AI DEBATE : Yoshua Bengio | Gary Marcus
https://www.youtube.com/watch?v=EeqwFjqFvJA
and thinking, that around minute 59, Gary Marcus points out that symbolic AI can do what networks do, and networks can do what symbolic AI does. Then my question is how can we define the differences....
Here are a couple of thoughts:
1. Gradient descent defines neural networks (Bengio style)
2. Dealing with changes in the data distribution is a challenge (Bengio), but that is only a challenge when the learning methodology assumes distributions of the data (i.e. gradient descent)
3. 'attention' is the solution (Bengio style), however, 'attention' can exist also in the symbolic AI approach -- I have been playing with that in unsupervised hierarchical models since 1996...see my masters..it relates to ensemble learning as well...
4. Gradient descent wants numeric data (Bengio), Symbolic AI does not work with numeric data. See my upcoming presentation: 'Symbolic Learning and Quantitative Analysis of categories', (https://www.youtube.com/watch?v=HwmqbUVF26g) the key idea is that numeric data learning methodologies typically leverage assumptions in the distribution of the data (see 2), categorical data learning methodologies typically do not (this is not so accurate -- here I try again)
Numeric space implies a mapping. A mapping implies a distribution to set the coordinates. It is possible to arbitrarily set coordinates but that would be a poor mapping
Second point:
Classification is a judgement statement that creates a symbol.
Learning creates a mapping
Continuous learning adjusts the mapping.
The question Bengio is asking with attention is how to continuously learn, adjust the mapping while enabling higher levels to retain their understating of the lower level
