Listening to
Melanie Mitchell - The Collapse of Artificial Intelligence
https://www.youtube.com/watch?v=4QBvSVYotVc
and;
Yoshua Bengio - From Deep Learning of Disentangled Representations to Higher-level Cognition
https://www.youtube.com/watch?v=Yr1mOzC93xs
Melanie clearly describes the problem with narrow focus, overfitting to a specific domain, where apparent success in modeling the domain is not indicative of success in learning from the domain. I think Yoshua tangentially is speaking of the same issue and proposing a framework wherein a higher-level description forces a learning. I described this, and created a methodology around it, in my paper in Neural Computation.
However, I think both of them are ignoring the fundamental flaw in supervised learning. The supervision creates a coupling between the specific domain knowledge and the sensory input, it imposes a way of thinking about the problem.
I assume it is unavoidable, any system that is connected to a reality must impose some structure. Said better, any system that communicates with another system must impose some structure. And any closed system that does not communicate does not enable creativity, it may utilize a Wittgenstein private language but that is not helpful.
So the trick is creating a smooth function from heuristic to statistics.
The trick is creating metrics and recognizing the assumptions
The real trick is weak learning. Recognizing that any learning is limiting, so best not to learn too much.
What is perfection?
If we assume a causative world, one in which every action predicts a result. We can envision perfection as modeling the environment, understanding the results of our actions.
In Ashlagi terms, the direct relationship between the giver and the recipient provides for a simple relationship.
But in a complex system, where the results are not predictable perfection is a form of evil. Different than deconstruction.
Deconstruction devolves the organic whole into its parts, denigrates the complex action into a complicated and then finally simple one.
Modeling in the simple world is understanding the parts
Modeling in the complex world is describing the whole
A perfect model in the simple (Ashlagi) world enumerates the parts of the system/machine and their relationships, the newtonian forces between the elements. Who gives who receives
A perfect model, i.e. the best model possible since perfection is by definition impossible, in the complex world, provides the minimum description length of the whole.
In the complex world transitions necessitate a loss of information, hence there is no perfect transmission. A perfect communication without loss would not enable learning (see 'why judgement is necessary')
