Actually you can learn a lot of stuff unsupervised, and then just use supervised learning to extract the information from within the net. So you can just throw huge amounts of letters at a deep net then later look at the activation at the top and say "Ok, so when neurons X, Y and Z are on it's when it's looking at a 1". I'm pretty sure you can just connect two at the top and have it say the number it sees.

> In order to make AI that really has a chance of reaching what humans would call intelligence, I firmly believe it has to make meaning out of some kind of raw sensory experience analogous to ours.

And why must it be able to do this from scratch? Why must it be hampered with the same limitations we have?