Dileep George
Electrical Engineering, Stanford University
and Redwood Neuroscience Institute
Real world objects have persistent structure. However, as we move about in the world the
spatio-temporal patterns coming from our sensory organs vary continuously. How the brain creates
invariant representations from the always-changing input patterns is a major unanswered question.
We propose that the neocortex solves the invariance problem by using a hierarchical structure. Each
region in the hierarchy learns and recalls sequences of inputs. Temporal sequences at each level of
the hierarchy become the spatial inputs to the next higher regions. Thus the entire memory system
stores sequences in sequences. The hierarchical model is highly efficient in that object
representations at any level in the hierarchy can be shared among multiple higher order objects,
therefore, transformations learned for one set of objects will automatically apply to others.
Assuming a hierarchy of sequences, and assuming that each region in the hierarchy behaves
equivalently, we derive the optimal Bayes inference rules for any level in the cortical hierarchy
and we show how feedfoward and feedback can be understood within this probabilistic framework.
We discuss how the hierarchical nested structure of sequences can be learned. We
show that static group formation and probability density formation are special cases of remembering
sequences, thus although normal vision is a temporal process we are able to recognize flashed
static images as well. We use the most basic form of one of these special cases to train an object
recognition system that exhibits robust invariant recognition
