Neural Network Learning Principles

  
  Because of a recent string of educational fads which sought authority by claiming (often without much justification) to be derived from neuroscience, many educators now react badly to what they call 'neuro-babble', and have convinced themselves that neuroscience has nothing to tell us, yet, about how we learn, and therefore has nothing to contribute to the design of more effective approaches to education.

  
 

It is true that neuroscientists don't say much about education, but it is not because they have nothing to contribute, it's because they are focused on a different set of problems and are working at a different level of detail. They are necessarily bogged down in the highly intricate scientific (experimental, statistical and ethical) problems that arise when they try to unravel such complexities as; why the speed at which we can read and distinguish between past and future tense verbs is effected by whether the words are presented in the left or right visual field?

  
  Their tools and methods are designed to work at that small scale, the microcosm, and are not suitable for exploring the general principles, the macrocosm, of human learning. They know a lot about how our neural networks learn to perceive the world, but they don't talk about it.

  
  So it falls to others (who like myself, are less constrained by professional/academic group-think) to bridge the gaps between the different academic disciplines, to extract principles from one domain and try them out in others.

 
  Designers of artificial intelligence software and smart robot-builders spend a lot of time copying ideas and observations from neuroscience and working out how they can be applied to real world problem-solving. From their perspective, neuroscience has already discovered many new and fundamental principles of neural network learning which have profound implications for the design of human education systems. Here are few particularly important ones.

  
 

In evolutionary terms, high quality sensory information is a valuable but expensive commodity. A balance had to be struck. As a consequence, our sensory capability is surprisingly limited, much more limited than we normally realize. We feel as if we can see everything there is to see, and hear everything there is to hear, but actually we can only detect a tiny proportion of the vast amount of information that is bouncing around the universe and we are deaf and blind to the rest of it.

  
 

But evolution is an awesome problem-solver and it came up with a very smart solution.

The limitations of our sensory capacity are compensated for by the amazing experience-trapping ability of our neural networks.

The brain uses this trapped experience to pre-consciously enhance our limited real-time sensory information. Your understanding of how the world appeared to work in the past, (as encoded in the structure of your neural connections) is used to help make sense of the limited, vague, ambiguous and noisy information you are receiving from your senses right now.

 
 
It is important to understand how neural networks trap and use past experience.
  
  Our accumulated maps and models about the world are not stored as facts or data in a lookup table, they are 'encoded' in the current wiring of the network which continuously evolves in response to the sensory experiences that flow through it. New real-time sensory information passes through (selectively stimulates and is interpreted by) networks which have been shaped by past experience, networks which have been 'taught' by experience to be sensitive to particular patterns of associations and distinctions, and insensitive to others. The old term 'encoding' is too static and fails to capture the continuous perceptual evolution made possible by this neural network experience trapping.

  
  This is how we interpret our environment, how we recognize familiar objects, people, places, movements, smells, situations, threats and opportunities, from within the stream of information that flows through our senses. This is how past experience is pre-consciously employed to help make sense of limited, noisy, current information.

 
 

Our perceptions, judgments and reactions, feel much more solid than they really are. They are actually composed of a small measure of sensory information and a large measure of assumptions about how the world 'is' now, which are based on our experience of how the world appeared to us in the past. The quality of our current perception, our understanding and our behavioural choices is highly dependent on the quality of our accumulated maps and models about reality - and the quality of our maps and models changes significantly with time and experience.

 
 
Neural network learning curves.
 
  We are so confident in the truth of our current model of the world that we are not normally aware of the profile of our own learning curve. Adults forget - to use one of Piaget's examples - that there was a time in our childhood when we didn't know that 'number is conserved'. Then, one day we discovered, to our astonishment (amusement, delight), that counting a row of ten pebbles from left to right produces the same result as counting them from right to left. Adults forget that we had to learn how to see and experience the world, and we forget that the experience of learning, finding new ways to make sense of our world, is very good fun.

 
  We are also blissfully unaware of all the things that we are not yet aware of. Gradually, step by step, we become aware of, learn to distinguish between, more and more different aspects of the world. Once we start paying attention to something new, we learn fast, but as our maps and models improve, our attention reduces. We have the feeling that we know all about that now, and we only pay attention if something unexpected/unexplained happens. And then, too often, learning grinds to a complete halt and we enter the rigid belief phase, where our curious learning attention is no longer activated by that topic.

  
  Not even the arrival of some new experience that (to an independent observer) highlights glaring inconsistencies in our maps and models, is enough to shake our belief and trigger a review and update of our understanding. This is not because we are stupid, illogical or irrational, it is because our neural networks have settled into such a stable state that they can no longer detect, respond to, or learn from, these new inconsistencies.

 
  And so it goes - we stumble across our own individual cognitive landscape, from the blissful ignorance of inexperience, to the self-assured ignorance of too much experience, via a trail of temporary errors. But this neural learning mechanism works very well, most of the time. Well enough to have enabled an unbroken chain of our ancestors to cope with a huge variety of very challenging situations. Their struggles ensured that we, their offspring, inherited a neural platform that was very well adapted to working out how to cope with the types of problems that arise for a human being on the surface of planet earth. We are naturally very good at learning from experience and applying that experience to the business of staying alive. What we need is a rich supply of the right types of experiences to learn from, and a healthy cultural framework to tell us how to make sense of the experiences.
  
 


Structurally encoded experience-trapping is the crucial concept here.

 
  In order to understand how this experience-trapping is possible, you first need to understand both the concept and the mechanisms of neural plasticity. This will be a new and unfamiliar concept for most people because most of the machines and systems we encounter in everyday life do not work in this way.

 
  We naturally assume that the electrical wiring in our radios, DVD players, toasters, and washing machines, will remain the same from one day to the next, and that these pieces of equipment will behave in exactly the same way tomorrow as they did today. We do not expect that the wiring will rearrange itself every time it is used, and that its behaviour will evolve as a result. But that is what happens in our neural networks. The wiring in a neural network subtly rewires itself every time it is used, adapting and fine tuning its performance, growing new connections to speed up the reaction times of busy circuits, connecting subsystems that repeatedly get used at the same time, re-prioritising menus, adjusting switches and sensors to make them more or less sensitive, cutting away and recycling redundant wiring that hasn't been used for a long time.

 
 

There are some everyday examples of plasticity (things changing their structure as a result of experience) that may help in developing an understanding of these experience-trapping processes at work in our neural networks.

 
  For an example of a sequence of events causing structural changes in a system, which then influence the future behaviour of the system, think of rain falling on a hillside. The water flows downhill and collects at the lowest point to form a little lake. The level in the lake rises until the water overflows its retaining barrier and forms a powerful stream which washes away the land surface to cut a passage across the landscape to the next lowest level lake. Every time it rains the whole system evolves a little bit, and the flowing water cuts a slightly different and more direct path to the sea. Once a river system is well established, it is very difficult for a new river system to get started in that area, because the rain naturally flows into the existing system. Similarly, patterns of connections form in our neural networks, as a result of the sequence of experiences that have flowed through that area of the brain. The current state of these patterns determines how we will respond (interpret, perceive, react) to the next experience, and that new experience will play its part in further fine tuning the overall neural pattern.

 
  Like a well established river system, a well established neural pattern comes to dominate its landscape. Whilst it may still be able to adjust itself in response to trivial and fluffy new experiences at the edge of its territory, its core structures may become so well established that they become very insensitive to important new evidence (the belief stage). An important new experience might be able to start a new furrow in an unploughed region of the network but it probably won't have any effect on the core structures in an established pattern.

  
 

For an example of a more dynamic network plasticity, think about the evolution of a group of friends on a social networking website. Suppose one member of an existing group develops a new interest and tells his/her group friends about it. Some of them are interested in this new topic, others are not. The ones who are interested get excited about the new subject, and start telling other people, some of whom are not part of the original group. Some of these outsiders then join the original group, bringing new ideas and associations with them. This changes the nature and balance of the original group. As a result, the original group starts to split into two different groups with slightly different characteristics; one centered around the members who were interested in the new idea, and the other centered around the ones who were not. Because the two groups now have slightly different characteristics, they may react quite differently to the next new idea that comes along. This is quite a good model for neural network plasticity: a dynamic structure whose reactions to external events change over time, because its internal structure is constantly being reshaped by the flow of resonant experiences.

 
 
So what can these neural network principles tell us about human learning and teaching.
  
 
  • The performance of a neural net is not determined by size or power. What matters is that it is well-tuned to, sensitive to, the sensory stimulation it is receiving. The quality, the richness in the available information, is crucial. Networks detect patterns, connections and associations in a flow of experiences and information. If the information is bland, monotonous, unstructured, oversimplified, poorly connected or badly sequenced, then there is very little for the network to detect. If it is not resonating, it is not experiencing and it cannot learn - if it is, a lot can happen.
 
 
  • The brain is a fantastic natural learning machine. It even gets neurological pleasure from exploring, paying attention and learning. It will perform amazingly well if it is presented with resonant, rich, well sequenced, well connected, relevant experiences.
 
 
  • Neural networks make meaning; they categorize, generalize and abstract from experience. They turn a continuous flow of ever changing sensory information into relatively stable perceived objects and relationships. The meanings we attach to things are our own creation. Our culture has a very important role in telling us how to make sense of our experiences of 'reality', what things mean, relative values, how to think about different types of thing, what to call things, how they work.
  
 

  • Timing the sequence of exposure to these cultural building blocks is very important. Algebra makes no sense, and is impossible to learn, if you haven't already mastered arithmetic. When a hole appears in the concept wall, fix it.
 
 

  • Our sensory apparatus is very limited. We forget that we can only see what we can see; which is a tiny and peculiarly human portion of reality. We are overconfident in the general truth of our limited perspectives.
 
 
  • Young humans are fantastic copy cats, they will mimic all sorts of things, including; what to pay attention to, how to react to new information, what things mean, relative values, what's important and what is not, what to say, when to say it, etc. So be careful what they are exposed to. See to it that they get plenty of exposure to the things you want them to copy. Be, or provide, examples of successful learning and problem solving attitudes, successful social behaviours; build things, make things, cook things, design things, organize things, make things happen.
 
 

  • Humans enjoy learning. When something resonates and grabs their attention, take full advantage of it. Surf that learning curve as far as you can, and consolidate those experiences, giving them cultural meanings, names, etc.
 
 

  • Pay attention to the properties of things and the properties of the relationships between things. It is not enough to simply name things/object/ideas. Build these elements into systems / models of cause and effect in time and space. Pay attention to the emergent properties of these systems. The human brain is quite capable of, and thoroughly enjoys, doing this from a very early age.
 
 

So

1) Work with the learning curve. Surf the learning curve, moving from unaware, to attention, to detailed fascination, to cultural consolidation / explanation. Keep it moving and questioning, to postpone the onset of rigid beliefs and out-of-date models.

  
 

2) Natural learning is pleasurable. Evolution designed it that way to motivate us to get out there and find out about our environment. Use it. The neurochemicals that are released when clouds of new associations are being made, give us the experience of fun, pleasure, euphoria, humour. It raises self-esteem and self confidence - hence the 'eureka' moment.

  
 

3) The brain is primarily an experience machine. It uses personal experience, and cultural framing, to categorize, abstract and generalize. It is not a rules processor - but it can use rules to consolidate previous experiences, and to agree socially what things are to be called, what they mean, measurement systems, relative values, principles, ethics, etc.

  
 

4) Remember that we (experienced) adults can only guess what things look like to a child's (inexperienced) brain, The rules and distinctions that an adult brain uses to communicate its mature understanding of the world (rules of grammar, or (unreliable inconsistent) spelling rules, for example) may not have any meaning to, or be of any use to, a child. Trust that their neural network platform will do the job it evolved for - if - you present it with the kind of rich, well structured, well sequenced, and relevant experiences that evolution has designed them to work with.

What you can and must do, is provide the cultural framework that is needed to consolidate those experiences (the names, the concepts, the measure, the values, the basic facts). Inexperienced children cannot co-create that element for themselves. It is the adult's job to pass on this cultural store of human achievement. Be clear and confident what the goal is, in terms of the sequence of cultural building blocks you are aiming to pass on to the next generation. Do what you can to help their neural networks do what they do so incredibly well. Our inherited neural networks are much better at learning than we are at teaching. The teacher's job is to provide rich, well-designed experiences, and then provide culturally framed consolidation; explanations, links and associations, language, etc.

  
 
Be aware of the weaknesses in the neural network mechanism.
  
  Very Important - Our neural networks are very good at detecting local 'cause and effect' relationships. They can detect and store associations between things which happen close together in both space and time. We are very bad at detecting remote associations where the cause and effect are separated by either time or space. To ease our anxiety and reduce uncertainty, we invent reasons to explain remote causation. We have a long history of invented reasons to explain why droughts, floods, earthquakes, plagues, etc., occur.

  
  Because our neural network understanding is based on experience, we are very bad at assessing the probability of things we have not yet experienced. We do not expect the unexpected. We are not good at detecting random chaos, so we tidy-up and simplify the past, which causes us to underestimate the amount of sudden change there is in our environment. We assume the world is more stable than it is. We assume it has always been the way it is now. We get anxious if it looks like changing. We don't understand that it is and has always been changing. (This perception is more prominent in western culture which developed the idea that God would make the world ok for us if we behaved properly - eastern religions have always known the world is constantly changing.)

 		 
  We think / feel that our perception of reality is true. We don't realize that we invented our current interpretation of reality and that our perception is very limited. We have to learn by experience that the world looks very different from other people's points of view, and from within other cultural or ideological frameworks.

  
  We often distort our own experience-based perception and models of reality, in order to fit in with the orthodox views of significant social groups. Particularly if there are rewards and punishments involved. So watch out for the influence of fashion and group-think. Many of our attention mechanisms operate pre-consciously in the vast submerged iceberg of the mind. Advertisers, and persuaders of all types, will try to hijack these attention mechanisms to get you to buy their products or lend support to their causes. Be aware.

 
  And finally - our neural networks evolved to grow up in a more natural setting than the modern urban landscape most of us now inhabit. It remains to be seen how well they will function in this new setting.