AI (artificial intelligence) is a huge topic in recent years and a large part of the AI subject centres around Neural Networks, but how do they actually function? This article aims to be a short primer on the subject.
Just like the human brain, neural networks in computing function very differently to traditional computer programs.
In a traditional computing we have two parts, the architecture of the computer and the software that runs upon the architecture. However with artificial neural networks these two parts are the same thing. In order to change the output the physical structure of the network must change.
A traditional computer program is developed first and then used, generally in those two discreet steps. However a neural network can be molded over time, or ‘trained’ as its normally called.
I often think of a neural network like a piece of clay being forced into a mold. It’s slowly pushed into its environment until it starts to take the same shape. When you extract the clay its a proxy for the environment, you can make assumptions or use that model to simulate the original environment.
The physical structure of the network is also the actual logic of the program. Like the brain, it consists of neurons, synapses, axons etc. Neurons are like nodes and fire pulses to one another via the synapses and axon (connections). However the intelligence isn’t solely in the neurons, in fact its mostly the structure of the connections between neurons that forms the logic of the program.
There are approximately 86 billion neurons in the human brain, which is, of course, a lot. However when you consider that there are 100 trillion connections between neurons (each forming only part a particular pathway) we can start to see why the brain’s specific shape can bring about individual experience, memory and knowledge. Artifical neural networks leverage the same concept.
When we train a neural network we are in fact changing the connectedness of a particular pathway.
As a child you have fewer connections between neurons. The process of accidentally burning yourself is a learning experience whereby the connections change, so that the pathways that caused the action are less likely to be taken again in the future.
We may think of a synapse (connection between neurons) like a road. By reducing the road from a three lane highway to a single lane the traffic is far more likely to take a different route.
In neural networks this process of learning is often referred to as backwards propagation.
We observe the output of a brand new network when we expose it to its environment. We scold a child when they do something naughty and similarly we can ‘scold’ a neural network immediately after it produces a result that we don’t like. Knowing the result is incorrect and the pathway that produced the incorrect result we can reduce all these synaptic connections by just a little (perhaps a few percent) and by doing that we are pushing the clay just a little more into the mold.
This phase of testing the network uses training data, where we know the results in advance. For example we might have ten thousand pictures of cats and dogs. If the network says dog when it’s a cat, we know how to refine it. Once we’re done we can let the network lose in the wild.