Disclaimer: I’m only two months into my Deep Learning course so I think I still have a rudimentary understanding of neural nets but I’m going to try my best to describe them as I understand them today. Any comments or feedback is appreciated.
The brain, as an actor
The main unit of a brain is the neuron, which fires electrical impulses to neighboring neurons in a dense, interconnected web — hence “neural network”. Any one neuron can have many dendrites to receive impulses from other neurons, as well as an axon that transmits impulses to other neurons. These impulses are “all or nothing”, either fired or not fired, though neurons can use the firing frequency to convey intensity. And while we don’t actually understand the complexity if this system, it’s through this large network (100–500 trillion neurons that fire to any of 7,000 other neurons on average) that complex thought and motor function arises. It’s akin to a hyperactive lightning storm, constantly flashing paths in the sky. This is how the brain triggers actions.
The brain, as a learner
The literature is a bit hazy on how the brain learns. A common theory suggests learning is a physical process called neuroplasticity: as we continue to trigger a specific chain of neurons, the body adapts and strengthens the connections between these neurons. For example, in patients who’ve had brain damage from strokes, it’s hypothesized that physical therapy is a process in which the brain discovers or strengthens different neural pathways to motor function.
Modeling how to think, not what to think
Artificial neural networks (ANN) are marketed as a simulated version of a brain. The leftmost nodes take in numerically coded data, which is transformed and passed onto deeper nodes with different weights and activation functions. In the classic representation, every node in a layer has a connection to a node in the next layer, though these connections could have minuscule impact — for instance two nodes connected with a weight of 0 is effectively not connected. Eventually the nodes will end up predicting a value on the rightmost side.
For supervised learning, vast amounts of input data (i.e. pixels values for an CAT scan or words in an email) is put through the neural network and estimated results are obtained. These estimated results are compared to actual output results (i.e. medical diagnoses for the CAT scans or whether the email was flagged as spam) for the input data and the difference between estimated and actual results is iteratively minimized over many runs. This is done by recalculating the weights for each neural network node such that the error between estimated and actual results is lowered, in process called “back propagation” (because the calculations are done from right to left). Lowering the error margin is how the ANN “learns” to build a network that better predicts an outcome. Since the data provided is just inputs and outputs, neither the machine nor the human programming it needs to be familiar with the problem’s intricacies, or how a certain set of inputs ends up becoming a certain set of outputs.
A flawed model?
There is debate about how reasonable it is to compare an artificial neural network and a brain. When you think of the brain as an actor, data that flows from neuron to neuron is very similar to data flowing from node to node. But thinking of the brain as a learner is different because we don’t exactly know how neurons learn. We use back propagation to strengthen certain ANN pathways for learning, but biological neuron impulses only flow one way (forward), so this is a big flaw in the ANN/Brain analogy. Current sentiment about ANNs is that biological neurons were used as a springboard or inspiration, not a hard and fast model. Regardless, ANN’s have found success wide use in machine learning applications so the brain debate is mostly a pedagogical one.
Brain food for thought
If you take a decidedly unromantic view on how brains function, you can essentially boil it down to a series of chemical/physical reactions. There’s no magic there, it’s all a system of chemical messages rooted in the laws of physics. That means, there’s no reason why we can’t reproduce the same system of chemical messages in a different medium, say, in a computer program. So while current artificial neurons are suspect in their relationship to biological neurons, there’s nothing that physically precludes us from developing a true artificial brain sometime down the line.
Originally posted on Medium.
Reposted on Notion