As so neatly lampooned by xkcd, physicists have a reputation for being just a little reductionist at times. It’s a well-deserved critique in many cases — some sub-fields of our subject seem too often to forget Philip Anderson‘s timely admonishment back in the seventies that more is different:
The main fallacy in this kind of thinking is that the reductionist hypothesis does not by any means imply a “constructionist” one: the ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe.
Philip W Anderson, Science 177 393 (1972)
Nonetheless, there is an almost pathological drive in many physicists to deconstruct; to reduce a system to its most basic elements. It’s an approach that has, after all, served our discipline very well over the centuries and is the bedrock of so much of what we do and teach. As long as we bear in mind that more is indeed different — and don’t naively assume that each aspect of life, the universe, and everything can be modelled by a suitably parameterised Ising model — then there can be considerable value in getting down to brass tacks.
So how low can we go? Can physicists literally atomise the human brain? Is it possible to encode the functionality of a neuron or a synapse in atoms? This is the theme of the Computerphile video embedded below (filmed, edited, and uploaded by the prolific Sean Riley, who, in addition to his Computerphile roles, is the host of the Living With AI podcast, co-runs Pots and Trowels, and has most recently rockumented the trials and tribulations of a britpop band dealing with the pandemic. Note that we academics don’t contribute to the editing of the videos (or thumbnails…) in any way.)
Sean and I discuss an inspiring paper, recently published in Nature Nanotechnology, that describes the work of Brian Kiraly, Elze Knol and co-workers in Alex Khajetoorians‘ group at Radboud University in Nijmegen. Khajetoorians’ team have, in a nutshell, scaled the functionality of neurons and synapses all the way down to the single atom limit. To do this, they needed to find a system which, at the atomic limit, is capable of mimicking neuronal behaviour and, in particular, could capture the most fundamental aspects of the neural network firing inside your skull as you read this.
Making connections
How do we learn? In essence, it’s all down to having the right connections. Neurons communicate with their neighbours via synapses, gaps which are of order tens of nanometres in width and across which a neurotransmitter is passed as the communication signal. A neuron releases neurotransmitter when its electrical potential reaches a threshold value (known as the action potential.) But synapses aren’t just a binary switch that simply opens and closes. If they were, we’d have no memory and we couldn’t learn from previous inputs/experiences.
We learn because of the essential malleability of the neural network that is the bedrock of our brain (and, by extension, of ourselves.) The strength of synaptic connections is modified by repeated exposure to stimuli via two primary mechanisms: short term plasticity (STP) and long term potentiation (LTP). It is the latter that is responsible for long-term memory — synaptic connections are strengthened and establish specific neural pathways that encode the knowledge or skill we’ve learnt.
A decade ago, a research team led by Jim Gimzewski and Masakazu Aono — who, like Khajetoorians, are world-leading researchers in the area of scanning probe microscopy — used a scanning tunnelling microscope and a layer of silver sulfide, Ag2S, to generate, and control, artificial synapses, demonstrating both STP and LTP. (In the Computerphile video we briefly describe how a scanning tunnelling microscope works. This article provides a more in-depth overview.) Figure 1 from their paper is shown below and neatly encapsulates the core concept. Depending on the rate of the input voltage pulses, either short term- or long term memory (i.e. STP vs LTP) is observed. When enough pulses are applied quickly enough, the artificial, solid state synapse — essentially a nanoscopic cluster of silver atoms — eventually fused so that the electrical connection was permanent; the system had a memory of its previous “experience”.
[Taken from Ohno et al. Nature Materials 10 591 (2011)]
Remarkably, Khajetoorians’ team went even further. They scaled down the functionality of neurons and synapses to, essentially, the single atom limit. To do so, they encoded what’s known as a Boltzmann machine in just a handful of atoms*.
Boltzmann’s Legacy
Before I briefly describe what’s meant by a Boltzmann machine, I’ll just note that there is something very satisfying about Boltzmann’s ideas being realised at the atomic limit. Ludwig Boltzmann — of Maxwell-Boltzmann distribution and Boltzmann’s constant fame, and whose pioneering and brilliant work laid the foundations of a cornerstone of all of physics (i.e. statistical mechanics) — was a very troubled man, eventually taking his own life during a holiday with his wife and daughter. His deteriorating mental health has been suggested to arise from what would now be diagnosed as bipolar disorder.
One aspect of Boltzmann’s career that ultimately caused him a great deal of stress was that his ideas about the statistical nature of matter were based firmly on the atomic hypothesis; Boltzmann believed firmly that, to quote Feynmann decades later, “… all things are made of atoms –little particles that move around in perpetual motion…”. And yet the scientific establishment of the time largely rejected the “reality” of atoms, seeing them as merely a theoretical construct. Boltzmann was ostracised by the majority of his peers for the intellectual crime of being, as the title of Carlo Cercignani’s biography puts it, “the man who trusted atoms.”
Can you imagine Boltzmann’s delight, however, if he could have seen STM images of individual atoms — and not just see them, but, and as detailed in a recent comprehensive and engaging review written by Khajetoorians and co-authors, push, pull, prod, and poke them; to build nanostructures atom by atom? I find it wonderfully apt that Boltzmann’s ideas are now being realised in the very atoms he championed throughout his life.
A Boltzmann machine falls within the class of machine learning algorithms known as energy-based models: they map the solution space for a learning problem to an energy landscape. (I thoroughly recommend Ackley, Hinton, and Sejnowski’s original 1985 paper on Boltzmann-machine-based learning algorithms — it’s a model of clarity.) In other words, the system is modelled such that interactions are defined according to energy costs/gains, and the optimised solution is determined by finding the lowest energy state. This strategy thoroughly appeals to physicists, for whom energy is pretty much everything. (Just about any question in physics — apart from one or two thorny general relativity issues –can be answered by “The system wants to reach its lowest energy state.” The devil, of course, is always in the detail…)
Despite my somewhat disparaging comment about Ising models above, they are of immense value in so many scientific areas, including machine learning and artificial intelligence. (We just have to be careful to recognise their limitations.) I’m not going to rehearse a description of the Ising model here — I wrote about it at length in the post linked above. As Kiraly, Knol and colleagues explain in their paper, a Boltzmann machine is equivalent to an Ising model where, instead of representing electron spin, each element of the Ising “grid” represents a neuron that can fluctuate between different values; the neurons in turn interact with their neighbours with a weighting that can be history dependent. We can write down a simple equation (known as a Hamiltonian) that accounts for the energy balance in the Boltzmann machine:
where 
Building the bits of an atomic brain
Khajetoorians’ team encoded the neurons of the Boltzmann machine as single cobalt atoms (supported on a black phosphorous substrate, whose structure and importance I’ll not cover here. See Alex Khajetoorians’ seminar video embedded at the foot of this post.) The electronic configuration of the cobalt atom can be switched between two different states — the apparent “shape” of the atom is significantly different in each of the states (compare the upper and lower STM images shown in (B) below. Note that an STM image is, to a very good first approximation, a map of electron density, and so if the electron configuration changes, so too does the image.) If the voltage applied between the STM tip and the atom exceeds a threshold value, the atom switches back and forth between the two configurations randomly. This is detected by measuring the tiny electrical current (due to quantum mechanical tunnelling) that flows between the STM tip and the underlying atom; as can be seen in figure (B) below (taken from Kiraly, Knol et al.‘s paper), the current switches between a “low” and a “high” state.
That’s encouraging, but for a Boltzmann machine the neurons need to interact. And indeed they do. As shown in (C) below, bringing two cobalt atoms into close proximity leads to the current now switching between not just two, but four values: the atoms are interacting and, in a Boltzmann machine picture, changing the energy landscape as compared to that for an isolated atom/neuron.
arXiv version: https://arxiv.org/ftp/arxiv/papers/2005/2005.01547.pdf
The final piece of the Boltzmann machine jigsaw is the ability to control the weights of the interactions between neurons. Kiraly, Knol et al. achieve this by “gating” the interaction of atomic neurons with another atom. (The structure of the substrate plays an all-important role here but understanding that isn’t necessary for the core concept. Again, see the seminar video embedded below for the complete picture.) Here’s the evidence that an appropriately positioned cobalt atom can indeed control the interactions between its neighbours, i.e. that the weight, 
arXiv version: https://arxiv.org/ftp/arxiv/papers/2005/2005.01547.pdf
By changing the configuration of the uppermost cobalt atom, the coupling between its neighbours is modified, as demonstrated by the differences in the variation (or “spiking”) of the tunnel current. In effect, single cobalt atoms act as both neurons and synapses. The researchers then went on to show that the atomic Boltmann machine they had fabricated could learn, i.e. the weightings between the cobalt neurons retained a memory of their past experience (in this case, the voltage that had been applied.) But to find out more about that, I’m going to refer you to Alex Khajetoorians’ clear and compelling seminar on the subject. Much better that your particular neural network learns straight from the horse’s mouth…
* For the more literal-minded who may be reading, yes, I know that a literal handful of atoms would be rather a lot. I’m using the term in its colloquial sense. Dealing with the literal-mindedness that is seemingly the lifeblood of the web can often be exhausting (and a true test of any artificial neural net will be its ability to contextualise and read between the lines.) I’ll finish as I started and close with xkcd nailing it, as ever:
