Quantum Confidential: The Lost History of Quantum Mechanics

In 1879, the twenty-one-year-old Max Planck (1858–1947) received his PhD for his work on entropy, a concept that would later serve him very well. However, Planck’s career got off to a slow start. Originally focusing his attention on problems in physical chemistry, he found he was less competitive with the likes of Josiah Willard Gibbs (1839–1903) and Jacobus Henricus van ’t Hoff (1852–1911), who were better able to probe such problems with a deeper chemical insight. 

However, by 1894, now a full professor at the University of Berlin, Planck was ready to tackle one of the biggest problems facing physics at the time: the strange interaction between matter and light embodied in the blackbody spectrum. Enabled by the experimental successes in the early 1890s, Wilhelm Wien (1864–1928) proposed in 1893 a very general mathematical solution to the problem known as Wien’s Displacement Law, and in 1896, he provided an upgraded version known as Wien’s Radiation Law. However, Wien based the latter on a weak analogy (with another fundamental law), never providing an actual rigorous derivation. Nonetheless, it did show excellent agreement with experimental results, and so, it seemed the long-sought-after solution to the blackbody radiation problem had finally arrived. Planck decided to take a closer look.

Around 1898, after many failed approaches, Planck was able to obtain what would be his “fundamental equation,” which was nothing short of the equation for matter in equilibrium with light. Using this, Wien’s Displacement Law, and the entropy of a single resonator (which we would equate to an atom or molecule today), he was able to provide – unlike Wien – an actual derivation of Wien’s Displacement Law, which became known as the Wien-Planck Law. So, early in 1899, Planck was convinced he’d successfully found the universal form of the blackbody spectrum that physicists had desired for almost forty years. But then things began to unravel. 

No sooner had Planck recounted his efforts (in the fifth of a series of publications since he had started) than new experiments began to poke holes in the Wien-Planck Law – Planck had made a mistake somewhere. Quickly reassessing his original derivation, he found his mistake (it was in the entropy of the resonator calculation) and was able to arrive at the correct form of the blackbody spectrum (i.e., Planck’s Radiation Law), which was now in perfect agreement with experiment. At this point, Planck could’ve stopped and moved on to another problem, assured that he would win the Nobel Prize. However, Planck wasn’t satisfied yet.

Although he was able to obtain the correct answers (versus experiment) with his new law, the actual physical meaning was still not clear. Moreover, Planck had to use a bit of clever “inspired guessing” to obtain the final form, which also left him a bit unsettled. Planck had always approached solutions to physical problems with a “hunger of the soul” (as Einstein described it), and this time would be no different – other than costing him a bit of his soul in the process.

Ludwig Boltzmann (1844–1906) and Planck had butted heads on more than one occasion. At the heart of their conflicts was the fundamental nature of entropy. Whereas Boltzmann saw entropy as a property rooted in statistical interpretation, Planck saw it in more “absolute” terms devoid of such statistical nonsense. Planck wasn’t alone in his misconception. Indeed, Boltzmann himself made this same mistake earlier, as would Einstein later (who, like Boltzmann, realized and corrected it), and Rudolf Clausius (1822–1888) never conceded (he never saw a need for entropy to be described in terms other than its thermodynamic definition).

Whatever misgivings Planck had with Boltzmann’s work, he had already made concessions to him since he had started, and so he would, once again (he “had to obtain a positive result, under any circumstances and at whatever cost.”). He needed to, once again, revisit his expression for the entropy of his resonator, and so he appealed to Boltzmann’s (statistical version) of entropy:

The actual mathematics (of calculating the entropy) required that Planck chop up the energies available to his resonator into discrete “chunks.” Boltzmann had done this as well when considering the energy available to a gas atom (rather than a resonator). However, Boltzmann considered this nothing more than a mathematical trick, and he therefore eliminated these convenient yet – what he considered to be – nonphysical chunks of energy at the end of his calculation. But for Planck, this wasn’t an option; he needed the chunks in order to arrive at his radiation law. In this way, Boltzmann’s mathematical trick became Planck’s physical reality: energy comes in chunks, or energy quanta.  

Planck, now forty-two, was well aware that energy quanta would drastically change physics forever. As such, he (and pretty much everyone else) chose to ignore those annoying chunks and focus on the remarkable accuracy of Planck’s Radiation Law. In fact, it wasn’t until eight years later that Planck came around to energy quanta’s being physical reality, and even then, perhaps only half-heartedly. Nonetheless, in 1905, a twenty-six-year-old Einstein not only embraced the concept of energy coming in chunks, he extended the concept to light as well, to introduce light quanta, which we now call photons. 

Today, the “crazy” physical consequences that are implicated by the success of quantum mechanics are well known by many. However, the fascinating history and the scientific struggles surrounding its creation are much lesser known. Nonetheless, these amazing stories of science are truly inspiring, and they are as important as the actual discoveries themselves.

Dr. Scott Bembenek is a principal scientist in the Computer-Aided Drug Discovery group at Johnson & Johnson Pharmaceutical Research & Development in San Diego. He is also the author of The Cosmic Machine: The Science That Runs Our Universe and the Story Behind It. To learn more about Dr. Bembenek and his work, visit http://scottbembenek.com and connect with him on Twitter.