Steven Weinberg has an interesting article to appear in the forthcoming edition of the New York Review of Books on the foundations of, indeed the trouble with, quantum mechanics. Weinberg declares himself to be “not so sure about the future of quantum mechanics,” that is he is not so sure that it constitutes a complete description of physical reality and that it can form the basis of a further deepening of our understanding of nature.
In the late 19th century up to the very early 20th century statements about the completeness of physics were often explicitly made with great confidence, but such sentiments were to be shattered by conceptual revolutions occasioned by relativity and quantum mechanics. This was not unlike similar desires to demonstrate the completeness of mathematics which also were declared and shattered at about the same time.
Today that sentiment is quite a prevalent one, it seems to me, except this time it is largely tacit. To be sure one often hears about looming and promising theories of everything, whether they be supergravity, superstrings, M-theory, loop quantum gravity, and the like that will end theoretical physics but these are references to what is to come, whether soon or not.
What I am referring to is the implicit assumption that quantum mechanics represents a foundational picture of physical reality, and once it supersedes general relativity through a quantum theory of gravity we would then know everything there is to know of a fundamental character about physical reality.
Yet the failure to develop just such a theory of quantum gravity might suggest otherwise. Every road to quantum gravity, and they’ve been more than three, has reached a dead end. This persistent failure demonstrates that, regarding the quantum mechanical picture of nature, gravitation is a persistent anomaly and if we put our Kuhnian hats on what is there for us to do but declare;
Quantum mechanics is not complete and the anomalous nature of gravitation demonstrates this to be so.
Weinberg draws a useful historical link to the reception of Newton and his universal law of gravitation. Just as Newton dispensed with an intuitive picture of nature that had hitherto held sway, the mechanical philosophy, so did quantum mechanics and it struck our sensibilities for largely the same reason, namely spooky action at a distance and the response has been pretty much the same that of; shut up and calculate.
But doubts, including philosophical ones, nonetheless persisted after Newton. The same, seemingly, applies to quantum mechanics and Weinberg states here
The introduction of probability into the principles of physics was disturbing to past physicists, but the trouble with quantum mechanics is not that it involves probabilities. We can live with that. The trouble is that in quantum mechanics the way that wave functions change with time is governed by an equation, the Schrödinger equation, that does not involve probabilities. It is just as deterministic as Newton’s equations of motion and gravitation. That is, given the wave function at any moment, the Schrödinger equation will tell you precisely what the wave function will be at any future time. There is not even the possibility of chaos, the extreme sensitivity to initial conditions that is possible in Newtonian mechanics. So if we regard the whole process of measurement as being governed by the equations of quantum mechanics, and these equations are perfectly deterministic, how do probabilities get into quantum mechanics?
The traditional answer, the Copenhagen Interpretation, Weinberg finds wanting due to a demarcation problem
One response to this puzzle was given in the 1920s by Niels Bohr, in what came to be called the Copenhagen interpretation of quantum mechanics. According to Bohr, in a measurement the state of a system such as a spin collapses to one result or another in a way that cannot itself be described by quantum mechanics, and is truly unpredictable. This answer is now widely felt to be unacceptable. There seems no way to locate the boundary between the realms in which, according to Bohr, quantum mechanics does or does not apply. As it happens, I was a graduate student at Bohr’s institute in Copenhagen, but he was very great and I was very young, and I never had a chance to ask him about this.
There is also a problem of completeness as Weinberg relates above wave function collapse cannot be explained endogenously so the Copenhagen Interpretation comes at the expense of incompleteness. Weinberg goes on to articulate weaknesses in other approaches to the measurement problem, which he puts under the broad headings of instrumentalist and realist approaches taking the many world’s interpretation as a reductio against realism.
It may well be the case that further fundamental advances in theoretical physics are to be found with work on the foundations of quantum mechanics, not necessarily quantum gravity or theories of everything. As Weinberg points out in the article, such work is amenable to empirical inquiry through experiment unlike, it would appear, in the case of the latter theories.
An intriguing passage in the Weinberg article is a tacit reference to David Hume. It is easy to miss. But it is well and truly there
Thus the instrumentalist approach turns its back on a vision that became possible after Darwin, of a world governed by impersonal physical laws that control human behavior along with everything else. It is not that we object to thinking about humans. Rather, we want to understand the relation of humans to nature, not just assuming the character of this relation by incorporating it in what we suppose are nature’s fundamental laws, but rather by deduction from laws that make no explicit reference to humans. We may in the end have to give up this goal, but I think not yet.
Recall the master from his A Treatise of Human Nature
In pretending therefore to explain the principles of human nature, we in effect propose a compleat system of the sciences, built on a foundation almost entirely new, and the only one upon which they can stand with any security
Considerations on and of quantum mechanics may well be leading us toward an epistemology naturalised worthy of the name, and only an epistemology naturalised can pretend to offer some semblance of a foundation to our understanding of nature.
To understand nature is to understand how it is comprehensible.