Why Won’t You Look at My New Theory?

In any forum where science is discussed, there will always be people who have a great new personal theory and can’t understand why no one else is interested in it. Here at PF we have rules about this, but I want to look at the more general question of why there is so little interest in such personal theories, independently of whatever rules a particular forum might have. Is it just because people are closed-minded, and unwilling to consider new ideas? Or is there some more cogent reason?

Of course, personal theories cover a very wide range; but here I want to focus on a particular kind of personal theory, one which arises from the following scenario: A new observation or experimental result is reported that appears to be inconsistent with what we think we already know. Rather than pick on recent examples (of which there are plenty), I’ll give two examples from the history of Solar System astronomy in the 19th century, since the outcomes of these cases are both well established by now so they can serve as good test cases without raising anyone’s hackles. Here they are:

(1) In the 19th century the motion of the Moon appeared to be inconsistent with the predictions of Newtonian gravity–i.e., the Moon was observed in the sky at locations that were different from those predicted by Newtonian calculations from previous observations. The differences were small, but the calculations and observations were believed to be accurate enough to make them significant.

(2) In the 19th century the motion of Mercury also appeared to be inconsistent with the predictions of Newtonian gravity. Here, again, the differences were small, but it was believed that the calculations and observations were accurate enough that the discrepancy was significant.

The question then arises, what is the reason for the apparent inconsistency? There are two possibilities:

(A) The inconsistency is only apparent; it is because we haven’t worked out carefully enough the implications of what we already know. This was the case for the apparent anomaly in the motion of the Moon: it turned out that there were small perturbations due to the other planets that hadn’t been correctly calculated, and when the calculations were corrected, the discrepancy between the theory and observation went away. This means, of course, that people’s belief before this discovery, that the calculations of the Newtonian prediction were correct, was in error.

(B) The inconsistency is real; it is because there is some fundamentally new effect going on that our current theories don’t comprehend. This was the case for the anomaly in the motion of Mercury. It turned out that the current theory of gravity (Newton’s theory) was not correct. When Einstein replaced that theory with the general theory of relativity, one of the first predictions to be re-calculated based on the new theory was the motion of Mercury, and the correction to the Newtonian prediction due to general relativity brought the prediction into line with observation.

It is worth noting, by the way, that before GR was developed, scientists considered a more mundane explanation of the discrepancies in Mercury’s orbit: that there might be a small planet inside the orbit of Mercury that was perturbing its motion just enough to account for the discrepancy. But such a planet was never observed despite increasingly sensitive attempts to do so, and this possibility had been rejected by the time Einstein began working on GR.

I don’t think any reasonable person would disagree that, in principle, (A) and (B) above are both valid possibilities in any situation of the general type we are discussing. However, I think there is a vast disagreement between scientists and non-scientists about the relative frequency of occurrence of (A) and (B). Many nonscientists seem to believe that situations of type (B), where a fundamentally new effect is there and the theory has to be modified to account for it, are common in science; whereas all good scientists know that in fact, almost all situations turn out to be of type (A), where the theory is fundamentally correct but its implications haven’t been calculated accurately enough. This is not because scientists are lazy or incompetent: it’s because calculating the predictions of a known theory is not a cookie-cutter mechanical process but a separate intellectual effort in its own right, and it is subject to the same kinds of errors as any other theoretical efforts.

I don’t know exactly why so many nonscientists seem to believe that type (B) situations are vastly more common than they are, but I can think of several possible reasons:

(1) Type (B) situations are far more exciting, so historians of science tend to focus on them, while the vastly more common type (A) situations are left out of popular accounts. So the nonscientist’s erroneous belief about the frequency of type (B) situations is due to a straightforward sampling bias.

(2) Type (B) situations, because they intrinsically involve the overthrow of some part of an accepted theory and its replacement with a new theory, always involve a dynamic of resistance by the scientific community to the new theory. Scientists understand that this resistance, even to theories that ultimately win out, is rational, and a necessary part of science; but nonscientists just focus on the underdog fighting against the establishment because it feeds their pet beliefs about such situations. So nonscientists’ erroneous belief that type (B) situations are common is just a special case of the general belief (which is also erroneous) that underdogs fighting establishments are usually right.

(3) Type (B) situations appear to nonscientists to hold out the hope that, in principle, anybody can overthrow an accepted scientific theory. Newton was a lowly college student when he came up with his laws of motion and his theory of gravity. Einstein was a patent office clerk who had failed to obtain an academic job when he published his famous papers on special relativity and quantum theory. Nonscientists look at these examples and draw the (erroneous) conclusion that you don’t need to know anything about the established theories to overthrow them; you don’t need to go through all the bothersome stuff that members of the scientific establishment do, like taking classes, getting degrees, doing research, publishing papers, going through peer review, etc. Just come up with a great new idea and you’re set.

Scientists, though, understand that Newton, Einstein, and the other scientists who found themselves in real type (B) situations did do all that stuff–they did learn the established theories inside and out before they tried to overthrow them. They did their “homework” in an unconventional way, but they still did it. So nonscientists’ erroneous belief that type (B) situations are common is due to their erroneous belief that you can come up with a new scientific theory that works, without actually having to do the work involved in understanding what is currently known.

Of these possibilities, the third would appear to be the one most likely to spawn personal theories of the kind I referred to at the top of this article. And, conveniently, it also offers an explanation of why others are so seldom interested: because the obvious counterpoint to the view that anybody can overthrow an accepted scientific theory is to go too far in the other direction and believe that only professional scientists–those with degrees or other credentials, etc.–can come up with a valid scientific theory. So of course any random person posting on an internet forum can’t possibly have a valid theory.

But the fact that this heuristic works 99.9999% of the time still does not make it right. Unfortunately, I think a large part of the reason it is so often adopted is that professional scientists themselves promote it–wittingly or unwittingly. There is a flip side to the observation that nonscientists often come up with personal theories that nobody listens to: the observation that professional scientists, when talking to nonscientists, often fail to distinguish the varying levels of confidence we have in different parts of science, and often present science in a way that encourages people to say “Oh, wow!” and accept whatever they are told on the authority of the scientist, rather than to think critically and try to build an understanding of their own. This is why PF also has rules about acceptable sources: because even scientists can’t always be trusted to fairly represent science. At least in a peer-reviewed paper, other experts are looking at who can call them on it if they go too far afield (though admittedly that doesn’t always work either).

I’ve painted a fairly gloomy picture in this article, but please bear in mind that I’m focusing here on something that only makes up a small fraction of all the posts on PF. Most discussions here don’t raise either of the issues I describe above. But if you’re tempted to post about your theory, or if you’re tempted to ask a question based on a pop science source, it might be worth taking some time out to consider.