Occam's razor in science: all-time practice or modern fashion?

[BWV]

That last remark about scientific methods vs. science content caught my attention. Could you kindly elaborate on that? For I have been teaching such subjects for years and the syllabuses covered both.

And, please, don't just cite what the terms mean, we all know that.

Was just thinking of something along the lines of a civics class where students are taught about how science operates, it’s limits, how to recognize pseudoscience, general critical thinking etc.

 

[apostolosdt]

Was just thinking of something along the lines of a civics class where students are taught about how science operates, it’s limits, how to recognize pseudoscience, general critical thinking etc.

I see, yes, that would be an interesting class indeed. Thanks a lot for your answer.

 

[russ_watters]

It's common practice in government IT projects to decide not to make a decision. That's for sure!

It's a common part of a decision matrix that can apply to almost anything: Option 1 (or 0 if you prefer) can pretty much always be "do nothing".

Also, to me Occam's razor somewhat obscures it's own point. It's not that complexity itself is the enemy, it's all the added assumptions/speculation that tend to come with it. Epicycles, the aether, etc.

 

[Anachronist]

When I went to college, more than 30 years ago, as far as I can remember, nobody was talking about the use of Occam's razor in science.

I don't know where you went to college, but where I went to college (40 years ago) Occam's Razor was indeed a thing. I recall the topic coming up in the context of quantum mechanics, and at the time I remember thinking that some things about quantum mechanics could be considered an exception, but that was more because I wasn't very good at quantum mechanics. :)

 

[Vanadium 50]

So, let's pick on aether. ("There's nothing as helpless and irresposible and depraved as a man in the depths of an aether binge.") '

Which is less "complex"

1. Light waves behave just like sound waves, only in a different medium.
2. Light waves and sound waves are fundamentally different phenomena.

This is an example of what I called Monday morning quarterbacking. We decided #2 is simpler after we knew the answer.
Here's another one, aether-like:

1. General Relativity is not the correct theory of gravity at galactic scales.
2. There is an unseen and undetected Dark Matter permeating all of space, that can only be detected through its gravitational influence.

Which is the simpler theory? While #2 is most likely correct, I think it is at least questionable whether its simpler. And if "simple" and even "necessary" have subjective components, so does Occam.

Just like physical razor blades, it's possible - and uncomfortable - to overuse Occam's razor.

 

[BWV]

So, let's pick on aether. ("There's nothing as helpless and irresposible and depraved as a man in the depths of an aether binge.") '

Which is less "complex"

1. Light waves behave just like sound waves, only in a different medium.
2. Light waves and sound waves are fundamentally different phenomena.

This is an example of what I called Monday morning quarterbacking. We decided #2 is simpler after we knew the answer.
Here's another one, aether-like:

1. General Relativity is not the correct theory of gravity at galactic scales.
2. There is an unseen and undetected Dark Matter permeating all of space, that can only be detected through its gravitational influence.

Which is the simpler theory? While #2 is most likely correct, I think it is at least questionable whether its simpler. And if "simple" and even "necessary" have subjective components, so does Occam.

Just like physical razor blades, it's possible - and uncomfortable - to overuse Occam's razor.

Sure, but on aether, it remained preferred explanation because of the its simplicity until the evidence became overwhelming

Dark matter is different, because GR was already a successful theory, so the idea that something that fit the theory, i.e. unseen matter, was the OR answer. The analogous argument would be the ID 'we can't explain how this creature/organ or whatever evolved, so evolution must be wrong'

The Wiki entry offers a good explanation -

Similarly, in science, Occam's razor is used as an abductive heuristic in the development of theoretical models rather than as a rigorous arbiter between candidate models.[4][5] In the scientific method, Occam's razor is not considered an irrefutable principle of logic or a scientific result; the preference for simplicity in the scientific method is based on the falsifiability criterion. For each accepted explanation of a phenomenon, there may be an extremely large, perhaps even incomprehensible, number of possible and more complex alternatives. Since failing explanations can always be burdened with ad hoc hypotheses to prevent them from being falsified, simpler theories are preferable to more complex ones because they tend to be more testable.

 

[Dale]

Here's another one, aether-like:

1. General Relativity is not the correct theory of gravity at galactic scales.
2. There is an unseen and undetected Dark Matter permeating all of space, that can only be detected through its gravitational influence.

Which is the simpler theory?

There is a BIG difference. The aether has no measurable effect on anything. Dark matter does have a measurable effect on galactic rotation curves.

I always run back to home Bayes when talking about Occham's razor since it gives a systematic approach for figuring out exactly what we mean by "simple" and exactly when additional complexity is necessary.

For the case of the aether given any set of data it is equally as well explained by having the aether velocity be 0, or 0.1 c, or -0.6 c. It is a fudge-factor that doesn't change anything. The likelihood of the aether model gets "spread out" over this useless aether velocity parameter, and therefore it has a lower likelihood than a non-aether model for any set of data.

For the case of dark matter, given a specific galactic rotation curve there is a limited range of possible dark matter configurations. It is a fudge factor that must be set to a particular value to fix a given rotation curve. So the likelihood doesn't get spread out. The additional complexity is required to fit the data and only the prediction with the right amount of dark matter matches the data.

The issue with the dark matter is that there is no other constraint on that fudge factor. There is one distribution that works for a given galaxy, but no way to independently predict or confirm that that value is right. We like it when our theoretical constructs correspond to multiple observables. We want these dark matter curves that we can already measure from the velocity profile to match some "stuff" curve that we measure in some other manner.

 

[Vanadium 50]

There is a BIG difference. The aether has no measurable effect on anything.

We know that now. But in the second half of the 19th century, it was even predictive - Hertzian waves. (Which we now call "radio") But I would argue that in that period it was reasonable to consider aether models. The idea that Occam would have allowed that theory to be rejected back then is what I called Monday morning quarterbacking. It's easy to say now that those numbskulls back then should have just applied Occam and they would have gotten the right answer, I don't think that's right.

As far as rotation curves, in fact there are strong constraints on what you call a fudge factor. If you cast it in terms of an acceleration, you get the same value for every single rotationally supported galaxy. You could divide the sample in two, and use the data of one to predict, galaxy by galaxy what the rotation curves are for the others. There are good reasons to go for my #2 over my #1, but lack of predictive power or an ad hoc fudge factor would not be one.

I'll grant you that simple is better, but you have to grant me that simple is subjective.

 

[BWV]

I always run back to home Bayes when talking about Occham's razor since it gives a systematic approach for figuring out exactly what we mean by "simple" and exactly when additional complexity is necessary.

that looks an awful lot like a philosophical view

 

[Dale]

We know that now. But in the second half of the 19th century, it was even predicted - Hertzian waves. (Which we now call "radio") But I would argue that in that period it was reasonable to consider aether models. The idea that Occam would have allowed that theory to be rejected back then is what I called Monday morning quarterbacking.

I think that we are talking about different aether theories. You seem to be talking about models like the rigid aether that did have experimentally testable consequences. It is precisely those testable consequences that the MMX was designed to measure. Those models were not rejected based on Occham's razor, those models were rejected because they were contradicted by the data. So in the second half of the 19th century, before MMX, those models were indeed reasonable to consider.

Only the Lorentz aether was rejected due to Occham's razor. Occham's razor requires at least two candidate theories, so it couldn't have been rejected prior to 1905 when SR was developed. As soon as SR was developed then Occham's razor could be applied between LET and SR in favor of SR (regardless of the extant evidence), but not earlier. This, in fact, was probably one of the reasons that SR was so rapidly adopted. So I agree that the 19th century was too early to cut it, but I am also not sure that anybody even cared about LET until after other theories started being experimentally contradicted.

I'll grant you that simple is better, but you have to grant me that simple is subjective.

Yes in that the formal approach for deciding which is simpler relies on subjective probabilities and is sensitive to the priors.

 

[Dale]

that looks an awful lot like a philosophical view

But it isn't. I don't go to Bayes based on the convincing arguments of any pro-Bayes philosophers. Indeed, even now there is a considerable amount of anti-Bayes philosophical arguments. Frankly, I cannot judge the merits of these arguments nor can I usually spot the place where the anti-Bayes arguments go wrong.

I simply tried it out and found by my own experience that it works well. The philosophers may continue to argue, it is unimportant.

I suspect that average empirical evidence has resolved more philosophical debates that strong philosophical arguments have.

 

[Paul Colby]

that looks an awful lot like a philosophical view

Yeah, the problem I have is there seems no way to select which of many is the correct philosophical argument. Just being classified as a philosophy bring nothing to the table. In physics it’s empirical test that decides these things as far as I can tell.

 

[Klystron]

... Could you please cite some concrete references from modern literature to support them?

I can supply two references from literature that discuss or embody Occam's Razor as a principle in investigation. Both sources also include films derived from literature, perhaps more accessible to your students,

Arthur Conan Doyle describes essential principles of parsimony through fictional character Sherlock Holmes. While Doyle does not, as far as I remember, directly quote William of Ockham, his training as a medical doctor and subsequent writing style occupies similar philosophical space.

The detective's guiding principle, as he says in The Sign of Four, is: "When you have eliminated the impossible, whatever remains, however improbable, must be the truth."

Italian author and semiotics professor Umberto Eco brings William of Ockham to life as fictional Franciscan scholar 'William of Baskerville', a friend and colleague of the real William on a secretive mission to support the Franciscan viewpoints at an important European conclave in novel "Name of the Rose". William guides and instructs student Adso of Melk in the nuances of nominalism and mathematical deduction while solving several mysteries at the monastery hosting the conventicle.

 

[apostolosdt]

I can supply two references from literature that discuss or embody Occam's Razor as a principle in investigation. Both sources also include films derived from literature, perhaps more accessible to your students, [...]

I'm sorry to have put you in any trouble; it's my fault, I should have been more accurate in my wording. By "literature" I meant scientific literature, that is, published papers in scientific journals. I guess I thought that was obvious in a physics forum.

Anyway, apologies once again!

 

[russ_watters]

So, let's pick on aether. ("There's nothing as helpless and irresposible and depraved as a man in the depths of an aether binge.") '

Which is less "complex"

1. Light waves behave just like sound waves, only in a different medium.
2. Light waves and sound waves are fundamentally different phenomena.

That isn't a theory/phenomena, that's a procedure for finding it (and I agree it's generally a good/safe approach). The theory is "there's a phenomena there even though I can't see (detect) it." Occam's Razor can't do much when you don't have competing theories to compare. It might be worthwhile to ask "do I really need this thing that there's no evidence for yet?" but if you can't answer that or find a viable model that doesn't need it, then you're stuck with it until you do. Aether wasn't a bad idea until the MMX because it wasn't competing with/being compared to anything else..

 

[Lord Jestocost]

"Bringing out the Occam’s razor in peer-review"

Nature Nanotechnology, volume 17, page 561 (2022) https://www.nature.com/articles/s41565-022-01166-5

 

[apostolosdt]

"Bringing out the Occam’s razor in peer-review"

Nature Nanotechnology, volume 17, page 561 (2022) https://www.nature.com/articles/s41565-022-01166-5

Thanks a lot! Good editorial; I'll try to trace Hoffmann's, I liked his style!

 

[Klystron]

I'm sorry to have put you in any trouble; it's my fault, I should have been more accurate in my wording. By "literature" I meant scientific literature, that is, published papers in scientific journals. I guess I thought that was obvious in a physics forum.

Anyway, apologies once again!

Gracious response, thank you, but my error for not checking the subforum heading before responding. I was thinking "Art, History and Linguistics" instead of "Other Physics".

Discussing pre-Rennaissance philosophical concepts in light of modern science remains difficult no matter how useful.

 

[Vanadium 50]

"Bringing out the Occam’s razor in peer-review"

Nature Nanotechnology, volume 17, page 561 (2022) https://www.nature.com/articles/s41565-022-01166-5

I wonder how this will work in practice. "Why yes, there is a simpler explanation. The author is a fruitcake. Well, not exactly. More like a few pecans short of a fruitcake."

 

[apostolosdt]

I wonder how this will work in practice. "Why yes, there is a simpler explanation. The author is a fruitcake. Well, not exactly. More like a few pecans short of a fruitcake."

An editorial in Nature has always been carrying some weight, I guess. I downloaded Hoffmann's article (you can find it cited on that Nature page and it is free to download). It's 26 pages of chemistry stuff mostly (I feel comfortable with chemistry) and I'm going to read it all.

 

[JBGARNER]

Scientific practice does indeed change over time. For example, in the early 1900’s it was not even common to report error bars or uncertainty. Now it is effectively mandatory. It is not as though earlier scientists didn’t know about uncertainty, it just became more prevalent in scientific writing and more central in scientific thinking. Same with Occham’s razor.

By the way, in my opinion both Occham’s razor and Popper’s falsifiability are properly subsumed in Bayesian inference

I agree. I learned statistics from the text 'Theory of Probability' by Harold Jeffreys. He used William of Ockham's rule: 'Entities are not to be multiplied without necessity'. First edition 1939. The first convincing text about statistical inference that I found. I based my Ph.D on his work.
J.B. Garner.

 

[Rive]

I'm wondering if Occam's razor has always been an all-time practice in science, or if it has become a modern fashion?

As I see it, science always had an all-time practice (or, rather: requirement) about keeping things nice, clean, elegant (in scientific sense).

Later, some kind of formalization were required about this, either as a self-reflection and also as an explanation for the public. Thus, Occam's razor were invented/re-interpreted in that simple form.

My humble opinion is, that this attempt is at least partially backfired. Within science, it was not really needed and the question of 'what's simple' always/still remained a scientific one: while outside of science, the 'just because ... ' remained the simplest explanation, giving birth to a plethora of 'alternative sciences' now backed up with some popcult Occam's razor.

 

[Dale]

the 'just because ... ' remained the simplest explanation

”Just because” is a very complicated explanation. It has a separate term for each data point. There are, in principle, more complicated models possible, but “just because” is still quite complicated

 

[Rive]

”Just because” is a very complicated explanation. It has a separate term for each data point.

Yes, exactly that's the missing part which makes the difference between 'inside science' and 'outside science' interpretation of Occam's razor

'Outside science' any 'Joe' can easily provide (and: believe in!) even multiple versions of 'just because ... ' explanations (word salads) for each 'data point' without breaking a sweat - and it'll be a perfect 'simple' for him since it was soooo easily done.

So by my opinion it's a partial failure having Occam's razor publicized as a principle of science without the 'what's simple' part really strongly attached.

 

[Dale]

Yes, exactly that's the missing part which makes the difference between 'inside science' and 'outside science' interpretation of Occam's razor

Ah, I understand your point now.

Here is an Insights article I wrote on using Bayesian inference in place of the usual philosophy of science concepts like Ockham's razor.

https://www.physicsforums.com/insights/how-bayesian-inference-works-in-the-context-of-science/

Something like Bayesian inference allows the benefits of Ockham's razor (and other philosophical views) without promoting the "just because".

 

[Rive]

Here is an Insights article I wrote on using Bayesian inference

Seen that, but you know... 'inside science' it's nice but kind of redundant, while 'outside science' you'll lose the public at 'Baye...', because it continues with equations and not with football

I think to mitigate Occam's failure for the public you'll need something what can go with a beer or two too

 

[Dale]

'inside science' it's nice but kind of redundant

Well, I don't think it is redundant since many scientists are not familiar with Bayesian methods.

 

[apostolosdt]

I've read the first two references in Nature's editorial, as cited in Lord Jestocost's post, a few posts above, and I'd like to share some thoughts of mine about their claims.

In Hoffmann et al. review article, the authors discuss Occam's Razor, both historically and through comparisons with other "simplifying" principles, within the framework of reaction mechanisms in chemistry. In the last sections, they also mention its status in Bayesian analysis. As I understand from the numerous statistics-inclined posts in this thread, those sections will be most interesting. The article itself is rather long, twenty-one pages plus notes & references, but it's worth reading. The concluding passage is the authors' remark that Occam's Razor is more like "an instruction in an operating manual," rather than "a world view."

The second reference in Nature, however, is more intriguing. It's called "Inverse Occam's razor," by I. Mazin. It's not a research paper, call it a short review article. It talks about a current tendency among journal editors to prefer complicated papers rather than simpler ones---hence the name "inverse." The reasons appear to be less scientific and more about the impact such publications will make.

Philosophy is one discipline and science is another, and how much they overlap is, in my opinion, not an objective issue. I prefer to see them as `orthogonal' to each other and, personally, I'm reluctant to entrust the interpretation of a 21th-century set of experimental data to a 14th-century "scholastic Philosopher."

But let me, for the sake of argument, assume for a moment the validity of Occam's Razor. The `simpler approach' it suggests is how we, people, tend to model our data in the hope of tracing some pattern in them. That should never be taken as some indication that Nature prefers to be simpler as well! Natural phenomena can and, most probably, are too complicated. Practicing scientists experience that during all their daily work.

I particularly enjoyed what Feynman used to lecture about `simplicity' in Nature; the video is from his famous "Go somewhere else!" statement:

 

[Dale]

That should never be taken as some indication that Nature prefers to be simpler as well!

This is a good point. Sometimes you hear Occham’s razor mis-stated as “all other things equal the simplest explanation is usually right”. The simplest explanation is usually the best, even if it is not right. And in science we generally assume that none of our explanations are “right”, so “best” is all we can do.