Is Occam's Razor Obsolete in Light of Modern Scientific Complexity?

[Mentat]

IMO, Occam's Razor has not been properly defined here yet.

You see, Occam's Razor is not "When two hypotheses dictate the same thing, but one makes less assumptions than the other, that one is better", it is "When two hypotheses dictate the same thing, to the same level of accuracy and tenability, the one with less assumptions is better".

You mentioned QM, right. Well, QM needs to make more assumptions than Newtonian Mechanics would, if applied to the microscopic scale, doesn't it (so it seems that Occam's Razor would have failed here, right?)? However, QM is more accurate, and thus better by default.

 

[Ivan Seeking]

Originally posted by Mentat
IMO, Occam's Razor has not been properly defined here yet.

You see, Occam's Razor is not "When two hypotheses dictate the same thing, but one makes less assumptions than the other, that one is better", it is "When two hypotheses dictate the same thing, to the same level of accuracy and tenability, the one with less assumptions is better".

You mentioned QM, right. Well, QM needs to make more assumptions than Newtonian Mechanics would, if applied to the microscopic scale, doesn't it (so it seems that Occam's Razor would have failed here, right?)? However, QM is more accurate, and thus better by default.

The real point of this comparison is not that QM was less appropriate for describing the behavior of the very small than was Newtonian Mechanics, I was pointing to the continuing need for more complex descriptions of reality. Again, this and the fact that we do mostly assume and expect that a TOE can exist are somewhat of a dichotomy. The link here is not the explicit application of parsimony; it is the overall philosophy of William that we still see today. Obviously part of the reason that we expect a TOE is the elegance observed in nature through physics, so I don't mean to argue that such expectations are without merit, but that nature can be reduced to a finite set of equations is assumed without proof.

This seems to be one of the most basic assumptions of physics.

 

[Ivan Seeking]

Originally posted by Ambitwistor
I don't think it's the case that contradictory evidence is ignored, if that evidence is well-established. If it's on the edge of measurement, maybe. More rigorous evidence is required to cast doubt upon a well-established theory, after all. This can also be phrased rigorously in the language of probability theory.

Well, I didn't want to start a UFO discussion here, but I must, so I will tackle it this way. From what I have seen of RADAR data that corroborates multiple eyewitness testimony, some kind of unknown atmospheric phenomenon [at least] is seen and interferes with aircraft. Since this gets labeled UFO [ =ET ] a truly fascinating subject is generally ignored. The debunkers typically pull out Occam's Razor to discredit the entire event by way of the ET hypothesis. The assumption typically made is that this must be ET so it can't be real; so ignore the whole subject. I see similar thinking applied in most fringe subjects. So we don’t see Occam used to judge competing theories, we see it used discredit or ignore eyewitnesses, in addition to RADAR data and other evidence. The simplest explanation is always that the whole thing is a hoax; no matter how compelling the evidence.

 

[Ambitwistor]

Originally posted by Ivan Seeking
From what I have seen of RADAR data that corroborates multiple eyewitness testimony, some kind of unknown atmospheric phenomenon [at least] is seen and interferes with aircraft. Since this gets labeled UFO [ =ET ] a truly fascinating subject is generally ignored. The debunkers typically pull out Occam's Razor to discredit the entire event by way of the ET hypothesis. The assumption typically made is that this must be ET so it can't be real; so ignore the whole subject.

I don't really believe that anybody "typically" uses this argument. There have been plenty of weird "UFO" phenomena that were later found to have natural explanations, and educated debunkers know this. If I hear about some unexplained event in the sky, I certainly don't assume that it must be a hoax. I do assume that it probably has a non-ET explanation.

Moreover, assuming that "this must be ET so it can't be real" is not an application of Occam's Razor. It's not even an application of logic: "It must be ET. It can't be ET. Therefore it's not real." doesn't make any logical sense.

The simplest explanation is always that the whole thing is a hoax; no matter how compelling the evidence.

That's ridiculous. Conspiracy theorists are usually ignored precisely because deception is usually not the simplest explanation. Sometimes the hoax hypothesis is the simplest explanation consistent with the evidence, and sometimes it's not: it depends on what the alternate explanations are, and what the evidence is.

 

[Ambitwistor]

Originally posted by Ivan Seeking
Obviously part of the reason that we expect a TOE is the elegance observed in nature through physics, so I don't mean to argue that such expectations are without merit, but that nature can be reduced to a finite set of equations is assumed without proof.

This seems to be one of the most basic assumptions of physics.

First, this is independent of whether there is a TOE. (A TOE is not merely a finite description of nature, but a unified one.)

Then, it is not an assumption of physics. We'd come up with the same laws regardless of whether we believe there are finitely more or infinitely more laws yet to be discovered, because the goal is always to explain as much as we can: if we find a set of laws that happens to describe everything we can observe, then we don't have to assume that the set of laws is finite or infinite to do that. And supposing we do construct a set of self-consistent laws (TOE or not) that describe everything we observe. That still doesn't prove, and never can prove, that those are all the laws that exist. An assumption that there are a finite number of laws is just irrelevant to the process of physics.

 

[Mentat]

Originally posted by Ivan Seeking
The real point of this comparison is not that QM was less appropriate for describing the behavior of the very small than was Newtonian Mechanics, I was pointing to the continuing need for more complex descriptions of reality. Again, this and the fact that we do mostly assume and expect that a TOE can exist are somewhat of a dichotomy. The link here is not the explicit application of parsimony; it is the overall philosophy of William that we still see today. Obviously part of the reason that we expect a TOE is the elegance observed in nature through physics, so I don't mean to argue that such expectations are without merit, but that nature can be reduced to a finite set of equations is assumed without proof.

This seems to be one of the most basic assumptions of physics.

But wha thas it to do with Occam's Razor?

 

[Ivan Seeking]

Originally posted by Mentat
But wha thas it to do with Occam's Razor?

I assume that you stick with the strictest interpretation. As I said in the beginning, the interpretations of OR can run wild. I think the philosophical motivations for OR permeates the thinking of most people more than we generally realize. Again, the most general embedded philosophy here is that we should expect simplicity from nature. The linked article discusses how philosophy does play an active role in the scienctific process.

 

[Ivan Seeking]

Originally posted by Ambitwistor
I don't really believe that anybody "typically" uses this argument. There have been plenty of weird "UFO" phenomena that were later found to have natural explanations, and educated debunkers know this. If I hear about some unexplained event in the sky, I certainly don't assume that it must be a hoax. I do assume that it probably has a non-ET explanation.

I can only tell you what I hear and see.

Moreover, assuming that "this must be ET so it can't be real" is not an application of Occam's Razor. It's not even an application of logic: "It must be ET. It can't be ET. Therefore it's not real." doesn't make any logical sense.

From what I see, the problem is not the stories that can be explained, those are easy to deal with, it's the credible stories that can't be explained. Since we have no convienent explanations, the default explanation becomes ET - which of course is much less likely to be the explanation than a hoax. Therefore its a hoax. In other words, manipulate the argument so that we can wrongly apply OR.

That's ridiculous. Conspiracy theorists are usually ignored precisely because deception is usually not the simplest explanation.

I never said anything about conspiracies. ?

Sometimes the hoax hypothesis is the simplest explanation consistent with the evidence, and sometimes it's not: it depends on what the alternate explanations are, and what the evidence is. [/B]

The problems come when we have no viable alternative explantion.

 

[Ambitwistor]

Originally posted by Ivan Seeking
From what I see, the problem is not the stories that can be explained, those are easy to deal with, it's the credible stories that can't be explained. Since we have no convienent explanations, the default explanation becomes ET - which of course is much less likely to be the explanation than a hoax.

To a skeptic, ET isn't the default explanation. If anything, "hoax" or "unexplained natural phenomenon" is the default explanation.

I never said anything about conspiracies. ?

So? The point is that hoaxes or other acts of deception are not always the simplest explanation for anything.

The problems come when we have no viable alternative explantion.

If we have no viable alternative explanation to a hoax, then by definition, a hoax is the best explanation that we have. What is the problem?

 

[Ivan Seeking]

Originally posted by Ambitwistor
Then, it is not an assumption of physics. We'd come up with the same laws regardless of whether we believe there are finitely more or infinitely more laws yet to be discovered, because the goal is always to explain as much as we can: if we find a set of laws that happens to describe everything we can observe, then we don't have to assume that the set of laws is finite or infinite to do that. And supposing we do construct a set of self-consistent laws (TOE or not) that describe everything we observe. That still doesn't prove, and never can prove, that those are all the laws that exist. An assumption that there are a finite number of laws is just irrelevant to the process of physics.

True. We can always assume processes exist that we never observe; so there could be physical laws veiled forever from our view. However, today we find that a great disconnect exists between two highly successful theories - GR and QM. Unification of these two theories is considered the holy grail of modern physics. As an example of my argument, with one exception - my favorite college physics professor - I have never heard anyone suggest that unification may not ever be possible. I have surely never heard a discussion on why unification fundamentally may not be possible. Of course, if M theory ever leaps from the realm of philosophy to that of physics by way of testable predictions, I admit that my point could be moot.

 

[Ambitwistor]

Originally posted by Ivan Seeking
However, today we find that a great disconnect exists between two highly successful theories - GR and QM. Unification of these two theories is considered the holy grail of modern physics. As an example of my argument, with one exception - my favorite college physics professor - I have never heard anyone suggest that unification may not ever be possible.

I think you're confused about the difference between quantum gravity and a unified field theory. If quantum mechanics is valid, then a theory of quantum gravity must exist, because gravity exists. This is required just from the standpoint of logical consistency. (It's conceivable that quantum mechanics is wrong and must be modified, in which case we wouldn't strictly call a successor theory of gravity "quantum" gravity. However, we do know that general relativity must be replaced by some successor theory that is compatible with quantum mechanics as we know it.) This is not regarded as a "unification" of theories; quantum mechanics itself is not a theory, but a framework in which specific theories (quantum theories of gravity, quantum theories of electromagnetism, quantum theories of matter, etc.) can be cast.

On the other hand, string theory attempts to not only quantize gravity, but unify it with the other interactions. Unlike the quantization of gravity, there is no a priori reason to believe that unification must be possible.

 

[Ivan Seeking]

Originally posted by Ambitwistor
If we have no viable alternative explanation to a hoax, then by definition, a hoax is the best explanation that we have. What is the problem?

Exactly! You just did what I am claiming.

How about considering that we don't know all of the potential explanations? This is another option that gets ignored. This exactly is where I see that science goes blind. This is why so many people like me find this so frustrating. We are convinced that something vert strange is afoot, but we can't imagine what.

 

[Ambitwistor]

How about considering that we don't know all of the potential explanations? This is another option that gets ignored.

I didn't ignore that option. I said that if you have only one viable explanation, you have only one viable explanation. That doesn't preclude the possibility of other viable explanations that you don't have.

This exactly is where I see that science goes blind.

Utter nonsense. Science does not ignore the possibility of unknown explanations. If it did, then science wouldn't exist at all.

Occam's Razor applies to explanations that you do have; it can't be applied to explanations that you don't have, because you can't say anything about either how simple they are, or how well they fit the evidence.

 

[Ivan Seeking]

Originally posted by Ambitwistor
there is no a priori reason to believe that unification must be possible.

That makes two.

 

[Ivan Seeking]

Originally posted by Ambitwistor
Utter nonsense. Science does not ignore the possibility of unknown explanations. If it did, then science wouldn't exist at all.

I just gave an example of where this does often happen. Though in fairness, there are a few scientists willing to throw their careers to the wind and pursue unpopular subjects.

Occam's Razor applies to explanations that you do have; it can't be applied to explanations that you don't have, because you can't say anything about either how simple they are, or how well they fit the evidence.

However it can be used wrongly to defer further investigation. In short, it is used to rule out unknown explanations.

 

[Ambitwistor]

Originally posted by Ivan Seeking
I just gave an example of where this does often happen.

Ivan, in a field where hoaxes and mundane natural and man-made phenomena have often proven to be correct explanations, hoaxes and mundane natural and man-made phenomena are rightly a good assumption. There simply isn't the funding to go chasing after every weird thing that some guy claimed he saw in the sky, considering how often people have studied UFO phenomena in the past and turned up no amazing new phenomena.

However it can be used wrongly to defer further investigation. In short, it is used to rule out unknown explanations.

Unknown explanations are never "ruled out", certainly not by Occam's Razor. Unknown explanations may not be actively pursued because nobody is interested, in a field that historically has not proven a fruitful source of interesting new phenomena.

 

[Ivan Seeking]

Originally posted by Ambitwistor
Ivan, in a field where hoaxes and mundane natural and man-made phenomena have often proven to be correct explanations, hoaxes and mundane natural and man-made phenomena are rightly a good assumption.

Approximately 90-95% of all sightings. True.

There simply isn't the funding to go chasing after every weird thing that some guy claimed he saw in the sky, considering how often people have studied UFO phenomena in the past and turned up no amazing new phenomena.

Again, the cases that interest me do not depend on what "some guy claimed he saw in the sky". Next, only after many years of refusals has the journal Nature agreed [in principle at least] to publish well penned papers on the subject. Funding is not the only issue.

Unknown explanations are never "ruled out", certainly not by Occam's Razor. Unknown explanations may not be actively pursued because nobody is interested, in a field that historically has not proven a fruitful source of interesting new phenomena. [/B]

I think you've hit the nail on the head. The trouble is, no one is smart enough to come up with anything the rival the ET hypothesis. So, since nature abhors a vacuum, we get Art Bell.

 

[Iacchus32]

That which is most obvious -- and hence, easiest to deduce -- is usually that which is most prevalent. Whereas that which is most prevalent, is usually the "status quo."

Hence it would seem, Occam's Razor is quite often "a ploy" to maintain the status quo.

 

[geometer]

I have quickly skimmed the posts here, so I apologize in advance if I am restating something or missing the whole point here. I remember reading somewhere Occam's actual words which were: "Construct no unnecessary hypotheses." Our interpretation of this as "All things being equal, the simplest explanation is probably the right one" is an interpretation of this statement or maybe a corollary.

 

[vanesch]

I don't see any simple explanations in modern physics. All of our simple expectations failed.

One shouldn't confuse "simple" with "simplistic" or "naive". The first one refers to the use of only a few, even if very abstract and counter intuitive, concepts. Modern physics strives towards it, with success. The latter means: what I can find without much thinking.
Unification is the typical example in physics, and has already worked out with success several times. Take electricity and magnetism. Before Maxwell, there were several, apparently unrelated, different, but indeed simplistic, explanations of several phenomena: static electricity, currents, magnetic needles etc...
Maxwell 4 equations, using 2 vector fields, describing ALL those phenomena at once. Although conceptually harder to understand, LESS concepts were used, in a more abstract way. This is what "simple" means in physics. A bit later, it was possible to reformulate, in relativistic language, the same mathematical content, but using ONE SINGLE entity, the 4-potential, and 1 equation. Again, things became less obvious and more abstract, but with less different concepts: simplicity again.
This story went on and on, for most of the 20th century.

Simpler, and at the same time more abstract and difficult. But simpler, because of a lower number of different concepts.

cheers,
Patrick.