Are real theories undiscoverable from effective field theories?

[pines-demon]

TL;DR Summary

Can somebody provide some overview on how effective field theories can hide features of the real theory?

In a podcast episode, Sean Carroll discusses Wilson's renormalization group and effective field theories, arguing how Wilson showed that upper-level theories are not discoverable from effective field theories at low energy. Excerpt:

The idea of effective field theory is you can do quantum field theory perfectly well at energies below the ultraviolet cutoff without knowing what the physics is above the ultraviolet cutoff. Even if space and time themselves exist above the ultraviolet cutoff, you don't need to know that, it will not show up in your low energy effective quantum field theory. So this and other things, and not exactly this, but related things won Wilson, the Nobel Prize and the undying reverence of modern quantum field theorists.

Thee good news is that effective field theories are effective, in some sense we were already using that fact before we knew it. The first field theory, the first modern quantum field theory is written down by Enrico Fermi in his theory of beta decay. And the fields that he was considering were protons and neutrons and electrons and neutrinos. These days we know number one, that protons and neutrons are not elementary fields. You can still treat them as fields and the particles as vibrations in the fields, but we have a deeper picture in which they are quarks and glue-ons and so forth. And also we know that the beta decay interaction is an example of the weak interactions mediated by W Bosons. Fermi didn't know any of that.

Why was Fermi in the 1930s able to invent such a good theory without knowing about quarks and glue-ons and the W bosons? The answer is that secretly he has an effective field theory. His theory is valid below a certain energy scale. The thing about effective field theories is they will be valid below the ultraviolet cutoff. They may or may not continue to be valid above the ultraviolet cutoff. When you go above the ultraviolet cutoff, you are open to the possibility that very new physical phenomena kick in maybe new particles, new fields, maybe even something that is not in the realm of quantum field theory at all. That's the power of effective field theory. It doesn't need to assume that the fundamental nature of reality is a quantum field theory. The low energy visible nature of reality will still be very well, described by quantum field theory.

In what sense is this true? Does that mean that no matter how precise the experiment no upper-level theory can be discovered? No matter how many experiments on beta theory it was not possible to uncover the weak force at low energy? Does that mean we cannot discover fundamental physics without a larger collider?

I wonder if somebody can provide some insight on how this effective "hiddenness" works and what are the key concepts to understand here. What are the keywords and theorems to understand to make such a claim.

 

[ohwilleke]

I'm going to object to the phrase "real theories", because any explanation for how things work is a real theory. It exists. It works. It isn't fake.

I think that the phrase you are looking for is "more fundamental theories".

I will also briefly mention in passing some of the proposals that have been made for more fundamental theories in lieu of an effective field theory Standard Model, mostly because the concept is just easier to understand in the context of some concrete examples, rather than in terms of pure generalities.

But, to illustrate the basic idea of "more fundamental" as opposed to "real" theories, consider, for example, that Newtonian gravity is a theory (quite a useful one, in fact, and definitely not a "fake theory"), but Special and General Relativity are more fundamental theories, and a hypothetical theory of quantum gravity might be more fundamental still.

Newtonian gravity works fine for most practical purpose in its domain of applicability (i.e. the conditions and circumstances under which the theory works) but it breaks down at very high speeds and very strong gravitational fields which are called "relativistic conditions." Still, the fact that general relativity produces "singularities" where the math blows up in black holes and at the Big Bang, and the fact that general relativity is not mathematically consistent in some respects with the Standard Model of Particle Physics, are facts that are big hints that general relativity may not be the most fundamental of all possible theories of gravity.

As a 2022 paper by Sean Carroll about effective field theories explains in its abstract:

Effective Field Theory (EFT) is the successful paradigm underlying modern theoretical physics, including the "Core Theory" of the Standard Model of particle physics plus Einstein's general relativity. I will argue that EFT grants us a unique insight: each EFT model comes with a built-in specification of its domain of applicability. Hence, once a model is tested within some domain (of energies and interaction strengths), we can be confident that it will continue to be accurate within that domain. Currently, the Core Theory has been tested in regimes that include all of the energy scales relevant to the physics of everyday life (biology, chemistry, technology, etc.). Therefore, we have reason to be confident that the laws of physics underlying the phenomena of everyday life are completely known.

Similarly, classical electromagnetism as described by Maxwell's equations is a theory, quantum electrodynamics is a more fundamental theory, the Standard Model of Particle Physics provides a broad context for quantum electrodynamics that is more complete, and perhaps, there is a yet more fundamental grand unified theory or theory of everything yet to be discovered that is out there. (You can actually get a mathematical hint about the existence of special relativity and a fixed speed of light in a vacuum from Maxwell's Equations, although special relativity was proposed so soon after they were formulated that special relativity didn't end up primarily taking that path to being discovered.)

Consider another effective theory which we might call the PNE theory for proton-neutron-electron theory. This is perfectly adequate for describing low energy chemical reactions involving stable isotopes of atoms, in which atoms stay bound in their nuclei for reasons that aren't further explained (just like people just starting to learn about quantum chromodynamics are not really told "why" quarks and gluons are always confined in composite hadrons). But PNE theory is not adequate for describing what happens in a particle accelerator or a radioactive isotope of an atom. You can do chemical reactions day in and day out for decades in the domain of applicability of PNE Theory and never gain any insight into the composite nature of protons and neutrons or nuclear physics. You need to expand your theory to include quarks, gluons, and other relevant pieces of the Standard Model of Particle Physics to do that adequately, and you can only learn about those things in high energy physics experiments.

You can describe radioactive decay in an effective theory that identifies the possible modes of radioactive decay and treats the probability of different modes of radioactive decay occurring as experimentally measured constants, although this still requires the addition of the electron neutrino to your model. This is a real theory too. You can find those experimentally measured physical constants in standard reference books for chemistry, physics, and nuclear physics. It is taught to high school and college students on a regular basis. It is useful.

But you need a higher energy experiment (or some other clever natural experiment of some kind) to understand the inner structure of hadrons like protons or neutrons, that will tell you where all those physically measured properties of radioactive elements come from, or how nuclei can be fused as well as split.

Carroll is taking the common position that the Standard Model of Particle Physics is a low energy effective field theory whose upper domain of applicability isn't know, and which may be (and implicitly, probably is) a manifestation of a more fundamental theory whose properties are only observable at higher energies.

Carroll's position is one that isn't as heavily emphasized these days as it used to be, a few decades ago when I was a kid and then a college student, for a few reasons:

1. In the early days of renormalization (which makes quantum physics calculations possible and explains why the physical constants of the Standard Model of Particle Physics change in a systematic way at different energy scales), when Carroll starting thinking about these issues, we didn't know if renormalization was mathematically legitimate in a rigorous way, or if it was only a mathematical trick which might have a much more limited domain of applicability. It was only shown rigorously much later that it is mathematically legitimate to arbitrarily high energy scales.

2. The discovery of the Higgs boson and its relatively low mass did a couple of things: it made the Standard Model basically complete, and it made the Standard Model mathematically free of pathologies up to a very high energy scale (basically up to the "grand unified theory" a.k.a. "GUT" energy scale of 10¹⁶ GeV/c²). Before that, it was possible that we needed new physics just to make the Standard Model mathematically coherent up to high energies. Even as it is, the Higgs field that imparts mass to fundamental particles in the Standard Model reaches a point where the vacuum becomes only "metastable" (i.e. prone to collapse over times frames on the order of the age of the universe or more, which is predicted to happen at about 10¹² GeV/c² which is eight orders of magnitude higher in energy in than the LHC equivalent in the low energy direction to a drop to about the energy scale of a single electron) at high enough energies, which would be a natural place to look around for new physics if we could create high enough energies in controlled conditions (which we can't, which is probably for the best, since it could cause the universe to unravel which might be unpleasant).

3. A lot of the physicists who emphasized thinking about the Standard Model of Particle Physics as a mere effective field theory were almost sure that new physics, beyond the Standard Model, would be discovered at the Large Hadron Collider at its peak energies of about 14 TeV/c² or less.

This was a natural expectation. From the early 1970s until the Tevatron collider completed its run in the year 2011 (a year before the Higgs boson was discovered), it seemed like every a few years, high energy physicists were discovering new fundamental particles or other new physical phenomena (like neutrino oscillation), the discovery of dark matter and dark energy phenomena provided a motivation for more particles to be discovered, and one got in the habit of feeling like each new higher energy collider with discover more new physics.

But as it happened, none of the new physics - new particles, new forces, etc. other than what appears to all examination to be consistent with a Standard Model Higgs boson that was discovered more than twelve years ago, and the separate lower energy physics discovery that at least two of the three active neutrinos have a non-zero mass. New composite particles like tetraquarks and pentaquarks have been discovered, but they were predicted to exist in the Standard Model even though we hadn't observed them yet when they were discovered (obviously).

Supersymmetry, multiple Higgs doublets, technicolor, leptoquarks, and many other hypothetical beyond the Standard Model Particles and new physics which lots of high energy physicists and theorists expected to see in some form or another, have not been detected at the LHC and have been ruled out up to partially model-dependent mass and cross-section of interaction thresholds.

Of course, theorists promptly imagined new physics that could exist at energies just a little higher than those currently capable of being explored at the LHC (which are on the order of 10⁴ GeV/c²). They didn't ignore the evidence, they just adapted to this new information. But this gets somewhat less exciting when you've moved the energy scale at which you predict there will be new physics up a few times and no new physics appears.

Put another way, there is really no positive evidence from high energy physics experiments that are strongly calling for new physics.

Indeed, the evidence that Big Bang Nucleosynthesis (which predicts the ratio of atomic elements that would be formed based upon know high energy physics and chemistry and nuclear physics) at temperatures that are expected to have been present maybe 15 minutes after the Big Bang.

Lots of proposed kinds of new physics would screw up Big Bang Nucleosynthesis which has been largely confirmed with astronomy observations, so at energy scales present 15 minutes after the Big Bang, or at lower energy scales, there are a lot of serious constraints on high energy new physics, so the cosmology/energy scale window in which the Standard Model of Particle Physics might not work is quite constrained.

Conditioned by these experiences and this knowledge, the conventional wisdom among many younger high energy physicists is trending towards an expectation of what is sometimes called a "new physics desert" in which there is no beyond the Standard Model physics between the energy scale at which top quark pairs can be produced (in theory about 350 GeV/c², but an order of magnitude higher to produce them in quantity), and some new high energy scale where new physics appears again that is many orders of magnitude higher (for example, 10¹² GeV/c² or more). In this expectation, the Standard Model continues to operate normally as a "low energy effective field theory" up to this threshold over many orders of magnitude, and then some unknown and unanticipated new physics starts to crop up again at this extreme high energy threshold which is many generations of particle colliders away, if it can ever be observed and studied.

This said, dark matter and dark energy phenomena from astronomy and cosmology to provide some motivation for new physics of some, undetermined kind. Proposals for that are all over the map, and nothing proposed yet works perfectly in all circumstances and has wide acceptance among scientists. So, there totally could be some sort of more fundamental theory that explains these phenomena with new particles or forces or modifications to existing ones. But we know little about these kinds of new physics because we haven't found a way to study these new physics directly, as opposed to inferring them from astronomy observations that don't seem to make sense without new physics.

Other motivations for new physics like the baryon asymmetry of the universe (i.e. the excess of matter over antimatter in particles that involve quarks like protons and neutrons) also suggest that there might be some new physics at very high, perhaps shortly post-Big Bang scale energies (we are talking seconds or even a mere tiny fraction of a second after the Big Bang in this case).

But possible new physics at shortly post-Big Bang scale energies are only so interesting to high energy physicists, because they have no meaningful hope of every recreating those energies in an experiment man-made or natural, that would allow you to study and quantify these new physics. And, any new physics that are observed would be observed by astrophysicists and not by them.

Now, generally speaking, high energy physicists and astrophysicists, when they are talking about an effective field theory, assume that the place where new physics will be found is at high energies. They don't call them "high energy physicists" for nothing and this is their natural, disciplinary predisposition.

One could imagine some circumstances other than high energy physics experiments (or natural experiments) in which new physics manifests in a way that we wouldn't have discovered yet that could be outside the domain of applicability of an effective field theory.

For example, rather than having an ultra-high energy "UV" threshold, one can imagine an effective field theory that has an ultra-low energy infrared "IR" threshold in which the physics change at ultra-low energies or in places that are completely free of significant gravitational fields. But, since IR physics are much more throughly explored than UV physics, the conventional wisdom is that high energies is where we would expect effective field theories to break down.

Indeed, there is at least one phenomena, called a "sphaleron" interaction that doesn't follow the same rules about "baryon number conservation" and "lepton number conservation", which is supposed to take place only at extremely high energies, even within the Standard Model. But we don't know for sure that this piece of the Standard Model is correct because we haven't been able to reproduce those extremely high energy conditions yet (it might take 10-100 times the energies of the LHC to do so).

Similarly, the Standard Model predicts that composite particles bound by the strong force called hadrons break up and form a new state called "quark-gluon plasma" which was mathematically predicted by the Standard Model, but we couldn't be sure that the Standard Model was reliable up to those energies until we (probably) briefly produced some of it at the LHC.

Both of those are examples of high energy new physics phenomena that can be predicted from the mathematics of an effective field theory (a bit like predicting the speed of light from Maxwell's equations), but there is no guarantee that this will always be the case. It could be that similar phenomena to sphalerons or quark-gluon plasma, that are not mathematically predicted by the low energy effective field theory of the Standard Model also exist at high enough energies. But we have no hint of these new physics because we've done no lower energy physics that would provide us with a hint that they might exist. It isn't logically necessary that hints of new physics at high energies be reflected in the low energy behavior of a physical system.

Mathematically, one way that this can happen is that there is a "UV fixed point" analogous to the speed of light, and new physics only starts to appear as your system gets close to the "UV fixed point" which is at some extremely high energy scale.

Another way that it could happen is that there could be a state change, sort of like a melting point or a transition to quark-gluon plasma, where some force that you may not have even known existed gets overwhelmed by the high energies leading to qualitatively different behavior. Higgs field metastability is another example of that.

One commonly imagined kind of hidden high energy sector in a higher energy theory would be that the Standard Model forces might start to get more and more similar in strength and behavior at very high energies until they merge into a single unified force that spontaneous breaks the symmetry of being a unified single force at lower energies. Spontaneous symmetry breaking would be another good key word to search.

Another possible kind of high energy new physics might be high energy excited state resonances of particles that were thought to be fundamental, a bit like overtones in music, that only start to show up at higher energies. Composite hadrons can have higher energy excited states in addition to their minimum energy ground states, and it wouldn't be inconceivable that particles we think of as fundamental (particularly force carrying bosons, or the Higgs boson) might do so as well.

Yet another possibility is that at high enough energies the existence of additional dimensions beyond the usual three of space and one of time, become observable in some way.

Likewise, the fabric of space-time might seem different at the extremely short distances in time and space associated with Planck time and Planck space, at which the uncertainty principle and dimensional analysis suggests that new physics might be lurking in what is seemingly a continuous, smooth space-time to our macroscopic senses.

 

[pines-demon]

Thanks for your examples. They are helpful.

I'm going to object to the phrase "real theories", because any explanation for how things work is a real theory. It exists. It works. It isn't fake.

I think that the phrase you are looking for is "more fundamental theories".

Got it.

But, to illustrate the basic idea of "more fundamental" as opposed to "real" theories, consider, for example, that Newtonian gravity is a theory (quite a useful one, in fact, and definitely not a "fake theory"), but Special and General Relativity are more fundamental theories, and a hypothetical theory of quantum gravity might be more fundamental still.

Newtonian gravity works fine for most practical purpose in its domain of applicability (i.e. the conditions and circumstances under which the theory works) but it breaks down at very high speeds and very strong gravitational fields which are called "relativistic conditions." Still, the fact that general relativity produces "singularities" where the math blows up in black holes and at the Big Bang, and the fact that general relativity is not mathematically consistent in some respects with the Standard Model of Particle Physics, are facts that are big hints that general relativity may not be the most fundamental of all possible theories of gravity.

Is Newtonian gravity an effective theory though? Wouldn't the precession of Mercury tell us that there was something wrong with it (without the need to go higher in energy)?

Consider another effective theory which we might call the PNE theory for proton-neutron-electron theory. This is perfectly adequate for describing low energy chemical reactions involving stable isotopes of atoms, in which atoms stay bound in their nuclei for reasons that aren't further explained (just like people just starting to learn about quantum chromodynamics are not really told "why" quarks and gluons are always confined in composite hadrons). But PNE theory is not adequate for describing what happens in a particle accelerator or a radioactive isotope of an atom. You can do chemical reactions day in and day out for decades in the domain of applicability of PNE Theory and never gain any insight into the composite nature of protons and neutrons or nuclear physics. You need to expand your theory to include quarks, gluons, and other relevant pieces of the Standard Model of Particle Physics to do that adequately, and you can only learn about those things in high energy physics experiments.

You can describe radioactive decay in an effective theory that identifies the possible modes of radioactive decay and treats the probability of different modes of radioactive decay occurring as experimentally measured constants, although this still requires the addition of the electron neutrino to your model. This is a real theory too. You can find those experimentally measured physical constants in standard reference books for chemistry, physics, and nuclear physics. It is taught to high school and college students on a regular basis. It is useful.

Would it be considered an effective theory even if it had inconsistencies? For example the PNE theory without the neutrino does not work right, it needs the neutrino but going into higher energies was not necessary to discover it. So PNEn (n for neutrino) would be an effective theory but not the PNE without n. Right? Is this what you call IR physics?

Carroll is taking the common position that the Standard Model of Particle Physics is a low energy effective field theory whose upper domain of applicability isn't know, and which may be (and implicitly, probably is) a manifestation of a more fundamental theory whose properties are only observable at higher energies.

Carroll's position is one that isn't as heavily emphasized these days as it used to be, a few decades ago when I was a kid and then a college student, for a few reasons:

1. In the early days of renormalization (which makes quantum physics calculations possible and explains why the physical constants of the Standard Model of Particle Physics change in a systematic way at different energy scales), when Carroll starting thinking about these issues, we didn't know if renormalization was mathematically legitimate in a rigorous way, or if it was only a mathematical trick which might have a much more limited domain of applicability. It was only shown rigorously much later that it is mathematically legitimate to arbitrarily high energy scales.

How does quantized gravity fits into this effective field point of view? Its divergences could be avoided by just saying that it is just an effective field theory or renormalization is still important here?

Conditioned by these experiences and this knowledge, the conventional wisdom among many younger high energy physicists is trending towards an expectation of what is sometimes called a "new physics desert" in which there is no beyond the Standard Model physics between the energy scale at which top quark pairs can be produced (in theory about 350 GeV/c², but an order of magnitude higher to produce them in quantity), and some new high energy scale where new physics appears again that is many orders of magnitude higher (for example, 10¹² GeV/c² or more). In this expectation, the Standard Model continues to operate normally as a "low energy effective field theory" up to this threshold over many orders of magnitude, and then some unknown and unanticipated new physics starts to crop up again at this extreme high energy threshold which is many generations of particle colliders away, if it can ever be observed and studied.

Mathematically, one way that this can happen is that there is a "UV fixed point" analogous to the speed of light, and new physics only starts to appear as your system gets close to the "UV fixed point" which is at some extremely high energy scale.

One commonly imagined kind of hidden high energy sector in a higher energy theory would be that the Standard Model forces might start to get more and more similar in strength and behavior at very high energies until they merge into a single unified force that spontaneous breaks the symmetry of being a unified single force at lower energies. Spontaneous symmetry breaking would be another good key word to search.

Again could precision experiments tells us what are we missing? or is higher energies strictly necessary? I noticed that you have not talked about Wilson or the renormalization group here. I want to know if there is some kind or theorem or broader set of rules that show that for some reason any more fundamental theory would be "optimally hidden" below the UV cutoff as an effective field theory.

 

[Reggid]

The effects that the effective theory neglects are suppressed by terms of order of the energy scale of the process divided by the energy scale of the ultraviolet cutoff. In the case of Fermi-theory and neutron decay the energy scale of the process is or order 1 MeV (the mass difference of neutron and proton), and the cutoff scale is the mass of the W-boson of 80 GeV.

If you want to see some effects that the effective theory can not describe - which means that they hint at a more fundamental theory - that you currently can't see, you can look at processes at higher energies, for which the effects will be larger because the numerator of the ratio of energy scales will be larger, or you can increase your precision in order to be able to see also the smaller effects.

You do not necessarily always need higher energies. Higher precision and higher energies are two different frontiers in the search for new physics.

 

[pines-demon]

The effects that the effective theory neglects are suppressed by terms of order of the energy scale of the process divided by the energy scale of the ultraviolet cutoff. In the case of Fermi-theory and neutron decay the energy scale of the process is or order 1 MeV (the mass difference of neutron and proton), and the cutoff scale is the mass of the W-boson of 80 GeV.

If you want to see some effects that the effective theory can not describe - which means that they hint at a more fundamental theory - that you currently can't see, you can look at processes at higher energies, for which the effects will be larger because the numerator of the ratio of energy scales will be larger, or you can increase your precision in order to be able to see also the smaller effects.

Ok, but is this insight coming from some general theory of effective field theories (RG?), or is this coming from Taylor expanding specific examples? Is there a reason for example to consider that the corrections are in larger power than just quadratic and thus more hidden?

You do not necessarily always need higher energies. Higher precision and higher energies are two different frontiers in the search for new physics.

Thanks, that’s what I mainly wanted to understand.