On Quanta article on the recent tensions of cosmology

[pines-demon]

Quanta Magazine recently published "Clashing Cosmic Numbers Challenge Our Best Theory of the Universe"

I am no fan of Quanta and I am not a cosmologist but there is certainly the increasing buzz that something is wrong with $\Lambda$-CDM and cosmological models thanks to the JWST. Is astronomy always so hyped or is there something novel going on? Can somebody comment on how this is just noise or are we getting close to paradigm shift in the upcoming decades?

 

[PeterDonis]

Moderator's note: Thread level changed to "I".

 

[PeterDonis]

Can somebody comment on how this is just noise or are we getting close to paradigm shift in the upcoming decades?

I think the current state is somewhere in between. There are genuine questions raised by the new data from the JWST, but I think the jury is still out on whether those questions can be answered satisfactorily within the general framework of $\Lambda CDM$ or whether a true paradigm shift is needed.

Generally speaking, "buzz" and "hype" should be disregarded with respect to any scientific field; even in fields where there is no serious disagreement at all among actual scientists, you will see "buzz" and "hype" in the popular press that can make it seem like there are issues.

As far as this particular issue is concerned, you will find plenty of discussion in previous PF threads that references the published scientific literature, which is the best place to look to learn about the actual state of research in a scientific field. Be warned, though, that those papers will generally require an "A" level background to be properly understood.

 

[Ibix]

I think the answer is "maybe".

I think the JWST provides a lot of data that people can jump on to push their own pet theory. That's where all the hype is coming from.

I think it's worth bearing in mind that for every time we've stared at Mercury asking "why isn't it in the right place" there've been at least as many times we've been staring at the Pioneer probe asking "why isn't it in the right place". Even when there is a genuine anomaly, sometimes the source is mundane.

 

[PeterDonis]

at least as many

More like "many, many, many more". The vast majority of apparent "anomalies" in scientific data end up having mundane resolutions. (For an even better recent case than the Pioneer anomaly, consider the OPERA neutrino experiments that appeared to show neutrinos traveling FTL.) Cases like the perihelion precession of Mercury are extremely rare. The only reason the latter cases appear to be more common is that they're the ones that get remembered long after they have been resolved.

 

[ohwilleke]

Is astronomy always so hyped or is there something novel going on?

Astronomy is always so hyped, but there is genuinely something of importance going on.

Can somebody comment on how this is just noise or are we getting close to paradigm shift in the upcoming decades?

The Lambda-CDM theory faces lots of problems, at multiple scales and in multiple independent circumstances, that are not just noise. Many of these problems far pre-date the JWST. Most of the observations and theoretical predictions at the root of these problems have been repeatedly replicated by multiple independent groups of astronomers and physicists.

The role of the JWST in bringing a crisis to a head, in general

But the JWST due to its greater precision and greater reach into the far infrared spectrum has a tendency to increase the statistical significance of tensions with expected values.

The statistical significance of a tension with an expected or predicted value is equal to the difference between the observed value and the expected value divided by the one sigma (i.e. one standard deviation) uncertainty of the difference between the observed value and the expected value.

What the JWST is doing is decreasing the uncertainty (expressed as the standard deviation of the value) in the observed value, which in turn, reduced the uncertainty in the difference between the observed value and the expected value.

For example, suppose that the observed value of a physical constant is 75 and the predicted value (or the value expected from a different data set) is 65, for a difference of 10 in the appropriate units. If the pre-JWST one standard deviation uncertainty in this difference was 5 and the post-JWST one standard deviation uncertainty in this difference is 2, but the best fit values of the observed and predicted values didn't change, a 2 sigma tension pre-JWST turns into a 5 sigma tension post-JWST, which amounts to a scientific discovery that the theory used to make the prediction is wrong by consensus standards in the discipline, if two other conditions are met (replication and a theoretical explanation for what could be causing the new observation).

Basically, the JWST is pushing some old tensions between observation and the LambdaCDM model into the territory of scientific discoveries that are contrary to the LambdaCDM model's predictions.

The real question is why LambdaCDM is still the paradigm

What has kept this "Standard Model of Cosmology" alive as a paradigm is (1) there is no real consensus on what should replace it, and (2) old physicists (and scientists more generally) tend to cling to the theories that were considered well-established when they were in their prime, notwithstanding accumulating evidence to the contrary.

The German physicist Max Planck said that science advances one funeral at a time. Or more precisely: “A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.”

Still, there are advocates for a new paradigm. See, e.g., Fabrizio Nesti, Paolo Salucci, Nicola Turini, "The Quest for the Nature of the Dark Matter: The Need of a New Paradigm" arXiv:2308.02004 (August 3, 2023 (published in 2023(2) Astronomy 90-104).

There are several tensions that have grown a lot stronger which are discussed in the article.

The Hubble Tension

The first tension discussed is the Hubble tension.

Simply put, in the LambdaCDM model, the Hubble constant should be a constant, which is related to the cosmological constant, Lambda, which is how General Relativity explains the phenomena sometimes more generally described as "dark energy."

But, new measurement are making it increasingly clear that the low value of Hubble's constant inferred from the cosmic microwave background radiation (CMB) cannot be reconciled with the roughly 10% higher value of Hubble's constant inferred from observations of objects at distances/time depths of 1-2 billion light years/years, which are much more recent than the CMB observations.

When measurements were less precise, an astronomer could reasonably expect new measurements to converge and eliminate the 10% discrepancy as they became more precise. But, that didn't happen. The tension just became more significant.

A paper ruling out one fairly easy theoretical tweak to the model to solve the Hubble tension is Sunny Vagnozzi, "Seven hints that early-time new physics alone is not sufficient to solve the Hubble tension" arXiv:2308.16628 (August 31, 2023) (accepted for publication in Universe).

Other Tensions

The second issue discussed is a layman's language description of what is sometimes called the "impossible early galaxies" problem. Simply put, the JWST is seeing (as hinted by earlier observations with less statistical significance) more galaxies at any given level of maturity at very high redshifts than LambdaCDM predicted would be seen, and more generally, earlier structure formation than predicted by LambdaCDM prior to JWST. See, e.g., Labbé, I., van Dokkum, P., Nelson, E. et al. "A population of red candidate massive galaxies ~600 Myr after the Big Bang." Nature (February 22, 2023). https://doi.org/10.1038/s41586-023-05786-2 (Open access version available at https://arxiv.org/abs/2207.12446). The Quanta article is also partially alluding to the "S8 tension" which is tough to describe succinctly at an intermediate level.

A paragraph of the Quanta article accurately states:

Other inconsistencies abound. “There are many more smaller problems elsewhere,” said Eleonora Di Valentino, a theoretical cosmologist at the University of Sheffield. “This is why it’s puzzling. Because it’s not just these big problems.”

The possible new physics resolutions of these issues towards the end of the article shouldn't be taken all that seriously. These are the "flavors of the week." The bigger problem, that I mentioned above, is that there is no consensus on how to fix these problems, just lots of ideas that are only partial solutions and haven't secured widespread acceptance.

The concern has been there for a while, for example, two years ago, a January 13, 2021 article in issue 358 of BBC Science Focus Magazine entitled "The Cracks in Cosmology: Why Our Universe Doesn't Add Up?" by Marcus Chown identified three main problems: First, he points to the gravitational lensing of subhalos in galactic clusters recently observed to be much more compact and less "puffy" than LambdaCDM would predict. Secondly, he points to a KIDS telescope observation of very large scale structure which shows it to be 8.3% smoother (i.e. less clumpy) than predicted by LambdaCDM. Third, he points to the Hubble tension.

These are just the tip of the iceberg. My running list of problems with the LambdaCDM model, which is far from comprehensive, include the following:

* The halo shapes predicted by it are usually wrong (too cuspy and not in the NFW distribution predicted by the theory). A recent example of this kind of observation can be found at Jorge Sanchez Almeida, Angel R. Plastino, Ignacio Trujillo, "Can cuspy dark matter dominated halos hold cored stellar mass distributions?" arXiv:2307.01256 (July 3, 2023) (Accepted for publication in ApJ).

* The correspondence between the distribution of ordinary matter and inferred dark matter in galaxies is too tight; truly collisionless dark matter should have less of a tight fit in its distribution to ordinary matter distributions than is observed. This is also the case in galaxy clusters.

* It doesn't explain systemic variation in the amount of apparent dark matter in elliptical galaxies, or why spiral galaxies have smaller proportions of ordinary matter than elliptical galaxies in same sized inferred dark matter halos, or why thick spiral galaxies have more inferred dark matter than thin ones.

* It doesn't explain why satellite galaxies are consistently located in a two dimensional plane relative to the core galaxy.

* Not as many satellite galaxies are observed as predicted, and it doesn't explain why the number of satellite galaxies is related to budge mass in spiral galaxies.

* The aggregate statistical distribution of galaxy types and shapes, called the "halo mass function" is inconsistent with what the LambdaCDM model predicts.

* The temperature of the universe measured by 21cm background radio signals is consistent with no dark matter and inconsistent with sufficient dark matter for LambdaCDM to work.

* It doesn't explain strong statistical evidence of an external field effect that violates the strong equivalence principle.

* Observations are inconsistent with the "Cosmological principle" that LambdaCDM predicts, which is "the notion that the spatial distribution of matter in the universe is homogeneous and isotropic when viewed on a large enough scale."

* It doesn't do a good job of explaining the rare dwarf galaxies (that are usually dark matter dominated) that seem to have no dark matter.

* The recently discovered dark galaxy called "Nube" shouldn't exist in its model and defeats any attempt to utilize baryonic feedback (an ill-understood process) to justify the discrepancies between what is observed and the model.

* It doesn't explain deficits of X-ray emissions in low surface brightness galaxies.

* It predicts too few galaxy clusters.

* It gets globular cluster formation wrong (see also here).

* There are too many galaxy clusters colliding at speeds that are too high relative to each other.

* It doesn't explain the "cosmic coincidence" problem (that the amount of ordinary matter, dark matter and dark energy are of the same order of magnitude at this moment in the history of the Universe since the Big Bang).

* There are potential unresolved systemic error problems in current dark energy measurements.

* Every measure of detecting dark matter directly has come up empty (including not just dedicated direct detection experiments but particle collider searches, searches for cosmic ray signals of dark matter annihilation, and indirect searches combined with direct searches and also here). But it requires particles and forces of types not present in the Standard Model or general relativity to fit what is observed.

(The links are just intended for the limited purpose of understanding what is being talked about, not as definitive scientific evidence of the problems.)