- #1
ohwilleke
Gold Member
- 2,440
- 1,429
One way too look for previously unknown relationships in data is to sift through the data looking for relationships between datum with no known theoretical connection and to see if you can find a formula that can link them, without any real solid theoretical motivation for doing so. Then, having established the consistency of your formula with the data, you try to imagine a theoretical explanation for that relationship that would make sense.
If you have a large data set, you call this a phenomenological formula. If you have only a few data points, this approach is often, disparagingly called "numerology" because knowledgeable physicists in this field know much better than the average person that it is easier to devise formulas that fit small data sets (even to surprising precision) than one would naively expect. Also, given the fact that even formulas known to be perfectly correct descriptions of reality don't always exactly predict the data that is collected due to measurement errors, the fact that a "numerological" formula doesn't exactly predict the data isn't fatal to the formula so long as it is consistent with the data to within the range of measurement error.
The bigger the measurement error is, the bigger the space of possible formulas that can fit the data points will be. Conversely, the smaller the measurement error is, the more likely it is that formulas fit to the data will not be false positives.
The trouble is that there is no really reliable way to quantify this directly. You can't just look at a small set of data points and say, "there are 500 formulas" or "there are 12 formulas", for example, that can be fit to these data points with this margin of error.
Still, at some point, if the margin of error in the measurements (and the calculation) gets small enough, a formula that can fit a small number of data points starts to look more like something that could actually be true physical laws that are the true explanation of the data, and less like coincidental numerology.
The Example Of Muon g-2
For example, consider the comparison of the muon g-2 theoretical prediction and its experimentally measured value.
The most precise experimentally determined value of this observable property of the muon, announced in January of 2004, in units of
10-11 and combining the errors in quadrature was:
E821 116 592 091 ± 63
The current state of the art theoretical prediction from the Standard Model of particle physics regarding the value of muon g-2 in units of 10-11 is:
QED 116 584 718.95 ± 0.08
HVP 6 850.6 ± 43
HLbL 105 ± 26
EW 153.6 ± 1.0
Total SM 116 591 828 ± 49
In the same units, the experimental result from 2004 exceeds the theoretical prediction by:
Discrepancy 263
This is a 3.3 sigma discrepancy, which is notable in physics, but not considered definitive proof of beyond the Standard Model physics either.
(The exact theoretical determination is a moving target.)
Even if the theoretical prediction were based just on a formula picked out of the air, rather than having a basis in the Standard Model, you would be inclined to think that a formula that matches the experimental result to 7 significant digits, even if it isn't consistent with the experimental result at the 2 sigma level, probably has some close connection with reality, even though it isn't consistent with the experimental result at the 2 sigma level commonly used to determine if results are consistent or not.
On the other hand, if the new experimental measurement that is in the works also finds a 3.3 sigma or greater discrepancy or more between the theoretical prediction and the experimental measurement, then there is either something wrong with the theoretical calculation (in the muon g-2 context, most likely with the hadronic component of the calculation that is associated with about 98% of the uncertainty in the theoretical prediction), or something wrong with the estimated margin of error in the experimental measurement (for example, if the true experimental margin of error was 126 rather than 63, underestimated by a factor of two, the results with be consistent at the better than 2 sigma level and nobody would be concerned that they might actually be discrepant due to new physics; but experimental margins of error are much easier to underestimate by overlooking one or more possible sources of error than they are to overestimate by quantifying a known source of error in a grossly inaccurate way).
Bottom Line
The point is that at some point, if your formula is close to predicting the measured data and the data is measured to high precision, what would otherwise be called numerology ought to be taken seriously and should be pretty close to the mark, even if it may need to be tweaked slightly to reproduce the true formula that describes reality.
But, how does one draw the line between a proposed formula that should be taken seriously and one that should be dismissed as numerology?
Also, while I've looked at the precision of the result compared to the precision of the prediction, and at the consistency of the result and the prediction in terms of statistical significance, there are other factors one could look at as well.
* Have new results gotten closer to or further from the prediction as they have grown more accurate over time?
For example, one of the attractions of Koide's rule for charged lepton masses which predicted the tau lepton mass in 1981 based upon the electron and muon masses (and is still consistent at less than 1 sigma today) is that the discrepancy between the prediction and the measured result has shrunk between 1981 when the rule was suggested and the present.
* Are the inputs into the formula things that could plausibly have a relationship to each other?
For example, Koide's rule for charged leptons seeks to relate the masses of three particles that are identical in all respects except for mass and lepton family, so it makes sense that these quantities might have some sort of deeper relationship to each other, even though there isn't a real obvious theoretical basis to explain why they are related in this way that can be demonstrated with other evidence. In contrast, if I came up with a formula for muon g-2 that had the pass completion rate of the Denver Bronco's as an input, I wouldn't trust it no matter how good the fit was.
* Are the terms in the formula the kind of terms that are unexceptional in other physical laws in this field?
If your formula has terms with πe in it, I'm going to be more suspicious than if it has more ordinary terms in it. But, an infinite series with powers of a coupling constant in it might look pretty attractive because formulas in that form are so common in the Standard Model.
* If errors are more prone to be in one direction than the other, is that the direction in which errors are observed?
For example, if the measured quantity is a sum of observed events and it is easy to have a false negative and miss a signal of an event (like a neutrino hitting an atomic nucleus), but it is hard to have a false positive result, a measured value that is lower than the predicted value will be less troubling than a measured value that is higher than the predicted value.
I imagine that there could also be other criteria.
Implication
The better we can define the boundary between numerology and a phenomenological formula that should be taken seriously, the more disciplined we can be in devoting our speculations to circumstances where a formula we find could be established to be meaningful.
For example, looking for numerological relationships between quantities measured to 5 significant digits each is a lot more likely to produce a formula that should be taken seriously than looking for a numerological relationship between quantities measured to only 2 significant digits, which can be spoofed with spurious and coincidental numerical similarities much more easily.
If you have a large data set, you call this a phenomenological formula. If you have only a few data points, this approach is often, disparagingly called "numerology" because knowledgeable physicists in this field know much better than the average person that it is easier to devise formulas that fit small data sets (even to surprising precision) than one would naively expect. Also, given the fact that even formulas known to be perfectly correct descriptions of reality don't always exactly predict the data that is collected due to measurement errors, the fact that a "numerological" formula doesn't exactly predict the data isn't fatal to the formula so long as it is consistent with the data to within the range of measurement error.
The bigger the measurement error is, the bigger the space of possible formulas that can fit the data points will be. Conversely, the smaller the measurement error is, the more likely it is that formulas fit to the data will not be false positives.
The trouble is that there is no really reliable way to quantify this directly. You can't just look at a small set of data points and say, "there are 500 formulas" or "there are 12 formulas", for example, that can be fit to these data points with this margin of error.
Still, at some point, if the margin of error in the measurements (and the calculation) gets small enough, a formula that can fit a small number of data points starts to look more like something that could actually be true physical laws that are the true explanation of the data, and less like coincidental numerology.
The Example Of Muon g-2
For example, consider the comparison of the muon g-2 theoretical prediction and its experimentally measured value.
The most precise experimentally determined value of this observable property of the muon, announced in January of 2004, in units of
10-11 and combining the errors in quadrature was:
E821 116 592 091 ± 63
The current state of the art theoretical prediction from the Standard Model of particle physics regarding the value of muon g-2 in units of 10-11 is:
QED 116 584 718.95 ± 0.08
HVP 6 850.6 ± 43
HLbL 105 ± 26
EW 153.6 ± 1.0
Total SM 116 591 828 ± 49
In the same units, the experimental result from 2004 exceeds the theoretical prediction by:
Discrepancy 263
This is a 3.3 sigma discrepancy, which is notable in physics, but not considered definitive proof of beyond the Standard Model physics either.
(The exact theoretical determination is a moving target.)
Even if the theoretical prediction were based just on a formula picked out of the air, rather than having a basis in the Standard Model, you would be inclined to think that a formula that matches the experimental result to 7 significant digits, even if it isn't consistent with the experimental result at the 2 sigma level, probably has some close connection with reality, even though it isn't consistent with the experimental result at the 2 sigma level commonly used to determine if results are consistent or not.
On the other hand, if the new experimental measurement that is in the works also finds a 3.3 sigma or greater discrepancy or more between the theoretical prediction and the experimental measurement, then there is either something wrong with the theoretical calculation (in the muon g-2 context, most likely with the hadronic component of the calculation that is associated with about 98% of the uncertainty in the theoretical prediction), or something wrong with the estimated margin of error in the experimental measurement (for example, if the true experimental margin of error was 126 rather than 63, underestimated by a factor of two, the results with be consistent at the better than 2 sigma level and nobody would be concerned that they might actually be discrepant due to new physics; but experimental margins of error are much easier to underestimate by overlooking one or more possible sources of error than they are to overestimate by quantifying a known source of error in a grossly inaccurate way).
Bottom Line
The point is that at some point, if your formula is close to predicting the measured data and the data is measured to high precision, what would otherwise be called numerology ought to be taken seriously and should be pretty close to the mark, even if it may need to be tweaked slightly to reproduce the true formula that describes reality.
But, how does one draw the line between a proposed formula that should be taken seriously and one that should be dismissed as numerology?
Also, while I've looked at the precision of the result compared to the precision of the prediction, and at the consistency of the result and the prediction in terms of statistical significance, there are other factors one could look at as well.
* Have new results gotten closer to or further from the prediction as they have grown more accurate over time?
For example, one of the attractions of Koide's rule for charged lepton masses which predicted the tau lepton mass in 1981 based upon the electron and muon masses (and is still consistent at less than 1 sigma today) is that the discrepancy between the prediction and the measured result has shrunk between 1981 when the rule was suggested and the present.
* Are the inputs into the formula things that could plausibly have a relationship to each other?
For example, Koide's rule for charged leptons seeks to relate the masses of three particles that are identical in all respects except for mass and lepton family, so it makes sense that these quantities might have some sort of deeper relationship to each other, even though there isn't a real obvious theoretical basis to explain why they are related in this way that can be demonstrated with other evidence. In contrast, if I came up with a formula for muon g-2 that had the pass completion rate of the Denver Bronco's as an input, I wouldn't trust it no matter how good the fit was.
* Are the terms in the formula the kind of terms that are unexceptional in other physical laws in this field?
If your formula has terms with πe in it, I'm going to be more suspicious than if it has more ordinary terms in it. But, an infinite series with powers of a coupling constant in it might look pretty attractive because formulas in that form are so common in the Standard Model.
* If errors are more prone to be in one direction than the other, is that the direction in which errors are observed?
For example, if the measured quantity is a sum of observed events and it is easy to have a false negative and miss a signal of an event (like a neutrino hitting an atomic nucleus), but it is hard to have a false positive result, a measured value that is lower than the predicted value will be less troubling than a measured value that is higher than the predicted value.
I imagine that there could also be other criteria.
Implication
The better we can define the boundary between numerology and a phenomenological formula that should be taken seriously, the more disciplined we can be in devoting our speculations to circumstances where a formula we find could be established to be meaningful.
For example, looking for numerological relationships between quantities measured to 5 significant digits each is a lot more likely to produce a formula that should be taken seriously than looking for a numerological relationship between quantities measured to only 2 significant digits, which can be spoofed with spurious and coincidental numerical similarities much more easily.
Last edited: