Direct Echo-Based Measurement of the Speed of Sound - Comments

[Dr. Courtney]

I apologize for my wrong assumption. Based on your questions it seemed like you did not understand the statistical issues involved as you did not mention any of the relevant statistical issues but only the pedagogical/scientific issues. For me, if I had decided (due to pedagogical or scientific considerations) to use the no-intercept method then I would have gone through a few of the relevant statistical issues, identified them as being immaterial for the data sets in consideration, and only then proceeded with the pedagogical/scientific justification. I mistakenly thought that the absence of any mention of the statistical issues indicated an unfamiliarity with them.

That is not the only issue, nor even the most important. By far the most important one is the possibility of bias in the slope. It does not appear to be a substantial issue for your data, so that would be the justification I would use were I trying to justify this approach.

Or in the Bayesian framework you can directly compare the probability of different models.

This would be a good statistical justification. It is not a general justification, because the general rule remains that use of the intercept is preferred. It is a justification specific to this particular experiment that the violation of the usual process does not produce the primary effect of concern: a substantial bias in the other parameter estimates.

Then you should know that your Ockham's razor argument is not strong in this case. It is at best neutral.

In the Bayesian approach this can be decided formally, and in the frequentist framework this is a no-no which leads to p-value hacking and failure to replicate results.

All considerations from the viewpoint of doing science intended for the mainstream literature. But from the viewpoint of the high school or intro college science classroom, largely irrelevant. The papers I cited make a strong case for leaving out the constant term when physical considerations indicate a reasonable physical model will go through the origin, and I think this is sufficient peer-reviewed statistics work to justify widespread use in the classroom in applicable cases. I also pointed out the classroom case of Mass vs. Volume where leaving out the constant term consistently provides more accurate estimates of the material density than including it. Been at this a while and never seen a problem when the conditions are met that are pointed out in the statistics papers I cited. You seem to be maintaining a disagreement based on your own authority without a willingness to cite peer-reviewed support for your position that the favored (or valid) approach is to include a constant term.

I don't regard the Bayesian approach as appropriate for the abilities of high school students I've tended to encounter. In contrast, computing residuals (and their variance) can be useful and instructive and is well within their capabilities once they've grown in their skills through 10 or so quantitative laboratories.

But zooming out, the statistical details of the analysis approach are all less relevant if one has taught the students the effort, means, and care to acquire accurate data in the first place for the input and output variables. It may seem to some that I am cutting corners in teaching analysis due to time and pedagogical constraints. But start with 5-10 data points with all the x and y values measured to 1% and you can yield better results with simplified analysis than you can with the same number of data points with 5% errors and the most rigorous statistical approach available. Analysis is often the turd polishing stage of introductory labs. I don't teach turd polishing.

 

[Dale]

And I've seen scientists publish papers with vertical shifts that make no sense. The probability of an effect when the cause is reduced to zero should be exactly zero.

I disagree emphatically on this. Including a vertical intercept in your regression is always valid (categorically and without reservation). In specific restricted circumstances it may be OK to coerce the intercept to 0, but it is always appropriate to not coerce it.

You may have a theory that says that in your experiment the effect is zero when the cause is zero, but if you artificially coerce that value to be zero then you are ignoring the data.

If the data has a non-zero intercept then either your theory is wrong or your experiment is wrong. Coercing it to zero makes you ignore this red flag from the data.

If your experiment is right and your theory is right then the confidence interval for the intercept will naturally and automatically include zero. Coercing it to zero prevents you from being able to use the data to confirm that aspect of your theory.

You seem to be maintaining a disagreement based on your own authority without a willingness to cite peer-reviewed support for your position that the favored (or valid) approach is to include a constant term.

I am perfectly willing to do so, but it will have to wait until tomorrow when I am back in my office.

All considerations from the viewpoint of doing science intended for the mainstream literature. But from the viewpoint of the high school or intro college science classroom, largely irrelevant.

Hmm, irrelevant? The goal of the class is to teach them how to do science, is it not?

I don't regard the Bayesian approach as appropriate for the abilities of high school students I've tended to encounter.

Agreed. Those comments are for your benefit. You seem to think about statistics in a way that would benefit from Bayesian methods.

But start with 5-10 data points with all the x and y values measured to 1% and you can yield better results with simplified analysis than you can with the same number of data points with 5% errors and the most rigorous statistical approach available.

100% agree.

It may seem to some that I am cutting corners in teaching analysis due to time and pedagogical constraints.

That is not at all how I see it. I think that you are going out of your way to teach something that is OK in this specific circumstance but is not generally a valid approach. To me, leaving the intercept in without discussing why would be the corner cutting approach (and what I would do in the interest of focusing on the science instead of the statistics).

 

[Dale]

You seem to be maintaining a disagreement based on your own authority without a willingness to cite peer-reviewed support for your position that the favored (or valid) approach is to include a constant term.

The best reference I have is:
"it is generally a safe practice not to use regression-through-the origin model and instead use the intercept regression model. If the regression line does go through the origin, b0 with the intercept model will differ from 0 only by a small sampling error, and unless the sample size is very small use of the intercept regression model has no disadvantages of any consequence. If the regression line does not go through the origin, use of the intercept regression model will avoid potentially serious difficulties resulting from forcing the regression line through the origin when this is not appropriate." (Kutner, et al. Applied Linear Statistical Models. 2005. McGraw-Hill Irwin). This I think summarizes my view on the topic completely.

Other cautionary notes include:

"Even if the response variable is theoretically zero when the predictor variable is, this does not necessarily mean that the no-intercept model is appropriate" (Gunst. Regression Analysis and its Application: A Data-Oriented Approach. 2018. Routledge)
"It is relatively easy to misuse the no intercept model" (Montgomery, et al. Introduction to Linear Regression. 2015. Wiley)
“regression through the origin will bias the results” (Lefkovitch. The study of population growth in organisms grouped by stages. 1965. Biometrics)
"in the no-intercept model the sum of the residuals is not necessarily zero" (Rawlings. Applied Regression Analysis: A Research Tool. 2001. Springer).
"Caution in the use of the model is advised" (Hahn. Fitting Regression Models with No Intercept Term. 1977. J. Qual. Tech.)

All directly echoing comments I made and issues I raised earlier.

 

[Swamp Thing]

Fun tip: You can use the sound recorder app in a smartphone to record the bang and the echo. There are some good apps that you can then use to display the waveform and measure the delay within the phone, or else you can send the files to a desktop.

I used this approach to measure the muzzle velocity of things that I fired from my homemade blowgun. I recorded the "pop" from the exiting projectile, followed by the sound of said projectile smacking through a sheet of paper pinned to a backdrop.

BTW, making a blowgun is a fun way to learn a lot of physics and of course, to teach "safety first."

 

[A.T.]

to teach "safety first."

On that note: The woman in the video, who gets a marshmallow shot into her mouth, should wear safety glasses.

 

[sophiecentaur]

I explain it to students this way: the only possible distance any signal can travel in zero time is zero distance.

You also need to point out to them that there is always a finite (and often significant) offset in measurements and that the line of measured dots will not actually pass through 0,0. The actual values near the origin have the same significance as the rest of them.

 

[sophiecentaur]

On that note: The woman in the video, who gets a marshmallow shot into her mouth, should wear safety glasses.

AND she should go for a 1km run to make proper use of the energy consumed.

 

[Dr. Courtney]

You also need to point out to them that there is always a finite (and often significant) offset in measurements and that the line of measured dots will not actually pass through 0,0. The actual values near the origin have the same significance as the rest of them.

With sufficient experimental care, the line can be made to pass through the origin. Why insist on a vertical offset for a line but not a power law? Would it make sense to add a vertical offset fitting data to Kepler's Third Law? That just adds an unnecessary adjustable parameter.

 

[Dale]

Would it make sense to add a vertical offset fitting data to Kepler's Third Law?

Yes, it would. For all of the reasons already identified above.

 

[sophiecentaur]

Would it make sense to add a vertical offset fitting data to Kepler's Third Law?

A 'DC' offset would apply to all your measurements. if you were to crowbar your curve fit to a wrong value near the origin, it would result in the parameters of your best fit curve being corrupted. Remember that all your measurements are subject to all the error sources and the curve can't know about the law that you are trying to fit them to. They are telling you there is something wrong by predicting that a low pair of co ordinates wouldn't sit at 0,0. Any 'theory' you try to apply to a set of measurements has to be consistent (within bounds) with your measurements. It would be like saying "These are Keppler's Laws but they don't apply at four o'clock on Sunday afternoon".

 

[Dale]

if you were to crowbar your curve fit to a wrong value near the origin, it would result in the parameters of your best fit curve being corrupted

Meaning that the errors in your estimate of any other parameters would no longer have zero mean but would have some non-zero bias.

 

[Dr. Courtney]

A 'DC' offset would apply to all your measurements. if you were to crowbar your curve fit to a wrong value near the origin, it would result in the parameters of your best fit curve being corrupted. Remember that all your measurements are subject to all the error sources and the curve can't know about the law that you are trying to fit them to. They are telling you there is something wrong by predicting that a low pair of co ordinates wouldn't sit at 0,0. Any 'theory' you try to apply to a set of measurements has to be consistent (within bounds) with your measurements. It would be like saying "These are Keppler's Laws but they don't apply at four o'clock on Sunday afternoon".

Interesting theory, but I was skeptical since forcing the line through the origin has tended to give better agreement with "known good" values over lots and lots of intro physics experiments I've supervised.

So I just completed a numerical experiment. The the Area of a circle (A) vs. the square of the radius is a straight line through the origin with a slope of pi. Adding some Gaussian noise with a defined standard deviation will give values of the slope that differ a bit from pi. If allowing an offset is better, the RMS error of the slope from pi will be SMALLER than the RMS error of the slope without the offset.

I used values of r from 1 to 10 in steps of 1, and standard deviations in the error added to A varying from 0.01 to 1. The RMS errors of the slopes from pi LARGER in every case allowing the offset. For example, for a standard deviation in the error added to A of 0.1, the RMS error in the slope was 0.0015 allowing the offset and 0.00082 forcing the line through the origin. Repeating the numerical experiment with Circumference vs diameter yields the same result that the RMS error of the slope from the known good (pi) is always LARGER when the offset is allowed.

Nice theory, but when the value of the output is known to be zero for zero input, forcing the line through the origin provides a more accurate value of the slope.

 

[sophiecentaur]

So, imagine your stopwatch had a 0.1s delay before it starts counting and that is the only significant error in the experiment. An excellent straight line through the points would predict a permanent 0.1s offset at t=0. You have found in your experiments, apparently, that your other points ‘predict’ a zero crossing that’s at 0,0. Bells should ring and equipment examined, IMO.
You can either believe that your argument is simply ‘true’ or you can look deeper into your limited data set and find a good flaw in what you have been doing. It is always dangerous to rely on isolated experiments if you’re trying to argue with accepted (well founded) statistical theory.

 

[Dr. Courtney]

So, imagine your stopwatch had a 0.1s delay before it starts counting and that is the only significant error in the experiment. An excellent straight line through the points would predict a permanent 0.1s offset at t=0. You have found in your experiments, apparently, that your other points ‘predict’ a zero crossing that’s at 0,0. Bells should ring and equipment examined, IMO.
You can either believe that your argument is simply ‘true’ or you can look deeper into your limited data set and find a good flaw in what you have been doing. It is always dangerous to rely on isolated experiments if you’re trying to argue with accepted (well founded) statistical theory.

If the theory has not been tested with real data sets, it should not be widely accepted.

As I said above, I've tested lots of real experimental data sets with and without an offset. If the experiment makes sense that the line goes through the origin, doing the fit without an offset most often provides better agreement with the known good value.

Your experiment is the one that is cherry picked, or at least it has little to do with most careful experiments. There was one occasion where we added an offset, but the physical justification was well understood. See: https://arxiv.org/ftp/arxiv/papers/1102/1102.1635.pdf

But to the original experiment, the measured round trip time is a time DIFFERENCE. A systematic vertical shift much larger than the random errors introduced in the measurement technique would be unexpected. Finding one would be evidence that the intended procedure was not carried out. The bad experiment should be repeated. Trying to fix it with an offset is just bad science. Can you say "fudge factor"?

 

[Dale]

So I just completed a numerical experiment. The the Area of a circle (A) vs. the square of the radius is a straight line through the origin with a slope of pi. Adding some Gaussian noise with a defined standard deviation will give values of the slope that differ a bit from pi.

Your numerical experiment doesn’t address the objection. The objection was about bias introduced into the slope when you forced a non-zero intercept to be zero.

 

[Dr. Courtney]

Your numerical experiment doesn’t address the objection. The objection was about bias introduced into the slope when you forced a non-zero intercept to be zero.

My experiment showed that the slopes are more accurate - in better agreement with the known good value.

If the experimental goal is a more accurate determination of the slope, the method works well. The purpose of the original experiment is an accurate determination of the speed of sound as the slope of distance vs time. My experiment supported that usage.

 

[sophiecentaur]

Your experiment is the one that is cherry picked,

Cherry picked, of course but if you can't argue against it then your other arguments fail.

when you forced a non-zero intercept to be zero

Forcing a zero is what you are doing - by introducing an extra point at the origin with no evidence. 0,0 has no more significance than any other arbitrary point in a data set.
If you had an experiment that, whatever you did, produced a value bang on the origin then you would have discovered some non-linear function of your system. That would be fine but in a simple system like transit times and distances you would have to conclude that something else was going on. The problem with your idea is that you need to ask yourself just how extreme the intercept would have to be before you would realize that things are not as simple as you might have hoped. We're not really discussing your particular experiments as much as a general principle about the discipline of measurement in general and you cannot just stick in numbers from out of your head and expect it to be valid. You cannot 'test' a theory by injecting numbers into your data which happen to follow that theory. That would be really bad Science.

 

[Dale]

My experiment showed that the slopes are more accurate - in better agreement with the known good value.

Sure, but your experiment did not address @sophiecentaur's "interesting theory" that you were skeptical about.

So, I did my own numerical experiment which did address the "interesting theory". I did almost the same thing that you did. I simulated 10000 data sets each with r incremented from 1 to 10 in steps of 1, I calculated $r^2$ and $A_i=\pi r^2+b + \epsilon_i$ with the "DC offset" $b=0.1$ and the random noise $\epsilon_i \sim \mathcal{N}(0,0.1)$, and I did linear fits both with an intercept and without an intercept. I then subtracted $\pi$ from the fitted slope to get the error in the slope and plotted the histograms below. The orange histogram is the error in the slope with the intercept and the blue histogram is the error in the slope without the intercept.

Note that there is substantial bias without the intercept. The no-intercept estimate is slightly more precise (as you discovered), but it is less accurate. The fit with the intercept is unbiased while the no-intercept fit is biased and the overall RMS deviation is greater for the no-intercept model.

The increased precision of the no-intercept estimate is deceptive, it does not generally correspond to increased accuracy as you suggested. Furthermore, because the intercept model is unbiased the correct slope can be obtained simply by acquiring enough data, whereas (even in the limit of infinite data) the no-intercept model will not converge to the correct estimate.

@sophiecentaur 's concern is valid, as shown in a relevant simulation, and is one of the issues recognized and discussed in the literature I cited earlier

If the experimental goal is a more accurate determination of the slope, the method works well.

The method is not more accurate in general, particularly not in the actual scenario raised by @sophiecentaur

 

[sophiecentaur]

@Dale it doesn’t surprise me that injecting a data point that follows an accepted law will bias the measured data in that direction. But if there is a set of measurements of a system that, for good reasons, will not follow the law then forcing a certain data point can fool the experimenter that there is no anomalous behavior. I can’t see the point in that.
You might miss out on a new branch of research by trying to prove that present ideas are correct. It’s the anomalies that reveal new knowledge.
Pluto would not have been spotted if measurements of Neptune’s orbit had been frigged to follow Kepler perfectly.

 

[Dr. Courtney]

@Dale it doesn’t surprise me that injecting a data point that follows an accepted law will bias the measured data in that direction.

In the context of the original experiment, there is no assumption or injection of a data point assuming the original law. The only assumption is that for a time interval of ZERO, the distance sound travels can only be ZERO. So, Einstein is implicitly assumed - a signal cannot propagate faster than light. But there is no assumption that the relationship between distance and time is linear or that the slope of the line will be a certain value if it is linear.

 

[sophiecentaur]

In the context of the original experiment, there is no assumption or injection of a data point assuming the original law. The only assumption is that for a time interval of ZERO, the distance sound travels can only be ZERO. So, Einstein is implicitly assumed - a signal cannot propagate faster than light. But there is no assumption that the relationship between distance and time is linear or that the slope of the line will be a certain value if it is linear.

You are confusing the Law with the measurement system you have been using. No one doubts that a perfect experiment would give a perfect zero crossing. Would you dream of injecting a theoretical data point elsewhere?
The consequence of your method would be to upset the whole of Science.
Your method, whether numerical or experimental, needs examination to identify any bias before you
Start changing the textbooks. Where would you stop?

 

[sophiecentaur]

@Dr. Courtney Let's be practical here. Where and how were your distances measured? How big was your sound source (I mean treating it as a real wave source with a real extent)? What was the minimum distance that you reckon you could measure? Diffraction effects will be present around the source and detector to upset your zero distance measurement.
Many people post on PF and imply that they have found holes in accepted science. This seems to be just another example. Rule number one is to doubt yourself before you doubt Science. Only after extended work on a topic can anyone be sure that a change of model is justified. Read @Dale 's detailed post (carefully) to see the effect on accuracy when you do your sleight of hand trick.

 

[Dr. Courtney]

Many people post on PF and imply that they have found holes in accepted science. This seems to be just another example. Rule number one is to doubt yourself before you doubt Science. Only after extended work on a topic can anyone be sure that a change of model is justified.

And yet that is what you suggest for fits to power laws such as Kepler's third law. I have seen many published papers fitting data to power laws, including Kepler's third law. I don't recall any of them adding a third parameter (vertical shift) to their model. Yet you are insisting that a change of model is justified. Change of models is only justified from more accurate results, not theoretical arguments like you and Dale are making.

Consider the graph below showing an analysis of Robert Boyle's original data published in support of Boyle's law. Fitting this data to a traditional power law yields better agreement with the "known good" value for the exponent. Adding a third adjustable parameter (the vertical shift) gives a higher r-squared and a lower Chi-square, but it gives a less accurate value for the exponent in both the condensation and rarefaction cases. I prefer the method that gives an error of 0.004 over the method with an error of 0.089 (Rarefaction case). I prefer the method that gives an error of 0.002 over the method that gives an error of 0.01 (Condensation case).

 

[Dale]

Change of models is only justified from more accurate results,

You are confusing precision and accuracy. Removing the intercept is more precise, but it is not generally more accurate because it can introduce bias when there is an offset. Furthermore, an imprecise but unbiased method is preferable over a precise but biased method because imprecision can be overcome simply by acquiring more data while bias cannot.

 

[Dr. Courtney]

Also consider the inaccuracies in the parameters introduced by adding a vertical shift to the power law as applied to Kepler's Third Law. Adding the vertical shift yields less accurate determinations of both the lead coefficient and the exponent for both the case of Kepler's original data and the modern data. I don't care about theoretical arguments about whether a vertical shift will yield more accurate (or unbiased) parameter values. Forcing the power law model through the origin actually does provide more accurate parameter values.

 

[Dale]

Fitting this data to a traditional power law yields better agreement with the "known good" value for the exponent. Adding a third adjustable parameter (the vertical shift) gives a higher r-squared and a lower Chi-square, but it gives a less accurate value for the exponent in both the condensation and rarefaction cases.

By the way, you seem to have a misunderstanding here. You didn’t describe what you were doing in detail, but there should be no third term. In the model $y=bx^m$ the log of the parameter $b$ is the intercept. This type of model is fit by log-transforming “under the hood” so you actually fit $\ln (y) = m \ln (x) + \ln (b)$. Thus a no-intercept fit would be one that coerces $\ln (b)=0 \implies b=1$. That coercion would lead to bias in the estimate of $m$ if $b\ne 1$

 

[Dr. Courtney]

By the way, you seem to have a misunderstanding here. You didn’t describe what you were doing in detail, but there should be no third term. In the model y=bxmy=bxm the log of the parameter bb is the intercept. This type of model is fit by log-transforming “under the hood” so you actually fit ln(y)=mln(x)+ln(b)ln⁡(y)=mln⁡(x)+ln⁡(b). Thus a no-intercept fit would be one that coerces ln(b)=0⟹b=1ln⁡(b)=0⟹b=1. That coercion would lead to bias in the estimate of mm if b≠1b≠1

No, I used a non-linear least squares fit directly to the model without the log transformation. In that approach, the vertical offset is a third parameter. In the log transformation approach, the intercept is necessary, because it is the second parameter needed and has a physical meaning relating to the data. In direct application of non-linear least squares, the vertical shift is not necessary or helpful, since it is the third parameter, has no physical meaning, and makes the results less accurate. I can't see how you thought I used the obsolete log transform approach, since it is not possible to do that with the three parameter model (true power law plus vertical shift.)

 

[Dale]

I used a non-linear least squares fit directly to the model without the log transformation.

Ah. All of my comments above are specifically about linear least squares fits. Non-linear fits are more complicated and need much more care with regard to the errors. Biased estimates are pervasive for non linear fits.

In direct application of non-linear least squares, the vertical shift is not necessary or helpful, since it is the third parameter, has no physical meaning, and makes the results less accurate.

No objection here.

 

[Dr. Courtney]

Ah. All of my comments above are specifically about linear least squares fits.

To clarify, do you mean linear models or linear least squares fits? One can use Levenberg-Marqhardt (non-linear least squares method) on linear models. I don't think how one obtains the best-fit parameters matters as long as Chi-square in minimized and R-squared is closest to one in the fit determined to be "best." In any case, let's return to linear models.

The graph below shows linear best fits of Mass vs. Volume for distilled water. The slope of the best fit line should give the density of water. I feel using a line with a vertical offset set to zero is warranted, since I know the mass of water with zero volume is zero. "Forcing" the line through the origin yields a density of water of 0.9967 g/mL with an uncertainty of 0.0014 g/mL estimated from the algorithm in the spreadsheet LINEST command. This agrees reasonably with the known good density of distilled water at 70 deg F, which is 0.99802. Not bad for a student grade graduated cylinder and electronic balance with 0.1 g resolution. The experiment and analysis produced an accuracy better than 0.2%.

What happens when a vertical shift is allowed? The analysis suggests the density of water is 1.0045 g/mL with an estimated uncertainty of 0.001 g/mL. Even though the accuracy estimate is 0.1%, the actual error is over 0.6%.

 

[sophiecentaur]

since I know the mass of water with zero volume is zero.

This is where my problem lies. Of course we know that no volume will have zero mass but how can you be sure that your volume and mass measurements have no offset and that all measured points on your graph - except your artificial one - could be subject to the same offset. If the offset were greater than the other random factors and if you took enough readings (@Dale has already made this point) then the offset would reveal itself in the position of the intercept of a good straight line. Are you saying that you should just ignore this actual intercept value and not put the final dot on this line? It could represent an experimentally significant piece of information. Where is the 'reality' in this discussion?

I made the point, earlier, that there will always be an offset in time / distance measurements when the source and receiver are very close together because the positions are undefinable (there are no point sources or point receivers). You seem to suggest that there is a 'real' answer to the final value of velocity from your experiment but there will always be some uncertainty.

Introducing Power Laws and Log scales is merely clouding the issue. How is finding and demonstrating that there is an underlying linear relationship between two measured variables helped by forcing any single data point into the process which would, in fact suggest that the relationship is not linear?

 

[Dr. Courtney]

I've provided case after case of fitting where fits without an offset provide better agreement with known good parameter values.

You repeat the same tired theoretical considerations in the face of additional demonstrations that leaving the offset out produces more accurate parameter values. I have no need to answer theoretical objections when case after of case of experimental data supports leaving the offset out.

As Feynman said, "The easiest person to fool is yourself." But Feynman also noted the more important principle in the scientific method is that experiment is the ultimate arbiter. I have additional experimental data showing that leaving the offset out gives better agreement with known good parameter values. Would you like to see it?

 

[sophiecentaur]

I have additional experimental data showing that leaving the offset out gives better agreement with known good parameter values.

Could you tell me how you manage to measure the distance when it is small? How do you define the positions of source and detector (relative to the reflector) and how big are the two terminals?

 

[sophiecentaur]

But Feynman also noted the more important principle in the scientific method is that experiment is the ultimate arbiter.

Doesn't that imply that the experimental result for small distances is the arbiter? But Ad Hominem arguments are not really relevant here. We all know Richard was a smart guy but that doesn't mean his particular statements apply to your particular example.

 

[Vanadium 50]

I hope this adds more light than heat.

First, there is a theorem, I forget by whom, that states that in general the estimator with the smallest variance is not unbiased. You can sort of see why this might be the case: if you want the smallest σ/μ, if μ is biased high, σ/μ will be biased low.

Next, in this situation we are fitting models. x = vt is a model. So is x = vt + x₀. So is x = at² vt + x₀. Statistics can't tell you what model is right or wrong. It can only tell you what fits well and what doesn't.

It has been suggested to add extra terms and if they come out zero, they don't hurt anything. One (of several) problem with this is "where do you stop?" You can always add terms and the best fit will always be when the number of parameters equals the number of data points. These "fits" tend to be unphysical and wiggly.

Resoultion in x has been cited as a reason not to use the x = vt model. However, if you're going down that path, you also need to consider resolution in t. This opens up all sorts of cans of worms, previously discussed here.

 

[Swamp Thing]

Next, in this situation we are fitting models. x = vt is a model. So is x = vt + x0. So is $x = at^2 vt + x0$. ... One (of several) problem with this is "where do you stop?"

Perhaps we should think of the model as representing the physical phenomenon as well as the error mechanisms. For example, in the Insight article (distance measurement), if the air temperature during the experiment can vary as we move away from the reflecting wall, then $x = a_0vt+a_1t^2 vt + x0$ might not be such a ridiculous idea. On the other hand, if we are sure that the speed of sound is independent of distance from the wall, then our model should not include a square term. Based on this pholosophy, we should probably attribute the intercept $x_0$ to a measurement bias.

Once we are clear about which terms belong to the phenomenon model, and which ones belong to the error model, then we can subtract out the terms that we're sure come from the error model. Me, I would bet on subtracting out the $x0$.

Or, if we want to add some finesse, we can increase the weight of measurements in proportion to the distance from the wall, since we can be less confident about a measurement that is too close to the wall. (if we do 20 measurements at a place 2 meters from the wall and 20 measurements somewhere 50 meters from the wall, will the first set have a larger standard deviation?)

In some situations, the error model and the phenomenon model can both contribute to a particular term. In this case, a probabilistic approach can be used to partition the term between the error model and the phenomenon model. IIRC, something like this is done in Kalman filters, for example.