Puzzled about the non-teaching of Monte Carlo method(s) for error analysis

[fluidistic]

Say you have measured several quantities and you can calculate the value of a physical quantity as a sum/product of the several measured quantities, and you are interested to calculate the uncertainty of that physical quantity.

I have just learned about the Monte Carlo method applied to error (or uncertainty) analysis.
To me, it is much more intuitive than the formula giving the standard deviation of the mean as a square root of the sum squared of the partial derivatives of the expression related to the variable whose standard deviation is looked after, multiplied by the standard deviation of each variable, squared. This formula has severe limitations, such as it assumes that the variables are uncorrelated, not so big, linear, etc. Furthermore, as far as I have read, Monte Carlo is more accurate if well applied.

The "inputs" of the Monte carlo are the distributions related to each variables. For example, if we assume (educated guess) that a Gaussian represents well the values that a voltmeter shows, with the mean being the value displayed and the standard deviation between the last digit displayed, we "try out" plugging values satisfying this condition, in the relationship between the physical quantity of interest and its associated measured quantities. We do the same for all quantities, and we thus obtain a distribution of values for the one of interest. Then it is a matter of reporting the median, 32nd and 68th percentiles, I believe.

This technique is apparently very strong and can overcome a lot of problems the usually taught formula in undergraduate physics/engineering meets in real life problems.
To me, it is one step further away from plugging the minimum and the maximum value a variable can take, and see the obtained value of the quantity of interest (which is not that good of a way to get the uncertainty, but better than nothing), in terms of difficulty of understanding.

How is this Monte Carlo method not even mentioned in the usual curriculum in science? I am totally puzzled. (I hope the premise of my question isn't flawed).

 

[hutchphd]

To me, it is much more intuitive than the formula giving the standard deviation of the mean as a square root of the sum squared of the partial derivatives of the expression related to the variable whose standard deviation is looked after, multiplied by the standard deviation of each variable, squared.

As you rightly point out, intuitive (and useful) is in the eye of the beholder.

There is nothing wrong with Monte Carlo methods but in the end, all you get is a number, or a set of numbers.
If in fact you have multiple sources of error that are not independent it is not sufficient to look at only the mean and one SD numbers for each source but rather the mesh needs to fill the parameter space using appropriate density of points and skill at randomizing.. And then you gat a set of numbers that are only as good as the process you have used, and tell you nothing directly about the sources of error.

My experiences where I have been looking seriously at sources of error have usually been aimed at eliminating the volubility of measurement from, say, a medical diagnostic system. In this circumstance the Monte Carlo approach is far less useful than trying to orthogonalize the inputs of error and look at them analytically using the old math. In fact my most lucrative and perhaps satisfying contracts involved rescuing production engineers who knew only Monte Carlo methods. But they are indeed easy to understand.

The answer is of course that you need to know both methods. For complicated systems, Monte Carlo usually involves an app that is purchased... how hard is that to do?? However the answer is often found in a more nuanced analytical approach and that is why you are advised to learn it and understand it.

 

[mpresic3]

How is this Monte Carlo method not even mentioned in the usual curriculum in science? I am totally puzzled. (I hope the premise of my question isn't flawed).

In the physics curricula I am familiar with, the student learns mechanics, thermodynamics, electrodynamics, statistical mechanics, quantum mechanics, possibly optics, and electives. The student takes similar courses in mathematics, and engineering. I meet many graduates everyday. These graduates vary in their readiness to do the job, or conduct research. It is easy to find "skills and techniques" that the schools have not taught them, but look at all the important topics, the schools did teach. The curricula is already very full. What would you leave out?

Sometimes Monte-Carlo is addressed in computer classes, where large data sets are common.

I have used Monte-Carlo techniques after graduation, and I think it is OK to learn something after graduating with the Bachelor's degree, or teaching pre-baccalaureate interns in summer occupations.

 

[fluidistic]

In the physics curricula I am familiar with, the student learns mechanics, thermodynamics, electrodynamics, statistical mechanics, quantum mechanics, possibly optics, and electives. The student takes similar courses in mathematics, and engineering. I meet many graduates everyday. These graduates vary in their readiness to do the job, or conduct research. It is easy to find "skills and techniques" that the schools have not taught them, but look at all the important topics, the schools did teach. The curricula is already very full. What would you leave out?

Sometimes Monte-Carlo is addressed in computer classes, where large data sets are common.

I have used Monte-Carlo techniques after graduation, and I think it is OK to learn something after graduating with the Bachelor's degree, or teaching pre-baccalaureate interns in summer occupations.

In this case, simply replace the analytical formula teaching with the Monte Carlo method.

 

[mpresic3]

In this case, simply replace the analytical formula teaching with the Monte Carlo method.

This quote puzzles me. In what case? In all cases? If so, are you suggesting physics teaching should no longer treat physical systems with equations, but should in some manner, treat the system running a Monte Carlo to cover every conceivable calculation. I presume a knowledge of the analytical equation would guide us in setting up the Monte Carlo.
I think a linearized error analysis treatment, although it has limitations, is important for all scientists, and engineers to learn. One advantage in linear error analysis is that it can be applied to many areas, in many fields. Most theories in undergraduate physics are linear, for the most part. I have taken two to three courses that treat non-linear estimation and control, and this is useful, however, it was important to take courses in linear estimation and control as a prerequisite. Shouldn't Monte Carlo techniques come later?

 

[DaveE]

I think a key point here, as @hutchphd said, is to distinguish getting an answer versus understanding the system. In the engineering world "what" isn't any more important than "why" or "how". This is why, in my experience, computer simulations are 95% verification and 5% results. An answer that you don't understand, and therefore trust, is of questionable value. Likewise, a system model has value even when it gives bad results because you may then be informed about how to modify the model.

In education, I think it is more important to teach concepts than tools, you can learn tools later. For an extreme example, why teach how to solve integrals when Wolfram can do it for you?

I'm also not sure of your premise that Monte Carlo isn't taught, but you may have to search for it a bit.

 

[Vanadium 50]

In this case, simply replace the analytical formula teaching with the Monte Carlo method.

This strikes me as a bad idea.

(1) There are cases - quite a few - where the analytic method is perfectly fine. Simple cases should require a pen and paper, not a computer.

(2) It is important to understand the output of the Monte Carlo, oerhaps by understanding limiting cases that can be solved analytically. This is particularly important since there are non-intuitive aspects to statistics, fitting and uncertainty quantification. (e.g. the Punzi effect and the better-than-zero effect)

(3) If this were a good idea, why stop here? Why ever do an analytic calculation? Just load it into a computer and run a simulation.

 

[fluidistic]

This quote puzzles me. In what case? In all cases?

In all cases where you would apply the standard analytical formula.

If so, are you suggesting physics teaching should no longer treat physical systems with equations, but should in some manner, treat the system running a Monte Carlo to cover every conceivable calculation. I presume a knowledge of the analytical equation would guide us in setting up the Monte Carlo.

I do not understand why the first sentence follows from my thoughts.

I think a linearized error analysis treatment, although it has limitations, is important for all scientists, and engineers to learn. One advantage in linear error analysis is that it can be applied to many areas, in many fields. Most theories in undergraduate physics are linear, for the most part. I have taken two to three courses that treat non-linear estimation and control, and this is useful, however, it was important to take courses in linear estimation and control as a prerequisite. Shouldn't Monte Carlo techniques come later?

I do not disagree with your first sentence, but you are the one who mentioned the problem of too little time to cover everything. In this case, since Monte Carlo is more intuitive (subjective, I know, as been pointed out already) and very easy to implement, it wins the 1st place, ahead of the standard analytical formula which has limited scope and still requires a computer anyway (or a pocket calculator).

I think a key point here, as @hutchphd said, is to distinguish getting an answer versus understanding the system. In the engineering world "what" isn't any more important than "why" or "how". This is why, in my experience, computer simulations are 95% verification and 5% results. An answer that you don't understand, and therefore trust, is of questionable value. Likewise, a system model has value even when it gives bad results because you may then be informed about how to modify the model.
In education, I think it is more important to teach concepts than tools, you can learn tools later. For an extreme example, why teach how to solve integrals when Wolfram can do it for you?

Sure, but here again, Monte Carlo wins hands down compared to the analytical formula (in terms of understanding what is going on, what is happening, etc.). You need a computer for both anyways, unless you have learned and remember how to take the square root of real numbers, by hand. (I was not taught any algorithm to do so in my whole life, and I suspect this is the case for most people).

I'm also not sure of your premise that Monte Carlo isn't taught, but you may have to search for it a bit.

I meant for uncertainty analysis. But yes, I should search more to see if this is included as a standard in undergraduate classes.

This strikes me as a bad idea.

(1) There are cases - quite a few - where the analytic method is perfectly fine. Simple cases should require a pen and paper, not a computer.

In practice most students will use a computer, or a pocket calculator. The formula $\sigma_f = \sqrt{\left( \frac{\partial f}{\partial x} \right)^2 \sigma_x^2 + \left( \frac{\partial f}{\partial y} \right)^2 \sigma_y^2 + ...}$. Where the analytical formula shines would be where the numbers turn out to yield a number whose square root is well known, this does not happen frequently, to say the least, even in undergraduate classes.

(2) It is important to understand the output of the Monte Carlo, oerhaps by understanding limiting cases that can be solved analytically. This is particularly important since there are non-intuitive aspects to statistics, fitting and uncertainty quantification. (e.g. the Punzi effect and the better-than-zero effect)

Yes, I think you are right. By the way, what is the "better-than-zero" effect?

(3) If this were a good idea, why stop here? Why ever do an analytic calculation? Just load it into a computer and run a simulation.

You stop here because in this particular case, a very simple and intuitive algorithm replaces an analytical formula that requires a numerical output from a computer. When you don't need a numerical output, no need to push brainlessly further to apply Monte Carlo at all costs like a fanboy.

 

[hutchphd]

Where the analytical formula shines would be where the numbers turn out to yield a number whose square root is well known, this does not happen frequently, to say the least, even in undergraduate classes.

No this is not where the analytical formula "shines".

It useful because it allows you analyze the constituent sources of error without ever plugging in actual numbers. Hence the term analytical formula.

But badly educated engineers made me employable at a handsome rate, so I guess I shouldn't complain. Unless I desire quality results.

 

[fluidistic]

No this is not where the analytical formula "shines".

It useful because it allows you analyze the constituent sources of error without ever plugging in actual numbers. Hence the term analytical formula.

But badly educated engineers made me employable at a handsome rate, so I guess I shouldn't complain. Unless I desire quality results.

Let's take Wikipedia's Ohm's law example: $\sigma_R \approx R \sqrt{\left(\frac{\sigma_V}{V}\right)^2 + \left( \frac{\sigma_I}{I} \right)^2}$. What is the useful information you get without plugging numbers? Is it that you expect $\sigma_R$ to grow as the relative uncertainty in the variables involved in the relation between R and those variables grows? Or... something else?

 

[Vanadium 50]

what is the "better-than-zero" effect?

You can have a case where a systematic uncertainty that is non-zero can give a tighter limit than if it were zero. More specifically, in a Poisson counting experiment with a single source of events whose average event count is known exactly, the upper limit on when no events are observed is traditionally quoted to be 3 events at 95% CL. The “Better Than Zero” effect is the observation that the upper limit is seen to go below 3 with some methods when zero events are observed and systematic uncertainties are introduced.

 

[hutchphd]

Let's take Wikipedia's Ohm's law example:

reference please

 

[fluidistic]

reference please

Give me the time to edit it to my liking first.
Kidding! Of course, sorry for not providing it. Here it goes https://en.m.wikipedia.org/wiki/Propagation_of_uncertainty, go to the bottom at Resistance measurement.

 

[hutchphd]

Thanks. This provides an excellent example.

Suppose I wish to design an experiment to optimally measure this resistor.

There are random reading errors and my ammeter has a percent deviation of 10% versus 2% for my voltmeter. I can minimize these by repeated measurements, but I wish to utilize my time wisely: maybe ~100 measurements total.

I know immediately what the formula tells me to do.

How does one proceed in the world of Monte Carlo??.

 

[fluidistic]

Thanks. This provides an excellent example.

Suppose I wish to design an experiment to optimally measure this resistor.

There are random reading errors and my ammeter has a percent deviation of 10% versus 2% for my voltmeter. I can minimize these by repeated measurements, but I wish to utilize my time wisely: maybe ~100 measurements total.

I know immediately what the formula tells me to do.

How does one proceed in the world of Monte Carlo??.

I think I am slow and I am missing something. In your case, as I understand it, the voltmeter has a 2% deviation regardless of the current used (thus voltage read)? Similarly for the ammeter, except that this is 10%. Then the formula tells me that $\sigma_R$ is fixed, regardless of the current used. Surely I am missing something, or I haven't understood your case well?

 

[hutchphd]

Something else the Monte Carlo gives no direct insight about.
The standard error of the mean of N measurements equals $\frac \sigma {\sqrt N}$ so you can tune error analytically to optimize the number of repeat measurements for each.
This would reguire a lot of Monte Carlo. Physicists use analytical methods because they have power, not because they are the simplest or most transparent.

 

[fluidistic]

I am trying to convince myself that the analytical formula can give a deeper insight than Monte Carlo, but I did not understand your post about the resistor example. Could you clarify it please?

Something else the Monte Carlo gives no direct insight about.
The standard error of the mean of N measurements equals $\frac \sigma {\sqrt N}$ so you can tune error analytically to optimize the number of repeat measurements for each.
This would reguire a lot of Monte Carlo. Physicists use analytical methods because they have power, not because they are the simplest or most transparent.

I do not really understand again. For a given set of measurement(s) yielding the value of interest, Monte Carlo provides you sigma, right? In what way does this differ from the analytical formula yielding sigma? Only the computational cost?

 

[hutchphd]

I am attempting to design the experiment, knowing the single measurement precision of the ammeter and voltmeter. By taking multiple measurements I can produce different errors in each mean. Using the analytic formula I can figure out how to optimize the experimental procedure for a given expenditure of effort.
Such optimizations are often required in the real would with many input error sources. Repeatedly plugging guesses into a Monte Carlo simulation is not a good way to proceed.
The purpose of analytic forms is to allow your mind to symbolically manipulate complicated problems for systems too entangled to otherwise be amenable to such analysis. Asking a computer app for a number is not at all the same.

.

 

[fluidistic]

Dear hutchphd,

I am attempting to design the experiment, knowing the single measurement precision of the ammeter and voltmeter.

Ok, so you got a 10% deviation and a 2% only for 1 measurement... or any measurement, even though you change the current or voltage?

By taking multiple measurements I can produce different errors in each mean.

In each mean of what, exactly? And now the error is not 10% and 2% anymore for any single measurement?

Using the analytic formula I can figure out how to optimize the experimental procedure for a given expenditure of effort.

How, exactly? I do not see it. As you have worded it, I see no way to improve $\sigma _R$ by changing the current or the voltage applied. The only way to reduce the uncertainty would be to brainlessly repeat the measurement to reduce the standard deviation of the mean. Unless of course you throw out everything and focus on the real world problem, in which case you would use an AC ("high frequency" to kill off thermoelectric effects but not so high either as to prevent the skin effect, low intensity as not to heat up the resistor whose resistance you want to measure, but not too low either, else the uncertainty in the voltage starts to become an important part of the error, etc.). But none of this is determined with formulas, be it either numerical or analytical, to me.

Such optimizations are often required in the real would with many input error sources.

So I really, really, really hope I will be able to understand what you've written so far, because I want to do things in an efficient way.

Repeatedly plugging guesses into a Monte Carlo simulation is not a good way to proceed.

I mean, it is probably less useful than repeating a measurement (since you gain no more information with Monte Carlo), but isn't sensitifivity analysis done that way, i.e. fix all inputs of Monte Carlo but one, and vary this one to see how the output changes? That is a useful thing to do, or not?

The purpose of analytic forms is to allow your mind to symbolically manipulate complicated problems for systems too entangled to otherwise be amenable to such analysis. Asking a computer app for a number is not at all the same.

.

Sure, I do agree with you, in general. But in this particular case, I do not see how the analytical formula provides the information on how to use wisely the time, i.e. by performing 100 times the experiment. Do you do anything different between those 100 experiments? If you change the current or voltage for instance, will the voltmeter/ammeter still show a 10% and 2% deviation (in real world I know that no, of course, but the way you worded it, to me, points that yes... hence my confusion). I don't really see how the analytical formula provides any guidance at all. I am not even comparing it to MC yet, just common sense.

 

[hutchphd]

The only way to reduce the uncertainty would be to brainlessly repeat the measurement to reduce the standard deviation of the mean.

Not brainlessly. At least not when I do it. That is the whole point.

First, in the example, a single measurement means measuring V or measuring I not measuring both. I know that the imprecision in the I is 5 times worse than V for a single measurement. To make the measured mean value I to have the same precision as V one needs to repeat the current measurement (jiggle the wires, bump the table, stomp the floor in between as prudent) 25 times. This equality of errors will be the best solution (I can show this analytically but not right now) for the stated problem...so I would measure I 100 times and V 4 times to be near 100 measurements as desired. The RMS measurement error for this case $$\frac {\sigma _R} R =\sqrt {\frac {(.1)^2} {100} + \frac {(.02)^2} 4 }=\frac {.02} {\sqrt 2} $$ assuming I did the arithmetic OK. Notice this imprecision is smaller than the better (2%) measurement device random imprecision and optimizes the result at requested level of effort.

but isn't sensitifivity analysis done that way, i.e. fix all inputs of Monte Carlo but one, and vary this one to see how the output changes? That is a useful thing to do, or not?

Only if all you know how to do is a mindless Monte Carlo Simulation. Again that's the point. Giving it a name does not make it smart.

 

[Office_Shredder]

Based on my experience six years ago I think numerical simulations were probably undertaught, but this is clearly too extreme.

The post here is a bad idea, as proven by the fact that the OP doesn't understand any of the descriptions of why it's a bad idea. I think what probably happened is you didn't understand how standard deviations intuitively, found out you can kind of get a computer to do it for you, and then wished that you never had to learn it in the first place. But there are loads of examples where understanding the formula and a bit of algebra can solve a problem that would be mind numbingly annoying to deal with by just stimulating. It also means you're not going to be able to verify your simulator results - if there's a bug and it outputs bad numbers, many of the posters here would be able to catch it, but if you don't know how these errors propagate then you can't.

 

[fluidistic]

Office_shredder, I think you may have some cognitive bias against me. If you reread my first post, I am wondering why MC is not mentioned (nor taught) in standard undergraduate (physics) curriculum. I have nothing against what is taught (the mentioned analytical formula). It is only after someone stated that there is no time to cover everything that in this very restrained case I mentioned to then teach the most useful in real world (which also happens to be more intuitive to me) way to compute uncertainties, which I thought would be MC over the analytical formula. I would argue that this is as extreme as not mentioning MC (I am not even talking about teaching MC, just mentioning it).

Now hutchphd is teaching me that the analytical formula provides insights that the MC do not, it's taking time for me to see it, but does that mean I wish I had never been taught the analytical formula and used it with a button in a computer? :) Not necessarily. :)
With MC, you also get a mean and a standard deviation (same thing as the analytical formula), that's the reason why I did not see hutchphd's point at first, but now I see exactly what he meant. Thank you (and sorry for your time), hutchphd.

Ok, so, good, the analytical formula can be used to planify future measurements in a wise way that MC can't (or at least, not as quickly as one would hope. I am no expert at all, just learned about it, so I may very well miss things!). So it should not be thrown away in real life experiments and let its place to MC entirely. OK.

Then my original post still stand though. There might be no time to cover Monte Carlo (and thus ask students to use it), but shouldn't it at least be mentioned, possibly only covering its advantages and disadvantages over the standard propagation of errors formula? That does seem like a big deal to me. It is so easy to implement that... ok I stop here :D.

 

[Office_Shredder]

It sounds like we're basically in agreement actually, cool.

 

[hutchphd]

Thank you (and sorry for your time), hutchphd.

Happy to teach folks who are receptive to new ideas. And I am a really big fan of looking at scatter graphs of data chosen and computed in a Monte Carlo fashion. It can give really good insight.

 

[atyy]

The bootstrap, which is often taught, often uses Monte Carlo methods.
http://faculty.washington.edu/yenchic/17Sp_302/R11.pdf
https://en.wikipedia.org/wiki/Bootstrapping_(statistics)

The basic concepts of these methods are fundamental, but not so much used in ancient times because of the lack of computer power.

 

[FactChecker]

Based on my experience six years ago I think numerical simulations were probably undertaught, but this is clearly too extreme.

The post here is a bad idea, as proven by the fact that the OP doesn't understand any of the descriptions of why it's a bad idea. I think what probably happened is you didn't understand how standard deviations intuitively, found out you can kind of get a computer to do it for you, and then wished that you never had to learn it in the first place. But there are loads of examples where understanding the formula and a bit of algebra can solve a problem that would be mind numbingly annoying to deal with by just stimulating. It also means you're not going to be able to verify your simulator results - if there's a bug and it outputs bad numbers, many of the posters here would be able to catch it, but if you don't know how these errors propagate then you can't.

My experience has been the opposite. The theoretical models are very simple and only apply in the most ideal situations. Any complications like logic switching, situational behavior, etc., can turn a simple theoretical calculation into a nightmare. In fact, there have been threads in these forums where problems with apparently simple modifications turned the theoretical calculation into the subject of a hundred argumentative extremely complicated posts whereas the Monte Carlo estimate was easily obtained.

 

[hutchphd]

The "simple theoretical models" have enabled me to successfully design calibration protocols for dozens of medical diagnostic systems to pass FDA certification. In addition they were sufficient to rescue a production run of nearly a million blood glucose meters. If the Monte Carlo statistics worked, I wouldn't have been needed.
The bottom line is you need to know what the hell you are doing...The Monte Carlo methods give you numbers and very little insight. If you don't understand the theory, you will not know what you are doing.
The best tool is still a good graph and good eyeballs.
The fact that the Monte Carlo numbers are easy to obtain says nothing about their particular value.

 

[FactChecker]

The "simple theoretical models" have enabled me to successfully design calibration protocols for dozens of medical diagnostic systems to pass FDA certification. In addition they were sufficient to rescue a production run of nearly a million blood glucose meters. If the Monte Carlo statistics worked, I wouldn't have been needed.
The bottom line is you need to know what the hell you are doing...The Monte Carlo methods give you numbers and very little insight. If you don't understand the theory, you will not know what you are doing.
The best tool is still a good graph and good eyeballs.
The fact that the Monte Carlo numbers are easy to obtain says nothing about their particular value.

Monte Carlo does work, just like any sampling technique works. With enough simulations, you can get the entire distribution. I spent my entire working career in simulation and the work paid off. You can compare alternative designs, look for possible rare events, etc. The fact that there are many applications of the standard theoretical distributions does not mean that there are not also many cases where slight modifications make the theory extremely hard and impractical.
In other words, von Neumann was not dumb.

 

[hutchphd]

But the original issue here was "why didn't somebody teach me this?". My response is that it takes ten minutes to teach it and you should indeed know it, but it in no way substitutes for knowing the analytic method which is considerably more involved. In fact to use Monte Carlo well, one must know the analysis.
Also this was one of an increasing number of posts saying "why teach all this analytical crap with physics?"... Just plug it in and run a simulation whenever you need to know something. You understand that this is not a viable method for innovation and not even a good representation of the way Physics is done

 

[FactChecker]

But the original issue here was "why didn't somebody teach me this?". My response is that it takes ten minutes to teach it and you should indeed know it, but it in no way substitutes for knowing the analytic method which is considerably more involved. In fact to use Monte Carlo well, one must know the analysis.
Also this was one of an increasing number of posts saying "why teach all this analytical crap with physics?"... Just plug it in and run a simulation whenever you need to know something. You understand that this is not a viable method for innovation and not even a good representation of the way Physics is done

Ok. I'll buy that. Knowing the standard theoretical models is essential. But Monte Carlo simulations are often not that easy. There are simulation languages and techniques that can take a while to learn. A prior post mentioned buying apps that make it trivial. I do not know what that was referring to. Only trivial examples are easy, but the real world gets complicated fast.

 

[hutchphd]

Mostly I was talking about process control and medical statistical sampling applications.
Clearly techniques developed by von Neumann, Fermi, and Ulam will not be trivial, but they were using the techniques for simulations otherwise not doable (at least with their contemporary computing resources). Again they are certainly useful where they are useful.

 

[fluidistic]

I appreciate your exchanges, people.
Going back to the resistor measurement. Imagine I want to measure a resistor's resistance, and that it's true value is 308 ohm. I use a current source giving a current of intensity 25 uA (uncertainty or standard deviation of 10%) and I measure a voltage of 7.7 mV (SD 2% as in hutchphd's example). A lucky strike, I know, this yields the true value for R, but this doesn't matter.

Out of a single measurement, Monte Carlo gives me the following histogram for R:

The obtained simulations give me that R = 308 +16 -14 ohm. I reported the median and the 68% confidence interval, but of course I can get the mean, standard deviation and any other commonly reported stats. The analytical formula tells me that R = 308 +/- 32 Ohm (it makes sense, it's slightly higher than 10% of R, which it should since the biggest uncertainty comes from the current, which is 10%). I am not sure why there is such a disparity between MC and the analytical formula, as the histogram does not differ that much from a Gaussian. I probably goofed with the confidence interval.

Anyway, if I am not doing things the wrong way, a sensitivity analysis tells me what we already knew, i.e. that the histogram will not change much even if the standard deviation of the voltage reaches 0 (for example by repeating the voltage measurement reading an infinite number of times). But it also shows the histogram of R I would get if I managed to reduce the uncertainty in the current by a factor 10 (by making 100 measurements of the current, assuming that a lot of sources of error introduced by the use of an ammeter are negligible, which is not a given in real life conditions). See it for yourself:

With MC, I have to input the probability distributions of I and V, and it turns out that if I repeat the experiment N times, the uncertainty in either I or V will decrease by a factor of sqrt(N). This is not a property of the analytical formula alone, it's just a fact about the standard deviation of the mean, I believe. So I can use this factor and repeat the sensitivity analysis by reducing the standard deviation of I and/or V as I like. In this way, I do not quite entirely see why MC gives inferior insight than the analytical formula. The more I think about it, the more I get immersed in the complexity. For example, I would never do, in real life, the 4 measurements of I and 100 measurements of V. It is not as simple as that. If, for an experiment, only V is measured and in the other experiment only I is measured, then it's probably not exactly the same setup, I am not sure if it is as easy to analyze the results from such experiments. Here in my example of a resistor of 308 ohm, it may not matter, but if the resistance was about 1 ohm, plugging an ammeter could be quite annoying and change drastically the result.

I actually enjoy this example, because it could be a real life good example, since measuring accurately a resistance is sometimes required.

 

[hutchphd]

Out of a single measurement, Monte Carlo gives me the following histogram for R:

The obtained simulations give me that R = 308 +16 -14 ohm.

.

I do not understand quoted simulation passband and this graph. They are clearly inconsistent. The width on the graph is clearly 60 ohms or more. Am I not understanding?

 

[fluidistic]

.

I do not understand quoted simulation passband and this graph. They are clearly inconsistent. The width on the graph is clearly 60 ohms or more. Am I not understanding?

Yeah, like I said I probably goofed in the reported percentiles. I will check my code tomorrow.

 

[fluidistic]

It turns out I didn't make any mistake there (I think!). It may seem counter intuitive, so here is the same plot (well another such simulation) with the mean, 68.27 interval, 95.45 and 99.73 intervals (1, 2 and 3 SD for a Gaussian). It may be an artifact of the histogram, which contains "only" 250 bins, so beware.

There are 100,000 values of R.
Lower bound 3 sigma comes at number 270. (as it should, this is 2.7%, or 1-0.9973). The value of R is 239 ohm.
Lower bound 2 sigma comes at number 4,650. (again, trivially as it should). R = 263 ohm.
Lower bound 1 sigma comes at number 31,730. (idem, ad nauseam) R = 294 ohm.
Mean comes at roughly number 50,000. (do I even need to think anymore at this point?) R = 308 ohm.
Upper bound 1 sigma comes at number 68,270. R = 324 ohm.
Upper bound 2 sigma comes at number 95,450. R = 372 ohm.
Upper bound 3 sigma comes at number 99,730. R = 430 ohm.

If this holds, this also means that even in this simple case where the histogram looked somewhat similar to a Gaussian at very frst glance (sure, it's visibly not a Gaussian, but it doesn't look that different at first glance), the common analytical formula overestimates significantly the uncertainty, assuming MC is more accurate