- #36
- 42,141
- 10,205
When a formula involves taking the difference between two positive values, the potential fractional error in the result exceeds the sum of the fractional errors in the operands.haruspex said:At most, the number of [significant figures] should be the lower of those in the first two columns; mostly that's three.
But given the subtraction in the formula, even that may be too many. I'll explain why in a later post.
Your formula involves ##1-\frac{F_i}{F_g}##. The first data pair in your table is ##F_i=0.11, F_g=0.16##. That implies ##0.105<F_i<0.115, 0.155<F_g<0.165##, i.e. fractional errors of ##\pm 4.5\%, \pm 3.1\%##. The standard rule adds those to produce an error of ##\pm 7.6\%## for ##\frac{F_i}{F_g}##, i.e. a value from 0.64 to 0.74. So ##0.26<1-\frac{F_i}{F_g}<0.36##, an error of ##\pm 15##%. As a result, the answer should be given as ##3\cdot 10^3##.