- #1
0xDEADBEEF
- 816
- 1
Maybe this counts as computational physics, or statistics I don't know where this topic belongs.
Ok so I have this nice high quality Thermodynamic measurement data and I have to start the "data raping" for results. Let's call the series A(t), and B(t) and I have a few more.
So for different times I have different measurement points. And I am very much interested in [tex]\frac{\mathrm{d}A}{\mathrm{d}B}[/tex] A(B) undergoes a phase transition and will show kinks or jumps, and it is measured with high accuracy. B(t) is very smooth, but the measuring equipment introduces an error that is too large for my taste.
What are good methods or standard methods that are used in calculating derivatives. If you go from one point to the next and just divide differences, then the results are dominated by the noise. Then there might be some oversampling going on, where the value has hardly changed from one point to the next.
My ideas are so far:
(By the way I am using R for the evaluaton, so any cool statistics trick you can recommend can probably be found somewhere as a routine.)
1) take points [t-delta,t+delta] of (B(t),A(t)) do a linear fit, take the slope, do that for each point (this means an artificial correlation is introduced between neighboring points, because their linear fit goes through all the same points except for two). Is there a name for this? How do you choose a good value for delta.
2) somehow smooth the data of B(t), to get rid of noise. There is no exact theory that could be fitted. One idea was averaging neighbors. Maybe using the Simpson 3/8. When does the use of polynomials make sense? I know there is something called smoothing with cubic splines for example in gnuplot. Does this really smooth the data, or does it simply make cubic splines that run through all the data points (instead of averaging around them). What is the best way to include a lot of points like 10? Still integrating over a polynomial?
3) This is madness/Sparta/Fourier transforms: Somehow do a Fourier Transform over (A(B)) for example with a FFT over interpolated points, then multiply by [tex]\omega[\tex] and transforming back. Maybe I could also do something to the high frequency coefficients at the same time to reduce noise.
Any tips? How do you make derivatives?
Ok so I have this nice high quality Thermodynamic measurement data and I have to start the "data raping" for results. Let's call the series A(t), and B(t) and I have a few more.
So for different times I have different measurement points. And I am very much interested in [tex]\frac{\mathrm{d}A}{\mathrm{d}B}[/tex] A(B) undergoes a phase transition and will show kinks or jumps, and it is measured with high accuracy. B(t) is very smooth, but the measuring equipment introduces an error that is too large for my taste.
What are good methods or standard methods that are used in calculating derivatives. If you go from one point to the next and just divide differences, then the results are dominated by the noise. Then there might be some oversampling going on, where the value has hardly changed from one point to the next.
My ideas are so far:
(By the way I am using R for the evaluaton, so any cool statistics trick you can recommend can probably be found somewhere as a routine.)
1) take points [t-delta,t+delta] of (B(t),A(t)) do a linear fit, take the slope, do that for each point (this means an artificial correlation is introduced between neighboring points, because their linear fit goes through all the same points except for two). Is there a name for this? How do you choose a good value for delta.
2) somehow smooth the data of B(t), to get rid of noise. There is no exact theory that could be fitted. One idea was averaging neighbors. Maybe using the Simpson 3/8. When does the use of polynomials make sense? I know there is something called smoothing with cubic splines for example in gnuplot. Does this really smooth the data, or does it simply make cubic splines that run through all the data points (instead of averaging around them). What is the best way to include a lot of points like 10? Still integrating over a polynomial?
3) This is madness/Sparta/Fourier transforms: Somehow do a Fourier Transform over (A(B)) for example with a FFT over interpolated points, then multiply by [tex]\omega[\tex] and transforming back. Maybe I could also do something to the high frequency coefficients at the same time to reduce noise.
Any tips? How do you make derivatives?