How do I compute the second derivative of a one-dimensional array?

[btb4198]

How do I compute the second derivative of an one dimensional array?

 

[fresh_42]

How do I compute the second derivative of an one dimensional array?

An array of what?

You simply differentiate twice. If you have a function $f\, : \,\mathbb{R}^m\longrightarrow \mathbb{R}^n$ you get an array of Hesse matrices as a result, a tensor, a single matrix in case $n=1.$

 

[btb4198]

array of doubles

 

[berkeman]

array of doubles

LOL. Well, that's super helpful, thanks.

Seriously though, if you have two columns of numbers (in Excel for example), with the first column the time stamps of the samples, and the 2nd column the position at that time stamp, make two more columns for the velocity and acceleration as a function of time.

For those last 2 columns, you would compute each entry as the change in the previous column divided by the change in the time column.

Have you made these types of Excel spreadsheets in the past? You can do the same thing with a CSV file and doing the computations with your language of choice (C, Python, etc.).

 

[Mark44]

How do I compute the second derivative of an one dimensional array?

An array of what?

array of doubles

If the array consists of constants of type double, and not functions of some variable, then it doesn't make much sense to talk about the first derivative, let alone the second derivative. The derivative with respect to any variable of a constant is zero.

If you're asking about first differences and second differences, rather than first and second derivatives, that's something different. Solving difference equations, as opposed to differential equations (which involve derivatives) might be related to what you're asking. If you do a web search for either of these terms, or for difference equations, you'll get lots of hits.

 

[btb4198]

The array is of X-ray value sorted into bins from a DPP. so kve vs bin.
but I am not sure what that matter

 

[btb4198]

LOL. Well, that's super helpful, thanks.

Seriously though, if you have two columns of numbers (in Excel for example), with the first column the time stamps of the samples, and the 2nd column the position at that time stamp, make two more columns for the velocity and acceleration as a function of time.

For those last 2 columns, you would compute each entry as the change in the previous column divided by the change in the time column.

Have you made these types of Excel spreadsheets in the past? You can do the same thing with a CSV file and doing the computations with your language of choice (C, Python, etc.).

Can you do an example by hand?

 

[btb4198]

How do you get the change ?

 

[berkeman]

Can you do an example by hand?

I'd prefer to do it in Excel and post a PDF of it with the cell math shown.

Have you never done this in Excel? What is your exact task? Is it from a lab or a homework assignment?

I have meetings for the rest of the night, but will try to post an example Excel spreadsheet in my morning if nobody else beats me to it.

 

[btb4198]

It is not for school ... I am going to be coding this

 

[Mark44]

How do I compute the second derivative of an one dimensional array?

As already stated, this doesn't make much sense. You can't take the derivative of an array.
Also, it's meaningless to "take the derivative" of something. Differentiation always represents the change of one variable with respect to some other variable. So far you haven't told us what the other variable is.

The array is of X-ray value sorted into bins from a DPP. so kve vs bin.

DPP?
kve == kilovolt electrons?
What does "kve vs bin" mean?
How are the bins determined?
Please stop with the overly terse responses that tell us almost nothing.

It is not for school ... I am going to be coding this

Good luck with that. You haven't given us a clear explanation of what you're doing, which makes it extremely difficult for us to give meaningful help or for you to implement code for it.

 

[berkeman]

I'd prefer to do it in Excel and post a PDF of it with the cell math shown.

Here is a simple example using Excel. The sample time base is 0.1 seconds between samples, and I made up the position (Distance) numbers. The equations for the velocity and acceleration are as follows:

Velocity calculation in cell C3: =(B3-B2)/$A$2

Acceleration calculation in cell D4: =(C4-C3)/$A$2

 

[fresh_42]

Here is a simple example using Excel. The sample time base is 0.1 seconds between samples, and I made up the position (Distance) numbers. The equations for the velocity and acceleration are as follows:

Velocity calculation in cell C3: =(B3-B2)/$A$2

Acceleration calculation in cell D4: =(C4-C3)/$A$2View attachment 301649

Just as information. You can copy cells into the PF edit window which automatically turns into an editable table instead of an image.

14.502,4100​	7,0000​	4.395,0483​
14.248,9600​	7,0000​	4.386,3770​
13.974,7500​	7,0000​	4.238,1300​
13.975,2500​	7,0000​	4.222,6772​
13.749,2900​	7,0000​	4.196,6640​
13.915,9700​	7,0000​	4.203,5547​
14.097,8800​	7,5000​	4.203,5737​
13.947,2100​	7,5000​	4.155,7250​
14.039,4700​	7,5000​	4.173,9976​

 

[berkeman]

Just as information. You can copy cells into the PF edit window which automatically turns into an editable table instead of an image.

Neat. Trying that here:

Time	Distance	Velocity	Acceleration
0.1	0
0.2	1	10
0.3	3	20	100
0.4	4	10	-100
0.5	6	20	100
0.6	8	20	0
0.7	10	20	0
0.8	9	-10	-300
0.9	5	-40	-300
1	10	50	900
1.1	20	100	500
1.2	21	10	-900
1.3	22	10	0

 

[berkeman]

Hmm, do I need to paste it in with some option, or just ctrl-v? I don't see the cells above as editable or showing the formulas for the calculations.

 

[fresh_42]

Hmm, do I need to pate it in with some option, or just ctrl-v? I don't see the cells above as editable or showing the formulas for the calculations.

I only used Ctrl+V. And I edited your table. Your last row has now a mistake. Of course, it does not copy the formulas, only the content.

 

[berkeman]

I only used Ctrl+V. And I edited your table. Your last row has now a mistake. Of course, it does not copy the formulas, only the content.

Okay, I fixed the error in the last row of the table now. I agree that it just does paste-by-value.

 

[btb4198]

berkeman

Thanks! I understand how to do this now. One question
should it be
(y2-y1) / ( x2-x1)

because you have :
C3: =(B3-B2)/$A$2

what about x1 ?

 

[berkeman]

berkeman

Thanks! I understand how to do this now. One question
should it be
(y2-y1) / ( x2-x1)

because you have :
C3: =(B3-B2)/$A$2

what about x1 ?

My sample intervals were on the even 0.1 second marks, so t2-t1 = t1 for that case. Makes sense? If the sampling were not on the same time interval, then yes you would need to compute the delta-t for each case.

 

[Mark44]

One question
should it be
(y2-y1) / ( x2-x1)

In post #1 you asked about the "derivative of an array." As I mentioned earlier, a derivative represents the change of one variable with respect to or relative to another. As far as I can tell, and from what you described, you have only one quantity. @berkeman's spreadsheet had variables of distance vs. time and velocity vs. time.

 

[Mark44]

How do I compute the second derivative of an one dimensional array?

A better question, IMO, is to first tell us what you are trying to do rather than jumping into the details of how to do it. This is, in part, why we're 20 posts into this thread.

 

[jbriggs444]

In post #1 you asked about the "derivative of an array." As I mentioned earlier, a derivative represents the change of one variable with respect to or relative to another. As far as I can tell, and from what you described, you have only one quantity. @berkeman's spreadsheet had variables of distance vs. time and velocity vs. time.

If the data are not from evenly spaced intervals then the question of what "time" values one needs to use becomes trickier.

What is calculated for the "first derivative" is a reasonable estimate for the "velocity" of the measured "position" at the midpoint of each "time" interval. It is the average "velocity" for the interval. The "time" imputed for this estimated velocity should be the "time" of the interval midpoint.

The calculation for the "second derivative" would then be a reasonable estimate for the "acceleration" between the midpoint of the first interval and the midpoint of the second interval. But only if the "time" that one divides by is the "time" between the interval midpoints rather than "time" between the interval endpoints. The "time" imputed for this estimated "acceleration" should be the "time" of the midpoint between the two midpoints.

I do not know whether this will eliminate all bias in your estimation process, but it will at least eliminate some. Depending on what you are really trying to do, entirely different approaches might be more appropriate.

what about x1 ?

If I understand that concern properly, the above discussion takes care of it. If you had $n$ samples, you only get to calculate $n-1$ average rates of change.

Edit: DPP in context of X-Rays. Perhaps https://moxtek.com/xray-product/mxdpp-50-digital-pulse-processor/. KVe appears to be Kilo Volt effective.

 

[Mark44]

If the data are not from evenly spaced intervals then the question of what "time" values you need to use becomes trickier.

I don't think we've ever found out anything about intervals yet. In post #6 the OP said something about Xray values in bins, "kve vs bin," whatever that means.

 

[btb4198]

I don't think we've ever found out anything about intervals yet. In post #6 the OP said something about Xray values in bins, "kve vs bin," whatever that means.

Mark,
Please watch this video : What is X-ray Fluorescence (XRF)?
It will explain the spectrum output of my Digital Pulse Processor (DPP).

Does this help?

 

[btb4198]

If the data are not from evenly spaced intervals then the question of what "time" values one needs to use becomes trickier.

What is calculated for the "first derivative" is a reasonable estimate for the "velocity" of the measured "position" at the midpoint of each "time" interval. It is the average "velocity" for the interval. The "time" imputed for this estimated velocity should be the "time" of the interval midpoint.

The calculation for the "second derivative" would then be a reasonable estimate for the "acceleration" between the midpoint of the first interval and the midpoint of the second interval. But only if the "time" that one divides by is the "time" between the interval midpoints rather than "time" between the interval endpoints. The "time" imputed for this estimated "acceleration" should be the "time" of the midpoint between the two midpoints.

I do not know whether this will eliminate all bias in your estimation process, but it will at least eliminate some. Depending on what you are really trying to do, entirely different approaches might be more appropriate.If I understand that concern properly, the above discussion takes care of it. If you had $n$ samples, you only get to calculate $n-1$ average rates of change.

Edit: DPP in context of X-Rays. Perhaps https://moxtek.com/xray-product/mxdpp-50-digital-pulse-processor/. KVe appears to be Kilo Volt effective.

Hi sorry I just update. That is the DPP I am using.

 

[berkeman]

Please watch this video : What is X-ray Fluorescence (XRF)?
It will explain the spectrum output of my Digital Pulse Processor (DPP).

Does this help?

So your "1-dimensional data" is really this 2-dimensional data in the plot (counts per energy bin), and you want to take the 2nd derivative of the data for some reason?

 

[fresh_42]

Hi sorry I just update. That is the DPP I am using.

But what do you want to compute the second derivative from and why? From the curve that can be sensed from that graphic? If so, then you need an analytical approximation for that curve. One way to do so is to mark several crucial points $(x,y)$ where $x$ denotes the value of the horizontal axis and $y$ the one of the vertical axis. Then you can set up a polynomial $p(x)=a_0+a_1x+a_nx^2+\ldots+a_nx^n$ with $n$ one less than the number of points you have chosen, and solve for $p(x)=y$ for your points. This gives you the values for the $a_k$ and you can differentiate $p(x).$

 

[btb4198]

So your "1-dimensional data" is really this 2-dimensional data in the plot (counts per energy bin), and you want to take the 2nd derivative of the data for some reason?

Sorry, I am coding someone else math. I really do not have too many answers as to why.

Question: when you have:
C3: =(B3-B2)/$A$2

My A2 would always be 1 right ?

 

[Mark44]

Sorry, I am coding someone else math. I really do not have too many answers as to why.

Question: when you have:
C3: =(B3-B2)/$A$2

My A2 would always be 1 right ?

This is pretty much off the wall in this thread. The above looks to be an Excel formula. If you copy this formula to the cell below, the formula in cell C4 will be "=(B4 - B3)/ $A$2". The purpose of the $ signs is to leave this part of the formula the same whether it's copied to columns to the right ($A) or rows below ($2). If the value in A2 is 1, then that's the same value that will be used when the formula is copied.

But if A2 is 1, why is it in the formula? Division by 1 makes no change in the expression it's in.

 

[btb4198]

This is pretty much off the wall in this thread. The above looks to be an Excel formula. If you copy this formula to the cell below, the formula in cell C4 will be "=(B4 - B3)/ $A$2". The purpose of the $ signs is to leave this part of the formula the same whether it's copied to columns to the right ($A) or rows below ($2). If the value in A2 is 1, then that's the same value that will be used when the formula is copied.

But if A2 is 1, why is it in the formula? Division by 1 makes no change in the expression it's in.

Yeah, I am going to remove it. Sorry, I never code a function to take a derivative of a graph before and I could not find any YouTube videos on how to do it by hand.
So I post the question here.
anyhow,
So Basically to take the first derivative, I am going to go through the array and subtract index + 1 from index.
So:
double value = spectrumData[i + 1] - spectrumData;and repeat the same process for the second derivative .

is this how Matlab would do it ?

 

[fresh_42]

Yeah, I am going to remove it. Sorry, I never code a function to take a derivative of a graph before and I could not find any YouTube videos on how to do it by hand.

See my post #27. It is an instruction.

 

[berkeman]

So Basically to take the first derivative, I am going to go through the array and subtract index + 1 from index.
So:
double value = spectrumData[i + 1] - spectrumData;

You need to get your units right in your derivative calculations (both for the numerator and denominator). The units for the numerator look to be "counts" right? And for the denominator, you have bins separated by some delta-energy in keV? So the units of the first derivative would be delta-counts per delta keV?

 

[berkeman]

So the units of the first derivative would be delta-counts per delta keV?

It looks like the horizontal bins are spaced linearly? What is the increment in delta-keV per bin? It looks like a small fraction of a keV delta per bin?

 

[btb4198]

You need to get your units right in your derivative calculations (both for the numerator and denominator). The units for the numerator look to be "counts" right? And for the denominator, you have bins separated by some delta-energy in keV? So the units of the first derivative would be delta-counts per delta keV?

Yes, that is all correct.

 

[btb4198]

It looks like the horizontal bins are spaced linearly? What is the increment in delta-keV per bin? It looks like a small fraction of a keV delta per bin?

From this picture is looks like the bins increase by one .