Please discuss discrete Fourier analysis

[mattrix]

TL;DR Summary

What are the subtleties when using Fourier analysis?

It has been 35 years since I did the math for Fourier analysis, and I have forgotten what the subtleties are. Please be kind.
So this is not a how do I calculate a DFT (though that may be my next question) but rather how do I use it, and interpret the results.

All the online and software I find seems to use the FFT algorithm.
If I remember properly the FFT requires 2^n samples. I recall you can zero pad to get the samples, but doesn't this change the fundamental frequency/period. eg if I have data for 'm' complete periods I would expect a prominent peak on the m'th harmonic, but if I have zero padded that peak with be scattered across several harmonics, none of which are related to the period in the data ?

What is the FFT calculating anyway? I want the spectrum of a continuous signal. When I multiply this with a comb function (sample) is the FFT calculating the spectrum of the product or the original continuous signal?

Thats all I can put words to at the moment. I have vague recollections that how you choose the data is important but I can't remember how or why.

I look forward to your comments.

 

[mcastillo356]

Hi, @mattrix, nice question, though at the first glance, thougt might be helpful, I can only give a useless link:

https://www.nti-audio.com/en/support/know-how/fast-fourier-transform-fft

Sorry, Love

 

[sophiecentaur]

A DFT looks at a time window of a function and assumes that the data repeats endlessly. This contrasts with the FT which transforms an infinite set of time samples into an infinite set of frequency samples. The resulting DFT will be a comb of samples which are harmonics of the repeat frequency; there are no other frequencies involved.

The FFT is just a quick method of DFT which works only for blocks of 2ⁿ samples. I heard it described as a 'folding' of the array of samples which eliminates the need for many of the calculations that a full DFT will do.

The error that occurs when you try to DFT a non-repeating function by selecting an arbitrary number of samples can be reduced by suitable 'windowing' of the array of samples so that the samples near the centre are weighted bigger than the samples near the end. This has the effect of reducing errors in the fine structure of the signal.

Another bit of arm waving: you can say that the DFT calculates the correlation of the input signal with each of the harmonics of the repeat frequency. That's the same as the frequency domain description of the input signal. There are as many ways of describing these things as you've had hot dinners so you have to pick (a valid) one of them if you want a good feel for the subject.

 

[mattrix]

Thankyou everyone, I knew there was more to this question than I could recall.
The trouble with University math is that all the functions are easily describable in the time domain, and hence manipulable. Real data doesn't obey these rules.
I am assuming that the real data has,
1. high frequency noise
2. low frequency bias

What I wanted to do with Fourier analysis was to select the significant components from the frequency spectrum and to sum them to get an approximation in the time domain.

I had forgotten about 'windowing' and the artifacts that they produce (such are the ravages of time). However in my case I am 'imposing' a periodic structure on the data: according to me it should be periodic.

The 2^n samples does concern me. If I recall correctly, the lowest frequency detected will have a period of 2^n time samples long (or is that 2^(n-1)?), the 2nd harmonic will have a period half as long.
So if my data has 1260 time samples and I zero pad to 2048 samples, then there will be no component with a period of 1260 samples, although I have 'imposed' that it does repeat after 1260 time samples. What other artifacts does this padding cause?

I guess I could interpolate the data and resample so that there are 2^n samples, but I think this will also cause artifacts?

 

[DrClaude]

The restriction of FFT to $N=2^n$ is archaic. A good modern FFT implementation, such as FFTW or GSL, will be efficient for $N$ a product of small primes.

$1260 = 2^2 \times 3^2 \times 5 \times 7$ so it will work fine, no padding necessary.

 

[mattrix]

Thank you DrClaude,
That does what I intended well.
Though it has sent me down a rabbit hole of new searches.

 

[mattrix]

So does what I intend to do:

Sample the data for 'm' periods: do an FFTW: select suitable components: convert them back to a continuous time waveform.

seem reasonable?

Does it pose any problems or introduce significant artifacts?

 

[berkeman]

I am assuming that the real data has,
1. high frequency noise
2. low frequency bias

Sample the data for 'm' periods: do an FFTW: select suitable components: convert them back to a continuous time waveform.

I may have missed it, but what is the bandwidth of the signal that you want to digitize and process? What is your digitizing sample rate?

Where are you setting the cutoff of your anti-alias preamp filter, and what polynomial are you using in that analog filter? What order?

 

[Jarvis323]

Thankyou everyone, I knew there was more to this question than I could recall.
The trouble with University math is that all the functions are easily describable in the time domain, and hence manipulable. Real data doesn't obey these rules.
I am assuming that the real data has,
1. high frequency noise
2. low frequency bias

What I wanted to do with Fourier analysis was to select the significant components from the frequency spectrum and to sum them to get an approximation in the time domain.

I had forgotten about 'windowing' and the artifacts that they produce (such are the ravages of time). However in my case I am 'imposing' a periodic structure on the data: according to me it should be periodic.

The 2^n samples does concern me. If I recall correctly, the lowest frequency detected will have a period of 2^n time samples long (or is that 2^(n-1)?), the 2nd harmonic will have a period half as long.
So if my data has 1260 time samples and I zero pad to 2048 samples, then there will be no component with a period of 1260 samples, although I have 'imposed' that it does repeat after 1260 time samples. What other artifacts does this padding cause?

I guess I could interpolate the data and resample so that there are 2^n samples, but I think this will also cause artifacts?

You might be confusing the 2^n window size for an fft with the nyquist frequency? In that case you need to sample at twice the frequency that you want to detect.

The window size affects the resulution of the result, basically the number of bins.

 

[berkeman]

In that case you need to sample at twice the frequency that you want to detect.

Not in the real world...

 

[mattrix]

I may have missed it, but what is the bandwidth of the signal that you want to digitize and process? What is your digitizing sample rate?

Where are you setting the cutoff of your anti-alias preamp filter, and what polynomial are you using in that analog filter? What order?

No explicit filtering has been applied. So I guess bandwidth is infinite?

I'm intending to use FFTW to explore the data. And treating this as a math problem (maybe incorrectly?)

IIRC the number of samples I choose to use determines the longest period I can detect in the data.
In the proposed case, 1260 samples/period. By using the FFTW algorithm I am saying the sequence will repeat after these 1260 samples.

I actually expect the data to repeat 13 times in these 1260 odd samples.

The shortest period I can detect is 2 time samples long. Roughly 50 times the expected periodicity in the data.

But I am just pushing numbers around the page now. I don't know if I have answered your questions?

 

[berkeman]

I don't know if I have answered your questions?

Not really. Digitizing analog waveforms in the real world involves lots of important considerations, some of which led to my questions for you above.

It would be best if you read through some good references on how to digitize waveforms correctly. Otherwise you will be chasing your tail on a number of issues. This Application Note by Analog Devices is a good place to start:

https://www.google.com/url?sa=t&rct...s/an-282.pdf&usg=AOvVaw3qlhBVdp1LOYzPBtff1c9o

 

[mattrix]

Thanks berkeman,
A lot of information in that link. I will have to read it a few times to get all this to gel.

In my case, I am presented with data that has already been sampled for digital storage. I do know that it has been oversampled/decimated so I guess some care has been taken with the data.