How to determine if distributions are correlated?

In summary, the conversation discusses using fast-fourier transforms (FFT) in Matlab to analyze real data and identify correlations between different spurs in the data. The speaker proposes using probability distribution functions (PDFs) to determine correlation and asks for feedback on this method. They also mention the challenge of determining correlation when there are many spurs present.
  • #1
ggk
2
0
Hi Everyone,

I am analyzing real data using fast-fourier transforms (FFT) in Matlab. The FFT magnitude spectrum show some background noise floor with several sharp spurs popping up high out of the background noise. I need to figure out conclusively which of these spurs are correlated with which other spurs (if any).

To simplify this problem let me just analyze two spurs, to see if they are correlated or not. Let me call them spur1 and spur2. I process the data to obtain three probability distribution functions (PDFs):

1) I isolate spur1 and do an inverse FFT only on spur1 to obtain its' respective real-time waveform (a sinusoid of a certain frequency). I take the PDF of this waveform (PDFspur1).

2) I isolate spur2 and do an inverse FFT only on spur2 to obtain its' respective real-time waveform (a sinusoid of a different frequency). I take the PDF of this waveform (PDFspur2).

3) I isolate both spur1 and spur2 from the rest of the spectrum and take an inverse FFT on the spectrum containing both spur1 and spur2. This results in a real-time waveform whose PDF I'll call PDFspur12.

I want to conclusively determine if spur1 and spur2 are correlated with each other. How do I do it?

One thought I have is, if the distibutions (PFDs) are statistically independent (that is, uncorrelated) then PDFspur12 should EQUAL PDFspur1 CONVOLVED with PDFspur2. If they are NOT equal, then they are not correlated.

I think this is mathematically sound, but I'd appreciate any comments/feedback, especially if you know a better/faster/more conclusive way to determine correlation. -GK
 
Physics news on Phys.org
  • #2
what do you mean by two signals of different frequencies being correlated?
 
  • #3
Correlation meaning the spurs share the same origin (source). For example, a square-wave has odd harmonics. If you see these harmonics in the FFT magnitude spectrum, you can tell they are correlated because each spur has the same phase-angle and they are integer multiples of the fundamental frequency. In my case, however, there are so many spurs that phase-angle and multipler (if it's known, which isn't always known in my case) aren't sufficient to determine correlation.
 

FAQ: How to determine if distributions are correlated?

How do you determine if two distributions are correlated?

To determine if two distributions are correlated, you can use statistical methods such as calculating the correlation coefficient or performing a hypothesis test. The correlation coefficient measures the strength and direction of the relationship between two variables, while a hypothesis test can determine if there is a significant correlation between the two distributions.

What is the significance of a correlation coefficient?

The correlation coefficient, often denoted as r, ranges from -1 to 1. A positive value indicates a positive correlation, meaning that as one distribution increases, the other also tends to increase. A negative value indicates a negative correlation, meaning that as one distribution increases, the other tends to decrease. A correlation coefficient of 0 indicates no correlation between the two distributions.

Can you determine causation from a correlation?

No, correlation does not imply causation. Just because two distributions are correlated does not necessarily mean that one causes the other. It is important to conduct further research and consider other factors before making any causal claims.

How large of a sample size is needed to determine if distributions are correlated?

The larger the sample size, the more accurate the results will be. However, the minimum sample size needed to determine correlation depends on the strength of the relationship, the variability of the data, and the desired level of confidence. A larger sample size is typically needed for weaker correlations or more variable data.

Can you determine correlation with non-numerical data?

Yes, correlation can be determined with non-numerical data using techniques such as rank correlation or contingency tables. However, these methods may not provide as precise results as using numerical data. It is important to consult with a statistician to determine the best approach for analyzing non-numerical data.

Similar threads

Replies
13
Views
2K
Replies
14
Views
3K
Replies
1
Views
2K
Replies
24
Views
4K
Back
Top