Endpoint Detection: Distinguishing Peaks from Noise

  • Thread starter inadaze
  • Start date
  • Tags
    Detection
In summary: The "mean" in this context is the average value of the data in the bins. The "standard deviation" is a measure of the spread in the data. Since most of the bins contain nothing but noise and have roughly the same value, the data generally has a small spread. You would add the "standard deviation" to the "mean" to get the "peak value".
  • #1
inadaze
20
0
Hi all,
I am trying to write a code that can determine peaks in a signal. What I am doing is recording a signal then running an fft to calculate the fundamental frequency and the overtones (consonant and dissonant harmonics). The problem is that some of the peaks are very small but are always distinguishable by looking at the plotted results. Has anyone had any experience with writing a code that could distinguish peaks from noise.

Thanks
Jay
 
Computer science news on Phys.org
  • #2
Compute the mean and standard deviation of the values in the bins. Then it should be easy to recognize peaks that are say, two standard deviations greater than the mean.

- Warren
 
  • #3
Thanks for your reply, but unfortunately I am not that educated in these terms. Could you really spell it out for me...I don't know what is the "mean" and "standard deviation(although I can guess at this)" of values in the "bins".
Sorry to make you elaborate, but I am kinda new to this stuff.

Thanks
Jay
 
  • #4
When you take an FFT, the resulting array of numbers is often referred to as an array of "bins," since each number actually indicates the power in a small range of frequencies. You can just call them numbers if you like.

The arithmetic mean of the array is simply the average value. Add all the bins and divide by the number of bins to get the mean. I will denote the values in the bins as x, and the mean of those values as [itex]<x>[/itex].

The standard deviation is a measure of the spread in the data. Since most of the bins contain nothing but noise and have roughly the same value, the data generally has a small spread. Calculate the standard deviation ([itex]\sigma[/itex]) as

[tex]\sigma = \sqrt{<x^2> - <x>^2}[/tex]

In words, take the mean of the squares of all the bins, subtract the square of the mean of the bins, and take the square root of it all. The resulting number represents how much a typical bin differs from the mean. Any bin that contains significantly more power than the mean plus this typical difference -- say, a bin whose value is several standard deviations above the mean -- is probably a peak.

- Warren
 
  • #5
When I take an FFT I get a mirror reflection opposite the peaks. Do I take these as well when calculating the mean?
Also, when I calculate the standard deviation it gave me this number:
0 +5.8949e+004i
What does this mean?
do I add this to the mean to find out what is considered a peak?
Or did I do something wrong?


Thanks
Jay
 

FAQ: Endpoint Detection: Distinguishing Peaks from Noise

What is endpoint detection and why is it important in scientific research?

Endpoint detection is a technique used to distinguish true data peaks from background noise in scientific measurements. It is important because it allows researchers to accurately identify and analyze meaningful data points, while filtering out irrelevant or erroneous data that could skew results.

How does endpoint detection work?

Endpoint detection uses mathematical algorithms to analyze data and identify peaks. These algorithms look for patterns in the data, such as sudden changes in amplitude or frequency, to determine which data points are peaks and which are noise.

What are some common challenges in endpoint detection?

Some common challenges in endpoint detection include distinguishing between true peaks and noise, dealing with overlapping peaks, and identifying peaks in noisy or low-quality data. These challenges can be addressed by using advanced algorithms and techniques, as well as careful calibration and validation of data.

How can endpoint detection improve the accuracy and reliability of scientific results?

By filtering out noise and identifying only the most relevant data points, endpoint detection can help researchers obtain more accurate and reliable results. This is especially important in fields such as medicine and environmental science, where small variations in data can have significant implications.

Are there different types of endpoint detection methods?

Yes, there are several different types of endpoint detection methods, including threshold-based methods, derivative-based methods, and wavelet-based methods. Each method has its own strengths and limitations, and the most suitable method will depend on the specific research context and data being analyzed.

Back
Top