Correlation operation as measure of similarity between two functions

[HJ Farnsworth]

Hello,

I recently started going through some lecture notes on linear systems and Fourier optics. (By the way, I just started with these, but so far the lecture notes are excellent. If anyone is looking to learn the subject but doesn't want to spend money on a textbook, the lecture notes, and the videos of the lectures, can be found at https://itunes.apple.com/us/itunes-u/opti512r-linear-systems-fourier/id413140966?mt=10).

In lecture 5, it says,

A very important operation that appears in many aspects of physics is the correlation operation. Correlation is a measure of similarity between two functions, and is defined (for complex functions) as

$\gamma _{fg}(x)=\int _{-\infty}^{\infty} f(\alpha)g^{*}(\alpha -x)d\alpha.$

Does anyone know, in what sense is this integral a measure of the similarity between $f$ and $g$?

I'm looking for both conceptual answers, i.e., an intuitive sense of why this integral corresponds to similarity, and quantitative answers, i.e., if the integral evaluates to this, $f$ and $g$ are "similar", but if it evaluates to that, then $f$ and $g$ are "dissimilar".

Thanks very much.

-HJ Farnsworth

 

[micromass]

Let us first consider $x=0$ and let us make all functions positive valued, then we are dealing with the quantity

$$\int_{-\infty}^{+\infty} f(\alpha)g(\alpha)d\alpha$$

This formula can be used to see how much the functions $g$ and $f$ match. Indeed, this integral will be maximized if $f=g$. In a formal sense, we have

$$\int_{-\infty}^{+\infty} f(\alpha)g(\alpha)d\alpha\leq \int_{-\infty}^{+\infty} \max\{f(\alpha),g(\alpha)\} d\alpha$$

and equality holds if $f=g$. So if we get a "high" value for this, then the functions match a lot, otherwise, they don't match a lot.

But now let's look at $\sin$ and $\cos$. They certainly are not the same functions, but they are similar. In fact, they are the same function up to a phase factor. Indeed,

$$\sin(\alpha) = \cos(\alpha - \pi/2)$$

The cross-correlation function is used to deal with this case. So by definition, we have

$$(f\ast g)(x) = \int_{-\infty}^{+\infty} f(\alpha)g(\alpha - x)d\alpha$$

So for any phase constant, what this does is translate $g$ by that phase constant and then measure the similarity of the functions.

As such, we get several values, one for each $x$. So we get the function $(f\ast g)$. High values of this function indicate that $f$ and the translated $g$ match a lot, low values that they don't match a lot. So the cross-correlation can be used to indicate by how much we should translate a function for them to match.

 

[HJ Farnsworth]

Thanks for the reply, micromass!

I understand - like functions will be in the same quadrant of the complex plane (or even better, "point" in the same direction in the complex plane) at the same time, or restricting ourselves to real functions, like functions will be positive or negative at the same time. This results in (ignoring the $-x$ for the moment) higher values of the given integral, since $f\times g^{*}$ will be positive if $\mathrm{Re}(f)/\mathrm{Im}(f) = \mathrm{Re}(g)/\mathrm{Im}(g)$. Unlike functions won't be expected to point in the same direction in the complex plane at the same time, resulting in lower values for the given integral. Throwing the $-x$ back in there, we have a variable that can be adjusted to increase or decrease the similarity.

So, similarity in this sense means "degree to which the two functions point in the same direction in the complex plane".

Thanks again.

-HJ Farnsworth