Good kernels, convergence, and more

  • Thread starter stripes
  • Start date
  • Tags
    Convergence
In summary, the instructor asked that we start by writing F(x) = ∫ from -π to x of f(t)dt. Lastly, we should not require the use of uniform continuity.
  • #1
stripes
266
0

Homework Statement



Hi all, I am back with more questions. Thank you to those helped with my last assignment.

Question 1:

For |x| ≤ π, define a sequence of functions by:

Kn(x) = {n if -π/n ≤ x ≤ π/n, 0 otherwise} for natural numbers n. An earlier part of the question asked that I extend the function periodically. The question I need help on asks:

If f(x) is a Riemann integrable periodic function? R(T)?, then the limit as n approaches infinity, of the convolution (f * Kn)(x) = f(x) whenever f is continuous at x. (f * Kn)(x) denotes the convolution as a function of (x), not multiplied by (x).

We are to prove this statement directly using the Fundamental Theorem of Calculus.

Question 2:

Use the integral test for series to prove that Ʃ from k=1..n of 1/k is ≥ ln(n + 1) for all n≥1.

Homework Equations





The Attempt at a Solution



FIRST QUESTION: Instructor asked that we start by writing F(x) = ∫ from -π to x of f(t)dt. Lastly, we should not require the use of uniform continuity.

I seriously don't know where to start. I have written that much, and I've written out the convolution as an integral of f(y)Kn(x-y) dy, but I don't know where to go from there.

SECOND QUESTION: I have tried a few things: integral of 1/x from k to k+1 is ln|k+1| + ln|k| = ln|k+1|, which is good to see. Intuitively, I can understand this question, but I don't know what to write down or how to do it. The integral test tells us if the integral is finite, then the series converges. It does not state any equalities or inequalities; it is merely a test for convergence. I see how the integral test will help, but I don't know how to formally relate the test and the sum/inequality, since the integral test does not say anything about equality.

Any help would be much appreciated!
 
Physics news on Phys.org
  • #2
stripes said:

Homework Statement



Hi all, I am back with more questions. Thank you to those helped with my last assignment.

Question 1:

For |x| ≤ π, define a sequence of functions by:

Kn(x) = {n if -π/n ≤ x ≤ π/n, 0 otherwise} for natural numbers n. An earlier part of the question asked that I extend the function periodically. The question I need help on asks:

If f(x) is a Riemann integrable periodic function? R(T)?, then the limit as n approaches infinity, of the convolution (f * Kn)(x) = f(x) whenever f is continuous at x. (f * Kn)(x) denotes the convolution as a function of (x), not multiplied by (x).
Hi stripes, this looks like an interesting problem and I'll be happy to help if I can. However, you did not indicate how f(x) and R(T) are defined. Also, a typesetting request: at least in my browser, the symbol you are using for pi (π) looks almost indistinguishable from lower-case N (n). For pi, can you please use # #\pi# # (with the spaces removed) instead? Then it will look like this: ##\pi##
 
  • #3
Yeah, I couldn't find the pi symbol. I'll do that from now on.

I don't think we're given any function for f in particular. When I say R(T), I think the question means periodic and Riemann integrable. So the statement is true for any periodic, Riemann integrable function.
 
  • #4
OK, here is my interpretation of question 1:

Let
$$K_n(x) = \begin{cases}
n & \textrm{ if } -\pi/n \leq x \leq \pi/n \\
0 & \textrm{ otherwise }\end{cases}$$
and suppose ##f## is a Riemann integrable function with period ##T##. Prove that
$$\lim_{n \rightarrow \infty} f*K_n(x) = f(x)$$
whenever ##f## is continuous at ##x##.

Assuming that is correct, I would start by writing out the definition of the convolution:
$$f*K_n(x) = \int_{-\infty}^{\infty}K_n(y) f(x-y) dy$$
and then substituting the definition of ##K_n##:
$$f*K_n(x) = \int_{-\pi/n}^{\pi/n}n f(x-y) dy$$
Now the integral is taken over the interval ##[-\pi/n, \pi/n]##, so we are evaluating ##f## at values in the interval ## [x-\pi/n, x+\pi/n]##. This seems like the right place to consider the continuity of ##f## at ##x##. Intuitively, we want to choose ##n## small enough that ##f## cannot vary much over the interval ## [x-\pi/n, x+\pi/n]##.
 
  • #5
By the way, I wonder if there is a factor missing somewhere. As written above, if I integrate ##K_n## over the interval ##[-\pi/n, \pi/n]##, I get ##2\pi##, but I think this will only work if ##K_n## integrates to ##1##.
 
  • #6
For the second question, write out ln(n+1) as an integral of 1/x, taken over an appropriate interval. Then consider your sum as a sum of rectangles, each having width = 1 and height = 1/k. Compare the area of this set of rectangles with the area under the curve 1/x.
 
  • #7
Hi jbunniii, you are correct when you say the area is 2pi. I have previously proven that this sequence of functions is a good kernel, and one condition that must be met to be a good kernel is

[itex]\frac{1}{2\pi} \int ^{\pi}_{- \pi} K_{n} (x) dx = 1.[/itex]

At least I think so.

So I wrote out the convolution integral, but wouldn't it be

[itex]f*K_n(x) = \int_{-\infty}^{\infty}f(x) K_n(x-y) dy?[/itex]

I wrote it out as such and then started using the FTC, then took the limit of both sides as n goes to infinity. I manipulate both sides over and over, and I get f(x) on one side, but I don't get the convolution on the other. So how would I utilize the FTC here? Trying what you suggested doesn't seem like it would involve the FTC, unless I am misunderstanding you, which is likely.

As for the 2nd question, it seems clearer now. I don't have time to attempt it right now, but I will do so tomorrow after my class and hopefully it will be smooth sailing. I have another question on this assignment but it may be too much to do at this point. I will post it tomorrow. I sure hope I have enough time before I have to hand it in, which is in about 12 hours. I am just too tired and I need to sleep right now. Thanks again.
 
  • #8
stripes said:
Hi jbunniii, you are correct when you say the area is 2pi. I have previously proven that this sequence of functions is a good kernel, and one condition that must be met to be a good kernel is

[itex]\frac{1}{2\pi} \int ^{\pi}_{- \pi} K_{n} (x) dx = 1.[/itex]

At least I think so.
OK, there's probably a ##2\pi## factor somewhere else that compensates for it. This sort of thing happens all the time in Fourier analysis because different authors/lecturers use different definitions of various operations, which causes scale factor differences here and there.

So I wrote out the convolution integral, but wouldn't it be

[itex]f*K_n(x) = \int_{-\infty}^{\infty}f(x) K_n(x-y) dy?[/itex]
The nice thing about convolution is that it's commutative:
$$f*K_n(x) = K_n*f(x)$$
So either integral will give you the same answer:
$$\int_{-\infty}^{\infty}f(y) K_n(x-y) dy = \int_{-\infty}^{\infty} K_n(y)f(x-y) dy$$
(By the way, your integral is neither of these: you should have f(y), not f(x). What you wrote isn't a convolution. Just a typo, I assume.)
Often one of the forms is more convenient because it results in nicer integration limits and/or a more pleasant integrand. Choose whichever is more convenient for the problem. I chose the second form because then the integral is simply taken over ##[-\pi/n,\pi/n]##.

I wrote it out as such and then started using the FTC, then took the limit of both sides as n goes to infinity. I manipulate both sides over and over, and I get f(x) on one side, but I don't get the convolution on the other. So how would I utilize the FTC here? Trying what you suggested doesn't seem like it would involve the FTC, unless I am misunderstanding you, which is likely.
You're right, I don't see why the FTC is needed here, at least not directly. I guess your lecturer has something else in mind.

I will briefly sketch my proof; perhaps you can borrow some of these ideas for an FTC-oriented proof.

Let ##\epsilon > 0##. Since ##f## is continuous at ##x##, there exists ##\delta > 0## such that ##|f(x) - f(y)| < \epsilon## whenever ##|x-y| < \delta##. Choose ##n## large enough so that ##\pi/n < \delta##. Then
##f*K_n(x) = \int_{-\pi/n}^{\pi/n}n f(x-y) dy##
Within the limits of integration, we have ##|y| \leq \pi/n < \delta## so we have ##f(x) - \epsilon < f(x-y) < f(x) + \epsilon##. Therefore,
$$2\pi(f(x) - \epsilon) < f*K_n(x) < 2\pi(f(x) + \epsilon)$$
We would be done except for that pesky ##2\pi## factor. See if you can work out how to make it go away. Your instructor isn't doing something strange like defining the convolution integral to include a ##1/(2\pi)## factor, is he?
 
Last edited:
  • #9
jbunniii said:
We would be done except for that pesky ##2\pi## factor. See if you can work out how to make it go away. Your instructor isn't doing something strange like defining the convolution integral to include a ##1/(2\pi)## factor, is he?
Actually, I bet that's exactly what he has done. I just checked Stein and Shakarchi's book Fourier Analysis: An Introduction; they have the same ##2\pi## factor in their "good kernel" definition, and they define the convolution as
$$f*K_n(x) = \frac{1}{2\pi}\int_{-\pi}^{\pi} f(x-y) K_n(y) dy$$
They are dealing with ##2\pi##-periodic functions in that chapter, or equivalently, functions defined on the unit circle. Now that I look back over your original post, I guess that's what R(T) means: Riemann integrable and ##2\pi##-periodic. So all my integrals I wrote earlier should have been taken over ##[-\pi,\pi]## instead of ##(-\infty,\infty)##, and we can mentally insert the ##1/(2\pi)## factor on the convolution. The rest will still be valid.
 
Last edited:
  • #10
I figured out my other question (the Integral Test one), thanks for your advice there. Seemed pretty straightforward.

Regarding the other one, your proof is very clear, and I started the question off like that originally, but then I remembered I had to use the FTC. I'm still trying to pick things out of it to apply to the FTC. I will see if I can ask my instructor as well.

You must be psychic; I am using the book Fourier Analysis: An Introduction. That factor of 2pi is now gone, and I can clearly see the limit equals the function. Still plugging away at it...

Thanks again.
 
  • #11
jbunniii said:
The nice thing about convolution is that it's commutative:
$$f*K_n(x) = K_n*f(x)$$

I feel very stupid, because I actually proved this in a previous question in the assignment!
 

FAQ: Good kernels, convergence, and more

1. What is a kernel in statistics and machine learning?

A kernel is a mathematical function that transforms data points into a higher dimensional space. It is commonly used in support vector machines (SVMs) to find a linear decision boundary between classes that may not be linearly separable in the original feature space.

2. How do you determine if a kernel is a "good" kernel?

A "good" kernel is one that accurately captures the underlying patterns and structure in the data, leading to better classification or regression results. This can be determined through cross-validation or by comparing the performance of different kernels on a given dataset.

3. What is the difference between a parametric and a non-parametric kernel?

A parametric kernel has a fixed number of parameters that do not change with the size of the dataset, while a non-parametric kernel can adapt to the complexity of the data. Non-parametric kernels are often more flexible and can better capture complex relationships, but they may also be more prone to overfitting.

4. How do you measure the convergence of a kernel method?

The convergence of a kernel method can be measured by looking at the rate at which the algorithm converges to a solution, as well as the stability of the solution. This can be evaluated by monitoring the training error and the validation error over multiple iterations of the algorithm.

5. Can kernels be used for unsupervised learning?

Yes, kernels can also be used for unsupervised learning tasks such as clustering and dimensionality reduction. In these cases, the kernel function is used to measure the similarity between data points instead of being used as a decision boundary. Popular unsupervised kernel methods include kernel PCA and kernel k-means.

Similar threads

Back
Top