Approximation theorem of Weierstrass

[evinda]

Hello! (Wave)

I want to prove that each continuous function $f$ in a closed and bounded interval $[a,b]$ can be approximated uniformly with polynomials, as good as we want, i.e. for a given positive $\epsilon$, there is a polynomial $p$ such that

$$\max_{a \leq x \leq b} |f(x)-p(x)|< \epsilon.$$

Firstly, we should make sure that we can assume without loss of generality that the interval $[a,b]$ is contained in the open interval $(-\pi,\pi)$.

But why can we assume this? (Thinking)

 

[I like Serena]

Hello! (Wave)

I want to prove that each continuous function $f$ in a closed and bounded interval $[a,b]$ can be approximated uniformly with polynomials, as good as we want, i.e. for a given positive $\epsilon$, there is a polynomial $p$ such that

$$\max_{a \leq x \leq b} |f(x)-p(x)|< \epsilon.$$

Firstly, we should make sure that we can assume without loss of generality that the interval $[a,b]$ is contained in the open interval $(-\pi,\pi)$.

But why can we assume this? (Thinking)

Hey evinda! (Wave)

Suppose $[a,b]$ is not contained in the open interval $(-\pi,\pi)$.
Then we can define $g: x\mapsto f(cx+d)$ such that the domain of $g$ is a closed interval $J$ inside $(-\pi,\pi)$ can't we?
If $f$ is continuous on $[a,b]$, then $g$ is continuous on $J$ as well.

So if the condition holds for a closed interval in $(-\pi,\pi)$, it holds for this $g$.
That is, we have:
$$\max_{y\in J} |g(y)-q(y)|< \epsilon$$
where $q$ is a polynomial.
Now define $p(x)=q(\frac{x-d}c)$, which is again a polynomial isn't it?
It follows that:
$$\max_{a \leq x \leq b} |f(x)-q(\frac{x-d}c)| = \max_{a \leq x \leq b} |f(x)-p(x)|< \epsilon$$
(Thinking)

 

[evinda]

If $f$ is continuous on $[a,b]$, then $g$ is continuous on $J$ as well.

How does this follow? (Thinking)

Now define $p(x)=q(\frac{x-d}c)$, which is again a polynomial isn't it?

$q(\frac{x-d}c)$ is a polynomial, because $q(x)$ is a polynomial, and substituting $\frac{x-d}c$ at $q$ we do get positive integer powers of $x$ ? (Thinking)

It follows that:
$$\max_{a \leq x \leq b} |f(x)-q(\frac{x-d}c)| = \max_{a \leq x \leq b} |f(x)-p(x)|< \epsilon$$
(Thinking)

How does this follow? (Worried)

 

[I like Serena]

How does this follow?

Don't we have:
$$\lim_{y\to y_0} g(y) = \lim_{x\to x_0} f(cx+d) = f(cx_0+d)=g(y_0)$$
for an appropriate $x_0$? (Wondering)

$q(\frac{x-d}c)$ is a polynomial, because $q(x)$ is a polynomial, and substituting $\frac{x-d}c$ at $q$ we do get positive integer powers of $x$ ?

Yep. (Nod)

How does this follow?

Don't we have:
$$\max_{y\in J}|g(y)-q(y)|<\epsilon \quad\Rightarrow\quad
\max_{y\in J} |f(cy+d)-q(y)| = \max_{x\in[a,b]} |f(x)-q\left(\frac{x-d}c\right)|=\max_{x\in[a,b]} |f(x)-p(x)| < \epsilon
$$
(Wondering)

 

[evinda]

Don't we have:
$$\lim_{y\to y_0} g(y) = \lim_{x\to x_0} f(cx+d) = f(cx_0+d)=g(y_0)$$
for an appropriate $x_0$? (Wondering)

Doesn't it hold that $\lim_{x\to x_0} g(x) = \lim_{x\to x_0} f(cx+d) = f(cx_0+d)=g(x_0)$ for some $x_0$ ? Or am I wrong? (Thinking)

Don't we have:
$$\max_{y\in J}|g(y)-q(y)|<\epsilon \quad\Rightarrow\quad
\max_{y\in J} |f(cy+d)-q(y)| = \max_{x\in[a,b]} |f(x)-q\left(\frac{x-d}c\right)|=\max_{x\in[a,b]} |f(x)-p(x)| < \epsilon
$$
(Wondering)

So, in total, we show that given any function in some interval $[a,b]$, we can define a function the domain of which is in the interval $(-\pi, \pi)$.

And if the condition is satified for continuous functions in a closed interval in $(-\pi,\pi)$, then the condition holds also for functions in the arbitrary interval $[a,b]$.

And for this reason, we can assume that $[a,b]$ is contained in $(-\pi,\pi)$. Right?

 

[I like Serena]

Doesn't it hold that $\lim_{x\to x_0} g(x) = \lim_{x\to x_0} f(cx+d) = f(cx_0+d)=g(x_0)$ for some $x_0$ ? Or am I wrong?

So, in total, we show that given any function in some interval $[a,b]$, we can define a function the domain of which is in the interval $(-\pi, \pi)$.

And if the condition is satisfied for continuous functions in a closed interval in $(-\pi,\pi)$, then the condition holds also for functions in the arbitrary interval $[a,b]$.

And for this reason, we can assume that $[a,b]$ is contained in $(-\pi,\pi)$. Right?

All correct. (Happy)

 

[evinda]

All correct. (Happy)

Nice... (Happy)

Then it says that since $f$ is continuous in a closed and bounded interval, it is uniformly continuous in $[a,b]$. Consequently, there is a positive $\delta>0$ such that

$|f(x)-p(x)|< \frac{\epsilon}{3} \ \ \forall x,y \in [a,b], |x-y|<\delta$.

How does this follow?

Since $f$ is uniformly continuous, we have that $\forall \epsilon>0, \ \exists \delta>0$ such that when $|x-y|<\delta$ it follows that $|f(x)-f(y)|<\epsilon$. Right?

Do we use somehow the fact that $\max_{a \leq x \leq b}|f(x)-p(x)|< \epsilon$ ?

 

[I like Serena]

Nice... (Happy)

Then it says that since $f$ is continuous in a closed and bounded interval, it is uniformly continuous in $[a,b]$. Consequently, there is a positive $\delta>0$ such that

$|f(x)-p(x)|< \frac{\epsilon}{3} \ \ \forall x,y \in [a,b], |x-y|<\delta$.

How does this follow?

Since $f$ is uniformly continuous, we have that $\forall \epsilon>0, \ \exists \delta>0$ such that when $|x-y|<\delta$ it follows that $|f(x)-f(y)|<\epsilon$. Right?

It's not clear to me either. There seems to be something missing that has perhaps been proven earlier. (Wondering)

Do we use somehow the fact that $\max_{a \leq x \leq b}|f(x)-p(x)|< \epsilon$ ?

Isn't that what we want to prove?
Then we can't use it. (Shake)

 

[evinda]

It's not clear to me either. There seems to be something missing that has perhaps been proven earlier. (Wondering)
Isn't that what we want to prove?
Then we can't use it. (Shake)

Ok... (Thinking)

Then we choose a natural number $n$ such that $h:=\frac{b-a}{n}< \delta$, and we consider the uniform partition of the interval $[a,b]$ with step $h$, i.e. with nodes $x_i:=a+ih, i=0, \dots, n$. We symbolize with $f_h$ the continuous and piecewise linear function (i.e. polynomial of degree at most 1) that interpolates $f$ at the points $x_0, \dots, x_n$. I want to verify that

$$f_h(x)=f(x_i) \frac{x_{i+1}-x}{h}+f(x_{i+1}) \frac{x-x_i}{h}, x \in [x_i,x_{i+1}],$$

and so

$$f(x)-f_h(x)=[f(x)-f(x_i)] \frac{x_{i+1}-x}{h}+[f(x)-f(x_{i+1})] \frac{x-x_i}{h}, x \in [x_i, x_{i+1}].$$

Then we should get to the result that

$$|f(x)-f_h(x)| \leq |f(x)-f(x_i)| \frac{x_{i+1}-x}{h}+|f(x)-f(x_{i+1})| \frac{x-x_i}{h},$$

so since $h<\delta$,

$$|f(x)-f_h(x)| \leq \frac{\epsilon}{3} \frac{x_{i+1}-x}{h}+\frac{\epsilon}{3} \frac{x-x_i}{h}=\frac{\epsilon}{3}, x \in [x_i, x_{i+1}].$$

Consequently,

$$\max_{a \leq x \leq b} |f(x)-f_h(x)|< \frac{\epsilon}{3}.$$

Do we use this formula in order to verify $f_h$ ?

View attachment 8181

View attachment 8182

 

[I like Serena]

Ok... (Thinking)

Then we choose a natural number $n$ such that $h:=\frac{b-a}{n}< \delta$, and we consider the uniform partition of the interval $[a,b]$ with step $h$, i.e. with nodes $x_i:=a+ih, i=0, \dots, n$. We symbolize with $f_h$ the continuous and piecewise linear function (i.e. polynomial of degree at most 1) that interpolates $f$ at the points $x_0, \dots, x_n$. I want to verify that

$$f_h(x)=f(x_i) \frac{x_{i+1}-x}{h}+f(x_{i+1}) \frac{x-x_i}{h}, x \in [x_i,x_{i+1}],$$

and so

$$f(x)-f_h(x)=[f(x)-f(x_i)] \frac{x_{i+1}-x}{h}+[f(x)-f(x_{i+1})] \frac{x-x_i}{h}, x \in [x_i, x_{i+1}].$$

Then we should get to the result that

$$|f(x)-f_h(x)| \leq |f(x)-f(x_i)| \frac{x_{i+1}-x}{h}+|f(x)-f(x_{i+1})| \frac{x-x_i}{h},$$

so since $h<\delta$,
$$|f(x)-f_h(x)| \leq \frac{\epsilon}{3} \frac{x_{i+1}-x}{h}+\frac{\epsilon}{3} \frac{x-x_i}{h}=\frac{\epsilon}{3}, x \in [x_i, x_{i+1}].$$

There seems to be a missing assumption that $|f(x)-f(x_i)| <\frac \epsilon 3$ for $|x-x_i|<\delta$. (Thinking)

Consequently,

$$\max_{a \leq x \leq b} |f(x)-f_h(x)|< \frac{\epsilon}{3}.$$

Do we use this formula in order to verify $f_h$ ?

https://www.physicsforums.com/attachments/8181

https://www.physicsforums.com/attachments/8182

I don't think we use those formulas.
Instead I think that for a given $\epsilon>0$ we've chosen $n$ to be high enough so that $|f(x)-f(x_i)| <\frac \epsilon 3$ for $x_i \le x \le x_{i+1}$.
That is, for $|x-x_i|\le h < \delta$.

EDIT: It follows from the fact that $f$ is continuous.
That means that for a given $\epsilon>0$ we can find a $\delta>0$ such that $|f(x)-f(x_i)|<\frac\epsilon 3$ if $|x-x_i|<\delta$.

 

[evinda]

Nice... Then we expand $f_h$ in the whole interval $[-\pi,\pi]$ as a continuous function with two properties: To be a polynomial of degree at most 1 in each of the intervals $[-\pi,a)$ and $(b,\pi]$ and to be zero at $-\pi$ and $\pi$. We symbolize the new function again with $f_h$. Now $f_h$ is in $E'$ ($E'$ is the space of piecewise continuous functions $f:[-\pi, \pi]\rightarrow \mathbb{R}$ with finite number of discontinuities.) in the interval $[-\pi,\pi]$ and satisfies the relation $f_h(-\pi)=f_h(\pi)$. Consequently, according to Dirichlet's theorem, the Fourier series $SF_h$ of $f_h$ converges uniformly to $f_h$, in the interval $[-\pi,\pi]$. So, for a sufficiently large $m$, the partial Fourier sum $S_mf_h$ of $f_h$ differs from $f_h$ less than $\frac{\epsilon}{3}$,

$$\max_{-\pi \leq x \leq \pi} |f_h(x)-S_mF_h(x)|< \frac{\epsilon}{3}.$$I have thought the following:

$x \in [-\pi,a)$ $\to$ $f_h(x)$ continuous $\to$ $f_h(x)=f_h(x-)=f_h(x+)$

$x \in (b,\pi]$ $\to$ $f_h(x)$ continuous $\to$ $f_h(x)=f_h(x-)=f_h(x+)$

$x \in [a,b]$ $\to$ $f_h(x)$ continuous $\to$ $f_h(x)=f_h(x-)=f_h(x+)$.

So for $x \in (-\pi,a), (b,\pi), [a,b]$ respectively we have from Dirichlet's theorem that the Fourier series of $f_h$ converges in each of the three intervals to

$$\frac{f_h(x-)+f_h(x+)}{2}=\frac{f_h(x)+f_h(x)}{2}=f_h(x).$$

At the points $x=\pm p$, the series converges to the value $\frac{f_h(\pi-)+f_h((-\pi)+)}{2}=0$.

Thus, the Fourier series $Sf_h$ of $f_h$ converges uniformly to $f_h$ in the interval $[-\pi,\pi]$.

(The converge is uniform since $f(-\pi)=f(\pi)$ and $f' \in E$.)

From the definition of uniform convergence we have that $\forall \epsilon>0$, $\exists M>0$ such that $\forall m>M$:

$$\max_{-\pi \leq x \leq \pi} |f_h(x)-S_mf_h(x)|<\frac{\epsilon}{3}.$$

Am I right? (Thinking)

 

[I like Serena]

Yep. You are right. (Nod)

Btw, doesn't it suffice that we picked the previous $f_h$ to be a continuous and piecewise linear polynomial that we extend with another 2 pieces of linear polynomials that end in zero at $\pm\pi$?
So each of those 3 pieces is continuous and therefore the new $f_h$ is continuous with a finite number of discontinuities, and $f(-\pi)=f(\pi)$.
Thus we can apply Dirichlet's Theorem. (Thinking)

 

[evinda]

Yep. You are right. (Nod)

Btw, doesn't it suffice that we picked the previous $f_h$ to be a continuous and piecewise linear polynomial that we extend with another 2 pieces of linear polynomials that end in zero at $\pm\pi$?
So each of those 3 pieces is continuous and therefore the new $f_h$ is continuous with a finite number of discontinuities, and $f(-\pi)=f(\pi)$.
Thus we can apply Dirichlet's Theorem. (Thinking)

Nice... (Smile)
Now the functions $\cos{(\nu x)}$ and $\sin{(\nu x)}$ are approximated, uniformly in the interval $[-\pi,\pi]$, by polynomials, for example by the corresponding Taylor polynomials around $0$. Consequently, there is a polynomial $p$ such that

$$\max_{-\pi \leq x \leq \pi} |S_m f_h(x)-p(x)|<\frac{\epsilon}{3}.$$

In order to show this, I have thought the following:$$\cos{(\nu x)}=\sum_{i=0}^m \frac{(-1)^i (\nu x)^{2i}}{(2i)!}:=p_1$$

$$\sin{(\nu x)}=\sum_{i=0}^{m} \frac{(-1)^i (\nu x)^{2i+1}}{(2i+1)!}:=p_2$$

$$p:=p_1+p_2$$

$$|S_mf_h-p|=|S_mf_h-p_1-p_2|=\left| \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})-p_1-p_2\right|=\left| \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})-\sum_{j=0}^m \frac{(-1)^j (ix)^{2j}}{(2j)!}-\sum_{j=0}^m \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}\right|$$

Is it right so far? If so, how could we continue? (Thinking)

 

[evinda]

Do we proceed as follows? (Thinking)\begin{align*}|S_mf_h-p|&=|S_mf_h-p_1-p_2|\\ & =\left| \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})-p_1-p_2\right|\\ & =\left| \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})-\sum_{j=0}^m \frac{(-1)^j (ix)^{2j}}{(2j)!}-\sum_{j=0}^m \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}\right|\\ & =\left| \frac{a_0}{2}+ \sum_{i=1}^m a_i \cos{(ix)}+\sum_{i=1}^m b_i \sin{(ix)}-\sum_{j=1}^m \frac{(-1)^j (ix)^{2j}}{(2j)!}-1-\sum_{j=1}^m \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}- ix\right|\\ & = \left| \frac{a_0}{2}-1-ix+ \sum_{i=1}^m \left (a_i \cos{(ix)}-\frac{(-1)^j (ix)^{2j}}{(2j)!}\right )+\sum_{i=1}^m \left (b_i \sin{(ix)}- \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}\right )\right|\\ & \leq \left| \frac{a_0}{2}-1-ix\right |+\left | \sum_{i=1}^m \left (a_i \cos{(ix)}-\frac{(-1)^j (ix)^{2j}}{(2j)!}\right )\right |+\left |\sum_{i=1}^m \left (b_i \sin{(ix)}- \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}\right )\right|\\ & \leq \left| \frac{a_0}{2}-1-ix\right |+ \sum_{i=1}^m \left |a_i \cos{(ix)}-\frac{(-1)^j (ix)^{2j}}{(2j)!}\right |+\sum_{i=1}^m \left |b_i \sin{(ix)}- \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}\right|\end{align*}

 

[I like Serena]

Nice... (Smile)
Now the functions $\cos{(\nu x)}$ and $\sin{(\nu x)}$ are approximated, uniformly in the interval $[-\pi,\pi]$, by polynomials, for example by the corresponding Taylor polynomials around $0$. Consequently, there is a polynomial $p$ such that

$$\max_{-\pi \leq x \leq \pi} |S_m f_h(x)-p(x)|<\frac{\epsilon}{3}.$$

In order to show this, I have thought the following:$$\cos{(\nu x)}=\sum_{i=0}^m \frac{(-1)^i (\nu x)^{2i}}{(2i)!}:=p_1$$

$$\sin{(\nu x)}=\sum_{i=0}^{m} \frac{(-1)^i (\nu x)^{2i+1}}{(2i+1)!}:=p_2$$

$$p:=p_1+p_2$$

$$|S_mf_h-p|=|S_mf_h-p_1-p_2|=\left| \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})-p_1-p_2\right|=\left| \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})-\sum_{j=0}^m \frac{(-1)^j (ix)^{2j}}{(2j)!}-\sum_{j=0}^m \frac{(-1)^j (ix)^{2j+1}}{(2j+1)!}\right|$$

Is it right so far? If so, how could we continue? (Thinking)

Can't the coefficients $a_i$ and $b_i$ be anything?
So I don't think subtracting those fixed $p_1$ and $p_2$ will work, since they do not depend on $a_i$ and $b_i$. (Thinking)

How about instead we expand it like this:
$$S_mf_h= \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})
\approx \frac{a_0}{2}+ \sum_{i=1}^m (a_i p_1(ix)+b_i p_2(ix))
$$
where $p_1$ and $p_2$ are the Taylor expansions of $\cos x$ respectively $\sin x$ up to some number of terms.
Then the right hand side is a polynomial isn't it?
And we can make the difference with the left hand side as small as we want by expanding $\cos$ and $\sin$ far enough, can't we? (Wondering)

 

[evinda]

Can't the coefficients $a_i$ and $b_i$ be anything?
So I don't think subtracting those fixed $p_1$ and $p_2$ will work, since they do not depend on $a_i$ and $b_i$. (Thinking)

How about instead we expand it like this:
$$S_mf_h= \frac{a_0}{2}+ \sum_{i=1}^m (a_i \cos{(ix)}+b_i \sin{(ix)})
\approx \frac{a_0}{2}+ \sum_{i=1}^m (a_i p_1(ix)+b_i p_2(ix))
$$
where $p_1$ and $p_2$ are the Taylor expansions of $\cos x$ respectively $\sin x$ up to some number of terms.
Then the right hand side is a polynomial isn't it?
And we can make the difference with the left hand side as small as we want by expanding $\cos$ and $\sin$ far enough, can't we? (Wondering)

So then we have that $|S_mf_h(x)-p(x)|<\frac{\epsilon}{3}$.

How does it follow that $\max_{-\pi \leq x \leq \pi} |S_mf_h(x)-p(x)|<\frac{\epsilon}{3}$ ? (Thinking)

 

[I like Serena]

So then we have that $|S_mf_h(x)-p(x)|<\frac{\epsilon}{3}$.

How does it follow that $\max_{-\pi \leq x \leq \pi} |S_mf_h(x)-p(x)|<\frac{\epsilon}{3}$ ?

Can't we keep expanding the Taylor polynomials until the difference is less than the desired error everywhere on the interval? (Wondering)

 

[evinda]

Can't we keep expanding the Taylor polynomials until the difference is less than the desired error everywhere on the interval? (Wondering)

Ok, I see.. Thanks a lot! (Smirk)