How can I prove the Riesz representation theorem?

[Fredrik]

I decided that I wanted to learn how to prove Riesz representation therorem, so I looked it up. It looks simple enough, but I'm confused by one of the details and I'm hoping someone can tell me what I'm doing wrong.

The statement I'd like to prove: If H is a Hilbert space over the complex numbers, and [itex]T\rightarrow \mathbb C[/itex] is linear and bounded, then there's a unique $x_0\in H$ such that $Tx=\langle x_0,x\rangle$ for all x in H.

How I'd like to prove it: Define $M=\ker T=\{x\in H|Tx=0\}$. M is a subspace of H, and so is $M^\perp$, the orthogonal complement of M. If $M=H$, then we obviously have $x_0=0$, so let's focus on the more interesting case $M\neq H$. Let $x\in H$ be arbitrary and choose $y\in M^\perp$ such that $Ty=1$. Note that

$$T(x-(Tx)y)=Tx-(Tx)Ty=0$$

so $x-(Tx)y\in M$. That means that

$$0=\langle y,x-(Tx)y\rangle=\langle y,x\rangle-Tx\|y\|^2$$

$$Tx=\langle\frac{y}{\|y\|^2},x\rangle=\langle x_0,x\rangle$$

where we have defined $x_0=y/\|y\|^2$. We have proved existence. To prove uniqueness, suppose that $\langle x_0',x\rangle=\langle x_0,x\rangle$ for all $x\in H$. This implies $\langle x_0'-x_0,x\rangle$, but the only vector that's orthogonal to all vectors in H is 0, so $x_0'=x_0$.

The problem: Doesn't the first part of the above prove that x₀ isn't unique? I mean, it's not hard to find a y in the orthogonal complement of M such that Ty=1. For any z in that subspace, we have T(z/T(z))=1, so the x₀ we end up with can "point in any direction" of the orthogonal complement of ker T. What's wrong here?

 

[Hurkyl]

so the x₀ we end up with can "point in any direction" of the orthogonal complement of ker T. What's wrong here?

How many directions are there in the orthogonal complement? Consider finite dimensional H for a moment; you know how to compute the dimension of everything involved.

 

[Fredrik]

Let $H=\mathbb C^3$, and let T be defined by the matrix (a b 0), where a and b are complex numbers. Now ker T is the 1-dimensional subspace that consists of multiples of (0 0 1), and the orthogonal complement of ker T is 2-dimensional. It needs to be 1-dimensional for my problem to go away.

 

[Hurkyl]

You sure the kernel is 1-dimensional?

 

[Fredrik]

0=(a b 0)(x y z)^T=ax+by.

D'oh, it's 2-dimensional. (Basis vectors (0 0 1) and (b -a 0)). So the problem goes away in this case. And I see that dim(ker T)=2 when T=(a 0 0) too. I don't have time to think about the general case now, but I'll do it tomorrow. It looks non-trivial, at least when we allow infinite-dimensional H.

 

[Hurkyl]

Right. The finite dimensional case turns out to be easy, because of the rank-nullity theorem. (Or, by the fact we already know how to write every linear functional as a transposed vector -- we just have to make the right tweak to insert the inner product)

The infinite dimensional case is going to be non-trivial, because we know in general the dual space is not isomorphic to the space -- the fact you're using the continuous dual, and working with a Hilbert space must be used somehow.

 

[maze]

You can use Reisz's lemma to construct a Cauchy sequence of unit vectors xi "nearly perpendicular" to M, in the sense that d(xi,M) -> 1. Then verify that the limit of this sequence is actually perpendicular to M.

 

[Fredrik]

Maze, I don't see how that helps. What we need to prove is that $\dim M^\perp=1$ even when H is infinite-dimensional. (At least it seems that way to me).

Hurkyl, are you sure I'm not just overlooking something very simple? I checked two other books, and the proof is essentially the same in all three. No one even mentions the complication that bothers me.

 

[Hurkyl]

I think your calculation is a convincing argument that the dimension of M-perp is no greater that 1 -- so the problem must be how to prove the dimension of M-perp is not zero.

For example, let's analyse a case where we know Riesz's theorem doesn't apply!

Let V be the set of all real sequences with only finitely many nonzero entries, with the obvious inner product

Let T be the summation functional

Then the kernel of T is the set of all real sequences with only finitely many nonzero entries that sum to zero.

I think it's easy to see that the orthogonal complement of ker T is zero; for every vector v in V, you can find an element of ker T that has a nonzero inner product with v.

 

[Fredrik]

I think your calculation is a convincing argument that the dimension of M-perp is no greater that 1

Yes, of course. The first part of the calculation shows that there's an $x_0$ for each "direction" in $M^\perp$, and the next part shows that there can be at most one $x_0$. So there can only be one "direction", i.e. we must have $\dim M^\perp=1$. I actually thought that that was the solution for a while a couple of days ago, and then I dismissed it. Now I can't even remember why. I must have made some dumb mistake.

Thanks for the help. I appreciate it.

so the problem must be how to prove the dimension of M-perp is not zero.

All the books cover that part right before they state and prove this theorem.

 

[maze]

Maze, I don't see how that helps. What we need to prove is that $\dim M^\perp=1$ even when H is infinite-dimensional. (At least it seems that way to me).

Do you know the lemma that for any x in the Banach space and for any $v \notin M$ there exist unique $u \in M$ and scalar c such that x = u + cv?

 

[Fredrik]

Do you know the lemma that for any x in the Banach space and for any $v \notin M$ there exist unique $u \in M$ and scalar c such that x = u + cv?

No, I'm not familiar with that. I looked up Riesz lemma at Wikipidia and found one that said (roughly) that if every point on the unit sphere of H is "close to" a point in a subspace, then that subspace is dense in H.

 

[maze]

No, I'm not familiar with that. I looked up Riesz lemma at Wikipidia and found one that said (roughly) that if every point on the unit sphere of H is "close to" a point in a subspace, then that subspace is dense in H.

There are a couple lemmas at play here. Reisz's lemma, and the other one about the unique expression x=u+cv under some technical conditions (I don't think it has a name).

For Reisz's lemma, the interpretation on wikipedia could be a little misleading if you haven't seen it before. Reisz's lemma is all about finding vectors that are nearly "perpendicular" to a subspace. It also applies to all Banach spaces, not just Hilbert spaces.

Now, in a Banach space there is not necessairily an inner product (only a norm). Thus technically it is not proper to talk about vectors being "perpendicular". However, we can do almost as good by just using the norm. If M is a subspace, look for unit vectors x such that the minimum distance from x to any point in M is 1. ie: length 1 vectors that are distance 1 away from M.

Reisz's lemma doesn't say you can find such a vector, but it says you can get arbitrarily close. For any $\epsilon$ you can find a vector x such that $1-d(x,M) < \epsilon$ and $||x||=1$ where d(x,M) is the infimum distance from x to any point in M ($d(x,M) = inf_{m \in M} ||x-m||$).

Reisz's lemma is pretty fun to prove, you might want to give it a shot. Hint: draw diagrams for it in R2. The other lemma about the unique expression x=u+cv is also pretty good to prove. It requires the condition that the infimum d(x,M) is actually achieved (there exist w in M such that ||w-x|| = d(x,M)). Hint: use proof by contradiction.

Reisz's representation theorem is basically a more powerful version of Reisz's lemma on more restrictive Hilbert spaces. Whereas in Banach spaces you could only get arbitrarily close to finding the "perpendicular" vector, in Hilbert spaces you can actually find one, and it is truly perpendicular in the sense of the inner product.