Help in understanding Eigenvectors please

[Martyn Arthur]

TL;DR Summary

CALCULATING EIGEN VECTORS

Hi; struggling a little with eigenvectors;
I can get to the equation at the foot of the example but I can't understand the "formula" leading to the setting of x = 3 at the foot of the example?
thanks
martyn

 

[Hill]

TL;DR Summary: CALCULATING EIGEN VECTORS

setting of x = 3 at the foot of the example?

The $x$ there can be anything except $0$. Setting $x=3$ simplifies the numbers.

 

[Martyn Arthur]

Thanks, I am having a very senior moment. can you please take me through the steps from lamda = 4?
thanks
martyn

 

[Hill]

Thanks, I am having a very senior moment. can you please take me through the steps from lamda = 4?
thanks
martyn

Which steps are not clear? There are only three.

 

[Martyn Arthur]

Which steps are not clear? There are only three.

I know this is basic; sometimes with ADHD simple things like this get muddled. Can you please take me through each of the steps
thanks

 

[Ibix]

Do you understand how to get $y=2x/3$ in the penultimate line? If so, what that is telling you that any vector $(x\ y)^T$ where y is 2/3 of x is an eigenvector. So $(3\ 2)^T$ or $(48\ 32)^T$. Typically you pick the vector with magnitude 1, but here they've just picked one that has nice round values for thr components.

 

[Martyn Arthur]

Thanks for your patience, its just one of those things that won't equate, if you can just go through the steps showing me one by one that would be really helpful, I have had a problem with just this conversion from day one

 

[Mark44]

Thanks for your patience, its just one of those things that won't equate, if you can just go through the steps showing me one by one that would be really helpful, I have had a problem with just this conversion from day one

The problem doesn't seem to be about conversion, but rather, one of comprehension on just what an eigenvector and its associated eigenvalue are all about.

For a given matrix A, an eigenvalue (usually represented as $\lambda$, the Greek letter lambda -- note the spelling) is a number such that $A\vec x = \lambda \vec x$ for some nonzero vector $\vec x$. There is not just one eigenvector for a given matrix and one of its eigenvalues. For a given eigenvalue, any constant multiple of a known eigenvector will also be an eigenvector.

For the problem in this thread, with the eigenvalue $\lambda = 4$, all eigenvectors <x, y> satisfy the equation $y = \frac 2 3 x$, where x can be any real number except 0.

All of the following are eigenvectors:
<3, 2>
<6, 4>
<1, 2/3>
<-9, -6>
etc.
It might be useful for you to verify for each of the above vectors that $A \vec x = 4 \vec x$.

 

[DaveE]

You can think of this matrix as a linear transformation that takes any input vector $\vec u$ and creates a new vector $\mathbf A \vec u$. There are always some special vectors that are transformed into new vectors that have their direction unchanged. These are the eigenvectors. However, their length is changed by the factor $\lambda$, the eigenvalue. Every eigenvector has an associated eigenvalue. Hence the key formula $\mathbf A \vec v = \lambda \vec v$. A vector that satisfies this equation is, by definition, an eigenvector with it's associated eigenvalue. But since the transformation is linear, we also know that for a scalar constant c, then $\mathbf A (c \vec v) = c \mathbf A \vec v = c\lambda \vec v = \lambda (c \vec v)$. So the only important feature of the eigenvector is its direction. It's really a set of vectors with a fixed direction and any (non-zero) length. That is why once the derivation determines that $\vec v = \begin{bmatrix} x\\ \frac{2x}{3} \end{bmatrix}$, Then you can choose any value of x. Your choice only effects the length of that vector. They chose $x=3$ because it looks pretty.

Try it out. Multiply a few eigenvectors to see the results, it's not too hard in 2 dimensions.

PS: There is a confusing bit about how people often speak of the plurality of "eigenvectors".
$\begin{bmatrix} x\\ \frac{2x}{3} \end{bmatrix}$ is an eigenvector. $\begin{bmatrix} 3\\ 2 \end{bmatrix}$, $\begin{bmatrix} 6\\ 4 \end{bmatrix}$, and $\begin{bmatrix} -27\\ -18 \end{bmatrix}$ are the same eigenvector. As I said before it's a set of vectors.

OTOH, $\begin{bmatrix} x\\ -x \end{bmatrix}$ is the other eigenvector. It could also be $\begin{bmatrix} 1\\ -1 \end{bmatrix}$, $\begin{bmatrix} -1\\ 1 \end{bmatrix}$, etc.

People can be sloppy here and context is important. Normally when we say the eigenvectors we mean the different sets of eigenvectors. So, 2 eigenvectors really means 2 different sets of vectors with each set having a fixed direction for all of its vectors. i.e. 2 directions, not just 2 vectors.

 

[Ibix]

Just to make explicit a step that DaveE's excellent explanation skipped, if $\mathbf{A}\vec v=\lambda \vec v$ then $0=\mathbf{A}\vec v-\lambda \vec v$. We'd like to take the common factor of $\vec v$ out so we can write something like $0=(\mathbf{A}-\lambda)\vec v$. But how? What does it mean to subtract a number from a matrix? There isn't an answer to that question, but we can note that multiplying a vector by the identity matrix, $\mathbf{I}$ does nothing. So we can write $$\begin{eqnarray*}
0&=&\mathbf{A}\vec v-\lambda v\\
&=&\mathbf{A}\vec v-\lambda\mathbf{I}\vec v\\
&=&(\mathbf{A}-\lambda\mathbf{I})\vec v
\end{eqnarray*}$$We know how to multiply a constant by a matrix (multiply each element of the matrix by the constant) and we know how to subtract one matrix from another (subtract the corresponding elements in the two matrices).

That's where the matrix$$\left[\begin{array}{cc}-2&3\\2&-3\end{array}\right]$$comes from. The top left element of $\mathbf{A}$ is 2, the top left element of $\mathbf{I}$ is 1, and $\lambda$ is 4. So the top left element of the matrix is $2-4\times 1=-2$. Multiplying the matrix by the vector $(x\ y)^T$ gives you a vector equation containing the two equations immediately below.

 

[Bosko]

CALCULATING EIGEN VECTORS

Hi; struggling a little with eigenvectors;
I can get to the equation at the foot of the example but I can't understand the "formula" leading to the setting of x = 3 at the foot of the example?
thanks
martyn

All vectors on the strait line $y=-x$ are eigenvectors of A with the same eigenvalue $\lambda=-1$
also
All vectors on the strait line $y=2x/3$ are eigenvectors of A with the same eigenvalue $\lambda=4$

There is no "formula" to pick one vector from each of two strait lines.
You can choose any number for x . As you like .
( That is "THE FORMULA") :-) Sorry, but this is so funny to me ... "freedom of choice formula"
Probably the most important formula in a life.

In the example you posted
$x=1$ is chosen for the first line and eigenvector from the first strait line is (1,-1)
$x=7$ can be chosen for the first line and eigenvector from the first strait line will be (7,-7)
...
$x=3$ is chosen for the second line and eigenvector from the second strait line is (3,2)
$x=-6$ can be chosen for the second line and eigenvector from the second strait line will be (-6,-4)
...

 

[Martyn Arthur]

You have been so helpful, hopefully I am getting there.
As below given x=y can is assign any value to x so that y has a corresponding value?
thanks martyn

 

[Bosko]

Let me do this task from the previous post.

First you need to find eigenvalues.
$det \begin{bmatrix} 2-\lambda&1\\ 4&-1-\lambda \end{bmatrix} =0$

$(2-\lambda)(-1-\lambda)-1 \cdot 4=0$
$(2-\lambda)(-1-\lambda)- 4=0$
$-2-2\lambda+\lambda+\lambda^2-4=0$
$\lambda^2-\lambda-6=0$

I need to find solutions $\lambda_1$ and $\lambda_2$. There are many ways to find them.

[1] Standard formula (copy/paste from Forum > Help > LaTeX)
$\lambda_{1,2}= \frac {-b \pm \sqrt{b^2 -4ac}} {2a}$

[2] You can use Vieta's formulas. What are those two numbers?
When you add them together, you will get a minus coefficient in front of the lambda $\lambda^2+(-1)\lambda-6=0$
$\lambda_1+\lambda_2=-(-1)=1$
When you multiply them, you will get the last number from the left side of the equation $\lambda^2-1\lambda-6=0$
$\lambda_1 \cdot \lambda_2=-6$
Can you guess what those two numbers ( $\lambda_1$ and $\lambda_2$) are?

[3] Complete to a "complete square". Add 1/4 to both sides of the equation

$\lambda^2-\lambda+\frac{1}{4}-6=0+\frac{1}{4}$
$\lambda^2-\lambda+\frac{1}{4}=6+\frac{1}{4}$ ... left side is a "comlete square"

$(\lambda-\frac{1}{2})^2=\frac{24}{4}+\frac{1}{4}$
$(\lambda-\frac{1}{2})^2=\frac{25}{4}$
$\lambda-\frac{1}{2}=\pm \sqrt{\frac{25}{4}}$
$\lambda-\frac{1}{2}=\pm \frac{5}{2}$
$\lambda-\frac{1}{2}=\pm 2\frac{1}{2}$... two and a half

$\lambda_1=+ 2\frac{1}{2}+\frac{1}{2}=3$
$\lambda_2=- 2\frac{1}{2}+\frac{1}{2}=-2$
These are two eigenvalues.

Now I will find eigenvectors
[1] For eigenvalue $\lambda_1=3$ I get

$\begin{bmatrix} 2-3&1\\ 4&-1-3 \end{bmatrix} \begin{bmatrix} x\\ y \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$

$\begin{bmatrix} -1&1\\ 4&-4 \end{bmatrix} \begin{bmatrix} x\\ y \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$

$-x+y=0$
$4x-4y=0$
These two equations are equivalent (have the same set of solutions). If you multiply the first by -4 you will get the second. There is no need to solve them both. It is enough to solve only one.
$y=x$
The solution set is a straight line with infinitely many eigenvectors for the same eigenvalue $\lambda_1=3$. I need to choose one.
For x=42 I get eigenvector $v_1=(42,42)$
For x=1 I get eigenvector $v_1=(1,1)$
...

[2] For eigenvalue $\lambda_2=-2$ I get

$\begin{bmatrix} 2-(-2)&1\\ 4&-1-(-2) \end{bmatrix} \begin{bmatrix} x\\ y \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$

$\begin{bmatrix} 4&1\\ 4&1 \end{bmatrix} \begin{bmatrix} x\\ y \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$

$4x+y=0$
$4x+y=0$
These two equations are equivalent (have the same set of solutions) because they are the same .:-)
$y=-4x$
The solution set is a straight line with infinitely many eigenvectors. I need to choose one.
For x=1 I get eigenvector $v_2=(1,-4)$
That is it.

 

[Martyn Arthur]

You have been so helpful, hopefully I am getting there.
As attacd given x=y can is assign any value to x so that y has a corresponding value?
thanks martyn

View attachment 338417

 

[Martyn Arthur]

You guys are so incredibly helpful; with I could buy you all a pint

 

[Bosko]

You have been so helpful, hopefully I am getting there.
As attacd given x=y can is assign any value to x so that y has a corresponding value?
thanks martyn

View attachment 338417

You're welcome
Yes, you can assign any value to x (-13.5 , 1, 77,...) except of course 0.

This tread seems to me to fit better under "Math Homework Help"

 

[Mark44]

There are always some special vectors that are transformed into new vectors that have their direction unchanged.

If $\vec x$ is an eigenvector, then $-\vec x$ is also an eigenvector. You need to include vectors that have the opposite direction.

PS: There is a confusing bit about how people often speak of the plurality of "eigenvectors".$\begin{bmatrix} x\\ \frac{2x}{3} \end{bmatrix}$ is an eigenvector.
$\begin{bmatrix} 3\\ 2 \end{bmatrix}$, $\begin{bmatrix} 6\\ 4 \end{bmatrix}$, and $\begin{bmatrix} -27\\ -18 \end{bmatrix}$ are the same eigenvector.

I disagree about them being the same eigenvector. All of the vectors you show are eigenvectors of the same eigenvalue, but they are different vectors. For a given eigenvalue, there are an infinite number of eigenvectors, all of which are (nonzero) scalar multiples of one another. The collection of such vectors constitutes the eigenspace for the given eigenvalue.

 

[DaveE]

If $\vec x$ is an eigenvector, then $-\vec x$ is also an eigenvector. You need to include vectors that have the opposite direction.

Which is why I did include those in my examples. Fortunately, we don't have to write a thesis to answer on PF, or get past "reviewer #3". Thanks for the clarification, now we can all be on the same page.

I disagree about them being the same eigenvector. All of the vectors you show are eigenvectors of the same eigenvalue, but they are different vectors. For a given eigenvalue, there are an infinite number of eigenvectors, all of which are (nonzero) scalar multiples of one another. The collection of such vectors constitutes the eigenspace for the given eigenvalue.

OK, you win. I do like "eigenspace" better that the "set of vectors" that I used. Although I'm not sure that's best for a beginner that is having a hard time with the basics. Personally, I prefer an epistemic approach where you first describe things in an approximate way, often with analogies, before applying rigorously correct definitions.

This is a perfect example of my point that out in the real world you'll hear all sorts of off-the-cuff references to this stuff; especially from me.

 

[Mark44]

Personally, I prefer an epistemic approach where you first describe things in an approximate way, often with analogies, before applying rigorously correct definitions.

Sure, that's a reasonable approach, as long as the approximate description is close enough to the definition that they don't conflict with the correct definition. My comments were aimed at correcting statements that were a bit off from being correct; e.g., that all eigenvectors have the same direction (they don't) and that all possible eigenvectors are the same (they aren't -- they differ by a constant multiple).

 

[PeroK]

Let me do this task from the previous post.

First you need to find eigenvalues.
$det \begin{bmatrix} 2-\lambda&1\\ 4&-1-\lambda \end{bmatrix} =0$

$(2-\lambda)(-1-\lambda)-1 \cdot 4=0$
$(2-\lambda)(-1-\lambda)- 4=0$
$-2-2\lambda+\lambda+\lambda^2-4=0$
$\lambda^2-\lambda-6=0$

I need to find solutions $\lambda_1$ and $\lambda_2$.

Often you can solve such an equation by "inspection". You can try $\lambda = 1,2,3$ and see immediately that $\lambda =3$ is one solution. Then the other solution must be $-2$. You could also have tried $\lambda = -1,-2$.

 

[Bosko]

Often you can solve such an equation by "inspection". You can try $\lambda = 1,2,3$ and see immediately that $\lambda =3$ is one solution. Then the other solution must be $-2$. You could also have tried $\lambda = -1,-2$.

With little modification , that is in fact, IMO, the most universal method for solving any polynomial equation ...
$$a_n\lambda^n+a_{n-1}\lambda^{n-1}+...+a_1\lambda+a_0=0$$
When I first looked at $\lambda^2-\lambda-6=0$ my first thought was aha.. this U-shaped curve for $\lambda=0$ is -6 and for $\lambda=-10$ and $\lambda=10$ it is well above zero. Well , one solution must be somewhere between -10 and 0 and another between 0 and 10.

If the degree of the equation was greater than two, I would probably do a similar method. Bisection (splitting of an interval):

If value of e.g. $\lambda^2-\lambda-6=0$ for $\lambda=0$ is negative and for $\lambda=10$ is positive . Let's try on the middle, $\lambda=5$ .

If value of the polynomial is zero then $\lambda=5$ is a solution. The entire polynomial $a_n\lambda^n+a_{n-1}\lambda^{n-1}+...+a_1\lambda+a_0$ should be divided by $\lambda-5$ and a new polynomial whose degree is n-1 is obtained. The procedure is repeated ...

If the value of the polynomial for $\lambda=5$ is greater than 0 then we will try with something in the middle of 0...5 interval.
If the value of the polynomial for $\lambda=5$ is less than 0 then we will try with something in the middle of 5...10 interval.

For the quadratic equation I like to use the oldest piece of mathematics I know of.
Around 2000 BC in the first agricultural (Sumerian) civilisation, in the Third dynasty of the city of Ur, 300 years after Akkadi invasion ( haha ... and centuries before the dark lord Sauron forged the one ring ... haha )

they solved equations of this form ...
$\lambda_1+\lambda_2=7$
$\lambda_1 \cdot \lambda_2=12$
$\lambda_1$ and $\lambda_2$ where represented the sides of the rectangle, so $\lambda_1+\lambda_2=7$ was half the circumference and the product $\lambda_1 \cdot \lambda_2=12$ was its surface area.
The same form has the modern name :"Viet's formulae for a quadratic equation"

3500 years later in Italy in the 16th century, formulas were found for solving polynomial equations of the third and fourth degree.

People tried to find formulas for solving polynomial equations of degree 5 and higher ... until a brilliant young Frenchman Evariste Galois (1811-1832) proved that there are no formulas for equations of degree 5 and higher.

Computers and a method similar to the one you described are used.

 

[DaveE]

With little modification , that is in fact, IMO, the most universal method for solving any polynomial equation ...
$$a_n\lambda^n+a_{n-1}\lambda^{n-1}+...+a_1\lambda+a_0=0$$
When I first looked at $\lambda^2-\lambda-6=0$ my first thought was aha.. this U-shaped curve for $\lambda=0$ is -6 and for $\lambda=-10$ and $\lambda=10$ it is well above zero. Well , one solution must be somewhere between -10 and 0 and another between 0 and 10.

If the degree of the equation was greater than two, I would probably do a similar method. Bisection (splitting of an interval):

If value of e.g. $\lambda^2-\lambda-6=0$ for $\lambda=0$ is negative and for $\lambda=10$ is positive . Let's try on the middle, $\lambda=5$ .

If value of the polynomial is zero then $\lambda=5$ is a solution. The entire polynomial $a_n\lambda^n+a_{n-1}\lambda^{n-1}+...+a_1\lambda+a_0$ should be divided by $\lambda-5$ and a new polynomial whose degree is n-1 is obtained. The procedure is repeated ...

If the value of the polynomial for $\lambda=5$ is greater than 0 then we will try with something in the middle of 0...5 interval.
If the value of the polynomial for $\lambda=5$ is less than 0 then we will try with something in the middle of 5...10 interval.

For the quadratic equation I like to use the oldest piece of mathematics I know of.
Around 2000 BC in the first agricultural (Sumerian) civilisation, in the Third dynasty of the city of Ur, 300 years after Akkadi invasion ( haha ... and centuries before the dark lord Sauron forged the one ring ... haha )

they solved equations of this form ...
$\lambda_1+\lambda_2=7$
$\lambda_1 \cdot \lambda_2=12$
$\lambda_1$ and $\lambda_2$ where represented the sides of the rectangle, so $\lambda_1+\lambda_2=7$ was half the circumference and the product $\lambda_1 \cdot \lambda_2=12$ was its surface area.
The same form has the modern name :"Viet's formulae for a quadratic equation"

3500 years later in Italy in the 16th century, formulas were found for solving polynomial equations of the third and fourth degree.

People tried to find formulas for solving polynomial equations of degree 5 and higher ... until a brilliant young Frenchman Evariste Galois (1811-1832) proved that there are no formulas for equations of degree 5 and higher.

Computers and a method similar to the one you described are used.

Works great for HW problems. IRL, the answers are hardly ever integers that you can guess, and are often 'yet to be defined' constants.

 

[Martyn Arthur]

My sincere thanks to all of you