Is result of vector inner product retained after matrix multiplication?

[Master1022]

Homework Statement

Let us imagine that we have two vectors $\vec{a}$ and $\vec{b}$ and they point in similar directions, such that the inner-product is evaluated to be a +ve number. If we now multiply both of the vectors by a matrix $W$ which has real entries, will the inner product of the 'transformed' vectors also be positive?

Relevant Equations

Inner product

Hi,

I was thinking about the following problem, but I couldn't think of any conclusive reasons to support my idea.

Question:
Let us imagine that we have two vectors $\vec{a}$ and $\vec{b}$ and they point in similar directions, such that the inner-product is evaluated to be a +ve number. If we now multiply both of the vectors by a matrix $W$ which has real entries, will the inner product of the 'transformed' vectors also be positive?

Attempt:

Intuitively I think along the lines of: if we imagine the operation as transforming a vector in some way, then the two vectors $\vec{a}$ and $\vec{b}$, which were similar, should be transformed to similar vectors?

Mathematically, I can write the following:
$$<W \vec{a} , W \vec{b} > = (W \vec{a}) \cdot (W \vec{b}) = (W \vec{a})^{T} (W \vec{b}) = \vec{a}^{T} W^{T} W \vec{b}$$

- $W^{T} W$ is positive semi-definite.
- I suppose if $\vec{a}$ and $\vec{b}$, then if/how positive the outcome will depend on some type of sensitivity of $W$? That is, we could view $\vec{b} = \vec{a} + \vec{\epsilon}$ and think about that?

Any help is greatly appreciated.

 

[Orodruin]

If $\vec a$ and $\vec b$ are parallel, then the assertion follows directly from $W^T W$ being positive semi-definite (as long as you change the statement to be non-negative instead of positive - it should be pretty obvious that the result can be zero if $W^T W$ happens to have eigenvalues that are zero, such as when $W = 0$).

However, consider what happens when you let $\vec a$ and $\vec b$ be linearly independent and ask yourself the following questions:

• Taking any set of two vectors $\vec c$ and $\vec d$, can you find a linear transformation $W$ such that $W\vec a = \vec c$ and $W \vec b = \vec d$?
• What does this mean for your assertion?

 

[Master1022]

Thanks for the reply @Orodruin !

If $\vec a$ and $\vec b$ are parallel, then the assertion follows directly from $W^T W$ being positive semi-definite (as long as you change the statement to be non-negative instead of positive - it should be pretty obvious that the result can be zero if $W^T W$ happens to have eigenvalues that are zero, such as when $W = 0$).

Agreed

However, consider what happens when you let $\vec a$ and $\vec b$ be linearly independent and ask yourself the following questions:

• Taking any set of two vectors $\vec c$ and $\vec d$, can you find a linear transformation $W$ such that $W\vec a = \vec c$ and $W \vec b = \vec d$?

I think so... I could imagine forming some matrix equation like:
$$W [ \vec{a} \quad \vec{b}] = [\vec{c} \quad \vec{d}]$$
and if $\vec{a}$ and $\vec{b}$ are linearly independent, then the matrix is full-rank and that can be inverted to find $W$.

• What does this mean for your assertion?

That might suggest that the outcome of the transformed inner product is still positive? This doesn't necessarily seem correct, but I am confused still...

 

[PeroK]

That might suggest that the outcome of the transformed inner product is still positive? This doesn't necessarily seem correct, but I am confused still...

It took me about 30 seconds to find a counterexample. Hint. Let the vectors be $(1,0)$ and $(1, 1)$. Look for a 2 x 2 matrix that maps $(1, 0)$ to itself and $(1, 1)$ to something in approximately the opposite direction.

 

[Orodruin]

That might suggest that the outcome of the transformed inner product is still positive? This doesn't necessarily seem correct, but I am confused still...

So you are free to choose any $\vec c$ and $\vec d$. Are there any such pairs with a negative inner product?

 

[Master1022]

It took me about 30 seconds to find a counterexample. Hint. Let the vectors be $(1,0)$ and $(1, 1)$. Look for a 2 x 2 matrix that maps $(1, 0)$ to itself and $(1, 1)$ to something in approximately the opposite direction.

Okay thanks @PeroK ! That makes sense, and yes I can see how the answer to my question was 'not necessarily'.