What is the derivative of a vector?

[Lucid Dreamer]

Hello,

In lecture today, my professor told us that the derivative of a row vector is a column vector. I worked with vector calculus before and never came across this. I suspect it is a notational issue but would greatly appreciate it if someone could elaborate on this.

 

[Amir Livne]

I never encountered this convention, and I can't see where it can benefit the presentation.
This seems like a perfectly good thing to ask your prof.

 

[DivisionByZro]

Strictly speaking, a vector does not have a derivative. However, if you have a vector-valued function (for example a function representing position as a function of time), then you can certainly consider the derivative of that function, it will simply be another function (the velocity function).

Also, some people seem to not be bothered by switching a row vector to a column vector, so I suppose in a sense your prof can be right, but I don't like that approach. Suppose you have a vector v(t), then the derivative with respect to t is simply the gradient of that vector function, which yields a row or column vector, depending on how you had it to start with.

Does this help?

 

[jambaugh]

Hello,

In lecture today, my professor told us that the derivative of a row vector is a column vector. I worked with vector calculus before and never came across this. I suspect it is a notational issue but would greatly appreciate it if someone could elaborate on this.

The derivative (gradient) of a scalar with respect to a vector, i.e. of scalar function of a vector variable, should be expressed as a dual vector. If you represent the vector variable as a column vector of variables then the derivative (gradient) should be written as a row vector of partial derivatives.

Beyond that, (parametric) derivatives of row vectors yields row vectors and likewise with column vectors. (parametric = derivatives w.r.t. a parameter, the vectors being functions of a single variable. $\vec{x}(t)$).

Row vectors and column vectors live in different (though isomorphic) spaces. However when we work with general vectors (in yet another space) with a given basis we can write the basis expansion using both row and column vectors like this:

$$x\mathbf{i}+y\mathbf{j} + z\mathbf{k} = \left(\begin{array}{ccc} \mathbf{i} & \mathbf{j} & \mathbf{k} \end{array}\right)\left[\begin{array}{c}x \\ y \\ z \end{array}\right] = \left(\begin{array}{ccc} x & y & z \end{array}\right)\left[\begin{array}{c} \mathbf{i} \\ \mathbf{j} \\ \mathbf{k} \end{array}\right]$$

This way we can represent general vectors (via a given basis) as either row vectors or column vectors of components and do so interchangibly. I find it convenient to (mostly) use column vectors to represent coordinate vectors and then row vectors to represent the dual gradients. But it is all a matter of convention and convenience.

 

[Lucid Dreamer]

Perhaps I should have been more clear in my question. I am looking at dR(w)/dw where w is a vector and R(w) is a scalar function of a vector variable. In this case, jambaugh post seems to make sense as if w is a row vector, than dR(w)/dw is a column vector. Thanks for all your help!

 

[jambaugh]

Perhaps I should have been more clear in my question. I am looking at dR(w)/dw where w is a vector and R(w) is a scalar function of a vector variable. In this case, jambaugh post seems to make sense as if w is a row vector, than dR(w)/dw is a column vector. Thanks for all your help!

Let me further elaborate as to why you get a dual vector. To generalize the idea of a derivative to vector calculus use differentials as local coordinates in the local linear approximation to a function.

Given $y = f(x)$ then the local linear approximation is:
$$dy = f'(x)dx \quad\quad\text{ that is to say } y+dy = f(x)+f'(x)dx \approx f(x+dx)$$
Allow either x or y or both to be vectors here. You then have the derivative as a linear operator valued function of x, said linear operator maps $dx$ type objects to $dy$ type objects. We can define it as a limit of a difference quotient if we are careful to avoid what looks like division by a vector:
$$\mathbf{dy} = f'(\mathbf{x})\mathbf{dx} \equiv \lim_{h\to 0} \frac{ f(\mathbf{x}+h\mathbf{dx}) - f(\mathbf{x})}{h}$$
Here $h$ is a real number and the difference quotient is well defined provided the range and domain of f are vector spaces (so we can add elements and multiply by scalars h and 1/h). That includes of course the case of 1-dimensional vectors we call scalars.

The nature of $f'(x)$ then is as a linear operator and we have the following cases:

• dy,dx both scalars: the linear operator $f'$ is just multiplication by a number.
• dy vector and dx scalar: the linear operator $f'$ maps scalars to vectors and so is multiplication by a vector.
• dy scalar and dx vector: the linear operator $f'$ maps vectors to scalars and so is a dual vector (linear functional).
• dy vector and dx vector:$f'$ is a full blown linear operator representable by a matrix.

Now as I mentioned, I prefer to use column vectors as coordinate vectors and row vectors for dual vectors so that in matrix format the action of $f'$ is left multiplication by a matrix.

e.g.
$$u = f(x,y);\quad du = f'(x,y)\left[\begin{array}{c}dx \\ dy \end{array}\right] = ( {\scriptsize{\frac{\partial u}{\partial x}\quad \frac{\partial u}{\partial y}}} ) \left[\begin{array}{c}dx \\ dy \end{array}\right]$$
$$\left[\begin{array}{c}u \\ v\end{array}\right] = \mathbf{F}(x,y,z) ; \quad \left[\begin{array}{c}du \\ dv \end{array}\right] =\mathbf{F}' (x,y,z)\left[\begin{array}{c}dx \\ dy \\ dz \end{array}\right]= \left[\begin{array}{c c c} \scriptsize{ \frac{\partial u}{\partial x}} &\scriptsize{ \frac{\partial u}{\partial y}} &\scriptsize{ \frac{\partial u}{\partial z}}\\ \scriptsize{ \frac{\partial v}{\partial x}} &\scriptsize{ \frac{\partial v}{\partial y}} &\scriptsize{ \frac{\partial v}{\partial z}}\end{array}\right] \left[\begin{array}{c}dx \\ dy \\ dz \end{array}\right]$$
Note the derivative of a scalar valued function of vectors is just the gradient and in resolving change of variables one gets the correct form of the gradient by preserving the differential relationship: $du = \nabla u \cdot \mathbf{dr}$.

2nd Note: Here I'm using primed notation just to match up with single variable calc. notation. More traditionally one uses the $\nabla$ operator. $F' \to \nabla F$, or one may use a Leibniz type notation.

3rd Note: Reversing my use of rows vs columns would allow one to better express directional derivatives and the differential operator, e.g.:
$$\mathbf{d} =\left( \begin{array}{ccc}dx & dy & dz \end{array}\right) \left[ \begin{array}{c} \partial_x \\ \partial_y \\ \partial_z\end{array}\right]$$

A final note. Here I am treating the differentials simply as local coordinates and not as differential forms per se and not as infinitesimals. In full blown differential geometry of manifolds we can't add points and differentials become cotangent vectors while the partial derivatives become tangent vectors. What we call "vector" and what we call "dual vector" is relative. The distinction between "tangent vector" and "co-tangent vector" is not.

 

[Lucid Dreamer]

I'm not sure if this is right, but here's what I think. Suppose
$$\frac{df}{d\vec{x}}: \mathbb{R}^m \rightarrow \mathbb{R}^n$$
Then $\frac{df}{d\vec{x}}$ is an $n \times m$ matrix.

Let $\vec{x} \epsilon \mathbb{R}^m$ so that $\vec{x}$ is a $m \times 1$ column vector. In the special case where n = 1, $\frac{df}{d\vec{x}}$ is a $1 \times m$ row vector.

Any thoughts?

 

[jambaugh]

I'm not sure if this is right, but here's what I think. Suppose
$$\frac{df}{d\vec{x}}: \mathbb{R}^m \rightarrow \mathbb{R}^n$$
Then $\frac{df}{d\vec{x}}$ is an $n \times m$ matrix.

Let $\vec{x} \epsilon \mathbb{R}^m$ so that $\vec{x}$ is a $m \times 1$ column vector. In the special case where n = 1, $\frac{df}{d\vec{x}}$ is a $1 \times m$ row vector.

Any thoughts?

Did you mean to say: $f:\mathbb{R}^m\to \mathbb{R}^n$?

To be ultra-precise, $\frac{df}{d\vec{x}}$ is an $n\times m$ matrix valued function and so $\frac{df}{d\vec{x}}(\vec{x})$ is an $n\times m$ matrix.

But beyond pedantic trivialities, yes you've got the gist of it.

 

[Lucid Dreamer]

$f: \mathbb{R}^m \rightarrow \mathbb{R}^n$ is represented by a $n \times m$ matrix. I don't see how $\frac{df}{d\vec{x}}$ is also represented by a $n \times m$ matrix.

Would you be able to provide a reference text for vector calculus that also does a fair treatment of matricies?

 

[jambaugh]

$f: \mathbb{R}^m \rightarrow \mathbb{R}^n$ is represented by a $n \times m$ matrix.

Only if $f$ itself is a linear mapping!

I don't see how $\frac{df}{d\vec{x}}$ is also represented by a $n \times m$ matrix.

In the linear case, $f(x) = Mx,\quad \frac{df(x)}{dx} = M,\quad \frac{df(x)}{dx} \cdot dx = M\cdot dx$ where $M$ is an $n\times m$ matrix.

Would you be able to provide a reference text for vector calculus that also does a fair treatment of matricies?

I don't know of one offhand. You'll probably get more use from separate texts, one on linear algebra, the other a good calculus text.

 

[jambaugh]

Would you be able to provide a reference text for vector calculus that also does a fair treatment of matricies?

The table of contents of ...
Advanced Calculus of Several Variables
looks pretty good. I haven't seen the book itself.

 

[lugita15]

Vector Calculus by Colley does an excellent job of explaining multivariable calculus in matrix form, and it also explains the linear algebra you need to manipulate these matrices.