- #1
- 1,009
- 9
I've been studying the book "Numerical Optimization" by Jorge Nocedal and Stephan J. Wright published by Springer, 1999. I'm puzzled by the use of the Laplacian operator ##\nabla^2## in chapter 10 on nonlinear least squares and in the appendix to define the Hessian matrix. The following is from pages 252 and 582:
$$\begin{align*}
\nabla f(x) &= \sum_{j=1}^m r_j(x) \nabla r_j(x) = J(x)^T r(x) \\
\nabla^2 f(x) &= \sum_{j=1}^m \nabla r_j(x) \nabla r_j(x)^T + \sum_{j=1}^m r_j(x) \nabla^2 r_j(x) \\
&= J(x)^T J(x) + \sum_{j=1}^m r_j(x) \nabla^2 r_j(x)
\end{align*}$$
The matrix of second partial derivatives of ##f## is known as the Hessian, and is defined as
$$
\nabla^2 f(x) = \begin{bmatrix}
\frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & \dots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\
\frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \dots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\
\vdots & \vdots && \vdots \\
\frac{\partial^2 f}{\partial x_n \partial x_1} & \frac{\partial^2 f}{\partial x_n \partial x_2} & \dots & \frac{\partial^2 f}{\partial x_n^2}
\end{bmatrix}
$$
Is it my imagination that the Laplacian operator is being improperly used? My understanding is that the Laplacian is:
$$
\nabla^2 f(x) = \frac{\partial^2 f}{\partial x_1^2} + \frac{\partial^2 f}{\partial x_2^2} + \dots + \frac{\partial^2 f}{\partial x_n^2}
$$
which is the trace of the Hessian.
$$\begin{align*}
\nabla f(x) &= \sum_{j=1}^m r_j(x) \nabla r_j(x) = J(x)^T r(x) \\
\nabla^2 f(x) &= \sum_{j=1}^m \nabla r_j(x) \nabla r_j(x)^T + \sum_{j=1}^m r_j(x) \nabla^2 r_j(x) \\
&= J(x)^T J(x) + \sum_{j=1}^m r_j(x) \nabla^2 r_j(x)
\end{align*}$$
The matrix of second partial derivatives of ##f## is known as the Hessian, and is defined as
$$
\nabla^2 f(x) = \begin{bmatrix}
\frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & \dots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\
\frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \dots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\
\vdots & \vdots && \vdots \\
\frac{\partial^2 f}{\partial x_n \partial x_1} & \frac{\partial^2 f}{\partial x_n \partial x_2} & \dots & \frac{\partial^2 f}{\partial x_n^2}
\end{bmatrix}
$$
Is it my imagination that the Laplacian operator is being improperly used? My understanding is that the Laplacian is:
$$
\nabla^2 f(x) = \frac{\partial^2 f}{\partial x_1^2} + \frac{\partial^2 f}{\partial x_2^2} + \dots + \frac{\partial^2 f}{\partial x_n^2}
$$
which is the trace of the Hessian.