Is Function $f$ Convex if its Hessian Matrix is Positive Semidefinite?

  • MHB
  • Thread starter Euge
  • Start date
In summary, a Hessian matrix is a square matrix of second-order partial derivatives that provides information about the curvature of a function. A Hessian matrix is positive semidefinite if all of its eigenvalues are non-negative, indicating that the function's curvature is either flat or convex. A function is convex if its Hessian matrix is positive semidefinite, as this ensures that the function's curvature is always upward or flat. If a Hessian matrix is not positive semidefinite, it means that the function's curvature is downward in at least one direction, making it non-convex. However, a function can still be convex if its Hessian matrix is positive semidefinite, as other conditions must also be met for a function
  • #1
Euge
Gold Member
MHB
POTW Director
2,073
243
Here is this week's POTW:

-----
Let $f : \Bbb R^n \to \Bbb R$ be twice differentiable function whose Hessian matrix is everywhere positive semidefinite. Show that $f$ is convex.
-----

 
Physics news on Phys.org
  • #2
No one answered this week’s problem. You can read my solution below.

For fixed $\mathbf{x}, \mathbf{y}\in \Bbb R^n$, define $g(t;\mathbf{x}, \mathbf{y}) = f(\mathbf{x}+ t\mathbf{y})$, for all $t\in\Bbb R$. If $H_{\mathbf{z}}(f)$ denotes the Hessian matrix evaluated at $\mathbf{z}$, then $g’’(t;\mathbf{x}, \mathbf{y}) = \frac{1}{2}\mathbf{y}^TH_{\mathbf{x} + t\mathbf{y}}(f)\mathbf{y}$, which is nonnegative by the positive semidefiniteness of the Hessian matrix. Therefore $g(t;\mathbf{x},\mathbf{y})$ is convex. Since $\mathbf{x}$ and $\mathbf{y}$ are arbitrary, it follows that $f$ is convex. Indeed, for all $\mathbf{x},\mathbf{y}\in \Bbb R^n$ and $t\in [0,1]$, $$f((1-t)\mathbf{x} + t\mathbf{y}) = f(\mathbf{x} + t(\mathbf{y}-\mathbf{x})) = g(t; \mathbf{x},
\mathbf{y}-\mathbf{x}) \le (1-t) g(0;\mathbf{x}, \mathbf{y}-\mathbf{x}) + t g(1; \mathbf{y} - \mathbf{x}) = (1-t) f(\mathbf{x}) + t f(\mathbf{y})$$ as desired.
 

FAQ: Is Function $f$ Convex if its Hessian Matrix is Positive Semidefinite?

What is a Hessian matrix?

A Hessian matrix is a square matrix of second-order partial derivatives of a multivariate function. It provides information about the curvature of the function at a given point.

What does it mean for a Hessian matrix to be positive semidefinite?

A Hessian matrix is positive semidefinite if all of its eigenvalues are non-negative. This indicates that the function is convex, meaning that it curves upwards and has no local minima.

How can I determine if a function is convex using its Hessian matrix?

If the Hessian matrix is positive semidefinite, then the function is convex. This can be determined by calculating the eigenvalues of the matrix and ensuring they are all non-negative.

Can a function be convex if its Hessian matrix is not positive semidefinite?

No, a function must have a positive semidefinite Hessian matrix to be convex. If the Hessian matrix is not positive semidefinite, then the function is either concave or has a saddle point.

Why is the positive semidefinite property of the Hessian matrix important in determining convexity?

The positive semidefinite property of the Hessian matrix is important because it guarantees that the function is convex, which has many practical implications. For example, convex functions have a unique global minimum, making optimization easier. Additionally, convex functions have stable and predictable behavior, making them useful in many applications such as machine learning and economics.

Back
Top