Proof of convergence theory in optimization

[i_a_n]

The question is:Suppose that lim $x_k=x_*$, where $x_*$ is a local minimizer of the nonlinear function $f$. Assume that $\triangledown^2 f(x_*)$ is symmetric positive definite. Prove that the sequence $\left \{ f(x_k)-f(x_*) \right \}$ converges linearly if and only if $\left \{ ||x_k-x_*|| \right \}$ converges linearly. Prove that the two sequences converge at the same rate, regardless of what the rate is. What is the relationship between the rate constant for the two sequences?

(I guess we may use the orthogonal diagonalization of a symmetric matrix and $f(x_k)-f(x_*)=\triangledown f(x_*)+\frac{1}{2}(x_k-x_*)^T\cdot\triangledown^2 f(\xi)(x_k-x_*)$ and $\triangledown f(x_*)=0$... But I got stuck here. So what's your answer?)

 

[Aaron Bex]

Answer:

The statement is true. To prove this, we will use Taylor's theorem and the orthogonal diagonalization of a symmetric matrix.

First, recall Taylor's theorem, which states that for any point $x_*$ in the domain of a function $f$, there exists an open neighborhood of $x_*$ such that for any $x$ in this neighborhood, $$f(x) = f(x_*) + \triangledown f(x_*)\cdot (x-x_*) + \frac{1}{2}(x-x_*)^T\cdot\triangledown^2 f(\xi)(x-x_*)$$ for some $\xi$ between $x_*$ and $x$.

Since $\triangledown^2 f(x_*)$ is symmetric positive definite, it can be orthogonally diagonalized. That is, there exists an orthogonal matrix $Q$ such that $\triangledown^2 f(x_*) = Q \Lambda Q^T$, where $\Lambda$ is a diagonal matrix with the eigenvalues of $\triangledown^2 f(x_*)$ on its diagonal. Therefore, we can rewrite the second term as $$\frac{1}{2}(x-x_*)^T\cdot\triangledown^