- #1
brydustin
- 205
- 0
The title of an old paper... It mentions that in order to use the full information of a hessian in 2nd order optimization that you should make a part of your iterative step to include v (eigenvector corresponding to smallest eigenvalue, assuming that the eigenvalue is negative).
By doing the following: p = -sign(g'*v)*v : where g is the gradient. So here is the question, what is the geometrical meaning of the dot product of {g,v}? Because the idea is to find a local minimum but I'm trying to find a local maximum and would like to use similar information. Another condition for a local minimum would be that all the eigenvalues are positive, so in my case I would want all of them to be negative. So in my case would I set
p = + or - sign(g'*w)*w, where w is the eigenvalue corresponding to the largest eigenvalue (assuming that its also greater than 0 -- obviously if max(eigenvalue) < 0 then hessian is sufficiently conditioned to find a maximizer. Anyway, I appreciate any help on this... which sign do I pick and why (what's the geometry behind it?)
Thanks
By doing the following: p = -sign(g'*v)*v : where g is the gradient. So here is the question, what is the geometrical meaning of the dot product of {g,v}? Because the idea is to find a local minimum but I'm trying to find a local maximum and would like to use similar information. Another condition for a local minimum would be that all the eigenvalues are positive, so in my case I would want all of them to be negative. So in my case would I set
p = + or - sign(g'*w)*w, where w is the eigenvalue corresponding to the largest eigenvalue (assuming that its also greater than 0 -- obviously if max(eigenvalue) < 0 then hessian is sufficiently conditioned to find a maximizer. Anyway, I appreciate any help on this... which sign do I pick and why (what's the geometry behind it?)
Thanks