Formulating a Method of Steepest Ascent on Lie Groups

[Mandelbroth]

Suppose we have a compact Lie group $G$, and a differentiable function $f:G_0\to\mathbb{R}$ from the identity component of $G$ to the real numbers. I'm looking to maximize the value of this function.

Being something of a neophyte at optimization, especially of this kind, I decided to stick with something I thought I knew well: the method of steepest ascent. Long story short, I'm having trouble with generalizing the concept to Lie groups.

My idea was fairly simple. We start with some point $p$ on the identity component of $G$, and then we take the logarithm of this point (that is, we take the inverse of the exponential map) because we want to add something to it. As a note, I justified this by saying that, if the point had more than one value for the logarithm, I could just pick one (every point on $G_0$ has at least one logarithm). Then, I would add some multiple of $\nabla f_p$ to $\log(p)$, and finish by exponentiating to get $p'=\exp(c\nabla f_p + \log(p))=\exp(c\nabla f_p)p$. Repeat.

The problem is, I can't figure out what I would use for the analogue of the gradient. Maybe I'm just not seeing something? I don't know. Any nudge in the right direction would be greatly appreciated. Thank you.

 

[jgens]

The problem is, I can't figure out what I would use for the analogue of the gradient.

Not sure if this helps (the question is way outside my area of study), but you can always fix a (bi-invariant) Riemannian metric and define gradients using that.

 

[Mandelbroth]

Not sure if this helps (the question is way outside my area of study), but you can always fix a (bi-invariant) Riemannian metric and define gradients using that.

I'm unfamiliar with this concept, but Wikipedia claims to know something about this. To fact-check, you're suggesting that I could introduce a Riemannian metric $g$ and define $\nabla f_p$ by $g_p(\nabla f_p, X_p)=X_p(f)$?

Wouldn't that put $\nabla f_p$ on $T_p G$ and not $T_e G$ (where $e$ is the identity on $G$), though?

 

[jgens]

Correct definition. You could also use pushforwards on the relevant left-multiplication map to map vectors TG_p into vectors TG_e. Invariance of the metric should ensure this is pretty well-behaved with respect to the gradient too.

 

[blue_raver22]

As a fellow scientist, I can understand your struggle with generalizing the method of steepest ascent to Lie groups. Let's break down your idea and see where the issue might lie.

Firstly, the method of steepest ascent in its traditional form is used to maximize a function in Euclidean space. In this case, we are dealing with a compact Lie group, which is a more abstract mathematical structure. So, we need to think about how this method can be adapted to work in this setting.

You mentioned taking the logarithm of the point in order to add something to it. This is a reasonable approach, as the logarithm is the inverse of the exponential map and can be thought of as a way to "move" in the group. However, the issue here is that the gradient of a function in Euclidean space is a vector, whereas in a Lie group, the tangent space at a point is a Lie algebra. So, we need to think about how to define a "direction" or "vector" in the Lie algebra that corresponds to the gradient of the function.

One way to do this is to use the notion of the tangent space at a point in the Lie group. This tangent space can be thought of as the set of all tangent vectors at that point, and it is isomorphic to the Lie algebra of the group. So, in a sense, we can think of the tangent space as the "vector space" of the Lie group.

With this in mind, we can define the gradient of a function on a Lie group as a tangent vector at a point that points in the direction of steepest ascent. This can be done using the pushforward map, which maps vectors in the Lie algebra to tangent vectors on the Lie group.

So, in summary, to adapt the method of steepest ascent to Lie groups, we need to use the notion of the tangent space and the pushforward map to define the gradient of a function on the Lie group. This will allow us to apply the same principles of the method of steepest ascent to maximize a function on a compact Lie group.

I hope this helps nudge you in the right direction. Good luck with your research!