LoRA-Muon-OGD: Spectral Orthogonal Gradient Projection on the Low-Rank Manifold for LLM Continual Learning
Generalizing Orthogonal Gradient Projection to the low-rank case and to steepest descent under a larger family of norms.
Generalizing Orthogonal Gradient Projection to the low-rank case and to steepest descent under a larger family of norms.