Skip to main content
Search

# Hessian-Vector Product

2 related articles

Subscribe to updates
  1. arXiv

    Tracking fine-tuning trajectories shows safety-aligned LLMs are not steered toward harm but lose their safe subspace, and projecting out the harmful gradient subspace cuts free-generation emergent misalignment by up to 80.0% on Qwen2.5-14B-IT

    The work presents a dynamic second-order geometric study of emergent misalignment (EM): tracking training trajectories shows directional Hessian curvature concentrates on semantic pivot tokens, Grassmannian projections show the harmful-safe gap widens mainly because safe-gradient overlap declines rather than through rotation toward a new malicious circuit, and on this basis it introduces a parameter-level Geometric Mitigation Framework that orthogonally projects the empirical harmful gradient subspace out of LoRA updates, suppressing free-generation EM by up to 80.0% on Qwen2.5-14B-IT and, across four open-weight instruction-based model families (3B–20B), teacher-forced evaluation shows the same harmful subspace controls the conditional support of frozen EM responses.

  2. Replacing full-Hessian supervision with random projected Hessian-vector products matches second-order accuracy while speeding each epoch by more than 24x

    The authors introduce Projected Hessian Learning (PHL), which supervises second-order curvature in machine-learning interatomic potentials (MLIPs) through stochastic Hutchinson-trace-based Hessian-vector product (HVP) projections instead of explicit Hessian construction; on a ωB97XD/6-31G(d) dataset of reactants, products, transition states, IRC and normal-mode-sampled geometries they compare four schemes (E-F, E-F-HVP with one-column or Hutchinson probes, and E-F-H), finding that with probe vectors resampled each minibatch both HVP methods are statistically indistinguishable from full-Hessian training in energy, force and Hessian accuracy while giving more than 24x speedup per epoch, and that under the more realistic fixed-vector regime with one HVP per molecule Hutchinson projections con

You’re all caught up