Skip to main content
Search

Junhan Sun

1 related articles

Subscribe to updates
  1. arXiv

    JEPA world models lose sample differences from initialization: SADL constrains the encoder only when supply falls short, cutting collapse across ten seeds from 50% to 0%

    In JEPA-style world models the encoder representation is dominated from initialization by a component shared by all inputs, and batch normalization in the projector hides that direction from downstream losses, so sample differences shrink until they collapse. SADL compares the difference capacity a projector-side regularizer demands with the difference capacity the encoder can excite and pass on, and adds a constraint on the encoder only when supply falls short, with no auxiliary networks or trainable parameters. In single-task and jointly trained four-task LeWM and INTACT runs, collapse across ten random seeds falls from 50% to 0%, and after one epoch the encoder effective rank is 3.4–6.4 times that of the corresponding baseline.

You’re all caught up