Skip to main content
Search

Columbia University

19 related articles

Subscribe to updates
  1. arXiv

    JAX-WAVE turns inverse wave design into one forward pass: 45% lower error than FISTA inside 5 ms

    In a heterogeneous medium, making a wave field take a prescribed shape at a set time means working backward to an initial pressure distribution, and previously every target and every medium required a fresh numerical inversion. JAX-WAVE trains a shared neural network to map the desired terminal field and a known wave-speed map directly to a bounded initial pressure, with training signal supplied by a differentiable finite-difference time-domain solver and no pre-optimized control labels. On an independent confirmation set, three trained models reach 25.24% error on unseen geometric targets and 4.77% on unseen reachable targets; within a 5 ms median online budget the learned map cuts mean error by 45% relative to accelerated numerical inversion on the heterogeneous benchmark.

  2. arXiv

    DictLLM shrinks a 7B model 5.1× with shared dictionaries and sparse coefficients, adding 2.9 points of perplexity

    On mobile devices the binding constraint for large language models is weight memory, not computation. DictLLM splits weight matrices within grouped layers into one shared dictionary plus layer-specific sparse coefficients, all computed from a pretrained model without retraining. On LLaMA-7B it compresses the model 5.1× with only a 2.9-point rise in WikiText-2 perplexity, a compression ratio about 3.6× that of the state-of-the-art low-rank method.

  3. arXiv

    Diffusion language models can learn from autoregressive teachers: JustOPD lifts Dream-7B's math reasoning while keeping any-order decoding

    Autoregressive models generate strictly left to right, while diffusion language models denoise in arbitrary order, so the former's token-level supervision does not transfer directly. JustOPD forces the diffusion student to roll out left to right during training, letting a Qwen3-14B teacher supervise every intermediate state, while inference keeps any-order decoding. With Dream-7B as student and Qwen3-14B as teacher on GSM8K and MATH500, scores rise over the base model across several divergences.

  4. arXiv

    MLP linearization moves to low-degree rational fits: Qwen3-8B drops 14% of parameters and decodes faster

    Instead of relying only on polynomial random features, MLP linearization now approximates activations with low-degree rational functions and splits each layer, via partial fractions, into separable weight-side and input-side inner products. Across synthetic fitting, vision, 3D and language models this route gives lower error at the same feature budget; compressing Qwen3-8B by replacing its last eight feed-forward layers removes 1.17B parameters (14% of the model) and raises decoding throughput from 763.3 to 809.6 tokens/s, with reasoning and knowledge benchmarks roughly at parity with the uncompressed model.

  5. Topic Models Drop the Word List: Directions in LLM Activations Track Short and Abstract Texts Better

    Topic models can describe topics with interpretable directions learned by sparse autoencoders instead of word lists, and an LLM-based topic judge prefers that representation on eight datasets, with the largest margins on short and abstract texts.

  6. arXiv

    RELACE scores actions by comparing original-context and outcome-conditioned likelihoods, reaching the highest reported mean success among compared methods on ALFWorld and WebShop

    RELACE is a critic-free framework that uses a frozen behavior policy to teacher-force score each executed action under both its original context and an outcome-augmented context, normalizes and clips the ratio of these length-normalized scores within each trajectory to reweight discounted returns, and then builds local advantages by comparing weighted returns among actions from matched states, combining them with trajectory-level GRPO supervision; on ALFWorld and WebShop with Qwen2.5-1.5B-Instruct and Qwen2.5-7B-Instruct it reports the highest mean aggregate metrics among the compared methods, including GRPO, GiGPO, HCAPO, and GraphGPO.

  7. arXiv

    SteerScope benchmark: none of 23 steering methods beats the Prompt Steering baseline on the efficacy–side-effect trade-off

    The authors introduce SteerScope, a two-axis, 15-metric evaluation suite for LLM activation steering, and benchmark 23 methods spanning 4 families (including prompting, LoRA, and SFT baselines) on Gemma-2-2B-it and Gemma-2-9B-it under matched models, tasks, and protocols; they find that no activation steering method achieves higher target efficacy without greater composite side effects, so none surpasses the Prompt Steering baseline on the efficacy–side-effect trade-off, and that efficacy and side effects rise together with intervention strength.

  8. arXiv

    EmbodiedRSI uses a hypothesis graph and value-of-information experiment selection to lift a frozen VLA to 77.0% on RoboCasa365 and 86.8% on LIBERO-Pro, transferring zero-shot to a real robot at 71.3%

    EmbodiedRSI is a self-evolving agentic harness that, around a frozen robot foundation model, maintains code and skill hypotheses in a Hypothesis Graph, uses Value-of-Information Experiment Selection to pick the physical experiments that best distinguish them, and turns each interaction into reusable improvement through Code-Skill Co-Evolution and Reward-Grounded Memory Learning; it reaches 77.0% overall success on RoboCasa365 and 71.3% on Composite-Unseen (best baseline 40.1%), 86.8% overall on LIBERO-Pro, and transfers the simulation-evolved harness zero-shot to a real robot at 71.3% overall success.

  9. arXiv

    ENPO uses adversarial policy exploration to remove iterative NLHF's exponential dependence on the inverse KL-regularization parameter, and consistently beats the evaluated RLHF and NLHF baselines on Llama-3-8B-Instruct

    This work studies online iterative Nash learning from human feedback (NLHF) and identifies exploration as a key obstacle: standard iterative NLHF can incur an exponential dependence on the inverse KL-regularization parameter, showing that implicit exploration through policy updates can be insufficient; it proposes ENPO, which combines SFT-type regularization with adversarial policy exploration to eliminate this exponential dependence without minimax oracles or explicit preference-model estimation, further introduces BENPO, which uses additional oracles to achieve an O(log T) regret bound, and develops DENPO, a practical variant of ENPO for fine-tuning LLMs; with Llama-3-8B-Instruct, DENPO shows consistent improvements over the evaluated RLHF and NLHF baselines across multiple benchmarks.

  10. arXiv

    Fixed-parameter universal transformers simulate any model in their class via input embeddings, validated on parenthesis balancing and multi-hop reasoning

    The authors introduce universal transformers: fixed transformers whose internal parameters stay fixed while a suitable input embedding encodes a description of a target model, letting them simulate any transformer in a given class; they give explicit sparse constructions achieving universality when the embedding dimension is sufficiently large, show that randomly initialized transformers are universal almost surely, and empirically validate the theory on parenthesis balancing and multi-hop reasoning, suggesting much of a transformer's expressive power may reside in its input representation rather than its learned weights.