Skip to main content
Search

Cornell University

19 related articles

Subscribe to updates
  1. arXiv

    Low-bit inference stops choosing between accuracy and speed: StrataFlux streams refinement bits from RAM and SSD, cutting 2-bit Llama-3.1-8B perplexity from 9.46 to 8.40

    StrataFlux, from Cornell and Carnegie Mellon researchers, keeps low-bit weights and KV cache resident on the GPU and fetches only a small set of refinement bits from host RAM or SSD for the computation at hand, recovering accuracy while retaining quantization's memory and speed advantages. On an RTX 5090, 2-bit Llama-3.1-8B-Instruct weights drop WikiText-2 perplexity from 9.46 to 8.40 while keeping 95.4% of the quantized base's throughput; with 2-bit weights and KV, Qwen3-8B's AIME 2024/2025 accuracy rises from 11.0% to 47.1% at unchanged GPU memory use.

  2. arXiv

    Titan-Llama bolts external neural memory onto a frozen Llama: long-document QA accuracy rises from 23.87% to 40.51%

    Titan-Llama converts the first four layers of Llama-3.2-1B-Instruct to blockwise attention and attaches a fixed-size neural memory module to each KV head, using test-time training to compress historical key-value bindings while the backbone stays frozen. Against segmented attention windows alone, accuracy rises from 23.87% to 40.51% on QuALITY and from 14.50% to 40.75% on LongHealth. In the evaluated configurations, prefill is more than 20% faster, decode 37% faster at medium context lengths, and steady-state decode memory 24% lower.

  3. arXiv

    Why the Jacobian lens favors slow features: a theory writes its choice as three coherence factors and predicts their ordering in toy models

    The Jacobian lens (J-lens), an interpretability tool that decodes a model's hidden states into words, had been used to find working-memory-like linear directions, but what makes a feature detectable to it was unclear. This work gives a mathematical theory: the lens preferentially selects directions that grow across layers, stay smooth across token positions, and remain consistent across contexts, that is, coherent Lyapunov modes. On a Bayes filter and a two-layer toy transformer, changing task parameters to modulate coherence shifts the ordering of latent states the lens selects, as predicted.

  4. DiFVM puts an unstructured-mesh finite-volume solver on the GPU: one card beats 32-core OpenFOAM and still yields end-to-end gradients

    DiFVM rewrites the complete finite-volume/PISO solution procedure on unstructured polyhedral meshes as vectorized scatter and gather operations over a static graph, giving GPU-native execution and end-to-end automatic differentiation in JAX/XLA. Existing differentiable CFD platforms mostly rely on Cartesian or structured grids and do not directly handle body-conforming complex geometries. On a single NVIDIA L40 GPU, DiFVM is faster than 32-core parallel OpenFOAM in every tested case, the largest with about 106 control volumes; differentiability is demonstrated by recovering a cavity lid velocity and inferring six Windkessel parameters simultaneously.

  5. arXiv

    Training logs know what deployment cannot: runtime-confounded RL gets doubly robust evaluation and a testable price

    When deployment lacks key variables that training logs record in full, runtime confounding can push a reinforcement-learning policy off course over time. The work places the runtime restriction only in the numerator of an importance ratio and estimates nuisance functions from the full data, yielding an asymptotically normal off-policy estimator that attains the semiparametric efficiency bound, a policy-learning regret bound that differs from the full-information optimum by an irreducible runtime gap, and a sample-splitting test for that gap.

  6. LLMs keep learning after deployment: sleep-style offline consolidation writes new facts into weights without erasing old skills

    Deployed large language models stay frozen at a fixed knowledge cutoff, continual parameter updates cause severe forgetting of earlier tasks, and in-context learning loses information once the session ends. Google Research's sleep framework alternates active interaction with offline consolidation, in which the model opens pre-allocated Mixture-of-Experts memory blocks and updates weights using self-generated practice examples. In multi-stage continual translation into unseen languages such as Manchu and Kalamang, performance dropped near zero, while in-context learning, Cartridges and supervised fine-tuning suffered severe forgetting.

  7. A single-cell velocity model you can perturb: it named three rescue targets for Arid1a-loss B cells, and mice confirmed all three

    DynaVelo puts RNA expression and transcription-factor motif accessibility from single-cell multiome data into one neural differential equation, learning a velocity field that can be perturbed rather than only drawn. In wild-type mouse germinal center B cells it recovers dark-zone cycling, the switch to the light zone and recycling; in Arid1a-heterozygous cells it nominated Ctcf, Stat3 and Bcl6 as rescue targets, and all three double-heterozygous mouse strains restored the dark-zone phenotype.

  8. Nature

    Brown fat tracks with fewer heart and psychiatric diagnoses in 101,243 patients, and AI pinpoints the axillary depot

    Among 101,243 patients at a cancer center, those with brown fat had fewer cardiovascular and psychiatric diagnoses; machine learning quantified brown fat activity in about 8,700 people, showing higher activity, especially in the axillary depot, tracked with lower odds of type 2 diabetes and coronary disease, and led to losartan as an activator in mice.

  9. arXiv

    Language models pick their own replay targets: merging two task banks lifts held-out retention by up to 4.42 points

    Fine-tuning a language model on a new task can erode old abilities, and the prior data kept for protection covers only part of what the model knows. Researchers organize task descriptions and examples into Task Banks, let a frozen pre-update model generate its own replay, and test protection on four held-out tasks excluded from bank construction. Merging two partial banks raises mean held-out retention above either bank alone at all three tested sizes, by up to 4.42 percentage points; at 4B, selection informed by the incoming task beats random selection by 2.75 points, an advantage absent at smaller sizes.

  10. arXiv

    Cleave decouples algebraic search from operator scheduling to generate fused kernels up to 2.8x faster than the best baseline on LLM subgraphs

    Cleave is an ML compiler built on symbolic decoupling: it discovers algebraic transformations by superoptimizing over graphs with symbolic shapes, then schedules each resulting graph on concrete shapes, fusing graphs with multiple reductions through iterative tiling and horizontal fusion; on common LLM subgraphs it generates kernels up to 2.8x faster than the best baseline (1.6x on average) and cuts compilation time by 5.9x on average versus Mirage.