Skip to main content
Search

# Fine-tuning

30 related articles

Subscribe to updates
  1. arXiv

    Fine-tuning a MACE foundation model fixes sodium clustering in hard carbon, cutting formation-energy error from 32.43 to 1.65 meV/atom

    Pretrained MACE atomistic foundation models describe sodium intercalation in graphite-like domains but treat the metallic attraction between sodium atoms on graphene as repulsion, so they miss the clustering and pore filling that shape hard-carbon storage. Targeted fine-tuning on 1,985 sodium-cluster configurations plus 300 expanded graphite and sodium reference configurations lowers the formation-energy mean absolute error on the sodium-cluster data from 32.43 to 1.65 meV/atom and raises R² from -36.32 to 0.87. After fine-tuning on a broader sodium-carbon set, the model still gives a 2.03 meV/atom formation-energy error on sodium-filled carbon nanotubes excluded from training, and it links closed-pore radius to plateau voltage.

  2. arXiv

    UniPipe puts serving and fine-tuning in one CUDA context: on a ptychography stream its miss rate tracks an inference-only deployment

    For scientific workflows that must process sensor data within hard deadlines, UniPipe runs inference serving and online fine-tuning of a deep-neural-network surrogate in a single CUDA context on one edge GPU, handling inference requests and labeled samples in one forward pass and backpropagating only through the labeled subset. On a real ptychographic reconstruction workflow with PtychoNN it lowers inference and reconstruction MSE over a frozen pretrained baseline, reduces the deadline-miss rate over a two-context concurrent baseline, and nearly halves the 95th-percentile latency spread relative to the mean.

  3. bioRxiv

    TabPFN 3.5 matches species distribution models without tuning, and fine-tuning makes it the benchmark's best single model

    Predicting where a species occurs from presence records alone has long relied on models such as MaxEnt that must be tuned for each dataset. On a standard benchmark of 225 species in six regions, TabPFN 3.5 reached a mean AUC of 0.732 without tuning, matching the best published five-model ensemble; fine-tuned on presence–background tasks from other regions it rose to 0.736, the best single model on the benchmark and significantly above MaxEnt's 0.728. The authors caution that six regions cannot establish the advantage in new regions, and that the model is sensitive to environmentally biased records.

  4. arXiv

    Power-system dynamic simulation gets a fast stand-in: fine-tuned Chronos-2 beats a traditional AI model on error and runs 563 times faster

    The pretrained time-series foundation model Chronos-2 was tested as a surrogate for power-system dynamic simulation. On 1,800 load-disturbance samples from an IEEE 9-bus system, retrieval-augmented in-context learning beat random reference selection, and fine-tuned Chronos-2 had lower errors than a traditional spatiotemporal graph neural network baseline on the 178 held-out samples, with power-balance regularization improving every reported metric except voltage-angle MAE. Its 35.5 ms inference was 563 times faster than the 20 s ANDES simulation.

  5. Fireworks AI

    DeepSeek R1 and V3 get quantization-aware fine-tuning on Fireworks, keeping FP8 serving without the accuracy drop

    Fireworks has added quantization-aware fine-tuning to its FireOptimizer adaptation engine, letting teams tailor the open DeepSeek R1 and V3 models to a specific use case. Previously, LoRA-tuned weights merged and served in FP8 lost noticeable accuracy, and the 671-billion-parameter base alone occupies 671 GB in FP8, above the typical 640 GB of an H100 host. The method fake-quantizes merged weights and activations in the forward pass so LoRA gradients stay in bfloat16 while merged weights follow FP8 numerics; on a French-wine review classification task with 15,000 training and 5,000 evaluation records, QAT tuning beat naive FP8 LoRA tuning.

  6. sarvam.ai

    Sarvam's Prism decision model answers in 44 ms and averages 71.8 across three indices, the best average under 100 ms among 25 models tested

    Sarvam released Prism, a decision model built on Sarvam 105B for enterprise workflows that need fast, calibrated judgments. Across 25 decision models Sarvam tested, Prism averaged 71.8 over three indices with a 44 ms median response, the highest average among models answering under 100 ms; higher-scoring models took roughly two to six times as long. The weights are open so enterprises can train on their own data and run on their own infrastructure.

  7. huggingface.co

    NVIDIA folds transcription, speaker labeling and translation into one open-weight model: 40 languages, one-hour audio

    NVIDIA released Nemotron 3.5 Transcribe, an open-weight checkpoint that combines transcription, speaker diarization, translation and audio question answering, covering 40 languages for transcription, 31 languages translated to or from English, and speaker attribution in 13, with recordings up to an hour and up to eight speakers. Previously a transcript could capture every word yet still leave users unsure who said what or how to translate it. Developers steer output with natural-language prompts and adapt vocabulary through Granary v2 data and fine-tuning; in a medical case study, fine-tuning cut word error rate on held-out speakers by 45.8% relative to the best prompt.

  8. Fireworks AI

    Llama 3.2 lands on Fireworks: the 11B and 90B vision models cost the same as text models, with images billed as 6,400 text tokens

    Fireworks announced support for Meta's Llama 3.2 family, covering the text-only 1B and 3B models and the multimodal 11B Vision and 90B Vision models, with inference and fine-tuning available on its platform. That generation was not previously offered there; the 11B and 90B multimodal models are now priced the same as text-only models, and images are charged as text tokens, typically counted as 6,400 text tokens for the Llama 3.2 vision models. Fine-tuning is currently open for the 1B and 3B text-only models, while multimodal fine-tuning is not yet available.

  9. arXiv

    Language models don't learn in-context all at once: control of the answer flips first, and accuracy catches up trillions of tokens later

    Language models do not acquire in-context learning in one step. A controlled diagnostic separates copying the nearest demonstration from composing mappings spread across the prompt; across 17 models about a third prefer the copied answer and the rest the rule-consistent one, and in OLMo-2-7B dependence on the requested value overtakes the conflicting example hundreds of billions of tokens before accuracy rises above chance. Training on synthetic definition-and-query documents flips the preference, while exposure to definitions or suppressing copy-favoring heads does not.

  10. langchain.com

    LangChain moves agent operations into LangSmith: semantic trace search and red teaming before production

    LangChain released a set of LangSmith features at Interrupt NYC and London so teams can own their intelligence in production. Previously, finding conversations meant relying on traditional filters; AI Search now retrieves traces by semantic understanding, so a team can ask whether a user got frustrated or whether PII leaked. The same update adds Managed Deep Agents v0.9, Engine v2 with red teaming, Trajectories, Custom Apps, and Fine-Tuning, covering how agents are built, understood, and improved.