Skip to main content
Search

# LLM Routing

10 related articles

Subscribe to updates
  1. arXiv

    PISA-E turns ensemble routing into a per-prompt decision, beating the strongest ensemble by 2.09 points when quality comes first

    PISA-E decides both which large language models to combine and how many to invoke for each prompt at inference time, reusing PISA's per-model capability estimates to predict ensemble success without training separate estimators for model combinations. Across four routing benchmarks it reaches macro-average task performance 2.09 percentage points above the strongest ensemble competitor in-distribution and 2.87 points out-of-distribution when quality is prioritized, and at selected lower-cost operating points it also cuts average inference cost by roughly 36% and 25%.

  2. arXiv

    A small model's prefill activations can price output length too: reasoning-model routing saves 9% to 33% on eight of nine datasets

    When routing reasoning models, how many tokens each query will produce is unknown in advance, and most routers price candidates at a fixed or median length. One approach reads the prompt's prefill activations from a frozen small model to predict both each candidate's success probability and its output length; on eight of nine datasets, pricing by predicted length spends 9% to 33% less than median-length pricing at matched accuracy, up to 69% of what perfect per-query cost knowledge would save.

  3. arXiv

    Agents stop routing on the current request alone: session history lifts pass rate 3.3 points at matched offloading

    Model routing for multi-turn agents is studied at the user-turn boundary, where a router picks one model for the whole turn before it runs. At a matched offloading rate of roughly a fifth of turns, a router that reads session history passes 28.3% of SheetCopilot tasks against 25.0% for the same router seeing only the current request, a 3.3-point gap, while staying within one point of always using the expensive model. A closed-form cache analysis finds downward switches repay their cache cost in about two calls, while upward switches do not break even under the evaluated conditions.

  4. arXiv

    Routing stops ranking answers by one score: security queries get their own quality, closing nearly 33 points of the security gap at the same models and budgets

    SCRouter splits routing quality into utility for ordinary queries and security for security-sensitive ones, weights the two per query, and recovers 60.9% of the gap between random selection and the per-query oracle on its own benchmark, 12.9 points above the previous best router R2-Router.

  5. arXiv

    SWAP routes stepwise among VLA policies via offline RL, lifting real-world DROID task success from 63.3% to 96.7%

    SWAP casts robot policy routing as an offline reinforcement learning problem, training an IQL routing critic that picks, after each action chunk, the candidate pretrained VLA policy with the highest predicted long-horizon return given the current observation; on real-world DROID tabletop manipulation it raises success from 63.3% for the best single policy to 96.7% while cutting average action steps for successful trajectories from 269 to 193 (a 28.3% reduction), and it outperforms fixed-policy and random-routing baselines in LIBERO and LIBERO-Plus simulation.

  6. arXiv

    Semantic entropy flags small-model errors on GSM8K at AUROC 0.871, but its synthetic-arithmetic counterpart is fully matched by a regex difficulty baseline

    The work tests semantic entropy as an escalation signal for routing queries from a small to a large language model across three benchmarks and two model families, finding that on GSM8K it reliably distinguishes the small model's mistakes (AUROC 0.871) and improves routed accuracy over random escalation by up to nine points at matched cost, while an equally strong-looking synthetic-arithmetic result is almost exactly matched by a model-free rule based only on how hard the question looks on the page; it then offers a checklist of controls, including a difficulty baseline, three escalation targets, an oracle-headroom pre-flight check, per-generation cost accounting, and a difficulty probe that predicted in advance a retrieval shortcut's collapse from AUROC 0.908 to 0.518.

  7. arXiv

    RouteFM turns LLM routing into a reusable pretrained capability, beating the strongest baseline by 2.23 quality points on the unseen multimodal MMR-Bench with only eight observations per candidate

    The work introduces RouteFM, reframing large language model routing from environment-specific local fitting into a pretrain-once, route-anywhere foundation-model paradigm: episodic pretraining across heterogeneous routing environments learns a reusable, target-conditioned candidate-comparison capability in which candidates are represented only by anonymous behavioral evidence (historical query embeddings, observed quality, normalized relative cost) with no model names or identity embeddings, and at inference the router stays frozen while adapting to new domains, modalities, candidate pools, and smaller context budgets through context alone, outperforming the strongest non-RouteFM baseline by 2.23 quality points on the pretraining-excluded MMR-Bench with eight observations per candidate.

  8. arXiv

    FlexRouter picks complementary model subsets with determinantal point processes, lifting average Success@10 to 0.8632 and 0.9914 on RouterEval

    The work reframes LLM routing as coverage-oriented subset selection, uses a determinantal point process to capture both model competence and inter-model redundancy, and trains with a failure-set-marginalization objective that directly maximizes the probability that at least one selected model answers correctly, achieving higher average Success@10 (0.8632 and 0.9914) and more diverse subsets on the medium-pool (3811 candidate LLMs) and large-pool (5000 candidate LLMs) RouterEval settings, while an adaptive greedy stopping rule sets subset size per query.

  9. arXiv

    SeLMRoute turns query requirements into a probabilistic state via 16 interpretable semantic probes, reaching 72.08% average accuracy on LLMRouterBench

    SeLMRoute splits LLM routing into three stages: a decision model first answers 16 interpretable semantic questions about each query and keeps the probability distributions as a 40-dimensional semantic state, a lightweight supervised regressor then predicts each candidate model's performance, and deployment objectives are applied last; on LLMRouterBench (15 datasets, 20 candidate models, 11,481 queries) it reaches 72.08%±0.45 average accuracy and 72.64% under grouped five-fold out-of-fold evaluation, versus 69.23% for the strongest fixed candidate, and in a 13-model performance-cost setting it improves performance in all five grouped splits with a mean PerfGain of 2.66%.

  10. arXiv

    SaveRouter Cuts LLM Routing's Break-Even Deployment Volume by Up to About 9.5x Using Roughly a Third of the Supervision

    The work proposes SaveRouter, a sparse-supervision routing framework that selectively acquires informative query-model feedback and shares capability information across related queries, using only about 33-41% of available training feedback across four routing benchmarks while maintaining competitive or better routing quality and reducing the break-even deployment volume by roughly 1.9-9.5x relative to the fastest conventional fully supervised router.

You’re all caught up