Skip to main content
Back to timeline
arXivSource publication:

Expert knowledge, read by an LLM, cuts multi-fidelity design regret by 61% on a battery problem

Synopsis

Multi-fidelity Bayesian optimization pairs a cheap low-fidelity model with a few costly high-fidelity simulations, but when high-fidelity data are scarce the discrepancy caused by physics the low-fidelity model omits is hard to learn from numbers alone. LLM-Assisted MFBO has a large language model compare which of two designs needs the larger signed residual correction, and a pairwise Gaussian process turns those judgments into a continuous knowledge basis whose sign and weight high-fidelity data calibrate. On a lithium-ion battery design problem the method reduced the regret area under the curve of conventional multi-fidelity Bayesian optimization by 61.2%, and a controlled study of four knowledge qualities and four LLMs showed that knowledge quality, not the LLM, governed performance.

Source-provided article image: LLM-Assisted Multi-Fidelity Bayesian Optimization: Fusing Expert Knowledge with Numerical Data
Fig. 1 · arXiv · Page 5

Interpretation

Multi-fidelity optimization gains a second information source: an expert describes in natural language which physics the low-fidelity model omits, a large language model judges which of two designs needs the larger signed residual correction, and that information enters the surrogate alongside low-fidelity data. Earlier multi-fidelity models, including co-Kriging, hierarchical Kriging and nonlinear information fusion, inferred the low-to-high-fidelity relation from numerical observations alone; when the omitted mechanism varies with the design variables, a few high-fidelity observations cannot capture that discrepancy. On the lithium-ion battery design problem, the method's normalized regret area over 20 seeds was 0.0303, 61.2% below conventional multi-fidelity Bayesian optimization at 0.0781, 75.5% below high-fidelity-only at 0.1237, and 65.2% below the knowledge-basis-only variant at 0.0871.

The knowledge basis supplies only the shape of the discrepancy, not the response level; high-fidelity data set its sign and contribution, so expert knowledge is not trusted blindly. With low-fidelity data alone, the low-fidelity coefficient of conventional multi-fidelity Bayesian optimization fell from 0.72 to 0.55, absorbing the unexplained design-dependent discrepancy into the global scale; with the knowledge basis, that coefficient stayed near 0.76 throughout, close to the global scale of 0.766 computed post hoc from 256 low-high-fidelity pairs. A knowledge basis built from only 13 valid comparisons reached a Spearman rank correlation of 0.96 with the true residual discrepancy computed post hoc on 256 designs, and 0.87 among the top 25% of designs by objective value.

The large language model does not predict high-fidelity values; it answers A, B or TIE on actively selected design pairs, where TIE means abstention for insufficient evidence, and a pairwise Gaussian process converts these comparisons into a continuous knowledge basis. Existing LLM-based Bayesian optimization methods ask about the objective itself, such as where the optimum lies or which outcome is better, and route the output into the surrogate mean, candidate generation, low-fidelity data or a utility function; here the question is where the low-fidelity model errs more. In the battery problem, all 13 A/B judgments made by the LLM agreed with the true order over the entire global-scale range from 0.55 to 1.0, whereas 4 of the 11 abstained pairs reversed their order within that range.

Performance is governed by knowledge quality rather than model choice: a factorial study of four knowledge qualities and four LLMs attributed 92.8% of the between-condition performance variance to the knowledge condition and 1.9% to the LLM. Under anonymized no-physics knowledge, all four LLMs abstained on all 96 queries, the knowledge basis vanished, and the method matched conventional multi-fidelity Bayesian optimization exactly; under deliberately reversed misleading knowledge, mean regret area was about 20% higher than conventional. With accurate knowledge, regret area relative to conventional multi-fidelity Bayesian optimization fell by 61.2% with GPT-6.1 Sol, 55.7% with GPT-6 Astra, 37.7% with GPT-6 Sol and 23.9% with GPT-5.6 Sol, with the first three significant after Holm correction.

Perspective

Validation covers one seven-dimensional lithium-ion battery design problem: the high-fidelity model is DFN and the low-fidelity model is SPM, the objective is volumetric discharge energy density under moderate and demanding conditions, and each of 20 seeds runs from 7 initial high-fidelity evaluations through 8 sequential evaluations. The knowledge basis is built once with 24 queries before Bayesian optimization, and high-fidelity data are never fed back to the LLM during optimization.

When knowledge is only partly correct, the coefficient sign alone cannot fix it: under misleading knowledge mean regret area was about 20% higher than conventional, though that degradation was less than half the improvement under accurate knowledge, and among the 16 conditions only the P4 with GPT-5.6 Sol combination stayed significant after multiple-comparison correction. The variance decomposition uses condition means, each from a single query plan generated for that condition; computed per seed, the knowledge effect was larger in all 20 seeds but the ratio fell from 6.3 to 3.2. Predictive variance does not propagate uncertainty from the pairwise Gaussian process or from the coefficient estimates.