Skip to main content
Back to timeline
bioRxivSource publication:

statLens splits statistical design among code, models and researchers, lifting small-model accuracy from 81.0% to 93.6%

Synopsis

Differential expression analysis requires settling the model family, primary comparison and sample dependence before fitting, decisions previously left to analysts or to code-writing autonomous agents. statLens assigns data profiling and rule checks to code, study-context interpretation to a language model, and final confirmation to the researcher; across 102 tasks, seven models held 92.5–95.0% accuracy, against 17.5–46.8% for LLM-only inference. Mean accuracy for four small models rose from 81.0% as autonomous agents to 93.6%.

Source-provided article image: statLens enables accurate and interpretable statistical intelligence
Figure 1 · bioRxiv · Page 3

Interpretation

The statistical design that precedes differential expression analysis can now be carried out by a human-in-the-loop workflow rather than answered in one pass by a single language model. On the same tasks, LLM-only inference reached 17.5–46.8% accuracy and autonomous agents 58.7–99.5%, with wide variation across models. On 510 multiple-choice questions from 102 tasks drawn from 74 public datasets across six omics modalities, statLens reached 92.5–95.0% accuracy across seven language models, a 2.5-percentage-point spread versus 40.8 points for autonomous agents.

The workflow divides one analysis into seven steps across perception, reasoning and execution modules, and each step's output must pass code checks and researcher confirmation before the next. Code profiles the feature matrix, verifies sample correspondence and checks design-matrix rank and contrast estimability, while the language model only interprets study context and proposes a model family and design formula. Across four small language models, statLens raised mean accuracy from 81.0% as autonomous agents to 93.6%; Gemma 3 12B went from 58.7% to 92.5% and Phi-4 from 84.1% to 93.7%.

In a mouse liver RNA-seq reanalysis, the workflow ran from design card to an executable DESeq2 script and recovered expression changes in the reported directions. The original study reported 1,568 differentially expressed genes; statLens found 1,557 at adjusted P < 0.05, with the discrepancy confined near the threshold. The data were eight samples and 14,371 genes from GSE326073, four DHT-treated and four vehicle-treated mice; all 36 genes highlighted in the original study were significant and changed in the reported directions.

Perspective

The demonstrated scope is differential expression statistical design across six omics modalities plus execution through DESeq2 negative binomial models, with inputs requiring sufficiently informative study descriptions, cleaned metadata and a prepared feature matrix. End-to-end execution currently covers only negative binomial models, and random-effects and nested dependence structures are outside the supported scope.

The benchmark is multiple-choice and counts only answers matching predefined references, so acceptable alternative analyses are not scored as correct; the automated harness advanced the workflow without substantive researcher corrections, so the added benefit of human review is not quantified. The authors also note that the contribution of each component remains to be established through ablation studies.