Skip to main content
Back to timeline
arXivSource publication:

LLMs can analyze data without reading it: metadata-only code generation makes inference cost independent of data volume and reruns

Synopsis

Metadata-Driven AI confines the LLM's role in data analysis to writing programs: the model receives only metadata about the data, never the data itself, and the program it returns runs where the data lives and is kept. Repeating the same analysis on new data then needs no further inference, so inference cost stops growing with data volume and with the number of runs; on four public tabular datasets the metadata sent to the model was one to three orders of magnitude smaller than the raw data.

Source-provided article image: Metadata-Driven AI: An Architectural Pattern for Sustainable Data Analysis with LLMs
arXiv

Interpretation

In data analysis, an LLM can write a reusable program from a description of the data alone, while the data itself never enters the model's context. The common routes either read the data into the model's context and answer directly, or let a code-interpreter agent generate code, run it in a sandbox and feed execution outputs back to the model; both send the data or fragments of it to the model and pay for inference on every run. On four public tabular datasets from vega-datasets, the byte ratio of metadata to raw data was 302 for birdstrikes, 134 for seattle-weather, 59 for disasters and 21 for gapminder-health-income, so metadata was one to three orders of magnitude smaller, and the ratio grows with the number of rows.

The approach splits an analysis into design time, where the model generates a program from metadata, and run time, where the program executes next to the data without calling the model. The generated artifact, called a formula, is a readable, deterministic program that declares the columns or document anchors it depends on; once stored it can be applied to new data of the same structure, and a failed dependency check stops the run before a wrong result is produced. On the 10,000-row, 14-column birdstrikes table, the metadata description was 4,018 bytes, about 1,000 tokens, against roughly 300,000 tokens for the whole table; after one generation, later files of the same structure run the stored program with no inference.

The generator need not be a frontier cloud model; a small local model can handle the code-writing step. The benchmark grades generated formulas mechanically by executing them and comparing their output with golden values computed from the data, across aggregation, regression and refinement tasks. The 8-billion-parameter open-weight model qwen3:8b passed all three tasks on a laptop (Apple M5, 16 GB RAM, via Ollama) at 22.6 s latency; in the same table gemini-3-flash-preview at low thinking scored 100% on all three tasks at 7.4 s. The authors note this is indicative only: three tasks, three runs, one dataset.

Perspective

The pattern targets data-analysis tasks that can be expressed as a program, such as aggregation, extraction, joining external APIs and producing charts; tasks that require reading and interpreting the data itself, such as summarizing free text or judging tone, fall outside its scope. The three case studies cover geospatial photovoltaic prospection over live public APIs, extraction of portfolio positions from bank statements, and a conversational assistant over smart-city dashboards, and the authors state that these case studies are evaluated qualitatively, with quantitative evaluation left for future work.

Metadata is not zero information: value ranges, category lists and string examples are real values, and for documents a whole sample crosses the boundary; execution-log feedback can also carry data, and the current implementation does not yet provide redaction. A stored formula can silently return plausible but wrong results when the data's meaning changes without its structure changing, and the authors have not yet evaluated such drift detection. The small-model benchmark covers three tasks, three runs and one dataset, which the authors call indicative.