Skip to main content
Back to timeline
arXivSource publication:

SQL validation stops reading whole queries: splitting into blocks lifts human error-catching from 36% to 91%

Synopsis

SQL generated by large language models can run and return plausible rows yet fail to answer the intended question, and without a trusted reference query the only recourse is for a person or model to read the whole query. DecoVal splits a candidate into smaller blocks that recompose to it, gives each block its own sub-question and input description, and picks the split expected to suit a given validator. In a user study with 20 SQL-proficient students, decomposition raised the share of errors caught from 36% to 91%, with no false rejections of correct queries.

Source-provided article image: Optimizing Query Decomposition for Reliable Validation
arXiv

Interpretation

Validating an LLM-generated SQL query no longer asks the validator to hold every join, nested subquery and aggregation in mind at once; the candidate is cut into blocks, each carrying one local judgment. The prior setting presented the whole query as a single judgment, and enterprise questions often involve multiple joins, nested subqueries and several levels of aggregation, which makes one-shot judgment error-prone. In a user study with 20 SQL-proficient students, whole-query validation caught 36% of errors (4 of 11), while decomposition caught 91% (10 of 11); correct candidates were accepted in all six validations under each presentation.

The split is chosen by how reliably a given validator is expected to judge each block, trading smaller, easier blocks against the larger number of judgments they require. Decomposition has a cost: smaller blocks lighten each judgment but add boundaries, add interface context to absorb, and can separate information one judgment needs, so the best granularity differs by validator. On 305 questions from BEAVER and Spider 2.0 with three LLMs as validators, DecoVal raised error-detection recall by 7.0 to 16.9 percentage points over whole-query validation while keeping precision above 90%.

Perspective

The demonstrated scope is a 20-participant user study on two questions from BEAVER's dw split, plus experiments on 305 questions from BEAVER and Spider 2.0 with three LLMs as validators; DecoVal supports only queries it can import, decompose, and print such that the SQL passes three checks, and five candidates using LATERAL or a table function received no verdict from any method.

The reliability model uses preset default thresholds, which the authors note could instead be fitted to a validator's past verdicts; rewrites currently only reorder joins and move filters, and whether decorrelation or other rewrites help validators remains open; block requirements and input descriptions come from LLMs, and more reliable task assignment is also open.