Retrieval-augmented answers read well but resist tracing: retrieving implicit questions lifts textbook adherence by 34% on average
Retrieval-augmented generation (RAG) lets large language models answer from textbooks, yet their explanations often mix in prior knowledge that cannot be traced back to the retrieved text. Across 90 Stack Overflow questions and three programming textbooks, plain RAG reached only 22%–40% median source adherence, versus 0% without retrieval. The authors introduce chain-of-illocution prompting (CoI): implicit explanatory questions are extracted from the textbook, evidence is retrieved for a small set of them, and the explanation is then generated; adherence rose significantly for most models, by 34% on average, though absolute levels remain moderate.