Skip to main content
Back to timeline
arXivSource publication:

Agri-MRAG stops pasting retrieved text into farm queries: evidence becomes LoRA adapters, reaching ROUGE-L 46.58

Synopsis

Agricultural visual question answering must read crop symptoms and call on pathology knowledge at once, and general multimodal models often confuse visually similar diseases. Agri-MRAG clusters retrieved image-text evidence by semantic consistency, keeps only a high-confidence subset, encodes that evidence as LoRA adapters injected into the model's feed-forward layers, and uses perplexity to decide whether to add explicit retrieved context. On PlantVillageVQA and the newly built PlantVQA it reaches ROUGE-L of 41.21 and 46.58, 11.70 and 4.51 points above mRAG and 8.80 and 3.39 above VisRAG.

Source-provided article image: Agri-MRAG: Parameter-Enhanced Multimodal Retrieval-Augmented Generation for Agricultural Visual Question Answering
arXiv

Interpretation

Retrieval augmentation for agricultural visual question answering no longer pastes every retrieved image-text pair into the prompt; it first selects a semantically consistent evidence subset and then merges that evidence into the model as parameters. Earlier mRAG and VisRAG relied only on prompt-level text concatenation with a fixed retrieval count, so long or noisy context could distract attention. On PlantVillageVQA, Agri-MRAG reaches ROUGE-L 41.21, 11.70 points above mRAG's 29.51 and 8.80 above VisRAG's 32.41; on PlantVQA it reaches 46.58, 4.51 and 3.39 points above the two.

Retrieved agricultural knowledge is encoded into low-rank adapters that are dynamically concatenated at inference and injected into the feed-forward layers of the multimodal model, so external knowledge shapes generation through parameters rather than text. The method extends P-RAG-style parametric knowledge fusion, previously confined to text-only data, to multimodal image-text scenarios, with one independent adapter per knowledge sample. Experiments use Qwen3-VL-2B as the base model with LoRA rank 2 and scaling factor 32 on PlantVillageVQA and PlantVQA; in the ablation, adding the adapter on top of clustering raises PlantVillageVQA ROUGE-L from 34.93 to 38.88.

A perplexity-guided route uses the model's own uncertainty to decide whether to add explicit retrieved context, so confident queries rely on parametric reasoning alone. Without routing, simple questions still carry lengthy explicit context while complex ones may lack fine-grained evidence. At Top-K 10, Agri-MRAG's context length is 71.71 tokens per sample versus more than 172 for both mRAG and VisRAG; in the ablation, adding PPL routing raises PlantVillageVQA ROUGE-L from 38.88 to 40.75.

Perspective

Evaluation covers the public PlantVillageVQA (154,977 training QA pairs used and 4,000 test QA pairs randomly sampled) and the newly built PlantVQA (7,514 images and 15,028 expert-verified QA pairs, with 2,990 test pairs), aimed at pest and disease identification in crops such as jujube, fragrant pear, apple, citrus and grape. BLEU, ROUGE-L and METEOR measure textual agreement with reference answers rather than diagnostic correctness itself; a fuller assessment of diagnostic correctness would require expert evaluation, which the authors leave as future work.

The cluster count, Top-K and perplexity threshold were not chosen on an independent validation set but compared on the test sets of the current datasets; the authors note that other datasets should use an independent validation set. Failure cases remain from fine-grained visual localization and discrimination errors, and the perplexity route does not detect such perceptual errors. Adapter storage is non-negligible: full training yields 154,977 adapters occupying 801 GB for PlantVillageVQA and 12,038 adapters occupying 63 GB for PlantVQA, and the experiment of building adapters only for high-frequency samples has not yet been carried out.