Skip to main content
Search

# Multimodal Agent

5 related articles

Subscribe to updates
  1. arXiv

    Chemistry figures no longer need manual transcription: RxnMiner lifts stereochemistry-aware reaction F1 from 21.0% to 71.0%

    Reaction figures in the chemical literature often pack several experiments into one shared template, substituent table and product grid, and recovering concrete reaction records from them has meant manual transcription. RxnMiner combines an expert-informed extraction procedure with a chemistry toolbox in one multimodal agent: it reads structures, binds evidence to each experimental entry, and instantiates undrawn reactants with a deterministic R-group engine. On ChemRxn-Bench, 603 reaction figures with 8,547 annotated records, it reaches 71.0% stereochemistry-aware reaction macro-F1 with a Qwen3.8-Flash-Next backbone, against 21.0% for direct prediction and 25.0% for the strongest existing extraction system.

  2. arXiv

    GATOR reconstructs textured 3D objects and scene-relative pose from casual images with a generative and agentic framework

    GATOR is a generative and agentic framework that recovers textured object assets and their scene-relative pose from one or more casual images: a local modality mixer couples patch-aligned RGB, target-mask, and pointmap features before cross-view reasoning, text-guided semantic conditioning supplies category names and object captions through stage-specific adapters for structure, geometry, and appearance generation, and the generated asset initializes a multimodal agent that refines structure and texture through an observation-guided edit-render-review loop; across synthetic objects, cluttered tabletops, and indoor scenes it achieves strong geometric and appearance fidelity while recovering scene-relative pose from sparse observations.

  3. arXiv

    Long-MDR pushes online RL for multimodal deep-research agents to 128k context and 75 tool turns, with a 9B model leading five of six benchmarks among 7B–9B agents at a 50-turn evaluation budget

    This work scales online reinforcement learning for multimodal deep-research agents to 128k context and 75+ tool-interaction turns and introduces Long-MDR, a three-component recipe of On-Policy Distillation Warmup, Progressive Horizon Expansion, and Entropy-Triggered Rescue, to address slow early reward growth and later entropy collapse in long-horizon training; at a 50-turn tool-interaction evaluation budget, the trained Long-MDR-9B ranks first on five of six benchmarks among the compared 7B–9B agents.

  4. arXiv

    FaV-A generates positive-negative space compositions via a staged multimodal agent, with experiments and ablations suggesting gains over direct zero-shot MLLM baselines

    The work presents Form and Void Agent (FaV-A), a multimodal agent for staged positive-negative space generation: it first generates a base object, then analyzes its shape and spatial structure to identify candidate negative-space semantics, and finally produces compositional instructions for the final image generation stage; experiments and ablation analyses suggest FaV-A yields more visually coherent and semantically aligned positive-negative space compositions than direct zero-shot MLLM baselines.

  5. arXiv

    EpiCon builds a shared multimodal memory bank with two 2B models, letting different agent systems reuse each other's experience and lifting macro-average scores by 1.7 to 4.9 points

    EpiCon introduces a shared multimodal memory framework in which two independently trained 2B models, a memory controller and a tree self-organizer, co-evolve question-level textual guidance with visual evidence and link it to a persistent experience bank, so different multi-agent systems can reuse and contribute experience without updating host model parameters; across eleven benchmarks, four multimodal task domains, two harnesses and multiple backbones, a frozen bank improves other systems with a single solving attempt, a second harness raises the original system's macro-average by 2.6 points, the 2B variant improves macro-average scores by 1.7 to 4.9 points over No Memory across four host configurations, and memory-operation time drops 67% to 74% relative to backbone-sized memory models.

You’re all caught up