Skip to main content
Search

# Multimodal Large Language Model

7 related articles

Subscribe to updates
  1. arXiv

    Mountainous first arrivals go to a multimodal agent: FA-Agent reads the gather and deletes picks it cannot judge

    First arrivals in mountainous seismic data are often weak, intermittent, and masked by strong direct waves and noise, so energy-based pickers jump to later arrivals or noise, and deep-learning pickers often need retraining for a new survey. FA-Agent combines a multimodal large language model, deterministic processing tools, and skill files that carry processing experience into an extensible framework: the agent surveys the whole gather to mark doubtful zones, then examines them enlarged and decides per segment whether to keep, correct, or remove each pick. On semi-synthetic gathers with known first arrivals and real mountainous noise, it picked clearly visible first arrivals accurately and removed picks where the first arrival was invisible instead of picking later events.

  2. arXiv

    Xiaomi's 4B multimodal model sends mask supervision into the shared backbone, gaining 5.6 points on referring comprehension

    Xiaomi released Xiaomi-Lens-0, a 4B multimodal model that decodes region masks from its own visual representations instead of relying on an extra segmentation encoder. Earlier multimodal models either took spatial detail from a separate segmentation encoder or kept the vision encoder frozen, so mask loss never reached the shared features. Xiaomi-Lens-0 pairs vision-encoder patch features with visual latents, the language model's states at generated mask tokens, in a lightweight decoder, so pixel-level supervision reaches the backbone shared with language generation; it gains 5.6 points on average over Qwen3.5 across all eight RefCOCO, RefCOCO+, and RefCOCOg splits and reaches the highest segmentation cIoU among compared models.

  3. arXiv

    UniData turns a simple requirement into multi-round, nine-modality instructions, beating Self-Instruct and VIGC on reasonableness, clarity, detail and relevance

    The authors introduce UniData, a universal multimodal instruction generation pipeline that first expands user keywords into diverse events, then uses a LLaMA-3-backed multimodal instruction generator with image and music tokenizers to produce instructions round by round, correcting repetition and irrelevance at inference via inter-round correlation; they also build UniDataset with nine modalities, 20,000 entries and 17.5 rounds on average, and report that its data quality exceeds Self-Instruct and VIGC on all four GPT-guided metrics while fine-tuning on the generated data improves other multimodal models' understanding and generation.

  4. arXiv

    Zero-shot language reasoning verifies nearby cross-view candidates but cannot rerank a visual retriever's mistakes

    With no component trained, this work prompts a multimodal large language model to convert ground panoramas and satellite tiles into structured XML descriptions and localizes by comparing them as text; on 9,826 VIGOR pairs from four U.S. cities, the descriptions are faithful but not discriminative, full-pool retrieval almost never returns the correct tile, structured verification over ten geographically adjacent candidates reaches 21.0% Recall@1 with field-level rationales and matches TF-IDF, and on the queries a trained visual retriever ranks wrongly, image reranking recovers a substantial share while text reranking stays near chance.

  5. arXiv

    ArticuTable reconstructs tabletop scenes with executable part-level articulation from a single RGB image, reaching the lowest LPIPS of 0.2398 and highest DINOv2 of 0.8991 on 150 reference images and ranking first in 85.59% of user-study evaluations

    The work presents ArticuTable, a single-image tabletop scene reconstruction framework in which GRAM converts imperfect monolithic proxy meshes into executable URDF articulated assets via MLLM-guided joint fitting and semantic state reasoning, while PSGSR progressively registers point-cloud, top-view, and input-view constraints with structure-aware semantic pose selection to resolve yaw ambiguity; on 150 generated reference images it achieves LPIPS 0.2398, DINOv2 0.8991, and CLIP 0.9265, outperforming ACDC, Gen3DSR, MIDI, and TabletopGen with the lowest collision rate of 0.3819%, ranks first in 85.59% of user-study evaluations, and releases 100 USDZ scenes directly loadable in Isaac Sim.

  6. arXiv

    Fact-Flow uses LLM-bootstrapped multi-label fact guidance to raise factual accuracy of MLLM medical reports on tuberculosis and ophthalmology data

    The work introduces Fact-Flow, a framework that decouples visual fact identification from report generation: an LLM automatically extracts and merges clinical finding labels from training reports (7 labels for tuberculosis, 42 for ophthalmology), a multi-label classifier predicts findings, and the predicted labels are serialized into a prompt to guide an MLLM; on the tuberculosis chest X-ray dataset MedGemma + Fact-Flow reaches RadFact F1 0.3055 versus 0.2266 for MedGemma alone, and on the ophthalmology dataset Qwen2.5-VL + Fact-Flow is best on most NLG metrics.

  7. medRxiv

    The Multimodal Anonymizer: a fully local multi-agent AI system for medical data deidentification

    The study developed and evaluated the Multimodal Anonymizer, a modular, locally deployable multi-agent framework integrating multimodal large language models, task-specific neural networks, and rule-based transformations; on benchmarks spanning text, tables, PDFs, imaging, metadata, filenames, audio, handwriting, and 3D imaging, its best local configuration (orchestrator Qwen3-VL-235B-A22B-Thinking) achieved 98.80% per-patient deidentification sensitivity (95%-CI 97.20; 100) and 99.60% critical clinical preservation (95%-CI 98.80; 100), reached 100% sensitivity and critical preservation on 250 local Charité partograms, and outperformed established tools across most modalities.

You’re all caught up