VGBench's 1,018 controlled items show voice agents recognize the tool but rarely stay silent when the speaker changes
The authors introduce VGBench, a 1,018-item diagnostic benchmark that tests whether audio LLMs condition execution on acoustic and conversational context across side-talk, self-talk, and speaker-switch scenarios under one silence/tool-call/natural-language action space; six raw audio LLMs and three training-free adaptations often identify the target tool yet rarely withhold action under the speaker switch (highest raw switch mute rate 14%), while a VoxGate post-training case study raises switch muting to 91.3% while keeping tool selection correct for nearby wearer commands and text-only controls.