Downloaded GUI agents can ship with a backdoor: IDEA audits each step before it runs and averages 0.972 detection across five trigger families
Open-weight GUI agents circulating on public model hubs can be backdoored before release, passing acceptance tests on clean inputs and switching to an attacker-chosen action only when a visual trigger appears on screen. IDEA treats the clean-step relation, in which the instruction selects the action and the evidence lies where the instruction points, as its baseline, measures decision-state displacement under two interventions, instruction redirection and candidate-region ablation, and calibrates a threshold against clean steps at the same state distance. On Qwen2.5-VL-3B across five trigger families at a 1% target false-positive rate, IDEA averages 0.972 detection with a worst family of 0.898 and returns a trigger region with every alarm.