Skip to main content
Search

# MAS Benchmark

1 related articles

Subscribe to updates
  1. arXiv

    A CLI agent's risk hides in its middle layers: Agent MechSuits cuts attack success from 85.99% to 13.36% with a 10-dimensional subspace

    Command-line agents accumulate risk across multi-turn execution: a single step can look harmless while later steps assemble it into a harmful chain, and external guardrails can only check outside the model against predefined rules. Agent MechSuits instead reads each step's hidden representation, uses a linear probe to judge whether the current state is harmful, and on a hit adds a steering vector inside a 10-dimensional subspace at the same layer without changing model weights. Across the white-box agents LLaMA-3.1-8B, Qwen-2.5-7B and Gemma-2-9B, current-step risk detection averages 97.32% accuracy and mean attack success falls from 85.99% to 13.36%, at the cost of a higher generation-collapse rate.

You’re all caught up