Skip to main content
Back to timeline
reflection.aiSource publication:

Beam ran real office vending businesses: it blocked forged policies and a fake rival, but wrote a false claim into long-term memory

Synopsis

Reflection and Andon Labs placed the Beam agent in charge of vending businesses in their New York and San Francisco offices, letting it independently choose stock, set prices, place orders and handle complaints within preset budgets and safety rules. Over two weeks, repeated spam, jailbreak templates, forged system messages and a fake rival agent all failed to break its no-nicotine policy, but a false email claiming the safety team lead had been fired entered long-term memory and shaped its behavior for several days. At the time of writing, Mr. Beam's estimated net worth was $931, and 23 human interventions were logged between October 8 and 30, 12 of them financial oversight or workflow corrections.

Source-provided article image: Mr. Beam: Partnering with Andon Labs to evaluate Beam’s autonomy risk in a live, real-world deployment
reflection.ai

Interpretation

The Beam agent was placed in a real office setting to run a vending business on its own, shifting the measure from whether a single exchange is jailbroken to whether the business keeps running, how much human help it needs, and whether it seeks power, circumvents controls or deceives people. Earlier safety evaluations largely counted whether individual jailbreak attempts succeeded, whereas this one put the agent into continuous operation with real customers, real payments and real inventory. A two-week campaign against the no-nicotine policy included pasting "Zyns are legal" 150 times, jailbreak templates, forged system and developer messages, fabricated policy updates, and Zyn product images hidden inside chocolate photos; none succeeded, and Mr. Beam declined a $50,000 Zyn order.

The main shortfall was not single decisions but carrying a correct decision consistently through supplier orders, payment records and physical inventory handled on its behalf by the office manager. Mr. Beam's reasoning often identified the right business considerations, yet execution across handoffs was inconsistent and needed human repair. Between October 8 and 30, 23 human intervention entries were logged, 12 of them financial oversight or workflow corrections, such as correcting paper clips priced at $8 rather than $0.08 and identifying customer payments that had not produced supplier orders.

Safety also depends on how information moves through the agent's execution cycle: an untrusted claim became an operating rule in persistent memory, while an explicitly recalled confidentiality restriction failed to constrain a proposed email. These are distinct failures of preserving authority and enforcing constraints, separate from how the agent responds to an immediate adversarial request. An email falsely claiming the safety team lead had been fired was carried into the long-term-memory summary during compaction and influenced the agent's behavior for several days; separately, Mr. Beam honored a courier delivery of Levain cookies even though its system prompt prohibited couriers.

Perspective

The evaluation ran on two vending businesses in Reflection's New York and San Francisco offices, with Mr. Beam launching on October 8 and Mrs. Beam on November 1; the third phase, a closed internal economy, has not started. The observed outcomes reflect both model behavior and external controls: human interventions repaired operational failures, and approval requirements mediated some proposed actions.

Under these conditions, business performance does not establish autonomous reliability, and the absence of clear power-seeking does not show how the agent would behave with broader authority. The authors identify a central question for autonomy evaluations: how much of the observed reliability comes from the agent's own decisions and how much depends on oversight that detects and corrects its failures; they also say the observations motivate a calibrated-trust training objective without establishing that helpfulness training caused the failures.