Skip to main content
Back to timeline
arXivSource publication:

Stress-testing a black-box ship collision-avoidance system: two target ships steer themselves, and the R-PPO policy finds collisions in 68 of 100 scenarios

Synopsis

Safety assessment of ship collision-avoidance systems usually relies on simulation, but in co-simulation each subsystem exposes only its interfaces, so conventional stress testing written as a fully observable Markov decision process cannot see the full state. EB-ASTv2 reformulates the search as a partially observable Markov decision process, uses recurrent proximal policy optimization (R-PPO) to pick adversarial actions from observation history, and lets two target ships trigger intermediate waypoint sampling independently. Across 100 traffic scenarios in a ship-in-transit setting, the R-PPO policy produced 68 own-ship collision scenarios, versus 58 for PPO and 54 for SAC; over five independent training runs the mean collision rate was 66.80%, against 65.20% for PPO and 51.80% for SAC.

Source-provided article image: Co-simulation-Based Safety-Assessment of Black-Box Maritime Cyber-Physical Systems
arXiv

Interpretation

Failure-scenario search for ship collision-avoidance systems can now run inside a black-box co-simulation that exposes only interface outputs, without requiring the testing agent to read each subsystem's full internal state. Earlier adaptive stress testing framed failure discovery as a fully observable Markov decision process, with the agent choosing disturbances from the current state; in co-simulation, subsystems typically exchange information only through inputs and outputs, with internal dynamics sealed inside compiled models. Across 100 traffic scenarios in the ship-in-transit setting, the EB-ASTv2 policy with R-PPO produced 68 scenarios in which the own ship collided with a target ship, versus 58 for the PPO policy and 54 for the SAC policy.

The method writes stress testing as a partially observable Markov decision process, so the policy infers the current situation from the history of observations and actions rather than from a single current observation. Recurrent proximal policy optimization (R-PPO) replaces the actor and critic networks of proximal policy optimization, a reinforcement-learning algorithm that limits how far each policy update may move, with long short-term memory networks so the policy retains history. Training used the ship-in-transit co-simulation, with an own ship carrying the collision-avoidance system and two target ships controlled by the agent, over 100 traffic scenarios generated by the Ship Traffic Generator, one drawn at random on each reset.

Multi-ship adversarial control runs through an event-activity mask: each target ship changes its waypoints only when it triggers its own decision event, and actions for the other ships leave their trajectories unchanged. The original EB-AST controlled a single adversarial ship and had only a scope angle as the intermediate-waypoint action; EB-ASTv2 adds a variable sampling scope length as a second action component. Over five independent training runs, R-PPO averaged a 66.80% collision rate with standard deviation 2.17, PPO 65.20% with 4.97, and SAC 51.80% with 5.36; the highest single-run score was PPO's 71.0%.

Perspective

The evaluated scope is a ship-in-transit co-simulation: one own ship with a simplified collision-avoidance system and two agent-controlled target ships that can sample intermediate waypoints mid-voyage, with training and test scenarios produced by the Ship Traffic Generator. The intended reader is an engineering researcher working on simulation-based safety assessment and reinforcement-learning testing.

The collision rate is the share of simulated scenarios that produce a collision, not a real-vessel failure probability; the simplified collision-avoidance system can address only one target ship per decision interval, a known limitation that is the vulnerability being exploited. The policy does not find a failure scenario for every initial condition, and in some cases the reachable trajectories of the two ships cannot intersect the own ship's trajectory.