StarSling hands CI tuning to agents: Mastra's slowest workflow drops from 29m 56s to 5m 06s
Synopsis
StarSling released GitHub Actions runners bundled with AI agents: each CI job runs in its own ephemeral Zen 5 sandbox, and the agents read workflows, run logs and machine telemetry, propose an optimization, test it side by side against the unchanged workflow, confirm the difference is statistically significant with a deterministic judgment, and ship the winner as a PR. Mastra's slowest workflow fell from 29m 56s on GitHub-hosted runners to 5m 06s on StarSling, and the company reports an average 10x wall-clock speedup with 70% less runner time across several open-source repos. Agents start opening PRs only after about 10,000 minutes of usage.
Interpretation
StarSling bundles CI runners with a set of AI agents that keep improving CI, giving every CI job its own ephemeral sandbox and letting agents file optimization PRs on top of the runners. While building its own agents, the team saw agent PRs back job queues up for hours and push the CI bill close to the inference bill; Mastra's slowest workflow took 29m 56s on GitHub-hosted runners. Mastra now merges over 1,800 PRs a month, and its slowest workflow runs in 5m 06s on StarSling, 5.87x faster than the 29m 56s on GitHub-hosted runners.
The agent loop reads workflows, run logs and machine telemetry, proposes one optimization, runs the unchanged workflow side by side with the proposed change, uses a deterministic judgment to confirm the difference is statistically significant, and ships the winning version as a PR with benchmark evidence. This loop is called the CI harness, and its judging step relies on a statistical significance test of the side-by-side runs rather than a model's subjective call. On open-source repo benchmarks, StarSling reports an average 10x wall-clock speedup with 70% less runner time: benchmark-gogs 12.8x, benchmark-shellcheck 5.5x, benchmark-chatwoot 3.9x, benchmark-transfer.sh 11.7x and benchmark-mirotalksfu 36.4x.
The agents found an 11-month-old performance bottleneck in better-auth, and the fix made one of its test jobs 5.4x faster. That bottleneck had gone unaddressed for 11 months before the agents surfaced it. The change was submitted as PR 10762 in the better-auth repository.
Perspective
The report rests on StarSling's launch post and covers its runners, the CI harness agent loop, the Mastra and better-auth cases, and benchmarks on five open-source repos; these figures come from the vendor and apply to its stated test setup.
The vendor says users can do these optimizations themselves with prompts and has released skills for that; agents start opening PRs only after about 10,000 minutes of usage, with 2,000 free minutes; the CI Challenge promises $1,000 if StarSling cannot save at least 10%, with eligibility and rules in its blog.
