Skip to main content
Back to timeline
OllamaSource publication:

Agents stop holding API keys: Ollama cloud models reach Fly.io sandboxes through a gateway

Synopsis

Ollama Cloud now runs inside Fly.io Sprites, the full Linux virtual machines with their own persistent disks where coding agents such as Claude Code, Codex and Gemini CLI execute commands. Previously an agent inside such a sandbox needed its own usable model-service key. The new Ollama Connector keeps the key encrypted in the Fly.io organization, and a gateway identifies which Sprite is calling and attaches the key, so the Sprite only knows a gateway URL and a connection ID. New connectors block every Sprite by default, access is granted by label, name prefix or all, and calls only work from inside a Sprite.

Source-provided article image: Run AI agents safely with Ollama and Fly.io
Ollama

Interpretation

Coding agents running in a sandbox can call Ollama cloud models without the key entering the sandbox. Previously, an agent calling a model service had to hold a usable API key itself. Claude Code, Codex and Gemini CLI are supported out of the box and let agents run commands securely in a sandbox; the connector page shows the call command with the connection ID filled in.

The key is stored by the Fly.io organization and attached by the gateway as the request is forwarded. The gateway first checks which Sprite is calling, then adds the key to the request, while the Sprite itself only receives a gateway URL and a connection ID. Fly.io encrypts the key and never returns it through the API; new connectors block every Sprite by default, access is granted by label, name prefix or all, and calls only work from inside a Sprite.

Integration follows existing SDK and command-line habits, with no agent-specific rework. Any OpenAI SDK works by pointing base_url at the gateway and using any value for api_key, and Ollama's native /api/* routes go through the same gateway. Sprites ship with a skill that tells Claude Code, Cursor, Codex and Gemini CLI about Ollama; if the Sprite has access, the agent can use Ollama models without any setup.

Perspective

Intended for developers running coding agents in Fly.io Sprites who want to use Ollama cloud models. It requires an Ollama API key and at least one Sprite, plus adding an Ollama connector under Sprites, then Connectors in the Fly.io dashboard, choosing which Sprites may use it, saving, and checking reachability with Test reachability.

New connectors block every Sprite by default, so access must be granted explicitly by label, name prefix or all; calls only work from inside a Sprite. The key is encrypted by Fly.io and never returned through the API, and the Sprite holds only a gateway URL and a connection ID.