Skip to main content
Back to timeline
mastra.aiSource publication:

Mastra agents can now call self-hosted open-weight models: a private gateway keeps requests on your own network

Synopsis

Mastra introduced private model gateways, letting agent requests route to self-hosted open-weight models. Previously, using a hosted provider meant sensitive data left the local network and HIPAA or GDPR compliance became more complicated unless data retention agreements were in place; now a gateway gets a custom ID and the agent config references a model such as private/ollama/gpt-oss:20b. Self-hosting removes the third party, costs cover infrastructure rather than tokens, and an environment variable can hide external providers.

Interpretation

Mastra agents can now send model requests to self-hosted open-weight models instead of only to hosted providers. Using a hosted provider previously meant data left the local network and HIPAA or GDPR compliance became more complicated unless data retention agreements were in place. Self-hosting removes the third party so sensitive data does not reach models outside the network; costs may also fall because only infrastructure is paid for, not tokens.

The mechanism is a custom model gateway that handles every model call. A gateway supplies a custom ID and display name, plus fetchProviders() for the models it offers, buildUrl() for the call base URL, getApiKey() for the key sent with each call, and resolveLanguageModel() to create the language model. The example points the gateway at a local Ollama server whose OpenAI-compatible API sits at http://localhost:11434/v1, and references the model as [gatewayId]/[provider]/[model], such as private/ollama/gpt-oss:20b.

Perspective

The configuration targets developers using Mastra agents who want self-hosted models; the example setting is a locally running Ollama server with the gpt-oss:20b model.

The compliance and cost statements are conditional: hosted-provider compliance eases when data retention agreements exist, and costs may be reduced rather than certainly are.