Vercel's AI Gateway adds Private Inference: self-hosted and fine-tuned models now share one key, SDK and log
Synopsis
Vercel's AI Gateway has added Private Inference, which connects enterprise self-hosted, fine-tuned or prerelease models as private providers. Such models previously needed separate keys, client code and logs; they now share the same authentication, SDKs, APIs and observability as catalog models, and switching only changes the model ID to private/<slug> while the gateway translates each request to the endpoint's format. Private models are visible only to their team, stay out of the public catalog, store credentials encrypted, and can enable Zero Data Retention per provider.
Interpretation
Enterprise self-hosted, fine-tuned or prerelease models can now run through the same authentication, SDKs, APIs and observability as AI Gateway catalog models, so every model a team uses is managed in one place. Previously these models needed separate keys, client code and logs, maintained apart from catalog models. Once connected, private model requests appear in AI Gateway logs alongside catalog requests, and observability charts break down requests, tokens and time to first token by private model.
Private models are called through Chat Completions, Responses or Messages, and the gateway translates each request to the endpoint's own format, so switching between private and catalog models only changes the model ID. Calls keep the existing interfaces, with the model named by a private/<slug> ID such as private/support-fine-tune. Setup means adding the endpoint as a private provider with its base URL, API format and credentials, then adding the models it serves; the provider and model can also be registered with Vercel CLI 65.16.0 or later.
Private models are visible and callable only by their Vercel team, are not listed in the public catalog, store credentials encrypted, and can enable Zero Data Retention per provider. Access scope and credential handling are constrained by the gateway rather than by each endpoint separately. If a private endpoint fails, a request can optionally fall back to a catalog model, for example by listing models in the request's gateway options.
Perspective
The capability targets enterprise users running self-hosted, fine-tuned or prerelease models under a Vercel team, registering endpoints and models through the gateway page or Vercel CLI 65.16.0 or later; supported model types, requirements and billing details are in the Private Inference documentation.
Private models are not listed in the public catalog and are visible and callable only by their team; Zero Data Retention must be enabled per provider, and fallback to a catalog model on endpoint failure is optional rather than default.
