AI Gateway Quickstart
Connect Apinizer to at least one LLM provider — cloud (OpenAI, Anthropic, Azure, Vertex, Bedrock) or self-hosted (vLLM, Ollama) — under AI Gateway → Models. The connection stores the provider's credentials encrypted, so your application never handles the real key.
See LLM Providers and Models for the full setup steps.
Go to AI Gateway → LLM Gateway and click New. Enter a Name and a Relative Path, then save.
On the Routing tab of the new LLM Gateway, select the LLM Provider and the Model in the Default Target Group, and save again.
Deploy the LLM Gateway to an environment. The Endpoints panel at the bottom of the Routing tab then shows its address for each deployed environment, in the form https://<gateway-access-url>/<relative-path>, with a copy icon next to it. A Test button next to the address lets you try it quickly through the Test Console — see Routing and Failover for details.
Configure the OpenAI SDK (or any OpenAI-compatible client) with the copied address plus /v1 as base_url:
from openai import OpenAI
client = OpenAI(
api_key="your-consumer-key",
base_url="https://<gateway-access-url>/<relative-path>/v1"
)
No other client-side changes are needed — the SDK talks to Apinizer exactly as it would talk to OpenAI directly.
A new LLM Gateway has no policies, so Apinizer does not check api_key; the SDK only needs a non-empty value. The key is verified only after you add an authentication policy to the LLM Gateway, and then it must be the key of a Consumer.
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello, Apinizer!"}],
stream=True
)
for chunk in response:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
The response streams back chunk-by-chunk as it arrives from the provider — the same shape you'd get calling the provider directly, regardless of which provider actually served the request.
A new LLM Gateway follows the stream value your client sends. Guardrails, semantic cache and token limits apply only after you add them as policies. If the LLM Gateway was created earlier and its Client "stream" field is Overwrite with the proxy setting, the Streaming (SSE) switch on the Routing tab decides, and a gateway with the switch off returns a single response even for stream=True. See Client "stream" Field.