AI Gateway

AI Gateway Concept

AI Gateway is the AI layer added to API Gateway — the same high-performance Data Plane component that already processes your API traffic, not a separate product. It exposes a single, OpenAI-compatible entry point in front of multiple LLM providers, so client applications integrate once and Apinizer handles the rest.

Single Entry Point

One OpenAI-compatible interface in front of multiple LLM providers, cloud or self-hosted.

Shared Governance

The same security, traffic management, and policy model already used for ordinary API traffic.

Key Capabilities

Multi-Provider Routing & Failover

Route requests across providers and models, with automatic failover when one is unavailable.

Guardrails & Privacy

Mask personal data and secrets, and block prompt-injection and other unwanted request patterns.

Token Quota & Cost Governance

Enforce token and USD budgets at multiple ownership levels, with pricing and cost reconciliation per model.

RAG & Knowledge

Ground responses in your own documents through knowledge bases and retrieval-augmented generation.

Tool & Agent Integration

Connect LLMs to external tools and other agents through the Model Context Protocol and Agent-to-Agent protocol.

Tracing & Reports

Trace individual requests end to end and report usage by person, team, model, and deployment type.

AI Gateway and API Gateway Relationship

An AI Gateway runs through the same kind of request pipeline as any other API Proxy — security and traffic-management policies apply exactly as they already do — with AI-specific stages (guardrails, RAG, quota, routing to an LLM provider) added to that pipeline rather than replacing it.

AI Gateway and API Manager Relationship

Configuration and deployment of AI Gateways work the same way as any other proxy type:

API Manager (Control Plane)
   │
   │ 1. AI Proxy Configuration
   │ 2. Deployment
   │
   ▼
AI Gateway (Data Plane)
   │
   │ 3. Request Processing (guardrails, RAG, quota, routing)
   │ 4. LLM Provider Call
   │ 5. Sending Metrics
   │
   ▼
LLM Provider / Client

Shared Platform Services

AI Gateway does not duplicate platform infrastructure — it reuses what already exists:

Credentials

Provider API keys are stored and encrypted the same way as any other Apinizer credential.

Distributed Cache

Token quotas, throttling, and the semantic cache share the same Cache Component as the rest of the platform.

Analytics & Observability

Usage, cost, and guardrail reports run on the same reporting engine used for API traffic.

Next Steps

AI Gateway Overview

See the full request pipeline and supported providers

AI Core Concepts

Review the terminology used throughout AI Gateway

API Gateway

Learn about the Data Plane component AI Gateway extends

API Manager

Learn about the Control Plane component that configures and deploys AI Gateways