Cache Component

Cache Component Concept

Performance Improvement

Reduces response time

Backend Load

Lightens backend load

High Availability

Protection against backend errors

Cost Savings

Reduces backend resource usage

Cache Component Types

Local Cache

Configuration Cache

• API Proxy configurations
• Policy definitions
• Routing information
• Metadata information

Token Cache

• OAuth2/JWT tokens
• Token validation results
• User session information

Response Cache

• API responses
• Cache key-based storage
• TTL (Time To Live) management
• Can be configured as local or distributed

ipucu

When response cache is configured as local, Hazelcast is not used. When configured as distributed, Hazelcast is used and shared among all API Gateway instances.

Distributed Cache (Hazelcast)

Hazelcast

Distributed cache infrastructure

• Multi-node support
• High availability
• Management and monitoring can be done in Apinizer
• Can run in independent namespaces from Gateway pods

Shared State

Data such as throttling and quota are stored distributed

• Throttling data
• Quota data
• Shared state information
• Cross-namespace access support

Response Cache

Response cache can be both local and distributed

• Local response cache: Hazelcast is not used
• Distributed response cache: Hazelcast is used

Cache Invalidation

Automatic and event-based invalidation

• Automatic invalidation
• Event-based invalidation
• Pattern-based invalidation

bilgi

Hazelcast is managed and can be monitored by Apinizer. When distributed cache is used, data such as throttling and quota are shared among all API Gateway instances.

Namespace Independence:

Cache Server pods can now run in different Kubernetes namespaces than Gateway pods. Gateway pods can access cache servers in other namespaces using Kubernetes service discovery (e.g., http://cache-http-service.apinizer-cache.svc.cluster.local:8090). This provides more flexible infrastructure management and allows you to separate Gateway and Cache workloads. For Cache Server configuration, see the Distributed Cache page.

Cache Strategies

Cache-Aside (Lazy Loading)

1. Search in cache
2. If not found, get from backend
3. Save to cache
4. Return to client

Write-Through

1. Write to backend
2. Write to cache
3. Return to client

Write-Back (Write-Behind)

1. Write to cache
2. Return to client
3. Write to backend asynchronously

Cache Configuration

Cache Key Strategy

URL Based

URL path is used as cache key

Example: GET /api/v1/products → Cache Key: /api/v1/products

Query Parameter

URL and query parameters are included in cache key

Example: GET /api/v1/products?category=electronics → Cache Key: /api/v1/products?category=electronics

Header Based

URL and specific header values are included in cache key

Cache Key: URL + Header values

Custom Key

Creating custom cache key

User-defined cache key strategies can be used.

Cache TTL (Time To Live)

Fixed TTL

• Keeping in cache for a specific period
• Example: 5 minutes, 1 hour

Dynamic TTL

• TTL based on response headers
• Cache-Control header
• Expires header

Conditional TTL

• Conditional TTL
• Based on response status code
• Based on content type

Cache Invalidation

TTL Expiry

• Automatic deletion with timeout

Manual Invalidation

• Manual cache clearing
• Invalidation via API

Event-Based

• Event-based invalidation
• Invalidation with webhook

Pattern-Based

• Pattern-based invalidation
• Wildcard invalidation

Cache Usage Scenarios

Response Caching

1. First request: Get from backend, save to cache
2. Subsequent requests: Return from cache
3. After TTL: Clear cache, get from backend again

Token Caching

1. Token validation: Check if token exists in cache
2. If in cache: Return without validation
3. If not in cache: Validate from Identity Manager, save to cache

Configuration Caching

1. API Proxy configuration: Get from Management API
2. Save to local cache
3. On configuration change: Invalidate cache

Throttling and Quota

1. Throttling and quota data are stored in distributed cache
2. Shared among all API Gateway instances (even if in different namespaces)
3. Synchronized via Hazelcast
4. Gateway pods access cache servers in different namespaces via Kubernetes service discovery

Cache Best Practices

TTL Management

• Select appropriate TTL values
• Too short TTL reduces performance
• Too long TTL risks stale data

Cache Key Design

• Meaningful cache keys
• Prevent collisions
• Pattern-based invalidation

Memory Management

• Cache size limits
• Eviction policies
• Memory monitoring

Error Handling

• Fallback on cache errors
• Routing to backend
• Error logging

Next Steps

Data Log Layer

Learn about Data Log Layer architecture

API Gateway

Learn about API Gateway component