In the dynamic world of API development and management, controlling the flow of incoming requests is crucial to maintain system stability, ensure fair usage and prevent abuse.

Apinizer provides easy-to-use and high-performance solutions to these challenges with its advanced rate limiting, throttling and quota management features.

In this article, we will examine these basic concepts and explain their implementation in Apinizer.



Rate Limit and ThrottlingLink to Rate Limit and Throttling

Rate limit and throttling are often used interchangeably and both refer to the process of controlling the rate of incoming requests to the API. 

Time Scale: Usually measured in seconds or minutes.

Examples

  • 100 requests per second
  • 1000 requests per minute

Application: Apinizer uses Distributed Cache to perform the relevant rule calculations, store the results and distribute them between different systems. 

Data Storage: Based on cache only. This means that data is transient and may be lost when the system is restarted, which is acceptable for short-term limits.

Usage Areas:

  • Protecting APIs against sudden traffic spikes
  • Ensure fair utilization among multiple customers in short time frames
  • Prevent misuse or DoS attacks

Quota ManagementLink to Quota Management

Quota management works with similar goals, but with a different scale and target.

Time Scale: Usually measured in hours, days or months.

Examples

  • 10,000 requests a day
  • 1.000.000 requests per month

Application: Uses a combination of cache for fast access and real-time updates and database for persistent long-term storage.

Data Storage:

  • Cache: Primary source for real-time quota checks, updated synchronously with each API call.
  • Database: The database is used for secondary, persistent storage. Updated asynchronously to reduce API response time, providing data persistence in case of system reboot or cache failure.

Areas of Use:

  • Enforce API usage limits at the business level
  • Billing and accounting for API consumption
  • Long-term utilization analysis and planning

Throttling and Quota Management Workflow Link to Throttling and Quota Management Workflow 

It uses a multi-layered approach to monitor and manage API usage quotas. This system works through four basic components: Client, API Gateway, Distributed Cache and Database.

The process flow is as follows:

  1. First Request Check: Every API request from the Client first passes through the API Gateway. The Gateway quickly refers to the Distributed Cache to check if the request meets the quota.
  2. Cache Layer: Distributed Cache holds quota information in a high performance and low latency manner. This layer is a critical component that enables the system to respond quickly. The current quota status from the Cache is evaluated by the API Gateway.
  3. Decision Mechanism: API Gateway follows two different paths based on the information it receives from the cache:
    • Successful Scenario: If the quota limit is not exceeded, the request is processed and a successful response is returned to the client. Then, the quota counter is updated.
    • Failed Scenario: If the quota has been exceeded, the request is rejected and an appropriate error message is sent to the client.
  4. Data Synchronization: On successful transactions, the quota information is updated in two stages:
    • First, the Distributed Cache is quickly updated (synchronous)
    • The database update is then performed asynchronously
  5. Durability and Consistency: The database ensures data continuity in case of long-term quota tracking and system reboots. Asynchronous update approach guarantees data durability without impacting API performance.

This flow strikes an optimal balance between high performance and data consistency. Cache utilization ensures fast response times, while database support guarantees data persistence. Asynchronous database updates provide reliable quota tracking while maintaining system performance.

The visual summarizing this flow is given below:

Critical “Interval Window Type” ParameterLink to Critical “Interval Window Type” Parameter

Central to the business logic of both throttling and quota management is the `Interval Window Type` parameter.

This parameter determines how to manage and update the time window in which requests are counted.

Possible Values

  • Fixed Window
  • Sliding Window

Fixed WindowLink to Fixed Window

In the Fixed Window approach, time is divided into fixed, non-overlapping periods.

Behavior:

  • All requests within a given period are counted together.
  • The counter is reset at the start of each new period.
  • The TTL (Time to Live) of the cache entry is set to the time remaining in the current period.

Example:

  • If the period is set to 1 minute and starts at 12:00:00:
  • Requests between 12:00:00:00 and 12:00:59 are counted in the same window.
  • At 12:01:00 a new window starts and the counter is reset.

Sliding WindowLink to Sliding Window

In the Sliding Window approach, each request starts its own time window.

Behavior:

  • The window “slides” with each new request.
  • Counts and continuously updates all requests during the past period.
  • The TTL value of the cache entry is always set to the full period length.

Example:

  • If the period is set to 1 minute:
  • A request at 12:00:30 counts all requests between 11:59:30 and 12:00:30.
  • A request at 12:00:45 counts all requests between 11:59:45 and 12:00:45.

Management of Longer Time SpansLink to Management of Longer Time Spans

The calculation of fixed windows becomes more complex, especially when working with longer time intervals for quota management.

Here's how to manage it for different scenarios:

For Intervals Less Than One Day (e.g. 15 minutes, 12 hours)Link to For Intervals Less Than One Day (e.g. 15 minutes, 12 hours)

Formula: Window Start = Day Start + (Elapsed Periods * Period Duration)

Example: For a 15-minute period:

  • Current time 14:37
  • Calculation
    • Day Start: 00.00
    • The last 15 minute periods: 58
  • Window start 14:30
  • This request is for the window 14:30 - 14:44:59.

For One Day or Longer Intervals (e.g. 3 days)

Formula: Window Start = Unix Epoch + (Elapsed Periods * Period Duration)

Example: For a period of 3 days:

  • Current date: 2023-10-15
  • Computing:
    • Days since the Unix epoch: 19,645
    • Past 3-day intervals: 6,548 (19,645/3, rounded down)
  • Window start 2023-10-13 00:00:00
  • This request is for the period 2023-10-13 00:00:00 - 2023-10-15 23:59:59.


Be careful to set the time zone value to manage day transitions!

Flexible Limitations with Dynamic Key GenerationLink to Flexible Limitations with Dynamic Key Generation

An important feature in our Rate limit, throttling and quota implementation is the ability to dynamically generate throttling keys based on various aspects of the incoming request. This approach allows for more granular and flexible control of API usage without the need for complex code changes.

Here is an overview of how this is done in our current system:

  • Flexible Switch Components: The system allows creation using any combination of delimitation keys:
    • Credentials (e.g. user ID, API key)
    • Request metadata (e.g. IP address, request headers)Request metadata (e.g. IP address, request headers)
    • Content from request payload
  • Dynamic Key Generation:
    • The system dynamically reads the components specified in each request and generates keys.
    • This reading is based on predefined paths or patterns set in the configuration.
  • Key Aggregation:
    • The extracted components are concatenated into a single string to form the delimitation key.
    • A consistent delimiter (for example, “-”) is used to separate the different components within the key.
  • Scenario Based Keys: This approach enables a variety of restriction scenarios:
    • Limitations per user: Using the user ID as the key
    • Limitations per endpoint: Merge the API endpoint's ID with the user ID
    • Content-based limitations: Including specific fields from the payload of the request in the key
  • Scalability and Performance:
    • The key generation process is designed to be lightweight and fast.
    • Complex computations are avoided to ensure minimal impact on request processing time.
  • Security Considerations:
    • The system ensures that sensitive information is not disclosed in the generated keys.
  • Consistency Across Services: 
    • This key generation logic is applied consistently across all API endpoints.
    • It is usually implemented as a centralized service or utility that all endpoints can benefit from.

By applying this flexible key generation approach, our system can adapt to a variety of rate capping and limitation requirements without the need for code changes. Whether it is applying different limits for different types of transactions, controlling access based on user roles, or applying complex multi-factor limitation rules, the system can meet these needs through configuration changes alone.

This level of flexibility provides the ability to fine-tune API usage and enables businesses to enforce fair usage policies, prevent abuse and effectively optimize resource allocation. It also offers the ability to quickly adjust limitation strategies in response to changing business needs or observed usage patterns.

ConclusionLink to Conclusion

Effective API traffic control, through request limiting, throttling and quota management, is essential for building robust, fair and scalable API services. By understanding the intricacies of fixed and sliding windows, implementing appropriate timescales strategies and following best practices, you can ensure that your APIs remain performant, protected and profitable.

Remember, the choice between request limiting strategies and Interval Window Type configuration should be based on your specific use case, traffic patterns and business requirements. Regular review and adjustment of these parameters will help you achieve an optimal balance between protection and availability for your API services.