Rate Limiting

Understanding API rate limits, how to handle them gracefully, and best practices for efficient API usage in MegaVault applications.

Rate Limiting Overview
Rate Limit Tiers
Rate Limit Headers
Handling Rate Limits
Best Practices
Monitoring Usage

Rate Limiting Overview

MegaVault implements rate limiting to ensure fair usage, maintain system stability, and provide consistent performance for all users. Rate limits vary by endpoint type and user plan.

Fair Usage

Ensure equal access

✅ Prevents API abuse
✅ Maintains service quality
✅ Protects infrastructure
✅ Equal access for all users

Flexible Limits

Plan-based tiers

✅ Free plan limits
✅ Pro plan increases
✅ Enterprise customization
✅ Endpoint-specific limits

Developer Tools

Monitor and optimize

✅ Rate limit headers
✅ Usage monitoring
✅ Retry guidance
✅ Optimization tips

💡

Rate Limiting Strategy

MegaVault uses a sliding window approach for rate limiting, providing more flexibility than fixed-window systems while maintaining fairness and preventing abuse.

Rate Limit Tiers

Rate limits vary based on your subscription plan and the type of operation.

Authentication Endpoints

Endpoint	Free Plan	Pro Plan	Enterprise	Window
/api/auth/signin	5 requests	10 requests	20 requests	1 minute
/api/auth/signup	3 requests	5 requests	10 requests	1 hour
/api/auth/refresh	10 requests	50 requests	100 requests	1 hour

File Operations

Operation	Free Plan	Pro Plan	Enterprise	Window
File Uploads	10 uploads	100 uploads	1000 uploads	1 hour
File Downloads	50 downloads	500 downloads	Unlimited	1 hour
File Listing	100 requests	1000 requests	10000 requests	1 hour
File Operations	50 requests	500 requests	5000 requests	1 hour

General API Endpoints

Free Plan

General API: 1,000 requests/hour
User Profile: 100 requests/hour
Usage Stats: 50 requests/hour
Mobile Sync: 200 requests/hour

Pro Plan

General API: 10,000 requests/hour
User Profile: 1,000 requests/hour
Usage Stats: 500 requests/hour
Mobile Sync: 2,000 requests/hour

Enterprise

General API: 100,000 requests/hour
User Profile: 10,000 requests/hour
Usage Stats: 5,000 requests/hour
Mobile Sync: 20,000 requests/hour

Rate Limit Headers

Every API response includes headers with rate limit information for monitoring usage.

Rate Limit Response Headers

HTTP/1.1 200 OK
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 995
X-RateLimit-Reset: 1640995200
X-RateLimit-Window: 3600
X-RateLimit-Resource: general-api
Retry-After: 300

{
  "success": true,
  "data": { ... }
}

Header Descriptions

Header	Description	Example
X-RateLimit-Limit	Maximum requests allowed in the current window	1000
X-RateLimit-Remaining	Requests remaining in the current window	995
X-RateLimit-Reset	Unix timestamp when the window resets	1640995200
X-RateLimit-Window	Window duration in seconds	3600
Retry-After	Seconds to wait before retrying (429 responses only)	300

Handling Rate Limits

Best practices for detecting and responding to rate limit conditions.

Detecting Rate Limits

Rate Limit Detection

function checkRateLimit(response) {
  const limit = parseInt(response.headers.get('X-RateLimit-Limit'));
  const remaining = parseInt(response.headers.get('X-RateLimit-Remaining'));
  const reset = parseInt(response.headers.get('X-RateLimit-Reset'));
  
  const usage = ((limit - remaining) / limit) * 100;
  
  // Warning at 80% usage
  if (usage >= 80) {
    console.warn(`Rate limit warning: ${usage.toFixed(1)}% used`);
    
    // Slow down requests when approaching limit
    if (usage >= 90) {
      const resetTime = new Date(reset * 1000);
      console.warn(`Rate limit critical: Resets at ${resetTime.toISOString()}`);
      return { shouldDelay: true, delayMs: 1000 };
    }
  }
  
  return { shouldDelay: false, delayMs: 0 };
}

async function makeRequest(url, options) {
  const response = await fetch(url, options);
  
  // Check rate limit headers
  const rateLimitInfo = checkRateLimit(response);
  if (rateLimitInfo.shouldDelay) {
    console.log(`Delaying next request by ${rateLimitInfo.delayMs}ms`);
    await new Promise(resolve => setTimeout(resolve, rateLimitInfo.delayMs));
  }
  
  // Handle 429 Too Many Requests
  if (response.status === 429) {
    const retryAfter = parseInt(response.headers.get('Retry-After')) || 60;
    throw new RateLimitError(`Rate limited. Retry after ${retryAfter} seconds`, retryAfter);
  }
  
  return response;
}

Exponential Backoff

Exponential Backoff Implementation

class RateLimitedClient {
  constructor(baseDelay = 1000, maxDelay = 30000, maxRetries = 5) {
    this.baseDelay = baseDelay;
    this.maxDelay = maxDelay;
    this.maxRetries = maxRetries;
  }
  
  async makeRequestWithBackoff(url, options, attempt = 1) {
    try {
      const response = await fetch(url, options);
      
      if (response.status === 429) {
        if (attempt > this.maxRetries) {
          throw new Error('Max retries exceeded');
        }
        
        const retryAfter = parseInt(response.headers.get('Retry-After')) || 0;
        const backoffDelay = Math.min(
          this.baseDelay * Math.pow(2, attempt - 1),
          this.maxDelay
        );
        
        // Use the larger of retry-after or exponential backoff
        const delay = Math.max(retryAfter * 1000, backoffDelay);
        
        console.log(`Rate limited. Retrying in ${delay}ms (attempt ${attempt})`);
        await new Promise(resolve => setTimeout(resolve, delay));
        
        return this.makeRequestWithBackoff(url, options, attempt + 1);
      }
      
      return response;
    } catch (error) {
      if (attempt > this.maxRetries) {
        throw error;
      }
      
      const delay = Math.min(
        this.baseDelay * Math.pow(2, attempt - 1),
        this.maxDelay
      );
      
      console.log(`Request failed. Retrying in ${delay}ms (attempt ${attempt})`);
      await new Promise(resolve => setTimeout(resolve, delay));
      
      return this.makeRequestWithBackoff(url, options, attempt + 1);
    }
  }
}

Request Queuing

Request Queue Manager

class RequestQueue {
  constructor(maxConcurrent = 5, requestsPerSecond = 10) {
    this.maxConcurrent = maxConcurrent;
    this.requestsPerSecond = requestsPerSecond;
    this.queue = [];
    this.active = 0;
    this.lastRequestTime = 0;
  }
  
  async enqueue(requestFn) {
    return new Promise((resolve, reject) => {
      this.queue.push({ requestFn, resolve, reject });
      this.processQueue();
    });
  }
  
  async processQueue() {
    if (this.active >= this.maxConcurrent || this.queue.length === 0) {
      return;
    }
    
    // Rate limiting: ensure minimum time between requests
    const now = Date.now();
    const minInterval = 1000 / this.requestsPerSecond;
    const timeSinceLastRequest = now - this.lastRequestTime;
    
    if (timeSinceLastRequest < minInterval) {
      setTimeout(() => this.processQueue(), minInterval - timeSinceLastRequest);
      return;
    }
    
    const { requestFn, resolve, reject } = this.queue.shift();
    this.active++;
    this.lastRequestTime = Date.now();
    
    try {
      const result = await requestFn();
      resolve(result);
    } catch (error) {
      reject(error);
    } finally {
      this.active--;
      // Process next item in queue
      setTimeout(() => this.processQueue(), 0);
    }
  }
}

Best Practices

Recommendations for efficient API usage and rate limit management.

Optimization Strategies

Batch Operations: Combine multiple operations into single requests when possible
Caching: Cache frequently accessed data to reduce API calls
Pagination: Use appropriate page sizes to balance performance and rate limits
Conditional Requests: Use ETags and If-Modified-Since headers to avoid unnecessary transfers
Background Processing: Move non-critical operations to background jobs

Monitoring Usage

Usage Monitoring

class RateLimitMonitor {
  constructor() {
    this.usage = new Map();
    this.alerts = [];
  }
  
  recordRequest(endpoint, rateLimitHeaders) {
    const key = this.getEndpointKey(endpoint);
    const limit = parseInt(rateLimitHeaders['x-ratelimit-limit']);
    const remaining = parseInt(rateLimitHeaders['x-ratelimit-remaining']);
    const reset = parseInt(rateLimitHeaders['x-ratelimit-reset']);
    
    this.usage.set(key, {
      endpoint,
      limit,
      remaining,
      reset,
      usage: ((limit - remaining) / limit) * 100,
      timestamp: Date.now()
    });
    
    // Check for high usage
    if (remaining / limit < 0.1) { // Less than 10% remaining
      this.triggerAlert(key, 'HIGH_USAGE', `Only ${remaining} requests remaining`);
    }
  }
  
  getUsageReport() {
    const report = Array.from(this.usage.values()).map(usage => ({
      endpoint: usage.endpoint,
      usagePercentage: usage.usage.toFixed(1),
      remaining: usage.remaining,
      resetTime: new Date(usage.reset * 1000).toISOString()
    }));
    
    return report.sort((a, b) => b.usagePercentage - a.usagePercentage);
  }
  
  triggerAlert(key, type, message) {
    console.warn(`Rate limit alert [${type}]: ${message}`);
    this.alerts.push({
      key,
      type,
      message,
      timestamp: Date.now()
    });
  }
}

Application Architecture

Request Queues: Implement request queuing to control API call frequency
Circuit Breakers: Use circuit breakers to handle rate limit failures gracefully
Local Caching: Cache API responses locally to reduce redundant calls
Lazy Loading: Load data on-demand rather than pre-fetching everything
Error Boundaries: Implement proper error handling for rate limit scenarios

Monitoring Usage

Track your API usage and optimize performance to stay within rate limits.

Usage Dashboard

Get Usage Statistics

GET /api/users/usage/rate-limits
Authorization: Bearer YOUR_JWT_TOKEN

Usage Response

{
  "success": true,
  "data": {
    "period": "current_hour",
    "endpoints": [
      {
        "endpoint": "/api/files",
        "limit": 1000,
        "used": 245,
        "remaining": 755,
        "usagePercentage": 24.5,
        "resetsAt": "2024-01-20T17:00:00Z"
      },
      {
        "endpoint": "/api/files/upload",
        "limit": 100,
        "used": 15,
        "remaining": 85,
        "usagePercentage": 15.0,
        "resetsAt": "2024-01-20T17:00:00Z"
      }
    ],
    "overallUsage": {
      "totalRequests": 260,
      "successfulRequests": 258,
      "rateLimitedRequests": 2,
      "successRate": 99.2
    }
  }
}

Alerting

Usage Alerts: Set up alerts when approaching rate limits (80%, 90%)
Error Monitoring: Monitor 429 responses and implement proper retry logic
Performance Tracking: Track API response times and success rates
Cost Optimization: Monitor API usage to optimize costs and performance

✅

Optimization Tips

Use webhooks instead of polling for real-time updates
Implement client-side caching with appropriate TTLs
Batch multiple operations when possible
Use streaming APIs for large data transfers

Rate Limiting

Table of Contents

Rate Limiting Overview

Fair Usage

Flexible Limits

Developer Tools

Rate Limiting Strategy

Rate Limit Tiers

Authentication Endpoints

File Operations

General API Endpoints

Free Plan

Pro Plan

Enterprise

Rate Limit Headers

Header Descriptions

Handling Rate Limits

Detecting Rate Limits

Exponential Backoff

Request Queuing

Best Practices

Optimization Strategies

Monitoring Usage

Application Architecture

Monitoring Usage

Usage Dashboard

Alerting

Optimization Tips

Related Topics

Error Handling →

API Overview →

Authentication →