Files

IGNY8 VPS (Salman) 069e0a24d8 Remove comprehensive API endpoints analysis document from the repository. This file contained detailed information on request/response formats, authentication methods, and endpoint statistics.

2025-11-14 14:48:48 +00:00

34 KiB

Raw Blame History

API Implementation Plan - Section 4: Apply Rate Limiting and Throttling Rules

Date: 2025-01-XX
Status: Planning
Priority: High
Related Document: API-ENDPOINTS-ANALYSIS.md

Executive Summary

This document outlines the implementation plan for Section 4: Apply Rate Limiting and Throttling Rules across the IGNY8 API layer. The goal is to prevent abuse, ensure fair resource usage, and enforce plan limits by introducing consistent per-user and per-function rate limits using DRF throttling and internal usage tracking.

Key Objectives:

Enable DRF throttling system with scoped rate limits
Assign throttle scopes to all major functions (AI, auth, content, images)
Implement plan-based throttling (optional, future phase)
Log and expose rate limit information
Integrate with existing credit system for quota enforcement

Current State Analysis
Implementation Tasks
Task 1: Enable DRF Throttling System
Task 2: Assign Throttle Scopes in ViewSets
Task 3: Override User/Account-Level Throttles (Optional)
Task 4: Log + Expose Rate Limit Info
Task 5: Advanced Daily/Monthly Quotas Per Plan
Task 6: Testing Matrix
Task 7: Changelog Entry
Testing Strategy
Rollout Plan
Success Criteria

Current State Analysis

Current Rate Limiting Status

Based on API-ENDPOINTS-ANALYSIS.md:

Rate Limiting: ❌ Not Implemented
- All endpoints are currently unlimited
- No throttling configuration in settings
- No rate limit enforcement
Credit System:
- CreditUsageLog model exists for tracking usage
- CreditBalance model tracks account credits
- Credit deduction happens but no rate limiting based on credits
Plan System:
- Plan model exists with different tiers
- Plans have credit limits but no API rate limits defined
- No plan-based throttling implemented
AI Functions:
- High-cost operations (AI generation, image generation)
- Currently no rate limiting on these expensive endpoints
- Risk of abuse and cost overruns

Implementation Tasks

Overview

Task ID	Task Name	Priority	Estimated Effort	Dependencies
1.1	Configure DRF throttling in settings	High	2 hours	None
1.2	Add debug throttle bypass	High	1 hour	1.1
2.1	Audit endpoints for throttle scopes	Medium	3 hours	None
2.2	Assign throttle scopes to ViewSets	High	6 hours	1.1
3.1	Create plan-based throttle class (optional)	Low	8 hours	1.1
4.1	Verify throttle headers	High	2 hours	2.2
4.2	Add frontend rate limit handling	Medium	4 hours	4.1
4.3	Implement abuse detection logging	Medium	3 hours	2.2
5.1	Integrate with credit system	Low	6 hours	3.1
6.1	Create test scenarios	High	4 hours	2.2
7.1	Create changelog entry	Low	1 hour	All tasks

Total Estimated Effort: ~40 hours

Task 1: Enable DRF Throttling System

Goal

Configure Django REST Framework to use scoped rate throttling with appropriate rate limits for different endpoint types.

Implementation Steps

Step 1.1: Configure DRF Throttling in Settings

File: backend/igny8_core/settings.py

Update REST_FRAMEWORK configuration:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'igny8_core.api.pagination.CustomPageNumberPagination',
    'PAGE_SIZE': 10,
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.SearchFilter',
        'rest_framework.filters.OrderingFilter',
    ],
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.AllowAny',  # Will be changed in Section 2
    ],
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'igny8_core.api.authentication.JWTAuthentication',
        'igny8_core.api.authentication.CSRFExemptSessionAuthentication',
        'rest_framework.authentication.BasicAuthentication',
    ],
    'EXCEPTION_HANDLER': 'igny8_core.api.exception_handlers.custom_exception_handler',
    # Throttling Configuration
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.ScopedRateThrottle',
    ],
    'DEFAULT_THROTTLE_RATES': {
        # AI Functions - Expensive operations
        'ai_function': '10/min',           # AI content generation, clustering
        'image_gen': '15/min',            # Image generation
        
        # Content Operations
        'content_write': '30/min',        # Content creation, updates
        'content_read': '100/min',        # Content listing, retrieval
        
        # Authentication
        'auth': '20/min',                 # Login, register, password reset
        'auth_strict': '5/min',           # Sensitive auth operations
        
        # Planner Operations
        'planner': '60/min',              # Keyword, cluster, idea operations
        'planner_ai': '10/min',           # AI-powered planner operations
        
        # Writer Operations
        'writer': '60/min',               # Task, content management
        'writer_ai': '10/min',            # AI-powered writer operations
        
        # System Operations
        'system': '100/min',              # Settings, prompts, profiles
        'system_admin': '30/min',         # Admin-only system operations
        
        # Billing Operations
        'billing': '30/min',              # Credit queries, usage logs
        'billing_admin': '10/min',        # Credit management (admin)
        
        # Default fallback
        'default': '100/min',             # Default for endpoints without scope
    },
}

Step 1.2: Add Debug Throttle Bypass

File: backend/igny8_core/settings.py

Add environment variable:

import os

# Throttling Configuration
IGNY8_DEBUG_THROTTLE = os.getenv('IGNY8_DEBUG_THROTTLE', 'False').lower() == 'true'

# Only apply throttling if not in debug mode or explicitly enabled
if not DEBUG and not IGNY8_DEBUG_THROTTLE:
    REST_FRAMEWORK['DEFAULT_THROTTLE_CLASSES'] = [
        'rest_framework.throttling.ScopedRateThrottle',
    ]
else:
    # In debug mode, use a no-op throttle class
    REST_FRAMEWORK['DEFAULT_THROTTLE_CLASSES'] = []

Create Custom Throttle Class with Debug Bypass:

File: backend/igny8_core/api/throttles.py

Implementation:

"""
Custom Throttle Classes for IGNY8 API
"""

from django.conf import settings
from rest_framework.throttling import ScopedRateThrottle


class DebugScopedRateThrottle(ScopedRateThrottle):
    """
    Scoped rate throttle that can be bypassed in debug mode.
    
    Usage:
        throttle_scope = 'ai_function'
    """
    
    def allow_request(self, request, view):
        """
        Check if request should be throttled.
        Bypass throttling if DEBUG=True or IGNY8_DEBUG_THROTTLE=True.
        """
        # Bypass throttling in debug mode
        if settings.DEBUG or getattr(settings, 'IGNY8_DEBUG_THROTTLE', False):
            return True
        
        # Use parent class throttling logic
        return super().allow_request(request, view)

Update settings to use custom throttle:

'DEFAULT_THROTTLE_CLASSES': [
    'igny8_core.api.throttles.DebugScopedRateThrottle',
],

Estimated Time: 2 hours (settings) + 1 hour (debug bypass) = 3 hours

Task 2: Assign Throttle Scopes in ViewSets

Goal

Assign appropriate throttle scopes to all ViewSets and custom actions based on their function and resource cost.

Implementation Steps

Step 2.1: Audit Endpoints for Throttle Scopes

Action Items:

List all ViewSets and custom actions
Categorize by function type (AI, content, auth, etc.)
Identify high-cost operations (AI, image generation)
Create mapping of endpoints to throttle scopes

Endpoint Categories:

AI Functions (10/min):

POST /api/v1/planner/keywords/auto_cluster/ - AI clustering
POST /api/v1/planner/clusters/auto_generate_ideas/ - AI idea generation
POST /api/v1/writer/tasks/auto_generate_content/ - AI content generation
POST /api/v1/writer/content/generate_image_prompts/ - AI prompt generation

Image Generation (15/min):

POST /api/v1/writer/images/generate_images/ - Image generation
POST /api/v1/writer/images/auto_generate/ - Legacy image generation

Content Write (30/min):

POST /api/v1/writer/tasks/ - Create task
POST /api/v1/writer/content/ - Create content
PUT /api/v1/writer/tasks/{id}/ - Update task
PUT /api/v1/writer/content/{id}/ - Update content

Planner AI (10/min):

POST /api/v1/planner/keywords/auto_cluster/
POST /api/v1/planner/clusters/auto_generate_ideas/

Planner Standard (60/min):

GET /api/v1/planner/keywords/ - List keywords
POST /api/v1/planner/keywords/ - Create keyword
GET /api/v1/planner/clusters/ - List clusters
POST /api/v1/planner/ideas/ - Create idea

Auth (20/min):

POST /api/v1/auth/login/
POST /api/v1/auth/register/
POST /api/v1/auth/change-password/

Auth Strict (5/min):

POST /api/v1/auth/password-reset/ - If exists
POST /api/v1/auth/users/invite/ - User invitation

System Admin (30/min):

POST /api/v1/system/settings/integrations/{pk}/save/
POST /api/v1/system/settings/integrations/{pk}/test/

Billing Admin (10/min):

POST /api/v1/billing/credits/transactions/ - Create transaction

Estimated Time: 3 hours

Step 2.2: Assign Throttle Scopes to ViewSets

Example Implementation:

Planner Module:

File: backend/planner/api/viewsets.py

from igny8_core.api.viewsets import BaseTenantViewSet

class KeywordViewSet(BaseTenantViewSet):
    throttle_scope = 'planner'  # Default for standard operations
    queryset = Keyword.objects.all()
    serializer_class = KeywordSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='ai_function'  # Override for AI operations
    )
    def auto_cluster(self, request):
        # AI clustering logic
        pass


class ClusterViewSet(BaseTenantViewSet):
    throttle_scope = 'planner'
    queryset = Cluster.objects.all()
    serializer_class = ClusterSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='planner_ai'  # AI-powered operations
    )
    def auto_generate_ideas(self, request):
        # AI idea generation logic
        pass

Writer Module:

File: backend/writer/api/viewsets.py

class TasksViewSet(BaseTenantViewSet):
    throttle_scope = 'writer'  # Default for standard operations
    queryset = Task.objects.all()
    serializer_class = TaskSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='ai_function'  # AI content generation
    )
    def auto_generate_content(self, request):
        # AI content generation logic
        pass


class ImagesViewSet(BaseTenantViewSet):
    throttle_scope = 'writer'
    queryset = Image.objects.all()
    serializer_class = ImageSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='image_gen'  # Image generation
    )
    def generate_images(self, request):
        # Image generation logic
        pass

Auth Module:

File: backend/auth/api/views.py

from rest_framework.views import APIView
from rest_framework.throttling import ScopedRateThrottle

class LoginView(APIView):
    throttle_scope = 'auth'
    throttle_classes = [DebugScopedRateThrottle]
    
    def post(self, request):
        # Login logic
        pass


class RegisterView(APIView):
    throttle_scope = 'auth'
    throttle_classes = [DebugScopedRateThrottle]
    
    def post(self, request):
        # Registration logic
        pass


class ChangePasswordView(APIView):
    throttle_scope = 'auth'
    throttle_classes = [DebugScopedRateThrottle]
    
    def post(self, request):
        # Password change logic
        pass

System Module:

File: backend/system/api/viewsets.py

class IntegrationSettingsViewSet(BaseTenantViewSet):
    throttle_scope = 'system'
    
    @action(
        detail=True,
        methods=['post'],
        throttle_scope='system_admin'  # Admin operations
    )
    def save(self, request, pk=None):
        # Save integration settings
        pass
    
    @action(
        detail=True,
        methods=['post'],
        throttle_scope='system_admin'
    )
    def test(self, request, pk=None):
        # Test integration connection
        pass

Billing Module:

File: backend/billing/api/viewsets.py

class CreditTransactionViewSet(BaseTenantViewSet):
    throttle_scope = 'billing'
    
    # Read-only operations use 'billing' scope
    # Write operations would use 'billing_admin' if implemented

Estimated Time: 6 hours

Task 3: Override User/Account-Level Throttles (Optional)

Goal

Implement plan-based throttling that applies different rate limits based on user's subscription plan.

Implementation Steps

Step 3.1: Create Plan-Based Throttle Class

File: backend/igny8_core/api/throttles.py

Implementation:

"""
Custom Throttle Classes for IGNY8 API
"""

from django.conf import settings
from rest_framework.throttling import SimpleRateThrottle


class PlanBasedRateThrottle(SimpleRateThrottle):
    """
    Rate throttle that applies different limits based on user's plan.
    
    Plan tiers:
    - Free: 10/min for AI functions
    - Pro: 100/min for AI functions
    - Enterprise: Unlimited (or very high limit)
    
    Usage:
        throttle_scope = 'ai_function'  # Base scope
        # Plan limits override base scope
    """
    
    # Plan-based rate limits
    PLAN_RATES = {
        'free': {
            'ai_function': '10/min',
            'image_gen': '15/min',
            'content_write': '30/min',
        },
        'pro': {
            'ai_function': '100/min',
            'image_gen': '50/min',
            'content_write': '100/min',
        },
        'enterprise': {
            'ai_function': '1000/min',  # Effectively unlimited
            'image_gen': '200/min',
            'content_write': '500/min',
        },
    }
    
    def get_rate(self):
        """
        Get rate limit based on user's plan.
        """
        # Get scope from view
        scope = getattr(self.view, 'throttle_scope', None)
        if not scope:
            return None
        
        # Get user's plan
        user = self.request.user
        if not user or not user.is_authenticated:
            # Anonymous users use default rate
            return self.get_default_rate(scope)
        
        # Get account and plan
        account = getattr(user, 'account', None)
        if not account:
            return self.get_default_rate(scope)
        
        plan = getattr(account, 'plan', None)
        if not plan:
            return self.get_default_rate(scope)
        
        plan_name = plan.name.lower() if hasattr(plan, 'name') else 'free'
        
        # Get rate for plan and scope
        plan_rates = self.PLAN_RATES.get(plan_name, self.PLAN_RATES['free'])
        rate = plan_rates.get(scope, self.get_default_rate(scope))
        
        return rate
    
    def get_default_rate(self, scope):
        """
        Get default rate from settings if plan-based rate not found.
        """
        throttle_rates = getattr(settings, 'REST_FRAMEWORK', {}).get('DEFAULT_THROTTLE_RATES', {})
        return throttle_rates.get(scope, '100/min')
    
    def get_cache_key(self, request, view):
        """
        Generate cache key based on user ID and scope.
        """
        if request.user.is_authenticated:
            ident = request.user.pk
        else:
            ident = self.get_ident(request)
        
        scope = getattr(view, 'throttle_scope', None)
        return self.cache_format % {
            'scope': scope,
            'ident': ident
        }

Step 3.2: Apply Plan-Based Throttling

Update settings to use plan-based throttle for AI functions:

File: backend/igny8_core/settings.py

# For AI functions, use plan-based throttling
# For other functions, use scoped throttling

# This would require custom throttle class selection per ViewSet
# Or use a mixin that selects throttle class based on scope

Alternative: Use in specific ViewSets:

from igny8_core.api.throttles import PlanBasedRateThrottle

class TasksViewSet(BaseTenantViewSet):
    throttle_scope = 'ai_function'
    throttle_classes = [PlanBasedRateThrottle]  # Use plan-based for this ViewSet
    
    @action(detail=False, methods=['post'])
    def auto_generate_content(self, request):
        # AI content generation
        pass

Estimated Time: 8 hours (optional, future phase)

Task 4: Log + Expose Rate Limit Info

Goal

Ensure rate limit information is properly exposed to clients and logged for abuse detection.

Implementation Steps

Step 4.1: Verify Throttle Headers

DRF automatically adds these headers:

X-Throttle-Limit: Maximum number of requests allowed
X-Throttle-Remaining: Number of requests remaining
Retry-After: Seconds to wait before retrying (when throttled)

Test in Postman:

Make requests to throttled endpoint
Check response headers
Verify headers are present and correct

Example Response Headers:

HTTP/1.1 200 OK
X-Throttle-Limit: 10
X-Throttle-Remaining: 9
X-Throttle-Reset: 1640995200

When Throttled (429):

HTTP/1.1 429 Too Many Requests
X-Throttle-Limit: 10
X-Throttle-Remaining: 0
Retry-After: 60

Estimated Time: 2 hours

Step 4.2: Add Frontend Rate Limit Handling

File: frontend/src/services/api.ts

Update fetchAPI to handle rate limits:

export async function fetchAPI(endpoint: string, options?: RequestInit) {
  const response = await fetch(`${API_BASE_URL}${endpoint}`, {
    ...options,
    headers: {
      ...options?.headers,
      'Authorization': `Bearer ${getToken()}`,
    },
  });

  // Check for rate limiting
  if (response.status === 429) {
    const retryAfter = response.headers.get('Retry-After');
    const throttleLimit = response.headers.get('X-Throttle-Limit');
    const throttleRemaining = response.headers.get('X-Throttle-Remaining');
    
    // Show user-friendly error
    showNotification(
      `Rate limit exceeded. Please wait ${retryAfter} seconds before trying again.`,
      'error'
    );
    
    // Store rate limit info for UI
    if (throttleLimit && throttleRemaining !== null) {
      storeRateLimitInfo({
        limit: parseInt(throttleLimit),
        remaining: parseInt(throttleRemaining),
        retryAfter: retryAfter ? parseInt(retryAfter) : 60,
      });
    }
    
    throw new Error('Rate limit exceeded');
  }

  // Check throttle headers on successful requests
  const throttleLimit = response.headers.get('X-Throttle-Limit');
  const throttleRemaining = response.headers.get('X-Throttle-Remaining');
  
  if (throttleLimit && throttleRemaining !== null) {
    const remaining = parseInt(throttleRemaining);
    const limit = parseInt(throttleLimit);
    const percentage = (remaining / limit) * 100;
    
    // Show warning if close to limit
    if (percentage < 20) {
      showNotification(
        `Warning: You have ${remaining} of ${limit} requests remaining.`,
        'warning'
      );
    }
    
    // Store for UI display
    storeRateLimitInfo({
      limit,
      remaining,
      percentage,
    });
  }

  return response;
}

Add Rate Limit Display Component:

File: frontend/src/components/RateLimitIndicator.tsx

import { useRateLimitStore } from '@/stores/rateLimitStore';

export function RateLimitIndicator() {
  const { limit, remaining, percentage } = useRateLimitStore();
  
  if (!limit) return null;
  
  return (
    <div className="rate-limit-indicator">
      <span>API Calls: {remaining} / {limit}</span>
      <div className="progress-bar">
        <div 
          className="progress-fill" 
          style={{ width: `${percentage}%` }}
        />
      </div>
      {percentage < 20 && (
        <span className="warning">Low on requests</span>
      )}
    </div>
  );
}

Estimated Time: 4 hours

Step 4.3: Implement Abuse Detection Logging

File: backend/igny8_core/api/middleware.py

Add rate limit logging middleware:

"""
Rate Limit Logging Middleware
"""

import logging
from django.utils.deprecation import MiddlewareMixin

logger = logging.getLogger(__name__)


class RateLimitLoggingMiddleware(MiddlewareMixin):
    """
    Middleware that logs rate limit hits for abuse detection.
    """
    
    def process_response(self, request, response):
        """
        Log rate limit hits (429 responses).
        """
        if response.status_code == 429:
            # Log rate limit hit
            logger.warning(
                f"Rate limit exceeded: {request.method} {request.path}",
                extra={
                    'user_id': request.user.id if request.user.is_authenticated else None,
                    'ip_address': self.get_client_ip(request),
                    'user_agent': request.META.get('HTTP_USER_AGENT'),
                    'throttle_scope': getattr(request, 'throttle_scope', None),
                }
            )
            
            # Check for repeated hits (potential abuse)
            # This could be enhanced with Redis/cache to track frequency
            self.check_for_abuse(request)
        
        return response
    
    def get_client_ip(self, request):
        """
        Get client IP address.
        """
        x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
        if x_forwarded_for:
            ip = x_forwarded_for.split(',')[0]
        else:
            ip = request.META.get('REMOTE_ADDR')
        return ip
    
    def check_for_abuse(self, request):
        """
        Check if user is repeatedly hitting rate limits (potential abuse).
        """
        # TODO: Implement abuse detection logic
        # - Track rate limit hits per user/IP
        # - Alert if threshold exceeded
        # - Optionally block user/IP temporarily
        pass

Register middleware:

File: backend/igny8_core/settings.py

MIDDLEWARE = [
    # ... other middleware ...
    'igny8_core.api.middleware.RateLimitLoggingMiddleware',
    # ... other middleware ...
]

Estimated Time: 3 hours

Task 5: Advanced Daily/Monthly Quotas Per Plan

Goal

Implement quota-based usage limits that go beyond rate limiting, integrated with the existing credit system.

Implementation Steps

Step 5.1: Integrate with Credit System

This task extends the existing credit system to enforce quotas:

File: backend/igny8_core/api/middleware.py

Add quota check middleware:

"""
Quota Enforcement Middleware
"""

from django.http import JsonResponse
from rest_framework import status
from igny8_core.api.response import error_response


class QuotaEnforcementMiddleware(MiddlewareMixin):
    """
    Middleware that enforces daily/monthly quotas based on plan and credits.
    """
    
    def process_request(self, request):
        """
        Check if user has exceeded quota before processing request.
        """
        # Only check for authenticated users
        if not request.user or not request.user.is_authenticated:
            return None
        
        # Get account
        account = getattr(request.user, 'account', None)
        if not account:
            return None
        
        # Check if endpoint requires quota check
        if not self.requires_quota_check(request):
            return None
        
        # Check credits
        from igny8_core.modules.billing.models import CreditBalance
        
        try:
            balance = CreditBalance.objects.get(account=account)
            
            # Check if credits are exhausted
            if balance.credits <= 0:
                return error_response(
                    error="Credits exhausted. Please purchase more credits to continue.",
                    status_code=status.HTTP_402_PAYMENT_REQUIRED
                )
            
            # Check daily/monthly quotas (if implemented)
            # This would require additional quota tracking models
            
        except CreditBalance.DoesNotExist:
            # No balance record - allow request (will be created on first use)
            pass
        
        return None
    
    def requires_quota_check(self, request):
        """
        Check if this endpoint requires quota validation.
        """
        # Check if it's an AI function or expensive operation
        throttle_scope = getattr(request, 'throttle_scope', None)
        expensive_scopes = ['ai_function', 'image_gen', 'planner_ai', 'writer_ai']
        
        return throttle_scope in expensive_scopes

Estimated Time: 6 hours (optional, future phase)

Task 6: Testing Matrix

Goal

Create comprehensive test scenarios to validate rate limiting works correctly.

Implementation Steps

Step 6.1: Create Test Scenarios

Test Cases:

1. AI Function Rate Limiting (10/min)

# Test: Make 11 requests to /api/v1/writer/tasks/auto_generate_content/
# Expected: 
# - Requests 1-10: 200 OK
# - Request 11: 429 Too Many Requests
# - Headers: X-Throttle-Limit: 10, X-Throttle-Remaining: 0, Retry-After: 60

2. Image Generation Rate Limiting (15/min)

# Test: Make 16 requests to /api/v1/writer/images/generate_images/
# Expected:
# - Requests 1-15: 200 OK
# - Request 16: 429 Too Many Requests

3. Auth Rate Limiting (20/min)

# Test: Make 21 login attempts
# Expected:
# - Requests 1-20: 200 OK or 401 Unauthorized (depending on credentials)
# - Request 21: 429 Too Many Requests

4. Debug Mode Bypass

# Test: Set IGNY8_DEBUG_THROTTLE=True
# Expected: All requests allowed regardless of rate limits

5. Throttle Headers Verification

# Test: Make requests and check headers
# Expected:
# - X-Throttle-Limit header present
# - X-Throttle-Remaining header present and decrements
# - Retry-After header present when throttled

6. Plan-Based Throttling (if implemented)

# Test: Free plan user vs Pro plan user
# Expected:
# - Free plan: 10/min for AI functions
# - Pro plan: 100/min for AI functions

Automated Test Implementation:

File: backend/igny8_core/api/tests/test_throttling.py

"""
Tests for Rate Limiting and Throttling
"""

from django.test import TestCase
from rest_framework.test import APIClient
from rest_framework import status
from django.contrib.auth import get_user_model
from igny8_core.auth.models import Account

User = get_user_model()


class ThrottlingTestCase(TestCase):
    def setUp(self):
        self.client = APIClient()
        self.account = Account.objects.create(name="Test Account")
        self.user = User.objects.create_user(
            email="test@example.com",
            password="testpass123",
            account=self.account
        )
        self.client.force_authenticate(user=self.user)
    
    def test_ai_function_rate_limit(self):
        """Test that AI functions are rate limited to 10/min"""
        endpoint = '/api/v1/writer/tasks/auto_generate_content/'
        
        # Make 10 requests (should all succeed)
        for i in range(10):
            response = self.client.post(endpoint, {})
            self.assertIn(response.status_code, [200, 400, 500])  # Not throttled
        
        # 11th request should be throttled
        response = self.client.post(endpoint, {})
        self.assertEqual(response.status_code, status.HTTP_429_TOO_MANY_REQUESTS)
        self.assertIn('Retry-After', response.headers)
    
    def test_throttle_headers_present(self):
        """Test that throttle headers are present in responses"""
        endpoint = '/api/v1/writer/tasks/auto_generate_content/'
        response = self.client.post(endpoint, {})
        
        self.assertIn('X-Throttle-Limit', response.headers)
        self.assertIn('X-Throttle-Remaining', response.headers)
    
    def test_debug_throttle_bypass(self):
        """Test that throttling is bypassed in debug mode"""
        from django.conf import settings
        original_debug = settings.DEBUG
        
        try:
            settings.DEBUG = True
            
            # Make 20 requests (should all succeed in debug mode)
            endpoint = '/api/v1/writer/tasks/auto_generate_content/'
            for i in range(20):
                response = self.client.post(endpoint, {})
                self.assertNotEqual(response.status_code, status.HTTP_429_TOO_MANY_REQUESTS)
        finally:
            settings.DEBUG = original_debug

Estimated Time: 4 hours

Task 7: Changelog Entry

Goal

Document the rate limiting implementation in the changelog.

Implementation Steps

Step 7.1: Update CHANGELOG.md

File: CHANGELOG.md

Add Entry:

## [Unreleased] - 2025-01-XX

### Added
- **Rate Limiting**: Added scoped API rate limiting for AI, image, auth, and content actions
  - Configured DRF throttling with scoped rate limits
  - AI functions: 10 requests/minute
  - Image generation: 15 requests/minute
  - Content operations: 30 requests/minute
  - Authentication: 20 requests/minute
  - Added `DebugScopedRateThrottle` with debug mode bypass
  - Rate limit information exposed via `X-Throttle-Limit` and `X-Throttle-Remaining` headers
  - Added rate limit logging for abuse detection
  - Frontend rate limit handling and UI indicators

### Changed
- All ViewSets now have explicit `throttle_scope` assignments
- Rate limiting can be bypassed in development mode via `IGNY8_DEBUG_THROTTLE=True`

### Security
- Rate limiting prevents API abuse and ensures fair resource usage
- Abuse detection logging identifies repeated rate limit hits

### Affected Areas
- API Layer (`igny8_core/api/throttles.py`)
- All ViewSets (throttle scope assignments)
- Frontend (`frontend/src/services/api.ts`)
- Settings (`backend/igny8_core/settings.py`)

### Migration Guide
1. **Development:**
   - Set `IGNY8_DEBUG_THROTTLE=True` to bypass throttling during development
   - Or set `DEBUG=True` in Django settings

2. **Production:**
   - Rate limits are automatically enforced
   - Monitor rate limit logs for abuse patterns
   - Adjust rate limits in `DEFAULT_THROTTLE_RATES` if needed

3. **Frontend:**
   - Handle 429 responses gracefully
   - Display rate limit information to users
   - Show warnings when approaching limits

Estimated Time: 1 hour

Testing Strategy

Unit Tests

File: backend/igny8_core/api/tests/test_throttles.py

Test Cases:

Throttle class rate limit enforcement
Debug mode bypass
Plan-based throttling (if implemented)
Throttle header generation
Cache key generation

Integration Tests

Test Cases:

End-to-end rate limiting on actual endpoints
Throttle header presence and correctness
Rate limit reset after time window
Multiple users with separate rate limits
Abuse detection logging

Manual Testing

Checklist:

AI functions rate limited to 10/min
Image generation rate limited to 15/min
Auth endpoints rate limited to 20/min
Throttle headers present in responses
429 responses include Retry-After header
Debug mode bypasses throttling
Frontend displays rate limit warnings
Abuse detection logs rate limit hits

Rollout Plan

Phase 1: Foundation (Week 1)

✅ Task 1: Configure DRF throttling
✅ Task 2.1: Audit endpoints
✅ Unit tests for throttle classes

Phase 2: Scope Assignment (Week 2)

✅ Task 2.2: Assign throttle scopes
✅ Task 4.1: Verify throttle headers
✅ Integration testing

Phase 3: Frontend & Logging (Week 3)

✅ Task 4.2: Frontend rate limit handling
✅ Task 4.3: Abuse detection logging
✅ End-to-end testing

Phase 4: Advanced Features (Week 4 - Optional)

✅ Task 3: Plan-based throttling (if needed)
✅ Task 5: Quota integration (if needed)

Phase 5: Documentation & Release (Week 5)

✅ Task 6: Complete test scenarios
✅ Task 7: Changelog entry
✅ Release to staging
✅ Production deployment

Success Criteria

Definition of Done

✅ DRF throttling configured with appropriate rate limits
✅ All ViewSets have explicit throttle scopes
✅ Throttle headers exposed in responses
✅ Frontend handles rate limits gracefully
✅ Abuse detection logging implemented
✅ Debug mode bypass works correctly
✅ All test scenarios pass
✅ Documentation updated
✅ Changelog entry created

Metrics

Coverage: 100% of expensive endpoints have throttle scopes
Security: Rate limiting prevents abuse
User Experience: Rate limit information clearly communicated
Performance: Throttling has minimal overhead

Risk Assessment

Risks

Too Restrictive Limits: Rate limits may be too low for legitimate users
- Mitigation: Start with conservative limits, adjust based on usage patterns
Performance Impact: Throttling checks may add overhead
- Mitigation: Use efficient caching (Redis), minimal overhead expected
Frontend Breaking: Frontend may not handle 429 responses
- Mitigation: Update frontend error handling, test thoroughly
Debug Mode in Production: Accidental debug mode could bypass throttling
- Mitigation: Ensure DEBUG=False in production, use environment variables

Rollback Plan

If issues arise:

Increase rate limits temporarily
Disable throttling for specific scopes if needed
Revert throttle scope assignments
Document issues for future fixes

Document Status: Implementation Plan
Last Updated: 2025-01-XX
Next Review: After Phase 1 completion

34 KiB Raw Blame History

API Implementation Plan - Section 4: Apply Rate Limiting and Throttling Rules

Executive Summary

Table of Contents

Current State Analysis

Current Rate Limiting Status

Implementation Tasks

Overview

Task 1: Enable DRF Throttling System

Goal

Implementation Steps

Step 1.1: Configure DRF Throttling in Settings

Step 1.2: Add Debug Throttle Bypass

Task 2: Assign Throttle Scopes in ViewSets

Goal

Implementation Steps

Step 2.1: Audit Endpoints for Throttle Scopes

Step 2.2: Assign Throttle Scopes to ViewSets

Task 3: Override User/Account-Level Throttles (Optional)

Goal

Implementation Steps

Step 3.1: Create Plan-Based Throttle Class

Step 3.2: Apply Plan-Based Throttling

Task 4: Log + Expose Rate Limit Info

Goal

Implementation Steps

Step 4.1: Verify Throttle Headers

Step 4.2: Add Frontend Rate Limit Handling

Step 4.3: Implement Abuse Detection Logging

Task 5: Advanced Daily/Monthly Quotas Per Plan

Goal

Implementation Steps

Step 5.1: Integrate with Credit System

Task 6: Testing Matrix

Goal

Implementation Steps

Step 6.1: Create Test Scenarios

Task 7: Changelog Entry

Goal

Implementation Steps

Step 7.1: Update CHANGELOG.md

Testing Strategy

Unit Tests

Integration Tests

Manual Testing

Rollout Plan

Phase 1: Foundation (Week 1)

Phase 2: Scope Assignment (Week 2)

Phase 3: Frontend & Logging (Week 3)

Phase 4: Advanced Features (Week 4 - Optional)

Phase 5: Documentation & Release (Week 5)

Success Criteria

Definition of Done

Metrics

Risk Assessment

Risks

Rollback Plan

34 KiB

Raw Blame History