igny8/unified-api/API-IMPLEMENTATION-PLAN-SECTION4.md

# API Implementation Plan - Section 4: Apply Rate Limiting and Throttling Rules

**Date:** 2025-01-XX  
**Status:** Planning  
**Priority:** High  
**Related Document:** `API-ENDPOINTS-ANALYSIS.md`

---

## Executive Summary

This document outlines the implementation plan for **Section 4: Apply Rate Limiting and Throttling Rules** across the IGNY8 API layer. The goal is to prevent abuse, ensure fair resource usage, and enforce plan limits by introducing consistent per-user and per-function rate limits using DRF throttling and internal usage tracking.

**Key Objectives:**
- Enable DRF throttling system with scoped rate limits
- Assign throttle scopes to all major functions (AI, auth, content, images)
- Implement plan-based throttling (optional, future phase)
- Log and expose rate limit information
- Integrate with existing credit system for quota enforcement

---

## Table of Contents

1. [Current State Analysis](#current-state-analysis)
2. [Implementation Tasks](#implementation-tasks)
3. [Task 1: Enable DRF Throttling System](#task-1-enable-drf-throttling-system)
4. [Task 2: Assign Throttle Scopes in ViewSets](#task-2-assign-throttle-scopes-in-viewsets)
5. [Task 3: Override User/Account-Level Throttles (Optional)](#task-3-override-useraccount-level-throttles-optional)
6. [Task 4: Log + Expose Rate Limit Info](#task-4-log--expose-rate-limit-info)
7. [Task 5: Advanced Daily/Monthly Quotas Per Plan](#task-5-advanced-dailymonthly-quotas-per-plan)
8. [Task 6: Testing Matrix](#task-6-testing-matrix)
9. [Task 7: Changelog Entry](#task-7-changelog-entry)
10. [Testing Strategy](#testing-strategy)
11. [Rollout Plan](#rollout-plan)
12. [Success Criteria](#success-criteria)

---

## Current State Analysis

### Current Rate Limiting Status

Based on `API-ENDPOINTS-ANALYSIS.md`:

1. **Rate Limiting:** ❌ **Not Implemented**
   - All endpoints are currently unlimited
   - No throttling configuration in settings
   - No rate limit enforcement

2. **Credit System:**
   - `CreditUsageLog` model exists for tracking usage
   - `CreditBalance` model tracks account credits
   - Credit deduction happens but no rate limiting based on credits

3. **Plan System:**
   - `Plan` model exists with different tiers
   - Plans have credit limits but no API rate limits defined
   - No plan-based throttling implemented

4. **AI Functions:**
   - High-cost operations (AI generation, image generation)
   - Currently no rate limiting on these expensive endpoints
   - Risk of abuse and cost overruns

---

## Implementation Tasks

### Overview

| Task ID | Task Name | Priority | Estimated Effort | Dependencies |
|---------|-----------|----------|------------------|--------------|
| 1.1 | Configure DRF throttling in settings | High | 2 hours | None |
| 1.2 | Add debug throttle bypass | High | 1 hour | 1.1 |
| 2.1 | Audit endpoints for throttle scopes | Medium | 3 hours | None |
| 2.2 | Assign throttle scopes to ViewSets | High | 6 hours | 1.1 |
| 3.1 | Create plan-based throttle class (optional) | Low | 8 hours | 1.1 |
| 4.1 | Verify throttle headers | High | 2 hours | 2.2 |
| 4.2 | Add frontend rate limit handling | Medium | 4 hours | 4.1 |
| 4.3 | Implement abuse detection logging | Medium | 3 hours | 2.2 |
| 5.1 | Integrate with credit system | Low | 6 hours | 3.1 |
| 6.1 | Create test scenarios | High | 4 hours | 2.2 |
| 7.1 | Create changelog entry | Low | 1 hour | All tasks |

**Total Estimated Effort:** ~40 hours

---

## Task 1: Enable DRF Throttling System

### Goal
Configure Django REST Framework to use scoped rate throttling with appropriate rate limits for different endpoint types.

### Implementation Steps

#### Step 1.1: Configure DRF Throttling in Settings

**File:** `backend/igny8_core/settings.py`

**Update REST_FRAMEWORK configuration:**
```python
REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'igny8_core.api.pagination.CustomPageNumberPagination',
    'PAGE_SIZE': 10,
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.SearchFilter',
        'rest_framework.filters.OrderingFilter',
    ],
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.AllowAny',  # Will be changed in Section 2
    ],
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'igny8_core.api.authentication.JWTAuthentication',
        'igny8_core.api.authentication.CSRFExemptSessionAuthentication',
        'rest_framework.authentication.BasicAuthentication',
    ],
    'EXCEPTION_HANDLER': 'igny8_core.api.exception_handlers.custom_exception_handler',
    # Throttling Configuration
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.ScopedRateThrottle',
    ],
    'DEFAULT_THROTTLE_RATES': {
        # AI Functions - Expensive operations
        'ai_function': '10/min',           # AI content generation, clustering
        'image_gen': '15/min',            # Image generation
        
        # Content Operations
        'content_write': '30/min',        # Content creation, updates
        'content_read': '100/min',        # Content listing, retrieval
        
        # Authentication
        'auth': '20/min',                 # Login, register, password reset
        'auth_strict': '5/min',           # Sensitive auth operations
        
        # Planner Operations
        'planner': '60/min',              # Keyword, cluster, idea operations
        'planner_ai': '10/min',           # AI-powered planner operations
        
        # Writer Operations
        'writer': '60/min',               # Task, content management
        'writer_ai': '10/min',            # AI-powered writer operations
        
        # System Operations
        'system': '100/min',              # Settings, prompts, profiles
        'system_admin': '30/min',         # Admin-only system operations
        
        # Billing Operations
        'billing': '30/min',              # Credit queries, usage logs
        'billing_admin': '10/min',        # Credit management (admin)
        
        # Default fallback
        'default': '100/min',             # Default for endpoints without scope
    },
}
```

#### Step 1.2: Add Debug Throttle Bypass

**File:** `backend/igny8_core/settings.py`

**Add environment variable:**
```python
import os

# Throttling Configuration
IGNY8_DEBUG_THROTTLE = os.getenv('IGNY8_DEBUG_THROTTLE', 'False').lower() == 'true'

# Only apply throttling if not in debug mode or explicitly enabled
if not DEBUG and not IGNY8_DEBUG_THROTTLE:
    REST_FRAMEWORK['DEFAULT_THROTTLE_CLASSES'] = [
        'rest_framework.throttling.ScopedRateThrottle',
    ]
else:
    # In debug mode, use a no-op throttle class
    REST_FRAMEWORK['DEFAULT_THROTTLE_CLASSES'] = []
```

**Create Custom Throttle Class with Debug Bypass:**

**File:** `backend/igny8_core/api/throttles.py`

**Implementation:**
```python
"""
Custom Throttle Classes for IGNY8 API
"""

from django.conf import settings
from rest_framework.throttling import ScopedRateThrottle


class DebugScopedRateThrottle(ScopedRateThrottle):
    """
    Scoped rate throttle that can be bypassed in debug mode.
    
    Usage:
        throttle_scope = 'ai_function'
    """
    
    def allow_request(self, request, view):
        """
        Check if request should be throttled.
        Bypass throttling if DEBUG=True or IGNY8_DEBUG_THROTTLE=True.
        """
        # Bypass throttling in debug mode
        if settings.DEBUG or getattr(settings, 'IGNY8_DEBUG_THROTTLE', False):
            return True
        
        # Use parent class throttling logic
        return super().allow_request(request, view)
```

**Update settings to use custom throttle:**
```python
'DEFAULT_THROTTLE_CLASSES': [
    'igny8_core.api.throttles.DebugScopedRateThrottle',
],
```

**Estimated Time:** 2 hours (settings) + 1 hour (debug bypass) = 3 hours

---

## Task 2: Assign Throttle Scopes in ViewSets

### Goal
Assign appropriate throttle scopes to all ViewSets and custom actions based on their function and resource cost.

### Implementation Steps

#### Step 2.1: Audit Endpoints for Throttle Scopes

**Action Items:**
1. List all ViewSets and custom actions
2. Categorize by function type (AI, content, auth, etc.)
3. Identify high-cost operations (AI, image generation)
4. Create mapping of endpoints to throttle scopes

**Endpoint Categories:**

**AI Functions (10/min):**
- `POST /api/v1/planner/keywords/auto_cluster/` - AI clustering
- `POST /api/v1/planner/clusters/auto_generate_ideas/` - AI idea generation
- `POST /api/v1/writer/tasks/auto_generate_content/` - AI content generation
- `POST /api/v1/writer/content/generate_image_prompts/` - AI prompt generation

**Image Generation (15/min):**
- `POST /api/v1/writer/images/generate_images/` - Image generation
- `POST /api/v1/writer/images/auto_generate/` - Legacy image generation

**Content Write (30/min):**
- `POST /api/v1/writer/tasks/` - Create task
- `POST /api/v1/writer/content/` - Create content
- `PUT /api/v1/writer/tasks/{id}/` - Update task
- `PUT /api/v1/writer/content/{id}/` - Update content

**Planner AI (10/min):**
- `POST /api/v1/planner/keywords/auto_cluster/`
- `POST /api/v1/planner/clusters/auto_generate_ideas/`

**Planner Standard (60/min):**
- `GET /api/v1/planner/keywords/` - List keywords
- `POST /api/v1/planner/keywords/` - Create keyword
- `GET /api/v1/planner/clusters/` - List clusters
- `POST /api/v1/planner/ideas/` - Create idea

**Auth (20/min):**
- `POST /api/v1/auth/login/`
- `POST /api/v1/auth/register/`
- `POST /api/v1/auth/change-password/`

**Auth Strict (5/min):**
- `POST /api/v1/auth/password-reset/` - If exists
- `POST /api/v1/auth/users/invite/` - User invitation

**System Admin (30/min):**
- `POST /api/v1/system/settings/integrations/{pk}/save/`
- `POST /api/v1/system/settings/integrations/{pk}/test/`

**Billing Admin (10/min):**
- `POST /api/v1/billing/credits/transactions/` - Create transaction

**Estimated Time:** 3 hours

#### Step 2.2: Assign Throttle Scopes to ViewSets

**Example Implementation:**

**Planner Module:**

**File:** `backend/planner/api/viewsets.py`

```python
from igny8_core.api.viewsets import BaseTenantViewSet

class KeywordViewSet(BaseTenantViewSet):
    throttle_scope = 'planner'  # Default for standard operations
    queryset = Keyword.objects.all()
    serializer_class = KeywordSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='ai_function'  # Override for AI operations
    )
    def auto_cluster(self, request):
        # AI clustering logic
        pass


class ClusterViewSet(BaseTenantViewSet):
    throttle_scope = 'planner'
    queryset = Cluster.objects.all()
    serializer_class = ClusterSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='planner_ai'  # AI-powered operations
    )
    def auto_generate_ideas(self, request):
        # AI idea generation logic
        pass
```

**Writer Module:**

**File:** `backend/writer/api/viewsets.py`

```python
class TasksViewSet(BaseTenantViewSet):
    throttle_scope = 'writer'  # Default for standard operations
    queryset = Task.objects.all()
    serializer_class = TaskSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='ai_function'  # AI content generation
    )
    def auto_generate_content(self, request):
        # AI content generation logic
        pass


class ImagesViewSet(BaseTenantViewSet):
    throttle_scope = 'writer'
    queryset = Image.objects.all()
    serializer_class = ImageSerializer
    
    @action(
        detail=False,
        methods=['post'],
        throttle_scope='image_gen'  # Image generation
    )
    def generate_images(self, request):
        # Image generation logic
        pass
```

**Auth Module:**

**File:** `backend/auth/api/views.py`

```python
from rest_framework.views import APIView
from rest_framework.throttling import ScopedRateThrottle

class LoginView(APIView):
    throttle_scope = 'auth'
    throttle_classes = [DebugScopedRateThrottle]
    
    def post(self, request):
        # Login logic
        pass


class RegisterView(APIView):
    throttle_scope = 'auth'
    throttle_classes = [DebugScopedRateThrottle]
    
    def post(self, request):
        # Registration logic
        pass


class ChangePasswordView(APIView):
    throttle_scope = 'auth'
    throttle_classes = [DebugScopedRateThrottle]
    
    def post(self, request):
        # Password change logic
        pass
```

**System Module:**

**File:** `backend/system/api/viewsets.py`

```python
class IntegrationSettingsViewSet(BaseTenantViewSet):
    throttle_scope = 'system'
    
    @action(
        detail=True,
        methods=['post'],
        throttle_scope='system_admin'  # Admin operations
    )
    def save(self, request, pk=None):
        # Save integration settings
        pass
    
    @action(
        detail=True,
        methods=['post'],
        throttle_scope='system_admin'
    )
    def test(self, request, pk=None):
        # Test integration connection
        pass
```

**Billing Module:**

**File:** `backend/billing/api/viewsets.py`

```python
class CreditTransactionViewSet(BaseTenantViewSet):
    throttle_scope = 'billing'
    
    # Read-only operations use 'billing' scope
    # Write operations would use 'billing_admin' if implemented
```

**Estimated Time:** 6 hours

---

## Task 3: Override User/Account-Level Throttles (Optional)

### Goal
Implement plan-based throttling that applies different rate limits based on user's subscription plan.

### Implementation Steps

#### Step 3.1: Create Plan-Based Throttle Class

**File:** `backend/igny8_core/api/throttles.py`

**Implementation:**
```python
"""
Custom Throttle Classes for IGNY8 API
"""

from django.conf import settings
from rest_framework.throttling import SimpleRateThrottle


class PlanBasedRateThrottle(SimpleRateThrottle):
    """
    Rate throttle that applies different limits based on user's plan.
    
    Plan tiers:
    - Free: 10/min for AI functions
    - Pro: 100/min for AI functions
    - Enterprise: Unlimited (or very high limit)
    
    Usage:
        throttle_scope = 'ai_function'  # Base scope
        # Plan limits override base scope
    """
    
    # Plan-based rate limits
    PLAN_RATES = {
        'free': {
            'ai_function': '10/min',
            'image_gen': '15/min',
            'content_write': '30/min',
        },
        'pro': {
            'ai_function': '100/min',
            'image_gen': '50/min',
            'content_write': '100/min',
        },
        'enterprise': {
            'ai_function': '1000/min',  # Effectively unlimited
            'image_gen': '200/min',
            'content_write': '500/min',
        },
    }
    
    def get_rate(self):
        """
        Get rate limit based on user's plan.
        """
        # Get scope from view
        scope = getattr(self.view, 'throttle_scope', None)
        if not scope:
            return None
        
        # Get user's plan
        user = self.request.user
        if not user or not user.is_authenticated:
            # Anonymous users use default rate
            return self.get_default_rate(scope)
        
        # Get account and plan
        account = getattr(user, 'account', None)
        if not account:
            return self.get_default_rate(scope)
        
        plan = getattr(account, 'plan', None)
        if not plan:
            return self.get_default_rate(scope)
        
        plan_name = plan.name.lower() if hasattr(plan, 'name') else 'free'
        
        # Get rate for plan and scope
        plan_rates = self.PLAN_RATES.get(plan_name, self.PLAN_RATES['free'])
        rate = plan_rates.get(scope, self.get_default_rate(scope))
        
        return rate
    
    def get_default_rate(self, scope):
        """
        Get default rate from settings if plan-based rate not found.
        """
        throttle_rates = getattr(settings, 'REST_FRAMEWORK', {}).get('DEFAULT_THROTTLE_RATES', {})
        return throttle_rates.get(scope, '100/min')
    
    def get_cache_key(self, request, view):
        """
        Generate cache key based on user ID and scope.
        """
        if request.user.is_authenticated:
            ident = request.user.pk
        else:
            ident = self.get_ident(request)
        
        scope = getattr(view, 'throttle_scope', None)
        return self.cache_format % {
            'scope': scope,
            'ident': ident
        }
```

#### Step 3.2: Apply Plan-Based Throttling

**Update settings to use plan-based throttle for AI functions:**

**File:** `backend/igny8_core/settings.py`

```python
# For AI functions, use plan-based throttling
# For other functions, use scoped throttling

# This would require custom throttle class selection per ViewSet
# Or use a mixin that selects throttle class based on scope
```

**Alternative: Use in specific ViewSets:**

```python
from igny8_core.api.throttles import PlanBasedRateThrottle

class TasksViewSet(BaseTenantViewSet):
    throttle_scope = 'ai_function'
    throttle_classes = [PlanBasedRateThrottle]  # Use plan-based for this ViewSet
    
    @action(detail=False, methods=['post'])
    def auto_generate_content(self, request):
        # AI content generation
        pass
```

**Estimated Time:** 8 hours (optional, future phase)

---

## Task 4: Log + Expose Rate Limit Info

### Goal
Ensure rate limit information is properly exposed to clients and logged for abuse detection.

### Implementation Steps

#### Step 4.1: Verify Throttle Headers

**DRF automatically adds these headers:**
- `X-Throttle-Limit`: Maximum number of requests allowed
- `X-Throttle-Remaining`: Number of requests remaining
- `Retry-After`: Seconds to wait before retrying (when throttled)

**Test in Postman:**
1. Make requests to throttled endpoint
2. Check response headers
3. Verify headers are present and correct

**Example Response Headers:**
```
HTTP/1.1 200 OK
X-Throttle-Limit: 10
X-Throttle-Remaining: 9
X-Throttle-Reset: 1640995200
```

**When Throttled (429):**
```
HTTP/1.1 429 Too Many Requests
X-Throttle-Limit: 10
X-Throttle-Remaining: 0
Retry-After: 60
```

**Estimated Time:** 2 hours

#### Step 4.2: Add Frontend Rate Limit Handling

**File:** `frontend/src/services/api.ts`

**Update fetchAPI to handle rate limits:**
```typescript
export async function fetchAPI(endpoint: string, options?: RequestInit) {
  const response = await fetch(`${API_BASE_URL}${endpoint}`, {
    ...options,
    headers: {
      ...options?.headers,
      'Authorization': `Bearer ${getToken()}`,
    },
  });

  // Check for rate limiting
  if (response.status === 429) {
    const retryAfter = response.headers.get('Retry-After');
    const throttleLimit = response.headers.get('X-Throttle-Limit');
    const throttleRemaining = response.headers.get('X-Throttle-Remaining');
    
    // Show user-friendly error
    showNotification(
      `Rate limit exceeded. Please wait ${retryAfter} seconds before trying again.`,
      'error'
    );
    
    // Store rate limit info for UI
    if (throttleLimit && throttleRemaining !== null) {
      storeRateLimitInfo({
        limit: parseInt(throttleLimit),
        remaining: parseInt(throttleRemaining),
        retryAfter: retryAfter ? parseInt(retryAfter) : 60,
      });
    }
    
    throw new Error('Rate limit exceeded');
  }

  // Check throttle headers on successful requests
  const throttleLimit = response.headers.get('X-Throttle-Limit');
  const throttleRemaining = response.headers.get('X-Throttle-Remaining');
  
  if (throttleLimit && throttleRemaining !== null) {
    const remaining = parseInt(throttleRemaining);
    const limit = parseInt(throttleLimit);
    const percentage = (remaining / limit) * 100;
    
    // Show warning if close to limit
    if (percentage < 20) {
      showNotification(
        `Warning: You have ${remaining} of ${limit} requests remaining.`,
        'warning'
      );
    }
    
    // Store for UI display
    storeRateLimitInfo({
      limit,
      remaining,
      percentage,
    });
  }

  return response;
}
```

**Add Rate Limit Display Component:**

**File:** `frontend/src/components/RateLimitIndicator.tsx`

```typescript
import { useRateLimitStore } from '@/stores/rateLimitStore';

export function RateLimitIndicator() {
  const { limit, remaining, percentage } = useRateLimitStore();
  
  if (!limit) return null;
  
  return (
    <div className="rate-limit-indicator">
      <span>API Calls: {remaining} / {limit}</span>
      <div className="progress-bar">
        <div 
          className="progress-fill" 
          style={{ width: `${percentage}%` }}
        />
      </div>
      {percentage < 20 && (
        <span className="warning">Low on requests</span>
      )}
    </div>
  );
}
```

**Estimated Time:** 4 hours

#### Step 4.3: Implement Abuse Detection Logging

**File:** `backend/igny8_core/api/middleware.py`

**Add rate limit logging middleware:**
```python
"""
Rate Limit Logging Middleware
"""

import logging
from django.utils.deprecation import MiddlewareMixin

logger = logging.getLogger(__name__)


class RateLimitLoggingMiddleware(MiddlewareMixin):
    """
    Middleware that logs rate limit hits for abuse detection.
    """
    
    def process_response(self, request, response):
        """
        Log rate limit hits (429 responses).
        """
        if response.status_code == 429:
            # Log rate limit hit
            logger.warning(
                f"Rate limit exceeded: {request.method} {request.path}",
                extra={
                    'user_id': request.user.id if request.user.is_authenticated else None,
                    'ip_address': self.get_client_ip(request),
                    'user_agent': request.META.get('HTTP_USER_AGENT'),
                    'throttle_scope': getattr(request, 'throttle_scope', None),
                }
            )
            
            # Check for repeated hits (potential abuse)
            # This could be enhanced with Redis/cache to track frequency
            self.check_for_abuse(request)
        
        return response
    
    def get_client_ip(self, request):
        """
        Get client IP address.
        """
        x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
        if x_forwarded_for:
            ip = x_forwarded_for.split(',')[0]
        else:
            ip = request.META.get('REMOTE_ADDR')
        return ip
    
    def check_for_abuse(self, request):
        """
        Check if user is repeatedly hitting rate limits (potential abuse).
        """
        # TODO: Implement abuse detection logic
        # - Track rate limit hits per user/IP
        # - Alert if threshold exceeded
        # - Optionally block user/IP temporarily
        pass
```

**Register middleware:**

**File:** `backend/igny8_core/settings.py`

```python
MIDDLEWARE = [
    # ... other middleware ...
    'igny8_core.api.middleware.RateLimitLoggingMiddleware',
    # ... other middleware ...
]
```

**Estimated Time:** 3 hours

---

## Task 5: Advanced Daily/Monthly Quotas Per Plan

### Goal
Implement quota-based usage limits that go beyond rate limiting, integrated with the existing credit system.

### Implementation Steps

#### Step 5.1: Integrate with Credit System

**This task extends the existing credit system to enforce quotas:**

**File:** `backend/igny8_core/api/middleware.py`

**Add quota check middleware:**
```python
"""
Quota Enforcement Middleware
"""

from django.http import JsonResponse
from rest_framework import status
from igny8_core.api.response import error_response


class QuotaEnforcementMiddleware(MiddlewareMixin):
    """
    Middleware that enforces daily/monthly quotas based on plan and credits.
    """
    
    def process_request(self, request):
        """
        Check if user has exceeded quota before processing request.
        """
        # Only check for authenticated users
        if not request.user or not request.user.is_authenticated:
            return None
        
        # Get account
        account = getattr(request.user, 'account', None)
        if not account:
            return None
        
        # Check if endpoint requires quota check
        if not self.requires_quota_check(request):
            return None
        
        # Check credits
        from igny8_core.modules.billing.models import CreditBalance
        
        try:
            balance = CreditBalance.objects.get(account=account)
            
            # Check if credits are exhausted
            if balance.credits <= 0:
                return error_response(
                    error="Credits exhausted. Please purchase more credits to continue.",
                    status_code=status.HTTP_402_PAYMENT_REQUIRED
                )
            
            # Check daily/monthly quotas (if implemented)
            # This would require additional quota tracking models
            
        except CreditBalance.DoesNotExist:
            # No balance record - allow request (will be created on first use)
            pass
        
        return None
    
    def requires_quota_check(self, request):
        """
        Check if this endpoint requires quota validation.
        """
        # Check if it's an AI function or expensive operation
        throttle_scope = getattr(request, 'throttle_scope', None)
        expensive_scopes = ['ai_function', 'image_gen', 'planner_ai', 'writer_ai']
        
        return throttle_scope in expensive_scopes
```

**Estimated Time:** 6 hours (optional, future phase)

---

## Task 6: Testing Matrix

### Goal
Create comprehensive test scenarios to validate rate limiting works correctly.

### Implementation Steps

#### Step 6.1: Create Test Scenarios

**Test Cases:**

**1. AI Function Rate Limiting (10/min)**
```python
# Test: Make 11 requests to /api/v1/writer/tasks/auto_generate_content/
# Expected: 
# - Requests 1-10: 200 OK
# - Request 11: 429 Too Many Requests
# - Headers: X-Throttle-Limit: 10, X-Throttle-Remaining: 0, Retry-After: 60
```

**2. Image Generation Rate Limiting (15/min)**
```python
# Test: Make 16 requests to /api/v1/writer/images/generate_images/
# Expected:
# - Requests 1-15: 200 OK
# - Request 16: 429 Too Many Requests
```

**3. Auth Rate Limiting (20/min)**
```python
# Test: Make 21 login attempts
# Expected:
# - Requests 1-20: 200 OK or 401 Unauthorized (depending on credentials)
# - Request 21: 429 Too Many Requests
```

**4. Debug Mode Bypass**
```python
# Test: Set IGNY8_DEBUG_THROTTLE=True
# Expected: All requests allowed regardless of rate limits
```

**5. Throttle Headers Verification**
```python
# Test: Make requests and check headers
# Expected:
# - X-Throttle-Limit header present
# - X-Throttle-Remaining header present and decrements
# - Retry-After header present when throttled
```

**6. Plan-Based Throttling (if implemented)**
```python
# Test: Free plan user vs Pro plan user
# Expected:
# - Free plan: 10/min for AI functions
# - Pro plan: 100/min for AI functions
```

**Automated Test Implementation:**

**File:** `backend/igny8_core/api/tests/test_throttling.py`

```python
"""
Tests for Rate Limiting and Throttling
"""

from django.test import TestCase
from rest_framework.test import APIClient
from rest_framework import status
from django.contrib.auth import get_user_model
from igny8_core.auth.models import Account

User = get_user_model()


class ThrottlingTestCase(TestCase):
    def setUp(self):
        self.client = APIClient()
        self.account = Account.objects.create(name="Test Account")
        self.user = User.objects.create_user(
            email="test@example.com",
            password="testpass123",
            account=self.account
        )
        self.client.force_authenticate(user=self.user)
    
    def test_ai_function_rate_limit(self):
        """Test that AI functions are rate limited to 10/min"""
        endpoint = '/api/v1/writer/tasks/auto_generate_content/'
        
        # Make 10 requests (should all succeed)
        for i in range(10):
            response = self.client.post(endpoint, {})
            self.assertIn(response.status_code, [200, 400, 500])  # Not throttled
        
        # 11th request should be throttled
        response = self.client.post(endpoint, {})
        self.assertEqual(response.status_code, status.HTTP_429_TOO_MANY_REQUESTS)
        self.assertIn('Retry-After', response.headers)
    
    def test_throttle_headers_present(self):
        """Test that throttle headers are present in responses"""
        endpoint = '/api/v1/writer/tasks/auto_generate_content/'
        response = self.client.post(endpoint, {})
        
        self.assertIn('X-Throttle-Limit', response.headers)
        self.assertIn('X-Throttle-Remaining', response.headers)
    
    def test_debug_throttle_bypass(self):
        """Test that throttling is bypassed in debug mode"""
        from django.conf import settings
        original_debug = settings.DEBUG
        
        try:
            settings.DEBUG = True
            
            # Make 20 requests (should all succeed in debug mode)
            endpoint = '/api/v1/writer/tasks/auto_generate_content/'
            for i in range(20):
                response = self.client.post(endpoint, {})
                self.assertNotEqual(response.status_code, status.HTTP_429_TOO_MANY_REQUESTS)
        finally:
            settings.DEBUG = original_debug
```

**Estimated Time:** 4 hours

---

## Task 7: Changelog Entry

### Goal
Document the rate limiting implementation in the changelog.

### Implementation Steps

#### Step 7.1: Update CHANGELOG.md

**File:** `CHANGELOG.md`

**Add Entry:**
```markdown
## [Unreleased] - 2025-01-XX

### Added
- **Rate Limiting**: Added scoped API rate limiting for AI, image, auth, and content actions
  - Configured DRF throttling with scoped rate limits
  - AI functions: 10 requests/minute
  - Image generation: 15 requests/minute
  - Content operations: 30 requests/minute
  - Authentication: 20 requests/minute
  - Added `DebugScopedRateThrottle` with debug mode bypass
  - Rate limit information exposed via `X-Throttle-Limit` and `X-Throttle-Remaining` headers
  - Added rate limit logging for abuse detection
  - Frontend rate limit handling and UI indicators

### Changed
- All ViewSets now have explicit `throttle_scope` assignments
- Rate limiting can be bypassed in development mode via `IGNY8_DEBUG_THROTTLE=True`

### Security
- Rate limiting prevents API abuse and ensures fair resource usage
- Abuse detection logging identifies repeated rate limit hits

### Affected Areas
- API Layer (`igny8_core/api/throttles.py`)
- All ViewSets (throttle scope assignments)
- Frontend (`frontend/src/services/api.ts`)
- Settings (`backend/igny8_core/settings.py`)

### Migration Guide
1. **Development:**
   - Set `IGNY8_DEBUG_THROTTLE=True` to bypass throttling during development
   - Or set `DEBUG=True` in Django settings

2. **Production:**
   - Rate limits are automatically enforced
   - Monitor rate limit logs for abuse patterns
   - Adjust rate limits in `DEFAULT_THROTTLE_RATES` if needed

3. **Frontend:**
   - Handle 429 responses gracefully
   - Display rate limit information to users
   - Show warnings when approaching limits
```

**Estimated Time:** 1 hour

---

## Testing Strategy

### Unit Tests

**File:** `backend/igny8_core/api/tests/test_throttles.py`

**Test Cases:**
- Throttle class rate limit enforcement
- Debug mode bypass
- Plan-based throttling (if implemented)
- Throttle header generation
- Cache key generation

### Integration Tests

**Test Cases:**
- End-to-end rate limiting on actual endpoints
- Throttle header presence and correctness
- Rate limit reset after time window
- Multiple users with separate rate limits
- Abuse detection logging

### Manual Testing

**Checklist:**
- [ ] AI functions rate limited to 10/min
- [ ] Image generation rate limited to 15/min
- [ ] Auth endpoints rate limited to 20/min
- [ ] Throttle headers present in responses
- [ ] 429 responses include Retry-After header
- [ ] Debug mode bypasses throttling
- [ ] Frontend displays rate limit warnings
- [ ] Abuse detection logs rate limit hits

---

## Rollout Plan

### Phase 1: Foundation (Week 1)
- ✅ Task 1: Configure DRF throttling
- ✅ Task 2.1: Audit endpoints
- ✅ Unit tests for throttle classes

### Phase 2: Scope Assignment (Week 2)
- ✅ Task 2.2: Assign throttle scopes
- ✅ Task 4.1: Verify throttle headers
- ✅ Integration testing

### Phase 3: Frontend & Logging (Week 3)
- ✅ Task 4.2: Frontend rate limit handling
- ✅ Task 4.3: Abuse detection logging
- ✅ End-to-end testing

### Phase 4: Advanced Features (Week 4 - Optional)
- ✅ Task 3: Plan-based throttling (if needed)
- ✅ Task 5: Quota integration (if needed)

### Phase 5: Documentation & Release (Week 5)
- ✅ Task 6: Complete test scenarios
- ✅ Task 7: Changelog entry
- ✅ Release to staging
- ✅ Production deployment

---

## Success Criteria

### Definition of Done

1. ✅ DRF throttling configured with appropriate rate limits
2. ✅ All ViewSets have explicit throttle scopes
3. ✅ Throttle headers exposed in responses
4. ✅ Frontend handles rate limits gracefully
5. ✅ Abuse detection logging implemented
6. ✅ Debug mode bypass works correctly
7. ✅ All test scenarios pass
8. ✅ Documentation updated
9. ✅ Changelog entry created

### Metrics

- **Coverage:** 100% of expensive endpoints have throttle scopes
- **Security:** Rate limiting prevents abuse
- **User Experience:** Rate limit information clearly communicated
- **Performance:** Throttling has minimal overhead

---

## Risk Assessment

### Risks

1. **Too Restrictive Limits:** Rate limits may be too low for legitimate users
   - **Mitigation:** Start with conservative limits, adjust based on usage patterns

2. **Performance Impact:** Throttling checks may add overhead
   - **Mitigation:** Use efficient caching (Redis), minimal overhead expected

3. **Frontend Breaking:** Frontend may not handle 429 responses
   - **Mitigation:** Update frontend error handling, test thoroughly

4. **Debug Mode in Production:** Accidental debug mode could bypass throttling
   - **Mitigation:** Ensure DEBUG=False in production, use environment variables

### Rollback Plan

If issues arise:
1. Increase rate limits temporarily
2. Disable throttling for specific scopes if needed
3. Revert throttle scope assignments
4. Document issues for future fixes

---

**Document Status:** Implementation Plan  
**Last Updated:** 2025-01-XX  
**Next Review:** After Phase 1 completion