igny8/unified-api/API-IMPLEMENTATION-PLAN-SECTION3.md

API Implementation Plan - Section 3: Implement Consistent Error Handling

Date: 2025-01-XX
Status: Planning
Priority: High
Related Document: API-ENDPOINTS-ANALYSIS.md, API-IMPLEMENTATION-PLAN-SECTION1.md

---

Executive Summary

This document outlines the implementation plan for Section 3: Implement Consistent Error Handling across the IGNY8 API layer. The goal is to ensure all exceptions, validation errors, and system-level failures return uniform, predictable JSON responses with proper HTTP status codes and user-readable messages.

Key Objectives:

• Create centralized exception handler that wraps all errors in unified format
• Unify validation error format across all endpoints
• Implement proper logging for debugging and monitoring
• Add developer-focused debug support for development environments
• Ensure consistent error responses for all failure scenarios

Note: This section builds on Section 1 Task 4 (Exception Handlers) and extends it with logging, debugging, and validation error unification.

---

Table of Contents

1. Current State Analysis
2. Implementation Tasks
3. Task 1: Create Centralized Exception Handler
4. Task 2: Register in Global DRF Settings
5. Task 3: Unify Validation Error Format
6. Task 4: Patch or Log Internal Server Errors
7. Task 5: Add Developer-Focused Debug Support
8. Task 6: Test Scenarios
9. Task 7: Changelog Entry
10. Testing Strategy
11. Rollout Plan
12. Success Criteria

---

Current State Analysis

Current Error Handling Issues

Based on API-ENDPOINTS-ANALYSIS.md and codebase analysis:

1. Inconsistent Error Formats:

   • Some endpoints return { error: "..." }
   • Some return { success: false, message: "..." }
   • Some return DRF default format
   • Validation errors may not be consistently formatted

2. Exception Handling:

   • Section 1 Task 4 creates basic exception handler
   • No logging for unhandled exceptions
   • No request ID tracking for error correlation
   • No debug information in development mode

3. Validation Errors:

   • DRF serializer errors are handled automatically
   • Manual validation errors may not follow unified format
   • Field-specific errors may not be consistently structured

4. Error Logging:

   • Some errors are logged, but not consistently
   • No centralized error logging strategy
   • No integration with error tracking services (Sentry, etc.)

5. Frontend Error Handling:

   • Frontend has error parsing logic but may not handle all formats
   • Error messages may not be user-friendly
   • No consistent error display strategy

---

Implementation Tasks

Overview

Task ID	Task Name	Priority	Estimated Effort	Dependencies
1.1	Create centralized exception handler	High	4 hours	Section 1 Task 1 (response.py)
1.2	Enhance exception handler with logging	High	3 hours	1.1
2.1	Register exception handler in settings	High	1 hour	1.1
3.1	Audit validation error usage	Medium	3 hours	None
3.2	Unify validation error format	High	4 hours	1.1
4.1	Configure error logging	High	3 hours	None
4.2	Add request ID tracking	Medium	2 hours	4.1
4.3	Integrate error tracking service (optional)	Low	4 hours	4.1
5.1	Add debug mode support	Low	3 hours	1.1
6.1	Create test scenarios	High	4 hours	All tasks
7.1	Create changelog entry	Low	1 hour	All tasks

Total Estimated Effort: ~32 hours

---

Task 1: Create Centralized Exception Handler

Goal

Create a comprehensive exception handler that wraps all exceptions in the unified error format with proper logging and debugging support.

Implementation Steps

Step 1.1: Create Enhanced Exception Handler

File: backend/igny8_core/api/exception_handlers.py

Note: This extends Section 1 Task 4 with enhanced error handling, logging, and debug support.

Implementation:

"""
Centralized Exception Handler for IGNY8 API

This module provides a comprehensive exception handler that:
- Wraps all exceptions in unified error format
- Logs errors for debugging and monitoring
- Provides debug information in development mode
- Tracks request IDs for error correlation
"""

import logging
import traceback
import uuid
from django.conf import settings
from rest_framework.views import exception_handler as drf_exception_handler
from rest_framework import status
from rest_framework.response import Response
from igny8_core.api.response import error_response

logger = logging.getLogger(__name__)


def get_request_id(request):
    """
    Get or create request ID for error tracking.
    
    Request ID can be set by middleware or generated here.
    """
    # Check if request ID is already set (e.g., by middleware)
    request_id = getattr(request, 'request_id', None)
    
    if not request_id:
        # Generate new request ID
        request_id = str(uuid.uuid4())
        request.request_id = request_id
    
    return request_id


def extract_error_message(exc, response):
    """
    Extract user-friendly error message from exception.
    
    Args:
        exc: The exception instance
        response: DRF's exception handler response
    
    Returns:
        tuple: (error_message, errors_dict)
    """
    error_message = "An error occurred"
    errors = None
    
    # Handle DRF exceptions with 'detail' attribute
    if hasattr(exc, 'detail'):
        if isinstance(exc.detail, dict):
            # Validation errors - use as errors dict
            errors = exc.detail
            # Extract first error message as top-level error
            if errors:
                first_key = list(errors.keys())[0]
                first_error = errors[first_key]
                if isinstance(first_error, list) and first_error:
                    error_message = f"{first_key}: {first_error[0]}"
                else:
                    error_message = f"{first_key}: {first_error}"
            else:
                error_message = "Validation failed"
        elif isinstance(exc.detail, list):
            # List of errors
            error_message = exc.detail[0] if exc.detail else "An error occurred"
            errors = {"non_field_errors": exc.detail}
        else:
            # String error message
            error_message = str(exc.detail)
    elif response and hasattr(response, 'data'):
        # Try to extract from response data
        if isinstance(response.data, dict):
            # Check for common error message fields
            error_message = (
                response.data.get('error') or
                response.data.get('message') or
                response.data.get('detail') or
                str(response.data)
            )
            errors = response.data if 'error' not in response.data else None
        elif isinstance(response.data, list):
            error_message = response.data[0] if response.data else "An error occurred"
            errors = {"non_field_errors": response.data}
        else:
            error_message = str(response.data) if response.data else "An error occurred"
    else:
        # Generic exception
        error_message = str(exc) if exc else "An error occurred"
    
    return error_message, errors


def get_status_code_message(status_code):
    """
    Get default error message for HTTP status code.
    """
    status_messages = {
        status.HTTP_400_BAD_REQUEST: "Bad request",
        status.HTTP_401_UNAUTHORIZED: "Authentication required",
        status.HTTP_403_FORBIDDEN: "Permission denied",
        status.HTTP_404_NOT_FOUND: "Resource not found",
        status.HTTP_405_METHOD_NOT_ALLOWED: "Method not allowed",
        status.HTTP_409_CONFLICT: "Conflict",
        status.HTTP_422_UNPROCESSABLE_ENTITY: "Unprocessable entity",
        status.HTTP_429_TOO_MANY_REQUESTS: "Too many requests",
        status.HTTP_500_INTERNAL_SERVER_ERROR: "Internal server error",
        status.HTTP_502_BAD_GATEWAY: "Bad gateway",
        status.HTTP_503_SERVICE_UNAVAILABLE: "Service unavailable",
    }
    return status_messages.get(status_code, "An error occurred")


def custom_exception_handler(exc, context):
    """
    Centralized exception handler that wraps all exceptions in unified format.
    
    This handler:
    - Wraps all exceptions in unified error format
    - Logs errors with request context
    - Provides debug information in development mode
    - Tracks request IDs for error correlation
    
    Args:
        exc: The exception instance
        context: Dictionary containing request, view, and args
    
    Returns:
        Response: Error response in unified format, or None to use default
    """
    # Get request from context
    request = context.get('request')
    view = context.get('view')
    
    # Get request ID for tracking
    request_id = None
    if request:
        request_id = get_request_id(request)
    
    # Call DRF's default exception handler first
    response = drf_exception_handler(exc, context)
    
    # Determine status code
    if response is not None:
        status_code = response.status_code
    else:
        # Unhandled exception - default to 500
        status_code = status.HTTP_500_INTERNAL_SERVER_ERROR
    
    # Extract error message and details
    error_message, errors = extract_error_message(exc, response)
    
    # Use default message if error message is generic
    if error_message == "An error occurred" or not error_message:
        error_message = get_status_code_message(status_code)
    
    # Log the error
    log_level = logging.ERROR if status_code >= 500 else logging.WARNING
    log_message = f"API Error [{status_code}]: {error_message}"
    
    if request_id:
        log_message += f" (Request ID: {request_id})"
    
    # Include exception details in log
    logger.log(
        log_level,
        log_message,
        extra={
            'request_id': request_id,
            'status_code': status_code,
            'exception_type': type(exc).__name__,
            'view': view.__class__.__name__ if view else None,
            'path': request.path if request else None,
            'method': request.method if request else None,
        },
        exc_info=status_code >= 500,  # Include traceback for server errors
    )
    
    # Build error response
    error_response_data = {
        "success": False,
        "error": error_message,
    }
    
    # Add errors dict if present
    if errors:
        error_response_data["errors"] = errors
    
    # Add request ID for error tracking
    if request_id:
        error_response_data["request_id"] = request_id
    
    # Add debug information in development mode
    if settings.DEBUG:
        error_response_data["debug"] = {
            "exception_type": type(exc).__name__,
            "exception_message": str(exc),
            "view": view.__class__.__name__ if view else None,
            "path": request.path if request else None,
            "method": request.method if request else None,
        }
        
        # Include traceback in debug mode for server errors
        if status_code >= 500:
            error_response_data["debug"]["traceback"] = traceback.format_exc()
    
    return Response(error_response_data, status=status_code)

Step 1.2: Create Request ID Middleware (Optional but Recommended)

File: backend/igny8_core/api/middleware.py

Implementation:

"""
Request ID Middleware for Error Tracking
"""

import uuid
from django.utils.deprecation import MiddlewareMixin


class RequestIDMiddleware(MiddlewareMixin):
    """
    Middleware that adds a unique request ID to each request.
    
    This request ID is used for error tracking and correlation.
    """
    
    def process_request(self, request):
        """
        Generate and attach request ID to request.
        """
        # Check if request ID is already set (e.g., by reverse proxy)
        request_id = request.META.get('HTTP_X_REQUEST_ID')
        
        if not request_id:
            # Generate new request ID
            request_id = str(uuid.uuid4())
        
        # Attach to request
        request.request_id = request_id
        
        # Add to response headers
        # This will be done in process_response
    
    def process_response(self, request, response):
        """
        Add request ID to response headers.
        """
        request_id = getattr(request, 'request_id', None)
        if request_id:
            response['X-Request-ID'] = request_id
        
        return response

Register in settings:

MIDDLEWARE = [
    # ... other middleware ...
    'igny8_core.api.middleware.RequestIDMiddleware',
    # ... other middleware ...
]

Estimated Time: 4 hours (handler) + 2 hours (middleware) = 6 hours

---

Task 2: Register in Global DRF Settings

Goal

Register the custom exception handler in Django REST Framework settings.

Implementation Steps

Step 2.1: Update REST_FRAMEWORK Settings

File: backend/igny8_core/settings.py

Update REST_FRAMEWORK configuration:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'igny8_core.api.pagination.CustomPageNumberPagination',
    'PAGE_SIZE': 10,
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.SearchFilter',
        'rest_framework.filters.OrderingFilter',
    ],
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.AllowAny',  # Will be changed in Section 2
    ],
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'igny8_core.api.authentication.JWTAuthentication',
        'igny8_core.api.authentication.CSRFExemptSessionAuthentication',
        'rest_framework.authentication.BasicAuthentication',
    ],
    'EXCEPTION_HANDLER': 'igny8_core.api.exception_handlers.custom_exception_handler',  # Add this
}

Estimated Time: 1 hour

---

Task 3: Unify Validation Error Format

Goal

Ensure all validation errors (serializer and manual) use the unified error format.

Implementation Steps

Step 3.1: Audit Validation Error Usage

Action Items:

1. Search for manual validation in ViewSets
2. Identify places where error_response is not used
3. Document current validation error patterns
4. Create refactoring checklist

Files to Audit:

• All ViewSets with custom create, update, or @action methods
• Custom validation logic in serializers
• Manual validation in views

Estimated Time: 3 hours

Step 3.2: Unify Validation Error Format

DRF Serializer Errors: DRF automatically handles serializer validation errors and passes them to the exception handler. The exception handler (Task 1) will wrap them in the unified format.

Manual Validation Errors:

Before:

class KeywordViewSet(BaseTenantViewSet):
    @action(detail=False, methods=['post'])
    def bulk_delete(self, request):
        ids = request.data.get('ids', [])
        if not ids:
            return Response({
                "error": "No IDs provided"
            }, status=400)

After:

from igny8_core.api.response import error_response

class KeywordViewSet(BaseTenantViewSet):
    @action(detail=False, methods=['post'])
    def bulk_delete(self, request):
        ids = request.data.get('ids', [])
        if not ids:
            return error_response(
                error="No IDs provided",
                errors={"ids": ["This field is required"]},
                status_code=status.HTTP_400_BAD_REQUEST
            )

Serializer Validation Override (if needed):

File: backend/igny8_core/api/serializers.py (create if needed)

"""
Base Serializer with Unified Error Format
"""

from rest_framework import serializers
from rest_framework.exceptions import ValidationError


class BaseSerializer(serializers.Serializer):
    """
    Base serializer that ensures validation errors follow unified format.
    
    DRF's default validation error handling should work with the exception handler,
    but this can be used if custom validation error formatting is needed.
    """
    
    def validate(self, attrs):
        """
        Override to add custom validation logic.
        """
        return attrs
    
    def to_internal_value(self, data):
        """
        Override to customize validation error format if needed.
        """
        try:
            return super().to_internal_value(data)
        except ValidationError as e:
            # Validation errors are already in the correct format
            # The exception handler will wrap them
            raise


class BaseModelSerializer(serializers.ModelSerializer, BaseSerializer):
    """
    Base model serializer with unified error format support.
    """
    pass

Estimated Time: 4 hours

---

Task 4: Patch or Log Internal Server Errors

Goal

Implement comprehensive error logging for debugging and monitoring.

Implementation Steps

Step 4.1: Configure Error Logging

File: backend/igny8_core/settings.py

Add/Update LOGGING configuration:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
            'style': '{',
        },
        'detailed': {
            'format': '{levelname} {asctime} [{request_id}] {module} {path} {method} {status_code} {message}',
            'style': '{',
        },
    },
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'verbose',
        },
        'file': {
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': os.path.join(BASE_DIR, 'logs', 'django.log'),
            'maxBytes': 1024 * 1024 * 10,  # 10 MB
            'backupCount': 5,
            'formatter': 'detailed',
        },
        'error_file': {
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': os.path.join(BASE_DIR, 'logs', 'errors.log'),
            'maxBytes': 1024 * 1024 * 10,  # 10 MB
            'backupCount': 10,
            'formatter': 'detailed',
            'level': 'ERROR',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console', 'file'],
            'level': 'INFO',
            'propagate': False,
        },
        'igny8_core.api': {
            'handlers': ['console', 'file', 'error_file'],
            'level': 'INFO',
            'propagate': False,
        },
        'django.request': {
            'handlers': ['error_file'],
            'level': 'ERROR',
            'propagate': False,
        },
    },
    'root': {
        'handlers': ['console'],
        'level': 'INFO',
    },
}

Create logs directory:

mkdir -p logs
touch logs/django.log logs/errors.log

Estimated Time: 3 hours

Step 4.2: Add Request ID Tracking

Implementation:

• Request ID middleware (Task 1.2) already adds request IDs
• Exception handler (Task 1.1) already logs request IDs
• Ensure request ID is included in all error responses

Estimated Time: Already included in Task 1

Step 4.3: Integrate Error Tracking Service (Optional)

Sentry Integration Example:

Install Sentry:

pip install sentry-sdk

File: backend/igny8_core/settings.py

Add Sentry configuration:

import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.logging import LoggingIntegration

# Only enable Sentry in production
if not settings.DEBUG and os.getenv('SENTRY_DSN'):
    sentry_sdk.init(
        dsn=os.getenv('SENTRY_DSN'),
        integrations=[
            DjangoIntegration(),
            LoggingIntegration(level=logging.INFO, event_level=logging.ERROR),
        ],
        traces_sample_rate=0.1,  # 10% of transactions
        send_default_pii=True,  # Include user information
        environment=os.getenv('ENVIRONMENT', 'production'),
    )

Update Exception Handler to Send to Sentry:

File: backend/igny8_core/api/exception_handlers.py

Add Sentry integration:

def custom_exception_handler(exc, context):
    # ... existing code ...
    
    # Send to Sentry for server errors
    if status_code >= 500:
        try:
            import sentry_sdk
            with sentry_sdk.push_scope() as scope:
                scope.set_tag("request_id", request_id)
                scope.set_tag("status_code", status_code)
                scope.set_context("request", {
                    "path": request.path if request else None,
                    "method": request.method if request else None,
                    "view": view.__class__.__name__ if view else None,
                })
                sentry_sdk.capture_exception(exc)
        except ImportError:
            # Sentry not installed
            pass
    
    # ... rest of handler ...

Estimated Time: 4 hours (optional)

---

Task 5: Add Developer-Focused Debug Support

Goal

Add debug information to error responses in development mode for easier debugging.

Implementation Steps

Step 5.1: Enhance Exception Handler with Debug Mode

Implementation: The exception handler (Task 1.1) already includes debug information when settings.DEBUG is True. This includes:

• Exception type and message
• View class name
• Request path and method
• Full traceback for server errors

Additional Debug Features (Optional):

File: backend/igny8_core/api/exception_handlers.py

Add environment variable control:

def custom_exception_handler(exc, context):
    # ... existing code ...
    
    # Add debug information in development mode
    # Can be controlled by DEBUG setting or API_DEBUG environment variable
    api_debug = settings.DEBUG or os.getenv('API_DEBUG', 'False').lower() == 'true'
    
    if api_debug:
        error_response_data["debug"] = {
            "exception_type": type(exc).__name__,
            "exception_message": str(exc),
            "view": view.__class__.__name__ if view else None,
            "path": request.path if request else None,
            "method": request.method if request else None,
            "user": str(request.user) if request and hasattr(request, 'user') else None,
            "account": str(request.account) if request and hasattr(request, 'account') else None,
        }
        
        # Include traceback in debug mode for server errors
        if status_code >= 500:
            error_response_data["debug"]["traceback"] = traceback.format_exc()
        
        # Include request data (sanitized)
        if request:
            error_response_data["debug"]["request_data"] = {
                "GET": dict(request.GET),
                "POST": {k: str(v)[:100] for k, v in dict(request.POST).items()},  # Truncate long values
            }
    
    # ... rest of handler ...

Security Note:

• Only include debug information when DEBUG=True or API_DEBUG=True
• Never include sensitive data (passwords, tokens) in debug output
• Sanitize request data before including in debug response

Estimated Time: 3 hours

---

Task 6: Test Scenarios

Goal

Create comprehensive test scenarios to validate error handling works correctly.

Implementation Steps

Step 6.1: Create Test Scenarios

Test Cases:

1. Missing Required Field (400)

# Test: POST /api/v1/planner/keywords/ without required fields
# Expected: 400 Bad Request
{
  "success": false,
  "error": "keyword: This field is required",
  "errors": {
    "keyword": ["This field is required"],
    "site_id": ["This field is required"]
  },
  "request_id": "abc-123-def"
}

2. Invalid Token (401)

# Test: GET /api/v1/auth/me/ with invalid token
# Expected: 401 Unauthorized
{
  "success": false,
  "error": "Authentication required",
  "request_id": "abc-123-def"
}

3. Permission Denied (403)

# Test: DELETE /api/v1/auth/users/1/ as non-admin user
# Expected: 403 Forbidden
{
  "success": false,
  "error": "Permission denied",
  "request_id": "abc-123-def"
}

4. Not Found (404)

# Test: GET /api/v1/planner/keywords/99999/ (non-existent ID)
# Expected: 404 Not Found
{
  "success": false,
  "error": "Resource not found",
  "request_id": "abc-123-def"
}

5. Server Crash (500)

# Test: Endpoint that raises unhandled exception
# Expected: 500 Internal Server Error
{
  "success": false,
  "error": "Internal server error",
  "request_id": "abc-123-def",
  "debug": {  # Only in DEBUG mode
    "exception_type": "ValueError",
    "exception_message": "...",
    "traceback": "..."
  }
}

6. Manual Check Failure

# Test: Custom action with manual validation
# Expected: Custom error message
{
  "success": false,
  "error": "No IDs provided",
  "errors": {
    "ids": ["This field is required"]
  },
  "request_id": "abc-123-def"
}

7. Validation Error with Multiple Fields

# Test: POST with multiple validation errors
# Expected: All errors in errors dict
{
  "success": false,
  "error": "email: Invalid email format",
  "errors": {
    "email": ["Invalid email format"],
    "password": ["Password must be at least 8 characters"],
    "password_confirm": ["Passwords do not match"]
  },
  "request_id": "abc-123-def"
}

Step 6.2: Create Automated Tests

File: backend/igny8_core/api/tests/test_exception_handlers.py

Test Implementation:

"""
Tests for Exception Handler
"""

from django.test import TestCase
from rest_framework.test import APIClient
from rest_framework import status
from django.contrib.auth import get_user_model
from igny8_core.auth.models import Account

User = get_user_model()


class ExceptionHandlerTestCase(TestCase):
    def setUp(self):
        self.client = APIClient()
        self.account = Account.objects.create(name="Test Account")
        self.user = User.objects.create_user(
            email="test@example.com",
            password="testpass123",
            account=self.account
        )
    
    def test_validation_error_format(self):
        """Test that validation errors return unified format"""
        response = self.client.post('/api/v1/planner/keywords/', {})
        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
        data = response.json()
        self.assertFalse(data['success'])
        self.assertIn('error', data)
        self.assertIn('errors', data)
        self.assertIn('request_id', data)
    
    def test_unauthorized_error_format(self):
        """Test that 401 errors return unified format"""
        response = self.client.get('/api/v1/auth/me/')
        self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
        data = response.json()
        self.assertFalse(data['success'])
        self.assertEqual(data['error'], 'Authentication required')
    
    def test_not_found_error_format(self):
        """Test that 404 errors return unified format"""
        self.client.force_authenticate(user=self.user)
        response = self.client.get('/api/v1/planner/keywords/99999/')
        self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
        data = response.json()
        self.assertFalse(data['success'])
        self.assertEqual(data['error'], 'Resource not found')
    
    def test_request_id_present(self):
        """Test that request ID is included in error responses"""
        response = self.client.post('/api/v1/planner/keywords/', {})
        data = response.json()
        self.assertIn('request_id', data)
        self.assertIsNotNone(data['request_id'])
    
    def test_debug_info_in_development(self):
        """Test that debug info is included when DEBUG=True"""
        from django.conf import settings
        original_debug = settings.DEBUG
        settings.DEBUG = True
        
        try:
            # Trigger a server error
            response = self.client.get('/api/v1/nonexistent-endpoint/')
            if response.status_code >= 500:
                data = response.json()
                if 'debug' in data:
                    self.assertIn('exception_type', data['debug'])
        finally:
            settings.DEBUG = original_debug

Estimated Time: 4 hours

---

Task 7: Changelog Entry

Goal

Document the error handling standardization changes.

Implementation Steps

Step 7.1: Update CHANGELOG.md

File: CHANGELOG.md (or similar)

Add Entry:

## [Unreleased] - 2025-01-XX

### Changed
- **Error Handling**: Added centralized exception handler and standardized all API error formats
  - Created `custom_exception_handler` in `igny8_core/api/exception_handlers.py`
  - All exceptions now return unified error format: `{ success: false, error: "...", errors: {...}, request_id: "..." }`
  - Added request ID tracking for error correlation
  - Enhanced error logging with request context
  - Added debug information in development mode (exception type, traceback, request details)
  - Unified validation error format across all endpoints

### Added
- **Request ID Middleware**: Added `RequestIDMiddleware` for error tracking and correlation
- **Error Logging**: Configured comprehensive error logging to files and console
- **Debug Support**: Added debug information in error responses when `DEBUG=True`
  - Includes exception type, message, view, path, method
  - Includes full traceback for server errors (500+)
  - Includes sanitized request data

### Security
- Debug information only included when `DEBUG=True` or `API_DEBUG=True`
- Sensitive data (passwords, tokens) excluded from debug output
- Request data sanitized before inclusion in debug responses

### Affected Areas
- API Layer (`igny8_core/api/exception_handlers.py`, `igny8_core/api/middleware.py`)
- All API endpoints (error responses now unified)
- Error logging configuration (`settings.py`)

### Migration Guide
1. **Frontend Updates:**
   - Update error handling to check `success: false` field
   - Use `error` field for top-level error message
   - Use `errors` field for field-specific validation errors
   - Use `request_id` for error reporting and support

2. **Error Response Format:**
   ```json
   {
     "success": false,
     "error": "Top-level error message",
     "errors": {
       "field_name": ["Field-specific error messages"]
     },
     "request_id": "uuid-for-tracking",
     "debug": {  // Only in DEBUG mode
       "exception_type": "ValidationError",
       "traceback": "..."
     }
   }

3. Testing:
   • All error responses now follow unified format
   • Request IDs are included for error tracking
   • Debug information available in development mode

**Estimated Time:** 1 hour

---

## Testing Strategy

### Unit Tests

**File:** `backend/igny8_core/api/tests/test_exception_handlers.py`

**Test Cases:**
- Exception handler wraps DRF exceptions correctly
- Error message extraction from various exception types
- Request ID generation and inclusion
- Debug information inclusion in development mode
- Status code mapping to error messages
- Validation error format handling

### Integration Tests

**Test Cases:**
- End-to-end API calls that trigger various error scenarios
- Error response format validation
- Request ID propagation
- Error logging verification
- Debug information in development mode

### Manual Testing

**Checklist:**
- [ ] Validation errors return unified format
- [ ] Authentication errors return unified format
- [ ] Permission errors return unified format
- [ ] Not found errors return unified format
- [ ] Server errors return unified format
- [ ] Request ID is included in all error responses
- [ ] Error logging works correctly
- [ ] Debug information appears in development mode
- [ ] Debug information is excluded in production mode
- [ ] Sensitive data is not included in debug output

---

## Rollout Plan

### Phase 1: Foundation (Week 1)
- ✅ Task 1: Create centralized exception handler
- ✅ Task 2: Register exception handler in settings
- ✅ Unit tests for exception handler

### Phase 2: Logging & Debug (Week 2)
- ✅ Task 4: Configure error logging
- ✅ Task 5: Add debug support
- ✅ Task 1.2: Add request ID middleware (optional)

### Phase 3: Validation & Testing (Week 3)
- ✅ Task 3: Unify validation error format
- ✅ Task 6: Create test scenarios
- ✅ Integration testing

### Phase 4: Documentation & Release (Week 4)
- ✅ Task 7: Changelog entry
- ✅ Documentation updates
- ✅ Release to staging
- ✅ Production deployment

---

## Success Criteria

### Definition of Done

1. ✅ Centralized exception handler is created and registered
2. ✅ All exceptions return unified error format
3. ✅ Request ID tracking is implemented
4. ✅ Error logging is configured and working
5. ✅ Debug information is available in development mode
6. ✅ Validation errors use unified format
7. ✅ All test scenarios pass
8. ✅ Documentation is updated
9. ✅ Changelog entry is created
10. ✅ No sensitive data in debug output

### Metrics

- **Coverage:** 100% of exceptions use unified format
- **Logging:** All errors are logged with request context
- **Test Coverage:** >90% for exception handler
- **Debug Support:** Debug information available in development mode

---

## Risk Assessment

### Risks

1. **Debug Information Leakage:** Debug info might leak sensitive data
   - **Mitigation:** Only include debug info when `DEBUG=True`, sanitize request data

2. **Performance Impact:** Exception handling and logging may add overhead
   - **Mitigation:** Minimal overhead, use async logging if needed

3. **Breaking Changes:** Frontend may expect different error format
   - **Mitigation:** Document new format, provide migration guide

4. **Log File Growth:** Error logs may grow large
   - **Mitigation:** Use rotating file handlers, set up log rotation

### Rollback Plan

If issues arise:
1. Revert exception handler registration (use DRF default)
2. Keep response format utilities (non-breaking)
3. Disable debug mode if sensitive data is exposed
4. Document issues for future fixes

---

## Appendix

### Error Response Format Examples

#### Validation Error (400)
```json
{
  "success": false,
  "error": "email: Invalid email format",
  "errors": {
    "email": ["Invalid email format"],
    "password": ["Password must be at least 8 characters"]
  },
  "request_id": "550e8400-e29b-41d4-a716-446655440000"
}

Authentication Error (401)

{
  "success": false,
  "error": "Authentication required",
  "request_id": "550e8400-e29b-41d4-a716-446655440000"
}

Permission Error (403)

{
  "success": false,
  "error": "Permission denied",
  "request_id": "550e8400-e29b-41d4-a716-446655440000"
}

Not Found Error (404)

{
  "success": false,
  "error": "Resource not found",
  "request_id": "550e8400-e29b-41d4-a716-446655440000"
}

Server Error (500) - Production

{
  "success": false,
  "error": "Internal server error",
  "request_id": "550e8400-e29b-41d4-a716-446655440000"
}

Server Error (500) - Development

{
  "success": false,
  "error": "Internal server error",
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "debug": {
    "exception_type": "ValueError",
    "exception_message": "Invalid value",
    "view": "KeywordViewSet",
    "path": "/api/v1/planner/keywords/",
    "method": "POST",
    "traceback": "Traceback (most recent call last):\n  ..."
  }
}

---

Document Status: Implementation Plan
Last Updated: 2025-01-XX
Next Review: After Phase 1 completion