salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ecda7e0ef59fbdb471ccd13171fa63738a12cf81
igny8/unified-api/API-IMPLEMENTATION-PLAN-SECTION3.md

1208 lines
36 KiB
Markdown
Raw Blame History

# API Implementation Plan - Section 3: Implement Consistent Error Handling
**Date:** 2025-01-XX
**Status:** Planning
**Priority:** High
**Related Document:** `API-ENDPOINTS-ANALYSIS.md`, `API-IMPLEMENTATION-PLAN-SECTION1.md`
---
## Executive Summary
This document outlines the implementation plan for **Section 3: Implement Consistent Error Handling** across the IGNY8 API layer. The goal is to ensure all exceptions, validation errors, and system-level failures return uniform, predictable JSON responses with proper HTTP status codes and user-readable messages.
**Key Objectives:**
- Create centralized exception handler that wraps all errors in unified format
- Unify validation error format across all endpoints
- Implement proper logging for debugging and monitoring
- Add developer-focused debug support for development environments
- Ensure consistent error responses for all failure scenarios
**Note:** This section builds on Section 1 Task 4 (Exception Handlers) and extends it with logging, debugging, and validation error unification.
---
## Table of Contents
1. [Current State Analysis](#current-state-analysis)
2. [Implementation Tasks](#implementation-tasks)
3. [Task 1: Create Centralized Exception Handler](#task-1-create-centralized-exception-handler)
4. [Task 2: Register in Global DRF Settings](#task-2-register-in-global-drf-settings)
5. [Task 3: Unify Validation Error Format](#task-3-unify-validation-error-format)
6. [Task 4: Patch or Log Internal Server Errors](#task-4-patch-or-log-internal-server-errors)
7. [Task 5: Add Developer-Focused Debug Support](#task-5-add-developer-focused-debug-support)
8. [Task 6: Test Scenarios](#task-6-test-scenarios)
9. [Task 7: Changelog Entry](#task-7-changelog-entry)
10. [Testing Strategy](#testing-strategy)
11. [Rollout Plan](#rollout-plan)
12. [Success Criteria](#success-criteria)
---
## Current State Analysis
### Current Error Handling Issues
Based on `API-ENDPOINTS-ANALYSIS.md` and codebase analysis:
1. **Inconsistent Error Formats:**
- Some endpoints return `{ error: "..." }`
- Some return `{ success: false, message: "..." }`
- Some return DRF default format
- Validation errors may not be consistently formatted
2. **Exception Handling:**
- Section 1 Task 4 creates basic exception handler
- No logging for unhandled exceptions
- No request ID tracking for error correlation
- No debug information in development mode
3. **Validation Errors:**
- DRF serializer errors are handled automatically
- Manual validation errors may not follow unified format
- Field-specific errors may not be consistently structured
4. **Error Logging:**
- Some errors are logged, but not consistently
- No centralized error logging strategy
- No integration with error tracking services (Sentry, etc.)
5. **Frontend Error Handling:**
- Frontend has error parsing logic but may not handle all formats
- Error messages may not be user-friendly
- No consistent error display strategy
---
## Implementation Tasks
### Overview
| Task ID | Task Name | Priority | Estimated Effort | Dependencies |
|---------|-----------|----------|------------------|--------------|
| 1.1 | Create centralized exception handler | High | 4 hours | Section 1 Task 1 (response.py) |
| 1.2 | Enhance exception handler with logging | High | 3 hours | 1.1 |
| 2.1 | Register exception handler in settings | High | 1 hour | 1.1 |
| 3.1 | Audit validation error usage | Medium | 3 hours | None |
| 3.2 | Unify validation error format | High | 4 hours | 1.1 |
| 4.1 | Configure error logging | High | 3 hours | None |
| 4.2 | Add request ID tracking | Medium | 2 hours | 4.1 |
| 4.3 | Integrate error tracking service (optional) | Low | 4 hours | 4.1 |
| 5.1 | Add debug mode support | Low | 3 hours | 1.1 |
| 6.1 | Create test scenarios | High | 4 hours | All tasks |
| 7.1 | Create changelog entry | Low | 1 hour | All tasks |
**Total Estimated Effort:** ~32 hours
---
## Task 1: Create Centralized Exception Handler
### Goal
Create a comprehensive exception handler that wraps all exceptions in the unified error format with proper logging and debugging support.
### Implementation Steps
#### Step 1.1: Create Enhanced Exception Handler
**File:** `backend/igny8_core/api/exception_handlers.py`
**Note:** This extends Section 1 Task 4 with enhanced error handling, logging, and debug support.
**Implementation:**
```python
"""
Centralized Exception Handler for IGNY8 API
This module provides a comprehensive exception handler that:
- Wraps all exceptions in unified error format
- Logs errors for debugging and monitoring
- Provides debug information in development mode
- Tracks request IDs for error correlation
"""
import logging
import traceback
import uuid
from django.conf import settings
from rest_framework.views import exception_handler as drf_exception_handler
from rest_framework import status
from rest_framework.response import Response
from igny8_core.api.response import error_response
logger = logging.getLogger(__name__)
def get_request_id(request):
"""
Get or create request ID for error tracking.
Request ID can be set by middleware or generated here.
"""
# Check if request ID is already set (e.g., by middleware)
request_id = getattr(request, 'request_id', None)
if not request_id:
# Generate new request ID
request_id = str(uuid.uuid4())
request.request_id = request_id
return request_id
def extract_error_message(exc, response):
"""
Extract user-friendly error message from exception.
Args:
exc: The exception instance
response: DRF's exception handler response
Returns:
tuple: (error_message, errors_dict)
"""
error_message = "An error occurred"
errors = None
# Handle DRF exceptions with 'detail' attribute
if hasattr(exc, 'detail'):
if isinstance(exc.detail, dict):
# Validation errors - use as errors dict
errors = exc.detail
# Extract first error message as top-level error
if errors:
first_key = list(errors.keys())[0]
first_error = errors[first_key]
if isinstance(first_error, list) and first_error:
error_message = f"{first_key}: {first_error[0]}"
else:
error_message = f"{first_key}: {first_error}"
else:
error_message = "Validation failed"
elif isinstance(exc.detail, list):
# List of errors
error_message = exc.detail[0] if exc.detail else "An error occurred"
errors = {"non_field_errors": exc.detail}
else:
# String error message
error_message = str(exc.detail)
elif response and hasattr(response, 'data'):
# Try to extract from response data
if isinstance(response.data, dict):
# Check for common error message fields
error_message = (
response.data.get('error') or
response.data.get('message') or
response.data.get('detail') or
str(response.data)
)
errors = response.data if 'error' not in response.data else None
elif isinstance(response.data, list):
error_message = response.data[0] if response.data else "An error occurred"
errors = {"non_field_errors": response.data}
else:
error_message = str(response.data) if response.data else "An error occurred"
else:
# Generic exception
error_message = str(exc) if exc else "An error occurred"
return error_message, errors
def get_status_code_message(status_code):
"""
Get default error message for HTTP status code.
"""
status_messages = {
status.HTTP_400_BAD_REQUEST: "Bad request",
status.HTTP_401_UNAUTHORIZED: "Authentication required",
status.HTTP_403_FORBIDDEN: "Permission denied",
status.HTTP_404_NOT_FOUND: "Resource not found",
status.HTTP_405_METHOD_NOT_ALLOWED: "Method not allowed",
status.HTTP_409_CONFLICT: "Conflict",
status.HTTP_422_UNPROCESSABLE_ENTITY: "Unprocessable entity",
status.HTTP_429_TOO_MANY_REQUESTS: "Too many requests",
status.HTTP_500_INTERNAL_SERVER_ERROR: "Internal server error",
status.HTTP_502_BAD_GATEWAY: "Bad gateway",
status.HTTP_503_SERVICE_UNAVAILABLE: "Service unavailable",
}
return status_messages.get(status_code, "An error occurred")
def custom_exception_handler(exc, context):
"""
Centralized exception handler that wraps all exceptions in unified format.
This handler:
- Wraps all exceptions in unified error format
- Logs errors with request context
- Provides debug information in development mode
- Tracks request IDs for error correlation
Args:
exc: The exception instance
context: Dictionary containing request, view, and args
Returns:
Response: Error response in unified format, or None to use default
"""
# Get request from context
request = context.get('request')
view = context.get('view')
# Get request ID for tracking
request_id = None
if request:
request_id = get_request_id(request)
# Call DRF's default exception handler first
response = drf_exception_handler(exc, context)
# Determine status code
if response is not None:
status_code = response.status_code
else:
# Unhandled exception - default to 500
status_code = status.HTTP_500_INTERNAL_SERVER_ERROR
# Extract error message and details
error_message, errors = extract_error_message(exc, response)
# Use default message if error message is generic
if error_message == "An error occurred" or not error_message:
error_message = get_status_code_message(status_code)
# Log the error
log_level = logging.ERROR if status_code >= 500 else logging.WARNING
log_message = f"API Error [{status_code}]: {error_message}"
if request_id:
log_message += f" (Request ID: {request_id})"
# Include exception details in log
logger.log(
log_level,
log_message,
extra={
'request_id': request_id,
'status_code': status_code,
'exception_type': type(exc).__name__,
'view': view.__class__.__name__ if view else None,
'path': request.path if request else None,
'method': request.method if request else None,
},
exc_info=status_code >= 500, # Include traceback for server errors
)
# Build error response
error_response_data = {
"success": False,
"error": error_message,
}
# Add errors dict if present
if errors:
error_response_data["errors"] = errors
# Add request ID for error tracking
if request_id:
error_response_data["request_id"] = request_id
# Add debug information in development mode
if settings.DEBUG:
error_response_data["debug"] = {
"exception_type": type(exc).__name__,
"exception_message": str(exc),
"view": view.__class__.__name__ if view else None,
"path": request.path if request else None,
"method": request.method if request else None,
}
# Include traceback in debug mode for server errors
if status_code >= 500:
error_response_data["debug"]["traceback"] = traceback.format_exc()
return Response(error_response_data, status=status_code)
```
#### Step 1.2: Create Request ID Middleware (Optional but Recommended)
**File:** `backend/igny8_core/api/middleware.py`
**Implementation:**
```python
"""
Request ID Middleware for Error Tracking
"""
import uuid
from django.utils.deprecation import MiddlewareMixin
class RequestIDMiddleware(MiddlewareMixin):
"""
Middleware that adds a unique request ID to each request.
This request ID is used for error tracking and correlation.
"""
def process_request(self, request):
"""
Generate and attach request ID to request.
"""
# Check if request ID is already set (e.g., by reverse proxy)
request_id = request.META.get('HTTP_X_REQUEST_ID')
if not request_id:
# Generate new request ID
request_id = str(uuid.uuid4())
# Attach to request
request.request_id = request_id
# Add to response headers
# This will be done in process_response
def process_response(self, request, response):
"""
Add request ID to response headers.
"""
request_id = getattr(request, 'request_id', None)
if request_id:
response['X-Request-ID'] = request_id
return response
```
**Register in settings:**
```python
MIDDLEWARE = [
# ... other middleware ...
'igny8_core.api.middleware.RequestIDMiddleware',
# ... other middleware ...
]
```
**Estimated Time:** 4 hours (handler) + 2 hours (middleware) = 6 hours
---
## Task 2: Register in Global DRF Settings
### Goal
Register the custom exception handler in Django REST Framework settings.
### Implementation Steps
#### Step 2.1: Update REST_FRAMEWORK Settings
**File:** `backend/igny8_core/settings.py`
**Update REST_FRAMEWORK configuration:**
```python
REST_FRAMEWORK = {
'DEFAULT_PAGINATION_CLASS': 'igny8_core.api.pagination.CustomPageNumberPagination',
'PAGE_SIZE': 10,
'DEFAULT_FILTER_BACKENDS': [
'django_filters.rest_framework.DjangoFilterBackend',
'rest_framework.filters.SearchFilter',
'rest_framework.filters.OrderingFilter',
],
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.AllowAny', # Will be changed in Section 2
],
'DEFAULT_AUTHENTICATION_CLASSES': [
'igny8_core.api.authentication.JWTAuthentication',
'igny8_core.api.authentication.CSRFExemptSessionAuthentication',
'rest_framework.authentication.BasicAuthentication',
],
'EXCEPTION_HANDLER': 'igny8_core.api.exception_handlers.custom_exception_handler', # Add this
}
```
**Estimated Time:** 1 hour
---
## Task 3: Unify Validation Error Format
### Goal
Ensure all validation errors (serializer and manual) use the unified error format.
### Implementation Steps
#### Step 3.1: Audit Validation Error Usage
**Action Items:**
1. Search for manual validation in ViewSets
2. Identify places where `error_response` is not used
3. Document current validation error patterns
4. Create refactoring checklist
**Files to Audit:**
- All ViewSets with custom `create`, `update`, or `@action` methods
- Custom validation logic in serializers
- Manual validation in views
**Estimated Time:** 3 hours
#### Step 3.2: Unify Validation Error Format
**DRF Serializer Errors:**
DRF automatically handles serializer validation errors and passes them to the exception handler. The exception handler (Task 1) will wrap them in the unified format.
**Manual Validation Errors:**
**Before:**
```python
class KeywordViewSet(BaseTenantViewSet):
@action(detail=False, methods=['post'])
def bulk_delete(self, request):
ids = request.data.get('ids', [])
if not ids:
return Response({
"error": "No IDs provided"
}, status=400)
```
**After:**
```python
from igny8_core.api.response import error_response
class KeywordViewSet(BaseTenantViewSet):
@action(detail=False, methods=['post'])
def bulk_delete(self, request):
ids = request.data.get('ids', [])
if not ids:
return error_response(
error="No IDs provided",
errors={"ids": ["This field is required"]},
status_code=status.HTTP_400_BAD_REQUEST
)
```
**Serializer Validation Override (if needed):**
**File:** `backend/igny8_core/api/serializers.py` (create if needed)
```python
"""
Base Serializer with Unified Error Format
"""
from rest_framework import serializers
from rest_framework.exceptions import ValidationError
class BaseSerializer(serializers.Serializer):
"""
Base serializer that ensures validation errors follow unified format.
DRF's default validation error handling should work with the exception handler,
but this can be used if custom validation error formatting is needed.
"""
def validate(self, attrs):
"""
Override to add custom validation logic.
"""
return attrs
def to_internal_value(self, data):
"""
Override to customize validation error format if needed.
"""
try:
return super().to_internal_value(data)
except ValidationError as e:
# Validation errors are already in the correct format
# The exception handler will wrap them
raise
class BaseModelSerializer(serializers.ModelSerializer, BaseSerializer):
"""
Base model serializer with unified error format support.
"""
pass
```
**Estimated Time:** 4 hours
---
## Task 4: Patch or Log Internal Server Errors
### Goal
Implement comprehensive error logging for debugging and monitoring.
### Implementation Steps
#### Step 4.1: Configure Error Logging
**File:** `backend/igny8_core/settings.py`
**Add/Update LOGGING configuration:**
```python
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
'style': '{',
},
'detailed': {
'format': '{levelname} {asctime} [{request_id}] {module} {path} {method} {status_code} {message}',
'style': '{',
},
},
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse',
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'verbose',
},
'file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'logs', 'django.log'),
'maxBytes': 1024 * 1024 * 10, # 10 MB
'backupCount': 5,
'formatter': 'detailed',
},
'error_file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'logs', 'errors.log'),
'maxBytes': 1024 * 1024 * 10, # 10 MB
'backupCount': 10,
'formatter': 'detailed',
'level': 'ERROR',
},
},
'loggers': {
'django': {
'handlers': ['console', 'file'],
'level': 'INFO',
'propagate': False,
},
'igny8_core.api': {
'handlers': ['console', 'file', 'error_file'],
'level': 'INFO',
'propagate': False,
},
'django.request': {
'handlers': ['error_file'],
'level': 'ERROR',
'propagate': False,
},
},
'root': {
'handlers': ['console'],
'level': 'INFO',
},
}
```
**Create logs directory:**
```bash
mkdir -p logs
touch logs/django.log logs/errors.log
```
**Estimated Time:** 3 hours
#### Step 4.2: Add Request ID Tracking
**Implementation:**
- Request ID middleware (Task 1.2) already adds request IDs
- Exception handler (Task 1.1) already logs request IDs
- Ensure request ID is included in all error responses
**Estimated Time:** Already included in Task 1
#### Step 4.3: Integrate Error Tracking Service (Optional)
**Sentry Integration Example:**
**Install Sentry:**
```bash
pip install sentry-sdk
```
**File:** `backend/igny8_core/settings.py`
**Add Sentry configuration:**
```python
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.logging import LoggingIntegration
# Only enable Sentry in production
if not settings.DEBUG and os.getenv('SENTRY_DSN'):
sentry_sdk.init(
dsn=os.getenv('SENTRY_DSN'),
integrations=[
DjangoIntegration(),
LoggingIntegration(level=logging.INFO, event_level=logging.ERROR),
],
traces_sample_rate=0.1, # 10% of transactions
send_default_pii=True, # Include user information
environment=os.getenv('ENVIRONMENT', 'production'),
)
```
**Update Exception Handler to Send to Sentry:**
**File:** `backend/igny8_core/api/exception_handlers.py`
**Add Sentry integration:**
```python
def custom_exception_handler(exc, context):
# ... existing code ...
# Send to Sentry for server errors
if status_code >= 500:
try:
import sentry_sdk
with sentry_sdk.push_scope() as scope:
scope.set_tag("request_id", request_id)
scope.set_tag("status_code", status_code)
scope.set_context("request", {
"path": request.path if request else None,
"method": request.method if request else None,
"view": view.__class__.__name__ if view else None,
})
sentry_sdk.capture_exception(exc)
except ImportError:
# Sentry not installed
pass
# ... rest of handler ...
```
**Estimated Time:** 4 hours (optional)
---
## Task 5: Add Developer-Focused Debug Support
### Goal
Add debug information to error responses in development mode for easier debugging.
### Implementation Steps
#### Step 5.1: Enhance Exception Handler with Debug Mode
**Implementation:**
The exception handler (Task 1.1) already includes debug information when `settings.DEBUG` is `True`. This includes:
- Exception type and message
- View class name
- Request path and method
- Full traceback for server errors
**Additional Debug Features (Optional):**
**File:** `backend/igny8_core/api/exception_handlers.py`
**Add environment variable control:**
```python
def custom_exception_handler(exc, context):
# ... existing code ...
# Add debug information in development mode
# Can be controlled by DEBUG setting or API_DEBUG environment variable
api_debug = settings.DEBUG or os.getenv('API_DEBUG', 'False').lower() == 'true'
if api_debug:
error_response_data["debug"] = {
"exception_type": type(exc).__name__,
"exception_message": str(exc),
"view": view.__class__.__name__ if view else None,
"path": request.path if request else None,
"method": request.method if request else None,
"user": str(request.user) if request and hasattr(request, 'user') else None,
"account": str(request.account) if request and hasattr(request, 'account') else None,
}
# Include traceback in debug mode for server errors
if status_code >= 500:
error_response_data["debug"]["traceback"] = traceback.format_exc()
# Include request data (sanitized)
if request:
error_response_data["debug"]["request_data"] = {
"GET": dict(request.GET),
"POST": {k: str(v)[:100] for k, v in dict(request.POST).items()}, # Truncate long values
}
# ... rest of handler ...
```
**Security Note:**
- Only include debug information when `DEBUG=True` or `API_DEBUG=True`
- Never include sensitive data (passwords, tokens) in debug output
- Sanitize request data before including in debug response
**Estimated Time:** 3 hours
---
## Task 6: Test Scenarios
### Goal
Create comprehensive test scenarios to validate error handling works correctly.
### Implementation Steps
#### Step 6.1: Create Test Scenarios
**Test Cases:**
**1. Missing Required Field (400)**
```python
# Test: POST /api/v1/planner/keywords/ without required fields
# Expected: 400 Bad Request
{
"success": false,
"error": "keyword: This field is required",
"errors": {
"keyword": ["This field is required"],
"site_id": ["This field is required"]
},
"request_id": "abc-123-def"
}
```
**2. Invalid Token (401)**
```python
# Test: GET /api/v1/auth/me/ with invalid token
# Expected: 401 Unauthorized
{
"success": false,
"error": "Authentication required",
"request_id": "abc-123-def"
}
```
**3. Permission Denied (403)**
```python
# Test: DELETE /api/v1/auth/users/1/ as non-admin user
# Expected: 403 Forbidden
{
"success": false,
"error": "Permission denied",
"request_id": "abc-123-def"
}
```
**4. Not Found (404)**
```python
# Test: GET /api/v1/planner/keywords/99999/ (non-existent ID)
# Expected: 404 Not Found
{
"success": false,
"error": "Resource not found",
"request_id": "abc-123-def"
}
```
**5. Server Crash (500)**
```python
# Test: Endpoint that raises unhandled exception
# Expected: 500 Internal Server Error
{
"success": false,
"error": "Internal server error",
"request_id": "abc-123-def",
"debug": { # Only in DEBUG mode
"exception_type": "ValueError",
"exception_message": "...",
"traceback": "..."
}
}
```
**6. Manual Check Failure**
```python
# Test: Custom action with manual validation
# Expected: Custom error message
{
"success": false,
"error": "No IDs provided",
"errors": {
"ids": ["This field is required"]
},
"request_id": "abc-123-def"
}
```
**7. Validation Error with Multiple Fields**
```python
# Test: POST with multiple validation errors
# Expected: All errors in errors dict
{
"success": false,
"error": "email: Invalid email format",
"errors": {
"email": ["Invalid email format"],
"password": ["Password must be at least 8 characters"],
"password_confirm": ["Passwords do not match"]
},
"request_id": "abc-123-def"
}
```
#### Step 6.2: Create Automated Tests
**File:** `backend/igny8_core/api/tests/test_exception_handlers.py`
**Test Implementation:**
```python
"""
Tests for Exception Handler
"""
from django.test import TestCase
from rest_framework.test import APIClient
from rest_framework import status
from django.contrib.auth import get_user_model
from igny8_core.auth.models import Account
User = get_user_model()
class ExceptionHandlerTestCase(TestCase):
def setUp(self):
self.client = APIClient()
self.account = Account.objects.create(name="Test Account")
self.user = User.objects.create_user(
email="test@example.com",
password="testpass123",
account=self.account
)
def test_validation_error_format(self):
"""Test that validation errors return unified format"""
response = self.client.post('/api/v1/planner/keywords/', {})
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
data = response.json()
self.assertFalse(data['success'])
self.assertIn('error', data)
self.assertIn('errors', data)
self.assertIn('request_id', data)
def test_unauthorized_error_format(self):
"""Test that 401 errors return unified format"""
response = self.client.get('/api/v1/auth/me/')
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
data = response.json()
self.assertFalse(data['success'])
self.assertEqual(data['error'], 'Authentication required')
def test_not_found_error_format(self):
"""Test that 404 errors return unified format"""
self.client.force_authenticate(user=self.user)
response = self.client.get('/api/v1/planner/keywords/99999/')
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
data = response.json()
self.assertFalse(data['success'])
self.assertEqual(data['error'], 'Resource not found')
def test_request_id_present(self):
"""Test that request ID is included in error responses"""
response = self.client.post('/api/v1/planner/keywords/', {})
data = response.json()
self.assertIn('request_id', data)
self.assertIsNotNone(data['request_id'])
def test_debug_info_in_development(self):
"""Test that debug info is included when DEBUG=True"""
from django.conf import settings
original_debug = settings.DEBUG
settings.DEBUG = True
try:
# Trigger a server error
response = self.client.get('/api/v1/nonexistent-endpoint/')
if response.status_code >= 500:
data = response.json()
if 'debug' in data:
self.assertIn('exception_type', data['debug'])
finally:
settings.DEBUG = original_debug
```
**Estimated Time:** 4 hours
---
## Task 7: Changelog Entry
### Goal
Document the error handling standardization changes.
### Implementation Steps
#### Step 7.1: Update CHANGELOG.md
**File:** `CHANGELOG.md` (or similar)
**Add Entry:**
```markdown
## [Unreleased] - 2025-01-XX
### Changed
- **Error Handling**: Added centralized exception handler and standardized all API error formats
- Created `custom_exception_handler` in `igny8_core/api/exception_handlers.py`
- All exceptions now return unified error format: `{ success: false, error: "...", errors: {...}, request_id: "..." }`
- Added request ID tracking for error correlation
- Enhanced error logging with request context
- Added debug information in development mode (exception type, traceback, request details)
- Unified validation error format across all endpoints
### Added
- **Request ID Middleware**: Added `RequestIDMiddleware` for error tracking and correlation
- **Error Logging**: Configured comprehensive error logging to files and console
- **Debug Support**: Added debug information in error responses when `DEBUG=True`
- Includes exception type, message, view, path, method
- Includes full traceback for server errors (500+)
- Includes sanitized request data
### Security
- Debug information only included when `DEBUG=True` or `API_DEBUG=True`
- Sensitive data (passwords, tokens) excluded from debug output
- Request data sanitized before inclusion in debug responses
### Affected Areas
- API Layer (`igny8_core/api/exception_handlers.py`, `igny8_core/api/middleware.py`)
- All API endpoints (error responses now unified)
- Error logging configuration (`settings.py`)
### Migration Guide
1. **Frontend Updates:**
- Update error handling to check `success: false` field
- Use `error` field for top-level error message
- Use `errors` field for field-specific validation errors
- Use `request_id` for error reporting and support
2. **Error Response Format:**
```json
{
"success": false,
"error": "Top-level error message",
"errors": {
"field_name": ["Field-specific error messages"]
},
"request_id": "uuid-for-tracking",
"debug": { // Only in DEBUG mode
"exception_type": "ValidationError",
"traceback": "..."
}
}
```
3. **Testing:**
- All error responses now follow unified format
- Request IDs are included for error tracking
- Debug information available in development mode
```
**Estimated Time:** 1 hour
---
## Testing Strategy
### Unit Tests
**File:** `backend/igny8_core/api/tests/test_exception_handlers.py`
**Test Cases:**
- Exception handler wraps DRF exceptions correctly
- Error message extraction from various exception types
- Request ID generation and inclusion
- Debug information inclusion in development mode
- Status code mapping to error messages
- Validation error format handling
### Integration Tests
**Test Cases:**
- End-to-end API calls that trigger various error scenarios
- Error response format validation
- Request ID propagation
- Error logging verification
- Debug information in development mode
### Manual Testing
**Checklist:**
- [ ] Validation errors return unified format
- [ ] Authentication errors return unified format
- [ ] Permission errors return unified format
- [ ] Not found errors return unified format
- [ ] Server errors return unified format
- [ ] Request ID is included in all error responses
- [ ] Error logging works correctly
- [ ] Debug information appears in development mode
- [ ] Debug information is excluded in production mode
- [ ] Sensitive data is not included in debug output
---
## Rollout Plan
### Phase 1: Foundation (Week 1)
- ✅ Task 1: Create centralized exception handler
- ✅ Task 2: Register exception handler in settings
- ✅ Unit tests for exception handler
### Phase 2: Logging & Debug (Week 2)
- ✅ Task 4: Configure error logging
- ✅ Task 5: Add debug support
- ✅ Task 1.2: Add request ID middleware (optional)
### Phase 3: Validation & Testing (Week 3)
- ✅ Task 3: Unify validation error format
- ✅ Task 6: Create test scenarios
- ✅ Integration testing
### Phase 4: Documentation & Release (Week 4)
- ✅ Task 7: Changelog entry
- ✅ Documentation updates
- ✅ Release to staging
- ✅ Production deployment
---
## Success Criteria
### Definition of Done
1. ✅ Centralized exception handler is created and registered
2. ✅ All exceptions return unified error format
3. ✅ Request ID tracking is implemented
4. ✅ Error logging is configured and working
5. ✅ Debug information is available in development mode
6. ✅ Validation errors use unified format
7. ✅ All test scenarios pass
8. ✅ Documentation is updated
9. ✅ Changelog entry is created
10. ✅ No sensitive data in debug output
### Metrics
- **Coverage:** 100% of exceptions use unified format
- **Logging:** All errors are logged with request context
- **Test Coverage:** >90% for exception handler
- **Debug Support:** Debug information available in development mode
---
## Risk Assessment
### Risks
1. **Debug Information Leakage:** Debug info might leak sensitive data
- **Mitigation:** Only include debug info when `DEBUG=True`, sanitize request data
2. **Performance Impact:** Exception handling and logging may add overhead
- **Mitigation:** Minimal overhead, use async logging if needed
3. **Breaking Changes:** Frontend may expect different error format
- **Mitigation:** Document new format, provide migration guide
4. **Log File Growth:** Error logs may grow large
- **Mitigation:** Use rotating file handlers, set up log rotation
### Rollback Plan
If issues arise:
1. Revert exception handler registration (use DRF default)
2. Keep response format utilities (non-breaking)
3. Disable debug mode if sensitive data is exposed
4. Document issues for future fixes
---
## Appendix
### Error Response Format Examples
#### Validation Error (400)
```json
{
"success": false,
"error": "email: Invalid email format",
"errors": {
"email": ["Invalid email format"],
"password": ["Password must be at least 8 characters"]
},
"request_id": "550e8400-e29b-41d4-a716-446655440000"
}
```
#### Authentication Error (401)
```json
{
"success": false,
"error": "Authentication required",
"request_id": "550e8400-e29b-41d4-a716-446655440000"
}
```
#### Permission Error (403)
```json
{
"success": false,
"error": "Permission denied",
"request_id": "550e8400-e29b-41d4-a716-446655440000"
}
```
#### Not Found Error (404)
```json
{
"success": false,
"error": "Resource not found",
"request_id": "550e8400-e29b-41d4-a716-446655440000"
}
```
#### Server Error (500) - Production
```json
{
"success": false,
"error": "Internal server error",
"request_id": "550e8400-e29b-41d4-a716-446655440000"
}
```
#### Server Error (500) - Development
```json
{
"success": false,
"error": "Internal server error",
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"debug": {
"exception_type": "ValueError",
"exception_message": "Invalid value",
"view": "KeywordViewSet",
"path": "/api/v1/planner/keywords/",
"method": "POST",
"traceback": "Traceback (most recent call last):\n ..."
}
}
```
---
**Document Status:** Implementation Plan
**Last Updated:** 2025-01-XX
**Next Review:** After Phase 1 completion
Reference in New Issue View Git Blame Copy Permalink