# API Implementation Plan - Section 1: Unified Response Format

**Date:** 2025-01-XX  
**Status:** Planning  
**Priority:** High  
**Related Document:** `API-ENDPOINTS-ANALYSIS.md`

---

## Executive Summary

This document outlines the implementation plan for **Section 1: Define a Unified Response Format** across the IGNY8 API layer. The goal is to ensure every API endpoint in all modules responds with a consistent JSON structure, improving API predictability, frontend integration, and developer experience.

**Target Format:**
```json
{
  "success": true | false,
  "data": {...} | [...],
  "message": "Human-readable success/failure message",
  "error": "Top-level error message (if any)",
  "errors": { "field": ["Error list"] } (optional, for validation failures)
}
```

---

## Table of Contents

1. [Current State Analysis](#current-state-analysis)
2. [Implementation Tasks](#implementation-tasks)
3. [Task 1: Define Global Response Wrapper Functions](#task-1-define-global-response-wrapper-functions)
4. [Task 2: Refactor All API Views and ViewSets](#task-2-refactor-all-api-views-and-viewsets)
5. [Task 3: Patch DRF Auto-generated Responses (Optional)](#task-3-patch-drf-auto-generated-responses-optional)
6. [Task 4: Update Exception Handlers](#task-4-update-exception-handlers)
7. [Task 5: Validation via Postman/Frontend Logs](#task-5-validation-via-postmanfrontend-logs)
8. [Task 6: Documentation Update](#task-6-documentation-update)
9. [Task 7: Changelog Entry](#task-7-changelog-entry)
10. [Testing Strategy](#testing-strategy)
11. [Rollout Plan](#rollout-plan)
12. [Success Criteria](#success-criteria)

---

## Current State Analysis

### Current Response Format Issues

Based on `API-ENDPOINTS-ANALYSIS.md`, the following inconsistencies exist:

1. **Mixed Response Formats:**
   - Some endpoints return `{ success: true, data: ... }`
   - Some endpoints return DRF standard format `{ count, next, previous, results: [...] }`
   - Some endpoints return custom formats
   - Error responses vary: `{ error: "..." }` vs `{ success: false, message: "..." }`

2. **Affected Modules:**
   - **Auth Module**: 15+ endpoints (register, login, user management, etc.)
   - **Planner Module**: 20+ endpoints (keywords, clusters, ideas)
   - **Writer Module**: 20+ endpoints (tasks, content, images)
   - **System Module**: 30+ endpoints (settings, prompts, integrations)
   - **Billing Module**: 10+ endpoints (credits, transactions, usage)

3. **Base Classes to Update:**
   - `AccountModelViewSet` - Used by many ViewSets
   - `SiteSectorModelViewSet` - Used by Planner/Writer modules
   - Custom ViewSets with overridden `create`, `update`, `destroy` methods

---

## Implementation Tasks

### Overview

| Task ID | Task Name | Priority | Estimated Effort | Dependencies |
|---------|-----------|----------|------------------|--------------|
| 1.1 | Create response wrapper functions | High | 2 hours | None |
| 1.2 | Update exception handlers | High | 3 hours | 1.1 |
| 2.1 | Audit all ViewSets | Medium | 4 hours | None |
| 2.2 | Refactor Auth module ViewSets | High | 6 hours | 1.1, 1.2 |
| 2.3 | Refactor Planner module ViewSets | High | 8 hours | 1.1, 1.2 |
| 2.4 | Refactor Writer module ViewSets | High | 8 hours | 1.1, 1.2 |
| 2.5 | Refactor System module ViewSets | High | 10 hours | 1.1, 1.2 |
| 2.6 | Refactor Billing module ViewSets | High | 4 hours | 1.1, 1.2 |
| 3.1 | Patch DRF auto-generated responses (optional) | Low | 4 hours | 1.1 |
| 5.1 | Manual testing | High | 8 hours | 2.2-2.6 |
| 6.1 | Update documentation | Medium | 4 hours | 5.1 |

**Total Estimated Effort:** ~61 hours

---

## Task 1: Define Global Response Wrapper Functions

### Goal
Create reusable helper functions that standardize all API responses.

### Implementation Steps

#### Step 1.1: Create Response Utility Module

**File:** `backend/igny8_core/api/response.py`

**Implementation:**
```python
"""
Unified API Response Format Utilities

This module provides helper functions to ensure all API endpoints
return a consistent response format.
"""

from rest_framework.response import Response
from rest_framework import status


def success_response(data=None, message=None, status_code=status.HTTP_200_OK):
    """
    Create a standardized success response.
    
    Args:
        data: Response data (dict, list, or any serializable object)
        message: Optional human-readable success message
        status_code: HTTP status code (default: 200)
    
    Returns:
        Response: DRF Response object with unified format
    
    Example:
        return success_response(
            data={"id": 1, "name": "Example"},
            message="Resource created successfully"
        )
    """
    response_data = {
        "success": True,
        "data": data,
    }
    
    if message:
        response_data["message"] = message
    
    return Response(response_data, status=status_code)


def error_response(
    error,
    errors=None,
    status_code=status.HTTP_400_BAD_REQUEST,
    message=None
):
    """
    Create a standardized error response.
    
    Args:
        error: Top-level error message (string)
        errors: Optional field-specific validation errors (dict)
        status_code: HTTP status code (default: 400)
        message: Optional additional message (deprecated, use error)
    
    Returns:
        Response: DRF Response object with unified format
    
    Example:
        return error_response(
            error="Validation failed",
            errors={"email": ["Invalid email format"]},
            status_code=status.HTTP_400_BAD_REQUEST
        )
    """
    response_data = {
        "success": False,
        "error": error,
    }
    
    if errors:
        response_data["errors"] = errors
    
    # Backward compatibility: if message is provided, use it as error
    if message and not error:
        response_data["error"] = message
    
    return Response(response_data, status=status_code)


def paginated_response(paginated_data, message=None):
    """
    Create a standardized paginated response.
    
    This wraps DRF's pagination response to include success flag
    and optional message while preserving pagination metadata.
    
    Args:
        paginated_data: DRF paginated response data (dict with count, next, previous, results)
        message: Optional human-readable message
    
    Returns:
        Response: DRF Response object with unified format
    
    Example:
        paginator = CustomPageNumberPagination()
        page = paginator.paginate_queryset(queryset, request)
        serializer = MySerializer(page, many=True)
        paginated_data = paginator.get_paginated_response(serializer.data).data
        return paginated_response(paginated_data, message="Keywords retrieved successfully")
    """
    response_data = {
        "success": True,
        **paginated_data  # Unpack count, next, previous, results
    }
    
    if message:
        response_data["message"] = message
    
    return Response(response_data)
```

#### Step 1.2: Update `__init__.py` for Easy Import

**File:** `backend/igny8_core/api/__init__.py`

**Add:**
```python
from .response import success_response, error_response, paginated_response

__all__ = ['success_response', 'error_response', 'paginated_response']
```

#### Step 1.3: Create Unit Tests

**File:** `backend/igny8_core/api/tests/test_response.py`

**Test Cases:**
- Test `success_response` with data only
- Test `success_response` with data and message
- Test `success_response` with custom status code (201, 204)
- Test `error_response` with error only
- Test `error_response` with error and errors dict
- Test `error_response` with custom status codes (400, 403, 404, 500)
- Test `paginated_response` with standard pagination data
- Test `paginated_response` with message

**Estimated Time:** 2 hours

---

## Task 2: Refactor All API Views and ViewSets

### Goal
Update all ViewSets to use the new response wrapper functions.

### Implementation Strategy

#### Step 2.1: Audit All ViewSets

**Action Items:**
1. List all ViewSets in each module
2. Identify custom `create`, `update`, `destroy`, `list`, `retrieve` methods
3. Identify custom action methods (`@action` decorator)
4. Document current response formats
5. Create refactoring checklist

**Files to Audit:**

**Auth Module:**
- `backend/auth/api/views.py` - UsersViewSet, AccountsViewSet, SiteViewSet, etc.
- `backend/auth/api/viewsets.py` - Custom authentication views

**Planner Module:**
- `backend/planner/api/viewsets.py` - KeywordViewSet, ClusterViewSet, ContentIdeasViewSet

**Writer Module:**
- `backend/writer/api/viewsets.py` - TasksViewSet, ContentViewSet, ImagesViewSet

**System Module:**
- `backend/system/api/viewsets.py` - AIPromptViewSet, AuthorProfileViewSet, StrategyViewSet
- `backend/system/api/viewsets.py` - IntegrationSettingsViewSet, SystemSettingsViewSet, etc.

**Billing Module:**
- `backend/billing/api/viewsets.py` - CreditBalanceViewSet, CreditUsageViewSet, CreditTransactionViewSet

**Estimated Time:** 4 hours

#### Step 2.2: Refactor Auth Module ViewSets

**Priority Endpoints:**
1. `POST /api/v1/auth/register/` - Registration endpoint
2. `POST /api/v1/auth/login/` - Login endpoint
3. `POST /api/v1/auth/change-password/` - Password change
4. `GET /api/v1/auth/me/` - Current user info
5. All ViewSet CRUD operations
6. Custom actions (invite, activate, etc.)

**Example Refactoring:**

**Before:**
```python
class RegisterView(APIView):
    def post(self, request):
        # ... validation logic ...
        return Response({
            "success": True,
            "message": "Registration successful",
            "user": user_data
        }, status=201)
```

**After:**
```python
from igny8_core.api.response import success_response

class RegisterView(APIView):
    def post(self, request):
        # ... validation logic ...
        return success_response(
            data={"user": user_data},
            message="Registration successful",
            status_code=status.HTTP_201_CREATED
        )
```

**Estimated Time:** 6 hours

#### Step 2.3: Refactor Planner Module ViewSets

**Priority Endpoints:**
1. KeywordViewSet - All CRUD + custom actions (bulk_delete, auto_cluster, etc.)
2. ClusterViewSet - All CRUD + custom actions (auto_generate_ideas)
3. ContentIdeasViewSet - All CRUD + custom actions (bulk_queue_to_writer)

**Example Refactoring:**

**Before:**
```python
class KeywordViewSet(SiteSectorModelViewSet):
    @action(detail=False, methods=['post'])
    def bulk_delete(self, request):
        ids = request.data.get('ids', [])
        deleted_count = Keyword.objects.filter(id__in=ids).delete()[0]
        return Response({"deleted_count": deleted_count})
```

**After:**
```python
from igny8_core.api.response import success_response, error_response

class KeywordViewSet(SiteSectorModelViewSet):
    @action(detail=False, methods=['post'])
    def bulk_delete(self, request):
        ids = request.data.get('ids', [])
        if not ids:
            return error_response(
                error="No IDs provided",
                status_code=status.HTTP_400_BAD_REQUEST
            )
        deleted_count = Keyword.objects.filter(id__in=ids).delete()[0]
        return success_response(
            data={"deleted_count": deleted_count},
            message=f"Successfully deleted {deleted_count} keywords"
        )
```

**Estimated Time:** 8 hours

#### Step 2.4: Refactor Writer Module ViewSets

**Priority Endpoints:**
1. TasksViewSet - All CRUD + custom actions (auto_generate_content, bulk_update)
2. ContentViewSet - All CRUD + custom actions (generate_image_prompts)
3. ImagesViewSet - All CRUD + custom actions (generate_images, bulk_update)

**Estimated Time:** 8 hours

#### Step 2.5: Refactor System Module ViewSets

**Priority Endpoints:**
1. AIPromptViewSet - All CRUD + custom actions (save, reset, by_type)
2. IntegrationSettingsViewSet - Custom URL patterns (save, test, generate)
3. All Settings ViewSets (SystemSettings, AccountSettings, UserSettings, etc.)

**Special Considerations:**
- IntegrationSettingsViewSet uses custom URL patterns (not standard ViewSet routes)
- Task progress endpoint returns Celery task status (may need special handling)

**Estimated Time:** 10 hours

#### Step 2.6: Refactor Billing Module ViewSets

**Priority Endpoints:**
1. CreditBalanceViewSet - Custom action (balance)
2. CreditUsageViewSet - Read-only + custom actions (summary, limits)
3. CreditTransactionViewSet - Read-only

**Estimated Time:** 4 hours

### Refactoring Checklist Template

For each ViewSet, check:

- [ ] Import response wrapper functions
- [ ] Update `create` method (if overridden)
- [ ] Update `update` method (if overridden)
- [ ] Update `destroy` method (if overridden)
- [ ] Update `list` method (if overridden) - Use `paginated_response` for paginated lists
- [ ] Update `retrieve` method (if overridden)
- [ ] Update all custom `@action` methods
- [ ] Update error handling to use `error_response`
- [ ] Test each endpoint manually
- [ ] Verify response format matches specification

---

## Task 3: Patch DRF Auto-generated Responses (Optional)

### Goal
Ensure DRF's auto-generated responses (from ModelViewSet default methods) also follow the unified format.

### Implementation Steps

#### Step 3.1: Create Base API View Class

**File:** `backend/igny8_core/api/views.py`

**Implementation:**
```python
"""
Base API View Classes with Unified Response Format
"""

from rest_framework.views import APIView
from rest_framework.viewsets import ModelViewSet
from rest_framework.response import Response
from rest_framework import status
from .response import success_response, error_response, paginated_response


class CustomAPIView(APIView):
    """
    Base APIView that automatically wraps responses in unified format.
    
    Override finalize_response to ensure all responses follow the format.
    """
    
    def finalize_response(self, request, response, *args, **kwargs):
        """
        Override to wrap DRF responses in unified format.
        
        This is called for all responses, including exceptions.
        """
        # If response is already in unified format, return as-is
        if isinstance(response.data, dict) and 'success' in response.data:
            return super().finalize_response(request, response, *args, **kwargs)
        
        # Wrap standard DRF responses
        if response.status_code < 400:
            # Success response
            if isinstance(response.data, dict):
                # Check if it's paginated response
                if 'results' in response.data and 'count' in response.data:
                    # Paginated response
                    return super().finalize_response(
                        request,
                        paginated_response(response.data),
                        *args,
                        **kwargs
                    )
                else:
                    # Regular success response
                    return super().finalize_response(
                        request,
                        success_response(response.data),
                        *args,
                        **kwargs
                    )
            else:
                # List or other data
                return super().finalize_response(
                    request,
                    success_response(response.data),
                    *args,
                    **kwargs
                )
        else:
            # Error response
            error_msg = str(response.data) if response.data else "An error occurred"
            errors = response.data if isinstance(response.data, dict) else None
            return super().finalize_response(
                request,
                error_response(
                    error=error_msg,
                    errors=errors,
                    status_code=response.status_code
                ),
                *args,
                **kwargs
            )


class CustomModelViewSet(ModelViewSet):
    """
    Base ModelViewSet that automatically wraps responses in unified format.
    
    Inherit from this instead of ModelViewSet to get automatic response formatting.
    """
    
    def finalize_response(self, request, response, *args, **kwargs):
        """
        Override to wrap DRF responses in unified format.
        """
        # Same logic as CustomAPIView
        if isinstance(response.data, dict) and 'success' in response.data:
            return super().finalize_response(request, response, *args, **kwargs)
        
        if response.status_code < 400:
            if isinstance(response.data, dict) and 'results' in response.data:
                return super().finalize_response(
                    request,
                    paginated_response(response.data),
                    *args,
                    **kwargs
                )
            else:
                return super().finalize_response(
                    request,
                    success_response(response.data),
                    *args,
                    **kwargs
                )
        else:
            error_msg = str(response.data) if response.data else "An error occurred"
            errors = response.data if isinstance(response.data, dict) else None
            return super().finalize_response(
                request,
                error_response(
                    error=error_msg,
                    errors=errors,
                    status_code=response.status_code
                ),
                *args,
                **kwargs
            )
```

#### Step 3.2: Update Base ViewSets (Optional)

**Decision Point:** 
- **Option A:** Update `AccountModelViewSet` and `SiteSectorModelViewSet` to inherit from `CustomModelViewSet`
- **Option B:** Keep current inheritance, manually wrap responses in each ViewSet

**Recommendation:** Option B (manual wrapping) for more control and explicit behavior.

**Estimated Time:** 4 hours (if implemented)

---

## Task 4: Update Exception Handlers

### Goal
Ensure all exceptions (400, 403, 404, 500) are wrapped in the unified error format.

### Implementation Steps

#### Step 4.1: Create Custom Exception Handler

**File:** `backend/igny8_core/api/exception_handlers.py`

**Implementation:**
```python
"""
Custom Exception Handlers for Unified Response Format
"""

from rest_framework.views import exception_handler as drf_exception_handler
from rest_framework import status
from .response import error_response


def custom_exception_handler(exc, context):
    """
    Custom exception handler that wraps all exceptions in unified format.
    
    This handler is called for all exceptions raised in API views.
    It ensures all error responses follow the unified format.
    
    Args:
        exc: The exception instance
        context: Dictionary containing request, view, and args
    
    Returns:
        Response: Error response in unified format, or None to use default
    """
    # Call DRF's default exception handler first
    response = drf_exception_handler(exc, context)
    
    if response is not None:
        # Extract error message and details
        error_message = "An error occurred"
        errors = None
        status_code = response.status_code
        
        # Handle different exception types
        if hasattr(exc, 'detail'):
            # DRF exceptions have 'detail' attribute
            if isinstance(exc.detail, dict):
                # Validation errors - use as errors dict
                errors = exc.detail
                # Extract first error message as top-level error
                if errors:
                    first_key = list(errors.keys())[0]
                    first_error = errors[first_key]
                    if isinstance(first_error, list) and first_error:
                        error_message = f"{first_key}: {first_error[0]}"
                    else:
                        error_message = f"{first_key}: {first_error}"
                else:
                    error_message = "Validation failed"
            elif isinstance(exc.detail, list):
                # List of errors
                error_message = exc.detail[0] if exc.detail else "An error occurred"
                errors = {"non_field_errors": exc.detail}
            else:
                # String error message
                error_message = str(exc.detail)
        else:
            # Generic exception
            error_message = str(exc) if exc else "An error occurred"
        
        # Map status codes to appropriate error messages
        if status_code == status.HTTP_400_BAD_REQUEST:
            if not error_message or error_message == "An error occurred":
                error_message = "Bad request"
        elif status_code == status.HTTP_401_UNAUTHORIZED:
            error_message = "Authentication required" if error_message == "An error occurred" else error_message
        elif status_code == status.HTTP_403_FORBIDDEN:
            error_message = "Permission denied" if error_message == "An error occurred" else error_message
        elif status_code == status.HTTP_404_NOT_FOUND:
            error_message = "Resource not found" if error_message == "An error occurred" else error_message
        elif status_code == status.HTTP_500_INTERNAL_SERVER_ERROR:
            error_message = "Internal server error" if error_message == "An error occurred" else error_message
        
        # Return unified error response
        return error_response(
            error=error_message,
            errors=errors,
            status_code=status_code
        )
    
    # Return None to use default exception handling for unhandled exceptions
    return None
```

#### Step 4.2: Register Exception Handler in Settings

**File:** `backend/igny8_core/settings.py`

**Update REST_FRAMEWORK configuration:**
```python
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'igny8_core.api.authentication.JWTAuthentication',
        'igny8_core.api.authentication.CSRFExemptSessionAuthentication',
        'rest_framework.authentication.BasicAuthentication',
    ],
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.AllowAny',
    ],
    'EXCEPTION_HANDLER': 'igny8_core.api.exception_handlers.custom_exception_handler',  # Add this
    # ... other settings ...
}
```

#### Step 4.3: Test Exception Handling

**Test Cases:**
1. ValidationError (400) - Should return `{ success: false, error: "...", errors: {...} }`
2. PermissionDenied (403) - Should return `{ success: false, error: "Permission denied" }`
3. NotFound (404) - Should return `{ success: false, error: "Resource not found" }`
4. Generic Exception (500) - Should return `{ success: false, error: "Internal server error" }`

**Estimated Time:** 3 hours

---

## Task 5: Validation via Postman/Frontend Logs

### Goal
Manually test endpoints to ensure responses match the unified format.

### Implementation Steps

#### Step 5.1: Create Test Checklist

**Test Endpoints by Module:**

**Auth Module (5 endpoints):**
- [ ] `POST /api/v1/auth/register/` - Success and error cases
- [ ] `POST /api/v1/auth/login/` - Success and error cases
- [ ] `POST /api/v1/auth/change-password/` - Success and error cases
- [ ] `GET /api/v1/auth/me/` - Success and unauthorized cases
- [ ] `GET /api/v1/auth/users/` - List with pagination

**Planner Module (6 endpoints):**
- [ ] `GET /api/v1/planner/keywords/` - List with pagination
- [ ] `POST /api/v1/planner/keywords/` - Create success and validation error
- [ ] `POST /api/v1/planner/keywords/bulk_delete/` - Success and error
- [ ] `POST /api/v1/planner/keywords/auto_cluster/` - Success response
- [ ] `GET /api/v1/planner/clusters/` - List with pagination
- [ ] `POST /api/v1/planner/ideas/bulk_queue_to_writer/` - Success response

**Writer Module (6 endpoints):**
- [ ] `GET /api/v1/writer/tasks/` - List with pagination
- [ ] `POST /api/v1/writer/tasks/` - Create success and validation error
- [ ] `POST /api/v1/writer/tasks/auto_generate_content/` - Success response
- [ ] `GET /api/v1/writer/content/` - List with pagination
- [ ] `POST /api/v1/writer/images/generate_images/` - Success response
- [ ] `GET /api/v1/writer/images/{id}/file/` - Success response

**System Module (5 endpoints):**
- [ ] `GET /api/v1/system/prompts/` - List with pagination
- [ ] `POST /api/v1/system/prompts/save/` - Success and error
- [ ] `GET /api/v1/system/settings/integrations/{pk}/` - Success response
- [ ] `POST /api/v1/system/settings/integrations/{pk}/test/` - Success and error
- [ ] `GET /api/v1/system/status/` - Success response

**Billing Module (3 endpoints):**
- [ ] `GET /api/v1/billing/credits/balance/balance/` - Success response
- [ ] `GET /api/v1/billing/credits/usage/` - List with pagination
- [ ] `GET /api/v1/billing/credits/usage/summary/` - Success response

#### Step 5.2: Create Postman Collection

**File:** `unified-api/postman/API-Response-Format-Tests.postman_collection.json`

**Include:**
- All test endpoints from checklist
- Success case tests
- Error case tests (validation, 404, 403, 500)
- Response format validation (JSON schema)

#### Step 5.3: Test Frontend Integration

**Action Items:**
1. Check browser console for API response logs
2. Verify frontend code handles new response format
3. Test error handling in frontend
4. Verify pagination still works correctly
5. Check for any breaking changes in response structure

**Files to Check:**
- `frontend/src/services/api.ts` - API client
- Frontend stores (Zustand) - Response parsing
- Error handling components

**Estimated Time:** 8 hours

---

## Task 6: Documentation Update

### Goal
Update all relevant documentation to reflect the new unified response format.

### Implementation Steps

#### Step 6.1: Update Backend Implementation Docs

**File:** `docs/04-BACKEND-IMPLEMENTATION.md`

**Add Section: "Standard Response Format"**

```markdown
## Standard Response Format

All API endpoints return responses in a unified JSON format:

### Success Response

```json
{
  "success": true,
  "data": {
    // Response data (object or array)
  },
  "message": "Optional success message"
}
```

### Error Response

```json
{
  "success": false,
  "error": "Top-level error message",
  "errors": {
    "field_name": ["Field-specific error messages"]
  }
}
```

### Paginated Response

```json
{
  "success": true,
  "count": 100,
  "next": "http://api.igny8.com/api/v1/endpoint/?page=2",
  "previous": null,
  "results": [ ... ],
  "message": "Optional message"
}
```

### Status Codes

- `200 OK` - Success
- `201 Created` - Resource created successfully
- `400 Bad Request` - Validation error or bad request
- `401 Unauthorized` - Authentication required
- `403 Forbidden` - Permission denied
- `404 Not Found` - Resource not found
- `500 Internal Server Error` - Server error
```

#### Step 6.2: Update API Endpoints Analysis

**File:** `unified-api/API-ENDPOINTS-ANALYSIS.md`

**Update Section:** "Response Format Standards"

- Mark as "Implemented" or "Standardized"
- Add examples using new format
- Update any outdated examples

#### Step 6.3: Update API Schema (if using Swagger/OpenAPI)

**If using drf-spectacular or similar:**

**File:** `backend/igny8_core/api/schema.py` (if exists)

**Add response schema:**
```python
from drf_spectacular.utils import extend_schema, OpenApiResponse
from drf_spectacular.types import OpenApiTypes

# Define unified response schemas
SUCCESS_RESPONSE_SCHEMA = {
    "type": "object",
    "properties": {
        "success": {"type": "boolean", "example": True},
        "data": {"type": "object"},
        "message": {"type": "string", "nullable": True}
    }
}

ERROR_RESPONSE_SCHEMA = {
    "type": "object",
    "properties": {
        "success": {"type": "boolean", "example": False},
        "error": {"type": "string"},
        "errors": {"type": "object", "nullable": True}
    }
}
```

#### Step 6.4: Create API Design Guide Section

**File:** `docs/API-DESIGN-GUIDE.md` (create if doesn't exist)

**Add:**
- Response format policy
- When to use `success_response` vs `error_response`
- Pagination guidelines
- Error message best practices
- Status code guidelines

**Estimated Time:** 4 hours

---

## Task 7: Changelog Entry

### Goal
Document the changes in the project changelog.

### Implementation Steps

#### Step 7.1: Update CHANGELOG.md

**File:** `CHANGELOG.md` (or similar)

**Add Entry:**
```markdown
## [Unreleased] - 2025-01-XX

### Changed
- **API Response Format**: Unified all API responses under standardized success/error format
  - All endpoints now return `{ success, data, message, error, errors }` format
  - Created `igny8_core.api.response` module with `success_response()`, `error_response()`, and `paginated_response()` helpers
  - Updated custom exception handler to wrap all errors in unified format
  - Refactored all ViewSets across Auth, Planner, Writer, System, and Billing modules
  - Updated documentation with response format examples

### Affected Areas
- API Layer (`igny8_core/api/`)
- Auth Module (`auth/api/`)
- Planner Module (`planner/api/`)
- Writer Module (`writer/api/`)
- System Module (`system/api/`)
- Billing Module (`billing/api/`)

### Breaking Changes
- ⚠️ **Response Format Change**: All API responses now follow unified format
  - Frontend code may need updates if it directly accesses response fields
  - Paginated responses now include `success: true` and optional `message` field
  - Error responses now use `error` field instead of `message` for top-level errors

### Migration Guide
1. Update frontend API client to handle new response format
2. Update error handling to check `success` field
3. Update pagination handling to account for `success` and `message` fields in paginated responses
```

**Estimated Time:** 1 hour

---

## Testing Strategy

### Unit Tests

**File:** `backend/igny8_core/api/tests/test_response.py`

**Test Cases:**
- Response wrapper functions
- Exception handler
- Edge cases (None data, empty lists, etc.)

### Integration Tests

**Test Cases:**
- End-to-end API calls
- Error scenarios (validation, 404, 500)
- Pagination with unified format
- Authentication errors

### Manual Testing

**Checklist:**
- [ ] All endpoints return unified format
- [ ] Success responses have `success: true`
- [ ] Error responses have `success: false`
- [ ] Pagination includes `success: true`
- [ ] Error messages are human-readable
- [ ] Validation errors include field-specific errors

---

## Rollout Plan

### Phase 1: Foundation (Week 1)
- ✅ Task 1: Create response wrapper functions
- ✅ Task 4: Update exception handlers
- ✅ Unit tests for response utilities

### Phase 2: Module Refactoring (Week 2-3)
- ✅ Task 2.2: Refactor Auth module
- ✅ Task 2.3: Refactor Planner module
- ✅ Task 2.4: Refactor Writer module
- ✅ Task 2.5: Refactor System module
- ✅ Task 2.6: Refactor Billing module

### Phase 3: Testing & Validation (Week 4)
- ✅ Task 5: Manual testing
- ✅ Frontend integration testing
- ✅ Bug fixes and adjustments

### Phase 4: Documentation & Release (Week 5)
- ✅ Task 6: Documentation updates
- ✅ Task 7: Changelog entry
- ✅ Release to staging
- ✅ Production deployment

---

## Success Criteria

### Definition of Done

1. ✅ All API endpoints return unified response format
2. ✅ Response wrapper functions are unit tested
3. ✅ Exception handler wraps all errors correctly
4. ✅ All ViewSets use response wrappers
5. ✅ Documentation is updated
6. ✅ Frontend integration tested and working
7. ✅ No breaking changes for critical endpoints (or migration guide provided)
8. ✅ Postman collection updated with new format
9. ✅ Changelog entry created

### Metrics

- **Coverage:** 100% of API endpoints use unified format
- **Test Coverage:** >90% for response utilities
- **Documentation:** All modules documented
- **Breaking Changes:** Documented and migration guide provided

---

## Risk Assessment

### Risks

1. **Breaking Changes:** Frontend may break if it expects old format
   - **Mitigation:** Provide migration guide, test frontend integration early

2. **Performance Impact:** Wrapping responses may add overhead
   - **Mitigation:** Minimal overhead, test performance if concerned

3. **Incomplete Migration:** Some endpoints may be missed
   - **Mitigation:** Comprehensive audit checklist, automated tests

4. **Exception Handler Conflicts:** Custom exception handling may conflict
   - **Mitigation:** Test thoroughly, handle edge cases

### Rollback Plan

If issues arise:
1. Revert exception handler changes (keep in settings but disable)
2. Keep response wrapper functions (non-breaking)
3. Gradually roll back ViewSet changes if needed
4. Document issues for future fixes

---

## Appendix

### Response Format Examples

#### Success with Data
```json
{
  "success": true,
  "data": {
    "id": 1,
    "name": "Example Keyword",
    "status": "active"
  },
  "message": "Keyword created successfully"
}
```

#### Success with List
```json
{
  "success": true,
  "data": [
    {"id": 1, "name": "Keyword 1"},
    {"id": 2, "name": "Keyword 2"}
  ],
  "message": "Keywords retrieved successfully"
}
```

#### Error with Validation
```json
{
  "success": false,
  "error": "Validation failed",
  "errors": {
    "email": ["Invalid email format"],
    "password": ["Password must be at least 8 characters"]
  }
}
```

#### Error without Details
```json
{
  "success": false,
  "error": "Resource not found"
}
```

#### Paginated Response
```json
{
  "success": true,
  "count": 150,
  "next": "http://api.igny8.com/api/v1/planner/keywords/?page=2",
  "previous": null,
  "results": [
    {"id": 1, "name": "Keyword 1"},
    {"id": 2, "name": "Keyword 2"}
  ],
  "message": "Keywords retrieved successfully"
}
```

---

**Document Status:** Implementation Plan  
**Last Updated:** 2025-01-XX  
**Next Review:** After Phase 1 completion