diff --git a/unified-api/API-STANDARD-v1.0.md b/unified-api/API-STANDARD-v1.0.md new file mode 100644 index 00000000..39ca1a95 --- /dev/null +++ b/unified-api/API-STANDARD-v1.0.md @@ -0,0 +1,1303 @@ +# IGNY8 API STANDARD v1.0 + +**Version:** 1.0 +**Date:** 2025-01-XX +**Status:** Active Standard +**Purpose:** Single source of truth for IGNY8 API contract, implementation, and integration + +**Applies To:** +- Django backend +- React frontend +- WordPress plugin +- 3rd-party API consumers +- Internal automations / CRON / AI functions + +--- + +## Table of Contents + +1. [Overview](#overview) +2. [API Versioning & Base URLs](#api-versioning--base-urls) +3. [Authentication & Authorization](#authentication--authorization) +4. [Response Format Standard](#response-format-standard) +5. [Error Handling Standard](#error-handling-standard) +6. [Rate Limits & Governance](#rate-limits--governance) +7. [Pagination & Query Rules](#pagination--query-rules) +8. [Standard Request & Response Shapes](#standard-request--response-shapes) +9. [Roles & Permissions Matrix](#roles--permissions-matrix) +10. [Tenant / Site / Sector Scoping](#tenant--site--sector-scoping) +11. [Module-wise Endpoint Rules](#module-wise-endpoint-rules) +12. [Logging, Request ID, and Debugging](#logging-request-id-and-debugging) +13. [Frontend and Plugin Integration Requirements](#frontend-and-plugin-integration-requirements) +14. [Migration Plan for Existing Code](#migration-plan-for-existing-code) +15. [Testing Strategy](#testing-strategy) +16. [Change Management](#change-management) + +--- + +## Overview + +The IGNY8 API Standard v1.0 establishes a unified, consistent interface for all API endpoints across the platform. This standard ensures: + +- **Predictability**: All endpoints follow the same patterns +- **Security**: Consistent authentication and authorization +- **Maintainability**: Standardized error handling and logging +- **Scalability**: Rate limiting and governance controls +- **Developer Experience**: Clear documentation and integration patterns + +### Key Principles + +1. **Unified Response Format**: All endpoints return consistent JSON structure +2. **Layered Authorization**: Authentication → Tenant Access → Role → Site/Sector +3. **Centralized Error Handling**: All errors wrapped in unified format +4. **Scoped Rate Limiting**: Different limits for different operation types +5. **Tenant Isolation**: All resources scoped by account/site/sector +6. **Request Tracking**: Every request has a unique ID for debugging + +--- + +## API Versioning & Base URLs + +### Base URL Structure + +``` +Production: https://api.igny8.com/api/v1/ +Development: http://localhost:8000/api/v1/ +``` + +### Module Namespaces + +All endpoints are organized under these namespaces: + +``` +/api/v1/ +├── auth/ # Authentication and user management +├── planner/ # Keywords, clusters, content ideas +├── writer/ # Tasks, content, images +├── system/ # Settings, prompts, integrations +└── billing/ # Credits, transactions, usage +``` + +### Versioning Rules + +- **v1** remains stable long-term +- Breaking changes require **v2** +- Deprecations allowed only with explicit timeline +- All endpoints must follow predictable REST patterns + +### Endpoint Naming Conventions + +- Use plural nouns: `/api/v1/planner/keywords/` +- Use snake_case for query parameters: `?site_id=1§or_id=2` +- Use kebab-case for custom actions: `/api/v1/planner/keywords/auto-cluster/` + +--- + +## Authentication & Authorization + +### Authentication Methods + +#### Primary: JWT Bearer Token + +``` +Authorization: Bearer +``` + +**Token Characteristics:** +- Contains `user_id` and `account_id` +- Type: `access` (15-minute expiry) +- Automatically sets `request.account` via middleware +- Resolves account → tenant context automatically + +**Token Payload:** +```json +{ + "user_id": 1, + "account_id": 1, + "type": "access", + "exp": 1234567890 +} +``` + +#### Fallback Methods + +1. **Session Authentication** (admin panel) + - Class: `CSRFExemptSessionAuthentication` + - Use case: Django admin panel (`/admin/`) + +2. **Basic Authentication** (debug/testing) + - Class: `rest_framework.authentication.BasicAuthentication` + - Use case: API testing tools (Postman, curl) + +### Authentication Order + +1. JWT Token Authentication (tried first) +2. Session Authentication (fallback) +3. Basic Authentication (last fallback) +4. If all fail: 401 Unauthorized + +### Public Endpoints (No Authentication Required) + +- `POST /api/v1/auth/register/` +- `POST /api/v1/auth/login/` +- `GET /api/v1/auth/plans/` +- `GET /api/v1/auth/industries/` +- `GET /api/v1/system/status/` +- `GET /api/ping/` (health check) + +**All other endpoints require JWT authentication.** + +### Authorization Layers + +Every endpoint enforces layered authorization: + +1. **User Authentication**: User must be authenticated +2. **Tenant Access**: User must belong to the tenant/account +3. **Role Authorization**: User must have appropriate role +4. **Site/Sector Access**: User must have access to requested site/sector + +--- + +## Response Format Standard + +### Mandatory Format + +**This is the global standard for all endpoints - no exceptions.** + +### Success Response + +```json +{ + "success": true, + "data": { + // Response data (object or array) + }, + "message": "Optional human-readable success message" +} +``` + +**Example:** +```json +{ + "success": true, + "data": { + "id": 1, + "name": "Example Keyword", + "status": "active" + }, + "message": "Keyword created successfully" +} +``` + +### Error Response + +```json +{ + "success": false, + "error": "Readable top-level error message", + "errors": { + "field_name": ["Field-specific error messages"] + }, + "request_id": "uuid-for-error-tracking" +} +``` + +**Example:** +```json +{ + "success": false, + "error": "Validation failed", + "errors": { + "email": ["Invalid email format"], + "password": ["Password must be at least 8 characters"] + }, + "request_id": "550e8400-e29b-41d4-a716-446655440000" +} +``` + +### Paginated Response + +```json +{ + "success": true, + "count": 120, + "next": "http://api.igny8.com/api/v1/endpoint/?page=2", + "previous": null, + "results": [ + // Array of results + ], + "message": "Optional message" +} +``` + +### Response Helper Functions + +**File:** `backend/igny8_core/api/response.py` + +```python +from igny8_core.api.response import success_response, error_response, paginated_response + +# Success response +return success_response( + data={"id": 1, "name": "Example"}, + message="Resource created successfully", + status_code=status.HTTP_201_CREATED +) + +# Error response +return error_response( + error="Validation failed", + errors={"email": ["Invalid email format"]}, + status_code=status.HTTP_400_BAD_REQUEST +) + +# Paginated response +paginator = CustomPageNumberPagination() +page = paginator.paginate_queryset(queryset, request) +serializer = MySerializer(page, many=True) +paginated_data = paginator.get_paginated_response(serializer.data).data +return paginated_response(paginated_data, message="Resources retrieved successfully") +``` + +--- + +## Error Handling Standard + +### Centralized Exception Handler + +**File:** `backend/igny8_core/api/exception_handlers.py` + +All exceptions are handled by a centralized exception handler that: + +- Wraps all errors in unified format +- Uses proper HTTP status codes (400, 401, 403, 404, 409, 422, 500) +- Includes sanitized validation errors under `errors` +- Always attaches `request_id` for error tracking +- Logs full exception details +- In DEBUG mode: includes traceback + request context + +### Status Code Mapping + +| Status Code | Meaning | Response Format | +|------------|---------|----------------| +| 200 | OK | `{ success: true, data: {...} }` | +| 201 | Created | `{ success: true, data: {...} }` | +| 400 | Bad Request | `{ success: false, error: "...", errors: {...} }` | +| 401 | Unauthorized | `{ success: false, error: "Authentication required" }` | +| 403 | Forbidden | `{ success: false, error: "Permission denied" }` | +| 404 | Not Found | `{ success: false, error: "Resource not found" }` | +| 409 | Conflict | `{ success: false, error: "Conflict" }` | +| 422 | Unprocessable Entity | `{ success: false, error: "...", errors: {...} }` | +| 429 | Too Many Requests | `{ success: false, error: "Rate limit exceeded" }` | +| 500 | Internal Server Error | `{ success: false, error: "Internal server error" }` | + +### Error Response Structure + +**Production Mode:** +```json +{ + "success": false, + "error": "Internal server error", + "request_id": "550e8400-e29b-41d4-a716-446655440000" +} +``` + +**Development Mode (DEBUG=True):** +```json +{ + "success": false, + "error": "Internal server error", + "request_id": "550e8400-e29b-41d4-a716-446655440000", + "debug": { + "exception_type": "ValueError", + "exception_message": "Invalid value", + "view": "KeywordViewSet", + "path": "/api/v1/planner/keywords/", + "method": "POST", + "traceback": "Traceback (most recent call last):\n ..." + } +} +``` + +### Server-side Logging + +- All 4xx errors logged as **warning** +- All 5xx errors logged as **error** +- Structured format with: + - timestamp + - request_id + - endpoint + - user_id + - account_id + - status_code + - error_message + - traceback (for 5xx) +- Rotating log files +- Sentry integration hooks for production + +--- + +## Rate Limits & Governance + +### Rate Limiting Configuration + +**File:** `backend/igny8_core/settings.py` + +```python +REST_FRAMEWORK = { + 'DEFAULT_THROTTLE_CLASSES': [ + 'igny8_core.api.throttles.DebugScopedRateThrottle', + ], + 'DEFAULT_THROTTLE_RATES': { + # AI Functions - Expensive operations + 'ai_function': '10/min', # AI content generation, clustering + 'image_gen': '15/min', # Image generation + + # Content Operations + 'content_write': '30/min', # Content creation, updates + 'content_read': '100/min', # Content listing, retrieval + + # Authentication + 'auth': '20/min', # Login, register, password reset + 'auth_strict': '5/min', # Sensitive auth operations + + # Planner Operations + 'planner': '60/min', # Keyword, cluster, idea operations + 'planner_ai': '10/min', # AI-powered planner operations + + # Writer Operations + 'writer': '60/min', # Task, content management + 'writer_ai': '10/min', # AI-powered writer operations + + # System Operations + 'system': '100/min', # Settings, prompts, profiles + 'system_admin': '30/min', # Admin-only system operations + + # Billing Operations + 'billing': '30/min', # Credit queries, usage logs + 'billing_admin': '10/min', # Credit management (admin) + + # Default fallback + 'default': '100/min', # Default for endpoints without scope + }, +} +``` + +### Throttle Headers + +All responses include rate limit information: + +``` +X-Throttle-Limit: 10 +X-Throttle-Remaining: 9 +X-Throttle-Reset: 1640995200 +``` + +When throttled (429): +``` +HTTP/1.1 429 Too Many Requests +X-Throttle-Limit: 10 +X-Throttle-Remaining: 0 +Retry-After: 60 +``` + +### Soft Limits (Business Logic) + +- **50 tasks** per bulk action +- **20 keywords** per cluster batch +- **1 cluster** → ideas generation +- **1 content** → image prompt extraction +- **1 image** generation per job + +### Safety Checks + +- Validate all IDs belong to tenant +- Validate dependent records exist +- Reject oversized payloads +- Reject invalid site/sector assignments +- Reject stale/wrong tenant tokens + +### Debug Mode Bypass + +Set `IGNY8_DEBUG_THROTTLE=True` or `DEBUG=True` to bypass throttling in development. + +--- + +## Pagination & Query Rules + +### Pagination Configuration + +**Default Settings:** +- Default page size: **10** +- Maximum page size: **100** +- Query parameter: `page_size` (optional) +- Page parameter: `page` (default: 1) + +### Query Parameters + +**Pagination:** +``` +?page=2&page_size=25 +``` + +**Filtering:** +``` +?status=active +?site_id=1 +?sector_id=2 +?cluster_id=5 +``` + +**Search:** +``` +?search=keyword +``` + +**Ordering:** +``` +?ordering=-created_at +?ordering=name,status +``` + +### Pagination Response Format + +```json +{ + "success": true, + "count": 150, + "next": "http://api.igny8.com/api/v1/planner/keywords/?page=3&page_size=25", + "previous": "http://api.igny8.com/api/v1/planner/keywords/?page=1&page_size=25", + "results": [ + // Array of results + ], + "message": "Keywords retrieved successfully" +} +``` + +--- + +## Standard Request & Response Shapes + +### Request Payload Rules + +All endpoints must follow: + +- **snake_case** field names +- Predictable field groups +- Standardized pagination params +- Standardized filters +- Standardized bulk update patterns +- Standardized action payloads +- Standardized error messages + +### Bulk Actions + +**Standard Format:** +```json +{ + "ids": [1, 2, 3], + // Additional fields as needed +} +``` + +**Example:** +```json +{ + "ids": [1, 2, 3], + "status": "active" +} +``` + +### Filtering Patterns + +**Standard Query Parameters:** +- `?search=` - Text search +- `?status=` - Status filter +- `?site_id=` - Site filter +- `?sector_id=` - Sector filter +- `?cluster_id=` - Cluster filter +- `?content_type=` - Content type filter + +### Request Examples + +**Create Resource:** +```json +POST /api/v1/planner/keywords/ +{ + "keyword": "example keyword", + "site_id": 1, + "sector_id": 2, + "status": "active" +} +``` + +**Bulk Update:** +```json +POST /api/v1/planner/keywords/bulk_update_status/ +{ + "ids": [1, 2, 3], + "status": "active" +} +``` + +**Custom Action:** +```json +POST /api/v1/planner/keywords/auto_cluster/ +{ + "ids": [1, 2, 3, 4, 5], + "sector_id": 2 +} +``` + +--- + +## Roles & Permissions Matrix + +### Role Hierarchy + +``` +owner > admin > editor > viewer > system_bot +``` + +### Standard Permission Classes + +**File:** `backend/igny8_core/api/permissions.py` + +| Permission Class | Description | Use Case | +|-----------------|------------|----------| +| `IsAuthenticatedAndActive` | User authenticated and active | Base permission for most endpoints | +| `HasTenantAccess` | User belongs to tenant/account | Tenant isolation | +| `IsViewerOrAbove` | Viewer, editor, admin, or owner | Read-only operations | +| `IsEditorOrAbove` | Editor, admin, or owner | Content operations | +| `IsAdminOrOwner` | Admin or owner only | Settings, keys, billing | + +### Permission Matrix by Endpoint Type + +| Endpoint Type | Required Permissions | Roles Allowed | +|--------------|---------------------|---------------| +| Public (register, login) | `AllowAny` | Anyone | +| Read-only (list, retrieve) | `IsAuthenticatedAndActive` + `HasTenantAccess` | All authenticated users | +| Content operations | `IsAuthenticatedAndActive` + `HasTenantAccess` + `IsEditorOrAbove` | Editor, Admin, Owner | +| User management | `IsAuthenticatedAndActive` + `HasTenantAccess` + `IsAdminOrOwner` | Admin, Owner | +| Billing/Transactions | `IsAuthenticatedAndActive` + `HasTenantAccess` + `IsAdminOrOwner` | Admin, Owner | +| Integration settings | `IsAuthenticatedAndActive` + `HasTenantAccess` + `IsAdminOrOwner` | Admin, Owner | + +### Base ViewSet Class + +**File:** `backend/igny8_core/api/viewsets.py` + +```python +from igny8_core.api.viewsets import BaseTenantViewSet +from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess + +class MyViewSet(BaseTenantViewSet): + permission_classes = [ + IsAuthenticatedAndActive, + HasTenantAccess, + ] + throttle_scope = 'planner' + queryset = MyModel.objects.all() + serializer_class = MySerializer +``` + +--- + +## Tenant / Site / Sector Scoping + +### Scoping Rules + +Every resource created or fetched must be scoped by: + +1. **Account/Tenant** - User's account +2. **Site** - Specific site within account +3. **Sector** - Specific sector within site + +### Enforcement + +**Base Classes:** +- `AccountModelViewSet` - Handles account isolation +- `SiteSectorModelViewSet` - Filters queries by site/sector + +**Requirements:** +- All custom actions must use `.get_queryset()` to avoid bypassing filters +- Any ID list must be verified to belong to the authenticated tenant +- Site/sector access validated based on user role + +### Scoping Example + +```python +class KeywordViewSet(SiteSectorModelViewSet): + # Automatically filters by: + # 1. account (from request.account) + # 2. site_id (from query params or request) + # 3. sector_id (from query params or request) + + queryset = Keyword.objects.all() + serializer_class = KeywordSerializer + + def get_queryset(self): + # Base class handles account/site/sector filtering + queryset = super().get_queryset() + # Additional filtering can be added here + return queryset +``` + +--- + +## Module-wise Endpoint Rules + +### Planner Module + +**Covers:** Keywords → Clusters → Ideas + +**Rules:** +- All endpoints require authentication +- Auto-cluster limited to **20 keywords/request** +- Ideas generation limited to **1 cluster/request** +- Standardized payloads for imports/exports +- Bulk actions must validate tenant ownership +- Throttle scope: `planner` (standard), `planner_ai` (AI operations) + +**Key Endpoints:** +- `GET/POST /api/v1/planner/keywords/` +- `POST /api/v1/planner/keywords/auto_cluster/` +- `POST /api/v1/planner/clusters/auto_generate_ideas/` +- `POST /api/v1/planner/ideas/bulk_queue_to_writer/` + +### Writer Module + +**Covers:** Tasks → Content → Image Prompts → Images + +**Rules:** +- 1 AI content generation per task +- 1 image generation per request +- Bulk actions max **50** +- Image prompt extraction uses unified response +- Must align with progress tracking +- Throttle scope: `writer` (standard), `writer_ai` (AI operations), `image_gen` (image generation) + +**Key Endpoints:** +- `GET/POST /api/v1/writer/tasks/` +- `POST /api/v1/writer/tasks/auto_generate_content/` +- `POST /api/v1/writer/content/generate_image_prompts/` +- `POST /api/v1/writer/images/generate_images/` + +### System Module + +**Covers:** Prompts, Strategies, Profiles, Integrations, Task Progress + +**Rules:** +- Saving prompts requires **editor/admin** +- Integration settings require **admin/owner** +- All tests (OpenAI/Runware) return unified format +- Progress endpoint returns stable JSON for progress modal +- Throttle scope: `system` (standard), `system_admin` (admin operations) + +**Key Endpoints:** +- `GET/POST /api/v1/system/prompts/` +- `POST /api/v1/system/prompts/save/` +- `POST /api/v1/system/settings/integrations/{pk}/test/` +- `GET /api/v1/system/settings/task_progress/{task_id}/` + +### Billing Module + +**Covers:** Credit Balance, Usage Logs, Transactions + +**Rules:** +- Balance/usage require authenticated tenant +- Transactions require **admin/owner** +- Unified reporting structure +- No raw model data returned +- Throttle scope: `billing` (standard), `billing_admin` (admin operations) + +**Key Endpoints:** +- `GET /api/v1/billing/credits/balance/balance/` +- `GET /api/v1/billing/credits/usage/` +- `GET /api/v1/billing/credits/usage/summary/` +- `GET /api/v1/billing/credits/transactions/` + +--- + +## Logging, Request ID, and Debugging + +### Request ID Tracking + +**Middleware:** `RequestIDMiddleware` + +Every request gets a unique request ID: +- Generated UUID if not provided +- Included in response headers: `X-Request-ID` +- Included in all error responses +- Used for log correlation + +### Backend Logging + +**Structured Log Format:** +```json +{ + "timestamp": "2025-01-15T12:30:00Z", + "request_id": "550e8400-e29b-41d4-a716-446655440000", + "endpoint": "/api/v1/planner/keywords/", + "method": "POST", + "user_id": 1, + "account_id": 1, + "status_code": 200, + "error_message": null, + "traceback": null +} +``` + +**Log Levels:** +- **INFO**: Normal operations +- **WARNING**: 4xx errors (client errors) +- **ERROR**: 5xx errors (server errors) +- **DEBUG**: Detailed debugging (only in DEBUG mode) + +**Log Files:** +- `logs/django.log` - General logs +- `logs/errors.log` - Error logs only + +### Frontend Debug Panel Integration + +All responses must support: + +- **Request ID** - For error tracking +- **Readable error messages** - User-friendly messages +- **Consistent progress steps** - For AI functions +- **Error visibility in real time** - For debugging + +### Progress Modal Standard (AI Functions) + +All AI functions must output uniform progress metadata: + +**Example Steps (Image Prompt Extraction):** +``` +1. Smart Image Prompts +2. Checking content and image slots +3. Mapping Content for X Image Prompts +4. Writing Featured Image Prompts +5. Writing X In-article Image Prompts +6. Assigning Prompts to Slots +``` + +**Success Message:** +``` +Featured Image and X In-article Image Prompts ready for image generation +``` + +**Progress Response Format:** +```json +{ + "state": "PROGRESS", + "meta": { + "phase": "AI_CALL", + "percentage": 50, + "current_step": "Writing Featured Image Prompts", + "total_steps": 6, + "steps": [ + "Smart Image Prompts", + "Checking content and image slots", + "Mapping Content for X Image Prompts", + "Writing Featured Image Prompts", + "Writing X In-article Image Prompts", + "Assigning Prompts to Slots" + ] + } +} +``` + +--- + +## Frontend and Plugin Integration Requirements + +### Frontend Integration + +**Response Parsing:** +```typescript +const response = await fetchAPI('/v1/planner/keywords/'); +const data = await response.json(); + +if (data.success) { + // Handle success + const keywords = data.data; // or data.results for paginated +} else { + // Handle error + console.error(data.error); + if (data.errors) { + // Handle field-specific errors + Object.keys(data.errors).forEach(field => { + console.error(`${field}: ${data.errors[field].join(', ')}`); + }); + } +} +``` + +**Error Handling:** +- **401 Unauthorized**: Trigger logout → redirect to login +- **403 Forbidden**: Show permission alert +- **429 Too Many Requests**: Show rate limit warning with retry time +- **4xx/5xx**: Display error message from `error` field + +**Pagination:** +```typescript +const response = await fetchAPI('/v1/planner/keywords/?page=1&page_size=25'); +const data = await response.json(); + +if (data.success) { + const keywords = data.results; // Paginated results + const count = data.count; + const next = data.next; + const previous = data.previous; +} +``` + +**Rate Limit Handling:** +```typescript +const throttleLimit = response.headers.get('X-Throttle-Limit'); +const throttleRemaining = response.headers.get('X-Throttle-Remaining'); + +if (parseInt(throttleRemaining) < 5) { + showNotification('Approaching rate limit', 'warning'); +} +``` + +### WordPress Plugin Integration + +**Requirements:** +- Uses the same JWT flow +- No custom plugin-specific endpoints +- Uses the app's `/api/v1/` fully +- WP only handles mapping + meta assignment + +**Authentication:** +```php +$token = get_option('igny8_api_token'); +$response = wp_remote_get('https://api.igny8.com/api/v1/planner/keywords/', [ + 'headers' => [ + 'Authorization' => 'Bearer ' . $token, + 'Content-Type' => 'application/json', + ], +]); +``` + +### 3rd-Party API Integration + +**Requirements:** +- Full compatibility with unified format +- Full access where tenant/permissions allow +- No special handling needed + +**Example (Python):** +```python +import requests + +def get_keywords(access_token): + url = 'https://api.igny8.com/api/v1/planner/keywords/' + headers = { + 'Authorization': f'Bearer {access_token}', + 'Content-Type': 'application/json' + } + + response = requests.get(url, headers=headers) + result = response.json() + + if result['success']: + return result['data'] # or result['results'] for paginated + else: + raise Exception(result['error']) +``` + +--- + +## Migration Plan for Existing Code + +### Phase 1: Foundation (Week 1) + +1. **Create Response Utilities** + - File: `backend/igny8_core/api/response.py` + - Functions: `success_response()`, `error_response()`, `paginated_response()` + - Unit tests + +2. **Create Permission Classes** + - File: `backend/igny8_core/api/permissions.py` + - Classes: `IsAuthenticatedAndActive`, `HasTenantAccess`, `IsAdminOrOwner`, etc. + - Unit tests + +3. **Create Base ViewSet** + - File: `backend/igny8_core/api/viewsets.py` + - Class: `BaseTenantViewSet` + - Automatic permission injection + +4. **Create Exception Handler** + - File: `backend/igny8_core/api/exception_handlers.py` + - Function: `custom_exception_handler()` + - Register in settings + +5. **Configure Rate Limiting** + - File: `backend/igny8_core/api/throttles.py` + - Class: `DebugScopedRateThrottle` + - Configure in settings + +### Phase 2: Module Refactoring (Week 2-3) + +**Refactoring Order (Least to Most Dependent):** + +1. **System Module** (12 hours) + - Refactor all ViewSets to use `BaseTenantViewSet` + - Add throttle scopes + - Replace Response() with helper functions + +2. **Billing Module** (8 hours) + - Same refactoring steps + +3. **Planner Module** (16 hours) + - Same refactoring steps + - Special attention to AI functions + +4. **Writer Module** (16 hours) + - Same refactoring steps + - Special attention to AI functions and image generation + +5. **Auth Module** (12 hours) + - Same refactoring steps + - Keep public endpoints (register, login) as `AllowAny` + +### Phase 3: Testing & Validation (Week 4) + +1. **Unit Tests** + - Test all response helpers + - Test permission classes + - Test exception handler + - Test rate limiting + +2. **Integration Tests** + - Test all endpoints return unified format + - Test error handling + - Test rate limiting + - Test permissions + +3. **Frontend Integration Testing** + - Test all workflows + - Verify error handling + - Verify rate limit warnings + - Verify pagination + +### Phase 4: Documentation & Release (Week 5) + +1. **Update Documentation** + - API usage guide + - Backend implementation docs + - Swagger/OpenAPI (optional) + +2. **Changelog Entry** + - Document all changes + - Breaking changes + - Migration guide + +3. **Production Deployment** + - Staging testing + - Production deployment + - Monitoring setup + +### Refactoring Checklist Template + +For each ViewSet: + +- [ ] Inherit from `BaseTenantViewSet` +- [ ] Set appropriate `permission_classes` +- [ ] Add `throttle_scope` +- [ ] Replace `Response()` with `success_response()` or `error_response()` +- [ ] Update custom actions with explicit permissions +- [ ] Validate payloads in custom actions +- [ ] Test endpoint manually +- [ ] Verify response format matches specification +- [ ] Update documentation + +--- + +## Testing Strategy + +### Unit Tests + +**File:** `backend/igny8_core/api/tests/test_response.py` + +```python +def test_success_response(): + response = success_response(data={"id": 1}, message="Success") + assert response.status_code == 200 + data = response.data + assert data['success'] == True + assert data['data'] == {"id": 1} + assert data['message'] == "Success" + +def test_error_response(): + response = error_response( + error="Validation failed", + errors={"email": ["Invalid"]} + ) + assert response.status_code == 400 + data = response.data + assert data['success'] == False + assert data['error'] == "Validation failed" + assert data['errors'] == {"email": ["Invalid"]} +``` + +**File:** `backend/igny8_core/api/tests/test_permissions.py` + +```python +def test_has_tenant_access(): + user = User.objects.create(account=account1) + request.user = user + request.account = account1 + + permission = HasTenantAccess() + assert permission.has_permission(request, view) == True + + request.account = account2 + assert permission.has_permission(request, view) == False +``` + +### Integration Tests + +**File:** `backend/igny8_core/api/tests/test_standardized_endpoints.py` + +```python +class StandardizedEndpointTestCase(TestCase): + def setUp(self): + self.client = APIClient() + self.user = User.objects.create_user(...) + self.client.force_authenticate(user=self.user) + + def test_list_endpoint_returns_unified_format(self): + response = self.client.get('/api/v1/planner/keywords/') + self.assertEqual(response.status_code, 200) + data = response.json() + self.assertIn('success', data) + self.assertTrue(data['success']) + self.assertIn('results', data) + + def test_create_endpoint_returns_unified_format(self): + response = self.client.post('/api/v1/planner/keywords/', { + 'keyword': 'test', + 'site_id': 1, + 'sector_id': 1 + }) + self.assertEqual(response.status_code, 201) + data = response.json() + self.assertIn('success', data) + self.assertTrue(data['success']) + self.assertIn('data', data) + + def test_error_returns_unified_format(self): + response = self.client.post('/api/v1/planner/keywords/', {}) + self.assertEqual(response.status_code, 400) + data = response.json() + self.assertIn('success', data) + self.assertFalse(data['success']) + self.assertIn('error', data) + self.assertIn('errors', data) + self.assertIn('request_id', data) +``` + +### Manual Testing + +**Postman Collection:** +- Test all CRUD operations +- Test custom actions +- Test error scenarios +- Test rate limiting +- Verify response format + +**Frontend Testing:** +- Test all workflows +- Verify error messages display correctly +- Verify rate limit warnings +- Verify pagination works +- Check browser console for errors + +### Test Coverage Goals + +- **Unit Tests**: >90% coverage for response utilities, permissions, exception handler +- **Integration Tests**: All major endpoints tested +- **Manual Tests**: All workflows tested + +--- + +## Change Management + +### Versioning Strategy + +- **v1** remains stable long-term +- Breaking changes require **v2** +- Deprecations allowed only with explicit timeline +- Non-breaking changes can be added to v1 + +### Breaking Changes + +**Definition:** Changes that require client code updates + +**Examples:** +- Removing an endpoint +- Changing response structure +- Changing authentication method +- Removing a field from response + +**Process:** +1. Document breaking change +2. Provide migration guide +3. Deprecate in v1 with timeline +4. Implement in v2 +5. Maintain v1 for deprecation period + +### Non-Breaking Changes + +**Definition:** Changes that don't require client code updates + +**Examples:** +- Adding new endpoints +- Adding optional fields to response +- Adding new query parameters +- Performance improvements + +**Process:** +1. Implement in v1 +2. Document in changelog +3. No migration required + +### Changelog Format + +**File:** `CHANGELOG.md` + +```markdown +## [1.1.0] - 2025-01-XX + +### Added +- New endpoint: `GET /api/v1/system/health/` +- Optional field `metadata` in keyword response + +### Changed +- Improved error messages for validation failures +- Rate limits increased for AI functions (10/min → 20/min) + +### Deprecated +- `GET /api/v1/planner/keywords/legacy/` (will be removed in v2.0.0) + +### Removed +- (None) + +### Security +- Fixed authentication bypass in custom actions +``` + +### Communication Plan + +1. **Internal Team** + - Document changes in changelog + - Update internal documentation + - Notify team via Slack/email + +2. **External API Consumers** + - Update API documentation + - Send email notification for breaking changes + - Provide migration guide + +3. **WordPress Plugin** + - Update plugin code if needed + - Test compatibility + - Release plugin update + +### Rollback Plan + +If issues arise: + +1. **Immediate Rollback** + - Revert code changes + - Restore previous version + - Monitor for issues + +2. **Partial Rollback** + - Revert specific module changes + - Keep other improvements + - Fix issues incrementally + +3. **Feature Flag** + - Disable new features via feature flags + - Keep code in place + - Re-enable after fixes + +--- + +## Success Criteria + +### Definition of Done + +- [ ] Every endpoint returns unified format +- [ ] All errors fully unified +- [ ] No endpoint leaks raw DRF format +- [ ] No inconsistent auth behavior +- [ ] No inconsistent pagination +- [ ] No inconsistent validation errors +- [ ] Plugin works using same exact API +- [ ] Third-party clients can implement without special rules +- [ ] Frontend debug panel reads everything correctly +- [ ] CRON + automations structured identically +- [ ] All tests pass +- [ ] Documentation complete +- [ ] Changelog updated + +### Metrics + +- **Coverage**: 100% of endpoints use unified format +- **Test Coverage**: >90% for core utilities +- **Documentation**: All endpoints documented +- **Breaking Changes**: Documented and migration guide provided +- **Performance**: No significant performance degradation + +--- + +## Appendix + +### Quick Reference + +**Response Helpers:** +```python +from igny8_core.api.response import success_response, error_response, paginated_response +``` + +**Permissions:** +```python +from igny8_core.api.permissions import ( + IsAuthenticatedAndActive, + HasTenantAccess, + IsAdminOrOwner, + IsEditorOrAbove, + IsViewerOrAbove, +) +``` + +**Base ViewSet:** +```python +from igny8_core.api.viewsets import BaseTenantViewSet +``` + +**Throttle Scopes:** +- `ai_function`: 10/min +- `image_gen`: 15/min +- `content_write`: 30/min +- `auth`: 20/min +- `planner`: 60/min +- `writer`: 60/min +- `system`: 100/min +- `billing`: 30/min + +### Related Documents + +- `API-ENDPOINTS-ANALYSIS.md` - Complete endpoint analysis +- `API-IMPLEMENTATION-PLAN-SECTION*.md` - Detailed implementation guides (reference) + +--- + +**Document Status:** Active Standard +**Last Updated:** 2025-01-XX +**Next Review:** Quarterly or as needed + diff --git a/unified-api/sample -singel-doc-for-6-sections.md b/unified-api/sample -singel-doc-for-6-sections.md new file mode 100644 index 00000000..36e840d0 --- /dev/null +++ b/unified-api/sample -singel-doc-for-6-sections.md @@ -0,0 +1,419 @@ +Understood. +Here is the **complete consolidated master document**, combining all 6 sections into a single, clean, unified API Standard for IGNY8. +No questions. No fragmentation. This becomes the **single source of truth** for backend, frontend, plugin, and 3rd-party integration. + +--- + +# **IGNY8 API STANDARDIZATION – MASTER CONSOLIDATED DOCUMENT** + +**Version:** 1.0 +**Purpose:** Establish a single unified API contract for the IGNY8 platform (app, plugin, and external API consumers). +**Applies To:** + +* Django backend +* React frontend +* WordPress plugin +* 3rd-party API consumers +* Internal automations / CRON / AI functions + +--- + +# **1. API Versioning & Structure** + +**Base URL:** + +``` +/api/v1/ +``` + +**Module Namespaces:** + +``` +auth/ +planner/ +writer/ +system/ +billing/ +``` + +All current and future endpoints must follow predictable REST patterns under these namespaces. + +**Versioning Rules:** + +* v1 remains stable long-term +* Breaking changes require v2 +* Deprecations allowed only with explicit timeline + +--- + +# **2. Authentication Standard (Unified)** + +**Primary method:** +`JWT Bearer token` + +``` +Authorization: Bearer +``` + +**Fallbacks:** + +* Session auth (admin panel) +* Basic auth (debug/testing) + +**Token Guarantees:** + +* Contains user_id and account_id +* Resolves account → tenant context automatically +* Sets `request.account` in all ViewSets + +**Public endpoints allowed:** + +* register +* login +* plans +* industries +* system/status + +Everything else requires JWT. + +--- + +# **3. Authorization & Permissions Standard** + +Every endpoint must use consistent layered authorization: + +### **Base layers:** + +1. User must be authenticated +2. User must belong to the tenant/account +3. User must have appropriate role +4. User must have access to requested site/sector + +### **Role hierarchy:** + +``` +owner > admin > editor > viewer > system_bot +``` + +### **Standard permission classes:** + +* IsAuthenticatedAndActive +* HasTenantAccess +* IsViewerOrAbove (read-only) +* IsEditorOrAbove (content operations) +* IsAdminOrOwner (settings, keys, billing) + +All ViewSets must explicitly declare the correct class. + +--- + +# **4. Unified Response Format (Mandatory)** + +This is **the global standard for all endpoints**, no exception. + +### **Success** + +``` +{ + "success": true, + "data": {}, + "message": "Optional readable message" +} +``` + +### **Error** + +``` +{ + "success": false, + "error": "Readable top-level message", + "errors": { "field": ["details"] }, + "request_id": "uuid" +} +``` + +### **Paginated** + +``` +{ + "success": true, + "count": 120, + "next": "...", + "previous": "...", + "results": [...], + "message": "Optional" +} +``` + +This single structure is used across: + +* App +* Plugin +* External API clients +* AI pipeline +* CRON jobs + +--- + +# **5. Unified Error Handling (Global Standard)** + +### **Centralized exception handler requirements:** + +* Wrap all errors in unified format +* Use proper status codes (400, 401, 403, 404, 409, 422, 500) +* Include sanitised validation errors under `errors` +* Always attach `request_id` +* Log full exception details +* In DEBUG mode: include traceback + request context + +### **Server-side Logging** + +* All 4xx logs as warning +* All 5xx logs as error +* Use structured format +* Store in rotating log files +* Provide hooks for Sentry in production + +--- + +# **6. Unified Request Payload Rules** + +All endpoints must follow: + +* snake_case field names +* predictable field groups +* standardized pagination params +* standardized filters +* standardized bulk update patterns +* standardized action payloads +* standardized error messages + +**Bulk actions:** + +``` +{ "ids": [1,2,3], ... } +``` + +**Filtering:** + +``` +?search= +?status= +?site_id= +?sector_id= +``` + +--- + +# **7. Tenant / Site / Sector Scoping Rules** + +Every resource created or fetched must be scoped by: + +1. **Account/Tenant** +2. **Site** +3. **Sector** + +### Enforcement: + +* `AccountModelViewSet` handles account isolation +* `SiteSectorModelViewSet` filters queries by site/sector +* All custom actions must also use `.get_queryset()` to avoid bypassing filters +* Any ID list must be verified to belong to the authenticated tenant + +--- + +# **8. Module-Wise API Rules (Planner / Writer / System / Billing)** + +## **Planner Module** + +Covers keywords → clusters → ideas. + +Rules: + +* All endpoints require authentication +* Auto-cluster limited to 20 keywords/request +* Ideas generation limited to 1 cluster/request +* Standardized payloads for imports/exports +* Bulk actions must validate tenant ownership + +--- + +## **Writer Module** + +Covers tasks → content → image prompts → images. + +Rules: + +* 1 AI content generation per task +* 1 image generation per request +* Bulk actions max 50 +* Image prompt extraction uses unified response +* Must align with progress tracking + +--- + +## **System Module** + +Covers prompts, strategies, profiles, integrations, task progress. + +Rules: + +* Saving prompts requires editor/admin +* Integration settings require admin/owner +* All tests (OpenAI/Runware) return unified format +* Progress endpoint returns stable JSON for progress modal + +--- + +## **Billing Module** + +Covers credit balance, usage logs, transactions. + +Rules: + +* Balance/usage require authenticated tenant +* Transactions require admin/owner +* Unified reporting structure +* No raw model data returned + +--- + +# **9. API Governance, Rate Limits, and Safety Controls** + +### **Soft limits** + +* 50 tasks per bulk action +* 20 keywords per cluster batch +* 1 cluster → ideas generation +* 1 content → image prompt extraction +* 1 image generation per job +* Per-tenant throttling (future) + +### **Safety checks** + +* Validate all IDs belong to tenant +* Validate dependent records exist +* Reject oversized payloads +* Reject invalid site/sector assignments +* Reject stale/wrong tenant tokens + +--- + +# **10. Logging, Monitoring, and Diagnostics** + +### **Backend logging** + +* timestamp +* request_id +* endpoint +* user_id +* account_id +* status_code +* error_message +* traceback for 5xx + +### **Frontend Debug Panel Integration** + +All responses must support: + +* request ID +* readable error messages +* consistent progress steps +* error visibility in real time + +--- + +# **11. Progress Modal Standard (AI Functions)** + +All AI functions must output uniform progress metadata: + +**Steps example for image prompt extraction** + +``` +Smart Image Prompts +Checking content and image slots +Mapping Content for X Image Prompts +Writing Featured Image Prompts +Writing X In-article Image Prompts +Assigning Prompts to Slots +``` + +Success message: + +``` +Featured Image and X In-article Image Prompts ready for image generation +``` + +All four AI functions must follow identical UI messaging structure. + +--- + +# **12. Standard Frontend/Plugin Integration Rules** + +### **Frontend** + +* All responses parsed through the unified envelope +* 401 triggers logout → redirect to login +* 403 triggers permission alert +* Errors read from `.error` or `.errors` +* Pagination reads `.results` + +### **WordPress Plugin** + +* Uses the same JWT flow +* No custom plugin-specific endpoints +* Uses the app’s `/api/v1/` fully +* WP only handles mapping + meta assignment + +### **3rd-Party APIs** + +* Full compatibility with unified format +* Full access where tenant/permissions allow +* No special handling needed + +--- + +# **13. Refactor & Migration Plan (One-Time Update)** + +1. Introduce unified response wrapper +2. Replace all custom responses in ViewSets +3. Inject consistent permission classes +4. Wrap all exceptions with centralized handler +5. Update serializers to rely on unified errors +6. Patch all custom actions to validate payloads +7. Update app’s frontend API client +8. Remove deprecated endpoint patterns +9. Regenerate Postman/OpenAPI schema +10. Update WordPress plugin integration +11. Run full-suite regression tests + +This whole migration becomes the Phase-4 API refactor inside the IGNY8 full migration pipeline. + +--- + +# **14. Success Criteria** + +* Every endpoint returns same envelope +* All errors fully unified +* No endpoint leaks raw DRF format +* No inconsistent auth behavior +* No inconsistent pagination +* No inconsistent validation errors +* Plugin works using same exact API +* Third-party clients can implement without special rules +* Frontend debug panel reads everything correctly +* CRON + automations structured identically + +--- + +# **15. Final Notes** + +This consolidated document **replaces all six API planning files**. +It is now the single standard for: + +* Backend implementation +* Frontend integration +* Plugin development +* External API docs +* AI function progress UI +* DevOps & monitoring