diff --git a/CHANGELOG.md b/CHANGELOG.md index d8b9b3fb..69f5cce9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,9 @@ -# IGNY8 Changelog +# Changelog -**Current Version:** `1.0.0` -**Last Updated:** 2025-01-XX -**Purpose:** Complete changelog of all changes, fixes, and features. Only updated after user confirmation. +All notable changes to the IGNY8 platform will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). --- @@ -10,8 +11,6 @@ **IMPORTANT**: This changelog is only updated after user confirmation that a fix or feature is complete and working. -**For AI Agents**: Read `docs/00-DOCUMENTATION-MANAGEMENT.md` before making any changes to this file. - ### Changelog Structure Each entry follows this format: @@ -20,579 +19,340 @@ Each entry follows this format: - **Type**: Added, Changed, Fixed, Deprecated, Removed, Security - **Description**: Clear description of the change - **Affected Areas**: Modules, components, or features affected -- **Documentation**: Reference to updated documentation files --- -## [Unreleased] - -### Added -- **Planning Documents Organization**: Organized architecture and implementation planning documents - - Created `docs/planning/` directory for all planning documents - - Moved `IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md` to `docs/planning/` - - Moved `IGNY8-IMPLEMENTATION-PLAN.md` to `docs/planning/` - - Moved `Igny8-phase-2-plan.md` to `docs/planning/` - - Moved `CONTENT-WORKFLOW-DIAGRAM.md` to `docs/planning/` - - Moved `ARCHITECTURE_CONTEXT.md` to `docs/planning/` - - Moved `sample-usage-limits-credit-system` to `docs/planning/` - - Created `docs/refactor/` directory for refactoring plans - - Updated `README.md` to reflect new document structure - - **Impact**: Better organization of planning documents, easier to find and maintain - -### Changed -- **API Documentation Consolidation**: Consolidated all API documentation into single comprehensive reference - - Created `docs/API-COMPLETE-REFERENCE.md` - Unified API documentation covering all endpoints, authentication, response formats, error handling, rate limiting, permissions, and integration examples - - Removed redundant documentation files: - - `docs/API-DOCUMENTATION.md` (consolidated into complete reference) - - `docs/DOCUMENTATION-SUMMARY.md` (consolidated into complete reference) - - `unified-api/API-ENDPOINTS-ANALYSIS.md` (consolidated into complete reference) - - `unified-api/API-STANDARD-v1.0.md` (consolidated into complete reference) - - New unified document includes: complete endpoint reference, authentication guide, response format standards, error handling, rate limiting, pagination, roles & permissions, tenant/site/sector scoping, integration examples (Python, JavaScript, cURL, PHP), testing & debugging, and change management - - **Impact**: Single source of truth for all API documentation, easier to maintain and navigate - -### Added -- Unified API Standard v1.0 implementation -- API Monitor page for endpoint health monitoring -- CRUD operations monitoring for Planner and Writer modules -- Sidebar API status indicator for aws-admin accounts -- **Health Check Endpoint**: `GET /api/v1/system/ping/` - Public health check endpoint per API Standard v1.0 requirement - - Returns unified format: `{success: true, data: {status: 'ok'}}` - - Tagged as 'System' in Swagger/ReDoc documentation - - Public endpoint (AllowAny permission) - -### Changed -- All API endpoints now return unified response format (`{success, data, message, errors}`) -- Frontend `fetchAPI` wrapper automatically extracts data from unified format -- All error responses follow unified format with `request_id` tracking -- Rate limiting configured with scoped throttles per module -- **Integration Views**: All integration endpoints now use unified response format - - Replaced 40+ raw `Response()` calls with `success_response()`/`error_response()` helpers - - All responses include `request_id` for tracking - - Updated frontend components to handle extracted data format -- **API Documentation**: Updated Swagger/ReDoc description to include all public endpoints - - Added `/api/v1/system/ping/` to public endpoints list - - Updated schema extensions to properly tag ping endpoint -- **AI Framework Refactoring**: Removed hardcoded model defaults, IntegrationSettings is now the single source of truth - - Removed `MODEL_CONFIG` dictionary with hardcoded defaults - - Removed Django settings `DEFAULT_AI_MODEL` fallback - - `get_model_config()` now requires `account` parameter and raises clear errors if IntegrationSettings not configured - - All AI functions now require account-specific model configuration - - Removed orphan code: `get_model()`, `get_max_tokens()`, `get_temperature()` helper functions - - Removed unused exports from `__init__.py`: `register_function`, `list_functions`, `get_model`, `get_max_tokens`, `get_temperature` - - **Impact**: Each account must configure their own AI models in IntegrationSettings - - **Documentation**: See `backend/igny8_core/ai/REFACTORING-IMPLEMENTED.md` for complete details - -### Fixed -- Keyword edit form now correctly populates existing values -- Auto-cluster function now works correctly with unified API format -- ResourceDebugOverlay now correctly extracts data from unified API responses -- All frontend pages now correctly handle unified API response format -- **Integration Views**: Fixed all integration endpoints not using unified response format - - `_test_openai()` and `_test_runware()` methods now use unified format - - `generate_image()`, `create()`, `save_settings()` methods now use unified format - - `get_image_generation_settings()` and `task_progress()` methods now use unified format - - All error responses now include `request_id` and follow unified format - - Fixed OpenAI integration endpoint error handling - invalid API keys now return 400 (Bad Request) instead of 401 (Unauthorized) -- **Frontend Components**: Updated to work with unified format - - `ValidationCard.tsx` - Removed dual-format handling, now works with extracted data - - `Integration.tsx` - Simplified to work with unified format - - `ImageGenerationCard.tsx` - Updated to work with extracted data format -- **Frontend Authentication**: Fixed `getAuthToken is not defined` error in `authStore.ts` - - Updated `refreshUser()` to use `fetchAPI()` instead of manual fetch with `getAuthToken()` - - Removed error throwing from catch block to prevent error accumulation -- **Frontend Error Handling**: Fixed console error accumulation - - `ResourceDebugOverlay.tsx` now silently ignores 404 errors for request-metrics endpoint - - Removed error throwing from `refreshUser()` catch block to prevent error spam -- **AI Framework Error Handling**: Improved error messages and exception handling - - `AIEngine._handle_error()` now preserves exception types for better error messages - - All AI function errors now include proper `error_type` (ConfigurationError, AccountNotFound, etc.) - - Fixed "Task failed - exception details unavailable" by improving error type preservation - - Error messages now clearly indicate when IntegrationSettings are missing or misconfigured - ---- - -## [1.1.1] - 2025-01-XX - -### Security -- **CRITICAL**: Fixed `AIPromptViewSet` security vulnerability - changed from `permission_classes = []` (allowing unauthenticated access) to `IsAuthenticatedAndActive + HasTenantAccess` -- Added `IsEditorOrAbove` permission check for `save_prompt` and `reset_prompt` actions in `AIPromptViewSet` -- All billing ViewSets now require `IsAuthenticatedAndActive + HasTenantAccess` for proper tenant isolation -- `CreditTransactionViewSet` now requires `IsAdminOrOwner` per API Standard v1.0 (billing/transactions require admin/owner) -- All system settings ViewSets now use standard permissions (`IsAuthenticatedAndActive + HasTenantAccess`) -- All auth ViewSets now explicitly include `IsAuthenticatedAndActive + HasTenantAccess` for proper tenant isolation - -### Changed -- **Auth Endpoints**: All authentication endpoints (`RegisterView`, `LoginView`, `ChangePasswordView`, `MeView`) now use unified response format with `success_response()` and `error_response()` helpers - - All responses now include `request_id` for error tracking - - Error responses follow unified format with `error` and `errors` fields - - Success responses follow unified format with `success`, `data`, and `message` fields -- **Billing Module**: Refactored `CreditUsageViewSet` and `CreditTransactionViewSet` to inherit from `AccountModelViewSet` instead of manual account filtering - - Account filtering now handled automatically by base class - - Improved code maintainability and consistency -- **System Settings**: All 5 system settings ViewSets now use standard permission classes - - `SystemSettingsViewSet`, `AccountSettingsViewSet`, `UserSettingsViewSet`, `ModuleSettingsViewSet`, `AISettingsViewSet` - - Write operations require `IsAdminOrOwner` per standard -- **Integration Settings**: Added `HasTenantAccess` permission to `IntegrationSettingsViewSet` for proper tenant isolation -- **Auth ViewSets**: Added explicit standard permissions to all auth ViewSets - - `UsersViewSet`, `AccountsViewSet`, `SubscriptionsViewSet`, `SiteUserAccessViewSet` now include `IsAuthenticatedAndActive + HasTenantAccess` - - `SiteViewSet`, `SectorViewSet` now include `IsAuthenticatedAndActive + HasTenantAccess` - -### Fixed -- Fixed auth endpoints not returning unified format (were using raw `Response()` instead of helpers) -- Fixed missing `request_id` in auth endpoint responses -- Fixed inconsistent error response format in auth endpoints -- Fixed billing ViewSets not using base classes (manual account filtering replaced with `AccountModelViewSet`) -- Fixed all ViewSets missing standard permissions (`IsAuthenticatedAndActive + HasTenantAccess`) - -### Documentation -- Updated implementation plan to reflect completion of all remaining API Standard v1.0 items -- All 8 remaining items from audit completed (100% compliance achieved) -- **API Standard v1.0**: Full compliance achieved - - All 10 audit tasks completed and verified - - All custom @action methods use unified response format - - All ViewSets use proper base classes, pagination, throttles, and permissions - - All error responses include `request_id` tracking - - No raw `Response()` calls remaining (except file downloads) - - All endpoints documented in Swagger/ReDoc with proper tags - ---- - -## [1.1.0] - 2025-01-XX +## [1.0.0] - 2025-11-24 ### Added -#### Unified API Standard v1.0 -- **Response Format Standardization** - - All endpoints return unified format: `{success: true/false, data: {...}, message: "...", errors: {...}}` - - Paginated responses include `success`, `count`, `next`, `previous`, `results` - - Error responses include `success: false`, `error`, `errors`, `request_id` - - Response helper functions: `success_response()`, `error_response()`, `paginated_response()` +#### Backend Core +- **Multi-Tenancy System** + - Complete account isolation with Account → Site → Sector hierarchy + - User roles: Developer, Owner, Admin, Editor, Viewer, System Bot + - Automatic account filtering via `AccountContextMiddleware` + - Base classes: `AccountModelViewSet` and `SiteSectorModelViewSet` + - Account isolation at database level with ForeignKey relationships -- **Custom Exception Handler** - - Centralized exception handling in `backend/igny8_core/api/exception_handlers.py` - - All exceptions wrapped in unified format - - Proper HTTP status code mapping (400, 401, 403, 404, 409, 422, 429, 500) - - Debug information included in development mode +- **Authentication & Authorization** + - JWT token authentication with PyJWT 2.8+ + - 15-minute access token expiry + - Refresh token support + - API key authentication for WordPress integration + - Permission classes: `IsAuthenticatedAndActive`, `HasTenantAccess`, `IsEditorOrAbove`, `IsAdminOrOwner` + - Role-based access control matrix -- **Custom Pagination** - - `CustomPageNumberPagination` class with unified format support - - Default page size: 10, max: 100 - - Dynamic page size via `page_size` query parameter - - Includes `success` field in paginated responses +- **Planner Module** (`modules/planner/`) + - Keywords model with search volume, difficulty, intent tracking + - Clusters model with many-to-many keyword relationships + - ContentIdeas model linked to clusters + - AI-powered keyword clustering via `auto_cluster` function + - AI-powered content idea generation via `generate_ideas` function + - Bulk keyword import (CSV and manual) + - ViewSets: `KeywordViewSet`, `ClusterViewSet`, `ContentIdeasViewSet` -- **Base ViewSets** - - `AccountModelViewSet` - Handles account isolation and unified CRUD responses - - `SiteSectorModelViewSet` - Extends account isolation with site/sector filtering - - All CRUD operations (create, retrieve, update, destroy) return unified format +- **Writer Module** (`modules/writer/`) + - Tasks model for content creation workflow + - Content model with HTML/plain text support and unified taxonomy structure + - Images model with featured/in-article image support + - AI content generation via GPT-4 (`generate_content` function) + - AI image generation via DALL-E 3 and Runware (`generate_images` function) + - Content taxonomies (`ContentTaxonomy`) and attributes (unified JSON structure) + - ViewSets: `TasksViewSet`, `ContentViewSet`, `ImagesViewSet`, `ContentTaxonomyViewSet`, `ContentAttributeViewSet` -- **Rate Limiting** - - `DebugScopedRateThrottle` with debug bypass for development - - Scoped rate limits per module (planner, writer, system, billing, auth) - - AI function rate limits (10/min for expensive operations) - - Bypass for aws-admin accounts and admin/developer roles - - Rate limit headers: `X-Throttle-Limit`, `X-Throttle-Remaining`, `X-Throttle-Reset` +- **Linker Module** (`modules/linker/`) + - Internal linking recommendations + - Link graph analysis + - `LinkerViewSet` for link operations -- **Request ID Tracking** - - `RequestIDMiddleware` generates unique UUID for each request - - Request ID included in all error responses - - Request ID in response headers: `X-Request-ID` - - Used for log correlation and debugging +- **Optimizer Module** (`modules/optimizer/`) + - Content quality scoring + - SEO optimization suggestions + - `OptimizerViewSet` for optimization operations -- **API Monitor** - - New page: `/settings/api-monitor` for endpoint health monitoring - - Monitors API status (HTTP response) and data status (page population) - - Endpoint groups: Core Health, Auth, Planner, Writer, System, Billing, CRUD Operations - - Sorting by status (errors first, then warnings, then healthy) - - Real-time endpoint health checks with configurable refresh interval - - Only accessible to aws-admin accounts +- **Publisher Module** (`modules/publisher/`) + - Publishing records tracking (`PublishingRecord` model) + - Deployment records (`DeploymentRecord` model) + - `PublisherViewSet`, `PublishingRecordViewSet`, `DeploymentRecordViewSet` -- **Sidebar API Status Indicator** - - Visual indicator circles for each endpoint group - - Color-coded status (green = healthy, yellow = warning) - - Abbreviations: CO, AU, PM, WM, PC, WC, SY - - Only visible and active for aws-admin accounts on API monitor page - - Prevents console errors on other pages +- **Site Builder Module** (`modules/site_builder/`) + - Site blueprint management (`SiteBlueprint` model) + - Page blueprint creation (`PageBlueprint` model) + - Site structure planning with clusters and taxonomies + - `SiteBlueprintViewSet`, `PageBlueprintViewSet`, `SiteAssetView`, `SiteBuilderMetadataView` -### Changed +- **Automation Module** (`modules/automation/`) + - Automation rules engine (`AutomationRule` model) + - Scheduled tasks (`ScheduledTask` model) + - Rule execution via Celery beat + - `AutomationRuleViewSet`, `ScheduledTaskViewSet` -#### Backend Refactoring -- **Planner Module** - All ViewSets refactored to unified format - - `KeywordViewSet` - CRUD + `auto_cluster` action - - `ClusterViewSet` - CRUD + `auto_generate_ideas` action - - `ContentIdeasViewSet` - CRUD + `bulk_queue_to_writer` action +- **Integration Module** (`modules/integration/`) + - WordPress integration support via `SiteIntegration` model + - Connection testing: `POST /integrations/test-connection/` + - Structure update: `POST /integrations/{id}/update-structure/` + - Content type summary: `GET /integrations/{id}/content-types/` + - Bidirectional sync support with sync status and logs + - Integration services: `IntegrationService`, `SyncService`, `ContentSyncService`, `SyncHealthService` + - Site-level sync endpoints: `/sites/{id}/sync/status/`, `/sites/{id}/sync/run/` + - `IntegrationViewSet` with comprehensive sync management -- **Writer Module** - All ViewSets refactored to unified format - - `TasksViewSet` - CRUD + `auto_generate_content` action - - `ContentViewSet` - CRUD + `generate_image_prompts` action - - `ImagesViewSet` - CRUD + `generate_images` action +- **System Module** (`modules/system/`) + - AI prompt management (`AIPrompt` model with types: cluster, ideas, content, image_prompts) + - Integration settings (`IntegrationSettings` model for OpenAI/Runware config) + - Author profiles for content style (`AuthorProfile` model) + - Content strategies (`Strategy` model) + - System and account settings endpoints + - ViewSets: `AIPromptViewSet`, `AuthorProfileViewSet`, `StrategyViewSet` + - Settings ViewSets: `SystemSettingsViewSet`, `AccountSettingsViewSet`, `UserSettingsViewSet`, `ModuleSettingsViewSet`, `AISettingsViewSet` -- **System Module** - All ViewSets refactored to unified format - - `AIPromptViewSet` - CRUD + `get_by_type`, `save_prompt`, `reset_prompt` actions - - `SystemSettingsViewSet`, `AccountSettingsViewSet`, `UserSettingsViewSet` - - `ModuleSettingsViewSet`, `AISettingsViewSet` - - `IntegrationSettingsViewSet` - Integration management and testing +- **Billing Module** (`modules/billing/`) + - Credit-based usage system + - Credit transactions tracking (`CreditTransaction` model) + - Usage analytics (`CreditUsageLog` model) + - `CreditService` for credit operations + - Monthly credit replenishment via Celery beat task + - ViewSets: `CreditBalanceViewSet`, `CreditUsageViewSet`, `CreditTransactionViewSet` -- **Billing Module** - All ViewSets refactored to unified format - - `CreditBalanceViewSet` - `balance` action - - `CreditUsageViewSet` - `summary`, `limits` actions - - `CreditTransactionViewSet` - CRUD operations +- **AI Framework** (`ai/`) + - `AIEngine` - Central orchestrator for all AI operations with progress tracking + - `AICore` - Unified API client for OpenAI/Runware with rate calculation + - `BaseAIFunction` - Abstract interface for AI function implementations + - AI function registry system with 5 core functions + - Progress tracking with Celery task state updates + - Cost tracking (token usage and pricing) + - Console step tracker for debugging (`ConsoleStepTracker`) + - Prompt registry system (`PromptRegistry`) + - Model configuration per account via `IntegrationSettings` + - Validators for payload validation + - Celery task wrapper: `run_ai_task` -- **Auth Module** - All ViewSets refactored to unified format - - `AuthViewSet` - `register`, `login`, `change_password`, `refresh_token`, `reset_password` - - `UsersViewSet` - CRUD + `create_user`, `update_role` actions - - `GroupsViewSet`, `AccountsViewSet`, `SubscriptionsViewSet` - - `SiteUserAccessViewSet`, `PlanViewSet`, `IndustryViewSet`, `SeedKeywordViewSet` +- **AI Functions** (`ai/functions/`) + - `auto_cluster.py` - Semantic keyword clustering + - `generate_ideas.py` - Content idea generation from clusters + - `generate_content.py` - Full blog post generation with HTML + - `generate_image_prompts.py` - Extract image prompts from content + - `generate_images.py` - Image generation via DALL-E 3 / Runware -#### Frontend Refactoring -- **fetchAPI Wrapper** (`frontend/src/services/api.ts`) - - Automatically extracts `data` field from unified format responses - - Handles paginated responses (`results` at top level) - - Properly throws errors for `success: false` responses - - Removed redundant `response?.data || response` checks across codebase +- **API Infrastructure** (`api/`) + - Unified response format: `{success, data, message, request_id}` + - Response helpers: `success_response()`, `error_response()`, `paginated_response()` + - Request ID tracking via `RequestIDMiddleware` + - Resource tracking via `ResourceTrackingMiddleware` + - Rate limiting with `DebugScopedRateThrottle` (bypass in DEBUG mode) + - Custom pagination: `CustomPageNumberPagination` (default: 10, max: 100) + - OpenAPI 3.0 schema via drf-spectacular + - Swagger UI at `/api/docs/` + - ReDoc at `/api/redoc/` + - Centralized exception handling with unified error format -- **All Frontend Pages Updated** - - Removed redundant response data extraction - - All pages now correctly consume unified API format - - Error handling standardized across all components - - Pagination handling standardized +- **Celery Integration** (`celery.py`) + - Async task processing for AI operations + - Celery beat for scheduled tasks + - Beat schedule: monthly credit replenishment, automation rule execution (every 5 min) + - Task autodiscovery from all installed apps -- **Component Updates** - - `FormModal` - Now accepts `React.ReactNode` for title prop - - `ComponentCard` - Updated to support status badges in titles - - `ResourceDebugOverlay` - Fixed to extract data from unified format - - `ApiStatusIndicator` - Restricted to aws-admin accounts and API monitor page +#### Frontend Core +- **React 19 Application** (`frontend/`) + - Vite 6.1.0 build system with fast HMR + - TypeScript 5.7.2 for type safety + - React Router v7.9.5 for client-side routing + - Zustand 5.0.8 state management with localStorage persistence + - Lazy loading for all module pages (code splitting) + - Tailwind CSS 4.0.8 for styling -### Fixed +- **Authentication** (`pages/AuthPages/`, `store/authStore.ts`) + - JWT-based authentication with automatic token refresh + - Login/signup pages + - Protected routes with `ProtectedRoute` component + - Auth store with Zustand (user, token, refreshToken state) + - Auto token refresh on 401 errors -#### Bug Fixes -- **Keyword Edit Form** - Now correctly populates existing values when editing - - Added `key` prop to force re-render when form data changes - - Fixed `seed_keyword_id` value handling for select dropdown +- **Module Pages** + - **Planner** (`pages/Planner/`): Keywords list, Clusters view, Ideas dashboard, Keyword opportunities + - **Writer** (`pages/Writer/`): Tasks dashboard, Content viewer with HTML preview, Drafts, Published list, Images gallery + - **Linker** (`pages/Linker/`): Link analysis dashboard, Content list with link suggestions + - **Optimizer** (`pages/Optimizer/`): Optimization dashboard, Content selector, Analysis preview + - **Thinker** (`pages/Thinker/`): AI prompts editor, Author profiles, Strategies, Image testing + - **Billing** (`pages/Billing/`): Credit balance, Transaction history, Usage analytics + - **Automation** (`pages/Automation/`): Rules management, Scheduled tasks list + - **Settings** (`pages/Settings/`): General, Users, Subscriptions, System, AI settings, API monitor -- **Auto-Cluster Function** - Now works correctly with unified API format - - Updated `autoClusterKeywords()` to wrap response with `success` field - - Proper error handling and response extraction +- **UI Components** (`components/`) + - Reusable components library + - ApexCharts for data visualization + - Heroicons for icons + - Responsive layouts with Tailwind + - Form components with validation + - Modal system + - Loading states and skeletons -- **ResourceDebugOverlay** - Fixed data extraction from unified API responses - - Extracts `data` field from `{success: true, data: {...}}` responses - - Added null safety checks for all property accesses - - Validates data structure before adding to metrics +- **API Integration** (`api/`, `services/`) + - Axios client with automatic token injection + - Request/response interceptors + - Automatic token refresh on 401 + - Module-specific API clients (planner, writer, linker, etc.) + - `fetchAPI` wrapper with unified format handling + - Error handling with toast notifications -- **API Response Handling** - Fixed all instances of incorrect data extraction - - Removed `response?.data || response` redundant checks - - Removed `response.results || []` redundant checks - - All API functions now correctly handle unified format +- **State Management** (`store/`) + - `authStore` - User authentication state + - `siteStore` - Site and sector context + - Module-specific stores (plannerStore, writerStore, etc.) + - Global app state (loading, errors, notifications) + - Zustand persist middleware for localStorage -- **React Hooks Error** - Fixed "Rendered more hooks than during the previous render" - - Moved all hooks to top of component before conditional returns - - Fixed `ApiStatusIndicator` component hook ordering +#### WordPress Plugin +- **Core Plugin** (`igny8-wp-integration/`) + - Plugin version 1.0.0 + - WordPress 5.0+ compatibility + - PHP 7.4+ requirement + - Main file: `igny8-bridge.php` + - Uninstall cleanup: `uninstall.php` -- **TypeScript Errors** - Fixed all type errors related to unified API format - - Added nullish coalescing for `toLocaleString()` calls - - Added null checks before `Object.entries()` calls - - Fixed all undefined property access errors +- **API Client** (`includes/class-igny8-api.php`) + - `Igny8API` class for backend communication + - API key authentication (primary method) + - JWT token authentication support (fallback) + - Automatic retry on 401 errors + - Request/response parsing with unified format support + - Methods: `connect()`, `get()`, `post()`, `put()`, `delete()`, `parse_response()` -#### System Health -- **System Status Page** - Fixed redundant data extraction - - Now correctly uses extracted data from `fetchAPI` - - All system metrics display correctly +- **REST API Endpoints** (`includes/class-igny8-rest-api.php`) + - `GET /wp-json/igny8/v1/site-metadata/` - Export site structure + - `GET /wp-json/igny8/v1/post-by-task-id/{id}` - Lookup by task ID + - `GET /wp-json/igny8/v1/post-by-content-id/{id}` - Lookup by content ID + - `POST /wp-json/igny8/v1/posts/` - Create post from IGNY8 + - `PUT /wp-json/igny8/v1/posts/{id}` - Update post from IGNY8 + - API key authentication via `X-IGNY8-API-Key` header -### Security -- Rate limiting bypass only for aws-admin accounts and admin/developer roles -- Request ID tracking for all API requests -- Centralized error handling prevents information leakage +- **Synchronization** (`sync/`) + - **WP → IGNY8** (`post-sync.php`): Post status sync on `save_post` hook + - **IGNY8 → WP** (`igny8-to-wp.php`): Content publishing via REST API + - **Taxonomy Sync** (`taxonomy-sync.php`): Taxonomy term synchronization + - **Hooks** (`hooks.php`): WordPress action/filter hook registration + - Bidirectional sync with status mapping + - Post meta tracking: `_igny8_task_id`, `_igny8_content_id`, `_igny8_wordpress_status`, `_igny8_last_synced`, `_igny8_managed` -### Testing +- **Admin Interface** (`admin/`) + - Settings page at Settings → IGNY8 API (`settings.php`) + - API key, email, password configuration + - Connection testing button + - Post columns showing IGNY8 status (`class-admin-columns.php`) + - Post meta boxes in editor (`class-post-meta-boxes.php`) + - Admin class: `Igny8Admin` (`class-admin.php`) -- **Comprehensive Test Suite** - - Created complete unit and integration test suite for Unified API Standard v1.0 - - 13 test files with ~115 test methods covering all API components - - Test coverage: 100% of API Standard components +- **Data Collection** (`data/`) + - Full site metadata collection (`site-collection.php`) + - Post type discovery with counts + - Taxonomy discovery with counts + - Link graph extraction (`link-graph.php`) + - Semantic mapping (`semantic-mapping.php`) + - WooCommerce integration support (`woocommerce.php`) -- **Unit Tests** (`backend/igny8_core/api/tests/`) - - `test_response.py` - Tests for response helper functions (18 tests) - - Tests `success_response()`, `error_response()`, `paginated_response()` - - Tests request ID generation and inclusion - - Tests status code mapping and error messages - - `test_exception_handler.py` - Tests for custom exception handler (12 tests) - - Tests all exception types (ValidationError, AuthenticationFailed, PermissionDenied, NotFound, Throttled, etc.) - - Tests debug mode behavior and debug info inclusion - - Tests field-specific and non-field error handling - - `test_permissions.py` - Tests for permission classes (20 tests) - - Tests `IsAuthenticatedAndActive`, `HasTenantAccess`, `IsViewerOrAbove`, `IsEditorOrAbove`, `IsAdminOrOwner` - - Tests role-based access control and tenant isolation - - Tests admin/system account bypass logic - - `test_throttles.py` - Tests for rate limiting (11 tests) - - Tests `DebugScopedRateThrottle` bypass logic (DEBUG mode, env flag, admin/system accounts) - - Tests rate parsing and throttle header generation - -- **Integration Tests** (`backend/igny8_core/api/tests/`) - - `test_integration_base.py` - Base test class with common fixtures and helper methods - - `test_integration_planner.py` - Planner module endpoint tests (12 tests) - - Tests CRUD operations for keywords, clusters, ideas - - Tests AI actions (auto_cluster) - - Tests error scenarios and validation - - `test_integration_writer.py` - Writer module endpoint tests (6 tests) - - Tests CRUD operations for tasks, content, images - - Tests error scenarios - - `test_integration_system.py` - System module endpoint tests (5 tests) - - Tests status, prompts, settings, integrations endpoints - - `test_integration_billing.py` - Billing module endpoint tests (5 tests) - - Tests credits, usage, transactions endpoints - - `test_integration_auth.py` - Auth module endpoint tests (8 tests) - - Tests login, register, user management endpoints - - Tests authentication flows and error scenarios - - `test_integration_errors.py` - Error scenario tests (6 tests) - - Tests 400, 401, 403, 404, 429, 500 error responses - - Tests unified error format across all error types - - `test_integration_pagination.py` - Pagination tests (10 tests) - - Tests pagination across all modules - - Tests page size, page parameter, max page size limits - - Tests empty results handling - - `test_integration_rate_limiting.py` - Rate limiting integration tests (7 tests) - - Tests throttle headers presence - - Tests bypass logic for admin/system accounts and DEBUG mode - - Tests different throttle scopes per module - -- **Test Verification** - - All tests verify unified response format (`{success, data/results, message, errors, request_id}`) - - All tests verify proper HTTP status codes - - All tests verify error format consistency - - All tests verify pagination format consistency - - All tests verify request ID inclusion - -- **Test Documentation** - - Created `backend/igny8_core/api/tests/README.md` with test structure and running instructions - - Created `backend/igny8_core/api/tests/TEST_SUMMARY.md` with comprehensive test statistics - - Created `backend/igny8_core/api/tests/run_tests.py` test runner script - -### Documentation - -- **OpenAPI/Swagger Integration** - - Installed and configured `drf-spectacular` for OpenAPI 3.0 schema generation - - Created Swagger UI endpoint: `/api/docs/` - - Created ReDoc endpoint: `/api/redoc/` - - Created OpenAPI schema endpoint: `/api/schema/` - - Configured comprehensive API documentation with code samples - - Added custom authentication extensions for JWT Bearer tokens - -- **Comprehensive Documentation Files** - - `docs/API-COMPLETE-REFERENCE.md` - Complete unified API reference (consolidated from multiple files) - - Quick start guide - - Endpoint reference - - Code examples (Python, JavaScript, cURL) - - Response format details - - `docs/AUTHENTICATION-GUIDE.md` - Authentication and authorization guide - - JWT Bearer token authentication - - Token management and refresh - - Code examples in Python and JavaScript - - Security best practices - - `docs/ERROR-CODES.md` - Complete error code reference - - HTTP status codes (200, 201, 400, 401, 403, 404, 409, 422, 429, 500) - - Field-specific error messages - - Error handling best practices - - Common error scenarios and solutions - - `docs/RATE-LIMITING.md` - Rate limiting and throttling guide - - Rate limit scopes and limits - - Handling rate limits (429 responses) - - Best practices and code examples - - Request queuing and caching strategies - - `docs/MIGRATION-GUIDE.md` - Migration guide for API consumers - - What changed in v1.0 - - Step-by-step migration instructions - - Code examples (before/after) - - Breaking and non-breaking changes - - `docs/WORDPRESS-PLUGIN-INTEGRATION.md` - WordPress plugin integration guide - - Complete PHP API client class - - Authentication implementation - - Error handling - - WordPress admin integration - - Best practices - - `docs/README.md` - Documentation index and quick start - -- **OpenAPI Schema Configuration** - - Configured comprehensive API description with features overview - - Added authentication documentation - - Added response format examples - - Added rate limiting documentation - - Added pagination documentation - - Configured endpoint tags (Authentication, Planner, Writer, System, Billing) - - Added code samples in Python and JavaScript - -- **Schema Extensions** - - Created `backend/igny8_core/api/schema_extensions.py` for custom authentication - - JWT Bearer token authentication extension - - CSRF-exempt session authentication extension - - Proper OpenAPI security scheme definitions - ---- - -## [1.0.0] - 2025-01-XX - -### Added - -#### Documentation System -- Complete documentation structure with 7 core documents -- Documentation management system with versioning -- Changelog management system -- DRY principles documentation -- Self-explaining documentation for AI agents - -#### Core Features -- Multi-tenancy system with account isolation -- Authentication (login/register) with JWT -- RBAC permissions (Developer, Owner, Admin, Editor, Viewer, System Bot) -- Account > Site > Sector hierarchy -- Multiple sites can be active simultaneously -- Maximum 5 active sectors per site - -#### Planner Module -- Keywords CRUD operations -- Keyword import/export (CSV) -- Keyword filtering and organization -- AI-powered keyword clustering -- Clusters CRUD operations -- Content ideas generation from clusters -- Content ideas CRUD operations -- Keyword-to-cluster mapping -- Cluster metrics and analytics - -#### Writer Module -- Tasks CRUD operations -- AI-powered content generation -- Content editing and review -- Image prompt extraction -- AI-powered image generation (OpenAI DALL-E, Runware) -- Image management -- WordPress integration (publishing) - -#### Thinker Module -- AI prompt management -- Author profile management -- Content strategy management -- Image generation testing - -#### System Module -- Integration settings (OpenAI, Runware) -- API key configuration -- Connection testing -- System status and monitoring - -#### Billing Module -- Credit balance tracking -- Credit transactions -- Usage logging -- Cost tracking - -#### Frontend -- Configuration-driven UI system -- 4 universal templates (Dashboard, Table, Form, System) -- Complete component library -- Zustand state management -- React Router v7 routing -- Progress tracking for AI tasks -- Responsive design - -#### Backend -- RESTful API with DRF -- Automatic account isolation -- Site access control -- Celery async task processing -- Progress tracking for Celery tasks -- Unified AI framework -- Database logging - -#### AI Functions -- Auto Cluster Keywords -- Generate Ideas -- Generate Content -- Generate Image Prompts -- Generate Images -- Test OpenAI connection -- Test Runware connection -- Test image generation +- **Supporting Classes** (`includes/`) + - `Igny8Site` - Site data collection and export + - `Igny8Webhooks` - Webhook handlers (legacy) + - `Igny8WebhookLogs` - Webhook logging + - `Igny8LinkQueue` - Link processing queue + - Helper functions in `functions.php` #### Infrastructure -- Docker-based containerization -- Two-stack architecture (infra, app) -- Caddy reverse proxy -- PostgreSQL database -- Redis cache and Celery broker -- pgAdmin database administration -- FileBrowser file management +- **Docker Deployment** (`docker-compose.app.yml`) + - Multi-container setup for development + - Services: PostgreSQL, Redis, Django backend, Celery worker, Celery beat + - Volume persistence for database and media + - Network isolation + +- **Database** (PostgreSQL 14+) + - Multi-tenant data model with account isolation + - Optimized indexes for common queries + - JSON fields for flexible configuration storage + - Database migrations system + +- **Task Queue** (Celery + Redis) + - Redis as message broker + - Celery worker for async tasks + - Celery beat for scheduled tasks + - Progress tracking with task state updates ### Documentation +- **Master Documentation** (`master-docs/`) + - Complete architecture documentation + - API reference with endpoint details + - Frontend and backend implementation guides + - WordPress plugin integration guide -#### Documentation Files Created -- `docs/00-DOCUMENTATION-MANAGEMENT.md` - Documentation and changelog management system -- `docs/01-TECH-STACK-AND-INFRASTRUCTURE.md` - Technology stack and infrastructure -- `docs/02-APPLICATION-ARCHITECTURE.md` - Application architecture with workflows -- `docs/03-FRONTEND-ARCHITECTURE.md` - Frontend architecture documentation -- `docs/04-BACKEND-IMPLEMENTATION.md` - Backend implementation reference -- `docs/05-AI-FRAMEWORK-IMPLEMENTATION.md` - AI framework implementation reference -- `docs/06-FUNCTIONAL-BUSINESS-LOGIC.md` - Functional business logic documentation +- **Project Documentation** + - `MASTER_REFERENCE.md` - Complete system architecture and navigation + - `README.md` - Quick start and setup guide + - `CHANGELOG.md` - This file -#### Documentation Features -- Complete workflow documentation -- Feature completeness -- No code snippets (workflow-focused) -- Accurate state reflection -- Cross-referenced documents -- Self-explaining structure for AI agents +### Changed +- N/A (Initial release) + +### Deprecated +- N/A (Initial release) + +### Removed +- N/A (Initial release) + +### Fixed +- N/A (Initial release) + +### Security +- JWT token authentication with 15-minute expiry and refresh tokens +- Secure API key storage in WordPress using encryption helpers +- CSRF protection on all forms +- SQL injection prevention via Django ORM +- XSS protection via Django template escaping +- HTTPS enforcement in production +- Account isolation at database level with automatic filtering +- Permission-based access control with role matrix +- Rate limiting to prevent abuse +- Request ID tracking for audit trails --- ## Version History -### Current Version: 1.0.0 - -**Status**: Production -**Date**: 2025-01-XX - -### Version Format - -- **MAJOR**: Breaking changes, major feature additions, architecture changes -- **MINOR**: New features, new modules, significant enhancements -- **PATCH**: Bug fixes, small improvements, documentation updates - -### Version Update Rules - -1. **MAJOR**: Only updated when user confirms major release -2. **MINOR**: Updated when user confirms new feature is complete -3. **PATCH**: Updated when user confirms bug fix is complete - -**IMPORTANT**: Never update version without user confirmation. +- **1.0.0** (2025-11-24) - Initial release with complete feature set --- -## Planned Features +## Versioning Strategy -### In Progress -- Planner Dashboard enhancement with KPIs -- Automation & CRON tasks -- Advanced analytics +- **Major version** (X.0.0) - Breaking changes, major architectural changes +- **Minor version** (1.X.0) - New features, new modules, backward compatible +- **Patch version** (1.0.X) - Bug fixes, minor improvements, documentation updates -### Future -- Analytics module enhancements -- Advanced scheduling features +--- + +## Upgrade Notes + +### Upgrading to 1.0.0 +This is the initial release. No upgrade path required. + +Future upgrade notes will be documented here for each version. + +--- + +## Future Roadmap + +Planned features and improvements (subject to change): + +- Enhanced analytics and reporting - Additional AI model integrations - Stripe payment integration -- Plan limits enforcement -- Advanced reporting +- Advanced automation workflows - Mobile app support -- API documentation (Swagger/OpenAPI) -- Unit and integration tests for unified API +- API webhooks +- Collaborative editing features +- Multi-language support + +See project board for current priorities and development status. --- -## Notes +**Note:** This changelog is maintained according to [Keep a Changelog](https://keepachangelog.com/) principles and only updated after user confirmation of completed features and fixes. -- All features are documented in detail in the respective documentation files -- Workflows are complete and accurate -- System is production-ready -- Documentation is maintained and updated regularly -- Changelog is only updated after user confirmation - ---- - -**For AI Agents**: Before making any changes, read `docs/00-DOCUMENTATION-MANAGEMENT.md` for complete guidelines on versioning, changelog management, and DRY principles. diff --git a/MASTER_REFERENCE.md b/MASTER_REFERENCE.md new file mode 100644 index 00000000..00b3fb44 --- /dev/null +++ b/MASTER_REFERENCE.md @@ -0,0 +1,1360 @@ +# IGNY8 System Master Reference + +**Version:** 1.0.0 +**Last Updated:** November 24, 2025 +**Purpose:** Complete system architecture and navigation reference for the IGNY8 SaaS platform and WordPress integration plugin. + +--- + +## Table of Contents + +1. [System Overview](#system-overview) +2. [Architecture](#architecture) +3. [Backend Deep Dive](#backend-deep-dive) +4. [Frontend Deep Dive](#frontend-deep-dive) +5. [WordPress Plugin Deep Dive](#wordpress-plugin-deep-dive) +6. [Integration Architecture](#integration-architecture) +7. [AI Framework](#ai-framework) +8. [Data Models](#data-models) +9. [API Reference](#api-reference) +10. [Workflows](#workflows) +11. [Developer Navigation Map](#developer-navigation-map) +12. [Troubleshooting Map](#troubleshooting-map) + +--- + +## System Overview + +IGNY8 is a full-stack AI-powered SEO content management platform consisting of three main components: + +1. **IGNY8 SaaS Application** - Multi-tenant Django + React application +2. **WordPress Bridge Plugin** - Bidirectional sync between WordPress and IGNY8 +3. **Sites Renderer** - Public-facing site deployment system + +### Core Capabilities + +| Capability | Implementation | Status | +|------------|----------------|--------| +| **Multi-Tenancy** | Account → Site → Sector hierarchy | ✅ Live | +| **Keyword Management** | Planner module (Keywords, Clusters, Ideas) | ✅ Live | +| **AI Content Generation** | Writer module with OpenAI integration | ✅ Live | +| **Image Generation** | AI-powered via DALL-E and Runware | ✅ Live | +| **WordPress Publishing** | Bidirectional sync via REST API | ✅ Live | +| **Internal Linking** | Linker module for SEO optimization | ✅ Live | +| **Content Optimization** | Optimizer module for scoring | ✅ Live | +| **Site Blueprints** | Site Builder for site structure | ✅ Live | +| **Automation** | Scheduled tasks and rules | ✅ Live | +| **Credit System** | Usage-based billing | ✅ Live | + +--- + +## Architecture + +### High-Level System Architecture + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ User Layer │ +├─────────────────────────────────────────────────────────────────┤ +│ React Frontend (app.igny8.com) │ WordPress Site (client) │ +│ - Vite 6.1.0 + React 19 │ - WP Bridge Plugin │ +│ - Zustand State Management │ - REST API Endpoints │ +│ - React Router v7 │ - Bidirectional Sync │ +└─────────────────────┬───────────────────────────┬───────────────┘ + │ │ + ▼ ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ API Gateway (Django) │ +├─────────────────────────────────────────────────────────────────┤ +│ - JWT Authentication │ +│ - Unified Response Format (success, data, message, request_id) │ +│ - Rate Limiting & Throttling │ +│ - Account Context Middleware │ +└─────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ Business Logic Layer │ +├─────────────────────────────────────────────────────────────────┤ +│ Modules (ViewSets) │ Business Logic (Services) │ +│ ─────────────────────────── │ ──────────────────────────── │ +│ • Planner │ • Planning Services │ +│ • Writer │ • Content Services │ +│ • Linker │ • Linking Services │ +│ • Optimizer │ • Optimization Services │ +│ • Publisher │ • Publishing Services │ +│ • Site Builder │ • Site Building Services │ +│ • Automation │ • Automation Services │ +│ • Integration │ • Integration Services │ +│ • System │ • System Settings │ +│ • Billing │ • Credit Services │ +└─────────────────────┬───────────────────┬───────────────────────┘ + │ │ + ▼ ▼ +┌──────────────────────────────┐ ┌──────────────────────────────┐ +│ AI Framework (Celery) │ │ Database (PostgreSQL) │ +├──────────────────────────────┤ ├──────────────────────────────┤ +│ • AIEngine │ │ • Account Isolation │ +│ • AICore │ │ • Site/Sector Scoping │ +│ • AI Functions Registry │ │ • Relational Models │ +│ • Progress Tracking │ │ • JSON Fields for Config │ +│ • Cost Tracking │ │ • Indexed Queries │ +└──────────────────────────────┘ └──────────────────────────────┘ +``` + +### Data Hierarchy + +``` +Account (Tenant) +├── Users (with roles) +├── Plan & Subscription +├── Credit Balance +└── Sites (1-N based on plan) + ├── Industry (optional) + ├── WordPress Integration (SiteIntegration) + └── Sectors (1-5 per site) + ├── Keywords + ├── Clusters + ├── Content Ideas + ├── Tasks + ├── Content + └── Images +``` + +--- + +## Backend Deep Dive + +### Technology Stack + +- **Framework:** Django 5.2+ +- **API:** Django REST Framework (DRF) with drf-spectacular +- **Database:** PostgreSQL +- **Task Queue:** Celery 5.3+ with Redis broker +- **Authentication:** JWT (PyJWT 2.8+) +- **AI:** OpenAI API, Runware API + +### Project Structure + +``` +backend/ +├── igny8_core/ # Django project root +│ ├── settings.py # Django configuration +│ ├── urls.py # Root URL routing +│ ├── celery.py # Celery configuration +│ ├── wsgi.py # WSGI entry point +│ ├── asgi.py # ASGI entry point (future) +│ │ +│ ├── auth/ # Multi-tenancy & authentication +│ │ ├── models.py # Account, User, Plan, Site, Sector +│ │ ├── views.py # Auth ViewSets +│ │ ├── serializers.py # Auth serializers +│ │ ├── middleware.py # AccountContextMiddleware +│ │ └── urls.py # /api/v1/auth/ routes +│ │ +│ ├── modules/ # Feature modules (ViewSets) +│ │ ├── planner/ # Keywords, Clusters, Ideas +│ │ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet +│ │ │ ├── serializers.py # Serializers +│ │ │ ├── urls.py # /api/v1/planner/ routes +│ │ │ └── apps.py # App config +│ │ │ +│ │ ├── writer/ # Tasks, Content, Images +│ │ │ ├── views.py # TasksViewSet, ContentViewSet, ImagesViewSet +│ │ │ ├── serializers.py # Serializers +│ │ │ └── urls.py # /api/v1/writer/ routes +│ │ │ +│ │ ├── linker/ # Internal linking +│ │ │ ├── views.py # LinkerViewSet +│ │ │ └── urls.py # /api/v1/linker/ routes +│ │ │ +│ │ ├── optimizer/ # Content optimization +│ │ │ ├── views.py # OptimizerViewSet +│ │ │ └── urls.py # /api/v1/optimizer/ routes +│ │ │ +│ │ ├── publisher/ # Publishing & deployment +│ │ │ ├── views.py # PublisherViewSet, PublishingRecordViewSet +│ │ │ └── urls.py # /api/v1/publisher/ routes +│ │ │ +│ │ ├── site_builder/ # Site blueprints & pages +│ │ │ ├── views.py # SiteBlueprintViewSet, PageBlueprintViewSet +│ │ │ ├── serializers.py # Serializers +│ │ │ └── urls.py # /api/v1/site-builder/ routes +│ │ │ +│ │ ├── automation/ # Automation rules & tasks +│ │ │ ├── views.py # AutomationRuleViewSet, ScheduledTaskViewSet +│ │ │ └── urls.py # /api/v1/automation/ routes +│ │ │ +│ │ ├── integration/ # External platform integrations +│ │ │ ├── views.py # IntegrationViewSet +│ │ │ └── urls.py # /api/v1/integration/ routes +│ │ │ +│ │ ├── system/ # Settings, prompts, integrations +│ │ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile +│ │ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet +│ │ │ ├── settings_views.py # SystemSettingsViewSet +│ │ │ └── urls.py # /api/v1/system/ routes +│ │ │ +│ │ └── billing/ # Credits & transactions +│ │ ├── views.py # CreditBalanceViewSet, CreditTransactionViewSet +│ │ ├── services.py # CreditService +│ │ └── urls.py # /api/v1/billing/ routes +│ │ +│ ├── business/ # Business logic & models +│ │ ├── planning/ # Planner models +│ │ │ └── models.py # Keywords, Clusters, ContentIdeas +│ │ ├── content/ # Writer models +│ │ │ └── models.py # Tasks, Content, Images +│ │ ├── linking/ # Linker services +│ │ ├── optimization/ # Optimizer services +│ │ ├── publishing/ # Publisher models +│ │ ├── site_building/ # Site builder models +│ │ ├── automation/ # Automation models +│ │ ├── integration/ # Integration models +│ │ │ ├── models.py # SiteIntegration +│ │ │ └── services/ # IntegrationService, SyncService +│ │ └── billing/ # Billing models +│ │ +│ ├── ai/ # AI Framework +│ │ ├── engine.py # AIEngine - Central orchestrator +│ │ ├── ai_core.py # AICore - API calls & model selection +│ │ ├── base.py # BaseAIFunction - Function interface +│ │ ├── registry.py # AI function registry +│ │ ├── prompts.py # PromptRegistry +│ │ ├── tracker.py # Progress & cost tracking +│ │ ├── tasks.py # run_ai_task Celery wrapper +│ │ ├── settings.py # Model configuration +│ │ ├── validators.py # Validation functions +│ │ ├── constants.py # AI constants +│ │ └── functions/ # AI function implementations +│ │ ├── auto_cluster.py # Keyword clustering +│ │ ├── generate_ideas.py # Content idea generation +│ │ ├── generate_content.py # Blog post generation +│ │ ├── generate_image_prompts.py # Image prompt extraction +│ │ └── generate_images.py # Image generation (DALL-E/Runware) +│ │ +│ ├── api/ # API base classes +│ │ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet +│ │ ├── response.py # success_response, error_response +│ │ ├── pagination.py # CustomPageNumberPagination +│ │ ├── permissions.py # Custom permissions +│ │ ├── throttles.py # Rate limiting +│ │ └── exception_handlers.py # Unified error handling +│ │ +│ ├── middleware/ # Custom middleware +│ │ ├── request_id.py # RequestIDMiddleware +│ │ ├── resource_tracker.py # ResourceTrackingMiddleware +│ │ └── __init__.py +│ │ +│ ├── utils/ # Shared utilities +│ │ ├── wordpress.py # WordPress helpers +│ │ ├── content_normalizer.py # Content processing +│ │ ├── queue_manager.py # Queue utilities +│ │ └── ai_processor.py # Legacy AI utilities +│ │ +│ └── admin/ # Django admin customization +│ ├── base.py # BaseAdmin +│ └── site.py # Admin site configuration +│ +├── manage.py # Django management CLI +├── requirements.txt # Python dependencies +└── Dockerfile # Docker configuration +``` + +### Core Base Classes + +#### AccountModelViewSet +**Location:** `igny8_core/api/base.py` + +Automatic account filtering for all resources. Used by modules that don't need site/sector filtering. + +```python +class AccountModelViewSet(ModelViewSet): + """Filters queryset by request.account automatically""" + + def get_queryset(self): + return super().get_queryset().filter(account=self.request.account) +``` + +**Used by:** System, Billing, Auth modules + +#### SiteSectorModelViewSet +**Location:** `igny8_core/api/base.py` + +Automatic filtering by account, site, and sector. Used by content-related modules. + +```python +class SiteSectorModelViewSet(ModelViewSet): + """Filters by account, site, and sector from query params""" + + def get_queryset(self): + queryset = super().get_queryset() + site_id = self.request.query_params.get('site_id') + sector_id = self.request.query_params.get('sector_id') + # ... filtering logic +``` + +**Used by:** Planner, Writer, Site Builder, Publisher, Automation, Integration modules + +### Middleware Stack + +1. **RequestIDMiddleware** - Assigns unique ID to each request +2. **AccountContextMiddleware** - Sets `request.account` from JWT token +3. **ResourceTrackingMiddleware** - Tracks API usage metrics + +--- + +## Frontend Deep Dive + +### Technology Stack + +- **Framework:** React 19.0.0 +- **Build Tool:** Vite 6.1.0 +- **Routing:** React Router v7.9.5 +- **State:** Zustand 5.0.8 +- **Styling:** Tailwind CSS 4.0.8 +- **Charts:** ApexCharts 4.1.0 +- **Icons:** Heroicons 2.2.0 + +### Project Structure + +``` +frontend/ +├── src/ +│ ├── main.tsx # Application entry point +│ ├── App.tsx # Main app with routing +│ ├── index.css # Global styles +│ │ +│ ├── api/ # API client layer +│ │ ├── client.ts # Axios instance with interceptors +│ │ ├── auth.ts # Auth API calls +│ │ ├── planner.ts # Planner API calls +│ │ ├── writer.ts # Writer API calls +│ │ └── [module].ts # Other module APIs +│ │ +│ ├── store/ # Zustand state management +│ │ ├── authStore.ts # Auth state (user, tokens) +│ │ ├── siteStore.ts # Site context +│ │ ├── moduleStore.ts # Module-specific state +│ │ └── globalStore.ts # Global app state +│ │ +│ ├── pages/ # Page components +│ │ ├── AuthPages/ +│ │ │ ├── SignIn.tsx +│ │ │ └── SignUp.tsx +│ │ ├── Dashboard/ +│ │ │ └── Home.tsx +│ │ ├── Planner/ +│ │ │ ├── Dashboard.tsx +│ │ │ ├── Keywords.tsx +│ │ │ ├── Clusters.tsx +│ │ │ └── Ideas.tsx +│ │ ├── Writer/ +│ │ │ ├── Dashboard.tsx +│ │ │ ├── Tasks.tsx +│ │ │ ├── Content.tsx +│ │ │ ├── ContentView.tsx +│ │ │ ├── Drafts.tsx +│ │ │ ├── Images.tsx +│ │ │ └── Published.tsx +│ │ ├── Linker/ +│ │ │ ├── Dashboard.tsx +│ │ │ └── ContentList.tsx +│ │ ├── Optimizer/ +│ │ │ ├── Dashboard.tsx +│ │ │ ├── ContentSelector.tsx +│ │ │ └── AnalysisPreview.tsx +│ │ ├── Thinker/ # System settings +│ │ │ ├── Dashboard.tsx +│ │ │ ├── Prompts.tsx +│ │ │ ├── AuthorProfiles.tsx +│ │ │ └── Strategies.tsx +│ │ ├── Billing/ +│ │ │ ├── Credits.tsx +│ │ │ ├── Transactions.tsx +│ │ │ └── Usage.tsx +│ │ ├── Automation/ +│ │ │ ├── Dashboard.tsx +│ │ │ ├── Rules.tsx +│ │ │ └── Tasks.tsx +│ │ └── Settings/ +│ │ ├── General.tsx +│ │ ├── Users.tsx +│ │ ├── Subscriptions.tsx +│ │ ├── System.tsx +│ │ └── AI.tsx +│ │ +│ ├── components/ # Reusable components +│ │ ├── common/ # Common UI components +│ │ ├── auth/ # Auth components +│ │ ├── forms/ # Form components +│ │ └── [module]/ # Module-specific components +│ │ +│ ├── layout/ # Layout components +│ │ ├── AppLayout.tsx # Main app layout +│ │ ├── Sidebar.tsx # Navigation sidebar +│ │ ├── Header.tsx # Top header +│ │ └── Footer.tsx # Footer +│ │ +│ ├── hooks/ # Custom React hooks +│ │ ├── useAuth.ts # Auth hook +│ │ ├── useApi.ts # API call hook +│ │ └── useSite.ts # Site context hook +│ │ +│ ├── services/ # Business logic +│ │ ├── authService.ts # Auth logic +│ │ └── apiService.ts # API utilities +│ │ +│ ├── config/ # Configuration +│ │ └── constants.ts # App constants +│ │ +│ └── types/ # TypeScript types +│ ├── api.ts # API types +│ └── models.ts # Data model types +│ +├── public/ # Static assets +├── package.json # Node dependencies +├── vite.config.ts # Vite configuration +├── tsconfig.json # TypeScript configuration +└── tailwind.config.js # Tailwind configuration +``` + +### State Management (Zustand) + +**Auth Store** (`store/authStore.ts`) +```typescript +interface AuthState { + user: User | null; + token: string | null; + refreshToken: string | null; + isAuthenticated: boolean; + login: (email: string, password: string) => Promise; + logout: () => void; + refreshAccessToken: () => Promise; +} +``` + +**Site Store** (`store/siteStore.ts`) +```typescript +interface SiteState { + currentSite: Site | null; + currentSector: Sector | null; + sites: Site[]; + setSite: (site: Site) => void; + setSector: (sector: Sector) => void; +} +``` + +--- + +## WordPress Plugin Deep Dive + +### Plugin Structure + +``` +igny8-wp-integration/ +├── igny8-bridge.php # Main plugin file & bootstrap +├── uninstall.php # Cleanup on uninstall +│ +├── includes/ # Core classes +│ ├── class-igny8-api.php # API client (authentication, requests) +│ ├── class-igny8-rest-api.php # WP REST endpoints (/wp-json/igny8/v1/) +│ ├── class-igny8-site.php # Site data collection +│ ├── class-igny8-webhooks.php # Webhook handlers (incoming from IGNY8) +│ ├── class-igny8-webhook-logs.php # Webhook logging +│ ├── class-igny8-link-queue.php # Link processing queue +│ └── functions.php # Helper functions +│ +├── admin/ # WordPress admin UI +│ ├── class-admin.php # Settings page controller +│ ├── settings.php # Settings page UI +│ ├── class-admin-columns.php # Custom post columns +│ ├── class-post-meta-boxes.php # Post editor meta boxes +│ └── assets/ +│ ├── css/admin.css # Admin styles +│ └── js/ +│ ├── admin.js # Admin scripts +│ └── post-editor.js # Post editor enhancements +│ +├── sync/ # Synchronization logic +│ ├── hooks.php # WordPress action/filter hooks +│ ├── post-sync.php # WP → IGNY8 post sync +│ ├── taxonomy-sync.php # WP → IGNY8 taxonomy sync +│ └── igny8-to-wp.php # IGNY8 → WP content sync +│ +├── data/ # Data collection & mapping +│ ├── site-collection.php # Full site data collection +│ ├── semantic-mapping.php # Semantic metadata extraction +│ ├── link-graph.php # Internal link graph builder +│ └── woocommerce.php # WooCommerce integration +│ +├── docs/ # Plugin documentation +│ ├── WORDPRESS-PLUGIN-INTEGRATION.md +│ ├── AUTHENTICATION-AUDIT.md +│ └── SYNC-DATA-FLOW-DIAGRAM.md +│ +└── tests/ # Test scripts + ├── test-api-authentication.php + ├── test-site-metadata.php + └── test-sync-structure.php +``` + +### Core Classes + +#### Igny8API +**File:** `includes/class-igny8-api.php` + +Handles all API communication with IGNY8 backend. + +**Key Methods:** +- `connect($api_key, $site_id)` - Establish connection using API key +- `get($endpoint)` - GET request +- `post($endpoint, $data)` - POST request +- `put($endpoint, $data)` - PUT request +- `delete($endpoint)` - DELETE request +- `parse_response($response)` - Normalize API responses + +**Authentication:** +- Primary: API key (stored in `igny8_api_key` option) +- Sends as `X-IGNY8-API-Key` header +- Automatic retry on 401 errors + +#### Igny8RestAPI +**File:** `includes/class-igny8-rest-api.php` + +Exposes WordPress data via REST API for IGNY8 to consume. + +**Endpoints:** +- `GET /wp-json/igny8/v1/site-metadata/` - Site structure and metadata +- `GET /wp-json/igny8/v1/post-by-task-id/{task_id}` - Find post by IGNY8 task ID +- `GET /wp-json/igny8/v1/post-by-content-id/{content_id}` - Find post by content ID +- `POST /wp-json/igny8/v1/posts/` - Create post from IGNY8 content +- `PUT /wp-json/igny8/v1/posts/{id}` - Update post from IGNY8 + +**Authentication:** +- Accepts `X-IGNY8-API-Key` header +- Validates against stored API key + +#### Igny8Site +**File:** `includes/class-igny8-site.php` + +Collects and formats site data for IGNY8. + +**Key Methods:** +- `get_site_metadata()` - Collect post types, taxonomies, counts +- `get_posts_data($post_type, $limit)` - Export posts +- `get_taxonomy_data($taxonomy, $limit)` - Export taxonomy terms +- `send_to_igny8()` - Push site data to IGNY8 API + +### Sync Flow + +#### WP → IGNY8 (Post Status Updates) + +1. User edits/publishes post in WordPress +2. `save_post` hook triggers → `sync/post-sync.php` +3. Check if post has `_igny8_task_id` meta +4. Map WP status to IGNY8 status +5. Send PUT request to `/api/v1/writer/tasks/{task_id}/` +6. Update `_igny8_last_synced` meta + +#### IGNY8 → WP (Content Publishing) + +1. IGNY8 creates content via AI Writer +2. IGNY8 sends POST to `/wp-json/igny8/v1/posts/` +3. Plugin validates API key +4. Create WP post with content +5. Store `_igny8_task_id` and `_igny8_content_id` meta +6. Return post ID and URL to IGNY8 + +### Post Meta Fields + +All IGNY8-managed posts use prefixed meta keys: + +- `_igny8_task_id` - IGNY8 task ID +- `_igny8_content_id` - IGNY8 content ID +- `_igny8_wordpress_status` - Current WP status +- `_igny8_last_synced` - Last sync timestamp +- `_igny8_managed` - Flag for IGNY8-managed posts + +--- + +## Integration Architecture + +### WordPress ↔ IGNY8 Integration + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ WordPress Site │ +├─────────────────────────────────────────────────────────────────┤ +│ IGNY8 Bridge Plugin │ +│ ───────────────────────────────────────────────────────────── │ +│ 1. REST API Endpoints (/wp-json/igny8/v1/) │ +│ • site-metadata (expose WP structure) │ +│ • posts (CRUD operations) │ +│ • post-by-task-id (lookup) │ +│ │ +│ 2. Sync Handlers (sync/) │ +│ • post-sync.php (WP → IGNY8) │ +│ • igny8-to-wp.php (IGNY8 → WP) │ +│ │ +│ 3. API Client (Igny8API class) │ +│ • Authentication (API key) │ +│ • Request/Response handling │ +└─────────────────────┬───────────────────────────────────────────┘ + │ + │ HTTPS REST API + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ IGNY8 Backend (Django) │ +├─────────────────────────────────────────────────────────────────┤ +│ Integration Module (/api/v1/integration/) │ +│ ───────────────────────────────────────────────────────────── │ +│ 1. IntegrationViewSet │ +│ • test-connection (validate WP connection) │ +│ • update-structure (receive WP metadata) │ +│ • sync (bidirectional sync trigger) │ +│ │ +│ 2. SiteIntegration Model │ +│ • platform: 'wordpress' │ +│ • credentials_json: {api_key, username, app_password} │ +│ • config_json: {site_url, content_types, sync_settings} │ +│ │ +│ 3. Integration Services │ +│ • IntegrationService (connection testing) │ +│ • SyncService (bidirectional sync) │ +│ • ContentSyncService (content mapping) │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### Authentication Flow + +**Plugin → Backend:** +1. Plugin stores API key in WP options (`igny8_api_key`) +2. All API requests include `X-IGNY8-API-Key` header +3. Backend validates key against `Site.wp_api_key` field +4. Backend sets account context from API key + +**Backend → Plugin:** +1. Backend sends requests to WP REST endpoints +2. Includes `X-IGNY8-API-Key` header +3. Plugin validates key matches stored value +4. Plugin processes request and returns unified response + +--- + +## AI Framework + +### Architecture + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ AI Framework Entry Point │ +├─────────────────────────────────────────────────────────────────┤ +│ run_ai_task(function_name, payload, account, celery_task) │ +│ • Called from ViewSets for async AI operations │ +│ • Wraps AI execution in Celery task │ +└─────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ AIEngine │ +│ Location: igny8_core/ai/engine.py │ +├─────────────────────────────────────────────────────────────────┤ +│ • Central orchestrator for all AI functions │ +│ • Progress tracking (Celery task updates) │ +│ • Cost tracking (token usage) │ +│ • Retry logic │ +│ • Error handling │ +│ │ +│ Key Methods: │ +│ • execute(function_name, payload) → result │ +│ • _execute_function(func, data) → processes function │ +│ • _update_progress(step, status, message) → UI updates │ +└─────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ AI Function Registry │ +│ Location: igny8_core/ai/registry.py │ +├─────────────────────────────────────────────────────────────────┤ +│ Registered Functions: │ +│ • auto_cluster (keywords → clusters) │ +│ • generate_ideas (clusters → content ideas) │ +│ • generate_content (ideas → blog posts) │ +│ • generate_image_prompts (content → image prompts) │ +│ • generate_images (prompts → images) │ +└─────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ AICore │ +│ Location: igny8_core/ai/ai_core.py │ +├─────────────────────────────────────────────────────────────────┤ +│ • Handles API calls to OpenAI/Runware │ +│ • Model selection and configuration │ +│ • Response parsing (JSON/text) │ +│ • Console logging │ +│ • Token counting and cost calculation │ +│ │ +│ Key Methods: │ +│ • run_ai_request(prompt, model, ...) → AI response │ +│ • generate_image(prompt, model, ...) → image URL │ +│ • _call_openai_api(messages, model, ...) → raw response │ +│ • _parse_json_response(text) → parsed JSON │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### AI Functions + +#### auto_cluster +**File:** `ai/functions/auto_cluster.py` +**Input:** List of keyword IDs +**Output:** Clusters with assigned keywords +**Model:** GPT-4 or configured model +**Process:** +1. Load keywords from database +2. Build prompt with keyword list +3. Call AI to group semantically similar keywords +4. Parse JSON response +5. Create Cluster records +6. Assign keywords to clusters + +#### generate_ideas +**File:** `ai/functions/generate_ideas.py` +**Input:** List of cluster IDs +**Output:** Content ideas for each cluster +**Model:** GPT-4 or configured model +**Process:** +1. Load clusters and keywords +2. Build prompt for content ideation +3. Call AI to generate ideas +4. Parse response +5. Create ContentIdea records + +#### generate_content +**File:** `ai/functions/generate_content.py` +**Input:** List of task IDs +**Output:** Full blog posts with HTML +**Model:** GPT-4 or configured model +**Process:** +1. Load tasks and content ideas +2. Build detailed prompt with brief +3. Call AI to generate article +4. Parse HTML content +5. Create/update Content records +6. Update task status + +#### generate_images +**File:** `ai/functions/generate_images.py` +**Input:** List of task IDs with image prompts +**Output:** Generated image URLs +**Models:** DALL-E 3 or Runware +**Process:** +1. Load tasks with image prompts +2. For each prompt, call image API +3. Upload images to storage +4. Create Image records +5. Link images to content + +--- + +## Data Models + +### Account & Auth Models + +**Account** (`auth/models.py`) +```python +Account +├── name: str +├── slug: str (unique) +├── owner: User (FK) +├── plan: Plan (FK) +├── credits: int +├── status: str (active, suspended, trial, cancelled) +└── created_at, updated_at +``` + +**User** (`auth/models.py`) +```python +User (extends AbstractUser) +├── email: str (unique) +├── username: str (unique) +├── account: Account (FK) +├── role: str (developer, owner, admin, editor, viewer, system_bot) +└── is_active, is_staff, is_superuser +``` + +**Site** (`auth/models.py`) +```python +Site +├── account: Account (FK) +├── name: str +├── slug: str +├── domain: URL +├── industry: Industry (FK, nullable) +├── wordpress_url: URL +├── wordpress_username: str +├── wordpress_password: str (encrypted) +├── wp_api_key: str (for bridge auth) +├── status: str (active, inactive, suspended) +└── created_at, updated_at +``` + +**Sector** (`auth/models.py`) +```python +Sector +├── account: Account (FK) +├── site: Site (FK) +├── name: str +├── industry_sector: IndustrySector (FK, nullable) +├── status: str (active, inactive) +└── created_at, updated_at +``` + +### Planner Models + +**Keywords** (`business/planning/models.py`) +```python +Keywords (inherits SiteSectorBaseModel) +├── account, site, sector (from base) +├── keyword: str +├── search_volume: int +├── difficulty: int +├── intent: str (informational, commercial, transactional, navigational) +├── status: str (active, inactive, used) +├── source: str (manual, csv_import, api_import) +└── created_at, updated_at +``` + +**Clusters** (`business/planning/models.py`) +```python +Clusters (inherits SiteSectorBaseModel) +├── account, site, sector (from base) +├── name: str +├── description: text +├── keywords: ManyToMany(Keywords) +├── status: str (active, archived) +└── created_at, updated_at +``` + +**ContentIdeas** (`business/planning/models.py`) +```python +ContentIdeas (inherits SiteSectorBaseModel) +├── account, site, sector (from base) +├── cluster: Cluster (FK) +├── title: str +├── description: text +├── suggested_keywords: JSON +├── status: str (draft, approved, in_progress, completed) +└── created_at, updated_at +``` + +### Writer Models + +**Tasks** (`business/content/models.py`) +```python +Tasks (inherits SiteSectorBaseModel) +├── account, site, sector (from base) +├── content_idea: ContentIdea (FK, nullable) +├── cluster: Cluster (FK, nullable) +├── title: str +├── brief: text +├── target_keywords: JSON +├── status: str (pending, in_progress, completed, published) +├── assigned_post_id: int (WP post ID) +├── post_url: URL +├── content: Content (OneToOne, reverse) +└── created_at, updated_at +``` + +**Content** (`business/content/models.py`) +```python +Content (inherits SiteSectorBaseModel) +├── account, site, sector (from base) +├── task: Tasks (OneToOne, nullable) +├── title: str +├── content_html: text +├── content_plain: text +├── excerpt: text +├── meta_description: str +├── entity_type: str (post, page, product, service) +├── external_id: str (WP post ID) +├── external_type: str (post, page) +├── sync_status: str (draft, imported, synced, published) +├── taxonomies: ManyToMany(ContentTaxonomy) +├── attributes: JSON +└── created_at, updated_at +``` + +**Images** (`business/content/models.py`) +```python +Images (inherits SiteSectorBaseModel) +├── account, site, sector (from base) +├── task: Tasks (FK) +├── image_url: URL +├── prompt: text +├── is_featured: bool +├── position: int +├── alt_text: str +├── caption: str +└── created_at, updated_at +``` + +### Integration Models + +**SiteIntegration** (`business/integration/models.py`) +```python +SiteIntegration +├── account: Account (FK) +├── site: Site (FK) +├── platform: str (wordpress, shopify, wix) +├── platform_type: str (cms, ecommerce, builder) +├── config_json: JSON (site_url, content_types, sync_settings) +├── credentials_json: JSON (api_key, username, app_password) +├── is_active: bool +├── sync_enabled: bool +├── last_sync_at: datetime +└── created_at, updated_at +``` + +--- + +## API Reference + +### Base URL + +``` +Production: https://api.igny8.com/api/v1/ +Development: http://localhost:8000/api/v1/ +``` + +### Unified Response Format + +All endpoints return: + +```json +{ + "success": true, + "data": { ... }, + "message": "Success message", + "request_id": "uuid", + "count": 100, // For paginated endpoints + "next": "url", // Pagination + "previous": "url", // Pagination + "results": [...] // Paginated data +} +``` + +Error response: +```json +{ + "success": false, + "error": "Error message", + "errors": {...}, // Field-level errors + "request_id": "uuid" +} +``` + +### Authentication + +**Login:** +```http +POST /api/v1/auth/login/ +Content-Type: application/json + +{ + "email": "user@example.com", + "password": "password" +} + +Response: +{ + "success": true, + "data": { + "access": "jwt_token", + "refresh": "refresh_token", + "user": {...} + } +} +``` + +**Use Token:** +```http +GET /api/v1/planner/keywords/ +Authorization: Bearer {access_token} +``` + +### Key Endpoints by Module + +#### Planner Module + +``` +GET /api/v1/planner/keywords/ # List keywords +POST /api/v1/planner/keywords/ # Create keyword +GET /api/v1/planner/keywords/{id}/ # Get keyword +PUT /api/v1/planner/keywords/{id}/ # Update keyword +DELETE /api/v1/planner/keywords/{id}/ # Delete keyword +POST /api/v1/planner/keywords/bulk_create/ # Bulk import +POST /api/v1/planner/keywords/auto_cluster/ # AI clustering + +GET /api/v1/planner/clusters/ # List clusters +POST /api/v1/planner/clusters/ # Create cluster +POST /api/v1/planner/clusters/generate_ideas/ # AI idea generation + +GET /api/v1/planner/ideas/ # List ideas +POST /api/v1/planner/ideas/ # Create idea +``` + +#### Writer Module + +``` +GET /api/v1/writer/tasks/ # List tasks +POST /api/v1/writer/tasks/ # Create task +GET /api/v1/writer/tasks/{id}/ # Get task +PUT /api/v1/writer/tasks/{id}/ # Update task +POST /api/v1/writer/tasks/generate_content/ # AI content generation +POST /api/v1/writer/tasks/generate_images/ # AI image generation + +GET /api/v1/writer/content/ # List content +GET /api/v1/writer/content/{id}/ # Get content + +GET /api/v1/writer/images/ # List images +``` + +#### Integration Module + +``` +GET /api/v1/integration/integrations/ # List integrations +POST /api/v1/integration/integrations/ # Create integration +POST /api/v1/integration/integrations/test-connection/ # Test WP connection +POST /api/v1/integration/integrations/{id}/update-structure/ # Update WP metadata +GET /api/v1/integration/integrations/{id}/content-types/ # Get content types +POST /api/v1/integration/integrations/{id}/sync/ # Trigger sync +``` + +#### System Module + +``` +GET /api/v1/system/prompts/ # List AI prompts +POST /api/v1/system/prompts/ # Create prompt +GET /api/v1/system/author-profiles/ # List author profiles +GET /api/v1/system/settings/ # System settings +``` + +#### Billing Module + +``` +GET /api/v1/billing/balance/ # Get credit balance +GET /api/v1/billing/transactions/ # List transactions +GET /api/v1/billing/usage/ # Usage analytics +``` + +--- + +## Workflows + +### 1. Keyword to Published Post (Full Workflow) + +``` +1. Import Keywords + ↓ + POST /api/v1/planner/keywords/bulk_create/ + • Upload CSV or manual entry + • Keywords stored in database + +2. AI Clustering + ↓ + POST /api/v1/planner/keywords/auto_cluster/ + • Celery task: auto_cluster + • AI groups keywords → Clusters + • Keywords assigned to clusters + +3. Generate Content Ideas + ↓ + POST /api/v1/planner/clusters/generate_ideas/ + • Celery task: generate_ideas + • AI creates ContentIdeas from clusters + +4. Create Writer Task + ↓ + POST /api/v1/writer/tasks/ + • Link task to ContentIdea + • Set brief and target keywords + +5. Generate Content + ↓ + POST /api/v1/writer/tasks/generate_content/ + • Celery task: generate_content + • AI writes full blog post → Content record + • Task status: completed + +6. Generate Images (optional) + ↓ + POST /api/v1/writer/tasks/generate_images/ + • Celery task: generate_images + • AI creates images → Image records + +7. Publish to WordPress + ↓ + POST /wp-json/igny8/v1/posts/ + • IGNY8 sends content to WP plugin + • Plugin creates WP post + • Stores _igny8_task_id meta + +8. Status Sync (WP → IGNY8) + ↓ + PUT /api/v1/writer/tasks/{id}/ + • User publishes in WP + • Plugin syncs status back to IGNY8 +``` + +### 2. WordPress Integration Setup + +``` +1. Install Plugin + ↓ + • Upload igny8-bridge.zip to WordPress + • Activate plugin + +2. Generate API Key (IGNY8 App) + ↓ + • Login to app.igny8.com + • Navigate to Settings → Sites + • Generate API key for site + +3. Connect Plugin + ↓ + • In WP admin: Settings → IGNY8 API + • Enter email, password, API key + • Click "Test Connection" + +4. Initial Sync + ↓ + • Plugin sends site metadata to IGNY8 + • POST /api/v1/integration/integrations/test-connection/ + • IGNY8 creates SiteIntegration record + +5. Ongoing Sync + ↓ + • WP → IGNY8: Post status updates (save_post hook) + • IGNY8 → WP: Content publishing (REST API) +``` + +--- + +## Developer Navigation Map + +### "I need to..." + +#### Add a new API endpoint +1. Create method in ViewSet (`modules/[module]/views.py`) +2. Use `@action` decorator for custom endpoints +3. Add route to `urls.py` if needed +4. Return `success_response()` or `error_response()` + +#### Add a new model field +1. Edit model in `business/[domain]/models.py` +2. Run `python manage.py makemigrations` +3. Run `python manage.py migrate` +4. Update serializers in `modules/[module]/serializers.py` + +#### Add AI functionality +1. Create function class in `ai/functions/[name].py` +2. Inherit from `BaseAIFunction` +3. Implement `validate()`, `prepare()`, `execute()`, `save()` +4. Register in `ai/registry.py` +5. Call via `run_ai_task()` from ViewSet + +#### Debug API call +1. Check request ID in response +2. Search logs: `grep {request_id} logs/` +3. Check middleware: `RequestIDMiddleware` +4. Check account context: `AccountContextMiddleware` + +#### Add WordPress sync logic +1. Add hook in `sync/hooks.php` +2. Implement handler in `sync/post-sync.php` or `sync/taxonomy-sync.php` +3. Use `Igny8API` class for API calls +4. Update post meta with `_igny8_*` keys + +#### Add frontend page +1. Create component in `frontend/src/pages/[Module]/[Page].tsx` +2. Add route in `App.tsx` +3. Add to sidebar navigation in `layout/Sidebar.tsx` +4. Create API calls in `api/[module].ts` +5. Use Zustand store for state + +--- + +## Troubleshooting Map + +### Common Issues and Where to Look + +#### "Authentication failed / 401 errors" +**Check:** +- `igny8_core/auth/middleware.py` - AccountContextMiddleware +- JWT token expiry - tokens expire in 15 minutes +- API key validation - `Site.wp_api_key` field +- Plugin: `includes/class-igny8-api.php` - connect() method + +#### "Posts not syncing to IGNY8" +**Check:** +- `sync/hooks.php` - save_post hook registered? +- `sync/post-sync.php` - igny8_sync_post_status_to_igny8() +- Post meta: `_igny8_task_id` exists? +- `igny8_is_connection_enabled()` returns true? +- API key stored: `get_option('igny8_api_key')` + +#### "AI task stuck/failed" +**Check:** +- Celery worker running: `celery -A igny8_core worker -l info` +- Task logs: Check Celery console output +- `ai/engine.py` - AIEngine.execute() +- `ai/ai_core.py` - APICore.run_ai_request() +- Credit balance: Check `Account.credits` +- Model configuration: `system/models.py` - IntegrationSettings + +#### "Frontend can't fetch data" +**Check:** +- CORS settings: `igny8_core/settings.py` - CORS_ALLOWED_ORIGINS +- API base URL: `frontend/src/api/client.ts` +- Auth token: Check `authStore` in browser DevTools +- Network tab: Check request/response +- Backend logs: Check for errors + +#### "WordPress plugin can't connect" +**Check:** +- API key matches: WP option vs `Site.wp_api_key` +- Site URL correct: WP site URL vs `SiteIntegration.config_json.site_url` +- REST API enabled: Test `/wp-json/` endpoint +- PHP errors: Check WP debug log +- Connection test: `includes/class-igny8-api.php` - connect() + +#### "Rate limiting errors" +**Check:** +- `api/throttles.py` - DebugScopedRateThrottle +- `settings.py` - IGNY8_DEBUG_THROTTLE flag +- User role has sufficient permissions +- Check throttle scope in ViewSet + +#### "Account isolation not working" +**Check:** +- `auth/middleware.py` - AccountContextMiddleware sets request.account? +- ViewSet inherits from `AccountModelViewSet` or `SiteSectorModelViewSet`? +- JWT token contains `account_id`? +- Query filtering: `get_queryset()` filters by account? + +--- + +## File Quick Reference + +### Most Important Files + +| File | Purpose | When to Check | +|------|---------|---------------| +| `backend/igny8_core/settings.py` | Django configuration | Environment issues, installed apps | +| `backend/igny8_core/urls.py` | URL routing | Adding endpoints, 404 errors | +| `backend/igny8_core/celery.py` | Celery config | Task queue issues | +| `backend/igny8_core/auth/models.py` | Account, User, Site models | Multi-tenancy, auth issues | +| `backend/igny8_core/auth/middleware.py` | Account context | Request.account issues | +| `backend/igny8_core/api/base.py` | Base ViewSets | Queryset filtering | +| `backend/igny8_core/ai/engine.py` | AI orchestrator | AI task failures | +| `backend/igny8_core/ai/ai_core.py` | AI API calls | OpenAI errors | +| `frontend/src/App.tsx` | Frontend routing | Navigation issues | +| `frontend/src/store/authStore.ts` | Auth state | Login/logout issues | +| `frontend/src/api/client.ts` | API client | API call failures | +| `igny8-wp-integration/igny8-bridge.php` | Plugin bootstrap | Plugin initialization | +| `igny8-wp-integration/includes/class-igny8-api.php` | WP API client | API communication | +| `igny8-wp-integration/includes/class-igny8-rest-api.php` | WP REST endpoints | Endpoint errors | +| `igny8-wp-integration/sync/post-sync.php` | WP → IGNY8 sync | Sync failures | + +--- + +## Technology Dependencies + +### Backend + +``` +Django==5.2.7 +djangorestframework +psycopg2-binary # PostgreSQL +redis # Celery broker +celery==5.3.0 +PyJWT==2.8.0 +requests==2.31.0 +beautifulsoup4==4.12.0 +drf-spectacular==0.27.0 +django-filter +django-cors-headers +whitenoise +gunicorn +``` + +### Frontend + +``` +react==19.0.0 +react-dom==19.0.0 +react-router==7.1.5 +react-router-dom==7.9.5 +zustand==5.0.8 +vite==6.1.0 +typescript==5.7.2 +tailwindcss==4.0.8 +apexcharts==4.1.0 +@heroicons/react==2.2.0 +``` + +### WordPress Plugin + +``` +PHP >= 7.4 +WordPress >= 5.0 +``` + +--- + +## Deployment Architecture + +### Production Stack + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Caddy │ +│ (Reverse Proxy + SSL) │ +└─────────────────────┬───────────────────────────────────────────┘ + │ + ┌───────────┴───────────┐ + ▼ ▼ +┌──────────────────┐ ┌──────────────────┐ +│ Frontend (Vite) │ │ Backend (Gunicorn)│ +│ app.igny8.com │ │ api.igny8.com │ +│ Port: 5173 │ │ Port: 8000 │ +└──────────────────┘ └─────────┬──────────┘ + │ + ┌───────────────┴───────────────┐ + ▼ ▼ + ┌──────────────────┐ ┌──────────────────┐ + │ PostgreSQL │ │ Redis │ + │ Database │ │ Celery Broker │ + └──────────────────┘ └─────────┬────────┘ + │ + ▼ + ┌──────────────────┐ + │ Celery Worker │ + │ AI Tasks │ + └──────────────────┘ +``` + +### Docker Compose + +See `docker-compose.app.yml` for full configuration. + +--- + +**End of Master Reference** + +For setup instructions, see `README.md`. +For version history, see `CHANGELOG.md`. +For API details, see `master-docs/API-COMPLETE-REFERENCE.md`. diff --git a/README.md b/README.md new file mode 100644 index 00000000..a1fa0d83 --- /dev/null +++ b/README.md @@ -0,0 +1,358 @@ +# IGNY8 - AI-Powered SEO Content Platform + +**Version:** 1.0.0 +**License:** Proprietary +**Website:** https://igny8.com + +--- + +## What is IGNY8? + +IGNY8 is a full-stack SaaS platform that combines AI-powered content generation with intelligent SEO management. It helps content creators, marketers, and agencies streamline their content workflow from keyword research to published articles. + +### Key Features + +- 🔍 **Smart Keyword Management** - Import, cluster, and organize keywords with AI +- ✍️ **AI Content Generation** - Generate SEO-optimized blog posts using GPT-4 +- 🖼️ **AI Image Creation** - Auto-generate featured and in-article images +- 🔗 **Internal Linking** - AI-powered link suggestions for SEO +- 📊 **Content Optimization** - Analyze and score content quality +- 🔄 **WordPress Integration** - Bidirectional sync with WordPress sites +- 📈 **Usage-Based Billing** - Credit system for AI operations +- 👥 **Multi-Tenancy** - Manage multiple sites and teams + +--- + +## Repository Structure + +This monorepo contains two main applications: + +``` +igny8/ +├── backend/ # Django REST API + Celery +├── frontend/ # React + Vite SPA +├── master-docs/ # Architecture documentation +└── docker-compose.app.yml # Docker deployment config +``` + +**Separate Repository:** +- [igny8-wp-integration](https://github.com/alorig/igny8-wp-integration) - WordPress bridge plugin + +--- + +## Quick Start + +### Prerequisites + +- **Python 3.11+** +- **Node.js 18+** +- **PostgreSQL 14+** +- **Redis 7+** +- **Docker** (optional, recommended for local development) + +### Local Development with Docker + +1. **Clone the repository** + ```powershell + git clone https://github.com/alorig/igny8-app.git + cd igny8 + ``` + +2. **Set environment variables** + + Create `.env` file in `backend/` directory: + ```env + SECRET_KEY=your-secret-key-here + DEBUG=True + DATABASE_URL=postgresql://postgres:postgres@db:5432/igny8 + REDIS_URL=redis://redis:6379/0 + OPENAI_API_KEY=your-openai-key + RUNWARE_API_KEY=your-runware-key + ``` + +3. **Start services** + ```powershell + docker-compose -f docker-compose.app.yml up --build + ``` + +4. **Access applications** + - Frontend: http://localhost:5173 + - Backend API: http://localhost:8000 + - API Docs: http://localhost:8000/api/docs/ + - Django Admin: http://localhost:8000/admin/ + +### Manual Setup (Without Docker) + +#### Backend Setup + +```powershell +cd backend + +# Create virtual environment +python -m venv .venv +.\.venv\Scripts\Activate.ps1 + +# Install dependencies +pip install -r requirements.txt + +# Run migrations +python manage.py migrate + +# Create superuser +python manage.py createsuperuser + +# Run development server +python manage.py runserver +``` + +In separate terminals, start Celery: + +```powershell +# Celery worker +celery -A igny8_core worker -l info + +# Celery beat (scheduled tasks) +celery -A igny8_core beat -l info +``` + +#### Frontend Setup + +```powershell +cd frontend + +# Install dependencies +npm install + +# Start dev server +npm run dev +``` + +--- + +## Project Architecture + +### System Overview + +``` +User Interface (React) + ↓ + REST API (Django) + ↓ +┌───────┴────────┐ +│ │ +Database AI Engine +(PostgreSQL) (Celery + OpenAI) + │ + WordPress Plugin + (Bidirectional Sync) +``` + +### Tech Stack + +**Backend:** +- Django 5.2+ (Python web framework) +- Django REST Framework (API) +- PostgreSQL (Database) +- Celery (Async task queue) +- Redis (Message broker) +- OpenAI API (Content generation) + +**Frontend:** +- React 19 (UI library) +- Vite 6 (Build tool) +- Zustand (State management) +- React Router v7 (Routing) +- Tailwind CSS 4 (Styling) + +**WordPress Plugin:** +- PHP 7.4+ (WordPress compatibility) +- REST API integration +- Bidirectional sync + +--- + +## How IGNY8 Works + +### Content Creation Workflow + +``` +1. Import Keywords + ↓ +2. AI Clusters Keywords + ↓ +3. Generate Content Ideas + ↓ +4. Create Writer Tasks + ↓ +5. AI Generates Content + ↓ +6. AI Creates Images + ↓ +7. Publish to WordPress + ↓ +8. Sync Status Back +``` + +### WordPress Integration + +The WordPress bridge plugin (`igny8-wp-integration`) creates a bidirectional connection: + +- **IGNY8 → WordPress:** Publish AI-generated content to WordPress +- **WordPress → IGNY8:** Sync post status updates back to IGNY8 + +**Setup:** +1. Install WordPress plugin on your site +2. Generate API key in IGNY8 app +3. Connect plugin using email, password, and API key +4. Plugin syncs automatically + +--- + +## Documentation + +Comprehensive documentation is available in the `master-docs/` directory: + +- **[MASTER_REFERENCE.md](./MASTER_REFERENCE.md)** - Complete system architecture and navigation +- **[API-COMPLETE-REFERENCE.md](./master-docs/API-COMPLETE-REFERENCE.md)** - Full API documentation +- **[02-APPLICATION-ARCHITECTURE.md](./master-docs/02-APPLICATION-ARCHITECTURE.md)** - System design +- **[04-BACKEND-IMPLEMENTATION.md](./master-docs/04-BACKEND-IMPLEMENTATION.md)** - Backend details +- **[03-FRONTEND-ARCHITECTURE.md](./master-docs/03-FRONTEND-ARCHITECTURE.md)** - Frontend details +- **[WORDPRESS-PLUGIN-INTEGRATION.md](./master-docs/WORDPRESS-PLUGIN-INTEGRATION.md)** - Plugin integration guide + +--- + +## Development Workflow + +### Running Tests + +```powershell +# Backend tests +cd backend +python manage.py test + +# Frontend tests +cd frontend +npm run test +``` + +### Code Quality + +```powershell +# Frontend linting +cd frontend +npm run lint +``` + +### Building for Production + +```powershell +# Backend +cd backend +python manage.py collectstatic + +# Frontend +cd frontend +npm run build +``` + +--- + +## API Overview + +**Base URL:** `https://api.igny8.com/api/v1/` + +**Authentication:** JWT Bearer token + +**Key Endpoints:** +- `/auth/login/` - User authentication +- `/planner/keywords/` - Keyword management +- `/planner/clusters/` - Keyword clusters +- `/writer/tasks/` - Content tasks +- `/writer/content/` - Generated content +- `/integration/integrations/` - WordPress integrations + +**Interactive Docs:** +- Swagger UI: https://api.igny8.com/api/docs/ +- ReDoc: https://api.igny8.com/api/redoc/ + +See [API-COMPLETE-REFERENCE.md](./master-docs/API-COMPLETE-REFERENCE.md) for full documentation. + +--- + +## Multi-Tenancy + +IGNY8 supports complete account isolation: + +``` +Account (Organization) +├── Users (with roles: owner, admin, editor, viewer) +├── Sites (multiple WordPress sites) +└── Sectors (content categories) + └── Keywords, Clusters, Content +``` + +All data is automatically scoped to the authenticated user's account. + +--- + +## Contributing + +This is a private repository. For internal development: + +1. Create feature branch: `git checkout -b feature/your-feature` +2. Make changes and test thoroughly +3. Commit: `git commit -m "Add your feature"` +4. Push: `git push origin feature/your-feature` +5. Create Pull Request + +--- + +## Deployment + +### Production Deployment + +1. **Set production environment variables** +2. **Build frontend:** `npm run build` +3. **Collect static files:** `python manage.py collectstatic` +4. **Run migrations:** `python manage.py migrate` +5. **Use docker-compose:** `docker-compose -f docker-compose.app.yml up -d` + +### Environment Variables + +Required for production: + +```env +SECRET_KEY= +DEBUG=False +ALLOWED_HOSTS=api.igny8.com,app.igny8.com +DATABASE_URL=postgresql://user:pass@host:5432/dbname +REDIS_URL=redis://host:6379/0 +OPENAI_API_KEY= +RUNWARE_API_KEY= +USE_SECURE_COOKIES=True +``` + +--- + +## Support + +For support and questions: +- Check [MASTER_REFERENCE.md](./MASTER_REFERENCE.md) for detailed documentation +- Review API docs at `/api/docs/` +- Contact development team + +--- + +## License + +Proprietary. All rights reserved. + +--- + +## Changelog + +See [CHANGELOG.md](./CHANGELOG.md) for version history and updates. + +--- + +**Built with ❤️ by the IGNY8 team** diff --git a/master-docs/00-DOCUMENTATION-MANAGEMENT.md b/master-docs/00-DOCUMENTATION-MANAGEMENT.md deleted file mode 100644 index a5eced1f..00000000 --- a/master-docs/00-DOCUMENTATION-MANAGEMENT.md +++ /dev/null @@ -1,404 +0,0 @@ -# IGNY8 Documentation & Changelog Management System - -**Last Updated:** 2025-01-XX -**Purpose:** Complete guide for managing documentation versioning, changelog updates, and DRY principles. This document must be read by all AI agents at the start of any session. - ---- - -## Table of Contents - -1. [Versioning System](#versioning-system) -2. [Changelog Management](#changelog-management) -3. [Documentation Update Process](#documentation-update-process) -4. [DRY Principles & Standards](#dry-principles--standards) -5. [AI Agent Instructions](#ai-agent-instructions) - ---- - -## Versioning System - -### Version Format - -**Format:** `MAJOR.MINOR.PATCH` (Semantic Versioning) - -- **MAJOR**: Breaking changes, major feature additions, architecture changes -- **MINOR**: New features, new modules, significant enhancements -- **PATCH**: Bug fixes, small improvements, documentation updates - -**Current Version:** `1.0.0` - -### Version Tracking - -**Location:** -- Root `CHANGELOG.md` - Main version history -- Each documentation file header - Last updated date - -**Version Update Rules:** -- **MAJOR**: Only updated when user confirms major release -- **MINOR**: Updated when user confirms new feature is complete -- **PATCH**: Updated when user confirms bug fix is complete - -### Version Update Process - -1. **Code Change Made**: Developer/AI makes code changes -2. **User Confirmation**: User confirms fix/feature is complete -3. **Version Update**: Update version in CHANGELOG.md -4. **Changelog Entry**: Add entry to CHANGELOG.md -5. **Documentation Update**: Update relevant documentation files if needed - -**IMPORTANT**: Never update version or changelog without user confirmation that the change is complete and working. - ---- - -## Changelog Management - -### Changelog Location - -**File:** `/CHANGELOG.md` (root directory) - -### Changelog Structure - -```markdown -## [Version] - YYYY-MM-DD - -### Added -- New features, modules, or capabilities - -### Changed -- Changes to existing features or behavior - -### Fixed -- Bug fixes and corrections - -### Deprecated -- Features that will be removed in future versions - -### Removed -- Features that have been removed - -### Security -- Security fixes and improvements -``` - -### Changelog Entry Format - -Each entry must include: -- **Date**: YYYY-MM-DD format -- **Type**: Added, Changed, Fixed, Deprecated, Removed, Security -- **Description**: Clear, concise description of the change -- **Affected Areas**: Modules, components, or features affected -- **Documentation**: Reference to updated documentation files (if any) - -### Example Changelog Entry - -```markdown -## [1.0.1] - 2025-01-15 - -### Fixed -- Fixed keyword clustering issue where keywords were not properly linked to clusters - - **Affected**: Planner Module, Keyword Clustering - - **Documentation**: Updated 06-FUNCTIONAL-BUSINESS-LOGIC.md (Keyword Clustering section) - -### Added -- Added bulk delete functionality for content ideas - - **Affected**: Planner Module, Content Ideas - - **Documentation**: Updated 06-FUNCTIONAL-BUSINESS-LOGIC.md (Content Ideas Management section) -``` - -### Changelog Update Rules - -1. **Only Update After User Confirmation**: Never add changelog entries until user confirms the change is complete -2. **One Entry Per Change**: Each fix or feature gets its own entry -3. **Chronological Order**: Newest entries at the top -4. **Be Specific**: Include what was changed, why, and where -5. **Link Documentation**: Reference updated documentation files -6. **Version Bump**: Update version number when adding entries - -### Changelog Categories - -**Added**: New features, new modules, new endpoints, new functions -**Changed**: Modified existing features, updated workflows, refactored code -**Fixed**: Bug fixes, error corrections, issue resolutions -**Deprecated**: Features marked for removal (include migration path) -**Removed**: Features that have been completely removed -**Security**: Security patches, vulnerability fixes, access control updates - ---- - -## Documentation Update Process - -### When to Update Documentation - -1. **New Feature Added**: Update relevant documentation files -2. **Feature Changed**: Update affected sections in documentation -3. **Bug Fixed**: Update documentation if behavior changed -4. **Workflow Modified**: Update workflow documentation -5. **API Changed**: Update API documentation -6. **Architecture Changed**: Update architecture documentation - -### Documentation Files Structure - -``` -master-docs/ # Master documentation (permanent reference) -├── 00-DOCUMENTATION-MANAGEMENT.md # This file (management guide) -├── 01-TECH-STACK-AND-INFRASTRUCTURE.md -├── 02-APPLICATION-ARCHITECTURE.md -├── 03-FRONTEND-ARCHITECTURE.md -├── 04-BACKEND-IMPLEMENTATION.md -├── 05-AI-FRAMEWORK-IMPLEMENTATION.md -├── 06-FUNCTIONAL-BUSINESS-LOGIC.md # Includes complete workflow documentation -├── API-COMPLETE-REFERENCE.md -└── WORDPRESS-PLUGIN-INTEGRATION.md - -active-workflow-docs/ # Current state and active work (single file) -└── CURRENT_WORKFLOW_STATUS.md # Single source of truth for current system state -``` - -**Note:** The `refactor-plan/` folder has been removed after consolidating relevant content into master docs. - -### Documentation Update Checklist - -- [ ] Identify which documentation file(s) need updating -- [ ] Update the relevant section(s) -- [ ] Update "Last Updated" date in file header -- [ ] Add changelog entry (after user confirmation) -- [ ] Verify all links still work -- [ ] Ensure consistency across all documentation - -### Documentation Standards - -1. **No Code Snippets**: Documentation focuses on workflows, features, and architecture -2. **Complete Coverage**: All features and workflows must be documented -3. **Accurate State**: Documentation must reflect current system state -4. **Clear Structure**: Use consistent headings and formatting -5. **Cross-References**: Link related sections and documents - ---- - -## DRY Principles & Standards - -### DRY (Don't Repeat Yourself) Philosophy - -**Core Principle**: Use existing, predefined, standardized components, utilities, functions, and configurations instead of creating parallel systems or duplicating code. - -### Frontend DRY Standards - -#### Components - -**MUST USE Existing Components:** -- **Templates**: Use 4 universal templates (DashboardTemplate, TablePageTemplate, FormPageTemplate, SystemPageTemplate) -- **UI Components**: Use components from `/frontend/src/components/` -- **Common Components**: Use ScrollToTop, GlobalErrorDisplay, LoadingStateMonitor -- **Form Components**: Use existing form components with props and configs - -**DO NOT:** -- Create new templates when existing ones can be used -- Duplicate component logic -- Create parallel component systems -- Hardcode UI elements that can use config-driven approach - -#### Configuration-Driven Development - -**MUST USE Configuration Files:** -- **Page Configs**: `/frontend/src/config/pages/` - Define page structure -- **Snippet Configs**: `/frontend/src/config/snippets/` - Define reusable snippets -- **Route Configs**: `/frontend/src/config/routes.config.ts` - Define routes -- **API Configs**: Use existing API client patterns - -**DO NOT:** -- Hardcode page structures -- Create pages without config files -- Duplicate configuration patterns - -#### State Management - -**MUST USE Existing Stores:** -- **Zustand Stores**: Use stores from `/frontend/src/stores/` - - Auth Store, Site Store, Sector Store - - Planner Store, Writer Store, Billing Store - - Settings Store, Page Size Store, Column Visibility Store -- **React Contexts**: Use contexts from `/frontend/src/contexts/` - - Theme Context, Sidebar Context, Header Metrics Context, Toast Context - -**DO NOT:** -- Create new stores for existing functionality -- Duplicate state management logic -- Create parallel state systems - -#### Utilities & Helpers - -**MUST USE Existing Utilities:** -- **API Client**: Use `/frontend/src/services/api.ts` patterns -- **Hooks**: Use custom hooks from `/frontend/src/hooks/` -- **Utils**: Use utility functions from `/frontend/src/utils/` -- **Constants**: Use constants from `/frontend/src/constants/` - -**DO NOT:** -- Create duplicate utility functions -- Implement API calls without using existing patterns -- Create new helper functions when existing ones work - -#### CSS & Styling - -**MUST USE:** -- **Tailwind CSS**: Use Tailwind utility classes -- **Existing Styles**: Use predefined styles and classes -- **Component Styles**: Use component-level styles from existing components -- **Theme System**: Use theme context for dark/light mode - -**DO NOT:** -- Create custom CSS when Tailwind classes exist -- Duplicate styling patterns -- Create parallel style systems -- Hardcode colors or spacing values - -### Backend DRY Standards - -#### Base Classes - -**MUST USE Existing Base Classes:** -- **AccountModelViewSet**: For account-isolated models -- **SiteSectorModelViewSet**: For site/sector-scoped models -- **AccountBaseModel**: For account-isolated models -- **SiteSectorBaseModel**: For site/sector-scoped models - -**DO NOT:** -- Create new base classes when existing ones work -- Duplicate filtering logic -- Create parallel isolation systems - -#### AI Framework - -**MUST USE AI Framework:** -- **BaseAIFunction**: Inherit from this for all AI functions -- **AIEngine**: Use for executing AI functions -- **AICore**: Use for AI API calls -- **PromptRegistry**: Use for prompt management -- **run_ai_task**: Use this Celery task entry point - -**DO NOT:** -- Create new AI function patterns -- Duplicate AI execution logic -- Create parallel AI systems - -#### Utilities & Services - -**MUST USE Existing Services:** -- **CreditService**: For credit management -- **Content Normalizer**: For content processing -- **AI Functions**: Use existing 5 AI functions -- **Middleware**: Use AccountContextMiddleware, ResourceTrackerMiddleware - -**DO NOT:** -- Create duplicate service logic -- Implement credit management without CreditService -- Create parallel utility systems - -### DRY Violation Detection - -**Red Flags:** -- Creating new components when similar ones exist -- Duplicating API call patterns -- Creating new state management when stores exist -- Hardcoding values that should be config-driven -- Creating parallel systems for existing functionality - -**Action Required:** -- Check existing components, utilities, and patterns first -- Refactor to use existing systems -- Update documentation if new patterns are truly needed - ---- - -## AI Agent Instructions - -### Mandatory Reading - -**At the start of EVERY session, AI agents MUST:** -1. Read this file (`00-DOCUMENTATION-MANAGEMENT.md`) -2. Read root `README.md` -3. Read `CHANGELOG.md` -4. Understand versioning system -5. Understand changelog management -6. Understand DRY principles - -### Versioning & Changelog Rules for AI Agents - -1. **NEVER update version or changelog without user confirmation** -2. **ALWAYS ask user before adding changelog entries** -3. **ONLY update changelog after user confirms change is complete** -4. **ALWAYS follow changelog structure and format** -5. **ALWAYS reference updated documentation files in changelog** - -### DRY Principles for AI Agents - -1. **ALWAYS check for existing components/utilities before creating new ones** -2. **ALWAYS use configuration-driven approach when possible** -3. **ALWAYS use existing templates and base classes** -4. **NEVER create parallel systems** -5. **NEVER duplicate code that can be reused** - -### Documentation Update Rules for AI Agents - -1. **ALWAYS update documentation when making changes** -2. **ALWAYS update "Last Updated" date in file header** -3. **ALWAYS maintain consistency across documentation** -4. **ALWAYS verify links after updates** -5. **ALWAYS follow documentation standards** - -### Workflow for AI Agents - -**When Making Code Changes:** -1. Check existing components/utilities first (DRY) -2. Make code changes -3. Update relevant documentation -4. Wait for user confirmation -5. Add changelog entry (after confirmation) -6. Update version (if needed, after confirmation) - -**When User Confirms Fix/Feature:** -1. Add changelog entry following structure -2. Update version if needed -3. Update documentation "Last Updated" dates -4. Verify all changes are documented - -### Self-Explaining System - -This documentation management system is designed to be self-explaining: -- **Clear Rules**: All rules are explicitly stated -- **Examples**: Examples provided for clarity -- **Structure**: Consistent structure across all documents -- **Cross-References**: Links between related documents -- **Standards**: Clear standards for all operations - -**Any AI agent reading this file should understand:** -- How to manage versions -- How to update changelog -- How to follow DRY principles -- How to update documentation -- When to ask for user confirmation - ---- - -## Summary - -### Key Principles - -1. **Versioning**: Semantic versioning, only update after user confirmation -2. **Changelog**: Structured entries, only after user confirmation -3. **Documentation**: Always update when making changes -4. **DRY**: Always use existing components, utilities, and patterns -5. **Confirmation**: Never update version/changelog without user confirmation - -### Lock Status - -**Documentation Management System**: ✅ **LOCKED** - -This system is finalized and should not be changed without explicit user approval. All AI agents must follow these rules. - ---- - -**Last Updated:** 2025-01-XX (Documentation structure updated) -**Version:** 1.0.0 -**Status:** Locked - diff --git a/master-docs/01-TECH-STACK-AND-INFRASTRUCTURE.md b/master-docs/01-TECH-STACK-AND-INFRASTRUCTURE.md deleted file mode 100644 index 35435c17..00000000 --- a/master-docs/01-TECH-STACK-AND-INFRASTRUCTURE.md +++ /dev/null @@ -1,1180 +0,0 @@ -# IGNY8 Technology Stack & Infrastructure - -**Last Updated:** 2025-01-XX (Added Sites Renderer container, updated module count, fixed React Router versions) -**Purpose:** Complete technology stack, infrastructure setup, Docker deployment, and fresh installation guide for the IGNY8 platform. - ---- - -## Table of Contents - -1. [Executive Summary](#executive-summary) -2. [Technology Stack](#technology-stack) -3. [System Architecture](#system-architecture) -4. [Core Architecture Principles](#core-architecture-principles) -5. [Infrastructure Components](#infrastructure-components) -6. [External Service Integrations](#external-service-integrations) -7. [Deployment Architecture](#deployment-architecture) -8. [Complete Fresh Installation Guide](#complete-fresh-installation-guide) - ---- - -## Executive Summary - -**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system is built with modern technologies and follows a multi-tenant architecture with complete account isolation. - -### Key Metrics - -- **Architecture**: Multi-tenant SaaS with account isolation -- **Backend**: Django 5.2+ with Django REST Framework -- **Frontend**: React 19 with TypeScript -- **Database**: PostgreSQL 15 -- **Task Queue**: Celery with Redis -- **Deployment**: Docker-based containerization -- **Reverse Proxy**: Caddy (HTTPS termination) -- **AI Functions**: 5 primary AI operations -- **Modules**: 10 backend modules (Planner, Writer, System, Billing, Automation, Integration, Linker, Optimizer, Publisher, Site Builder) - ---- - -## Technology Stack - -### Backend Stack - -| Component | Technology | Version | Purpose | -|-----------|------------|---------|---------| -| **Framework** | Django | 5.2.7+ | Web framework | -| **API Framework** | Django REST Framework | Latest | RESTful API | -| **Database** | PostgreSQL | 15 | Primary database | -| **Database Adapter** | psycopg2-binary | Latest | PostgreSQL Python adapter | -| **Task Queue** | Celery | 5.3.0+ | Asynchronous tasks | -| **Cache/Broker** | Redis | 7 | Celery broker & caching | -| **Authentication** | JWT | PyJWT 2.8.0+ | Token-based auth | -| **HTTP Client** | Requests | 2.31.0+ | External API calls | -| **WSGI Server** | Gunicorn | Latest | Production server | -| **Static Files** | WhiteNoise | Latest | Static file serving | -| **HTML Parsing** | BeautifulSoup4 | 4.12.0+ | HTML parsing & processing | -| **System Utils** | psutil | 5.9.0+ | System & process utilities | -| **Docker Client** | docker | 7.0.0+ | Docker API client | -| **Django Packages** | django-filter | Latest | Advanced filtering | -| **Django Packages** | django-cors-headers | Latest | CORS handling | - -### Frontend Stack - -| Component | Technology | Version | Purpose | -|-----------|------------|---------|---------| -| **Framework** | React | 19.0.0 | UI library | -| **Framework** | React DOM | 19.0.0 | DOM rendering | -| **Language** | TypeScript | 5.7.2 | Type safety | -| **Build Tool** | Vite | 6.1.0 | Build tool & dev server | -| **Styling** | Tailwind CSS | 4.0.8 | Utility-first CSS | -| **State Management** | Zustand | 5.0.8 | Lightweight state | -| **Routing** | react-router | ^7.1.5 | Core routing library | -| **Routing** | react-router-dom | ^7.9.5 | DOM bindings for React Router | -| **HTTP Client** | Fetch API | Native | API communication | -| **SEO** | react-helmet-async | 2.0.5 | Dynamic head management | - -### Frontend UI Libraries - -| Library | Version | Purpose | -|---------|---------|---------| -| **Icons** | @heroicons/react | 2.2.0 | Icon library | -| **Charts** | apexcharts | 4.1.0 | Chart library | -| **Charts** | react-apexcharts | 1.7.0 | React wrapper for ApexCharts | -| **Calendar** | @fullcalendar/core | 6.1.15 | Calendar component | -| **Calendar** | @fullcalendar/react | 6.1.15 | React wrapper for FullCalendar | -| **Calendar** | @fullcalendar/daygrid | 6.1.15 | Day grid view | -| **Calendar** | @fullcalendar/timegrid | 6.1.15 | Time grid view | -| **Calendar** | @fullcalendar/interaction | 6.1.15 | Calendar interactions | -| **Calendar** | @fullcalendar/list | 6.1.15 | List view | -| **Maps** | @react-jvectormap/core | 1.0.4 | Vector map library | -| **Maps** | @react-jvectormap/world | 1.1.2 | World map component | -| **Carousel** | swiper | 11.2.3 | Carousel/slider component | -| **Date Picker** | flatpickr | 4.6.13 | Date/time picker | -| **Drag & Drop** | react-dnd | 16.0.1 | Drag and drop functionality | -| **Drag & Drop** | react-dnd-html5-backend | 16.0.1 | HTML5 drag & drop backend | -| **File Upload** | react-dropzone | 14.3.5 | File upload component | - -### Frontend Utilities - -| Library | Version | Purpose | -|---------|---------|---------| -| **Class Utils** | clsx | 2.1.1 | Conditional class names | -| **Tailwind Utils** | tailwind-merge | 3.0.1 | Merge Tailwind classes | - -### Frontend Development Tools - -| Tool | Version | Purpose | -|------|---------|---------| -| **Linter** | ESLint | 9.19.0 | Code linting | -| **TypeScript Linter** | typescript-eslint | 8.22.0 | TypeScript linting | -| **Vite Plugin** | @vitejs/plugin-react | 4.3.4 | React plugin for Vite | -| **SVG Plugin** | vite-plugin-svgr | 4.3.0 | SVG as React components | -| **PostCSS** | postcss | 8.5.2 | CSS processing | -| **Tailwind PostCSS** | @tailwindcss/postcss | 4.0.8 | Tailwind PostCSS plugin | - -### Infrastructure Stack - -| Component | Technology | Purpose | -|-----------|------------|---------| -| **Containerization** | Docker | Application containers | -| **Orchestration** | Docker Compose | Multi-container orchestration | -| **Reverse Proxy** | Caddy | HTTPS termination & routing | -| **Database Admin** | pgAdmin | PostgreSQL administration | -| **File Management** | FileBrowser | Web-based file management | -| **Container Management** | Portainer | Docker container management | - -### External Services - -| Service | Purpose | Integration | -|---------|---------|-------------| -| **OpenAI API** | Text generation (GPT models) | API integration | -| **OpenAI DALL-E** | Image generation | API integration | -| **Runware API** | Alternative image generation | API integration | -| **WordPress** | Content publishing | REST API integration | -| **Stripe** | Payment processing | Webhook integration (planned) | - ---- - -## System Architecture - -### High-Level Architecture - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Client Layer │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ -│ │ Browser │ │ Mobile │ │ Admin │ │ -│ │ (React) │ │ (Future) │ │ Panel │ │ -│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ -└─────────┼──────────────────┼──────────────────┼─────────────┘ - │ │ │ - └──────────────────┼──────────────────┘ - │ -┌────────────────────────────┼──────────────────────────────┐ -│ Reverse Proxy Layer │ -│ ┌───────────────┐ │ -│ │ Caddy │ │ -│ │ (HTTPS/443) │ │ -│ └───────┬───────┘ │ -└────────────────────────────┼──────────────────────────────┘ - │ -┌────────────────────────────┼──────────────────────────────┐ -│ Application Layer │ -│ ┌──────────────┐ ┌──────────────┐ │ -│ │ Frontend │ │ Backend │ │ -│ │ (React) │◄─────────────┤ (Django) │ │ -│ │ Port 8021 │ REST API │ Port 8011 │ │ -│ └──────────────┘ └──────┬───────┘ │ -│ │ │ -│ ┌────────┴────────┐ │ -│ │ Celery Worker │ │ -│ │ (Async Tasks) │ │ -│ └────────┬────────┘ │ -└───────────────────────────────────────┼──────────────────┘ - │ -┌───────────────────────────────────────┼──────────────────┐ -│ Data Layer │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ -│ │ PostgreSQL │ │ Redis │ │ Storage │ │ -│ │ (Database) │ │ (Cache/Broker)│ │ (Files) │ │ -│ └──────────────┘ └──────────────┘ └──────────────┘ │ -└──────────────────────────────────────────────────────────┘ - │ -┌───────────────────────────────────────┼──────────────────┐ -│ External Services │ -│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ -│ │ OpenAI │ │ Runware │ │ WordPress │ │ -│ │ (GPT/DALL-E)│ │ (Images) │ │ (Publish) │ │ -│ └──────────────┘ └──────────────┘ └──────────────┘ │ -└──────────────────────────────────────────────────────────┘ -``` - -### Request Flow - -``` -1. User Request - ↓ -2. Browser (React Frontend) - ↓ -3. Caddy Reverse Proxy (HTTPS Termination) - ↓ -4. Django Backend (API Endpoint) - ↓ -5. AccountContextMiddleware (Account Isolation) - ↓ -6. ViewSet (Business Logic) - ↓ -7. Serializer (Validation) - ↓ -8. Model (Database) - ↓ -9. Response (JSON) - ↓ -10. Frontend (UI Update) -``` - -### AI Task Flow - -``` -1. User Action (e.g., "Auto Cluster Keywords") - ↓ -2. Frontend API Call - ↓ -3. Backend Endpoint (ViewSet Action) - ↓ -4. Celery Task Queued - ↓ -5. Task ID Returned to Frontend - ↓ -6. Frontend Polls Progress Endpoint - ↓ -7. Celery Worker Processes Task - ↓ -8. AIProcessor Makes API Calls - ↓ -9. Results Saved to Database - ↓ -10. Progress Updates Sent - ↓ -11. Frontend Displays Results -``` - ---- - -## Core Architecture Principles - -### 1. Configuration-Driven Everything - -**Principle**: Zero HTML/JSX duplication - All UI rendered from configuration. - -**Implementation**: -- **Frontend**: Config files in `/config/pages/` and `/config/snippets/` -- **Backend**: DRF serializers and ViewSet actions -- **Templates**: 4 universal templates (Dashboard, Table, Form, System) - -**Benefits**: -- Single source of truth -- Easy maintenance -- Consistent UI/UX -- Rapid feature development - -### 2. Multi-Tenancy Foundation - -**Principle**: Complete account isolation with automatic filtering. - -**Implementation**: -- All models inherit `AccountBaseModel` -- All ViewSets inherit `AccountModelViewSet` -- Middleware injects account context from JWT -- Site > Sector hierarchy for content organization - -**Benefits**: -- Data security -- Scalability -- Resource isolation -- Simplified access control - -### 3. Template System (4 Universal Templates) - -**Principle**: Reusable templates for all page types. - -| Template | Purpose | Usage | -|----------|---------|-------| -| **DashboardTemplate** | Module home pages | KPIs, workflow steps, charts | -| **TablePageTemplate** | CRUD table pages | Keywords, Clusters, Tasks, etc. | -| **FormPageTemplate** | Settings/form pages | Settings, Integration, etc. | -| **SystemPageTemplate** | System/admin pages | Logs, Status, Monitoring | - -### 4. Unified AI Processor - -**Principle**: Single interface for all AI operations. - -**Implementation**: -- Single `AIEngine` class orchestrates all AI operations -- All AI functions inherit from `BaseAIFunction` -- Manual and automated workflows use same functions -- Account-specific API keys and model configuration - -**Benefits**: -- Code reusability -- Consistent error handling -- Unified logging -- Easy to extend - -### 5. Module-Based Organization - -**Principle**: Clear module boundaries with shared utilities. - -**Backend Modules**: -- **Planner**: Keywords, Clusters, Ideas -- **Writer**: Tasks, Content, Images -- **System**: Settings, Integrations, AI Configuration -- **Billing**: Credits, Transactions, Usage -- **Automation**: Automation workflows -- **Integration**: External integrations -- **Linker**: Internal linking -- **Optimizer**: Content optimization -- **Publisher**: Publishing workflows -- **Site Builder**: Site blueprint management -- **Auth**: Accounts, Users, Sites, Sectors - ---- - -## Infrastructure Components - -### Docker Architecture - -The system uses a two-stack Docker architecture: - -1. **Infrastructure Stack (`igny8-infra`)**: Shared services -2. **Application Stack (`igny8-app`)**: Application-specific services - -### Infrastructure Stack Services - -| Service | Container Name | Port | Purpose | -|---------|----------------|------|---------| -| **PostgreSQL** | `igny8_postgres` | 5432 (internal) | Database | -| **Redis** | `igny8_redis` | 6379 (internal) | Cache & Celery broker | -| **pgAdmin** | `igny8_pgadmin` | 5050:80 | Database administration | -| **FileBrowser** | `igny8_filebrowser` | 8080:80 | File management | -| **Caddy** | `igny8_caddy` | 80:80, 443:443 | Reverse proxy & HTTPS | -| **Setup Helper** | `setup-helper` | - | Utility container | - -### Application Stack Services - -| Service | Container Name | Port | Purpose | -|---------|----------------|------|---------| -| **Backend** | `igny8_backend` | 8011:8010 | Django REST API | -| **Frontend** | `igny8_frontend` | 8021:5173 | React application (main app) | -| **Marketing Dev** | `igny8_marketing_dev` | 8023:5174 | Marketing site (dev server) | -| **Sites Renderer** | `igny8_sites` | 8024:5176 | Serves deployed public sites (sites.igny8.com) | -| **Celery Worker** | `igny8_celery_worker` | - | Async task processing | -| **Celery Beat** | `igny8_celery_beat` | - | Scheduled tasks | - -### Network Configuration - -- **Network Name**: `igny8_net` -- **Type**: External bridge network -- **Purpose**: Inter-container communication -- **Creation**: Must be created manually before starting stacks - -### Volume Management - -**Infrastructure Volumes**: -- `pgdata`: PostgreSQL database data -- `redisdata`: Redis data -- `pgadmin_data`: pgAdmin configuration -- `filebrowser_db`: FileBrowser database -- `caddy_data`: Caddy SSL certificates -- `caddy_config`: Caddy configuration - -**Application Volumes**: -- Host mounts for application code -- Host mounts for logs -- Docker socket for container management - -### Port Allocation - -| Service | External Port | Internal Port | Access | -|---------|---------------|---------------|--------| -| **pgAdmin** | 5050 | 80 | http://localhost:5050 | -| **FileBrowser** | 8080 | 80 | http://localhost:8080 | -| **Caddy** | 80, 443 | 80, 443 | https://domain.com | -| **Backend** | 8011 | 8010 | http://localhost:8011 | -| **Frontend** | 8021 | 5173 | http://localhost:8021 | -| **Marketing Dev** | 8023 | 5174 | http://localhost:8023 | -| **Sites Renderer** | 8024 | 5176 | http://localhost:8024 | - ---- - -## External Service Integrations - -### OpenAI Integration - -**Purpose**: Text generation and image generation - -**Services Used**: -- GPT models for text generation -- DALL-E for image generation - -**Configuration**: -- API key stored per account in `IntegrationSettings` -- Model selection per account -- Cost tracking per request - -### Runware Integration - -**Purpose**: Alternative image generation service - -**Configuration**: -- API key stored per account -- Model selection (e.g., `runware:97@1`) -- Image type selection (realistic, artistic, cartoon) - -### WordPress Integration - -**Purpose**: Content publishing - -**Configuration**: -- WordPress URL per site -- Username and password stored per site -- REST API integration for publishing - -**Workflow**: -1. Content generated in IGNY8 -2. Images attached -3. Content published to WordPress via REST API -4. Status updated in IGNY8 - -### Stripe Integration (Planned) - -**Purpose**: Payment processing - -**Status**: Planned for future implementation - -**Features**: -- Subscription management -- Payment processing -- Webhook integration - ---- - -## Deployment Architecture - -### Deployment Model - -**Container-Based**: All services run in Docker containers - -**Stack Separation**: -- Infrastructure stack runs independently -- Application stack depends on infrastructure stack -- Both stacks share the same Docker network - -### Environment Configuration - -**Backend Environment Variables**: -- Database connection (PostgreSQL) -- Redis connection -- Django settings (DEBUG, SECRET_KEY, etc.) -- JWT settings -- Celery configuration - -**Frontend Environment Variables**: -- Backend API URL -- Environment (development/production) - -**Infrastructure Environment Variables**: -- PostgreSQL credentials -- pgAdmin credentials -- FileBrowser configuration - -### Health Checks - -**Backend Health Check**: -- Endpoint: `/api/v1/system/status/` -- Interval: 30 seconds -- Timeout: 10 seconds -- Retries: 3 - -**PostgreSQL Health Check**: -- Command: `pg_isready` -- Interval: 20 seconds -- Timeout: 3 seconds -- Retries: 5 - -**Redis Health Check**: -- Command: `redis-cli ping` -- Interval: 20 seconds -- Timeout: 3 seconds -- Retries: 5 - -### Scaling Considerations - -**Horizontal Scaling**: -- Multiple Celery workers can be added -- Multiple backend instances can be added (with load balancer) -- Frontend can be scaled independently - -**Vertical Scaling**: -- Database can be scaled with more resources -- Redis can be scaled with more memory -- Containers can be allocated more CPU/memory - -### Backup & Recovery - -**Database Backups**: -- PostgreSQL dumps stored in `/data/backups` -- Automated backup scripts -- Point-in-time recovery support - -**Volume Backups**: -- Docker volume backups -- Application code backups -- Configuration backups - ---- - -## Summary - -The IGNY8 platform is built on a modern, scalable architecture using: - -- **Django 5.2+** for the backend API -- **React 19** for the frontend -- **PostgreSQL 15** for data storage -- **Celery & Redis** for async processing -- **Docker** for containerization -- **Caddy** for reverse proxy and HTTPS - -The architecture follows principles of: -- Configuration-driven development -- Multi-tenancy with account isolation -- Module-based organization -- Unified AI processing -- Template-based UI rendering - -This architecture supports scalability, maintainability, and rapid feature development while ensuring data security and isolation. - ---- - -## Complete Fresh Installation Guide - -This guide covers installing IGNY8 on a fresh Ubuntu VPS from scratch, including all prerequisites, Docker, Portainer, and the complete IGNY8 application stack. - -### Prerequisites - -**System Requirements**: -- **OS**: Ubuntu 22.04 LTS or 24.04 LTS (recommended) -- **RAM**: Minimum 4GB (8GB+ recommended for production) -- **CPU**: 2+ cores (4+ cores recommended) -- **Storage**: 50GB+ free space -- **Network**: Static IP address or domain name -- **Access**: Root or sudo access - -**Domain Setup** (Optional but recommended): -- Domain name pointing to server IP -- DNS records configured: - - `igny8.com` → Server IP - - `www.igny8.com` → Server IP - - `app.igny8.com` → Server IP - - `api.igny8.com` → Server IP - ---- - -### Step 1: Initial Server Setup - -#### 1.1 Update System - -```bash -# Update package list -sudo apt update && sudo apt upgrade -y - -# Install essential tools -sudo apt install -y curl wget git vim ufw software-properties-common apt-transport-https ca-certificates gnupg lsb-release -``` - -#### 1.2 Configure Firewall - -```bash -# Enable UFW firewall -sudo ufw enable - -# Allow SSH (adjust port if needed) -sudo ufw allow 22/tcp - -# Allow HTTP and HTTPS -sudo ufw allow 80/tcp -sudo ufw allow 443/tcp - -# Allow Portainer (if accessing directly) -sudo ufw allow 9000/tcp - -# Check firewall status -sudo ufw status -``` - -#### 1.3 Create Application User (Optional but Recommended) - -```bash -# Create user for application -sudo adduser igny8 -sudo usermod -aG sudo igny8 - -# Switch to application user -su - igny8 -``` - ---- - -### Step 2: Install Docker - -#### 2.1 Install Docker Engine - -```bash -# Remove old Docker versions if any -sudo apt remove -y docker docker-engine docker.io containerd runc - -# Add Docker's official GPG key -sudo install -m 0755 -d /etc/apt/keyrings -curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg -sudo chmod a+r /etc/apt/keyrings/docker.gpg - -# Set up Docker repository -echo \ - "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ - $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ - sudo tee /etc/apt/sources.list.d/docker.list > /dev/null - -# Install Docker Engine -sudo apt update -sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin - -# Verify Docker installation -sudo docker --version -sudo docker compose version -``` - -#### 2.2 Configure Docker (Optional) - -```bash -# Add user to docker group (to run docker without sudo) -sudo usermod -aG docker $USER - -# Apply group changes (logout/login or use newgrp) -newgrp docker - -# Test Docker without sudo -docker run hello-world - -# Configure Docker to start on boot -sudo systemctl enable docker -sudo systemctl start docker -``` - ---- - -### Step 3: Install Portainer - -#### 3.1 Create Portainer Volume - -```bash -# Create volume for Portainer data -docker volume create portainer_data -``` - -#### 3.2 Run Portainer Container - -```bash -# Run Portainer Community Edition -docker run -d \ - -p 9000:9000 \ - --name=portainer \ - --restart=always \ - -v /var/run/docker.sock:/var/run/docker.sock \ - -v portainer_data:/data \ - portainer/portainer-ce:latest -``` - -#### 3.3 Access Portainer - -1. Open browser: `http://your-server-ip:9000` -2. Create admin account (first-time setup) -3. Select "Docker" environment -4. Portainer dashboard will load - -**Note**: For production, configure Portainer behind Caddy reverse proxy with HTTPS. - ---- - -### Step 4: Create Docker Network - -```bash -# Create external network for IGNY8 services -docker network create igny8_net - -# Verify network creation -docker network ls | grep igny8_net -``` - -**Important**: This network must exist before starting IGNY8 services. - ---- - -### Step 5: Prepare Application Directories - -#### 5.1 Create Directory Structure - -```bash -# Create main application directory -sudo mkdir -p /data/app -sudo chown $USER:$USER /data/app - -# Create subdirectories -mkdir -p /data/app/igny8 -mkdir -p /data/app/logs -mkdir -p /data/backups -``` - -#### 5.2 Clone or Copy IGNY8 Repository - -**Option A: Git Clone** (if repository is available): -```bash -cd /data/app -git clone igny8 -cd igny8 -``` - -**Option B: Copy Files** (if files are already available): -```bash -# Copy application files to /data/app/igny8 -# Ensure all files are in place: -# - backend/ -# - frontend/ -# - docker-compose.yml (infrastructure) -# - docker-compose.app.yml (application) -``` - ---- - -### Step 6: Install Infrastructure Stack - -#### 6.1 Navigate to Infrastructure Directory - -```bash -cd /data/app -# Infrastructure docker-compose.yml should be here -``` - -#### 6.2 Configure Infrastructure - -**Edit `docker-compose.yml`** (if needed): -- Update PostgreSQL credentials -- Update pgAdmin credentials -- Configure Caddy domain names -- Set volume paths - -#### 6.3 Start Infrastructure Stack - -```bash -# Start infrastructure services -docker compose -f docker-compose.yml -p igny8-infra up -d - -# Verify services are running -docker compose -f docker-compose.yml -p igny8-infra ps - -# Check logs -docker compose -f docker-compose.yml -p igny8-infra logs -f -``` - -**Infrastructure Services**: -- PostgreSQL (database) -- Redis (cache/broker) -- pgAdmin (database admin) -- FileBrowser (file management) -- Caddy (reverse proxy) - -#### 6.4 Verify Infrastructure - -```bash -# Check PostgreSQL -docker exec igny8_postgres pg_isready - -# Check Redis -docker exec igny8_redis redis-cli ping - -# Check all containers -docker ps | grep igny8 -``` - ---- - -### Step 7: Build Application Images - -#### 7.1 Build Backend Image - -```bash -cd /data/app/igny8/backend - -# Build backend Docker image -docker build -t igny8-backend:latest -f Dockerfile . - -# Verify image -docker images | grep igny8-backend -``` - -#### 7.2 Build Frontend Images - -```bash -cd /data/app/igny8/frontend - -# Build main app dev image -docker build -t igny8-frontend-dev:latest -f Dockerfile.dev . - -# Build marketing dev image -docker build -t igny8-marketing-dev:latest -f Dockerfile.marketing.dev . - -# Verify images -docker images | grep igny8-frontend -docker images | grep igny8-marketing -``` - -#### 7.3 Build Sites Renderer Image - -```bash -cd /data/app/igny8/sites - -# Build sites renderer dev image -docker build -t igny8-sites-dev:latest -f Dockerfile.dev . - -# Verify image -docker images | grep igny8-sites -``` - -**Note**: For production, also build production images: -```bash -# Production frontend image -docker build -t igny8-frontend:latest -f Dockerfile . - -# Production marketing image -docker build -t igny8-marketing:latest -f Dockerfile.marketing . -``` - ---- - -### Step 8: Configure Application - -#### 8.1 Configure Environment Variables - -**Backend Environment** (in `docker-compose.app.yml` or `.env`): -```bash -# Database -DB_HOST=postgres -DB_NAME=igny8_db -DB_USER=igny8 -DB_PASSWORD=your_secure_password - -# Redis -REDIS_HOST=redis -REDIS_PORT=6379 - -# Django -SECRET_KEY=your_django_secret_key -DEBUG=False -USE_SECURE_COOKIES=True -USE_SECURE_PROXY_HEADER=True - -# JWT Settings (if needed) -JWT_SECRET_KEY=your_jwt_secret_key -``` - -**Frontend Environment**: -```bash -# In docker-compose.app.yml -VITE_BACKEND_URL=https://api.igny8.com/api -``` - -#### 8.2 Configure Caddy - -**Edit Caddyfile** for domain routing: -- `igny8.com` → Marketing site -- `app.igny8.com` → Main application -- `sites.igny8.com` → Sites renderer (deployed public sites) -- `api.igny8.com` → Backend API - ---- - -### Step 9: Start Application Stack - -#### 9.1 Navigate to Application Directory - -```bash -cd /data/app/igny8 -``` - -#### 9.2 Start Application Services - -```bash -# Start application stack -docker compose -f docker-compose.app.yml -p igny8-app up -d - -# Verify services -docker compose -f docker-compose.app.yml -p igny8-app ps - -# Check logs -docker compose -f docker-compose.app.yml -p igny8-app logs -f -``` - -#### 9.3 Verify All Services - -```bash -# Check all containers -docker ps - -# Check network connectivity -docker network inspect igny8_net - -# Test backend health -curl http://localhost:8011/api/v1/system/status/ - -# Test frontend -curl http://localhost:8021 -``` - ---- - -### Step 10: Initialize Database - -#### 10.1 Run Migrations - -```bash -# Execute migrations -docker exec igny8_backend python manage.py migrate - -# Create superuser (optional) -docker exec -it igny8_backend python manage.py createsuperuser -``` - -#### 10.2 Load Initial Data (if needed) - -```bash -# Load default plans, industries, etc. -docker exec igny8_backend python manage.py loaddata initial_data.json -``` - ---- - -### Step 11: Configure Caddy for HTTPS - -#### 11.1 Update Caddyfile - -Ensure Caddyfile includes: -- Domain names -- SSL certificate auto-generation -- Reverse proxy rules - -#### 11.2 Restart Caddy - -```bash -# Restart Caddy container -docker compose -f docker-compose.yml -p igny8-infra restart igny8_caddy - -# Check Caddy logs -docker logs igny8_caddy -``` - ---- - -### Step 12: Verify Installation - -#### 12.1 Check Service Status - -```bash -# Infrastructure services -docker compose -f docker-compose.yml -p igny8-infra ps - -# Application services -docker compose -f docker-compose.app.yml -p igny8-app ps - -# All services should show "Up" status -``` - -#### 12.2 Test Access - -1. **Marketing Site**: `https://igny8.com` (or `http://your-ip:8023` for dev) -2. **Main App**: `https://app.igny8.com` (or `http://your-ip:8021` for dev) -3. **Sites Renderer**: `https://sites.igny8.com` (or `http://your-ip:8024` for dev) -4. **Backend API**: `https://api.igny8.com/api/v1/system/status/` -5. **Portainer**: `http://your-ip:9000` -6. **pgAdmin**: `http://your-ip:5050` - -#### 12.3 Check Logs - -```bash -# Backend logs -docker logs igny8_backend - -# Frontend logs -docker logs igny8_frontend - -# Marketing logs -docker logs igny8_marketing_dev - -# Sites renderer logs -docker logs igny8_sites - -# Check for errors -docker logs igny8_backend 2>&1 | grep -i error -``` - ---- - -### Step 13: Post-Installation Configuration - -#### 13.1 Configure Integration Settings - -1. Access application: `https://app.igny8.com` -2. Sign up or log in -3. Navigate to Settings → Integration -4. Configure: - - OpenAI API key - - Runware API key (optional) - - WordPress credentials (per site) - -#### 13.2 Create First Site - -1. Navigate to Settings → Sites -2. Create a new site -3. Configure WordPress integration (if needed) -4. Create sectors (1-5 per site) - -#### 13.3 Set Up Monitoring (Optional) - -- Configure health checks -- Set up log rotation -- Configure backup scripts -- Set up monitoring alerts - ---- - -### Troubleshooting - -#### Common Issues - -**1. Port Already in Use**: -```bash -# Check what's using the port -sudo lsof -i :8011 -sudo lsof -i :8021 - -# Stop conflicting services or change ports in docker-compose -``` - -**2. Network Not Found**: -```bash -# Recreate network -docker network rm igny8_net -docker network create igny8_net - -# Restart services -docker compose -f docker-compose.app.yml -p igny8-app restart -``` - -**3. Database Connection Failed**: -```bash -# Check PostgreSQL is running -docker ps | grep postgres - -# Check network connectivity -docker exec igny8_backend ping postgres - -# Verify credentials in docker-compose -``` - -**4. Permission Denied**: -```bash -# Fix directory permissions -sudo chown -R $USER:$USER /data/app -sudo chmod -R 755 /data/app -``` - -**5. Caddy SSL Issues**: -```bash -# Check Caddy logs -docker logs igny8_caddy - -# Verify domain DNS points to server -nslookup igny8.com - -# Check firewall allows port 80/443 -sudo ufw status -``` - ---- - -### Maintenance Commands - -#### View Logs -```bash -# All services -docker compose -f docker-compose.app.yml -p igny8-app logs -f - -# Specific service -docker logs -f igny8_backend -``` - -#### Restart Services -```bash -# Restart all app services -docker compose -f docker-compose.app.yml -p igny8-app restart - -# Restart specific service -docker restart igny8_backend -``` - -#### Update Application -```bash -# Pull latest code -cd /data/app/igny8 -git pull - -# Rebuild images -docker build -t igny8-backend:latest -f backend/Dockerfile ./backend -docker build -t igny8-frontend-dev:latest -f frontend/Dockerfile.dev ./frontend -docker build -t igny8-marketing-dev:latest -f frontend/Dockerfile.marketing.dev ./frontend -docker build -t igny8-sites-dev:latest -f sites/Dockerfile.dev ./sites - -# Restart services -docker compose -f docker-compose.app.yml -p igny8-app up -d -``` - -#### Backup Database -```bash -# Create backup -docker exec igny8_postgres pg_dump -U igny8 igny8_db > /data/backups/backup_$(date +%Y%m%d_%H%M%S).sql -``` - ---- - -### Security Checklist - -- [ ] Firewall configured (UFW) -- [ ] SSH key-based authentication enabled -- [ ] Strong passwords for all services -- [ ] Docker socket permissions secured -- [ ] Caddy HTTPS configured -- [ ] Environment variables secured -- [ ] Regular backups scheduled -- [ ] Log rotation configured -- [ ] Monitoring set up -- [ ] Security updates automated - ---- - -### Next Steps - -After installation: - -1. **Configure Domain**: Update DNS records and Caddy configuration -2. **Set Up Backups**: Configure automated database backups -3. **Monitor Logs**: Set up log aggregation and monitoring -4. **Performance Tuning**: Optimize Docker resources and database -5. **Security Hardening**: Review security settings and access controls - ---- - -## Summary - -The complete installation process includes: - -1. **Server Setup**: Ubuntu VPS preparation -2. **Docker Installation**: Docker Engine and Docker Compose -3. **Portainer Setup**: Container management UI -4. **Network Creation**: Docker network for services -5. **Infrastructure Stack**: PostgreSQL, Redis, Caddy, etc. -6. **Application Build**: Build Docker images -7. **Application Deployment**: Start application services -8. **Database Initialization**: Run migrations -9. **HTTPS Configuration**: Caddy SSL setup -10. **Verification**: Test all services - -This installation guide provides a complete path from a fresh Ubuntu server to a fully operational IGNY8 platform. - diff --git a/master-docs/02-APPLICATION-ARCHITECTURE.md b/master-docs/02-APPLICATION-ARCHITECTURE.md deleted file mode 100644 index 0a3f1ec2..00000000 --- a/master-docs/02-APPLICATION-ARCHITECTURE.md +++ /dev/null @@ -1,935 +0,0 @@ -# IGNY8 Application Architecture - -**Last Updated:** 2025-01-XX (Added 6 missing modules: Linker, Optimizer, Publisher, Site Builder, Automation, Integration) -**Purpose:** Complete application architecture documentation covering system hierarchy, user roles, access control, modules, workflows, data models, multi-tenancy, API architecture, and security. - ---- - -## Table of Contents - -1. [System Overview](#system-overview) -2. [System Hierarchy](#system-hierarchy) -3. [User Roles & Access Control](#user-roles--access-control) -4. [Module Organization](#module-organization) -5. [Complete Workflows](#complete-workflows) -6. [Data Models & Relationships](#data-models--relationships) -7. [Multi-Tenancy Architecture](#multi-tenancy-architecture) -8. [API Architecture](#api-architecture) -9. [Security Architecture](#security-architecture) -10. [Integration Points](#integration-points) - ---- - -## System Overview - -**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system provides a scalable, multi-account platform that enables users to manage keywords, generate content ideas, create AI-powered content, and publish to WordPress. - -### Platform Capabilities - -| Capability | Description | Module | -|------------|-------------|--------| -| **Keyword Management** | Import, organize, filter, and manage SEO keywords | Planner | -| **Keyword Clustering** | AI-powered semantic clustering of related keywords | Planner | -| **Content Ideas** | Generate content ideas from keyword clusters | Planner | -| **Content Generation** | AI-powered blog post and article generation | Writer | -| **Image Generation** | AI-powered image generation (DALL-E, Runware) | Writer | -| **WordPress Integration** | Direct publishing to WordPress sites | Writer | -| **Internal Linking** | AI-powered internal linking suggestions | Linker | -| **Content Optimization** | Content scoring and optimization suggestions | Optimizer | -| **Site Publishing** | Publishing records and site deployment | Publisher | -| **Site Blueprints** | Site structure and page blueprint management | Site Builder | -| **Automation** | Automation rules and scheduled tasks | Automation | -| **Site Integration** | External platform integrations and sync | Integration | -| **Account Management** | Multi-account SaaS with user roles | Auth | -| **Billing & Credits** | Credit-based billing system | Billing | -| **AI Configuration** | Customizable AI prompts and settings | System | - ---- - -## System Hierarchy - -### Entity Relationships - -``` -Account (1) ──< (N) User -Account (1) ──< (1) Subscription ──> (1) Plan -Account (1) ──< (N) Site -Site (1) ──< (1-5) Sector -Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks -Cluster (1) ──< (N) Keywords (Many-to-Many) -Cluster (1) ──< (N) ContentIdeas -ContentIdeas (1) ──< (N) Tasks -Task (1) ──> (1) Content -Task (1) ──< (N) Images -``` - -### Hierarchy Details - -**Account Level**: -- Top-level organization/workspace -- Contains users, sites, subscriptions, and all data -- Has credit balance and plan assignment -- Status: active, suspended, trial, cancelled - -**User Level**: -- Individual user accounts within an account -- Has role (developer, owner, admin, editor, viewer) -- Can belong to only one account -- Access controlled by role and site permissions - -**Site Level**: -- Workspace within an account (1-N relationship) -- Can have multiple active sites simultaneously -- Has WordPress integration settings (URL, username, password) -- Can be associated with an industry -- Status: active, inactive, suspended - -**Sector Level**: -- Content category within a site (1-5 per site) -- Organizes keywords, clusters, ideas, and tasks -- Can reference an industry sector template -- Status: active, inactive - -**Content Level**: -- Keywords, Clusters, ContentIdeas belong to Sector -- Tasks, Content, Images belong to Sector -- All content is automatically associated with Account and Site - ---- - -## User Roles & Access Control - -### User Roles - -| Role | Access Level | Description | Permissions | -|------|--------------|-------------|-------------| -| **Developer** | System-wide | Full system access, bypasses all restrictions | All accounts, all sites, all data, system settings | -| **Owner** | Account-wide | Full account access, can manage users and billing | All sites in account, user management, billing | -| **Admin** | Account-wide | Account admin access, can manage content and users | All sites in account, content management, user management | -| **Editor** | Site-specific | Content editing access, can manage clusters/tasks | Granted sites only, content editing | -| **Viewer** | Site-specific | Read-only access | Granted sites only, read-only | -| **System Bot** | System-wide | System automation user | All accounts, all sites, all data | - -### Access Control Matrix - -| User Role | Account Access | Site Access | Data Access | User Management | Billing | -|-----------|----------------|-------------|-------------|------------------|---------| -| Developer | All accounts | All sites | All data | Yes | Yes | -| System Bot | All accounts | All sites | All data | No | No | -| Owner | Own account | All sites in account | All data in account | Yes | Yes | -| Admin | Own account | All sites in account | All data in account | Yes | No | -| Editor | Own account | Granted sites only | Data in granted sites | No | No | -| Viewer | Own account | Granted sites only | Read-only in granted sites | No | No | - -### Site Access Control - -**Automatic Access**: -- Owners and Admins: Automatic access to all sites in their account -- Developers and System Bot: Access to all sites across all accounts - -**Explicit Access**: -- Editors and Viewers: Require explicit `SiteUserAccess` records -- Access granted by Owner or Admin -- Access can be revoked at any time - -### Account Isolation - -**Principle**: All data is automatically filtered by account. - -**Implementation**: -- All models inherit `AccountBaseModel` with `account` ForeignKey -- All ViewSets inherit `AccountModelViewSet` with automatic filtering -- Middleware sets `request.account` from JWT token -- Queries automatically filter by account (except for Developer/System Bot) - ---- - -## Module Organization - -### Module Structure - -| Module | Purpose | Models | ViewSets | Celery Tasks | -|--------|---------|--------|----------|--------------| -| **Planner** | Keyword management & content planning | Keywords, Clusters, ContentIdeas | KeywordViewSet, ClusterViewSet, ContentIdeasViewSet | auto_cluster_keywords_task, auto_generate_ideas_task | -| **Writer** | Content generation & management | Tasks, Content, Images | TasksViewSet, ContentViewSet, ImagesViewSet | auto_generate_content_task, auto_generate_images_task | -| **Linker** | Internal linking operations | - | LinkerViewSet | - | -| **Optimizer** | Content optimization & scoring | - | OptimizerViewSet | - | -| **Publisher** | Publishing records & site deployment | PublishingRecord, DeploymentRecord | PublishingRecordViewSet, DeploymentRecordViewSet, PublisherViewSet | - | -| **Site Builder** | Site blueprint management | SiteBlueprint, PageBlueprint, SiteBlueprintCluster, SiteBlueprintTaxonomy | SiteBlueprintViewSet, PageBlueprintViewSet, SiteAssetView, SiteBuilderMetadataView | - | -| **Automation** | Automation rules & scheduled tasks | AutomationRule, ScheduledTask | AutomationRuleViewSet, ScheduledTaskViewSet | - | -| **Integration** | External platform integrations | SiteIntegration | IntegrationViewSet | - | -| **System** | Settings, prompts, integrations | AIPrompt, IntegrationSettings, AuthorProfile, Strategy | AIPromptViewSet, IntegrationSettingsViewSet, AuthorProfileViewSet | - | -| **Billing** | Credits, transactions, usage | CreditTransaction, CreditUsageLog | CreditTransactionViewSet, CreditUsageLogViewSet | - | -| **Auth** | Multi-tenancy, users, accounts | Account, User, Plan, Site, Sector | AccountViewSet, UserViewSet, SiteViewSet, SectorViewSet | - | - -### Module Dependencies - -``` -Auth (Core) - ├── Planner (depends on Auth) - ├── Writer (depends on Auth, Planner) - ├── Linker (depends on Auth, Writer) - ├── Optimizer (depends on Auth, Writer) - ├── Publisher (depends on Auth, Writer, Site Builder) - ├── Site Builder (depends on Auth, Planner) - ├── Automation (depends on Auth, Planner, Writer) - ├── Integration (depends on Auth, Writer) - ├── System (depends on Auth) - └── Billing (depends on Auth) -``` - -### Module Features - -#### Planner Module - -**Purpose**: Keyword research, clustering, and content idea generation - -**Features**: -- Keyword import (CSV/manual) -- Keyword filtering and organization -- AI-powered keyword clustering -- Content idea generation from clusters -- Keyword-to-cluster mapping -- Cluster metrics and analytics - -**Models**: -- `Keywords`: Individual keywords with metrics (volume, difficulty, intent) -- `Clusters`: Groups of related keywords -- `ContentIdeas`: Content ideas generated from clusters - -#### Writer Module - -**Purpose**: Content generation, image generation, and publishing - -**Features**: -- Task creation from content ideas -- AI-powered content generation -- Content editing and review -- Image prompt extraction -- AI-powered image generation -- WordPress publishing - -**Models**: -- `Tasks`: Content generation tasks -- `Content`: Generated HTML content -- `Images`: Generated images (featured, in-article) - -#### System Module - -**Purpose**: System configuration and AI settings - -**Features**: -- AI prompt management -- Integration settings (OpenAI, Runware) -- Author profile management -- Content strategy management -- System status and monitoring - -**Models**: -- `AIPrompt`: AI prompt templates -- `IntegrationSettings`: API keys and configuration -- `AuthorProfile`: Writing style profiles -- `Strategy`: Content strategies - -#### Linker Module - -**Purpose**: Internal linking operations for content - -**Features**: -- AI-powered internal linking suggestions -- Content-to-content link analysis -- Cluster-based link recommendations -- Batch linking operations -- Link validation and management - -**ViewSets**: -- `LinkerViewSet`: Process content for internal linking - -#### Optimizer Module - -**Purpose**: Content optimization and scoring - -**Features**: -- Content analysis and scoring -- SEO optimization suggestions -- Readability improvements -- Content quality metrics -- Batch optimization operations - -**ViewSets**: -- `OptimizerViewSet`: Optimize and analyze content - -#### Publisher Module - -**Purpose**: Publishing records and site deployment - -**Features**: -- Publishing record tracking -- Deployment record management -- Site definition management -- Publishing status tracking -- Multi-destination publishing support - -**Models**: -- `PublishingRecord`: Records of content publishing -- `DeploymentRecord`: Site deployment records - -**ViewSets**: -- `PublishingRecordViewSet`: Manage publishing records -- `DeploymentRecordViewSet`: Manage deployment records -- `PublisherViewSet`: Publishing operations - -#### Site Builder Module - -**Purpose**: Site blueprint and structure management - -**Features**: -- Site blueprint creation and management -- Page blueprint management -- Cluster attachment to blueprints -- Taxonomy management -- Site structure generation -- Page content generation -- Site asset management - -**Models**: -- `SiteBlueprint`: Site structure blueprints -- `PageBlueprint`: Individual page blueprints -- `SiteBlueprintCluster`: Cluster-to-blueprint links -- `SiteBlueprintTaxonomy`: Taxonomy definitions - -**ViewSets**: -- `SiteBlueprintViewSet`: Manage site blueprints -- `PageBlueprintViewSet`: Manage page blueprints -- `SiteAssetView`: File management for site assets -- `SiteBuilderMetadataView`: Metadata for site builder - -**Note**: Site Builder Wizard UI has been removed. Blueprints can be managed via API. - -#### Automation Module - -**Purpose**: Automation rules and scheduled tasks - -**Features**: -- Automation rule creation and management -- Scheduled task management -- Rule execution tracking -- Trigger-based automation -- Conditional automation workflows - -**Models**: -- `AutomationRule`: Automation rule definitions -- `ScheduledTask`: Scheduled task records - -**ViewSets**: -- `AutomationRuleViewSet`: Manage automation rules -- `ScheduledTaskViewSet`: Manage scheduled tasks - -#### Integration Module - -**Purpose**: External platform integrations and synchronization - -**Features**: -- Site integration management (WordPress, etc.) -- Integration connection testing -- Content synchronization -- Sync health monitoring -- Multi-platform publishing support - -**Models**: -- `SiteIntegration`: Site-to-platform integration records - -**ViewSets**: -- `IntegrationViewSet`: Manage site integrations - -#### Billing Module - -**Purpose**: Credit management and usage tracking - -**Features**: -- Credit balance tracking -- Credit transactions -- Usage logging -- Cost tracking - -**Models**: -- `CreditTransaction`: Credit additions and deductions -- `CreditUsageLog`: Usage tracking with cost - -#### Auth Module - -**Purpose**: Multi-tenancy and user management - -**Features**: -- Account management -- User management -- Plan management -- Site and sector management -- Industry templates - -**Models**: -- `Account`: Top-level organization -- `User`: User accounts -- `Plan`: Subscription plans -- `Site`: Sites within accounts -- `Sector`: Sectors within sites -- `Industry`: Global industry templates -- `IndustrySector`: Industry sector templates - ---- - -## Complete Workflows - -### 1. Account Setup Workflow - -**Purpose**: Onboard a new account and user - -**Steps**: -1. User signs up via `/signup` -2. Account created with default plan -3. Owner user created and linked to account -4. User signs in via `/signin` -5. JWT token generated and returned -6. Frontend stores token and redirects to dashboard -7. User creates first site (optional) -8. User creates sectors (1-5 per site, optional) -9. User configures integration settings (OpenAI, Runware) -10. System ready for use - -**Data Created**: -- 1 Account record -- 1 User record (owner role) -- 1 Subscription record (default plan) -- 0-N Site records -- 0-N Sector records (per site) -- 1 IntegrationSettings record (per integration type) - -### 2. Keyword Management Workflow - -**Purpose**: Import, organize, and cluster keywords - -**Steps**: -1. User navigates to `/planner/keywords` -2. User imports keywords via CSV or manual entry -3. Keywords validated and stored in database -4. Keywords displayed in table with filters -5. User filters keywords by sector, status, intent, etc. -6. User selects keywords for clustering -7. User clicks "Auto Cluster" action -8. Backend validates keyword IDs -9. Celery task queued (`auto_cluster_keywords_task`) -10. Task ID returned to frontend -11. Frontend polls progress endpoint -12. Celery worker processes task: - - Loads keywords from database - - Builds AI prompt with keyword data - - Calls OpenAI API for clustering - - Parses cluster response - - Creates Cluster records - - Links keywords to clusters -13. Progress updates sent to frontend -14. Task completes -15. Frontend displays new clusters -16. User can view clusters, edit cluster names, add/remove keywords - -**Data Created**: -- N Keyword records (if imported) -- M Cluster records (created by AI) -- N-M Keyword-to-Cluster relationships - -**AI Function**: Auto Cluster Keywords - -### 3. Content Idea Generation Workflow - -**Purpose**: Generate content ideas from keyword clusters - -**Steps**: -1. User navigates to `/planner/clusters` -2. User selects one or more clusters -3. User clicks "Generate Ideas" action -4. Backend validates cluster IDs -5. Celery task queued (`auto_generate_ideas_task`) -6. Task ID returned to frontend -7. Frontend polls progress endpoint -8. Celery worker processes task: - - Loads clusters and keywords - - Builds AI prompt with cluster data - - Calls OpenAI API for idea generation - - Parses idea response - - Creates ContentIdeas records - - Links ideas to clusters -9. Progress updates sent to frontend -10. Task completes -11. Frontend displays new content ideas -12. User can view ideas, edit titles, prioritize ideas - -**Data Created**: -- N ContentIdeas records (created by AI) -- N ContentIdeas-to-Cluster relationships - -**AI Function**: Generate Ideas - -### 4. Content Generation Workflow - -**Purpose**: Generate AI-powered content from content ideas - -**Steps**: -1. User navigates to `/planner/ideas` -2. User selects one or more content ideas -3. User clicks "Create Tasks" action -4. Task records created for each idea -5. User navigates to `/writer/tasks` -6. User selects tasks for content generation -7. User clicks "Generate Content" action -8. Backend validates task IDs -9. Celery task queued (`auto_generate_content_task`) -10. Task ID returned to frontend -11. Frontend polls progress endpoint -12. Celery worker processes task: - - Loads tasks and related data (cluster, keywords, idea) - - Builds AI prompt with task data - - Calls OpenAI API for content generation - - Parses HTML content response - - Creates/updates Content records - - Updates task status -13. Progress updates sent to frontend -14. Task completes -15. Frontend displays generated content -16. User can review and edit content -17. User navigates to `/writer/content` to view content - -**Data Created**: -- N Task records (from ideas) -- N Content records (generated HTML) - -**AI Function**: Generate Content - -### 5. Image Generation Workflow - -**Purpose**: Generate images for content - -**Steps**: -1. User navigates to `/writer/content` -2. User selects content items -3. User clicks "Generate Image Prompts" action (optional, can skip) -4. Backend validates content IDs -5. Celery task queued (`auto_generate_image_prompts_task`) -6. Task processes: - - Loads content HTML - - Builds AI prompt for prompt extraction - - Calls OpenAI API - - Parses image prompts (featured, in-article) - - Creates/updates Images records with prompts -7. User clicks "Generate Images" action -8. Backend validates image IDs -9. Celery task queued (`auto_generate_images_task`) -10. Task processes: - - Loads images with prompts - - Builds image generation prompt - - Calls OpenAI DALL-E or Runware API - - Receives image URLs - - Updates Images records with URLs -11. Images displayed in frontend -12. User can view images, regenerate, or delete - -**Data Created**: -- N Images records (with prompts) -- N Images records (with image URLs) - -**AI Functions**: Generate Image Prompts, Generate Images - -### 6. WordPress Publishing Workflow - -**Purpose**: Publish content to WordPress - -**Steps**: -1. User navigates to `/writer/content` -2. User selects content to publish -3. User clicks "Publish to WordPress" action -4. Backend validates: - - Site has WordPress URL configured - - Site has WordPress credentials - - Content is ready (status: review or draft) -5. Backend calls WordPress REST API: - - Creates post with content HTML - - Uploads featured image (if available) - - Uploads in-article images (if available) - - Sets post status (draft, publish) -6. WordPress post ID stored in Content record -7. Content status updated to "published" -8. Frontend displays success message -9. User can view published content in WordPress - -**Data Updated**: -- Content record: `wp_post_id`, `status` = "published" - -**Integration**: WordPress REST API - -### 7. User Management Workflow - -**Purpose**: Add users to account and manage permissions - -**Steps**: -1. Owner/Admin navigates to `/settings/users` -2. User clicks "Add User" action -3. User enters email and selects role -4. Backend creates User record: - - Email validated (unique per account) - - Role assigned - - Account linked - - Password set (or invitation sent) -5. If role is Editor/Viewer: - - Owner/Admin grants site access - - SiteUserAccess records created -6. New user receives invitation email -7. New user signs in and accesses granted sites - -**Data Created**: -- 1 User record -- 0-N SiteUserAccess records (for Editor/Viewer roles) - -### 8. Integration Configuration Workflow - -**Purpose**: Configure AI service integrations - -**Steps**: -1. User navigates to `/settings/integration` -2. User selects integration type (OpenAI, Runware) -3. User enters API key -4. User clicks "Test Connection" (optional) -5. Backend validates API key: - - Makes test API call - - Returns connection status -6. User saves integration settings -7. Backend stores API key in IntegrationSettings -8. Integration ready for use in AI functions - -**Data Created/Updated**: -- 1 IntegrationSettings record (per integration type) - -**Test Functions**: Test OpenAI, Test Runware - ---- - -## Data Models & Relationships - -### Core Models - -**Account Model**: -- Fields: name, slug, owner, plan, credits, status -- Relationships: Users (1-N), Sites (1-N), Subscription (1-1) - -**User Model**: -- Fields: email, account, role -- Relationships: Account (N-1), SiteUserAccess (N-M via SiteUserAccess) - -**Plan Model**: -- Fields: name, price, limits (users, sites, keywords, clusters, etc.), credits -- Relationships: Accounts (1-N via Subscription) - -**Site Model**: -- Fields: name, slug, domain, industry, status, wp_url, wp_username, wp_app_password -- Relationships: Account (N-1), Sectors (1-N), Industry (N-1) - -**Sector Model**: -- Fields: name, slug, site, industry_sector, status -- Relationships: Site (N-1), IndustrySector (N-1), Keywords/Clusters/Ideas/Tasks (1-N) - -### Planner Models - -**Keywords Model**: -- Fields: keyword, volume, difficulty, intent, cluster (M-N) -- Relationships: Sector (N-1), Clusters (M-N) - -**Clusters Model**: -- Fields: name, description, keywords_count, volume -- Relationships: Sector (N-1), Keywords (M-N), ContentIdeas (1-N) - -**ContentIdeas Model**: -- Fields: idea_title, description, cluster, status -- Relationships: Sector (N-1), Cluster (N-1), Tasks (1-N) - -### Writer Models - -**Tasks Model**: -- Fields: title, description, cluster, idea, status, content -- Relationships: Sector (N-1), Cluster (N-1), ContentIdeas (N-1), Content (1-1), Images (1-N) - -**Content Model**: -- Fields: task, html_content, word_count, status, wp_post_id -- Relationships: Task (1-1), Images (1-N) - -**Images Model**: -- Fields: task, content, image_type, prompt, image_url, status -- Relationships: Task (N-1), Content (N-1) - -### System Models - -**AIPrompt Model**: -- Fields: prompt_type, prompt_value, account -- Relationships: Account (N-1) - -**IntegrationSettings Model**: -- Fields: integration_type, config (JSON), account -- Relationships: Account (N-1) - -**AuthorProfile Model**: -- Fields: name, description, tone, language, account -- Relationships: Account (N-1) - -**Strategy Model**: -- Fields: name, description, sector, prompt_types, account -- Relationships: Account (N-1), Sector (N-1) - -### Billing Models - -**CreditTransaction Model**: -- Fields: account, transaction_type, amount, balance_after -- Relationships: Account (N-1) - -**CreditUsageLog Model**: -- Fields: account, operation_type, credits_used, cost_usd -- Relationships: Account (N-1) - ---- - -## Multi-Tenancy Architecture - -### Account Isolation - -**Implementation Levels**: -1. **Model Level**: All models inherit `AccountBaseModel` with `account` ForeignKey -2. **ViewSet Level**: All ViewSets inherit `AccountModelViewSet` with automatic filtering -3. **Middleware Level**: `AccountContextMiddleware` sets `request.account` from JWT - -**Filtering**: -- All queries automatically filter by `account = request.account` -- Developer and System Bot users bypass account filtering -- System accounts bypass account filtering - -### Site/Sector Hierarchy - -**Purpose**: Organize content within accounts - -**Structure**: -``` -Account - └── Site 1 (Active) - ├── Sector 1 (Active) - │ ├── Keywords - │ ├── Clusters - │ ├── ContentIdeas - │ └── Tasks - ├── Sector 2 (Active) - └── Sector 3 (Inactive) - └── Site 2 (Active) - └── Sector 1 (Active) -``` - -**Constraints**: -- Maximum 5 active sectors per site -- Multiple sites can be active simultaneously -- All content must belong to a site and sector - -### Data Isolation Flow - -``` -Request with JWT Token - ↓ -AccountContextMiddleware - ├── Extract Account ID from JWT - ├── Load Account Object - └── Set request.account - ↓ -ViewSet.get_queryset() - ├── Check User Role - ├── Filter by Account (if not admin/developer) - └── Filter by Accessible Sites (if not owner/admin) - ↓ -Database Query - ↓ -Results (Account-Isolated) -``` - ---- - -## API Architecture - -### API Structure - -**Base URL**: `/api/v1/` - -**Module Endpoints**: -- `/api/v1/auth/` - Accounts, users, sites, sectors, plans -- `/api/v1/planner/` - Keywords, clusters, ideas -- `/api/v1/writer/` - Tasks, content, images -- `/api/v1/linker/` - Internal linking operations -- `/api/v1/optimizer/` - Content optimization operations -- `/api/v1/publisher/` - Publishing records, deployment records -- `/api/v1/site-builder/` - Site blueprints, page blueprints, assets -- `/api/v1/automation/` - Automation rules, scheduled tasks -- `/api/v1/integration/` - Site integrations, sync operations -- `/api/v1/system/` - Prompts, integrations, author-profiles, strategies -- `/api/v1/billing/` - Credits, transactions, usage - -### Authentication Flow - -``` -1. User Signs In - ↓ -2. Backend Validates Credentials - ↓ -3. JWT Token Generated - ↓ -4. Token Returned to Frontend - ↓ -5. Frontend Stores Token (localStorage) - ↓ -6. Frontend Includes Token in Requests (Authorization: Bearer {token}) - ↓ -7. Backend Validates Token - ↓ -8. Account Context Set - ↓ -9. Request Processed -``` - -### Response Format - -**Success Response**: -- `success`: true -- `data`: Response data -- `message`: Optional message - -**Error Response**: -- `success`: false -- `message`: Error message -- `errors`: Validation errors (if applicable) - -**Pagination Response**: -- `count`: Total count -- `next`: Next page URL -- `previous`: Previous page URL -- `results`: Array of results - ---- - -## Security Architecture - -### Authentication - -**Methods**: -- JWT (JSON Web Tokens) - Primary method -- Session-based auth - Fallback for admin - -**Token Management**: -- Tokens stored in localStorage -- Tokens included in `Authorization: Bearer {token}` header -- Token refresh mechanism (future implementation) - -### Authorization - -**Role-Based Access Control (RBAC)**: -- Role checked on every request -- Permissions enforced at ViewSet level -- Action-level permissions for sensitive operations - -**Data Access Control**: -- Account-level: Automatic filtering by account -- Site-level: Filtering by accessible sites -- Action-level: Permission checks in ViewSet actions - -### Data Security - -**Account Isolation**: -- All queries filtered by account -- Admin/Developer override for system accounts -- No cross-account data leakage - -**Site Access Control**: -- Users can only access granted sites -- Admin/Developer override for all sites -- Explicit access grants for Editor/Viewer roles - -**API Security**: -- CORS configured for frontend domain -- CSRF enabled for session-based auth -- Input validation via DRF serializers -- Rate limiting (future implementation) - ---- - -## Integration Points - -### OpenAI Integration - -**Purpose**: Text generation and image generation - -**Configuration**: -- API key stored per account in `IntegrationSettings` -- Model selection per account -- Cost tracking per request - -**Services Used**: -- GPT models for text generation -- DALL-E for image generation - -### Runware Integration - -**Purpose**: Alternative image generation service - -**Configuration**: -- API key stored per account -- Model selection (e.g., `runware:97@1`) -- Image type selection (realistic, artistic, cartoon) - -### WordPress Integration - -**Purpose**: Content publishing - -**Configuration**: -- WordPress URL per site -- Username and password stored per site -- REST API integration for publishing - -**Workflow**: -1. Content generated in IGNY8 -2. Images attached -3. Content published to WordPress via REST API -4. Status updated in IGNY8 - -### Stripe Integration (Planned) - -**Purpose**: Payment processing - -**Status**: Planned for future implementation - -**Features**: -- Subscription management -- Payment processing -- Webhook integration - ---- - -## Summary - -The IGNY8 application architecture provides: - -1. **Multi-Tenancy**: Complete account isolation with automatic filtering -2. **Hierarchical Organization**: Account > Site > Sector > Content structure -3. **Role-Based Access**: Granular permissions for different user roles -4. **Module-Based Design**: Clear separation of concerns across 10 backend modules -5. **Complete Workflows**: End-to-end workflows from keyword import to publishing -6. **AI Integration**: Unified AI framework for all AI operations -7. **Content Optimization**: Internal linking and content optimization tools -8. **Publishing System**: Multi-destination publishing with deployment tracking -9. **Site Builder**: Site blueprint and structure management (API-based) -10. **Automation**: Rule-based automation and scheduled tasks -11. **Integration**: External platform integrations and synchronization -12. **WordPress Integration**: Direct publishing to WordPress sites -13. **Credit System**: Credit-based billing and usage tracking -14. **Security First**: JWT auth, RBAC, data isolation -15. **Scalable Design**: Supports multiple accounts, sites, and users - -This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform. - diff --git a/master-docs/03-FRONTEND-ARCHITECTURE.md b/master-docs/03-FRONTEND-ARCHITECTURE.md deleted file mode 100644 index 7894e9fa..00000000 --- a/master-docs/03-FRONTEND-ARCHITECTURE.md +++ /dev/null @@ -1,1644 +0,0 @@ -# IGNY8 Frontend Architecture Documentation - -**Last Updated:** 2025-01-XX (Added Sites Renderer application, updated routing with Linker/Optimizer/Sites modules, fixed React Router versions) -**Version:** 1.0 -**Purpose:** Master-level architecture and functional documentation for the IGNY8 frontend application, covering structure, routing, state management, build system, and all functional details. - ---- - -## Table of Contents - -1. [Executive Summary](#executive-summary) -2. [Prerequisites](#prerequisites) -3. [Technology Stack](#technology-stack) -4. [Application Architecture Overview](#application-architecture-overview) -5. [Project Structure](#project-structure) -6. [Dual Application Architecture](#dual-application-architecture) -7. [Routing Architecture](#routing-architecture) -8. [State Management](#state-management) -9. [Component Architecture](#component-architecture) -10. [Template System](#template-system) -11. [API Integration Layer](#api-integration-layer) -12. [Build System & Configuration](#build-system--configuration) -13. [Development Workflow](#development-workflow) -14. [Deployment Architecture](#deployment-architecture) -15. [Key Features & Functions](#key-features--functions) -16. [Performance Optimizations](#performance-optimizations) -17. [Security Considerations](#security-considerations) - ---- - -## Executive Summary - -The IGNY8 frontend is a modern, triple-application React-based system consisting of: - -1. **Main Application** (`app.igny8.com`): A full-featured SaaS platform for SEO content management with authentication, multi-tenancy, and complex workflows -2. **Marketing Site** (`igny8.com`): A public-facing marketing website with SEO-optimized pages -3. **Sites Renderer** (`sites.igny8.com`): A public-facing site renderer that serves deployed IGNY8-hosted sites - -All three applications share components and utilities but are built and deployed separately, using different entry points, routing, and build configurations. - -### Key Characteristics - -- **Framework**: React 19 with TypeScript -- **Build Tool**: Vite 6.1.0 -- **Styling**: Tailwind CSS 4.0 -- **State Management**: Zustand with persistence -- **Routing**: React Router v7 -- **Code Splitting**: Lazy loading for optimal performance -- **Architecture**: Component-based with template system - ---- - -## Prerequisites - -### Development Environment - -| Requirement | Version | Purpose | -|------------|---------|---------| -| **Node.js** | 18+ | JavaScript runtime | -| **npm** | 9+ | Package manager | -| **TypeScript** | 5.7+ | Type checking | -| **Docker** | 20+ | Containerization (optional) | -| **Git** | 2.30+ | Version control | - -### Runtime Dependencies - -| Dependency | Version | Purpose | -|------------|---------|---------| -| **React** | 19.0.0 | UI library | -| **React DOM** | 19.0.0 | DOM rendering | -| **react-router** | ^7.1.5 | Core routing library | -| **react-router-dom** | ^7.9.5 | DOM bindings for React Router | -| **Zustand** | 5.0.8 | State management | -| **Vite** | 6.1.0 | Build tool & dev server | - -### External Services - -- **Backend API**: Django REST Framework API (default: `https://api.igny8.com/api`) -- **Authentication**: JWT-based (access + refresh tokens) -- **Reverse Proxy**: Caddy (for production HTTPS) - ---- - -## Technology Stack - -### Core Framework & Libraries - -```json -{ - "react": "^19.0.0", - "react-dom": "^19.0.0", - "react-router": "^7.1.5", - "react-router-dom": "^7.9.5", - "zustand": "^5.0.8", - "typescript": "~5.7.2" -} -``` - -### UI & Styling - -| Library | Version | Purpose | -|---------|---------|---------| -| **Tailwind CSS** | 4.0.8 | Utility-first CSS framework | -| **@heroicons/react** | 2.2.0 | Icon library | -| **clsx** | 2.1.1 | Conditional class names | -| **tailwind-merge** | 3.0.1 | Merge Tailwind classes | - -### Data Visualization - -| Library | Version | Purpose | -|---------|---------|---------| -| **apexcharts** | 4.1.0 | Chart library | -| **react-apexcharts** | 1.7.0 | React wrapper for ApexCharts | - -### Forms & Inputs - -| Library | Version | Purpose | -|---------|---------|---------| -| **flatpickr** | 4.6.13 | Date/time picker | -| **react-dropzone** | 14.3.5 | File upload component | -| **react-dnd** | 16.0.1 | Drag and drop | - -### SEO & Meta Tags - -| Library | Version | Purpose | -|---------|---------|---------| -| **react-helmet-async** | 2.0.5 | Dynamic head management | - -### Build & Development Tools - -| Tool | Version | Purpose | -|------|---------|---------| -| **Vite** | 6.1.0 | Build tool & dev server | -| **@vitejs/plugin-react** | 4.3.4 | React plugin for Vite | -| **vite-plugin-svgr** | 4.3.0 | SVG as React components | -| **ESLint** | 9.19.0 | Code linting | -| **TypeScript ESLint** | 8.22.0 | TypeScript linting | - ---- - -## Application Architecture Overview - -### High-Level Architecture - -``` -┌─────────────────────────────────────────────────────────────┐ -│ Frontend Application │ -├─────────────────────────────────────────────────────────────┤ -│ │ -│ ┌──────────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐ │ -│ │ Main Application │ │ Marketing Site │ │ Sites Renderer │ │ -│ │ (app.igny8.com) │ │ (igny8.com) │ │ (sites.igny8.com) │ │ -│ ├──────────────────────┤ ├──────────────────────┤ ├──────────────────────┤ │ -│ │ • Authentication │ │ • Public Pages │ │ • Public Sites │ │ -│ │ • Multi-tenant │ │ • SEO Optimized │ │ • Dynamic Content │ │ -│ │ • Protected Routes │ │ • No Authentication │ │ • No Authentication │ │ -│ │ • Complex Workflows │ │ • Static Content │ │ • Site Blueprints │ │ -│ └──────────────────────┘ └──────────────────────┘ └──────────────────────┘ │ -│ │ │ │ -│ └──────────┬───────────────────┘ │ -│ │ │ -│ ┌──────────▼──────────┐ │ -│ │ Shared Components │ │ -│ │ & Utilities │ │ -│ └──────────┬──────────┘ │ -│ │ │ -│ ┌──────────▼──────────┐ │ -│ │ API Service Layer │ │ -│ │ (services/api.ts) │ │ -│ └──────────┬──────────┘ │ -│ │ │ -│ ┌──────────▼──────────┐ │ -│ │ Backend API │ │ -│ │ (Django REST) │ │ -│ └─────────────────────┘ │ -└─────────────────────────────────────────────────────────────┘ -``` - -### Application Entry Points - -#### Main Application Entry -- **File**: `src/main.tsx` -- **HTML**: `index.html` -- **Port**: 5173 (dev), 8021 (Docker) -- **Domain**: `app.igny8.com` - -#### Marketing Site Entry -- **File**: `src/marketing/index.tsx` -- **HTML**: `marketing.html` -- **Port**: 5174 (dev), 8023 (Docker) -- **Domain**: `igny8.com` - -#### Sites Renderer Entry -- **File**: `sites/src/main.tsx` (separate repository/package) -- **HTML**: `sites/index.html` -- **Port**: 5176 (dev), 8024 (Docker) -- **Domain**: `sites.igny8.com` -- **Purpose**: Renders deployed public sites from Site Blueprints - ---- - -## Project Structure - -### Directory Hierarchy - -``` -frontend/ -├── public/ # Static assets -│ ├── favicon.png -│ └── images/ -├── src/ # Source code -│ ├── components/ # Reusable UI components -│ │ ├── auth/ # Authentication components -│ │ ├── charts/ # Chart components -│ │ ├── common/ # Common utilities -│ │ ├── dashboard/ # Dashboard-specific -│ │ ├── form/ # Form components -│ │ ├── header/ # Header components -│ │ ├── tables/ # Table components -│ │ ├── tasks/ # Task-related -│ │ ├── ui/ # Base UI components -│ │ └── UserProfile/ # User profile -│ ├── config/ # Configuration files -│ │ ├── forms/ # Form configurations -│ │ ├── pages/ # Page configurations -│ │ ├── routes.config.ts # Route metadata -│ │ └── snippets/ # Code snippets -│ ├── context/ # React contexts -│ │ ├── ThemeContext.tsx -│ │ ├── SidebarContext.tsx -│ │ └── HeaderMetricsContext.tsx -│ ├── hooks/ # Custom React hooks -│ ├── icons/ # Icon components -│ ├── layout/ # Layout components -│ │ ├── AppLayout.tsx # Main app layout -│ │ ├── AppHeader.tsx # Header component -│ │ └── AppSidebar.tsx # Sidebar navigation -│ ├── marketing/ # Marketing site -│ │ ├── components/ # Marketing components -│ │ ├── config/ # Marketing config -│ │ ├── data/ # Marketing data -│ │ ├── layout/ # Marketing layout -│ │ ├── pages/ # Marketing pages -│ │ ├ ├── Home.tsx -│ │ ├ ├── Product.tsx -│ │ ├ ├── Solutions.tsx -│ │ ├ ├── Pricing.tsx -│ │ └ └── ... -│ │ ├── styles/ # Marketing styles -│ │ ├── index.tsx # Marketing entry -│ │ └── MarketingApp.tsx # Marketing router -│ ├── pages/ # Main app pages -│ │ ├── AuthPages/ # Login/Signup -│ │ ├── Dashboard/ # Dashboard pages -│ │ ├── Planner/ # Planner module -│ │ ├── Writer/ # Writer module -│ │ ├── Thinker/ # Thinker module -│ │ ├── Billing/ # Billing module -│ │ ├── Settings/ # Settings pages -│ │ └── Help/ # Help pages -│ ├── services/ # API services -│ │ └── api.ts # Centralized API client -│ ├── store/ # Zustand stores -│ │ ├── authStore.ts # Authentication state -│ │ ├── siteStore.ts # Site management -│ │ ├── sectorStore.ts # Sector management -│ │ ├── plannerStore.ts # Planner state -│ │ ├── billingStore.ts # Billing state -│ │ └── ... -│ ├── styles/ # Global styles -│ │ ├── index.css # Main styles -│ │ └── igny8-colors.css # Custom colors -│ ├── templates/ # Page templates -│ │ ├── DashboardTemplate.tsx -│ │ ├── TablePageTemplate.tsx -│ │ ├── FormPageTemplate.tsx -│ │ ├── SystemPageTemplate.tsx -│ │ └── ContentViewTemplate.tsx -│ ├── utils/ # Utility functions -│ ├── App.tsx # Main app router -│ └── main.tsx # Main entry point -├── index.html # Main app HTML -├── marketing.html # Marketing HTML -├── vite.config.ts # Vite configuration -├── tsconfig.json # TypeScript config -├── package.json # Dependencies -├── Dockerfile* # Docker files -└── sites/ # Sites Renderer (separate app) - ├── src/ # Sites renderer source - ├── index.html # Sites renderer HTML - ├── vite.config.ts # Sites Vite config - ├── package.json # Sites dependencies - └── Dockerfile.dev # Sites dev Dockerfile -``` - ---- - -## Triple Application Architecture - -### Main Application (`app.igny8.com`) - -**Purpose**: Authenticated SaaS platform for content management - -**Characteristics**: -- Requires authentication (JWT tokens) -- Multi-tenant architecture -- Protected routes with role-based access -- Complex state management -- Real-time data updates -- Module-based organization (Planner, Writer, Thinker, Billing) - -**Entry Point**: `src/main.tsx` → `src/App.tsx` - -**Provider Hierarchy**: -```typescript - - - - - - - - - - - -``` - -**Global Components**: -- `GlobalErrorDisplay` - Displays global errors -- `LoadingStateMonitor` - Tracks loading states -- `ScrollToTop` - Scrolls to top on route change -- `ProtectedRoute` - Route protection wrapper - -**Key Features**: -- User authentication & authorization -- Site & sector management -- Keyword management (Planner) -- Content generation (Writer) -- AI configuration (Thinker) -- Billing & credits -- Settings & integrations - -### Marketing Site (`igny8.com`) - -**Purpose**: Public-facing marketing website - -**Characteristics**: -- No authentication required -- Static content with SEO optimization -- Simple routing -- Minimal state management -- Fast loading times -- SEO meta tags - -**Entry Point**: `src/marketing/index.tsx` → `src/marketing/MarketingApp.tsx` - -**Pages**: -- `/` - Home -- `/product` - Product overview -- `/solutions` - Solutions by industry -- `/pricing` - Pricing plans -- `/tour` - Product tour -- `/resources` - Resources & guides -- `/case-studies` - Case studies -- `/partners` - Partner program -- `/contact` - Contact page -- `/waitlist` - Waitlist signup - -**SEO Features**: -- Dynamic meta tags via `react-helmet-async` -- SEO component for each page -- Optimized meta descriptions -- Open Graph tags -- Twitter Card tags - -### Sites Renderer (`sites.igny8.com`) - -**Purpose**: Public-facing site renderer for deployed IGNY8-hosted sites - -**Characteristics**: -- No authentication required -- Dynamic content from Site Blueprints -- Public access to deployed sites -- Separate codebase (`sites/` directory) -- React Router for site navigation -- API integration for blueprint data - -**Entry Point**: `sites/src/main.tsx` → `sites/src/App.tsx` - -**Technology Stack**: -- React 19.2.0 -- React Router DOM 7.9.6 -- TypeScript 5.9.3 -- Vite 7.2.2 -- Zustand 5.0.8 -- Axios 1.13.2 - -**Features**: -- Loads site definitions from backend API -- Renders pages from Page Blueprints -- Dynamic routing based on site structure -- SEO-optimized pages -- Responsive design - -**Data Flow**: -1. Site slug extracted from URL -2. API call to fetch Site Blueprint -3. Page Blueprints loaded -4. Content rendered from blueprints -5. Navigation generated from site structure - ---- - -## Routing Architecture - -### Main Application Routing - -**Router**: React Router v7 (`BrowserRouter`) - -**Structure**: -```typescript - - - {/* Public Routes */} - } /> - } /> - - {/* Protected Routes */} - }> - } /> - - {/* Planner Module */} - } /> - } /> - } /> - } /> - } /> - - {/* Writer Module */} - } /> - } /> - } /> - } /> - } /> - } /> - } /> - - {/* Thinker Module */} - } /> - } /> - } /> - } /> - } /> - } /> - - {/* Billing Module */} - } /> - } /> - } /> - - {/* Reference Data */} - } /> - } /> - - {/* Linker Module */} - } /> - } /> - - {/* Optimizer Module */} - } /> - } /> - } /> - - {/* Automation */} - } /> - {/* Note: Schedules functionality is integrated into Automation Dashboard */} - - {/* Sites Module */} - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - - {/* Settings */} - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - } /> - - {/* Help */} - } /> - } /> - } /> - } /> - - {/* UI Elements Showcase */} - } /> - } /> - } /> - - {/* Fallback */} - } /> - - - -``` - -**Route Protection**: -- `ProtectedRoute` component checks authentication -- Redirects to `/signin` if not authenticated -- Validates JWT token on route access - -**Lazy Loading**: -- All page components are lazy-loaded using `React.lazy()` -- Reduces initial bundle size -- Improves time-to-interactive - -### Sites Renderer Routing - -**Router**: React Router v7 (`BrowserRouter`) - -**Structure**: -- Dynamic routing based on Site Blueprint structure -- Routes generated from Page Blueprints -- Public access (no authentication) -- SEO-friendly URLs - -**Key Routes**: -- `/:siteSlug` - Site home page -- `/:siteSlug/:pageSlug` - Individual pages -- Dynamic navigation from site structure - -**Data Loading**: -- Site Blueprint loaded on mount -- Page Blueprints loaded per route -- Content rendered from blueprint data - -### Marketing Site Routing - -**Router**: React Router v7 (`BrowserRouter`) - -**Structure**: -```typescript - - - - - } /> - } /> - } /> - {/* ... more routes ... */} - - - -``` - -**SPA Routing**: -- Client-side routing for all pages -- Server configuration (Caddy/Vite) serves `marketing.html` for all routes -- Enables direct URL access and browser navigation - -**ScrollToTop Component**: -- Automatically scrolls to top on route change -- Ensures proper navigation experience - ---- - -## State Management - -### Zustand Architecture - -**Library**: Zustand 5.0.8 with persistence middleware - -**Store Structure**: -``` -store/ -├── authStore.ts # Authentication & user -├── siteStore.ts # Active site management -├── sectorStore.ts # Active sector management -├── plannerStore.ts # Planner module state -├── billingStore.ts # Billing & credits -├── settingsStore.ts # Application settings -├── pageSizeStore.ts # Table pagination -└── columnVisibilityStore.ts # Table column visibility -``` - -### Context Providers - -**React Context Architecture**: - -The application uses React Context for cross-component state sharing: - -**1. ThemeContext** (`src/context/ThemeContext.tsx`): -- Manages light/dark theme -- Persists theme preference in localStorage -- Applies theme class to document root -- Provides `useTheme()` hook - -**2. SidebarContext** (`src/context/SidebarContext.tsx`): -- Manages sidebar state (expanded/collapsed, mobile open/closed) -- Tracks active menu items and submenus -- Provides `useSidebar()` hook -- Wrapped in `SidebarProvider` component - -**3. HeaderMetricsContext** (`src/context/HeaderMetricsContext.tsx`): -- Manages header metrics and statistics -- Provides metrics display functionality -- Used for dashboard KPIs and statistics - -**4. ToastProvider** (`src/components/ui/toast/ToastContainer.tsx`): -- Global toast notification system -- Provides `useToast()` hook -- Methods: `success()`, `error()`, `warning()`, `info()` - -**Context Provider Hierarchy**: -```typescript - - - - - - - - - -``` - -### Custom Hooks - -**Hook Library** (`src/hooks/`): - -**1. useErrorHandler**: -- Global error handling system -- Tracks errors with source and timestamp -- Provides `addError()`, `clearError()`, `clearAllErrors()` -- Maintains last 10 errors - -**2. usePageDataLoader**: -- Standardized data loading pattern -- Handles loading states and errors -- Auto-load and manual load modes -- Dependency-based reloading - -**3. useModal**: -- Modal state management -- Open/close functionality -- Modal data management - -**4. useGoBack**: -- Navigation history management -- Back button functionality - -**5. usePersistentToggle**: -- Toggle state with localStorage persistence -- Remembers user preferences - -**6. useProgressModal**: -- Progress tracking for long operations -- Multi-step progress display - -**7. useResourceDebug**: -- Resource tracking and debugging -- Development/debugging utilities - -### Store Patterns - -#### Authentication Store (`authStore.ts`) - -**State**: -```typescript -interface AuthState { - user: User | null; - token: string | null; - refreshToken: string | null; - isAuthenticated: boolean; - loading: boolean; -} -``` - -**Actions**: -- `login(email, password)` - Authenticate user -- `logout()` - Clear authentication -- `register(data)` - Create new account -- `refreshToken()` - Refresh JWT token -- `refreshUser()` - Update user data - -**Persistence**: LocalStorage (`auth-storage`) - -#### Site Store (`siteStore.ts`) - -**State**: -```typescript -interface SiteState { - sites: Site[]; - activeSite: Site | null; - loading: boolean; -} -``` - -**Actions**: -- `loadSites()` - Fetch all sites -- `loadActiveSite()` - Load current active site -- `setActiveSite(site)` - Switch active site -- `createSite(data)` - Create new site -- `updateSite(id, data)` - Update site -- `deleteSite(id)` - Delete site - -**Persistence**: LocalStorage (`site-storage`) - -#### Sector Store (`sectorStore.ts`) - -**State**: -```typescript -interface SectorState { - sectors: Sector[]; - activeSector: Sector | null; - loading: boolean; -} -``` - -**Actions**: -- `loadSectorsForSite(siteId)` - Load sectors for site -- `setActiveSector(sector)` - Switch active sector -- `createSector(data)` - Create new sector -- `updateSector(id, data)` - Update sector -- `deleteSector(id)` - Delete sector -- `clearActiveSector()` - Clear active sector - -**Persistence**: LocalStorage (`sector-storage`) - -#### Settings Store (`settingsStore.ts`) - -**State**: -```typescript -interface SettingsState { - accountSettings: Record; - moduleSettings: Record>; - loading: boolean; - error: string | null; -} -``` - -**Actions**: -- `loadAccountSettings()` - Load all account settings -- `loadAccountSetting(key)` - Load specific account setting -- `updateAccountSetting(key, value)` - Update account setting -- `loadModuleSettings(moduleName)` - Load module settings -- `updateModuleSetting(moduleName, key, value)` - Update module setting -- `reset()` - Reset store state - -**Persistence**: LocalStorage (`settings-storage`) - -#### Page Size Store (`pageSizeStore.ts`) - -**State**: -```typescript -interface PageSizeStore { - pageSizes: Record; - setPageSize: (key: string, size: number) => void; - getPageSize: (key: string) => number; -} -``` - -**Purpose**: Manages pagination page sizes per table/page -**Persistence**: LocalStorage (`page-size-storage`) - -#### Column Visibility Store (`columnVisibilityStore.ts`) - -**State**: -```typescript -interface ColumnVisibilityState { - visibleColumns: Record; - setVisibleColumns: (key: string, columns: string[]) => void; - toggleColumn: (key: string, column: string) => void; -} -``` - -**Purpose**: Manages table column visibility preferences -**Persistence**: LocalStorage (`column-visibility-storage`) - -### Store Usage Pattern - -```typescript -// In components -import { useAuthStore } from '../store/authStore'; -import { useSiteStore } from '../store/siteStore'; - -function MyComponent() { - const { user, isAuthenticated } = useAuthStore(); - const { activeSite, setActiveSite } = useSiteStore(); - - // Component logic -} -``` - ---- - -## Component Architecture - -### Component Hierarchy - -``` -App -├── ErrorBoundary -├── ThemeProvider -├── HeaderMetricsProvider -├── ToastProvider -└── Router - ├── GlobalErrorDisplay - ├── LoadingStateMonitor - └── Routes - ├── Public Routes (SignIn, SignUp) - └── Protected Routes - └── AppLayout - ├── AppHeader - ├── AppSidebar - └── Outlet (Page Content) - └── Page Components - └── Template Components - └── UI Components -``` - -### Component Categories - -#### 1. Layout Components (`src/layout/`) - -**AppLayout.tsx**: -- Main application layout wrapper -- Manages sidebar, header, and content area -- Handles site/sector initialization -- Provides context for child components - -**Key Features**: -- **Site Initialization**: Automatically loads active site on mount -- **Sector Loading**: Loads sectors when active site changes -- **User Refresh**: Periodic user data refresh (every 2 minutes, on visibility change, on focus) -- **Version Detection**: Detects app version changes and refreshes user data -- **Timeout Handling**: 35-second timeout for site/sector loading -- **Error Handling**: Integrated error handling with useErrorHandler -- **Debug Overlay**: Resource tracking overlay (enabled via localStorage flag) -- **Loading State Tracking**: Integrates with LoadingStateMonitor -- **Responsive Layout**: Adapts to sidebar state (expanded/collapsed, mobile) - -**AppHeader.tsx**: -- Top navigation bar -- User menu, notifications, site selector -- Breadcrumbs and page title - -**AppSidebar.tsx**: -- Left navigation menu -- Module navigation (Planner, Writer, Thinker, Billing) -- Collapsible/expandable -- Mobile-responsive - -#### 2. Page Components (`src/pages/`) - -**Organization**: -- Module-based folders (Planner/, Writer/, Thinker/, Billing/) -- Feature-based pages within modules -- Shared pages (Dashboard/, Settings/, Help/) - -**Pattern**: -- Pages use templates for consistent structure -- Pages handle data fetching and business logic -- Pages compose UI components - -#### 3. Template Components (`src/templates/`) - -**DashboardTemplate.tsx**: -- Module dashboard pages -- KPI cards, workflow steps, charts -- Recent activity sections - -**TablePageTemplate.tsx**: -- CRUD table pages -- Filtering, sorting, pagination -- Bulk actions, export -- Column visibility controls - -**FormPageTemplate.tsx**: -- Settings and form pages -- Sectioned forms -- Save/cancel actions -- Validation display - -**SystemPageTemplate.tsx**: -- System/admin pages -- Status cards, logs, monitoring -- System information - -**ContentViewTemplate.tsx**: -- Individual content view -- Metadata display -- Content rendering - -#### 4. UI Components (`src/components/ui/`) - -**Base Components**: -- Button, Input, Select, Checkbox, Radio -- Table, Modal, Dropdown, Toast -- Card, Badge, Avatar, Spinner -- Pagination, Tabs, Tooltip - -**Pattern**: Reusable, styled with Tailwind CSS - -#### 5. Common Components (`src/components/common/`) - -**Utilities**: -- ScrollToTop - Route change scroll -- GlobalErrorDisplay - Error handling -- LoadingStateMonitor - Loading states -- ErrorBoundary - React error boundaries -- ColumnSelector - Table column management - ---- - -## Template System - -### Template Architecture - -The application uses 4 universal templates for consistent page structure: - -#### 1. DashboardTemplate - -**Purpose**: Module home pages - -**Features**: -- KPI cards with metrics -- Workflow step indicators -- Charts and visualizations -- Recent activity feeds - -**Usage**: -```typescript - -``` - -#### 2. TablePageTemplate - -**Purpose**: CRUD table pages - -**Features**: -- Data table with sorting -- Filtering and search -- Pagination -- Bulk actions -- Export functionality -- Column visibility controls - -**Usage**: -```typescript - -``` - -#### 3. FormPageTemplate - -**Purpose**: Settings and form pages - -**Features**: -- Sectioned form layout -- Field grouping -- Save/cancel actions -- Validation display -- Loading states - -**Usage**: -```typescript - -``` - -#### 4. SystemPageTemplate - -**Purpose**: System/admin pages - -**Features**: -- Status cards -- Log displays -- System metrics -- Monitoring dashboards - -**Usage**: -```typescript - -``` - ---- - -## API Integration Layer - -### API Service Architecture - -**File**: `src/services/api.ts` - -**Purpose**: Centralized API client with automatic configuration - -### API Configuration - -**Base URL Detection**: -```typescript -function getApiBaseUrl(): string { - // 1. Check environment variables - const envUrl = import.meta.env.VITE_BACKEND_URL; - if (envUrl) return envUrl; - - // 2. Auto-detect from origin - const origin = window.location.origin; - - // 3. Localhost/IP detection - if (origin.includes('localhost')) { - return origin.replace(':3000', ':8011') + '/api'; - } - - // 4. Production default - return 'https://api.igny8.com/api'; -} -``` - -**Authentication**: -- JWT tokens from Zustand store -- Automatic token injection in headers -- Token refresh on 401 responses -- Site/sector context injection - -### API Functions - -**Pattern**: Module-based function organization - -**Example Structure**: -```typescript -// Keywords API -export const keywordsApi = { - list: (params) => fetch('/api/v1/planner/keywords/', { params }), - create: (data) => fetch('/api/v1/planner/keywords/', { method: 'POST', body: data }), - update: (id, data) => fetch(`/api/v1/planner/keywords/${id}/`, { method: 'PATCH', body: data }), - delete: (id) => fetch(`/api/v1/planner/keywords/${id}/`, { method: 'DELETE' }), -}; - -// Content API -export const contentApi = { - list: (params) => fetch('/api/v1/writer/content/', { params }), - create: (data) => fetch('/api/v1/writer/content/', { method: 'POST', body: data }), - // ... more functions -}; -``` - -**Request Interceptors**: -- Automatic site/sector context injection -- JWT token attachment -- Error handling -- Loading state management - -**Response Handling**: -- JSON parsing -- Error extraction -- Status code handling -- Token refresh on 401 - ---- - -## Build System & Configuration - -### Vite Configuration - -**File**: `vite.config.ts` - -**Key Features**: - -#### 1. Dual Build Support - -```typescript -const isMarketingBuild = mode === "marketing"; - -build: { - rollupOptions: { - input: isMarketingBuild - ? { marketing: resolve(__dirname, "marketing.html") } - : { - main: resolve(__dirname, "index.html"), - marketing: resolve(__dirname, "marketing.html"), - }, - }, -} -``` - -#### 2. Code Splitting - -- Automatic code splitting by route -- Lazy-loaded components -- Vendor chunk separation -- CSS code splitting - -#### 3. Development Server - -**Main App** (Port 5173): -- Hot Module Replacement (HMR) -- WebSocket for live updates -- Proxy configuration for API - -**Marketing Site** (Port 5174): -- SPA routing plugin -- Marketing-specific configuration -- SEO-friendly development - -**Sites Renderer** (Port 5176): -- Separate Vite configuration (`sites/vite.config.ts`) -- Public site rendering -- API integration for blueprints -- Dynamic routing from site structure - -#### 4. Custom Plugins - -**bypassHostCheck**: -- Workaround for Vite 6.1.0 host check bug -- Allows external connections in Docker - -**serveMarketingSPA**: -- SPA routing for marketing dev server -- Serves `marketing.html` for all routes -- Enables direct URL access - -#### 5. Optimization - -**Dependency Pre-bundling**: -- Frequently used dependencies pre-bundled -- Heavy dependencies excluded (lazy-loaded) -- ESBuild for fast bundling - -**Build Optimization**: -- Minification with ESBuild -- Source maps in dev only -- Chunk size warnings -- Compressed size reporting - -### TypeScript Configuration - -**Files**: -- `tsconfig.json` - Base configuration -- `tsconfig.app.json` - Application config -- `tsconfig.node.json` - Node/build config - -**Settings**: -- Strict type checking -- ES2020 target -- React JSX support -- Path aliases - -### Environment Variables - -**Development**: -```bash -VITE_BACKEND_URL=https://api.igny8.com/api -VITE_ALLOWED_HOSTS=igny8.com,localhost -``` - -**Production**: -- Set via Docker environment -- Caddy reverse proxy configuration -- Domain-based routing - ---- - -## Development Workflow - -### Local Development Setup - -#### 1. Install Dependencies - -```bash -cd frontend -npm install -``` - -#### 2. Start Development Server - -**Main Application**: -```bash -npm run dev -# Runs on http://localhost:5173 -``` - -**Marketing Site**: -```bash -npm run dev:marketing -# Runs on http://localhost:5174 -``` - -**Sites Renderer**: -```bash -cd sites -npm run dev -# Runs on http://localhost:5176 -``` - -#### 3. Build for Production - -**Main Application**: -```bash -npm run build -# Output: dist/ -``` - -**Marketing Site**: -```bash -npm run build:marketing -# Output: dist/marketing.html -``` - -**Sites Renderer**: -```bash -cd sites -npm run build -# Output: sites/dist/ -``` - -### Docker Development - -#### Main App Container - -```bash -# Build image -docker build -t igny8-frontend-dev -f Dockerfile.dev frontend/ - -# Run container -docker run -p 8021:5173 \ - -v $(pwd)/frontend:/app \ - -e VITE_BACKEND_URL=https://api.igny8.com/api \ - igny8-frontend-dev -``` - -#### Marketing Container - -```bash -# Build image -docker build -t igny8-marketing-dev -f Dockerfile.marketing.dev frontend/ - -# Run container -docker run -p 8023:5174 \ - -v $(pwd)/frontend:/app \ - igny8-marketing-dev -``` - -### Development Scripts - -| Script | Command | Purpose | -|--------|---------|---------| -| `dev` | `vite` | Start main app dev server | -| `dev:marketing` | `PORT=5174 vite --port 5174` | Start marketing dev server | -| `build` | `vite build` | Build main app | -| `build:marketing` | `vite build --mode marketing` | Build marketing site | -| `sites:dev` | `cd sites && npm run dev` | Start sites renderer dev server (Port 5176) | -| `sites:build` | `cd sites && npm run build` | Build sites renderer | -| `type-check` | `tsc -b --noEmit` | Type checking | -| `lint` | `eslint .` | Lint code | - ---- - -## Deployment Architecture - -### Production Build - -#### Main Application - -**Build Output**: -``` -dist/ -├── index.html -├── assets/ -│ ├── js/ -│ │ ├── main-[hash].js -│ │ └── vendor-[hash].js -│ └── css/ -│ └── main-[hash].css -└── marketing.html (optional) -``` - -**Deployment**: -- Served via Caddy reverse proxy -- Static file serving -- SPA routing fallback - -#### Sites Renderer - -**Build Output**: -``` -sites/dist/ -├── index.html -├── assets/ -│ ├── js/ -│ │ └── [hash].js -│ └── css/ -│ └── [hash].css -``` - -**Deployment**: -- Separate Docker container (`igny8_sites`) -- Served via Caddy reverse proxy -- Dynamic routing from Site Blueprints -- Public access (no authentication) - -#### Marketing Site - -**Build Output**: -``` -dist/ -├── marketing.html -├── assets/ -│ ├── js/ -│ │ └── marketing-[hash].js -│ └── css/ -│ └── marketing-[hash].css -``` - -**Deployment**: -- Separate Caddy configuration -- SPA routing enabled -- SEO optimization - -### Docker Deployment - -#### Production Dockerfile - -**Main App** (`Dockerfile`): -```dockerfile -FROM node:18-alpine AS builder -WORKDIR /app -COPY package*.json ./ -RUN npm install -COPY . . -RUN npm run build - -FROM caddy:latest -COPY --from=builder /app/dist /usr/share/caddy -COPY Caddyfile /etc/caddy/Caddyfile -EXPOSE 8020 -CMD ["caddy", "run", "--config", "/etc/caddy/Caddyfile"] -``` - -**Marketing** (`Dockerfile.marketing`): -```dockerfile -FROM node:18-alpine AS builder -WORKDIR /app -COPY package*.json ./ -RUN npm install -COPY . . -RUN npm run build:marketing - -FROM caddy:latest -COPY --from=builder /app/dist /usr/share/caddy -COPY Caddyfile.marketing /etc/caddy/Caddyfile -EXPOSE 8020 -CMD ["caddy", "run", "--config", "/etc/caddy/Caddyfile"] -``` - -### Caddy Configuration - -#### Main App (`Caddyfile`) - -```caddy -:8020 { - root * /usr/share/caddy - encode gzip zstd - - # SPA routing - handle { - try_files {path} /index.html - file_server - } -} -``` - -#### Sites Renderer (`Caddyfile.sites`) - -```caddy -sites.igny8.com { - reverse_proxy igny8_sites:5176 - encode gzip zstd -} -``` - -**Features**: -- Public access (no authentication) -- Dynamic routing from Site Blueprints -- SEO-friendly URLs -- HTTPS via Caddy - -#### Marketing Site (`Caddyfile.marketing`) - -```caddy -:8020 { - root * /usr/share/caddy - encode gzip zstd - - # SPA routing for marketing - handle { - try_files {path} /marketing.html - file_server - } -} -``` - ---- - -## Key Features & Functions - -### Authentication & Authorization - -**Features**: -- JWT-based authentication -- Access + refresh tokens -- Automatic token refresh -- Role-based access control (RBAC) -- Protected route guards - -**Implementation**: -- `ProtectedRoute` component -- `authStore` for state management -- API interceptors for token injection - -### Multi-Tenancy - -**Features**: -- Site-based isolation -- Sector organization within sites -- Active site/sector context -- Automatic context injection in API calls - -**Implementation**: -- `siteStore` and `sectorStore` -- Site/sector selectors in header -- Context-aware API functions - -### Module System - -**Modules**: -1. **Planner**: Keywords, clusters, ideas -2. **Writer**: Tasks, content, images -3. **Thinker**: Prompts, profiles, strategies -4. **Billing**: Credits, transactions, usage - -**Features**: -- Module-based routing -- Module-specific dashboards -- Module-specific stores -- Module-specific templates - -### Data Management - -**Features**: -- CRUD operations for all entities -- Filtering and sorting -- Pagination -- Bulk operations -- Export functionality -- Real-time updates - -**Implementation**: -- TablePageTemplate for lists -- API service layer -- Zustand stores for state - -### Error Handling - -**Features**: -- Global error boundary -- API error handling -- User-friendly error messages -- Error logging -- Retry mechanisms - -**Implementation**: -- `ErrorBoundary` component -- `GlobalErrorDisplay` component -- `useErrorHandler` hook -- API error interceptors - -### Loading States - -**Features**: -- Loading indicators -- Skeleton screens -- Progress tracking -- Loading state monitoring - -**Implementation**: -- `LoadingStateMonitor` component -- Loading states in stores -- Suspense boundaries - ---- - -## Performance Optimizations - -### Code Splitting - -**Strategy**: -- Route-based code splitting -- Lazy loading of page components -- Vendor chunk separation -- Dynamic imports for heavy dependencies - -**Implementation**: -```typescript -const Keywords = lazy(() => import("./pages/Planner/Keywords")); -``` - -### Bundle Optimization - -**Techniques**: -- Tree shaking -- Dead code elimination -- Minification (ESBuild) -- CSS code splitting -- Asset optimization - -### Runtime Performance - -**Optimizations**: -- Memoization with `useMemo` and `useCallback` -- Virtual scrolling for large lists -- Debounced search/filter -- Optimistic UI updates -- Request deduplication - -### Caching Strategy - -**Client-Side**: -- Zustand persistence (LocalStorage) -- API response caching -- Asset caching (browser cache) - -**Server-Side**: -- Static asset caching (Caddy) -- CDN for assets (if configured) - ---- - -## Security Considerations - -### Authentication Security - -**Measures**: -- JWT tokens stored securely -- Token refresh mechanism -- Automatic logout on token expiry -- Secure token transmission (HTTPS) - -### API Security - -**Measures**: -- HTTPS enforcement -- CORS configuration -- API key protection -- Request validation -- Rate limiting (backend) - -### Data Security - -**Measures**: -- No sensitive data in client code -- Secure storage of tokens -- XSS prevention -- CSRF protection (backend) -- Input sanitization - -### Content Security - -**Measures**: -- React's built-in XSS protection -- Sanitized user input -- Safe HTML rendering -- Secure file uploads - ---- - -## Conclusion - -The IGNY8 frontend architecture is designed for: - -1. **Scalability**: Modular structure supports growth -2. **Maintainability**: Clear separation of concerns -3. **Performance**: Optimized builds and lazy loading -4. **Developer Experience**: TypeScript, hot reload, clear patterns -5. **User Experience**: Fast loading, smooth navigation, responsive design - -The dual-application architecture allows for: -- Separate deployment of marketing and app -- Independent scaling -- Different optimization strategies -- Clear separation of public and authenticated content - -This architecture provides a solid foundation for continued development and scaling of the IGNY8 platform. - ---- - -**Document Version**: 1.0 -**Last Updated**: 2025-01-XX -**Maintained By**: IGNY8 Development Team - diff --git a/master-docs/04-BACKEND-IMPLEMENTATION.md b/master-docs/04-BACKEND-IMPLEMENTATION.md deleted file mode 100644 index ed87ad30..00000000 --- a/master-docs/04-BACKEND-IMPLEMENTATION.md +++ /dev/null @@ -1,1268 +0,0 @@ -# IGNY8 Backend Implementation Reference - -**Last Updated:** 2025-01-XX (Added 6 missing modules: Linker, Optimizer, Publisher, Site Builder, Automation, Integration) -**Purpose:** Complete backend implementation reference covering project structure, models, ViewSets, serializers, Celery tasks, API endpoints, base classes, middleware, and utilities. - ---- - -## Table of Contents - -1. [Backend Overview](#backend-overview) -2. [Tech Stack](#tech-stack) -3. [Project Structure](#project-structure) -4. [Models](#models) -5. [ViewSets](#viewsets) -6. [Serializers](#serializers) -7. [Celery Tasks](#celery-tasks) -8. [API Endpoints](#api-endpoints) -9. [Base Classes](#base-classes) -10. [Middleware](#middleware) -11. [Utilities](#utilities) -12. [Modules](#modules) - ---- - -## Backend Overview - -The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery. - -### Key Features - -- **Multi-Tenancy**: Complete account isolation with automatic filtering -- **RESTful API**: DRF ViewSets with consistent response format -- **Celery Integration**: Asynchronous task processing for long-running operations -- **Account/Site/Sector Hierarchy**: Hierarchical data organization -- **AI Integration**: Unified AI framework for all AI operations -- **Progress Tracking**: Real-time progress updates for Celery tasks - ---- - -## Tech Stack - -### Core Technologies - -- **Django 5.2+**: Web framework -- **Django REST Framework**: API framework -- **PostgreSQL**: Database -- **Celery**: Asynchronous task queue -- **Redis**: Celery broker and caching - -### Key Libraries - -- **django-filter**: Advanced filtering -- **djangorestframework-simplejwt**: JWT authentication -- **requests**: HTTP client for external APIs -- **python-dotenv**: Environment variable management - ---- - -## Project Structure - -``` -backend/igny8_core/ -├── auth/ # Multi-tenancy and authentication -│ ├── models.py # Account, User, Plan, Site, Sector, Industry models -│ ├── views.py # Account, User, Site, Sector ViewSets -│ ├── serializers.py # Account, User, Plan serializers -│ └── urls.py # Auth module URLs -├── modules/ # Feature modules -│ ├── planner/ # Keywords, Clusters, Ideas -│ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet -│ │ ├── serializers.py # Model serializers -│ │ └── urls.py # Planner module URLs -│ ├── writer/ # Tasks, Content, Images -│ │ ├── views.py # TasksViewSet, ContentViewSet, ImagesViewSet -│ │ ├── serializers.py # Model serializers -│ │ └── urls.py # Writer module URLs -│ ├── linker/ # Internal linking -│ │ ├── views.py # LinkerViewSet -│ │ ├── serializers.py # Link serializers -│ │ └── urls.py # Linker module URLs -│ ├── optimizer/ # Content optimization -│ │ ├── views.py # OptimizerViewSet -│ │ ├── serializers.py # Optimizer serializers -│ │ └── urls.py # Optimizer module URLs -│ ├── publisher/ # Publishing & deployment -│ │ ├── views.py # PublishingRecordViewSet, DeploymentRecordViewSet, PublisherViewSet -│ │ └── urls.py # Publisher module URLs -│ ├── site_builder/ # Site blueprint management -│ │ ├── views.py # SiteBlueprintViewSet, PageBlueprintViewSet, SiteAssetView -│ │ ├── serializers.py # Site builder serializers -│ │ └── urls.py # Site Builder module URLs -│ ├── automation/ # Automation rules -│ │ ├── views.py # AutomationRuleViewSet, ScheduledTaskViewSet -│ │ ├── serializers.py # Automation serializers -│ │ └── urls.py # Automation module URLs -│ ├── integration/ # External integrations -│ │ ├── views.py # IntegrationViewSet -│ │ └── urls.py # Integration module URLs -│ ├── system/ # Settings, Prompts, Integration -│ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy -│ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet -│ │ ├── settings_views.py # SystemSettingsViewSet, AccountSettingsViewSet -│ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress -│ │ ├── utils.py # Default prompts, prompt loading -│ │ └── urls.py # System module URLs -│ └── billing/ # Credits, Transactions, Usage -│ ├── views.py # CreditTransactionViewSet, CreditBalanceViewSet, CreditUsageViewSet -│ ├── services.py # CreditService -│ └── urls.py # Billing module URLs -├── business/ # Business logic layer -│ ├── planning/ # Planner models (Keywords, Clusters, ContentIdeas) -│ ├── content/ # Writer models (Tasks, Content, Images) -│ ├── linking/ # Linker services -│ ├── optimization/ # Optimizer services -│ ├── publishing/ # Publisher models (PublishingRecord, DeploymentRecord) -│ ├── site_building/ # Site Builder models (SiteBlueprint, PageBlueprint, etc.) -│ ├── automation/ # Automation models (AutomationRule, ScheduledTask) -│ ├── integration/ # Integration models (SiteIntegration) -│ └── billing/ # Billing models (CreditTransaction, CreditUsageLog) -├── api/ # API base classes -│ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet -│ └── pagination.py # CustomPageNumberPagination -├── utils/ # Shared utilities -│ ├── ai_processor.py # Unified AI interface (legacy, see AI functions) -│ └── content_normalizer.py # Content processing utilities -├── middleware/ # Custom middleware -│ ├── account.py # AccountContextMiddleware (sets request.account) -│ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics) -├── ai/ # AI framework -│ ├── base.py # BaseAIFunction -│ ├── engine.py # AIEngine -│ ├── tasks.py # run_ai_task -│ ├── registry.py # Function registry -│ ├── prompts.py # PromptRegistry -│ ├── ai_core.py # AICore -│ ├── settings.py # Model configuration -│ ├── validators.py # Validation functions -│ ├── tracker.py # Progress tracking -│ └── functions/ # AI function implementations -│ ├── auto_cluster.py -│ ├── generate_ideas.py -│ ├── generate_content.py -│ ├── generate_image_prompts.py -│ └── generate_images.py -├── settings.py # Django settings -├── urls.py # Root URL configuration -└── celery.py # Celery configuration -``` - ---- - -## Models - -### Base Models - -#### AccountBaseModel -**File**: `auth/models.py` - -**Purpose**: Base model for all account-isolated models. - -**Fields**: -- `account`: ForeignKey to Account -- `created_at`: DateTimeField (auto_now_add) -- `updated_at`: DateTimeField (auto_now) - -**Usage**: All models that need account isolation inherit from this. - -#### SiteSectorBaseModel -**File**: `auth/models.py` - -**Purpose**: Base model for models that belong to Site and Sector. - -**Fields**: -- Inherits from `AccountBaseModel` -- `site`: ForeignKey to Site -- `sector`: ForeignKey to Sector - -**Methods**: -- `save()`: Automatically sets `account` from `site.account` and validates sector belongs to site - -**Usage**: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this. - -### Auth Models - -#### Account -**Table**: `igny8_accounts` - -**Fields**: -- `name`: CharField -- `slug`: SlugField (unique) -- `owner`: ForeignKey to User -- `plan`: ForeignKey to Plan -- `credits`: IntegerField (default: 0) -- `status`: CharField (choices: active, suspended, trial, cancelled) - -**Methods**: -- `is_system_account()`: Returns True if account is a system account - -#### User -**Table**: `igny8_users` - -**Inherits**: `AbstractUser` - -**Fields**: -- `email`: EmailField (unique) -- `account`: ForeignKey to Account -- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot) - -**Methods**: -- `has_role(role)`: Checks if user has role -- `is_owner_or_admin()`: Checks if user is owner or admin -- `is_developer()`: Checks if user is developer -- `is_admin_or_developer()`: Checks if user is admin or developer -- `is_system_account_user()`: Checks if user belongs to system account -- `get_accessible_sites()`: Returns queryset of accessible sites - -#### Plan -**Table**: `igny8_plans` - -**Fields**: Extensive fields for limits (users, sites, keywords, clusters, content ideas, AI requests, word count, images, credits) - -**Methods**: -- `clean()`: Validates plan limits -- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month - -#### Site -**Table**: `igny8_sites` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `name`: CharField -- `slug`: SlugField (unique per account) -- `domain`: URLField (optional) -- `industry`: ForeignKey to Industry (optional) -- `is_active`: BooleanField -- `status`: CharField -- `wp_url`: URLField (optional) -- `wp_username`: CharField (optional) -- `wp_app_password`: CharField (optional) - -**Methods**: -- `get_active_sectors_count()`: Returns count of active sectors -- `can_add_sector()`: Returns True if site can add another sector (max 5) - -#### Sector -**Table**: `igny8_sectors` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `site`: ForeignKey to Site -- `industry_sector`: ForeignKey to IndustrySector (optional) -- `name`: CharField -- `slug`: SlugField (unique per site) -- `is_active`: BooleanField -- `status`: CharField - -**Validation**: Maximum 5 active sectors per site - -### Planner Models - -#### Keywords -**Table**: `igny8_keywords` - -**Inherits**: `SiteSectorBaseModel` - -**Fields**: -- `keyword`: CharField -- `volume`: IntegerField (optional) -- `difficulty`: IntegerField (optional) -- `intent`: CharField (optional) -- `status`: CharField -- `cluster`: ManyToManyField to Clusters - -#### Clusters -**Table**: `igny8_clusters` - -**Inherits**: `SiteSectorBaseModel` - -**Fields**: -- `name`: CharField -- `description`: TextField (optional) -- `keywords_count`: IntegerField (calculated) -- `volume`: IntegerField (calculated) -- `status`: CharField -- `keywords`: ManyToManyField to Keywords - -#### ContentIdeas -**Table**: `igny8_content_ideas` - -**Inherits**: `SiteSectorBaseModel` - -**Fields**: -- `idea_title`: CharField -- `description`: TextField -- `content_type`: CharField -- `content_structure`: CharField -- `target_keywords`: TextField -- `keyword_cluster`: ForeignKey to Clusters -- `estimated_word_count`: IntegerField -- `status`: CharField - -### Writer Models - -#### Tasks -**Table**: `igny8_tasks` - -**Inherits**: `SiteSectorBaseModel` - -**Fields**: -- `title`: CharField -- `description`: TextField -- `cluster`: ForeignKey to Clusters (optional) -- `idea`: ForeignKey to ContentIdeas (optional) -- `content_type`: CharField -- `content_structure`: CharField -- `status`: CharField - -#### Content -**Table**: `igny8_content` - -**Inherits**: `SiteSectorBaseModel` - -**Fields**: -- `task`: OneToOneField to Tasks -- `html_content`: TextField -- `word_count`: IntegerField -- `status`: CharField -- `wp_post_id`: IntegerField (optional) -- `meta_title`: CharField (optional) -- `meta_description`: TextField (optional) -- `primary_keyword`: CharField (optional) -- `secondary_keywords`: TextField (optional) - -#### Images -**Table**: `igny8_images` - -**Inherits**: `SiteSectorBaseModel` - -**Fields**: -- `task`: ForeignKey to Tasks (optional) -- `content`: ForeignKey to Content (optional) -- `image_type`: CharField (choices: featured, in_article, desktop, mobile) -- `prompt`: TextField -- `image_url`: CharField -- `status`: CharField - -### System Models - -#### AIPrompt -**Table**: `igny8_ai_prompts` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `prompt_type`: CharField -- `prompt_value`: TextField -- `function_name`: CharField (optional) - -#### IntegrationSettings -**Table**: `igny8_integration_settings` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `integration_type`: CharField (choices: openai, runware, image_generation) -- `config`: JSONField - -#### AuthorProfile -**Table**: `igny8_author_profiles` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `name`: CharField -- `description`: TextField -- `tone`: CharField -- `language`: CharField - -#### Strategy -**Table**: `igny8_strategies` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `name`: CharField -- `description`: TextField -- `sector`: ForeignKey to Sector (optional) -- `prompt_types`: JSONField -- `section_logic`: JSONField -- `is_active`: BooleanField - -### Billing Models - -#### CreditTransaction -**Table**: `igny8_credit_transactions` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `transaction_type`: CharField (choices: purchase, usage, refund, adjustment) -- `amount`: IntegerField -- `balance_after`: IntegerField -- `description`: TextField (optional) - -#### CreditUsageLog -**Table**: `igny8_credit_usage_logs` - -**Inherits**: `AccountBaseModel` - -**Fields**: -- `operation_type`: CharField -- `credits_used`: IntegerField -- `cost_usd`: DecimalField -- `details`: JSONField (optional) - ---- - -## ViewSets - -### Base ViewSets - -#### AccountModelViewSet -**File**: `api/base.py` - -**Purpose**: Base ViewSet with automatic account filtering. - -**Methods**: -- `get_queryset()`: Filters queryset by `request.account` (with admin/developer override) -- `perform_create()`: Sets account on created objects -- `get_serializer_context()`: Adds account to serializer context - -**Access Control**: -- Admin/Developer users: Bypass account filtering -- System account users: Bypass account filtering -- Regular users: Only see data from their account - -#### SiteSectorModelViewSet -**File**: `api/base.py` - -**Purpose**: Base ViewSet with site/sector filtering and access control. - -**Inherits**: `AccountModelViewSet` - -**Methods**: -- `get_queryset()`: Filters by account, accessible sites, and optional site_id/sector_id -- `perform_create()`: Validates site access and sector-site relationship -- `get_serializer_context()`: Adds accessible sites and sectors to context - -**Access Control**: -- Developers: All active sites -- System account users: All active sites -- Owners/Admins: All sites in their account -- Editors/Viewers: Only sites granted via `SiteUserAccess` - -### Planner ViewSets - -#### KeywordViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List keywords with filtering -- `create()`: Create keyword -- `retrieve()`: Get keyword details -- `update()`: Update keyword -- `destroy()`: Delete keyword -- `auto_cluster()`: Auto-cluster keywords using AI -- `bulk_delete()`: Bulk delete keywords -- `bulk_update_status()`: Bulk update keyword status -- `export_csv()`: Export keywords to CSV -- `import_csv()`: Import keywords from CSV - -**Filtering**: -- Search: `keyword` field -- Filters: `status`, `cluster_id`, `intent` -- Custom: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max` -- Ordering: `created_at`, `volume`, `difficulty` - -#### ClusterViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List clusters -- `create()`: Create cluster -- `retrieve()`: Get cluster details -- `update()`: Update cluster -- `destroy()`: Delete cluster -- `auto_generate_ideas()`: Auto-generate content ideas for clusters - -#### ContentIdeasViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List content ideas -- `create()`: Create content idea -- `retrieve()`: Get content idea details -- `update()`: Update content idea -- `destroy()`: Delete content idea - -### Writer ViewSets - -#### TasksViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List tasks -- `create()`: Create task -- `retrieve()`: Get task details -- `update()`: Update task -- `destroy()`: Delete task -- `auto_generate_content()`: Auto-generate content for tasks -- `generate_images()`: Generate images for image records -- `bulk_delete()`: Bulk delete tasks -- `bulk_update()`: Bulk update task status - -**Filtering**: -- Search: `title`, `keywords` -- Filters: `status`, `cluster_id`, `content_type`, `content_structure` -- Ordering: `title`, `created_at`, `word_count`, `status` - -#### ContentViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List content -- `create()`: Create content -- `retrieve()`: Get content details -- `update()`: Update content -- `destroy()`: Delete content -- `publish_to_wordpress()`: Publish content to WordPress - -#### ImagesViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List images -- `create()`: Create image -- `retrieve()`: Get image details -- `update()`: Update image -- `destroy()`: Delete image -- `generate_images()`: Generate images using AI - -### Linker ViewSets - -#### LinkerViewSet -**Inherits**: `viewsets.ViewSet` - -**Actions**: -- `process()`: Process a single content item for internal linking -- `batch_process()`: Process multiple content items for internal linking - -**Purpose**: AI-powered internal linking suggestions based on cluster matches - -### Optimizer ViewSets - -#### OptimizerViewSet -**Inherits**: `viewsets.ViewSet` - -**Actions**: -- `optimize()`: Optimize content (auto-detects entry point) -- `batch_optimize()`: Batch optimize multiple content items -- `analyze()`: Analyze content without optimization - -**Purpose**: Content optimization and scoring - -### Publisher ViewSets - -#### PublishingRecordViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List publishing records -- `create()`: Create publishing record -- `retrieve()`: Get publishing record details -- `update()`: Update publishing record -- `destroy()`: Delete publishing record - -#### DeploymentRecordViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List deployment records -- `create()`: Create deployment record -- `retrieve()`: Get deployment record details -- `update()`: Update deployment record -- `destroy()`: Delete deployment record - -#### PublisherViewSet -**Inherits**: `viewsets.ViewSet` - -**Actions**: -- `publish()`: Publish content to destination -- `deploy()`: Deploy site blueprint -- `check_readiness()`: Check deployment readiness - -#### SiteDefinitionView -**Inherits**: `APIView` - -**Purpose**: Public endpoint for Sites Renderer to fetch site definitions - -### Site Builder ViewSets - -#### SiteBlueprintViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List site blueprints -- `create()`: Create site blueprint -- `retrieve()`: Get site blueprint details -- `update()`: Update site blueprint -- `destroy()`: Delete site blueprint -- `generate_structure()`: Generate site structure using AI -- `generate_all_pages()`: Generate all pages for blueprint -- `create_tasks()`: Create Writer tasks for pages -- `progress()`: Get cluster-level completion status -- `attach_clusters()`: Attach planner clusters to blueprint -- `detach_clusters()`: Detach clusters from blueprint -- `list_taxonomies()`: List taxonomies for blueprint -- `create_taxonomy()`: Create taxonomy for blueprint -- `import_taxonomies()`: Import taxonomies from external source -- `bulk_delete()`: Bulk delete blueprints - -#### PageBlueprintViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List page blueprints -- `create()`: Create page blueprint -- `retrieve()`: Get page blueprint details -- `update()`: Update page blueprint -- `destroy()`: Delete page blueprint -- `generate_content()`: Generate content for page -- `regenerate()`: Regenerate page content - -#### SiteAssetView -**Inherits**: `APIView` - -**Actions**: -- `GET`: List files for site -- `POST`: Upload file -- `DELETE`: Delete file - -#### SiteBuilderMetadataView -**Inherits**: `APIView` - -**Actions**: -- `GET`: Get metadata (business types, audience profiles, brand personalities, hero imagery) - -### Automation ViewSets - -#### AutomationRuleViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List automation rules -- `create()`: Create automation rule -- `retrieve()`: Get automation rule details -- `update()`: Update automation rule -- `destroy()`: Delete automation rule -- `execute()`: Manually execute automation rule - -#### ScheduledTaskViewSet -**Inherits**: `AccountModelViewSet` - -**Actions**: -- `list()`: List scheduled tasks -- `create()`: Create scheduled task -- `retrieve()`: Get scheduled task details -- `update()`: Update scheduled task -- `destroy()`: Delete scheduled task - -### Integration ViewSets - -#### IntegrationViewSet -**Inherits**: `SiteSectorModelViewSet` - -**Actions**: -- `list()`: List site integrations -- `create()`: Create site integration -- `retrieve()`: Get integration details -- `update()`: Update integration -- `destroy()`: Delete integration -- `test_connection()`: Test connection to integrated platform -- `sync()`: Sync content with integrated platform -- `sync_health()`: Check sync health status - -### System ViewSets - -#### IntegrationSettingsViewSet -**Inherits**: `viewsets.ViewSet` - -**Actions**: -- `list()`: List integrations -- `retrieve()`: Get integration settings -- `update()`: Save integration settings -- `save_post()`: Save integration settings (POST) -- `test_connection()`: Test API connection -- `test_openai()`: Test OpenAI connection -- `test_runware()`: Test Runware connection -- `generate_image()`: Test image generation -- `task_progress()`: Get Celery task progress - -#### AIPromptViewSet -**Inherits**: `AccountModelViewSet` - -**Actions**: -- `list()`: List prompts -- `create()`: Create prompt -- `retrieve()`: Get prompt details -- `update()`: Update prompt -- `destroy()`: Delete prompt -- `reset_to_default()`: Reset prompt to default value - -#### AuthorProfileViewSet -**Inherits**: `AccountModelViewSet` - -**Actions**: -- `list()`: List author profiles -- `create()`: Create author profile -- `retrieve()`: Get author profile details -- `update()`: Update author profile -- `destroy()`: Delete author profile - ---- - -## Serializers - -### Planner Serializers - -#### KeywordSerializer -**Fields**: All Keyword model fields - -**Validation**: Validates keyword uniqueness, cluster belongs to same sector - -#### ClusterSerializer -**Fields**: All Cluster model fields - -**Read-Only Fields**: `keywords_count`, `volume` (calculated) - -#### ContentIdeasSerializer -**Fields**: All ContentIdeas model fields - -### Writer Serializers - -#### TasksSerializer -**Fields**: All Tasks model fields - -#### ContentSerializer -**Fields**: All Content model fields - -#### ImagesSerializer -**Fields**: All Images model fields - -### System Serializers - -#### AIPromptSerializer -**Fields**: All AIPrompt model fields - -#### IntegrationSettingsSerializer -**Fields**: All IntegrationSettings model fields - -### Linker Serializers - -#### LinkContentSerializer -**Fields**: `content_id` - -#### BatchLinkContentSerializer -**Fields**: `content_ids` (array) - -### Optimizer Serializers - -#### OptimizeContentSerializer -**Fields**: `content_id`, `entry_point` (optional) - -#### BatchOptimizeContentSerializer -**Fields**: `content_ids` (array), `entry_point` (optional) - -#### AnalyzeContentSerializer -**Fields**: `content_id` - -### Publisher Serializers - -#### PublishingRecordSerializer -**Fields**: All PublishingRecord model fields - -#### DeploymentRecordSerializer -**Fields**: All DeploymentRecord model fields - -### Site Builder Serializers - -#### SiteBlueprintSerializer -**Fields**: All SiteBlueprint model fields, includes nested `pages` - -#### PageBlueprintSerializer -**Fields**: All PageBlueprint model fields - -#### SiteBuilderMetadataSerializer -**Fields**: `business_types`, `audience_profiles`, `brand_personalities`, `hero_imagery_directions` - -### Automation Serializers - -#### AutomationRuleSerializer -**Fields**: All AutomationRule model fields - -#### ScheduledTaskSerializer -**Fields**: All ScheduledTask model fields - -### Integration Serializers - -#### SiteIntegrationSerializer -**Fields**: All SiteIntegration model fields - ---- - -## Celery Tasks - -### AI Task Entry Point - -#### run_ai_task -**File**: `ai/tasks.py` - -**Purpose**: Unified Celery task entrypoint for all AI functions - -**Parameters**: -- `function_name`: Function name (e.g., 'auto_cluster') -- `payload`: Function-specific payload -- `account_id`: Account ID - -**Flow**: -1. Gets account from account_id -2. Gets function instance from registry -3. Creates AIEngine -4. Executes function via AIEngine - -### Planner Tasks - -#### auto_cluster_keywords_task -**File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`) - -**Purpose**: Auto-cluster keywords using AI - -**Parameters**: -- `keyword_ids`: List of keyword IDs -- `account_id`: Account ID -- `site_id`: Site ID -- `sector_id`: Sector ID - -**Progress Tracking**: Updates progress with request_steps and response_steps - -#### auto_generate_ideas_task -**File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`) - -**Purpose**: Auto-generate content ideas for clusters - -**Parameters**: -- `cluster_ids`: List of cluster IDs -- `account_id`: Account ID - -**Progress Tracking**: Updates progress for each cluster - -### Writer Tasks - -#### auto_generate_content_task -**File**: `modules/writer/tasks.py` (legacy, now uses `run_ai_task`) - -**Purpose**: Auto-generate content for tasks - -**Parameters**: -- `task_ids`: List of task IDs -- `account_id`: Account ID - -**Progress Tracking**: Updates progress for each task - -#### process_image_generation_queue -**File**: `modules/writer/views.py` (via `ai/tasks.py`) - -**Purpose**: Generate images for image records - -**Parameters**: -- `image_ids`: List of image IDs -- `account_id`: Account ID -- `content_id`: Content ID (optional) - -**Progress Tracking**: Updates progress for each image sequentially - ---- - -## API Endpoints - -### Base URL - -`/api/v1/` - -### Planner Endpoints - -- `GET /api/v1/planner/keywords/` - List keywords -- `POST /api/v1/planner/keywords/` - Create keyword -- `GET /api/v1/planner/keywords/{id}/` - Get keyword -- `PUT /api/v1/planner/keywords/{id}/` - Update keyword -- `DELETE /api/v1/planner/keywords/{id}/` - Delete keyword -- `POST /api/v1/planner/keywords/auto_cluster/` - Auto-cluster keywords -- `POST /api/v1/planner/keywords/bulk_delete/` - Bulk delete keywords -- `POST /api/v1/planner/keywords/bulk_update_status/` - Bulk update status -- `GET /api/v1/planner/keywords/export_csv/` - Export keywords -- `POST /api/v1/planner/keywords/import_csv/` - Import keywords -- `GET /api/v1/planner/clusters/` - List clusters -- `POST /api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas -- `GET /api/v1/planner/ideas/` - List content ideas - -### Writer Endpoints - -- `GET /api/v1/writer/tasks/` - List tasks -- `POST /api/v1/writer/tasks/` - Create task -- `GET /api/v1/writer/tasks/{id}/` - Get task details -- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content -- `GET /api/v1/writer/content/` - List content -- `GET /api/v1/writer/content/{id}/` - Get content details -- `POST /api/v1/writer/images/generate_images/` - Generate images - -### Linker Endpoints - -- `POST /api/v1/linker/process/` - Process content for internal linking -- `POST /api/v1/linker/batch_process/` - Batch process content for linking - -### Optimizer Endpoints - -- `POST /api/v1/optimizer/optimize/` - Optimize content -- `POST /api/v1/optimizer/batch_optimize/` - Batch optimize content -- `POST /api/v1/optimizer/analyze/` - Analyze content - -### Publisher Endpoints - -- `GET /api/v1/publisher/publishing-records/` - List publishing records -- `POST /api/v1/publisher/publishing-records/` - Create publishing record -- `GET /api/v1/publisher/deployments/` - List deployment records -- `POST /api/v1/publisher/deployments/` - Create deployment record -- `POST /api/v1/publisher/publish/` - Publish content -- `POST /api/v1/publisher/deploy/` - Deploy site blueprint -- `GET /api/v1/publisher/sites/{site_id}/definition/` - Get site definition (public) - -### Site Builder Endpoints - -- `GET /api/v1/site-builder/blueprints/` - List site blueprints -- `POST /api/v1/site-builder/blueprints/` - Create site blueprint -- `GET /api/v1/site-builder/blueprints/{id}/` - Get blueprint details -- `POST /api/v1/site-builder/blueprints/{id}/generate_structure/` - Generate structure -- `POST /api/v1/site-builder/blueprints/{id}/generate_all_pages/` - Generate all pages -- `POST /api/v1/site-builder/blueprints/{id}/create_tasks/` - Create tasks for pages -- `GET /api/v1/site-builder/blueprints/{id}/progress/` - Get progress -- `POST /api/v1/site-builder/blueprints/{id}/clusters/attach/` - Attach clusters -- `POST /api/v1/site-builder/blueprints/{id}/clusters/detach/` - Detach clusters -- `GET /api/v1/site-builder/blueprints/{id}/taxonomies/` - List taxonomies -- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/` - Create taxonomy -- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/import/` - Import taxonomies -- `POST /api/v1/site-builder/blueprints/bulk_delete/` - Bulk delete blueprints -- `GET /api/v1/site-builder/pages/` - List page blueprints -- `POST /api/v1/site-builder/pages/` - Create page blueprint -- `GET /api/v1/site-builder/pages/{id}/` - Get page details -- `POST /api/v1/site-builder/pages/{id}/generate_content/` - Generate content -- `POST /api/v1/site-builder/pages/{id}/regenerate/` - Regenerate page -- `GET /api/v1/site-builder/assets/` - List assets -- `POST /api/v1/site-builder/assets/` - Upload asset -- `DELETE /api/v1/site-builder/assets/` - Delete asset -- `GET /api/v1/site-builder/metadata/` - Get metadata - -### Automation Endpoints - -- `GET /api/v1/automation/rules/` - List automation rules -- `POST /api/v1/automation/rules/` - Create automation rule -- `GET /api/v1/automation/rules/{id}/` - Get rule details -- `POST /api/v1/automation/rules/{id}/execute/` - Execute rule -- `GET /api/v1/automation/scheduled-tasks/` - List scheduled tasks -- `POST /api/v1/automation/scheduled-tasks/` - Create scheduled task - -### Integration Endpoints - -- `GET /api/v1/integration/integrations/` - List site integrations -- `POST /api/v1/integration/integrations/` - Create integration -- `GET /api/v1/integration/integrations/{id}/` - Get integration details -- `POST /api/v1/integration/integrations/{id}/test_connection/` - Test connection -- `POST /api/v1/integration/integrations/{id}/sync/` - Sync content -- `GET /api/v1/integration/integrations/{id}/sync_health/` - Get sync health - -### System Endpoints - -- `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings -- `PUT /api/v1/system/settings/integrations/{pk}/` - Save integration settings -- `POST /api/v1/system/settings/integrations/{pk}/test_openai/` - Test OpenAI -- `POST /api/v1/system/settings/integrations/{pk}/test_runware/` - Test Runware -- `POST /api/v1/system/settings/integrations/{pk}/generate_image/` - Test image generation -- `GET /api/v1/system/settings/task_progress/{task_id}/` - Get task progress - ---- - -## Base Classes - -### AccountModelViewSet - -**File**: `api/base.py` - -**Purpose**: Base ViewSet with automatic account filtering - -**Features**: -- Automatic account filtering -- Admin/Developer override -- Account context in serializers - -### SiteSectorModelViewSet - -**File**: `api/base.py` - -**Purpose**: Base ViewSet with site/sector filtering - -**Features**: -- Account filtering (inherited) -- Site access control -- Sector validation -- Accessible sites/sectors in serializer context - ---- - -## Middleware - -### AccountContextMiddleware - -**File**: `middleware/account.py` - -**Purpose**: Sets `request.account` from JWT token - -**Functionality**: -- Extracts account ID from JWT token -- Loads Account object -- Sets `request.account` - -### ResourceTrackerMiddleware - -**File**: `middleware/resource_tracker.py` - -**Purpose**: Tracks API request metrics - -**Functionality**: -- Tracks CPU, memory, I/O usage -- Stores metrics in cache -- Provides metrics endpoint - ---- - -## Utilities - -### Content Normalizer - -**File**: `utils/content_normalizer.py` - -**Purpose**: Content processing utilities - -**Functions**: -- `normalize_content()`: Converts plain text to HTML -- `_extract_body_content()`: Extracts body content from HTML - ---- - -## Modules - -### Planner Module - -**Purpose**: Keyword management and content planning - -**Models**: Keywords, Clusters, ContentIdeas - -**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet - -**Tasks**: Auto-cluster keywords, Auto-generate ideas - -### Writer Module - -**Purpose**: Content generation and management - -**Models** (in `business/content/`): -- `Tasks`: Content generation tasks -- `Content`: Generated content pieces -- `Images`: Generated images -- `ContentClusterMap`: Content-to-cluster mapping -- `ContentTaxonomyMap`: Content taxonomy mapping -- `ContentAttributeMap`: Content attribute mapping - -**ViewSets**: -- `TasksViewSet`: CRUD + auto-generate content, bulk operations -- `ContentViewSet`: CRUD + publish to WordPress -- `ImagesViewSet`: CRUD + generate images - -**URLs**: `/api/v1/writer/` - -### Linker Module - -**Purpose**: Internal linking operations - -**ViewSets**: -- `LinkerViewSet`: Process content for internal linking - -**Actions**: -- `process()`: Process single content item -- `batch_process()`: Process multiple content items - -**URLs**: `/api/v1/linker/` - -### Optimizer Module - -**Purpose**: Content optimization and scoring - -**ViewSets**: -- `OptimizerViewSet`: Content optimization operations - -**Actions**: -- `optimize()`: Optimize content (auto-detects entry point) -- `batch_optimize()`: Batch optimize content -- `analyze()`: Analyze content without optimization - -**URLs**: `/api/v1/optimizer/` - -### Publisher Module - -**Purpose**: Publishing records and site deployment - -**Models** (in `business/publishing/`): -- `PublishingRecord`: Records of content publishing -- `DeploymentRecord`: Records of site deployments - -**ViewSets**: -- `PublishingRecordViewSet`: CRUD for publishing records -- `DeploymentRecordViewSet`: CRUD for deployment records -- `PublisherViewSet`: Publishing and deployment actions -- `SiteDefinitionView`: Public endpoint for Sites Renderer - -**Actions**: -- `publish()`: Publish content to destination -- `deploy()`: Deploy site blueprint -- `check_readiness()`: Check deployment readiness - -**URLs**: `/api/v1/publisher/` - -### Site Builder Module - -**Purpose**: Site blueprint management - -**Models** (in `business/site_building/`): -- `SiteBlueprint`: Site structure definitions -- `PageBlueprint`: Page structure definitions -- `SiteBlueprintCluster`: Cluster-to-blueprint mapping -- `SiteBlueprintTaxonomy`: Taxonomy definitions -- `SiteBuilderOption`: Site builder configuration options - -**ViewSets**: -- `SiteBlueprintViewSet`: CRUD + structure generation, cluster management -- `PageBlueprintViewSet`: CRUD + content generation -- `SiteAssetView`: Asset management -- `SiteBuilderMetadataView`: Metadata retrieval - -**Actions**: -- `generate_structure()`: Generate site structure using AI -- `generate_all_pages()`: Generate all pages for blueprint -- `create_tasks()`: Create Writer tasks for pages -- `progress()`: Get cluster-level completion status -- `attach_clusters()`: Attach planner clusters -- `detach_clusters()`: Detach clusters -- `generate_content()`: Generate content for page -- `regenerate()`: Regenerate page content - -**URLs**: `/api/v1/site-builder/` - -**Note**: Site Builder Wizard UI has been removed. Only API-based blueprint management remains. - -### Automation Module - -**Purpose**: Automation rules and scheduled tasks - -**Models** (in `business/automation/`): -- `AutomationRule`: Rule-based automation definitions -- `ScheduledTask`: Scheduled task definitions - -**ViewSets**: -- `AutomationRuleViewSet`: CRUD + execute rule -- `ScheduledTaskViewSet`: CRUD for scheduled tasks - -**Actions**: -- `execute()`: Manually execute automation rule - -**URLs**: `/api/v1/automation/` - -### Integration Module - -**Purpose**: External platform integrations - -**Models** (in `business/integration/`): -- `SiteIntegration`: Site-level integration configurations - -**ViewSets**: -- `IntegrationViewSet`: CRUD + sync operations - -**Actions**: -- `test_connection()`: Test connection to integrated platform -- `sync()`: Sync content with integrated platform -- `sync_health()`: Check sync health status - -**URLs**: `/api/v1/integration/` - -### System Module - -**Purpose**: System configuration and AI settings - -**Models**: -- `AIPrompt`: AI prompt templates -- `IntegrationSettings`: Account-level integration settings -- `AuthorProfile`: Author profile definitions -- `Strategy`: Content strategy definitions - -**ViewSets**: -- `AIPromptViewSet`: CRUD + reset to default -- `IntegrationSettingsViewSet`: Integration settings management -- `AuthorProfileViewSet`: CRUD for author profiles -- `SystemSettingsViewSet`: System settings management -- `AccountSettingsViewSet`: Account settings management - -**URLs**: `/api/v1/system/` - -### Billing Module - -**Purpose**: Credit management and usage tracking - -**Models** (in `business/billing/`): -- `CreditTransaction`: Credit transaction records -- `CreditUsageLog`: Credit usage logging - -**ViewSets**: -- `CreditTransactionViewSet`: CRUD for transactions -- `CreditBalanceViewSet`: Credit balance retrieval -- `CreditUsageViewSet`: Credit usage tracking - -**Services**: -- `CreditService`: Credit operations (check, deduct, add, calculate) - -**URLs**: `/api/v1/billing/` - ---- - -## Summary - -The IGNY8 backend provides: - -1. **Multi-Tenancy**: Complete account isolation with automatic filtering -2. **RESTful API**: DRF ViewSets with consistent response format -3. **Celery Integration**: Asynchronous task processing -4. **Hierarchical Organization**: Account > Site > Sector > Content structure -5. **AI Framework**: Unified AI framework for all AI operations -6. **Progress Tracking**: Real-time progress updates for Celery tasks -7. **Module-Based Design**: Clear separation of concerns across 10 modules -8. **Base Classes**: Reusable ViewSets for common patterns -9. **Middleware**: Account context and resource tracking -10. **Utilities**: Content processing and AI integration -11. **Internal Linking**: AI-powered internal linking suggestions -12. **Content Optimization**: Content optimization and scoring -13. **Publishing System**: Multi-destination publishing with deployment tracking -14. **Site Builder**: Site blueprint and structure management (API-based) -15. **Automation**: Rule-based automation and scheduled tasks -16. **Integration**: External platform integrations and synchronization - -This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform with 10 backend modules covering the complete content workflow from planning to publishing. - diff --git a/master-docs/05-AI-FRAMEWORK-IMPLEMENTATION.md b/master-docs/05-AI-FRAMEWORK-IMPLEMENTATION.md deleted file mode 100644 index b9340a73..00000000 --- a/master-docs/05-AI-FRAMEWORK-IMPLEMENTATION.md +++ /dev/null @@ -1,676 +0,0 @@ -# IGNY8 AI Framework Implementation Reference - -**Last Updated:** 2025-01-XX (Added 3 missing AI functions: generate_site_structure, optimize_content, generate_page_content) -**Purpose:** Complete AI framework implementation reference covering architecture, code structure, all 8 AI functions, execution flow, progress tracking, cost tracking, prompt management, and model configuration. - ---- - -## Table of Contents - -1. [AI Framework Overview](#ai-framework-overview) -2. [Common Architecture](#common-architecture) -3. [AI Function Execution Flow](#ai-function-execution-flow) -4. [AI Functions](#ai-functions) -5. [Progress Tracking](#progress-tracking) -6. [Cost Tracking](#cost-tracking) -7. [Prompt Management](#prompt-management) -8. [Model Configuration](#model-configuration) - ---- - -## AI Framework Overview - -The IGNY8 AI framework provides a unified interface for all AI operations. All AI functions inherit from `BaseAIFunction` and are orchestrated by `AIEngine`, ensuring consistent execution, progress tracking, error handling, and cost tracking. - -### Key Components - -- **BaseAIFunction**: Abstract base class for all AI functions -- **AIEngine**: Central orchestrator managing lifecycle, progress, logging, cost tracking -- **AICore**: Centralized AI request handler for all AI operations -- **PromptRegistry**: Centralized prompt management with hierarchical resolution -- **Function Registry**: Lazy-loaded function registry -- **Progress Tracking**: Real-time progress updates via Celery -- **Cost Tracking**: Automatic cost and token tracking - -### AI Functions - -1. **Auto Cluster Keywords**: Group related keywords into semantic clusters -2. **Generate Ideas**: Generate content ideas from keyword clusters -3. **Generate Content**: Generate blog post and article content -4. **Generate Image Prompts**: Extract image prompts from content -5. **Generate Images**: Generate images using OpenAI DALL-E or Runware -6. **Generate Site Structure**: Create site/page architecture from business brief -7. **Optimize Content**: Optimize content for SEO, readability, and engagement -8. **Generate Page Content**: Generate structured page content with JSON blocks for Site Builder - ---- - -## Common Architecture - -### Core Framework Files - -#### Entry Point -**File**: `backend/igny8_core/ai/tasks.py` -**Function**: `run_ai_task` -**Purpose**: Unified Celery task entrypoint for all AI functions -**Parameters**: `function_name` (str), `payload` (dict), `account_id` (int) -**Flow**: Loads function from registry → Creates AIEngine → Executes function - -#### Engine Orchestrator -**File**: `backend/igny8_core/ai/engine.py` -**Class**: `AIEngine` -**Purpose**: Central orchestrator managing lifecycle, progress, logging, cost tracking -**Methods**: -- `execute` - Main execution pipeline (6 phases: INIT, PREP, AI_CALL, PARSE, SAVE, DONE) -- `_handle_error` - Centralized error handling -- `_log_to_database` - Logs to AITaskLog model - -#### Base Function Class -**File**: `backend/igny8_core/ai/base.py` -**Class**: `BaseAIFunction` -**Purpose**: Abstract base class defining interface for all AI functions -**Abstract Methods**: -- `get_name` - Returns function name (e.g., 'auto_cluster') -- `prepare` - Loads and prepares data -- `build_prompt` - Builds AI prompt -- `parse_response` - Parses AI response -- `save_output` - Saves results to database -**Optional Methods**: -- `get_metadata` - Returns display name, description, phases -- `get_max_items` - Returns max items limit (or None) -- `validate` - Validates input payload (default: checks for 'ids') -- `get_model` - Returns model override (default: None, uses account default) - -#### Function Registry -**File**: `backend/igny8_core/ai/registry.py` -**Functions**: -- `register_function` - Registers function class -- `register_lazy_function` - Registers lazy loader -- `get_function` - Gets function class by name (lazy loads if needed) -- `get_function_instance` - Gets function instance by name -- `list_functions` - Lists all registered functions - -**Registered Functions** (7 lazy-loaded): -1. `auto_cluster` - AutoClusterFunction -2. `generate_ideas` - GenerateIdeasFunction -3. `generate_content` - GenerateContentFunction -4. `generate_images` - GenerateImagesFunction -5. `generate_image_prompts` - GenerateImagePromptsFunction -6. `generate_site_structure` - GenerateSiteStructureFunction -7. `optimize_content` - OptimizeContentFunction - -**Note**: `generate_page_content` exists as a function but may be called directly without registry registration. - -#### AI Core Handler -**File**: `backend/igny8_core/ai/ai_core.py` -**Class**: `AICore` -**Purpose**: Centralized AI request handler for all AI operations (text and image generation) -**Methods**: -- `run_ai_request` - Makes API call to OpenAI/Runware for text generation -- `generate_image` - Makes API call to OpenAI DALL-E or Runware for image generation -- `extract_json` - Extracts JSON from response (handles markdown code blocks) - -#### Prompt Registry -**File**: `backend/igny8_core/ai/prompts.py` -**Class**: `PromptRegistry` -**Purpose**: Centralized prompt management with hierarchical resolution -**Method**: `get_prompt` - Gets prompt with resolution order: -1. Task-level prompt_override (if exists) -2. DB prompt for (account, function) -3. Default fallback from DEFAULT_PROMPTS registry -**Prompt Types**: -- `clustering` - For auto_cluster function -- `ideas` - For generate_ideas function -- `content_generation` - For generate_content function -- `image_prompt_extraction` - For extract_image_prompts function -- `image_prompt_template` - Template for formatting image prompts -- `negative_prompt` - Negative prompt for Runware image generation -- `site_structure_generation` - For generate_site_structure function -- `content_optimization` - For optimize_content function -- `page_content_generation` - For generate_page_content function - -#### Model Settings -**File**: `backend/igny8_core/ai/settings.py` -**Constants**: `FUNCTION_ALIASES` - Function name aliases for backward compatibility -**Functions**: -- `get_model_config(function_name, account)` - Gets model config from IntegrationSettings (account required, no fallbacks) - - Raises `ValueError` if IntegrationSettings not configured - - Returns dict with `model`, `max_tokens`, `temperature`, `response_format` - ---- - -## AI Function Execution Flow - -### Complete Execution Pipeline - -``` -1. API Endpoint (views.py) - ↓ -2. run_ai_task (tasks.py) - - Gets account from account_id - - Gets function instance from registry - - Creates AIEngine - ↓ -3. AIEngine.execute (engine.py) - Phase 1: INIT (0-10%) - - Calls function.validate() - - Updates progress tracker - ↓ - Phase 2: PREP (10-25%) - - Calls function.prepare() - - Calls function.build_prompt() - - Updates progress tracker - ↓ - Phase 3: AI_CALL (25-70%) - - Gets model config from settings - - Calls AICore.run_ai_request() or AICore.generate_image() - - Tracks cost and tokens - - Updates progress tracker - ↓ - Phase 4: PARSE (70-85%) - - Calls function.parse_response() - - Updates progress tracker - ↓ - Phase 5: SAVE (85-98%) - - Calls function.save_output() - - Logs credit usage - - Updates progress tracker - ↓ - Phase 6: DONE (98-100%) - - Logs to AITaskLog - - Returns result -``` - -### Progress Updates - -**Progress Endpoint**: `/api/v1/system/settings/task_progress/{task_id}/` - -**Response Format**: -- `state`: Task state (PENDING, PROGRESS, SUCCESS, FAILURE) -- `meta`: Progress metadata - - `phase`: Current phase (INIT, PREP, AI_CALL, PARSE, SAVE, DONE) - - `percentage`: Progress percentage (0-100) - - `message`: User-friendly message - - `request_steps`: Array of request steps - - `response_steps`: Array of response steps - - `cost`: API cost in USD - - `tokens`: Token count - ---- - -## AI Functions - -### 1. Auto Cluster Keywords - -**Purpose**: Group related keywords into semantic clusters using AI - -**Function Class**: `AutoClusterFunction` -**File**: `backend/igny8_core/ai/functions/auto_cluster.py` - -**API Endpoint**: -- **ViewSet**: `KeywordViewSet` -- **Action**: `auto_cluster` -- **Method**: POST -- **URL Path**: `/v1/planner/keywords/auto_cluster/` -- **Payload**: `ids` (list[int]) - Keyword IDs - -**Function Methods**: -- `get_name()`: Returns `'auto_cluster'` -- `validate(payload, account)`: Validates keyword IDs exist -- `prepare(payload, account)`: Loads keywords from database -- `build_prompt(data, account)`: Builds clustering prompt with keyword data -- `parse_response(response, step_tracker)`: Parses cluster JSON response -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates Cluster records and links keywords - -**Input**: List of keyword IDs -**Output**: Cluster records created, keywords linked to clusters - -**Progress Messages**: -- INIT: "Validating keywords" -- PREP: "Preparing keyword clustering" -- AI_CALL: "Analyzing keyword relationships" -- PARSE: "Processing cluster data" -- SAVE: "Creating clusters" - -### 2. Generate Ideas - -**Purpose**: Generate content ideas from keyword clusters - -**Function Class**: `GenerateIdeasFunction` -**File**: `backend/igny8_core/ai/functions/generate_ideas.py` - -**API Endpoint**: -- **ViewSet**: `ClusterViewSet` -- **Action**: `auto_generate_ideas` -- **Method**: POST -- **URL Path**: `/v1/planner/clusters/auto_generate_ideas/` -- **Payload**: `ids` (list[int]) - Cluster IDs - -**Function Methods**: -- `get_name()`: Returns `'generate_ideas'` -- `validate(payload, account)`: Validates cluster IDs exist -- `prepare(payload, account)`: Loads clusters and keywords -- `build_prompt(data, account)`: Builds idea generation prompt with cluster data -- `parse_response(response, step_tracker)`: Parses ideas JSON response -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates ContentIdeas records - -**Input**: List of cluster IDs -**Output**: ContentIdeas records created - -**Progress Messages**: -- INIT: "Verifying cluster integrity" -- PREP: "Loading cluster keywords" -- AI_CALL: "Generating ideas with Igny8 Semantic AI" -- PARSE: "{count} high-opportunity idea(s) generated" -- SAVE: "Content Outline for Ideas generated" - -### 3. Generate Content - -**Purpose**: Generate blog post and article content from tasks - -**Function Class**: `GenerateContentFunction` -**File**: `backend/igny8_core/ai/functions/generate_content.py` - -**API Endpoint**: -- **ViewSet**: `TasksViewSet` -- **Action**: `auto_generate_content` -- **Method**: POST -- **URL Path**: `/v1/writer/tasks/auto_generate_content/` -- **Payload**: `ids` (list[int]) - Task IDs (max 50) - -**Function Methods**: -- `get_name()`: Returns `'generate_content'` -- `get_max_items()`: Returns `50` (max tasks per batch) -- `validate(payload, account)`: Validates task IDs exist -- `prepare(payload, account)`: Loads tasks with related data -- `build_prompt(data, account)`: Builds content generation prompt with task data -- `parse_response(response, step_tracker)`: Parses content (JSON or plain text) -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates/updates Content records - -**Input**: List of task IDs -**Output**: Content records created/updated with HTML content - -**Progress Messages**: -- INIT: "Initializing content generation" -- PREP: "Loading tasks and building prompts" -- AI_CALL: "Generating content with AI" -- PARSE: "Processing content" -- SAVE: "Saving content" - -### 4. Generate Image Prompts - -**Purpose**: Extract image prompts from content for generating images - -**Function Class**: `GenerateImagePromptsFunction` -**File**: `backend/igny8_core/ai/functions/generate_image_prompts.py` - -**API Endpoint**: -- **ViewSet**: `TasksViewSet` (via content) -- **Action**: `generate_image_prompts` -- **Method**: POST -- **URL Path**: `/v1/writer/content/generate_image_prompts/` -- **Payload**: `ids` (list[int]) - Content IDs - -**Function Methods**: -- `get_name()`: Returns `'generate_image_prompts'` -- `validate(payload, account)`: Validates content IDs exist -- `prepare(payload, account)`: Loads content records -- `build_prompt(data, account)`: Builds prompt extraction prompt with content HTML -- `parse_response(response, step_tracker)`: Parses image prompts JSON -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Updates Images records with prompts - -**Input**: List of content IDs -**Output**: Images records updated with prompts (featured, in-article) - -**Progress Messages**: -- INIT: "Validating content" -- PREP: "Preparing image prompt extraction" -- AI_CALL: "Extracting image prompts" -- PARSE: "Processing image prompts" -- SAVE: "Saving image prompts" - -### 5. Generate Images - -**Purpose**: Generate images using AI (OpenAI DALL-E or Runware) based on prompts - -**Function Class**: `GenerateImagesFunction` -**File**: `backend/igny8_core/ai/functions/generate_images.py` - -**API Endpoint**: -- **ViewSet**: `ImagesViewSet` -- **Action**: `generate_images` -- **Method**: POST -- **URL Path**: `/v1/writer/images/generate_images/` -- **Payload**: `ids` (list[int]) - Image IDs - -**Function Methods**: -- `get_name()`: Returns `'generate_images'` -- `validate(payload, account)`: Validates image IDs exist and have prompts -- `prepare(payload, account)`: Loads images with prompts -- `build_prompt(data, account)`: Formats image prompt with context -- `parse_response(response, step_tracker)`: Parses image URL from API response -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Updates Images records with image URLs - -**Input**: List of image IDs (with prompts) -**Output**: Images records updated with image URLs - -**Image Generation Settings**: -- Provider: 'openai' or 'runware' -- Model: Model name (e.g., 'dall-e-3', 'runware:97@1') -- Image Type: 'realistic', 'artistic', 'cartoon' -- Max In-Article Images: Max images per content -- Image Format: 'webp', 'jpg', 'png' -- Desktop/Mobile: Boolean flags - -**Progress Messages**: -- INIT: "Validating image prompts" -- PREP: "Preparing image generation" -- AI_CALL: "Creating image(s) with AI" -- PARSE: "Processing image response" -- SAVE: "Saving generated image(s)" - -### 6. Generate Site Structure - -**Purpose**: Create site/page architecture from business brief, objectives, and style guides - -**Function Class**: `GenerateSiteStructureFunction` -**File**: `backend/igny8_core/ai/functions/generate_site_structure.py` - -**API Endpoint**: -- **ViewSet**: `SiteBlueprintViewSet` -- **Action**: `generate_structure` -- **Method**: POST -- **URL Path**: `/v1/site-builder/blueprints/{id}/generate_structure/` -- **Payload**: `ids` (list[int]) - Site Blueprint IDs - -**Function Methods**: -- `get_name()`: Returns `'generate_site_structure'` -- `validate(payload, account)`: Validates blueprint ID exists -- `prepare(payload, account)`: Loads site blueprint with related data -- `build_prompt(data, account)`: Builds site structure generation prompt with business brief, objectives, style guides -- `parse_response(response, step_tracker)`: Parses site structure JSON (pages, blocks, navigation) -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates PageBlueprint records with blocks - -**Input**: Site Blueprint ID -**Output**: PageBlueprint records created with structured blocks - -**Progress Messages**: -- INIT: "Validating blueprint data…" -- PREP: "Preparing site context…" -- AI_CALL: "Generating site structure with AI…" -- PARSE: "Parsing generated blueprint…" -- SAVE: "Saving pages and blocks…" -- DONE: "Site structure ready!" - -### 7. Optimize Content - -**Purpose**: Optimize content for SEO, readability, and engagement - -**Function Class**: `OptimizeContentFunction` -**File**: `backend/igny8_core/ai/functions/optimize_content.py` - -**API Endpoint**: -- **ViewSet**: `OptimizerViewSet` -- **Action**: `optimize` or `analyze` -- **Method**: POST -- **URL Path**: `/v1/optimizer/optimize/` or `/v1/optimizer/analyze/` -- **Payload**: `ids` (list[int]) - Content IDs, `entry_point` (optional, str) - -**Function Methods**: -- `get_name()`: Returns `'optimize_content'` -- `validate(payload, account)`: Validates content IDs exist -- `prepare(payload, account)`: Loads content with related data -- `build_prompt(data, account)`: Builds optimization prompt with content, SEO targets, readability goals -- `parse_response(response, step_tracker)`: Parses optimized content JSON -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Updates Content records with optimized content and scores - -**Input**: List of content IDs -**Output**: Content records updated with optimized content, SEO scores, readability scores - -**Optimization Features**: -- SEO optimization (keyword density, meta descriptions, headings) -- Readability improvements (sentence length, paragraph structure) -- Engagement enhancements (call-to-actions, internal links) -- Content scoring (SEO score, readability score, engagement score) - -**Progress Messages**: -- INIT: "Validating content data…" -- PREP: "Preparing content context…" -- AI_CALL: "Optimizing content with AI…" -- PARSE: "Parsing optimized content…" -- SAVE: "Saving optimized content…" -- DONE: "Content optimized!" - -### 8. Generate Page Content - -**Purpose**: Generate structured page content with JSON blocks for Site Builder pages - -**Function Class**: `GeneratePageContentFunction` -**File**: `backend/igny8_core/ai/functions/generate_page_content.py` - -**API Endpoint**: -- **ViewSet**: `PageBlueprintViewSet` -- **Action**: `generate_content` or `regenerate` -- **Method**: POST -- **URL Path**: `/v1/site-builder/pages/{id}/generate_content/` or `/v1/site-builder/pages/{id}/regenerate/` -- **Payload**: `ids` (list[int]) - Page Blueprint IDs (max 20) - -**Function Methods**: -- `get_name()`: Returns `'generate_page_content'` -- `get_max_items()`: Returns `20` (max pages per batch) -- `validate(payload, account)`: Validates page blueprint IDs exist -- `prepare(payload, account)`: Loads page blueprints with related data -- `build_prompt(data, account)`: Builds page content generation prompt optimized for Site Builder -- `parse_response(response, step_tracker)`: Parses JSON blocks format -- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Updates PageBlueprint records with blocks_json - -**Input**: List of page blueprint IDs -**Output**: PageBlueprint records updated with structured blocks_json - -**Key Differences from Generate Content**: -- Outputs structured JSON blocks format instead of HTML -- Optimized prompts for site builder pages -- Supports page-specific context (site structure, navigation, related pages) -- Blocks can include: headings, paragraphs, images, CTAs, forms, etc. - -**Progress Messages**: -- INIT: "Initializing page content generation..." -- PREP: "Loading page blueprint and building prompt..." -- AI_CALL: "Generating structured content with AI..." -- PARSE: "Parsing JSON blocks..." -- SAVE: "Saving blocks to page..." -- DONE: "Page content generated!" - ---- - -## Progress Tracking - -### Progress Phases - -All AI functions follow the same 6-phase execution: - -1. **INIT** (0-10%): Validation phase -2. **PREP** (10-25%): Data preparation phase -3. **AI_CALL** (25-70%): AI API call phase -4. **PARSE** (70-85%): Response parsing phase -5. **SAVE** (85-98%): Database save phase -6. **DONE** (98-100%): Completion phase - -### Progress Updates - -**Frontend Polling**: Frontend polls `/api/v1/system/settings/task_progress/{task_id}/` every 1-2 seconds - -**Progress Response**: -- `state`: Task state -- `meta`: Progress metadata - - `phase`: Current phase - - `percentage`: Progress percentage - - `message`: User-friendly message - - `request_steps`: Request steps array - - `response_steps`: Response steps array - - `cost`: API cost - - `tokens`: Token count - -### Step Tracking - -**Request Steps**: Tracked during prompt building and AI call -**Response Steps**: Tracked during response parsing - -**Purpose**: Provides detailed logging for debugging and transparency - ---- - -## Cost Tracking - -### Cost Calculation - -**Text Generation**: -- Cost calculated based on model pricing (input tokens + output tokens) -- Tracked per request -- Stored in `CostTracker` - -**Image Generation**: -- Cost calculated based on provider pricing -- OpenAI DALL-E: Fixed cost per image -- Runware: Variable cost per image -- Tracked per image - -### Cost Storage - -**AITaskLog Model**: -- `cost`: Total cost for task -- `tokens`: Total tokens used - -**CreditUsageLog Model**: -- `cost_usd`: Cost in USD -- `credits_used`: Credits deducted - ---- - -## Prompt Management - -### Prompt Resolution Order - -1. **Task-Level Override**: If task has `prompt_override`, use it -2. **Database Prompt**: If account has custom prompt in database, use it -3. **Default Prompt**: Use default prompt from `DEFAULT_PROMPTS` registry - -### Prompt Customization - -**Per Account**: Custom prompts stored in `AIPrompt` model -**Per Function**: Different prompts for each function type -**Context Variables**: Prompts support context placeholders: -- `[IGNY8_KEYWORDS]` - Keyword list -- `[IGNY8_CLUSTERS]` - Cluster list -- `[IGNY8_CLUSTER_KEYWORDS]` - Cluster keywords -- `[IGNY8_IDEA]` - Idea data -- `[IGNY8_CLUSTER]` - Cluster data - ---- - -## Model Configuration - -### IntegrationSettings - Single Source of Truth - -**IMPORTANT**: As of the refactoring completed in 2025-01-XX, the AI framework uses **IntegrationSettings only** for model configuration. There are no hardcoded defaults or fallbacks. - -**IntegrationSettings Model**: -- `integration_type`: 'openai' or 'runware' (required) -- `account`: Account instance (required) - each account must configure their own models -- `is_active`: Boolean (must be True for configuration to be used) -- `config`: JSONField with model configuration (required) - - `model`: Model name (required) - e.g., 'gpt-4o-mini', 'gpt-4o', 'dall-e-3' - - `max_tokens`: Max tokens (optional, defaults to 4000) - - `temperature`: Temperature (optional, defaults to 0.7) - - `response_format`: Response format (optional, automatically set for JSON mode models) - -### Model Configuration Function - -**File**: `backend/igny8_core/ai/settings.py` - -**Function**: `get_model_config(function_name: str, account) -> Dict[str, Any]` - -**Behavior**: -- **Requires** `account` parameter (no longer optional) -- **Requires** IntegrationSettings to be configured for the account -- **Raises** `ValueError` with clear error messages if: - - Account not provided - - IntegrationSettings not found for account - - Model not configured in IntegrationSettings - - IntegrationSettings is inactive - -**Error Messages**: -- Missing account: `"Account is required for model configuration"` -- Missing IntegrationSettings: `"OpenAI IntegrationSettings not configured for account {id}. Please configure OpenAI settings in the integration page."` -- Missing model: `"Model not configured in IntegrationSettings for account {id}. Please set 'model' in OpenAI integration settings."` - -**Returns**: -```python -{ - 'model': str, # Model name from IntegrationSettings - 'max_tokens': int, # From config or default 4000 - 'temperature': float, # From config or default 0.7 - 'response_format': dict, # JSON mode for supported models, or None -} -``` - -### Account-Specific Configuration - -**Key Principle**: Each account must configure their own AI models. There are no global defaults. - -**Configuration Steps**: -1. Navigate to Settings → Integrations -2. Configure OpenAI integration settings -3. Set `model` in the configuration (required) -4. Optionally set `max_tokens` and `temperature` -5. Ensure integration is active - -**Supported Models**: -- Text generation: `gpt-4o-mini`, `gpt-4o`, `gpt-4-turbo`, etc. -- Image generation: `dall-e-3` (OpenAI) or `runware:97@1` (Runware) -- JSON mode: Automatically enabled for supported models (gpt-4o, gpt-4-turbo, etc.) - -### Function Aliases - -**File**: `backend/igny8_core/ai/settings.py` - -**FUNCTION_ALIASES**: Dictionary mapping legacy function names to current names -- `cluster_keywords` → `auto_cluster` -- `auto_cluster_keywords` → `auto_cluster` -- `auto_generate_ideas` → `generate_ideas` -- `auto_generate_content` → `generate_content` -- `auto_generate_images` → `generate_images` - -**Purpose**: Maintains backward compatibility with legacy function names. - -### Removed Functions - -The following helper functions were removed as part of the refactoring (they were never used): -- `get_model()` - Removed (use `get_model_config()['model']` instead) -- `get_max_tokens()` - Removed (use `get_model_config()['max_tokens']` instead) -- `get_temperature()` - Removed (use `get_model_config()['temperature']` instead) - -**Rationale**: These functions were redundant - `get_model_config()` already returns all needed values. - ---- - -## Summary - -The IGNY8 AI framework provides: - -1. **Unified Interface**: All AI functions use the same execution pipeline -2. **Consistent Execution**: All functions follow the same 6-phase flow -3. **Progress Tracking**: Real-time progress updates via Celery -4. **Cost Tracking**: Automatic cost and token tracking -5. **Error Handling**: Centralized error handling in AIEngine -6. **Prompt Management**: Hierarchical prompt resolution -7. **Model Configuration**: Per-account model overrides -8. **Database Logging**: Automatic logging to AITaskLog -9. **Extensibility**: Easy to add new AI functions -10. **Reliability**: Retry logic and error recovery -11. **8 AI Functions**: Complete coverage from planning to publishing -12. **Site Builder Integration**: Specialized functions for site structure and page content -13. **Content Optimization**: AI-powered content optimization and scoring - -This architecture ensures consistency, maintainability, and extensibility while providing a robust foundation for all AI operations in the IGNY8 platform, supporting 8 AI functions across the complete content workflow. - diff --git a/master-docs/06-FUNCTIONAL-BUSINESS-LOGIC.md b/master-docs/06-FUNCTIONAL-BUSINESS-LOGIC.md deleted file mode 100644 index 1cfc6a83..00000000 --- a/master-docs/06-FUNCTIONAL-BUSINESS-LOGIC.md +++ /dev/null @@ -1,1395 +0,0 @@ -# IGNY8 Functional Business Logic Documentation - -**Last Updated:** 2025-01-XX (Workflow updated - Wizard removed) -**Purpose:** Complete functional and business logic documentation covering all workflows, functions, features, and how the application works from a business perspective. - ---- - -## Table of Contents - -1. [Prerequisites](#prerequisites) -2. [System Overview](#system-overview) -3. [Complete User Workflow](#complete-user-workflow) -4. [Account and User Management](#account-and-user-management) -5. [Planner Module - Complete Workflows](#planner-module---complete-workflows) -6. [Writer Module - Complete Workflows](#writer-module---complete-workflows) -7. [Thinker Module - Complete Workflows](#thinker-module---complete-workflows) -8. [System Module - Complete Workflows](#system-module---complete-workflows) -9. [Billing Module - Complete Workflows](#billing-module---complete-workflows) -10. [AI Functions - Complete Details](#ai-functions---complete-details) -11. [Credit System - How It Works](#credit-system---how-it-works) -12. [WordPress Integration](#wordpress-integration) -13. [Data Flow and State Management](#data-flow-and-state-management) -14. [Complete Feature List](#complete-feature-list) - ---- - -## Prerequisites - -### Technology Stack Required - -**Backend Technologies:** -- Django -- Django REST Framework -- PostgreSQL -- Celery -- Redis -- Python - -**Frontend Technologies:** -- React -- TypeScript -- Vite -- Tailwind CSS -- Zustand -- React Router - -**Infrastructure:** -- Docker -- Docker Compose -- Caddy - -**External Services:** -- OpenAI API -- Runware API (optional) -- WordPress (optional) - ---- - -## System Overview - -IGNY8 is a SaaS platform for SEO keyword management and AI-driven content generation. The system operates on a multi-tenant architecture where each account is completely isolated. Users work within accounts, manage sites, organize content by sectors, and use AI to generate content from keywords. - -### Core Business Flow - -1. **Account Setup**: User creates account, sets up sites and sectors -2. **Keyword Management**: Import keywords, organize by sector, cluster related keywords -3. **Content Planning**: Generate content ideas from keyword clusters -4. **Content Creation**: Create tasks from ideas, generate AI content -5. **Image Generation**: Generate images for content -6. **Publishing**: Publish content to WordPress sites - -### Key Business Concepts - -- **Account**: Top-level organization containing all data -- **Site**: Workspace within account (can have multiple sites) -- **Sector**: Content category within site (1-5 per site) -- **Keywords**: SEO keywords organized by sector -- **Clusters**: AI-grouped related keywords -- **Content Ideas**: AI-generated content ideas from clusters -- **Tasks**: Content generation tasks created from ideas -- **Content**: Generated HTML blog posts and articles -- **Images**: AI-generated images for content -- **Credits**: Currency for AI operations - ---- - -## Complete User Workflow - -### Current Workflow Path (Updated 2025-01-XX) - -The Site Builder Wizard has been removed. The workflow now follows a direct path: - -``` -PLANNING → WRITER → OPTIMIZE → PUBLISH -``` - -**Direct Path:** Keywords/Clusters → Ideas → Tasks → Content (no wizard intermediary) - -### Phase Flow Summary - -``` -┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ -│ PLANNING │ --> │ WRITER │ --> │OPTIMIZE │ --> │ PUBLISH │ -│ │ │ │ │ │ │ │ -│Keywords │ │Content │ │Link & │ │Deploy │ -│Clusters │ │Generate │ │Optimize │ │Site │ -│ Ideas │ │ Images │ │ │ │ │ -└──────────┘ └──────────┘ └──────────┘ └──────────┘ -``` - -### Complete Workflow Steps - -#### Phase 1: Planning & Keyword Research - -1. **Keyword Discovery** - - Navigate to `/planner/keyword-opportunities` - - Browse keyword database - - Identify target keywords - -2. **Keyword Import** - - Navigate to `/planner/keywords` - - Import CSV or manual entry - - Keywords stored by sector - -3. **Keyword Clustering** - - Select keywords → "Auto-Cluster" - - AI clustering (1 credit per 30 keywords) - - Clusters created automatically - -4. **Cluster Management** - - Navigate to `/planner/clusters` - - Review & refine clusters - - Merge/split clusters as needed - - Clusters ready for content generation - -5. **Content Ideas Generation** - - Navigate to `/planner/ideas` - - Generate ideas from clusters (1 credit per idea) - - Review and refine ideas - - Queue ideas to Writer - -#### Phase 2: Content Creation (Writer) - -1. **Task Creation** - - Ideas automatically create tasks - - Navigate to `/writer/tasks` - - Review tasks from ideas - -2. **Content Generation** - - Select tasks → "Generate Content" - - AI content generation (3 credits per content) - - HTML content with structured blocks - - Meta tags automatically generated - -3. **Content Review** - - Navigate to `/writer/content` - - Review & edit HTML content - - Update metadata (title, description, keywords) - - Validate content structure - -4. **Image Generation** - - Navigate to `/writer/images` - - Generate image prompts (optional) - - Generate images (1 credit per image) - - Featured and in-article images - -#### Phase 3: Optimization & Linking - -1. **Internal Linking** - - Navigate to `/linker/content` - - System analyzes content - - Suggests internal links (cluster matches prioritized) - - Accept/reject link suggestions - -2. **Content Optimization** - - Navigate to `/optimizer/content` - - System scores content - - Review optimization suggestions - - Apply improvements - -#### Phase 4: Publishing - -1. **Content Validation** - - Navigate to `/sites/:id/posts/:postId` - - Validate entity type, cluster mapping, taxonomy - - Ensure all attributes complete - -2. **WordPress Publishing** - - Select content → "Publish to WordPress" - - System validates WordPress connection - - Content published to WordPress site - - Images uploaded and linked - -3. **Site Deployment** (for IGNY8-hosted sites) - - Navigate to `/sites/:id/deploy` - - Review cluster coverage - - Validate content completeness - - Deploy site to sites.igny8.com/{slug} - -### Workflow Variations - -#### Workflow A: New IGNY8-Hosted Site -- Complete build from scratch -- Full workflow: Planning → Writer → Optimize → Publish → Deploy - -#### Workflow B: Existing Site Management -- Add new content to existing site -- Manage existing content -- Update and republish content - -#### Workflow C: WordPress Sync - New Site -- Connect new WordPress site -- Sync existing content -- Continue with normal workflow - -#### Workflow D: WordPress Sync - Existing Site -- Connect existing WordPress site -- Import existing posts -- Sync and manage content - -### Site Builder Status - -**Removed Features:** -- ❌ Site Builder Wizard (6-step guided process) -- ❌ WorkflowState model and services -- ❌ Wizard UI components and routes - -**Available Features:** -- ✅ Site Blueprint APIs (create/manage via API) -- ✅ Site Builder models (SiteBlueprint, PageBlueprint, etc.) -- ✅ Direct workflow path (Planning → Writer) - -**Note:** Site blueprints can still be created and managed through API endpoints, but there is no guided UI wizard. Users proceed directly from Planning to Writer. - ---- - -## Account and User Management - -### Account Creation and Setup - -**Workflow:** -1. User visits signup page -2. User provides email and password -3. System creates Account record with default plan -4. System creates User record with "owner" role -5. System creates Subscription record linking account to plan -6. User receives confirmation -7. User signs in and receives JWT token -8. Frontend stores token and redirects to dashboard - -**Data Created:** -- Account record (name, slug, owner, plan, credits, status) -- User record (email, account, role: owner) -- Subscription record (account, plan, status) - -### User Roles and Permissions - -**Roles:** -- **Developer**: Full system access, bypasses all restrictions -- **Owner**: Full account access, manages users and billing -- **Admin**: Account admin, manages content and users -- **Editor**: Content editing access for granted sites only -- **Viewer**: Read-only access for granted sites only -- **System Bot**: System automation user - -**Access Control:** -- Owners and Admins: Automatic access to all sites in account -- Editors and Viewers: Require explicit site access grants -- Developers and System Bot: Access to all sites across all accounts - -### Site and Sector Management - -**Site Creation:** -1. User navigates to Settings → Sites -2. User creates site with name, domain, industry -3. System creates Site record linked to account -4. User can configure WordPress integration (URL, username, password) -5. Site becomes available for content organization - -**Sector Creation:** -1. User navigates to site details -2. User creates sector (1-5 active sectors per site) -3. System creates Sector record linked to site -4. Sector becomes available for keyword and content organization - -**Constraints:** -- Maximum 5 active sectors per site -- Multiple sites can be active simultaneously -- All content must belong to a site and sector - ---- - -## Planner Module - Complete Workflows - -### Keyword Management - -**Keyword Import (CSV):** -1. User navigates to Planner → Keywords -2. User clicks "Import CSV" action -3. User uploads CSV file with columns: keyword, volume, difficulty, intent -4. System validates CSV format and data -5. System checks for duplicates -6. System creates Keyword records for each row -7. Keywords are linked to selected site and sector -8. Keywords appear in table with status "active" - -**Keyword Import (Manual):** -1. User navigates to Planner → Keywords -2. User clicks "Add Keyword" action -3. User enters keyword, volume, difficulty, intent -4. System validates keyword uniqueness within sector -5. System creates Keyword record -6. Keyword appears in table - -**Keyword Filtering:** -- Filter by sector -- Filter by status (active, inactive, archived) -- Filter by intent (informational, navigational, transactional, commercial) -- Filter by cluster (clustered, unclustered) -- Filter by difficulty range (min/max) -- Filter by volume range (min/max) -- Search by keyword text - -**Keyword Bulk Operations:** -- Bulk delete: Select multiple keywords, delete all -- Bulk update status: Select multiple keywords, change status -- Bulk add from seed: Add keywords from seed keyword library - -**Keyword Export:** -1. User applies filters (optional) -2. User clicks "Export CSV" action -3. System generates CSV file with filtered keywords -4. User downloads CSV file - -### Keyword Clustering - -**Auto Cluster Workflow:** -1. User navigates to Planner → Keywords -2. User selects keywords (maximum 20 per batch) -3. User clicks "Auto Cluster" action -4. System validates keyword IDs and account credits -5. System queues Celery task (`run_ai_task` with function `auto_cluster`) -6. System returns task ID to frontend -7. Frontend polls progress endpoint (`/api/v1/system/settings/task_progress/{task_id}/`) -8. Celery worker processes task: - - Loads keywords from database - - Builds AI prompt with keyword data (keyword, volume, difficulty, intent) - - Calls OpenAI API with clustering prompt - - Parses AI response (cluster names and keyword assignments) - - Creates Cluster records for each cluster - - Links keywords to clusters (Many-to-Many relationship) - - Updates cluster metrics (keywords_count, volume) -9. Progress updates sent to frontend (phases: INIT, PREP, AI_CALL, PARSE, SAVE, DONE) -10. Task completes -11. Frontend displays new clusters -12. Credits deducted from account (1 credit per 30 keywords) - -**Manual Cluster Management:** -1. User navigates to Planner → Clusters -2. User views clusters with keyword counts and volumes -3. User can edit cluster name and description -4. User can add keywords to cluster -5. User can remove keywords from cluster -6. User can delete cluster (keywords remain, just unlinked) - -**Cluster Filtering:** -- Filter by sector -- Filter by status -- Search by cluster name -- Sort by keywords_count, volume, created_at - -### Content Idea Generation - -**Generate Ideas Workflow:** -1. User navigates to Planner → Clusters -2. User selects one cluster (maximum 1 per batch) -3. User clicks "Generate Ideas" action -4. System validates cluster ID and account credits -5. System queues Celery task (`run_ai_task` with function `generate_ideas`) -6. System returns task ID to frontend -7. Frontend polls progress endpoint -8. Celery worker processes task: - - Loads cluster and associated keywords - - Builds AI prompt with cluster data (cluster name, keywords, sector context) - - Calls OpenAI API with idea generation prompt - - Parses AI response (idea titles, descriptions, content types, structures) - - Creates ContentIdeas records for each idea - - Links ideas to cluster -9. Progress updates sent to frontend -10. Task completes -11. Frontend displays new content ideas -12. Credits deducted from account (1 credit per idea) - -**Content Ideas Management:** -1. User navigates to Planner → Ideas -2. User views ideas with titles, descriptions, content types -3. User can edit idea title and description -4. User can change idea status (active, inactive, archived) -5. User can delete idea -6. User can create tasks from ideas (moves to Writer module) - -**Idea Filtering:** -- Filter by sector -- Filter by cluster -- Filter by status -- Filter by content type -- Search by idea title - -**Bulk Queue to Writer:** -1. User selects multiple ideas -2. User clicks "Queue to Writer" action -3. System creates Task records for each idea -4. Tasks are linked to ideas and clusters -5. Tasks appear in Writer → Tasks page - ---- - -## Writer Module - Complete Workflows - -### Task Management - -**Task Creation:** -1. User navigates to Writer → Tasks -2. User can create task manually or from content ideas -3. If from ideas: System creates task with idea data (title, description, cluster, idea) -4. If manual: User enters title, description, selects cluster, content type, structure -5. System creates Task record -6. Task appears in table with status "draft" - -**Task Filtering:** -- Filter by sector -- Filter by cluster -- Filter by status (draft, in_progress, review, completed, archived) -- Filter by content type -- Filter by content structure -- Search by title or keywords - -**Task Bulk Operations:** -- Bulk delete: Select multiple tasks, delete all -- Bulk update status: Select multiple tasks, change status - -### Content Generation - -**Generate Content Workflow:** -1. User navigates to Writer → Tasks -2. User selects tasks (maximum 50 per batch) -3. User clicks "Generate Content" action -4. System validates task IDs and account credits -5. System queues Celery task (`run_ai_task` with function `generate_content`) -6. System returns task ID to frontend -7. Frontend polls progress endpoint -8. Celery worker processes task (for each task): - - Loads task and related data (cluster, keywords, idea, sector, site) - - Builds AI prompt with task data (title, description, keywords, content type, structure) - - Calls OpenAI API with content generation prompt - - Parses HTML content response - - Creates or updates Content record with HTML - - Calculates word count - - Updates task status to "review" -9. Progress updates sent to frontend (per task) -10. Task completes -11. Frontend displays generated content -12. Credits deducted from account (3 credits per content piece) - -**Content Management:** -1. User navigates to Writer → Content -2. User views content list with titles, word counts, status -3. User clicks content to view full HTML -4. User can edit content HTML directly -5. User can update content status (draft, review, published) -6. User can update meta title and description -7. User can update primary and secondary keywords - -**Content Filtering:** -- Filter by sector -- Filter by status -- Filter by content type -- Search by title or keywords -- Sort by word count, created_at, status - -**Content Views:** -- Drafts: Content with status "draft" -- Published: Content with status "published" and wp_post_id set -- All Content: All content regardless of status - -### Image Generation - -**Generate Image Prompts Workflow:** -1. User navigates to Writer → Content -2. User selects content items -3. User clicks "Generate Image Prompts" action (optional) -4. System validates content IDs -5. System queues Celery task (`run_ai_task` with function `generate_image_prompts`) -6. Celery worker processes task: - - Loads content HTML - - Builds AI prompt for image prompt extraction - - Calls OpenAI API - - Parses image prompts (featured image prompt, 1-6 in-article image prompts) - - Creates or updates Images records with prompts - - Sets image_type (featured, in_article) -7. Task completes -8. Images appear in Writer → Images page with prompts ready - -**Generate Images Workflow:** -1. User navigates to Writer → Images -2. User selects images with prompts -3. User clicks "Generate Images" action -4. System validates image IDs and account credits -5. System queues Celery task (`run_ai_task` with function `generate_images`) -6. System returns task ID to frontend -7. Frontend polls progress endpoint -8. Celery worker processes task (for each image sequentially): - - Loads image with prompt - - Determines image provider (OpenAI DALL-E or Runware) from integration settings - - Builds image generation request - - Calls OpenAI DALL-E API or Runware API - - Receives image URL - - Updates Images record with image_url - - Sets image status to "completed" -9. Progress updates sent to frontend (per image) -10. Task completes -11. Frontend displays generated images -12. Credits deducted from account (1 credit per image) - -**Image Management:** -1. User navigates to Writer → Images -2. User views images with prompts, URLs, types, status -3. User can view image by clicking -4. User can regenerate image (creates new image with same prompt) -5. User can delete image -6. User can update image status - -**Image Filtering:** -- Filter by content -- Filter by image type (featured, in_article, desktop, mobile) -- Filter by status -- Filter by task - -**Image Bulk Operations:** -- Bulk update status: Select multiple images, change status - -### WordPress Publishing - -**Publish to WordPress Workflow:** -1. User navigates to Writer → Content -2. User selects content to publish -3. User clicks "Publish to WordPress" action -4. System validates: - - Site has WordPress URL configured (wp_url field) - - Site has WordPress credentials (wp_username, wp_app_password) - - Content status is "review" or "draft" -5. System calls WordPress REST API: - - Authenticates with WordPress using credentials - - Creates post with content HTML (title from content meta_title or task title) - - Sets post content to content HTML - - Uploads featured image if available (Images record with image_type="featured") - - Uploads in-article images if available (Images records with image_type="in_article") - - Sets post status (draft or publish based on content status) - - Sets post meta (primary_keyword, secondary_keywords) -6. WordPress returns post ID -7. System updates Content record: - - Sets wp_post_id field - - Sets status to "published" -8. Frontend displays success message -9. User can view published content in WordPress - -**Publishing Requirements:** -- Site must have WordPress URL configured -- Site must have WordPress username and app password -- Content must have status "review" or "draft" -- Featured image is optional but recommended - ---- - -## Thinker Module - Complete Workflows - -### AI Prompts Management - -**Prompt Management:** -1. User navigates to Thinker → Prompts -2. User views all AI prompts used in the system -3. User can view prompt details (prompt_type, prompt_value, function_name) -4. User can edit prompt value -5. User can reset prompt to default value -6. System uses prompts in AI function execution - -**Prompt Types:** -- `auto_cluster`: Prompt for keyword clustering -- `generate_ideas`: Prompt for content idea generation -- `generate_content`: Prompt for content generation -- `generate_image_prompts`: Prompt for extracting image prompts from content -- `generate_images`: Prompt for image generation - -**Prompt Hierarchy:** -1. Account-specific prompt (if exists) -2. Default system prompt (fallback) - -### Author Profiles Management - -**Author Profile Creation:** -1. User navigates to Thinker → Author Profiles -2. User clicks "Add Profile" action -3. User enters name, description, tone, language -4. System creates AuthorProfile record -5. Profile becomes available for content generation - -**Author Profile Usage:** -- Author profiles are used in content generation prompts -- Profile tone and language influence AI-generated content style -- Multiple profiles can be created for different content styles - -**Author Profile Management:** -- View all profiles -- Edit profile details -- Delete profile - -### Strategies Management - -**Strategy Creation:** -1. User navigates to Thinker → Strategies -2. User clicks "Add Strategy" action -3. User enters name, description -4. User selects sector (optional, for sector-specific strategies) -5. User configures prompt types (which prompts this strategy affects) -6. User configures section logic (content structure preferences) -7. System creates Strategy record -8. Strategy becomes active and influences AI generation - -**Strategy Usage:** -- Strategies influence content generation when linked to sectors -- Strategy prompt types override default prompts -- Strategy section logic affects content structure - -**Strategy Management:** -- View all strategies -- Edit strategy details -- Activate/deactivate strategy -- Delete strategy - -### Image Testing - -**Image Testing Workflow:** -1. User navigates to Thinker → Image Testing -2. User enters image prompt -3. User selects image provider (OpenAI DALL-E or Runware) -4. User clicks "Generate Test Image" action -5. System calls image generation API -6. System displays generated image -7. User can test different prompts and providers -8. No credits deducted for testing (or minimal test credits) - ---- - -## System Module - Complete Workflows - -### Integration Settings - -**OpenAI Integration Setup:** -1. User navigates to Settings → Integration -2. User selects "OpenAI" integration type -3. User enters OpenAI API key -4. User clicks "Test Connection" (optional) -5. System makes test API call to OpenAI -6. System returns connection status -7. User saves integration settings -8. System stores API key in IntegrationSettings record -9. OpenAI integration ready for AI functions - -**Runware Integration Setup:** -1. User navigates to Settings → Integration -2. User selects "Runware" integration type -3. User enters Runware API key -4. User selects Runware model (e.g., `runware:97@1`) -5. User clicks "Test Connection" (optional) -6. System makes test API call to Runware -7. System returns connection status -8. User saves integration settings -9. System stores API key and model in IntegrationSettings record -10. Runware integration ready for image generation - -**Integration Test Functions:** -- Test OpenAI: Makes test API call, returns success/failure -- Test Runware: Makes test API call, returns success/failure -- Generate Test Image: Generates test image using selected provider - -### System Settings - -**General Settings:** -- Account name and details -- Site management -- Sector management -- Industry selection - -**User Management:** -- Add users to account -- Assign roles (owner, admin, editor, viewer) -- Grant site access (for editors/viewers) -- Remove users - -**Module Settings:** -- Enable/disable modules -- Configure module-specific settings - -**AI Settings:** -- Configure AI model preferences -- Set default models per function -- Configure token limits -- Set cost tracking preferences - -### System Status and Monitoring - -**Status Endpoint:** -- System health check -- Database connectivity -- Redis connectivity -- Celery worker status -- Service availability - -**Request Metrics:** -- API request counts -- Response times -- Error rates -- Resource usage (CPU, memory, I/O) - ---- - -## Billing Module - Complete Workflows - -### Credit Management - -**Credit Balance:** -- Each account has a credit balance -- Credits are displayed in Billing → Credits page -- Credits are deducted for AI operations -- Credits can be added via transactions - -**Credit Transactions:** -1. User navigates to Billing → Transactions -2. User views all credit transactions -3. Transaction types: - - Purchase: Credits purchased - - Subscription: Credits from subscription renewal - - Refund: Credits refunded - - Deduction: Credits used for AI operations - - Adjustment: Manual credit adjustments -4. Each transaction shows amount, balance_after, description, timestamp - -**Credit Usage Logs:** -1. User navigates to Billing → Usage -2. User views detailed usage logs -3. Each log entry shows: - - Operation type (clustering, ideas, content, images, reparse) - - Credits used - - Cost in USD - - Model used - - Input/output tokens - - Related object (keyword, cluster, task) - - Timestamp -4. Logs are filterable by operation type, date range - -### Credit Costs - -**Credit Costs per Operation:** -- Clustering: 1 credit per 30 keywords (base: 1 credit) -- Ideas: 1 credit per idea (base: 1 credit) -- Content: 3 credits per content piece (base: 3 credits) -- Images: 1 credit per image (base: 1 credit) -- Reparse: 1 credit per reparse (base: 1 credit) - -**Credit Deduction Process:** -1. User initiates AI operation -2. System checks account credit balance -3. System calculates required credits based on operation type -4. If insufficient credits: Operation fails with error -5. If sufficient credits: Operation proceeds -6. After operation completes: Credits deducted via CreditService -7. CreditTransaction record created (deduction type) -8. CreditUsageLog record created with operation details -9. Account credit balance updated - -### Plan Management - -**Plan Features:** -- Each account is linked to a plan via subscription -- Plans define limits: - - Maximum users - - Maximum sites - - Maximum keywords - - Maximum clusters - - Maximum content ideas - - Maximum AI requests - - Maximum word count - - Maximum images - - Included credits per month - - Credits per month (if different from included) - -**Plan Limits Enforcement:** -- System checks plan limits before operations -- Operations fail if limits exceeded -- Limits are checked at: - - Keyword import - - Cluster creation - - Idea generation - - Content generation - - Image generation - ---- - -## AI Functions - Complete Details - -### AI Function Execution Flow - -**Unified Entry Point:** -- All AI functions use `run_ai_task` Celery task -- Function name passed as parameter (auto_cluster, generate_ideas, generate_content, generate_image_prompts, generate_images) -- Payload contains function-specific data -- Account ID passed for credit checking and account isolation - -**Execution Phases:** -1. **INIT**: Initialize function, validate payload -2. **PREP**: Load data from database, prepare for AI call -3. **AI_CALL**: Make API call to OpenAI or Runware -4. **PARSE**: Parse AI response, extract structured data -5. **SAVE**: Save results to database -6. **DONE**: Task complete, return results - -**Progress Tracking:** -- Progress updates sent via Celery task progress -- Frontend polls progress endpoint -- Progress shows current phase and percentage -- Progress shows items processed (e.g., "3 of 10 keywords clustered") - -**Error Handling:** -- Validation errors: Returned immediately -- API errors: Logged, error returned to frontend -- Insufficient credits: Operation fails before AI call -- Network errors: Retried with exponential backoff - -### Auto Cluster Function - -**Function Name:** `auto_cluster` - -**Purpose:** Group related keywords into semantic clusters using AI - -**Input:** -- Keyword IDs (list, maximum 20) -- Sector ID (optional) - -**Process:** -1. Load keywords from database -2. Build prompt with keyword data (keyword, volume, difficulty, intent) -3. Call OpenAI API with clustering prompt -4. Parse response (cluster names, keyword assignments) -5. Create Cluster records -6. Link keywords to clusters -7. Update cluster metrics - -**Output:** -- Cluster records created -- Keywords linked to clusters -- Cluster metrics updated (keywords_count, volume) - -**Credits:** 1 credit per 30 keywords - -### Generate Ideas Function - -**Function Name:** `generate_ideas` - -**Purpose:** Generate SEO-optimized content ideas from keyword clusters - -**Input:** -- Cluster IDs (list, maximum 1 per batch) - -**Process:** -1. Load cluster and associated keywords -2. Build prompt with cluster data (name, keywords, sector context) -3. Call OpenAI API with idea generation prompt -4. Parse response (idea titles, descriptions, content types, structures) -5. Create ContentIdeas records -6. Link ideas to cluster - -**Output:** -- ContentIdeas records created -- Ideas linked to clusters - -**Credits:** 1 credit per idea - -### Generate Content Function - -**Function Name:** `generate_content` - -**Purpose:** Generate article content from task ideas - -**Input:** -- Task IDs (list, maximum 50 per batch) - -**Process:** -1. Load tasks with relationships (cluster, keywords, idea, sector, site) -2. For each task: - - Build prompt with task data (title, description, keywords, content type, structure) - - Call OpenAI API with content generation prompt - - Parse HTML content response - - Create or update Content record - - Calculate word count - - Update task status -3. Process tasks sequentially or in parallel (configurable) - -**Output:** -- Content records created/updated with HTML -- Task status updated to "review" -- Word count calculated - -**Credits:** 3 credits per content piece - -### Generate Image Prompts Function - -**Function Name:** `generate_image_prompts` - -**Purpose:** Extract image prompts from content HTML - -**Input:** -- Content IDs (list) - -**Process:** -1. Load content HTML -2. Build prompt for image prompt extraction -3. Call OpenAI API -4. Parse response (featured image prompt, 1-6 in-article image prompts) -5. Create or update Images records with prompts -6. Set image_type (featured, in_article) - -**Output:** -- Images records created/updated with prompts -- Image types set - -**Credits:** Included in content generation (no separate charge) - -### Generate Images Function - -**Function Name:** `generate_images` - -**Purpose:** Generate images using OpenAI DALL-E or Runware - -**Input:** -- Image IDs (list) - -**Process:** -1. Load images with prompts -2. Determine image provider from integration settings -3. For each image: - - Build image generation request - - Call OpenAI DALL-E API or Runware API - - Receive image URL - - Update Images record with image_url - - Set status to "completed" -4. Process images sequentially - -**Output:** -- Images records updated with image URLs -- Image status updated - -**Credits:** 1 credit per image - ---- - -## Credit System - How It Works - -### Credit Balance Management - -**Account Credits:** -- Each account has a `credits` field (integer) -- Credits start at 0 or plan-included credits -- Credits are deducted for AI operations -- Credits can be added via transactions - -**Credit Checking:** -- Before AI operation: System checks if account has sufficient credits -- If insufficient: Operation fails with `InsufficientCreditsError` -- If sufficient: Operation proceeds - -**Credit Deduction:** -- After AI operation completes: Credits deducted via `CreditService.deduct_credits()` -- Account credits field updated -- CreditTransaction record created (type: deduction, amount: negative) -- CreditUsageLog record created with operation details - -**Credit Addition:** -- Via transactions: `CreditService.add_credits()` called -- Account credits field updated -- CreditTransaction record created (type: purchase/subscription/refund/adjustment, amount: positive) - -### Credit Transaction Types - -**Purchase:** -- User purchases credits -- Amount: Positive integer -- Description: "Credit purchase" -- Balance after: Account credits + purchase amount - -**Subscription:** -- Credits from subscription renewal -- Amount: Plan's included_credits or credits_per_month -- Description: "Subscription renewal credits" -- Balance after: Account credits + subscription credits - -**Refund:** -- Credits refunded to account -- Amount: Positive integer -- Description: "Credit refund" -- Balance after: Account credits + refund amount - -**Deduction:** -- Credits used for AI operations -- Amount: Negative integer -- Description: Operation-specific (e.g., "Content generation") -- Balance after: Account credits - deduction amount - -**Adjustment:** -- Manual credit adjustments by admin -- Amount: Positive or negative integer -- Description: Admin-provided reason -- Balance after: Account credits + adjustment amount - -### Credit Usage Logging - -**Operation Types:** -- `clustering`: Keyword clustering operations -- `ideas`: Content idea generation -- `content`: Content generation -- `images`: Image generation -- `reparse`: Content reparse operations - -**Log Details:** -- Operation type -- Credits used -- Cost in USD (calculated from API costs) -- Model used (e.g., "gpt-4", "dall-e-3") -- Input tokens (if available) -- Output tokens (if available) -- Related object type (keyword, cluster, task, content, image) -- Related object ID -- Metadata (additional context) - ---- - -## WordPress Integration - -### WordPress Configuration - -**Site-Level Configuration:** -- Each site can have WordPress integration configured -- Fields: `wp_url`, `wp_username`, `wp_app_password` -- Configuration stored in Site model - -**WordPress App Password:** -- WordPress requires app password (not regular password) -- App password generated in WordPress user settings -- App password used for REST API authentication - -### Publishing Process - -**Publishing Workflow:** -1. User selects content to publish -2. System validates WordPress configuration -3. System authenticates with WordPress REST API -4. System creates WordPress post: - - Title: Content meta_title or task title - - Content: Content HTML - - Status: Draft or Publish (based on content status) - - Featured image: Uploaded if available - - In-article images: Uploaded if available - - Meta fields: Primary keyword, secondary keywords -5. WordPress returns post ID -6. System updates Content record: - - Sets `wp_post_id` field - - Sets `status` to "published" -7. User can view published content in WordPress - -**Image Upload:** -- Featured image: Images record with `image_type="featured"` -- In-article images: Images records with `image_type="in_article"` -- Images uploaded to WordPress media library -- Image URLs updated in WordPress post - -**Publishing Requirements:** -- Site must have `wp_url` configured -- Site must have `wp_username` and `wp_app_password` -- Content must have status "review" or "draft" -- WordPress REST API must be accessible - ---- - -## Data Flow and State Management - -### Frontend State Management - -**Zustand Stores:** -- **Auth Store**: User authentication, JWT token, account info -- **Site Store**: Current site selection, accessible sites -- **Sector Store**: Current sector selection, accessible sectors -- **Planner Store**: Planner module state (keywords, clusters, ideas) -- **Writer Store**: Writer module state (tasks, content, images) -- **Billing Store**: Credit balance, transactions, usage logs -- **Settings Store**: User settings, preferences -- **Page Size Store**: Table page size preferences -- **Column Visibility Store**: Table column visibility preferences - -**React Contexts:** -- **Theme Context**: Dark/light theme -- **Sidebar Context**: Sidebar open/closed state -- **Header Metrics Context**: Header KPI metrics -- **Toast Context**: Toast notifications - -### API Communication - -**API Client:** -- Frontend uses Fetch API for HTTP requests -- Base URL: `/api/v1/` -- JWT token included in Authorization header -- Request/response format: JSON - -**Request Flow:** -1. User action in frontend -2. Frontend makes API request -3. Backend middleware extracts account from JWT -4. Backend ViewSet processes request -5. Backend returns JSON response -6. Frontend updates state -7. Frontend updates UI - -**Response Format:** -- Success: `{success: true, data: {...}, message: "..."}` -- Error: `{success: false, error: "...", errors: {...}}` -- Pagination: `{count: N, next: "...", previous: "...", results: [...]}` - -### Celery Task Flow - -**Task Queueing:** -1. User action triggers API request -2. Backend ViewSet action queues Celery task -3. Task ID returned to frontend -4. Frontend stores task ID - -**Progress Polling:** -1. Frontend polls progress endpoint every 2-3 seconds -2. Backend returns progress (phase, percentage, items processed) -3. Frontend updates progress UI -4. Polling continues until task completes - -**Task Completion:** -1. Celery worker completes task -2. Results saved to database -3. Progress endpoint returns completion status -4. Frontend stops polling -5. Frontend refreshes data -6. Frontend displays results - ---- - -## Complete Feature List - -### Planner Module Features - -**Keywords:** -- CSV import -- Manual keyword entry -- Keyword filtering (sector, status, intent, cluster, difficulty, volume) -- Keyword search -- Keyword sorting -- Bulk delete -- Bulk status update -- Bulk add from seed keywords -- CSV export -- Auto cluster (AI-powered clustering) -- Keyword-to-cluster linking -- Keyword metrics display (volume, difficulty, intent) - -**Clusters:** -- Cluster listing -- Cluster creation (manual) -- Cluster editing (name, description) -- Cluster deletion -- Keyword-to-cluster management (add/remove) -- Cluster metrics (keywords_count, volume) -- Auto generate ideas (AI-powered idea generation) -- Cluster filtering (sector, status) -- Cluster search - -**Content Ideas:** -- Idea listing -- Idea creation (manual) -- Idea editing (title, description) -- Idea deletion -- Idea status management -- Bulk queue to writer (create tasks from ideas) -- Idea filtering (sector, cluster, status, content type) -- Idea search - -**Keyword Opportunities:** -- Opportunity analysis -- Keyword gap analysis -- Competitive analysis - -### Writer Module Features - -**Tasks:** -- Task listing -- Task creation (manual or from ideas) -- Task editing (title, description, content type, structure) -- Task deletion -- Task status management -- Auto generate content (AI-powered content generation) -- Task filtering (sector, cluster, status, content type, structure) -- Task search -- Bulk delete -- Bulk status update - -**Content:** -- Content listing -- Content viewing (full HTML) -- Content editing (HTML editor) -- Content status management -- Meta title and description editing -- Primary and secondary keywords editing -- Word count display -- Publish to WordPress -- Content filtering (sector, status, content type) -- Content search -- Drafts view -- Published view - -**Images:** -- Image listing -- Image viewing -- Image creation (manual) -- Image editing (prompt, type) -- Image deletion -- Generate image prompts (AI-powered prompt extraction) -- Generate images (AI-powered image generation) -- Image filtering (content, type, status, task) -- Image bulk status update -- Image regeneration - -### Thinker Module Features - -**Prompts:** -- Prompt listing -- Prompt viewing -- Prompt editing -- Prompt reset to default -- Prompt type filtering - -**Author Profiles:** -- Profile listing -- Profile creation -- Profile editing -- Profile deletion -- Profile usage in content generation - -**Strategies:** -- Strategy listing -- Strategy creation -- Strategy editing -- Strategy activation/deactivation -- Strategy deletion -- Sector-specific strategies -- Strategy prompt type configuration -- Strategy section logic configuration - -**Image Testing:** -- Test image generation -- Prompt testing -- Provider testing (OpenAI DALL-E, Runware) -- Image preview - -### System Module Features - -**Integration Settings:** -- OpenAI integration setup -- Runware integration setup -- API key management -- Connection testing -- Test image generation -- Model selection (Runware) - -**Settings:** -- General settings -- Account settings -- User settings -- Module settings -- AI settings - -**User Management:** -- User listing -- User creation -- User editing -- User deletion -- Role assignment -- Site access management - -**Site Management:** -- Site listing -- Site creation -- Site editing -- Site deletion -- WordPress integration configuration -- Industry assignment - -**Sector Management:** -- Sector listing -- Sector creation -- Sector editing -- Sector deletion -- Sector activation/deactivation -- Maximum 5 active sectors per site - -### Billing Module Features - -**Credits:** -- Credit balance display -- Credit transaction history -- Credit usage logs -- Credit cost display per operation - -**Transactions:** -- Transaction listing -- Transaction filtering (type, date range) -- Transaction details -- Balance tracking - -**Usage:** -- Usage log listing -- Usage log filtering (operation type, date range) -- Cost tracking -- Token tracking -- Model usage tracking - -**Plans:** -- Plan listing -- Plan details -- Plan limits display -- Subscription management - -### Authentication Features - -**Sign Up:** -- Account creation -- User creation (owner role) -- Subscription creation (default plan) -- Email validation -- Password validation - -**Sign In:** -- Email/password authentication -- JWT token generation -- Token storage (localStorage) -- Account context loading - -**Password Management:** -- Password change -- Password reset (planned) - -**Session Management:** -- Token expiration handling -- Token refresh (planned) -- Logout - -### Dashboard Features - -**Home Dashboard:** -- Account overview -- KPI metrics -- Recent activity -- Quick actions - -**Module Dashboards:** -- Planner Dashboard: Keyword metrics, cluster metrics, idea metrics -- Writer Dashboard: Task metrics, content metrics, image metrics -- Thinker Dashboard: Prompt metrics, strategy metrics -- Billing Dashboard: Credit balance, usage metrics - ---- - -## Summary - -IGNY8 operates as a complete SaaS platform for SEO keyword management and AI-driven content generation. The system follows a clear business flow: - -1. **Setup**: Users create accounts, set up sites and sectors -2. **Planning**: Users import keywords, cluster them with AI, generate content ideas -3. **Creation**: Users create tasks from ideas, generate AI content, generate images -4. **Publishing**: Users publish content to WordPress sites -5. **Management**: Users manage credits, transactions, settings, and integrations - -All operations are account-isolated, credit-based, and AI-powered. The system provides complete workflows from keyword import to WordPress publishing, with comprehensive filtering, search, and bulk operations throughout. - diff --git a/master-docs/AGENT_MASTER_REFERENCE.md b/master-docs/AGENT_MASTER_REFERENCE.md deleted file mode 100644 index ffbee4e5..00000000 --- a/master-docs/AGENT_MASTER_REFERENCE.md +++ /dev/null @@ -1,156 +0,0 @@ -# Agent Master Reference — IGNY8 - -Purpose: provide a single, navigable reference for any new agent (human or automated) to locate the exact files, entry points, and functional responsibilities across the IGNY8 monorepo. Use this as the starting point for feature work, bug fixes, or integrations. - -Contents -- Repository snapshot (concise tree) -- Key modules & responsibilities (table) -- Important functions / entrypoints (table) -- Quick agent workflow (5 steps) -- Setup & common commands - ---- - -**Repository Snapshot (Top-level, concise)** - -- `backend/` : Django project and related management scripts - - `manage.py` : Django CLI entry - - `requirements.txt` : Python deps - - `igny8_core/` : main Django package - - `settings.py`, `asgi.py`, `wsgi.py`, `celery.py` - - `api/`, `ai/`, `auth/`, `admin/`, `business/`, `modules/`, `utils/` - - `frontend/` (legacy frontend bundle inside backend) - -- `frontend/` : Vite + React application (modern front-end) - - `package.json`, `vite.config.ts`, `src/` (React + TS), `public/` - -- `master-docs/` : documentation; authoritative notes about architecture - -- `docker-compose.app.yml` : local dev stack (DB, cache, web, worker) - ---- - -**Key Modules & Purpose** - -| Module / Path | Purpose / Responsibility | Quick Notes | -|---|---|---| -| `backend/manage.py` | Django management entrypoint | Run migrations, tests, server | -| `igny8_core/settings.py` | Central config for Django | Env-driven; source of secrets/flags | -| `igny8_core/celery.py` | Celery app configuration | Worker + beat scheduling setup | -| `igny8_core/ai/` | AI orchestration layer | Models and engine integration | -| `igny8_core/api/` | REST endpoints for backend | Look here for API implementations | -| `frontend/` | Modern frontend app | `src/` contains components, hooks, services | -| `docker-compose.app.yml` | Local dev orchestration | DB, cache, web, worker definitions | -| `master-docs/` | Documentation and process| Use for architecture decisions and public docs | - ---- - -**Important Files & Entrypoints (where to start for common tasks)** - -| Task | File / Function | Where to look | -|---|---|---| -| Start backend locally | `backend/manage.py` | `python manage.py runserver` (see file for settings) | -| Celery worker/beat | `igny8_core/celery.py` | `celery -A igny8_core worker` and `celery -A igny8_core beat` | -| Add/inspect REST endpoint | `igny8_core/api/` | Search for view classes or `@api_view` functions | -| AI model orchestration | `igny8_core/ai/engine.py` (or `ai_core.py`) | See `05-AI-FRAMEWORK-IMPLEMENTATION.md` in `master-docs/` | -| Frontend page/component | `frontend/src/pages/` and `frontend/src/components/` | Use the `App` and route files in `src/` | - ---- - -**Function / Area Quick-Reference (examples)** - -- Celery and periodic tasks - - File: `igny8_core/celery.py` - - Purpose: configure Celery app, load periodic tasks (beat schedule is present at `backend/celerybeat-schedule`). - -- AI orchestration - - Files: `igny8_core/ai/ai_core.py`, `igny8_core/ai/engine.py` (or similarly named) - - Purpose: implement model selection, prompt orchestration, caching, and engine interfaces. - -- API handlers - - Path: `igny8_core/api/` (search for submodules per resource) - - Purpose: HTTP endpoints used by `frontend/` and external integrations. - -- Frontend data services - - Path: `frontend/src/services/` and `frontend/src/api/` - - Purpose: client-side API wrappers; coordinate expected backend response shapes. - ---- - -**Agent Quick Workflow — 5 Steps** - -1. Identify the feature or bug area - - Use this reference to map the user-facing feature to `frontend/` pages and `igny8_core/api/` endpoints. - - Search for the route or identifiable component in `frontend/src/`. - -2. Locate the backend API handler - - From the frontend service or API call, open the corresponding URL and find the view in `igny8_core/api/`. - - Inspect serializer, view, and model (if present) in the same directory. - -3. Check configuration and environment - - Open `igny8_core/settings.py` to find related flags and env vars. - - For scheduled/background work, inspect `igny8_core/celery.py` and `backend/celerybeat-schedule`. - -4. Implement change and run local checks - - Backend: create/modify files under `igny8_core/` and run `python manage.py test` or `pytest` as configured. - - Frontend: modify `frontend/src/` and run `npm run dev` or `npm run build` for production checks. - -5. Update documentation & tests - - Add or update `master-docs/*` when making cross-cutting changes. - - Update the `API-COMPLETE-REFERENCE.md` if API shapes change. - ---- - -**Common Commands (PowerShell-friendly)** - -Backend setup and run (PowerShell): - -``` -cd backend -python -m venv .venv -.\.venv\Scripts\Activate.ps1 -pip install -r requirements.txt -python manage.py migrate -python manage.py runserver -``` - -Run Celery worker/beat (PowerShell): - -``` -cd backend -.\.venv\Scripts\Activate.ps1 -celery -A igny8_core worker -l info -celery -A igny8_core beat -l info -``` - -Frontend dev (Windows PowerShell): - -``` -cd frontend -npm install -npm run dev -``` - -Docker compose local stack: - -``` -docker-compose -f docker-compose.app.yml up --build -``` - ---- - -**Guidance for new agents** - -- Always cross-check `master-docs/` entries when making architectural changes. -- When changing an API, update `master-docs/API-COMPLETE-REFERENCE.md` and the frontend wrappers in `frontend/src/services/`. -- For AI changes, refer to `master-docs/05-AI-FRAMEWORK-IMPLEMENTATION.md` before editing `igny8_core/ai/`. - ---- - -If you want, I can: - -- Expand the file tree to include all files per directory (full tree). -- Generate a CSV or JSON mapping of file → responsibilities for programmatic use. -- Add an index of search terms and likely filenames for quick grep commands. - -End of reference. diff --git a/master-docs/CURRENT_WORKFLOW_STATUS.md b/master-docs/CURRENT_WORKFLOW_STATUS.md deleted file mode 100644 index 8c0dc9c8..00000000 --- a/master-docs/CURRENT_WORKFLOW_STATUS.md +++ /dev/null @@ -1,193 +0,0 @@ -# IGNY8 Current Workflow Status - -**Last Updated:** 2025-01-XX -**Purpose:** Single source of truth for current system state, active work, and recent changes - ---- - -## Current System State - -### Workflow Path (Current) - -``` -PLANNING → WRITER → OPTIMIZE → PUBLISH -``` - -**Direct Path:** Keywords/Clusters → Ideas → Tasks → Content (no wizard intermediary) - -### Removed Features (2025-11-20) - -- **Site Builder Wizard** - 6-step guided wizard process completely removed -- **WorkflowState Model** - Backend model and services removed -- **Wizard Routes** - `/sites/builder`, `/sites/builder/preview`, `/sites/blueprints` routes removed -- **Wizard UI Components** - All wizard step components removed from frontend - -### Active Features - -- ✅ **Site Blueprint APIs** - All backend APIs for creating/managing blueprints still work -- ✅ **Site Builder Models** - SiteBlueprint, PageBlueprint, SiteBlueprintCluster, SiteBlueprintTaxonomy models active -- ✅ **Direct Workflow** - Users can go directly from Planning → Writer → Optimize → Publish -- ✅ **All Core Modules** - Planner, Writer, Optimizer, Publisher all functional - ---- - -## Recent Changes - -### Site Builder Wizard Removal (2025-11-20) - -**What Was Deleted:** - -#### Frontend Files -- `/frontend/src/pages/Sites/Builder/Wizard.tsx` -- `/frontend/src/pages/Sites/Builder/Preview.tsx` -- `/frontend/src/pages/Sites/Builder/Blueprints.tsx` -- All wizard step components (`steps/*.tsx`) -- Wizard components (`components/WizardProgress.tsx`, `components/HelperDrawer.tsx`) -- `/frontend/src/store/builderStore.ts` - -#### Backend Files -- `/backend/igny8_core/business/site_building/services/workflow_state_service.py` -- `/backend/igny8_core/business/site_building/services/wizard_context_service.py` -- `/backend/igny8_core/business/site_building/services/validators.py` - -#### Backend Model -- `WorkflowState` model removed from `models.py` - -#### Database -- Table: `igny8_site_blueprint_workflow_states` (dropped) - -### Leftover Code Cleanup (2025-01-XX) - -**Additional Cleanup Completed:** - -#### Frontend API Functions Removed -- `fetchWizardContext()` - Called non-existent `/workflow/context/` endpoint -- `updateWorkflowStep()` - Called non-existent `/workflow/step/` endpoint - -#### TypeScript Interfaces Removed -- `WorkflowState` interface - No longer needed (model removed) -- `WizardContext` interface - No longer needed (wizard removed) -- `workflow_state` field removed from `SiteBlueprint` interface - -#### Frontend Components Fixed -- **Planner Dashboard** (`frontend/src/pages/Planner/Dashboard.tsx`): - - Removed `incompleteBlueprints` state and filtering logic - - Removed incomplete workflows alert banner - - Removed unused `Alert` import - -**Files Modified:** -1. `frontend/src/services/api.ts` - Removed wizard/workflow functions and interfaces -2. `frontend/src/pages/Planner/Dashboard.tsx` - Removed workflow state references - -**Status:** ✅ All leftover wizard/workflow code has been cleaned up - ---- - -## What's Still Active (Site Builder) - -### Models Still Active -1. **SiteBlueprint** (`igny8_site_blueprints`) -2. **PageBlueprint** (`igny8_page_blueprints`) -3. **SiteBlueprintCluster** (`igny8_site_blueprint_clusters`) -4. **SiteBlueprintTaxonomy** (`igny8_site_blueprint_taxonomies`) -5. **BusinessType** (`igny8_site_builder_business_types`) -6. **AudienceProfile** (`igny8_site_builder_audience_profiles`) -7. **BrandPersonality** (`igny8_site_builder_brand_personalities`) -8. **HeroImageryDirection** (`igny8_site_builder_hero_imagery`) - -### API Endpoints Still Available -- `GET/POST /api/v1/site-builder/blueprints/` -- `GET/POST /api/v1/site-builder/pages/` -- `POST /api/v1/site-builder/blueprints/{id}/generate_structure/` -- `POST /api/v1/site-builder/blueprints/{id}/generate_all_pages/` -- `POST /api/v1/site-builder/blueprints/{id}/clusters/attach` -- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/` -- `GET /api/v1/site-builder/metadata/` - -### Services Still Active -- `StructureGenerationService` - AI structure generation -- `PageGenerationService` - Page content generation -- `TaxonomyService` - Taxonomy management -- `SiteBuilderFileService` - File management - ---- - -## Migration & Database State - -### Migration State -- All apps have single `0001_initial.py` migrations -- Clean migration state (no migration history) -- Database structure matches current models -- WorkflowState table removed from database - -### Database Backup -- **Backup:** `backup_postgres_20251120_232816.sql` (646KB) -- Contains all data except WorkflowState (intentionally excluded) - ---- - -## Current Workflow Details - -### Site Setup & Integration (New Flow) - -#### Step 1: Add Site (WordPress) -- User adds a new WordPress site from the welcome screen or sites page -- During site creation, user selects: - - **Industry**: From available industries (e.g., Technology, Healthcare, Finance) - - **Sectors**: Multiple sectors within the selected industry (e.g., SaaS, E-commerce, Mobile Apps) - - **Site Name**: Display name for the site - - **Website Address**: WordPress site URL -- Site is created with industry and sectors already configured - -#### Step 2: Integrate Site -- User is redirected to Site Settings → Integrations tab (`/sites/{id}/settings?tab=integrations`) -- **API Key Generation**: - - User generates a unique API key for the site - - API key is displayed and can be copied - - Key is stored securely in the integration credentials -- **Plugin Download**: - - User downloads the IGNY8 WP Bridge plugin directly from the page - - Plugin provides deeper WordPress integration than default WordPress app -- **WordPress Configuration**: - - User enters WordPress site URL - - User enters WordPress username - - User enters Application Password (created in WordPress admin) - - User enables/disables integration and two-way sync -- Integration is saved and connection is tested - -### Phase 1: Planning -- Keyword import and management -- Auto-clustering (1 credit per 30 keywords) -- Content idea generation (1 credit per idea) -- Queue ideas to Writer - -### Phase 2: Writer -- Task creation from ideas -- Content generation (3 credits per content) -- Image generation (1 credit per image) -- Content review and editing - -### Phase 3: Optimize -- Internal linking suggestions -- Content optimization scoring -- Apply improvements - -### Phase 4: Publish -- WordPress publishing (via IGNY8 WP Bridge plugin) -- Site deployment (for IGNY8-hosted sites) -- Content validation - ---- - -## Notes - -- Site blueprints can still be created/managed through API endpoints -- No guided UI wizard available -- Direct path from Planning to Writer is the standard workflow -- All core functionality remains intact - ---- - -**Last Updated:** 2025-01-XX -**Status:** Current and Active - diff --git a/master-docs/file-tree.md b/master-docs/file-tree.md deleted file mode 100644 index 236c3cfe..00000000 Binary files a/master-docs/file-tree.md and /dev/null differ