Files
igny8/master-docs/04-BACKEND-IMPLEMENTATION.md

1269 lines
38 KiB
Markdown

# IGNY8 Backend Implementation Reference
**Last Updated:** 2025-01-XX (Added 6 missing modules: Linker, Optimizer, Publisher, Site Builder, Automation, Integration)
**Purpose:** Complete backend implementation reference covering project structure, models, ViewSets, serializers, Celery tasks, API endpoints, base classes, middleware, and utilities.
---
## Table of Contents
1. [Backend Overview](#backend-overview)
2. [Tech Stack](#tech-stack)
3. [Project Structure](#project-structure)
4. [Models](#models)
5. [ViewSets](#viewsets)
6. [Serializers](#serializers)
7. [Celery Tasks](#celery-tasks)
8. [API Endpoints](#api-endpoints)
9. [Base Classes](#base-classes)
10. [Middleware](#middleware)
11. [Utilities](#utilities)
12. [Modules](#modules)
---
## Backend Overview
The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery.
### Key Features
- **Multi-Tenancy**: Complete account isolation with automatic filtering
- **RESTful API**: DRF ViewSets with consistent response format
- **Celery Integration**: Asynchronous task processing for long-running operations
- **Account/Site/Sector Hierarchy**: Hierarchical data organization
- **AI Integration**: Unified AI framework for all AI operations
- **Progress Tracking**: Real-time progress updates for Celery tasks
---
## Tech Stack
### Core Technologies
- **Django 5.2+**: Web framework
- **Django REST Framework**: API framework
- **PostgreSQL**: Database
- **Celery**: Asynchronous task queue
- **Redis**: Celery broker and caching
### Key Libraries
- **django-filter**: Advanced filtering
- **djangorestframework-simplejwt**: JWT authentication
- **requests**: HTTP client for external APIs
- **python-dotenv**: Environment variable management
---
## Project Structure
```
backend/igny8_core/
├── auth/ # Multi-tenancy and authentication
│ ├── models.py # Account, User, Plan, Site, Sector, Industry models
│ ├── views.py # Account, User, Site, Sector ViewSets
│ ├── serializers.py # Account, User, Plan serializers
│ └── urls.py # Auth module URLs
├── modules/ # Feature modules
│ ├── planner/ # Keywords, Clusters, Ideas
│ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│ │ ├── serializers.py # Model serializers
│ │ └── urls.py # Planner module URLs
│ ├── writer/ # Tasks, Content, Images
│ │ ├── views.py # TasksViewSet, ContentViewSet, ImagesViewSet
│ │ ├── serializers.py # Model serializers
│ │ └── urls.py # Writer module URLs
│ ├── linker/ # Internal linking
│ │ ├── views.py # LinkerViewSet
│ │ ├── serializers.py # Link serializers
│ │ └── urls.py # Linker module URLs
│ ├── optimizer/ # Content optimization
│ │ ├── views.py # OptimizerViewSet
│ │ ├── serializers.py # Optimizer serializers
│ │ └── urls.py # Optimizer module URLs
│ ├── publisher/ # Publishing & deployment
│ │ ├── views.py # PublishingRecordViewSet, DeploymentRecordViewSet, PublisherViewSet
│ │ └── urls.py # Publisher module URLs
│ ├── site_builder/ # Site blueprint management
│ │ ├── views.py # SiteBlueprintViewSet, PageBlueprintViewSet, SiteAssetView
│ │ ├── serializers.py # Site builder serializers
│ │ └── urls.py # Site Builder module URLs
│ ├── automation/ # Automation rules
│ │ ├── views.py # AutomationRuleViewSet, ScheduledTaskViewSet
│ │ ├── serializers.py # Automation serializers
│ │ └── urls.py # Automation module URLs
│ ├── integration/ # External integrations
│ │ ├── views.py # IntegrationViewSet
│ │ └── urls.py # Integration module URLs
│ ├── system/ # Settings, Prompts, Integration
│ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet
│ │ ├── settings_views.py # SystemSettingsViewSet, AccountSettingsViewSet
│ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress
│ │ ├── utils.py # Default prompts, prompt loading
│ │ └── urls.py # System module URLs
│ └── billing/ # Credits, Transactions, Usage
│ ├── views.py # CreditTransactionViewSet, CreditBalanceViewSet, CreditUsageViewSet
│ ├── services.py # CreditService
│ └── urls.py # Billing module URLs
├── business/ # Business logic layer
│ ├── planning/ # Planner models (Keywords, Clusters, ContentIdeas)
│ ├── content/ # Writer models (Tasks, Content, Images)
│ ├── linking/ # Linker services
│ ├── optimization/ # Optimizer services
│ ├── publishing/ # Publisher models (PublishingRecord, DeploymentRecord)
│ ├── site_building/ # Site Builder models (SiteBlueprint, PageBlueprint, etc.)
│ ├── automation/ # Automation models (AutomationRule, ScheduledTask)
│ ├── integration/ # Integration models (SiteIntegration)
│ └── billing/ # Billing models (CreditTransaction, CreditUsageLog)
├── api/ # API base classes
│ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet
│ └── pagination.py # CustomPageNumberPagination
├── utils/ # Shared utilities
│ ├── ai_processor.py # Unified AI interface (legacy, see AI functions)
│ └── content_normalizer.py # Content processing utilities
├── middleware/ # Custom middleware
│ ├── account.py # AccountContextMiddleware (sets request.account)
│ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
├── ai/ # AI framework
│ ├── base.py # BaseAIFunction
│ ├── engine.py # AIEngine
│ ├── tasks.py # run_ai_task
│ ├── registry.py # Function registry
│ ├── prompts.py # PromptRegistry
│ ├── ai_core.py # AICore
│ ├── settings.py # Model configuration
│ ├── validators.py # Validation functions
│ ├── tracker.py # Progress tracking
│ └── functions/ # AI function implementations
│ ├── auto_cluster.py
│ ├── generate_ideas.py
│ ├── generate_content.py
│ ├── generate_image_prompts.py
│ └── generate_images.py
├── settings.py # Django settings
├── urls.py # Root URL configuration
└── celery.py # Celery configuration
```
---
## Models
### Base Models
#### AccountBaseModel
**File**: `auth/models.py`
**Purpose**: Base model for all account-isolated models.
**Fields**:
- `account`: ForeignKey to Account
- `created_at`: DateTimeField (auto_now_add)
- `updated_at`: DateTimeField (auto_now)
**Usage**: All models that need account isolation inherit from this.
#### SiteSectorBaseModel
**File**: `auth/models.py`
**Purpose**: Base model for models that belong to Site and Sector.
**Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site
- `sector`: ForeignKey to Sector
**Methods**:
- `save()`: Automatically sets `account` from `site.account` and validates sector belongs to site
**Usage**: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this.
### Auth Models
#### Account
**Table**: `igny8_accounts`
**Fields**:
- `name`: CharField
- `slug`: SlugField (unique)
- `owner`: ForeignKey to User
- `plan`: ForeignKey to Plan
- `credits`: IntegerField (default: 0)
- `status`: CharField (choices: active, suspended, trial, cancelled)
**Methods**:
- `is_system_account()`: Returns True if account is a system account
#### User
**Table**: `igny8_users`
**Inherits**: `AbstractUser`
**Fields**:
- `email`: EmailField (unique)
- `account`: ForeignKey to Account
- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot)
**Methods**:
- `has_role(role)`: Checks if user has role
- `is_owner_or_admin()`: Checks if user is owner or admin
- `is_developer()`: Checks if user is developer
- `is_admin_or_developer()`: Checks if user is admin or developer
- `is_system_account_user()`: Checks if user belongs to system account
- `get_accessible_sites()`: Returns queryset of accessible sites
#### Plan
**Table**: `igny8_plans`
**Fields**: Extensive fields for limits (users, sites, keywords, clusters, content ideas, AI requests, word count, images, credits)
**Methods**:
- `clean()`: Validates plan limits
- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month
#### Site
**Table**: `igny8_sites`
**Inherits**: `AccountBaseModel`
**Fields**:
- `name`: CharField
- `slug`: SlugField (unique per account)
- `domain`: URLField (optional)
- `industry`: ForeignKey to Industry (optional)
- `is_active`: BooleanField
- `status`: CharField
- `wp_url`: URLField (optional)
- `wp_username`: CharField (optional)
- `wp_app_password`: CharField (optional)
**Methods**:
- `get_active_sectors_count()`: Returns count of active sectors
- `can_add_sector()`: Returns True if site can add another sector (max 5)
#### Sector
**Table**: `igny8_sectors`
**Inherits**: `AccountBaseModel`
**Fields**:
- `site`: ForeignKey to Site
- `industry_sector`: ForeignKey to IndustrySector (optional)
- `name`: CharField
- `slug`: SlugField (unique per site)
- `is_active`: BooleanField
- `status`: CharField
**Validation**: Maximum 5 active sectors per site
### Planner Models
#### Keywords
**Table**: `igny8_keywords`
**Inherits**: `SiteSectorBaseModel`
**Fields**:
- `keyword`: CharField
- `volume`: IntegerField (optional)
- `difficulty`: IntegerField (optional)
- `intent`: CharField (optional)
- `status`: CharField
- `cluster`: ManyToManyField to Clusters
#### Clusters
**Table**: `igny8_clusters`
**Inherits**: `SiteSectorBaseModel`
**Fields**:
- `name`: CharField
- `description`: TextField (optional)
- `keywords_count`: IntegerField (calculated)
- `volume`: IntegerField (calculated)
- `status`: CharField
- `keywords`: ManyToManyField to Keywords
#### ContentIdeas
**Table**: `igny8_content_ideas`
**Inherits**: `SiteSectorBaseModel`
**Fields**:
- `idea_title`: CharField
- `description`: TextField
- `content_type`: CharField
- `content_structure`: CharField
- `target_keywords`: TextField
- `keyword_cluster`: ForeignKey to Clusters
- `estimated_word_count`: IntegerField
- `status`: CharField
### Writer Models
#### Tasks
**Table**: `igny8_tasks`
**Inherits**: `SiteSectorBaseModel`
**Fields**:
- `title`: CharField
- `description`: TextField
- `cluster`: ForeignKey to Clusters (optional)
- `idea`: ForeignKey to ContentIdeas (optional)
- `content_type`: CharField
- `content_structure`: CharField
- `status`: CharField
#### Content
**Table**: `igny8_content`
**Inherits**: `SiteSectorBaseModel`
**Fields**:
- `task`: OneToOneField to Tasks
- `html_content`: TextField
- `word_count`: IntegerField
- `status`: CharField
- `wp_post_id`: IntegerField (optional)
- `meta_title`: CharField (optional)
- `meta_description`: TextField (optional)
- `primary_keyword`: CharField (optional)
- `secondary_keywords`: TextField (optional)
#### Images
**Table**: `igny8_images`
**Inherits**: `SiteSectorBaseModel`
**Fields**:
- `task`: ForeignKey to Tasks (optional)
- `content`: ForeignKey to Content (optional)
- `image_type`: CharField (choices: featured, in_article, desktop, mobile)
- `prompt`: TextField
- `image_url`: CharField
- `status`: CharField
### System Models
#### AIPrompt
**Table**: `igny8_ai_prompts`
**Inherits**: `AccountBaseModel`
**Fields**:
- `prompt_type`: CharField
- `prompt_value`: TextField
- `function_name`: CharField (optional)
#### IntegrationSettings
**Table**: `igny8_integration_settings`
**Inherits**: `AccountBaseModel`
**Fields**:
- `integration_type`: CharField (choices: openai, runware, image_generation)
- `config`: JSONField
#### AuthorProfile
**Table**: `igny8_author_profiles`
**Inherits**: `AccountBaseModel`
**Fields**:
- `name`: CharField
- `description`: TextField
- `tone`: CharField
- `language`: CharField
#### Strategy
**Table**: `igny8_strategies`
**Inherits**: `AccountBaseModel`
**Fields**:
- `name`: CharField
- `description`: TextField
- `sector`: ForeignKey to Sector (optional)
- `prompt_types`: JSONField
- `section_logic`: JSONField
- `is_active`: BooleanField
### Billing Models
#### CreditTransaction
**Table**: `igny8_credit_transactions`
**Inherits**: `AccountBaseModel`
**Fields**:
- `transaction_type`: CharField (choices: purchase, usage, refund, adjustment)
- `amount`: IntegerField
- `balance_after`: IntegerField
- `description`: TextField (optional)
#### CreditUsageLog
**Table**: `igny8_credit_usage_logs`
**Inherits**: `AccountBaseModel`
**Fields**:
- `operation_type`: CharField
- `credits_used`: IntegerField
- `cost_usd`: DecimalField
- `details`: JSONField (optional)
---
## ViewSets
### Base ViewSets
#### AccountModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with automatic account filtering.
**Methods**:
- `get_queryset()`: Filters queryset by `request.account` (with admin/developer override)
- `perform_create()`: Sets account on created objects
- `get_serializer_context()`: Adds account to serializer context
**Access Control**:
- Admin/Developer users: Bypass account filtering
- System account users: Bypass account filtering
- Regular users: Only see data from their account
#### SiteSectorModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with site/sector filtering and access control.
**Inherits**: `AccountModelViewSet`
**Methods**:
- `get_queryset()`: Filters by account, accessible sites, and optional site_id/sector_id
- `perform_create()`: Validates site access and sector-site relationship
- `get_serializer_context()`: Adds accessible sites and sectors to context
**Access Control**:
- Developers: All active sites
- System account users: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`
### Planner ViewSets
#### KeywordViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List keywords with filtering
- `create()`: Create keyword
- `retrieve()`: Get keyword details
- `update()`: Update keyword
- `destroy()`: Delete keyword
- `auto_cluster()`: Auto-cluster keywords using AI
- `bulk_delete()`: Bulk delete keywords
- `bulk_update_status()`: Bulk update keyword status
- `export_csv()`: Export keywords to CSV
- `import_csv()`: Import keywords from CSV
**Filtering**:
- Search: `keyword` field
- Filters: `status`, `cluster_id`, `intent`
- Custom: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max`
- Ordering: `created_at`, `volume`, `difficulty`
#### ClusterViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List clusters
- `create()`: Create cluster
- `retrieve()`: Get cluster details
- `update()`: Update cluster
- `destroy()`: Delete cluster
- `auto_generate_ideas()`: Auto-generate content ideas for clusters
#### ContentIdeasViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List content ideas
- `create()`: Create content idea
- `retrieve()`: Get content idea details
- `update()`: Update content idea
- `destroy()`: Delete content idea
### Writer ViewSets
#### TasksViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List tasks
- `create()`: Create task
- `retrieve()`: Get task details
- `update()`: Update task
- `destroy()`: Delete task
- `auto_generate_content()`: Auto-generate content for tasks
- `generate_images()`: Generate images for image records
- `bulk_delete()`: Bulk delete tasks
- `bulk_update()`: Bulk update task status
**Filtering**:
- Search: `title`, `keywords`
- Filters: `status`, `cluster_id`, `content_type`, `content_structure`
- Ordering: `title`, `created_at`, `word_count`, `status`
#### ContentViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List content
- `create()`: Create content
- `retrieve()`: Get content details
- `update()`: Update content
- `destroy()`: Delete content
- `publish_to_wordpress()`: Publish content to WordPress
#### ImagesViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List images
- `create()`: Create image
- `retrieve()`: Get image details
- `update()`: Update image
- `destroy()`: Delete image
- `generate_images()`: Generate images using AI
### Linker ViewSets
#### LinkerViewSet
**Inherits**: `viewsets.ViewSet`
**Actions**:
- `process()`: Process a single content item for internal linking
- `batch_process()`: Process multiple content items for internal linking
**Purpose**: AI-powered internal linking suggestions based on cluster matches
### Optimizer ViewSets
#### OptimizerViewSet
**Inherits**: `viewsets.ViewSet`
**Actions**:
- `optimize()`: Optimize content (auto-detects entry point)
- `batch_optimize()`: Batch optimize multiple content items
- `analyze()`: Analyze content without optimization
**Purpose**: Content optimization and scoring
### Publisher ViewSets
#### PublishingRecordViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List publishing records
- `create()`: Create publishing record
- `retrieve()`: Get publishing record details
- `update()`: Update publishing record
- `destroy()`: Delete publishing record
#### DeploymentRecordViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List deployment records
- `create()`: Create deployment record
- `retrieve()`: Get deployment record details
- `update()`: Update deployment record
- `destroy()`: Delete deployment record
#### PublisherViewSet
**Inherits**: `viewsets.ViewSet`
**Actions**:
- `publish()`: Publish content to destination
- `deploy()`: Deploy site blueprint
- `check_readiness()`: Check deployment readiness
#### SiteDefinitionView
**Inherits**: `APIView`
**Purpose**: Public endpoint for Sites Renderer to fetch site definitions
### Site Builder ViewSets
#### SiteBlueprintViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List site blueprints
- `create()`: Create site blueprint
- `retrieve()`: Get site blueprint details
- `update()`: Update site blueprint
- `destroy()`: Delete site blueprint
- `generate_structure()`: Generate site structure using AI
- `generate_all_pages()`: Generate all pages for blueprint
- `create_tasks()`: Create Writer tasks for pages
- `progress()`: Get cluster-level completion status
- `attach_clusters()`: Attach planner clusters to blueprint
- `detach_clusters()`: Detach clusters from blueprint
- `list_taxonomies()`: List taxonomies for blueprint
- `create_taxonomy()`: Create taxonomy for blueprint
- `import_taxonomies()`: Import taxonomies from external source
- `bulk_delete()`: Bulk delete blueprints
#### PageBlueprintViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List page blueprints
- `create()`: Create page blueprint
- `retrieve()`: Get page blueprint details
- `update()`: Update page blueprint
- `destroy()`: Delete page blueprint
- `generate_content()`: Generate content for page
- `regenerate()`: Regenerate page content
#### SiteAssetView
**Inherits**: `APIView`
**Actions**:
- `GET`: List files for site
- `POST`: Upload file
- `DELETE`: Delete file
#### SiteBuilderMetadataView
**Inherits**: `APIView`
**Actions**:
- `GET`: Get metadata (business types, audience profiles, brand personalities, hero imagery)
### Automation ViewSets
#### AutomationRuleViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List automation rules
- `create()`: Create automation rule
- `retrieve()`: Get automation rule details
- `update()`: Update automation rule
- `destroy()`: Delete automation rule
- `execute()`: Manually execute automation rule
#### ScheduledTaskViewSet
**Inherits**: `AccountModelViewSet`
**Actions**:
- `list()`: List scheduled tasks
- `create()`: Create scheduled task
- `retrieve()`: Get scheduled task details
- `update()`: Update scheduled task
- `destroy()`: Delete scheduled task
### Integration ViewSets
#### IntegrationViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List site integrations
- `create()`: Create site integration
- `retrieve()`: Get integration details
- `update()`: Update integration
- `destroy()`: Delete integration
- `test_connection()`: Test connection to integrated platform
- `sync()`: Sync content with integrated platform
- `sync_health()`: Check sync health status
### System ViewSets
#### IntegrationSettingsViewSet
**Inherits**: `viewsets.ViewSet`
**Actions**:
- `list()`: List integrations
- `retrieve()`: Get integration settings
- `update()`: Save integration settings
- `save_post()`: Save integration settings (POST)
- `test_connection()`: Test API connection
- `test_openai()`: Test OpenAI connection
- `test_runware()`: Test Runware connection
- `generate_image()`: Test image generation
- `task_progress()`: Get Celery task progress
#### AIPromptViewSet
**Inherits**: `AccountModelViewSet`
**Actions**:
- `list()`: List prompts
- `create()`: Create prompt
- `retrieve()`: Get prompt details
- `update()`: Update prompt
- `destroy()`: Delete prompt
- `reset_to_default()`: Reset prompt to default value
#### AuthorProfileViewSet
**Inherits**: `AccountModelViewSet`
**Actions**:
- `list()`: List author profiles
- `create()`: Create author profile
- `retrieve()`: Get author profile details
- `update()`: Update author profile
- `destroy()`: Delete author profile
---
## Serializers
### Planner Serializers
#### KeywordSerializer
**Fields**: All Keyword model fields
**Validation**: Validates keyword uniqueness, cluster belongs to same sector
#### ClusterSerializer
**Fields**: All Cluster model fields
**Read-Only Fields**: `keywords_count`, `volume` (calculated)
#### ContentIdeasSerializer
**Fields**: All ContentIdeas model fields
### Writer Serializers
#### TasksSerializer
**Fields**: All Tasks model fields
#### ContentSerializer
**Fields**: All Content model fields
#### ImagesSerializer
**Fields**: All Images model fields
### System Serializers
#### AIPromptSerializer
**Fields**: All AIPrompt model fields
#### IntegrationSettingsSerializer
**Fields**: All IntegrationSettings model fields
### Linker Serializers
#### LinkContentSerializer
**Fields**: `content_id`
#### BatchLinkContentSerializer
**Fields**: `content_ids` (array)
### Optimizer Serializers
#### OptimizeContentSerializer
**Fields**: `content_id`, `entry_point` (optional)
#### BatchOptimizeContentSerializer
**Fields**: `content_ids` (array), `entry_point` (optional)
#### AnalyzeContentSerializer
**Fields**: `content_id`
### Publisher Serializers
#### PublishingRecordSerializer
**Fields**: All PublishingRecord model fields
#### DeploymentRecordSerializer
**Fields**: All DeploymentRecord model fields
### Site Builder Serializers
#### SiteBlueprintSerializer
**Fields**: All SiteBlueprint model fields, includes nested `pages`
#### PageBlueprintSerializer
**Fields**: All PageBlueprint model fields
#### SiteBuilderMetadataSerializer
**Fields**: `business_types`, `audience_profiles`, `brand_personalities`, `hero_imagery_directions`
### Automation Serializers
#### AutomationRuleSerializer
**Fields**: All AutomationRule model fields
#### ScheduledTaskSerializer
**Fields**: All ScheduledTask model fields
### Integration Serializers
#### SiteIntegrationSerializer
**Fields**: All SiteIntegration model fields
---
## Celery Tasks
### AI Task Entry Point
#### run_ai_task
**File**: `ai/tasks.py`
**Purpose**: Unified Celery task entrypoint for all AI functions
**Parameters**:
- `function_name`: Function name (e.g., 'auto_cluster')
- `payload`: Function-specific payload
- `account_id`: Account ID
**Flow**:
1. Gets account from account_id
2. Gets function instance from registry
3. Creates AIEngine
4. Executes function via AIEngine
### Planner Tasks
#### auto_cluster_keywords_task
**File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`)
**Purpose**: Auto-cluster keywords using AI
**Parameters**:
- `keyword_ids`: List of keyword IDs
- `account_id`: Account ID
- `site_id`: Site ID
- `sector_id`: Sector ID
**Progress Tracking**: Updates progress with request_steps and response_steps
#### auto_generate_ideas_task
**File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`)
**Purpose**: Auto-generate content ideas for clusters
**Parameters**:
- `cluster_ids`: List of cluster IDs
- `account_id`: Account ID
**Progress Tracking**: Updates progress for each cluster
### Writer Tasks
#### auto_generate_content_task
**File**: `modules/writer/tasks.py` (legacy, now uses `run_ai_task`)
**Purpose**: Auto-generate content for tasks
**Parameters**:
- `task_ids`: List of task IDs
- `account_id`: Account ID
**Progress Tracking**: Updates progress for each task
#### process_image_generation_queue
**File**: `modules/writer/views.py` (via `ai/tasks.py`)
**Purpose**: Generate images for image records
**Parameters**:
- `image_ids`: List of image IDs
- `account_id`: Account ID
- `content_id`: Content ID (optional)
**Progress Tracking**: Updates progress for each image sequentially
---
## API Endpoints
### Base URL
`/api/v1/`
### Planner Endpoints
- `GET /api/v1/planner/keywords/` - List keywords
- `POST /api/v1/planner/keywords/` - Create keyword
- `GET /api/v1/planner/keywords/{id}/` - Get keyword
- `PUT /api/v1/planner/keywords/{id}/` - Update keyword
- `DELETE /api/v1/planner/keywords/{id}/` - Delete keyword
- `POST /api/v1/planner/keywords/auto_cluster/` - Auto-cluster keywords
- `POST /api/v1/planner/keywords/bulk_delete/` - Bulk delete keywords
- `POST /api/v1/planner/keywords/bulk_update_status/` - Bulk update status
- `GET /api/v1/planner/keywords/export_csv/` - Export keywords
- `POST /api/v1/planner/keywords/import_csv/` - Import keywords
- `GET /api/v1/planner/clusters/` - List clusters
- `POST /api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas
- `GET /api/v1/planner/ideas/` - List content ideas
### Writer Endpoints
- `GET /api/v1/writer/tasks/` - List tasks
- `POST /api/v1/writer/tasks/` - Create task
- `GET /api/v1/writer/tasks/{id}/` - Get task details
- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
- `GET /api/v1/writer/content/` - List content
- `GET /api/v1/writer/content/{id}/` - Get content details
- `POST /api/v1/writer/images/generate_images/` - Generate images
### Linker Endpoints
- `POST /api/v1/linker/process/` - Process content for internal linking
- `POST /api/v1/linker/batch_process/` - Batch process content for linking
### Optimizer Endpoints
- `POST /api/v1/optimizer/optimize/` - Optimize content
- `POST /api/v1/optimizer/batch_optimize/` - Batch optimize content
- `POST /api/v1/optimizer/analyze/` - Analyze content
### Publisher Endpoints
- `GET /api/v1/publisher/publishing-records/` - List publishing records
- `POST /api/v1/publisher/publishing-records/` - Create publishing record
- `GET /api/v1/publisher/deployments/` - List deployment records
- `POST /api/v1/publisher/deployments/` - Create deployment record
- `POST /api/v1/publisher/publish/` - Publish content
- `POST /api/v1/publisher/deploy/` - Deploy site blueprint
- `GET /api/v1/publisher/sites/{site_id}/definition/` - Get site definition (public)
### Site Builder Endpoints
- `GET /api/v1/site-builder/blueprints/` - List site blueprints
- `POST /api/v1/site-builder/blueprints/` - Create site blueprint
- `GET /api/v1/site-builder/blueprints/{id}/` - Get blueprint details
- `POST /api/v1/site-builder/blueprints/{id}/generate_structure/` - Generate structure
- `POST /api/v1/site-builder/blueprints/{id}/generate_all_pages/` - Generate all pages
- `POST /api/v1/site-builder/blueprints/{id}/create_tasks/` - Create tasks for pages
- `GET /api/v1/site-builder/blueprints/{id}/progress/` - Get progress
- `POST /api/v1/site-builder/blueprints/{id}/clusters/attach/` - Attach clusters
- `POST /api/v1/site-builder/blueprints/{id}/clusters/detach/` - Detach clusters
- `GET /api/v1/site-builder/blueprints/{id}/taxonomies/` - List taxonomies
- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/` - Create taxonomy
- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/import/` - Import taxonomies
- `POST /api/v1/site-builder/blueprints/bulk_delete/` - Bulk delete blueprints
- `GET /api/v1/site-builder/pages/` - List page blueprints
- `POST /api/v1/site-builder/pages/` - Create page blueprint
- `GET /api/v1/site-builder/pages/{id}/` - Get page details
- `POST /api/v1/site-builder/pages/{id}/generate_content/` - Generate content
- `POST /api/v1/site-builder/pages/{id}/regenerate/` - Regenerate page
- `GET /api/v1/site-builder/assets/` - List assets
- `POST /api/v1/site-builder/assets/` - Upload asset
- `DELETE /api/v1/site-builder/assets/` - Delete asset
- `GET /api/v1/site-builder/metadata/` - Get metadata
### Automation Endpoints
- `GET /api/v1/automation/rules/` - List automation rules
- `POST /api/v1/automation/rules/` - Create automation rule
- `GET /api/v1/automation/rules/{id}/` - Get rule details
- `POST /api/v1/automation/rules/{id}/execute/` - Execute rule
- `GET /api/v1/automation/scheduled-tasks/` - List scheduled tasks
- `POST /api/v1/automation/scheduled-tasks/` - Create scheduled task
### Integration Endpoints
- `GET /api/v1/integration/integrations/` - List site integrations
- `POST /api/v1/integration/integrations/` - Create integration
- `GET /api/v1/integration/integrations/{id}/` - Get integration details
- `POST /api/v1/integration/integrations/{id}/test_connection/` - Test connection
- `POST /api/v1/integration/integrations/{id}/sync/` - Sync content
- `GET /api/v1/integration/integrations/{id}/sync_health/` - Get sync health
### System Endpoints
- `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings
- `PUT /api/v1/system/settings/integrations/{pk}/` - Save integration settings
- `POST /api/v1/system/settings/integrations/{pk}/test_openai/` - Test OpenAI
- `POST /api/v1/system/settings/integrations/{pk}/test_runware/` - Test Runware
- `POST /api/v1/system/settings/integrations/{pk}/generate_image/` - Test image generation
- `GET /api/v1/system/settings/task_progress/{task_id}/` - Get task progress
---
## Base Classes
### AccountModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with automatic account filtering
**Features**:
- Automatic account filtering
- Admin/Developer override
- Account context in serializers
### SiteSectorModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with site/sector filtering
**Features**:
- Account filtering (inherited)
- Site access control
- Sector validation
- Accessible sites/sectors in serializer context
---
## Middleware
### AccountContextMiddleware
**File**: `middleware/account.py`
**Purpose**: Sets `request.account` from JWT token
**Functionality**:
- Extracts account ID from JWT token
- Loads Account object
- Sets `request.account`
### ResourceTrackerMiddleware
**File**: `middleware/resource_tracker.py`
**Purpose**: Tracks API request metrics
**Functionality**:
- Tracks CPU, memory, I/O usage
- Stores metrics in cache
- Provides metrics endpoint
---
## Utilities
### Content Normalizer
**File**: `utils/content_normalizer.py`
**Purpose**: Content processing utilities
**Functions**:
- `normalize_content()`: Converts plain text to HTML
- `_extract_body_content()`: Extracts body content from HTML
---
## Modules
### Planner Module
**Purpose**: Keyword management and content planning
**Models**: Keywords, Clusters, ContentIdeas
**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
**Tasks**: Auto-cluster keywords, Auto-generate ideas
### Writer Module
**Purpose**: Content generation and management
**Models** (in `business/content/`):
- `Tasks`: Content generation tasks
- `Content`: Generated content pieces
- `Images`: Generated images
- `ContentClusterMap`: Content-to-cluster mapping
- `ContentTaxonomyMap`: Content taxonomy mapping
- `ContentAttributeMap`: Content attribute mapping
**ViewSets**:
- `TasksViewSet`: CRUD + auto-generate content, bulk operations
- `ContentViewSet`: CRUD + publish to WordPress
- `ImagesViewSet`: CRUD + generate images
**URLs**: `/api/v1/writer/`
### Linker Module
**Purpose**: Internal linking operations
**ViewSets**:
- `LinkerViewSet`: Process content for internal linking
**Actions**:
- `process()`: Process single content item
- `batch_process()`: Process multiple content items
**URLs**: `/api/v1/linker/`
### Optimizer Module
**Purpose**: Content optimization and scoring
**ViewSets**:
- `OptimizerViewSet`: Content optimization operations
**Actions**:
- `optimize()`: Optimize content (auto-detects entry point)
- `batch_optimize()`: Batch optimize content
- `analyze()`: Analyze content without optimization
**URLs**: `/api/v1/optimizer/`
### Publisher Module
**Purpose**: Publishing records and site deployment
**Models** (in `business/publishing/`):
- `PublishingRecord`: Records of content publishing
- `DeploymentRecord`: Records of site deployments
**ViewSets**:
- `PublishingRecordViewSet`: CRUD for publishing records
- `DeploymentRecordViewSet`: CRUD for deployment records
- `PublisherViewSet`: Publishing and deployment actions
- `SiteDefinitionView`: Public endpoint for Sites Renderer
**Actions**:
- `publish()`: Publish content to destination
- `deploy()`: Deploy site blueprint
- `check_readiness()`: Check deployment readiness
**URLs**: `/api/v1/publisher/`
### Site Builder Module
**Purpose**: Site blueprint management
**Models** (in `business/site_building/`):
- `SiteBlueprint`: Site structure definitions
- `PageBlueprint`: Page structure definitions
- `SiteBlueprintCluster`: Cluster-to-blueprint mapping
- `SiteBlueprintTaxonomy`: Taxonomy definitions
- `SiteBuilderOption`: Site builder configuration options
**ViewSets**:
- `SiteBlueprintViewSet`: CRUD + structure generation, cluster management
- `PageBlueprintViewSet`: CRUD + content generation
- `SiteAssetView`: Asset management
- `SiteBuilderMetadataView`: Metadata retrieval
**Actions**:
- `generate_structure()`: Generate site structure using AI
- `generate_all_pages()`: Generate all pages for blueprint
- `create_tasks()`: Create Writer tasks for pages
- `progress()`: Get cluster-level completion status
- `attach_clusters()`: Attach planner clusters
- `detach_clusters()`: Detach clusters
- `generate_content()`: Generate content for page
- `regenerate()`: Regenerate page content
**URLs**: `/api/v1/site-builder/`
**Note**: Site Builder Wizard UI has been removed. Only API-based blueprint management remains.
### Automation Module
**Purpose**: Automation rules and scheduled tasks
**Models** (in `business/automation/`):
- `AutomationRule`: Rule-based automation definitions
- `ScheduledTask`: Scheduled task definitions
**ViewSets**:
- `AutomationRuleViewSet`: CRUD + execute rule
- `ScheduledTaskViewSet`: CRUD for scheduled tasks
**Actions**:
- `execute()`: Manually execute automation rule
**URLs**: `/api/v1/automation/`
### Integration Module
**Purpose**: External platform integrations
**Models** (in `business/integration/`):
- `SiteIntegration`: Site-level integration configurations
**ViewSets**:
- `IntegrationViewSet`: CRUD + sync operations
**Actions**:
- `test_connection()`: Test connection to integrated platform
- `sync()`: Sync content with integrated platform
- `sync_health()`: Check sync health status
**URLs**: `/api/v1/integration/`
### System Module
**Purpose**: System configuration and AI settings
**Models**:
- `AIPrompt`: AI prompt templates
- `IntegrationSettings`: Account-level integration settings
- `AuthorProfile`: Author profile definitions
- `Strategy`: Content strategy definitions
**ViewSets**:
- `AIPromptViewSet`: CRUD + reset to default
- `IntegrationSettingsViewSet`: Integration settings management
- `AuthorProfileViewSet`: CRUD for author profiles
- `SystemSettingsViewSet`: System settings management
- `AccountSettingsViewSet`: Account settings management
**URLs**: `/api/v1/system/`
### Billing Module
**Purpose**: Credit management and usage tracking
**Models** (in `business/billing/`):
- `CreditTransaction`: Credit transaction records
- `CreditUsageLog`: Credit usage logging
**ViewSets**:
- `CreditTransactionViewSet`: CRUD for transactions
- `CreditBalanceViewSet`: Credit balance retrieval
- `CreditUsageViewSet`: Credit usage tracking
**Services**:
- `CreditService`: Credit operations (check, deduct, add, calculate)
**URLs**: `/api/v1/billing/`
---
## Summary
The IGNY8 backend provides:
1. **Multi-Tenancy**: Complete account isolation with automatic filtering
2. **RESTful API**: DRF ViewSets with consistent response format
3. **Celery Integration**: Asynchronous task processing
4. **Hierarchical Organization**: Account > Site > Sector > Content structure
5. **AI Framework**: Unified AI framework for all AI operations
6. **Progress Tracking**: Real-time progress updates for Celery tasks
7. **Module-Based Design**: Clear separation of concerns across 10 modules
8. **Base Classes**: Reusable ViewSets for common patterns
9. **Middleware**: Account context and resource tracking
10. **Utilities**: Content processing and AI integration
11. **Internal Linking**: AI-powered internal linking suggestions
12. **Content Optimization**: Content optimization and scoring
13. **Publishing System**: Multi-destination publishing with deployment tracking
14. **Site Builder**: Site blueprint and structure management (API-based)
15. **Automation**: Rule-based automation and scheduled tasks
16. **Integration**: External platform integrations and synchronization
This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform with 10 backend modules covering the complete content workflow from planning to publishing.