Remove obsolete migration and workflow files; delete Site Builder Wizard references and related components. Update documentation to reflect the removal of the WorkflowState model and streamline the site building process.

2025-11-21 00:59:34 +00:00
parent 6bb918bad6
commit c8adfe06d1
23 changed files with 1914 additions and 3493 deletions
--- a/master-docs/04-BACKEND-IMPLEMENTATION.md
+++ b/master-docs/04-BACKEND-IMPLEMENTATION.md
@@ -1,6 +1,6 @@
 # IGNY8 Backend Implementation Reference

-**Last Updated:** 2025-01-XX  
+**Last Updated:** 2025-01-XX (Added 6 missing modules: Linker, Optimizer, Publisher, Site Builder, Automation, Integration)  
 **Purpose:** Complete backend implementation reference covering project structure, models, ViewSets, serializers, Celery tasks, API endpoints, base classes, middleware, and utilities.

 ---
@@ -67,26 +67,56 @@ backend/igny8_core/
 │   └── urls.py               # Auth module URLs
 ├── modules/                  # Feature modules
 │   ├── planner/              # Keywords, Clusters, Ideas
-│   │   ├── models.py         # Keywords, Clusters, ContentIdeas models
 │   │   ├── views.py          # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
-│   │   ├── tasks.py          # Celery tasks for AI operations
 │   │   ├── serializers.py    # Model serializers
 │   │   └── urls.py           # Planner module URLs
 │   ├── writer/               # Tasks, Content, Images
-│   │   ├── models.py         # Tasks, Content, Images models
-│   │   ├── views.py          # TasksViewSet, ImagesViewSet
-│   │   ├── tasks.py          # Celery tasks for content/image generation
+│   │   ├── views.py          # TasksViewSet, ContentViewSet, ImagesViewSet
+│   │   ├── serializers.py    # Model serializers
 │   │   └── urls.py           # Writer module URLs
+│   ├── linker/               # Internal linking
+│   │   ├── views.py          # LinkerViewSet
+│   │   ├── serializers.py    # Link serializers
+│   │   └── urls.py           # Linker module URLs
+│   ├── optimizer/            # Content optimization
+│   │   ├── views.py          # OptimizerViewSet
+│   │   ├── serializers.py    # Optimizer serializers
+│   │   └── urls.py           # Optimizer module URLs
+│   ├── publisher/            # Publishing & deployment
+│   │   ├── views.py          # PublishingRecordViewSet, DeploymentRecordViewSet, PublisherViewSet
+│   │   └── urls.py           # Publisher module URLs
+│   ├── site_builder/         # Site blueprint management
+│   │   ├── views.py          # SiteBlueprintViewSet, PageBlueprintViewSet, SiteAssetView
+│   │   ├── serializers.py    # Site builder serializers
+│   │   └── urls.py           # Site Builder module URLs
+│   ├── automation/           # Automation rules
+│   │   ├── views.py          # AutomationRuleViewSet, ScheduledTaskViewSet
+│   │   ├── serializers.py   # Automation serializers
+│   │   └── urls.py           # Automation module URLs
+│   ├── integration/          # External integrations
+│   │   ├── views.py          # IntegrationViewSet
+│   │   └── urls.py           # Integration module URLs
 │   ├── system/               # Settings, Prompts, Integration
 │   │   ├── models.py         # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
 │   │   ├── views.py          # AIPromptViewSet, AuthorProfileViewSet
+│   │   ├── settings_views.py # SystemSettingsViewSet, AccountSettingsViewSet
 │   │   ├── integration_views.py  # IntegrationSettingsViewSet, task_progress
 │   │   ├── utils.py          # Default prompts, prompt loading
 │   │   └── urls.py           # System module URLs
 │   └── billing/              # Credits, Transactions, Usage
-│       ├── models.py         # CreditTransaction, CreditUsageLog models
-│       ├── views.py          # Billing ViewSets
-│       └── services.py       # CreditService
+│       ├── views.py          # CreditTransactionViewSet, CreditBalanceViewSet, CreditUsageViewSet
+│       ├── services.py       # CreditService
+│       └── urls.py           # Billing module URLs
+├── business/                 # Business logic layer
+│   ├── planning/             # Planner models (Keywords, Clusters, ContentIdeas)
+│   ├── content/              # Writer models (Tasks, Content, Images)
+│   ├── linking/              # Linker services
+│   ├── optimization/        # Optimizer services
+│   ├── publishing/           # Publisher models (PublishingRecord, DeploymentRecord)
+│   ├── site_building/        # Site Builder models (SiteBlueprint, PageBlueprint, etc.)
+│   ├── automation/           # Automation models (AutomationRule, ScheduledTask)
+│   ├── integration/          # Integration models (SiteIntegration)
+│   └── billing/              # Billing models (CreditTransaction, CreditUsageLog)
 ├── api/                      # API base classes
 │   ├── base.py               # AccountModelViewSet, SiteSectorModelViewSet
 │   └── pagination.py         # CustomPageNumberPagination
@@ -489,6 +519,17 @@ backend/igny8_core/
 - Filters: `status`, `cluster_id`, `content_type`, `content_structure`
 - Ordering: `title`, `created_at`, `word_count`, `status`

+#### ContentViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List content
+- `create()`: Create content
+- `retrieve()`: Get content details
+- `update()`: Update content
+- `destroy()`: Delete content
+- `publish_to_wordpress()`: Publish content to WordPress
+
 #### ImagesViewSet
 **Inherits**: `SiteSectorModelViewSet`

@@ -500,6 +541,150 @@ backend/igny8_core/
 - `destroy()`: Delete image
 - `generate_images()`: Generate images using AI

+### Linker ViewSets
+
+#### LinkerViewSet
+**Inherits**: `viewsets.ViewSet`
+
+**Actions**:
+- `process()`: Process a single content item for internal linking
+- `batch_process()`: Process multiple content items for internal linking
+
+**Purpose**: AI-powered internal linking suggestions based on cluster matches
+
+### Optimizer ViewSets
+
+#### OptimizerViewSet
+**Inherits**: `viewsets.ViewSet`
+
+**Actions**:
+- `optimize()`: Optimize content (auto-detects entry point)
+- `batch_optimize()`: Batch optimize multiple content items
+- `analyze()`: Analyze content without optimization
+
+**Purpose**: Content optimization and scoring
+
+### Publisher ViewSets
+
+#### PublishingRecordViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List publishing records
+- `create()`: Create publishing record
+- `retrieve()`: Get publishing record details
+- `update()`: Update publishing record
+- `destroy()`: Delete publishing record
+
+#### DeploymentRecordViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List deployment records
+- `create()`: Create deployment record
+- `retrieve()`: Get deployment record details
+- `update()`: Update deployment record
+- `destroy()`: Delete deployment record
+
+#### PublisherViewSet
+**Inherits**: `viewsets.ViewSet`
+
+**Actions**:
+- `publish()`: Publish content to destination
+- `deploy()`: Deploy site blueprint
+- `check_readiness()`: Check deployment readiness
+
+#### SiteDefinitionView
+**Inherits**: `APIView`
+
+**Purpose**: Public endpoint for Sites Renderer to fetch site definitions
+
+### Site Builder ViewSets
+
+#### SiteBlueprintViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List site blueprints
+- `create()`: Create site blueprint
+- `retrieve()`: Get site blueprint details
+- `update()`: Update site blueprint
+- `destroy()`: Delete site blueprint
+- `generate_structure()`: Generate site structure using AI
+- `generate_all_pages()`: Generate all pages for blueprint
+- `create_tasks()`: Create Writer tasks for pages
+- `progress()`: Get cluster-level completion status
+- `attach_clusters()`: Attach planner clusters to blueprint
+- `detach_clusters()`: Detach clusters from blueprint
+- `list_taxonomies()`: List taxonomies for blueprint
+- `create_taxonomy()`: Create taxonomy for blueprint
+- `import_taxonomies()`: Import taxonomies from external source
+- `bulk_delete()`: Bulk delete blueprints
+
+#### PageBlueprintViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List page blueprints
+- `create()`: Create page blueprint
+- `retrieve()`: Get page blueprint details
+- `update()`: Update page blueprint
+- `destroy()`: Delete page blueprint
+- `generate_content()`: Generate content for page
+- `regenerate()`: Regenerate page content
+
+#### SiteAssetView
+**Inherits**: `APIView`
+
+**Actions**:
+- `GET`: List files for site
+- `POST`: Upload file
+- `DELETE`: Delete file
+
+#### SiteBuilderMetadataView
+**Inherits**: `APIView`
+
+**Actions**:
+- `GET`: Get metadata (business types, audience profiles, brand personalities, hero imagery)
+
+### Automation ViewSets
+
+#### AutomationRuleViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List automation rules
+- `create()`: Create automation rule
+- `retrieve()`: Get automation rule details
+- `update()`: Update automation rule
+- `destroy()`: Delete automation rule
+- `execute()`: Manually execute automation rule
+
+#### ScheduledTaskViewSet
+**Inherits**: `AccountModelViewSet`
+
+**Actions**:
+- `list()`: List scheduled tasks
+- `create()`: Create scheduled task
+- `retrieve()`: Get scheduled task details
+- `update()`: Update scheduled task
+- `destroy()`: Delete scheduled task
+
+### Integration ViewSets
+
+#### IntegrationViewSet
+**Inherits**: `SiteSectorModelViewSet`
+
+**Actions**:
+- `list()`: List site integrations
+- `create()`: Create site integration
+- `retrieve()`: Get integration details
+- `update()`: Update integration
+- `destroy()`: Delete integration
+- `test_connection()`: Test connection to integrated platform
+- `sync()`: Sync content with integrated platform
+- `sync_health()`: Check sync health status
+
 ### System ViewSets

 #### IntegrationSettingsViewSet
@@ -575,6 +760,57 @@ backend/igny8_core/
 #### IntegrationSettingsSerializer
 **Fields**: All IntegrationSettings model fields

+### Linker Serializers
+
+#### LinkContentSerializer
+**Fields**: `content_id`
+
+#### BatchLinkContentSerializer
+**Fields**: `content_ids` (array)
+
+### Optimizer Serializers
+
+#### OptimizeContentSerializer
+**Fields**: `content_id`, `entry_point` (optional)
+
+#### BatchOptimizeContentSerializer
+**Fields**: `content_ids` (array), `entry_point` (optional)
+
+#### AnalyzeContentSerializer
+**Fields**: `content_id`
+
+### Publisher Serializers
+
+#### PublishingRecordSerializer
+**Fields**: All PublishingRecord model fields
+
+#### DeploymentRecordSerializer
+**Fields**: All DeploymentRecord model fields
+
+### Site Builder Serializers
+
+#### SiteBlueprintSerializer
+**Fields**: All SiteBlueprint model fields, includes nested `pages`
+
+#### PageBlueprintSerializer
+**Fields**: All PageBlueprint model fields
+
+#### SiteBuilderMetadataSerializer
+**Fields**: `business_types`, `audience_profiles`, `brand_personalities`, `hero_imagery_directions`
+
+### Automation Serializers
+
+#### AutomationRuleSerializer
+**Fields**: All AutomationRule model fields
+
+#### ScheduledTaskSerializer
+**Fields**: All ScheduledTask model fields
+
+### Integration Serializers
+
+#### SiteIntegrationSerializer
+**Fields**: All SiteIntegration model fields
+
 ---

 ## Celery Tasks
@@ -675,9 +911,77 @@ backend/igny8_core/
 ### Writer Endpoints

 - `GET /api/v1/writer/tasks/` - List tasks
+- `POST /api/v1/writer/tasks/` - Create task
+- `GET /api/v1/writer/tasks/{id}/` - Get task details
 - `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
+- `GET /api/v1/writer/content/` - List content
+- `GET /api/v1/writer/content/{id}/` - Get content details
 - `POST /api/v1/writer/images/generate_images/` - Generate images

+### Linker Endpoints
+
+- `POST /api/v1/linker/process/` - Process content for internal linking
+- `POST /api/v1/linker/batch_process/` - Batch process content for linking
+
+### Optimizer Endpoints
+
+- `POST /api/v1/optimizer/optimize/` - Optimize content
+- `POST /api/v1/optimizer/batch_optimize/` - Batch optimize content
+- `POST /api/v1/optimizer/analyze/` - Analyze content
+
+### Publisher Endpoints
+
+- `GET /api/v1/publisher/publishing-records/` - List publishing records
+- `POST /api/v1/publisher/publishing-records/` - Create publishing record
+- `GET /api/v1/publisher/deployments/` - List deployment records
+- `POST /api/v1/publisher/deployments/` - Create deployment record
+- `POST /api/v1/publisher/publish/` - Publish content
+- `POST /api/v1/publisher/deploy/` - Deploy site blueprint
+- `GET /api/v1/publisher/sites/{site_id}/definition/` - Get site definition (public)
+
+### Site Builder Endpoints
+
+- `GET /api/v1/site-builder/blueprints/` - List site blueprints
+- `POST /api/v1/site-builder/blueprints/` - Create site blueprint
+- `GET /api/v1/site-builder/blueprints/{id}/` - Get blueprint details
+- `POST /api/v1/site-builder/blueprints/{id}/generate_structure/` - Generate structure
+- `POST /api/v1/site-builder/blueprints/{id}/generate_all_pages/` - Generate all pages
+- `POST /api/v1/site-builder/blueprints/{id}/create_tasks/` - Create tasks for pages
+- `GET /api/v1/site-builder/blueprints/{id}/progress/` - Get progress
+- `POST /api/v1/site-builder/blueprints/{id}/clusters/attach/` - Attach clusters
+- `POST /api/v1/site-builder/blueprints/{id}/clusters/detach/` - Detach clusters
+- `GET /api/v1/site-builder/blueprints/{id}/taxonomies/` - List taxonomies
+- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/` - Create taxonomy
+- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/import/` - Import taxonomies
+- `POST /api/v1/site-builder/blueprints/bulk_delete/` - Bulk delete blueprints
+- `GET /api/v1/site-builder/pages/` - List page blueprints
+- `POST /api/v1/site-builder/pages/` - Create page blueprint
+- `GET /api/v1/site-builder/pages/{id}/` - Get page details
+- `POST /api/v1/site-builder/pages/{id}/generate_content/` - Generate content
+- `POST /api/v1/site-builder/pages/{id}/regenerate/` - Regenerate page
+- `GET /api/v1/site-builder/assets/` - List assets
+- `POST /api/v1/site-builder/assets/` - Upload asset
+- `DELETE /api/v1/site-builder/assets/` - Delete asset
+- `GET /api/v1/site-builder/metadata/` - Get metadata
+
+### Automation Endpoints
+
+- `GET /api/v1/automation/rules/` - List automation rules
+- `POST /api/v1/automation/rules/` - Create automation rule
+- `GET /api/v1/automation/rules/{id}/` - Get rule details
+- `POST /api/v1/automation/rules/{id}/execute/` - Execute rule
+- `GET /api/v1/automation/scheduled-tasks/` - List scheduled tasks
+- `POST /api/v1/automation/scheduled-tasks/` - Create scheduled task
+
+### Integration Endpoints
+
+- `GET /api/v1/integration/integrations/` - List site integrations
+- `POST /api/v1/integration/integrations/` - Create integration
+- `GET /api/v1/integration/integrations/{id}/` - Get integration details
+- `POST /api/v1/integration/integrations/{id}/test_connection/` - Test connection
+- `POST /api/v1/integration/integrations/{id}/sync/` - Sync content
+- `GET /api/v1/integration/integrations/{id}/sync_health/` - Get sync health
+
 ### System Endpoints

 - `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings
@@ -772,29 +1076,170 @@ backend/igny8_core/

 **Purpose**: Content generation and management

-**Models**: Tasks, Content, Images
+**Models** (in `business/content/`):
+- `Tasks`: Content generation tasks
+- `Content`: Generated content pieces
+- `Images`: Generated images
+- `ContentClusterMap`: Content-to-cluster mapping
+- `ContentTaxonomyMap`: Content taxonomy mapping
+- `ContentAttributeMap`: Content attribute mapping

-**ViewSets**: TasksViewSet, ImagesViewSet
+**ViewSets**:
+- `TasksViewSet`: CRUD + auto-generate content, bulk operations
+- `ContentViewSet`: CRUD + publish to WordPress
+- `ImagesViewSet`: CRUD + generate images

-**Tasks**: Auto-generate content, Generate images
+**URLs**: `/api/v1/writer/`
+
+### Linker Module
+
+**Purpose**: Internal linking operations
+
+**ViewSets**:
+- `LinkerViewSet`: Process content for internal linking
+
+**Actions**:
+- `process()`: Process single content item
+- `batch_process()`: Process multiple content items
+
+**URLs**: `/api/v1/linker/`
+
+### Optimizer Module
+
+**Purpose**: Content optimization and scoring
+
+**ViewSets**:
+- `OptimizerViewSet`: Content optimization operations
+
+**Actions**:
+- `optimize()`: Optimize content (auto-detects entry point)
+- `batch_optimize()`: Batch optimize content
+- `analyze()`: Analyze content without optimization
+
+**URLs**: `/api/v1/optimizer/`
+
+### Publisher Module
+
+**Purpose**: Publishing records and site deployment
+
+**Models** (in `business/publishing/`):
+- `PublishingRecord`: Records of content publishing
+- `DeploymentRecord`: Records of site deployments
+
+**ViewSets**:
+- `PublishingRecordViewSet`: CRUD for publishing records
+- `DeploymentRecordViewSet`: CRUD for deployment records
+- `PublisherViewSet`: Publishing and deployment actions
+- `SiteDefinitionView`: Public endpoint for Sites Renderer
+
+**Actions**:
+- `publish()`: Publish content to destination
+- `deploy()`: Deploy site blueprint
+- `check_readiness()`: Check deployment readiness
+
+**URLs**: `/api/v1/publisher/`
+
+### Site Builder Module
+
+**Purpose**: Site blueprint management
+
+**Models** (in `business/site_building/`):
+- `SiteBlueprint`: Site structure definitions
+- `PageBlueprint`: Page structure definitions
+- `SiteBlueprintCluster`: Cluster-to-blueprint mapping
+- `SiteBlueprintTaxonomy`: Taxonomy definitions
+- `SiteBuilderOption`: Site builder configuration options
+
+**ViewSets**:
+- `SiteBlueprintViewSet`: CRUD + structure generation, cluster management
+- `PageBlueprintViewSet`: CRUD + content generation
+- `SiteAssetView`: Asset management
+- `SiteBuilderMetadataView`: Metadata retrieval
+
+**Actions**:
+- `generate_structure()`: Generate site structure using AI
+- `generate_all_pages()`: Generate all pages for blueprint
+- `create_tasks()`: Create Writer tasks for pages
+- `progress()`: Get cluster-level completion status
+- `attach_clusters()`: Attach planner clusters
+- `detach_clusters()`: Detach clusters
+- `generate_content()`: Generate content for page
+- `regenerate()`: Regenerate page content
+
+**URLs**: `/api/v1/site-builder/`
+
+**Note**: Site Builder Wizard UI has been removed. Only API-based blueprint management remains.
+
+### Automation Module
+
+**Purpose**: Automation rules and scheduled tasks
+
+**Models** (in `business/automation/`):
+- `AutomationRule`: Rule-based automation definitions
+- `ScheduledTask`: Scheduled task definitions
+
+**ViewSets**:
+- `AutomationRuleViewSet`: CRUD + execute rule
+- `ScheduledTaskViewSet`: CRUD for scheduled tasks
+
+**Actions**:
+- `execute()`: Manually execute automation rule
+
+**URLs**: `/api/v1/automation/`
+
+### Integration Module
+
+**Purpose**: External platform integrations
+
+**Models** (in `business/integration/`):
+- `SiteIntegration`: Site-level integration configurations
+
+**ViewSets**:
+- `IntegrationViewSet`: CRUD + sync operations
+
+**Actions**:
+- `test_connection()`: Test connection to integrated platform
+- `sync()`: Sync content with integrated platform
+- `sync_health()`: Check sync health status
+
+**URLs**: `/api/v1/integration/`

 ### System Module

 **Purpose**: System configuration and AI settings

-**Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy
+**Models**:
+- `AIPrompt`: AI prompt templates
+- `IntegrationSettings`: Account-level integration settings
+- `AuthorProfile`: Author profile definitions
+- `Strategy`: Content strategy definitions

-**ViewSets**: AIPromptViewSet, IntegrationSettingsViewSet, AuthorProfileViewSet
+**ViewSets**:
+- `AIPromptViewSet`: CRUD + reset to default
+- `IntegrationSettingsViewSet`: Integration settings management
+- `AuthorProfileViewSet`: CRUD for author profiles
+- `SystemSettingsViewSet`: System settings management
+- `AccountSettingsViewSet`: Account settings management
+
+**URLs**: `/api/v1/system/`

 ### Billing Module

 **Purpose**: Credit management and usage tracking

-**Models**: CreditTransaction, CreditUsageLog
+**Models** (in `business/billing/`):
+- `CreditTransaction`: Credit transaction records
+- `CreditUsageLog`: Credit usage logging

-**ViewSets**: CreditTransactionViewSet, CreditUsageLogViewSet
+**ViewSets**:
+- `CreditTransactionViewSet`: CRUD for transactions
+- `CreditBalanceViewSet`: Credit balance retrieval
+- `CreditUsageViewSet`: Credit usage tracking

-**Services**: CreditService (check, deduct, add, calculate credits)
+**Services**:
+- `CreditService`: Credit operations (check, deduct, add, calculate)
+
+**URLs**: `/api/v1/billing/`

 ---

@@ -808,10 +1253,16 @@ The IGNY8 backend provides:
 4. **Hierarchical Organization**: Account > Site > Sector > Content structure
 5. **AI Framework**: Unified AI framework for all AI operations
 6. **Progress Tracking**: Real-time progress updates for Celery tasks
-7. **Module-Based Design**: Clear separation of concerns
+7. **Module-Based Design**: Clear separation of concerns across 10 modules
 8. **Base Classes**: Reusable ViewSets for common patterns
 9. **Middleware**: Account context and resource tracking
 10. **Utilities**: Content processing and AI integration
+11. **Internal Linking**: AI-powered internal linking suggestions
+12. **Content Optimization**: Content optimization and scoring
+13. **Publishing System**: Multi-destination publishing with deployment tracking
+14. **Site Builder**: Site blueprint and structure management (API-based)
+15. **Automation**: Rule-based automation and scheduled tasks
+16. **Integration**: External platform integrations and synchronization

-This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform.
+This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform with 10 backend modules covering the complete content workflow from planning to publishing.