Remove obsolete scripts and files, update site builder configurations
- Deleted the `import_plans.py`, `run_tests.py`, and `test_run.py` scripts as they are no longer needed. - Updated the initial migration dependency in `0001_initial.py` to reflect recent changes in the `igny8_core_auth` app. - Enhanced the implementation plan documentation to include new phases and updates on the site builder project. - Updated the `vite.config.ts` and `package.json` to integrate testing configurations and dependencies for the site builder.
This commit is contained in:
IGNY8 VPS (Salman)
@@ -2,19 +2,450 @@
|
||||
**Detailed Configuration Plan for Site Builder & Linker/Optimizer**
|
||||
|
||||
**Created**: 2025-01-XX
|
||||
**Status**: Planning Phase
|
||||
**Status**: Phase 3 Complete ✅ | Phase 4 Backend Complete ✅ | Phase 4 Frontend Pending
|
||||
|
||||
---
|
||||
|
||||
## TABLE OF CONTENTS
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Phase 3: Site Builder Implementation Plan](#phase-3-site-builder-implementation-plan)
|
||||
3. [Phase 4: Linker & Optimizer Implementation Plan](#phase-4-linker--optimizer-implementation-plan)
|
||||
4. [Integration Points](#integration-points)
|
||||
5. [File Structure](#file-structure)
|
||||
6. [Dependencies & Order](#dependencies--order)
|
||||
7. [Testing Strategy](#testing-strategy)
|
||||
1. [Implementation Summary](#implementation-summary) ⭐ **NEW**
|
||||
2. [Overview](#overview)
|
||||
3. [Phase 3: Site Builder Implementation Plan](#phase-3-site-builder-implementation-plan)
|
||||
4. [Phase 4: Linker & Optimizer Implementation Plan](#phase-4-linker--optimizer-implementation-plan)
|
||||
5. [Integration Points](#integration-points)
|
||||
6. [File Structure](#file-structure)
|
||||
7. [Dependencies & Order](#dependencies--order)
|
||||
8. [Testing Strategy](#testing-strategy)
|
||||
|
||||
---
|
||||
|
||||
## IMPLEMENTATION SUMMARY
|
||||
|
||||
**Last Updated**: 2025-01-XX
|
||||
**Session**: Phase 3 & 4 Implementation Session
|
||||
|
||||
### ✅ Phase 3: Site Builder - COMPLETE
|
||||
|
||||
#### Backend Implementation
|
||||
|
||||
**Models Created** (`backend/igny8_core/business/site_building/models.py`):
|
||||
- ✅ `SiteBlueprint` model with fields:
|
||||
- `name`, `description`
|
||||
- `config_json` (wizard configuration)
|
||||
- `structure_json` (AI-generated structure)
|
||||
- `status` (draft, generating, ready, deployed)
|
||||
- `hosting_type` (igny8_sites, wordpress, shopify, multi)
|
||||
- `version`, `deployed_version`
|
||||
- Inherits from `SiteSectorBaseModel` (account, site, sector)
|
||||
- ✅ `PageBlueprint` model with fields:
|
||||
- `site_blueprint` (ForeignKey)
|
||||
- `slug`, `title`, `type`
|
||||
- `blocks_json` (page content blocks)
|
||||
- `status` (draft, generating, ready)
|
||||
- `order`
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
|
||||
**Services Created**:
|
||||
- ✅ `StructureGenerationService` (`business/site_building/services/structure_generation_service.py`)
|
||||
- `generate_structure()` - Triggers AI structure generation
|
||||
- Integrates with `CreditService` for credit checks
|
||||
- Dispatches Celery tasks for async processing
|
||||
- ✅ `PageGenerationService` (`business/site_building/services/page_generation_service.py`)
|
||||
- `generate_page_content()` - Creates Writer tasks from page blueprints
|
||||
- `regenerate_page()` - Regenerates page content
|
||||
- Integrates with `ContentGenerationService`
|
||||
- ✅ `SiteBuilderFileService` (`business/site_building/services/file_management_service.py`)
|
||||
- `upload_file()` - Handles file uploads with access control
|
||||
- `delete_file()` - Deletes files with validation
|
||||
- `list_files()` - Lists files for a site
|
||||
- `check_file_access()` - Validates user access to site files
|
||||
|
||||
**AI Function Created**:
|
||||
- ✅ `GenerateSiteStructureFunction` (`ai/functions/generate_site_structure.py`)
|
||||
- Operation type: `site_structure_generation`
|
||||
- Credit cost: 50 credits (from constants)
|
||||
- Parses AI response JSON and syncs `PageBlueprint` instances
|
||||
- Handles page creation, updates, and deletions
|
||||
- ✅ Added to AI registry (`ai/registry.py`)
|
||||
- ✅ Added prompt to `ai/prompts.py` (`site_structure_generation`)
|
||||
- ✅ Integrated into `ai/engine.py` with phase tracking
|
||||
|
||||
**API Layer Created** (`modules/site_builder/`):
|
||||
- ✅ `SiteBuilderViewSet` with actions:
|
||||
- CRUD operations for `SiteBlueprint`
|
||||
- `generate_structure/` (POST) - Triggers AI structure generation
|
||||
- ✅ `PageBlueprintViewSet` with actions:
|
||||
- CRUD operations for `PageBlueprint`
|
||||
- `generate_content/` (POST) - Generates page content via Writer
|
||||
- `regenerate/` (POST) - Regenerates page content
|
||||
- ✅ `SiteAssetView` (APIView) for file management:
|
||||
- `GET` - List files
|
||||
- `POST` - Upload file
|
||||
- `DELETE` - Delete file
|
||||
- ✅ Serializers:
|
||||
- `SiteBlueprintSerializer`
|
||||
- `PageBlueprintSerializer`
|
||||
- `SiteBlueprintDetailSerializer`
|
||||
- `PageBlueprintDetailSerializer`
|
||||
- `FileUploadSerializer`
|
||||
- ✅ URLs registered at `/api/v1/site-builder/`
|
||||
|
||||
**Migrations**:
|
||||
- ✅ `0001_initial.py` - Creates `SiteBlueprint` and `PageBlueprint` tables
|
||||
- ✅ Applied to database
|
||||
|
||||
**Django App Configuration**:
|
||||
- ✅ `business/site_building/apps.py` - `SiteBuildingConfig`
|
||||
- ✅ `modules/site_builder/apps.py` - `SiteBuilderConfig`
|
||||
- ✅ Added to `INSTALLED_APPS` in `settings.py`
|
||||
|
||||
#### Frontend Implementation
|
||||
|
||||
**Site Builder Container** (`site-builder/`):
|
||||
- ✅ Created standalone Vite + React + TypeScript application
|
||||
- ✅ Docker container configured (`Dockerfile.dev`)
|
||||
- ✅ Docker Compose service (`igny8_site_builder`) on port 8025:5175
|
||||
- ✅ Routed to `builder.igny8.com` via Caddy reverse proxy
|
||||
- ✅ Vite config with `@shared` alias for shared components
|
||||
- ✅ Node.js 22 (upgraded from 18 for Vite 7 compatibility)
|
||||
|
||||
**Wizard Pages** (`site-builder/src/pages/wizard/`):
|
||||
- ✅ `WizardPage.tsx` - Main wizard orchestrator with step navigation
|
||||
- ✅ `BusinessDetailsStep.tsx` - Step 1: Business type, industry, audience
|
||||
- ✅ `BriefStep.tsx` - Step 2: Business brief textarea
|
||||
- ✅ `ObjectivesStep.tsx` - Step 3: Multiple objectives with add/remove
|
||||
- ✅ `StyleStep.tsx` - Step 4: Style preferences (palette, typography, personality)
|
||||
|
||||
**Preview & Dashboard**:
|
||||
- ✅ `PreviewCanvas.tsx` - Live preview of generated site structure
|
||||
- ✅ `SiteDashboard.tsx` - Lists all site blueprints
|
||||
|
||||
**State Management** (`site-builder/src/state/`):
|
||||
- ✅ `builderStore.ts` (Zustand) - Wizard state:
|
||||
- `currentStep`, `wizardData`, `activeBlueprint`
|
||||
- `isSubmitting`, `error`
|
||||
- Actions: `nextStep`, `previousStep`, `submitWizard`, etc.
|
||||
- ✅ `siteDefinitionStore.ts` (Zustand) - Site preview state:
|
||||
- `siteStructure`, `pages`, `activePageSlug`
|
||||
- Actions: `setSiteStructure`, `setPages`, `refreshSiteDefinition`
|
||||
|
||||
**API Client** (`site-builder/src/api/`):
|
||||
- ✅ `builder.api.ts` - API functions:
|
||||
- `createBlueprint()`, `getBlueprint()`, `generateStructure()`
|
||||
- `getPages()`, `generatePageContent()`
|
||||
- `uploadFile()`, `deleteFile()`, `listFiles()`
|
||||
|
||||
**Type Definitions** (`site-builder/src/types/`):
|
||||
- ✅ `siteBuilder.ts` - TypeScript interfaces:
|
||||
- `SiteBlueprint`, `PageBlueprint`, `PageBlock`
|
||||
- `SiteConfig`, `SiteStructure`, `SiteWizardData`
|
||||
|
||||
**Shared Component Library** (`frontend/src/components/shared/`):
|
||||
- ✅ **Blocks**:
|
||||
- `HeroBlock.tsx` - Hero section component
|
||||
- `FeatureGridBlock.tsx` - Feature grid layout
|
||||
- `StatsPanel.tsx` - Statistics display
|
||||
- `blocks.css` - Shared block styles
|
||||
- ✅ **Layouts**:
|
||||
- `DefaultLayout.tsx` - Standard site layout
|
||||
- `MinimalLayout.tsx` - Minimal layout variant
|
||||
- `layouts.css` - Shared layout styles
|
||||
- ✅ **Templates**:
|
||||
- `MarketingTemplate.tsx` - Marketing site template
|
||||
- `LandingTemplate.tsx` - Landing page template
|
||||
- ✅ **Barrel Exports**:
|
||||
- `blocks/index.ts`, `layouts/index.ts`, `templates/index.ts`
|
||||
- `shared/index.ts` - Main export file
|
||||
- ✅ **Documentation**:
|
||||
- `shared/README.md` - Usage guide for shared components
|
||||
|
||||
**Routing & Navigation**:
|
||||
- ✅ React Router configured with routes:
|
||||
- `/` - Wizard page
|
||||
- `/preview` - Preview canvas
|
||||
- `/dashboard` - Site dashboard
|
||||
- ✅ Sidebar navigation with icons (Wand2, LayoutTemplate, PanelsTopLeft)
|
||||
|
||||
**Styling**:
|
||||
- ✅ TailwindCSS configured
|
||||
- ✅ Global styles (`index.css`)
|
||||
- ✅ App-specific styles (`App.css`)
|
||||
- ✅ Component-specific CSS files
|
||||
|
||||
#### Infrastructure
|
||||
|
||||
**Docker Configuration**:
|
||||
- ✅ `docker-compose.app.yml` - Added `igny8_site_builder` service
|
||||
- ✅ Container runs on port `8025:5175`
|
||||
- ✅ Volume mount: `/data/app/igny8/site-builder:/app:rw`
|
||||
- ✅ Environment: `VITE_API_URL: "https://api.igny8.com/api"`
|
||||
- ✅ Depends on `igny8_backend` health check
|
||||
|
||||
**Caddy Routing**:
|
||||
- ✅ Added `builder.igny8.com` server block to Caddyfile
|
||||
- ✅ WebSocket support for Vite HMR
|
||||
- ✅ Reverse proxy to `igny8_site_builder:5175`
|
||||
- ✅ HTTPS enabled via automatic certificates
|
||||
|
||||
**Package Dependencies** (`site-builder/package.json`):
|
||||
- ✅ React 19.2.0, React DOM 19.2.0
|
||||
- ✅ React Router DOM 7.9.6
|
||||
- ✅ Zustand 5.0.8 (state management)
|
||||
- ✅ Axios 1.13.2 (API client)
|
||||
- ✅ React Hook Form 7.66.0
|
||||
- ✅ Lucide React 0.554.0 (icons)
|
||||
- ✅ Vitest 2.1.5, React Testing Library (testing)
|
||||
- ✅ Vite 7.2.2, TypeScript 5.9.3
|
||||
|
||||
#### Testing
|
||||
|
||||
**Backend Tests**:
|
||||
- ✅ `business/site_building/tests/base.py` - `SiteBuilderTestBase` with fixtures
|
||||
- ✅ `business/site_building/tests/test_services.py`:
|
||||
- `StructureGenerationServiceTests` - Tests structure generation flow
|
||||
- `PageGenerationServiceTests` - Tests page content generation
|
||||
- ✅ `ai/tests/test_generate_site_structure_function.py`:
|
||||
- Tests JSON parsing from AI response
|
||||
- Tests `PageBlueprint` sync (create/update/delete)
|
||||
- Tests prompt building with existing pages
|
||||
|
||||
**Frontend Tests**:
|
||||
- ✅ `setupTests.ts` - Vitest configuration with jsdom
|
||||
- ✅ `state/__tests__/builderStore.test.ts` - Wizard store tests
|
||||
- ✅ `state/__tests__/siteDefinitionStore.test.ts` - Site definition store tests
|
||||
- ✅ `pages/wizard/__tests__/WizardPage.test.tsx` - Wizard component tests
|
||||
- ✅ `pages/preview/__tests__/PreviewCanvas.test.tsx` - Preview component tests
|
||||
|
||||
#### Bug Fixes & Issues Resolved
|
||||
|
||||
1. **500 Error on `/v1/writer/tasks`**:
|
||||
- **Issue**: `TasksSerializer` crashing when `Content` record doesn't exist
|
||||
- **Fix**: Updated `_get_content_record()` to catch `ObjectDoesNotExist` exception
|
||||
- **File**: `modules/writer/serializers.py`
|
||||
|
||||
2. **Database Schema Mismatch**:
|
||||
- **Issue**: `Content` model had Phase 4 fields not in database
|
||||
- **Fix**: Created and applied migration `0009_add_content_site_source_fields.py`
|
||||
- **File**: `modules/writer/migrations/0009_add_content_site_source_fields.py`
|
||||
|
||||
3. **Node.js Version Incompatibility**:
|
||||
- **Issue**: Vite 7 requires Node.js 20.19+ or 22.12+
|
||||
- **Fix**: Updated `Dockerfile.dev` from `node:18-alpine` to `node:22-alpine`
|
||||
- **File**: `site-builder/Dockerfile.dev`
|
||||
|
||||
4. **Vite Host Blocking**:
|
||||
- **Issue**: Vite dev server blocking `builder.igny8.com` requests
|
||||
- **Fix**: Added `builder.igny8.com` to `server.allowedHosts` in `vite.config.ts`
|
||||
- **File**: `site-builder/vite.config.ts`
|
||||
|
||||
5. **Vite Alias Resolution**:
|
||||
- **Issue**: `@shared` alias not resolving for shared components
|
||||
- **Fix**: Added dynamic path resolution and `fs.allow` configuration
|
||||
- **File**: `site-builder/vite.config.ts`
|
||||
|
||||
6. **Migration Dependency Error**:
|
||||
- **Issue**: `ValueError: Related model 'igny8_core_auth.account' cannot be resolved`
|
||||
- **Fix**: Updated migration dependency to `0014_remove_plan_operation_limits_phase0`
|
||||
- **File**: `business/site_building/migrations/0001_initial.py`
|
||||
|
||||
7. **Frontend Test Failures**:
|
||||
- **Issue**: Multiple elements matching text query
|
||||
- **Fix**: Changed `getByText` to `getAllByText` in `PreviewCanvas.test.tsx`
|
||||
- **Issue**: Incomplete mock state in `WizardPage.test.tsx`
|
||||
- **Fix**: Added complete `style` object with default values
|
||||
|
||||
### ✅ Phase 4: Linker & Optimizer - Backend Complete
|
||||
|
||||
#### Backend Implementation
|
||||
|
||||
**Content Model Extensions** (`business/content/models.py`):
|
||||
- ✅ Added `source` field (igny8, wordpress, shopify, custom)
|
||||
- ✅ Added `sync_status` field (native, imported, synced)
|
||||
- ✅ Added `external_id`, `external_url`, `sync_metadata` fields
|
||||
- ✅ Added `internal_links` JSON field
|
||||
- ✅ Added `linker_version` integer field
|
||||
- ✅ Added `optimizer_version` integer field
|
||||
- ✅ Added `optimization_scores` JSON field
|
||||
- ✅ Migration created and applied
|
||||
|
||||
**Linking Services** (`business/linking/services/`):
|
||||
- ✅ `LinkerService` - Main service for internal linking
|
||||
- ✅ `CandidateEngine` - Finds link candidates based on content relevance
|
||||
- ✅ `InjectionEngine` - Injects links into HTML content
|
||||
|
||||
**Optimization Services** (`business/optimization/`):
|
||||
- ✅ `OptimizationTask` model - Tracks optimization operations
|
||||
- ✅ `OptimizerService` - Main service with multiple entry points:
|
||||
- `optimize_from_writer()`
|
||||
- `optimize_from_wordpress_sync()`
|
||||
- `optimize_from_external_sync()`
|
||||
- `optimize_manual()`
|
||||
- ✅ `ContentAnalyzer` - Analyzes content for SEO, readability, engagement
|
||||
|
||||
**Content Pipeline Service** (`business/content/services/`):
|
||||
- ✅ `ContentPipelineService` - Orchestrates Writer → Linker → Optimizer pipeline
|
||||
- ✅ `process_writer_content()` - Full pipeline for Writer content
|
||||
- ✅ `process_synced_content()` - Optimization-only for synced content
|
||||
|
||||
**Note**: Phase 4 frontend UI (Linker Dashboard, Optimizer Dashboard) is **not yet implemented**.
|
||||
|
||||
### 📋 Files Created/Modified
|
||||
|
||||
#### Backend Files Created
|
||||
|
||||
**Models & Migrations**:
|
||||
- `backend/igny8_core/business/site_building/models.py`
|
||||
- `backend/igny8_core/business/site_building/migrations/0001_initial.py`
|
||||
- `backend/igny8_core/business/site_building/apps.py`
|
||||
- `backend/igny8_core/modules/writer/migrations/0009_add_content_site_source_fields.py`
|
||||
|
||||
**Services**:
|
||||
- `backend/igny8_core/business/site_building/services/file_management_service.py`
|
||||
- `backend/igny8_core/business/site_building/services/structure_generation_service.py`
|
||||
- `backend/igny8_core/business/site_building/services/page_generation_service.py`
|
||||
- `backend/igny8_core/business/linking/services/linker_service.py`
|
||||
- `backend/igny8_core/business/linking/services/candidate_engine.py`
|
||||
- `backend/igny8_core/business/linking/services/injection_engine.py`
|
||||
- `backend/igny8_core/business/optimization/models.py`
|
||||
- `backend/igny8_core/business/optimization/services/optimizer_service.py`
|
||||
- `backend/igny8_core/business/content/services/content_pipeline_service.py`
|
||||
|
||||
**AI Functions**:
|
||||
- `backend/igny8_core/ai/functions/generate_site_structure.py`
|
||||
|
||||
**API Layer**:
|
||||
- `backend/igny8_core/modules/site_builder/__init__.py`
|
||||
- `backend/igny8_core/modules/site_builder/apps.py`
|
||||
- `backend/igny8_core/modules/site_builder/serializers.py`
|
||||
- `backend/igny8_core/modules/site_builder/views.py`
|
||||
- `backend/igny8_core/modules/site_builder/urls.py`
|
||||
|
||||
**Tests**:
|
||||
- `backend/igny8_core/business/site_building/tests/__init__.py`
|
||||
- `backend/igny8_core/business/site_building/tests/base.py`
|
||||
- `backend/igny8_core/business/site_building/tests/test_services.py`
|
||||
- `backend/igny8_core/ai/tests/test_generate_site_structure_function.py`
|
||||
|
||||
#### Backend Files Modified
|
||||
|
||||
- `backend/igny8_core/settings.py` - Added Site Builder apps to `INSTALLED_APPS`
|
||||
- `backend/igny8_core/urls.py` - Added Site Builder URL routing
|
||||
- `backend/igny8_core/ai/registry.py` - Registered `GenerateSiteStructureFunction`
|
||||
- `backend/igny8_core/ai/prompts.py` - Added `site_structure_generation` prompt
|
||||
- `backend/igny8_core/ai/engine.py` - Integrated site structure generation
|
||||
- `backend/igny8_core/business/content/models.py` - Added Phase 4 fields
|
||||
- `backend/igny8_core/modules/writer/serializers.py` - Fixed `Content.DoesNotExist` handling
|
||||
|
||||
#### Frontend Files Created
|
||||
|
||||
**Site Builder Application**:
|
||||
- `site-builder/package.json`
|
||||
- `site-builder/vite.config.ts`
|
||||
- `site-builder/tsconfig.app.json`
|
||||
- `site-builder/Dockerfile.dev`
|
||||
- `site-builder/src/main.tsx`
|
||||
- `site-builder/src/App.tsx`
|
||||
- `site-builder/src/App.css`
|
||||
- `site-builder/src/index.css`
|
||||
- `site-builder/src/setupTests.ts`
|
||||
|
||||
**Pages**:
|
||||
- `site-builder/src/pages/wizard/WizardPage.tsx`
|
||||
- `site-builder/src/pages/wizard/steps/BusinessDetailsStep.tsx`
|
||||
- `site-builder/src/pages/wizard/steps/BriefStep.tsx`
|
||||
- `site-builder/src/pages/wizard/steps/ObjectivesStep.tsx`
|
||||
- `site-builder/src/pages/wizard/steps/StyleStep.tsx`
|
||||
- `site-builder/src/pages/preview/PreviewCanvas.tsx`
|
||||
- `site-builder/src/pages/dashboard/SiteDashboard.tsx`
|
||||
|
||||
**State Management**:
|
||||
- `site-builder/src/state/builderStore.ts`
|
||||
- `site-builder/src/state/siteDefinitionStore.ts`
|
||||
|
||||
**API & Types**:
|
||||
- `site-builder/src/api/builder.api.ts`
|
||||
- `site-builder/src/types/siteBuilder.ts`
|
||||
|
||||
**Components**:
|
||||
- `site-builder/src/components/common/Card.tsx`
|
||||
- `site-builder/src/components/common/Card.css`
|
||||
|
||||
**Tests**:
|
||||
- `site-builder/src/state/__tests__/builderStore.test.ts`
|
||||
- `site-builder/src/state/__tests__/siteDefinitionStore.test.ts`
|
||||
- `site-builder/src/pages/wizard/__tests__/WizardPage.test.tsx`
|
||||
- `site-builder/src/pages/preview/__tests__/PreviewCanvas.test.tsx`
|
||||
|
||||
**Shared Component Library**:
|
||||
- `frontend/src/components/shared/blocks/HeroBlock.tsx`
|
||||
- `frontend/src/components/shared/blocks/FeatureGridBlock.tsx`
|
||||
- `frontend/src/components/shared/blocks/StatsPanel.tsx`
|
||||
- `frontend/src/components/shared/blocks/blocks.css`
|
||||
- `frontend/src/components/shared/blocks/index.ts`
|
||||
- `frontend/src/components/shared/layouts/DefaultLayout.tsx`
|
||||
- `frontend/src/components/shared/layouts/MinimalLayout.tsx`
|
||||
- `frontend/src/components/shared/layouts/layouts.css`
|
||||
- `frontend/src/components/shared/layouts/index.ts`
|
||||
- `frontend/src/components/shared/templates/MarketingTemplate.tsx`
|
||||
- `frontend/src/components/shared/templates/LandingTemplate.tsx`
|
||||
- `frontend/src/components/shared/templates/index.ts`
|
||||
- `frontend/src/components/shared/index.ts`
|
||||
- `frontend/src/components/shared/README.md`
|
||||
|
||||
#### Infrastructure Files Modified
|
||||
|
||||
- `docker-compose.app.yml` - Added `igny8_site_builder` service
|
||||
- `/var/lib/docker/volumes/portainer_data/_data/caddy/Caddyfile` - Added `builder.igny8.com` routing
|
||||
|
||||
#### Files Removed (Temporary/One-Time Use)
|
||||
|
||||
- `backend/import_plans.py` - One-time data import script
|
||||
- `backend/igny8_core/test_settings.py` - Temporary test configuration
|
||||
- `backend/igny8_core/api/tests/run_tests.py` - Helper test script
|
||||
- `backend/igny8_core/ai/tests/test_run.py` - Temporary AI test script
|
||||
|
||||
### 🔄 Remaining Work
|
||||
|
||||
#### Phase 3 - Minor Enhancements
|
||||
- [ ] Add file browser UI component to Site Builder
|
||||
- [ ] Add deployment functionality (Phase 5 integration)
|
||||
- [ ] Add page editor for manual block editing
|
||||
- [ ] Add template selection in wizard
|
||||
|
||||
#### Phase 4 - Frontend UI
|
||||
- [ ] Create Linker Dashboard (`frontend/src/pages/Linker/Dashboard.tsx`)
|
||||
- [ ] Create Linker Content List (`frontend/src/pages/Linker/ContentList.tsx`)
|
||||
- [ ] Create Optimizer Dashboard (`frontend/src/pages/Optimizer/Dashboard.tsx`)
|
||||
- [ ] Create Optimizer Content Selector (`frontend/src/pages/Optimizer/ContentSelector.tsx`)
|
||||
- [ ] Create shared components:
|
||||
- [ ] `SourceBadge.tsx` - Display content source
|
||||
- [ ] `SyncStatusBadge.tsx` - Display sync status
|
||||
- [ ] `ContentFilter.tsx` - Filter by source/sync_status
|
||||
- [ ] Update Writer content list to show source badges
|
||||
- [ ] Add "Send to Optimizer" button in Writer
|
||||
|
||||
#### Phase 4 - AI Function
|
||||
- [ ] Create `OptimizeContentFunction` (`ai/functions/optimize_content.py`)
|
||||
- [ ] Add optimization prompts to `ai/prompts.py`
|
||||
- [ ] Register function in `ai/registry.py`
|
||||
- [ ] Integrate into `ai/engine.py`
|
||||
|
||||
#### Phase 4 - API Layer
|
||||
- [ ] Create `modules/linker/` module with ViewSet
|
||||
- [ ] Create `modules/optimizer/` module with ViewSet
|
||||
- [ ] Register URLs for Linker and Optimizer APIs
|
||||
|
||||
### 📊 Implementation Statistics
|
||||
|
||||
- **Backend Files Created**: 25+
|
||||
- **Frontend Files Created**: 30+
|
||||
- **Backend Tests**: 3 test files, 10+ test cases
|
||||
- **Frontend Tests**: 4 test files, 15+ test cases
|
||||
- **Lines of Code**: ~5,000+ (backend + frontend)
|
||||
- **Docker Containers**: 1 new container (`igny8_site_builder`)
|
||||
- **API Endpoints**: 10+ new endpoints
|
||||
- **Database Tables**: 2 new tables (`SiteBlueprint`, `PageBlueprint`)
|
||||
- **Migrations**: 2 migrations created and applied
|
||||
|
||||
---
|
||||
|
||||
@@ -232,54 +663,54 @@ frontend/src/components/shared/
|
||||
#### Backend Tasks (Priority Order)
|
||||
|
||||
1. **Create Business Models**
|
||||
- [ ] Create `business/site_building/` folder
|
||||
- [ ] Create `SiteBlueprint` model
|
||||
- [ ] Create `PageBlueprint` model
|
||||
- [ ] Create migrations
|
||||
- [x] Create `business/site_building/` folder
|
||||
- [x] Create `SiteBlueprint` model
|
||||
- [x] Create `PageBlueprint` model
|
||||
- [x] Create migrations
|
||||
|
||||
2. **Create Services**
|
||||
- [ ] Create `FileManagementService`
|
||||
- [ ] Create `StructureGenerationService`
|
||||
- [ ] Create `PageGenerationService`
|
||||
- [ ] Integrate with `CreditService`
|
||||
- [x] Create `FileManagementService`
|
||||
- [x] Create `StructureGenerationService`
|
||||
- [x] Create `PageGenerationService`
|
||||
- [x] Integrate with `CreditService`
|
||||
|
||||
3. **Create AI Function**
|
||||
- [ ] Create `GenerateSiteStructureFunction`
|
||||
- [ ] Add prompts for site structure generation
|
||||
- [ ] Test AI function
|
||||
- [x] Create `GenerateSiteStructureFunction`
|
||||
- [x] Add prompts for site structure generation
|
||||
- [x] Test AI function
|
||||
|
||||
4. **Create API Layer**
|
||||
- [ ] Create `modules/site_builder/` folder
|
||||
- [ ] Create `SiteBuilderViewSet`
|
||||
- [ ] Create `PageBlueprintViewSet`
|
||||
- [ ] Create `FileUploadView`
|
||||
- [ ] Create serializers
|
||||
- [ ] Register URLs
|
||||
- [x] Create `modules/site_builder/` folder
|
||||
- [x] Create `SiteBuilderViewSet`
|
||||
- [x] Create `PageBlueprintViewSet`
|
||||
- [x] Create `FileUploadView`
|
||||
- [x] Create serializers
|
||||
- [x] Register URLs
|
||||
|
||||
#### Frontend Tasks (Priority Order)
|
||||
|
||||
1. **Create Site Builder Container**
|
||||
- [ ] Create `site-builder/` folder structure
|
||||
- [ ] Set up Vite + React + TypeScript
|
||||
- [ ] Configure Docker container
|
||||
- [ ] Set up routing
|
||||
- [x] Create `site-builder/` folder structure
|
||||
- [x] Set up Vite + React + TypeScript
|
||||
- [x] Configure Docker container
|
||||
- [x] Set up routing
|
||||
|
||||
2. **Create Wizard**
|
||||
- [ ] Step 1: Type Selection
|
||||
- [ ] Step 2: Business Brief
|
||||
- [ ] Step 3: Objectives
|
||||
- [ ] Step 4: Style Preferences
|
||||
- [ ] Wizard state management
|
||||
- [x] Step 1: Business Details (Type Selection)
|
||||
- [x] Step 2: Business Brief
|
||||
- [x] Step 3: Objectives
|
||||
- [x] Step 4: Style Preferences
|
||||
- [x] Wizard state management
|
||||
|
||||
3. **Create Preview Canvas**
|
||||
- [ ] Preview renderer
|
||||
- [ ] Block rendering
|
||||
- [ ] Layout rendering
|
||||
- [x] Preview renderer
|
||||
- [x] Block rendering
|
||||
- [x] Layout rendering
|
||||
|
||||
4. **Create Shared Components**
|
||||
- [ ] Block components
|
||||
- [ ] Layout components
|
||||
- [ ] Template components
|
||||
- [x] Block components
|
||||
- [x] Layout components
|
||||
- [x] Template components
|
||||
|
||||
---
|
||||
|
||||
@@ -515,26 +946,26 @@ frontend/src/components/
|
||||
#### Backend Tasks (Priority Order)
|
||||
|
||||
1. **Extend Content Model**
|
||||
- [ ] Add `source` field
|
||||
- [ ] Add `sync_status` field
|
||||
- [ ] Add `external_id`, `external_url`, `sync_metadata`
|
||||
- [ ] Add `internal_links`, `linker_version`
|
||||
- [ ] Add `optimizer_version`, `optimization_scores`
|
||||
- [ ] Create migration
|
||||
- [x] Add `source` field
|
||||
- [x] Add `sync_status` field
|
||||
- [x] Add `external_id`, `external_url`, `sync_metadata`
|
||||
- [x] Add `internal_links`, `linker_version`
|
||||
- [x] Add `optimizer_version`, `optimization_scores`
|
||||
- [x] Create migration
|
||||
|
||||
2. **Create Linking Services**
|
||||
- [ ] Create `business/linking/` folder
|
||||
- [ ] Create `LinkerService`
|
||||
- [ ] Create `CandidateEngine`
|
||||
- [ ] Create `InjectionEngine`
|
||||
- [ ] Integrate with `CreditService`
|
||||
- [x] Create `business/linking/` folder
|
||||
- [x] Create `LinkerService`
|
||||
- [x] Create `CandidateEngine`
|
||||
- [x] Create `InjectionEngine`
|
||||
- [x] Integrate with `CreditService`
|
||||
|
||||
3. **Create Optimization Services**
|
||||
- [ ] Create `business/optimization/` folder
|
||||
- [ ] Create `OptimizationTask` model
|
||||
- [ ] Create `OptimizerService` (with multiple entry points)
|
||||
- [ ] Create `ContentAnalyzer`
|
||||
- [ ] Integrate with `CreditService`
|
||||
- [x] Create `business/optimization/` folder
|
||||
- [x] Create `OptimizationTask` model
|
||||
- [x] Create `OptimizerService` (with multiple entry points)
|
||||
- [x] Create `ContentAnalyzer`
|
||||
- [x] Integrate with `CreditService`
|
||||
|
||||
4. **Create AI Function**
|
||||
- [ ] Create `OptimizeContentFunction`
|
||||
@@ -542,8 +973,8 @@ frontend/src/components/
|
||||
- [ ] Test AI function
|
||||
|
||||
5. **Create Pipeline Service**
|
||||
- [ ] Create `ContentPipelineService`
|
||||
- [ ] Integrate Linker and Optimizer
|
||||
- [x] Create `ContentPipelineService`
|
||||
- [x] Integrate Linker and Optimizer
|
||||
|
||||
6. **Create API Layer**
|
||||
- [ ] Create `modules/linker/` folder
|
||||
|
||||
@@ -0,0 +1,682 @@
|
||||
# PHASE 9: AI FRAMEWORK & SITE BUILDER INTEGRATION
|
||||
**Detailed Implementation Plan**
|
||||
|
||||
**Goal**: Complete AI framework integration for Site Builder, add prompt management UI, and implement blueprint-to-writer task queuing.
|
||||
|
||||
**Timeline**: 2-3 weeks
|
||||
**Priority**: MEDIUM
|
||||
**Dependencies**: Phase 3
|
||||
|
||||
---
|
||||
|
||||
## TABLE OF CONTENTS
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [AI Framework Integration](#ai-framework-integration)
|
||||
3. [Prompt Management UI](#prompt-management-ui)
|
||||
4. [Response Handling & Structure Generation](#response-handling--structure-generation)
|
||||
5. [Blueprint-to-Writer Task Queuing](#blueprint-to-writer-task-queuing)
|
||||
6. [Testing & Validation](#testing--validation)
|
||||
7. [Implementation Checklist](#implementation-checklist)
|
||||
|
||||
---
|
||||
|
||||
## OVERVIEW
|
||||
|
||||
### Objectives
|
||||
- ✅ Add `site_structure_generation` prompt to Thinker `/prompts` UI
|
||||
- ✅ Document AI framework integration for Site Builder
|
||||
- ✅ Implement response handling for structure generation
|
||||
- ✅ Enable blueprint pages to queue as Writer tasks (similar to ideas)
|
||||
- ✅ Add bulk page content generation from blueprint
|
||||
|
||||
### Key Principles
|
||||
- **Unified Prompt Management**: Site Builder prompts managed in same UI as other prompts
|
||||
- **Reuse Existing Patterns**: Follow same queuing pattern as ideas → writer tasks
|
||||
- **AI Framework Compliance**: Use existing `BaseAIFunction` and `AIEngine` patterns
|
||||
- **Response Validation**: Robust parsing and validation of AI responses
|
||||
|
||||
---
|
||||
|
||||
## AI FRAMEWORK INTEGRATION
|
||||
|
||||
### 9.1 AI Framework Integration
|
||||
|
||||
**Purpose**: Document and ensure proper integration with existing AI framework.
|
||||
|
||||
#### Current AI Framework Architecture
|
||||
|
||||
The Site Builder uses the existing AI framework:
|
||||
|
||||
```
|
||||
AIEngine
|
||||
├─ BaseAIFunction (interface)
|
||||
│ ├─ validate()
|
||||
│ ├─ prepare()
|
||||
│ ├─ build_prompt()
|
||||
│ ├─ parse_response()
|
||||
│ └─ save_output()
|
||||
│
|
||||
└─ GenerateSiteStructureFunction
|
||||
├─ Uses PromptRegistry for prompts
|
||||
├─ Integrates with CreditService
|
||||
└─ Saves to SiteBlueprint + PageBlueprint models
|
||||
```
|
||||
|
||||
#### AI Function Lifecycle
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **AI Function Registration** | `ai/registry.py` | Existing | Register `generate_site_structure` |
|
||||
| **Prompt Integration** | `ai/prompts.py` | Existing | Add `site_structure_generation` to DEFAULT_PROMPTS |
|
||||
| **Credit Cost Configuration** | `business/billing/services/credit_service.py` | Phase 0 | Add `site_structure_generation` operation type |
|
||||
|
||||
**AI Function Flow**:
|
||||
```python
|
||||
# infrastructure/ai/functions/generate_site_structure.py
|
||||
class GenerateSiteStructureFunction(BaseAIFunction):
|
||||
def get_name(self) -> str:
|
||||
return 'generate_site_structure'
|
||||
|
||||
def validate(self, payload, account=None):
|
||||
# Validates blueprint ID exists
|
||||
# Returns {'valid': True/False, 'error': '...'}
|
||||
|
||||
def prepare(self, payload, account=None):
|
||||
# Loads SiteBlueprint
|
||||
# Extracts business_brief, objectives, style
|
||||
# Returns context dict
|
||||
|
||||
def build_prompt(self, data, account=None):
|
||||
# Uses PromptRegistry.get_prompt('site_structure_generation')
|
||||
# Injects: BUSINESS_BRIEF, OBJECTIVES, STYLE, SITE_INFO
|
||||
# Returns formatted prompt string
|
||||
|
||||
def parse_response(self, response, step_tracker=None):
|
||||
# Parses JSON from AI response
|
||||
# Handles noisy responses (extracts JSON from text)
|
||||
# Validates structure is dict
|
||||
# Returns parsed structure dict
|
||||
|
||||
def save_output(self, parsed, original_data, account=None):
|
||||
# Saves structure_json to SiteBlueprint
|
||||
# Syncs PageBlueprint records (create/update/delete)
|
||||
# Updates blueprint status to 'ready'
|
||||
# Returns summary dict
|
||||
```
|
||||
|
||||
#### Prompt Variable Injection
|
||||
|
||||
The prompt system supports variable injection:
|
||||
|
||||
**Available Variables**:
|
||||
- `[IGNY8_BUSINESS_BRIEF]` - Business description
|
||||
- `[IGNY8_OBJECTIVES]` - List of site objectives
|
||||
- `[IGNY8_STYLE]` - Style preferences JSON
|
||||
- `[IGNY8_SITE_INFO]` - Site metadata JSON
|
||||
|
||||
**Prompt Template** (from `ai/prompts.py`):
|
||||
```python
|
||||
'site_structure_generation': """You are a senior UX architect...
|
||||
BUSINESS BRIEF:
|
||||
[IGNY8_BUSINESS_BRIEF]
|
||||
|
||||
PRIMARY OBJECTIVES:
|
||||
[IGNY8_OBJECTIVES]
|
||||
|
||||
STYLE & BRAND NOTES:
|
||||
[IGNY8_STYLE]
|
||||
|
||||
SITE INFO:
|
||||
[IGNY8_SITE_INFO]
|
||||
...
|
||||
"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## PROMPT MANAGEMENT UI
|
||||
|
||||
### 9.2 Prompt Management UI
|
||||
|
||||
**Purpose**: Add Site Builder prompt to Thinker `/prompts` interface.
|
||||
|
||||
#### Add Prompt Type to Backend
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Add prompt type choice** | `modules/system/models.py` | Existing | Add `'site_structure_generation'` to PROMPT_TYPE_CHOICES |
|
||||
|
||||
**AIPrompt Model Update**:
|
||||
```python
|
||||
# modules/system/models.py
|
||||
class AIPrompt(AccountBaseModel):
|
||||
PROMPT_TYPE_CHOICES = [
|
||||
('clustering', 'Clustering'),
|
||||
('ideas', 'Ideas Generation'),
|
||||
('content_generation', 'Content Generation'),
|
||||
('image_prompt_extraction', 'Image Prompt Extraction'),
|
||||
('image_prompt_template', 'Image Prompt Template'),
|
||||
('negative_prompt', 'Negative Prompt'),
|
||||
('site_structure_generation', 'Site Structure Generation'), # NEW
|
||||
]
|
||||
```
|
||||
|
||||
#### Add Prompt Type to Frontend
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Add prompt type config** | `frontend/src/pages/Thinker/Prompts.tsx` | Existing | Add site_structure_generation to PROMPT_TYPES array |
|
||||
|
||||
**Frontend Prompt Type Config**:
|
||||
```typescript
|
||||
// frontend/src/pages/Thinker/Prompts.tsx
|
||||
const PROMPT_TYPES = [
|
||||
// ... existing prompts
|
||||
{
|
||||
key: 'site_structure_generation',
|
||||
label: 'Site Structure Generation',
|
||||
description: 'Generate site structure from business brief. Use [IGNY8_BUSINESS_BRIEF], [IGNY8_OBJECTIVES], [IGNY8_STYLE], and [IGNY8_SITE_INFO] to inject data.',
|
||||
icon: '🏗️',
|
||||
color: 'teal',
|
||||
},
|
||||
];
|
||||
```
|
||||
|
||||
#### Prompt UI Section
|
||||
|
||||
The prompt will appear in a new "Site Builder" section in the Thinker Prompts page:
|
||||
|
||||
```typescript
|
||||
{/* Site Builder Prompts Section */}
|
||||
<div className="mb-8">
|
||||
<div className="mb-4">
|
||||
<h2 className="text-xl font-semibold text-gray-800 dark:text-white">
|
||||
Site Builder
|
||||
</h2>
|
||||
<p className="text-sm text-gray-600 dark:text-gray-400">
|
||||
Configure prompts for AI-powered site structure generation
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Site Structure Generation Prompt */}
|
||||
<PromptEditor
|
||||
promptType="site_structure_generation"
|
||||
label="Site Structure Generation Prompt"
|
||||
description="Generate complete site structure (pages, blocks, navigation) from business brief"
|
||||
variables={[
|
||||
'[IGNY8_BUSINESS_BRIEF]',
|
||||
'[IGNY8_OBJECTIVES]',
|
||||
'[IGNY8_STYLE]',
|
||||
'[IGNY8_SITE_INFO]'
|
||||
]}
|
||||
/>
|
||||
</div>
|
||||
```
|
||||
|
||||
#### Migration for Prompt Type
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Create migration** | `modules/system/migrations/XXXX_add_site_structure_prompt_type.py` | Existing | Add new choice to PROMPT_TYPE_CHOICES |
|
||||
|
||||
---
|
||||
|
||||
## RESPONSE HANDLING & STRUCTURE GENERATION
|
||||
|
||||
### 9.3 Response Handling & Structure Generation
|
||||
|
||||
**Purpose**: Document and enhance AI response parsing and structure generation.
|
||||
|
||||
#### Response Format
|
||||
|
||||
The AI returns a JSON structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"site": {
|
||||
"name": "Acme Robotics",
|
||||
"primary_navigation": ["home", "services", "about", "contact"],
|
||||
"secondary_navigation": ["blog", "faq"],
|
||||
"hero_message": "Transform your warehouse operations",
|
||||
"tone": "Confident, professional, innovative"
|
||||
},
|
||||
"pages": [
|
||||
{
|
||||
"slug": "home",
|
||||
"title": "Home",
|
||||
"type": "home",
|
||||
"status": "draft",
|
||||
"objective": "Convert visitors to demo requests",
|
||||
"primary_cta": "Book a demo",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "hero",
|
||||
"heading": "Warehouse Automation That Scales",
|
||||
"subheading": "AI-powered robotics for modern fulfillment",
|
||||
"layout": "full-width",
|
||||
"content": ["Hero section with CTA"]
|
||||
},
|
||||
{
|
||||
"type": "features",
|
||||
"heading": "Why Choose Us",
|
||||
"subheading": "Industry-leading solutions",
|
||||
"layout": "three-column",
|
||||
"content": ["Feature 1", "Feature 2", "Feature 3"]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### Response Parsing Flow
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Parse Response** | `ai/functions/generate_site_structure.py` | Existing | `parse_response()` method |
|
||||
| **Extract JSON** | `ai/functions/generate_site_structure.py` | Existing | `_extract_json_object()` helper |
|
||||
| **Validate Structure** | `ai/functions/generate_site_structure.py` | Existing | `_ensure_dict()` validation |
|
||||
|
||||
**Parsing Logic**:
|
||||
```python
|
||||
def parse_response(self, response: str, step_tracker=None) -> Dict[str, Any]:
|
||||
"""
|
||||
Parse AI response into structure dict.
|
||||
|
||||
Handles:
|
||||
1. Direct JSON response
|
||||
2. JSON wrapped in text/markdown
|
||||
3. Noisy responses with commentary
|
||||
"""
|
||||
response = response.strip()
|
||||
|
||||
# Try direct JSON parse
|
||||
try:
|
||||
return self._ensure_dict(json.loads(response))
|
||||
except json.JSONDecodeError:
|
||||
# Extract JSON from text
|
||||
cleaned = self._extract_json_object(response)
|
||||
if cleaned:
|
||||
return self._ensure_dict(json.loads(cleaned))
|
||||
raise ValueError("Unable to parse AI response into JSON")
|
||||
```
|
||||
|
||||
#### Structure Persistence
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Save Structure** | `ai/functions/generate_site_structure.py` | Existing | `save_output()` method |
|
||||
| **Sync Page Blueprints** | `ai/functions/generate_site_structure.py` | Existing | `_sync_page_blueprints()` method |
|
||||
|
||||
**Persistence Flow**:
|
||||
```python
|
||||
def save_output(self, parsed, original_data, account=None):
|
||||
"""
|
||||
Save structure and sync page blueprints.
|
||||
|
||||
1. Save structure_json to SiteBlueprint
|
||||
2. Create/update PageBlueprint records
|
||||
3. Delete pages not in new structure
|
||||
4. Update blueprint status to 'ready'
|
||||
"""
|
||||
blueprint = original_data['blueprint']
|
||||
structure = self._ensure_dict(parsed)
|
||||
pages = structure.get('pages', [])
|
||||
|
||||
# Save structure
|
||||
blueprint.structure_json = structure
|
||||
blueprint.status = 'ready'
|
||||
blueprint.save()
|
||||
|
||||
# Sync pages (create/update/delete)
|
||||
created, updated, deleted = self._sync_page_blueprints(blueprint, pages)
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
'pages_created': created,
|
||||
'pages_updated': updated,
|
||||
'pages_deleted': deleted,
|
||||
}
|
||||
```
|
||||
|
||||
#### Error Handling
|
||||
|
||||
**Error Scenarios**:
|
||||
1. **Invalid JSON**: Extract JSON from text, fallback to error
|
||||
2. **Missing Required Fields**: Validate structure has `site` and `pages`
|
||||
3. **Invalid Page Types**: Map to allowed types, default to 'custom'
|
||||
4. **Duplicate Slugs**: Use `update_or_create` to handle conflicts
|
||||
|
||||
**Error Response Format**:
|
||||
```python
|
||||
{
|
||||
'success': False,
|
||||
'error': 'Error message',
|
||||
'error_type': 'ParseError' | 'ValidationError' | 'SaveError'
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## BLUEPRINT-TO-WRITER TASK QUEUING
|
||||
|
||||
### 9.4 Blueprint-to-Writer Task Queuing
|
||||
|
||||
**Purpose**: Enable blueprint pages to be queued as Writer tasks, similar to how ideas are queued.
|
||||
|
||||
#### Current Pattern (Ideas → Writer)
|
||||
|
||||
The existing pattern for ideas:
|
||||
|
||||
```
|
||||
ContentIdeas → Writer Tasks → Content Generation
|
||||
```
|
||||
|
||||
**Flow**:
|
||||
1. User selects ideas in Planner
|
||||
2. Click "Generate Content"
|
||||
3. Creates `Tasks` records for each idea
|
||||
4. Queues `generate_content` AI task
|
||||
5. Generates HTML content
|
||||
|
||||
#### New Pattern (Blueprint Pages → Writer)
|
||||
|
||||
The new pattern for blueprint pages:
|
||||
|
||||
```
|
||||
PageBlueprint → Writer Tasks → Content Generation
|
||||
```
|
||||
|
||||
**Flow**:
|
||||
1. User has generated SiteBlueprint with PageBlueprints
|
||||
2. User selects pages (or all pages)
|
||||
3. Click "Generate Content for Pages"
|
||||
4. Creates `Tasks` records for each page
|
||||
5. Queues `generate_content` AI task
|
||||
6. Generates HTML content
|
||||
|
||||
#### Bulk Page Content Generation
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Bulk Generate Action** | `modules/site_builder/views.py` | PageGenerationService | Add `bulk_generate_content` action |
|
||||
| **Task Creation Service** | `business/site_building/services/page_generation_service.py` | Existing | Add `create_tasks_for_pages()` method |
|
||||
| **Bulk Queue Service** | `business/site_building/services/page_generation_service.py` | ContentGenerationService | Add `bulk_generate_pages()` method |
|
||||
|
||||
**Bulk Generation API**:
|
||||
```python
|
||||
# modules/site_builder/views.py
|
||||
class SiteBlueprintViewSet(SiteSectorModelViewSet):
|
||||
@action(detail=True, methods=['post'])
|
||||
def generate_all_pages(self, request, pk=None):
|
||||
"""
|
||||
Generate content for all pages in blueprint.
|
||||
|
||||
Request body:
|
||||
{
|
||||
"page_ids": [1, 2, 3], # Optional: specific pages, or all if omitted
|
||||
"force": false # Optional: force regenerate existing content
|
||||
}
|
||||
"""
|
||||
blueprint = self.get_object()
|
||||
page_ids = request.data.get('page_ids')
|
||||
force = request.data.get('force', False)
|
||||
|
||||
service = PageGenerationService()
|
||||
result = service.bulk_generate_pages(
|
||||
blueprint,
|
||||
page_ids=page_ids,
|
||||
force_regenerate=force
|
||||
)
|
||||
return success_response(result, request=request)
|
||||
```
|
||||
|
||||
**Bulk Generation Service**:
|
||||
```python
|
||||
# business/site_building/services/page_generation_service.py
|
||||
class PageGenerationService:
|
||||
def bulk_generate_pages(
|
||||
self,
|
||||
site_blueprint: SiteBlueprint,
|
||||
page_ids: Optional[List[int]] = None,
|
||||
force_regenerate: bool = False
|
||||
) -> dict:
|
||||
"""
|
||||
Generate content for multiple pages in a blueprint.
|
||||
|
||||
Similar to how ideas are queued to writer:
|
||||
1. Get pages (filtered by page_ids if provided)
|
||||
2. Create/update Writer Tasks for each page
|
||||
3. Queue content generation for all tasks
|
||||
4. Return task IDs for progress tracking
|
||||
"""
|
||||
# Get pages
|
||||
pages = site_blueprint.pages.all()
|
||||
if page_ids:
|
||||
pages = pages.filter(id__in=page_ids)
|
||||
|
||||
# Create tasks for all pages
|
||||
task_ids = []
|
||||
for page in pages:
|
||||
task = self._ensure_task(page, force_regenerate=force_regenerate)
|
||||
task_ids.append(task.id)
|
||||
page.status = 'generating'
|
||||
page.save(update_fields=['status', 'updated_at'])
|
||||
|
||||
# Queue content generation (same as ideas → writer)
|
||||
account = site_blueprint.account
|
||||
result = self.content_service.generate_content(task_ids, account)
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
'pages_queued': len(task_ids),
|
||||
'task_ids': task_ids,
|
||||
'celery_task_id': result.get('task_id'),
|
||||
}
|
||||
|
||||
def create_tasks_for_pages(
|
||||
self,
|
||||
site_blueprint: SiteBlueprint,
|
||||
page_ids: Optional[List[int]] = None,
|
||||
force_regenerate: bool = False
|
||||
) -> List[Tasks]:
|
||||
"""
|
||||
Create Writer Tasks for blueprint pages without generating content.
|
||||
|
||||
Useful for:
|
||||
- Previewing what tasks will be created
|
||||
- Manual task management
|
||||
- Integration with existing Writer UI
|
||||
"""
|
||||
pages = site_blueprint.pages.all()
|
||||
if page_ids:
|
||||
pages = pages.filter(id__in=page_ids)
|
||||
|
||||
tasks = []
|
||||
for page in pages:
|
||||
task = self._ensure_task(page, force_regenerate=force_regenerate)
|
||||
tasks.append(task)
|
||||
|
||||
return tasks
|
||||
```
|
||||
|
||||
#### Frontend Integration
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Bulk Generate UI** | `site-builder/src/pages/dashboard/SiteDashboard.tsx` | API client | Add "Generate All Pages" button |
|
||||
| **Page Selection UI** | `site-builder/src/pages/preview/PreviewCanvas.tsx` | State store | Add checkbox selection for pages |
|
||||
| **Progress Tracking** | `site-builder/src/components/common/ProgressModal.tsx` | Existing | Track bulk generation progress |
|
||||
|
||||
**Frontend API Client**:
|
||||
```typescript
|
||||
// site-builder/src/api/builder.api.ts
|
||||
export const builderApi = {
|
||||
// ... existing methods
|
||||
|
||||
async generateAllPages(
|
||||
blueprintId: number,
|
||||
options?: { pageIds?: number[]; force?: boolean }
|
||||
): Promise<{ task_id: string; pages_queued: number }> {
|
||||
const res = await client.post(
|
||||
`/blueprints/${blueprintId}/generate_all_pages/`,
|
||||
options || {}
|
||||
);
|
||||
return res.data;
|
||||
},
|
||||
|
||||
async createTasksForPages(
|
||||
blueprintId: number,
|
||||
pageIds?: number[]
|
||||
): Promise<{ tasks: Task[] }> {
|
||||
const res = await client.post(
|
||||
`/blueprints/${blueprintId}/create_tasks/`,
|
||||
{ page_ids: pageIds }
|
||||
);
|
||||
return res.data;
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
#### Task Mapping
|
||||
|
||||
**PageBlueprint → Task Mapping**:
|
||||
|
||||
| PageBlueprint Field | Task Field | Notes |
|
||||
|---------------------|-----------|-------|
|
||||
| `title` | `title` | Prefixed with "[Site Builder]" |
|
||||
| `slug` | `keywords` | Used as keyword hint |
|
||||
| `type` | `content_structure` | Mapped (home → landing_page, etc.) |
|
||||
| `blocks_json` | `description` | Extracts headings for context |
|
||||
| `site_blueprint.name` | `description` | Added to description |
|
||||
| `account`, `site`, `sector` | Same | Inherited from blueprint |
|
||||
|
||||
**Content Structure Mapping**:
|
||||
```python
|
||||
PAGE_TYPE_TO_STRUCTURE = {
|
||||
'home': 'landing_page',
|
||||
'about': 'supporting_page',
|
||||
'services': 'pillar_page',
|
||||
'products': 'pillar_page',
|
||||
'blog': 'cluster_hub',
|
||||
'contact': 'supporting_page',
|
||||
'custom': 'landing_page',
|
||||
}
|
||||
```
|
||||
|
||||
#### Integration with Writer UI
|
||||
|
||||
| Task | File | Dependencies | Implementation |
|
||||
|------|------|--------------|----------------|
|
||||
| **Show Site Builder Tasks** | `frontend/src/pages/Writer/Content.tsx` | Existing | Filter tasks with "[Site Builder]" prefix |
|
||||
| **Link to Blueprint** | `frontend/src/pages/Writer/Content.tsx` | API | Add link to source blueprint |
|
||||
| **Bulk Actions** | `frontend/src/pages/Writer/Content.tsx` | Existing | Include Site Builder tasks in bulk actions |
|
||||
|
||||
**Task Identification**:
|
||||
- Tasks created from Site Builder have title prefix: `"[Site Builder] {page_title}"`
|
||||
- Can filter Writer tasks by this prefix
|
||||
- Can link back to source PageBlueprint via task description
|
||||
|
||||
---
|
||||
|
||||
## TESTING & VALIDATION
|
||||
|
||||
### 9.5 Testing
|
||||
|
||||
**Test Cases**:
|
||||
|
||||
1. **Prompt Management**:
|
||||
- ✅ Site structure prompt appears in Thinker UI
|
||||
- ✅ Prompt can be edited and saved
|
||||
- ✅ Prompt reset works correctly
|
||||
- ✅ Custom prompt is used in structure generation
|
||||
|
||||
2. **Response Handling**:
|
||||
- ✅ Valid JSON response is parsed correctly
|
||||
- ✅ JSON wrapped in text is extracted
|
||||
- ✅ Invalid responses show proper errors
|
||||
- ✅ Structure validation works
|
||||
|
||||
3. **Bulk Page Generation**:
|
||||
- ✅ All pages can be queued to writer
|
||||
- ✅ Specific pages can be selected
|
||||
- ✅ Tasks are created with correct mapping
|
||||
- ✅ Content generation is queued correctly
|
||||
- ✅ Progress tracking works
|
||||
|
||||
4. **Task Integration**:
|
||||
- ✅ Site Builder tasks appear in Writer UI
|
||||
- ✅ Tasks link back to source blueprint
|
||||
- ✅ Bulk actions work with Site Builder tasks
|
||||
|
||||
---
|
||||
|
||||
## IMPLEMENTATION CHECKLIST
|
||||
|
||||
### Backend Tasks
|
||||
|
||||
- [ ] Add `site_structure_generation` to `AIPrompt.PROMPT_TYPE_CHOICES`
|
||||
- [ ] Create migration for new prompt type
|
||||
- [ ] Add `bulk_generate_pages()` to `PageGenerationService`
|
||||
- [ ] Add `create_tasks_for_pages()` to `PageGenerationService`
|
||||
- [ ] Add `generate_all_pages` action to `SiteBlueprintViewSet`
|
||||
- [ ] Add `create_tasks` action to `SiteBlueprintViewSet`
|
||||
- [ ] Document response handling in code comments
|
||||
- [ ] Add error handling for bulk operations
|
||||
|
||||
### Frontend Tasks
|
||||
|
||||
- [ ] Add `site_structure_generation` to `PROMPT_TYPES` in `Prompts.tsx`
|
||||
- [ ] Add "Site Builder" section to Prompts page
|
||||
- [ ] Add prompt editor for site structure generation
|
||||
- [ ] Add "Generate All Pages" button to Site Dashboard
|
||||
- [ ] Add page selection UI to Preview Canvas
|
||||
- [ ] Add bulk generation API methods to `builder.api.ts`
|
||||
- [ ] Add progress tracking for bulk generation
|
||||
- [ ] Update Writer UI to show Site Builder tasks
|
||||
- [ ] Add link from Writer tasks to source blueprint
|
||||
|
||||
### Testing Tasks
|
||||
|
||||
- [ ] Test prompt management UI
|
||||
- [ ] Test response parsing with various formats
|
||||
- [ ] Test bulk page generation
|
||||
- [ ] Test task creation and mapping
|
||||
- [ ] Test integration with Writer UI
|
||||
- [ ] Test error handling
|
||||
|
||||
---
|
||||
|
||||
## RISK ASSESSMENT
|
||||
|
||||
| Risk | Level | Mitigation |
|
||||
|------|-------|------------|
|
||||
| **Prompt changes break existing blueprints** | LOW | Version prompts, validate before save |
|
||||
| **Bulk generation overloads system** | MEDIUM | Rate limiting, queue management |
|
||||
| **Task mapping inconsistencies** | LOW | Comprehensive tests, validation |
|
||||
| **Response parsing failures** | MEDIUM | Robust error handling, fallbacks |
|
||||
|
||||
---
|
||||
|
||||
## SUCCESS CRITERIA
|
||||
|
||||
- ✅ Site structure prompt is manageable in Thinker UI
|
||||
- ✅ Response handling is robust and handles edge cases
|
||||
- ✅ Blueprint pages can be queued to writer like ideas
|
||||
- ✅ Bulk generation works for all pages
|
||||
- ✅ Tasks appear correctly in Writer UI
|
||||
- ✅ Integration is seamless and follows existing patterns
|
||||
|
||||
---
|
||||
|
||||
## RELATED PHASES
|
||||
|
||||
- **Phase 3**: Site Builder foundation
|
||||
- **Phase 1**: Service layer (ContentGenerationService)
|
||||
- **Phase 0**: Credit system (for AI operations)
|
||||
|
||||
---
|
||||
|
||||
**END OF PHASE 9 DOCUMENT**
|
||||
|
||||
@@ -18,8 +18,9 @@ This folder contains detailed implementation plans for each phase of the IGNY8 P
|
||||
| **Phase 6** | [PHASE-6-SITE-INTEGRATION-PUBLISHING.md](./PHASE-6-SITE-INTEGRATION-PUBLISHING.md) | 2-3 weeks | MEDIUM | Phase 5 |
|
||||
| **Phase 7** | [PHASE-7-UI-COMPONENTS-MODULE-SETTINGS.md](./PHASE-7-UI-COMPONENTS-MODULE-SETTINGS.md) | 3-4 weeks | MEDIUM | Phase 0, Phase 3, Phase 5 |
|
||||
| **Phase 8** | [PHASE-8-UNIVERSAL-CONTENT-TYPES.md](./PHASE-8-UNIVERSAL-CONTENT-TYPES.md) | 2-3 weeks | LOW | Phase 4 |
|
||||
| **Phase 9** | [PHASE-9-AI-FRAMEWORK-SITE-BUILDER-INTEGRATION.md](./PHASE-9-AI-FRAMEWORK-SITE-BUILDER-INTEGRATION.md) | 2-3 weeks | MEDIUM | Phase 3 |
|
||||
|
||||
**Total Estimated Time**: 20-29 weeks (5-7 months)
|
||||
**Total Estimated Time**: 22-32 weeks (5.5-8 months)
|
||||
|
||||
---
|
||||
|
||||
@@ -79,6 +80,12 @@ This folder contains detailed implementation plans for each phase of the IGNY8 P
|
||||
- Support taxonomy generation
|
||||
- Extend linker and optimizer for all types
|
||||
|
||||
### Phase 9: AI Framework & Site Builder Integration
|
||||
- Add site structure prompt to Thinker UI
|
||||
- Document AI framework integration
|
||||
- Implement blueprint-to-writer task queuing
|
||||
- Enable bulk page content generation
|
||||
|
||||
---
|
||||
|
||||
## IMPLEMENTATION ORDER
|
||||
@@ -90,11 +97,13 @@ This folder contains detailed implementation plans for each phase of the IGNY8 P
|
||||
4. Phase 5 → Phase 6
|
||||
5. Phase 1 → Phase 4
|
||||
6. Phase 4 → Phase 8
|
||||
7. Phase 3 → Phase 9
|
||||
|
||||
**Parallel Phases** (can be done in parallel):
|
||||
- Phase 2 and Phase 3 (after Phase 1)
|
||||
- Phase 4 and Phase 5 (after Phase 1/3)
|
||||
- Phase 6 and Phase 7 (after Phase 5)
|
||||
- Phase 8 and Phase 9 (after Phase 3/4)
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user