241 lines
15 KiB
Markdown
241 lines
15 KiB
Markdown
# Site Builder & AI Functions - Comprehensive Audit Report
|
|
|
|
## Executive Summary
|
|
This audit identifies all requirements, dependencies, and gaps for the Site Builder functionality and related AI generation features. The primary issues are:
|
|
1. **Database migrations not applied** - Site builder tables don't exist
|
|
2. **Redis/Celery not running** - AI tasks can't execute asynchronously
|
|
3. All code components are present and properly configured
|
|
|
|
---
|
|
|
|
## Requirements Audit Table
|
|
|
|
| Category | Component | Requirement | Status | Location/Notes | Action Required |
|
|
|----------|-----------|-------------|--------|----------------|-----------------|
|
|
| **Database** | Migrations | `0001_initial.py` - SiteBlueprint, PageBlueprint tables | ✅ EXISTS | `backend/igny8_core/business/site_building/migrations/0001_initial.py` | ❌ **NOT APPLIED** - Run migrations |
|
|
| **Database** | Migrations | `0002_sitebuilder_metadata.py` - BusinessType, AudienceProfile, BrandPersonality, HeroImageryDirection | ✅ EXISTS | `backend/igny8_core/business/site_building/migrations/0002_sitebuilder_metadata.py` | ❌ **NOT APPLIED** - Run migrations |
|
|
| **Database** | Table | `igny8_site_blueprints` | ❌ MISSING | Should be created by 0001_initial.py | Run `python manage.py migrate site_building` |
|
|
| **Database** | Table | `igny8_page_blueprints` | ❌ MISSING | Should be created by 0001_initial.py | Run `python manage.py migrate site_building` |
|
|
| **Database** | Table | `igny8_site_builder_business_types` | ❌ MISSING | Should be created by 0002_sitebuilder_metadata.py | Run `python manage.py migrate site_building` |
|
|
| **Database** | Table | `igny8_site_builder_audience_profiles` | ❌ MISSING | Should be created by 0002_sitebuilder_metadata.py | Run `python manage.py migrate site_building` |
|
|
| **Database** | Table | `igny8_site_builder_brand_personalities` | ❌ MISSING | Should be created by 0002_sitebuilder_metadata.py | Run `python manage.py migrate site_building` |
|
|
| **Database** | Table | `igny8_site_builder_hero_imagery` | ❌ MISSING | Should be created by 0002_sitebuilder_metadata.py | Run `python manage.py migrate site_building` |
|
|
| **Database** | Seed Data | Business types, audience profiles, brand personalities, hero imagery | ✅ DEFINED | Migration 0002 includes `seed_site_builder_metadata()` function | Will seed automatically when migration runs |
|
|
| **App Config** | Django App | `igny8_core.business.site_building.apps.SiteBuildingConfig` | ✅ REGISTERED | `backend/igny8_core/settings.py:55` | No action needed |
|
|
| **App Config** | Django App | `igny8_core.modules.site_builder.apps.SiteBuilderConfig` | ✅ REGISTERED | `backend/igny8_core/settings.py:59` | No action needed |
|
|
| **Models** | SiteBlueprint | Model with all fields (name, description, config_json, structure_json, status, hosting_type, version) | ✅ EXISTS | `backend/igny8_core/business/site_building/models.py:10-83` | No action needed |
|
|
| **Models** | PageBlueprint | Model with all fields (site_blueprint, slug, title, type, blocks_json, status, order) | ✅ EXISTS | `backend/igny8_core/business/site_building/models.py:85-166` | No action needed |
|
|
| **Models** | BusinessType | Model extending SiteBuilderOption | ✅ EXISTS | `backend/igny8_core/business/site_building/models.py:189-194` | No action needed |
|
|
| **Models** | AudienceProfile | Model extending SiteBuilderOption | ✅ EXISTS | `backend/igny8_core/business/site_building/models.py:197-202` | No action needed |
|
|
| **Models** | BrandPersonality | Model extending SiteBuilderOption | ✅ EXISTS | `backend/igny8_core/business/site_building/models.py:205-210` | No action needed |
|
|
| **Models** | HeroImageryDirection | Model extending SiteBuilderOption | ✅ EXISTS | `backend/igny8_core/business/site_building/models.py:213-218` | No action needed |
|
|
| **Services** | StructureGenerationService | Service to trigger AI structure generation | ✅ EXISTS | `backend/igny8_core/business/site_building/services/structure_generation_service.py` | No action needed |
|
|
| **Services** | PageGenerationService | Service to generate page content via Writer | ✅ EXISTS | `backend/igny8_core/business/site_building/services/page_generation_service.py` | No action needed |
|
|
| **Services** | SiteBuilderFileService | File management service | ✅ EXISTS | `backend/igny8_core/business/site_building/services/file_management_service.py` | No action needed |
|
|
| **AI Functions** | GenerateSiteStructureFunction | AI function class | ✅ EXISTS | `backend/igny8_core/ai/functions/generate_site_structure.py` | No action needed |
|
|
| **AI Functions** | Function Registration | `generate_site_structure` registered in registry | ✅ REGISTERED | `backend/igny8_core/ai/registry.py:97-112` | No action needed |
|
|
| **AI Functions** | AI Task Dispatch | `run_ai_task` Celery task | ✅ EXISTS | `backend/igny8_core/ai/tasks.py:12-147` | No action needed |
|
|
| **AI Functions** | Prompt Template | `site_structure_generation` prompt | ✅ EXISTS | `backend/igny8_core/ai/prompts.py:242-307` | No action needed |
|
|
| **AI Functions** | Prompt Mapping | Function name → prompt type mapping | ✅ CONFIGURED | `backend/igny8_core/ai/prompts.py:599` | No action needed |
|
|
| **API Endpoints** | SiteBlueprintViewSet | CRUD + generate_structure action | ✅ EXISTS | `backend/igny8_core/modules/site_builder/views.py:32-81` | No action needed |
|
|
| **API Endpoints** | PageBlueprintViewSet | CRUD + generate_content action | ✅ EXISTS | `backend/igny8_core/modules/site_builder/views.py:141-172` | No action needed |
|
|
| **API Endpoints** | SiteBuilderMetadataView | Metadata endpoint for dropdowns | ✅ EXISTS | `backend/igny8_core/modules/site_builder/views.py:216-250` | ❌ **FAILS** - Needs DB tables |
|
|
| **API Endpoints** | SiteAssetView | File upload/download | ✅ EXISTS | `backend/igny8_core/modules/site_builder/views.py:175-213` | No action needed |
|
|
| **API URLs** | Site Builder URLs | `/api/v1/site-builder/` routes | ✅ CONFIGURED | `backend/igny8_core/modules/site_builder/urls.py` | No action needed |
|
|
| **API URLs** | URL Registration | Included in main urls.py | ✅ REGISTERED | `backend/igny8_core/urls.py:30` | No action needed |
|
|
| **Billing** | Credit Cost | `site_structure_generation: 50 credits` | ✅ DEFINED | `backend/igny8_core/business/billing/constants.py:13` | No action needed |
|
|
| **Billing** | Credit Cost | `site_page_generation: 20 credits` | ✅ DEFINED | `backend/igny8_core/business/billing/constants.py:14` | No action needed |
|
|
| **Billing** | Credit Check | CreditService.check_credits() called | ✅ IMPLEMENTED | `backend/igny8_core/business/site_building/services/structure_generation_service.py:57` | No action needed |
|
|
| **Celery** | Configuration | Celery app configured | ✅ CONFIGURED | `backend/igny8_core/celery.py` | No action needed |
|
|
| **Celery** | Settings | CELERY_BROKER_URL, CELERY_RESULT_BACKEND | ✅ CONFIGURED | `backend/igny8_core/settings.py:497-508` | No action needed |
|
|
| **Celery** | Worker Container | `igny8_celery_worker` in docker-compose | ✅ DEFINED | `docker-compose.app.yml:105-128` | ❌ **NOT RUNNING** - Start container |
|
|
| **Celery** | Beat Container | `igny8_celery_beat` in docker-compose | ✅ DEFINED | `docker-compose.app.yml:130-153` | ❌ **NOT RUNNING** - Start container |
|
|
| **Redis** | Configuration | REDIS_HOST, REDIS_PORT env vars | ✅ CONFIGURED | `docker-compose.app.yml:38-39, 115-116, 140-141` | No action needed |
|
|
| **Redis** | Redis Service | External Redis service | ❌ **NOT ACCESSIBLE** | Expected from infra stack | ❌ **NOT RUNNING** - Start Redis service |
|
|
| **Dependencies** | Python Packages | celery>=5.3.0 | ✅ IN REQUIREMENTS | `backend/requirements.txt:11` | No action needed |
|
|
| **Dependencies** | Python Packages | redis | ✅ IN REQUIREMENTS | `backend/requirements.txt:4` | No action needed |
|
|
| **Dependencies** | Python Packages | Django>=5.2.7 | ✅ IN REQUIREMENTS | `backend/requirements.txt:1` | No action needed |
|
|
| **Dependencies** | Python Packages | djangorestframework | ✅ IN REQUIREMENTS | `backend/requirements.txt:6` | No action needed |
|
|
| **Frontend** | API Client | siteBuilderApi functions | ✅ EXISTS | `frontend/src/services/siteBuilder.api.ts` | No action needed |
|
|
| **Frontend** | Store | useBuilderStore with metadata loading | ✅ EXISTS | `frontend/src/store/builderStore.ts:399-412` | No action needed |
|
|
| **Logging** | StructureGenerationService | Logging statements | ✅ IMPLEMENTED | `backend/igny8_core/business/site_building/services/structure_generation_service.py:49-53, 98-102` | No action needed |
|
|
| **Logging** | GenerateSiteStructure | Logging statements | ✅ IMPLEMENTED | `backend/igny8_core/ai/functions/generate_site_structure.py:135` | No action needed |
|
|
| **Error Handling** | InsufficientCreditsError | Handled in StructureGenerationService | ✅ IMPLEMENTED | `backend/igny8_core/business/site_building/services/structure_generation_service.py:58-61` | No action needed |
|
|
| **Error Handling** | Celery Fallback | Synchronous execution if Celery unavailable | ✅ IMPLEMENTED | `backend/igny8_core/business/site_building/services/structure_generation_service.py:109-115` | No action needed |
|
|
|
|
---
|
|
|
|
## Critical Issues Summary
|
|
|
|
### 🔴 **CRITICAL - Database Migrations Not Applied**
|
|
**Impact**: All Site Builder endpoints fail with `ProgrammingError: relation "igny8_site_builder_business_types" does not exist`
|
|
|
|
**Root Cause**: Migrations exist but haven't been run on the database
|
|
|
|
**Fix Required**:
|
|
```bash
|
|
# Step 1: Navigate to app directory
|
|
cd /data/app/igny8
|
|
|
|
# Step 2: Create migrations if model changes exist
|
|
docker compose -f docker-compose.app.yml -p igny8-app exec igny8_backend python manage.py makemigrations site_building
|
|
|
|
# Step 3: Apply migrations
|
|
docker compose -f docker-compose.app.yml -p igny8-app exec igny8_backend python manage.py migrate site_building
|
|
|
|
# Alternative: If backend is in infra stack, use:
|
|
# cd /data/app
|
|
# docker compose -f docker-compose.yml -p igny8-infra exec <backend_container_name> python manage.py makemigrations site_building
|
|
# docker compose -f docker-compose.yml -p igny8-infra exec <backend_container_name> python manage.py migrate site_building
|
|
```
|
|
|
|
**Files Affected**:
|
|
- `backend/igny8_core/business/site_building/migrations/0001_initial.py`
|
|
- `backend/igny8_core/business/site_building/migrations/0002_sitebuilder_metadata.py`
|
|
|
|
**Tables That Will Be Created**:
|
|
1. `igny8_site_blueprints`
|
|
2. `igny8_page_blueprints`
|
|
3. `igny8_site_builder_business_types`
|
|
4. `igny8_site_builder_audience_profiles`
|
|
5. `igny8_site_builder_brand_personalities`
|
|
6. `igny8_site_builder_hero_imagery`
|
|
|
|
---
|
|
|
|
### 🔴 **CRITICAL - Redis Not Running**
|
|
**Impact**: Celery tasks can't be queued, AI generation fails silently or runs synchronously
|
|
|
|
**Root Cause**: Redis service is not accessible (Connection refused errors in logs)
|
|
|
|
**Fix Required**:
|
|
1. Ensure Redis service is running in the infra stack
|
|
2. Verify network connectivity between backend and Redis
|
|
3. Check Redis is accessible at `redis:6379` from backend container
|
|
|
|
**Configuration**:
|
|
- Expected: `redis://redis:6379/0`
|
|
- Environment vars: `REDIS_HOST=redis`, `REDIS_PORT=6379`
|
|
|
|
---
|
|
|
|
### 🔴 **CRITICAL - Celery Worker Not Running**
|
|
**Impact**: Even if Redis is fixed, AI tasks won't execute because no worker is processing them
|
|
|
|
**Root Cause**: `igny8_celery_worker` container is not running
|
|
|
|
**Fix Required**:
|
|
```bash
|
|
# Navigate to app directory
|
|
cd /data/app/igny8
|
|
|
|
# Start Celery worker
|
|
docker compose -f docker-compose.app.yml -p igny8-app up -d igny8_celery_worker
|
|
|
|
# Verify it's running
|
|
docker compose -f docker-compose.app.yml -p igny8-app ps igny8_celery_worker
|
|
```
|
|
|
|
**Container Configuration**: `docker-compose.app.yml:105-128`
|
|
|
|
**Note**: If backend is in infra stack, Celery worker may also be there. Check which stack contains the backend service.
|
|
|
|
---
|
|
|
|
### 🟡 **WARNING - Celery Beat Not Running**
|
|
**Impact**: Periodic tasks (credit replenishment, automation rules) won't run
|
|
|
|
**Root Cause**: `igny8_celery_beat` container is not running
|
|
|
|
**Fix Required**:
|
|
```bash
|
|
# Navigate to app directory
|
|
cd /data/app/igny8
|
|
|
|
# Start Celery beat
|
|
docker compose -f docker-compose.app.yml -p igny8-app up -d igny8_celery_beat
|
|
|
|
# Verify it's running
|
|
docker compose -f docker-compose.app.yml -p igny8-app ps igny8_celery_beat
|
|
```
|
|
|
|
---
|
|
|
|
## Verification Checklist
|
|
|
|
After fixing the critical issues, verify:
|
|
|
|
- [ ] Database migrations applied: `python manage.py showmigrations site_building` shows all as `[X]`
|
|
- [ ] Tables exist: Query `igny8_site_builder_business_types` returns data
|
|
- [ ] Redis accessible: `redis-cli -h redis ping` returns `PONG`
|
|
- [ ] Celery worker running: `docker ps | grep celery_worker` shows container
|
|
- [ ] Celery worker connected: Check logs for "celery@hostname ready"
|
|
- [ ] Metadata endpoint works: `GET /api/v1/site-builder/metadata/` returns 200 with data
|
|
- [ ] AI task can be queued: Check logs for `[StructureGenerationService] Queued AI task`
|
|
- [ ] AI task executes: Check logs for `run_ai_task STARTED: generate_site_structure`
|
|
|
|
---
|
|
|
|
## Code Quality Assessment
|
|
|
|
### ✅ **Strengths**
|
|
1. **Complete Implementation**: All core components are implemented
|
|
2. **Proper Separation**: Business logic separated from API layer
|
|
3. **Error Handling**: Graceful fallbacks for Celery unavailability
|
|
4. **Credit System**: Properly integrated with billing system
|
|
5. **Logging**: Comprehensive logging throughout the flow
|
|
6. **Type Hints**: Good use of type hints in services
|
|
7. **Documentation**: Models and services have docstrings
|
|
|
|
### ⚠️ **Potential Improvements**
|
|
1. **Migration Dependencies**: Ensure migration dependencies are correct (currently depends on `igny8_core_auth.0014` and `writer.0009`)
|
|
2. **Error Messages**: Could be more user-friendly in API responses
|
|
3. **Testing**: Test files exist but may need updates for current implementation
|
|
4. **Monitoring**: Consider adding metrics for AI task success/failure rates
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. **Immediate**: Run database migrations
|
|
2. **Immediate**: Start Redis service (if in infra stack)
|
|
3. **Immediate**: Start Celery worker container
|
|
4. **Immediate**: Start Celery beat container (optional but recommended)
|
|
5. **Verification**: Test metadata endpoint
|
|
6. **Verification**: Test structure generation endpoint
|
|
7. **Monitoring**: Watch logs for AI task execution
|
|
|
|
---
|
|
|
|
## Files Reference
|
|
|
|
### Core Models
|
|
- `backend/igny8_core/business/site_building/models.py` - All Site Builder models
|
|
|
|
### Services
|
|
- `backend/igny8_core/business/site_building/services/structure_generation_service.py` - AI structure generation
|
|
- `backend/igny8_core/business/site_building/services/page_generation_service.py` - Page content generation
|
|
- `backend/igny8_core/business/site_building/services/file_management_service.py` - File handling
|
|
|
|
### AI Functions
|
|
- `backend/igny8_core/ai/functions/generate_site_structure.py` - AI function implementation
|
|
- `backend/igny8_core/ai/tasks.py` - Celery task dispatcher
|
|
- `backend/igny8_core/ai/registry.py` - Function registry
|
|
|
|
### API Layer
|
|
- `backend/igny8_core/modules/site_builder/views.py` - API endpoints
|
|
- `backend/igny8_core/modules/site_builder/urls.py` - URL routing
|
|
- `backend/igny8_core/modules/site_builder/serializers.py` - Request/response serialization
|
|
|
|
### Configuration
|
|
- `backend/igny8_core/settings.py` - Django settings
|
|
- `backend/igny8_core/celery.py` - Celery configuration
|
|
- `docker-compose.app.yml` - Container definitions
|
|
- `backend/requirements.txt` - Python dependencies
|
|
|
|
---
|
|
|
|
**Report Generated**: Based on comprehensive codebase analysis
|
|
**Status**: All code components present, infrastructure needs attention
|
|
|