igny8/SITE_BUILDER_AUDIT_REPORT.md

Site Builder & AI Functions - Comprehensive Audit Report

Executive Summary

This audit identifies all requirements, dependencies, and gaps for the Site Builder functionality and related AI generation features. The primary issues are:

1. Database migrations not applied - Site builder tables don't exist
2. Redis/Celery not running - AI tasks can't execute asynchronously
3. All code components are present and properly configured

---

Requirements Audit Table

Category	Component	Requirement	Status	Location/Notes	Action Required
Database	Migrations	0001_initial.py - SiteBlueprint, PageBlueprint tables	✅ EXISTS	backend/igny8_core/business/site_building/migrations/0001_initial.py	❌ NOT APPLIED - Run migrations
Database	Migrations	0002_sitebuilder_metadata.py - BusinessType, AudienceProfile, BrandPersonality, HeroImageryDirection	✅ EXISTS	backend/igny8_core/business/site_building/migrations/0002_sitebuilder_metadata.py	❌ NOT APPLIED - Run migrations
Database	Table	igny8_site_blueprints	❌ MISSING	Should be created by 0001_initial.py	Run python manage.py migrate site_building
Database	Table	igny8_page_blueprints	❌ MISSING	Should be created by 0001_initial.py	Run python manage.py migrate site_building
Database	Table	igny8_site_builder_business_types	❌ MISSING	Should be created by 0002_sitebuilder_metadata.py	Run python manage.py migrate site_building
Database	Table	igny8_site_builder_audience_profiles	❌ MISSING	Should be created by 0002_sitebuilder_metadata.py	Run python manage.py migrate site_building
Database	Table	igny8_site_builder_brand_personalities	❌ MISSING	Should be created by 0002_sitebuilder_metadata.py	Run python manage.py migrate site_building
Database	Table	igny8_site_builder_hero_imagery	❌ MISSING	Should be created by 0002_sitebuilder_metadata.py	Run python manage.py migrate site_building
Database	Seed Data	Business types, audience profiles, brand personalities, hero imagery	✅ DEFINED	Migration 0002 includes seed_site_builder_metadata() function	Will seed automatically when migration runs
App Config	Django App	igny8_core.business.site_building.apps.SiteBuildingConfig	✅ REGISTERED	backend/igny8_core/settings.py:55	No action needed
App Config	Django App	igny8_core.modules.site_builder.apps.SiteBuilderConfig	✅ REGISTERED	backend/igny8_core/settings.py:59	No action needed
Models	SiteBlueprint	Model with all fields (name, description, config_json, structure_json, status, hosting_type, version)	✅ EXISTS	backend/igny8_core/business/site_building/models.py:10-83	No action needed
Models	PageBlueprint	Model with all fields (site_blueprint, slug, title, type, blocks_json, status, order)	✅ EXISTS	backend/igny8_core/business/site_building/models.py:85-166	No action needed
Models	BusinessType	Model extending SiteBuilderOption	✅ EXISTS	backend/igny8_core/business/site_building/models.py:189-194	No action needed
Models	AudienceProfile	Model extending SiteBuilderOption	✅ EXISTS	backend/igny8_core/business/site_building/models.py:197-202	No action needed
Models	BrandPersonality	Model extending SiteBuilderOption	✅ EXISTS	backend/igny8_core/business/site_building/models.py:205-210	No action needed
Models	HeroImageryDirection	Model extending SiteBuilderOption	✅ EXISTS	backend/igny8_core/business/site_building/models.py:213-218	No action needed
Services	StructureGenerationService	Service to trigger AI structure generation	✅ EXISTS	backend/igny8_core/business/site_building/services/structure_generation_service.py	No action needed
Services	PageGenerationService	Service to generate page content via Writer	✅ EXISTS	backend/igny8_core/business/site_building/services/page_generation_service.py	No action needed
Services	SiteBuilderFileService	File management service	✅ EXISTS	backend/igny8_core/business/site_building/services/file_management_service.py	No action needed
AI Functions	GenerateSiteStructureFunction	AI function class	✅ EXISTS	backend/igny8_core/ai/functions/generate_site_structure.py	No action needed
AI Functions	Function Registration	generate_site_structure registered in registry	✅ REGISTERED	backend/igny8_core/ai/registry.py:97-112	No action needed
AI Functions	AI Task Dispatch	run_ai_task Celery task	✅ EXISTS	backend/igny8_core/ai/tasks.py:12-147	No action needed
AI Functions	Prompt Template	site_structure_generation prompt	✅ EXISTS	backend/igny8_core/ai/prompts.py:242-307	No action needed
AI Functions	Prompt Mapping	Function name → prompt type mapping	✅ CONFIGURED	backend/igny8_core/ai/prompts.py:599	No action needed
API Endpoints	SiteBlueprintViewSet	CRUD + generate_structure action	✅ EXISTS	backend/igny8_core/modules/site_builder/views.py:32-81	No action needed
API Endpoints	PageBlueprintViewSet	CRUD + generate_content action	✅ EXISTS	backend/igny8_core/modules/site_builder/views.py:141-172	No action needed
API Endpoints	SiteBuilderMetadataView	Metadata endpoint for dropdowns	✅ EXISTS	backend/igny8_core/modules/site_builder/views.py:216-250	❌ FAILS - Needs DB tables
API Endpoints	SiteAssetView	File upload/download	✅ EXISTS	backend/igny8_core/modules/site_builder/views.py:175-213	No action needed
API URLs	Site Builder URLs	/api/v1/site-builder/ routes	✅ CONFIGURED	backend/igny8_core/modules/site_builder/urls.py	No action needed
API URLs	URL Registration	Included in main urls.py	✅ REGISTERED	backend/igny8_core/urls.py:30	No action needed
Billing	Credit Cost	site_structure_generation: 50 credits	✅ DEFINED	backend/igny8_core/business/billing/constants.py:13	No action needed
Billing	Credit Cost	site_page_generation: 20 credits	✅ DEFINED	backend/igny8_core/business/billing/constants.py:14	No action needed
Billing	Credit Check	CreditService.check_credits() called	✅ IMPLEMENTED	backend/igny8_core/business/site_building/services/structure_generation_service.py:57	No action needed
Celery	Configuration	Celery app configured	✅ CONFIGURED	backend/igny8_core/celery.py	No action needed
Celery	Settings	CELERY_BROKER_URL, CELERY_RESULT_BACKEND	✅ CONFIGURED	backend/igny8_core/settings.py:497-508	No action needed
Celery	Worker Container	igny8_celery_worker in docker-compose	✅ DEFINED	docker-compose.app.yml:105-128	❌ NOT RUNNING - Start container
Celery	Beat Container	igny8_celery_beat in docker-compose	✅ DEFINED	docker-compose.app.yml:130-153	❌ NOT RUNNING - Start container
Redis	Configuration	REDIS_HOST, REDIS_PORT env vars	✅ CONFIGURED	docker-compose.app.yml:38-39, 115-116, 140-141	No action needed
Redis	Redis Service	External Redis service	❌ NOT ACCESSIBLE	Expected from infra stack	❌ NOT RUNNING - Start Redis service
Dependencies	Python Packages	celery>=5.3.0	✅ IN REQUIREMENTS	backend/requirements.txt:11	No action needed
Dependencies	Python Packages	redis	✅ IN REQUIREMENTS	backend/requirements.txt:4	No action needed
Dependencies	Python Packages	Django>=5.2.7	✅ IN REQUIREMENTS	backend/requirements.txt:1	No action needed
Dependencies	Python Packages	djangorestframework	✅ IN REQUIREMENTS	backend/requirements.txt:6	No action needed
Frontend	API Client	siteBuilderApi functions	✅ EXISTS	frontend/src/services/siteBuilder.api.ts	No action needed
Frontend	Store	useBuilderStore with metadata loading	✅ EXISTS	frontend/src/store/builderStore.ts:399-412	No action needed
Logging	StructureGenerationService	Logging statements	✅ IMPLEMENTED	backend/igny8_core/business/site_building/services/structure_generation_service.py:49-53, 98-102	No action needed
Logging	GenerateSiteStructure	Logging statements	✅ IMPLEMENTED	backend/igny8_core/ai/functions/generate_site_structure.py:135	No action needed
Error Handling	InsufficientCreditsError	Handled in StructureGenerationService	✅ IMPLEMENTED	backend/igny8_core/business/site_building/services/structure_generation_service.py:58-61	No action needed
Error Handling	Celery Fallback	Synchronous execution if Celery unavailable	✅ IMPLEMENTED	backend/igny8_core/business/site_building/services/structure_generation_service.py:109-115	No action needed

---

Critical Issues Summary

🔴 CRITICAL - Database Migrations Not Applied

Impact: All Site Builder endpoints fail with ProgrammingError: relation "igny8_site_builder_business_types" does not exist

Root Cause: Migrations exist but haven't been run on the database

Fix Required:

# Step 1: Navigate to app directory
cd /data/app/igny8

# Step 2: Create migrations if model changes exist
docker compose -f docker-compose.app.yml -p igny8-app exec igny8_backend python manage.py makemigrations site_building

# Step 3: Apply migrations
docker compose -f docker-compose.app.yml -p igny8-app exec igny8_backend python manage.py migrate site_building

# Alternative: If backend is in infra stack, use:
# cd /data/app
# docker compose -f docker-compose.yml -p igny8-infra exec <backend_container_name> python manage.py makemigrations site_building
# docker compose -f docker-compose.yml -p igny8-infra exec <backend_container_name> python manage.py migrate site_building

Files Affected:

• backend/igny8_core/business/site_building/migrations/0001_initial.py
• backend/igny8_core/business/site_building/migrations/0002_sitebuilder_metadata.py

Tables That Will Be Created:

1. igny8_site_blueprints
2. igny8_page_blueprints
3. igny8_site_builder_business_types
4. igny8_site_builder_audience_profiles
5. igny8_site_builder_brand_personalities
6. igny8_site_builder_hero_imagery

---

🔴 CRITICAL - Redis Not Running

Impact: Celery tasks can't be queued, AI generation fails silently or runs synchronously

Root Cause: Redis service is not accessible (Connection refused errors in logs)

Fix Required:

1. Ensure Redis service is running in the infra stack
2. Verify network connectivity between backend and Redis
3. Check Redis is accessible at redis:6379 from backend container

Configuration:

• Expected: redis://redis:6379/0
• Environment vars: REDIS_HOST=redis, REDIS_PORT=6379

---

🔴 CRITICAL - Celery Worker Not Running

Impact: Even if Redis is fixed, AI tasks won't execute because no worker is processing them

Root Cause: igny8_celery_worker container is not running

Fix Required:

# Navigate to app directory
cd /data/app/igny8

# Start Celery worker
docker compose -f docker-compose.app.yml -p igny8-app up -d igny8_celery_worker

# Verify it's running
docker compose -f docker-compose.app.yml -p igny8-app ps igny8_celery_worker

Container Configuration: docker-compose.app.yml:105-128

Note: If backend is in infra stack, Celery worker may also be there. Check which stack contains the backend service.

---

🟡 WARNING - Celery Beat Not Running

Impact: Periodic tasks (credit replenishment, automation rules) won't run

Root Cause: igny8_celery_beat container is not running

Fix Required:

# Navigate to app directory
cd /data/app/igny8

# Start Celery beat
docker compose -f docker-compose.app.yml -p igny8-app up -d igny8_celery_beat

# Verify it's running
docker compose -f docker-compose.app.yml -p igny8-app ps igny8_celery_beat

---

Verification Checklist

After fixing the critical issues, verify:

• Database migrations applied: python manage.py showmigrations site_building shows all as [X]
• Tables exist: Query igny8_site_builder_business_types returns data
• Redis accessible: redis-cli -h redis ping returns PONG
• Celery worker running: docker ps | grep celery_worker shows container
• Celery worker connected: Check logs for "celery@hostname ready"
• Metadata endpoint works: GET /api/v1/site-builder/metadata/ returns 200 with data
• AI task can be queued: Check logs for [StructureGenerationService] Queued AI task
• AI task executes: Check logs for run_ai_task STARTED: generate_site_structure

---

Code Quality Assessment

✅ Strengths

1. Complete Implementation: All core components are implemented
2. Proper Separation: Business logic separated from API layer
3. Error Handling: Graceful fallbacks for Celery unavailability
4. Credit System: Properly integrated with billing system
5. Logging: Comprehensive logging throughout the flow
6. Type Hints: Good use of type hints in services
7. Documentation: Models and services have docstrings

⚠️ Potential Improvements

1. Migration Dependencies: Ensure migration dependencies are correct (currently depends on igny8_core_auth.0014 and writer.0009)
2. Error Messages: Could be more user-friendly in API responses
3. Testing: Test files exist but may need updates for current implementation
4. Monitoring: Consider adding metrics for AI task success/failure rates

---

Next Steps

1. Immediate: Run database migrations
2. Immediate: Start Redis service (if in infra stack)
3. Immediate: Start Celery worker container
4. Immediate: Start Celery beat container (optional but recommended)
5. Verification: Test metadata endpoint
6. Verification: Test structure generation endpoint
7. Monitoring: Watch logs for AI task execution

---

Files Reference

Core Models

• backend/igny8_core/business/site_building/models.py - All Site Builder models

Services

• backend/igny8_core/business/site_building/services/structure_generation_service.py - AI structure generation
• backend/igny8_core/business/site_building/services/page_generation_service.py - Page content generation
• backend/igny8_core/business/site_building/services/file_management_service.py - File handling

AI Functions

• backend/igny8_core/ai/functions/generate_site_structure.py - AI function implementation
• backend/igny8_core/ai/tasks.py - Celery task dispatcher
• backend/igny8_core/ai/registry.py - Function registry

API Layer

• backend/igny8_core/modules/site_builder/views.py - API endpoints
• backend/igny8_core/modules/site_builder/urls.py - URL routing
• backend/igny8_core/modules/site_builder/serializers.py - Request/response serialization

Configuration

• backend/igny8_core/settings.py - Django settings
• backend/igny8_core/celery.py - Celery configuration
• docker-compose.app.yml - Container definitions
• backend/requirements.txt - Python dependencies

---

Report Generated: Based on comprehensive codebase analysis Status: All code components present, infrastructure needs attention