igny8/docs/automation/AUTOMATION-DEPLOYMENT-CHECKLIST.md

Automation Implementation - Deployment Checklist

✅ Completed Components

Backend

• Database models created (AutomationConfig, AutomationRun)
• AutomationLogger service (file-based logging)
• AutomationService orchestrator (7-stage pipeline)
• API endpoints (AutomationViewSet)
• Celery tasks (scheduled checks, run execution, resume)
• URL routing registered
• Celery beat schedule configured
• Migration file created

Frontend

• TypeScript API service (automationService.ts)
• Main dashboard page (AutomationPage.tsx)
• StageCard component
• ActivityLog component
• ConfigModal component
• RunHistory component

Documentation

• Comprehensive README (AUTOMATION-IMPLEMENTATION-README.md)
• Original plan corrected (automation-plan.md)

⏳ Remaining Tasks

1. Run Database Migration

cd /data/app/igny8/backend
python manage.py migrate

This will create the automation_config and automation_run tables.

2. Register Frontend Route

Add to your React Router configuration:

import AutomationPage from './pages/Automation/AutomationPage';

// In your route definitions:
{
  path: '/automation',
  element: <AutomationPage />,
}

3. Add Navigation Link

Add link to main navigation menu:

{
  name: 'Automation',
  href: '/automation',
  icon: /* automation icon */
}

4. Verify Infrastructure

Celery Worker

# Check if running
docker ps | grep celery

# Start if needed
docker-compose up -d celery

Celery Beat

# Check if running
docker ps | grep beat

# Start if needed  
docker-compose up -d celery-beat

Redis/Cache

# Verify cache backend in settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': 'redis://redis:6379/1',
    }
}

5. Create Log Directory

mkdir -p /data/app/igny8/backend/logs/automation
chmod 755 /data/app/igny8/backend/logs/automation

6. Test API Endpoints

# Get config (should return default config)
curl -X GET "http://localhost:8000/api/v1/automation/config/?site_id=1" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Estimate credits
curl -X GET "http://localhost:8000/api/v1/automation/estimate/?site_id=1" \
  -H "Authorization: Bearer YOUR_TOKEN"

7. Test Frontend

1. Navigate to /automation page
2. Click [Configure] - modal should open
3. Save configuration
4. Click [Run Now] - should trigger automation
5. Verify real-time updates in stage cards
6. Check activity log is streaming

8. Test Scheduled Automation

1. Enable automation in config
2. Set scheduled time to 1 minute from now
3. Wait for next hour (beat checks hourly at :00)
4. Verify automation starts automatically

9. Monitor First Run

Watch logs in real-time:

# Backend logs
tail -f /data/app/igny8/backend/logs/automation/{account_id}/{site_id}/{run_id}/automation_run.log

# Celery worker logs
docker logs -f <celery_container>

# Django logs
docker logs -f <backend_container>

10. Verify Database Records

from igny8_core.business.automation.models import AutomationConfig, AutomationRun

# Check config created
AutomationConfig.objects.all()

# Check runs recorded
AutomationRun.objects.all()

# View stage results
run = AutomationRun.objects.latest('started_at')
print(run.stage_1_result)
print(run.stage_2_result)
# ... etc

Quick Start Commands

# 1. Run migration
cd /data/app/igny8/backend
python manage.py migrate

# 2. Create log directory
mkdir -p logs/automation
chmod 755 logs/automation

# 3. Restart services
docker-compose restart celery celery-beat

# 4. Verify Celery beat schedule
docker exec <celery_container> celery -A igny8_core inspect scheduled

# 5. Test automation (Django shell)
python manage.py shell
>>> from igny8_core.business.automation.services import AutomationService
>>> from igny8_core.modules.system.models import Account, Site
>>> account = Account.objects.first()
>>> site = Site.objects.first()
>>> service = AutomationService(account, site)
>>> service.estimate_credits()  # Should return number
>>> # Don't run start_automation() yet - test via UI first

Expected Behavior

First Successful Run

1. Stage 1: Process keywords → create clusters (2-5 min)
2. Stage 2: Generate ideas from clusters (1-2 min per cluster)
3. Stage 3: Create tasks from ideas (instant)
4. Stage 4: Generate content from tasks (3-5 min per task)
5. Stage 5: Extract image prompts (1-2 min per content)
6. Stage 6: Generate images (2-3 min per image)
7. Stage 7: Count content ready for review (instant)

Total time: 15-45 minutes depending on batch sizes

Stage Results Example

{
  "stage_1_result": {
    "keywords_processed": 20,
    "clusters_created": 4,
    "batches_run": 1,
    "credits_used": 4
  },
  "stage_2_result": {
    "clusters_processed": 4,
    "ideas_created": 16,
    "credits_used": 8
  },
  "stage_3_result": {
    "ideas_processed": 16,
    "tasks_created": 16,
    "batches_run": 1
  },
  "stage_4_result": {
    "tasks_processed": 16,
    "content_created": 16,
    "total_words": 40000,
    "credits_used": 80
  },
  "stage_5_result": {
    "content_processed": 16,
    "prompts_created": 64,
    "credits_used": 32
  },
  "stage_6_result": {
    "images_processed": 64,
    "images_generated": 64,
    "content_moved_to_review": 16,
    "credits_used": 128
  },
  "stage_7_result": {
    "ready_for_review": 16,
    "content_ids": [1, 2, 3, ...]
  }
}

Troubleshooting

"Module not found" errors

• Restart Django server after adding new models
• Run python manage.py collectstatic if needed

"Table does not exist" errors

• Run migration: python manage.py migrate

"No module named automation"

• Check __init__.py files exist in all directories
• Verify imports in urls.py

Celery tasks not running

• Check worker is running: docker ps | grep celery
• Check beat is running: docker ps | grep beat
• Verify tasks registered: celery -A igny8_core inspect registered

Logs not appearing

• Check directory permissions: ls -la logs/automation
• Check AutomationLogger.start_run() creates directories
• Verify log file path in code matches actual filesystem

Frontend errors

• Check API service imported correctly
• Verify route registered in router
• Check for TypeScript compilation errors
• Verify API endpoints returning expected data

Success Criteria

• Migration runs without errors
• Frontend /automation page loads
• Config modal opens and saves
• Credit estimate shows reasonable number
• "Run Now" starts automation successfully
• Stage cards update in real-time
• Activity log shows progress
• All 7 stages complete successfully
• Content moved to review status
• Run History table shows completed run
• Scheduled automation triggers at configured time

Post-Deployment

1. Monitor first few runs closely
2. Adjust batch sizes based on performance
3. Set up alerts for failed runs
4. Document any issues encountered
5. Train users on automation features
6. Gather feedback for improvements