igny8/AUTOMATION-DEPLOYMENT-CHECKLIST.md

# Automation Implementation - Deployment Checklist

## ✅ Completed Components

### Backend
- [x] Database models created (`AutomationConfig`, `AutomationRun`)
- [x] AutomationLogger service (file-based logging)
- [x] AutomationService orchestrator (7-stage pipeline)
- [x] API endpoints (`AutomationViewSet`)
- [x] Celery tasks (scheduled checks, run execution, resume)
- [x] URL routing registered
- [x] Celery beat schedule configured
- [x] Migration file created

### Frontend
- [x] TypeScript API service (`automationService.ts`)
- [x] Main dashboard page (`AutomationPage.tsx`)
- [x] StageCard component
- [x] ActivityLog component  
- [x] ConfigModal component
- [x] RunHistory component

### Documentation
- [x] Comprehensive README (`AUTOMATION-IMPLEMENTATION-README.md`)
- [x] Original plan corrected (`automation-plan.md`)

## ⏳ Remaining Tasks

### 1. Run Database Migration

```bash
cd /data/app/igny8/backend
python manage.py migrate
```

This will create the `automation_config` and `automation_run` tables.

### 2. Register Frontend Route

Add to your React Router configuration:

```typescript
import AutomationPage from './pages/Automation/AutomationPage';

// In your route definitions:
{
  path: '/automation',
  element: <AutomationPage />,
}
```

### 3. Add Navigation Link

Add link to main navigation menu:

```typescript
{
  name: 'Automation',
  href: '/automation',
  icon: /* automation icon */
}
```

### 4. Verify Infrastructure

**Celery Worker**
```bash
# Check if running
docker ps | grep celery

# Start if needed
docker-compose up -d celery
```

**Celery Beat**
```bash
# Check if running
docker ps | grep beat

# Start if needed  
docker-compose up -d celery-beat
```

**Redis/Cache**
```bash
# Verify cache backend in settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': 'redis://redis:6379/1',
    }
}
```

### 5. Create Log Directory

```bash
mkdir -p /data/app/igny8/backend/logs/automation
chmod 755 /data/app/igny8/backend/logs/automation
```

### 6. Test API Endpoints

```bash
# Get config (should return default config)
curl -X GET "http://localhost:8000/api/v1/automation/config/?site_id=1" \
  -H "Authorization: Bearer YOUR_TOKEN"

# Estimate credits
curl -X GET "http://localhost:8000/api/v1/automation/estimate/?site_id=1" \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### 7. Test Frontend

1. Navigate to `/automation` page
2. Click [Configure] - modal should open
3. Save configuration
4. Click [Run Now] - should trigger automation
5. Verify real-time updates in stage cards
6. Check activity log is streaming

### 8. Test Scheduled Automation

1. Enable automation in config
2. Set scheduled time to 1 minute from now
3. Wait for next hour (beat checks hourly at :00)
4. Verify automation starts automatically

### 9. Monitor First Run

Watch logs in real-time:

```bash
# Backend logs
tail -f /data/app/igny8/backend/logs/automation/{account_id}/{site_id}/{run_id}/automation_run.log

# Celery worker logs
docker logs -f <celery_container>

# Django logs
docker logs -f <backend_container>
```

### 10. Verify Database Records

```python
from igny8_core.business.automation.models import AutomationConfig, AutomationRun

# Check config created
AutomationConfig.objects.all()

# Check runs recorded
AutomationRun.objects.all()

# View stage results
run = AutomationRun.objects.latest('started_at')
print(run.stage_1_result)
print(run.stage_2_result)
# ... etc
```

## Quick Start Commands

```bash
# 1. Run migration
cd /data/app/igny8/backend
python manage.py migrate

# 2. Create log directory
mkdir -p logs/automation
chmod 755 logs/automation

# 3. Restart services
docker-compose restart celery celery-beat

# 4. Verify Celery beat schedule
docker exec <celery_container> celery -A igny8_core inspect scheduled

# 5. Test automation (Django shell)
python manage.py shell
>>> from igny8_core.business.automation.services import AutomationService
>>> from igny8_core.modules.system.models import Account, Site
>>> account = Account.objects.first()
>>> site = Site.objects.first()
>>> service = AutomationService(account, site)
>>> service.estimate_credits()  # Should return number
>>> # Don't run start_automation() yet - test via UI first
```

## Expected Behavior

### First Successful Run

1. **Stage 1**: Process keywords → create clusters (2-5 min)
2. **Stage 2**: Generate ideas from clusters (1-2 min per cluster)
3. **Stage 3**: Create tasks from ideas (instant)
4. **Stage 4**: Generate content from tasks (3-5 min per task)
5. **Stage 5**: Extract image prompts (1-2 min per content)
6. **Stage 6**: Generate images (2-3 min per image)
7. **Stage 7**: Count content ready for review (instant)

Total time: 15-45 minutes depending on batch sizes

### Stage Results Example

```json
{
  "stage_1_result": {
    "keywords_processed": 20,
    "clusters_created": 4,
    "batches_run": 1,
    "credits_used": 4
  },
  "stage_2_result": {
    "clusters_processed": 4,
    "ideas_created": 16,
    "credits_used": 8
  },
  "stage_3_result": {
    "ideas_processed": 16,
    "tasks_created": 16,
    "batches_run": 1
  },
  "stage_4_result": {
    "tasks_processed": 16,
    "content_created": 16,
    "total_words": 40000,
    "credits_used": 80
  },
  "stage_5_result": {
    "content_processed": 16,
    "prompts_created": 64,
    "credits_used": 32
  },
  "stage_6_result": {
    "images_processed": 64,
    "images_generated": 64,
    "content_moved_to_review": 16,
    "credits_used": 128
  },
  "stage_7_result": {
    "ready_for_review": 16,
    "content_ids": [1, 2, 3, ...]
  }
}
```

## Troubleshooting

### "Module not found" errors
- Restart Django server after adding new models
- Run `python manage.py collectstatic` if needed

### "Table does not exist" errors
- Run migration: `python manage.py migrate`

### "No module named automation"
- Check `__init__.py` files exist in all directories
- Verify imports in `urls.py`

### Celery tasks not running
- Check worker is running: `docker ps | grep celery`
- Check beat is running: `docker ps | grep beat`
- Verify tasks registered: `celery -A igny8_core inspect registered`

### Logs not appearing
- Check directory permissions: `ls -la logs/automation`
- Check AutomationLogger.start_run() creates directories
- Verify log file path in code matches actual filesystem

### Frontend errors
- Check API service imported correctly
- Verify route registered in router
- Check for TypeScript compilation errors
- Verify API endpoints returning expected data

## Success Criteria

- [ ] Migration runs without errors
- [ ] Frontend `/automation` page loads
- [ ] Config modal opens and saves
- [ ] Credit estimate shows reasonable number
- [ ] "Run Now" starts automation successfully
- [ ] Stage cards update in real-time
- [ ] Activity log shows progress
- [ ] All 7 stages complete successfully
- [ ] Content moved to review status
- [ ] Run History table shows completed run
- [ ] Scheduled automation triggers at configured time

## Post-Deployment

1. Monitor first few runs closely
2. Adjust batch sizes based on performance
3. Set up alerts for failed runs
4. Document any issues encountered
5. Train users on automation features
6. Gather feedback for improvements