28 KiB
IGNY8 Automation System Implementation Plan
Overview
This document provides a detailed, step-by-step implementation plan for completing the IGNY8 Automation system. The backend infrastructure is already in place; this plan focuses on extending functionality and building the frontend interface.
Current State Analysis
✅ Already Implemented (Backend)
-
Data Models (
backend/igny8_core/business/automation/models.py)AutomationRule: Complete with trigger types, conditions, actionsScheduledTask: Task scheduling and tracking
-
Core Services (
backend/igny8_core/business/automation/services/)AutomationService: Main orchestratorRuleEngine: Condition evaluationConditionEvaluator: Field path resolutionActionExecutor: Action execution (partial)
-
API Endpoints (
backend/igny8_core/modules/automation/views.py)AutomationRuleViewSet: CRUD + execute endpointScheduledTaskViewSet: Task management
❌ Missing Components
-
Backend Extensions:
- ActionExecutor only supports 3 actions (cluster_keywords, generate_ideas, generate_content)
- Missing: generate_images, generate_site_structure, generate_page_content, optimize_content, publish_to_wordpress
- No Celery task integration for async execution
- No progress tracking integration
- No webhook support for event triggers
-
Frontend (Complete):
- No automation module pages
- No rule management interface
- No workflow visualizer
- No execution monitor
- No progress tracking UI
-
Integration:
- No connection to existing AI functions
- No WordPress publish integration
- No credit cost estimation
Implementation Phases
Phase 1: Backend Extensions (Week 1)
1.1 Extend ActionExecutor
File: backend/igny8_core/business/automation/services/action_executor.py
Changes Required:
# Add new action methods to ActionExecutor class
def _execute_generate_images(self, params, rule):
"""Execute generate images action"""
content_id = params.get('content_id')
provider = params.get('provider', 'dall-e')
try:
from igny8_core.business.content.services.image_generation_service import ImageGenerationService
service = ImageGenerationService()
result = service.generate_images(
content_id=content_id,
provider=provider,
account=rule.account
)
return result
except Exception as e:
logger.error(f"Error generating images: {str(e)}", exc_info=True)
return {'success': False, 'error': str(e)}
def _execute_generate_site_structure(self, params, rule):
"""Execute generate site structure action"""
site_id = params.get('site_id') or (rule.site.id if rule.site else None)
try:
from igny8_core.business.sites.services.site_generation_service import SiteGenerationService
service = SiteGenerationService()
result = service.generate_structure(
site_id=site_id,
account=rule.account
)
return result
except Exception as e:
logger.error(f"Error generating site structure: {str(e)}", exc_info=True)
return {'success': False, 'error': str(e)}
def _execute_generate_page_content(self, params, rule):
"""Execute generate page content action"""
page_ids = params.get('page_ids', [])
try:
from igny8_core.business.sites.services.page_generation_service import PageGenerationService
service = PageGenerationService()
result = service.generate_content(
page_ids=page_ids,
account=rule.account
)
return result
except Exception as e:
logger.error(f"Error generating page content: {str(e)}", exc_info=True)
return {'success': False, 'error': str(e)}
def _execute_optimize_content(self, params, rule):
"""Execute optimize content action"""
content_ids = params.get('content_ids', [])
try:
from igny8_core.business.optimizer.services.optimization_service import OptimizationService
service = OptimizationService()
result = service.optimize_content(
content_ids=content_ids,
account=rule.account
)
return result
except Exception as e:
logger.error(f"Error optimizing content: {str(e)}", exc_info=True)
return {'success': False, 'error': str(e)}
def _execute_publish_to_wordpress(self, params, rule):
"""Execute publish to WordPress action"""
content_ids = params.get('content_ids', [])
integration_id = params.get('integration_id')
try:
from igny8_core.business.integration.services.wordpress_service import WordPressService
service = WordPressService()
result = service.publish_content(
content_ids=content_ids,
integration_id=integration_id,
account=rule.account
)
return result
except Exception as e:
logger.error(f"Error publishing to WordPress: {str(e)}", exc_info=True)
return {'success': False, 'error': str(e)}
# Update execute() method to handle new action types
def execute(self, action, context, rule):
"""Execute a single action"""
action_type = action.get('type')
params = action.get('params', {})
action_map = {
'cluster_keywords': self._execute_cluster_keywords,
'generate_ideas': self._execute_generate_ideas,
'generate_content': self._execute_generate_content,
'generate_images': self._execute_generate_images,
'generate_site_structure': self._execute_generate_site_structure,
'generate_page_content': self._execute_generate_page_content,
'optimize_content': self._execute_optimize_content,
'publish_to_wordpress': self._execute_publish_to_wordpress,
}
executor = action_map.get(action_type)
if executor:
return executor(params, rule)
else:
logger.warning(f"Unknown action type: {action_type}")
return {'success': False, 'error': f'Unknown action type: {action_type}'}
Estimated Time: 4 hours
1.2 Create Celery Tasks
File: backend/igny8_core/business/automation/tasks.py
Changes Required:
"""
Celery tasks for automation
"""
from celery import shared_task
from django.utils import timezone
from igny8_core.business.automation.models import AutomationRule, ScheduledTask
from igny8_core.business.automation.services.automation_service import AutomationService
import logging
logger = logging.getLogger(__name__)
@shared_task(bind=True, name='automation.execute_rule')
def execute_rule_task(self, rule_id, context=None):
"""
Execute an automation rule asynchronously
Args:
rule_id: AutomationRule ID
context: Optional context dict
"""
try:
rule = AutomationRule.objects.get(id=rule_id)
service = AutomationService()
# Update task progress
self.update_state(
state='PROGRESS',
meta={'phase': 'Initializing', 'progress': 0}
)
result = service.execute_rule(rule, context or {})
self.update_state(
state='SUCCESS',
meta={'phase': 'Completed', 'progress': 100, 'result': result}
)
return result
except Exception as e:
logger.error(f"Error executing rule {rule_id}: {str(e)}", exc_info=True)
self.update_state(
state='FAILURE',
meta={'error': str(e)}
)
raise
@shared_task(name='automation.execute_scheduled_rules')
def execute_scheduled_rules_task():
"""
Execute all scheduled rules that are due
Called by Celery Beat
"""
service = AutomationService()
result = service.execute_scheduled_rules()
logger.info(f"Scheduled rules execution: {result}")
return result
@shared_task(name='automation.cleanup_old_tasks')
def cleanup_old_tasks():
"""
Clean up old scheduled tasks
Keep last 30 days only
"""
from datetime import timedelta
cutoff_date = timezone.now() - timedelta(days=30)
deleted_count = ScheduledTask.objects.filter(
created_at__lt=cutoff_date,
status__in=['completed', 'failed', 'cancelled']
).delete()[0]
logger.info(f"Cleaned up {deleted_count} old scheduled tasks")
return {'deleted': deleted_count}
Estimated Time: 2 hours
1.3 Add Progress Tracking
File: backend/igny8_core/business/automation/services/automation_service.py
Changes Required:
# Add to execute_rule method
def execute_rule(self, rule, context=None):
"""Execute an automation rule with progress tracking"""
# ... existing code ...
# Execute via rule engine with progress tracking
try:
from celery import current_task
# Update progress for each action
total_actions = len(rule.actions)
for idx, action in enumerate(rule.actions):
if current_task:
current_task.update_state(
state='PROGRESS',
meta={
'phase': f'Executing action {idx + 1}/{total_actions}',
'progress': int((idx / total_actions) * 100),
'action': action.get('type')
}
)
# Execute action
action_result = self.rule_engine.execute_action(action, context, rule)
# Store result
results.append(action_result)
# ... rest of existing code ...
Estimated Time: 2 hours
1.4 Add API Endpoints
File: backend/igny8_core/modules/automation/views.py
Changes Required:
# Add new endpoints to AutomationRuleViewSet
@action(detail=True, methods=['get'], url_path='progress', url_name='progress')
def get_progress(self, request, pk=None):
"""Get execution progress for a rule"""
rule = self.get_object()
# Get latest task for this rule
from celery.result import AsyncResult
from igny8_core.business.automation.models import ScheduledTask
latest_task = ScheduledTask.objects.filter(
automation_rule=rule,
status='running'
).order_by('-created_at').first()
if not latest_task or not latest_task.metadata.get('task_id'):
return success_response(
data={'status': 'idle'},
request=request
)
task_id = latest_task.metadata['task_id']
task_result = AsyncResult(task_id)
return success_response(
data={
'status': task_result.state,
'progress': task_result.info.get('progress', 0) if task_result.info else 0,
'phase': task_result.info.get('phase', 'Unknown') if task_result.info else 'Unknown',
'result': task_result.result if task_result.successful() else None
},
request=request
)
@action(detail=False, methods=['get'], url_path='action-types', url_name='action-types')
def get_action_types(self, request):
"""Get available action types"""
action_types = [
{
'type': 'cluster_keywords',
'name': 'Cluster Keywords',
'description': 'Automatically cluster keywords by topic',
'params': ['keyword_ids', 'sector_id']
},
{
'type': 'generate_ideas',
'name': 'Generate Ideas',
'description': 'Generate content ideas from clusters',
'params': ['cluster_ids']
},
{
'type': 'generate_content',
'name': 'Generate Content',
'description': 'Generate article content from tasks',
'params': ['task_ids']
},
{
'type': 'generate_images',
'name': 'Generate Images',
'description': 'Generate images for content',
'params': ['content_id', 'provider']
},
{
'type': 'generate_site_structure',
'name': 'Generate Site Structure',
'description': 'Generate website structure',
'params': ['site_id']
},
{
'type': 'generate_page_content',
'name': 'Generate Page Content',
'description': 'Generate content for pages',
'params': ['page_ids']
},
{
'type': 'optimize_content',
'name': 'Optimize Content',
'description': 'Optimize content for SEO',
'params': ['content_ids']
},
{
'type': 'publish_to_wordpress',
'name': 'Publish to WordPress',
'description': 'Publish content to WordPress',
'params': ['content_ids', 'integration_id']
}
]
return success_response(
data=action_types,
request=request
)
@action(detail=False, methods=['get'], url_path='execution-history', url_name='execution-history')
def get_execution_history(self, request):
"""Get execution history for all rules"""
from igny8_core.business.automation.models import ScheduledTask
tasks = ScheduledTask.objects.filter(
automation_rule__account=request.account
).select_related('automation_rule').order_by('-executed_at')[:50]
serializer = ScheduledTaskSerializer(tasks, many=True)
return success_response(
data=serializer.data,
request=request
)
Estimated Time: 3 hours
Phase 2: Frontend Foundation (Week 2)
2.1 Create Automation Store
File: frontend/src/stores/automationStore.ts (NEW)
Implementation:
import { create } from 'zustand';
import { automationApi } from '../api/automation.api';
interface AutomationRule {
id: string;
name: string;
description: string;
trigger: 'schedule' | 'event' | 'manual';
schedule?: string;
conditions: any[];
actions: any[];
is_active: boolean;
status: 'active' | 'inactive' | 'paused';
last_executed_at?: string;
execution_count: number;
}
interface ScheduledTask {
id: string;
automation_rule: string;
scheduled_at: string;
executed_at?: string;
status: 'pending' | 'running' | 'completed' | 'failed' | 'cancelled';
result: any;
error_message?: string;
}
interface AutomationStore {
// State
rules: AutomationRule[];
tasks: ScheduledTask[];
actionTypes: any[];
loading: boolean;
error: string | null;
executionProgress: {
ruleId: string | null;
status: string;
progress: number;
phase: string;
} | null;
// Actions
fetchRules: () => Promise<void>;
fetchRule: (id: string) => Promise<AutomationRule>;
createRule: (rule: Partial<AutomationRule>) => Promise<AutomationRule>;
updateRule: (id: string, rule: Partial<AutomationRule>) => Promise<AutomationRule>;
deleteRule: (id: string) => Promise<void>;
executeRule: (id: string, context?: any) => Promise<void>;
fetchActionTypes: () => Promise<void>;
fetchExecutionHistory: () => Promise<void>;
fetchProgress: (ruleId: string) => Promise<void>;
clearError: () => void;
}
export const useAutomationStore = create<AutomationStore>((set, get) => ({
// Initial state
rules: [],
tasks: [],
actionTypes: [],
loading: false,
error: null,
executionProgress: null,
// Actions
fetchRules: async () => {
set({ loading: true, error: null });
try {
const response = await automationApi.getRules();
set({ rules: response.data, loading: false });
} catch (error: any) {
set({ error: error.message, loading: false });
}
},
fetchRule: async (id: string) => {
set({ loading: true, error: null });
try {
const response = await automationApi.getRule(id);
set({ loading: false });
return response.data;
} catch (error: any) {
set({ error: error.message, loading: false });
throw error;
}
},
createRule: async (rule: Partial<AutomationRule>) => {
set({ loading: true, error: null });
try {
const response = await automationApi.createRule(rule);
set(state => ({
rules: [...state.rules, response.data],
loading: false
}));
return response.data;
} catch (error: any) {
set({ error: error.message, loading: false });
throw error;
}
},
updateRule: async (id: string, rule: Partial<AutomationRule>) => {
set({ loading: true, error: null });
try {
const response = await automationApi.updateRule(id, rule);
set(state => ({
rules: state.rules.map(r => r.id === id ? response.data : r),
loading: false
}));
return response.data;
} catch (error: any) {
set({ error: error.message, loading: false });
throw error;
}
},
deleteRule: async (id: string) => {
set({ loading: true, error: null });
try {
await automationApi.deleteRule(id);
set(state => ({
rules: state.rules.filter(r => r.id !== id),
loading: false
}));
} catch (error: any) {
set({ error: error.message, loading: false });
throw error;
}
},
executeRule: async (id: string, context?: any) => {
set({ loading: true, error: null });
try {
await automationApi.executeRule(id, context);
set({ loading: false });
// Start polling for progress
const pollProgress = setInterval(async () => {
await get().fetchProgress(id);
const progress = get().executionProgress;
if (progress && (progress.status === 'SUCCESS' || progress.status === 'FAILURE')) {
clearInterval(pollProgress);
}
}, 2000);
} catch (error: any) {
set({ error: error.message, loading: false });
throw error;
}
},
fetchActionTypes: async () => {
try {
const response = await automationApi.getActionTypes();
set({ actionTypes: response.data });
} catch (error: any) {
set({ error: error.message });
}
},
fetchExecutionHistory: async () => {
set({ loading: true, error: null });
try {
const response = await automationApi.getExecutionHistory();
set({ tasks: response.data, loading: false });
} catch (error: any) {
set({ error: error.message, loading: false });
}
},
fetchProgress: async (ruleId: string) => {
try {
const response = await automationApi.getProgress(ruleId);
set({ executionProgress: { ruleId, ...response.data } });
} catch (error: any) {
console.error('Error fetching progress:', error);
}
},
clearError: () => set({ error: null })
}));
Estimated Time: 3 hours
2.2 Create Automation API Client
File: frontend/src/api/automation.api.ts (NEW)
Implementation:
import apiClient from './client';
export const automationApi = {
// Rules
getRules: () => apiClient.get('/automation/rules/'),
getRule: (id: string) => apiClient.get(`/automation/rules/${id}/`),
createRule: (data: any) => apiClient.post('/automation/rules/', data),
updateRule: (id: string, data: any) => apiClient.patch(`/automation/rules/${id}/`, data),
deleteRule: (id: string) => apiClient.delete(`/automation/rules/${id}/`),
executeRule: (id: string, context?: any) =>
apiClient.post(`/automation/rules/${id}/execute/`, { context }),
// Progress
getProgress: (ruleId: string) =>
apiClient.get(`/automation/rules/${ruleId}/progress/`),
// Action types
getActionTypes: () => apiClient.get('/automation/rules/action-types/'),
// Execution history
getExecutionHistory: () => apiClient.get('/automation/rules/execution-history/'),
// Scheduled tasks
getTasks: () => apiClient.get('/automation/scheduled-tasks/'),
getTask: (id: string) => apiClient.get(`/automation/scheduled-tasks/${id}/`)
};
Estimated Time: 1 hour
Phase 3: Frontend UI Components (Week 3)
3.1 Rule List Page
File: frontend/src/pages/automation/AutomationRulesPage.tsx (NEW)
Features:
- Table view of all automation rules
- Status indicators (active/inactive/paused)
- Last execution time and count
- Quick actions (execute, edit, delete)
- Create new rule button
Estimated Time: 6 hours
3.2 Rule Editor Modal
File: frontend/src/components/automation/RuleEditorModal.tsx (NEW)
Features:
- Form for rule name, description, trigger type
- Schedule configuration (cron expression builder)
- Condition builder (field, operator, value)
- Action builder (type selector, parameter inputs)
- Save/cancel buttons
Estimated Time: 8 hours
3.3 Workflow Visualizer
File: frontend/src/components/automation/WorkflowVisualizer.tsx (NEW)
Features:
- Visual flowchart of rule actions
- Drag-and-drop action reordering
- Action configuration panel
- Connection lines between actions
- Status indicators during execution
Estimated Time: 12 hours
3.4 Execution Monitor
File: frontend/src/components/automation/ExecutionMonitor.tsx (NEW)
Features:
- Real-time progress display
- Current phase indicator
- Progress percentage
- Action-by-action status
- Error display
- Cancel execution button
Estimated Time: 6 hours
3.5 Execution History
File: frontend/src/components/automation/ExecutionHistory.tsx (NEW)
Features:
- Table of past executions
- Status badges (completed/failed/cancelled)
- Execution time and duration
- Result summary
- Error details (if failed)
- Re-run button
Estimated Time: 4 hours
Phase 4: Integration & Testing (Week 4)
4.1 Backend Integration Tests
File: backend/igny8_core/business/automation/tests/test_automation.py (NEW)
Test Cases:
- Rule creation and validation
- Condition evaluation
- Action execution
- Progress tracking
- Error handling
- Credit checking
Estimated Time: 8 hours
4.2 Frontend Component Tests
File: frontend/src/components/automation/__tests__/ (NEW)
Test Cases:
- Rule list rendering
- Rule editor form validation
- Workflow visualizer interactions
- Execution monitor updates
- API integration
Estimated Time: 6 hours
4.3 End-to-End Testing
Manual Testing:
- Create automation rule
- Execute rule manually
- Monitor progress
- Verify results
- Test scheduled execution
- Test error scenarios
Estimated Time: 4 hours
4.4 Documentation
Files to Update:
- User guide for automation module
- API documentation
- Developer guide for adding new actions
Estimated Time: 4 hours
Implementation Timeline
Week 1: Backend Extensions
- Day 1-2: Extend ActionExecutor with all action types
- Day 3: Create Celery tasks
- Day 4: Add progress tracking
- Day 5: Add API endpoints and testing
Week 2: Frontend Foundation
- Day 1: Create automation store
- Day 2: Create API client
- Day 3: Setup routing and page structure
- Day 4-5: Build rule list page
Week 3: Frontend UI
- Day 1-2: Build rule editor modal
- Day 3-4: Build workflow visualizer
- Day 5: Build execution monitor and history
Week 4: Integration & Polish
- Day 1-2: Backend integration tests
- Day 3: Frontend component tests
- Day 4: End-to-end testing
- Day 5: Documentation and polish
Resource Requirements
Development Team
- 1 Backend Developer (Full-time, 4 weeks)
- 1 Frontend Developer (Full-time, 4 weeks)
- 1 QA Engineer (Part-time, Week 4)
Infrastructure
- Development environment
- Staging environment for testing
- Celery worker and beat scheduler
- Redis for task queue
Risk Assessment
High Risk
-
Workflow Visualizer Complexity
- Mitigation: Use existing React Flow library
- Fallback: Simpler list-based interface
-
Real-time Progress Tracking
- Mitigation: Implement polling with WebSocket fallback
- Fallback: Manual refresh button
Medium Risk
-
Action Execution Failures
- Mitigation: Comprehensive error handling and retry logic
- Fallback: Manual intervention and re-execution
-
Credit Estimation Accuracy
- Mitigation: Conservative estimates with buffer
- Fallback: Manual credit approval for large operations
Low Risk
- UI/UX Complexity
- Mitigation: Iterative design with user feedback
- Fallback: Simplified interface
Success Criteria
Functional Requirements
- ✅ Users can create automation rules with conditions and actions
- ✅ Rules can be executed manually or on schedule
- ✅ Real-time progress tracking during execution
- ✅ Execution history with success/failure status
- ✅ All AI functions available as actions
- ✅ WordPress publishing integration
Performance Requirements
- ✅ Rule execution starts within 2 seconds
- ✅ Progress updates every 2 seconds
- ✅ UI remains responsive during execution
- ✅ Support for 100+ concurrent rule executions
Quality Requirements
- ✅ 80%+ test coverage
- ✅ Zero critical bugs
- ✅ Comprehensive documentation
- ✅ User acceptance testing passed
Post-Implementation
Monitoring
- Track rule execution success rate
- Monitor execution duration
- Track credit consumption
- Monitor error rates
Optimization
- Optimize slow-running actions
- Improve credit estimation accuracy
- Enhance error messages
- Add more action types based on usage
Future Enhancements
- Conditional branching (if/else logic)
- Parallel action execution
- Action templates and presets
- Rule sharing and marketplace
- Advanced scheduling (multiple schedules per rule)
- Webhook triggers for external events
Appendix
Action Type Reference
| Action Type | Description | Parameters | Credits |
|---|---|---|---|
| cluster_keywords | Cluster keywords by topic | keyword_ids, sector_id | 10 |
| generate_ideas | Generate content ideas | cluster_ids | 15 |
| generate_content | Generate article content | task_ids | 50 |
| generate_images | Generate images | content_id, provider | 1-4 |
| generate_site_structure | Generate site structure | site_id | 20 |
| generate_page_content | Generate page content | page_ids | 30 |
| optimize_content | Optimize for SEO | content_ids | 10 |
| publish_to_wordpress | Publish to WordPress | content_ids, integration_id | 5 |
Condition Operators
| Operator | Description | Example |
|---|---|---|
| equals | Exact match | status equals "draft" |
| not_equals | Not equal | status not_equals "published" |
| contains | String contains | title contains "guide" |
| greater_than | Numeric > | word_count greater_than 1000 |
| less_than | Numeric < | word_count less_than 5000 |
| in | Value in list | status in ["draft", "pending"] |
| not_in | Value not in list | status not_in ["archived"] |
API Endpoint Summary
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/automation/rules/ | List all rules |
| POST | /api/v1/automation/rules/ | Create new rule |
| GET | /api/v1/automation/rules/{id}/ | Get rule details |
| PUT | /api/v1/automation/rules/{id}/ | Update rule |
| DELETE | /api/v1/automation/rules/{id}/ | Delete rule |
| POST | /api/v1/automation/rules/{id}/execute/ | Execute rule |
| GET | /api/v1/automation/rules/{id}/progress/ | Get execution progress |
| GET | /api/v1/automation/rules/action-types/ | Get available actions |
| GET | /api/v1/automation/rules/execution-history/ | Get execution history |
| GET | /api/v1/automation/scheduled-tasks/ | List scheduled tasks |
Document Version: 1.0
Created: 2024-11-30
Author: IGNY8 Development Team
Status: Ready for Implementation