9 phases

2025-11-16 23:15:45 +05:00
parent 1c8c44ebe0
commit f84be4194f
10 changed files with 3514 additions and 0 deletions
--- a/docs/planning/phases/PHASE-2-AUTOMATION-SYSTEM.md
+++ b/docs/planning/phases/PHASE-2-AUTOMATION-SYSTEM.md
@@ -0,0 +1,596 @@
+# PHASE 2: AUTOMATION SYSTEM
+**Detailed Implementation Plan**
+
+**Goal**: Implement automation rules and scheduled tasks.
+
+**Timeline**: 2-3 weeks  
+**Priority**: HIGH  
+**Dependencies**: Phase 1
+
+---
+
+## TABLE OF CONTENTS
+
+1. [Overview](#overview)
+2. [Automation Models](#automation-models)
+3. [Automation Service](#automation-service)
+4. [Celery Beat Tasks](#celery-beat-tasks)
+5. [Automation API](#automation-api)
+6. [Automation UI](#automation-ui)
+7. [Testing & Validation](#testing--validation)
+8. [Implementation Checklist](#implementation-checklist)
+
+---
+
+## OVERVIEW
+
+### Objectives
+- ✅ Create AutomationRule and ScheduledTask models
+- ✅ Build AutomationService with rule execution engine
+- ✅ Implement Celery Beat scheduled tasks
+- ✅ Create automation API endpoints
+- ✅ Build automation UI (Dashboard, Rules, History)
+
+### Key Principles
+- **Rule-Based**: Users create rules with triggers, conditions, actions
+- **Scheduled Execution**: Rules can run on schedule or event triggers
+- **Credit-Aware**: Automation respects credit limits
+- **Audit Trail**: All automation executions logged
+
+---
+
+## AUTOMATION MODELS
+
+### 2.1 Automation Models
+
+**Purpose**: Store automation rules and scheduled task records.
+
+#### AutomationRule Model
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **AutomationRule Model** | `domain/automation/models.py` | Phase 1 | Create model with trigger, conditions, actions, schedule |
+
+**AutomationRule Model**:
+```python
+# domain/automation/models.py
+class AutomationRule(SiteSectorBaseModel):
+    name = models.CharField(max_length=255)
+    description = models.TextField(blank=True)
+    
+    # Trigger configuration
+    trigger = models.CharField(
+        max_length=50,
+        choices=[
+            ('schedule', 'Scheduled'),
+            ('keyword_added', 'Keyword Added'),
+            ('cluster_created', 'Cluster Created'),
+            ('idea_created', 'Idea Created'),
+            ('content_generated', 'Content Generated'),
+            ('task_created', 'Task Created'),
+        ]
+    )
+    
+    # Condition evaluation
+    conditions = models.JSONField(default=dict)
+    # Example: {'field': 'status', 'operator': 'eq', 'value': 'draft'}
+    
+    # Actions to execute
+    actions = models.JSONField(default=list)
+    # Example: [{'type': 'generate_ideas', 'params': {'cluster_ids': [1, 2]}}]
+    
+    # Schedule configuration (for scheduled triggers)
+    schedule = models.JSONField(default=dict)
+    # Example: {'cron': '0 9 * * *', 'timezone': 'UTC'}
+    
+    # Execution limits
+    is_active = models.BooleanField(default=True)
+    max_executions_per_day = models.IntegerField(default=10)
+    credit_limit_per_execution = models.IntegerField(default=100)
+    
+    # Tracking
+    last_executed_at = models.DateTimeField(null=True, blank=True)
+    execution_count_today = models.IntegerField(default=0)
+    last_reset_at = models.DateTimeField(auto_now_add=True)
+    
+    class Meta:
+        ordering = ['-created_at']
+```
+
+#### ScheduledTask Model
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **ScheduledTask Model** | `domain/automation/models.py` | Phase 1 | Create model to track scheduled executions |
+
+**ScheduledTask Model**:
+```python
+# domain/automation/models.py
+class ScheduledTask(SiteSectorBaseModel):
+    automation_rule = models.ForeignKey(AutomationRule, on_delete=models.CASCADE)
+    scheduled_at = models.DateTimeField()
+    executed_at = models.DateTimeField(null=True, blank=True)
+    status = models.CharField(
+        max_length=20,
+        choices=[
+            ('pending', 'Pending'),
+            ('running', 'Running'),
+            ('completed', 'Completed'),
+            ('failed', 'Failed'),
+            ('skipped', 'Skipped'),
+        ],
+        default='pending'
+    )
+    result = models.JSONField(default=dict, blank=True)
+    error_message = models.TextField(blank=True)
+    credits_used = models.IntegerField(default=0)
+    
+    class Meta:
+        ordering = ['-scheduled_at']
+```
+
+#### Automation Migrations
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Automation Migrations** | `domain/automation/migrations/` | Phase 1 | Create initial migrations |
+
+---
+
+## AUTOMATION SERVICE
+
+### 2.2 Automation Service
+
+**Purpose**: Execute automation rules with condition evaluation and action execution.
+
+#### AutomationService
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **AutomationService** | `domain/automation/services/automation_service.py` | Phase 1 services | Main service for rule execution |
+
+**AutomationService Methods**:
+```python
+# domain/automation/services/automation_service.py
+class AutomationService:
+    def __init__(self):
+        self.rule_engine = RuleEngine()
+        self.condition_evaluator = ConditionEvaluator()
+        self.action_executor = ActionExecutor()
+        self.credit_service = CreditService()
+    
+    def execute_rule(self, rule, context=None):
+        """Execute an automation rule"""
+        # Check if rule is active
+        if not rule.is_active:
+            return {'status': 'skipped', 'reason': 'Rule is inactive'}
+        
+        # Check execution limits
+        if not self._check_execution_limits(rule):
+            return {'status': 'skipped', 'reason': 'Execution limit reached'}
+        
+        # Check credits
+        if not self.credit_service.check_credits(rule.account, 'automation', rule.credit_limit_per_execution):
+            return {'status': 'skipped', 'reason': 'Insufficient credits'}
+        
+        # Evaluate conditions
+        if not self.condition_evaluator.evaluate(rule.conditions, context):
+            return {'status': 'skipped', 'reason': 'Conditions not met'}
+        
+        # Execute actions
+        results = self.action_executor.execute(rule.actions, context)
+        
+        # Update rule tracking
+        rule.last_executed_at = timezone.now()
+        rule.execution_count_today += 1
+        rule.save()
+        
+        return {'status': 'completed', 'results': results}
+    
+    def _check_execution_limits(self, rule):
+        """Check if rule can execute (daily limit)"""
+        # Reset counter if new day
+        if rule.last_reset_at.date() < timezone.now().date():
+            rule.execution_count_today = 0
+            rule.last_reset_at = timezone.now()
+            rule.save()
+        
+        return rule.execution_count_today < rule.max_executions_per_day
+```
+
+#### Rule Execution Engine
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Rule Execution Engine** | `domain/automation/services/rule_engine.py` | Phase 1 services | Orchestrates rule execution |
+
+**RuleEngine Methods**:
+```python
+# domain/automation/services/rule_engine.py
+class RuleEngine:
+    def execute_rule(self, rule, context):
+        """Orchestrate rule execution"""
+        # Validate rule
+        # Check conditions
+        # Execute actions
+        # Handle errors
+        pass
+```
+
+#### Condition Evaluator
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Condition Evaluator** | `domain/automation/services/condition_evaluator.py` | None | Evaluates rule conditions |
+
+**ConditionEvaluator Methods**:
+```python
+# domain/automation/services/condition_evaluator.py
+class ConditionEvaluator:
+    def evaluate(self, conditions, context):
+        """Evaluate rule conditions"""
+        # Support operators: eq, ne, gt, gte, lt, lte, in, contains
+        # Example: {'field': 'status', 'operator': 'eq', 'value': 'draft'}
+        pass
+```
+
+#### Action Executor
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Action Executor** | `domain/automation/services/action_executor.py` | Phase 1 services | Executes rule actions |
+
+**ActionExecutor Methods**:
+```python
+# domain/automation/services/action_executor.py
+class ActionExecutor:
+    def __init__(self):
+        self.clustering_service = ClusteringService()
+        self.ideas_service = IdeasService()
+        self.content_service = ContentGenerationService()
+    
+    def execute(self, actions, context):
+        """Execute rule actions"""
+        results = []
+        for action in actions:
+            action_type = action['type']
+            params = action.get('params', {})
+            
+            if action_type == 'generate_ideas':
+                result = self.ideas_service.generate_ideas(params['cluster_ids'], context['account'])
+            elif action_type == 'generate_content':
+                result = self.content_service.generate_content(params['task_id'], context['account'])
+            # ... other action types
+            
+            results.append(result)
+        
+        return results
+```
+
+---
+
+## CELERY BEAT TASKS
+
+### 2.3 Celery Beat Tasks
+
+**Purpose**: Schedule automation rules and monthly credit replenishment.
+
+#### Scheduled Automation Task
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Scheduled Automation Task** | `infrastructure/messaging/automation_tasks.py` | AutomationService | Periodic task to execute scheduled rules |
+
+**Scheduled Automation Task**:
+```python
+# infrastructure/messaging/automation_tasks.py
+from celery import shared_task
+from celery.schedules import crontab
+
+@shared_task
+def execute_scheduled_automation_rules():
+    """Execute all scheduled automation rules"""
+    from domain.automation.services.automation_service import AutomationService
+    
+    service = AutomationService()
+    rules = AutomationRule.objects.filter(
+        trigger='schedule',
+        is_active=True
+    )
+    
+    for rule in rules:
+        # Check if rule should execute based on schedule
+        if should_execute_now(rule.schedule):
+            service.execute_rule(rule)
+```
+
+#### Monthly Credit Replenishment
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Monthly Credit Replenishment** | `infrastructure/messaging/automation_tasks.py` | CreditService | Add credits monthly to accounts |
+
+**Monthly Credit Replenishment Task**:
+```python
+# infrastructure/messaging/automation_tasks.py
+@shared_task
+def replenish_monthly_credits():
+    """Replenish monthly credits for all active accounts"""
+    from domain.billing.services.credit_service import CreditService
+    
+    service = CreditService()
+    accounts = Account.objects.filter(status='active')
+    
+    for account in accounts:
+        if account.plan:
+            monthly_credits = account.plan.monthly_credits
+            if monthly_credits > 0:
+                service.add_credits(account, monthly_credits, 'monthly_replenishment')
+```
+
+#### Celery Beat Configuration
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Celery Beat Configuration** | `backend/igny8_core/celery.py` | None | Configure periodic tasks |
+
+**Celery Beat Configuration**:
+```python
+# backend/igny8_core/celery.py
+from celery.schedules import crontab
+
+app.conf.beat_schedule = {
+    'execute-scheduled-automation-rules': {
+        'task': 'infrastructure.messaging.automation_tasks.execute_scheduled_automation_rules',
+        'schedule': crontab(minute='*/15'),  # Every 15 minutes
+    },
+    'replenish-monthly-credits': {
+        'task': 'infrastructure.messaging.automation_tasks.replenish_monthly_credits',
+        'schedule': crontab(hour=0, minute=0, day_of_month=1),  # First day of month
+    },
+}
+```
+
+---
+
+## AUTOMATION API
+
+### 2.4 Automation API
+
+**Purpose**: CRUD API for automation rules and scheduled tasks.
+
+#### AutomationRule ViewSet
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **AutomationRule ViewSet** | `modules/automation/views.py` | AutomationService | CRUD operations for rules |
+
+**AutomationRule ViewSet**:
+```python
+# modules/automation/views.py
+class AutomationRuleViewSet(AccountModelViewSet):
+    queryset = AutomationRule.objects.all()
+    serializer_class = AutomationRuleSerializer
+    
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.automation_service = AutomationService()
+    
+    @action(detail=True, methods=['post'])
+    def execute(self, request, pk=None):
+        """Manually execute a rule"""
+        rule = self.get_object()
+        result = self.automation_service.execute_rule(rule, {'account': request.account})
+        return Response(result)
+    
+    @action(detail=True, methods=['post'])
+    def test(self, request, pk=None):
+        """Test rule conditions without executing"""
+        rule = self.get_object()
+        # Test condition evaluation
+        pass
+```
+
+#### ScheduledTask ViewSet
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **ScheduledTask ViewSet** | `modules/automation/views.py` | AutomationService | View scheduled task history |
+
+**ScheduledTask ViewSet**:
+```python
+# modules/automation/views.py
+class ScheduledTaskViewSet(AccountModelViewSet):
+    queryset = ScheduledTask.objects.all()
+    serializer_class = ScheduledTaskSerializer
+    filterset_fields = ['status', 'automation_rule']
+    ordering = ['-scheduled_at']
+```
+
+#### Automation URLs
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Automation URLs** | `modules/automation/urls.py` | None | Register automation routes |
+
+**Automation URLs**:
+```python
+# modules/automation/urls.py
+from rest_framework.routers import DefaultRouter
+from .views import AutomationRuleViewSet, ScheduledTaskViewSet
+
+router = DefaultRouter()
+router.register(r'rules', AutomationRuleViewSet, basename='automation-rule')
+router.register(r'scheduled-tasks', ScheduledTaskViewSet, basename='scheduled-task')
+
+urlpatterns = router.urls
+```
+
+---
+
+## AUTOMATION UI
+
+### 2.5 Automation UI
+
+**Purpose**: User interface for managing automation rules and viewing history.
+
+#### Automation Dashboard
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Automation Dashboard** | `frontend/src/pages/Automation/Dashboard.tsx` | EXISTING (placeholder) | Overview of automation status |
+
+**Dashboard Features**:
+- Active rules count
+- Recent executions
+- Success/failure rates
+- Credit usage from automation
+- Quick actions (create rule, view history)
+
+#### Rules Management
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Rules Management** | `frontend/src/pages/Automation/Rules.tsx` | NEW | CRUD interface for rules |
+
+**Rules Management Features**:
+- List all rules
+- Create new rule (wizard)
+- Edit existing rule
+- Enable/disable rule
+- Delete rule
+- Test rule
+- Manual execution
+
+#### Schedules Page
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Schedules Page** | `frontend/src/pages/Schedules.tsx` | EXISTING (placeholder) | View scheduled task history |
+
+**Schedules Page Features**:
+- List scheduled tasks
+- Filter by status, rule, date
+- View execution results
+- View error messages
+- Retry failed tasks
+
+#### Automation API Client
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Automation API Client** | `frontend/src/services/automation.api.ts` | NEW | API client for automation endpoints |
+
+**Automation API Client**:
+```typescript
+// frontend/src/services/automation.api.ts
+export const automationApi = {
+  getRules: () => fetchAPI('/automation/rules/'),
+  createRule: (data) => fetchAPI('/automation/rules/', { method: 'POST', body: data }),
+  updateRule: (id, data) => fetchAPI(`/automation/rules/${id}/`, { method: 'PUT', body: data }),
+  deleteRule: (id) => fetchAPI(`/automation/rules/${id}/`, { method: 'DELETE' }),
+  executeRule: (id) => fetchAPI(`/automation/rules/${id}/execute/`, { method: 'POST' }),
+  getScheduledTasks: (filters) => fetchAPI('/automation/scheduled-tasks/', { params: filters }),
+};
+```
+
+---
+
+## TESTING & VALIDATION
+
+### 2.6 Testing
+
+**Test Cases**:
+
+1. **Automation Service Tests**:
+   - ✅ Rules execute correctly
+   - ✅ Conditions evaluate correctly
+   - ✅ Actions execute correctly
+   - ✅ Execution limits enforced
+   - ✅ Credit checks work
+
+2. **Scheduled Tasks Tests**:
+   - ✅ Scheduled tasks run on time
+   - ✅ Credit replenishment works monthly
+   - ✅ Task status tracking works
+
+3. **API Tests**:
+   - ✅ CRUD operations work
+   - ✅ Rule execution endpoint works
+   - ✅ Scheduled task history works
+
+4. **UI Tests**:
+   - ✅ Dashboard displays correctly
+   - ✅ Rules management works
+   - ✅ Schedule history displays correctly
+
+---
+
+## IMPLEMENTATION CHECKLIST
+
+### Backend Tasks
+
+- [ ] Create `domain/automation/models.py`
+- [ ] Create AutomationRule model
+- [ ] Create ScheduledTask model
+- [ ] Create automation migrations
+- [ ] Create `domain/automation/services/automation_service.py`
+- [ ] Create `domain/automation/services/rule_engine.py`
+- [ ] Create `domain/automation/services/condition_evaluator.py`
+- [ ] Create `domain/automation/services/action_executor.py`
+- [ ] Create `infrastructure/messaging/automation_tasks.py`
+- [ ] Add scheduled automation task
+- [ ] Add monthly credit replenishment task
+- [ ] Configure Celery Beat
+- [ ] Create `modules/automation/views.py`
+- [ ] Create AutomationRule ViewSet
+- [ ] Create ScheduledTask ViewSet
+- [ ] Create `modules/automation/serializers.py`
+- [ ] Create `modules/automation/urls.py`
+- [ ] Register automation URLs in main urls.py
+
+### Frontend Tasks
+
+- [ ] Implement `frontend/src/pages/Automation/Dashboard.tsx`
+- [ ] Create `frontend/src/pages/Automation/Rules.tsx`
+- [ ] Implement `frontend/src/pages/Schedules.tsx`
+- [ ] Create `frontend/src/services/automation.api.ts`
+- [ ] Create rule creation wizard
+- [ ] Create rule editor
+- [ ] Create schedule history table
+
+### Testing Tasks
+
+- [ ] Test automation rule execution
+- [ ] Test scheduled tasks
+- [ ] Test credit replenishment
+- [ ] Test API endpoints
+- [ ] Test UI components
+
+---
+
+## RISK ASSESSMENT
+
+| Risk | Level | Mitigation |
+|------|-------|------------|
+| **Rule execution errors** | MEDIUM | Comprehensive error handling, logging |
+| **Credit limit violations** | MEDIUM | Credit checks before execution |
+| **Scheduled task failures** | MEDIUM | Retry mechanism, error logging |
+| **Performance issues** | LOW | Background processing, rate limiting |
+
+---
+
+## SUCCESS CRITERIA
+
+- ✅ Automation rules execute correctly
+- ✅ Scheduled tasks run on time
+- ✅ Credit replenishment works monthly
+- ✅ UI shows automation status
+- ✅ Rules can be created, edited, deleted
+- ✅ Execution history is tracked
+- ✅ All automation respects credit limits
+
+---
+
+**END OF PHASE 2 DOCUMENT**
+