Cleanup

AI_FILES_ANALYSIS.md

@@ -0,0 +1,136 @@
+# AI Folder Files Analysis
+## Which Files Are Actually Needed for 3 Active Functions
+
+### ✅ **REQUIRED FILES** (All needed for active functions)
+
+#### 1. **tasks.py** - ✅ REQUIRED
+- **Purpose:** Unified Celery task entrypoint
+- **Used by:** All views (planner/views.py, writer/views.py)
+- **Dependencies:** engine.py, registry.py
+- **Status:** KEEP
+
+#### 2. **engine.py** - ✅ REQUIRED
+- **Purpose:** Central orchestrator for all AI functions
+- **Used by:** tasks.py
+- **Dependencies:** base.py, tracker.py, ai_core.py, settings.py, models.py
+- **Status:** KEEP
+
+#### 3. **base.py** - ✅ REQUIRED
+- **Purpose:** Abstract base class for all AI functions
+- **Used by:** All function classes (AutoClusterFunction, GenerateIdeasFunction, GenerateContentFunction)
+- **Dependencies:** None
+- **Status:** KEEP
+
+#### 4. **registry.py** - ✅ REQUIRED
+- **Purpose:** Function registry with lazy loading
+- **Used by:** tasks.py
+- **Dependencies:** base.py, function modules
+- **Status:** KEEP
+
+#### 5. **ai_core.py** - ✅ REQUIRED
+- **Purpose:** Centralized AI request handler
+- **Used by:** engine.py, all function classes
+- **Dependencies:** constants.py, tracker.py
+- **Status:** KEEP
+
+#### 6. **prompts.py** - ✅ REQUIRED
+- **Purpose:** Centralized prompt management
+- **Used by:** All function classes (build_prompt methods)
+- **Dependencies:** None
+- **Status:** KEEP
+
+#### 7. **settings.py** - ✅ REQUIRED
+- **Purpose:** Model configurations per function
+- **Used by:** engine.py
+- **Dependencies:** constants.py (via ai_processor import)
+- **Status:** KEEP
+
+#### 8. **validators.py** - ✅ REQUIRED
+- **Purpose:** Validation functions
+- **Used by:** All function classes (validate methods)
+- **Dependencies:** constants.py, models
+- **Status:** KEEP
+
+#### 9. **tracker.py** - ✅ REQUIRED
+- **Purpose:** Progress tracking utilities
+- **Used by:** engine.py, ai_core.py
+- **Dependencies:** types.py (imported but NOT USED), constants.py
+- **Status:** KEEP (but can remove types.py import)
+
+#### 10. **constants.py** - ✅ REQUIRED
+- **Purpose:** AI constants (model rates, valid models)
+- **Used by:** ai_core.py, settings.py, validators.py
+- **Dependencies:** None
+- **Status:** KEEP
+
+#### 11. **models.py** - ✅ REQUIRED
+- **Purpose:** AITaskLog database model
+- **Used by:** engine.py (for logging), admin.py
+- **Dependencies:** AccountBaseModel
+- **Status:** KEEP
+
+#### 12. **apps.py** - ✅ REQUIRED
+- **Purpose:** Django app configuration
+- **Used by:** Django (app registration)
+- **Dependencies:** admin.py
+- **Status:** KEEP
+
+#### 13. **admin.py** - ✅ REQUIRED (Optional but recommended)
+- **Purpose:** Django admin interface for AITaskLog
+- **Used by:** apps.py (auto-imported)
+- **Dependencies:** models.py
+- **Status:** KEEP (useful for debugging/admin)
+
+---
+
+### ❌ **UNUSED FILES** (Can be removed)
+
+#### 1. **types.py** - ❌ NOT USED
+- **Purpose:** Type definitions (StepLog, ProgressState, AITaskResult dataclasses)
+- **Used by:** tracker.py (imported but never instantiated)
+- **Actual usage:** None - tracker.py uses plain Dict types instead
+- **Status:** **SAFE TO REMOVE**
+- **Action:** Remove file and remove import from tracker.py
+
+---
+
+## Summary
+
+### Files to KEEP (13 files):
+1. ✅ tasks.py
+2. ✅ engine.py
+3. ✅ base.py
+4. ✅ registry.py
+5. ✅ ai_core.py
+6. ✅ prompts.py
+7. ✅ settings.py
+8. ✅ validators.py
+9. ✅ tracker.py
+10. ✅ constants.py
+11. ✅ models.py
+12. ✅ apps.py
+13. ✅ admin.py
+
+### Files to REMOVE (1 file):
+1. ❌ **types.py** - Imported but never used
+
+---
+
+## Action Items
+
+1. **Remove types.py** - File is not used
+2. **Remove import from tracker.py** - Remove `from igny8_core.ai.types import StepLog, ProgressState` (line 8)
+
+---
+
+## Verification
+
+All other files are actively used in the execution flow:
+- **tasks.py** → **registry.py** → **engine.py** → **base.py** (function classes)
+- **engine.py** → **ai_core.py** → **constants.py**
+- **engine.py** → **tracker.py** → **constants.py**
+- **engine.py** → **settings.py** → **constants.py**
+- **engine.py** → **models.py** (AITaskLog)
+- **function classes** → **prompts.py**, **validators.py**
+- **apps.py** → **admin.py** → **models.py**
+

backend/igny8_core/ai/functions/__init__.py

@@ -2,16 +2,14 @@
 AI Function implementations
 """
 from igny8_core.ai.functions.auto_cluster import AutoClusterFunction
-from igny8_core.ai.functions.generate_ideas import GenerateIdeasFunction, generate_ideas_core
+from igny8_core.ai.functions.generate_ideas import GenerateIdeasFunction
-from igny8_core.ai.functions.generate_content import GenerateContentFunction, generate_content_core
+from igny8_core.ai.functions.generate_content import GenerateContentFunction
 from igny8_core.ai.functions.generate_images import GenerateImagesFunction, generate_images_core
 
 __all__ = [
     'AutoClusterFunction',
     'GenerateIdeasFunction',
-    'generate_ideas_core',
     'GenerateContentFunction',
-    'generate_content_core',
     'GenerateImagesFunction',
     'generate_images_core',
 ]

backend/igny8_core/ai/functions/generate_content.py

@@ -300,88 +300,3 @@ class GenerateContentFunction(BaseAIFunction):
         }
 
 
-def generate_content_core(task_ids: List[int], account_id: int = None, progress_callback=None):
-    """
-    Core logic for generating content (legacy function signature for backward compatibility).
-    Can be called with or without Celery.
-    
-    Args:
-        task_ids: List of task IDs
-        account_id: Account ID for account isolation
-        progress_callback: Optional function to call for progress updates
-    
-    Returns:
-        Dict with 'success', 'tasks_updated', 'message', etc.
-    """
-    try:
-        from igny8_core.auth.models import Account
-        
-        account = None
-        if account_id:
-            account = Account.objects.get(id=account_id)
-        
-        # Use the new function class
-        fn = GenerateContentFunction()
-        fn.account = account
-        
-        # Prepare payload
-        payload = {'ids': task_ids}
-        
-        # Validate
-        validated = fn.validate(payload, account)
-        if not validated['valid']:
-            return {'success': False, 'error': validated['error']}
-        
-        # Prepare data
-        tasks = fn.prepare(payload, account)
-        
-        tasks_updated = 0
-        
-        # Process each task
-        for task in tasks:
-            # Build prompt for this task
-            prompt = fn.build_prompt([task], account)
-            
-            # Get model config from settings
-            model_config = get_model_config('generate_content')
-            
-            # Generate function_id for tracking (ai-generate-content-02 for legacy path)
-            function_id = "ai-generate-content-02"
-            
-            # Call AI using centralized request handler
-            ai_core = AICore(account=account)
-            result = ai_core.run_ai_request(
-                prompt=prompt,
-                model=model_config.get('model'),
-                max_tokens=model_config.get('max_tokens'),
-                temperature=model_config.get('temperature'),
-                response_format=model_config.get('response_format'),
-                function_name='generate_content',
-                function_id=function_id  # Pass function_id for tracking
-            )
-            
-            if result.get('error'):
-                logger.error(f"AI error for task {task.id}: {result['error']}")
-                continue
-            
-            # Parse response
-            content = fn.parse_response(result['content'])
-            
-            if not content:
-                logger.warning(f"No content generated for task {task.id}")
-                continue
-            
-            # Save output
-            save_result = fn.save_output(content, [task], account)
-            tasks_updated += save_result.get('tasks_updated', 0)
-        
-        return {
-            'success': True,
-            'tasks_updated': tasks_updated,
-            'message': f'Content generation complete: {tasks_updated} articles generated'
-        }
-        
-    except Exception as e:
-        logger.error(f"Error in generate_content_core: {str(e)}", exc_info=True)
-        return {'success': False, 'error': str(e)}
-

backend/igny8_core/ai/functions/generate_ideas.py

@@ -10,7 +10,6 @@ from igny8_core.ai.base import BaseAIFunction
 from igny8_core.modules.planner.models import Clusters, ContentIdeas
 from igny8_core.ai.ai_core import AICore
 from igny8_core.ai.validators import validate_cluster_exists, validate_cluster_limits
-from igny8_core.ai.tracker import ConsoleStepTracker
 from igny8_core.ai.prompts import PromptRegistry
 from igny8_core.ai.settings import get_model_config
 
@@ -231,104 +230,3 @@ class GenerateIdeasFunction(BaseAIFunction):
         }
 
 
-def generate_ideas_core(cluster_id: int, account_id: int = None, progress_callback=None):
-    """
-    Core logic for generating ideas (legacy function signature for backward compatibility).
-    Can be called with or without Celery.
-    
-    Args:
-        cluster_id: Cluster ID to generate idea for
-        account_id: Account ID for account isolation
-        progress_callback: Optional function to call for progress updates
-    
-    Returns:
-        Dict with 'success', 'idea_created', 'message', etc.
-    """
-    tracker = ConsoleStepTracker('generate_ideas')
-    tracker.init("Task started")
-    
-    try:
-        from igny8_core.auth.models import Account
-        
-        account = None
-        if account_id:
-            account = Account.objects.get(id=account_id)
-        
-        tracker.prep("Loading account and cluster data...")
-        
-        # Use the new function class
-        fn = GenerateIdeasFunction()
-        # Store account for use in methods
-        fn.account = account
-        
-        # Prepare payload
-        payload = {'ids': [cluster_id]}
-        
-        # Validate
-        tracker.prep("Validating input...")
-        validated = fn.validate(payload, account)
-        if not validated['valid']:
-            tracker.error('ValidationError', validated['error'])
-            return {'success': False, 'error': validated['error']}
-        
-        # Prepare data
-        tracker.prep("Loading cluster with keywords...")
-        data = fn.prepare(payload, account)
-        
-        # Build prompt
-        tracker.prep("Building prompt...")
-        prompt = fn.build_prompt(data, account)
-        
-        # Get model config from settings
-        model_config = get_model_config('generate_ideas')
-        
-        # Generate function_id for tracking (ai-generate-ideas-02 for legacy path)
-        function_id = "ai-generate-ideas-02-desktop"
-        
-        # Call AI using centralized request handler
-        ai_core = AICore(account=account)
-        result = ai_core.run_ai_request(
-            prompt=prompt,
-            model=model_config.get('model'),
-            max_tokens=model_config.get('max_tokens'),
-            temperature=model_config.get('temperature'),
-            response_format=model_config.get('response_format'),
-            function_name='generate_ideas',
-            function_id=function_id,  # Pass function_id for tracking
-            tracker=tracker
-        )
-        
-        if result.get('error'):
-            return {'success': False, 'error': result['error']}
-        
-        # Parse response
-        tracker.parse("Parsing AI response...")
-        ideas_data = fn.parse_response(result['content'])
-        
-        if not ideas_data:
-            tracker.error('ParseError', 'No ideas generated by AI')
-            return {'success': False, 'error': 'No ideas generated by AI'}
-        
-        tracker.parse(f"Parsed {len(ideas_data)} idea(s)")
-        
-        # Take first idea
-        idea_data = ideas_data[0]
-        
-        # Save output
-        tracker.save("Saving idea to database...")
-        save_result = fn.save_output(ideas_data, data, account)
-        tracker.save(f"Saved {save_result['ideas_created']} idea(s)")
-        
-        tracker.done(f"Idea '{idea_data.get('title', 'Untitled')}' created successfully")
-        
-        return {
-            'success': True,
-            'idea_created': save_result['ideas_created'],
-            'message': f"Idea '{idea_data.get('title', 'Untitled')}' created"
-        }
-        
-    except Exception as e:
-        tracker.error('Exception', str(e), e)
-        logger.error(f"Error in generate_ideas_core: {str(e)}", exc_info=True)
-        return {'success': False, 'error': str(e)}
-

backend/igny8_core/ai/tests/test_run.py

@@ -12,9 +12,6 @@ os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'igny8.settings')
 django.setup()
 
 from igny8_core.ai.functions.auto_cluster import AutoClusterFunction
-# REMOVED: generate_ideas function removed
-# from igny8_core.ai.functions.generate_ideas import generate_ideas_core
-from igny8_core.ai.functions.generate_content import generate_content_core
 from igny8_core.ai.functions.generate_images import generate_images_core
 from igny8_core.ai.ai_core import AICore
 
@@ -52,34 +49,19 @@ def test_auto_cluster():
     # print(f"Validation result: {result}")
 
 
-def test_generate_ideas():
-    """Test generate ideas function"""
-    print("\n" + "="*80)
-    print("TEST 3: Generate Ideas Function")
-    print("="*80)
-    print("Note: This requires actual cluster ID in the database")
-    print("Skipping - requires database setup")
-    # Uncomment to test with real data:
-    # result = generate_ideas_core(cluster_id=1, account_id=1)
-    # print(f"Result: {result}")
-
-
 def test_generate_content():
     """Test generate content function"""
     print("\n" + "="*80)
-    print("TEST 4: Generate Content Function")
+    print("TEST 3: Generate Content Function")
     print("="*80)
     print("Note: This requires actual task IDs in the database")
     print("Skipping - requires database setup")
-    # Uncomment to test with real data:
-    # result = generate_content_core(task_ids=[1], account_id=1)
-    # print(f"Result: {result}")
 
 
 def test_generate_images():
     """Test generate images function"""
     print("\n" + "="*80)
-    print("TEST 5: Generate Images Function")
+    print("TEST 4: Generate Images Function")
     print("="*80)
     print("Note: This requires actual task IDs in the database")
     print("Skipping - requires database setup")
@@ -91,7 +73,7 @@ def test_generate_images():
 def test_json_extraction():
     """Test JSON extraction"""
     print("\n" + "="*80)
-    print("TEST 6: JSON Extraction")
+    print("TEST 5: JSON Extraction")
     print("="*80)
     
     ai_core = AICore()
@@ -123,8 +105,6 @@ if __name__ == '__main__':
     test_ai_core()
     test_json_extraction()
     test_auto_cluster()
-    # REMOVED: generate_ideas function removed
-    # test_generate_ideas()
     test_generate_content()
     test_generate_images()
     

backend/igny8_core/ai/tracker.py

@@ -5,7 +5,6 @@ import time
 import logging
 from typing import List, Dict, Any, Optional, Callable
 from datetime import datetime
-from igny8_core.ai.types import StepLog, ProgressState
 from igny8_core.ai.constants import DEBUG_MODE
 
 logger = logging.getLogger(__name__)

backend/igny8_core/ai/types.py

@@ -1,44 +0,0 @@
-"""
-Shared types and dataclasses for AI framework
-"""
-from dataclasses import dataclass
-from typing import Dict, List, Any, Optional
-from datetime import datetime
-
-
-@dataclass
-class StepLog:
-    """Single step in request/response tracking"""
-    stepNumber: int
-    stepName: str
-    functionName: str
-    status: str  # 'success' or 'error'
-    message: str
-    error: Optional[str] = None
-    duration: Optional[int] = None  # milliseconds
-
-
-@dataclass
-class ProgressState:
-    """Progress state for AI tasks"""
-    phase: str  # INIT, PREP, AI_CALL, PARSE, SAVE, DONE
-    percentage: int  # 0-100
-    message: str
-    current: Optional[int] = None
-    total: Optional[int] = None
-    current_item: Optional[str] = None
-
-
-@dataclass
-class AITaskResult:
-    """Result from AI function execution"""
-    success: bool
-    function_name: str
-    result_data: Dict[str, Any]
-    request_steps: List[StepLog]
-    response_steps: List[StepLog]
-    cost: float = 0.0
-    tokens: int = 0
-    error: Optional[str] = None
-    duration: Optional[int] = None  # milliseconds
-