reference plugin and image gen analysis
This commit is contained in:
Desktop
277
docs/ai-docs/AI_FUNCTIONS_AUDIT_REPORT.md
Normal file
277
docs/ai-docs/AI_FUNCTIONS_AUDIT_REPORT.md
Normal file
@@ -0,0 +1,277 @@
|
||||
# AI Functions Deep Audit Report
|
||||
## Clustering, Idea Generation, and Content Generation
|
||||
|
||||
**Date:** 2025-01-XX
|
||||
**Scope:** Complete audit of AI functions for clustering, idea generation, and content generation to identify unused code and files safe to remove.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This audit identifies **legacy code, deprecated functions, and unused implementations** that are safe to remove from the codebase. The current system uses a unified AI framework through `run_ai_task` → `AIEngine` → `BaseAIFunction` implementations.
|
||||
|
||||
---
|
||||
|
||||
## 1. CURRENT ACTIVE ARCHITECTURE
|
||||
|
||||
### 1.1 Active Flow (What's Actually Used)
|
||||
|
||||
```
|
||||
Frontend API Call
|
||||
↓
|
||||
views.py (auto_cluster/auto_generate_ideas/auto_generate_content)
|
||||
↓
|
||||
run_ai_task (ai/tasks.py) - Unified Celery task entrypoint
|
||||
↓
|
||||
AIEngine (ai/engine.py) - Orchestrator
|
||||
↓
|
||||
BaseAIFunction implementations:
|
||||
- AutoClusterFunction (ai/functions/auto_cluster.py)
|
||||
- GenerateIdeasFunction (ai/functions/generate_ideas.py)
|
||||
- GenerateContentFunction (ai/functions/generate_content.py)
|
||||
↓
|
||||
AICore (ai/ai_core.py) - Centralized AI request handler
|
||||
↓
|
||||
AIProvider (OpenAI/Runware)
|
||||
```
|
||||
|
||||
### 1.2 Active Files (KEEP)
|
||||
|
||||
#### Core Framework
|
||||
- ✅ `backend/igny8_core/ai/tasks.py` - Unified Celery task (`run_ai_task`)
|
||||
- ✅ `backend/igny8_core/ai/engine.py` - AIEngine orchestrator
|
||||
- ✅ `backend/igny8_core/ai/base.py` - BaseAIFunction abstract class
|
||||
- ✅ `backend/igny8_core/ai/ai_core.py` - AICore centralized handler
|
||||
- ✅ `backend/igny8_core/ai/registry.py` - Function registry
|
||||
- ✅ `backend/igny8_core/ai/prompts.py` - PromptRegistry
|
||||
- ✅ `backend/igny8_core/ai/settings.py` - Model configurations
|
||||
- ✅ `backend/igny8_core/ai/constants.py` - AI constants
|
||||
- ✅ `backend/igny8_core/ai/tracker.py` - Progress tracking
|
||||
- ✅ `backend/igny8_core/ai/validators.py` - Validation functions
|
||||
|
||||
#### Function Implementations
|
||||
- ✅ `backend/igny8_core/ai/functions/auto_cluster.py` - AutoClusterFunction
|
||||
- ✅ `backend/igny8_core/ai/functions/generate_ideas.py` - GenerateIdeasFunction
|
||||
- ✅ `backend/igny8_core/ai/functions/generate_content.py` - GenerateContentFunction
|
||||
- ✅ `backend/igny8_core/ai/functions/__init__.py` - Exports
|
||||
|
||||
#### API Endpoints
|
||||
- ✅ `backend/igny8_core/modules/planner/views.py` - KeywordViewSet.auto_cluster(), ClusterViewSet.auto_generate_ideas()
|
||||
- ✅ `backend/igny8_core/modules/writer/views.py` - TasksViewSet.auto_generate_content()
|
||||
|
||||
---
|
||||
|
||||
## 2. DEPRECATED / UNUSED CODE (SAFE TO REMOVE)
|
||||
|
||||
### 2.1 Legacy Wrapper Functions (NOT USED)
|
||||
|
||||
#### ❌ `generate_ideas_core()` - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/ai/functions/generate_ideas.py:234`
|
||||
- **Status:** Legacy wrapper function for backward compatibility
|
||||
- **Usage:** ❌ **NOT CALLED ANYWHERE** (only in commented test code)
|
||||
- **Purpose:** Was meant for direct calls without Celery, but all calls now go through `run_ai_task`
|
||||
- **Action:** **REMOVE** - Function and export from `__init__.py`
|
||||
|
||||
#### ❌ `generate_content_core()` - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/ai/functions/generate_content.py:303`
|
||||
- **Status:** Legacy wrapper function for backward compatibility
|
||||
- **Usage:** ❌ **NOT CALLED ANYWHERE** (only in commented test code)
|
||||
- **Purpose:** Was meant for direct calls without Celery, but all calls now go through `run_ai_task`
|
||||
- **Action:** **REMOVE** - Function and export from `__init__.py`
|
||||
|
||||
### 2.2 AIProcessor Deprecated Methods (NOT USED)
|
||||
|
||||
#### ❌ `AIProcessor.cluster_keywords()` - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/utils/ai_processor.py:1049-1282`
|
||||
- **Status:** ⚠️ **DEPRECATED** (marked with deprecation warning)
|
||||
- **Usage:** ❌ **NOT CALLED ANYWHERE**
|
||||
- **Lines:** ~233 lines of code
|
||||
- **Action:** **REMOVE** entire method
|
||||
|
||||
#### ❌ `AIProcessor.generate_ideas()` - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/utils/ai_processor.py:1284-1363`
|
||||
- **Status:** Legacy method
|
||||
- **Usage:** ❌ **NOT CALLED ANYWHERE**
|
||||
- **Lines:** ~80 lines of code
|
||||
- **Action:** **REMOVE** entire method
|
||||
|
||||
#### ❌ `AIProcessor.generate_content()` - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/utils/ai_processor.py:433-531`
|
||||
- **Status:** Legacy method
|
||||
- **Usage:** ❌ **NOT CALLED ANYWHERE** (only called internally by `extract_image_prompts`, which is also unused)
|
||||
- **Lines:** ~98 lines of code
|
||||
- **Action:** **REMOVE** entire method
|
||||
|
||||
#### ❌ `AIProcessor.extract_image_prompts()` - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/utils/ai_processor.py:471-580`
|
||||
- **Status:** Legacy method
|
||||
- **Usage:** ❌ **NOT CALLED ANYWHERE** (new framework uses `GenerateImagesFunction` which uses `AICore.run_ai_request` directly)
|
||||
- **Lines:** ~110 lines of code
|
||||
- **Action:** **REMOVE** entire method
|
||||
|
||||
**Note:** `AIProcessor.generate_image()` is **STILL USED** in `integration_views.py` for image generation, so keep the class and that method.
|
||||
|
||||
### 2.3 Unused Exports
|
||||
|
||||
#### ❌ `generate_ideas_core` export - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/ai/functions/__init__.py:5,12`
|
||||
- **Action:** Remove from imports and `__all__`
|
||||
|
||||
#### ❌ `generate_content_core` export - **SAFE TO REMOVE**
|
||||
- **Location:** `backend/igny8_core/ai/functions/__init__.py:6,14`
|
||||
- **Action:** Remove from imports and `__all__`
|
||||
|
||||
### 2.4 Test File Cleanup
|
||||
|
||||
#### ⚠️ `backend/igny8_core/ai/tests/test_run.py`
|
||||
- **Status:** Contains commented-out code referencing removed functions
|
||||
- **Action:** Clean up commented code (lines 16, 63, 75, 126-127)
|
||||
|
||||
---
|
||||
|
||||
## 3. AIProcessor - PARTIAL CLEANUP
|
||||
|
||||
### 3.1 What to KEEP in AIProcessor
|
||||
|
||||
✅ **KEEP:**
|
||||
- `AIProcessor.__init__()` - Initialization
|
||||
- `AIProcessor._get_api_key()` - API key retrieval
|
||||
- `AIProcessor._get_model()` - Model retrieval
|
||||
- `AIProcessor._call_openai()` - OpenAI API calls (used by generate_image)
|
||||
- `AIProcessor._extract_json_from_response()` - JSON extraction
|
||||
- `AIProcessor.generate_image()` - **STILL USED** in `integration_views.py`
|
||||
- `AIProcessor.get_prompt()` - May be used by generate_image
|
||||
- Constants imports (MODEL_RATES, etc.) - Used by AICore
|
||||
|
||||
### 3.2 What to REMOVE from AIProcessor
|
||||
|
||||
❌ **REMOVE:**
|
||||
- `AIProcessor.cluster_keywords()` - ~233 lines
|
||||
- `AIProcessor.generate_ideas()` - ~80 lines
|
||||
- `AIProcessor.generate_content()` - ~98 lines
|
||||
- `AIProcessor.extract_image_prompts()` - ~110 lines
|
||||
- **Total:** ~521 lines of unused code
|
||||
|
||||
---
|
||||
|
||||
## 4. FILES ALREADY DELETED (Good!)
|
||||
|
||||
✅ **Already Removed:**
|
||||
- `backend/igny8_core/modules/planner/tasks.py` - ✅ Already deleted
|
||||
- `backend/igny8_core/modules/writer/tasks.py` - ✅ Already deleted
|
||||
|
||||
---
|
||||
|
||||
## 5. SUMMARY OF REMOVALS
|
||||
|
||||
### 5.1 Functions to Remove
|
||||
|
||||
| Function | Location | Lines | Status |
|
||||
|----------|----------|-------|--------|
|
||||
| `generate_ideas_core()` | `ai/functions/generate_ideas.py:234` | ~100 | ❌ Remove |
|
||||
| `generate_content_core()` | `ai/functions/generate_content.py:303` | ~85 | ❌ Remove |
|
||||
| `AIProcessor.cluster_keywords()` | `utils/ai_processor.py:1049` | ~233 | ❌ Remove |
|
||||
| `AIProcessor.generate_ideas()` | `utils/ai_processor.py:1284` | ~80 | ❌ Remove |
|
||||
| `AIProcessor.generate_content()` | `utils/ai_processor.py:433` | ~98 | ❌ Remove |
|
||||
| `AIProcessor.extract_image_prompts()` | `utils/ai_processor.py:471` | ~110 | ❌ Remove |
|
||||
|
||||
**Total Lines to Remove:** ~706 lines
|
||||
|
||||
### 5.2 Exports to Remove
|
||||
|
||||
| Export | Location |
|
||||
|--------|----------|
|
||||
| `generate_ideas_core` | `ai/functions/__init__.py:5,12` |
|
||||
| `generate_content_core` | `ai/functions/__init__.py:6,14` |
|
||||
|
||||
### 5.3 Test Cleanup
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `ai/tests/test_run.py` | Remove commented code (lines 16, 63, 75, 126-127) |
|
||||
|
||||
---
|
||||
|
||||
## 6. VERIFICATION CHECKLIST
|
||||
|
||||
Before removing, verify:
|
||||
|
||||
- [ ] ✅ No imports of `generate_ideas_core` or `generate_content_core` anywhere
|
||||
- [ ] ✅ No calls to `AIProcessor.cluster_keywords()`, `generate_ideas()`, or `generate_content()`
|
||||
- [ ] ✅ All active code paths use `run_ai_task` → `AIEngine` → `BaseAIFunction`
|
||||
- [ ] ✅ `AIProcessor.generate_image()` is still used (verify in `integration_views.py`)
|
||||
- [ ] ✅ Constants from `ai_processor.py` are still imported by `AICore` (verify)
|
||||
|
||||
---
|
||||
|
||||
## 7. RECOMMENDED REMOVAL ORDER
|
||||
|
||||
1. **Phase 1: Remove Legacy Wrapper Functions**
|
||||
- Remove `generate_ideas_core()` from `generate_ideas.py`
|
||||
- Remove `generate_content_core()` from `generate_content.py`
|
||||
- Remove exports from `__init__.py`
|
||||
|
||||
2. **Phase 2: Remove AIProcessor Deprecated Methods**
|
||||
- Remove `AIProcessor.cluster_keywords()`
|
||||
- Remove `AIProcessor.generate_ideas()`
|
||||
- Remove `AIProcessor.generate_content()`
|
||||
- Remove `AIProcessor.extract_image_prompts()` (depends on generate_content, so remove after)
|
||||
|
||||
3. **Phase 3: Cleanup Tests**
|
||||
- Clean up commented code in `test_run.py`
|
||||
|
||||
4. **Phase 4: Verification**
|
||||
- Run full test suite
|
||||
- Verify all AI functions still work
|
||||
- Check for any broken imports
|
||||
|
||||
---
|
||||
|
||||
## 8. IMPACT ANALYSIS
|
||||
|
||||
### 8.1 No Breaking Changes Expected
|
||||
|
||||
- ✅ All active code paths use the new framework
|
||||
- ✅ No external dependencies on removed functions
|
||||
- ✅ Views only use `run_ai_task` (verified)
|
||||
|
||||
### 8.2 Benefits
|
||||
|
||||
- 🎯 **~706 lines of dead code removed**
|
||||
- 🎯 **Clearer codebase** - Only active code remains
|
||||
- 🎯 **Easier maintenance** - No confusion about which path to use
|
||||
- 🎯 **Reduced technical debt**
|
||||
|
||||
---
|
||||
|
||||
## 9. FILES TO MODIFY
|
||||
|
||||
### 9.1 Files to Edit
|
||||
|
||||
1. `backend/igny8_core/ai/functions/generate_ideas.py`
|
||||
- Remove `generate_ideas_core()` function (lines 234-333)
|
||||
|
||||
2. `backend/igny8_core/ai/functions/generate_content.py`
|
||||
- Remove `generate_content_core()` function (lines 303-386)
|
||||
|
||||
3. `backend/igny8_core/ai/functions/__init__.py`
|
||||
- Remove `generate_ideas_core` import and export
|
||||
- Remove `generate_content_core` import and export
|
||||
|
||||
4. `backend/igny8_core/utils/ai_processor.py`
|
||||
- Remove `cluster_keywords()` method (lines 1049-1282)
|
||||
- Remove `generate_ideas()` method (lines 1284-1363)
|
||||
- Remove `generate_content()` method (lines 433-531)
|
||||
- Remove `extract_image_prompts()` method (lines 471-580)
|
||||
|
||||
5. `backend/igny8_core/ai/tests/test_run.py`
|
||||
- Remove commented code referencing removed functions
|
||||
|
||||
---
|
||||
|
||||
## 10. CONCLUSION
|
||||
|
||||
This audit identifies **~706 lines of unused legacy code** that can be safely removed without impacting functionality. All active code paths use the unified AI framework (`run_ai_task` → `AIEngine` → `BaseAIFunction`), making these legacy functions obsolete.
|
||||
|
||||
**Recommendation:** Proceed with removal in phases as outlined above, with thorough testing after each phase.
|
||||
|
||||
Reference in New Issue
Block a user