fixes of ai toke limit standrd 8192

AI_SYSTEM_AUDIT.md

@@ -0,0 +1,79 @@
+# AI System Audit Report
+
+## Current State
+
+### Active AI System (New Architecture)
+**Location:** `backend/igny8_core/ai/`
+
+**Core Components:**
+- `ai_core.py` - Central AI request handler (run_ai_request method)
+- `engine.py` - Orchestrator for all AI functions
+- `settings.py` - Model configuration loader
+- `prompts.py` - Prompt templates
+- `base.py` - Base class for AI functions
+- `tasks.py` - Celery tasks
+- `models.py` - AITaskLog for logging
+- `tracker.py` - Progress/step tracking
+- `registry.py` - Function registry
+- `constants.py` - Shared constants (MODEL_RATES, etc.)
+
+**AI Functions:**
+- `functions/auto_cluster.py` - Keyword clustering
+- `functions/generate_ideas.py` - Content idea generation
+- `functions/generate_content.py` - Article content generation
+- `functions/generate_images.py` - Image generation
+- `functions/generate_image_prompts.py` - Image prompt generation
+- `functions/optimize_content.py` - Content optimization
+
+**Usage:** All new code uses `AIEngine` + function classes
+
+### Legacy AI System (Old Architecture)
+**Location:** `backend/igny8_core/utils/ai_processor.py`
+
+**Purpose:** Original AI interface from reference plugin migration
+**Size:** 1390 lines
+**Status:** PARTIALLY USED - Only for:
+- MODEL_RATES constant (imported by settings.py and integration_views.py)
+- Integration test views
+
+**NOT USED FOR:** Actual AI function execution (replaced by AIEngine)
+
+## max_tokens Fallback Analysis
+
+### Current Fallbacks Found:
+
+1. **settings.py:103** - `config.get('max_tokens', 16384)`
+   - Falls back to 16384 if not in IntegrationSettings
+   
+2. **ai_core.py:116** - `max_tokens: int = 8192`
+   - Default parameter in run_ai_request()
+   
+3. **ai_core.py:856** - `max_tokens: int = 4000` **[LEGACY]**
+   - Legacy call_openai() method
+   
+4. **ai_processor.py:111** - `max_tokens: int = 4000` **[LEGACY]**
+   - Legacy _call_openai() method
+   
+5. **ai_processor.py:437** - `max_tokens: int = 4000` **[LEGACY]**
+   - Legacy generate_content() method
+   
+6. **ai_processor.py:531** - Hardcoded `max_tokens=1000`
+   
+7. **ai_processor.py:1133** - Hardcoded `max_tokens=3000`
+   
+8. **ai_processor.py:1340** - Hardcoded `max_tokens=4000`
+
+## Recommended Actions
+
+### 1. Standardize max_tokens to 8192
+- Remove fallback in settings.py (line 103): Change to just `config['max_tokens']` and require it
+- Keep ai_core.py:116 default at 8192 (main entry point)
+- Update IntegrationSettings to have 8192 as required value
+
+### 2. Mark Legacy Code
+- Add deprecation warnings to ai_processor.py
+- Document that it's only kept for MODEL_RATES constant
+- Consider extracting MODEL_RATES to constants.py and removing ai_processor.py entirely
+
+### 3. Remove Dead Code
+- call_openai() legacy method in ai_core.py (if not used)