# IGNY8 AI System Audit — Current Structure & Flow Mapping (Baseline Report)

**Date:** 2024-12-19  
**Scope:** Complete structural and functional audit of the IGNY8 AI subsystem  
**Methodology:** Code analysis without modifications or assumptions

---

## Executive Summary

The IGNY8 AI subsystem is a comprehensive framework for AI-powered content operations including keyword clustering, idea generation, content generation, and image generation. The system uses a unified framework architecture with a centralized execution engine, but also maintains legacy code paths for backward compatibility.

**Key Findings:**
- **20 core AI files** in `backend/igny8_core/ai/`
- **4 AI function implementations** following BaseAIFunction pattern
- **Dual execution paths:** New unified framework (`run_ai_task` → `AIEngine`) and legacy paths (direct task calls)
- **Centralized AI request handling** via `AICore.run_ai_request()`
- **Comprehensive tracking** via `StepTracker`, `ProgressTracker`, `CostTracker`, and `ConsoleStepTracker`
- **Multiple prompt management systems:** `PromptRegistry` (new) and direct database queries (legacy)

---

## 1. Current File Inventory

### 1.1 Core AI Framework (`backend/igny8_core/ai/`)

| File | Lines | Purpose |
|------|-------|---------|
| `__init__.py` | 73 | Package exports - exposes main classes and functions |
| `admin.py` | 60 | Django admin configuration for `AITaskLog` model |
| `ai_core.py` | 756 | Centralized AI request handler - `AICore` class with `run_ai_request()` and `generate_image()` |
| `apps.py` | 21 | Django app configuration |
| `base.py` | 95 | Abstract base class `BaseAIFunction` - defines function interface |
| `constants.py` | 42 | Model pricing, valid models, configuration constants |
| `engine.py` | 375 | Central orchestrator `AIEngine` - manages function lifecycle |
| `models.py` | 52 | Database model `AITaskLog` for unified logging |
| `processor.py` | 76 | **DEPRECATED** wrapper around `AICore` for backward compatibility |
| `prompts.py` | 432 | `PromptRegistry` - centralized prompt management with hierarchical resolution |
| `registry.py` | 97 | Function registry with lazy loading - `register_function()`, `get_function()` |
| `settings.py` | 117 | Model configurations per function - `MODEL_CONFIG`, `get_model_config()` |
| `tasks.py` | 131 | Unified Celery task entrypoint `run_ai_task()` - single entry point for all AI functions |
| `tracker.py` | 348 | Progress tracking utilities - `StepTracker`, `ProgressTracker`, `CostTracker`, `ConsoleStepTracker` |
| `types.py` | 44 | Type definitions - `StepLog`, `ProgressState`, `AITaskResult` dataclasses |
| `validators.py` | 187 | Validation functions - `validate_ids()`, `validate_keywords_exist()`, etc. |

### 1.2 AI Function Implementations (`backend/igny8_core/ai/functions/`)

| File | Lines | Purpose |
|------|-------|---------|
| `__init__.py` | 18 | Function package exports |
| `auto_cluster.py` | 330 | `AutoClusterFunction` - Groups keywords into semantic clusters |
| `generate_content.py` | 388 | `GenerateContentFunction` + `generate_content_core()` - Generates article content |
| `generate_ideas.py` | 335 | `GenerateIdeasFunction` + `generate_ideas_core()` - Generates content ideas from clusters |
| `generate_images.py` | 279 | `GenerateImagesFunction` + `generate_images_core()` - Generates images for tasks |

### 1.3 Utility Files (`backend/igny8_core/utils/`)

| File | Lines | Purpose |
|------|-------|---------|
| `ai_processor.py` | 1407 | **LEGACY** Unified AI interface - `AIProcessor` class with OpenAI/Runware support. Contains duplicate constants and methods. |
| `content_normalizer.py` | 273 | Content normalization - converts AI responses to HTML format |
| `queue_manager.py` | 90 | Queue abstraction (currently placeholder, not fully implemented) |
| `wordpress.py` | (not read) | WordPress integration utilities |

### 1.4 Module Task Files

| File | Lines | Purpose |
|------|-------|---------|
| `modules/planner/tasks.py` | 736 | **DEPRECATED** Legacy clustering task `auto_cluster_keywords_task()` - uses old `AIProcessor` |
| `modules/writer/tasks.py` | 1156 | Legacy content/image generation tasks - `auto_generate_content_task()`, `auto_generate_images_task()` |

### 1.5 Configuration Files

| File | Purpose |
|------|---------|
| `celery.py` | Celery app configuration - auto-discovers tasks from all Django apps |
| `modules/system/models.py` | `AIPrompt`, `IntegrationSettings` models for AI configuration |

---

## 2. Function Inventory

### 2.1 Core Framework Functions

#### `AICore` (ai_core.py)
- **`__init__(account=None)`** - Initialize with account context, loads API keys and model from `IntegrationSettings`
- **`_load_account_settings()`** - Loads OpenAI/Runware API keys and model from `IntegrationSettings` or Django settings
- **`get_api_key(integration_type='openai')`** - Returns API key for integration type
- **`get_model(integration_type='openai')`** - Returns model name for integration type
- **`run_ai_request(prompt, model=None, max_tokens=4000, temperature=0.7, response_format=None, api_key=None, function_name='ai_request', function_id=None, tracker=None)`** - **CENTRAL METHOD** - Handles all OpenAI text generation requests with console logging
- **`extract_json(response_text)`** - Extracts JSON from response text (handles markdown code blocks)
- **`generate_image(prompt, provider='openai', model=None, size='1024x1024', n=1, api_key=None, negative_prompt=None, function_name='generate_image')`** - Generates images via OpenAI DALL-E or Runware
- **`_generate_image_openai(...)`** - Internal method for OpenAI image generation
- **`_generate_image_runware(...)`** - Internal method for Runware image generation
- **`calculate_cost(model, input_tokens, output_tokens, model_type='text')`** - Calculates API cost
- **`call_openai(...)`** - **LEGACY** - Redirects to `run_ai_request()`

#### `AIEngine` (engine.py)
- **`__init__(celery_task=None, account=None)`** - Initialize with Celery task and account
- **`execute(fn: BaseAIFunction, payload: dict)`** - **CENTRAL ORCHESTRATOR** - Unified execution pipeline:
  - Phase 1: INIT (0-10%) - Validation
  - Phase 2: PREP (10-25%) - Data loading & prompt building
  - Phase 3: AI_CALL (25-70%) - API call to provider
  - Phase 4: PARSE (70-85%) - Response parsing
  - Phase 5: SAVE (85-98%) - Database operations
  - Phase 6: DONE (98-100%) - Finalization
- **`_handle_error(error, fn=None, exc_info=False)`** - Centralized error handling
- **`_log_to_database(fn, payload, parsed, save_result, error=None)`** - Logs to `AITaskLog` model
- **`_calculate_credits_for_clustering(keyword_count, tokens, cost)`** - Calculates credits for clustering operations

#### `PromptRegistry` (prompts.py)
- **`get_prompt(function_name, account=None, task=None, context=None)`** - Hierarchical prompt resolution:
  1. Task-level `prompt_override` (if exists)
  2. DB prompt for (account, function)
  3. Default fallback from registry
- **`_render_prompt(prompt_template, context)`** - Renders template with `[IGNY8_*]` placeholders and `{variable}` format
- **`get_image_prompt_template(account=None)`** - Gets image prompt template
- **`get_negative_prompt(account=None)`** - Gets negative prompt

#### `BaseAIFunction` (base.py) - Abstract Interface
- **`get_name()`** - Returns function name (abstract)
- **`get_metadata()`** - Returns function metadata (display name, description, phases)
- **`validate(payload, account=None)`** - Validates input payload (default: checks for 'ids')
- **`get_max_items()`** - Returns max items limit (optional)
- **`prepare(payload, account=None)`** - Loads and prepares data (abstract)
- **`build_prompt(data, account=None)`** - Builds AI prompt (abstract)
- **`get_model(account=None)`** - Returns model override (optional)
- **`parse_response(response, step_tracker=None)`** - Parses AI response (abstract)
- **`save_output(parsed, original_data, account=None, progress_tracker=None, step_tracker=None)`** - Saves results to database (abstract)

### 2.2 AI Function Implementations

#### `AutoClusterFunction` (functions/auto_cluster.py)
- **`get_name()`** - Returns `'auto_cluster'`
- **`validate(payload, account=None)`** - Validates keyword IDs exist
- **`prepare(payload, account=None)`** - Loads keywords with relationships
- **`build_prompt(data, account=None)`** - Builds clustering prompt using `PromptRegistry`
- **`parse_response(response, step_tracker=None)`** - Parses JSON cluster data
- **`save_output(parsed, original_data, account=None, ...)`** - Creates/updates clusters and assigns keywords

#### `GenerateIdeasFunction` (functions/generate_ideas.py)
- **`get_name()`** - Returns `'generate_ideas'`
- **`validate(payload, account=None)`** - Validates cluster IDs exist
- **`prepare(payload, account=None)`** - Loads clusters with keywords
- **`build_prompt(data, account=None)`** - Builds ideas generation prompt
- **`parse_response(response, step_tracker=None)`** - Parses JSON ideas data
- **`save_output(parsed, original_data, account=None, ...)`** - Creates `ContentIdeas` records

#### `GenerateContentFunction` (functions/generate_content.py)
- **`get_name()`** - Returns `'generate_content'`
- **`validate(payload, account=None)`** - Validates task IDs exist
- **`prepare(payload, account=None)`** - Loads tasks with relationships
- **`build_prompt(data, account=None)`** - Builds content generation prompt
- **`parse_response(response, step_tracker=None)`** - Parses JSON or plain text content
- **`save_output(parsed, original_data, account=None, ...)`** - Saves content to `Content` model

#### `GenerateImagesFunction` (functions/generate_images.py)
- **`get_name()`** - Returns `'generate_images'`
- **`validate(payload, account=None)`** - Validates task IDs exist
- **`prepare(payload, account=None)`** - Loads tasks and image settings
- **`build_prompt(data, account=None)`** - Extracts image prompts from task content (calls AI)
- **`parse_response(response, step_tracker=None)`** - Returns parsed response (already parsed)
- **`save_output(parsed, original_data, account=None, ...)`** - Creates `Images` records

### 2.3 Tracking Functions

#### `StepTracker` (tracker.py)
- **`add_request_step(step_name, status='success', message='', error=None, duration=None)`** - Adds request step
- **`add_response_step(step_name, status='success', message='', error=None, duration=None)`** - Adds response step
- **`get_meta()`** - Returns metadata dict with request/response steps

#### `ProgressTracker` (tracker.py)
- **`update(phase, percentage, message, current=None, total=None, current_item=None, meta=None)`** - Updates Celery task state
- **`set_phase(phase, percentage, message, meta=None)`** - Sets progress phase
- **`complete(message='Task complete!', meta=None)`** - Marks task as complete
- **`error(error_message, meta=None)`** - Marks task as failed
- **`get_duration()`** - Returns elapsed time in milliseconds

#### `ConsoleStepTracker` (tracker.py)
- **`init(message='Task started')`** - Logs initialization
- **`prep(message)`** - Logs preparation phase
- **`ai_call(message)`** - Logs AI call phase
- **`parse(message)`** - Logs parsing phase
- **`save(message)`** - Logs save phase
- **`done(message='Execution completed')`** - Logs completion
- **`error(error_type, message, exception=None)`** - Logs error
- **`retry(attempt, max_attempts, reason='')`** - Logs retry
- **`timeout(timeout_seconds)`** - Logs timeout
- **`rate_limit(retry_after)`** - Logs rate limit
- **`malformed_json(details='')`** - Logs JSON parsing error

#### `CostTracker` (tracker.py)
- **`record(function_name, cost, tokens, model=None)`** - Records API call cost
- **`get_total()`** - Returns total cost
- **`get_total_tokens()`** - Returns total tokens
- **`get_operations()`** - Returns all operations list

### 2.4 Celery Tasks

#### `run_ai_task` (ai/tasks.py)
- **`run_ai_task(self, function_name: str, payload: dict, account_id: int = None)`** - **UNIFIED ENTRYPOINT** - Dynamically loads and executes AI functions via `AIEngine`

#### Legacy Tasks (modules/*/tasks.py)
- **`auto_cluster_keywords_task`** (planner/tasks.py) - **DEPRECATED** - Uses old `AIProcessor`
- **`auto_generate_content_task`** (writer/tasks.py) - Uses `AIProcessor` directly (not via framework)
- **`auto_generate_images_task`** (writer/tasks.py) - Uses `AIProcessor` directly

### 2.5 Legacy Functions (ai_processor.py)

#### `AIProcessor` - **LEGACY/DEPRECATED**
- **`_call_openai(prompt, model=None, max_tokens=4000, temperature=0.7, response_format=None, api_key=None, function_id=None, response_steps=None)`** - Internal OpenAI API caller
- **`_extract_json_from_response(response_text)`** - JSON extraction (duplicate of `AICore.extract_json()`)
- **`generate_content(prompt, model=None, max_tokens=4000, temperature=0.7, **kwargs)`** - Generates text content
- **`extract_image_prompts(content, title, max_images=3, account=None)`** - Extracts image prompts from content
- **`check_moderation(text, api_key=None)`** - Checks content moderation
- **`generate_image(prompt, provider='openai', model=None, size='1024x1024', n=1, api_key=None, **kwargs)`** - Generates images
- **`cluster_keywords(keywords, sector_name=None, account=None, response_steps=None, progress_callback=None, tracker=None, **kwargs)`** - **DEPRECATED** - Clusters keywords (old method)
- **`generate_ideas(clusters, account=None, **kwargs)`** - Generates ideas (old method)
- **`get_prompt(prompt_type, account=None)`** - Gets prompt from database (old method)
- **`estimate_cost(operation, tokens_or_prompt, model=None)`** - Estimates cost (not implemented)

---

## 3. Class Inventory

### 3.1 Core Classes

| Class | File | Purpose | Inheritance |
|-------|------|---------|-------------|
| `AICore` | ai_core.py | Centralized AI request handler | - |
| `AIEngine` | engine.py | Central orchestrator for AI functions | - |
| `BaseAIFunction` | base.py | Abstract base for all AI functions | ABC |
| `PromptRegistry` | prompts.py | Centralized prompt management | - |
| `StepTracker` | tracker.py | Tracks request/response steps | - |
| `ProgressTracker` | tracker.py | Tracks Celery progress updates | - |
| `CostTracker` | tracker.py | Tracks API costs and tokens | - |
| `ConsoleStepTracker` | tracker.py | Console-based step logging | - |
| `AITaskLog` | models.py | Database model for AI task logging | AccountBaseModel |
| `AIProcessor` | utils/ai_processor.py | **LEGACY** Unified AI interface | - |

### 3.2 Function Classes

| Class | File | Purpose | Inheritance |
|-------|------|---------|-------------|
| `AutoClusterFunction` | functions/auto_cluster.py | Keyword clustering | BaseAIFunction |
| `GenerateIdeasFunction` | functions/generate_ideas.py | Idea generation | BaseAIFunction |
| `GenerateContentFunction` | functions/generate_content.py | Content generation | BaseAIFunction |
| `GenerateImagesFunction` | functions/generate_images.py | Image generation | BaseAIFunction |

### 3.3 Data Classes

| Class | File | Purpose |
|-------|------|---------|
| `StepLog` | types.py | Single step in request/response tracking |
| `ProgressState` | types.py | Progress state for AI tasks |
| `AITaskResult` | types.py | Result from AI function execution |

---

## 4. Dependency Graph/Table

### 4.1 Import Relationships

```
ai/__init__.py
  ├─> registry.py (register_function, get_function, list_functions)
  ├─> engine.py (AIEngine)
  ├─> base.py (BaseAIFunction)
  ├─> ai_core.py (AICore)
  ├─> validators.py (all validators)
  ├─> constants.py (all constants)
  ├─> prompts.py (PromptRegistry, get_prompt)
  └─> settings.py (MODEL_CONFIG, get_model_config, etc.)

ai/tasks.py
  ├─> engine.py (AIEngine)
  └─> registry.py (get_function_instance)

ai/engine.py
  ├─> base.py (BaseAIFunction)
  ├─> tracker.py (StepTracker, ProgressTracker, CostTracker, ConsoleStepTracker)
  ├─> ai_core.py (AICore)
  └─> settings.py (get_model_config)

ai/ai_core.py
  ├─> constants.py (MODEL_RATES, IMAGE_MODEL_RATES, etc.)
  └─> tracker.py (ConsoleStepTracker)

ai/functions/auto_cluster.py
  ├─> base.py (BaseAIFunction)
  ├─> ai_core.py (AICore)
  ├─> prompts.py (PromptRegistry)
  └─> settings.py (get_model_config)

ai/functions/generate_content.py
  ├─> base.py (BaseAIFunction)
  ├─> ai_core.py (AICore)
  ├─> prompts.py (PromptRegistry)
  └─> settings.py (get_model_config)

ai/functions/generate_ideas.py
  ├─> base.py (BaseAIFunction)
  ├─> ai_core.py (AICore)
  ├─> prompts.py (PromptRegistry)
  └─> settings.py (get_model_config)

ai/functions/generate_images.py
  ├─> base.py (BaseAIFunction)
  ├─> ai_core.py (AICore)
  ├─> prompts.py (PromptRegistry)
  └─> settings.py (get_model_config)

utils/ai_processor.py
  ├─> modules/system/models.py (IntegrationSettings)
  └─> modules/system/utils.py (get_prompt_value, get_default_prompt)

modules/planner/tasks.py
  └─> utils/ai_processor.py (AIProcessor) [DEPRECATED PATH]

modules/writer/tasks.py
  ├─> utils/ai_processor.py (AIProcessor) [LEGACY PATH]
  └─> ai/functions/generate_content.py (generate_content_core)
```

### 4.2 External Dependencies

| Dependency | Used By | Purpose |
|------------|---------|---------|
| `django` | All files | Django ORM, models, settings |
| `celery` | tasks.py, engine.py | Async task execution |
| `requests` | ai_core.py, ai_processor.py | HTTP requests to OpenAI/Runware APIs |
| `json` | Multiple files | JSON parsing |
| `re` | ai_core.py, ai_processor.py, content_normalizer.py | Regex for JSON extraction |
| `logging` | All files | Logging |
| `time` | tracker.py, ai_core.py | Timing and duration tracking |
| `bs4` (BeautifulSoup) | content_normalizer.py | HTML parsing (optional) |

### 4.3 Database Models Dependencies

| Model | Used By | Purpose |
|-------|---------|---------|
| `AITaskLog` | engine.py | Unified AI task logging |
| `IntegrationSettings` | ai_core.py, ai_processor.py, settings.py | API keys and model configuration |
| `AIPrompt` | prompts.py | Custom prompt templates |
| `Keywords` | auto_cluster.py, validators.py | Keyword data |
| `Clusters` | auto_cluster.py, generate_ideas.py | Cluster data |
| `ContentIdeas` | generate_ideas.py | Content ideas |
| `Tasks` | generate_content.py, generate_images.py | Writer tasks |
| `Content` | generate_content.py | Generated content |
| `Images` | generate_images.py | Generated images |

---

## 5. System Flow Description

### 5.1 New Unified Framework Flow (Recommended Path)

```
Frontend API Call
  ↓
ViewSet Action (e.g., planner/views.py::auto_cluster)
  ↓
run_ai_task.delay(function_name='auto_cluster', payload={ids: [...]}, account_id=123)
  ↓
Celery Worker: run_ai_task (ai/tasks.py)
  ├─> Load Account
  ├─> get_function_instance('auto_cluster') → AutoClusterFunction
  └─> AIEngine.execute(AutoClusterFunction, payload)
       ├─> Phase 1: INIT (0-10%)
       │   └─> fn.validate(payload, account)
       ├─> Phase 2: PREP (10-25%)
       │   ├─> fn.prepare(payload, account) → Load keywords
       │   └─> fn.build_prompt(data, account) → PromptRegistry.get_prompt()
       ├─> Phase 3: AI_CALL (25-70%)
       │   ├─> AICore.run_ai_request(prompt, model, ...)
       │   │   ├─> Load API key from IntegrationSettings
       │   │   ├─> Validate model
       │   │   ├─> Build OpenAI request
       │   │   ├─> Send HTTP request
       │   │   ├─> Parse response
       │   │   └─> Calculate cost
       │   └─> Track cost via CostTracker
       ├─> Phase 4: PARSE (70-85%)
       │   └─> fn.parse_response(response_content, step_tracker)
       ├─> Phase 5: SAVE (85-98%)
       │   └─> fn.save_output(parsed, original_data, account, ...)
       │       └─> Database transaction: Create/update clusters
       └─> Phase 6: DONE (98-100%)
           ├─> Log to AITaskLog
           └─> Return result dict
  ↓
Celery Task State Update (SUCCESS/FAILURE)
  ↓
Frontend Polls Task Status
  ↓
Progress Modal Displays Steps
```

### 5.2 Legacy Content Generation Flow (Still Active)

```
Frontend API Call
  ↓
ViewSet Action (writer/views.py::auto_generate_content)
  ↓
auto_generate_content_task.delay(task_ids, account_id)
  ↓
Celery Worker: auto_generate_content_task (modules/writer/tasks.py)
  ├─> Load Tasks from database
  ├─> For each task:
  │   ├─> Load prompt template (get_prompt_value)
  │   ├─> Format prompt with task data
  │   ├─> AIProcessor.generate_content(prompt)
  │   │   └─> AIProcessor._call_openai() [DUPLICATE OF AICore.run_ai_request]
  │   ├─> Parse response (GenerateContentFunction.parse_response)
  │   └─> Save content (GenerateContentFunction.save_output)
  └─> Return result
```

### 5.3 Legacy Clustering Flow (Deprecated)

```
Frontend API Call
  ↓
ViewSet Action (planner/views.py::auto_cluster) [OLD PATH]
  ↓
_auto_cluster_keywords_core() (modules/planner/tasks.py)
  ├─> Load keywords
  ├─> AIProcessor.cluster_keywords() [DEPRECATED]
  │   └─> AIProcessor._call_openai() [DUPLICATE]
  ├─> Parse clusters
  └─> Save to database
```

### 5.4 Image Generation Flow

```
Frontend API Call
  ↓
ViewSet Action (writer/views.py::auto_generate_images)
  ↓
auto_generate_images_task.delay(task_ids, account_id)
  ↓
Celery Worker: auto_generate_images_task (modules/writer/tasks.py)
  ├─> Load tasks
  ├─> For each task:
  │   ├─> Extract image prompts (AIProcessor.extract_image_prompts)
  │   │   └─> Calls AI to extract prompts from content
  │   ├─> Generate featured image (AIProcessor.generate_image)
  │   ├─> Generate desktop images (if enabled)
  │   └─> Generate mobile images (if enabled)
  └─> Save Images records
```

---

## 6. Integration Points

### 6.1 Celery Integration

**Task Registration:**
- `celery.py` uses `app.autodiscover_tasks()` to auto-discover tasks from all Django apps
- Tasks are registered via `@shared_task` decorator

**Registered Tasks:**
1. `ai.tasks.run_ai_task` - Unified entrypoint (NEW)
2. `planner.tasks.auto_cluster_keywords_task` - **DEPRECATED**
3. `writer.tasks.auto_generate_content_task` - Legacy (still active)
4. `writer.tasks.auto_generate_images_task` - Legacy (still active)

**Task State Management:**
- `ProgressTracker.update()` calls `task.update_state(state='PROGRESS', meta={...})`
- Progress metadata includes: `phase`, `percentage`, `message`, `request_steps`, `response_steps`
- Final states: `SUCCESS`, `FAILURE`

### 6.2 Database Integration

**Models:**
- `AITaskLog` - Unified logging table (used by `AIEngine._log_to_database()`)
- `IntegrationSettings` - API keys and model configuration (used by `AICore._load_account_settings()`)
- `AIPrompt` - Custom prompt templates (used by `PromptRegistry.get_prompt()`)

**Data Models:**
- `Keywords` - Input for clustering
- `Clusters` - Output of clustering
- `ContentIdeas` - Output of idea generation
- `Tasks` - Input for content/image generation
- `Content` - Output of content generation
- `Images` - Output of image generation

### 6.3 Frontend API Integration

**Endpoints:**
1. `POST /v1/planner/keywords/auto_cluster/` → `planner.views.ClusterViewSet.auto_cluster()`
   - Calls `run_ai_task.delay(function_name='auto_cluster', ...)`

2. `POST /v1/planner/clusters/auto_generate_ideas/` → `planner.views.ClusterViewSet.auto_generate_ideas()`
   - Calls `run_ai_task.delay(function_name='auto_generate_ideas', ...)`

3. `POST /v1/writer/tasks/auto_generate_content/` → `writer.views.TasksViewSet.auto_generate_content()`
   - Calls `auto_generate_content_task.delay(...)` [LEGACY PATH]

4. `POST /v1/writer/tasks/auto_generate_images/` → `writer.views.TasksViewSet.auto_generate_images()`
   - Calls `auto_generate_images_task.delay(...)` [LEGACY PATH]

**Response Format:**
- Success: `{success: true, task_id: "...", message: "..."}`
- Error: `{success: false, error: "..."}`

**Progress Tracking:**
- Frontend polls Celery task status via `GET /v1/system/tasks/{task_id}/status/`
- Progress modal displays `request_steps` and `response_steps` from task meta

### 6.4 Configuration Integration

**Settings Loading Hierarchy:**
1. **Account-level** (`IntegrationSettings` model):
   - API keys: `IntegrationSettings.config['apiKey']`
   - Model: `IntegrationSettings.config['model']`
   - Image settings: `IntegrationSettings.config` (for image_generation type)

2. **Django Settings** (fallback):
   - `OPENAI_API_KEY`
   - `RUNWARE_API_KEY`
   - `DEFAULT_AI_MODEL`

3. **Function-level** (`ai/settings.py`):
   - `MODEL_CONFIG[function_name]` - Default model, max_tokens, temperature per function

**Prompt Loading Hierarchy:**
1. Task-level override: `task.prompt_override` (if exists)
2. Account-level: `AIPrompt` model (account, prompt_type)
3. Default: `PromptRegistry.DEFAULT_PROMPTS[prompt_type]`

### 6.5 Debug Panel Integration

**Console Logging:**
- `ConsoleStepTracker` logs to stdout/stderr (only if `DEBUG_MODE=True`)
- Logs include timestamps, phases, messages, errors

**Step Tracking:**
- `StepTracker` maintains `request_steps` and `response_steps` arrays
- Steps include: `stepNumber`, `stepName`, `status`, `message`, `duration`, `error`
- Steps are included in Celery task meta and displayed in progress modal

**Database Logging:**
- `AITaskLog` records all AI task executions
- Fields: `task_id`, `function_name`, `phase`, `status`, `cost`, `tokens`, `request_steps`, `response_steps`, `error`, `payload`, `result`

---

## 7. Identified Redundancies or Repetition

### 7.1 Duplicate Constants

**Location 1:** `ai/constants.py`
- `MODEL_RATES`, `IMAGE_MODEL_RATES`, `VALID_OPENAI_IMAGE_MODELS`, `VALID_SIZES_BY_MODEL`, `DEFAULT_AI_MODEL`, `JSON_MODE_MODELS`

**Location 2:** `utils/ai_processor.py` (lines 18-44)
- **EXACT DUPLICATE** of all constants from `constants.py`

**Impact:** Constants are defined in two places, risking inconsistency.

### 7.2 Duplicate JSON Extraction Logic

**Location 1:** `ai/ai_core.py::extract_json()` (lines 391-429)
- Handles markdown code blocks, multiline JSON, direct JSON

**Location 2:** `utils/ai_processor.py::_extract_json_from_response()` (lines 342-449)
- **MORE COMPREHENSIVE** - handles more edge cases, balanced brace matching

**Impact:** Two implementations with different capabilities. `ai_processor.py` version is more robust.

### 7.3 Duplicate OpenAI API Calling Logic

**Location 1:** `ai/ai_core.py::run_ai_request()` (lines 106-389)
- Centralized method with console logging via `ConsoleStepTracker`
- Handles validation, model selection, cost calculation
- Returns standardized dict format

**Location 2:** `utils/ai_processor.py::_call_openai()` (lines 125-340)
- **SIMILAR LOGIC** but with `response_steps` parameter instead of `tracker`
- Less comprehensive error handling
- Different return format

**Impact:** Two code paths for the same operation. New code should use `AICore.run_ai_request()`.

### 7.4 Duplicate Image Generation Logic

**Location 1:** `ai/ai_core.py::generate_image()` + `_generate_image_openai()` + `_generate_image_runware()` (lines 431-728)
- Uses `print()` statements for logging (inconsistent with console tracker)

**Location 2:** `utils/ai_processor.py::generate_image()` (lines 667-1043)
- **MORE COMPREHENSIVE** - extensive logging, better error handling
- Handles Runware authentication flow

**Impact:** Two implementations. `ai_processor.py` version has better logging.

### 7.5 Duplicate Prompt Loading Logic

**Location 1:** `ai/prompts.py::PromptRegistry.get_prompt()` (lines 280-333)
- Hierarchical resolution: task override → DB prompt → default
- Supports `[IGNY8_*]` placeholders and `{variable}` format

**Location 2:** `utils/ai_processor.py::get_prompt()` (lines 1044-1057)
- Simple database lookup via `modules/system/utils.get_prompt_value()`
- No hierarchical resolution

**Location 3:** Direct calls in `modules/writer/tasks.py` (lines 343, 959, 964)
- Uses `get_prompt_value()` and `get_default_prompt()` directly

**Impact:** Three different ways to load prompts. New code should use `PromptRegistry`.

### 7.6 Duplicate Model Configuration Logic

**Location 1:** `ai/settings.py::get_model_config()` (lines 49-97)
- Reads from `IntegrationSettings` if account provided
- Falls back to `MODEL_CONFIG` defaults

**Location 2:** `ai/ai_core.py::_load_account_settings()` (lines 46-90)
- Reads model from `IntegrationSettings` directly
- Similar logic but embedded in `AICore.__init__()`

**Location 3:** `utils/ai_processor.py::_get_model()` (lines 98-123)
- Reads model from `IntegrationSettings` directly
- Similar logic but embedded in `AIProcessor.__init__()`

**Impact:** Model loading logic duplicated in three places.

### 7.7 Duplicate API Key Loading Logic

**Location 1:** `ai/ai_core.py::_load_account_settings()` (lines 46-90)
- Loads OpenAI and Runware keys from `IntegrationSettings`

**Location 2:** `utils/ai_processor.py::_get_api_key()` (lines 73-96)
- **EXACT SAME LOGIC** for loading API keys

**Impact:** Identical code in two places.

### 7.8 Repeated Error Handling Patterns

**Pattern:** Multiple files have similar try/except blocks for:
- API request errors
- JSON parsing errors
- Database errors
- Validation errors

**Impact:** Error handling is not centralized, making it harder to maintain consistent error messages and logging.

### 7.9 Repeated Progress Update Patterns

**Pattern:** Multiple places manually build progress update dicts:
- `modules/writer/tasks.py` (lines 62-73, 220-231, etc.)
- `modules/planner/tasks.py` (lines 59-71, 203-215, etc.)
- `ai/engine.py` (lines 57, 79, 141, etc.)

**Impact:** Progress update format is not standardized, though `ProgressTracker` exists to handle this.

---

## 8. Summary of Potential Consolidation Areas

### 8.1 Constants Consolidation

**Observation:** Model rates, valid models, and configuration constants are duplicated between `ai/constants.py` and `utils/ai_processor.py`.

**Potential Action:** Remove constants from `ai_processor.py` and import from `constants.py`. However, `ai_processor.py` is marked as legacy, so this may not be necessary if it's being phased out.

### 8.2 JSON Extraction Consolidation

**Observation:** Two JSON extraction methods exist with different capabilities. `ai_processor.py::_extract_json_from_response()` is more comprehensive.

**Potential Action:** Enhance `AICore.extract_json()` with logic from `ai_processor.py`, or create a shared utility function.

### 8.3 API Request Consolidation

**Observation:** `AICore.run_ai_request()` and `AIProcessor._call_openai()` perform the same operation with different interfaces.

**Potential Action:** All new code should use `AICore.run_ai_request()`. Legacy code in `ai_processor.py` can remain for backward compatibility but should be marked as deprecated.

### 8.4 Image Generation Consolidation

**Observation:** Two image generation implementations exist. `ai_processor.py` version has better logging.

**Potential Action:** Enhance `AICore.generate_image()` with logging improvements from `ai_processor.py`, or migrate all code to use `AICore.generate_image()`.

### 8.5 Prompt Loading Consolidation

**Observation:** Three different methods exist for loading prompts: `PromptRegistry.get_prompt()`, `AIProcessor.get_prompt()`, and direct `get_prompt_value()` calls.

**Potential Action:** Migrate all code to use `PromptRegistry.get_prompt()` for consistency. Update legacy code paths gradually.

### 8.6 Model/API Key Loading Consolidation

**Observation:** Model and API key loading logic is duplicated in `AICore`, `AIProcessor`, and `settings.py`.

**Potential Action:** Create shared utility functions for loading settings from `IntegrationSettings`, used by all classes.

### 8.7 Error Handling Consolidation

**Observation:** Error handling patterns are repeated across multiple files.

**Potential Action:** Create centralized error handling utilities or enhance `AIEngine._handle_error()` to be more reusable.

### 8.8 Progress Tracking Consolidation

**Observation:** Some code manually builds progress update dicts instead of using `ProgressTracker`.

**Potential Action:** Migrate all progress updates to use `ProgressTracker.update()` for consistency.

### 8.9 Legacy Code Path Elimination

**Observation:** Multiple execution paths exist:
- New: `run_ai_task` → `AIEngine` → `BaseAIFunction` implementations
- Legacy: Direct `AIProcessor` calls in `modules/*/tasks.py`

**Potential Action:** Gradually migrate all legacy tasks to use the new framework. Mark legacy code as deprecated.

---

## 9. Assumptions Made

1. **File `wordpress.py`** was not read - assumed to be unrelated to AI processing based on name.

2. **Frontend code** was partially analyzed via search results - full frontend audit not performed.

3. **Database migrations** were not analyzed - assumed to be standard Django migrations.

4. **Test files** were not analyzed - `ai/tests/test_run.py` exists but was not read.

5. **Settings file** (`backend/igny8_core/settings.py`) was not read - assumed to contain standard Django settings.

6. **System module utilities** (`modules/system/utils.py`) were referenced but not fully read - assumed to contain `get_prompt_value()` and `get_default_prompt()` functions.

---

## 10. Appendix

### 10.1 File Line Counts

| Directory | Files | Total Lines |
|-----------|-------|-------------|
| `ai/` | 15 | ~2,500 |
| `ai/functions/` | 5 | ~1,300 |
| `utils/` (AI-related) | 3 | ~1,800 |
| `modules/planner/tasks.py` | 1 | 736 |
| `modules/writer/tasks.py` | 1 | 1,156 |
| **Total** | **25** | **~7,500** |

### 10.2 Function Count Summary

- **Core Framework Functions:** ~30
- **AI Function Implementations:** 4 classes × ~6 methods = ~24 methods
- **Tracking Functions:** ~20
- **Legacy Functions:** ~15
- **Celery Tasks:** 4
- **Total:** ~90 functions/methods

### 10.3 Key Design Patterns

1. **Template Method Pattern:** `BaseAIFunction` defines algorithm skeleton, subclasses implement steps
2. **Registry Pattern:** `FunctionRegistry` for dynamic function discovery
3. **Factory Pattern:** `get_function_instance()` creates function instances
4. **Strategy Pattern:** Different AI functions implement same interface
5. **Observer Pattern:** `ProgressTracker` updates Celery task state
6. **Facade Pattern:** `AICore` provides simplified interface to OpenAI/Runware APIs

---

**End of Report**