# IGNY8 AI Functions Documentation

**Version:** 1.0  
**Last Updated:** 2025-01-XX  
**Purpose:** Complete documentation of all AI functions in the IGNY8 system, their workflows, dependencies, and integration points.

---

## Table of Contents

1. [AI Functions Overview](#ai-functions-overview)
2. [AIProcessor Class](#aiprocessor-class)
3. [Planner Module AI Functions](#planner-module-ai-functions)
4. [Writer Module AI Functions](#writer-module-ai-functions)
5. [System Module AI Functions](#system-module-ai-functions)
6. [Progress Tracking](#progress-tracking)
7. [Error Handling](#error-handling)
8. [Dependencies](#dependencies)

---

## AI Functions Overview

The IGNY8 system includes **8 primary AI functions** across 3 modules:

1. **Planner Module (2 functions):**
   - Auto Cluster Keywords (`auto_cluster`)
   - Auto Generate Ideas (`auto_generate_ideas`)

2. **Writer Module (3 functions):**
   - Auto Generate Content (`auto_generate_content`)
   - Auto Generate Images (`auto_generate_images`)
   - Extract Image Prompts (`extract_image_prompts`)

3. **System Module (3 functions):**
   - Test OpenAI Integration (`test_openai`)
   - Test Runware Integration (`test_runware`)
   - Generate Image (Test) (`generate_image`)

All AI functions use the unified `AIProcessor` class as the core interface for AI operations.

---

## AIProcessor Class

**Location**: `backend/igny8_core/utils/ai_processor.py`

**Purpose**: Unified AI interface for all AI operations.

### Initialization

**Method**: `AIProcessor.__init__()`

**Parameters**:
- `account` (optional): Account object for loading account-specific settings

**Dependencies**:
- `IntegrationSettings` model (for API keys and model configuration)
- Django settings (fallback for API keys)

**Initializes**:
- `self.openai_api_key`: Loaded from IntegrationSettings or Django settings
- `self.runware_api_key`: Loaded from IntegrationSettings or Django settings
- `self.default_model`: Loaded from IntegrationSettings or Django settings
- `self.model_rates`: Model pricing rates (per 1M tokens)
- `self.image_model_rates`: Image model pricing rates (per image)

### Core Methods

#### cluster_keywords()
**Purpose**: Cluster keywords using AI-based semantic similarity.

**Parameters**:
- `keywords`: List of keyword dictionaries
- `sector_name`: Optional sector name for context
- `account`: Optional account object
- `response_steps`: Optional list for step tracking
- `progress_callback`: Optional callback function

**Dependencies**:
- `AIPrompt` model (for clustering prompt template)
- `get_default_prompt()` utility function
- `_call_openai()` method
- `_extract_json_from_response()` method

**Returns**: Dictionary with `clusters` array and `response_steps`

#### generate_ideas()
**Purpose**: Generate content ideas for clusters using AI.

**Parameters**:
- `clusters`: List of cluster dictionaries
- `account`: Optional account object

**Dependencies**:
- `AIPrompt` model (for ideas prompt template)
- `get_default_prompt()` utility function
- `_call_openai()` method
- `_extract_json_from_response()` method

**Returns**: Dictionary with `ideas` array

#### generate_content()
**Purpose**: Generate text content using OpenAI GPT models.

**Parameters**:
- `prompt`: The prompt text
- `model`: Optional model name (defaults to self.default_model)
- `max_tokens`: Maximum tokens to generate
- `temperature`: Sampling temperature

**Dependencies**:
- `_call_openai()` method

**Returns**: Dictionary with `content`, `tokens_used`, `model`, `cost`, `error`

#### extract_image_prompts()
**Purpose**: Extract image prompts from content using AI.

**Parameters**:
- `content`: Article content text
- `title`: Article title
- `max_images`: Maximum number of images to extract prompts for
- `account`: Optional account object

**Dependencies**:
- `AIPrompt` model (for image prompt extraction template)
- `get_default_prompt()` utility function
- `_call_openai()` method
- `_extract_json_from_response()` method

**Returns**: Dictionary with `featured_prompt` and `in_article_prompts` array

#### generate_image()
**Purpose**: Generate images using OpenAI DALL-E or Runware.

**Parameters**:
- `prompt`: Image generation prompt
- `model`: Image model name (dall-e-3, dall-e-2, or runware model)
- `size`: Image size (e.g., "1024x1024")
- `n`: Number of images to generate
- `account`: Optional account object

**Dependencies**:
- `IntegrationSettings` model (for API keys)
- OpenAI API or Runware API

**Returns**: Dictionary with `image_url`, `revised_prompt`, `cost`, `provider`, `model`

### Internal Methods

#### _call_openai()
**Purpose**: Make HTTP request to OpenAI API.

**Parameters**:
- `prompt`: The prompt text
- `model`: Model name
- `max_tokens`: Maximum tokens
- `temperature`: Sampling temperature
- `response_format`: Optional JSON response format
- `response_steps`: Optional list for step tracking

**Dependencies**:
- `requests` library
- OpenAI API endpoint

**Returns**: Dictionary with `content`, `input_tokens`, `output_tokens`, `total_tokens`, `model`, `cost`, `error`, `api_id`

#### _extract_json_from_response()
**Purpose**: Extract JSON from OpenAI response (handles various formats).

**Parameters**:
- `response_text`: Raw response text from OpenAI

**Dependencies**:
- `json` library
- `re` library (for regex patterns)

**Returns**: Parsed JSON dictionary or None

#### _get_api_key()
**Purpose**: Get API key from IntegrationSettings or Django settings.

**Parameters**:
- `integration_type`: Type of integration ('openai' or 'runware')
- `account`: Optional account object

**Dependencies**:
- `IntegrationSettings` model
- Django settings

**Returns**: API key string or None

#### _get_model()
**Purpose**: Get default model from IntegrationSettings or Django settings.

**Parameters**:
- `integration_type`: Type of integration ('openai')
- `account`: Optional account object

**Dependencies**:
- `IntegrationSettings` model
- Django settings

**Returns**: Model name string or default

#### get_prompt()
**Purpose**: Get prompt template from AIPrompt model or default.

**Parameters**:
- `prompt_type`: Type of prompt ('clustering', 'ideas', 'content_generation', etc.)
- `account`: Optional account object

**Dependencies**:
- `AIPrompt` model
- `get_default_prompt()` utility function

**Returns**: Prompt template string

---

## Planner Module AI Functions

### 1. Auto Cluster Keywords

**Function Name**: `auto_cluster`

**API Endpoint**: `POST /v1/planner/keywords/auto_cluster/`

**ViewSet Action**: `KeywordViewSet.auto_cluster()`

**Celery Task**: `auto_cluster_keywords_task()`

**Core Function**: `_auto_cluster_keywords_core()`

**AIProcessor Method**: `AIProcessor.cluster_keywords()`

**Purpose**: Automatically group related keywords into semantic clusters using AI.

#### Complete Workflow (16 Steps)

**Request Phase (Steps 1-7)**:
1. **API Endpoint Validation**: Validate keyword IDs array
2. **Account/Site Extraction**: Extract account and site from request
3. **Task Queuing**: Queue Celery task
4. **Keyword Loading & Validation**: Load keywords from database
5. **Relationship Validation**: Validate account, site, sector relationships
6. **AIProcessor Creation**: Create AIProcessor instance
7. **AI Call Preparation**: Format keywords for AI processing

**AI Processing Phase (Steps 8-12)**:
8. **HTTP Response Validation**: Validate OpenAI API response
9. **Content Extraction**: Extract content from response
10. **Cost Calculation**: Calculate API cost
11. **JSON Extraction & Parsing**: Extract and parse JSON from response
12. **Cluster Data Validation**: Validate cluster data structure

**Database Phase (Steps 13-16)**:
13. **Database Transaction Start**: Begin atomic transaction
14. **Cluster Creation/Update**: Create or update clusters
15. **Keyword Matching & Assignment**: Match keywords to clusters
16. **Metrics Recalculation & Commit**: Recalculate cluster metrics and commit

#### Dependencies

**Models**:
- `Keywords` model
- `Clusters` model
- `Account` model
- `Site` model
- `Sector` model
- `IntegrationSettings` model (for OpenAI API key)
- `AIPrompt` model (for clustering prompt template)
- `Plan` model (for plan limits)

**Functions**:
- `KeywordViewSet.auto_cluster()` (API endpoint)
- `auto_cluster_keywords_task()` (Celery task)
- `_auto_cluster_keywords_core()` (core function)
- `AIProcessor.cluster_keywords()` (AI processing)

**Files**:
- `backend/igny8_core/modules/planner/views.py`
- `backend/igny8_core/modules/planner/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`

### 2. Auto Generate Ideas

**Function Name**: `auto_generate_ideas`

**API Endpoint**: `POST /v1/planner/clusters/auto_generate_ideas/`

**ViewSet Action**: `ClusterViewSet.auto_generate_ideas()`

**Celery Task**: `auto_generate_ideas_task()`

**Core Function**: `_generate_single_idea_core()`

**AIProcessor Method**: `AIProcessor.generate_ideas()`

**Purpose**: Generate content ideas for keyword clusters using AI.

#### Workflow

1. **API Endpoint**: Validate cluster IDs, extract account
2. **Task Queuing**: Queue Celery task
3. **Cluster Loading**: Load clusters from database
4. **AI Processing**: Call `AIProcessor.generate_ideas()`
5. **Idea Creation**: Create ContentIdeas records

#### Dependencies

**Models**:
- `Clusters` model
- `ContentIdeas` model
- `Keywords` model
- `Account` model
- `Site` model
- `Sector` model
- `IntegrationSettings` model (for OpenAI API key)
- `AIPrompt` model (for ideas prompt template)
- `Plan` model (for plan limits)

**Functions**:
- `ClusterViewSet.auto_generate_ideas()` (API endpoint)
- `auto_generate_ideas_task()` (Celery task)
- `_generate_single_idea_core()` (core function)
- `AIProcessor.generate_ideas()` (AI processing)

**Files**:
- `backend/igny8_core/modules/planner/views.py`
- `backend/igny8_core/modules/planner/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`

---

## Writer Module AI Functions

### 3. Auto Generate Content

**Function Name**: `auto_generate_content`

**API Endpoint**: `POST /v1/writer/tasks/auto_generate_content/`

**ViewSet Action**: `TasksViewSet.auto_generate_content()`

**Celery Task**: `auto_generate_content_task()`

**AIProcessor Method**: `AIProcessor.generate_content()`

**Purpose**: Generate complete blog post/article content using AI.

#### Workflow

1. **API Endpoint**: Validate task IDs, extract account
2. **Task Queuing**: Queue Celery task
3. **Task Loading**: Load tasks with relationships
4. **Content Generation**: Call `AIProcessor.generate_content()` for each task
5. **Content Saving**: Save generated content to Content model

#### Dependencies

**Models**:
- `Tasks` model
- `Content` model
- `ContentIdeas` model
- `Clusters` model
- `Keywords` model
- `Account` model
- `Site` model
- `Sector` model
- `IntegrationSettings` model (for OpenAI API key)
- `AIPrompt` model (for content generation prompt template)
- `AuthorProfile` model (for writing style)
- `Plan` model (for plan limits)

**Functions**:
- `TasksViewSet.auto_generate_content()` (API endpoint)
- `auto_generate_content_task()` (Celery task)
- `AIProcessor.generate_content()` (AI processing)

**Files**:
- `backend/igny8_core/modules/writer/views.py`
- `backend/igny8_core/modules/writer/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`

### 4. Auto Generate Images

**Function Name**: `auto_generate_images`

**API Endpoint**: `POST /v1/writer/tasks/auto_generate_images/`

**ViewSet Action**: `TasksViewSet.auto_generate_images()`

**Celery Task**: `auto_generate_images_task()`

**AIProcessor Methods**: `AIProcessor.extract_image_prompts()`, `AIProcessor.generate_image()`

**Purpose**: Generate images for content tasks using AI (OpenAI DALL-E or Runware).

#### Workflow

1. **API Endpoint**: Validate task IDs, extract account
2. **Task Queuing**: Queue Celery task
3. **Task Loading**: Load tasks with content
4. **Image Prompt Extraction**: Call `AIProcessor.extract_image_prompts()`
5. **Image Generation**: Call `AIProcessor.generate_image()` for each prompt
6. **Image Saving**: Save generated images to Images model

#### Dependencies

**Models**:
- `Tasks` model
- `Images` model
- `Content` model
- `Account` model
- `IntegrationSettings` model (for OpenAI/Runware API keys)
- `AIPrompt` model (for image prompt templates)
- `Plan` model (for plan limits and image model choices)

**Functions**:
- `TasksViewSet.auto_generate_images()` (API endpoint)
- `auto_generate_images_task()` (Celery task)
- `AIProcessor.extract_image_prompts()` (prompt extraction)
- `AIProcessor.generate_image()` (image generation)

**Files**:
- `backend/igny8_core/modules/writer/views.py`
- `backend/igny8_core/modules/writer/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`

### 5. Extract Image Prompts

**Function Name**: `extract_image_prompts`

**AIProcessor Method**: `AIProcessor.extract_image_prompts()`

**Purpose**: Extract image prompts from generated content for image generation.

**Note**: This is typically called internally by `auto_generate_images`, but can be called directly.

#### Dependencies

**Models**:
- `Content` model
- `AIPrompt` model (for image prompt extraction template)
- `IntegrationSettings` model (for OpenAI API key)

**Functions**:
- `AIProcessor.extract_image_prompts()` (AI processing)

**Files**:
- `backend/igny8_core/utils/ai_processor.py`

---

## System Module AI Functions

### 6. Test OpenAI Integration

**Function Name**: `test_openai`

**API Endpoint**: `POST /v1/system/settings/integrations/{id}/test_openai/`

**ViewSet Action**: `IntegrationSettingsViewSet.test_openai()`

**Purpose**: Test OpenAI API connection and validate API key.

#### Workflow

1. **Get API Key**: From request or saved IntegrationSettings
2. **Test Connection**: Call OpenAI API (`/v1/models` or `/v1/chat/completions`)
3. **Return Result**: Success or error message

#### Dependencies

**Models**:
- `IntegrationSettings` model
- `Account` model

**Functions**:
- `IntegrationSettingsViewSet.test_openai()` (API endpoint)
- `AIProcessor` class (for API key loading)

**Files**:
- `backend/igny8_core/modules/system/integration_views.py`
- `backend/igny8_core/utils/ai_processor.py`

### 7. Test Runware Integration

**Function Name**: `test_runware`

**API Endpoint**: `POST /v1/system/settings/integrations/{id}/test_runware/`

**ViewSet Action**: `IntegrationSettingsViewSet.test_runware()`

**Purpose**: Test Runware API connection and validate API key.

#### Workflow

1. **Get API Key**: From request or saved IntegrationSettings
2. **Test Connection**: Call Runware API (`/v1`)
3. **Return Result**: Success or error message

#### Dependencies

**Models**:
- `IntegrationSettings` model
- `Account` model

**Functions**:
- `IntegrationSettingsViewSet.test_runware()` (API endpoint)
- `AIProcessor` class (for API key loading)

**Files**:
- `backend/igny8_core/modules/system/integration_views.py`
- `backend/igny8_core/utils/ai_processor.py`

### 8. Generate Image (Test)

**Function Name**: `generate_image`

**API Endpoint**: `POST /v1/system/settings/integrations/{id}/generate_image/`

**ViewSet Action**: `IntegrationSettingsViewSet.generate_image()`

**Purpose**: Test image generation with OpenAI DALL-E or Runware.

#### Workflow

1. **Get Configuration**: From request or saved IntegrationSettings
2. **Generate Image**: Call `AIProcessor.generate_image()`
3. **Return Result**: Image URL or error message

#### Dependencies

**Models**:
- `IntegrationSettings` model
- `Account` model

**Functions**:
- `IntegrationSettingsViewSet.generate_image()` (API endpoint)
- `AIProcessor.generate_image()` (image generation)

**Files**:
- `backend/igny8_core/modules/system/integration_views.py`
- `backend/igny8_core/utils/ai_processor.py`

---

## Progress Tracking

### Task Progress Endpoint

**Endpoint**: `GET /v1/system/settings/task_progress/{task_id}/`

**ViewSet Action**: `IntegrationSettingsViewSet.task_progress()`

**Purpose**: Get Celery task status and progress.

**Response Format**:
```json
{
  "state": "PROGRESS",
  "meta": {
    "current": 2,
    "total": 10,
    "percentage": 20,
    "message": "Processing...",
    "request_steps": [...],
    "response_steps": [...]
  }
}
```

**States**:
- `PENDING`: Task is queued but not started
- `PROGRESS`: Task is in progress
- `SUCCESS`: Task completed successfully
- `FAILURE`: Task failed with error

### Frontend Integration

**Hook**: `useProgressModal`

**Functionality**:
- Polls task progress endpoint every 2 seconds
- Displays progress percentage and message
- Shows step-by-step logs (request_steps and response_steps)
- Auto-closes on completion

**Store**: `aiRequestLogsStore`

**Functionality**:
- Stores AI request/response logs
- Tracks step-by-step execution
- Displays logs in UI

---

## Error Handling

### Error Types

1. **API Errors**: OpenAI/Runware API errors
2. **Validation Errors**: Input validation failures
3. **Database Errors**: Database operation failures
4. **Connection Errors**: Celery/Redis connection issues

### Error Propagation

1. **API Endpoint**: Catches errors, returns error response with request_steps
2. **Celery Task**: Catches errors, updates task state to FAILURE with error details
3. **Core Function**: Catches errors, logs to request_steps/response_steps
4. **AIProcessor**: Catches errors, returns error in result dictionary

### Error Logging

- All errors logged to `request_steps` or `response_steps`
- Error details include: error type, message, function name, step number
- Frontend displays errors in AI Request/Response Logs section

---

## Dependencies

### Database Models

- `Account`: Account isolation
- `Site`, `Sector`: Site/sector hierarchy
- `Keywords`, `Clusters`, `ContentIdeas`: Planner data
- `Tasks`, `Content`, `Images`: Writer data
- `IntegrationSettings`: API keys and configuration
- `AIPrompt`: Prompt templates
- `Plan`: Plan limits

### External Services

- **OpenAI API**: GPT models for text generation
- **OpenAI DALL-E**: Image generation
- **Runware API**: Alternative image generation

### Utilities

- `get_default_prompt()`: Default prompt templates
- `AIProcessor`: Unified AI interface
- Celery: Asynchronous task processing
- Redis: Celery broker

---

## Summary

The IGNY8 AI functions system provides:

1. **Unified Interface**: Single AIProcessor class for all AI operations
2. **Account Isolation**: All AI functions respect account boundaries
3. **Progress Tracking**: Real-time progress updates with step-by-step logs
4. **Error Handling**: Comprehensive error logging and propagation
5. **Flexible Configuration**: Account-specific API keys and prompts
6. **Asynchronous Processing**: Celery tasks for long-running operations
7. **Multiple Providers**: Support for OpenAI and Runware
8. **Complete Workflows**: End-to-end workflows from API to database

All AI functions integrate seamlessly with the account/site/user/plan architecture and provide detailed logging for debugging and monitoring.