igny8/docs/ai-docs/AI_MASTER_ARCHITECTURE.md

AI Master Architecture Document

Clustering, Idea Generation, and Content Generation

Version: 1.0
Date: 2025-01-XX
Scope: Complete architecture for 3 verified AI functions (clustering, idea generation, content generation)

---

Table of Contents

1. Common Architecture
2. Auto Cluster Keywords
3. Generate Ideas
4. Generate Content

---

1. Common Architecture

1.1 Core Framework Files

Entry Point

• File: backend/igny8_core/ai/tasks.py
• Function: run_ai_task
• Purpose: Unified Celery task entrypoint for all AI functions
• Parameters: function_name (str), payload (dict), account_id (int)
• Flow: Loads function from registry → Creates AIEngine → Executes function

Engine Orchestrator

• File: backend/igny8_core/ai/engine.py
• Class: AIEngine
• Purpose: Central orchestrator managing lifecycle, progress, logging, cost tracking
• Methods:
  • execute - Main execution pipeline (6 phases: INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
  • _handle_error - Centralized error handling
  • _log_to_database - Logs to AITaskLog model
  • Helper methods: _get_input_description, _build_validation_message, _get_prep_message, _get_ai_call_message, _get_parse_message, _get_parse_message_with_count, _get_save_message, _calculate_credits_for_clustering

Base Function Class

• File: backend/igny8_core/ai/base.py
• Class: BaseAIFunction
• Purpose: Abstract base class defining interface for all AI functions
• Abstract Methods:
  • get_name - Returns function name (e.g., 'auto_cluster')
  • prepare - Loads and prepares data
  • build_prompt - Builds AI prompt
  • parse_response - Parses AI response
  • save_output - Saves results to database
• Optional Methods:
  • get_metadata - Returns display name, description, phases
  • get_max_items - Returns max items limit (or None)
  • validate - Validates input payload (default: checks for 'ids')
  • get_model - Returns model override (default: None, uses account default)

Function Registry

• File: backend/igny8_core/ai/registry.py
• Functions:
  • register_function - Registers function class
  • register_lazy_function - Registers lazy loader
  • get_function - Gets function class by name (lazy loads if needed)
  • get_function_instance - Gets function instance by name
  • list_functions - Lists all registered functions
• Lazy Loaders:
  • _load_auto_cluster - Loads AutoClusterFunction
  • _load_generate_ideas - Loads GenerateIdeasFunction
  • _load_generate_content - Loads GenerateContentFunction

AI Core Handler

• File: backend/igny8_core/ai/ai_core.py
• Class: AICore
• Purpose: Centralized AI request handler for all text generation
• Methods:
  • run_ai_request - Makes API call to OpenAI/Runware
  • extract_json - Extracts JSON from response (handles markdown code blocks)

Prompt Registry

• File: backend/igny8_core/ai/prompts.py
• Class: PromptRegistry
• Purpose: Centralized prompt management with hierarchical resolution
• Method: get_prompt - Gets prompt with resolution order:
  1. Task-level prompt_override (if exists)
  2. DB prompt for (account, function)
  3. Default fallback from DEFAULT_PROMPTS registry
• Prompt Types:
  • clustering - For auto_cluster function
  • ideas - For generate_ideas function
  • content_generation - For generate_content function
• Context Placeholders:
  • [IGNY8_KEYWORDS] - Replaced with keyword list
  • [IGNY8_CLUSTERS] - Replaced with cluster list
  • [IGNY8_CLUSTER_KEYWORDS] - Replaced with cluster keywords
  • [IGNY8_IDEA] - Replaced with idea data
  • [IGNY8_CLUSTER] - Replaced with cluster data
  • [IGNY8_KEYWORDS] - Replaced with keywords (for content)

Model Settings

• File: backend/igny8_core/ai/settings.py
• Constants:
  • MODEL_CONFIG - Model configurations per function (model, max_tokens, temperature, response_format)
  • FUNCTION_ALIASES - Legacy function name mappings
• Functions:
  • get_model_config - Gets model config for function (reads from IntegrationSettings if account provided)
  • get_model - Gets model name for function
  • get_max_tokens - Gets max tokens for function
  • get_temperature - Gets temperature for function

Validators

• File: backend/igny8_core/ai/validators.py
• Functions:
  • validate_ids - Validates 'ids' array in payload
  • validate_keywords_exist - Validates keywords exist in database
  • validate_cluster_exists - Validates cluster exists
  • validate_tasks_exist - Validates tasks exist
  • validate_cluster_limits - Validates plan limits (currently disabled - always returns valid)
  • validate_api_key - Validates API key is configured
  • validate_model - Validates model is in supported list
  • validate_image_size - Validates image size for model

Progress Tracking

• File: backend/igny8_core/ai/tracker.py
• Classes:
  • StepTracker - Tracks request/response steps
  • ProgressTracker - Tracks Celery progress updates
  • CostTracker - Tracks API costs and tokens
  • ConsoleStepTracker - Console-based step logging

Database Logging

• File: backend/igny8_core/ai/models.py
• Model: AITaskLog
• Fields: task_id, function_name, account, phase, message, status, duration, cost, tokens, request_steps, response_steps, error, payload, result

1.2 Execution Flow (All Functions)

1. API Endpoint (views.py)
   ↓
2. run_ai_task (tasks.py)
   - Gets account from account_id
   - Gets function instance from registry
   - Creates AIEngine
   ↓
3. AIEngine.execute (engine.py)
   Phase 1: INIT (0-10%)
   - Calls function.validate()
   - Updates progress tracker
   ↓
   Phase 2: PREP (10-25%)
   - Calls function.prepare()
   - Calls function.build_prompt()
   - Updates progress tracker
   ↓
   Phase 3: AI_CALL (25-70%)
   - Gets model config from settings
   - Calls AICore.run_ai_request()
   - Tracks cost and tokens
   - Updates progress tracker
   ↓
   Phase 4: PARSE (70-85%)
   - Calls function.parse_response()
   - Updates progress tracker
   ↓
   Phase 5: SAVE (85-98%)
   - Calls function.save_output()
   - Logs credit usage
   - Updates progress tracker
   ↓
   Phase 6: DONE (98-100%)
   - Logs to AITaskLog
   - Returns result

---

2. Auto Cluster Keywords

2.1 Function Implementation

• File: backend/igny8_core/ai/functions/auto_cluster.py
• Class: AutoClusterFunction
• Inherits: BaseAIFunction

2.2 API Endpoint

• File: backend/igny8_core/modules/planner/views.py
• ViewSet: KeywordViewSet
• Action: auto_cluster
• Method: POST
• URL Path: /v1/planner/keywords/auto_cluster/
• Payload:
  • ids (list[int]) - Keyword IDs to cluster
  • sector_id (int, optional) - Sector ID for filtering
• Response:
  • success (bool)
  • task_id (str) - Celery task ID if async
  • clusters_created (int) - Number of clusters created
  • keywords_updated (int) - Number of keywords updated
  • message (str)

2.3 Function Methods

get_name()

• Returns: 'auto_cluster'

get_metadata()

• Returns: Dict with display_name, description, phases (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)

get_max_items()

• Returns: None (no limit)

validate(payload, account)

• Validates:
  • Calls validate_ids to check for 'ids' array
  • Calls validate_keywords_exist to verify keywords exist
• Returns: Dict with valid (bool) and optional error (str)

prepare(payload, account)

• Loads:
  • Keywords from database (filters by ids, account, optional sector_id)
  • Uses select_related for: account, site, site__account, sector, sector__site
• Returns: Dict with:
  • keywords (list[Keyword objects])
  • keyword_data (list[dict]) - Formatted data with: id, keyword, volume, difficulty, intent
  • sector_id (int, optional)

build_prompt(data, account)

• Gets Prompt:
  • Calls PromptRegistry.get_prompt(function_name='auto_cluster', account, context)
  • Context includes: KEYWORDS (formatted keyword list), optional SECTOR (sector name)
• Formatting:
  • Formats keywords as: "- {keyword} (Volume: {volume}, Difficulty: {difficulty}, Intent: {intent})"
  • Replaces [IGNY8_KEYWORDS] placeholder
  • Adds JSON mode instruction if not present
• Returns: Prompt string

parse_response(response, step_tracker)

• Parsing:
  • Tries direct JSON parse first
  • Falls back to AICore.extract_json() if needed (handles markdown code blocks)
• Extraction:
  • Extracts clusters array from JSON
  • Handles both dict with 'clusters' key and direct array
• Returns: List[Dict] with cluster data:
  • name (str) - Cluster name
  • description (str) - Cluster description
  • keywords (list[str]) - List of keyword strings

save_output(parsed, original_data, account, progress_tracker, step_tracker)

• Input:
  • parsed - List of cluster dicts from parse_response
  • original_data - Dict from prepare() with keywords and sector_id
• Process:
  • Gets account, site, sector from first keyword
  • For each cluster in parsed:
    • Gets or creates Clusters record:
      • Fields: name, description, account, site, sector, status='active'
      • Uses get_or_create with name + account + site + sector
    • Matches keywords (case-insensitive):
      • Normalizes cluster keywords and available keywords to lowercase
      • Updates matched Keywords records:
        • Sets cluster foreign key
        • Sets status='mapped'
  • Recalculates cluster metrics:
    • keywords_count - Count of keywords in cluster
    • volume - Sum of keyword volumes (uses volume_override if available, else seed_keyword__volume)
• Returns: Dict with:
  • count (int) - Clusters created
  • clusters_created (int) - Clusters created
  • keywords_updated (int) - Keywords updated

2.4 Database Models

Keywords Model

• File: backend/igny8_core/modules/planner/models.py
• Model: Keywords
• Fields Used:
  • id - Keyword ID
  • seed_keyword (ForeignKey) - Reference to SeedKeyword
  • keyword (property) - Gets keyword text from seed_keyword
  • volume (property) - Gets volume from volume_override or seed_keyword
  • difficulty (property) - Gets difficulty from difficulty_override or seed_keyword
  • intent (property) - Gets intent from seed_keyword
  • cluster (ForeignKey) - Assigned cluster
  • status - Status ('active', 'pending', 'mapped', 'archived')
  • account, site, sector - From SiteSectorBaseModel

Clusters Model

• File: backend/igny8_core/modules/planner/models.py
• Model: Clusters
• Fields Used:
  • name - Cluster name (unique)
  • description - Cluster description
  • keywords_count - Count of keywords (recalculated)
  • volume - Sum of keyword volumes (recalculated)
  • status - Status ('active')
  • account, site, sector - From SiteSectorBaseModel

2.5 AI Response Format

Expected JSON:

{
  "clusters": [
    {
      "name": "Cluster Name",
      "description": "Cluster description",
      "keywords": ["keyword1", "keyword2", "keyword3"]
    }
  ]
}

2.6 Progress Messages

• INIT: "Validating {keyword1}, {keyword2}, {keyword3} and {X} more keywords" (shows first 3, then count)
• PREP: "Loading {count} keyword(s)"
• AI_CALL: "Generating clusters with Igny8 Semantic SEO Model"
• PARSE: "{count} cluster(s) created"
• SAVE: "Saving {count} cluster(s)"

---

3. Generate Ideas

3.1 Function Implementation

• File: backend/igny8_core/ai/functions/generate_ideas.py
• Class: GenerateIdeasFunction
• Inherits: BaseAIFunction

3.2 API Endpoint

• File: backend/igny8_core/modules/planner/views.py
• ViewSet: ClusterViewSet
• Action: auto_generate_ideas
• Method: POST
• URL Path: /v1/planner/clusters/auto_generate_ideas/
• Payload:
  • ids (list[int]) - Cluster IDs (max 10)
• Response:
  • success (bool)
  • task_id (str) - Celery task ID if async
  • ideas_created (int) - Number of ideas created
  • message (str)

3.3 Function Methods

get_name()

• Returns: 'generate_ideas'

get_metadata()

• Returns: Dict with display_name, description, phases (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)

get_max_items()

• Returns: 10 (max clusters per generation)

validate(payload, account)

• Validates:
  • Calls super().validate() to check for 'ids' array and max_items limit
  • Calls validate_cluster_exists for first cluster ID
  • Calls validate_cluster_limits for plan limits (currently disabled)
• Returns: Dict with valid (bool) and optional error (str)

prepare(payload, account)

• Loads:
  • Clusters from database (filters by ids, account)
  • Uses select_related for: sector, account, site, sector__site
  • Uses prefetch_related for: keywords
• Gets Keywords:
  • For each cluster, loads Keywords with select_related('seed_keyword')
  • Extracts keyword text from seed_keyword.keyword
• Returns: Dict with:
  • clusters (list[Cluster objects])
  • cluster_data (list[dict]) - Formatted data with: id, name, description, keywords (list[str])
  • account (Account object)

build_prompt(data, account)

• Gets Prompt:
  • Calls PromptRegistry.get_prompt(function_name='generate_ideas', account, context)
  • Context includes:
    • CLUSTERS - Formatted cluster list: "Cluster ID: {id} | Name: {name} | Description: {description}"
    • CLUSTER_KEYWORDS - Formatted cluster keywords: "Cluster ID: {id} | Name: {name} | Keywords: {keyword1}, {keyword2}"
• Replaces Placeholders:
  • [IGNY8_CLUSTERS] → clusters_text
  • [IGNY8_CLUSTER_KEYWORDS] → cluster_keywords_text
• Returns: Prompt string

parse_response(response, step_tracker)

• Parsing:
  • Calls AICore.extract_json() to extract JSON from response
  • Validates 'ideas' key exists in JSON
• Returns: List[Dict] with idea data:
  • title (str) - Idea title
  • description (str or dict) - Idea description (can be JSON string)
  • content_type (str) - Content type ('blog_post', 'article', etc.)
  • content_structure (str) - Content structure ('cluster_hub', 'supporting_page', etc.)
  • cluster_id (int, optional) - Cluster ID reference
  • cluster_name (str, optional) - Cluster name reference
  • estimated_word_count (int) - Estimated word count
  • covered_keywords or target_keywords (str) - Target keywords

save_output(parsed, original_data, account, progress_tracker, step_tracker)

• Input:
  • parsed - List of idea dicts from parse_response
  • original_data - Dict from prepare() with clusters and cluster_data
• Process:
  • For each idea in parsed:
    • Matches cluster:
      • First tries by cluster_id from AI response
      • Falls back to cluster_name matching
      • Last resort: position-based matching (first idea → first cluster)
    • Gets site from cluster (or cluster.sector.site)
    • Handles description:
      • If dict, converts to JSON string
      • If not string, converts to string
    • Creates ContentIdeas record:
      • Fields:
        • idea_title - From title
        • description - Processed description
        • content_type - From content_type (default: 'blog_post')
        • content_structure - From content_structure (default: 'supporting_page')
        • target_keywords - From covered_keywords or target_keywords
        • keyword_cluster - Matched cluster
        • estimated_word_count - From estimated_word_count (default: 1500)
        • status - 'new'
        • account, site, sector - From cluster
• Returns: Dict with:
  • count (int) - Ideas created
  • ideas_created (int) - Ideas created

3.4 Database Models

Clusters Model

• File: backend/igny8_core/modules/planner/models.py
• Model: Clusters
• Fields Used:
  • id - Cluster ID
  • name - Cluster name
  • description - Cluster description
  • keywords (related_name) - Related Keywords
  • account, site, sector - From SiteSectorBaseModel

ContentIdeas Model

• File: backend/igny8_core/modules/planner/models.py
• Model: ContentIdeas
• Fields Used:
  • idea_title - Idea title
  • description - Idea description (can be JSON string)
  • content_type - Content type ('blog_post', 'article', 'guide', 'tutorial')
  • content_structure - Content structure ('cluster_hub', 'landing_page', 'pillar_page', 'supporting_page')
  • target_keywords - Target keywords string
  • keyword_cluster (ForeignKey) - Related cluster
  • estimated_word_count - Estimated word count
  • status - Status ('new', 'scheduled', 'published')
  • account, site, sector - From SiteSectorBaseModel

3.5 AI Response Format

Expected JSON:

{
  "ideas": [
    {
      "title": "Idea Title",
      "description": "Idea description or JSON structure",
      "content_type": "blog_post",
      "content_structure": "supporting_page",
      "cluster_id": 1,
      "cluster_name": "Cluster Name",
      "estimated_word_count": 1500,
      "covered_keywords": "keyword1, keyword2"
    }
  ]
}

3.6 Progress Messages

• INIT: "Verifying cluster integrity"
• PREP: "Loading cluster keywords"
• AI_CALL: "Generating ideas with Igny8 Semantic AI"
• PARSE: "{count} high-opportunity idea(s) generated"
• SAVE: "Content Outline for Ideas generated"

---

4. Generate Content

4.1 Function Implementation

• File: backend/igny8_core/ai/functions/generate_content.py
• Class: GenerateContentFunction
• Inherits: BaseAIFunction

4.2 API Endpoint

• File: backend/igny8_core/modules/writer/views.py
• ViewSet: TasksViewSet
• Action: auto_generate_content
• Method: POST
• URL Path: /v1/writer/tasks/auto_generate_content/
• Payload:
  • ids (list[int]) - Task IDs (max 10)
• Response:
  • success (bool)
  • task_id (str) - Celery task ID if async
  • tasks_updated (int) - Number of tasks updated
  • message (str)

4.3 Function Methods

get_name()

• Returns: 'generate_content'

get_metadata()

• Returns: Dict with display_name, description, phases (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)

get_max_items()

• Returns: 50 (max tasks per batch)

validate(payload, account)

• Validates:
  • Calls super().validate() to check for 'ids' array and max_items limit
  • Calls validate_tasks_exist to verify tasks exist
• Returns: Dict with valid (bool) and optional error (str)

prepare(payload, account)

• Loads:
  • Tasks from database (filters by ids, account)
  • Uses select_related for: account, site, sector, cluster, idea
• Returns: List[Task objects]

build_prompt(data, account)

• Input: Can be single Task or list[Task] (handles first task if list)
• Builds Idea Data:
  • title - From task.title
  • description - From task.description
  • outline - From task.idea.description (handles JSON structure):
    • If JSON, formats as: "## {H2 heading}\n### {H3 subheading}\nContent Type: {type}\nDetails: {details}"
    • If plain text, uses as-is
  • structure - From task.idea.content_structure or task.content_structure
  • type - From task.idea.content_type or task.content_type
  • estimated_word_count - From task.idea.estimated_word_count
• Builds Cluster Data:
  • cluster_name - From task.cluster.name
  • description - From task.cluster.description
  • status - From task.cluster.status
• Builds Keywords Data:
  • From task.keywords (legacy) or task.idea.target_keywords
• Gets Prompt:
  • Calls PromptRegistry.get_prompt(function_name='generate_content', account, task, context)
  • Context includes:
    • IDEA - Formatted idea data string
    • CLUSTER - Formatted cluster data string
    • KEYWORDS - Keywords string
• Returns: Prompt string

parse_response(response, step_tracker)

• Parsing:
  • First tries JSON parse:
    • If successful and dict, returns dict
  • Falls back to plain text:
    • Calls normalize_content() from content_normalizer to convert to HTML
    • Returns dict with content field
• Returns: Dict with:
  • If JSON:
    • content (str) - HTML content
    • title (str, optional) - Content title
    • meta_title (str, optional) - Meta title
    • meta_description (str, optional) - Meta description
    • word_count (int, optional) - Word count
    • primary_keyword (str, optional) - Primary keyword
    • secondary_keywords (list, optional) - Secondary keywords
    • tags (list, optional) - Tags
    • categories (list, optional) - Categories
  • If Plain Text:
    • content (str) - Normalized HTML content

save_output(parsed, original_data, account, progress_tracker, step_tracker)

• Input:
  • parsed - Dict from parse_response
  • original_data - Task object or list[Task] (handles first task if list)
• Process:
  • Extracts content fields from parsed dict:
    • content_html - From content field
    • title - From title or task.title
    • meta_title - From meta_title or task.meta_title or task.title
    • meta_description - From meta_description or task.meta_description or task.description
    • word_count - From word_count or calculated from content
    • primary_keyword - From primary_keyword
    • secondary_keywords - From secondary_keywords (converts to list if needed)
    • tags - From tags (converts to list if needed)
    • categories - From categories (converts to list if needed)
  • Calculates word count if not provided:
    • Strips HTML tags and counts words
  • Gets or creates Content record:
    • Uses get_or_create with task (OneToOne relationship)
    • Defaults: html_content, word_count, status='draft', account, site, sector
  • Updates Content fields:
    • html_content - Content HTML
    • word_count - Word count
    • title - Content title
    • meta_title - Meta title
    • meta_description - Meta description
    • primary_keyword - Primary keyword
    • secondary_keywords - Secondary keywords (JSONField)
    • tags - Tags (JSONField)
    • categories - Categories (JSONField)
    • status - Always 'draft' for newly generated content
    • metadata - Extra fields from parsed dict (excludes standard fields)
    • account, site, sector, task - Aligned from task
  • Updates Tasks record:
    • Sets status='completed'
    • Updates updated_at
• Returns: Dict with:
  • count (int) - Tasks updated (always 1 per task)
  • tasks_updated (int) - Tasks updated
  • word_count (int) - Word count

4.4 Database Models

Tasks Model

• File: backend/igny8_core/modules/writer/models.py
• Model: Tasks
• Fields Used:
  • id - Task ID
  • title - Task title
  • description - Task description
  • keywords - Keywords string (legacy)
  • cluster (ForeignKey) - Related cluster
  • idea (ForeignKey) - Related ContentIdeas
  • content_structure - Content structure
  • content_type - Content type
  • status - Status ('queued', 'completed')
  • meta_title - Meta title
  • meta_description - Meta description
  • account, site, sector - From SiteSectorBaseModel

Content Model

• File: backend/igny8_core/modules/writer/models.py
• Model: Content
• Fields Used:
  • task (OneToOneField) - Related task
  • html_content - HTML content
  • word_count - Word count
  • title - Content title
  • meta_title - Meta title
  • meta_description - Meta description
  • primary_keyword - Primary keyword
  • secondary_keywords (JSONField) - Secondary keywords list
  • tags (JSONField) - Tags list
  • categories (JSONField) - Categories list
  • status - Status ('draft', 'review', 'published')
  • metadata (JSONField) - Additional metadata
  • account, site, sector - From SiteSectorBaseModel (auto-set from task)

4.5 AI Response Format

Expected JSON:

{
  "content": "<html>Content HTML</html>",
  "title": "Content Title",
  "meta_title": "Meta Title",
  "meta_description": "Meta description",
  "word_count": 1500,
  "primary_keyword": "primary keyword",
  "secondary_keywords": ["keyword1", "keyword2"],
  "tags": ["tag1", "tag2"],
  "categories": ["category1"]
}

Or Plain Text:

Plain text content that will be normalized to HTML

4.6 Progress Messages

• INIT: "Validating task"
• PREP: "Preparing content idea"
• AI_CALL: "Writing article with Igny8 Semantic AI"
• PARSE: "{count} article(s) created"
• SAVE: "Saving article"

---

5. Change Guide

5.1 Where to Change Validation Logic

• File: backend/igny8_core/ai/validators.py
• Functions: validate_ids, validate_keywords_exist, validate_cluster_exists, validate_tasks_exist
• Or: Override validate() method in function class

5.2 Where to Change Data Loading

• File: Function-specific file (e.g., auto_cluster.py)
• Method: prepare()
• Change: Modify queryset filters, select_related, prefetch_related

5.3 Where to Change Prompts

• File: backend/igny8_core/ai/prompts.py
• Method: PromptRegistry.get_prompt()
• Change: Modify DEFAULT_PROMPTS dict or update database prompts

5.4 Where to Change Model Configuration

• File: backend/igny8_core/ai/settings.py
• Constant: MODEL_CONFIG
• Change: Update model, max_tokens, temperature, response_format per function

5.5 Where to Change Response Parsing

• File: Function-specific file (e.g., generate_content.py)
• Method: parse_response()
• Change: Modify JSON extraction or plain text handling

5.6 Where to Change Database Saving

• File: Function-specific file (e.g., auto_cluster.py)
• Method: save_output()
• Change: Modify model creation/update logic, field mappings

5.7 Where to Change Progress Messages

• File: backend/igny8_core/ai/engine.py
• Methods: _get_prep_message(), _get_ai_call_message(), _get_parse_message(), _get_save_message()
• Or: Override in function class get_metadata() phases

5.8 Where to Change Error Handling

• File: backend/igny8_core/ai/engine.py
• Method: _handle_error()
• Change: Modify error logging, error response format

---

6. Dependencies

6.1 Function Dependencies

• All functions depend on: BaseAIFunction, AICore, PromptRegistry, get_model_config
• Clustering depends on: Keywords, Clusters models
• Ideas depends on: Clusters, ContentIdeas, Keywords models
• Content depends on: Tasks, Content, ContentIdeas, Clusters models

6.2 External Dependencies

• Celery: For async task execution (run_ai_task)
• OpenAI API: For AI text generation (via AICore.run_ai_request)
• Django ORM: For database operations
• IntegrationSettings: For account-specific model configuration

---

7. Key Relationships

7.1 Clustering Flow

Keywords → Clusters (many-to-one)
- Keywords.cluster (ForeignKey)
- Clusters.keywords (related_name)

7.2 Ideas Flow

Clusters → ContentIdeas (one-to-many)
- ContentIdeas.keyword_cluster (ForeignKey)
- Clusters.ideas (related_name, if exists)

7.3 Content Flow

Tasks → Content (one-to-one)
- Content.task (OneToOneField)
- Tasks.content_record (related_name)

Tasks → ContentIdeas (many-to-one)
- Tasks.idea (ForeignKey)
- ContentIdeas.tasks (related_name)

Tasks → Clusters (many-to-one)
- Tasks.cluster (ForeignKey)
- Clusters.tasks (related_name)

---

8. Notes

• All functions use the same execution pipeline through AIEngine.execute()
• Progress tracking is handled automatically by AIEngine
• Cost tracking is handled automatically by CostTracker
• Database logging is handled automatically by AITaskLog
• Model configuration can be overridden per account via IntegrationSettings
• Prompts can be overridden per account via database prompts
• All functions support both async (Celery) and sync execution
• Error handling is centralized in AIEngine._handle_error()