salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
65a7d00fba857122870685cd8b82e1b24c25241c
igny8/planner-writer-workflow.md
alorig 0bd603f925 docs
2025-11-24 11:52:43 +05:00

50 KiB
Raw Blame History

IGNY8 Planner → Writer Workflow Documentation

Complete Technical Implementation Guide
Date: November 24, 2025
Version: 1.0

Table of Contents

  1. Overview
  2. Database Tables & Models
  3. Frontend Pages & Components
  4. Backend API Endpoints
  5. Complete Workflow
  6. AI Functions
  7. Flowchart Diagrams

Overview

The IGNY8 content workflow consists of two primary modules:

  • Planner Module: Keyword research, clustering, and content ideation
  • Writer Module: Task management, content generation, and publishing

Data flows sequentially through these stages:

Keywords → Clusters → Ideas → Tasks → Content → Images

Database Tables & Models

Planner Module Tables

1. igny8_keywords (Keywords Model)

Location: backend/igny8_core/business/planning/models.py

Field Type Description Choices
id Integer Primary key -
seed_keyword_id ForeignKey Reference to global SeedKeyword -
volume_override Integer Site-specific volume override -
difficulty_override Integer Site-specific difficulty override -
cluster_id ForeignKey Parent cluster Clusters
status CharField(50) Keyword status active, pending, archived
account ForeignKey Owner account -
site ForeignKey Parent site -
sector ForeignKey Parent sector -
created_at DateTime Creation timestamp -
updated_at DateTime Last update timestamp -

Indexes:

  • seed_keyword
  • status
  • cluster
  • site, sector
  • seed_keyword, site, sector (unique)

Properties (Computed from SeedKeyword):

  • keyword - Text from seed_keyword.keyword
  • volume - volume_override or seed_keyword.volume
  • difficulty - difficulty_override or seed_keyword.difficulty
  • intent - seed_keyword.intent

2. igny8_clusters (Clusters Model)

Location: backend/igny8_core/business/planning/models.py

Field Type Description Choices
id Integer Primary key -
name CharField(255) Cluster name (unique) -
description TextField Cluster description -
keywords_count Integer Number of keywords -
volume Integer Total search volume -
mapped_pages Integer Number of mapped pages -
status CharField(50) Cluster status active, archived
context_type CharField(50) Cluster dimension topic, attribute, service_line
dimension_meta JSONField Extended metadata -
account ForeignKey Owner account -
site ForeignKey Parent site -
sector ForeignKey Parent sector -
created_at DateTime Creation timestamp -
updated_at DateTime Last update timestamp -

Indexes:

  • name
  • status
  • site, sector
  • context_type

Relationships:

  • keywords (1:M) - Related Keywords
  • ideas (1:M) - Related ContentIdeas
  • tasks (1:M) - Related Tasks
  • contents (1:M) - Related Content

3. igny8_content_ideas (ContentIdeas Model)

Location: backend/igny8_core/business/planning/models.py

Field Type Description Choices
id Integer Primary key -
idea_title CharField(255) Idea title -
description TextField Idea description -
target_keywords CharField(500) Comma-separated keywords (legacy) -
keyword_objects ManyToManyField Keywords linked to idea Keywords
keyword_cluster_id ForeignKey Parent cluster Clusters
taxonomy_id ForeignKey Optional taxonomy association SiteBlueprintTaxonomy
status CharField(50) Idea status new, scheduled, published
estimated_word_count Integer Target word count -
site_entity_type CharField(50) Target entity type post, page, product, service, taxonomy_term
cluster_role CharField(50) Role within cluster hub, supporting, attribute
account ForeignKey Owner account -
site ForeignKey Parent site -
sector ForeignKey Parent sector -
created_at DateTime Creation timestamp -
updated_at DateTime Last update timestamp -

Indexes:

  • idea_title
  • status
  • keyword_cluster
  • site_entity_type
  • cluster_role
  • site, sector

Writer Module Tables

4. igny8_tasks (Tasks Model)

Location: backend/igny8_core/business/content/models.py

Field Type Description Choices
id Integer Primary key -
title CharField(255) Task title -
description TextField Task description -
keywords CharField(500) Comma-separated keywords (legacy) -
cluster_id ForeignKey Parent cluster Clusters
keyword_objects ManyToManyField Keywords linked to task Keywords
idea_id ForeignKey Source idea ContentIdeas
status CharField(50) Task status queued, in_progress, completed, failed
entity_type CharField(50) Content entity type post, page, product, service, taxonomy_term
taxonomy_id ForeignKey Taxonomy association SiteBlueprintTaxonomy
cluster_role CharField(50) Role within cluster hub, supporting, attribute
account ForeignKey Owner account -
site ForeignKey Parent site -
sector ForeignKey Parent sector -
created_at DateTime Creation timestamp -
updated_at DateTime Last update timestamp -

Indexes:

  • title
  • status
  • cluster
  • entity_type
  • cluster_role
  • site, sector

Relationships:

  • content_record (1:1) - Related Content (OneToOneField)

5. igny8_content (Content Model)

Location: backend/igny8_core/business/content/models.py

Field Type Description Choices
id Integer Primary key -
task_id OneToOneField Parent task Tasks
html_content TextField Final HTML content -
word_count Integer Content word count -
metadata JSONField Additional metadata -
title CharField(255) Content title -
meta_title CharField(255) SEO meta title -
meta_description TextField SEO meta description -
primary_keyword CharField(255) Primary keyword -
secondary_keywords JSONField List of secondary keywords -
status CharField(50) Content workflow status draft, review, publish
source CharField(50) Content source igny8, wordpress, shopify, custom
sync_status CharField(50) Sync status native, imported, synced
external_id CharField(255) External platform ID -
external_url URLField External platform URL -
external_type CharField(100) External post type -
sync_metadata JSONField Platform-specific sync metadata -
internal_links JSONField Internal links (linker) -
linker_version Integer Linker processing version -
optimizer_version Integer Optimizer processing version -
optimization_scores JSONField Optimization scores -
entity_type CharField(50) Content entity type post, page, product, service, taxonomy_term
content_format CharField(50) Content format (posts only) article, listicle, guide, comparison, review, roundup
cluster_role CharField(50) Role within cluster hub, supporting, attribute
json_blocks JSONField Structured content blocks -
structure_data JSONField Content structure data -
taxonomies ManyToManyField Associated taxonomy terms ContentTaxonomy
cluster_id ForeignKey Primary semantic cluster Clusters
account ForeignKey Owner account -
site ForeignKey Parent site -
sector ForeignKey Parent sector -
generated_at DateTime Generation timestamp -
updated_at DateTime Last update timestamp -

Indexes:

  • task
  • generated_at
  • source
  • sync_status
  • source, sync_status
  • entity_type
  • content_format
  • cluster_role
  • cluster
  • external_type
  • site, entity_type

6. igny8_images (Images Model)

Location: backend/igny8_core/business/content/models.py

Field Type Description Choices
id Integer Primary key -
content_id ForeignKey Parent content Content
image_type CharField(50) Image type featured, in_article
position Integer Position in article -
prompt TextField Image generation prompt -
image_url URLField Generated image URL -
status CharField(50) Image status pending, generated, failed
provider CharField(50) Image generation provider dall-e, stable-diffusion, midjourney
model CharField(100) Image generation model -
error_message TextField Error message (if failed) -
metadata JSONField Additional metadata -
account ForeignKey Owner account -
site ForeignKey Parent site -
sector ForeignKey Parent sector -
created_at DateTime Creation timestamp -
updated_at DateTime Last update timestamp -

Indexes:

  • content
  • image_type
  • status
  • site, sector

Frontend Pages & Components

Planner Module Pages

1. Keywords Page (/planner/keywords)

Component: frontend/src/pages/Planner/Keywords.tsx

Key Functions:

  • loadKeywords() - Fetch keywords from API with filters/pagination
  • handleCreateSeedKeyword() - Attach SeedKeyword to site/sector
  • handleEdit(keyword) - Edit keyword (override volume/difficulty, assign cluster)
  • handleBulkAction('auto_cluster', ids) - AI clustering (max 20 keywords)
  • handleBulkDelete(ids) - Delete keywords
  • handleBulkUpdateStatus(ids, status) - Update status
  • handleExport() - Export keywords to CSV
  • handleImportClick() - Import keywords from CSV

API Calls:

fetchKeywords(filters: KeywordFilters): Promise<PaginatedResponse<Keyword>>
createKeyword(data: KeywordCreateData): Promise<Keyword>
updateKeyword(id: number, data: KeywordCreateData): Promise<Keyword>
deleteKeyword(id: number): Promise<void>
bulkDeleteKeywords(ids: number[]): Promise<{ deleted_count: number }>
bulkUpdateKeywordsStatus(ids: number[], status: string): Promise<{ updated_count: number }>
autoClusterKeywords(ids: number[], sectorId?: number): Promise<AITaskResponse>

State Management:

  • keywords - Keyword list
  • clusters - Cluster dropdown options
  • availableSeedKeywords - Available SeedKeywords for attachment
  • searchTerm, statusFilter, clusterFilter, intentFilter, difficultyFilter, volumeMin, volumeMax - Filters
  • currentPage, totalPages, totalCount - Pagination
  • sortBy, sortDirection - Sorting
  • selectedIds - Bulk selection

AI Function Logs: When Resource Debug is enabled, logs all AI function calls:

  • Request: keyword_ids, sector_id
  • Success: task_id (async) or clusters_created, keywords_updated (sync)
  • Progress steps: phase, percentage, message
  • Errors: error message

2. Clusters Page (/planner/clusters)

Component: frontend/src/pages/Planner/Clusters.tsx

Key Functions:

  • loadClusters() - Fetch clusters from API
  • handleCreateCluster() - Create new cluster manually
  • handleEdit(cluster) - Edit cluster name/description
  • handleRowAction('generate_ideas', cluster) - Generate ideas for single cluster
  • handleBulkAction('auto_generate_ideas', ids) - Generate ideas for multiple clusters (max 5)
  • handleBulkDelete(ids) - Delete clusters
  • handleBulkUpdateStatus(ids, status) - Update status

API Calls:

fetchClusters(filters: ClusterFilters): Promise<PaginatedResponse<Cluster>>
createCluster(data: ClusterCreateData): Promise<Cluster>
updateCluster(id: number, data: ClusterCreateData): Promise<Cluster>
deleteCluster(id: number): Promise<void>
bulkDeleteClusters(ids: number[]): Promise<{ deleted_count: number }>
bulkUpdateClustersStatus(ids: number[], status: string): Promise<{ updated_count: number }>
autoGenerateIdeas(clusterIds: number[]): Promise<AITaskResponse>

State Management:

  • clusters - Cluster list
  • searchTerm, statusFilter, difficultyFilter, volumeMin, volumeMax - Filters
  • currentPage, totalPages, totalCount - Pagination
  • sortBy, sortDirection - Sorting
  • selectedIds - Bulk selection

AI Progress Modal: Shows progress for auto_generate_ideas async tasks:

  • Title: "Generating Content Ideas"
  • Function ID: ai-generate-ideas-01-desktop
  • Displays: phase, percentage, message, step logs

3. Ideas Page (/planner/ideas)

Component: frontend/src/pages/Planner/Ideas.tsx

Key Functions:

  • loadIdeas() - Fetch content ideas from API
  • handleCreateIdea() - Create new idea manually
  • handleEdit(idea) - Edit idea title/description/cluster
  • handleRowAction('queue_to_writer', idea) - Convert single idea to task
  • handleBulkAction('queue_to_writer', ids) - Convert multiple ideas to tasks
  • handleBulkDelete(ids) - Delete ideas
  • handleBulkUpdateStatus(ids, status) - Update status

API Calls:

fetchContentIdeas(filters: ContentIdeasFilters): Promise<PaginatedResponse<ContentIdea>>
createContentIdea(data: ContentIdeaCreateData): Promise<ContentIdea>
updateContentIdea(id: number, data: ContentIdeaCreateData): Promise<ContentIdea>
deleteContentIdea(id: number): Promise<void>
bulkDeleteContentIdeas(ids: number[]): Promise<{ deleted_count: number }>
bulkUpdateContentIdeasStatus(ids: number[], status: string): Promise<{ updated_count: number }>
bulkQueueIdeasToWriter(ids: number[]): Promise<{ created_count: number }>

State Management:

  • ideas - Idea list
  • clusters - Cluster dropdown options
  • searchTerm, statusFilter, clusterFilter, structureFilter, typeFilter, entityTypeFilter - Filters
  • currentPage, totalPages, totalCount - Pagination
  • sortBy, sortDirection - Sorting
  • selectedIds - Bulk selection

Queue to Writer Logic: Only ideas with status='new' can be queued. Creates Tasks with:

  • title = idea_title
  • description = idea.description
  • keywords = idea.target_keywords
  • cluster = idea.keyword_cluster
  • idea = idea.id
  • status = 'queued'
  • entity_type = idea.site_entity_type
  • cluster_role = idea.cluster_role
  • taxonomy = idea.taxonomy

Writer Module Pages

4. Tasks Page (/writer/tasks)

Component: frontend/src/pages/Writer/Tasks.tsx

Key Functions:

  • loadTasks() - Fetch tasks from API
  • handleCreateTask() - Create new task manually
  • handleEdit(task) - Edit task title/description/cluster
  • handleRowAction('generate_content', task) - Generate content for single task (AI)
  • handleBulkAction('generate_images', ids) - Generate images for multiple tasks (max 10)
  • handleBulkDelete(ids) - Delete tasks
  • handleBulkUpdateStatus(ids, status) - Update status

API Calls:

fetchTasks(filters: TasksFilters): Promise<PaginatedResponse<Task>>
createTask(data: TaskCreateData): Promise<Task>
updateTask(id: number, data: TaskCreateData): Promise<Task>
deleteTask(id: number): Promise<void>
bulkDeleteTasks(ids: number[]): Promise<{ deleted_count: number }>
bulkUpdateTasksStatus(ids: number[], status: string): Promise<{ updated_count: number }>
autoGenerateContent(taskIds: number[]): Promise<AITaskResponse>
autoGenerateImages(taskIds: number[]): Promise<AITaskResponse>

State Management:

  • tasks - Task list
  • clusters - Cluster dropdown options
  • searchTerm, statusFilter, clusterFilter, structureFilter, typeFilter, sourceFilter, entityTypeFilter - Filters
  • currentPage, totalPages, totalCount - Pagination
  • sortBy, sortDirection - Sorting
  • selectedIds - Bulk selection
  • aiLogs - AI function logs (when Resource Debug enabled)

AI Function Logs: Logs all AI function calls:

  • Request: task_ids, task_title
  • Success: task_id (async) or tasks_updated, images_created (sync)
  • Progress steps: phase, percentage, message
  • Errors: error message

AI Progress Modal: Shows progress for auto_generate_content async tasks:

  • Title: "Generating Content"
  • Function ID: ai-generate-content-03
  • Displays: phase, percentage, message, step logs

5. Content Page (/writer/content)

Component: frontend/src/pages/Writer/Content.tsx

Key Functions:

  • loadContent() - Fetch content from API
  • handleRowAction('generate_image_prompts', content) - Generate AI image prompts
  • handleRowAction('optimize', content) - Run optimizer on content
  • handleRowAction('send_to_optimizer', content) - Navigate to optimizer page

API Calls:

fetchContent(filters: ContentFilters): Promise<PaginatedResponse<Content>>
generateImagePrompts(contentIds: number[]): Promise<AITaskResponse>
optimizerApi.optimize(contentId: number, source: string): Promise<OptimizationResult>

State Management:

  • content - Content list
  • searchTerm, statusFilter, sourceFilter, syncStatusFilter - Filters
  • currentPage, totalPages, totalCount - Pagination
  • sortBy, sortDirection - Sorting
  • selectedIds - Bulk selection

Content Row Actions:

  1. View - Navigate to /writer/content/:id (ContentView page)
  2. Generate Image Prompts - AI function to create smart image prompts
  3. Optimize - Run optimizer to improve SEO scores
  4. Send to Optimizer - Navigate to /optimizer/content?contentId=:id

6. Images Page (/writer/images)

Component: frontend/src/pages/Writer/Images.tsx

Key Functions:

  • loadImages() - Fetch content images grouped by content
  • handleRowAction('update_status', imageGroup) - Update all images for content
  • handleGenerateImages(contentId) - Generate images using AI (opens ImageQueueModal)
  • handleBulkExport(ids) - Export image metadata

API Calls:

fetchContentImages(filters: {}): Promise<ContentImagesResponse>
fetchImageGenerationSettings(): Promise<ImageGenerationSettings>
generateImages(contentId: number, imageType: string, prompt: string, provider: string, model: string): Promise<{ image_url: string }>
bulkUpdateImagesStatus(contentId: number, status: string): Promise<{ updated_count: number }>

State Management:

  • images - ContentImagesGroup list (one row per content)
  • searchTerm, statusFilter - Filters
  • currentPage, totalPages, totalCount - Client-side pagination
  • sortBy, sortDirection - Client-side sorting
  • selectedIds - Bulk selection
  • imageQueue - Image generation queue items
  • taskId, imageModel, imageProvider - Image generation settings

ImageQueueModal: Displays real-time image generation progress:

  • Featured image (always first)
  • In-article images (up to max_in_article_images from settings)
  • Shows: index, label, type, prompt, status, progress, image preview
  • Updates via polling or WebSocket

Content Images Group Structure:

{
  content_id: number;
  content_title: string;
  featured_image: { id, prompt, image_url, status, provider, model };
  in_article_images: [{ id, position, prompt, image_url, status, provider, model }];
  overall_status: 'pending' | 'partial' | 'complete' | 'failed';
  total_images: number;
  generated_images: number;
}

Backend API Endpoints

Planner Module Endpoints

Keywords API

Base URL: /v1/planner/keywords/

Endpoint Method Function Description
/ GET list() List keywords with filters/pagination
/ POST create() Create new keyword (attach SeedKeyword)
/:id/ GET retrieve() Get single keyword details
/:id/ PUT/PATCH update() Update keyword
/:id/ DELETE destroy() Delete keyword
/bulk_delete/ POST bulk_delete() Delete multiple keywords
/bulk_update_status/ POST bulk_update_status() Update status for multiple keywords
/auto_cluster/ POST auto_cluster() AI clustering function
/export/ GET export() Export keywords to CSV
/import/ POST import() Import keywords from CSV

ViewSet: KeywordsViewSet in backend/igny8_core/modules/planner/views.py

Clusters API

Base URL: /v1/planner/clusters/

Endpoint Method Function Description
/ GET list() List clusters with filters/pagination
/ POST create() Create new cluster
/:id/ GET retrieve() Get single cluster details
/:id/ PUT/PATCH update() Update cluster
/:id/ DELETE destroy() Delete cluster
/bulk_delete/ POST bulk_delete() Delete multiple clusters
/bulk_update_status/ POST bulk_update_status() Update status for multiple clusters
/auto_generate_ideas/ POST auto_generate_ideas() AI idea generation function

ViewSet: ClusterViewSet in backend/igny8_core/modules/planner/views.py

Content Ideas API

Base URL: /v1/planner/content-ideas/

Endpoint Method Function Description
/ GET list() List ideas with filters/pagination
/ POST create() Create new idea
/:id/ GET retrieve() Get single idea details
/:id/ PUT/PATCH update() Update idea
/:id/ DELETE destroy() Delete idea
/bulk_delete/ POST bulk_delete() Delete multiple ideas
/bulk_update_status/ POST bulk_update_status() Update status for multiple ideas
/bulk_queue_to_writer/ POST bulk_queue_to_writer() Convert ideas to tasks

ViewSet: ContentIdeasViewSet in backend/igny8_core/modules/planner/views.py

Writer Module Endpoints

Tasks API

Base URL: /v1/writer/tasks/

Endpoint Method Function Description
/ GET list() List tasks with filters/pagination
/ POST create() Create new task
/:id/ GET retrieve() Get single task details
/:id/ PUT/PATCH update() Update task
/:id/ DELETE destroy() Delete task
/bulk_delete/ POST bulk_delete() Delete multiple tasks
/bulk_update_status/ POST bulk_update_status() Update status for multiple tasks
/auto_generate_content/ POST auto_generate_content() AI content generation function

ViewSet: TasksViewSet in backend/igny8_core/modules/writer/views.py

Content API

Base URL: /v1/writer/content/

Endpoint Method Function Description
/ GET list() List content with filters/pagination
/:id/ GET retrieve() Get single content details
/:id/ PUT/PATCH update() Update content
/:id/ DELETE destroy() Delete content
/generate_image_prompts/ POST generate_image_prompts() AI image prompt generation

ViewSet: ContentViewSet in backend/igny8_core/modules/writer/views.py

Images API

Base URL: /v1/writer/images/

Endpoint Method Function Description
/ GET list() List images with filters/pagination
/content/ GET by_content() List images grouped by content
/settings/ GET get_settings() Get image generation settings
/auto_generate/ POST auto_generate_images() AI bulk image generation (deprecated)
/generate_images/ POST generate_images() Generate images for content
/bulk_update_status/ POST bulk_update_status() Update status for content images

ViewSet: ImagesViewSet in backend/igny8_core/modules/writer/views.py

Complete Workflow

Stage 1: Keyword Research & Clustering

Step 1.1: Attach SeedKeywords to Site

Frontend: Keywords Page → "Add Keyword" button Function: handleCreateSeedKeyword() API Call: POST /v1/planner/keywords/

Request Payload:

{
  "seed_keyword_id": 123,
  "site_id": 1,
  "sector_id": 2,
  "volume_override": null,
  "difficulty_override": null,
  "cluster_id": null,
  "status": "pending"
}

Backend Process:

  1. Validate seed_keyword_id exists in seed_keywords table
  2. Validate site_id and sector_id exist and match
  3. Check unique constraint: seed_keyword + site + sector
  4. Create Keywords record with account/site/sector
  5. Return created keyword with computed properties

Response:

{
  "success": true,
  "data": {
    "id": 456,
    "seed_keyword_id": 123,
    "keyword": "best coffee beans",
    "volume": 12000,
    "difficulty": 45,
    "intent": "commercial",
    "cluster_id": null,
    "status": "pending",
    "created_at": "2025-11-24T10:30:00Z"
  },
  "message": "Keyword attached successfully"
}

Step 1.2: AI Auto-Clustering

Frontend: Keywords Page → Select keywords → Bulk Actions → "Auto-Cluster" Function: handleBulkAction('auto_cluster', ids) API Call: POST /v1/planner/keywords/auto_cluster/

Request Payload:

{
  "ids": [456, 457, 458, 459, 460],
  "sector_id": 2
}

Backend Process (auto_cluster() view):

# backend/igny8_core/modules/planner/views.py line 573
def auto_cluster(self, request):
    keyword_ids = request.data.get('ids', [])
    sector_id = request.data.get('sector_id')
    account = getattr(request, 'account', None)
    
    # Use ClusteringService
    service = ClusteringService()
    result = service.cluster_keywords(keyword_ids, account, sector_id)
    
    if result.get('success'):
        if 'task_id' in result:
            # Async task queued
            return success_response(data={'task_id': result['task_id']}, message='Clustering started')
        else:
            # Synchronous execution
            return success_response(data=result)

ClusteringService Logic: Location: backend/igny8_core/business/planning/services/clustering_service.py

  1. Load Keywords:

    • Fetch Keywords by IDs
    • Get keyword text, volume, difficulty, intent from seed_keyword

  2. Check Credits:

    • Calculate credits needed (based on keyword count)
    • Deduct credits from account

  3. AI Clustering:

    • Send keywords to AI engine
    • Prompt: "Group these keywords into semantic clusters..."
    • AI returns: { clusters: [{ name, keywords: [ids] }] }

  4. Create/Update Clusters:

    • For each AI cluster:
      • Check if cluster with same name exists
      • If exists, use existing; else create new Clusters record
      • Update Keywords.cluster_id for all keywords in group
      • Calculate keywords_count, volume, difficulty for cluster

  5. Return Result:

    • Sync: { success: true, clusters_created: 3, keywords_updated: 20 }
    • Async: { success: true, task_id: 'uuid', message: 'Clustering started' }

Response (Async):

{
  "success": true,
  "data": {
    "task_id": "abc123-def456-ghi789"
  },
  "message": "Clustering started"
}

Frontend Progress Tracking:

// Opens ProgressModal
progressModal.openModal(result.task_id, 'Auto-Clustering Keywords', 'ai-auto-cluster-01');

// ProgressModal polls /v1/system/task-progress/:task_id/ every 2 seconds
// Shows: phase, percentage, message, step logs

AI Task Progress Phases:

  1. Initializing (0-10%) - Validating keywords, loading data
  2. Processing (10-40%) - Sending to AI engine
  3. AI Analysis (40-80%) - AI clustering keywords
  4. Saving Results (80-95%) - Creating clusters, updating keywords
  5. Completed (100%) - Done

Stage 2: Content Idea Generation

Step 2.1: AI Generate Ideas from Clusters

Frontend: Clusters Page → Select clusters → Bulk Actions → "Generate Ideas" Function: handleBulkAction('auto_generate_ideas', ids) API Call: POST /v1/planner/clusters/auto_generate_ideas/

Request Payload:

{
  "ids": [10, 11, 12]
}

Backend Process (auto_generate_ideas() view):

# backend/igny8_core/modules/planner/views.py line 810
def auto_generate_ideas(self, request):
    cluster_ids = request.data.get('ids', [])
    account = getattr(request, 'account', None)
    
    # Use IdeasService
    service = IdeasService()
    result = service.generate_ideas(cluster_ids, account)
    
    if result.get('success'):
        if 'task_id' in result:
            # Async task queued
            return success_response(data={'task_id': result['task_id']}, message='Idea generation started')
        else:
            # Synchronous execution
            return success_response(data=result)

IdeasService Logic: Location: backend/igny8_core/business/planning/services/ideas_service.py

  1. Load Clusters:

    • Fetch Clusters by IDs with keywords
    • Get cluster name, description, keywords, volume

  2. Check Credits:

    • Calculate credits needed (based on cluster count)
    • Deduct credits from account

  3. AI Idea Generation:

    • For each cluster:
      • Send cluster data to AI engine
      • Prompt: "Generate 5-10 content ideas for cluster '{name}' with keywords [{keywords}]..."
      • AI returns: [{ title, description, target_keywords, entity_type, cluster_role }]

  4. Create Ideas:

    • For each AI idea:
      • Create ContentIdeas record
      • Set keyword_cluster_id = cluster ID
      • Set status = 'new'
      • Set site_entity_type, cluster_role from AI
      • Link keywords via keyword_objects M2M

  5. Return Result:

    • Sync: { success: true, ideas_created: 25 }
    • Async: { success: true, task_id: 'uuid', message: 'Idea generation started' }

Response (Async):

{
  "success": true,
  "data": {
    "task_id": "xyz789-uvw456-rst123"
  },
  "message": "Idea generation started"
}

AI Task Progress Phases:

  1. Initializing (0-10%) - Loading clusters, keywords
  2. Analyzing Clusters (10-30%) - Analyzing keyword patterns
  3. Generating Ideas (30-90%) - AI creating ideas (per cluster)
  4. Saving Ideas (90-100%) - Creating ContentIdeas records

Step 2.2: Queue Ideas to Writer

Frontend: Ideas Page → Select ideas (status='new') → Bulk Actions → "Queue to Writer" Function: handleBulkAction('queue_to_writer', ids) API Call: POST /v1/planner/content-ideas/bulk_queue_to_writer/

Request Payload:

{
  "ids": [50, 51, 52]
}

Backend Process (bulk_queue_to_writer() view):

# backend/igny8_core/modules/planner/views.py line 997
def bulk_queue_to_writer(self, request):
    ids = request.data.get('ids', [])
    queryset = self.get_queryset()
    ideas = queryset.filter(id__in=ids, status='new')  # Only 'new' ideas
    
    created_tasks = []
    for idea in ideas:
        task = Tasks.objects.create(
            title=idea.idea_title,
            description=idea.description or '',
            keywords=idea.target_keywords or '',
            cluster=idea.keyword_cluster,
            idea=idea,
            status='queued',
            account=idea.account,
            site=idea.site,
            sector=idea.sector,
            entity_type=(idea.site_entity_type or 'post'),
            taxonomy=idea.taxonomy,
            cluster_role=(idea.cluster_role or 'hub'),
        )
        created_tasks.append(task.id)
        
        # Update idea status to 'scheduled'
        idea.status = 'scheduled'
        idea.save()
    
    return success_response(data={'created_count': len(created_tasks)})

Response:

{
  "success": true,
  "data": {
    "created_count": 3
  },
  "message": "Queue complete: 3 tasks created from 3 ideas"
}

Stage 3: Content Generation

Step 3.1: AI Generate Content for Task

Frontend: Tasks Page → Row Action → "Generate Content" Function: handleRowAction('generate_content', task) API Call: POST /v1/writer/tasks/auto_generate_content/

Request Payload:

{
  "ids": [100]
}

Backend Process (auto_generate_content() view):

# backend/igny8_core/modules/writer/views.py line 150
def auto_generate_content(self, request):
    ids = request.data.get('ids', [])
    account = getattr(request, 'account', None)
    
    # Validate tasks exist
    existing_tasks = queryset.filter(id__in=ids, account=account)
    
    # Use ContentGenerationService
    service = ContentGenerationService()
    result = service.generate_content(ids, account)
    
    if result.get('success'):
        if 'task_id' in result:
            # Async task queued
            return success_response(data={'task_id': result['task_id']}, message='Content generation started')
        else:
            # Synchronous execution
            return success_response(data=result, message='Content generated successfully')

ContentGenerationService Logic: Location: backend/igny8_core/business/content/services/content_generation_service.py

  1. Load Tasks:

    • Fetch Tasks by IDs with cluster, keywords
    • Validate task status (can generate for any status)

  2. Check Credits:

    • Calculate credits needed (based on word count target)
    • Deduct credits from account

  3. AI Content Generation:

    • For each task:
      • Build AI prompt with:
        • Title: task.title
        • Keywords: task.keywords or cluster keywords
        • Entity type: task.entity_type
        • Cluster role: task.cluster_role
        • Word count: estimate from idea or default
      • Send to AI engine
      • AI returns: { html_content, word_count, meta_title, meta_description, primary_keyword, secondary_keywords }

  4. Create/Update Content:

    • Check if task.content_record exists (OneToOne)
    • If exists: update content
    • If not: create new Content record
    • Set fields:
      • html_content = AI output
      • word_count = AI calculated
      • meta_title, meta_description = AI SEO
      • primary_keyword, secondary_keywords = AI keywords
      • status = 'draft'
      • source = 'igny8'
      • sync_status = 'native'
      • entity_type = task.entity_type
      • cluster_role = task.cluster_role
      • cluster = task.cluster
      • structure_data = AI metadata

  5. Update Task Status:

    • Set task.status = 'completed'

  6. Return Result:

    • Sync: { success: true, tasks_updated: 1 }
    • Async: { success: true, task_id: 'uuid', message: 'Content generation started' }

Response (Async):

{
  "success": true,
  "data": {
    "task_id": "content-abc123-def456"
  },
  "message": "Content generation started"
}

AI Task Progress Phases:

  1. Initializing (0-10%) - Loading tasks, keywords
  2. Research (10-30%) - Analyzing keywords, cluster
  3. Outlining (30-50%) - Creating content structure
  4. Writing (50-90%) - Generating content sections
  5. SEO Optimization (90-95%) - Adding meta tags, keywords
  6. Saving (95-100%) - Creating Content record

Stage 4: Image Generation

Step 4.1: Generate Image Prompts (Smart Prompts)

Frontend: Content Page → Row Action → "Generate Image Prompts" Function: handleRowAction('generate_image_prompts', content) API Call: POST /v1/writer/content/generate_image_prompts/

Request Payload:

{
  "ids": [200]
}

Backend Process (generate_image_prompts() view):

# backend/igny8_core/modules/writer/views.py line 793
@action(detail=False, methods=['post'], url_path='generate_image_prompts', url_name='generate_image_prompts')
def generate_image_prompts(self, request):
    ids = request.data.get('ids', [])
    account = getattr(request, 'account', None)
    
    # Use ImagePromptService
    service = ImagePromptService()
    result = service.generate_prompts(ids, account)
    
    if result.get('success'):
        if 'task_id' in result:
            return success_response(data={'task_id': result['task_id']}, message='Prompt generation started')
        else:
            return success_response(data=result)

ImagePromptService Logic:

  1. Load Content:

    • Fetch Content by IDs with html_content
    • Parse HTML to identify image placement needs

  2. Analyze Content:

    • Identify sections needing images
    • Extract context for each image (surrounding text)
    • Determine image type (featured, in-article)

  3. AI Prompt Generation:

    • For each image placement:
      • Send section context to AI
      • Prompt: "Create a detailed image prompt for section about '{context}'..."
      • AI returns: detailed DALL-E/Stable Diffusion prompt

  4. Create Image Records:

    • For featured image: create Images record with image_type='featured'
    • For in-article images: create Images records with image_type='in_article', position=N
    • Set prompt = AI output
    • Set status = 'pending'
    • Set provider, model from settings

  5. Return Result:

    • { success: true, prompts_created: 5 }

Step 4.2: Generate Images from Prompts

Frontend: Images Page → Row Action → "Generate Images" Function: handleGenerateImages(contentId) Opens: ImageQueueModal

Modal Process:

  1. Fetch Settings:

    • GET /v1/writer/images/settings/
    • Returns: { max_in_article_images, default_provider, default_model }

  2. Build Queue:

    • Load content images: fetchContentImages({ content_id: contentId })
    • Filter images with status='pending' and prompt
    • Order: featured first, then in-article by position
    • Limit: max_in_article_images from settings

  3. Generate Images Sequentially:

    • For each queue item:
      • Update UI: status='generating', progress=10%
      • Call: POST /v1/writer/images/generate_images/ 
        {
  "content_id": 200,
  "image_type": "featured",
  "prompt": "Professional coffee beans in burlap sack...",
  "provider": "dall-e",
  "model": "dall-e-3"
}
      • Backend calls image provider API (DALL-E, Stable Diffusion)
      • Updates Images record:
        • image_url = provider response
        • status = 'generated' or 'failed'
        • error_message = if failed
      • Update UI: status='generated', progress=100%, imageUrl=...

  4. Completion:

    • All images generated
    • Update overall status badge
    • Reload images list

Backend Image Generation:

# backend/igny8_core/modules/writer/views.py line 634
@action(detail=False, methods=['post'], url_path='generate_images', url_name='generate_images')
def generate_images(self, request):
    content_id = request.data.get('content_id')
    image_type = request.data.get('image_type')
    prompt = request.data.get('prompt')
    provider = request.data.get('provider', 'dall-e')
    model = request.data.get('model', 'dall-e-3')
    
    # Deduct credits
    # Call image provider API
    # Update Images record with image_url or error
    
    return success_response(data={'image_url': 'https://...'})

AI Functions

1. Auto-Cluster Keywords

Function ID: ai-auto-cluster-01 API Endpoint: POST /v1/planner/keywords/auto_cluster/ Service: ClusteringService Location: backend/igny8_core/business/planning/services/clustering_service.py

Input:

  • ids: Keyword IDs (max 20)
  • sector_id: Sector ID

Process:

  1. Load keywords with seed_keyword data
  2. Check credits (1 credit per 5 keywords)
  3. Send to AI: keyword texts, volumes, difficulties, intents
  4. AI groups keywords into semantic clusters
  5. Create/update Clusters records
  6. Update Keywords.cluster_id

Output:

  • Async: { task_id }
  • Sync: { clusters_created, keywords_updated }

Credits: 1 credit per 5 keywords (rounded up)

2. Auto-Generate Ideas

Function ID: ai-generate-ideas-01-desktop API Endpoint: POST /v1/planner/clusters/auto_generate_ideas/ Service: IdeasService Location: backend/igny8_core/business/planning/services/ideas_service.py

Input:

  • ids: Cluster IDs (max 5)

Process:

  1. Load clusters with keywords
  2. Check credits (2 credits per cluster)
  3. For each cluster:
    • Send cluster name, keywords, volumes to AI
    • AI generates 5-10 content ideas
  4. Create ContentIdeas records

Output:

  • Async: { task_id }
  • Sync: { ideas_created }

Credits: 2 credits per cluster

3. Auto-Generate Content

Function ID: ai-generate-content-03 API Endpoint: POST /v1/writer/tasks/auto_generate_content/ Service: ContentGenerationService Location: backend/igny8_core/business/content/services/content_generation_service.py

Input:

  • ids: Task IDs (max 10)

Process:

  1. Load tasks with clusters, keywords
  2. Check credits (based on word count target)
  3. For each task:
    • Build AI prompt with title, keywords, entity_type
    • AI generates HTML content, meta tags, SEO
  4. Create/update Content records
  5. Update task status to 'completed'

Output:

  • Async: { task_id }
  • Sync: { tasks_updated }

Credits: 1 credit per 500 words (estimated)

4. Generate Image Prompts

Function ID: ai-generate-image-prompts-01-desktop API Endpoint: POST /v1/writer/content/generate_image_prompts/ Service: ImagePromptService Location: backend/igny8_core/business/content/services/image_prompt_service.py

Input:

  • ids: Content IDs

Process:

  1. Load content with HTML
  2. Analyze content sections
  3. For each image placement:
    • Send section context to AI
    • AI creates detailed image prompt
  4. Create Images records with prompts

Output:

  • Async: { task_id }
  • Sync: { prompts_created }

Credits: 0.5 credits per prompt

5. Generate Images

Function ID: N/A (synchronous) API Endpoint: POST /v1/writer/images/generate_images/ Service: Direct image provider call

Input:

  • content_id: Content ID
  • image_type: 'featured' or 'in_article'
  • prompt: Image generation prompt
  • provider: 'dall-e', 'stable-diffusion', 'midjourney'
  • model: Provider-specific model

Process:

  1. Deduct credits (based on provider/model)
  2. Call image provider API (DALL-E, Stable Diffusion)
  3. Update Images record with image_url or error

Output:

  • { image_url: 'https://...' }

Credits:

  • DALL-E 3: 4 credits per image
  • DALL-E 2: 2 credits per image
  • Stable Diffusion: 1 credit per image

Flowchart Diagrams

Complete Workflow Flowchart

graph TD
    Start([User starts workflow]) --> SelectSite[Select Site & Sector]
    SelectSite --> AttachKeywords[Attach SeedKeywords<br/>POST /keywords/]
    
    AttachKeywords --> ManualCluster{Manual or<br/>AI Clustering?}
    ManualCluster -->|Manual| EditKeywords[Edit Keywords<br/>Assign to Cluster]
    ManualCluster -->|AI| AutoCluster[Auto-Cluster<br/>POST /keywords/auto_cluster/]
    
    AutoCluster --> ClusteringProgress{Async<br/>Task?}
    ClusteringProgress -->|Yes| PollClustering[Poll Progress<br/>GET /task-progress/:id/]
    ClusteringProgress -->|No| ClustersCreated[Clusters Created]
    PollClustering --> ClustersCreated
    EditKeywords --> ClustersCreated
    
    ClustersCreated --> ManualIdeas{Manual or<br/>AI Ideas?}
    ManualIdeas -->|Manual| CreateIdeas[Create Ideas Manually<br/>POST /content-ideas/]
    ManualIdeas -->|AI| AutoIdeas[Auto-Generate Ideas<br/>POST /clusters/auto_generate_ideas/]
    
    AutoIdeas --> IdeasProgress{Async<br/>Task?}
    IdeasProgress -->|Yes| PollIdeas[Poll Progress<br/>GET /task-progress/:id/]
    IdeasProgress -->|No| IdeasCreated[Ideas Created]
    PollIdeas --> IdeasCreated
    CreateIdeas --> IdeasCreated
    
    IdeasCreated --> QueueToWriter[Queue to Writer<br/>POST /content-ideas/bulk_queue_to_writer/]
    QueueToWriter --> TasksCreated[Tasks Created<br/>status='queued']
    
    TasksCreated --> ManualContent{Manual or<br/>AI Content?}
    ManualContent -->|Manual| WriteContent[Write Content Manually]
    ManualContent -->|AI| GenerateContent[Generate Content<br/>POST /tasks/auto_generate_content/]
    
    GenerateContent --> ContentProgress{Async<br/>Task?}
    ContentProgress -->|Yes| PollContent[Poll Progress<br/>GET /task-progress/:id/]
    ContentProgress -->|No| ContentCreated[Content Created]
    PollContent --> ContentCreated
    WriteContent --> ContentCreated
    
    ContentCreated --> ImagePrompts{Generate<br/>Image Prompts?}
    ImagePrompts -->|Yes| AIPrompts[Generate Image Prompts<br/>POST /content/generate_image_prompts/]
    ImagePrompts -->|No| ManualPrompts[Write Prompts Manually]
    
    AIPrompts --> PromptsCreated[Image Records Created<br/>status='pending']
    ManualPrompts --> PromptsCreated
    
    PromptsCreated --> GenerateImages{Generate<br/>Images?}
    GenerateImages -->|Yes| ImageQueue[Open ImageQueueModal<br/>Sequential Generation]
    GenerateImages -->|No| SkipImages[Skip Images]
    
    ImageQueue --> ImageLoop[For Each Image:<br/>POST /images/generate_images/]
    ImageLoop --> ImageCreated[Image Generated<br/>status='generated']
    ImageCreated --> MoreImages{More<br/>Images?}
    MoreImages -->|Yes| ImageLoop
    MoreImages -->|No| AllImagesComplete[All Images Complete]
    
    AllImagesComplete --> ReviewContent[Review Content<br/>status='review']
    SkipImages --> ReviewContent
    
    ReviewContent --> PublishContent[Publish Content<br/>status='publish']
    PublishContent --> End([Workflow Complete])

    %% Styling
    classDef aiFunction fill:#9f7aea,stroke:#805ad5,stroke-width:2px,color:#fff
    classDef userAction fill:#48bb78,stroke:#38a169,stroke-width:2px,color:#fff
    classDef decision fill:#ed8936,stroke:#dd6b20,stroke-width:2px,color:#fff
    classDef progress fill:#4299e1,stroke:#3182ce,stroke-width:2px,color:#fff
    
    class AutoCluster,AutoIdeas,GenerateContent,AIPrompts,ImageQueue aiFunction
    class AttachKeywords,EditKeywords,CreateIdeas,WriteContent,ManualPrompts,ReviewContent,PublishContent userAction
    class ManualCluster,ManualIdeas,ManualContent,ImagePrompts,GenerateImages,ClusteringProgress,IdeasProgress,ContentProgress,MoreImages decision
    class PollClustering,PollIdeas,PollContent progress

AI Function Progress Tracking

sequenceDiagram
    participant U as User (Frontend)
    participant A as API Endpoint
    participant S as Service Layer
    participant AI as AI Engine
    participant C as Celery Worker
    participant DB as Database
    
    U->>A: POST /keywords/auto_cluster/ {ids: [1,2,3]}
    A->>S: ClusteringService.cluster_keywords()
    S->>DB: Check credits
    DB-->>S: Credits sufficient
    S->>C: Queue async task
    C-->>S: task_id
    S-->>A: {success: true, task_id: 'abc123'}
    A-->>U: {task_id: 'abc123', message: 'Clustering started'}
    
    U->>U: Open ProgressModal
    
    loop Poll every 2 seconds
        U->>A: GET /task-progress/abc123/
        A->>DB: Get task progress
        DB-->>A: {phase, percentage, message}
        A-->>U: {status, percentage, message, details}
        U->>U: Update progress bar
    end
    
    Note over C,AI: Celery worker processes task
    C->>AI: Send keywords to AI
    AI-->>C: {clusters: [{name, keywords}]}
    C->>DB: Create Clusters, update Keywords
    DB-->>C: Success
    C->>DB: Update task progress: 100%, completed
    
    U->>A: GET /task-progress/abc123/
    A->>DB: Get task progress
    DB-->>A: {status: 'completed', percentage: 100}
    A-->>U: {status: 'completed', percentage: 100}
    U->>U: Close modal, show success toast
    U->>A: GET /keywords/ (reload data)
    A->>DB: Fetch updated keywords
    DB-->>A: Keywords with cluster_id set
    A-->>U: Updated keywords list

Image Generation Queue Flow

sequenceDiagram
    participant U as User (Frontend)
    participant M as ImageQueueModal
    participant A as API Endpoint
    participant P as Image Provider
    participant DB as Database
    
    U->>A: GET /images/content/?content_id=200
    A->>DB: Fetch content images
    DB-->>A: {featured_image, in_article_images[]}
    A-->>U: Content images data
    
    U->>U: Click "Generate Images"
    U->>A: GET /images/settings/
    A-->>U: {max_in_article_images: 3, default_provider: 'dall-e'}
    
    U->>M: Open ImageQueueModal
    M->>M: Build queue: [featured, in-article-1, in-article-2, in-article-3]
    M->>U: Display queue items (status: pending)
    
    loop For each queue item
        M->>M: Update UI: status='generating', progress=10%
        M->>A: POST /images/generate_images/ {content_id, image_type, prompt, provider, model}
        A->>DB: Deduct credits
        A->>P: Generate image (DALL-E API)
        P-->>A: {image_url: 'https://...'}
        A->>DB: Update Images record (image_url, status='generated')
        DB-->>A: Success
        A-->>M: {success: true, image_url: 'https://...'}
        M->>M: Update UI: status='generated', progress=100%, show image
    end
    
    M->>M: All images complete
    M->>U: Show success message, close modal option
    U->>U: Close modal
    U->>A: GET /images/content/ (reload)
    A->>DB: Fetch updated images
    DB-->>A: All images with status='generated'
    A-->>U: Updated images list

Summary

This workflow documentation covers:

✅ Database Tables: All 6 core models with fields, types, indexes, and relationships
✅ Frontend Pages: 6 pages with all functions, API calls, and state management
✅ Backend APIs: All endpoints with methods, functions, and ViewSets
✅ Complete Workflow: Step-by-step from keywords to published content
✅ AI Functions: 5 AI functions with inputs, processes, outputs, and credits
✅ Flowcharts: 3 detailed diagrams (complete workflow, progress tracking, image generation)

Key Workflow Paths:

  1. Manual Path: User creates everything manually (keywords → clusters → ideas → tasks → content)
  2. AI-Assisted Path: User uses AI functions at each stage (auto-cluster, auto-generate ideas, auto-generate content)
  3. Hybrid Path: Mix of manual and AI (manual keywords, AI clustering, manual ideas, AI content)

Credits System:

  • Auto-Cluster: 1 credit per 5 keywords
  • Generate Ideas: 2 credits per cluster
  • Generate Content: 1 credit per 500 words
  • Image Prompts: 0.5 credits per prompt
  • Generate Images: 1-4 credits per image (provider-dependent)

Progress Tracking: All async AI functions return task_id, which is polled via /task-progress/:id/ to show real-time progress with phases, percentages, and step logs.

End of Documentation

Reference in New Issue View Git Blame Copy Permalink