igny8/STATUS-UNIFIED-STRUCTURE-PLAN.md

Unified Status Structure Plan - Keywords, Clusters, Ideas

Date: December 3, 2025
Objective: Implement unified, workflow-driven status values across all three modules

---

🎯 Proposed Unified Status Structure

Module-Specific Status Values

KEYWORDS Module

Current: pending, active, archived
Proposed: new, mapped, disabled

Status	Meaning	When Used	Next Step
new	Keyword just attached to site, not yet assigned to cluster	User adds keyword	AI auto-cluster runs → mapped
mapped	Keyword is assigned to a cluster	AI clustering completes OR manual assignment	Can generate ideas from cluster
disabled	Keyword is archived/inactive	User manually disables	No workflow activity

Workflow:

new → mapped → (can be used for ideas)
  ↓
disabled (any time)

---

CLUSTERS Module

Current: active, pending, archived
Proposed: new, mapped, disabled

Status	Meaning	When Used	Next Step
new	Cluster created but no content ideas generated yet	AI clustering creates cluster OR manual creation	AI generate_ideas runs → mapped
mapped	Cluster has ideas generated from it	AI generate_ideas completes	Ideas ready to queue to writer
disabled	Cluster is archived/inactive	User manually disables	No workflow activity

Workflow:

new → mapped → (ideas queue to writer)
  ↓
disabled (any time)

---

IDEAS Module

Current: new, scheduled, completed, published
Proposed: new, queued, completed, published

Status	Meaning	When Used	Next Step
new	Idea generated by AI but not queued to writer yet	AI generate_ideas creates idea	User queues to writer → queued
queued	Idea queued to writer, task created in Writer module	Bulk queue to writer API called	Content generated by writer → completed
completed	Content generated from idea (Task completed with content)	generate_content AI function creates Content record	Manual publish → published
published	Content published to site	User publishes OR auto-publish	Final state

Workflow:

new → queued → completed → published

---

📊 Status Update Flow Diagram

KEYWORDS WORKFLOW:
1. User adds SeedKeyword → Keywords status='new'
2. AI auto_cluster runs → Keywords status='mapped' (assigned to cluster)
3. User can disable → Keywords status='disabled'

CLUSTERS WORKFLOW:
1. AI creates cluster → Clusters status='new'
2. AI generate_ideas runs → Clusters status='mapped' (has ideas)
3. User can disable → Clusters status='disabled'

IDEAS WORKFLOW:
1. AI creates idea → Ideas status='new'
2. User bulk queues ideas → Ideas status='queued' (task created in Writer)
3. Writer AI generates content → Ideas status='completed' (Content record created)
4. User publishes content → Ideas status='published'

---

🔧 Change Locations & Impact

Backend Model Files to Update

1. /backend/igny8_core/business/planning/models.py

Keywords Model (Lines 42-46):

# CURRENT:
STATUS_CHOICES = [
    ('active', 'Active'),
    ('pending', 'Pending'),
    ('archived', 'Archived'),
]

# PROPOSED:
STATUS_CHOICES = [
    ('new', 'New'),
    ('mapped', 'Mapped'),
    ('disabled', 'Disabled'),
]

# DEFAULT: Change from 'pending' → 'new'

Clusters Model (Line 12):

# CURRENT:
status = models.CharField(max_length=50, default='active')

# PROPOSED:
status = models.CharField(max_length=50, default='new', 
    choices=[('new', 'New'), ('mapped', 'Mapped'), ('disabled', 'Disabled')])

ContentIdeas Model (Lines 150-157):

# CURRENT:
STATUS_CHOICES = [
    ('new', 'New'),
    ('scheduled', 'Scheduled'),
    ('published', 'Published'),
]

# PROPOSED:
STATUS_CHOICES = [
    ('new', 'New'),
    ('queued', 'Queued'),
    ('completed', 'Completed'),
    ('published', 'Published'),
]

# DEFAULT: Keep 'new'

---

Backend AI Function Files to Update

2. /backend/igny8_core/ai/functions/auto_cluster.py (Line 297)

When keywords are assigned to clusters:

# CURRENT:
status='active'

# PROPOSED:
status='mapped'

Location: In save_output() method when updating Keywords

---

3. /backend/igny8_core/ai/functions/generate_ideas.py (Line 224)

When clusters change status after ideas generated:

# CURRENT: No cluster status update

# PROPOSED: After creating ideas, update cluster status:
# In save_output() method, after creating ideas:
cluster.status = 'mapped'
cluster.save()

Additional: Ideas default stays status='new'

---

4. /backend/igny8_core/ai/functions/generate_content.py (Line 318)

When content is generated from a task:

# CURRENT:
task.status = 'completed'
task.save()

# PROPOSED: Also update the related Idea status:
task.status = 'completed'
task.save()

# NEW: Update idea to completed
if hasattr(task, 'idea') and task.idea:
    idea = task.idea
    idea.status = 'completed'
    idea.save()

Location: In save_output() method after content is created

---

Backend API Endpoint Files to Update

5. /backend/igny8_core/modules/planner/views.py

Line 1029 - Queue ideas to writer filter:

# CURRENT:
queueable_ideas = all_ideas.filter(status='new')

# NO CHANGE NEEDED (stays 'new')

Line 1084 - Update idea status when queued:

# CURRENT:
idea.status = 'scheduled'
idea.save()

# PROPOSED:
idea.status = 'queued'
idea.save()

Line 563 - Bulk create keywords from CSV:

# CURRENT:
status=row.get('status', 'pending') or 'pending'

# PROPOSED:
status=row.get('status', 'new') or 'new'

---

Frontend Configuration Files to Update

6. /frontend/src/config/pages/keywords.config.tsx

Table Column Status Badge (Lines 230-248):

// CURRENT:
pending: 'amber', active: 'green', archived: 'red'

// PROPOSED:
new: 'amber', mapped: 'green', disabled: 'red'

Filter Dropdown (Lines 310-318):

// CURRENT:
options: ['pending', 'active', 'archived']

// PROPOSED:
options: ['new', 'mapped', 'disabled']

Form Field Default (Lines 560-570):

// CURRENT:
default: 'pending'

// PROPOSED:
default: 'new'

---

7. /frontend/src/config/pages/clusters.config.tsx

Table Column Status Badge (Lines 190-200):

// CURRENT:
active: 'green', archived: 'red'  (inconsistent from backend)

// PROPOSED:
new: 'amber', mapped: 'green', disabled: 'red'

Filter Dropdown (Lines 240-253):

// CURRENT:
options: ['active', 'archived']  (missing 'new')

// PROPOSED:
options: ['new', 'mapped', 'disabled']

Form Field Default (Lines 405-418):

// CURRENT:
default: 'new'

// PROPOSED:
default: 'new'  (no change, already correct)

---

8. /frontend/src/config/pages/ideas.config.tsx

Table Column Status Badge (Lines 170-185):

// CURRENT:
new: 'amber', scheduled: 'blue', completed: 'blue', published: 'green'

// PROPOSED:
new: 'amber', queued: 'blue', completed: 'blue', published: 'green'

Filter Dropdown (Lines 218-228):

// CURRENT:
options: ['new', 'scheduled', 'completed', 'published']
// NOTE: missing 'completed' in some places

// PROPOSED:
options: ['new', 'queued', 'completed', 'published']

Form Field Default (Lines 372-385):

// CURRENT:
default: 'new'

// PROPOSED:
default: 'new'  (no change)

---

Frontend Dashboard & State Files

9. /frontend/src/pages/Planner/Dashboard.tsx

Status Metrics (Lines 121-155):

// Updates status counting logic
// From: keywordsByStatus, clustersByStatus, ideaByStatus
// To: Use new status values

// Example refactor:
// OLD: keywordsByStatus['active'] vs keywordsByStatus['pending']
// NEW: keywordsByStatus['mapped'] vs keywordsByStatus['new']

No API call changes needed - API returns correct values

---

10. /frontend/src/services/api.ts

No changes needed - Only returns what backend provides

---

Frontend Component Files

11. /frontend/src/pages/Planner/Keywords.tsx

Status Filter Logic (if hardcoded):

• Search for status comparisons
• Update from old values → new values

12. /frontend/src/pages/Planner/Clusters.tsx

Status Filter Logic (if hardcoded):

• Search for status comparisons
• Update from old values → new values

13. /frontend/src/pages/Planner/Ideas.tsx

Status Filter Logic (if hardcoded):

• Update from scheduled → queued
• Keep new, completed, published

---

Database Migration Files

14. New Django Migration Required

File to create: /backend/igny8_core/business/planning/migrations/NNNN_update_status_choices.py

# Migration tasks:
# 1. Update Keywords STATUS_CHOICES in model
# 2. Data migration: pending → new, active → mapped, archived → disabled
# 3. Update Clusters status field with choices
# 4. Data migration: active → new (or map by count of ideas)
# 5. Update ContentIdeas STATUS_CHOICES
# 6. Data migration: scheduled → queued

Changes needed:

• Keywords: pending→new, active→mapped, archived→disabled
• Clusters: active→new or mapped (based on ideas_count), archived→disabled
• ContentIdeas: scheduled→queued

---

📍 Location Summary

Total Files to Update: 13

Category	Count	Files
Backend Models	1	models.py
Backend AI Functions	3	auto_cluster.py, generate_ideas.py, generate_content.py
Backend API Views	1	views.py
Frontend Configs	3	keywords.config.tsx, clusters.config.tsx, ideas.config.tsx
Frontend Pages	3	Dashboard.tsx, Keywords.tsx, Clusters.tsx, Ideas.tsx
Database Migrations	1	New migration file
TOTAL	13

---

🔄 Dependency Order

Phase 1: Backend Models & Data Migration (BLOCKER)

1. Update models.py STATUS_CHOICES for all 3 models
2. Create Django migration with data transformation
3. Run migration on database
4. Test: Backend API returns new status values

Phase 2: AI Functions (Status Update Logic)

5. Update auto_cluster.py line 297: keywords status → 'mapped'
6. Add to generate_ideas.py: clusters status → 'mapped' after ideas created
7. Update generate_content.py line 318: ideas status → 'completed' after content created
8. Test: AI workflows set correct status values

Phase 3: Backend API (Response Values)

9. Update views.py line 563: CSV import defaults to 'new'
10. Update views.py line 1084: queue to writer uses 'queued'
11. Test: API responses have new status values

Phase 4: Frontend Configuration (Display Logic)

12. Update keywords.config.tsx (3 locations: badge, filter, form)
13. Update clusters.config.tsx (3 locations: badge, filter, form)
14. Update ideas.config.tsx (3 locations: badge, filter, form)
15. Test: Filters and forms display new status values

Phase 5: Frontend Pages (State Management)

16. Update Dashboard.tsx status metrics calculations
17. Update Keywords.tsx status filters (if hardcoded)
18. Update Clusters.tsx status filters (if hardcoded)
19. Update Ideas.tsx status filters and comparisons
20. Test: Dashboard shows correct counts, filters work

---

🧪 Testing Checklist

Backend Tests

• Models accept new status values only (no old values allowed)
• Migration transforms data correctly
• Auto-cluster: Keywords change from new → mapped
• Generate ideas: Clusters change from new → mapped
• Generate content: Ideas change from queued → completed
• API returns new status values

Frontend Tests

• Keywords: Table shows new/mapped/disabled badges with correct colors
• Keywords: Filter dropdown shows new/mapped/disabled options
• Keywords: Create form defaults to 'new'
• Clusters: Table shows new/mapped/disabled badges
• Clusters: Filter shows new/mapped/disabled options
• Clusters: Create form defaults to 'new'
• Ideas: Table shows new/queued/completed/published badges
• Ideas: Filter shows all 4 status options
• Ideas: Queue to writer changes status to 'queued'
• Dashboard: Metrics count correctly with new status values
• Dashboard: Workflow percentage bars calculate correctly

Integration Tests

• Full workflow: keyword → cluster → idea → queued → completed → published
• Status transitions are unidirectional (new→mapped, queued→completed, etc.)
• Disabled items don't appear in workflow suggestions
• Bulk operations update status correctly

---

🎯 Benefits of Unified Structure

1. Consistency: All modules follow same pattern (new → mapped → disabled)
2. Clarity: Status names reflect actual workflow state, not just data state
3. Scalability: Easy to add new statuses without confusion
4. Maintainability: Single place to understand entire status flow
5. User Understanding: Users immediately understand workflow progression
6. AI Integration: AI functions have clear status update points

---

⚠️ Breaking Changes Alert

For API Consumers

• Old status values pending, active, archived, scheduled will no longer be accepted
• API will return only new, mapped, disabled, queued, completed, published
• Clients must update filtering logic

For Database

• Existing records will be migrated with data transformation
• No data loss, but status values will change
• Recommended: Backup database before running migration

For Frontend

• Old hardcoded status comparisons must be updated
• Filter options in dropdowns must match new values
• Status badges must use new color mappings

---

📋 Rollback Plan

If issues arise:

1. Reverse Django migration (restores old status values and data)
2. Revert backend code to previous version
3. Revert frontend configs to previous version
4. Test API and UI return to old behavior

---

Status: Plan Complete - Ready for Implementation
Next Step: Execute Phase 1 (Backend Models & Migration)