527 lines
14 KiB
Markdown
527 lines
14 KiB
Markdown
# Unified Status Structure Plan - Keywords, Clusters, Ideas
|
|
|
|
**Date:** December 3, 2025
|
|
**Objective:** Implement unified, workflow-driven status values across all three modules
|
|
|
|
---
|
|
|
|
## 🎯 Proposed Unified Status Structure
|
|
|
|
### Module-Specific Status Values
|
|
|
|
#### KEYWORDS Module
|
|
**Current:** `pending`, `active`, `archived`
|
|
**Proposed:** `new`, `mapped`, `disabled`
|
|
|
|
| Status | Meaning | When Used | Next Step |
|
|
|--------|---------|-----------|-----------|
|
|
| **new** | Keyword just attached to site, not yet assigned to cluster | User adds keyword | AI auto-cluster runs → `mapped` |
|
|
| **mapped** | Keyword is assigned to a cluster | AI clustering completes OR manual assignment | Can generate ideas from cluster |
|
|
| **disabled** | Keyword is archived/inactive | User manually disables | No workflow activity |
|
|
|
|
**Workflow:**
|
|
```
|
|
new → mapped → (can be used for ideas)
|
|
↓
|
|
disabled (any time)
|
|
```
|
|
|
|
---
|
|
|
|
#### CLUSTERS Module
|
|
**Current:** `active`, `pending`, `archived`
|
|
**Proposed:** `new`, `mapped`, `disabled`
|
|
|
|
| Status | Meaning | When Used | Next Step |
|
|
|--------|---------|-----------|-----------|
|
|
| **new** | Cluster created but no content ideas generated yet | AI clustering creates cluster OR manual creation | AI generate_ideas runs → `mapped` |
|
|
| **mapped** | Cluster has ideas generated from it | AI generate_ideas completes | Ideas ready to queue to writer |
|
|
| **disabled** | Cluster is archived/inactive | User manually disables | No workflow activity |
|
|
|
|
**Workflow:**
|
|
```
|
|
new → mapped → (ideas queue to writer)
|
|
↓
|
|
disabled (any time)
|
|
```
|
|
|
|
---
|
|
|
|
#### IDEAS Module
|
|
**Current:** `new`, `scheduled`, `completed`, `published`
|
|
**Proposed:** `new`, `queued`, `completed`, `published`
|
|
|
|
| Status | Meaning | When Used | Next Step |
|
|
|--------|---------|-----------|-----------|
|
|
| **new** | Idea generated by AI but not queued to writer yet | AI generate_ideas creates idea | User queues to writer → `queued` |
|
|
| **queued** | Idea queued to writer, task created in Writer module | Bulk queue to writer API called | Content generated by writer → `completed` |
|
|
| **completed** | Content generated from idea (Task completed with content) | generate_content AI function creates Content record | Manual publish → `published` |
|
|
| **published** | Content published to site | User publishes OR auto-publish | Final state |
|
|
|
|
**Workflow:**
|
|
```
|
|
new → queued → completed → published
|
|
```
|
|
|
|
---
|
|
|
|
## 📊 Status Update Flow Diagram
|
|
|
|
```
|
|
KEYWORDS WORKFLOW:
|
|
1. User adds SeedKeyword → Keywords status='new'
|
|
2. AI auto_cluster runs → Keywords status='mapped' (assigned to cluster)
|
|
3. User can disable → Keywords status='disabled'
|
|
|
|
CLUSTERS WORKFLOW:
|
|
1. AI creates cluster → Clusters status='new'
|
|
2. AI generate_ideas runs → Clusters status='mapped' (has ideas)
|
|
3. User can disable → Clusters status='disabled'
|
|
|
|
IDEAS WORKFLOW:
|
|
1. AI creates idea → Ideas status='new'
|
|
2. User bulk queues ideas → Ideas status='queued' (task created in Writer)
|
|
3. Writer AI generates content → Ideas status='completed' (Content record created)
|
|
4. User publishes content → Ideas status='published'
|
|
```
|
|
|
|
---
|
|
|
|
## 🔧 Change Locations & Impact
|
|
|
|
### Backend Model Files to Update
|
|
|
|
#### 1. `/backend/igny8_core/business/planning/models.py`
|
|
|
|
**Keywords Model (Lines 42-46):**
|
|
```python
|
|
# CURRENT:
|
|
STATUS_CHOICES = [
|
|
('active', 'Active'),
|
|
('pending', 'Pending'),
|
|
('archived', 'Archived'),
|
|
]
|
|
|
|
# PROPOSED:
|
|
STATUS_CHOICES = [
|
|
('new', 'New'),
|
|
('mapped', 'Mapped'),
|
|
('disabled', 'Disabled'),
|
|
]
|
|
|
|
# DEFAULT: Change from 'pending' → 'new'
|
|
```
|
|
|
|
**Clusters Model (Line 12):**
|
|
```python
|
|
# CURRENT:
|
|
status = models.CharField(max_length=50, default='active')
|
|
|
|
# PROPOSED:
|
|
status = models.CharField(max_length=50, default='new',
|
|
choices=[('new', 'New'), ('mapped', 'Mapped'), ('disabled', 'Disabled')])
|
|
```
|
|
|
|
**ContentIdeas Model (Lines 150-157):**
|
|
```python
|
|
# CURRENT:
|
|
STATUS_CHOICES = [
|
|
('new', 'New'),
|
|
('scheduled', 'Scheduled'),
|
|
('published', 'Published'),
|
|
]
|
|
|
|
# PROPOSED:
|
|
STATUS_CHOICES = [
|
|
('new', 'New'),
|
|
('queued', 'Queued'),
|
|
('completed', 'Completed'),
|
|
('published', 'Published'),
|
|
]
|
|
|
|
# DEFAULT: Keep 'new'
|
|
```
|
|
|
|
---
|
|
|
|
### Backend AI Function Files to Update
|
|
|
|
#### 2. `/backend/igny8_core/ai/functions/auto_cluster.py` (Line 297)
|
|
|
|
**When keywords are assigned to clusters:**
|
|
```python
|
|
# CURRENT:
|
|
status='active'
|
|
|
|
# PROPOSED:
|
|
status='mapped'
|
|
```
|
|
|
|
**Location:** In `save_output()` method when updating Keywords
|
|
|
|
---
|
|
|
|
#### 3. `/backend/igny8_core/ai/functions/generate_ideas.py` (Line 224)
|
|
|
|
**When clusters change status after ideas generated:**
|
|
```python
|
|
# CURRENT: No cluster status update
|
|
|
|
# PROPOSED: After creating ideas, update cluster status:
|
|
# In save_output() method, after creating ideas:
|
|
cluster.status = 'mapped'
|
|
cluster.save()
|
|
```
|
|
|
|
**Additional:** Ideas default stays `status='new'`
|
|
|
|
---
|
|
|
|
#### 4. `/backend/igny8_core/ai/functions/generate_content.py` (Line 318)
|
|
|
|
**When content is generated from a task:**
|
|
```python
|
|
# CURRENT:
|
|
task.status = 'completed'
|
|
task.save()
|
|
|
|
# PROPOSED: Also update the related Idea status:
|
|
task.status = 'completed'
|
|
task.save()
|
|
|
|
# NEW: Update idea to completed
|
|
if hasattr(task, 'idea') and task.idea:
|
|
idea = task.idea
|
|
idea.status = 'completed'
|
|
idea.save()
|
|
```
|
|
|
|
**Location:** In `save_output()` method after content is created
|
|
|
|
---
|
|
|
|
### Backend API Endpoint Files to Update
|
|
|
|
#### 5. `/backend/igny8_core/modules/planner/views.py`
|
|
|
|
**Line 1029 - Queue ideas to writer filter:**
|
|
```python
|
|
# CURRENT:
|
|
queueable_ideas = all_ideas.filter(status='new')
|
|
|
|
# NO CHANGE NEEDED (stays 'new')
|
|
```
|
|
|
|
**Line 1084 - Update idea status when queued:**
|
|
```python
|
|
# CURRENT:
|
|
idea.status = 'scheduled'
|
|
idea.save()
|
|
|
|
# PROPOSED:
|
|
idea.status = 'queued'
|
|
idea.save()
|
|
```
|
|
|
|
**Line 563 - Bulk create keywords from CSV:**
|
|
```python
|
|
# CURRENT:
|
|
status=row.get('status', 'pending') or 'pending'
|
|
|
|
# PROPOSED:
|
|
status=row.get('status', 'new') or 'new'
|
|
```
|
|
|
|
---
|
|
|
|
### Frontend Configuration Files to Update
|
|
|
|
#### 6. `/frontend/src/config/pages/keywords.config.tsx`
|
|
|
|
**Table Column Status Badge (Lines 230-248):**
|
|
```typescript
|
|
// CURRENT:
|
|
pending: 'amber', active: 'green', archived: 'red'
|
|
|
|
// PROPOSED:
|
|
new: 'amber', mapped: 'green', disabled: 'red'
|
|
```
|
|
|
|
**Filter Dropdown (Lines 310-318):**
|
|
```typescript
|
|
// CURRENT:
|
|
options: ['pending', 'active', 'archived']
|
|
|
|
// PROPOSED:
|
|
options: ['new', 'mapped', 'disabled']
|
|
```
|
|
|
|
**Form Field Default (Lines 560-570):**
|
|
```typescript
|
|
// CURRENT:
|
|
default: 'pending'
|
|
|
|
// PROPOSED:
|
|
default: 'new'
|
|
```
|
|
|
|
---
|
|
|
|
#### 7. `/frontend/src/config/pages/clusters.config.tsx`
|
|
|
|
**Table Column Status Badge (Lines 190-200):**
|
|
```typescript
|
|
// CURRENT:
|
|
active: 'green', archived: 'red' (inconsistent from backend)
|
|
|
|
// PROPOSED:
|
|
new: 'amber', mapped: 'green', disabled: 'red'
|
|
```
|
|
|
|
**Filter Dropdown (Lines 240-253):**
|
|
```typescript
|
|
// CURRENT:
|
|
options: ['active', 'archived'] (missing 'new')
|
|
|
|
// PROPOSED:
|
|
options: ['new', 'mapped', 'disabled']
|
|
```
|
|
|
|
**Form Field Default (Lines 405-418):**
|
|
```typescript
|
|
// CURRENT:
|
|
default: 'new'
|
|
|
|
// PROPOSED:
|
|
default: 'new' (no change, already correct)
|
|
```
|
|
|
|
---
|
|
|
|
#### 8. `/frontend/src/config/pages/ideas.config.tsx`
|
|
|
|
**Table Column Status Badge (Lines 170-185):**
|
|
```typescript
|
|
// CURRENT:
|
|
new: 'amber', scheduled: 'blue', completed: 'blue', published: 'green'
|
|
|
|
// PROPOSED:
|
|
new: 'amber', queued: 'blue', completed: 'blue', published: 'green'
|
|
```
|
|
|
|
**Filter Dropdown (Lines 218-228):**
|
|
```typescript
|
|
// CURRENT:
|
|
options: ['new', 'scheduled', 'completed', 'published']
|
|
// NOTE: missing 'completed' in some places
|
|
|
|
// PROPOSED:
|
|
options: ['new', 'queued', 'completed', 'published']
|
|
```
|
|
|
|
**Form Field Default (Lines 372-385):**
|
|
```typescript
|
|
// CURRENT:
|
|
default: 'new'
|
|
|
|
// PROPOSED:
|
|
default: 'new' (no change)
|
|
```
|
|
|
|
---
|
|
|
|
### Frontend Dashboard & State Files
|
|
|
|
#### 9. `/frontend/src/pages/Planner/Dashboard.tsx`
|
|
|
|
**Status Metrics (Lines 121-155):**
|
|
```typescript
|
|
// Updates status counting logic
|
|
// From: keywordsByStatus, clustersByStatus, ideaByStatus
|
|
// To: Use new status values
|
|
|
|
// Example refactor:
|
|
// OLD: keywordsByStatus['active'] vs keywordsByStatus['pending']
|
|
// NEW: keywordsByStatus['mapped'] vs keywordsByStatus['new']
|
|
```
|
|
|
|
**No API call changes needed** - API returns correct values
|
|
|
|
---
|
|
|
|
#### 10. `/frontend/src/services/api.ts`
|
|
|
|
**No changes needed** - Only returns what backend provides
|
|
|
|
---
|
|
|
|
### Frontend Component Files
|
|
|
|
#### 11. `/frontend/src/pages/Planner/Keywords.tsx`
|
|
|
|
**Status Filter Logic (if hardcoded):**
|
|
- Search for status comparisons
|
|
- Update from old values → new values
|
|
|
|
#### 12. `/frontend/src/pages/Planner/Clusters.tsx`
|
|
|
|
**Status Filter Logic (if hardcoded):**
|
|
- Search for status comparisons
|
|
- Update from old values → new values
|
|
|
|
#### 13. `/frontend/src/pages/Planner/Ideas.tsx`
|
|
|
|
**Status Filter Logic (if hardcoded):**
|
|
- Update from `scheduled` → `queued`
|
|
- Keep `new`, `completed`, `published`
|
|
|
|
---
|
|
|
|
### Database Migration Files
|
|
|
|
#### 14. New Django Migration Required
|
|
|
|
**File to create:** `/backend/igny8_core/business/planning/migrations/NNNN_update_status_choices.py`
|
|
|
|
```python
|
|
# Migration tasks:
|
|
# 1. Update Keywords STATUS_CHOICES in model
|
|
# 2. Data migration: pending → new, active → mapped, archived → disabled
|
|
# 3. Update Clusters status field with choices
|
|
# 4. Data migration: active → new (or map by count of ideas)
|
|
# 5. Update ContentIdeas STATUS_CHOICES
|
|
# 6. Data migration: scheduled → queued
|
|
```
|
|
|
|
**Changes needed:**
|
|
- Keywords: pending→new, active→mapped, archived→disabled
|
|
- Clusters: active→new or mapped (based on ideas_count), archived→disabled
|
|
- ContentIdeas: scheduled→queued
|
|
|
|
---
|
|
|
|
## 📍 Location Summary
|
|
|
|
### Total Files to Update: 13
|
|
|
|
| Category | Count | Files |
|
|
|----------|-------|-------|
|
|
| Backend Models | 1 | models.py |
|
|
| Backend AI Functions | 3 | auto_cluster.py, generate_ideas.py, generate_content.py |
|
|
| Backend API Views | 1 | views.py |
|
|
| Frontend Configs | 3 | keywords.config.tsx, clusters.config.tsx, ideas.config.tsx |
|
|
| Frontend Pages | 3 | Dashboard.tsx, Keywords.tsx, Clusters.tsx, Ideas.tsx |
|
|
| Database Migrations | 1 | New migration file |
|
|
| **TOTAL** | **13** | |
|
|
|
|
---
|
|
|
|
## 🔄 Dependency Order
|
|
|
|
### Phase 1: Backend Models & Data Migration (BLOCKER)
|
|
1. Update `models.py` STATUS_CHOICES for all 3 models
|
|
2. Create Django migration with data transformation
|
|
3. Run migration on database
|
|
4. Test: Backend API returns new status values
|
|
|
|
### Phase 2: AI Functions (Status Update Logic)
|
|
5. Update `auto_cluster.py` line 297: keywords status → 'mapped'
|
|
6. Add to `generate_ideas.py`: clusters status → 'mapped' after ideas created
|
|
7. Update `generate_content.py` line 318: ideas status → 'completed' after content created
|
|
8. Test: AI workflows set correct status values
|
|
|
|
### Phase 3: Backend API (Response Values)
|
|
9. Update `views.py` line 563: CSV import defaults to 'new'
|
|
10. Update `views.py` line 1084: queue to writer uses 'queued'
|
|
11. Test: API responses have new status values
|
|
|
|
### Phase 4: Frontend Configuration (Display Logic)
|
|
12. Update `keywords.config.tsx` (3 locations: badge, filter, form)
|
|
13. Update `clusters.config.tsx` (3 locations: badge, filter, form)
|
|
14. Update `ideas.config.tsx` (3 locations: badge, filter, form)
|
|
15. Test: Filters and forms display new status values
|
|
|
|
### Phase 5: Frontend Pages (State Management)
|
|
16. Update `Dashboard.tsx` status metrics calculations
|
|
17. Update `Keywords.tsx` status filters (if hardcoded)
|
|
18. Update `Clusters.tsx` status filters (if hardcoded)
|
|
19. Update `Ideas.tsx` status filters and comparisons
|
|
20. Test: Dashboard shows correct counts, filters work
|
|
|
|
---
|
|
|
|
## 🧪 Testing Checklist
|
|
|
|
### Backend Tests
|
|
- [ ] Models accept new status values only (no old values allowed)
|
|
- [ ] Migration transforms data correctly
|
|
- [ ] Auto-cluster: Keywords change from new → mapped
|
|
- [ ] Generate ideas: Clusters change from new → mapped
|
|
- [ ] Generate content: Ideas change from queued → completed
|
|
- [ ] API returns new status values
|
|
|
|
### Frontend Tests
|
|
- [ ] Keywords: Table shows new/mapped/disabled badges with correct colors
|
|
- [ ] Keywords: Filter dropdown shows new/mapped/disabled options
|
|
- [ ] Keywords: Create form defaults to 'new'
|
|
- [ ] Clusters: Table shows new/mapped/disabled badges
|
|
- [ ] Clusters: Filter shows new/mapped/disabled options
|
|
- [ ] Clusters: Create form defaults to 'new'
|
|
- [ ] Ideas: Table shows new/queued/completed/published badges
|
|
- [ ] Ideas: Filter shows all 4 status options
|
|
- [ ] Ideas: Queue to writer changes status to 'queued'
|
|
- [ ] Dashboard: Metrics count correctly with new status values
|
|
- [ ] Dashboard: Workflow percentage bars calculate correctly
|
|
|
|
### Integration Tests
|
|
- [ ] Full workflow: keyword → cluster → idea → queued → completed → published
|
|
- [ ] Status transitions are unidirectional (new→mapped, queued→completed, etc.)
|
|
- [ ] Disabled items don't appear in workflow suggestions
|
|
- [ ] Bulk operations update status correctly
|
|
|
|
---
|
|
|
|
## 🎯 Benefits of Unified Structure
|
|
|
|
1. **Consistency:** All modules follow same pattern (new → mapped → disabled)
|
|
2. **Clarity:** Status names reflect actual workflow state, not just data state
|
|
3. **Scalability:** Easy to add new statuses without confusion
|
|
4. **Maintainability:** Single place to understand entire status flow
|
|
5. **User Understanding:** Users immediately understand workflow progression
|
|
6. **AI Integration:** AI functions have clear status update points
|
|
|
|
---
|
|
|
|
## ⚠️ Breaking Changes Alert
|
|
|
|
### For API Consumers
|
|
- Old status values `pending`, `active`, `archived`, `scheduled` will no longer be accepted
|
|
- API will return only `new`, `mapped`, `disabled`, `queued`, `completed`, `published`
|
|
- Clients must update filtering logic
|
|
|
|
### For Database
|
|
- Existing records will be migrated with data transformation
|
|
- No data loss, but status values will change
|
|
- Recommended: Backup database before running migration
|
|
|
|
### For Frontend
|
|
- Old hardcoded status comparisons must be updated
|
|
- Filter options in dropdowns must match new values
|
|
- Status badges must use new color mappings
|
|
|
|
---
|
|
|
|
## 📋 Rollback Plan
|
|
|
|
If issues arise:
|
|
1. Reverse Django migration (restores old status values and data)
|
|
2. Revert backend code to previous version
|
|
3. Revert frontend configs to previous version
|
|
4. Test API and UI return to old behavior
|
|
|
|
---
|
|
|
|
**Status:** Plan Complete - Ready for Implementation
|
|
**Next Step:** Execute Phase 1 (Backend Models & Migration)
|
|
|