From 8162b6ae92eab5cb390fa0bc040b01374e0b1492 Mon Sep 17 00:00:00 2001 From: "IGNY8 VPS (Salman)" Date: Mon, 15 Dec 2025 03:15:31 +0000 Subject: [PATCH] sadas --- IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md | 1393 ++++++----------------- 1 file changed, 338 insertions(+), 1055 deletions(-) diff --git a/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md b/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md index c44749b4..81feba6a 100644 --- a/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md +++ b/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md @@ -1,54 +1,161 @@ -# Implementation Audit & Action Plan -**Pre-Launch Items 1 & 4: Status Modules + Page Flow UX** +# Implementation Audit - Reality Check +**Based on Actual Codebase Analysis** **Date:** December 15, 2025 -**Status:** Complete System Audit -**Purpose:** Identify current implementation state and define exact changes needed +**Status:** Code-Verified Action Plan +**Purpose:** Ground implementation in what actually exists, not assumptions --- -## Table of Contents +## Executive Summary -1. [UX Components Audit](#ux-components-audit) -2. [Page Structure Audit](#page-structure-audit) -3. [Module-by-Module Implementation Status](#module-by-module-implementation-status) +### What Actually Works (Code-Verified) + +✅ **PageHeader Component** (`frontend/src/components/common/PageHeader.tsx`, 170 lines) +- Handles site/sector selectors, badges, navigation tabs, last updated timestamp +- Used consistently across all Planner/Writer pages +- **No changes needed** + +✅ **ProgressModal Component** (`frontend/src/components/common/ProgressModal.tsx`, 822 lines) +- Shows step-by-step progress for AI operations +- Has visual step indicators, handles all AI functions +- Extracts success messages with counts +- **Works well, but has fragile regex parsing** + +✅ **useProgressModal Hook** (`frontend/src/hooks/useProgressModal.ts`, 815 lines) +- Polls Celery task status every 2 seconds +- Has elaborate `getStepInfo()` function (146 lines) that uses regex to extract counts +- **Problem**: Relies on message format, not structured data + +✅ **Backend ProgressTracker** (`backend/igny8_core/ai/tracker.py`, 347 lines) +- Has `update(phase, percentage, message, current, total, current_item, meta)` method +- **Supports structured data via `meta` dict** +- **Problem**: AI functions don't actually use `meta` to pass structured details + +✅ **Model Status Fields Work Fine** +- Keywords: `status` = `('new', 'New'), ('mapped', 'Mapped')` +- Clusters: `status` = `('new', 'New'), ('mapped', 'Mapped')` +- ContentIdeas: `status` = `('new', 'New'), ('queued', 'Queued'), ('completed', 'Completed')` +- Tasks: `status` = `('queued', 'Queued'), ('completed', 'Completed')` +- Content: `status` = `('draft', 'Draft'), ('review', 'Review'), ('published', 'Published')` +- **These work for workflow state tracking - don't need to change** + +### The Actual Problems + +❌ **Problem 1: Fragile Progress Data Extraction** +- Frontend uses regex to parse counts from messages: `/(\d+)\s+keyword/i`, `/(\d+)\s+cluster/i` +- **146 lines** of extraction logic in `getStepInfo()` function +- Breaks if message wording changes +- **Solution**: Backend should pass structured details, frontend reads directly + +❌ **Problem 2: Missing Workflow Guidance** +- Users don't know where they are in the workflow (no "Step 2 of 3") +- No contextual help for first-time users +- No quick metrics summary visible +- **Solution**: Add StepBanner, HelperNotification, MetricsPanel components + +❌ **Problem 3: Automation Progress Parsing Issues** +- AutomationPage tries to parse `stage_X_result` JSON +- Queue item display incorrect +- Progress bar doesn't reflect actual progress +- **Solution**: Fix stage result structure and parsing logic --- +## Action Plan (Reality-Based) -## UX Components Audit +### Phase 1: Backend - Emit Structured Progress Details (Week 1) -### Components Required by ITEM-4 but Missing +**Goal**: Make AI functions pass structured data instead of just messages -| Component | Status | Purpose | Priority | -|-----------|--------|---------|----------| -| **StepBanner** | 🔴 Missing | Show workflow step progress (Step 2/5) | High | -| **HelperNotification** | 🔴 Missing | Contextual guidance messages | High | -| **InlineGuidance** | 🔴 Missing | Help text under form fields | Medium | -| **Breadcrumbs** | 🔴 Missing | Navigation breadcrumb trail | Medium | -| **MetricsPanel** | 🔴 Missing | Collapsible bottom metrics panel | High | -| **NotificationDropdown** | 🔴 Missing | Bell icon notifications | Medium | +**Files to Update:** +1. `backend/igny8_core/ai/functions/auto_cluster.py` (347 lines) +2. `backend/igny8_core/ai/functions/generate_ideas.py` +3. `backend/igny8_core/ai/functions/generate_content.py` +4. `backend/igny8_core/ai/functions/generate_image_prompts.py` +5. `backend/igny8_core/ai/functions/generate_images.py` -### Existing Components (Good) +**Change Pattern:** -| Component | Status | Location | Notes | -|-----------|--------|----------|-------| -| **PageHeader** | ✅ Exists | `frontend/src/components/common/PageHeader.tsx` | Well-implemented, standardized | -| **ModuleNavigationTabs** | ✅ Exists | `frontend/src/components/navigation/ModuleNavigationTabs.tsx` | Works well, needs tooltips | -| **Tooltip** | ✅ Exists | `frontend/src/components/ui/tooltip/Tooltip.tsx` | Basic implementation, needs enhancement | -| **ProgressModal** | ✅ Exists | `frontend/src/components/common/ProgressModal.tsx` | Needs refactoring | +**Before:** +```python +tracker.update( + phase='SAVE', + percentage=90, + message=f"Creating {len(clusters)} clusters" +) +``` + +**After:** +```python +tracker.update( + phase='SAVE', + percentage=90, + message=f"Creating {len(clusters)} clusters", + meta={ + 'items_total': len(keywords), + 'items_processed': processed_count, + 'items_created': len(clusters), + 'current_item_name': current_cluster.name if current_cluster else None + } +) +``` + +**Effort:** ~3-4 hours per function = 15-20 hours total --- -### New Components to Create +### Phase 2: Frontend - Simplify Progress Modal (Week 1-2) -#### 1. StepBanner Component +**Goal**: Remove regex parsing, read structured details directly -**Purpose:** Display workflow step progress (e.g., "Step 2 of 5: Cluster Keywords") +**Files to Update:** +1. `frontend/src/hooks/useProgressModal.ts` (815 lines) + - **Remove**: Lines 96-242 (146 lines of regex extraction) + - **Add**: Simple `details` accessor from `state.meta.details` -**Location:** `frontend/src/components/workflow/StepBanner.tsx` (NEW) +2. `frontend/src/components/common/ProgressModal.tsx` (822 lines) + - **Simplify**: `getSuccessMessage()` to read from backend + - **Remove**: Regex patterns for extracting counts -**Props:** +**Before (useProgressModal.ts, ~146 lines):** +```typescript +const getStepInfo = (stepName: string, message: string, allSteps: any[]) => { + // Extract keyword count + let keywordCount = extractNumber(/(\d+)\s+keyword/i, message); + // ... 140 more lines of extraction logic +}; +``` + +**After (~20 lines):** +```typescript +const getStepInfo = (state: any): ProgressDisplay => { + const details = state.meta?.details || {}; + + return { + percentage: state.meta?.percentage || 0, + message: state.meta?.message || 'Processing...', + itemsTotal: details.items_total, + itemsProcessed: details.items_processed, + itemsCreated: details.items_created, + currentItemName: details.current_item_name, + }; +}; +``` + +**Effort:** ~6-8 hours + +--- + +### Phase 3: Create Missing UX Components (Week 2) + +#### 3.1 StepBanner Component + +**File:** `frontend/src/components/workflow/StepBanner.tsx` (NEW) + +**Purpose:** Show "Step 2 of 3: Organize Clusters" with clickable progress + +**Interface:** ```typescript interface StepBannerProps { currentStep: number; @@ -58,830 +165,24 @@ interface StepBannerProps { href: string; completed: boolean; }>; - onStepClick?: (stepIndex: number) => void; } ``` -**Visual:** -``` -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Step 2 of 5: Organize into Clusters - -1. Extract Keywords ✓ → 2. Cluster Keywords ● → 3. Generate Ideas → 4. Create Content → 5. Publish -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -``` - **Usage:** ```tsx +// In Clusters.tsx ``` ---- - -#### 2. HelperNotification Component - -**Purpose:** Show contextual guidance (welcome, tips, warnings) - -**Location:** `frontend/src/components/helper/HelperNotification.tsx` (NEW) - -**Props:** -```typescript -interface HelperNotificationProps { - type: 'welcome' | 'info' | 'tip' | 'warning' | 'success'; - title: string; - message: string; - actions?: Array<{ - label: string; - href?: string; - onClick?: () => void; - }>; - dismissible?: boolean; - onDismiss?: () => void; -} -``` - -**Visual:** -``` -┌─────────────────────────────────────────────────┐ -│ 💡 Welcome to Keywords │ -│ │ -│ Extract and manage SEO keywords. Start by │ -│ adding keywords manually or importing CSV. │ -│ │ -│ [Import CSV] [Learn More] [Got it] [×] │ -└─────────────────────────────────────────────────┘ -``` - -**Persistence:** Store dismissed state in localStorage per page - ---- - -#### 3. InlineGuidance Component - -**Purpose:** Small help text under form fields - -**Location:** `frontend/src/components/helper/InlineGuidance.tsx` (NEW) - -**Props:** -```typescript -interface InlineGuidanceProps { - text: string; - type?: 'info' | 'warning' | 'error'; - icon?: ReactNode; -} -``` - -**Visual:** -``` -[Input Field: Enter keyword] -ℹ️ Use lowercase, no special characters. Separate with commas. -``` - ---- - -#### 4. Breadcrumbs Component - -**Purpose:** Navigation breadcrumb trail - -**Location:** `frontend/src/components/navigation/Breadcrumbs.tsx` (NEW) - -**Props:** -```typescript -interface BreadcrumbsProps { - items: Array<{ - label: string; - href?: string; // No href = current page - }>; -} -``` - -**Visual:** -``` -Home > Setup > Sites > Site Settings -``` - ---- - -#### 5. MetricsPanel Component - -**Purpose:** Collapsible metrics panel at page bottom - -**Location:** `frontend/src/components/dashboard/MetricsPanel.tsx` (NEW) - -**Props:** -```typescript -interface MetricsPanelProps { - title: string; - metrics: Array<{ - label: string; - value: string | number; - subtitle?: string; - icon?: ReactNode; - tooltip?: string; - onClick?: () => void; - }>; - collapsible?: boolean; - defaultCollapsed?: boolean; -} -``` - -**Visual (Expanded):** -``` -┌─────────────────────────────────────────────────────────┐ -│ ▼ Keyword Metrics [Collapse] × │ -├─────────────────────────────────────────────────────────┤ -│ Total: 450 │ New: 120 │ Mapped: 330 │ Vol: 125K │ -│ ℹ️ All saved │ ℹ️ Unmapped │ ℹ️ In cluster │ ℹ️ Search │ -└─────────────────────────────────────────────────────────┘ -``` - ---- - -#### 6. NotificationDropdown Component - -**Purpose:** Bell icon with notification list - -**Location:** `frontend/src/components/notifications/NotificationDropdown.tsx` (NEW) - -**Backend Required:** `Notification` model with account FK - -**Props:** -```typescript -interface NotificationDropdownProps { - notifications: Notification[]; - unreadCount: number; - onMarkRead: (id: number) => void; - onClearAll: () => void; -} -``` - ---- - -### Tooltip Component Enhancement - -**Current Location:** `frontend/src/components/ui/tooltip/Tooltip.tsx` - -**Current Implementation:** Basic hover tooltip with portal rendering - -**Required Enhancements:** - -| Enhancement | Current | Required | -|-------------|---------|----------| -| Keyboard accessibility | ❌ No ESC handler | ✅ ESC to close, tab navigation | -| Hover delay | ❌ Immediate show | ✅ 300ms delay before show | -| Max width | ❌ Whitespace nowrap | ✅ Max-width with wrapping | -| Rich content | ❌ Text only | ✅ Support ReactNode content | -| Arrow positioning | ✅ Has arrow | ✅ Keep current (good) | - -**Example Usage (Enhanced):** -```tsx - - Clustering -

Groups keywords by semantic similarity

- Learn more → - -}> - - -``` - ---- - -## Page Structure Audit - -### Current Page Structure (Planner/Writer Pages) - -**Example: Keywords.tsx** - -```tsx -
- , color: 'blue' }} - navigation={} - /> - - {/* Actions bar */} -
- - -
- - {/* Main content - table */} - - - {/* No metrics panel */} - -``` - -### Required Page Structure (Standardized) - -**All workflow pages should follow:** - -```tsx -
- {/* 1. Page Header (EXISTS) */} - } - /> - - {/* 2. Step Banner (MISSING - needs creation) */} - {isWorkflowPage && } - - {/* 3. Helper Notification (MISSING - needs creation) */} - {showHelper && } - - {/* 4. Breadcrumbs (MISSING - needs creation) */} - {showBreadcrumbs && } - - {/* 5. Page Actions Bar (EXISTS, needs standardization) */} - - - {/* 6. Main Content (EXISTS) */} - - {/* Tables, forms, cards */} - - - {/* 7. Metrics Panel (MISSING - needs creation) */} - -
-``` - -### Page Header Usage Analysis - -**Current Status:** ✅ PageHeader is used on all major pages - -| Module | Page | Has PageHeader | Has Navigation | Missing Elements | -|--------|------|----------------|----------------|------------------| -| **Planner** | Keywords | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | -| Planner | Clusters | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | -| Planner | Ideas | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | -| **Writer** | Tasks | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | -| Writer | Content | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | -| Writer | Images | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | -| **Automation** | Automation | ✅ Yes | ❌ No tabs | Helper, stage progress fixes | - ---- - -## Module-by-Module Implementation Status - -### PLANNER MODULE - -#### Keywords Page - -**Location:** `frontend/src/pages/Planner/Keywords.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | -| StepBanner | ❌ | None | Show "Step 1/3: Extract Keywords" | Add component | -| Helper | ❌ | None | "Import keywords or add manually" | Add welcome message | -| Progress Modal | ✅ | useProgressModal for clustering | Fix message counts | Update backend tracker | -| Metrics Panel | ❌ | None | Total, New, Mapped, Avg Volume | Add component | -| Tooltips | ❌ | None | "Cluster" button, import icon | Add tooltips | -| Next Step Button | ❌ | None | "Cluster Keywords →" | Add button to actions | - -**Required Changes:** -1. Add StepBanner showing step 1/3 in Planner workflow -2. Add HelperNotification on first visit: "Import keywords or add manually to start" -3. Add Tooltip on "Cluster" button: "Group keywords by semantic similarity" -4. Add MetricsPanel at bottom with: Total keywords, New (unmapped), Mapped, Avg volume -5. Add "Next Step" button: "Cluster Keywords →" navigating to /planner/clusters -6. Backend: Update clustering progress to include keyword counts in details - ---- - -#### Clusters Page - -**Location:** `frontend/src/pages/Planner/Clusters.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | -| StepBanner | ❌ | None | Show "Step 2/3: Organize Clusters" | Add component | -| Helper | ❌ | None | "Clusters organize keywords into topic groups" | Add welcome message | -| Progress Modal | ❌ | No generation function | N/A (future feature) | None | -| Metrics Panel | ❌ | None | Total clusters, Avg size, Total volume | Add component | -| Tooltips | ❌ | None | Cluster cards, action buttons | Add tooltips | -| Next Step Button | ❌ | None | "Generate Ideas →" | Add button to actions | - -**Required Changes:** -1. Add StepBanner showing step 2/3 in Planner workflow -2. Add HelperNotification: "Clusters organize keywords into topic groups" -3. Add Tooltip on cluster cards showing keyword count and volume -4. Add MetricsPanel: Total clusters, Avg cluster size, Total volume -5. Add "Next Step" button: "Generate Ideas →" to /planner/ideas - ---- - -#### Ideas Page - -**Location:** `frontend/src/pages/Planner/Ideas.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | -| StepBanner | ❌ | None | Show "Step 3/3: Generate Ideas" | Add component | -| Helper | ❌ | None | "Content ideas generated from clusters" | Add welcome message | -| Progress Modal | ✅ | useProgressModal for ideas | Fix generic "preparing clusters" | Update backend messages | -| Metrics Panel | ❌ | None | Ideas, Cluster hubs, Word count estimates | Add component | -| Content Structure Badge | ✅ | Shows badge | Add tooltip to badge | Add tooltip | -| Next Step Button | ❌ | None | "Create Tasks →" | Add button | - -**Required Changes:** -1. Add StepBanner showing step 3/3 in Planner workflow -2. Add HelperNotification: "Content ideas are generated from your clusters" -3. Add Tooltip on content structure badge explaining structure type -4. Add MetricsPanel: Total ideas, Cluster hubs, Est. word count -5. Add "Next Step" button: "Create Tasks →" to convert ideas to tasks -6. Backend: Update idea generation to include cluster names in details - ---- - -### WRITER MODULE - -#### Tasks Page - -**Location:** `frontend/src/pages/Writer/Tasks.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | -| StepBanner | ❌ | None | Show "Step 1/3: Queue Tasks" | Add component | -| Helper | ❌ | None | "Tasks are content generation jobs" | Add welcome message | -| Progress Modal | ✅ | useProgressModal for content | Fix generic "writing article" | Update backend with title & word count | -| Metrics Panel | ❌ | None | Total, Queued, Completed, Total words | Add component | -| Status Badges | ✅ | Shows status | Add tooltips to badges | Add tooltips | -| Next Step Button | ❌ | None | "Generate Content →" | Add button | - -**Required Changes:** -1. Add StepBanner showing step 1/3 in Writer workflow -2. Add HelperNotification: "Tasks are content generation jobs queued for AI" -3. Add Tooltip on status badges explaining each status -4. Add MetricsPanel: Total tasks, Queued, Completed, Total words -5. Add "Next Step" button: "Generate Content →" trigger bulk generation -6. Backend: Include task title and target word count in progress details - ---- - -#### Content Page - -**Location:** `frontend/src/pages/Writer/Content.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | -| StepBanner | ❌ | None | Show "Step 2/3: Review Content" | Add component | -| Helper | ❌ | None | "Review and edit AI-generated content" | Add welcome message | -| Progress Modal | ✅ | For image prompts | Fix prompt count extraction | Update backend to return counts | -| Metrics Panel | ❌ | None | Total, Drafts, Published, Total words | Add component | -| Status Badges | ✅ | Shows status | Add tooltips | Add tooltips | -| Next Step Button | ❌ | None | "Generate Images →" | Add button | - -**Required Changes:** -1. Add StepBanner showing step 2/3 in Writer workflow -2. Add HelperNotification: "Review and edit AI-generated content before publishing" -3. Add Tooltip on status badges (draft/review/published) -4. Add MetricsPanel: Total content, Drafts, Published, Total words -5. Add "Next Step" button: "Generate Images →" to /writer/images -6. Backend: Return structured prompt counts in result - ---- - -#### Images Page - -**Location:** `frontend/src/pages/Writer/Images.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | -| StepBanner | ❌ | None | Show "Step 3/3: Generate Images" | Add component | -| Helper | ❌ | None | "Generate images from AI prompts" | Add welcome message | -| Progress Modal | ✅ | ImageQueueModal | Harmonize with ProgressModal | Consider refactor | -| Metrics Panel | ❌ | None | Total images, Generated, Published | Add component | -| Image Preview | ✅ | Hover preview | Add tooltip | None needed | -| Next Step Button | ❌ | None | "Publish to WordPress →" | Add button | - -**Required Changes:** -1. Add StepBanner showing step 3/3 in Writer workflow -2. Add HelperNotification: "Generate images for your content from AI prompts" -3. Add MetricsPanel: Total images, Generated, Published -4. Add "Next Step" button: "Publish to WordPress →" to publish flow -5. Consider harmonizing ImageQueueModal with ProgressModal pattern - ---- - -### AUTOMATION MODULE - -#### Automation Page - -**Location:** `frontend/src/pages/Automation/AutomationPage.tsx` - -| Feature | Status | Current | Required | Action | -|---------|--------|---------|----------|--------| -| PageHeader | ✅ | Has title, badge | No tabs (not needed) | None | -| StepBanner | ❌ | Shows stage cards instead | Keep current approach | None | -| Helper | ❌ | None | "Automation runs workflow automatically" | Add helper | -| Stage Progress | 🔴 | Broken | Fix queue items, counts | Critical fix needed | -| Progress Bar | 🔴 | Inaccurate | Fix calculation | Fix | -| Real-time Updates | 🟡 | Partial | Optimize polling | Improve | -| Metrics Display | ✅ | Stage cards show metrics | Fix accuracy | Update logic | - -**Critical Issues:** - -| Issue | Impact | Location | Fix Required | -|-------|--------|----------|--------------| -| Wrong queue items displayed | Users see incorrect tasks in queue | Stage cards | Fix stage result parsing | -| Missing queue items | Some items don't appear | Stage cards | Fix API response structure | -| Progress bar doesn't progress | Users can't see actual progress | Progress bar component | Fix percentage calculation | -| Total/processed counts buggy | Metrics don't add up | Stage cards | Fix backend stage_X_result JSON | - -**Required Changes:** -1. Add HelperNotification: "Automation runs the entire workflow automatically" -2. Fix stage progress cards to show accurate queue items: - - Parse `stage_X_result` JSON correctly - - Show pending, processing, completed, failed lists -3. Fix progress bar calculation: - - Formula: `(completed_stages * 100 / 7) + (current_stage_progress / 7)` -4. Fix stage metrics to show accurate counts: - - Keywords processed → clusters created - - Clusters processed → ideas created - - Ideas → tasks → content → images -5. Add real-time stage updates without excessive polling -6. Display estimated completion time -7. Show credit consumption per stage - ---- - -**End of Audit Document** - - -**File:** `backend/igny8_core/common/status_enums.py` (NEW) - -```python -GENERATION_STATUS_CHOICES = [ - ('pending', 'Pending'), - ('queued', 'Queued'), - ('processing', 'Processing'), - ('completed', 'Completed'), - ('failed', 'Failed'), - ('cancelled', 'Cancelled'), -] - -CONTENT_STATUS_CHOICES = [ - ('draft', 'Draft'), - ('review', 'Review'), - ('published', 'Published'), - ('archived', 'Archived'), -] -``` - ---- - -#### 1.2 Add generation_status Fields - -**Migrations Required:** - -| Model | Field | Default | Migration File | -|-------|-------|---------|----------------| -| Keywords | `generation_status` | `'pending'` | `0014_add_keywords_generation_status.py` | -| Clusters | `generation_status` | `'pending'` | `0015_add_clusters_generation_status.py` | -| Content | `generation_status` | `'pending'` | `0016_add_content_generation_status.py` | - -**Update Status Choices:** - -| Model | Action | Migration File | -|-------|--------|----------------| -| ContentIdeas | Add `processing`, `failed`, `cancelled` | `0017_update_ideas_status_choices.py` | -| Tasks | Add `processing`, `failed`, `cancelled` | `0018_update_tasks_status_choices.py` | - ---- - -#### 1.3 Enhance ProgressTracker - -**File:** `backend/igny8_core/ai/tracker.py` - -**Changes:** - -```python -class ProgressTracker: - def update( - self, - phase: str, - percentage: int, - message: str, - details: Dict = None # NEW - ): - """ - details = { - 'items_total': int, - 'items_processed': int, - 'current_item': str, - 'current_item_name': str, - 'estimated_seconds_remaining': int, - } - """ - if self.task: - self.task.update_state( - state='PROGRESS', - meta={ - 'phase': phase, - 'percentage': percentage, - 'message': message, - 'details': details or {}, # Include details - 'current': self.current, - 'total': self.total, - } - ) -``` - ---- - -#### 1.4 Update AI Function Progress Messages - -**Files to Update:** - -1. **auto_cluster.py:** - ```python - # PREP phase - tracker.update( - phase='PREP', - percentage=10, - message=f"Loading {len(keywords)} keywords for clustering", - details={ - 'items_total': len(keywords), - 'items_processed': 0, - } - ) - - # AI_CALL phase (in loop) - tracker.update( - phase='AI_CALL', - percentage=20 + (processed * 60 / total), - message=f"Analyzing keyword relationships ({processed}/{total})", - details={ - 'items_total': total, - 'items_processed': processed, - 'current_item': 'keyword', - 'current_item_name': current_keyword.keyword, - } - ) - - # SAVE phase - tracker.update( - phase='SAVE', - percentage=90, - message=f"Creating {len(clusters)} clusters", - details={ - 'items_total': len(clusters), - } - ) - ``` - -2. **generate_ideas.py:** - ```python - # PREP phase - tracker.update( - phase='PREP', - percentage=10, - message=f"Preparing {len(clusters)} cluster(s) for idea generation", - details={'items_total': len(clusters)} - ) - - # AI_CALL phase (per cluster) - tracker.update( - phase='AI_CALL', - percentage=20 + (idx * 60 / total), - message=f"Generating ideas for cluster: {cluster.name}", - details={ - 'items_total': total, - 'items_processed': idx, - 'current_item_name': cluster.name, - } - ) - ``` - -3. **generate_content.py:** - ```python - # PREP phase - tracker.update( - phase='PREP', - percentage=10, - message=f"Preparing task: {task.title}", - details={'current_item_name': task.title} - ) - - # AI_CALL phase - tracker.update( - phase='AI_CALL', - percentage=50, - message=f"Writing {task.word_count}-word article: {task.title}", - details={ - 'current_item_name': task.title, - 'word_count': task.word_count, - } - ) - ``` - -4. **generate_image_prompts.py:** - ```python - # PARSE phase - tracker.update( - phase='PARSE', - percentage=70, - message=f"Writing {len(in_article_prompts)} in-article image prompts", - details={'prompt_count': len(in_article_prompts)} - ) - ``` - -5. **generate_images.py:** - ```python - # AI_CALL phase (in loop) - tracker.update( - phase='AI_CALL', - percentage=20 + (idx * 70 / total), - message=f"Generating image {idx + 1}/{total}: {prompt[:50]}...", - details={ - 'items_total': total, - 'items_processed': idx, - 'current_item_name': prompt[:100], - } - ) - ``` - ---- - -#### 1.5 Return Structured Completion Data - -**Update Celery task return values:** - -```python -# In each AI function, return structured summary -return { - 'status': 'completed', - 'summary': { - 'message': 'Clustering complete', - 'details': f"{keywords_processed} keywords mapped into {clusters_created} clusters", - 'counts': { - 'keywords_processed': keywords_processed, - 'clusters_created': clusters_created, - } - }, - 'results': [...], -} -``` - ---- - -### Phase 2: Frontend Progress Refactor (Week 1-2) - -#### 2.1 Simplify useProgressModal Hook - -**File:** `frontend/src/hooks/useProgressModal.ts` - -**Changes:** - -1. **Remove regex extraction logic** (Lines 90-120, ~100 lines) -2. **Simplify getStepInfo():** - ```typescript - const getStepInfo = (state: any): ProgressDisplay => { - const details = state.meta?.details || {}; - - return { - percentage: state.meta?.percentage || 0, - message: state.meta?.message || 'Processing...', - itemProgress: details.items_total - ? `${details.items_processed || 0}/${details.items_total}` - : null, - currentItem: details.current_item_name || null, - }; - }; - ``` - -3. **Update polling to use backend details directly** - ---- - -#### 2.2 Simplify ProgressModal Component - -**File:** `frontend/src/components/common/ProgressModal.tsx` - -**Changes:** - -1. **Replace getSuccessMessage() with simple backend lookup:** - ```typescript - const getSuccessMessage = (taskResult: any): string => { - return taskResult?.summary?.details || 'Task completed successfully'; - }; - ``` - -2. **Remove all regex extraction** (Lines 30-130) - -3. **Use backend step definitions** instead of hardcoded function checks - ---- - -#### 2.3 Add Real-Time Status Updates to Tables - -**Files:** All page components with tables - -**Changes:** - -1. Add `status` column to tables showing badge -2. Update row status in real-time during processing -3. Show animated spinner for "processing" status -4. Auto-reload row data on completion - -**Example:** -```tsx -// In Keywords table - -``` - ---- - -### Phase 3: Create New UX Components (Week 2) - -#### 3.1 StepBanner Component - -**File:** `frontend/src/components/workflow/StepBanner.tsx` (NEW) - -**Implementation:** -```tsx -import React from 'react'; -import { Link } from 'react-router-dom'; - -interface Step { - label: string; - href: string; - completed: boolean; -} - -interface StepBannerProps { - currentStep: number; - totalSteps: number; - steps: Step[]; -} - -export default function StepBanner({ currentStep, totalSteps, steps }: StepBannerProps) { - return ( -
-

- Step {currentStep} of {totalSteps}: {steps[currentStep - 1]?.label} -

-
- {steps.map((step, idx) => ( - - {step.completed ? ( - - ✓ {idx + 1}. {step.label} - - ) : idx === currentStep - 1 ? ( - - ● {idx + 1}. {step.label} - - ) : ( - - {idx + 1}. {step.label} - - )} - {idx < steps.length - 1 && ( - → - )} - - ))} -
-
- ); -} -``` +**Effort:** ~4 hours --- @@ -889,285 +190,267 @@ export default function StepBanner({ currentStep, totalSteps, steps }: StepBanne **File:** `frontend/src/components/helper/HelperNotification.tsx` (NEW) -**Implementation:** -```tsx -import React, { useState, useEffect } from 'react'; -import Button from '../ui/button/Button'; - -interface HelperAction { - label: string; - href?: string; - onClick?: () => void; -} +**Purpose:** Dismissible contextual help (stored in localStorage) +**Interface:** +```typescript interface HelperNotificationProps { type: 'welcome' | 'info' | 'tip' | 'warning' | 'success'; title: string; message: string; - actions?: HelperAction[]; + actions?: Array<{ label: string; onClick?: () => void; }>; dismissible?: boolean; - pageKey: string; // Used for localStorage persistence -} - -export default function HelperNotification({ - type, - title, - message, - actions, - dismissible = true, - pageKey, -}: HelperNotificationProps) { - const storageKey = `helper_dismissed_${pageKey}`; - const [isDismissed, setIsDismissed] = useState(false); - - useEffect(() => { - const dismissed = localStorage.getItem(storageKey); - if (dismissed === 'true') { - setIsDismissed(true); - } - }, [storageKey]); - - const handleDismiss = () => { - localStorage.setItem(storageKey, 'true'); - setIsDismissed(true); - }; - - if (isDismissed) return null; - - const colors = { - welcome: 'bg-blue-50 border-blue-200 text-blue-900 dark:bg-blue-900/20 dark:border-blue-700 dark:text-blue-100', - info: 'bg-gray-50 border-gray-200 text-gray-900 dark:bg-gray-800/50 dark:border-gray-700 dark:text-gray-100', - tip: 'bg-green-50 border-green-200 text-green-900 dark:bg-green-900/20 dark:border-green-700 dark:text-green-100', - warning: 'bg-orange-50 border-orange-200 text-orange-900 dark:bg-orange-900/20 dark:border-orange-700 dark:text-orange-100', - success: 'bg-emerald-50 border-emerald-200 text-emerald-900 dark:bg-emerald-900/20 dark:border-emerald-700 dark:text-emerald-100', - }; - - const icons = { - welcome: '💡', - info: 'ℹ️', - tip: '✨', - warning: '⚠️', - success: '✅', - }; - - return ( -
-
- {icons[type]} -
-

{title}

-

{message}

- {actions && actions.length > 0 && ( -
- {actions.map((action, idx) => ( - - ))} -
- )} -
- {dismissible && ( - - )} -
-
- ); + pageKey: string; // For localStorage persistence } ``` ---- +**Storage:** `localStorage.setItem('helper_dismissed_keywords', 'true')` -#### 3.3 Other Components +**Usage:** +```tsx +// In Keywords.tsx + window.open('/docs/keywords') } + ]} +/> +``` -**Create these components following similar patterns:** - -1. **InlineGuidance.tsx** - Simple text with icon -2. **Breadcrumbs.tsx** - Navigation trail with links -3. **MetricsPanel.tsx** - Collapsible metrics display -4. **NotificationDropdown.tsx** - Bell icon with dropdown list +**Effort:** ~4 hours --- -### Phase 4: Update All Pages (Week 3) +#### 3.3 MetricsPanel Component -#### 4.1 Planner Pages +**File:** `frontend/src/components/dashboard/MetricsPanel.tsx` (NEW) -**Apply to:** Keywords.tsx, Clusters.tsx, Ideas.tsx +**Purpose:** Collapsible metrics summary (alternative to ModuleMetricsFooter) -**Changes per page:** -1. Import new components -2. Add StepBanner at top -3. Add HelperNotification below StepBanner -4. Add Tooltips to icons and buttons -5. Add MetricsPanel at bottom -6. Add "Next Step" button to actions bar +**Interface:** +```typescript +interface MetricsPanelProps { + title: string; + metrics: Array<{ + label: string; + value: string | number; + subtitle?: string; + tooltip?: string; + }>; + collapsible?: boolean; + defaultCollapsed?: boolean; +} +``` + +**Usage:** +```tsx +// In Keywords.tsx + +``` + +**Effort:** ~6 hours --- -#### 4.2 Writer Pages +### Phase 4: Update Pages with New Components (Week 2-3) -**Apply to:** Tasks.tsx, Content.tsx, Images.tsx +#### Planner Module -**Changes per page:** -1. Import new components -2. Add StepBanner at top -3. Add HelperNotification below StepBanner -4. Add Tooltips to status badges -5. Add MetricsPanel at bottom -6. Add "Next Step" button to actions bar +**Keywords.tsx** (998 lines) +- Add StepBanner (currentStep=1, totalSteps=3) +- Add HelperNotification for first-time guidance +- Add MetricsPanel (or keep existing ModuleMetricsFooter) +- **Effort:** 2-3 hours + +**Clusters.tsx** +- Add StepBanner (currentStep=2, totalSteps=3) +- Add HelperNotification +- Add MetricsPanel +- **Effort:** 2-3 hours + +**Ideas.tsx** +- Add StepBanner (currentStep=3, totalSteps=3) +- Add HelperNotification +- Add MetricsPanel +- **Effort:** 2-3 hours + +#### Writer Module + +**Tasks.tsx** +- Add StepBanner (currentStep=1, totalSteps=3) +- Add HelperNotification +- Add MetricsPanel +- **Effort:** 2-3 hours + +**Content.tsx** +- Add StepBanner (currentStep=2, totalSteps=3) +- Add HelperNotification +- Add MetricsPanel +- **Effort:** 2-3 hours + +**Images.tsx** +- Add StepBanner (currentStep=3, totalSteps=3) +- Add HelperNotification +- Add MetricsPanel +- **Effort:** 2-3 hours + +**Total Page Updates:** 12-18 hours --- -#### 4.3 Automation Page +### Phase 5: Fix Automation Progress Issues (Week 3) -**File:** `frontend/src/pages/Automation/AutomationPage.tsx` +**File:** `frontend/src/pages/Automation/AutomationPage.tsx` (995 lines) -**Critical Fixes:** -1. Fix stage progress card data parsing -2. Fix queue item display (pending/processing/completed/failed) -3. Fix progress bar calculation -4. Add real-time stage updates -5. Add HelperNotification +**Current Issues:** +1. Stage progress cards show wrong/missing queue items +2. Progress bar calculation inaccurate +3. Stage metrics don't add up + +**Root Cause:** Stage result JSON structure not being parsed correctly + +**Fix:** +1. Add HelperNotification for automation guidance +2. Fix stage result parsing logic +3. Fix progress bar calculation: `(completed_stages * 100 / 7) + (current_stage_progress / 7)` +4. Ensure stage metrics accurately reflect backend counts + +**Effort:** 8-10 hours --- -### Phase 5: Testing & QA (Week 3) +## Total Effort Estimate -#### 5.1 Manual AI Function Tests - -| Test | Page | Action | Expected Result | -|------|------|--------|-----------------| -| Clustering | Keywords | Select 50 keywords, run clustering | Modal shows accurate counts, creates clusters | -| Ideas | Ideas | Select 3 clusters, generate ideas | Modal shows cluster names, creates ideas | -| Content | Tasks | Select 1 task, generate | Modal shows task title & word count | -| Image Prompts | Content | Generate prompts | Modal shows "1 Featured + X In-article prompts" | -| Images | Images | Generate images | Shows each image with accurate progress | +| Phase | Description | Hours | Duration | +|-------|-------------|-------|----------| +| Phase 1 | Backend structured details | 15-20h | Week 1 | +| Phase 2 | Frontend simplification | 6-8h | Week 1-2 | +| Phase 3 | New UX components | 14h | Week 2 | +| Phase 4 | Update pages | 12-18h | Week 2-3 | +| Phase 5 | Fix automation | 8-10h | Week 3 | +| **TOTAL** | **55-70 hours** | **~2-3 weeks** | --- -#### 5.2 Automation Wizard Tests +## What NOT To Do -| Test | Expected Result | -|------|-----------------| -| Stage 1: Clustering | Shows accurate keyword → cluster counts | -| Stage 2: Ideas | Shows cluster → idea counts | -| Stage 3: Tasks | Shows idea → task conversion | -| Stage 4: Content | Shows task → content generation | -| Pause/Resume | Continues from pause point | -| Cancel | Stops and shows cancelled state | -| Queue Display | Shows correct pending/processing/completed items per stage | +❌ **Don't add `generation_status` field to models** +- Current status fields work fine for workflow tracking +- No need for DB migrations + +❌ **Don't create status enum files** +- Models already have STATUS_CHOICES inline +- No need for centralized enums + +❌ **Don't refactor ProgressModal completely** +- It works well, just needs to read structured data instead of parsing messages +- Keep existing step visualization logic + +❌ **Don't change Celery task structure** +- Task state updates work fine +- Just need to pass more structured data in `meta` --- -#### 5.3 UX Component Tests +## Testing Strategy -| Component | Test | Expected | -|-----------|------|----------| -| StepBanner | Click completed step | Navigates to that page | -| HelperNotification | Dismiss | Persists dismissal in localStorage | -| Tooltip | Hover | Shows after 300ms delay | -| MetricsPanel | Collapse | Collapses and shows summary | -| Breadcrumbs | Click | Navigates to parent pages | +### Phase 1-2 Testing (Progress System) ---- +**Test Each AI Function:** +1. Keywords → Clustering (auto_cluster.py) + - Verify `details.items_total`, `details.items_processed`, `details.items_created` appear in state + - Verify frontend displays counts without regex + - **Expected:** "50 keywords processed → 8 clusters created" -## Summary of Required Changes +2. Clusters → Ideas (generate_ideas.py) + - Verify cluster names in `details.current_item_name` + - Verify idea counts + - **Expected:** "3 clusters processed → 12 ideas created" -### Backend Changes +3. Tasks → Content (generate_content.py) + - Verify task title and word count in details + - **Expected:** "Task: How to Train Your Dog → 1500 words generated" -| Priority | Change | Files | Effort | -|----------|--------|-------|--------| -| P0 | Create unified status enums | `common/status_enums.py` (NEW) | 2h | -| P0 | Add migrations for generation_status | 4 migration files | 4h | -| P0 | Enhance ProgressTracker with details | `ai/tracker.py` | 3h | -| P0 | Update AI function messages | 5 AI function files | 8h | -| P1 | Return structured completion data | All AI functions | 4h | +4. Content → Image Prompts (generate_image_prompts.py) + - Verify prompt counts (featured + in-article) + - **Expected:** "1 featured + 5 in-article prompts created" -**Total Backend Effort:** ~21 hours +5. Image Prompts → Images (generate_images.py) + - Verify image generation progress + - **Expected:** "Generating image 3/6" ---- +### Phase 3-4 Testing (UX Components) -### Frontend Changes +**Test Each New Component:** +1. **StepBanner** + - Verify workflow position shown correctly + - Verify completed steps clickable, navigate correctly + - Verify current step highlighted -| Priority | Change | Files | Effort | -|----------|--------|-------|--------| -| P0 | Simplify useProgressModal | `hooks/useProgressModal.ts` | 4h | -| P0 | Simplify ProgressModal | `components/common/ProgressModal.tsx` | 3h | -| P0 | Create StepBanner | `components/workflow/StepBanner.tsx` (NEW) | 4h | -| P0 | Create HelperNotification | `components/helper/HelperNotification.tsx` (NEW) | 4h | -| P1 | Create MetricsPanel | `components/dashboard/MetricsPanel.tsx` (NEW) | 6h | -| P1 | Create Breadcrumbs | `components/navigation/Breadcrumbs.tsx` (NEW) | 3h | -| P2 | Create InlineGuidance | `components/helper/InlineGuidance.tsx` (NEW) | 2h | -| P2 | Enhance Tooltip | `components/ui/tooltip/Tooltip.tsx` | 3h | -| P0 | Fix Automation stage progress | `pages/Automation/AutomationPage.tsx` | 8h | -| P0 | Update Keywords page | `pages/Planner/Keywords.tsx` | 4h | -| P0 | Update Clusters page | `pages/Planner/Clusters.tsx` | 3h | -| P0 | Update Ideas page | `pages/Planner/Ideas.tsx` | 3h | -| P0 | Update Tasks page | `pages/Writer/Tasks.tsx` | 4h | -| P0 | Update Content page | `pages/Writer/Content.tsx` | 3h | -| P0 | Update Images page | `pages/Writer/Images.tsx` | 3h | +2. **HelperNotification** + - Verify dismissal persists in localStorage + - Verify actions work + - Verify doesn't show again after dismissal -**Total Frontend Effort:** ~57 hours +3. **MetricsPanel** + - Verify collapse/expand works + - Verify tooltips show on hover + - Verify metrics update in real-time ---- +### Phase 5 Testing (Automation) -### Total Project Effort - -| Phase | Effort | Duration | -|-------|--------|----------| -| Backend Foundation | 21h | Week 1 | -| Frontend Refactor | 20h | Week 1-2 | -| New Components | 22h | Week 2 | -| Page Updates | 20h | Week 3 | -| Testing & QA | 15h | Week 3 | -| **TOTAL** | **98h** | **3 weeks** | +**Test Automation Pipeline:** +1. Start automation run +2. Verify each stage shows accurate pending/processing/completed counts +3. Verify progress bar advances correctly +4. Pause and verify state persists +5. Resume and verify continues from correct stage +6. Verify stage result JSON contains all expected fields --- ## Success Criteria ### Backend -- ✅ All models have proper status fields with full choices -- ✅ All AI functions emit detailed progress with item counts -- ✅ All AI functions return structured completion data -- ✅ Progress messages are specific and accurate +✅ All AI functions emit structured details via `tracker.update(meta={...})` +✅ Details include: `items_total`, `items_processed`, `items_created`, `current_item_name` +✅ Celery task state includes details in `meta.details` ### Frontend -- ✅ Progress modal shows accurate counts without regex parsing -- ✅ All 5 new UX components created and working -- ✅ All Planner/Writer pages have StepBanner, Helper, Metrics -- ✅ Automation stage progress is accurate -- ✅ All tooltips working with keyboard accessibility +✅ useProgressModal reads details directly from `state.meta.details` +✅ No regex parsing in useProgressModal or ProgressModal +✅ Progress messages show accurate counts +✅ All new UX components created and working ### UX -- ✅ Users understand current workflow step (StepBanner) -- ✅ Users get contextual help (HelperNotification) -- ✅ Users see real-time metrics (MetricsPanel) -- ✅ Progress messages are clear and accurate -- ✅ Automation progress is transparent and accurate +✅ Users see workflow step via StepBanner +✅ Users get contextual help via HelperNotification +✅ Users see metrics summary via MetricsPanel +✅ Automation progress accurate and transparent --- ## Implementation Notes -1. **Backwards Compatibility:** New `generation_status` fields should not break existing queries -2. **Migration Strategy:** Run migrations in dev → staging → production with testing at each stage -3. **localStorage Cleanup:** Consider adding "Reset all helpers" option in settings -4. **Performance:** MetricsPanel should not cause extra API calls (use existing data) -5. **Accessibility:** All new components must support keyboard navigation -6. **Dark Mode:** All new components must have proper dark mode styles +1. **Backwards Compatibility**: New `meta.details` is additive, doesn't break existing code +2. **No DB Changes**: All changes are in application layer (tracker, frontend) +3. **localStorage Management**: Consider adding "Reset all helpers" in settings +4. **Performance**: MetricsPanel uses existing data, no extra API calls +5. **Dark Mode**: All new components must have dark mode styles +6. **Accessibility**: All new components must support keyboard navigation --- -**End of Audit Document** +**End of Reality-Based Action Plan**
- {keyword.generation_status === 'processing' && ( - }> - Processing - - )} - {keyword.generation_status === 'completed' && ( - Completed - )} -