diff --git a/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md b/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md new file mode 100644 index 00000000..c44749b4 --- /dev/null +++ b/IMPLEMENTATION-AUDIT-AND-ACTION-PLAN.md @@ -0,0 +1,1173 @@ +# Implementation Audit & Action Plan +**Pre-Launch Items 1 & 4: Status Modules + Page Flow UX** + +**Date:** December 15, 2025 +**Status:** Complete System Audit +**Purpose:** Identify current implementation state and define exact changes needed + +--- + +## Table of Contents + +1. [UX Components Audit](#ux-components-audit) +2. [Page Structure Audit](#page-structure-audit) +3. [Module-by-Module Implementation Status](#module-by-module-implementation-status) + +--- + + +## UX Components Audit + +### Components Required by ITEM-4 but Missing + +| Component | Status | Purpose | Priority | +|-----------|--------|---------|----------| +| **StepBanner** | 🔴 Missing | Show workflow step progress (Step 2/5) | High | +| **HelperNotification** | 🔴 Missing | Contextual guidance messages | High | +| **InlineGuidance** | 🔴 Missing | Help text under form fields | Medium | +| **Breadcrumbs** | 🔴 Missing | Navigation breadcrumb trail | Medium | +| **MetricsPanel** | 🔴 Missing | Collapsible bottom metrics panel | High | +| **NotificationDropdown** | 🔴 Missing | Bell icon notifications | Medium | + +### Existing Components (Good) + +| Component | Status | Location | Notes | +|-----------|--------|----------|-------| +| **PageHeader** | ✅ Exists | `frontend/src/components/common/PageHeader.tsx` | Well-implemented, standardized | +| **ModuleNavigationTabs** | ✅ Exists | `frontend/src/components/navigation/ModuleNavigationTabs.tsx` | Works well, needs tooltips | +| **Tooltip** | ✅ Exists | `frontend/src/components/ui/tooltip/Tooltip.tsx` | Basic implementation, needs enhancement | +| **ProgressModal** | ✅ Exists | `frontend/src/components/common/ProgressModal.tsx` | Needs refactoring | + +--- + +### New Components to Create + +#### 1. StepBanner Component + +**Purpose:** Display workflow step progress (e.g., "Step 2 of 5: Cluster Keywords") + +**Location:** `frontend/src/components/workflow/StepBanner.tsx` (NEW) + +**Props:** +```typescript +interface StepBannerProps { + currentStep: number; + totalSteps: number; + steps: Array<{ + label: string; + href: string; + completed: boolean; + }>; + onStepClick?: (stepIndex: number) => void; +} +``` + +**Visual:** +``` +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +Step 2 of 5: Organize into Clusters + +1. Extract Keywords ✓ → 2. Cluster Keywords ● → 3. Generate Ideas → 4. Create Content → 5. Publish +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` + +**Usage:** +```tsx + +``` + +--- + +#### 2. HelperNotification Component + +**Purpose:** Show contextual guidance (welcome, tips, warnings) + +**Location:** `frontend/src/components/helper/HelperNotification.tsx` (NEW) + +**Props:** +```typescript +interface HelperNotificationProps { + type: 'welcome' | 'info' | 'tip' | 'warning' | 'success'; + title: string; + message: string; + actions?: Array<{ + label: string; + href?: string; + onClick?: () => void; + }>; + dismissible?: boolean; + onDismiss?: () => void; +} +``` + +**Visual:** +``` +┌─────────────────────────────────────────────────┐ +│ 💡 Welcome to Keywords │ +│ │ +│ Extract and manage SEO keywords. Start by │ +│ adding keywords manually or importing CSV. │ +│ │ +│ [Import CSV] [Learn More] [Got it] [×] │ +└─────────────────────────────────────────────────┘ +``` + +**Persistence:** Store dismissed state in localStorage per page + +--- + +#### 3. InlineGuidance Component + +**Purpose:** Small help text under form fields + +**Location:** `frontend/src/components/helper/InlineGuidance.tsx` (NEW) + +**Props:** +```typescript +interface InlineGuidanceProps { + text: string; + type?: 'info' | 'warning' | 'error'; + icon?: ReactNode; +} +``` + +**Visual:** +``` +[Input Field: Enter keyword] +â„šī¸ Use lowercase, no special characters. Separate with commas. +``` + +--- + +#### 4. Breadcrumbs Component + +**Purpose:** Navigation breadcrumb trail + +**Location:** `frontend/src/components/navigation/Breadcrumbs.tsx` (NEW) + +**Props:** +```typescript +interface BreadcrumbsProps { + items: Array<{ + label: string; + href?: string; // No href = current page + }>; +} +``` + +**Visual:** +``` +Home > Setup > Sites > Site Settings +``` + +--- + +#### 5. MetricsPanel Component + +**Purpose:** Collapsible metrics panel at page bottom + +**Location:** `frontend/src/components/dashboard/MetricsPanel.tsx` (NEW) + +**Props:** +```typescript +interface MetricsPanelProps { + title: string; + metrics: Array<{ + label: string; + value: string | number; + subtitle?: string; + icon?: ReactNode; + tooltip?: string; + onClick?: () => void; + }>; + collapsible?: boolean; + defaultCollapsed?: boolean; +} +``` + +**Visual (Expanded):** +``` +┌─────────────────────────────────────────────────────────┐ +│ â–ŧ Keyword Metrics [Collapse] × │ +├─────────────────────────────────────────────────────────┤ +│ Total: 450 │ New: 120 │ Mapped: 330 │ Vol: 125K │ +│ â„šī¸ All saved │ â„šī¸ Unmapped │ â„šī¸ In cluster │ â„šī¸ Search │ +└─────────────────────────────────────────────────────────┘ +``` + +--- + +#### 6. NotificationDropdown Component + +**Purpose:** Bell icon with notification list + +**Location:** `frontend/src/components/notifications/NotificationDropdown.tsx` (NEW) + +**Backend Required:** `Notification` model with account FK + +**Props:** +```typescript +interface NotificationDropdownProps { + notifications: Notification[]; + unreadCount: number; + onMarkRead: (id: number) => void; + onClearAll: () => void; +} +``` + +--- + +### Tooltip Component Enhancement + +**Current Location:** `frontend/src/components/ui/tooltip/Tooltip.tsx` + +**Current Implementation:** Basic hover tooltip with portal rendering + +**Required Enhancements:** + +| Enhancement | Current | Required | +|-------------|---------|----------| +| Keyboard accessibility | ❌ No ESC handler | ✅ ESC to close, tab navigation | +| Hover delay | ❌ Immediate show | ✅ 300ms delay before show | +| Max width | ❌ Whitespace nowrap | ✅ Max-width with wrapping | +| Rich content | ❌ Text only | ✅ Support ReactNode content | +| Arrow positioning | ✅ Has arrow | ✅ Keep current (good) | + +**Example Usage (Enhanced):** +```tsx + + Clustering +

Groups keywords by semantic similarity

+ Learn more → + +}> + +
+``` + +--- + +## Page Structure Audit + +### Current Page Structure (Planner/Writer Pages) + +**Example: Keywords.tsx** + +```tsx +
+ , color: 'blue' }} + navigation={} + /> + + {/* Actions bar */} +
+ + +
+ + {/* Main content - table */} + + + {/* No metrics panel */} + +``` + +### Required Page Structure (Standardized) + +**All workflow pages should follow:** + +```tsx +
+ {/* 1. Page Header (EXISTS) */} + } + /> + + {/* 2. Step Banner (MISSING - needs creation) */} + {isWorkflowPage && } + + {/* 3. Helper Notification (MISSING - needs creation) */} + {showHelper && } + + {/* 4. Breadcrumbs (MISSING - needs creation) */} + {showBreadcrumbs && } + + {/* 5. Page Actions Bar (EXISTS, needs standardization) */} + + + {/* 6. Main Content (EXISTS) */} + + {/* Tables, forms, cards */} + + + {/* 7. Metrics Panel (MISSING - needs creation) */} + +
+``` + +### Page Header Usage Analysis + +**Current Status:** ✅ PageHeader is used on all major pages + +| Module | Page | Has PageHeader | Has Navigation | Missing Elements | +|--------|------|----------------|----------------|------------------| +| **Planner** | Keywords | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | +| Planner | Clusters | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | +| Planner | Ideas | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | +| **Writer** | Tasks | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | +| Writer | Content | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | +| Writer | Images | ✅ Yes | ✅ Yes | StepBanner, Helper, Metrics | +| **Automation** | Automation | ✅ Yes | ❌ No tabs | Helper, stage progress fixes | + +--- + +## Module-by-Module Implementation Status + +### PLANNER MODULE + +#### Keywords Page + +**Location:** `frontend/src/pages/Planner/Keywords.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | +| StepBanner | ❌ | None | Show "Step 1/3: Extract Keywords" | Add component | +| Helper | ❌ | None | "Import keywords or add manually" | Add welcome message | +| Progress Modal | ✅ | useProgressModal for clustering | Fix message counts | Update backend tracker | +| Metrics Panel | ❌ | None | Total, New, Mapped, Avg Volume | Add component | +| Tooltips | ❌ | None | "Cluster" button, import icon | Add tooltips | +| Next Step Button | ❌ | None | "Cluster Keywords →" | Add button to actions | + +**Required Changes:** +1. Add StepBanner showing step 1/3 in Planner workflow +2. Add HelperNotification on first visit: "Import keywords or add manually to start" +3. Add Tooltip on "Cluster" button: "Group keywords by semantic similarity" +4. Add MetricsPanel at bottom with: Total keywords, New (unmapped), Mapped, Avg volume +5. Add "Next Step" button: "Cluster Keywords →" navigating to /planner/clusters +6. Backend: Update clustering progress to include keyword counts in details + +--- + +#### Clusters Page + +**Location:** `frontend/src/pages/Planner/Clusters.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | +| StepBanner | ❌ | None | Show "Step 2/3: Organize Clusters" | Add component | +| Helper | ❌ | None | "Clusters organize keywords into topic groups" | Add welcome message | +| Progress Modal | ❌ | No generation function | N/A (future feature) | None | +| Metrics Panel | ❌ | None | Total clusters, Avg size, Total volume | Add component | +| Tooltips | ❌ | None | Cluster cards, action buttons | Add tooltips | +| Next Step Button | ❌ | None | "Generate Ideas →" | Add button to actions | + +**Required Changes:** +1. Add StepBanner showing step 2/3 in Planner workflow +2. Add HelperNotification: "Clusters organize keywords into topic groups" +3. Add Tooltip on cluster cards showing keyword count and volume +4. Add MetricsPanel: Total clusters, Avg cluster size, Total volume +5. Add "Next Step" button: "Generate Ideas →" to /planner/ideas + +--- + +#### Ideas Page + +**Location:** `frontend/src/pages/Planner/Ideas.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | +| StepBanner | ❌ | None | Show "Step 3/3: Generate Ideas" | Add component | +| Helper | ❌ | None | "Content ideas generated from clusters" | Add welcome message | +| Progress Modal | ✅ | useProgressModal for ideas | Fix generic "preparing clusters" | Update backend messages | +| Metrics Panel | ❌ | None | Ideas, Cluster hubs, Word count estimates | Add component | +| Content Structure Badge | ✅ | Shows badge | Add tooltip to badge | Add tooltip | +| Next Step Button | ❌ | None | "Create Tasks →" | Add button | + +**Required Changes:** +1. Add StepBanner showing step 3/3 in Planner workflow +2. Add HelperNotification: "Content ideas are generated from your clusters" +3. Add Tooltip on content structure badge explaining structure type +4. Add MetricsPanel: Total ideas, Cluster hubs, Est. word count +5. Add "Next Step" button: "Create Tasks →" to convert ideas to tasks +6. Backend: Update idea generation to include cluster names in details + +--- + +### WRITER MODULE + +#### Tasks Page + +**Location:** `frontend/src/pages/Writer/Tasks.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | +| StepBanner | ❌ | None | Show "Step 1/3: Queue Tasks" | Add component | +| Helper | ❌ | None | "Tasks are content generation jobs" | Add welcome message | +| Progress Modal | ✅ | useProgressModal for content | Fix generic "writing article" | Update backend with title & word count | +| Metrics Panel | ❌ | None | Total, Queued, Completed, Total words | Add component | +| Status Badges | ✅ | Shows status | Add tooltips to badges | Add tooltips | +| Next Step Button | ❌ | None | "Generate Content →" | Add button | + +**Required Changes:** +1. Add StepBanner showing step 1/3 in Writer workflow +2. Add HelperNotification: "Tasks are content generation jobs queued for AI" +3. Add Tooltip on status badges explaining each status +4. Add MetricsPanel: Total tasks, Queued, Completed, Total words +5. Add "Next Step" button: "Generate Content →" trigger bulk generation +6. Backend: Include task title and target word count in progress details + +--- + +#### Content Page + +**Location:** `frontend/src/pages/Writer/Content.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | +| StepBanner | ❌ | None | Show "Step 2/3: Review Content" | Add component | +| Helper | ❌ | None | "Review and edit AI-generated content" | Add welcome message | +| Progress Modal | ✅ | For image prompts | Fix prompt count extraction | Update backend to return counts | +| Metrics Panel | ❌ | None | Total, Drafts, Published, Total words | Add component | +| Status Badges | ✅ | Shows status | Add tooltips | Add tooltips | +| Next Step Button | ❌ | None | "Generate Images →" | Add button | + +**Required Changes:** +1. Add StepBanner showing step 2/3 in Writer workflow +2. Add HelperNotification: "Review and edit AI-generated content before publishing" +3. Add Tooltip on status badges (draft/review/published) +4. Add MetricsPanel: Total content, Drafts, Published, Total words +5. Add "Next Step" button: "Generate Images →" to /writer/images +6. Backend: Return structured prompt counts in result + +--- + +#### Images Page + +**Location:** `frontend/src/pages/Writer/Images.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge, tabs | ✅ Good | None | +| StepBanner | ❌ | None | Show "Step 3/3: Generate Images" | Add component | +| Helper | ❌ | None | "Generate images from AI prompts" | Add welcome message | +| Progress Modal | ✅ | ImageQueueModal | Harmonize with ProgressModal | Consider refactor | +| Metrics Panel | ❌ | None | Total images, Generated, Published | Add component | +| Image Preview | ✅ | Hover preview | Add tooltip | None needed | +| Next Step Button | ❌ | None | "Publish to WordPress →" | Add button | + +**Required Changes:** +1. Add StepBanner showing step 3/3 in Writer workflow +2. Add HelperNotification: "Generate images for your content from AI prompts" +3. Add MetricsPanel: Total images, Generated, Published +4. Add "Next Step" button: "Publish to WordPress →" to publish flow +5. Consider harmonizing ImageQueueModal with ProgressModal pattern + +--- + +### AUTOMATION MODULE + +#### Automation Page + +**Location:** `frontend/src/pages/Automation/AutomationPage.tsx` + +| Feature | Status | Current | Required | Action | +|---------|--------|---------|----------|--------| +| PageHeader | ✅ | Has title, badge | No tabs (not needed) | None | +| StepBanner | ❌ | Shows stage cards instead | Keep current approach | None | +| Helper | ❌ | None | "Automation runs workflow automatically" | Add helper | +| Stage Progress | 🔴 | Broken | Fix queue items, counts | Critical fix needed | +| Progress Bar | 🔴 | Inaccurate | Fix calculation | Fix | +| Real-time Updates | 🟡 | Partial | Optimize polling | Improve | +| Metrics Display | ✅ | Stage cards show metrics | Fix accuracy | Update logic | + +**Critical Issues:** + +| Issue | Impact | Location | Fix Required | +|-------|--------|----------|--------------| +| Wrong queue items displayed | Users see incorrect tasks in queue | Stage cards | Fix stage result parsing | +| Missing queue items | Some items don't appear | Stage cards | Fix API response structure | +| Progress bar doesn't progress | Users can't see actual progress | Progress bar component | Fix percentage calculation | +| Total/processed counts buggy | Metrics don't add up | Stage cards | Fix backend stage_X_result JSON | + +**Required Changes:** +1. Add HelperNotification: "Automation runs the entire workflow automatically" +2. Fix stage progress cards to show accurate queue items: + - Parse `stage_X_result` JSON correctly + - Show pending, processing, completed, failed lists +3. Fix progress bar calculation: + - Formula: `(completed_stages * 100 / 7) + (current_stage_progress / 7)` +4. Fix stage metrics to show accurate counts: + - Keywords processed → clusters created + - Clusters processed → ideas created + - Ideas → tasks → content → images +5. Add real-time stage updates without excessive polling +6. Display estimated completion time +7. Show credit consumption per stage + +--- + +**End of Audit Document** + + +**File:** `backend/igny8_core/common/status_enums.py` (NEW) + +```python +GENERATION_STATUS_CHOICES = [ + ('pending', 'Pending'), + ('queued', 'Queued'), + ('processing', 'Processing'), + ('completed', 'Completed'), + ('failed', 'Failed'), + ('cancelled', 'Cancelled'), +] + +CONTENT_STATUS_CHOICES = [ + ('draft', 'Draft'), + ('review', 'Review'), + ('published', 'Published'), + ('archived', 'Archived'), +] +``` + +--- + +#### 1.2 Add generation_status Fields + +**Migrations Required:** + +| Model | Field | Default | Migration File | +|-------|-------|---------|----------------| +| Keywords | `generation_status` | `'pending'` | `0014_add_keywords_generation_status.py` | +| Clusters | `generation_status` | `'pending'` | `0015_add_clusters_generation_status.py` | +| Content | `generation_status` | `'pending'` | `0016_add_content_generation_status.py` | + +**Update Status Choices:** + +| Model | Action | Migration File | +|-------|--------|----------------| +| ContentIdeas | Add `processing`, `failed`, `cancelled` | `0017_update_ideas_status_choices.py` | +| Tasks | Add `processing`, `failed`, `cancelled` | `0018_update_tasks_status_choices.py` | + +--- + +#### 1.3 Enhance ProgressTracker + +**File:** `backend/igny8_core/ai/tracker.py` + +**Changes:** + +```python +class ProgressTracker: + def update( + self, + phase: str, + percentage: int, + message: str, + details: Dict = None # NEW + ): + """ + details = { + 'items_total': int, + 'items_processed': int, + 'current_item': str, + 'current_item_name': str, + 'estimated_seconds_remaining': int, + } + """ + if self.task: + self.task.update_state( + state='PROGRESS', + meta={ + 'phase': phase, + 'percentage': percentage, + 'message': message, + 'details': details or {}, # Include details + 'current': self.current, + 'total': self.total, + } + ) +``` + +--- + +#### 1.4 Update AI Function Progress Messages + +**Files to Update:** + +1. **auto_cluster.py:** + ```python + # PREP phase + tracker.update( + phase='PREP', + percentage=10, + message=f"Loading {len(keywords)} keywords for clustering", + details={ + 'items_total': len(keywords), + 'items_processed': 0, + } + ) + + # AI_CALL phase (in loop) + tracker.update( + phase='AI_CALL', + percentage=20 + (processed * 60 / total), + message=f"Analyzing keyword relationships ({processed}/{total})", + details={ + 'items_total': total, + 'items_processed': processed, + 'current_item': 'keyword', + 'current_item_name': current_keyword.keyword, + } + ) + + # SAVE phase + tracker.update( + phase='SAVE', + percentage=90, + message=f"Creating {len(clusters)} clusters", + details={ + 'items_total': len(clusters), + } + ) + ``` + +2. **generate_ideas.py:** + ```python + # PREP phase + tracker.update( + phase='PREP', + percentage=10, + message=f"Preparing {len(clusters)} cluster(s) for idea generation", + details={'items_total': len(clusters)} + ) + + # AI_CALL phase (per cluster) + tracker.update( + phase='AI_CALL', + percentage=20 + (idx * 60 / total), + message=f"Generating ideas for cluster: {cluster.name}", + details={ + 'items_total': total, + 'items_processed': idx, + 'current_item_name': cluster.name, + } + ) + ``` + +3. **generate_content.py:** + ```python + # PREP phase + tracker.update( + phase='PREP', + percentage=10, + message=f"Preparing task: {task.title}", + details={'current_item_name': task.title} + ) + + # AI_CALL phase + tracker.update( + phase='AI_CALL', + percentage=50, + message=f"Writing {task.word_count}-word article: {task.title}", + details={ + 'current_item_name': task.title, + 'word_count': task.word_count, + } + ) + ``` + +4. **generate_image_prompts.py:** + ```python + # PARSE phase + tracker.update( + phase='PARSE', + percentage=70, + message=f"Writing {len(in_article_prompts)} in-article image prompts", + details={'prompt_count': len(in_article_prompts)} + ) + ``` + +5. **generate_images.py:** + ```python + # AI_CALL phase (in loop) + tracker.update( + phase='AI_CALL', + percentage=20 + (idx * 70 / total), + message=f"Generating image {idx + 1}/{total}: {prompt[:50]}...", + details={ + 'items_total': total, + 'items_processed': idx, + 'current_item_name': prompt[:100], + } + ) + ``` + +--- + +#### 1.5 Return Structured Completion Data + +**Update Celery task return values:** + +```python +# In each AI function, return structured summary +return { + 'status': 'completed', + 'summary': { + 'message': 'Clustering complete', + 'details': f"{keywords_processed} keywords mapped into {clusters_created} clusters", + 'counts': { + 'keywords_processed': keywords_processed, + 'clusters_created': clusters_created, + } + }, + 'results': [...], +} +``` + +--- + +### Phase 2: Frontend Progress Refactor (Week 1-2) + +#### 2.1 Simplify useProgressModal Hook + +**File:** `frontend/src/hooks/useProgressModal.ts` + +**Changes:** + +1. **Remove regex extraction logic** (Lines 90-120, ~100 lines) +2. **Simplify getStepInfo():** + ```typescript + const getStepInfo = (state: any): ProgressDisplay => { + const details = state.meta?.details || {}; + + return { + percentage: state.meta?.percentage || 0, + message: state.meta?.message || 'Processing...', + itemProgress: details.items_total + ? `${details.items_processed || 0}/${details.items_total}` + : null, + currentItem: details.current_item_name || null, + }; + }; + ``` + +3. **Update polling to use backend details directly** + +--- + +#### 2.2 Simplify ProgressModal Component + +**File:** `frontend/src/components/common/ProgressModal.tsx` + +**Changes:** + +1. **Replace getSuccessMessage() with simple backend lookup:** + ```typescript + const getSuccessMessage = (taskResult: any): string => { + return taskResult?.summary?.details || 'Task completed successfully'; + }; + ``` + +2. **Remove all regex extraction** (Lines 30-130) + +3. **Use backend step definitions** instead of hardcoded function checks + +--- + +#### 2.3 Add Real-Time Status Updates to Tables + +**Files:** All page components with tables + +**Changes:** + +1. Add `status` column to tables showing badge +2. Update row status in real-time during processing +3. Show animated spinner for "processing" status +4. Auto-reload row data on completion + +**Example:** +```tsx +// In Keywords table + +``` + +--- + +### Phase 3: Create New UX Components (Week 2) + +#### 3.1 StepBanner Component + +**File:** `frontend/src/components/workflow/StepBanner.tsx` (NEW) + +**Implementation:** +```tsx +import React from 'react'; +import { Link } from 'react-router-dom'; + +interface Step { + label: string; + href: string; + completed: boolean; +} + +interface StepBannerProps { + currentStep: number; + totalSteps: number; + steps: Step[]; +} + +export default function StepBanner({ currentStep, totalSteps, steps }: StepBannerProps) { + return ( +
+

+ Step {currentStep} of {totalSteps}: {steps[currentStep - 1]?.label} +

+
+ {steps.map((step, idx) => ( + + {step.completed ? ( + + ✓ {idx + 1}. {step.label} + + ) : idx === currentStep - 1 ? ( + + ● {idx + 1}. {step.label} + + ) : ( + + {idx + 1}. {step.label} + + )} + {idx < steps.length - 1 && ( + → + )} + + ))} +
+
+ ); +} +``` + +--- + +#### 3.2 HelperNotification Component + +**File:** `frontend/src/components/helper/HelperNotification.tsx` (NEW) + +**Implementation:** +```tsx +import React, { useState, useEffect } from 'react'; +import Button from '../ui/button/Button'; + +interface HelperAction { + label: string; + href?: string; + onClick?: () => void; +} + +interface HelperNotificationProps { + type: 'welcome' | 'info' | 'tip' | 'warning' | 'success'; + title: string; + message: string; + actions?: HelperAction[]; + dismissible?: boolean; + pageKey: string; // Used for localStorage persistence +} + +export default function HelperNotification({ + type, + title, + message, + actions, + dismissible = true, + pageKey, +}: HelperNotificationProps) { + const storageKey = `helper_dismissed_${pageKey}`; + const [isDismissed, setIsDismissed] = useState(false); + + useEffect(() => { + const dismissed = localStorage.getItem(storageKey); + if (dismissed === 'true') { + setIsDismissed(true); + } + }, [storageKey]); + + const handleDismiss = () => { + localStorage.setItem(storageKey, 'true'); + setIsDismissed(true); + }; + + if (isDismissed) return null; + + const colors = { + welcome: 'bg-blue-50 border-blue-200 text-blue-900 dark:bg-blue-900/20 dark:border-blue-700 dark:text-blue-100', + info: 'bg-gray-50 border-gray-200 text-gray-900 dark:bg-gray-800/50 dark:border-gray-700 dark:text-gray-100', + tip: 'bg-green-50 border-green-200 text-green-900 dark:bg-green-900/20 dark:border-green-700 dark:text-green-100', + warning: 'bg-orange-50 border-orange-200 text-orange-900 dark:bg-orange-900/20 dark:border-orange-700 dark:text-orange-100', + success: 'bg-emerald-50 border-emerald-200 text-emerald-900 dark:bg-emerald-900/20 dark:border-emerald-700 dark:text-emerald-100', + }; + + const icons = { + welcome: '💡', + info: 'â„šī¸', + tip: '✨', + warning: 'âš ī¸', + success: '✅', + }; + + return ( +
+
+ {icons[type]} +
+

{title}

+

{message}

+ {actions && actions.length > 0 && ( +
+ {actions.map((action, idx) => ( + + ))} +
+ )} +
+ {dismissible && ( + + )} +
+
+ ); +} +``` + +--- + +#### 3.3 Other Components + +**Create these components following similar patterns:** + +1. **InlineGuidance.tsx** - Simple text with icon +2. **Breadcrumbs.tsx** - Navigation trail with links +3. **MetricsPanel.tsx** - Collapsible metrics display +4. **NotificationDropdown.tsx** - Bell icon with dropdown list + +--- + +### Phase 4: Update All Pages (Week 3) + +#### 4.1 Planner Pages + +**Apply to:** Keywords.tsx, Clusters.tsx, Ideas.tsx + +**Changes per page:** +1. Import new components +2. Add StepBanner at top +3. Add HelperNotification below StepBanner +4. Add Tooltips to icons and buttons +5. Add MetricsPanel at bottom +6. Add "Next Step" button to actions bar + +--- + +#### 4.2 Writer Pages + +**Apply to:** Tasks.tsx, Content.tsx, Images.tsx + +**Changes per page:** +1. Import new components +2. Add StepBanner at top +3. Add HelperNotification below StepBanner +4. Add Tooltips to status badges +5. Add MetricsPanel at bottom +6. Add "Next Step" button to actions bar + +--- + +#### 4.3 Automation Page + +**File:** `frontend/src/pages/Automation/AutomationPage.tsx` + +**Critical Fixes:** +1. Fix stage progress card data parsing +2. Fix queue item display (pending/processing/completed/failed) +3. Fix progress bar calculation +4. Add real-time stage updates +5. Add HelperNotification + +--- + +### Phase 5: Testing & QA (Week 3) + +#### 5.1 Manual AI Function Tests + +| Test | Page | Action | Expected Result | +|------|------|--------|-----------------| +| Clustering | Keywords | Select 50 keywords, run clustering | Modal shows accurate counts, creates clusters | +| Ideas | Ideas | Select 3 clusters, generate ideas | Modal shows cluster names, creates ideas | +| Content | Tasks | Select 1 task, generate | Modal shows task title & word count | +| Image Prompts | Content | Generate prompts | Modal shows "1 Featured + X In-article prompts" | +| Images | Images | Generate images | Shows each image with accurate progress | + +--- + +#### 5.2 Automation Wizard Tests + +| Test | Expected Result | +|------|-----------------| +| Stage 1: Clustering | Shows accurate keyword → cluster counts | +| Stage 2: Ideas | Shows cluster → idea counts | +| Stage 3: Tasks | Shows idea → task conversion | +| Stage 4: Content | Shows task → content generation | +| Pause/Resume | Continues from pause point | +| Cancel | Stops and shows cancelled state | +| Queue Display | Shows correct pending/processing/completed items per stage | + +--- + +#### 5.3 UX Component Tests + +| Component | Test | Expected | +|-----------|------|----------| +| StepBanner | Click completed step | Navigates to that page | +| HelperNotification | Dismiss | Persists dismissal in localStorage | +| Tooltip | Hover | Shows after 300ms delay | +| MetricsPanel | Collapse | Collapses and shows summary | +| Breadcrumbs | Click | Navigates to parent pages | + +--- + +## Summary of Required Changes + +### Backend Changes + +| Priority | Change | Files | Effort | +|----------|--------|-------|--------| +| P0 | Create unified status enums | `common/status_enums.py` (NEW) | 2h | +| P0 | Add migrations for generation_status | 4 migration files | 4h | +| P0 | Enhance ProgressTracker with details | `ai/tracker.py` | 3h | +| P0 | Update AI function messages | 5 AI function files | 8h | +| P1 | Return structured completion data | All AI functions | 4h | + +**Total Backend Effort:** ~21 hours + +--- + +### Frontend Changes + +| Priority | Change | Files | Effort | +|----------|--------|-------|--------| +| P0 | Simplify useProgressModal | `hooks/useProgressModal.ts` | 4h | +| P0 | Simplify ProgressModal | `components/common/ProgressModal.tsx` | 3h | +| P0 | Create StepBanner | `components/workflow/StepBanner.tsx` (NEW) | 4h | +| P0 | Create HelperNotification | `components/helper/HelperNotification.tsx` (NEW) | 4h | +| P1 | Create MetricsPanel | `components/dashboard/MetricsPanel.tsx` (NEW) | 6h | +| P1 | Create Breadcrumbs | `components/navigation/Breadcrumbs.tsx` (NEW) | 3h | +| P2 | Create InlineGuidance | `components/helper/InlineGuidance.tsx` (NEW) | 2h | +| P2 | Enhance Tooltip | `components/ui/tooltip/Tooltip.tsx` | 3h | +| P0 | Fix Automation stage progress | `pages/Automation/AutomationPage.tsx` | 8h | +| P0 | Update Keywords page | `pages/Planner/Keywords.tsx` | 4h | +| P0 | Update Clusters page | `pages/Planner/Clusters.tsx` | 3h | +| P0 | Update Ideas page | `pages/Planner/Ideas.tsx` | 3h | +| P0 | Update Tasks page | `pages/Writer/Tasks.tsx` | 4h | +| P0 | Update Content page | `pages/Writer/Content.tsx` | 3h | +| P0 | Update Images page | `pages/Writer/Images.tsx` | 3h | + +**Total Frontend Effort:** ~57 hours + +--- + +### Total Project Effort + +| Phase | Effort | Duration | +|-------|--------|----------| +| Backend Foundation | 21h | Week 1 | +| Frontend Refactor | 20h | Week 1-2 | +| New Components | 22h | Week 2 | +| Page Updates | 20h | Week 3 | +| Testing & QA | 15h | Week 3 | +| **TOTAL** | **98h** | **3 weeks** | + +--- + +## Success Criteria + +### Backend +- ✅ All models have proper status fields with full choices +- ✅ All AI functions emit detailed progress with item counts +- ✅ All AI functions return structured completion data +- ✅ Progress messages are specific and accurate + +### Frontend +- ✅ Progress modal shows accurate counts without regex parsing +- ✅ All 5 new UX components created and working +- ✅ All Planner/Writer pages have StepBanner, Helper, Metrics +- ✅ Automation stage progress is accurate +- ✅ All tooltips working with keyboard accessibility + +### UX +- ✅ Users understand current workflow step (StepBanner) +- ✅ Users get contextual help (HelperNotification) +- ✅ Users see real-time metrics (MetricsPanel) +- ✅ Progress messages are clear and accurate +- ✅ Automation progress is transparent and accurate + +--- + +## Implementation Notes + +1. **Backwards Compatibility:** New `generation_status` fields should not break existing queries +2. **Migration Strategy:** Run migrations in dev → staging → production with testing at each stage +3. **localStorage Cleanup:** Consider adding "Reset all helpers" option in settings +4. **Performance:** MetricsPanel should not cause extra API calls (use existing data) +5. **Accessibility:** All new components must support keyboard navigation +6. **Dark Mode:** All new components must have proper dark mode styles + +--- + +**End of Audit Document** diff --git a/PHASE-2-COMPLETION-SUMMARY.md b/PHASE-2-COMPLETION-SUMMARY.md deleted file mode 100644 index 2390625a..00000000 --- a/PHASE-2-COMPLETION-SUMMARY.md +++ /dev/null @@ -1,242 +0,0 @@ -# Phase 2 Implementation Summary -**Date:** December 14, 2025 -**Status:** ✅ COMPLETED + Critical Fix Applied - ---- - -## What Was Accomplished - -### 🔴 **CRITICAL FIX: Issue #5 - Sidebar Navigation** - -**Problem:** Custom sidebar with organized groups only appeared on home and group-level pages, but showed default Django/Unfold sidebar on model list/detail pages. - -**Solution Implemented:** -Modified `/data/app/igny8/backend/igny8_core/admin/site.py`: - -```python -def each_context(self, request): - """Force custom sidebar on ALL pages""" - context = super().each_context(request) - custom_apps = self.get_app_list(request, app_label=None) - context['available_apps'] = custom_apps - context['app_list'] = custom_apps # Added for full compatibility - return context - -def get_app_list(self, request, app_label=None): - """ALWAYS ignore app_label parameter""" - app_dict = self._build_app_dict(request, None) # Force None - # ... builds custom organized sidebar -``` - -**Result:** -- ✅ Custom sidebar now appears consistently on ALL admin pages -- ✅ Navigation is consistent throughout entire admin interface -- ✅ Users no longer lose context when drilling into models -- ✅ Overall admin grade improved from C+ to B - ---- - -## Phase 2: Bulk Operations & Export - COMPLETED - -### Export Functionality Added - -**Auth Module:** -1. **Account Admin** - Added `AccountResource` and `ExportMixin` - - Exports: id, name, slug, owner email, plan name, status, credits, billing_country, timestamps - -2. **Site Admin** - Added `SiteResource` and `ExportMixin` - - Exports: id, name, slug, account name, industry name, domain, status, is_active, site_type, hosting_type, created_at - -3. **User Admin** - Added `UserResource` and `ExportMixin` - - Exports: id, email, username, account name, role, is_active, is_staff, created_at, last_login - -**Publishing Module:** -4. **PublishingRecord Admin** - Added `PublishingRecordResource` and `ExportMixin` - - Exports: id, content title, site name, sector name, destination, status, destination_url, published_at, created_at - -**Integration Module:** -5. **SyncEvent Admin** - Added `SyncEventResource` and `ExportMixin` - - Exports: id, integration site name, site name, event_type, action, success, external_id, description, created_at - -### Bulk Actions Added - -**Automation Module:** -- `AutomationConfig Admin`: - - ✅ `bulk_enable` - Enable selected automation configs - - ✅ `bulk_disable` - Disable selected automation configs - -**Publishing Module:** -- `PublishingRecord Admin`: - - ✅ `bulk_retry_failed` - Retry failed publishing records - -**Integration Module:** -- `SiteIntegration Admin`: - - ✅ `bulk_enable_sync` - Enable sync for selected integrations - - ✅ `bulk_disable_sync` - Disable sync for selected integrations - - ✅ `bulk_trigger_sync` - Trigger sync now for selected integrations - -- `SyncEvent Admin`: - - ✅ `bulk_mark_reviewed` - Mark sync events as reviewed (placeholder for future 'reviewed' field) - -### Previously Completed (Earlier in Phase 2) - -- ✅ Advanced Unfold filters on Tasks, Content, Keywords, Clusters, ContentIdeas -- ✅ Export on Payment, CreditTransaction, Keywords -- ✅ Bulk actions on Tasks (status changes), Content (status changes), Keywords (cluster assignment) - ---- - -## Files Modified - -1. **`/data/app/igny8/backend/igny8_core/admin/site.py`** - - Fixed `each_context()` method - - Fixed `get_app_list()` method - - Added documentation comments - -2. **`/data/app/igny8/backend/igny8_core/auth/admin.py`** - - Added import_export imports - - Created AccountResource, SiteResource, UserResource - - Updated AccountAdmin, SiteAdmin, UserAdmin with ExportMixin - -3. **`/data/app/igny8/backend/igny8_core/business/automation/admin.py`** - - Added bulk_enable and bulk_disable actions - - Added messages import - -4. **`/data/app/igny8/backend/igny8_core/business/publishing/admin.py`** - - Added import_export imports - - Created PublishingRecordResource - - Updated PublishingRecordAdmin with ExportMixin - - Added bulk_retry_failed action - -5. **`/data/app/igny8/backend/igny8_core/business/integration/admin.py`** - - Added import_export imports - - Created SyncEventResource - - Updated SiteIntegrationAdmin with bulk sync actions - - Updated SyncEventAdmin with ExportMixin and bulk_mark_reviewed - -6. **Documentation Updates:** - - `/data/app/igny8/ADMIN-IMPLEMENTATION-STATUS.md` - - `/data/app/igny8/DJANGO-ADMIN-AUDIT-REPORT.md` - ---- - -## Impact & Benefits - -### User Experience -- **Consistent Navigation:** Custom sidebar appears on every admin page -- **Better Data Export:** 10+ critical models now exportable to CSV/Excel -- **Efficient Operations:** Bulk actions reduce time for common tasks -- **Professional UI:** Unfold provides modern, clean interface throughout - -### Operational Efficiency -- **Time Savings:** - - Bulk automation enable/disable: ~30 seconds vs 5+ minutes manually - - Bulk sync operations: ~15 seconds vs 3+ minutes manually - - Export large datasets: ~10 seconds vs hours of manual data collection - -- **Reduced Errors:** Bulk operations reduce chance of missing items -- **Better Monitoring:** Export functionality enables data analysis and reporting - -### Technical Quality -- **Code Consistency:** All admins now follow same patterns -- **Maintainability:** Clear documentation and comments added -- **Performance:** No performance impact - all operations use Django ORM efficiently - ---- - -## Testing Performed - -### Manual Testing -- ✅ Verified sidebar appears on all admin pages (home, app index, model list, model detail) -- ✅ Tested export functionality on Account, Site, User models -- ✅ Tested bulk enable/disable on AutomationConfig -- ✅ Tested bulk retry on PublishingRecord -- ✅ Tested bulk sync actions on SiteIntegration -- ✅ Verified backend restart successful (no errors) -- ✅ Checked all admin pages load correctly - -### Container Health -- ✅ igny8_backend: Running -- ✅ igny8_celery_worker: Running -- ✅ igny8_celery_beat: Running -- ✅ igny8_flower: Running -- ✅ igny8_postgres: Running - ---- - -## Next Steps - -### Phase 3: Monitoring & Dashboards (Starting Next) - -**Priority Tasks:** -1. Create Celery task monitoring admin with enhanced UI -2. Build operational dashboard with key metrics: - - Account health scores - - Content production metrics - - Billing overview - - Automation status - - Integration health - -3. Implement alert system for: - - Low credits (< 100) - - Failed automations (> 5 in 7 days) - - Failed syncs (today) - - Pending payments - -4. Add account health indicators to Account admin - -**Estimated Effort:** 1-2 weeks - ---- - -## Success Metrics - -| Metric | Target | Status | -|--------|--------|--------| -| Custom sidebar on all pages | 100% | ✅ 100% | -| Export functionality coverage | 80% of critical models | ✅ 85%+ | -| Bulk action coverage | Key operational modules | ✅ Automation, Publishing, Integration | -| Backend restart time | < 60 seconds | ✅ ~45 seconds | -| No admin errors after changes | 0 errors | ✅ 0 errors | -| Overall admin grade | B or higher | ✅ B (upgraded from C+) | - ---- - -## Known Issues Remaining - -From the audit report, these are next priorities: - -1. **Issue #3 (High):** Phantom Models in System Configuration Group - - Some models listed in sidebar don't exist - - Action: Clean up sidebar configuration - -2. **Issue #25-26 (High):** Query Optimization - - Missing select_related/prefetch_related - - Action: Add to all admins with foreign keys - -3. **Issue #24 (High):** Missing Settings Admins - - SystemSettings, AccountSettings, etc. - - Action: Create admin classes - -4. **Issue #12 (Low):** Export still missing on System, Optimization, AI models - - Action: Add when those modules become more critical - ---- - -## Conclusion - -Phase 2 is **FULLY COMPLETED** with a critical bonus fix (Issue #5). The admin interface now provides: - -✅ Consistent navigation throughout -✅ Modern, professional UI via Unfold -✅ Extensive export capabilities -✅ Efficient bulk operations -✅ No styling work needed (Unfold handles it) - -**Ready to proceed to Phase 3: Monitoring & Dashboards** - ---- - -**Completed By:** GitHub Copilot -**Date:** December 14, 2025 -**Next Phase Start Date:** December 14, 2025 (can start immediately) diff --git a/django-updates/DJANGO-ADMIN-AUDIT-REPORT.md b/django-updates/DJANGO-ADMIN-AUDIT-REPORT.md deleted file mode 100644 index 27147c8f..00000000 --- a/django-updates/DJANGO-ADMIN-AUDIT-REPORT.md +++ /dev/null @@ -1,800 +0,0 @@ -# Django Admin Backend Audit Report -**Date:** December 14, 2025 -**Scope:** Complete Django Admin Implementation including Unfold Integration - ---- - -## Executive Summary - -This comprehensive audit examines the Django admin backend implementation for the IGNY8 platform, including Unfold theme integration, model registrations, sidebar organization, and admin configurations across all modules. - -### Overall Assessment -- **Admin Framework:** Unfold (Modern Django Admin Theme) -- **Total Admin Files:** 11 -- **Total Models Registered:** 42+ -- **Sidebar Groups:** 14 -- **Custom Admin Site:** `Igny8AdminSite` (extends `UnfoldAdminSite`) - ---- - -## 1. Configuration Analysis - -### 1.1 Settings Configuration (`backend/igny8_core/settings.py`) - -#### ✅ **CORRECTLY CONFIGURED** -- **Unfold installed BEFORE `django.contrib.admin`** (Line 40) -- Unfold contrib packages properly included: - - `unfold.contrib.filters` - - `unfold.contrib.import_export` - - `unfold.contrib.simple_history` -- Custom admin config: `igny8_core.admin.apps.Igny8AdminConfig` - -#### Unfold Settings (Lines 623-658) -```python -UNFOLD = { - "SITE_TITLE": "IGNY8 Administration", - "SITE_HEADER": "IGNY8 Admin", - "SITE_URL": "/", - "SITE_SYMBOL": "rocket_launch", - "SHOW_HISTORY": True, - "SHOW_VIEW_ON_SITE": True, - "SIDEBAR": { - "show_search": True, - "show_all_applications": False, # Uses custom app_list - }, -} -``` - -### 1.2 Admin App Configuration (`backend/igny8_core/admin/apps.py`) - -#### ✅ **STRENGTHS** -1. Custom `Igny8AdminConfig` properly extends `AdminConfig` -2. Registry copying mechanism preserves model registrations -3. Enhanced Celery admin setup with proper unregister/register -4. Django internal models registered with appropriate permissions - -#### âš ī¸ **ISSUES IDENTIFIED** - -**Issue #1: Registry Replacement Timing** -- **Location:** `apps.py` lines 29-34 -- **Problem:** Registry copying happens in `ready()`, but some models may register after this point -- **Impact:** Potential race conditions with late-registering models -- **Severity:** Medium - -**Issue #2: Silent Error Handling** -- **Location:** `apps.py` lines 85-89 -- **Problem:** Celery admin setup errors are logged as warnings but never surface to developers -- **Impact:** Missing enhanced Celery monitoring without notification -- **Severity:** Low - ---- - -## 2. Sidebar Organization Audit - -### 2.1 Custom Admin Site (`backend/igny8_core/admin/site.py`) - -#### Current Sidebar Structure (14 Groups) - -| Group Name | Models Count | App Label | Status | -|-----------|--------------|-----------|--------| -| **Accounts & Users** | 11 | igny8_core_auth | ✅ Complete | -| **Billing & Tenancy** | 9 | billing | ✅ Complete | -| **Writer Module** | 7 | writer | ✅ Complete | -| **Planner** | 3 | planner | ✅ Complete | -| **Publishing** | 2 | publishing | ✅ Complete | -| **Optimization** | 1 | optimization | ✅ Complete | -| **Automation** | 2 | automation | ✅ Complete | -| **Integration** | 2 | integration | ✅ Complete | -| **AI Framework** | 1 | ai | ✅ Complete | -| **System Configuration** | 12 | system | âš ī¸ **Issues Found** | -| **Celery Results** | 2 | django_celery_results | ✅ Complete | -| **Content Types** | 1 | contenttypes | ✅ Complete | -| **Administration** | 1 | admin | ✅ Complete | -| **Auth & Authorization** | 2 | auth | ✅ Complete | -| **Sessions** | 1 | sessions | ✅ Complete | - -### 2.2 Sidebar Issues Identified - -#### âš ī¸ **Issue #3: Phantom Models in System Configuration Group** -- **Location:** `site.py` lines 129-141 -- **Problem:** 12 models listed, but only 4 exist in system module -- **Missing Models:** - - `ContentTemplate` ❌ - - `TaxonomyConfig` ❌ - - `SystemSetting` ❌ - - `ContentTypeConfig` ❌ - - `PublishingChannel` ❌ - - `APIKey` ❌ - - `WebhookConfig` ❌ - - `NotificationConfig` ❌ - - `AuditLog` ❌ - -- **Actual Models in System:** - - `AIPrompt` ✅ (registered) - - `IntegrationSettings` ✅ (registered) - - `AuthorProfile` ✅ (registered) - - `Strategy` ✅ (registered) - - `SystemLog` ✅ (conditionally registered) - - `SystemStatus` ✅ (conditionally registered) - -- **Impact:** Sidebar will not display these phantom models, creating gaps in expected admin interface -- **Severity:** High - UX confusion - -#### âš ī¸ **Issue #4: Inconsistent Group Naming** -- **Location:** `site.py` line 165 -- **Problem:** Group name is "Authentication and Authorization" but should match Django's standard "Auth" -- **Impact:** Minor - inconsistent naming convention -- **Severity:** Low - -#### ✅ **Issue #5 (RESOLVED): Custom Sidebar Only Shows on Home/Group Pages** -- **Location:** `site.py` - `get_app_list()` and `each_context()` methods -- **Problem:** Custom sidebar with organized groups only displayed on: - - `/admin/` (home page) ✅ - - `/admin/{group-level-page}/` (app index pages) ✅ - - **BUT NOT ON:** `/admin/{app}/{model}/` (model list pages) ❌ - - **AND NOT ON:** `/admin/{app}/{model}/{id}/change/` (model detail pages) ❌ - -- **Symptom:** Sub-pages showed default Unfold/Django sidebar instead of custom defined groups -- **Additional Issue:** Model pages didn't show app title and icon in sidebar - -**✅ FIXED (December 14, 2025):** -- Modified `get_app_list()` to ALWAYS ignore `app_label` parameter (always pass `None` to `_build_app_dict`) -- Modified `each_context()` to set BOTH `available_apps` AND `app_list` in context -- Added documentation comments explaining the fix -- Backend restarted and tested successfully - -**Root Cause:** -- Unfold/Django passes `app_label` to `get_app_list()` on model detail pages -- This caused the method to filter apps instead of returning full custom sidebar -- Setting only `available_apps` wasn't enough - needed `app_list` too for full compatibility - -**Solution Implemented:** -```python -def each_context(self, request): - context = super().each_context(request) - custom_apps = self.get_app_list(request, app_label=None) - context['available_apps'] = custom_apps - context['app_list'] = custom_apps # Added for compatibility - return context - -def get_app_list(self, request, app_label=None): - # ALWAYS pass None to _build_app_dict - app_dict = self._build_app_dict(request, None) - # ... rest of method -``` - -- **Status:** **RESOLVED** ✅ -- **Severity:** Was CRITICAL - Now fixed - ---- - -## 3. Model Registration Audit - -### 3.1 Registration Coverage by Module - -#### ✅ **Auth Module** (`igny8_core/auth/admin.py`) -**Models Registered:** 11/11 (100%) -- User ✅ -- Account ✅ -- Plan ✅ -- Subscription ✅ -- Site ✅ -- Sector ✅ -- SiteUserAccess ✅ -- Industry ✅ -- IndustrySector ✅ -- SeedKeyword ✅ -- PasswordResetToken ✅ - -**Admin Features:** -- Custom forms with dynamic payment method choices -- Health indicators with visual styling -- Inline admins (SectorInline, IndustrySectorInline) -- Bulk actions (generate API keys) -- Account filtering with `AccountAdminMixin` - -#### ✅ **Billing Module** (`modules/billing/admin.py` + `business/billing/admin.py`) - -**âš ī¸ Issue #6: Duplicate Registrations** -- **Location:** `business/billing/admin.py` -- **Problem:** File contains commented-out duplicate registrations -- **Models Affected:** - - `CreditCostConfig` - - `Invoice` - - `Payment` - - `CreditPackage` - -- **Current State:** Only active registrations in `modules/billing/admin.py` -- **Impact:** Confusing codebase, technical debt -- **Severity:** Medium - maintainability issue - -**Models Registered:** 9/9 (100%) -- CreditTransaction ✅ -- CreditUsageLog ✅ -- Invoice ✅ -- Payment ✅ (with approval workflow) -- CreditPackage ✅ -- PaymentMethodConfig ✅ -- AccountPaymentMethod ✅ (registered in both places) -- CreditCostConfig ✅ -- PlanLimitUsage ✅ - -**Admin Features:** -- Export functionality with import_export -- Approval workflow for manual payments -- Bulk actions (approve, reject payments) -- Date range filters -- Cost change indicators -- Audit trails - -#### ✅ **Writer Module** (`modules/writer/admin.py`) -**Models Registered:** 7/7 (100%) - -Models are actually in `business/content/models.py` but registered through writer module: -- Tasks ✅ -- Content ✅ -- Images ✅ -- ContentTaxonomy ✅ -- ContentAttribute ✅ -- ContentTaxonomyRelation ✅ -- ContentClusterMap ✅ - -**âš ī¸ Issue #7: Model Location Confusion** -- **Problem:** Writer module models are actually in `business/content/models.py` -- **Location:** `modules/writer/models.py` only contains import aliases -- **Impact:** Confusing architecture, hard to locate actual model definitions -- **Severity:** Medium - maintainability issue - -**Admin Features:** -- Inline taxonomy management -- Bulk actions (status changes, taxonomy assignment, cluster assignment) -- Export functionality -- Advanced filters (Unfold contrib filters) -- Autocomplete fields - -#### ✅ **Planner Module** (`modules/planner/admin.py`) -**Models Registered:** 3/3 (100%) - -Models are in `business/planning/models.py`: -- Clusters ✅ -- Keywords ✅ -- ContentIdeas ✅ - -**âš ī¸ Issue #8: Same Model Location Confusion** -- **Problem:** Planner models are in `business/planning/` not in `modules/planner/` -- **Impact:** Architecture inconsistency -- **Severity:** Medium - -**Admin Features:** -- Bulk cluster assignment -- Status management actions -- Export functionality for Keywords -- Advanced Unfold filters - -#### ✅ **System Module** (`modules/system/admin.py`) -**Models Registered:** 6/6 (100% of existing models) -- AIPrompt ✅ -- IntegrationSettings ✅ -- AuthorProfile ✅ -- Strategy ✅ -- SystemLog ✅ (conditional) -- SystemStatus ✅ (conditional) - -**âš ī¸ Issue #9: Conditional Imports** -- **Location:** `admin.py` lines 15-32 -- **Problem:** SystemLog and SystemStatus registration wrapped in try/except -- **Impact:** Silent failures if models don't exist -- **Severity:** Low - but unclear why conditional - -**Admin Features:** -- Account-based filtering -- Read-only config fields (security) -- Import of separate settings admin modules - -#### ✅ **Publishing Module** (`business/publishing/admin.py`) -**Models Registered:** 2/2 (100%) -- PublishingRecord ✅ -- DeploymentRecord ✅ - -**Admin Features:** -- Site/Sector filtering with `SiteSectorAdminMixin` - -#### ✅ **Automation Module** (`business/automation/admin.py`) -**Models Registered:** 2/2 (100%) -- AutomationConfig ✅ -- AutomationRun ✅ - -**Admin Features:** -- Account-based filtering -- Basic list display and filters - -#### ✅ **Integration Module** (`business/integration/admin.py`) -**Models Registered:** 2/2 (100%) -- SiteIntegration ✅ -- SyncEvent ✅ - -**Admin Features:** -- Account-based filtering -- Comprehensive sync status tracking - -#### ✅ **Optimization Module** (`business/optimization/admin.py`) -**Models Registered:** 1/1 (100%) -- OptimizationTask ✅ - -**Admin Features:** -- Account-based filtering -- Credits tracking - -#### ✅ **AI Module** (`ai/admin.py`) -**Models Registered:** 1/1 (100%) -- AITaskLog ✅ - -**Admin Features:** -- Read-only (logs cannot be modified) -- Comprehensive tracking fields -- No add permission (auto-created) - ---- - -## 4. Admin Base Mixins Analysis (`admin/base.py`) - -### 4.1 AccountAdminMixin - -**Purpose:** Filter queryset by account and enforce account-based permissions - -**✅ Strengths:** -- Properly checks for superuser and developer roles -- Filters by user's account -- Implements view/change/delete permissions - -**âš ī¸ Issue #10: Inconsistent Developer Check** -- **Location:** `base.py` multiple locations -- **Problem:** Uses `hasattr(request.user, 'is_developer') and request.user.is_developer()` -- **Issue:** Assumes `is_developer` is a method, but it might be a property -- **Impact:** Potential AttributeError if implementation changes -- **Severity:** Low - but should be standardized - -### 4.2 SiteSectorAdminMixin - -**Purpose:** Filter queryset by site/sector and enforce site-based access - -**✅ Strengths:** -- Checks user's accessible sites via `get_accessible_sites()` -- Properly implements permission checks - -**âš ī¸ Issue #11: No Fallback for Missing `get_accessible_sites`** -- **Location:** `base.py` lines 71, 84, 95, 105 -- **Problem:** Uses `hasattr` check but no error handling if method exists but fails -- **Impact:** Silent failures or unexpected empty querysets -- **Severity:** Low - ---- - -## 5. Admin Features Consistency Audit - -### 5.1 Common Features Matrix - -| Feature | Auth | Billing | Writer | Planner | System | Publishing | Automation | Integration | Optimization | AI | -|---------|------|---------|--------|---------|--------|-----------|-----------|-------------|--------------|-----| -| **Unfold ModelAdmin** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| **List Display** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| **List Filters** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| **Search Fields** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| **Readonly Fields** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Partial | ✅ | ✅ | ✅ | -| **Fieldsets** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | -| **Inline Admins** | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | -| **Bulk Actions** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | -| **Export (import_export)** | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | -| **Unfold Advanced Filters** | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | -| **Autocomplete Fields** | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | -| **Date Hierarchy** | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | - -### 5.2 Inconsistency Issues - -#### âš ī¸ **Issue #12: Inconsistent Fieldsets Usage** -- **Problem:** Only Auth, Billing, Writer, Planner, and System modules use fieldsets -- **Missing In:** Publishing, Automation, Integration, Optimization, AI -- **Impact:** Inconsistent admin UI experience -- **Severity:** Low - cosmetic but affects UX - -#### âš ī¸ **Issue #12: Inconsistent Export Functionality** - **PARTIALLY RESOLVED ✅** -- **Problem:** Only Billing, Writer, and Planner had export functionality -- **Missing In:** Auth, System, Publishing, Automation, Integration, Optimization, AI - -**✅ FIXED (December 14, 2025):** -- Added export to Account admin (AccountResource) -- Added export to Site admin (SiteResource) -- Added export to User admin (UserResource) -- Added export to PublishingRecord admin (PublishingRecordResource) -- Added export to SyncEvent admin (SyncEventResource) - -**Still Missing:** -- System models (AIPrompt, Strategy, AuthorProfile) -- Optimization, AI modules - -- **Impact:** Significantly improved - most critical models now exportable -- **Severity:** Low (was Medium) - remaining gaps are lower priority models - -#### âš ī¸ **Issue #14: Inconsistent Advanced Filters** -- **Problem:** Only Billing, Writer, and Planner use Unfold's advanced filters -- **Impact:** Inconsistent filtering experience across admin -- **Severity:** Low - UX inconsistency - ---- - -## 6. Unfold Integration Analysis - -### 6.1 Theme Integration - -**✅ Strengths:** -1. All ModelAdmin classes properly extend `unfold.admin.ModelAdmin` -2. Inline admins use `unfold.admin.TabularInline` -3. Advanced filters properly imported from `unfold.contrib.filters.admin` -4. Import/export integration with `unfold.contrib.import_export` -5. Simple history integration with `unfold.contrib.simple_history` - -### 6.2 Unfold Settings - -**✅ Properly Configured:** -- Site branding (title, header, symbol) -- Color scheme (custom primary colors) -- Sidebar configuration -- History and view-on-site enabled - -**âš ī¸ Issue #15: Limited Unfold Feature Usage** -- **Problem:** Not utilizing all available Unfold features: - - No dashboard customization - - No environment badges - - No custom actions with icons - - No tabs in change forms - -- **Impact:** Missing out on enhanced admin UX -- **Severity:** Low - optional features - ---- - -## 7. Security & Permissions Audit - -### 7.1 Permission Controls - -**✅ Strengths:** -1. Account-based filtering prevents cross-account data access -2. Site/Sector filtering enforces multi-tenancy -3. Superuser and developer bypass for administrative tasks -4. Read-only admin for system tables (ContentType, LogEntry, Session) -5. Sensitive data handling (IntegrationSettings config field becomes readonly) - -**âš ī¸ Issue #16: Inconsistent Permission Checks** -- **Problem:** Some admins check permissions, others don't -- **Example:** Industry and IndustrySector have `has_delete_permission` checks, but similar global models don't -- **Impact:** Inconsistent permission enforcement -- **Severity:** Medium - security concern - -**âš ī¸ Issue #17: No Audit Trail for Sensitive Changes** -- **Problem:** CreditCostConfig tracks updater, but Payment approvals don't track all details -- **Impact:** Incomplete audit trail for billing operations -- **Severity:** Medium - compliance concern - ---- - -## 8. Code Quality & Maintainability Issues - -### 8.1 Architecture Issues - -#### âš ī¸ **Issue #18: Module vs Business Package Confusion** -- **Problem:** Models split between `modules/` and `business/` packages -- **Examples:** - - Writer models in `business/content/` - - Planner models in `business/planning/` - - Billing models in `business/billing/` - -- **Impact:** Hard to locate model definitions, confusing for new developers -- **Severity:** High - architecture issue - -#### âš ī¸ **Issue #19: Duplicate Admin Files** -- **Location:** `business/billing/admin.py` with commented-out registrations -- **Problem:** Dead code and confusion about which admin is active -- **Impact:** Technical debt, confusion -- **Severity:** Medium - -### 8.2 Code Duplication - -#### âš ī¸ **Issue #20: Repeated Safe Display Methods** -- **Problem:** Almost every admin has identical `get_X_display` methods for safe attribute access -- **Example:** `get_site_display`, `get_sector_display`, `get_account_display` -- **Impact:** Code duplication, harder maintenance -- **Severity:** Medium - should be in base mixin - -#### âš ī¸ **Issue #21: Repeated Permission Checks** -- **Problem:** Developer permission checks repeated in multiple places -- **Impact:** Hard to maintain if permission logic changes -- **Severity:** Medium - -### 8.3 Documentation Issues - -#### âš ī¸ **Issue #22: Missing Admin Docstrings** -- **Problem:** Most ModelAdmin classes lack comprehensive docstrings -- **Impact:** Hard to understand purpose and functionality -- **Severity:** Low - documentation issue - -#### âš ī¸ **Issue #23: Inconsistent Commenting** -- **Problem:** Some admins have detailed comments, others have none -- **Impact:** Inconsistent code quality -- **Severity:** Low - ---- - -## 9. Missing Features & Gaps - -### 9.1 Missing Admin Interfaces - -#### âš ī¸ **Issue #24: No Admin for Settings Models** -- **Location:** `modules/system/settings_models.py` -- **Missing Admins:** - - `SystemSettings` - - `AccountSettings` - - `UserSettings` - - `ModuleSettings` - - `ModuleEnableSettings` - - `AISettings` - -- **Note:** Admin imports reference them but they're in separate file -- **Impact:** Cannot manage system settings through admin -- **Severity:** High - functional gap - -### 9.2 Missing Bulk Actions - -**Models That Should Have Bulk Actions:** -1. Publishing (bulk publish, bulk unpublish) -2. Automation (bulk enable/disable) -3. Integration (bulk sync) -4. Optimization (bulk reoptimize) - -**Severity:** Medium - functionality limitation - -### 9.3 Missing Filters - -**Models That Need Better Filters:** -1. AITaskLog - needs phase filter, cost range filter -2. AutomationRun - needs duration filter -3. PublishingRecord - needs date range filter - -**Severity:** Low - UX improvement needed - ---- - -## 10. Performance Concerns - -### 10.1 Query Optimization - -#### âš ī¸ **Issue #25: Missing select_related/prefetch_related** -- **Problem:** Most admins don't optimize queries -- **Exceptions:** - - ContentTaxonomy admin uses `select_related` - - ContentAttribute admin uses `select_related` - -- **Impact:** N+1 query problems, slow admin pages -- **Severity:** High - performance issue - -#### âš ī¸ **Issue #26: No List Select Related** -- **Problem:** None of the admins define `list_select_related` -- **Impact:** Multiple queries for foreign key displays in list view -- **Severity:** High - performance issue - -### 10.2 Large Dataset Handling - -#### âš ī¸ **Issue #27: No Pagination Configuration** -- **Problem:** No custom `list_per_page` settings for models with large datasets -- **Models Affected:** Content, Tasks, Keywords, Payment, CreditTransaction -- **Impact:** Slow page loads for large datasets -- **Severity:** Medium - ---- - -## 11. Critical Issues Summary - -### ✅ Critical/Blocker Issues - RESOLVED - -1. **Issue #5: Custom Sidebar Only Shows on Home/Group Pages** - **RESOLVED ✅** - - **Fix Applied:** Modified `get_app_list()` and `each_context()` to force custom sidebar on all pages - - **Date Fixed:** December 14, 2025 - - **Files Modified:** `/data/app/igny8/backend/igny8_core/admin/site.py` - -### High Severity Issues (Must Fix) - -2. **Issue #3:** Phantom Models in System Configuration Group - - **Action:** Remove non-existent models from sidebar configuration - -3. **Issue #18:** Module vs Business Package Confusion - - **Action:** Consolidate models or document architecture clearly - -4. **Issue #24:** Missing Admin for Settings Models - - **Action:** Create admin interfaces for system settings - -5. **Issue #25:** Missing select_related/prefetch_related - - **Action:** Add query optimization to all admins - -6. **Issue #26:** No List Select Related - - **Action:** Add `list_select_related` to all relevant admins - -### Medium Severity Issues (Should Fix) - -7. **Issue #1:** Registry Replacement Timing -8. **Issue #6:** Duplicate Registrations -9. **Issue #7:** Model Location Confusion -10. **Issue #8:** Same Model Location Confusion -11. **Issue #13:** Inconsistent Export Functionality -12. **Issue #16:** Inconsistent Permission Checks -13. **Issue #17:** No Audit Trail for Sensitive Changes -14. **Issue #19:** Duplicate Admin Files -15. **Issue #20:** Repeated Safe Display Methods -16. **Issue #21:** Repeated Permission Checks -17. **Issue #27:** No Pagination Configuration - -### Low Severity Issues (Nice to Have) - -18. All remaining issues (2, 4, 9, 10, 11, 12, 14, 15, 22, 23) - ---- - -## 12. Recommendations - -### 12.1 Immediate Actions (Critical - Fix Today) - -1. **✅ COMPLETED: Fix Custom Sidebar on All Pages (Issue #5)** - - The custom sidebar now appears on ALL admin pages including model list/detail views. - - **Solution Implemented:** - ```python - # In Igny8AdminSite class - def each_context(self, request): - """Ensure custom app_list is ALWAYS used""" - context = super().each_context(request) - custom_apps = self.get_app_list(request, app_label=None) - context['available_apps'] = custom_apps - context['app_list'] = custom_apps # Also set for compatibility - return context - - def get_app_list(self, request, app_label=None): - """IGNORE app_label to always show full custom sidebar""" - app_dict = self._build_app_dict(request, None) # Always pass None - # ... rest of method - ``` - -2. **Fix Sidebar Configuration (Issue #3)** - ```python - # Remove phantom models from System Configuration group - 'System Configuration': { - 'models': [ - ('system', 'AIPrompt'), - ('system', 'IntegrationSettings'), - ('system', 'Strategy'), - ('system', 'AuthorProfile'), - ], - }, - ``` - -3. **Add Query Optimization** - ```python - # Example for all admins with foreign keys - list_select_related = ['account', 'site', 'sector'] - - def get_queryset(self, request): - qs = super().get_queryset(request) - return qs.select_related('account', 'site', 'sector') - ``` - -4. **Create Missing Settings Admins** - - Implement admin classes for all settings models - - Add proper permissions and filtering - -### 12.2 Short-term Improvements (1-2 weeks) - -1. **Consolidate Safe Display Methods** - ```python - # Add to base.py - class EnhancedAdminMixin: - def get_safe_related_display(self, obj, field_name, display_attr='name'): - try: - related = getattr(obj, field_name, None) - return getattr(related, display_attr, '-') if related else '-' - except: - return '-' - ``` - -2. **Add Export to Critical Models** - - Auth models (User, Account, Site) - - System models (AIPrompt, Strategy) - - Publishing, Automation, Integration models - -3. **Standardize Bulk Actions** - - Add status change actions to all models with status fields - - Add enable/disable actions where applicable - -4. **Clean Up Dead Code** - - Remove commented-out code in `business/billing/admin.py` - - Remove backup files (`site_backup.py`, `site_old.py`) - -### 12.3 Long-term Enhancements (1+ months) - -1. **Architecture Reorganization** - - Decide on single location for models (business/ or modules/) - - Update imports and references - - Document architecture decisions - -2. **Enhanced Unfold Integration** - - Add custom dashboard - - Implement environment badges - - Add tabs for complex forms - - Custom actions with icons - -3. **Comprehensive Admin Documentation** - - Document each admin class purpose - - Create admin user guide - - Add inline help text - -4. **Advanced Features** - - Implement admin actions logging - - Add data visualization for analytics - - Create custom admin reports - ---- - -## 13. Testing Recommendations - -### 13.1 Manual Testing Checklist - -- [ ] Verify all sidebar groups display correctly -- [ ] Check that all models appear in correct groups -- [ ] Test account filtering for all admins -- [ ] Test site/sector filtering for relevant admins -- [ ] Verify bulk actions work correctly -- [ ] Test export functionality -- [ ] Check permission enforcement -- [ ] Test search functionality -- [ ] Verify filters work properly -- [ ] Test inline admins - -### 13.2 Automated Testing - -Create admin tests for: -1. Model registration coverage -2. Permission checks -3. Query optimization (query count tests) -4. Bulk action functionality -5. Export functionality - ---- - -## 14. Conclusion - -The IGNY8 Django admin implementation is **functionally complete** with comprehensive model coverage and modern UI via Unfold integration. However, there are **significant inconsistencies, architectural issues, and performance concerns** that need to be addressed. - -### Key Metrics - -- **Registration Coverage:** 42+ models, ~98% coverage -- **Unfold Integration:** Strong (all admins use Unfold) -- **Feature Consistency:** Moderate (60-70%) -- **Code Quality:** Moderate (significant duplication) -- **Performance:** Poor (missing query optimization) -- **Documentation:** Poor (minimal docstrings) - -### Priority Fixes - -**✅ Day 1 COMPLETED (Dec 14, 2025):** Fixed custom sidebar on all admin pages (Issue #5) -**Week 1:** Fix sidebar phantom models, add query optimization -**Week 2:** Add settings admins, consolidate safe display methods -**Week 3:** Add export functionality to remaining models, clean up dead code -**Week 4:** Standardize bulk actions and filters - -### Overall Grade: **B** - -*Upgraded from C+ due to critical sidebar navigation issue being RESOLVED.* - -The admin works well for daily use but needs refactoring for maintainability and performance optimization. - ---- - -**Audit Completed By:** GitHub Copilot -**Date:** December 14, 2025 diff --git a/django-updates/DJANGO-ADMIN-IMPROVEMENT-PLAN.md b/django-updates/DJANGO-ADMIN-IMPROVEMENT-PLAN.md deleted file mode 100644 index fff9addf..00000000 --- a/django-updates/DJANGO-ADMIN-IMPROVEMENT-PLAN.md +++ /dev/null @@ -1,1551 +0,0 @@ -# Django Admin Improvement Plan - Unfold Edition - -**Version:** 2.0.0 -**Created:** December 14, 2025 -**Status:** Implementation Phase -**Priority:** 🔴 High - 3-4 weeks implementation -**Dependencies:** After Plan Management Implementation -**Theme:** Django Unfold (Modern Admin Theme) - ---- - -## 🔧 CRITICAL FIXES APPLIED (December 14, 2025) - -### Problem: Multiple Conflicting Admin Systems -The system had **3 conflicting admin systems running simultaneously**: -1. Default Django admin -2. Custom IGNY8 admin modifications -3. Unfold theme (partially installed) - -This caused style conflicts, crashes, and inconsistent UI. - -### Solution: Clean Unfold-Only Installation - -**✅ Fixed Issues:** -1. **Backend Container Crashing** - Missing `unfold` module - - Added `django-unfold==0.73.1` to requirements.txt - - Rebuilt Docker image: `igny8-backend:latest` - - All containers now use new image (backend, celery_worker, celery_beat, flower) - -2. **Admin Apps Configuration Error** - `AttributeError: module 'django.contrib.admin' has no attribute 'apps'` - - Fixed import in `admin/apps.py`: `from django.contrib.admin.apps import AdminConfig` - - Import Unfold AFTER apps are ready in `ready()` method - -3. **Admin Site Inheritance** - Mixed admin systems - - Changed `Igny8AdminSite` from `admin.AdminSite` → `UnfoldAdminSite` - - All admin classes now inherit from Unfold's `ModelAdmin` - -4. **Celery Admin Filters** - Using wrong filter classes - - Changed from `DateRangeFilter` → `RangeDateFilter` (Unfold version) - - Updated `CeleryTaskResultAdmin` to use `ModelAdmin` from Unfold - -5. **Middleware Configuration** - - Added `simple_history.middleware.HistoryRequestMiddleware` - -6. **INSTALLED_APPS Order** - - Unfold apps MUST be before `django.contrib.admin` - - Configured properly in settings.py - -7. **UserAdmin Popup Styling** - User edit form in popup used default Django styling - - Changed `UserAdmin` to inherit from both `BaseUserAdmin` and `ModelAdmin` - - Multiple inheritance preserves Django user functionality + Unfold styling - - Popups now use Unfold templates with modern icons and widgets - -**✅ Result: Single Clean Admin System** -- **ONLY Unfold** - No more conflicts -- Modern, responsive UI with Tailwind CSS -- Dark mode support -- Advanced filters and bulk operations built-in -- All popups and forms use Unfold styling -- All containers running healthy - ---- - -## Executive Summary - -This document outlines a comprehensive improvement plan for the IGNY8 Django Admin interface using **Django Unfold**, a modern, feature-rich admin theme. Unfold provides a beautiful UI based on Tailwind CSS, advanced filtering, bulk operations, dark mode, and extensive customization options - eliminating the need for custom CSS/JS while providing enterprise-grade functionality out of the box. - -**Key Objectives:** -- ✅ **COMPLETED:** Install and configure Unfold theme -- Leverage Unfold's built-in features for UI/UX excellence -- Reorganize sidebar menu using Unfold's navigation system -- Remove unused/empty models -- Implement bulk operations with Unfold's action system -- Add Celery task monitoring with Unfold integration -- Create operational dashboards using Unfold's dashboard tools -- Implement advanced filtering with Unfold's filter extensions - -**What Makes This Different:** -- **Modern UI Out-of-the-Box:** No custom CSS needed - Unfold provides beautiful, responsive design -- **Built-in Features:** Bulk operations, advanced filters, charts, dark mode included -- **Django Integration:** Works seamlessly with django-import-export, django-simple-history, django-celery-results -- **Extensible:** Easy to customize with Unfold's configuration system - ---- - -## Current State Analysis - -### ✅ Completed (Phase 0) - December 14, 2025 - -1. ✅ **Unfold Installation** - django-unfold==0.73.1 installed in requirements.txt -2. ✅ **Docker Image Rebuilt** - igny8-backend:latest rebuilt with all dependencies -3. ✅ **Settings Configuration** - UNFOLD settings configured in settings.py -4. ✅ **Admin Site Update** - Igny8AdminSite now inherits from UnfoldAdminSite -5. ✅ **Admin Apps Fixed** - Igny8AdminConfig properly imports Unfold after apps ready -6. ✅ **Celery Admin Updated** - CeleryTaskResultAdmin uses Unfold ModelAdmin & filters -7. ✅ **Supporting Packages** - django-simple-history, django-import-export, django-celery-results installed -8. ✅ **Middleware Updated** - simple_history.middleware.HistoryRequestMiddleware added -9. ✅ **Static Files** - Unfold assets collected via collectstatic -10. ✅ **All Containers Running** - Backend, Celery Worker, Celery Beat, Flower all healthy -11. ✅ **Conflicts Resolved** - No more mixed admin systems (default + custom + Unfold) -12. ✅ **Single Admin System** - **Unfold ONLY** - clean, modern, no conflicts - -### ✅ Strengths - -1. **Modern UI Theme** - Unfold provides beautiful, responsive Tailwind-based design with zero custom CSS -2. **Consistent Styling** - All admin pages, popups, and forms use Unfold templates -3. **Custom Admin Site** - Igny8AdminSite with logical grouping (maintained) -4. **Multi-Tenancy Support** - AccountAdminMixin and SiteSectorAdminMixin -5. **Payment Approval Workflow** - Comprehensive payment approval system -6. **Custom Actions** - API key generation, payment approval/rejection -7. **Field Customization** - Custom fieldsets and readonly fields - -### âš ī¸ Issues Remaining - -#### 1. **Sidebar Menu Organization** -- ✅ Current get_app_list() structure works -- Missing PlanLimitUsage model (needs to be added to admin) -- Some empty groups appearing (site_building models don't exist) -- Need to clean up model groupings - -#### 2. **Unused/Empty Models** (Same as before) -- **site_building** models referenced but don't exist -- Duplicate model registrations need cleanup - -#### 3. **Missing Features** (Now easier with Unfold) -- CSV/Excel export - Can use Unfold's import_export integration -- Bulk operations - Use Unfold's enhanced action system -- Celery monitoring - Use Unfold's contrib package for better UI -- Admin dashboard - Use Unfold's dashboard widgets -- Advanced filtering - Use Unfold's filter contrib package - ---- - -## Unfold Features Available - -### Built-in Features We'll Use: - -1. **Visual Interface** - - Modern Tailwind CSS-based design - - Dark mode support (automatic) - - Responsive layout (mobile-friendly) - - Beautiful forms and tables - -2. **Advanced Filtering** - - `unfold.contrib.filters` - Enhanced filter UI - - Date range filters with calendar - - Autocomplete filters for foreign keys - - Numeric range filters - -3. **Import/Export** - - `unfold.contrib.import_export` - Styled import/export UI - - Works seamlessly with django-import-export - - Beautiful file upload/download interface - -4. **History/Audit Trail** - - `unfold.contrib.simple_history` - Enhanced history UI - - Timeline view of changes - - User attribution and timestamps - -5. **Actions** - - Enhanced bulk actions UI - - Custom action styling - - Progress indicators - -6. **Dashboard Components** - - Cards for metrics - - Charts (Chart.js integration) - - Custom widgets - - Activity feeds - -7. **Other Features** - - Inline tabs for grouping - - Conditional field visibility - - WYSIWYG editor (Trix) - - Sortable inlines - - Command palette (Cmd+K search) - ---- - -## Phase 1: Critical Fixes & Model Updates (Week 1) - -### 1.1 Clean Up Sidebar Menu Organization - -**Problem:** site_building models referenced but don't exist, causing confusion. Some empty groups appearing. - -**Action:** - -1. **Remove non-existent site_building models** from `backend/igny8_core/admin/site.py` custom_groups: - - BusinessType - - AudienceProfile - - BrandPersonality - - HeroImageryDirection - -2. **Add missing PlanLimitUsage model** to Billing group: - ```python - 'Billing & Accounts': { - 'models': [ - ('igny8_core_auth', 'Plan'), - ('billing', 'PlanLimitUsage'), # ADD THIS - ('igny8_core_auth', 'Account'), - # ... rest of models - ], - }, - ``` - -3. **Verify all referenced models exist** - check that each model in custom_groups is actually registered - -**Files to modify:** -- `/data/app/igny8/backend/igny8_core/admin/site.py` -- Verify PlanLimitUsage is registered in admin - -### 1.2 Resolve Duplicate Model Registrations - -**Problem:** Multiple models registered twice causing conflicts - -**Action:** -- Keep only `modules/billing/admin.py` versions (they have full workflow) -- Remove or comment out duplicates in `business/billing/admin.py`: - - Invoice - - Payment - - CreditPackage - - CreditCostConfig - -**Files to modify:** -- `/data/app/igny8/backend/igny8_core/business/billing/admin.py` - -### 1.3 Update Admin Classes to Use Unfold - -**Current State:** Most admin classes inherit from `admin.ModelAdmin` - -**Action:** Update all admin classes to inherit from Unfold's ModelAdmin: - -```python -from unfold.admin import ModelAdmin -from unfold.contrib.filters.admin import RangeDateFilter -from unfold.contrib.import_export.forms import ExportForm, ImportForm - -class TasksAdmin(SiteSectorAdminMixin, ModelAdmin): - # Unfold-specific features - compressed_fields = True # Compact form layout - warn_unsaved_form = True # Warn before leaving unsaved form - - # Standard Django admin - list_display = ['title', 'status', 'cluster', 'created_at'] - list_filter = [ - ('created_at', RangeDateFilter), # Unfold date range filter - 'status', - ] - search_fields = ['title', 'description'] - actions_detail = ['generate_content', 'assign_cluster'] # Actions in detail view -``` - -**Key Files to Update:** -- `/data/app/igny8/backend/igny8_core/modules/writer/admin.py` -- `/data/app/igny8/backend/igny8_core/modules/planner/admin.py` -- `/data/app/igny8/backend/igny8_core/modules/billing/admin.py` -- `/data/app/igny8/backend/igny8_core/business/automation/admin.py` -- `/data/app/igny8/backend/igny8_core/business/integration/admin.py` -- All other admin files - -### 1.4 Test Admin Configuration - -**Action:** After cleanup, verify: -- All sidebar groups display correctly -- No 404 errors when clicking model links -- All registered models appear in appropriate groups -- No empty groups in sidebar - ---- - -## Phase 2: Bulk Operations & Export (Week 2) - -### 2.1 Implement CSV/Excel Export with Unfold - -**Install Already Complete:** django-import-export and unfold.contrib.import_export - -**Action:** Add ExportMixin and ImportMixin to admin classes: - -```python -from import_export.admin import ImportExportModelAdmin -from unfold.contrib.import_export.forms import ExportForm, ImportForm, SelectableFieldsExportForm - -class TasksAdmin(SiteSectorAdminMixin, ImportExportModelAdmin, ModelAdmin): - import_form_class = ImportForm - export_form_class = ExportForm - - # Define what fields to export - class TaskResource(resources.ModelResource): - class Meta: - model = Tasks - fields = ('id', 'title', 'status', 'cluster__name', 'site__name', 'created_at') - export_order = fields - - resource_class = TaskResource -``` - -**Priority Models for Export:** -- ✅ Tasks -- ✅ Content -- ✅ Keywords -- ✅ Payments -- ✅ CreditTransactions -- ✅ Clusters - -### 2.2 Add Bulk Operations with Unfold Actions - -**Unfold provides enhanced action UI automatically** - -**Action:** Add actions to admin classes: - -```python -from django.contrib import admin -from unfold.decorators import action - -class TasksAdmin(SiteSectorAdminMixin, ModelAdmin): - actions = ['bulk_set_in_progress', 'bulk_set_completed', 'bulk_assign_cluster'] - - @action(description="Mark as In Progress") - def bulk_set_in_progress(self, request, queryset): - updated = queryset.update(status='in_progress') - self.message_user(request, f'{updated} tasks marked as in progress', 'SUCCESS') - - @action(description="Mark as Completed") - def bulk_set_completed(self, request, queryset): - updated = queryset.update(status='completed') - self.message_user(request, f'{updated} tasks completed', 'SUCCESS') - - @action(description="Assign to Cluster", form_class=BulkAssignClusterForm) - def bulk_assign_cluster(self, request, queryset): - # Unfold will show form with cluster selection - if 'cluster' in request.POST: - cluster_id = request.POST['cluster'] - queryset.update(cluster_id=cluster_id) - self.message_user(request, f'Assigned to cluster', 'SUCCESS') -``` - -**Bulk Actions to Add:** - -#### Tasks Admin -- Mark as Draft -- Mark as In Progress -- Mark as Completed -- Assign to Cluster -- Generate Content (trigger AI) - -#### Content Admin -- Publish to WordPress -- Change Status -- Add Taxonomy -- Update SEO Settings - -#### Keywords Admin -- Assign to Cluster -- Set Priority -- Mark for Research - -#### Payments Admin -- Approve Payments (already exists) -- Reject Payments (already exists) -- Export Transactions (add) - -### 2.3 Advanced Filtering with Unfold - -**Unfold provides beautiful filter UI out of the box** - -**Action:** Use Unfold's filter contrib for enhanced filtering: - -```python -from unfold.contrib.filters.admin import ( - RangeDateFilter, - RangeDateTimeFilter, - RangeNumericFilter, - SingleNumericFilter, - SliderNumericFilter, -) -from unfold.contrib.filters.admin import RelatedDropdownFilter, ChoicesDropdownFilter - -class CreditTransactionAdmin(AccountAdminMixin, ModelAdmin): - list_filter = [ - ('created_at', RangeDateFilter), # Beautiful date range picker - ('amount', RangeNumericFilter), # Numeric range with slider - ('account', RelatedDropdownFilter), # Dropdown with search - ('transaction_type', ChoicesDropdownFilter), # Enhanced dropdown - ] -``` - -**Filter Types to Implement:** - -| Model | Filters | -|-------|---------| -| CreditTransaction | Date range, Amount range, Account dropdown, Type | -| Payment | Date range, Status, Amount range, Method | -| Content | Date range, Status, Site, Sector, Word count range | -| Tasks | Date range, Status, Cluster, Priority | -| AutomationRun | Date range, Status, Site, Success/Fail | - ---- - -## Phase 3: Monitoring & Dashboards (Week 5-6) - -### 3.1 Celery Task Monitoring - -**Install django-celery-results:** - -```bash -pip install django-celery-results -``` - -**Add to settings.py:** - -```python -INSTALLED_APPS = [ - # ... - 'django_celery_results', -] - -CELERY_RESULT_BACKEND = 'django-db' -CELERY_CACHE_BACKEND = 'django-cache' -``` - -**Create Celery Task Admin:** - -File: `backend/igny8_core/admin/celery_admin.py` - -```python -from django.contrib import admin -from django_celery_results.models import TaskResult - -@admin.register(TaskResult) -class CeleryTaskResultAdmin(admin.ModelAdmin): - list_display = [ - 'task_id', - 'task_name', - 'status', - 'date_created', - 'date_done', - 'colored_status', - ] - list_filter = [ - 'status', - 'task_name', - ('date_created', DateRangeFilter), - ] - search_fields = ['task_id', 'task_name', 'task_args'] - readonly_fields = ['task_id', 'task_name', 'task_args', 'task_kwargs', 'result', 'traceback'] - - actions = ['retry_failed_tasks', 'clear_old_tasks'] - - def colored_status(self, obj): - colors = { - 'SUCCESS': 'green', - 'FAILURE': 'red', - 'PENDING': 'orange', - 'STARTED': 'blue', - } - color = colors.get(obj.status, 'gray') - return format_html( - '{}', - color, - obj.status - ) - colored_status.short_description = 'Status' - - def retry_failed_tasks(self, request, queryset): - """Retry failed celery tasks""" - from celery import current_app - count = 0 - for task in queryset.filter(status='FAILURE'): - try: - # Get task function and retry - task_func = current_app.tasks.get(task.task_name) - if task_func: - task_func.apply_async() - count += 1 - except Exception as e: - self.message_user(request, f'Error retrying task {task.task_id}: {str(e)}', level='ERROR') - - self.message_user(request, f'Retried {count} failed task(s)', level='SUCCESS') - retry_failed_tasks.short_description = 'Retry failed tasks' -``` - -**Add to admin groups:** - -```python -'🤖 AI & Automation': { - 'models': [ - # ... - ('django_celery_results', 'TaskResult'), - ], -}, -``` - -### 3.2 Admin Dashboard with Metrics - -**Create custom admin index view:** - -File: `backend/igny8_core/admin/dashboard.py` - -```python -from django.contrib.admin.views.decorators import staff_member_required -from django.shortcuts import render -from django.db.models import Count, Sum, Q -from django.utils import timezone -from datetime import timedelta - -@staff_member_required -def admin_dashboard(request): - """Custom admin dashboard with key metrics""" - - # Date ranges - today = timezone.now().date() - week_ago = today - timedelta(days=7) - month_ago = today - timedelta(days=30) - - # Account metrics - from igny8_core.auth.models import Account - total_accounts = Account.objects.count() - active_accounts = Account.objects.filter(status='active').count() - low_credit_accounts = Account.objects.filter(credits__lt=100).count() - - # Content metrics - from igny8_core.modules.writer.models import Content, Tasks - content_this_week = Content.objects.filter(created_at__gte=week_ago).count() - content_this_month = Content.objects.filter(created_at__gte=month_ago).count() - tasks_pending = Tasks.objects.filter(status='pending').count() - - # Billing metrics - from igny8_core.modules.billing.models import Payment, CreditTransaction - pending_payments = Payment.objects.filter(status='pending_approval').count() - payments_this_month = Payment.objects.filter( - created_at__gte=month_ago, - status='succeeded' - ).aggregate(total=Sum('amount'))['total'] or 0 - - credit_usage_this_month = CreditTransaction.objects.filter( - created_at__gte=month_ago, - transaction_type='deduction' - ).aggregate(total=Sum('amount'))['total'] or 0 - - # Automation metrics - from igny8_core.business.automation.models import AutomationRun - automation_running = AutomationRun.objects.filter(status='running').count() - automation_failed = AutomationRun.objects.filter( - status='failed', - created_at__gte=week_ago - ).count() - - # WordPress sync metrics - from igny8_core.business.integration.models import SyncEvent - sync_failed = SyncEvent.objects.filter( - success=False, - created_at__gte=today - ).count() - - context = { - 'title': 'Dashboard', - 'accounts': { - 'total': total_accounts, - 'active': active_accounts, - 'low_credit': low_credit_accounts, - }, - 'content': { - 'this_week': content_this_week, - 'this_month': content_this_month, - 'tasks_pending': tasks_pending, - }, - 'billing': { - 'pending_payments': pending_payments, - 'payments_this_month': payments_this_month, - 'credit_usage_this_month': abs(credit_usage_this_month), - }, - 'automation': { - 'running': automation_running, - 'failed_this_week': automation_failed, - }, - 'integration': { - 'sync_failed_today': sync_failed, - }, - } - - return render(request, 'admin/dashboard.html', context) -``` - -**Create dashboard template:** - -File: `backend/igny8_core/templates/admin/dashboard.html` - -```html -{% extends "admin/base_site.html" %} -{% load static %} - -{% block content %} -

IGNY8 Admin Dashboard

- - -
- -
-

Accounts

-
-
- {{ accounts.total }} - Total Accounts -
-
- {{ accounts.active }} - Active -
-
- {{ accounts.low_credit }} - Low Credits (< 100) -
-
-
- - -
-

Content Production

-
-
- {{ content.this_week }} - This Week -
-
- {{ content.this_month }} - This Month -
-
- {{ content.tasks_pending }} - Tasks Pending -
-
-
- - -
-

Billing

-
-
- {{ billing.pending_payments }} - Pending Approvals - - Review - -
-
- ${{ billing.payments_this_month|floatformat:2 }} - Revenue This Month -
-
- {{ billing.credit_usage_this_month }} - Credits Used -
-
-
- - -
-

Automation

-
-
- {{ automation.running }} - Running Now -
-
- {{ automation.failed_this_week }} - Failed (7 days) - {% if automation.failed_this_week > 0 %} - - Review - - {% endif %} -
-
-
- - -
-

Integration Health

-
-
- {{ integration.sync_failed_today }} - Failed Syncs Today - {% if integration.sync_failed_today > 0 %} - - Review - - {% endif %} -
-
-
-
-{% endblock %} -``` - -**Note:** Unfold automatically provides styling via Tailwind CSS. No custom CSS needed - use Unfold's utility classes if customization is required. - -**Update admin site URLs:** - -```python -# backend/igny8_core/admin/site.py -from .dashboard import admin_dashboard - -class Igny8AdminSite(admin.AdminSite): - # ... existing code - - def get_urls(self): - from django.urls import path - urls = super().get_urls() - custom_urls = [ - path('dashboard/', self.admin_view(admin_dashboard), name='dashboard'), - ] - return custom_urls + urls - - def index(self, request, extra_context=None): - """Redirect to custom dashboard""" - from django.shortcuts import redirect - return redirect('admin:dashboard') -``` - -### 3.3 Account Health Indicators - -**Add to Account Admin:** - -```python -@admin.register(Account) -class AccountAdmin(AccountAdminMixin, admin.ModelAdmin): - # ... existing code - - list_display = [ - 'name', - 'slug', - 'owner', - 'plan', - 'status', - 'credits', - 'health_indicator', # ADD - 'created_at' - ] - - def health_indicator(self, obj): - """Visual health score""" - score = 100 - issues = [] - - # Check credit balance - if obj.credits < 50: - score -= 30 - issues.append('Low credits') - elif obj.credits < 100: - score -= 15 - - # Check recent activity - from django.utils import timezone - from datetime import timedelta - week_ago = timezone.now() - timedelta(days=7) - - from igny8_core.modules.writer.models import Content - recent_content = Content.objects.filter( - account=obj, - created_at__gte=week_ago - ).count() - - if recent_content == 0: - score -= 20 - issues.append('No recent activity') - - # Check failed automations - from igny8_core.business.automation.models import AutomationRun - failed_runs = AutomationRun.objects.filter( - site__account=obj, - status='failed', - created_at__gte=week_ago - ).count() - - if failed_runs > 0: - score -= 15 - issues.append(f'{failed_runs} failed automation(s)') - - # Determine color - if score >= 80: - color = 'green' - icon = '✓' - elif score >= 60: - color = 'orange' - icon = 'âš ī¸' - else: - color = 'red' - icon = '✗' - - issue_text = ', '.join(issues) if issues else 'Healthy' - - # Use Unfold badge classes instead of inline styles - if score >= 80: - badge_class = 'success' - elif score >= 60: - badge_class = 'warning' - else: - badge_class = 'danger' - - return format_html( - '{}', - badge_class, issue_text - ) - health_indicator.short_description = 'Health' -``` - -### 3.4 Alert System - -**Create alerts utility:** - -File: `backend/igny8_core/admin/alerts.py` - -```python -from django.core.cache import cache -from django.utils import timezone -from datetime import timedelta - -class AdminAlerts: - """System for admin alerts and notifications""" - - @staticmethod - def get_alerts(): - """Get all active alerts""" - alerts = [] - - # Check for pending payments - from igny8_core.modules.billing.models import Payment - pending_payments = Payment.objects.filter(status='pending_approval').count() - if pending_payments > 0: - alerts.append({ - 'level': 'warning', - 'message': f'{pending_payments} payment(s) awaiting approval', - 'url': '/admin/billing/payment/?status=pending_approval', - 'action': 'Review Payments' - }) - - # Check for low credit accounts - from igny8_core.auth.models import Account - low_credit_accounts = Account.objects.filter( - status='active', - credits__lt=100 - ).count() - if low_credit_accounts > 0: - alerts.append({ - 'level': 'info', - 'message': f'{low_credit_accounts} account(s) with low credits', - 'url': '/admin/igny8_core_auth/account/?credits__lt=100', - 'action': 'View Accounts' - }) - - # Check for failed automations - from igny8_core.business.automation.models import AutomationRun - today = timezone.now().date() - failed_today = AutomationRun.objects.filter( - status='failed', - created_at__date=today - ).count() - if failed_today > 0: - alerts.append({ - 'level': 'error', - 'message': f'{failed_today} automation(s) failed today', - 'url': '/admin/automation/automationrun/?status=failed', - 'action': 'Review Failures' - }) - - # Check for failed syncs - from igny8_core.business.integration.models import SyncEvent - failed_syncs = SyncEvent.objects.filter( - success=False, - created_at__date=today - ).count() - if failed_syncs > 5: # Only alert if more than 5 - alerts.append({ - 'level': 'warning', - 'message': f'{failed_syncs} WordPress sync failures today', - 'url': '/admin/integration/syncevent/?success=False', - 'action': 'Review Syncs' - }) - - return alerts -``` - -**Add to dashboard template:** - -```html - -{% if alerts %} -
- {% for alert in alerts %} -
- {{ alert.message }} - {{ alert.action }} -
- {% endfor %} -
-{% endif %} -``` - -**Note:** Use Unfold's alert styling classes. No custom CSS or emoji icons needed. - ---- - -## Phase 4: Analytics & Reporting (Week 7-8) - -### 4.1 Business Intelligence Reports - -**Create reports module:** - -File: `backend/igny8_core/admin/reports.py` - -```python -from django.contrib.admin.views.decorators import staff_member_required -from django.shortcuts import render -from django.db.models import Count, Sum, Avg, Q -from django.utils import timezone -from datetime import timedelta -import json - -@staff_member_required -def revenue_report(request): - """Revenue and billing analytics""" - from igny8_core.modules.billing.models import Payment - from igny8_core.auth.models import Plan - - # Date ranges - today = timezone.now() - months = [] - monthly_revenue = [] - - for i in range(6): - month_start = today.replace(day=1) - timedelta(days=30*i) - month_end = month_start.replace(day=28) + timedelta(days=4) - - revenue = Payment.objects.filter( - status='succeeded', - processed_at__gte=month_start, - processed_at__lt=month_end - ).aggregate(total=Sum('amount'))['total'] or 0 - - months.insert(0, month_start.strftime('%b %Y')) - monthly_revenue.insert(0, float(revenue)) - - # Plan distribution - plan_distribution = Plan.objects.annotate( - account_count=Count('account') - ).values('name', 'account_count') - - # Payment method breakdown - payment_methods = Payment.objects.filter( - status='succeeded' - ).values('payment_method').annotate( - count=Count('id'), - total=Sum('amount') - ).order_by('-total') - - context = { - 'title': 'Revenue Report', - 'months': json.dumps(months), - 'monthly_revenue': json.dumps(monthly_revenue), - 'plan_distribution': plan_distribution, - 'payment_methods': payment_methods, - } - - return render(request, 'admin/reports/revenue.html', context) - -@staff_member_required -def usage_report(request): - """Credit usage and AI operations analytics""" - from igny8_core.modules.billing.models import CreditUsageLog - - # Usage by operation type - usage_by_operation = CreditUsageLog.objects.values( - 'operation_type' - ).annotate( - total_credits=Sum('credits_used'), - total_cost=Sum('cost_usd'), - operation_count=Count('id') - ).order_by('-total_credits') - - # Top credit consumers - from django.db.models import Count - top_consumers = CreditUsageLog.objects.values( - 'account__name' - ).annotate( - total_credits=Sum('credits_used'), - operation_count=Count('id') - ).order_by('-total_credits')[:10] - - # Model usage distribution - model_usage = CreditUsageLog.objects.values( - 'model_used' - ).annotate( - usage_count=Count('id') - ).order_by('-usage_count') - - context = { - 'title': 'Usage Report', - 'usage_by_operation': usage_by_operation, - 'top_consumers': top_consumers, - 'model_usage': model_usage, - } - - return render(request, 'admin/reports/usage.html', context) - -@staff_member_required -def content_report(request): - """Content production analytics""" - from igny8_core.modules.writer.models import Content, Tasks - - # Content by type - content_by_type = Content.objects.values( - 'content_type' - ).annotate(count=Count('id')).order_by('-count') - - # Production timeline (last 30 days) - days = [] - daily_counts = [] - for i in range(30): - day = timezone.now().date() - timedelta(days=i) - count = Content.objects.filter(created_at__date=day).count() - days.insert(0, day.strftime('%m/%d')) - daily_counts.insert(0, count) - - # Average word count by content type - avg_words = Content.objects.values('content_type').annotate( - avg_words=Avg('word_count') - ).order_by('-avg_words') - - # Task completion rate - total_tasks = Tasks.objects.count() - completed_tasks = Tasks.objects.filter(status='completed').count() - completion_rate = (completed_tasks / total_tasks * 100) if total_tasks > 0 else 0 - - context = { - 'title': 'Content Production Report', - 'content_by_type': content_by_type, - 'days': json.dumps(days), - 'daily_counts': json.dumps(daily_counts), - 'avg_words': avg_words, - 'completion_rate': completion_rate, - } - - return render(request, 'admin/reports/content.html', context) -``` - -**Add report links to admin site:** - -```python -# In Igny8AdminSite.get_urls() -custom_urls = [ - path('dashboard/', self.admin_view(admin_dashboard), name='dashboard'), - path('reports/revenue/', self.admin_view(revenue_report), name='report_revenue'), - path('reports/usage/', self.admin_view(usage_report), name='report_usage'), - path('reports/content/', self.admin_view(content_report), name='report_content'), -] -``` - -### 4.2 Data Quality Dashboard - -**Create data quality checker:** - -```python -@staff_member_required -def data_quality_report(request): - """Check data quality and integrity""" - issues = [] - - # Orphaned content (no site) - from igny8_core.modules.writer.models import Content - orphaned_content = Content.objects.filter(site__isnull=True).count() - if orphaned_content > 0: - issues.append({ - 'severity': 'warning', - 'type': 'Orphaned Records', - 'count': orphaned_content, - 'description': 'Content items without assigned site', - 'action_url': '/admin/writer/content/?site__isnull=True' - }) - - # Tasks without clusters - from igny8_core.modules.writer.models import Tasks - tasks_no_cluster = Tasks.objects.filter(cluster__isnull=True).count() - if tasks_no_cluster > 0: - issues.append({ - 'severity': 'info', - 'type': 'Missing Relationships', - 'count': tasks_no_cluster, - 'description': 'Tasks without assigned cluster', - 'action_url': '/admin/writer/tasks/?cluster__isnull=True' - }) - - # Accounts with negative credits - from igny8_core.auth.models import Account - negative_credits = Account.objects.filter(credits__lt=0).count() - if negative_credits > 0: - issues.append({ - 'severity': 'error', - 'type': 'Data Integrity', - 'count': negative_credits, - 'description': 'Accounts with negative credit balance', - 'action_url': '/admin/igny8_core_auth/account/?credits__lt=0' - }) - - # Duplicate keywords - from igny8_core.modules.planner.models import Keywords - from django.db.models import Count - duplicates = Keywords.objects.values('keyword', 'site', 'sector').annotate( - count=Count('id') - ).filter(count__gt=1).count() - if duplicates > 0: - issues.append({ - 'severity': 'warning', - 'type': 'Duplicates', - 'count': duplicates, - 'description': 'Duplicate keywords for same site/sector', - 'action_url': '/admin/planner/keywords/' - }) - - # Content without SEO data - no_seo = Content.objects.filter( - Q(meta_title__isnull=True) | Q(meta_title='') | - Q(meta_description__isnull=True) | Q(meta_description='') - ).count() - if no_seo > 0: - issues.append({ - 'severity': 'info', - 'type': 'Incomplete Data', - 'count': no_seo, - 'description': 'Content missing SEO metadata', - 'action_url': '/admin/writer/content/' - }) - - context = { - 'title': 'Data Quality Report', - 'issues': issues, - 'total_issues': len(issues), - } - - return render(request, 'admin/reports/data_quality.html', context) -``` - ---- - -## Phase 5: Advanced Features (Week 9-10) - -### 5.1 Inline Editing - -**Enable list_editable for common fields:** - -```python -# Tasks Admin -class TasksAdmin(SiteSectorAdminMixin, admin.ModelAdmin): - list_editable = ['status', 'cluster'] # Quick edit in list view - -# Keywords Admin -class KeywordsAdmin(SiteSectorAdminMixin, admin.ModelAdmin): - list_editable = ['cluster', 'status'] - -# Payment Admin - already has workflow, keep as-is -``` - -### 5.2 Enhanced Audit Trail - -**Install django-simple-history:** - -```bash -pip install django-simple-history -``` - -**Add to critical models:** - -```python -from simple_history.models import HistoricalRecords - -class Payment(models.Model): - # ... existing fields - history = HistoricalRecords() - -class Account(AccountBaseModel): - # ... existing fields - history = HistoricalRecords() - -class CreditCostConfig(models.Model): - # ... existing fields - history = HistoricalRecords() -``` - -**Register history admin:** - -```python -from simple_history.admin import SimpleHistoryAdmin - -@admin.register(Payment) -class PaymentAdmin(SimpleHistoryAdmin, AccountAdminMixin, admin.ModelAdmin): - # ... existing code - # This adds a "History" button to view all changes -``` - -### 5.3 Permission Groups - -**Create permission groups:** - -```python -# backend/igny8_core/management/commands/create_admin_groups.py -from django.core.management.base import BaseCommand -from django.contrib.auth.models import Group, Permission -from django.contrib.contenttypes.models import ContentType - -class Command(BaseCommand): - help = 'Create admin permission groups' - - def handle(self, *args, **kwargs): - # Content Manager Group - content_manager, _ = Group.objects.get_or_create(name='Content Manager') - content_perms = Permission.objects.filter( - content_type__app_label__in=['writer', 'planner'], - codename__in=['view_content', 'change_content', 'view_tasks', 'change_tasks'] - ) - content_manager.permissions.set(content_perms) - - # Billing Administrator Group - billing_admin, _ = Group.objects.get_or_create(name='Billing Administrator') - billing_perms = Permission.objects.filter( - content_type__app_label='billing' - ) - billing_admin.permissions.set(billing_perms) - - # Support Agent Group (Read-Only) - support_agent, _ = Group.objects.get_or_create(name='Support Agent') - support_perms = Permission.objects.filter( - codename__startswith='view_' - ) - support_agent.permissions.set(support_perms) - - self.stdout.write(self.style.SUCCESS('Successfully created admin groups')) -``` - ---- - -## Implementation Checklist - -### ✅ Phase 0: Foundation (COMPLETED - Dec 14, 2025) - -- [x] Install and configure Unfold theme -- [x] Fix all admin classes to inherit from Unfold ModelAdmin -- [x] Fix UserAdmin popup styling (multiple inheritance) -- [x] Configure UNFOLD settings in settings.py -- [x] Update middleware and INSTALLED_APPS -- [x] Test all containers running - -### Phase 1: Configuration Cleanup (Week 1) - IN PROGRESS - -- [x] Fix UserAdmin popup styling -- [ ] Remove unused site_building models from admin site config -- [ ] Remove duplicate model registrations (keep modules/ versions) -- [ ] Add PlanLimitUsage to Billing & Accounts group -- [ ] Verify all model links work -- [ ] Test admin configuration - -### Phase 2: Bulk Operations & Export (Week 2-3) - NOT STARTED - -- [ ] Add bulk status update actions to Tasks admin -- [ ] Add bulk operations to Content admin -- [ ] Add bulk operations to Keywords admin -- [ ] Add export action to Payments admin -- [ ] Install django-import-export -- [ ] Create Resource classes for Tasks, Content, Keywords, Payments -- [ ] Add export buttons to admin list views -- [ ] Install django-admin-rangefilter -- [ ] Add date range filters to key models -- [ ] Configure autocomplete_fields for large foreign keys -- [ ] Test all bulk operations -- [ ] Test export functionality - -### ✅ Phase 3: Monitoring & Dashboards (COMPLETED - Dec 14, 2025) - -- [x] Install django-celery-results -- [x] Configure Celery to use django-db backend -- [x] Create CeleryTaskResultAdmin with colored status -- [x] Create CeleryGroupResultAdmin with colored status -- [x] Add retry_failed_tasks action -- [x] Add clear_old_tasks action -- [x] Create admin_dashboard view function -- [x] Create dashboard.html template with metrics -- [x] Add dashboard route to admin site URLs -- [x] Redirect admin index to dashboard -- [x] Add health_indicator to Account admin -- [x] Add health_details to Account admin -- [x] Create AdminAlerts utility class -- [x] Add alerts section to dashboard template -- [x] Fix execution_time format_html issue -- [x] Test dashboard metrics accuracy -- [x] Test alert system functionality -- [x] Verify all Celery admin pages work (200 status) - -### ✅ Phase 4: Analytics & Reporting (COMPLETED - Dec 15, 2025) - -- [x] Create reports.py module -- [x] Implement revenue_report view -- [x] Implement usage_report view -- [x] Implement content_report view -- [x] Implement data_quality_report view -- [x] Create report templates (revenue, usage, content, data_quality) -- [x] Add chart.js for visualizations -- [x] Add report routes to admin site URLs -- [x] Add report links to admin sidebar navigation -- [x] Create report permission checks -- [ ] Test all reports with real data (operational task) -- [ ] Optimize report queries for performance (operational task) - -### ✅ Phase 5: Advanced Features (COMPLETED - Dec 15, 2025) - -- [x] Enable list_editable for Tasks and Keywords -- [x] django-simple-history already installed -- [x] Add HistoricalRecords to Payment model -- [x] Add HistoricalRecords to Account model -- [x] Add HistoricalRecords to CreditCostConfig model -- [x] Run migrations for history tables -- [x] Update admins to use SimpleHistoryAdmin -- [x] Create create_admin_groups management command -- [x] Define permission groups (Content Manager, Billing Admin, Support Agent) -- [x] Assign permissions to groups -- [ ] Test permission restrictions (operational task) -- [ ] Document permission group usage (operational task) - ---- - -## Technical Requirements - -### Python Packages - -```txt -# Already installed: -django-unfold==0.73.1 # Modern admin theme (✅ INSTALLED) -django-import-export==3.3.1 # CSV/Excel import/export (✅ INSTALLED) -django-celery-results==2.5.1 # Celery monitoring (✅ INSTALLED) -django-simple-history==3.4.0 # Audit trail (✅ INSTALLED) - -# No additional packages needed for styling - Unfold handles everything -``` - -### Settings Configuration - -```python -# backend/igny8_core/settings.py -# ✅ Already configured properly - -INSTALLED_APPS = [ - 'unfold', # ✅ Already installed - Must be before django.contrib.admin - 'unfold.contrib.filters', # ✅ Already installed - 'unfold.contrib.import_export', # ✅ Already installed - 'unfold.contrib.simple_history', # ✅ Already installed - 'igny8_core.admin.apps.Igny8AdminConfig', - # ... rest of apps - 'import_export', # ✅ Already installed - 'django_celery_results', # ✅ Already installed - 'simple_history', # ✅ Already installed -] - -# Celery Results - ✅ Already configured -CELERY_RESULT_BACKEND = 'django-db' -CELERY_CACHE_BACKEND = 'django-cache' - -# History Middleware - ✅ Already configured -MIDDLEWARE = [ - # ... existing middleware - 'simple_history.middleware.HistoryRequestMiddleware', -] - -# Import/Export Settings -IMPORT_EXPORT_USE_TRANSACTIONS = True - -# UNFOLD configuration - ✅ Already configured in settings.py -``` - -### Database Migrations - -```bash -# Already completed - no additional migrations needed for Unfold -# Unfold uses Django's existing admin tables - -# For future features: -# python manage.py migrate django_celery_results # When adding Celery monitoring -# python manage.py migrate simple_history # When adding history to models -``` - -### Template Structure - -``` -backend/igny8_core/templates/ -├── admin/ -│ ├── base_site.html -│ ├── dashboard.html -│ └── reports/ -│ ├── revenue.html -│ ├── usage.html -│ ├── content.html -│ └── data_quality.html -``` - ---- - -## Testing Plan - -### Manual Testing Checklist - -- [ ] Admin login works -- [ ] Dashboard displays correctly -- [ ] All sidebar groups show correct models -- [ ] No 404 errors for any admin links -- [ ] Bulk actions work on all enabled models -- [ ] CSV export generates valid files -- [ ] Date range filters work correctly -- [ ] Autocomplete fields load data -- [ ] Celery task monitoring shows tasks -- [ ] Dashboard metrics are accurate -- [ ] Alerts show for real issues -- [ ] Reports generate without errors -- [ ] Charts render correctly -- [ ] History tracking works for enabled models -- [ ] Permission groups restrict access correctly - -### Performance Testing - -- [ ] Dashboard loads in < 2 seconds -- [ ] List views with 1000+ records paginate correctly -- [ ] Bulk operations on 100+ records complete -- [ ] Export of 1000+ records completes -- [ ] Reports with large datasets load within 5 seconds -- [ ] No N+1 query issues in list views - ---- - -## Rollback Plan - -If issues arise during implementation: - -1. **Unfold Issues (unlikely):** Already stable and working - - Unfold is well-tested and production-ready - - No custom styling to break - - All standard Django admin features work - -2. **Duplicate Registration Issues:** - - Keep `modules/` admin files active - - Comment out `business/` admin registrations - -3. **Performance Issues:** - - Disable expensive dashboard queries - - Use caching for metrics - - Reduce pagination size - -4. **Database Issues:** - - All migrations are reversible - - Backup before major changes - - Test on staging first - ---- - -## Success Metrics - -Track these metrics to measure improvement success: - -1. **Time Savings** - - Payment approval time: Target < 2 minutes - - Content review time: Target < 5 minutes per item - - Report generation time: Target < 30 seconds - -2. **User Satisfaction** - - Admin user feedback: Target 8+/10 - - Feature usage: 80%+ of new features used weekly - -3. **Data Quality** - - Orphaned records: Reduce by 90% - - Duplicate data: Reduce by 80% - - Data completeness: Increase to 95%+ - -4. **Operational Efficiency** - - Failed automation detection: < 1 hour - - Payment processing: Same-day completion - - Account health monitoring: 100% visibility - ---- - -## Maintenance & Support - -### Regular Maintenance Tasks - -- **Weekly:** - - Review dashboard alerts - - Check failed celery tasks - - Monitor data quality issues - -- **Monthly:** - - Review permission group assignments - - Update credit cost configs if needed - - Clean up old celery task results - - Archive old audit logs - -- **Quarterly:** - - Review and optimize slow admin queries - - Update admin theme if needed - - Review and update permission groups - - Analyze usage patterns for improvements - -### Documentation Updates - -Keep these docs updated: -- Admin user guide with new features -- Permission group definitions -- Report interpretation guides -- Troubleshooting common issues - ---- - -## Future Enhancements (Post-Launch) - -1. **AI-Powered Insights** - - Anomaly detection in usage patterns - - Predictive account churn analysis - - Automated optimization recommendations - -2. **Advanced Automation** - - Workflow builder for common tasks - - Scheduled report delivery - - Auto-responses to common issues - -3. **Mobile Admin** - - Responsive mobile design - - Mobile-optimized dashboards - - Push notifications for critical alerts - -4. **Integration Enhancements** - - Slack/Discord alert integration - - Zapier/Make.com workflow triggers - - API for external admin tools - ---- - -## Conclusion - -This comprehensive plan transforms the Django admin from a basic management tool into a powerful operational hub. The phased approach ensures steady progress while minimizing disruption. Focus on Phases 1-3 for immediate operational impact, then expand to analytics and advanced features in Phases 4-5. - -**Estimated Total Effort:** 2-4 weeks with 1-2 developers (reduced since styling is already complete via Unfold) - -**Priority:** 🔴 High - Critical for efficient operations at scale - -**Dependencies:** Should be implemented after Plan Management (#2) is complete - - - diff --git a/docs/.structure-plan.txt b/docs/.structure-plan.txt deleted file mode 100644 index d2e9c37a..00000000 --- a/docs/.structure-plan.txt +++ /dev/null @@ -1,119 +0,0 @@ -FINAL DOCUMENTATION STRUCTURE -============================== - -docs/ -├── README.md (Master navigation - find anything in 1 step) -├── CHANGELOG.md (All changes across system) -│ -├── 00-SYSTEM/ -│ ├── ARCHITECTURE-OVERVIEW.md (High-level system design) -│ ├── TECH-STACK.md (All technologies used) -│ ├── MULTITENANCY.md (Account isolation, tenant context) -│ ├── AUTHENTICATION.md (JWT, sessions, permissions) -│ └── DATA-FLOWS.md (Visual workflows across system) -│ -├── 10-BACKEND/ -│ ├── OVERVIEW.md (Backend architecture) -│ ├── MODELS.md (All database models with fields) -│ ├── SERVICES.md (Business logic services) -│ │ -│ ├── accounts/ -│ │ └── ACCOUNTS-REFERENCE.md (User, Account, Role models + endpoints) -│ │ -│ ├── billing/ -│ │ ├── BILLING-REFERENCE.md (Plans, Subscriptions, Invoices) -│ │ ├── CREDITS-SYSTEM.md (Credit allocation/deduction) -│ │ └── PAYMENT-METHODS.md (Payment processing) -│ │ -│ ├── planner/ -│ │ ├── PLANNER-REFERENCE.md (Keywords → Clusters → Ideas) -│ │ ├── KEYWORD-CLUSTERING.md (Clustering algorithm) -│ │ └── IDEA-GENERATION.md (AI-powered idea generation) -│ │ -│ ├── writer/ -│ │ ├── WRITER-REFERENCE.md (Content generation) -│ │ ├── CONTENT-GENERATION.md (AI content creation flow) -│ │ ├── IMAGES-SYSTEM.md (Image generation & management) -│ │ └── PUBLISHING.md (Review & publish workflow) -│ │ -│ ├── automation/ -│ │ ├── AUTOMATION-REFERENCE.md (Full automation pipeline) -│ │ ├── PIPELINE-STAGES.md (Stage-by-stage breakdown) -│ │ └── SCHEDULER.md (Celery tasks & scheduling) -│ │ -│ ├── integrations/ -│ │ ├── WORDPRESS-INTEGRATION.md (WP sync & publishing) -│ │ ├── AI-SERVICES.md (OpenAI, Anthropic integration) -│ │ └── IMAGE-GENERATION.md (DALL-E, Stability AI) -│ │ -│ └── sites/ -│ └── SITES-REFERENCE.md (Site & Sector management) -│ -├── 20-API/ -│ ├── API-REFERENCE.md (All REST endpoints) -│ ├── AUTHENTICATION-ENDPOINTS.md (Login, register, tokens) -│ ├── PLANNER-ENDPOINTS.md (Keywords, clusters, ideas) -│ ├── WRITER-ENDPOINTS.md (Content, tasks, images) -│ ├── AUTOMATION-ENDPOINTS.md (Automation runs) -│ ├── BILLING-ENDPOINTS.md (Invoices, payments, credits) -│ └── INTEGRATION-ENDPOINTS.md (WordPress, external services) -│ -├── 30-FRONTEND/ -│ ├── FRONTEND-ARCHITECTURE.md (React structure, routing) -│ ├── STATE-MANAGEMENT.md (Zustand stores) -│ ├── COMPONENTS.md (Reusable UI components) -│ │ -│ ├── planner/ -│ │ └── PLANNER-UI.md (Keywords, clusters, ideas pages) -│ │ -│ ├── writer/ -│ │ └── WRITER-UI.md (Content, tasks, publishing pages) -│ │ -│ ├── automation/ -│ │ └── AUTOMATION-UI.md (Automation dashboard) -│ │ -│ └── billing/ -│ └── BILLING-UI.md (Plans, payments, invoices) -│ -├── 40-WORKFLOWS/ -│ ├── SIGNUP-TO-ACTIVE.md (User journey from signup to active) -│ ├── CONTENT-LIFECYCLE.md (Keyword → Idea → Task → Content → Published) -│ ├── PAYMENT-WORKFLOW.md (Manual payment approval flow) -│ ├── AUTOMATION-WORKFLOW.md (Full automation run lifecycle) -│ └── WORDPRESS-SYNC.md (Bidirectional sync workflow) -│ -├── 50-DEPLOYMENT/ -│ ├── ENVIRONMENT-SETUP.md (Local, staging, production) -│ ├── DOCKER-DEPLOYMENT.md (Container setup) -│ └── DATABASE-MIGRATIONS.md (Migration strategy) -│ -└── 90-ARCHIVED/ - └── (Old deprecated docs for reference) - -NAVIGATION RULES -================ - -1. Master README.md has "Quick Find" table: - - Want to add feature? → Find module → Find file - - Want to troubleshoot? → Find workflow → Find exact function - - Want API details? → Find endpoint → See request/response/location - -2. Every doc file has: - - Purpose statement - - File locations (exact paths) - - Function/Class names (no code) - - Related files (cross-references) - - Data flow (if applicable) - -3. No code snippets, only: - - File paths: backend/igny8_core/business/billing/services/credit_service.py - - Function names: CreditService.add_credits() - - Model fields: account.credits, invoice.total - - Endpoints: POST /v1/billing/admin/payments/confirm/ - -4. Visual elements allowed: - - ASCII flow diagrams - - State transition tables - - Field mapping tables - - Workflow sequences - diff --git a/docs/PLAN-LIMITS.md b/docs/PLAN-LIMITS.md deleted file mode 100644 index 0d3e3a66..00000000 --- a/docs/PLAN-LIMITS.md +++ /dev/null @@ -1,677 +0,0 @@ -# Plan Limits System - -## Overview - -The Plan Limits System enforces subscription-based usage restrictions in IGNY8. It tracks both **hard limits** (persistent throughout subscription) and **monthly limits** (reset on billing cycle). - -**File:** `/docs/PLAN-LIMITS.md` -**Version:** 1.0.0 -**Last Updated:** December 12, 2025 - ---- - -## Architecture - -### Limit Types - -#### Hard Limits (Never Reset) -These limits persist for the lifetime of the subscription and represent total capacity: - -| Limit Type | Field Name | Description | Example Value | -|------------|------------|-------------|---------------| -| Sites | `max_sites` | Maximum number of sites per account | Starter: 2, Growth: 5, Scale: Unlimited | -| Team Users | `max_users` | Maximum team members | Starter: 1, Growth: 3, Scale: 10 | -| Keywords | `max_keywords` | Total keywords allowed | Starter: 500, Growth: 1000, Scale: Unlimited | -| Clusters | `max_clusters` | Total clusters allowed | Starter: 50, Growth: 100, Scale: Unlimited | - -#### Monthly Limits (Reset on Billing Cycle) -These limits reset automatically at the start of each billing period: - -| Limit Type | Field Name | Description | Example Value | -|------------|------------|-------------|---------------| -| Content Ideas | `max_content_ideas` | New ideas generated per month | Starter: 100, Growth: 300, Scale: 600 | -| Content Words | `max_content_words` | Total words generated per month | Starter: 100K, Growth: 300K, Scale: 500K | -| Basic Images | `max_images_basic` | Basic AI images per month | Starter: 100, Growth: 300, Scale: 500 | -| Premium Images | `max_images_premium` | Premium AI images (DALL-E) per month | Starter: 20, Growth: 60, Scale: 100 | -| Image Prompts | `max_image_prompts` | AI-generated prompts per month | Starter: 100, Growth: 300, Scale: 500 | - ---- - -## Database Schema - -### Plan Model Extensions -**Location:** `backend/igny8_core/auth/models.py` - -```python -class Plan(models.Model): - # ... existing fields ... - - # Hard Limits - max_sites = IntegerField(default=2, validators=[MinValueValidator(1)]) - max_users = IntegerField(default=1, validators=[MinValueValidator(1)]) - max_keywords = IntegerField(default=500, validators=[MinValueValidator(1)]) - max_clusters = IntegerField(default=50, validators=[MinValueValidator(1)]) - - # Monthly Limits - max_content_ideas = IntegerField(default=100, validators=[MinValueValidator(1)]) - max_content_words = IntegerField(default=100000, validators=[MinValueValidator(1)]) - max_images_basic = IntegerField(default=100, validators=[MinValueValidator(1)]) - max_images_premium = IntegerField(default=20, validators=[MinValueValidator(1)]) - max_image_prompts = IntegerField(default=100, validators=[MinValueValidator(1)]) -``` - -### PlanLimitUsage Model -**Location:** `backend/igny8_core/business/billing/models.py` - -Tracks monthly consumption for each limit type: - -```python -class PlanLimitUsage(AccountBaseModel): - LIMIT_TYPE_CHOICES = [ - ('content_ideas', 'Content Ideas'), - ('content_words', 'Content Words'), - ('images_basic', 'Basic Images'), - ('images_premium', 'Premium Images'), - ('image_prompts', 'Image Prompts'), - ] - - limit_type = CharField(max_length=50, choices=LIMIT_TYPE_CHOICES, db_index=True) - amount_used = IntegerField(default=0, validators=[MinValueValidator(0)]) - period_start = DateField() - period_end = DateField() - metadata = JSONField(default=dict) # Stores breakdown by site, content_id, etc. - - class Meta: - unique_together = [['account', 'limit_type', 'period_start']] - indexes = [ - Index(fields=['account', 'period_start']), - Index(fields=['period_end']), - ] -``` - -**Migration:** `backend/igny8_core/modules/billing/migrations/0015_planlimitusage.py` - ---- - -## Service Layer - -### LimitService -**Location:** `backend/igny8_core/business/billing/services/limit_service.py` - -Central service for all limit operations. - -#### Key Methods - -##### 1. Check Hard Limit -```python -LimitService.check_hard_limit(account, limit_type, additional_count=1) -``` - -**Purpose:** Validate if adding items would exceed hard limit -**Raises:** `HardLimitExceededError` if limit exceeded -**Example:** -```python -try: - LimitService.check_hard_limit(account, 'sites', additional_count=1) - # Proceed with site creation -except HardLimitExceededError as e: - raise PermissionDenied(str(e)) -``` - -##### 2. Check Monthly Limit -```python -LimitService.check_monthly_limit(account, limit_type, amount) -``` - -**Purpose:** Validate if operation would exceed monthly allowance -**Raises:** `MonthlyLimitExceededError` if limit exceeded -**Example:** -```python -try: - LimitService.check_monthly_limit(account, 'content_words', amount=2500) - # Proceed with content generation -except MonthlyLimitExceededError as e: - raise InsufficientCreditsError(str(e)) -``` - -##### 3. Increment Usage -```python -LimitService.increment_usage(account, limit_type, amount, metadata=None) -``` - -**Purpose:** Record usage after successful operation -**Returns:** New total usage -**Example:** -```python -LimitService.increment_usage( - account=account, - limit_type='content_words', - amount=2500, - metadata={ - 'content_id': 123, - 'content_title': 'My Article', - 'site_id': 456 - } -) -``` - -##### 4. Get Usage Summary -```python -LimitService.get_usage_summary(account) -``` - -**Purpose:** Comprehensive usage report for all limits -**Returns:** Dictionary with hard_limits, monthly_limits, period info -**Example Response:** -```json -{ - "account_id": 1, - "account_name": "Acme Corp", - "plan_name": "Growth Plan", - "period_start": "2025-12-01", - "period_end": "2025-12-31", - "days_until_reset": 19, - "hard_limits": { - "sites": { - "display_name": "Sites", - "current": 3, - "limit": 5, - "remaining": 2, - "percentage_used": 60 - }, - "keywords": { - "display_name": "Keywords", - "current": 750, - "limit": 1000, - "remaining": 250, - "percentage_used": 75 - } - }, - "monthly_limits": { - "content_words": { - "display_name": "Content Words", - "current": 245000, - "limit": 300000, - "remaining": 55000, - "percentage_used": 82 - }, - "images_basic": { - "display_name": "Basic Images", - "current": 120, - "limit": 300, - "remaining": 180, - "percentage_used": 40 - } - } -} -``` - -##### 5. Reset Monthly Limits -```python -LimitService.reset_monthly_limits(account) -``` - -**Purpose:** Reset all monthly usage at period end (called by Celery task) -**Returns:** Dictionary with reset summary -**Note:** Called automatically by scheduled task, not manually - ---- - -## Enforcement Points - -### 1. Site Creation -**File:** `backend/igny8_core/auth/views.py` (SiteViewSet.perform_create) - -```python -LimitService.check_hard_limit(account, 'sites', additional_count=1) -``` - -### 2. Content Generation -**File:** `backend/igny8_core/business/content/services/content_generation_service.py` - -```python -# Check limit before generation -LimitService.check_monthly_limit(account, 'content_words', amount=total_word_count) - -# Increment usage after successful generation -LimitService.increment_usage(account, 'content_words', amount=actual_word_count) -``` - -### 3. Content Save Hook -**File:** `backend/igny8_core/business/content/models.py` (Content.save) - -Automatically increments `content_words` usage when content_html is saved: - -```python -def save(self, *args, **kwargs): - # Auto-calculate word count - if self.content_html: - calculated_count = calculate_word_count(self.content_html) - self.word_count = calculated_count - - super().save(*args, **kwargs) - - # Increment usage for newly generated words - if new_words > 0: - LimitService.increment_usage(account, 'content_words', amount=new_words) -``` - -### 4. Additional Enforcement Points (To Be Implemented) - -Following the same pattern, add checks to: -- **Keyword Import:** Check `max_keywords` before bulk import -- **Clustering:** Check `max_clusters` before creating new clusters -- **Idea Generation:** Check `max_content_ideas` before generating ideas -- **Image Generation:** Check `max_images_basic`/`max_images_premium` before AI call - ---- - -## Word Counting Utility - -**Location:** `backend/igny8_core/utils/word_counter.py` - -Provides accurate word counting from HTML content. - -### Functions - -#### calculate_word_count(html_content) -```python -from igny8_core.utils.word_counter import calculate_word_count - -word_count = calculate_word_count('

Hello world!

') -# Returns: 2 -``` - -**Method:** -1. Strips HTML tags using BeautifulSoup -2. Fallback to regex if BeautifulSoup fails -3. Counts words (sequences of alphanumeric characters) - -#### format_word_count(count) -```python -formatted = format_word_count(1500) # "1.5K" -formatted = format_word_count(125000) # "125K" -``` - -#### validate_word_count_limit(html_content, limit) -```python -result = validate_word_count_limit(html, limit=100000) -# Returns: { -# 'allowed': True, -# 'word_count': 2500, -# 'limit': 100000, -# 'remaining': 97500, -# 'would_exceed_by': 0 -# } -``` - ---- - -## Scheduled Tasks - -**Location:** `backend/igny8_core/tasks/plan_limits.py` - -### 1. Reset Monthly Plan Limits -**Task Name:** `reset_monthly_plan_limits` -**Schedule:** Daily at 00:30 UTC -**Purpose:** Reset monthly usage for accounts at period end - -**Process:** -1. Find all active accounts with subscriptions -2. Check if `current_period_end` <= today -3. Call `LimitService.reset_monthly_limits(account)` -4. Update subscription period dates -5. Log reset summary - -### 2. Check Approaching Limits -**Task Name:** `check_approaching_limits` -**Schedule:** Daily at 09:00 UTC -**Purpose:** Warn users when usage exceeds 80% threshold - -**Process:** -1. Find all active accounts -2. Get usage summary -3. Check if any limit >= 80% -4. Log warnings (future: send email notifications) - -**Celery Beat Configuration:** -`backend/igny8_core/celery.py` - -```python -app.conf.beat_schedule = { - 'reset-monthly-plan-limits': { - 'task': 'reset_monthly_plan_limits', - 'schedule': crontab(hour=0, minute=30), - }, - 'check-approaching-limits': { - 'task': 'check_approaching_limits', - 'schedule': crontab(hour=9, minute=0), - }, -} -``` - ---- - -## API Endpoints - -### Get Usage Summary -**Endpoint:** `GET /api/v1/billing/usage-summary/` -**Authentication:** Required (IsAuthenticatedAndActive) -**Response:** Usage summary for current account - -**Example Request:** -```bash -curl -H "Authorization: Bearer " \ - /api/v1/billing/usage-summary/ -``` - -**Example Response:** -```json -{ - "success": true, - "message": "Usage summary retrieved successfully.", - "data": { - "account_id": 1, - "account_name": "Acme Corp", - "plan_name": "Growth Plan", - "period_start": "2025-12-01", - "period_end": "2025-12-31", - "days_until_reset": 19, - "hard_limits": { ... }, - "monthly_limits": { ... } - } -} -``` - ---- - -## Error Handling - -### HardLimitExceededError -```python -raise HardLimitExceededError( - f"Sites limit exceeded. Current: 5, Limit: 5. " - f"Upgrade your plan to increase this limit." -) -``` - -**HTTP Status:** 403 Forbidden -**User Action:** Upgrade plan or delete unused resources - -### MonthlyLimitExceededError -```python -raise MonthlyLimitExceededError( - f"Content Words limit exceeded. Used: 295000, Requested: 8000, Limit: 300000. " - f"Resets on December 31, 2025. Upgrade your plan or wait for reset." -) -``` - -**HTTP Status:** 403 Forbidden -**User Action:** Wait for reset, upgrade plan, or reduce request size - ---- - -## Frontend Integration Guide - -### TypeScript Types -```typescript -interface Plan { - id: number; - name: string; - // Hard limits - max_sites: number; - max_users: number; - max_keywords: number; - max_clusters: number; - // Monthly limits - max_content_ideas: number; - max_content_words: number; - max_images_basic: number; - max_images_premium: number; - max_image_prompts: number; -} - -interface UsageSummary { - account_id: number; - account_name: string; - plan_name: string; - period_start: string; - period_end: string; - days_until_reset: number; - hard_limits: { - [key: string]: { - display_name: string; - current: number; - limit: number; - remaining: number; - percentage_used: number; - }; - }; - monthly_limits: { - [key: string]: { - display_name: string; - current: number; - limit: number; - remaining: number; - percentage_used: number; - }; - }; -} -``` - -### API Hook Example -```typescript -// src/services/api/billing.ts -export const getUsageSummary = async (): Promise => { - const response = await apiClient.get('/billing/usage-summary/'); - return response.data.data; -}; - -// src/pages/Dashboard.tsx -const { data: usage } = useQuery('usage-summary', getUsageSummary); -``` - -### UI Components - -#### Usage Widget -```tsx - - -

Usage This Month

- {usage.days_until_reset} days until reset -
- - {Object.entries(usage.monthly_limits).map(([key, data]) => ( -
-
{data.display_name}
- = 80 ? 'warning' : 'primary'} - /> - {data.current.toLocaleString()} / {data.limit.toLocaleString()} -
- ))} -
-
-``` - -#### Limit Warning Alert -```tsx -{usage.monthly_limits.content_words.percentage_used >= 80 && ( - - âš ī¸ You've used {usage.monthly_limits.content_words.percentage_used}% of your - monthly word limit. Resets in {usage.days_until_reset} days. - Upgrade Plan - -)} -``` - ---- - -## Testing - -### Manual Testing Checklist - -1. **Hard Limit - Sites:** - - Set plan `max_sites = 2` - - Create 2 sites successfully - - Attempt to create 3rd site → should fail with error - -2. **Monthly Limit - Words:** - - Set plan `max_content_words = 5000` - - Generate content with 3000 words - - Generate content with 2500 words → should fail - - Check usage API shows 3000/5000 - -3. **Usage Increment:** - - Generate content - - Verify `PlanLimitUsage.amount_used` increments correctly - - Check metadata contains content_id - -4. **Monthly Reset:** - - Manually run: `docker exec igny8_backend python manage.py shell` - - Execute: - ```python - from igny8_core.tasks.plan_limits import reset_monthly_plan_limits - reset_monthly_plan_limits() - ``` - - Verify usage resets to 0 - - Verify new period records created - -5. **Usage Summary API:** - - Call GET `/api/v1/billing/usage-summary/` - - Verify all limits present - - Verify percentages calculated correctly - -### Unit Test Example -```python -# tests/test_limit_service.py -def test_check_hard_limit_exceeded(): - account = create_test_account(plan_max_sites=2) - create_test_sites(account, count=2) - - with pytest.raises(HardLimitExceededError): - LimitService.check_hard_limit(account, 'sites', additional_count=1) - -def test_increment_monthly_usage(): - account = create_test_account() - - LimitService.increment_usage(account, 'content_words', amount=1000) - - usage = PlanLimitUsage.objects.get(account=account, limit_type='content_words') - assert usage.amount_used == 1000 -``` - ---- - -## Monitoring & Logs - -### Key Log Messages - -**Successful limit check:** -``` -INFO Hard limit check: sites - Current: 2, Requested: 1, Limit: 5 -INFO Monthly limit check: content_words - Current: 50000, Requested: 2500, Limit: 100000 -``` - -**Limit exceeded:** -``` -WARNING Hard limit exceeded: sites - Current: 5, Requested: 1, Limit: 5 -WARNING Monthly limit exceeded: content_words - Used: 98000, Requested: 5000, Limit: 100000 -``` - -**Usage increment:** -``` -INFO Incremented content_words usage by 2500. New total: 52500 -``` - -**Monthly reset:** -``` -INFO Resetting limits for account 123 (Acme Corp) - period ended 2025-12-31 -INFO Reset complete for account 123: New period 2026-01-01 to 2026-01-31 -INFO Monthly plan limits reset task complete: 45 accounts reset, 0 errors -``` - ---- - -## Troubleshooting - -### Issue: Limits not enforcing -**Check:** -1. Verify Plan has non-zero limit values: `Plan.objects.get(id=X)` -2. Check if service calling LimitService methods -3. Review logs for exceptions being caught - -### Issue: Usage not incrementing -**Check:** -1. Verify Content.save() executing successfully -2. Check for exceptions in logs during increment_usage -3. Query `PlanLimitUsage` table directly - -### Issue: Reset task not running -**Check:** -1. Celery Beat is running: `docker exec igny8_backend celery -A igny8_core inspect active` -2. Check Celery Beat schedule: `docker exec igny8_backend celery -A igny8_core inspect scheduled` -3. Review Celery logs: `docker logs igny8_celery_beat` - ---- - -## Future Enhancements - -1. **Email Notifications:** - - Send warning emails at 80%, 90%, 100% thresholds - - Weekly usage summary reports - - Monthly reset confirmations - -2. **Additional Enforcement:** - - Keyword bulk import limit check - - Cluster creation limit check - - Idea generation limit check - - Image generation limit checks - -3. **Usage Analytics:** - - Historical usage trends - - Projection of limit exhaustion date - - Recommendations for plan upgrades - -4. **Soft Limits:** - - Allow slight overages with warnings - - Grace period before hard enforcement - -5. **Admin Tools:** - - Override limits for specific accounts - - One-time usage bonuses - - Custom limit adjustments - ---- - -## Related Files - -**Models:** -- `backend/igny8_core/auth/models.py` - Plan model -- `backend/igny8_core/business/billing/models.py` - PlanLimitUsage model - -**Services:** -- `backend/igny8_core/business/billing/services/limit_service.py` - LimitService -- `backend/igny8_core/utils/word_counter.py` - Word counting utility - -**Views:** -- `backend/igny8_core/auth/views.py` - Site creation enforcement -- `backend/igny8_core/business/billing/views.py` - Usage summary API -- `backend/igny8_core/business/content/services/content_generation_service.py` - Content generation enforcement - -**Tasks:** -- `backend/igny8_core/tasks/plan_limits.py` - Reset and warning tasks -- `backend/igny8_core/celery.py` - Celery Beat schedule - -**Migrations:** -- `backend/igny8_core/auth/migrations/0013_plan_max_clusters_plan_max_content_ideas_and_more.py` -- `backend/igny8_core/modules/billing/migrations/0015_planlimitusage.py` - -**Documentation:** -- `CHANGELOG.md` - Version history with plan limits feature -- `.cursorrules` - Development standards and versioning rules - ---- - -**End of Document** diff --git a/django-updates/ADMIN-IMPLEMENTATION-STATUS.md b/docs/PRE-LAUNCH/ADMIN-IMPLEMENTATION-STATUS.md similarity index 100% rename from django-updates/ADMIN-IMPLEMENTATION-STATUS.md rename to docs/PRE-LAUNCH/ADMIN-IMPLEMENTATION-STATUS.md diff --git a/docs/PRE-LAUNCH/ITEM-2-CREDITS-BILLING-PRICING.md b/docs/PRE-LAUNCH/ITEM-2-CREDITS-BILLING-PRICING.md deleted file mode 100644 index 03b2ba44..00000000 --- a/docs/PRE-LAUNCH/ITEM-2-CREDITS-BILLING-PRICING.md +++ /dev/null @@ -1,628 +0,0 @@ -# Item 2: Credits, Billing, Pricing Logic, and Usage Limits - -**Priority:** Critical -**Target:** Production Launch -**Last Updated:** December 11, 2025 - ---- - -## Overview - -Define and implement a comprehensive credit cost system, plan-based usage limits, and billing logic for all AI operations. This includes setting credit costs per function, establishing plan tiers with limits, and implementing enforcement mechanisms across backend and frontend. - ---- - -## Current Implementation Analysis - -### Credit System Architecture - -**Location:** `backend/igny8_core/business/billing/` - -#### Credit Models - -| Model | Purpose | Key Fields | -|-------|---------|------------| -| **CreditTransaction** | Tracks all credit additions/deductions | `transaction_type`, `amount`, `balance_after`, `description` | -| **CreditUsageLog** | Detailed log per AI operation | `operation_type`, `credits_used`, `cost_usd`, `model_used`, `tokens_input`, `tokens_output` | -| **CreditCostConfig** | Admin-configurable credit costs | `operation_type`, `credits_cost`, `unit`, `display_name` | - -**Credit Transaction Types:** -- `purchase` - Credit purchase -- `subscription` - Monthly subscription renewal -- `refund` - Credit refund -- `deduction` - Usage deduction (AI operations) -- `adjustment` - Manual admin adjustment - -#### Credit Service - -**Location:** `backend/igny8_core/business/billing/services/credit_service.py` - -**Methods:** -- `get_credit_cost(operation_type, amount)` - Calculate cost for operation -- `check_credits(account, operation_type, amount)` - Validate sufficient credits -- `deduct_credits(account, amount, operation_type, ...)` - Deduct and log -- `deduct_credits_for_operation(...)` - Convenience method with auto-calculation - -**Logic:** -1. Checks database `CreditCostConfig` first -2. Falls back to hardcoded `CREDIT_COSTS` constants -3. Applies unit-based calculation (per 100 words, per image, etc.) -4. Validates sufficient balance before deduction -5. Creates both `CreditTransaction` and `CreditUsageLog` records - ---- - -### Current Credit Costs - -**Location:** `backend/igny8_core/business/billing/constants.py` - -| Operation | Current Cost | Unit | Notes | -|-----------|--------------|------|-------| -| `clustering` | 10 credits | per request | Clusters all submitted keywords | -| `idea_generation` | 15 credits | per request | Ideas for one cluster | -| `content_generation` | 1 credit | per 100 words | Word-count based | -| `image_prompt_extraction` | 2 credits | per content | Extract prompts from content | -| `image_generation` | 5 credits | per image | Generate single image | -| `linking` | 8 credits | per content | Internal linking (NEW) | -| `optimization` | 1 credit | per 200 words | Content optimization (NEW) | -| `site_structure_generation` | 50 credits | per site | Site blueprint (Phase 7) | -| `site_page_generation` | 20 credits | per page | Page generation (Phase 7) | - -**Legacy Aliases:** -- `ideas` → `idea_generation` -- `content` → 3 credits fixed (legacy) -- `images` → `image_generation` -- `reparse` → 1 credit - -**Issues with Current Costs:** -1. **Not optimized for profitability** - Costs may not reflect actual AI provider costs -2. **Arbitrary values** - No clear formula based on model costs, processing time, or value -3. **Inconsistent granularity** - Some per-request, some per-word, some per-item -4. **No differentiation by quality** - Same cost regardless of model quality (GPT-4 vs GPT-3.5) - ---- - -### Plan Model and Limits - -**Location:** `backend/igny8_core/auth/models.py` - `Plan` model - -#### Current Plan Structure - -| Field | Purpose | Current State | -|-------|---------|---------------| -| `name`, `slug` | Plan identification | ✅ Implemented | -| `price`, `billing_cycle` | Pricing | ✅ Monthly/Annual support | -| `included_credits` | Monthly credit allocation | ✅ Implemented | -| `extra_credit_price` | Per-credit overage cost | ✅ Default $0.01 | -| `allow_credit_topup` | Can buy more credits | ✅ Boolean flag | -| `auto_credit_topup_threshold` | Auto-buy trigger | ✅ Optional | -| `auto_credit_topup_amount` | Auto-buy amount | ✅ Optional | -| `max_users` | Users per account | ✅ Implemented | -| `max_sites` | Sites per account | ✅ Implemented | -| `max_industries` | Industries/sectors limit | ✅ Optional | -| `max_author_profiles` | Writing styles limit | ✅ Default 5 | - -**What's MISSING:** -- ❌ Max keywords limit -- ❌ Max clusters limit -- ❌ Max ideas limit -- ❌ Max content pieces limit -- ❌ Max images limit -- ❌ Max tasks in queue limit -- ❌ Daily/monthly usage caps (beyond credits) -- ❌ Per-user vs per-account limits distinction - ---- - -### Account Credit Balance - -**Location:** `backend/igny8_core/auth/models.py` - `Account` model - -**Field:** `credits` (IntegerField with MinValueValidator(0)) - -**Current Behavior:** -- Credits deducted on AI operation completion -- Credit balance checked before operation starts -- `InsufficientCreditsError` raised if balance < required - -**No Implementation For:** -- Credit expiration dates -- Credit rollover rules (monthly vs annual) -- Negative balance prevention (hard stop vs warning) -- Credit reserve for pending operations - ---- - -## Pricing Plan Requirements - -### Recommended Plan Tiers - -Based on industry standards and target market: - -| Plan | Monthly Price | Annual Price | Included Credits | Target User | -|------|---------------|--------------|------------------|-------------| -| **Free** | $0 | $0 | 50 | Trial users, hobbyists | -| **Starter** | $29 | $299 (15% off) | 500 | Solo creators, small blogs | -| **Growth** | $99 | $1,019 (15% off) | 2,000 | Growing sites, agencies | -| **Pro** | $299 | $3,077 (15% off) | 7,500 | Power users, large agencies | -| **Enterprise** | Custom | Custom | Custom | Enterprise clients | - -**Free Plan Considerations:** -- Should be marked `is_internal=True` to hide from public pricing -- Limits should be strict enough to encourage upgrade -- Should not include advanced features (automation, API access) - ---- - -### Usage Limits Per Plan - -**Proposed Limits** (to be finalized): - -| Limit Type | Free | Starter | Growth | Pro | Enterprise | -|------------|------|---------|--------|-----|------------| -| **Monthly Credits** | 50 | 500 | 2,000 | 7,500 | Custom | -| **Max Users** | 1 | 2 | 5 | 15 | Unlimited | -| **Max Sites** | 1 | 3 | 10 | 50 | Unlimited | -| **Max Keywords (saved)** | 100 | 1,000 | 5,000 | 25,000 | Unlimited | -| **Max Clusters** | 20 | 100 | 500 | 2,500 | Unlimited | -| **Max Ideas (saved)** | 50 | 500 | 2,500 | 12,500 | Unlimited | -| **Max Content Pieces** | 25 | 250 | 1,250 | 6,250 | Unlimited | -| **Max Images** | 25 | 250 | 1,250 | 6,250 | Unlimited | -| **Max Queue Size** | 5 | 20 | 50 | 200 | Unlimited | -| **Automation Enabled** | ❌ | ❌ | ✅ | ✅ | ✅ | -| **API Access** | ❌ | ❌ | ✅ | ✅ | ✅ | -| **Priority Support** | ❌ | ❌ | ❌ | ✅ | ✅ | - -**Notes:** -- "Unlimited" means no hard limit, but still subject to fair use policy -- Limits apply per account (across all sites in account) -- Deleted items don't count toward limits (soft-delete system) - ---- - -### Credit Cost Optimization Strategy - -**Goal:** Define credit costs that: -1. Cover AI provider costs + margin -2. Are competitive with market rates -3. Encourage usage without abuse -4. Scale predictably with usage - -#### Recommended Credit Cost Revisions - -**Analysis Required:** -- [ ] Calculate actual AI provider costs per operation (OpenAI, Runware, etc.) -- [ ] Add 30-50% margin for infrastructure, support, and profit -- [ ] Compare with competitor pricing (Jasper, Copy.ai, Writesonic) -- [ ] Test with sample use cases to ensure plan value - -**Proposed Adjustments** (pending analysis): - -| Operation | Current | Proposed | Reasoning | -|-----------|---------|----------|-----------| -| Clustering | 10 | **8** | Lower barrier for discovery phase | -| Idea Generation | 15 | **12** | Encourage ideation before writing | -| Content (100 words) | 1 | **1.5** | Reflect actual GPT-4 costs | -| Image Prompts | 2 | **3** | More complex extraction logic | -| Image Generation | 5 | **6** | Runware/DALL-E costs increasing | -| Optimization | 1 per 200 words | **0.5 per 100 words** | Encourage optimization usage | - -**Variable Costs by Model Quality:** -- **Option:** Charge more for GPT-4 vs GPT-3.5, DALL-E 3 vs DALL-E 2 -- **Implementation:** Add `model_tier` multiplier in `get_credit_cost()` - ---- - -## Required Implementation - -### A. Expand Plan Model with Usage Limits - -**File:** `backend/igny8_core/auth/models.py` - `Plan` model - -**Add Fields:** - -```python -# Content Creation Limits (NULL = unlimited) -max_keywords = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum keywords saved per account" -) -max_clusters = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum clusters per account" -) -max_ideas = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum content ideas saved per account" -) -max_content = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum content pieces per account" -) -max_images = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum images per account" -) - -# Queue and Rate Limits -max_queue_size = models.IntegerField( - default=10, - validators=[MinValueValidator(1)], - help_text="Maximum concurrent items in queue" -) -max_daily_ai_requests = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum AI requests per day (prevents abuse)" -) -max_monthly_content_generated = models.IntegerField( - null=True, blank=True, - validators=[MinValueValidator(1)], - help_text="Maximum content pieces generated per month" -) - -# Feature Access Flags -allow_automation = models.BooleanField( - default=False, - help_text="Enable automation wizard" -) -allow_api_access = models.BooleanField( - default=False, - help_text="Enable API access" -) -allow_bulk_operations = models.BooleanField( - default=True, - help_text="Enable bulk actions (delete, export, etc.)" -) -``` - -**Migration:** Create Django migration to add these fields with default NULL values - ---- - -### B. Create Limit Enforcement Service - -**File:** `backend/igny8_core/business/billing/services/limit_service.py` (NEW) - -**Service Class:** `LimitService` - -**Methods to Implement:** - -| Method | Purpose | Returns | -|--------|---------|---------| -| `check_keyword_limit(account)` | Check if can add more keywords | `bool` or raises `LimitExceededError` | -| `check_cluster_limit(account)` | Check if can add more clusters | `bool` or raises `LimitExceededError` | -| `check_idea_limit(account)` | Check if can add more ideas | `bool` or raises `LimitExceededError` | -| `check_content_limit(account)` | Check if can add more content | `bool` or raises `LimitExceededError` | -| `check_image_limit(account)` | Check if can add more images | `bool` or raises `LimitExceededError` | -| `check_queue_limit(account)` | Check queue capacity | `bool` or raises `LimitExceededError` | -| `check_daily_request_limit(account)` | Check daily AI request quota | `bool` or raises `LimitExceededError` | -| `get_usage_stats(account)` | Get current usage counts | `dict` with all counters | -| `get_limit_stats(account)` | Get limits and remaining capacity | `dict` with limits | - -**Implementation Logic:** - -```python -def check_keyword_limit(account): - plan = account.plan - if plan.max_keywords is None: - return True # Unlimited - - current_count = Keywords.objects.filter( - account=account, - deleted_at__isnull=True # Exclude soft-deleted - ).count() - - if current_count >= plan.max_keywords: - raise LimitExceededError( - f"Keyword limit reached ({plan.max_keywords}). Upgrade your plan." - ) - - return True -``` - -**Exception:** `LimitExceededError` (inherit from `BillingException`) - ---- - -### C. Integrate Limit Checks in API Views - -**Files to Update:** - -- `backend/igny8_core/modules/planner/views.py` - KeywordsViewSet -- `backend/igny8_core/modules/planner/views.py` - ClustersViewSet -- `backend/igny8_core/modules/planner/views.py` - ContentIdeasViewSet -- `backend/igny8_core/modules/writer/views.py` - TasksViewSet -- `backend/igny8_core/modules/writer/views.py` - ContentViewSet - -**Integration Points:** - -| ViewSet | Action | Check to Add | -|---------|--------|--------------| -| KeywordsViewSet | `create()` | `LimitService.check_keyword_limit(account)` | -| KeywordsViewSet | `bulk_create()` | Check limit with proposed count | -| ClustersViewSet | `create()` | `LimitService.check_cluster_limit(account)` | -| ContentIdeasViewSet | `create()` | `LimitService.check_idea_limit(account)` | -| TasksViewSet | `create()` | `LimitService.check_content_limit(account)` + `check_queue_limit()` | -| ContentViewSet | `create()` | `LimitService.check_content_limit(account)` | - -**Example Integration:** - -```python -def create(self, request, *args, **kwargs): - account = request.user.account - - # Check limit before creating - try: - LimitService.check_keyword_limit(account) - except LimitExceededError as e: - return Response({ - 'error': str(e), - 'error_code': 'LIMIT_EXCEEDED', - 'upgrade_url': '/pricing' - }, status=403) - - # Proceed with creation - return super().create(request, *args, **kwargs) -``` - ---- - -### D. Add Usage Tracking and Counter Cache - -**Optimization:** Instead of counting records on every request, cache counts - -**Implementation Options:** - -#### Option 1: Add Counter Fields to Account Model - -```python -# Add to Account model -keyword_count = models.IntegerField(default=0) -cluster_count = models.IntegerField(default=0) -idea_count = models.IntegerField(default=0) -content_count = models.IntegerField(default=0) -image_count = models.IntegerField(default=0) -``` - -**Update counters in signals:** -- `post_save` signal: increment counter -- `post_delete` signal: decrement counter -- Periodic reconciliation task to fix drift - -#### Option 2: Cache Usage Stats (Recommended) - -Use Django cache with 5-minute TTL: - -```python -def get_cached_usage_stats(account): - cache_key = f'usage_stats_{account.id}' - stats = cache.get(cache_key) - - if stats is None: - stats = { - 'keywords': Keywords.objects.filter(account=account, deleted_at__isnull=True).count(), - 'clusters': Clusters.objects.filter(account=account, deleted_at__isnull=True).count(), - # ... etc - } - cache.set(cache_key, stats, 300) # 5 minutes - - return stats -``` - -**Invalidate cache on:** -- Create operations -- Delete operations -- Soft-delete operations - ---- - -### E. Frontend Limit Display - -#### 1. Usage Dashboard Widget - -**Location:** `frontend/src/components/dashboard/UsageLimitsWidget.tsx` (NEW) - -**Display:** -- Current usage vs limit for each resource -- Progress bars with color coding: - - Green: < 70% used - - Yellow: 70-90% used - - Red: > 90% used -- "Upgrade Plan" button when approaching limits - -**Example UI:** - -``` -Usage & Limits -━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Keywords: 750 / 1,000 ████████░░ 75% -Clusters: 45 / 100 ████░░░░░░ 45% -Ideas: 380 / 500 ███████░░░ 76% -Content: 120 / 250 ████░░░░░░ 48% - -[Upgrade Plan] -``` - -#### 2. Inline Warnings - -**Show warnings when approaching limits:** - -- At 80%: Yellow badge "Approaching limit" -- At 90%: Orange warning "Near limit - Upgrade recommended" -- At 100%: Red error "Limit reached - Upgrade required" - -**Display in:** -- Header metrics -- Page headers -- Before bulk operations -- In forms (disable submit if limit reached) - -#### 3. Create/Import Dialogs - -**Add limit check before showing form:** - -```typescript -const handleCreateKeyword = () => { - const stats = usageStats; // from API - const limit = account.plan.max_keywords; - - if (limit && stats.keywords >= limit) { - toast.error('Keyword limit reached. Upgrade your plan.'); - navigate('/settings/billing'); - return; - } - - setShowCreateModal(true); -}; -``` - -#### 4. Upgrade Prompts - -**When limit error occurs:** -- Show modal with: - - Current plan - - Current limit - - Recommended plan - - Benefits of upgrading - - "Upgrade Now" CTA - ---- - -### F. Credit Cost Configuration UI (Admin) - -**Location:** Django Admin or custom Admin Panel page - -**Feature:** Allow superusers to edit credit costs without code changes - -**Admin Interface:** -- List all operations with current costs -- Edit cost, unit, and display name -- Track change history (previous_cost field) -- Enable/disable operations -- Preview impact on sample use cases - -**Models Used:** -- `CreditCostConfig` - Admin-editable costs -- Falls back to `CREDIT_COSTS` constants if not configured - ---- - -## Testing Requirements - -### Limit Enforcement Tests - -| Test Case | Expected Result | -|-----------|-----------------| -| Create keyword at limit | Error: "Keyword limit reached" | -| Create keyword below limit | Success | -| Create 10 keywords via bulk import at limit | Error with count blocked | -| Delete keyword then create | Success (count decremented) | -| Soft-delete keyword then restore | Counts update correctly | -| Upgrade plan mid-session | New limits apply immediately | - -### Credit Deduction Tests - -| Test Case | Expected Result | -|-----------|-----------------| -| Generate content with sufficient credits | Content created, credits deducted | -| Generate content with insufficient credits | Error: "Insufficient credits" | -| Generate content at exact credit balance | Success, balance = 0 | -| Generate multiple items in queue | Each deducts credits sequentially | -| Credit deduction failure mid-operation | Transaction rolled back, no partial deduction | - -### Plan Limit Tests - -| Plan | Test Case | Expected Result | -|------|-----------|-----------------| -| Free | Create 101st keyword (limit: 100) | Blocked | -| Starter | Create 6 queue items (limit: 5) | Blocked | -| Growth | Enable automation | Success (has access) | -| Pro | Create unlimited keywords | Success (no limit) | -| Enterprise | All operations | No limits enforced | - ---- - -## Pricing Page Updates - -**Location:** `frontend/src/pages/marketing/Pricing.tsx` - -### Required Elements - -1. **Plan Comparison Table** - - All tiers side-by-side - - Feature checkmarks - - Highlight "Most Popular" plan - - Monthly/Annual toggle with savings badge - -2. **Usage Limits Display** - - Show key limits per plan - - Use "Unlimited" label for null limits - - Tooltip explanations for complex limits - -3. **Credit System Explanation** - - What credits are - - How they're consumed - - How to buy more - - Credit rollover rules - -4. **FAQ Section** - - "What happens when I run out of credits?" - - "Can I change plans mid-month?" - - "Do unused credits roll over?" - - "What's included in Enterprise?" - -5. **Calculator Widget** (Optional) - - Estimate monthly usage - - Recommend plan based on needs - - Show credit consumption breakdown - ---- - -## Success Metrics - -- ✅ All AI operations enforce credit checks -- ✅ All create operations enforce limit checks -- ✅ Credit costs reflect actual provider costs + margin -- ✅ Plans are competitively priced -- ✅ Usage dashboard shows accurate counts -- ✅ Limit warnings prevent user frustration -- ✅ Upgrade flow is clear and frictionless -- ✅ Admin can adjust costs without code changes -- ✅ All tests pass - ---- - -## Related Files Reference - -### Backend -- `backend/igny8_core/auth/models.py` - Account, Plan models -- `backend/igny8_core/business/billing/models.py` - Credit models -- `backend/igny8_core/business/billing/constants.py` - Credit costs -- `backend/igny8_core/business/billing/services/credit_service.py` - Credit logic -- `backend/igny8_core/business/billing/services/limit_service.py` - **NEW** Limit enforcement -- `backend/igny8_core/modules/planner/views.py` - Planner API views -- `backend/igny8_core/modules/writer/views.py` - Writer API views - -### Frontend -- `frontend/src/components/dashboard/UsageLimitsWidget.tsx` - **NEW** Usage display -- `frontend/src/pages/marketing/Pricing.tsx` - Pricing page -- `frontend/src/pages/Planner/*.tsx` - Planner pages (add limit checks) -- `frontend/src/pages/Writer/*.tsx` - Writer pages (add limit checks) -- `frontend/src/services/api.ts` - API service (handle limit errors) - ---- - -## Notes - -- Limits should be enforced at API level, not just UI level -- Consider "soft limits" with warnings vs "hard limits" with blocks -- Credit expiration and rollover rules need business decision -- Enterprise pricing needs custom quote system -- Monitor actual usage patterns to optimize costs and limits -- A/B test different pricing tiers to maximize conversion
+ {keyword.generation_status === 'processing' && ( + }> + Processing + + )} + {keyword.generation_status === 'completed' && ( + Completed + )} +