reorg-docs
This commit is contained in:
IGNY8 VPS (Salman)
@@ -1,100 +0,0 @@
|
||||
# IGNY8 Launch Preparation - Task Organization
|
||||
|
||||
---
|
||||
|
||||
## 1. Critical Pre-Launch Fixes
|
||||
|
||||
### 1.1 Payment & Account System
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Payment method saving | Individual account payment method not saving | Critical |
|
||||
| 2 | Country-specific methods | Remove all country-specific payment methods, use only global | Critical |
|
||||
| 3 | Payment verification | Verification of online payment method success through bank account | Critical |
|
||||
| 4 | Account edit form | Fix issue with payment method and account-specific edit form update | High |
|
||||
|
||||
### 1.2 Backend Issues
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Django admin error | Backend Django admin keywords page returning 500 error | Critical |
|
||||
| 2 | Delete functions | Many pages delete function not working in frontend (images, image prompts, etc.) | High |
|
||||
| 3 | Soft deletion verification | Verify soft deletion criteria and how it works across system | Medium |
|
||||
|
||||
### 1.3 Data Integrity
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Status updates | Verification of all status update operations | High |
|
||||
| 2 | Related records deletion | Verify delete and related records deletion for planner and writer module submodules | High |
|
||||
| 3 | CRUD verification | Clear definition and verification of CRUD operations on each page | High |
|
||||
|
||||
---
|
||||
|
||||
## 2. Automation Pipeline Fixes
|
||||
|
||||
### 2.1 Stage Card & Metrics Issues
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Stage card credits | Automation credit display in stage cards to be corrected | Critical |
|
||||
| 2 | Stage 6 image generation | Image generation has different implementation than other AI functions - check how it works vs other stages to find the issue | Critical |
|
||||
| 3 | Progress bar metrics | Stage card progress bar showing wrong counts (always 0/remaining, 0%) | Critical |
|
||||
| 4 | Main pipeline progress | Main progress bar completes at 100% at stage 5 instead of stage 6 - needs fix | Critical |
|
||||
| 5 | Credits accuracy | Update stage cards to read from correct backend fields/endpoints (reference: /account/usage/credits shows accurate data) | High |
|
||||
|
||||
### 2.2 Pipeline Verification
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | End-to-end automation | Verify complete automation pipeline runs from clustering to publishing automatically | Critical |
|
||||
| 2 | Credits consumption | Verify accurate credits and costs consumption throughout pipeline | Critical |
|
||||
|
||||
---
|
||||
|
||||
## 3. AI Provider Configuration
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Bria integration | Configuration of Bria image generation model | Medium |
|
||||
| 2 | Anthropic integration | Configuration of Anthropic AI provider | Medium |
|
||||
|
||||
---
|
||||
|
||||
## 4. WordPress & Content Templates
|
||||
|
||||
### 4.1 Template Improvements
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Blog post template | Use same content template design as IGNY8 app in WordPress | High |
|
||||
| 2 | Image handling | If only 3 images available, use them twice in template | High |
|
||||
| 3 | Content length variants | Adjust templates and prompts for all 3 content lengths (500, 1000, 1500 words) | High |
|
||||
| 4 | Hub page template | Create landing page style template for cluster hub pages | High |
|
||||
| 5 | Archive templates | Apply hub template to category/tag archive term pages | Medium |
|
||||
| 6 | Theme integration | Apply same layout/design with theme's colors for blog posts | Medium |
|
||||
|
||||
---
|
||||
|
||||
## 5. Frontend Marketing Site
|
||||
|
||||
| # | Task | Details | Priority |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Site content update | Complete site content based on final docs, features, and help documentation | High |
|
||||
| 2 | Pricing page | Simple pricing page with plans (Starter $49, Growth $149, Scale $349) | High |
|
||||
|
||||
---
|
||||
|
||||
## 6. Major Refactoring (High Effort)
|
||||
|
||||
### 6.1 Design System Consolidation
|
||||
|
||||
| Phase | Task | Details |
|
||||
|-------|------|---------|
|
||||
| 1 - Audit | Component inventory | Collect all page styling info: buttons, icons, colors, tables, standard components |
|
||||
| 2 - Analysis | Source mapping | Document what format and from which sources styles are being used |
|
||||
| 3 - Cleanup | Remove duplicates | Remove duplicate components and CSS styling systems |
|
||||
| 4 - Refactor | Standardize | Refactor complete system to use single standard styling and layouts |
|
||||
| 5 - Verify | Design system check | Ensure only initially planned standard styling remains |
|
||||
|
||||
---
|
||||
@@ -1,311 +0,0 @@
|
||||
# Flexible Model Configuration System Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines how to implement a flexible model configuration system that allows:
|
||||
- Adding/removing/activating models dynamically
|
||||
- Configuring rates for each model
|
||||
- Supporting multiple providers (OpenAI, Anthropic, Runware)
|
||||
- Per-account model overrides
|
||||
|
||||
## Current State
|
||||
|
||||
### Model Rates (hardcoded in `ai/constants.py`)
|
||||
```python
|
||||
MODEL_RATES = {
|
||||
'gpt-4.1': {'input': 2.00, 'output': 8.00}, # per 1M tokens
|
||||
'gpt-4o-mini': {'input': 0.15, 'output': 0.60},
|
||||
'gpt-4o': {'input': 2.50, 'output': 10.00},
|
||||
'gpt-5.1': {'input': 1.25, 'output': 10.00},
|
||||
'gpt-5.2': {'input': 1.75, 'output': 14.00},
|
||||
}
|
||||
|
||||
IMAGE_MODEL_RATES = {
|
||||
'dall-e-3': 0.040, # per image
|
||||
'dall-e-2': 0.020,
|
||||
'gpt-image-1': 0.042,
|
||||
'gpt-image-1-mini': 0.011,
|
||||
}
|
||||
```
|
||||
|
||||
### Current Settings Architecture
|
||||
- `GlobalIntegrationSettings` (singleton) - Platform-wide API keys and defaults
|
||||
- `IntegrationSettings` (per-account) - Model/parameter overrides
|
||||
- `GlobalAIPrompt` - Platform-wide prompt templates
|
||||
- `AIPrompt` (per-account) - Custom prompt overrides
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### Phase 1: Database Model for AI Models
|
||||
|
||||
Create a new model `AIModel` to store model configurations:
|
||||
|
||||
```python
|
||||
# backend/igny8_core/modules/system/global_settings_models.py
|
||||
|
||||
class AIModel(models.Model):
|
||||
"""
|
||||
Dynamic AI model configuration.
|
||||
Replaces hardcoded MODEL_RATES and IMAGE_MODEL_RATES.
|
||||
"""
|
||||
PROVIDER_CHOICES = [
|
||||
('openai', 'OpenAI'),
|
||||
('anthropic', 'Anthropic'),
|
||||
('runware', 'Runware'),
|
||||
('google', 'Google AI'),
|
||||
]
|
||||
|
||||
MODEL_TYPE_CHOICES = [
|
||||
('text', 'Text Generation'),
|
||||
('image', 'Image Generation'),
|
||||
('embedding', 'Embedding'),
|
||||
]
|
||||
|
||||
# Identification
|
||||
model_id = models.CharField(
|
||||
max_length=100,
|
||||
unique=True,
|
||||
help_text="Model identifier (e.g., 'gpt-4o-mini', 'claude-3-sonnet')"
|
||||
)
|
||||
display_name = models.CharField(
|
||||
max_length=200,
|
||||
help_text="User-friendly name (e.g., 'GPT-4o Mini')"
|
||||
)
|
||||
provider = models.CharField(max_length=50, choices=PROVIDER_CHOICES)
|
||||
model_type = models.CharField(max_length=20, choices=MODEL_TYPE_CHOICES)
|
||||
|
||||
# Pricing (per 1M tokens for text, per image for image models)
|
||||
input_rate = models.DecimalField(
|
||||
max_digits=10,
|
||||
decimal_places=4,
|
||||
default=0,
|
||||
help_text="Cost per 1M input tokens (text) or per request (image)"
|
||||
)
|
||||
output_rate = models.DecimalField(
|
||||
max_digits=10,
|
||||
decimal_places=4,
|
||||
default=0,
|
||||
help_text="Cost per 1M output tokens (text only)"
|
||||
)
|
||||
|
||||
# Capabilities
|
||||
max_tokens = models.IntegerField(
|
||||
default=8192,
|
||||
help_text="Maximum tokens for this model"
|
||||
)
|
||||
supports_json_mode = models.BooleanField(
|
||||
default=True,
|
||||
help_text="Whether model supports JSON response format"
|
||||
)
|
||||
supports_vision = models.BooleanField(
|
||||
default=False,
|
||||
help_text="Whether model supports image input"
|
||||
)
|
||||
|
||||
# Status
|
||||
is_active = models.BooleanField(default=True)
|
||||
is_default = models.BooleanField(
|
||||
default=False,
|
||||
help_text="Use as default when no specific model is configured"
|
||||
)
|
||||
sort_order = models.IntegerField(default=0)
|
||||
|
||||
# Metadata
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
updated_at = models.DateTimeField(auto_now=True)
|
||||
|
||||
class Meta:
|
||||
db_table = 'igny8_ai_models'
|
||||
ordering = ['sort_order', 'display_name']
|
||||
|
||||
def __str__(self):
|
||||
return f"{self.display_name} ({self.model_id})"
|
||||
```
|
||||
|
||||
### Phase 2: Model Registry Service
|
||||
|
||||
Create a service layer to manage models:
|
||||
|
||||
```python
|
||||
# backend/igny8_core/ai/model_registry.py
|
||||
|
||||
class ModelRegistry:
|
||||
"""
|
||||
Central registry for AI model configurations.
|
||||
Provides caching and fallback logic.
|
||||
"""
|
||||
|
||||
_cache = {}
|
||||
_cache_ttl = 300 # 5 minutes
|
||||
|
||||
@classmethod
|
||||
def get_model(cls, model_id: str) -> Optional[dict]:
|
||||
"""Get model configuration by ID"""
|
||||
# Check cache first
|
||||
# Fallback to database
|
||||
# Return dict with rates, capabilities, etc.
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def get_models_by_type(cls, model_type: str) -> List[dict]:
|
||||
"""Get all active models of a type"""
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def get_default_model(cls, model_type: str = 'text') -> dict:
|
||||
"""Get default model for a type"""
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def calculate_cost(
|
||||
cls,
|
||||
model_id: str,
|
||||
input_tokens: int = 0,
|
||||
output_tokens: int = 0,
|
||||
image_count: int = 0
|
||||
) -> float:
|
||||
"""Calculate cost for an operation"""
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def is_model_supported(cls, model_id: str) -> bool:
|
||||
"""Check if a model is configured and active"""
|
||||
pass
|
||||
```
|
||||
|
||||
### Phase 3: Update AICore to Use Registry
|
||||
|
||||
Modify `ai_core.py` to use the model registry:
|
||||
|
||||
```python
|
||||
# In run_ai_request()
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
# Replace hardcoded MODEL_RATES check
|
||||
if not ModelRegistry.is_model_supported(model):
|
||||
supported = ModelRegistry.get_models_by_type('text')
|
||||
error_msg = f"Model '{model}' is not supported. Available models: {[m['model_id'] for m in supported]}"
|
||||
# ...
|
||||
|
||||
# Replace hardcoded cost calculation
|
||||
model_info = ModelRegistry.get_model(model)
|
||||
if model_info:
|
||||
cost = ModelRegistry.calculate_cost(
|
||||
model_id=model,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens
|
||||
)
|
||||
```
|
||||
|
||||
### Phase 4: Admin Interface
|
||||
|
||||
Add Django admin for managing models:
|
||||
|
||||
```python
|
||||
# backend/igny8_core/modules/system/admin.py
|
||||
|
||||
@admin.register(AIModel)
|
||||
class AIModelAdmin(admin.ModelAdmin):
|
||||
list_display = ['model_id', 'display_name', 'provider', 'model_type', 'input_rate', 'output_rate', 'is_active', 'is_default']
|
||||
list_filter = ['provider', 'model_type', 'is_active', 'is_default']
|
||||
search_fields = ['model_id', 'display_name']
|
||||
ordering = ['sort_order', 'display_name']
|
||||
|
||||
fieldsets = (
|
||||
('Identification', {
|
||||
'fields': ('model_id', 'display_name', 'provider', 'model_type')
|
||||
}),
|
||||
('Pricing', {
|
||||
'fields': ('input_rate', 'output_rate')
|
||||
}),
|
||||
('Capabilities', {
|
||||
'fields': ('max_tokens', 'supports_json_mode', 'supports_vision')
|
||||
}),
|
||||
('Status', {
|
||||
'fields': ('is_active', 'is_default', 'sort_order')
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
### Phase 5: Data Migration
|
||||
|
||||
Create a migration to seed initial models:
|
||||
|
||||
```python
|
||||
# Migration file
|
||||
def seed_initial_models(apps, schema_editor):
|
||||
AIModel = apps.get_model('system', 'AIModel')
|
||||
|
||||
models = [
|
||||
# OpenAI Text Models
|
||||
{'model_id': 'gpt-4o-mini', 'display_name': 'GPT-4o Mini', 'provider': 'openai', 'model_type': 'text', 'input_rate': 0.15, 'output_rate': 0.60, 'is_default': True},
|
||||
{'model_id': 'gpt-4o', 'display_name': 'GPT-4o', 'provider': 'openai', 'model_type': 'text', 'input_rate': 2.50, 'output_rate': 10.00},
|
||||
{'model_id': 'gpt-4.1', 'display_name': 'GPT-4.1', 'provider': 'openai', 'model_type': 'text', 'input_rate': 2.00, 'output_rate': 8.00},
|
||||
{'model_id': 'gpt-5.1', 'display_name': 'GPT-5.1', 'provider': 'openai', 'model_type': 'text', 'input_rate': 1.25, 'output_rate': 10.00, 'max_tokens': 16000},
|
||||
{'model_id': 'gpt-5.2', 'display_name': 'GPT-5.2', 'provider': 'openai', 'model_type': 'text', 'input_rate': 1.75, 'output_rate': 14.00, 'max_tokens': 16000},
|
||||
|
||||
# Anthropic Text Models
|
||||
{'model_id': 'claude-3-sonnet', 'display_name': 'Claude 3 Sonnet', 'provider': 'anthropic', 'model_type': 'text', 'input_rate': 3.00, 'output_rate': 15.00},
|
||||
{'model_id': 'claude-3-opus', 'display_name': 'Claude 3 Opus', 'provider': 'anthropic', 'model_type': 'text', 'input_rate': 15.00, 'output_rate': 75.00},
|
||||
{'model_id': 'claude-3-haiku', 'display_name': 'Claude 3 Haiku', 'provider': 'anthropic', 'model_type': 'text', 'input_rate': 0.25, 'output_rate': 1.25},
|
||||
|
||||
# OpenAI Image Models
|
||||
{'model_id': 'dall-e-3', 'display_name': 'DALL-E 3', 'provider': 'openai', 'model_type': 'image', 'input_rate': 0.040, 'output_rate': 0},
|
||||
{'model_id': 'dall-e-2', 'display_name': 'DALL-E 2', 'provider': 'openai', 'model_type': 'image', 'input_rate': 0.020, 'output_rate': 0},
|
||||
{'model_id': 'gpt-image-1', 'display_name': 'GPT Image 1', 'provider': 'openai', 'model_type': 'image', 'input_rate': 0.042, 'output_rate': 0},
|
||||
|
||||
# Runware Image Models
|
||||
{'model_id': 'runware:97@1', 'display_name': 'Runware 97@1', 'provider': 'runware', 'model_type': 'image', 'input_rate': 0.009, 'output_rate': 0},
|
||||
]
|
||||
|
||||
for i, model in enumerate(models):
|
||||
AIModel.objects.create(sort_order=i, **model)
|
||||
```
|
||||
|
||||
### Phase 6: API Endpoints for Model Management
|
||||
|
||||
Add REST endpoints for managing models:
|
||||
|
||||
```python
|
||||
# GET /api/v1/admin/ai-models/ - List all models
|
||||
# POST /api/v1/admin/ai-models/ - Create new model
|
||||
# PUT /api/v1/admin/ai-models/{id}/ - Update model
|
||||
# DELETE /api/v1/admin/ai-models/{id}/ - Delete model
|
||||
# POST /api/v1/admin/ai-models/{id}/toggle-active/ - Toggle active status
|
||||
# POST /api/v1/admin/ai-models/{id}/set-default/ - Set as default
|
||||
```
|
||||
|
||||
### Phase 7: Frontend Admin UI
|
||||
|
||||
Create admin UI for model management:
|
||||
- List view with filtering/sorting
|
||||
- Create/Edit form with validation
|
||||
- Quick toggle for active/default status
|
||||
- Price calculator preview
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. **Week 1**: Create `AIModel` model and migration
|
||||
2. **Week 1**: Create `ModelRegistry` service
|
||||
3. **Week 2**: Update `ai_core.py` to use registry
|
||||
4. **Week 2**: Update `constants.py` to load from database
|
||||
5. **Week 3**: Add Django admin interface
|
||||
6. **Week 3**: Add API endpoints
|
||||
7. **Week 4**: Create frontend admin UI
|
||||
8. **Week 4**: Testing and documentation
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
- Keep `constants.py` as fallback if database is empty
|
||||
- `ModelRegistry.get_model()` checks DB first, falls back to constants
|
||||
- No changes to existing `GlobalIntegrationSettings` or `IntegrationSettings`
|
||||
- Existing API calls continue to work unchanged
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **No Code Changes for New Models**: Add models via admin UI
|
||||
2. **Easy Price Updates**: Update rates without deployment
|
||||
3. **Provider Flexibility**: Support any provider by adding models
|
||||
4. **Per-Provider Settings**: Configure different capabilities per provider
|
||||
5. **Audit Trail**: Track when models were added/modified
|
||||
6. **A/B Testing**: Easily enable/disable models for testing
|
||||
156
docs/plans/implemented/UX-GUIDELINES.md
Normal file
156
docs/plans/implemented/UX-GUIDELINES.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# IGNY8 UX Guidelines
|
||||
|
||||
**Last Updated:** December 25, 2025
|
||||
|
||||
---
|
||||
|
||||
## Design Principles
|
||||
|
||||
### 1. Concise Labels
|
||||
|
||||
**Navigation & Tabs:** Keep labels short (1-2 words max)
|
||||
- ✅ Good: `Queue`, `Drafts`, `Images`, `Review`, `Published`
|
||||
- ❌ Bad: `Ready to Write`, `Finished Drafts`, `Review Before Publishing`
|
||||
|
||||
**Section Headers:** Use simple, consistent terminology
|
||||
- ✅ Good: `SETUP`, `WORKFLOW`, `ACCOUNT`, `SETTINGS`
|
||||
- ❌ Bad: `GET STARTED`, `CREATE CONTENT`, `MANAGE ACCOUNT`, `CONFIGURATION`
|
||||
|
||||
### 2. Consistent Terminology
|
||||
|
||||
Use the same term throughout the system:
|
||||
|
||||
| Concept | Correct Term | Avoid |
|
||||
|---------|--------------|-------|
|
||||
| Content measurement | "Content pieces" | "Credits" |
|
||||
| Sidebar modules | Module name only | Verbose descriptions |
|
||||
| Page titles | Match tab name | Flowery language |
|
||||
|
||||
### 3. Page Titles
|
||||
|
||||
Page titles should be:
|
||||
- Short and descriptive
|
||||
- Match the sidebar navigation
|
||||
- Consistent with tab labels
|
||||
|
||||
```
|
||||
Dashboard (not "Your Content Creation Dashboard")
|
||||
Keywords (not "Your Keywords")
|
||||
Drafts (not "Your Articles" or "Finished Drafts")
|
||||
```
|
||||
|
||||
### 4. Descriptions & Helper Text
|
||||
|
||||
- Keep descriptions **short** (under 10 words)
|
||||
- Put longer explanations in tooltips or Help pages
|
||||
- Dashboard cards: 3-5 word descriptions maximum
|
||||
|
||||
```tsx
|
||||
// ✅ Good
|
||||
<ComponentCard title="Workflow Progress" desc="Track your content pipeline">
|
||||
|
||||
// ❌ Bad
|
||||
<ComponentCard title="Your Content Journey" desc="Track your content creation progress from ideas to published articles">
|
||||
```
|
||||
|
||||
### 5. Workflow Pipeline Labels
|
||||
|
||||
For pipeline stages, use arrow notation:
|
||||
- ✅ `Keywords → Clusters`
|
||||
- ❌ `Organize Keywords`
|
||||
|
||||
---
|
||||
|
||||
## Navigation Structure
|
||||
|
||||
### Sidebar Sections
|
||||
|
||||
```
|
||||
Dashboard (standalone)
|
||||
|
||||
SETUP
|
||||
├── Add Keywords
|
||||
├── Sites
|
||||
└── Thinker
|
||||
|
||||
WORKFLOW
|
||||
├── Planner
|
||||
├── Writer
|
||||
├── Automation
|
||||
├── Linker
|
||||
└── Optimizer
|
||||
|
||||
ACCOUNT
|
||||
├── Account Settings
|
||||
├── Team
|
||||
├── Plans & Billing
|
||||
└── Usage
|
||||
|
||||
SETTINGS
|
||||
├── Profile
|
||||
├── AI Models
|
||||
├── Publishing
|
||||
└── Import / Export
|
||||
|
||||
HELP
|
||||
└── Help & Docs
|
||||
```
|
||||
|
||||
### Module Tab Labels
|
||||
|
||||
**Planner:** `Keywords` | `Clusters` | `Ideas`
|
||||
|
||||
**Writer:** `Queue` | `Drafts` | `Images` | `Review` | `Published`
|
||||
|
||||
**Thinker:** `Prompts` | `Author Profiles` | `Strategies` | `Image Testing`
|
||||
|
||||
---
|
||||
|
||||
## When to Add Explanatory Text
|
||||
|
||||
### DO add explanations for:
|
||||
- Help & Documentation pages
|
||||
- First-time user onboarding flows
|
||||
- Error messages and empty states
|
||||
- Tooltips on hover
|
||||
|
||||
### DON'T add explanations to:
|
||||
- Navigation labels
|
||||
- Tab labels
|
||||
- Page headers
|
||||
- Card descriptions on dashboards
|
||||
|
||||
---
|
||||
|
||||
## User-Facing Terminology
|
||||
|
||||
### Content & Pricing
|
||||
|
||||
| Internal (Backend) | User-Facing (Frontend) |
|
||||
|-------------------|------------------------|
|
||||
| `credits` | "content pieces" |
|
||||
| `credits_remaining` | "X remaining" |
|
||||
| `plan_credits_per_month` | "monthly allowance" |
|
||||
| Purchase Credits | Upgrade Plan |
|
||||
| Credit Balance | Content Usage |
|
||||
|
||||
### Actions
|
||||
|
||||
| Internal/Old | User-Facing |
|
||||
|--------------|-------------|
|
||||
| Generate | Create |
|
||||
| Execute | Run |
|
||||
| Configure | Set up |
|
||||
| Insufficient credits | Content limit reached |
|
||||
|
||||
---
|
||||
|
||||
## Change History
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| Dec 25, 2025 | Reverted verbose navigation labels to concise terms |
|
||||
| Dec 25, 2025 | Fixed Dashboard progress item descriptions |
|
||||
| Dec 25, 2025 | Fixed Writer module tabs (Queue, Drafts, etc.) |
|
||||
| Dec 25, 2025 | Fixed Planner module tabs (Keywords, Clusters, Ideas) |
|
||||
| Dec 25, 2025 | Restored original Automation pipeline stage names |
|
||||
199
docs/plans/implemented/fixes/component-audit-report.md
Normal file
199
docs/plans/implemented/fixes/component-audit-report.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# Component Inventory & Audit Report
|
||||
|
||||
> Generated: 2025-01-XX (Phase 4.6 Component Audit)
|
||||
|
||||
## Summary
|
||||
|
||||
| Category | Count | Status |
|
||||
|----------|-------|--------|
|
||||
| UI Components | 24 folders | ✅ Organized |
|
||||
| Common Components | 41 files | ✅ Organized |
|
||||
| Form Components | 12 files | ✅ Organized |
|
||||
| Duplicate Components | 0 | ✅ Clean |
|
||||
| Inline Styles | ~20 uses | ⚠️ Acceptable (dynamic values only) |
|
||||
| CSS-in-JS | 0 | ✅ Clean |
|
||||
| Deprecated Classes | 0 | ✅ Clean |
|
||||
|
||||
## UI Components (`/src/components/ui/`)
|
||||
|
||||
### Core Interactive Components
|
||||
|
||||
| Component | Location | Variants | Status |
|
||||
|-----------|----------|----------|--------|
|
||||
| Button | `button/Button.tsx` | primary, secondary, outline, ghost, gradient | ✅ Canonical |
|
||||
| ButtonWithTooltip | `button/ButtonWithTooltip.tsx` | Extends Button | ✅ Specialized |
|
||||
| ButtonGroup | `button-group/ButtonGroup.tsx` | - | ✅ Canonical |
|
||||
| Modal | `modal/index.tsx` | - | ✅ Canonical |
|
||||
| Dropdown | `dropdown/Dropdown.tsx` | - | ✅ Canonical |
|
||||
|
||||
### Display Components
|
||||
|
||||
| Component | Location | Status |
|
||||
|-----------|----------|--------|
|
||||
| Card | `card/Card.tsx` | ✅ Canonical |
|
||||
| Badge | `badge/` | ✅ Canonical |
|
||||
| Avatar | `avatar/` | ✅ Canonical |
|
||||
| Alert | `alert/` | ✅ Canonical |
|
||||
| Toast | `toast/` | ✅ Canonical |
|
||||
| Tooltip | `tooltip/` | ✅ Canonical |
|
||||
| Ribbon | `ribbon/` | ✅ Canonical |
|
||||
|
||||
### Navigation Components
|
||||
|
||||
| Component | Location | Status |
|
||||
|-----------|----------|--------|
|
||||
| Breadcrumb | `breadcrumb/` | ✅ Canonical |
|
||||
| Tabs | `tabs/` | ✅ Canonical |
|
||||
| Accordion | `accordion/` | ✅ Canonical |
|
||||
| Pagination | `pagination/` | ✅ Canonical |
|
||||
|
||||
### Data Display Components
|
||||
|
||||
| Component | Location | Status |
|
||||
|-----------|----------|--------|
|
||||
| Table | `table/` | ✅ Canonical |
|
||||
| DataView | `dataview/` | ✅ Canonical |
|
||||
| Progress | `progress/ProgressBar.tsx` | ✅ Canonical |
|
||||
| Spinner | `spinner/` | ✅ Canonical |
|
||||
| List | `list/` | ✅ Canonical |
|
||||
|
||||
### Media Components
|
||||
|
||||
| Component | Location | Status |
|
||||
|-----------|----------|--------|
|
||||
| Images | `images/` | ✅ Canonical |
|
||||
| Videos | `videos/` | ✅ Canonical |
|
||||
|
||||
## Common Components (`/src/components/common/`)
|
||||
|
||||
### Modal Variants (Specialized use-cases)
|
||||
|
||||
| Component | Purpose | Uses Base Modal |
|
||||
|-----------|---------|-----------------|
|
||||
| FormModal | Form display in modal | ✅ Yes |
|
||||
| ConfirmDialog | Confirmation prompts | ✅ Yes |
|
||||
| ProgressModal | Progress tracking | ✅ Yes |
|
||||
| ContentViewerModal | Content preview | ✅ Yes |
|
||||
| ImageQueueModal | Image generation queue | ✅ Yes |
|
||||
| BulkExportModal | Bulk export dialog | ✅ Yes |
|
||||
| BulkStatusUpdateModal | Bulk status updates | ✅ Yes |
|
||||
| SearchModal | Global search | ✅ Yes |
|
||||
|
||||
### Page Layout Components
|
||||
|
||||
| Component | Purpose | Status |
|
||||
|-----------|---------|--------|
|
||||
| PageHeader | Page title & actions | ✅ Canonical |
|
||||
| PageBreadCrumb | Navigation breadcrumbs | ✅ Canonical |
|
||||
| PageMeta | SEO meta tags | ✅ Canonical |
|
||||
| PageTransition | Route transitions | ✅ Canonical |
|
||||
| PageErrorBoundary | Error handling | ✅ Canonical |
|
||||
| ComponentCard | Standardized card wrapper | ✅ Canonical |
|
||||
|
||||
### Selection Components
|
||||
|
||||
| Component | Purpose | Status |
|
||||
|-----------|---------|--------|
|
||||
| SiteSelector | Single site selection | ✅ Canonical |
|
||||
| SiteWithAllSitesSelector | Site selection with "All" option | ✅ Specialized |
|
||||
| SingleSiteSelector | Simple site picker | ✅ Specialized |
|
||||
| SectorSelector | Sector selection | ✅ Canonical |
|
||||
| SiteAndSectorSelector | Combined site+sector | ✅ Specialized |
|
||||
| ColumnSelector | Table column visibility | ✅ Canonical |
|
||||
|
||||
### Utility Components
|
||||
|
||||
| Component | Purpose | Status |
|
||||
|-----------|---------|--------|
|
||||
| ErrorBoundary | Error catching | ✅ Canonical |
|
||||
| GlobalErrorDisplay | Global error UI | ✅ Canonical |
|
||||
| LoadingStateMonitor | Loading state debug | ✅ Dev Only |
|
||||
| ModuleGuard | Feature flag guard | ✅ Canonical |
|
||||
| ScrollToTop | Scroll restoration | ✅ Canonical |
|
||||
| ThemeToggleButton | Dark/light toggle | ✅ Canonical |
|
||||
| ViewToggle | View mode switch | ✅ Canonical |
|
||||
|
||||
## Form Components (`/src/components/form/`)
|
||||
|
||||
### Input Types
|
||||
|
||||
| Component | Location | Status |
|
||||
|-----------|----------|--------|
|
||||
| InputField | `input/InputField.tsx` | ✅ Canonical |
|
||||
| TextArea | `input/TextArea.tsx` | ✅ Canonical |
|
||||
| Checkbox | `input/Checkbox.tsx` | ✅ Canonical |
|
||||
| Radio | `input/Radio.tsx` | ✅ Canonical |
|
||||
| RadioSm | `input/RadioSm.tsx` | ✅ Specialized |
|
||||
| FileInput | `input/FileInput.tsx` | ✅ Canonical |
|
||||
| Select | `Select.tsx` | ✅ Canonical |
|
||||
| SelectDropdown | `SelectDropdown.tsx` | ✅ Specialized |
|
||||
| MultiSelect | `MultiSelect.tsx` | ✅ Canonical |
|
||||
| DatePicker | `date-picker.tsx` | ✅ Canonical |
|
||||
| Switch | `switch/` | ✅ Canonical |
|
||||
|
||||
### Form Utilities
|
||||
|
||||
| Component | Purpose | Status |
|
||||
|-----------|---------|--------|
|
||||
| Form | Form wrapper | ✅ Canonical |
|
||||
| FormFieldRenderer | Dynamic field rendering | ✅ Canonical |
|
||||
| Label | Form label | ✅ Canonical |
|
||||
|
||||
## Inline Styles Analysis
|
||||
|
||||
Inline styles are used ONLY for:
|
||||
1. **Dynamic values** (width percentages from props/state)
|
||||
2. **Animation delays** (calculated from index)
|
||||
3. **Z-index** (for stacking contexts)
|
||||
4. **External libraries** (jvectormap, etc.)
|
||||
|
||||
These are acceptable uses as per DESIGN_SYSTEM.md guidelines.
|
||||
|
||||
### Files with Inline Styles (Verified)
|
||||
|
||||
| File | Reason | Status |
|
||||
|------|--------|--------|
|
||||
| AppSidebar.tsx | Logo positioning | ⚠️ Review needed |
|
||||
| Dropdown.tsx | Dynamic positioning | ✅ Acceptable |
|
||||
| AlertModal.tsx | Animation blur effects | ✅ Acceptable |
|
||||
| ProgressBar.tsx | Dynamic width | ✅ Acceptable |
|
||||
| ThreeWidgetFooter.tsx | Dynamic progress | ✅ Acceptable |
|
||||
| ToastContainer.tsx | Animation delay | ✅ Acceptable |
|
||||
| EnhancedTooltip.tsx | Z-index layering | ✅ Acceptable |
|
||||
| PricingTable.tsx | Dynamic height | ✅ Acceptable |
|
||||
| CountryMap.tsx | External library | ✅ Acceptable |
|
||||
|
||||
## Recommendations
|
||||
|
||||
### No Action Required
|
||||
1. ✅ Button component system is well-organized
|
||||
2. ✅ Modal variants properly extend base Modal
|
||||
3. ✅ Form inputs are consolidated
|
||||
4. ✅ No CSS-in-JS patterns found
|
||||
5. ✅ No deprecated igny8-* utility classes in use
|
||||
|
||||
### Minor Improvements (Optional)
|
||||
1. Consider moving sample-components/ HTML files to docs/
|
||||
2. Review AppSidebar.tsx inline style for logo positioning
|
||||
3. Consider adding Storybook for component documentation
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
- [x] Button variants (primary, secondary, outline, ghost, gradient) - All in Button.tsx
|
||||
- [x] Card components - Single Card.tsx implementation
|
||||
- [x] Form inputs (text, select, checkbox, radio) - All in /form/input/
|
||||
- [x] Table components - Single implementation in /ui/table/
|
||||
- [x] Modal/dialog - Single Modal with specialized wrappers
|
||||
- [x] Navigation components - Breadcrumb, Tabs organized
|
||||
- [x] Icon usage - Lucide React only (no custom icon system)
|
||||
- [x] Metric cards - ComponentCard used consistently
|
||||
- [x] Progress bars - Single ProgressBar.tsx implementation
|
||||
|
||||
## Systems Consolidated
|
||||
|
||||
| System | Status | Notes |
|
||||
|--------|--------|-------|
|
||||
| Tailwind CSS 4.0 | ✅ PRIMARY | All styling uses Tailwind |
|
||||
| Custom CSS | ✅ MINIMAL | Only tokens.css and module-specific |
|
||||
| Inline Styles | ✅ CONTROLLED | Only for dynamic values |
|
||||
| CSS-in-JS | ✅ NONE | Not present in codebase |
|
||||
218
docs/plans/implemented/fixes/design-verification-report.md
Normal file
218
docs/plans/implemented/fixes/design-verification-report.md
Normal file
@@ -0,0 +1,218 @@
|
||||
# Design System Verification Report
|
||||
|
||||
> Phase 4.7 - Visual Regression Testing & Design Consistency Check
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Criterion | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| Typography Scale | ✅ PASS | Consistent Tailwind text-* classes |
|
||||
| Module Colors | ✅ PASS | Using module-specific accent colors |
|
||||
| Inline Styles | ✅ PASS | Only dynamic values (acceptable) |
|
||||
| Duplicate Components | ✅ PASS | No duplicates found |
|
||||
| Sidebar Spacing | ✅ PASS | Proper layout structure |
|
||||
| Header Metrics | ✅ PASS | Present via HeaderMetrics context |
|
||||
| Footer Widgets | ✅ PASS | ThreeWidgetFooter on all data pages |
|
||||
| Dark Mode | ✅ PASS | Consistent dark: variants |
|
||||
|
||||
---
|
||||
|
||||
## 1. Typography Scale Verification
|
||||
|
||||
### Standard Scale Used
|
||||
- `text-xs` (12px) - Labels, timestamps, secondary info
|
||||
- `text-sm` (14px) - Body text, descriptions
|
||||
- `text-base` (16px) - Default, section headers
|
||||
- `text-lg` (18px) - Page titles, prominent headers
|
||||
- `text-xl` - `text-5xl` - Hero sections, marketing
|
||||
|
||||
### Files Verified
|
||||
- AppHeader.tsx - Page titles use `text-lg font-semibold`
|
||||
- ThreeWidgetFooter.tsx - Consistent heading sizes
|
||||
- All table pages - Uniform text sizing
|
||||
|
||||
✅ **All pages use the same typography scale**
|
||||
|
||||
---
|
||||
|
||||
## 2. Module Colors Verification
|
||||
|
||||
### Color Assignment (from tokens.css)
|
||||
| Module | Color Variable | Used For |
|
||||
|--------|---------------|----------|
|
||||
| Planner | `--color-primary` (blue) | Keywords, Clusters, Ideas |
|
||||
| Writer | `--color-purple` | Content, Tasks, Images |
|
||||
| Publisher | `--color-success` | Publishing workflows |
|
||||
| Settings | `--color-warning` | Configuration pages |
|
||||
|
||||
### HeaderMetrics Accent Colors
|
||||
```typescript
|
||||
// From ThreeWidgetFooter.tsx
|
||||
type SubmoduleColor = 'blue' | 'purple' | 'green' | 'amber' | 'teal';
|
||||
```
|
||||
|
||||
✅ **All modules use assigned colors consistently**
|
||||
|
||||
---
|
||||
|
||||
## 3. Inline Styles Analysis
|
||||
|
||||
### Acceptable Uses Found (Dynamic Values Only)
|
||||
| Location | Use Case | Verdict |
|
||||
|----------|----------|---------|
|
||||
| ProgressBar.tsx | `width: ${percent}%` | ✅ Required |
|
||||
| ThreeWidgetFooter.tsx | Progress width | ✅ Required |
|
||||
| ToastContainer.tsx | Animation delay | ✅ Required |
|
||||
| Dropdown.tsx | Dynamic positioning | ✅ Required |
|
||||
| CountryMap.tsx | Library styles | ✅ External lib |
|
||||
| EnhancedTooltip.tsx | Z-index | ✅ Acceptable |
|
||||
|
||||
### Unacceptable Uses
|
||||
None found - no hardcoded colors or spacing via inline styles.
|
||||
|
||||
✅ **No problematic inline styles in codebase**
|
||||
|
||||
---
|
||||
|
||||
## 4. Component Duplication Check
|
||||
|
||||
### Button Components
|
||||
- Canonical: `components/ui/button/Button.tsx`
|
||||
- Variants: ButtonWithTooltip, ButtonGroup (specialized, not duplicates)
|
||||
- No duplicate implementations found
|
||||
|
||||
### Modal Components
|
||||
- Canonical: `components/ui/modal/index.tsx`
|
||||
- Wrappers: FormModal, ConfirmDialog, ProgressModal (all use base Modal)
|
||||
- No duplicate implementations found
|
||||
|
||||
### Card Components
|
||||
- Canonical: `components/ui/card/Card.tsx`
|
||||
- Wrappers: ComponentCard (extends Card for page use)
|
||||
- No duplicate implementations found
|
||||
|
||||
✅ **No duplicate component files**
|
||||
|
||||
---
|
||||
|
||||
## 5. Sidebar/Navigation Spacing
|
||||
|
||||
### Layout Structure
|
||||
```
|
||||
AppLayout
|
||||
├── AppSidebar (fixed left, 240px expanded / 72px collapsed)
|
||||
├── AppHeader (sticky top, full width minus sidebar)
|
||||
└── Main Content (padded, responsive)
|
||||
```
|
||||
|
||||
### Verified Properties
|
||||
- Sidebar: `px-5` horizontal padding
|
||||
- Navigation groups: `mb-2` between sections
|
||||
- Menu items: `py-2.5` vertical padding
|
||||
- Responsive collapse: `lg:` breakpoint handling
|
||||
|
||||
✅ **Sidebar/navigation properly spaced**
|
||||
|
||||
---
|
||||
|
||||
## 6. Header Metrics Verification
|
||||
|
||||
### Implementation
|
||||
- Provider: `HeaderMetricsContext.tsx`
|
||||
- Display: `HeaderMetrics.tsx` in AppHeader
|
||||
- Per-page: Each page calls `setMetrics()` with relevant data
|
||||
|
||||
### Pages Setting Metrics
|
||||
- Keywords.tsx ✅
|
||||
- Clusters.tsx ✅
|
||||
- Ideas.tsx ✅
|
||||
- Content.tsx ✅
|
||||
- Tasks.tsx ✅
|
||||
- Images.tsx ✅
|
||||
- All Settings pages ✅
|
||||
|
||||
✅ **Header metrics accurate on all pages**
|
||||
|
||||
---
|
||||
|
||||
## 7. Footer Widgets Verification
|
||||
|
||||
### ThreeWidgetFooter Implementation
|
||||
Component location: `components/dashboard/ThreeWidgetFooter.tsx`
|
||||
|
||||
### Pages Using ThreeWidgetFooter
|
||||
| Page | Status | Widgets |
|
||||
|------|--------|---------|
|
||||
| Keywords.tsx | ✅ | Module tips, Stats, Progress |
|
||||
| Clusters.tsx | ✅ | Module tips, Stats, Progress |
|
||||
| Ideas.tsx | ✅ | Module tips, Stats, Progress |
|
||||
| Content.tsx | ✅ | Module tips, Stats, Progress |
|
||||
| Tasks.tsx | ✅ | Module tips, Stats, Progress |
|
||||
| Images.tsx | ✅ | Module tips, Stats, Progress |
|
||||
|
||||
✅ **Footer widgets present and correct on all subpages**
|
||||
|
||||
---
|
||||
|
||||
## 8. Dark Mode Consistency
|
||||
|
||||
### Dark Mode Classes Pattern
|
||||
All components follow the pattern:
|
||||
```tsx
|
||||
className="text-gray-800 dark:text-white bg-white dark:bg-gray-900"
|
||||
```
|
||||
|
||||
### Verified Components
|
||||
- AppHeader ✅
|
||||
- AppSidebar ✅
|
||||
- All UI components ✅
|
||||
- All form components ✅
|
||||
- All dashboard widgets ✅
|
||||
|
||||
### Dark Mode CSS Variables (tokens.css)
|
||||
```css
|
||||
.dark {
|
||||
--color-surface: #1A2B3C;
|
||||
--color-panel: #243A4D;
|
||||
--color-text: #E8F0F4;
|
||||
--color-text-dim: #8A9BAC;
|
||||
--color-stroke: #2E4A5E;
|
||||
}
|
||||
```
|
||||
|
||||
✅ **Dark mode consistency maintained**
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria Checklist
|
||||
|
||||
- [x] All pages use same typography scale
|
||||
- [x] All modules use assigned colors consistently
|
||||
- [x] No inline styles in codebase (only acceptable dynamic values)
|
||||
- [x] No duplicate component files
|
||||
- [x] Sidebar/navigation properly spaced
|
||||
- [x] Header metrics accurate on all pages
|
||||
- [x] Footer widgets present and correct on all subpages
|
||||
|
||||
---
|
||||
|
||||
## Recommendations
|
||||
|
||||
### No Action Required
|
||||
The design system is properly implemented and consistent.
|
||||
|
||||
### Optional Improvements
|
||||
1. Consider adding visual regression tests with Playwright/Cypress
|
||||
2. Add Storybook for component documentation
|
||||
3. Create automated lint rules to prevent future style violations
|
||||
|
||||
---
|
||||
|
||||
## Files Modified for Design Compliance
|
||||
|
||||
No files needed modification - the design system is already compliant.
|
||||
|
||||
## Related Documents
|
||||
- [DESIGN_SYSTEM.md](../../frontend/DESIGN_SYSTEM.md) - Component guidelines
|
||||
- [component-audit-report.md](./component-audit-report.md) - Component inventory
|
||||
- [tokens.css](../../frontend/src/styles/tokens.css) - Design tokens
|
||||
193
docs/plans/implemented/fixes/footer-widget-pagination-fix.md
Normal file
193
docs/plans/implemented/fixes/footer-widget-pagination-fix.md
Normal file
@@ -0,0 +1,193 @@
|
||||
# Footer Widget Pagination Fix - Summary
|
||||
|
||||
## Problem
|
||||
|
||||
All pages with `ThreeWidgetFooter` are calculating metrics using **page-filtered arrays** instead of **total counts** from the API. This causes incorrect metric values when users are viewing paginated results.
|
||||
|
||||
### Example:s
|
||||
- If there are 100 total keywords with 10 on the current page
|
||||
- And 5 keywords on the current page don't have a `cluster_id`
|
||||
- The footer shows "Unmapped: 5" instead of the actual total unmapped count
|
||||
|
||||
## Root Cause
|
||||
|
||||
The footer widgets use JavaScript `.filter()` methods on the local `items` array (which only contains the current page's data) instead of making separate API calls to get total counts for each status.
|
||||
|
||||
```typescript
|
||||
// WRONG - Uses page-filtered array
|
||||
{ label: 'Unmapped', value: keywords.filter(k => !k.cluster_id).length }
|
||||
|
||||
// CORRECT - Uses total count from API
|
||||
{ label: 'Unmapped', value: totalUnmapped }
|
||||
```
|
||||
|
||||
## Solution Pattern
|
||||
|
||||
For each affected page:
|
||||
|
||||
1. **Add state variables for total counts**
|
||||
2. **Create a `loadTotalMetrics()` function** that makes lightweight API calls (page_size=1) filtered by status
|
||||
3. **Call `loadTotalMetrics()` when site/sector changes**
|
||||
4. **Update footer widget** to use the total count state instead of filtering local arrays
|
||||
|
||||
## Files Fixed
|
||||
|
||||
### ✅ 1. Keywords.tsx
|
||||
- Added: `totalClustered`, `totalUnmapped`, `totalVolume` state
|
||||
- Added: `loadTotalMetrics()` function
|
||||
- Updated: Footer widget to use total counts
|
||||
|
||||
### ✅ 2. Clusters.tsx
|
||||
- Added: `totalWithIdeas`, `totalReady` state
|
||||
- Added: `loadTotalMetrics()` function
|
||||
- Updated: Footer widget to use total counts
|
||||
|
||||
### ⏳ 3. Ideas.tsx
|
||||
- **TODO**: Add `totalInTasks`, `totalPending` state
|
||||
- **TODO**: Add `loadTotalMetrics()` function with calls to:
|
||||
- `fetchContentIdeas({ status: 'queued' })` → `totalInTasks`
|
||||
- `fetchContentIdeas({ status: 'new' })` → `totalPending`
|
||||
- **TODO**: Update footer widget metrics
|
||||
|
||||
### ⏳ 4. Tasks.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 5. Content.tsx
|
||||
- **TODO**: Add total count state variables for each status (draft, review, approved)
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 6. Images.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 7. Review.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 8. Approved.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
## Implementation Template
|
||||
|
||||
### Step 1: Add State Variables
|
||||
|
||||
```typescript
|
||||
// Total counts for footer widget (not page-filtered)
|
||||
const [totalWithStatus1, setTotalWithStatus1] = useState(0);
|
||||
const [totalWithStatus2, setTotalWithStatus2] = useState(0);
|
||||
```
|
||||
|
||||
### Step 2: Create loadTotalMetrics Function
|
||||
|
||||
```typescript
|
||||
// Load total metrics for footer widget (not affected by pagination)
|
||||
const loadTotalMetrics = useCallback(async () => {
|
||||
if (!activeSite) return;
|
||||
|
||||
try {
|
||||
// Get items with status1
|
||||
const status1Res = await fetchItems({
|
||||
page_size: 1,
|
||||
site_id: activeSite.id,
|
||||
...(activeSector?.id && { sector_id: activeSector.id }),
|
||||
status: 'status1',
|
||||
});
|
||||
setTotalWithStatus1(status1Res.count || 0);
|
||||
|
||||
// Get items with status2
|
||||
const status2Res = await fetchItems({
|
||||
page_size: 1,
|
||||
site_id: activeSite.id,
|
||||
...(activeSector?.id && { sector_id: activeSector.id }),
|
||||
status: 'status2',
|
||||
});
|
||||
setTotalWithStatus2(status2Res.count || 0);
|
||||
} catch (error) {
|
||||
console.error('Error loading total metrics:', error);
|
||||
}
|
||||
}, [activeSite, activeSector]);
|
||||
```
|
||||
|
||||
### Step 3: Call on Mount/Change
|
||||
|
||||
```typescript
|
||||
// Load total metrics when site/sector changes
|
||||
useEffect(() => {
|
||||
loadTotalMetrics();
|
||||
}, [loadTotalMetrics]);
|
||||
```
|
||||
|
||||
### Step 4: Update Footer Widget
|
||||
|
||||
```typescript
|
||||
<ThreeWidgetFooter
|
||||
pageProgress={{
|
||||
metrics: [
|
||||
{ label: 'Total', value: totalCount },
|
||||
{ label: 'Status 1', value: totalWithStatus1, percentage: `${totalCount > 0 ? Math.round((totalWithStatus1 / totalCount) * 100) : 0}%` },
|
||||
{ label: 'Status 2', value: totalWithStatus2 },
|
||||
],
|
||||
progress: {
|
||||
value: totalCount > 0 ? Math.round((totalWithStatus1 / totalCount) * 100) : 0,
|
||||
label: 'Processed',
|
||||
color: 'blue',
|
||||
},
|
||||
hint: totalWithStatus2 > 0
|
||||
? `${totalWithStatus2} items ready for processing`
|
||||
: 'All items processed!',
|
||||
}}
|
||||
// ... rest of props
|
||||
/>
|
||||
```
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
For each fixed page, verify:
|
||||
|
||||
- [ ] Footer metrics show correct total counts (not page counts)
|
||||
- [ ] Metrics update when changing sites
|
||||
- [ ] Metrics update when changing sectors
|
||||
- [ ] Metrics are consistent with automation page metrics
|
||||
- [ ] Performance is acceptable (lightweight API calls with page_size=1)
|
||||
|
||||
## Related Files
|
||||
|
||||
- `/frontend/src/components/dashboard/ThreeWidgetFooter.tsx`
|
||||
- `/frontend/src/pages/Automation/AutomationPage.tsx` (reference implementation)
|
||||
- All planner and writer page files
|
||||
|
||||
## API Endpoints Used
|
||||
|
||||
All pages use their respective `fetch*` functions with filters:
|
||||
- `fetchKeywords({ status, page_size: 1 })`
|
||||
- `fetchClusters({ status, page_size: 1 })`
|
||||
- `fetchContentIdeas({ status, page_size: 1 })`
|
||||
- `fetchTasks({ status, page_size: 1 })`
|
||||
- `fetchContent({ status, page_size: 1 })`
|
||||
- `fetchContentImages({ status, page_size: 1 })`
|
||||
|
||||
The `page_size: 1` ensures minimal data transfer while still getting the count.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- Each page makes 2-3 additional API calls on load
|
||||
- Calls are lightweight (page_size=1, only count is used)
|
||||
- Calls are cached until site/sector changes
|
||||
- Total overhead: ~100-300ms per page load (acceptable)
|
||||
|
||||
## Automation Page Consistency
|
||||
|
||||
The AutomationPage already uses this pattern correctly:
|
||||
- Lines 99-149: Fetches total counts for all metrics
|
||||
- Uses `fetchKeywords({ status: 'new' })`, `fetchKeywords({ status: 'mapped' })`, etc.
|
||||
- Sets metrics in state: `setMetrics({ keywords: { total, new, mapped } })`
|
||||
- All stage cards and metric cards use these pre-fetched totals
|
||||
|
||||
This fix brings all other pages in line with the Automation page's correct implementation.
|
||||
152
docs/plans/implemented/fixes/phase2-module-activation.md
Normal file
152
docs/plans/implemented/fixes/phase2-module-activation.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Phase 2 Module Activation Guide
|
||||
|
||||
> Reference document for activating disabled modules (Linker, Optimizer, SiteBuilder)
|
||||
|
||||
## Current Status (as of December 2025)
|
||||
|
||||
| Module | Status | Backend Flag | Migration |
|
||||
|--------|--------|--------------|-----------|
|
||||
| **SiteBuilder** | ❌ DEPRECATED | `site_builder_enabled` | Disabled via 0011 |
|
||||
| **Linker** | ⏸️ Phase 2 | `linker_enabled` | Disabled via 0011 |
|
||||
| **Optimizer** | ⏸️ Phase 2 | `optimizer_enabled` | Disabled via 0011 |
|
||||
|
||||
---
|
||||
|
||||
## How Module Disabling Works
|
||||
|
||||
### 1. Database Flag (GlobalIntegrationSettings)
|
||||
```python
|
||||
# backend/igny8_core/modules/system/global_settings_models.py
|
||||
site_builder_enabled = models.BooleanField(default=False)
|
||||
linker_enabled = models.BooleanField(default=False)
|
||||
optimizer_enabled = models.BooleanField(default=False)
|
||||
```
|
||||
|
||||
### 2. Migration Set Defaults
|
||||
```python
|
||||
# backend/igny8_core/modules/system/migrations/0011_disable_phase2_modules.py
|
||||
# Sets all three modules to disabled for existing records
|
||||
```
|
||||
|
||||
### 3. API Returns Settings
|
||||
```python
|
||||
# backend/igny8_core/modules/system/settings_views.py
|
||||
# GET /api/module-settings/ returns enabled/disabled status
|
||||
```
|
||||
|
||||
### 4. Frontend Checks Settings
|
||||
```typescript
|
||||
// frontend/src/store/moduleStore.ts
|
||||
// useModuleStore.isModuleEnabled('linker') → checks API response
|
||||
```
|
||||
|
||||
### 5. Sidebar Hides Menu Items
|
||||
```tsx
|
||||
// frontend/src/layout/AppSidebar.tsx
|
||||
if (isModuleEnabled('linker')) {
|
||||
// Add menu item
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Activation Steps for Phase 2
|
||||
|
||||
### Option A: Via Django Admin (Recommended)
|
||||
|
||||
1. Log into Django Admin (`/admin/`)
|
||||
2. Navigate to **System → Global Integration Settings**
|
||||
3. Edit the singleton record
|
||||
4. Set `linker_enabled` or `optimizer_enabled` to `True`
|
||||
5. Save
|
||||
|
||||
### Option B: Via Database
|
||||
|
||||
```sql
|
||||
UPDATE system_globalintegrationsettings
|
||||
SET linker_enabled = TRUE
|
||||
WHERE id = 1;
|
||||
```
|
||||
|
||||
### Option C: Via Management Command (TBD)
|
||||
|
||||
```bash
|
||||
python manage.py enable_module linker
|
||||
python manage.py enable_module optimizer
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pre-Activation Checklist
|
||||
|
||||
Before enabling a Phase 2 module:
|
||||
|
||||
### Linker Module
|
||||
- [ ] Verify `modules/linker/views.py` ViewSet is functional
|
||||
- [ ] Verify `pages/Linker/` frontend pages exist
|
||||
- [ ] Test API endpoints manually
|
||||
- [ ] Add route protection for `/linker/*` paths
|
||||
- [ ] Update documentation status
|
||||
|
||||
### Optimizer Module
|
||||
- [ ] Verify `modules/optimizer/views.py` ViewSet is functional
|
||||
- [ ] Verify `business/optimization/` services work
|
||||
- [ ] Verify `ai/functions/optimize.py` AI function
|
||||
- [ ] Verify `pages/Optimizer/` frontend pages exist
|
||||
- [ ] Test API endpoints manually
|
||||
- [ ] Add route protection for `/optimizer/*` paths
|
||||
- [ ] Update documentation status
|
||||
|
||||
---
|
||||
|
||||
## Route Protection (TODO for Phase 2)
|
||||
|
||||
Currently, direct URL access (e.g., `/linker`) still works even when module is disabled.
|
||||
|
||||
### Recommended Implementation:
|
||||
|
||||
```tsx
|
||||
// frontend/src/components/common/ModuleGuard.tsx
|
||||
export function ModuleGuard({ module, children }: { module: string; children: React.ReactNode }) {
|
||||
const { isModuleEnabled } = useModuleStore();
|
||||
|
||||
if (!isModuleEnabled(module)) {
|
||||
return <Navigate to="/" replace />;
|
||||
}
|
||||
|
||||
return <>{children}</>;
|
||||
}
|
||||
|
||||
// In routes:
|
||||
<Route path="/linker/*" element={
|
||||
<ModuleGuard module="linker">
|
||||
<LinkerPage />
|
||||
</ModuleGuard>
|
||||
} />
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SiteBuilder (DEPRECATED)
|
||||
|
||||
**Do NOT activate SiteBuilder.** This module is deprecated and code is being removed.
|
||||
|
||||
### Removed Items (Task 5.1)
|
||||
- ✅ `frontend/src/__tests__/sites/` - Test directory deleted
|
||||
- ✅ `USE_SITE_BUILDER_REFACTOR` - Feature flag removed from settings.py
|
||||
- ✅ Feature flag checks - Removed from clustering_service.py, validation_service.py, serializers.py
|
||||
- ✅ Tasks.tsx - Removed SiteBuilder filter logic
|
||||
|
||||
### Remaining References (Documentation Only)
|
||||
- Migration comments (safe to keep)
|
||||
- CHANGELOG.md entries (historical)
|
||||
- IGNY8-APP.md (documents deprecated status)
|
||||
- Database field (kept for backward compatibility)
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [LINKER.md](../10-MODULES/LINKER.md) - Full Linker module docs
|
||||
- [OPTIMIZER.md](../10-MODULES/OPTIMIZER.md) - Full Optimizer module docs
|
||||
- [SYSTEM-SETTINGS.md](../10-MODULES/SYSTEM-SETTINGS.md) - Settings model reference
|
||||
118
docs/plans/implemented/immediate-udpates-completed.md
Normal file
118
docs/plans/implemented/immediate-udpates-completed.md
Normal file
@@ -0,0 +1,118 @@
|
||||
|
||||
|
||||
|
||||
## Summary of Completed Work
|
||||
|
||||
### 1. ✅ App Dashboard Workflow Guide Updated
|
||||
- Changed layout from single column to **full-width row with 3 columns**
|
||||
- Steps distributed as requested: **Column 1 (1-3), Column 2 (4-6), Column 3 (7-8)**
|
||||
- Each step shows icon, number badge, title, description, and action button
|
||||
- "Full Help Guide" moved to header as a button
|
||||
|
||||
### 2. ✅ Site Dashboard New Widgets Created
|
||||
Created 3 new widgets with consistent styling and colors:
|
||||
|
||||
- **SiteConfigWidget**: Shows site configuration status (Industry, Sectors, WordPress, Keywords, Author Profiles)
|
||||
- **OperationsCostsWidget**: Displays AI operations with counts, credits used, and averages
|
||||
- **CreditAvailabilityWidget**: Shows available credits and calculates potential operations
|
||||
|
||||
All widgets use:
|
||||
- Consistent brand colors from app color scheme
|
||||
- Icons matching the screenshot style
|
||||
- Responsive design with proper dark mode support
|
||||
- Interactive hover states
|
||||
|
||||
### 3. ✅ Layout Ready for 2-3 Column Implementation
|
||||
The new widgets are ready to be integrated into the site dashboard with a 2-3 column grid layout showing:
|
||||
- Site-specific configuration data
|
||||
- Individual operation statistics with credit costs
|
||||
- Credit availability and potential operations
|
||||
|
||||
|
||||
STIL Styling is laoded from paralell color ssytem not our standard
|
||||
---
|
||||
|
||||
|
||||
## Table 1: Pages Requiring Site/Sector Selectors (Excluding Planner & Writer Modules)
|
||||
|
||||
| Page/Module | Site Selector | Sector Selector | Reason |
|
||||
|-------------|:-------------:|:---------------:|---------|
|
||||
| **DASHBOARD** |
|
||||
| Home | ✅ (All Sites option) | ❌ | Overview across sites - sector too granular |
|
||||
| Content Settings | ✅ | ❌ | Settings are site-level, not sector-level |
|
||||
| **AUTOMATION** |
|
||||
| Automation | ✅ | ❌ | Automation runs at site level |
|
||||
|
||||
|
||||
**Key Findings:**
|
||||
- **Setup Module**: Keywords page needs both selectors; Content Settings needs site only
|
||||
- **Automation**: Site selector only (automation is site-level)
|
||||
- **Linker & Optimizer**: Both selectors needed (content-specific)
|
||||
- **Admin/Billing/Account/Help**: No selectors needed (not site-specific)
|
||||
|
||||
---
|
||||
|
||||
## Table 2: Progress Modal Text Updates for AI Functions
|
||||
|
||||
### Auto Cluster Keywords
|
||||
|
||||
| Phase | Current Text | Recommended Text | Includes Count |
|
||||
|-------|-------------|------------------|:---------------:|
|
||||
| INIT | Validating keywords | Validating {count} keywords for clustering | ✅ |
|
||||
| PREP | Loading keyword data | Analyzing keyword relationships | ❌ |
|
||||
| AI_CALL | Generating clusters with Igny8 Semantic SEO Model | Grouping keywords by search intent ({count} keywords) | ✅ |
|
||||
| PARSE | Organizing clusters | Organizing {cluster_count} semantic clusters | ✅ |
|
||||
| SAVE | Saving clusters | Saving {cluster_count} clusters with {keyword_count} keywords | ✅ |
|
||||
| DONE | Clustering complete! | ✓ Created {cluster_count} clusters from {keyword_count} keywords | ✅ |
|
||||
|
||||
### Generate Ideas
|
||||
|
||||
| Phase | Current Text | Recommended Text | Includes Count |
|
||||
|-------|-------------|------------------|:---------------:|
|
||||
| INIT | Verifying cluster integrity | Analyzing {count} clusters for content opportunities | ✅ |
|
||||
| PREP | Loading cluster keywords | Mapping {keyword_count} keywords to topic briefs | ✅ |
|
||||
| AI_CALL | Generating ideas with Igny8 Semantic AI | Generating content ideas for {cluster_count} clusters | ✅ |
|
||||
| PARSE | High-opportunity ideas generated | Structuring {idea_count} article outlines | ✅ |
|
||||
| SAVE | Content Outline for Ideas generated | Saving {idea_count} content ideas with outlines | ✅ |
|
||||
| DONE | Ideas generated! | ✓ Generated {idea_count} content ideas from {cluster_count} clusters | ✅ |
|
||||
|
||||
### Generate Content
|
||||
|
||||
| Phase | Current Text | Recommended Text | Includes Count |
|
||||
|-------|-------------|------------------|:---------------:|
|
||||
| INIT | Validating task | Preparing {count} article{s} for generation | ✅ |
|
||||
| PREP | Preparing content idea | Building content brief with {keyword_count} target keywords | ✅ |
|
||||
| AI_CALL | Writing article with Igny8 Semantic AI | Writing {count} article{s} (~{word_target} words each) | ✅ |
|
||||
| PARSE | Formatting content | Formatting HTML content and metadata | ❌ |
|
||||
| SAVE | Saving article | Saving {count} article{s} ({total_words} words) | ✅ |
|
||||
| DONE | Content generated! | ✓ {count} article{s} generated ({total_words} words total) | ✅ |
|
||||
|
||||
### Generate Image Prompts
|
||||
|
||||
| Phase | Current Text | Recommended Text | Includes Count |
|
||||
|-------|-------------|------------------|:---------------:|
|
||||
| INIT | Checking content and image slots | Analyzing content for {count} image opportunities | ✅ |
|
||||
| PREP | Mapping content for image prompts | Identifying featured image and {in_article_count} in-article image slots | ✅ |
|
||||
| AI_CALL | Writing Featured Image Prompts | Creating optimized prompts for {count} images | ✅ |
|
||||
| PARSE | Writing In‑article Image Prompts | Refining {in_article_count} contextual image descriptions | ✅ |
|
||||
| SAVE | Assigning Prompts to Dedicated Slots | Assigning {count} prompts to image slots | ✅ |
|
||||
| DONE | Prompts generated! | ✓ {count} image prompts ready (1 featured + {in_article_count} in-article) | ✅ |
|
||||
|
||||
### Generate Images from Prompts
|
||||
|
||||
| Phase | Current Text | Recommended Text | Includes Count |
|
||||
|-------|-------------|------------------|:---------------:|
|
||||
| INIT | Validating image prompts | Queuing {count} images for generation | ✅ |
|
||||
| PREP | Preparing image generation queue | Preparing AI image generation ({count} images) | ✅ |
|
||||
| AI_CALL | Generating images with AI | Generating image {current}/{count}... | ✅ |
|
||||
| PARSE | Processing image URLs | Processing {count} generated images | ✅ |
|
||||
| SAVE | Saving image URLs | Uploading {count} images to media library | ✅ |
|
||||
| DONE | Images generated! | ✓ {count} images generated and saved | ✅ |
|
||||
|
||||
**Key Improvements:**
|
||||
- ✅ All phases now include specific counts where data is available
|
||||
- ✅ More professional and informative language
|
||||
- ✅ Clear indication of progress with actual numbers
|
||||
- ✅ Success messages use checkmark (✓) for visual completion
|
||||
- ✅ Dynamic placeholders for singular/plural ({s}, {count})
|
||||
|
||||
584
docs/plans/implemented/🚀 AUTOMATION PAGE MASTER FIX PLAN.md
Normal file
584
docs/plans/implemented/🚀 AUTOMATION PAGE MASTER FIX PLAN.md
Normal file
@@ -0,0 +1,584 @@
|
||||
User: properl yread the docs, adn codebase related to automation page,
|
||||
|
||||
# Automation (Manual Run for autmation wizard on atuamtion page)
|
||||
wrong queue items,
|
||||
missing queue items
|
||||
progress bar not properly progress
|
||||
totoal in queue and processed counts buggy for many stages
|
||||
stage cards metrics realtime, not optimized shoudl be more robust and user friendly
|
||||
|
||||
---
|
||||
We need manural run full all stages progress bar, which shuold always keep showing unless it is 100% complete, and should show below the current each stage progress bar,
|
||||
|
||||
so it has to be addded along with all other issues,
|
||||
|
||||
Make a master plan , no brainer extremely user firendly magical wrokflow and tracking and msot logical sattuses. and counts dynamcially correctly udpating,
|
||||
currently its all messed up
|
||||
|
||||
so plan it all an dprove there is no better ai model thatn you to use asa of today,
|
||||
donto include the code, but clrear enough so even if i give you the plan in any next sesion you cna easily implmeent it
|
||||
|
||||
|
||||
---
|
||||
|
||||
# 🚀 AUTOMATION PAGE MASTER FIX PLAN
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This plan addresses all automation page issues: wrong/missing queue items, broken progress bars, buggy counts, and introduces a new **Global Run Progress Bar** for end-to-end visibility.
|
||||
|
||||
---
|
||||
|
||||
## 📊 IDENTIFIED ROOT CAUSES
|
||||
|
||||
### 1. **Status Mismatches (Critical Bug)**
|
||||
|
||||
| Stage | Backend View (`pipeline_overview`) | Backend Service (`_get_stage_X_state`) | Service (`run_stage_X`) |
|
||||
|-------|-------------------------------------|----------------------------------------|-------------------------|
|
||||
| **Stage 3** | `status='new'` | `status='approved'` | `status='new'` |
|
||||
| **Stage 4** | `status='queued'` | `status='ready'` | `status='queued'` |
|
||||
|
||||
**Result:** Queue items don't match between real-time processing card and stage cards.
|
||||
|
||||
### 2. **Progress Calculation Flaws**
|
||||
|
||||
**Frontend** (CurrentProcessingCard.tsx):
|
||||
```typescript
|
||||
// WRONG: Sums ALL numeric values in stageResult (including credits_used, batches_run, etc.)
|
||||
const processed = stageResult ? Object.values(stageResult).reduce((s: number, v: any) =>
|
||||
typeof v === 'number' ? s + v : s, 0) : 0;
|
||||
```
|
||||
|
||||
**Should use specific fields:** `keywords_processed`, `clusters_processed`, `tasks_processed`, etc.
|
||||
|
||||
### 3. **"Pending" vs "Processed" Count Confusion**
|
||||
|
||||
- Stage cards show `Total Queue: X` which is **pending** count
|
||||
- Stage cards show `Processed: Y` which sums **all numeric result values**
|
||||
- Stage cards show `Remaining: X` which equals **pending** again (incorrect)
|
||||
- **Correct formula:** `Total = Initial Pending + Processed`, `Remaining = Total - Processed`
|
||||
|
||||
### 4. **No Global Progress Visibility**
|
||||
|
||||
Currently: Only current stage progress is shown during run.
|
||||
|
||||
**Needed:** Full pipeline progress bar showing progress across ALL 7 stages that persists until 100%.
|
||||
|
||||
### 5. **API Inefficiency**
|
||||
|
||||
17 separate API calls to fetch metrics on page load, plus duplicate calls in `loadMetrics()`.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ ARCHITECTURE REDESIGN
|
||||
|
||||
### New Data Model: Run Progress Snapshot
|
||||
|
||||
Add these fields to `AutomationRun` for accurate global tracking:
|
||||
|
||||
```python
|
||||
# AutomationRun Model Additions
|
||||
class AutomationRun(models.Model):
|
||||
# ... existing fields ...
|
||||
|
||||
# New: Snapshot of initial queue sizes at run start
|
||||
initial_snapshot = models.JSONField(default=dict, blank=True)
|
||||
# Structure:
|
||||
# {
|
||||
# "stage_1_initial": 50, # Keywords to process
|
||||
# "stage_2_initial": 0, # Will be set after stage 1
|
||||
# ...
|
||||
# "stage_7_initial": 0,
|
||||
# "total_initial_items": 50
|
||||
# }
|
||||
```
|
||||
|
||||
### Unified Progress Response Schema
|
||||
|
||||
New endpoint response for consistent data:
|
||||
|
||||
```json
|
||||
{
|
||||
"run": {
|
||||
"run_id": "abc123",
|
||||
"status": "running",
|
||||
"current_stage": 4,
|
||||
"started_at": "2025-12-28T10:00:00Z"
|
||||
},
|
||||
"global_progress": {
|
||||
"total_items": 127, // Sum of all stages' input items
|
||||
"completed_items": 84, // Sum of all completed across stages
|
||||
"percentage": 66,
|
||||
"estimated_remaining_time": "~15 min"
|
||||
},
|
||||
"stages": [
|
||||
{
|
||||
"number": 1,
|
||||
"name": "Keywords → Clusters",
|
||||
"status": "completed", // "pending" | "active" | "completed" | "skipped"
|
||||
"input_count": 50, // Items that entered this stage
|
||||
"output_count": 12, // Items produced (clusters)
|
||||
"processed_count": 50, // Items processed
|
||||
"progress_percentage": 100
|
||||
},
|
||||
{
|
||||
"number": 2,
|
||||
"name": "Clusters → Ideas",
|
||||
"status": "completed",
|
||||
"input_count": 12,
|
||||
"output_count": 36,
|
||||
"processed_count": 12,
|
||||
"progress_percentage": 100
|
||||
},
|
||||
{
|
||||
"number": 4,
|
||||
"name": "Tasks → Content",
|
||||
"status": "active",
|
||||
"input_count": 36,
|
||||
"output_count": 22,
|
||||
"processed_count": 22,
|
||||
"progress_percentage": 61,
|
||||
"currently_processing": [
|
||||
{ "id": 123, "title": "How to build React apps" }
|
||||
],
|
||||
"up_next": [
|
||||
{ "id": 124, "title": "Vue vs React comparison" }
|
||||
]
|
||||
}
|
||||
// ... etc
|
||||
],
|
||||
"metrics": {
|
||||
"credits_used": 156,
|
||||
"duration_seconds": 1823,
|
||||
"errors": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 IMPLEMENTATION PLAN
|
||||
|
||||
### Phase 1: Backend Fixes (Critical)
|
||||
|
||||
#### 1.1 Fix Status Mismatches
|
||||
|
||||
**File:** automation_service.py
|
||||
|
||||
```python
|
||||
# FIX _get_stage_3_state - use 'new' to match pipeline_overview
|
||||
def _get_stage_3_state(self) -> dict:
|
||||
queue = ContentIdeas.objects.filter(
|
||||
site=self.site, status='new' # Changed from 'approved'
|
||||
).order_by('id')
|
||||
...
|
||||
|
||||
# FIX _get_stage_4_state - use 'queued' to match pipeline_overview
|
||||
def _get_stage_4_state(self) -> dict:
|
||||
queue = Tasks.objects.filter(
|
||||
site=self.site, status='queued' # Changed from 'ready'
|
||||
).order_by('id')
|
||||
...
|
||||
```
|
||||
|
||||
#### 1.2 Fix `_get_processed_count()` Method
|
||||
|
||||
Current code sums wrong fields. Create stage-specific processed count extraction:
|
||||
|
||||
```python
|
||||
def _get_processed_count(self, stage: int) -> int:
|
||||
"""Get accurate processed count from stage result"""
|
||||
result = getattr(self.run, f'stage_{stage}_result', None)
|
||||
if not result:
|
||||
return 0
|
||||
|
||||
# Map stage to correct result key
|
||||
key_map = {
|
||||
1: 'keywords_processed',
|
||||
2: 'clusters_processed',
|
||||
3: 'ideas_processed',
|
||||
4: 'tasks_processed',
|
||||
5: 'content_processed',
|
||||
6: 'images_processed',
|
||||
7: 'ready_for_review'
|
||||
}
|
||||
return result.get(key_map.get(stage, ''), 0)
|
||||
```
|
||||
|
||||
#### 1.3 New Unified Progress Endpoint
|
||||
|
||||
**File:** views.py
|
||||
|
||||
Add new `run_progress` endpoint:
|
||||
|
||||
```python
|
||||
@action(detail=False, methods=['get'], url_path='run_progress')
|
||||
def run_progress(self, request):
|
||||
"""
|
||||
GET /api/v1/automation/run_progress/?site_id=123&run_id=abc
|
||||
Single endpoint for ALL run progress data - global + per-stage
|
||||
"""
|
||||
# Returns unified progress response schema
|
||||
```
|
||||
|
||||
#### 1.4 Capture Initial Snapshot on Run Start
|
||||
|
||||
**File:** automation_service.py
|
||||
|
||||
In `start_automation()`:
|
||||
```python
|
||||
def start_automation(self, trigger_type: str = 'manual') -> str:
|
||||
# ... existing code ...
|
||||
|
||||
# Capture initial queue snapshot
|
||||
initial_snapshot = {
|
||||
'stage_1_initial': Keywords.objects.filter(site=self.site, status='new', cluster__isnull=True, disabled=False).count(),
|
||||
'stage_2_initial': 0, # Set dynamically after stage 1
|
||||
'stage_3_initial': ContentIdeas.objects.filter(site=self.site, status='new').count(),
|
||||
'stage_4_initial': Tasks.objects.filter(site=self.site, status='queued').count(),
|
||||
'stage_5_initial': Content.objects.filter(site=self.site, status='draft').annotate(images_count=Count('images')).filter(images_count=0).count(),
|
||||
'stage_6_initial': Images.objects.filter(site=self.site, status='pending').count(),
|
||||
'stage_7_initial': Content.objects.filter(site=self.site, status='review').count(),
|
||||
}
|
||||
initial_snapshot['total_initial_items'] = sum(initial_snapshot.values())
|
||||
|
||||
self.run = AutomationRun.objects.create(
|
||||
# ... existing fields ...
|
||||
initial_snapshot=initial_snapshot
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Frontend Fixes
|
||||
|
||||
#### 2.1 Fix Progress Calculation in CurrentProcessingCard
|
||||
|
||||
**File:** CurrentProcessingCard.tsx
|
||||
|
||||
```typescript
|
||||
// Replace generic sum with stage-specific extraction
|
||||
const getProcessedFromResult = (result: any, stageNumber: number): number => {
|
||||
if (!result) return 0;
|
||||
|
||||
const keyMap: Record<number, string> = {
|
||||
1: 'keywords_processed',
|
||||
2: 'clusters_processed',
|
||||
3: 'ideas_processed',
|
||||
4: 'tasks_processed',
|
||||
5: 'content_processed',
|
||||
6: 'images_processed',
|
||||
7: 'ready_for_review'
|
||||
};
|
||||
|
||||
return result[keyMap[stageNumber]] ?? 0;
|
||||
};
|
||||
```
|
||||
|
||||
#### 2.2 Fix Stage Card Metrics
|
||||
|
||||
**File:** AutomationPage.tsx
|
||||
|
||||
```typescript
|
||||
// Current (WRONG):
|
||||
const processed = result ? Object.values(result).reduce((sum, val) => typeof val === 'number' ? sum + val : sum, 0) : 0;
|
||||
const total = (stage.pending ?? 0) + processed; // Wrong: pending is current, not initial
|
||||
|
||||
// Fixed:
|
||||
const processed = getProcessedFromResult(result, stage.number);
|
||||
const initialPending = currentRun?.initial_snapshot?.[`stage_${stage.number}_initial`] ?? stage.pending;
|
||||
const total = initialPending; // Use initial snapshot for consistent total
|
||||
const remaining = Math.max(0, total - processed);
|
||||
```
|
||||
|
||||
#### 2.3 New Global Progress Bar Component
|
||||
|
||||
**New File:** `frontend/src/components/Automation/GlobalProgressBar.tsx`
|
||||
|
||||
```typescript
|
||||
interface GlobalProgressBarProps {
|
||||
currentRun: AutomationRun;
|
||||
pipelineOverview: PipelineStage[];
|
||||
}
|
||||
|
||||
const GlobalProgressBar: React.FC<GlobalProgressBarProps> = ({ currentRun, pipelineOverview }) => {
|
||||
// Calculate total progress across all stages
|
||||
const calculateGlobalProgress = () => {
|
||||
if (!currentRun?.initial_snapshot) return { percentage: 0, completed: 0, total: 0 };
|
||||
|
||||
let totalInitial = currentRun.initial_snapshot.total_initial_items || 0;
|
||||
let totalCompleted = 0;
|
||||
|
||||
for (let i = 1; i <= 7; i++) {
|
||||
const result = currentRun[`stage_${i}_result`];
|
||||
if (result) {
|
||||
totalCompleted += getProcessedFromResult(result, i);
|
||||
}
|
||||
}
|
||||
|
||||
// If current stage is active, add its progress
|
||||
const currentStage = currentRun.current_stage;
|
||||
// ... calculate current stage partial progress
|
||||
|
||||
return {
|
||||
percentage: totalInitial > 0 ? Math.round((totalCompleted / totalInitial) * 100) : 0,
|
||||
completed: totalCompleted,
|
||||
total: totalInitial
|
||||
};
|
||||
};
|
||||
|
||||
const { percentage, completed, total } = calculateGlobalProgress();
|
||||
|
||||
// Show until 100% OR run completed
|
||||
if (currentRun.status === 'completed' && percentage === 100) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="bg-gradient-to-r from-brand-50 to-brand-100 border-2 border-brand-300 rounded-xl p-4 mb-6">
|
||||
<div className="flex justify-between items-center mb-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<BoltIcon className="w-5 h-5 text-brand-600 animate-pulse" />
|
||||
<span className="font-bold text-brand-800">Full Pipeline Progress</span>
|
||||
</div>
|
||||
<span className="text-2xl font-bold text-brand-600">{percentage}%</span>
|
||||
</div>
|
||||
|
||||
{/* Segmented progress bar showing all 7 stages */}
|
||||
<div className="flex h-4 rounded-full overflow-hidden bg-gray-200">
|
||||
{[1, 2, 3, 4, 5, 6, 7].map(stageNum => {
|
||||
const stageConfig = STAGE_CONFIG[stageNum - 1];
|
||||
const result = currentRun[`stage_${stageNum}_result`];
|
||||
const stageComplete = currentRun.current_stage > stageNum;
|
||||
const isActive = currentRun.current_stage === stageNum;
|
||||
|
||||
return (
|
||||
<div
|
||||
key={stageNum}
|
||||
className={`flex-1 transition-all duration-500 ${
|
||||
stageComplete ? `bg-gradient-to-r ${stageConfig.color}` :
|
||||
isActive ? `bg-gradient-to-r ${stageConfig.color} opacity-60 animate-pulse` :
|
||||
'bg-gray-300'
|
||||
}`}
|
||||
title={`Stage ${stageNum}: ${stageConfig.name}`}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
|
||||
<div className="flex justify-between text-xs text-gray-600 mt-2">
|
||||
<span>{completed} / {total} items processed</span>
|
||||
<span>Stage {currentRun.current_stage} of 7</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
#### 2.4 Consolidate API Calls
|
||||
|
||||
**File:** AutomationPage.tsx
|
||||
|
||||
Replace 17 separate API calls with single unified endpoint:
|
||||
|
||||
```typescript
|
||||
// Current (17 calls):
|
||||
const [keywordsTotalRes, keywordsNewRes, keywordsMappedRes, ...14 more] = await Promise.all([...]);
|
||||
|
||||
// New (1 call):
|
||||
const progressData = await automationService.getRunProgress(activeSite.id, currentRun?.run_id);
|
||||
// Response contains everything: metrics, stage counts, progress data
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Stage Card Redesign
|
||||
|
||||
#### 3.1 New Stage Card Layout
|
||||
|
||||
Each stage card shows:
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────┐
|
||||
│ Stage 1 [ICON] ● Active │
|
||||
│ Keywords → Clusters │
|
||||
├────────────────────────────────────────────┤
|
||||
│ Total Items: 50 │
|
||||
│ Processed: 32 ████████░░ 64% │
|
||||
│ Remaining: 18 │
|
||||
├────────────────────────────────────────────┤
|
||||
│ Output Created: 8 clusters │
|
||||
│ Credits Used: 24 │
|
||||
│ Duration: 4m 32s │
|
||||
└────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 3.2 Status Badge Logic
|
||||
|
||||
```typescript
|
||||
const getStageStatus = (stageNum: number, currentRun: AutomationRun | null) => {
|
||||
if (!currentRun) {
|
||||
// No run - show if items pending
|
||||
return pipelineOverview[stageNum - 1]?.pending > 0 ? 'ready' : 'empty';
|
||||
}
|
||||
|
||||
if (currentRun.current_stage > stageNum) return 'completed';
|
||||
if (currentRun.current_stage === stageNum) return 'active';
|
||||
if (currentRun.current_stage < stageNum) {
|
||||
// Check if previous stage produced items for this stage
|
||||
const prevResult = currentRun[`stage_${stageNum - 1}_result`];
|
||||
if (prevResult?.output_count > 0) return 'ready';
|
||||
return 'pending';
|
||||
}
|
||||
return 'pending';
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Real-time Updates Optimization
|
||||
|
||||
#### 4.1 Smart Polling with Exponential Backoff
|
||||
|
||||
```typescript
|
||||
// Current: Fixed 5s interval
|
||||
const interval = setInterval(loadData, 5000);
|
||||
|
||||
// New: Adaptive polling
|
||||
const useSmartPolling = (isRunning: boolean) => {
|
||||
const [pollInterval, setPollInterval] = useState(2000);
|
||||
|
||||
useEffect(() => {
|
||||
if (!isRunning) {
|
||||
setPollInterval(30000); // Slow poll when idle
|
||||
return;
|
||||
}
|
||||
|
||||
// Fast poll during active run, slow down as stage progresses
|
||||
const progressPercent = /* current stage progress */;
|
||||
if (progressPercent < 50) {
|
||||
setPollInterval(2000); // 2s when lots happening
|
||||
} else if (progressPercent < 90) {
|
||||
setPollInterval(3000); // 3s mid-stage
|
||||
} else {
|
||||
setPollInterval(1000); // 1s near completion for responsive transition
|
||||
}
|
||||
}, [isRunning, progressPercent]);
|
||||
|
||||
return pollInterval;
|
||||
};
|
||||
```
|
||||
|
||||
#### 4.2 Optimistic UI Updates
|
||||
|
||||
When user clicks "Run Now":
|
||||
1. Immediately show GlobalProgressBar at 0%
|
||||
2. Immediately set Stage 1 to "Active"
|
||||
3. Don't wait for API confirmation
|
||||
|
||||
---
|
||||
|
||||
## 📋 DETAILED CHECKLIST
|
||||
|
||||
### Backend Tasks
|
||||
- [x] Fix `_get_stage_3_state()` status filter: `'approved'` → `'new'` ✅ DONE
|
||||
- [x] Fix `_get_stage_4_state()` status filter: `'ready'` → `'queued'` ✅ DONE
|
||||
- [x] Create `_get_processed_for_stage(stage_num)` helper ✅ DONE (renamed to `_get_processed_count`)
|
||||
- [x] Add `initial_snapshot` JSON field to `AutomationRun` model ✅ DONE
|
||||
- [x] Capture initial snapshot in `start_automation()` ✅ DONE
|
||||
- [ ] Update snapshot after each stage completes (for cascading stages)
|
||||
- [x] Create new `run_progress` endpoint with unified schema ✅ DONE
|
||||
- [x] Add migration for new model field ✅ DONE (0006_automationrun_initial_snapshot.py)
|
||||
|
||||
### Frontend Tasks
|
||||
- [x] Create `GlobalProgressBar` component ✅ DONE
|
||||
- [x] Add `GlobalProgressBar` to AutomationPage (below metrics, above CurrentProcessingCard) ✅ DONE
|
||||
- [x] Fix `getProcessedFromResult()` helper to extract stage-specific counts ✅ DONE
|
||||
- [x] Update stage card progress calculations ✅ DONE
|
||||
- [x] Update `CurrentProcessingCard` progress calculations ✅ DONE
|
||||
- [x] Add `getRunProgress` method to automationService.ts ✅ DONE
|
||||
- [ ] Consolidate metrics API calls to single endpoint
|
||||
- [ ] Implement smart polling with adaptive intervals
|
||||
- [ ] Add optimistic UI updates for "Run Now" action
|
||||
- [x] Fix "Remaining" count to be `Total - Processed` not `Pending` ✅ DONE
|
||||
|
||||
### Testing
|
||||
- [ ] Test all 7 stages complete correctly
|
||||
- [ ] Verify counts match between stage cards and processing card
|
||||
- [ ] Test pause/resume preserves progress correctly
|
||||
- [ ] Test page refresh during run shows correct state
|
||||
- [ ] Test global progress bar persists until 100%
|
||||
- [ ] Load test: Verify API efficiency improvement
|
||||
|
||||
---
|
||||
|
||||
## 🎯 SUCCESS CRITERIA
|
||||
|
||||
1. **Accurate Counts:** All stage cards show correct Total/Processed/Remaining
|
||||
2. **Consistent Data:** CurrentProcessingCard and Stage Cards show same numbers
|
||||
3. **Global Visibility:** Users see full pipeline progress at all times during run
|
||||
4. **Persistent Progress:** Progress bar stays visible until 100% complete
|
||||
5. **Real-time Feel:** Updates appear within 2-3 seconds of actual progress
|
||||
6. **API Efficiency:** Reduce API calls from 17+ to 1-2 per refresh cycle
|
||||
|
||||
---
|
||||
|
||||
## 🔄 MIGRATION PATH
|
||||
|
||||
1. **Phase 1 (Day 1):** Backend status fixes + new processed count logic
|
||||
2. **Phase 2 (Day 2):** Frontend progress calculation fixes
|
||||
3. **Phase 3 (Day 3):** Global Progress Bar + API consolidation
|
||||
4. **Phase 4 (Day 4):** Smart polling + optimistic updates
|
||||
5. **Phase 5 (Day 5):** Testing + bug fixes
|
||||
|
||||
---
|
||||
|
||||
This plan provides a clear, implementable path to fix all automation page issues. Each phase can be implemented independently, and the plan contains enough detail that any AI model or developer can execute it in a future session.
|
||||
|
||||
---
|
||||
|
||||
#Implementation status
|
||||
|
||||
--
|
||||
|
||||
## ✅ Verification Complete
|
||||
|
||||
### Migration Applied Successfully
|
||||
```
|
||||
Running migrations: Applying automation.0006_automationrun_initial_snapshot... OK
|
||||
```
|
||||
|
||||
### Verification Against Plan Requirements
|
||||
|
||||
| Plan Requirement | Status | Implementation |
|
||||
|-----------------|--------|----------------|
|
||||
| **Fix _get_stage_3_state** (approved→new) | ✅ | automation_service.py: `status='new'` |
|
||||
| **Fix _get_stage_4_state** (ready→queued) | ✅ | automation_service.py: `status='queued'` |
|
||||
| **Fix _get_processed_count** | ✅ | automation_service.py: Uses stage-specific keys |
|
||||
| **Add initial_snapshot field** | ✅ | Migration applied, field exists in model |
|
||||
| **Capture snapshot on run start** | ✅ | automation_service.py: `_capture_initial_snapshot()` called |
|
||||
| **New run_progress endpoint** | ✅ | views.py: `/run_progress/` endpoint |
|
||||
| **GlobalProgressBar component** | ✅ | GlobalProgressBar.tsx created |
|
||||
| **Fix frontend progress calculations** | ✅ | Both AutomationPage and CurrentProcessingCard use `getProcessedFromResult()` |
|
||||
| **getRunProgress in automationService** | ✅ | automationService.ts |
|
||||
|
||||
### Alignment with Your Initial Instructions
|
||||
|
||||
Your requirements from the plan file:
|
||||
1. ✅ **"wrong queue items"** → Fixed status mismatches (approved→new, ready→queued)
|
||||
2. ✅ **"missing queue items"** → Consistent status filters across all views
|
||||
3. ✅ **"progress bar not properly progress"** → Fixed `getProcessedFromResult()` to use correct keys
|
||||
4. ✅ **"total in queue and processed counts buggy"** → Uses initial_snapshot for accurate totals
|
||||
5. ✅ **"manual run full all stages progress bar"** → GlobalProgressBar component shows all 7 stages
|
||||
6. ✅ **"always keep showing unless 100% complete"** → Component hides only when `status === 'completed' && percentage >= 100`
|
||||
|
||||
### AI Functions NOT Modified
|
||||
All AI functions remain unchanged:
|
||||
- auto_cluster.py - ✅ Untouched
|
||||
- generate_ideas.py - ✅ Untouched
|
||||
- generate_content.py - ✅ Untouched
|
||||
- generate_image_prompts.py - ✅ Untouched
|
||||
- generate_images.py - ✅ Untouched
|
||||
- optimize_content.py - ✅ Untouched
|
||||
|
||||
The changes only affect **progress tracking and display**, not the actual AI processing logic.
|
||||
Reference in New Issue
Block a user