726 lines
20 KiB
Markdown
726 lines
20 KiB
Markdown
# Automation Progress UX Improvement Plan
|
|
|
|
**Date:** December 4, 2025
|
|
**Status:** Design Phase
|
|
**Priority:** MEDIUM
|
|
|
|
---
|
|
|
|
## 🎯 OBJECTIVE
|
|
|
|
Improve the automation progress tracking UX to show **real-time processing status** for currently processing items, making it easier for users to understand what's happening during automation runs.
|
|
|
|
---
|
|
|
|
## 🔍 CURRENT STATE ANALYSIS
|
|
|
|
### Current Behavior
|
|
|
|
**What Users See Now:**
|
|
1. A "Current State" card that shows the stage being processed
|
|
2. Stage number and status (e.g., "Stage 3: Ideas → Tasks")
|
|
3. **BUT:** No visibility into which specific records are being processed
|
|
4. **Problem:** User only knows when a full stage completes
|
|
|
|
**Example Current Experience:**
|
|
```
|
|
┌─────────────────────────────────────┐
|
|
│ Current State: Stage 2 │
|
|
│ Clusters → Ideas (AI) │
|
|
│ │
|
|
│ Status: Processing │
|
|
└─────────────────────────────────────┘
|
|
|
|
[User waits... no updates until stage completes]
|
|
```
|
|
|
|
### User Pain Points
|
|
|
|
1. ❌ **No Record-Level Progress:** Can't see which keywords/ideas/content are being processed
|
|
2. ❌ **No Queue Visibility:** Don't know what's coming up next
|
|
3. ❌ **No Item Count Progress:** "Processing 15 of 50 keywords..." is missing
|
|
4. ❌ **Card Position:** Current state card is at bottom, requires scrolling
|
|
5. ❌ **No Percentage Progress:** Just a spinner, no quantitative feedback
|
|
|
|
---
|
|
|
|
## ✅ PROPOSED SOLUTION
|
|
|
|
### New Design Concept
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ 🔄 AUTOMATION IN PROGRESS │
|
|
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67% │
|
|
│ │
|
|
│ Stage 2: Clusters → Ideas (AI) │
|
|
│Column 1 │
|
|
│ Currently Processing: │
|
|
│ • "Best SEO tools for small business" (Cluster #42) │
|
|
│ Column 2 │
|
|
│ Up Next: │
|
|
│ • "Content marketing automation platforms" │
|
|
│ • "AI-powered content creation tools" │
|
|
│ Sinngle row centered │
|
|
│ Progress: 34/50 clusters processed │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
[STAGES SECTION BELOW - All 7 stages in grid view]
|
|
```
|
|
|
|
---
|
|
|
|
## 📐 DETAILED DESIGN SPECIFICATIONS
|
|
|
|
### 1. Card Repositioning
|
|
|
|
**Move from:** Bottom of page (after stages)
|
|
**Move to:** Top of page (above stages section)
|
|
**Layout:** Max width 1200px horizontal card
|
|
**Visibility:** Only shown when `currentRun?.status === 'running'`
|
|
|
|
### 2. Card Structure
|
|
|
|
#### Header Section
|
|
- **Left:** Large stage number icon (animated pulse)
|
|
- **Center:** Stage name + type badge (AI/Local/Manual)
|
|
- **Right:** Percentage complete (calculated from processed/total)
|
|
|
|
#### Progress Bar
|
|
- **Type:** Animated linear progress bar
|
|
- **Colors:**
|
|
- Blue for active stage
|
|
- Green for completed
|
|
- Gray for pending
|
|
- **Updates:** Refresh every 3-5 seconds via polling
|
|
|
|
#### Currently Processing Section
|
|
- **For Keywords Stage:**
|
|
```
|
|
Currently Processing:
|
|
• "keyword 1"
|
|
• "keyword 2"
|
|
• "keyword 3"
|
|
|
|
+ 47 more in queue
|
|
```
|
|
|
|
- **For Ideas Stage:**
|
|
```
|
|
Currently Processing:
|
|
• "10 Ways to Improve SEO Rankings"
|
|
|
|
Up Next:
|
|
• "Content Marketing Best Practices 2025"
|
|
• "AI Tools for Content Writers"
|
|
```
|
|
|
|
- **For Content Stage:**
|
|
```
|
|
Currently Processing:
|
|
• "How to Use ChatGPT for Content Creation" (2,500 words)
|
|
|
|
Up Next:
|
|
• "Best AI Image Generators in 2025"
|
|
```
|
|
|
|
#### Record Counter
|
|
```
|
|
Progress: [current]/[total] [items] processed
|
|
Example: Progress: 15/50 keywords processed
|
|
```
|
|
|
|
### 3. Refresh Strategy
|
|
|
|
**Polling Approach:**
|
|
```typescript
|
|
// Poll every 3 seconds while automation is running
|
|
useEffect(() => {
|
|
if (currentRun?.status === 'running') {
|
|
const interval = setInterval(() => {
|
|
// Refresh ONLY the current processing data
|
|
fetchCurrentProcessingState();
|
|
}, 3000);
|
|
|
|
return () => clearInterval(interval);
|
|
}
|
|
}, [currentRun]);
|
|
```
|
|
|
|
**Partial Refresh:**
|
|
- Only refresh the "Currently Processing" component
|
|
- Don't reload entire page
|
|
- Don't re-fetch stage cards
|
|
- Smooth transition (no flickering)
|
|
|
|
---
|
|
|
|
## 🗄️ BACKEND CHANGES REQUIRED
|
|
|
|
### New API Endpoint
|
|
|
|
**URL:** `GET /api/automation/current_processing/`
|
|
**Params:** `?site_id={id}&run_id={run_id}`
|
|
|
|
**Response Format:**
|
|
```json
|
|
{
|
|
"run_id": "abc123",
|
|
"current_stage": 2,
|
|
"stage_name": "Clusters → Ideas",
|
|
"stage_type": "AI",
|
|
"total_items": 50,
|
|
"processed_items": 34,
|
|
"percentage": 68,
|
|
"currently_processing": [
|
|
{
|
|
"id": 42,
|
|
"title": "Best SEO tools for small business",
|
|
"type": "cluster"
|
|
}
|
|
],
|
|
"up_next": [
|
|
{
|
|
"id": 43,
|
|
"title": "Content marketing automation platforms",
|
|
"type": "cluster"
|
|
},
|
|
{
|
|
"id": 44,
|
|
"title": "AI-powered content creation tools",
|
|
"type": "cluster"
|
|
}
|
|
],
|
|
"remaining_count": 16
|
|
}
|
|
```
|
|
|
|
### Implementation in AutomationService
|
|
|
|
**File:** `backend/igny8_core/business/automation/services/automation_service.py`
|
|
|
|
**Add method:**
|
|
```python
|
|
def get_current_processing_state(self) -> dict:
|
|
"""
|
|
Get real-time processing state for current automation run
|
|
"""
|
|
if not self.run or self.run.status != 'running':
|
|
return None
|
|
|
|
stage = self.run.current_stage
|
|
|
|
# Get stage-specific data
|
|
if stage == 1: # Keywords → Clusters
|
|
queue = Keywords.objects.filter(
|
|
site=self.site, status='new'
|
|
).order_by('id')
|
|
|
|
return {
|
|
'stage_number': 1,
|
|
'stage_name': 'Keywords → Clusters',
|
|
'stage_type': 'AI',
|
|
'total_items': queue.count() + self._get_processed_count(stage),
|
|
'processed_items': self._get_processed_count(stage),
|
|
'currently_processing': self._get_current_items(queue, 3),
|
|
'up_next': self._get_next_items(queue, 2, skip=3),
|
|
}
|
|
|
|
elif stage == 2: # Clusters → Ideas
|
|
queue = Clusters.objects.filter(
|
|
site=self.site, status='new', disabled=False
|
|
).order_by('id')
|
|
|
|
return {
|
|
'stage_number': 2,
|
|
'stage_name': 'Clusters → Ideas',
|
|
'stage_type': 'AI',
|
|
'total_items': queue.count() + self._get_processed_count(stage),
|
|
'processed_items': self._get_processed_count(stage),
|
|
'currently_processing': self._get_current_items(queue, 1),
|
|
'up_next': self._get_next_items(queue, 2, skip=1),
|
|
}
|
|
|
|
# ... similar for stages 3-6
|
|
|
|
def _get_processed_count(self, stage: int) -> int:
|
|
"""Get count of items processed in current stage"""
|
|
result_key = f'stage_{stage}_result'
|
|
result = getattr(self.run, result_key, {})
|
|
|
|
# Extract appropriate count from result
|
|
if stage == 1:
|
|
return result.get('keywords_processed', 0)
|
|
elif stage == 2:
|
|
return result.get('clusters_processed', 0)
|
|
# ... etc
|
|
|
|
def _get_current_items(self, queryset, count: int) -> list:
|
|
"""Get currently processing items"""
|
|
items = queryset[:count]
|
|
return [
|
|
{
|
|
'id': item.id,
|
|
'title': getattr(item, 'keyword', None) or
|
|
getattr(item, 'cluster_name', None) or
|
|
getattr(item, 'idea_title', None) or
|
|
getattr(item, 'title', None),
|
|
'type': queryset.model.__name__.lower()
|
|
}
|
|
for item in items
|
|
]
|
|
```
|
|
|
|
### Add View in AutomationViewSet
|
|
|
|
**File:** `backend/igny8_core/business/automation/views.py`
|
|
|
|
```python
|
|
@action(detail=False, methods=['get'], url_path='current_processing')
|
|
def current_processing(self, request):
|
|
"""Get current processing state for active automation run"""
|
|
site_id = request.GET.get('site_id')
|
|
run_id = request.GET.get('run_id')
|
|
|
|
if not site_id or not run_id:
|
|
return error_response(
|
|
error='site_id and run_id required',
|
|
status_code=400,
|
|
request=request
|
|
)
|
|
|
|
try:
|
|
run = AutomationRun.objects.get(run_id=run_id, site_id=site_id)
|
|
|
|
if run.status != 'running':
|
|
return success_response(data=None, request=request)
|
|
|
|
service = AutomationService.from_run_id(run_id)
|
|
state = service.get_current_processing_state()
|
|
|
|
return success_response(data=state, request=request)
|
|
|
|
except AutomationRun.DoesNotExist:
|
|
return error_response(
|
|
error='Run not found',
|
|
status_code=404,
|
|
request=request
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## 🎨 FRONTEND CHANGES REQUIRED
|
|
|
|
### 1. New Component: CurrentProcessingCard
|
|
|
|
**File:** `frontend/src/components/Automation/CurrentProcessingCard.tsx`
|
|
|
|
```typescript
|
|
interface CurrentProcessingCardProps {
|
|
runId: string;
|
|
siteId: number;
|
|
currentStage: number;
|
|
onComplete?: () => void;
|
|
}
|
|
|
|
const CurrentProcessingCard: React.FC<CurrentProcessingCardProps> = ({
|
|
runId,
|
|
siteId,
|
|
currentStage,
|
|
onComplete
|
|
}) => {
|
|
const [processingState, setProcessingState] = useState<ProcessingState | null>(null);
|
|
|
|
// Poll every 3 seconds
|
|
useEffect(() => {
|
|
const fetchState = async () => {
|
|
const state = await automationService.getCurrentProcessing(siteId, runId);
|
|
setProcessingState(state);
|
|
|
|
// If stage completed, trigger refresh
|
|
if (state && state.processed_items === state.total_items) {
|
|
onComplete?.();
|
|
}
|
|
};
|
|
|
|
fetchState();
|
|
const interval = setInterval(fetchState, 3000);
|
|
|
|
return () => clearInterval(interval);
|
|
}, [siteId, runId]);
|
|
|
|
if (!processingState) return null;
|
|
|
|
const percentage = Math.round(
|
|
(processingState.processed_items / processingState.total_items) * 100
|
|
);
|
|
|
|
return (
|
|
<div className="bg-blue-50 dark:bg-blue-900/20 border-2 border-blue-500 rounded-lg p-6 mb-6">
|
|
{/* Header */}
|
|
<div className="flex items-center justify-between mb-4">
|
|
<div className="flex items-center gap-3">
|
|
<div className="animate-pulse">
|
|
<BoltIcon className="w-8 h-8 text-blue-600" />
|
|
</div>
|
|
<div>
|
|
<h2 className="text-2xl font-bold text-gray-900 dark:text-white">
|
|
Automation In Progress
|
|
</h2>
|
|
<p className="text-sm text-gray-600 dark:text-gray-400">
|
|
Stage {currentStage}: {processingState.stage_name}
|
|
<span className="ml-2 px-2 py-0.5 bg-blue-100 dark:bg-blue-900 text-blue-700 dark:text-blue-300 rounded text-xs">
|
|
{processingState.stage_type}
|
|
</span>
|
|
</p>
|
|
</div>
|
|
</div>
|
|
<div className="text-right">
|
|
<div className="text-4xl font-bold text-blue-600">{percentage}%</div>
|
|
<div className="text-sm text-gray-600 dark:text-gray-400">
|
|
{processingState.processed_items}/{processingState.total_items} processed
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
{/* Progress Bar */}
|
|
<div className="mb-6">
|
|
<div className="w-full bg-gray-200 dark:bg-gray-700 rounded-full h-3">
|
|
<div
|
|
className="bg-blue-600 h-3 rounded-full transition-all duration-500"
|
|
style={{ width: `${percentage}%` }}
|
|
/>
|
|
</div>
|
|
</div>
|
|
|
|
{/* Currently Processing */}
|
|
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
|
|
<div>
|
|
<h3 className="text-sm font-semibold text-gray-700 dark:text-gray-300 mb-2">
|
|
Currently Processing:
|
|
</h3>
|
|
<div className="space-y-1">
|
|
{processingState.currently_processing.map((item, idx) => (
|
|
<div key={idx} className="flex items-start gap-2 text-sm">
|
|
<span className="text-blue-600 mt-1">•</span>
|
|
<span className="text-gray-800 dark:text-gray-200 font-medium">
|
|
{item.title}
|
|
</span>
|
|
</div>
|
|
))}
|
|
</div>
|
|
</div>
|
|
|
|
<div>
|
|
<h3 className="text-sm font-semibold text-gray-700 dark:text-gray-300 mb-2">
|
|
Up Next:
|
|
</h3>
|
|
<div className="space-y-1">
|
|
{processingState.up_next.map((item, idx) => (
|
|
<div key={idx} className="flex items-start gap-2 text-sm">
|
|
<span className="text-gray-400 mt-1">•</span>
|
|
<span className="text-gray-600 dark:text-gray-400">
|
|
{item.title}
|
|
</span>
|
|
</div>
|
|
))}
|
|
{processingState.remaining_count > processingState.up_next.length && (
|
|
<div className="text-xs text-gray-500 mt-2">
|
|
+ {processingState.remaining_count - processingState.up_next.length} more in queue
|
|
</div>
|
|
)}
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
);
|
|
};
|
|
```
|
|
|
|
### 2. Update AutomationPage.tsx
|
|
|
|
**File:** `frontend/src/pages/Automation/AutomationPage.tsx`
|
|
|
|
```typescript
|
|
// Add new import
|
|
import CurrentProcessingCard from '../../components/Automation/CurrentProcessingCard';
|
|
|
|
// In the component
|
|
return (
|
|
<div className="p-6">
|
|
<PageMeta title="Automation" description="AI automation pipeline" />
|
|
|
|
{/* Current Processing Card - MOVE TO TOP */}
|
|
{currentRun?.status === 'running' && (
|
|
<CurrentProcessingCard
|
|
runId={currentRun.run_id}
|
|
siteId={selectedSite.id}
|
|
currentStage={currentRun.current_stage}
|
|
onComplete={() => {
|
|
// Refresh full page metrics when stage completes
|
|
loadAutomationData();
|
|
}}
|
|
/>
|
|
)}
|
|
|
|
{/* Metrics Cards */}
|
|
<div className="grid grid-cols-1 md:grid-cols-3 gap-6 mb-6">
|
|
{/* ... existing metrics ... */}
|
|
</div>
|
|
|
|
{/* Stages Section */}
|
|
<ComponentCard>
|
|
<h2 className="text-xl font-semibold mb-4">Pipeline Stages</h2>
|
|
{/* ... existing stages ... */}
|
|
</ComponentCard>
|
|
|
|
{/* Rest of the page ... */}
|
|
</div>
|
|
);
|
|
```
|
|
|
|
### 3. Add Service Method
|
|
|
|
**File:** `frontend/src/services/automationService.ts`
|
|
|
|
```typescript
|
|
export interface ProcessingState {
|
|
run_id: string;
|
|
current_stage: number;
|
|
stage_name: string;
|
|
stage_type: 'AI' | 'Local' | 'Manual';
|
|
total_items: number;
|
|
processed_items: number;
|
|
percentage: number;
|
|
currently_processing: Array<{
|
|
id: number;
|
|
title: string;
|
|
type: string;
|
|
}>;
|
|
up_next: Array<{
|
|
id: number;
|
|
title: string;
|
|
type: string;
|
|
}>;
|
|
remaining_count: number;
|
|
}
|
|
|
|
// Add to automationService
|
|
getCurrentProcessing: async (
|
|
siteId: number,
|
|
runId: string
|
|
): Promise<ProcessingState | null> => {
|
|
return fetchAPI(
|
|
buildUrl('/current_processing/', { site_id: siteId, run_id: runId })
|
|
);
|
|
},
|
|
```
|
|
|
|
---
|
|
|
|
## 🧪 TESTING PLAN
|
|
|
|
### Unit Tests
|
|
|
|
- [ ] Test `get_current_processing_state()` for each stage
|
|
- [ ] Test `_get_processed_count()` calculation
|
|
- [ ] Test `_get_current_items()` formatting
|
|
- [ ] Test API endpoint with various run states
|
|
|
|
### Integration Tests
|
|
|
|
- [ ] Test polling updates every 3 seconds
|
|
- [ ] Test stage completion triggers full refresh
|
|
- [ ] Test card disappears when automation completes
|
|
- [ ] Test with 0 items (edge case)
|
|
- [ ] Test with 1000+ items (performance)
|
|
|
|
### Visual/UX Tests
|
|
|
|
- [ ] Card positioned at top of page
|
|
- [ ] Progress bar animates smoothly
|
|
- [ ] Record names display correctly
|
|
- [ ] Responsive design (mobile/tablet/desktop)
|
|
- [ ] Dark mode support
|
|
- [ ] Loading states
|
|
- [ ] Error states
|
|
|
|
---
|
|
|
|
## 📊 STAGE-SPECIFIC DISPLAY FORMATS
|
|
|
|
### Stage 1: Keywords → Clusters
|
|
|
|
```
|
|
Currently Processing:
|
|
• "best seo tools"
|
|
• "content marketing platforms"
|
|
• "ai writing assistants"
|
|
|
|
+ 47 more keywords in queue
|
|
|
|
Progress: 3/50 keywords processed
|
|
```
|
|
|
|
### Stage 2: Clusters → Ideas
|
|
|
|
```
|
|
Currently Processing:
|
|
• "SEO Tools and Software" (Cluster #12)
|
|
|
|
Up Next:
|
|
• "Content Marketing Strategies"
|
|
• "AI Content Generation"
|
|
|
|
Progress: 12/25 clusters processed
|
|
```
|
|
|
|
### Stage 3: Ideas → Tasks
|
|
|
|
```
|
|
Currently Processing:
|
|
• "10 Best SEO Tools for 2025"
|
|
|
|
Up Next:
|
|
• "How to Create Content with AI"
|
|
• "Content Marketing ROI Calculator"
|
|
|
|
Progress: 8/30 ideas processed
|
|
```
|
|
|
|
### Stage 4: Tasks → Content
|
|
|
|
```
|
|
Currently Processing:
|
|
• "Ultimate Guide to SEO in 2025" (2,500 words)
|
|
|
|
Up Next:
|
|
• "AI Content Creation Best Practices"
|
|
|
|
Progress: 5/15 tasks processed
|
|
```
|
|
|
|
### Stage 5: Content → Image Prompts
|
|
|
|
```
|
|
Currently Processing:
|
|
• "How to Use ChatGPT for Content" (Extracting 3 image prompts)
|
|
|
|
Up Next:
|
|
• "Best AI Image Generators 2025"
|
|
|
|
Progress: 10/15 content pieces processed
|
|
```
|
|
|
|
### Stage 6: Image Prompts → Images
|
|
|
|
```
|
|
Currently Processing:
|
|
• Featured image for "SEO Guide 2025"
|
|
|
|
Up Next:
|
|
• In-article image #1 for "SEO Guide 2025"
|
|
• In-article image #2 for "SEO Guide 2025"
|
|
|
|
Progress: 15/45 images generated
|
|
```
|
|
|
|
### Stage 7: Manual Review Gate
|
|
|
|
```
|
|
Automation Complete! ✅
|
|
|
|
Ready for Review:
|
|
• "Ultimate Guide to SEO in 2025"
|
|
• "AI Content Creation Best Practices"
|
|
• "Best Image Generators 2025"
|
|
|
|
+ 12 more content pieces
|
|
|
|
Total: 15 content pieces ready for review
|
|
```
|
|
|
|
---
|
|
|
|
## 🎯 SUCCESS METRICS
|
|
|
|
### User Experience
|
|
|
|
✅ Users can see **exactly what's being processed** at any moment
|
|
✅ Users know **what's coming up next** in the queue
|
|
✅ Users can estimate **remaining time** based on progress
|
|
✅ Users get **quantitative feedback** (percentage, counts)
|
|
✅ Users see **smooth, non-disruptive updates** (no page flicker)
|
|
|
|
### Technical
|
|
|
|
✅ Polling interval: 3 seconds (balance between freshness and load)
|
|
✅ API response time: < 200ms
|
|
✅ Component re-render: Only the processing card, not entire page
|
|
✅ Memory usage: No memory leaks from polling
|
|
✅ Error handling: Graceful degradation if API fails
|
|
|
|
---
|
|
|
|
## 🚀 IMPLEMENTATION PHASES
|
|
|
|
### Phase 1: Backend (1-2 days)
|
|
- [ ] Implement `get_current_processing_state()` method
|
|
- [ ] Add `/current_processing/` API endpoint
|
|
- [ ] Test with all 7 stages
|
|
- [ ] Add unit tests
|
|
|
|
### Phase 2: Frontend (2-3 days)
|
|
- [ ] Create `CurrentProcessingCard` component
|
|
- [ ] Add polling logic with cleanup
|
|
- [ ] Style with Tailwind (match existing design system)
|
|
- [ ] Add dark mode support
|
|
- [ ] Integrate into `AutomationPage`
|
|
|
|
### Phase 3: Testing & Refinement (1-2 days)
|
|
- [ ] Integration testing
|
|
- [ ] Performance testing
|
|
- [ ] UX testing
|
|
- [ ] Bug fixes
|
|
|
|
### Phase 4: Deployment
|
|
- [ ] Deploy backend changes
|
|
- [ ] Deploy frontend changes
|
|
- [ ] Monitor first automation runs
|
|
- [ ] Collect user feedback
|
|
|
|
---
|
|
|
|
## 🔄 FUTURE ENHANCEMENTS
|
|
|
|
### V2 Features (Post-MVP)
|
|
|
|
1. **Estimated Time Remaining:**
|
|
```
|
|
Progress: 15/50 keywords processed
|
|
Estimated time remaining: ~8 minutes
|
|
```
|
|
|
|
2. **Stage-Level Progress Bar:**
|
|
- Each stage shows its own mini progress bar
|
|
- Visual indicator of which stages are complete
|
|
|
|
3. **Click to View Details:**
|
|
- Click on a record name to see modal with details
|
|
- Preview generated content/images
|
|
|
|
4. **Pause/Resume from Card:**
|
|
- Add pause button directly in the card
|
|
- Quick action without scrolling
|
|
|
|
5. **Export Processing Log:**
|
|
- Download real-time processing log
|
|
- CSV of all processed items with timestamps
|
|
|
|
---
|
|
|
|
## END OF PLAN
|
|
|
|
This plan provides a comprehensive UX improvement for automation progress tracking, making the process transparent and user-friendly while maintaining system performance.
|