20 KiB
20 KiB
Automation Progress UX Improvement Plan
Date: December 4, 2025
Status: Design Phase
Priority: MEDIUM
🎯 OBJECTIVE
Improve the automation progress tracking UX to show real-time processing status for currently processing items, making it easier for users to understand what's happening during automation runs.
🔍 CURRENT STATE ANALYSIS
Current Behavior
What Users See Now:
- A "Current State" card that shows the stage being processed
- Stage number and status (e.g., "Stage 3: Ideas → Tasks")
- BUT: No visibility into which specific records are being processed
- Problem: User only knows when a full stage completes
Example Current Experience:
┌─────────────────────────────────────┐
│ Current State: Stage 2 │
│ Clusters → Ideas (AI) │
│ │
│ Status: Processing │
└─────────────────────────────────────┘
[User waits... no updates until stage completes]
User Pain Points
- ❌ No Record-Level Progress: Can't see which keywords/ideas/content are being processed
- ❌ No Queue Visibility: Don't know what's coming up next
- ❌ No Item Count Progress: "Processing 15 of 50 keywords..." is missing
- ❌ Card Position: Current state card is at bottom, requires scrolling
- ❌ No Percentage Progress: Just a spinner, no quantitative feedback
✅ PROPOSED SOLUTION
New Design Concept
┌─────────────────────────────────────────────────────────────────────────────┐
│ 🔄 AUTOMATION IN PROGRESS │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67% │
│ │
│ Stage 2: Clusters → Ideas (AI) │
│Column 1 │
│ Currently Processing: │
│ • "Best SEO tools for small business" (Cluster #42) │
│ Column 2 │
│ Up Next: │
│ • "Content marketing automation platforms" │
│ • "AI-powered content creation tools" │
│ Sinngle row centered │
│ Progress: 34/50 clusters processed │
└─────────────────────────────────────────────────────────────────────────────┘
[STAGES SECTION BELOW - All 7 stages in grid view]
📐 DETAILED DESIGN SPECIFICATIONS
1. Card Repositioning
Move from: Bottom of page (after stages)
Move to: Top of page (above stages section)
Layout: Max width 1200px horizontal card
Visibility: Only shown when currentRun?.status === 'running'
2. Card Structure
Header Section
- Left: Large stage number icon (animated pulse)
- Center: Stage name + type badge (AI/Local/Manual)
- Right: Percentage complete (calculated from processed/total)
Progress Bar
- Type: Animated linear progress bar
- Colors:
- Blue for active stage
- Green for completed
- Gray for pending
- Updates: Refresh every 3-5 seconds via polling
Currently Processing Section
-
For Keywords Stage:
Currently Processing: • "keyword 1" • "keyword 2" • "keyword 3" + 47 more in queue -
For Ideas Stage:
Currently Processing: • "10 Ways to Improve SEO Rankings" Up Next: • "Content Marketing Best Practices 2025" • "AI Tools for Content Writers" -
For Content Stage:
Currently Processing: • "How to Use ChatGPT for Content Creation" (2,500 words) Up Next: • "Best AI Image Generators in 2025"
Record Counter
Progress: [current]/[total] [items] processed
Example: Progress: 15/50 keywords processed
3. Refresh Strategy
Polling Approach:
// Poll every 3 seconds while automation is running
useEffect(() => {
if (currentRun?.status === 'running') {
const interval = setInterval(() => {
// Refresh ONLY the current processing data
fetchCurrentProcessingState();
}, 3000);
return () => clearInterval(interval);
}
}, [currentRun]);
Partial Refresh:
- Only refresh the "Currently Processing" component
- Don't reload entire page
- Don't re-fetch stage cards
- Smooth transition (no flickering)
🗄️ BACKEND CHANGES REQUIRED
New API Endpoint
URL: GET /api/automation/current_processing/
Params: ?site_id={id}&run_id={run_id}
Response Format:
{
"run_id": "abc123",
"current_stage": 2,
"stage_name": "Clusters → Ideas",
"stage_type": "AI",
"total_items": 50,
"processed_items": 34,
"percentage": 68,
"currently_processing": [
{
"id": 42,
"title": "Best SEO tools for small business",
"type": "cluster"
}
],
"up_next": [
{
"id": 43,
"title": "Content marketing automation platforms",
"type": "cluster"
},
{
"id": 44,
"title": "AI-powered content creation tools",
"type": "cluster"
}
],
"remaining_count": 16
}
Implementation in AutomationService
File: backend/igny8_core/business/automation/services/automation_service.py
Add method:
def get_current_processing_state(self) -> dict:
"""
Get real-time processing state for current automation run
"""
if not self.run or self.run.status != 'running':
return None
stage = self.run.current_stage
# Get stage-specific data
if stage == 1: # Keywords → Clusters
queue = Keywords.objects.filter(
site=self.site, status='new'
).order_by('id')
return {
'stage_number': 1,
'stage_name': 'Keywords → Clusters',
'stage_type': 'AI',
'total_items': queue.count() + self._get_processed_count(stage),
'processed_items': self._get_processed_count(stage),
'currently_processing': self._get_current_items(queue, 3),
'up_next': self._get_next_items(queue, 2, skip=3),
}
elif stage == 2: # Clusters → Ideas
queue = Clusters.objects.filter(
site=self.site, status='new', disabled=False
).order_by('id')
return {
'stage_number': 2,
'stage_name': 'Clusters → Ideas',
'stage_type': 'AI',
'total_items': queue.count() + self._get_processed_count(stage),
'processed_items': self._get_processed_count(stage),
'currently_processing': self._get_current_items(queue, 1),
'up_next': self._get_next_items(queue, 2, skip=1),
}
# ... similar for stages 3-6
def _get_processed_count(self, stage: int) -> int:
"""Get count of items processed in current stage"""
result_key = f'stage_{stage}_result'
result = getattr(self.run, result_key, {})
# Extract appropriate count from result
if stage == 1:
return result.get('keywords_processed', 0)
elif stage == 2:
return result.get('clusters_processed', 0)
# ... etc
def _get_current_items(self, queryset, count: int) -> list:
"""Get currently processing items"""
items = queryset[:count]
return [
{
'id': item.id,
'title': getattr(item, 'keyword', None) or
getattr(item, 'cluster_name', None) or
getattr(item, 'idea_title', None) or
getattr(item, 'title', None),
'type': queryset.model.__name__.lower()
}
for item in items
]
Add View in AutomationViewSet
File: backend/igny8_core/business/automation/views.py
@action(detail=False, methods=['get'], url_path='current_processing')
def current_processing(self, request):
"""Get current processing state for active automation run"""
site_id = request.GET.get('site_id')
run_id = request.GET.get('run_id')
if not site_id or not run_id:
return error_response(
error='site_id and run_id required',
status_code=400,
request=request
)
try:
run = AutomationRun.objects.get(run_id=run_id, site_id=site_id)
if run.status != 'running':
return success_response(data=None, request=request)
service = AutomationService.from_run_id(run_id)
state = service.get_current_processing_state()
return success_response(data=state, request=request)
except AutomationRun.DoesNotExist:
return error_response(
error='Run not found',
status_code=404,
request=request
)
🎨 FRONTEND CHANGES REQUIRED
1. New Component: CurrentProcessingCard
File: frontend/src/components/Automation/CurrentProcessingCard.tsx
interface CurrentProcessingCardProps {
runId: string;
siteId: number;
currentStage: number;
onComplete?: () => void;
}
const CurrentProcessingCard: React.FC<CurrentProcessingCardProps> = ({
runId,
siteId,
currentStage,
onComplete
}) => {
const [processingState, setProcessingState] = useState<ProcessingState | null>(null);
// Poll every 3 seconds
useEffect(() => {
const fetchState = async () => {
const state = await automationService.getCurrentProcessing(siteId, runId);
setProcessingState(state);
// If stage completed, trigger refresh
if (state && state.processed_items === state.total_items) {
onComplete?.();
}
};
fetchState();
const interval = setInterval(fetchState, 3000);
return () => clearInterval(interval);
}, [siteId, runId]);
if (!processingState) return null;
const percentage = Math.round(
(processingState.processed_items / processingState.total_items) * 100
);
return (
<div className="bg-blue-50 dark:bg-blue-900/20 border-2 border-blue-500 rounded-lg p-6 mb-6">
{/* Header */}
<div className="flex items-center justify-between mb-4">
<div className="flex items-center gap-3">
<div className="animate-pulse">
<BoltIcon className="w-8 h-8 text-blue-600" />
</div>
<div>
<h2 className="text-2xl font-bold text-gray-900 dark:text-white">
Automation In Progress
</h2>
<p className="text-sm text-gray-600 dark:text-gray-400">
Stage {currentStage}: {processingState.stage_name}
<span className="ml-2 px-2 py-0.5 bg-blue-100 dark:bg-blue-900 text-blue-700 dark:text-blue-300 rounded text-xs">
{processingState.stage_type}
</span>
</p>
</div>
</div>
<div className="text-right">
<div className="text-4xl font-bold text-blue-600">{percentage}%</div>
<div className="text-sm text-gray-600 dark:text-gray-400">
{processingState.processed_items}/{processingState.total_items} processed
</div>
</div>
</div>
{/* Progress Bar */}
<div className="mb-6">
<div className="w-full bg-gray-200 dark:bg-gray-700 rounded-full h-3">
<div
className="bg-blue-600 h-3 rounded-full transition-all duration-500"
style={{ width: `${percentage}%` }}
/>
</div>
</div>
{/* Currently Processing */}
<div className="grid grid-cols-1 md:grid-cols-2 gap-4">
<div>
<h3 className="text-sm font-semibold text-gray-700 dark:text-gray-300 mb-2">
Currently Processing:
</h3>
<div className="space-y-1">
{processingState.currently_processing.map((item, idx) => (
<div key={idx} className="flex items-start gap-2 text-sm">
<span className="text-blue-600 mt-1">•</span>
<span className="text-gray-800 dark:text-gray-200 font-medium">
{item.title}
</span>
</div>
))}
</div>
</div>
<div>
<h3 className="text-sm font-semibold text-gray-700 dark:text-gray-300 mb-2">
Up Next:
</h3>
<div className="space-y-1">
{processingState.up_next.map((item, idx) => (
<div key={idx} className="flex items-start gap-2 text-sm">
<span className="text-gray-400 mt-1">•</span>
<span className="text-gray-600 dark:text-gray-400">
{item.title}
</span>
</div>
))}
{processingState.remaining_count > processingState.up_next.length && (
<div className="text-xs text-gray-500 mt-2">
+ {processingState.remaining_count - processingState.up_next.length} more in queue
</div>
)}
</div>
</div>
</div>
</div>
);
};
2. Update AutomationPage.tsx
File: frontend/src/pages/Automation/AutomationPage.tsx
// Add new import
import CurrentProcessingCard from '../../components/Automation/CurrentProcessingCard';
// In the component
return (
<div className="p-6">
<PageMeta title="Automation" description="AI automation pipeline" />
{/* Current Processing Card - MOVE TO TOP */}
{currentRun?.status === 'running' && (
<CurrentProcessingCard
runId={currentRun.run_id}
siteId={selectedSite.id}
currentStage={currentRun.current_stage}
onComplete={() => {
// Refresh full page metrics when stage completes
loadAutomationData();
}}
/>
)}
{/* Metrics Cards */}
<div className="grid grid-cols-1 md:grid-cols-3 gap-6 mb-6">
{/* ... existing metrics ... */}
</div>
{/* Stages Section */}
<ComponentCard>
<h2 className="text-xl font-semibold mb-4">Pipeline Stages</h2>
{/* ... existing stages ... */}
</ComponentCard>
{/* Rest of the page ... */}
</div>
);
3. Add Service Method
File: frontend/src/services/automationService.ts
export interface ProcessingState {
run_id: string;
current_stage: number;
stage_name: string;
stage_type: 'AI' | 'Local' | 'Manual';
total_items: number;
processed_items: number;
percentage: number;
currently_processing: Array<{
id: number;
title: string;
type: string;
}>;
up_next: Array<{
id: number;
title: string;
type: string;
}>;
remaining_count: number;
}
// Add to automationService
getCurrentProcessing: async (
siteId: number,
runId: string
): Promise<ProcessingState | null> => {
return fetchAPI(
buildUrl('/current_processing/', { site_id: siteId, run_id: runId })
);
},
🧪 TESTING PLAN
Unit Tests
- Test
get_current_processing_state()for each stage - Test
_get_processed_count()calculation - Test
_get_current_items()formatting - Test API endpoint with various run states
Integration Tests
- Test polling updates every 3 seconds
- Test stage completion triggers full refresh
- Test card disappears when automation completes
- Test with 0 items (edge case)
- Test with 1000+ items (performance)
Visual/UX Tests
- Card positioned at top of page
- Progress bar animates smoothly
- Record names display correctly
- Responsive design (mobile/tablet/desktop)
- Dark mode support
- Loading states
- Error states
📊 STAGE-SPECIFIC DISPLAY FORMATS
Stage 1: Keywords → Clusters
Currently Processing:
• "best seo tools"
• "content marketing platforms"
• "ai writing assistants"
+ 47 more keywords in queue
Progress: 3/50 keywords processed
Stage 2: Clusters → Ideas
Currently Processing:
• "SEO Tools and Software" (Cluster #12)
Up Next:
• "Content Marketing Strategies"
• "AI Content Generation"
Progress: 12/25 clusters processed
Stage 3: Ideas → Tasks
Currently Processing:
• "10 Best SEO Tools for 2025"
Up Next:
• "How to Create Content with AI"
• "Content Marketing ROI Calculator"
Progress: 8/30 ideas processed
Stage 4: Tasks → Content
Currently Processing:
• "Ultimate Guide to SEO in 2025" (2,500 words)
Up Next:
• "AI Content Creation Best Practices"
Progress: 5/15 tasks processed
Stage 5: Content → Image Prompts
Currently Processing:
• "How to Use ChatGPT for Content" (Extracting 3 image prompts)
Up Next:
• "Best AI Image Generators 2025"
Progress: 10/15 content pieces processed
Stage 6: Image Prompts → Images
Currently Processing:
• Featured image for "SEO Guide 2025"
Up Next:
• In-article image #1 for "SEO Guide 2025"
• In-article image #2 for "SEO Guide 2025"
Progress: 15/45 images generated
Stage 7: Manual Review Gate
Automation Complete! ✅
Ready for Review:
• "Ultimate Guide to SEO in 2025"
• "AI Content Creation Best Practices"
• "Best Image Generators 2025"
+ 12 more content pieces
Total: 15 content pieces ready for review
🎯 SUCCESS METRICS
User Experience
✅ Users can see exactly what's being processed at any moment
✅ Users know what's coming up next in the queue
✅ Users can estimate remaining time based on progress
✅ Users get quantitative feedback (percentage, counts)
✅ Users see smooth, non-disruptive updates (no page flicker)
Technical
✅ Polling interval: 3 seconds (balance between freshness and load)
✅ API response time: < 200ms
✅ Component re-render: Only the processing card, not entire page
✅ Memory usage: No memory leaks from polling
✅ Error handling: Graceful degradation if API fails
🚀 IMPLEMENTATION PHASES
Phase 1: Backend (1-2 days)
- Implement
get_current_processing_state()method - Add
/current_processing/API endpoint - Test with all 7 stages
- Add unit tests
Phase 2: Frontend (2-3 days)
- Create
CurrentProcessingCardcomponent - Add polling logic with cleanup
- Style with Tailwind (match existing design system)
- Add dark mode support
- Integrate into
AutomationPage
Phase 3: Testing & Refinement (1-2 days)
- Integration testing
- Performance testing
- UX testing
- Bug fixes
Phase 4: Deployment
- Deploy backend changes
- Deploy frontend changes
- Monitor first automation runs
- Collect user feedback
🔄 FUTURE ENHANCEMENTS
V2 Features (Post-MVP)
-
Estimated Time Remaining:
Progress: 15/50 keywords processed Estimated time remaining: ~8 minutes -
Stage-Level Progress Bar:
- Each stage shows its own mini progress bar
- Visual indicator of which stages are complete
-
Click to View Details:
- Click on a record name to see modal with details
- Preview generated content/images
-
Pause/Resume from Card:
- Add pause button directly in the card
- Quick action without scrolling
-
Export Processing Log:
- Download real-time processing log
- CSV of all processed items with timestamps
END OF PLAN
This plan provides a comprehensive UX improvement for automation progress tracking, making the process transparent and user-friendly while maintaining system performance.