automation and ai and some planning and fixes adn docs reorg
This commit is contained in:
IGNY8 VPS (Salman)
193
docs/fixes/footer-widget-pagination-fix.md
Normal file
193
docs/fixes/footer-widget-pagination-fix.md
Normal file
@@ -0,0 +1,193 @@
|
||||
# Footer Widget Pagination Fix - Summary
|
||||
|
||||
## Problem
|
||||
|
||||
All pages with `ThreeWidgetFooter` are calculating metrics using **page-filtered arrays** instead of **total counts** from the API. This causes incorrect metric values when users are viewing paginated results.
|
||||
|
||||
### Example:
|
||||
- If there are 100 total keywords with 10 on the current page
|
||||
- And 5 keywords on the current page don't have a `cluster_id`
|
||||
- The footer shows "Unmapped: 5" instead of the actual total unmapped count
|
||||
|
||||
## Root Cause
|
||||
|
||||
The footer widgets use JavaScript `.filter()` methods on the local `items` array (which only contains the current page's data) instead of making separate API calls to get total counts for each status.
|
||||
|
||||
```typescript
|
||||
// WRONG - Uses page-filtered array
|
||||
{ label: 'Unmapped', value: keywords.filter(k => !k.cluster_id).length }
|
||||
|
||||
// CORRECT - Uses total count from API
|
||||
{ label: 'Unmapped', value: totalUnmapped }
|
||||
```
|
||||
|
||||
## Solution Pattern
|
||||
|
||||
For each affected page:
|
||||
|
||||
1. **Add state variables for total counts**
|
||||
2. **Create a `loadTotalMetrics()` function** that makes lightweight API calls (page_size=1) filtered by status
|
||||
3. **Call `loadTotalMetrics()` when site/sector changes**
|
||||
4. **Update footer widget** to use the total count state instead of filtering local arrays
|
||||
|
||||
## Files Fixed
|
||||
|
||||
### ✅ 1. Keywords.tsx
|
||||
- Added: `totalClustered`, `totalUnmapped`, `totalVolume` state
|
||||
- Added: `loadTotalMetrics()` function
|
||||
- Updated: Footer widget to use total counts
|
||||
|
||||
### ✅ 2. Clusters.tsx
|
||||
- Added: `totalWithIdeas`, `totalReady` state
|
||||
- Added: `loadTotalMetrics()` function
|
||||
- Updated: Footer widget to use total counts
|
||||
|
||||
### ⏳ 3. Ideas.tsx
|
||||
- **TODO**: Add `totalInTasks`, `totalPending` state
|
||||
- **TODO**: Add `loadTotalMetrics()` function with calls to:
|
||||
- `fetchContentIdeas({ status: 'queued' })` → `totalInTasks`
|
||||
- `fetchContentIdeas({ status: 'new' })` → `totalPending`
|
||||
- **TODO**: Update footer widget metrics
|
||||
|
||||
### ⏳ 4. Tasks.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 5. Content.tsx
|
||||
- **TODO**: Add total count state variables for each status (draft, review, approved)
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 6. Images.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 7. Review.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
### ⏳ 8. Approved.tsx
|
||||
- **TODO**: Add total count state variables
|
||||
- **TODO**: Add `loadTotalMetrics()` function
|
||||
- **TODO**: Update footer widget
|
||||
|
||||
## Implementation Template
|
||||
|
||||
### Step 1: Add State Variables
|
||||
|
||||
```typescript
|
||||
// Total counts for footer widget (not page-filtered)
|
||||
const [totalWithStatus1, setTotalWithStatus1] = useState(0);
|
||||
const [totalWithStatus2, setTotalWithStatus2] = useState(0);
|
||||
```
|
||||
|
||||
### Step 2: Create loadTotalMetrics Function
|
||||
|
||||
```typescript
|
||||
// Load total metrics for footer widget (not affected by pagination)
|
||||
const loadTotalMetrics = useCallback(async () => {
|
||||
if (!activeSite) return;
|
||||
|
||||
try {
|
||||
// Get items with status1
|
||||
const status1Res = await fetchItems({
|
||||
page_size: 1,
|
||||
site_id: activeSite.id,
|
||||
...(activeSector?.id && { sector_id: activeSector.id }),
|
||||
status: 'status1',
|
||||
});
|
||||
setTotalWithStatus1(status1Res.count || 0);
|
||||
|
||||
// Get items with status2
|
||||
const status2Res = await fetchItems({
|
||||
page_size: 1,
|
||||
site_id: activeSite.id,
|
||||
...(activeSector?.id && { sector_id: activeSector.id }),
|
||||
status: 'status2',
|
||||
});
|
||||
setTotalWithStatus2(status2Res.count || 0);
|
||||
} catch (error) {
|
||||
console.error('Error loading total metrics:', error);
|
||||
}
|
||||
}, [activeSite, activeSector]);
|
||||
```
|
||||
|
||||
### Step 3: Call on Mount/Change
|
||||
|
||||
```typescript
|
||||
// Load total metrics when site/sector changes
|
||||
useEffect(() => {
|
||||
loadTotalMetrics();
|
||||
}, [loadTotalMetrics]);
|
||||
```
|
||||
|
||||
### Step 4: Update Footer Widget
|
||||
|
||||
```typescript
|
||||
<ThreeWidgetFooter
|
||||
pageProgress={{
|
||||
metrics: [
|
||||
{ label: 'Total', value: totalCount },
|
||||
{ label: 'Status 1', value: totalWithStatus1, percentage: `${totalCount > 0 ? Math.round((totalWithStatus1 / totalCount) * 100) : 0}%` },
|
||||
{ label: 'Status 2', value: totalWithStatus2 },
|
||||
],
|
||||
progress: {
|
||||
value: totalCount > 0 ? Math.round((totalWithStatus1 / totalCount) * 100) : 0,
|
||||
label: 'Processed',
|
||||
color: 'blue',
|
||||
},
|
||||
hint: totalWithStatus2 > 0
|
||||
? `${totalWithStatus2} items ready for processing`
|
||||
: 'All items processed!',
|
||||
}}
|
||||
// ... rest of props
|
||||
/>
|
||||
```
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
For each fixed page, verify:
|
||||
|
||||
- [ ] Footer metrics show correct total counts (not page counts)
|
||||
- [ ] Metrics update when changing sites
|
||||
- [ ] Metrics update when changing sectors
|
||||
- [ ] Metrics are consistent with automation page metrics
|
||||
- [ ] Performance is acceptable (lightweight API calls with page_size=1)
|
||||
|
||||
## Related Files
|
||||
|
||||
- `/frontend/src/components/dashboard/ThreeWidgetFooter.tsx`
|
||||
- `/frontend/src/pages/Automation/AutomationPage.tsx` (reference implementation)
|
||||
- All planner and writer page files
|
||||
|
||||
## API Endpoints Used
|
||||
|
||||
All pages use their respective `fetch*` functions with filters:
|
||||
- `fetchKeywords({ status, page_size: 1 })`
|
||||
- `fetchClusters({ status, page_size: 1 })`
|
||||
- `fetchContentIdeas({ status, page_size: 1 })`
|
||||
- `fetchTasks({ status, page_size: 1 })`
|
||||
- `fetchContent({ status, page_size: 1 })`
|
||||
- `fetchContentImages({ status, page_size: 1 })`
|
||||
|
||||
The `page_size: 1` ensures minimal data transfer while still getting the count.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- Each page makes 2-3 additional API calls on load
|
||||
- Calls are lightweight (page_size=1, only count is used)
|
||||
- Calls are cached until site/sector changes
|
||||
- Total overhead: ~100-300ms per page load (acceptable)
|
||||
|
||||
## Automation Page Consistency
|
||||
|
||||
The AutomationPage already uses this pattern correctly:
|
||||
- Lines 99-149: Fetches total counts for all metrics
|
||||
- Uses `fetchKeywords({ status: 'new' })`, `fetchKeywords({ status: 'mapped' })`, etc.
|
||||
- Sets metrics in state: `setMetrics({ keywords: { total, new, mapped } })`
|
||||
- All stage cards and metric cards use these pre-fetched totals
|
||||
|
||||
This fix brings all other pages in line with the Automation page's correct implementation.
|
||||
Reference in New Issue
Block a user