igny8/docs/plans/implemented/fixes/footer-widget-pagination-fix.md

Footer Widget Pagination Fix - Summary

Problem

All pages with ThreeWidgetFooter are calculating metrics using page-filtered arrays instead of total counts from the API. This causes incorrect metric values when users are viewing paginated results.

Example:s

• If there are 100 total keywords with 10 on the current page
• And 5 keywords on the current page don't have a cluster_id
• The footer shows "Unmapped: 5" instead of the actual total unmapped count

Root Cause

The footer widgets use JavaScript .filter() methods on the local items array (which only contains the current page's data) instead of making separate API calls to get total counts for each status.

// WRONG - Uses page-filtered array
{ label: 'Unmapped', value: keywords.filter(k => !k.cluster_id).length }

// CORRECT - Uses total count from API
{ label: 'Unmapped', value: totalUnmapped }

Solution Pattern

For each affected page:

1. Add state variables for total counts
2. Create a loadTotalMetrics() function that makes lightweight API calls (page_size=1) filtered by status
3. Call loadTotalMetrics() when site/sector changes
4. Update footer widget to use the total count state instead of filtering local arrays

Files Fixed

✅ 1. Keywords.tsx

• Added: totalClustered, totalUnmapped, totalVolume state
• Added: loadTotalMetrics() function
• Updated: Footer widget to use total counts

✅ 2. Clusters.tsx

• Added: totalWithIdeas, totalReady state
• Added: loadTotalMetrics() function
• Updated: Footer widget to use total counts

⏳ 3. Ideas.tsx

• TODO: Add totalInTasks, totalPending state
• TODO: Add loadTotalMetrics() function with calls to:
  • fetchContentIdeas({ status: 'queued' }) → totalInTasks
  • fetchContentIdeas({ status: 'new' }) → totalPending
• TODO: Update footer widget metrics

⏳ 4. Tasks.tsx

• TODO: Add total count state variables
• TODO: Add loadTotalMetrics() function
• TODO: Update footer widget

⏳ 5. Content.tsx

• TODO: Add total count state variables for each status (draft, review, approved)
• TODO: Add loadTotalMetrics() function
• TODO: Update footer widget

⏳ 6. Images.tsx

• TODO: Add total count state variables
• TODO: Add loadTotalMetrics() function
• TODO: Update footer widget

⏳ 7. Review.tsx

• TODO: Add total count state variables
• TODO: Add loadTotalMetrics() function
• TODO: Update footer widget

⏳ 8. Approved.tsx

• TODO: Add total count state variables
• TODO: Add loadTotalMetrics() function
• TODO: Update footer widget

Implementation Template

Step 1: Add State Variables

// Total counts for footer widget (not page-filtered)
const [totalWithStatus1, setTotalWithStatus1] = useState(0);
const [totalWithStatus2, setTotalWithStatus2] = useState(0);

Step 2: Create loadTotalMetrics Function

// Load total metrics for footer widget (not affected by pagination)
const loadTotalMetrics = useCallback(async () => {
  if (!activeSite) return;

  try {
    // Get items with status1
    const status1Res = await fetchItems({
      page_size: 1,
      site_id: activeSite.id,
      ...(activeSector?.id && { sector_id: activeSector.id }),
      status: 'status1',
    });
    setTotalWithStatus1(status1Res.count || 0);

    // Get items with status2
    const status2Res = await fetchItems({
      page_size: 1,
      site_id: activeSite.id,
      ...(activeSector?.id && { sector_id: activeSector.id }),
      status: 'status2',
    });
    setTotalWithStatus2(status2Res.count || 0);
  } catch (error) {
    console.error('Error loading total metrics:', error);
  }
}, [activeSite, activeSector]);

Step 3: Call on Mount/Change

// Load total metrics when site/sector changes
useEffect(() => {
  loadTotalMetrics();
}, [loadTotalMetrics]);

Step 4: Update Footer Widget

<ThreeWidgetFooter
  pageProgress={{
    metrics: [
      { label: 'Total', value: totalCount },
      { label: 'Status 1', value: totalWithStatus1, percentage: `${totalCount > 0 ? Math.round((totalWithStatus1 / totalCount) * 100) : 0}%` },
      { label: 'Status 2', value: totalWithStatus2 },
    ],
    progress: {
      value: totalCount > 0 ? Math.round((totalWithStatus1 / totalCount) * 100) : 0,
      label: 'Processed',
      color: 'blue',
    },
    hint: totalWithStatus2 > 0
      ? `${totalWithStatus2} items ready for processing`
      : 'All items processed!',
  }}
  // ... rest of props
/>

Testing Checklist

For each fixed page, verify:

• Footer metrics show correct total counts (not page counts)
• Metrics update when changing sites
• Metrics update when changing sectors
• Metrics are consistent with automation page metrics
• Performance is acceptable (lightweight API calls with page_size=1)

Related Files

• /frontend/src/components/dashboard/ThreeWidgetFooter.tsx
• /frontend/src/pages/Automation/AutomationPage.tsx (reference implementation)
• All planner and writer page files

API Endpoints Used

All pages use their respective fetch* functions with filters:

• fetchKeywords({ status, page_size: 1 })
• fetchClusters({ status, page_size: 1 })
• fetchContentIdeas({ status, page_size: 1 })
• fetchTasks({ status, page_size: 1 })
• fetchContent({ status, page_size: 1 })
• fetchContentImages({ status, page_size: 1 })

The page_size: 1 ensures minimal data transfer while still getting the count.

Performance Considerations

• Each page makes 2-3 additional API calls on load
• Calls are lightweight (page_size=1, only count is used)
• Calls are cached until site/sector changes
• Total overhead: ~100-300ms per page load (acceptable)

Automation Page Consistency

The AutomationPage already uses this pattern correctly:

• Lines 99-149: Fetches total counts for all metrics
• Uses fetchKeywords({ status: 'new' }), fetchKeywords({ status: 'mapped' }), etc.
• Sets metrics in state: setMetrics({ keywords: { total, new, mapped } })
• All stage cards and metric cards use these pre-fetched totals

This fix brings all other pages in line with the Automation page's correct implementation.