notifciations issues fixed final

2025-12-28 00:52:14 +00:00
parent 28a60f8141
commit 0605f650b1
12 changed files with 1384 additions and 18 deletions
--- a/docs/10-MODULES/NOTIFICATIONS.md
+++ b/docs/10-MODULES/NOTIFICATIONS.md
@@ -0,0 +1,593 @@
+# Notifications Module
+
+**Last Verified:** December 28, 2025  
+**Status:** ✅ Active (v1.2.2 - Notifications Page Added)  
+**Backend Path:** `backend/igny8_core/business/notifications/`  
+**Frontend Path:** `frontend/src/store/notificationStore.ts`, `frontend/src/components/header/NotificationDropdown.tsx`, `frontend/src/pages/account/NotificationsPage.tsx`
+
+---
+
+## Quick Reference
+
+| What | File | Key Items |
+|------|------|-----------|
+| Models | `business/notifications/models.py` | `Notification`, `NotificationPreference` |
+| Views | `business/notifications/views.py` | `NotificationViewSet` |
+| Serializers | `business/notifications/serializers.py` | `NotificationSerializer` |
+| Services | `business/notifications/services.py` | `NotificationService` |
+| Frontend Store | `store/notificationStore.ts` | Zustand notification store |
+| Frontend Dropdown | `components/header/NotificationDropdown.tsx` | Notification UI (header bell) |
+| Frontend Page | `pages/account/NotificationsPage.tsx` | Full notifications history page |
+| API Client | `services/notifications.api.ts` | API methods |
+
+---
+
+## Purpose
+
+The Notifications module provides real-time user notifications for AI operations, system events, and workflow milestones. Notifications appear in the header dropdown and persist in the database.
+
+**Key Features:**
+- Real-time notifications for AI task completion/failure
+- Account-wide or user-specific notifications
+- Read/unread state tracking
+- Action buttons with navigation
+- Auto-fetch and auto-sync with API
+
+---
+
+## Data Models
+
+### Notification
+
+| Field | Type | Currently Used? | Purpose |
+|-------|------|----------------|---------|
+| account | FK | ✅ **Yes** | Owner account (required for multi-tenancy) |
+| user | FK | ❌ **No** | User-specific notifications (nullable - if null, visible to all account users). **Currently unused** - all notifications are account-wide |
+| notification_type | CharField | ✅ **Yes** | Type from NotificationType choices (for filtering, icons) |
+| title | CharField(200) | ✅ **Yes** | Notification title shown in dropdown |
+| message | TextField | ✅ **Yes** | Notification body text shown in dropdown |
+| severity | CharField | ✅ **Yes** | info/success/warning/error (affects icon color) |
+| site | FK | ✅ **Partial** | Related site (nullable). Created but **not displayed** in current dropdown |
+| content_type | FK | ❌ **No** | Generic relation to any object. **Future:** Link notification to specific Content/Task/etc. for direct access |
+| object_id | PositiveInteger | ❌ **No** | Generic relation ID. **Future:** Used with content_type for object linking |
+| action_url | CharField(500) | ✅ **Yes** | Frontend route for action button (e.g., `/planner/clusters`) |
+| action_label | CharField(50) | ✅ **Yes** | Action button text (e.g., "View Clusters") |
+| is_read | Boolean | ✅ **Yes** | Read status (default: False). Used for unread badge count |
+| read_at | DateTime | ✅ **Partial** | When marked read (nullable). Created but **not displayed** in dropdown |
+| metadata | JSON | ✅ **Partial** | Additional data (counts, details). Stored but **not displayed** in dropdown |
+| created_at | DateTime | ✅ **Yes** | Creation timestamp. Shown as relative time ("2 minutes ago") |
+| updated_at | DateTime | ❌ **No** | Last update timestamp. **Currently unused** |
+
+**Indexes:**
+- `(account, -created_at)` - List notifications by account ✅ **Used**
+- `(account, is_read, -created_at)` - Filter unread ✅ **Used**
+- `(user, -created_at)` - User-specific notifications ❌ **Unused** (all notifications are account-wide)
+
+---
+
+### Why So Many Unused Fields?
+
+The model was designed with **future expansion** in mind, but the current "dropdown-only" implementation uses less than half of the fields. Here's why they exist:
+
+**Over-Engineered for Current Use:**
+- `content_type` + `object_id` → For direct object linking (planned: click notification, go to that exact content item)
+- `user` → For user-specific notifications (planned: "@John, your content is ready")
+- `metadata` → For rich data (planned: show counts, progress, details in dedicated page)
+- `read_at` → For analytics (planned: "You read this 2 days ago")
+- `updated_at` → For editing notifications (planned: update notification instead of creating duplicate)
+
+**Currently Essential:**
+- `account`, `title`, `message`, `severity`, `notification_type` → Core notification data
+- `action_url`, `action_label` → Navigation buttons
+- `is_read`, `created_at` → Dropdown functionality
+
+**The Design Mismatch:**
+You're right - this is a **database schema designed for a full-featured notification system** (with dedicated page, filtering, search, history) but the frontend only implements a **simple dropdown**. The backend is over-built for the current use case.
+
+---
+
+## Notification Types
+
+### AI Operations
+
+| Type | Severity | Trigger | Message Format |
+|------|----------|---------|----------------|
+| `ai_cluster_complete` | success | Clustering finishes | "Created X clusters from Y keywords" |
+| `ai_cluster_failed` | error | Clustering fails | "Failed to cluster keywords: [error]" |
+| `ai_ideas_complete` | success | Idea generation finishes | "Generated X content ideas from Y clusters" |
+| `ai_ideas_failed` | error | Idea generation fails | "Failed to generate ideas: [error]" |
+| `ai_content_complete` | success | Content generation finishes | "Generated X articles (Y words)" |
+| `ai_content_failed` | error | Content generation fails | "Failed to generate content: [error]" |
+| `ai_images_complete` | success | Image generation finishes | "Generated X images" |
+| `ai_images_failed` | error | Image generation fails | "Failed to generate X images: [error]" |
+| `ai_prompts_complete` | success | Image prompts created | "X image prompts ready (1 featured + Y in-article)" |
+| `ai_prompts_failed` | error | Image prompt creation fails | "Failed to create image prompts: [error]" |
+
+### Workflow Events
+
+| Type | Severity | Trigger | Message Format |
+|------|----------|---------|----------------|
+| `content_ready_review` | info | Content moved to review | "[Title] ready for review" |
+| `content_published` | success | Content published | '"[Title]" published to [site]' |
+| `content_publish_failed` | error | Publishing fails | 'Failed to publish "[Title]": [error]' |
+
+### WordPress Sync
+
+| Type | Severity | Trigger | Message Format |
+|------|----------|---------|----------------|
+| `wordpress_sync_success` | success | Sync completes | "Synced X items with [site]" |
+| `wordpress_sync_failed` | error | Sync fails | "WordPress sync failed for [site]: [error]" |
+
+### Credits & Billing
+
+| Type | Severity | Trigger | Message Format |
+|------|----------|---------|----------------|
+| `credits_low` | warning | Credits < 20% | "You've used X% of your credits. Y remaining." |
+| `credits_depleted` | error | Credits exhausted | "Your credits are exhausted. Upgrade to continue." |
+
+### Setup & System
+
+| Type | Severity | Trigger | Message Format |
+|------|----------|---------|----------------|
+| `site_setup_complete` | success | All setup steps done | "[Site] is fully configured and ready!" |
+| `keywords_imported` | info | Keywords added | "Added X keywords to [site]" |
+| `system_info` | info | System messages | Custom message |
+
+---
+
+## Notification Triggers
+
+### AI Task Completion (AIEngine)
+
+**Location:** `backend/igny8_core/ai/engine.py`  
+**Methods:** `_create_success_notification()`, `_create_failure_notification()`
+
+**Flow:**
+1. AIEngine executes AI function (`auto_cluster`, `generate_ideas`, etc.)
+2. On **success** (after DONE phase):
+   - Calls `_create_success_notification(function_name, save_result, payload)`
+   - Maps function to NotificationService method
+   - Creates notification with counts/metrics
+3. On **failure** (in `_handle_error()`):
+   - Calls `_create_failure_notification(function_name, error)`
+   - Creates error notification with error message
+
+**Mapping:**
+```python
+auto_cluster → NotificationService.notify_clustering_complete/failed
+generate_ideas → NotificationService.notify_ideas_complete/failed
+generate_content → NotificationService.notify_content_complete/failed
+generate_image_prompts → NotificationService.notify_prompts_complete/failed
+generate_images → NotificationService.notify_images_complete/failed
+```
+
+### WordPress Publishing
+
+**Location:** `backend/igny8_core/tasks/wordpress_publishing.py`  
+**Trigger:** After publishing content to WordPress
+
+```python
+NotificationService.notify_content_published(
+    account=account,
+    site=site,
+    title=content.title,
+    content_object=content
+)
+```
+
+### WordPress Sync
+
+**Location:** `backend/igny8_core/tasks/wordpress_publishing.py`  
+**Trigger:** After syncing posts from WordPress
+
+```python
+NotificationService.notify_wordpress_sync_success(
+    account=account,
+    site=site,
+    count=synced_count
+)
+```
+
+### Credit Alerts
+
+**Location:** `backend/igny8_core/business/billing/services/credit_service.py`  
+**Trigger:** After deducting credits
+
+```python
+# When credits drop below threshold
+NotificationService.notify_credits_low(
+    account=account,
+    percentage_used=80,
+    credits_remaining=remaining
+)
+
+# When credits exhausted
+NotificationService.notify_credits_depleted(account=account)
+```
+
+### Manual Triggers (Optional)
+
+Notifications can also be created manually from anywhere:
+
+```python
+from igny8_core.business.notifications.services import NotificationService
+
+NotificationService.notify_keywords_imported(
+    account=account,
+    site=site,
+    count=keyword_count
+)
+```
+
+**Note:** As of v1.2.1, the following actions **DO create notifications**:
+- ✅ AI task completion/failure (clustering, ideas, content, images, prompts)
+- ✅ Keyword import via "Add to Workflow" - **Fixed in v1.2.1**
+
+**Actions that DON'T yet create notifications** (planned):
+- ❌ WordPress publishing (needs integration in wordpress_publishing.py)
+- ❌ WordPress sync (needs integration in wordpress_publishing.py)
+- ❌ Credit alerts (needs integration in credit_service.py)
+- ❌ Automation completion (planned for future)
+
+---
+
+## User Interface
+
+### Notification Dropdown (Header)
+
+**Location:** Header bell icon (top right)  
+**File:** `frontend/src/components/header/NotificationDropdown.tsx`
+
+**Features:**
+- Shows last 50 notifications
+- Animated badge with unread count
+- Click notification to mark read + navigate
+- "Mark all read" button
+- Auto-refreshes every 30 seconds
+- Refreshes when opened (if stale > 1 minute)
+- "View All Notifications" link to full page
+
+### Notifications Page (Full History)
+
+**Location:** `/account/notifications`  
+**File:** `frontend/src/pages/account/NotificationsPage.tsx`  
+**Access:** Sidebar → ACCOUNT → Notifications OR NotificationDropdown → "View All Notifications"
+
+**Features:**
+- **Filters:**
+  - Severity (info/success/warning/error)
+  - Notification type (AI operations, WordPress sync, credits, etc.)
+  - Read status (all/unread/read)
+  - Site (filter by specific site)
+  - Date range (from/to dates)
+- **Actions:**
+  - Mark individual notifications as read
+  - Mark all as read (bulk action)
+  - Delete individual notifications
+  - Click notification to navigate to action URL
+- **Display:**
+  - Full notification history with pagination
+  - Severity icons with color coding
+  - Relative timestamps ("2 hours ago")
+  - Site badge when applicable
+  - Action buttons for related pages
+  - Unread badge in sidebar menu
+
+**v1.2.2 Implementation:**
+- ✅ Full-page notifications view created
+- ✅ Advanced filtering by severity, type, read status, site, date range
+- ✅ Bulk actions (mark all read)
+- ✅ Individual actions (mark read, delete)
+- ✅ Added to sidebar under ACCOUNT section
+- ✅ Unread count badge in sidebar
+- ✅ Fixed broken link in NotificationDropdown (was `/notifications`, now `/account/notifications`)
+
+---
+
+## API Endpoints
+
+### List Notifications
+
+```http
+GET /api/v1/notifications/
+```
+
+**Query Parameters:**
+- `page` - Page number (default: 1)
+- `page_size` - Results per page (default: 20)
+- `is_read` - Filter by read status (`true`/`false`)
+- `notification_type` - Filter by type (e.g., `ai_cluster_complete`)
+- `severity` - Filter by severity (`info`/`success`/`warning`/`error`)
+
+**Response:**
+```json
+{
+  "count": 42,
+  "next": "/api/v1/notifications/?page=2",
+  "previous": null,
+  "results": [
+    {
+      "id": 123,
+      "notification_type": "ai_cluster_complete",
+      "severity": "success",
+      "title": "Clustering Complete",
+      "message": "Created 5 clusters from 50 keywords",
+      "is_read": false,
+      "created_at": "2025-12-28T10:30:00Z",
+      "read_at": null,
+      "action_label": "View Clusters",
+      "action_url": "/planner/clusters",
+      "site": {"id": 1, "name": "My Blog"},
+      "metadata": {
+        "cluster_count": 5,
+        "keyword_count": 50
+      }
+    }
+  ]
+}
+```
+
+### Get Unread Count
+
+```http
+GET /api/v1/notifications/unread-count/
+```
+
+**Response:**
+```json
+{
+  "unread_count": 7
+}
+```
+
+### Mark as Read
+
+```http
+POST /api/v1/notifications/{id}/read/
+```
+
+**Response:**
+```json
+{
+  "id": 123,
+  "is_read": true,
+  "read_at": "2025-12-28T10:35:00Z"
+}
+```
+
+### Mark All as Read
+
+```http
+POST /api/v1/notifications/read-all/
+```
+
+**Response:**
+```json
+{
+  "updated_count": 7,
+  "message": "Marked 7 notifications as read"
+}
+```
+
+### Delete Notification
+
+```http
+DELETE /api/v1/notifications/{id}/
+```
+
+**Response:** `204 No Content`
+
+---
+
+## Frontend Implementation
+
+### Notification Store
+
+**File:** `frontend/src/store/notificationStore.ts`
+
+**Features:**
+- Zustand store for state management
+- In-memory queue for optimistic UI updates
+- API sync for persistent notifications
+- Auto-fetch on mount and periodic sync (every 30 seconds)
+- Auto-refresh when dropdown opens (if stale > 1 minute)
+
+**Key Methods:**
+```typescript
+// Add local notification (optimistic)
+addNotification(notification)
+
+// Mark as read (optimistic + API sync)
+markAsRead(id)
+
+// Mark all as read
+markAllAsRead()
+
+// Fetch from API
+fetchNotifications()
+
+// Sync unread count
+syncUnreadCount()
+```
+
+### Notification Dropdown
+
+**File:** `frontend/src/components/header/NotificationDropdown.tsx`
+
+**Features:**
+- Badge with unread count (animated ping when > 0)
+- Dropdown with notification list
+- Click notification to mark read and navigate
+- "Mark all read" action
+- Empty state when no notifications
+- Auto-fetch on open if stale
+
+**Icon Mapping:**
+```typescript
+auto_cluster → GroupIcon
+generate_ideas → BoltIcon
+generate_content → FileTextIcon
+generate_images → FileIcon
+system → AlertIcon
+success → CheckCircleIcon
+```
+
+---
+
+## Business Logic
+
+### Visibility Rules
+
+1. **Account-wide notifications** (`user=NULL`):
+   - Visible to ALL users in the account
+   - Example: "Automation completed 10 tasks"
+
+2. **User-specific notifications** (`user=User`):
+   - Only visible to that specific user
+   - Example: "Your content is ready for review"
+
+3. **Site-filtered** (frontend):
+   - Frontend can filter by site in UI
+   - Backend always returns all account notifications
+
+### Read Status
+
+- Notifications start as `is_read=False`
+- Clicking notification marks it read (API call + optimistic update)
+- "Mark all read" bulk updates all unread notifications
+- Read notifications stay in dropdown (can be filtered out in future)
+
+### Retention Policy
+
+- Notifications never auto-delete (future: add retention policy)
+- Users can manually delete notifications
+- Admin can clean up old notifications via management command (future)
+
+---
+
+## Common Issues
+
+| Issue | Cause | Fix |
+|-------|-------|-----|
+| Notifications not appearing | AIEngine not calling NotificationService | Fixed in v1.2.1 |
+| "Add to workflow" no notification | KeywordViewSet not calling NotificationService | Fixed in v1.2.1 |
+| Can't see notification history | No dedicated notifications page | Fixed in v1.2.2 - Page created at /account/notifications |
+| "View All" button → 404 | Link to `/notifications` but page doesn't exist | Fixed in v1.2.2 - Link updated to /account/notifications |
+| Duplicate notifications | Multiple AI task retries | Check task retry logic |
+| Missing notifications | Celery worker crashed | Check Celery logs |
+| Unread count wrong | Race condition in state | Refresh page or wait for sync |
+| Action URL not working | Incorrect route in action_url | Check NotificationService methods |
+
+---
+
+## Integration Points
+
+| From | To | Trigger |
+|------|----|---------| 
+| AIEngine | NotificationService | AI task success/failure |
+| WordPress Publisher | NotificationService | Publishing/sync events |
+| Credit Service | NotificationService | Low credits/depleted |
+| Automation | NotificationService | Automation milestones (future) |
+
+---
+
+## Planned Changes
+
+| Feature | Status | Description |
+|---------|--------|-------------|
+| Notification preferences | 🔜 Planned | User can toggle notification types |
+| Email notifications | 🔜 Planned | Send email for critical notifications |
+| Push notifications | 🔜 Planned | Browser push for real-time alerts |
+| Notification retention | 🔜 Planned | Auto-delete after 30/60 days |
+| Notification categories | 🔜 Planned | Group by module (Planner, Writer, etc.) |
+| Notification sounds | 🔜 Planned | Audio alerts for important events |
+| Webhook notifications | 🔜 Planned | POST to external webhook URLs |
+
+---
+
+## Version History
+
+| Version | Date | Changes |
+|---------|------|---------|
+| v1.2.2 | Dec 28, 2025 | **NEW:** Full notifications page at /account/notifications with filtering, bulk actions, and sidebar integration |
+| v1.2.1 | Dec 28, 2025 | **Fixed:** Notifications now created on AI task completion + keyword import |
+| v1.2.0 | Dec 27, 2025 | Initial notifications system implementation |
+
+---
+
+## Testing
+
+### Test Notification Creation
+
+```python
+# In Django shell: docker exec -it igny8_backend python manage.py shell
+from igny8_core.business.notifications.services import NotificationService
+from igny8_core.auth.models import Account
+
+account = Account.objects.first()
+
+# Test clustering notification
+NotificationService.notify_clustering_complete(
+    account=account,
+    cluster_count=5,
+    keyword_count=50
+)
+
+# Test error notification
+NotificationService.notify_clustering_failed(
+    account=account,
+    error="Insufficient credits"
+)
+```
+
+### Test Frontend
+
+1. Run AI operation (clustering, idea generation, etc.)
+2. Check notification dropdown (bell icon in header)
+3. Verify unread badge appears
+4. Click notification to mark read and navigate
+5. Click "Mark all read" to clear badge
+
+---
+
+## Debug Commands
+
+```bash
+# Check Celery logs for notification creation
+docker logs igny8_celery_worker -f | grep -i "notification"
+
+# Check notifications in database
+docker exec -it igny8_backend python manage.py shell
+>>> from igny8_core.business.notifications.models import Notification
+>>> Notification.objects.count()
+>>> Notification.objects.filter(is_read=False).count()
+>>> Notification.objects.last().title
+
+# Test notification API
+curl -H "Authorization: Bearer <token>" \
+  http://localhost:8011/api/v1/notifications/
+
+# Check notification creation code
+grep -r "notify_clustering_complete" backend/
+```
+
+---
+
+## Architecture Notes
+
+**Design Pattern:** Service Layer + Repository Pattern
+- `NotificationService` provides static methods for creating notifications
+- Each notification type has dedicated method with validation
+- `Notification.create_notification()` is class method for low-level creation
+- Views use `AccountModelViewSet` for automatic account filtering
+
+**Why Not Signals?**
+- Explicit is better than implicit
+- Clear call sites for debugging
+- Easier to test and mock
+- No hidden side effects
+- Can pass context-specific data
+
+**Lazy Imports:**
+- NotificationService uses lazy imports in AIEngine to avoid Django app loading issues
+- Import inside method, not at module level
--- a/docs/30-FRONTEND/PAGES.md
+++ b/docs/30-FRONTEND/PAGES.md
@@ -150,6 +150,29 @@ Routes defined in `/frontend/src/App.tsx`:

 ## ACCOUNT Routes

+### Notifications
+
+| Route | File | Description |
+|-------|------|-------------|
+| `/account/notifications` | `account/NotificationsPage.tsx` | Full notifications history with filters and bulk actions |
+
+**Features:**
+- Filter by severity (info/success/warning/error)
+- Filter by notification type (AI operations, WordPress sync, credits, etc.)
+- Filter by read status (all/unread/read)
+- Filter by site
+- Filter by date range (from/to)
+- Mark individual notifications as read
+- Mark all as read (bulk action)
+- Delete individual notifications
+- Click notification to navigate to related page
+- Real-time unread count badge
+
+**v1.2.2 Changes:**
+- NEW page created with full filtering capabilities
+- Linked from NotificationDropdown "View All Notifications" button
+- Added to sidebar under ACCOUNT section with unread badge
+
 ### Account Settings

 | Route | File | Description |
--- a/docs/INDEX.md
+++ b/docs/INDEX.md
@@ -40,6 +40,7 @@
 | **Automation** | ✅ Active | 7-stage automated pipeline | [AUTOMATION.md](10-MODULES/AUTOMATION.md) |
 | **Billing** | ✅ Active | Credits, plans, payments | [BILLING.md](10-MODULES/BILLING.md) |
 | **Integrations** | ✅ Active | WordPress sync, webhooks | [INTEGRATIONS.md](10-MODULES/INTEGRATIONS.md) |
+| **Notifications** | ✅ Active | Real-time notifications for AI tasks | [NOTIFICATIONS.md](10-MODULES/NOTIFICATIONS.md) |
 | **System** | ✅ Active | Settings, prompts, AI config | [SYSTEM-SETTINGS.md](10-MODULES/SYSTEM-SETTINGS.md) |
 | **Publisher** | ✅ Active | Content publishing pipeline | [PUBLISHER.md](10-MODULES/PUBLISHER.md) |
 | **Linker** | ⏸️ Inactive | Internal linking (disabled by default) | [LINKER.md](10-MODULES/LINKER.md) |