re org

2026-01-17 16:11:31 +00:00
parent d3b3e1c0d4
commit ccb5b1d26d
9 changed files with 381 additions and 48 deletions
--- a/docs/10-MODULES/AUTOMATION.md
+++ b/docs/10-MODULES/AUTOMATION.md
@@ -1,7 +1,7 @@
 # Automation Module

-**Last Verified:** January 3, 2026  
-**Version:** 1.3.2  
+**Last Verified:** January 17, 2026  
+**Version:** 1.8.0  
 **Status:** ✅ Active  
 **Backend Path:** `backend/igny8_core/business/automation/`  
 **Frontend Path:** `frontend/src/pages/Automation/`
@@ -16,8 +16,12 @@
 | Service | `business/automation/services/automation_service.py` | `AutomationService` |
 | Logger | `business/automation/services/automation_logger.py` | `AutomationLogger` |
 | Celery Tasks | `business/automation/tasks.py` | `run_automation_task`, `check_scheduled_automations` |
-| Publishing Tasks | `igny8_core/tasks/publishing_scheduler.py` | Scheduled publishing (v1.3.2) |
-| Frontend | `pages/Automation/AutomationPage.tsx` | Main automation UI |
+| Publishing Tasks | `igny8_core/tasks/publishing_scheduler.py` | Scheduled publishing |
+| **Unified Settings** | `modules/integration/views/unified_settings.py` | **v1.8.0** Consolidated settings API |
+| Frontend Overview | `pages/Automation/AutomationOverview.tsx` | **v1.8.0** Run history dashboard |
+| Frontend Run Detail | `pages/Automation/AutomationRunDetail.tsx` | **v1.8.0** Detailed run view |
+| Frontend Manual Run | `pages/Automation/AutomationPage.tsx` | Manual run UI |
+| **Site Settings** | `pages/Sites/AIAutomationSettings.tsx` | **v1.8.0** Unified settings UI |
 | Progress Bar | `components/Automation/GlobalProgressBar.tsx` | Full pipeline progress |
 | Processing Card | `components/Automation/CurrentProcessingCard.tsx` | Real-time progress |

@@ -31,21 +35,23 @@ The Automation module runs the complete 7-stage content pipeline automatically:
 Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Images → Published
 ```

+**Settings Location (v1.8.0):** Site Settings → Automation tab
+
 ---

 ## 7-Stage Pipeline

-| Stage | Name | AI Function | Credit Cost |
-|-------|------|-------------|-------------|
-| 1 | Keywords → Clusters | `AutoClusterFunction` | Per batch |
-| 2 | Clusters → Ideas | `GenerateIdeasFunction` | Per idea |
-| 3 | Ideas → Tasks | None (local) | None |
-| 4 | Tasks → Content | `GenerateContentFunction` | Per 100 words |
-| 5 | Content → Image Prompts | `GenerateImagePromptsFunction` | Per prompt |
-| 6 | Image Prompts → Images | `process_image_generation_queue` | Per image |
-| 7 | Review → Published | Publishing Scheduler (v1.3.2) | None |
+| Stage | Name | AI Function | Credit Cost | Can Skip |
+|-------|------|-------------|-------------|----------|
+| 1 | Keywords → Clusters | `AutoClusterFunction` | Per batch | ✅ |
+| 2 | Clusters → Ideas | `GenerateIdeasFunction` | Per idea | ✅ |
+| 3 | Ideas → Tasks | None (local) | None | ✅ |
+| 4 | Tasks → Content | `GenerateContentFunction` | Per 100 words | ✅ |
+| 5 | Content → Image Prompts | `GenerateImagePromptsFunction` | Per prompt | ✅ |
+| 6 | Image Prompts → Images | `process_image_generation_queue` | Per image | ✅ |
+| 7 | Review → Published | Publishing Scheduler | None | ✅ |

-**Note:** Stage 7 uses the Publishing Scheduler with `PublishingSettings` to auto-approve and schedule content for publication. See [PUBLISHER.md](PUBLISHER.md) for details.
+**Note:** Stage 7 uses the Publishing Scheduler with `PublishingSettings` for auto-approval and scheduling.

 ---

@@ -58,7 +64,7 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag
 | account | FK | Owner account |
 | site | FK | Target site |
 | enabled | Boolean | Enable/disable automation |
-| frequency | CharField | daily/weekly/monthly |
+| frequency | CharField | hourly/daily/weekly |
 | scheduled_time | TimeField | Time to run |
 | stage_1_batch_size | Integer | Keywords per batch |
 | stage_2_batch_size | Integer | Clusters per batch |
@@ -66,6 +72,19 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag
 | stage_4_batch_size | Integer | Tasks per batch |
 | stage_5_batch_size | Integer | Content per batch |
 | stage_6_batch_size | Integer | Images per batch |
+| **stage_1_enabled** | Boolean | **v1.8.0** Enable stage 1 |
+| **stage_2_enabled** | Boolean | **v1.8.0** Enable stage 2 |
+| **stage_3_enabled** | Boolean | **v1.8.0** Enable stage 3 |
+| **stage_4_enabled** | Boolean | **v1.8.0** Enable stage 4 |
+| **stage_5_enabled** | Boolean | **v1.8.0** Enable stage 5 |
+| **stage_6_enabled** | Boolean | **v1.8.0** Enable stage 6 |
+| **stage_7_enabled** | Boolean | **v1.8.0** Enable stage 7 |
+| **max_keywords_per_run** | Integer | **v1.8.0** Per-run limit stage 1 |
+| **max_clusters_per_run** | Integer | **v1.8.0** Per-run limit stage 2 |
+| **max_ideas_per_run** | Integer | **v1.8.0** Per-run limit stage 3 |
+| **max_tasks_per_run** | Integer | **v1.8.0** Per-run limit stage 4 |
+| **max_content_per_run** | Integer | **v1.8.0** Per-run limit stage 5 |
+| **max_images_per_run** | Integer | **v1.8.0** Per-run limit stage 6 |
 | within_stage_delay | Integer | Seconds between batches |
 | between_stage_delay | Integer | Seconds between stages |
 | last_run_at | DateTime | Last execution |
@@ -234,6 +253,42 @@ For each stage (1-7):

 ---

+## Settings Configuration (v1.8.0)
+
+**Location:** Site Settings → Automation tab  
+**API:** `GET/PATCH /api/v1/integration/sites/{site_id}/unified-settings/`
+
+> ⚠️ **v1.8.0 Change:** Settings are now consolidated in Site Settings. The previous standalone `/automation/settings` page has been removed.
+
+### Settings UI Sections
+
+1. **Schedule & Frequency Card**
+   - Enable/disable toggle
+   - Frequency (hourly/daily/weekly)
+   - Days of week selection
+   - Time slot selection
+   - Next run display
+
+2. **Capacity Card**
+   - Total items per run (calculated)
+   - Stage-by-stage breakdown
+
+3. **AI Configuration Card**
+   - Testing model selection (is_testing=true)
+   - Live model selection (is_testing=false)
+   - Image model selection
+
+4. **Stage Configuration (Matrix)**
+   - Per-stage Enable/Disable toggle
+   - Per-stage batch size
+   - Per-stage max items per run (**new in v1.8.0**)
+
+5. **Help Cards**
+   - Pipeline flow visualization
+   - Stage descriptions
+
+---
+
 ## Scheduling

 **Celery Beat Task:** `check_scheduled_automations`  
@@ -276,14 +331,30 @@ During execution:

 ## Frontend Integration

-### AutomationPage Components
+### AutomationOverview (v1.8.0)

- **Config Panel:** Enable/disable, schedule settings
+**Path:** `/automation/overview`  
+**Purpose:** Run history dashboard with:
+- All automation runs with status
+- Filter by status, date range
+- Click to view detailed run information
+
+### AutomationRunDetail (v1.8.0)
+
+**Path:** `/automation/runs/:runId`  
+**Purpose:** Detailed view of individual run with:
+- Stage-by-stage progress
+- Items processed per stage
+- Errors and logs
+
+### AutomationPage
+
+**Path:** `/automation`  
+**Purpose:** Manual run control with:
 - **Pipeline Cards:** Stage-by-stage status with pending counts
 - **Processing Card:** Live processing status during run
 - **Control Buttons:** Run Now, Pause, Resume, Cancel
 - **Activity Log:** Real-time log streaming
- **History Table:** Past 20 runs with status

 ### Polling

@@ -305,11 +376,11 @@ During execution:

 ---

-## Planned Changes
+## Changelog

-| Feature | Status | Description |
-|---------|--------|-------------|
-| Progress bar accuracy | 🐛 Bug | Fix progress calculation based on actual stage |
-| Completed count display | 🐛 Bug | Fix count display in UI |
-| Stage skip configuration | 🔜 Planned | Allow skipping certain stages |
-| Notification on complete | 🔜 Planned | Email/webhook when done |
+| Version | Changes |
+|---------|---------|
+| v1.8.0 | Unified settings in Site Settings, per-run limits, skip stage functionality |
+| v1.7.0 | AutomationOverview dashboard, AutomationRunDetail page |
+| v1.3.2 | Stage 7 publishing scheduler integration |
+| v1.3.0 | Progress tracking with initial snapshots |
--- a/docs/10-MODULES/PUBLISHER.md
+++ b/docs/10-MODULES/PUBLISHER.md
@@ -1,8 +1,8 @@
 # Publisher Module

-**Last Verified:** January 3, 2026  
+**Last Verified:** January 17, 2026  
 **Status:** ✅ Active  
-**Version:** 1.3.2
+**Version:** 1.8.0
 **Backend Path:** `backend/igny8_core/modules/publisher/` + `backend/igny8_core/business/publishing/`  
 **Frontend Path:** `frontend/src/pages/Publisher/`

@@ -15,10 +15,12 @@
 | Views | `modules/publisher/views.py` | `PublishingRecordViewSet`, `DeploymentViewSet`, `PublishContentViewSet` |
 | Models | `business/publishing/models.py` | `PublishingRecord`, `DeploymentRecord` |
 | Integration Models | `modules/integration/models.py` | `PublishingSettings` |
+| **Unified Settings** | `modules/integration/views/unified_settings.py` | **v1.8.0** Consolidated settings API |
 | Services | `business/publishing/services/*.py` | Publishing orchestration |
 | Scheduler Tasks | `igny8_core/tasks/publishing_scheduler.py` | Celery beat tasks |
 | URLs | `modules/publisher/urls.py` | Publisher endpoints |
 | Frontend | `pages/Publisher/ContentCalendar.tsx` | Content calendar view |
+| **Site Settings** | `pages/Sites/AIAutomationSettings.tsx` | **v1.8.0** Unified settings UI |

 ---

@@ -32,6 +34,10 @@ The Publisher module manages:
 - Multi-destination publishing
 - **Content calendar visualization**

+**Settings Location (v1.8.0):** Site Settings → Automation tab
+
+> ⚠️ **v1.8.0 Change:** The standalone `/publisher/settings` page has been removed. Publishing settings are now configured in Site Settings → Automation tab under "Capacity" and "Schedule" sections.
+
 ---

 ## Data Models
@@ -65,7 +71,7 @@ The Publisher module manages:
 | error_log | TextField | Errors encountered |
 | metadata | JSON | Deployment details |

-### PublishingSettings (v1.3.2)
+### PublishingSettings (v1.3.2, updated v1.8.0)

 Site-level publishing configuration:

@@ -79,6 +85,7 @@ Site-level publishing configuration:
 | monthly_publish_limit | Integer | Max publications per month |
 | publish_days | JSON | Days of week for publishing ["mon","wed","fri"] |
 | publish_time_slots | JSON | Time slots for publishing [{"start":"09:00","end":"17:00"}] |
+| **total_items_per_run** | Integer | **v1.8.0** Computed capacity display |

 ### Content Site Status Fields (v1.3.2)

@@ -94,6 +101,13 @@ Added to Content model for scheduling:

 ## API Endpoints

+### Unified Settings API (v1.8.0)
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| **GET** | `/api/v1/integration/sites/{site_id}/unified-settings/` | Get all automation + publishing settings |
+| **PATCH** | `/api/v1/integration/sites/{site_id}/unified-settings/` | Update settings |
+
 ### Publishing & Records

 | Method | Path | Handler | Purpose |
@@ -115,12 +129,14 @@ Added to Content model for scheduling:
 | **POST** | `/api/v1/writer/content/bulk_schedule/` | Bulk schedule with site defaults | `{ "content_ids": [1,2,3], "use_site_defaults": true, "site_id": 5 }` | `{ "success": true, "scheduled_count": 3, "schedule_preview": [...] }` |
 | **POST** | `/api/v1/writer/content/bulk_schedule_preview/` | Preview bulk schedule times | `{ "content_ids": [1,2,3], "site_id": 5 }` | `{ "schedule_preview": [...], "site_settings": {...} }` |

-### Publishing Settings
+### Legacy Publishing Settings (deprecated)
+
+> ⚠️ **Deprecated in v1.8.0:** Use unified-settings API instead

 | Method | Path | Purpose |
 |--------|------|---------|
-| **GET** | `/api/v1/sites/{site_id}/settings?tab=publishing` | Get site publishing settings (default schedule, stagger, limits) |
-| **PUT** | `/api/v1/sites/{site_id}/publishing-settings/` | Update publishing settings |
+| ~~GET~~ | ~~`/api/v1/sites/{site_id}/settings?tab=publishing`~~ | ~~Get site publishing settings~~ |
+| ~~PUT~~ | ~~`/api/v1/sites/{site_id}/publishing-settings/`~~ | ~~Update publishing settings~~ |

 ---

--- a/docs/20-API/ENDPOINTS.md
+++ b/docs/20-API/ENDPOINTS.md
@@ -1,7 +1,7 @@
 # API Endpoints Reference

-**Last Verified:** January 3, 2026  
-**Version:** 1.3.2  
+**Last Verified:** January 17, 2026  
+**Version:** 1.8.0  
 **Base URL:** `/api/v1/`  
 **Documentation:** `/api/docs/` (Swagger) | `/api/redoc/` (ReDoc)

@@ -149,7 +149,51 @@ POST /api/v1/writer/content/{id}/schedule/
 | POST | `/site_sync/` | Site-level sync | Sync by site ID |
 | POST | `/webhook/wordpress/` | WordPress webhook | Receive WP updates |

-### Publishing Settings Endpoints (v1.3.2)
+### Unified Settings Endpoints (v1.8.0)
+
+| Method | Path | Handler | Purpose |
+|--------|------|---------|---------|
+| GET | `/sites/{site_id}/unified-settings/` | `UnifiedSiteSettingsViewSet.retrieve` | Get all site settings (automation, publishing, AI) |
+| PATCH | `/sites/{site_id}/unified-settings/` | `UnifiedSiteSettingsViewSet.partial_update` | Update settings (partial) |
+
+**Unified Settings Response:**
+```json
+{
+  "automation_config": {
+    "enabled": true,
+    "frequency": "daily",
+    "scheduled_days": ["mon", "wed", "fri"],
+    "scheduled_time": "09:00:00",
+    "next_run_at": "2026-01-18T09:00:00Z",
+    "stage_1_enabled": true, "stage_1_batch_size": 50, "max_keywords_per_run": 100,
+    "stage_2_enabled": true, "stage_2_batch_size": 10, "max_clusters_per_run": 20,
+    // ... stages 3-7
+  },
+  "publishing_settings": {
+    "auto_approval_enabled": true,
+    "auto_publish_enabled": true,
+    "daily_publish_limit": 3,
+    "weekly_publish_limit": 15,
+    "monthly_publish_limit": 50,
+    "publish_days": ["mon", "tue", "wed", "thu", "fri"],
+    "publish_time_slots": [{"start": "09:00", "end": "17:00"}],
+    "total_items_per_run": 500
+  },
+  "stage_configs": [
+    {"stage": 1, "enabled": true, "batch_size": 50, "max_per_run": 100},
+    // ... stages 2-7
+  ],
+  "ai_settings": {
+    "testing_model_id": 2,
+    "live_model_id": 1,
+    "image_model_id": 3
+  }
+}
+```
+
+### Publishing Settings Endpoints (deprecated v1.8.0)
+
+> ⚠️ **Deprecated:** Use `/sites/{site_id}/unified-settings/` instead

 | Method | Path | Handler | Purpose |
 |--------|------|---------|---------|
--- a/docs/30-FRONTEND/PAGES.md
+++ b/docs/30-FRONTEND/PAGES.md
@@ -1,7 +1,7 @@
 # Frontend Pages & Routes

-**Last Verified:** January 3, 2026  
-**Version:** 1.3.2  
+**Last Verified:** January 17, 2026  
+**Version:** 1.8.0  
 **Framework:** React 19 + TypeScript + React Router 6 + Vite

 ---
@@ -108,18 +108,24 @@ Routes defined in `/frontend/src/App.tsx`:
 |-------|------|-------------|
 | `/sites` | `Sites/List.tsx` | Site listing with setup checklist per site |
 | `/sites/:id` | `Sites/Dashboard.tsx` | Site dashboard with quick actions and widgets |
-| `/sites/:id/settings` | `Sites/Settings.tsx` | Site settings (General, Integrations, Publishing, Content Types) |
+| `/sites/:id/settings` | `Sites/Settings.tsx` | Site settings (General, Automation, Integrations) |
+
+**v1.8.0 Changes:**
+- Site Settings now has 3 tabs: General, Automation, Integrations
+- Automation tab consolidates all AI, Automation, and Publishing settings
+- Removed separate AI Settings tab
+- Uses `AIAutomationSettings.tsx` component for unified settings UI

 **v1.3.2 Changes:**
 - Site Dashboard redesigned with SiteInfoBar component
 - Quick actions now 2-column card grid layout
 - AI Operations widget loads real data from getDashboardStats API
 - Fixed race condition in async loadSiteData
- Settings page has new Publishing Settings tab

 **Components:**
 - `SiteSetupChecklist` - Shows setup progress (site created, industry/sectors, WordPress, keywords)
 - `SiteInfoBar` - Reusable site info header for site-specific pages
+- `AIAutomationSettings` - Unified settings component (v1.8.0)

 ### Thinker (Admin Only)

@@ -165,7 +171,17 @@ Routes defined in `/frontend/src/App.tsx`:

 | Route | File | Description |
 |-------|------|-------------|
-| `/automation` | `Automation/AutomationPage.tsx` | 7-stage pipeline, schedule config, run controls |
+| `/automation` | `Automation/AutomationPage.tsx` | 7-stage pipeline, run controls |
+| `/automation/overview` | `Automation/AutomationOverview.tsx` | Run history dashboard (v1.7.0) |
+| `/automation/runs/:id` | `Automation/AutomationRunDetail.tsx` | Detailed run view (v1.7.0) |
+
+**v1.8.0 Changes:**
+- Removed `/automation/settings` route (merged into Site Settings → Automation tab)
+- Added redirect from `/automation/settings` to `/sites/settings?tab=automation`
+
+**v1.7.0 Changes:**
+- Added AutomationOverview page with run history
+- Added AutomationRunDetail page for detailed run information

 ---

@@ -201,6 +217,10 @@ Routes defined in `/frontend/src/App.tsx`:
 | Old Route | Redirects To |
 |-----------|--------------|
 | `/sites/:id/publishing-queue` | `/publisher/content-calendar` |
+| `/automation/settings` | `/sites/settings?tab=automation` (v1.8.0) |
+
+**Removed in v1.8.0:**
+- `/publisher/settings` - No longer exists (settings in Site Settings → Automation)

 ### Linker (Optional Module)

--- a/docs/INDEX.md
+++ b/docs/INDEX.md
@@ -1,7 +1,7 @@
 # IGNY8 Technical Documentation

-**Version:** 1.7.1
-**Last Updated:** January 11, 2026  
+**Version:** 1.8.0
+**Last Updated:** January 17, 2026  
 **Purpose:** Complete technical reference for the IGNY8 AI content platform

 ---
@@ -77,7 +77,7 @@
 | [PAGE-REQUIREMENTS.md](30-FRONTEND/PAGE-REQUIREMENTS.md) | Site/sector selector requirements |
 | [STORES.md](30-FRONTEND/STORES.md) | Zustand state management |

-### Current Page Structure (v1.4.0)
+### Current Page Structure (v1.8.0)

 ```
 /                           → Dashboard (Home.tsx) - with workflow widgets
@@ -88,7 +88,7 @@
 │   /sites                  → Sites List (List.tsx)
 │   /sites/:id              → Site Dashboard (Dashboard.tsx)
 │   /sites/:id/settings     → Site Settings (Settings.tsx)
-│                             - Tabs: General, AI Settings (v1.4.0), Integrations, Publishing, Content Types
+│                             - Tabs: General, Automation (v1.8.0), Integrations
 │   /thinker/prompts        → Thinker Prompts (Prompts.tsx) [Admin]
 │   /thinker/author-profiles → Author Profiles (AuthorProfiles.tsx) [Admin]
 ├── WORKFLOW
@@ -101,6 +101,8 @@
 │   /writer/review          → Review (Review.tsx)
 │   /writer/approved        → Approved (Approved.tsx)
 │   /automation             → Automation Dashboard (AutomationPage.tsx)
+│   /automation/overview    → Run History (AutomationOverview.tsx) [v1.7.0]
+│   /automation/runs/:id    → Run Detail (AutomationRunDetail.tsx) [v1.7.0]
 ├── PUBLISHER
 │   /publisher/content-calendar → Content Calendar (ContentCalendar.tsx)
 ├── OPTIONAL MODULES
@@ -121,6 +123,11 @@
    /ui-elements            → UI Elements (UIElements.tsx) [Design System Ref]
 ```

+**Removed in v1.8.0:**
+- `/automation/settings` - Merged into Site Settings → Automation tab
+- `/publisher/settings` - Merged into Site Settings → Automation tab
+- `/sites/:id/settings` AI Settings tab - Merged into Automation tab
+
 **Removed in v1.4.0:**
 - `/settings/ai` - AI Settings page merged into Site Settings AI tab