diff --git a/CHANGELOG.md b/CHANGELOG.md index bd7b28b6..e3888ca2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,7 @@ # IGNY8 Change Log -**Current Version:** 1.2.2 -**Last Updated:** December 28, 2025 +**Current Version:** 1.3.0 +**Last Updated:** January 1, 2026 --- @@ -9,6 +9,7 @@ | Version | Date | Summary | |---------|------|---------| +| 1.3.0 | Jan 1, 2026 | **Major** - Automation overhaul, AI provider integrations (Anthropic/Bria), Model Registry, Global Progress tracking, CSS globalization | | 1.2.2 | Dec 28, 2025 | **NEW** - Full notifications page with filtering and bulk actions | | 1.2.1 | Dec 28, 2025 | **Critical Fix** - AI task notifications now working | | 1.2.0 | Dec 27, 2025 | **Final Launch Release** - Notifications system, Dashboard widgets, ThreeWidgetFooter, UI polish | @@ -31,6 +32,250 @@ --- +## v1.3.0 - January 1, 2026 + +### Major Release: Automation Overhaul & AI Provider Integrations + +This release introduces significant improvements to the automation pipeline, adds support for multiple AI providers (Anthropic Claude, Bria AI), implements a centralized Model Registry for cost calculation, and includes comprehensive UI/UX improvements with CSS globalization. + +--- + +### 🚀 Automation Pipeline Overhaul + +**Complete Rewrite of Automation System:** +- Redesigned 7-stage automation pipeline with accurate progress tracking +- Added initial snapshot capture for proper progress percentage calculation +- Implemented incremental saves after each item for real-time UI updates +- Fixed progress calculation using stage-specific keys instead of summing all numeric values +- Stage 7 converted from "Manual Review Gate" to "Auto-Approve and Publish" + +**New Components:** +- `GlobalProgressBar.tsx` - Full pipeline progress across all 7 stages with segmented visualization +- `CurrentProcessingCardV2.tsx` - Simplified real-time progress card + +**Backend Improvements:** +- `automation_service.py` - Complete rewrite with proper item counting +- `_capture_initial_snapshot()` - Captures queue sizes at run start +- `_update_snapshot_after_stage()` - Updates counts for cascading stages +- Fixed `_get_processed_count()` to use stage-specific keys +- Credits now tracked via `CreditUsageLog` (matches billing source of truth) +- New `run_progress` API endpoint for unified progress data + +**Stage Card UI Improvements:** +- New layout with icon, stage number, and function name in header +- Larger Pending/Processed counts with semantic colors +- Breathing circle indicator for active stages +- Progress bar with stage-specific gradient colors +- Credits and duration displayed inline + +**Pipeline Overview Fixes:** +- Stage 1 pending now matches Keywords "New" count (removed `cluster__isnull` filter) +- Stage 3 uses `status='new'` instead of `status='approved'` +- Stage 4 uses `status='queued'` matching pipeline_overview + +--- + +### 🤖 AI Provider Integrations + +**Anthropic Claude Support:** +- Full integration with Claude API (claude-3-5-sonnet, claude-3-opus, claude-3-haiku) +- New `_make_anthropic_request()` method in AICore +- Cost calculation using ModelRegistry with fallback rates +- Token tracking (input/output) for accurate billing +- Added `anthropic_api_key` to GlobalIntegrationSettings +- Migration: `0013_add_anthropic_integration.py` + +**Bria AI Image Generation:** +- Support for Bria AI image generation API +- New `_generate_image_bria()` method in AICore +- Models: bria-2.3, bria-2.3-fast, bria-2.2 +- Added `bria_api_key` to GlobalIntegrationSettings +- Migration: `0012_add_bria_integration.py` + +--- + +### 📊 Model Registry Service + +**New Centralized Model Configuration:** +- `backend/igny8_core/ai/model_registry.py` - Central registry with caching +- Database-driven model configs via `AIModelConfig` +- Fallback to `constants.py` for backward compatibility +- 5-minute cache TTL for performance + +**Features:** +- `ModelRegistry.get_model(model_id)` - Get config from cache/DB/constants +- `ModelRegistry.get_rate(model_id, rate_type)` - Get input/output/image rates +- `ModelRegistry.calculate_cost()` - Unified cost calculation +- `ModelRegistry.list_models()` - List all active models with filters +- `ModelRegistry.validate_model()` - Check if model is valid and active + +**Migration:** +- `0009_seed_ai_model_configs.py` - Seeds initial model configurations + +--- + +### 🎨 CSS Globalization & Design System + +**Semantic Color Standardization:** +- Replaced hardcoded Tailwind colors with semantic tokens throughout: + - `blue-*` → `brand-*` (primary brand color) + - `green-*` → `success-*` (success states) + - `red-*` → `error-*` (error states) + - `yellow-*` → `warning-*` (warning states) + - `slate-*` → `gray-*` (neutral grays) + +**Files Updated (100+ files):** +- All Automation components +- All Billing components +- All Auth components (SignIn, SignUp forms) +- All Marketing pages +- Account pages (Settings, Billing, Usage, Notifications) +- Planner, Writer, Thinker pages +- Common components (ProgressModal, ColumnSelector, etc.) + +**CSS Cleanup:** +- Removed `igny8-colors.css` (617 lines deleted) +- Consolidated color definitions in `tokens.css` +- Updated `index.css` with streamlined utilities + +--- + +### 📝 Prompt System Enhancements + +**Prompt Prefix Tracking:** +- New prompt prefix system for tracking prompt source +- `##GP01-Clustering` (Global Prompt) vs `##CP01-Clustering` (Custom Prompt) +- `get_prompt_with_metadata()` returns prompt source information +- `get_prompt_prefix_for_function()` for tracking without full prompt fetch + +**Prompt Types Added:** +- `PROMPT_PREFIX_MAP` with numbered prefixes for all 11 prompt types +- Enables tracking whether AI used global or customized prompts + +--- + +### 📄 New Pages & Routes + +**Legal Pages:** +- `frontend/src/pages/legal/Privacy.tsx` - Privacy Policy page +- `frontend/src/pages/legal/Terms.tsx` - Terms of Service page +- Routes added to App.tsx + +--- + +### 🔧 Backend Improvements + +**New Migrations:** +- `0006_automationrun_initial_snapshot.py` - Stores snapshot at run start +- `0008_global_payment_methods.py` - Global payment method support +- `0009_seed_ai_model_configs.py` - AI model configurations +- `0011_disable_phase2_modules.py` - Phase 2 module toggles +- `0012_add_bria_integration.py` - Bria AI integration settings +- `0013_add_anthropic_integration.py` - Anthropic integration settings + +**API Enhancements:** +- New `run_progress` endpoint in automation views +- Account API expansions for usage tracking +- Billing API improvements for payment methods + +**Planning Module:** +- Added `signals.py` for planning model signals +- Improved clustering service +- Fixed keyword status queries + +--- + +### 🐛 Bug Fixes + +**Automation:** +- Fixed progress percentage calculation (was summing all result values) +- Fixed Stage 1 pending count (was filtering by cluster__isnull) +- Fixed remaining count calculation during active processing +- Fixed credits tracking to use CreditUsageLog + +**UI/UX:** +- Fixed PageHeader description rendering (removed from body, title in AppHeader) +- Fixed account page JSX syntax errors causing module import failures +- Fixed notification count display location +- Fixed duplicate headings on Plans & Billing page + +**Backend:** +- Fixed content validation service +- Fixed clustering service dependencies +- Fixed writer admin configurations + +--- + +### 📚 Documentation + +**New Docs:** +- `docs/fixes/component-audit-report.md` +- `docs/fixes/design-verification-report.md` +- `docs/fixes/footer-widget-pagination-fix.md` +- `docs/fixes/phase2-module-activation.md` +- `docs/plans/MASTER-IMPLEMENTATION-PLAN.md` +- `docs/plans/flexible-model-configuration-plan.md` +- `docs/plans/final-fixes.md` + +**Moved/Reorganized:** +- `TODOS.md` → `docs/plans/TODOS.md` +- `immediate-updates-completed.md` → `docs/immediate-updates-completed.md` + +--- + +### Files Summary + +**New Files (15):** +``` +frontend/src/components/Automation/GlobalProgressBar.tsx (248 lines) +frontend/src/components/Automation/CurrentProcessingCardV2.tsx (344 lines) +frontend/src/pages/legal/Privacy.tsx (260 lines) +frontend/src/pages/legal/Terms.tsx (186 lines) +frontend/src/hooks/useWorkflowStats.ts (294 lines) +frontend/src/services/automationService.ts (71 lines) +backend/igny8_core/ai/model_registry.py (339 lines) +backend/igny8_core/business/planning/signals.py (130 lines) +backend/igny8_core/migrations/0006_automationrun_initial_snapshot.py +backend/igny8_core/business/billing/migrations/0008_global_payment_methods.py +backend/igny8_core/business/billing/migrations/0009_seed_ai_model_configs.py +backend/igny8_core/modules/system/migrations/0011_disable_phase2_modules.py +backend/igny8_core/modules/system/migrations/0012_add_bria_integration.py +backend/igny8_core/modules/system/migrations/0013_add_anthropic_integration.py +🚀 AUTOMATION PAGE MASTER FIX PLAN.md +``` + +**Major Changes (20+ files):** +- `frontend/src/pages/Automation/AutomationPage.tsx` - Complete UI overhaul +- `backend/igny8_core/ai/ai_core.py` - Anthropic & Bria integration (+557 lines) +- `backend/igny8_core/business/automation/services/automation_service.py` - Pipeline rewrite (+619 lines) +- `backend/igny8_core/business/automation/views.py` - New endpoints (+241 lines) +- `frontend/src/index.css` - CSS consolidation +- 100+ files updated for semantic color tokens + +**Deleted Files:** +- `frontend/src/styles/igny8-colors.css` (617 lines) +- `COMPREHENSIVE-AUDIT-REPORT.md` +- Multiple test files in `frontend/src/__tests__/sites/` +- `to-do-s/` folder contents (migrated to docs) + +--- + +### Git Reference +```bash +# Commits since v1.2.2 (20 commits) +git log 0605f650..HEAD --oneline + +# Key commits: +# ea9125b8 - Automation revamp part 1 +# 53fdebf7 - automation and ai and some planning and fixes +# c91175fd - styling css globalization +# 2af7bb72 - master plan implementation +# 89b64cd7 - modules widgets and colors and styling +# 0d3e25e5 - automation and other pages updates +``` + +--- + ## v1.2.2 - December 28, 2025 ### Full Notifications Page Implementation diff --git a/docs/00-SYSTEM/ARCHITECTURE.md b/docs/00-SYSTEM/ARCHITECTURE.md index aa384984..484abf27 100644 --- a/docs/00-SYSTEM/ARCHITECTURE.md +++ b/docs/00-SYSTEM/ARCHITECTURE.md @@ -1,7 +1,7 @@ # System Architecture -**Last Verified:** December 27, 2025 -**Version:** 1.2.0 +**Last Verified:** January 1, 2026 +**Version:** 1.3.0 **Backend Path:** `backend/igny8_core/` **Frontend Path:** `frontend/src/` @@ -67,7 +67,7 @@ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ EXTERNAL SERVICES │ -│ OpenAI API (GPT-4, DALL-E) │ Runware │ WordPress Sites │ +│ OpenAI │ Anthropic │ Runware │ Bria │ WordPress Sites │ └─────────────────────────────────────────────────────────────────┘ ``` @@ -97,6 +97,7 @@ backend/igny8_core/ │ ├── engine.py # AIEngine orchestrator │ ├── functions/ # AutoCluster, GenerateIdeas, GenerateContent, etc. │ ├── registry.py # Function registry +│ ├── model_registry.py # Model Registry service (v1.3.0) │ └── progress.py # Progress tracking │ ├── modules/ # Feature modules (API layer) @@ -192,6 +193,7 @@ frontend/src/ │ └── AppSidebar.tsx │ ├── hooks/ # Custom hooks +│ └── useWorkflowStats.ts # Automation stats hook (v1.3.0) ├── context/ # React contexts └── utils/ # Utility functions ``` @@ -299,10 +301,46 @@ AI keys are stored in `GlobalIntegrationSettings` (database), not env vars. --- +## Model Registry (v1.3.0) + +The Model Registry provides centralized AI model configuration with database-stored pricing and settings: + +```python +# backend/igny8_core/ai/model_registry.py +from igny8_core.modules.system.models import AIModelConfig + +class ModelRegistry: + """Centralized AI model configuration service""" + + @classmethod + def get_model(cls, model_id: str) -> AIModelConfig: + """Get model config by ID""" + return AIModelConfig.objects.get(model_id=model_id) + + @classmethod + def get_active_models_by_type(cls, model_type: str) -> QuerySet: + """Get all active models for a type (text, image, etc.)""" + return AIModelConfig.objects.filter( + model_type=model_type, + is_active=True + ) +``` + +**Supported Providers:** +| Provider | Types | Integration | +|----------|-------|-------------| +| OpenAI | Text, Image | GPT-4o, GPT-4-turbo, DALL-E 3 | +| Anthropic | Text | Claude 3.5 Sonnet, Claude 3 Opus | +| Runware | Image | RunwareFLUX, SD 3.5 | +| Bria | Image | Bria 2.3 | + +--- + ## Planned Changes | Feature | Status | Description | |---------|--------|-------------| -| AIModelConfig Database | 🔜 Planned | Move AI model pricing from constants to database | +| ~~AIModelConfig Database~~ | ✅ v1.3.0 | ~~Move AI model pricing from constants to database~~ | +| ~~Multi-provider AI~~ | ✅ v1.3.0 | ~~Support for Anthropic, Bria~~ | | Module Guard Extension | 🔜 Planned | Extend linker/optimizer disable to all pages (currently sidebar only) | -| Multi-provider AI | 🔜 Planned | Support for Anthropic, Google AI | +| Google AI Integration | 🔜 Planned | Support for Gemini models | diff --git a/docs/10-MODULES/AUTOMATION.md b/docs/10-MODULES/AUTOMATION.md index cb9d4b8e..502eab02 100644 --- a/docs/10-MODULES/AUTOMATION.md +++ b/docs/10-MODULES/AUTOMATION.md @@ -1,6 +1,7 @@ # Automation Module -**Last Verified:** December 25, 2025 +**Last Verified:** January 1, 2026 +**Version:** 1.3.0 **Status:** ✅ Active **Backend Path:** `backend/igny8_core/business/automation/` **Frontend Path:** `frontend/src/pages/Automation/` @@ -16,6 +17,8 @@ | Logger | `business/automation/services/automation_logger.py` | `AutomationLogger` | | Celery Tasks | `business/automation/tasks.py` | `run_automation_task`, `check_scheduled_automations` | | Frontend | `pages/Automation/AutomationPage.tsx` | Main automation UI | +| Progress Bar | `components/Automation/GlobalProgressBar.tsx` | Full pipeline progress | +| Processing Card | `components/Automation/CurrentProcessingCard.tsx` | Real-time progress | --- @@ -24,7 +27,7 @@ The Automation module runs the complete 7-stage content pipeline automatically: ``` -Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Images → Review +Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Images → Published ``` --- @@ -39,7 +42,9 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag | 4 | Tasks → Content | `GenerateContentFunction` | Per 100 words | | 5 | Content → Image Prompts | `GenerateImagePromptsFunction` | Per prompt | | 6 | Image Prompts → Images | `process_image_generation_queue` | Per image | -| 7 | Manual Review Gate | None | None | +| 7 | Review → Published | None (auto-approve) | None | + +**Note:** Stage 7 changed from "Manual Review Gate" to auto-approve and publish in v1.3.0. --- @@ -79,6 +84,7 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag | cancelled_at | DateTime | Cancel time (nullable) | | completed_at | DateTime | Completion time (nullable) | | total_credits_used | Decimal | Total credits consumed | +| **initial_snapshot** | JSON | **v1.3.0** Queue sizes at run start | | stage_1_result | JSON | Stage 1 results | | stage_2_result | JSON | Stage 2 results | | stage_3_result | JSON | Stage 3 results | @@ -100,6 +106,7 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag | GET | `/api/v1/automation/current_run/` | Get current run | Run status/progress | | GET | `/api/v1/automation/pipeline_overview/` | Get pipeline | Stage status counts | | GET | `/api/v1/automation/current_processing/` | Get processing | Live processing status | +| **GET** | `/api/v1/automation/run_progress/` | **v1.3.0** | Unified progress data | | POST | `/api/v1/automation/pause/` | Pause run | Pause after current item | | POST | `/api/v1/automation/resume/` | Resume run | Resume from saved stage | | POST | `/api/v1/automation/cancel/` | Cancel run | Cancel after current item | @@ -109,6 +116,22 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag **Query Parameters:** All require `?site_id=`, run-specific require `?run_id=` +### run_progress Endpoint (v1.3.0) + +Returns unified progress data for frontend: +```json +{ + "run": { "run_id": "...", "status": "running", "current_stage": 3 }, + "global_progress": { "total_items": 100, "completed_items": 45, "percentage": 45 }, + "stages": [ + { "number": 1, "status": "completed", "input_count": 50, "processed_count": 50 }, + ... + ], + "metrics": { "credits_used": 120, "duration_seconds": 3600 }, + "initial_snapshot": { "stage_1_initial": 50, ... } +} +``` + --- ## Execution Flow @@ -118,10 +141,11 @@ Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Imag 1. User clicks "Run Now" on frontend 2. Frontend calls `POST /automation/run_now/?site_id=X` 3. Backend acquires cache lock `automation_lock_{site_id}` -4. Estimates credits required (1.2x buffer) -5. Validates balance >= estimate -6. Creates `AutomationRun` record -7. Enqueues `run_automation_task` Celery task +4. **v1.3.0:** Captures initial snapshot with `_capture_initial_snapshot()` +5. Estimates credits required (1.2x buffer) +6. Validates balance >= estimate +7. Creates `AutomationRun` record +8. Enqueues `run_automation_task` Celery task 8. Returns run ID immediately ### Stage Execution diff --git a/docs/10-MODULES/INTEGRATIONS.md b/docs/10-MODULES/INTEGRATIONS.md index 9c86d1be..0af713d9 100644 --- a/docs/10-MODULES/INTEGRATIONS.md +++ b/docs/10-MODULES/INTEGRATIONS.md @@ -1,6 +1,7 @@ # Integrations Module -**Last Verified:** December 25, 2025 +**Last Verified:** January 1, 2026 +**Version:** 1.3.0 **Status:** ✅ Active **Backend Path:** `backend/igny8_core/modules/integration/` + `backend/igny8_core/business/integration/` **Frontend Path:** `frontend/src/pages/Settings/IntegrationSettings.tsx` @@ -15,13 +16,66 @@ | Views | `modules/integration/views.py` | `SiteIntegrationViewSet` | | Webhooks | `modules/integration/webhooks.py` | `wordpress_webhook` | | Services | `business/integration/services/*.py` | Sync services | +| AI Core | `ai/ai_core.py` | OpenAI, Anthropic, Runware, Bria clients | +| Model Registry | `ai/model_registry.py` | Centralized model configs | | Frontend | `pages/Settings/IntegrationSettings.tsx` | Integration UI | --- +## AI Provider Integrations (v1.3.0) + +### Supported Providers + +| Provider | Type | Models | API Key Field | +|----------|------|--------|---------------| +| OpenAI | Text/Image | gpt-4o, gpt-4o-mini, dall-e-3 | `openai_api_key` | +| Anthropic | Text | claude-3-5-sonnet, claude-3-opus, claude-3-haiku | `anthropic_api_key` | +| Runware | Image | runware-v2 | `runware_api_key` | +| Bria AI | Image | bria-2.3, bria-2.3-fast, bria-2.2 | `bria_api_key` | + +### GlobalIntegrationSettings + +Located in `modules/system/global_settings_models.py`: + +| Field | Type | Purpose | +|-------|------|---------| +| openai_api_key | CharField | OpenAI API key | +| anthropic_api_key | CharField | Anthropic API key (v1.3.0) | +| runware_api_key | CharField | Runware API key | +| bria_api_key | CharField | Bria AI API key (v1.3.0) | +| default_text_model | CharField | Default text model | +| default_image_model | CharField | Default image model | +| default_image_provider | CharField | openai/runware/bria | + +### Model Registry + +Centralized model configuration with caching: + +```python +from igny8_core.ai.model_registry import ModelRegistry + +# Get model config +model = ModelRegistry.get_model('gpt-4o-mini') + +# Calculate cost +cost = ModelRegistry.calculate_cost('gpt-4o-mini', input_tokens=1000, output_tokens=500) + +# List all models +models = ModelRegistry.list_models(model_type='text', provider='openai') +``` + +### Migrations + +- `0012_add_bria_integration.py` - Adds Bria AI integration settings +- `0013_add_anthropic_integration.py` - Adds Anthropic integration settings +- `0009_seed_ai_model_configs.py` - Seeds model configurations + +--- + ## Purpose The Integrations module manages: +- AI provider connections (OpenAI, Anthropic, Runware, Bria) - WordPress site connections - Two-way content synchronization - Webhook handling diff --git a/docs/30-FRONTEND/PAGES.md b/docs/30-FRONTEND/PAGES.md index 011f1643..1bb81f92 100644 --- a/docs/30-FRONTEND/PAGES.md +++ b/docs/30-FRONTEND/PAGES.md @@ -1,7 +1,7 @@ # Frontend Pages & Routes -**Last Verified:** December 27, 2025 -**Version:** 1.2.0 +**Last Verified:** January 1, 2026 +**Version:** 1.3.0 **Framework:** React 19 + TypeScript + React Router 6 + Vite --- @@ -27,6 +27,15 @@ Routes defined in `/frontend/src/App.tsx`: --- +## Legal Pages (v1.3.0) + +| Route | File | Description | +|-------|------|-------------| +| `/privacy` | `legal/Privacy.tsx` | Privacy Policy | +| `/terms` | `legal/Terms.tsx` | Terms of Service | + +--- + ## Dashboard | Route | File | Description |