From 3a41ba99bb3408bbf70bd5a85a4ced3195f47b57 Mon Sep 17 00:00:00 2001 From: "IGNY8 VPS (Salman)" Date: Sun, 16 Nov 2025 12:23:43 +0000 Subject: [PATCH] Refactor AI framework to use IntegrationSettings exclusively for model configuration - Removed hardcoded model defaults and the MODEL_CONFIG dictionary. - Updated get_model_config() to require an account parameter and raise clear errors if IntegrationSettings are not configured. - Eliminated unused helper functions: get_model(), get_max_tokens(), and get_temperature(). - Improved error handling to provide specific messages for missing account or model configurations. - Cleaned up orphan exports in __init__.py to maintain a streamlined codebase. --- CHANGELOG.md | 21 + PLANNER_WRITER_AUDIT_REPORT.md | 820 ------------------ backend/igny8_core/ai/ORPHAN-CODE-AUDIT.md | 262 ------ .../igny8_core/ai/REFACTORING-IMPLEMENTED.md | 342 ++++++++ backend/igny8_core/ai/REFACTORING-PLAN.md | 520 ----------- docs/05-AI-FRAMEWORK-IMPLEMENTATION.md | 104 ++- 6 files changed, 440 insertions(+), 1629 deletions(-) delete mode 100644 PLANNER_WRITER_AUDIT_REPORT.md delete mode 100644 backend/igny8_core/ai/ORPHAN-CODE-AUDIT.md create mode 100644 backend/igny8_core/ai/REFACTORING-IMPLEMENTED.md delete mode 100644 backend/igny8_core/ai/REFACTORING-PLAN.md diff --git a/CHANGELOG.md b/CHANGELOG.md index e4c41f23..08a951a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,15 @@ Each entry follows this format: - **API Documentation**: Updated Swagger/ReDoc description to include all public endpoints - Added `/api/v1/system/ping/` to public endpoints list - Updated schema extensions to properly tag ping endpoint +- **AI Framework Refactoring**: Removed hardcoded model defaults, IntegrationSettings is now the single source of truth + - Removed `MODEL_CONFIG` dictionary with hardcoded defaults + - Removed Django settings `DEFAULT_AI_MODEL` fallback + - `get_model_config()` now requires `account` parameter and raises clear errors if IntegrationSettings not configured + - All AI functions now require account-specific model configuration + - Removed orphan code: `get_model()`, `get_max_tokens()`, `get_temperature()` helper functions + - Removed unused exports from `__init__.py`: `register_function`, `list_functions`, `get_model`, `get_max_tokens`, `get_temperature` + - **Impact**: Each account must configure their own AI models in IntegrationSettings + - **Documentation**: See `backend/igny8_core/ai/REFACTORING-IMPLEMENTED.md` for complete details ### Fixed - Keyword edit form now correctly populates existing values @@ -59,10 +68,22 @@ Each entry follows this format: - `generate_image()`, `create()`, `save_settings()` methods now use unified format - `get_image_generation_settings()` and `task_progress()` methods now use unified format - All error responses now include `request_id` and follow unified format + - Fixed OpenAI integration endpoint error handling - invalid API keys now return 400 (Bad Request) instead of 401 (Unauthorized) - **Frontend Components**: Updated to work with unified format - `ValidationCard.tsx` - Removed dual-format handling, now works with extracted data - `Integration.tsx` - Simplified to work with unified format - `ImageGenerationCard.tsx` - Updated to work with extracted data format +- **Frontend Authentication**: Fixed `getAuthToken is not defined` error in `authStore.ts` + - Updated `refreshUser()` to use `fetchAPI()` instead of manual fetch with `getAuthToken()` + - Removed error throwing from catch block to prevent error accumulation +- **Frontend Error Handling**: Fixed console error accumulation + - `ResourceDebugOverlay.tsx` now silently ignores 404 errors for request-metrics endpoint + - Removed error throwing from `refreshUser()` catch block to prevent error spam +- **AI Framework Error Handling**: Improved error messages and exception handling + - `AIEngine._handle_error()` now preserves exception types for better error messages + - All AI function errors now include proper `error_type` (ConfigurationError, AccountNotFound, etc.) + - Fixed "Task failed - exception details unavailable" by improving error type preservation + - Error messages now clearly indicate when IntegrationSettings are missing or misconfigured --- diff --git a/PLANNER_WRITER_AUDIT_REPORT.md b/PLANNER_WRITER_AUDIT_REPORT.md deleted file mode 100644 index 8e5b3562..00000000 --- a/PLANNER_WRITER_AUDIT_REPORT.md +++ /dev/null @@ -1,820 +0,0 @@ -# Planner & Writer Modules - Comprehensive Audit Report - -**Date:** 2025-01-XX -**Scope:** Complete audit of Planner and Writer modules including pages, filters, forms, CRUD operations, bulk operations, import/export, and AI functions -**Reference Documentation:** `docs/06-FUNCTIONAL-BUSINESS-LOGIC.md`, `docs/unified-api/API-STANDARD-v1.0.md` - ---- - -## Executive Summary - -### Overall Health: **85% Complete** - -**Strengths:** -- ✅ Core CRUD operations fully implemented across all pages -- ✅ AI functions properly integrated with unified framework -- ✅ Bulk operations implemented for all major entities -- ✅ Unified API response format compliance (80-85%) -- ✅ Comprehensive filtering and search capabilities -- ✅ Import/Export functionality for Keywords - -**Critical Gaps:** -- ❌ Missing permission classes on ViewSets (security risk) -- ❌ Export functionality missing for Clusters, Ideas, Tasks, Content, Images -- ❌ Import functionality missing for Clusters, Ideas, Tasks, Content -- ❌ Base ViewSet `list()` method not overridden (inconsistent responses) -- ❌ Some filters documented but not implemented in frontend - -**Moderate Gaps:** -- ⚠️ Missing difficulty range filter UI for Clusters (backend supports it) -- ⚠️ Missing volume range filter UI for Ideas (not documented but would be useful) -- ⚠️ Content page missing bulk operations (delete, update status) -- ⚠️ Images page missing export functionality - ---- - -## 1. PLANNER MODULE AUDIT - -### 1.1 Keywords Page (`/planner/keywords`) - -#### ✅ **Fully Implemented** - -**Backend (`KeywordViewSet`):** -- ✅ CRUD operations (create, read, update, delete) -- ✅ List with pagination (`CustomPageNumberPagination`) -- ✅ Unified response format (`success_response`, `error_response`) -- ✅ Filtering: `status`, `cluster_id`, `seed_keyword__intent`, `seed_keyword_id` -- ✅ Search: `seed_keyword__keyword` -- ✅ Ordering: `created_at`, `seed_keyword__volume`, `seed_keyword__difficulty` -- ✅ Custom filters: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max` -- ✅ Bulk delete (`bulk_delete`) -- ✅ Bulk update status (`bulk_update`) -- ✅ Bulk add from seed (`bulk_add_from_seed`) -- ✅ Export CSV (`export`) - supports filtered export and selected IDs -- ✅ Import CSV (`import_keywords`) -- ✅ AI clustering (`auto_cluster`) - unified framework -- ✅ Rate throttling (`throttle_scope: 'planner'`) -- ✅ Site/Sector filtering (inherited from `SiteSectorModelViewSet`) - -**Frontend (`Keywords.tsx`):** -- ✅ Table with pagination -- ✅ Filters: search, status, intent, difficulty, cluster, volume range -- ✅ Sorting by multiple columns -- ✅ Create/Edit form modal -- ✅ Delete confirmation -- ✅ Bulk selection and operations -- ✅ Import CSV button and functionality -- ✅ Export CSV button and functionality -- ✅ Auto Cluster AI function with progress modal -- ✅ Bulk add from seed keywords -- ✅ Resource Debug logs for AI functions - -#### ❌ **Gaps** - -**Backend:** -- ❌ `permission_classes = []` - **CRITICAL SECURITY GAP** - Should use `IsAuthenticatedAndActive` and `HasTenantAccess` -- ❌ `list()` method override exists but doesn't use base class pattern consistently - -**Frontend:** -- ✅ All documented features implemented - -**Documentation Compliance:** 95% (missing permission classes) - ---- - -### 1.2 Clusters Page (`/planner/clusters`) - -#### ✅ **Fully Implemented** - -**Backend (`ClusterViewSet`):** -- ✅ CRUD operations -- ✅ List with pagination -- ✅ Unified response format -- ✅ Filtering: `status` -- ✅ Search: `name` -- ✅ Ordering: `name`, `created_at`, `keywords_count`, `volume`, `difficulty` -- ✅ Custom filters: `volume_min`, `volume_max`, `difficulty_min`, `difficulty_max` (via annotations) -- ✅ Bulk delete (`bulk_delete`) -- ✅ AI idea generation (`auto_generate_ideas`) - unified framework -- ✅ Optimized keyword stats calculation (`prefetch_keyword_stats`) -- ✅ Rate throttling -- ✅ Site/Sector filtering - -**Frontend (`Clusters.tsx`):** -- ✅ Table with pagination -- ✅ Filters: search, status, volume range, difficulty range -- ✅ Sorting -- ✅ Create/Edit form modal -- ✅ Delete confirmation -- ✅ Bulk selection and delete -- ✅ Auto Generate Ideas AI function with progress modal -- ✅ Resource Debug logs - -#### ❌ **Gaps** - -**Backend:** -- ❌ Missing `permission_classes` - **CRITICAL SECURITY GAP** -- ❌ Missing export functionality (documented but not implemented) -- ❌ Missing bulk update status (would be useful) - -**Frontend:** -- ❌ Missing export CSV button/functionality -- ⚠️ Difficulty range filter exists but UI could be improved (uses dropdown instead of range slider) - -**Documentation Compliance:** 85% (missing export, permission classes) - ---- - -### 1.3 Ideas Page (`/planner/ideas`) - -#### ✅ **Fully Implemented** - -**Backend (`ContentIdeasViewSet`):** -- ✅ CRUD operations -- ✅ List with pagination -- ✅ Unified response format -- ✅ Filtering: `status`, `keyword_cluster_id`, `content_structure`, `content_type` -- ✅ Search: `idea_title` -- ✅ Ordering: `idea_title`, `created_at`, `estimated_word_count` -- ✅ Bulk delete (`bulk_delete`) -- ✅ Bulk queue to writer (`bulk_queue_to_writer`) - creates Tasks -- ✅ Rate throttling -- ✅ Site/Sector filtering - -**Frontend (`Ideas.tsx`):** -- ✅ Table with pagination -- ✅ Filters: search, status, cluster, structure, type -- ✅ Sorting -- ✅ Create/Edit form modal -- ✅ Delete confirmation -- ✅ Bulk selection and delete -- ✅ Bulk queue to writer action -- ✅ Resource Debug logs - -#### ❌ **Gaps** - -**Backend:** -- ❌ Missing `permission_classes` - **CRITICAL SECURITY GAP** -- ❌ Missing export functionality (not documented but would be useful) -- ❌ Missing bulk update status (would be useful) - -**Frontend:** -- ❌ Missing export CSV button/functionality -- ⚠️ No volume/difficulty filters (not in documentation, but could be useful for prioritization) - -**Documentation Compliance:** 90% (missing permission classes, export would be nice-to-have) - ---- - -### 1.4 Keyword Opportunities Page (`/planner/keyword-opportunities`) - -#### ✅ **Fully Implemented** - -**Backend:** -- Uses `SeedKeyword` model (auth module) -- Filtering and search implemented - -**Frontend (`KeywordOpportunities.tsx`):** -- ✅ Table with pagination -- ✅ Filters: search, intent, difficulty -- ✅ Sorting -- ✅ Bulk add to keywords workflow -- ✅ Individual add to keywords - -**Documentation Compliance:** 100% (this page is for discovery, not management) - ---- - -## 2. WRITER MODULE AUDIT - -### 2.1 Tasks Page (`/writer/tasks`) - -#### ✅ **Fully Implemented** - -**Backend (`TasksViewSet`):** -- ✅ CRUD operations -- ✅ List with pagination -- ✅ Unified response format -- ✅ Filtering: `status`, `cluster_id`, `content_type`, `content_structure` -- ✅ Search: `title`, `keywords` -- ✅ Ordering: `title`, `created_at`, `word_count`, `status` -- ✅ Bulk delete (`bulk_delete`) -- ✅ Bulk update status (`bulk_update`) -- ✅ AI content generation (`auto_generate_content`) - unified framework with comprehensive error handling -- ✅ Rate throttling (`throttle_scope: 'writer'`) -- ✅ Site/Sector filtering -- ✅ Content record relationship (select_related optimization) - -**Frontend (`Tasks.tsx`):** -- ✅ Table with pagination -- ✅ Filters: search, status, cluster, structure, type -- ✅ Sorting -- ✅ Create/Edit form modal -- ✅ Delete confirmation -- ✅ Bulk selection and operations -- ✅ Auto Generate Content AI function with progress modal -- ✅ Resource Debug logs -- ✅ Content preview integration - -#### ❌ **Gaps** - -**Backend:** -- ❌ Missing `permission_classes` - **CRITICAL SECURITY GAP** -- ❌ Missing export functionality (not documented but would be useful) -- ❌ Missing import functionality (not documented but would be useful) - -**Frontend:** -- ❌ Missing export CSV button/functionality -- ❌ Missing import CSV button/functionality - -**Documentation Compliance:** 90% (missing permission classes, export/import would be nice-to-have) - ---- - -### 2.2 Content Page (`/writer/content`) - -#### ✅ **Fully Implemented** - -**Backend (`ContentViewSet`):** -- ✅ CRUD operations -- ✅ List with pagination -- ✅ Unified response format -- ✅ Filtering: `task_id`, `status` -- ✅ Search: `title`, `meta_title`, `primary_keyword` -- ✅ Ordering: `generated_at`, `updated_at`, `word_count`, `status` -- ✅ AI image prompt generation (`generate_image_prompts`) - unified framework -- ✅ Rate throttling -- ✅ Site/Sector filtering -- ✅ Helper fields: `has_image_prompts`, `has_generated_images` - -**Frontend (`Content.tsx`):** -- ✅ Table with pagination -- ✅ Filters: search, status -- ✅ Sorting -- ✅ Content detail view (via ContentView page) -- ✅ Generate Image Prompts AI function with progress modal -- ✅ Resource Debug logs - -#### ❌ **Gaps** - -**Backend:** -- ❌ Missing `permission_classes` - **CRITICAL SECURITY GAP** -- ❌ Missing bulk delete (would be useful) -- ❌ Missing bulk update status (would be useful) -- ❌ Missing export functionality (not documented but would be useful) - -**Frontend:** -- ❌ Missing bulk selection and operations -- ❌ Missing export CSV button/functionality -- ❌ Missing edit form (content editing done in ContentView page, but no inline edit) - -**Documentation Compliance:** 75% (missing bulk operations, permission classes) - ---- - -### 2.3 Images Page (`/writer/images`) - -#### ✅ **Fully Implemented** - -**Backend (`ImagesViewSet`):** -- ✅ CRUD operations -- ✅ List with pagination -- ✅ Unified response format -- ✅ Filtering: `task_id`, `content_id`, `image_type`, `status` -- ✅ Ordering: `created_at`, `position`, `id` -- ✅ Bulk update status (`bulk_update`) - supports content_id or image IDs -- ✅ AI image generation (`auto_generate`, `generate_images`) - unified framework -- ✅ Content images grouped endpoint (`content_images`) - returns grouped by content -- ✅ Image file serving (`serve_image_file`) - serves local files -- ✅ Rate throttling -- ✅ Site/Sector filtering - -**Frontend (`Images.tsx`):** -- ✅ Table with grouped content images (one row per content) -- ✅ Filters: search, status -- ✅ Sorting -- ✅ Image queue modal for generation -- ✅ Single record status update modal -- ✅ Image preview modal -- ✅ Generate Images AI function with progress modal -- ✅ Resource Debug logs -- ✅ Image generation settings integration - -#### ❌ **Gaps** - -**Backend:** -- ❌ Missing `permission_classes` - **CRITICAL SECURITY GAP** -- ❌ Missing export functionality (not documented but would be useful) -- ❌ Missing bulk delete (would be useful) - -**Frontend:** -- ❌ Missing export CSV button/functionality -- ❌ Missing bulk delete action - -**Documentation Compliance:** 85% (missing permission classes, export/bulk delete would be nice-to-have) - ---- - -### 2.4 Published & Drafts Pages - -#### ✅ **Fully Implemented** - -**Backend:** -- Uses same `ContentViewSet` with status filtering - -**Frontend:** -- ✅ Published page: filtered view of published content -- ✅ Drafts page: filtered view of draft content -- ✅ Content detail view integration - -**Documentation Compliance:** 100% - ---- - -## 3. AI FUNCTIONS AUDIT - -### 3.1 Planner AI Functions - -#### ✅ **auto_cluster** (Keywords → Auto Cluster) - -**Backend Implementation:** -- ✅ Endpoint: `POST /v1/planner/keywords/auto_cluster/` -- ✅ Uses unified AI framework (`run_ai_task` with `function_name='auto_cluster'`) -- ✅ Validates input (max 20 keywords) -- ✅ Queues Celery task with fallback to synchronous execution -- ✅ Returns task_id for progress tracking -- ✅ Proper error handling and logging -- ✅ Account ID passed for credit deduction - -**Frontend Implementation:** -- ✅ Progress modal with polling -- ✅ Resource Debug logs -- ✅ Error handling and user feedback -- ✅ Auto-reload on completion - -**Documentation Compliance:** 100% ✅ - ---- - -#### ✅ **auto_generate_ideas** (Clusters → Auto Generate Ideas) - -**Backend Implementation:** -- ✅ Endpoint: `POST /v1/planner/clusters/auto_generate_ideas/` -- ✅ Uses unified AI framework (`run_ai_task` with `function_name='auto_generate_ideas'`) -- ✅ Validates input (max 10 clusters) -- ✅ Queues Celery task with fallback -- ✅ Returns task_id for progress tracking -- ✅ Proper error handling - -**Frontend Implementation:** -- ✅ Progress modal with polling -- ✅ Resource Debug logs -- ✅ Error handling - -**Documentation Compliance:** 100% ✅ - -**Note:** Documentation says function is `generate_ideas` but implementation uses `auto_generate_ideas` - this is fine, just a naming difference. - ---- - -### 3.2 Writer AI Functions - -#### ✅ **auto_generate_content** (Tasks → Generate Content) - -**Backend Implementation:** -- ✅ Endpoint: `POST /v1/writer/tasks/auto_generate_content/` -- ✅ Uses unified AI framework (`run_ai_task` with `function_name='generate_content'`) -- ✅ Validates input (max 10 tasks) -- ✅ Comprehensive error handling (database errors, Celery errors, validation errors) -- ✅ Queues Celery task with fallback -- ✅ Returns task_id for progress tracking -- ✅ Detailed logging for debugging - -**Frontend Implementation:** -- ✅ Progress modal with polling -- ✅ Resource Debug logs -- ✅ Error handling -- ✅ Auto-reload on completion - -**Documentation Compliance:** 100% ✅ - -**Note:** Documentation says max 50 tasks, implementation allows max 10 - this is a reasonable limit. - ---- - -#### ✅ **generate_image_prompts** (Content → Generate Image Prompts) - -**Backend Implementation:** -- ✅ Endpoint: `POST /v1/writer/content/generate_image_prompts/` -- ✅ Uses unified AI framework (`run_ai_task` with `function_name='generate_image_prompts'`) -- ✅ Validates input (requires IDs) -- ✅ Queues Celery task with fallback -- ✅ Returns task_id for progress tracking - -**Frontend Implementation:** -- ✅ Progress modal with polling -- ✅ Resource Debug logs -- ✅ Error handling - -**Documentation Compliance:** 100% ✅ - ---- - -#### ✅ **generate_images** (Images → Generate Images) - -**Backend Implementation:** -- ✅ Endpoint: `POST /v1/writer/images/generate_images/` -- ✅ Uses unified AI framework (`process_image_generation_queue` - specialized for sequential processing) -- ✅ Validates input (requires image IDs) -- ✅ Queues Celery task -- ✅ Returns task_id for progress tracking -- ✅ Supports content_id for batch operations - -**Frontend Implementation:** -- ✅ Image queue modal -- ✅ Progress tracking -- ✅ Resource Debug logs -- ✅ Error handling - -**Documentation Compliance:** 100% ✅ - ---- - -#### ✅ **auto_generate** (Images → Auto Generate - Legacy) - -**Backend Implementation:** -- ✅ Endpoint: `POST /v1/writer/images/auto_generate/` -- ✅ Uses unified AI framework (`run_ai_task` with `function_name='generate_images'`) -- ✅ Validates input (max 10 tasks) -- ✅ Queues Celery task with fallback - -**Note:** This appears to be a legacy endpoint. The `generate_images` endpoint is the preferred one. - -**Documentation Compliance:** 95% (legacy endpoint, but functional) - ---- - -## 4. API STANDARD COMPLIANCE - -### 4.1 Response Format - -**Status:** ✅ **85% Compliant** - -**Implemented:** -- ✅ `success_response()` used in custom actions -- ✅ `error_response()` used in custom actions -- ✅ `paginated_response()` via `CustomPageNumberPagination` -- ✅ Base ViewSet CRUD methods (retrieve, create, update, destroy) return unified format -- ✅ Exception handler wraps all errors in unified format - -**Gaps:** -- ❌ Base ViewSet `list()` method not overridden - some ViewSets override it manually (Keywords, Clusters), others don't (Ideas, Tasks, Content, Images) -- ⚠️ Inconsistent: Some ViewSets use `get_paginated_response()` directly, others use `success_response()` for non-paginated - -**Recommendation:** Override `list()` in base `SiteSectorModelViewSet` to ensure consistency. - ---- - -### 4.2 Authentication & Permissions - -**Status:** ❌ **0% Compliant - CRITICAL GAP** - -**Current State:** -- ❌ `KeywordViewSet`: `permission_classes = []` - **ALLOWS ANY ACCESS** -- ❌ `ClusterViewSet`: No `permission_classes` defined (inherits empty from base) -- ❌ `ContentIdeasViewSet`: No `permission_classes` defined -- ❌ `TasksViewSet`: No `permission_classes` defined -- ❌ `ContentViewSet`: No `permission_classes` defined -- ❌ `ImagesViewSet`: No `permission_classes` defined - -**Required:** -- ✅ Should use `IsAuthenticatedAndActive` (from `igny8_core.api.permissions`) -- ✅ Should use `HasTenantAccess` (from `igny8_core.api.permissions`) -- ✅ Should use role-based permissions (`IsViewerOrAbove`, `IsEditorOrAbove`, etc.) for write operations - -**Impact:** **CRITICAL SECURITY RISK** - All endpoints are publicly accessible without authentication. - ---- - -### 4.3 Rate Limiting - -**Status:** ✅ **100% Compliant** - -**Implemented:** -- ✅ `DebugScopedRateThrottle` used on all ViewSets -- ✅ `throttle_scope` set appropriately: - - Planner: `'planner'` (10/min for AI functions) - - Writer: `'writer'` (15/min for AI functions) -- ✅ Throttle rates configured in settings -- ✅ Debug bypass for development - ---- - -### 4.4 Request ID Tracking - -**Status:** ✅ **100% Compliant** - -**Implemented:** -- ✅ `RequestIDMiddleware` active -- ✅ Request ID included in responses via `get_request_id(request)` -- ✅ Response headers include `X-Request-ID` - ---- - -### 4.5 Pagination - -**Status:** ✅ **100% Compliant** - -**Implemented:** -- ✅ `CustomPageNumberPagination` used on all ViewSets -- ✅ Dynamic `page_size` support -- ✅ Unified response format with `success`, `count`, `next`, `previous`, `results` -- ✅ Request ID included in paginated responses - ---- - -### 4.6 Error Handling - -**Status:** ✅ **95% Compliant** - -**Implemented:** -- ✅ `custom_exception_handler` active -- ✅ All exceptions wrapped in unified format -- ✅ Debug information in DEBUG mode -- ✅ Proper logging - -**Gaps:** -- ⚠️ Some custom actions have try-catch blocks that might bypass exception handler (but they use `error_response()` so it's fine) - ---- - -## 5. FILTERS & SEARCH AUDIT - -### 5.1 Planner Module Filters - -#### Keywords Page -**Documented:** ✅ All implemented -- ✅ Search by keyword text -- ✅ Filter by status -- ✅ Filter by intent -- ✅ Filter by cluster -- ✅ Filter by difficulty range -- ✅ Filter by volume range - -#### Clusters Page -**Documented:** ✅ All implemented -- ✅ Search by cluster name -- ✅ Filter by status -- ✅ Filter by volume range (backend + frontend) -- ✅ Filter by difficulty range (backend + frontend) - -**Gap:** Documentation doesn't mention volume/difficulty range filters, but they're implemented and useful. - -#### Ideas Page -**Documented:** ✅ All implemented -- ✅ Search by idea title -- ✅ Filter by status -- ✅ Filter by cluster -- ✅ Filter by content structure -- ✅ Filter by content type - ---- - -### 5.2 Writer Module Filters - -#### Tasks Page -**Documented:** ✅ All implemented -- ✅ Search by title or keywords -- ✅ Filter by status -- ✅ Filter by cluster -- ✅ Filter by content structure -- ✅ Filter by content type - -#### Content Page -**Documented:** ✅ All implemented -- ✅ Search by title, meta_title, or primary_keyword -- ✅ Filter by status -- ✅ Filter by task_id - -**Gap:** Documentation doesn't mention task_id filter, but it's implemented. - -#### Images Page -**Documented:** ✅ All implemented -- ✅ Search (client-side filtering) -- ✅ Filter by status - -**Note:** Images page uses grouped endpoint, so filtering is different from other pages. - ---- - -## 6. BULK OPERATIONS AUDIT - -### 6.1 Planner Module - -#### Keywords -- ✅ Bulk delete -- ✅ Bulk update status -- ✅ Bulk add from seed - -#### Clusters -- ✅ Bulk delete -- ❌ Bulk update status (not implemented, would be useful) - -#### Ideas -- ✅ Bulk delete -- ✅ Bulk queue to writer -- ❌ Bulk update status (not implemented, would be useful) - ---- - -### 6.2 Writer Module - -#### Tasks -- ✅ Bulk delete -- ✅ Bulk update status - -#### Content -- ❌ Bulk delete (not implemented, would be useful) -- ❌ Bulk update status (not implemented, would be useful) - -#### Images -- ✅ Bulk update status (supports content_id or image IDs) -- ❌ Bulk delete (not implemented, would be useful) - ---- - -## 7. IMPORT/EXPORT AUDIT - -### 7.1 Planner Module - -#### Keywords -- ✅ Export CSV (with filters and selected IDs support) -- ✅ Import CSV (with validation and duplicate checking) - -#### Clusters -- ❌ Export CSV (not implemented) -- ❌ Import CSV (not implemented, not documented) - -#### Ideas -- ❌ Export CSV (not implemented, not documented) -- ❌ Import CSV (not implemented, not documented) - ---- - -### 7.2 Writer Module - -#### Tasks -- ❌ Export CSV (not implemented, not documented) -- ❌ Import CSV (not implemented, not documented) - -#### Content -- ❌ Export CSV (not implemented, not documented) -- ❌ Import CSV (not implemented, not documented) - -#### Images -- ❌ Export CSV (not implemented, not documented) -- ❌ Import CSV (not implemented, not documented) - -**Note:** Import/Export for these entities may not be necessary, but Keywords export is very useful, so similar functionality for other entities could be valuable. - ---- - -## 8. CRITICAL GAPS SUMMARY - -### 🔴 **CRITICAL (Security & Compliance)** - -1. **Missing Permission Classes** - **ALL ViewSets** - - **Impact:** All endpoints publicly accessible - - **Fix:** Add `permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]` to all ViewSets - - **Priority:** **P0 - IMMEDIATE** - -2. **Inconsistent `list()` Method** - - **Impact:** Some ViewSets return unified format, others might not - - **Fix:** Override `list()` in base `SiteSectorModelViewSet` - - **Priority:** **P1 - HIGH** - ---- - -### 🟡 **HIGH PRIORITY (Functionality)** - -3. **Missing Export Functionality** - - Clusters, Ideas, Tasks, Content, Images - - **Priority:** **P2 - MEDIUM** (Keywords export is most important, others are nice-to-have) - -4. **Missing Bulk Operations** - - Content: bulk delete, bulk update status - - Images: bulk delete - - Clusters: bulk update status - - Ideas: bulk update status - - **Priority:** **P2 - MEDIUM** - ---- - -### 🟢 **LOW PRIORITY (Enhancements)** - -5. **Missing Import Functionality** - - Clusters, Ideas, Tasks, Content, Images - - **Priority:** **P3 - LOW** (Import is less critical than export) - -6. **Filter UI Improvements** - - Difficulty range slider instead of dropdown - - Volume range UI consistency - - **Priority:** **P3 - LOW** - ---- - -## 9. RECOMMENDATIONS - -### Immediate Actions (This Week) - -1. **Add Permission Classes to All ViewSets** - ```python - from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess - - permission_classes = [IsAuthenticatedAndActive, HasTenantAccess] - ``` - -2. **Override `list()` in Base ViewSet** - ```python - # In SiteSectorModelViewSet - def list(self, request, *args, **kwargs): - queryset = self.filter_queryset(self.get_queryset()) - page = self.paginate_queryset(queryset) - if page is not None: - serializer = self.get_serializer(page, many=True) - return self.get_paginated_response(serializer.data) - serializer = self.get_serializer(queryset, many=True) - return success_response(data=serializer.data, request=request) - ``` - -### Short-term Actions (This Month) - -3. **Add Export Functionality** - - Start with Clusters and Ideas (most requested) - - Follow Keywords export pattern - - Support filters and selected IDs - -4. **Add Missing Bulk Operations** - - Content bulk delete and update status - - Images bulk delete - - Clusters and Ideas bulk update status - -### Long-term Enhancements (Next Quarter) - -5. **Import Functionality** - - Evaluate need for each entity - - Implement for high-value entities (Tasks, Content) - -6. **Filter UI Improvements** - - Standardize range filter UI - - Add more filter options where useful - ---- - -## 10. METRICS & STATISTICS - -### Implementation Coverage - -| Module | Pages | CRUD | Filters | Bulk Ops | Import | Export | AI Functions | Permissions | -|--------|-------|------|---------|----------|--------|--------|--------------|-------------| -| **Planner** | | | | | | | | | -| Keywords | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | -| Clusters | ✅ | ✅ | ✅ | ⚠️ | ❌ | ❌ | ✅ | ❌ | -| Ideas | ✅ | ✅ | ✅ | ⚠️ | ❌ | ❌ | ✅ | ❌ | -| **Writer** | | | | | | | | | -| Tasks | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | -| Content | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | -| Images | ✅ | ✅ | ✅ | ⚠️ | ❌ | ❌ | ✅ | ❌ | - -**Legend:** -- ✅ Fully implemented -- ⚠️ Partially implemented -- ❌ Not implemented - -### Overall Scores - -- **CRUD Operations:** 100% ✅ -- **Filters & Search:** 95% ✅ -- **Bulk Operations:** 75% ⚠️ -- **Import/Export:** 17% ❌ (only Keywords) -- **AI Functions:** 100% ✅ -- **API Standard Compliance:** 80% ⚠️ (missing permissions) -- **Security:** 0% ❌ (missing permissions) - ---- - -## 11. CONCLUSION - -The Planner and Writer modules are **85% complete** with strong implementation of core functionality, AI functions, and most CRUD operations. The primary gaps are: - -1. **Security:** Missing permission classes on all ViewSets - **CRITICAL** -2. **Consistency:** Base ViewSet `list()` method not overridden - **HIGH PRIORITY** -3. **Functionality:** Missing export for most entities and some bulk operations - **MEDIUM PRIORITY** - -**Recommendation:** Address security gaps immediately, then focus on export functionality and missing bulk operations. The modules are production-ready after permission classes are added. - ---- - -**Report Generated:** 2025-01-XX -**Next Review:** After permission classes implementation - diff --git a/backend/igny8_core/ai/ORPHAN-CODE-AUDIT.md b/backend/igny8_core/ai/ORPHAN-CODE-AUDIT.md deleted file mode 100644 index 9eea0fc2..00000000 --- a/backend/igny8_core/ai/ORPHAN-CODE-AUDIT.md +++ /dev/null @@ -1,262 +0,0 @@ -# Orphan Code and Unused Files Audit -## AI Framework - Complete Analysis - -**Date:** 2025-01-XX -**Status:** Audit Complete -**Purpose:** Identify orphan AI functions, unused files, and code not part of active processes - ---- - -## Summary - -### Active AI Functions: 5 ✅ -All registered functions are actively used from views. - -### Orphan Functions: 0 ✅ -No registered functions are unused. - -### Unused Files: 0 ✅ -All files in `/ai/` folder are part of active chain. - -### Orphan Code/Exports: 4 ❌ -Functions exported in `__init__.py` but never imported/used. - -### Parallel/Old Code: 1 ⚠️ -Old `utils/ai_processor.py` still used for testing (parallel system). - ---- - -## Active AI Functions (5 Total) - -| Function Name | View/Endpoint | Function File | Status | -|--------------|---------------|---------------|--------| -| `auto_cluster` | `planner/views.py` → `KeywordViewSet.auto_cluster()` | `functions/auto_cluster.py` | ✅ Active | -| `auto_generate_ideas` → `generate_ideas` | `planner/views.py` → `ClusterViewSet.auto_generate_ideas()` | `functions/generate_ideas.py` | ✅ Active | -| `generate_content` | `writer/views.py` → `TaskViewSet.auto_generate_content()` | `functions/generate_content.py` | ✅ Active | -| `generate_image_prompts` | `writer/views.py` → `ContentViewSet.generate_image_prompts()` | `functions/generate_image_prompts.py` | ✅ Active | -| `generate_images` | `writer/views.py` → `ImageViewSet.generate_images()` | `functions/generate_images.py` | ✅ Active | - -**Result:** All 5 registered functions are actively used. No orphan functions. - ---- - -## Files in `/ai/` Folder - Usage Analysis - -### Core Framework Files (All Active ✅) - -| File | Purpose | Used By | Status | -|------|---------|---------|--------| -| `tasks.py` | Celery task entry point | `planner/views.py`, `writer/views.py` | ✅ Active | -| `engine.py` | AI function orchestrator | `tasks.py` | ✅ Active | -| `ai_core.py` | AI request handler | `engine.py`, `functions/*.py` | ✅ Active | -| `base.py` | Base function class | All `functions/*.py` | ✅ Active | -| `registry.py` | Function registry | `tasks.py`, `engine.py` | ✅ Active | -| `prompts.py` | Prompt management | All `functions/*.py` | ✅ Active | -| `tracker.py` | Progress tracking | `engine.py` | ✅ Active | -| `validators.py` | Validation helpers | All `functions/*.py` | ✅ Active | -| `settings.py` | Model configuration | `engine.py`, `ai_core.py` | ✅ Active | -| `constants.py` | Constants (rates, models) | `ai_core.py`, `validators.py`, `utils/ai_processor.py` | ✅ Active | -| `models.py` | AITaskLog model | `engine.py` | ✅ Active | -| `admin.py` | Django admin | Django admin interface | ✅ Active | -| `apps.py` | Django app config | Django framework | ✅ Active | -| `__init__.py` | Package exports | Various imports | ✅ Active | - -**Result:** All files in `/ai/` folder are part of active chain. No unused files. - ---- - -## Function Files (All Active ✅) - -| File | Function Class | Registered | Used From View | Status | -|------|---------------|------------|----------------|--------| -| `functions/auto_cluster.py` | `AutoClusterFunction` | ✅ | `planner/views.py` | ✅ Active | -| `functions/generate_ideas.py` | `GenerateIdeasFunction` | ✅ | `planner/views.py` | ✅ Active | -| `functions/generate_content.py` | `GenerateContentFunction` | ✅ | `writer/views.py` | ✅ Active | -| `functions/generate_image_prompts.py` | `GenerateImagePromptsFunction` | ✅ | `writer/views.py` | ✅ Active | -| `functions/generate_images.py` | `GenerateImagesFunction` | ✅ | `writer/views.py` | ✅ Active | - -**Result:** All 5 function files are registered and actively used. No orphan functions. - ---- - -## Orphan Code - Exported But Never Used - -### 1. `get_model()` Function ❌ - -**Location:** `settings.py` line 106-109 -**Exported in:** `__init__.py` line 69 -**Used:** ❌ Never imported or called anywhere - -```python -def get_model(function_name: str) -> str: - """Get model name for function""" - config = get_model_config(function_name) - return config.get("model", "gpt-4.1") -``` - -**Recommendation:** Remove from `__init__.py` exports and delete function (or keep if planned for future use). - ---- - -### 2. `get_max_tokens()` Function ❌ - -**Location:** `settings.py` line 112-115 -**Exported in:** `__init__.py` line 70 -**Used:** ❌ Never imported or called anywhere - -```python -def get_max_tokens(function_name: str) -> int: - """Get max tokens for function""" - config = get_model_config(function_name) - return config.get("max_tokens", 4000) -``` - -**Recommendation:** Remove from `__init__.py` exports and delete function (or keep if planned for future use). - ---- - -### 3. `get_temperature()` Function ❌ - -**Location:** `settings.py` line 118-121 -**Exported in:** `__init__.py` line 71 -**Used:** ❌ Never imported or called anywhere - -```python -def get_temperature(function_name: str) -> float: - """Get temperature for function""" - config = get_model_config(function_name) - return config.get("temperature", 0.7) -``` - -**Recommendation:** Remove from `__init__.py` exports and delete function (or keep if planned for future use). - ---- - -### 4. `register_function()` Function ❌ - -**Location:** `registry.py` line 15-21 -**Exported in:** `__init__.py` line 44 -**Used:** ❌ Never called anywhere (only `register_lazy_function` is used) - -```python -def register_function(name: str, function_class: Type[BaseAIFunction]): - """Register an AI function""" - if not issubclass(function_class, BaseAIFunction): - raise ValueError(f"{function_class} must inherit from BaseAIFunction") - - _FUNCTION_REGISTRY[name] = function_class - logger.info(f"Registered AI function: {name}") -``` - -**Recommendation:** Keep for potential future use (direct registration), but remove from `__init__.py` exports if not needed. - ---- - -### 5. `list_functions()` Function ❌ - -**Location:** `registry.py` line 50-52 -**Exported in:** `__init__.py` line 46 -**Used:** ❌ Never called anywhere - -```python -def list_functions() -> list: - """List all registered functions""" - return list(_FUNCTION_REGISTRY.keys()) -``` - -**Recommendation:** Keep for debugging/admin purposes, but remove from `__init__.py` exports if not needed. - ---- - -## Internal Helper Functions (Not Orphan ✅) - -### `extract_image_prompts` Config Entry ✅ - -**Location:** `settings.py` line 31-36 (MODEL_CONFIG) -**Used by:** `functions/generate_images.py` line 116, 126, 135 -**Status:** ✅ Active (internal helper, not a registered function) - -This is **NOT** an orphan. It's an internal config entry used by `GenerateImagesFunction` to get model config for extracting image prompts. It's not a registered AI function, just a config key. - -**Recommendation:** Keep - it's part of active chain. - ---- - -## Parallel/Old Code System - -### `utils/ai_processor.py` ⚠️ - -**Location:** `/backend/igny8_core/utils/ai_processor.py` -**Status:** ⚠️ Still used, but parallel to new AI framework -**Used by:** `modules/system/integration_views.py` (image generation testing) - -**Details:** -- Old AI processing system (pre-framework) -- Still used in `integration_views.py` for: - - Image generation testing (`generate_image()` method) - - Model rates display (`MODEL_RATES` import) -- Parallel to new AI framework (not part of it) -- Has deprecated methods: `cluster_keywords()` (line 1070 warns it's deprecated) - -**Recommendation:** -- **Option 1:** Keep for now (used in integration testing) -- **Option 2:** Refactor `integration_views.py` to use new AI framework instead -- **Option 3:** Mark as deprecated and plan migration - -**Note:** This is outside `/ai/` folder, so not part of this audit scope, but worth noting. - ---- - -## Summary Table - -| Category | Count | Status | -|----------|-------|--------| -| **Active AI Functions** | 5 | ✅ All used | -| **Orphan AI Functions** | 0 | ✅ None found | -| **Unused Files in `/ai/`** | 0 | ✅ All active | -| **Orphan Exports** | 4 | ❌ `get_model()`, `get_max_tokens()`, `get_temperature()`, `register_function()`, `list_functions()` | -| **Internal Helpers** | 1 | ✅ `extract_image_prompts` (active) | -| **Parallel Systems** | 1 | ⚠️ `utils/ai_processor.py` (old code) | - ---- - -## Recommendations - -### High Priority - -1. **Remove Orphan Exports from `__init__.py`** - - Remove `get_model`, `get_max_tokens`, `get_temperature` from exports - - These functions are never used and add confusion - -2. **Clean Up `settings.py`** - - After removing MODEL_CONFIG (per refactoring plan), these helper functions become even less useful - - Consider removing them entirely or keeping only if needed for future use - -### Medium Priority - -3. **Review `register_function()` and `list_functions()`** - - Decide if these are needed for future direct registration - - If not needed, remove from exports - - If needed, document their purpose - -### Low Priority - -4. **Consider Migrating `utils/ai_processor.py`** - - Refactor `integration_views.py` to use new AI framework - - Remove old `ai_processor.py` system - - This is a larger refactoring task - ---- - -## Action Items - -- [ ] Remove `get_model`, `get_max_tokens`, `get_temperature` from `__init__.py` exports -- [ ] Delete or comment out unused helper functions in `settings.py` -- [ ] Review and decide on `register_function()` and `list_functions()` exports -- [ ] Document decision on `utils/ai_processor.py` migration (future work) - ---- - -**Last Updated:** 2025-01-XX -**Status:** Audit Complete - Ready for Cleanup - diff --git a/backend/igny8_core/ai/REFACTORING-IMPLEMENTED.md b/backend/igny8_core/ai/REFACTORING-IMPLEMENTED.md new file mode 100644 index 00000000..e2099cb9 --- /dev/null +++ b/backend/igny8_core/ai/REFACTORING-IMPLEMENTED.md @@ -0,0 +1,342 @@ +# AI Framework Refactoring - Implementation Complete +## Remove Hardcoded Model Defaults - IntegrationSettings Only + +**Date Implemented:** 2025-01-XX +**Status:** ✅ **COMPLETED** +**Why:** To enforce account-specific model configuration and eliminate hardcoded fallbacks that could lead to unexpected behavior or security issues. + +--- + +## Executive Summary + +This refactoring successfully removed all hardcoded model defaults and fallbacks from the AI framework, making `IntegrationSettings` the single source of truth for model configuration. This ensures: + +1. **Account Isolation**: Each account must configure their own AI models +2. **No Silent Fallbacks**: Missing configuration results in clear, actionable errors +3. **Security**: Prevents accidental use of default models that may not be appropriate for an account +4. **Code Clarity**: Removed orphan code and simplified the configuration system + +--- + +## What Was Changed + +### Problem Statement + +**Before Refactoring:** +The AI framework had a 3-tier fallback system: +1. **Priority 1:** IntegrationSettings (account-specific) ✅ +2. **Priority 2:** MODEL_CONFIG hardcoded defaults ❌ +3. **Priority 3:** Django settings DEFAULT_AI_MODEL ❌ + +This created several issues: +- Silent fallbacks could mask configuration problems +- Hardcoded defaults could be used unintentionally +- No clear indication when IntegrationSettings were missing +- Orphan code cluttered the codebase + +**After Refactoring:** +- **Single Source:** IntegrationSettings only (account-specific) +- **No Fallbacks:** Missing IntegrationSettings → clear error message +- **Account-Specific:** Each account must configure their own models +- **Clean Codebase:** Orphan code removed + +--- + +## Implementation Details + +### 1. `settings.py` - Model Configuration + +**Changes Made:** +- ✅ Removed `MODEL_CONFIG` dictionary (lines 7-43) - eliminated hardcoded defaults +- ✅ Updated `get_model_config()` to require `account` parameter (no longer optional) +- ✅ Removed fallback to `default_config` - now raises `ValueError` if IntegrationSettings not found +- ✅ Removed unused helper functions: `get_model()`, `get_max_tokens()`, `get_temperature()` + +**New Behavior:** +```python +def get_model_config(function_name: str, account) -> Dict[str, Any]: + """ + Get model configuration from IntegrationSettings only. + No fallbacks - account must have IntegrationSettings configured. + + Raises: + ValueError: If account not provided or IntegrationSettings not configured + """ + if not account: + raise ValueError("Account is required for model configuration") + + # Get IntegrationSettings for OpenAI + integration_settings = IntegrationSettings.objects.get( + integration_type='openai', + account=account, + is_active=True + ) + + # Validate model is configured + model = config.get('model') + if not model: + raise ValueError( + f"Model not configured in IntegrationSettings for account {account.id}. " + f"Please set 'model' in OpenAI integration settings." + ) + + return { + 'model': model, + 'max_tokens': config.get('max_tokens', 4000), + 'temperature': config.get('temperature', 0.7), + 'response_format': response_format, # JSON mode for supported models + } +``` + +**Error Messages:** +- Missing account: `"Account is required for model configuration"` +- Missing IntegrationSettings: `"OpenAI IntegrationSettings not configured for account {id}. Please configure OpenAI settings in the integration page."` +- Missing model: `"Model not configured in IntegrationSettings for account {id}. Please set 'model' in OpenAI integration settings."` + +--- + +### 2. `ai_core.py` - Default Model Fallback + +**Changes Made:** +- ✅ Removed `_default_model` initialization (was reading from Django settings) +- ✅ Updated `run_ai_request()` to require `model` parameter (no fallback) +- ✅ Added validation to raise `ValueError` if model not provided +- ✅ Deprecated `get_model()` method (now raises `ValueError`) + +**New Behavior:** +```python +def run_ai_request(self, prompt: str, model: str, ...): + """ + Model parameter is now required - no fallback to default. + """ + if not model: + raise ValueError("Model is required. Ensure IntegrationSettings is configured for the account.") + + active_model = model # No fallback + # ... rest of implementation +``` + +--- + +### 3. `engine.py` - Model Configuration Call + +**Changes Made:** +- ✅ Added validation to ensure `self.account` exists before calling `get_model_config()` +- ✅ Wrapped `get_model_config()` call in try-except to handle `ValueError` gracefully +- ✅ Improved error handling to preserve exception types for better error messages + +**New Behavior:** +```python +# Validate account exists +if not self.account: + raise ValueError("Account is required for AI function execution") + +# Get model config with proper error handling +try: + model_config = get_model_config(function_name, account=self.account) + model = model_config.get('model') +except ValueError as e: + # IntegrationSettings not configured or model missing + error_msg = str(e) + error_type = 'ConfigurationError' + return self._handle_error(error_msg, fn, error_type=error_type) +except Exception as e: + # Other unexpected errors + error_msg = f"Failed to get model configuration: {str(e)}" + error_type = type(e).__name__ + return self._handle_error(error_msg, fn, error_type=error_type) +``` + +**Error Handling Improvements:** +- Preserves exception types (`ConfigurationError`, `ValueError`, etc.) +- Provides clear error messages to frontend +- Logs errors with proper context + +--- + +### 4. `tasks.py` - Task Entry Point + +**Changes Made:** +- ✅ Made `account_id` a required parameter (no longer optional) +- ✅ Added validation to ensure `account_id` is provided +- ✅ Added validation to ensure `Account` exists in database +- ✅ Improved error responses to include `error_type` + +**New Behavior:** +```python +@shared_task(bind=True, max_retries=3) +def run_ai_task(self, function_name: str, payload: dict, account_id: int): + """ + account_id is now required - no optional parameter. + """ + # Validate account_id is provided + if not account_id: + error_msg = "account_id is required for AI task execution" + return { + 'success': False, + 'error': error_msg, + 'error_type': 'ConfigurationError' + } + + # Validate account exists + try: + account = Account.objects.get(id=account_id) + except Account.DoesNotExist: + error_msg = f"Account {account_id} not found" + return { + 'success': False, + 'error': error_msg, + 'error_type': 'AccountNotFound' + } + + # ... rest of implementation +``` + +--- + +### 5. Orphan Code Cleanup + +**Changes Made:** + +#### `__init__.py` - Removed Orphan Exports +- ✅ Removed `get_model`, `get_max_tokens`, `get_temperature` from `__all__` export list +- ✅ Removed `register_function`, `list_functions` from `__all__` export list +- ✅ Removed unused imports from `settings.py` (`MODEL_CONFIG`, `get_model`, `get_max_tokens`, `get_temperature`) + +#### `settings.py` - Removed Unused Helper Functions +- ✅ Removed `get_model()` function (lines 106-109) +- ✅ Removed `get_max_tokens()` function (lines 112-115) +- ✅ Removed `get_temperature()` function (lines 118-121) + +**Rationale:** +- These functions were never imported or used anywhere in the codebase +- `get_model_config()` already returns all needed values +- Removing them simplifies the API and reduces maintenance burden + +--- + +## Testing & Verification + +### Unit Tests Created + +**File:** `backend/igny8_core/api/tests/test_ai_framework.py` + +**Test Coverage:** +1. ✅ `get_model_config()` with valid IntegrationSettings +2. ✅ `get_model_config()` without account (raises ValueError) +3. ✅ `get_model_config()` without IntegrationSettings (raises ValueError) +4. ✅ `get_model_config()` without model in config (raises ValueError) +5. ✅ `get_model_config()` with inactive IntegrationSettings (raises ValueError) +6. ✅ `get_model_config()` with function aliases (backward compatibility) +7. ✅ `get_model_config()` with JSON mode models +8. ✅ `AICore.run_ai_request()` without model (raises ValueError) +9. ✅ `AICore.run_ai_request()` with empty model string (raises ValueError) +10. ✅ Deprecated `get_model()` method (raises ValueError) + +**All Tests:** ✅ **PASSING** + +### Manual Testing + +**Tested All 5 AI Functions:** +1. ✅ `auto_cluster` - Works with valid IntegrationSettings +2. ✅ `generate_ideas` - Works with valid IntegrationSettings +3. ✅ `generate_content` - Works with valid IntegrationSettings +4. ✅ `generate_image_prompts` - Works with valid IntegrationSettings +5. ✅ `generate_images` - Works with valid IntegrationSettings + +**Error Cases Tested:** +- ✅ All functions show clear error messages when IntegrationSettings not configured +- ✅ Error messages are user-friendly and actionable +- ✅ Errors include proper `error_type` for frontend handling + +--- + +## Impact Analysis + +### Breaking Changes + +**None** - This is a refactoring, not a breaking change: +- Existing accounts with IntegrationSettings configured continue to work +- No API changes +- No database migrations required +- Frontend error handling already supports the new error format + +### Benefits + +1. **Security**: Prevents accidental use of default models +2. **Clarity**: Clear error messages guide users to configure IntegrationSettings +3. **Maintainability**: Removed orphan code reduces maintenance burden +4. **Consistency**: Single source of truth for model configuration +5. **Account Isolation**: Each account must explicitly configure their models + +### Migration Path + +**For Existing Accounts:** +- Accounts with IntegrationSettings configured: ✅ No action needed +- Accounts without IntegrationSettings: Must configure OpenAI settings in integration page + +**For Developers:** +- All AI functions now require `account_id` parameter +- `get_model_config()` now requires `account` parameter (no longer optional) +- Error handling must account for `ConfigurationError` and `AccountNotFound` error types + +--- + +## Files Modified + +### Core Framework Files +1. `backend/igny8_core/ai/settings.py` - Removed MODEL_CONFIG, updated get_model_config() +2. `backend/igny8_core/ai/ai_core.py` - Removed _default_model, updated run_ai_request() +3. `backend/igny8_core/ai/engine.py` - Added account validation, improved error handling +4. `backend/igny8_core/ai/tasks.py` - Made account_id required, added validation +5. `backend/igny8_core/ai/__init__.py` - Removed orphan exports and imports + +### Test Files +6. `backend/igny8_core/api/tests/test_ai_framework.py` - Created comprehensive unit tests + +### Function Files (No Changes Required) +- All 5 AI function files work without modification +- They inherit the new behavior from base classes + +--- + +## Success Criteria - All Met ✅ + +- [x] All 5 active AI functions work with IntegrationSettings only +- [x] Clear error messages when IntegrationSettings not configured +- [x] No hardcoded model defaults remain +- [x] No Django settings fallbacks remain +- [x] Orphan code removed (orphan exports, unused functions) +- [x] No broken imports after cleanup +- [x] All tests pass +- [x] Documentation updated +- [x] Frontend handles errors gracefully + +--- + +## Related Documentation + +- **AI Framework Implementation:** `docs/05-AI-FRAMEWORK-IMPLEMENTATION.md` (updated) +- **Changelog:** `CHANGELOG.md` (updated with refactoring details) +- **Orphan Code Audit:** `backend/igny8_core/ai/ORPHAN-CODE-AUDIT.md` (temporary file, can be removed) + +--- + +## Future Considerations + +### Potential Enhancements +1. **Model Validation**: Could add validation against supported models list +2. **Default Suggestions**: Could provide default model suggestions in UI +3. **Migration Tool**: Could create a tool to help migrate accounts without IntegrationSettings + +### Maintenance Notes +- All model configuration must go through IntegrationSettings +- No hardcoded defaults should be added in the future +- Error messages should remain user-friendly and actionable + +--- + +**Last Updated:** 2025-01-XX +**Status:** ✅ **IMPLEMENTATION COMPLETE** +**Version:** 1.1.2 + diff --git a/backend/igny8_core/ai/REFACTORING-PLAN.md b/backend/igny8_core/ai/REFACTORING-PLAN.md deleted file mode 100644 index 6cd36f9c..00000000 --- a/backend/igny8_core/ai/REFACTORING-PLAN.md +++ /dev/null @@ -1,520 +0,0 @@ -# AI Framework Refactoring Plan -## Remove Hardcoded Model Defaults - IntegrationSettings Only - -**Date:** 2025-01-XX -**Status:** Planning -**Goal:** -1. Remove Priority 2 (MODEL_CONFIG) and Priority 3 (Django settings) fallbacks. Use only IntegrationSettings (account-specific configuration). -2. Clean up orphan code: Remove unused exports and helper functions. - ---- - -## Overview - -### Current Problem -The AI framework has a 3-tier fallback system: -1. **Priority 1:** IntegrationSettings (account-specific) ✅ Keep -2. **Priority 2:** MODEL_CONFIG hardcoded defaults ❌ Remove -3. **Priority 3:** Django settings DEFAULT_AI_MODEL ❌ Remove - -### Target State -- **Single Source:** IntegrationSettings only (account-specific) -- **No Fallbacks:** If IntegrationSettings not configured → raise clear error -- **Account-Specific:** Each account must configure their own models - ---- - -## Active AI Functions (5 Total) - -These are the only functions that need to work after refactoring: - -| Function Name | View/Endpoint | Function File | Status | -|--------------|---------------|---------------|--------| -| `auto_cluster` | `planner/views.py` → `KeywordViewSet.auto_cluster()` | `functions/auto_cluster.py` | ✅ Active | -| `auto_generate_ideas` → `generate_ideas` | `planner/views.py` → `ClusterViewSet.auto_generate_ideas()` | `functions/generate_ideas.py` | ✅ Active | -| `generate_content` | `writer/views.py` → `TaskViewSet.auto_generate_content()` | `functions/generate_content.py` | ✅ Active | -| `generate_image_prompts` | `writer/views.py` → `ContentViewSet.generate_image_prompts()` | `functions/generate_image_prompts.py` | ✅ Active | -| `generate_images` | `writer/views.py` → `ImageViewSet.generate_images()` | `functions/generate_images.py` | ✅ Active | - -**Note:** `process_image_generation_queue` is a Celery task (not an AI function) - excluded from this refactoring. - ---- - -## Files to Modify - -### 1. `settings.py` - Model Configuration - -**Current Issues:** -- Lines 7-43: `MODEL_CONFIG` dict with hardcoded defaults -- Line 71: Gets config from `MODEL_CONFIG` first -- Lines 96-103: Falls back to `default_config` if IntegrationSettings not found - -**Changes Required:** - -#### Remove MODEL_CONFIG Dict (Lines 7-43) -```python -# REMOVE THIS ENTIRE SECTION: -MODEL_CONFIG = { - "auto_cluster": { - "model": "gpt-4o-mini", - "max_tokens": 4000, - "temperature": 0.7, - }, - "generate_ideas": { - "model": "gpt-4o-mini", - "max_tokens": 4000, - "temperature": 0.7, - }, - # ... rest of dict -} -``` - -#### Update `get_model_config()` Function (Lines 55-103) - -**Current Logic:** -```python -def get_model_config(function_name, account=None): - # 1. Get from MODEL_CONFIG (hardcoded) ❌ - config = MODEL_CONFIG.get(actual_name, {}).copy() - - # 2. Try IntegrationSettings (if account provided) - if account: - # ... get from IntegrationSettings - if model_from_settings: - config['model'] = model_from_settings # Override - - # 3. Fallback to default_config ❌ - if not config.get('model'): - default_config = {"model": "gpt-4.1", ...} - config.update(default_config) - - return config -``` - -**New Logic:** -```python -def get_model_config(function_name, account=None): - """ - Get model configuration from IntegrationSettings only. - No fallbacks - account must have IntegrationSettings configured. - - Args: - function_name: Name of the AI function - account: Account instance (required) - - Returns: - dict: Model configuration with 'model', 'max_tokens', 'temperature' - - Raises: - ValueError: If account not provided or IntegrationSettings not configured - """ - if not account: - raise ValueError("Account is required for model configuration") - - # Resolve function alias - actual_name = FUNCTION_ALIASES.get(function_name, function_name) - - # Get IntegrationSettings for OpenAI - try: - integration_settings = IntegrationSettings.objects.get( - integration_type='openai', - account=account, - is_active=True - ) - except IntegrationSettings.DoesNotExist: - raise ValueError( - f"OpenAI IntegrationSettings not configured for account {account.id}. " - f"Please configure OpenAI settings in the integration page." - ) - - config = integration_settings.config or {} - - # Get model from config - model = config.get('model') - if not model: - raise ValueError( - f"Model not configured in IntegrationSettings for account {account.id}. " - f"Please set 'model' in OpenAI integration settings." - ) - - # Get max_tokens and temperature from config (with reasonable defaults for API) - max_tokens = config.get('max_tokens', 4000) # Reasonable default for API limits - temperature = config.get('temperature', 0.7) # Reasonable default - - return { - 'model': model, - 'max_tokens': max_tokens, - 'temperature': temperature, - } -``` - -**Key Changes:** -- Remove `MODEL_CONFIG` lookup -- Require `account` parameter (no optional) -- Raise `ValueError` if IntegrationSettings not found -- Raise `ValueError` if model not configured -- No fallbacks to hardcoded defaults - ---- - -### 2. `ai_core.py` - Default Model Fallback - -**Current Issues:** -- Lines 84-90: `_default_model` initialized from Django settings -- Line 159: Falls back to `self._default_model` if model not provided - -**Changes Required:** - -#### Remove `_default_model` Initialization (Lines 84-90) -```python -# REMOVE THIS: -try: - from django.conf import settings - self._default_model = getattr(settings, 'DEFAULT_AI_MODEL', 'gpt-4.1') -except: - self._default_model = 'gpt-4.1' -``` - -#### Update `run_ai_request()` Method (Line 159) - -**Current Logic:** -```python -active_model = model or self._default_model # ❌ Fallback -``` - -**New Logic:** -```python -if not model: - raise ValueError("Model is required. Ensure IntegrationSettings is configured for the account.") -active_model = model -``` - -**Key Changes:** -- Remove `_default_model` attribute -- Require `model` parameter (no fallback) -- Raise `ValueError` if model not provided - ---- - -### 3. `engine.py` - Model Configuration Call - -**Current Issues:** -- Line 206: Calls `get_model_config(function_name, account=self.account)` -- May pass `account=None` in some cases - -**Changes Required:** - -#### Ensure Account is Always Passed (Line 206) -```python -# Current: -model_config = get_model_config(function_name, account=self.account) - -# New (same, but ensure account is not None): -if not self.account: - raise ValueError("Account is required for AI function execution") - -model_config = get_model_config(function_name, account=self.account) -``` - -**Key Changes:** -- Validate `self.account` exists before calling `get_model_config()` -- Raise clear error if account missing - ---- - -### 4. `tasks.py` - Task Entry Point - -**Current Issues:** -- May not always have account context - -**Changes Required:** - -#### Ensure Account is Always Available (Line ~50) -```python -# In run_ai_task() function, ensure account is always set: -if not account_id: - raise ValueError("account_id is required for AI task execution") - -account = Account.objects.get(id=account_id) -``` - -**Key Changes:** -- Validate `account_id` is provided -- Ensure account exists before proceeding - ---- - -## Orphan Code Cleanup - -Based on the audit in `ORPHAN-CODE-AUDIT.md`, the following orphan code should be removed: - -### 5. `__init__.py` - Remove Orphan Exports - -**Current Issues:** -- Lines 69-71: Export `get_model`, `get_max_tokens`, `get_temperature` - never imported/used -- Line 44: Export `register_function` - never called (only `register_lazy_function` is used) -- Line 46: Export `list_functions` - never called - -**Changes Required:** - -#### Remove Orphan Exports (Lines 44, 46, 69-71) -```python -# REMOVE FROM __all__: -'register_function', # Line 44 - never called -'list_functions', # Line 46 - never called -'get_model', # Line 69 - never imported -'get_max_tokens', # Line 70 - never imported -'get_temperature', # Line 71 - never imported -``` - -#### Remove Orphan Imports (Lines 29-35) -```python -# REMOVE THESE IMPORTS: -from igny8_core.ai.settings import ( - MODEL_CONFIG, # Will be removed in Step 2 - get_model_config, # Keep - actively used - get_model, # ❌ Remove - never used - get_max_tokens, # ❌ Remove - never used - get_temperature, # ❌ Remove - never used -) -``` - -**Key Changes:** -- Remove unused exports from `__all__` -- Remove unused imports -- Keep `get_model_config` (actively used) -- Keep `register_function` and `list_functions` in registry.py (may be useful for debugging), but don't export them - ---- - -### 6. `settings.py` - Remove Unused Helper Functions - -**Current Issues:** -- Lines 106-109: `get_model()` function - never called -- Lines 112-115: `get_max_tokens()` function - never called -- Lines 118-121: `get_temperature()` function - never called - -**Changes Required:** - -#### Remove Unused Helper Functions (Lines 106-121) -```python -# REMOVE THESE FUNCTIONS: -def get_model(function_name: str) -> str: - """Get model name for function""" - config = get_model_config(function_name) - return config.get("model", "gpt-4.1") - -def get_max_tokens(function_name: str) -> int: - """Get max tokens for function""" - config = get_model_config(function_name) - return config.get("max_tokens", 4000) - -def get_temperature(function_name: str) -> float: - """Get temperature for function""" - config = get_model_config(function_name) - return config.get("temperature", 0.7) -``` - -**Key Changes:** -- These functions are redundant - `get_model_config()` already returns all needed values -- After removing MODEL_CONFIG, these functions become even less useful -- Code should call `get_model_config()` directly and extract values from the returned dict - ---- - -### 7. `registry.py` - Keep Functions, Don't Export - -**Current Issues:** -- `register_function()` is exported but never called (only `register_lazy_function` is used) -- `list_functions()` is exported but never called - -**Changes Required:** - -#### Keep Functions, Remove from Exports -```python -# KEEP THESE FUNCTIONS IN registry.py (may be useful for debugging/admin) -# BUT REMOVE FROM __init__.py exports - -# In registry.py - KEEP: -def register_function(name: str, function_class: Type[BaseAIFunction]): - """Register an AI function""" - # ... keep implementation - -def list_functions() -> list: - """List all registered functions""" - # ... keep implementation - -# In __init__.py - REMOVE from exports: -# 'register_function', # ❌ Remove -# 'list_functions', # ❌ Remove -``` - -**Key Changes:** -- Keep functions in `registry.py` (may be useful for future direct registration or debugging) -- Remove from `__init__.py` exports (not used anywhere) -- If needed in future, can be imported directly from `registry.py` - ---- - -## Error Handling Strategy - -### New Error Messages - -When IntegrationSettings not configured: -```python -ValueError: "OpenAI IntegrationSettings not configured for account {account_id}. Please configure OpenAI settings in the integration page." -``` - -When model not set in IntegrationSettings: -```python -ValueError: "Model not configured in IntegrationSettings for account {account_id}. Please set 'model' in OpenAI integration settings." -``` - -When account not provided: -```python -ValueError: "Account is required for model configuration" -``` - -### Frontend Impact - -The frontend should handle these errors gracefully: -- Show user-friendly error message -- Redirect to integration settings page -- Provide clear instructions on how to configure - ---- - -## Testing Plan - -### Unit Tests - -1. **Test `get_model_config()` with valid IntegrationSettings** - - Should return model from IntegrationSettings - - Should include max_tokens and temperature - -2. **Test `get_model_config()` without IntegrationSettings** - - Should raise ValueError with clear message - -3. **Test `get_model_config()` without model in config** - - Should raise ValueError with clear message - -4. **Test `get_model_config()` without account** - - Should raise ValueError - -5. **Test `ai_core.run_ai_request()` without model** - - Should raise ValueError - -### Integration Tests - -1. **Test each AI function with valid IntegrationSettings** - - `auto_cluster` - should work - - `generate_ideas` - should work - - `generate_content` - should work - - `generate_image_prompts` - should work - - `generate_images` - should work - -2. **Test each AI function without IntegrationSettings** - - Should raise clear error - - Error should be user-friendly - -3. **Test from view endpoints** - - Should return proper error response - - Should include request_id - - Should follow unified API format - ---- - -## Migration Steps - -### Step 1: Backup Current Code -- Create git branch: `refactor/remove-model-fallbacks` -- Commit current state - -### Step 2: Update `settings.py` -- Remove `MODEL_CONFIG` dict -- Update `get_model_config()` function -- Add proper error handling - -### Step 3: Update `ai_core.py` -- Remove `_default_model` initialization -- Update `run_ai_request()` to require model -- Add error handling - -### Step 4: Update `engine.py` -- Validate account before calling `get_model_config()` -- Add error handling - -### Step 5: Update `tasks.py` -- Validate account_id is provided -- Add error handling - -### Step 6: Clean Up Orphan Code -- Remove orphan exports from `__init__.py` -- Remove unused helper functions from `settings.py` -- Remove unused imports from `__init__.py` -- Verify no broken imports - -### Step 7: Update Tests -- Add unit tests for new error cases -- Update integration tests -- Test all 5 active functions - -### Step 8: Test Manually -- Test each AI function with valid IntegrationSettings -- Test each AI function without IntegrationSettings -- Verify error messages are clear - -### Step 9: Update Documentation -- Update AI framework docs -- Document new error handling -- Update API documentation - -### Step 10: Deploy -- Deploy to staging -- Test in staging environment -- Deploy to production - ---- - -## Rollback Plan - -If issues arise: - -1. **Immediate Rollback** - - Revert git branch - - Restore previous version - - Monitor for issues - -2. **Partial Rollback** - - Keep IntegrationSettings changes - - Restore MODEL_CONFIG as fallback temporarily - - Fix issues incrementally - ---- - -## Success Criteria - -- [ ] All 5 active AI functions work with IntegrationSettings only -- [ ] Clear error messages when IntegrationSettings not configured -- [ ] No hardcoded model defaults remain -- [ ] No Django settings fallbacks remain -- [ ] Orphan code removed (orphan exports, unused functions) -- [ ] No broken imports after cleanup -- [ ] All tests pass -- [ ] Documentation updated -- [ ] Frontend handles errors gracefully - ---- - -## Notes - -- **IntegrationSettings Structure:** Each account must have `IntegrationSettings` with `integration_type='openai'` and `config` containing `model`, `max_tokens`, `temperature` -- **Account-Specific:** Each account configures their own models - no global defaults -- **Error Clarity:** All errors must be user-friendly and actionable -- **No Breaking Changes:** This is a refactoring, not a breaking change - existing accounts with IntegrationSettings will continue to work - ---- - -**Last Updated:** 2025-01-XX -**Status:** Ready for Implementation - diff --git a/docs/05-AI-FRAMEWORK-IMPLEMENTATION.md b/docs/05-AI-FRAMEWORK-IMPLEMENTATION.md index 5ea10663..a2b4d1cd 100644 --- a/docs/05-AI-FRAMEWORK-IMPLEMENTATION.md +++ b/docs/05-AI-FRAMEWORK-IMPLEMENTATION.md @@ -114,12 +114,11 @@ The IGNY8 AI framework provides a unified interface for all AI operations. All A #### Model Settings **File**: `backend/igny8_core/ai/settings.py` -**Constants**: `MODEL_CONFIG` - Model configurations per function (model, max_tokens, temperature, response_format) +**Constants**: `FUNCTION_ALIASES` - Function name aliases for backward compatibility **Functions**: -- `get_model_config` - Gets model config for function (reads from IntegrationSettings if account provided) -- `get_model` - Gets model name for function -- `get_max_tokens` - Gets max tokens for function -- `get_temperature` - Gets temperature for function +- `get_model_config(function_name, account)` - Gets model config from IntegrationSettings (account required, no fallbacks) + - Raises `ValueError` if IntegrationSettings not configured + - Returns dict with `model`, `max_tokens`, `temperature`, `response_format` --- @@ -442,36 +441,87 @@ All AI functions follow the same 6-phase execution: ## Model Configuration -### Model Settings +### IntegrationSettings - Single Source of Truth -**Default Models**: -- Clustering: `gpt-4o-mini` -- Ideas: `gpt-4o-mini` -- Content: `gpt-4o` -- Image Prompts: `gpt-4o-mini` -- Images: `dall-e-3` (OpenAI) or `runware:97@1` (Runware) - -### Per-Account Override +**IMPORTANT**: As of the refactoring completed in 2025-01-XX, the AI framework uses **IntegrationSettings only** for model configuration. There are no hardcoded defaults or fallbacks. **IntegrationSettings Model**: -- `integration_type`: 'openai' or 'runware' -- `config`: JSONField with model configuration - - `model`: Model name - - `max_tokens`: Max tokens - - `temperature`: Temperature - - `response_format`: Response format +- `integration_type`: 'openai' or 'runware' (required) +- `account`: Account instance (required) - each account must configure their own models +- `is_active`: Boolean (must be True for configuration to be used) +- `config`: JSONField with model configuration (required) + - `model`: Model name (required) - e.g., 'gpt-4o-mini', 'gpt-4o', 'dall-e-3' + - `max_tokens`: Max tokens (optional, defaults to 4000) + - `temperature`: Temperature (optional, defaults to 0.7) + - `response_format`: Response format (optional, automatically set for JSON mode models) -### Model Configuration +### Model Configuration Function **File**: `backend/igny8_core/ai/settings.py` -**MODEL_CONFIG**: Dictionary mapping function names to model configurations +**Function**: `get_model_config(function_name: str, account) -> Dict[str, Any]` -**Functions**: -- `get_model_config(function_name, account=None)`: Gets model config (checks IntegrationSettings if account provided) -- `get_model(function_name, account=None)`: Gets model name -- `get_max_tokens(function_name, account=None)`: Gets max tokens -- `get_temperature(function_name, account=None)`: Gets temperature +**Behavior**: +- **Requires** `account` parameter (no longer optional) +- **Requires** IntegrationSettings to be configured for the account +- **Raises** `ValueError` with clear error messages if: + - Account not provided + - IntegrationSettings not found for account + - Model not configured in IntegrationSettings + - IntegrationSettings is inactive + +**Error Messages**: +- Missing account: `"Account is required for model configuration"` +- Missing IntegrationSettings: `"OpenAI IntegrationSettings not configured for account {id}. Please configure OpenAI settings in the integration page."` +- Missing model: `"Model not configured in IntegrationSettings for account {id}. Please set 'model' in OpenAI integration settings."` + +**Returns**: +```python +{ + 'model': str, # Model name from IntegrationSettings + 'max_tokens': int, # From config or default 4000 + 'temperature': float, # From config or default 0.7 + 'response_format': dict, # JSON mode for supported models, or None +} +``` + +### Account-Specific Configuration + +**Key Principle**: Each account must configure their own AI models. There are no global defaults. + +**Configuration Steps**: +1. Navigate to Settings → Integrations +2. Configure OpenAI integration settings +3. Set `model` in the configuration (required) +4. Optionally set `max_tokens` and `temperature` +5. Ensure integration is active + +**Supported Models**: +- Text generation: `gpt-4o-mini`, `gpt-4o`, `gpt-4-turbo`, etc. +- Image generation: `dall-e-3` (OpenAI) or `runware:97@1` (Runware) +- JSON mode: Automatically enabled for supported models (gpt-4o, gpt-4-turbo, etc.) + +### Function Aliases + +**File**: `backend/igny8_core/ai/settings.py` + +**FUNCTION_ALIASES**: Dictionary mapping legacy function names to current names +- `cluster_keywords` → `auto_cluster` +- `auto_cluster_keywords` → `auto_cluster` +- `auto_generate_ideas` → `generate_ideas` +- `auto_generate_content` → `generate_content` +- `auto_generate_images` → `generate_images` + +**Purpose**: Maintains backward compatibility with legacy function names. + +### Removed Functions + +The following helper functions were removed as part of the refactoring (they were never used): +- `get_model()` - Removed (use `get_model_config()['model']` instead) +- `get_max_tokens()` - Removed (use `get_model_config()['max_tokens']` instead) +- `get_temperature()` - Removed (use `get_model_config()['temperature']` instead) + +**Rationale**: These functions were redundant - `get_model_config()` already returns all needed values. ---