# AI Framework Refactoring Plan
## Remove Hardcoded Model Defaults - IntegrationSettings Only

**Date:** 2025-01-XX  
**Status:** Planning  
**Goal:** 
1. Remove Priority 2 (MODEL_CONFIG) and Priority 3 (Django settings) fallbacks. Use only IntegrationSettings (account-specific configuration).
2. Clean up orphan code: Remove unused exports and helper functions.

---

## Overview

### Current Problem
The AI framework has a 3-tier fallback system:
1. **Priority 1:** IntegrationSettings (account-specific) ✅ Keep
2. **Priority 2:** MODEL_CONFIG hardcoded defaults ❌ Remove
3. **Priority 3:** Django settings DEFAULT_AI_MODEL ❌ Remove

### Target State
- **Single Source:** IntegrationSettings only (account-specific)
- **No Fallbacks:** If IntegrationSettings not configured → raise clear error
- **Account-Specific:** Each account must configure their own models

---

## Active AI Functions (5 Total)

These are the only functions that need to work after refactoring:

| Function Name | View/Endpoint | Function File | Status |
|--------------|---------------|---------------|--------|
| `auto_cluster` | `planner/views.py` → `KeywordViewSet.auto_cluster()` | `functions/auto_cluster.py` | ✅ Active |
| `auto_generate_ideas` → `generate_ideas` | `planner/views.py` → `ClusterViewSet.auto_generate_ideas()` | `functions/generate_ideas.py` | ✅ Active |
| `generate_content` | `writer/views.py` → `TaskViewSet.auto_generate_content()` | `functions/generate_content.py` | ✅ Active |
| `generate_image_prompts` | `writer/views.py` → `ContentViewSet.generate_image_prompts()` | `functions/generate_image_prompts.py` | ✅ Active |
| `generate_images` | `writer/views.py` → `ImageViewSet.generate_images()` | `functions/generate_images.py` | ✅ Active |

**Note:** `process_image_generation_queue` is a Celery task (not an AI function) - excluded from this refactoring.

---

## Files to Modify

### 1. `settings.py` - Model Configuration

**Current Issues:**
- Lines 7-43: `MODEL_CONFIG` dict with hardcoded defaults
- Line 71: Gets config from `MODEL_CONFIG` first
- Lines 96-103: Falls back to `default_config` if IntegrationSettings not found

**Changes Required:**

#### Remove MODEL_CONFIG Dict (Lines 7-43)
```python
# REMOVE THIS ENTIRE SECTION:
MODEL_CONFIG = {
    "auto_cluster": {
        "model": "gpt-4o-mini",
        "max_tokens": 4000,
        "temperature": 0.7,
    },
    "generate_ideas": {
        "model": "gpt-4o-mini",
        "max_tokens": 4000,
        "temperature": 0.7,
    },
    # ... rest of dict
}
```

#### Update `get_model_config()` Function (Lines 55-103)

**Current Logic:**
```python
def get_model_config(function_name, account=None):
    # 1. Get from MODEL_CONFIG (hardcoded) ❌
    config = MODEL_CONFIG.get(actual_name, {}).copy()
    
    # 2. Try IntegrationSettings (if account provided)
    if account:
        # ... get from IntegrationSettings
        if model_from_settings:
            config['model'] = model_from_settings  # Override
    
    # 3. Fallback to default_config ❌
    if not config.get('model'):
        default_config = {"model": "gpt-4.1", ...}
        config.update(default_config)
    
    return config
```

**New Logic:**
```python
def get_model_config(function_name, account=None):
    """
    Get model configuration from IntegrationSettings only.
    No fallbacks - account must have IntegrationSettings configured.
    
    Args:
        function_name: Name of the AI function
        account: Account instance (required)
    
    Returns:
        dict: Model configuration with 'model', 'max_tokens', 'temperature'
    
    Raises:
        ValueError: If account not provided or IntegrationSettings not configured
    """
    if not account:
        raise ValueError("Account is required for model configuration")
    
    # Resolve function alias
    actual_name = FUNCTION_ALIASES.get(function_name, function_name)
    
    # Get IntegrationSettings for OpenAI
    try:
        integration_settings = IntegrationSettings.objects.get(
            integration_type='openai',
            account=account,
            is_active=True
        )
    except IntegrationSettings.DoesNotExist:
        raise ValueError(
            f"OpenAI IntegrationSettings not configured for account {account.id}. "
            f"Please configure OpenAI settings in the integration page."
        )
    
    config = integration_settings.config or {}
    
    # Get model from config
    model = config.get('model')
    if not model:
        raise ValueError(
            f"Model not configured in IntegrationSettings for account {account.id}. "
            f"Please set 'model' in OpenAI integration settings."
        )
    
    # Get max_tokens and temperature from config (with reasonable defaults for API)
    max_tokens = config.get('max_tokens', 4000)  # Reasonable default for API limits
    temperature = config.get('temperature', 0.7)  # Reasonable default
    
    return {
        'model': model,
        'max_tokens': max_tokens,
        'temperature': temperature,
    }
```

**Key Changes:**
- Remove `MODEL_CONFIG` lookup
- Require `account` parameter (no optional)
- Raise `ValueError` if IntegrationSettings not found
- Raise `ValueError` if model not configured
- No fallbacks to hardcoded defaults

---

### 2. `ai_core.py` - Default Model Fallback

**Current Issues:**
- Lines 84-90: `_default_model` initialized from Django settings
- Line 159: Falls back to `self._default_model` if model not provided

**Changes Required:**

#### Remove `_default_model` Initialization (Lines 84-90)
```python
# REMOVE THIS:
try:
    from django.conf import settings
    self._default_model = getattr(settings, 'DEFAULT_AI_MODEL', 'gpt-4.1')
except:
    self._default_model = 'gpt-4.1'
```

#### Update `run_ai_request()` Method (Line 159)

**Current Logic:**
```python
active_model = model or self._default_model  # ❌ Fallback
```

**New Logic:**
```python
if not model:
    raise ValueError("Model is required. Ensure IntegrationSettings is configured for the account.")
active_model = model
```

**Key Changes:**
- Remove `_default_model` attribute
- Require `model` parameter (no fallback)
- Raise `ValueError` if model not provided

---

### 3. `engine.py` - Model Configuration Call

**Current Issues:**
- Line 206: Calls `get_model_config(function_name, account=self.account)`
- May pass `account=None` in some cases

**Changes Required:**

#### Ensure Account is Always Passed (Line 206)
```python
# Current:
model_config = get_model_config(function_name, account=self.account)

# New (same, but ensure account is not None):
if not self.account:
    raise ValueError("Account is required for AI function execution")

model_config = get_model_config(function_name, account=self.account)
```

**Key Changes:**
- Validate `self.account` exists before calling `get_model_config()`
- Raise clear error if account missing

---

### 4. `tasks.py` - Task Entry Point

**Current Issues:**
- May not always have account context

**Changes Required:**

#### Ensure Account is Always Available (Line ~50)
```python
# In run_ai_task() function, ensure account is always set:
if not account_id:
    raise ValueError("account_id is required for AI task execution")

account = Account.objects.get(id=account_id)
```

**Key Changes:**
- Validate `account_id` is provided
- Ensure account exists before proceeding

---

## Orphan Code Cleanup

Based on the audit in `ORPHAN-CODE-AUDIT.md`, the following orphan code should be removed:

### 5. `__init__.py` - Remove Orphan Exports

**Current Issues:**
- Lines 69-71: Export `get_model`, `get_max_tokens`, `get_temperature` - never imported/used
- Line 44: Export `register_function` - never called (only `register_lazy_function` is used)
- Line 46: Export `list_functions` - never called

**Changes Required:**

#### Remove Orphan Exports (Lines 44, 46, 69-71)
```python
# REMOVE FROM __all__:
'register_function',  # Line 44 - never called
'list_functions',     # Line 46 - never called
'get_model',          # Line 69 - never imported
'get_max_tokens',     # Line 70 - never imported
'get_temperature',    # Line 71 - never imported
```

#### Remove Orphan Imports (Lines 29-35)
```python
# REMOVE THESE IMPORTS:
from igny8_core.ai.settings import (
    MODEL_CONFIG,        # Will be removed in Step 2
    get_model_config,    # Keep - actively used
    get_model,           # ❌ Remove - never used
    get_max_tokens,      # ❌ Remove - never used
    get_temperature,     # ❌ Remove - never used
)
```

**Key Changes:**
- Remove unused exports from `__all__`
- Remove unused imports
- Keep `get_model_config` (actively used)
- Keep `register_function` and `list_functions` in registry.py (may be useful for debugging), but don't export them

---

### 6. `settings.py` - Remove Unused Helper Functions

**Current Issues:**
- Lines 106-109: `get_model()` function - never called
- Lines 112-115: `get_max_tokens()` function - never called
- Lines 118-121: `get_temperature()` function - never called

**Changes Required:**

#### Remove Unused Helper Functions (Lines 106-121)
```python
# REMOVE THESE FUNCTIONS:
def get_model(function_name: str) -> str:
    """Get model name for function"""
    config = get_model_config(function_name)
    return config.get("model", "gpt-4.1")

def get_max_tokens(function_name: str) -> int:
    """Get max tokens for function"""
    config = get_model_config(function_name)
    return config.get("max_tokens", 4000)

def get_temperature(function_name: str) -> float:
    """Get temperature for function"""
    config = get_model_config(function_name)
    return config.get("temperature", 0.7)
```

**Key Changes:**
- These functions are redundant - `get_model_config()` already returns all needed values
- After removing MODEL_CONFIG, these functions become even less useful
- Code should call `get_model_config()` directly and extract values from the returned dict

---

### 7. `registry.py` - Keep Functions, Don't Export

**Current Issues:**
- `register_function()` is exported but never called (only `register_lazy_function` is used)
- `list_functions()` is exported but never called

**Changes Required:**

#### Keep Functions, Remove from Exports
```python
# KEEP THESE FUNCTIONS IN registry.py (may be useful for debugging/admin)
# BUT REMOVE FROM __init__.py exports

# In registry.py - KEEP:
def register_function(name: str, function_class: Type[BaseAIFunction]):
    """Register an AI function"""
    # ... keep implementation

def list_functions() -> list:
    """List all registered functions"""
    # ... keep implementation

# In __init__.py - REMOVE from exports:
# 'register_function',  # ❌ Remove
# 'list_functions',    # ❌ Remove
```

**Key Changes:**
- Keep functions in `registry.py` (may be useful for future direct registration or debugging)
- Remove from `__init__.py` exports (not used anywhere)
- If needed in future, can be imported directly from `registry.py`

---

## Error Handling Strategy

### New Error Messages

When IntegrationSettings not configured:
```python
ValueError: "OpenAI IntegrationSettings not configured for account {account_id}. Please configure OpenAI settings in the integration page."
```

When model not set in IntegrationSettings:
```python
ValueError: "Model not configured in IntegrationSettings for account {account_id}. Please set 'model' in OpenAI integration settings."
```

When account not provided:
```python
ValueError: "Account is required for model configuration"
```

### Frontend Impact

The frontend should handle these errors gracefully:
- Show user-friendly error message
- Redirect to integration settings page
- Provide clear instructions on how to configure

---

## Testing Plan

### Unit Tests

1. **Test `get_model_config()` with valid IntegrationSettings**
   - Should return model from IntegrationSettings
   - Should include max_tokens and temperature

2. **Test `get_model_config()` without IntegrationSettings**
   - Should raise ValueError with clear message

3. **Test `get_model_config()` without model in config**
   - Should raise ValueError with clear message

4. **Test `get_model_config()` without account**
   - Should raise ValueError

5. **Test `ai_core.run_ai_request()` without model**
   - Should raise ValueError

### Integration Tests

1. **Test each AI function with valid IntegrationSettings**
   - `auto_cluster` - should work
   - `generate_ideas` - should work
   - `generate_content` - should work
   - `generate_image_prompts` - should work
   - `generate_images` - should work

2. **Test each AI function without IntegrationSettings**
   - Should raise clear error
   - Error should be user-friendly

3. **Test from view endpoints**
   - Should return proper error response
   - Should include request_id
   - Should follow unified API format

---

## Migration Steps

### Step 1: Backup Current Code
- Create git branch: `refactor/remove-model-fallbacks`
- Commit current state

### Step 2: Update `settings.py`
- Remove `MODEL_CONFIG` dict
- Update `get_model_config()` function
- Add proper error handling

### Step 3: Update `ai_core.py`
- Remove `_default_model` initialization
- Update `run_ai_request()` to require model
- Add error handling

### Step 4: Update `engine.py`
- Validate account before calling `get_model_config()`
- Add error handling

### Step 5: Update `tasks.py`
- Validate account_id is provided
- Add error handling

### Step 6: Clean Up Orphan Code
- Remove orphan exports from `__init__.py`
- Remove unused helper functions from `settings.py`
- Remove unused imports from `__init__.py`
- Verify no broken imports

### Step 7: Update Tests
- Add unit tests for new error cases
- Update integration tests
- Test all 5 active functions

### Step 8: Test Manually
- Test each AI function with valid IntegrationSettings
- Test each AI function without IntegrationSettings
- Verify error messages are clear

### Step 9: Update Documentation
- Update AI framework docs
- Document new error handling
- Update API documentation

### Step 10: Deploy
- Deploy to staging
- Test in staging environment
- Deploy to production

---

## Rollback Plan

If issues arise:

1. **Immediate Rollback**
   - Revert git branch
   - Restore previous version
   - Monitor for issues

2. **Partial Rollback**
   - Keep IntegrationSettings changes
   - Restore MODEL_CONFIG as fallback temporarily
   - Fix issues incrementally

---

## Success Criteria

- [ ] All 5 active AI functions work with IntegrationSettings only
- [ ] Clear error messages when IntegrationSettings not configured
- [ ] No hardcoded model defaults remain
- [ ] No Django settings fallbacks remain
- [ ] Orphan code removed (orphan exports, unused functions)
- [ ] No broken imports after cleanup
- [ ] All tests pass
- [ ] Documentation updated
- [ ] Frontend handles errors gracefully

---

## Notes

- **IntegrationSettings Structure:** Each account must have `IntegrationSettings` with `integration_type='openai'` and `config` containing `model`, `max_tokens`, `temperature`
- **Account-Specific:** Each account configures their own models - no global defaults
- **Error Clarity:** All errors must be user-friendly and actionable
- **No Breaking Changes:** This is a refactoring, not a breaking change - existing accounts with IntegrationSettings will continue to work

---

**Last Updated:** 2025-01-XX  
**Status:** Ready for Implementation