275 lines
6.6 KiB
Markdown
275 lines
6.6 KiB
Markdown
# IGNY8 AI Framework Documentation
|
|
|
|
**Version:** 1.0
|
|
**Last Updated:** 2025-01-XX
|
|
**Purpose:** Complete documentation of the unified AI framework architecture.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
The IGNY8 AI Framework provides a unified, consistent architecture for all AI functions. It eliminates code duplication, standardizes progress tracking, and provides a single interface for all AI operations.
|
|
|
|
### Key Benefits
|
|
|
|
- **90% Code Reduction**: Functions are now ~100 lines instead of ~600
|
|
- **Consistent UX**: All functions use the same progress modal and tracking
|
|
- **Unified Logging**: Single `AITaskLog` table for all AI operations
|
|
- **Easy Extension**: Add new functions by creating one class
|
|
- **Better Debugging**: Detailed step-by-step tracking for all operations
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
igny8_core/ai/
|
|
├── __init__.py # Auto-registers all functions
|
|
├── apps.py # Django app configuration
|
|
├── admin.py # Admin interface for AITaskLog
|
|
├── base.py # BaseAIFunction abstract class
|
|
├── engine.py # AIEngine orchestrator
|
|
├── processor.py # AIProcessor wrapper
|
|
├── registry.py # Function registry
|
|
├── tracker.py # StepTracker, ProgressTracker, CostTracker
|
|
├── tasks.py # Unified Celery task entrypoint
|
|
├── types.py # Shared dataclasses
|
|
├── models.py # AITaskLog model
|
|
└── functions/ # Function implementations
|
|
├── __init__.py
|
|
└── auto_cluster.py # Auto cluster function
|
|
```
|
|
|
|
---
|
|
|
|
## Core Components
|
|
|
|
### 1. BaseAIFunction
|
|
|
|
Abstract base class that all AI functions inherit from.
|
|
|
|
**Methods to implement:**
|
|
- `get_name()`: Return function name
|
|
- `prepare()`: Load and prepare data
|
|
- `build_prompt()`: Build AI prompt
|
|
- `parse_response()`: Parse AI response
|
|
- `save_output()`: Save results to database
|
|
|
|
**Optional overrides:**
|
|
- `validate()`: Custom validation
|
|
- `get_max_items()`: Set item limit
|
|
- `get_model()`: Specify AI model
|
|
- `get_metadata()`: Function metadata
|
|
|
|
### 2. AIEngine
|
|
|
|
Central orchestrator that manages the execution pipeline.
|
|
|
|
**Phases:**
|
|
- INIT (0-10%): Validation & setup
|
|
- PREP (10-25%): Data loading & prompt building
|
|
- AI_CALL (25-60%): API call to provider
|
|
- PARSE (60-80%): Response parsing
|
|
- SAVE (80-95%): Database operations
|
|
- DONE (95-100%): Finalization
|
|
|
|
### 3. Function Registry
|
|
|
|
Dynamic function discovery system.
|
|
|
|
**Usage:**
|
|
```python
|
|
from igny8_core.ai.registry import register_function, get_function
|
|
|
|
# Register function
|
|
register_function('auto_cluster', AutoClusterFunction)
|
|
|
|
# Get function
|
|
fn = get_function('auto_cluster')
|
|
```
|
|
|
|
### 4. Unified Celery Task
|
|
|
|
Single entrypoint for all AI functions.
|
|
|
|
**Endpoint:** `run_ai_task(function_name, payload, account_id)`
|
|
|
|
**Example:**
|
|
```python
|
|
from igny8_core.ai.tasks import run_ai_task
|
|
|
|
task = run_ai_task.delay(
|
|
function_name='auto_cluster',
|
|
payload={'ids': [1, 2, 3], 'sector_id': 1},
|
|
account_id=1
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## Function Implementation Example
|
|
|
|
### Auto Cluster Function
|
|
|
|
```python
|
|
from igny8_core.ai.base import BaseAIFunction
|
|
|
|
class AutoClusterFunction(BaseAIFunction):
|
|
def get_name(self) -> str:
|
|
return 'auto_cluster'
|
|
|
|
def get_max_items(self) -> int:
|
|
return 20
|
|
|
|
def prepare(self, payload: dict, account=None) -> Dict:
|
|
# Load keywords
|
|
ids = payload.get('ids', [])
|
|
keywords = Keywords.objects.filter(id__in=ids)
|
|
return {'keywords': keywords, ...}
|
|
|
|
def build_prompt(self, data: Dict, account=None) -> str:
|
|
# Build clustering prompt
|
|
return prompt_template.replace('[IGNY8_KEYWORDS]', keywords_text)
|
|
|
|
def parse_response(self, response: str, step_tracker=None) -> List[Dict]:
|
|
# Parse AI response
|
|
return clusters
|
|
|
|
def save_output(self, parsed, original_data, account, progress_tracker) -> Dict:
|
|
# Save clusters to database
|
|
return {'clusters_created': 5, 'keywords_updated': 20}
|
|
```
|
|
|
|
---
|
|
|
|
## API Endpoint Example
|
|
|
|
### Before (Old): ~300 lines
|
|
|
|
### After (New): ~50 lines
|
|
|
|
```python
|
|
@action(detail=False, methods=['post'], url_path='auto_cluster')
|
|
def auto_cluster(self, request):
|
|
from igny8_core.ai.tasks import run_ai_task
|
|
|
|
account = getattr(request, 'account', None)
|
|
account_id = account.id if account else None
|
|
|
|
payload = {
|
|
'ids': request.data.get('ids', []),
|
|
'sector_id': request.data.get('sector_id')
|
|
}
|
|
|
|
task = run_ai_task.delay(
|
|
function_name='auto_cluster',
|
|
payload=payload,
|
|
account_id=account_id
|
|
)
|
|
|
|
return Response({
|
|
'success': True,
|
|
'task_id': str(task.id),
|
|
'message': 'Clustering started'
|
|
})
|
|
```
|
|
|
|
---
|
|
|
|
## Progress Tracking
|
|
|
|
### Unified Progress Endpoint
|
|
|
|
**URL:** `/api/v1/system/settings/task_progress/<task_id>/`
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"state": "PROGRESS",
|
|
"meta": {
|
|
"phase": "AI_CALL",
|
|
"percentage": 45,
|
|
"message": "Analyzing keyword relationships...",
|
|
"request_steps": [...],
|
|
"response_steps": [...],
|
|
"cost": 0.000123,
|
|
"tokens": 1500
|
|
}
|
|
}
|
|
```
|
|
|
|
### Frontend Integration
|
|
|
|
All AI functions use the same progress modal:
|
|
- Single `useProgressModal` hook
|
|
- Unified progress endpoint
|
|
- Consistent phase labels
|
|
- Step-by-step logs
|
|
|
|
---
|
|
|
|
## Database Logging
|
|
|
|
### AITaskLog Model
|
|
|
|
Unified logging table for all AI operations.
|
|
|
|
**Fields:**
|
|
- `task_id`: Celery task ID
|
|
- `function_name`: Function name
|
|
- `account`: Account (required)
|
|
- `phase`: Current phase
|
|
- `status`: success/error/pending
|
|
- `cost`: API cost
|
|
- `tokens`: Token usage
|
|
- `request_steps`: Request step logs
|
|
- `response_steps`: Response step logs
|
|
- `error`: Error message (if any)
|
|
|
|
---
|
|
|
|
## Migration Guide
|
|
|
|
### Migrating Existing Functions
|
|
|
|
1. Create function class inheriting `BaseAIFunction`
|
|
2. Implement required methods
|
|
3. Register function in `ai/__init__.py`
|
|
4. Update API endpoint to use `run_ai_task`
|
|
5. Test and remove old code
|
|
|
|
### Example Migration
|
|
|
|
**Old code:**
|
|
```python
|
|
@action(...)
|
|
def auto_cluster(self, request):
|
|
# 300 lines of code
|
|
```
|
|
|
|
**New code:**
|
|
```python
|
|
@action(...)
|
|
def auto_cluster(self, request):
|
|
# 20 lines using framework
|
|
```
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
The AI Framework provides:
|
|
|
|
1. **Unified Architecture**: Single framework for all AI functions
|
|
2. **Code Reduction**: 90% less code per function
|
|
3. **Consistent UX**: Same progress modal for all functions
|
|
4. **Better Debugging**: Detailed step tracking
|
|
5. **Easy Extension**: Add functions quickly
|
|
6. **Unified Logging**: Single log table
|
|
7. **Cost Tracking**: Automatic cost calculation
|
|
|
|
This architecture ensures maintainability, consistency, and extensibility while dramatically reducing code duplication.
|
|
|