Add AI framework refactoring plan and orphan code audit

- Add REFACTORING-PLAN.md: Plan to remove MODEL_CONFIG and Django settings fallbacks
- Add ORPHAN-CODE-AUDIT.md: Audit of unused code and exports
- Update CHANGELOG.md: Document API Standard v1.0 completion
- Update documentation files
This commit is contained in:
IGNY8 VPS (Salman)
2025-11-16 09:15:07 +00:00
parent 7cd0e1a807
commit 60ffc12e8c
6 changed files with 821 additions and 594 deletions

View File

@@ -1298,7 +1298,15 @@ from igny8_core.api.viewsets import BaseTenantViewSet
---
**Document Status:** Active Standard
**Document Status:** Active Standard - ✅ **100% IMPLEMENTED**
**Last Updated:** 2025-01-XX
**Implementation Status:** All requirements implemented and verified
- ✅ All endpoints return unified response format
- ✅ All errors fully unified with request_id tracking
- ✅ All ViewSets use proper base classes, pagination, throttles, and permissions
- ✅ All custom @action methods use unified format
- ✅ All public endpoints properly configured
- ✅ Health check endpoint `/api/v1/system/ping/` implemented
- ✅ All endpoints documented in Swagger/ReDoc
**Next Review:** Quarterly or as needed

View File

@@ -1,593 +0,0 @@
# Implementation Plan - Remaining API Standard v1.0 Items
**Date:** 2025-01-XX
**Status:** Planning
**Purpose:** Complete implementation plan for remaining API Standard v1.0 compliance items
---
## Executive Summary
**Total Items:** 10 remaining items
**Critical Priority:** 2 items
**High Priority:** 4 items
**Medium Priority:** 4 items
**Estimated Time:** 12-16 hours
**Risk Level:** Low (mostly adding permissions and refactoring response formats)
---
## Table of Contents
1. [Critical Priority Items](#critical-priority-items)
2. [High Priority Items](#high-priority-items)
3. [Medium Priority Items](#medium-priority-items)
4. [Implementation Order](#implementation-order)
5. [Testing Strategy](#testing-strategy)
6. [Rollback Plan](#rollback-plan)
---
## Critical Priority Items
### 1. Fix Auth Endpoints to Use Unified Format
**Issue:** `RegisterView`, `LoginView`, `ChangePasswordView`, `MeView` in `backend/igny8_core/auth/urls.py` use raw `Response()` instead of unified format helpers.
**Impact:** These are the most frequently used endpoints and don't comply with the standard.
**Files to Modify:**
- `backend/igny8_core/auth/urls.py`
**Implementation Steps:**
1. **Update RegisterView** (lines 41-58)
```python
# BEFORE:
return Response({
'success': True,
'message': 'Registration successful',
'user': user_serializer.data
}, status=status.HTTP_201_CREATED)
# AFTER:
from igny8_core.api.response import success_response, error_response
return success_response(
data={'user': user_serializer.data},
message='Registration successful',
status_code=status.HTTP_201_CREATED,
request=request
)
```
2. **Update LoginView** (lines 66-134)
- Replace all `Response()` calls with `success_response()` or `error_response()`
- Ensure `request_id` is included
- Fix error response format to use `error` + `errors` structure
3. **Update ChangePasswordView** (lines 142-167)
- Replace `Response()` with `success_response()` and `error_response()`
- Ensure proper error format
4. **Update MeView** (lines 175-188)
- Replace `Response()` with `success_response()`
- Ensure `request_id` is included
5. **Add imports at top of file:**
```python
from igny8_core.api.response import success_response, error_response
```
**Testing:**
- Test registration with valid/invalid data
- Test login with valid/invalid credentials
- Test change password with valid/invalid old password
- Test /me endpoint
- Verify all responses have `success`, `data`/`error`, `request_id` fields
**Estimated Time:** 2 hours
---
### 2. Fix AIPromptViewSet Permissions
**Issue:** `AIPromptViewSet` has `permission_classes = []` which allows unauthenticated access.
**Impact:** Security vulnerability - anyone can access/modify AI prompts.
**Files to Modify:**
- `backend/igny8_core/modules/system/views.py` (line 42)
**Implementation Steps:**
1. **Update permission_classes:**
```python
# BEFORE:
permission_classes = [] # Allow any for now (backward compatibility)
# AFTER:
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
```
2. **Verify custom actions have proper permissions:**
- `get_by_type` - Should allow authenticated users
- `save_prompt` - Should require `IsEditorOrAbove` (per standard: "Saving prompts requires editor/admin")
- `reset_prompt` - Should require `IsEditorOrAbove`
3. **Update save_prompt action:**
```python
@action(detail=False, methods=['post'], url_path='save', url_name='save')
def save_prompt(self, request):
"""Save or update a prompt - requires editor or above"""
# Add permission check
from igny8_core.api.permissions import IsEditorOrAbove
if not IsEditorOrAbove().has_permission(request, self):
return error_response(
error='Permission denied',
status_code=status.HTTP_403_FORBIDDEN,
request=request
)
# ... rest of implementation
```
**Testing:**
- Test unauthenticated access (should return 401)
- Test authenticated user access (should work)
- Test save_prompt with viewer role (should return 403)
- Test save_prompt with editor role (should work)
**Estimated Time:** 1 hour
---
## High Priority Items
### 3. Update Billing ViewSets to Use Standard Permissions
**Issue:** All billing ViewSets use `permissions.IsAuthenticated` instead of `IsAuthenticatedAndActive + HasTenantAccess`.
**Files to Modify:**
- `backend/igny8_core/modules/billing/views.py`
**Implementation Steps:**
1. **Update CreditBalanceViewSet** (line 35)
```python
# BEFORE:
permission_classes = [permissions.IsAuthenticated]
# AFTER:
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
```
2. **Update CreditUsageViewSet** (line 91)
```python
# BEFORE:
permission_classes = [permissions.IsAuthenticated]
# AFTER:
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
```
3. **Update CreditTransactionViewSet** (line 466)
```python
# BEFORE:
permission_classes = [permissions.IsAuthenticated]
# AFTER:
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner]
```
Note: Transactions require admin/owner per standard.
**Testing:**
- Test with unauthenticated user (should return 401)
- Test with authenticated user from different account (should return 403)
- Test with authenticated user from same account (should work)
- Test CreditTransactionViewSet with viewer/editor (should return 403)
- Test CreditTransactionViewSet with admin/owner (should work)
**Estimated Time:** 1.5 hours
---
### 4. Update System Settings ViewSets to Use Standard Permissions
**Issue:** All 5 system settings ViewSets use `permissions.IsAuthenticated` instead of standard permissions.
**Files to Modify:**
- `backend/igny8_core/modules/system/settings_views.py`
**Implementation Steps:**
1. **Update SystemSettingsViewSet** (line 37)
```python
# BEFORE:
permission_classes = [permissions.IsAuthenticated]
# AFTER:
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
# Update get_permissions() method to use IsAdminOrOwner for write operations
def get_permissions(self):
if self.action in ['create', 'update', 'partial_update', 'destroy']:
return [IsAdminOrOwner()]
return [IsAuthenticatedAndActive(), HasTenantAccess()]
```
2. **Update AccountSettingsViewSet** (line 88)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
# Write operations should require IsAdminOrOwner (already handled by get_permissions)
```
3. **Update UserSettingsViewSet** (line 148)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
```
4. **Update ModuleSettingsViewSet** (line 214)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
```
5. **Update AISettingsViewSet** (line 293)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess]
```
**Testing:**
- Test read operations with authenticated user (should work)
- Test write operations with viewer/editor (should return 403)
- Test write operations with admin/owner (should work)
- Test with user from different account (should return 403)
**Estimated Time:** 2 hours
---
## Medium Priority Items
### 5. Refactor Billing ViewSets to Use AccountModelViewSet
**Issue:** Billing ViewSets manually filter by account instead of using base class.
**Files to Modify:**
- `backend/igny8_core/modules/billing/views.py`
**Implementation Steps:**
1. **Refactor CreditBalanceViewSet:**
- Change from `viewsets.ViewSet` to inherit from a base class
- Since it's a custom action-only ViewSet, keep as ViewSet but ensure account filtering
- Verify account is set correctly from request
2. **Refactor CreditUsageViewSet:**
```python
# BEFORE:
class CreditUsageViewSet(viewsets.ReadOnlyModelViewSet):
def get_queryset(self):
account = getattr(self.request, 'account', None)
# ... manual filtering
# AFTER:
from igny8_core.api.base import AccountModelViewSet
class CreditUsageViewSet(AccountModelViewSet):
# Base class handles account filtering automatically
def get_queryset(self):
queryset = super().get_queryset()
# Only add custom filtering here (operation_type, date range)
# Account filtering is automatic
```
3. **Refactor CreditTransactionViewSet:**
```python
# BEFORE:
class CreditTransactionViewSet(viewsets.ReadOnlyModelViewSet):
def get_queryset(self):
account = getattr(self.request, 'account', None)
# ... manual filtering
# AFTER:
class CreditTransactionViewSet(AccountModelViewSet):
# Base class handles account filtering automatically
```
**Testing:**
- Verify account filtering works correctly
- Test with admin/developer (should see all accounts)
- Test with regular user (should only see their account)
- Verify custom filters still work (operation_type, date range)
**Estimated Time:** 2 hours
---
### 6. Refactor IntegrationSettingsViewSet to Use AccountModelViewSet
**Issue:** `IntegrationSettingsViewSet` inherits from `viewsets.ViewSet` instead of `AccountModelViewSet`.
**Files to Modify:**
- `backend/igny8_core/modules/system/integration_views.py`
**Implementation Steps:**
1. **Change base class:**
```python
# BEFORE:
class IntegrationSettingsViewSet(viewsets.ViewSet):
# AFTER:
from igny8_core.api.base import AccountModelViewSet
class IntegrationSettingsViewSet(AccountModelViewSet):
```
2. **Update methods to use base class features:**
- `get_queryset()` - Use base class method if model has account field
- `perform_create()` - Use base class to set account
- Verify account scoping works correctly
3. **Note:** This ViewSet uses custom URL patterns, so ensure base class doesn't conflict.
**Testing:**
- Test integration settings retrieval (should be account-scoped)
- Test saving integration settings (should set account automatically)
- Test with user from different account (should not see other account's settings)
**Estimated Time:** 1.5 hours
---
### 7. Add Explicit Standard Permissions to Auth ViewSets
**Issue:** Auth ViewSets inherit from `AccountModelViewSet` but don't explicitly set standard permissions.
**Files to Modify:**
- `backend/igny8_core/auth/views.py`
**Implementation Steps:**
1. **Update UsersViewSet** (line 135)
```python
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner]
```
2. **Update AccountsViewSet** (line 270)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner]
```
3. **Update SubscriptionsViewSet** (line 335)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner]
```
4. **Update SiteViewSet** (line 472)
```python
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess, IsEditorOrAbove
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsEditorOrAbove]
```
5. **Update SectorViewSet** (line 715)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsEditorOrAbove]
```
6. **Update SiteUserAccessViewSet** (line 396)
```python
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner]
```
**Testing:**
- Test each ViewSet with appropriate roles
- Verify tenant isolation works
- Test with users from different accounts
**Estimated Time:** 1.5 hours
---
### 8. Add Role-Based Permissions to CreditTransactionViewSet
**Issue:** `CreditTransactionViewSet` should require `IsAdminOrOwner` per standard but currently only requires `IsAuthenticated`.
**Files to Modify:**
- `backend/igny8_core/modules/billing/views.py` (line 466)
**Implementation Steps:**
1. **Update permissions:**
```python
# This is already covered in item #3, but ensure it's implemented:
from igny8_core.api.permissions import IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner
permission_classes = [IsAuthenticatedAndActive, HasTenantAccess, IsAdminOrOwner]
```
2. **Verify in standard:** Per API Standard v1.0, billing/transactions require admin/owner.
**Testing:**
- Test with viewer role (should return 403)
- Test with editor role (should return 403)
- Test with admin role (should work)
- Test with owner role (should work)
**Estimated Time:** 0.5 hours (included in item #3)
---
## Implementation Order
### Phase 1: Critical Security Fixes (3 hours)
1. ✅ Fix AIPromptViewSet permissions (1 hour) - **CRITICAL SECURITY**
2. ✅ Fix auth endpoints unified format (2 hours) - **HIGH VISIBILITY**
### Phase 2: High Priority Permissions (3.5 hours)
3. ✅ Update billing ViewSets permissions (1.5 hours)
4. ✅ Update system settings ViewSets permissions (2 hours)
### Phase 3: Base Class Refactoring (3.5 hours)
5. ✅ Refactor billing ViewSets to use AccountModelViewSet (2 hours)
6. ✅ Refactor IntegrationSettingsViewSet (1.5 hours)
### Phase 4: Auth Module Permissions (1.5 hours)
7. ✅ Add explicit permissions to auth ViewSets (1.5 hours)
**Total Estimated Time:** 11.5 hours
---
## Testing Strategy
### Unit Tests
**File:** `backend/igny8_core/api/tests/test_auth_endpoints.py` (NEW)
```python
class AuthEndpointsTestCase(TestCase):
def test_register_returns_unified_format(self):
"""Test register endpoint returns unified format"""
response = self.client.post('/api/v1/auth/register/', {
'email': 'test@example.com',
'password': 'testpass123',
'password_confirm': 'testpass123'
})
data = response.json()
self.assertIn('success', data)
self.assertTrue(data['success'])
self.assertIn('data', data)
self.assertIn('request_id', data)
def test_login_returns_unified_format(self):
"""Test login endpoint returns unified format"""
# ... test implementation
def test_change_password_returns_unified_format(self):
"""Test change password endpoint returns unified format"""
# ... test implementation
def test_me_returns_unified_format(self):
"""Test /me endpoint returns unified format"""
# ... test implementation
```
**File:** `backend/igny8_core/api/tests/test_permissions_billing.py` (NEW)
```python
class BillingPermissionsTestCase(TestCase):
def test_credit_balance_requires_authentication(self):
"""Test credit balance requires authentication"""
# ... test implementation
def test_credit_transactions_requires_admin(self):
"""Test credit transactions requires admin/owner"""
# ... test implementation
```
### Integration Tests
**Update:** `backend/igny8_core/api/tests/test_integration_auth.py`
- Add tests for auth endpoints unified format
- Test permission enforcement
**Update:** `backend/igny8_core/api/tests/test_integration_billing.py`
- Add tests for permission enforcement
- Test account scoping
**Update:** `backend/igny8_core/api/tests/test_integration_system.py`
- Add tests for AIPromptViewSet permissions
- Test system settings permissions
### Manual Testing Checklist
- [ ] Test registration with valid/invalid data
- [ ] Test login with valid/invalid credentials
- [ ] Test change password
- [ ] Test /me endpoint
- [ ] Test AIPromptViewSet with unauthenticated user (should fail)
- [ ] Test AIPromptViewSet save_prompt with viewer (should fail)
- [ ] Test billing endpoints with different roles
- [ ] Test system settings with different roles
- [ ] Test account isolation on all endpoints
- [ ] Verify all responses have unified format
---
## Rollback Plan
### If Issues Arise:
1. **Immediate Rollback:**
- Revert git commits for problematic changes
- Restore previous version
- Monitor for issues
2. **Partial Rollback:**
- Revert specific module changes
- Keep other improvements
- Fix issues incrementally
3. **Feature Flag:**
- Add feature flag for new permissions
- Disable if issues found
- Re-enable after fixes
### Rollback Order (if needed):
1. Auth endpoints (most visible)
2. Permissions (security critical)
3. Base class refactoring (less visible)
---
## Success Criteria
### Definition of Done
- [ ] All auth endpoints return unified format
- [ ] All ViewSets use standard permissions
- [ ] All ViewSets use base classes where applicable
- [ ] All tests pass
- [ ] Manual testing completed
- [ ] No security vulnerabilities
- [ ] CHANGELOG updated with accurate information
### Metrics
- **Coverage:** 100% of remaining items implemented
- **Test Coverage:** >90% for modified code
- **Security:** No endpoints allow unauthenticated access (except public endpoints)
- **Compliance:** 100% compliance with API Standard v1.0
---
## Notes
- All changes should maintain backward compatibility where possible
- Permission changes may require frontend updates if error handling changes
- Test thoroughly before deploying to production
- Update CHANGELOG.md after completion to reflect accurate status
---
**Document Status:** Implementation Plan
**Last Updated:** 2025-01-XX
**Next Steps:** Begin Phase 1 implementation