igny8/docs/07-IGNY8-SYSTEM-COMPLETE-OVERVIEW.md

# IGNY8 System Complete Overview

## Executive Summary

IGNY8 is a comprehensive AI-powered content generation and management platform built on Django (backend) and React (frontend). The system enables automated content creation workflows from keyword research through publication, with WordPress integration for seamless content distribution.

## System Architecture

### Core Technology Stack

**Backend:**
- Django 4.2+ with Django REST Framework
- PostgreSQL database
- Celery for async task processing
- Redis for caching and message broker
- JWT authentication

**Frontend:**
- React 18+ with TypeScript
- Vite build system
- Zustand for state management
- TailwindCSS for styling
- React Router for navigation

**AI Integration:**
- OpenAI GPT-4 for content generation
- Runware for image generation
- Custom AI orchestration layer

## Multi-Tenant Architecture

### Account Hierarchy

```
Account (Organization)
  └── Site (Website/Project)
      └── Sector (Content Category)
          └── Content/Keywords/Tasks
```

**Key Models:**
- [`Account`](backend/igny8_core/auth/models.py:15): Top-level organization
- [`Site`](backend/igny8_core/auth/models.py:45): Individual website/project
- [`Sector`](backend/igny8_core/auth/models.py:75): Content category within site
- [`User`](backend/igny8_core/auth/models.py:105): User with role-based permissions

### Data Isolation

All business models inherit from:
- [`AccountBaseModel`](backend/igny8_core/auth/models.py:135): Account-level isolation
- [`SiteSectorBaseModel`](backend/igny8_core/auth/models.py:155): Site/Sector-level isolation

## Authentication & Authorization

### JWT Authentication Flow

1. User logs in via [`/api/v1/auth/login/`](backend/igny8_core/modules/auth/views.py:25)
2. Backend validates credentials and returns JWT tokens
3. Frontend stores tokens in localStorage
4. [`AccountContextMiddleware`](backend/igny8_core/auth/middleware.py:15) extracts account from token
5. All requests include JWT in Authorization header

### Permission System

**Roles (Hierarchical):**
- Owner: Full access
- Admin: Manage users and settings
- Editor: Create and edit content
- Viewer: Read-only access

**Implementation:**
- [`IsAuthenticatedAndActive`](backend/igny8_core/api/permissions.py:10): Base permission
- [`IsViewerOrAbove`](backend/igny8_core/api/permissions.py:25): Minimum viewer role
- [`IsEditorOrAbove`](backend/igny8_core/api/permissions.py:40): Minimum editor role

## API Architecture

### Unified API Standard v1.0

All API responses follow this format:

```json
{
  "success": true,
  "data": { /* response data */ },
  "message": "Operation completed successfully",
  "request_id": "uuid-v4",
  "timestamp": "2024-01-01T00:00:00Z"
}
```

**Error Response:**
```json
{
  "success": false,
  "error": "Error message",
  "error_code": "ERROR_CODE",
  "request_id": "uuid-v4",
  "timestamp": "2024-01-01T00:00:00Z"
}
```

### Base ViewSets

**[`AccountModelViewSet`](backend/igny8_core/api/base.py:25):**
- Automatic account filtering
- Used for account-level resources

**[`SiteSectorModelViewSet`](backend/igny8_core/api/base.py:65):**
- Automatic site/sector filtering
- Used for content-related resources
- Supports site/sector switching

## Core Modules

### 1. Planner Module

**Purpose:** Keyword research and content planning

**Key Components:**
- [`Keyword`](backend/igny8_core/business/planning/models.py:15): Individual keywords with metrics
- [`Cluster`](backend/igny8_core/business/planning/models.py:85): Grouped keywords by topic
- [`Task`](backend/igny8_core/business/planning/models.py:155): Content creation tasks

**AI Functions:**
- [`auto_cluster()`](backend/igny8_core/ai/functions/auto_cluster.py:15): Automatic keyword clustering
- [`generate_ideas()`](backend/igny8_core/ai/functions/generate_ideas.py:15): Content idea generation

**API Endpoints:**
- `GET/POST /api/v1/planner/keywords/`
- `GET/POST /api/v1/planner/clusters/`
- `GET/POST /api/v1/planner/tasks/`
- `POST /api/v1/planner/keywords/bulk-import/`
- `POST /api/v1/planner/clusters/{id}/auto-cluster/`

### 2. Writer Module

**Purpose:** AI-powered content generation

**Key Components:**
- [`Content`](backend/igny8_core/business/content/models.py:15): Generated content with metadata
- [`ContentVersion`](backend/igny8_core/business/content/models.py:125): Version history
- [`ImagePrompt`](backend/igny8_core/business/content/models.py:195): AI image generation prompts
- [`GeneratedImage`](backend/igny8_core/business/content/models.py:265): Generated images

**AI Functions:**
- [`generate_content()`](backend/igny8_core/ai/functions/generate_content.py:15): Article generation
- [`generate_image_prompts()`](backend/igny8_core/ai/functions/generate_image_prompts.py:15): Image prompt creation
- [`generate_images()`](backend/igny8_core/ai/functions/generate_images.py:15): Image generation via Runware

**API Endpoints:**
- `GET/POST /api/v1/writer/content/`
- `POST /api/v1/writer/content/{id}/generate/`
- `POST /api/v1/writer/content/{id}/regenerate/`
- `GET/POST /api/v1/writer/image-prompts/`
- `POST /api/v1/writer/images/generate/`

### 3. Integration Module

**Purpose:** WordPress bidirectional sync

**Key Components:**
- [`Integration`](backend/igny8_core/business/integration/models.py:15): WordPress site connection
- [`SyncLog`](backend/igny8_core/business/integration/models.py:85): Sync operation history

**WordPress Plugin:**
- REST API endpoints for content sync
- Webhook notifications for changes
- Authentication via API keys

**API Endpoints:**
- `GET/POST /api/v1/integration/integrations/`
- `POST /api/v1/integration/integrations/{id}/test-connection/`
- `POST /api/v1/integration/integrations/{id}/sync/`
- `POST /api/v1/integration/integrations/{id}/publish/`

**Sync Flow:**
1. User creates Integration with WordPress URL and credentials
2. System validates connection
3. Content can be pushed to WordPress
4. WordPress plugin sends webhooks on changes
5. System pulls updates and maintains sync

### 4. Automation Module

**Purpose:** Automated workflow execution

**Key Components:**
- [`AutomationRule`](backend/igny8_core/business/automation/models.py:11): Workflow definitions
- [`ScheduledTask`](backend/igny8_core/business/automation/models.py:93): Scheduled executions

**Services:**
- [`AutomationService`](backend/igny8_core/business/automation/services/automation_service.py:15): Main orchestrator
- [`RuleEngine`](backend/igny8_core/business/automation/services/rule_engine.py:15): Condition evaluation
- [`ActionExecutor`](backend/igny8_core/business/automation/services/action_executor.py:13): Action execution

**Rule Structure:**
```json
{
  "name": "Auto Content Pipeline",
  "trigger": "schedule",
  "schedule": "0 0 * * *",
  "conditions": [
    {
      "field": "task.status",
      "operator": "equals",
      "value": "pending"
    }
  ],
  "actions": [
    {
      "type": "generate_content",
      "params": {
        "task_ids": ["task-id"]
      }
    },
    {
      "type": "generate_images",
      "params": {
        "content_id": "content-id"
      }
    }
  ]
}
```

**API Endpoints:**
- `GET/POST /api/v1/automation/rules/`
- `POST /api/v1/automation/rules/{id}/execute/`
- `GET /api/v1/automation/scheduled-tasks/`

## AI Framework

### Architecture

**[`AIEngine`](backend/igny8_core/ai/engine.py:25):**
- Central orchestrator for all AI operations
- Manages function registry and execution
- Handles progress tracking and error handling

**[`AICore`](backend/igny8_core/ai/core.py:15):**
- Low-level API client for OpenAI/Runware
- Request/response handling
- Rate limiting and retry logic

**[`AI Function Registry`](backend/igny8_core/ai/registry.py:15):**
- Lazy loading of AI functions
- Function metadata and validation
- Dependency injection

### AI Function Structure

Each AI function follows this pattern:

```python
@ai_function(
    name="function_name",
    description="Function description",
    cost_credits=10
)
def my_ai_function(account, **kwargs):
    """
    AI function implementation
    
    Args:
        account: Account instance
        **kwargs: Function-specific parameters
    
    Returns:
        dict: Result with success status
    """
    # Implementation
    pass
```

### Progress Tracking

**Celery Integration:**
- Each AI operation runs as Celery task
- Progress updates via task state
- Real-time updates to frontend

**Progress Phases:**
1. Initializing: Setup and validation
2. Processing: Main AI operation
3. AI Analysis: Post-processing
4. Saving: Database persistence
5. Completed: Final state

### Cost Management

**Credit System:**
- [`CreditService`](backend/igny8_core/business/billing/services/credit_service.py:15): Credit management
- Pre-execution credit checks
- Post-execution credit deduction
- Usage tracking and reporting

**Credit Costs:**
- Content generation: 1 credit per 500 words
- Image prompts: 0.5 credits per prompt
- Image generation: 1-4 credits (provider-dependent)
- Keyword clustering: 10 credits per operation

## Frontend Architecture

### State Management (Zustand)

**Global Stores:**
- [`authStore`](frontend/src/stores/authStore.ts:10): Authentication state
- [`siteStore`](frontend/src/stores/siteStore.ts:10): Site/sector selection
- [`contentStore`](frontend/src/stores/contentStore.ts:10): Content management

**Store Pattern:**
```typescript
interface Store {
  // State
  data: any[];
  loading: boolean;
  error: string | null;
  
  // Actions
  fetchData: () => Promise<void>;
  createItem: (item: any) => Promise<void>;
  updateItem: (id: string, item: any) => Promise<void>;
  deleteItem: (id: string) => Promise<void>;
}
```

### API Client

**[`apiClient`](frontend/src/api/client.ts:15):**
- Axios-based HTTP client
- Automatic JWT token injection
- Request/response interceptors
- Error handling and retry logic

### Component Structure

**Layout Components:**
- [`DefaultLayout`](frontend/src/components/shared/layouts/DefaultLayout.tsx:10): Main app layout
- [`Header`](frontend/src/components/header/Header.tsx:10): Top navigation
- [`Sidebar`](frontend/src/components/sidebar/Sidebar.tsx:10): Side navigation

**Common Components:**
- [`ProgressModal`](frontend/src/components/common/ProgressModal.tsx:10): AI operation progress
- [`ConfirmDialog`](frontend/src/components/common/ConfirmDialog.tsx:10): Confirmation dialogs
- [`FormModal`](frontend/src/components/common/FormModal.tsx:10): Generic form modals

### Routing

**Protected Routes:**
```typescript
<ProtectedRoute>
  <Route path="/planner" element={<PlannerPage />} />
  <Route path="/writer" element={<WriterPage />} />
  <Route path="/integration" element={<IntegrationPage />} />
  <Route path="/automation" element={<AutomationPage />} />
</ProtectedRoute>
```

## Database Schema

### Core Tables

**Authentication:**
- `igny8_accounts`: Organizations
- `igny8_sites`: Websites/projects
- `igny8_sectors`: Content categories
- `igny8_users`: User accounts

**Planning:**
- `igny8_keywords`: Keyword data
- `igny8_clusters`: Keyword clusters
- `igny8_tasks`: Content tasks

**Content:**
- `igny8_content`: Generated content
- `igny8_content_versions`: Version history
- `igny8_image_prompts`: Image prompts
- `igny8_generated_images`: Generated images

**Integration:**
- `igny8_integrations`: WordPress connections
- `igny8_sync_logs`: Sync history

**Automation:**
- `igny8_automation_rules`: Workflow rules
- `igny8_scheduled_tasks`: Scheduled executions

**Billing:**
- `igny8_credit_transactions`: Credit usage
- `igny8_subscriptions`: Subscription plans

## Deployment

### Docker Configuration

**Services:**
- `backend`: Django application
- `frontend`: React application
- `postgres`: PostgreSQL database
- `redis`: Redis cache/broker
- `celery`: Celery worker
- `celery-beat`: Celery scheduler

**[`docker-compose.app.yml`](docker-compose.app.yml:1):**
```yaml
services:
  backend:
    build: ./backend
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://...
      - REDIS_URL=redis://...
  
  frontend:
    build: ./frontend
    ports:
      - "3000:3000"
  
  postgres:
    image: postgres:15
    volumes:
      - postgres_data:/var/lib/postgresql/data
  
  redis:
    image: redis:7-alpine
  
  celery:
    build: ./backend
    command: celery -A igny8_core worker -l info
  
  celery-beat:
    build: ./backend
    command: celery -A igny8_core beat -l info
```

### Environment Variables

**Backend (.env):**
```
DEBUG=False
SECRET_KEY=your-secret-key
DATABASE_URL=postgresql://user:pass@host:5432/db
REDIS_URL=redis://host:6379/0
OPENAI_API_KEY=sk-...
RUNWARE_API_KEY=...
```

**Frontend (.env):**
```
VITE_API_URL=http://localhost:8000/api/v1
VITE_WS_URL=ws://localhost:8000/ws
```

## Security Considerations

### Authentication Security

1. **JWT Tokens:**
   - Access token: 1 hour expiry
   - Refresh token: 7 days expiry
   - Secure HTTP-only cookies (optional)

2. **Password Security:**
   - Django's PBKDF2 hashing
   - Minimum 8 characters
   - Complexity requirements

3. **API Security:**
   - Rate limiting per endpoint
   - CORS configuration
   - CSRF protection

### Data Security

1. **Multi-Tenant Isolation:**
   - Automatic account filtering
   - Row-level security
   - No cross-account data access

2. **Permission Checks:**
   - Role-based access control
   - Object-level permissions
   - Audit logging

3. **Sensitive Data:**
   - API keys encrypted at rest
   - WordPress credentials secured
   - PII data protection

## Performance Optimization

### Backend Optimization

1. **Database:**
   - Indexed foreign keys
   - Composite indexes for common queries
   - Query optimization with select_related/prefetch_related

2. **Caching:**
   - Redis for session storage
   - API response caching
   - Database query caching

3. **Async Processing:**
   - Celery for long-running tasks
   - Background job processing
   - Task prioritization

### Frontend Optimization

1. **Code Splitting:**
   - Route-based splitting
   - Lazy loading components
   - Dynamic imports

2. **State Management:**
   - Zustand for minimal re-renders
   - Selective subscriptions
   - Memoization

3. **Asset Optimization:**
   - Image lazy loading
   - SVG optimization
   - Bundle size monitoring

## Monitoring & Logging

### Backend Logging

**Log Levels:**
- DEBUG: Development debugging
- INFO: General information
- WARNING: Warning messages
- ERROR: Error conditions
- CRITICAL: Critical failures

**Log Destinations:**
- Console output
- File rotation
- External logging service (optional)

### Frontend Monitoring

**Error Tracking:**
- React Error Boundaries
- API error logging
- User action tracking

**Performance Monitoring:**
- Page load times
- API response times
- Component render times

## Testing Strategy

### Backend Testing

**Unit Tests:**
- Model validation
- Service logic
- Utility functions

**Integration Tests:**
- API endpoints
- Database operations
- External API calls

**Test Coverage:**
- Minimum 80% coverage
- Critical paths 100% coverage

### Frontend Testing

**Component Tests:**
- React Testing Library
- Component rendering
- User interactions

**Integration Tests:**
- API integration
- State management
- Routing

## Development Workflow

### Local Development

1. **Backend Setup:**
```bash
cd backend
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt
python manage.py migrate
python manage.py runserver
```

2. **Frontend Setup:**
```bash
cd frontend
npm install
npm run dev
```

3. **Celery Worker:**
```bash
cd backend
celery -A igny8_core worker -l info
```

### Git Workflow

1. Create feature branch from `develop`
2. Implement changes with tests
3. Create pull request
4. Code review and approval
5. Merge to `develop`
6. Deploy to staging
7. Merge to `main` for production

## API Documentation

### Interactive Documentation

**Swagger UI:**
- URL: `/api/schema/swagger-ui/`
- Interactive API testing
- Request/response examples

**ReDoc:**
- URL: `/api/schema/redoc/`
- Clean documentation view
- Searchable endpoints

### API Versioning

Current version: **v1.0**

All endpoints prefixed with `/api/v1/`

Future versions will use `/api/v2/`, etc.

## Troubleshooting

### Common Issues

**1. Database Connection Errors:**
- Check DATABASE_URL in .env
- Verify PostgreSQL is running
- Check network connectivity

**2. Celery Task Failures:**
- Check Redis connection
- Verify Celery worker is running
- Check task logs

**3. Authentication Issues:**
- Verify JWT token validity
- Check token expiration
- Validate user permissions

**4. WordPress Sync Failures:**
- Test WordPress connection
- Verify API credentials
- Check WordPress plugin version

## Future Enhancements

### Planned Features

1. **Advanced Automation:**
   - Conditional workflows
   - Multi-step pipelines
   - Error recovery

2. **Enhanced AI:**
   - Multiple AI providers
   - Custom model fine-tuning
   - Advanced prompt engineering

3. **Collaboration:**
   - Team workspaces
   - Content approval workflows
   - Comment system

4. **Analytics:**
   - Content performance tracking
   - SEO metrics
   - Usage analytics

5. **Integrations:**
   - Additional CMS platforms
   - Social media publishing
   - Email marketing tools

## Support & Resources

### Documentation

- System Architecture: [`docs/00-SYSTEM-ARCHITECTURE-MASTER-REFERENCE.md`](docs/00-SYSTEM-ARCHITECTURE-MASTER-REFERENCE.md:1)
- REST API: [`docs/01-IGNY8-REST-API-COMPLETE-REFERENCE.md`](docs/01-IGNY8-REST-API-COMPLETE-REFERENCE.md:1)
- WordPress Integration: [`docs/03-WORDPRESS-PLUGIN-API-INTEGRATION-GUIDE.md`](docs/03-WORDPRESS-PLUGIN-API-INTEGRATION-GUIDE.md:1)
- Automation: [`docs/automation-designer-mode-specification.md`](docs/automation-designer-mode-specification.md:1)

### Contact

- Technical Support: support@igny8.com
- Documentation: docs.igny8.com
- GitHub: github.com/igny8/igny8-app

---

**Document Version:** 1.0  
**Last Updated:** 2024-11-30  
**Maintained By:** IGNY8 Development Team