salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

1

Browse Source
This commit is contained in:
alorig
2025-11-09 23:16:15 +05:00
parent 69d58d8bef
commit a622a3ffe3
11 changed files with 0 additions and 698 deletions
Download Patch File Download Diff File Expand all files Collapse all files

749
docs/Architecture/01-ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,749 @@
# IGNY8 System Architecture
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete system architecture documentation covering design patterns, principles, tech stack, and structural organization.
---
## Table of Contents
1. [System Overview](#system-overview)
2. [Tech Stack](#tech-stack)
3. [Core Architecture Principles](#core-architecture-principles)
4. [Project Structure](#project-structure)
5. [Key Architectural Patterns](#key-architectural-patterns)
6. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
7. [Module Organization](#module-organization)
8. [API Architecture](#api-architecture)
9. [Frontend Architecture](#frontend-architecture)
10. [Backend Architecture](#backend-architecture)
11. [Database Architecture](#database-architecture)
12. [Security Architecture](#security-architecture)
13. [Deployment Architecture](#deployment-architecture)
---
## System Overview
**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system migrated from a WordPress plugin architecture to a modern Django + React architecture, providing a scalable, multi-account platform for content planning and generation.
### Core Capabilities
- **Multi-Account SaaS Platform**: Complete account isolation with site/sector hierarchy
- **SEO Keyword Management**: Import, organize, and cluster keywords
- **AI-Powered Content Planning**: Automated keyword clustering and content idea generation
- **AI Content Generation**: Automated blog post and article generation
- **Image Generation**: AI-powered image generation for content
- **WordPress Integration**: Publish content directly to WordPress sites
- **Subscription Management**: Plan-based limits and billing integration
---
## Tech Stack
### Backend
- **Framework**: Django 5.2+ with Django REST Framework (DRF)
- **Database**: PostgreSQL
- **Task Queue**: Celery with Redis broker
- **Authentication**: JWT (JSON Web Tokens) + Session-based auth
- **API**: RESTful API with DRF ViewSets
- **Caching**: Redis
- **File Storage**: Local filesystem (configurable for S3)
### Frontend
- **Framework**: React 19 with TypeScript
- **Build Tool**: Vite
- **Styling**: Tailwind CSS
- **State Management**: Zustand
- **Routing**: React Router v6
- **HTTP Client**: Fetch API with custom wrapper
- **UI Components**: Custom component library
### Infrastructure
- **Containerization**: Docker & Docker Compose
- **Reverse Proxy**: Caddy (HTTPS on port 443)
- **Process Management**: Supervisor (for Celery workers)
- **Monitoring**: Portainer (optional)
### Development Tools
- **Backend**: Python 3.11+, pip, virtualenv
- **Frontend**: Node.js 18+, npm/yarn
- **Version Control**: Git
- **Code Quality**: ESLint, Prettier (frontend), Black, Flake8 (backend)
---
## Core Architecture Principles
### 1. Configuration-Driven Everything
**Principle**: Zero HTML/JSX duplication - All UI rendered from configuration.
- **Tables, filters, forms** all driven by config files
- **Single source of truth** - Change config, UI updates everywhere
- **Page-local config** for page-specific settings
- **Shared snippets** for reusable column/filter/action definitions
**Implementation**:
- Frontend: Config files in `/config/pages/` and `/config/snippets/`
- Backend: DRF serializers and ViewSet actions
### 2. Multi-Tenancy Foundation
**Principle**: Complete account isolation with automatic filtering.
- All models inherit `AccountBaseModel` with automatic account isolation
- All ViewSets inherit `AccountModelViewSet` with automatic account filtering
- Middleware injects account context from JWT on every request
- Site > Sector hierarchy for content organization
**Implementation**:
- Base models: `AccountBaseModel`, `SiteSectorBaseModel`
- Base ViewSets: `AccountModelViewSet`, `SiteSectorModelViewSet`
- Middleware: `AccountContextMiddleware` sets `request.account`
### 3. Template System (4 Universal Templates)
**Principle**: Reusable templates for all page types.
- **DashboardTemplate**: Module home pages (KPIs, workflow steps, charts)
- **TablePageTemplate**: CRUD table pages (Keywords, Clusters, Tasks, etc.)
- **FormPageTemplate**: Settings/form pages (Settings, Integration, etc.)
- **SystemPageTemplate**: System/admin pages (Logs, Status, Monitoring)
**Implementation**:
- Frontend: `/templates/` directory with 4 template components
- Config-driven: Templates accept config objects for customization
### 4. Unified AI Processor
**Principle**: Single interface for all AI operations.
- Single `AIProcessor` class handles all AI operations
- Manual and automated workflows use same functions
- Action-based routing: 'clustering', 'ideas', 'content_generation', 'image_generation'
- Account-specific API keys and model configuration
**Implementation**:
- Backend: `AIProcessor` class in `/utils/ai_processor.py`
- Integration: Loads API keys from `IntegrationSettings` model
### 5. Module-Based Organization
**Principle**: Clear module boundaries with shared utilities.
- Each module = Django app (`/modules/{module}/`)
- Clear module boundaries with shared utilities
- Module router pattern for subpage routing
- Consistent structure across modules
**Implementation**:
- Backend: `/modules/planner/`, `/modules/writer/`, `/modules/system/`, `/modules/billing/`
- Frontend: `/pages/Planner/`, `/pages/Writer/`, `/pages/Settings/`, etc.
---
## Project Structure
```
igny8/
├── backend/ # Django backend
│ └── igny8_core/ # Django project
│ ├── auth/ # Multi-tenancy, User, Account, Plan models
│ │ ├── models.py # Account, User, Plan, Site, Sector, Industry models
│ │ ├── views.py # Account, User, Site, Sector ViewSets
│ │ ├── serializers.py # Account, User, Plan serializers
│ │ └── urls.py # Auth module URLs
│ ├── modules/ # Feature modules
│ │ ├── planner/ # Keywords, Clusters, Ideas
│ │ │ ├── models.py # Keywords, Clusters, ContentIdeas models
│ │ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│ │ │ ├── tasks.py # Celery tasks for AI operations
│ │ │ ├── serializers.py # Model serializers
│ │ │ └── urls.py # Planner module URLs
│ │ ├── writer/ # Tasks, Content, Images
│ │ │ ├── models.py # Tasks, Content, Images models
│ │ │ ├── views.py # TasksViewSet
│ │ │ ├── tasks.py # Celery tasks for content/image generation
│ │ │ └── urls.py # Writer module URLs
│ │ ├── system/ # Settings, Prompts, Integration
│ │ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│ │ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet
│ │ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress
│ │ │ ├── utils.py # Default prompts, prompt loading
│ │ │ └── urls.py # System module URLs
│ │ └── billing/ # Credits, Transactions, Usage
│ │ ├── models.py # CreditTransaction, UsageLog models
│ │ ├── views.py # Billing ViewSets
│ │ └── services.py # CreditService
│ ├── api/ # API base classes
│ │ └── base.py # AccountModelViewSet, SiteSectorModelViewSet
│ ├── utils/ # Shared utilities
│ │ ├── ai_processor.py # Unified AI interface
│ │ └── content_normalizer.py # Content processing utilities
│ ├── middleware/ # Custom middleware
│ │ ├── account.py # AccountContextMiddleware (sets request.account)
│ │ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
│ ├── settings.py # Django settings
│ ├── urls.py # Root URL configuration
│ └── celery.py # Celery configuration
│
├── frontend/ # React frontend
│ └── src/
│ ├── pages/ # Page components
│ │ ├── Planner/ # KeywordsPage, ClustersPage, IdeasPage, Dashboard
│ │ ├── Writer/ # TasksPage, DraftsPage, PublishedPage, Dashboard
│ │ ├── Settings/ # General, Integration, Status, ImportExport
│ │ ├── Billing/ # Credits, Transactions, Usage
│ │ └── AuthPages/ # SignIn, SignUp
│ ├── templates/ # 4 master templates
│ │ ├── DashboardTemplate.tsx
│ │ ├── TablePageTemplate.tsx
│ │ ├── FormPageTemplate.tsx
│ │ └── SystemPageTemplate.tsx
│ ├── components/ # UI components
│ │ ├── layout/ # AppLayout, Sidebar, Header, Breadcrumbs
│ │ ├── table/ # DataTable, Filters, Actions, Pagination
│ │ ├── ui/ # Button, Card, Modal, Toast, etc.
│ │ └── auth/ # ProtectedRoute, Auth components
│ ├── config/ # Configuration files
│ │ ├── pages/ # Page-specific configs
│ │ │ └── keywords.config.tsx
│ │ ├── snippets/ # Shared column/filter/action definitions
│ │ │ ├── columns.snippets.ts
│ │ │ ├── filters.snippets.ts
│ │ │ └── actions.snippets.ts
│ │ └── routes.config.ts # Route configuration
│ ├── store/ # Zustand stores
│ │ ├── authStore.ts # Authentication state
│ │ ├── plannerStore.ts # Planner module state
│ │ ├── siteStore.ts # Site selection state
│ │ └── aiRequestLogsStore.ts # AI request/response logs
│ ├── services/ # API clients
│ │ └── api.ts # fetchAPI, API functions
│ ├── hooks/ # Custom React hooks
│ │ ├── useProgressModal.ts # Progress modal for long-running tasks
│ │ └── useAuth.ts # Authentication hook
│ ├── layout/ # Layout components
│ │ └── AppLayout.tsx # Main app layout wrapper
│ ├── App.tsx # Root component with routing
│ └── main.tsx # Entry point
│
└── docs/ # Documentation
└── ActiveDocs/ # Active documentation
├── 01-ARCHITECTURE.md # This file
├── 02-FRONTEND.md # Frontend documentation
├── 03-BACKEND.md # Backend documentation
├── 04-AI-FUNCTIONS.md # AI functions documentation
└── 05-ACCOUNT-USER-PLAN.md # Account/User/Plan documentation
```
---
## Key Architectural Patterns
### Configuration System (Page-Local Config + Shared Snippets)
**Rule**: Config = Page-Local, Snippets = Shared
**Structure**:
```
/pages/Planner/KeywordsPage.tsx
├── Imports snippets from /config/snippets/
├── Defines page-local tableConfig, filterConfig, actionsConfig
└── Passes config to TablePageTemplate
/config/snippets/
├── columns.snippets.ts # statusColumn, titleColumn, etc.
├── filters.snippets.ts # statusFilter, dateRangeFilter, etc.
├── actions.snippets.ts # commonActions, bulkActions, etc.
```
**Benefits**:
- Reusable components across pages
- Page-specific customization
- Single source of truth for shared definitions
### Base ViewSet Pattern
**Backend Pattern**: All ViewSets inherit from base classes for consistent behavior.
**Base Classes**:
- `AccountModelViewSet`: Automatic account filtering
- `SiteSectorModelViewSet`: Account + site/sector filtering + access control
**Benefits**:
- Consistent access control
- Automatic account isolation
- Reduced code duplication
### Celery Task Pattern
**Pattern**: Long-running operations use Celery tasks with progress tracking.
**Structure**:
1. API endpoint queues Celery task
2. Task updates progress via `update_state()`
3. Frontend polls `task_progress` endpoint
4. Progress displayed in modal
**Benefits**:
- Non-blocking API responses
- Real-time progress updates
- Scalable background processing
### AI Function Pattern
**Pattern**: All AI functions follow consistent structure.
**Structure**:
1. API Endpoint: Validates input, queues Celery task
2. Celery Task: Wraps core function with progress tracking
3. Core Function: Business logic, calls AIProcessor
4. AIProcessor: Makes API calls, returns structured data
5. Core Function: Saves results to database
**Benefits**:
- Consistent error handling
- Progress tracking
- Reusable AI interface
---
## Multi-Tenancy Architecture
### Account Isolation
**Principle**: All data is isolated by account.
**Implementation**:
- All models inherit `AccountBaseModel` (has `account` ForeignKey)
- All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
- Middleware sets `request.account` from JWT token
**Access Control**:
- Admin/Developer users: Bypass account filtering (see all accounts)
- System account users: Bypass account filtering (see all accounts)
- Regular users: Only see data from their account
### Site/Sector Hierarchy
**Structure**:
```
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
```
**Implementation**:
- Models inherit `SiteSectorBaseModel` (has `site` and `sector` ForeignKeys)
- ViewSets inherit `SiteSectorModelViewSet` (filters by accessible sites)
- User access control via `User.get_accessible_sites()`
**Site Access Control**:
- System account users: All active sites
- Developers: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`
---
## Module Organization
### Planner Module
**Purpose**: Keyword management and content planning.
**Models**:
- `Keywords`: Individual keywords with volume, difficulty, intent
- `Clusters`: Keyword clusters (groups of related keywords)
- `ContentIdeas`: Content ideas generated from clusters
**ViewSets**:
- `KeywordViewSet`: CRUD + `auto_cluster` action
- `ClusterViewSet`: CRUD + `auto_generate_ideas` action
- `ContentIdeasViewSet`: CRUD operations
**Tasks**:
- `auto_cluster_keywords_task`: AI-powered keyword clustering
- `auto_generate_ideas_task`: AI-powered content idea generation
### Writer Module
**Purpose**: Content generation and management.
**Models**:
- `Tasks`: Content generation tasks
- `Content`: Generated content (HTML)
- `Images`: Generated images for tasks
**ViewSets**:
- `TasksViewSet`: CRUD + `auto_generate_content`, `auto_generate_images` actions
**Tasks**:
- `auto_generate_content_task`: AI-powered content generation
- `auto_generate_images_task`: AI-powered image generation
### System Module
**Purpose**: System settings, prompts, and integrations.
**Models**:
- `AIPrompt`: AI prompt templates (clustering, ideas, content, images)
- `IntegrationSettings`: API keys and configuration (OpenAI, Runware, etc.)
- `AuthorProfile`: Writing style profiles
- `Strategy`: Content strategies per sector
**ViewSets**:
- `AIPromptViewSet`: CRUD for prompts
- `IntegrationSettingsViewSet`: CRUD + `test_openai`, `test_runware`, `generate_image`, `task_progress` actions
- `AuthorProfileViewSet`: CRUD for author profiles
- `StrategyViewSet`: CRUD for strategies
### Billing Module
**Purpose**: Credits, transactions, and usage tracking.
**Models**:
- `CreditTransaction`: Credit purchase/usage transactions
- `UsageLog`: Daily/monthly usage tracking
**ViewSets**:
- `CreditTransactionViewSet`: CRUD for transactions
- `UsageLogViewSet`: Read-only usage logs
**Services**:
- `CreditService`: Credit calculation and deduction logic
---
## API Architecture
### RESTful API Design
**Base URL**: `/api/v1/`
**Endpoint Structure**:
- `/api/v1/planner/keywords/` - Keywords CRUD
- `/api/v1/planner/keywords/auto_cluster/` - Auto-cluster action
- `/api/v1/planner/clusters/` - Clusters CRUD
- `/api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas action
- `/api/v1/writer/tasks/` - Tasks CRUD
- `/api/v1/writer/tasks/auto_generate_content/` - Auto-generate content action
- `/api/v1/system/settings/task_progress/{task_id}/` - Task progress polling
### Authentication
**Methods**:
- JWT (JSON Web Tokens) - Primary method
- Session-based auth - Fallback for admin
**Flow**:
1. User signs in → Backend returns JWT token
2. Frontend stores token in localStorage
3. Frontend includes token in `Authorization: Bearer {token}` header
4. Backend middleware validates token and sets `request.user` and `request.account`
### Response Format
**Success Response**:
```json
{
"success": true,
"data": { ... },
"message": "Optional message"
}
```
**Error Response**:
```json
{
"success": false,
"message": "Error message",
"errors": { ... }
}
```
### Pagination
**Format**: Page-based pagination
**Response**:
```json
{
"count": 100,
"next": "http://api.example.com/api/v1/resource/?page=2",
"previous": null,
"results": [ ... ]
}
```
---
## Frontend Architecture
### Component Hierarchy
```
App
└── AppLayout
├── Sidebar (navigation)
├── Header (user menu, notifications)
└── Main Content
└── Page Component
└── Template (DashboardTemplate, TablePageTemplate, etc.)
└── Components (DataTable, Filters, etc.)
```
### State Management
**Zustand Stores**:
- `authStore`: Authentication state (user, token, account)
- `plannerStore`: Planner module state
- `siteStore`: Selected site/sector
- `aiRequestLogsStore`: AI request/response logs
- `pageSizeStore`: Table page size preference
**Local State**: React `useState` for component-specific state
### Routing
**Structure**: React Router v6 with nested routes
**Routes**:
- `/` - Home/Dashboard
- `/planner` - Planner Dashboard
- `/planner/keywords` - Keywords page
- `/planner/clusters` - Clusters page
- `/planner/ideas` - Ideas page
- `/writer` - Writer Dashboard
- `/writer/tasks` - Tasks page
- `/settings` - Settings pages
**Protected Routes**: All routes except `/signin` and `/signup` require authentication
---
## Backend Architecture
### Model Inheritance Hierarchy
```
models.Model
└── AccountBaseModel (adds account ForeignKey)
└── SiteSectorBaseModel (adds site, sector ForeignKeys)
└── Keywords, Clusters, ContentIdeas, Tasks, etc.
```
### ViewSet Inheritance Hierarchy
```
viewsets.ModelViewSet
└── AccountModelViewSet (adds account filtering)
└── SiteSectorModelViewSet (adds site/sector filtering)
└── KeywordViewSet, ClusterViewSet, TasksViewSet, etc.
```
### Middleware Stack
1. **SecurityMiddleware**: Django security middleware
2. **SessionMiddleware**: Session management
3. **AuthenticationMiddleware**: User authentication
4. **AccountContextMiddleware**: Sets `request.account` from JWT
5. **ResourceTrackerMiddleware**: Tracks API request metrics
### Celery Task Architecture
**Broker**: Redis
**Workers**: Separate Celery worker processes
**Task Structure**:
```python
@shared_task(bind=True)
def my_task(self, ...):
# Update progress
self.update_state(state='PROGRESS', meta={...})
# Do work
result = do_work()
# Return result
return result
```
**Progress Tracking**:
- Frontend polls `/api/v1/system/settings/task_progress/{task_id}/`
- Backend returns task state and meta information
- Progress displayed in modal
---
## Database Architecture
### Core Tables
- `igny8_accounts`: Account information
- `igny8_users`: User accounts
- `igny8_plans`: Subscription plans
- `igny8_subscriptions`: Active subscriptions
- `igny8_sites`: Sites within accounts
- `igny8_sectors`: Sectors within sites
- `igny8_industries`: Global industry templates
- `igny8_industry_sectors`: Industry sector templates
### Planner Tables
- `igny8_keywords`: Keywords
- `igny8_clusters`: Keyword clusters
- `igny8_content_ideas`: Content ideas
### Writer Tables
- `igny8_tasks`: Content generation tasks
- `igny8_content`: Generated content
- `igny8_images`: Generated images
### System Tables
- `igny8_ai_prompts`: AI prompt templates
- `igny8_integration_settings`: API keys and configuration
- `igny8_author_profiles`: Writing style profiles
- `igny8_strategies`: Content strategies
### Billing Tables
- `igny8_credit_transactions`: Credit transactions
- `igny8_usage_logs`: Usage tracking
### Indexes
**Account Isolation**: All tables have indexes on `account`
**Site/Sector Filtering**: Tables with site/sector have composite indexes on `(account, site, sector)`
**Performance**: Indexes on frequently queried fields (status, created_at, etc.)
---
## Security Architecture
### Authentication
**JWT Tokens**:
- Signed with secret key
- Contains user ID and account ID
- Expires after configured time
- Stored in localStorage (frontend)
**Session Auth**:
- Fallback for admin interface
- Django session framework
### Authorization
**Role-Based Access Control (RBAC)**:
- `developer`: Full system access
- `owner`: Full account access
- `admin`: Account admin access
- `editor`: Content editing access
- `viewer`: Read-only access
**Access Control**:
- Account-level: Automatic filtering by `request.account`
- Site-level: Filtering by `user.get_accessible_sites()`
- Action-level: Permission checks in ViewSet actions
### Data Isolation
**Account Isolation**:
- All queries filtered by account
- Admin/Developer override for system accounts
**Site Access Control**:
- Users can only access granted sites
- Admin/Developer override for all sites
### API Security
**CORS**: Configured for frontend domain
**CSRF**: Enabled for session-based auth
**Rate Limiting**: (Future implementation)
**Input Validation**: DRF serializers validate all input
---
## Deployment Architecture
### Docker Compose Setup
**Services**:
- `backend`: Django application (port 8010/8011)
- `frontend`: React application (port 5173/8021)
- `db`: PostgreSQL database
- `redis`: Redis for Celery broker and caching
- `caddy`: Reverse proxy (HTTPS on port 443)
- `celery-worker`: Celery worker process
- `celery-beat`: Celery beat scheduler (optional)
### Environment Configuration
**Backend**:
- `DJANGO_SETTINGS_MODULE`: Django settings module
- `DATABASE_URL`: PostgreSQL connection string
- `REDIS_URL`: Redis connection string
- `SECRET_KEY`: Django secret key
- `OPENAI_API_KEY`: OpenAI API key (fallback)
**Frontend**:
- `VITE_API_URL`: Backend API URL
- `VITE_APP_NAME`: Application name
### Scaling Considerations
**Horizontal Scaling**:
- Multiple Celery workers
- Multiple backend instances (load balanced)
- Multiple frontend instances (static files)
**Vertical Scaling**:
- Database connection pooling
- Redis connection pooling
- Celery worker concurrency
### Monitoring
**Application Monitoring**:
- ResourceTrackerMiddleware tracks API request metrics
- Celery task monitoring via Flower (optional)
**Infrastructure Monitoring**:
- Portainer for container monitoring
- Database monitoring via PostgreSQL logs
- Redis monitoring via Redis CLI
---
## Summary
The IGNY8 architecture is built on:
1. **Configuration-Driven Design**: Zero duplication, single source of truth
2. **Multi-Tenancy Foundation**: Complete account isolation with site/sector hierarchy
3. **Template System**: 4 universal templates for all page types
4. **Unified AI Interface**: Single AIProcessor for all AI operations
5. **Module-Based Organization**: Clear boundaries with shared utilities
6. **RESTful API**: Consistent API design with DRF
7. **Modern Frontend**: React + TypeScript with Zustand state management
8. **Scalable Backend**: Django + Celery for async processing
9. **Security First**: JWT auth, RBAC, data isolation
10. **Docker Deployment**: Containerized for easy deployment and scaling
This architecture ensures scalability, maintainability, and extensibility while maintaining a clean separation of concerns across modules.