Organize planning documents and update README structure

- Created a new `docs/planning/` directory to better organize architecture and implementation planning documents.
- Moved existing planning documents into the new directory for improved accessibility.
- Updated `README.md` to reflect the new document structure and added references to the organized planning documents.
- Enhanced overall documentation management for easier navigation and maintenance.
This commit is contained in:
IGNY8 VPS (Salman)
2025-11-16 17:41:30 +00:00
parent 4561f73afb
commit 76a363b3d5
14 changed files with 2922 additions and 664 deletions

View File

@@ -146,7 +146,21 @@ docs/
├── 03-FRONTEND-ARCHITECTURE.md
├── 04-BACKEND-IMPLEMENTATION.md
├── 05-AI-FRAMEWORK-IMPLEMENTATION.md
── 06-FUNCTIONAL-BUSINESS-LOGIC.md
── 06-FUNCTIONAL-BUSINESS-LOGIC.md
├── API-COMPLETE-REFERENCE.md
├── WORDPRESS-PLUGIN-INTEGRATION.md
├── planning/ # Architecture & implementation planning
│ ├── IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md
│ ├── IGNY8-IMPLEMENTATION-PLAN.md
│ ├── Igny8-phase-2-plan.md
│ ├── CONTENT-WORKFLOW-DIAGRAM.md
│ ├── ARCHITECTURE_CONTEXT.md
│ └── sample-usage-limits-credit-system
└── refactor/ # Refactoring plans and documentation
├── README.md
├── routes/
├── folder-structure/
└── migrations/
```
### Documentation Update Checklist

View File

@@ -0,0 +1,924 @@
# IGNY8 Complete Architecture Context
**Created:** 2025-01-XX
**Purpose:** Comprehensive context document for understanding the complete IGNY8 system architecture, workflows, and implementation details.
---
## Executive Summary
IGNY8 is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system operates on a multi-tenant architecture with complete account isolation, hierarchical organization (Account > Site > Sector > Content), and unified AI processing framework.
**Key Characteristics:**
- Multi-tenant SaaS with account isolation
- Django 5.2+ backend with DRF API
- React 19 frontend with TypeScript
- PostgreSQL 15 database
- Celery + Redis for async tasks
- Docker-based containerization
- Caddy reverse proxy for HTTPS
---
## System Architecture Overview
### High-Level Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Client Layer (Browser) │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Main App │ │ Marketing │ │ Admin │ │
│ │ (app.igny8) │ │ (igny8.com) │ │ Panel │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
└─────────┼──────────────────┼──────────────────┼─────────────┘
│ │ │
└──────────────────┼──────────────────┘
┌────────────────────────────┼──────────────────────────────┐
│ Reverse Proxy Layer │
│ ┌───────────────┐ │
│ │ Caddy │ │
│ │ (HTTPS/443) │ │
│ └───────┬───────┘ │
└────────────────────────────┼──────────────────────────────┘
┌────────────────────────────┼──────────────────────────────┐
│ Application Layer │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Frontend │ │ Backend │ │
│ │ (React) │◄─────────────┤ (Django) │ │
│ │ Port 8021 │ REST API │ Port 8011 │ │
│ └──────────────┘ └──────┬───────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ Celery Worker │ │
│ │ (Async Tasks) │ │
│ └────────┬────────┘ │
└───────────────────────────────────────┼──────────────────┘
┌───────────────────────────────────────┼──────────────────┐
│ Data Layer │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ PostgreSQL │ │ Redis │ │ Storage │ │
│ │ (Database) │ │ (Cache/Broker)│ │ (Files) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└──────────────────────────────────────────────────────────┘
┌───────────────────────────────────────┼──────────────────┐
│ External Services │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ OpenAI │ │ Runware │ │ WordPress │ │
│ │ (GPT/DALL-E)│ │ (Images) │ │ (Publish) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└──────────────────────────────────────────────────────────┘
```
### Current Infrastructure Status
**Running Containers:**
- `igny8_backend` - Django API (Port 8011, healthy)
- `igny8_frontend` - React app (Port 8021)
- `igny8_marketing_dev` - Marketing site (Port 8023)
- `igny8_celery_worker` - Async task processor
- `igny8_celery_beat` - Scheduled tasks
- `igny8_postgres` - Database (healthy)
- `igny8_redis` - Cache/Broker (healthy)
- `igny8_caddy` - Reverse proxy (Ports 80, 443)
- `igny8_pgadmin` - DB admin (Port 5050)
- `igny8_filebrowser` - File manager (Port 8080)
- `portainer` - Container management (Ports 8000, 9443)
**Network:** `igny8_net` (bridge network, external)
---
## Technology Stack
### Backend Stack
- **Framework:** Django 5.2.7+
- **API:** Django REST Framework
- **Database:** PostgreSQL 15
- **Task Queue:** Celery 5.3.0+ with Redis 7
- **Auth:** JWT (PyJWT 2.8.0+)
- **Server:** Gunicorn
- **Static Files:** WhiteNoise
### Frontend Stack
- **Framework:** React 19.0.0
- **Language:** TypeScript 5.7.2
- **Build Tool:** Vite 6.1.0
- **Styling:** Tailwind CSS 4.0.8
- **State:** Zustand 5.0.8
- **Routing:** React Router v7.9.5
- **Icons:** @heroicons/react 2.2.0
### Infrastructure
- **Containerization:** Docker + Docker Compose
- **Reverse Proxy:** Caddy (HTTPS termination)
- **Container Management:** Portainer
---
## Core Architecture Principles
### 1. Multi-Tenancy Foundation
- **Account Isolation:** All models inherit `AccountBaseModel` with `account` ForeignKey
- **Automatic Filtering:** All ViewSets inherit `AccountModelViewSet` with automatic filtering
- **Middleware:** `AccountContextMiddleware` sets `request.account` from JWT token
- **Hierarchy:** Account > Site > Sector > Content
### 2. Configuration-Driven Everything
- **Frontend:** Config files in `/config/pages/` and `/config/snippets/`
- **Backend:** DRF serializers and ViewSet actions
- **Templates:** 4 universal templates (Dashboard, Table, Form, System)
### 3. Unified AI Framework
- **Single Interface:** All AI operations use `AIEngine` orchestrator
- **Base Class:** All AI functions inherit from `BaseAIFunction`
- **Execution Pipeline:** 6 phases (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
- **Progress Tracking:** Real-time updates via Celery
### 4. Module-Based Organization
- **Planner:** Keywords, Clusters, Ideas
- **Writer:** Tasks, Content, Images
- **Thinker:** Prompts, Author Profiles, Strategies
- **System:** Settings, Integrations, AI Configuration
- **Billing:** Credits, Transactions, Usage
- **Auth:** Accounts, Users, Sites, Sectors
---
## System Hierarchy
### Entity Relationships
```
Account (1) ──< (N) User
Account (1) ──< (1) Subscription ──> (1) Plan
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
Cluster (1) ──< (N) Keywords (Many-to-Many)
Cluster (1) ──< (N) ContentIdeas
ContentIdeas (1) ──< (N) Tasks
Task (1) ──> (1) Content
Task (1) ──< (N) Images
```
### Hierarchy Details
**Account Level:**
- Top-level organization/workspace
- Contains users, sites, subscriptions, and all data
- Has credit balance and plan assignment
- Status: active, suspended, trial, cancelled
**User Level:**
- Individual user accounts within an account
- Has role (developer, owner, admin, editor, viewer)
- Can belong to only one account
- Access controlled by role and site permissions
**Site Level:**
- Workspace within an account (1-N relationship)
- Can have multiple active sites simultaneously
- Has WordPress integration settings (URL, username, password)
- Can be associated with an industry
- Status: active, inactive, suspended
**Sector Level:**
- Content category within a site (1-5 per site)
- Organizes keywords, clusters, ideas, and tasks
- Can reference an industry sector template
- Status: active, inactive
**Content Level:**
- Keywords, Clusters, ContentIdeas belong to Sector
- Tasks, Content, Images belong to Sector
- All content is automatically associated with Account and Site
---
## User Roles & Access Control
### Role Hierarchy
```
developer > owner > admin > editor > viewer > system_bot
```
### Role Permissions
| Role | Account Access | Site Access | Data Access | User Management | Billing |
|------|----------------|-------------|-------------|-----------------|---------|
| Developer | All accounts | All sites | All data | Yes | Yes |
| System Bot | All accounts | All sites | All data | No | No |
| Owner | Own account | All sites in account | All data in account | Yes | Yes |
| Admin | Own account | All sites in account | All data in account | Yes | No |
| Editor | Own account | Granted sites only | Data in granted sites | No | No |
| Viewer | Own account | Granted sites only | Read-only in granted sites | No | No |
### Access Control Implementation
**Automatic Access:**
- Owners and Admins: Automatic access to all sites in their account
- Developers and System Bot: Access to all sites across all accounts
**Explicit Access:**
- Editors and Viewers: Require explicit `SiteUserAccess` records
- Access granted by Owner or Admin
- Access can be revoked at any time
---
## Complete Workflows
### 1. Account Setup Workflow
**Steps:**
1. User signs up via `/signup`
2. Account created with default plan
3. Owner user created and linked to account
4. User signs in via `/signin`
5. JWT token generated and returned
6. Frontend stores token and redirects to dashboard
7. User creates first site (optional)
8. User creates sectors (1-5 per site, optional)
9. User configures integration settings (OpenAI, Runware)
10. System ready for use
**Data Created:**
- 1 Account record
- 1 User record (owner role)
- 1 Subscription record (default plan)
- 0-N Site records
- 0-N Sector records (per site)
- 1 IntegrationSettings record (per integration type)
### 2. Keyword Management Workflow
**Steps:**
1. User navigates to `/planner/keywords`
2. User imports keywords via CSV or manual entry
3. Keywords validated and stored in database
4. Keywords displayed in table with filters
5. User filters keywords by sector, status, intent, etc.
6. User selects keywords for clustering
7. User clicks "Auto Cluster" action
8. Backend validates keyword IDs
9. Celery task queued (`run_ai_task` with function `auto_cluster`)
10. Task ID returned to frontend
11. Frontend polls progress endpoint
12. Celery worker processes task:
- Loads keywords from database
- Builds AI prompt with keyword data
- Calls OpenAI API for clustering
- Parses cluster response
- Creates Cluster records
- Links keywords to clusters
13. Progress updates sent to frontend
14. Task completes
15. Frontend displays new clusters
16. Credits deducted from account
**AI Function:** Auto Cluster Keywords
### 3. Content Generation Workflow
**Steps:**
1. User navigates to `/planner/ideas`
2. User selects content ideas
3. User clicks "Create Tasks" action
4. Task records created for each idea
5. User navigates to `/writer/tasks`
6. User selects tasks for content generation
7. User clicks "Generate Content" action
8. Backend validates task IDs
9. Celery task queued (`run_ai_task` with function `generate_content`)
10. Task ID returned to frontend
11. Frontend polls progress endpoint
12. Celery worker processes task:
- Loads tasks and related data (cluster, keywords, idea)
- Builds AI prompt with task data
- Calls OpenAI API for content generation
- Parses HTML content response
- Creates/updates Content records
- Updates task status
13. Progress updates sent to frontend
14. Task completes
15. Frontend displays generated content
16. Credits deducted from account
**AI Function:** Generate Content
### 4. WordPress Publishing Workflow
**Steps:**
1. User navigates to `/writer/content`
2. User selects content to publish
3. User clicks "Publish to WordPress" action
4. Backend validates:
- Site has WordPress URL configured
- Site has WordPress credentials
- Content is ready (status: review or draft)
5. Backend calls WordPress REST API:
- Creates post with content HTML
- Uploads featured image (if available)
- Uploads in-article images (if available)
- Sets post status (draft, publish)
6. WordPress post ID stored in Content record
7. Content status updated to "published"
8. Frontend displays success message
**Integration:** WordPress REST API
---
## AI Framework Architecture
### Unified Execution Pipeline
**Entry Point:** `run_ai_task` (Celery task)
- Location: `backend/igny8_core/ai/tasks.py`
- Parameters: `function_name`, `payload`, `account_id`
- Flow: Loads function from registry → Creates AIEngine → Executes function
**Engine Orchestrator:** `AIEngine`
- Location: `backend/igny8_core/ai/engine.py`
- Purpose: Central orchestrator managing lifecycle, progress, logging, cost tracking
- Methods:
- `execute` - Main execution pipeline (6 phases)
- `_handle_error` - Centralized error handling
- `_log_to_database` - Logs to AITaskLog model
**Base Function Class:** `BaseAIFunction`
- Location: `backend/igny8_core/ai/base.py`
- Purpose: Abstract base class defining interface for all AI functions
- Abstract Methods:
- `get_name()` - Returns function name
- `prepare()` - Loads and prepares data
- `build_prompt()` - Builds AI prompt
- `parse_response()` - Parses AI response
- `save_output()` - Saves results to database
### AI Function Execution Flow
```
1. API Endpoint (views.py)
2. run_ai_task (tasks.py)
- Gets account from account_id
- Gets function instance from registry
- Creates AIEngine
3. AIEngine.execute (engine.py)
Phase 1: INIT (0-10%)
- Calls function.validate()
- Updates progress tracker
Phase 2: PREP (10-25%)
- Calls function.prepare()
- Calls function.build_prompt()
- Updates progress tracker
Phase 3: AI_CALL (25-70%)
- Gets model config from settings
- Calls AICore.run_ai_request() or AICore.generate_image()
- Tracks cost and tokens
- Updates progress tracker
Phase 4: PARSE (70-85%)
- Calls function.parse_response()
- Updates progress tracker
Phase 5: SAVE (85-98%)
- Calls function.save_output()
- Logs credit usage
- Updates progress tracker
Phase 6: DONE (98-100%)
- Logs to AITaskLog
- Returns result
```
### AI Functions
1. **Auto Cluster Keywords** (`auto_cluster`)
- Purpose: Group related keywords into semantic clusters
- Input: Keyword IDs (max 20)
- Output: Cluster records created, keywords linked
- Credits: 1 credit per 30 keywords
2. **Generate Ideas** (`generate_ideas`)
- Purpose: Generate content ideas from keyword clusters
- Input: Cluster IDs (max 1 per batch)
- Output: ContentIdeas records created
- Credits: 1 credit per idea
3. **Generate Content** (`generate_content`)
- Purpose: Generate blog post and article content
- Input: Task IDs (max 50 per batch)
- Output: Content records created/updated with HTML
- Credits: 3 credits per content piece
4. **Generate Image Prompts** (`generate_image_prompts`)
- Purpose: Extract image prompts from content HTML
- Input: Content IDs
- Output: Images records updated with prompts
- Credits: Included in content generation
5. **Generate Images** (`generate_images`)
- Purpose: Generate images using OpenAI DALL-E or Runware
- Input: Image IDs (with prompts)
- Output: Images records updated with image URLs
- Credits: 1 credit per image
---
## Frontend Architecture
### Application Structure
**Dual Application Architecture:**
1. **Main Application** (`app.igny8.com`): Authenticated SaaS platform
2. **Marketing Site** (`igny8.com`): Public-facing marketing website
**Entry Points:**
- Main App: `src/main.tsx``src/App.tsx`
- Marketing: `src/marketing/index.tsx``src/marketing/MarketingApp.tsx`
### State Management
**Zustand Stores:**
- `authStore` - Authentication & user
- `siteStore` - Active site management
- `sectorStore` - Active sector management
- `plannerStore` - Planner module state
- `billingStore` - Billing & credits
- `settingsStore` - Application settings
- `pageSizeStore` - Table pagination
- `columnVisibilityStore` - Table column visibility
**React Contexts:**
- `ThemeContext` - Light/dark theme
- `SidebarContext` - Sidebar state
- `HeaderMetricsContext` - Header metrics
- `ToastProvider` - Toast notifications
### Template System
**4 Universal Templates:**
1. **DashboardTemplate** - Module home pages (KPIs, workflow steps, charts)
2. **TablePageTemplate** - CRUD table pages (filtering, sorting, pagination)
3. **FormPageTemplate** - Settings/form pages (sectioned forms)
4. **SystemPageTemplate** - System/admin pages (status cards, logs)
### API Integration
**API Service Layer:**
- Location: `frontend/src/services/api.ts`
- Function: `fetchAPI()` - Centralized API client
- Features:
- Automatic token injection
- Token refresh on 401
- Site/sector context injection
- Unified error handling
- Timeout handling
**Request Flow:**
1. User action in frontend
2. Frontend makes API request via `fetchAPI()`
3. JWT token included in Authorization header
4. Backend middleware extracts account from JWT
5. Backend ViewSet processes request
6. Backend returns JSON response (unified format)
7. Frontend updates state
8. Frontend updates UI
---
## Backend Architecture
### Multi-Tenancy Implementation
**Account Isolation:**
- **Model Level:** All models inherit `AccountBaseModel` with `account` ForeignKey
- **ViewSet Level:** All ViewSets inherit `AccountModelViewSet` with automatic filtering
- **Middleware Level:** `AccountContextMiddleware` sets `request.account` from JWT
**Middleware Flow:**
```
Request with JWT Token
AccountContextMiddleware
├── Extract Account ID from JWT
├── Load Account Object
└── Set request.account
ViewSet.get_queryset()
├── Check User Role
├── Filter by Account (if not admin/developer)
└── Filter by Accessible Sites (if not owner/admin)
Database Query
Results (Account-Isolated)
```
### Base Classes
**AccountModelViewSet:**
- Location: `backend/igny8_core/api/base.py`
- Purpose: Base ViewSet with automatic account filtering
- Features:
- Automatic account filtering
- Admin/Developer override
- Account context in serializers
**SiteSectorModelViewSet:**
- Location: `backend/igny8_core/api/base.py`
- Purpose: Base ViewSet with site/sector filtering
- Features:
- Account filtering (inherited)
- Site access control
- Sector validation
- Accessible sites/sectors in serializer context
### API Response Format
**Unified Format:**
```json
{
"success": true,
"data": {...},
"message": "Optional message",
"request_id": "uuid"
}
```
**Error Format:**
```json
{
"success": false,
"error": "Error message",
"errors": {
"field_name": ["Field-specific errors"]
},
"request_id": "uuid"
}
```
**Paginated Format:**
```json
{
"success": true,
"count": 120,
"next": "url",
"previous": "url",
"results": [...],
"request_id": "uuid"
}
```
---
## Module Organization
### Planner Module
- **Purpose:** Keyword management & content planning
- **Models:** Keywords, Clusters, ContentIdeas
- **ViewSets:** KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
- **Celery Tasks:** `auto_cluster_keywords_task`, `auto_generate_ideas_task`
- **Features:**
- Keyword import (CSV/manual)
- Keyword filtering and organization
- AI-powered keyword clustering
- Content idea generation from clusters
- Keyword-to-cluster mapping
### Writer Module
- **Purpose:** Content generation & management
- **Models:** Tasks, Content, Images
- **ViewSets:** TasksViewSet, ImagesViewSet
- **Celery Tasks:** `auto_generate_content_task`, `auto_generate_images_task`
- **Features:**
- Task creation from content ideas
- AI-powered content generation
- Content editing and review
- Image prompt extraction
- AI-powered image generation
- WordPress publishing
### Thinker Module
- **Purpose:** AI configuration and strategy
- **Models:** AIPrompt, AuthorProfile, Strategy
- **ViewSets:** AIPromptViewSet, AuthorProfileViewSet
- **Features:**
- AI prompt management
- Author profile management
- Content strategy management
- Image testing
### System Module
- **Purpose:** System configuration and AI settings
- **Models:** IntegrationSettings, AIPrompt, AuthorProfile, Strategy
- **ViewSets:** IntegrationSettingsViewSet, AIPromptViewSet, AuthorProfileViewSet
- **Features:**
- Integration settings (OpenAI, Runware)
- AI prompt management
- System status and monitoring
### Billing Module
- **Purpose:** Credit management and usage tracking
- **Models:** CreditTransaction, CreditUsageLog
- **ViewSets:** CreditTransactionViewSet, CreditUsageLogViewSet
- **Services:** CreditService
- **Features:**
- Credit balance tracking
- Credit transactions
- Usage logging
- Cost tracking
### Auth Module
- **Purpose:** Multi-tenancy and user management
- **Models:** Account, User, Plan, Site, Sector, Industry
- **ViewSets:** AccountViewSet, UserViewSet, SiteViewSet, SectorViewSet
- **Features:**
- Account management
- User management
- Plan management
- Site and sector management
- Industry templates
---
## Credit System
### Credit Balance Management
**Account Credits:**
- Each account has a `credits` field (integer)
- Credits start at 0 or plan-included credits
- Credits are deducted for AI operations
- Credits can be added via transactions
**Credit Checking:**
- Before AI operation: System checks if account has sufficient credits
- If insufficient: Operation fails with `InsufficientCreditsError`
- If sufficient: Operation proceeds
**Credit Deduction:**
- After AI operation completes: Credits deducted via `CreditService.deduct_credits()`
- Account credits field updated
- CreditTransaction record created (type: deduction, amount: negative)
- CreditUsageLog record created with operation details
### Credit Costs per Operation
- **Clustering:** 1 credit per 30 keywords (base: 1 credit)
- **Ideas:** 1 credit per idea (base: 1 credit)
- **Content:** 3 credits per content piece (base: 3 credits)
- **Images:** 1 credit per image (base: 1 credit)
- **Reparse:** 1 credit per reparse (base: 1 credit)
---
## WordPress Integration
### Publishing Process
**Workflow:**
1. User selects content to publish
2. System validates WordPress configuration
3. System authenticates with WordPress REST API
4. System creates WordPress post:
- Title: Content meta_title or task title
- Content: Content HTML
- Status: Draft or Publish (based on content status)
- Featured image: Uploaded if available
- In-article images: Uploaded if available
- Meta fields: Primary keyword, secondary keywords
5. WordPress returns post ID
6. System updates Content record:
- Sets `wp_post_id` field
- Sets `status` to "published"
**Requirements:**
- Site must have WordPress URL configured (`wp_url`)
- Site must have WordPress username and app password
- Content must have status "review" or "draft"
- WordPress REST API must be accessible
---
## Docker Architecture
### Infrastructure Stack (`igny8-infra`)
- **PostgreSQL** - Database (Port 5432 internal)
- **Redis** - Cache & Celery broker (Port 6379 internal)
- **pgAdmin** - Database admin (Port 5050)
- **FileBrowser** - File management (Port 8080)
- **Caddy** - Reverse proxy (Ports 80, 443)
- **Setup Helper** - Utility container
### Application Stack (`igny8-app`)
- **Backend** - Django API (Port 8011:8010)
- **Frontend** - React app (Port 8021:5173)
- **Marketing Dev** - Marketing site (Port 8023:5174)
- **Celery Worker** - Async task processing
- **Celery Beat** - Scheduled tasks
### Network Configuration
- **Network Name:** `igny8_net`
- **Type:** External bridge network
- **Purpose:** Inter-container communication
---
## Key Files and Locations
### Backend Key Files
- `backend/igny8_core/auth/middleware.py` - AccountContextMiddleware
- `backend/igny8_core/api/base.py` - AccountModelViewSet, SiteSectorModelViewSet
- `backend/igny8_core/ai/engine.py` - AIEngine orchestrator
- `backend/igny8_core/ai/base.py` - BaseAIFunction
- `backend/igny8_core/ai/tasks.py` - run_ai_task entrypoint
- `backend/igny8_core/api/response.py` - Unified response helpers
### Frontend Key Files
- `frontend/src/services/api.ts` - API client
- `frontend/src/store/authStore.ts` - Authentication state
- `frontend/src/store/siteStore.ts` - Site management
- `frontend/src/templates/` - 4 universal templates
- `frontend/src/config/pages/` - Page configurations
### Documentation
- `docs/01-TECH-STACK-AND-INFRASTRUCTURE.md` - Tech stack
- `docs/02-APPLICATION-ARCHITECTURE.md` - Application architecture
- `docs/03-FRONTEND-ARCHITECTURE.md` - Frontend architecture
- `docs/04-BACKEND-IMPLEMENTATION.md` - Backend implementation
- `docs/05-AI-FRAMEWORK-IMPLEMENTATION.md` - AI framework
- `docs/06-FUNCTIONAL-BUSINESS-LOGIC.md` - Business logic
- `docs/API-COMPLETE-REFERENCE.md` - Complete API reference
---
## Data Flow Examples
### Request Flow
```
1. User Action (e.g., "Auto Cluster Keywords")
2. Frontend API Call (fetchAPI)
3. Backend Endpoint (ViewSet Action)
4. Celery Task Queued
5. Task ID Returned to Frontend
6. Frontend Polls Progress Endpoint
7. Celery Worker Processes Task
8. AIProcessor Makes API Calls
9. Results Saved to Database
10. Progress Updates Sent
11. Frontend Displays Results
```
### Authentication Flow
```
1. User Signs In
2. Backend Validates Credentials
3. JWT Token Generated
4. Token Returned to Frontend
5. Frontend Stores Token (localStorage)
6. Frontend Includes Token in Requests (Authorization: Bearer {token})
7. Backend Validates Token
8. Account Context Set (AccountContextMiddleware)
9. Request Processed
```
---
## Security Architecture
### Authentication
- **Primary:** JWT Bearer tokens
- **Fallback:** Session-based auth (admin panel)
- **Token Storage:** localStorage (frontend)
- **Token Expiry:** 15 minutes (access), 7 days (refresh)
### Authorization
- **Role-Based Access Control (RBAC):** Role checked on every request
- **Data Access Control:**
- Account-level: Automatic filtering by account
- Site-level: Filtering by accessible sites
- Action-level: Permission checks in ViewSet actions
### Account Isolation
- All queries filtered by account
- Admin/Developer override for system accounts
- No cross-account data leakage
---
## External Service Integrations
### OpenAI Integration
- **Purpose:** Text generation and image generation
- **Configuration:** API key stored per account in `IntegrationSettings`
- **Services Used:**
- GPT models for text generation
- DALL-E for image generation
- **Cost Tracking:** Tracked per request
### Runware Integration
- **Purpose:** Alternative image generation service
- **Configuration:** API key stored per account
- **Model Selection:** e.g., `runware:97@1`
- **Image Type:** realistic, artistic, cartoon
### WordPress Integration
- **Purpose:** Content publishing
- **Configuration:** WordPress URL per site, username and password stored per site
- **Workflow:**
1. Content generated in IGNY8
2. Images attached
3. Content published to WordPress via REST API
4. Status updated in IGNY8
---
## Development Workflow
### Local Development
1. **Backend:**
```bash
cd backend
pip install -r requirements.txt
python manage.py migrate
python manage.py runserver
```
2. **Frontend:**
```bash
cd frontend
npm install
npm run dev
```
### Docker Development
1. **Build Images:**
```bash
docker build -t igny8-backend -f backend/Dockerfile ./backend
docker build -t igny8-frontend-dev -f frontend/Dockerfile.dev ./frontend
```
2. **Start Services:**
```bash
docker compose -f docker-compose.app.yml -p igny8-app up -d
```
---
## Summary
This context document provides a comprehensive overview of the IGNY8 system architecture, including:
1. **System Architecture:** High-level architecture, infrastructure status, technology stack
2. **Core Principles:** Multi-tenancy, configuration-driven, unified AI framework, module-based
3. **System Hierarchy:** Entity relationships, account/site/sector structure
4. **User Roles:** Role hierarchy, permissions, access control
5. **Workflows:** Complete workflows for account setup, keyword management, content generation, WordPress publishing
6. **AI Framework:** Unified execution pipeline, AI functions, progress tracking
7. **Frontend Architecture:** Dual application structure, state management, templates, API integration
8. **Backend Architecture:** Multi-tenancy implementation, base classes, API response format
9. **Module Organization:** Planner, Writer, Thinker, System, Billing, Auth modules
10. **Credit System:** Credit balance management, costs per operation
11. **WordPress Integration:** Publishing process, requirements
12. **Docker Architecture:** Infrastructure and application stacks
13. **Key Files:** Important file locations
14. **Data Flow:** Request and authentication flows
15. **Security:** Authentication, authorization, account isolation
16. **External Services:** OpenAI, Runware, WordPress integrations
17. **Development:** Local and Docker development workflows
This document serves as a comprehensive reference for understanding the complete IGNY8 system architecture and implementation details.
---
**Last Updated:** 2025-01-XX
**Version:** 1.0.0

View File

@@ -0,0 +1,309 @@
# CONTENT WORKFLOW & ENTRY POINTS
**Complete Workflow Diagrams for Writer → Linker → Optimizer**
---
## WORKFLOW 1: WRITER → LINKER → OPTIMIZER → PUBLISH
```
┌─────────────┐
│ Writer │
│ Generates │
│ Content │
└──────┬──────┘
┌─────────────────────┐
│ Content Saved │
│ source='igny8' │
│ sync_status='native'│
│ status='draft' │
└──────┬──────────────┘
┌─────────────────────┐
│ Linker Trigger │
│ (Auto or Manual) │
└──────┬──────────────┘
┌─────────────────────┐
│ LinkerService │
│ - Finds candidates │
│ - Injects links │
│ - Updates content │
└──────┬──────────────┘
┌─────────────────────┐
│ Content Updated │
│ linker_version++ │
│ internal_links[] │
│ status='linked' │
└──────┬──────────────┘
┌─────────────────────┐
│ Optimizer Trigger │
│ (Auto or Manual) │
└──────┬──────────────┘
┌─────────────────────┐
│ OptimizerService │
│ - Analyzes content │
│ - Optimizes │
│ - Stores results │
└──────┬──────────────┘
┌─────────────────────┐
│ Content Updated │
│ optimizer_version++ │
│ status='optimized' │
└──────┬──────────────┘
┌─────────────────────┐
│ PublisherService │
│ - WordPress │
│ - Sites Renderer │
│ - Shopify │
└─────────────────────┘
```
---
## WORKFLOW 2: WORDPRESS SYNC → OPTIMIZER → PUBLISH
```
┌─────────────────┐
│ WordPress │
│ Plugin Syncs │
│ Posts to IGNY8 │
└────────┬────────┘
┌─────────────────────────┐
│ ContentSyncService │
│ sync_from_wordpress() │
└────────┬────────────────┘
┌─────────────────────────┐
│ Content Created │
│ source='wordpress' │
│ sync_status='synced' │
│ external_id=wp_post_id │
│ external_url=wp_url │
└────────┬────────────────┘
┌─────────────────────────┐
│ Content Visible │
│ in Writer/Content List │
│ (Filterable by source) │
└────────┬────────────────┘
┌─────────────────────────┐
│ User Selects Content │
│ Clicks "Optimize" │
└────────┬────────────────┘
┌─────────────────────────┐
│ OptimizerService │
│ optimize_from_wordpress_│
│ sync(content_id) │
└────────┬────────────────┘
┌─────────────────────────┐
│ Optimizer Processes │
│ (Same logic as IGNY8) │
│ - Analyzes │
│ - Optimizes │
│ - Stores results │
└────────┬────────────────┘
┌─────────────────────────┐
│ OptimizationTask │
│ Created │
│ Original preserved │
└────────┬────────────────┘
┌─────────────────────────┐
│ Optional: Sync Back │
│ to WordPress │
│ (Two-way sync) │
└─────────────────────────┘
```
---
## WORKFLOW 3: 3RD PARTY SYNC → OPTIMIZER → PUBLISH
```
┌─────────────────┐
│ Shopify/API │
│ Syncs Content │
│ to IGNY8 │
└────────┬────────┘
┌─────────────────────────┐
│ ContentSyncService │
│ sync_from_shopify() │
│ or sync_from_custom() │
└────────┬────────────────┘
┌─────────────────────────┐
│ Content Created │
│ source='shopify'/'custom'│
│ sync_status='imported' │
│ external_id=external_id │
│ external_url=external_url│
└────────┬────────────────┘
┌─────────────────────────┐
│ Content Visible │
│ in Writer/Content List │
│ (Filterable by source) │
└────────┬────────────────┘
┌─────────────────────────┐
│ User Selects Content │
│ Clicks "Optimize" │
└────────┬────────────────┘
┌─────────────────────────┐
│ OptimizerService │
│ optimize_from_external_ │
│ sync(content_id) │
└────────┬────────────────┘
┌─────────────────────────┐
│ Optimizer Processes │
│ (Same logic) │
└────────┬────────────────┘
┌─────────────────────────┐
│ OptimizationTask │
│ Created │
└─────────────────────────┘
```
---
## WORKFLOW 4: MANUAL SELECTION → LINKER/OPTIMIZER
```
┌─────────────────────────┐
│ User Views Content List │
│ (Any source) │
└────────┬────────────────┘
┌─────────────────────────┐
│ User Selects Content │
│ (Can filter by source) │
└────────┬────────────────┘
┌─────────────────────────┐
│ User Clicks Action: │
│ - "Add Links" │
│ - "Optimize" │
│ - "Link & Optimize" │
└────────┬────────────────┘
┌─────────────────────────┐
│ LinkerService or │
│ OptimizerService │
│ (Works for any source) │
└────────┬────────────────┘
┌─────────────────────────┐
│ Content Processed │
│ Results Stored │
└─────────────────────────┘
```
---
## CONTENT STORAGE STRATEGY
### Unified Content Model
All content stored in same `Content` model, differentiated by flags:
| Field | Values | Purpose |
|-------|--------|---------|
| `source` | `'igny8'`, `'wordpress'`, `'shopify'`, `'custom'` | Where content came from |
| `sync_status` | `'native'`, `'imported'`, `'synced'` | How content was added |
| `external_id` | String | External platform ID |
| `external_url` | URL | External platform URL |
| `sync_metadata` | JSON | Platform-specific data |
### Content Filtering
**Frontend Filters**:
- By source: Show only IGNY8, WordPress, Shopify, or All
- By sync_status: Show Native, Imported, Synced, or All
- By optimization status: Not optimized, Optimized, Needs optimization
- By linking status: Not linked, Linked, Needs linking
**Backend Queries**:
```python
# Get all IGNY8 content
Content.objects.filter(source='igny8', sync_status='native')
# Get all WordPress synced content
Content.objects.filter(source='wordpress', sync_status='synced')
# Get all content ready for optimization
Content.objects.filter(optimizer_version=0)
# Get all content ready for linking
Content.objects.filter(linker_version=0)
```
---
## ENTRY POINT SUMMARY
| Entry Point | Trigger | Content Source | Goes Through |
|-------------|---------|----------------|--------------|
| **Writer → Linker** | Auto or Manual | `source='igny8'` | Linker → Optimizer |
| **Writer → Optimizer** | Auto or Manual | `source='igny8'` | Optimizer (skip linker) |
| **WordPress Sync → Optimizer** | Manual or Auto | `source='wordpress'` | Optimizer only |
| **3rd Party Sync → Optimizer** | Manual or Auto | `source='shopify'/'custom'` | Optimizer only |
| **Manual Selection → Linker** | Manual | Any source | Linker only |
| **Manual Selection → Optimizer** | Manual | Any source | Optimizer only |
---
## KEY PRINCIPLES
1. **Unified Storage**: All content in same model, filtered by flags
2. **Source Agnostic**: Linker/Optimizer work on any content source
3. **Flexible Entry**: Multiple ways to enter pipeline
4. **Preserve Original**: Original content always preserved
5. **Version Tracking**: `linker_version` and `optimizer_version` track processing
6. **Filterable**: Content can be filtered by source, sync_status, processing status
---
**END OF DOCUMENT**

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,124 @@
# Unified Credit-Based Usage System — Instructions for Cursor
## Objective
Transition IGNY8 from a limit-based subscription model to a fully credit-driven usage model.
Every feature is unlocked for all paid users, and only credits determine how much the platform can be used.
---
## 1. Core Principle
Remove all numerical limits tied to subscription plans.
Examples of limits that must no longer exist:
- Number of keywords allowed
- Number of clusters
- Number of content ideas
- Daily or monthly content tasks
- Monthly word count
- Image generation limits
- Site limits
- User limits
- Any plan-based feature restrictions
The only limiter in the entire system should be **credits**.
---
## 2. Purpose of Subscription Plans
Plans should no longer define usage capacity.
Plans only define:
- Monthly credits added to the account
- Support and onboarding level
- Billing cycle (monthly or yearly)
Every paid user gets full access to all features without restrictions.
---
## 3. Credit Economy
Every system action should consume a defined number of credits.
The values can be adjusted later, but the mechanism must be universal.
Example conceptual structure:
- One clustering request consumes a small number of credits
- One idea generation request consumes more credits
- Content generation consumes credits based on word count
- Image generation consumes credits per image
- Optimization consumes credits per thousand words
- Linking operations consume a fixed number of credits
Credits should be deducted at the moment the action is initiated.
---
## 4. System Behavior
Cursor should apply these global rules:
- If the account has enough credits, the action proceeds
- If credits are insufficient, the system shows a clear warning and does not run the action
- All features remain available, but actions require credits
- Credit consumption must be tracked and logged for transparency
- Credit usage must be consistent across all modules
This ensures that usage is pay-as-you-go within the subscription credit budget.
---
## 5. Removal of Old Logic
All of the following must be removed from the system:
- Any checks for keyword count
- Any maximum allowed clusters
- Any caps on ideas
- Any daily task limits
- Any monthly generation limits
- Any image generation limits
- Any limits based on the users plan
- Any feature locking based on plan tier
The platform should feel “unlimited,” with only credit balance controlling usage.
---
## 6. Frontend Adjustments
On the interface:
- Remove UI elements showing limits
- Replace them with a simple display of remaining credits
- Show estimated credit cost before performing any action
- If credits are insufficient, display a clear prompt to buy more credits
- All features must appear unlocked and available
The user should understand that only credits restrict their usage, not the plan.
---
## 7. Billing and Top-ups
Plans supply monthly credits, but users should be able to purchase extra credits anytime.
The system must support:
- Monthly credit replenishment
- On-demand credit top-ups
- Tracking of credit usage history
- Notifications when credits run low
This ensures consistent monetization and scalability.
---
## 8. Guiding Principle for Cursor
Cursor must treat **credits as the universal currency of usage**, and remove every other form of operational restriction.
The product becomes:
- Simpler for users
- Simpler for engineering
- Easier to maintain
- Scalable in pricing
- Fully usage-based
All future modules (Optimizer, Linker, etc.) must also follow the same credit model.
---

89
docs/refactor/README.md Normal file
View File

@@ -0,0 +1,89 @@
# REFACTORING DOCUMENTATION
**Purpose**: This directory contains refactoring plans, migration guides, and architectural refactoring documentation.
---
## Directory Structure
```
docs/refactor/
├── README.md # This file
├── routes/ # Route refactoring plans
├── folder-structure/ # Folder structure refactoring plans
└── migrations/ # Migration guides for refactoring
```
---
## Refactoring Plans
### Current Refactoring Status
**Phase 0: Foundation & Credit System**
- [ ] Credit-only model migration
- [ ] Plan model simplification
- [ ] Module settings system
**Phase 1: Service Layer Refactoring**
- [ ] Domain-driven structure
- [ ] Service layer implementation
- [ ] Model migrations
**Phase 2: Automation System**
- [ ] AutomationRule model
- [ ] ScheduledTask model
- [ ] Celery integration
**Phase 3: Site Builder**
- [ ] Site Builder models
- [ ] File management service
- [ ] Sites folder access
**Phase 4: Linker & Optimizer**
- [ ] Content model extensions
- [ ] Multiple entry points
- [ ] Workflow implementation
**Phase 5: Sites Renderer**
- [ ] Sites container
- [ ] Layout system
- [ ] Template system
**Phase 6: Site Integration**
- [ ] SiteIntegration model
- [ ] Multi-destination publishing
- [ ] Integration adapters
**Phase 7: UI Components**
- [ ] Global component library
- [ ] Module settings UI
- [ ] Site management UI
**Phase 8: Universal Content Types**
- [ ] Content type extensions
- [ ] Taxonomy support
- [ ] Product/Service pages
---
## Route Refactoring
See `routes/` directory for route refactoring plans.
---
## Folder Structure Refactoring
See `folder-structure/` directory for folder structure refactoring plans.
---
## Migration Guides
See `migrations/` directory for step-by-step migration guides.
---
**Last Updated**: 2025-01-XX

View File

@@ -0,0 +1,64 @@
# FOLDER STRUCTURE REFACTORING PLANS
**Purpose**: Documentation for folder structure refactoring and reorganization.
---
## Current Structure
```
backend/igny8_core/
├── modules/ # Feature modules
├── ai/ # AI framework
├── api/ # API base classes
└── middleware/ # Custom middleware
```
---
## Planned Structure (Domain-Driven)
```
backend/igny8_core/
├── core/ # Core models (Account, User, Site, Sector)
├── domain/ # Domain-specific code
│ ├── content/ # Content domain
│ ├── planning/ # Planning domain
│ ├── linking/ # Linking domain
│ ├── optimization/# Optimization domain
│ ├── site_building/# Site building domain
│ ├── integration/ # Integration domain
│ └── billing/ # Billing domain
├── infrastructure/ # Infrastructure code
│ ├── ai/ # AI framework
│ ├── storage/ # Storage services
│ └── queue/ # Queue management
├── modules/ # Module ViewSets (thin layer)
├── shared/ # Shared utilities
└── api/ # API base classes
```
---
## Migration Strategy
1. Create new domain folders
2. Move models to domain folders
3. Create service layer in domain folders
4. Update imports incrementally
5. Keep modules folder for ViewSets only
---
## File Organization Rules
- **Models**: `domain/{domain}/models.py`
- **Services**: `domain/{domain}/services/`
- **Serializers**: `modules/{module}/serializers.py`
- **ViewSets**: `modules/{module}/views.py`
- **URLs**: `modules/{module}/urls.py`
---
**Last Updated**: 2025-01-XX

View File

@@ -0,0 +1,52 @@
# MIGRATION GUIDES
**Purpose**: Step-by-step migration guides for refactoring.
---
## Migration Guides
### Phase 0: Credit System Migration
- [ ] Remove plan limit fields
- [ ] Update Plan model
- [ ] Update CreditService
- [ ] Update AI Engine
- [ ] Update frontend
### Phase 1: Service Layer Migration
- [ ] Create domain folders
- [ ] Move models
- [ ] Create services
- [ ] Update ViewSets
- [ ] Update imports
### Phase 2: Content Model Extensions
- [ ] Add source field
- [ ] Add sync_status field
- [ ] Add external_id fields
- [ ] Create migrations
- [ ] Update serializers
### Phase 3: New Module Integration
- [ ] Automation module
- [ ] Linker module
- [ ] Optimizer module
- [ ] Site Builder module
- [ ] Integration module
---
## Migration Checklist
For each migration:
- [ ] Create migration files
- [ ] Test migrations
- [ ] Update code references
- [ ] Update tests
- [ ] Update documentation
- [ ] Deploy incrementally
---
**Last Updated**: 2025-01-XX

View File

@@ -0,0 +1,56 @@
# ROUTE REFACTORING PLANS
**Purpose**: Documentation for API route refactoring and reorganization.
---
## Current Route Structure
### Backend Routes
- `/api/v1/planner/` - Planner module routes
- `/api/v1/writer/` - Writer module routes
- `/api/v1/thinker/` - Thinker module routes
- `/api/v1/system/` - System module routes
- `/api/v1/billing/` - Billing module routes
- `/api/v1/auth/` - Authentication routes
### Frontend Routes
- `/planner/*` - Planner pages
- `/writer/*` - Writer pages
- `/thinker/*` - Thinker pages
- `/settings/*` - Settings pages
- `/billing/*` - Billing pages
---
## Planned Route Changes
### Phase 1: Service Layer Routes
- New routes for service-based endpoints
- Domain-specific route organization
### Phase 2: New Module Routes
- `/api/v1/automation/` - Automation routes
- `/api/v1/linker/` - Linker routes
- `/api/v1/optimizer/` - Optimizer routes
- `/api/v1/site-builder/` - Site Builder routes
- `/api/v1/integration/` - Integration routes
### Phase 3: Frontend Route Updates
- New module pages
- Route guards for module access
- Conditional route loading
---
## Migration Strategy
1. Add new routes alongside existing routes
2. Gradually migrate endpoints to new structure
3. Maintain backward compatibility
4. Update frontend routes incrementally
---
**Last Updated**: 2025-01-XX