igny8/igny8-ai-seo-wp-plugin/docs/MIGRATION_PLAN_EXTRACTED.txt

IGNY8 WP PLUGIN TO IGNY8 APP MIGRATION PLAN
============================================

SECTION 1 - CODEBASE & DIRECTORY STRUCTURE (IGNY8 APP)
Goal: Design the top-level and module-level folder hierarchy for the new Django + React (Vite + Tailwind) full-stack IGNY8 App, preserving 1:1 parity with the current WordPress plugin modules while introducing clear API, automation, and AI boundaries.

1.1 Top-Level Repository Layout
igny8-app/
  backend/                    # Django backend (core API, models, automation)
    igny8_core/              # Main Django app (replaces plugin core)
      __init__.py
      settings.py            # Env-aware settings (local / staging / prod)
      urls.py
      wsgi.py
      asgi.py
      modules/               # 1:1 migration of plugin modules
        planner/
          models.py
          views.py
          serializers.py
          tasks.py           # CRON/async/queue handlers
          urls.py
        writer/
        thinker/
        ... (future: linker, optimizer, etc.)
      api/                   # Unified REST entrypoints
        routes/
          ai/
            content.py
            images.py
            publish.py
            reparse.py
          system/
        middleware/
      auth/                  # Multi-tenant + RBAC + Stripe linkage
        models.py
        roles.py
        views.py
        signals.py
        serializers.py
      utils/                 # Shared helpers (AI, logs, etc.)
    cron/                    # Timed jobs (mapped from plugin CRON)
    migrations/
    manage.py
  frontend/                  # React + Vite + Tailwind UI
    src/
      app/
        globals.css
        layout.tsx
        pages/               # Page-per-module parity with WP admin
          PlannerPage.tsx
          WriterPage.tsx
          ThinkerPage.tsx
          DashboardPage.tsx
        components/
          ui/                # Local UI library (Card, Button, etc.)
          layout/
          charts/
        hooks/
        services/            # API clients (axios/fetch for each module)
        store/               # Zustand/Redux state
      vite.config.js
      package.json
      tsconfig.json
  infra/                     # Docker, Portainer, Caddy, Postgres, Redis
    docker-compose.yml
    caddy/
    scripts/
    dev-config.yml
    README.md
  docs/
    MIGRATION_PLAN.md
    API_REFERENCE.md
    DATABASE_MAP.md
  .env / .env.dev / .env.prod  # environment-specific configs

1.2 Key Design Rules
- Modules: Each former WordPress module = isolated Django app inside /backend/modules/
- API Layer: Use /backend/api/routes/{domain}/ for all external triggers
- Frontend: Page = Module parity (PlannerPage.tsx, etc.)
- UI Library: /frontend/src/components/ui/ replaces @/components/ui/ from TailAdmin
- DevOps: Single infra/docker-compose.yml defines the full stack
- Roles/Billing: /backend/auth/roles.py + Stripe integration
- Docs: /docs folder in-repo

1.3 Role-Based Access (RBAC) Foundation
Roles:
- Owner: Full access to all tenants, billing, and automation
- Admin: Manage content modules, view billing but not edit
- Editor: Generate AI content, manage clusters/tasks
- Viewer: Read-only dashboards
- SystemBot: Automation + CRON actions (No UI, API-only)

1.4 AI-Assisted Dev Environment (Cursor / Local PC)
- Expose selective sub-repos /backend/modules/, /frontend/src/, /docs/ to GitHub/GitLab
- Keep /infra & .env excluded (.gitignore) for security
- AI Context Indexing: Each module includes README.md describing schema, routes, and functions
- Dockerized Dev Setup: Cursor runs Node + Python containers via docker-compose
- Source Sync Hooks: Git hooks ensure migrations and schema docs stay synchronized


SECTION 2 - DATABASE & SCHEMA MIGRATION PLAN
Goal: Migrate all IGNY8 custom database tables and essential data from WordPress to a Django + PostgreSQL schema.

2.1 IGNY8 Core Tables to Migrate 1:1
WP Table → Django Model:
- igny8_keywords → Keywords
- igny8_clusters → Clusters
- igny8_content_ideas → ContentIdeas
- igny8_tasks → Tasks
- igny8_variations → Variations
- igny8_prompts → Prompts
- igny8_logs → Logs
- igny8_images → Images
- igny8_schedules → Schedules
- igny8_settings → Settings (Split into SystemSettings and UserSettings)
- igny8_accounts → Accounts
- igny8_links → Links (Future)
- igny8_metrics → Metrics

2.2 WordPress Default Tables (Partial Extraction)
- wp_posts → WPPosts (ID, post_title, post_content, post_status, post_type, post_date)
- wp_postmeta → WPPostMeta (post_id, meta_key, meta_value - only IGNY8 keys)
- wp_users → WPUserMap (ID, user_email, display_name)
- wp_options → WPOptions (option_name, option_value - plugin config only)

2.3 Cross-System Integration Map
- Keywords/Clusters: App → WP via WP REST API
- Tasks/Content: App → WP via WP /wp-json/igny8/v1/import-content
- Images: App → WP via WP Media REST endpoint
- Settings: Bidirectional sync
- Logs/Metrics: App-only

2.4 Schema Example: Tasks Model (Django ORM)
Includes: id, cluster, idea, task_type, ai_model, prompt, raw_ai_response, status, result, timestamps, user, tenant

2.5 Database Migration & Sync Process
Steps: Export → Create Django models → Import data → Verify → Migrate WP data → Run integrity tests → Enable periodic sync

2.6 Tenant-Aware Model Design
Every model inherits from TenantBaseModel with tenant, created_at, updated_at fields.


SECTION 3 - COMPONENT & UI MAPPING PLAN
Goal: Rebuild the WordPress plugin's admin interface as React/Vite pages.

3.1 One-to-One Module → Page Mapping
- Planner Dashboard → PlannerPage.tsx (/planner)
- Writer Dashboard → WriterPage.tsx (/writer)
- Thinker → ThinkerPage.tsx (/thinker)
- Optimizer (Phase 2) → OptimizerPage.tsx (/optimizer)
- Linker (Phase 2) → LinkerPage.tsx (/linker)
- Settings → SettingsPage.tsx (/settings)
- Dashboard → DashboardPage.tsx (/dashboard)

3.2 Shared Frontend Component Map
- DataTable, FilterPanel, FormModal, ActionButtons, StatsCards, ToastNotifications, AIStatusBar, ConfirmDialog, TenantSwitcher

3.3 UI Data Flow Diagram (Per Module)
User → React Page → Zustand Store → API Service → Django API → AI Processing → Database → React Refresh

3.4 State Management Rules (Zustand)
Each module defines its store following the same interface pattern.


SECTION 4 - API ROUTES & AUTOMATION TRIGGERS
Goal: Define all Django REST API endpoints and automation triggers.

4.1 Unified API Structure
- /api/planner/ - Keyword → Cluster → Idea generation
- /api/writer/ - Content generation, reparsing, image injection
- /api/thinker/ - Prompt testing, image model handling
- /api/settings/ - Model rates, limits, keys, Stripe data
- /api/system/ - Metrics, logs, and CRON control
- /api/auth/ - Tenant registration, login, roles
- /api/publish/ - Push content/images to WP site

4.2 Example Endpoint Tree
Detailed endpoint listing for each module with GET/POST/DELETE methods

4.3 API → AI Execution Chain
Common AI service layer (/backend/utils/ai.py) handles all AI requests

4.4 Automation Triggers (Replacing WordPress CRON)
- auto_cluster_keywords: every 6h
- auto_generate_ideas: daily
- auto_generate_content: hourly
- auto_generate_images: hourly
- auto_publish_posts: daily
- cleanup_logs: weekly

4.5 Queue Control Logic
Shared queue core (/backend/utils/queue_manager.py) for automation and manual triggers

4.6 Stripe Integration (API)
Endpoints for billing, subscription, webhooks

4.7 AI Key & Model Management
Configured through /api/settings/ai-models

4.8 Example: /api/writer/generate/<id> Workflow
8-step process from React frontend to WordPress publish

4.9 Authentication & Role Guard Middleware
All endpoints protected by RoleRequired decorator

4.10 CRON & API Coherence
CRON functions reuse the same endpoints as manual buttons


SECTION 5 - WORDPRESS INTEGRATION & SYNC LAYER
Goal: Design a clean, secure bridge between Django backend and WordPress sites.

5.1 Integration Overview
- AI Content Publish: App → WP
- Image Upload: App → WP
- Settings Sync: Bidirectional
- Post Status Check: WP → App
- Keyword/Cluster Sync: App → WP
- Auth Link: Manual JWT-based connection

5.2 Connection Setup (One-Time Auth Handshake)
WPIntegration model stores site_url, api_key, tenant, status, last_sync

5.3 WordPress REST Endpoints (Added to Plugin)
- /wp-json/igny8/v1/import-content
- /wp-json/igny8/v1/upload-image
- /wp-json/igny8/v1/sync-settings
- /wp-json/igny8/v1/status
- /wp-json/igny8/v1/pull-updates

5.4 Django → WordPress Communication Flow
Example publishing workflow with request/response structure

5.5 WordPress Plugin: Receiving Side Implementation
Minimalistic handler example code

5.6 Bidirectional Settings Sync
Field mapping between App and WP

5.7 Multi-Tenant Integration Mapping
Each tenant can connect multiple WP sites

5.8 Sync Safety & Rate Control Layer
Queue throttling, error handling, fallback mode, audit logs

5.9 Security Model
JWT with 12h expiry, HMAC signatures, CORS whitelist

5.10 Example: Full Publish & Feedback Cycle
Complete workflow from user action to periodic sync

5.11 Future Extension (Phase-2+)
Planned integrations for metrics, links, user data, schema


SECTION 6 - DEPLOYMENT, ENVIRONMENT & LOCAL DEV PLAN
Goal: Define complete development, deployment, and synchronization environment.

6.1 Full Stack Architecture Summary
- Reverse Proxy: Caddy (80/443)
- Frontend: React (Vite) + Node 20 (8020→8021)
- Backend: Django + Gunicorn (8010→8011)
- Database: PostgreSQL 15 (5432)
- Cache/Queue: Redis 7 (6379)
- Admin DB UI: pgAdmin4 (5050)
- File Manager: Filebrowser (8080)
- Docker Manager: Portainer CE (9443/8000)

6.2 Environment File (.env)
Variables for DB, Redis, AI keys, Stripe, domains

6.3 Docker Compose Structure
Service definitions with volumes, networks, environment variables

6.4 Local Development Setup (Cursor AI)
Steps for local development with live mounts

6.5 Portainer Stack Deployment
Production deployment via Portainer stacks

6.6 Environment-Specific Configs
Separate configs for local, staging, production

6.7 Backup & Recovery Procedures
Automated backups for database, media, configs


SECTION 7 - DATA MIGRATION & SYNC STRATEGY
Goal: Extract, transform, and import all IGNY8 plugin data from WordPress MySQL to PostgreSQL.

7.1 Migration Overview
Extract from WP → Transform schema → Import to Django → Validate → Sync

7.2 Table Mapping Details
Complete mapping of all WP tables to Django models

7.3 Migration Phases
1. Extraction (Dump plugin tables)
2. Transformation (Convert to Postgres schema)
3. Import (Bulk insert via Django)
4. Validation (Compare counts, hashes)
5. Sync (Enable real-time sync to WP)

7.4 Extraction Script Example
Python script using mysql.connector

7.5 Transformation Example
Data transformation script

7.6 Import to Django
Django management command for bulk import

7.7 Verification Step
Comparison script for validation

7.8 Syncable Tables (Remain Linked to WP)
Tables that maintain bidirectional sync

7.9 Migration Validation Dashboard
UI section showing migration status

7.10 Rollback Strategy
Procedure for handling migration failures

7.11 Final Verification Checklist
Checkpoints for successful migration

7.12 Post-Migration Tasks
Deactivate old plugin CRON, update plugin config to Remote Mode


SECTION 8 - TENANCY, BILLING & USER ACCESS MODEL
Goal: Define multi-tenant access, workspace isolation, role permissions, and subscription billing.

8.1 Core Concepts
- Tenant: Logical workspace
- User: Authenticated account inside tenant
- Subscription: Stripe-linked billing plan
- Workspace: UI grouping under tenant
- Site Integration: Connected WordPress instances

8.2 Data Model Overview
Django Models: Tenant, User, Plan, Subscription

8.3 Stripe Integration Workflow
6-step process from plan selection to webhook handling

8.4 Credit System Logic
Credit costs per action:
- Keyword clustering: 1 credit / 30 keywords
- Content idea generation: 1 credit / idea
- Full blog content: 3 credits
- AI image generation: 1 credit / image
- Reparse content: 1 credit
- Auto publish: Free if already generated

8.5 Roles & Access Permissions
Owner, Admin, Editor, Viewer roles with specific permissions

8.6 Tenant Isolation Enforcement
Database-level, file-level, API-level, worker-level isolation

8.7 Tenant Dashboard Layout (Frontend)
React components for tenant management

8.8 Billing Plans (Finalized Baseline)
- Free/Trial: $1, 25 credits, 1 user, 1 site
- Starter: $39, 200 credits, 3 users, 3 sites
- Pro: $89, 600 credits, 5 users, 5 sites
- Agency: $199, 2000 credits, 15 users, 10 sites
- Custom/Enterprise: Variable

8.9 Webhooks for Sync
Stripe and WordPress webhook handlers

8.10 Suspended Tenant Behavior
Handling payment failures

8.11 Multi-Tenant CRON Scheduler
Per-tenant queue system

8.12 Cross-Module Tenant Integration
How each module enforces tenant boundaries

8.13 Stripe Integration Files
Backend implementation structure

8.14 Security Enforcement
Authentication, authorization, webhook validation, file access, admin audit

8.15 Example Flow - New Tenant Signup
Complete signup to activation workflow


SECTION 9 - AI AUTOMATION PIPELINE & TASK ENGINE
Goal: Establish unified, tenant-aware automation system for all AI-based tasks.

9.1 Core Design Principles
- Single-Item Execution
- Tenant Isolation
- Unified Scheduler
- Configurable Limits
- Recoverable Jobs

9.2 AI Task Types & Flow
- Planner: Keyword Clustering → Idea Generation
- Writer: AI Draft Generation → Reparse → Publish
- Thinker: Prompt Creation / Persona / Strategy
- Image Generator: DALL-E / Runware image tasks
- Linker (Phase-2): Backlink planner

9.3 Queue Architecture (Redis-Backed)
Redis 7.x with Celery or django-q, tenant-based queue naming

9.4 AI Execution Workflow
Scheduler → Redis Queue → Django Worker → AI Engine API → Parser Layer → DB + Logs

9.5 Execution Limits & Global Settings
Parameters for AI_MAX_ITEMS_PER_REQUEST, batch sizes, timeouts, retries, cost caps

9.6 Task Lifecycle (Writer Example)
Queued → Dispatch → AI Request → Response Handling → Validation → Storage → Optional Actions

9.7 CRON Automation Flows
- cron_auto_cluster(): 6h
- cron_auto_ideas(): 6h
- cron_auto_writer(): 3h
- cron_auto_image(): 3h
- cron_auto_publish(): 12h
- cron_cleanup_logs(): 24h

9.8 AI Model Routing Logic
Model selector with auto-balancing and fallback

9.9 AI Pipeline Directory Map
Backend module structure for AI pipeline

9.10 Credit Deduction Example
Credit deduction logic

9.11 Error Recovery Flow
Handling timeouts, invalid JSON, parser failures, sync failures, credit errors

9.12 Frontend Task Control (React Components)
TaskQueueTable, TaskControlPanel, AutomationConfigForm with WebSocket feed

9.13 Monitoring & Telemetry
Structured logging with tenant, task_type, status, duration, model, credits_spent, output_size

9.14 Local AI Dev Mode (Cursor-Ready)
Development configuration for local testing

9.15 Verification Checklist
Checkpoints for queue, worker, task enqueue, credit deduction, AI response, CRON logs

9.16 Future Enhancements
Parallel pipelines, semantic caching, rate shaping, cost anomaly alerts, tracing integration


SECTION 10 - SECURITY, LOGGING & MONITORING FRAMEWORK
Goal: Define full-stack security, audit, and observability framework.

10.1 Security Architecture Overview
Layers: Frontend, Backend, Database, AI Layer, Infrastructure, Integrations

10.2 Authentication & Token System
JWT authentication with tenant context injection, role verification, session expiry

10.3 Authorization & Role-Based Rules
Role access scope definitions

10.4 Secure API Architecture
Endpoint pattern, JWT verification, tenant match, role access, input validation

10.5 Stripe & Webhook Security
HMAC validation for webhooks

10.6 Data Encryption & Storage Policy
Encryption for credentials/keys, storage policies for AI responses, files, backups, logs

10.7 API Rate Limiting
Per tenant (30 req/min), per IP (100 req/min), per worker (1 AI request/iteration), per model (60 req/min)

10.8 Logging System Overview
Centralized logging: App → Django Logger → Postgres → Loki Stream → Grafana

10.9 Monitoring & Alerts Stack
Tools: Portainer, Loki, Grafana, Alertmanager, Redis Inspector
Custom alerts for AI timeout, failed tasks, Stripe failures, CRON lag

10.10 Data Backup & Recovery
Backup frequencies and retention:
- Postgres DB: Every 6 hours, 7 days retention
- FileBrowser/Media: Daily, 14 days
- Caddy/Config: Daily, 7 days
- Logs (Loki): Rolling window, 30 days

10.11 Error & Exception Handling
Handling for AI API errors, DB write errors, worker crashes, CRON failures, user API errors

10.12 Developer Audit Trail
Critical system events logged in igny8_audit with 1 year retention

10.13 Local Dev & Security Mirrors
Development configuration with AI_DEV_MODE, CORS settings, mock data

10.14 Security Verification Checklist
Checkpoints for HTTPS, JWT validation, tenant isolation, webhook verification, file access, backups, Redis security, network isolation, rate limiting

10.15 Future Enhancements
OAuth 2.0 login, tenant-level dashboards, ElasticSearch sink, offsite sync, smart anomaly detection

10.16 Final Summary
Security model fully container-aware and tenant-isolated
Logging & metrics unified under Loki + Grafana
Stripe billing, AI cost tracking, and access control audited
Developer-friendly dev mode supported
Production deployment validated under current Docker infrastructure


END OF DOCUMENT