From 264d98f146ba64d2266df585437087420fd0eb3d Mon Sep 17 00:00:00 2001 From: "IGNY8 VPS (Salman)" Date: Mon, 23 Feb 2026 22:47:48 +0000 Subject: [PATCH] soclaizer and video plan --- .../SOCIALIZER-AND-VIDEO-CREATOR-MODULES.md | 1376 +++++++++++++++++ 1 file changed, 1376 insertions(+) create mode 100644 docs/plans/SOCIALIZER-AND-VIDEO-CREATOR-MODULES.md diff --git a/docs/plans/SOCIALIZER-AND-VIDEO-CREATOR-MODULES.md b/docs/plans/SOCIALIZER-AND-VIDEO-CREATOR-MODULES.md new file mode 100644 index 00000000..03f7a714 --- /dev/null +++ b/docs/plans/SOCIALIZER-AND-VIDEO-CREATOR-MODULES.md @@ -0,0 +1,1376 @@ +# Socializer & Video Content Creator — Development Plan + +> **Created:** February 22, 2026 +> **Target Version:** v2.0.0 +> **Status:** Planning +> **Dependencies:** Existing IGNY8 v1.8.4 architecture + +--- + +## Table of Contents + +1. [Executive Summary](#1-executive-summary) +2. [Module A: Socializer](#2-module-a-socializer) +3. [Module B: Video Content Creator](#3-module-b-video-content-creator) +4. [Shared Infrastructure](#4-shared-infrastructure) +5. [Self-Hosted GPU Option](#5-self-hosted-gpu-option) +6. [Database Schema (All New Models)](#6-database-schema) +7. [AI Functions (New)](#7-ai-functions) +8. [API Endpoints (New)](#8-api-endpoints) +9. [Frontend Pages (New)](#9-frontend-pages) +10. [Celery Tasks (New)](#10-celery-tasks) +11. [Credit Cost Matrix](#11-credit-cost-matrix) +12. [Development Phases](#12-development-phases) +13. [Risk & Dependency Matrix](#13-risk--dependency-matrix) + +--- + +## 1. Executive Summary + +Two new active modules extending the existing content pipeline: + +``` +CURRENT PIPELINE (v1.8.4): +Keywords → Clusters → Ideas → Tasks → Content → Images → Published (WordPress) + +EXTENDED PIPELINE (v2.0.0): + ┌→ WordPress (existing) +Keywords → Clusters → Ideas → Tasks → Content → Images ──┼→ Socializer → 5 Platforms + └→ Video Creator → 3+ Platforms +``` + +### What We're Building + +| Module | Purpose | Platforms | AI Involved | +|--------|---------|-----------|-------------| +| **Socializer** | Adapt published content into social posts, schedule across platforms | LinkedIn, Twitter/X, Facebook, Instagram, TikTok | Text adaptation, image resizing/generation, hashtag optimization | +| **Video Creator** | Convert articles into video content, publish to video platforms | YouTube, Instagram Reels, TikTok, YouTube Shorts | Script generation, TTS voiceover, video assembly, thumbnail generation | + +### Integration Points with Existing System + +| Existing Component | How It's Used | +|--------------------|---------------| +| `Content` model | Source material for both modules | +| `Images` model | Source images for social posts and video thumbnails | +| `AIEngine` | Extended with new functions for social/video generation | +| `ModelRegistry` / `AIModelConfig` | New models registered (TTS, video) | +| `IntegrationProvider` | New providers added (social APIs, video APIs) | +| `CreditService` | All operations are credit-based | +| `AutomationConfig` | New stages 8 (Socializer) and 9 (Video) added to pipeline | +| `NotificationService` | Notifications for publish success/failure | +| `PublishingSettings` | Extended with social + video scheduling rules | +| `Celery Beat` | New scheduled tasks for social posting windows | + +--- + +## 2. Module A: Socializer + +### 2.1 Overview + +``` +┌──────────────────────────────────────────────────────────────────────────────┐ +│ SOCIALIZER MODULE │ +├──────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ CONTENT (Published) │ +│ │ │ +│ ▼ │ +│ ┌─────────────────┐ ┌────────────────────────────────────────────────┐ │ +│ │ AI Adaptation │──►│ Platform-Specific Posts │ │ +│ │ Engine │ │ │ │ +│ │ │ │ LinkedIn → Professional tone, 1300 chars │ │ +│ │ • Tone shift │ │ Twitter/X → Punchy, 280 chars, thread option │ │ +│ │ • Length trim │ │ Facebook → Conversational, 500 chars │ │ +│ │ • Hashtag gen │ │ Instagram → Visual-first, 2200 chars+30 tags │ │ +│ │ • CTA insertion │ │ TikTok → Gen-Z tone, trending hooks │ │ +│ └─────────────────┘ └──────────────────────┬─────────────────────────┘ │ +│ │ │ +│ ┌────────────────────────────────────────┤ │ +│ ▼ ▼ │ +│ ┌─────────────────┐ ┌─────────────────┐ │ +│ │ Image Optimizer │ │ Social Calendar │ │ +│ │ │ │ │ │ +│ │ LinkedIn: 1200 │ │ Best-time slots │ │ +│ │ ×627 │ │ Queue management │ │ +│ │ Twitter: 1600 │ │ Drag-and-drop │ │ +│ │ ×900 │ │ Integrated with │ │ +│ │ Facebook: 1200 │ │ content calendar │ │ +│ │ ×630 │ └────────┬────────┘ │ +│ │ Instagram: 1080 │ │ │ +│ │ ×1080 │ ▼ │ +│ │ TikTok: 1080 │ ┌─────────────────┐ │ +│ │ ×1920 │ │ Platform APIs │ │ +│ │ │ │ (OAuth2 + SDK) │ │ +│ └─────────────────┘ └─────────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────────────────────────┘ +``` + +### 2.2 Platform API Requirements + +| Platform | API | Auth | Post Types | Rate Limits | Developer Access | +|----------|-----|------|------------|-------------|-----------------| +| **LinkedIn** | LinkedIn Marketing API v2 | OAuth 2.0 (3-legged) | Text, Image, Article, Carousel | 100 posts/day | Company page required, app review | +| **Twitter/X** | X API v2 (Free/Basic/Pro) | OAuth 2.0 + PKCE | Tweet, Thread, Image, Poll | 1500 tweets/mo (Free), 3000 (Basic) | Developer account, app approval | +| **Facebook** | Meta Graph API v19 | OAuth 2.0 (Page token) | Text, Photo, Link, Video, Carousel | 200 posts/hr per page | Meta Business verification | +| **Instagram** | Instagram Graph API | OAuth 2.0 (via Facebook) | Photo, Carousel, Reel, Story | 25 API calls/hr per user | Facebook Business account required | +| **TikTok** | TikTok Content Posting API | OAuth 2.0 | Video, Photo | 10k requests/day | TikTok for Developers access | + +### 2.3 Features Breakdown + +#### A. Content Adaptation (AI) + +| Feature | Description | AI Provider | Existing/New | +|---------|-------------|-------------|--------------| +| Tone adaptation | Rewrite content excerpt per platform voice | OpenAI GPT-4o / Anthropic Claude | New AI Function | +| Length optimization | Trim/expand to platform char limits | OpenAI GPT-4o-mini | New AI Function | +| Hashtag generation | Generate relevant, trending hashtags | OpenAI GPT-4o-mini | New AI Function | +| CTA insertion | Add platform-appropriate calls-to-action | OpenAI GPT-4o-mini | Part of adaptation | +| Thread splitting | Break long content into Twitter threads | OpenAI GPT-4o-mini | New AI Function | +| Carousel creation | Split article into carousel slides (IG/LinkedIn) | OpenAI GPT-4o | New AI Function | +| Emoji optimization | Add appropriate emojis per platform norms | Included in adaptation | Part of adaptation | + +#### B. Image Optimization + +| Feature | Description | AI Provider | Existing/New | +|---------|-------------|-------------|--------------| +| Resize/crop | Resize source images to platform specs | Pillow (local) | New utility | +| Text overlay | Add headline text on images for social | Pillow + custom fonts | New utility | +| Thumbnail generation | Create eye-catching social thumbnails | Runware / DALL-E / Bria | Extend existing | +| Format conversion | Convert to platform-optimal format (WebP/JPG) | Pillow (local) | New utility | +| Carousel image gen | Generate multiple slide images | Runware / DALL-E | Extend existing | + +**Platform Image Specs:** + +| Platform | Post Image | Story/Reel Cover | Carousel Slide | Max File Size | +|----------|-----------|-------------------|----------------|---------------| +| LinkedIn | 1200×627 | N/A | 1080×1080 | 10 MB | +| Twitter/X | 1600×900 | N/A | N/A | 5 MB | +| Facebook | 1200×630 | 1080×1920 | 1080×1080 | 10 MB | +| Instagram | 1080×1080 | 1080×1920 | 1080×1080 | 8 MB | +| TikTok | 1080×1920 | 1080×1920 | N/A | 72 MB (video) | + +#### C. Social Calendar + +| Feature | Description | +|---------|-------------| +| Calendar view | Monthly/weekly/daily calendar showing scheduled social posts | +| Best-time scheduling | AI-suggested optimal posting times per platform | +| Drag-and-drop | Reschedule posts by dragging on calendar | +| Queue management | FIFO queue with priority override | +| Integration | Merged view with existing Content Calendar (`/publisher/content-calendar`) | +| Bulk scheduling | Schedule posts for multiple platforms in one action | +| Recurring posts | Evergreen content re-posting on schedule | + +#### D. Analytics & Tracking + +| Metric | Source | Storage | +|--------|--------|---------| +| Impressions | Platform APIs (delayed) | `SocialPostMetrics` model | +| Clicks | Platform APIs OR UTM + our analytics | `SocialPostMetrics` model | +| Engagement rate | Calculated (likes+comments+shares / impressions) | Computed | +| Best performing platform | Aggregated | Dashboard widget | +| Best posting times | Historical analysis | `SocialAnalytics` model | + +### 2.4 User Flow + +``` +1. Content is published (or approved) in Writer/Publisher + │ + ▼ +2. User navigates to /socializer/posts (or auto-triggered by automation) + │ + ▼ +3. AI generates platform-specific versions (all 5 or selected) + │ + ├── LinkedIn post (professional tone, image 1200×627) + ├── Twitter/X tweet or thread (punchy, image 1600×900) + ├── Facebook post (conversational, image 1200×630) + ├── Instagram caption + image (1080×1080) or carousel + └── TikTok caption (for image-to-video or article link) + │ + ▼ +4. User reviews/edits each platform version + │ + ▼ +5. User schedules posting time (or uses AI-suggested best time) + │ + ▼ +6. Celery task publishes at scheduled time via platform APIs + │ + ▼ +7. Engagement metrics fetched periodically (every 6 hours) +``` + +--- + +## 3. Module B: Video Content Creator + +### 3.1 Overview + +``` +┌──────────────────────────────────────────────────────────────────────────────┐ +│ VIDEO CONTENT CREATOR MODULE │ +├──────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ CONTENT (Published Article) │ +│ │ │ +│ ▼ │ +│ ┌─────────────────┐ │ +│ │ Script Engine │ AI converts article → video script │ +│ │ (GPT-4o) │ • Sections with narration text │ +│ │ │ • Visual cues per section │ +│ │ │ • Timing estimates │ +│ └────────┬──────────┘ │ +│ │ │ +│ ┌─────┴──────┐ │ +│ ▼ ▼ │ +│ ┌────────┐ ┌──────────┐ │ +│ │Short │ │Long Form │ │ +│ │Form │ │(YouTube) │ │ +│ │30-90s │ │5-15 min │ │ +│ └───┬────┘ └────┬─────┘ │ +│ │ │ │ +│ ▼ ▼ │ +│ ┌─────────────────────────┐ │ +│ │ TTS Engine │ AI voiceover from script │ +│ │ • OpenAI TTS │ • Multiple voices │ +│ │ • ElevenLabs (opt) │ • Speed/tone control │ +│ │ • Self-hosted Coqui │ • Multi-language │ +│ └────────────┬─────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────┐ │ +│ │ Visual Assembly │ │ +│ │ │ │ +│ │ • Stock footage search │ Pexels/Pixabay API (free) │ +│ │ • Article images │ From existing Images model │ +│ │ • AI-generated scenes │ Runware/DALL-E/Stable Diffusion │ +│ │ • Text overlays │ Key points, statistics │ +│ │ • Transitions │ Fade, slide, zoom │ +│ └────────────┬─────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────────────────┐ │ +│ │ Video Composer │ FFmpeg / MoviePy │ +│ │ │ │ +│ │ • Merge audio+visuals │ │ +│ │ • Add captions/subs │ Burned-in or SRT │ +│ │ • Add music (opt) │ Royalty-free background │ +│ │ • Render final MP4 │ H.264 / H.265 │ +│ │ • Multi-resolution │ 1080p, 720p │ +│ └────────────┬─────────────┘ │ +│ │ │ +│ ┌─────────┼──────────┐ │ +│ ▼ ▼ ▼ │ +│ ┌──────┐ ┌───────┐ ┌────────┐ │ +│ │ YT │ │ Reels │ │ TikTok │ + YouTube Shorts │ +│ │Long │ │ Short │ │ Short │ │ +│ │form │ │ form │ │ form │ │ +│ └──────┘ └───────┘ └────────┘ │ +│ │ +└──────────────────────────────────────────────────────────────────────────────┘ +``` + +### 3.2 Platform Requirements + +| Platform | Format | Max Duration | Resolution | Max Size | API | +|----------|--------|-------------|------------|----------|-----| +| **YouTube** (long) | MP4 (H.264) | 12 hours | 1920×1080 (16:9) | 256 GB | YouTube Data API v3 | +| **YouTube Shorts** | MP4 (H.264) | 60 seconds | 1080×1920 (9:16) | 256 GB | YouTube Data API v3 | +| **Instagram Reels** | MP4 (H.264) | 90 seconds | 1080×1920 (9:16) | 650 MB | Instagram Graph API | +| **TikTok** | MP4 (H.264) | 10 minutes | 1080×1920 (9:16) | 72 MB | TikTok Content Posting API | + +### 3.3 Video Generation Pipeline + +#### Stage 1: Script Generation (AI) + +| Input | Process | Output | +|-------|---------|--------| +| `Content.content_body` (HTML) | GPT-4o extracts key points, writes narration script | `VideoScript` JSON | +| `Content.meta_description` | Creates hook/intro | Opening section | +| `Content.target_keywords` | SEO optimization for video title/description | Video SEO metadata | + +**VideoScript Structure:** +```json +{ + "title": "10 Python Tips Every Beginner Should Know", + "description": "SEO-optimized video description...", + "tags": ["python", "programming", "tutorial"], + "sections": [ + { + "section_id": 1, + "type": "hook", + "narration": "Did you know that 80% of Python beginners...", + "visual_cue": "animated_text", + "text_overlay": "80% of beginners make this mistake", + "duration_estimate": 5.0 + }, + { + "section_id": 2, + "type": "intro", + "narration": "In this video, we'll cover 10 essential tips...", + "visual_cue": "title_card", + "text_overlay": "10 Essential Python Tips", + "duration_estimate": 8.0 + }, + { + "section_id": 3, + "type": "point", + "narration": "Tip number one: Use list comprehensions...", + "visual_cue": "code_snippet", + "text_overlay": "Tip 1: List Comprehensions", + "code_block": "result = [x*2 for x in range(10)]", + "duration_estimate": 20.0 + } + ], + "total_duration_estimate": 480.0, + "chapter_markers": [ + {"time": "0:00", "title": "Introduction"}, + {"time": "0:13", "title": "Tip 1: List Comprehensions"} + ] +} +``` + +#### Stage 2: Voiceover (TTS) + +| Provider | Model | Quality | Latency | Cost Model | Self-Hostable | +|----------|-------|---------|---------|------------|---------------| +| **OpenAI TTS** | `tts-1` / `tts-1-hd` | Good / High | Fast | $15 / 1M chars (tts-1), $30 (hd) | No | +| **ElevenLabs** | Various | Very High | Medium | $5-$330/mo plans | No | +| **Coqui TTS** | XTTS-v2 | Good | Medium | Free (self-hosted) | **Yes** ✅ | +| **Bark** | Bark | Medium | Slow | Free (self-hosted) | **Yes** ✅ | +| **Piper TTS** | Piper | Good (fast) | Very Fast | Free (self-hosted) | **Yes** ✅ | + +#### Stage 3: Visual Asset Assembly + +| Asset Type | Source | How | +|-----------|--------|-----| +| Article images | Existing `Images` model | Pull from content | +| AI-generated scenes | Runware / DALL-E / Bria / Stable Diffusion | Generate from visual cues | +| Stock footage | Pexels API / Pixabay API | Free stock video clips | +| Text overlays | Pillow / FFmpeg drawtext | Rendered per section | +| Code snippets | Syntax-highlighted rendering | Pillow + Pygments | +| Background music | Royalty-free library (local) | Bundled tracks | + +#### Stage 4: Video Composition + +| Component | Technology | Purpose | +|-----------|-----------|---------| +| **FFmpeg** | System binary | Core video encoding, merging, transcoding | +| **MoviePy** | Python library (wraps FFmpeg) | High-level composition API | +| **Pillow** | Python library | Image manipulation, text overlays | +| **pydub** | Python library | Audio manipulation, mixing | +| **Pygments** | Python library | Code syntax highlighting for tech content | + +**Render Presets:** + +| Preset | Resolution | Aspect | Duration | Use Case | +|--------|-----------|--------|----------|----------| +| `youtube_long` | 1920×1080 | 16:9 | 3-15 min | YouTube main video | +| `youtube_short` | 1080×1920 | 9:16 | 30-60s | YouTube Shorts | +| `instagram_reel` | 1080×1920 | 9:16 | 30-90s | Instagram Reels | +| `tiktok` | 1080×1920 | 9:16 | 30-180s | TikTok | + +#### Stage 5: Video SEO & Publishing + +| Feature | Description | +|---------|-------------| +| Title optimization | AI-generated SEO titles per platform | +| Description | Keyword-rich descriptions with timestamps (YouTube) | +| Tags | AI-extracted relevant tags from content | +| Thumbnails | AI-generated eye-catching thumbnails | +| Captions/Subtitles | Auto-generated SRT from TTS transcript | +| Chapter markers | YouTube chapters from script sections | + +### 3.4 User Flow + +``` +1. Content is published (or user manually selects content) + │ + ▼ +2. User navigates to /video/create (or auto-triggered by automation) + │ + ▼ +3. Select video type: Short-form (Reels/Shorts/TikTok) or Long-form (YouTube) + │ + ▼ +4. AI generates video script → User reviews/edits script + │ + ▼ +5. Select voice (from TTS provider voice list) + │ + ▼ +6. Generate voiceover (TTS) → Preview/re-generate + │ + ▼ +7. AI assembles visuals (images + stock + overlays) + │ + ▼ +8. Render final video → Preview + │ + ▼ +9. Optimize SEO metadata (title, description, tags) + │ + ▼ +10. Schedule/publish to selected platform(s) + │ + ▼ +11. Track views, engagement (fetched from platform APIs) +``` + +--- + +## 4. Shared Infrastructure + +### 4.1 New Integration Providers + +Add to `IntegrationProvider` table: + +| provider_id | display_name | provider_type | Purpose | +|-------------|-------------|---------------|---------| +| `linkedin` | LinkedIn | social | OAuth tokens, Company Page ID | +| `twitter` | Twitter/X | social | OAuth tokens, API keys | +| `facebook` | Facebook/Meta | social | Page tokens, App credentials | +| `instagram` | Instagram | social | Linked via Facebook, IG Business Account ID | +| `tiktok` | TikTok | social | OAuth tokens, Creator credentials | +| `youtube` | YouTube | video | OAuth tokens, Channel ID | +| `elevenlabs` | ElevenLabs | ai | API key for TTS | +| `pexels` | Pexels | stock | API key for stock video/images | +| `pixabay` | Pixabay | stock | API key for stock video/images | +| `gpu_server` | Self-Hosted GPU | ai | URL + auth for custom GPU inference | + +### 4.2 New AIModelConfig Entries + +| model_name | model_type | provider | Purpose | +|-----------|-----------|----------|---------| +| `tts-1` | tts | openai | Standard TTS voice | +| `tts-1-hd` | tts | openai | High-quality TTS voice | +| `eleven-multilingual-v2` | tts | elevenlabs | Premium TTS (optional) | +| `coqui-xtts-v2` | tts | gpu_server | Self-hosted TTS | +| `piper-en-us` | tts | gpu_server | Self-hosted fast TTS | +| `stable-diffusion-xl` | image | gpu_server | Self-hosted image gen | +| `llama-3.1-70b` | text | gpu_server | Self-hosted text gen | + +### 4.3 File Storage + +New requirement: **video and audio file storage**. + +| Option | For Development | For Production | +|--------|----------------|----------------| +| **Local filesystem** | `MEDIA_ROOT/videos/`, `MEDIA_ROOT/audio/` | Not recommended | +| **S3-compatible** | MinIO (local Docker) | AWS S3 / DigitalOcean Spaces / Cloudflare R2 | +| **CDN** | Not needed | CloudFront / Bunny CDN (for video delivery) | + +**Recommended:** Add `django-storages` with S3 backend. All video/audio files stored in S3 with CDN for delivery. + +```python +# settings.py additions +DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage' +AWS_STORAGE_BUCKET_NAME = 'igny8-media' +AWS_S3_REGION_NAME = 'us-east-1' +MEDIA_URL = 'https://cdn.igny8.com/media/' +``` + +### 4.4 Extended Automation Pipeline + +Current 7-stage pipeline extended to 9: + +| Stage | Name | Module | New? | +|-------|------|--------|------| +| 1 | Keywords → Clusters | Planner | Existing | +| 2 | Clusters → Ideas | Planner | Existing | +| 3 | Ideas → Tasks | Writer | Existing | +| 4 | Tasks → Content | Writer | Existing | +| 5 | Content → Image Prompts | Writer | Existing | +| 6 | Image Prompts → Images | Writer | Existing | +| 7 | Review → Published | Publisher | Existing | +| **8** | **Published → Social Posts** | **Socializer** | **NEW** | +| **9** | **Published → Video** | **Video Creator** | **NEW** | + +New fields on `AutomationConfig`: + +```python +# Stage 8 - Socializer +stage_8_enabled = BooleanField(default=False) +stage_8_platforms = JSONField(default=list) # ["linkedin", "twitter", "facebook"] +stage_8_batch_size = IntegerField(default=5) +max_social_posts_per_run = IntegerField(default=20) + +# Stage 9 - Video Creator +stage_9_enabled = BooleanField(default=False) +stage_9_video_type = CharField(default='short') # short / long / both +stage_9_batch_size = IntegerField(default=2) +max_videos_per_run = IntegerField(default=5) +``` + +--- + +## 5. Self-Hosted GPU Option + +### 5.1 Architecture + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ IGNY8 APP SERVER │ +│ (Django + Celery Workers) │ +└───────────────────────────┬─────────────────────────────────────────────┘ + │ + ┌─────────────┼─────────────────────┐ + ▼ ▼ ▼ + ┌────────────┐ ┌─────────────┐ ┌────────────────┐ + │ OpenAI │ │ Anthropic │ │ GPU Server │ + │ Cloud API │ │ Cloud API │ │ (Self-Hosted) │ + │ │ │ │ │ │ + │ GPT-4o │ │ Claude 3.5 │ │ vLLM API │ + │ DALL-E 3 │ │ │ │ ├ Llama 3.1 │ + │ TTS-1 │ │ │ │ ├ Mistral │ + └────────────┘ └─────────────┘ │ ComfyUI API │ + │ ├ SDXL │ + │ ├ FLUX │ + │ Coqui TTS │ + │ ├ XTTS-v2 │ + └────────────────┘ +``` + +### 5.2 GPU Server Stack + +| Component | Purpose | GPU VRAM Needed | Docker Image | +|-----------|---------|----------------|--------------| +| **vLLM** | Text generation (Llama, Mistral) | 40 GB (70B) / 16 GB (7B) | `vllm/vllm-openai` | +| **ComfyUI** | Image generation (SDXL, FLUX) | 12-24 GB | `comfyanonymous/comfyui` | +| **Coqui TTS** | Voice synthesis (XTTS-v2) | 4-6 GB | `ghcr.io/coqui-ai/tts` | +| **Piper TTS** | Fast TTS (lower quality) | 1 GB (CPU ok) | `rhasspy/piper` | +| **Whisper** | Speech-to-text (for captioning) | 4-10 GB | `openai/whisper` | + +**Minimum GPU requirement:** 1× NVIDIA A100 80GB or 2× NVIDIA RTX 4090 (24GB each) +**Budget option:** 1× NVIDIA RTX 4090 running 7B text + SDXL + Coqui TTS + +### 5.3 GPU Server Integration with ModelRegistry + +```python +# New provider type in IntegrationProvider +IntegrationProvider.objects.create( + provider_id='gpu_server', + display_name='Self-Hosted GPU Server', + provider_type='ai', + api_endpoint='https://gpu.igny8.internal:8000', + api_key='internal-auth-token', + config={ + 'vllm_base_url': 'http://gpu:8000/v1', # OpenAI-compatible + 'comfyui_base_url': 'http://gpu:8188', # ComfyUI API + 'tts_base_url': 'http://gpu:5002', # Coqui TTS API + 'piper_base_url': 'http://gpu:10200', # Piper TTS API + } +) + +# ModelRegistry already supports this — just add AIModelConfig entries +# vLLM serves an OpenAI-compatible API, so existing OpenAI client code reuses +``` + +### 5.4 Fallback Strategy + +``` +Request → Check gpu_server availability + │ + ┌─────────┴──────────┐ + │ GPU Online │ GPU Offline/Overloaded + ▼ ▼ +Use self-hosted Fallback to cloud +(zero API cost) (OpenAI/Anthropic) +``` + +Implementation: `ModelRegistry.get_model()` checks `IntegrationProvider.is_active` and a health-check endpoint before routing. + +--- + +## 6. Database Schema + +### 6.1 Socializer Models + +**File:** `backend/igny8_core/business/socializer/models.py` + +```python +class SocialAccount(AccountBaseModel): + """Connected social media accounts""" + platform = CharField(max_length=20, choices=SOCIAL_PLATFORMS) + # linkedin, twitter, facebook, instagram, tiktok + platform_user_id = CharField(max_length=255) + platform_username = CharField(max_length=255) + platform_page_name = CharField(max_length=255, blank=True) + access_token = TextField() # Encrypted + refresh_token = TextField(blank=True) # Encrypted + token_expires_at = DateTimeField(null=True) + scopes = JSONField(default=list) + is_active = BooleanField(default=True) + connected_at = DateTimeField(auto_now_add=True) + last_used_at = DateTimeField(null=True) + metadata = JSONField(default=dict) # Platform-specific data + + +class SocialPost(SiteSectorBaseModel): + """A social media post for a specific platform""" + content = ForeignKey('writer.Content', on_delete=CASCADE, related_name='social_posts') + social_account = ForeignKey(SocialAccount, on_delete=CASCADE) + platform = CharField(max_length=20, choices=SOCIAL_PLATFORMS) + + # Content + post_text = TextField() + hashtags = JSONField(default=list) + link_url = URLField(blank=True) + call_to_action = CharField(max_length=255, blank=True) + post_type = CharField(max_length=20) + # text, image, carousel, video, thread, story + + # Media + media_urls = JSONField(default=list) # URLs of attached images/videos + thumbnail_url = URLField(blank=True) + + # Scheduling + status = CharField(max_length=20, choices=SOCIAL_POST_STATUS) + # draft, scheduled, publishing, published, failed, cancelled + scheduled_at = DateTimeField(null=True) + published_at = DateTimeField(null=True) + platform_post_id = CharField(max_length=255, blank=True) + platform_post_url = URLField(blank=True) + error_message = TextField(blank=True) + + # AI Generation + ai_model_used = CharField(max_length=100, blank=True) + credits_used = DecimalField(max_digits=10, decimal_places=2, default=0) + + created_at = DateTimeField(auto_now_add=True) + updated_at = DateTimeField(auto_now=True) + + +class SocialPostMetrics(models.Model): + """Engagement metrics for a published social post""" + social_post = OneToOneField(SocialPost, on_delete=CASCADE, related_name='metrics') + impressions = IntegerField(default=0) + reach = IntegerField(default=0) + likes = IntegerField(default=0) + comments = IntegerField(default=0) + shares = IntegerField(default=0) + clicks = IntegerField(default=0) + saves = IntegerField(default=0) + engagement_rate = DecimalField(max_digits=5, decimal_places=2, default=0) + last_fetched_at = DateTimeField(null=True) + raw_data = JSONField(default=dict) # Full API response + + +class SocialCalendarSlot(AccountBaseModel): + """Preferred posting times per platform per site""" + site = ForeignKey('auth.Site', on_delete=CASCADE) + platform = CharField(max_length=20, choices=SOCIAL_PLATFORMS) + day_of_week = IntegerField() # 0=Monday, 6=Sunday + time_slot = TimeField() + is_active = BooleanField(default=True) +``` + +### 6.2 Video Creator Models + +**File:** `backend/igny8_core/business/video/models.py` + +```python +class VideoProject(SiteSectorBaseModel): + """A video project derived from content""" + content = ForeignKey('writer.Content', on_delete=CASCADE, related_name='video_projects') + title = CharField(max_length=500) + description = TextField(blank=True) + + # Configuration + video_type = CharField(max_length=20, choices=VIDEO_TYPES) + # youtube_long, youtube_short, instagram_reel, tiktok + target_duration = IntegerField(help_text="Target duration in seconds") + voice_id = CharField(max_length=100, blank=True) + tts_provider = CharField(max_length=50, default='openai') + background_music = CharField(max_length=255, blank=True) + + # Script + script = JSONField(default=dict) # VideoScript structure + script_status = CharField(max_length=20, default='pending') + # pending, generating, ready, approved + + # Status + status = CharField(max_length=20, choices=VIDEO_STATUS) + # draft, script_ready, generating_audio, generating_visuals, + # composing, rendering, ready, published, failed + progress_pct = IntegerField(default=0) + error_message = TextField(blank=True) + + # Output + video_url = URLField(blank=True) # S3/CDN URL of final video + thumbnail_url = URLField(blank=True) + subtitle_url = URLField(blank=True) # SRT file URL + duration_seconds = IntegerField(null=True) + file_size_bytes = BigIntegerField(null=True) + resolution = CharField(max_length=20, blank=True) # 1920x1080 + + # AI/Credits + ai_model_used = CharField(max_length=100, blank=True) + tts_model_used = CharField(max_length=100, blank=True) + credits_used = DecimalField(max_digits=10, decimal_places=2, default=0) + + created_at = DateTimeField(auto_now_add=True) + updated_at = DateTimeField(auto_now=True) + + +class VideoAsset(models.Model): + """Individual assets used in a video project""" + project = ForeignKey(VideoProject, on_delete=CASCADE, related_name='assets') + asset_type = CharField(max_length=20) + # voiceover, image, stock_video, text_overlay, music, subtitle + section_id = IntegerField(null=True) # Maps to script section + file_url = URLField() + duration_seconds = DecimalField(max_digits=8, decimal_places=2, null=True) + position = IntegerField(default=0) # Order in sequence + metadata = JSONField(default=dict) + created_at = DateTimeField(auto_now_add=True) + + +class VideoPublishRecord(AccountBaseModel): + """Publishing record for a video to a platform""" + project = ForeignKey(VideoProject, on_delete=CASCADE, related_name='publish_records') + site = ForeignKey('auth.Site', on_delete=CASCADE) + platform = CharField(max_length=20) + # youtube, instagram, tiktok + platform_video_id = CharField(max_length=255, blank=True) + platform_url = URLField(blank=True) + status = CharField(max_length=20) + # pending, uploading, processing, published, failed + seo_title = CharField(max_length=500, blank=True) + seo_description = TextField(blank=True) + seo_tags = JSONField(default=list) + chapter_markers = JSONField(default=list) + published_at = DateTimeField(null=True) + error_message = TextField(blank=True) + metadata = JSONField(default=dict) + + +class VideoMetrics(models.Model): + """Video performance metrics from platform APIs""" + publish_record = OneToOneField(VideoPublishRecord, on_delete=CASCADE, related_name='metrics') + views = IntegerField(default=0) + likes = IntegerField(default=0) + comments = IntegerField(default=0) + shares = IntegerField(default=0) + watch_time_seconds = IntegerField(default=0) + average_view_duration = DecimalField(max_digits=8, decimal_places=2, default=0) + click_through_rate = DecimalField(max_digits=5, decimal_places=2, default=0) + last_fetched_at = DateTimeField(null=True) + raw_data = JSONField(default=dict) +``` + +### 6.3 Model Count Impact + +| Current (v1.8.4) | New Models | New Total | +|-------------------|-----------|-----------| +| 52+ models | +8 (Socializer: 4, Video: 4) | 60+ models | + +--- + +## 7. AI Functions + +### 7.1 New AI Functions + +**Location:** `backend/igny8_core/ai/functions/` + +| Function | File | Input | Output | Provider | +|----------|------|-------|--------|----------| +| `AdaptContentForSocial` | `social.py` | Content + platform | Platform-specific post text | GPT-4o-mini / Claude | +| `GenerateHashtags` | `social.py` | Content + platform | Hashtag list | GPT-4o-mini | +| `GenerateTwitterThread` | `social.py` | Content | Thread of tweets | GPT-4o | +| `GenerateCarouselSlides` | `social.py` | Content | Slide texts + prompts | GPT-4o | +| `GenerateVideoScript` | `video.py` | Content + video_type | VideoScript JSON | GPT-4o | +| `GenerateVideoSEO` | `video.py` | VideoScript + platform | Title, desc, tags | GPT-4o-mini | +| `GenerateVideoThumbnailPrompt` | `video.py` | VideoScript | Image prompt for thumbnail | GPT-4o-mini | + +### 7.2 New Non-AI Processing Functions + +| Function | File | Input | Output | Technology | +|----------|------|-------|--------|------------| +| `TextToSpeech` | `tts.py` | Script narration text | Audio MP3/WAV | OpenAI TTS / ElevenLabs / Coqui | +| `ResizeImageForPlatform` | `image_processing.py` | Image + platform specs | Resized image | Pillow | +| `RenderTextOverlay` | `image_processing.py` | Image + text | Image with text | Pillow | +| `ComposeVideo` | `video_composer.py` | Assets (audio, images, overlays) | MP4 video | FFmpeg / MoviePy | +| `GenerateSubtitles` | `subtitle.py` | TTS audio + script | SRT file | Whisper / text-alignment | +| `FetchStockVideo` | `stock.py` | Search query | Video clip URL | Pexels / Pixabay API | + +### 7.3 Registration in AIEngine + +```python +# ai/engine.py — extend AIEngine +class AIEngine: + # Existing + def auto_cluster(self, keywords): ... + def generate_ideas(self, cluster): ... + def generate_content(self, task): ... + def generate_images(self, content): ... + + # NEW — Socializer + def adapt_for_social(self, content, platform, social_account): ... + def generate_hashtags(self, content, platform): ... + def generate_thread(self, content): ... + def generate_carousel(self, content): ... + + # NEW — Video Creator + def generate_video_script(self, content, video_type): ... + def generate_voiceover(self, script, voice_id, provider): ... + def compose_video(self, project): ... + def generate_video_seo(self, project, platform): ... +``` + +--- + +## 8. API Endpoints + +### 8.1 Socializer Endpoints + +**Base:** `/api/v1/socializer/` + +| Method | Path | Handler | Purpose | +|--------|------|---------|---------| +| **Accounts** | | | | +| GET | `/accounts/` | `SocialAccountViewSet.list` | List connected social accounts | +| POST | `/accounts/connect/` | `SocialAccountViewSet.connect` | Start OAuth flow for platform | +| POST | `/accounts/{id}/disconnect/` | `SocialAccountViewSet.disconnect` | Disconnect account | +| POST | `/accounts/{id}/refresh/` | `SocialAccountViewSet.refresh_token` | Refresh OAuth token | +| **Posts** | | | | +| GET | `/posts/` | `SocialPostViewSet.list` | List social posts (filterable) | +| POST | `/posts/generate/` | `SocialPostViewSet.generate` | AI-generate posts for content | +| PUT | `/posts/{id}/` | `SocialPostViewSet.update` | Edit post text/media | +| POST | `/posts/{id}/schedule/` | `SocialPostViewSet.schedule` | Schedule for publishing | +| POST | `/posts/{id}/publish-now/` | `SocialPostViewSet.publish_now` | Publish immediately | +| POST | `/posts/{id}/cancel/` | `SocialPostViewSet.cancel` | Cancel scheduled post | +| POST | `/posts/bulk-generate/` | `SocialPostViewSet.bulk_generate` | Generate for multiple contents | +| POST | `/posts/bulk-schedule/` | `SocialPostViewSet.bulk_schedule` | Schedule multiple posts | +| **Calendar** | | | | +| GET | `/calendar/` | `SocialCalendarViewSet.list` | Get calendar events (date range) | +| GET | `/calendar/slots/` | `SocialCalendarViewSet.slots` | Get configured time slots | +| PUT | `/calendar/slots/` | `SocialCalendarViewSet.update_slots` | Update time slots | +| **Analytics** | | | | +| GET | `/analytics/overview/` | `SocialAnalyticsViewSet.overview` | Aggregated metrics | +| GET | `/analytics/platform/{platform}/` | `SocialAnalyticsViewSet.by_platform` | Per-platform metrics | +| GET | `/analytics/post/{id}/` | `SocialAnalyticsViewSet.by_post` | Single post metrics | + +### 8.2 Video Creator Endpoints + +**Base:** `/api/v1/video/` + +| Method | Path | Handler | Purpose | +|--------|------|---------|---------| +| **Projects** | | | | +| GET | `/projects/` | `VideoProjectViewSet.list` | List video projects | +| POST | `/projects/` | `VideoProjectViewSet.create` | Create project from content | +| GET | `/projects/{id}/` | `VideoProjectViewSet.retrieve` | Get project detail | +| DELETE | `/projects/{id}/` | `VideoProjectViewSet.destroy` | Delete project + assets | +| **Script** | | | | +| POST | `/projects/{id}/generate-script/` | `VideoProjectViewSet.generate_script` | AI-generate video script | +| PUT | `/projects/{id}/script/` | `VideoProjectViewSet.update_script` | Edit script manually | +| POST | `/projects/{id}/approve-script/` | `VideoProjectViewSet.approve_script` | Approve script for production | +| **Production** | | | | +| POST | `/projects/{id}/generate-voiceover/` | `VideoProjectViewSet.generate_voiceover` | Generate TTS audio | +| POST | `/projects/{id}/generate-visuals/` | `VideoProjectViewSet.generate_visuals` | Generate/collect visual assets | +| POST | `/projects/{id}/compose/` | `VideoProjectViewSet.compose` | Render final video | +| POST | `/projects/{id}/regenerate-section/{section_id}/` | `VideoProjectViewSet.regenerate_section` | Re-do a specific section | +| **Publishing** | | | | +| POST | `/projects/{id}/publish/` | `VideoProjectViewSet.publish` | Publish to platform(s) | +| GET | `/projects/{id}/publish-status/` | `VideoProjectViewSet.publish_status` | Check publishing status | +| **Assets & Voices** | | | | +| GET | `/voices/` | `VoiceViewSet.list` | List available TTS voices | +| GET | `/projects/{id}/assets/` | `VideoAssetViewSet.list` | List project assets | +| **Analytics** | | | | +| GET | `/analytics/overview/` | `VideoAnalyticsViewSet.overview` | Aggregated video metrics | +| GET | `/analytics/project/{id}/` | `VideoAnalyticsViewSet.by_project` | Single video metrics | + +--- + +## 9. Frontend Pages + +### 9.1 New Routes + +``` +├── SOCIALIZER +│ /socializer/dashboard → Social Dashboard (overview, metrics) +│ /socializer/accounts → Connected Accounts (manage OAuth) +│ /socializer/posts → Social Posts (list, filter, bulk actions) +│ /socializer/posts/:id → Post Detail (edit, preview per platform) +│ /socializer/calendar → Social Calendar (scheduling view) +│ /socializer/analytics → Social Analytics (engagement dashboard) +│ +├── VIDEO CREATOR +│ /video/dashboard → Video Dashboard (overview, metrics) +│ /video/projects → Video Projects (list) +│ /video/create → New Video (select content, configure) +│ /video/projects/:id → Video Project Detail (script, preview, publish) +│ /video/projects/:id/editor → Video Editor (timeline, preview) +│ /video/analytics → Video Analytics +``` + +### 9.2 New Zustand Stores + +| Store | File | Purpose | +|-------|------|---------| +| `socialStore` | `store/socialStore.ts` | Social accounts, posts, calendar state | +| `videoStore` | `store/videoStore.ts` | Video projects, rendering state, voices | + +### 9.3 New Components + +| Component | Purpose | +|-----------|---------| +| `SocialPostPreview` | Platform-mockup preview (shows how post will look) | +| `SocialCalendar` | Calendar widget for scheduling | +| `PlatformSelector` | Multi-select for target platforms | +| `VideoTimeline` | Timeline editor showing sections, audio, visuals | +| `VideoPlayer` | In-app video preview player | +| `ScriptEditor` | Editable script with section management | +| `VoiceSelector` | TTS voice selector with audio samples | + +### 9.4 Sidebar Navigation Updates + +``` +Current: New: +───────── ───────── +Dashboard Dashboard +Planner Planner + Keywords Keywords + Clusters Clusters + Ideas Ideas +Writer Writer + Tasks Tasks + Content Content + Images Images + Review Review +Automation Automation +Publisher Publisher + Socializer ← NEW + Dashboard + Posts + Calendar + Analytics + Video Creator ← NEW + Dashboard + Projects + Analytics +``` + +--- + +## 10. Celery Tasks + +### 10.1 Socializer Tasks + +| Task | Schedule | Purpose | +|------|----------|---------| +| `socializer.publish_scheduled_posts` | Every 5 min | Publish posts where `scheduled_at <= now` | +| `socializer.refresh_expired_tokens` | Every 6 hours | Refresh OAuth tokens nearing expiration | +| `socializer.fetch_post_metrics` | Every 6 hours | Fetch engagement metrics for published posts | +| `socializer.cleanup_failed_posts` | Daily at 3 AM | Retry or mark stale failed posts | + +### 10.2 Video Creator Tasks + +| Task | Schedule | Purpose | +|------|----------|---------| +| `video.process_video_project` | On-demand (triggered) | Full pipeline: script → TTS → visuals → compose | +| `video.generate_voiceover` | On-demand | Generate TTS audio for a project | +| `video.compose_video` | On-demand | Render final video (CPU/GPU intensive) | +| `video.publish_video` | On-demand | Upload video to platform | +| `video.fetch_video_metrics` | Every 12 hours | Fetch view/engagement metrics | +| `video.cleanup_temp_assets` | Daily at 4 AM | Remove temporary render files | + +### 10.3 Celery Worker Configuration + +Video rendering is CPU/GPU intensive — use a separate queue: + +```python +# celery.py additions +CELERY_TASK_ROUTES = { + 'igny8_core.business.video.tasks.compose_video': {'queue': 'video_render'}, + 'igny8_core.business.video.tasks.generate_voiceover': {'queue': 'video_render'}, + 'igny8_core.business.socializer.tasks.*': {'queue': 'social'}, + # Existing tasks stay on 'default' queue +} + +# docker-compose additions +# worker-video: +# command: celery -A igny8_core worker -Q video_render --concurrency=2 +# worker-social: +# command: celery -A igny8_core worker -Q social --concurrency=4 +``` + +--- + +## 11. Credit Cost Matrix + +### 11.1 Socializer Operations + +| Operation | AI Model | Est. Tokens | Credits | Notes | +|-----------|----------|------------|---------|-------| +| Adapt content for 1 platform | GPT-4o-mini | ~500 | 1 | Per platform | +| Generate hashtags | GPT-4o-mini | ~200 | 0.5 | Per platform | +| Generate Twitter thread | GPT-4o-mini | ~1000 | 2 | Full thread | +| Generate carousel slides | GPT-4o | ~2000 | 5 | Per carousel | +| Generate social image | Runware/DALL-E | N/A | 3-10 | Per image | +| **Full social suite (5 platforms)** | Mixed | ~5000 | **~15-25** | Per content item | + +### 11.2 Video Creator Operations + +| Operation | Model/Tool | Est. Usage | Credits | Notes | +|-----------|-----------|------------|---------|-------| +| Generate video script | GPT-4o | ~3000 tokens | 5 | Per script | +| Generate TTS voiceover | OpenAI TTS-1 | ~5000 chars | 10 | Per minute audio | +| Generate TTS voiceover (HD) | OpenAI TTS-1-HD | ~5000 chars | 20 | Per minute audio | +| Generate TTS (self-hosted) | Coqui XTTS | ~5000 chars | 2 | Reduced (own GPU) | +| Generate visual assets | Runware/DALL-E | ~5 images | 15-50 | Per video | +| Generate thumbnail | DALL-E/Runware | 1 image | 3-10 | Per video | +| Video composition | FFmpeg | CPU time | 5 | Per render | +| Video SEO metadata | GPT-4o-mini | ~500 tokens | 1 | Per platform | +| **Full short-form video** | Mixed | - | **~40-80** | 30-60s video | +| **Full long-form video** | Mixed | - | **~100-250** | 5-15 min video | + +### 11.3 Self-Hosted GPU Savings + +| Operation | Cloud Cost (credits) | Self-Hosted (credits) | Saving | +|-----------|---------------------|----------------------|--------| +| TTS voiceover (1 min) | 10-20 | 2 | 80-90% | +| Image generation (per image) | 5-10 | 1 | 80-90% | +| Text generation (per 1K tokens) | 1-3 | 0.5 | 50-83% | + +> Credits for self-hosted operations are lower because there's no API cost—only a flat infrastructure fee per account or globally amortized. + +--- + +## 12. Development Phases + +### Phase 1: Foundation (Weeks 1-3) + +| # | Task | Type | Effort | Dependencies | +|---|------|------|--------|-------------| +| 1.1 | Add `django-storages` + S3 config for media | Backend infra | 2 days | S3 bucket provisioned | +| 1.2 | Create `SocialAccount`, `SocialPost`, `SocialPostMetrics`, `SocialCalendarSlot` models + migrations | Backend models | 2 days | None | +| 1.3 | Create `VideoProject`, `VideoAsset`, `VideoPublishRecord`, `VideoMetrics` models + migrations | Backend models | 2 days | 1.1 | +| 1.4 | Add new `IntegrationProvider` entries (social, video, GPU) | Backend seed data | 0.5 days | None | +| 1.5 | Add new `AIModelConfig` entries (TTS models, GPU models) | Backend seed data | 0.5 days | None | +| 1.6 | Extend `AutomationConfig` with stage 8 & 9 fields + migration | Backend models | 1 day | None | +| 1.7 | Add FFmpeg + MoviePy + pydub to `requirements.txt` and Dockerfile | Backend infra | 1 day | None | +| 1.8 | Create backend module scaffolding (`modules/socializer/`, `modules/video/`, `business/socializer/`, `business/video/`) | Backend structure | 1 day | None | +| 1.9 | Admin registration for all new models | Backend admin | 1 day | 1.2, 1.3 | + +### Phase 2: Socializer — Core (Weeks 3-6) + +| # | Task | Type | Effort | Dependencies | +|---|------|------|--------|-------------| +| 2.1 | OAuth2 flow for LinkedIn, Twitter/X, Facebook/Instagram, TikTok | Backend auth | 5 days | Platform developer accounts | +| 2.2 | `AdaptContentForSocial` AI function + prompt templates | Backend AI | 2 days | None | +| 2.3 | `GenerateHashtags` AI function | Backend AI | 1 day | None | +| 2.4 | `GenerateTwitterThread` AI function | Backend AI | 1 day | None | +| 2.5 | Image resizing utility (Pillow) for all platform specs | Backend utility | 2 days | None | +| 2.6 | SocialPost CRUD ViewSet + serializers | Backend API | 2 days | 1.2 | +| 2.7 | Social post publishing service (per-platform SDK wrappers) | Backend service | 5 days | 2.1 | +| 2.8 | `publish_scheduled_posts` Celery task | Backend task | 1 day | 2.7 | +| 2.9 | `fetch_post_metrics` Celery task | Backend task | 2 days | 2.7 | +| 2.10 | Social Calendar API endpoints | Backend API | 2 days | 1.2 | +| 2.11 | Frontend: Social accounts page (OAuth connect/disconnect) | Frontend | 3 days | 2.1 | +| 2.12 | Frontend: Social posts page (list, generate, edit) | Frontend | 4 days | 2.6 | +| 2.13 | Frontend: Platform post preview components | Frontend | 3 days | 2.12 | +| 2.14 | Frontend: Social calendar page | Frontend | 4 days | 2.10 | +| 2.15 | Frontend: Social analytics dashboard | Frontend | 3 days | 2.9 | +| 2.16 | Frontend: Zustand social store | Frontend | 1 day | 2.11 | +| 2.17 | Notifications integration (publish success/failure) | Backend | 1 day | 2.7 | +| 2.18 | Automation stage 8 integration | Backend | 2 days | 2.7, 1.6 | + +### Phase 3: Video Creator — Core (Weeks 6-10) + +| # | Task | Type | Effort | Dependencies | +|---|------|------|--------|-------------| +| 3.1 | `GenerateVideoScript` AI function + prompt templates | Backend AI | 3 days | None | +| 3.2 | TTS integration service (OpenAI TTS API) | Backend service | 2 days | None | +| 3.3 | TTS integration service (ElevenLabs — optional) | Backend service | 2 days | ElevenLabs API key | +| 3.4 | Stock video/image fetcher (Pexels + Pixabay) | Backend service | 2 days | API keys | +| 3.5 | Visual asset assembler (collect images, stock, generate missing) | Backend service | 3 days | 3.4, existing ImageGen | +| 3.6 | Video composer service (FFmpeg + MoviePy pipeline) | Backend service | 5 days | 1.7 | +| 3.7 | Subtitle generator (SRT from script timestamps) | Backend utility | 1 day | None | +| 3.8 | Text overlay renderer (Pillow + Pygments for code) | Backend utility | 2 days | None | +| 3.9 | `GenerateVideoSEO` AI function | Backend AI | 1 day | None | +| 3.10 | Thumbnail generator (AI image + text overlay) | Backend service | 2 days | Existing ImageGen | +| 3.11 | VideoProject CRUD ViewSet + serializers | Backend API | 2 days | 1.3 | +| 3.12 | Video rendering Celery tasks (separate queue) | Backend task | 2 days | 3.6 | +| 3.13 | YouTube upload service (YouTube Data API v3) | Backend service | 3 days | YouTube API credentials | +| 3.14 | Instagram Reels upload service | Backend service | 2 days | 2.1 (Facebook OAuth) | +| 3.15 | TikTok video upload service | Backend service | 2 days | TikTok API credentials | +| 3.16 | Frontend: Video projects page (list, create) | Frontend | 3 days | 3.11 | +| 3.17 | Frontend: Script editor page | Frontend | 3 days | 3.1 | +| 3.18 | Frontend: Video timeline/preview page | Frontend | 5 days | 3.6 | +| 3.19 | Frontend: Voice selector component | Frontend | 2 days | 3.2 | +| 3.20 | Frontend: Video analytics page | Frontend | 2 days | 3.13 | +| 3.21 | Frontend: Zustand video store | Frontend | 1 day | 3.16 | +| 3.22 | Notifications integration | Backend | 1 day | 3.12 | +| 3.23 | Automation stage 9 integration | Backend | 2 days | 3.12, 1.6 | + +### Phase 4: Self-Hosted GPU (Weeks 10-12) — OPTIONAL + +| # | Task | Type | Effort | Dependencies | +|---|------|------|--------|-------------| +| 4.1 | GPU server Docker Compose (vLLM + ComfyUI + Coqui TTS) | DevOps | 3 days | GPU hardware | +| 4.2 | vLLM adapter in ModelRegistry (OpenAI-compatible client) | Backend | 2 days | 4.1 | +| 4.3 | ComfyUI adapter in image generation service | Backend | 2 days | 4.1 | +| 4.4 | Coqui TTS adapter in TTS service | Backend | 1 day | 4.1 | +| 4.5 | Health check + fallback logic in ModelRegistry | Backend | 2 days | 4.2, 4.3, 4.4 | +| 4.6 | GPU provider admin UI (settings, monitoring) | Frontend | 2 days | 4.5 | +| 4.7 | Load testing and optimization | QA | 3 days | All above | + +### Phase 5: Polish & Integration (Weeks 12-14) + +| # | Task | Type | Effort | Dependencies | +|---|------|------|--------|-------------| +| 5.1 | Module enable/disable (ModuleEnableSettings: `socializer_enabled`, `video_creator_enabled`) | Both | 1 day | All above | +| 5.2 | Dashboard widgets (Social Overview, Video Pipeline) | Frontend | 2 days | Phases 2-3 | +| 5.3 | Unified settings: social + video config in Site Settings | Both | 2 days | Phases 2-3 | +| 5.4 | End-to-end testing (full pipeline: content → social + video → published) | QA | 3 days | All above | +| 5.5 | Credit system testing (all new operations correctly deducted) | QA | 2 days | Phases 2-3 | +| 5.6 | Documentation (update all docs: ARCHITECTURE, MODULES, ENDPOINTS, MODELS) | Docs | 2 days | All above | +| 5.7 | Rate limiting and error handling for all platform APIs | Backend | 2 days | Phases 2-3 | +| 5.8 | Data migration for existing users (create default social calendar slots) | Backend | 1 day | Phase 2 | + +### Timeline Summary + +| Phase | Duration | Focus | +|-------|----------|-------| +| Phase 1 | Weeks 1-3 | Foundation: models, migrations, infra | +| Phase 2 | Weeks 3-6 | Socializer: OAuth, AI, scheduling, frontend | +| Phase 3 | Weeks 6-10 | Video Creator: script, TTS, compositor, frontend | +| Phase 4 | Weeks 10-12 | Self-hosted GPU (optional) | +| Phase 5 | Weeks 12-14 | Polish, integration, testing, docs | +| **Total** | **14 weeks** | **~70 working days** | + +--- + +## 13. Risk & Dependency Matrix + +### External Dependencies + +| Dependency | Risk Level | Mitigation | +|-----------|------------|------------| +| **Platform API access** (LinkedIn, Twitter, Meta, TikTok) | 🔴 High | Apply for developer access ASAP (can take 2-4 weeks for approval). Twitter/X API pricing changes are unpredictable. | +| **YouTube Data API** | 🟡 Medium | Well-documented, generous quota (10K units/day). Need Google Cloud project + OAuth consent screen review. | +| **ElevenLabs API** | 🟢 Low | Optional — OpenAI TTS is primary. ElevenLabs is premium add-on. | +| **FFmpeg on server** | 🟢 Low | Well-supported, already common in Docker images. Add to Dockerfile. | +| **S3-compatible storage** | 🟢 Low | Many providers (AWS, DO Spaces, Cloudflare R2). Straightforward setup. | +| **GPU hardware** (self-hosted) | 🟡 Medium | Optional phase. Cloud GPU rental (RunPod, Lambda) as interim. | + +### Technical Risks + +| Risk | Impact | Probability | Mitigation | +|------|--------|------------|------------| +| Video rendering is slow (minutes per video) | User experience | High | Separate Celery queue, progress tracking, async preview | +| Platform API rate limits | Publishing delays | Medium | Queue system with backoff, respect rate limits, spread over time | +| OAuth token expiration | Silent failures | Medium | Proactive token refresh task, user notification on failure | +| Video file sizes (100MB+) | Storage costs, slow uploads | High | Compression optimization, CDN delivery, tiered render quality | +| TTS quality inconsistency | User satisfaction | Medium | Voice preview before generation, multiple provider options | +| Platform API breaking changes | Feature outages | Medium | Abstract platform SDKs behind interface, monitor changelogs | + +### File & Directory Structure (New) + +``` +backend/igny8_core/ +├── modules/ +│ ├── socializer/ ← NEW +│ │ ├── __init__.py +│ │ ├── views.py # SocialPostViewSet, SocialAccountViewSet, etc. +│ │ ├── serializers.py +│ │ ├── urls.py +│ │ └── admin.py +│ ├── video/ ← NEW +│ │ ├── __init__.py +│ │ ├── views.py # VideoProjectViewSet, etc. +│ │ ├── serializers.py +│ │ ├── urls.py +│ │ └── admin.py +│ +├── business/ +│ ├── socializer/ ← NEW +│ │ ├── __init__.py +│ │ ├── models.py # SocialAccount, SocialPost, SocialPostMetrics, SocialCalendarSlot +│ │ ├── services/ +│ │ │ ├── adaptation_service.py # AI content adaptation +│ │ │ ├── publishing_service.py # Platform-specific publishers +│ │ │ ├── oauth_service.py # OAuth2 flows for each platform +│ │ │ ├── metrics_service.py # Fetch engagement metrics +│ │ │ └── calendar_service.py # Scheduling logic +│ │ ├── tasks.py # Celery tasks +│ │ └── platform_adapters/ +│ │ ├── base.py # Abstract platform adapter +│ │ ├── linkedin.py +│ │ ├── twitter.py +│ │ ├── facebook.py +│ │ ├── instagram.py +│ │ └── tiktok.py +│ ├── video/ ← NEW +│ │ ├── __init__.py +│ │ ├── models.py # VideoProject, VideoAsset, VideoPublishRecord, VideoMetrics +│ │ ├── services/ +│ │ │ ├── script_service.py # AI script generation +│ │ │ ├── tts_service.py # TTS provider abstraction +│ │ │ ├── asset_service.py # Visual asset collection +│ │ │ ├── composer_service.py # FFmpeg/MoviePy video composition +│ │ │ ├── subtitle_service.py # SRT generation +│ │ │ ├── publishing_service.py # YouTube/IG/TikTok upload +│ │ │ └── metrics_service.py # Fetch video metrics +│ │ ├── tasks.py # Celery tasks +│ │ └── platform_adapters/ +│ │ ├── base.py +│ │ ├── youtube.py +│ │ ├── instagram.py +│ │ └── tiktok.py +│ +├── ai/functions/ +│ ├── social.py ← NEW (AdaptContentForSocial, GenerateHashtags, etc.) +│ ├── video.py ← NEW (GenerateVideoScript, GenerateVideoSEO, etc.) +│ └── tts.py ← NEW (TextToSpeech provider abstraction) + +frontend/src/ +├── pages/ +│ ├── Socializer/ ← NEW +│ │ ├── SocialDashboard.tsx +│ │ ├── SocialAccounts.tsx +│ │ ├── SocialPosts.tsx +│ │ ├── SocialPostDetail.tsx +│ │ ├── SocialCalendar.tsx +│ │ └── SocialAnalytics.tsx +│ ├── Video/ ← NEW +│ │ ├── VideoDashboard.tsx +│ │ ├── VideoProjects.tsx +│ │ ├── VideoCreate.tsx +│ │ ├── VideoProjectDetail.tsx +│ │ ├── VideoEditor.tsx +│ │ └── VideoAnalytics.tsx +├── store/ +│ ├── socialStore.ts ← NEW +│ └── videoStore.ts ← NEW +├── api/ +│ ├── socializer.api.ts ← NEW +│ └── video.api.ts ← NEW +├── components/ +│ ├── socializer/ ← NEW +│ │ ├── SocialPostPreview.tsx +│ │ ├── PlatformSelector.tsx +│ │ └── SocialCalendarWidget.tsx +│ ├── video/ ← NEW +│ │ ├── VideoPlayer.tsx +│ │ ├── VideoTimeline.tsx +│ │ ├── ScriptEditor.tsx +│ │ └── VoiceSelector.tsx +│ └── dashboard/ +│ ├── SocialOverviewWidget.tsx ← NEW +│ └── VideoPipelineWidget.tsx ← NEW +``` + +--- + +## Appendix A: Python Dependencies (New) + +```txt +# requirements.txt additions + +# Media/Video +moviepy>=1.0.3 +pydub>=0.25.1 +Pillow>=10.0.0 # Already present +ffmpeg-python>=0.2.0 + +# Storage +django-storages[s3]>=1.14 +boto3>=1.34 + +# Social Platform SDKs +python-linkedin-v2>=0.9.0 +tweepy>=4.14 # Twitter/X +facebook-sdk>=3.1.0 +python-instagram>=1.3.2 +tiktok-api>=0.1.0 # Or direct HTTP client + +# TTS +openai>=1.0 # Already present (for TTS endpoints) +elevenlabs>=0.2.0 # Optional + +# Video +google-api-python-client>=2.0 # YouTube Data API +google-auth-oauthlib>=1.0 + +# Self-hosted (optional) +# vllm served externally — uses OpenAI-compatible client (already has openai package) +# coqui-tts served externally — HTTP API calls +``` + +## Appendix B: Environment Variables (New) + +```env +# Storage +AWS_ACCESS_KEY_ID=xxx +AWS_SECRET_ACCESS_KEY=xxx +AWS_STORAGE_BUCKET_NAME=igny8-media +AWS_S3_REGION_NAME=us-east-1 +MEDIA_CDN_URL=https://cdn.igny8.com + +# Social Platform OAuth Apps (stored in IntegrationProvider, but app-level config) +LINKEDIN_CLIENT_ID=xxx +LINKEDIN_CLIENT_SECRET=xxx +TWITTER_CLIENT_ID=xxx +TWITTER_CLIENT_SECRET=xxx +META_APP_ID=xxx +META_APP_SECRET=xxx +TIKTOK_CLIENT_KEY=xxx +TIKTOK_CLIENT_SECRET=xxx +GOOGLE_CLIENT_ID=xxx +GOOGLE_CLIENT_SECRET=xxx + +# Optional: Self-hosted GPU +GPU_SERVER_URL=https://gpu.igny8.internal:8000 +GPU_SERVER_AUTH_TOKEN=xxx + +# Optional: ElevenLabs +ELEVENLABS_API_KEY=xxx +``` + +## Appendix C: Docker Compose Additions + +```yaml +# docker-compose.app.yml additions + + worker-social: + build: ./backend + command: celery -A igny8_core worker -Q social --concurrency=4 -l info + depends_on: + - redis + - db + env_file: ./backend/.env + + worker-video: + build: + context: ./backend + dockerfile: Dockerfile.video # Includes FFmpeg + command: celery -A igny8_core worker -Q video_render --concurrency=2 -l info + depends_on: + - redis + - db + env_file: ./backend/.env + volumes: + - video_temp:/tmp/igny8_video # Temp render storage + +volumes: + video_temp: +``` + +```dockerfile +# backend/Dockerfile.video +FROM python:3.11-slim + +# Install FFmpeg +RUN apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/* + +COPY requirements.txt . +RUN pip install -r requirements.txt + +COPY . /app +WORKDIR /app +```