docs & ux improvmeents
This commit is contained in:
IGNY8 VPS (Salman)
337
docs/40-WORKFLOWS/CONTENT-PIPELINE.md
Normal file
337
docs/40-WORKFLOWS/CONTENT-PIPELINE.md
Normal file
@@ -0,0 +1,337 @@
|
||||
# Content Pipeline Workflow
|
||||
|
||||
**Last Verified:** December 25, 2025
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The IGNY8 content pipeline transforms raw keywords into published WordPress articles through a multi-stage workflow. This can run manually (step-by-step) or automatically via the Automation module.
|
||||
|
||||
---
|
||||
|
||||
## Pipeline Stages
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ CONTENT PIPELINE │
|
||||
├─────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ KEYWORDS │───►│ CLUSTERS │───►│ IDEAS │───►│ TASKS │ │
|
||||
│ │ Stage 1 │ │ Stage 2 │ │ Stage 3 │ │ Stage 4 │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ CONTENT │───►│ IMAGES │───►│ REVIEW │───►│ PUBLISH │ │
|
||||
│ │ Stage 5 │ │ Stage 6 │ │ Stage 7 │ │ Stage 8 │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stage 1: Keywords
|
||||
|
||||
**Module:** Planner
|
||||
**Status Values:** `pending`, `clustered`, `used`, `archived`
|
||||
|
||||
### Input
|
||||
- Seed keywords (manually added or from SEO tools)
|
||||
- Optional: search volume, difficulty, CPC data
|
||||
|
||||
### Process
|
||||
1. User adds keywords via UI or bulk import
|
||||
2. Keywords validated and deduplicated
|
||||
3. Assigned to site + sector
|
||||
|
||||
### Output
|
||||
- Keyword records in `Keyword` model
|
||||
- Status: `pending`
|
||||
|
||||
### Credit Usage
|
||||
- None (free operation)
|
||||
|
||||
---
|
||||
|
||||
## Stage 2: Clustering
|
||||
|
||||
**Module:** Planner
|
||||
**AI Function:** `AutoClusterKeywords`
|
||||
|
||||
### Input
|
||||
- Selected pending keywords (2-100)
|
||||
|
||||
### Process
|
||||
1. AI analyzes semantic relationships
|
||||
2. Groups keywords by topic/intent
|
||||
3. Creates cluster with name + description
|
||||
|
||||
### Output
|
||||
- `Cluster` records created
|
||||
- Keywords linked to clusters
|
||||
- Keyword status → `clustered`
|
||||
|
||||
### Credit Usage
|
||||
- 1 idea credit per clustering operation
|
||||
|
||||
---
|
||||
|
||||
## Stage 3: Ideas
|
||||
|
||||
**Module:** Planner
|
||||
**AI Function:** `GenerateContentIdeas`
|
||||
|
||||
### Input
|
||||
- Cluster with 2+ keywords
|
||||
|
||||
### Process
|
||||
1. AI generates content idea titles
|
||||
2. Creates brief description for each
|
||||
3. Suggests primary + secondary keywords
|
||||
|
||||
### Output
|
||||
- `ContentIdea` records created
|
||||
- Status: `pending`
|
||||
|
||||
### Credit Usage
|
||||
- 1 idea credit per idea generated
|
||||
|
||||
---
|
||||
|
||||
## Stage 4: Tasks
|
||||
|
||||
**Module:** Writer
|
||||
**Status Values:** `pending`, `in_progress`, `completed`, `cancelled`
|
||||
|
||||
### Input
|
||||
- Selected content ideas
|
||||
|
||||
### Process
|
||||
1. Ideas converted to tasks
|
||||
2. Task gets content brief + keywords
|
||||
3. Optional: set due date, assign user
|
||||
|
||||
### Output
|
||||
- `Task` records created
|
||||
- Status: `pending`
|
||||
- Ideas status → `used`
|
||||
|
||||
### Credit Usage
|
||||
- None (free operation)
|
||||
|
||||
---
|
||||
|
||||
## Stage 5: Content Generation
|
||||
|
||||
**Module:** Writer
|
||||
**AI Function:** `GenerateContent`
|
||||
|
||||
### Input
|
||||
- Task with title + keywords + brief
|
||||
|
||||
### Process
|
||||
1. AI generates full article content
|
||||
2. Creates structured HTML output
|
||||
3. Adds meta title + description
|
||||
|
||||
### Output
|
||||
- `Content` record created
|
||||
- Full HTML body
|
||||
- SEO metadata
|
||||
- Task status → `completed`
|
||||
|
||||
### Credit Usage
|
||||
- 1 content credit per generation
|
||||
|
||||
---
|
||||
|
||||
## Stage 6: Image Generation
|
||||
|
||||
**Module:** Writer
|
||||
**AI Function:** `GenerateImages`
|
||||
|
||||
### Input
|
||||
- Content record
|
||||
- Number of images (default: 1-3)
|
||||
|
||||
### Process
|
||||
1. AI analyzes content for image prompts
|
||||
2. Generates images via DALL-E/Runware
|
||||
3. Creates thumbnail + full versions
|
||||
|
||||
### Output
|
||||
- `ContentImage` records created
|
||||
- Image URLs + alt text
|
||||
- Featured image assigned
|
||||
|
||||
### Credit Usage
|
||||
- 1 image credit per image generated
|
||||
|
||||
---
|
||||
|
||||
## Stage 7: Review
|
||||
|
||||
**Module:** Writer
|
||||
**Status Values:** `draft`, `review`, `approved`, `published`
|
||||
|
||||
### Input
|
||||
- Generated content + images
|
||||
|
||||
### Process
|
||||
1. Content displayed in rich editor
|
||||
2. User reviews + edits if needed
|
||||
3. User approves for publishing
|
||||
|
||||
### Output
|
||||
- Content status → `approved`
|
||||
- Any manual edits saved
|
||||
|
||||
### Credit Usage
|
||||
- None (free operation)
|
||||
- Regeneration costs additional credits
|
||||
|
||||
---
|
||||
|
||||
## Stage 8: Publishing
|
||||
|
||||
**Module:** Publisher
|
||||
**Integration:** WordPress REST API
|
||||
|
||||
### Input
|
||||
- Approved content
|
||||
- WordPress integration credentials
|
||||
|
||||
### Process
|
||||
1. Content formatted for WordPress
|
||||
2. Images uploaded to WP media
|
||||
3. Post created with categories/tags
|
||||
4. Status set to draft/published
|
||||
|
||||
### Output
|
||||
- `PublishingRecord` created
|
||||
- WordPress post ID stored
|
||||
- Content status → `published`
|
||||
|
||||
### Credit Usage
|
||||
- None (free operation)
|
||||
|
||||
---
|
||||
|
||||
## Automation Mode
|
||||
|
||||
When running via Automation module:
|
||||
|
||||
1. **Configuration** - Set limits per stage
|
||||
2. **Execution** - Pipeline runs automatically
|
||||
3. **Pacing** - Configurable delays between operations
|
||||
4. **Monitoring** - Real-time status updates
|
||||
|
||||
### Automation Config Options
|
||||
|
||||
```
|
||||
Stage Limits:
|
||||
- clustering_limit: Max keywords to cluster
|
||||
- ideas_limit: Max ideas to generate
|
||||
- content_limit: Max content to generate
|
||||
- image_limit: Max images to generate
|
||||
- publish_limit: Max content to publish
|
||||
|
||||
Timing:
|
||||
- delay_between_operations: Seconds between API calls
|
||||
- max_runtime: Maximum run duration
|
||||
|
||||
Behavior:
|
||||
- auto_approve: Skip review stage
|
||||
- auto_publish: Publish immediately
|
||||
- stop_on_error: Halt pipeline on failure
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Diagram
|
||||
|
||||
```
|
||||
Seed Keywords
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Keyword │ status: pending
|
||||
│ Model │ belongs_to: site, sector
|
||||
└────────┬────────┘
|
||||
│ AI: AutoCluster
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Cluster │ keywords: [...]
|
||||
│ Model │ belongs_to: site, sector
|
||||
└────────┬────────┘
|
||||
│ AI: GenerateIdeas
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ ContentIdea │ primary_keyword, secondaries
|
||||
│ Model │ cluster_id, status
|
||||
└────────┬────────┘
|
||||
│ Convert to Task
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Task │ idea_id, brief
|
||||
│ Model │ assigned_to, status
|
||||
└────────┬────────┘
|
||||
│ AI: GenerateContent
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Content │ task_id, body, meta
|
||||
│ Model │ status: draft
|
||||
└────────┬────────┘
|
||||
│ AI: GenerateImages
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ ContentImage │ content_id, url
|
||||
│ Model │ alt_text, is_featured
|
||||
└────────┬────────┘
|
||||
│ User Review
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Content │ status: approved
|
||||
│ (updated) │
|
||||
└────────┬────────┘
|
||||
│ WordPress API
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│PublishingRecord │ content_id, wp_post_id
|
||||
│ Model │ status, published_at
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Stage | Common Errors | Recovery |
|
||||
|-------|---------------|----------|
|
||||
| Clustering | API timeout | Retry with smaller batch |
|
||||
| Ideas | API rate limit | Wait and retry |
|
||||
| Content | Insufficient credits | Add credits, retry |
|
||||
| Images | Image API failure | Skip images, continue |
|
||||
| Publish | WordPress auth fail | Reauth integration |
|
||||
|
||||
---
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Pipeline Stats (Dashboard)
|
||||
|
||||
- Keywords pending clustering
|
||||
- Ideas pending task creation
|
||||
- Tasks pending generation
|
||||
- Content pending review
|
||||
- Content pending publish
|
||||
|
||||
### Automation Logs
|
||||
|
||||
- Run ID + timestamps
|
||||
- Stage + item processed
|
||||
- Success/failure status
|
||||
- Credit deductions
|
||||
- Error messages
|
||||
Reference in New Issue
Block a user