salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs & ux improvmeents

Browse Source
This commit is contained in:
IGNY8 VPS (Salman)
2025-12-25 20:31:58 +00:00
parent 90e6e96b2b
commit 4bffede052
247 changed files with 6869 additions and 53517 deletions
Download Patch File Download Diff File Expand all files Collapse all files

148
docs/40-WORKFLOWS/AUTOMATION-WORKFLOW.md
View File

@@ -1,148 +0,0 @@
# Automation Module (Code-Sourced, Dec 2025)
Single canonical reference for IGNY8 automation (backend, frontend, and runtime behavior). Replaces all prior automation docs in this folder.
---
## 1) What Automation Does
- Runs the 7-stage pipeline across Planner/Writer:
1) Keywords → Clusters (AI)
2) Clusters → Ideas (AI)
3) Ideas → Tasks (Local)
4) Tasks → Content (AI)
5) Content → Image Prompts (AI)
6) Image Prompts → Images (AI)
7) Manual Review Gate (Manual)
- Per-site, per-account isolation. One run at a time per site; guarded by cache lock `automation_lock_{site_id}`.
- Scheduling via Celery beat (`automation.check_scheduled_automations`); execution via Celery tasks (`run_automation_task`, `resume_automation_task` / `continue_automation_task`).
---
## 2) Backend API (behavior + payloads)
Base: `/api/v1/automation/` (auth required; site must belong to user’s account).
- `GET config?site_id=`: returns or creates config with enable flag, frequency (`daily|weekly|monthly`), scheduled_time, stage_1..6 batch sizes, delays (`within_stage_delay`, `between_stage_delay`), last_run_at, next_run_at.
- `PUT update_config?site_id=`: same fields as above, updates in-place.
- `POST run_now?site_id=`: starts a manual run; enqueues `run_automation_task`. Fails if a run is already active or lock exists.
- `GET current_run?site_id=`: current running/paused run with status, current_stage, totals, and stage_1..7_result blobs (counts, credits, partial flags, skip reasons).
- `GET pipeline_overview?site_id=`: per-stage status counts and “pending” numbers for UI cards.
- `GET current_processing?site_id=&run_id=`: live processing snapshot for an active run; null if not running.
- `POST pause|resume|cancel?site_id=&run_id=`: pause after current item; resume from saved `current_stage`; cancel after current item and stamp cancelled_at/completed_at.
- `GET history?site_id=`: last 20 runs (id, status, trigger, timestamps, total_credits_used, current_stage).
- `GET logs?run_id=&lines=100`: tail of the per-run activity log written by AutomationLogger.
- `GET estimate?site_id=`: estimated_credits, current_balance, sufficient (balance >= 1.2x estimate).
Error behaviors:
- Missing site_id/run_id → 400.
- Site not in account → 404.
- Run not found → 404 on run-specific endpoints.
- Already running / lock held → 400 on run_now.
---
## 3) Data Model (runtime state)
- `AutomationConfig` (one per site): enable flag, schedule (frequency, time), batch sizes per stage (1–6), delays (within-stage, between-stage), last_run_at, next_run_at.
- `AutomationRun`: run_id, trigger_type (manual/scheduled), status (running/paused/cancelled/completed/failed), current_stage, timestamps (start/pause/resume/cancel/complete), total_credits_used, per-stage result JSON (stage_1_result … stage_7_result), error_message.
- Activity logs: one file per run via AutomationLogger; streamed through the `logs` endpoint.
---
## 4) How Execution Works (AutomationService)
- Start: grabs cache lock `automation_lock_{site_id}`, estimates credits, enforces 1.2x balance check, creates AutomationRun and log file.
- AI functions used: Stage 1 `AutoClusterFunction`; Stage 2 `GenerateIdeasFunction`; Stage 4 `GenerateContentFunction`; Stage 5 `GenerateImagePromptsFunction`; Stage 6 uses `process_image_generation_queue` (not the partial `generate_images` AI function).
- Stage flow (per code):
- Stage 1 Keywords → Clusters: require ≥5 keywords (validate_minimum_keywords); batch by config; AIEngine clustering; records keywords_processed, clusters_created, batches, credits, time; skips if insufficient keywords.
- Stage 2 Clusters → Ideas: batch by config; AIEngine ideas; records ideas_created.
- Stage 3 Ideas → Tasks: local conversion of queued ideas to tasks; batches by config; no AI.
- Stage 4 Tasks → Content: batch by config; AIEngine content; records content count + word totals.
- Stage 5 Content → Image Prompts: batch by config; AIEngine image-prompts into Images (featured + in-article).
- Stage 6 Image Prompts → Images: uses `process_image_generation_queue` with provider/model from IntegrationSettings; updates Images status.
- Stage 7 Manual Review Gate: marks ready-for-review counts; no AI.
- Control: each stage checks `_check_should_stop` (paused/cancelled); saves partial progress (counts, credits) before returning; resume continues from `current_stage`.
- Credits: upfront estimate check (1.2x buffer) before starting; AIEngine per-call pre-checks and post-SAVE deductions; `total_credits_used` accumulates.
- Locks: acquired on start; cleared on completion or failure; also cleared on fatal errors in tasks.
- Errors: any unhandled exception marks run failed, sets error_message, logs error, clears lock; pipeline_overview/history reflect status.
- Stage result fields (persisted):
- S1: keywords_processed, clusters_created, batches_run, credits_used, skipped/partial flags, time_elapsed.
- S2: clusters_processed, ideas_created, batches_run, credits_used.
- S3: ideas_processed, tasks_created, batches_run.
- S4: tasks_processed, content_created, total_words, batches_run, credits_used.
- S5: content_processed, prompts_created, batches_run, credits_used.
- S6: images_processed, images_generated, batches_run.
- S7: ready_for_review counts.
Batching & delays:
- Configurable per site; stage_1..6 batch sizes control how many items per batch; `within_stage_delay` pauses between batches; `between_stage_delay` between stages.
Scheduling:
- `check_scheduled_automations` runs hourly; respects frequency/time and last_run_at (~23h guard); skips if a run is active; sets next_run_at; starts `run_automation_task`.
Celery execution:
- `run_automation_task` runs stages 1→7 sequentially for a run_id; failures mark run failed and clear lock.
- `resume_automation_task` / `continue_automation_task` continue from saved `current_stage`.
- Workers need access to cache (locks) and IntegrationSettings (models/providers).
Image pipeline specifics:
- Stage 5 writes prompts to Images (featured + ordered in-article).
- Stage 6 generates images via queue helper; AI `generate_images` remains partial/broken and is not used by automation.
---
## 5) Frontend Behavior (AutomationPage)
- Route: `/automation`.
- What the user can do: run now, pause, resume, cancel; edit config (enable/schedule, batch sizes, delays); view activity log; view history; watch live processing card and pipeline cards update.
- Polling: every ~5s while a run is running/paused for current_run, pipeline_overview, metrics, current_processing; lighter polling when idle.
- Metrics: fetched via low-level endpoints (keywords/clusters/ideas/tasks/content/images) for authoritative counts.
- States shown: running, paused, cancelled, failed, completed; processing card shown when a run exists; pipeline cards use “pending” counts from pipeline_overview.
- Activity log: pulled from `logs` endpoint; shown in UI for live tailing.
---
## 6) Configuration & Dependencies
- Needs IntegrationSettings for AI models and image providers (OpenAI/runware).
- Requires Celery beat and workers; cache backend required for locks.
- Tenant scoping everywhere: site + account filtering on all automation queries.
---
## 7) Known Limitations and Gaps
- `generate_images` AI function is partial/broken; automation uses queue helper instead.
- Pause/Cancel stop after the current item; no mid-item abort.
- Batch defaults are conservative (e.g., stage_2=1, stage_4=1); tune per site for throughput.
- Stage 7 is manual; no automated review step.
- No automated test suite observed for automation pipeline (stage transitions, pause/resume/cancel, scheduling guards, credit estimation/deduction).
- Enhancements to consider: fix or replace `generate_images`; add mid-item abort; surface lock status/owner; broaden batch defaults after validation; add operator-facing doc in app; add tests.
---
## 8) Field/Behavior Quick Tables
### Pipeline “pending” definitions (pipeline_overview)
- Stage 1: Keywords with status `new`, cluster is null, not disabled.
- Stage 2: Clusters status `new`, not disabled, with no ideas.
- Stage 3: ContentIdeas status `new`.
- Stage 4: Tasks status `queued`.
- Stage 5: Content status `draft` with zero images.
- Stage 6: Images status `pending`.
- Stage 7: Content status `review`.
### Stage result fields (stored on AutomationRun)
- S1: keywords_processed, clusters_created, batches_run, credits_used, skipped, partial, time_elapsed.
- S2: clusters_processed, ideas_created, batches_run, credits_used.
- S3: ideas_processed, tasks_created, batches_run.
- S4: tasks_processed, content_created, total_words, batches_run, credits_used.
- S5: content_processed, prompts_created, batches_run, credits_used.
- S6: images_processed, images_generated, batches_run.
- S7: ready_for_review.
### Credit handling
- Pre-run: estimate_credits * 1.2 vs account.credits (fails if insufficient).
- Per AI call: AIEngine pre-check credits; post-SAVE deduction with cost/tokens tracked; total_credits_used aggregates deductions.
### Logging
- Per-run log file via AutomationLogger; accessed with `GET logs?run_id=&lines=`; includes stage start/progress/errors and batch info.
### Polling (frontend)
- Active run: ~5s cadence for current_run, pipeline_overview, metrics, current_processing, logs tail.
- Idle: lighter polling (current_run/pipeline_overview) to show readiness and pending counts.

73
docs/40-WORKFLOWS/CONTENT-LIFECYCLE.md
View File

@@ -1,73 +0,0 @@
# Content Lifecycle
## Purpose
Trace content from keyword intake through clustering, idea generation, task creation, AI writing, image generation, and publishing/sync, mapping to backend services and frontend pages.
## Code Locations (exact paths)
- Planner models: `backend/igny8_core/business/planning/models.py` (`Keywords`, `Clusters`, `ContentIdeas`)
- Writer models: `backend/igny8_core/business/content/models.py` (`Tasks`, `Content`, `Images`, `ContentTaxonomy`)
- ViewSets: `backend/igny8_core/modules/planner/views.py`; `backend/igny8_core/modules/writer/views.py`
- Automation orchestration: `backend/igny8_core/business/automation/services/automation_service.py`; Celery tasks `business/automation/tasks.py`
- WordPress sync/publish: `backend/igny8_core/business/integration/services/integration_service.py`, `modules/integration/views.py`
- Frontend pages: `frontend/src/pages/Planner/*`, `frontend/src/pages/Writer/*`, `frontend/src/pages/Automation/AutomationPage.tsx`, `frontend/src/pages/Sites/*`
## High-Level Responsibilities
- Collect and cluster keywords, generate ideas, create tasks, generate content and images, then publish/sync.
- Support manual steps via Planner/Writer pages and automated runs via Automation pipeline.
- Maintain site/sector scoping and credit enforcement throughout AI operations.
## Detailed Behavior
- Keywords → Clusters: `KeywordViewSet` (upload/create) and `ClusterViewSet` (manual or AI auto_cluster). Models inherit `SiteSectorBaseModel` for scoping.
- Clusters → ContentIdeas: `ContentIdeasViewSet` with AI generation endpoint; ideas track status.
- Ideas/Tasks: `Tasks` model holds brief/keywords/structure/status; created manually or by automation stage.
- Tasks → Content: Writer endpoints generate content (AI) and update `Content` records; statuses managed in writer views.
- Content → Images: `ImagesViewSet` handles image generation and storage; images linked to tasks/content.
- Publishing: Integration service sends content to WordPress via `/api/v1/integration` endpoints; WP plugin responds with IDs and syncs status back.
- Automation: `automation_service.run_automation` executes 7 stages with delays/retries/credit estimates; run status tracked in `AutomationRun`.
## Data Structures / Models Involved (no code)
- Planner: `Keywords`, `Clusters`, `ContentIdeas`.
- Writer: `Tasks`, `Content`, `Images`, `ContentTaxonomy`.
- Automation: `AutomationConfig`, `AutomationRun`.
- Integration: `SiteIntegration`.
## Execution Flow
- Manual path: upload keywords → cluster (auto/manual) → generate ideas → create tasks → generate content → generate images → publish to WP → WP syncs status back.
- Automated path: Automation pipeline stages perform the same transitions in sequence, logging progress and deducting credits.
- Each stage uses DRF viewsets; automation uses Celery tasks (`run_automation_task`, `resume_automation_task`) and logger for trace files.
## Cross-Module Interactions
- Automation consumes planner/writer endpoints and triggers integration publish.
- Integration relies on site’s WP API key (`Site.wp_api_key`) and `SiteIntegration` config for credentials/URLs.
- Billing deducts credits for AI operations through `CreditService`.
## State Transitions (if applicable)
- Keywords: status (active/inactive/used) per model.
- Clusters: status (active/archived).
- ContentIdeas: status (draft/approved/in_progress/completed).
- Tasks: status simplified (queued/completed) in Stage 1; content status (draft/published) and image status stored on related models.
- AutomationRun: status across run lifecycle (created/running/paused/completed/failed).
## Error Handling
- Viewsets rely on unified DRF error responses; automation logger records failures per stage; Celery retries configured where applicable.
- WordPress sync errors surfaced via integration service responses.
## Tenancy Rules
- All planner/writer/automation/integration models inherit account/site/sector via base models; viewsets filter by `request.account` and query params.
## Billing Rules (if applicable)
- AI clustering/idea/content/image generation deduct credits via `CreditService` cost map; automation blocks run if estimated credits insufficient.
## Background Tasks / Schedulers (if applicable)
- Automation scheduler: `check_scheduled_automations` enqueues runs; `run_automation_task`/`resume_automation_task` execute pipeline in Celery.
- AI tasks run async via Celery wrappers (`ai/tasks.py`).
## Key Design Considerations
- Single source of truth for tenancy via base models; prevents cross-tenant data exposure.
- Automation mirrors manual flow; manual pages remain usable when automation is paused or disabled.
- WordPress publish/sync uses API key auth to avoid storing user credentials beyond site integration config.
## How Developers Should Work With This Module
- When adding new content stages, update both manual endpoints and automation stages.
- Ensure every new action that calls AI is wired through `CreditService` and respects site/sector filters.
- For new publishing targets, extend `SiteIntegration` and integration service while keeping site/account scoping.

337
docs/40-WORKFLOWS/CONTENT-PIPELINE.md Normal file
View File

@@ -0,0 +1,337 @@
# Content Pipeline Workflow
**Last Verified:** December 25, 2025
---
## Overview
The IGNY8 content pipeline transforms raw keywords into published WordPress articles through a multi-stage workflow. This can run manually (step-by-step) or automatically via the Automation module.
---
## Pipeline Stages
```
┌─────────────────────────────────────────────────────────────────────────┐
│ CONTENT PIPELINE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ KEYWORDS │───►│ CLUSTERS │───►│ IDEAS │───►│ TASKS │ │
│ │ Stage 1 │ │ Stage 2 │ │ Stage 3 │ │ Stage 4 │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ CONTENT │───►│ IMAGES │───►│ REVIEW │───►│ PUBLISH │ │
│ │ Stage 5 │ │ Stage 6 │ │ Stage 7 │ │ Stage 8 │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
---
## Stage 1: Keywords
**Module:** Planner
**Status Values:** `pending`, `clustered`, `used`, `archived`
### Input
- Seed keywords (manually added or from SEO tools)
- Optional: search volume, difficulty, CPC data
### Process
1. User adds keywords via UI or bulk import
2. Keywords validated and deduplicated
3. Assigned to site + sector
### Output
- Keyword records in `Keyword` model
- Status: `pending`
### Credit Usage
- None (free operation)
---
## Stage 2: Clustering
**Module:** Planner
**AI Function:** `AutoClusterKeywords`
### Input
- Selected pending keywords (2-100)
### Process
1. AI analyzes semantic relationships
2. Groups keywords by topic/intent
3. Creates cluster with name + description
### Output
- `Cluster` records created
- Keywords linked to clusters
- Keyword status → `clustered`
### Credit Usage
- 1 idea credit per clustering operation
---
## Stage 3: Ideas
**Module:** Planner
**AI Function:** `GenerateContentIdeas`
### Input
- Cluster with 2+ keywords
### Process
1. AI generates content idea titles
2. Creates brief description for each
3. Suggests primary + secondary keywords
### Output
- `ContentIdea` records created
- Status: `pending`
### Credit Usage
- 1 idea credit per idea generated
---
## Stage 4: Tasks
**Module:** Writer
**Status Values:** `pending`, `in_progress`, `completed`, `cancelled`
### Input
- Selected content ideas
### Process
1. Ideas converted to tasks
2. Task gets content brief + keywords
3. Optional: set due date, assign user
### Output
- `Task` records created
- Status: `pending`
- Ideas status → `used`
### Credit Usage
- None (free operation)
---
## Stage 5: Content Generation
**Module:** Writer
**AI Function:** `GenerateContent`
### Input
- Task with title + keywords + brief
### Process
1. AI generates full article content
2. Creates structured HTML output
3. Adds meta title + description
### Output
- `Content` record created
- Full HTML body
- SEO metadata
- Task status → `completed`
### Credit Usage
- 1 content credit per generation
---
## Stage 6: Image Generation
**Module:** Writer
**AI Function:** `GenerateImages`
### Input
- Content record
- Number of images (default: 1-3)
### Process
1. AI analyzes content for image prompts
2. Generates images via DALL-E/Runware
3. Creates thumbnail + full versions
### Output
- `ContentImage` records created
- Image URLs + alt text
- Featured image assigned
### Credit Usage
- 1 image credit per image generated
---
## Stage 7: Review
**Module:** Writer
**Status Values:** `draft`, `review`, `approved`, `published`
### Input
- Generated content + images
### Process
1. Content displayed in rich editor
2. User reviews + edits if needed
3. User approves for publishing
### Output
- Content status → `approved`
- Any manual edits saved
### Credit Usage
- None (free operation)
- Regeneration costs additional credits
---
## Stage 8: Publishing
**Module:** Publisher
**Integration:** WordPress REST API
### Input
- Approved content
- WordPress integration credentials
### Process
1. Content formatted for WordPress
2. Images uploaded to WP media
3. Post created with categories/tags
4. Status set to draft/published
### Output
- `PublishingRecord` created
- WordPress post ID stored
- Content status → `published`
### Credit Usage
- None (free operation)
---
## Automation Mode
When running via Automation module:
1. **Configuration** - Set limits per stage
2. **Execution** - Pipeline runs automatically
3. **Pacing** - Configurable delays between operations
4. **Monitoring** - Real-time status updates
### Automation Config Options
```
Stage Limits:
- clustering_limit: Max keywords to cluster
- ideas_limit: Max ideas to generate
- content_limit: Max content to generate
- image_limit: Max images to generate
- publish_limit: Max content to publish
Timing:
- delay_between_operations: Seconds between API calls
- max_runtime: Maximum run duration
Behavior:
- auto_approve: Skip review stage
- auto_publish: Publish immediately
- stop_on_error: Halt pipeline on failure
```
---
## Data Flow Diagram
```
Seed Keywords
│
▼
┌─────────────────┐
│ Keyword │ status: pending
│ Model │ belongs_to: site, sector
└────────┬────────┘
│ AI: AutoCluster
▼
┌─────────────────┐
│ Cluster │ keywords: [...]
│ Model │ belongs_to: site, sector
└────────┬────────┘
│ AI: GenerateIdeas
▼
┌─────────────────┐
│ ContentIdea │ primary_keyword, secondaries
│ Model │ cluster_id, status
└────────┬────────┘
│ Convert to Task
▼
┌─────────────────┐
│ Task │ idea_id, brief
│ Model │ assigned_to, status
└────────┬────────┘
│ AI: GenerateContent
▼
┌─────────────────┐
│ Content │ task_id, body, meta
│ Model │ status: draft
└────────┬────────┘
│ AI: GenerateImages
▼
┌─────────────────┐
│ ContentImage │ content_id, url
│ Model │ alt_text, is_featured
└────────┬────────┘
│ User Review
▼
┌─────────────────┐
│ Content │ status: approved
│ (updated) │
└────────┬────────┘
│ WordPress API
▼
┌─────────────────┐
│PublishingRecord │ content_id, wp_post_id
│ Model │ status, published_at
└─────────────────┘
```
---
## Error Handling
| Stage | Common Errors | Recovery |
|-------|---------------|----------|
| Clustering | API timeout | Retry with smaller batch |
| Ideas | API rate limit | Wait and retry |
| Content | Insufficient credits | Add credits, retry |
| Images | Image API failure | Skip images, continue |
| Publish | WordPress auth fail | Reauth integration |
---
## Monitoring
### Pipeline Stats (Dashboard)
- Keywords pending clustering
- Ideas pending task creation
- Tasks pending generation
- Content pending review
- Content pending publish
### Automation Logs
- Run ID + timestamps
- Stage + item processed
- Success/failure status
- Credit deductions
- Error messages

399
docs/40-WORKFLOWS/CREDIT-SYSTEM.md Normal file
View File

@@ -0,0 +1,399 @@
# Usage & Content System
**Last Verified:** December 25, 2025
---
## Overview
IGNY8 uses a content-based allowance system. Users see "Content Pieces" while the backend tracks detailed credit consumption for internal cost monitoring.
**User View:** `47/50 Content Pieces Remaining`
**Backend Tracks:** Idea credits, content credits, image credits (for cost analysis)
---
## How It Works
### User-Facing (Simple)
| What Users See | Description |
|----------------|-------------|
| **Content Pieces** | Monthly allowance of pages/articles |
| **X/Y Remaining** | Used vs total for the month |
| **Upgrade Plan** | Get more content pieces |
### Backend (Detailed - Internal Only)
| Credit Type | Used For | Tracked For |
|-------------|----------|-------------|
| Idea Credits | Clustering, idea generation | Cost analysis |
| Content Credits | Article generation | Usage limits |
| Image Credits | Image generation | Cost analysis |
| Optimization Credits | SEO optimization (future) | Cost analysis |
---
## Plan Allowances
| Plan | Content Pieces/Month | Sites | Users |
|------|---------------------|-------|-------|
| Starter | 50 | 2 | 2 |
| Growth | 200 | 5 | 3 |
| Scale | 500 | Unlimited | 5 |
**Included with every content piece:**
- AI keyword clustering
- AI idea generation
- AI content writing (1000-2000 words)
- 3 images (1 featured + 2 in-article)
- Internal linking
- SEO optimization
- WordPress publishing
---
## Backend Soft Limits (Hidden from Users)
To prevent abuse, the backend enforces hidden limits:
| Limit | Starter | Growth | Scale |
|-------|---------|--------|-------|
| Keyword imports/mo | 500 | 2,000 | 5,000 |
| Clustering operations | 100 | 400 | 1,000 |
| Idea generations | 150 | 600 | 1,500 |
| Images generated | 200 | 800 | 2,000 |
If users hit these limits, they see: "You've reached your preparation limit for this month."
---
## Content Deduction Flow
```
┌─────────────────────────────────────────────────────────────────┐
│ CONTENT CREATION FLOW │
├─────────────────────────────────────────────────────────────────┤
│ │
│ User clicks Check Generate │
│ "Generate" ──────► Allowance ──────► Content │
│ │ │ │
│ │ Limit │ │
│ │ Reached ▼ │
│ ▼ Deduct 1 │
│ Show Upgrade Content │
│ Modal Piece │
│ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Operations & Credit Costs
### Planner Operations
| Operation | Credits | Type |
|-----------|---------|------|
| Add keyword | 0 | Free |
| Auto-cluster keywords | 1 | Idea |
| Generate content ideas | 1 per idea | Idea |
### Writer Operations
| Operation | Credits | Type |
|-----------|---------|------|
| Create task | 0 | Free |
| Generate content | 1 | Content |
| Regenerate content | 1 | Content |
| Generate images | 1 per image | Image |
| Regenerate image | 1 | Image |
| Edit content | 0 | Free |
### Automation Operations
| Operation | Credits | Type |
|-----------|---------|------|
| Run automation | Sum of operations | Mixed |
| Pause/resume | 0 | Free |
### Publisher Operations
| Operation | Credits | Type |
|-----------|---------|------|
| Publish to WordPress | 0 | Free |
| Sync from WordPress | 0 | Free |
### Optimizer Operations (Future)
| Operation | Credits | Type |
|-----------|---------|------|
| Optimize content | 1 | Optimization |
| Batch optimize | 1 per item | Optimization |
---
## Database Models
### CreditBalance
```python
class CreditBalance(models.Model):
account = models.ForeignKey(Account)
site = models.ForeignKey(Site, null=True)
idea_credits = models.IntegerField(default=0)
content_credits = models.IntegerField(default=0)
image_credits = models.IntegerField(default=0)
optimization_credits = models.IntegerField(default=0)
period_start = models.DateField()
period_end = models.DateField()
```
### CreditUsage
```python
class CreditUsage(models.Model):
account = models.ForeignKey(Account)
site = models.ForeignKey(Site, null=True)
user = models.ForeignKey(User)
credit_type = models.CharField() # idea/content/image/optimization
amount = models.IntegerField()
operation = models.CharField() # generate_content, etc.
created_at = models.DateTimeField(auto_now_add=True)
```
---
## Business Logic
### CreditService
Location: `backend/igny8_core/business/billing/services.py`
**Key Methods:**
```python
class CreditService:
def check_balance(account, site, credit_type, amount) -> bool:
"""Check if sufficient credits available"""
def deduct_credits(account, site, user, credit_type, amount, operation) -> bool:
"""Deduct credits and log usage"""
def get_balance(account, site) -> CreditBalance:
"""Get current balance"""
def reset_monthly_credits(account) -> None:
"""Reset credits at period start"""
def add_credits(account, credit_type, amount, reason) -> None:
"""Add credits (admin/purchase)"""
```
### Usage in AI Operations
```python
# In content generation service
def generate_content(task, user):
# 1. Check balance
if not credit_service.check_balance(
account=task.site.account,
site=task.site,
credit_type='content',
amount=1
):
raise InsufficientCreditsError()
# 2. Execute AI function
content = ai_engine.generate_content(task)
# 3. Deduct credits
credit_service.deduct_credits(
account=task.site.account,
site=task.site,
user=user,
credit_type='content',
amount=1,
operation='generate_content'
)
return content
```
---
## API Responses
### Successful Deduction
```json
{
"success": true,
"data": { ... },
"credits_used": {
"type": "content",
"amount": 1
},
"balance": {
"content_credits": 49
}
}
```
### Insufficient Credits
```json
HTTP 402 Payment Required
{
"success": false,
"error": "Insufficient content credits",
"code": "INSUFFICIENT_CREDITS",
"required": 1,
"available": 0
}
```
---
## Frontend Handling
### Balance Display
- Header shows credit balances
- Updates after each operation
- Warning at low balance (< 10%)
### Error Handling
```typescript
// In writer store
async generateContent(taskId: string) {
try {
const response = await api.generateContent(taskId);
// Update billing store
billingStore.fetchBalance();
return response;
} catch (error) {
if (error.code === 'INSUFFICIENT_CREDITS') {
// Show upgrade modal
uiStore.showUpgradeModal();
}
throw error;
}
}
```
---
## Usage Tracking
### Usage Summary Endpoint
```
GET /api/v1/billing/usage/summary/?period=month
```
Response:
```json
{
"period": "2025-01",
"usage": {
"idea_credits": 45,
"content_credits": 23,
"image_credits": 67,
"optimization_credits": 0
},
"by_operation": {
"auto_cluster": 12,
"generate_ideas": 33,
"generate_content": 23,
"generate_images": 67
}
}
```
---
## Automation Credit Estimation
Before running automation:
```
GET /api/v1/automation/estimate/?site_id=...
```
Response:
```json
{
"estimated_credits": {
"idea_credits": 25,
"content_credits": 10,
"image_credits": 30
},
"stages": {
"clustering": 5,
"ideas": 20,
"content": 10,
"images": 30
},
"has_sufficient_credits": true
}
```
---
## Credit Reset
Credits reset monthly based on billing cycle:
1. **Monthly Reset Job** runs at period end
2. **Unused credits** do not roll over
3. **Purchased credits** may have different expiry
### Celery Task
```python
@celery.task
def reset_monthly_credits():
"""
Run daily, resets credits for accounts
whose period_end is today
"""
today = date.today()
balances = CreditBalance.objects.filter(period_end=today)
for balance in balances:
credit_service.reset_monthly_credits(balance.account)
```
---
## Admin Operations
### Manual Credit Adjustment
Via Django Admin or API:
```python
# Add credits
credit_service.add_credits(
account=account,
credit_type='content',
amount=100,
reason='Customer support adjustment'
)
```
### Usage Audit
All credit changes logged in `CreditUsage` with:
- Timestamp
- User who triggered
- Operation type
- Amount deducted
- Related object ID

69
docs/40-WORKFLOWS/PAYMENT-WORKFLOW.md
View File

@@ -1,69 +0,0 @@
# Billing Lifecycle
## Purpose
Detail how credits, plans, subscriptions, invoices, and payments flow through the system, and where deductions/additions are enforced.
## Code Locations (exact paths)
- Models: `backend/igny8_core/business/billing/models.py` (`CreditTransaction`, `CreditUsageLog`, `CreditCostConfig`, `Invoice`, `Payment`, `CreditPackage`, `PaymentMethodConfig`, `AccountPaymentMethod`)
- Services: `backend/igny8_core/business/billing/services/credit_service.py`, `services/invoice_service.py`, `services/payment_service.py`
- Views/Endpoints: `backend/igny8_core/modules/billing/views.py`
- Frontend billing pages: `frontend/src/pages/account/{AccountBillingPage.tsx,PlansAndBillingPage.tsx,PurchaseCreditsPage.tsx}`
## High-Level Responsibilities
- Track credit balance, record usage, and enforce costs for AI/automation actions.
- Manage plans/subscriptions, generate invoices, and record payments.
- Expose account-scoped billing data (balance, usage, invoices, payments, packages, payment methods).
## Detailed Behavior
- Credit balance: stored on `Account.credits`; `CreditService` adds/deducts via `CreditTransaction` and logs usage in `CreditUsageLog`.
- Costs: read from `CreditCostConfig` or hardcoded `CREDIT_COSTS` fallbacks inside `credit_service.py`.
- Deductions: called from AI/automation flows to ensure sufficient credits; `InsufficientCreditsError` thrown on shortage.
- Plans/Subscriptions: `Plan` defines price, billing_cycle, included_credits, max sites/sectors/users; `Subscription` links account to plan. Plan selection handled via billing endpoints and frontend plan tabs.
- Invoices: `InvoiceService` creates invoices for subscriptions and credit packages; generates unique invoice numbers and line items.
- Payments: `PaymentService` records payments; manual payments stored with status (pending/processing/etc.); payment methods configured via `PaymentMethodConfig`/`AccountPaymentMethod`.
- Packages: `CreditPackage` defines purchasable credit bundles; purchasing triggers invoice/payment flows.
## Data Structures / Models Involved (no code)
- Billing: `CreditTransaction`, `CreditUsageLog`, `CreditCostConfig`, `Invoice`, `Payment`, `CreditPackage`, `PaymentMethodConfig`, `AccountPaymentMethod`.
- Plan: `Plan`, `Subscription` (in `auth/models.py`).
- Account linkage: `Account` holds `credits` and `plan`.
## Execution Flow
- Balance/Usage read: billing endpoints return account-scoped balance and usage summaries.
- AI/Automation call: service invokes `CreditService.deduct_credits_for_action` → creates `CreditTransaction` + `CreditUsageLog`.
- Purchase credits: frontend calls billing endpoint → `CreditService.add_credits` + `InvoiceService` for package; payment recorded via `PaymentService` (manual/other).
- Subscription change: endpoint updates `Subscription` + `Account.plan`, generates invoice as needed; payment recorded if required.
- Invoice download: `modules/billing/views.py` exposes invoice retrieval; frontend uses `downloadInvoicePDF`.
## Cross-Module Interactions
- Automation/writer AI calls depend on credit checks; insufficient balance blocks operations.
- Account settings feed invoice billing details; payments/invoices are tied to account.
## State Transitions (if applicable)
- Payment status: `pending` → `processing/pending_approval` → `succeeded/completed` or `failed/refunded/cancelled/void/uncollectible`.
- Subscription status: `active`/`cancelled` (persisted in `Subscription`); plan reference on account updates accordingly.
## Error Handling
- `InsufficientCreditsError` for low balance; surfaces to API as error response.
- Payment/Invoice service raise validation errors (e.g., already paid invoice).
## Tenancy Rules
- All billing queries filter by `request.user.account`; viewsets inherit from `AccountModelViewSet`.
- Payment methods, invoices, payments, credit transactions are account-scoped; no cross-tenant access.
## Billing Rules (if applicable)
- Costs derived from config constants or `CreditCostConfig`.
- Included credits and limits come from `Plan`; extra purchases via `CreditPackage`.
- Credits must be available before AI/automation runs proceed.
## Background Tasks / Schedulers (if applicable)
- None dedicated; billing operations are request-driven. Automation tasks carry account context to deduct credits within Celery runs.
## Key Design Considerations
- CreditService centralizes deductions to avoid scattered logic.
- Manual payment flow avoids storing sensitive data; relies on transaction_reference and admin approval.
## How Developers Should Work With This Module
- When adding new billable actions, route them through `CreditService` with a defined cost key.
- To change pricing, update `CreditCostConfig` data or the `CREDIT_COSTS` fallback map.
- Keep all billing endpoints inheriting `AccountModelViewSet` to maintain isolation.

77
docs/40-WORKFLOWS/SIGNUP-TO-ACTIVE.md
View File

@@ -1,77 +0,0 @@
# User Flow Overview
## Purpose
Describe the end-to-end journey from signup through planning, writing, automation, publishing, and billing within IGNY8, mapping each step to concrete backend/frontend modules so engineers can navigate without scanning code.
## Code Locations (exact paths)
- Auth & account: `backend/igny8_core/auth/{views.py,serializers.py,models.py,middleware.py}`, `frontend/src/store/authStore.ts`, routes in `frontend/src/App.tsx`
- Planner: `backend/igny8_core/modules/planner/views.py`, `business/planning/models.py`, `frontend/src/pages/Planner/*`
- Writer: `backend/igny8_core/modules/writer/views.py`, `business/content/models.py`, `frontend/src/pages/Writer/*`
- Automation: `backend/igny8_core/business/automation/{services/automation_service.py,tasks.py,views.py}`, `frontend/src/pages/Automation/AutomationPage.tsx`
- Publishing/Integration: `backend/igny8_core/business/integration/{models.py,services/integration_service.py}`, `modules/integration/views.py`, WordPress plugin endpoints consumed via `/api/v1/integration/`; `frontend/src/pages/Sites/*`
- Billing: `backend/igny8_core/business/billing/{models.py,services/credit_service.py,services/invoice_service.py,views.py}`, `frontend/src/pages/account/*`
- Tenancy enforcement: `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/api/base.py`
## High-Level Responsibilities
- Authenticate users and attach account context to every request.
- Let users plan keywords/clusters/ideas, create tasks and content, optionally automate all seven pipeline stages.
- Manage sites/sectors and connect WordPress for publishing/sync.
- Track credits, plans, subscriptions, invoices, and payments through billing services.
## Detailed Behavior
- Signup/Login: `auth/views.py` issues JWTs; `AccountContextMiddleware` sets `request.account`; frontend `authStore` stores tokens and refreshes them.
- Planning: `KeywordViewSet`, `ClusterViewSet`, `ContentIdeasViewSet` create/list/update scoped by `SiteSectorModelViewSet` filters; frontend planner pages drive these endpoints.
- Writing: `TasksViewSet`, `ContentViewSet`, `ImagesViewSet` manage tasks/content/images; AI generation endpoints trigger Celery-backed functions.
- Automation: `AutomationViewSet` + `automation_service.py` orchestrate 7 stages (keywords→clusters→ideas→tasks→content→image-prompts→images/manual review) with pause/resume and credit estimation; Celery tasks `run_automation_task`/`resume_automation_task` execute runs.
- Publishing/Integration: `IntegrationViewSet` handles WP connection tests and sync; WP plugin sends/receives data via API key; content publish endpoints in writer module update WordPress via integration services.
- Billing: credit balances and costs computed in `credit_service.py`; invoices via `invoice_service.py`; payments via `payment_service.py`; endpoints in `modules/billing/views.py` feed frontend billing pages.
## Data Structures / Models Involved (no code)
- Tenancy: `Account`, `Site`, `Sector` (plus `SiteUserAccess`), base models `AccountBaseModel`/`SiteSectorBaseModel`.
- Planning: `Keywords`, `Clusters`, `ContentIdeas`.
- Writing: `Tasks`, `Content`, `Images`.
- Automation: `AutomationConfig`, `AutomationRun`.
- Billing: `CreditTransaction`, `CreditUsageLog`, `CreditCostConfig`, `Invoice`, `Payment`, `CreditPackage`, `Subscription`, `Plan`.
- Integration: `SiteIntegration`.
## Execution Flow
1) User signs in → JWT stored → `AccountContextMiddleware` populates `request.account`.
2) Tenant creates sites/sectors (`SiteViewSet`), selects industry/sectors, optionally connects WordPress.
3) Planner: upload/enter keywords → cluster → generate ideas (manual or via automation/AI functions).
4) Writer: create tasks from ideas, generate content/images (manual endpoints or automation stages).
5) Publish: send to WordPress via integration endpoints or automation publish step; WP plugin syncs back statuses.
6) Automation (optional): run 7-stage pipeline via `AutomationViewSet` + Celery tasks; pause/resume supported.
7) Billing: credits deducted per AI/pipeline usage (`credit_service`), invoices/payments recorded; users view in billing pages.
## Cross-Module Interactions
- Automation invokes planner/writer endpoints/services and logs credit estimates.
- Billing hooks into automation and writer AI calls via credit deduction utilities.
- Integration uses site/account context and WP API key for sync; writer publish flows depend on integration configuration.
## State Transitions (if applicable)
- Account status (`active/suspended/trial/cancelled`) governs access; plan/subscription affect limits.
- Tasks/content/images status transitions handled in writer endpoints; automation run status moves through `created/running/paused/completed/failed`.
- Site activation via `set_active`; sectors toggled via `select_sectors`.
## Error Handling
- Unified API responses via `api/response.py`; DRF exception handler configured in settings; frontend shows toasts/banners.
- Automation errors logged via `automation_logger`; tasks wrapped in Celery with retries where defined.
## Tenancy Rules
- `AccountContextMiddleware` sets `request.account`; base viewsets filter by account/site/sector; API key auth sets account from `Site.wp_api_key`; public site slug reads limited to active sites.
## Billing Rules (if applicable)
- AI/automation uses `CreditService.deduct_credits_for_action`; credit balance required before runs; plans/subscriptions define included credits and sector/site limits.
## Background Tasks / Schedulers (if applicable)
- Celery tasks: automation runs (`run_automation_task`, `resume_automation_task`), scheduler `check_scheduled_automations`, AI functions (`ai/tasks.py`), publishing tasks (`tasks/wordpress_publishing.py`).
## Key Design Considerations
- Strong tenant isolation via middleware + filtered querysets.
- Automation relies on polling and Celery; frontend automation page polls every 5s during runs.
- Billing is authoritative in backend; frontend is read-only except initiating purchases/subscriptions.
## How Developers Should Work With This Module
- Trace user-visible flows by following router → page component → service call → backend viewset.
- When adding steps, ensure credit deductions and tenancy filters are applied in the corresponding backend service/viewset.
- Keep WP integration changes consistent with API key auth and `SiteIntegration` schema.

1367
docs/40-WORKFLOWS/WORDPRESS-SYNC.md
View File

File diff suppressed because it is too large Load Diff