salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

1

Browse Source
This commit is contained in:
IGNY8 VPS (Salman)
2026-03-23 17:20:51 +00:00
parent e78a41f11c
commit 0570052fec
21 changed files with 15889 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

552
v2/V2-Execution-Docs/02A-content-types-extension.md Normal file
View File

@@ -0,0 +1,552 @@
# IGNY8 Phase 2: Content Types Extension (02A)
## Type-Specific Content Generation Pipeline
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Content Model Today
The `Content` model (app_label=`writer`, db_table=`igny8_content`) already supports multiple content types via two fields:
```python
CONTENT_TYPE_CHOICES = [
('post', 'Post'),
('page', 'Page'),
('product', 'Product'),
('taxonomy', 'Taxonomy'),
]
CONTENT_STRUCTURE_CHOICES = [
('article', 'Article'),
('guide', 'Guide'),
('comparison', 'Comparison'),
('review', 'Review'),
('listicle', 'Listicle'),
('landing_page', 'Landing Page'),
('business_page', 'Business Page'),
('service_page', 'Service Page'),
('general', 'General'),
('cluster_hub', 'Cluster Hub'),
('product_page', 'Product Page'),
('category_archive', 'Category Archive'),
('tag_archive', 'Tag Archive'),
('attribute_archive', 'Attribute Archive'),
]
```
These same choices exist on `Tasks` (db_table=`igny8_tasks`) and `ContentIdeas` (db_table=`igny8_content_ideas`).
### What Works Now
- The `generate_content` AI function in `igny8_core/ai/functions/generate_content.py` produces blog-style articles regardless of the `content_type` field
- Only `content_type='post'` with `content_structure='article'` is actively used by the automation pipeline
- Pipeline Stage 4 (Tasks → Content) does not route to type-specific prompts
- No type-specific section layouts, presets, or schema generation exist
### Phase 1 Foundation Available
- `SAGCluster.cluster_type` choices: `product_category`, `condition_problem`, `feature`, `brand`, `informational`, `comparison`
- `SAGCluster.hub_page_type` (default `cluster_hub`) and `hub_page_structure` (guide_tutorial, product_comparison, category_overview, problem_solution, resource_library)
- 01E blueprint-aware pipeline provides `blueprint_context` to each stage with `cluster_type`, `content_structure`, and `content_type` fields
- 01E defines 12 content type → template key mappings (e.g., `sag_hub_guide`, `sag_blog_comparison`, `sag_product_page`)
### Gap
The template keys from 01E (`sag_hub_guide`, `sag_blog_comparison`, etc.) route to LLM prompt templates — but those templates don't exist yet. The actual type-specific prompt logic, section layouts, field schemas, and generation presets are what this doc delivers.
---
## 2. WHAT TO BUILD
### Overview
Extend the content generation pipeline to produce structurally different output for 6 content type categories. Each type gets:
- **Section layout templates** — defining the structure of the generated content (sections, order, constraints)
- **Type-specific AI prompts** — prompt templates tailored to the content type's purpose
- **Generation presets** — default word counts, image counts, schema types, tone
- **Structured data fields** — type-specific data (product specs, service steps, comparison items) stored in a JSONField
### Content Type Extensions
**Type 1: Pages** (content_type=`page`)
| Structure | Purpose | Section Layout |
|-----------|---------|----------------|
| `landing_page` | Conversion-focused landing page | Hero → Features → Benefits → Social Proof → CTA |
| `business_page` | About/company page | Company Intro → History → Values → Team → CTA |
| `service_page` | Service offering page | Problem → Solution → Process → Outcomes → Pricing → FAQ → CTA |
| `general` | Generic page | Intro → Body Sections → CTA |
| `cluster_hub` | Cluster hub/pillar page | Overview → Subtopic Grid → Detailed Guides → Internal Links → FAQ |
- **AI prompt tone:** Professional, conversion-focused, concise, benefit-driven
- **Schema:** WebPage (default), AboutPage, ContactPage as appropriate
- **Default word count:** 1,000–3,000 depending on structure
**Type 2: Products** (content_type=`product`)
| Structure | Purpose | Section Layout |
|-----------|---------|----------------|
| `product_page` | Single product review/description | Overview → Features → Specifications → Pros/Cons → Verdict |
| `comparison` | A vs B product comparison | Introduction → Feature Matrix → Category Breakdown → Verdict |
| `listicle` | Top products roundup | Introduction → Product Cards (ranked) → Comparison Table → Verdict |
- **Structured data fields:** `price_range` (JSON), `features` (JSON array), `specifications` (JSON), `pros_cons` (JSON {pros: [], cons: []})
- **AI prompt tone:** Feature-benefit mapping, objective analysis, buyer-persona aware
- **Schema:** Product with offers, aggregateRating, review
- **Image presets:** Product hero image, feature highlight visuals, comparison table graphic
- **Default word count:** 1,500–4,000
**Type 3: Services** (content_type=`page`, content_structure=`service_page`)
| Structure | Purpose | Section Layout |
|-----------|---------|----------------|
| `service_page` | Core service offering page | Problem → Solution → Process Steps → Outcomes → Pricing Tiers → FAQ → CTA |
| `landing_page` | Service-specific landing (area variant) | Hero → Service Intro → Benefits → Testimonials → Area Info → CTA |
- **Structured data fields:** `process_steps` (JSON array), `outcomes` (JSON array), `pricing_tiers` (JSON), `areas_served` (JSON array), `faqs` (JSON array of {question, answer})
- **AI prompt tone:** Problem-solution, trust-building, process explanation, CTA-heavy
- **Schema:** Service, ProfessionalService, LocalBusiness+hasOfferCatalog
- **Geographic targeting:** Generate area-specific variations from a base service page
- **Default word count:** 1,500–3,500
**Type 4: Company Pages** (content_type=`page`, content_structure=`business_page`)
| Structure | Purpose |
|-----------|---------|
| `business_page` | About company, team, careers, press pages |
- **AI prompt tone:** Brand voice emphasis, story-driven, credibility markers
- **Schema:** Organization, AboutPage
- **Default word count:** 800–2,000
**Type 5: Comparison Pages** (content_type=`post`, content_structure=`comparison`)
| Structure | Purpose | Section Layout |
|-----------|---------|----------------|
| `comparison` | A vs B analysis | Introduction → Feature Matrix → Category-by-Category → Winner → FAQ |
| `listicle` | Top N alternatives/roundup | Introduction → Comparison Table → Individual Reviews → Verdict |
- **Structured data fields:** `comparison_items` (JSON array of {name, features, pros, cons, rating, verdict})
- **AI prompt tone:** Objective analysis, data-driven, comparison tables, winner selection with reasoning
- **Schema:** Article with itemListElement (for multi-comparison)
- **Default word count:** 2,000–5,000
**Type 6: Brand Pages** (content_type=`page`, content_structure=`brand_page`)
| Structure | Purpose |
|-----------|---------|
| `brand_page` | Brand overview, review, or alternative recommendation page |
- **AI prompt tone:** Brand-focused, factual, company background included
- **Schema:** Organization (for overview), Article (for review)
- **Default word count:** 1,000–3,000
### Blueprint-to-Type Mapping
When the pipeline executes with SAG context (01E), the `SAGCluster.cluster_type` informs which content types to generate:
| SAGCluster.cluster_type | Primary content_type | Primary content_structure |
|-------------------------|---------------------|--------------------------|
| `informational` | post | article, guide |
| `comparison` | post | comparison, listicle |
| `product_category` | product | product_page, listicle |
| `feature` | page | landing_page, service_page |
| `brand` | page | brand_page |
| `condition_problem` | post | guide, article |
Hub pages for any cluster type use `content_type=page`, `content_structure=cluster_hub`.
---
## 3. DATA MODELS & APIs
### New Choices (add to existing choice lists)
```python
# Add to CONTENT_STRUCTURE_CHOICES on Content, Tasks, ContentIdeas
('brand_page', 'Brand Page'),
```
Note: Most structures already exist in the codebase. Only `brand_page` is new.
### Modified Models
**Content** (db_table=`igny8_content`) — add fields:
```python
sections = models.JSONField(
default=list, blank=True,
help_text="Ordered section data for structured content types"
)
# Structure: [{"type": "hero", "heading": "...", "body": "...", "cta": "..."}, ...]
structured_data = models.JSONField(
default=dict, blank=True,
help_text="Type-specific data: product specs, service steps, comparison items"
)
# Structure varies by content_type — see type definitions above
```
**Tasks** (db_table=`igny8_tasks`) — add fields:
```python
structure_template = models.JSONField(
default=dict, blank=True,
help_text="Section layout template for content generation"
)
# Structure: {"sections": [{"type": "hero", "required": true, "max_words": 200}, ...]}
type_presets = models.JSONField(
default=dict, blank=True,
help_text="Type-specific generation parameters"
)
# Structure: {"tone": "professional", "schema_type": "Product", "image_count": 3, ...}
```
### New Model
```python
class ContentTypeTemplate(AccountBaseModel):
"""
Defines section layout and AI prompt templates per content_type + content_structure combination.
System-provided defaults (is_system=True) plus per-account custom templates.
"""
content_type = models.CharField(max_length=50, choices=CONTENT_TYPE_CHOICES, db_index=True)
content_structure = models.CharField(max_length=50, choices=CONTENT_STRUCTURE_CHOICES, db_index=True)
template_name = models.CharField(max_length=200)
section_layout = models.JSONField(
default=list,
help_text="Ordered sections: [{type, label, required, max_words, guidance}]"
)
ai_prompt_template = models.TextField(
help_text="Base AI prompt template for this type. Supports {variables}."
)
default_schema_type = models.CharField(max_length=100, blank=True, default='')
default_word_count_min = models.IntegerField(default=1000)
default_word_count_max = models.IntegerField(default=3000)
default_image_count = models.IntegerField(default=2)
tone = models.CharField(max_length=100, default='professional')
is_system = models.BooleanField(default=False, help_text="System-provided template (not editable by users)")
is_active = models.BooleanField(default=True)
class Meta:
app_label = 'writer'
db_table = 'igny8_content_type_templates'
unique_together = [['account', 'content_type', 'content_structure', 'template_name']]
ordering = ['content_type', 'content_structure']
```
### Migration
```
igny8_core/migrations/XXXX_content_types_extension.py
```
Fields added:
1. `Content.sections` — JSONField, default=list
2. `Content.structured_data` — JSONField, default=dict
3. `Tasks.structure_template` — JSONField, default=dict
4. `Tasks.type_presets` — JSONField, default=dict
5. `ContentTypeTemplate` new table
6. Add `('brand_page', 'Brand Page')` to CONTENT_STRUCTURE_CHOICES
### API Endpoints
**Content Type Templates:**
```
GET /api/v1/writer/content-type-templates/ # List templates (filtered by content_type, content_structure)
POST /api/v1/writer/content-type-templates/ # Create custom template
GET /api/v1/writer/content-type-templates/{id}/ # Template detail
PUT /api/v1/writer/content-type-templates/{id}/ # Update custom template
DELETE /api/v1/writer/content-type-templates/{id}/ # Delete custom template (system templates cannot be deleted)
GET /api/v1/writer/content-type-templates/{id}/preview/ # Preview: generate sample section layout
```
**Modified Endpoints:**
```
POST /api/v1/writer/tasks/ # Extend: accepts structure_template + type_presets
POST /api/v1/writer/generate/ # Extend: routes to type-specific AI prompt
GET /api/v1/writer/content/{id}/ # Response now includes sections + structured_data
```
**ViewSet:**
```python
# igny8_core/modules/writer/views/content_type_template_views.py
class ContentTypeTemplateViewSet(AccountModelViewSet):
serializer_class = ContentTypeTemplateSerializer
queryset = ContentTypeTemplate.objects.all()
filterset_fields = ['content_type', 'content_structure', 'is_system', 'is_active']
def get_queryset(self):
# Return system templates + account's custom templates
return ContentTypeTemplate.objects.filter(
models.Q(is_system=True) | models.Q(account=self.request.account)
)
@action(detail=True, methods=['get'])
def preview(self, request, pk=None):
template = self.get_object()
# Return rendered section layout with placeholder content
return Response({'sections': template.section_layout})
```
**URL Registration:**
```python
# igny8_core/modules/writer/urls.py — add to existing router
router.register('content-type-templates', ContentTypeTemplateViewSet, basename='content-type-template')
```
### AI Function Extension
Extend `GenerateContentFunction` in `igny8_core/ai/functions/generate_content.py`:
```python
class GenerateContentFunction(BaseAIFunction):
def prepare(self, payload: dict, account=None) -> Any:
tasks = super().prepare(payload, account)
for task in tasks:
# Load template if not already set on task
if not task.structure_template:
template = ContentTypeTemplate.objects.filter(
models.Q(account=account) | models.Q(is_system=True),
content_type=task.content_type,
content_structure=task.content_structure,
is_active=True
).order_by('-is_system').first() # Prefer account-specific over system
if template:
task._template = template
return tasks
def build_prompt(self, data: Any, account=None) -> str:
task = data # Single task (batch_size=1)
template = getattr(task, '_template', None)
if template:
prompt = template.ai_prompt_template.format(
title=task.title,
keywords=task.keywords or '',
word_count=task.word_count,
content_type=task.content_type,
content_structure=task.content_structure,
sections=json.dumps(template.section_layout),
schema_type=template.default_schema_type,
tone=template.tone,
)
else:
# Fallback to existing blog-style prompt
prompt = self._build_default_prompt(task)
# Inject blueprint context if available (from 01E)
blueprint_context = getattr(task, 'blueprint_context', None)
if blueprint_context:
prompt += f"\n\nCluster Context: {json.dumps(blueprint_context)}"
return prompt
def parse_response(self, response: str, step_tracker=None) -> Any:
parsed = super().parse_response(response, step_tracker)
# Extract sections array and structured_data if present in AI response
if isinstance(parsed, dict):
parsed.setdefault('sections', [])
parsed.setdefault('structured_data', {})
return parsed
def save_output(self, parsed, original_data, account=None, **kwargs) -> Dict:
result = super().save_output(parsed, original_data, account, **kwargs)
# Persist sections and structured_data on Content
if 'content_id' in result:
Content.objects.filter(id=result['content_id']).update(
sections=parsed.get('sections', []),
structured_data=parsed.get('structured_data', {}),
)
return result
```
### System Template Seed Data
Create a management command to seed default templates:
```python
# igny8_core/management/commands/seed_content_type_templates.py
```
Seed templates (is_system=True, account=None):
| content_type | content_structure | template_name | default_schema_type | word_count_range |
|---|---|---|---|---|
| post | article | Blog Article | Article | 1000–2500 |
| post | guide | Comprehensive Guide | Article | 2000–4000 |
| post | comparison | Comparison Article | Article | 2000–5000 |
| post | review | Product Review | Review | 1500–3000 |
| post | listicle | Listicle | Article | 1500–3500 |
| page | landing_page | Landing Page | WebPage | 1000–2500 |
| page | business_page | Business Page | AboutPage | 800–2000 |
| page | service_page | Service Page | Service | 1500–3500 |
| page | general | General Page | WebPage | 500–2000 |
| page | cluster_hub | Cluster Hub Page | CollectionPage | 2000–5000 |
| page | brand_page | Brand Page | Organization | 1000–3000 |
| product | product_page | Product Page | Product | 1500–4000 |
| taxonomy | category_archive | Category Archive | CollectionPage | 500–1500 |
| taxonomy | tag_archive | Tag Archive | CollectionPage | 500–1500 |
| taxonomy | attribute_archive | Attribute Archive | CollectionPage | 500–1500 |
### Credit Costs
No change to existing credit costs. Type routing changes the prompt structure but not the token volume — still 4–8 credits per content generation via `CreditCostConfig(operation_type='content_generation')`.
---
## 4. IMPLEMENTATION STEPS
### Step 1: Add New Fields to Existing Models
```bash
# Add fields to Content, Tasks models
# Add 'brand_page' to CONTENT_STRUCTURE_CHOICES
```
Files to modify:
- `backend/igny8_core/business/content/models.py` — add `sections`, `structured_data` to Content; add `structure_template`, `type_presets` to Tasks; add `brand_page` to CONTENT_STRUCTURE_CHOICES
- `backend/igny8_core/business/planning/models.py` — add `brand_page` to ContentIdeas CONTENT_STRUCTURE_CHOICES
### Step 2: Create ContentTypeTemplate Model
```bash
# Create new model in writer app
```
File to create:
- `backend/igny8_core/business/content/content_type_template.py` (or add to existing `models.py`)
### Step 3: Create and Run Migration
```bash
cd /data/app/igny8/backend
python manage.py makemigrations --name content_types_extension
python manage.py migrate
```
### Step 4: Create Serializers
Files to create/modify:
- `backend/igny8_core/modules/writer/serializers/content_type_template_serializer.py`
- Modify existing content serializer to include `sections` and `structured_data`
- Modify existing task serializer to include `structure_template` and `type_presets`
### Step 5: Create ViewSet and URLs
Files to create:
- `backend/igny8_core/modules/writer/views/content_type_template_views.py`
- Modify `backend/igny8_core/modules/writer/urls.py` — register new ViewSet
### Step 6: Extend GenerateContentFunction
File to modify:
- `backend/igny8_core/ai/functions/generate_content.py` — add type routing logic
### Step 7: Create System Template Seed Command
File to create:
- `backend/igny8_core/management/commands/seed_content_type_templates.py`
```bash
python manage.py seed_content_type_templates
```
### Step 8: Create Type-Specific Prompt Templates
Files to create in `backend/igny8_core/ai/prompts/`:
- `page_prompts.py` — landing_page, business_page, service_page, general, cluster_hub
- `product_prompts.py` — product_page, product comparison, product roundup
- `comparison_prompts.py` — versus, multi-comparison, alternatives
- `brand_prompts.py` — brand overview, brand review
### Step 9: Frontend Updates
Files to create/modify in `frontend/src/`:
- `pages/Writer/ContentTypeTemplates.tsx` — template management page
- `stores/contentTypeTemplateStore.ts` — Zustand store
- `api/contentTypeTemplates.ts` — API client
- Modify task creation form to show type-specific template selection
- Modify content viewer to render section-based content
### Step 10: Tests
```bash
cd /data/app/igny8/backend
python manage.py test igny8_core.business.content.tests.test_content_type_templates
python manage.py test igny8_core.ai.tests.test_generate_content_types
```
---
## 5. ACCEPTANCE CRITERIA
- [ ] `Content.sections` and `Content.structured_data` fields exist and migrate successfully
- [ ] `Tasks.structure_template` and `Tasks.type_presets` fields exist and migrate successfully
- [ ] `ContentTypeTemplate` model created with all fields, `igny8_content_type_templates` table exists
- [ ] `brand_page` added to CONTENT_STRUCTURE_CHOICES on Content, Tasks, ContentIdeas
- [ ] 15 system templates seeded via management command
- [ ] `GET /api/v1/writer/content-type-templates/` returns system + account templates
- [ ] `POST /api/v1/writer/content-type-templates/` creates custom template (is_system=False)
- [ ] System templates cannot be modified or deleted by account users
- [ ] `GenerateContentFunction` routes to type-specific prompt when template exists
- [ ] `GenerateContentFunction` falls back to existing blog-style prompt when no template found
- [ ] Content generated with type template populates `sections` and `structured_data` fields
- [ ] Blueprint context from 01E is injected into prompts when SAG data available
- [ ] Frontend template management page allows CRUD on custom templates
- [ ] Task creation form shows template selection filtered by content_type + content_structure
- [ ] All new API endpoints require authentication and enforce account isolation
- [ ] Existing `content_type='post'` + `content_structure='article'` generation works unchanged (backward compatible)
---
## 6. CLAUDE CODE INSTRUCTIONS
### Execution Order
1. Read `backend/igny8_core/business/content/models.py` — understand existing Content and Tasks models
2. Read `backend/igny8_core/ai/functions/generate_content.py` — understand current generation logic
3. Read `backend/igny8_core/ai/base.py` and `backend/igny8_core/ai/registry.py` — understand base pattern
4. Add new fields + ContentTypeTemplate model
5. Create migration, run `makemigrations` + `migrate`
6. Build serializers, ViewSet, URLs
7. Extend GenerateContentFunction with type routing
8. Create seed command and run it
9. Build prompt templates per type
10. Build frontend components
### Key Constraints
- ALL primary keys are `BigAutoField` (integer). No UUIDs anywhere.
- Model class names are PLURAL where applicable: `Tasks`, `ContentIdeas`, `Clusters`, `Keywords`, `Images`. `Content` stays singular.
- Frontend files use `.tsx` extension, Zustand for state management, Vitest for testing
- Celery app name is `igny8_core`
- All new tables use `igny8_` prefix
- Follow existing ViewSet pattern: `AccountModelViewSet` for account-scoped resources
- Follow existing serializer pattern: `ModelSerializer` with explicit `fields`
- Follow existing URL pattern: register on `DefaultRouter` in `igny8_core/modules/writer/urls.py`
### File Tree (New/Modified)
```
backend/igny8_core/
├── business/content/
│ └── models.py # MODIFY: add sections, structured_data, structure_template, type_presets, brand_page choice
│ └── content_type_template.py # NEW: ContentTypeTemplate model (or add to models.py)
├── business/planning/
│ └── models.py # MODIFY: add brand_page to ContentIdeas CONTENT_STRUCTURE_CHOICES
├── ai/functions/
│ └── generate_content.py # MODIFY: type routing + template-aware prompt building
├── ai/prompts/
│ ├── page_prompts.py # NEW: landing page, business page, service page prompts
│ ├── product_prompts.py # NEW: product page, comparison, roundup prompts
│ ├── comparison_prompts.py # NEW: versus, multi-comparison prompts
│ └── brand_prompts.py # NEW: brand overview/review prompts
├── management/commands/
│ └── seed_content_type_templates.py # NEW: seed system templates
├── modules/writer/
│ ├── serializers/
│ │ └── content_type_template_serializer.py # NEW
│ ├── views/
│ │ └── content_type_template_views.py # NEW
│ └── urls.py # MODIFY: register new ViewSet
├── migrations/
│ └── XXXX_content_types_extension.py # NEW: auto-generated
frontend/src/
├── pages/Writer/
│ └── ContentTypeTemplates.tsx # NEW: template management
├── stores/
│ └── contentTypeTemplateStore.ts # NEW: Zustand store
├── api/
│ └── contentTypeTemplates.ts # NEW: API client
```
### Cross-References
- **01E** (blueprint-aware pipeline): blueprint_context injection, cluster_type → content_type mapping
- **01A** (SAG data foundation): SAGCluster.cluster_type, hub_page_type, hub_page_structure
- **02B** (taxonomy term content): uses content_type=taxonomy with ContentTypeTemplate
- **02G** (rich schema): schema_type from ContentTypeTemplate.default_schema_type
- **03B** (WP plugin connected): content sync maps content_type to WordPress post types

642
v2/V2-Execution-Docs/02B-taxonomy-term-content.md Normal file
View File

@@ -0,0 +1,642 @@
# IGNY8 Phase 2: Taxonomy Term Content (02B)
## Rich Content Generation for Taxonomy Terms
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Existing Taxonomy Infrastructure
The taxonomy system is partially built:
**ContentTaxonomy** (writer app, db_table=`igny8_content_taxonomies`):
- Stores taxonomy term references synced from WordPress
- Fields: `name`, `slug`, `external_id` (WP term ID), `taxonomy_type` (category/tag/product_cat/product_tag/attribute)
- No content generation — terms are metadata only (name + slug + external reference)
**ContentTaxonomyRelation** (writer app):
- Links `Content` to `ContentTaxonomy` (many-to-many through table)
- Allows assigning existing taxonomy terms to content pieces
**Content Model** (writer app, db_table=`igny8_content`):
- `content_type='taxonomy'` exists in CONTENT_TYPE_CHOICES but is unused by the generation pipeline
- CONTENT_STRUCTURE_CHOICES includes `category_archive`, `tag_archive`, `attribute_archive`
- `taxonomy_terms` ManyToManyField through ContentTaxonomyRelation
**Tasks Model** (writer app, db_table=`igny8_tasks`):
- `taxonomy_term` ForeignKey to ContentTaxonomy (nullable, db_column='taxonomy_id')
- Not used by automation pipeline — present as a field only
**SiteIntegration** (integration app):
- WordPress connections exist via `SiteIntegration` model
- `SyncEvent` logs operations but taxonomy sync is stubbed/incomplete
### What Doesn't Exist
- No content generation for taxonomy terms (categories, tags, attributes)
- No cluster mapping for taxonomy terms
- No WordPress → IGNY8 taxonomy sync (full fetch and reconcile)
- No IGNY8 → WordPress term content push
- No AI function for term content generation
- No admin interface for managing term-to-cluster mapping
---
## 2. WHAT TO BUILD
### Overview
Make taxonomy terms first-class SEO content pages by:
1. **Syncing terms from WordPress** — fetch all categories, tags, WooCommerce taxonomies
2. **Mapping terms to clusters** — automatic keyword-overlap + semantic matching
3. **Generating rich content** — AI-generated landing page content for each term
4. **Pushing content back** — sync generated content to WordPress term descriptions + meta
### Taxonomy Sync (WordPress → IGNY8)
Full bidirectional sync leveraging existing `SiteIntegration`:
**Fetch targets:**
- WordPress categories (`taxonomy_type='category'`)
- WordPress tags (`taxonomy_type='tag'`)
- WooCommerce product categories (`taxonomy_type='product_cat'`)
- WooCommerce product tags (`taxonomy_type='product_tag'`)
- WooCommerce product attributes (`taxonomy_type='attribute'`, e.g., `pa_color`, `pa_size`)
**Sync logic:**
1. Use existing `SiteIntegration.credentials_json` to authenticate WP REST API
2. Fetch all terms via `GET /wp-json/wp/v2/categories`, `/tags`, `/product_cat`, etc.
3. Reconcile: create new `ContentTaxonomy` records, update changed ones, flag deleted
4. Store parent/child hierarchy for categories
5. Log sync as `SyncEvent` with `event_type='metadata_sync'`
### Cluster Mapping Service
A shared service (`cluster_mapping_service.py`) that maps taxonomy terms to keyword clusters:
**Algorithm:**
| Factor | Weight | Method |
|--------|--------|--------|
| Keyword overlap | 40% | Compare term name + slug against cluster keywords |
| Semantic similarity | 40% | Embedding-based cosine similarity (term name vs cluster description) |
| Title match | 20% | Exact/partial match of term name in cluster name |
**Output per term:**
- `primary_cluster_id` — best-match cluster
- `secondary_cluster_ids` — additional related clusters (up to 3)
- `mapping_confidence` — 0.0 to 1.0 score
- `mapping_status`:
- `auto_mapped` (confidence ≥ 0.6) — assigned automatically
- `suggested` (confidence 0.3–0.6) — suggested for manual review
- `unmapped` (confidence < 0.3) — no good match found
### Term Content Generation
Each taxonomy term gets rich, SEO-optimized content:
**Generated sections:**
1. **H1 Title** — optimized for the term + primary cluster keywords
2. **Rich description** — 500–1,500 words covering the topic
3. **FAQ section** — 5–8 questions and answers
4. **Related terms** — links to sibling/child terms
5. **Meta title** — 50–60 characters
6. **Meta description** — 150–160 characters
**AI function:** `GenerateTermContentFunction(BaseAIFunction)`:
- Input: term name, taxonomy_type, assigned cluster keywords, existing content titles under term, parent/sibling terms for context
- Output: structured JSON with sections (intro, overview, FAQ, related)
- Uses `ContentTypeTemplate` from 02A where `content_type='taxonomy'`
### Term Content Sync (IGNY8 → WordPress)
Push generated content to WordPress:
- Custom WP REST endpoint: `POST /wp-json/igny8/v1/terms/{id}/content`
- Stores in WordPress term meta:
- `_igny8_term_content` — HTML content
- `_igny8_term_faq` — JSON FAQ array
- `_igny8_term_meta_title` — SEO title
- `_igny8_term_meta_description` — SEO description
- Updates native WordPress term description with the generated content
- Schema: CollectionPage with itemListElement for listed content
---
## 3. DATA MODELS & APIs
### Modified Models
**ContentTaxonomy** (db_table=`igny8_content_taxonomies`) — add fields:
```python
# Cluster mapping
cluster = models.ForeignKey(
'planner.Clusters', on_delete=models.SET_NULL,
null=True, blank=True, related_name='taxonomy_terms',
help_text="Primary cluster this term maps to"
)
secondary_cluster_ids = models.JSONField(
default=list, blank=True,
help_text="Additional related cluster IDs"
)
mapping_confidence = models.FloatField(
default=0.0,
help_text="Cluster mapping confidence score 0.0-1.0"
)
mapping_status = models.CharField(
max_length=20, default='unmapped',
choices=[
('auto_mapped', 'Auto Mapped'),
('manual_mapped', 'Manual Mapped'),
('suggested', 'Suggested'),
('unmapped', 'Unmapped'),
],
db_index=True
)
# Generated content
term_content = models.TextField(
blank=True, default='',
help_text="Generated rich HTML content for the term page"
)
term_faq = models.JSONField(
default=list, blank=True,
help_text="Generated FAQ: [{question, answer}]"
)
meta_title = models.CharField(max_length=255, blank=True, default='')
meta_description = models.TextField(blank=True, default='')
content_status = models.CharField(
max_length=20, default='none',
choices=[
('none', 'No Content'),
('generating', 'Generating'),
('generated', 'Generated'),
('published', 'Published to WP'),
],
db_index=True
)
# Hierarchy
parent_term = models.ForeignKey(
'self', on_delete=models.SET_NULL,
null=True, blank=True, related_name='child_terms'
)
term_count = models.IntegerField(
default=0,
help_text="Number of posts/products using this term"
)
# Sync tracking
last_synced_from_wp = models.DateTimeField(null=True, blank=True)
last_pushed_to_wp = models.DateTimeField(null=True, blank=True)
```
### New AI Function
```python
# igny8_core/ai/functions/generate_term_content.py
class GenerateTermContentFunction(BaseAIFunction):
"""Generate rich SEO content for taxonomy terms."""
def get_name(self) -> str:
return 'generate_term_content'
def get_metadata(self) -> Dict:
return {
'display_name': 'Generate Term Content',
'description': 'Generate rich landing page content for taxonomy terms',
'phases': {
'INIT': 'Initializing...',
'PREP': 'Loading term and cluster data...',
'AI_CALL': 'Generating term content...',
'PARSE': 'Parsing response...',
'SAVE': 'Saving term content...',
'DONE': 'Complete!'
}
}
def get_max_items(self) -> int:
return 10 # Process up to 10 terms per batch
def validate(self, payload: dict, account=None) -> Dict:
term_ids = payload.get('ids', [])
if not term_ids:
return {'valid': False, 'error': 'No term IDs provided'}
return {'valid': True}
def prepare(self, payload: dict, account=None) -> List:
term_ids = payload.get('ids', [])
terms = ContentTaxonomy.objects.filter(
id__in=term_ids,
account=account
).select_related('cluster', 'parent_term')
return list(terms)
def build_prompt(self, data: Any, account=None) -> str:
term = data # Single term
# Build context: cluster keywords, existing content, siblings
cluster_keywords = []
if term.cluster:
cluster_keywords = list(
term.cluster.keywords.values_list('keyword', flat=True)[:20]
)
sibling_terms = list(
ContentTaxonomy.objects.filter(
taxonomy_type=term.taxonomy_type,
site=term.site,
parent_term=term.parent_term
).exclude(id=term.id).values_list('name', flat=True)[:10]
)
# Use ContentTypeTemplate from 02A if available
# Fall back to default term prompt
return self._build_term_prompt(term, cluster_keywords, sibling_terms)
def parse_response(self, response: str, step_tracker=None) -> Dict:
# Parse structured JSON: {content_html, faq, meta_title, meta_description}
pass
def save_output(self, parsed, original_data, account=None, **kwargs) -> Dict:
term = original_data
term.term_content = parsed.get('content_html', '')
term.term_faq = parsed.get('faq', [])
term.meta_title = parsed.get('meta_title', '')
term.meta_description = parsed.get('meta_description', '')
term.content_status = 'generated'
term.save()
return {'count': 1, 'items_updated': [term.id]}
```
Register in `igny8_core/ai/registry.py`:
```python
register_lazy_function('generate_term_content', lambda: GenerateTermContentFunction)
```
### New Service
```python
# igny8_core/business/content/cluster_mapping_service.py
class ClusterMappingService:
"""Maps taxonomy terms to keyword clusters using multi-factor scoring."""
KEYWORD_OVERLAP_WEIGHT = 0.4
SEMANTIC_SIMILARITY_WEIGHT = 0.4
TITLE_MATCH_WEIGHT = 0.2
AUTO_MAP_THRESHOLD = 0.6
SUGGEST_THRESHOLD = 0.3
def map_terms_to_clusters(self, site_id: int, account_id: int) -> Dict:
"""
Map all unmapped ContentTaxonomy terms to Clusters for a site.
Returns: {mapped: int, suggested: int, unmapped: int}
"""
pass
def map_single_term(self, term: ContentTaxonomy) -> Dict:
"""
Map a single term. Returns:
{cluster_id, secondary_ids, confidence, status}
"""
pass
def _keyword_overlap_score(self, term_name: str, cluster_keywords: list) -> float:
pass
def _semantic_similarity_score(self, term_name: str, cluster_description: str) -> float:
pass
def _title_match_score(self, term_name: str, cluster_name: str) -> float:
pass
```
### New Celery Tasks
```python
# igny8_core/tasks/taxonomy_tasks.py
@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def sync_taxonomy_from_wordpress(self, site_id: int, account_id: int):
"""Fetch all taxonomy terms from WordPress and reconcile with ContentTaxonomy."""
pass
@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def map_terms_to_clusters(self, site_id: int, account_id: int):
"""Run cluster mapping on all unmapped terms for a site."""
pass
@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def generate_term_content_task(self, term_ids: list, account_id: int):
"""Generate content for a batch of taxonomy terms."""
pass
@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def push_term_content_to_wordpress(self, term_id: int, account_id: int):
"""Push generated term content to WordPress via REST API."""
pass
```
### Migration
```
igny8_core/migrations/XXXX_taxonomy_term_content.py
```
Fields added to `ContentTaxonomy`:
1. `cluster` — ForeignKey to Clusters (nullable)
2. `secondary_cluster_ids` — JSONField
3. `mapping_confidence` — FloatField
4. `mapping_status` — CharField
5. `term_content` — TextField
6. `term_faq` — JSONField
7. `meta_title` — CharField
8. `meta_description` — TextField
9. `content_status` — CharField
10. `parent_term` — ForeignKey to self (nullable)
11. `term_count` — IntegerField
12. `last_synced_from_wp` — DateTimeField (nullable)
13. `last_pushed_to_wp` — DateTimeField (nullable)
### API Endpoints
```
# Taxonomy Term Management
GET /api/v1/writer/taxonomy/terms/ # List terms with mapping status (filterable)
GET /api/v1/writer/taxonomy/terms/{id}/ # Term detail
GET /api/v1/writer/taxonomy/terms/unmapped/ # Terms needing cluster assignment
GET /api/v1/writer/taxonomy/terms/stats/ # Summary: mapped/unmapped/generated/published counts
# WordPress Sync
POST /api/v1/writer/taxonomy/terms/sync/ # Trigger WP → IGNY8 sync
GET /api/v1/writer/taxonomy/terms/sync/status/ # Last sync time + status
# Cluster Mapping
POST /api/v1/writer/taxonomy/terms/{id}/map-cluster/ # Manual cluster assignment
POST /api/v1/writer/taxonomy/terms/auto-map/ # Run auto-mapping for all unmapped terms
GET /api/v1/writer/taxonomy/terms/{id}/cluster-suggestions/ # Get AI cluster suggestions for a term
# Content Generation
POST /api/v1/writer/taxonomy/terms/create-tasks/ # Bulk create generation tasks for selected terms
POST /api/v1/writer/taxonomy/terms/{id}/generate/ # Generate content for single term
POST /api/v1/writer/taxonomy/terms/generate-bulk/ # Generate content for multiple terms
# Publishing to WordPress
POST /api/v1/writer/taxonomy/terms/{id}/publish/ # Push single term content to WP
POST /api/v1/writer/taxonomy/terms/publish-bulk/ # Push multiple terms to WP
```
**ViewSet:**
```python
# igny8_core/modules/writer/views/taxonomy_term_views.py
class TaxonomyTermViewSet(SiteSectorModelViewSet):
serializer_class = TaxonomyTermSerializer
queryset = ContentTaxonomy.objects.all()
filterset_fields = ['taxonomy_type', 'mapping_status', 'content_status', 'site']
@action(detail=False, methods=['get'])
def unmapped(self, request):
qs = self.get_queryset().filter(mapping_status='unmapped')
return self.paginate_and_respond(qs)
@action(detail=False, methods=['get'])
def stats(self, request):
site_id = request.query_params.get('site_id')
qs = self.get_queryset().filter(site_id=site_id)
return Response({
'total': qs.count(),
'mapped': qs.filter(mapping_status__in=['auto_mapped', 'manual_mapped']).count(),
'suggested': qs.filter(mapping_status='suggested').count(),
'unmapped': qs.filter(mapping_status='unmapped').count(),
'content_generated': qs.filter(content_status='generated').count(),
'content_published': qs.filter(content_status='published').count(),
})
@action(detail=False, methods=['post'])
def sync(self, request):
site_id = request.data.get('site_id')
sync_taxonomy_from_wordpress.delay(site_id, request.account.id)
return Response({'message': 'Taxonomy sync started'})
@action(detail=True, methods=['post'], url_path='map-cluster')
def map_cluster(self, request, pk=None):
term = self.get_object()
cluster_id = request.data.get('cluster_id')
term.cluster_id = cluster_id
term.mapping_status = 'manual_mapped'
term.mapping_confidence = 1.0
term.save()
return Response(TaxonomyTermSerializer(term).data)
@action(detail=False, methods=['post'], url_path='auto-map')
def auto_map(self, request):
site_id = request.data.get('site_id')
map_terms_to_clusters.delay(site_id, request.account.id)
return Response({'message': 'Auto-mapping started'})
@action(detail=True, methods=['get'], url_path='cluster-suggestions')
def cluster_suggestions(self, request, pk=None):
term = self.get_object()
service = ClusterMappingService()
suggestions = service.get_suggestions(term, top_n=5)
return Response({'suggestions': suggestions})
@action(detail=True, methods=['post'])
def generate(self, request, pk=None):
term = self.get_object()
generate_term_content_task.delay([term.id], request.account.id)
return Response({'message': 'Content generation started'})
@action(detail=True, methods=['post'])
def publish(self, request, pk=None):
term = self.get_object()
push_term_content_to_wordpress.delay(term.id, request.account.id)
return Response({'message': 'Publishing to WordPress started'})
```
**URL Registration:**
```python
# igny8_core/modules/writer/urls.py — add to existing router
router.register('taxonomy/terms', TaxonomyTermViewSet, basename='taxonomy-term')
```
### Credit Costs
| Operation | Credits | Via |
|-----------|---------|-----|
| Taxonomy sync (WordPress → IGNY8) | 1 per batch | CreditCostConfig: `taxonomy_sync` |
| Term content generation | 4–6 per term | CreditCostConfig: `term_content_generation` |
| Term content optimization | 3–5 per term | CreditCostConfig: `term_content_optimization` |
Add to `CreditCostConfig`:
```python
CreditCostConfig.objects.get_or_create(
operation_type='taxonomy_sync',
defaults={'display_name': 'Taxonomy Sync', 'base_credits': 1}
)
CreditCostConfig.objects.get_or_create(
operation_type='term_content_generation',
defaults={'display_name': 'Term Content Generation', 'base_credits': 5}
)
```
Add to `CreditUsageLog.OPERATION_TYPE_CHOICES`:
```python
('taxonomy_sync', 'Taxonomy Sync'),
('term_content_generation', 'Term Content Generation'),
```
---
## 4. IMPLEMENTATION STEPS
### Step 1: Add Fields to ContentTaxonomy
File to modify:
- `backend/igny8_core/business/content/models.py` (or wherever ContentTaxonomy is defined)
- Add all 13 new fields listed in migration section
### Step 2: Create and Run Migration
```bash
cd /data/app/igny8/backend
python manage.py makemigrations --name taxonomy_term_content
python manage.py migrate
```
### Step 3: Build ClusterMappingService
File to create:
- `backend/igny8_core/business/content/cluster_mapping_service.py`
### Step 4: Create GenerateTermContentFunction
File to create:
- `backend/igny8_core/ai/functions/generate_term_content.py`
Register in:
- `backend/igny8_core/ai/registry.py`
### Step 5: Create Celery Tasks
File to create:
- `backend/igny8_core/tasks/taxonomy_tasks.py`
Register in Celery beat schedule (optional — these are primarily on-demand):
- `sync_taxonomy_from_wordpress` — can be periodic (weekly) or on-demand
### Step 6: Add Credit Cost Entries
Add `taxonomy_sync` and `term_content_generation` to:
- `CreditCostConfig` seed data
- `CreditUsageLog.OPERATION_TYPE_CHOICES`
### Step 7: Build Serializers
File to create:
- `backend/igny8_core/modules/writer/serializers/taxonomy_term_serializer.py`
### Step 8: Build ViewSet and URLs
File to create:
- `backend/igny8_core/modules/writer/views/taxonomy_term_views.py`
Modify:
- `backend/igny8_core/modules/writer/urls.py`
### Step 9: Frontend
Files to create/modify in `frontend/src/`:
- `pages/Writer/TaxonomyTerms.tsx` — term list with mapping status indicators
- `pages/Writer/TaxonomyTermDetail.tsx` — term detail with generated content preview
- `components/Writer/ClusterMappingPanel.tsx` — cluster assignment/suggestion UI
- `stores/taxonomyTermStore.ts` — Zustand store
- `api/taxonomyTerms.ts` — API client
### Step 10: Tests
```bash
cd /data/app/igny8/backend
python manage.py test igny8_core.business.content.tests.test_cluster_mapping
python manage.py test igny8_core.ai.tests.test_generate_term_content
python manage.py test igny8_core.modules.writer.tests.test_taxonomy_term_views
```
---
## 5. ACCEPTANCE CRITERIA
- [ ] All 13 new fields on ContentTaxonomy migrate successfully
- [ ] `GenerateTermContentFunction` registered in AI function registry
- [ ] WordPress → IGNY8 taxonomy sync fetches categories, tags, WooCommerce taxonomies
- [ ] Sync creates/updates ContentTaxonomy records with correct taxonomy_type
- [ ] Parent/child hierarchy preserved via parent_term FK
- [ ] SyncEvent logged with event_type='metadata_sync' after each sync operation
- [ ] ClusterMappingService maps terms with confidence scores
- [ ] Terms with confidence ≥ 0.6 auto-mapped, 0.3–0.6 suggested, < 0.3 unmapped
- [ ] Manual cluster assignment sets mapping_status='manual_mapped' with confidence=1.0
- [ ] Term content generation produces: content_html, FAQ, meta_title, meta_description
- [ ] content_status transitions: none → generating → generated → published
- [ ] Publishing pushes content to WordPress via `POST /wp-json/igny8/v1/terms/{id}/content`
- [ ] All API endpoints require authentication and enforce account isolation
- [ ] Frontend term list shows mapping status badges (mapped/suggested/unmapped)
- [ ] Frontend supports manual cluster assignment from suggestion list
- [ ] Credit deduction works for taxonomy_sync and term_content_generation operations
- [ ] Backward compatible — existing ContentTaxonomy records unaffected (new fields nullable/defaulted)
---
## 6. CLAUDE CODE INSTRUCTIONS
### Execution Order
1. Read `backend/igny8_core/business/content/models.py` — find ContentTaxonomy and ContentTaxonomyRelation
2. Read `backend/igny8_core/business/planning/models.py` — understand Clusters model for FK reference
3. Read `backend/igny8_core/ai/functions/generate_content.py` — reference pattern for new AI function
4. Read `backend/igny8_core/ai/registry.py` — understand registration pattern
5. Add fields to ContentTaxonomy model
6. Create migration and run it
7. Build ClusterMappingService
8. Build GenerateTermContentFunction + register it
9. Build Celery tasks
10. Build serializers, ViewSet, URLs
11. Build frontend components
### Key Constraints
- ALL primary keys are `BigAutoField` (integer). No UUIDs.
- Model class names PLURAL: `Clusters`, `Keywords`, `Tasks`, `ContentIdeas`, `Images`. `Content` stays singular. `ContentTaxonomy` stays singular.
- Frontend: `.tsx` files, Zustand stores, Vitest testing
- Celery app name: `igny8_core`
- All new db_tables use `igny8_` prefix
- Follow existing ViewSet pattern: `SiteSectorModelViewSet` for site-scoped resources
- AI functions follow `BaseAIFunction` pattern with lazy registry
### File Tree (New/Modified)
```
backend/igny8_core/
├── business/content/
│ ├── models.py # MODIFY: add fields to ContentTaxonomy
│ └── cluster_mapping_service.py # NEW: ClusterMappingService
├── ai/functions/
│ └── generate_term_content.py # NEW: GenerateTermContentFunction
├── ai/
│ └── registry.py # MODIFY: register generate_term_content
├── tasks/
│ └── taxonomy_tasks.py # NEW: sync, map, generate, publish tasks
├── modules/writer/
│ ├── serializers/
│ │ └── taxonomy_term_serializer.py # NEW
│ ├── views/
│ │ └── taxonomy_term_views.py # NEW
│ └── urls.py # MODIFY: register taxonomy/terms route
├── migrations/
│ └── XXXX_taxonomy_term_content.py # NEW: auto-generated
frontend/src/
├── pages/Writer/
│ ├── TaxonomyTerms.tsx # NEW: term list page
│ └── TaxonomyTermDetail.tsx # NEW: term detail + content preview
├── components/Writer/
│ └── ClusterMappingPanel.tsx # NEW: cluster assignment UI
├── stores/
│ └── taxonomyTermStore.ts # NEW: Zustand store
├── api/
│ └── taxonomyTerms.ts # NEW: API client
```
### Cross-References
- **02A** (content types extension): ContentTypeTemplate for content_type='taxonomy' provides prompt template
- **01A** (SAG data foundation): SAGAttribute → taxonomy mapping context
- **01D** (setup wizard): wizard creates initial taxonomy plan used for cluster mapping
- **03B** (WP plugin connected): connected plugin receives term content via REST endpoint
- **03C** (companion theme): theme renders term landing pages using pushed content

774
v2/V2-Execution-Docs/02C-gsc-integration.md Normal file
View File

@@ -0,0 +1,774 @@
# IGNY8 Phase 2: GSC Integration (02C)
## Google Search Console — Indexing, Inspection & Analytics
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Existing Integration Infrastructure
- `SiteIntegration` model (db_table=`igny8_site_integrations`) stores WordPress connections with `platform='wordpress'`
- `SyncEvent` model (db_table=`igny8_sync_events`) logs publish/sync operations
- Integration app registered at `/api/v1/integration/`
- No Google API connections of any kind exist
- No OAuth 2.0 infrastructure for third-party APIs
- `IntegrationProvider` model supports `provider_type`: ai/payment/email/storage — no `search_engine` type yet
### Content Publishing Flow
- When `Content.site_status` changes to `published`, a `PublishingRecord` is created
- Content gets `external_url` and `external_id` after WordPress publish
- No automatic indexing request after publish
- No tracking of whether published URLs are indexed by Google
### What Doesn't Exist
- Google Search Console OAuth connection
- URL Inspection API integration
- Indexing queue with priority and quota management
- Search analytics data collection/dashboard
- Re-inspection scheduling
- Plugin-side index status display
- Any GSC-related models, endpoints, or tasks
---
## 2. WHAT TO BUILD
### Overview
Full Google Search Console integration with four capabilities:
1. **OAuth Connection** — connect GSC property via Google OAuth 2.0
2. **URL Inspection** — inspect URLs via Google's URL Inspection API (2K/day quota), auto-inspect after publish
3. **Indexing Queue** — priority-based queue with quota management and re-inspection scheduling
4. **Search Analytics** — fetch and cache search performance data (clicks, impressions, CTR, position)
### OAuth 2.0 Connection Flow
```
User clicks "Connect GSC" →
IGNY8 backend generates Google OAuth URL →
User authorizes in Google consent screen →
Google redirects to /api/v1/integration/gsc/callback/ →
Backend stores encrypted access_token + refresh_token →
Backend fetches user's GSC properties →
User selects property → GSCConnection created
```
**Google Cloud project requirements:**
- Search Console API enabled
- URL Inspection API enabled (separate from Search Console API)
- OAuth 2.0 client ID (Web application type)
- Scopes: `https://www.googleapis.com/auth/webmasters.readonly`, `https://www.googleapis.com/auth/indexing`
- Redirect URI: `https://{domain}/api/v1/integration/gsc/callback/`
**Token management:**
- Access tokens expire after 1 hour → background refresh via Celery task
- Refresh tokens stored encrypted in `GSCConnection.refresh_token`
- If refresh fails → set `GSCConnection.status='expired'`, notify user
### URL Inspection API
**Google API endpoint:**
```
POST https://searchconsole.googleapis.com/v1/urlInspection/index:inspect
Body: {"inspectionUrl": "https://example.com/page", "siteUrl": "sc-domain:example.com"}
```
**Response fields tracked:**
| Field | Type | Stored In |
|-------|------|-----------|
| `verdict` | PASS/PARTIAL/FAIL/NEUTRAL | URLInspectionRecord.verdict |
| `coverageState` | e.g., "Submitted and indexed" | URLInspectionRecord.coverage_state |
| `indexingState` | e.g., "INDEXING_ALLOWED" | URLInspectionRecord.indexing_state |
| `robotsTxtState` | e.g., "ALLOWED" | URLInspectionRecord.last_inspection_result (JSON) |
| `lastCrawlTime` | ISO datetime | URLInspectionRecord.last_crawled |
| Full response | JSON | URLInspectionRecord.last_inspection_result |
**Quota:** 2,000 inspections per day per GSC property (resets midnight Pacific Time)
**Rate limit:** 1 request per 3 seconds (safe limit: 600 per 30 minutes)
### Indexing Queue System
Priority-based queue that respects daily quota:
| Priority | Trigger | Description |
|----------|---------|-------------|
| 100 | Content published (auto) | Newly published content auto-queued |
| 90 | Re-inspection (auto) | Scheduled follow-up check |
| 70 | Manual inspect request | User requests specific URL inspection |
| 50 | Information query | Check status only, no submit |
| 30 | Scheduled bulk re-inspect | Periodic re-check of all URLs |
**Queue processing:**
- Celery task runs every 5 minutes
- Checks `GSCDailyQuota` for remaining capacity
- Processes items in priority order (highest first)
- Respects 1 request/3 second rate limit
- Status flow: `queued → processing → completed/failed/quota_exceeded`
### Re-Inspection Schedule
After initial inspection, automatically schedule follow-up checks:
| Check | Timing | Purpose |
|-------|--------|---------|
| Check 1 | 24 hours after submission | Quick verification |
| Check 2 | 3 days after | Give Google time to crawl |
| Check 3 | 6 days after | Most URLs indexed by now |
| Check 4 | 13 days after | Final automatic check |
If still not indexed after Check 4 → mark `status='manual_review'`, stop auto-checking.
### Search Analytics
**Google API endpoint:**
```
POST https://searchconsole.googleapis.com/v3/sites/{siteUrl}/searchAnalytics/query
Body: {
"startDate": "2025-01-01",
"endDate": "2025-03-23",
"dimensions": ["page", "query", "date"],
"rowLimit": 25000
}
```
**Metrics collected:** clicks, impressions, ctr, position
**Dimensions:** page, query (keyword), country, device, date
**Date range:** up to 16 months historical
**Caching:** Results cached in `GSCMetricsCache` with 24-hour TTL, refreshed daily via Celery
### Auto-Indexing After Publish
When `Content.site_status` changes to `'published'` and the content has an `external_url`:
1. Check if `GSCConnection` exists for the site with status='active'
2. Create or update `URLInspectionRecord` for the URL
3. Add to `IndexingQueue` with priority=100
4. If SAG data available: hub pages get inspected before supporting articles (blueprint-aware priority)
### Plugin-Side Status Sync
IGNY8 pushes index statuses to the WordPress plugin:
- Endpoint: `POST /wp-json/igny8/v1/gsc/status-sync`
- Payload: `{urls: [{url, status, verdict, last_inspected}]}`
- Plugin displays status badges on WP post list table:
- ⏳ `pending_inspection`
- ✓ `indexed`
- ✗ `not_indexed`
- ➡ `indexing_requested`
- 🚫 `error_noindex`
---
## 3. DATA MODELS & APIs
### New Models (integration app)
```python
class GSCConnection(AccountBaseModel):
"""Google Search Console OAuth connection per site."""
site = models.ForeignKey(
'igny8_core_auth.Site', on_delete=models.CASCADE,
related_name='gsc_connections'
)
google_email = models.CharField(max_length=255)
access_token = models.TextField(help_text="Encrypted OAuth access token")
refresh_token = models.TextField(help_text="Encrypted OAuth refresh token")
token_expiry = models.DateTimeField()
gsc_property_url = models.CharField(
max_length=500,
help_text="GSC property URL, e.g., sc-domain:example.com"
)
status = models.CharField(
max_length=20, default='active',
choices=[
('active', 'Active'),
('expired', 'Token Expired'),
('revoked', 'Access Revoked'),
],
db_index=True
)
class Meta:
app_label = 'integration'
db_table = 'igny8_gsc_connections'
unique_together = [['site', 'gsc_property_url']]
class URLInspectionRecord(SiteSectorBaseModel):
"""Tracks URL inspection history and indexing status."""
url = models.URLField(max_length=2000, db_index=True)
content = models.ForeignKey(
'writer.Content', on_delete=models.SET_NULL,
null=True, blank=True, related_name='inspection_records',
help_text="Linked IGNY8 content (null for external/non-IGNY8 URLs)"
)
last_inspection_result = models.JSONField(
default=dict, help_text="Full Google API response"
)
verdict = models.CharField(
max_length=20, blank=True, default='',
help_text="PASS/PARTIAL/FAIL/NEUTRAL"
)
coverage_state = models.CharField(max_length=100, blank=True, default='')
indexing_state = models.CharField(max_length=100, blank=True, default='')
last_crawled = models.DateTimeField(null=True, blank=True)
last_inspected = models.DateTimeField(null=True, blank=True)
inspection_count = models.IntegerField(default=0)
next_inspection = models.DateTimeField(
null=True, blank=True,
help_text="Scheduled next re-inspection datetime"
)
status = models.CharField(
max_length=30, default='pending_inspection',
choices=[
('pending_inspection', 'Pending Inspection'),
('indexed', 'Indexed'),
('not_indexed', 'Not Indexed'),
('indexing_requested', 'Indexing Requested'),
('error_noindex', 'Error / No Index'),
('manual_review', 'Manual Review Needed'),
],
db_index=True
)
class Meta:
app_label = 'integration'
db_table = 'igny8_url_inspection_records'
unique_together = [['site', 'url']]
ordering = ['-last_inspected']
class IndexingQueue(SiteSectorBaseModel):
"""Priority queue for URL inspection API requests."""
url = models.URLField(max_length=2000)
url_inspection_record = models.ForeignKey(
URLInspectionRecord, on_delete=models.SET_NULL,
null=True, blank=True, related_name='queue_entries'
)
priority = models.IntegerField(
default=50, db_index=True,
help_text="100=auto-publish, 90=re-inspect, 70=manual, 50=info, 30=bulk"
)
status = models.CharField(
max_length=20, default='queued',
choices=[
('queued', 'Queued'),
('processing', 'Processing'),
('completed', 'Completed'),
('failed', 'Failed'),
('quota_exceeded', 'Quota Exceeded'),
],
db_index=True
)
date_added = models.DateTimeField(auto_now_add=True)
date_processed = models.DateTimeField(null=True, blank=True)
error_message = models.TextField(blank=True, default='')
class Meta:
app_label = 'integration'
db_table = 'igny8_indexing_queue'
ordering = ['-priority', 'date_added']
class GSCMetricsCache(SiteSectorBaseModel):
"""Cached search analytics data from GSC API."""
metric_type = models.CharField(
max_length=50, db_index=True,
choices=[
('search_analytics', 'Search Analytics'),
('page_performance', 'Page Performance'),
('keyword_performance', 'Keyword Performance'),
]
)
dimension_filters = models.JSONField(
default=dict,
help_text="Filters used for this query: {dimensions, filters}"
)
data = models.JSONField(
default=list,
help_text="Cached query results"
)
date_range_start = models.DateField()
date_range_end = models.DateField()
expires_at = models.DateTimeField(
help_text="Cache expiry — refresh after this time"
)
class Meta:
app_label = 'integration'
db_table = 'igny8_gsc_metrics_cache'
ordering = ['-date_range_end']
class GSCDailyQuota(SiteSectorBaseModel):
"""Tracks daily URL Inspection API usage per site/property."""
date = models.DateField(db_index=True)
inspections_used = models.IntegerField(default=0)
quota_limit = models.IntegerField(default=2000)
class Meta:
app_label = 'integration'
db_table = 'igny8_gsc_daily_quota'
unique_together = [['site', 'date']]
```
### Migration
```
igny8_core/migrations/XXXX_gsc_integration.py
```
New tables:
1. `igny8_gsc_connections`
2. `igny8_url_inspection_records`
3. `igny8_indexing_queue`
4. `igny8_gsc_metrics_cache`
5. `igny8_gsc_daily_quota`
### API Endpoints
```
# OAuth Connection
POST /api/v1/integration/gsc/connect/ # Initiate OAuth (returns redirect URL)
GET /api/v1/integration/gsc/callback/ # OAuth callback (stores tokens)
DELETE /api/v1/integration/gsc/disconnect/ # Revoke + delete connection
GET /api/v1/integration/gsc/properties/ # List connected GSC properties
GET /api/v1/integration/gsc/status/ # Connection status
# Quota
GET /api/v1/integration/gsc/quota/ # Today's quota usage (used/limit)
# URL Inspection
POST /api/v1/integration/gsc/inspect/ # Queue single URL for inspection
POST /api/v1/integration/gsc/inspect/bulk/ # Queue multiple URLs
GET /api/v1/integration/gsc/inspections/ # List inspection records (filterable)
GET /api/v1/integration/gsc/inspections/{id}/ # Single inspection detail
# Search Analytics
GET /api/v1/integration/gsc/analytics/ # Search analytics (cached)
GET /api/v1/integration/gsc/analytics/keywords/ # Keyword performance
GET /api/v1/integration/gsc/analytics/pages/ # Page performance
GET /api/v1/integration/gsc/analytics/export/ # CSV export
# Queue Management (admin)
GET /api/v1/integration/gsc/queue/ # View queue status
POST /api/v1/integration/gsc/queue/clear/ # Clear failed/quota_exceeded items
```
### Services
```python
# igny8_core/business/integration/gsc_service.py
class GSCService:
"""Google Search Console API client wrapper."""
def get_oauth_url(self, site_id: int, redirect_uri: str) -> str:
"""Generate Google OAuth consent URL."""
pass
def handle_oauth_callback(self, code: str, site_id: int, account_id: int) -> GSCConnection:
"""Exchange auth code for tokens, create GSCConnection."""
pass
def refresh_access_token(self, connection: GSCConnection) -> bool:
"""Refresh expired access token using refresh_token."""
pass
def inspect_url(self, connection: GSCConnection, url: str) -> Dict:
"""Call URL Inspection API. Returns parsed response."""
pass
def fetch_search_analytics(
self, connection: GSCConnection,
start_date: str, end_date: str,
dimensions: list, row_limit: int = 25000
) -> List[Dict]:
"""Fetch search analytics data from GSC API."""
pass
def list_properties(self, connection: GSCConnection) -> List[str]:
"""List all GSC properties accessible by the connected account."""
pass
class IndexingQueueProcessor:
"""Processes the indexing queue respecting quota and rate limits."""
RATE_LIMIT_SECONDS = 3 # 1 request per 3 seconds
QUOTA_BUFFER = 50 # Reserve 50 inspections for manual use
def process_queue(self, site_id: int):
"""Process queued items for a site, respecting daily quota."""
quota = GSCDailyQuota.objects.get_or_create(
site_id=site_id,
date=timezone.now().date(),
defaults={'quota_limit': 2000}
)[0]
remaining = quota.quota_limit - quota.inspections_used - self.QUOTA_BUFFER
if remaining <= 0:
return {'processed': 0, 'reason': 'quota_exceeded'}
items = IndexingQueue.objects.filter(
site_id=site_id,
status='queued'
).order_by('-priority', 'date_added')[:remaining]
processed = 0
for item in items:
item.status = 'processing'
item.save()
try:
result = self.gsc_service.inspect_url(connection, item.url)
self._update_inspection_record(item, result)
item.status = 'completed'
item.date_processed = timezone.now()
quota.inspections_used += 1
processed += 1
time.sleep(self.RATE_LIMIT_SECONDS)
except QuotaExceededException:
item.status = 'quota_exceeded'
break
except Exception as e:
item.status = 'failed'
item.error_message = str(e)
finally:
item.save()
quota.save()
return {'processed': processed}
def _update_inspection_record(self, queue_item, result):
"""Create/update URLInspectionRecord from API result."""
record, created = URLInspectionRecord.objects.update_or_create(
site=queue_item.site,
url=queue_item.url,
defaults={
'last_inspection_result': result,
'verdict': result.get('inspectionResult', {}).get('indexStatusResult', {}).get('verdict', ''),
'coverage_state': result.get('inspectionResult', {}).get('indexStatusResult', {}).get('coverageState', ''),
'indexing_state': result.get('inspectionResult', {}).get('indexStatusResult', {}).get('indexingState', ''),
'last_inspected': timezone.now(),
'inspection_count': models.F('inspection_count') + 1,
}
)
# Schedule re-inspection
self._schedule_next_inspection(record)
def _schedule_next_inspection(self, record):
"""Schedule follow-up inspection based on inspection count."""
delays = {1: 1, 2: 3, 3: 6, 4: 13} # days after inspection
if record.inspection_count in delays:
record.next_inspection = timezone.now() + timedelta(days=delays[record.inspection_count])
record.save()
elif record.inspection_count > 4 and record.verdict != 'PASS':
record.status = 'manual_review'
record.next_inspection = None
record.save()
```
### Celery Tasks
```python
# igny8_core/tasks/gsc_tasks.py
@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def process_indexing_queue(self, site_id: int = None):
"""Process pending indexing queue items. Runs every 5 minutes."""
# If site_id provided, process that site only
# Otherwise, process all sites with active GSCConnection
pass
@shared_task(bind=True, max_retries=3, default_retry_delay=300)
def refresh_gsc_tokens(self):
"""Refresh expiring GSC OAuth tokens. Runs hourly."""
expiring = GSCConnection.objects.filter(
status='active',
token_expiry__lte=timezone.now() + timedelta(minutes=10)
)
for conn in expiring:
GSCService().refresh_access_token(conn)
@shared_task(bind=True, max_retries=3, default_retry_delay=300)
def fetch_search_analytics(self):
"""Fetch and cache search analytics for all connected sites. Runs daily."""
pass
@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def schedule_reinspections(self):
"""Add due re-inspections to the queue. Runs daily."""
due = URLInspectionRecord.objects.filter(
next_inspection__lte=timezone.now(),
status__in=['not_indexed', 'indexing_requested']
)
for record in due:
IndexingQueue.objects.get_or_create(
site=record.site,
url=record.url,
status='queued',
defaults={'priority': 90, 'url_inspection_record': record}
)
@shared_task(bind=True)
def auto_queue_published_content(self, content_id: int):
"""Queue newly published content for GSC inspection. Triggered by publish signal."""
content = Content.objects.get(id=content_id)
if not content.external_url:
return
connection = GSCConnection.objects.filter(
site=content.site, status='active'
).first()
if not connection:
return
record, _ = URLInspectionRecord.objects.get_or_create(
site=content.site, url=content.external_url,
defaults={'content': content, 'sector': content.sector, 'account': content.account}
)
IndexingQueue.objects.create(
site=content.site, sector=content.sector, account=content.account,
url=content.external_url, url_inspection_record=record,
priority=100, status='queued'
)
```
**Beat schedule additions** (add to `igny8_core/celery.py`):
```python
'process-indexing-queue': {
'task': 'gsc.process_indexing_queue',
'schedule': crontab(minute='*/5'), # Every 5 minutes
},
'refresh-gsc-tokens': {
'task': 'gsc.refresh_gsc_tokens',
'schedule': crontab(minute=30), # Every hour at :30
},
'fetch-search-analytics': {
'task': 'gsc.fetch_search_analytics',
'schedule': crontab(hour=4, minute=0), # Daily at 4 AM
},
'schedule-reinspections': {
'task': 'gsc.schedule_reinspections',
'schedule': crontab(hour=5, minute=0), # Daily at 5 AM
},
```
### Auto-Indexing Signal
Connect to Content model's post-publish flow:
```python
# igny8_core/business/integration/signals.py
from django.db.models.signals import post_save
from django.dispatch import receiver
@receiver(post_save, sender='writer.Content')
def queue_for_gsc_inspection(sender, instance, **kwargs):
"""When content is published, auto-queue for GSC inspection."""
if instance.site_status == 'published' and instance.external_url:
auto_queue_published_content.delay(instance.id)
```
### Credit Costs
| Operation | Credits | Notes |
|-----------|---------|-------|
| GSC OAuth connection setup | 1 | One-time per connection |
| URL inspections (per 100) | 0.1 | Batch pricing |
| Indexing request (per URL) | 0.05 | Minimal cost |
| Analytics caching (per site/month) | 0.5 | Monthly recurring |
Add to `CreditCostConfig`:
```python
CreditCostConfig.objects.get_or_create(
operation_type='gsc_inspection',
defaults={'display_name': 'GSC URL Inspection', 'base_credits': 1}
)
CreditCostConfig.objects.get_or_create(
operation_type='gsc_analytics',
defaults={'display_name': 'GSC Analytics Sync', 'base_credits': 1}
)
```
---
## 4. IMPLEMENTATION STEPS
### Step 1: Create GSC Models
File to create/modify:
- `backend/igny8_core/business/integration/gsc_models.py` (or add to existing `models.py`)
- 5 new models: GSCConnection, URLInspectionRecord, IndexingQueue, GSCMetricsCache, GSCDailyQuota
### Step 2: Create and Run Migration
```bash
cd /data/app/igny8/backend
python manage.py makemigrations --name gsc_integration
python manage.py migrate
```
### Step 3: Build GSCService
File to create:
- `backend/igny8_core/business/integration/gsc_service.py`
- Requires `google-auth`, `google-auth-oauthlib`, `google-api-python-client` packages
Add to `requirements.txt`:
```
google-auth>=2.0.0
google-auth-oauthlib>=1.0.0
google-api-python-client>=2.0.0
```
### Step 4: Build IndexingQueueProcessor
File to create:
- `backend/igny8_core/business/integration/indexing_queue_processor.py`
### Step 5: Build Celery Tasks
File to create:
- `backend/igny8_core/tasks/gsc_tasks.py`
Add beat schedule entries to:
- `backend/igny8_core/celery.py`
### Step 6: Build Auto-Indexing Signal
File to create:
- `backend/igny8_core/business/integration/signals.py`
Register in:
- `backend/igny8_core/business/integration/apps.py` — `ready()` method
### Step 7: Build Serializers
File to create:
- `backend/igny8_core/modules/integration/serializers/gsc_serializers.py`
### Step 8: Build ViewSets and URLs
Files to create:
- `backend/igny8_core/modules/integration/views/gsc_views.py`
- Modify `backend/igny8_core/modules/integration/urls.py` — register GSC endpoints
### Step 9: Add OAuth Settings
Add to `backend/igny8_core/settings.py`:
```python
# Google OAuth 2.0 (GSC Integration)
GOOGLE_CLIENT_ID = env('GOOGLE_CLIENT_ID', default='')
GOOGLE_CLIENT_SECRET = env('GOOGLE_CLIENT_SECRET', default='')
GOOGLE_REDIRECT_URI = env('GOOGLE_REDIRECT_URI', default='')
```
### Step 10: Frontend
Files to create in `frontend/src/`:
- `pages/Integration/GSCDashboard.tsx` — main GSC dashboard
- `pages/Integration/GSCAnalytics.tsx` — search analytics with charts
- `pages/Integration/GSCInspections.tsx` — URL inspection list with status badges
- `pages/Integration/GSCConnect.tsx` — OAuth connection flow
- `components/Integration/QuotaIndicator.tsx` — daily quota usage bar
- `components/Integration/InspectionStatusBadge.tsx` — status badges
- `stores/gscStore.ts` — Zustand store
- `api/gsc.ts` — API client
### Step 11: Tests
```bash
cd /data/app/igny8/backend
python manage.py test igny8_core.business.integration.tests.test_gsc_service
python manage.py test igny8_core.business.integration.tests.test_indexing_queue
python manage.py test igny8_core.modules.integration.tests.test_gsc_views
```
---
## 5. ACCEPTANCE CRITERIA
- [ ] 5 new database tables created and migrated successfully
- [ ] Google OAuth 2.0 flow works: connect → consent → callback → tokens stored
- [ ] GSC properties listed after successful OAuth connection
- [ ] Token refresh works automatically before expiry via Celery task
- [ ] URL Inspection API calls succeed and results stored in URLInspectionRecord
- [ ] Daily quota tracked in GSCDailyQuota, respects 2,000/day limit
- [ ] Rate limit of 1 request/3 seconds enforced in queue processor
- [ ] Re-inspection schedule runs: 1 day, 3 days, 6 days, 13 days after initial check
- [ ] URLs not indexed after Check 4 marked as 'manual_review'
- [ ] Content publish triggers auto-queue at priority 100
- [ ] Search analytics data fetched and cached with 24-hour TTL
- [ ] Analytics endpoints return cached data with date range filtering
- [ ] All endpoints require authentication and enforce account isolation
- [ ] Frontend GSC dashboard shows: connection status, quota usage, inspection list, analytics charts
- [ ] inspection status badges display correctly on URL list
- [ ] `google-auth`, `google-auth-oauthlib`, `google-api-python-client` added to requirements.txt
- [ ] Disconnecting GSC revokes token and deletes GSCConnection
---
## 6. CLAUDE CODE INSTRUCTIONS
### Execution Order
1. Read `backend/igny8_core/business/integration/models.py` — understand existing SiteIntegration, SyncEvent
2. Read `backend/igny8_core/modules/integration/urls.py` — understand existing URL patterns
3. Read `backend/igny8_core/celery.py` — understand beat schedule registration
4. Add new packages to requirements.txt
5. Create GSC models (5 models)
6. Create migration, run it
7. Build GSCService (OAuth + API client)
8. Build IndexingQueueProcessor
9. Build Celery tasks (4 tasks) + register in beat schedule
10. Build auto-indexing signal
11. Build serializers, ViewSets, URLs
12. Build frontend components
### Key Constraints
- ALL primary keys are `BigAutoField` (integer). No UUIDs.
- Model class names: GSCConnection, URLInspectionRecord, IndexingQueue, GSCMetricsCache, GSCDailyQuota (descriptive names, not plural)
- Frontend: `.tsx` files, Zustand stores, Vitest testing
- Celery app name: `igny8_core`
- All db_tables use `igny8_` prefix
- Tokens MUST be encrypted at rest (use same encryption as SiteIntegration.credentials_json)
- OAuth client_id/secret must be in environment variables, never in code
- Follow existing integration app patterns for URL structure
### File Tree (New/Modified)
```
backend/igny8_core/
├── business/integration/
│ ├── models.py # MODIFY or NEW gsc_models.py: 5 new models
│ ├── gsc_service.py # NEW: GSCService (OAuth + API)
│ ├── indexing_queue_processor.py # NEW: IndexingQueueProcessor
│ ├── signals.py # NEW: auto-indexing signal
│ └── apps.py # MODIFY: register signals in ready()
├── tasks/
│ └── gsc_tasks.py # NEW: 4 Celery tasks
├── celery.py # MODIFY: add 4 beat schedule entries
├── settings.py # MODIFY: add GOOGLE_* settings
├── modules/integration/
│ ├── serializers/
│ │ └── gsc_serializers.py # NEW
│ ├── views/
│ │ └── gsc_views.py # NEW
│ └── urls.py # MODIFY: register GSC endpoints
├── migrations/
│ └── XXXX_gsc_integration.py # NEW: auto-generated
├── requirements.txt # MODIFY: add google-auth packages
frontend/src/
├── pages/Integration/
│ ├── GSCDashboard.tsx # NEW
│ ├── GSCAnalytics.tsx # NEW
│ ├── GSCInspections.tsx # NEW
│ └── GSCConnect.tsx # NEW
├── components/Integration/
│ ├── QuotaIndicator.tsx # NEW
│ └── InspectionStatusBadge.tsx # NEW
├── stores/
│ └── gscStore.ts # NEW: Zustand store
├── api/
│ └── gsc.ts # NEW: API client
```
### Cross-References
- **01E** (blueprint-aware pipeline): triggers auto-indexing after publish, hub pages prioritized
- **02E** (backlinks): GSC impressions data feeds backlink KPI dashboard
- **02F** (optimizer): GSC position data identifies optimization candidates
- **03A** (WP plugin standalone): standalone plugin has GSC dashboard tab
- **03B** (WP plugin connected): connected mode syncs index statuses from IGNY8 to WP
- **04B** (reporting): GSC metrics (clicks, impressions, CTR) feed into service reports

735
v2/V2-Execution-Docs/02D-linker-internal.md Normal file
View File

@@ -0,0 +1,735 @@
# IGNY8 Phase 2: Internal Linker (02D)
## SAG-Based Internal Linking Engine
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Internal Linking Today
There is **no** internal linking system in IGNY8. Content is generated and published without any cross-linking strategy. Links within content are only those the AI incidentally includes during generation.
### What Exists
- `Content` model (app_label=`writer`, db_table=`igny8_content`) — stores `content_html` where links would be inserted
- `SAGCluster` and `SAGBlueprint` models (from 01A) — provide the cluster hierarchy for link topology
- The 7-stage automation pipeline (01E) generates and publishes content but has no linking stage between generation and publish
- `SiteIntegration` model (app_label=`integration`) tracks WordPress connections
### What Does Not Exist
- No SAGLink model, no LinkMap model, no SAGLinkAudit model
- No link scoring algorithm
- No anchor text management
- No link density enforcement
- No link insertion into content_html
- No orphan page detection
- No link health monitoring
- No link audit system
### Foundation Available
- `SAGBlueprint` (01A) — defines the SAG hierarchy (site → sectors → clusters → content)
- `SAGCluster` (01A) — cluster_type, hub_page_type, hub_page_structure
- `SAGAttribute` (01A) — attribute values shared across clusters (basis for cross-cluster linking)
- 01E pipeline — post-generation hook point available between Stage 4 (Content) and Stage 7 (Publish)
- `Content.content_type` and `Content.content_structure` — determines link density rules
- 02B `ContentTaxonomy` with cluster mapping — taxonomy-to-cluster relationships for taxonomy contextual links
---
## 2. WHAT TO BUILD
### Overview
Build a SAG-aware internal linking engine that automatically plans, scores, and inserts internal links into content. The system operates in two modes: new content mode (pipeline integration) and existing content remediation (audit + fix).
### 2.1 Seven Link Types
| # | Link Type | Direction | Description | Limit | Placement |
|---|-----------|-----------|-------------|-------|-----------|
| 1 | **Vertical Upward** | Supporting → Hub | MANDATORY: every supporting article links to its cluster hub | 1 per article | First 2 paragraphs |
| 2 | **Vertical Downward** | Hub → Supporting | Hub lists ALL its supporting articles | No cap | "Related Articles" section + contextual body links |
| 3 | **Horizontal Sibling** | Supporting ↔ Supporting | Same-cluster articles linking to each other | Max 2 per article | Natural content overlap points |
| 4 | **Cross-Cluster** | Hub ↔ Hub | Hubs sharing a SAGAttribute value can cross-link | Max 2 per hub | Contextual body links |
| 5 | **Taxonomy Contextual** | Term Page → Hubs | Term pages link to ALL cluster hubs using that attribute | No cap | Auto-generated from 02B taxonomy-cluster mapping |
| 6 | **Breadcrumb** | Hierarchical | Home → Sector → [Attribute] → Hub → Current Page | 1 chain per page | Top of page (auto-generated from SAG hierarchy) |
| 7 | **Related Content** | Cross-cluster allowed | 2-3 links in "Related Reading" section at end of article | 2-3 per article | End of article section |
**Link Density Rules (outbound per page type, by word count):**
| Page Type | <1000 words | 1000-2000 words | 2000+ words |
|-----------|------------|-----------------|-------------|
| Hub (`cluster_hub`) | 5-10 | 10-15 | 15-20 |
| Blog (article/guide/etc.) | 2-5 | 3-8 | 4-12 |
| Product/Service | 2-3 | 3-5 | 3-5 |
| Term Page (taxonomy) | 3+ | 3+ | unlimited |
### 2.2 Link Scoring Algorithm (5 Factors)
Each candidate link target receives a score (0-100):
| Factor | Weight | Description |
|--------|--------|-------------|
| Shared attribute values | 40% | Count of SAGAttribute values shared between source and target clusters |
| Target page authority | 25% | Inbound link count of target page (from LinkMap) |
| Keyword overlap | 20% | Common keywords between source cluster and target content |
| Content recency | 10% | Newer content gets a boost (exponential decay over 6 months) |
| Link count gap | 5% | Pages with fewest inbound links get a priority boost |
**Threshold:** Score ≥ 60 qualifies for automatic linking. Scores 40-59 are suggested for manual review.
### 2.3 Anchor Text Rules
| Rule | Value |
|------|-------|
| Min length | 2 words |
| Max length | 8 words |
| Grammatically natural | Must read naturally in surrounding sentence |
| No exact-match overuse | Same exact anchor cannot be used >3 times to same target URL |
| Anchor distribution per target | Primary keyword 60%, page title 30%, natural phrase 10% |
| Diversification audit | Flag if any single anchor accounts for >40% of links to a target |
**Anchor Types:**
- `primary_keyword` — cluster primary keyword
- `page_title` — target content's title (or shortened version)
- `natural` — AI-selected contextually appropriate phrase
- `branded` — brand/site name (for homepage links)
### 2.4 Two Operating Modes
#### A. New Content Mode (Pipeline Integration)
Runs after Stage 4 (content generated), before Stage 7 (publish):
1. Content generated by pipeline → link planning triggers
2. Calculate link targets using scoring algorithm
3. Insert links into `content_html` at natural positions
4. Store link plan in SAGLink records
5. If content is a hub → auto-generate "Related Articles" section with links to all supporting articles in cluster
6. **Mandatory check:** if content is a supporting article, verify vertical_up link to hub exists; insert if missing
#### B. Existing Content Remediation (Audit + Fix)
For already-published content without proper internal linking:
1. **Crawl phase:** Scan all published content for a site, extract all `<a>` tags, build LinkMap
2. **Audit analysis:**
- Orphan pages: 0 inbound internal links
- Over-linked pages: outbound > density max for page type/word count
- Under-linked pages: outbound < density min
- Missing mandatory links: supporting articles without hub uplink
- Broken links: target URL returns 4xx/5xx
3. **Recommendation generation:** Priority-scored fix recommendations with AI-suggested anchor text
4. **Batch application:** Insert missing links across multiple content records
### 2.5 Cluster-Level Link Health Score
Per-cluster health score (0-100) for link coverage:
| Factor | Points |
|--------|--------|
| Hub published and linked (has outbound + inbound links) | 25 |
| All supporting articles have mandatory uplink to hub | 25 |
| At least 1 cross-cluster link from hub | 15 |
| Term pages link to hub | 15 |
| No broken links in cluster | 10 |
| Link density within range for all pages | 10 |
Site-wide link health = average of all cluster scores. Feeds into SAG health monitoring (01G).
---
## 3. DATA MODELS & APIS
### 3.1 New Models
#### SAGLink (new `linker` app)
```python
class SAGLink(SiteSectorBaseModel):
"""
Represents a planned or inserted internal link between two content pages.
Tracks link type, anchor text, score, and status through lifecycle.
"""
blueprint = models.ForeignKey(
'planner.SAGBlueprint',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='sag_links'
)
source_content = models.ForeignKey(
'writer.Content',
on_delete=models.CASCADE,
related_name='outbound_sag_links'
)
target_content = models.ForeignKey(
'writer.Content',
on_delete=models.CASCADE,
related_name='inbound_sag_links'
)
link_type = models.CharField(
max_length=20,
choices=[
('vertical_up', 'Vertical Upward'),
('vertical_down', 'Vertical Downward'),
('horizontal', 'Horizontal Sibling'),
('cross_cluster', 'Cross-Cluster'),
('taxonomy', 'Taxonomy Contextual'),
('breadcrumb', 'Breadcrumb'),
('related', 'Related Content'),
]
)
anchor_text = models.CharField(max_length=200)
anchor_type = models.CharField(
max_length=20,
choices=[
('primary_keyword', 'Primary Keyword'),
('page_title', 'Page Title'),
('natural', 'Natural Phrase'),
('branded', 'Branded'),
]
)
placement_zone = models.CharField(
max_length=20,
choices=[
('in_body', 'In Body'),
('related_section', 'Related Section'),
('breadcrumb', 'Breadcrumb'),
('sidebar', 'Sidebar'),
]
)
placement_position = models.IntegerField(
null=True,
blank=True,
help_text='Paragraph number for in_body placement'
)
score = models.FloatField(
default=0,
help_text='Link scoring algorithm result (0-100)'
)
status = models.CharField(
max_length=15,
choices=[
('planned', 'Planned'),
('inserted', 'Inserted'),
('verified', 'Verified'),
('broken', 'Broken'),
('removed', 'Removed'),
],
default='planned'
)
is_mandatory = models.BooleanField(
default=False,
help_text='True for vertical_up links (supporting → hub)'
)
inserted_at = models.DateTimeField(null=True, blank=True)
class Meta:
app_label = 'linker'
db_table = 'igny8_sag_links'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### SAGLinkAudit (linker app)
```python
class SAGLinkAudit(SiteSectorBaseModel):
"""
Stores results of a site-wide or cluster-level link audit.
"""
blueprint = models.ForeignKey(
'planner.SAGBlueprint',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='link_audits'
)
audit_date = models.DateTimeField(auto_now_add=True)
total_links = models.IntegerField(default=0)
missing_mandatory = models.IntegerField(default=0)
orphan_pages = models.IntegerField(default=0)
broken_links = models.IntegerField(default=0)
over_linked_pages = models.IntegerField(default=0)
under_linked_pages = models.IntegerField(default=0)
cluster_scores = models.JSONField(
default=dict,
help_text='{cluster_id: {score, missing, issues[]}}'
)
recommendations = models.JSONField(
default=list,
help_text='[{content_id, action, link_type, target_id, anchor_suggestion, priority}]'
)
overall_health_score = models.FloatField(
default=0,
help_text='Average of cluster scores (0-100)'
)
class Meta:
app_label = 'linker'
db_table = 'igny8_sag_link_audits'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### LinkMap (linker app)
```python
class LinkMap(SiteSectorBaseModel):
"""
Full link map of all internal (and external) links found in published content.
Built by crawling content_html of all published content records.
"""
source_url = models.URLField()
source_content = models.ForeignKey(
'writer.Content',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='outbound_link_map'
)
target_url = models.URLField()
target_content = models.ForeignKey(
'writer.Content',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='inbound_link_map'
)
anchor_text = models.CharField(max_length=500)
is_internal = models.BooleanField(default=True)
is_follow = models.BooleanField(default=True)
position = models.CharField(
max_length=20,
choices=[
('in_content', 'In Content'),
('navigation', 'Navigation'),
('footer', 'Footer'),
('sidebar', 'Sidebar'),
],
default='in_content'
)
last_verified = models.DateTimeField(null=True, blank=True)
status = models.CharField(
max_length=15,
choices=[
('active', 'Active'),
('broken', 'Broken'),
('removed', 'Removed'),
],
default='active'
)
class Meta:
app_label = 'linker'
db_table = 'igny8_link_map'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
### 3.2 Modified Models
#### Content (writer app) — add 4 fields
```python
# Add to Content model:
link_plan = models.JSONField(
null=True,
blank=True,
help_text='Planned links before insertion: [{target_id, link_type, anchor, score}]'
)
links_inserted = models.BooleanField(
default=False,
help_text='Whether link plan has been applied to content_html'
)
inbound_link_count = models.IntegerField(
default=0,
help_text='Cached count of inbound internal links'
)
outbound_link_count = models.IntegerField(
default=0,
help_text='Cached count of outbound internal links'
)
```
### 3.3 New App Registration
Create linker app:
- **App config:** `igny8_core/modules/linker/apps.py` with `app_label = 'linker'`
- **Add to INSTALLED_APPS** in `igny8_core/settings.py`
### 3.4 Migration
```
igny8_core/migrations/XXXX_add_linker_models.py
```
**Operations:**
1. `CreateModel('SAGLink', ...)` — with indexes on source_content, target_content, link_type, status
2. `CreateModel('SAGLinkAudit', ...)`
3. `CreateModel('LinkMap', ...)` — with index on source_url, target_url
4. `AddField('Content', 'link_plan', JSONField(null=True, blank=True))`
5. `AddField('Content', 'links_inserted', BooleanField(default=False))`
6. `AddField('Content', 'inbound_link_count', IntegerField(default=0))`
7. `AddField('Content', 'outbound_link_count', IntegerField(default=0))`
### 3.5 API Endpoints
All endpoints under `/api/v1/linker/`:
#### Link Management
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/links/?site_id=X` | List all SAGLink records with filters (link_type, status, cluster_id, source_content_id) |
| POST | `/api/v1/linker/links/plan/` | Generate link plan for a content piece. Body: `{content_id}`. Returns planned SAGLink records. |
| POST | `/api/v1/linker/links/insert/` | Insert planned links into content_html. Body: `{content_id}`. Modifies Content.content_html. |
| POST | `/api/v1/linker/links/batch-insert/` | Batch insert for multiple content. Body: `{content_ids: [int]}`. Queues Celery task. |
| GET | `/api/v1/linker/content/{id}/links/` | All inbound + outbound links for a specific content piece. |
#### Link Audit
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/audit/?site_id=X` | Latest SAGLinkAudit results. |
| POST | `/api/v1/linker/audit/run/` | Trigger site-wide link audit. Body: `{site_id}`. Queues Celery task. Returns task ID. |
| GET | `/api/v1/linker/audit/recommendations/?site_id=X` | Get fix recommendations from latest audit. |
| POST | `/api/v1/linker/audit/apply/` | Apply recommended fixes in batch. Body: `{site_id, recommendation_ids: [int]}`. |
#### Link Map & Health
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/link-map/?site_id=X` | Full LinkMap for site with pagination. |
| GET | `/api/v1/linker/orphans/?site_id=X` | List orphan pages (0 inbound internal links). |
| GET | `/api/v1/linker/health/?site_id=X` | Cluster-level link health scores. |
**Permissions:** All endpoints use `SiteSectorModelViewSet` permission patterns.
### 3.6 Link Planning Service
**Location:** `igny8_core/business/link_planning.py`
```python
class LinkPlanningService:
"""
Generates internal link plans for content based on SAG hierarchy
and scoring algorithm.
"""
SCORE_WEIGHTS = {
'shared_attributes': 0.40,
'target_authority': 0.25,
'keyword_overlap': 0.20,
'content_recency': 0.10,
'link_count_gap': 0.05,
}
AUTO_LINK_THRESHOLD = 60
REVIEW_THRESHOLD = 40
def plan(self, content_id):
"""
Generate link plan for a content piece.
1. Identify content's cluster and role (hub vs supporting)
2. Determine mandatory links (vertical_up for supporting)
3. Score all candidate targets
4. Select targets within density limits
5. Generate anchor text per link
6. Create SAGLink records with status='planned'
Returns list of planned SAGLink records.
"""
pass
def _get_mandatory_links(self, content, cluster):
"""Vertical upward: supporting → hub. Always added."""
pass
def _get_candidates(self, content, cluster, blueprint):
"""Gather all potential link targets from cluster and related clusters."""
pass
def _score_candidate(self, source_content, target_content, source_cluster,
target_cluster, blueprint):
"""Calculate 0-100 score using 5-factor algorithm."""
pass
def _select_within_density(self, content, scored_candidates):
"""Filter candidates to stay within density limits for page type and word count."""
pass
def _generate_anchor_text(self, source_content, target_content, link_type):
"""AI-generate contextually appropriate anchor text."""
pass
```
### 3.7 Link Insertion Service
**Location:** `igny8_core/business/link_insertion.py`
```python
class LinkInsertionService:
"""
Inserts planned links into content_html.
Handles placement, anchor text insertion, and collision avoidance.
"""
def insert(self, content_id):
"""
Insert all planned SAGLink records into Content.content_html.
1. Load all SAGLinks where source_content=content_id, status='planned'
2. Parse content_html
3. For each link, find insertion point based on placement_zone + position
4. Insert <a> tag with anchor text
5. Update SAGLink status='inserted', set inserted_at
6. Update Content.content_html, links_inserted=True, outbound_link_count
7. Update target Content.inbound_link_count
"""
pass
def _find_insertion_point(self, html_tree, link):
"""
Find best insertion point in parsed HTML:
- in_body: find paragraph at placement_position, find natural spot for anchor
- related_section: append to "Related Articles" section (create if missing)
- breadcrumb: insert breadcrumb trail at top
"""
pass
def _insert_link(self, html_tree, position, anchor_text, target_url):
"""Insert <a href> tag at position without breaking existing HTML."""
pass
```
### 3.8 Link Audit Service
**Location:** `igny8_core/business/link_audit.py`
```python
class LinkAuditService:
"""
Runs site-wide link audits: builds link map, identifies issues,
generates recommendations.
"""
def run_audit(self, site_id):
"""
Full audit:
1. Crawl all published Content for site
2. Extract all <a> tags, build/update LinkMap records
3. Identify orphan pages, over/under-linked, missing mandatory, broken
4. Calculate per-cluster health scores
5. Generate prioritized recommendations
6. Create SAGLinkAudit record
Returns SAGLinkAudit instance.
"""
pass
def _build_link_map(self, site_id):
"""Extract links from all published content_html, create LinkMap records."""
pass
def _find_orphans(self, site_id):
"""Content with 0 inbound internal links."""
pass
def _check_density(self, site_id):
"""Compare outbound counts against density rules per page type."""
pass
def _check_mandatory(self, site_id):
"""Verify all supporting articles have vertical_up link to their hub."""
pass
def _calculate_cluster_health(self, site_id, cluster):
"""Calculate 0-100 health score per cluster."""
pass
def _generate_recommendations(self, issues):
"""Priority-scored recommendations with AI-suggested anchor text."""
pass
```
### 3.9 Celery Tasks
**Location:** `igny8_core/tasks/linker_tasks.py`
```python
@shared_task(name='generate_link_plan')
def generate_link_plan(content_id):
"""Runs after content generation, before publish. Creates SAGLink records."""
pass
@shared_task(name='run_link_audit')
def run_link_audit(site_id):
"""Scheduled weekly or triggered manually. Full site-wide audit."""
pass
@shared_task(name='verify_links')
def verify_links(site_id):
"""Check for broken links via HTTP status checks on LinkMap URLs."""
pass
@shared_task(name='rebuild_link_map')
def rebuild_link_map(site_id):
"""Full crawl of published content to rebuild LinkMap from scratch."""
pass
```
**Beat Schedule Additions:**
| Task | Schedule | Notes |
|------|----------|-------|
| `run_link_audit` | Weekly (Sunday 1:00 AM) | Site-wide audit for all active sites |
| `verify_links` | Weekly (Wednesday 2:00 AM) | HTTP check all active LinkMap entries |
---
## 4. IMPLEMENTATION STEPS
### Step 1: Create Linker App
1. Create `igny8_core/modules/linker/` directory with `__init__.py` and `apps.py`
2. Add `linker` to `INSTALLED_APPS` in settings.py
3. Create models: SAGLink, SAGLinkAudit, LinkMap
### Step 2: Migration
1. Create migration for 3 new models
2. Add 4 new fields to Content model (link_plan, links_inserted, inbound_link_count, outbound_link_count)
3. Run migration
### Step 3: Services
1. Implement `LinkPlanningService` in `igny8_core/business/link_planning.py`
2. Implement `LinkInsertionService` in `igny8_core/business/link_insertion.py`
3. Implement `LinkAuditService` in `igny8_core/business/link_audit.py`
### Step 4: Pipeline Integration
Insert link planning + insertion between Stage 4 and Stage 7:
```python
# After content generation completes in pipeline:
def post_content_generation(content_id):
# 02G: Generate schema + SERP elements
# ...
# 02D: Plan and insert internal links
link_service = LinkPlanningService()
link_service.plan(content_id)
insertion_service = LinkInsertionService()
insertion_service.insert(content_id)
```
### Step 5: API Endpoints
1. Create `igny8_core/urls/linker.py` with link, audit, and health endpoints
2. Create views extending `SiteSectorModelViewSet`
3. Register URL patterns under `/api/v1/linker/`
### Step 6: Celery Tasks
1. Implement all 4 tasks in `igny8_core/tasks/linker_tasks.py`
2. Add `run_link_audit` and `verify_links` to Celery beat schedule
### Step 7: Serializers & Admin
1. Create DRF serializers for SAGLink, SAGLinkAudit, LinkMap
2. Register models in Django admin
### Step 8: Credit Cost Configuration
Add to `CreditCostConfig` (billing app):
| operation_type | default_cost | description |
|---------------|-------------|-------------|
| `link_audit` | 1 | Site-wide link audit |
| `link_generation` | 0.5 | Generate 1-5 links with AI anchor text |
| `link_audit_full` | 3-5 | Full site audit with recommendations |
---
## 5. ACCEPTANCE CRITERIA
### Link Types
- [ ] Vertical upward link (supporting → hub) automatically inserted for all supporting articles
- [ ] Vertical downward links (hub → supporting) generated with "Related Articles" section
- [ ] Horizontal sibling links (max 2) between same-cluster supporting articles
- [ ] Cross-cluster links (max 2) between hubs sharing SAGAttribute values
- [ ] Taxonomy contextual links from term pages to all relevant cluster hubs
- [ ] Breadcrumb chain generated from SAG hierarchy for all content
- [ ] Related content section (2-3 links) generated at end of article
### Link Scoring
- [ ] 5-factor scoring algorithm produces 0-100 scores
- [ ] Links with score ≥ 60 auto-inserted
- [ ] Links with score 40-59 suggested for manual review
- [ ] Score algorithm uses: shared attributes (40%), authority (25%), keyword overlap (20%), recency (10%), gap boost (5%)
### Anchor Text
- [ ] Anchor text 2-8 words, grammatically natural
- [ ] Same exact anchor not used >3 times to same target
- [ ] Distribution per target: 60% primary keyword, 30% page title, 10% natural
- [ ] Diversification audit flags if any anchor >40% of links to a target
### Link Density
- [ ] Hub pages: 5-20 outbound links based on word count
- [ ] Blog pages: 2-12 outbound links based on word count
- [ ] Product/Service pages: 2-5 outbound links
- [ ] Term pages: 3+ outbound, unlimited for taxonomy contextual
### Audit & Remediation
- [ ] Link audit identifies orphan pages, over/under-linked, missing mandatory, broken links
- [ ] Cluster-level health score (0-100) calculated per cluster
- [ ] Recommendations generated with priority scores and AI-suggested anchors
- [ ] Batch application of recommendations modifies content_html correctly
### Pipeline Integration
- [ ] Link plan generated automatically after content generation in pipeline
- [ ] Links inserted before publish stage
- [ ] Mandatory vertical_up link verified before allowing publish
- [ ] Content.inbound_link_count and outbound_link_count updated on insert
---
## 6. CLAUDE CODE INSTRUCTIONS
### File Locations
```
igny8_core/
├── modules/
│ └── linker/
│ ├── __init__.py
│ ├── apps.py # app_label = 'linker'
│ └── models.py # SAGLink, SAGLinkAudit, LinkMap
├── business/
│ ├── link_planning.py # LinkPlanningService
│ ├── link_insertion.py # LinkInsertionService
│ └── link_audit.py # LinkAuditService
├── tasks/
│ └── linker_tasks.py # Celery tasks
├── urls/
│ └── linker.py # Linker endpoints
└── migrations/
└── XXXX_add_linker_models.py
```
### Conventions
- **PKs:** BigAutoField (integer) — do NOT use UUIDs
- **Table prefix:** `igny8_` on all new tables
- **App label:** `linker` (new app)
- **Celery app name:** `igny8_core`
- **URL pattern:** `/api/v1/linker/...`
- **Permissions:** Use `SiteSectorModelViewSet` permission pattern
- **Model inheritance:** SAGLink and SAGLinkAudit extend `SiteSectorBaseModel`; LinkMap extends `SiteSectorBaseModel`
- **Frontend:** `.tsx` files with Zustand stores for state management
### Cross-References
| Doc | Relationship |
|-----|-------------|
| **01A** | SAGBlueprint/SAGCluster/SAGAttribute provide hierarchy and cross-cluster relationships |
| **01E** | Pipeline integration — link planning hooks after Stage 4, before Stage 7 |
| **01G** | SAG health monitoring incorporates cluster link health scores |
| **02B** | ContentTaxonomy cluster mapping enables taxonomy contextual links |
| **02E** | External backlinks complement internal links; authority distributed by internal links |
| **02F** | Optimizer identifies internal link opportunities and feeds to linker |
| **03A** | WP plugin standalone mode has its own internal linking module — separate from this |
| **03C** | Theme renders breadcrumbs and related content sections generated by linker |
### Key Decisions
1. **New `linker` app** — Separate app because linking is a distinct domain with its own models, not tightly coupled to writer or planner
2. **SAGLink stores planned AND inserted** — Single model tracks the full lifecycle from planning through insertion to verification
3. **LinkMap is separate from SAGLink** — LinkMap stores the actual crawled link state (including non-SAG links); SAGLink stores the planned/managed links
4. **Cached counts on Content** — `inbound_link_count` and `outbound_link_count` are denormalized for fast queries; updated on insert/removal
5. **HTML parsing for insertion** — Use Python HTML parser (BeautifulSoup or lxml) for safe link insertion without corrupting content_html

734
v2/V2-Execution-Docs/02E-linker-external-backlinks.md Normal file
View File

@@ -0,0 +1,734 @@
# IGNY8 Phase 2: External Linker & Backlinks (02E)
## SAG-Based External Backlink Campaign Engine
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### External Linking Today
There is **no** backlink management in IGNY8. No external API integrations exist for link building platforms. No campaign generation, tracking, or KPI monitoring. Backlink building is entirely manual and external to the platform.
### What Exists
- `SAGBlueprint` and `SAGCluster` models (01A) — provide the hierarchy and cluster assignments for target page identification
- `Keywords` model (planner app) — provides search volume data for tier assignment
- `Content` model with `content_type` and `content_structure` — classifies pages for tiering
- `GSCMetricsCache` (02C) — provides organic traffic and impression data for KPI tracking
- `SAGLink` and `LinkMap` models (02D) — internal link data complements external strategy
- The `linker` app (02D) — provides app namespace for related models
### What Does Not Exist
- No SAGCampaign model, SAGBacklink model, or CampaignKPISnapshot model
- No page tier assignment system
- No country-specific strategy profiles
- No marketplace API integrations (FatGrid, PRNews.io, etc.)
- No campaign generation algorithm
- No anchor text planning
- No quality scoring for backlink opportunities
- No tipping point detection
- No dead link monitoring for placed backlinks
---
## 2. WHAT TO BUILD
### Overview
Build a SAG-based backlink campaign engine that generates country-specific link-building campaigns targeting hub pages. The system leverages the SAG hierarchy to focus backlinks on high-value pages (T1-T3) and lets internal linking (02D) distribute authority to supporting content.
### 2.1 Hub-Only Strategy (Core Principle)
Backlinks target ONLY T1-T3 pages (homepage + cluster hubs + key service/product pages). SAG internal linking (02D) distributes authority from hubs downstream to 70+ supporting pages per cluster.
**Budget Allocation:**
- 70-85% to T1-T3 (homepage, top hubs, products/services)
- 15-30% to T4-T5 (authority magnets: guides, tools, supporting articles)
- 0% to term/taxonomy pages (get authority via internal links)
Typically 20-30 target pages per site.
### 2.2 Page Tier Assignment
| Tier | Pages | Links/Page Target | Description |
|------|-------|-------------------|-------------|
| **T1** | Homepage (1 page) | 10-30 | Brand authority anchor |
| **T2** | Top 40% hubs by search volume | 5-15 | Primary money pages |
| **T3** | Remaining hubs + products/services | 3-10 | Supporting money pages |
| **T4** | Supporting blog articles | 1-4 | Content authority |
| **T5** | Authority magnets (guides, tools) | 2-6 | Link bait pages |
**Tier Assignment Algorithm:**
1. Load SAGBlueprint → identify all published hub pages
2. Sort hubs by total cluster keyword search volume (from Keywords model)
3. Top 40% of hubs = T2, remaining hubs = T3
4. Products/services pages = T3
5. Supporting blog articles = T4
6. Content with `content_structure` in (`guide`, `comparison`, `listicle`) and high word count = T5
### 2.3 Country-Specific Strategy Profiles
Four pre-built profiles with different timelines, budgets, and quality thresholds:
| Parameter | Pakistan (PK) | Canada (CA) | UK | USA |
|-----------|--------------|-------------|-----|------|
| Timeline | 8 months | 12 months | 14 months | 18 months |
| Budget range | $2-5K | $3-7K | $3-9K | $5-13K |
| Target DR | 25-30 | 35-40 | 35-40 | 40-45 |
| Quality threshold | ≥5/11 | ≥6/11 | ≥6/11 | ≥7/11 |
| Exact match anchor | 5-10% | 3-7% | 3-7% | 2-5% |
| Velocity phases | ramp→peak→cruise→maintenance | Same 4 phases | Same | Same |
**Anchor Text Mix by Country:**
| Anchor Type | PK | CA | UK | USA |
|-------------|-----|-----|-----|------|
| Branded | 30-35% | 35-40% | 35-40% | 35-45% |
| Naked URL | 15-20% | 15-20% | 15-20% | 15-20% |
| Generic | 15-20% | 15-20% | 15-20% | 15-20% |
| Partial Match | 15-20% | 12-18% | 12-18% | 10-15% |
| Exact Match | 5-10% | 3-7% | 3-7% | 2-5% |
| LSI/Topical | 5-10% | 5-10% | 5-10% | 5-8% |
| Brand+Keyword | — | 3-5% | 3-5% | 3-5% |
### 2.4 Campaign Generation Algorithm
1. Load SAGBlueprint → identify all published hub pages
2. Assign tiers based on search volume data (from Keywords model)
3. Select country profile → calculate `referring_domains_needed`
4. `links_per_tier = referring_domains_needed × tier_allocation_%`
5. `budget_estimate = links × cost_per_link × link_mix_%`
6. Distribute across monthly velocity curve (ramp → peak → cruise → maintenance)
7. Assign pages to months by priority (keyword difficulty, search volume, commercial value)
8. Pre-generate 3 anchor text variants per page per anchor type
9. Set quality requirements per country threshold
### 2.5 Marketplace Integrations
#### FatGrid API
- **Base URL:** `https://api.fatgrid.com/api/public`
- **Auth:** API key in request header
- **Endpoints:**
- Domain Lookup — DR, DA, traffic, niche for a domain
- Marketplace Browse — filterable by DR, traffic, price, niche
- Bulk Domain Lookup — up to 1,000 domains per request
- **15+ Aggregated Marketplaces:** Collaborator.pro, PRNews.io, Adsy.com, WhitePress.com, Bazoom.com, MeUp.com, etc.
- **Usage:** IGNY8 proxies FatGrid API calls to find and filter link placement opportunities
#### PR Distribution (3 Tiers)
| Tier | Provider | Price Range | Reach |
|------|----------|-------------|-------|
| PR Basic | EIN Presswire | $99-499/release | AP News, Bloomberg, 115+ US TV |
| PR Premium | PRNews.io | $500-5K/placement | Yahoo Finance, Forbes-tier publications |
| PR Enterprise | Linking News (white-label) | $500-2K/distribution | ABC, NBC, FOX, Yahoo, Bloomberg |
### 2.6 Quality Scoring (Per Backlink Opportunity)
**Auto-Checkable Factors (7 points):**
1. Organic traffic >500/month
2. Domain Rating / Domain Authority > country threshold
3. Indexed in Google
4. Not on known PBN/spam farm blocklist
5. Traffic trend stable or growing
6. Niche relevance to content topic
7. Dofollow link confirmed
**Manual Review Factors (4 points):**
8. Outbound links <100 on linking page
9. Niche relevance (editorial check)
10. Editorial quality of surrounding content
11. Dofollow confirmed (manual verification)
**Total: 0-11 points. Country thresholds:** PK ≥5, CA ≥6, UK ≥6, USA ≥7
### 2.7 Authority Tipping Point Detection
Monitor for 3+ simultaneous indicators:
- Domain Rating reached country target
- Pages with GSC impressions >100 but 0 SAGBacklinks start getting organic clicks
- Un-linked pages rank on page 2-3 (positions 11-30)
- New content ranks passively without dedicated backlinks
- Keywords in top 10 exceed threshold: PK 10+, UK/CA 15+, USA 20+
**When triggered:** Recommend: reduce link-building velocity, shift budget to content creation, enter maintenance mode.
### 2.8 Dead Link Monitoring
- Periodic HTTP checks on all placed backlinks (status=`live`)
- Status tracking: `live` → `dead` (404/403/removed) → `replaced`
- Impact scoring: estimate authority loss based on source DR and link type
- Auto-generate replacement recommendations
- Reserve 10-15% monthly budget for replacements
---
## 3. DATA MODELS & APIS
### 3.1 New Models
All models in the `linker` app (same app as 02D).
#### SAGCampaign (linker app)
```python
class SAGCampaign(SiteSectorBaseModel):
"""
Backlink campaign generated from SAG data + country profile.
"""
blueprint = models.ForeignKey(
'planner.SAGBlueprint',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='backlink_campaigns'
)
country_code = models.CharField(
max_length=3,
help_text='PK, CA, UK, or USA'
)
status = models.CharField(
max_length=15,
choices=[
('draft', 'Draft'),
('active', 'Active'),
('paused', 'Paused'),
('completed', 'Completed'),
],
default='draft'
)
tier_assignments = models.JSONField(
default=dict,
help_text='{content_id: tier_level (T1/T2/T3/T4/T5)}'
)
total_links_target = models.IntegerField(default=0)
budget_estimate_min = models.DecimalField(
max_digits=10, decimal_places=2, default=0
)
budget_estimate_max = models.DecimalField(
max_digits=10, decimal_places=2, default=0
)
timeline_months = models.IntegerField(default=12)
monthly_plan = models.JSONField(
default=list,
help_text='[{month, links_target, pages[], budget}]'
)
anchor_text_plan = models.JSONField(
default=dict,
help_text='{content_id: [{text, type, allocated}]}'
)
country_profile = models.JSONField(
default=dict,
help_text='Full profile snapshot at campaign creation'
)
kpi_data = models.JSONField(
default=dict,
help_text='Monthly KPI snapshots summary'
)
started_at = models.DateTimeField(null=True, blank=True)
class Meta:
app_label = 'linker'
db_table = 'igny8_sag_campaigns'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### SAGBacklink (linker app)
```python
class SAGBacklink(SiteSectorBaseModel):
"""
Individual backlink record within a campaign.
Tracks from planning through placement to ongoing monitoring.
"""
campaign = models.ForeignKey(
'linker.SAGCampaign',
on_delete=models.CASCADE,
related_name='backlinks'
)
blueprint = models.ForeignKey(
'planner.SAGBlueprint',
on_delete=models.SET_NULL,
null=True,
blank=True
)
target_content = models.ForeignKey(
'writer.Content',
on_delete=models.CASCADE,
related_name='backlinks'
)
target_url = models.URLField()
target_tier = models.CharField(
max_length=3,
choices=[
('T1', 'Tier 1 — Homepage'),
('T2', 'Tier 2 — Top Hubs'),
('T3', 'Tier 3 — Other Hubs/Products'),
('T4', 'Tier 4 — Supporting Articles'),
('T5', 'Tier 5 — Authority Magnets'),
]
)
source_url = models.URLField(
blank=True,
default='',
help_text='May not be known at planning stage'
)
source_domain = models.CharField(max_length=255, blank=True, default='')
source_dr = models.IntegerField(null=True, blank=True)
source_traffic = models.IntegerField(null=True, blank=True)
anchor_text = models.CharField(max_length=200)
anchor_type = models.CharField(
max_length=20,
choices=[
('branded', 'Branded'),
('naked_url', 'Naked URL'),
('generic', 'Generic'),
('partial_match', 'Partial Match'),
('exact_match', 'Exact Match'),
('lsi', 'LSI/Topical'),
('brand_keyword', 'Brand + Keyword'),
]
)
link_type = models.CharField(
max_length=20,
choices=[
('guest_post', 'Guest Post'),
('niche_edit', 'Niche Edit'),
('pr_distribution', 'PR Distribution'),
('directory', 'Directory'),
('resource_page', 'Resource Page'),
('broken_link', 'Broken Link Building'),
('haro', 'HARO/Journalist Query'),
]
)
marketplace = models.CharField(
max_length=20,
blank=True,
default='',
help_text='fatgrid, prnews, ein, linking_news, manual'
)
cost = models.DecimalField(
max_digits=10, decimal_places=2, null=True, blank=True
)
quality_score = models.FloatField(
null=True,
blank=True,
help_text='0-11 quality score'
)
country_relevant = models.BooleanField(default=True)
date_ordered = models.DateField(null=True, blank=True)
date_live = models.DateField(null=True, blank=True)
date_last_checked = models.DateField(null=True, blank=True)
status = models.CharField(
max_length=15,
choices=[
('planned', 'Planned'),
('ordered', 'Ordered'),
('live', 'Live'),
('dead', 'Dead'),
('replaced', 'Replaced'),
('rejected', 'Rejected'),
],
default='planned'
)
notes = models.TextField(blank=True, default='')
class Meta:
app_label = 'linker'
db_table = 'igny8_sag_backlinks'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### CampaignKPISnapshot (linker app)
```python
class CampaignKPISnapshot(SiteSectorBaseModel):
"""
Monthly KPI snapshot for a backlink campaign.
Tracks domain metrics, link counts, keyword rankings, and tipping point indicators.
"""
campaign = models.ForeignKey(
'linker.SAGCampaign',
on_delete=models.CASCADE,
related_name='kpi_snapshots'
)
snapshot_date = models.DateField()
dr = models.FloatField(null=True, blank=True, help_text='Domain Rating')
da = models.FloatField(null=True, blank=True, help_text='Domain Authority')
referring_domains = models.IntegerField(default=0)
new_links_this_month = models.IntegerField(default=0)
links_by_tier = models.JSONField(default=dict, help_text='{T1: count, T2: count, ...}')
cost_this_month = models.DecimalField(max_digits=10, decimal_places=2, default=0)
cost_per_link_avg = models.DecimalField(max_digits=10, decimal_places=2, default=0)
keywords_top_10 = models.IntegerField(default=0)
keywords_top_20 = models.IntegerField(default=0)
keywords_top_50 = models.IntegerField(default=0)
organic_traffic = models.IntegerField(
null=True, blank=True,
help_text='From GSC via 02C GSCMetricsCache'
)
impressions = models.IntegerField(
null=True, blank=True,
help_text='From GSC via 02C GSCMetricsCache'
)
pages_ranking_without_backlinks = models.IntegerField(default=0)
tipping_point_indicators = models.JSONField(
default=dict,
help_text='{indicator_name: True/False}'
)
tipping_point_triggered = models.BooleanField(default=False)
class Meta:
app_label = 'linker'
db_table = 'igny8_campaign_kpi_snapshots'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
### 3.2 Migration
```
igny8_core/migrations/XXXX_add_backlink_models.py
```
**Operations:**
1. `CreateModel('SAGCampaign', ...)` — with index on country_code, status
2. `CreateModel('SAGBacklink', ...)` — with indexes on campaign, status, target_content
3. `CreateModel('CampaignKPISnapshot', ...)` — with index on campaign, snapshot_date
### 3.3 API Endpoints
All endpoints under `/api/v1/linker/` (extending the linker URL namespace from 02D):
#### Campaign Management
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/campaigns/?site_id=X` | List backlink campaigns |
| POST | `/api/v1/linker/campaigns/generate/` | AI-generate campaign from blueprint + country. Body: `{site_id, blueprint_id, country_code}`. |
| GET | `/api/v1/linker/campaigns/{id}/` | Campaign detail with monthly plan |
| PUT | `/api/v1/linker/campaigns/{id}/` | Update campaign (adjust plan, budget) |
| POST | `/api/v1/linker/campaigns/{id}/activate/` | Start campaign (set status=active, started_at) |
| POST | `/api/v1/linker/campaigns/{id}/pause/` | Pause active campaign |
#### KPI Tracking
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/campaigns/{id}/kpi/` | KPI snapshot timeline for campaign |
| POST | `/api/v1/linker/campaigns/{id}/kpi/snapshot/` | Record monthly KPI. Body: `{dr, da, referring_domains, ...}`. |
| GET | `/api/v1/linker/tipping-point/?campaign_id=X` | Tipping point analysis — current indicator state |
#### Backlink Records
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/backlinks/?campaign_id=X` | List backlinks with filters (status, tier, anchor_type) |
| POST | `/api/v1/linker/backlinks/` | Add backlink record. Body: `{campaign_id, target_content_id, ...}`. |
| PUT | `/api/v1/linker/backlinks/{id}/` | Update backlink status/details |
| POST | `/api/v1/linker/backlinks/check/` | Trigger dead link check for campaign. Body: `{campaign_id}`. |
#### Marketplace Proxy
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/linker/marketplace/search/` | FatGrid marketplace search. Query params: `dr_min, traffic_min, price_max, niche`. |
| GET | `/api/v1/linker/marketplace/domain/{domain}/` | FatGrid domain lookup — DR, DA, traffic, niche. |
**Permissions:** All endpoints use `SiteSectorModelViewSet` permission patterns.
### 3.4 Campaign Generation Service
**Location:** `igny8_core/business/campaign_generation.py`
```python
class CampaignGenerationService:
"""
Generates a backlink campaign from SAG data + country profile.
"""
COUNTRY_PROFILES = {
'PK': {
'timeline_months': 8,
'budget_min': 2000, 'budget_max': 5000,
'target_dr': 25, 'quality_threshold': 5,
'exact_match_max': 0.10,
'tier_allocation': {'T1': 0.25, 'T2': 0.35, 'T3': 0.25, 'T4': 0.10, 'T5': 0.05},
# ... full profile
},
'CA': { ... },
'UK': { ... },
'USA': { ... },
}
def generate(self, site_id, blueprint_id, country_code):
"""
1. Load blueprint → published hub pages
2. Assign tiers by search volume
3. Select country profile
4. Calculate total links needed + budget
5. Build monthly plan with velocity curve
6. Pre-generate anchor text variants
7. Create SAGCampaign record
Returns SAGCampaign instance.
"""
pass
def _assign_tiers(self, blueprint, published_content):
"""Sort hubs by search volume, assign T1-T5."""
pass
def _build_monthly_plan(self, tier_assignments, country_profile):
"""Distribute links across months using velocity curve."""
pass
def _generate_anchor_plans(self, tier_assignments, country_profile):
"""Pre-generate 3 anchor text variants per page per type."""
pass
```
### 3.5 Quality Scoring Service
**Location:** `igny8_core/business/backlink_quality.py`
```python
class BacklinkQualityService:
"""
Scores backlink opportunities on 0-11 scale.
"""
def score(self, domain_data, country_code):
"""
Auto-check 7 factors:
1. Organic traffic >500/mo
2. DR/DA > country threshold
3. Indexed in Google
4. Not on blocklist
5. Traffic trend stable/growing
6. Niche relevance
7. Dofollow confirmed
Returns (score, breakdown).
"""
pass
def meets_threshold(self, score, country_code):
"""Check if score meets country minimum."""
pass
```
### 3.6 Tipping Point Detector
**Location:** `igny8_core/business/tipping_point.py`
```python
class TippingPointDetector:
"""
Monitors campaign KPI snapshots for authority tipping point indicators.
"""
def evaluate(self, campaign_id):
"""
Check 5 indicators against latest KPI data + GSC data.
If 3+ triggered, set tipping_point_triggered=True.
Returns {indicators: {name: bool}, triggered: bool, recommendation: str}
"""
pass
```
### 3.7 FatGrid API Client
**Location:** `igny8_core/integration/fatgrid_client.py`
```python
class FatGridClient:
"""
Client for FatGrid marketplace API.
API key stored in SiteIntegration or account-level settings.
"""
BASE_URL = 'https://api.fatgrid.com/api/public'
def __init__(self, api_key):
self.api_key = api_key
def search_marketplace(self, dr_min=None, traffic_min=None,
price_max=None, niche=None, limit=50):
"""Browse marketplace with filters."""
pass
def lookup_domain(self, domain):
"""Get DR, DA, traffic, niche for a domain."""
pass
def bulk_lookup(self, domains):
"""Lookup up to 1,000 domains."""
pass
```
### 3.8 Celery Tasks
**Location:** `igny8_core/tasks/backlink_tasks.py`
```python
@shared_task(name='check_backlink_status')
def check_backlink_status(campaign_id):
"""Weekly HTTP check on all 'live' backlinks. Updates status to 'dead' if 4xx/5xx."""
pass
@shared_task(name='record_kpi_snapshot')
def record_kpi_snapshot(campaign_id):
"""Monthly KPI recording: pull GSC data, calculate keyword rankings."""
pass
@shared_task(name='evaluate_tipping_point')
def evaluate_tipping_point(campaign_id):
"""Monthly, after KPI snapshot. Check tipping point indicators."""
pass
@shared_task(name='generate_replacement_recommendations')
def generate_replacement_recommendations(campaign_id):
"""Triggered when dead links detected. Generate replacement suggestions."""
pass
```
**Beat Schedule Additions:**
| Task | Schedule | Notes |
|------|----------|-------|
| `check_backlink_status` | Weekly (Thursday 3:00 AM) | HTTP check all live backlinks |
| `record_kpi_snapshot` | Monthly (1st of month, 5:00 AM) | Record KPI for all active campaigns |
| `evaluate_tipping_point` | Monthly (1st of month, 6:00 AM) | After KPI snapshot |
---
## 4. IMPLEMENTATION STEPS
### Step 1: Models
1. Create SAGCampaign, SAGBacklink, CampaignKPISnapshot in linker app (extends 02D)
2. Run migration
### Step 2: Country Profiles
1. Define 4 country profiles (PK, CA, UK, USA) as configuration data
2. Store as constants in `CampaignGenerationService` or as a seed data migration
### Step 3: Services
1. Implement `CampaignGenerationService` in `igny8_core/business/campaign_generation.py`
2. Implement `BacklinkQualityService` in `igny8_core/business/backlink_quality.py`
3. Implement `TippingPointDetector` in `igny8_core/business/tipping_point.py`
4. Implement `FatGridClient` in `igny8_core/integration/fatgrid_client.py`
### Step 4: API Endpoints
1. Add campaign, backlink, KPI, and marketplace endpoints to `igny8_core/urls/linker.py`
2. Create views: `CampaignViewSet`, `BacklinkViewSet`, `KPISnapshotView`, `TippingPointView`
3. Create `MarketplaceSearchView`, `MarketplaceDomainLookupView` (FatGrid proxy)
### Step 5: Celery Tasks
1. Implement 4 tasks in `igny8_core/tasks/backlink_tasks.py`
2. Add to Celery beat schedule (weekly backlink check, monthly KPI + tipping point)
### Step 6: Serializers & Admin
1. Create DRF serializers for SAGCampaign, SAGBacklink, CampaignKPISnapshot
2. Register models in Django admin
### Step 7: Credit Cost Configuration
Add to `CreditCostConfig` (billing app):
| operation_type | default_cost | description |
|---------------|-------------|-------------|
| `campaign_generation` | 2 | AI-generate campaign from blueprint + country |
| `backlink_audit` | 1 | Dead link check for campaign |
| `kpi_snapshot` | 0.5 | Monthly KPI recording |
| `dead_link_detection` | 1 | Dead link detection + replacement recommendations |
**Note:** FatGrid API calls consume FatGrid credits (separate billing — not IGNY8 credits). Store FatGrid API key in account-level settings or SiteIntegration.
---
## 5. ACCEPTANCE CRITERIA
### Campaign Generation
- [ ] Campaign generated from SAGBlueprint + country code
- [ ] Page tiers assigned based on search volume from Keywords model
- [ ] Budget estimates calculated from country profile + tier allocation
- [ ] Monthly plan distributed across velocity curve (ramp → peak → cruise → maintenance)
- [ ] Anchor text plan pre-generated with 3 variants per page per anchor type
- [ ] Country profile snapshot stored in campaign record
### Country Profiles
- [ ] All 4 profiles (PK, CA, UK, USA) defined with correct parameters
- [ ] Anchor text mix enforced per country (branded %, exact match %, etc.)
- [ ] Quality thresholds enforced per country (PK ≥5, CA ≥6, UK ≥6, USA ≥7)
- [ ] Timeline and budget ranges match specification
### Quality Scoring
- [ ] 7 auto-checkable factors scored per backlink opportunity
- [ ] Country-specific threshold enforcement
- [ ] Quality score stored on SAGBacklink record
### Tipping Point
- [ ] 5 indicators monitored from KPI data + GSC data
- [ ] Triggered when 3+ indicators simultaneous
- [ ] Recommendation generated when triggered (reduce velocity, shift to content)
- [ ] tipping_point_triggered flag set on KPI snapshot
### Dead Link Monitoring
- [ ] Weekly HTTP checks on all live backlinks
- [ ] Status transitions: live → dead, with date tracking
- [ ] Replacement recommendations generated for dead links
- [ ] 10-15% budget reserve recommended in campaign plan
### Marketplace Integration
- [ ] FatGrid marketplace search proxied through IGNY8 API
- [ ] FatGrid domain lookup returns DR, DA, traffic, niche
- [ ] API key stored securely (not exposed to frontend)
---
## 6. CLAUDE CODE INSTRUCTIONS
### File Locations
```
igny8_core/
├── modules/
│ └── linker/
│ └── models.py # Add SAGCampaign, SAGBacklink, CampaignKPISnapshot
├── business/
│ ├── campaign_generation.py # CampaignGenerationService
│ ├── backlink_quality.py # BacklinkQualityService
│ └── tipping_point.py # TippingPointDetector
├── integration/
│ └── fatgrid_client.py # FatGridClient
├── tasks/
│ └── backlink_tasks.py # Celery tasks
├── urls/
│ └── linker.py # Extend with campaign/backlink endpoints
└── migrations/
└── XXXX_add_backlink_models.py
```
### Conventions
- **PKs:** BigAutoField (integer) — do NOT use UUIDs
- **Table prefix:** `igny8_` on all new tables
- **App label:** `linker` (same app as 02D)
- **Celery app name:** `igny8_core`
- **URL pattern:** `/api/v1/linker/campaigns/...`, `/api/v1/linker/backlinks/...`, `/api/v1/linker/marketplace/...`
- **Permissions:** Use `SiteSectorModelViewSet` permission pattern
- **Model inheritance:** All new models extend `SiteSectorBaseModel`
- **Frontend:** `.tsx` files with Zustand stores
### Cross-References
| Doc | Relationship |
|-----|-------------|
| **02D** | Internal linking distributes authority from hub pages (backlink targets) to supporting content |
| **02C** | GSC data feeds KPIs (organic traffic, impressions, keyword positions) |
| **01A** | SAGBlueprint provides cluster hierarchy for tier assignment |
| **04A** | Managed services package includes backlink campaign management for clients |
| **01G** | SAG health monitoring can incorporate backlink campaign progress |
### Key Decisions
1. **Same `linker` app as 02D** — Internal and external linking share the same app since they're conceptually related; campaigns reference the same Content and Blueprint models
2. **Country profiles as code constants** — Stored in CampaignGenerationService class, not database, to prevent accidental modification; versioned with code
3. **FatGrid proxy** — Never expose FatGrid API key to frontend; all marketplace calls routed through IGNY8 backend
4. **KPI snapshots are manual + automatic** — Monthly auto-recording via Celery, but manual recording also supported for ad-hoc updates
5. **Separate billing for marketplace** — FatGrid credits are external; IGNY8 credits cover campaign generation, audits, and AI-powered anchor text generation

602
v2/V2-Execution-Docs/02F-optimizer.md Normal file
View File

@@ -0,0 +1,602 @@
# IGNY8 Phase 2: Content Optimizer (02F)
## Cluster-Aligned Content Optimization Engine
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Optimization App Today
The `optimization` Django app exists in `INSTALLED_APPS` but is **inactive** (behind feature flag). The following exist:
- **`OptimizationTask` model** — exists with minimal fields (basic task tracking only)
- **`optimize_content` AI function** — registered in `igny8_core/ai/registry.py` as one of the 7 registered functions, but only does basic content rewriting without cluster awareness, keyword coverage analysis, or scoring
- **`optimization` app label** — app exists at `igny8_core/modules/optimization/`
### What Does Not Exist
- No cluster-alignment during optimization
- No keyword coverage analysis against cluster keyword sets
- No heading restructure logic
- No intent-based content rewrite
- No schema gap detection
- No before/after scoring system (0-100)
- No batch optimization
- No integration with SAG data (01A) or taxonomy terms (02B)
### Foundation Available
- `Clusters` model (app_label=`planner`, db_table=`igny8_clusters`) with cluster keywords
- `Keywords` model (app_label=`planner`, db_table=`igny8_keywords`) linked to clusters
- `Content.schema_markup` JSONField — used by 02G for JSON-LD
- `Content.content_type` and `Content.content_structure` — routing context
- `Content.structured_data` JSONField (added by 02A)
- `ContentTaxonomy` cluster mapping (added by 02B) with `mapping_confidence`
- `GSCMetricsCache` (added by 02C) — position data identifies pages needing optimization
- `SchemaValidationService` (added by 02G) — schema gap detection reuse
- `BaseAIFunction` with `validate()`, `prepare()`, `build_prompt()`, `parse_response()`, `save_output()`
---
## 2. WHAT TO BUILD
### Overview
Extend the existing `OptimizationTask` model and `optimize_content` AI function into a full cluster-aligned optimization engine. The system analyzes content against its cluster's keyword set, scores quality on a 0-100 scale, and produces optimized content with tracked before/after metrics.
### 2.1 Cluster Matching (Auto-Assign Optimization Context)
When content has no cluster assignment, the optimizer auto-detects the best-fit cluster:
**Scoring Algorithm:**
- Keyword overlap (40%): count of cluster keywords found in content title + headings + body
- Semantic similarity (40%): AI-scored relevance between content topic and cluster theme
- Title match (20%): similarity between content title and cluster name/keywords
**Thresholds:**
- Confidence ≥ 0.6 → auto-assign cluster
- Confidence < 0.6 → flag for manual review, suggest top 3 candidates
This reuses the same scoring pattern as `ClusterMappingService` from 02B.
### 2.2 Keyword Coverage Analysis
For content with an assigned cluster:
1. Load all `Keywords` records belonging to that cluster
2. Scan `content_html` for each keyword: exact match, partial match (stemmed), semantic presence
3. Report per keyword: `{keyword, target_density, current_density, status: present|missing|low_density}`
4. Coverage targets:
- Hub content (`cluster_hub`): 70%+ of cluster keywords covered
- Supporting articles: 40%+ of cluster keywords covered
- Product/service pages: 30%+ (focused on commercial keywords)
### 2.3 Heading Restructure
Analyze H1/H2/H3 hierarchy for SEO best practices:
| Check | Rule | Fix |
|-------|------|-----|
| Single H1 | Content must have exactly one H1 | Merge or demote extra H1s |
| H2 keyword coverage | H2s should contain target keywords from cluster | AI rewrites H2s with keyword incorporation |
| Logical hierarchy | No skipped levels (H1 → H3 without H2) | Insert missing levels |
| H2 count | Minimum 3 H2s for content >1000 words | AI suggests additional H2 sections |
| Missing keyword themes | Cluster keywords not represented in any heading | AI suggests new H2/H3 sections for missing themes |
### 2.4 Content Rewrite (Intent-Aligned)
**Intent Classification:**
- **Informational**: expand explanations, add examples, increase depth, add definitions
- **Commercial**: add comparison tables, pros/cons, feature highlights, trust signals
- **Transactional**: strengthen CTAs, add urgency, streamline conversion path, social proof
**Content Adjustments:**
- Expand thin content (<500 words) to minimum viable length for the content structure
- Compress bloated content (detect and remove redundancy)
- Add missing sections identified by keyword coverage analysis
- Maintain existing tone and style while improving SEO alignment
### 2.5 Schema Gap Detection
Leverages `SchemaValidationService` from 02G:
1. Check existing `Content.schema_markup` against expected schemas for the content type
2. Expected schema by type: Article (post), Product (product), Service (service_page), FAQPage (if FAQ detected), BreadcrumbList (all), HowTo (if steps detected)
3. Identify missing required fields per schema type
4. Generate corrected/complete schema JSON-LD
5. Schema-only optimization mode available (no content rewrite, just schema fix)
### 2.6 Before/After Scoring
**Content Quality Score (0-100):**
| Factor | Weight | Score Criteria |
|--------|--------|---------------|
| Keyword Coverage | 30% | % of cluster keywords present vs target |
| Heading Structure | 20% | Single H1, keyword H2s, logical hierarchy, no skipped levels |
| Content Depth | 20% | Word count vs structure minimum, section completeness, detail level |
| Readability | 15% | Sentence length, paragraph length, Flesch-Kincaid approximation |
| Schema Completeness | 15% | Required schema fields present, validation passes |
Every optimization records `score_before` and `score_after`. Dashboard aggregates show average improvement across all optimizations.
### 2.7 Batch Optimization
- Select content by: cluster ID, score threshold (e.g., all content scoring < 50), content type, date range
- Queue as Celery tasks with priority ordering (lowest scores first)
- Concurrency: max 3 concurrent optimization tasks per account
- Progress tracking via OptimizationTask status field
- Cancel capability: change status to `rejected` to stop processing
---
## 3. DATA MODELS & APIS
### 3.1 Modified Model — OptimizationTask (optimization app)
Extend the existing `OptimizationTask` model with 16 new fields:
```python
# Add to existing OptimizationTask model:
content = models.ForeignKey(
'writer.Content',
on_delete=models.CASCADE,
related_name='optimization_tasks'
)
primary_cluster = models.ForeignKey(
'planner.Clusters',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='optimization_tasks'
)
secondary_clusters = models.JSONField(
default=list,
blank=True,
help_text='List of Clusters IDs for secondary relevance'
)
keyword_targets = models.JSONField(
default=list,
blank=True,
help_text='[{keyword, target_density, current_density, status}]'
)
optimization_type = models.CharField(
max_length=20,
choices=[
('full_rewrite', 'Full Rewrite'),
('heading_only', 'Heading Only'),
('schema_only', 'Schema Only'),
('keyword_coverage', 'Keyword Coverage'),
('batch', 'Batch'),
],
default='full_rewrite'
)
intent_classification = models.CharField(
max_length=15,
choices=[
('informational', 'Informational'),
('commercial', 'Commercial'),
('transactional', 'Transactional'),
],
blank=True,
default=''
)
score_before = models.FloatField(null=True, blank=True)
score_after = models.FloatField(null=True, blank=True)
content_before = models.TextField(
blank=True,
default='',
help_text='Snapshot of original content_html'
)
content_after = models.TextField(
blank=True,
default='',
help_text='Optimized HTML (null until optimization completes)'
)
metadata_before = models.JSONField(
default=dict,
blank=True,
help_text='{meta_title, meta_description, headings[]}'
)
metadata_after = models.JSONField(
default=dict,
blank=True
)
schema_before = models.JSONField(default=dict, blank=True)
schema_after = models.JSONField(default=dict, blank=True)
structure_changes = models.JSONField(
default=list,
blank=True,
help_text='[{change_type, description, before, after}]'
)
confidence_score = models.FloatField(
null=True,
blank=True,
help_text='AI confidence in the quality of changes (0-1)'
)
applied = models.BooleanField(default=False)
applied_at = models.DateTimeField(null=True, blank=True)
```
**Update STATUS choices on OptimizationTask:**
```python
STATUS_CHOICES = [
('pending', 'Pending'),
('analyzing', 'Analyzing'),
('optimizing', 'Optimizing'),
('review', 'Ready for Review'),
('applied', 'Applied'),
('rejected', 'Rejected'),
]
```
**PK:** BigAutoField (integer) — existing model
**Table:** existing `igny8_optimization_tasks` table (no rename needed)
### 3.2 Migration
Single migration in the optimization app (or igny8_core migrations):
```
igny8_core/migrations/XXXX_extend_optimization_task.py
```
**Operations:**
1. `AddField('OptimizationTask', 'content', ...)` — FK to Content
2. `AddField('OptimizationTask', 'primary_cluster', ...)` — FK to Clusters
3. `AddField('OptimizationTask', 'secondary_clusters', ...)` — JSONField
4. `AddField('OptimizationTask', 'keyword_targets', ...)` — JSONField
5. `AddField('OptimizationTask', 'optimization_type', ...)` — CharField
6. `AddField('OptimizationTask', 'intent_classification', ...)` — CharField
7. `AddField('OptimizationTask', 'score_before', ...)` — FloatField
8. `AddField('OptimizationTask', 'score_after', ...)` — FloatField
9. `AddField('OptimizationTask', 'content_before', ...)` — TextField
10. `AddField('OptimizationTask', 'content_after', ...)` — TextField
11. `AddField('OptimizationTask', 'metadata_before', ...)` — JSONField
12. `AddField('OptimizationTask', 'metadata_after', ...)` — JSONField
13. `AddField('OptimizationTask', 'schema_before', ...)` — JSONField
14. `AddField('OptimizationTask', 'schema_after', ...)` — JSONField
15. `AddField('OptimizationTask', 'structure_changes', ...)` — JSONField
16. `AddField('OptimizationTask', 'confidence_score', ...)` — FloatField
17. `AddField('OptimizationTask', 'applied', ...)` — BooleanField
18. `AddField('OptimizationTask', 'applied_at', ...)` — DateTimeField
### 3.3 API Endpoints
All endpoints under `/api/v1/optimizer/`:
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/optimizer/analyze/` | Analyze single content piece. Body: `{content_id}`. Returns scores + keyword coverage + heading analysis + recommendations. Does NOT rewrite. |
| POST | `/api/v1/optimizer/optimize/` | Run full optimization. Body: `{content_id, optimization_type}`. Creates OptimizationTask, runs analysis + rewrite, returns preview. |
| POST | `/api/v1/optimizer/preview/` | Preview changes without creating task. Body: `{content_id}`. Returns diff-style output. |
| POST | `/api/v1/optimizer/apply/{id}/` | Apply optimized version. Copies `content_after` → `Content.content_html`, updates metadata, sets `applied=True`. |
| POST | `/api/v1/optimizer/reject/{id}/` | Reject optimization. Sets status=`rejected`, keeps original content. |
| POST | `/api/v1/optimizer/batch/` | Queue batch optimization. Body: `{site_id, cluster_id?, score_threshold?, content_type?, content_ids?}`. Returns batch task ID. |
| GET | `/api/v1/optimizer/tasks/?site_id=X` | List OptimizationTask records with filters (status, optimization_type, cluster_id, date range). |
| GET | `/api/v1/optimizer/tasks/{id}/` | Single optimization detail with full before/after data. |
| GET | `/api/v1/optimizer/tasks/{id}/diff/` | HTML diff view — visual comparison of content_before vs content_after. |
| GET | `/api/v1/optimizer/cluster-suggestions/?content_id=X` | Suggest best-fit cluster for unassigned content. Returns top 3 candidates with confidence scores. |
| POST | `/api/v1/optimizer/assign-cluster/` | Assign cluster to content. Body: `{content_id, cluster_id}`. Updates Content record. |
| GET | `/api/v1/optimizer/dashboard/?site_id=X` | Optimization stats: avg score improvement, count by status, top improved, lowest scoring content. |
**Permissions:** All endpoints use `SiteSectorModelViewSet` permission patterns.
### 3.4 AI Function — Enhanced optimize_content
Extend the existing registered `optimize_content` AI function:
**Registry key:** `optimize_content` (already registered — enhance, not replace)
**Location:** `igny8_core/ai/functions/optimize_content.py` (existing file)
```python
class OptimizeContentFunction(BaseAIFunction):
"""
Enhanced cluster-aligned content optimization.
Extends existing optimize_content with keyword coverage,
heading restructure, intent classification, and scoring.
"""
function_name = 'optimize_content'
def validate(self, content_id, optimization_type='full_rewrite', **kwargs):
# Verify content exists, has content_html
# Verify optimization_type is valid
pass
def prepare(self, content_id, optimization_type='full_rewrite', **kwargs):
# Load Content record
# Determine cluster (from Content or auto-match)
# Load cluster Keywords
# Analyze current keyword coverage
# Parse heading structure
# Classify intent
# Calculate score_before
# Snapshot content_before, metadata_before, schema_before
pass
def build_prompt(self):
# Build type-specific optimization prompt:
# - Include current content_html
# - Include cluster keywords with coverage status
# - Include heading analysis results
# - Include intent classification
# - Include optimization_type instructions:
# full_rewrite: all optimizations
# heading_only: heading restructure only
# schema_only: schema fix only (no content change)
# keyword_coverage: add missing keyword sections only
pass
def parse_response(self, response):
# Parse optimized HTML
# Parse updated metadata (meta_title, meta_description)
# Parse structure_changes list
# Parse confidence_score
pass
def save_output(self, parsed):
# Create OptimizationTask with all before/after data
# Calculate score_after
# Set status='review'
pass
```
### 3.5 Content Scoring Service
**Location:** `igny8_core/business/content_scoring.py`
```python
class ContentScoringService:
"""
Calculates Content Quality Score (0-100) using 5 weighted factors.
Used by optimizer for before/after and by dashboard for overview.
"""
WEIGHTS = {
'keyword_coverage': 0.30,
'heading_structure': 0.20,
'content_depth': 0.20,
'readability': 0.15,
'schema_completeness': 0.15,
}
def score(self, content_id, cluster_id=None):
"""
Calculate composite score for a content record.
Returns: {total: float, breakdown: {factor: score}}
"""
pass
def _score_keyword_coverage(self, content, cluster):
"""0-100: % of cluster keywords found in content."""
pass
def _score_heading_structure(self, content_html):
"""0-100: single H1, keyword H2s, no skipped levels, H2 count."""
pass
def _score_content_depth(self, content_html, content_structure):
"""0-100: word count vs minimum for structure type, section completeness."""
pass
def _score_readability(self, content_html):
"""0-100: avg sentence length, paragraph length, Flesch-Kincaid approx."""
pass
def _score_schema_completeness(self, content):
"""0-100: required schema fields present, from SchemaValidationService (02G)."""
pass
```
### 3.6 Keyword Coverage Analyzer
**Location:** `igny8_core/business/keyword_coverage.py`
```python
class KeywordCoverageAnalyzer:
"""
Analyzes content against cluster keyword set.
Returns per-keyword presence and overall coverage percentage.
"""
def analyze(self, content_id, cluster_id):
"""
Returns {
total_keywords: int,
covered: int,
missing: int,
coverage_pct: float,
keywords: [{keyword, target_density, current_density, status}]
}
"""
pass
def _extract_text(self, content_html):
"""Strip HTML, return plain text for analysis."""
pass
def _check_keyword(self, keyword, text):
"""Check for exact, partial (stemmed), and semantic presence."""
pass
```
### 3.7 Celery Tasks
**Location:** `igny8_core/tasks/optimization_tasks.py`
```python
@shared_task(name='run_optimization')
def run_optimization(optimization_task_id):
"""Process a single OptimizationTask. Called by API endpoints."""
pass
@shared_task(name='run_batch_optimization')
def run_batch_optimization(site_id, cluster_id=None, score_threshold=None,
content_type=None, content_ids=None, batch_size=10):
"""
Process batch of content for optimization.
Selects content matching filters, creates OptimizationTask per item,
processes sequentially with max 3 concurrent per account.
"""
pass
@shared_task(name='identify_optimization_candidates')
def identify_optimization_candidates(site_id, threshold=50):
"""
Weekly scan: find content with quality score below threshold.
Creates report, does NOT auto-optimize.
"""
pass
```
**Beat Schedule Addition:**
| Task | Schedule | Notes |
|------|----------|-------|
| `identify_optimization_candidates` | Weekly (Monday 4:00 AM) | Scans all sites, identifies low-scoring content |
---
## 4. IMPLEMENTATION STEPS
### Step 1: Migration
1. Add 16 new fields to `OptimizationTask` model
2. Update STATUS_CHOICES on OptimizationTask
3. Run migration
### Step 2: Services
1. Implement `ContentScoringService` in `igny8_core/business/content_scoring.py`
2. Implement `KeywordCoverageAnalyzer` in `igny8_core/business/keyword_coverage.py`
### Step 3: AI Function Enhancement
1. Extend `OptimizeContentFunction` in `igny8_core/ai/functions/optimize_content.py`
2. Add cluster-alignment, keyword coverage, heading analysis, intent classification, scoring
3. Maintain backward compatibility — existing `optimize_content` calls still work
### Step 4: API Endpoints
1. Add optimizer endpoints to `igny8_core/urls/optimizer.py` (or create if doesn't exist)
2. Create views: `AnalyzeView`, `OptimizeView`, `PreviewView`, `ApplyView`, `RejectView`, `BatchView`
3. Create `ClusterSuggestionsView`, `AssignClusterView`, `DashboardView`, `DiffView`
4. Register URL patterns under `/api/v1/optimizer/`
### Step 5: Celery Tasks
1. Implement `run_optimization`, `run_batch_optimization`, `identify_optimization_candidates`
2. Add `identify_optimization_candidates` to Celery beat schedule
### Step 6: Serializers & Admin
1. Update DRF serializer for extended OptimizationTask (include all 16 new fields)
2. Create nested serializers for before/after views
3. Update Django admin registration
### Step 7: Credit Cost Configuration
Add to `CreditCostConfig` (billing app):
| operation_type | default_cost | description |
|---------------|-------------|-------------|
| `optimization_analysis` | 2 | Analyze single content (scoring + keyword coverage) |
| `optimization_full_rewrite` | 5-8 | Full rewrite optimization (varies by content length) |
| `optimization_schema_only` | 1 | Schema gap fix only |
| `optimization_batch` | 15-25 | Batch optimization for 10 items |
Credit deduction follows existing `CreditUsageLog` pattern.
---
## 5. ACCEPTANCE CRITERIA
### Cluster Matching
- [ ] Content without cluster assignment gets auto-matched with confidence scoring
- [ ] Confidence ≥ 0.6 auto-assigns; < 0.6 flags for manual review with top 3 suggestions
- [ ] Cluster suggestions endpoint returns ranked candidates
### Keyword Coverage
- [ ] All cluster keywords analyzed for presence in content
- [ ] Coverage report includes exact match, partial match, and missing keywords
- [ ] Hub content targets 70%+, supporting articles 40%+, product/service 30%+
### Heading Restructure
- [ ] H1/H2/H3 hierarchy validated (single H1, no skipped levels)
- [ ] Missing keyword themes identified and new headings suggested
- [ ] AI rewrites headings incorporating target keywords while maintaining meaning
### Content Rewrite
- [ ] Intent classified correctly (informational/commercial/transactional)
- [ ] Rewrite adjusts content structure based on intent
- [ ] Thin content expanded, bloated content compressed
- [ ] Missing keyword sections added
### Scoring
- [ ] Score 0-100 calculated with 5 weighted factors
- [ ] score_before recorded before any changes
- [ ] score_after recorded after optimization
- [ ] Dashboard shows average improvement and distribution
### Before/After
- [ ] Full snapshot of original content preserved in content_before
- [ ] Optimized version stored in content_after without auto-applying
- [ ] Diff view provides visual HTML comparison
- [ ] Apply action copies content_after → Content.content_html
- [ ] Reject action preserves original, marks task rejected
### Batch
- [ ] Batch optimization selects content by cluster, score threshold, type, or explicit IDs
- [ ] Max 3 concurrent optimizations per account enforced
- [ ] Progress trackable via OptimizationTask status
- [ ] Weekly candidate identification runs without auto-optimizing
### Integration
- [ ] Schema gap detection leverages SchemaValidationService from 02G
- [ ] Credit costs deducted per CreditCostConfig entries
- [ ] All API endpoints respect account/site permission boundaries
---
## 6. CLAUDE CODE INSTRUCTIONS
### File Locations
```
igny8_core/
├── ai/
│ └── functions/
│ └── optimize_content.py # Enhance existing function
├── business/
│ ├── content_scoring.py # ContentScoringService
│ └── keyword_coverage.py # KeywordCoverageAnalyzer
├── tasks/
│ └── optimization_tasks.py # Celery tasks
├── urls/
│ └── optimizer.py # Optimizer endpoints
└── migrations/
└── XXXX_extend_optimization_task.py
```
### Conventions
- **PKs:** BigAutoField (integer) — do NOT use UUIDs
- **Table prefix:** `igny8_` (existing table `igny8_optimization_tasks`)
- **Celery app name:** `igny8_core`
- **URL pattern:** `/api/v1/optimizer/...`
- **Permissions:** Use `SiteSectorModelViewSet` permission pattern
- **AI functions:** Extend existing `BaseAIFunction` subclass — do NOT create a new registration key, enhance the existing `optimize_content`
- **Frontend:** `.tsx` files with Zustand stores for state management
### Cross-References
| Doc | Relationship |
|-----|-------------|
| **02B** | Taxonomy terms get cluster context for optimization; ClusterMappingService scoring pattern reused |
| **02G** | SchemaValidationService used for schema gap detection; schema_only optimization triggers 02G schema generation |
| **02C** | GSC position data identifies pages needing optimization (high impressions, low clicks) |
| **02D** | Optimizer identifies internal link opportunities and feeds them to linker |
| **01E** | Blueprint-aware pipeline sets initial content quality; optimizer improves post-generation |
| **01A** | SAGBlueprint/SAGCluster data provides cluster context for optimization |
| **01G** | SAG health monitoring can incorporate content quality scores as a health factor |
### Key Decisions
1. **Extend, don't replace** — The existing `OptimizationTask` model and `optimize_content` AI function are enhanced, not replaced with new models
2. **Preview-first workflow** — Optimizations always produce a preview (status=`review`) before applying to Content
3. **Content snapshot** — Full HTML snapshot stored in `content_before` for rollback capability
4. **Score reuse** — `ContentScoringService` is a standalone service usable by other modules (02G schema audit, 01G health monitoring)
5. **Schema delegation** — Schema gap detection reuses 02G's `SchemaValidationService` rather than duplicating logic

702
v2/V2-Execution-Docs/02G-rich-schema-serp.md Normal file
View File

@@ -0,0 +1,702 @@
# IGNY8 Phase 2: Rich Schema & SERP Enhancement (02G)
## JSON-LD Schema Generation & On-Page SERP Element Injection
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Schema Markup Today
The `Content` model (app_label=`writer`, db_table=`igny8_content`) has a `schema_markup` JSONField that stores raw JSON-LD. The AI function `generate_content` occasionally includes basic Article schema, but the output is inconsistent and unvalidated.
### What Works Now
- `Content.schema_markup` — JSONField exists, sometimes populated during generation
- `generate_content` AI function — may produce rudimentary Article schema as part of content output
- `ContentTypeTemplate` model (added by 02A) defines section layouts and presets per content type
- 02A added `Content.structured_data` JSONField for type-specific data (product specs, service steps, etc.)
### What Does Not Exist
- No systematic schema generation by content type
- No on-page SERP element injection (TL;DR, TOC, Key Takeaways, etc.)
- No schema validation against Google Rich Results requirements
- No retroactive enhancement of already-published content
- No SchemaTemplate model, no SERPEnhancement model, no validation records
- No SERP element tracking per content
### Phase 1 & 2A Foundation Available
- `SAGCluster.cluster_type` choices: `product_category`, `condition_problem`, `feature`, `brand`, `informational`, `comparison`
- 01E blueprint-aware pipeline provides `blueprint_context` with `cluster_type`, `content_structure`, `content_type`
- 02A content type routing provides type-specific generation with section layouts
- `Content.content_type` choices: `post`, `page`, `product`, `taxonomy`
- `Content.content_structure` choices: 14 structure types including `cluster_hub`, `product_page`, `service_page`, `comparison`, `review`
---
## 2. WHAT TO BUILD
### Overview
Build a schema generation and SERP enhancement system that:
1. Generates correct JSON-LD structured data for 10 schema types, mapped to content type/structure
2. Injects 8 on-page SERP elements into `content_html` to improve rich snippet eligibility
3. Validates schema against Google Rich Results requirements
4. Retroactively enhances existing published content with missing schema and SERP elements
### 2.1 JSON-LD Schema Types (10 Types)
Each schema type maps to specific `content_type` + `content_structure` combinations:
| # | Schema Type | Applies To | Key Fields |
|---|------------|-----------|------------|
| 1 | **Article / BlogPosting** | `post` (all structures) | headline, datePublished, dateModified, author (Person/Organization), publisher, image, description, mainEntityOfPage, wordCount, articleSection |
| 2 | **Product** | `product` / `product_page` | name, description, image, brand, offers (price, priceCurrency, availability, url), aggregateRating, review, sku, gtin |
| 3 | **Service** | `page` / `service_page` | name, description, provider (Organization), serviceType, areaServed, hasOfferCatalog, offers |
| 4 | **LocalBusiness** | Sites with physical location (site-level config) | name, address, telephone, openingHours, geo, image, priceRange, sameAs, hasMap |
| 5 | **Organization** | Site-wide (homepage schema) | name, url, logo, sameAs[], contactPoint, foundingDate, founders |
| 6 | **BreadcrumbList** | All pages | itemListElement [{position, name, item(URL)}] — auto-generated from SAG hierarchy or WP breadcrumb trail |
| 7 | **FAQPage** | Content with FAQ sections (auto-detected from H2/H3 question patterns) | mainEntity [{@type: Question, name, acceptedAnswer: {text}}] |
| 8 | **HowTo** | Step-by-step content (detected from ordered lists with process indicators) | name, step [{@type: HowToStep, name, text, image, url}], totalTime, estimatedCost |
| 9 | **VideoObject** | Content with video embeds (02I integration) | name, description, thumbnailUrl, uploadDate, duration, contentUrl, embedUrl |
| 10 | **WebSite + SearchAction** | Site-wide (homepage) | name, url, potentialAction (SearchAction with query-input) |
**Auto-Detection Rules:**
- FAQPage: detected when content has H2/H3 headings matching question patterns (starts with "What", "How", "Why", "When", "Is", "Can", "Does", "Should") or explicit `<div class="faq-section">` blocks
- HowTo: detected when content has ordered lists (`<ol>`) combined with process language ("Step 1", "First", "Next", etc.)
- VideoObject: detected when `<iframe>` or `<video>` tags present, or when 02I VideoProject is linked to content
- BreadcrumbList: always generated — uses SAG hierarchy (Site → Sector → Cluster → Content) or WordPress breadcrumb trail from SiteIntegration sync
**Schema Stacking:** A single content piece can have multiple schemas. An article with FAQ and video gets Article + FAQPage + VideoObject + BreadcrumbList — all in a single `<script type="application/ld+json">` array.
### 2.2 On-Page SERP Elements (8 Types)
SERP elements are HTML blocks injected into `content_html` to improve featured snippet and rich result eligibility:
| # | Element | Description | Insertion Point | Detection / Source |
|---|---------|-------------|----------------|-------------------|
| 1 | **TL;DR Box** | 2-3 sentence summary in styled box | Top of article, after first paragraph | AI-generated from content |
| 2 | **Table of Contents** | Auto-generated from H2/H3 headings with anchor links | After intro paragraph, before first H2 | Parsed from content headings |
| 3 | **Key Takeaways** | Bullet list of main points in styled box | After TL;DR or after intro | AI-generated from content |
| 4 | **Definition Boxes** | Highlighted term definitions | Inline, after first use of defined term | AI detects key terms + generates definitions |
| 5 | **Comparison Tables** | Structured HTML tables for comparison content | Within body, at relevant H2 section | AI-generated for `comparison`, `review` structures |
| 6 | **People Also Ask** | Related questions with expandable answers | Before conclusion or after last H2 | AI-generated from content + cluster keywords |
| 7 | **Statistics Callouts** | Visual callout boxes for key numbers/stats | Inline, wrapping existing stats in text | Regex detection of numbers/percentages in text |
| 8 | **Pro/Con Boxes** | Structured pros and cons for review/comparison content | Within body, at relevant section | AI-generated for `review`, `comparison`, `product_page` structures |
**SERP Element Applicability by Content Structure:**
| Structure | TL;DR | TOC | Key Takeaways | Definitions | Comparison | PAA | Stats | Pro/Con |
|-----------|-------|-----|---------------|-------------|------------|-----|-------|---------|
| `article` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
| `guide` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
| `comparison` | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
| `review` | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ |
| `listicle` | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ |
| `landing_page` | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ |
| `service_page` | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ |
| `product_page` | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ |
| `cluster_hub` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
### 2.3 Retroactive Enhancement Engine
For existing published content that was generated before this module:
1. **Scan Phase:** Query all Content records where `schema_markup` is empty/incomplete OR `serp_elements` is null/empty
2. **Priority Ordering:** Highest-traffic pages first (using GSC data from 02C `GSCMetricsCache` if available, otherwise by `created_at` DESC)
3. **Generate Phase:** For each content, determine applicable schema types + SERP elements based on `content_type`, `content_structure`, and HTML analysis
4. **Preview Mode:** Store generated schema and SERP HTML in model records without modifying Content — user reviews before applying
5. **Apply Phase:** On approval, update `Content.schema_markup` and inject SERP element HTML into `Content.content_html`
6. **Batch Processing:** Process 10 content items per Celery task, with configurable batch size
---
## 3. DATA MODELS & APIS
### 3.1 New Models
#### SchemaTemplate (writer app)
```python
class SchemaTemplate(AccountBaseModel):
"""
Reusable JSON-LD schema templates with placeholder fields.
Account-level: account admins can customize templates.
"""
schema_type = models.CharField(
max_length=30,
choices=[
('article', 'Article / BlogPosting'),
('product', 'Product'),
('service', 'Service'),
('localbusiness', 'LocalBusiness'),
('organization', 'Organization'),
('breadcrumb', 'BreadcrumbList'),
('faq', 'FAQPage'),
('howto', 'HowTo'),
('video', 'VideoObject'),
('website', 'WebSite + SearchAction'),
]
)
content_type_match = models.CharField(
max_length=20,
choices=CONTENT_TYPE_CHOICES,
help_text='Which content_type this template applies to'
)
content_structure_match = models.CharField(
max_length=30,
choices=CONTENT_STRUCTURE_CHOICES,
blank=True,
null=True,
help_text='Further filter by content_structure (null = any)'
)
template_json = models.JSONField(
help_text='JSON-LD template with {{placeholder}} fields'
)
required_fields = models.JSONField(
default=list,
help_text='List of required field paths for validation'
)
is_default = models.BooleanField(default=False)
class Meta:
app_label = 'writer'
db_table = 'igny8_schema_templates'
unique_together = [
('account', 'schema_type', 'content_type_match', 'content_structure_match')
]
```
**PK:** BigAutoField (integer) — inherits from AccountBaseModel
**Relationships:** account FK (from AccountBaseModel)
#### SERPEnhancement (writer app)
```python
class SERPEnhancement(SiteSectorBaseModel):
"""
Tracks individual SERP enhancement elements generated for content.
One record per enhancement type per content.
"""
ENHANCEMENT_TYPE_CHOICES = [
('tldr', 'TL;DR Box'),
('toc', 'Table of Contents'),
('key_takeaways', 'Key Takeaways'),
('definition', 'Definition Box'),
('comparison_table', 'Comparison Table'),
('paa', 'People Also Ask'),
('stats_callout', 'Statistics Callout'),
('pro_con', 'Pro/Con Box'),
]
content = models.ForeignKey(
'writer.Content',
on_delete=models.CASCADE,
related_name='serp_enhancement_records'
)
enhancement_type = models.CharField(max_length=20, choices=ENHANCEMENT_TYPE_CHOICES)
html_snippet = models.TextField(
help_text='Generated HTML block to inject into content_html'
)
insertion_point = models.CharField(
max_length=30,
help_text='Where in content: top, after_intro, before_h2_N, bottom'
)
status = models.CharField(
max_length=15,
choices=[
('generated', 'Generated'),
('inserted', 'Inserted'),
('removed', 'Removed'),
],
default='generated'
)
generated_at = models.DateTimeField(auto_now_add=True)
class Meta:
app_label = 'writer'
db_table = 'igny8_serp_enhancements'
unique_together = [('content', 'enhancement_type')]
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
**Relationships:** content FK → Content, site FK + sector FK + account FK (from SiteSectorBaseModel)
#### SchemaValidationResult (writer app)
```python
class SchemaValidationResult(SiteSectorBaseModel):
"""
Stores schema validation results per content per schema type.
"""
content = models.ForeignKey(
'writer.Content',
on_delete=models.CASCADE,
related_name='schema_validations'
)
schema_type = models.CharField(max_length=30)
is_valid = models.BooleanField(default=False)
errors = models.JSONField(default=list, help_text='List of validation error strings')
warnings = models.JSONField(default=list, help_text='List of validation warning strings')
validated_at = models.DateTimeField(auto_now_add=True)
class Meta:
app_label = 'writer'
db_table = 'igny8_schema_validation_results'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
### 3.2 Modified Models
#### Content (writer app) — add field
```python
# Add to Content model:
serp_elements = models.JSONField(
default=dict,
blank=True,
help_text='Tracks which SERP enhancements are active: {type: True/False}'
)
```
**Existing field used:** `Content.schema_markup` (JSONField) — now systematically populated by this module instead of ad-hoc AI output.
### 3.3 Migration
Single migration in writer app:
```
igny8_core/migrations/XXXX_add_schema_serp_models.py
```
**Operations:**
1. `CreateModel('SchemaTemplate', ...)` — with unique_together constraint
2. `CreateModel('SERPEnhancement', ...)` — with unique_together constraint
3. `CreateModel('SchemaValidationResult', ...)`
4. `AddField('Content', 'serp_elements', JSONField(default=dict, blank=True))`
### 3.4 API Endpoints
All endpoints under `/api/v1/writer/` — extends the existing writer app URL namespace.
#### Schema Generation
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/writer/schema/generate/` | Generate schema for single content. Body: `{content_id}`. Returns JSON-LD + updates `Content.schema_markup`. |
| POST | `/api/v1/writer/schema/validate/` | Validate existing schema against Google requirements. Body: `{content_id}`. Returns SchemaValidationResult. |
| POST | `/api/v1/writer/schema/batch-generate/` | Batch generate schema. Body: `{content_ids: [int], site_id}`. Queues Celery task. Returns task ID. |
| GET | `/api/v1/writer/schema/templates/` | List SchemaTemplate records. Query params: `account_id`, `schema_type`, `content_type_match`. |
| GET | `/api/v1/writer/schema/audit/?site_id=X` | Schema coverage audit — returns counts of content with/without schema per type. |
| POST | `/api/v1/writer/schema/retroactive/` | Trigger retroactive schema scan. Body: `{site_id, batch_size}`. Queues Celery task. |
#### SERP Enhancement
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/writer/serp/enhance/` | Generate SERP elements for single content. Body: `{content_id, element_types: []}`. Returns SERPEnhancement records. |
| POST | `/api/v1/writer/serp/batch-enhance/` | Batch enhancement. Body: `{content_ids: [int], site_id}`. Queues Celery task. |
| GET | `/api/v1/writer/serp/preview/{content_id}/` | Preview enhancements — returns modified HTML without applying. |
| POST | `/api/v1/writer/serp/apply/{content_id}/` | Apply enhancements — injects HTML into `Content.content_html` and updates `Content.serp_elements`. |
| POST | `/api/v1/writer/serp/remove/{content_id}/` | Remove specific SERP elements. Body: `{element_types: []}`. |
**Permissions:** All endpoints use `AccountModelViewSet` or `SiteSectorModelViewSet` permission patterns from existing codebase.
### 3.5 AI Functions
#### GenerateSchemaFunction (extends BaseAIFunction)
**Registry key:** `generate_schema`
**Location:** `igny8_core/ai/functions/generate_schema.py`
```python
class GenerateSchemaFunction(BaseAIFunction):
"""
Generates JSON-LD structured data for content.
Determines applicable schema types from content_type, content_structure,
and HTML analysis. Produces schema-stacked output.
"""
function_name = 'generate_schema'
def validate(self, content_id, **kwargs):
# Verify content exists and has content_html
pass
def prepare(self, content_id, **kwargs):
# Load Content, determine applicable schema types
# Load matching SchemaTemplate records
# Extract structured_data from Content (from 02A)
pass
def build_prompt(self):
# Include: content title, meta_description, content_html excerpt,
# content_type, content_structure, structured_data,
# schema template as example, required_fields list
pass
def parse_response(self, response):
# Parse JSON-LD array from AI response
# Validate against required_fields
pass
def save_output(self, parsed):
# Save to Content.schema_markup
# Create SchemaValidationResult records
pass
```
**Input:** `content_id` (int)
**Output:** JSON-LD array saved to `Content.schema_markup`
#### GenerateSERPElementsFunction (extends BaseAIFunction)
**Registry key:** `generate_serp_elements`
**Location:** `igny8_core/ai/functions/generate_serp_elements.py`
```python
class GenerateSERPElementsFunction(BaseAIFunction):
"""
Generates on-page SERP enhancement HTML for content.
Uses content structure and applicability matrix to determine which elements
to generate. Returns HTML snippets for each element.
"""
function_name = 'generate_serp_elements'
def validate(self, content_id, element_types=None, **kwargs):
# Verify content exists
# If element_types not specified, determine from applicability matrix
pass
def prepare(self, content_id, element_types=None, **kwargs):
# Load Content, parse content_html for headings/stats/terms
# Load cluster keywords for PAA generation
pass
def build_prompt(self):
# Per element type, build specific sub-prompts:
# - TL;DR: "Summarize in 2-3 sentences..."
# - Key Takeaways: "Extract 3-5 main points..."
# - PAA: "Generate 4-6 related questions..."
# - Definitions: "Identify key terms and define..."
# etc.
pass
def parse_response(self, response):
# Parse per-element HTML snippets from AI response
pass
def save_output(self, parsed):
# Create/update SERPEnhancement records per element
pass
```
**Input:** `content_id` (int), optional `element_types` (list of strings)
**Output:** SERPEnhancement records created, not yet injected into content_html
### 3.6 Schema Validation Service
**Location:** `igny8_core/business/schema_validation.py`
```python
class SchemaValidationService:
"""
Validates JSON-LD schema against Google Rich Results requirements.
Not just schema.org compliance — checks Google-specific required fields.
"""
GOOGLE_REQUIRED_FIELDS = {
'article': ['headline', 'datePublished', 'author', 'image', 'publisher'],
'product': ['name', 'image', 'offers'],
'service': ['name', 'description', 'provider'],
'localbusiness': ['name', 'address'],
'organization': ['name', 'url', 'logo'],
'breadcrumb': ['itemListElement'],
'faq': ['mainEntity'],
'howto': ['name', 'step'],
'video': ['name', 'description', 'thumbnailUrl', 'uploadDate'],
'website': ['name', 'url', 'potentialAction'],
}
def validate(self, content_id):
"""
Validate all schema_markup entries for a content record.
Returns list of SchemaValidationResult records.
"""
pass
def _validate_single(self, schema_json, schema_type):
"""
Validate a single schema entry against required fields.
Returns (is_valid, errors[], warnings[]).
"""
pass
def auto_fix(self, content_id):
"""
Attempt to fix common schema issues:
- Missing dateModified → copy from updated_at
- Missing image → use first image from Images model
- Missing publisher → use site/account Organization schema
"""
pass
```
### 3.7 SERP Element Injection Service
**Location:** `igny8_core/business/serp_injection.py`
```python
class SERPInjectionService:
"""
Injects SERP enhancement HTML snippets into content_html.
Handles insertion point resolution and collision avoidance.
"""
INSERTION_ORDER = [
'tldr', # After first paragraph
'toc', # After intro, before first H2
'key_takeaways', # After TL;DR or after intro
'definition', # Inline, after first use of term
'comparison_table', # Within body at relevant H2
'stats_callout', # Inline, wrapping existing stats
'pro_con', # Within body at relevant section
'paa', # Before conclusion or after last H2
]
def inject(self, content_id):
"""
Inject all 'generated' SERPEnhancement records into content_html.
Updates Content.content_html and Content.serp_elements tracking field.
Marks SERPEnhancement records as 'inserted'.
"""
pass
def remove(self, content_id, element_types):
"""
Remove specified SERP elements from content_html.
Each element is wrapped in <div data-serp-element="{type}"> for removal.
"""
pass
def preview(self, content_id):
"""
Return modified content_html with enhancements WITHOUT saving.
"""
pass
```
**SERP Element HTML Wrapping Convention:**
All injected elements are wrapped with a data attribute for identification:
```html
<div data-serp-element="tldr" class="igny8-serp-tldr">
<!-- Generated TL;DR content -->
</div>
```
This allows reliable removal/replacement without corrupting surrounding content.
---
## 4. IMPLEMENTATION STEPS
### Step 1: Migration & Models
1. Create `SchemaTemplate` model in writer app
2. Create `SERPEnhancement` model in writer app
3. Create `SchemaValidationResult` model in writer app
4. Add `serp_elements` JSONField to Content model
5. Run migration
### Step 2: Schema Templates Seed Data
Create default SchemaTemplate records for each of the 10 schema types:
| schema_type | content_type_match | content_structure_match | is_default |
|------------|-------------------|------------------------|------------|
| `article` | `post` | `null` (any) | True |
| `product` | `product` | `null` | True |
| `product` | `post` | `product_page` | True |
| `service` | `page` | `service_page` | True |
| `localbusiness` | `page` | `null` | True |
| `organization` | `page` | `business_page` | True |
| `breadcrumb` | `post` | `null` | True |
| `breadcrumb` | `page` | `null` | True |
| `breadcrumb` | `product` | `null` | True |
| `faq` | `post` | `null` | True |
| `howto` | `post` | `null` | True |
| `video` | `post` | `null` | True |
| `website` | `page` | `null` | True |
Seed via data migration or management command `seed_schema_templates`.
### Step 3: AI Functions
1. Implement `GenerateSchemaFunction` in `igny8_core/ai/functions/generate_schema.py`
2. Implement `GenerateSERPElementsFunction` in `igny8_core/ai/functions/generate_serp_elements.py`
3. Register both in `igny8_core/ai/registry.py`
### Step 4: Services
1. Implement `SchemaValidationService` in `igny8_core/business/schema_validation.py`
2. Implement `SERPInjectionService` in `igny8_core/business/serp_injection.py`
### Step 5: Pipeline Integration
Integrate schema generation into the content pipeline after Stage 4 (content generation):
```python
# In content generation pipeline (01E blueprint-aware-pipeline):
# After GenerateContentFunction completes:
def post_content_generation(content_id):
# Auto-generate schema based on content type
generate_schema_fn = registry.get('generate_schema')
generate_schema_fn.execute(content_id=content_id)
# Auto-generate applicable SERP elements
generate_serp_fn = registry.get('generate_serp_elements')
generate_serp_fn.execute(content_id=content_id)
# Inject SERP elements into content_html
injection_service = SERPInjectionService()
injection_service.inject(content_id)
```
### Step 6: API Endpoints
1. Add schema endpoints to `igny8_core/urls/writer.py`
2. Create `SchemaGenerateView`, `SchemaValidateView`, `SchemaBatchGenerateView`
3. Create `SERPEnhanceView`, `SERPBatchEnhanceView`, `SERPPreviewView`, `SERPApplyView`
4. Create `SchemaAuditView`, `SchemaRetroactiveView`
### Step 7: Celery Tasks
Register in `igny8_core/tasks/` and add beat schedule entries:
```python
# igny8_core/tasks/schema_tasks.py
@shared_task(name='generate_schema_for_content')
def generate_schema_for_content(content_id):
"""After content generation, auto-generate schema."""
pass
@shared_task(name='retroactive_schema_scan')
def retroactive_schema_scan(site_id, batch_size=10):
"""Scan existing content and generate missing schemas in batches."""
pass
@shared_task(name='validate_schemas_batch')
def validate_schemas_batch(site_id):
"""Periodic validation of all schemas for a site."""
pass
```
**Beat Schedule Additions:**
| Task | Schedule | Notes |
|------|----------|-------|
| `validate_schemas_batch` | Weekly (Sunday 3:00 AM) | Validates all schemas, creates SchemaValidationResult records |
### Step 8: Serializers & Admin
1. Create DRF serializers for SchemaTemplate, SERPEnhancement, SchemaValidationResult
2. Register models in Django admin for inspection
### Step 9: Credit Cost Configuration
Add to `CreditCostConfig` (billing app):
| operation_type | default_cost | description |
|---------------|-------------|-------------|
| `schema_generation` | 1 | Generate JSON-LD schema for one content |
| `serp_element_generation` | 0.5 | Generate one SERP element |
| `schema_validation` | 0.1 | Validate schema for one content |
| `schema_batch` | 8-12 | Batch generate for 10 items (varies by content) |
Credit deduction follows existing `CreditUsageLog` pattern: log entry created per operation with `operation_type`, `credits_used`, `content` FK.
---
## 5. ACCEPTANCE CRITERIA
### Schema Generation
- [ ] Article/BlogPosting schema generated for all `content_type='post'` content
- [ ] Product schema generated for `content_type='product'` and `content_structure='product_page'` content
- [ ] Service schema generated for `content_structure='service_page'` content
- [ ] BreadcrumbList schema generated for all content using SAG hierarchy
- [ ] FAQPage schema auto-detected and generated when content has question-pattern headings
- [ ] HowTo schema auto-detected and generated when content has step-by-step lists
- [ ] Schema stacking works — content with FAQ + Article gets both schemas in array
- [ ] All schemas pass SchemaValidationService checks for Google required fields
### SERP Enhancement
- [ ] TL;DR box generated and injected for applicable content structures
- [ ] Table of Contents auto-generated from H2/H3 headings with working anchor links
- [ ] Key Takeaways bullet list generated for applicable content
- [ ] People Also Ask section generated with 4-6 questions + answers
- [ ] Comparison Tables generated for comparison/review content
- [ ] Pro/Con boxes generated for review/product_page content
- [ ] All SERP elements wrapped in `<div data-serp-element="{type}">` for reliable removal
- [ ] SERP elements can be removed without corrupting content
- [ ] Applicability matrix enforced — no TL;DR on landing_page, etc.
### Retroactive Enhancement
- [ ] Retroactive scan identifies content missing schema by type
- [ ] Priority ordering by traffic (GSC data) or creation date
- [ ] Preview mode shows changes without modifying Content
- [ ] Batch processing handles 10 items per task run
- [ ] Applied enhancements update Content.schema_markup and Content.serp_elements
### Validation
- [ ] SchemaValidationResult records created for each validation run
- [ ] Validation checks Google-specific required fields (not just schema.org)
- [ ] Auto-fix resolves common issues (missing dateModified, image, publisher)
- [ ] Weekly batch validation catches schema drift
### Integration
- [ ] Schema generation triggers automatically after content generation in pipeline
- [ ] SERP elements generated and injected as part of pipeline flow
- [ ] Credit costs deducted per CreditCostConfig entries
- [ ] All API endpoints respect account/site permission boundaries
---
## 6. CLAUDE CODE INSTRUCTIONS
### File Locations
```
igny8_core/
├── ai/
│ └── functions/
│ ├── generate_schema.py # GenerateSchemaFunction
│ └── generate_serp_elements.py # GenerateSERPElementsFunction
├── business/
│ ├── schema_validation.py # SchemaValidationService
│ └── serp_injection.py # SERPInjectionService
├── tasks/
│ └── schema_tasks.py # Celery tasks
├── urls/
│ └── writer.py # Add schema + serp endpoints
└── migrations/
└── XXXX_add_schema_serp_models.py # Models + Content.serp_elements
```
### Conventions
- **PKs:** BigAutoField (integer) — do NOT use UUIDs
- **Table prefix:** `igny8_` on all new tables
- **Celery app name:** `igny8_core`
- **URL pattern:** `/api/v1/writer/schema/...` and `/api/v1/writer/serp/...`
- **Permissions:** Use `AccountModelViewSet` / `SiteSectorModelViewSet` patterns
- **AI functions:** Extend `BaseAIFunction` with `validate()`, `prepare()`, `build_prompt()`, `parse_response()`, `save_output()`
- **Registry:** Register new AI functions in `igny8_core/ai/registry.py`
- **Frontend:** `.tsx` files with Zustand stores for state management
### Cross-References
| Doc | Relationship |
|-----|-------------|
| **02A** | Content type determines which schema type to generate; ContentTypeTemplate section layouts inform schema field population |
| **02F** | Optimizer detects schema gaps and triggers schema generation/fix |
| **02I** | VideoObject schema generated for content with linked VideoProject |
| **03A** | WP plugin standalone mode has its own schema module — different from this IGNY8-native implementation |
| **03B** | Connected mode pushes schema to WordPress via bulk endpoint |
| **01E** | Pipeline integration — schema generation hooks after Stage 4 content generation |
| **01G** | SAG health monitoring can incorporate schema completeness as a health factor |
### Key Decisions
1. **Writer app, not separate app** — SchemaTemplate, SERPEnhancement, SchemaValidationResult all live in the `writer` app since they are tightly coupled to Content
2. **Schema stacking** — multiple schemas per content stored as JSON array in `Content.schema_markup`
3. **SERP element wrapping** — all injected HTML uses `data-serp-element` attribute for non-destructive add/remove
4. **Preview before apply** — retroactive enhancements always go through preview state
5. **Content.serp_elements tracking field** — JSONField dict `{type: True/False}` for fast lookups without querying SERPEnhancement table

782
v2/V2-Execution-Docs/02H-socializer.md Normal file
View File

@@ -0,0 +1,782 @@
# IGNY8 Phase 2: Socializer (02H)
## Native Social Media Posting & Engagement Engine — Stage 8
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Social Media Today
There is **no** social media functionality in IGNY8. No OAuth connections to any social platform exist. No social post generation, scheduling, or analytics. The automation pipeline ends at Stage 7 (publish to WordPress).
### What Exists
- 7-stage automation pipeline (01E) terminates at publish — no post-publish social distribution
- `SiteIntegration` model (integration app) — handles WordPress OAuth; pattern reusable for social OAuth
- `Content` model — stores the content that feeds social post generation
- `Images` model (writer app) — stores generated images that can be resized for social platforms
- `AutomationConfig` model (automation app) — per-site automation settings; extendable for social toggles
- Celery infrastructure with beat scheduler — ready for social scheduling tasks
### What Does Not Exist
- No social app or models
- No OAuth connections for LinkedIn, Twitter/X, Facebook, Instagram, TikTok
- No AI social post generation or platform adaptation
- No scheduling/calendar system
- No engagement tracking
- No Stage 8 pipeline integration
---
## 2. WHAT TO BUILD
### Overview
Build Stage 8 of the automation pipeline: a social media engine that connects to 5 platforms via OAuth, generates AI-adapted posts from published content, schedules posts with best-time optimization, and tracks engagement metrics.
### 2.1 Platform Support
| Platform | Max Length | Key Format | OAuth Flow | API |
|----------|-----------|------------|------------|-----|
| **LinkedIn** | 1,300 chars | Professional, industry insights | OAuth 2.0 (3-legged) | Marketing API v2 |
| **Twitter/X** | 280 chars (thread option) | Concise, hashtags, hooks | OAuth 2.0 (PKCE) | API v2 |
| **Facebook** | ~63,206 chars (500 optimal) | Conversational, visual | OAuth 2.0 (Facebook Login) | Graph API v18+ |
| **Instagram** | 2,200 chars, 30 hashtags | Visual-first, hashtag-heavy | Via Facebook Business | Graph API (IG) |
| **TikTok** | varies | Gen-Z tone, trending hooks | OAuth 2.0 | Content Posting API |
### 2.2 OAuth Infrastructure
Reusable OAuth service structure:
**Location:** `igny8_core/integration/oauth/`
- `base_oauth.py` — base class with token encrypt/decrypt, refresh logic
- `linkedin_oauth.py` — LinkedIn 3-legged OAuth, scopes: `w_member_social`, `r_organization_social`, `w_organization_social`
- `twitter_oauth.py` — Twitter OAuth 2.0 with PKCE
- `facebook_oauth.py` — Facebook Login, page selection, scopes: `pages_manage_posts`, `pages_read_engagement`
- `instagram_oauth.py` — via Facebook Business, Instagram Basic Display API
- `tiktok_oauth.py` — TikTok OAuth 2.0
**Token Security:**
- Access tokens and refresh tokens encrypted using the same encryption pattern as GSC tokens (02C)
- Tokens stored in `SocialAccount.access_token` and `SocialAccount.refresh_token` (encrypted TextField)
- Auto-refresh via Celery task (`refresh_social_tokens`) runs hourly
**Platform-Specific Auth Notes:**
- **Facebook:** Requires App Review for `pages_manage_posts` permission. Flow: Login → Page Selection → Permission Grant → Store `page_access_token`. Webhook subscription for engagement updates.
- **LinkedIn:** Requires Marketing API Partner Program access. Flow: Authorization → Organization Selection → Store `access_token`.
- **Instagram:** No direct OAuth — connected via Facebook Business account. Requires Facebook Page linked to Instagram Professional account.
- **Multi-account support:** Multiple SocialAccount records per platform per site.
### 2.3 AI Content Adaptation
**Input:** Content record (title, content_html, meta_description, cluster keywords from SAGCluster)
**Per platform, AI generates:**
- Adapted text matching platform tone + length limits
- Hashtag set (topic-aware, platform-appropriate count)
- CTA appropriate to platform conventions
- Image sizing recommendations
**Platform-Specific Generation:**
- **LinkedIn:** Professional tone, industry framing, 3-5 hashtags, link in text
- **Twitter/X:** Hook + key point + CTA in 280 chars. Thread generation option (2-10 tweets) for long content
- **Facebook:** Conversational, 2-3 paragraphs, 3-5 hashtags, link at end
- **Instagram:** Visual-first caption, 20-30 hashtags in first comment or caption, no clickable link (use "link in bio" CTA)
- **TikTok:** Gen-Z/casual tone, trending hook format, 3-5 hashtags
### 2.4 Post Types
| # | Post Type | Description | Platforms |
|---|-----------|-------------|-----------|
| 1 | **Announcement** | Link post with title + excerpt + URL | All |
| 2 | **Highlights** | Key takeaways from article + CTA | All |
| 3 | **Quote Card** | Branded insight/statistic as image post | All |
| 4 | **FAQ Snippet** | Single FAQ from content as post | LinkedIn, Twitter, Facebook |
| 5 | **Carousel** | Multi-image posts (future) | Instagram, LinkedIn |
### 2.5 Image Sizing per Platform
| Platform | Dimension | Aspect Ratio |
|----------|-----------|-------------|
| LinkedIn | 1200×627px | ~1.91:1 |
| Twitter | 1600×900px | ~16:9 |
| Facebook | 1200×630px | ~1.91:1 |
| Instagram (square) | 1080×1080px | 1:1 |
| Instagram (portrait) | 1080×1350px | 4:5 |
| TikTok | 1080×1920px | 9:16 |
Image resizing uses the `Images` model records generated by pipeline Stages 5-6, resized to platform specs using Pillow.
### 2.6 Scheduling & Calendar
**Best-Time Algorithm:**
- Configurable default best-time slots per platform
- Per-account override via `SocialAccount.settings.best_times[]`
- If engagement data available, learn optimal posting times from past performance
**Queue System:**
- Posts with `status='scheduled'` and `scheduled_at` in the past → Celery task publishes
- Celery task `publish_scheduled_posts` runs every minute
- Frequency caps: max posts per day per platform (configurable, default: 2)
- Cooldown: 4-6 hours between posts to same platform (configurable via `SocialAccount.settings.cooldown_hours`)
**Calendar View:**
- API endpoint returns week/month layout of scheduled + published posts
- Frontend renders visual calendar with drag-drop rescheduling (`.tsx` component)
**Bulk Scheduling:**
- Select multiple content → generate social posts for all connected platforms
- Auto-space across optimal times over days/weeks
- Review before confirming schedule
### 2.7 Engagement Tracking
- Celery task `fetch_engagement` runs every 6 hours
- Fetches metrics from each platform API for recent posts (last 30 days)
- Metrics: likes, comments, shares/retweets, impressions, clicks, engagement_rate
- Aggregate dashboard: per-platform performance, best performing posts, optimal posting times
- Attribution: UTM parameters appended to all shared URLs (`utm_source={platform}&utm_medium=social&utm_campaign=igny8`)
### 2.8 Stage 8 Pipeline Integration
After Stage 7 (publish to WordPress), if social accounts are connected for the site:
1. Pipeline triggers Stage 8
2. Load all connected `SocialAccount` records for the site
3. For each connected account, AI generates platform-adapted post
4. **If `AutomationConfig.auto_social_publish` enabled:** posts queue immediately at next best-time slot
5. **If manual review enabled:** posts go to `status='draft'` for user review
6. Published posts logged with `platform_post_id` for engagement tracking
**AutomationConfig Extension:**
```python
# Add to AutomationConfig.settings JSONField:
{
"social_enabled": True, # Master toggle
"auto_social_publish": False, # Auto-queue or manual review
"social_platforms": ["linkedin", "twitter"], # Which platforms to auto-post
"social_post_types": ["announcement", "highlights"], # Which post types
}
```
---
## 3. DATA MODELS & APIS
### 3.1 New Models
All models in a new `social` app.
#### SocialAccount (social app)
```python
class SocialAccount(SiteSectorBaseModel):
"""
Connected social media account for a site.
Stores OAuth tokens (encrypted) and per-account settings.
Multi-account support: multiple accounts per platform per site.
"""
platform = models.CharField(
max_length=15,
choices=[
('linkedin', 'LinkedIn'),
('twitter', 'Twitter/X'),
('facebook', 'Facebook'),
('instagram', 'Instagram'),
('tiktok', 'TikTok'),
]
)
platform_account_id = models.CharField(
max_length=255,
help_text='External platform account/page ID'
)
account_name = models.CharField(max_length=255, help_text='Display name')
account_type = models.CharField(
max_length=20,
choices=[
('personal', 'Personal'),
('page', 'Page'),
('organization', 'Organization'),
],
default='personal'
)
access_token = models.TextField(
help_text='Encrypted OAuth access token'
)
refresh_token = models.TextField(
blank=True,
default='',
help_text='Encrypted OAuth refresh token (nullable for platforms without refresh)'
)
token_expiry = models.DateTimeField(null=True, blank=True)
permissions = models.JSONField(
default=list,
help_text='Granted OAuth scopes'
)
status = models.CharField(
max_length=10,
choices=[
('active', 'Active'),
('expired', 'Expired'),
('revoked', 'Revoked'),
('error', 'Error'),
],
default='active'
)
settings = models.JSONField(
default=dict,
help_text='{auto_publish, max_per_day, best_times[], cooldown_hours}'
)
class Meta:
app_label = 'social'
db_table = 'igny8_social_accounts'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### SocialPost (social app)
```python
class SocialPost(SiteSectorBaseModel):
"""
A social media post — generated from content or standalone.
Tracks full lifecycle from draft to published with engagement.
"""
content = models.ForeignKey(
'writer.Content',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='social_posts',
help_text='Source content (null for standalone posts)'
)
social_account = models.ForeignKey(
'social.SocialAccount',
on_delete=models.CASCADE,
related_name='posts'
)
platform = models.CharField(
max_length=15,
help_text='Denormalized from social_account for filtering'
)
post_type = models.CharField(
max_length=15,
choices=[
('announcement', 'Announcement'),
('highlights', 'Highlights'),
('quote_card', 'Quote Card'),
('faq_snippet', 'FAQ Snippet'),
('carousel', 'Carousel'),
]
)
text = models.TextField()
hashtags = models.JSONField(default=list, help_text='List of hashtag strings')
media_urls = models.JSONField(default=list, help_text='List of image/video URLs')
link_url = models.URLField(blank=True, default='')
utm_params = models.JSONField(
default=dict,
blank=True,
help_text='{utm_source, utm_medium, utm_campaign}'
)
thread_posts = models.JSONField(
null=True,
blank=True,
help_text='For Twitter threads: [{text, media_url}]'
)
scheduled_at = models.DateTimeField(null=True, blank=True)
published_at = models.DateTimeField(null=True, blank=True)
platform_post_id = models.CharField(
max_length=255,
blank=True,
default='',
help_text='External post ID after publishing'
)
platform_post_url = models.URLField(blank=True, default='')
status = models.CharField(
max_length=15,
choices=[
('draft', 'Draft'),
('scheduled', 'Scheduled'),
('publishing', 'Publishing'),
('published', 'Published'),
('failed', 'Failed'),
('cancelled', 'Cancelled'),
],
default='draft'
)
error_message = models.TextField(blank=True, default='')
class Meta:
app_label = 'social'
db_table = 'igny8_social_posts'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### SocialEngagement (social app)
```python
class SocialEngagement(models.Model):
"""
Engagement metrics for a published social post.
Fetched periodically from platform APIs.
"""
social_post = models.ForeignKey(
'social.SocialPost',
on_delete=models.CASCADE,
related_name='engagement_records'
)
likes = models.IntegerField(default=0)
comments = models.IntegerField(default=0)
shares = models.IntegerField(default=0)
impressions = models.IntegerField(default=0)
clicks = models.IntegerField(default=0)
engagement_rate = models.FloatField(default=0.0)
raw_data = models.JSONField(default=dict, help_text='Full platform API response')
fetched_at = models.DateTimeField(auto_now_add=True)
class Meta:
app_label = 'social'
db_table = 'igny8_social_engagement'
```
**PK:** BigAutoField (integer) — standard Django Model (not AccountBaseModel since engagement is tied to post)
### 3.2 New App Registration
Create social app:
- **App config:** `igny8_core/modules/social/apps.py` with `app_label = 'social'`
- **Add to INSTALLED_APPS** in `igny8_core/settings.py`
### 3.3 Migration
```
igny8_core/migrations/XXXX_add_social_models.py
```
**Operations:**
1. `CreateModel('SocialAccount', ...)` — with index on platform, status
2. `CreateModel('SocialPost', ...)` — with indexes on social_account, platform, status, scheduled_at
3. `CreateModel('SocialEngagement', ...)` — with index on social_post
### 3.4 API Endpoints
All endpoints under `/api/v1/social/`:
#### Account Management
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/social/accounts/connect/{platform}/` | Initiate OAuth flow for platform. Returns redirect URL. |
| GET | `/api/v1/social/accounts/callback/{platform}/` | OAuth callback handler. Exchanges code for tokens, creates SocialAccount. |
| GET | `/api/v1/social/accounts/?site_id=X` | List connected accounts for site. |
| DELETE | `/api/v1/social/accounts/{id}/` | Disconnect account (revoke tokens, soft-delete). |
| PUT | `/api/v1/social/accounts/{id}/settings/` | Update account settings (max_per_day, best_times, cooldown). |
#### Post Management
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/social/posts/generate/` | AI-generate posts for content across all connected platforms. Body: `{content_id, post_types: []}`. |
| POST | `/api/v1/social/posts/` | Create manual post. Body: `{social_account_id, text, ...}`. |
| GET | `/api/v1/social/posts/?site_id=X` | List posts with filters (platform, status, date range). |
| PUT | `/api/v1/social/posts/{id}/` | Edit draft/scheduled post. |
| POST | `/api/v1/social/posts/{id}/publish/` | Publish immediately. |
| POST | `/api/v1/social/posts/{id}/schedule/` | Schedule for later. Body: `{scheduled_at}`. |
| DELETE | `/api/v1/social/posts/{id}/` | Cancel/delete post. |
| POST | `/api/v1/social/posts/bulk-generate/` | Generate for multiple content. Body: `{content_ids: [int]}`. |
| POST | `/api/v1/social/posts/bulk-schedule/` | Schedule multiple posts. Body: `{post_ids: [int], auto_space: true}`. |
#### Calendar & Analytics
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/social/calendar/?site_id=X&month=YYYY-MM` | Calendar view — scheduled + published posts for month. |
| GET | `/api/v1/social/analytics/?site_id=X` | Aggregate analytics: per-platform performance, top posts. |
| GET | `/api/v1/social/analytics/posts/?site_id=X` | Per-post analytics with engagement breakdown. |
**Permissions:** All endpoints use `SiteSectorModelViewSet` permission patterns.
### 3.5 AI Function — GenerateSocialPostsFunction
**Registry key:** `generate_social_posts`
**Location:** `igny8_core/ai/functions/generate_social_posts.py`
```python
class GenerateSocialPostsFunction(BaseAIFunction):
"""
Generates platform-adapted social media posts from content.
One call produces posts for all requested platforms.
"""
function_name = 'generate_social_posts'
def validate(self, content_id, platforms=None, post_types=None, **kwargs):
# Verify content exists, has content_html
# Verify requested platforms have connected accounts
pass
def prepare(self, content_id, platforms=None, post_types=None, **kwargs):
# Load Content record
# Load cluster keywords for hashtag generation
# Load connected SocialAccount records for the site
# Determine applicable post_types per platform
pass
def build_prompt(self):
# Per platform, build adaptation instructions:
# - Platform tone + length limits
# - Content title, meta_description, key points from content_html
# - Hashtag generation instructions
# - CTA format
# - Thread instructions (Twitter if content > 280 chars summary)
pass
def parse_response(self, response):
# Parse per-platform posts:
# {platform: {text, hashtags[], cta, thread_posts?}}
pass
def save_output(self, parsed):
# Create SocialPost records per platform per post_type
# Attach UTM parameters
# Set status='draft' (default) or 'scheduled' if auto-publish
pass
```
### 3.6 OAuth Service
**Location:** `igny8_core/integration/oauth/`
```python
# base_oauth.py
class BaseOAuthService:
"""Base class for OAuth integrations. Handles token encryption/decryption."""
def encrypt_token(self, token):
"""Encrypt token using Django's Fernet key (same pattern as 02C GSC tokens)."""
pass
def decrypt_token(self, encrypted_token):
"""Decrypt stored token."""
pass
def get_authorization_url(self, site_id, account_id=None):
"""Generate OAuth authorization URL with state parameter."""
pass
def handle_callback(self, code, state):
"""Exchange authorization code for tokens, create/update SocialAccount."""
pass
def refresh_tokens(self, social_account):
"""Refresh expired access token using refresh_token."""
pass
# linkedin_oauth.py
class LinkedInOAuthService(BaseOAuthService):
"""LinkedIn Marketing API OAuth 2.0 (3-legged)."""
SCOPES = ['w_member_social', 'r_organization_social', 'w_organization_social']
# ...
# twitter_oauth.py
class TwitterOAuthService(BaseOAuthService):
"""Twitter/X OAuth 2.0 with PKCE."""
SCOPES = ['tweet.read', 'tweet.write', 'users.read']
# ...
# facebook_oauth.py
class FacebookOAuthService(BaseOAuthService):
"""Facebook Login with page selection."""
SCOPES = ['pages_manage_posts', 'pages_read_engagement']
# ...
# tiktok_oauth.py
class TikTokOAuthService(BaseOAuthService):
"""TikTok Content Posting API OAuth."""
# ...
```
### 3.7 Social Publisher Service
**Location:** `igny8_core/business/social_publisher.py`
```python
class SocialPublisherService:
"""
Handles actual posting to platform APIs.
Routes to platform-specific publishers.
"""
PUBLISHERS = {
'linkedin': LinkedInPublisher,
'twitter': TwitterPublisher,
'facebook': FacebookPublisher,
'instagram': InstagramPublisher,
'tiktok': TikTokPublisher,
}
def publish(self, social_post_id):
"""
Publish a SocialPost to its target platform.
1. Load SocialPost + SocialAccount
2. Decrypt access token
3. Route to platform-specific publisher
4. Update SocialPost with platform_post_id, platform_post_url, published_at
5. Set status='published' or 'failed'
"""
pass
```
### 3.8 Celery Tasks
**Location:** `igny8_core/tasks/social_tasks.py`
```python
@shared_task(name='publish_scheduled_posts')
def publish_scheduled_posts():
"""
Runs every minute. Finds SocialPost records with
status='scheduled' and scheduled_at <= now(). Publishes each.
"""
pass
@shared_task(name='refresh_social_tokens')
def refresh_social_tokens():
"""
Runs hourly. Refreshes tokens expiring within next 2 hours.
Updates SocialAccount.access_token, token_expiry.
Sets status='expired' if refresh fails.
"""
pass
@shared_task(name='fetch_engagement')
def fetch_engagement():
"""
Runs every 6 hours. Fetches engagement metrics from platform APIs
for all published posts in last 30 days.
Creates SocialEngagement records.
"""
pass
@shared_task(name='generate_social_posts_stage8')
def generate_social_posts_stage8(content_id):
"""
Triggered by pipeline after Stage 7 (publish).
Generates social posts for all connected platforms for the site.
"""
pass
```
**Beat Schedule Additions:**
| Task | Schedule | Notes |
|------|----------|-------|
| `publish_scheduled_posts` | Every minute | Publishes due scheduled posts |
| `refresh_social_tokens` | Hourly | Refreshes expiring OAuth tokens |
| `fetch_engagement` | Every 6 hours | Fetches engagement metrics for recent posts |
### 3.9 Image Resizing Service
**Location:** `igny8_core/business/social_image_resize.py`
```python
class SocialImageResizeService:
"""
Resizes images from the Images model to platform-specific dimensions.
Uses Pillow for image processing.
"""
PLATFORM_SIZES = {
'linkedin': (1200, 627),
'twitter': (1600, 900),
'facebook': (1200, 630),
'instagram_square': (1080, 1080),
'instagram_portrait': (1080, 1350),
'tiktok': (1080, 1920),
}
def resize(self, image_path, platform, variant='default'):
"""Resize image, return new file path."""
pass
```
---
## 4. IMPLEMENTATION STEPS
### Step 1: Create Social App
1. Create `igny8_core/modules/social/` directory with `__init__.py` and `apps.py`
2. Add `social` to `INSTALLED_APPS` in settings.py
3. Create models: SocialAccount, SocialPost, SocialEngagement
### Step 2: Migration
1. Create migration for 3 new models
2. Run migration
### Step 3: OAuth Infrastructure
1. Create `igny8_core/integration/oauth/` directory
2. Implement `BaseOAuthService` with token encryption/decryption
3. Implement platform-specific OAuth handlers (LinkedIn, Twitter, Facebook, TikTok)
4. Instagram OAuth routes through Facebook Business
### Step 4: AI Function
1. Implement `GenerateSocialPostsFunction` in `igny8_core/ai/functions/generate_social_posts.py`
2. Register `generate_social_posts` in `igny8_core/ai/registry.py`
### Step 5: Services
1. Implement `SocialPublisherService` in `igny8_core/business/social_publisher.py`
2. Implement platform-specific publishers (LinkedIn, Twitter, Facebook, Instagram, TikTok)
3. Implement `SocialImageResizeService` in `igny8_core/business/social_image_resize.py`
### Step 6: Pipeline Integration
Add Stage 8 trigger after Stage 7 (publish):
```python
# In pipeline after publish completes:
def post_publish(content_id):
site = content.site
# Check if social is enabled for this site
config = AutomationConfig.objects.get(site=site)
if config.settings.get('social_enabled'):
generate_social_posts_stage8.delay(content_id)
```
### Step 7: API Endpoints
1. Create `igny8_core/urls/social.py` with account, post, calendar, and analytics endpoints
2. Create views: `AccountConnectView`, `AccountCallbackView`, `AccountListView`
3. Create `PostGenerateView`, `PostViewSet`, `CalendarView`, `AnalyticsView`
4. Register URL patterns under `/api/v1/social/`
### Step 8: Celery Tasks
1. Implement 4 tasks in `igny8_core/tasks/social_tasks.py`
2. Add `publish_scheduled_posts`, `refresh_social_tokens`, `fetch_engagement` to beat schedule
### Step 9: Serializers & Admin
1. Create DRF serializers for SocialAccount (exclude encrypted tokens from response), SocialPost, SocialEngagement
2. Register models in Django admin
### Step 10: Credit Cost Configuration
Add to `CreditCostConfig` (billing app):
| operation_type | default_cost | description |
|---------------|-------------|-------------|
| `social_adaptation` | 1 | AI-generate post for one platform |
| `social_hashtag` | 0.5 | Hashtag generation for one post |
| `social_thread` | 2 | Twitter thread generation (2-10 tweets) |
| `social_carousel` | 5 | Carousel post generation |
| `social_image_resize` | 3-10 | Image resizing/generation per platform |
| `social_suite` | 15-25 | Full suite for all 5 platforms |
---
## 5. ACCEPTANCE CRITERIA
### OAuth Connections
- [ ] LinkedIn OAuth 2.0 connects and stores encrypted tokens
- [ ] Twitter/X OAuth 2.0 with PKCE connects successfully
- [ ] Facebook Login with page selection works, stores page_access_token
- [ ] Instagram connects via Facebook Business account
- [ ] TikTok OAuth connects and stores tokens
- [ ] Token refresh runs hourly, updates expired tokens
- [ ] Multi-account support: multiple accounts per platform per site
### Post Generation
- [ ] AI generates platform-adapted text for each connected platform
- [ ] LinkedIn posts use professional tone, 1,300 char limit
- [ ] Twitter posts respect 280 char limit, thread option for long content
- [ ] Facebook posts use conversational tone, optimal 500 char length
- [ ] Instagram captions include 20-30 hashtags, "link in bio" CTA
- [ ] TikTok posts use casual/Gen-Z tone, trending hook format
- [ ] UTM parameters appended to all shared URLs
### Post Types
- [ ] Announcement, Highlights, Quote Card, FAQ Snippet post types generated
- [ ] Post type selection respects platform compatibility
### Scheduling
- [ ] Posts can be scheduled for future time
- [ ] `publish_scheduled_posts` task runs every minute, publishes due posts
- [ ] Frequency caps enforced (max per day per platform)
- [ ] Cooldown period enforced between posts to same platform
- [ ] Calendar endpoint returns month view of scheduled/published posts
- [ ] Bulk scheduling auto-spaces posts across optimal times
### Engagement
- [ ] Engagement metrics fetched every 6 hours for recent posts
- [ ] SocialEngagement records created with likes, comments, shares, impressions, clicks
- [ ] Aggregate analytics endpoint returns per-platform performance + top posts
- [ ] Per-post analytics available with engagement breakdown
### Pipeline Integration
- [ ] Stage 8 triggers automatically after Stage 7 (publish)
- [ ] Social posts generated for all connected platforms
- [ ] auto_social_publish toggle controls immediate vs manual review
- [ ] Platform selection configurable via AutomationConfig.settings
### Images
- [ ] Images from pipeline resized to platform-specific dimensions
- [ ] media_urls populated on SocialPost records
---
## 6. CLAUDE CODE INSTRUCTIONS
### File Locations
```
igny8_core/
├── modules/
│ └── social/
│ ├── __init__.py
│ ├── apps.py # app_label = 'social'
│ └── models.py # SocialAccount, SocialPost, SocialEngagement
├── ai/
│ └── functions/
│ └── generate_social_posts.py # GenerateSocialPostsFunction
├── integration/
│ └── oauth/
│ ├── __init__.py
│ ├── base_oauth.py # BaseOAuthService
│ ├── linkedin_oauth.py
│ ├── twitter_oauth.py
│ ├── facebook_oauth.py
│ ├── instagram_oauth.py # Routes through Facebook
│ └── tiktok_oauth.py
├── business/
│ ├── social_publisher.py # SocialPublisherService + platform publishers
│ └── social_image_resize.py # SocialImageResizeService
├── tasks/
│ └── social_tasks.py # Celery tasks
├── urls/
│ └── social.py # Social endpoints
└── migrations/
└── XXXX_add_social_models.py
```
### Conventions
- **PKs:** BigAutoField (integer) — do NOT use UUIDs
- **Table prefix:** `igny8_` on all new tables
- **App label:** `social` (new app)
- **Celery app name:** `igny8_core`
- **URL pattern:** `/api/v1/social/...`
- **Permissions:** Use `SiteSectorModelViewSet` permission pattern
- **Token encryption:** Same Fernet pattern as 02C GSC tokens — NEVER expose raw tokens in API responses
- **AI functions:** Extend `BaseAIFunction`; register as `generate_social_posts`
- **Frontend:** `.tsx` files with Zustand stores
### Cross-References
| Doc | Relationship |
|-----|-------------|
| **01E** | Pipeline Stage 8 integration — hooks after Stage 7 publish |
| **02I** | Video creator shares social posting for video content; reuses SocialAccount OAuth for YouTube/TikTok |
| **02C** | GSC token encryption pattern reused for social OAuth tokens |
| **03A** | WP plugin standalone has share buttons — different from this (posting FROM IGNY8) |
| **04A** | Managed services include social media management as a service tier |
### Key Decisions
1. **New `social` app** — Separate from integration because social media is a distinct domain with its own models and business logic
2. **SocialEngagement extends models.Model, not SiteSectorBaseModel** — Engagement records are tied to posts, not directly to sites/sectors
3. **Denormalized `platform` on SocialPost** — Avoids join to SocialAccount for platform-based queries and filtering
4. **OAuth tokens encrypted at rest** — Same Fernet encryption as 02C; tokens never returned in API responses
5. **AutomationConfig.settings extension** — Social pipeline toggles added to existing settings JSONField rather than new model fields

896
v2/V2-Execution-Docs/02I-video-creator.md Normal file
View File

@@ -0,0 +1,896 @@
# IGNY8 Phase 2: Video Creator (02I)
## AI Video Creation Pipeline — Stage 9
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 2 — Feature Expansion
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects
---
## 1. CURRENT STATE
### Video Today
There is **no** video creation capability in IGNY8. No TTS, no FFmpeg pipeline, no video publishing. Images exist (generated by pipeline Stages 5-6) and can feed into video as visual assets.
### What Exists
- `Images` model (writer app) — generated images from pipeline, usable as video visual assets
- `SocialAccount` model (02H) — provides OAuth connections to YouTube, Instagram, TikTok for video publishing
- Self-hosted AI infrastructure (Phase 0F) — provides GPU for TTS and AI image generation
- Content generation pipeline (01E) — content records provide source material for video scripts
- Celery infrastructure with multiple queues — supports dedicated `video` queue for long-running renders
### What Does Not Exist
- No video app or models
- No script generation from articles
- No TTS (text-to-speech) voiceover generation
- No FFmpeg/MoviePy video composition pipeline
- No subtitle generation
- No video publishing to platforms
- No Stage 9 pipeline integration
---
## 2. WHAT TO BUILD
### Overview
Build Stage 9 of the automation pipeline: an AI video creation system that converts published content into videos. The pipeline has 5 stages: script generation → voiceover → visual assets → composition → publishing. Videos publish to YouTube, Instagram Reels, and TikTok.
### 2.1 Video Types
| Type | Duration | Aspect Ratio | Primary Platform |
|------|----------|-------------|-----------------|
| **Short** | 30-90s | 9:16 vertical | YouTube Shorts, Instagram Reels, TikTok |
| **Medium** | 60-180s | 9:16 or 16:9 | TikTok, YouTube |
| **Long** | 5-15m | 16:9 horizontal | YouTube |
### 2.2 Platform Specs
| Platform | Max Duration | Resolution | Encoding | Max File Size |
|----------|-------------|------------|----------|--------------|
| YouTube Long | Up to 12h | 1920×1080 | MP4 H.264, AAC audio | 256GB |
| YouTube Shorts | ≤60s | 1080×1920 | MP4 H.264, AAC audio | 256GB |
| Instagram Reels | ≤90s | 1080×1920 | MP4 H.264, AAC audio | 650MB |
| TikTok | ≤10m | 1080×1920 | MP4 H.264, AAC audio | 72MB |
### 2.3 Five-Stage Video Pipeline
#### Stage 1 — Script Generation (AI)
**Input:** Content record (title, content_html, meta_description, keywords, images)
AI extracts key points and produces:
```json
{
"hook": "text (3-5 sec)",
"intro": "text (10-15 sec)",
"points": [
{"text": "...", "duration_est": 20, "visual_cue": "show chart", "text_overlay": "Key stat"}
],
"cta": "text (5-10 sec)",
"chapter_markers": [{"time": 0, "title": "Intro"}],
"total_estimated_duration": 120
}
```
SEO: AI generates platform-specific title, description, tags for each target platform.
#### Stage 2 — Voiceover (TTS)
**Cloud Providers:**
| Provider | Cost | Quality | Features |
|----------|------|---------|----------|
| OpenAI TTS | $15-30/1M chars | High | Voices: alloy, echo, fable, onyx, nova, shimmer |
| ElevenLabs | Plan-based | Highest | Voice cloning, ultra-realistic |
**Self-Hosted (via 0F GPU):**
| Model | Quality | Speed | Notes |
|-------|---------|-------|-------|
| Coqui XTTS-v2 | Good | Medium | Multi-language, free |
| Bark | Expressive | Slow | Emotional speech |
| Piper TTS | Moderate | Fast | Lightweight |
**Features:** Voice selection, speed control, multi-language support
**Output:** WAV/MP3 audio file + word-level timestamps (for subtitle sync)
#### Stage 3 — Visual Assets
**Sources:**
- Article images from `Images` model (already generated by pipeline Stages 5-6)
- AI-generated scenes (Runware/DALL-E/Stable Diffusion via 0F)
- Stock footage APIs: Pexels, Pixabay (free, API key required)
- Text overlay frames (rendered via Pillow)
- Code snippet frames (via Pygments syntax highlighting)
**Effects:**
- Ken Burns effect on still images (zoom/pan animation)
- Transition effects between scenes (fade, slide, dissolve)
#### Stage 4 — Video Composition (FFmpeg + MoviePy)
**Libraries:** FFmpeg (encoding), MoviePy (high-level composition), Pillow (text overlays), pydub (audio processing)
**Process:**
1. Create visual timeline from script sections
2. Assign visuals to each section (image/video clip per point)
3. Add text overlays at specified timestamps
4. Mix voiceover audio with background music (royalty-free, 20% volume)
5. Apply transitions between sections
6. Render to target resolution/format
**Render Presets:**
| Preset | Resolution | Duration Range | Encoding |
|--------|-----------|----------------|----------|
| `youtube_long` | 1920×1080 | 3-15m | H.264/AAC |
| `youtube_short` | 1080×1920 | 30-60s | H.264/AAC |
| `instagram_reel` | 1080×1920 | 30-90s | H.264/AAC |
| `tiktok` | 1080×1920 | 30-180s | H.264/AAC |
#### Stage 5 — SEO & Publishing
- Auto-generate SRT subtitle file from TTS word-level timestamps
- AI thumbnail: hero image with title text overlay
- Platform-specific metadata: title (optimized per platform), description (with timestamps for YouTube), tags, category
- Publishing via platform APIs (reuses OAuth from 02H SocialAccount)
- Confirmation logging with platform video ID
### 2.4 User Flow
1. Select content → choose video type (short/medium/long) → target platforms
2. AI generates script → user reviews/edits script
3. Select voice → preview audio → approve
4. Auto-assign visuals → user can swap images → preview composition
5. Render video → preview final → approve
6. Publish to selected platforms → track performance
### 2.5 Dedicated Celery Queue
Video rendering is CPU/GPU intensive and requires isolation:
- **Dedicated `video` queue:** `celery -A igny8_core worker -Q video --concurrency=1`
- **Long-running tasks:** 5-30 minutes per video render
- **Progress tracking:** via Celery result backend (task status updates)
- **Temp file cleanup:** after publish, clean up intermediate files (audio, frames, raw renders)
---
## 3. DATA MODELS & APIS
### 3.1 New Models
All models in a new `video` app.
#### VideoProject (video app)
```python
class VideoProject(SiteSectorBaseModel):
"""
Top-level container for a video creation project.
Links to source content and tracks overall progress.
"""
content = models.ForeignKey(
'writer.Content',
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name='video_projects',
help_text='Source content (null for standalone video)'
)
project_type = models.CharField(
max_length=10,
choices=[
('short', 'Short (30-90s)'),
('medium', 'Medium (60-180s)'),
('long', 'Long (5-15m)'),
]
)
target_platforms = models.JSONField(
default=list,
help_text='List of target platform strings: youtube_long, youtube_short, instagram_reel, tiktok'
)
status = models.CharField(
max_length=15,
choices=[
('draft', 'Draft'),
('scripting', 'Script Generation'),
('voiceover', 'Voiceover Generation'),
('composing', 'Visual Composition'),
('rendering', 'Rendering'),
('review', 'Ready for Review'),
('published', 'Published'),
('failed', 'Failed'),
],
default='draft'
)
settings = models.JSONField(
default=dict,
help_text='{voice_id, voice_provider, music_track, transition_style}'
)
class Meta:
app_label = 'video'
db_table = 'igny8_video_projects'
```
**PK:** BigAutoField (integer) — inherits from SiteSectorBaseModel
#### VideoScript (video app)
```python
class VideoScript(models.Model):
"""
Script for a video project — generated by AI, editable by user.
"""
project = models.OneToOneField(
'video.VideoProject',
on_delete=models.CASCADE,
related_name='script'
)
script_text = models.TextField(help_text='Full narration text')
sections = models.JSONField(
default=list,
help_text='[{text, duration_est, visual_cue, text_overlay}]'
)
hook = models.TextField(blank=True, default='')
cta = models.TextField(blank=True, default='')
chapter_markers = models.JSONField(
default=list,
help_text='[{time, title}]'
)
total_estimated_duration = models.IntegerField(
default=0,
help_text='Total estimated duration in seconds'
)
seo_metadata = models.JSONField(
default=dict,
help_text='{platform: {title, description, tags}}'
)
version = models.IntegerField(default=1)
class Meta:
app_label = 'video'
db_table = 'igny8_video_scripts'
```
**PK:** BigAutoField (integer) — standard Django Model
#### VideoAsset (video app)
```python
class VideoAsset(models.Model):
"""
Individual asset (image, footage, music, overlay, subtitle) for a video project.
"""
project = models.ForeignKey(
'video.VideoProject',
on_delete=models.CASCADE,
related_name='assets'
)
asset_type = models.CharField(
max_length=15,
choices=[
('image', 'Image'),
('footage', 'Footage'),
('music', 'Background Music'),
('overlay', 'Text Overlay'),
('subtitle', 'Subtitle File'),
]
)
source = models.CharField(
max_length=20,
choices=[
('article_image', 'Article Image'),
('ai_generated', 'AI Generated'),
('stock_pexels', 'Pexels Stock'),
('stock_pixabay', 'Pixabay Stock'),
('uploaded', 'User Uploaded'),
('rendered', 'Rendered'),
]
)
file_path = models.CharField(max_length=500, help_text='Path in media storage')
file_url = models.URLField(blank=True, default='')
duration = models.FloatField(
null=True, blank=True,
help_text='Duration in seconds (for video/audio assets)'
)
section_index = models.IntegerField(
null=True, blank=True,
help_text='Which script section this asset belongs to'
)
order = models.IntegerField(default=0)
class Meta:
app_label = 'video'
db_table = 'igny8_video_assets'
```
**PK:** BigAutoField (integer) — standard Django Model
#### RenderedVideo (video app)
```python
class RenderedVideo(models.Model):
"""
A rendered video file for a specific platform preset.
One project can have multiple renders (one per target platform).
"""
project = models.ForeignKey(
'video.VideoProject',
on_delete=models.CASCADE,
related_name='rendered_videos'
)
preset = models.CharField(
max_length=20,
choices=[
('youtube_long', 'YouTube Long'),
('youtube_short', 'YouTube Short'),
('instagram_reel', 'Instagram Reel'),
('tiktok', 'TikTok'),
]
)
resolution = models.CharField(
max_length=15,
help_text='e.g. 1920x1080 or 1080x1920'
)
duration = models.FloatField(help_text='Duration in seconds')
file_size = models.BigIntegerField(help_text='File size in bytes')
file_path = models.CharField(max_length=500)
file_url = models.URLField(blank=True, default='')
subtitle_file_path = models.CharField(max_length=500, blank=True, default='')
thumbnail_path = models.CharField(max_length=500, blank=True, default='')
render_started_at = models.DateTimeField()
render_completed_at = models.DateTimeField(null=True, blank=True)
status = models.CharField(
max_length=15,
choices=[
('queued', 'Queued'),
('rendering', 'Rendering'),
('completed', 'Completed'),
('failed', 'Failed'),
],
default='queued'
)
class Meta:
app_label = 'video'
db_table = 'igny8_rendered_videos'
```
**PK:** BigAutoField (integer) — standard Django Model
#### PublishedVideo (video app)
```python
class PublishedVideo(models.Model):
"""
Tracks a rendered video published to a social platform.
Uses SocialAccount from 02H for OAuth credentials.
"""
rendered_video = models.ForeignKey(
'video.RenderedVideo',
on_delete=models.CASCADE,
related_name='publications'
)
social_account = models.ForeignKey(
'social.SocialAccount',
on_delete=models.CASCADE,
related_name='published_videos'
)
platform = models.CharField(max_length=15)
platform_video_id = models.CharField(max_length=255, blank=True, default='')
published_url = models.URLField(blank=True, default='')
title = models.CharField(max_length=255)
description = models.TextField()
tags = models.JSONField(default=list)
thumbnail_url = models.URLField(blank=True, default='')
published_at = models.DateTimeField(null=True, blank=True)
status = models.CharField(
max_length=15,
choices=[
('publishing', 'Publishing'),
('published', 'Published'),
('failed', 'Failed'),
('removed', 'Removed'),
],
default='publishing'
)
class Meta:
app_label = 'video'
db_table = 'igny8_published_videos'
```
**PK:** BigAutoField (integer) — standard Django Model
#### VideoEngagement (video app)
```python
class VideoEngagement(models.Model):
"""
Engagement metrics for a published video.
Fetched periodically from platform APIs.
"""
published_video = models.ForeignKey(
'video.PublishedVideo',
on_delete=models.CASCADE,
related_name='engagement_records'
)
views = models.IntegerField(default=0)
likes = models.IntegerField(default=0)
comments = models.IntegerField(default=0)
shares = models.IntegerField(default=0)
watch_time_seconds = models.IntegerField(default=0)
avg_view_duration = models.FloatField(default=0.0)
raw_data = models.JSONField(default=dict, help_text='Full platform API response')
fetched_at = models.DateTimeField(auto_now_add=True)
class Meta:
app_label = 'video'
db_table = 'igny8_video_engagement'
```
**PK:** BigAutoField (integer) — standard Django Model
### 3.2 New App Registration
Create video app:
- **App config:** `igny8_core/modules/video/apps.py` with `app_label = 'video'`
- **Add to INSTALLED_APPS** in `igny8_core/settings.py`
### 3.3 Migration
```
igny8_core/migrations/XXXX_add_video_models.py
```
**Operations:**
1. `CreateModel('VideoProject', ...)` — with indexes on content, status
2. `CreateModel('VideoScript', ...)` — OneToOne to VideoProject
3. `CreateModel('VideoAsset', ...)` — with index on project
4. `CreateModel('RenderedVideo', ...)` — with index on project, status
5. `CreateModel('PublishedVideo', ...)` — with indexes on rendered_video, social_account
6. `CreateModel('VideoEngagement', ...)` — with index on published_video
### 3.4 API Endpoints
All endpoints under `/api/v1/video/`:
#### Project Management
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/video/projects/` | Create video project. Body: `{content_id, project_type, target_platforms}`. |
| GET | `/api/v1/video/projects/?site_id=X` | List projects with filters (status, project_type). |
| GET | `/api/v1/video/projects/{id}/` | Project detail with script, assets, renders. |
#### Script
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/video/scripts/generate/` | AI-generate script from content. Body: `{project_id}`. |
| PUT | `/api/v1/video/scripts/{project_id}/` | Edit script (user modifications). |
#### Voiceover
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/video/voiceover/generate/` | Generate TTS audio. Body: `{project_id, voice_id, provider}`. |
| POST | `/api/v1/video/voiceover/preview/` | Preview voice sample (short clip). Body: `{text, voice_id, provider}`. |
#### Assets
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/video/assets/{project_id}/` | List project assets. |
| POST | `/api/v1/video/assets/{project_id}/` | Add/replace asset. |
#### Rendering & Publishing
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/v1/video/render/` | Queue video render. Body: `{project_id, presets: []}`. |
| GET | `/api/v1/video/render/{id}/status/` | Render progress (queued/rendering/completed/failed). |
| GET | `/api/v1/video/rendered/{project_id}/` | List rendered videos for project. |
| POST | `/api/v1/video/publish/` | Publish to platform. Body: `{rendered_video_id, social_account_id}`. |
#### Analytics
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/video/analytics/?site_id=X` | Aggregate video analytics across projects. |
| GET | `/api/v1/video/analytics/{published_video_id}/` | Single video analytics with engagement timeline. |
**Permissions:** All endpoints use `SiteSectorModelViewSet` permission patterns.
### 3.5 AI Functions
#### GenerateVideoScriptFunction
**Registry key:** `generate_video_script`
**Location:** `igny8_core/ai/functions/generate_video_script.py`
```python
class GenerateVideoScriptFunction(BaseAIFunction):
"""
Generates video script from content record.
Produces hook, intro, body points, CTA, chapter markers,
and platform-specific SEO metadata.
"""
function_name = 'generate_video_script'
def validate(self, project_id, **kwargs):
# Verify project exists, has linked content with content_html
pass
def prepare(self, project_id, **kwargs):
# Load VideoProject + Content
# Extract key points, images, meta_description
# Determine target duration from project_type
pass
def build_prompt(self):
# Include: content title, key points, meta_description
# Target duration constraints
# Per target_platform: SEO metadata requirements
pass
def parse_response(self, response):
# Parse script structure: hook, intro, points[], cta, chapter_markers
# Parse seo_metadata per platform
pass
def save_output(self, parsed):
# Create/update VideoScript record
# Update VideoProject.status = 'scripting' → 'voiceover'
pass
```
### 3.6 TTS Service
**Location:** `igny8_core/business/tts_service.py`
```python
class TTSService:
"""
Text-to-speech service. Supports cloud providers and self-hosted models.
Returns audio file + word-level timestamps.
"""
PROVIDERS = {
'openai': OpenAITTSProvider,
'elevenlabs': ElevenLabsTTSProvider,
'coqui': CoquiTTSProvider, # Self-hosted via 0F
'bark': BarkTTSProvider, # Self-hosted via 0F
'piper': PiperTTSProvider, # Self-hosted via 0F
}
def generate(self, text, voice_id, provider='openai'):
"""
Generate voiceover audio.
Returns {audio_path, duration, timestamps: [{word, start, end}]}
"""
pass
def preview(self, text, voice_id, provider='openai', max_chars=200):
"""Generate short preview clip."""
pass
def list_voices(self, provider='openai'):
"""List available voices for provider."""
pass
```
### 3.7 Video Composition Service
**Location:** `igny8_core/business/video_composition.py`
```python
class VideoCompositionService:
"""
Composes video from script + audio + visual assets using FFmpeg/MoviePy.
"""
PRESETS = {
'youtube_long': {'width': 1920, 'height': 1080, 'min_dur': 180, 'max_dur': 900},
'youtube_short': {'width': 1080, 'height': 1920, 'min_dur': 30, 'max_dur': 60},
'instagram_reel': {'width': 1080, 'height': 1920, 'min_dur': 30, 'max_dur': 90},
'tiktok': {'width': 1080, 'height': 1920, 'min_dur': 30, 'max_dur': 180},
}
def compose(self, project_id, preset):
"""
Full composition:
1. Load script + audio + visual assets
2. Create visual timeline from script sections
3. Assign visuals to sections
4. Add text overlays at timestamps
5. Mix voiceover + background music (20% volume)
6. Apply transitions
7. Render to preset resolution/format
8. Generate SRT subtitles from TTS timestamps
9. Generate thumbnail
10. Create RenderedVideo record
Returns RenderedVideo instance.
"""
pass
def _apply_ken_burns(self, image_clip, duration):
"""Apply zoom/pan animation to still image."""
pass
def _generate_subtitles(self, timestamps, output_path):
"""Generate SRT file from word-level timestamps."""
pass
def _generate_thumbnail(self, project, output_path):
"""Create thumbnail: hero image + title text overlay."""
pass
```
### 3.8 Video Publisher Service
**Location:** `igny8_core/business/video_publisher.py`
```python
class VideoPublisherService:
"""
Publishes rendered videos to platforms via their APIs.
Reuses SocialAccount OAuth from 02H.
"""
def publish(self, rendered_video_id, social_account_id):
"""
Upload and publish video to platform.
1. Load RenderedVideo + SocialAccount
2. Decrypt OAuth tokens
3. Upload video file via platform API
4. Set metadata (title, description, tags, thumbnail)
5. Create PublishedVideo record
"""
pass
```
### 3.9 Celery Tasks
**Location:** `igny8_core/tasks/video_tasks.py`
All video tasks run on the dedicated `video` queue:
```python
@shared_task(name='generate_video_script', queue='video')
def generate_video_script_task(project_id):
"""AI script generation from content."""
pass
@shared_task(name='generate_voiceover', queue='video')
def generate_voiceover_task(project_id):
"""TTS audio generation."""
pass
@shared_task(name='render_video', queue='video')
def render_video_task(project_id, preset):
"""
FFmpeg/MoviePy video composition. Long-running (5-30 min).
Updates RenderedVideo.status through lifecycle.
"""
pass
@shared_task(name='generate_thumbnail', queue='video')
def generate_thumbnail_task(project_id):
"""AI thumbnail creation with title overlay."""
pass
@shared_task(name='generate_subtitles', queue='video')
def generate_subtitles_task(project_id):
"""SRT generation from TTS timestamps."""
pass
@shared_task(name='publish_video', queue='video')
def publish_video_task(rendered_video_id, social_account_id):
"""Upload video to platform API."""
pass
@shared_task(name='fetch_video_engagement')
def fetch_video_engagement_task():
"""Periodic metric fetch for published videos. Runs on default queue."""
pass
@shared_task(name='video_pipeline_stage9', queue='video')
def video_pipeline_stage9(content_id):
"""
Full pipeline: script → voice → render → publish.
Triggered after Stage 8 (social) or directly after Stage 7 (publish).
"""
pass
```
**Beat Schedule Additions:**
| Task | Schedule | Notes |
|------|----------|-------|
| `fetch_video_engagement` | Every 12 hours | Fetches engagement metrics for published videos |
**Docker Configuration:**
```yaml
# Add to docker-compose.app.yml:
celery_video_worker:
build: ./backend
command: celery -A igny8_core worker -Q video --concurrency=1 --loglevel=info
# Requires FFmpeg installed in Docker image
```
**Dockerfile Addition:**
```dockerfile
# Add to backend/Dockerfile:
RUN apt-get update && apt-get install -y ffmpeg
```
---
## 4. IMPLEMENTATION STEPS
### Step 1: Create Video App
1. Create `igny8_core/modules/video/` directory with `__init__.py` and `apps.py`
2. Add `video` to `INSTALLED_APPS` in settings.py
3. Create 6 models: VideoProject, VideoScript, VideoAsset, RenderedVideo, PublishedVideo, VideoEngagement
### Step 2: Migration
1. Create migration for 6 new models
2. Run migration
### Step 3: System Dependencies
1. Add FFmpeg to Docker image (`apt-get install -y ffmpeg`)
2. Add to `requirements.txt`: `moviepy`, `pydub`, `Pillow` (already present), `pysrt`
3. Add docker-compose service for `celery_video_worker` with `-Q video --concurrency=1`
### Step 4: AI Function
1. Implement `GenerateVideoScriptFunction` in `igny8_core/ai/functions/generate_video_script.py`
2. Register `generate_video_script` in `igny8_core/ai/registry.py`
### Step 5: Services
1. Implement `TTSService` in `igny8_core/business/tts_service.py` (cloud + self-hosted providers)
2. Implement `VideoCompositionService` in `igny8_core/business/video_composition.py`
3. Implement `VideoPublisherService` in `igny8_core/business/video_publisher.py`
### Step 6: Pipeline Integration
Add Stage 9 trigger:
```python
# After Stage 8 (social posts) or Stage 7 (publish):
def post_social_or_publish(content_id):
site = content.site
config = AutomationConfig.objects.get(site=site)
if config.settings.get('video_enabled'):
video_pipeline_stage9.delay(content_id)
```
### Step 7: API Endpoints
1. Create `igny8_core/urls/video.py` with project, script, voiceover, asset, render, publish, analytics endpoints
2. Create views extending `SiteSectorModelViewSet`
3. Register URL patterns under `/api/v1/video/`
### Step 8: Celery Tasks
1. Implement 8 tasks in `igny8_core/tasks/video_tasks.py`
2. Add `fetch_video_engagement` to beat schedule
3. Ensure render tasks target `video` queue
### Step 9: Serializers & Admin
1. Create DRF serializers for all 6 models
2. Register models in Django admin
### Step 10: Credit Cost Configuration
Add to `CreditCostConfig` (billing app):
| operation_type | default_cost | description |
|---------------|-------------|-------------|
| `video_script_generation` | 5 | AI script generation from content |
| `video_tts_standard` | 10/min | Cloud TTS (OpenAI) — per minute of audio |
| `video_tts_selfhosted` | 2/min | Self-hosted TTS (Coqui/Piper via 0F) |
| `video_tts_hd` | 20/min | HD TTS (ElevenLabs) — per minute |
| `video_visual_generation` | 15-50 | AI visual asset generation (varies by count) |
| `video_thumbnail` | 3-10 | AI thumbnail creation |
| `video_composition` | 5 | FFmpeg render |
| `video_seo_metadata` | 1 | SEO metadata per platform |
| `video_short_total` | 40-80 | Total for short-form video |
| `video_long_total` | 100-250 | Total for long-form video |
---
## 5. ACCEPTANCE CRITERIA
### Script Generation
- [ ] AI generates structured script from content with hook, intro, body points, CTA
- [ ] Script includes chapter markers with timestamps
- [ ] Platform-specific SEO metadata generated (title, description, tags)
- [ ] Script duration estimates match project_type constraints
- [ ] User can edit script before proceeding
### Voiceover
- [ ] OpenAI TTS generates audio with voice selection
- [ ] ElevenLabs TTS works as premium option
- [ ] Self-hosted TTS (Coqui XTTS-v2) works via 0F GPU
- [ ] Word-level timestamps generated for subtitle sync
- [ ] Voice preview endpoint allows testing before full generation
### Visual Assets
- [ ] Article images from Images model used as visual assets
- [ ] Ken Burns effect applied to still images
- [ ] Text overlay frames rendered via Pillow
- [ ] Transitions applied between scenes
- [ ] User can swap assets before rendering
### Rendering
- [ ] FFmpeg/MoviePy composition produces correct resolution per preset
- [ ] Audio mix: voiceover at 100% + background music at 20%
- [ ] SRT subtitle file generated from TTS timestamps
- [ ] AI thumbnail generated with title text overlay
- [ ] Render runs on dedicated `video` Celery queue with concurrency=1
- [ ] Render progress trackable via status endpoint
### Publishing
- [ ] Video uploads to YouTube via API (reusing 02H SocialAccount)
- [ ] Video uploads to Instagram Reels
- [ ] Video uploads to TikTok
- [ ] Platform video ID and URL stored on PublishedVideo
- [ ] Engagement metrics fetched every 12 hours
### Pipeline Integration
- [ ] Stage 9 triggers automatically when video_enabled in AutomationConfig
- [ ] Full pipeline (script → voice → render → publish) runs as single Celery chain
- [ ] VideoObject schema (02G) generated for published video content
---
## 6. CLAUDE CODE INSTRUCTIONS
### File Locations
```
igny8_core/
├── modules/
│ └── video/
│ ├── __init__.py
│ ├── apps.py # app_label = 'video'
│ └── models.py # 6 models
├── ai/
│ └── functions/
│ └── generate_video_script.py # GenerateVideoScriptFunction
├── business/
│ ├── tts_service.py # TTSService (cloud + self-hosted)
│ ├── video_composition.py # VideoCompositionService
│ └── video_publisher.py # VideoPublisherService
├── tasks/
│ └── video_tasks.py # Celery tasks (video queue)
├── urls/
│ └── video.py # Video endpoints
└── migrations/
└── XXXX_add_video_models.py
```
### Conventions
- **PKs:** BigAutoField (integer) — do NOT use UUIDs
- **Table prefix:** `igny8_` on all new tables
- **App label:** `video` (new app)
- **Celery app name:** `igny8_core`
- **Celery queue:** `video` for all render/composition tasks (default queue for engagement fetch)
- **URL pattern:** `/api/v1/video/...`
- **Permissions:** Use `SiteSectorModelViewSet` permission pattern
- **Docker:** FFmpeg must be installed in Docker image; dedicated `celery_video_worker` service
- **AI functions:** Extend `BaseAIFunction`; register as `generate_video_script`
- **Frontend:** `.tsx` files with Zustand stores
### Cross-References
| Doc | Relationship |
|-----|-------------|
| **02H** | Socializer provides SocialAccount model + OAuth for YouTube/Instagram/TikTok publishing |
| **0F** | Self-hosted AI infrastructure provides GPU for TTS + image generation |
| **01E** | Pipeline Stage 9 integration — hooks after Stage 8 (social) or Stage 7 (publish) |
| **02G** | VideoObject schema generated for content with published video |
| **04A** | Managed services may include video creation as premium tier |
### Key Decisions
1. **New `video` app** — Separate app because video has 6 models and complex pipeline logic distinct from social posting
2. **Dedicated Celery queue** — Video rendering is CPU/GPU intensive (5-30 min); isolated `video` queue with concurrency=1 prevents blocking other tasks
3. **VideoScript, VideoAsset as plain models.Model** — Not SiteSectorBaseModel because they're children of VideoProject which carries the site/sector context
4. **Multiple RenderedVideo per project** — One project can target multiple platforms; each gets its own render at the correct resolution
5. **Reuse 02H OAuth** — PublishedVideo references SocialAccount from 02H; no duplicate OAuth infrastructure for video platforms
6. **Temp file cleanup** — Intermediate files (raw audio, image frames, non-final renders) cleaned up after successful publish to manage disk space
### System Requirements
- FFmpeg installed on server (add to Docker image via `apt-get install -y ffmpeg`)
- Python packages: `moviepy`, `pydub`, `Pillow`, `pysrt`
- Sufficient disk space for video temp files (cleanup after publish)
- Self-hosted GPU (from 0F) for TTS + AI image generation (optional — cloud fallback available)
- Dedicated Celery worker for `video` queue: `celery -A igny8_core worker -Q video --concurrency=1`

911
v2/V2-Execution-Docs/03A-wp-plugin-standalone.md Normal file
View File

@@ -0,0 +1,911 @@
# IGNY8 Phase 3: WordPress Plugin — Standalone SEO (03A)
## 10-Module Free SEO Plugin for WordPress
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 3 — WordPress Ecosystem
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, WordPress Developers, Architects
---
## 1. CURRENT STATE
### Existing WordPress Integration
The IGNY8 WordPress Bridge Plugin v1.5.2 exists at `/data/app/igny8/plugins/wordpress/source/igny8-wp-bridge/`. It provides:
- Content sync from IGNY8 SaaS → WordPress via `POST /wp-json/igny8/v1/sync/push`
- SAG blueprint sync via `POST /wp-json/igny8/v1/sag/sync-blueprint`
- Taxonomy creation via `POST /wp-json/igny8/v1/sag/create-taxonomies`
- API key authentication using `X-IGNY8-API-KEY` header
- REST namespace: `/wp-json/igny8/v1/`
### What Does NOT Exist
- No standalone SEO capabilities (users rely on Yoast/RankMath alongside IGNY8)
- No schema generation, sitemap control, redirect management, or site audit
- No internal link mapping or social sharing features
- No analytics connector or SMTP mail management
- No migration path from competing plugins
### Phase 2 Foundation Available
- 02D Linker Internal: SAGLink model and link scoring logic in IGNY8 backend — feeds the plugin's internal linking module in connected mode
- 02G Rich Schema SERP: 10 JSON-LD schema types and schema validation in IGNY8 backend — feeds the plugin's schema module in connected mode
### What v2 Changes
Phase 3 builds a **completely new plugin** (not a patch to v1.5.2). The v1.5.2 bridge becomes deprecated once v2 is deployed. The new plugin works in two modes:
- **Standalone mode** (this doc): Full free SEO plugin — no IGNY8 subscription required
- **Connected mode** (03B): Extends standalone with IGNY8 SaaS integration
---
## 2. WHAT TO BUILD
### Overview
A comprehensive WordPress SEO plugin with 10 standalone modules that function without an IGNY8 SaaS subscription. Each module is independently toggleable from Settings → Modules. The plugin replaces Yoast SEO, RankMath, or All in One SEO as a complete, free alternative.
### Plugin Identity
- **Plugin Name:** IGNY8
- **Text Domain:** `igny8`
- **REST Namespace:** `/wp-json/igny8/v1/`
- **Option Prefix:** `igny8_`
- **Post Meta Prefix:** `_igny8_`
- **Term Meta Prefix:** `_igny8_term_`
- **Table Prefix:** `{wp_prefix}igny8_`
- **Min Requirements:** WordPress 5.9+, PHP 7.4+
- **License:** GPL v2+
- **Public API:** `igny8()` returns main singleton — e.g., `igny8()->seo->get_title()`, `igny8()->schema->get_json_ld()`
### Module 1: SEO Core
**Purpose:** Focus keyword management, title/meta tag control, content analysis, Open Graph, breadcrumbs, and robots meta — the foundation of on-page SEO.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_SEO_Module` | `modules/seo/class-seo-module.php` | Module bootstrap, hook registration |
| `IGNY8_Meta_Box` | `modules/seo/class-meta-box.php` | Post editor meta box UI + save handler |
| `IGNY8_Title_Tag` | `modules/seo/class-title-tag.php` | `<title>` management via `document_title_parts` filter |
| `IGNY8_Meta_Tags` | `modules/seo/class-meta-tags.php` | Output `<meta>` description, robots, canonical in `<head>` |
| `IGNY8_Content_Analysis` | `modules/seo/class-content-analysis.php` | Real-time SEO scoring in admin JS |
| `IGNY8_Breadcrumbs` | `modules/seo/class-breadcrumbs.php` | Breadcrumb HTML generation + BreadcrumbList schema |
| `IGNY8_OpenGraph` | `modules/seo/class-opengraph.php` | OG + Twitter Card meta tags |
| `IGNY8_Robots_Txt` | `modules/seo/class-robots-txt.php` | Virtual robots.txt manager |
| `IGNY8_Verification` | `modules/seo/class-verification.php` | Webmaster verification code output |
**Features:**
- Focus keyword per post/page/product (primary + secondary keywords array)
- SEO title + meta description editor with live SERP preview
- Content analysis: keyword density, heading structure, readability (Flesch-Kincaid)
- Title tag templates per post type, taxonomy, author, and date archives
- Breadcrumbs via shortcode `[igny8_breadcrumbs]` + function `igny8()->seo->get_breadcrumbs()`
- Open Graph meta tags (og:title, og:description, og:image, og:type)
- Twitter Cards (twitter:card, twitter:title, twitter:description, twitter:image)
- Robots meta per post (noindex, nofollow, noarchive)
- Canonical URL override
- Webmaster verification codes (Google, Bing, Yandex, Pinterest)
- Consolidated `wp_head` output at priority 1 — single hook for all `<title>`, meta, OG, robots, canonical, schema
**Post Meta Keys:**
```
_igny8_focus_keyword -- string: primary focus keyword
_igny8_secondary_keywords -- JSON array: additional target keywords
_igny8_seo_title -- string: custom SEO title
_igny8_meta_description -- string: custom meta description
_igny8_canonical_url -- string: canonical URL override
_igny8_robots_index -- string: index/noindex
_igny8_robots_follow -- string: follow/nofollow
_igny8_og_title -- string: OG title override
_igny8_og_description -- string: OG description override
_igny8_og_image -- int: attachment ID for OG image
_igny8_twitter_title -- string: Twitter title override
_igny8_twitter_description -- string: Twitter description override
_igny8_seo_score -- int: 0-100 overall SEO score
_igny8_content_score -- int: 0-100 content quality score
_igny8_readability_score -- float: Flesch-Kincaid readability
_igny8_last_analysis -- string: ISO timestamp of last analysis
```
**Term Meta Keys:**
```
_igny8_term_seo_title -- string: term archive SEO title
_igny8_term_meta_description -- string: term archive meta description
_igny8_term_robots_index -- string: index/noindex
_igny8_term_og_image -- int: attachment ID
```
**Options:**
```
igny8_seo_settings -- JSON: {title_templates: {...}, separator: " | ", defaults: {...}}
igny8_social_profiles -- JSON: {facebook, twitter, linkedin, youtube, instagram, pinterest}
```
**Hooks:**
- Actions: `wp_head` (priority 1), `add_meta_boxes`, `save_post`, `document_title_parts`
- Filters: `igny8_title_parts`, `igny8_title_separator`, `igny8_meta_description`, `igny8_robots_meta`, `igny8_canonical_url`, `igny8_breadcrumbs`, `igny8_seo_score`
### Module 2: Schema (JSON-LD)
**Purpose:** Automatic and customizable structured data markup for rich search results.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Schema_Module` | `modules/schema/class-schema-module.php` | Module bootstrap |
| `IGNY8_Schema_Generator` | `modules/schema/class-schema-generator.php` | Main dispatcher, merges all types |
| `IGNY8_Schema_Article` | `modules/schema/class-schema-article.php` | Article/BlogPosting schema |
| `IGNY8_Schema_LocalBusiness` | `modules/schema/class-schema-local-business.php` | LocalBusiness schema |
| `IGNY8_Schema_Product` | `modules/schema/class-schema-product.php` | Product schema (WooCommerce) |
| `IGNY8_Schema_FAQ` | `modules/schema/class-schema-faq.php` | FAQPage schema |
| `IGNY8_Schema_HowTo` | `modules/schema/class-schema-howto.php` | HowTo schema |
| `IGNY8_Schema_Organization` | `modules/schema/class-schema-organization.php` | Organization/Person |
| `IGNY8_Schema_WebPage` | `modules/schema/class-schema-webpage.php` | WebPage/CollectionPage |
| `IGNY8_Schema_Breadcrumb` | `modules/schema/class-schema-breadcrumb.php` | BreadcrumbList |
| `IGNY8_Schema_Service` | `modules/schema/class-schema-service.php` | Service schema |
| `IGNY8_Schema_Custom` | `modules/schema/class-schema-custom.php` | Raw JSON-LD editor |
**Features:**
- Auto-detect schema type per post type
- 10 supported types: Article/BlogPosting, Organization/Person, WebPage/CollectionPage, BreadcrumbList, FAQPage, HowTo, LocalBusiness, Service, Product (WooCommerce), WebSite+SearchAction
- Custom schema editor (raw JSON-LD input)
- Schema validation against Google Rich Results requirements
- Global settings: Organization vs Person, default types per post type
- Single `<script type="application/ld+json">` block in head (merged from all types)
**Methods:** `get_json_ld($post_id)`, `render_schema($post_id)`, `detect_schema_type($post_id)`, `get_organization_schema()`, `get_breadcrumb_schema($post_id)`
**Post Meta:** `_igny8_schema_type`, `_igny8_schema_custom`, `_igny8_schema_faq` (connected), `_igny8_schema_howto` (connected)
**Options:** `igny8_schema_settings` — JSON: `{org_type, name, logo, knowledge_graph}`
**Hooks:** `igny8_schema_types`, `igny8_schema_{type}`, `igny8_json_ld`
### Module 3: Sitemap (XML)
**Purpose:** XML sitemaps with image support and search engine notification.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Sitemap_Module` | `modules/sitemap/class-sitemap-module.php` | Module bootstrap |
| `IGNY8_Sitemap_Index` | `modules/sitemap/class-sitemap-index.php` | Sitemap index file |
| `IGNY8_Sitemap_Posts` | `modules/sitemap/class-sitemap-posts.php` | Post type sub-sitemaps |
| `IGNY8_Sitemap_Taxonomies` | `modules/sitemap/class-sitemap-taxonomies.php` | Taxonomy sub-sitemaps |
| `IGNY8_Sitemap_Images` | `modules/sitemap/class-sitemap-images.php` | Image sitemap |
| `IGNY8_Sitemap_XSL` | `modules/sitemap/class-sitemap-xsl.php` | XSL stylesheet for browser view |
**Rewrite Rules:**
```
/sitemap_index.xml → sitemap index
/sitemap-{post_type}.xml → post type sitemaps
/sitemap-{taxonomy}.xml → taxonomy sitemaps
/sitemap-images.xml → image sitemap
/sitemap.xsl → XSL stylesheet
```
**Features:**
- Sitemap index + sub-sitemaps per post type and taxonomy
- Image sitemaps
- XSL stylesheet for human-readable browser view
- Respects noindex settings from SEO Core module
- Auto-ping search engines on update
- Exclusion rules (specific posts, taxonomies)
**Options:** `igny8_sitemap_settings` — JSON: `{included_post_types, included_taxonomies, excluded_urls}`
**Hooks:** `igny8_sitemap_post_types`, `igny8_sitemap_taxonomies`, `igny8_sitemap_excluded_urls`
### Module 4: Redirects
**Purpose:** 301/302/307 redirect management with 404 monitoring and auto-redirect on slug changes.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Redirects_Module` | `modules/redirects/class-redirects-module.php` | Module bootstrap |
| `IGNY8_Redirect_Manager` | `modules/redirects/class-redirect-manager.php` | CRUD + matching engine |
| `IGNY8_404_Monitor` | `modules/redirects/class-404-monitor.php` | 404 logging with hit counts |
| `IGNY8_Auto_Redirect` | `modules/redirects/class-auto-redirect.php` | Auto-redirect on slug change via `post_updated` |
**Features:**
- CRUD for 301/302/307 redirects
- 404 monitoring with hit counts, referrer, user agent (hashed IP for privacy)
- Auto-redirect on slug change via `post_updated` hook
- CSV import/export
- Regex redirect support (advanced)
- One-click redirect creation from 404 log
**Methods:** `add_redirect($source, $target, $type)`, `log_404($url, $referrer)`, `create_redirect_from_404($url, $target)`
**Options:** `igny8_redirect_settings` — JSON: `{auto_redirect_on_slug_change: true}`
**Database Table: `{prefix}igny8_redirects`**
```sql
CREATE TABLE {prefix}igny8_redirects (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
source_url VARCHAR(500) NOT NULL,
target_url VARCHAR(500) NOT NULL,
type SMALLINT DEFAULT 301,
hits INT DEFAULT 0,
last_hit DATETIME NULL,
created_at DATETIME NOT NULL,
is_active TINYINT(1) DEFAULT 1,
INDEX idx_source (source_url(191))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
**Database Table: `{prefix}igny8_404_log`**
```sql
CREATE TABLE {prefix}igny8_404_log (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
url VARCHAR(500) NOT NULL,
referrer VARCHAR(500) NULL,
user_agent VARCHAR(500) NULL,
ip_hash VARCHAR(64) NULL,
hits INT DEFAULT 1,
first_hit DATETIME NOT NULL,
last_hit DATETIME NOT NULL,
is_resolved TINYINT(1) DEFAULT 0,
redirect_id BIGINT NULL,
INDEX idx_url (url(191))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 5: Site Intelligence
**Purpose:** Automated site audit with orphan page detection, thin content scanning, duplicate meta finder, and keyword cannibalization detection.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Intelligence_Module` | `modules/site-intelligence/class-intelligence-module.php` | Module bootstrap |
| `IGNY8_Site_Audit` | `modules/site-intelligence/class-site-audit.php` | Full audit runner |
| `IGNY8_Orphan_Detector` | `modules/site-intelligence/class-orphan-detector.php` | Pages with no inbound links |
| `IGNY8_Thin_Content` | `modules/site-intelligence/class-thin-content.php` | Below threshold word count |
| `IGNY8_Empty_Terms` | `modules/site-intelligence/class-empty-terms.php` | Taxonomy terms with 0 posts |
| `IGNY8_Cannibalization` | `modules/site-intelligence/class-cannibalization.php` | Multiple pages targeting same keyword |
| `IGNY8_Duplicate_Meta` | `modules/site-intelligence/class-duplicate-meta.php` | Duplicate title/description finder |
| `IGNY8_Cluster_Detector` | `modules/site-intelligence/class-cluster-detector.php` | Topic grouping via keyword similarity |
| `IGNY8_Gap_Analysis` | `modules/site-intelligence/class-gap-analysis.php` | SAG gap analysis (connected mode only) |
| `IGNY8_Cluster_Health` | `modules/site-intelligence/class-cluster-health.php` | Cluster health scoring (connected mode only) |
**Features:**
- Site audit runner via WP Cron (weekly) + manual trigger
- Orphan page detection (no internal links pointing to it)
- Thin content scanner (configurable word count threshold, default 300)
- Empty taxonomy term detection
- Duplicate title/meta description finder
- Keyword cannibalization detection (multiple pages targeting same keyword)
- Auto-cluster detection (topic grouping across posts via keyword similarity)
- Dashboard with severity badges (critical/warning/info) and actionable fix suggestions
- Connected mode adds: SAG gap analysis, cluster health scoring (from 01G)
**Methods:** `run_full_audit()`, `get_audit_results($args)`, `get_orphan_pages()`, `get_thin_content()`, `detect_clusters($posts)`
**Options:** `igny8_intelligence_settings` — JSON: `{audit_schedule, thresholds: {thin_content_min: 300}, exclusions}`
**Database Table: `{prefix}igny8_audit_results`**
```sql
CREATE TABLE {prefix}igny8_audit_results (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
audit_type VARCHAR(50) NOT NULL,
post_id BIGINT NULL,
term_id BIGINT NULL,
severity VARCHAR(20) NOT NULL,
message TEXT NOT NULL,
data LONGTEXT NULL,
audit_date DATETIME NOT NULL,
is_resolved TINYINT(1) DEFAULT 0,
INDEX idx_type_date (audit_type, audit_date)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 6: Internal Linking
**Purpose:** Site-wide link crawl, per-post link audit, link suggestions, and orphan content detection.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Linking_Module` | `modules/linking/class-linking-module.php` | Module bootstrap |
| `IGNY8_Link_Auditor` | `modules/linking/class-link-auditor.php` | Per-post link audit |
| `IGNY8_Link_Suggestions` | `modules/linking/class-link-suggestions.php` | Content similarity link suggestions |
| `IGNY8_Orphan_Links` | `modules/linking/class-orphan-links.php` | Pages with zero inbound links |
| `IGNY8_Link_Graph` | `modules/linking/class-link-graph.php` | Link equity visualization |
| `IGNY8_Auto_Linker` | `modules/linking/class-auto-linker.php` | Auto-link insertion (connected mode only) |
| `IGNY8_Link_Rules` | `modules/linking/class-link-rules.php` | Keyword-based link rules (connected mode only) |
**Features:**
- Full site link crawl + link map stored in `{prefix}igny8_link_map`
- Per-post link audit: outbound count, inbound count, orphan status
- Link suggestions: "This post should link to X" based on content similarity
- Link equity visualization (which pages pass authority)
- Orphan content detection (pages with zero inbound internal links)
- Connected mode adds: auto-link insertion based on keyword rules from IGNY8 backend (02D)
**Methods:** `crawl_links()`, `get_post_links($post_id)`, `get_link_suggestions($post_id)`, `get_orphan_pages()`, `auto_link_content($post_id)` (connected)
**Post Meta:** `_igny8_related_links` (JSON: `[{post_id, anchor_text, priority}]`), `_igny8_link_suggestions` (JSON)
**Options:** `igny8_linking_settings` — JSON: `{auto_link_rules, max_links_per_post}`
**Database Table: `{prefix}igny8_link_map`**
```sql
CREATE TABLE {prefix}igny8_link_map (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
source_post_id BIGINT NOT NULL,
target_post_id BIGINT NOT NULL,
anchor_text VARCHAR(500) NULL,
is_follow TINYINT(1) DEFAULT 1,
link_position VARCHAR(20) NULL,
last_crawled DATETIME NOT NULL,
INDEX idx_source (source_post_id),
INDEX idx_target (target_post_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 7: Google Search Console
**Purpose:** OAuth 2.0 GSC connection with search performance dashboard and keyword tracking.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_GSC_Module` | `modules/gsc/class-gsc-module.php` | Module bootstrap |
| `IGNY8_GSC_Auth` | `modules/gsc/class-gsc-auth.php` | OAuth 2.0 flow |
| `IGNY8_GSC_Dashboard` | `modules/gsc/class-gsc-dashboard.php` | Performance dashboard |
| `IGNY8_GSC_Keywords` | `modules/gsc/class-gsc-keywords.php` | Keyword position tracking |
| `IGNY8_GSC_Indexing` | `modules/gsc/class-gsc-indexing.php` | URL indexing requests (connected mode only) |
| `IGNY8_GSC_URL_Inspection` | `modules/gsc/class-gsc-url-inspection.php` | URL Inspection API (connected mode only) |
**Features:**
- OAuth 2.0 connection flow to Google Search Console
- Search performance dashboard: clicks, impressions, CTR, avg position
- Keyword position tracking over time
- Date range filtering
- Top pages by performance
- Data stored locally for historical tracking
- Connected mode adds: URL Inspection API, indexing requests, status sync from IGNY8 backend (02C)
**Methods:** `authorize_gsc()`, `get_search_metrics($args)`, `get_keyword_positions($args)`, `inspect_url($url)` (connected), `request_indexing($url)` (connected)
**Options:** `igny8_gsc_token` (encrypted), `igny8_gsc_property` (selected GSC property URL)
### Module 8: Socializer
**Purpose:** Lightweight share buttons and social profile management.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Socializer_Module` | `modules/socializer/class-socializer-module.php` | Module bootstrap |
| `IGNY8_Share_Buttons` | `modules/socializer/class-share-buttons.php` | CSS/JS share button rendering |
| `IGNY8_Social_Profiles` | `modules/socializer/class-social-profiles.php` | Organization social URLs |
| `IGNY8_Auto_Poster` | `modules/socializer/class-auto-poster.php` | Auto-post on publish (connected mode only) |
| `IGNY8_Twitter_API` | `modules/socializer/class-twitter-api.php` | Twitter/X API (connected mode only) |
| `IGNY8_Facebook_API` | `modules/socializer/class-facebook-api.php` | Facebook Pages API (connected mode only) |
| `IGNY8_LinkedIn_API` | `modules/socializer/class-linkedin-api.php` | LinkedIn API (connected mode only) |
| `IGNY8_OG_Image_Generator` | `modules/socializer/class-og-image-generator.php` | Dynamic OG image (connected mode only) |
**Features:**
- Pure CSS/JS share buttons (<5KB total, no external scripts)
- Position options: floating sidebar, above content, below content, inline
- Networks: Facebook, X/Twitter, LinkedIn, Pinterest, WhatsApp, Telegram, Email, Copy Link
- Social profile management (org social URLs)
- Share count display (optional, via API)
- Connected mode adds: auto-post on publish via 02H SocialAccount OAuth, dynamic OG image generation
**Methods:** `render_share_buttons($post_id)`, `get_social_profiles()`, `post_to_social($post_id)` (connected)
**Options:** `igny8_share_settings` — JSON: `{networks, position, style}`
**Frontend Assets:** `share-buttons.css` (<3KB), `share-buttons.js` (<2KB) — conditionally enqueued only when enabled
### Module 9: Analytics + SMTP
**Purpose:** Analytics script injection (GA4, GTM, pixels) and SMTP mail override.
**Classes (Analytics):**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Analytics_Module` | `modules/analytics/class-analytics-module.php` | Module bootstrap |
| `IGNY8_GA_Connector` | `modules/analytics/class-ga-connector.php` | GA4 measurement ID injection |
| `IGNY8_GTM_Connector` | `modules/analytics/class-gtm-connector.php` | Google Tag Manager container |
| `IGNY8_Pixel_Manager` | `modules/analytics/class-pixel-manager.php` | Facebook, TikTok, Pinterest pixels |
| `IGNY8_Script_Manager` | `modules/analytics/class-script-manager.php` | Custom header/footer scripts |
**Classes (SMTP):**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_SMTP_Module` | `modules/smtp/class-smtp-module.php` | Module bootstrap |
| `IGNY8_SMTP_Mailer` | `modules/smtp/class-smtp-mailer.php` | SMTP mail override |
| `IGNY8_Email_Log` | `modules/smtp/class-email-log.php` | Delivery log with status |
| `IGNY8_Email_Test` | `modules/smtp/class-email-test.php` | Test email sender |
**Features (Analytics):**
- GA4 measurement ID connector (script injection via `wp_head`)
- Google Tag Manager container ID
- Facebook Pixel, TikTok Pixel, Pinterest Tag
- Custom header/footer scripts (with placement control)
**Features (SMTP):**
- SMTP mail override (host, port, user, pass, encryption SSL/TLS)
- Email delivery log with status tracking
- Test email sender
- From name + from email override
**Options:**
```
igny8_analytics_settings -- JSON: {ga_id, gtm_id, fb_pixel, tiktok_pixel, pinterest_tag, custom_scripts}
igny8_smtp_settings -- JSON: {host, port, user, pass, encryption, from_name, from_email}
```
**SMTP Hooks:** `wp_mail` (override), `wp_mail_from`, `wp_mail_from_name`
**Database Table: `{prefix}igny8_email_log`**
```sql
CREATE TABLE {prefix}igny8_email_log (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
to_email VARCHAR(320) NOT NULL,
subject VARCHAR(500) NOT NULL,
status VARCHAR(20) NOT NULL,
error_message TEXT NULL,
sent_at DATETIME NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 10: Import (Migration)
**Purpose:** One-click migration from Yoast SEO, RankMath, and All in One SEO.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Importer` | `data/class-importer.php` | Base importer with rollback |
| `IGNY8_Yoast_Importer` | `data/class-yoast-importer.php` | Yoast SEO migration |
| `IGNY8_RankMath_Importer` | `data/class-rankmath-importer.php` | RankMath migration |
| `IGNY8_AIOSEO_Importer` | `data/class-aioseo-importer.php` | All in One SEO migration |
**Features:**
- One-click import from: Yoast SEO, RankMath, All in One SEO
- Imports: focus keywords, SEO titles, meta descriptions, robots settings, redirects, schema settings
- Pre-import preview showing what will be migrated (dry run mode)
- Rollback capability (stores original values before overwrite)
**Meta Key Mapping (Yoast → IGNY8):**
```
_yoast_wpseo_focuskw → _igny8_focus_keyword
_yoast_wpseo_title → _igny8_seo_title
_yoast_wpseo_metadesc → _igny8_meta_description
_yoast_wpseo_canonical → _igny8_canonical_url
_yoast_wpseo_meta-robots-noindex → _igny8_robots_index
_yoast_wpseo_opengraph-title → _igny8_og_title
_yoast_wpseo_opengraph-description → _igny8_og_description
```
**Meta Key Mapping (RankMath → IGNY8):**
```
rank_math_focus_keyword → _igny8_focus_keyword
rank_math_title → _igny8_seo_title
rank_math_description → _igny8_meta_description
rank_math_canonical_url → _igny8_canonical_url
rank_math_robots → _igny8_robots_index + _igny8_robots_follow
rank_math_facebook_title → _igny8_og_title
rank_math_facebook_description → _igny8_og_description
```
**Meta Key Mapping (AIOSEO → IGNY8):**
```
_aioseo_title → _igny8_seo_title
_aioseo_description → _igny8_meta_description
_aioseo_canonical_url → _igny8_canonical_url
_aioseo_robots_noindex → _igny8_robots_index
_aioseo_og_title → _igny8_og_title
_aioseo_og_description → _igny8_og_description
```
---
## 3. DATA MODELS & APIS
### Database Tables (6 total)
All tables created on plugin activation by `class-installer.php`:
| # | Table | Module | Purpose |
|---|-------|--------|---------|
| 1 | `{prefix}igny8_redirects` | Redirects | 301/302/307 redirect rules |
| 2 | `{prefix}igny8_404_log` | Redirects | 404 error logging |
| 3 | `{prefix}igny8_link_map` | Internal Linking | Page-to-page link relationships |
| 4 | `{prefix}igny8_audit_results` | Site Intelligence | Audit findings |
| 5 | `{prefix}igny8_email_log` | SMTP | Email delivery log |
| 6 | `{prefix}igny8_sync_queue` | (Connected prep) | Sync queue for connected mode readiness |
**Table 6: `{prefix}igny8_sync_queue`** (created in standalone for connected mode readiness)
```sql
CREATE TABLE {prefix}igny8_sync_queue (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
igny8_content_id VARCHAR(100) NOT NULL,
content_type VARCHAR(50) NOT NULL,
sync_status VARCHAR(20) NOT NULL DEFAULT 'pending',
wp_post_id BIGINT NULL,
wp_term_id BIGINT NULL,
data LONGTEXT NULL,
created_at DATETIME NOT NULL,
synced_at DATETIME NULL,
error_message TEXT NULL,
INDEX idx_status (sync_status)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### REST API Endpoints (public)
All endpoints under `/wp-json/igny8/v1/`:
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET | `/related/{post_id}` | Related content links | None (public) |
| GET | `/cluster/{cluster_id}` | Cluster content list | None (public) |
| GET | `/term/{term_id}/content` | Rich term content | None (public) |
| GET | `/audit/summary` | Site audit results | Admin nonce |
| GET | `/structure/overview` | Site structure summary | Admin nonce |
### Options Summary
All options use `igny8_` prefix and store JSON:
| Option Key | Module | Content |
|------------|--------|---------|
| `igny8_seo_settings` | SEO Core | Title templates, separator, defaults |
| `igny8_social_profiles` | SEO Core | Organization social URLs |
| `igny8_schema_settings` | Schema | Org type, name, logo, knowledge graph |
| `igny8_sitemap_settings` | Sitemap | Included post types, taxonomies |
| `igny8_redirect_settings` | Redirects | Auto-redirect on slug change |
| `igny8_intelligence_settings` | Site Intelligence | Audit schedule, thresholds, exclusions |
| `igny8_linking_settings` | Internal Linking | Auto-link rules, max links per post |
| `igny8_gsc_token` | GSC | Encrypted OAuth token |
| `igny8_gsc_property` | GSC | Selected GSC property URL |
| `igny8_share_settings` | Socializer | Networks, position, style |
| `igny8_analytics_settings` | Analytics | GA4, GTM, pixel IDs, custom scripts |
| `igny8_smtp_settings` | SMTP | Host, port, user, pass, encryption |
---
## 4. IMPLEMENTATION STEPS
### File Structure
```
igny8/
├── igny8.php # Plugin header, constants, bootstrap
├── readme.txt # WordPress.org readme
├── uninstall.php # Full cleanup on uninstall
├── includes/
│ ├── class-igny8.php # Main singleton class
│ ├── class-module-manager.php # Register/activate/deactivate modules
│ ├── class-api-client.php # IGNY8 SaaS API client (connected mode)
│ ├── class-connection.php # API key validation, connection status
│ ├── class-utils.php # Shared utilities
│ ├── class-rest-api.php # Plugin REST endpoints
│ ├── class-compatibility.php # Detect Yoast/RankMath conflicts
│ ├── modules/
│ │ ├── seo/ # Module 1 — 9 class files
│ │ ├── schema/ # Module 2 — 12 class files
│ │ ├── sitemap/ # Module 3 — 6 class files
│ │ ├── redirects/ # Module 4 — 4 class files
│ │ ├── site-intelligence/ # Module 5 — 10 class files
│ │ ├── linking/ # Module 6 — 7 class files
│ │ ├── gsc/ # Module 7 — 6 class files
│ │ ├── socializer/ # Module 8 — 8 class files
│ │ ├── analytics/ # Module 9a — 5 class files
│ │ ├── smtp/ # Module 9b — 4 class files
│ │ ├── content-sync/ # Module 11 [CONNECTED] — 6 class files
│ │ └── sag/ # Module 12 [CONNECTED] — 6 class files
│ ├── admin/
│ │ ├── class-admin-menu.php # Top-level menu + submenus
│ │ ├── class-dashboard.php # Overview + quick actions
│ │ ├── class-setup-wizard.php # 6-step first-run wizard
│ │ ├── class-compatibility-notice.php
│ │ ├── views/ # PHP template files
│ │ └── assets/ # admin.css, admin.js, meta-box.css, meta-box.js
│ ├── frontend/
│ │ ├── class-head-output.php # All <head> tag consolidation
│ │ ├── class-share-output.php # Share button rendering
│ │ └── assets/ # share-buttons.css, share-buttons.js
│ ├── data/
│ │ ├── class-installer.php # Create all DB tables on activation
│ │ ├── class-migrator.php # Version-based migration handler
│ │ └── class-importer.php # Base importer + Yoast/RankMath/AIOSEO
│ └── integrations/
│ ├── class-woocommerce.php # WooCommerce SEO + schema
│ └── class-theme-bridge.php # Communication with companion theme
└── languages/
└── igny8.pot # Translation template
```
### Build Sequence
**Phase 1: Foundation (Days 1-3)**
1. Create plugin skeleton: `igny8.php` with header, constants, autoloader
2. Build `class-igny8.php` singleton — `instance()`, `init()`, `load_modules()`
3. Build `class-module-manager.php` — `register()`, `activate()`, `deactivate()`, `is_active()`, `boot()`
4. Build admin menu structure: `class-admin-menu.php` with top-level + submenus
5. Build setup wizard: `class-setup-wizard.php` with 6 steps
6. Build `class-installer.php` — create all 6 DB tables via `dbDelta()`
7. Build `class-rest-api.php` — register 5 public endpoints
8. Build `class-compatibility.php` — detect Yoast/RankMath/AIOSEO active
**Phase 2: SEO Core (Days 4-7)**
1. Build `IGNY8_SEO_Module` — module bootstrap + all hook registrations
2. Build `IGNY8_Meta_Box` — post editor meta box with all 16 post meta fields
3. Build `IGNY8_Title_Tag` — template system via `document_title_parts` filter
4. Build `IGNY8_Meta_Tags` — `<meta>` output in `<head>`
5. Build `IGNY8_Content_Analysis` — real-time SEO scoring (keyword density, heading structure, readability)
6. Build `IGNY8_Breadcrumbs` — shortcode + BreadcrumbList schema
7. Build `IGNY8_OpenGraph` — OG + Twitter Card tags
8. Build `IGNY8_Robots_Txt` — virtual robots.txt
9. Build `IGNY8_Verification` — webmaster codes
10. Build `class-head-output.php` — consolidated `wp_head` at priority 1
**Phase 3: Schema + Sitemap + Redirects (Days 8-10)**
1. Build schema module: generator dispatcher + 10 type-specific classes
2. Build sitemap module: index + post/taxonomy/image sub-sitemaps + XSL
3. Build redirects module: CRUD + 404 monitor + auto-redirect
**Phase 4: Site Intelligence (Days 11-13)**
1. Build audit runner with WP Cron weekly schedule
2. Build 7 audit type classes (orphan, thin, empty terms, cannibalization, duplicate meta, cluster detector)
3. Build audit dashboard with severity badges
**Phase 5: Internal Linking (Days 14-15)**
1. Build link crawl engine + link map table
2. Build per-post link audit + suggestions
3. Build link graph visualization
**Phase 6: Socializer + Analytics/SMTP + GSC (Days 16-18)**
1. Build share buttons (<5KB CSS/JS)
2. Build analytics script connectors
3. Build SMTP override + email log
4. Build GSC OAuth flow + dashboard
**Phase 7: Import (Days 19-20)**
1. Build base importer with dry run + rollback
2. Build Yoast → IGNY8 mapping
3. Build RankMath → IGNY8 mapping
4. Build AIOSEO → IGNY8 mapping
**Phase 8: Compliance (Days 28-30)**
1. WordPress.org compliance audit — sanitize all inputs, escape all outputs
2. i18n pass — all strings through `__()` / `esc_html__()` with `igny8` domain
3. readme.txt with full feature list, changelog, screenshots
4. Accessibility review — ARIA labels, keyboard navigation in admin
### Setup Wizard (6 Steps)
| Step | Title | Content |
|------|-------|---------|
| 1 | Site Type | Blog, Business, eCommerce, Local Business, SaaS, Portfolio |
| 2 | SEO Import | Detect existing Yoast/RankMath/AIOSEO, offer import |
| 3 | Site Basics | Site name, Organization/Person, logo, title separator |
| 4 | Social Profiles | Social URLs + default sharing image |
| 5 | Connect to IGNY8 | API key input (optional, clear skip button) |
| 6 | Done | Summary, run first audit CTA |
### Admin Menu Structure
```
IGNY8 (top-level, dashicons-chart-area)
├── Dashboard — Overview + quick actions
├── SEO — Meta box settings, title templates
├── Schema — Default types, organization settings
├── Sitemaps — Post type/taxonomy inclusion
├── Redirects — CRUD + 404 monitor
├── Site Intelligence — Audit dashboard
├── Internal Links — Link map, suggestions
├── Search Console — GSC dashboard
├── Socializer — Buttons, profiles
├── Analytics — GA4, GTM, pixels
├── SMTP — Mail settings, log
├── Import — Migration wizard
└── Settings — Modules toggle, general settings
```
### Performance Rules
| Rule | Implementation |
|------|---------------|
| Zero CSS on frontend unless share buttons enabled | Conditional `wp_enqueue_style` check |
| Zero JS on frontend unless interactive features active | Conditional `wp_enqueue_script` check |
| All `<head>` output in single hook at priority 1 | `class-head-output.php` consolidation |
| Schema as single JSON-LD block | `class-schema-generator.php` merges all types |
| Admin assets only on IGNY8 admin pages | `get_current_screen()` check |
| Meta box assets only on post editor | Screen check before enqueue |
| Site audit via WP Cron, never on page load | `wp_schedule_event()` weekly |
| Link crawl as background process | WP Cron, batch processing |
| Connected API calls never block page requests | WP Cron queue, async |
| Option reads via object cache | `wp_cache_get/set` wrapping |
### WordPress.org Compliance
- GPL v2+ license header in all files
- No phone-home without explicit user consent
- All user-facing strings translatable via `igny8` text domain using `__()`, `esc_html__()`, `_e()`, `esc_html_e()`
- Sanitize all inputs: `sanitize_text_field()`, `wp_kses_post()`, `absint()`, `sanitize_url()`
- Escape all outputs: `esc_html()`, `esc_attr()`, `esc_url()`, `wp_kses_post()`
- Use WP APIs: Settings API, REST API, `WP_Query` — no direct DB queries where WP functions exist
- Nonce verification on all form submissions: `wp_nonce_field()` / `wp_verify_nonce()`
- Capability checks: `current_user_can('manage_options')` for admin pages
---
## 5. ACCEPTANCE CRITERIA
### Foundation
- [ ] Plugin activates without errors on WordPress 5.9+ with PHP 7.4+
- [ ] All 6 database tables created on activation via `dbDelta()`
- [ ] Setup wizard runs on first activation with all 6 steps functional
- [ ] Module manager toggles modules on/off without errors
- [ ] `igny8()` public API returns singleton with module access
- [ ] Compatibility checker detects active Yoast/RankMath/AIOSEO and shows admin notice
### SEO Core (Module 1)
- [ ] Meta box renders on post/page/product edit screens with all fields
- [ ] SEO title and meta description save to post meta and render in `<head>`
- [ ] Title tag templates work with variables: `{title}`, `{site_name}`, `{separator}`
- [ ] Content analysis scores keyword density, heading structure, readability (0-100)
- [ ] Breadcrumbs render via shortcode `[igny8_breadcrumbs]` with BreadcrumbList schema
- [ ] Open Graph + Twitter Card tags render correctly in `<head>`
- [ ] Robots meta (noindex/nofollow) respected by output and sitemaps
- [ ] Canonical URL override renders in `<head>`
### Schema (Module 2)
- [ ] Auto-detects schema type per post type
- [ ] Renders valid JSON-LD in single `<script>` block
- [ ] All 10 schema types produce valid structured data
- [ ] Custom JSON-LD editor accepts and renders user input
- [ ] Schema passes Google Rich Results Test validation
### Sitemap (Module 3)
- [ ] `/sitemap_index.xml` renders with sub-sitemap links
- [ ] Per-post-type and per-taxonomy sub-sitemaps render
- [ ] Image sitemap included
- [ ] XSL stylesheet provides human-readable browser view
- [ ] Noindexed posts excluded from sitemap
- [ ] Search engines pinged on sitemap update
### Redirects (Module 4)
- [ ] Create/read/update/delete 301/302/307 redirects
- [ ] 404 errors logged with hit count, referrer, hashed IP
- [ ] Auto-redirect created on slug change
- [ ] One-click redirect from 404 log entry
- [ ] CSV import/export functional
### Site Intelligence (Module 5)
- [ ] Full audit runs via WP Cron weekly and manual trigger
- [ ] Orphan pages, thin content, empty terms, duplicates, cannibalization detected
- [ ] Dashboard shows severity badges (critical/warning/info) with counts
- [ ] Individual audit items dismissible/resolvable
### Internal Linking (Module 6)
- [ ] Full site link crawl completes and populates `igny8_link_map`
- [ ] Per-post link audit shows inbound/outbound counts
- [ ] Link suggestions generated based on content similarity
- [ ] Orphan pages (zero inbound links) identified
### GSC (Module 7)
- [ ] OAuth 2.0 flow connects to Google Search Console
- [ ] Dashboard shows clicks, impressions, CTR, avg position with date range
- [ ] Keyword position tracking displays historical data
### Socializer (Module 8)
- [ ] Share buttons render at configured position (<5KB total)
- [ ] All 8 networks functional (Facebook, X, LinkedIn, Pinterest, WhatsApp, Telegram, Email, Copy)
- [ ] Button display conditionally enqueued — zero CSS/JS when disabled
### Analytics + SMTP (Module 9)
- [ ] GA4, GTM, Facebook Pixel, TikTok Pixel scripts inject correctly
- [ ] Custom header/footer scripts render in correct position
- [ ] SMTP override sends mail via configured server
- [ ] Email log records all sent/failed emails
- [ ] Test email sends successfully
### Import (Module 10)
- [ ] Yoast SEO meta data imports correctly with key mapping
- [ ] RankMath meta data imports correctly with key mapping
- [ ] All in One SEO meta data imports correctly with key mapping
- [ ] Dry run preview shows migration summary without writing
- [ ] Rollback restores original values
### Performance & Compliance
- [ ] Zero frontend CSS/JS impact when share buttons disabled
- [ ] All `<head>` output consolidated in single `wp_head` hook at priority 1
- [ ] Admin assets loaded only on IGNY8 admin pages
- [ ] All user strings translatable via `igny8` text domain
- [ ] All inputs sanitized, all outputs escaped
- [ ] Nonce verification on all form submissions
- [ ] Plugin passes WordPress Plugin Check (PCP) tool
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
Before starting implementation:
1. Read existing WP Bridge plugin at `/data/app/igny8/plugins/wordpress/source/igny8-wp-bridge/` — understand current structure
2. Read 02D (Linker Internal) for IGNY8 backend linking logic that feeds Module 6
3. Read 02G (Rich Schema SERP) for schema types that feed Module 2
### Execution Order
```
Foundation → SEO Core → Schema+Sitemap+Redirects → Intelligence → Linking → Social+Analytics+GSC → Import → Compliance
```
### Critical Rules
1. **All IDs are integers** — BigAutoField on IGNY8 side, BIGINT AUTO_INCREMENT on WP side
2. **IGNY8 model names are PLURAL** — Clusters, Keywords, Tasks, ContentIdeas, Images, Content (stays singular)
3. **Table prefix** — All WP tables use `{wp_prefix}igny8_` prefix
4. **Post meta prefix** — All `_igny8_` prefixed
5. **Term meta prefix** — All `_igny8_term_` prefixed
6. **Option prefix** — All `igny8_` prefixed
7. **Text domain** — `igny8` for all translatable strings
8. **REST namespace** — `/wp-json/igny8/v1/`
9. **No direct DB queries** where WP functions exist (`WP_Query`, `get_post_meta`, `get_option`)
10. **Sanitize inputs** — `sanitize_text_field()`, `wp_kses_post()`, `absint()`, `sanitize_url()`
11. **Escape outputs** — `esc_html()`, `esc_attr()`, `esc_url()`, `wp_kses_post()`
12. **Nonce on forms** — `wp_nonce_field()` / `wp_verify_nonce()` / `check_ajax_referer()`
13. **Capability checks** — `current_user_can('manage_options')` for admin pages
14. **Module pattern** — Every module implements `register()`, `activate()`, `deactivate()`, `is_active()`, `boot()`
### Testing
- Activate plugin on WordPress 5.9+ with PHP 7.4+
- Run setup wizard through all 6 steps
- Toggle each module on/off — verify no errors
- Create a post, fill all SEO meta, verify `<head>` output
- Check sitemap renders at `/sitemap_index.xml`
- Create redirect, trigger 404, verify logging
- Run manual audit, verify dashboard results
- Run link crawl, verify link map populated
- Connect GSC via OAuth, verify dashboard data
- Test share buttons at all positions
- Configure GA4 + SMTP, verify injection + mail delivery
- Import from Yoast test data, verify mapping
- Run WordPress Plugin Check (PCP) tool for compliance
### File Creation Order
```
1. igny8.php (plugin header + bootstrap)
2. includes/class-igny8.php (singleton)
3. includes/class-module-manager.php (module orchestration)
4. includes/data/class-installer.php (DB tables)
5. includes/admin/class-admin-menu.php (menu structure)
6. includes/admin/class-setup-wizard.php (first-run wizard)
7. includes/class-rest-api.php (public endpoints)
8. includes/class-compatibility.php (conflict detection)
9. includes/modules/seo/* (Module 1 — 9 files)
10. includes/frontend/class-head-output.php (consolidated <head>)
11. includes/modules/schema/* (Module 2 — 12 files)
12. includes/modules/sitemap/* (Module 3 — 6 files)
13. includes/modules/redirects/* (Module 4 — 4 files)
14. includes/modules/site-intelligence/* (Module 5 — 10 files)
15. includes/modules/linking/* (Module 6 — 7 files)
16. includes/modules/gsc/* (Module 7 — 6 files)
17. includes/modules/socializer/* (Module 8 — 8 files)
18. includes/modules/analytics/* (Module 9a — 5 files)
19. includes/modules/smtp/* (Module 9b — 4 files)
20. includes/data/class-importer.php + importers (Module 10 — 4 files)
21. includes/integrations/class-woocommerce.php (WooCommerce)
22. includes/integrations/class-theme-bridge.php (theme communication)
23. uninstall.php (cleanup)
24. languages/igny8.pot (i18n template)
```
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 02D Linker Internal | IGNY8 backend SAGLink scoring → feeds Module 6 auto-linker in connected mode |
| 02G Rich Schema SERP | IGNY8 backend schema generation → feeds Module 2 via bulk push in connected mode |
| 02C GSC Integration | IGNY8 backend GSC metrics → feeds Module 7 URL Inspection in connected mode |
| 02H Socializer | IGNY8 backend SocialAccount OAuth → feeds Module 8 auto-poster in connected mode |
| 03B WP Plugin Connected | Extends this plugin with 2 connected-mode modules (Content Sync, SAG Structure) |
| 03C Companion Theme | Theme consumes plugin API: `igny8()->seo`, `igny8()->schema`, `igny8()->linking` |
| 03D Toolkit Plugin | If both installed, toolkit SMTP defers to this plugin's SMTP module |

487
v2/V2-Execution-Docs/03B-wp-plugin-connected.md Normal file
View File

@@ -0,0 +1,487 @@
# IGNY8 Phase 3: WordPress Plugin — Connected Mode (03B)
## IGNY8 SaaS ↔ WordPress Bidirectional Integration
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 3 — WordPress Ecosystem
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, WordPress Developers, Architects
---
## 1. CURRENT STATE
### Existing WordPress Bridge
The IGNY8 WordPress Bridge Plugin v1.5.2 at `/data/app/igny8/plugins/wordpress/source/igny8-wp-bridge/` provides basic content sync:
- Content push from IGNY8 → WordPress via `POST /wp-json/igny8/v1/sync/push`
- SAG blueprint sync via `POST /wp-json/igny8/v1/sag/sync-blueprint`
- Taxonomy creation via `POST /wp-json/igny8/v1/sag/create-taxonomies`
- API key authentication using `X-IGNY8-API-KEY` header
### What v2 Connected Mode Replaces
Phase 3 connected mode is part of the new v2 plugin (03A). It replaces and extends all v1.5.2 capabilities with:
- Full content queue + review workflow (not just raw push)
- Image downloading and attachment creation
- SAG blueprint → taxonomy + term → cluster hub page mapping
- Bidirectional sync events and confirmation callbacks
- Schema bulk push from IGNY8 backend (02G)
- GSC status sync from IGNY8 backend (02C)
- Auto-linking from IGNY8 backend (02D)
- Social auto-post from IGNY8 backend (02H)
### Prerequisites
- Standalone plugin (03A) must be installed and active
- IGNY8 SaaS subscription with valid API key
- Connection established via Setup Wizard Step 5 or Settings → Connect
### IGNY8 Backend Models That WordPress Interacts With
- `Content` (writer app, db_table=`igny8_content`) — content_type: post/page/product/taxonomy; content_structure choices
- `Tasks` (writer app, db_table=`igny8_tasks`) — writing tasks, status: queued/completed
- `Images` (writer app) — generated images for content
- `Clusters` (planner app, db_table=`igny8_clusters`) — keyword clusters
- `SAGBlueprint`, `SAGAttribute`, `SAGCluster` (sag app) — Phase 1 SAG models
- `SiteIntegration` (integration app) — WordPress connection details
- `SyncEvent` (integration app) — sync operation logs
- `PublishingSettings`, `PublishingRecord` (publishing app)
- All IDs are integers (BigAutoField) — never UUIDs
---
## 2. WHAT TO BUILD
### Overview
Two connected-mode modules that extend the standalone plugin (03A) when an IGNY8 SaaS subscription is active. These modules are only loaded when `igny8_api_connected` option is `true`.
### Module 11: Content Sync
**Purpose:** Receive content from IGNY8 SaaS, queue it, download images, map types, create/update WordPress posts, and confirm back.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_Sync_Module` | `modules/content-sync/class-sync-module.php` | Module entry, webhook receiver |
| `IGNY8_Content_Puller` | `modules/content-sync/class-content-puller.php` | Fetch content from IGNY8 SaaS API |
| `IGNY8_Content_Mapper` | `modules/content-sync/class-content-mapper.php` | Map IGNY8 content_type → WP post_type |
| `IGNY8_Image_Downloader` | `modules/content-sync/class-image-downloader.php` | Download images → `wp_upload_dir()`, `wp_insert_attachment()` |
| `IGNY8_Sync_Queue` | `modules/content-sync/class-sync-queue.php` | Sync queue DB table management |
| `IGNY8_Publish_Scheduler` | `modules/content-sync/class-publish-scheduler.php` | Review queue, scheduled publishing |
**Content Type Mapping:**
| IGNY8 content_type | WP post_type | Notes |
|----|----|----|
| `post` | `post` | Default blog post |
| `page` | `page` | Standard page |
| `product` | `product` | Requires WooCommerce active |
| `service` | `service` | Requires companion theme CPT (03C) |
| `company_page` | `page` | Uses `page-company` template |
| `taxonomy_landing` | — | Written to term description (not a post) |
| `cluster_hub` | `page` | Landing page preset applied |
| `comparison` | `post` | Standard post with comparison structure |
| `brand_page` | `page` | Standard page |
**Content Sync Flow:**
```
1. IGNY8 SaaS publishes content
2. → POST /wp-json/igny8/v1/sync/push (validated via X-IGNY8-API-KEY)
Payload: {igny8_content_id, content_type, title, content_html, excerpt,
featured_image, meta: {focus_keyword, secondary_keywords,
cluster_id, taxonomies: {service_category: [...], cluster: [...]}}}
3. → Plugin creates {prefix}igny8_sync_queue entry (status: pending)
4. → WP Cron job (igny8_process_sync_queue) processes queue:
a. Download images → wp_upload_dir(), wp_insert_attachment()
b. Map content_type → WP post_type via IGNY8_Content_Mapper
c. wp_insert_post() or wp_update_post()
d. Set post meta (_igny8_content_id, _igny8_cluster_id, _igny8_sync_status)
e. wp_set_object_terms() for taxonomy assignment
f. Status: 'review' (manual mode) or 'synced' (auto-publish mode)
5. → Plugin POSTs confirmation to IGNY8: POST /api/v1/integration/sync-events/
```
**Methods:**
- `push_content_to_queue($data)` — validate payload, insert into sync queue
- `process_sync_queue()` — WP Cron handler, batch process pending items
- `map_content_type($igny8_type)` — return WP post_type + template
- `create_post_from_content($data)` — `wp_insert_post()` + meta + terms
- `download_images($content)` — download to `wp_upload_dir()`, create attachments
- `schedule_publish($post_id, $date)` — set future publish date
- `get_review_queue()` — list items with status `review`
**Post Meta (Connected Mode):**
```
_igny8_content_id -- int: links WP post to IGNY8 Content record
_igny8_cluster_id -- int: links WP post to IGNY8 SAGCluster
_igny8_sync_status -- string: pending/synced/failed/modified
_igny8_last_sync -- string: ISO timestamp
_igny8_related_links -- JSON: [{post_id, anchor_text, priority}]
_igny8_link_suggestions -- JSON: AI-generated link suggestions from 02D
```
**Hooks:**
- Actions: `wp_scheduled_event` (`igny8_process_sync_queue`), `rest_api_init`
- Filters: `igny8_content_before_sync`, `igny8_post_mapped`, `igny8_sync_status_updated`
### Module 12: SAG Structure (Blueprint Sync)
**Purpose:** Receive SAG blueprint from IGNY8, create WordPress taxonomies and terms matching SAG dimensions and attributes, map clusters to hub pages, and provide structure visualization.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `IGNY8_SAG_Module` | `modules/sag/class-sag-module.php` | Module entry |
| `IGNY8_Blueprint_Sync` | `modules/sag/class-blueprint-sync.php` | Receive + cache blueprint |
| `IGNY8_Taxonomy_Builder` | `modules/sag/class-taxonomy-builder.php` | `register_taxonomy()` from SAG attributes |
| `IGNY8_Term_Builder` | `modules/sag/class-term-builder.php` | `wp_insert_term()` with hierarchy |
| `IGNY8_Cluster_Manager` | `modules/sag/class-cluster-manager.php` | Map clusters → hub pages/terms |
| `IGNY8_Structure_Visualizer` | `modules/sag/class-structure-visualizer.php` | Visual site structure admin page |
**SAG Blueprint Sync Flow:**
```
1. IGNY8 confirms blueprint (01D setup wizard)
2. → POST /wp-json/igny8/v1/sag/sync-blueprint (validated via X-IGNY8-API-KEY)
Payload: {blueprint: {dimensions: [{id, name, slug, attributes: [{id, name, slug}]}],
clusters: [{id, name, dimension_ids, hub_page_id, recommended_content_count}]}}
3. → Plugin stores in option igny8_sag_blueprint
4. → Admin sees "Blueprint Review" notice
5. → User clicks "Approve Blueprint":
a. Loop dimensions → register_taxonomy() if not exists
b. Loop attribute values → wp_insert_term() with parent hierarchy
c. Set term meta (_igny8_term_sag_attribute, _igny8_term_sag_level)
d. Loop clusters → create/link hub pages
6. → POST confirmation back: {term_ids_map, hub_page_ids_map}
```
**Methods:**
- `sync_blueprint($json)` — validate, store in option, trigger admin notice
- `create_taxonomies_from_blueprint($bp)` — `register_taxonomy()` per dimension
- `create_terms_from_blueprint($bp)` — `wp_insert_term()` per attribute value
- `map_cluster_to_page($cluster_id, $page_id)` — link SAGCluster to WP hub page
- `get_blueprint()` — return cached blueprint from option
- `get_structure_overview()` — summary for structure visualizer admin page
**Term Meta (Connected Mode):**
```
_igny8_term_content -- string: rich HTML for term landing page
_igny8_term_faq -- JSON: [{question, answer}]
_igny8_term_related_terms -- JSON: [term_id, term_id, ...]
_igny8_term_sag_attribute -- string: attribute name from SAG
_igny8_term_sag_level -- string: primary/secondary/tertiary
_igny8_term_cluster_id -- int: linked SAGCluster ID
_igny8_term_igny8_id -- int: ID in IGNY8 platform
_igny8_term_seo_title -- string: SEO title for term archive
_igny8_term_meta_description -- string: meta description for term archive
_igny8_term_robots_index -- string: index/noindex
_igny8_term_og_image -- int: OG image attachment ID
```
---
## 3. DATA MODELS & APIS
### Database Table
The sync queue table is created by 03A's `class-installer.php` (table 6), used here:
**`{prefix}igny8_sync_queue`**
```sql
CREATE TABLE {prefix}igny8_sync_queue (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
igny8_content_id VARCHAR(100) NOT NULL,
content_type VARCHAR(50) NOT NULL,
sync_status VARCHAR(20) NOT NULL DEFAULT 'pending',
wp_post_id BIGINT NULL,
wp_term_id BIGINT NULL,
data LONGTEXT NULL,
created_at DATETIME NOT NULL,
synced_at DATETIME NULL,
error_message TEXT NULL,
INDEX idx_status (sync_status)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Options (Connected Mode)
| Option Key | Purpose |
|------------|---------|
| `igny8_api_key` | API key for IGNY8 SaaS (encrypted via `wp_encrypt()` or openssl) |
| `igny8_api_connected` | Boolean — is connected mode active |
| `igny8_api_url` | Base URL (default: `https://api.igny8.com/api/v1/`) |
| `igny8_site_id` | IGNY8 Site record ID (integer) |
| `igny8_last_sync` | ISO timestamp of last successful sync |
| `igny8_sag_blueprint` | JSON: full blueprint data from IGNY8 |
### API Client (`class-api-client.php`)
Outbound requests from WordPress to IGNY8 SaaS API:
| Property | Value |
|----------|-------|
| Base URL | From `igny8_api_url` option (default: `https://api.igny8.com/api/v1/`) |
| Auth Header | `Authorization: Bearer {igny8_api_key}` |
| Methods | `get($endpoint, $params)`, `post($endpoint, $data)`, `put($endpoint, $data)`, `delete($endpoint)` |
| Error Handling | `WP_Error` wrapping, log failures via `error_log()` |
| Rate Limiting | Respect 429 responses with exponential backoff (1s → 2s → 4s → 8s) |
| Connection Test | `GET /api/v1/system/status/` |
**IGNY8 Backend Endpoints Consumed by Plugin:**
```
GET /api/v1/writer/content/?site_id=X&status=approved -- content ready for sync
GET /api/v1/writer/content/{id}/ -- single content with HTML, images, taxonomy data
GET /api/v1/planner/clusters/?site_id=X -- cluster data
GET /api/v1/sag/blueprints/?site_id=X -- SAG blueprint (Phase 1)
POST /api/v1/integration/sync-events/ -- log sync results
POST /api/v1/publishing/records/ -- log publishing
```
### REST Endpoints (Inbound from IGNY8)
All endpoints validate `X-IGNY8-API-KEY` header. Under `/wp-json/igny8/v1/`:
| Method | Endpoint | Purpose | Payload |
|--------|----------|---------|---------|
| POST | `/sync/push` | Content push from IGNY8 | `{igny8_content_id, content_type, title, content, excerpt, featured_image, meta: {...}}` |
| POST | `/sag/sync-blueprint` | Blueprint sync | `{blueprint: {dimensions: [...], clusters: [...]}}` |
| POST | `/sag/create-taxonomies` | Taxonomy creation | `{taxonomies: [{name, slug, hierarchical, post_types}]}` |
| POST | `/terms/{id}/content` | Term content push | `{term_id, content, faq, related_terms, meta: {...}}` |
| POST | `/schema/bulk-update` | Schema JSON-LD bulk push | `{schemas: [{post_id, schema_type, schema_data}]}` |
| POST | `/gsc/status-sync` | GSC index statuses | `{statuses: [{url, index_status, coverage_state, last_checked}]}` |
| POST | `/event` | Webhook events | `{event_type, data: {...}}` |
### Endpoint Payload Examples
**POST `/sync/push`:**
```json
{
"igny8_content_id": 12345,
"content_type": "post",
"title": "Content Title",
"content": "<p>HTML content...</p>",
"excerpt": "Short excerpt",
"featured_image": "https://cdn.igny8.com/image.jpg",
"meta": {
"focus_keyword": "main keyword",
"secondary_keywords": ["kw2", "kw3"],
"cluster_id": 123,
"taxonomies": {
"service_category": ["emergency-plumbing"],
"cluster": ["drain-services"]
}
}
}
```
**POST `/sag/sync-blueprint`:**
```json
{
"blueprint": {
"dimensions": [
{
"id": 1,
"name": "Service Type",
"slug": "service_type",
"attributes": [
{"id": 1, "name": "Emergency Plumbing", "slug": "emergency-plumbing"}
]
}
],
"clusters": [
{
"id": 1,
"name": "Emergency Drain Cleaning",
"dimension_ids": [1, 2],
"hub_page_id": null,
"recommended_content_count": 12
}
]
}
}
```
**POST `/terms/{id}/content`:**
```json
{
"term_id": 1001,
"content": "<p>HTML content for term landing page</p>",
"faq": [{"question": "Q1?", "answer": "A1"}],
"related_terms": [1002, 1003],
"meta": {
"seo_title": "Custom title",
"meta_description": "Custom description"
}
}
```
**POST `/schema/bulk-update`:**
```json
{
"schemas": [
{
"post_id": 123,
"schema_type": "Article",
"schema_data": {"@context": "https://schema.org", "@type": "Article"}
}
]
}
```
**POST `/gsc/status-sync`:**
```json
{
"statuses": [
{
"url": "https://example.com/post",
"index_status": "Indexed",
"coverage_state": "Submitted and indexed",
"last_checked": "2026-03-22T12:00:00Z"
}
]
}
```
---
## 4. IMPLEMENTATION STEPS
### Admin UI Additions (Connected Mode)
When connected, two new submenus appear:
```
IGNY8 (top-level)
├── ... (all standalone modules from 03A)
├── Content Sync — Sync queue, review queue, settings, sync log
└── SAG Structure — Blueprint review, cluster manager, structure visualizer
```
### Build Sequence
**Phase 8: Content Sync Module (Days 21-24)**
1. Build `IGNY8_Sync_Module` — module entry, register REST endpoints
2. Build `IGNY8_Content_Mapper` — content_type → post_type mapping table
3. Build `IGNY8_Image_Downloader` — download remote images, create WP attachments
4. Build `IGNY8_Sync_Queue` — queue management, status transitions
5. Build `IGNY8_Content_Puller` — fetch content from IGNY8 API
6. Build `IGNY8_Publish_Scheduler` — review queue UI, scheduled publishing
7. Register WP Cron event: `igny8_process_sync_queue` (every 5 minutes)
**Phase 9: SAG Structure Module (Days 25-27)**
1. Build `IGNY8_SAG_Module` — module entry, register REST endpoints
2. Build `IGNY8_Blueprint_Sync` — receive, validate, store blueprint
3. Build `IGNY8_Taxonomy_Builder` — `register_taxonomy()` from blueprint dimensions
4. Build `IGNY8_Term_Builder` — `wp_insert_term()` from attribute values
5. Build `IGNY8_Cluster_Manager` — map clusters → hub pages
6. Build `IGNY8_Structure_Visualizer` — admin page with visual site structure
7. Build `class-api-client.php` — outbound IGNY8 API client with rate limiting
### Security Considerations
1. **API Key Validation**: Every inbound REST endpoint validates `X-IGNY8-API-KEY` header against stored `igny8_api_key` option using `hash_equals()` for timing-safe comparison
2. **Payload Sanitization**: All incoming content HTML sanitized via `wp_kses_post()`. All text fields via `sanitize_text_field()`. All URLs via `sanitize_url()`. All integers via `absint()`
3. **API Key Storage**: Encrypted at rest. Never exposed in admin UI after initial entry. Use `wp_options` with obfuscated display (`****...last4`)
4. **Rate Limiting**: API client respects 429 responses. Exponential backoff: 1s → 2s → 4s → 8s max
5. **Nonce Bypass for REST**: Inbound webhooks use API key auth, not nonce. WordPress REST permission callbacks enforce `X-IGNY8-API-KEY` validation
---
## 5. ACCEPTANCE CRITERIA
### Content Sync (Module 11)
- [ ] `POST /wp-json/igny8/v1/sync/push` receives content and creates sync queue entry
- [ ] WP Cron processes sync queue — creates/updates WordPress posts
- [ ] Content type mapping works for all 9 IGNY8 → WP type combinations
- [ ] Images downloaded from IGNY8 CDN, created as WP attachments, set as featured
- [ ] Post meta set: `_igny8_content_id`, `_igny8_cluster_id`, `_igny8_sync_status`
- [ ] Taxonomy terms assigned via `wp_set_object_terms()`
- [ ] Review mode: items land in review queue (status `review`) for manual approval
- [ ] Auto-publish mode: items publish directly (status `synced`)
- [ ] Confirmation callback POSTs to `POST /api/v1/integration/sync-events/`
- [ ] Sync queue admin page shows all items with status, errors, retry button
### SAG Structure (Module 12)
- [ ] `POST /wp-json/igny8/v1/sag/sync-blueprint` receives and stores blueprint
- [ ] Admin sees "Blueprint Review" notice after blueprint sync
- [ ] "Approve Blueprint" creates taxonomies from SAG dimensions
- [ ] Terms created from attribute values with correct hierarchy
- [ ] Term meta set: `_igny8_term_sag_attribute`, `_igny8_term_sag_level`, `_igny8_term_cluster_id`
- [ ] Clusters mapped to hub pages
- [ ] `POST /wp-json/igny8/v1/terms/{id}/content` pushes rich content to term meta
- [ ] `POST /wp-json/igny8/v1/schema/bulk-update` updates schema meta on posts
- [ ] `POST /wp-json/igny8/v1/gsc/status-sync` updates GSC status display
- [ ] Structure visualizer admin page renders site taxonomy/cluster hierarchy
### API Client
- [ ] Connection test via `GET /api/v1/system/status/` returns success
- [ ] All outbound requests include `Authorization: Bearer {api_key}` header
- [ ] 429 responses trigger exponential backoff (1s → 2s → 4s → 8s)
- [ ] Failed requests wrapped in `WP_Error` and logged
### Security
- [ ] All inbound REST endpoints validate `X-IGNY8-API-KEY` via `hash_equals()`
- [ ] Incoming HTML sanitized via `wp_kses_post()`
- [ ] API key stored encrypted, displayed as `****...last4` in admin
- [ ] No API key exposed in client-side JavaScript or HTML source
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
Before starting implementation:
1. Read 03A (standalone plugin) — understand module manager, admin menu, DB tables
2. Read existing WP Bridge v1.5.2 at `/data/app/igny8/plugins/wordpress/source/igny8-wp-bridge/`
3. Read 01A (SAG Data Foundation) for SAGBlueprint, SAGAttribute, SAGCluster models
4. Read 01D (Setup Wizard) for blueprint generation flow that triggers sync
5. Read 02B (Taxonomy Term Content) for term content that pushes via `/terms/{id}/content`
6. Read 01G (SAG Health Monitoring) for health scoring context
### Execution Order
```
Content Sync module (6 files) → SAG Structure module (6 files) → API Client → Admin UI views
```
### Critical Rules
1. **All IGNY8 IDs are integers** — `igny8_content_id`, `cluster_id`, `term_id` are all integer (BigAutoField), never UUIDs
2. **IGNY8 model names are PLURAL** — Clusters, Keywords, Tasks, ContentIdeas, Images, Content (stays singular)
3. **Connected modules only load when `igny8_api_connected` is `true`**
4. **REST security** — every inbound endpoint validates `X-IGNY8-API-KEY` with `hash_equals()`
5. **No blocking** — all API calls and sync processing via WP Cron, never on page load
6. **Image downloads** — use `wp_remote_get()` for fetching, `wp_handle_sideload()` for attachment creation
7. **Taxonomy registration** — use `register_taxonomy()` with proper `show_in_rest => true` for block editor support
8. **Term creation** — use `wp_insert_term()`, not direct DB insert
### File Creation Order
```
1. includes/modules/content-sync/class-sync-module.php
2. includes/modules/content-sync/class-content-mapper.php
3. includes/modules/content-sync/class-image-downloader.php
4. includes/modules/content-sync/class-sync-queue.php
5. includes/modules/content-sync/class-content-puller.php
6. includes/modules/content-sync/class-publish-scheduler.php
7. includes/modules/sag/class-sag-module.php
8. includes/modules/sag/class-blueprint-sync.php
9. includes/modules/sag/class-taxonomy-builder.php
10. includes/modules/sag/class-term-builder.php
11. includes/modules/sag/class-cluster-manager.php
12. includes/modules/sag/class-structure-visualizer.php
13. includes/class-api-client.php
14. includes/admin/views/sync-queue.php
15. includes/admin/views/blueprint-review.php
16. includes/admin/views/structure-visualizer.php
```
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 03A WP Plugin Standalone | Base plugin — module manager, DB tables, admin menu, `igny8()` singleton |
| 01A SAG Data Foundation | SAGBlueprint, SAGAttribute, SAGCluster models — data source for Module 12 |
| 01B SAG Blueprint Engine | Blueprint generation that produces data synced to WordPress |
| 01D Setup Wizard | Blueprint generation triggers `POST /sag/sync-blueprint` to WordPress |
| 01G SAG Health Monitoring | Health scoring data available in connected mode intelligence module |
| 02B Taxonomy Term Content | Generates term content pushed via `POST /terms/{id}/content` |
| 02C GSC Integration | GSC metrics pushed via `POST /gsc/status-sync` |
| 02D Linker Internal | SAGLink data feeds connected mode auto-linker in Module 6 |
| 02G Rich Schema SERP | Schema data pushed via `POST /schema/bulk-update` |
| 02H Socializer | SocialAccount OAuth feeds connected mode auto-poster in Module 8 |
| 03C Companion Theme | Theme consumes blueprint data, term meta, cluster navigation |

633
v2/V2-Execution-Docs/03C-companion-theme.md Normal file
View File

@@ -0,0 +1,633 @@
# IGNY8 Phase 3: Companion Theme (03C)
## FSE Block Theme Optimized for IGNY8 + SAG Architecture
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 3 — WordPress Ecosystem
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, WordPress Developers, Theme Developers, Architects
---
## 1. CURRENT STATE
### No Companion Theme Exists
- Users run generic themes (Astra, GeneratePress, Kadence, etc.) alongside the IGNY8 plugin
- Plugin provides data (SEO meta, schema, internal links) but cannot control template rendering
- SAG taxonomy structures (service_category, service_area, clusters) have no optimized archive templates
- Term landing pages render as basic post lists — no rich content, FAQ, or cross-linking
- Cluster hub pages have no dedicated template with navigation and content grid
### Plugin API Available (03A + 03B)
When the IGNY8 plugin is active, the theme can access:
- `igny8()->seo->get_title()`, `igny8()->seo->get_breadcrumbs()` — SEO data
- `igny8()->schema->get_json_ld($post_id)` — schema markup
- `igny8()->linking->get_link_suggestions($post_id)` — internal link suggestions
- `igny8()->linking->get_cluster_navigation()` — cluster navigation data
- Term meta: `_igny8_term_content`, `_igny8_term_faq`, `_igny8_term_related_terms`
- Blueprint data: `get_option('igny8_sag_blueprint')` — full SAG structure
### Graceful Degradation
The theme checks `function_exists('igny8')` before calling any plugin API:
- **Without plugin:** Theme breadcrumbs, basic related content by category, no SAG features
- **With plugin standalone:** SEO breadcrumbs, schema, link suggestions, share buttons
- **With plugin connected:** Full SAG term pages, cluster navigation, blueprint visualization, rich term content
---
## 2. WHAT TO BUILD
### Overview
A Full Site Editing (FSE) block theme built for WordPress 5.9+ that provides:
- 7 custom post types (CPTs) optimized for service businesses, portfolios, and SaaS
- 9 custom taxonomies including SAG-mapped service categories and clusters
- Rich term landing page templates with 7-section layout
- Cluster hub template with dimensional navigation
- 15 landing page section types with 8 presets
- 50+ block patterns across 8 categories
- 13 custom Gutenberg blocks
- 5 site-type starter templates with demo content
- 6 interlinking display components
- WooCommerce integration when active
### Design System (theme.json)
- **Colors:** Primary palette (primary, light, dark, accent), neutrals (background, surface, text, text-secondary, border), semantic (success, warning, error, info)
- **Typography:** Heading + body font pairings from curated list. Sizes: xs, sm, base, lg, xl, 2xl, 3xl, 4xl
- **Spacing:** xs (0.25rem), sm (0.5rem), base (1rem), lg (1.5rem), xl (2rem), 2xl (3rem), 3xl (4rem)
- **Border Radius:** none, sm, base, lg, full
- **CSS:** BEM naming `.tn-block__element--modifier`, <35KB gzipped total
- **JS:** ES6+, vanilla (no jQuery on frontend), <15KB gzipped
- **Performance:** FCP <1.0s, LCP <1.5s, TBT <50ms, CLS <0.05, PageSpeed 95+ mobile
### 7 Custom Post Types
**CPT 1: `service`**
- Archive enabled, supports: title, editor, thumbnail, excerpt, revisions, page-attributes
- Taxonomies: `service_category`, `service_area`, `service_attribute`, `cluster`
- Rewrite: `/services/%service_category%/%postname%/`
- Templates: `single-service.html`, `archive-service.html`
- 10 meta fields:
| Meta Key | Type | Purpose |
|----------|------|---------|
| `_theme_service_price_range` | string | Price display ("$150 - $500") |
| `_theme_service_duration` | string | Time estimate ("2-4 hours") |
| `_theme_service_process_steps` | JSON | `[{step_number, title, desc, icon}]` |
| `_theme_service_outcomes` | JSON | `[{title, description}]` |
| `_theme_service_faqs` | JSON | `[{question, answer}]` |
| `_theme_service_cta_text` | string | CTA button text |
| `_theme_service_cta_url` | string | CTA button URL |
| `_theme_service_areas_served` | JSON | Geographic areas array |
| `_theme_service_gallery` | JSON | Attachment IDs array |
| `_theme_service_related_services` | JSON | Related service post IDs |
**CPT 2: `landing_page`**
- NO archive, does NOT support editor (section meta box instead)
- Rewrite: `/l/%postname%/`
- Template: `single-landing_page.html`
- Section-based rendering from `_theme_landing_sections` repeater meta (15 section types, see below)
**CPT 3: `portfolio`**
- Archive enabled, supports: title, editor, thumbnail, excerpt, revisions
- Taxonomies: `portfolio_category`, `portfolio_tag`, `service_category` (shared)
- Rewrite: `/portfolio/%portfolio_category%/%postname%/`
- Templates: `single-portfolio.html`, `archive-portfolio.html`
- 8 meta fields:
| Meta Key | Type | Purpose |
|----------|------|---------|
| `_theme_portfolio_client` | string | Client name |
| `_theme_portfolio_date` | string | YYYY-MM-DD |
| `_theme_portfolio_url` | string | Live project URL |
| `_theme_portfolio_results` | JSON | `[{metric, value, description}]` |
| `_theme_portfolio_technologies` | string | Comma-separated tech list |
| `_theme_portfolio_testimonial_id` | int | Linked testimonial post ID |
| `_theme_portfolio_gallery` | JSON | Attachment IDs array |
| `_theme_portfolio_before_after` | JSON | `{before_img, after_img}` |
**CPT 4: `team_member`**
- Archive enabled (team page), supports: title, editor, thumbnail
- Rewrite: `/team/%postname%/`
- Templates: `single-team-member.html`, `archive-team-member.html`
- 6 meta fields: `_theme_team_position`, `_theme_team_email`, `_theme_team_phone`, `_theme_team_social` (JSON: `{linkedin, twitter, facebook, website}`), `_theme_team_order` (int), `_theme_team_department`
**CPT 5: `testimonial`**
- Public: false (displayed via blocks/shortcodes only, no individual pages)
- Supports: title, editor, thumbnail
- 8 meta fields: `_theme_testimonial_author_name`, `_theme_testimonial_author_title`, `_theme_testimonial_company`, `_theme_testimonial_rating` (1-5 int), `_theme_testimonial_image` (attachment ID), `_theme_testimonial_featured` (bool), `_theme_testimonial_service_id` (linked service), `_theme_testimonial_product_id`
**CPT 6: `faq`**
- Archive enabled, supports: title, editor (title=question, editor=answer)
- Taxonomies: `faq_category`, `cluster`
- Rewrite: `/faq/%faq_category%/%postname%/`
- Templates: `single-faq.html`, `archive-faq.html`
- 2 meta fields: `_theme_faq_order` (int), `_theme_faq_schema_enabled` (bool)
**CPT 7: `documentation`**
- Archive enabled, hierarchical (parent/child docs)
- Supports: title, editor, thumbnail, page-attributes
- Taxonomies: `doc_category`
- Rewrite: `/docs/%doc_category%/%postname%/`
- Templates: `single-documentation.html`, `archive-documentation.html`
- 4 meta fields: `_theme_doc_sidebar_enabled` (bool), `_theme_doc_toc_enabled` (bool), `_theme_doc_last_reviewed` (date), `_theme_doc_related_docs` (JSON: post IDs)
> **Note:** WooCommerce taxonomies (`product_cat`, `product_tag`, `pa_*`) are NOT registered by theme. Theme provides rich templates for them. Theme registers `cluster` for `product` post type.
### 9 Custom Taxonomies
| # | Slug | For CPTs | Hierarchical | SAG Mapping | Rewrite |
|---|------|----------|-------------|-------------|---------|
| 1 | `service_category` | service, portfolio | Yes | Primary attribute | `/services/{slug}/` |
| 2 | `service_area` | service | Yes | Tertiary (geographic) | `/area/{slug}/` |
| 3 | `service_attribute` | service | No | Secondary attributes | `/attribute/{slug}/` |
| 4 | `portfolio_category` | portfolio | Yes | — | `/portfolio/{slug}/` |
| 5 | `portfolio_tag` | portfolio | No | — | `/tag/{slug}/` |
| 6 | `faq_category` | faq | Yes | Cluster assignment | `/faq/{slug}/` |
| 7 | `doc_category` | documentation | Yes | — | `/docs/{slug}/` |
| 8 | `cluster` | post, page, service, faq, product | No | Core cluster assignment | `/topic/{slug}/` |
| 9 | `topic_tag` | post, service, faq, documentation | No | Granular tagging | `/topic/{slug}/` |
### Term Landing Page Templates (7-Section Structure)
Every taxonomy term archive renders as a rich landing page:
| # | Section | Content |
|---|---------|---------|
| 1 | **Term Hero** | H1 (term name), description (from `_igny8_term_content` if active), breadcrumbs, post count |
| 2 | **Key Subtopics** | Grid of child term cards (if hierarchical + children exist) |
| 3 | **Content Grid** | Posts/services/products in term, filterable, paginated, layout options (grid 2/3/4 col, list) |
| 4 | **Related Terms** | Sibling terms or `_igny8_term_related_terms` from connected mode |
| 5 | **Cluster Cross-Links** | Related clusters sharing dimensional axes (if IGNY8 connected) |
| 6 | **FAQ Section** | Accordion from `_igny8_term_faq` or matching `faq_category` posts |
| 7 | **CTA** | Configurable per term or global default |
### Cluster Hub Template (`taxonomy-cluster.html`)
| # | Section | Content |
|---|---------|---------|
| 1 | **Cluster Hero** | H1, description, dimensional breadcrumb, content count |
| 2 | **Hub Content** | Pillar article from `_igny8_term_content` |
| 3 | **Supporting Content Grid** | Posts/services/products grouped by type |
| 4 | **Attribute Navigation** | Browse intersecting attributes via SAG data |
| 5 | **Related Clusters** | Cross-linking via SAG dimensional relationships |
| 6 | **FAQ** | Cluster-specific FAQ from `_igny8_term_faq` |
| 7 | **CTA** | Configurable CTA block |
### 6 Interlinking Display Components
| # | Template Part | Source | Fallback |
|---|--------------|--------|----------|
| 1 | `parts/related-content.html` | `_igny8_related_links` post meta | Category match, 3-4 cards |
| 2 | `parts/cluster-navigation.html` | `igny8()->linking->get_cluster_navigation()` | Same cluster term posts |
| 3 | `parts/attribute-browse.html` | Term archive sidebar pills/tags | All terms for taxonomy |
| 4 | `parts/breadcrumbs.html` | `igny8()->seo->get_breadcrumbs()` | Theme breadcrumb fallback |
| 5 | `parts/term-quick-links.html` | All terms assigned to post grouped by taxonomy | Standard tag list |
| 6 | `parts/child-term-grid.html` | Child terms with image/name/description/post count | `get_term_children()` |
### Landing Page System (15 Section Types)
| # | Type | Fields |
|---|------|--------|
| 1 | `hero` | headline, subheadline, CTA, background image/video/gradient |
| 2 | `features-grid` | columns 2/3/4, items: icon/title/description |
| 3 | `how-it-works` | numbered steps: title/description/icon |
| 4 | `social-proof` | testimonials: grid/slider/single layout |
| 5 | `pricing` | billing toggle monthly/annual, plans: features/price/CTA |
| 6 | `faq` | Q&A items, `schema_enabled` option for FAQPage JSON-LD |
| 7 | `cta-band` | headline, subheadline, CTA, background colors |
| 8 | `content-block` | title, WYSIWYG editor, image left/right, background |
| 9 | `stats-counter` | stats: number/suffix/label, animated counters |
| 10 | `team` | From `team_member` CPT or manual entries |
| 11 | `portfolio-grid` | From `portfolio` CPT |
| 12 | `contact-form` | Form shortcode, optional map, address block |
| 13 | `video-embed` | URL, poster image, autoplay, full-width options |
| 14 | `comparison-table` | columns, rows, highlight column option |
| 15 | `logo-bar` | Logo images, grid or horizontal scroll |
Section template files in `inc/landing-pages/sections/`: one PHP file per section type.
### 8 Landing Page Presets
| # | Preset | Section Order |
|---|--------|---------------|
| 1 | SaaS Product | hero → logo-bar → features → how-it-works → social-proof → pricing → faq → cta |
| 2 | Service Business | hero → features → content → stats → social-proof → team → contact-form → cta |
| 3 | Product Launch | hero → content → features → video → social-proof → pricing → faq → cta |
| 4 | Lead Generation | hero → features → social-proof → cta |
| 5 | Portfolio / Agency | hero → portfolio → how-it-works → social-proof → stats → contact-form |
| 6 | Event / Webinar | hero → content → features → social-proof → faq → cta |
| 7 | Cluster Hub | hero → content → features → faq → cta |
| 8 | Term Landing | hero → content → features → social-proof → faq → cta |
### 50+ Block Patterns (by Category)
| Category | Patterns |
|----------|----------|
| Heroes (4) | hero-centered, hero-split, hero-video-bg, hero-slider |
| Content (4) | features-3col, features-alternating, how-it-works, benefits-icons |
| Social Proof (4) | testimonials-grid, testimonials-slider, logo-bar, stats-row |
| Conversion (4) | cta-simple, cta-with-form, pricing-section, newsletter-signup |
| Services (3) | service-cards, service-list, service-with-sidebar |
| Portfolio (3) | portfolio-grid, portfolio-masonry, case-study-layout |
| Footers (3) | footer-4col, footer-centered, footer-minimal |
| Custom (13+) | pricing-table, feature-grid, testimonial-slider, stats-counter, team-grid, faq-accordion, cta-banner, logo-carousel, comparison-table, icon-box, service-grid, portfolio-showcase, doc-sidebar |
### 13 Custom Gutenberg Blocks
Registered in `inc/blocks/`, each with `block.json`, `edit.js`, `save.js`, `style.css`:
1. `pricing-table` — configurable pricing plans with toggle
2. `feature-grid` — icon + title + description grid
3. `testimonial-slider` — carousel from `testimonial` CPT
4. `stats-counter` — animated number counters
5. `team-grid` — from `team_member` CPT
6. `faq-accordion` — collapsible FAQ with optional FAQPage schema
7. `cta-banner` — full-width call-to-action band
8. `logo-carousel` — partner/client logo showcase
9. `comparison-table` — feature comparison matrix
10. `icon-box` — icon + heading + text block
11. `service-grid` — from `service` CPT (dynamic)
12. `portfolio-showcase` — from `portfolio` CPT (dynamic)
13. `doc-sidebar` — documentation navigation tree
### 5 Site-Type Starter Templates
Each includes full template files, sample CPT entries, sample taxonomy terms, demo images:
| # | Starter | Key Templates |
|---|---------|---------------|
| 1 | Blog | front-page, single.html, archive.html |
| 2 | SaaS | front-page, page-features.html, page-pricing.html, page-docs.html |
| 3 | Corporate | front-page, page-about.html, page-services.html, page-contact.html |
| 4 | eCommerce | front-page, archive-product.html, single-product.html |
| 5 | Portfolio | front-page, archive-portfolio.html, single-portfolio.html |
### WooCommerce Integration (when active)
- Register `cluster` taxonomy for `product` post type
- Rich category pages following term landing structure
- Product attribute support in templates
- Integration with IGNY8 schema (Product schema type from 02G)
- Template overrides: `archive-product.php`, `single-product.php`, `taxonomy-product_cat.php`, `content-product.php`
---
## 3. DATA MODELS & APIS
### No Custom Database Tables
The theme uses only WordPress native storage:
- **Post meta** — CPT meta fields (all `_theme_` prefixed)
- **Term meta** — Plugin term meta (all `_igny8_term_` prefixed, read-only from theme)
- **Options** — Theme settings via Customizer/theme.json
### Post Meta Summary (by CPT)
| CPT | Total Meta Fields | Prefix |
|-----|-------------------|--------|
| `service` | 10 | `_theme_service_` |
| `landing_page` | 1 (`_theme_landing_sections` repeater) | `_theme_landing_` |
| `portfolio` | 8 | `_theme_portfolio_` |
| `team_member` | 6 | `_theme_team_` |
| `testimonial` | 8 | `_theme_testimonial_` |
| `faq` | 2 | `_theme_faq_` |
| `documentation` | 4 | `_theme_doc_` |
### Plugin API Consumed (read-only)
| API Call | Purpose | Fallback Without Plugin |
|----------|---------|------------------------|
| `igny8()->seo->get_title()` | SEO title | `wp_title()` |
| `igny8()->seo->get_breadcrumbs()` | Breadcrumb HTML | Theme breadcrumb in `inc/breadcrumbs.php` |
| `igny8()->schema->get_json_ld($post_id)` | JSON-LD output | None (omitted) |
| `igny8()->linking->get_link_suggestions($post_id)` | Related content | Category-based `WP_Query` |
| `igny8()->linking->get_cluster_navigation()` | Cluster nav | Same-taxonomy term list |
| `get_term_meta($id, '_igny8_term_content')` | Term landing content | Term description |
| `get_term_meta($id, '_igny8_term_faq')` | Term FAQ data | `faq_category` posts query |
| `get_term_meta($id, '_igny8_term_related_terms')` | Related terms | Sibling terms |
| `get_option('igny8_sag_blueprint')` | SAG structure | Hidden (no SAG features) |
---
## 4. IMPLEMENTATION STEPS
### Theme Architecture
```
igny8-theme/
├── style.css # Theme header (required)
├── theme.json # Design system tokens, settings, styles
├── functions.php # Bootstrap, includes loader
├── screenshot.png # Theme screenshot (1200x900)
├── assets/
│ ├── css/ # Compiled CSS (BEM, <35KB gzipped)
│ ├── js/ # Vanilla ES6+ JS (<15KB gzipped)
│ ├── fonts/ # Self-hosted font files
│ └── images/ # Theme images, icons
├── inc/
│ ├── setup.php # Theme supports, menus, sidebars, image sizes
│ ├── cleanup.php # Remove WP bloat (emojis, embeds, etc.)
│ ├── enqueue.php # Conditional CSS/JS loading
│ ├── template-tags.php # Custom template tags
│ ├── breadcrumbs.php # Theme breadcrumb fallback (when plugin not active)
│ ├── walkers.php # Custom nav walkers
│ ├── customizer.php # Logo, colors, fonts, layout, social URLs, footer text
│ ├── theme-bridge.php # Communication layer with IGNY8 plugin
│ ├── cpt/
│ │ ├── register.php # All CPT registration
│ │ ├── service.php # service CPT config
│ │ ├── landing-page.php # landing_page CPT config
│ │ ├── portfolio.php # portfolio CPT config
│ │ ├── team-member.php # team_member CPT config
│ │ ├── testimonial.php # testimonial CPT config
│ │ ├── faq.php # faq CPT config
│ │ └── documentation.php # documentation CPT config
│ ├── taxonomies/
│ │ ├── register.php # All taxonomy registration
│ │ ├── service-category.php # service_category config
│ │ ├── service-area.php # service_area config
│ │ ├── service-attribute.php # service_attribute config
│ │ ├── portfolio-category.php # portfolio_category config
│ │ ├── portfolio-tag.php # portfolio_tag config
│ │ ├── faq-category.php # faq_category config
│ │ ├── doc-category.php # doc_category config
│ │ ├── cluster.php # cluster taxonomy config
│ │ └── topic-tag.php # topic_tag config
│ ├── meta-fields/
│ │ ├── class-meta-box-base.php # Base meta box class
│ │ ├── service-meta.php # service CPT meta handler
│ │ ├── landing-page-meta.php # landing_page section repeater
│ │ ├── portfolio-meta.php # portfolio CPT meta handler
│ │ ├── team-member-meta.php # team_member meta handler
│ │ ├── testimonial-meta.php # testimonial meta handler
│ │ ├── faq-meta.php # faq meta handler
│ │ └── documentation-meta.php # documentation meta handler
│ ├── landing-pages/
│ │ ├── register-cpt.php # Landing page CPT specifics
│ │ ├── meta-boxes.php # Section repeater meta box
│ │ ├── presets.php # 8 landing page presets
│ │ ├── render.php # Section rendering engine
│ │ └── sections/ # 15 section PHP templates
│ │ ├── hero.php
│ │ ├── features-grid.php
│ │ ├── how-it-works.php
│ │ ├── social-proof.php
│ │ ├── pricing.php
│ │ ├── faq.php
│ │ ├── cta-band.php
│ │ ├── content-block.php
│ │ ├── stats-counter.php
│ │ ├── team.php
│ │ ├── portfolio-grid.php
│ │ ├── contact-form.php
│ │ ├── video-embed.php
│ │ ├── comparison-table.php
│ │ └── logo-bar.php
│ ├── interlinking/
│ │ ├── related-content.php # After-post related content
│ │ ├── cluster-navigation.php # Cluster nav component
│ │ ├── attribute-browse.php # Attribute browsing sidebar
│ │ ├── breadcrumbs-component.php # SAG-aware breadcrumbs
│ │ ├── term-quick-links.php # Post sidebar term links
│ │ └── child-term-grid.php # Child term card grid
│ ├── blocks/ # 13 custom block registrations
│ │ ├── pricing-table/
│ │ ├── feature-grid/
│ │ ├── testimonial-slider/
│ │ ├── stats-counter/
│ │ ├── team-grid/
│ │ ├── faq-accordion/
│ │ ├── cta-banner/
│ │ ├── logo-carousel/
│ │ ├── comparison-table/
│ │ ├── icon-box/
│ │ ├── service-grid/
│ │ ├── portfolio-showcase/
│ │ └── doc-sidebar/
│ └── patterns/
│ ├── register.php # Pattern category + registration
│ ├── heroes/
│ ├── content/
│ ├── social-proof/
│ ├── conversion/
│ ├── services/
│ ├── portfolio/
│ └── footers/
├── templates/ # FSE templates (20+)
│ ├── index.html
│ ├── home.html
│ ├── single.html
│ ├── page.html
│ ├── archive.html
│ ├── single-service.html
│ ├── single-portfolio.html
│ ├── single-landing_page.html
│ ├── single-documentation.html
│ ├── single-faq.html
│ ├── single-team-member.html
│ ├── archive-service.html
│ ├── archive-portfolio.html
│ ├── archive-faq.html
│ ├── archive-documentation.html
│ ├── archive-team-member.html
│ ├── taxonomy-service_category.html
│ ├── taxonomy-service_area.html
│ ├── taxonomy-service_attribute.html
│ ├── taxonomy-cluster.html
│ ├── taxonomy-faq_category.html
│ ├── taxonomy-product_cat.html # WooCommerce
│ ├── taxonomy-product_tag.html # WooCommerce
│ ├── search.html
│ ├── 404.html
│ └── starters/ # 5 starter template sets
├── parts/ # FSE template parts (20+)
│ ├── header.html
│ ├── header-transparent.html
│ ├── header-minimal.html
│ ├── footer.html
│ ├── footer-minimal.html
│ ├── sidebar.html
│ ├── sidebar-docs.html
│ ├── post-meta.html
│ ├── author-box.html
│ ├── comments.html
│ ├── pagination.html
│ ├── related-content.html
│ ├── cluster-navigation.html
│ ├── attribute-browse.html
│ ├── breadcrumbs.html
│ ├── term-quick-links.html
│ ├── child-term-grid.html
│ ├── term-hero.html
│ ├── term-content.html
│ └── term-faq.html
└── woocommerce/ # WooCommerce template overrides
├── archive-product.php
├── single-product.php
├── taxonomy-product_cat.php
├── content-product.php
├── cart/
├── checkout/
└── myaccount/
```
### Build Sequence
**Phase 1: Foundation (Days 1-3)**
1. Create `style.css` header, `theme.json` design tokens, `functions.php` bootstrap
2. Build `inc/setup.php` — theme supports, menus, sidebars, image sizes
3. Build `inc/cleanup.php` — remove WP bloat (emojis, embed scripts, etc.)
4. Build `inc/enqueue.php` — conditional CSS/JS loading
5. Build `inc/theme-bridge.php` — plugin detection + API wrapper
6. Build `inc/customizer.php` — logo, colors, fonts, layout options
**Phase 2: CPTs + Taxonomies (Days 4-7)**
1. Register all 7 CPTs in `inc/cpt/` (7 config files + register.php)
2. Register all 9 taxonomies in `inc/taxonomies/` (9 config files + register.php)
3. Build `inc/meta-fields/class-meta-box-base.php` + 7 per-CPT meta handlers
4. Build basic FSE templates for all CPTs: single-*.html, archive-*.html
**Phase 3: Term Landing Pages + Cluster Hub (Days 8-10)**
1. Build 7-section term landing page template structure
2. Build `taxonomy-cluster.html` hub template
3. Build all taxonomy-*.html templates
4. Build 6 interlinking components in `inc/interlinking/`
5. Build template parts: term-hero, term-content, term-faq, child-term-grid, etc.
**Phase 4: Landing Page System (Days 11-14)**
1. Build section repeater meta box for `landing_page` CPT
2. Build 15 section PHP templates in `inc/landing-pages/sections/`
3. Build section rendering engine (`inc/landing-pages/render.php`)
4. Build 8 landing page presets (`inc/landing-pages/presets.php`)
**Phase 5: Blocks + Patterns (Days 15-19)**
1. Register 13 custom Gutenberg blocks with `block.json` + edit/save JS
2. Register 50+ block patterns across 8 categories
3. Build `inc/breadcrumbs.php` fallback
4. Build `inc/walkers.php` for nav menus
5. Build `inc/template-tags.php` for template helper functions
**Phase 6: Starters + WooCommerce (Days 20-23)**
1. Build 5 site-type starter template sets with demo content
2. Build WooCommerce template overrides (archive-product, single-product, taxonomy-product_cat)
3. Register `cluster` taxonomy for `product` post type when WooCommerce active
**Phase 7: Polish (Days 24-26)**
1. CSS optimization — ensure <35KB gzipped total
2. JS optimization — ensure <15KB gzipped, vanilla ES6+
3. Performance audit — meet FCP <1.0s, LCP <1.5s, TBT <50ms targets
4. Accessibility review — ARIA labels, keyboard nav, focus states
5. i18n pass — all strings via `igny8-theme` text domain
6. Screenshot + readme
---
## 5. ACCEPTANCE CRITERIA
### CPTs & Taxonomies
- [ ] All 7 CPTs registered with correct supports, rewrite rules, and templates
- [ ] All 9 taxonomies registered with correct hierarchy, post type assignments, and rewrites
- [ ] All CPT meta boxes render and save correctly
- [ ] `service` CPT displays with all 10 meta fields in single-service template
- [ ] `landing_page` CPT renders sections from repeater meta
- [ ] `portfolio` CPT shows results, gallery, before/after
- [ ] `faq` CPT renders as FAQ with optional schema
### Term Landing Pages
- [ ] Every taxonomy term archive renders 7-section landing page
- [ ] Term hero shows name, description (from `_igny8_term_content` or fallback), post count
- [ ] Child term grid, content grid, related terms, FAQ accordion all render
- [ ] Cluster cross-links appear when IGNY8 plugin connected
- [ ] Graceful fallback when plugin is not active
### Cluster Hub
- [ ] `taxonomy-cluster.html` renders hub with pillar content, supporting grid, attribute nav
- [ ] Dimensional navigation works (browse intersecting attributes)
- [ ] Related clusters cross-link correctly
### Landing Pages
- [ ] All 15 section types render correctly
- [ ] All 8 presets pre-populate correct section order
- [ ] Section repeater meta box allows add/remove/reorder sections
- [ ] FAQ section generates FAQPage schema when `schema_enabled` is true
### Blocks & Patterns
- [ ] All 13 custom blocks registered and functional in block editor
- [ ] 50+ patterns available in pattern inserter, categorized correctly
- [ ] Dynamic blocks (service-grid, portfolio-showcase, doc-sidebar) query correct CPTs
### Interlinking Components
- [ ] Related content shows `_igny8_related_links` data or category fallback
- [ ] Cluster navigation renders when plugin provides data
- [ ] Breadcrumbs use plugin or theme fallback
- [ ] All 6 template parts render without errors
### Starters & WooCommerce
- [ ] All 5 starter templates import with demo content
- [ ] WooCommerce product templates render with cluster taxonomy
- [ ] Product category archives follow term landing structure
### Performance & Quality
- [ ] CSS <35KB gzipped, JS <15KB gzipped
- [ ] PageSpeed Insights 95+ mobile
- [ ] FCP <1.0s, LCP <1.5s, TBT <50ms, CLS <0.05
- [ ] All strings translatable via `igny8-theme` text domain
- [ ] Keyboard navigation works on all interactive elements
- [ ] ARIA labels on all custom blocks and components
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
Before starting implementation:
1. Read 03A (plugin standalone) — understand `igny8()` public API, module methods
2. Read 03B (plugin connected) — understand term meta keys, blueprint data structure
3. Read 01A (SAG Data Foundation) — understand SAGCluster, SAGAttribute relationships
4. Read 01B (SAG Blueprint Engine) — understand dimension/attribute hierarchy
### Execution Order
```
Foundation → CPTs+Taxonomies → Term Landing+Cluster Hub → Landing Page System → Blocks+Patterns → Starters+WooCommerce → Polish
```
### Critical Rules
1. **FSE block theme** — use `templates/` and `parts/` directories with HTML files, not classic PHP templates
2. **theme.json** — all design tokens defined here, not in CSS variables directly
3. **No jQuery on frontend** — vanilla ES6+ only, jQuery allowed in admin only
4. **Conditional plugin features** — always check `function_exists('igny8')` before plugin API calls
5. **All IDs are integers** — IGNY8 content IDs, cluster IDs, term IDs are all integers (BigAutoField)
6. **Meta prefix** — theme meta uses `_theme_` prefix, plugin meta uses `_igny8_` prefix (read-only from theme)
7. **Text domain** — `igny8-theme` (different from plugin's `igny8` domain)
8. **Sanitize/escape** — all meta saves via `sanitize_text_field()` / `wp_kses_post()`, all output via `esc_html()` / `esc_attr()` / `esc_url()`
9. **Performance first** — aim for zero JS on pages that don't need it, conditional CSS loading
### File Creation Order
```
1. style.css + theme.json + functions.php (foundation)
2. inc/setup.php + inc/cleanup.php + inc/enqueue.php
3. inc/theme-bridge.php + inc/customizer.php
4. inc/cpt/register.php + 7 CPT files
5. inc/taxonomies/register.php + 9 taxonomy files
6. inc/meta-fields/class-meta-box-base.php + 7 meta handlers
7. templates/ (20+ FSE template files)
8. parts/ (20+ template part files)
9. inc/interlinking/ (6 component files)
10. inc/landing-pages/ (15 section templates + render engine + presets)
11. inc/blocks/ (13 block directories)
12. inc/patterns/ (50+ patterns)
13. woocommerce/ (template overrides)
14. templates/starters/ (5 starter sets)
15. inc/breadcrumbs.php + inc/walkers.php + inc/template-tags.php
```
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 03A WP Plugin Standalone | Plugin API: `igny8()->seo`, `igny8()->schema`, `igny8()->linking` consumed by theme |
| 03B WP Plugin Connected | Connected mode provides: term meta, blueprint data, cluster navigation |
| 01A SAG Data Foundation | SAGCluster → `cluster` taxonomy, SAGAttribute → `service_category`/`service_attribute`/`service_area` |
| 01B SAG Blueprint Engine | Blueprint dimensions → theme taxonomies, blueprint clusters → hub pages |
| 02D Linker Internal | Link suggestions feed `parts/related-content.html` via plugin API |
| 02G Rich Schema SERP | Schema types feed Module 2 which theme triggers via `igny8()->schema` |
| 03D Toolkit Plugin | Toolkit complements theme (forms, performance, security) |

467
v2/V2-Execution-Docs/03D-toolkit-plugin.md Normal file
View File

@@ -0,0 +1,467 @@
# IGNY8 Phase 3: Toolkit Plugin (03D)
## Utility Plugin — Performance, Forms, Security, SMTP, WooCommerce
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 3 — WordPress Ecosystem
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, WordPress Developers, Architects
---
## 1. CURRENT STATE
### No Toolkit Exists
- Users rely on multiple plugins for common utilities: WP Super Cache, Contact Form 7, Wordfence, WP Mail SMTP, etc.
- Each additional plugin adds overhead, potential conflicts, and security surface area
- No IGNY8-branded utility layer exists
### Relationship to IGNY8 Ecosystem
- **Independent** — toolkit works without the main IGNY8 plugin or companion theme
- **Complementary** — designed to pair with the companion theme (03C) for a complete site stack
- **Defers** — if both toolkit and main IGNY8 plugin are installed, toolkit's SMTP module defers to IGNY8's SMTP (03A Module 9b) to avoid conflict
- **Same module pattern** — uses identical `register()`, `activate()`, `deactivate()`, `is_active()`, `boot()` module architecture as 03A
### Plugin Identity
- **Plugin Name:** IGNY8 Toolkit
- **Text Domain:** `igny8-toolkit`
- **Option Prefix:** `igny8_toolkit_`
- **Table Prefix:** `{wp_prefix}igny8_toolkit_`
- **Min Requirements:** WordPress 5.9+, PHP 7.4+
- **License:** GPL v2+
- **Freemium Model:** Core features free, advanced features premium (detailed per module below)
---
## 2. WHAT TO BUILD
### Overview
A separate utility plugin with 5 infrastructure modules. Each module is independently toggleable and has zero performance impact when disabled. Follows the same module manager pattern as the main IGNY8 plugin (03A).
### Module 1: Performance
**Purpose:** Page caching, asset optimization, image optimization, and database cleanup.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `Toolkit_Performance_Module` | `modules/performance/class-performance-module.php` | Module bootstrap |
| `Toolkit_Page_Cache` | `modules/performance/class-page-cache.php` | File-based page cache |
| `Toolkit_Asset_Optimizer` | `modules/performance/class-asset-optimizer.php` | CSS/JS combine + minify |
| `Toolkit_Image_Optimizer` | `modules/performance/class-image-optimizer.php` | WebP conversion, lazy loading |
| `Toolkit_Critical_CSS` | `modules/performance/class-critical-css.php` | Above-fold CSS extraction |
| `Toolkit_HTML_Minifier` | `modules/performance/class-html-minifier.php` | Remove whitespace/comments |
| `Toolkit_Browser_Cache` | `modules/performance/class-browser-cache.php` | Expires + cache-control headers |
| `Toolkit_Preload` | `modules/performance/class-preload.php` | `<link rel="preload">` injection |
| `Toolkit_Database_Optimizer` | `modules/performance/class-database-optimizer.php` | Clean transients, revisions, spam |
**Features:**
| Feature | Tier | Description |
|---------|------|-------------|
| Page cache (basic) | Free | File-based cache, auto-purge on `save_post` |
| HTML minification | Free | Strip whitespace and comments from HTML output |
| Asset optimizer | Premium | Combine + minify CSS/JS with exclusion rules |
| Image optimizer | Premium | WebP conversion via GD/Imagick, lazy loading, responsive srcset |
| Critical CSS | Premium | Extract above-fold CSS for faster FCP |
| Browser cache headers | Premium | Expires + cache-control via `.htaccess` or `send_headers` |
| Preload/prefetch | Premium | `<link rel="preload">` for key resources |
| Gzip compression check | Free | Display gzip status in dashboard |
| Database optimizer | Premium | Clean transients, post revisions, spam comments, orphan meta |
**Database Table: `{prefix}igny8_toolkit_cache_stats`**
```sql
CREATE TABLE {prefix}igny8_toolkit_cache_stats (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
hits INT DEFAULT 0,
misses INT DEFAULT 0,
purges INT DEFAULT 0,
last_purge DATETIME NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 2: Forms
**Purpose:** Drag-and-drop form builder with submissions storage, email notifications, and anti-spam.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `Toolkit_Forms_Module` | `modules/forms/class-forms-module.php` | Module bootstrap |
| `Toolkit_Form_Builder` | `modules/forms/class-form-builder.php` | Admin form editor |
| `Toolkit_Form_Renderer` | `modules/forms/class-form-renderer.php` | Frontend rendering via shortcode/block |
| `Toolkit_Form_Processor` | `modules/forms/class-form-processor.php` | Submission handling |
| `Toolkit_Form_Notifications` | `modules/forms/class-form-notifications.php` | Email notifications |
| `Toolkit_Form_Entries` | `modules/forms/class-form-entries.php` | Submission viewer + export |
| `Toolkit_Form_AntiSpam` | `modules/forms/class-form-antispam.php` | Honeypot + token validation |
**Features:**
| Feature | Tier | Description |
|---------|------|-------------|
| Basic builder (5 field types) | Free | text, email, textarea, select, checkbox |
| Simple rendering | Free | Shortcode `[igny8_form id="X"]` |
| All field types | Premium | + radio, file upload, hidden, number, date |
| Conditional logic | Premium | Show/hide fields based on other field values |
| Multi-step forms | Premium | Step-by-step form wizard |
| Submissions dashboard | Premium | View, search, export CSV, delete entries |
| Email templates | Premium | Customizable notification templates |
**Anti-spam:** Honeypot field + basic token validation — no external service dependency.
**Embed:** Shortcode `[igny8_form id="X"]` + Gutenberg block for insertion.
**Database Table: `{prefix}igny8_toolkit_submissions`**
```sql
CREATE TABLE {prefix}igny8_toolkit_submissions (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
form_id INT NOT NULL,
data LONGTEXT NOT NULL,
ip_hash VARCHAR(64) NULL,
user_agent VARCHAR(500) NULL,
created_at DATETIME NOT NULL,
INDEX idx_form (form_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 3: Security
**Purpose:** Login protection, basic firewall, WordPress hardening, and audit logging.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `Toolkit_Security_Module` | `modules/security/class-security-module.php` | Module bootstrap |
| `Toolkit_Login_Protection` | `modules/security/class-login-protection.php` | Brute-force protection |
| `Toolkit_Firewall` | `modules/security/class-firewall.php` | Bad user agent blocking, upload protection |
| `Toolkit_Hardening` | `modules/security/class-hardening.php` | XML-RPC disable, file editor, WP version |
| `Toolkit_Headers` | `modules/security/class-headers.php` | Security headers |
| `Toolkit_Audit_Log` | `modules/security/class-audit-log.php` | Event tracking |
**Features:**
| Feature | Tier | Description |
|---------|------|-------------|
| Login attempts limit | Free | Default 5 attempts, 15min lockout |
| File editor disable | Free | `DISALLOW_FILE_EDIT` constant |
| Security headers | Free | X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy |
| Hide WP version | Free | Remove generator meta tag |
| XML-RPC disable | Free | Block XML-RPC requests |
| Advanced firewall | Premium | Block bad user agents, PHP execution in uploads |
| 2FA via email code | Premium | Two-factor authentication on login |
| Full audit logging | Premium | Track logins, settings changes, plugin activations, role changes |
| IP allowlist/blocklist | Premium | Manage access by IP address |
**Database Table: `{prefix}igny8_toolkit_audit_log`**
```sql
CREATE TABLE {prefix}igny8_toolkit_audit_log (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
event_type VARCHAR(50) NOT NULL,
user_id BIGINT NULL,
object_type VARCHAR(50) NULL,
object_id BIGINT NULL,
details LONGTEXT NULL,
ip VARCHAR(45) NULL,
created_at DATETIME NOT NULL,
INDEX idx_event (event_type, created_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 4: SMTP
**Purpose:** SMTP mail override for sites NOT using the main IGNY8 plugin. If both installed, this module defers to IGNY8's SMTP (03A Module 9b).
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `Toolkit_SMTP_Module` | `modules/smtp/class-smtp-module.php` | Module bootstrap (checks for main plugin) |
| `Toolkit_SMTP_Mailer` | `modules/smtp/class-smtp-mailer.php` | SMTP mail override |
| `Toolkit_Email_Log` | `modules/smtp/class-email-log.php` | Delivery log with status |
| `Toolkit_Email_Test` | `modules/smtp/class-email-test.php` | Test email sender |
**Features:**
| Feature | Tier | Description |
|---------|------|-------------|
| SMTP override | Free | Host, port, username, password, encryption (SSL/TLS) |
| From name/email | Free | Override WordPress default sender |
| Mail sending | Free | Route all `wp_mail()` through SMTP |
| Email delivery log | Premium | Track sent/failed with error details |
| Email templates | Premium | Customizable notification email templates |
| Resend failed | Premium | Retry failed deliveries |
**Conflict Resolution:** On `boot()`, check `class_exists('IGNY8_SMTP_Module')` — if main plugin SMTP is active, this module auto-deactivates and shows admin notice: "SMTP is handled by the IGNY8 plugin."
**Database Table: `{prefix}igny8_toolkit_email_log`**
```sql
CREATE TABLE {prefix}igny8_toolkit_email_log (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
to_email VARCHAR(320) NOT NULL,
subject VARCHAR(500) NOT NULL,
status VARCHAR(20) NOT NULL,
error_message TEXT NULL,
sent_at DATETIME NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```
### Module 5: WooCommerce Enhancements
**Purpose:** UX improvements for WooCommerce stores. Entirely premium — only available in paid version.
**Classes:**
| Class | File | Purpose |
|-------|------|---------|
| `Toolkit_WooCommerce_Module` | `modules/woocommerce/class-woocommerce-module.php` | Module bootstrap (requires WooCommerce) |
| `Toolkit_Quick_View` | `modules/woocommerce/class-quick-view.php` | Modal product preview on shop pages |
| `Toolkit_Wishlist` | `modules/woocommerce/class-wishlist.php` | Cookie-based (guest) + user meta (logged in) |
| `Toolkit_AJAX_Cart` | `modules/woocommerce/class-ajax-cart.php` | AJAX add-to-cart on archive pages |
| `Toolkit_Product_Filters` | `modules/woocommerce/class-product-filters.php` | Price/attribute/category AJAX filters |
| `Toolkit_Product_Gallery` | `modules/woocommerce/class-product-gallery.php` | Zoom, lightbox, thumbnails |
**Features (all Premium):**
- Quick view modal on shop/archive pages
- Wishlist (cookie-based for guests, user meta for logged-in)
- AJAX add-to-cart on archive pages (no page reload)
- Product filters (price range, attributes, categories) with AJAX
- Product gallery enhancement (zoom, lightbox, thumbnail navigation)
**Conditional Loading:** Module only loads when WooCommerce is active (`class_exists('WooCommerce')`).
---
## 3. DATA MODELS & APIS
### Database Tables (4 total)
All tables created on plugin activation by installer class:
| # | Table | Module | Purpose |
|---|-------|--------|---------|
| 1 | `{prefix}igny8_toolkit_submissions` | Forms | Form submission data (JSON) |
| 2 | `{prefix}igny8_toolkit_email_log` | SMTP | Email delivery tracking |
| 3 | `{prefix}igny8_toolkit_audit_log` | Security | Security event log |
| 4 | `{prefix}igny8_toolkit_cache_stats` | Performance | Cache hit/miss stats |
### Options Summary
All options use `igny8_toolkit_` prefix:
| Option Key | Module | Content |
|------------|--------|---------|
| `igny8_toolkit_performance_settings` | Performance | Cache, minify, optimize toggles |
| `igny8_toolkit_forms` | Forms | JSON: `[{id, name, fields, settings}]` |
| `igny8_toolkit_security_settings` | Security | Login limits, hardening toggles |
| `igny8_toolkit_smtp_settings` | SMTP | Host, port, user, pass, encryption |
| `igny8_toolkit_woocommerce_settings` | WooCommerce | Feature toggles |
| `igny8_toolkit_license_key` | Global | Premium license key |
| `igny8_toolkit_license_status` | Global | active/expired/invalid |
### No REST API Endpoints
The toolkit plugin does not expose REST endpoints. All admin pages are standard WordPress admin screens.
---
## 4. IMPLEMENTATION STEPS
### Plugin Architecture
```
igny8-toolkit/
├── igny8-toolkit.php # Plugin header, constants, module loader
├── readme.txt # WordPress.org readme
├── uninstall.php # Full cleanup on uninstall
├── includes/
│ ├── class-toolkit.php # Main singleton class
│ ├── class-module-manager.php # Module toggle (same pattern as 03A)
│ ├── class-installer.php # Create 4 DB tables on activation
│ ├── class-license.php # Premium license validation
│ ├── modules/
│ │ ├── performance/ # Module 1 — 9 class files
│ │ ├── forms/ # Module 2 — 7 class files
│ │ ├── security/ # Module 3 — 6 class files
│ │ ├── smtp/ # Module 4 — 4 class files
│ │ └── woocommerce/ # Module 5 — 6 class files
│ └── admin/
│ ├── class-dashboard.php # Toolkit dashboard
│ └── views/ # Admin page templates
├── admin/ # Admin CSS + JS
│ ├── css/
│ └── js/
├── frontend/ # Frontend CSS + JS
│ ├── css/
│ └── js/
└── languages/
└── igny8-toolkit.pot # Translation template
```
### Build Sequence
**Phase 1: Foundation (Days 1-2)**
1. Create `igny8-toolkit.php` with header, constants, autoloader
2. Build `class-toolkit.php` singleton — `instance()`, `init()`, `load_modules()`
3. Build `class-module-manager.php` — same pattern as 03A
4. Build `class-installer.php` — create 4 DB tables via `dbDelta()`
5. Build `class-license.php` — premium license validation
6. Build `admin/class-dashboard.php` — toolkit overview page
**Phase 2: Performance Module (Days 3-5)**
1. Build page cache (file-based, purge on `save_post`)
2. Build HTML minifier (output buffer processing)
3. Build asset optimizer (combine/minify with exclusions)
4. Build image optimizer (WebP via GD/Imagick, lazy loading)
5. Build critical CSS generator
6. Build browser cache + preload
7. Build database optimizer
**Phase 3: Forms Module (Days 6-8)**
1. Build form builder admin UI
2. Build form renderer (shortcode + block)
3. Build form processor (validation + sanitization)
4. Build notifications (email on submission)
5. Build anti-spam (honeypot + token)
6. Build entries admin page (view + CSV export)
**Phase 4: Security Module (Days 9-11)**
1. Build login protection (attempt tracking, lockout)
2. Build firewall (bad user agents, upload protection)
3. Build hardening (XML-RPC, file editor, version hide, headers)
4. Build audit log (event tracking)
5. Build 2FA (premium — email code on login)
**Phase 5: SMTP Module (Days 12-13)**
1. Build SMTP mailer (override `wp_mail`)
2. Build conflict check (defer to main IGNY8 plugin if active)
3. Build email log
4. Build test email sender
**Phase 6: WooCommerce Module (Days 14-16)**
1. Build quick view modal
2. Build wishlist (cookie + user meta)
3. Build AJAX cart
4. Build product filters
5. Build gallery enhancements
**Phase 7: Compliance (Days 17-18)**
1. WordPress.org compliance — sanitize/escape, nonces, capability checks
2. i18n pass — all strings via `igny8-toolkit` text domain
3. readme.txt with feature list, changelog
4. Accessibility review
### Admin Menu Structure
```
IGNY8 Toolkit (top-level, dashicons-admin-tools)
├── Dashboard — Overview, module status, cache stats
├── Performance — Cache, minify, optimize, image settings
├── Forms — Form builder, entries, settings
├── Security — Login protection, hardening, audit log
├── SMTP — Mail settings, log, test
├── WooCommerce — Feature toggles (when WooCommerce active)
└── License — Premium license management
```
---
## 5. ACCEPTANCE CRITERIA
### Performance (Module 1)
- [ ] Page cache creates static HTML files and serves them on repeat visits
- [ ] Cache auto-purges on `save_post` for the updated post's URL
- [ ] HTML minification reduces output size by removing whitespace/comments
- [ ] Image optimizer converts uploads to WebP format
- [ ] Lazy loading adds `loading="lazy"` to images below fold
- [ ] Database optimizer cleans transients, revisions, spam comments
- [ ] Cache stats table tracks hits/misses/purges
### Forms (Module 2)
- [ ] Form builder creates forms with configurable fields
- [ ] `[igny8_form id="X"]` shortcode renders form on frontend
- [ ] Form submissions stored in `igny8_toolkit_submissions` table
- [ ] Email notification sent on submission
- [ ] Honeypot field blocks spam submissions
- [ ] Entries admin page shows submissions with CSV export
- [ ] All form data sanitized: `sanitize_text_field()`, `sanitize_email()`, `wp_kses_post()` per field type
### Security (Module 3)
- [ ] Login attempts limited to 5 per 15 minutes (configurable)
- [ ] Lockout message displayed after threshold exceeded
- [ ] XML-RPC disabled when hardening active
- [ ] File editor disabled via `DISALLOW_FILE_EDIT`
- [ ] WP version removed from `<head>` and feeds
- [ ] Security headers set: X-Frame-Options, X-Content-Type-Options, Referrer-Policy
- [ ] Audit log records login attempts, settings changes, plugin activations
### SMTP (Module 4)
- [ ] SMTP override routes `wp_mail()` through configured server
- [ ] Test email sends successfully with configured SMTP
- [ ] Module auto-deactivates when main IGNY8 plugin SMTP is active
- [ ] Email log records sent/failed status with error messages
### WooCommerce (Module 5)
- [ ] Quick view modal opens on shop/archive pages
- [ ] Wishlist persists for guests (cookie) and users (user meta)
- [ ] AJAX add-to-cart works without page reload
- [ ] Product filters update results via AJAX
- [ ] Module only loads when WooCommerce is active
### Cross-Module
- [ ] All 5 modules independently toggleable — zero impact when disabled
- [ ] Plugin activates without errors on WordPress 5.9+ with PHP 7.4+
- [ ] All 4 database tables created on activation
- [ ] All strings translatable via `igny8-toolkit` text domain
- [ ] All inputs sanitized, all outputs escaped
- [ ] Nonce verification on all form submissions
- [ ] Premium features gated behind license validation
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
Before starting implementation:
1. Read 03A (main plugin) — understand module manager pattern to replicate
2. Read 03C (companion theme) — understand theme as primary consumer of toolkit features
### Execution Order
```
Foundation → Performance → Forms → Security → SMTP → WooCommerce → Compliance
```
### Critical Rules
1. **Independent plugin** — must work without main IGNY8 plugin or companion theme
2. **Same module pattern** — `register()`, `activate()`, `deactivate()`, `is_active()`, `boot()` — identical to 03A
3. **SMTP conflict resolution** — check `class_exists('IGNY8_SMTP_Module')` on boot, auto-defer if main plugin active
4. **WooCommerce conditional** — Module 5 only loads when `class_exists('WooCommerce')` is true
5. **Table prefix** — all tables use `{wp_prefix}igny8_toolkit_` (different from main plugin's `{wp_prefix}igny8_`)
6. **Option prefix** — all options use `igny8_toolkit_` prefix
7. **Text domain** — `igny8-toolkit` (different from plugin's `igny8` and theme's `igny8-theme`)
8. **Freemium** — free features work without license, premium features check `class-license.php` validation
9. **Sanitize/escape** — same WordPress standards as 03A
10. **No external dependencies** — no CDN scripts, no external API calls (except SMTP + optional license check)
### File Creation Order
```
1. igny8-toolkit.php (plugin header + bootstrap)
2. includes/class-toolkit.php (singleton)
3. includes/class-module-manager.php (module orchestration)
4. includes/class-installer.php (4 DB tables)
5. includes/class-license.php (premium validation)
6. includes/admin/class-dashboard.php (overview page)
7. includes/modules/performance/* (9 files)
8. includes/modules/forms/* (7 files)
9. includes/modules/security/* (6 files)
10. includes/modules/smtp/* (4 files)
11. includes/modules/woocommerce/* (6 files)
12. admin/ (CSS + JS assets)
13. frontend/ (CSS + JS assets)
14. uninstall.php (cleanup)
15. languages/igny8-toolkit.pot (i18n)
```
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 03A WP Plugin Standalone | Module manager pattern replicated; SMTP conflict resolution when both installed |
| 03C Companion Theme | Theme is primary consumer of toolkit features (forms, performance) |

486
v2/V2-Execution-Docs/04A-managed-services.md Normal file
View File

@@ -0,0 +1,486 @@
# IGNY8 Phase 4: Managed Services (04A)
## Service Tiers, Backlink Orders, Ahrefs Integration, Backlink Indexing
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 4 — Business Layer
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Business Operations, Architects
---
## 1. CURRENT STATE
### Billing Infrastructure Exists
The billing app provides:
- `Plan` — subscription tier definition (Free/$49/$149/$349)
- `Subscription` — active subscription per account, links to Plan
- `CreditTransaction` — credit debit/credit events
- `CreditUsageLog` — per-operation usage tracking
- `CreditCostConfig` — configurable cost per operation type (PK = operation_type)
- `AccountPaymentMethod` — saved Stripe/PayPal payment methods
- `Payment` + `Invoice` — payment and invoice records
- `CreditService` in `business/billing/` — pattern: `check_credits()` → execute → `deduct_credits()` → `log_usage()`
- Stripe + PayPal live and integrated
### Current Plans
| Plan | Price | Sites | Users | Credits/Mo |
|------|-------|-------|-------|------------|
| Free | $0 | 1 | 1 | 100 |
| Starter | $49 | 3 | 3 | 1,000 |
| Growth | $149 | 10 | 10 | 5,000 |
| Scale | $349 | Unlimited | Unlimited | 25,000 |
### What Does NOT Exist
- No managed services layer — Alorig manages client sites manually
- No backlink order tracking — orders placed manually on FatGrid/PRNews
- No Ahrefs integration — domain/backlink data checked manually
- No backlink indexing service — links indexed organically or manually submitted
- No automated onboarding — each client setup is manual
- 6 existing Alorig clients ready for managed service conversion
### Phase 2 Models Available
- `SAGCampaign`, `SAGBacklink`, `CampaignKPISnapshot` (02E) — backlink campaign engine
- `GSCConnection`, `URLInspectionRecord`, `GSCMetricsCache` (02C) — GSC data
- `SAGLink`, `SAGLinkAudit`, `LinkMap` (02D) — internal linking
- `SocialPost`, `SocialAccount`, `SocialEngagement` (02H) — socializer
- `SchemaTemplate`, `SERPEnhancement`, `SchemaValidationResult` (02G) — schema
- `AutomationConfig`, `AutomationRun` (automation app) — pipeline orchestration
- All IDs are integers (BigAutoField)
---
## 2. WHAT TO BUILD
### Overview
A managed services layer that lets Alorig sell done-for-you SEO services at two tiers, with integrated backlink ordering, Ahrefs API monitoring, and backlink indexing. Everything builds on existing IGNY8 models and infrastructure.
### Managed Service Tiers
**Managed Lite ($100/site/month):**
- 10 articles/month (published to WordPress via pipeline)
- Basic SAG setup + automation configuration (first month)
- Monthly PDF report
- Email support (48-hour response)
- Automation config: weekly schedule, 3 articles/run, post + cluster_hub types
- Review required before publish (Alorig team reviews in Writer)
**Managed Pro ($399/site/month):**
- 30 articles/month (hubs prioritized early, supporting content later)
- Full SAG architecture with detailed mode
- Backlink campaign management (budget separate — pass-through + 20% markup)
- Social media content generation + scheduling (configured platforms)
- Retroactive schema enhancement on existing pages
- GSC monitoring: auto-indexing, re-inspection, issue alerts
- Weekly PDF report with full KPIs
- Dedicated account manager (24-hour response)
- Taxonomy term content generation
### Backlink Service Packages
**By Quality Level:**
| Tier | Service | Alorig Cost | Client Price | Margin |
|------|---------|-------------|-------------|--------|
| Basic Guest Post | DR 30-50 via FatGrid | $30-80 | $150-300 | 3-5× |
| Standard Guest Post | DR 50-70 via FatGrid | $100-300 | $400-800 | 2-3× |
| Premium Guest Post | DR 70+ via FatGrid | $500-2,000 | $1,500-5,000 | 2-3× |
| PR Basic | 300+ outlets (EIN Presswire) | $99-499 | $500-1,500 | 3-5× |
| PR Premium | Yahoo/Bloomberg/Fox | $500-5,000 | $2,000-15,000 | 3-4× |
**Monthly Retainer Packages:**
| Package | Links/Mo | DR Range | Monthly Cost | Market |
|---------|----------|----------|-------------|--------|
| Starter | 5-8 | DR 30-50 | $800-1,500 | PK |
| Growth | 10-15 | DR 30-70 mix | $2,000-4,000 | UK/CA |
| Authority | 15-25 | DR 40-70+ mix + PR | $4,000-8,000 | USA |
| Enterprise | Custom | Custom | Custom | Multi-site/agency |
**Niche Surcharges:** Crypto/Casino 2-3×, Finance/Insurance 1.5-2×, Health/Medical 1.5-2×, Tech/SaaS 1.2-1.5×, General 1× baseline.
### Ahrefs API Integration
IGNY8 integrates with Ahrefs API v3 for automated domain metrics and backlink monitoring.
**API Endpoints Used:**
| Ahrefs Endpoint | IGNY8 Use Case |
|-----------------|----------------|
| `GET /v3/site-explorer/domain-rating` | DR for any domain (backlink quality scoring) |
| `GET /v3/site-explorer/backlinks` | Verify placed backlinks are live |
| `GET /v3/site-explorer/referring-domains` | Referring domain list + metrics |
| `GET /v3/site-explorer/organic-keywords` | Keyword rankings (supplement GSC data) |
| `GET /v3/site-explorer/pages-by-traffic` | Top pages by organic traffic |
| `GET /v3/site-explorer/metrics` | Domain summary (DR, referring domains, traffic, keywords) |
| `GET /v3/site-explorer/metrics-history` | Historical DR + traffic trend |
| `GET /v3/site-explorer/refdomains-history` | Referring domains over time |
**Use Cases:**
1. **Backlink verification** — after SAGBacklink placed, verify via Ahrefs crawl data
2. **Quality scoring** — auto-populate `source_dr`, `source_traffic` on SAGBacklink
3. **Campaign KPIs** — pull DR, referring domains, keywords for CampaignKPISnapshot
4. **Competitor analysis** — compare client's metrics against competitors
5. **Dead link detection** — cross-reference lost backlinks with SAGBacklink records
6. **Reporting** — feed ServiceReport data (04B)
7. **Link prospecting** — find competitor backlinks for outreach
**Pricing:** Credits-based (varies by Ahrefs plan). API key stored per account in `AhrefsConnection`.
### Backlink Indexing Service
After a backlink is placed and verified live, submit it for Google indexing to accelerate authority transfer.
**Indexing Services (integrate best available):**
- IndexNow API (free) — Bing/Yandex submission
- SpeedyIndex — fast indexing, API available, credit-based
- Omega Indexer — bulk indexing, ~$0.02-0.05/URL
**Integration Flow:**
```
SAGBacklink.status → 'live'
→ Verify URL accessible (HTTP 200)
→ Queue BacklinkIndexingRequest (status: queued)
→ Submit via indexing service API (status: submitted)
→ Wait 7 days, check if indexed
→ If indexed → is_indexed = True, status → indexed
→ If not after 14 days → re-submit once (status: resubmitted)
→ If still not after 28 days → status → failed, flag for review
```
**Client Pricing:** $0.10-0.20 per URL (5-10× markup); included in Managed Pro tier.
### Client Onboarding Automation
When `ManagedServiceSubscription` is created:
1. Verify site has IGNY8 plugin installed + connected (`SiteIntegration` exists)
2. If no blueprint → queue Celery task to auto-generate SAG blueprint
3. Configure `AutomationConfig` based on `service_config` (schedule, stages, content types)
4. Send welcome email (expectations, content approval process, contact info)
5. Create first month's content tasks in pipeline queue
6. Notify account manager: "New client onboarded, first content due"
7. Update subscription status: pending → active
**Monthly Delivery Workflow (Lite):**
```
Month Start → Verify blueprint → Run pipeline (10 articles) →
Alorig reviews → Approve/edit → Publish to WP →
Update articles_published counter → Month End → Generate report → Send → Bill
```
**Monthly Delivery Workflow (Pro — extends Lite):**
```
+ Run pipeline for 30 articles (hubs prioritized)
+ Taxonomy term content generation
+ Backlink: Load SAGCampaign plan → Browse FatGrid → Place orders →
Track delivery → Quality check → Log SAGBacklink → Submit for indexing
+ Social: Generate adapted posts → Schedule via best-time algorithm
+ Schema: Retroactive scan → Push to WP
+ GSC: Check indexing → Re-request pending → Flag issues
+ Weekly: Generate report → Send
```
### Margin Analysis
| Component | Managed Lite | Managed Pro |
|-----------|-------------|-------------|
| Service Fee | $100/month | $399/month |
| Platform Cost (credits) | ~$15-25 | ~$50-80 |
| Human Time | ~2-3 hrs/month | ~8-12 hrs/month |
| Ahrefs API cost | ~$2-5 | ~$10-20 |
| Indexing service cost | $0 | ~$5-15 |
| **Gross Margin** | **~70-75%** | **~65-75%** |
Backlink purchase costs are pass-through with 20% markup — separate from service fee.
---
## 3. DATA MODELS & APIS
### New App: `services`
**App Label:** `services`
**Location:** `igny8_core/services/`
**Table Prefix:** `igny8_`
### New Models
**`ManagedServiceSubscription`** (extends `AccountBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `site` | ForeignKey(Site) | Which site is being managed |
| `tier` | CharField(max_length=20) | `lite` / `pro` |
| `status` | CharField(max_length=20) | `pending` / `active` / `paused` / `cancelled` |
| `monthly_price` | DecimalField(8,2) | Monthly fee |
| `articles_per_month` | IntegerField | Target articles per month |
| `account_manager` | ForeignKey(User, null=True) | Assigned team member |
| `current_month_articles_published` | IntegerField(default=0) | Counter |
| `current_month_start` | DateField(null=True) | Current billing period start |
| `service_config` | JSONField | Per-site settings (automation, social, backlinks, reporting) |
| `started_at` | DateTimeField(null=True) | Service start timestamp |
| `cancelled_at` | DateTimeField(null=True) | Cancellation timestamp |
| `next_billing_date` | DateField(null=True) | Next billing date |
Table: `igny8_managed_service_subscriptions`
**`BacklinkServiceOrder`** (extends `AccountBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `site` | ForeignKey(Site) | Target site |
| `campaign` | ForeignKey(SAGCampaign, null=True) | Linked campaign from 02E |
| `managed_subscription` | ForeignKey(ManagedServiceSubscription, null=True) | If part of managed service |
| `order_type` | CharField(max_length=20) | `one_time` / `monthly` |
| `package` | CharField(max_length=50) | `starter` / `growth` / `authority` / `enterprise` / `custom` |
| `client_price` | DecimalField(10,2) | Price charged to client |
| `actual_cost` | DecimalField(10,2, default=0) | Actual cost to Alorig |
| `margin` | DecimalField(10,2, default=0) | Calculated margin |
| `niche_surcharge` | FloatField(default=1.0) | Niche multiplier |
| `links_ordered` | IntegerField(default=0) | Total links ordered |
| `links_delivered` | IntegerField(default=0) | Links delivered by vendor |
| `links_live` | IntegerField(default=0) | Confirmed live |
| `links_indexed` | IntegerField(default=0) | Confirmed indexed |
| `status` | CharField(max_length=20) | `draft` / `approved` / `in_progress` / `delivered` / `completed` |
| `ordered_at` | DateTimeField(null=True) | Order placement date |
| `delivered_at` | DateTimeField(null=True) | Full delivery date |
| `notes` | TextField(blank=True) | Internal notes |
Table: `igny8_backlink_service_orders`
**`AhrefsConnection`** (extends `AccountBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `api_key` | TextField | Encrypted Ahrefs API key |
| `plan_type` | CharField(null=True) | `lite` / `standard` / `advanced` / `enterprise` |
| `credits_remaining` | IntegerField(null=True) | Remaining API credits |
| `last_synced` | DateTimeField(null=True) | Last sync timestamp |
| `status` | CharField(max_length=20) | `active` / `expired` / `error` |
Table: `igny8_ahrefs_connections`
**`AhrefsDomainSnapshot`** (extends `SiteSectorBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `domain` | CharField(max_length=255) | Domain name |
| `domain_rating` | FloatField | DR score |
| `referring_domains` | IntegerField | Count |
| `organic_keywords` | IntegerField | Count |
| `organic_traffic` | IntegerField | Estimated monthly |
| `backlinks_total` | IntegerField | Total backlinks |
| `snapshot_date` | DateField | Date of snapshot |
| `raw_data` | JSONField | Full API response |
Table: `igny8_ahrefs_domain_snapshots`
**`BacklinkIndexingRequest`** (extends `SiteSectorBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `backlink` | ForeignKey(SAGBacklink) | Linked backlink from 02E |
| `url` | URLField | Source URL to be indexed |
| `indexing_service` | CharField(max_length=30) | `speedyindex` / `omega` / `colossus` / `indexnow` / `manual` |
| `service_request_id` | CharField(null=True) | External service reference |
| `submitted_at` | DateTimeField(null=True) | Submission timestamp |
| `first_check_at` | DateTimeField(null=True) | First index check |
| `last_check_at` | DateTimeField(null=True) | Last index check |
| `check_count` | IntegerField(default=0) | Times checked |
| `is_indexed` | BooleanField(default=False) | Is indexed |
| `indexed_at` | DateTimeField(null=True) | When indexed |
| `status` | CharField(max_length=20) | `queued` / `submitted` / `checking` / `indexed` / `failed` / `resubmitted` |
| `cost` | DecimalField(6,4, default=0) | Cost per submission |
Table: `igny8_backlink_indexing_requests`
### Modified Models
**`SAGBacklink`** (from 02E linker app) — add fields:
| Field | Type | Description |
|-------|------|-------------|
| `is_indexed` | BooleanField(default=False) | Indexed in Google |
| `indexed_at` | DateTimeField(null=True) | When indexed |
| `ahrefs_verified` | BooleanField(default=False) | Verified via Ahrefs |
| `ahrefs_last_seen` | DateTimeField(null=True) | Last seen in Ahrefs |
| `indexing_submitted` | BooleanField(default=False) | Submitted for indexing |
### API Endpoints
All under `/api/v1/`:
**Managed Service Subscriptions:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/subscriptions/` | List all (admin only) |
| POST | `/services/subscriptions/` | Create subscription |
| GET | `/services/subscriptions/{id}/` | Detail |
| PATCH | `/services/subscriptions/{id}/` | Update config/status |
| POST | `/services/subscriptions/{id}/onboard/` | Trigger onboarding flow |
| POST | `/services/subscriptions/{id}/pause/` | Pause service |
| POST | `/services/subscriptions/{id}/resume/` | Resume service |
| POST | `/services/subscriptions/{id}/cancel/` | Cancel service |
**Backlink Service Orders:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/backlink-orders/` | List all (admin) |
| POST | `/services/backlink-orders/` | Create order |
| GET | `/services/backlink-orders/{id}/` | Detail |
| PATCH | `/services/backlink-orders/{id}/` | Update status |
| GET | `/services/backlink-orders/{id}/links/` | Links in order |
| POST | `/services/backlink-orders/{id}/approve/` | Client approval |
**Ahrefs Integration:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| POST | `/integration/ahrefs/connect/` | Save API key |
| DELETE | `/integration/ahrefs/disconnect/` | Remove connection |
| GET | `/integration/ahrefs/status/` | Connection status + credits |
| GET | `/integration/ahrefs/domain/{domain}/` | Domain metrics |
| GET | `/integration/ahrefs/domain/{domain}/history/` | DR + traffic history |
| GET | `/integration/ahrefs/backlinks/{domain}/` | Backlinks list |
| GET | `/integration/ahrefs/referring-domains/{domain}/` | Referring domains |
| GET | `/integration/ahrefs/organic-keywords/{domain}/` | Organic keywords |
| POST | `/integration/ahrefs/verify-backlinks/` | Bulk verify SAGBacklinks |
| POST | `/integration/ahrefs/snapshot/` | Take domain snapshot |
**Backlink Indexing:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| POST | `/linker/indexing/submit/` | Submit URLs for indexing |
| POST | `/linker/indexing/bulk-submit/` | Bulk submit all live unindexed |
| GET | `/linker/indexing/requests/` | List requests (filter by site_id) |
| GET | `/linker/indexing/requests/{id}/` | Single request status |
| GET | `/linker/indexing/stats/` | Success rate, pending count |
**Admin Dashboard:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/dashboard/` | Revenue summary, client count, MRR |
| GET | `/services/dashboard/overdue/` | Overdue deliverables |
| GET | `/services/dashboard/margin/` | Margin tracking across orders |
### Celery Tasks
| Task | Schedule | Purpose |
|------|----------|---------|
| `onboard_managed_client` | On-demand | Full onboarding flow |
| `process_managed_billing` | Monthly (1st) | Invoice all active subscriptions |
| `reset_monthly_counters` | Monthly (1st) | Reset articles_published counters |
| `check_delivery_status` | Daily | Flag subscriptions behind on deliverables |
| `sync_ahrefs_domain_snapshot` | Weekly | Pull DR/traffic/referring domains per connected site |
| `verify_backlinks_via_ahrefs` | Weekly | Cross-reference SAGBacklinks with Ahrefs data |
| `submit_backlinks_for_indexing` | Daily | Submit live unindexed backlinks |
| `check_indexing_status` | Daily | Check submitted URLs (after 7-day wait) |
| `detect_lost_backlinks_ahrefs` | Weekly | Compare Ahrefs lost links with SAGBacklink records |
### Services
| Service | Purpose |
|---------|---------|
| `ManagedBillingService` | Subscription CRUD, billing, margin calculation |
| `AhrefsService` | API client wrapper, rate limiting, credit tracking |
| `BacklinkIndexingService` | Submit to indexing service, track status, retry logic |
---
## 4. IMPLEMENTATION STEPS
### Build Sequence
**Week 1: Foundation**
1. Create `services` app under `igny8_core/services/`
2. Create `ManagedServiceSubscription` model + migration
3. Create `BacklinkServiceOrder` model + migration
4. Create serializers + viewsets for both
5. Build `ManagedBillingService` — subscription CRUD, billing integration
6. Build onboarding Celery task
**Week 2: Ahrefs + Indexing**
1. Create `AhrefsConnection` + `AhrefsDomainSnapshot` models + migration
2. Build `AhrefsService` — API client with rate limiting and credit tracking
3. Create `BacklinkIndexingRequest` model + migration
4. Build `BacklinkIndexingService` — multi-provider submission
5. Add 5 new fields to `SAGBacklink` model + migration
6. Wire up all Celery tasks
**Week 3: API + Dashboard**
1. Register all API endpoints
2. Build admin dashboard endpoints (revenue, overdue, margin)
3. Build onboarding flow end-to-end
4. Test full managed service lifecycle
---
## 5. ACCEPTANCE CRITERIA
### Managed Services
- [ ] `ManagedServiceSubscription` creates for Lite and Pro tiers
- [ ] Onboarding flow: verify plugin → generate blueprint → configure automation → welcome email
- [ ] Monthly billing generates invoices via existing Stripe/PayPal integration
- [ ] `articles_published` counter tracks against `articles_per_month` target
- [ ] Pause/resume/cancel flows work without data loss
- [ ] Service config JSON stores per-site automation, social, backlink settings
### Backlink Orders
- [ ] `BacklinkServiceOrder` tracks full lifecycle: draft → approved → in_progress → delivered → completed
- [ ] Links tracker: ordered vs delivered vs live vs indexed counts accurate
- [ ] Margin calculated correctly with niche surcharges
- [ ] Orders linkable to SAGCampaign (02E)
### Ahrefs Integration
- [ ] API key stored encrypted, connection test validates key
- [ ] Domain metrics endpoint returns DR, referring domains, traffic, keywords
- [ ] Weekly snapshots auto-populate `AhrefsDomainSnapshot`
- [ ] Backlink verification cross-references SAGBacklink records
- [ ] Lost backlink detection flags dead links
- [ ] Credits consumption tracked
### Backlink Indexing
- [ ] `BacklinkIndexingRequest` tracks full lifecycle: queued → submitted → checking → indexed/failed
- [ ] Submissions go to configured indexing service API
- [ ] 7-day wait before first check, re-submit at 14 days, fail at 28 days
- [ ] Cost tracking per submission
- [ ] `SAGBacklink.is_indexed` updated when confirmed
### Admin Dashboard
- [ ] Revenue summary shows total MRR, client count by tier
- [ ] Overdue endpoint flags behind-schedule content/links
- [ ] Margin tracking shows per-order and aggregate margins
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
1. Read 02E (Linker External) — SAGCampaign, SAGBacklink, CampaignKPISnapshot models
2. Read existing billing app models — Plan, Subscription, CreditService
3. Read 03B (WP Plugin Connected) — content delivery flow that managed services triggers
4. Read automation app — AutomationConfig, AutomationRun for pipeline configuration
### Execution Order
```
services app skeleton → models + migrations → ManagedBillingService → AhrefsService →
BacklinkIndexingService → API endpoints → Celery tasks → admin dashboard
```
### Critical Rules
1. **All IDs are integers** — BigAutoField PKs, integer FKs
2. **Model names PLURAL** — Clusters, Keywords, Tasks, etc.
3. **Table prefix** — `igny8_` for all tables
4. **App label** — `services` for new app
5. **Celery app** — `igny8_core` (same as all other tasks)
6. **API pattern** — `/api/v1/services/` for managed service endpoints, `/api/v1/integration/ahrefs/` for Ahrefs, `/api/v1/linker/indexing/` for indexing
7. **AccountBaseModel** for account-scoped models, **SiteSectorBaseModel** for site-scoped
8. **Encrypt API keys** — Ahrefs API key encrypted at rest
9. **Rate limiting** — respect Ahrefs API rate limits, exponential backoff on 429
10. **No blocking** — all external API calls in Celery tasks, never in request/response cycle
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 02E Linker External | SAGCampaign, SAGBacklink models — backlink orders link to these |
| 02C GSC Integration | GSCMetricsCache — monitoring data for Pro tier, indexing status |
| 02D Linker Internal | SAGLink, LinkMap — internal linking health for service delivery |
| 02H Socializer | SocialPost, SocialAccount — social content for Pro tier |
| 02G Rich Schema SERP | SchemaTemplate — retroactive schema enhancement for Pro tier |
| 01A SAG Data Foundation | SAGBlueprint — auto-generated during onboarding |
| 03B WP Plugin Connected | Content delivery to WordPress via sync endpoints |
| 04B Whitelabel Reporting | Reports consume subscription + order data from this doc |
| 04C Pricing Launch | Feature gates control who can purchase managed services |

393
v2/V2-Execution-Docs/04B-whitelabel-reporting.md Normal file
View File

@@ -0,0 +1,393 @@
# IGNY8 Phase 4: White-Label & Reporting (04B)
## Automated Reports, Agency Branding, Admin Dashboard, Client View
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 4 — Business Layer
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Frontend Developers, Architects
---
## 1. CURRENT STATE
### No Reporting Infrastructure
- Reports created manually by Alorig team via Google Docs/Sheets
- No automated metric collection across IGNY8 modules
- No PDF generation capability
- No scheduled report delivery
### No Agency/Reseller Model
- No white-label capability — all reports show IGNY8 branding
- No custom branding configuration
- No agency account structure
### Phase 4A Foundation Available (04A)
- `ManagedServiceSubscription` — client subscription data
- `BacklinkServiceOrder` — order tracking with margin
- `AhrefsDomainSnapshot` — DR, traffic, referring domains data
- `BacklinkIndexingRequest` — indexing status tracking
### Phase 2 Data Sources Available
- `Content` (writer app) — articles published, word counts
- `URLInspectionRecord` + `GSCMetricsCache` (02C) — indexing status, search performance
- `SAGBacklink` + `CampaignKPISnapshot` (02E) — backlink campaign metrics
- `SocialPost` + `SocialEngagement` (02H) — social media metrics
- `SAGBlueprint` health score (01G) — SAG completion and authority
- All IDs are integers (BigAutoField)
---
## 2. WHAT TO BUILD
### Overview
An automated report generation engine with 9 configurable sections, PDF output, white-label branding for agencies, an admin services dashboard for Alorig, and a client-facing read-only view for managed service subscribers.
### Report Generation Engine
**9 Configurable Report Sections** (each can be included/excluded per client):
| # | Section | Data Sources |
|---|---------|-------------|
| 1 | **Executive Summary** | AI-generated from all other sections — key wins, KPI highlights |
| 2 | **Content Performance** | `Content` model — articles published by type, word count, images created |
| 3 | **Indexing Status** | `URLInspectionRecord` + `GSCMetricsCache` (02C) — indexed/pending/error trend |
| 4 | **Keyword Rankings** | `GSCMetricsCache` (02C) + `AhrefsDomainSnapshot` (04A) — top 10/20/50, movers |
| 5 | **Organic Traffic** | `GSCMetricsCache` (02C) — clicks, impressions, CTR, avg position, MoM change |
| 6 | **Backlink Campaign** | `SAGBacklink` (02E) + `BacklinkServiceOrder` (04A) + `AhrefsDomainSnapshot` (04A) — links built, DR dist, budget, quality, referring domain growth |
| 7 | **Social Media** | `SocialPost` + `SocialEngagement` (02H) — posts by platform, engagement, top performers |
| 8 | **SAG Health** | `SAGBlueprint` (01G) — completion %, cluster status, authority score (0-100), content gaps |
| 9 | **Next Month Plan** | `SAGCampaign` (02E) + automation queue — upcoming content topics, backlink targets, action items |
### PDF Report Generation
| Component | Choice | Rationale |
|-----------|--------|-----------|
| Template engine | WeasyPrint (HTML → PDF) | CSS-based layout, full typography control |
| Charts | matplotlib (server-side rendering) | SVG/PNG charts embedded in HTML |
| Storage | S3/media storage | URL saved in `ServiceReport.report_pdf_url` |
| Themes | Light + Dark | Configurable per template |
**Report Flow:**
```
Celery task triggers → Collect metrics from all sources →
Render section HTML per included sections →
Generate charts via matplotlib → Merge into base template →
WeasyPrint converts HTML → PDF → Upload to storage →
Save ServiceReport with PDF URL → Email to recipients if configured
```
### White-Label Branding (Agency Play)
For agencies reselling IGNY8 services:
**`AgencyBranding` provides:**
- Agency name (replaces "IGNY8" / "Alorig" throughout PDF)
- Logo URL (replaces IGNY8 logo in header)
- Primary + secondary colors (brand color scheme)
- Contact email + phone + website URL
- Footer text
**When `ServiceReport.is_white_label = True`:**
- All "IGNY8" and "Alorig" text replaced with `brand_name`
- Logo swapped to `brand_logo_url`
- Footer shows agency contact info
- Color scheme uses agency brand colors
- Report URL domain can be custom (via Caddy subdomain routing)
### Admin Services Dashboard (Alorig Team Only)
**Frontend Pages (.tsx + Zustand):**
```
frontend/src/pages/Services/
├── ServicesDashboard.tsx # Overview: all managed clients, MRR, revenue
│ ├── ClientList.tsx # All subscriptions with status, tier, articles count
│ ├── RevenueSummary.tsx # MRR total, margin total, trend chart
│ └── UpcomingActions.tsx # Overdue: reports, content, renewals
├── ClientDetail.tsx # Single client management
│ ├── ClientConfig.tsx # Service config editor (JSON form)
│ ├── ContentTracker.tsx # Articles published this month vs target
│ ├── BacklinkTracker.tsx # Links ordered vs delivered vs live vs indexed
│ ├── AhrefsMetrics.tsx # DR history, referring domains, top pages
│ ├── ReportHistory.tsx # Past reports with resend option
│ └── BillingHistory.tsx # Invoices, payments, margin per order
├── BacklinkOrders.tsx # All orders across clients
│ ├── OrderList.tsx # Filterable by client, status, package
│ └── OrderDetail.tsx # Individual order with link-by-link tracking
├── IndexingDashboard.tsx # Backlink indexing status across all clients
└── ReportGenerator.tsx # Generate report for any client + period
```
**Sidebar addition** (admin-only, behind `managed_services_enabled` feature flag):
```
ADMIN
├── Sector Templates
└── Managed Services (NEW)
```
### Client-Facing View (Managed Service Subscribers)
For managed clients who also have IGNY8 SaaS access:
**Frontend Pages (.tsx + Zustand):**
```
frontend/src/pages/ManagedService/
├── ServiceOverview.tsx # What's included, current month progress
├── ReportViewer.tsx # View/download past reports
└── ApprovalQueue.tsx # Approve blueprint, review content (if review_required)
```
**Sidebar addition** (only shows if `ManagedServiceSubscription` exists for account):
```
ACCOUNT
├── Account Settings
├── Plans & Billing
├── Managed Service (NEW — conditional)
├── Usage
└── AI Models
```
---
## 3. DATA MODELS & APIS
### New Models (in `services` app)
**`ServiceReport`** (extends `AccountBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `site` | ForeignKey(Site) | Report target site |
| `managed_subscription` | ForeignKey(ManagedServiceSubscription, null=True) | Linked subscription |
| `report_type` | CharField(max_length=20) | `weekly` / `monthly` / `quarterly` |
| `period_start` | DateField | Report period start |
| `period_end` | DateField | Report period end |
| `report_data` | JSONField | All metrics for period (structured by section) |
| `report_pdf_url` | URLField(blank=True) | Generated PDF in S3 |
| `is_white_label` | BooleanField(default=False) | Use agency branding |
| `branding` | ForeignKey(AgencyBranding, null=True) | Agency branding reference |
| `sections_included` | JSONField | List of included section keys |
| `generated_at` | DateTimeField(auto_now_add=True) | Generation timestamp |
| `sent_at` | DateTimeField(null=True) | Email sent timestamp |
| `sent_to` | JSONField(default=list) | List of recipient emails |
Table: `igny8_service_reports`
**`AgencyBranding`** (extends `AccountBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `brand_name` | CharField(max_length=200) | Agency name |
| `brand_logo_url` | URLField(blank=True) | Logo URL |
| `primary_color` | CharField(max_length=7) | Hex color code |
| `secondary_color` | CharField(max_length=7) | Hex color code |
| `contact_email` | EmailField(blank=True) | Agency contact email |
| `contact_phone` | CharField(blank=True) | Agency phone |
| `website_url` | URLField(blank=True) | Agency website |
| `footer_text` | TextField(blank=True) | Custom footer text |
| `is_active` | BooleanField(default=True) | Active flag |
Table: `igny8_agency_branding`
**`ReportTemplate`** (extends `AccountBaseModel`)
| Field | Type | Description |
|-------|------|-------------|
| `name` | CharField(max_length=200) | Template name |
| `template_html` | TextField | Base HTML template |
| `template_css` | TextField | Custom CSS |
| `is_default` | BooleanField(default=False) | System default |
| `branding` | ForeignKey(AgencyBranding, null=True) | Agency-specific template |
Table: `igny8_report_templates`
### API Endpoints
All under `/api/v1/`:
**Service Reports:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/reports/` | List (admin + client's own) |
| POST | `/services/reports/generate/` | Generate report for site + period |
| GET | `/services/reports/{id}/` | Detail + PDF URL |
| POST | `/services/reports/{id}/send/` | Email to recipients |
| POST | `/services/reports/{id}/regenerate/` | Regenerate PDF |
| GET | `/services/reports/preview/` | Preview with data (no save) |
**Agency Branding:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/branding/` | List brandings |
| POST | `/services/branding/` | Create branding |
| GET | `/services/branding/{id}/` | Detail |
| PUT | `/services/branding/{id}/` | Update |
| DELETE | `/services/branding/{id}/` | Delete |
**Report Templates:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/report-templates/` | List templates |
| POST | `/services/report-templates/` | Create custom template |
| PUT | `/services/report-templates/{id}/` | Update template |
**Dashboard (Admin):**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/services/dashboard/` | Revenue, clients, MRR |
| GET | `/services/dashboard/overdue/` | Overdue deliverables |
| GET | `/services/dashboard/margin/` | Margin tracking |
| GET | `/services/dashboard/clients/` | Client list with status summary |
### Celery Tasks
| Task | Schedule | Purpose |
|------|----------|---------|
| `generate_weekly_reports` | Every Monday 6am | Generate for all Pro clients |
| `generate_monthly_reports` | 1st of month | Generate for all Lite + Pro clients |
| `send_pending_reports` | Daily | Email any generated but unsent reports |
| `check_overdue_deliverables` | Daily | Flag clients behind on content/links |
| `collect_report_metrics` | Nightly | Pre-aggregate metrics for faster generation |
### Services
| Service | Purpose |
|---------|---------|
| `ReportService` | `generate_report()` — collect metrics, render PDF |
| `ReportEmailService` | Send report emails with PDF attachment or link |
### New Python Dependencies
- `weasyprint` — HTML → PDF rendering
- `matplotlib` — chart generation (server-side SVG/PNG)
---
## 4. IMPLEMENTATION STEPS
### Build Sequence
**Week 1: Models + Report Engine**
1. Create `ServiceReport`, `AgencyBranding`, `ReportTemplate` models + migration
2. Create serializers + viewsets
3. Build `ReportService` — metric collection from all 9 data sources
4. Build PDF template (base HTML + CSS) with section rendering
5. Integrate WeasyPrint for HTML → PDF conversion
6. Build chart generation via matplotlib (DR trend, traffic chart, etc.)
**Week 2: White-Label + Dashboard**
1. Build `AgencyBranding` CRUD endpoints
2. Implement branding injection in report templates
3. Build admin dashboard API endpoints
4. Build Celery tasks for scheduled report generation
**Week 3: Frontend**
1. Build `ServicesDashboard.tsx` + child components
2. Build `ClientDetail.tsx` + child components
3. Build `BacklinkOrders.tsx` + `IndexingDashboard.tsx`
4. Build `ReportGenerator.tsx`
5. Build client-facing `ManagedService/` pages
6. Add sidebar items (admin + client conditional)
### Zustand Stores
```typescript
// frontend/src/stores/servicesStore.ts
interface ServicesState {
subscriptions: ManagedServiceSubscription[]
reports: ServiceReport[]
orders: BacklinkServiceOrder[]
dashboard: DashboardSummary | null
loading: boolean
fetchSubscriptions: () => Promise<void>
fetchReports: (siteId: number) => Promise<void>
generateReport: (params: ReportParams) => Promise<void>
fetchDashboard: () => Promise<void>
}
// frontend/src/stores/brandingStore.ts
interface BrandingState {
brandings: AgencyBranding[]
templates: ReportTemplate[]
fetchBrandings: () => Promise<void>
createBranding: (data: AgencyBrandingInput) => Promise<void>
updateBranding: (id: number, data: AgencyBrandingInput) => Promise<void>
}
```
---
## 5. ACCEPTANCE CRITERIA
### Report Generation
- [ ] Reports generate with data from all 9 sections when data available
- [ ] Sections configurable per client — excluded sections omitted from PDF
- [ ] PDF renders correctly with charts, tables, and branding
- [ ] WeasyPrint produces valid PDF files uploadable to S3
- [ ] Charts render via matplotlib (DR trend, traffic chart, ranking distribution)
- [ ] Generated PDF URL saved in `ServiceReport.report_pdf_url`
### Scheduled Reports
- [ ] Weekly reports auto-generate on Monday 6am for all Pro clients
- [ ] Monthly reports auto-generate on 1st for all Lite + Pro clients
- [ ] `send_pending_reports` emails generated-but-unsent reports
- [ ] Pre-aggregation task collects metrics nightly for faster generation
### White-Label Branding
- [ ] `AgencyBranding` stores name, logo, colors, contact info
- [ ] When `is_white_label = True`, "IGNY8"/"Alorig" replaced with `brand_name`
- [ ] Logo, colors, and footer text swapped in PDF output
- [ ] Multiple brandings supported per account
### Admin Dashboard
- [ ] `ServicesDashboard.tsx` shows all managed clients, MRR, revenue
- [ ] `ClientDetail.tsx` shows config, content tracker, backlink tracker, reports, billing
- [ ] `BacklinkOrders.tsx` lists all orders filterable by client, status, package
- [ ] `IndexingDashboard.tsx` shows indexing status across all clients
- [ ] `ReportGenerator.tsx` generates on-demand report for any client + period
- [ ] Sidebar shows "Managed Services" link only for admin users
### Client-Facing View
- [ ] `ServiceOverview.tsx` shows tier, articles progress, current month status
- [ ] `ReportViewer.tsx` lists past reports with download links
- [ ] `ApprovalQueue.tsx` shows pending blueprint/content approvals
- [ ] Sidebar shows "Managed Service" only when subscription exists
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
1. Read 04A (Managed Services) — subscription, order, Ahrefs, indexing models
2. Read 02C (GSC Integration) — GSCMetricsCache, URLInspectionRecord for report sections
3. Read 02E (Linker External) — SAGBacklink, CampaignKPISnapshot for backlink section
4. Read 02H (Socializer) — SocialPost, SocialEngagement for social section
5. Read 01G (SAG Health Monitoring) — health score for SAG section
6. Read existing billing app — understand existing admin sidebar structure
### Execution Order
```
Models + migrations → ReportService → PDF template + WeasyPrint →
AgencyBranding CRUD → Celery tasks → Admin dashboard frontend → Client frontend
```
### Critical Rules
1. **All IDs are integers** — BigAutoField PKs
2. **Model names PLURAL** — Clusters, Keywords, etc.
3. **Table prefix** — `igny8_` for all tables
4. **App** — models go in `services` app (same as 04A)
5. **Frontend** — `.tsx` files, Zustand stores, in `frontend/src/pages/Services/` and `frontend/src/pages/ManagedService/`
6. **Feature flag** — admin dashboard behind `managed_services_enabled` flag
7. **Report data** — always query by site + date range, never expose cross-account data
8. **PDF security** — generated PDFs accessible only to account owner or admin
9. **Celery app** — `igny8_core` for all tasks
10. **New dependencies** — `weasyprint`, `matplotlib` added to `requirements.txt`
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 04A Managed Services | Subscription + order data feeds all reports |
| 02C GSC Integration | GSCMetricsCache → sections 3 (Indexing), 4 (Rankings), 5 (Traffic) |
| 02E Linker External | SAGBacklink + CampaignKPISnapshot → section 6 (Backlinks) |
| 02H Socializer | SocialPost + SocialEngagement → section 7 (Social Media) |
| 01G SAG Health Monitoring | Health score → section 8 (SAG Health) |
| 04C Pricing Launch | Feature gates control who can access reports + white-label |

576
v2/V2-Execution-Docs/04C-pricing-launch.md Normal file
View File

@@ -0,0 +1,576 @@
# IGNY8 Phase 4: Pricing & Launch (04C)
## Feature Gating, Plan Matrix, WP.org Submission, Go-to-Market, Migration
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 4 — Business Layer
**Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Frontend Developers, Product
---
## 1. CURRENT STATE
### Current Plan Structure
- 4 plans live: Free ($0), Starter ($49), Growth ($149), Scale ($349)
- Stripe + PayPal integrated for payment processing
- Plan model exists in `billing` app with basic feature limits (sites, credits)
- Feature gating is minimal — no granular per-feature capability matrix
- No `plan_features` JSONField on Plan
### No WordPress.org Plugin Submission
- WP plugin built in 03A but not submitted to WP.org directory
- No SVN deploy pipeline configured
- No readme.txt in WP.org format
- No plugin banner/icon assets
### No Go-to-Market Strategy Implemented
- Current clients are direct Alorig relationships
- No organic acquisition funnel from WP.org
- No marketing automation for plan upgrade nudges
- No add-on purchase flow for managed services
### Phase 4A + 4B Complete
- `ManagedServiceSubscription` model (04A) — tiers, pricing, monthly targets
- `BacklinkServiceOrder` model (04A) — packages, pricing, margin tracking
- `ServiceReport` model (04B) — automated reports with PDF generation
- `AgencyBranding` model (04B) — white-label branding config
- All Phase 2 features built — each needs plan-level gating
---
## 2. WHAT TO BUILD
### Updated Plan Feature Matrix
Core SaaS plan prices unchanged. All V2 features gated by plan level:
| Feature | Free | Starter ($49) | Growth ($149) | Scale ($349) |
|---------|------|---------------|---------------|--------------|
| Sites | 1 | 3 | 10 | Unlimited |
| Credits/month | 100 | 1,000 | 5,000 | 25,000 |
| SAG Blueprint | Quick mode only | Quick + Detailed | Full | Full |
| Content Types | Post only | Post + Page | All types | All types |
| Taxonomy Content (02B) | — | — | Yes | Yes |
| GSC Integration (02C) | — | Basic (analytics) | Full (+ indexing) | Full |
| Internal Linker (02D) | — | Audit only | Audit + auto-insert | Full + remediation |
| External Backlinks (02E) | — | — | Self-service | Self-service + API |
| Optimizer (02F) | — | Basic analysis | Full rewrite | Full + batch |
| Rich Schema (02G) | Basic (Article) | 5 types | All 10 types | All + retroactive |
| Socializer (02H) | — | 2 platforms | All platforms | All + auto-schedule |
| Video Creator (02I) | — | — | Short-form only | All formats |
| Ahrefs Integration (04A) | — | — | Read-only metrics | Full (verify + prospect) |
| Backlink Indexing (04A) | — | — | — | Included |
| Reports (04B) | — | — | Monthly PDF | Weekly + custom |
| White-Label (04B) | — | — | — | Yes |
| API Access | — | — | Read-only | Full CRUD |
| Managed Services (04A) | — | — | Can purchase Lite | Can purchase Lite/Pro |
### Feature Flag System
**Approach:** Add `plan_features` JSONField on existing `Plan` model.
**JSON structure per plan:**
```json
{
"sag_mode": "quick",
"content_types": ["post"],
"taxonomy_content": false,
"gsc_level": "none",
"linker_level": "none",
"backlinks_level": "none",
"optimizer_level": "none",
"schema_types": 0,
"socializer_platforms": 0,
"video_level": "none",
"ahrefs_level": "none",
"backlink_indexing": false,
"report_level": "none",
"white_label": false,
"api_access": "none",
"managed_services": "none"
}
```
**Feature level hierarchies:**
```python
LEVEL_HIERARCHY = {
"sag_mode": ["quick", "detailed", "full"],
"gsc_level": ["none", "basic", "full"],
"linker_level": ["none", "audit", "auto", "full"],
"backlinks_level": ["none", "self_service", "self_service_api"],
"optimizer_level": ["none", "basic", "full", "batch"],
"video_level": ["none", "short", "all"],
"ahrefs_level": ["none", "readonly", "full"],
"report_level": ["none", "monthly", "weekly_custom"],
"api_access": ["none", "readonly", "full"],
"managed_services": ["none", "lite", "lite_pro"],
}
```
**Feature gate checking pattern:**
```python
class FeatureGateService:
@staticmethod
def check_feature(account, feature, required_level):
plan = account.subscription.plan
features = plan.plan_features or {}
current = features.get(feature, "none")
hierarchy = LEVEL_HIERARCHY.get(feature)
if hierarchy is None:
# Boolean or list feature — direct comparison
return bool(current)
return hierarchy.index(current) >= hierarchy.index(required_level)
```
**Decorator for ViewSets:**
```python
def require_feature(feature, required_level):
def decorator(view_func):
@wraps(view_func)
def wrapper(self, request, *args, **kwargs):
account = request.user.account
if not FeatureGateService.check_feature(account, feature, required_level):
return Response(
{"detail": f"Feature '{feature}' requires plan upgrade."},
status=403
)
FeatureGateService.log_usage(account, feature, required_level, True, request.path)
return view_func(self, request, *args, **kwargs)
return wrapper
return decorator
```
**Usage example:**
```python
class InternalLinkViewSet(viewsets.ModelViewSet):
@require_feature('linker_level', 'auto')
def auto_insert(self, request):
...
```
### Managed Services Add-On Pricing (in Billing Flow)
| Add-On | Price | Available To |
|--------|-------|-------------|
| Managed Lite | $100/site/month | Growth + Scale plans |
| Managed Pro | $399/site/month | Scale plan only |
| Backlink Package (Starter) | $800–$1,500/month | Growth + Scale |
| Backlink Package (Growth) | $2,000–$4,000/month | Growth + Scale |
| Backlink Package (Authority) | $4,000–$8,000/month | Scale only |
| Indexing Boost (standalone) | $0.15/URL | Growth + Scale |
Add-ons are purchased via existing Stripe/PayPal subscription modification. Each add-on creates related model records:
- Managed services → `ManagedServiceSubscription` (04A)
- Backlink packages → `BacklinkServiceOrder` (04A)
- Indexing boost → credit balance increase for indexing operations
### WordPress.org Plugin Submission
**Plugin identity:** `igny8-seo` (standalone plugin from 03A, rebranded for WP.org)
**Version:** 2.0.0
**License:** GPL v2+
**Submission checklist (10 items):**
| # | Requirement | Details |
|---|------------|---------|
| 1 | Input sanitization | All user inputs sanitized with `sanitize_text_field()`, `intval()`, `wp_kses_post()` |
| 2 | Output escaping | All outputs escaped with `esc_html()`, `esc_attr()`, `esc_url()`, `wp_kses_post()` |
| 3 | Nonce verification | All form submissions verified with `wp_verify_nonce()` / `check_admin_referer()` |
| 4 | No unauthorized external calls | Phone-home disabled by default, Settings API toggle for optional usage analytics |
| 5 | GPL v2+ license | `LICENSE` file in plugin root, `License: GPLv2 or later` in plugin header |
| 6 | `readme.txt` | WP.org formatted: short description, installation, FAQ, changelog, screenshots |
| 7 | Assets | Banner 1544×500 + 772×250, icon 256×256 + 128×128, 8+ screenshots |
| 8 | Translation-ready | All strings in `__()`, `_e()`, `esc_html__()` with `igny8` text domain |
| 9 | No minified JS without source | Unminified versions included alongside minified |
| 10 | Security hardening | `ABSPATH` check at top of every PHP file, no `eval()`, no `file_get_contents()` for remote |
**Privacy compliance:**
- No tracking without consent
- No storing user IPs (hash only if needed)
- Privacy policy section in readme.txt
**Post-approval setup:**
- SVN deploy pipeline: GitHub tag → GitHub Actions → WP.org SVN
- Auto-deploy on version tag push
- Update checker: `/api/v1/plugins/igny8-seo/check-update/` for connected users
**Review timeline:** Typically 1–4 weeks for initial WP.org review.
### Go-to-Market Strategy
**5 Target Audiences:**
| # | Audience | How They Find Us | Conversion Path |
|---|----------|-----------------|----------------|
| 1 | WordPress site owners | WP.org plugin directory search | Free plugin → Setup Wizard → Free Plan → Upgrade |
| 2 | SEO agencies | Direct outreach, white-label pitch | Demo → Managed Pro → Backlink packages |
| 3 | Content creators/bloggers | WP.org + SEO community + content about SAG | Free → Growth for full content pipeline |
| 4 | eCommerce stores | WP.org (WooCommerce integration mention) | Free → Growth for product schema + services |
| 5 | Local businesses | WP.org (LocalBusiness schema, service areas) | Free → Starter for GSC + schema |
**Acquisition Funnel:**
```
WP.org Plugin (free) → Install → Setup Wizard →
→ Free Plan (100 credits) → Experience value →
→ Upgrade to Starter/Growth/Scale →
→ Optional: Purchase Managed Services add-on →
→ Optional: Purchase Backlink Packages
```
### Launch Sequence
| Step | Action | Timing |
|------|--------|--------|
| 1 | Enable `managed_services_enabled` flag in staging | Week 1 |
| 2 | Onboard 2–3 existing Alorig clients as beta managed service subscribers | Week 1–2 |
| 3 | Test full cycle: onboarding → content → backlinks → indexing → reporting | Week 2–3 |
| 4 | Fix issues from beta testing | Week 3 |
| 5 | Submit WP.org plugin (parallel with beta testing) | Week 2 |
| 6 | Enable feature flags in production | Week 3–4 |
| 7 | Update pricing page on marketing site | Week 4 |
| 8 | Send announcement email to existing users | Week 4 |
| 9 | Begin WP.org organic acquisition (post-approval) | Ongoing |
### Existing User Migration
**Principle:** Nothing existing breaks. New feature gates apply to new features only.
**Migration steps:**
1. Run one-time `migrate_plan_features` Celery task
2. Populate `plan_features` JSONField on all 4 existing Plan records with appropriate values
3. Existing users retain current plan pricing (grandfathered)
4. Existing credits + billing cycles unchanged
5. Users gain access to new V2 features according to their plan level
**Migration data per plan:**
```python
PLAN_FEATURE_DEFAULTS = {
"free": {
"sag_mode": "quick",
"content_types": ["post"],
"taxonomy_content": False,
"gsc_level": "none",
"linker_level": "none",
"backlinks_level": "none",
"optimizer_level": "none",
"schema_types": 0,
"socializer_platforms": 0,
"video_level": "none",
"ahrefs_level": "none",
"backlink_indexing": False,
"report_level": "none",
"white_label": False,
"api_access": "none",
"managed_services": "none",
},
"starter": {
"sag_mode": "detailed",
"content_types": ["post", "page"],
"taxonomy_content": False,
"gsc_level": "basic",
"linker_level": "audit",
"backlinks_level": "none",
"optimizer_level": "basic",
"schema_types": 5,
"socializer_platforms": 2,
"video_level": "none",
"ahrefs_level": "none",
"backlink_indexing": False,
"report_level": "none",
"white_label": False,
"api_access": "none",
"managed_services": "none",
},
"growth": {
"sag_mode": "full",
"content_types": "all",
"taxonomy_content": True,
"gsc_level": "full",
"linker_level": "auto",
"backlinks_level": "self_service",
"optimizer_level": "full",
"schema_types": 10,
"socializer_platforms": "all",
"video_level": "short",
"ahrefs_level": "readonly",
"backlink_indexing": False,
"report_level": "monthly",
"white_label": False,
"api_access": "readonly",
"managed_services": "lite",
},
"scale": {
"sag_mode": "full",
"content_types": "all",
"taxonomy_content": True,
"gsc_level": "full",
"linker_level": "full",
"backlinks_level": "self_service_api",
"optimizer_level": "batch",
"schema_types": "all_retroactive",
"socializer_platforms": "all_auto",
"video_level": "all",
"ahrefs_level": "full",
"backlink_indexing": True,
"report_level": "weekly_custom",
"white_label": True,
"api_access": "full",
"managed_services": "lite_pro",
},
}
```
---
## 3. DATA MODELS & APIS
### Modified Models
**`Plan` (billing app)** — add field:
| Field | Type | Description |
|-------|------|-------------|
| `plan_features` | JSONField(default=dict) | Feature capability matrix per plan |
No new table — existing `igny8_plans` table gets new column.
### New Models
**`FeatureUsageLog`** (extends `AccountBaseModel`, billing app)
| Field | Type | Description |
|-------|------|-------------|
| `feature` | CharField(max_length=50) | Feature key checked (e.g., `linker_level`) |
| `required_level` | CharField(max_length=30) | Level that was required |
| `current_level` | CharField(max_length=30) | Level user's plan has |
| `was_allowed` | BooleanField | Whether access was granted |
| `endpoint` | CharField(max_length=200) | API endpoint that triggered the check |
| `timestamp` | DateTimeField(auto_now_add=True) | When the check happened |
Table: `igny8_feature_usage_log`
Purpose: Track which features users hit gates on → inform pricing decisions and identify upgrade opportunities.
### API Endpoints
All under `/api/v1/`:
**Feature Gates:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/billing/features/` | Current account's available features |
| GET | `/billing/features/check/{feature}/` | Check specific feature access |
**Plan Management (admin only):**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/billing/plans/` | List all plans with features |
| PUT | `/billing/plans/{id}/features/` | Update plan feature matrix |
**Add-On Purchases:**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/billing/add-ons/` | List available add-ons for current plan |
| POST | `/billing/add-ons/managed-service/` | Purchase managed service add-on |
| POST | `/billing/add-ons/backlink-package/` | Purchase backlink package |
| POST | `/billing/add-ons/indexing-boost/` | Purchase indexing credits |
**WP.org Plugin (public, no auth required):**
| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/plugins/igny8-seo/info/` | Plugin info for WP update checker |
| GET | `/plugins/igny8-seo/download/` | Download latest ZIP |
| GET | `/plugins/igny8-seo/check-update/` | Version check endpoint |
### Celery Tasks
| Task | Schedule | Purpose |
|------|----------|---------|
| `migrate_plan_features` | One-time | Populate `plan_features` on existing Plan records |
| `sync_wporg_stats` | Daily | Fetch WP.org download/rating stats |
| `feature_usage_report` | Weekly | Aggregate FeatureUsageLog for product decisions |
### Services
| Service | Purpose |
|---------|---------|
| `FeatureGateService` | `check_feature()`, `get_plan_features()`, `log_usage()` |
| `AddOnPurchaseService` | Process add-on purchases via existing Stripe/PayPal integration |
| `WPOrgDeployService` | SVN deploy pipeline config for WP.org (GitHub Actions) |
---
## 4. IMPLEMENTATION STEPS
### Build Sequence
**Week 1: Feature Gate System**
1. Add `plan_features` JSONField to `Plan` model + migration
2. Create `FeatureUsageLog` model + migration
3. Build `FeatureGateService` with `check_feature()`, `log_usage()`
4. Build `@require_feature()` decorator
5. Create feature check API endpoints (`/billing/features/`)
6. Write `migrate_plan_features` one-time task with `PLAN_FEATURE_DEFAULTS`
**Week 2: Plan Feature Integration**
1. Apply `@require_feature()` decorator to all Phase 2 module ViewSets:
- 02B taxonomy content endpoints → `taxonomy_content`
- 02C GSC endpoints → `gsc_level`
- 02D internal linker endpoints → `linker_level`
- 02E external backlinks endpoints → `backlinks_level`
- 02F optimizer endpoints → `optimizer_level`
- 02G schema endpoints → `schema_types`
- 02H socializer endpoints → `socializer_platforms`
- 02I video endpoints → `video_level`
- 04A Ahrefs endpoints → `ahrefs_level`
- 04A indexing endpoints → `backlink_indexing`
- 04B report endpoints → `report_level`
- 04B white-label endpoints → `white_label`
2. Build add-on purchase endpoints using existing Stripe/PayPal
3. Build plan admin feature update endpoint
**Week 3: WP.org Submission + Launch**
1. Build WP.org plugin info/download/check-update public endpoints
2. Prepare plugin for WP.org: readme.txt, assets, security audit
3. Set up GitHub Actions → WP.org SVN deploy pipeline
4. Submit plugin to WP.org review queue
5. Run `migrate_plan_features` on production
6. Update marketing site pricing page
### Frontend Changes
**Billing pages (existing):**
- Update plan comparison table to show all V2 features
- Add "Feature Unavailable" overlay when gated feature accessed on lower plan
- Add upgrade CTA buttons inline with gated features
**Add-on purchase flow:**
```
frontend/src/pages/Billing/
├── AddOns.tsx # Available add-ons for current plan
│ ├── ManagedServiceCard.tsx # Lite/Pro tier selection → purchase
│ ├── BacklinkPackageCard.tsx # Package selection → purchase
│ └── IndexingBoostCard.tsx # Quantity slider → purchase
└── PlanComparison.tsx # Updated with all V2 features
```
**Zustand store addition:**
```typescript
// Extend existing billing store
interface BillingState {
// ... existing fields
planFeatures: PlanFeatures | null
addOns: AddOn[]
fetchPlanFeatures: () => Promise<void>
purchaseAddOn: (type: string, params: AddOnParams) => Promise<void>
checkFeature: (feature: string) => boolean
}
```
**Feature gate UI pattern:**
```tsx
// Reusable component for gated features
const FeatureGate: React.FC<{ feature: string; requiredLevel: string }> = ({
feature, requiredLevel, children
}) => {
const { checkFeature } = useBillingStore()
if (!checkFeature(feature)) {
return <UpgradePrompt feature={feature} requiredLevel={requiredLevel} />
}
return <>{children}</>
}
```
---
## 5. ACCEPTANCE CRITERIA
### Feature Gate System
- [ ] `plan_features` JSONField added to `Plan` model
- [ ] `FeatureGateService.check_feature()` correctly evaluates level hierarchy
- [ ] `@require_feature()` decorator returns 403 with upgrade message for insufficient plan
- [ ] `FeatureUsageLog` records all gate checks (allowed and denied)
- [ ] All Phase 2 module ViewSets decorated with appropriate feature gates
- [ ] `/billing/features/` returns current account's feature matrix
- [ ] `/billing/features/check/{feature}/` returns allowed/denied for specific feature
### Plan Feature Defaults
- [ ] Free plan: quick SAG, post only, no V2 features
- [ ] Starter plan: detailed SAG, post + page, basic GSC, audit linker, basic optimizer, 5 schema types, 2 social platforms
- [ ] Growth plan: full SAG, all content types, full GSC, auto linker, full optimizer, all schema, all social, short video, readonly Ahrefs, monthly reports, readonly API, can purchase Lite managed
- [ ] Scale plan: everything at maximum level including white-label, weekly reports, full API, both managed tiers
### Add-On Purchases
- [ ] Add-on availability respects plan level (e.g., Authority package only for Scale)
- [ ] Purchase creates appropriate model records (ManagedServiceSubscription, BacklinkServiceOrder)
- [ ] Stripe/PayPal subscription modification works for recurring add-ons
- [ ] One-time purchases (indexing boost) add credits correctly
### Migration
- [ ] `migrate_plan_features` task populates all 4 existing plans correctly
- [ ] Existing users see no change in accessible features they already had
- [ ] New V2 features available according to plan level after migration
### WP.org Plugin
- [ ] `/plugins/igny8-seo/info/` returns valid plugin info JSON
- [ ] `/plugins/igny8-seo/check-update/` returns latest version for WP update checker
- [ ] `/plugins/igny8-seo/download/` serves latest plugin ZIP
- [ ] readme.txt passes WP.org validator
- [ ] All 10 submission checklist items verified
### Frontend
- [ ] Plan comparison page shows all V2 features by plan
- [ ] `FeatureGate` component shows upgrade prompt for gated features
- [ ] Add-on purchase flow completes end-to-end
- [ ] Feature denied state shows clear upgrade path
---
## 6. CLAUDE CODE INSTRUCTIONS
### Context Requirements
1. Read existing `Plan` model in billing app — understand current fields before adding `plan_features`
2. Read existing billing ViewSets — understand current plan/subscription endpoints
3. Read all Phase 2 execution docs (02B–02I) — identify every ViewSet that needs `@require_feature()`
4. Read 04A (Managed Services) — subscription + order models for add-on purchase integration
5. Read 04B (Whitelabel Reporting) — report + branding endpoints to gate by plan
6. Read 03A (WP Plugin Standalone) — plugin structure for WP.org submission preparation
### Execution Order
```
plan_features migration → FeatureGateService → @require_feature decorator →
Apply to all Phase 2 ViewSets → Add-on purchase endpoints →
WP.org public endpoints → Frontend feature gate component →
migrate_plan_features task → Launch checklist
```
### Critical Rules
1. **All IDs are integers** — BigAutoField PKs
2. **Model names PLURAL** — consistent with existing codebase
3. **Table prefix** — `igny8_` for all tables
4. **Billing app** — `Plan` modification and `FeatureUsageLog` go in existing billing app
5. **Frontend** — `.tsx` files, Zustand stores
6. **Decorator stacks** — `@require_feature()` stacks with existing `@permission_classes` and DRF auth
7. **No breaking changes** — existing Plan records must work before and after migration
8. **Celery app** — `igny8_core` for all tasks
9. **Add-on pricing** — use existing Stripe Product/Price objects, create new ones only for new add-ons
10. **WP.org endpoints** — public (no auth), rate-limited, no sensitive data exposed
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 04A Managed Services | Add-on purchase creates ManagedServiceSubscription + BacklinkServiceOrder |
| 04B Whitelabel Reporting | Report + white-label features gated by plan level |
| 02B Taxonomy Content | Gated by `taxonomy_content` boolean |
| 02C GSC Integration | Gated by `gsc_level` (none/basic/full) |
| 02D Linker Internal | Gated by `linker_level` (none/audit/auto/full) |
| 02E Linker External | Gated by `backlinks_level` (none/self_service/self_service_api) |
| 02F Optimizer | Gated by `optimizer_level` (none/basic/full/batch) |
| 02G Rich Schema | Gated by `schema_types` (0/5/10/all_retroactive) |
| 02H Socializer | Gated by `socializer_platforms` (0/2/all/all_auto) |
| 02I Video Creator | Gated by `video_level` (none/short/all) |
| 03A WP Plugin Standalone | Plugin submitted to WP.org, update checker endpoints |
| Billing App (existing) | Plan model modified, Stripe/PayPal integration for add-ons |

840
v2/V2-Execution-Docs/05-other-apps-deployment.md Normal file
View File

@@ -0,0 +1,840 @@
# Phase 5 — Multi-App Deployment (05)
## 6 Alorig Apps on Shared Infrastructure
**Source of Truth:** Alorig-Master-Plan-v1.md + individual app blueprints
**Document Version:** 1.0
**Date:** 2026-03-23
**Phase:** IGNY8 Phase 5 — Multi-App Deployment
**Status:** Build Ready
**Audience:** Claude Code, DevOps, Backend Developers, Frontend Developers
---
## 1. CURRENT STATE
### Shared Infrastructure Operational (Phase 0)
- **VPS:** Hostinger KVM 4 — 4 vCPU, 16GB RAM, 200GB NVMe
- **Docker Network:** `alorig_net` (external, all apps join)
- **Shared Services** (`docker-compose.infra.yml`):
- `alorig_postgres` — PostgreSQL 16, separate database per app
- `alorig_redis` — Redis 7, app-specific key prefixes + DB numbers
- `alorig_caddy` — Caddy 2, reverse proxy + SSL via Cloudflare DNS challenge
- `portainer` — Portainer CE, Docker GUI for 25+ containers
- `flower` — mher/flower, Celery monitoring (on-demand)
- **DNS:** All domains in Cloudflare (free tier), wildcard A records to VPS IP
### IGNY8 Production + Staging Running
- IGNY8 Phase 0–4 complete and stable
- Containers: `igny8_backend`, `igny8_frontend`, `igny8_celery_worker`, `igny8_celery_beat`, `igny8_celery_video_worker`, `flower`
- RAM usage: ~1.8 GB
- Ports: 8011 (prod), 8031 (staging)
- Redis DBs: 0 (prod), 1 (staging)
### 6 Apps Awaiting Deployment
All apps have blueprints finalized. None have GitHub repos or containers yet.
### Resource Budget
| Component | RAM Estimate |
|-----------|-------------|
| Shared infra (PG + Redis + Caddy + Portainer) | ~2.5 GB |
| IGNY8 (prod + staging) | ~1.8 GB |
| Psydge | ~1.75 GB |
| Snapify | ~700 MB |
| Observer OS | ~700 MB |
| Alorig Site Builder | ~1.25 GB |
| AstroTiming | ~400 MB |
| ASMS Phase 1 | ~600 MB |
| OS + Docker overhead | ~1.0 GB |
| **TOTAL STEADY STATE** | **~10.7 GB / 16 GB** |
**Headroom:** ~5 GB for spikes. Upgrade trigger: steady-state >13 GB or OOM kills.
---
## 2. WHAT TO BUILD
Deploy 6 apps sequentially on the shared infrastructure. Each app follows the standard Alorig container pattern with app-specific exceptions. Deploy one at a time, monitor RAM after each before proceeding.
### Deployment Order
| # | App | Rationale | Expected RAM |
|---|-----|-----------|-------------|
| 1 | **Psydge** | Most complex (WebSockets, Channels, real-time). Validate server under real load early. | ~1.75 GB |
| 2 | **Snapify** | Standard Django. Validates multi-app pattern. WhatsApp adds external service testing. | ~700 MB |
| 3 | **Alorig Site Builder** | Dual runtime (Django + Payload CMS/Node.js). Tests Node.js container pattern. | ~1.25 GB |
| 4 | **Observer OS** | Standard Django + React. Light resources. Shares ephemeris logic with AstroTiming. | ~700 MB |
| 5 | **AstroTiming** | Lightest app. Shares `pyswisseph` ephemeris data approach with Observer OS. | ~400 MB |
| 6 | **ASMS Phase 1** | School management MVP for 74-school pilot. Basic modules only. | ~600 MB |
### Shared Tech Stack (All Apps)
| Layer | Technology | Version |
|-------|-----------|---------|
| Backend | Django | 5.1–5.2+ |
| API | Django REST Framework | 3.15+ |
| Database | PostgreSQL | 16 (shared instance, separate DB per app) |
| Cache/Broker | Redis | 7 (shared instance, separate DB per app) |
| Task Queue | Celery | 5.3+ |
| Frontend | React + TypeScript | React 19, TS 5+ |
| State Management | Zustand | 4–5 |
| Styling | Tailwind CSS | 3–4 |
| Build | Vite | 5–6 |
| Containers | Docker + Docker Compose | External network (`alorig_net`) |
| Reverse Proxy | Caddy 2 | Cloudflare DNS challenge SSL |
---
## 3. SHARED DEPLOYMENT PATTERN
### Standard Per-App Container Set (3 containers)
| Container | Role |
|-----------|------|
| `{app}_backend` | Django + Gunicorn (WSGI) |
| `{app}_celery_worker` | Background task processing |
| `{app}_celery_beat` | Scheduled task scheduling |
React frontend built as static files → served via `alorig_caddy` (no separate frontend container).
**Exceptions:**
- Psydge: +1 container (`psydge_channels` for Django Channels/ASGI/WebSocket)
- Site Builder: +1 container (`sitebuilder_payload` for Node.js/Payload CMS)
### Port Assignment Map
| App | Backend Port | Additional Ports |
|-----|-------------|-----------------|
| IGNY8 (prod) | 8011 | — |
| IGNY8 (staging) | 8031 | — |
| Psydge | 8012 | 8013 (WebSocket/Channels) |
| Snapify | 8014 | — |
| Alorig Site Builder | 8016 | 8017 (Payload CMS) |
| Observer OS | 8018 | — |
| AstroTiming | 8020 | — |
| ASMS | 8022 | — |
### Redis DB Assignment Map
| DB | App |
|----|-----|
| 0 | IGNY8 production |
| 1 | IGNY8 staging |
| 2 | Psydge |
| 3 | Snapify |
| 4 | Alorig Site Builder |
| 5 | Observer OS |
| 6 | AstroTiming |
| 7 | ASMS |
### Docker Compose File Naming
```
docker-compose.infra.yml ← Shared infrastructure
docker-compose.igny8.yml ← IGNY8 production
docker-compose.igny8-staging.yml ← IGNY8 staging
docker-compose.psydge.yml ← Psydge
docker-compose.snapify.yml ← Snapify
docker-compose.sitebuilder.yml ← Alorig Site Builder
docker-compose.observer.yml ← Observer OS
docker-compose.astrotiming.yml ← AstroTiming
docker-compose.asms.yml ← ASMS Phase 1
```
All compose files: `networks: { alorig_net: { external: true } }`
---
## 4. APP 1: PSYDGE — AI Trading Discipline Platform
### Product Summary
Psydge (Psychological Edge) — web-based trading discipline and execution platform. Sits between trader and MT4/MT5 broker via MetaApi, enforcing hard-coded discipline rules that structurally prevent self-destructive trading. Two products: Psydge Core (discipline engine) + Vista Strategies (pre-defined strategy suite).
### Tech Stack Additions
| Addition | Technology | Purpose |
|----------|-----------|---------|
| Real-Time | Django Channels + Redis | WebSocket: live prices, position updates, gate status, AI messages |
| Broker Bridge | MetaApi (Cloud) | MT4/MT5 connectivity, order execution, position polling |
| AI | Claude API (Anthropic) | Trade supervision, behavioral prediction, pre-entry validation, session debriefs |
| Charts | TradingView Lightweight Charts | Live candlestick charts, M1–D1 timeframes |
| Billing | Stripe | Subscription (Psydge Core + Vista Strategies tiers) |
### Container Architecture (4 containers — exception)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `psydge_backend` | psydge-backend:latest | Django + Gunicorn (WSGI for REST API) | 8012 |
| `psydge_channels` | psydge-backend:latest | Django + Daphne (ASGI for WebSocket) | 8013 |
| `psydge_celery_worker` | psydge-backend:latest | Celery worker (AI calls, behavioral analysis, scoring) | — |
| `psydge_celery_beat` | psydge-backend:latest | Celery beat scheduler | — |
### Database
- Database: `psydge_db` on `alorig_postgres`
- Redis prefix: `psydge:`, Redis DB: 2
### Core Data Models
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| User | email, timezone, broker_credentials (encrypted), subscription_tier | User account |
| BrokerConnection | user_id, broker, server, account_number, connection_status | MT4/MT5 connection state |
| DisciplineConfig | user_id, loss_lock_r, profit_lock_r, cooldown_hours, max_risk_pct, session_hours, symbol_whitelist | Per-user discipline parameters |
| GateState | Redis hash: user_id → 6 buffer values | Real-time gate buffers (sub-ms reads) |
| Trade | user_id, symbol, direction, entry, sl, tp, lot, risk_pct, r_result, strategy, status, opened_at, closed_at | Complete trade record |
| Violation | user_id, violation_code, attempted_action, gate_state_snapshot | Every blocked trade attempt |
| DisciplineScore | user_id, date, score, metric_breakdown (JSON) | Daily discipline score history |
| ReflectionEntry | user_id, date, session_notes, ai_prompt, ai_summary | Session journal entries |
| AIInteraction | user_id, trade_id, role, input_context, output | AI call history for audit |
| BehavioralPattern | user_id, pattern_type, occurrence_count, last_instance, trend | Detected violation patterns |
### Redis State Schema
| Key Pattern | Type | TTL | Content |
|-------------|------|-----|---------|
| `psydge:gate:{user_id}` | Hash | Persistent | All 6 buffer values |
| `psydge:cooldown:{user_id}` | String | Cooldown duration | Cooldown expiry timestamp |
| `psydge:session_r:{user_id}` | String | End of day | Cumulative R for session |
| `psydge:positions:{user_id}` | Hash | Live sync | Open positions from MetaApi |
| `psydge:prices:{symbol}` | String | 5 seconds | Latest bid/ask |
| `psydge:lock:{user_id}` | String | End of session | Session lock flag |
### Discipline Engine
- **6 Gate Buffers:** GO Signal (B), Symbol+TF Whitelist (C), Session Filter (D), R-Based Equity Guard (E), Cooldown Timer (H), Risk % Check (K)
- **Locked Rules:** Loss Lock (−2R daily), Profit Lock (+3R daily), 4hr cooldown, 1% risk/trade, London+NY sessions only, web panel only execution
- **Violation Levels:** L1 Alert → L2 Overlay → L3 Blackout → L4 Session Lock
- **Discipline Score (0–100):** Rule Compliance 25%, Session Discipline 15%, Risk Management 15%, Patience 15%, Post-Win Control 15%, Cooldown Respect 10%, Revenge Prevention 5%
### 4 AI Roles (Claude API + Celery)
| Role | Trigger | Output |
|------|---------|--------|
| Trade Supervisor | Trade at TP1 or R threshold | scale / hold / tighten_sl / exit |
| Pre-Entry Validator | User initiates trade (if enabled) | approve / stand_down + reasoning |
| Behavioral Predictor | Session start + post-trade | Risk alerts + enforcement tightening |
| Session Debrief | Session end | Natural language summary + insights |
### Vista Strategy Suite (4 Tiers)
| Strategy | Timeframe | Target R |
|----------|-----------|----------|
| Scalper Engine | M1–M5 | 1.5–2R |
| Intraday Momentum | M15–H1 | 2–7R |
| Swing Position | H1–D1 | 5–12R |
| Vault Trail | H4–D1 | 15–20R |
6 proprietary indicators: NDNS, SV, TM, MOM, FHS, HAMFIST.
### Celery Tasks
| Task | Schedule |
|------|----------|
| `calculate_discipline_score` | End of each trading session |
| `detect_behavioral_patterns` | After every violation |
| `ai_session_debrief` | Session end |
| `sync_metaapi_positions` | Every 5 seconds (per connected user) |
| `weekly_behavioral_report` | Sunday 23:00 UTC |
### Caddy Routes
```
psydge.com → static React build (frontend)
api.psydge.com → psydge_backend:8012 (REST API)
ws.psydge.com → psydge_channels:8013 (WebSocket)
```
### Implementation Steps
1. Create GitHub repo `psydge` with Django project scaffold
2. Configure `docker-compose.psydge.yml` (4 containers on `alorig_net`)
3. Create `psydge_db` on shared PostgreSQL
4. Build Django models, migrations, admin
5. Build Discipline Engine (gate buffers, violation logging, enforcer)
6. Integrate MetaApi SDK (broker connection, order flow, position sync)
7. Build WebSocket consumers (Django Channels) for real-time data
8. Build Vista strategy engine (indicator calculations, TM scoring)
9. Integrate Claude API for 4 AI roles via Celery tasks
10. Build React frontend (trade panel, analytics dashboard, settings)
11. Configure Caddy routes for psydge.com, api.psydge.com, ws.psydge.com
12. Build Stripe subscription integration
13. Deploy, smoke test with paper trading account
14. Monitor RAM — expected ~1.75 GB steady state
---
## 5. APP 2: SNAPIFY — Pakistani E-Commerce Platform
### Product Summary
Snapify.pk — mobile-first, AI-powered e-commerce platform for Pakistani micro-sellers to create online stores via WhatsApp. Three-interface architecture: buyer-facing store (server-rendered Django for SEO), seller dashboard (React 19 PWA), admin panel (React 19 SPA). Target: 5M+ informal sellers, breakeven at ~150 paying users.
### Tech Stack Additions
| Addition | Technology | Purpose |
|----------|-----------|---------|
| WhatsApp | WhatsApp Business Cloud API | Onboarding bot, order notifications, seller commands |
| Image CDN | Cloudinary | Image optimization, responsive srcset, WebP |
| File Storage | DigitalOcean Spaces (S3-compatible) | Product images, generated assets |
| Social APIs | Meta Graph API (FB + Instagram) | Auto-posting, story creation for Pro sellers |
| AI Engine | OpenAI GPT-4 + Whisper API | Content generation, image analysis, speech-to-text (Urdu/Punjabi) |
| Email | SendGrid | Transactional emails |
| Search | PostgreSQL Full-Text + pg_trgm | Product search, store discovery |
| Payment | JazzCash / EasyPaisa | Pakistani payment gateways |
### Container Architecture (3 containers — standard)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `snapify_backend` | snapify-backend:latest | Django + Gunicorn (REST API + server-rendered store pages) | 8014 |
| `snapify_celery_worker` | snapify-backend:latest | Celery worker (AI, images, social posting, SEO gen) | — |
| `snapify_celery_beat` | snapify-backend:latest | Celery beat (sitemap regen, hub pages, analytics) | — |
**Note:** Buyer-facing store is server-rendered Django templates (NOT React) — critical for SEO on 3G connections.
### Database
- Database: `snapify_db` on `alorig_postgres`
- Redis prefix: `snapify:`, Redis DB: 3
### Three-Interface Architecture
| Interface | Users | Technology | URL |
|-----------|-------|-----------|-----|
| Buyer-Facing Store | Buyers / public | Server-rendered Django + Tailwind | snapify.pk/store-name |
| Seller Dashboard | Sellers | React 19 PWA (installable) | app.snapify.pk |
| Admin Panel | Alorig team | React 19 SPA | admin.snapify.pk |
### Core Data Models
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| Store | seller_id, name, slug, city, category, custom_domain, template, subscription_tier, is_verified | Multi-tenant store entity |
| Product | store_id, title, description, price, images (JSON), category_id, variants (JSON), seo_meta | Product listing |
| Order | store_id, buyer_name, buyer_phone, products (JSON), total, status, tracking_number | WhatsApp-native order lifecycle |
| Category | store_id, name, slug, product_count | Store-level product categories |
| Analytics | store_id, page_type, page_slug, visitor_ip_hash, referrer, city | Per-pageview analytics |
| Subscription | store_id, plan (free/pro), started_at, expires_at, payment_method | Seller subscription |
| Post | store_id, platform, content, image_url, scheduled_at, status, engagement_metrics | Social auto-posting |
| WhatsAppSession | store_id, phone_hash, state, last_message_at | Onboarding conversation state machine |
### Multi-Tenant Architecture
- `StoreBaseModel` — FK to Store (equivalent to IGNY8's `SiteSectorBaseModel`)
- `StoreModelViewSet` — queries scoped to store
- `StoreMiddleware` — resolves tenant from URL path or custom domain lookup
### AI Pipeline (5 Stages via Celery)
| Stage | Input → Output | Technology |
|-------|----------------|-----------|
| Image Analysis | Product photos → category, color, material, style | GPT-4 Vision |
| Voice Processing | WhatsApp voice notes → structured text | Whisper API |
| Content Generation | Product metadata → title, description, bullets, meta | GPT-4 |
| SEO Optimization | Content + city → city-specific landing pages, schema | Custom templates + GPT |
| Social Content | Images + content → post-ready images with overlays | Pillow + GPT captions |
### Store Page Types (Server-Rendered)
| Page Type | URL Pattern |
|-----------|------------|
| Store Homepage | snapify.pk/store-name |
| Product Detail | snapify.pk/store-name/product-slug |
| Category Page | snapify.pk/store-name/category-slug |
| City Landing (Pro) | snapify.pk/store-name/delivery-to-lahore |
| Store About | snapify.pk/store-name/about |
| Category Hub (Platform) | snapify.pk/best-clothing-stores-punjab |
### Performance Targets (3G Pakistan)
| Metric | Target |
|--------|--------|
| First Contentful Paint | < 1.0s on 3G |
| Largest Contentful Paint | < 2.0s on 3G |
| Total Page Size | < 200KB |
| PageSpeed Score | 95+ (mobile) |
### Pricing
| Plan | Price | Features |
|------|-------|---------|
| Free | Rs. 0 | 15 products, basic dashboard, Snapify branding |
| Pro | Rs. 2,500/month | Unlimited products, custom domain, analytics, social posting, no branding |
### Celery Tasks
| Task | Schedule |
|------|----------|
| `process_whatsapp_webhook` | On webhook receipt |
| `generate_product_content` | On product photo upload |
| `generate_city_pages` | Weekly (Pro stores) |
| `regenerate_sitemap` | Daily 2 AM |
| `regenerate_hub_pages` | Weekly Sunday 3 AM |
| `social_auto_post` | Per schedule |
| `invalidate_store_cache` | On product/store change |
### Caddy Routes
```
snapify.pk → snapify_backend:8014 (server-rendered store pages)
api.snapify.pk → snapify_backend:8014/api/ (REST API)
app.snapify.pk → static React PWA build (seller dashboard)
admin.snapify.pk → static React SPA build (admin, IP-restricted)
```
### Implementation Steps
1. Create GitHub repo `snapify` with Django project scaffold
2. Configure `docker-compose.snapify.yml` (3 containers on `alorig_net`)
3. Create `snapify_db` on shared PostgreSQL
4. Build multi-tenant models (Store, Product, Order, Category, Analytics)
5. Build WhatsApp Business API integration (webhook receiver, state machine onboarding)
6. Build server-rendered Django store templates (6 page types, 7 store templates)
7. Build AI content generation pipeline (5 stages via Celery)
8. Build React seller PWA dashboard (products, orders, analytics, settings)
9. Build React admin panel (store management, moderation, platform metrics)
10. Build SEO engine (programmatic city pages, schema markup, sitemap generator)
11. Integrate Cloudinary for image CDN
12. Configure Caddy routes for all 4 subdomains
13. Integrate JazzCash/EasyPaisa for Pro subscriptions
14. Deploy, test WhatsApp onboarding flow end-to-end
15. Monitor RAM — expected ~700 MB steady state
---
## 6. APP 3: ALORIG SITE BUILDER — Multi-Tenant Website Platform
### Product Summary
AI-powered multi-tenant website builder. Dual runtime: Django (business logic/API) + Payload CMS (Node.js, content management/visual editing). Only app with a Node.js container alongside Django.
### Tech Stack Additions
| Addition | Technology | Purpose |
|----------|-----------|---------|
| CMS | Payload CMS (Node.js) | Visual content editing, block-based page builder |
| Node Runtime | Node.js 18+ | Payload CMS server |
| AI | OpenAI GPT-4 / Claude | Content generation, design suggestions |
### Container Architecture (4 containers — exception with Node.js)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `sitebuilder_backend` | sitebuilder-backend:latest | Django + Gunicorn (REST API, auth, billing, tenants) | 8016 |
| `sitebuilder_payload` | sitebuilder-payload:latest | Payload CMS (Node.js, content editing, block rendering) | 8017 |
| `sitebuilder_celery_worker` | sitebuilder-backend:latest | Celery worker (AI generation, site builds, deployments) | — |
| `sitebuilder_celery_beat` | sitebuilder-backend:latest | Celery beat scheduler | — |
### Database
- Database: `sitebuilder_db` on `alorig_postgres`
- Redis prefix: `sitebuilder:`, Redis DB: 4
- Payload CMS: can share same PostgreSQL or use separate `sitebuilder_payload_db`
### Caddy Routes
```
alorig.com/builder → static React build (builder dashboard)
api.alorig.com/builder → sitebuilder_backend:8016 (Django API)
cms.alorig.com → sitebuilder_payload:8017 (Payload CMS)
*.sites.alorig.com → rendered sites (wildcard)
```
### Implementation Steps
1. Create GitHub repo `alorig-site-builder` with Django + Payload dual scaffold
2. Configure `docker-compose.sitebuilder.yml` (4 containers on `alorig_net`)
3. Create `sitebuilder_db` database
4. Build Django models (Tenant/Site, User, Subscription, Deployment)
5. Build Payload CMS configuration (block types, page schemas, media)
6. Build Django↔Payload sync layer (auth sharing, data bridge)
7. Build AI content generation pipeline
8. Build React dashboard (site management, templates, billing)
9. Configure Caddy routes including wildcard for rendered sites
10. Deploy, test multi-tenant site creation end-to-end
11. Monitor RAM — expected ~1.25 GB steady state
---
## 7. APP 4: OBSERVER OS — Consciousness/Behavioral Awareness Platform
### Product Summary
Observer OS (0·i·G·8) — self-operating behavioral operating system. Four progressive symbolic modules (0: Dissolution → i: Identity → G: The Gate → 8: Infinity) guide users through observation of ego patterns via AI-powered journaling, pattern detection, ambient environment adaptation, and planetary alignment. NOT a meditation app — an architecture of awakening.
### Tech Stack Additions
| Addition | Technology | Purpose |
|----------|-----------|---------|
| AI (Primary) | OpenAI GPT-4o | Text analysis, pattern detection, ambient generation, module transitions |
| AI (Secondary) | Anthropic Claude | Paradox detection, reflective prompt generation |
| TTS/Audio | OpenAI TTS / ElevenLabs | Ambient sound generation, optional voice responses |
| Image Gen | Runware / Stable Diffusion | Ambient visual generation |
| Ephemeris | Swiss Ephemeris (pyswisseph) | Planetary positions for alignment layer |
### Container Architecture (3 containers — standard)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `observer_backend` | observer-backend:latest | Django + Gunicorn (REST API) | 8018 |
| `observer_celery_worker` | observer-backend:latest | Celery worker (AI analysis, patterns, ambient refresh) | — |
| `observer_celery_beat` | observer-backend:latest | Celery beat (pattern analysis, planetary updates, cleanup) | — |
### Database
- Database: `observer_db` on `alorig_postgres`
- Redis prefix: `observer:`, Redis DB: 5
### Django App Structure
| Directory | Purpose |
|-----------|---------|
| `auth/` | User model, anonymous auth, JWT |
| `api/` | Base ViewSets, pagination, response format |
| `modules/observer/` | Module state, journal, pattern endpoints |
| `modules/ambient/` | Color state, sound state, planetary overlay |
| `modules/system/` | Settings, AI config, prompt templates |
| `business/patterns/` | PatternService, EgoLoopDetector, FlowAnalyzer, ContradictionEngine |
| `business/modules/` | ModuleTransitionService, state machine, trigger evaluation |
| `business/journal/` | JournalService, NLP processing, prompt generation |
| `business/ambient/` | ColorEngine, SoundEngine, PlanetaryService |
| `ai/` | AIEngine, function registry, providers (OpenAI, Anthropic) |
| `ai/functions/` | AnalyzeJournal, DetectPatterns, GeneratePrompt, DetectParadox, GenerateAmbient, AssessTransition |
| `tasks/` | Celery task definitions |
### Core Data Models
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| User | email (optional), current_module, preferences, anonymous_token | Anonymous-first account |
| ModuleState | user_id, active_module (0/i/G/8), transition_signals, accumulated_score | Module tracking + transition readiness |
| JournalEntry | user_id, text (encrypted), emotion_tag, module_at_time, ai_analysis_summary | Encrypted journal entries |
| Pattern | user_id, pattern_type, label, frequency, last_occurrence, evidence_refs | Detected behavioral patterns |
| StateLoop | user_id, loop_id, triggers, behaviors, resolution_state, occurrence_count | Ego loops: trigger → reaction → resolution |
| Event | user_id, module, trigger_type, text_input, system_reaction | Granular event log |
| AmbientState | user_id, current_palette, current_sound, planetary_context | Current ambient config |
| PlanetarySnapshot | date, planetary_positions, major_transits, awareness_theme | Daily planetary data cache |
| PromptTemplate | module, trigger_context, prompt_text, effectiveness_score | AI prompt templates per context |
### 4 Core Modules (0 → i → G → 8)
| Module | Symbol | Role |
|--------|--------|------|
| Dissolution | 0 | Disengage the machinery of "I" — break reactive identification |
| Identity | i | Surface and expose egoic patterning — mirror the constructed self |
| The Gate | G | Trigger collapse of division between observer and observed |
| Infinity | 8 | Sustain choiceless awareness and non-fragmented perception |
### Module Transitions (Automatic, Never Announced)
| From → To | Trigger |
|-----------|---------|
| 0 → i | Emergence of habitual pattern, ego language, resistance to stillness |
| i → G | Observer begins observing the observer — paradox arises |
| G → 8 | Surrender — "I" loses center stage, ambient being replaces it |
| 8 → 0 | Identity re-coagulates — gentle rhythmic restart |
### Ambient Color System
| State | Palette | Module |
|-------|---------|--------|
| Calm / Presence | Deep indigo, soft violet, midnight blue | 8 (Infinity) |
| Alert / Activation | Amber, warm gold, burnt orange | i (Identity) |
| Stillness / Dissolution | Charcoal, muted slate, near-black | 0 (Dissolution) |
| Threshold / Paradox | Silver-white, pale lavender, stark contrast | G (Gate) |
| Agitation / Resistance | Desaturated tones, slight red warmth | Any (friction) |
### Privacy Architecture
| Tier | Location | Encryption |
|------|----------|-----------|
| Tier 1 (Local Only) | Device storage | Device-level encryption |
| Tier 2 (Encrypted Sync) | Cloud database | AES-256 at rest, TLS in transit |
| Tier 3 (Ephemeral) | Cloud AI API | Processed and discarded, never stored |
### API Endpoints
| Group | Base Path |
|-------|-----------|
| Auth | /api/v1/auth/ |
| Module State | /api/v1/observer/module/ |
| Journal | /api/v1/journal/ |
| Patterns | /api/v1/patterns/ |
| Ambient | /api/v1/ambient/ |
| Dashboard | /api/v1/dashboard/ |
| Chat | /api/v1/chat/ |
| Settings | /api/v1/settings/ |
### Celery Tasks
| Task | Schedule |
|------|----------|
| Pattern Analysis | Every 6 hours (active users) |
| Module Transition Check | Every 2 hours (active users) |
| Planetary Update | Daily at midnight |
| Ambient Refresh | On state change or hourly |
| Journal NLP Processing | On new entry (debounced 5 min) |
| Data Cleanup | Weekly |
### Caddy Routes
```
observeros.app → static React build (web app)
api.observeros.app → observer_backend:8018 (REST API)
```
### Implementation Steps
1. Create GitHub repo `observer-os` with Django project scaffold
2. Configure `docker-compose.observer.yml` (3 containers on `alorig_net`)
3. Create `observer_db` database
4. Build Django models with encryption (JournalEntry text encrypted at rest)
5. Build anonymous auth system (no email required for core use)
6. Build 4-module state machine (ModuleTransitionService)
7. Build pattern detection engine (PatternService, EgoLoopDetector, FlowAnalyzer, ContradictionEngine)
8. Build journaling API with NLP processing pipeline
9. Build ambient environment system (ColorEngine, SoundEngine, PlanetaryService)
10. Integrate AI providers (OpenAI primary, Claude secondary) via function registry
11. Build React frontend (conversational chat + visual dashboard)
12. Build ambient components (color engine, sound player, planetary overlay)
13. Configure Caddy routes
14. Deploy, test module transition flow end-to-end
15. Monitor RAM — expected ~700 MB steady state
---
## 8. APP 5: ASTROTIMING — Planetary Hours Intelligence Engine
### Product Summary
AstroTiming transforms ancient celestial timing wisdom (Al Saat by Kash Al Barni) into a modern API-powered productivity tool. Three layers: Layer 1 (Classical Planetary Hours), Layer 2 (Live Planetary Positions via Swiss Ephemeris), Layer 3 (Intelligence Engine — Timing Quality Score 0–100). Three products: consumer web app, developer API, premium scheduling.
### Tech Stack Additions
| Addition | Technology | Purpose |
|----------|-----------|---------|
| Ephemeris | pyswisseph | Sub-arcsecond planetary position calculations |
| Sunrise/Sunset | SunCalc / astronomical calc | Precise sunrise/sunset for planetary hour computation |
| AI (optional) | OpenAI GPT-4o | Natural language timing recommendations |
| Calendar | Google Calendar API / CalDAV | Premium scheduling integration |
### Container Architecture (3 containers — standard, lightest app)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `astrotiming_backend` | astrotiming-backend:latest | Django + Gunicorn (REST API + consumer web) | 8020 |
| `astrotiming_celery_worker` | astrotiming-backend:latest | Celery worker (ephemeris pre-computation, AI recommendations) | — |
| `astrotiming_celery_beat` | astrotiming-backend:latest | Celery beat (daily planetary cache, calendar sync) | — |
### Database
- Database: `astrotiming_db` on `alorig_postgres`
- Redis prefix: `astrotiming:`, Redis DB: 6
### Three-Layer Engine
**Layer 1: Classical Planetary Hours (Al Saat)**
- Deterministic, requires only location + date/time
- Sunrise/sunset → 12 unequal daytime + 12 unequal nighttime hours (24 total)
- Chaldean Order: Saturn → Jupiter → Mars → Sun → Venus → Mercury → Moon
- Activity-planet mapping from Al Saat taxonomy
- Works entirely offline
**Layer 2: Live Planetary Positions (Swiss Ephemeris)**
- Exact ecliptic longitude of all 7 classical planets
- Dignities: domicile, exaltation, detriment, fall
- Retrogrades, aspects (conjunction, sextile, square, trine, opposition)
- Lunar phase, Moon sign, void-of-course periods
- NASA JPL DE431 data: 13201 BCE–17191 CE, 0.1 arcsecond precision
**Layer 3: Intelligence Engine (TQS)**
- Timing Quality Score (0–100) for any activity at any moment
- Forward search: scan upcoming hours/days, find optimal windows
- Smart scheduler: generate optimized calendar from activities + priorities
- Conflict detection: flag unfavorable windows, suggest alternatives
### TQS Scoring Model
| Factor | Weight |
|--------|--------|
| Hour Ruler Match | 30% |
| Day Ruler Harmony | 15% |
| Planetary Dignity | 15% |
| Lunar Condition | 15% |
| Planetary Aspects | 15% |
| Retrograde Status | 10% |
Interpretation: 80–100 Excellent, 60–79 Good, 40–59 Neutral, 20–39 Unfavorable, 0–19 Avoid.
### Planet-Activity Mapping (Al Saat)
| Planet | Domains |
|--------|---------|
| Sun (Shams) | Leadership, health, government, recognition |
| Moon (Qamar) | Travel, social, domestic, emotional |
| Mars (Mirrikh) | Physical action, competition, bold ventures |
| Mercury (Utarid) | Writing, commerce, learning, technology |
| Jupiter (Mushtari) | Business expansion, education, spiritual |
| Venus (Zuhra) | Arts, relationships, beauty, social |
| Saturn (Zuhal) | Organization, long-term projects, boundaries |
### Core Data Models
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| User | email, location (lat/lng), timezone, preferences | User with location |
| Location | name, latitude, longitude, timezone, user_id | Saved locations |
| ActivityCategory | planet, category_name, activities (JSON) | Al Saat activity-planet mapping |
| TimingQuery | user_id, activity, location_id, datetime, tqs_score, breakdown (JSON) | Logged queries with scores |
| ScheduleEntry | user_id, activity, preferred_window, assigned_window, tqs_score | Smart scheduler entries |
| PlanetaryCache | date, location_hash, sunrise, sunset, planetary_hours (JSON), positions (JSON), aspects (JSON) | Pre-computed daily data |
| APIKey | user_id, key_hash, tier (free/developer/premium), requests_today, rate_limit | Developer API key management |
### Pricing
| Tier | Price | Features |
|------|-------|---------|
| Free (Consumer) | $0 | Web app: current hour, TQS for any activity, daily view |
| Developer API | $9–49/month | REST API: TQS, forward search, bulk queries, webhooks |
| Premium | $4.99/month | Smart scheduler, calendar sync, daily briefings, optimal window alerts |
### Celery Tasks
| Task | Schedule |
|------|----------|
| `precompute_daily_planetary_data` | Daily 00:00 UTC |
| `generate_daily_briefing` | Daily per user timezone (sunrise − 30min) |
| `sync_user_calendars` | Every 30 min (premium users) |
| `cleanup_old_queries` | Weekly |
### Caddy Routes
```
astrotiming.com → static React build (consumer web app)
api.astrotiming.com → astrotiming_backend:8020 (REST API + developer API)
```
### Implementation Steps
1. Create GitHub repo `astrotiming` with Django project scaffold
2. Install `pyswisseph` and verify ephemeris calculations
3. Configure `docker-compose.astrotiming.yml` (3 containers on `alorig_net`)
4. Create `astrotiming_db` database
5. Build Layer 1: Classical planetary hours engine (Al Saat ruleset)
6. Build Layer 2: Swiss Ephemeris integration (positions, dignities, aspects, retrogrades, lunar)
7. Build Layer 3: TQS scoring engine combining Layer 1 + Layer 2
8. Build forward search and smart scheduler algorithms
9. Build Django models, REST API endpoints
10. Build developer API with key management and rate limiting
11. Build React frontend (daily view, activity search, score display)
12. Configure Caddy routes
13. Implement PlanetaryCache pre-computation pipeline
14. Deploy, validate TQS against known Al Saat test cases
15. Monitor RAM — expected ~400 MB steady state
---
## 9. APP 6: ASMS PHASE 1 — School Management System
### Product Summary
Alorig School Management System (ASMS) Phase 1 — pilot deployment for 74 schools in Pakistan. Foundation platform: student/guardian management, multi-school tenancy, admissions, gradebook, attendance, courses, RBAC, LMS core. Phase 1 scope only (700–1,030 hours); Advanced Modules & Intelligence Layer are Phase 2+.
### Tech Stack Additions
| Addition | Technology | Purpose |
|----------|-----------|---------|
| Multi-Tenant | Head Office → Branch → School hierarchy | 74-campus data isolation |
| RBAC | Django permission groups | 10 user roles |
| Attendance | GPS + geo-fence (Phase 2) | 100m radius per school |
| WhatsApp | WhatsApp Business API (Phase 2) | Parent notifications |
| AI (Phase 2) | Predictive analytics, adaptive assessment | At-risk identification |
### Container Architecture (3 containers — standard)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `asms_backend` | asms-backend:latest | Django + Gunicorn (REST API) | 8022 |
| `asms_celery_worker` | asms-backend:latest | Celery worker (reports, data import, notifications) | — |
| `asms_celery_beat` | asms-backend:latest | Celery beat (attendance reports, grade calculations) | — |
### Database
- Database: `asms_db` on `alorig_postgres`
- Redis prefix: `asms:`, Redis DB: 7
### Multi-School Tenant Architecture
- 3-level hierarchy: Head Office → Branch (regional) → School (campus)
- `SchoolBaseModel` — FK to School with tenant isolation
- `SchoolModelViewSet` — queries scoped to user's school(s)
- HO Admin sees all; Branch Admin sees branch schools; Principal sees own school only
### Phase 1 Foundation Modules
| Module | Key Models |
|--------|-----------|
| Student & Guardian Management | Student, Guardian, Enrollment, Transfer |
| Multi-School Tenant | HeadOffice, Branch, School, SchoolUser |
| Admissions & Registration | Application, AdmissionLevel, AdmissionStep |
| Gradebook & Report Cards | Grade, MarkingPeriod, Transcript, ReportCard |
| Discipline & Behavior | Infraction, DisciplineAction, BehaviorLog |
| Attendance (Base) | Attendance, AttendanceRecord, AbsenteeFlag |
| Courses & Scheduling | Course, ClassSection, TeacherAssignment, Schedule |
| RBAC (10 Roles) | Role, Permission, UserSchoolAccess |
| LMS Core | Curriculum, LessonPlan, Quiz, Question, LearningOutcome |
### 10 User Roles
HO Admin, Branch Admin, Principal, Vice Principal, Teacher, Class Teacher, Accountant, Parent, Student, Support Staff
### Caddy Routes
```
schools.alorig.com → static React build (web app)
api.schools.alorig.com → asms_backend:8022 (REST API)
```
### Implementation Steps
1. Create GitHub repo `asms` with Django project scaffold
2. Configure `docker-compose.asms.yml` (3 containers on `alorig_net`)
3. Create `asms_db` database
4. Build multi-tenant hierarchy models (HeadOffice, Branch, School)
5. Build RBAC system (10 roles with granular permissions)
6. Build Student & Guardian management module
7. Build Admissions workflow module
8. Build Courses & Scheduling module with conflict detection
9. Build Attendance module (daily homeroom/class, absence tracking)
10. Build Gradebook module (standards-based grading, report cards, transcripts)
11. Build Discipline & Behavior tracking module
12. Build LMS core (curriculum, lesson plans, quiz engine)
13. Build React frontend (role-based dashboards, CRUD screens, reports)
14. Build REST API layer for all modules
15. Seed with pilot school data (5 schools initially)
16. Deploy, run pilot testing with 5 schools
17. Monitor RAM — expected ~600 MB steady state
---
## 10. POST-DEPLOYMENT MONITORING
### Immediate (First 24 Hours After Each App)
1. `docker stats` — verify container RAM matches budget
2. PostgreSQL: `SELECT count(*) FROM pg_stat_activity;` — check connection count
3. Redis: `redis-cli INFO memory` — verify key prefix isolation
4. Caddy routes — verify SSL and domain routing
5. Smoke tests — login, core functionality, API responses
### Ongoing (First Week)
1. Monitor RAM over full business cycle
2. Check Celery task queue depths — no backlogs
3. Verify inter-app isolation (no Redis key collisions, no DB cross-contamination)
4. Test under expected concurrent user load
### Upgrade Triggers
| Condition | Action |
|-----------|--------|
| Steady-state RAM > 13 GB | Upgrade to KVM 8 or split heaviest app |
| OOM kills on any container | Investigate — reduce workers or split |
| PostgreSQL connections > 80 | Add PgBouncer connection pooling |
| Redis memory > 2 GB | Review TTLs, add eviction policies |
| Celery queue > 1000 pending | Add worker or split queues |
---
## 11. ACCEPTANCE CRITERIA
- [ ] Each of the 6 apps has: containers, database, Redis config, Caddy routes, implementation steps
- [ ] Deployment order specified with rationale and triggers
- [ ] Post-deployment monitoring protocol documented
- [ ] Resource budget per app with upgrade triggers
- [ ] Docker compose naming convention established
- [ ] Each app references its source blueprint document
- [ ] Exception patterns (Psydge WebSocket, Site Builder Node.js) clearly documented
- [ ] Port assignments verified — no conflicts across all apps
- [ ] Redis DB numbers verified — no conflicts across all apps
- [ ] All Caddy routes listed — no domain conflicts
- [ ] Total steady-state RAM estimate ~10.7 GB within 16 GB budget
---
## 12. CLAUDE CODE INSTRUCTIONS
### Context Requirements
1. Read Phase 0 infrastructure docs — understand shared services configuration
2. Read IGNY8 docker-compose files — understand existing port/Redis/DB allocation
3. Read each app blueprint document for deep-dive implementation details
4. Verify port and Redis DB assignment maps against all existing compose files
### Execution Order
```
For each app (in deployment order):
1. Create GitHub repo with Django scaffold
2. Create docker-compose.{app}.yml on alorig_net
3. Create database on shared PostgreSQL
4. Build models + migrations
5. Build app-specific business logic
6. Build REST API endpoints
7. Build React frontend (static build for Caddy)
8. Configure Caddy routes
9. Deploy + smoke test
10. Monitor RAM for 24 hours
11. Proceed to next app only when stable
```
### Critical Rules
1. **One app at a time** — deploy sequentially, monitor between each
2. **Shared infrastructure** — every app joins `alorig_net`, uses shared PG/Redis
3. **Database per app** — separate database name on shared PostgreSQL instance
4. **Redis per app** — separate DB number + key prefix
5. **Port uniqueness** — use assigned ports from port map (no conflicts)
6. **Static frontend** — React builds served by Caddy (no frontend containers)
7. **Exception containers** — only Psydge (Channels) and Site Builder (Payload) get extra containers
8. **RAM monitoring** — check after each app, stop if approaching 13 GB
9. **Compose naming** — `docker-compose.{appname}.yml` convention
### Cross-References
| Doc | Relationship |
|-----|-------------|
| 00-MASTER | Phase 0 infrastructure that all apps run on |
| 00A Server + Docker | Shared services, `alorig_net` network, Caddy config |
| 00B Database Schema | PostgreSQL setup, per-app databases |
| 00C Caddy Configuration | Reverse proxy rules, SSL, domain routing |
| 04A–04C | IGNY8 must be stable (Phase 4 complete) before Phase 5 begins |

1628
v2/V2-Execution-Docs/PHASE-2-BUILD-PLAN.md Normal file
View File

File diff suppressed because it is too large Load Diff

1305
v2/V2-Execution-Docs/PHASE-3-BUILD-PLAN.md Normal file
View File

File diff suppressed because it is too large Load Diff

883
v2/V2-Execution-Docs/PHASE-4-BUILD-PLAN.md Normal file
View File

@@ -0,0 +1,883 @@
# Phase 4 Build Plan — Business Layer (3 Docs)
**Purpose:** This is a single instruction set for Claude Code to build all 3 Phase 4 execution documents. Read this entire file, then build each doc one at a time.
---
## CRITICAL BASELINE RULES (from V2-Execution-Docs-Updated)
Before writing ANY doc, internalize these codebase-verified facts:
### Primary Keys
- ALL models use `BigAutoField` (integer PKs, NOT UUIDs)
- `DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'`
- Every ID in API requests/responses is an integer
### Model Names (PLURAL)
- `Clusters` (not Cluster), `Keywords` (not Keyword), `Tasks` (not Task)
- `ContentIdeas` (not ContentIdea), `Images` (not Image), `Content` (stays singular)
### App Structure
- Project root: `igny8_core/`
- SAG app: `igny8_core/sag/`
- App labels: `igny8_core_auth`, `planner`, `writer`, `automation`, `integration`, etc.
- All tables use `igny8_` prefix
### Verified Codebase Versions
- Python 3.11, Django >=5.2.7, Node 18, React ^19.0.0, TypeScript ~5.7.2, Zustand ^5.0.8, Vite ^6.1.0, Celery >=5.3.0
### Model Inheritance
- `AccountBaseModel` provides: `account` FK, `created_by`, `created_at`, `updated_at`
- `SiteSectorBaseModel` extends AccountBaseModel with: `site` FK, `sector` FK
- `SoftDeletableModel` mixin provides soft delete via `SoftDeleteManager`
### Frontend Stack
- TypeScript `.tsx` files, Zustand state management, Vitest + Playwright testing
### Source of Truth
- Start every doc with: `**Source of Truth:** Codebase at /data/app/igny8/`
---
## DOC STRUCTURE STANDARD
Every doc MUST follow this structure:
```
# [Phase.Sub] — Title
**Source of Truth:** Codebase at `/data/app/igny8/`
**Version:** 1.0
**Date:** 2026-03-23
## 1. Current State
## 2. What to Build
## 3. Data Models & APIs
## 4. Implementation Steps
## 5. Acceptance Criteria
## 6. Claude Code Instructions
```
---
## EXISTING BILLING CONTEXT (Phase 4 builds on top of this)
### Existing Billing Models (billing app)
- `Plan` — subscription tier definition (Free/$49/$149/$349)
- `Subscription` — active subscription per account, links to Plan
- `CreditTransaction` — individual credit debit/credit events
- `CreditUsageLog` — per-operation usage tracking
- `CreditCostConfig` — configurable cost per operation type
- `AccountPaymentMethod` — saved Stripe/PayPal payment methods
- `Payment` — payment records
- `Invoice` — invoice records
### Existing Credit System
- `CreditService` in `business/billing/` manages all credit operations
- Pattern: `check_credits()` → execute operation → `deduct_credits()` → `log_usage()`
- `AIModelConfig.tokens_per_credit` (text) and `credits_per_image` (images)
### Current Plans (live)
| Plan | Price | Sites | Users | Credits/Month |
|------|-------|-------|-------|---------------|
| Free | $0 | 1 | 1 | 100 |
| Starter | $49 | 3 | 3 | 1,000 |
| Growth | $149 | 10 | 10 | 5,000 |
| Scale | $349 | Unlimited | Unlimited | 25,000 |
### Payment Infrastructure
- Stripe + PayPal integrated (live credentials)
- Manual payment methods available
- Existing webhook handlers for subscription events
### Existing Integration Infrastructure
- `SiteIntegration` — stores WordPress + other connections
- `SyncEvent` — logs sync operations
- `IntegrationProvider` — provider definitions (OpenAI, Anthropic, Runware, etc.)
- `IntegrationSettings` — per-account provider settings
### Phase 2 Models Phase 4 Depends On
- `SAGCampaign`, `SAGBacklink`, `CampaignKPISnapshot` (02E — backlink engine)
- `GSCConnection`, `URLInspectionRecord`, `GSCMetricsCache` (02C — GSC data)
- `SAGLink`, `SAGLinkAudit`, `LinkMap` (02D — internal linking)
- `SocialPost`, `SocialAccount`, `SocialEngagement` (02H — socializer)
- `SchemaTemplate`, `SERPEnhancement`, `SchemaValidationResult` (02G — schema)
- `SAGBlueprint`, `SAGAttribute`, `SAGCluster` (01A — SAG core)
- `AutomationConfig`, `AutomationRun` (automation app)
---
## DOC 04A: managed-services.md
**File:** `V2-Execution-Docs/04A-managed-services.md`
**Scope:** Managed service tiers (Lite/Pro), backlink service packages, Ahrefs API integration, backlink indexing service, client onboarding automation, service delivery workflows, admin dashboard.
**Depends On:** 02E (backlink campaign engine) + 03B (WP plugin connected mode)
### Source Material:
- Live Docs on Server/igny8-app-docs/plans/final-dev-guides-of-futute-implementation/DocD-Business-Services-Dev-Guide.md
- temp/IGNY8-Project-Files/SAG-Doc4-External-Backlink-Campaign-PLAN.md (managed service pricing)
### What to Cover:
**1. Current State:**
- Billing infrastructure exists (Plan, Subscription, CreditTransaction, Payment, Invoice)
- Stripe/PayPal integrated and live
- No managed services layer — Alorig manages client sites manually
- No backlink order tracking — orders placed manually on FatGrid/PRNews
- No automated reporting — reports created manually
- No Ahrefs integration — domain/backlink data checked manually in Ahrefs dashboard
- No backlink indexing service — links indexed organically or manually submitted
- 6 existing Alorig clients ready for managed service conversion
**2. What to Build:**
**Managed Service Tiers:**
**Managed Lite ($100/site/month):**
- 10 articles/month (published to WordPress via pipeline)
- Basic SAG setup + automation configuration (first month)
- Monthly PDF report
- Email support (48-hour response)
- Automation config: weekly schedule, 3 articles/run, post + cluster_hub types
- Review required before publish (Alorig team reviews in Writer)
**Managed Pro ($399/site/month):**
- 30 articles/month (hubs prioritized early, supporting content later)
- Full SAG architecture with detailed mode
- Backlink campaign management (budget separate, pass-through + 20% markup)
- Social media content generation + scheduling (configured platforms)
- Retroactive schema enhancement on existing pages
- GSC monitoring: auto-indexing, re-inspection, issue alerts
- Weekly PDF report with full KPIs
- Dedicated account manager (24-hour response)
- Taxonomy term content generation
**Backlink Service Packages (Managed Mode — Alorig Executes):**
By quality level:
| Tier | Service | Alorig Cost | Client Price | Margin |
|------|---------|-------------|-------------|--------|
| Basic Guest Post | DR 30-50 via FatGrid | $30-80 | $150-300 | 3-5× |
| Standard Guest Post | DR 50-70 via FatGrid | $100-300 | $400-800 | 2-3× |
| Premium Guest Post | DR 70+ via FatGrid | $500-2,000 | $1,500-5,000 | 2-3× |
| PR Basic | 300+ outlets via EIN Presswire | $99-499 | $500-1,500 | 3-5× |
| PR Premium | Yahoo/Bloomberg/Fox via PRNews.io/Linking News | $500-5,000 | $2,000-15,000 | 3-4× |
Monthly retainer packages:
| Package | Links/Month | DR Range | Monthly Cost | Market |
|---------|------------|----------|-------------|--------|
| Starter | 5-8 links | DR 30-50 | $800-1,500 | PK market |
| Growth | 10-15 links | DR 30-70 mix | $2,000-4,000 | UK/CA market |
| Authority | 15-25 links | DR 40-70+ mix + PR | $4,000-8,000 | USA market |
| Enterprise | Custom | Custom | Custom | Multi-site/agency |
Niche surcharges: Crypto/Casino 2-3×, Finance/Insurance 1.5-2×, Health/Medical 1.5-2×, Tech/SaaS 1.2-1.5×, General 1× baseline.
**Ahrefs API Integration (NEW — not in original planning docs):**
IGNY8 integrates with Ahrefs API v3 for automated domain metrics, backlink monitoring, and competitive intelligence. This serves both the managed services reporting and the self-service backlink module.
Ahrefs API endpoints to integrate:
- `GET /v3/site-explorer/domain-rating` — DR for any domain (used in backlink quality scoring)
- `GET /v3/site-explorer/backlinks` — all backlinks to a target URL/domain (verify placed links)
- `GET /v3/site-explorer/referring-domains` — referring domain list + metrics
- `GET /v3/site-explorer/organic-keywords` — organic keyword rankings (supplement GSC data)
- `GET /v3/site-explorer/pages-by-traffic` — top pages by organic traffic
- `GET /v3/site-explorer/metrics` — domain metrics summary (DR, referring domains, organic traffic, organic keywords)
- `GET /v3/site-explorer/metrics-history` — historical domain metrics (DR trend, traffic trend)
- `GET /v3/site-explorer/refdomains-history` — referring domains over time
Use cases in IGNY8:
1. **Backlink verification** — after SAGBacklink is placed, verify it's live via Ahrefs crawl data (supplements HTTP check)
2. **Quality scoring** — auto-populate source_dr, source_traffic on SAGBacklink from Ahrefs
3. **Campaign KPIs** — pull DR, referring domains, organic keywords for CampaignKPISnapshot automatically
4. **Competitor analysis** — compare client's domain metrics against competitors
5. **Dead link detection** — cross-reference Ahrefs lost backlinks with SAGBacklink records
6. **Reporting** — Ahrefs metrics feed into ServiceReport data (DR history, referring domain growth, top pages)
7. **Link prospecting** — find competitor backlinks for outreach opportunities
Ahrefs API pricing: credits-based (varies by plan), IGNY8 stores API key per account in IntegrationSettings.
**Backlink Indexing Service (NEW — not in original planning docs):**
After a backlink is placed and verified live, it needs to be crawled by Google to pass authority. Natural indexing can take weeks/months. IGNY8 integrates an on-demand backlink indexing service.
Indexing service options (integrate best available):
- **IndexNow API** (free) — submit URLs to Bing/Yandex (not Google, but signals exist)
- **Google Indexing API** — only for JobPosting/BroadcastEvent pages (limited use)
- **Omega Indexer** — bulk indexing service, API available, ~$0.02-0.05/URL
- **Colossus Indexer** — high success rate, ~$0.03-0.07/URL
- **SpeedyIndex** — fast indexing, API available, credit-based pricing
- **LinkCentaur** — drip-feed indexing, API available
Recommended primary: **SpeedyIndex or Omega Indexer** (both have APIs, reasonable pricing, good success rates)
Integration flow:
```
SAGBacklink status changes to 'live'
→ Verify URL is accessible (HTTP 200)
→ Queue for indexing service submission
→ Submit via indexing service API
→ Track: submitted_for_indexing_at, indexing_service, indexing_status
→ Check back after 7 days (Ahrefs or HTTP cache header check)
→ If indexed → update SAGBacklink.is_indexed = True
→ If not after 14 days → re-submit once
→ If still not after 28 days → mark as indexing_failed, flag for review
```
Client-facing: "Indexing Boost" add-on or included in Pro tier
Pricing to client: $0.10-0.20 per URL (5-10× markup on service cost)
**Client Onboarding Automation:**
When `ManagedServiceSubscription` is created:
1. Check site has IGNY8 plugin installed + connected (verify `SiteIntegration` exists)
2. If no blueprint → queue Celery task to run SAG wizard (auto-generate blueprint)
3. Configure `AutomationConfig` based on service_config (schedule, stages, content types)
4. Send welcome email to client (what to expect, how to approve content, contact info)
5. Create first month content tasks in pipeline queue
6. Notify account manager: "New client onboarded, first content due"
7. Update subscription status: pending → active
Monthly delivery workflow (Lite):
```
Month Start → Check blueprint exists → Run pipeline (10 articles) →
Alorig reviews → Approve/edit → Publish to WP →
Update articles_published counter → Month End → Generate report → Send → Bill
```
Monthly delivery workflow (Pro — adds to Lite):
```
+ Run pipeline for 30 articles (hubs first) + Taxonomy term content
+ Backlink: Load SAGCampaign monthly plan → Browse FatGrid → Place orders →
Track delivery → Quality check → Log SAGBacklink → Submit for indexing
+ Social: Generate platform-adapted posts → Schedule via best-time algorithm
+ Schema: Run retroactive scan → Push to WP
+ GSC: Check indexing → Re-request pending → Flag issues
+ Weekly: Generate report → Send
```
**3. Data Models & APIs:**
New models (in new `services` app):
- `ManagedServiceSubscription` (extends AccountBaseModel)
- `site` ForeignKey to Site
- `tier` CharField (lite/pro)
- `status` CharField (pending/active/paused/cancelled)
- `monthly_price` DecimalField(max_digits=8, decimal_places=2)
- `articles_per_month` IntegerField
- `account_manager` ForeignKey to User (nullable) — assigned team member
- `current_month_articles_published` IntegerField (default 0)
- `current_month_start` DateField (nullable)
- `service_config` JSONField — per-site settings (automation, social, backlinks, reporting)
- `started_at` DateTimeField (nullable)
- `cancelled_at` DateTimeField (nullable)
- `next_billing_date` DateField (nullable)
- Table: `igny8_managed_service_subscriptions`
- `BacklinkServiceOrder` (extends AccountBaseModel)
- `site` ForeignKey to Site
- `campaign` ForeignKey to SAGCampaign (nullable)
- `managed_subscription` ForeignKey to ManagedServiceSubscription (nullable)
- `order_type` CharField (one_time/monthly)
- `package` CharField (starter/growth/authority/enterprise/custom)
- `client_price` DecimalField(max_digits=10, decimal_places=2)
- `actual_cost` DecimalField(max_digits=10, decimal_places=2, default=0)
- `margin` DecimalField(max_digits=10, decimal_places=2, default=0)
- `niche_surcharge` FloatField (default 1.0)
- `links_ordered` IntegerField (default 0)
- `links_delivered` IntegerField (default 0)
- `links_live` IntegerField (default 0)
- `links_indexed` IntegerField (default 0)
- `status` CharField (draft/approved/in_progress/delivered/completed)
- `ordered_at` DateTimeField (nullable)
- `delivered_at` DateTimeField (nullable)
- `notes` TextField (blank)
- Table: `igny8_backlink_service_orders`
- `AhrefsConnection` (extends AccountBaseModel)
- `api_key` TextField (encrypted)
- `plan_type` CharField (nullable) — lite/standard/advanced/enterprise
- `credits_remaining` IntegerField (nullable)
- `last_synced` DateTimeField (nullable)
- `status` CharField (active/expired/error)
- Table: `igny8_ahrefs_connections`
- `AhrefsDomainSnapshot` (extends SiteSectorBaseModel)
- `domain` CharField
- `domain_rating` FloatField
- `referring_domains` IntegerField
- `organic_keywords` IntegerField
- `organic_traffic` IntegerField
- `backlinks_total` IntegerField
- `snapshot_date` DateField
- `raw_data` JSONField — full API response
- Table: `igny8_ahrefs_domain_snapshots`
- `BacklinkIndexingRequest` (extends SiteSectorBaseModel)
- `backlink` ForeignKey to SAGBacklink
- `url` URLField — the backlink source URL to be indexed
- `indexing_service` CharField (speedyindex/omega/colossus/indexnow/manual)
- `service_request_id` CharField (nullable) — external service reference
- `submitted_at` DateTimeField (nullable)
- `first_check_at` DateTimeField (nullable)
- `last_check_at` DateTimeField (nullable)
- `check_count` IntegerField (default 0)
- `is_indexed` BooleanField (default False)
- `indexed_at` DateTimeField (nullable)
- `status` CharField (queued/submitted/checking/indexed/failed/resubmitted)
- `cost` DecimalField(max_digits=6, decimal_places=4, default=0)
- Table: `igny8_backlink_indexing_requests`
Modified models:
- `SAGBacklink` (from 02E) — add fields:
- `is_indexed` BooleanField (default False)
- `indexed_at` DateTimeField (nullable)
- `ahrefs_verified` BooleanField (default False)
- `ahrefs_last_seen` DateTimeField (nullable)
- `indexing_submitted` BooleanField (default False)
New endpoints:
```
# Managed Service Subscriptions
GET /api/v1/services/subscriptions/ # List (admin only)
POST /api/v1/services/subscriptions/ # Create
GET /api/v1/services/subscriptions/{id}/ # Detail
PATCH /api/v1/services/subscriptions/{id}/ # Update config/status
POST /api/v1/services/subscriptions/{id}/onboard/ # Trigger onboarding flow
POST /api/v1/services/subscriptions/{id}/pause/ # Pause service
POST /api/v1/services/subscriptions/{id}/resume/ # Resume service
POST /api/v1/services/subscriptions/{id}/cancel/ # Cancel service
# Backlink Service Orders
GET /api/v1/services/backlink-orders/ # List (admin)
POST /api/v1/services/backlink-orders/ # Create order
GET /api/v1/services/backlink-orders/{id}/ # Detail
PATCH /api/v1/services/backlink-orders/{id}/ # Update status/details
GET /api/v1/services/backlink-orders/{id}/links/ # Links in this order
POST /api/v1/services/backlink-orders/{id}/approve/ # Client approves order
# Ahrefs Integration
POST /api/v1/integration/ahrefs/connect/ # Save API key
DELETE /api/v1/integration/ahrefs/disconnect/ # Remove connection
GET /api/v1/integration/ahrefs/status/ # Connection status + credits
GET /api/v1/integration/ahrefs/domain/{domain}/ # Domain metrics (proxy)
GET /api/v1/integration/ahrefs/domain/{domain}/history/ # DR + traffic history
GET /api/v1/integration/ahrefs/backlinks/{domain}/ # Backlinks list
GET /api/v1/integration/ahrefs/referring-domains/{domain}/ # Referring domains
GET /api/v1/integration/ahrefs/organic-keywords/{domain}/ # Organic keywords
POST /api/v1/integration/ahrefs/verify-backlinks/ # Bulk verify SAGBacklinks via Ahrefs
POST /api/v1/integration/ahrefs/snapshot/ # Take domain snapshot for KPIs
# Backlink Indexing
POST /api/v1/linker/indexing/submit/ # Submit URLs for indexing
POST /api/v1/linker/indexing/bulk-submit/ # Bulk submit (all live unindexed)
GET /api/v1/linker/indexing/requests/?site_id=X # List indexing requests
GET /api/v1/linker/indexing/requests/{id}/ # Single request status
GET /api/v1/linker/indexing/stats/?site_id=X # Indexing success rate, pending count
# Admin Dashboard
GET /api/v1/services/dashboard/ # Revenue summary, client count, MRR
GET /api/v1/services/dashboard/overdue/ # Overdue content/links deliverables
GET /api/v1/services/dashboard/margin/ # Margin tracking across all orders
```
Celery tasks:
- `onboard_managed_client` — triggered on subscription creation, runs full onboarding
- `process_managed_billing` — monthly, invoice all active subscriptions
- `reset_monthly_counters` — monthly, reset articles_published counters
- `check_delivery_status` — daily, flag subscriptions behind on content/links
- `sync_ahrefs_domain_snapshot` — weekly per connected site, pull DR/traffic/etc.
- `verify_backlinks_via_ahrefs` — weekly, cross-reference SAGBacklink records with Ahrefs data
- `submit_backlinks_for_indexing` — daily, submit live unindexed backlinks to indexing service
- `check_indexing_status` — daily, check submitted URLs (after 7-day wait)
- `detect_lost_backlinks_ahrefs` — weekly, compare Ahrefs data with SAGBacklink records
Services:
- `ManagedBillingService` — subscription CRUD, billing, margin calculation
- `AhrefsService` — API client wrapper, rate limiting, credit tracking
- `BacklinkIndexingService` — submit to indexing service, track status, retry logic
**Margin Analysis:**
| Component | Managed Lite | Managed Pro |
|-----------|-------------|-------------|
| Service Fee | $100/month | $399/month |
| Platform Cost (credits) | ~$15-25 | ~$50-80 |
| Human Time | ~2-3 hrs/month | ~8-12 hrs/month |
| Ahrefs API cost | ~$2-5 | ~$10-20 |
| Indexing service cost | $0 | ~$5-15 |
| Gross Margin | ~70-75% | ~65-75% |
Note: Backlink purchase costs are pass-through with 20% markup — separate from service fee.
**Cross-references:** 02E (SAGCampaign/SAGBacklink models), 02C (GSC data for indexing status), 02D (internal linking health), 02H (socializer for Pro tier), 02G (schema enhancement for Pro tier), 01A (SAGBlueprint for onboarding), 03B (WP plugin connected for content delivery)
---
## DOC 04B: whitelabel-reporting.md
**File:** `V2-Execution-Docs/04B-whitelabel-reporting.md`
**Scope:** Agency account model, automated report generation (weekly/monthly/quarterly), white-label branding, admin services dashboard, client-facing read-only view, PDF report templates, Linking News white-label integration.
**Depends On:** 04A (managed services provides subscriptions + order data)
### Source Material:
- Live Docs on Server/igny8-app-docs/plans/final-dev-guides-of-futute-implementation/DocD-Business-Services-Dev-Guide.md
### What to Cover:
**1. Current State:**
- No reporting infrastructure in IGNY8
- No agency/reseller model
- No white-label capability
- Reports created manually by Alorig team (Google Docs/Sheets)
- No admin dashboard for managed service oversight
**2. What to Build:**
**Report Generation Engine:**
9 configurable report sections (each can be included/excluded per client):
1. **Executive Summary** — key wins, KPI highlights, AI-generated summary paragraph
2. **Content Performance** — articles published (count, by type), word count, images created
3. **Indexing Status** — pages indexed/pending/error (from GSC 02C), trend chart
4. **Keyword Rankings** — keywords in top 10/20/50, position movers up/down, new entries (from GSC + Ahrefs)
5. **Organic Traffic** — clicks, impressions, CTR, avg position (from GSC), month-over-month change
6. **Backlink Campaign** (if active) — links built, DR distribution chart, budget spent vs allocated, quality scores, Ahrefs referring domain growth
7. **Social Media** (if active) — posts published by platform, engagement (likes/comments/shares), top performers
8. **SAG Health** — overall blueprint completion %, cluster status, authority score (0-100), content gaps remaining
9. **Next Month Plan** — upcoming content topics, backlink targets, action items for client approval
Data sources per section:
- Content: `Content` model (writer app) filtered by site + date range
- Indexing: `URLInspectionRecord` (02C) + `GSCMetricsCache`
- Rankings: `GSCMetricsCache` (02C) + `AhrefsDomainSnapshot` (04A)
- Traffic: `GSCMetricsCache` (02C)
- Backlinks: `SAGBacklink` (02E) + `BacklinkServiceOrder` (04A) + `CampaignKPISnapshot` (02E) + `AhrefsDomainSnapshot` (04A)
- Social: `SocialPost` + `SocialEngagement` (02H)
- SAG Health: `SAGBlueprint` health score (01G)
- Next Month: `SAGCampaign` monthly plan (02E) + automation queue
**PDF Report Generation:**
- Template engine: WeasyPrint (HTML → PDF) or ReportLab
- Base HTML template with CSS styling
- Sections rendered conditionally based on client config
- Charts rendered via matplotlib or chart.js (server-side rendering)
- Output stored in S3/media storage, URL saved in `ServiceReport.report_pdf_url`
- Template supports light/dark themes
**White-Label Branding (Agency Play):**
For agencies reselling IGNY8 services to their clients:
- `AgencyBranding` model stores: agency name, logo URL, primary color, secondary color, contact email, contact phone, website URL, footer text
- When `ServiceReport.is_white_label = True`:
- "IGNY8" / "Alorig" replaced with `brand_name` throughout PDF
- Logo replaced with `brand_logo_url` in header
- Footer shows agency contact info
- Color scheme uses agency brand colors
- Report URL domain can be custom (via Caddy subdomain routing)
- Linking News integration: white-label PR distribution reports use agency branding
**Admin Services Dashboard (Alorig team only):**
Frontend pages (TypeScript .tsx):
```
frontend/src/pages/Services/
├── ServicesDashboard.tsx # Overview: all managed clients, MRR, revenue
│ ├── ClientList.tsx # All subscriptions with status, tier, articles count
│ ├── RevenueSummary.tsx # MRR total, margin total, trend chart
│ └── UpcomingActions.tsx # Overdue: reports, content, renewals
├── ClientDetail.tsx # Single client management
│ ├── ClientConfig.tsx # Service config editor (JSON form)
│ ├── ContentTracker.tsx # Articles published this month vs target
│ ├── BacklinkTracker.tsx # Links ordered vs delivered vs live vs indexed
│ ├── AhrefsMetrics.tsx # DR history, referring domains, top pages
│ ├── ReportHistory.tsx # Past reports with resend option
│ └── BillingHistory.tsx # Invoices, payments, margin per order
├── BacklinkOrders.tsx # All orders across clients
│ ├── OrderList.tsx # Filterable by client, status, package
│ └── OrderDetail.tsx # Individual order with link-by-link tracking
├── IndexingDashboard.tsx # Backlink indexing status across all clients
└── ReportGenerator.tsx # Generate report for any client + period
```
Sidebar addition (admin-only, behind `managed_services_enabled` feature flag):
```
ADMIN
├── Sector Templates
└── Managed Services (NEW)
```
**Client-Facing View (for managed clients who also have IGNY8 access):**
Frontend pages:
```
frontend/src/pages/ManagedService/
├── ServiceOverview.tsx # What's included, current month progress
├── ReportViewer.tsx # View/download past reports
└── ApprovalQueue.tsx # Approve blueprint, review content (if review_required)
```
Sidebar addition (only shows if `ManagedServiceSubscription` exists for account):
```
ACCOUNT
├── Account Settings
├── Plans & Billing
├── Managed Service (NEW — conditional)
├── Usage
└── AI Models
```
**3. Data Models & APIs:**
New models (in `services` app):
- `ServiceReport` (extends AccountBaseModel)
- `site` ForeignKey to Site
- `managed_subscription` ForeignKey to ManagedServiceSubscription (nullable)
- `report_type` CharField (weekly/monthly/quarterly)
- `period_start` DateField
- `period_end` DateField
- `report_data` JSONField — all metrics for period (structured by section)
- `report_pdf_url` URLField (blank) — generated PDF in S3
- `is_white_label` BooleanField (default False)
- `branding` ForeignKey to AgencyBranding (nullable)
- `sections_included` JSONField — list of included section keys
- `generated_at` DateTimeField (auto_now_add)
- `sent_at` DateTimeField (nullable)
- `sent_to` JSONField — list of email addresses sent to
- Table: `igny8_service_reports`
- `AgencyBranding` (extends AccountBaseModel)
- `brand_name` CharField(max_length=200)
- `brand_logo_url` URLField (blank)
- `primary_color` CharField(max_length=7) — hex color
- `secondary_color` CharField(max_length=7) — hex color
- `contact_email` EmailField (blank)
- `contact_phone` CharField (blank)
- `website_url` URLField (blank)
- `footer_text` TextField (blank)
- `is_active` BooleanField (default True)
- Table: `igny8_agency_branding`
- `ReportTemplate` (extends AccountBaseModel)
- `name` CharField
- `template_html` TextField — base HTML template
- `template_css` TextField — custom CSS
- `is_default` BooleanField
- `branding` ForeignKey to AgencyBranding (nullable) — agency-specific template
- Table: `igny8_report_templates`
New endpoints:
```
# Service Reports
GET /api/v1/services/reports/ # List (admin + client's own)
POST /api/v1/services/reports/generate/ # Generate report
GET /api/v1/services/reports/{id}/ # Detail + PDF URL
POST /api/v1/services/reports/{id}/send/ # Email to recipients
POST /api/v1/services/reports/{id}/regenerate/ # Regenerate PDF
GET /api/v1/services/reports/preview/ # Preview with data (no save)
# Agency Branding
GET /api/v1/services/branding/ # List brandings
POST /api/v1/services/branding/ # Create branding
GET /api/v1/services/branding/{id}/ # Detail
PUT /api/v1/services/branding/{id}/ # Update
DELETE /api/v1/services/branding/{id}/ # Delete
# Report Templates
GET /api/v1/services/report-templates/ # List templates
POST /api/v1/services/report-templates/ # Create custom template
PUT /api/v1/services/report-templates/{id}/ # Update template
# Dashboard (Admin)
GET /api/v1/services/dashboard/ # Revenue, clients, MRR
GET /api/v1/services/dashboard/overdue/ # Overdue deliverables
GET /api/v1/services/dashboard/margin/ # Margin tracking
GET /api/v1/services/dashboard/clients/ # Client list with status summary
```
Celery tasks:
- `generate_weekly_reports` — every Monday 6am, generate for all Pro clients
- `generate_monthly_reports` — 1st of month, generate for all Lite + Pro clients
- `send_pending_reports` — daily, email any generated but unsent reports
- `check_overdue_deliverables` — daily, flag clients behind on content/links
- `collect_report_metrics` — nightly, pre-aggregate metrics for faster report generation
Services:
- `ReportService` — generate_report(), collect metrics from all sources, render PDF
- `ReportEmailService` — send report emails with PDF attachment or link
**Cross-references:** 04A (subscription + order data feeds reports), 02C (GSC data), 02E (backlink KPIs), 02H (social metrics), 01G (SAG health), 04C (feature flags control access)
---
## DOC 04C: pricing-launch.md
**File:** `V2-Execution-Docs/04C-pricing-launch.md`
**Scope:** Updated plan pricing structure, feature gating (which features at which plan level), WordPress.org plugin submission, go-to-market strategy, launch sequence, existing user migration.
**Depends On:** 04B (reporting and agency features complete)
### Source Material:
- Live Docs on Server/igny8-app-docs/plans/final-dev-guides-of-futute-implementation/DocD-Business-Services-Dev-Guide.md
- temp/IGNY8-Project-Files/IGNY8-Current-State.md (current plan structure)
### What to Cover:
**1. Current State:**
- 4 plans live (Free/$49/$149/$349)
- Stripe + PayPal integrated
- Feature gating exists via `Plan` model but minimal differentiation
- No WordPress.org submission done
- No formal go-to-market — current clients are direct Alorig relationships
- No managed services add-ons in billing flow
**2. What to Build:**
**Updated Plan Feature Matrix:**
Core SaaS plans remain unchanged in price. New V2 features are gated by plan:
| Feature | Free | Starter ($49) | Growth ($149) | Scale ($349) |
|---------|------|--------------|---------------|-------------|
| Sites | 1 | 3 | 10 | Unlimited |
| Credits/month | 100 | 1,000 | 5,000 | 25,000 |
| SAG Blueprint | Quick mode only | Quick + Detailed | Full | Full |
| Content Types | Post only | Post + Page | All types | All types |
| Taxonomy Content | — | — | ✓ | ✓ |
| GSC Integration | — | Basic (analytics only) | Full (+ indexing) | Full |
| Internal Linker | — | Audit only | Audit + auto-insert | Full + remediation |
| External Backlinks | — | — | Self-service | Self-service + API |
| Optimizer | — | Basic analysis | Full rewrite | Full + batch |
| Rich Schema | Basic (Article only) | 5 types | All 10 types | All + retroactive |
| Socializer | — | 2 platforms | All platforms | All + auto-schedule |
| Video Creator | — | — | Short-form only | All formats |
| Ahrefs Integration | — | — | Read-only metrics | Full (verify + prospect) |
| Backlink Indexing | — | — | — | ✓ (included) |
| Reports | — | — | Monthly PDF | Weekly + custom |
| White-Label | — | — | — | ✓ |
| API Access | — | — | Read-only | Full CRUD |
| Managed Services | — | — | Can purchase Lite | Can purchase Lite/Pro |
**Feature Flag System:**
Extend existing `Plan` model with feature capabilities:
New model: `PlanFeature` (or JSON field on Plan):
```
plan_features = {
"sag_mode": "quick|detailed|full",
"content_types": ["post"] | ["post", "page"] | "all",
"taxonomy_content": false | true,
"gsc_level": "none|basic|full",
"linker_level": "none|audit|auto|full",
"backlinks_level": "none|self_service|self_service_api",
"optimizer_level": "none|basic|full|batch",
"schema_types": 0 | 5 | 10 | "all_retroactive",
"socializer_platforms": 0 | 2 | "all" | "all_auto",
"video_level": "none|short|all",
"ahrefs_level": "none|readonly|full",
"backlink_indexing": false | true,
"report_level": "none|monthly|weekly_custom",
"white_label": false | true,
"api_access": "none|readonly|full",
"managed_services": "none|lite|lite_pro"
}
```
Feature gate checking pattern:
```python
# In any ViewSet or service
def check_feature(account, feature, required_level):
plan = account.subscription.plan
features = plan.plan_features # JSONField
current = features.get(feature, "none")
return LEVEL_HIERARCHY[feature].index(current) >= LEVEL_HIERARCHY[feature].index(required_level)
```
Middleware/decorator:
```python
@require_feature('linker_level', 'auto')
def auto_insert_links(self, request):
...
```
**Managed Services Add-On Pricing (in billing flow):**
| Add-On | Price | Available To |
|--------|-------|-------------|
| Managed Lite | $100/site/month | Growth + Scale plans |
| Managed Pro | $399/site/month | Scale plan only |
| Backlink Package (Starter) | $800-1,500/month | Growth + Scale |
| Backlink Package (Growth) | $2,000-4,000/month | Growth + Scale |
| Backlink Package (Authority) | $4,000-8,000/month | Scale only |
| Indexing Boost (standalone) | $0.15/URL | Growth + Scale |
**WordPress.org Plugin Submission:**
Plugin: IGNY8 SEO (standalone plugin from 03A, rebranded for WP.org)
Slug: `igny8-seo`
Version: 2.0.0
Submission checklist:
1. Code review: sanitization (all inputs), escaping (all outputs), nonce verification (all forms)
2. No external calls without user consent (phone-home disabled by default)
3. GPL v2+ license file included
4. readme.txt formatted per WP.org standards (short description, installation, FAQ, changelog, screenshots)
5. Assets: plugin banner (1544×500 and 772×250), icon (256×256 and 128×128), 8+ screenshots
6. Translation-ready: all strings wrapped in `__()`, `_e()`, `esc_html__()` with `igny8` text domain
7. No minified JS without source (include unminified versions)
8. Security: no direct file access without `ABSPATH` check, no `eval()`, no `file_get_contents()` for remote
9. Privacy: no tracking without consent, no storing user IPs (hash only)
10. Performance: no admin-wide asset loading, conditional enqueue
Review timeline: typically 1-4 weeks for initial review
Post-approval:
- Set up SVN deploy pipeline (GitHub → WP.org SVN)
- Auto-deploy on version tag via GitHub Actions
- Update checker: existing `/api/plugins/{slug}/check-update/` endpoint continues working for connected users
**Go-to-Market Strategy:**
Target audiences:
1. **WordPress site owners** — discover via WP.org plugin directory, free plugin → upsell to SaaS
2. **SEO agencies** — managed services + white-label, scale with backlink packages
3. **Content creators/bloggers** — SAG-powered content strategy, AI content generation
4. **eCommerce stores** — WooCommerce integration, product schema, service pages
5. **Local businesses** — LocalBusiness schema, service area pages, GSC monitoring
Funnel:
```
WP.org Plugin (free) → Install → Setup Wizard →
→ Free Plan (limited, 100 credits) → Experience value →
→ Upgrade to Starter/Growth/Scale →
→ Optional: Purchase Managed Services add-on →
→ Optional: Purchase Backlink Packages
```
**Launch Sequence:**
Phase 4A (Week 1-2): Build managed services backend + Ahrefs + indexing
Phase 4B (Week 2-3): Build reporting engine + white-label + dashboards
Phase 4C (Week 3-4): Feature gating + WP.org submission + pricing update
Launch steps:
1. Enable `managed_services_enabled` feature flag in staging
2. Onboard 2-3 existing Alorig clients as beta managed service subscribers
3. Test full cycle: onboarding → content → backlinks → indexing → reporting
4. Fix issues from beta
5. Submit WP.org plugin (parallel with beta testing)
6. Enable feature flags in production
7. Update pricing page on marketing site
8. Send announcement email to existing users
9. Begin WP.org organic acquisition
**Existing User Migration:**
- Current users keep their existing plan (grandfathered)
- New feature gates apply to new features only — nothing existing breaks
- Migration script: populate `plan_features` JSONField on all existing Plan records
- Existing credits + billing unchanged
**3. Data Models & APIs:**
Modified models:
- `Plan` (billing app) — add field:
- `plan_features` JSONField — feature capability matrix (see above)
New model:
- `FeatureUsageLog` (extends AccountBaseModel)
- `feature` CharField — feature key checked
- `required_level` CharField
- `current_level` CharField
- `was_allowed` BooleanField
- `endpoint` CharField — API endpoint that triggered check
- `timestamp` DateTimeField (auto_now_add)
- Table: `igny8_feature_usage_log`
- Purpose: track which features users hit gates on → inform pricing decisions
New endpoints:
```
# Feature Gates
GET /api/v1/billing/features/ # Current account's available features
GET /api/v1/billing/features/check/{feature}/ # Check specific feature access
# Plan Management (admin)
GET /api/v1/billing/plans/ # List all plans with features
PUT /api/v1/billing/plans/{id}/features/ # Update plan feature matrix
# Add-On Purchases
POST /api/v1/billing/add-ons/managed-service/ # Purchase managed service
POST /api/v1/billing/add-ons/backlink-package/ # Purchase backlink package
POST /api/v1/billing/add-ons/indexing-boost/ # Purchase indexing credits
GET /api/v1/billing/add-ons/ # List available add-ons for current plan
# WP.org Plugin (public)
GET /api/v1/plugins/igny8-seo/info/ # Plugin info for WP update checker
GET /api/v1/plugins/igny8-seo/download/ # Download latest ZIP
GET /api/v1/plugins/igny8-seo/check-update/ # Version check
```
Celery tasks:
- `migrate_plan_features` — one-time, populate plan_features on existing plans
- `sync_wporg_stats` — daily, fetch WP.org download/rating stats (if API available)
- `feature_usage_report` — weekly, aggregate FeatureUsageLog for product decisions
Services:
- `FeatureGateService` — check_feature(), get_plan_features(), log_usage()
- `AddOnPurchaseService` — process add-on purchases via existing Stripe/PayPal
- `WPOrgDeployService` — SVN deploy pipeline for WP.org (GitHub Actions config)
**Cross-references:** 04A (managed services pricing), 04B (reporting features gated by plan), All Phase 2 modules (each gated by plan level), 03A (WP.org plugin submission), existing billing app (Plan, Subscription models)
---
## BUILD SEQUENCE
```
Phase 4 Doc Build Order
═══════════════════════
1. 04A — Managed Services
Foundation: subscription model, onboarding, Ahrefs, indexing
(All other Phase 4 docs depend on this)
2. 04B — Whitelabel & Reporting
Builds on 04A subscriptions + pulls data from Phase 2 modules
3. 04C — Pricing & Launch
Wraps everything: feature gates across all modules, WP.org, go-to-market
(Must be last — references all features from all phases)
```
For each doc:
1. Read this plan's section for that doc
2. Read the source files listed in "Source Material"
3. Read relevant Phase 2 docs for data model cross-references
4. Write the doc following the DOC STRUCTURE STANDARD
5. Verify all PKs are BigAutoField (integer, not UUID)
6. Verify all model names are PLURAL where applicable
7. Verify all frontend references use .tsx + Zustand
8. Save to `V2-Execution-Docs/04X-filename.md`
---
## SOURCE FILES TO READ
```
Live Docs on Server/igny8-app-docs/plans/final-dev-guides-of-futute-implementation/DocD-Business-Services-Dev-Guide.md
temp/IGNY8-Project-Files/IGNY8-Current-State.md
temp/IGNY8-Project-Files/SAG-Doc4-External-Backlink-Campaign-PLAN.md
V2-Execution-Docs/00-MASTER-EXECUTION-PLAN.md
V2-Execution-Docs/02C-gsc-integration.md (reporting data source)
V2-Execution-Docs/02E-linker-external-backlinks.md (campaign/backlink models)
V2-Execution-Docs/02H-socializer.md (social metrics for reports)
V2-Execution-Docs/01G-sag-health-monitoring.md (health score for reports)
```
---
*This document is the single instruction set for building all 3 Phase 4 execution documents. Each doc section above contains the complete spec needed to write a self-contained execution doc that Claude Code can pick up and build independently.*

861
v2/V2-Execution-Docs/PHASE-5-BUILD-PLAN.md Normal file
View File

@@ -0,0 +1,861 @@
# Phase 5 Build Plan — Multi-App Deployment (1 Doc)
**Purpose:** This is the single instruction set for Claude Code to build the Phase 5 execution document (`05-other-apps-deployment.md`). This doc covers deploying all 6 remaining Alorig apps on the shared infrastructure established in Phase 0.
---
## CRITICAL BASELINE RULES
Before writing the doc, internalize these infrastructure-verified facts:
### Shared Infrastructure (Established in Phase 0)
- **VPS:** Hostinger KVM 4 — 4 vCPU, 16GB RAM, 200GB NVMe
- **Docker Network:** `alorig_net` (external, all apps join this network)
- **Shared Services** (alorig-infra docker-compose.yml):
- `alorig_postgres` — PostgreSQL 16, separate database per app
- `alorig_redis` — Redis 7, app-specific key prefixes (`psydge:`, `snapify:`, etc.)
- `alorig_caddy` — Caddy 2, reverse proxy + SSL via Cloudflare DNS challenge
- `portainer` — Portainer CE, Docker GUI for managing 25+ containers
- `flower` — mher/flower, Celery monitoring (on-demand)
- **DNS:** All domains managed in Cloudflare (free tier), wildcard A records to VPS IP
- **GitHub:** All repos on GitHub (private), one repo per app
- **IGNY8 already running:** Production + staging on new server (Phase 0–4 complete)
### Per-App Container Pattern (Standard)
Every Django app follows this identical structure:
- `{appname}_backend` — Django + Gunicorn (WSGI) or Daphne (ASGI)
- `{appname}_celery_worker` — Background task processing
- `{appname}_celery_beat` — Scheduled task scheduling
- React frontend built as static files → served via `alorig_caddy` (NO separate frontend container)
**Exceptions:**
- **Psydge:** Adds 4th container `psydge_channels` for Django Channels (ASGI/WebSocket)
- **Alorig Site Builder:** Adds `sitebuilder_payload` container for Node.js/Payload CMS alongside Django containers
### Alorig Tech Stack (Shared Across All Apps)
| Layer | Technology | Version |
|-------|-----------|---------|
| Backend | Django | 5.1–5.2+ |
| API | Django REST Framework | 3.15+ |
| Database | PostgreSQL | 16 (shared instance) |
| Cache/Broker | Redis | 7 (shared instance) |
| Task Queue | Celery | 5.3+ |
| Frontend | React + TypeScript | React 19, TS 5+ |
| State Mgmt | Zustand | 4–5 |
| Styling | Tailwind CSS | 3–4 |
| Build Tool | Vite | 5–6 |
| Containerization | Docker + Docker Compose | External network (alorig_net) |
| Reverse Proxy | Caddy 2 | Cloudflare DNS challenge SSL |
### Resource Budget (from Alorig Master Plan)
| Component | RAM Estimate | Notes |
|-----------|-------------|-------|
| Shared infra (PG + Redis + Caddy + Portainer) | ~2.5 GB | PostgreSQL serving 7 databases is biggest consumer |
| IGNY8 (prod + staging) | ~1.8 GB | Already running |
| Psydge | ~1.75 GB | WebSockets hold memory per connection; heaviest at 30 users |
| Snapify | ~700 MB | Standard CRUD with image processing |
| Observer OS | ~700 MB | AI pattern detection is external API calls |
| Alorig Site Builder | ~1.25 GB | Django + Payload/Node.js dual runtime |
| AstroTiming | ~400 MB | Lightest app. Brief CPU bursts for ephemeris |
| ASMS Phase 1 | ~600 MB | Pilot scale, basic modules only |
| OS + Docker overhead | ~1.0 GB | |
| **TOTAL STEADY STATE** | **~10.7 GB / 16 GB** | **~5 GB headroom for spikes** |
**Upgrade trigger:** Steady-state RAM consistently above 13 GB, or OOM kills on any container. Options: upgrade to KVM 8, or split Psydge to its own KVM 1.
### Source of Truth
- Start the doc with: `**Source of Truth:** Alorig-Master-Plan-v1.md + individual app blueprints`
- Infrastructure decisions from Alorig Master Plan override individual app docs
---
## DOC STRUCTURE STANDARD
The `05-other-apps-deployment.md` doc MUST follow this structure:
```
# Phase 5 — Multi-App Deployment
**Source of Truth:** Alorig-Master-Plan-v1.md + individual app blueprints
**Version:** 1.0
**Date:** 2026-03-23
## 1. Current State
## 2. What to Build
## 3. Shared Deployment Pattern
## 4. App 1: Psydge
## 5. App 2: Snapify
## 6. App 3: Alorig Site Builder
## 7. App 4: Observer OS
## 8. App 5: AstroTiming
## 9. App 6: ASMS Phase 1
## 10. Post-Deployment Monitoring
## 11. Acceptance Criteria
## 12. Claude Code Instructions
```
---
## DEPLOYMENT ORDER & RATIONALE
The apps deploy in this specific order, one at a time. Monitor RAM after each before proceeding to next.
| # | App | Why This Order | Trigger |
|---|-----|----------------|---------|
| 1 | **Psydge** | Most complex (WebSockets, Django Channels, real-time broker connection). Deploy early to validate server under real load. | After IGNY8 stable on new server |
| 2 | **Snapify** | Standard Django stack. Good validation that multi-app pattern works. WhatsApp integration adds external service validation. | After Psydge stable |
| 3 | **Alorig Site Builder** | Dual runtime (Django + Payload CMS/Node.js). Tests Node.js container alongside Django stack. | After Snapify stable |
| 4 | **Observer OS** | Standard Django + React. Light resource requirements initially. Planetary alignment layer shares ephemeris logic with AstroTiming. | When ready |
| 5 | **AstroTiming** | Lightest app. Minimal containers needed. Can share `pyswisseph` ephemeris data cache with Observer OS. | When ready |
| 6 | **ASMS Phase 1** | School management MVP. Basic modules for pilot deployment at 74 Schools. | When ready |
---
## APP 1: PSYDGE — AI Trading Discipline Platform
### Product Summary
Psydge (Psychological Edge) is a web-based trading discipline and execution platform. It sits between the trader and their MT4/MT5 broker via MetaApi, enforcing hard-coded discipline rules that structurally prevent self-destructive trading behavior. Two products under one platform: Psydge Core (discipline engine) and Vista Strategies (pre-defined strategy suite).
### Source Documents
- `Psydge/Psydge_Technical_Documentation.pdf` (26 pages)
- `Psydge/psydge-project/` (project scaffold if exists)
### Tech Stack Specifics (Beyond Alorig Standard)
| Addition | Technology | Purpose |
|----------|-----------|---------|
| Real-Time | Django Channels + Redis | WebSocket for live price feeds, position updates, gate status, AI messages |
| Broker Bridge | MetaApi (Cloud) | MT4/MT5 connectivity, order execution, position polling via REST + WebSocket |
| AI | Claude API (Anthropic) | Trade supervision, behavioral prediction, pre-entry validation, session debriefs |
| Charts | TradingView Lightweight Charts | Live candlestick charts, M1 through D1 timeframes |
| Billing | Stripe | Subscription management (Psydge Core + Vista Strategies tiers) |
### Container Architecture (4 containers — exception to standard 3)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `psydge_backend` | psydge-backend:latest | Django + Gunicorn (WSGI for REST API) | 8012 |
| `psydge_channels` | psydge-backend:latest | Django + Daphne (ASGI for WebSocket) | 8013 |
| `psydge_celery_worker` | psydge-backend:latest | Celery worker (AI calls, behavioral analysis, scoring) | — |
| `psydge_celery_beat` | psydge-backend:latest | Celery beat scheduler | — |
### Database
- Database name: `psydge_db` on shared `alorig_postgres`
- Redis key prefix: `psydge:`
- Redis DB: 2 (separate from IGNY8's DB 0 and staging DB 1)
### Core Data Models
| Model | Storage | Key Fields | Purpose |
|-------|---------|-----------|---------|
| User | PostgreSQL | id, email, timezone, broker_credentials (encrypted), subscription_tier | User account |
| BrokerConnection | PostgreSQL + Redis | user_id, broker, server, account_number, connection_status | MT4/MT5 connection state |
| DisciplineConfig | PostgreSQL | user_id, loss_lock_r, profit_lock_r, cooldown_hours, max_risk_pct, session_hours, symbol_whitelist | Per-user discipline parameters |
| GateState | Redis | user_id → hash: buffer_go, buffer_whitelist, buffer_session, buffer_equity, buffer_cooldown, buffer_risk | Real-time gate buffer values (sub-ms reads) |
| Trade | PostgreSQL | id, user_id, symbol, direction, entry, sl, tp, lot, risk_pct, r_result, strategy, status, opened_at, closed_at | Complete trade record |
| Violation | PostgreSQL | id, user_id, violation_code, attempted_action, timestamp, gate_state_snapshot | Every blocked trade attempt |
| DisciplineScore | PostgreSQL | user_id, date, score, metric_breakdown (JSON) | Daily discipline score history |
| ReflectionEntry | PostgreSQL | user_id, date, session_notes, ai_prompt, ai_summary | Session journal entries |
| AIInteraction | PostgreSQL | id, user_id, trade_id, role (supervisor/validator/predictor), input_context, output, timestamp | AI call history for audit |
| BehavioralPattern | PostgreSQL | user_id, pattern_type, occurrence_count, last_instance, trend | Detected violation patterns |
### Redis State Schema
| Key Pattern | Type | TTL | Content |
|-------------|------|-----|---------|
| `psydge:gate:{user_id}` | Hash | None (persistent) | All 6 buffer values |
| `psydge:cooldown:{user_id}` | String | Cooldown duration | Timestamp when cooldown expires |
| `psydge:session_r:{user_id}` | String | End of day | Cumulative R for current session |
| `psydge:positions:{user_id}` | Hash | None (live sync) | Open position data from MetaApi |
| `psydge:prices:{symbol}` | String | 5 seconds | Latest bid/ask from MetaApi stream |
| `psydge:lock:{user_id}` | String | End of session | Session lock flag when equity guard triggers |
### Caddy Routes
```
psydge.com → static React build (frontend)
api.psydge.com → psydge_backend:8012 (REST API)
ws.psydge.com → psydge_channels:8013 (WebSocket)
```
### Key Discipline Engine Components
- **6 Gate Buffers:** GO Signal (B), Symbol+TF Whitelist (C), Session Filter (D), R-Based Equity Guard (E), Cooldown Timer (H), Risk % Check (K)
- **Locked Rules (Non-Negotiable):** Loss Lock (-2R daily), Profit Lock (+3R daily), 4hr Cooldown, 1.0% risk per trade, London+NY sessions only, Psydge web panel only execution surface
- **Violation Response Levels:** Level 1 (Alert) → Level 2 (Overlay) → Level 3 (Blackout) → Level 4 (Session Lock)
- **Discipline Score:** 0–100 weighted composite: Rule Compliance 25%, Session Discipline 15%, Risk Management 15%, Patience Score 15%, Post-Win Control 15%, Cooldown Respect 10%, Revenge Prevention 5%
### 4 AI Roles (All via Claude API + Celery)
| Role | Trigger | Input | Output |
|------|---------|-------|--------|
| Trade Supervisor | Trade hits TP1 or R threshold | Symbol, entry, SL/TP, duration, delta, trend, TM score | scale / hold / tighten_sl / exit |
| Pre-Entry Validator | User initiates new trade (if enabled) | Setup context, TM score, multi-TF alignment, recent trade history | approve / stand_down + reasoning |
| Behavioral Predictor | Session start + post-trade | User's violation history, trade patterns, time patterns, win/loss sequence | Risk alerts + enforcement tightening suggestions |
| Session Debrief | Session end | All trades, violations, discipline score, behavioral data | Natural language session summary + improvement insights |
### Vista Strategy Suite (4 Tiers)
| Strategy | Timeframe | Style | Target R |
|----------|-----------|-------|----------|
| Scalper Engine | M1–M5 | Delta breakouts, microstructure | 1.5–2R |
| Intraday Momentum | M15–H1 | Trend trades with intelligent scaling | 2–7R |
| Swing Position | H1–D1 | HTF structure, trend reclaim | 5–12R |
| Vault Trail | H4–D1 | Receives winning handoffs from other strategies | 15–20R |
### 6 Proprietary Indicators
NDNS (Net Delta/Net Strength), SV (Structure Validation), TM (Trend-Momentum), MOM (Momentum Oscillator), FHS (Fractal/Harmonic Structure), HAMFIST (Harmonic + Fib Integrated)
### Celery Tasks
| Task | Schedule | Function |
|------|----------|----------|
| `calculate_discipline_score` | End of each trading session | Compute daily discipline score from all trade/violation data |
| `detect_behavioral_patterns` | After every violation | Update BehavioralPattern records, check for new patterns |
| `ai_session_debrief` | Session end | Generate AI session summary via Claude API |
| `sync_metaapi_positions` | Every 5 seconds (per connected user) | Pull position/equity data from MetaApi WebSocket |
| `weekly_behavioral_report` | Sunday 23:00 UTC | Generate weekly AI behavioral report per user |
### Implementation Steps (for 05 doc)
1. Create GitHub repo `psydge` with Django project scaffold
2. Configure `docker-compose.psydge.yml` (4 containers on `alorig_net`)
3. Create `psydge_db` database on shared PostgreSQL
4. Build Django models, migrations, admin
5. Build Discipline Engine (gate buffers, violation logging, enforcer)
6. Integrate MetaApi SDK (broker connection, order flow, position sync)
7. Build WebSocket consumers (Django Channels) for real-time data
8. Build Vista strategy engine (indicator calculations, TM scoring)
9. Integrate Claude API for 4 AI roles via Celery tasks
10. Build React frontend (trade panel, analytics dashboard, settings)
11. Configure Caddy routes for psydge.com, api.psydge.com, ws.psydge.com
12. Build Stripe subscription integration
13. Deploy, smoke test with paper trading account
14. Monitor RAM usage — expected ~1.75 GB steady state
---
## APP 2: SNAPIFY — Pakistani E-Commerce Platform
### Product Summary
Snapify.pk is a mobile-first, AI-powered e-commerce platform enabling Pakistani micro-sellers to create online stores via WhatsApp. Three-interface architecture: buyer-facing store (server-rendered Django + Tailwind for SEO), seller dashboard (React 19 PWA), admin panel (React 19 SPA). Target: 5M+ informal sellers, breakeven at ~150 paying users.
### Source Documents
- `Snapify/Snapify-Blueprint-v3-Complete.pdf` (34 pages)
### Tech Stack Specifics (Beyond Alorig Standard)
| Addition | Technology | Purpose |
|----------|-----------|---------|
| WhatsApp | WhatsApp Business Cloud API | Onboarding bot, order notifications, seller commands |
| Image CDN | Cloudinary (free tier + scale) | Image optimization, responsive srcset, WebP |
| File Storage | DigitalOcean Spaces (S3-compatible) | Product images, generated assets, backups |
| Social APIs | Meta Graph API (FB + Instagram) | Auto-posting, story creation for Pro sellers |
| AI Engine | OpenAI GPT-4 API + Whisper API | Content generation, image analysis, speech-to-text (Urdu/Punjabi) |
| Email | SendGrid | Transactional emails, seller notifications |
| Search | PostgreSQL Full-Text + pg_trgm | Product search, store discovery |
| Monitoring | Sentry + custom middleware | Error tracking, performance monitoring |
| Payment | JazzCash / EasyPaisa / Stripe (future) | Pakistani payment gateways |
### Container Architecture (3 containers — standard pattern)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `snapify_backend` | snapify-backend:latest | Django + Gunicorn (serves REST API + server-rendered store pages) | 8014 |
| `snapify_celery_worker` | snapify-backend:latest | Celery worker (AI processing, image optimization, social posting, SEO gen) | — |
| `snapify_celery_beat` | snapify-backend:latest | Celery beat (sitemap regen, hub page regen, analytics aggregation) | — |
**Note:** Buyer-facing store is server-rendered Django templates (NOT React) — critical for SEO on 3G connections. Seller dashboard and admin panel are React builds served as static files via Caddy.
### Database
- Database name: `snapify_db` on shared `alorig_postgres`
- Redis key prefix: `snapify:`
- Redis DB: 3
### Three-Interface Architecture
| Interface | Users | Technology | URL |
|-----------|-------|-----------|-----|
| Buyer-Facing Store | Buyers / public | Server-rendered Django + Tailwind CSS | snapify.pk/store-name |
| Seller Dashboard | Store owners / sellers | React 19 PWA (installable on phone) | app.snapify.pk |
| Admin Panel | Alorig team / support | React 19 internal SPA | admin.snapify.pk |
### Core Data Models (Key Entities)
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| Store | id, seller_id, name, slug, city, category, custom_domain, template, subscription_tier, is_verified | Multi-tenant store entity |
| Product | id, store_id, title, description, price, images (JSON), category_id, variants (JSON), is_active, seo_meta | Product listing |
| Order | id, store_id, buyer_name, buyer_phone, products (JSON), total, status (pending/confirmed/shipped/delivered/cancelled), tracking_number | WhatsApp-native order lifecycle |
| Category | id, store_id, name, slug, product_count | Store-level product categories |
| Analytics | id, store_id, page_type, page_slug, visitor_ip_hash, referrer, city, timestamp | Per-pageview analytics |
| Subscription | id, store_id, plan (free/pro), started_at, expires_at, payment_method | Seller subscription |
| Post | id, store_id, platform, content, image_url, scheduled_at, status, engagement_metrics | Social auto-posting records |
| WhatsAppSession | id, store_id, phone_hash, state, last_message_at | Onboarding conversation state machine |
### Multi-Tenant Architecture
- `StoreBaseModel` — FK to Store with auto-assignment (equivalent to IGNY8's `SiteSectorBaseModel`)
- `StoreModelViewSet` — all queries scoped to store
- `StoreMiddleware` — resolves tenant from URL path (snapify.pk/store) or custom domain lookup
- Custom domain support: buyer visits `silkhouse.pk` → CNAME to snapify.pk → StoreMiddleware resolves via `custom_domain` field
### AI Pipeline (5 Stages via Celery)
| Stage | Input | Output | Technology |
|-------|-------|--------|-----------|
| Image Analysis | Product photos | Category, color, material, style detection | GPT-4 Vision API |
| Voice Processing | WhatsApp voice notes | Structured text (product, price, description) | Whisper API + custom parser |
| Content Generation | Product metadata + category | Title, description, bullets, meta tags | GPT-4 with retail prompts |
| SEO Optimization | Generated content + city data | City-specific landing pages, schema markup | Custom SEO templates + GPT |
| Social Content | Product images + AI content | Post-ready images with overlays and CTAs | Pillow image engine + GPT captions |
### Store Page Types (Server-Rendered Django Templates)
| Page Type | URL Pattern | Rendering |
|-----------|------------|-----------|
| Store Homepage | snapify.pk/store-name | Django template, product grid, category filters |
| Product Detail | snapify.pk/store-name/product-slug | Full product layout with Schema.org markup |
| Category Page | snapify.pk/store-name/category-slug | Filtered product grid |
| City Landing (Pro) | snapify.pk/store-name/delivery-to-lahore | Programmatically generated, geo-targeted copy |
| Store About | snapify.pk/store-name/about | Seller info, trust signals, verified badge |
| Category Hub (Platform) | snapify.pk/best-clothing-stores-punjab | Aggregated cross-store listings |
### Performance Targets (3G Pakistan)
| Metric | Target |
|--------|--------|
| First Contentful Paint | < 1.0s on 3G |
| Largest Contentful Paint | < 2.0s on 3G |
| Total Page Size | < 200KB |
| PageSpeed Score | 95+ (mobile) |
### Caddy Routes
```
snapify.pk → snapify_backend:8014 (server-rendered store pages)
api.snapify.pk → snapify_backend:8014/api/ (REST API)
app.snapify.pk → static React PWA build (seller dashboard)
admin.snapify.pk → static React SPA build (admin panel, IP-restricted)
```
### Pricing
| Plan | Price | Limits |
|------|-------|--------|
| Free | Rs. 0 | 15 products, basic dashboard, Snapify branding |
| Pro | Rs. 2,500/month | Unlimited products, custom domain, analytics, social posting, no branding |
### Celery Tasks
| Task | Schedule | Function |
|------|----------|----------|
| `process_whatsapp_webhook` | On webhook receipt | Parse incoming WhatsApp messages, route to handler |
| `generate_product_content` | On product photo upload | AI image analysis + content generation pipeline |
| `generate_city_pages` | Weekly (Pro stores) | Programmatic city landing page generation |
| `regenerate_sitemap` | Daily 2 AM | Per-store XML sitemap regeneration |
| `regenerate_hub_pages` | Weekly Sunday 3 AM | Platform-level category hub page regeneration |
| `social_auto_post` | Per schedule | Post to FB/IG via Meta Graph API |
| `invalidate_store_cache` | On product/store change | Clear Redis-cached rendered HTML pages |
### Implementation Steps
1. Create GitHub repo `snapify` with Django project scaffold
2. Configure `docker-compose.snapify.yml` (3 containers on `alorig_net`)
3. Create `snapify_db` database on shared PostgreSQL
4. Build multi-tenant models (Store, Product, Order, Category, Analytics)
5. Build WhatsApp Business API integration (webhook receiver, state machine onboarding)
6. Build server-rendered Django store templates (6 page types, 7 store templates)
7. Build AI content generation pipeline (5 stages via Celery)
8. Build React seller PWA dashboard (products, orders, analytics, settings)
9. Build React admin panel (store management, moderation, platform metrics)
10. Build SEO engine (programmatic city pages, schema markup, sitemap generator)
11. Integrate Cloudinary for image CDN
12. Configure Caddy routes for all 4 subdomains
13. Integrate JazzCash/EasyPaisa payment for Pro subscriptions
14. Deploy, test WhatsApp onboarding flow end-to-end
15. Monitor RAM usage — expected ~700 MB steady state
---
## APP 3: ALORIG SITE BUILDER — Multi-Tenant Website Platform
### Product Summary
AI-powered multi-tenant website builder platform. Dual runtime architecture with Django handling business logic/API and Payload CMS (Node.js) handling content management and visual editing. This is the only app with a Node.js container alongside Django.
### Source Documents
- `Alorig Site Builder/Alorig-Site-Builder-Blueprint.docx`
### Tech Stack Specifics (Beyond Alorig Standard)
| Addition | Technology | Purpose |
|----------|-----------|---------|
| CMS | Payload CMS (Node.js) | Visual content editing, block-based page builder |
| Node Runtime | Node.js 18+ | Payload CMS server alongside Django |
| AI | OpenAI GPT-4 / Claude | AI-powered content generation, design suggestions |
### Container Architecture (4 containers — exception with Node.js)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `sitebuilder_backend` | sitebuilder-backend:latest | Django + Gunicorn (REST API, auth, billing, tenant management) | 8016 |
| `sitebuilder_payload` | sitebuilder-payload:latest | Payload CMS (Node.js, content editing, block rendering) | 8017 |
| `sitebuilder_celery_worker` | sitebuilder-backend:latest | Celery worker (AI generation, site builds, deployments) | — |
| `sitebuilder_celery_beat` | sitebuilder-backend:latest | Celery beat scheduler | — |
### Database
- Database name: `sitebuilder_db` on shared `alorig_postgres`
- Redis key prefix: `sitebuilder:`
- Redis DB: 4
- Payload CMS: Can share same PostgreSQL database or use separate `sitebuilder_payload_db`
### Caddy Routes
```
alorig.com/builder → static React build (builder dashboard)
api.alorig.com/builder → sitebuilder_backend:8016 (Django API)
cms.alorig.com → sitebuilder_payload:8017 (Payload CMS)
*.sites.alorig.com → rendered sites (wildcard)
```
### Implementation Steps
1. Create GitHub repo `alorig-site-builder` with Django + Payload dual scaffold
2. Configure `docker-compose.sitebuilder.yml` (4 containers on `alorig_net`)
3. Create `sitebuilder_db` database
4. Build Django models (Tenant/Site, User, Subscription, Deployment)
5. Build Payload CMS configuration (block types, page schemas, media)
6. Build Django↔Payload sync layer (auth sharing, data bridge)
7. Build AI content generation pipeline
8. Build React dashboard (site management, templates, billing)
9. Configure Caddy routes including wildcard for rendered sites
10. Deploy, test multi-tenant site creation end-to-end
11. Monitor RAM usage — expected ~1.25 GB steady state
---
## APP 4: OBSERVER OS — Consciousness/Behavioral Awareness Platform
### Product Summary
Observer OS (0·i·G·8) is a self-operating behavioral operating system. Four progressive symbolic modules (0: Dissolution → i: Identity → G: The Gate → 8: Infinity) guide users through observation of ego patterns via AI-powered journaling, pattern detection, ambient environment adaptation, and planetary alignment layers. NOT a meditation app — it's an architecture of awakening.
### Source Documents
- `oigate/Observer-OS-Project-Specification-v1.pdf` (34 pages)
- `oigate/site/` (project scaffold if exists)
### Tech Stack Specifics (Beyond Alorig Standard)
| Addition | Technology | Purpose |
|----------|-----------|---------|
| AI (Primary) | OpenAI GPT-4o | Text analysis, pattern detection, ambient generation, module transitions |
| AI (Secondary) | Anthropic Claude | Nuanced paradox detection, reflective prompt generation |
| TTS/Audio | OpenAI TTS / ElevenLabs | Ambient sound generation, optional voice responses |
| Image Gen | Runware / Stable Diffusion | Ambient visual generation, thumbnail creation |
| Ephemeris | Swiss Ephemeris (pyswisseph) or AstroAPI | Planetary position data for alignment layer |
| Push | FCM/APNS | Push notifications (web + future mobile) |
| Mobile (Phase 2) | React Native or Flutter | iOS + Android (future) |
### Container Architecture (3 containers — standard)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `observer_backend` | observer-backend:latest | Django + Gunicorn (REST API) | 8018 |
| `observer_celery_worker` | observer-backend:latest | Celery worker (AI analysis, pattern detection, ambient refresh) | — |
| `observer_celery_beat` | observer-backend:latest | Celery beat (scheduled pattern analysis, planetary updates, data cleanup) | — |
### Database
- Database name: `observer_db` on shared `alorig_postgres`
- Redis key prefix: `observer:`
- Redis DB: 5
### Django App Structure
| Directory | Purpose | Contents |
|-----------|---------|----------|
| `auth/` | Authentication | User model, anonymous auth, JWT, account context middleware |
| `api/` | API infrastructure | Base ViewSets, authentication classes, pagination, response format |
| `modules/observer/` | Observer module API | Module state endpoints, journal API, pattern API, dashboard API |
| `modules/ambient/` | Ambient environment API | Color state, sound state, planetary overlay endpoints |
| `modules/system/` | System configuration | Settings, AI model config, prompt templates |
| `business/patterns/` | Pattern detection services | PatternService, EgoLoopDetector, FlowAnalyzer, ContradictionEngine |
| `business/modules/` | Module transition logic | ModuleTransitionService, state machine, trigger evaluation |
| `business/journal/` | Journaling services | JournalService, NLP processing, prompt generation |
| `business/ambient/` | Ambient environment | ColorEngine, SoundEngine, PlanetaryService |
| `ai/` | AI engine | AIEngine, function registry, providers (OpenAI, Anthropic), model registry |
| `ai/functions/` | AI function implementations | AnalyzeJournal, DetectPatterns, GeneratePrompt, DetectParadox, GenerateAmbient, AssessTransition |
| `tasks/` | Celery task definitions | PatternAnalysisTask, ModuleTransitionTask, PlanetaryUpdateTask, AmbientRefreshTask |
### Core Data Models
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| User | id, email (optional), created_at, current_module, preferences, anonymous_token | Anonymous-first user account |
| ModuleState | user_id, active_module (0/i/G/8), entered_at, transition_signals, accumulated_score | Tracks active module and transition readiness |
| JournalEntry | user_id, text (encrypted), emotion_tag, timestamp, module_at_time, ai_analysis_summary | Encrypted journal entries with optional AI analysis |
| Pattern | user_id, pattern_type, label, frequency, last_occurrence, first_detected, evidence_refs | Detected behavioral patterns across all input channels |
| StateLoop | user_id, loop_id, triggers, behaviors, resolution_state, occurrence_count | Ego loops: trigger → reaction → resolution tracking |
| Event | user_id, timestamp, module, trigger_type, text_input, system_reaction | Granular event log for behavioral analysis |
| AmbientState | user_id, current_palette, current_sound, planetary_context, last_updated | Current ambient environment configuration |
| PlanetarySnapshot | date, planetary_positions, major_transits, awareness_theme | Daily planetary data cache for alignment layer |
| PromptTemplate | module, trigger_context, prompt_text, usage_count, effectiveness_score | AI prompt templates per module and context |
| SystemConfig | key, value, category | Global system configuration (AI models, feature flags) |
### 4 Core Modules (0 → i → G → 8)
| Module | Symbol | Role | AI Processing |
|--------|--------|------|--------------|
| Dissolution | 0 | Disengage the machinery of "I" — break reactive identification | NLP analysis for identity-reinforcement patterns, ambient adaptation |
| Identity | i | Surface and expose egoic patterning — mirror the constructed self | Longitudinal pattern recognition (2-4 weeks), ego-pattern classification, dynamic prompt generation |
| The Gate | G | Trigger collapse of division between observer and observed | Contradiction detection, paradox generation, context-aware silence |
| Infinity | 8 | Sustain choiceless awareness and non-fragmented perception | Flow state detection, ambient adaptation, self-obsolescence logic |
### Module Transition Logic (Automatic, Never Announced)
| From → To | Trigger | User Experience |
|-----------|---------|---------------|
| 0 → i | Emergence of habitual pattern, ego language, or resistance to stillness | Prompts shift from silence to reflection; journal prompts appear |
| i → G | Observer begins observing the observer — paradox or deep contradiction arises | Interface becomes more minimal; paradox questions surface |
| G → 8 | Surrender occurs — "I" loses center stage, replaced by ambient being | System recedes; ambient environment takes over; almost no text |
| 8 → 0 | Identity re-coagulates or new life loop forms | Gentle restart — not regressive, but rhythmic; cycle begins again |
### Ambient Environment System
**Color System:**
| State | Color Palette | Module Alignment |
|-------|--------------|-----------------|
| Calm / Presence | Deep indigo, soft violet, midnight blue | Module 8 (Infinity) |
| Alert / Activation | Amber, warm gold, burnt orange | Module i (Identity) |
| Stillness / Dissolution | Charcoal, muted slate, near-black | Module 0 (Dissolution) |
| Threshold / Paradox | Silver-white, pale lavender, stark contrast | Module G (Gate) |
| Agitation / Resistance | Desaturated tones, slight red warmth | Any module (friction detected) |
### Privacy Architecture (Foundational, Not Feature)
| Tier | Location | Data Type | Encryption |
|------|----------|-----------|-----------|
| Tier 1 (Local Only) | Device storage | Journal entries, emotion tags, raw behavioral data | Device-level encryption |
| Tier 2 (Encrypted Sync) | Cloud database | Pattern summaries, module state, preferences, anonymized loop data | AES-256 at rest, TLS in transit |
| Tier 3 (Ephemeral) | Cloud AI API | Anonymized text snippets for NLP processing | Processed and discarded, never stored |
### API Endpoint Structure
| Endpoint Group | Base Path | Key Endpoints |
|---------------|-----------|--------------|
| Auth | /api/v1/auth/ | register, login, anonymous-login, token-refresh, delete-account |
| Module State | /api/v1/observer/module/ | current-state, transition-history, signal-log |
| Journal | /api/v1/journal/ | create, list, export, delete-all |
| Patterns | /api/v1/patterns/ | list, detail, loops, flow-states, contradictions |
| Ambient | /api/v1/ambient/ | current-state, update-palette, update-sound, planetary-context |
| Dashboard | /api/v1/dashboard/ | flow-summary, module-view, pattern-overview |
| Chat | /api/v1/chat/ | send-message, get-prompt, silence-mode |
| Settings | /api/v1/settings/ | preferences, notifications, privacy, data-export |
### Celery Tasks
| Task | Queue | Schedule | Function |
|------|-------|----------|----------|
| Pattern Analysis | analysis | Every 6 hours (active users) | Process accumulated behavioral data, update Pattern records |
| Module Transition Check | transitions | Every 2 hours (active users) | Evaluate transition signals, trigger module shifts if thresholds met |
| Planetary Update | planetary | Daily at midnight | Pull ephemeris data, generate awareness themes, cache snapshot |
| Ambient Refresh | ambient | On state change or hourly | Recalculate color/sound environment based on current signals |
| Journal NLP Processing | analysis | On new entry (debounced 5 min) | Run AI analysis on journal entry, update pattern data |
| Data Cleanup | maintenance | Weekly | Purge ephemeral AI data, compress old event logs, update aggregates |
### Caddy Routes
```
observeros.app → static React build (web app)
api.observeros.app → observer_backend:8018 (REST API)
```
### Implementation Steps
1. Create GitHub repo `observer-os` with Django project scaffold
2. Configure `docker-compose.observer.yml` (3 containers on `alorig_net`)
3. Create `observer_db` database
4. Build Django models with encryption (JournalEntry text field encrypted at rest)
5. Build anonymous auth system (no email required for core functionality)
6. Build 4-module state machine (ModuleTransitionService)
7. Build pattern detection engine (PatternService, EgoLoopDetector, FlowAnalyzer, ContradictionEngine)
8. Build journaling API with NLP processing pipeline
9. Build ambient environment system (ColorEngine, SoundEngine, PlanetaryService)
10. Integrate AI providers (OpenAI primary, Claude secondary) via function registry
11. Build React frontend (dual interface: conversational chat + visual dashboard)
12. Build ambient components (color engine, sound player, planetary overlay)
13. Configure Caddy routes
14. Deploy, test module transition flow end-to-end
15. Monitor RAM usage — expected ~700 MB steady state
---
## APP 5: ASTROTIMING — Planetary Hours Intelligence Engine
### Product Summary
AstroTiming transforms ancient celestial timing wisdom (Al Saat by Kash Al Barni) into a modern API-powered productivity tool. Three-layer architecture: Layer 1 (Classical Planetary Hours), Layer 2 (Live Planetary Positions via Swiss Ephemeris), Layer 3 (Intelligence Engine with Timing Quality Score 0–100). Three products: consumer web app, developer API, premium scheduling layer.
### Source Documents
- `Astro Timing/AstroTiming-System-Blueprint.pdf` (5+ pages)
### Tech Stack Specifics (Beyond Alorig Standard)
| Addition | Technology | Purpose |
|----------|-----------|---------|
| Ephemeris | pyswisseph (Python Swiss Ephemeris bindings) | Sub-arcsecond planetary position calculations |
| Sunrise/Sunset | SunCalc or astronomical calculation | Precise sunrise/sunset for planetary hour computation |
| AI (optional) | OpenAI GPT-4o | Natural language timing recommendations, scheduling suggestions |
| Calendar | Google Calendar API / CalDAV | Premium scheduling layer integration |
### Container Architecture (3 containers — standard, lightest app)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `astrotiming_backend` | astrotiming-backend:latest | Django + Gunicorn (REST API + consumer web) | 8020 |
| `astrotiming_celery_worker` | astrotiming-backend:latest | Celery worker (ephemeris pre-computation, AI recommendations) | — |
| `astrotiming_celery_beat` | astrotiming-backend:latest | Celery beat (daily planetary data cache, user calendar sync) | — |
**Note:** AstroTiming is the lightest app (~400 MB). Ephemeris calculations are CPU-burst (brief) but the pyswisseph library uses zero external dependencies — all calculations are local.
### Database
- Database name: `astrotiming_db` on shared `alorig_postgres`
- Redis key prefix: `astrotiming:`
- Redis DB: 6
### Three-Layer Engine Architecture
**Layer 1: Classical Planetary Hours (Al Saat Ruleset)**
- Deterministic. Requires only geographic location + date/time
- Sunrise/sunset → 12 unequal daytime + 12 unequal nighttime planetary hours (24 total)
- Chaldean Order assignment: Saturn → Jupiter → Mars → Sun → Venus → Mercury → Moon
- Day ruler = first hour ruler
- Activity-planet mapping from Al Saat's complete taxonomy
- Works entirely offline, zero external dependencies
**Layer 2: Live Planetary Positions (Swiss Ephemeris)**
- Exact ecliptic longitude of all 7 classical planets at any given moment
- Planetary dignities: domicile, exaltation, detriment, fall
- Retrograde status
- Planetary aspects: conjunction, sextile, square, trine, opposition
- Lunar phase and Moon sign
- Moon void-of-course periods
- NASA JPL DE431 data: 13201 BCE to 17191 CE, 0.1 arcsecond precision
**Layer 3: Intelligence Engine (Scoring & Recommendations)**
- **Timing Quality Score (TQS):** 0–100 composite score for any activity at any moment
- **Forward Search:** Given an activity, scan upcoming hours/days and find optimal windows
- **Smart Scheduler:** Given user activities + priorities, generate optimized weekly/monthly calendar
- **Conflict Detection:** Flag activities in unfavorable windows with suggested alternatives
### TQS Scoring Model
| Factor | Weight | Description |
|--------|--------|-------------|
| Hour Ruler Match | 30% | Does the planetary hour ruler align with the activity? (Al Saat mapping) |
| Day Ruler Harmony | 15% | Does the day ruler support or conflict with the hour ruler? |
| Planetary Dignity | 15% | Is the hour ruler in domicile/exaltation (boost) or detriment/fall (penalty)? |
| Lunar Condition | 15% | Moon phase, sign, void-of-course status, aspects to hour ruler |
| Planetary Aspects | 15% | Major aspects between hour ruler and benefics/malefics |
| Retrograde Status | 10% | Is the hour ruler or key planet for the activity retrograde? |
Score interpretation: 80–100 Excellent, 60–79 Good, 40–59 Neutral, 20–39 Unfavorable, 0–19 Avoid.
### Planet-Activity Mapping (Core Ruleset from Al Saat)
| Planet | Arabic | Domains |
|--------|--------|---------|
| Sun (Shams) | Authority, Vitality, Success | Leadership, health, government, recognition |
| Moon (Qamar) | Emotions, Travel, Public | Travel, public/social, domestic, emotional |
| Mars (Mirrikh) | Action, Competition, Courage | Physical action, competition, bold ventures |
| Mercury (Utarid) | Communication, Commerce, Learning | Writing, commerce, learning, technology |
| Jupiter (Mushtari) | Expansion, Wealth, Spirituality | Business expansion, education, spiritual practice |
| Venus (Zuhra) | Beauty, Love, Harmony | Arts, relationships, beauty, social gatherings |
| Saturn (Zuhal) | Discipline, Structure, Endings | Organization, long-term projects, boundaries, endings |
### Core Data Models
| Model | Key Fields | Purpose |
|-------|-----------|---------|
| User | id, email, location (lat/lng), timezone, preferences | User with location for planetary calculations |
| Location | id, name, latitude, longitude, timezone, user_id | Saved locations for quick access |
| ActivityCategory | id, planet, category_name, activities (JSON) | Al Saat activity-planet mapping reference |
| TimingQuery | id, user_id, activity, location_id, datetime, tqs_score, breakdown (JSON) | Logged timing queries with scores |
| ScheduleEntry | id, user_id, activity, preferred_window, assigned_window, tqs_score | Smart scheduler entries |
| PlanetaryCache | date, location_hash, sunrise, sunset, planetary_hours (JSON), positions (JSON), aspects (JSON) | Pre-computed daily planetary data |
| APIKey | id, user_id, key_hash, tier (free/developer/premium), requests_today, rate_limit | Developer API key management |
### API Products (3 Tiers)
| Tier | Price | Features |
|------|-------|----------|
| Free (Consumer) | $0 | Web app: current planetary hour, TQS for any activity, daily view |
| Developer API | $9–49/month | REST API: TQS scoring, forward search, bulk queries, webhooks |
| Premium | $4.99/month | Smart scheduler, calendar sync, personalized daily briefings, optimal window alerts |
### Caddy Routes
```
astrotiming.com → static React build (consumer web app)
api.astrotiming.com → astrotiming_backend:8020 (REST API + developer API)
```
### Celery Tasks
| Task | Schedule | Function |
|------|----------|----------|
| `precompute_daily_planetary_data` | Daily 00:00 UTC | Calculate planetary hours + positions for all saved locations, cache in PlanetaryCache |
| `generate_daily_briefing` | Daily per user timezone (sunrise - 30min) | Premium users: generate personalized daily timing briefing |
| `sync_user_calendars` | Every 30 minutes (premium users) | Pull calendar events, run conflict detection, suggest rescheduling |
| `cleanup_old_queries` | Weekly | Purge old TimingQuery records beyond retention period |
### Implementation Steps
1. Create GitHub repo `astrotiming` with Django project scaffold
2. Install `pyswisseph` and verify ephemeris calculations work
3. Configure `docker-compose.astrotiming.yml` (3 containers on `alorig_net`)
4. Create `astrotiming_db` database
5. Build Layer 1: Classical planetary hours calculation engine (Al Saat ruleset)
6. Build Layer 2: Swiss Ephemeris integration (positions, dignities, aspects, retrogrades, lunar data)
7. Build Layer 3: TQS scoring engine combining Layer 1 + Layer 2
8. Build forward search and smart scheduler algorithms
9. Build Django models, REST API endpoints
10. Build developer API with key management and rate limiting
11. Build React frontend (consumer app: daily view, activity search, score display)
12. Configure Caddy routes
13. Implement PlanetaryCache pre-computation pipeline
14. Deploy, validate TQS scores against known Al Saat test cases
15. Monitor RAM usage — expected ~400 MB steady state
---
## APP 6: ASMS PHASE 1 — School Management System
### Product Summary
Alorig School Management System (ASMS) Phase 1 is a pilot deployment for 74 Schools in Pakistan. Foundation platform: student/guardian management, multi-school tenant architecture, admissions, gradebook, attendance, courses, RBAC, LMS core. Phase 1 focuses on the Foundation Platform (700–1,030 hours scope); Advanced Modules & Intelligence Layer (3,010–4,480 hours) are Phase 2+ scope.
### Source Documents
- `ASMS/documentation/74-Schools-Development-Effort-Breakdown.md`
- `ASMS/documentation/74_Schools_Digital_Transformation_Project_Scope.docx`
- `ASMS/documentation/74_Schools_Project_Scope_v2_AI_Layer.docx`
### Tech Stack Specifics (Beyond Alorig Standard)
| Addition | Technology | Purpose |
|----------|-----------|---------|
| Multi-Tenant | Head Office → Branch → School hierarchy | 74-campus data isolation |
| RBAC | Django permission groups | 10 user roles (HO Admin, Branch Admin, Principal, Teacher, Parent, Student, etc.) |
| Attendance | GPS + geo-fence (future) | 100m radius per school, spoof detection (Phase 2) |
| WhatsApp | WhatsApp Business API (future) | Parent notifications, auto-translated messages (Phase 2) |
| AI (future) | Predictive analytics, adaptive assessment | At-risk identification, auto-report cards (Phase 2) |
| PWA (future) | Service worker, offline-first | Low-bandwidth optimization for school environments (Phase 2) |
### Container Architecture (3 containers — standard)
| Container | Image | Role | Port |
|-----------|-------|------|------|
| `asms_backend` | asms-backend:latest | Django + Gunicorn (REST API) | 8022 |
| `asms_celery_worker` | asms-backend:latest | Celery worker (report generation, data import, notifications) | — |
| `asms_celery_beat` | asms-backend:latest | Celery beat (attendance reports, grade calculations) | — |
### Database
- Database name: `asms_db` on shared `alorig_postgres`
- Redis key prefix: `asms:`
- Redis DB: 7
### Multi-School Tenant Architecture
- 3-level hierarchy: Head Office → Branch (regional) → School (campus)
- `SchoolBaseModel` — FK to School with tenant isolation
- `SchoolModelViewSet` — queries scoped to user's school(s)
- HO Admin sees all schools; Branch Admin sees branch schools; Principal sees own school only
### Phase 1 Foundation Modules (Scope)
| Module | Scope | Key Models |
|--------|-------|-----------|
| Student & Guardian Management | Registration, profiles, guardian contacts, transfers, graduation | Student, Guardian, Enrollment, Transfer |
| Multi-School Tenant | 74-campus data isolation with hierarchy | HeadOffice, Branch, School, SchoolUser |
| Admissions & Registration | Application tracking, admission levels, step workflows | Application, AdmissionLevel, AdmissionStep |
| Gradebook & Report Cards | Standards-based grading, marking periods, GPA, printable transcripts | Grade, MarkingPeriod, Transcript, ReportCard |
| Discipline & Behavior | Infraction logging, action tracking, parent notification triggers | Infraction, DisciplineAction, BehaviorLog |
| Attendance (Base) | Daily by homeroom/class, absence reasons, chronic flags | Attendance, AttendanceRecord, AbsenteeFlag |
| Courses & Scheduling | Subject allocation, class sections, teacher assignments, conflict detection | Course, ClassSection, TeacherAssignment, Schedule |
| RBAC (10 Roles) | Granular permissions per role per school level | Role, Permission, UserSchoolAccess |
| LMS Core | Curriculum structure, lesson plans, quiz engine, grade calculator | Curriculum, LessonPlan, Quiz, Question, LearningOutcome |
| REST API Layer | DRF API for all foundation modules | ViewSets, serializers, permissions |
| React Frontend | Web application for all modules | Dashboards, forms, tables, reports |
### Caddy Routes
```
schools.alorig.com → static React build (web app)
api.schools.alorig.com → asms_backend:8022 (REST API)
```
### Implementation Steps
1. Create GitHub repo `asms` with Django project scaffold
2. Configure `docker-compose.asms.yml` (3 containers on `alorig_net`)
3. Create `asms_db` database
4. Build multi-tenant hierarchy models (HeadOffice, Branch, School)
5. Build RBAC system (10 roles with granular permissions)
6. Build Student & Guardian management module
7. Build Admissions workflow module
8. Build Courses & Scheduling module with conflict detection
9. Build Attendance module (daily homeroom/class, absence tracking)
10. Build Gradebook module (standards-based grading, report cards, transcripts)
11. Build Discipline & Behavior tracking module
12. Build LMS core (curriculum, lesson plans, quiz engine)
13. Build React frontend (role-based dashboards, CRUD screens, reports)
14. Build REST API layer for all modules
15. Seed with pilot school data (5 schools initially)
16. Deploy, run pilot testing with 5 schools
17. Monitor RAM usage — expected ~600 MB steady state
---
## POST-DEPLOYMENT MONITORING PROTOCOL
After EACH app deployment:
### Immediate (First 24 Hours)
1. Run `docker stats` — verify container RAM matches budget estimates
2. Check PostgreSQL connections: `SELECT count(*) FROM pg_stat_activity;` — ensure not approaching `max_connections`
3. Check Redis memory: `redis-cli INFO memory` — verify key prefix isolation
4. Verify Caddy routes resolve correctly (SSL, domain routing)
5. Run basic smoke tests (login, core functionality, API responses)
### Ongoing (First Week)
1. Monitor RAM usage pattern over full business cycle
2. Check Celery task queue depths — ensure no backlogs
3. Verify inter-app isolation (no Redis key collisions, no DB cross-contamination)
4. Test under expected concurrent user load
### Upgrade Triggers
| Condition | Action |
|-----------|--------|
| Steady-state RAM > 13 GB | Consider KVM 8 upgrade or split heaviest app |
| OOM kills on any container | Immediate investigation — reduce worker count or split app |
| PostgreSQL connections > 80 | Review connection pooling (PgBouncer) |
| Redis memory > 2 GB | Review TTLs, add eviction policies |
| Celery task queue > 1000 pending | Add dedicated worker or split queues |
### Docker Compose File Naming Convention
```
docker-compose.infra.yml ← Shared infrastructure (PG, Redis, Caddy, Portainer)
docker-compose.igny8.yml ← IGNY8 production
docker-compose.igny8-staging.yml ← IGNY8 staging
docker-compose.psydge.yml ← Psydge
docker-compose.snapify.yml ← Snapify
docker-compose.sitebuilder.yml ← Alorig Site Builder
docker-compose.observer.yml ← Observer OS
docker-compose.astrotiming.yml ← AstroTiming
docker-compose.asms.yml ← ASMS Phase 1
```
All compose files use `external: true` for `alorig_net` network.
---
## ACCEPTANCE CRITERIA (for the 05 doc)
The `05-other-apps-deployment.md` doc is complete when:
1. Each of the 6 apps has a self-contained deployment section with: containers, database, Redis config, Caddy routes, implementation steps
2. Deployment order is specified with rationale and triggers
3. Post-deployment monitoring protocol is documented
4. Resource budget is specified per app with upgrade triggers
5. Docker compose file naming convention is established
6. Each app section references its source blueprint document
7. Exception patterns (Psydge WebSocket, Site Builder Node.js) are clearly documented
8. Port assignments don't conflict across apps
9. Redis DB numbers don't conflict across apps
10. All Caddy routes are listed with no domain conflicts
---
## PORT ASSIGNMENT MAP (No Conflicts)
| App | Backend Port | Additional Ports |
|-----|-------------|-----------------|
| IGNY8 (prod) | 8011 | — |
| IGNY8 (staging) | 8031 | — |
| Psydge | 8012 | 8013 (WebSocket/Channels) |
| Snapify | 8014 | — |
| Alorig Site Builder | 8016 | 8017 (Payload CMS) |
| Observer OS | 8018 | — |
| AstroTiming | 8020 | — |
| ASMS | 8022 | — |
## REDIS DB ASSIGNMENT MAP (No Conflicts)
| DB | App |
|----|-----|
| 0 | IGNY8 production |
| 1 | IGNY8 staging |
| 2 | Psydge |
| 3 | Snapify |
| 4 | Alorig Site Builder |
| 5 | Observer OS |
| 6 | AstroTiming |
| 7 | ASMS |
---
## BUILD SEQUENCE
Claude Code builds the single doc `05-other-apps-deployment.md` using this plan as the instruction set. The doc follows the standard structure (Current State → What to Build → Implementation Steps → Acceptance Criteria → Claude Code Instructions) but is organized per-app within those sections.
**Dependencies:** Phase 0 (infrastructure) must be complete. Phase 1–4 (IGNY8) should be complete or stable. Each app in Phase 5 deploys independently and sequentially.
---
*This build plan is the single instruction set for Phase 5. All 6 app deployment specifications are contained here with full technical depth extracted from individual app blueprints.*