docs rearrange

This commit is contained in:
alorig
2025-12-07 16:49:30 +05:00
parent 8aef9c7727
commit c87bc7266c
45 changed files with 12452 additions and 1516 deletions

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,263 @@
# Keywords, Clusters & Ideas - Complete Field Mapping
**Date:** December 3, 2025
---
## 📊 KEYWORDS - Complete Field Reference
### Backend Model Fields
**File:** `backend/igny8_core/business/planning/models.py`
| Field Name | Type | Default | Required | Choices | Notes |
|------------|------|---------|----------|---------|-------|
| `id` | Integer PK | Auto | ✅ | - | Primary key |
| `seed_keyword` | ForeignKey | - | ✅ | - | Links to SeedKeyword (PROTECT on delete) |
| `volume_override` | Integer | NULL | ❌ | - | Site-specific override (uses seed_keyword.volume if NULL) |
| `difficulty_override` | Integer | NULL | ❌ | - | Site-specific override (uses seed_keyword.difficulty if NULL) |
| `attribute_values` | JSONField | [] | ❌ | - | Optional metadata (product specs, modifiers) |
| `cluster` | ForeignKey | NULL | ❌ | - | Parent cluster (SET_NULL on delete) |
| `status` | CharField(50) | `pending` | ✅ | `pending`, `active`, `archived` | Keyword status |
| `site` | ForeignKey | - | ✅ | - | Owner site (inherited) |
| `sector` | ForeignKey | - | ✅ | - | Owner sector (inherited) |
| `account` | ForeignKey | - | ✅ | - | Owner account (inherited) |
| `created_at` | DateTime | Auto | ✅ | - | Auto-generated |
| `updated_at` | DateTime | Auto | ✅ | - | Auto-updated |
### Frontend Table Columns
**File:** `frontend/src/config/pages/keywords.config.tsx`
| Column Key | Label | Visible Default | Sortable | Render Notes |
|----------|--------|-----------------|----------|--------------|
| `keyword` | Keyword | ✅ Yes | ✅ Yes | From seed_keyword.keyword (links to Keywords page) |
| `sector_name` | Sector | Conditional* | ✅ Yes | Badge (blue) - shown when viewing all sectors |
| `volume` | Volume | ✅ Yes | ✅ Yes | Formatted as number (e.g., 1,250) |
| `cluster_name` | Cluster | ✅ Yes | ✅ Yes | Parent cluster name or "-" |
| `difficulty` | Difficulty | ✅ Yes | ✅ Yes | Badge (1-5): 1-2=green, 3=amber, 4-5=red |
| `intent` | Intent | ✅ Yes | ✅ Yes | Badge colors: Transactional/Commercial=green, Navigational=amber |
| `status` | Status | ✅ Yes | ✅ Yes | Badge: pending=amber, active=green, archived=red |
| `created_at` | Created | ✅ Yes | ✅ Yes | Relative date (e.g., "2 hours ago") |
| (Hidden by default) | | | | |
| `updated_at` | Updated | ❌ No | ✅ Yes | Relative date |
### Frontend Filter Dropdown
**File:** `frontend/src/config/pages/keywords.config.tsx` (Lines 310-360)
| Filter Key | Label | Type | Options | Dynamic |
|-----------|-------|------|---------|---------|
| `search` | Search | Text | N/A | - |
| `status` | Status | Select | `pending`, `active`, `archived` | ❌ No |
| `intent` | Intent | Select | `informational`, `navigational`, `transactional`, `commercial` | ❌ No |
| `difficulty` | Difficulty | Select | `1-5` with labels | ❌ No |
| `cluster` | Cluster | Select | Dynamic from database | ✅ Yes (loads clusters) |
| `volume` | Volume Range | Custom | Min/Max number inputs | ❌ No (range picker) |
### Frontend Create/Edit Form
**File:** `frontend/src/config/pages/keywords.config.tsx` (Lines 560-586)
| Field Key | Label | Type | Required | Default | Options |
|-----------|-------|------|----------|---------|---------|
| `seed_keyword_id` | Seed Keyword | Select | ✅ Yes | - | Dynamic from availableSeedKeywords |
| `volume_override` | Volume Override | Number | ❌ No | NULL | Numeric input (optional override) |
| `difficulty_override` | Difficulty Override | Number | ❌ No | NULL | Select 1-5 |
| `cluster_id` | Cluster | Select | ❌ No | NULL | Dynamic from clusters array |
| `status` | Status | Select | ✅ Yes | `pending` | `pending`, `active`, `archived` |
---
## 📊 CLUSTERS - Complete Field Reference
### Backend Model Fields
**File:** `backend/igny8_core/business/planning/models.py`
| Field Name | Type | Default | Required | Choices | Notes |
|------------|------|---------|----------|---------|-------|
| `id` | Integer PK | Auto | ✅ | - | Primary key |
| `name` | CharField(255) | - | ✅ | - | Unique cluster name (unique=True) |
| `description` | TextField | NULL | ❌ | - | Optional cluster description |
| `keywords_count` | Integer | 0 | ✅ | - | Cached count of linked keywords |
| `ideas_count` | Integer | 0 | ✅ | - | Cached count of linked ideas |
| `volume` | Integer | 0 | ✅ | - | Cached total volume from keywords |
| `mapped_pages` | Integer | 0 | ✅ | - | Number of mapped pages |
| `status` | CharField(50) | `new` | ✅ | `new`, `idea`, `mapped` | Cluster status |
| `site` | ForeignKey | - | ✅ | - | Owner site (inherited) |
| `sector` | ForeignKey | - | ✅ | - | Owner sector (inherited) |
| `account` | ForeignKey | - | ✅ | - | Owner account (inherited) |
| `created_at` | DateTime | Auto | ✅ | - | Auto-generated |
| `updated_at` | DateTime | Auto | ✅ | - | Auto-updated |
### Frontend Table Columns
**File:** `frontend/src/config/pages/clusters.config.tsx`
| Column Key | Label | Visible Default | Sortable | Render Notes |
|----------|--------|-----------------|----------|--------------|
| `name` | Cluster Name | ✅ Yes | ✅ Yes | Link to cluster detail page |
| `sector_name` | Sector | Conditional* | ✅ Yes | Badge (blue) - shown when viewing all sectors |
| `keywords_count` | Keywords | ✅ Yes | ✅ Yes | Formatted as number (e.g., 45) |
| `ideas_count` | Ideas | ✅ Yes | ✅ Yes | Formatted as number (e.g., 12) |
| `volume` | Volume | ✅ Yes | ✅ Yes | Formatted as number (e.g., 5,280) |
| `difficulty` | Difficulty | ✅ Yes | ✅ Yes | Badge (1-5): 1-2=green, 3=amber, 4-5=red |
| `content_count` | Content | ✅ Yes | ✅ Yes | Formatted as number |
| `status` | Status | ✅ Yes | ✅ Yes | Badge: new=amber, idea=blue, mapped=green |
| `created_at` | Created | ✅ Yes | ✅ Yes | Relative date |
| (Hidden by default) | | | | |
| `description` | Description | ❌ No | ❌ No | Text truncated to 250px |
| `mapped_pages` | Mapped Pages | ❌ No | ✅ Yes | Formatted number |
| `updated_at` | Updated | ❌ No | ✅ Yes | Relative date |
### Frontend Filter Dropdown
**File:** `frontend/src/config/pages/clusters.config.tsx` (Lines 240-290)
| Filter Key | Label | Type | Options | Dynamic |
|-----------|-------|------|---------|---------|
| `search` | Search | Text | N/A | - |
| `status` | Status | Select | `new`, `idea`, `mapped` | ❌ No |
| `difficulty` | Difficulty | Select | `1-5` with labels | ❌ No |
| `volume` | Volume Range | Custom | Min/Max number inputs | ❌ No (range picker) |
### Frontend Create/Edit Form
**File:** `frontend/src/config/pages/clusters.config.tsx` (Lines 405-418)
| Field Key | Label | Type | Required | Default | Options |
|-----------|-------|------|----------|---------|---------|
| `name` | Cluster Name | Text | ✅ Yes | - | Text input (placeholder: "Enter cluster name") |
| `description` | Description | Textarea | ❌ No | NULL | Textarea (placeholder: "Enter cluster description") |
| `status` | Status | Select | ✅ Yes | `new` | `new`, `idea`, `mapped` |
---
## 📊 CONTENT IDEAS - Complete Field Reference
### Backend Model Fields
**File:** `backend/igny8_core/business/planning/models.py`
| Field Name | Type | Default | Required | Choices | Notes |
|------------|------|---------|----------|---------|-------|
| `id` | Integer PK | Auto | ✅ | - | Primary key |
| `idea_title` | CharField(255) | - | ✅ | - | Content idea title |
| `description` | TextField | NULL | ❌ | - | Content outline/description |
| `target_keywords` | CharField(500) | '' | ❌ | - | Comma-separated keywords (legacy) |
| `keyword_objects` | M2M(Keywords) | - | ❌ | - | Individual keywords linked to idea |
| `keyword_cluster` | ForeignKey(Clusters) | NULL | ❌ | - | Parent cluster (SET_NULL on delete) |
| `status` | CharField(50) | `new` | ✅ | `new`, `scheduled`, `completed`, `published` | Idea workflow status |
| `estimated_word_count` | Integer | 1000 | ✅ | - | Target article length |
| `content_type` | CharField(50) | `post` | ✅ | `post`, `page`, `product`, `taxonomy` | Content type |
| `content_structure` | CharField(50) | `article` | ✅ | See structures below | Content format/structure |
| `site` | ForeignKey | - | ✅ | - | Owner site (inherited) |
| `sector` | ForeignKey | - | ✅ | - | Owner sector (inherited) |
| `account` | ForeignKey | - | ✅ | - | Owner account (inherited) |
| `created_at` | DateTime | Auto | ✅ | - | Auto-generated |
| `updated_at` | DateTime | Auto | ✅ | - | Auto-updated |
**Content Structure Choices (based on content_type):**
- **Post:** `article`, `guide`, `comparison`, `review`, `listicle`
- **Page:** `landing_page`, `business_page`, `service_page`, `general`, `cluster_hub`
- **Product:** `product_page`
- **Taxonomy:** `category_archive`, `tag_archive`, `attribute_archive`
### Frontend Table Columns
**File:** `frontend/src/config/pages/ideas.config.tsx`
| Column Key | Label | Visible Default | Sortable | Render Notes |
|----------|--------|-----------------|----------|--------------|
| `idea_title` | Title | ✅ Yes | ✅ Yes | Expandable (shows description) |
| `sector_name` | Sector | Conditional* | ✅ Yes | Badge (blue) - shown when viewing all sectors |
| `content_structure` | Structure | ✅ Yes | ✅ Yes | Badge (purple): article, guide, guide, etc. |
| `content_type` | Type | ✅ Yes | ✅ Yes | Badge (blue): post, page, product, taxonomy |
| `target_keywords` | Target Keywords | ✅ Yes | ❌ No | Text truncated to 250px |
| `keyword_cluster_name` | Cluster | ✅ Yes | ✅ Yes | Parent cluster name or "-" |
| `status` | Status | ✅ Yes | ✅ Yes | Badge: new=amber, scheduled=blue, completed=blue, published=green |
| `estimated_word_count` | Words | ✅ Yes | ✅ Yes | Formatted as number (e.g., 1,500) |
| `created_at` | Created | ✅ Yes | ✅ Yes | Relative date |
| (Hidden by default) | | | | |
| `updated_at` | Updated | ❌ No | ✅ Yes | Relative date |
### Frontend Filter Dropdown
**File:** `frontend/src/config/pages/ideas.config.tsx` (Lines 218-270)
| Filter Key | Label | Type | Options | Dynamic |
|-----------|-------|------|---------|---------|
| `search` | Search | Text | N/A | - |
| `status` | Status | Select | `new`, `scheduled`, `completed`, `published` | ❌ No |
| `content_structure` | Structure | Select | article, guide, comparison, etc. (all 13 options) | ❌ No |
| `content_type` | Type | Select | `post`, `page`, `product`, `taxonomy` | ❌ No |
| `cluster` | Cluster | Select | Dynamic from database | ✅ Yes (loads clusters) |
### Frontend Create/Edit Form
**File:** `frontend/src/config/pages/ideas.config.tsx` (Lines 372-417)
| Field Key | Label | Type | Required | Default | Options |
|-----------|-------|------|----------|---------|---------|
| `idea_title` | Title | Text | ✅ Yes | - | Text input (placeholder: "Enter idea title") |
| `description` | Description | Textarea | ❌ No | NULL | Textarea (placeholder: "Enter content outline") |
| `content_type` | Content Type | Select | ✅ Yes | `post` | `post`, `page`, `product`, `taxonomy` |
| `content_structure` | Content Structure | Select | ✅ Yes | `article` | 13 structure options (depends on content_type) |
| `target_keywords` | Target Keywords | Text | ❌ No | NULL | Text input (comma-separated) |
| `keyword_cluster_id` | Cluster | Select | ❌ No | NULL | Dynamic from clusters array |
| `status` | Status | Select | ✅ Yes | `new` | `new`, `scheduled`, `completed`, `published` |
| `estimated_word_count` | Word Count | Number | ❌ No | 1000 | Numeric input |
---
## 📝 SUMMARY COMPARISON
### Status Fields
| Module | Backend Default | Backend Choices | Frontend Form Default | Frontend Form Choices |
|--------|-----------------|-----------------|----------------------|----------------------|
| **Keywords** | `pending` | pending, active, archived | `pending` | pending, active, archived |
| **Clusters** | `new` | new, idea, mapped | `new` | new, idea, mapped |
| **Ideas** | `new` | new, scheduled, completed, published | `new` | new, scheduled, completed, published |
### Required Fields (Must be filled)
| Module | Required Fields |
|--------|-----------------|
| **Keywords** | seed_keyword_id, status, site, sector, account |
| **Clusters** | name, status, site, sector, account |
| **Ideas** | idea_title, status, content_type, content_structure, estimated_word_count, site, sector, account |
### Optional Fields
| Module | Optional Fields |
|--------|-----------------|
| **Keywords** | volume_override, difficulty_override, attribute_values, cluster_id |
| **Clusters** | description |
| **Ideas** | description, target_keywords, keyword_objects, keyword_cluster_id |
### Dynamic Dropdowns (Loaded from DB)
| Module | Filter | Form |
|--------|--------|------|
| **Keywords** | cluster (dropdown) | seed_keyword_id (all available), cluster_id (all clusters for sector) |
| **Clusters** | - | - |
| **Ideas** | cluster (dropdown) | keyword_cluster_id (all clusters for sector) |
### Visible-by-Default Table Columns
| Module | Count | Primary Columns |
|--------|-------|-----------------|
| **Keywords** | 9 | keyword, volume, cluster, difficulty, intent, status, created_at |
| **Clusters** | 11 | name, keywords_count, ideas_count, volume, difficulty, content_count, status, created_at |
| **Ideas** | 10 | idea_title, content_structure, content_type, target_keywords, cluster, status, word_count, created_at |
---
## 🔑 Key Differences
### Keywords
- **Uses:** SeedKeyword (global pool) - one keyword per site/sector
- **Overrideable:** volume, difficulty (site-specific)
- **Links to:** Clusters (via cluster FK)
- **Status:** pending (awaiting cluster), active (clustered), archived
### Clusters
- **Type:** Pure topic clusters (semantic groupings)
- **Auto-updated:** keywords_count, ideas_count, volume (cached from keywords)
- **Status:** new (no ideas), idea (has ideas), mapped (has content)
- **No overrides:** All values are cached/calculated
### Ideas
- **Type:** Content concepts ready for production
- **Links:** Cluster (required for workflow), Keywords (optional M2M)
- **Customizable:** content_type, content_structure, word_count
- **Status:** new → scheduled (queued to writer) → completed (content generated) → published
---
**END OF COMPLETE FIELD MAPPING**

View File

@@ -0,0 +1,258 @@
# Content Taxonomy Relationship Diagram
## Current Architecture (Simplified)
```
┌─────────────────────────────────────────────────────────────────────┐
│ IGNY8 Content System │
└─────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────┐
│ Multi-Tenant Hierarchy │
│ │
│ Account ──┬── Site ──┬── Sector ──┬── Keywords │
│ │ │ ├── Clusters │
│ │ │ ├── Ideas │
│ │ │ ├── Tasks │
│ │ │ ├── Content │
│ │ │ └── ContentTaxonomy │
│ │ └── Sector 2 │
│ └── Site 2 │
└──────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────┐
│ Planner → Writer → Publisher Workflow │
│ │
│ Phase 1-3: PLANNER │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Keywords │ ──> │ Clusters │ ──> │ Ideas │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ │ │ │ │
│ Phase 4: WRITER │
│ │ │ │ │
│ └─────────────────┴─────────────────┘ │
│ │ │
│ v │
│ ┌──────────┐ ┌──────────┐ │
│ │ Tasks │ ──> │ Content │ │
│ └──────────┘ └──────────┘ │
│ │ │
│ │ │
│ Phase 5: PUBLISHER │ │
│ v │
│ ┌─────────────┐ │
│ │ WordPress │ │
│ │ Shopify │ │
│ │ Sites │ │
│ └─────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────┐
│ Content ↔ Taxonomy Relationship (Many-to-Many) │
└──────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────┐
│ Content Model │
│─────────────────────────────────│
│ PK: id │
│ FK: site_id ──────┐ │
│ FK: sector_id │ │
│ FK: cluster_id │ │
│ │ │
│ title │ │
│ content_html │ │
│ word_count │ │
│ meta_title │ │
│ meta_description │ │
│ │ │
│ content_type │ │
│ content_structure │ │
│ │ │
│ external_id │ │
│ external_url │ │
│ sync_status │ │
│ │ │
│ source │ │
│ status │ │
└───────────┬───────┘ │
│ │
│ Many-to-Many │
│ (via ContentTaxonomyRelation)
│ │
v │
┌─────────────────────────────────┐│
│ ContentTaxonomyRelation Model ││
│─────────────────────────────────││
│ PK: id ││
│ FK: content_id ──────────────────┘
│ FK: taxonomy_id ────────────┐
│ │
│ created_at │
│ updated_at │
│ │
│ UNIQUE(content, taxonomy) │
└─────────────────────────────┘
v
┌─────────────────────────────────┐
│ ContentTaxonomy Model │
│─────────────────────────────────│
│ PK: id │
│ FK: site_id ─────┐ │
│ FK: sector_id │ │
│ │ │
│ name │ │
│ slug │ │
│ taxonomy_type │ ◄─── "category" or "tag"
│ description │ │
│ count │ │
│ │ │
│ WordPress Sync: │ │
│ external_taxonomy│ ◄─── "category", "post_tag"
│ external_id │ ◄─── WordPress term_id
│ │ │
│ metadata (JSON) │ │
└──────────────────┘ │
│ │
│ │
UNIQUE(site, slug, taxonomy_type)│
UNIQUE(site, external_id, external_taxonomy)
│ │
└──────────────┘
┌──────────────────────────────────────────────────────────────────────┐
│ Usage Example │
└──────────────────────────────────────────────────────────────────────┘
# Get all categories for a content piece
categories = content.taxonomy_terms.filter(taxonomy_type='category')
# Get all tags for a content piece
tags = content.taxonomy_terms.filter(taxonomy_type='tag')
# Add a category to content
tech_category = ContentTaxonomy.objects.get(name='Technology')
content.taxonomy_terms.add(tech_category)
# Get all content with a specific tag
tutorial_tag = ContentTaxonomy.objects.get(name='Tutorial')
contents = tutorial_tag.contents.all()
# WordPress publishing (automatic)
wp_categories = [term.name for term in content.taxonomy_terms.filter(taxonomy_type='category')]
wp_tags = [term.name for term in content.taxonomy_terms.filter(taxonomy_type='tag')]
┌──────────────────────────────────────────────────────────────────────┐
│ WordPress Integration (Bidirectional Sync) │
└──────────────────────────────────────────────────────────────────────┘
IGNY8 → WordPress:
─────────────────
1. Content created in IGNY8
2. Categories/tags assigned via taxonomy_terms M2M
3. Publishing task created
4. wordpress_publishing.py extracts:
- categories = content.taxonomy_terms.filter(taxonomy_type='category')
- tags = content.taxonomy_terms.filter(taxonomy_type='tag')
5. REST API creates WordPress post with terms
6. external_id saved back to Content model
7. Log: [5-homeg8.com] [POST] Published: "Article Title" (ID: 123)
WordPress → IGNY8:
─────────────────
1. WordPress plugin detects post update
2. REST API sends post data + terms to IGNY8
3. Content updated/created with external_id
4. ContentTaxonomy created/updated with external_id
5. ContentTaxonomyRelation created linking them
6. Log: [5-homeg8.com] [SYNC] Imported: "Article Title"
┌──────────────────────────────────────────────────────────────────────┐
│ Database Tables Summary │
└──────────────────────────────────────────────────────────────────────┘
igny8_content
├─ id (PK)
├─ site_id (FK → sites)
├─ sector_id (FK → sectors)
├─ cluster_id (FK → clusters)
├─ title, content_html, word_count
├─ meta_title, meta_description, keywords
├─ content_type, content_structure
├─ external_id, external_url, sync_status
└─ source, status, timestamps
igny8_content_taxonomy_relations (Through Table)
├─ id (PK)
├─ content_id (FK → igny8_content)
├─ taxonomy_id (FK → igny8_content_taxonomy_terms)
└─ timestamps
UNIQUE(content_id, taxonomy_id)
igny8_content_taxonomy_terms
├─ id (PK)
├─ site_id (FK → sites)
├─ sector_id (FK → sectors)
├─ name, slug
├─ taxonomy_type ('category' | 'tag')
├─ description, count
├─ external_taxonomy ('category' | 'post_tag')
├─ external_id (WordPress term_id)
├─ metadata (JSON)
└─ timestamps
UNIQUE(site_id, slug, taxonomy_type)
UNIQUE(site_id, external_id, external_taxonomy)
┌──────────────────────────────────────────────────────────────────────┐
│ Before vs After Comparison │
└──────────────────────────────────────────────────────────────────────┘
BEFORE (Complex - with SiteBlueprint):
────────────────────────────────────
SiteBlueprint ──┬── SiteBlueprintCluster ──> Clusters
├── SiteBlueprintTaxonomy ──> ContentTaxonomy
└── PageBlueprint ──> Content
Content ──> ContentTaxonomyMap ──> ContentTaxonomy
(separate table with FK)
PublishingRecord ──┬── content_id
└── site_blueprint_id
DeploymentRecord ──┬── content_id
└── site_blueprint_id
AFTER (Simple - SiteBlueprint Removed):
──────────────────────────────────────
Keywords ──> Clusters ──> Ideas ──> Tasks ──> Content
Content ↔ ContentTaxonomy
(M2M via ContentTaxonomyRelation)
PublishingRecord ──> content_id
DeploymentRecord ──> content_id
┌──────────────────────────────────────────────────────────────────────┐
│ Key Benefits │
└──────────────────────────────────────────────────────────────────────┘
✅ Simpler architecture - removed 4 models
✅ Direct M2M relationship - easier to query
✅ Less database joins - better performance
✅ Clear taxonomy model in Django admin
✅ WordPress sync unchanged - still works perfectly
✅ Planner-Writer-Publisher workflow intact
✅ Multi-tenant security maintained
✅ No circular dependencies
✅ Clean codebase - no legacy blueprint code

View File

@@ -0,0 +1,237 @@
bluesalman: gothrough readme and tell me which folders/files inside backeedn adn frontedn folders are acutaly part of igny8 app, and which folders are part of some tech stack which will be enever modifed by app only isntaleld when tech stack isntaleled
GitHub Copilot: I'll create a short plan, then list the backend and frontend directories to inspect their contents.
Created 3 todos
Now I'll list both directories (backend and frontend) to collect the items to classify.
Read [](file:///data/app/igny8/backend)
Read [](file:///data/app/igny8/frontend)
Marking plan steps completed and finishing classification.
Completed: *List backend files* (1/3)
Progress: I scanned backend and frontend and classified items below.
**Backend — Classified**
- **App (igny8 application core):**
- `igny8_core/`: The Django project / app code (settings, apps, views, models, tasks). This is the main application source you will modify.
- `manage.py`: Django management entrypoint — part of the app.
- `migrations/`: Django DB migrations — app schema changes (source-controlled).
- `staticfiles/`: Collected static assets (app assets). May be generated but belongs to the app.
- `requirements.txt`: Dependency manifest for the app (used to install the Python packages the app needs). Not runtime third-party code, but part of the app repo and should be version-controlled.
- **Repo tooling / app-maintenance scripts (part of the repo; maintained by your team):**
- `check_api_response.py`, `check_recent_keywords.py`, `cleanup_structure_categories.py`, `diagnose_generate_content.py`, `inject_test_data.py`, `sync_idea_status.py`, `test_tags_categories.py`, `verify_*.py`, `fix_*` scripts, `final_verify.py`, `deploy_automation.sh`, `example_csv/`: These are maintenance, verification, or migration scripts that are part of the repository and used by developers/ops. They are not third-party code but are repo-level tooling you may edit.
- **Tech stack / infra artifacts (should not be modified by the app runtime; installed or produced by infra):**
- `.venv/`: Virtual environment — environment artifact; not part of app source and typically excluded from commits.
- `Dockerfile`: Container build file — infra/build config (used by ops or CI).
- `backup_postgres_20251120_232816.sql`, `db_backup_20251120_232646.sqlite3`, `rename_fields_migration.sql`: Database backups / SQL dumps — artifacts, not app source.
- `celerybeat-schedule`: Celery scheduler state file — runtime artifact.
- `logs/`: Runtime logs — generated artifacts.
- Any large generated files (e.g., DB dumps) are infra artifacts and shouldn't be edited as source.
Notes:
- Some items like `deploy_automation.sh` or `fix_*` scripts are operational but live in the repo — treat them as repository tooling (editable by you), whereas backups, `.venv`, and docker images/configs are infra artifacts.
**Frontend — Classified**
- **App (igny8 frontend source / code you will modify):**
- `src/`: Application source (React/Vue/Svelte/TSX/etc.) — primary frontend code to edit.
- `public/`, `index.html`, `marketing.html`, `banner.png`: Static assets and HTML entrypoints — part of the app.
- `package.json`, `package-lock.json`: NPM manifests — define packages the frontend needs (repo-managed).
- `eslint.config.js`, `postcss.config.js`, `vite.config.ts`, `vitest.config.ts`, `tsconfig*.json`: Build and tooling config that belong to the frontend source and are edited as part of app development.
- README.md, `DESIGN_SYSTEM.md`, `MIGRATION_GUIDE.md`: Documentation for the frontend — repo docs.
- **Tech stack / infra artifacts (installed/generated by stack — do not edit at runtime):**
- `node_modules/`: Installed dependencies — generated by `npm` / `pnpm` / `yarn`; should not be manually edited and typically ignored by VCS.
- `dist/`: Built production bundle — generated artifact (output of the build).
- `Caddyfile`, `Caddyfile.marketing`, `Dockerfile`, `Dockerfile.dev`, `Dockerfile.marketing`, `Dockerfile.marketing.dev`: Webserver and container build configs — deployment/infra files. They belong to ops/build but do live in the repo; treat as infra configuration, not application logic.
- `LICENSE.md`, other top-level docs: docs, not runtime app code.
Notes:
- Build-tool config files (e.g., `vite.config.ts`, `tsconfig.json`) are part of the app repository and are changed by developers. By contrast, `node_modules` and `dist` are installed/generated by the tech stack and should not be edited directly.
Quick summary (one-line):
- Edit: igny8_core, `manage.py`, migrations, src, public, `package.json`, and similar source/config files — these are part of the igny8 app.
- Don't manually edit: runtime/generated/installed artifacts like `.venv/`, `node_modules/`, `dist/`, DB backups, `celerybeat-schedule`, and use `Dockerfile`/`Caddyfile` only as infra config (ops-managed).
Would you like:
- a) A file added to the repo (e.g., `ARCHITECTURE.md` or `IGNY8-CONTENTS.md`) that labels each top-level item as "app" vs "infra", or
- b) I can produce a JSON/CSV mapping of items for automation (useful for CI), or
- c) Walk through any specific file/folder you want deeper inspection of?
Which option do you prefer?
bluesalman: ok now cehck teh strcutre of /data adn recrussively all its fodlers including igny8/
waht can we do if we want to make igny8 folder as the only full app foodler requried to move to any new location,
and jsut install the portainer as current envirnpmaent and isntall teh app package on nwe locatink build the package and it runs there,
teh curretn setup isnto standrd as it ocntians some node )moduels dist and env,
venv i know can be removed completely
so waht do you think
GitHub Copilot: I'll make a short plan, then list data and the key subfolders (app, igny8, and their important children) so I can analyze what to keep, remove, or package.
Created 4 todos
Read [](file:///data)
Updating the plan: mark the scan step done (noting data access blocked) and keep other steps ready. Then I'll provide analysis, exact commands, Docker and non-Docker packaging options, and a sample `docker-compose.yml` and `install.sh`.
bluesalman: waht happend you are stucl
GitHub Copilot: I'll create a short todo plan, then list data, app, and igny8 to inspect contents before proposing a packaging strategy.
Created 4 todos
Read [](file:///data)
Now I'll run a quick directory size and shallow tree under the workspace `igny8/` so we can see which folders (like `node_modules`, `.venv`, `dist`) are large and where they are.
Ran terminal command: cd /data/app/igny8 && echo 'Top-level sizes:' && du -sh * 2>/dev/null | sort -h && echo && echo 'Directory tree (depth 3):' && find . -maxdepth 3 -type d | sed 's|^\./||' | sort
Updating the todo list: mark the listing steps done and the analysis/packaging step in-progress.
Starting: *Analyze and propose packaging* (4/4)
Progress update: I inspected the workspace `igny8/` (sizes and a shallow tree). Next I analyzed which files to keep/exclude and drafted packaging, cleanup commands, and a sample `docker-compose.yml` plus a `prepare_package.sh` plan. Below are clear findings and actionable steps.
**Findings**
- **Large/generated artifacts present:**
- .venv — virtualenv (can be removed for packaging).
- node_modules, node_modules — installed npm dependencies (do not include in package).
- dist — built frontend assets (can be produced during build; you can include or build on deploy).
- DB dumps and scheduler/state files appear under backend (e.g., `*.sql`, `*.sqlite3`, `celerybeat-schedule`) — these are artifacts and should not be bundled as app source.
- **App source & config (keep these):**
- igny8_core, manage.py, migrations, requirements.txt — Django app and deps manifest.
- src, public, package.json and build config files — frontend source and build config.
- `Dockerfile` / `Dockerfile.*` and `Caddyfile` — infra config that should remain in the repo (useful for containerization).
- **Repo tooling and docs:** `deploy_automation.sh`, `verify_*.py`, docs, README.md — keep in repo; useful for ops/dev.
**Goal**
Make `igny8/` a single, relocatable application folder that you can move to a new host, then deploy via Portainer (or Docker Compose) without shipping generated artifacts like `node_modules`, `.venv`, or large backups.
Recommendations and exact steps
- **Strategy (recommended): Docker image + single `docker-compose.yml` stack**
- Keep only source, config, Dockerfiles and manifests in the app folder.
- Exclude generated artifacts from the package (`.venv`, `node_modules`, `dist`, backups, logs).
- Provide a `docker-compose.yml` that builds the backend and frontend images from the repo and defines services: `db` (Postgres), `redis`, backend (Gunicorn/Django), `worker` (Celery), and frontend (built and served by Caddy or served as static in a small nginx/caddy image).
- Deploy via Portainer by uploading the `docker-compose.yml` (Portainer can build images from contexts if configured), or pre-build images, push to container registry, and use compose with image references.
- **Alternative:** Tarball + install script that runs `docker compose build` and `docker compose up -d` on the new host.
- **Safety note:** Always create a backup or branch of the repo before mass-deleting artifacts.
Exact commands to produce a clean package (run from igny8):
```bash
# make a copy first (safe)
cp -a /data/app/igny8 /tmp/igny8-package-source
# inside the copy, remove generated artifacts
cd /tmp/igny8-package-source
rm -rf backend/.venv
rm -rf backend/*.sql backend/*.sqlite3 backend/celerybeat-schedule backend/logs || true
rm -rf frontend/node_modules sites/node_modules
rm -rf frontend/dist
# optional: remove any large backups in root
rm -f backup_*.sql || true
# create archive
tar -czf /tmp/igny8-clean.tgz .
# verify size
ls -lh /tmp/igny8-clean.tgz
```
Suggested `.dockerignore` (place in repo root) to keep image builds small:
```
backend/.venv
frontend/node_modules
sites/node_modules
frontend/dist
*.sql
*.sqlite3
logs/
*.log
.git
node_modules
```
Sample minimal `docker-compose.yml` (adapt to your app's env vars and Dockerfiles):
```yaml
version: '3.8'
services:
db:
image: postgres:15
environment:
POSTGRES_DB: igny8
POSTGRES_USER: igny8
POSTGRES_PASSWORD: changeme
volumes:
- db_data:/var/lib/postgresql/data
redis:
image: redis:7
command: redis-server --appendonly yes
backend:
build:
context: ./backend
dockerfile: Dockerfile
depends_on:
- db
- redis
environment:
DATABASE_URL: postgres://igny8:changeme@db:5432/igny8
REDIS_URL: redis://redis:6379/0
ports:
- "8000:8000"
volumes:
- static_volume:/app/staticfiles
worker:
build:
context: ./backend
dockerfile: Dockerfile
command: celery -A igny8_core worker --loglevel=info
depends_on:
- redis
- db
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
ports:
- "3000:80"
depends_on:
- backend
volumes:
db_data:
static_volume:
```
Notes:
- The Dockerfile should install Python deps from `requirements.txt`, collect static, run migrations in a start-up script, and run Gunicorn. If not present, we can produce a small `Dockerfile`.
- The Dockerfile should `npm ci && npm run build` and copy `dist/` into a small static-server image (Caddy or nginx). If you prefer to serve frontend via the backend/CDN, build and copy the static files into backend static volume.
What I'd do next (I can do these for you)
- Option A (recommended): Create `docker-compose.yml` and `prepare_package.sh` in the repo (I can add both files), plus `.dockerignore` and a minimal Dockerfile / Dockerfile if missing. Then test `docker compose up --build` locally.
- Option B: Create an archive script only (if you prefer to handle Dockerfiles yourself).
Would you like me to:
- **(1)** Create the `prepare_package.sh`, `.dockerignore`, and a `docker-compose.yml` in the repository and test the compose up here, or
- **(2)** Only produce the files and commands you can run on your target host (no changes to repo), or
- **(3)** Generate a simple Dockerfile and Dockerfile for a fully self-contained build?
Pick an option and Ill implement it.

View File

@@ -0,0 +1,46 @@
## Session Configuration Summary (Signup → Activation → Billing)
This doc captures all changes made in this session across backend and frontend to stabilize signup, onboarding, plans, payments, throttling, and admin flows.
### Backend Changes
- Registration slug collision handling
- Now generates unique account slugs by appending numeric suffixes to avoid `IntegrityError` on duplicate slugs during signup.
- File: `backend/igny8_core/auth/serializers.py`
- Plans endpoint throttling removed
- `/api/v1/auth/plans/` no longer uses throttling to prevent 429 responses during onboarding/plan fetch.
- File: `backend/igny8_core/auth/views.py`
### Frontend Changes
- Plans visibility and throttling behavior
- Removed plan filtering to only Starter/Growth/Scale; all active plans returned by the API are shown.
- Added a 429 retry: on throttling while loading billing/plans, shows “throttled, retrying” and auto-retries once after 2 seconds.
- File: `frontend/src/pages/account/PlansAndBillingPage.tsx`
- Signup redirect hard fallback
- After successful signup, still calls router navigate to `/account/plans`, and also forces a 500ms hard redirect to `/account/plans` if navigation stalls.
- File: `frontend/src/components/auth/SignUpForm.tsx`
- Payment method preference
- Auto-selects payment method id 14 when available and best-effort sets it as default; prefers enabled methods.
- File: `frontend/src/pages/account/PlansAndBillingPage.tsx`
- Admin subscriptions actions and linking
- Added Activate/Cancel and Refresh actions on `/admin/subscriptions`.
- `/admin/accounts` “Manage” links now deep-link to `/admin/subscriptions?account_id=...`.
- Files: `frontend/src/pages/admin/AdminSubscriptionsPage.tsx`, `frontend/src/pages/admin/AdminAllAccountsPage.tsx`
- Upgrade gating helper
- Introduced `isUpgradeError` / `showUpgradeToast` and applied to Sites list to surface upgrade prompts on 402/403. ModuleGuard imported/prepared.
- Files: `frontend/src/utils/upgrade.ts`, `frontend/src/pages/Sites/List.tsx`, `frontend/src/components/common/ModuleGuard.tsx`
- Balance error UX
- Shows “Balance unavailable” with retry; clears stale balance on error instead of silently showing defaults.
- Files: `frontend/src/components/dashboard/CreditBalanceWidget.tsx`, `frontend/src/store/billingStore.ts`
### Behavior Notes / Outcomes
- Signup now survives duplicate slug cases (unique slug generation).
- Plans fetch should not be throttled; all active plans returned by the API will render.
- If SPA navigation fails post-signup, the hard redirect ensures landing on `/account/plans`.
- Payment method id 14 is preferred when present; otherwise any default/available method can be used.
- Admins can activate/cancel subscriptions from the subscriptions page; Manage links carry `account_id`.
- Upgrade prompts now surface on 402/403 in Sites list; balance widgets show proper error/retry state.
### Operational Dependencies / Reminders
- Ensure the backend exposes at least one active plan; otherwise the list will remain empty.
- Ensure at least one payment method exists (id 14 preferred) so plan selection/purchases are not blocked.

View File

@@ -0,0 +1,13 @@
Differences from the two docs (user-flow + session-config-summary) in the current implementation:
1) Subscriptions throttling still impacts normal users
- Docs expect “plans fetch should not be throttled” and smooth onboarding; in practice, `/v1/auth/subscriptions/` is still returning 429s for normal users until the backend restart picks up the relaxed scope.
2) Payment methods are not auto-present for all accounts
- Docs assume payment methods are available so plan selection isnt blocked. Account 29 had none until we inserted bank_transfer and PK manual; the UI showed “No payment methods available.”
3) Enterprise plan is hidden for non-aws-admin
- Not called out in the docs; current UI filters out Enterprise for normal tenants.
4) Onboarding friction from auth/plan calls
- Docs expect a seamless redirect and retry-once for 429; current flow still shows visible errors (429/403) when token is missing/expired or when subscriptions are throttled.

View File

@@ -0,0 +1,200 @@
## IGNY8 User & Billing Flow — Current State and Remediation Plan
### 0) Scope / Sources
- Frontend routing: `frontend/src/App.tsx`
- Auth store and token handling: `frontend/src/store/authStore.ts`
- Balance/usage/billing client: `frontend/src/services/api.ts`, `frontend/src/services/billing.api.ts`
- Admin billing UI: `frontend/src/pages/Admin/AdminBilling.tsx`
- Backend auth/billing routers: `backend/igny8_core/auth/urls.py`, `backend/igny8_core/business/billing/urls.py`, `backend/igny8_core/modules/billing/urls.py`
---
### 1) Signup / Login
**Backend endpoints (auth/urls.py)**
- POST `/api/v1/auth/register/` → creates user; no email verification enforced.
- POST `/api/v1/auth/login/` → issues JWT access/refresh; requires valid account + plan for “happy path”.
- POST `/api/v1/auth/refresh/` → refresh token.
**Frontend behavior (authStore)**
- `register()` hits `/v1/auth/register/`; on success stores tokens and marks authenticated.
- `login()` hits `/v1/auth/login/`; if user.account missing → throws `ACCOUNT_REQUIRED`; if plan missing → `PLAN_REQUIRED`. So the UI blocks login success if the account/plan is absent.
- After login, routes are guarded by `ProtectedRoute` → land on main dashboard (`/`).
**Gaps**
- No email verification or onboarding; user is dropped to dashboard.
- If account/plan is missing, errors surface but no guided remediation (no redirect to plan selection).
---
### 2) Tenant Billing Data & Balance
**Current endpoints used**
- Balance: `/v1/billing/transactions/balance/` (business billing CreditTransactionViewSet.balance).
- Credit transactions: `/api/v1/billing/transactions/`.
- Usage logs/summary/limits: `/api/v1/billing/credits/usage/…`, `/v1/billing/credits/usage/summary/`, `/v1/billing/credits/usage/limits/`.
- Invoices: `/api/v1/billing/invoices/` (+ detail, pdf).
- Payments: `/api/v1/billing/payments/`.
- Credit packages: `/v1/billing/credit-packages/`.
**UI touchpoints**
- Header metric uses `fetchCreditBalance()``/v1/billing/transactions/balance/`.
- Usage & Analytics, Credits pages, Billing panels use `getCreditBalance()` → same endpoint.
**Gaps**
- Subscription lifecycle (select plan, subscribe/upgrade/cancel) not wired in UI.
- Payment methods add/remove not exposed in UI.
- Balance fetch falls back to zero on any non-200; no inline “balance unavailable” messaging.
---
### 3) Admin Billing (pages/Admin/AdminBilling.tsx)
**Current tabs & data**
- Overview: stats from `/v1/admin/billing/stats/` (totals, usage).
- User Management: `/v1/admin/users/` (credit balances), adjust credits via `/v1/admin/users/{id}/adjust-credits/`.
- Credit Pricing: `/v1/admin/credit-costs/` with editable cost per operation; Save posts updates.
- Credit Packages: read-only cards from `/v1/billing/credit-packages/`.
**Gaps**
- No admin CRUD for packages (only read).
- No exports/filters for payments/invoices in this page (handled elsewhere).
---
### 4) Plan Limits / Access
**Backend**
- Auth login blocks if account plan missing (frontend enforced). Backend exposes plans/subscriptions via `auth/urls.py` (routers: `plans`, `subscriptions`).
- Automation service checks `account.credits` directly before heavy AI work (no HTTP).
**Frontend**
- No explicit plan-limit enforcement in UI (sites, seats, module gating) beyond route/module guards already present per module.
- No upgrade prompts when hitting limits.
---
### 5) AI / Automation Credit Checks
- Backend: `automation_service.py` estimates credits, adds 20% buffer, aborts if `account.credits` < required. Not dependent on billing APIs.
- Frontend: no preflight credit warning; errors are backend-driven.
---
### 6) Whats Missing for a Complete E2E Paid Flow
1) **Subscription UX**: Add tenant plan selection + subscribe/upgrade/cancel wired to `/api/v1/auth/subscriptions/...` (or billing subscription endpoints if exposed separately). Show current plan and renewal status.
2) **Payment Methods**: Expose add/list/remove default; required for Stripe/PayPal/manual flows.
3) **Invoices/Payments (tenant)**: List history with PDF download; surface payment status.
4) **Manual payments (if supported)**: Provide upload/reference input; show approval status.
5) **Plan limits**: Enforce and message limits on sites, seats, modules; show “Upgrade to increase limit.”
6) **Balance UX**: If `/v1/billing/transactions/balance/` fails, show inline warning and retry instead of silently showing 0.
7) **Admin packages**: If needed, add CRUD for credit packages (activate/feature/sort/price IDs).
8) **Onboarding**: Post-signup redirect to “Choose plan” if no active plan; optional email verification.
---
### 7) Recommended Happy Path (Target)
1) Signup → email verify (optional) → redirect to Plan selection.
2) Choose plan → subscribe (create payment method + subscription intent) → confirm → landing on dashboard with non-zero credits and plan limits loaded.
3) User can:
- Add sites up to plan limit.
- Use Planner/Writer modules; preflight check shows remaining credits.
- View Credits/Usage/Invoices/Payments; purchase extra credit packages if needed.
4) Header shows live balance from `/v1/billing/transactions/balance/`; errors visible if fetch fails.
5) Admin can:
- Adjust credit costs, see stats/payments/invoices/approvals.
- Manage credit packages (CRUD) if enabled.
---
### 8) Quick Endpoint Reference (current wiring)
- Balance: `/v1/billing/transactions/balance/`
- Tenant: invoices `/api/v1/billing/invoices/`, payments `/api/v1/billing/payments/`, credit-packages `/v1/billing/credit-packages/`, usage `/api/v1/billing/credits/usage/`
- Admin: stats `/v1/admin/billing/stats/`, users `/v1/admin/users/`, credit-costs `/v1/admin/credit-costs/`, admin payments `/v1/billing/admin/payments/`, pending approvals `/v1/billing/admin/pending_payments/`
---
## Remediation Plan (Do Now)
### A) Access control / onboarding
- After register → force redirect to Plans page.
- Lock all app routes except Account Profile, Plans, Billing until subscription is active (route guard).
- Optional: email verification gate before plan selection.
### B) Plans & subscription UX (tenant)
- Plans page: list plans from `/api/v1/auth/plans/` (or billing subscriptions endpoint), show limits.
- Actions: subscribe/upgrade/cancel wired to subscription endpoints; show status/renewal date.
- CTA in header/sidebar if no active plan.
### C) Payment methods
- Add/list/remove default payment methods (Stripe/PayPal/manual) via billing endpoints.
- Show “add payment method” inline on Plans/Billing if none exists.
### D) Checkout / payments
- Credit packages purchase flow wired to `/v1/billing/credit-packages/…/purchase/`.
- If manual payments allowed: upload/reference form and surface status; tie to admin approvals.
- Invoice list/history with PDF download; payment history with status badges.
### E) Balance and usage UX
- Use single endpoint `/v1/billing/transactions/balance/`; on non-200 show inline warning + retry (no silent zero).
- Surface plan limits and usage (sites, seats, module access) with upgrade prompts when near/at limit.
### F) Module access & limits
- Enforce plan limits on: add site, team invites, module guards (planner/writer/automation/etc.).
- On 403/422 from backend: show upgrade prompt with link to Plans.
### G) Admin
- Optional: add CRUD for credit packages (create/edit/sort/feature) if backend endpoints available.
- Keep credit costs editable (already wired); add filters/exports later if needed.
### H) AI task preflight
- Before costly AI actions, fetch balance and show “insufficient credits—purchase/upgrade” prompt.
- Handle backend credit errors with actionable UI (purchase/upgrade CTA).
### I) Error handling / UX polish
- Balance widget: display “Balance unavailable” on error; retry button.
- Centralize billing error toasts with clear messages (404 vs 402/403 vs 500).
### J) Implementation order
1) Route guard + post-signup redirect to Plans; hide other routes until active plan.
2) Plans page with subscribe/upgrade/cancel wiring; add payment method UI.
3) Billing page: invoices/payments/history + credit package purchase CTA.
4) Balance/usage error handling; plan limit prompts; AI preflight checks.
5) Admin package CRUD (if needed) and exports/filters.
[User lands]
|
v
[/signup] -- create account --> [Tokens stored]
|
v
[/account/plans] (onboarding gate)
|
|-- list plans (/v1/auth/plans/)
|-- choose plan + payment method (default/selected)
|-- subscribe (/v1/auth/subscriptions/)
v
[Plan active] -----> [Billing tabs available]
| | \
| | \
| v v
| [Purchase Credits] [Invoices/Payments]
| /api/v1/... (list + PDF download)
| |
| v
| [Balance/Usage/Limits] (shared store, retry)
|
v
[Protected routes unlocked: /, modules, account]
|
v
[Sites -> Add/Manage site]
|
|-- check plan limits (sites/seats/modules)
|-- on limit: show upgrade/purchase prompt
v
[Site created]
|
v
[Start workflow]
|-- Planner/Writer/Automation, etc.
|-- preflight: balance/limits; on 402/403 -> prompt upgrade/purchase
v
[User executes tasks with credits]