docs 1

docs/backend/IGNY8-BACKEND-ARCHITECTURE.md

@@ -0,0 +1,55 @@
+# IGNY8 Backend Architecture (code-sourced, Dec 2025)
+
+Purpose: backend-only view based on current code under `backend/`. No legacy docs or assumptions.
+
+## 1) Stack & Cross-Cutting
+- Django + DRF; Celery present via `backend/igny8_core/celery.py` and `tasks/`.
+- Multi-tenancy: models inherit `AccountBaseModel` (tenant FK `tenant_id`) or `SiteSectorBaseModel` for site/sector scoping (`backend/igny8_core/auth/models.py`). Account carries billing fields (email/address/tax_id).
+- Project URLs wired in `backend/igny8_core/urls.py`.
+
+## 2) API Namespaces (as routed)
+- `/api/v1/auth/` → `igny8_core.auth.urls` (auth + CSV admin helpers).
+- `/api/v1/account/` → `igny8_core.api.urls` (account settings, team, usage analytics).
+- `/api/v1/planner/` → `igny8_core.modules.planner.urls` (keywords, clusters, ideas ViewSets).
+- `/api/v1/writer/` → `igny8_core.modules.writer.urls` (tasks, images, content, taxonomies ViewSets).
+- `/api/v1/system/` → `igny8_core.modules.system.urls` (prompts, author profiles, strategies, settings: system/account/user/modules/ai, module-enable toggle, health/status/metrics, Gitea webhook, integration settings save/test/generate/progress).
+- `/api/v1/billing/` → `igny8_core.business.billing.urls` (tenant billing: invoices, payments, credit-packages, credit-transactions, payment-methods CRUD/default, credits balance/usage/transactions; manual payment submit; available methods).
+- `/api/v1/admin/` → `igny8_core.modules.billing.admin_urls` (billing admin: stats, user credit adjustments, credit costs, invoices/payments/pending approvals, approve/reject, payment-method configs, account payment methods CRUD/default).
+- `/api/v1/automation/` → `igny8_core.business.automation.urls` (automation ViewSet).
+- `/api/v1/linker/` → `igny8_core.modules.linker.urls` (linker ViewSet).
+- `/api/v1/optimizer/` → `igny8_core.modules.optimizer.urls` (optimizer ViewSet).
+- `/api/v1/publisher/` → `igny8_core.modules.publisher.urls` (publishing records, deployments, root publisher actions, public site definition).
+- `/api/v1/integration/` → `igny8_core.modules.integration.urls` (integrations ViewSet + WordPress status/metadata webhooks).
+- OpenAPI docs: `/api/schema/`, `/api/docs/`, `/api/redoc/`.
+
+## 3) Key Domain Models (code references)
+- Billing (`business/billing/models.py`):
+  - Invoice: `subtotal`, `tax`, `total`, `currency`, `status` (`draft|pending|paid|void|uncollectible`), dates, `line_items`, `stripe_invoice_id`, `payment_method`, billing period, helpers `subtotal_amount|tax_amount|total_amount`.
+  - Payment: statuses include `pending_approval`, `processing`, `completed/succeeded`, `failed/refunded/cancelled`; methods `stripe|paypal|bank_transfer|local_wallet|manual`; intent/charge ids, manual references, approvals, timestamps, `failure_reason`.
+  - CreditPackage: slugged packages with `price`, `credits`, `discount_percentage`, stripe/paypal ids, `is_active`, `is_featured`, `sort_order`, `features`.
+  - CreditTransaction: `transaction_type`, `amount`, `balance_after`, `metadata`, `reference_id`.
+  - CreditUsageLog: `operation_type`, `credits_used`, tokens/model, related object refs, metadata.
+  - CreditCostConfig: per-operation configurable costs (admin-editable).
+  - AccountPaymentMethod: CRUD + default toggle; PaymentMethodConfig for availability by country/method.
+- Account (`auth/models.py`): Account with billing fields, credits balance, status, owner, stripe_customer_id; base models enforce tenant scoping.
+- Other business areas (content, planning, optimization, publishing, integration) define models/services under `backend/igny8_core/business/*` (models present in `content`, `planning`, `optimization`, `publishing`, `integration`; `linking` uses services).
+
+## 4) Services & Admin Surfaces
+- Billing services: `business/billing/services/invoice_service.py`, `payment_service.py` (used by InvoiceViewSet/PaymentViewSet/admin aliases).
+- Admin billing aliases in `modules/billing/admin_urls.py` map to `AdminBillingViewSet` plus legacy stats/credit-cost endpoints.
+- Module enable and settings handled in `modules/system/settings_views.py` and integration settings in `modules/system/integration_views.py`.
+
+## 5) Automation & Tasks
+- Automation API exposed via `business/automation/urls.py` (AutomationViewSet).
+- Celery tasks scaffolded under `igny8_core/tasks` and `tasks.py`; workers started via standard Celery entrypoints (see repo root README for commands).
+
+## 6) Integration & Webhooks
+- WordPress webhooks: `/api/v1/integration/webhooks/wordpress/status/`, `/metadata/`.
+- Gitea webhook: `/api/v1/system/webhook/`.
+- Public site definition: `/api/v1/publisher/sites/<site_id>/definition/`.
+
+## 7) Observations / gaps for follow-up
+- `docs/user-flow/` lacks flows; backend routes above should anchor those docs.
+- Ensure billing/admin docs use the namespaces and models listed here (avoid legacy paths). Data shape examples still need to be added alongside serializers.
+
+

docs/backend/IGNY8-PLANNER-BACKEND.md

@@ -0,0 +1,54 @@
+# Planner Module (Backend) — Code-Sourced Overview (Dec 2025)
+
+Scope: `backend/igny8_core/modules/planner` (ViewSets/serializers/urls) and backing models in `backend/igny8_core/business/planning`.
+
+## Endpoints (routed from `modules/planner/urls.py`)
+- `/api/v1/planner/keywords/` — CRUD + bulk actions, CSV import/export, clustering trigger.
+- `/api/v1/planner/clusters/` — CRUD with aggregated stats (keywords/volume/difficulty/ideas/content).
+- `/api/v1/planner/ideas/` — CRUD for content ideas tied to clusters/keywords.
+
+## Models (business/planning/models.py)
+- **Clusters** (`SiteSectorBaseModel`, soft-delete): `name`, `description`, `keywords_count`, `volume`, `mapped_pages`, `status(new|mapped)`, `disabled`; unique per `site, sector`. Indexes on `name`, `status`, `site, sector`.
+- **Keywords** (`SiteSectorBaseModel`, soft-delete): FK `seed_keyword` (global), overrides `volume_override`, `difficulty_override`, `attribute_values`; FK `cluster` (same sector); `status(new|mapped)`, `disabled`. Unique per `seed_keyword, site, sector`. Properties expose `keyword`, `volume`, `difficulty`, `intent` from seed keyword. Save enforces industry/sector alignment between site/sector and seed keyword.
+- **ContentIdeas** (`SiteSectorBaseModel`, soft-delete): `idea_title`, `description`, `target_keywords` (legacy), M2M `keyword_objects`, FK `keyword_cluster` (same sector), `status(new|queued|completed)`, `disabled`, `estimated_word_count`, `content_type` (post/page/product/taxonomy), `content_structure` (article/guide/etc.). Tracks metadata (audience, tone, outlines, brief), CTA fields, review flags (`is_priority`, `is_assigned`, `is_review_required`), language, published URL, audit fields. Indexes on `site, sector, keyword_cluster, status, idea_title`.
+
+## Serializers
+- **KeywordSerializer**: read-only keyword metrics from seed keyword; requires `seed_keyword_id` on create; optional on update; validates site/sector via the ViewSet; surfaces `cluster_name`, `sector_name`, `volume/difficulty/intent`.
+- **ClusterSerializer** (`cluster_serializers.py`): computes `keywords_count`, `volume`, `difficulty`, `ideas_count`, `content_count`; bulk prefetch helper to avoid N+1. Annotated volume/difficulty via overrides or seed keyword values.
+- **ContentIdeasSerializer**: links to clusters and keywords; exposes cluster/sector names; supports site/sector write-only ids; legacy taxonomy getter retained for backward compatibility.
+
+## ViewSets (modules/planner/views.py)
+- **KeywordViewSet** (`SiteSectorModelViewSet`):
+  - Filtering/search/order: search by seed keyword text; filter by status, cluster, intent, seed_keyword_id; ordering by created_at, volume, difficulty.
+  - Custom filters: difficulty_min/max, volume_min/max using override-first logic.
+  - Bulk actions: `bulk_delete`, `bulk_update` (status), `bulk_add_from_seed` (validates industry/sector alignment, requires site/sector), `import_keywords` CSV (site/sector required, creates Keywords), `export` CSV (ids filter supported).
+  - Clustering: `auto_cluster` calls `ClusteringService.cluster_keywords`; enforces min keyword count (5) and credits; returns async task_id or sync result.
+  - Create requires site_id and sector_id (validated against site/sector and account).
+- **ClusterViewSet**:
+  - Filters/search/order: search by name; filter by status; ordering on name/created_at/keywords_count/volume/difficulty.
+  - Annotates volume/difficulty using overrides; uses serializer prefetch to reduce N+1.
+  - Export endpoints not present; CRUD via standard actions.
+- **ContentIdeasViewSet**:
+  - CRUD for ideas; filters on status/cluster; search by title/description/keywords; ordering by created_at/status.
+  - Generates title/brief via `IdeasService.generate_content_idea`; enforces credits (catches `InsufficientCreditsError`).
+  - Bulk assign status not present; focused on single-item generation and listing.
+
+## Services (business/planning/services)
+- **ClusteringService**: clusters keywords; invoked by `auto_cluster`; credit-aware.
+- **IdeasService**: generates content ideas (title/brief) from keywords; used in `ContentIdeasViewSet`.
+
+## Permissions, tenancy, and throttling
+- Permissions: `IsAuthenticatedAndActive` + `IsViewerOrAbove` for all planner endpoints.
+- Tenancy: `SiteSectorModelViewSet` base ensures account/site/sector scoping; create/import/add-from-seed require explicit site_id + sector_id with consistency checks.
+- Throttling: `DebugScopedRateThrottle` with scope `planner`.
+
+## CSV Import/Export
+- Export: `/planner/keywords/export` supports ids filter; outputs CSV with keyword, volume, difficulty, intent, status, cluster.
+- Import: `/planner/keywords/import_keywords` expects CSV with keyword, volume, difficulty, intent, status; site_id and sector_id required in query params; skips duplicates per site/sector/account.
+
+## Notes / gaps
+- Two `bulk_update` methods exist in `KeywordViewSet` (duplicate definitions); consolidate to one.
+- No dedicated bulk idea status update; consider parity with keywords.
+- Legacy taxonomy references removed; ContentIdeas retains a legacy getter for taxonomy name but model FK is removed.
+
+

docs/backend/IGNY8-WRITER-BACKEND.md

@@ -0,0 +1,73 @@
+# Writer Module (Backend) — Code-Sourced Overview (Dec 2025)
+
+Scope: `backend/igny8_core/modules/writer` (ViewSets/serializers/urls) backed by models in `backend/igny8_core/business/content`.
+
+## Endpoints (from `modules/writer/urls.py`)
+- `/api/v1/writer/tasks/` — CRUD + bulk delete/update, auto content generation.
+- `/api/v1/writer/images/` — CRUD + serve file, auto generate images, bulk status update (by content or ids), grouped content images.
+- `/api/v1/writer/content/` — CRUD + publish/unpublish to WordPress, check WordPress status, AI prompt generation, validation, metadata mapping, taxonomy/tag management.
+- `/api/v1/writer/taxonomies/` — CRUD for content taxonomies (categories/tags). Attributes endpoint disabled (serializer removed).
+
+## Models (business/content/models.py)
+- **Tasks** (`SiteSectorBaseModel`, soft-delete): title, description, FK `cluster` (required, same sector), optional `idea`, `content_type` (post/page/product/taxonomy), `content_structure` (article/guide/landing_page/etc.), optional `taxonomy_term`, comma keywords, word_count, status (`queued|completed`). Indexes on title/status/cluster/type/structure/site+sector.
+- **Content** (`SiteSectorBaseModel`, soft-delete): title, HTML, word_count, SEO fields, FK `cluster`, `content_type/structure`, M2M `taxonomy_terms` (through `ContentTaxonomyRelation`), external publishing fields (`external_id/url/type/metadata/sync_status`), source (`igny8|wordpress`), status (`draft|review|published`). Indexes on title/cluster/type/structure/source/status/external_id/site+sector.
+- **ContentTaxonomy** (`SiteSectorBaseModel`): name, slug, `taxonomy_type (category|tag)`, external_taxonomy/id, sync_status, description, count, metadata. Unique per site+slug+type and site+external_id+external_taxonomy.
+- **Images** (`SiteSectorBaseModel`, soft-delete): FK `content` (preferred) or `task` (legacy), `image_type (featured|desktop|mobile|in_article)`, `image_url/path`, prompt, status (`pending|generated|failed`), position. save() inherits account/site/sector from content/task. Indexes on content/task type/status/position.
+- **ContentClusterMap** (summary): links content/tasks to clusters with roles (hub/supporting/attribute) and source (blueprint/manual/import); tracks keywords + slugs.
+- **ContentTaxonomyRelation**: through table for content↔taxonomy.
+
+## Serializers (modules/writer/serializers.py)
+- **TasksSerializer**: requires cluster/content_type/content_structure on create; defaults status=queued; exposes cluster_name/sector_name; accepts site_id/sector_id (write-only).
+- **ImagesSerializer**: exposes task_title/content_title; read-only account/timestamps.
+- **ContentSerializer**: requires cluster/content_type/content_structure/title on create; defaults source=igny8, status=draft; exposes taxonomy terms, tags, categories, image flags/status; site_id/sector_id write-only.
+- **ContentTaxonomySerializer** (in file but not shown above): manages taxonomy fields and external sync data.
+- Group serializers for content images (`ContentImageSerializer`, `ContentImagesGroupSerializer`).
+
+## ViewSets & Actions (modules/writer/views.py)
+- **TasksViewSet** (`SiteSectorModelViewSet`):
+  - Filters/search/order: search title/keywords; filter status/cluster/content_type/structure; order by title/created_at/status.
+  - Bulk: `bulk_delete`, `bulk_update` (status).
+  - AI: `auto_generate_content` → `ContentGenerationService.generate_content(ids, account)`; limits 10 ids; returns task_id or sync result; 402 on `InsufficientCreditsError`.
+  - Create requires explicit `site_id` and `sector_id`; validates site/sector association; account resolved from request user/site.
+- **ImagesViewSet**:
+  - Filters/order: filter task_id/content_id/image_type/status; order by created_at/position/id.
+  - Create enforces site/sector (from request context, falling back to user active site/default sector); sets account/site/sector.
+  - Actions:
+    - `serve_image_file` streams local file if `image_path` exists.
+    - `auto_generate_images` queues `run_ai_task(generate_images)` (Celery if available) for up to 10 task_ids.
+    - `bulk_update` sets status by `content_id` or `ids`.
+    - `content_images` returns grouped images (featured + in-article) via grouped serializer.
+- **ContentViewSet** (not fully shown above but key actions):
+  - CRUD with filters/search/order (title, status, cluster, content_type/structure, source); tenant scoping via base class.
+  - Taxonomy management: add/remove terms, sync helpers (uses `ContentTaxonomy`).
+  - AI helpers: `generate_image_prompts` (queues `run_ai_task(generate_image_prompts)`), validation (`ContentValidationService`), metadata mapping (`MetadataMappingService`), content generation pathways tied to `ContentGenerationService`.
+  - Publishing: `publish` queues `publish_content_to_wordpress` Celery task (optimistically sets status=published), `wordpress_status` fetches WP status via integration, `unpublish` clears external links and reverts to draft.
+  - Image status helpers and grouped image retrieval also exposed.
+- **ContentTaxonomyViewSet**: CRUD for categories/tags; supports external sync fields.
+
+## Permissions, tenancy, throttling
+- Permissions: `IsAuthenticatedAndActive` + `IsViewerOrAbove` across Writer; certain actions (e.g., `unpublish`) use `IsEditorOrAbove`.
+- Tenancy: `SiteSectorModelViewSet` enforces account/site/sector scoping; create operations require site_id+sector_id or context site/sector; Images save derives account/site/sector from content/task.
+- Throttling: `DebugScopedRateThrottle` with scope `writer`.
+
+## AI/Background Tasks & Limits
+- Content generation: `auto_generate_content` (max 10 task IDs) → `ContentGenerationService`; credits enforced (402 on insufficient).
+- Image generation: `auto_generate_images` (max 10 task IDs) via `run_ai_task`; Celery preferred, sync fallback.
+- Image prompt generation: `generate_image_prompts` via `run_ai_task`.
+
+## Publishing Integration
+- Publish: `/writer/content/{id}/publish/` → queues WordPress publish; refuses if already published.
+- Status: `/writer/content/{id}/wordpress_status/` → checks WordPress via site integration.
+- Unpublish: clears external_id/url, sets status to draft (does not delete WP post).
+
+## CSV / Bulk Notes
+- Tasks: bulk delete/update supported.
+- Images: bulk status update by content or ids; grouped retrieval.
+- Content: bulk operations primarily around AI prompt/image generation and publishing; no CSV import/export.
+
+## Observations / gaps
+- Tasks status choices limited to `queued/completed`; no explicit error/cancel states.
+- Images bulk_update exposed on same ViewSet name as Tasks bulk_update (distinct routes under images vs tasks); OK but keep consistent naming.
+- ContentViewSet file is large; ensure doc readers reference actions for validation/metadata mapping if extending.
+
+