docs re-org

docs/10-BACKEND/planner/PLANNER-REFERENCE.md

@@ -0,0 +1,80 @@
+# Planner Overview
+
+## Purpose
+Explain how the planner module manages keywords, clusters, and content ideas, including tenancy, validation, and API behaviors.
+
+## Code Locations (exact paths)
+- Models: `backend/igny8_core/business/planning/models.py`
+- Serializers: `backend/igny8_core/modules/planner/serializers.py`
+- Views: `backend/igny8_core/modules/planner/views.py`
+- URLs: `backend/igny8_core/modules/planner/urls.py`
+- Services: `backend/igny8_core/business/planning/services/clustering_service.py`, `ideas_service.py`
+
+## High-Level Responsibilities
+- Store site/sector-scoped keywords, clusters, and content ideas linked to global seed keywords.
+- Provide CRUD and bulk operations for keywords/clusters/ideas with search, filter, and ordering.
+- Validate industry/sector alignment between seed keywords and site/sector.
+- Feed downstream writer/automation stages with structured ideas and clusters.
+
+## Detailed Behavior
+- Models (site/sector scoped via `SiteSectorBaseModel` and soft-delete):
+  - `Keywords`: references global `SeedKeyword`; optional volume/difficulty overrides; optional cluster; status `new/mapped`; disabled flag; validation ensures seed keyword industry/sector match the site/sector.
+  - `Clusters`: topic groups with counts, volume, mapped_pages, status `new/mapped`; unique per site/sector by name.
+  - `ContentIdeas`: ideas tied to clusters and optional keyword objects; status `new/queued/completed`; content_type/structure and estimated_word_count; disabled flag.
+- Serializers:
+  - `KeywordSerializer` enforces `seed_keyword_id` on create; exposes seed keyword-derived fields (keyword/volume/difficulty/intent) read-only; optional overrides; includes site_id/sector_id write-only; provides cluster/sector names via getters; optional `attribute_values` when `USE_SITE_BUILDER_REFACTOR` flag is enabled.
+  - `ClusterSerializer` exposes site/sector IDs, sector name, read-only counts/volume/mapped_pages.
+  - `ContentIdeasSerializer` exposes cluster/sector names, content type/structure, keyword/target fields, site/sector write-only.
+- Views (`KeywordViewSet`, `ClusterViewSet`, `ContentIdeasViewSet`):
+  - Inherit `SiteSectorModelViewSet` for tenant/site/sector filtering and unified responses.
+  - Permissions: `IsAuthenticatedAndActive` + viewer-or-above (search/list CRUD).
+  - Pagination: `CustomPageNumberPagination`; throttle scope `planner`.
+  - Filters/Search/Ordering: keywords searchable by seed_keyword.keyword; filter by status, cluster_id, seed intent; ordering by created_at, volume, difficulty; custom range filters for difficulty/volume; clusters filter by status; ideas filter by status/cluster.
+  - Creation requires explicit `site_id` and `sector_id`; viewsets validate site/sector existence and alignment; set `account/site/sector` explicitly on save.
+  - Bulk endpoints:
+    - Keywords: `bulk_delete`, two definitions of `bulk_update` (status update) present; `bulk_add_from_seed` to create Keywords from SeedKeywords with site/sector validation.
+    - Clusters: bulk update endpoints in view (not shown above) adjust status.
+    - Ideas: creation and status updates follow standard CRUD; not shown as bulk endpoints in code snippet.
+  - Error handling: viewsets wrap errors into unified responses; return 400/404/500 as appropriate.
+- Services:
+  - `ClusteringService` and `IdeasService` (not detailed here) provide additional business logic for clustering/idea generation when invoked by automation or endpoints.
+
+## Data Structures / Models Involved (no code)
+- `Keywords`, `Clusters`, `ContentIdeas`; global `SeedKeyword`.
+- Tenancy entities: `Account`, `Site`, `Sector`.
+
+## Execution Flow
+- Requests → DRF auth → `SiteSectorModelViewSet` filters by account/site/sector → serializers validate seed/site/sector alignment → models persisted → responses returned via unified helpers; bulk actions operate on filtered querysets.
+
+## Cross-Module Interactions
+- Automation stages 1–3 consume planner data (keywords → clusters → ideas → tasks).
+- Writer tasks reference clusters/ideas produced here.
+- Billing credits may be checked upstream for AI clustering/idea generation (via services invoked in automation or planner actions).
+
+## State Transitions
+- Keywords: `new` → `mapped`; Clusters: `new` → `mapped`; ContentIdeas: `new` → `queued` → `completed`.
+- Disabled flag can exclude records from processes.
+
+## Error Handling
+- Validation enforces seed/site/sector industry alignment; missing site/sector on create raises validation errors.
+- Bulk operations return 400 on missing IDs/status; general errors return unified 500 responses.
+
+## Tenancy Rules
+- All planner models inherit `SiteSectorBaseModel`; queries are filtered by account/site/sector; admin/developer/system roles can bypass filtering via base viewset logic.
+- Create requires explicit site and sector; viewsets fetch and set account from site/user/middleware.
+
+## Billing Rules
+- None directly in the planner endpoints; credit costs for clustering/idea generation are enforced where those services are called (automation or planner service usage).
+
+## Background Tasks / Schedulers
+- Planner endpoints run synchronously; automation may batch planner data into AI tasks via Celery (documented in automation).
+
+## Key Design Considerations
+- Strict site/sector validation prevents cross-tenant or cross-site leaks.
+- Seed keyword linkage centralizes keyword metadata; overrides allow per-site tuning.
+- Pagination/filtering/search are optimized for large keyword sets.
+
+## How Developers Should Work With This Module
+- Use provided viewsets and serializers; ensure site_id/sector_id are supplied on create.
+- When adding fields, extend serializers with read-only/write-only behavior consistent with current patterns.
+- For new bulk operations, follow existing patterns: validate IDs, operate on filtered queryset, return unified responses.