igny8/master-docs/10-backend/planner/PLANNER-OVERVIEW.md

Planner Overview

Purpose

Explain how the planner module manages keywords, clusters, and content ideas, including tenancy, validation, and API behaviors.

Code Locations (exact paths)

• Models: backend/igny8_core/business/planning/models.py
• Serializers: backend/igny8_core/modules/planner/serializers.py
• Views: backend/igny8_core/modules/planner/views.py
• URLs: backend/igny8_core/modules/planner/urls.py
• Services: backend/igny8_core/business/planning/services/clustering_service.py, ideas_service.py

High-Level Responsibilities

• Store site/sector-scoped keywords, clusters, and content ideas linked to global seed keywords.
• Provide CRUD and bulk operations for keywords/clusters/ideas with search, filter, and ordering.
• Validate industry/sector alignment between seed keywords and site/sector.
• Feed downstream writer/automation stages with structured ideas and clusters.

Detailed Behavior

• Models (site/sector scoped via SiteSectorBaseModel and soft-delete):
  • Keywords: references global SeedKeyword; optional volume/difficulty overrides; optional cluster; status new/mapped; disabled flag; validation ensures seed keyword industry/sector match the site/sector.
  • Clusters: topic groups with counts, volume, mapped_pages, status new/mapped; unique per site/sector by name.
  • ContentIdeas: ideas tied to clusters and optional keyword objects; status new/queued/completed; content_type/structure and estimated_word_count; disabled flag.
• Serializers:
  • KeywordSerializer enforces seed_keyword_id on create; exposes seed keyword-derived fields (keyword/volume/difficulty/intent) read-only; optional overrides; includes site_id/sector_id write-only; provides cluster/sector names via getters; optional attribute_values when USE_SITE_BUILDER_REFACTOR flag is enabled.
  • ClusterSerializer exposes site/sector IDs, sector name, read-only counts/volume/mapped_pages.
  • ContentIdeasSerializer exposes cluster/sector names, content type/structure, keyword/target fields, site/sector write-only.
• Views (KeywordViewSet, ClusterViewSet, ContentIdeasViewSet):
  • Inherit SiteSectorModelViewSet for tenant/site/sector filtering and unified responses.
  • Permissions: IsAuthenticatedAndActive + viewer-or-above (search/list CRUD).
  • Pagination: CustomPageNumberPagination; throttle scope planner.
  • Filters/Search/Ordering: keywords searchable by seed_keyword.keyword; filter by status, cluster_id, seed intent; ordering by created_at, volume, difficulty; custom range filters for difficulty/volume; clusters filter by status; ideas filter by status/cluster.
  • Creation requires explicit site_id and sector_id; viewsets validate site/sector existence and alignment; set account/site/sector explicitly on save.
  • Bulk endpoints:
    • Keywords: bulk_delete, two definitions of bulk_update (status update) present; bulk_add_from_seed to create Keywords from SeedKeywords with site/sector validation.
    • Clusters: bulk update endpoints in view (not shown above) adjust status.
    • Ideas: creation and status updates follow standard CRUD; not shown as bulk endpoints in code snippet.
  • Error handling: viewsets wrap errors into unified responses; return 400/404/500 as appropriate.
• Services:
  • ClusteringService and IdeasService (not detailed here) provide additional business logic for clustering/idea generation when invoked by automation or endpoints.

Data Structures / Models Involved (no code)

• Keywords, Clusters, ContentIdeas; global SeedKeyword.
• Tenancy entities: Account, Site, Sector.

Execution Flow

• Requests → DRF auth → SiteSectorModelViewSet filters by account/site/sector → serializers validate seed/site/sector alignment → models persisted → responses returned via unified helpers; bulk actions operate on filtered querysets.

Cross-Module Interactions

• Automation stages 1–3 consume planner data (keywords → clusters → ideas → tasks).
• Writer tasks reference clusters/ideas produced here.
• Billing credits may be checked upstream for AI clustering/idea generation (via services invoked in automation or planner actions).

State Transitions

• Keywords: new → mapped; Clusters: new → mapped; ContentIdeas: new → queued → completed.
• Disabled flag can exclude records from processes.

Error Handling

• Validation enforces seed/site/sector industry alignment; missing site/sector on create raises validation errors.
• Bulk operations return 400 on missing IDs/status; general errors return unified 500 responses.

Tenancy Rules

• All planner models inherit SiteSectorBaseModel; queries are filtered by account/site/sector; admin/developer/system roles can bypass filtering via base viewset logic.
• Create requires explicit site and sector; viewsets fetch and set account from site/user/middleware.

Billing Rules

• None directly in the planner endpoints; credit costs for clustering/idea generation are enforced where those services are called (automation or planner service usage).

Background Tasks / Schedulers

• Planner endpoints run synchronously; automation may batch planner data into AI tasks via Celery (documented in automation).

Key Design Considerations

• Strict site/sector validation prevents cross-tenant or cross-site leaks.
• Seed keyword linkage centralizes keyword metadata; overrides allow per-site tuning.
• Pagination/filtering/search are optimized for large keyword sets.

How Developers Should Work With This Module

• Use provided viewsets and serializers; ensure site_id/sector_id are supplied on create.
• When adding fields, extend serializers with read-only/write-only behavior consistent with current patterns.
• For new bulk operations, follow existing patterns: validate IDs, operate on filtered queryset, return unified responses.