docs rearrange
This commit is contained in:
alorig
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,19 +0,0 @@
|
||||
# IGNY8 Docs Changelog (Dec 2025)
|
||||
|
||||
## 2025-12-07
|
||||
### Features captured (code-sourced)
|
||||
- Added backend-only architecture doc (`docs/backend/IGNY8-BACKEND-ARCHITECTURE.md`) reflecting current namespaces, billing models, integrations, and webhooks.
|
||||
- Added planner backend doc (`docs/backend/IGNY8-PLANNER-BACKEND.md`) and writer backend doc (`docs/backend/IGNY8-WRITER-BACKEND.md`) from live code.
|
||||
- Replaced automation folder with single canonical `docs/automation/AUTOMATION-REFERENCE.md` (code-sourced pipeline, APIs, models, tasks, frontend).
|
||||
- Multi-tenant foundation via `AccountBaseModel` / `SiteSectorBaseModel` with tenant-scoped billing fields on `Account`.
|
||||
- Backend namespaces wired in `backend/igny8_core/urls.py` covering auth, account, planner, writer, system, billing (tenant), admin billing, automation, linker, optimizer, publisher, integration.
|
||||
- Billing tenant surface: invoices, payments (including manual submission + available methods), credit-packages, credit-transactions, payment-methods CRUD/default, credits balance/usage/transactions.
|
||||
- Billing admin surface under `/api/v1/admin/`: stats, users credit adjustments, credit costs, invoices/payments/pending approvals, payment-method configs, account payment methods.
|
||||
- WordPress integration webhooks (`/api/v1/integration/webhooks/wordpress/status|metadata/`) and Gitea webhook (`/api/v1/system/webhook/`).
|
||||
- Frontend routes mapped for Planner, Writer, Automation, Linker, Optimizer, Thinker, Billing module, Account pages, and Admin billing/management; sidebar paths defined in `frontend/src/layout/AppSidebar.tsx`.
|
||||
|
||||
### Issues / gaps to fix
|
||||
- `docs/user-flow/` is empty; needs end-to-end flows (account, billing, content) based on current routes and APIs.
|
||||
- Billing/account documentation must be aligned to the live namespaces and models captured in `docs/igny8-app/IGNY8-APP-ARCHITECTURE.md` (avoid legacy/retired paths).
|
||||
- Add API request/response samples for billing/admin endpoints to the billing docs to match the current serializers (not yet documented here).
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
# IGNY8 Documentation Index (Dec 2025)
|
||||
|
||||
Purpose: single entry point for all project docs with live links to the current sources. This index reflects the codebase as implemented, not legacy placeholders.
|
||||
|
||||
## Codebase Snapshot
|
||||
- **Backend (Django/DRF):** Multi-tenant models inherit `AccountBaseModel`/`SiteSectorBaseModel` for automatic tenant scoping (`backend/igny8_core/auth/models.py`). API namespaces are wired in `backend/igny8_core/urls.py`: `/api/v1/auth/`, `/api/v1/account/`, `/api/v1/planner/`, `/api/v1/writer/`, `/api/v1/system/`, `/api/v1/billing/` (tenant billing/invoices/credits), `/api/v1/admin/` (billing admin + credit costs/stats), `/api/v1/automation/`, `/api/v1/linker/`, `/api/v1/optimizer/`, `/api/v1/publisher/`, `/api/v1/integration/`.
|
||||
- **Frontend (React/Vite):** Routing lives in `frontend/src/App.tsx` with lazy-loaded pages; account/billing pages at `/account/plans`, `/account/billing`, `/account/purchase-credits`, `/account/settings`, `/account/team`, `/account/usage`, plus tenant billing module routes under `/billing/*` and admin billing under `/admin/*`. Sidebar labels/paths are defined in `frontend/src/layout/AppSidebar.tsx`.
|
||||
|
||||
## Document Map
|
||||
- **App Architecture (code-sourced):** `docs/igny8-app/IGNY8-APP-ARCHITECTURE.md`
|
||||
- **Backend Architecture (code-sourced):** `docs/backend/IGNY8-BACKEND-ARCHITECTURE.md`
|
||||
- **Planner Backend Detail:** `docs/backend/IGNY8-PLANNER-BACKEND.md`
|
||||
- **Writer Backend Detail:** `docs/backend/IGNY8-WRITER-BACKEND.md`
|
||||
- **Platform / Tech Stack:** `docs/tech-stack/00-SYSTEM-ARCHITECTURE-MASTER-REFERENCE.md`
|
||||
- **API:** `docs/API/01-IGNY8-REST-API-COMPLETE-REFERENCE.md`, `docs/API/API-COMPLETE-REFERENCE-LATEST.md`
|
||||
- **Billing & Credits (canonical):** `docs/billing/billing-account-final-plan-2025-12-05.md`, `docs/billing/credits-system-audit-and-improvement-plan.md`
|
||||
- **App Feature Guides:** `docs/igny8-app/02-PLANNER-WRITER-WORKFLOW-TECHNICAL-GUIDE.md`, `docs/igny8-app/05-WRITER-IMAGES-PAGE-SYSTEM-DESIGN.md`, `docs/igny8-app/06-FEATURE-MODIFICATION-DEVELOPER-GUIDE.md`, `docs/igny8-app/KEYWORDS-CLUSTERS-IDEAS-COMPLETE-MAPPING.md`, `docs/igny8-app/app-packaging-backaup-plan.md`, `docs/igny8-app/TAXONOMY/*`, `docs/igny8-app/status-related-temporary/*`
|
||||
- **Automation:** `docs/automation/` (automation-plan, implementation analysis, UX improvements, stage-6 image generation fix, cluster validation fix plan)
|
||||
- **AI Functions:** `docs/ai/AI-FUNCTIONS-COMPLETE-REFERENCE.md`
|
||||
- **Automation:** `docs/automation/AUTOMATION-REFERENCE.md`
|
||||
- **WordPress Integration:** `docs/wp/` (API integration guide, bidirectional sync reference, publishing field mapping, refactor/implementation summaries, deployment fixes)
|
||||
- **User Flow Drafts:** root-level `user-flow-plan*.md` (in-progress drafts); `docs/user-flow/` is currently empty and needs coverage.
|
||||
- **Docs Changelog:** `docs/CHANGELOG.md`
|
||||
|
||||
## Known Documentation Gaps to Close
|
||||
- App-level architecture summary sourced from code (backend modules and frontend routes), including tenant scoping rules.
|
||||
- Billing/account accuracy pass: align docs to current endpoints in `igny8_core/urls.py` and admin aliases in `modules/billing/admin_urls.py`, plus the active frontend routes in `App.tsx`.
|
||||
- Frontend route/UX map for account/billing/admin sections tied to sidebar paths.
|
||||
- Multi-tenancy guardrails doc (account/site/sector isolation, role expectations) based on `auth/models.py`.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,148 +0,0 @@
|
||||
# Automation Module (Code-Sourced, Dec 2025)
|
||||
|
||||
Single canonical reference for IGNY8 automation (backend, frontend, and runtime behavior). Replaces all prior automation docs in this folder.
|
||||
|
||||
---
|
||||
|
||||
## 1) What Automation Does
|
||||
- Runs the 7-stage pipeline across Planner/Writer:
|
||||
1) Keywords → Clusters (AI)
|
||||
2) Clusters → Ideas (AI)
|
||||
3) Ideas → Tasks (Local)
|
||||
4) Tasks → Content (AI)
|
||||
5) Content → Image Prompts (AI)
|
||||
6) Image Prompts → Images (AI)
|
||||
7) Manual Review Gate (Manual)
|
||||
- Per-site, per-account isolation. One run at a time per site; guarded by cache lock `automation_lock_{site_id}`.
|
||||
- Scheduling via Celery beat (`automation.check_scheduled_automations`); execution via Celery tasks (`run_automation_task`, `resume_automation_task` / `continue_automation_task`).
|
||||
|
||||
---
|
||||
|
||||
## 2) Backend API (behavior + payloads)
|
||||
Base: `/api/v1/automation/` (auth required; site must belong to user’s account).
|
||||
|
||||
- `GET config?site_id=`: returns or creates config with enable flag, frequency (`daily|weekly|monthly`), scheduled_time, stage_1..6 batch sizes, delays (`within_stage_delay`, `between_stage_delay`), last_run_at, next_run_at.
|
||||
- `PUT update_config?site_id=`: same fields as above, updates in-place.
|
||||
- `POST run_now?site_id=`: starts a manual run; enqueues `run_automation_task`. Fails if a run is already active or lock exists.
|
||||
- `GET current_run?site_id=`: current running/paused run with status, current_stage, totals, and stage_1..7_result blobs (counts, credits, partial flags, skip reasons).
|
||||
- `GET pipeline_overview?site_id=`: per-stage status counts and “pending” numbers for UI cards.
|
||||
- `GET current_processing?site_id=&run_id=`: live processing snapshot for an active run; null if not running.
|
||||
- `POST pause|resume|cancel?site_id=&run_id=`: pause after current item; resume from saved `current_stage`; cancel after current item and stamp cancelled_at/completed_at.
|
||||
- `GET history?site_id=`: last 20 runs (id, status, trigger, timestamps, total_credits_used, current_stage).
|
||||
- `GET logs?run_id=&lines=100`: tail of the per-run activity log written by AutomationLogger.
|
||||
- `GET estimate?site_id=`: estimated_credits, current_balance, sufficient (balance >= 1.2x estimate).
|
||||
|
||||
Error behaviors:
|
||||
- Missing site_id/run_id → 400.
|
||||
- Site not in account → 404.
|
||||
- Run not found → 404 on run-specific endpoints.
|
||||
- Already running / lock held → 400 on run_now.
|
||||
|
||||
---
|
||||
|
||||
## 3) Data Model (runtime state)
|
||||
- `AutomationConfig` (one per site): enable flag, schedule (frequency, time), batch sizes per stage (1–6), delays (within-stage, between-stage), last_run_at, next_run_at.
|
||||
- `AutomationRun`: run_id, trigger_type (manual/scheduled), status (running/paused/cancelled/completed/failed), current_stage, timestamps (start/pause/resume/cancel/complete), total_credits_used, per-stage result JSON (stage_1_result … stage_7_result), error_message.
|
||||
- Activity logs: one file per run via AutomationLogger; streamed through the `logs` endpoint.
|
||||
|
||||
---
|
||||
|
||||
## 4) How Execution Works (AutomationService)
|
||||
- Start: grabs cache lock `automation_lock_{site_id}`, estimates credits, enforces 1.2x balance check, creates AutomationRun and log file.
|
||||
- AI functions used: Stage 1 `AutoClusterFunction`; Stage 2 `GenerateIdeasFunction`; Stage 4 `GenerateContentFunction`; Stage 5 `GenerateImagePromptsFunction`; Stage 6 uses `process_image_generation_queue` (not the partial `generate_images` AI function).
|
||||
- Stage flow (per code):
|
||||
- Stage 1 Keywords → Clusters: require ≥5 keywords (validate_minimum_keywords); batch by config; AIEngine clustering; records keywords_processed, clusters_created, batches, credits, time; skips if insufficient keywords.
|
||||
- Stage 2 Clusters → Ideas: batch by config; AIEngine ideas; records ideas_created.
|
||||
- Stage 3 Ideas → Tasks: local conversion of queued ideas to tasks; batches by config; no AI.
|
||||
- Stage 4 Tasks → Content: batch by config; AIEngine content; records content count + word totals.
|
||||
- Stage 5 Content → Image Prompts: batch by config; AIEngine image-prompts into Images (featured + in-article).
|
||||
- Stage 6 Image Prompts → Images: uses `process_image_generation_queue` with provider/model from IntegrationSettings; updates Images status.
|
||||
- Stage 7 Manual Review Gate: marks ready-for-review counts; no AI.
|
||||
- Control: each stage checks `_check_should_stop` (paused/cancelled); saves partial progress (counts, credits) before returning; resume continues from `current_stage`.
|
||||
- Credits: upfront estimate check (1.2x buffer) before starting; AIEngine per-call pre-checks and post-SAVE deductions; `total_credits_used` accumulates.
|
||||
- Locks: acquired on start; cleared on completion or failure; also cleared on fatal errors in tasks.
|
||||
- Errors: any unhandled exception marks run failed, sets error_message, logs error, clears lock; pipeline_overview/history reflect status.
|
||||
- Stage result fields (persisted):
|
||||
- S1: keywords_processed, clusters_created, batches_run, credits_used, skipped/partial flags, time_elapsed.
|
||||
- S2: clusters_processed, ideas_created, batches_run, credits_used.
|
||||
- S3: ideas_processed, tasks_created, batches_run.
|
||||
- S4: tasks_processed, content_created, total_words, batches_run, credits_used.
|
||||
- S5: content_processed, prompts_created, batches_run, credits_used.
|
||||
- S6: images_processed, images_generated, batches_run.
|
||||
- S7: ready_for_review counts.
|
||||
|
||||
Batching & delays:
|
||||
- Configurable per site; stage_1..6 batch sizes control how many items per batch; `within_stage_delay` pauses between batches; `between_stage_delay` between stages.
|
||||
|
||||
Scheduling:
|
||||
- `check_scheduled_automations` runs hourly; respects frequency/time and last_run_at (~23h guard); skips if a run is active; sets next_run_at; starts `run_automation_task`.
|
||||
|
||||
Celery execution:
|
||||
- `run_automation_task` runs stages 1→7 sequentially for a run_id; failures mark run failed and clear lock.
|
||||
- `resume_automation_task` / `continue_automation_task` continue from saved `current_stage`.
|
||||
- Workers need access to cache (locks) and IntegrationSettings (models/providers).
|
||||
|
||||
Image pipeline specifics:
|
||||
- Stage 5 writes prompts to Images (featured + ordered in-article).
|
||||
- Stage 6 generates images via queue helper; AI `generate_images` remains partial/broken and is not used by automation.
|
||||
|
||||
---
|
||||
|
||||
## 5) Frontend Behavior (AutomationPage)
|
||||
- Route: `/automation`.
|
||||
- What the user can do: run now, pause, resume, cancel; edit config (enable/schedule, batch sizes, delays); view activity log; view history; watch live processing card and pipeline cards update.
|
||||
- Polling: every ~5s while a run is running/paused for current_run, pipeline_overview, metrics, current_processing; lighter polling when idle.
|
||||
- Metrics: fetched via low-level endpoints (keywords/clusters/ideas/tasks/content/images) for authoritative counts.
|
||||
- States shown: running, paused, cancelled, failed, completed; processing card shown when a run exists; pipeline cards use “pending” counts from pipeline_overview.
|
||||
- Activity log: pulled from `logs` endpoint; shown in UI for live tailing.
|
||||
|
||||
---
|
||||
|
||||
## 6) Configuration & Dependencies
|
||||
- Needs IntegrationSettings for AI models and image providers (OpenAI/runware).
|
||||
- Requires Celery beat and workers; cache backend required for locks.
|
||||
- Tenant scoping everywhere: site + account filtering on all automation queries.
|
||||
|
||||
---
|
||||
|
||||
## 7) Known Limitations and Gaps
|
||||
- `generate_images` AI function is partial/broken; automation uses queue helper instead.
|
||||
- Pause/Cancel stop after the current item; no mid-item abort.
|
||||
- Batch defaults are conservative (e.g., stage_2=1, stage_4=1); tune per site for throughput.
|
||||
- Stage 7 is manual; no automated review step.
|
||||
- No automated test suite observed for automation pipeline (stage transitions, pause/resume/cancel, scheduling guards, credit estimation/deduction).
|
||||
- Enhancements to consider: fix or replace `generate_images`; add mid-item abort; surface lock status/owner; broaden batch defaults after validation; add operator-facing doc in app; add tests.
|
||||
|
||||
---
|
||||
|
||||
## 8) Field/Behavior Quick Tables
|
||||
|
||||
### Pipeline “pending” definitions (pipeline_overview)
|
||||
- Stage 1: Keywords with status `new`, cluster is null, not disabled.
|
||||
- Stage 2: Clusters status `new`, not disabled, with no ideas.
|
||||
- Stage 3: ContentIdeas status `new`.
|
||||
- Stage 4: Tasks status `queued`.
|
||||
- Stage 5: Content status `draft` with zero images.
|
||||
- Stage 6: Images status `pending`.
|
||||
- Stage 7: Content status `review`.
|
||||
|
||||
### Stage result fields (stored on AutomationRun)
|
||||
- S1: keywords_processed, clusters_created, batches_run, credits_used, skipped, partial, time_elapsed.
|
||||
- S2: clusters_processed, ideas_created, batches_run, credits_used.
|
||||
- S3: ideas_processed, tasks_created, batches_run.
|
||||
- S4: tasks_processed, content_created, total_words, batches_run, credits_used.
|
||||
- S5: content_processed, prompts_created, batches_run, credits_used.
|
||||
- S6: images_processed, images_generated, batches_run.
|
||||
- S7: ready_for_review.
|
||||
|
||||
### Credit handling
|
||||
- Pre-run: estimate_credits * 1.2 vs account.credits (fails if insufficient).
|
||||
- Per AI call: AIEngine pre-check credits; post-SAVE deduction with cost/tokens tracked; total_credits_used aggregates deductions.
|
||||
|
||||
### Logging
|
||||
- Per-run log file via AutomationLogger; accessed with `GET logs?run_id=&lines=`; includes stage start/progress/errors and batch info.
|
||||
|
||||
### Polling (frontend)
|
||||
- Active run: ~5s cadence for current_run, pipeline_overview, metrics, current_processing, logs tail.
|
||||
- Idle: lighter polling (current_run/pipeline_overview) to show readiness and pending counts.
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
# IGNY8 Backend Architecture (code-sourced, Dec 2025)
|
||||
|
||||
Purpose: backend-only view based on current code under `backend/`. No legacy docs or assumptions.
|
||||
|
||||
## 1) Stack & Cross-Cutting
|
||||
- Django + DRF; Celery present via `backend/igny8_core/celery.py` and `tasks/`.
|
||||
- Multi-tenancy: models inherit `AccountBaseModel` (tenant FK `tenant_id`) or `SiteSectorBaseModel` for site/sector scoping (`backend/igny8_core/auth/models.py`). Account carries billing fields (email/address/tax_id).
|
||||
- Project URLs wired in `backend/igny8_core/urls.py`.
|
||||
|
||||
## 2) API Namespaces (as routed)
|
||||
- `/api/v1/auth/` → `igny8_core.auth.urls` (auth + CSV admin helpers).
|
||||
- `/api/v1/account/` → `igny8_core.api.urls` (account settings, team, usage analytics).
|
||||
- `/api/v1/planner/` → `igny8_core.modules.planner.urls` (keywords, clusters, ideas ViewSets).
|
||||
- `/api/v1/writer/` → `igny8_core.modules.writer.urls` (tasks, images, content, taxonomies ViewSets).
|
||||
- `/api/v1/system/` → `igny8_core.modules.system.urls` (prompts, author profiles, strategies, settings: system/account/user/modules/ai, module-enable toggle, health/status/metrics, Gitea webhook, integration settings save/test/generate/progress).
|
||||
- `/api/v1/billing/` → `igny8_core.business.billing.urls` (tenant billing: invoices, payments, credit-packages, credit-transactions, payment-methods CRUD/default, credits balance/usage/transactions; manual payment submit; available methods).
|
||||
- `/api/v1/admin/` → `igny8_core.modules.billing.admin_urls` (billing admin: stats, user credit adjustments, credit costs, invoices/payments/pending approvals, approve/reject, payment-method configs, account payment methods CRUD/default).
|
||||
- `/api/v1/automation/` → `igny8_core.business.automation.urls` (automation ViewSet).
|
||||
- `/api/v1/linker/` → `igny8_core.modules.linker.urls` (linker ViewSet).
|
||||
- `/api/v1/optimizer/` → `igny8_core.modules.optimizer.urls` (optimizer ViewSet).
|
||||
- `/api/v1/publisher/` → `igny8_core.modules.publisher.urls` (publishing records, deployments, root publisher actions, public site definition).
|
||||
- `/api/v1/integration/` → `igny8_core.modules.integration.urls` (integrations ViewSet + WordPress status/metadata webhooks).
|
||||
- OpenAPI docs: `/api/schema/`, `/api/docs/`, `/api/redoc/`.
|
||||
|
||||
## 3) Key Domain Models (code references)
|
||||
- Billing (`business/billing/models.py`):
|
||||
- Invoice: `subtotal`, `tax`, `total`, `currency`, `status` (`draft|pending|paid|void|uncollectible`), dates, `line_items`, `stripe_invoice_id`, `payment_method`, billing period, helpers `subtotal_amount|tax_amount|total_amount`.
|
||||
- Payment: statuses include `pending_approval`, `processing`, `completed/succeeded`, `failed/refunded/cancelled`; methods `stripe|paypal|bank_transfer|local_wallet|manual`; intent/charge ids, manual references, approvals, timestamps, `failure_reason`.
|
||||
- CreditPackage: slugged packages with `price`, `credits`, `discount_percentage`, stripe/paypal ids, `is_active`, `is_featured`, `sort_order`, `features`.
|
||||
- CreditTransaction: `transaction_type`, `amount`, `balance_after`, `metadata`, `reference_id`.
|
||||
- CreditUsageLog: `operation_type`, `credits_used`, tokens/model, related object refs, metadata.
|
||||
- CreditCostConfig: per-operation configurable costs (admin-editable).
|
||||
- AccountPaymentMethod: CRUD + default toggle; PaymentMethodConfig for availability by country/method.
|
||||
- Account (`auth/models.py`): Account with billing fields, credits balance, status, owner, stripe_customer_id; base models enforce tenant scoping.
|
||||
- Other business areas (content, planning, optimization, publishing, integration) define models/services under `backend/igny8_core/business/*` (models present in `content`, `planning`, `optimization`, `publishing`, `integration`; `linking` uses services).
|
||||
|
||||
## 4) Services & Admin Surfaces
|
||||
- Billing services: `business/billing/services/invoice_service.py`, `payment_service.py` (used by InvoiceViewSet/PaymentViewSet/admin aliases).
|
||||
- Admin billing aliases in `modules/billing/admin_urls.py` map to `AdminBillingViewSet` plus legacy stats/credit-cost endpoints.
|
||||
- Module enable and settings handled in `modules/system/settings_views.py` and integration settings in `modules/system/integration_views.py`.
|
||||
|
||||
## 5) Automation & Tasks
|
||||
- Automation API exposed via `business/automation/urls.py` (AutomationViewSet).
|
||||
- Celery tasks scaffolded under `igny8_core/tasks` and `tasks.py`; workers started via standard Celery entrypoints (see repo root README for commands).
|
||||
|
||||
## 6) Integration & Webhooks
|
||||
- WordPress webhooks: `/api/v1/integration/webhooks/wordpress/status/`, `/metadata/`.
|
||||
- Gitea webhook: `/api/v1/system/webhook/`.
|
||||
- Public site definition: `/api/v1/publisher/sites/<site_id>/definition/`.
|
||||
|
||||
## 7) Observations / gaps for follow-up
|
||||
- `docs/user-flow/` lacks flows; backend routes above should anchor those docs.
|
||||
- Ensure billing/admin docs use the namespaces and models listed here (avoid legacy paths). Data shape examples still need to be added alongside serializers.
|
||||
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
# Planner Module (Backend) — Code-Sourced Overview (Dec 2025)
|
||||
|
||||
Scope: `backend/igny8_core/modules/planner` (ViewSets/serializers/urls) and backing models in `backend/igny8_core/business/planning`.
|
||||
|
||||
## Endpoints (routed from `modules/planner/urls.py`)
|
||||
- `/api/v1/planner/keywords/` — CRUD + bulk actions, CSV import/export, clustering trigger.
|
||||
- `/api/v1/planner/clusters/` — CRUD with aggregated stats (keywords/volume/difficulty/ideas/content).
|
||||
- `/api/v1/planner/ideas/` — CRUD for content ideas tied to clusters/keywords.
|
||||
|
||||
## Models (business/planning/models.py)
|
||||
- **Clusters** (`SiteSectorBaseModel`, soft-delete): `name`, `description`, `keywords_count`, `volume`, `mapped_pages`, `status(new|mapped)`, `disabled`; unique per `site, sector`. Indexes on `name`, `status`, `site, sector`.
|
||||
- **Keywords** (`SiteSectorBaseModel`, soft-delete): FK `seed_keyword` (global), overrides `volume_override`, `difficulty_override`, `attribute_values`; FK `cluster` (same sector); `status(new|mapped)`, `disabled`. Unique per `seed_keyword, site, sector`. Properties expose `keyword`, `volume`, `difficulty`, `intent` from seed keyword. Save enforces industry/sector alignment between site/sector and seed keyword.
|
||||
- **ContentIdeas** (`SiteSectorBaseModel`, soft-delete): `idea_title`, `description`, `target_keywords` (legacy), M2M `keyword_objects`, FK `keyword_cluster` (same sector), `status(new|queued|completed)`, `disabled`, `estimated_word_count`, `content_type` (post/page/product/taxonomy), `content_structure` (article/guide/etc.). Tracks metadata (audience, tone, outlines, brief), CTA fields, review flags (`is_priority`, `is_assigned`, `is_review_required`), language, published URL, audit fields. Indexes on `site, sector, keyword_cluster, status, idea_title`.
|
||||
|
||||
## Serializers
|
||||
- **KeywordSerializer**: read-only keyword metrics from seed keyword; requires `seed_keyword_id` on create; optional on update; validates site/sector via the ViewSet; surfaces `cluster_name`, `sector_name`, `volume/difficulty/intent`.
|
||||
- **ClusterSerializer** (`cluster_serializers.py`): computes `keywords_count`, `volume`, `difficulty`, `ideas_count`, `content_count`; bulk prefetch helper to avoid N+1. Annotated volume/difficulty via overrides or seed keyword values.
|
||||
- **ContentIdeasSerializer**: links to clusters and keywords; exposes cluster/sector names; supports site/sector write-only ids; legacy taxonomy getter retained for backward compatibility.
|
||||
|
||||
## ViewSets (modules/planner/views.py)
|
||||
- **KeywordViewSet** (`SiteSectorModelViewSet`):
|
||||
- Filtering/search/order: search by seed keyword text; filter by status, cluster, intent, seed_keyword_id; ordering by created_at, volume, difficulty.
|
||||
- Custom filters: difficulty_min/max, volume_min/max using override-first logic.
|
||||
- Bulk actions: `bulk_delete`, `bulk_update` (status), `bulk_add_from_seed` (validates industry/sector alignment, requires site/sector), `import_keywords` CSV (site/sector required, creates Keywords), `export` CSV (ids filter supported).
|
||||
- Clustering: `auto_cluster` calls `ClusteringService.cluster_keywords`; enforces min keyword count (5) and credits; returns async task_id or sync result.
|
||||
- Create requires site_id and sector_id (validated against site/sector and account).
|
||||
- **ClusterViewSet**:
|
||||
- Filters/search/order: search by name; filter by status; ordering on name/created_at/keywords_count/volume/difficulty.
|
||||
- Annotates volume/difficulty using overrides; uses serializer prefetch to reduce N+1.
|
||||
- Export endpoints not present; CRUD via standard actions.
|
||||
- **ContentIdeasViewSet**:
|
||||
- CRUD for ideas; filters on status/cluster; search by title/description/keywords; ordering by created_at/status.
|
||||
- Generates title/brief via `IdeasService.generate_content_idea`; enforces credits (catches `InsufficientCreditsError`).
|
||||
- Bulk assign status not present; focused on single-item generation and listing.
|
||||
|
||||
## Services (business/planning/services)
|
||||
- **ClusteringService**: clusters keywords; invoked by `auto_cluster`; credit-aware.
|
||||
- **IdeasService**: generates content ideas (title/brief) from keywords; used in `ContentIdeasViewSet`.
|
||||
|
||||
## Permissions, tenancy, and throttling
|
||||
- Permissions: `IsAuthenticatedAndActive` + `IsViewerOrAbove` for all planner endpoints.
|
||||
- Tenancy: `SiteSectorModelViewSet` base ensures account/site/sector scoping; create/import/add-from-seed require explicit site_id + sector_id with consistency checks.
|
||||
- Throttling: `DebugScopedRateThrottle` with scope `planner`.
|
||||
|
||||
## CSV Import/Export
|
||||
- Export: `/planner/keywords/export` supports ids filter; outputs CSV with keyword, volume, difficulty, intent, status, cluster.
|
||||
- Import: `/planner/keywords/import_keywords` expects CSV with keyword, volume, difficulty, intent, status; site_id and sector_id required in query params; skips duplicates per site/sector/account.
|
||||
|
||||
## Notes / gaps
|
||||
- Two `bulk_update` methods exist in `KeywordViewSet` (duplicate definitions); consolidate to one.
|
||||
- No dedicated bulk idea status update; consider parity with keywords.
|
||||
- Legacy taxonomy references removed; ContentIdeas retains a legacy getter for taxonomy name but model FK is removed.
|
||||
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
# Writer Module (Backend) — Code-Sourced Overview (Dec 2025)
|
||||
|
||||
Scope: `backend/igny8_core/modules/writer` (ViewSets/serializers/urls) backed by models in `backend/igny8_core/business/content`.
|
||||
|
||||
## Endpoints (from `modules/writer/urls.py`)
|
||||
- `/api/v1/writer/tasks/` — CRUD + bulk delete/update, auto content generation.
|
||||
- `/api/v1/writer/images/` — CRUD + serve file, auto generate images, bulk status update (by content or ids), grouped content images.
|
||||
- `/api/v1/writer/content/` — CRUD + publish/unpublish to WordPress, check WordPress status, AI prompt generation, validation, metadata mapping, taxonomy/tag management.
|
||||
- `/api/v1/writer/taxonomies/` — CRUD for content taxonomies (categories/tags). Attributes endpoint disabled (serializer removed).
|
||||
|
||||
## Models (business/content/models.py)
|
||||
- **Tasks** (`SiteSectorBaseModel`, soft-delete): title, description, FK `cluster` (required, same sector), optional `idea`, `content_type` (post/page/product/taxonomy), `content_structure` (article/guide/landing_page/etc.), optional `taxonomy_term`, comma keywords, word_count, status (`queued|completed`). Indexes on title/status/cluster/type/structure/site+sector.
|
||||
- **Content** (`SiteSectorBaseModel`, soft-delete): title, HTML, word_count, SEO fields, FK `cluster`, `content_type/structure`, M2M `taxonomy_terms` (through `ContentTaxonomyRelation`), external publishing fields (`external_id/url/type/metadata/sync_status`), source (`igny8|wordpress`), status (`draft|review|published`). Indexes on title/cluster/type/structure/source/status/external_id/site+sector.
|
||||
- **ContentTaxonomy** (`SiteSectorBaseModel`): name, slug, `taxonomy_type (category|tag)`, external_taxonomy/id, sync_status, description, count, metadata. Unique per site+slug+type and site+external_id+external_taxonomy.
|
||||
- **Images** (`SiteSectorBaseModel`, soft-delete): FK `content` (preferred) or `task` (legacy), `image_type (featured|desktop|mobile|in_article)`, `image_url/path`, prompt, status (`pending|generated|failed`), position. save() inherits account/site/sector from content/task. Indexes on content/task type/status/position.
|
||||
- **ContentClusterMap** (summary): links content/tasks to clusters with roles (hub/supporting/attribute) and source (blueprint/manual/import); tracks keywords + slugs.
|
||||
- **ContentTaxonomyRelation**: through table for content↔taxonomy.
|
||||
|
||||
## Serializers (modules/writer/serializers.py)
|
||||
- **TasksSerializer**: requires cluster/content_type/content_structure on create; defaults status=queued; exposes cluster_name/sector_name; accepts site_id/sector_id (write-only).
|
||||
- **ImagesSerializer**: exposes task_title/content_title; read-only account/timestamps.
|
||||
- **ContentSerializer**: requires cluster/content_type/content_structure/title on create; defaults source=igny8, status=draft; exposes taxonomy terms, tags, categories, image flags/status; site_id/sector_id write-only.
|
||||
- **ContentTaxonomySerializer** (in file but not shown above): manages taxonomy fields and external sync data.
|
||||
- Group serializers for content images (`ContentImageSerializer`, `ContentImagesGroupSerializer`).
|
||||
|
||||
## ViewSets & Actions (modules/writer/views.py)
|
||||
- **TasksViewSet** (`SiteSectorModelViewSet`):
|
||||
- Filters/search/order: search title/keywords; filter status/cluster/content_type/structure; order by title/created_at/status.
|
||||
- Bulk: `bulk_delete`, `bulk_update` (status).
|
||||
- AI: `auto_generate_content` → `ContentGenerationService.generate_content(ids, account)`; limits 10 ids; returns task_id or sync result; 402 on `InsufficientCreditsError`.
|
||||
- Create requires explicit `site_id` and `sector_id`; validates site/sector association; account resolved from request user/site.
|
||||
- **ImagesViewSet**:
|
||||
- Filters/order: filter task_id/content_id/image_type/status; order by created_at/position/id.
|
||||
- Create enforces site/sector (from request context, falling back to user active site/default sector); sets account/site/sector.
|
||||
- Actions:
|
||||
- `serve_image_file` streams local file if `image_path` exists.
|
||||
- `auto_generate_images` queues `run_ai_task(generate_images)` (Celery if available) for up to 10 task_ids.
|
||||
- `bulk_update` sets status by `content_id` or `ids`.
|
||||
- `content_images` returns grouped images (featured + in-article) via grouped serializer.
|
||||
- **ContentViewSet** (not fully shown above but key actions):
|
||||
- CRUD with filters/search/order (title, status, cluster, content_type/structure, source); tenant scoping via base class.
|
||||
- Taxonomy management: add/remove terms, sync helpers (uses `ContentTaxonomy`).
|
||||
- AI helpers: `generate_image_prompts` (queues `run_ai_task(generate_image_prompts)`), validation (`ContentValidationService`), metadata mapping (`MetadataMappingService`), content generation pathways tied to `ContentGenerationService`.
|
||||
- Publishing: `publish` queues `publish_content_to_wordpress` Celery task (optimistically sets status=published), `wordpress_status` fetches WP status via integration, `unpublish` clears external links and reverts to draft.
|
||||
- Image status helpers and grouped image retrieval also exposed.
|
||||
- **ContentTaxonomyViewSet**: CRUD for categories/tags; supports external sync fields.
|
||||
|
||||
## Permissions, tenancy, throttling
|
||||
- Permissions: `IsAuthenticatedAndActive` + `IsViewerOrAbove` across Writer; certain actions (e.g., `unpublish`) use `IsEditorOrAbove`.
|
||||
- Tenancy: `SiteSectorModelViewSet` enforces account/site/sector scoping; create operations require site_id+sector_id or context site/sector; Images save derives account/site/sector from content/task.
|
||||
- Throttling: `DebugScopedRateThrottle` with scope `writer`.
|
||||
|
||||
## AI/Background Tasks & Limits
|
||||
- Content generation: `auto_generate_content` (max 10 task IDs) → `ContentGenerationService`; credits enforced (402 on insufficient).
|
||||
- Image generation: `auto_generate_images` (max 10 task IDs) via `run_ai_task`; Celery preferred, sync fallback.
|
||||
- Image prompt generation: `generate_image_prompts` via `run_ai_task`.
|
||||
|
||||
## Publishing Integration
|
||||
- Publish: `/writer/content/{id}/publish/` → queues WordPress publish; refuses if already published.
|
||||
- Status: `/writer/content/{id}/wordpress_status/` → checks WordPress via site integration.
|
||||
- Unpublish: clears external_id/url, sets status to draft (does not delete WP post).
|
||||
|
||||
## CSV / Bulk Notes
|
||||
- Tasks: bulk delete/update supported.
|
||||
- Images: bulk status update by content or ids; grouped retrieval.
|
||||
- Content: bulk operations primarily around AI prompt/image generation and publishing; no CSV import/export.
|
||||
|
||||
## Observations / gaps
|
||||
- Tasks status choices limited to `queued/completed`; no explicit error/cancel states.
|
||||
- Images bulk_update exposed on same ViewSet name as Tasks bulk_update (distinct routes under images vs tasks); OK but keep consistent naming.
|
||||
- ContentViewSet file is large; ensure doc readers reference actions for validation/metadata mapping if extending.
|
||||
|
||||
|
||||
@@ -1,143 +0,0 @@
|
||||
# IGNY8 Billing & Account Platform — Final Plan
|
||||
**Date:** 2025-12-05
|
||||
**Status:** Canonical (supersedes all prior billing/account docs)
|
||||
**Scope:** Tenant + Admin experience across Billing, Account, Credits, Payment Methods, and Payment Gateways. Frontend + Backend must use the IGNY8 standard API system.
|
||||
|
||||
---
|
||||
|
||||
## 1) Purpose & Principles
|
||||
- Single source of truth for billing/account flows.
|
||||
- One canonical API namespace per audience (tenant vs admin).
|
||||
- Align models ↔ services ↔ serializers ↔ frontend calls; remove drift.
|
||||
- Keep manual payment path healthy while Stripe/PayPal mature.
|
||||
- Favor incremental rollout with feature-flagged transitions.
|
||||
|
||||
---
|
||||
|
||||
## 2) Canonical Namespaces
|
||||
- Tenant: `/api/v1/billing/...`
|
||||
- Admin: `/api/v1/admin/billing/...`
|
||||
- Deprecate legacy `/api/v1/admin/...` (modules/billing) and duplicated routers. Optionally dual-serve for one release behind a feature flag.
|
||||
|
||||
---
|
||||
|
||||
## 3) Reality Snapshot (current code)
|
||||
- Tenant billing viewsets live in `backend/igny8_core/business/billing/views.py` (invoices, payments, credit packages, transactions).
|
||||
- Admin billing viewset also exists there (`pending_payments`, approvals, stats) but mounted at `/api/v1/billing/admin/...`.
|
||||
- Legacy admin stack in `backend/igny8_core/modules/billing/views.py` exposes stats/users/credit-costs only; no invoices/payments.
|
||||
- Model drift: `InvoiceService` expects `currency`, `tax_amount`, `total_amount` that are absent on `Invoice`. `PaymentService` uses `pending_approval`/`completed` statuses not present in `Payment.STATUS_CHOICES`.
|
||||
- Frontend admin pages are wired but call nonexistent or legacy routes (e.g., `/v1/admin/payments/`).
|
||||
|
||||
---
|
||||
|
||||
## 4) Backend Plan
|
||||
### 4.1 Models & Migrations (billing app)
|
||||
- Invoice: add `currency`, `subtotal_amount`, `tax_amount`, `total_amount`, `line_items` (JSON list), `payment_method` (string), `stripe_invoice_id`/`paypal_invoice_id` (nullable), `metadata`. Keep existing dates/status. Add index on `status, created_at`.
|
||||
- Payment: extend `STATUS_CHOICES` to include `pending_approval`, `processing`, `completed`, `failed`, `refunded`, `cancelled`. Add `payment_method` enum (stripe, paypal, bank_transfer, local_wallet, manual), gateway intent/charge ids, `manual_reference`, `failure_reason`, `approved_by/at`, `metadata`.
|
||||
- CreditPackage: ensure `slug`, `price`, `credits`, `stripe_price_id` exist and `is_active`, `is_featured`. Add `sort_order` for admin ordering.
|
||||
- PaymentMethodConfig: keep as source for country-level enablement; ensure uniqueness `(country_code, payment_method)`.
|
||||
- Write forward/backward-safe migrations; backfill currency and totals with sensible defaults (e.g., `USD`, recompute `total_amount=subtotal_amount+tax_amount` where possible).
|
||||
|
||||
### 4.2 Services
|
||||
- `InvoiceService`: align fields; generate invoice numbers, compute subtotal/tax/total, store line items, mark paid/void/uncollectible, emit events, optionally render PDF stub.
|
||||
- `PaymentService`:
|
||||
- Stripe/PayPal: create intents/orders, store ids, update status on success/failure, attach to invoice, and trigger credit application when relevant.
|
||||
- Manual: create `pending_approval`, allow approve/reject endpoints to flip status and apply credits.
|
||||
- Sync helpers from webhooks/intents; emit structured logs.
|
||||
- `SubscriptionService` (existing): ensure plan ↔ subscription sync; use Stripe subscription id; update invoice/payment when Stripe posts events.
|
||||
- `CreditPackageService`: list active packages, create invoice + payment intent, on success add credits (`CreditTransaction`) and recalc balance.
|
||||
|
||||
### 4.3 Webhooks
|
||||
- Add `/api/v1/billing/webhooks/stripe/` and `/api/v1/billing/webhooks/paypal/` with signature verification.
|
||||
- Handle events: `invoice.paid`, `invoice.payment_failed`, `customer.subscription.updated/deleted`, `payment_intent.succeeded/failed`, PayPal equivalents.
|
||||
- Idempotent processing; log and surface dead-letter queue for failures.
|
||||
|
||||
### 4.4 APIs (Tenant)
|
||||
- `GET /api/v1/billing/account_balance/` – balance + monthly included.
|
||||
- `GET /api/v1/billing/transactions/` – credit transactions (paginated).
|
||||
- `GET /api/v1/billing/usage/` – usage logs with filters.
|
||||
- `GET /api/v1/billing/invoices/` | `GET /api/v1/billing/invoices/:id/` | `GET /api/v1/billing/invoices/:id/pdf/`.
|
||||
- `GET /api/v1/billing/payments/` | `GET /api/v1/billing/payments/:id/`.
|
||||
- `GET /api/v1/billing/credit-packages/`.
|
||||
- `POST /api/v1/billing/credits/purchase/` – create intent; returns client secret + invoice id.
|
||||
- `POST /api/v1/billing/subscriptions/create|upgrade|cancel/`.
|
||||
- `GET /api/v1/billing/subscriptions/` – current subscription status.
|
||||
- `GET/POST /api/v1/billing/payment-methods/` – list/add; store tokens/ids, not card data.
|
||||
|
||||
### 4.5 APIs (Admin)
|
||||
- `GET /api/v1/admin/billing/invoices/` | `GET /:id/` – all accounts, filters (status, account, method, date), pagination, export.
|
||||
- `GET /api/v1/admin/billing/payments/` | `GET /:id/` – all accounts, filters (status, method, gateway).
|
||||
- `POST /api/v1/admin/billing/payments/:id/approve/` | `/reject/` – manual payments only.
|
||||
- `GET/POST/PATCH/DELETE /api/v1/admin/billing/credit-packages/` – CRUD with `is_featured`, `sort_order`.
|
||||
- `GET /api/v1/admin/billing/stats/` – totals, MRR, pending approvals, failed webhooks.
|
||||
- `GET /api/v1/admin/billing/pending_payments/` – for approvals (same payload frontend expects).
|
||||
- AuthZ: developer/superuser only; enforce tenant scoping for non-admin routes.
|
||||
|
||||
### 4.6 Settings & Config
|
||||
- Add `STRIPE_PUBLIC_KEY`, `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET`, `PAYPAL_CLIENT_ID`, `PAYPAL_CLIENT_SECRET`, `PAYPAL_WEBHOOK_ID` to settings/env; feature-flag gateways.
|
||||
- Rate-limit payment endpoints; CSRF exempt webhooks only.
|
||||
|
||||
---
|
||||
|
||||
## 5) Frontend Plan
|
||||
### 5.1 Service Layer (`frontend/src/services/billing.api.ts`)
|
||||
- Point all admin calls to `/api/v1/admin/billing/...`; remove dead `/v1/admin/...` routes.
|
||||
- Reuse tenant calls for user-facing pages; ensure response shapes match new serializers (`InvoiceSerializer`, `PaymentSerializer`, `CreditPackageSerializer`).
|
||||
- Add helpers for manual approval actions and credit-package CRUD.
|
||||
|
||||
### 5.2 Tenant Pages
|
||||
- Plans & Billing (`/account/billing`): tabs for Plan, Credits, Billing History (invoices), Payment Methods. Uses balance, subscription, invoices, payment-methods, credit-packages, purchase intent.
|
||||
- Purchase Credits (`/account/credits/purchase`): list packages → create purchase intent → confirm via Stripe/PayPal/manual upload → poll payment.
|
||||
- Invoices (`/account/invoices`): paginated table, filters, PDF download.
|
||||
- Payment Methods: list/add/remove default (via tokens).
|
||||
|
||||
### 5.3 Admin Pages
|
||||
- Billing Overview (`/admin/billing`): stats from `/admin/billing/stats/`, pending approvals summary.
|
||||
- Invoices (`/admin/invoices`): uses `/admin/billing/invoices/`; add filters (account, status, method, date) and CSV/PDF export.
|
||||
- Payments (`/admin/payments`): uses `/admin/billing/payments/`; add filters and status badges.
|
||||
- Credit Packages (`/admin/credit-packages`): CRUD via admin endpoints; show `is_active`, `is_featured`, `sort_order`, Stripe price id.
|
||||
- Payment Approvals (`/admin/payments/approvals`): call `pending_payments`, approve/reject endpoints; show invoice link and manual references.
|
||||
- Sidebar links already present—only fix data wiring.
|
||||
|
||||
### 5.4 UX/Consistency
|
||||
- Standard empty/error/loading states across billing pages.
|
||||
- Currency display from invoice/payment `currency`.
|
||||
- Role-guard tenant routes (owner/admin only for purchases; read-only for others where applicable).
|
||||
|
||||
---
|
||||
|
||||
## 6) Sequenced Action Plan
|
||||
- **Day 0 (alignment):** Merge this doc; add “retired” note to older docs; choose namespace flag (`USE_NEW_ADMIN_BILLING_API=true`).
|
||||
- **Phase 1 (backend, 2-3 days):** Migrate models; update services; add admin list endpoints; webhook stubs; tests for serializers and viewsets.
|
||||
- **Phase 2 (gateways + frontend wiring, 3-4 days):** Implement Stripe/PayPal intent paths, hook webhooks, wire `billing.api.ts` to new endpoints, fix admin pages, enable manual approvals end-to-end.
|
||||
- **Phase 3 (polish, 2 days):** Add exports, filters, pagination polish; observability (structured logs, alerts); finalize docs and remove legacy router.
|
||||
|
||||
---
|
||||
|
||||
## 7) Deliverables & DoD
|
||||
- Schema migration applied and reversible; services match models.
|
||||
- Canonical endpoints live and documented; legacy admin endpoints removed/flagged off.
|
||||
- Tenant billing pages load without 404s; purchases and subscriptions succeed (manual + Stripe at minimum).
|
||||
- Admin pages show data, support approvals, and package CRUD.
|
||||
- Tests: unit (services), API (tenant/admin billing), integration (purchase + manual approval flow), webhook idempotency.
|
||||
- Docs: this file referenced from legacy docs; endpoint matrix embedded in `billing.api.ts` JSDoc or a short `/docs/billing/api-map.md`.
|
||||
|
||||
---
|
||||
|
||||
## 8) Quick Endpoint Matrix (canonical)
|
||||
- Tenant: invoices (`/api/v1/billing/invoices`), payments (`/api/v1/billing/payments`), credit packages + purchase (`/api/v1/billing/credit-packages`, `/credits/purchase`), subscription create/upgrade/cancel (`/subscriptions/...`), payment methods (`/payment-methods`), usage/transactions/balance.
|
||||
- Admin: invoices (`/api/v1/admin/billing/invoices`), payments (`/api/v1/admin/billing/payments`), pending approvals (`/api/v1/admin/billing/pending_payments`), approve/reject (`/api/v1/admin/billing/payments/:id/approve|reject`), credit packages CRUD (`/api/v1/admin/billing/credit-packages`), stats (`/api/v1/admin/billing/stats`).
|
||||
|
||||
---
|
||||
|
||||
## 9) Rollout & Migration Notes
|
||||
- Run migrations during maintenance window; backfill currency/totals.
|
||||
- If dual-stack needed, proxy legacy admin routes to new handlers temporarily.
|
||||
- Communicate new admin endpoints to frontend before flipping flags.
|
||||
- Validate webhook secrets in non-prod before enabling in prod.
|
||||
|
||||
---
|
||||
|
||||
## 10) Retirement Notice
|
||||
This document supersedes `docs/working-docs/Original-plan-may-have-wrong-logic.md` and `docs/working-docs/admin-billing-plan-2025-12-05.md`. Treat those as retired; keep only for historical reference.
|
||||
|
||||
@@ -1,858 +0,0 @@
|
||||
# Credits System - Complete Audit and Improvement Plan
|
||||
|
||||
**Date:** December 4, 2025
|
||||
**Status:** Audit Complete - Awaiting Implementation
|
||||
**Priority:** HIGH
|
||||
|
||||
---
|
||||
|
||||
## 📋 EXECUTIVE SUMMARY
|
||||
|
||||
This document provides a comprehensive audit of the IGNY8 credits system, identifies gaps and potential issues, and proposes improvements including backend admin configuration for credit costs per function.
|
||||
|
||||
**Current State:** ✅ Working
|
||||
**Areas for Improvement:** Configuration Management, Billing Integration, Admin UI, Reporting
|
||||
|
||||
---
|
||||
|
||||
## 🔍 SYSTEM ARCHITECTURE AUDIT
|
||||
|
||||
### Current Credit System Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ CREDITS SYSTEM │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 1. Account Model (credits field) │
|
||||
│ 2. CreditTransaction Model (history) │
|
||||
│ 3. CreditUsageLog Model (detailed usage) │
|
||||
│ 4. CreditService (business logic) │
|
||||
│ 5. CREDIT_COSTS (hardcoded constants) │
|
||||
│ 6. Credit API Endpoints │
|
||||
│ 7. Frontend Dashboard │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ DATABASE MODELS AUDIT
|
||||
|
||||
### 1. Account Model (`auth.Account`)
|
||||
|
||||
**Location:** `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Credits Field:**
|
||||
```python
|
||||
credits = models.IntegerField(default=0, help_text="Current credit balance")
|
||||
```
|
||||
|
||||
**Status:** ✅ Working
|
||||
**Findings:**
|
||||
- Simple integer field for balance
|
||||
- No soft delete or archive mechanism
|
||||
- No credit expiration tracking
|
||||
- No overdraft protection
|
||||
|
||||
**Recommendations:**
|
||||
- ✅ Keep simple design (no changes needed)
|
||||
- Add `credits_expires_at` field for subscription credits
|
||||
- Add `bonus_credits` field separate from subscription credits
|
||||
|
||||
---
|
||||
|
||||
### 2. CreditTransaction Model
|
||||
|
||||
**Location:** `backend/igny8_core/business/billing/models.py`
|
||||
|
||||
**Current Structure:**
|
||||
```python
|
||||
class CreditTransaction(AccountBaseModel):
|
||||
TRANSACTION_TYPE_CHOICES = [
|
||||
('purchase', 'Purchase'),
|
||||
('subscription', 'Subscription Renewal'),
|
||||
('refund', 'Refund'),
|
||||
('deduction', 'Usage Deduction'),
|
||||
('adjustment', 'Manual Adjustment'),
|
||||
]
|
||||
|
||||
transaction_type = models.CharField(max_length=20, choices=TRANSACTION_TYPE_CHOICES)
|
||||
amount = models.IntegerField() # + for add, - for deduct
|
||||
balance_after = models.IntegerField()
|
||||
description = models.CharField(max_length=255)
|
||||
metadata = models.JSONField(default=dict)
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
```
|
||||
|
||||
**Status:** ✅ Working
|
||||
**Findings:**
|
||||
- Comprehensive transaction types
|
||||
- Good metadata for context
|
||||
- Proper indexing on account/type/date
|
||||
- Missing: invoice_id FK, payment_method
|
||||
|
||||
**Recommendations:**
|
||||
- Add `invoice_id` FK (for billing integration)
|
||||
- Add `payment_method` field ('stripe', 'manual', 'free')
|
||||
- Add `status` field ('pending', 'completed', 'failed', 'refunded')
|
||||
- Add `external_transaction_id` for Stripe payment IDs
|
||||
|
||||
---
|
||||
|
||||
### 3. CreditUsageLog Model
|
||||
|
||||
**Location:** `backend/igny8_core/business/billing/models.py`
|
||||
|
||||
**Current Structure:**
|
||||
```python
|
||||
class CreditUsageLog(AccountBaseModel):
|
||||
OPERATION_TYPE_CHOICES = [
|
||||
('clustering', 'Clustering'),
|
||||
('idea_generation', 'Idea Generation'),
|
||||
('content_generation', 'Content Generation'),
|
||||
('image_prompt_extraction', 'Image Prompt Extraction'),
|
||||
('image_generation', 'Image Generation'),
|
||||
('linking', 'Content Linking'),
|
||||
('optimization', 'Content Optimization'),
|
||||
('site_structure_generation', 'Site Structure Generation'),
|
||||
('site_page_generation', 'Site Page Generation'),
|
||||
]
|
||||
|
||||
operation_type = models.CharField(max_length=50, choices=OPERATION_TYPE_CHOICES)
|
||||
credits_used = models.IntegerField()
|
||||
cost_usd = models.DecimalField(max_digits=10, decimal_places=2, null=True)
|
||||
model_used = models.CharField(max_length=100, blank=True)
|
||||
tokens_input = models.IntegerField(null=True)
|
||||
tokens_output = models.IntegerField(null=True)
|
||||
related_object_type = models.CharField(max_length=50, blank=True)
|
||||
related_object_id = models.IntegerField(null=True)
|
||||
metadata = models.JSONField(default=dict)
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
```
|
||||
|
||||
**Status:** ✅ Working
|
||||
**Findings:**
|
||||
- Excellent detail tracking (model, tokens, cost)
|
||||
- Good related object tracking
|
||||
- Proper operation type choices
|
||||
- Missing: site/sector isolation, duration tracking
|
||||
|
||||
**Recommendations:**
|
||||
- Add `site` FK for multi-tenant filtering
|
||||
- Add `sector` FK for isolation
|
||||
- Add `duration_seconds` field (API call time)
|
||||
- Add `success` boolean field (track failures)
|
||||
- Add `error_message` field for failed operations
|
||||
|
||||
---
|
||||
|
||||
## 💳 CREDIT COST CONFIGURATION AUDIT
|
||||
|
||||
### Current Implementation: Hardcoded Constants
|
||||
|
||||
**Location:** `backend/igny8_core/business/billing/constants.py`
|
||||
|
||||
```python
|
||||
CREDIT_COSTS = {
|
||||
'clustering': 10, # Per clustering request
|
||||
'idea_generation': 15, # Per cluster → ideas request
|
||||
'content_generation': 1, # Per 100 words
|
||||
'image_prompt_extraction': 2, # Per content piece
|
||||
'image_generation': 5, # Per image
|
||||
'linking': 8, # Per content piece
|
||||
'optimization': 1, # Per 200 words
|
||||
'site_structure_generation': 50, # Per site blueprint
|
||||
'site_page_generation': 20, # Per page
|
||||
}
|
||||
```
|
||||
|
||||
**Status:** ⚠️ Working but NOT configurable
|
||||
**Problems:**
|
||||
1. ❌ **Hardcoded values** - Requires code deployment to change
|
||||
2. ❌ **No admin UI** - Cannot adjust costs without developer
|
||||
3. ❌ **No versioning** - Cannot track cost changes over time
|
||||
4. ❌ **No A/B testing** - Cannot test different pricing
|
||||
5. ❌ **No per-account pricing** - All accounts same cost
|
||||
6. ❌ **No promotional pricing** - Cannot offer discounts
|
||||
|
||||
---
|
||||
|
||||
## 💰 BILLING & INVOICING GAPS
|
||||
|
||||
### Current State
|
||||
|
||||
**✅ Working:**
|
||||
- Credit deduction on AI operations
|
||||
- Credit transaction logging
|
||||
- Credit balance API
|
||||
- Credit usage API
|
||||
- Monthly credit replenishment (Celery task)
|
||||
|
||||
**❌ Missing (NOT Implemented):**
|
||||
1. **Invoice Generation** - No Invoice model or PDF generation
|
||||
2. **Payment Processing** - No Stripe/PayPal integration
|
||||
3. **Subscription Management** - No recurring billing
|
||||
4. **Purchase Credits** - No one-time credit purchase flow
|
||||
5. **Refund Processing** - No refund workflow
|
||||
6. **Payment History** - No payment records
|
||||
7. **Tax Calculation** - No tax/VAT handling
|
||||
8. **Billing Address** - No billing info storage
|
||||
|
||||
---
|
||||
|
||||
## 🎯 PROPOSED SOLUTION: Backend Admin Configuration
|
||||
|
||||
### New Model: CreditCostConfig
|
||||
|
||||
**Purpose:** Make credit costs configurable from Django Admin
|
||||
|
||||
**Location:** `backend/igny8_core/modules/billing/models.py`
|
||||
|
||||
```python
|
||||
class CreditCostConfig(models.Model):
|
||||
"""
|
||||
Configurable credit costs per AI function
|
||||
Admin-editable alternative to hardcoded constants
|
||||
"""
|
||||
# Operation identification
|
||||
operation_type = models.CharField(
|
||||
max_length=50,
|
||||
unique=True,
|
||||
choices=CreditUsageLog.OPERATION_TYPE_CHOICES,
|
||||
help_text="AI operation type"
|
||||
)
|
||||
|
||||
# Cost configuration
|
||||
credits_cost = models.IntegerField(
|
||||
validators=[MinValueValidator(0)],
|
||||
help_text="Credits required for this operation"
|
||||
)
|
||||
|
||||
# Unit of measurement
|
||||
unit = models.CharField(
|
||||
max_length=50,
|
||||
default='per_request',
|
||||
choices=[
|
||||
('per_request', 'Per Request'),
|
||||
('per_100_words', 'Per 100 Words'),
|
||||
('per_200_words', 'Per 200 Words'),
|
||||
('per_item', 'Per Item'),
|
||||
('per_image', 'Per Image'),
|
||||
],
|
||||
help_text="What the cost applies to"
|
||||
)
|
||||
|
||||
# Metadata
|
||||
display_name = models.CharField(max_length=100, help_text="Human-readable name")
|
||||
description = models.TextField(blank=True, help_text="What this operation does")
|
||||
|
||||
# Status
|
||||
is_active = models.BooleanField(default=True, help_text="Enable/disable this operation")
|
||||
|
||||
# Audit fields
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
updated_at = models.DateTimeField(auto_now=True)
|
||||
updated_by = models.ForeignKey(
|
||||
'auth.User',
|
||||
null=True,
|
||||
blank=True,
|
||||
on_delete=models.SET_NULL,
|
||||
help_text="Admin who last updated"
|
||||
)
|
||||
|
||||
# Change tracking
|
||||
previous_cost = models.IntegerField(
|
||||
null=True,
|
||||
blank=True,
|
||||
help_text="Cost before last update (for audit trail)"
|
||||
)
|
||||
|
||||
class Meta:
|
||||
db_table = 'igny8_credit_cost_config'
|
||||
verbose_name = 'Credit Cost Configuration'
|
||||
verbose_name_plural = 'Credit Cost Configurations'
|
||||
ordering = ['operation_type']
|
||||
|
||||
def __str__(self):
|
||||
return f"{self.display_name} - {self.credits_cost} credits {self.unit}"
|
||||
|
||||
def save(self, *args, **kwargs):
|
||||
# Track cost changes
|
||||
if self.pk:
|
||||
old = CreditCostConfig.objects.get(pk=self.pk)
|
||||
if old.credits_cost != self.credits_cost:
|
||||
self.previous_cost = old.credits_cost
|
||||
super().save(*args, **kwargs)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Django Admin Configuration
|
||||
|
||||
**Location:** `backend/igny8_core/modules/billing/admin.py`
|
||||
|
||||
```python
|
||||
from django.contrib import admin
|
||||
from django.utils.html import format_html
|
||||
from .models import CreditCostConfig
|
||||
|
||||
@admin.register(CreditCostConfig)
|
||||
class CreditCostConfigAdmin(admin.ModelAdmin):
|
||||
list_display = [
|
||||
'operation_type',
|
||||
'display_name',
|
||||
'credits_cost_display',
|
||||
'unit',
|
||||
'is_active',
|
||||
'cost_change_indicator',
|
||||
'updated_at',
|
||||
'updated_by'
|
||||
]
|
||||
|
||||
list_filter = ['is_active', 'unit', 'updated_at']
|
||||
search_fields = ['operation_type', 'display_name', 'description']
|
||||
|
||||
fieldsets = (
|
||||
('Operation', {
|
||||
'fields': ('operation_type', 'display_name', 'description')
|
||||
}),
|
||||
('Cost Configuration', {
|
||||
'fields': ('credits_cost', 'unit', 'is_active')
|
||||
}),
|
||||
('Audit Trail', {
|
||||
'fields': ('previous_cost', 'updated_by', 'created_at', 'updated_at'),
|
||||
'classes': ('collapse',)
|
||||
}),
|
||||
)
|
||||
|
||||
readonly_fields = ['created_at', 'updated_at', 'previous_cost']
|
||||
|
||||
def credits_cost_display(self, obj):
|
||||
"""Show cost with color coding"""
|
||||
if obj.credits_cost >= 20:
|
||||
color = 'red'
|
||||
elif obj.credits_cost >= 10:
|
||||
color = 'orange'
|
||||
else:
|
||||
color = 'green'
|
||||
return format_html(
|
||||
'<span style="color: {}; font-weight: bold;">{} credits</span>',
|
||||
color,
|
||||
obj.credits_cost
|
||||
)
|
||||
credits_cost_display.short_description = 'Cost'
|
||||
|
||||
def cost_change_indicator(self, obj):
|
||||
"""Show if cost changed recently"""
|
||||
if obj.previous_cost is not None:
|
||||
if obj.credits_cost > obj.previous_cost:
|
||||
icon = '📈' # Increased
|
||||
color = 'red'
|
||||
elif obj.credits_cost < obj.previous_cost:
|
||||
icon = '📉' # Decreased
|
||||
color = 'green'
|
||||
else:
|
||||
icon = '➡️' # Same
|
||||
color = 'gray'
|
||||
|
||||
return format_html(
|
||||
'{} <span style="color: {};">({} → {})</span>',
|
||||
icon,
|
||||
color,
|
||||
obj.previous_cost,
|
||||
obj.credits_cost
|
||||
)
|
||||
return '—'
|
||||
cost_change_indicator.short_description = 'Recent Change'
|
||||
|
||||
def save_model(self, request, obj, form, change):
|
||||
"""Track who made the change"""
|
||||
obj.updated_by = request.user
|
||||
super().save_model(request, obj, form, change)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Updated CreditService to Use Database
|
||||
|
||||
**Location:** `backend/igny8_core/business/billing/services/credit_service.py`
|
||||
|
||||
```python
|
||||
class CreditService:
|
||||
"""Service for managing credits"""
|
||||
|
||||
@staticmethod
|
||||
def get_credit_cost(operation_type, amount=None):
|
||||
"""
|
||||
Get credit cost for an operation.
|
||||
Now checks database config first, falls back to constants.
|
||||
|
||||
Args:
|
||||
operation_type: Type of operation
|
||||
amount: Optional amount (word count, image count, etc.)
|
||||
|
||||
Returns:
|
||||
int: Credit cost
|
||||
"""
|
||||
# Try to get from database config first
|
||||
try:
|
||||
from igny8_core.modules.billing.models import CreditCostConfig
|
||||
|
||||
config = CreditCostConfig.objects.filter(
|
||||
operation_type=operation_type,
|
||||
is_active=True
|
||||
).first()
|
||||
|
||||
if config:
|
||||
base_cost = config.credits_cost
|
||||
|
||||
# Apply unit-based calculation
|
||||
if config.unit == 'per_100_words' and amount:
|
||||
return max(1, (amount // 100)) * base_cost
|
||||
elif config.unit == 'per_200_words' and amount:
|
||||
return max(1, (amount // 200)) * base_cost
|
||||
elif config.unit in ['per_item', 'per_image'] and amount:
|
||||
return amount * base_cost
|
||||
else:
|
||||
return base_cost
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get cost from database, using constants: {e}")
|
||||
|
||||
# Fallback to hardcoded constants
|
||||
from igny8_core.business.billing.constants import CREDIT_COSTS
|
||||
|
||||
base_cost = CREDIT_COSTS.get(operation_type, 1)
|
||||
|
||||
# Apply multipliers for word-based operations
|
||||
if operation_type == 'content_generation' and amount:
|
||||
return max(1, (amount // 100)) # 1 credit per 100 words
|
||||
elif operation_type == 'optimization' and amount:
|
||||
return max(1, (amount // 200)) # 1 credit per 200 words
|
||||
elif operation_type in ['image_generation'] and amount:
|
||||
return amount * base_cost
|
||||
else:
|
||||
return base_cost
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 ADMIN UI - CREDIT COST CONFIGURATION
|
||||
|
||||
### Management Command: Initialize Credit Costs
|
||||
|
||||
**Location:** `backend/igny8_core/modules/billing/management/commands/init_credit_costs.py`
|
||||
|
||||
```python
|
||||
from django.core.management.base import BaseCommand
|
||||
from igny8_core.modules.billing.models import CreditCostConfig
|
||||
from igny8_core.business.billing.constants import CREDIT_COSTS
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = 'Initialize credit cost configurations from constants'
|
||||
|
||||
def handle(self, *args, **options):
|
||||
"""Migrate hardcoded costs to database"""
|
||||
|
||||
operation_metadata = {
|
||||
'clustering': {
|
||||
'display_name': 'Auto Clustering',
|
||||
'description': 'Group keywords into semantic clusters using AI',
|
||||
'unit': 'per_request'
|
||||
},
|
||||
'idea_generation': {
|
||||
'display_name': 'Idea Generation',
|
||||
'description': 'Generate content ideas from keyword clusters',
|
||||
'unit': 'per_request'
|
||||
},
|
||||
'content_generation': {
|
||||
'display_name': 'Content Generation',
|
||||
'description': 'Generate article content using AI',
|
||||
'unit': 'per_100_words'
|
||||
},
|
||||
'image_prompt_extraction': {
|
||||
'display_name': 'Image Prompt Extraction',
|
||||
'description': 'Extract image prompts from content',
|
||||
'unit': 'per_request'
|
||||
},
|
||||
'image_generation': {
|
||||
'display_name': 'Image Generation',
|
||||
'description': 'Generate images using AI (DALL-E, Runware)',
|
||||
'unit': 'per_image'
|
||||
},
|
||||
'linking': {
|
||||
'display_name': 'Content Linking',
|
||||
'description': 'Generate internal links between content',
|
||||
'unit': 'per_request'
|
||||
},
|
||||
'optimization': {
|
||||
'display_name': 'Content Optimization',
|
||||
'description': 'Optimize content for SEO',
|
||||
'unit': 'per_200_words'
|
||||
},
|
||||
'site_structure_generation': {
|
||||
'display_name': 'Site Structure Generation',
|
||||
'description': 'Generate complete site blueprint',
|
||||
'unit': 'per_request'
|
||||
},
|
||||
'site_page_generation': {
|
||||
'display_name': 'Site Page Generation',
|
||||
'description': 'Generate site pages from blueprint',
|
||||
'unit': 'per_item'
|
||||
},
|
||||
}
|
||||
|
||||
created_count = 0
|
||||
updated_count = 0
|
||||
|
||||
for operation_type, cost in CREDIT_COSTS.items():
|
||||
# Skip legacy aliases
|
||||
if operation_type in ['ideas', 'content', 'images', 'reparse']:
|
||||
continue
|
||||
|
||||
metadata = operation_metadata.get(operation_type, {})
|
||||
|
||||
config, created = CreditCostConfig.objects.get_or_create(
|
||||
operation_type=operation_type,
|
||||
defaults={
|
||||
'credits_cost': cost,
|
||||
'display_name': metadata.get('display_name', operation_type.replace('_', ' ').title()),
|
||||
'description': metadata.get('description', ''),
|
||||
'unit': metadata.get('unit', 'per_request'),
|
||||
'is_active': True
|
||||
}
|
||||
)
|
||||
|
||||
if created:
|
||||
created_count += 1
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(f'✅ Created: {config.display_name} - {cost} credits')
|
||||
)
|
||||
else:
|
||||
updated_count += 1
|
||||
self.stdout.write(
|
||||
self.style.WARNING(f'⚠️ Already exists: {config.display_name}')
|
||||
)
|
||||
|
||||
self.stdout.write(
|
||||
self.style.SUCCESS(f'\n✅ Complete: {created_count} created, {updated_count} already existed')
|
||||
)
|
||||
```
|
||||
|
||||
**Run command:**
|
||||
```bash
|
||||
python manage.py init_credit_costs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 POTENTIAL ISSUES & FIXES
|
||||
|
||||
### Issue 1: Race Conditions in Credit Deduction
|
||||
|
||||
**Problem:**
|
||||
```python
|
||||
# Current code (simplified)
|
||||
if account.credits < required:
|
||||
raise InsufficientCreditsError()
|
||||
account.credits -= required # Race condition here!
|
||||
account.save()
|
||||
```
|
||||
|
||||
**Risk:** Two requests could both check balance and deduct simultaneously
|
||||
|
||||
**Fix:** Use database-level atomic update
|
||||
```python
|
||||
from django.db.models import F
|
||||
|
||||
@transaction.atomic
|
||||
def deduct_credits(account, amount):
|
||||
# Atomic update with check
|
||||
updated = Account.objects.filter(
|
||||
id=account.id,
|
||||
credits__gte=amount # Check in database
|
||||
).update(
|
||||
credits=F('credits') - amount
|
||||
)
|
||||
|
||||
if updated == 0:
|
||||
raise InsufficientCreditsError()
|
||||
|
||||
account.refresh_from_db()
|
||||
return account.credits
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue 2: Negative Credit Balance
|
||||
|
||||
**Problem:** No hard constraint preventing negative credits
|
||||
|
||||
**Fix 1:** Database constraint
|
||||
```python
|
||||
# Migration
|
||||
operations = [
|
||||
migrations.AddConstraint(
|
||||
model_name='account',
|
||||
constraint=models.CheckConstraint(
|
||||
check=models.Q(credits__gte=0),
|
||||
name='credits_non_negative'
|
||||
),
|
||||
),
|
||||
]
|
||||
```
|
||||
|
||||
**Fix 2:** Service layer validation (current approach - OK)
|
||||
|
||||
---
|
||||
|
||||
### Issue 3: Missing Credit Expiration
|
||||
|
||||
**Problem:** Subscription credits never expire
|
||||
|
||||
**Fix:** Add expiration tracking
|
||||
```python
|
||||
# Account model
|
||||
credits_expires_at = models.DateTimeField(null=True, blank=True)
|
||||
|
||||
# Celery task (daily)
|
||||
@shared_task
|
||||
def expire_credits():
|
||||
"""Expire old credits"""
|
||||
expired = Account.objects.filter(
|
||||
credits_expires_at__lt=timezone.now(),
|
||||
credits__gt=0
|
||||
)
|
||||
|
||||
for account in expired:
|
||||
# Transfer to expired_credits field or log
|
||||
account.credits = 0
|
||||
account.save()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue 4: No Usage Analytics
|
||||
|
||||
**Problem:** Hard to analyze which functions cost most credits
|
||||
|
||||
**Fix:** Add aggregation views
|
||||
```python
|
||||
# Backend
|
||||
@action(detail=False, methods=['get'])
|
||||
def cost_breakdown(self, request):
|
||||
"""Get credit cost breakdown by operation"""
|
||||
from django.db.models import Sum
|
||||
|
||||
breakdown = CreditUsageLog.objects.filter(
|
||||
account=request.account
|
||||
).values('operation_type').annotate(
|
||||
total_credits=Sum('credits_used'),
|
||||
count=Count('id')
|
||||
).order_by('-total_credits')
|
||||
|
||||
return Response(breakdown)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Issue 5: No Budget Alerts
|
||||
|
||||
**Problem:** Users can run out of credits unexpectedly
|
||||
|
||||
**Fix:** Add threshold alerts
|
||||
```python
|
||||
# After each deduction
|
||||
def check_low_balance(account):
|
||||
if account.credits < 100: # Configurable threshold
|
||||
send_low_balance_email(account)
|
||||
|
||||
if account.credits < 50:
|
||||
send_critical_balance_email(account)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📈 FUTURE ENHANCEMENTS
|
||||
|
||||
### Phase 1: Billing Integration (Priority: HIGH)
|
||||
|
||||
**Models to Add:**
|
||||
1. `Invoice` - Store invoice records
|
||||
2. `Payment` - Track payments
|
||||
3. `Subscription` - Recurring billing
|
||||
4. `CreditPackage` - One-time credit purchases
|
||||
|
||||
**Integrations:**
|
||||
- Stripe for payments
|
||||
- PDF generation for invoices
|
||||
- Email notifications
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Advanced Pricing (Priority: MEDIUM)
|
||||
|
||||
**Features:**
|
||||
1. **Volume Discounts**
|
||||
- 1000+ credits/month: 10% discount
|
||||
- 5000+ credits/month: 20% discount
|
||||
|
||||
2. **Per-Account Pricing**
|
||||
- Enterprise accounts: Custom pricing
|
||||
- Trial accounts: Limited operations
|
||||
|
||||
3. **Promotional Codes**
|
||||
- Discount codes
|
||||
- Free credit grants
|
||||
|
||||
4. **Credit Bundles**
|
||||
- Starter: 500 credits
|
||||
- Pro: 2000 credits
|
||||
- Enterprise: 10000 credits
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Usage Analytics Dashboard (Priority: MEDIUM)
|
||||
|
||||
**Features:**
|
||||
1. **Cost Breakdown Charts**
|
||||
- By operation type
|
||||
- By time period
|
||||
- By site/sector
|
||||
|
||||
2. **Trend Analysis**
|
||||
- Daily/weekly/monthly usage
|
||||
- Forecasting
|
||||
- Budget alerts
|
||||
|
||||
3. **Comparison Reports**
|
||||
- Compare accounts
|
||||
- Compare time periods
|
||||
- Benchmark against averages
|
||||
|
||||
---
|
||||
|
||||
## 🧪 TESTING CHECKLIST
|
||||
|
||||
### Unit Tests Required
|
||||
|
||||
- [ ] Test CreditCostConfig model creation
|
||||
- [ ] Test CreditService with database config
|
||||
- [ ] Test fallback to constants
|
||||
- [ ] Test atomic credit deduction
|
||||
- [ ] Test negative balance prevention
|
||||
- [ ] Test cost calculation with units
|
||||
|
||||
### Integration Tests Required
|
||||
|
||||
- [ ] Test full credit deduction flow
|
||||
- [ ] Test monthly replenishment
|
||||
- [ ] Test admin UI operations
|
||||
- [ ] Test concurrent deductions (race conditions)
|
||||
- [ ] Test cost changes propagate correctly
|
||||
|
||||
### Manual Testing Required
|
||||
|
||||
- [ ] Create credit config in Django Admin
|
||||
- [ ] Update cost and verify in logs
|
||||
- [ ] Deactivate operation and verify rejection
|
||||
- [ ] Test with different units (per 100 words, per image)
|
||||
- [ ] Verify audit trail (previous_cost, updated_by)
|
||||
|
||||
---
|
||||
|
||||
## 📋 IMPLEMENTATION ROADMAP
|
||||
|
||||
### Week 1: Database Configuration
|
||||
- [ ] Create `CreditCostConfig` model
|
||||
- [ ] Create migration
|
||||
- [ ] Create Django Admin
|
||||
- [ ] Create init_credit_costs command
|
||||
- [ ] Update CreditService to use database
|
||||
|
||||
### Week 2: Testing & Refinement
|
||||
- [ ] Write unit tests
|
||||
- [ ] Write integration tests
|
||||
- [ ] Manual QA testing
|
||||
- [ ] Fix race conditions
|
||||
- [ ] Add constraints
|
||||
|
||||
### Week 3: Documentation & Deployment
|
||||
- [ ] Update API documentation
|
||||
- [ ] Create admin user guide
|
||||
- [ ] Deploy to staging
|
||||
- [ ] User acceptance testing
|
||||
- [ ] Deploy to production
|
||||
|
||||
### Week 4: Monitoring & Optimization
|
||||
- [ ] Monitor cost changes
|
||||
- [ ] Analyze usage patterns
|
||||
- [ ] Optimize slow queries
|
||||
- [ ] Plan Phase 2 features
|
||||
|
||||
---
|
||||
|
||||
## 🎯 SUCCESS CRITERIA
|
||||
|
||||
✅ **Backend Admin:** Credits configurable via Django Admin
|
||||
✅ **No Code Deploys:** Cost changes don't require deployment
|
||||
✅ **Audit Trail:** Track who changed costs and when
|
||||
✅ **Backward Compatible:** Existing code continues to work
|
||||
✅ **Performance:** No regression in credit deduction speed
|
||||
✅ **Data Integrity:** No race conditions or negative balances
|
||||
✅ **Testing:** 100% test coverage for critical paths
|
||||
|
||||
---
|
||||
|
||||
## 📊 CREDITS SYSTEM FLOWCHART
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[AI Operation Request] --> B{Check CreditCostConfig}
|
||||
B -->|Found| C[Get Cost from Database]
|
||||
B -->|Not Found| D[Get Cost from Constants]
|
||||
C --> E[Calculate Total Cost]
|
||||
D --> E
|
||||
E --> F{Sufficient Credits?}
|
||||
F -->|Yes| G[Atomic Deduct Credits]
|
||||
F -->|No| H[Raise InsufficientCreditsError]
|
||||
G --> I[Create CreditTransaction]
|
||||
I --> J[Create CreditUsageLog]
|
||||
J --> K[Return Success]
|
||||
H --> L[Return Error]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 SECURITY CONSIDERATIONS
|
||||
|
||||
### Credit Manipulation Prevention
|
||||
|
||||
1. **No Client-Side Credit Calculation**
|
||||
- All calculations server-side
|
||||
- Credits never exposed in API responses
|
||||
|
||||
2. **Atomic Transactions**
|
||||
- Use database transactions
|
||||
- Prevent race conditions
|
||||
|
||||
3. **Audit Logging**
|
||||
- Log all credit changes
|
||||
- Track who/when/why
|
||||
|
||||
4. **Rate Limiting**
|
||||
- Prevent credit abuse
|
||||
- Throttle expensive operations
|
||||
|
||||
5. **Admin Permissions**
|
||||
- Only superusers can modify costs
|
||||
- Track all admin changes
|
||||
|
||||
---
|
||||
|
||||
## END OF AUDIT
|
||||
|
||||
This comprehensive audit identifies all aspects of the credits system, proposes a database-driven configuration approach, and provides a clear roadmap for implementation. The system is currently working well but lacks flexibility for cost adjustments without code deployments.
|
||||
|
||||
**Recommendation:** Implement the CreditCostConfig model in Phase 1 to enable admin-configurable costs.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,59 +0,0 @@
|
||||
# IGNY8 App Architecture (code-sourced, Dec 2025)
|
||||
|
||||
Authoritative snapshot derived from the current codebase (backend Django/DRF, frontend React/Vite). No legacy docs referenced.
|
||||
|
||||
## 1) Platform Overview
|
||||
- **Multi-tenant:** All tenant-scoped models inherit `AccountBaseModel`; site/sector models inherit `SiteSectorBaseModel` (`backend/igny8_core/auth/models.py`).
|
||||
- **APIs (wired in `backend/igny8_core/urls.py`):** `/api/v1/auth/`, `/api/v1/account/`, `/api/v1/planner/`, `/api/v1/writer/`, `/api/v1/system/`, `/api/v1/billing/` (tenant billing), `/api/v1/admin/` (billing admin + credit costs), `/api/v1/automation/`, `/api/v1/linker/`, `/api/v1/optimizer/`, `/api/v1/publisher/`, `/api/v1/integration/`.
|
||||
- **Frontend routes (in `frontend/src/App.tsx`):** Planner, Writer, Automation, Linker, Optimizer, Thinker, Billing, Account, Admin sections; sidebar defined in `frontend/src/layout/AppSidebar.tsx`.
|
||||
|
||||
## 2) Backend Modules & Endpoints
|
||||
- **Planner** (`modules/planner/urls.py`): `/keywords`, `/clusters`, `/ideas` via ViewSets.
|
||||
- **Writer** (`modules/writer/urls.py`): `/tasks`, `/images`, `/content`, `/taxonomies`.
|
||||
- **Automation** (`business/automation/urls.py`): root ViewSet for automation orchestration.
|
||||
- **Linker** (`modules/linker/urls.py`): root ViewSet for internal linking.
|
||||
- **Optimizer** (`modules/optimizer/urls.py`): root ViewSet for optimization analysis.
|
||||
- **Publisher** (`modules/publisher/urls.py`): `/publishing-records`, `/deployments`, root publisher actions, plus public `sites/<id>/definition/`.
|
||||
- **Integration** (`modules/integration/urls.py`): `/integrations`, WordPress webhooks at `/webhooks/wordpress/status|metadata/`.
|
||||
- **System** (`modules/system/urls.py`): prompts, author profiles, strategies, settings (system/account/user/modules/ai), module enable toggle at `/settings/modules/enable/`, health/status/metrics, Gitea webhook, integration settings save/test/generate/progress routes.
|
||||
- **Account** (`api/urls.py`): `/settings`, `/team`, `/usage/analytics/`.
|
||||
- **Billing (tenant)** (`business/billing/urls.py`): invoices, payments (list + manual submit + available methods), credit-packages, credit-transactions, payment-methods (CRUD + set_default), credits balance/usage/transactions.
|
||||
- **Billing (admin)** (`modules/billing/admin_urls.py` under `/api/v1/admin/`): stats, users credit adjustments, credit costs, plus aliases to `AdminBillingViewSet` for invoices, payments, pending_payments, approve/reject, payment-method-configs, account-payment-methods (CRUD + set_default).
|
||||
|
||||
## 3) Billing & Credits Data Model (key fields)
|
||||
- **Invoice:** `subtotal`, `tax`, `total`, `currency`, `status`, `invoice_date`, `due_date`, `paid_at`, `line_items`, `stripe_invoice_id`, `payment_method`, billing period fields; helper properties expose `subtotal_amount`, `tax_amount`, `total_amount` for compatibility.
|
||||
- **Payment:** statuses include `pending_approval`, `processing`, `completed/succeeded`, `failed/refunded/cancelled`; methods include `stripe`, `paypal`, `bank_transfer`, `local_wallet`, `manual`; tracks intent/charge ids, manual references, approvals, timestamps, failure_reason.
|
||||
- **CreditPackage:** slugged packages with price, credits, stripe/paypal ids, `is_active`, `is_featured`, `sort_order`, features JSON.
|
||||
- **CreditTransaction:** transaction_type, amount, balance_after, metadata, reference_id (invoice/payment), account-scoped.
|
||||
- **CreditUsageLog:** operation_type, credits_used, tokens/model, related object refs, metadata; account-scoped.
|
||||
- **CreditCostConfig:** per-operation configurable costs (admin editable).
|
||||
- **Account billing fields:** billing email/address/tax_id on `Account` (multi-tenant).
|
||||
|
||||
## 4) Frontend Surface (routes & guards)
|
||||
- **Planner:** `/planner/keywords`, `/planner/clusters`, `/planner/clusters/:id`, `/planner/ideas`.
|
||||
- **Writer:** `/writer/tasks`, `/writer/content`, `/writer/content/:id`, `/writer/drafts` (redirect), `/writer/images`, `/writer/review`, `/writer/published`.
|
||||
- **Automation:** `/automation`.
|
||||
- **Linker:** `/linker`, `/linker/content`.
|
||||
- **Optimizer:** `/optimizer`, `/optimizer/content`, `/optimizer/analyze/:id`.
|
||||
- **Thinker:** `/thinker` redirect → `/thinker/prompts`; also `/thinker/author-profiles`, `/thinker/profile`, `/thinker/strategies`, `/thinker/image-testing`.
|
||||
- **Billing (tenant module):** `/billing/overview`, `/billing/credits`, `/billing/transactions`, `/billing/usage`.
|
||||
- **Account:** `/account/plans`, `/account/billing`, `/account/purchase-credits`, `/account/settings`, `/account/team`, `/account/usage`.
|
||||
- **Admin:** `/admin/dashboard`, `/admin/accounts`, `/admin/subscriptions`, `/admin/account-limits`, `/admin/billing`, `/admin/invoices`, `/admin/payments`, `/admin/payments/approvals`, `/admin/credit-packages`.
|
||||
- **Sidebar source:** `frontend/src/layout/AppSidebar.tsx` (Account group contains Plans & Billing, Plans, Team Management, Usage & Analytics).
|
||||
|
||||
## 5) Multi-Tenancy & Roles
|
||||
- Tenant isolation via `AccountBaseModel` FKs (`account_id`/`tenant_id` column) with indexes; site/sector models enforce belonging to same account.
|
||||
- Admin-only endpoints served under `/api/v1/admin/` (developer/superuser expected; see billing admin ViewSet).
|
||||
- Frontend ModuleGuard used for module-based access on feature modules (planner/writer/linker/optimizer/thinker); account/billing pages are standard routes.
|
||||
|
||||
## 6) Integration & Webhooks
|
||||
- WordPress status/metadata webhooks at `/api/v1/integration/webhooks/wordpress/status|metadata/`.
|
||||
- Gitea webhook at `/api/v1/system/webhook/`.
|
||||
- Health: `/api/v1/system/ping`, `/api/v1/system/status`, request metrics at `/api/v1/system/request-metrics/<request_id>/`.
|
||||
|
||||
## 7) Known Documentation Needs (to be maintained)
|
||||
- Align billing/account docs to the above code-level namespaces and models.
|
||||
- Add frontend route map (kept here) to the Account/Billing docs.
|
||||
- Fill `docs/user-flow/` (currently empty) with end-to-end user/account/billing flows based on this architecture.
|
||||
|
||||
|
||||
@@ -1,263 +0,0 @@
|
||||
# Keywords, Clusters & Ideas - Complete Field Mapping
|
||||
|
||||
**Date:** December 3, 2025
|
||||
|
||||
---
|
||||
|
||||
## 📊 KEYWORDS - Complete Field Reference
|
||||
|
||||
### Backend Model Fields
|
||||
**File:** `backend/igny8_core/business/planning/models.py`
|
||||
|
||||
| Field Name | Type | Default | Required | Choices | Notes |
|
||||
|------------|------|---------|----------|---------|-------|
|
||||
| `id` | Integer PK | Auto | ✅ | - | Primary key |
|
||||
| `seed_keyword` | ForeignKey | - | ✅ | - | Links to SeedKeyword (PROTECT on delete) |
|
||||
| `volume_override` | Integer | NULL | ❌ | - | Site-specific override (uses seed_keyword.volume if NULL) |
|
||||
| `difficulty_override` | Integer | NULL | ❌ | - | Site-specific override (uses seed_keyword.difficulty if NULL) |
|
||||
| `attribute_values` | JSONField | [] | ❌ | - | Optional metadata (product specs, modifiers) |
|
||||
| `cluster` | ForeignKey | NULL | ❌ | - | Parent cluster (SET_NULL on delete) |
|
||||
| `status` | CharField(50) | `pending` | ✅ | `pending`, `active`, `archived` | Keyword status |
|
||||
| `site` | ForeignKey | - | ✅ | - | Owner site (inherited) |
|
||||
| `sector` | ForeignKey | - | ✅ | - | Owner sector (inherited) |
|
||||
| `account` | ForeignKey | - | ✅ | - | Owner account (inherited) |
|
||||
| `created_at` | DateTime | Auto | ✅ | - | Auto-generated |
|
||||
| `updated_at` | DateTime | Auto | ✅ | - | Auto-updated |
|
||||
|
||||
### Frontend Table Columns
|
||||
**File:** `frontend/src/config/pages/keywords.config.tsx`
|
||||
|
||||
| Column Key | Label | Visible Default | Sortable | Render Notes |
|
||||
|----------|--------|-----------------|----------|--------------|
|
||||
| `keyword` | Keyword | ✅ Yes | ✅ Yes | From seed_keyword.keyword (links to Keywords page) |
|
||||
| `sector_name` | Sector | Conditional* | ✅ Yes | Badge (blue) - shown when viewing all sectors |
|
||||
| `volume` | Volume | ✅ Yes | ✅ Yes | Formatted as number (e.g., 1,250) |
|
||||
| `cluster_name` | Cluster | ✅ Yes | ✅ Yes | Parent cluster name or "-" |
|
||||
| `difficulty` | Difficulty | ✅ Yes | ✅ Yes | Badge (1-5): 1-2=green, 3=amber, 4-5=red |
|
||||
| `intent` | Intent | ✅ Yes | ✅ Yes | Badge colors: Transactional/Commercial=green, Navigational=amber |
|
||||
| `status` | Status | ✅ Yes | ✅ Yes | Badge: pending=amber, active=green, archived=red |
|
||||
| `created_at` | Created | ✅ Yes | ✅ Yes | Relative date (e.g., "2 hours ago") |
|
||||
| (Hidden by default) | | | | |
|
||||
| `updated_at` | Updated | ❌ No | ✅ Yes | Relative date |
|
||||
|
||||
### Frontend Filter Dropdown
|
||||
**File:** `frontend/src/config/pages/keywords.config.tsx` (Lines 310-360)
|
||||
|
||||
| Filter Key | Label | Type | Options | Dynamic |
|
||||
|-----------|-------|------|---------|---------|
|
||||
| `search` | Search | Text | N/A | - |
|
||||
| `status` | Status | Select | `pending`, `active`, `archived` | ❌ No |
|
||||
| `intent` | Intent | Select | `informational`, `navigational`, `transactional`, `commercial` | ❌ No |
|
||||
| `difficulty` | Difficulty | Select | `1-5` with labels | ❌ No |
|
||||
| `cluster` | Cluster | Select | Dynamic from database | ✅ Yes (loads clusters) |
|
||||
| `volume` | Volume Range | Custom | Min/Max number inputs | ❌ No (range picker) |
|
||||
|
||||
### Frontend Create/Edit Form
|
||||
**File:** `frontend/src/config/pages/keywords.config.tsx` (Lines 560-586)
|
||||
|
||||
| Field Key | Label | Type | Required | Default | Options |
|
||||
|-----------|-------|------|----------|---------|---------|
|
||||
| `seed_keyword_id` | Seed Keyword | Select | ✅ Yes | - | Dynamic from availableSeedKeywords |
|
||||
| `volume_override` | Volume Override | Number | ❌ No | NULL | Numeric input (optional override) |
|
||||
| `difficulty_override` | Difficulty Override | Number | ❌ No | NULL | Select 1-5 |
|
||||
| `cluster_id` | Cluster | Select | ❌ No | NULL | Dynamic from clusters array |
|
||||
| `status` | Status | Select | ✅ Yes | `pending` | `pending`, `active`, `archived` |
|
||||
|
||||
---
|
||||
|
||||
## 📊 CLUSTERS - Complete Field Reference
|
||||
|
||||
### Backend Model Fields
|
||||
**File:** `backend/igny8_core/business/planning/models.py`
|
||||
|
||||
| Field Name | Type | Default | Required | Choices | Notes |
|
||||
|------------|------|---------|----------|---------|-------|
|
||||
| `id` | Integer PK | Auto | ✅ | - | Primary key |
|
||||
| `name` | CharField(255) | - | ✅ | - | Unique cluster name (unique=True) |
|
||||
| `description` | TextField | NULL | ❌ | - | Optional cluster description |
|
||||
| `keywords_count` | Integer | 0 | ✅ | - | Cached count of linked keywords |
|
||||
| `ideas_count` | Integer | 0 | ✅ | - | Cached count of linked ideas |
|
||||
| `volume` | Integer | 0 | ✅ | - | Cached total volume from keywords |
|
||||
| `mapped_pages` | Integer | 0 | ✅ | - | Number of mapped pages |
|
||||
| `status` | CharField(50) | `new` | ✅ | `new`, `idea`, `mapped` | Cluster status |
|
||||
| `site` | ForeignKey | - | ✅ | - | Owner site (inherited) |
|
||||
| `sector` | ForeignKey | - | ✅ | - | Owner sector (inherited) |
|
||||
| `account` | ForeignKey | - | ✅ | - | Owner account (inherited) |
|
||||
| `created_at` | DateTime | Auto | ✅ | - | Auto-generated |
|
||||
| `updated_at` | DateTime | Auto | ✅ | - | Auto-updated |
|
||||
|
||||
### Frontend Table Columns
|
||||
**File:** `frontend/src/config/pages/clusters.config.tsx`
|
||||
|
||||
| Column Key | Label | Visible Default | Sortable | Render Notes |
|
||||
|----------|--------|-----------------|----------|--------------|
|
||||
| `name` | Cluster Name | ✅ Yes | ✅ Yes | Link to cluster detail page |
|
||||
| `sector_name` | Sector | Conditional* | ✅ Yes | Badge (blue) - shown when viewing all sectors |
|
||||
| `keywords_count` | Keywords | ✅ Yes | ✅ Yes | Formatted as number (e.g., 45) |
|
||||
| `ideas_count` | Ideas | ✅ Yes | ✅ Yes | Formatted as number (e.g., 12) |
|
||||
| `volume` | Volume | ✅ Yes | ✅ Yes | Formatted as number (e.g., 5,280) |
|
||||
| `difficulty` | Difficulty | ✅ Yes | ✅ Yes | Badge (1-5): 1-2=green, 3=amber, 4-5=red |
|
||||
| `content_count` | Content | ✅ Yes | ✅ Yes | Formatted as number |
|
||||
| `status` | Status | ✅ Yes | ✅ Yes | Badge: new=amber, idea=blue, mapped=green |
|
||||
| `created_at` | Created | ✅ Yes | ✅ Yes | Relative date |
|
||||
| (Hidden by default) | | | | |
|
||||
| `description` | Description | ❌ No | ❌ No | Text truncated to 250px |
|
||||
| `mapped_pages` | Mapped Pages | ❌ No | ✅ Yes | Formatted number |
|
||||
| `updated_at` | Updated | ❌ No | ✅ Yes | Relative date |
|
||||
|
||||
### Frontend Filter Dropdown
|
||||
**File:** `frontend/src/config/pages/clusters.config.tsx` (Lines 240-290)
|
||||
|
||||
| Filter Key | Label | Type | Options | Dynamic |
|
||||
|-----------|-------|------|---------|---------|
|
||||
| `search` | Search | Text | N/A | - |
|
||||
| `status` | Status | Select | `new`, `idea`, `mapped` | ❌ No |
|
||||
| `difficulty` | Difficulty | Select | `1-5` with labels | ❌ No |
|
||||
| `volume` | Volume Range | Custom | Min/Max number inputs | ❌ No (range picker) |
|
||||
|
||||
### Frontend Create/Edit Form
|
||||
**File:** `frontend/src/config/pages/clusters.config.tsx` (Lines 405-418)
|
||||
|
||||
| Field Key | Label | Type | Required | Default | Options |
|
||||
|-----------|-------|------|----------|---------|---------|
|
||||
| `name` | Cluster Name | Text | ✅ Yes | - | Text input (placeholder: "Enter cluster name") |
|
||||
| `description` | Description | Textarea | ❌ No | NULL | Textarea (placeholder: "Enter cluster description") |
|
||||
| `status` | Status | Select | ✅ Yes | `new` | `new`, `idea`, `mapped` |
|
||||
|
||||
---
|
||||
|
||||
## 📊 CONTENT IDEAS - Complete Field Reference
|
||||
|
||||
### Backend Model Fields
|
||||
**File:** `backend/igny8_core/business/planning/models.py`
|
||||
|
||||
| Field Name | Type | Default | Required | Choices | Notes |
|
||||
|------------|------|---------|----------|---------|-------|
|
||||
| `id` | Integer PK | Auto | ✅ | - | Primary key |
|
||||
| `idea_title` | CharField(255) | - | ✅ | - | Content idea title |
|
||||
| `description` | TextField | NULL | ❌ | - | Content outline/description |
|
||||
| `target_keywords` | CharField(500) | '' | ❌ | - | Comma-separated keywords (legacy) |
|
||||
| `keyword_objects` | M2M(Keywords) | - | ❌ | - | Individual keywords linked to idea |
|
||||
| `keyword_cluster` | ForeignKey(Clusters) | NULL | ❌ | - | Parent cluster (SET_NULL on delete) |
|
||||
| `status` | CharField(50) | `new` | ✅ | `new`, `scheduled`, `completed`, `published` | Idea workflow status |
|
||||
| `estimated_word_count` | Integer | 1000 | ✅ | - | Target article length |
|
||||
| `content_type` | CharField(50) | `post` | ✅ | `post`, `page`, `product`, `taxonomy` | Content type |
|
||||
| `content_structure` | CharField(50) | `article` | ✅ | See structures below | Content format/structure |
|
||||
| `site` | ForeignKey | - | ✅ | - | Owner site (inherited) |
|
||||
| `sector` | ForeignKey | - | ✅ | - | Owner sector (inherited) |
|
||||
| `account` | ForeignKey | - | ✅ | - | Owner account (inherited) |
|
||||
| `created_at` | DateTime | Auto | ✅ | - | Auto-generated |
|
||||
| `updated_at` | DateTime | Auto | ✅ | - | Auto-updated |
|
||||
|
||||
**Content Structure Choices (based on content_type):**
|
||||
- **Post:** `article`, `guide`, `comparison`, `review`, `listicle`
|
||||
- **Page:** `landing_page`, `business_page`, `service_page`, `general`, `cluster_hub`
|
||||
- **Product:** `product_page`
|
||||
- **Taxonomy:** `category_archive`, `tag_archive`, `attribute_archive`
|
||||
|
||||
### Frontend Table Columns
|
||||
**File:** `frontend/src/config/pages/ideas.config.tsx`
|
||||
|
||||
| Column Key | Label | Visible Default | Sortable | Render Notes |
|
||||
|----------|--------|-----------------|----------|--------------|
|
||||
| `idea_title` | Title | ✅ Yes | ✅ Yes | Expandable (shows description) |
|
||||
| `sector_name` | Sector | Conditional* | ✅ Yes | Badge (blue) - shown when viewing all sectors |
|
||||
| `content_structure` | Structure | ✅ Yes | ✅ Yes | Badge (purple): article, guide, guide, etc. |
|
||||
| `content_type` | Type | ✅ Yes | ✅ Yes | Badge (blue): post, page, product, taxonomy |
|
||||
| `target_keywords` | Target Keywords | ✅ Yes | ❌ No | Text truncated to 250px |
|
||||
| `keyword_cluster_name` | Cluster | ✅ Yes | ✅ Yes | Parent cluster name or "-" |
|
||||
| `status` | Status | ✅ Yes | ✅ Yes | Badge: new=amber, scheduled=blue, completed=blue, published=green |
|
||||
| `estimated_word_count` | Words | ✅ Yes | ✅ Yes | Formatted as number (e.g., 1,500) |
|
||||
| `created_at` | Created | ✅ Yes | ✅ Yes | Relative date |
|
||||
| (Hidden by default) | | | | |
|
||||
| `updated_at` | Updated | ❌ No | ✅ Yes | Relative date |
|
||||
|
||||
### Frontend Filter Dropdown
|
||||
**File:** `frontend/src/config/pages/ideas.config.tsx` (Lines 218-270)
|
||||
|
||||
| Filter Key | Label | Type | Options | Dynamic |
|
||||
|-----------|-------|------|---------|---------|
|
||||
| `search` | Search | Text | N/A | - |
|
||||
| `status` | Status | Select | `new`, `scheduled`, `completed`, `published` | ❌ No |
|
||||
| `content_structure` | Structure | Select | article, guide, comparison, etc. (all 13 options) | ❌ No |
|
||||
| `content_type` | Type | Select | `post`, `page`, `product`, `taxonomy` | ❌ No |
|
||||
| `cluster` | Cluster | Select | Dynamic from database | ✅ Yes (loads clusters) |
|
||||
|
||||
### Frontend Create/Edit Form
|
||||
**File:** `frontend/src/config/pages/ideas.config.tsx` (Lines 372-417)
|
||||
|
||||
| Field Key | Label | Type | Required | Default | Options |
|
||||
|-----------|-------|------|----------|---------|---------|
|
||||
| `idea_title` | Title | Text | ✅ Yes | - | Text input (placeholder: "Enter idea title") |
|
||||
| `description` | Description | Textarea | ❌ No | NULL | Textarea (placeholder: "Enter content outline") |
|
||||
| `content_type` | Content Type | Select | ✅ Yes | `post` | `post`, `page`, `product`, `taxonomy` |
|
||||
| `content_structure` | Content Structure | Select | ✅ Yes | `article` | 13 structure options (depends on content_type) |
|
||||
| `target_keywords` | Target Keywords | Text | ❌ No | NULL | Text input (comma-separated) |
|
||||
| `keyword_cluster_id` | Cluster | Select | ❌ No | NULL | Dynamic from clusters array |
|
||||
| `status` | Status | Select | ✅ Yes | `new` | `new`, `scheduled`, `completed`, `published` |
|
||||
| `estimated_word_count` | Word Count | Number | ❌ No | 1000 | Numeric input |
|
||||
|
||||
---
|
||||
|
||||
## 📝 SUMMARY COMPARISON
|
||||
|
||||
### Status Fields
|
||||
| Module | Backend Default | Backend Choices | Frontend Form Default | Frontend Form Choices |
|
||||
|--------|-----------------|-----------------|----------------------|----------------------|
|
||||
| **Keywords** | `pending` | pending, active, archived | `pending` | pending, active, archived |
|
||||
| **Clusters** | `new` | new, idea, mapped | `new` | new, idea, mapped |
|
||||
| **Ideas** | `new` | new, scheduled, completed, published | `new` | new, scheduled, completed, published |
|
||||
|
||||
### Required Fields (Must be filled)
|
||||
| Module | Required Fields |
|
||||
|--------|-----------------|
|
||||
| **Keywords** | seed_keyword_id, status, site, sector, account |
|
||||
| **Clusters** | name, status, site, sector, account |
|
||||
| **Ideas** | idea_title, status, content_type, content_structure, estimated_word_count, site, sector, account |
|
||||
|
||||
### Optional Fields
|
||||
| Module | Optional Fields |
|
||||
|--------|-----------------|
|
||||
| **Keywords** | volume_override, difficulty_override, attribute_values, cluster_id |
|
||||
| **Clusters** | description |
|
||||
| **Ideas** | description, target_keywords, keyword_objects, keyword_cluster_id |
|
||||
|
||||
### Dynamic Dropdowns (Loaded from DB)
|
||||
| Module | Filter | Form |
|
||||
|--------|--------|------|
|
||||
| **Keywords** | cluster (dropdown) | seed_keyword_id (all available), cluster_id (all clusters for sector) |
|
||||
| **Clusters** | - | - |
|
||||
| **Ideas** | cluster (dropdown) | keyword_cluster_id (all clusters for sector) |
|
||||
|
||||
### Visible-by-Default Table Columns
|
||||
| Module | Count | Primary Columns |
|
||||
|--------|-------|-----------------|
|
||||
| **Keywords** | 9 | keyword, volume, cluster, difficulty, intent, status, created_at |
|
||||
| **Clusters** | 11 | name, keywords_count, ideas_count, volume, difficulty, content_count, status, created_at |
|
||||
| **Ideas** | 10 | idea_title, content_structure, content_type, target_keywords, cluster, status, word_count, created_at |
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Key Differences
|
||||
|
||||
### Keywords
|
||||
- **Uses:** SeedKeyword (global pool) - one keyword per site/sector
|
||||
- **Overrideable:** volume, difficulty (site-specific)
|
||||
- **Links to:** Clusters (via cluster FK)
|
||||
- **Status:** pending (awaiting cluster), active (clustered), archived
|
||||
|
||||
### Clusters
|
||||
- **Type:** Pure topic clusters (semantic groupings)
|
||||
- **Auto-updated:** keywords_count, ideas_count, volume (cached from keywords)
|
||||
- **Status:** new (no ideas), idea (has ideas), mapped (has content)
|
||||
- **No overrides:** All values are cached/calculated
|
||||
|
||||
### Ideas
|
||||
- **Type:** Content concepts ready for production
|
||||
- **Links:** Cluster (required for workflow), Keywords (optional M2M)
|
||||
- **Customizable:** content_type, content_structure, word_count
|
||||
- **Status:** new → scheduled (queued to writer) → completed (content generated) → published
|
||||
|
||||
---
|
||||
|
||||
**END OF COMPLETE FIELD MAPPING**
|
||||
@@ -1,258 +0,0 @@
|
||||
# Content Taxonomy Relationship Diagram
|
||||
|
||||
## Current Architecture (Simplified)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ IGNY8 Content System │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Multi-Tenant Hierarchy │
|
||||
│ │
|
||||
│ Account ──┬── Site ──┬── Sector ──┬── Keywords │
|
||||
│ │ │ ├── Clusters │
|
||||
│ │ │ ├── Ideas │
|
||||
│ │ │ ├── Tasks │
|
||||
│ │ │ ├── Content │
|
||||
│ │ │ └── ContentTaxonomy │
|
||||
│ │ └── Sector 2 │
|
||||
│ └── Site 2 │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Planner → Writer → Publisher Workflow │
|
||||
│ │
|
||||
│ Phase 1-3: PLANNER │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Keywords │ ──> │ Clusters │ ──> │ Ideas │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ │
|
||||
│ │ │ │ │
|
||||
│ │ │ │ │
|
||||
│ Phase 4: WRITER │
|
||||
│ │ │ │ │
|
||||
│ └─────────────────┴─────────────────┘ │
|
||||
│ │ │
|
||||
│ v │
|
||||
│ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Tasks │ ──> │ Content │ │
|
||||
│ └──────────┘ └──────────┘ │
|
||||
│ │ │
|
||||
│ │ │
|
||||
│ Phase 5: PUBLISHER │ │
|
||||
│ v │
|
||||
│ ┌─────────────┐ │
|
||||
│ │ WordPress │ │
|
||||
│ │ Shopify │ │
|
||||
│ │ Sites │ │
|
||||
│ └─────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Content ↔ Taxonomy Relationship (Many-to-Many) │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
┌─────────────────────────────────┐
|
||||
│ Content Model │
|
||||
│─────────────────────────────────│
|
||||
│ PK: id │
|
||||
│ FK: site_id ──────┐ │
|
||||
│ FK: sector_id │ │
|
||||
│ FK: cluster_id │ │
|
||||
│ │ │
|
||||
│ title │ │
|
||||
│ content_html │ │
|
||||
│ word_count │ │
|
||||
│ meta_title │ │
|
||||
│ meta_description │ │
|
||||
│ │ │
|
||||
│ content_type │ │
|
||||
│ content_structure │ │
|
||||
│ │ │
|
||||
│ external_id │ │
|
||||
│ external_url │ │
|
||||
│ sync_status │ │
|
||||
│ │ │
|
||||
│ source │ │
|
||||
│ status │ │
|
||||
└───────────┬───────┘ │
|
||||
│ │
|
||||
│ Many-to-Many │
|
||||
│ (via ContentTaxonomyRelation)
|
||||
│ │
|
||||
v │
|
||||
┌─────────────────────────────────┐│
|
||||
│ ContentTaxonomyRelation Model ││
|
||||
│─────────────────────────────────││
|
||||
│ PK: id ││
|
||||
│ FK: content_id ──────────────────┘
|
||||
│ FK: taxonomy_id ────────────┐
|
||||
│ │
|
||||
│ created_at │
|
||||
│ updated_at │
|
||||
│ │
|
||||
│ UNIQUE(content, taxonomy) │
|
||||
└─────────────────────────────┘
|
||||
│
|
||||
│
|
||||
v
|
||||
┌─────────────────────────────────┐
|
||||
│ ContentTaxonomy Model │
|
||||
│─────────────────────────────────│
|
||||
│ PK: id │
|
||||
│ FK: site_id ─────┐ │
|
||||
│ FK: sector_id │ │
|
||||
│ │ │
|
||||
│ name │ │
|
||||
│ slug │ │
|
||||
│ taxonomy_type │ ◄─── "category" or "tag"
|
||||
│ description │ │
|
||||
│ count │ │
|
||||
│ │ │
|
||||
│ WordPress Sync: │ │
|
||||
│ external_taxonomy│ ◄─── "category", "post_tag"
|
||||
│ external_id │ ◄─── WordPress term_id
|
||||
│ │ │
|
||||
│ metadata (JSON) │ │
|
||||
└──────────────────┘ │
|
||||
│ │
|
||||
│ │
|
||||
UNIQUE(site, slug, taxonomy_type)│
|
||||
UNIQUE(site, external_id, external_taxonomy)
|
||||
│ │
|
||||
└──────────────┘
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Usage Example │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
# Get all categories for a content piece
|
||||
categories = content.taxonomy_terms.filter(taxonomy_type='category')
|
||||
|
||||
# Get all tags for a content piece
|
||||
tags = content.taxonomy_terms.filter(taxonomy_type='tag')
|
||||
|
||||
# Add a category to content
|
||||
tech_category = ContentTaxonomy.objects.get(name='Technology')
|
||||
content.taxonomy_terms.add(tech_category)
|
||||
|
||||
# Get all content with a specific tag
|
||||
tutorial_tag = ContentTaxonomy.objects.get(name='Tutorial')
|
||||
contents = tutorial_tag.contents.all()
|
||||
|
||||
# WordPress publishing (automatic)
|
||||
wp_categories = [term.name for term in content.taxonomy_terms.filter(taxonomy_type='category')]
|
||||
wp_tags = [term.name for term in content.taxonomy_terms.filter(taxonomy_type='tag')]
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ WordPress Integration (Bidirectional Sync) │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
IGNY8 → WordPress:
|
||||
─────────────────
|
||||
1. Content created in IGNY8
|
||||
2. Categories/tags assigned via taxonomy_terms M2M
|
||||
3. Publishing task created
|
||||
4. wordpress_publishing.py extracts:
|
||||
- categories = content.taxonomy_terms.filter(taxonomy_type='category')
|
||||
- tags = content.taxonomy_terms.filter(taxonomy_type='tag')
|
||||
5. REST API creates WordPress post with terms
|
||||
6. external_id saved back to Content model
|
||||
7. Log: [5-homeg8.com] [POST] Published: "Article Title" (ID: 123)
|
||||
|
||||
WordPress → IGNY8:
|
||||
─────────────────
|
||||
1. WordPress plugin detects post update
|
||||
2. REST API sends post data + terms to IGNY8
|
||||
3. Content updated/created with external_id
|
||||
4. ContentTaxonomy created/updated with external_id
|
||||
5. ContentTaxonomyRelation created linking them
|
||||
6. Log: [5-homeg8.com] [SYNC] Imported: "Article Title"
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Database Tables Summary │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
igny8_content
|
||||
├─ id (PK)
|
||||
├─ site_id (FK → sites)
|
||||
├─ sector_id (FK → sectors)
|
||||
├─ cluster_id (FK → clusters)
|
||||
├─ title, content_html, word_count
|
||||
├─ meta_title, meta_description, keywords
|
||||
├─ content_type, content_structure
|
||||
├─ external_id, external_url, sync_status
|
||||
└─ source, status, timestamps
|
||||
|
||||
igny8_content_taxonomy_relations (Through Table)
|
||||
├─ id (PK)
|
||||
├─ content_id (FK → igny8_content)
|
||||
├─ taxonomy_id (FK → igny8_content_taxonomy_terms)
|
||||
└─ timestamps
|
||||
UNIQUE(content_id, taxonomy_id)
|
||||
|
||||
igny8_content_taxonomy_terms
|
||||
├─ id (PK)
|
||||
├─ site_id (FK → sites)
|
||||
├─ sector_id (FK → sectors)
|
||||
├─ name, slug
|
||||
├─ taxonomy_type ('category' | 'tag')
|
||||
├─ description, count
|
||||
├─ external_taxonomy ('category' | 'post_tag')
|
||||
├─ external_id (WordPress term_id)
|
||||
├─ metadata (JSON)
|
||||
└─ timestamps
|
||||
UNIQUE(site_id, slug, taxonomy_type)
|
||||
UNIQUE(site_id, external_id, external_taxonomy)
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Before vs After Comparison │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
BEFORE (Complex - with SiteBlueprint):
|
||||
────────────────────────────────────
|
||||
SiteBlueprint ──┬── SiteBlueprintCluster ──> Clusters
|
||||
├── SiteBlueprintTaxonomy ──> ContentTaxonomy
|
||||
└── PageBlueprint ──> Content
|
||||
|
||||
Content ──> ContentTaxonomyMap ──> ContentTaxonomy
|
||||
(separate table with FK)
|
||||
|
||||
PublishingRecord ──┬── content_id
|
||||
└── site_blueprint_id
|
||||
|
||||
DeploymentRecord ──┬── content_id
|
||||
└── site_blueprint_id
|
||||
|
||||
|
||||
AFTER (Simple - SiteBlueprint Removed):
|
||||
──────────────────────────────────────
|
||||
Keywords ──> Clusters ──> Ideas ──> Tasks ──> Content
|
||||
|
||||
Content ↔ ContentTaxonomy
|
||||
(M2M via ContentTaxonomyRelation)
|
||||
|
||||
PublishingRecord ──> content_id
|
||||
|
||||
DeploymentRecord ──> content_id
|
||||
|
||||
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Key Benefits │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
✅ Simpler architecture - removed 4 models
|
||||
✅ Direct M2M relationship - easier to query
|
||||
✅ Less database joins - better performance
|
||||
✅ Clear taxonomy model in Django admin
|
||||
✅ WordPress sync unchanged - still works perfectly
|
||||
✅ Planner-Writer-Publisher workflow intact
|
||||
✅ Multi-tenant security maintained
|
||||
✅ No circular dependencies
|
||||
✅ Clean codebase - no legacy blueprint code
|
||||
@@ -1,237 +0,0 @@
|
||||
bluesalman: gothrough readme and tell me which folders/files inside backeedn adn frontedn folders are acutaly part of igny8 app, and which folders are part of some tech stack which will be enever modifed by app only isntaleld when tech stack isntaleled
|
||||
|
||||
GitHub Copilot: I'll create a short plan, then list the backend and frontend directories to inspect their contents.
|
||||
|
||||
Created 3 todos
|
||||
|
||||
Now I'll list both directories (backend and frontend) to collect the items to classify.
|
||||
|
||||
Read [](file:///data/app/igny8/backend)
|
||||
|
||||
Read [](file:///data/app/igny8/frontend)
|
||||
|
||||
Marking plan steps completed and finishing classification.
|
||||
|
||||
Completed: *List backend files* (1/3)
|
||||
|
||||
Progress: I scanned backend and frontend and classified items below.
|
||||
|
||||
**Backend — Classified**
|
||||
- **App (igny8 application core):**
|
||||
- `igny8_core/`: The Django project / app code (settings, apps, views, models, tasks). This is the main application source you will modify.
|
||||
- `manage.py`: Django management entrypoint — part of the app.
|
||||
- `migrations/`: Django DB migrations — app schema changes (source-controlled).
|
||||
- `staticfiles/`: Collected static assets (app assets). May be generated but belongs to the app.
|
||||
- `requirements.txt`: Dependency manifest for the app (used to install the Python packages the app needs). Not runtime third-party code, but part of the app repo and should be version-controlled.
|
||||
|
||||
- **Repo tooling / app-maintenance scripts (part of the repo; maintained by your team):**
|
||||
- `check_api_response.py`, `check_recent_keywords.py`, `cleanup_structure_categories.py`, `diagnose_generate_content.py`, `inject_test_data.py`, `sync_idea_status.py`, `test_tags_categories.py`, `verify_*.py`, `fix_*` scripts, `final_verify.py`, `deploy_automation.sh`, `example_csv/`: These are maintenance, verification, or migration scripts that are part of the repository and used by developers/ops. They are not third-party code but are repo-level tooling you may edit.
|
||||
|
||||
- **Tech stack / infra artifacts (should not be modified by the app runtime; installed or produced by infra):**
|
||||
- `.venv/`: Virtual environment — environment artifact; not part of app source and typically excluded from commits.
|
||||
- `Dockerfile`: Container build file — infra/build config (used by ops or CI).
|
||||
- `backup_postgres_20251120_232816.sql`, `db_backup_20251120_232646.sqlite3`, `rename_fields_migration.sql`: Database backups / SQL dumps — artifacts, not app source.
|
||||
- `celerybeat-schedule`: Celery scheduler state file — runtime artifact.
|
||||
- `logs/`: Runtime logs — generated artifacts.
|
||||
- Any large generated files (e.g., DB dumps) are infra artifacts and shouldn't be edited as source.
|
||||
|
||||
Notes:
|
||||
- Some items like `deploy_automation.sh` or `fix_*` scripts are operational but live in the repo — treat them as repository tooling (editable by you), whereas backups, `.venv`, and docker images/configs are infra artifacts.
|
||||
|
||||
**Frontend — Classified**
|
||||
- **App (igny8 frontend source / code you will modify):**
|
||||
- `src/`: Application source (React/Vue/Svelte/TSX/etc.) — primary frontend code to edit.
|
||||
- `public/`, `index.html`, `marketing.html`, `banner.png`: Static assets and HTML entrypoints — part of the app.
|
||||
- `package.json`, `package-lock.json`: NPM manifests — define packages the frontend needs (repo-managed).
|
||||
- `eslint.config.js`, `postcss.config.js`, `vite.config.ts`, `vitest.config.ts`, `tsconfig*.json`: Build and tooling config that belong to the frontend source and are edited as part of app development.
|
||||
- README.md, `DESIGN_SYSTEM.md`, `MIGRATION_GUIDE.md`: Documentation for the frontend — repo docs.
|
||||
|
||||
- **Tech stack / infra artifacts (installed/generated by stack — do not edit at runtime):**
|
||||
- `node_modules/`: Installed dependencies — generated by `npm` / `pnpm` / `yarn`; should not be manually edited and typically ignored by VCS.
|
||||
- `dist/`: Built production bundle — generated artifact (output of the build).
|
||||
- `Caddyfile`, `Caddyfile.marketing`, `Dockerfile`, `Dockerfile.dev`, `Dockerfile.marketing`, `Dockerfile.marketing.dev`: Webserver and container build configs — deployment/infra files. They belong to ops/build but do live in the repo; treat as infra configuration, not application logic.
|
||||
- `LICENSE.md`, other top-level docs: docs, not runtime app code.
|
||||
|
||||
Notes:
|
||||
- Build-tool config files (e.g., `vite.config.ts`, `tsconfig.json`) are part of the app repository and are changed by developers. By contrast, `node_modules` and `dist` are installed/generated by the tech stack and should not be edited directly.
|
||||
|
||||
Quick summary (one-line):
|
||||
- Edit: igny8_core, `manage.py`, migrations, src, public, `package.json`, and similar source/config files — these are part of the igny8 app.
|
||||
- Don't manually edit: runtime/generated/installed artifacts like `.venv/`, `node_modules/`, `dist/`, DB backups, `celerybeat-schedule`, and use `Dockerfile`/`Caddyfile` only as infra config (ops-managed).
|
||||
|
||||
Would you like:
|
||||
- a) A file added to the repo (e.g., `ARCHITECTURE.md` or `IGNY8-CONTENTS.md`) that labels each top-level item as "app" vs "infra", or
|
||||
- b) I can produce a JSON/CSV mapping of items for automation (useful for CI), or
|
||||
- c) Walk through any specific file/folder you want deeper inspection of?
|
||||
|
||||
Which option do you prefer?
|
||||
|
||||
bluesalman: ok now cehck teh strcutre of /data adn recrussively all its fodlers including igny8/
|
||||
waht can we do if we want to make igny8 folder as the only full app foodler requried to move to any new location,
|
||||
and jsut install the portainer as current envirnpmaent and isntall teh app package on nwe locatink build the package and it runs there,
|
||||
|
||||
teh curretn setup isnto standrd as it ocntians some node )moduels dist and env,
|
||||
venv i know can be removed completely
|
||||
|
||||
so waht do you think
|
||||
|
||||
GitHub Copilot: I'll make a short plan, then list data and the key subfolders (app, igny8, and their important children) so I can analyze what to keep, remove, or package.
|
||||
|
||||
Created 4 todos
|
||||
|
||||
Read [](file:///data)
|
||||
|
||||
Updating the plan: mark the scan step done (noting data access blocked) and keep other steps ready. Then I'll provide analysis, exact commands, Docker and non-Docker packaging options, and a sample `docker-compose.yml` and `install.sh`.
|
||||
|
||||
bluesalman: waht happend you are stucl
|
||||
|
||||
GitHub Copilot: I'll create a short todo plan, then list data, app, and igny8 to inspect contents before proposing a packaging strategy.
|
||||
|
||||
Created 4 todos
|
||||
|
||||
Read [](file:///data)
|
||||
|
||||
Now I'll run a quick directory size and shallow tree under the workspace `igny8/` so we can see which folders (like `node_modules`, `.venv`, `dist`) are large and where they are.
|
||||
|
||||
Ran terminal command: cd /data/app/igny8 && echo 'Top-level sizes:' && du -sh * 2>/dev/null | sort -h && echo && echo 'Directory tree (depth 3):' && find . -maxdepth 3 -type d | sed 's|^\./||' | sort
|
||||
|
||||
Updating the todo list: mark the listing steps done and the analysis/packaging step in-progress.
|
||||
|
||||
Starting: *Analyze and propose packaging* (4/4)
|
||||
|
||||
Progress update: I inspected the workspace `igny8/` (sizes and a shallow tree). Next I analyzed which files to keep/exclude and drafted packaging, cleanup commands, and a sample `docker-compose.yml` plus a `prepare_package.sh` plan. Below are clear findings and actionable steps.
|
||||
|
||||
**Findings**
|
||||
- **Large/generated artifacts present:**
|
||||
- .venv — virtualenv (can be removed for packaging).
|
||||
- node_modules, node_modules — installed npm dependencies (do not include in package).
|
||||
- dist — built frontend assets (can be produced during build; you can include or build on deploy).
|
||||
- DB dumps and scheduler/state files appear under backend (e.g., `*.sql`, `*.sqlite3`, `celerybeat-schedule`) — these are artifacts and should not be bundled as app source.
|
||||
- **App source & config (keep these):**
|
||||
- igny8_core, manage.py, migrations, requirements.txt — Django app and deps manifest.
|
||||
- src, public, package.json and build config files — frontend source and build config.
|
||||
- `Dockerfile` / `Dockerfile.*` and `Caddyfile` — infra config that should remain in the repo (useful for containerization).
|
||||
- **Repo tooling and docs:** `deploy_automation.sh`, `verify_*.py`, docs, README.md — keep in repo; useful for ops/dev.
|
||||
|
||||
**Goal**
|
||||
Make `igny8/` a single, relocatable application folder that you can move to a new host, then deploy via Portainer (or Docker Compose) without shipping generated artifacts like `node_modules`, `.venv`, or large backups.
|
||||
|
||||
Recommendations and exact steps
|
||||
|
||||
- **Strategy (recommended): Docker image + single `docker-compose.yml` stack**
|
||||
- Keep only source, config, Dockerfiles and manifests in the app folder.
|
||||
- Exclude generated artifacts from the package (`.venv`, `node_modules`, `dist`, backups, logs).
|
||||
- Provide a `docker-compose.yml` that builds the backend and frontend images from the repo and defines services: `db` (Postgres), `redis`, backend (Gunicorn/Django), `worker` (Celery), and frontend (built and served by Caddy or served as static in a small nginx/caddy image).
|
||||
- Deploy via Portainer by uploading the `docker-compose.yml` (Portainer can build images from contexts if configured), or pre-build images, push to container registry, and use compose with image references.
|
||||
|
||||
- **Alternative:** Tarball + install script that runs `docker compose build` and `docker compose up -d` on the new host.
|
||||
|
||||
- **Safety note:** Always create a backup or branch of the repo before mass-deleting artifacts.
|
||||
|
||||
Exact commands to produce a clean package (run from igny8):
|
||||
|
||||
```bash
|
||||
# make a copy first (safe)
|
||||
cp -a /data/app/igny8 /tmp/igny8-package-source
|
||||
|
||||
# inside the copy, remove generated artifacts
|
||||
cd /tmp/igny8-package-source
|
||||
rm -rf backend/.venv
|
||||
rm -rf backend/*.sql backend/*.sqlite3 backend/celerybeat-schedule backend/logs || true
|
||||
rm -rf frontend/node_modules sites/node_modules
|
||||
rm -rf frontend/dist
|
||||
# optional: remove any large backups in root
|
||||
rm -f backup_*.sql || true
|
||||
|
||||
# create archive
|
||||
tar -czf /tmp/igny8-clean.tgz .
|
||||
|
||||
# verify size
|
||||
ls -lh /tmp/igny8-clean.tgz
|
||||
```
|
||||
|
||||
Suggested `.dockerignore` (place in repo root) to keep image builds small:
|
||||
|
||||
```
|
||||
backend/.venv
|
||||
frontend/node_modules
|
||||
sites/node_modules
|
||||
frontend/dist
|
||||
*.sql
|
||||
*.sqlite3
|
||||
logs/
|
||||
*.log
|
||||
.git
|
||||
node_modules
|
||||
```
|
||||
|
||||
Sample minimal `docker-compose.yml` (adapt to your app's env vars and Dockerfiles):
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
db:
|
||||
image: postgres:15
|
||||
environment:
|
||||
POSTGRES_DB: igny8
|
||||
POSTGRES_USER: igny8
|
||||
POSTGRES_PASSWORD: changeme
|
||||
volumes:
|
||||
- db_data:/var/lib/postgresql/data
|
||||
|
||||
redis:
|
||||
image: redis:7
|
||||
command: redis-server --appendonly yes
|
||||
|
||||
backend:
|
||||
build:
|
||||
context: ./backend
|
||||
dockerfile: Dockerfile
|
||||
depends_on:
|
||||
- db
|
||||
- redis
|
||||
environment:
|
||||
DATABASE_URL: postgres://igny8:changeme@db:5432/igny8
|
||||
REDIS_URL: redis://redis:6379/0
|
||||
ports:
|
||||
- "8000:8000"
|
||||
volumes:
|
||||
- static_volume:/app/staticfiles
|
||||
|
||||
worker:
|
||||
build:
|
||||
context: ./backend
|
||||
dockerfile: Dockerfile
|
||||
command: celery -A igny8_core worker --loglevel=info
|
||||
depends_on:
|
||||
- redis
|
||||
- db
|
||||
|
||||
frontend:
|
||||
build:
|
||||
context: ./frontend
|
||||
dockerfile: Dockerfile
|
||||
ports:
|
||||
- "3000:80"
|
||||
depends_on:
|
||||
- backend
|
||||
|
||||
volumes:
|
||||
db_data:
|
||||
static_volume:
|
||||
```
|
||||
|
||||
Notes:
|
||||
- The Dockerfile should install Python deps from `requirements.txt`, collect static, run migrations in a start-up script, and run Gunicorn. If not present, we can produce a small `Dockerfile`.
|
||||
- The Dockerfile should `npm ci && npm run build` and copy `dist/` into a small static-server image (Caddy or nginx). If you prefer to serve frontend via the backend/CDN, build and copy the static files into backend static volume.
|
||||
|
||||
What I'd do next (I can do these for you)
|
||||
- Option A (recommended): Create `docker-compose.yml` and `prepare_package.sh` in the repo (I can add both files), plus `.dockerignore` and a minimal Dockerfile / Dockerfile if missing. Then test `docker compose up --build` locally.
|
||||
- Option B: Create an archive script only (if you prefer to handle Dockerfiles yourself).
|
||||
|
||||
Would you like me to:
|
||||
- **(1)** Create the `prepare_package.sh`, `.dockerignore`, and a `docker-compose.yml` in the repository and test the compose up here, or
|
||||
- **(2)** Only produce the files and commands you can run on your target host (no changes to repo), or
|
||||
- **(3)** Generate a simple Dockerfile and Dockerfile for a fully self-contained build?
|
||||
|
||||
Pick an option and I’ll implement it.
|
||||
@@ -1,335 +0,0 @@
|
||||
# Status Implementation Tables - Complete Reference
|
||||
|
||||
**Date:** December 3, 2025
|
||||
|
||||
---
|
||||
|
||||
## 🎯 KEYWORDS MODULE
|
||||
|
||||
### Backend Model
|
||||
**File:** `/backend/igny8_core/business/planning/models.py`
|
||||
|
||||
| Field | Type | Choices | Default | Required | Description |
|
||||
|-------|------|---------|---------|----------|-------------|
|
||||
| `status` | CharField(50) | `new`, `mapped` | `new` | ✅ Yes | Workflow status |
|
||||
| `disabled` | BooleanField | - | `False` | ✅ Yes | Filter control (exclude from processes) |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Table Column
|
||||
**File:** `/frontend/src/config/pages/keywords.config.tsx` (Lines ~230-248)
|
||||
|
||||
| Status Value | Badge Color | Badge Label | Display When |
|
||||
|--------------|-------------|-------------|--------------|
|
||||
| `new` | Amber/Yellow | New | Keyword not yet clustered |
|
||||
| `mapped` | Green | Mapped | Keyword assigned to cluster |
|
||||
| *(disabled=true)* | Red/Gray | Disabled | User manually disabled (optional display) |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Filter Dropdown
|
||||
**File:** `/frontend/src/config/pages/keywords.config.tsx` (Lines ~310-318)
|
||||
|
||||
| Filter Type | Options | Default | Description |
|
||||
|-------------|---------|---------|-------------|
|
||||
| Status Select | `new`, `mapped` | All | Workflow status filter |
|
||||
| Disabled Checkbox | Show/Hide disabled | Hide disabled | Filter control |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Form Fields
|
||||
**File:** `/frontend/src/config/pages/keywords.config.tsx` (Lines ~560-570)
|
||||
|
||||
| Field | Type | Options | Default | Required | Editable |
|
||||
|-------|------|---------|---------|----------|----------|
|
||||
| Status | Select | `new`, `mapped` | `new` | ✅ Yes | ✅ Yes |
|
||||
| Disabled | Checkbox | true/false | `false` | ❌ No | ✅ Yes |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 CLUSTERS MODULE
|
||||
|
||||
### Backend Model
|
||||
**File:** `/backend/igny8_core/business/planning/models.py`
|
||||
|
||||
| Field | Type | Choices | Default | Required | Description |
|
||||
|-------|------|---------|---------|----------|-------------|
|
||||
| `status` | CharField(50) | `new`, `mapped` | `new` | ✅ Yes | Workflow status |
|
||||
| `disabled` | BooleanField | - | `False` | ✅ Yes | Filter control (exclude from processes) |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Table Column
|
||||
**File:** `/frontend/src/config/pages/clusters.config.tsx` (Lines ~190-200)
|
||||
|
||||
| Status Value | Badge Color | Badge Label | Display When |
|
||||
|--------------|-------------|-------------|--------------|
|
||||
| `new` | Amber/Yellow | New | Cluster created, no ideas generated yet |
|
||||
| `mapped` | Green | Mapped | Ideas generated from cluster |
|
||||
| *(disabled=true)* | Red/Gray | Disabled | User manually disabled (optional display) |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Filter Dropdown
|
||||
**File:** `/frontend/src/config/pages/clusters.config.tsx` (Lines ~240-253)
|
||||
|
||||
| Filter Type | Options | Default | Description |
|
||||
|-------------|---------|---------|-------------|
|
||||
| Status Select | `new`, `mapped` | All | Workflow status filter |
|
||||
| Disabled Checkbox | Show/Hide disabled | Hide disabled | Filter control |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Form Fields
|
||||
**File:** `/frontend/src/config/pages/clusters.config.tsx` (Lines ~405-418)
|
||||
|
||||
| Field | Type | Options | Default | Required | Editable |
|
||||
|-------|------|---------|---------|----------|----------|
|
||||
| Status | Select | `new`, `mapped` | `new` | ✅ Yes | ✅ Yes |
|
||||
| Disabled | Checkbox | true/false | `false` | ❌ No | ✅ Yes |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 IDEAS MODULE
|
||||
|
||||
### Backend Model
|
||||
**File:** `/backend/igny8_core/business/planning/models.py`
|
||||
|
||||
| Field | Type | Choices | Default | Required | Description |
|
||||
|-------|------|---------|---------|----------|-------------|
|
||||
| `status` | CharField(50) | `new`, `queued`, `completed` | `new` | ✅ Yes | Workflow status |
|
||||
| `disabled` | BooleanField | - | `False` | ✅ Yes | Filter control (exclude from processes) |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Table Column
|
||||
**File:** `/frontend/src/config/pages/ideas.config.tsx` (Lines ~170-185)
|
||||
|
||||
| Status Value | Badge Color | Badge Label | Display When |
|
||||
|--------------|-------------|-------------|--------------|
|
||||
| `new` | Amber/Yellow | New | Idea generated, not queued yet |
|
||||
| `queued` | Blue | Queued | Task created in Writer module |
|
||||
| `completed` | Green | Completed | Content generated (Task completed) |
|
||||
| *(disabled=true)* | Red/Gray | Disabled | User manually disabled (optional display) |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Filter Dropdown
|
||||
**File:** `/frontend/src/config/pages/ideas.config.tsx` (Lines ~218-228)
|
||||
|
||||
| Filter Type | Options | Default | Description |
|
||||
|-------------|---------|---------|-------------|
|
||||
| Status Select | `new`, `queued`, `completed` | All | Workflow status filter |
|
||||
| Disabled Checkbox | Show/Hide disabled | Hide disabled | Filter control |
|
||||
|
||||
---
|
||||
|
||||
### Frontend Form Fields
|
||||
**File:** `/frontend/src/config/pages/ideas.config.tsx` (Lines ~372-385)
|
||||
|
||||
| Field | Type | Options | Default | Required | Editable |
|
||||
|-------|------|---------|---------|----------|----------|
|
||||
| Status | Select | `new`, `queued`, `completed` | `new` | ✅ Yes | ✅ Yes |
|
||||
| Disabled | Checkbox | true/false | `false` | ❌ No | ✅ Yes |
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
# 🔄 STATUS TRANSITION TABLES
|
||||
|
||||
## KEYWORDS Status Transitions
|
||||
|
||||
| Current Status | Trigger/Action | Next Status | Auto/Manual | Updated By |
|
||||
|----------------|----------------|-------------|-------------|------------|
|
||||
| `new` | AI auto_cluster runs | `mapped` | 🤖 Auto | `auto_cluster.py` Line 297 |
|
||||
| `new` | User manually assigns to cluster | `mapped` | 👤 Manual | User form edit |
|
||||
| `new` | User toggles disabled | *(stays new)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| `mapped` | User toggles disabled | *(stays mapped)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| *(any)* | User re-enables | *(stays same)* + `disabled=false` | 👤 Manual | User form edit |
|
||||
|
||||
**Workflow Path:**
|
||||
```
|
||||
new ──[auto_cluster AI]──> mapped
|
||||
│
|
||||
└──[user toggle disabled]──> (status unchanged, disabled flag set)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CLUSTERS Status Transitions
|
||||
|
||||
| Current Status | Trigger/Action | Next Status | Auto/Manual | Updated By |
|
||||
|----------------|----------------|-------------|-------------|------------|
|
||||
| `new` | AI generate_ideas runs | `mapped` | 🤖 Auto | `generate_ideas.py` (new code) |
|
||||
| `new` | User manually creates ideas | `mapped` | 👤 Manual | User workflow |
|
||||
| `new` | User toggles disabled | *(stays new)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| `mapped` | User toggles disabled | *(stays mapped)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| *(any)* | User re-enables | *(stays same)* + `disabled=false` | 👤 Manual | User form edit |
|
||||
|
||||
**Workflow Path:**
|
||||
```
|
||||
new ──[generate_ideas AI]──> mapped
|
||||
│
|
||||
└──[user toggle disabled]──> (status unchanged, disabled flag set)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## IDEAS Status Transitions
|
||||
|
||||
| Current Status | Trigger/Action | Next Status | Auto/Manual | Updated By |
|
||||
|----------------|----------------|-------------|-------------|------------|
|
||||
| `new` | User bulk-queues ideas to writer | `queued` | 👤 Manual | `views.py` Line 1084 |
|
||||
| `queued` | Writer AI generates content | `completed` | 🤖 Auto | `generate_content.py` Line 318 (syncs from Task) |
|
||||
| `queued` | Task status becomes 'completed' | `completed` | 🤖 Auto | Auto-sync from Task.status |
|
||||
| `new` | User toggles disabled | *(stays new)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| `queued` | User toggles disabled | *(stays queued)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| `completed` | User toggles disabled | *(stays completed)* + `disabled=true` | 👤 Manual | User form edit |
|
||||
| *(any)* | User re-enables | *(stays same)* + `disabled=false` | 👤 Manual | User form edit |
|
||||
|
||||
**Workflow Path:**
|
||||
```
|
||||
new ──[user bulk_queue]──> queued ──[generate_content AI]──> completed
|
||||
│ │ │
|
||||
└──[user toggle disabled]────┴───────────[user toggle]──────────┘
|
||||
(status unchanged, disabled flag set)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
# 📊 COMBINED STATUS OVERVIEW
|
||||
|
||||
## All Modules - Status Values
|
||||
|
||||
| Module | Workflow Statuses | Filter Status | Total Unique Values |
|
||||
|--------|-------------------|---------------|---------------------|
|
||||
| **Keywords** | `new`, `mapped` | `disabled` (boolean) | 2 status + 1 flag |
|
||||
| **Clusters** | `new`, `mapped` | `disabled` (boolean) | 2 status + 1 flag |
|
||||
| **Ideas** | `new`, `queued`, `completed` | `disabled` (boolean) | 3 status + 1 flag |
|
||||
|
||||
---
|
||||
|
||||
## Process Inclusion Matrix
|
||||
|
||||
| Status | Auto-Cluster AI | Generate-Ideas AI | Bulk-Queue | Generate-Content AI | Dashboard Metrics |
|
||||
|--------|-----------------|-------------------|------------|---------------------|-------------------|
|
||||
| Keywords: `new` | ✅ Included | ❌ N/A | ❌ N/A | ❌ N/A | ✅ Counted |
|
||||
| Keywords: `mapped` | ❌ Already clustered | ✅ Used for ideas | ❌ N/A | ❌ N/A | ✅ Counted |
|
||||
| Keywords: `disabled=true` | ❌ Excluded | ❌ Excluded | ❌ N/A | ❌ N/A | ❌ Excluded |
|
||||
| Clusters: `new` | ❌ N/A | ✅ Included | ❌ N/A | ❌ N/A | ✅ Counted |
|
||||
| Clusters: `mapped` | ❌ N/A | ❌ Already has ideas | ✅ Ideas can queue | ❌ N/A | ✅ Counted |
|
||||
| Clusters: `disabled=true` | ❌ N/A | ❌ Excluded | ❌ Excluded | ❌ N/A | ❌ Excluded |
|
||||
| Ideas: `new` | ❌ N/A | ❌ N/A | ✅ Included | ❌ N/A | ✅ Counted |
|
||||
| Ideas: `queued` | ❌ N/A | ❌ N/A | ❌ Already queued | ✅ Included | ✅ Counted |
|
||||
| Ideas: `completed` | ❌ N/A | ❌ N/A | ❌ Already done | ❌ Already done | ✅ Counted |
|
||||
| Ideas: `disabled=true` | ❌ N/A | ❌ N/A | ❌ Excluded | ❌ Excluded | ❌ Excluded |
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
# 🔄 DATA MIGRATION TRANSITION TABLE
|
||||
|
||||
## Keywords Migration
|
||||
|
||||
| Old Status | New Status | New Disabled Flag | Logic |
|
||||
|------------|------------|-------------------|-------|
|
||||
| `pending` | `new` | `false` | Direct mapping |
|
||||
| `active` | `mapped` | `false` | Keyword was clustered |
|
||||
| `archived` | `mapped` | `true` | Preserve data, mark as disabled |
|
||||
|
||||
**SQL Preview:**
|
||||
```sql
|
||||
-- Keywords migration
|
||||
UPDATE igny8_keywords
|
||||
SET status = 'new', disabled = false
|
||||
WHERE status = 'pending';
|
||||
|
||||
UPDATE igny8_keywords
|
||||
SET status = 'mapped', disabled = false
|
||||
WHERE status = 'active';
|
||||
|
||||
UPDATE igny8_keywords
|
||||
SET status = 'mapped', disabled = true
|
||||
WHERE status = 'archived';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Clusters Migration
|
||||
|
||||
| Old Status | New Status | New Disabled Flag | Logic |
|
||||
|------------|------------|-------------------|-------|
|
||||
| `active` (with ideas_count > 0) | `mapped` | `false` | Cluster has ideas |
|
||||
| `active` (with ideas_count = 0) | `new` | `false` | Cluster has no ideas yet |
|
||||
| *(no archived status exists)* | - | - | - |
|
||||
|
||||
**SQL Preview:**
|
||||
```sql
|
||||
-- Clusters migration
|
||||
UPDATE igny8_clusters
|
||||
SET status = 'mapped', disabled = false
|
||||
WHERE ideas_count > 0;
|
||||
|
||||
UPDATE igny8_clusters
|
||||
SET status = 'new', disabled = false
|
||||
WHERE ideas_count = 0;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Ideas Migration
|
||||
|
||||
| Old Status | New Status | New Disabled Flag | Logic |
|
||||
|------------|------------|-------------------|-------|
|
||||
| `new` | `new` | `false` | No change |
|
||||
| `scheduled` | `queued` | `false` | Rename to match writer workflow |
|
||||
| `published` | `completed` | `false` | Publishing is separate deployment |
|
||||
| *(no archived status)* | - | - | - |
|
||||
|
||||
**SQL Preview:**
|
||||
```sql
|
||||
-- Ideas migration
|
||||
UPDATE igny8_content_ideas
|
||||
SET status = 'new', disabled = false
|
||||
WHERE status = 'new';
|
||||
|
||||
UPDATE igny8_content_ideas
|
||||
SET status = 'queued', disabled = false
|
||||
WHERE status = 'scheduled';
|
||||
|
||||
UPDATE igny8_content_ideas
|
||||
SET status = 'completed', disabled = false
|
||||
WHERE status = 'published';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
# 📋 SUMMARY COMPARISON TABLE
|
||||
|
||||
## Before vs After
|
||||
|
||||
| Module | Current Status Values | New Status Values | Current Filter | New Filter |
|
||||
|--------|----------------------|-------------------|----------------|------------|
|
||||
| **Keywords** | `pending`, `active`, `archived` | `new`, `mapped` | Status dropdown | Status dropdown + disabled checkbox |
|
||||
| **Clusters** | `active` (hardcoded, no choices) | `new`, `mapped` | Status dropdown | Status dropdown + disabled checkbox |
|
||||
| **Ideas** | `new`, `scheduled`, `published` | `new`, `queued`, `completed` | Status dropdown | Status dropdown + disabled checkbox |
|
||||
|
||||
---
|
||||
|
||||
## Status Count Changes
|
||||
|
||||
| Module | Before | After | Change |
|
||||
|--------|--------|-------|--------|
|
||||
| **Keywords** | 3 status values | 2 workflow + 1 boolean flag | -1 status (simplified) |
|
||||
| **Clusters** | 1 hardcoded value | 2 workflow + 1 boolean flag | +1 status (proper choices) |
|
||||
| **Ideas** | 3 status values | 3 workflow + 1 boolean flag | Same count (renamed) |
|
||||
|
||||
---
|
||||
|
||||
**END OF IMPLEMENTATION TABLES**
|
||||
@@ -1,658 +0,0 @@
|
||||
bluesalman: https://app.igny8.com/setup/add-keywords
|
||||
|
||||
when keywrods added to wrokflow from this page, the stus in planner/keywrods for the keywrods is saved as pending, which is wrong as per new plan,
|
||||
please check and fix it so that newly added seed keywords in planner kkeywrod s will ahve sttus |New| instaead of pending
|
||||
|
||||
GitHub Copilot: I'll search the backend for the endpoint and code handling adding keywords, then locate where status is set to "pending" and change it to "New". I'll first add a short todo plan.
|
||||
|
||||
Created 5 todos
|
||||
|
||||
Now I'll search the backend for route/view names and keywords insertion code. I'll look for "add-keywords", "add_keywords", "seed keyword", and "planner" references.
|
||||
|
||||
Searched for regex `add-keywords|add_keywords|add_keyword|setup/add-keywords|seed keyword|seed_keywords|planner/keywords|planner keywords|save.*status.*pending` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for regex `SeedKeyword|seedkeyword|seed_keywords|SeedKeywords|Seed Keyword|SeedKeywordStatus|status='pending'` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for regex `setup/add-keywords|add_keywords|setup.views|def add_keywords|/setup/add-keywords` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for files matching `**/backend/**/setup*`, no matches
|
||||
|
||||
Searched for regex `'pending'|"pending"|Pending` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for files matching `**/backend/igny8_core/business/planning`, no matches
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/business)
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/business/planning)
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/business/planning/models.py)
|
||||
|
||||
Searched for regex `Keywords(objects|.create|.bulk_create|Keywords\()|add_keyword(s)?|seed_keyword.*create|save\(.*seed_keyword` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for regex `add-keywords|/setup/add-keywords|setup/add-keywords|addKeywords|add_seed` (`**/frontend/**`), 6 results
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/App.tsx#330-330), lines 330 to 410
|
||||
|
||||
Searched for text `IndustriesSectorsKeywords` (`**/frontend/src/**`), 5 results
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/pages/Setup/IndustriesSectorsKeywords.tsx#1-1), lines 1 to 400
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/pages/Setup/IndustriesSectorsKeywords.tsx#400-400), lines 400 to 800
|
||||
|
||||
Searched for text `addSeedKeywordsToWorkflow` (`**/backend/**|frontend/**`), no results
|
||||
|
||||
Searched for regex `addSeedKeywords|addSeedKeywordsToWorkflow|add_seed_keywords_to_workflow` (`**/frontend/**`), 6 results
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/services/api.ts#1880-1880), lines 1880 to 1935
|
||||
|
||||
Searched for text `bulk_add_from_seed|bulk_add_from_seed|bulk_add_from_seed` (`**/backend/**`), 3 results
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/modules/planner/views.py#1-1), lines 1 to 420
|
||||
|
||||
I'll change the default status from 'pending' to 'new' in the planner bulk add view, then update the todo list to mark steps done. Applying the patch now.
|
||||
|
||||
Using "Apply Patch"
|
||||
|
||||
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
|
||||
Completed: *Find add-keywords handler* (1/5)
|
||||
|
||||
Update complete — quick summary and next steps
|
||||
|
||||
- **What I changed**: In views.py I updated the `bulk_add_from_seed` action so newly created `Keywords` use `'status': 'new'` instead of `'pending'`.
|
||||
- **Why**: The `Keywords` model defines allowed statuses with `'new'` representing newly added seed keywords; setting `'pending'` was inconsistent with the new plan and caused incorrect state.
|
||||
- **Files modified**:
|
||||
- views.py — changed default `status` in `get_or_create(..., defaults={...})`.
|
||||
|
||||
Next recommended steps
|
||||
- Run the backend test suite or at least exercise the endpoint manually:
|
||||
- Example curl (replace token/site/sector IDs as appropriate):
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8000/api/v1/planner/keywords/bulk_add_from_seed/' \
|
||||
-H "Authorization: Token $AUTH_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"seed_keyword_ids":[1,2,3],"site_id":5,"sector_id":12}'
|
||||
```
|
||||
- Verify in the UI by adding keywords from `/setup/add-keywords` and checking that the corresponding records in Planner (`/planner/keywords`) have status `New`.
|
||||
- If you want, I can run the test suite or add a small unit test to assert the default status is `'new'`. Which would you prefer?
|
||||
|
||||
Made changes.
|
||||
|
||||
bluesalman: cehck agian, if wahtever you did is corrrect and logically true, since sitll new keywrods added have status pedning
|
||||
|
||||
GitHub Copilot: I'll search the backend for any other places that set a new Keyword's status to 'pending', then inspect the planner serializer to see default creation behavior. I'll run a code search across the backend.
|
||||
|
||||
Searched for regex `'pending'|"status': 'pending'|"status": "pending"|defaults={\s*'status': 'pending'` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for regex `Keywords.objects.get_or_create|Keywords.objects.create|create\(.*Keywords` (`**/backend/**`), 9 results
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/modules/planner/views.py#520-520), lines 520 to 640
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/api/tests/test_integration_planner.py)
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/modules/planner/management/commands/add_keywords_to_sectors.py)
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/modules/planner/serializers.py)
|
||||
|
||||
Searched for text `planner/keywords|KeywordOpportunities|bulk_update_keywords_status|bulkUpdateKeywordsStatus|status':'pending'` (`**/frontend/**`), 20 results
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/pages/Planner/Keywords.tsx#1-1), lines 1 to 400
|
||||
|
||||
Searched for text `createKeyword(` (`**/frontend/**`), 3 results
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/services/api.ts#480-480), lines 480 to 580
|
||||
|
||||
I'm about to patch the `bulk_add_from_seed` action to enforce `status='new'` after creation (safeguard and log), so we can be certain newly created keywords are marked correctly. I'll apply a small, focused change.
|
||||
|
||||
Using "Apply Patch"
|
||||
|
||||
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
|
||||
Starting: *Run tests or basic sanity checks* (4/5)
|
||||
|
||||
Progress update — what's done and next steps
|
||||
|
||||
- I added a safeguard in `bulk_add_from_seed` to explicitly set `keyword.status = 'new'` and save it when a Keyword is created. This ensures newly created keywords cannot remain `'pending'` due to DB or other unexpected defaults.
|
||||
- Files changed:
|
||||
- views.py (updated default in `get_or_create` and added explicit status enforcement)
|
||||
|
||||
Next actions I suggest (choose one):
|
||||
- I can run the backend test suite or run a quick Django shell command to simulate the `bulk_add_from_seed` call and verify the created Keywords have `status='new'`. (This requires a running dev environment; tell me if you want me to run tests locally.)
|
||||
- Or you can try adding keywords from `/setup/add-keywords` now and confirm — if still pending, provide a sample request/response or logs and I'll investigate further.
|
||||
|
||||
If you want me to run verification here, I can:
|
||||
- Run a targeted unit test or a Django management shell snippet to call `bulk_add_from_seed` and print the created Keyword statuses.
|
||||
Tell me which verification you prefer and I'll proceed.
|
||||
|
||||
Made changes.
|
||||
|
||||
bluesalman: \Planner\Keywords.tsx
|
||||
|
||||
cehck this if somethign is overrididng it on th page wiwh tpending status
|
||||
|
||||
GitHub Copilot: I'll add an updated TODO, search the frontend for any hardcoded `pending` defaults, then patch the `formData` initial `status` in Keywords.tsx to `'new'`.
|
||||
|
||||
Added 2 todos
|
||||
|
||||
Searched for regex `status: 'pending'|status: "pending"|'status': 'pending'|"status": "pending"` (`**/frontend/**`), 20 results
|
||||
|
||||
Read [](file:///data/app/igny8/frontend/src/pages/Planner/Keywords.tsx#70-70), lines 70 to 110
|
||||
|
||||
Using "Apply Patch"
|
||||
|
||||
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
|
||||
Completed: *Inspect frontend Keywords page* (4/7)
|
||||
|
||||
Done — what I checked and changed
|
||||
|
||||
- What I inspected:
|
||||
- Searched the frontend for hardcoded `pending` occurrences and found that Keywords.tsx initialized the create form's `formData.status` as `'pending'`.
|
||||
- Confirmed backend `bulk_add_from_seed` now sets `Keywords` created from seed to `'new'` (we previously updated this).
|
||||
|
||||
- What I changed:
|
||||
- Updated Keywords.tsx initial `formData.status` from `'pending'` to `'new'`.
|
||||
- Also added a safeguard in views.py (in `bulk_add_from_seed`) to explicitly set `keyword.status = 'new'` and save when a keyword is created.
|
||||
|
||||
- Files modified:
|
||||
- views.py — set defaults to `'new'` and enforce status after create.
|
||||
- Keywords.tsx — changed default `formData.status` to `'new'`.
|
||||
|
||||
Why this should fix the issue
|
||||
- The UI create form was explicitly sending `'pending'` when creating keywords via the Planner page; changing the form default to `'new'` prevents that.
|
||||
- The setup flow (`/setup/add-keywords`) uses the bulk-add API which now defaults to `'new'` and enforces it, preventing DB or other code from leaving it as `'pending'`.
|
||||
|
||||
Next steps / verification
|
||||
- To confirm end-to-end, either:
|
||||
- Add seed keywords from the UI (`/setup/add-keywords`) and check `/planner/keywords` — new records should show status `New`.
|
||||
- Or run this curl (replace token/site/sector IDs) to test bulk endpoint:
|
||||
```bash
|
||||
curl -X POST 'http://localhost:8000/api/v1/planner/keywords/bulk_add_from_seed/' \
|
||||
-H "Authorization: Token $AUTH_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"seed_keyword_ids":[1,2,3],"site_id":5,"sector_id":12}'
|
||||
```
|
||||
- Or run a Django shell check:
|
||||
```bash
|
||||
python manage.py shell
|
||||
# then in shell:
|
||||
from igny8_core.modules.planner.views import KeywordViewSet
|
||||
# (or directly create Keywords via ORM to verify defaults)
|
||||
```
|
||||
|
||||
Would you like me to:
|
||||
- Run a quick Django shell snippet here to simulate a `get_or_create` and print the created keyword status? (Requires dev environment / DB access)
|
||||
- Or run the integration tests that touch the planner endpoints? I'll proceed with whichever you prefer.
|
||||
|
||||
Made changes.
|
||||
|
||||
bluesalman: when cluster is delted teh mapeed keywros status which now have no cluster mapped their staus is notupdated,
|
||||
so when soem mapped clsuters are delted it should remove the mapping status and change the status of keywrods back to new,
|
||||
|
||||
similarly th whole plalner and workflwo pipeline works in one idmenstion as cgoing forward from keywrod to clsuter to idea to taks, to contetn to images to review to publish and preovious module statuas changes atumatically
|
||||
simialrrly we need to make sure that when in pipeline someone try to delte a record, it hsold not have any mapped or assocaited record upstream,
|
||||
|
||||
so if user try to delte a clsuter which ahs a mapped ide, or similarly try to delte keywrod and that has mapepd clsuter, it hsouldnott allow to delte,
|
||||
|
||||
this applies to both single record delete as well as bulk delete fatures for all moduel sin planner in writer
|
||||
|
||||
and when some record in any modeul is eligble to be edelted as it has no upstream matching//mapped record, then when delted it should updated the statuas of previous assocation back to orignal sattusu which was before mapping
|
||||
|
||||
this need s to be carefully planned,
|
||||
|
||||
udneratnd the current flow and statusus from docs drerelated douemetns of planner writer, and status mapping tbales docs,
|
||||
also see the code base,
|
||||
|
||||
then plan it and show me the plan of thiis backrward delttion as welll as limits applciatin for edlete in , show me this in chat in table format,
|
||||
|
||||
|
||||
GitHub Copilot: I'll add a small plan (todo list) and then scan the codebase and docs for planner/writer status flows, delete endpoints, and model relationships. I'll collect relevant files to build the deletion plan.
|
||||
|
||||
Starting: *Gather planner/writer docs* (1/5)
|
||||
|
||||
Searched for regex `class Clusters|class Keywords|class ContentIdeas|class Tasks|class Content|class Image|on_delete=|bulk_delete|bulk_delete_from_seed|@action\(|def destroy\(|def delete\(|bulk_delete` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for regex `class Tasks|class Task|class Content|class Image|ContentIdeas|keyword_objects|keyword_cluster|writer` (`**/backend/**`), 20 results
|
||||
|
||||
Now I'll collect the status fields and relationships for relevant models (planner + writer modules) to summarize current flow. I'll search models for status choices and FK relationships.
|
||||
|
||||
Searched for regex `class .*models.py|status = models.CharField|class .*\(|from .*modules.*writer.models|class Tasks|class Content` (`**/backend/**`), 20 results
|
||||
|
||||
Searched for files matching `**/backend/igny8_core/modules/writer/**/models.py`, 1 match
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/modules/writer/models.py)
|
||||
|
||||
Read [](file:///data/app/igny8/backend/igny8_core/business/content/models.py#1-1), lines 1 to 500
|
||||
|
||||
Next I'll outline what I found about the current flow, then propose a safe, testable plan (in table form) to implement the backward-deletion and deletion-limits behavior you described.
|
||||
|
||||
**Quick summary of current state (from code + docs)**
|
||||
- Core pipeline (forward): Keyword -> Cluster -> ContentIdea -> Task -> Content -> Images -> (Review -> Publish).
|
||||
- Relevant models & status fields:
|
||||
- `planner.Keywords` — statuses: `('new','New'), ('mapped','Mapped')` (field `status`, default `'new'`).
|
||||
- `planner.Clusters` — statuses: `('new','New'), ('mapped','Mapped')` (field `status`, default `'new'`).
|
||||
- `planner.ContentIdeas` — statuses: `('new','New'), ('queued','Queued'), ('completed','Completed')`.
|
||||
- `writer.Tasks` — statuses: `('queued','Queued'), ('completed','Completed')`.
|
||||
- `writer.Content` — statuses: `('draft','Draft'), ('review','Review'), ('published','Published')`.
|
||||
- `writer.Images` — status default `'pending'` (values: pending/generated/failed).
|
||||
- Important relationships (from models):
|
||||
- `Keywords.seed_keyword` — FK(PROTECT) to global SeedKeyword (prevents deleting seed keyword).
|
||||
- `Keywords.cluster` — FK(SET_NULL) to `Clusters`. (Deleting a Cluster currently sets `Keywords.cluster = NULL`.)
|
||||
- `ContentIdeas.keyword_objects` — M2M to `Keywords`.
|
||||
- `ContentIdeas.keyword_cluster` — FK(SET_NULL) to `Clusters`.
|
||||
- `Tasks.cluster` — FK(SET_NULL) to `Clusters`.
|
||||
- `Content.cluster` — FK(SET_NULL) to `Clusters`.
|
||||
- `writer.ContentClusterMap.cluster` — FK(CASCADE) to `Clusters` (this will delete mappings when cluster deleted).
|
||||
- Current behavior you observed:
|
||||
- When a `Cluster` is deleted, `Keywords.cluster` becomes NULL (because of SET_NULL) but the `Keywords.status` often remains `'mapped'` (incorrect). You want those Keywords to revert to `'new'`.
|
||||
- You want deletions to be blocked if there are dependent/mapped downstream records (both single and bulk deletes).
|
||||
- You want status rollback to original pre-mapping status where possible, not just a generic `'new'`.
|
||||
|
||||
Plan (high-level) — what we will implement
|
||||
- Enforce deletion guardrails: prevent deleting an entity if downstream mapped records exist (single + bulk).
|
||||
- On allowed deletions, perform controlled rollback updates to upstream/related records (e.g., set Keywords back to `'new'` if their cluster was removed).
|
||||
- Track previous statuses at mapping time so rollbacks can restore the original status instead of guessing.
|
||||
- Add tests and API-level checks to make behavior explicit and safe.
|
||||
|
||||
Action plan (step-by-step) — next work to do
|
||||
- I will implement this plan in increments and add tests. If you want me to start coding, I will.
|
||||
- Below I present a detailed plan table describing each entity, what blocks deletion, and what automatic rollback/update should happen when deletion is allowed.
|
||||
|
||||
**Plan table — Deletion rules & rollback behavior**
|
||||
|
||||
- Columns: **Entity** | **Blocks Deletion If** | **On Delete (allowed)** | **Implementation Notes / Where to change**
|
||||
|
||||
| **Entity** | **Blocks Deletion If** | **On Delete (allowed)** | **Implementation Notes / Where to change** |
|
||||
|-----------|------------------------|-------------------------|-------------------------------------------|
|
||||
| `Cluster` (`planner.Clusters`) | Any of: <br>- `ContentIdeas` with `keyword_cluster = cluster` (idea mapped) <br>- `writer.ContentClusterMap` entries referencing cluster (content mapped) <br>- `writer.Tasks` where `cluster = cluster` (active tasks mapped) <br>(Also consider queued writer jobs referencing cluster) | - If allowed (no blocking downstream): delete cluster. <br>- After deletion, update `planner.Keywords` that referenced this cluster to set `status` → previously-recorded status (fallback `'new'`) and ensure `cluster` is NULL. | - Add `pre_delete` signal or ViewSet checks to block deletion if blocking dependencies exist (return 400). <br>- Add `post_delete` signal to update `Keywords` status for keywords that had `cluster=deleted_pk` (safe because instance.pk available). <br>- Prefer pre_delete check (fail fast) and post_delete update (apply rollback). <br>- Also update any `ContentIdeas` keyword cluster references are already SET_NULL — but we will block deletion if ideas exist. |
|
||||
| `Keyword` (`planner.Keywords`) | Any of: <br>- `keyword.cluster` is not NULL (keyword mapped to a cluster) <br>- `ContentIdeas.keyword_objects` contains this keyword (idea uses keyword) <br>- Any other downstream references (e.g., queued processes referencing keyword) | - If allowed (no blocking references): delete keyword. <br>- On delete: if this keyword was previously mapped (we recorded previous status), restore previous status on other objects that referenced it (rare). | - Add `pre_delete` on Keywords to check M2M `contentideas` and `cluster`. <br>- For UI/bulk delete, ViewSet should call logic that attempts deletion per-keyword and returns details about blocked vs deleted items. |
|
||||
| `ContentIdea` (`planner.ContentIdeas`) | Any of: <br>- `writer.Tasks` referencing the idea (`Tasks.idea`) <br>- `writer.Content` or `ContentClusterMap` referencing the idea indirectly (if used) | - If allowed: delete idea; if idea was the reason a cluster was mapped to something, optionally revert cluster status — but clusters are normally independent. | - Add `pre_delete` to check `Tasks.exists()` and block if exists. <br>- On delete allowed, remove any leftover associations (M2M to keywords) and optionally update `Keywords` mapped status if mapping implied change. |
|
||||
| `Task` (`writer.Tasks`) | Content (final content) or images already generated for that task; or a Task is currently in-progress/queued | - If allowed: delete task; on delete, optionally set statuses on dependent `Images` or `Content` back to previous states if they were generated as result of this task. | - Check `Images` or `Content` referencing the task. Block if final content exists. |
|
||||
| `Content` (`writer.Content`) | If published or external sync state would be impacted; if `ContentClusterMap` or `Images` depend on it | - If allowed: delete content; update `ContentClusterMap` (cascade) and optionally change `cluster` or other upstream statuses if appropriate. | - Block deletion if it's published or external state exists (optional policy). |
|
||||
|
||||
Notes:
|
||||
- “Blocks Deletion If” checks must be applied both for single `DELETE` (ViewSet destroy) and bulk delete endpoints. Bulk deletes should iterate items and return a structured result: {deleted: n, skipped: [(id, reason), ...]}.
|
||||
- Restoration of "original status" requires recording the previous status at the time of the mapping operation (e.g., when a Keyword is mapped to a Cluster we should save prior status). Without that, we must fallback to safe defaults (Keywords -> `'new'`, Clusters -> `'new'`).
|
||||
- There are two layers for enforcement:
|
||||
1. Model-level signals (pre_delete/post_delete) — catches deletes triggered anywhere (shell, admin, bulk queryset.delete). pre_delete allows blocking; post_delete can update related rows.
|
||||
2. API-level ViewSet checks (destroy/bulk_delete override) — gives user-friendly error messages and prevents accidental DB operations. Both should be used together.
|
||||
|
||||
Implementation details / code plan (concrete changes)
|
||||
1. Add status-history (preferred): new DB model and migration
|
||||
- Model: `StatusHistory` (generic) or add `previous_status` JSONField on mapping models:
|
||||
- Fields: `model_name`, `object_id`, `field_name`, `previous_value`, `changed_by`, `changed_at`, `context` (why changed).
|
||||
- On mapping events (e.g., when `Keyword.cluster` is set or ContentIdea.keyword_cluster set), record the prior status in `StatusHistory`.
|
||||
- Benefit: allows precise rollback to "original status".
|
||||
|
||||
2. Signals & checks
|
||||
- `pre_delete` signal for `Clusters`:
|
||||
- Query for blocking dependencies (ContentIdeas, ContentClusterMap, Tasks). If any exist, raise `ProtectedError` or return API error.
|
||||
- `post_delete` signal for `Clusters`:
|
||||
- Update `Keywords.objects.filter(cluster=deleted_pk)` — but cluster FK will already become NULL (SET_NULL) when cluster deleted; use previous pk to find keywords and set `status` back to `previous_value` (from StatusHistory) OR `'new'` fallback.
|
||||
- `pre_delete` for `Keywords`:
|
||||
- Check `contentideas_set.exists()` or `keyword.cluster is not None` and block deletion if needed.
|
||||
- Similar pre_delete/post_delete logic for `ContentIdeas` and `Tasks` as required.
|
||||
|
||||
3. ViewSet changes (API UX)
|
||||
- Override `destroy` in `ClusterViewSet` / `KeywordViewSet` to:
|
||||
- Run dependency check and return 400 with a helpful message if blocked.
|
||||
- Improve `bulk_delete` actions so they iterate and for each record:
|
||||
- Attempt deletion (or perform the same checks) and return a detailed report of skipped items and reasons (do not fail the whole batch unless you prefer that policy).
|
||||
|
||||
4. Tests
|
||||
- Add unit & integration tests verifying:
|
||||
- Deletion blocked when downstream objects exist.
|
||||
- Deletion allowed when no downstream objects; verify keywords statuses changed to recorded previous values (or `'new'` fallback).
|
||||
- Bulk delete returns expected report for mixed batches.
|
||||
|
||||
5. Data migration (if implementing StatusHistory)
|
||||
- Optionally scan existing mapped keywords/clusters and set `previous_status` for future rollbacks (best-effort).
|
||||
- Communicate that existing historical pre-mapping states cannot be reconstructed and will default to safe values.
|
||||
|
||||
Example pseudocode for cluster pre/post delete signals
|
||||
- pre_delete (Cluster):
|
||||
- if ContentIdeas.objects.filter(keyword_cluster=cluster).exists(): raise ProtectedError("Cluster has mapped ideas")
|
||||
- if ContentClusterMap.objects.filter(cluster=cluster).exists(): raise ProtectedError("Cluster has mapped contents")
|
||||
- if Tasks.objects.filter(cluster=cluster).exists(): raise ProtectedError("Cluster has mapped tasks")
|
||||
- post_delete (Cluster):
|
||||
- keywords_to_fix = Keywords.objects.filter(cluster__isnull=True, previous_cluster_id=deleted_pk) OR Keywords.objects.filter(previous_cluster_id=deleted_pk) # or use stored history
|
||||
- For each keyword: set status = previous_status if present else 'new'; save()
|
||||
|
||||
Bulk delete endpoint pattern (safe)
|
||||
- For each id in request.data['ids']:
|
||||
- Try: run same pre-delete checks; if fail, append to skipped with reason
|
||||
- Else: perform instance.delete(); append to deleted list
|
||||
- Return success_response({deleted_count, skipped: [{id, reason}]})
|
||||
|
||||
Risk & considerations
|
||||
- Race conditions: concurrent operations could create race conditions; use DB transactions and row-locking for critical checks if necessary.
|
||||
- Legacy data: without historical statuses you cannot perfectly restore "previous status" — plan needs a small migration & fallback policy.
|
||||
- UX: blocking deletions can confuse users; consider adding a `force=true` option that will delete and also cascade/clear downstream (with warnings).
|
||||
- Performance: bulk-delete checks must be optimized (use `.exists()` queries and `prefetch_related`).
|
||||
|
||||
Estimated implementation steps & time (rough)
|
||||
- Phase 1 (1–2 days): Add model-level `pre_delete` checks and ViewSet-level guards for `Clusters` and `Keywords`; add post_delete handler to set `Keywords.status='new'` when their cluster was removed (fallback behavior). Add unit tests.
|
||||
- Phase 2 (1–2 days): Add StatusHistory model and update mapping operations to record previous statuses. Extend post_delete logic to restore recorded previous states. Add migration + tests.
|
||||
- Phase 3 (0.5–1 day): Apply same pattern across other modules (Ideas -> Tasks -> Content) with fine-grained policies. Add integration tests for bulk ops and UI acceptance tests.
|
||||
- Phase 4 (0.5 day): Documentation update and release notes.
|
||||
|
||||
Concrete next steps I can take now (pick any):
|
||||
- A: Implement Phase 1 now: add pre_delete + post_delete for `Clusters` & `Keywords` and update `bulk_delete` endpoints to return detailed reports. I will write tests for cluster delete behavior. (I can start coding immediately.)
|
||||
- B: Draft the exact schema for `StatusHistory` and a migration; then implement mapping-time recording hooks (this is needed to fully restore previous statuses).
|
||||
- C: Implement API-level UX: `bulk_delete` returns breakdown of deleted/skipped and reasons for all planner & writer modules.
|
||||
|
||||
Which step do you want me to implement first? If you want me to proceed, I’ll:
|
||||
- start with Phase 1 (safe changes + tests), or
|
||||
- prepare the `StatusHistory` schema and migration (Phase 2).
|
||||
|
||||
If you prefer, I can also open a PR-style patch and run the relevant unit tests locally — tell me to proceed and whether to start with Phase 1 (fast safety fix) or Phase 2 (longer, reversible solution).
|
||||
|
||||
bluesalman: ia sked for oa dcouement, with only safety, of record cano nt be delted if it has currently associated/mapped record upstream,
|
||||
that is enoigh no need to make a huge backup database, we already have daily backupds for that,
|
||||
so here only delte eleigbility is enough, which is no updstream recor dperesent, if ther eus it gives error on deletion, with mesage you cancno tdelted if associated, name of the upstream module and record titel show, and it says delte teh otehr xyz in xyz moduel record firest to delte this,
|
||||
|
||||
and 2nd thing ia sked is udpating the downstreem moduel stusu when an upstream record is delted
|
||||
|
||||
both cn be tracked easily through curent sttau sof record, we no all teh moduels defautl new stuas , if it didfertn then we need to hecek upstream and giv ethis rroero,
|
||||
adn when delted we just newewd to set the previous down stream moduel to whatever is the fresh new reacord status is,
|
||||
|
||||
simple very simple
|
||||
|
||||
|
||||
|
||||
GitHub Copilot: I'll add a short todo list for implementing the safety-only deletion policy and then present the requested document with rules, error messages, and implementation notes.
|
||||
|
||||
Completed: *Draft safety deletion policy doc* (1/6)
|
||||
|
||||
**Safety-Only Deletion Policy & Implementation Plan**
|
||||
|
||||
**Scope:**
|
||||
- Minimal, safe behavior only: block deletes when there are currently associated/mapped upstream records. Provide clear error message telling the user which upstream records block deletion and what to remove first.
|
||||
- When deletion is allowed (no upstream associations), perform the delete and set downstream (child) records' statuses to the module default (e.g., `Keywords` → `'new'`) — simple fallback, no full status history tracking.
|
||||
- Applies to single-record `DELETE` and API `bulk_delete` actions across planner and writer modules.
|
||||
|
||||
**Assumptions:**
|
||||
- Existing daily DB backups suffice; no historical status audit required here.
|
||||
- Default “new” statuses are known from models (planner/writer models). Use those defaults for rollback.
|
||||
- We implement checks at API level (ViewSets) — prevents user-driven deletes via UI/API and provides friendly messages. (Optional: add model `pre_delete` protections later.)
|
||||
|
||||
**Quick Implementation Summary (one-liner):**
|
||||
- Add a reusable helper that inspects upstream dependencies, returns blocking info; call it from `destroy` and `bulk_delete` for each module; if allowed, delete and update downstream statuses to default.
|
||||
|
||||
**Table: Entities, Block Conditions, Error Message, On-Delete Behavior, Implementation Location**
|
||||
|
||||
| Entity | Blocks Deletion If (upstream exists) | Example Error Message (returned to client) | On Delete (allowed) — downstream updates | Implementation Location |
|
||||
|--------|--------------------------------------|---------------------------------------------|------------------------------------------|-------------------------|
|
||||
| `Cluster` (`planner.Clusters`) | Any mapped: `ContentIdeas` with `keyword_cluster`, `writer.ContentClusterMap` entries, `writer.Tasks` where `cluster` set, or `writer.Content` referencing cluster | "Cannot delete Cluster 'NAME' — mapped records exist in: ContentIdeas: 'Idea title', Tasks: 'Task title'. Delete those first." | Keywords whose `cluster` becomes NULL should have `status` set to `'new'` (model default) after cluster deletion. | `ClusterViewSet.destroy` + `ClusterViewSet.bulk_delete` (in views.py). Use helper in shared utils. |
|
||||
| `Keyword` (`planner.Keywords`) | Mapped cluster (keyword.cluster != NULL) OR `ContentIdeas` (M2M) referencing the keyword | "Cannot delete Keyword 'TERM' — mapped to Cluster 'CLUSTER_NAME' and used in Idea 'IDEA_TITLE'. Delete those first." | N/A (deleting keyword only affects fewer downstream things); if allowed, remove M2M references in `ContentIdeas`. | `KeywordViewSet.destroy` + `bulk_delete` (in planner views). |
|
||||
| `ContentIdea` (`planner.ContentIdeas`) | `writer.Tasks` with `idea` set, or `Content` that used this idea during generation | "Cannot delete Idea 'TITLE' — tasks exist: 'Task title'. Delete those tasks first." | If allowed, clear M2M relations to `Keywords` (no status change needed), update any clusters if appropriate to stay consistent (usually no change). | `ContentIdeasViewSet.destroy` + `bulk_delete` (planner views). |
|
||||
| `Task` (`writer.Tasks`) | `writer.Content` generated from this task (`content` referencing it) or `Images` attached to it | "Cannot delete Task 'TITLE' — content exists: 'Content title'. Delete that content first." | If allowed, cascade-delete or set images' `task` to NULL (model currently CASCADE); adopt policy: block if `Content` exists. | `TaskViewSet.destroy` + `bulk_delete` (writer views). |
|
||||
| `Content` (`writer.Content`) | Published content or `ContentClusterMap`/`Images` referencing it (depending on policy) | "Cannot delete Content 'TITLE' — images or content mappings exist. Delete images/mappings first." | If allowed, `ContentClusterMap` entries cascade; update any upstream mapping flags if needed (e.g., cluster mapped counts). | `ContentViewSet.destroy` + `bulk_delete` (writer views). |
|
||||
|
||||
**API behavior / Error format**
|
||||
- Single delete (DELETE /{module}/{id}/): return HTTP 400 with JSON:
|
||||
- { "success": false, "error": "Cannot delete X: mapped records present", "details": [ { "module": "ContentIdeas", "id": 123, "title": "Idea title" }, ... ], "message": "Delete the listed records in module Y before deleting this." }
|
||||
- Bulk delete (POST /.../bulk_delete/): iterate IDs and return per-id result:
|
||||
- { "deleted": [1,2], "skipped": [ { "id": 3, "reason": "Mapped records exist", "details": [...] }, ... ] }
|
||||
|
||||
**Simple Helper (design, pseudo-code)**
|
||||
|
||||
- Add a small helper util `check_delete_blockers(instance)` that:
|
||||
- Accepts a model instance (Cluster/Keyword/Idea/Task/Content).
|
||||
- Runs fast `.exists()` queries for known upstream relationships.
|
||||
- Returns list of blocking tuples: [(module_name, id, title_field), ...] or empty list if safe.
|
||||
- Implement in `igny8_core.modules.planner.utils` or `igny8_core.api.utils` for reuse.
|
||||
|
||||
Example (concise Python-like pseudocode):
|
||||
```python
|
||||
def check_delete_blockers_cluster(cluster):
|
||||
blockers = []
|
||||
# Ideas
|
||||
qs = ContentIdeas.objects.filter(keyword_cluster=cluster).values('id','idea_title')[:5]
|
||||
blockers += [('ContentIdeas', r['id'], r['idea_title']) for r in qs]
|
||||
# Writer mappings
|
||||
from igny8_core.business.content.models import ContentClusterMap, Tasks, Content
|
||||
if ContentClusterMap.objects.filter(cluster=cluster).exists():
|
||||
cm = ContentClusterMap.objects.filter(cluster=cluster).values('id','content_id')[:5]
|
||||
# map to titles if possible
|
||||
if Tasks.objects.filter(cluster=cluster).exists():
|
||||
t = Tasks.objects.filter(cluster=cluster).values('id','title')[:5]
|
||||
blockers += [('Tasks', r['id'], r['title']) for r in t]
|
||||
if Content.objects.filter(cluster=cluster).exists():
|
||||
c = Content.objects.filter(cluster=cluster).values('id','title')[:5]
|
||||
blockers += [('Content', r['id'], r['title']) for r in c]
|
||||
return blockers
|
||||
```
|
||||
|
||||
**Using the helper in ViewSets**
|
||||
- In `destroy(self, request, *args, **kwargs)`:
|
||||
- instance = self.get_object()
|
||||
- blockers = check_delete_blockers_{model}(instance)
|
||||
- if blockers: return error_response(...) with details and message
|
||||
- else: proceed with `instance.delete()` and after delete run downstream simple updates:
|
||||
- e.g., for Cluster: Keywords.objects.filter(cluster=instance).update(status='new', cluster=None)
|
||||
- (Because cluster FK uses `SET_NULL` in models, you might need to find keywords that had cluster==pk before deletion. Best approach: perform updates before deletion: Keywords.objects.filter(cluster=instance).update(cluster=None, status='new') then delete cluster to avoid race.)
|
||||
|
||||
**Bulk delete approach**
|
||||
- For each id:
|
||||
- Try to fetch object
|
||||
- Run the same `check_delete_blockers`
|
||||
- If blockers -> add to `skipped` with reasons
|
||||
- Else -> perform controlled update of downstream fields (set cluster->NULL, status->default) then delete, add to `deleted`
|
||||
- Return both lists; HTTP 200.
|
||||
|
||||
**Downstream status update (simple):**
|
||||
- Before deleting an upstream record, find child records and set their status to default (from model definitions). Examples:
|
||||
- When deleting `Cluster`:
|
||||
- Keywords: `Keywords.objects.filter(cluster=cluster).update(cluster=None, status='new')`
|
||||
- ContentIdeas: if cluster deletion allowed, they shouldn't exist (we block), but if model had SET_NULL, ensure consistent state.
|
||||
- When deleting `Keyword`:
|
||||
- Remove from `ContentIdeas.keyword_objects` M2M (or block if used in ideas per policy).
|
||||
- Important: run these updates inside a DB transaction so partial states do not occur.
|
||||
|
||||
**User-friendly message examples**
|
||||
- Single delete error:
|
||||
- "Cannot delete Cluster 'Mobile Apps' — mapped records exist: ContentIdeas: 'How to build mobile apps' (ID 412), Tasks: 'Write mobile apps guide' (ID 776). Delete those first."
|
||||
- Bulk delete report:
|
||||
- { "deleted": [5,7], "skipped": [ {"id":3,"reason":"Mapped records exist","details":[{"module":"ContentIdeas","id":412,"title":"How to build mobile apps"}]} ] }
|
||||
|
||||
**Testing to add (minimal)**
|
||||
- Unit tests:
|
||||
- Attempt to delete a `Cluster` with a `ContentIdea` pointing to it -> assert 400 and error contains idea title.
|
||||
- Delete a `Cluster` with no upstream -> perform deletion and assert `Keywords` that referenced cluster now have `status == 'new'` and `cluster == None`.
|
||||
- Bulk-delete mix of deletable and blocked items -> assert correct `deleted`/`skipped` lists.
|
||||
- Integration tests for planner → writer interactions:
|
||||
- Create cluster → create content mapping → attempt cluster delete -> assert blocked.
|
||||
|
||||
**Developer notes & safe rollout**
|
||||
- Implement API-level checks first (fast). This prevents UI/API deletes and gives friendly messages. Tests added before merging. No DB schema changes needed.
|
||||
- Optional later improvement: add a small `StatusHistory` or `previous_status` if precise restoration ever required.
|
||||
- For backwards compatibility: existing code that uses `.delete()` outside viewsets (e.g., admin or scripts) won't get the friendly error unless we also add `pre_delete` model handlers — recommend adding `pre_delete` signals after the API behavior is in place.
|
||||
|
||||
**Minimal code snippet to return an error in a ViewSet (DRF style)**
|
||||
```python
|
||||
from igny8_core.api.response import error_response, success_response
|
||||
|
||||
def destroy(self, request, *args, **kwargs):
|
||||
instance = self.get_object()
|
||||
blockers = check_delete_blockers_cluster(instance)
|
||||
if blockers:
|
||||
details = [{'module': m, 'id': id, 'title': title} for m,id,title in blockers]
|
||||
return error_response(
|
||||
error=f"Cannot delete Cluster '{instance.name}' - mapped records exist",
|
||||
request=request,
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
data={'details': details, 'message': 'Delete the listed records first'}
|
||||
)
|
||||
# Safe path: update child records then delete inside transaction
|
||||
with transaction.atomic():
|
||||
Keywords.objects.filter(cluster=instance).update(cluster=None, status='new')
|
||||
instance.delete()
|
||||
return success_response(data={'deleted_id': kwargs.get('pk')}, request=request)
|
||||
```
|
||||
|
||||
**Next steps (recommended immediate actions)**
|
||||
- I can implement Phase 1 changes (API-level checks + bulk_delete reporting + tests) now. This is low-risk and requires no migrations.
|
||||
- If you want, I will:
|
||||
- Implement helper utils, update `ClusterViewSet.destroy` and `bulk_delete` and `KeywordViewSet` similarly, and add unit tests for the blocking behavior.
|
||||
- Run tests locally and show results.
|
||||
|
||||
Which immediate action do you want me to take now:
|
||||
- A) Implement Phase 1 (API-level checks + bulk delete + tests), or
|
||||
- B) Only produce the final doc (above) for handoff and stop here?
|
||||
|
||||
bluesalman: only make th edocument
|
||||
|
||||
GitHub Copilot: **Safety Deletion Policy — Backward-Deletion Rules & Implementation Doc**
|
||||
|
||||
**Purpose:**
|
||||
- Define a minimal, safe deletion policy: forbid deletes when there are currently associated/mapped upstream records; provide a clear error message listing blocking upstream records and how to resolve; when a deletion is permitted, update downstream statuses to the module default (no historical audit).
|
||||
- This is an API-level safety document only — no DB audit/history required.
|
||||
|
||||
**Scope:**
|
||||
- Planner + Writer modules (primary entities): `Clusters`, `Keywords`, `ContentIdeas`, `Tasks`, `Content`, `Images`.
|
||||
- Applies to single-record `DELETE` and API `bulk_delete` actions.
|
||||
- Behavior must be consistent and user-friendly (clear messages).
|
||||
|
||||
**High-level Rules (short):**
|
||||
- Block delete if any upstream/mapped record exists. Return HTTP 400 with structured details (module, id, title) and an instruction to delete upstream records first.
|
||||
- For an allowed delete: in a DB transaction, update affected downstream child records to model defaults (e.g., `Keywords.status = 'new'`) then delete the record.
|
||||
- Bulk delete: iterate IDs, perform same checks per item, do not abort whole batch. Return per-id results (deleted/skipped with reason).
|
||||
|
||||
**Entity Rules Table**
|
||||
|
||||
| Entity | Blocks Deletion If (upstream exists) | Error Message (client) | On Delete (allowed) — downstream updates | Implementation Notes |
|
||||
|---|---:|---|---|---|
|
||||
| Cluster (`planner.Clusters`) | Any mapped items: `ContentIdeas` (keyword_cluster), `writer.ContentClusterMap`, `writer.Tasks` with `cluster`, `writer.Content` referencing cluster | "Cannot delete Cluster 'NAME' — mapped records exist in: ContentIdeas: 'Idea title' (ID). Delete those first." | Before delete: set `Keywords` referencing cluster → `cluster = NULL` and `status = 'new'` (model default). Then delete cluster. | Implement in `ClusterViewSet.destroy` and `bulk_delete`. Use helper `check_delete_blockers(cluster)`. Run updates inside a `transaction.atomic()` and perform child updates before `instance.delete()` to avoid orphan visibility. |
|
||||
| Keyword (`planner.Keywords`) | `keyword.cluster != NULL` OR M2M `ContentIdeas` referencing this keyword | "Cannot delete Keyword 'TERM' — mapped to Cluster 'CLUSTER' and used in Idea 'IDEA_TITLE'. Delete those first." | If allowed, remove M2M references (orphan removal). No status rollback needed for upstream. | Implement in `KeywordViewSet.destroy` + `bulk_delete`. For M2M, block deletion unless explicit policy allows cascade. |
|
||||
| ContentIdea (`planner.ContentIdeas`) | `writer.Tasks` where `idea` set; `Content` generated from idea | "Cannot delete Idea 'TITLE' — tasks exist: 'Task title'. Delete those tasks first." | If allowed, clear M2M `keyword_objects`. | Implement in `ContentIdeasViewSet`. Block if `Tasks.exists()`; otherwise delete inside transaction. |
|
||||
| Task (`writer.Tasks`) | Exists `Content` generated from the task OR final images attached | "Cannot delete Task 'TITLE' — content exists: 'Content title'. Delete that content first." | If allowed, delete or clear `Images` per policy (prefer blocking if final content exists). | Implement in writer `TaskViewSet`. Prefer strict block if `Content` exists. |
|
||||
| Content (`writer.Content`) | Published content or `ContentClusterMap`/`Images` referencing it | "Cannot delete Content 'TITLE' — images or mappings exist. Delete them first." | If allowed, delete `ContentClusterMap` entries (they `CASCADE`) and `Images` as per current model behavior. Update cluster or counters if needed. | Implement in writer `ContentViewSet`. Consider blocking published content deletion by policy. |
|
||||
| Images (`writer.Images`) | Typically safe to delete unless other policies apply (e.g., external sync) | "Cannot delete Image 'ID' — blocked by external sync." | If allowed, delete image record. | Implement in `ImagesViewSet` if present; otherwise admin allowed. |
|
||||
|
||||
**API Response Examples**
|
||||
|
||||
- Single-delete error (HTTP 400):
|
||||
{
|
||||
"success": false,
|
||||
"error": "Cannot delete Cluster 'Mobile Apps' — mapped records exist",
|
||||
"details": [
|
||||
{"module": "ContentIdeas", "id": 412, "title": "How to build mobile apps"},
|
||||
{"module": "Tasks", "id": 776, "title": "Write mobile apps guide"}
|
||||
],
|
||||
"message": "Delete the listed records in their modules first, then retry."
|
||||
}
|
||||
|
||||
- Bulk delete response (HTTP 200):
|
||||
{
|
||||
"deleted": [5,7],
|
||||
"skipped": [
|
||||
{"id":3,"reason":"Mapped records exist","details":[{"module":"ContentIdeas","id":412,"title":"How to build mobile apps"}]}
|
||||
],
|
||||
"message":"Some items skipped due to mapped records. See details."
|
||||
}
|
||||
|
||||
- Success (single delete):
|
||||
{
|
||||
"success": true,
|
||||
"data": {"deleted_id": 12}
|
||||
}
|
||||
|
||||
**Helper design (concise)**
|
||||
|
||||
- New small utility in `igny8_core.api.utils` (or `igny8_core.modules.planner.utils`) with functions:
|
||||
- `check_delete_blockers_for_cluster(cluster) -> List[dict]`
|
||||
- `check_delete_blockers_for_keyword(keyword) -> List[dict]`
|
||||
- (others as needed)
|
||||
|
||||
- Implementation pattern:
|
||||
- Fast `.exists()` queries and `.values('id', 'title_field')[:5]` to show up to N blockers (avoid heavy scans).
|
||||
- Return a short list of blockers with module name, id, display title.
|
||||
|
||||
Example pseudocode:
|
||||
def check_delete_blockers_cluster(cluster):
|
||||
blockers = []
|
||||
ideas = ContentIdeas.objects.filter(keyword_cluster=cluster).values('id','idea_title')[:5]
|
||||
blockers += [ {'module':'ContentIdeas','id':r['id'],'title':r['idea_title']} for r in ideas ]
|
||||
tasks = Tasks.objects.filter(cluster=cluster).values('id','title')[:5]
|
||||
blockers += [ {'module':'Tasks','id':r['id'],'title':r['title']} for r in tasks ]
|
||||
if ContentClusterMap.objects.filter(cluster=cluster).exists():
|
||||
# get sample content titles
|
||||
return blockers
|
||||
|
||||
**Delete execution flow (safe sequence)**
|
||||
|
||||
- For single delete in ViewSet:
|
||||
1. Fetch instance via `self.get_object()`.
|
||||
2. Run `blockers = check_delete_blockers_*`.
|
||||
3. If `blockers`: return `error_response` (HTTP 400) with `details`.
|
||||
4. Else: inside `transaction.atomic()`:
|
||||
- Update children to defaults BEFORE deleting upstream:
|
||||
- Example: `Keywords.objects.filter(cluster=instance).update(cluster=None, status='new')`
|
||||
- Call `instance.delete()`.
|
||||
- Return success response.
|
||||
|
||||
- For bulk delete:
|
||||
- Iterate ids; for each id, apply same procedure; collect `deleted` and `skipped` with reasons; return both lists. Do not abort entire batch on one failure.
|
||||
|
||||
**Downstream status update rules (simple fallback)**
|
||||
|
||||
- When an upstream mapping is removed (allowed delete):
|
||||
- Use model default for downstream status:
|
||||
- `Keywords` default: `'new'` → set status `'new'`.
|
||||
- `Clusters` default: `'new'` (if applicable).
|
||||
- `ContentIdeas` default: `'new'`.
|
||||
- `Tasks` default: `'queued'`.
|
||||
- `Content` default: `'draft'`.
|
||||
- `Images` default: `'pending'`.
|
||||
- Implementation: set downstream `status` explicitly in the pre-delete update inside the same transaction.
|
||||
|
||||
**DB & Concurrency notes**
|
||||
- Run updates + delete in a single `transaction.atomic()` to avoid transient inconsistent states.
|
||||
- Prefer updating children before deletion to avoid races where FK ON DELETE behavior may set NULL after delete. Example: call `Keywords.objects.filter(cluster=instance).update(cluster=None, status='new')` then `instance.delete()`.
|
||||
|
||||
**Testing checklist (minimal, priority)**
|
||||
- Block tests:
|
||||
- Attempt to delete a `Cluster` that has `ContentIdeas` → assert HTTP 400 and response includes idea title.
|
||||
- Bulk delete mix of deletable and blocked clusters → assert deleted/skipped counts & reasons.
|
||||
- Allowed-delete tests:
|
||||
- Delete a `Cluster` with no blockers → assert `Keywords` previously referencing it have `status == 'new'` and `cluster == None`.
|
||||
- Regression check:
|
||||
- Ensure existing `bulk_delete` endpoints preserve format but return the enhanced report.
|
||||
|
||||
**Developer implementation notes**
|
||||
- Where to change:
|
||||
- Planner viewset: views.py — implement `destroy` override and enhanced `bulk_delete`.
|
||||
- Writer viewsets: `backend/igny8_core/modules/writer/*` (or business/content viewsets) — add `destroy` + `bulk_delete` checks similarly.
|
||||
- Add reusable helper module: `backend/igny8_core/api/utils/delete_checks.py` (or similar).
|
||||
- No DB migrations are required for the minimal policy.
|
||||
- Optional later improvement: add `StatusHistory` model if precise original-status restoration is desired.
|
||||
|
||||
**UX / Messaging guidance**
|
||||
- Keep messages short and actionable: include the blocking module, record title, and next step (delete the blocking records first). Example:
|
||||
- "Cannot delete Cluster 'NAME' — mapped records exist in ContentIdeas: 'Idea title' (ID). Delete those items in Planner → Ideas first, then retry."
|
||||
- For bulk operations, provide counts and a link or list to the specific blocking records.
|
||||
|
||||
**Rollout / Deployment**
|
||||
- Implement API-level checks and tests, deploy to staging. Verify UI flows (single and bulk deletes) show messages and blocked items correctly.
|
||||
- After staging verification, deploy to production. No DB schema migration required.
|
||||
|
||||
---
|
||||
|
||||
If you want, I can now:
|
||||
- Produce a ready-to-copy developer ticket/PR patch that adds the helper and one example ViewSet implementation (e.g., `ClusterViewSet.destroy` + `bulk_delete`) and unit tests — small, safe Phase 1 patch.
|
||||
- Or stop here with this document for handoff.
|
||||
|
||||
Which do you prefer?
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,272 +0,0 @@
|
||||
# Quick Deployment Guide - WordPress Integration Fixes
|
||||
**Date:** November 30, 2025
|
||||
|
||||
## Pre-Deployment Checklist
|
||||
|
||||
- [ ] Backup database
|
||||
- [ ] Backup WordPress site
|
||||
- [ ] Stop Celery workers
|
||||
- [ ] Note current content count in "Published" status
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Apply Database Migration
|
||||
|
||||
```bash
|
||||
cd /data/app/igny8/backend
|
||||
source .venv/bin/activate
|
||||
python manage.py migrate integration
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
Running migrations:
|
||||
Applying integration.0002_add_sync_event_model... OK
|
||||
```
|
||||
|
||||
**Verify migration:**
|
||||
```bash
|
||||
python manage.py showmigrations integration
|
||||
```
|
||||
|
||||
Should show:
|
||||
```
|
||||
integration
|
||||
[X] 0001_initial
|
||||
[X] 0002_add_sync_event_model
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Restart Backend Services
|
||||
|
||||
### If using systemd:
|
||||
```bash
|
||||
sudo systemctl restart igny8-backend
|
||||
sudo systemctl restart igny8-celery-worker
|
||||
sudo systemctl status igny8-backend
|
||||
sudo systemctl status igny8-celery-worker
|
||||
```
|
||||
|
||||
### If using Docker:
|
||||
```bash
|
||||
cd /data/app/igny8
|
||||
docker-compose restart backend
|
||||
docker-compose restart celery
|
||||
docker-compose logs -f celery # Check for errors
|
||||
```
|
||||
|
||||
### If using screen/tmux:
|
||||
```bash
|
||||
# Stop Celery worker (Ctrl+C in screen session)
|
||||
# Start again:
|
||||
celery -A igny8_core worker --loglevel=info
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Update WordPress Plugin (if needed)
|
||||
|
||||
**Option A: If plugin files were updated via git:**
|
||||
```bash
|
||||
# On server
|
||||
cd /data/app/igny8/igny8-wp-plugin
|
||||
git pull origin main
|
||||
|
||||
# Copy to WordPress plugins directory
|
||||
cp -r /data/app/igny8/igny8-wp-plugin /var/www/html/wp-content/plugins/igny8-bridge
|
||||
```
|
||||
|
||||
**Option B: Manual file transfer:**
|
||||
Upload these modified files to WordPress:
|
||||
- `igny8-wp-plugin/sync/igny8-to-wp.php`
|
||||
- `igny8-wp-plugin/sync/post-sync.php`
|
||||
|
||||
**No WordPress settings changes needed!**
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Verify Everything Works
|
||||
|
||||
### Test 1: Check Database Table
|
||||
```bash
|
||||
cd /data/app/igny8/backend
|
||||
source .venv/bin/activate
|
||||
python manage.py dbshell
|
||||
```
|
||||
|
||||
```sql
|
||||
-- Check table exists
|
||||
\dt igny8_sync_events
|
||||
|
||||
-- Check initial structure
|
||||
SELECT COUNT(*) FROM igny8_sync_events;
|
||||
```
|
||||
|
||||
### Test 2: Publish Test Content
|
||||
1. Go to IGNY8 Review page
|
||||
2. Click "Publish" on any content
|
||||
3. Wait 10 seconds
|
||||
4. Go to Published page
|
||||
5. **Expected:** WP Status shows "Published" (green)
|
||||
|
||||
### Test 3: Check Debug Status Page
|
||||
1. Go to Settings → Debug Status
|
||||
2. Select WordPress integration
|
||||
3. **Expected:** See sync event for the test publish
|
||||
|
||||
### Test 4: Check WordPress
|
||||
1. Go to WordPress admin → Posts
|
||||
2. Find the published post
|
||||
3. **Expected:** Post exists with all fields (categories, tags, SEO, image)
|
||||
|
||||
### Test 5: Test Status Sync from WordPress
|
||||
1. In WordPress, change post from "Published" to "Draft"
|
||||
2. Wait 5 seconds
|
||||
3. Go to IGNY8 Published page
|
||||
4. **Expected:** WP Status shows "Draft" (gray)
|
||||
5. **Expected:** Debug Status shows webhook event
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Monitor for Issues
|
||||
|
||||
### Watch Celery logs:
|
||||
```bash
|
||||
# Docker
|
||||
docker-compose logs -f celery | grep "publish_content_to_wordpress"
|
||||
|
||||
# Systemd
|
||||
sudo journalctl -u igny8-celery-worker -f
|
||||
|
||||
# Manual
|
||||
# Just check the screen/tmux session
|
||||
```
|
||||
|
||||
**Look for:**
|
||||
- ✅ "Successfully published content X to WordPress post Y"
|
||||
- ✅ "Content model updated: external_id=..."
|
||||
- ✅ "Status webhook sent for content..."
|
||||
|
||||
**Red flags:**
|
||||
- ❌ "Failed to publish"
|
||||
- ❌ "Exception during publish"
|
||||
- ❌ "Status webhook failed"
|
||||
|
||||
### Watch WordPress error log:
|
||||
```bash
|
||||
tail -f /var/www/html/wp-content/debug.log
|
||||
```
|
||||
|
||||
**Look for:**
|
||||
- ✅ "IGNY8: Status webhook sent for content..."
|
||||
- ✅ "IGNY8: ✅ WordPress post created"
|
||||
|
||||
**Red flags:**
|
||||
- ❌ "IGNY8: Status webhook failed"
|
||||
- ❌ "IGNY8: NOT AUTHENTICATED"
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan (if needed)
|
||||
|
||||
### If migration breaks:
|
||||
```bash
|
||||
cd /data/app/igny8/backend
|
||||
source .venv/bin/activate
|
||||
python manage.py migrate integration 0001_initial
|
||||
```
|
||||
|
||||
### If Celery errors:
|
||||
```bash
|
||||
# Restore old task file from git
|
||||
cd /data/app/igny8/backend
|
||||
git checkout HEAD~1 igny8_core/tasks/wordpress_publishing.py
|
||||
sudo systemctl restart igny8-celery-worker
|
||||
```
|
||||
|
||||
### If WordPress errors:
|
||||
```bash
|
||||
# Restore old plugin files from git
|
||||
cd /data/app/igny8/igny8-wp-plugin
|
||||
git checkout HEAD~1 sync/igny8-to-wp.php sync/post-sync.php
|
||||
# Copy to WordPress
|
||||
cp -r /data/app/igny8/igny8-wp-plugin /var/www/html/wp-content/plugins/igny8-bridge
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Issues & Fixes
|
||||
|
||||
### Issue: "No module named 'SyncEvent'"
|
||||
**Cause:** Migration not applied
|
||||
**Fix:** Run `python manage.py migrate integration`
|
||||
|
||||
### Issue: Celery task failing with "SyncEvent not found"
|
||||
**Cause:** Celery running old code
|
||||
**Fix:** `sudo systemctl restart igny8-celery-worker`
|
||||
|
||||
### Issue: Webhook returns 404
|
||||
**Cause:** URLs not registered
|
||||
**Fix:** `sudo systemctl restart igny8-backend`
|
||||
|
||||
### Issue: WordPress webhook not sending
|
||||
**Cause:** API key not set or wrong
|
||||
**Fix:** Check WordPress Settings → IGNY8 Bridge → API Key
|
||||
|
||||
### Issue: Debug status shows no events
|
||||
**Cause:** Database not created or migration failed
|
||||
**Fix:** Check migration status, verify table exists
|
||||
|
||||
---
|
||||
|
||||
## Performance Monitoring
|
||||
|
||||
### Check SyncEvent table size:
|
||||
```sql
|
||||
SELECT COUNT(*),
|
||||
pg_size_pretty(pg_total_relation_size('igny8_sync_events')) as size
|
||||
FROM igny8_sync_events;
|
||||
```
|
||||
|
||||
### Check recent events:
|
||||
```sql
|
||||
SELECT event_type, COUNT(*),
|
||||
AVG(duration_ms) as avg_duration,
|
||||
MAX(duration_ms) as max_duration
|
||||
FROM igny8_sync_events
|
||||
WHERE created_at > NOW() - INTERVAL '1 hour'
|
||||
GROUP BY event_type;
|
||||
```
|
||||
|
||||
### Cleanup old events (optional):
|
||||
```sql
|
||||
-- Delete events older than 30 days
|
||||
DELETE FROM igny8_sync_events WHERE created_at < NOW() - INTERVAL '30 days';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
After deployment, you should see:
|
||||
|
||||
- ✅ 0 errors in Celery logs for publishing
|
||||
- ✅ 100% of published content has `external_id` set
|
||||
- ✅ All published content shows WP Status on Published page
|
||||
- ✅ Debug Status page shows real events for each publish
|
||||
- ✅ WordPress posts have all fields (categories, tags, images, SEO)
|
||||
- ✅ Status changes in WordPress sync back to IGNY8 within 5 seconds
|
||||
|
||||
---
|
||||
|
||||
## Contact/Support
|
||||
|
||||
If you encounter issues:
|
||||
1. Check logs (Celery, Django, WordPress debug.log)
|
||||
2. Review troubleshooting section in main documentation
|
||||
3. Verify all services restarted after deployment
|
||||
4. Check network connectivity (IGNY8 ↔ WordPress)
|
||||
5. Verify API keys match on both sides
|
||||
|
||||
**Good luck with deployment!** 🚀
|
||||
@@ -1,589 +0,0 @@
|
||||
# WordPress Integration Fixes - Complete Diagnostic & Implementation Report
|
||||
**Date:** November 30, 2025
|
||||
**Status:** ✅ ALL ISSUES FIXED
|
||||
**Migration Created:** Yes - `0002_add_sync_event_model.py`
|
||||
|
||||
---
|
||||
|
||||
## Issues Identified and Fixed
|
||||
|
||||
### ✅ Issue 1: Content Status Not Changing from 'review' to 'published'
|
||||
|
||||
**Root Cause:**
|
||||
This was ALREADY FIXED in previous updates. The code in `ContentViewSet.publish()` (line 827-828) sets status to 'published' immediately when the publish button is clicked.
|
||||
|
||||
**Current Behavior:**
|
||||
- Status changes to 'published' immediately upon clicking publish
|
||||
- Celery task runs in background to actually publish to WordPress
|
||||
- No changes needed
|
||||
|
||||
**Files Verified:**
|
||||
- `backend/igny8_core/modules/writer/views.py` (lines 827-828)
|
||||
|
||||
---
|
||||
|
||||
### ✅ Issue 2: WP Status Column Not Updating
|
||||
|
||||
**Root Cause:**
|
||||
The `wordpress_status` field was not being stored in the Content model after WordPress responds. The Celery task was only updating `external_id` and `external_url`.
|
||||
|
||||
**Fix Applied:**
|
||||
Updated `publish_content_to_wordpress` task to:
|
||||
1. Extract `post_status` from WordPress API response
|
||||
2. Store in `content.metadata['wordpress_status']`
|
||||
3. Save to database alongside `external_id` and `external_url`
|
||||
|
||||
**Code Changes:**
|
||||
```python
|
||||
# File: backend/igny8_core/tasks/wordpress_publishing.py (lines 197-225)
|
||||
wp_data = response.json().get('data', {})
|
||||
wp_status = wp_data.get('post_status', 'publish')
|
||||
|
||||
# Update wordpress_status in metadata
|
||||
if not hasattr(content, 'metadata') or content.metadata is None:
|
||||
content.metadata = {}
|
||||
content.metadata['wordpress_status'] = wp_status
|
||||
|
||||
content.save(update_fields=[
|
||||
'external_id', 'external_url', 'status', 'metadata', 'updated_at'
|
||||
])
|
||||
```
|
||||
|
||||
**Files Modified:**
|
||||
- `backend/igny8_core/tasks/wordpress_publishing.py`
|
||||
|
||||
---
|
||||
|
||||
### ✅ Issue 3: WordPress Sync Back to IGNY8 Not Working
|
||||
|
||||
**Root Cause:**
|
||||
WordPress plugin was calling the old task API (`PUT /writer/tasks/{id}/`), which doesn't update the Content model. The Content model needs to be updated via webhook.
|
||||
|
||||
**Fix Applied:**
|
||||
1. Created webhook endpoints in IGNY8 backend:
|
||||
- `POST /api/v1/integration/webhooks/wordpress/status/` - Receives status updates
|
||||
- `POST /api/v1/integration/webhooks/wordpress/metadata/` - Receives metadata updates
|
||||
|
||||
2. Updated WordPress plugin to call webhook after creating/updating posts:
|
||||
- Added `igny8_send_status_webhook()` function in `sync/igny8-to-wp.php`
|
||||
- Added webhook call in `sync/post-sync.php` after status sync
|
||||
- Webhooks are non-blocking (async) for better performance
|
||||
|
||||
**Webhook Flow:**
|
||||
```
|
||||
WordPress Post Created/Updated
|
||||
↓
|
||||
igny8_send_status_webhook() called
|
||||
↓
|
||||
POST /api/v1/integration/webhooks/wordpress/status/
|
||||
↓
|
||||
Content model updated:
|
||||
- external_id = WordPress post ID
|
||||
- external_url = WordPress post URL
|
||||
- metadata.wordpress_status = WordPress status
|
||||
- status = mapped IGNY8 status (if applicable)
|
||||
↓
|
||||
SyncEvent logged for real-time monitoring
|
||||
```
|
||||
|
||||
**Files Created:**
|
||||
- `backend/igny8_core/modules/integration/webhooks.py`
|
||||
|
||||
**Files Modified:**
|
||||
- `backend/igny8_core/modules/integration/urls.py`
|
||||
- `igny8-wp-plugin/sync/igny8-to-wp.php` (added webhook function)
|
||||
- `igny8-wp-plugin/sync/post-sync.php` (added webhook call)
|
||||
|
||||
---
|
||||
|
||||
### ✅ Issue 4: Debug Status Page - No Real-Time Events
|
||||
|
||||
**Root Cause:**
|
||||
The debug status page was showing placeholder data. There was no real event logging system in the database.
|
||||
|
||||
**Fix Applied:**
|
||||
1. Created `SyncEvent` model to track all sync operations:
|
||||
- Stores event type (publish, sync, error, webhook, test)
|
||||
- Stores success/failure status
|
||||
- Stores content_id, external_id, error messages
|
||||
- Stores duration in milliseconds
|
||||
- Stores detailed JSON payload
|
||||
|
||||
2. Updated debug status endpoint to fetch real events from database:
|
||||
- `GET /api/v1/integration/integrations/{id}/debug-status/?include_events=true&event_limit=50`
|
||||
- Returns actual SyncEvent records ordered by newest first
|
||||
|
||||
3. Added event logging to all sync operations:
|
||||
- Publishing to WordPress (success/failure)
|
||||
- Webhook received from WordPress
|
||||
- Status updates
|
||||
- Errors with full details
|
||||
|
||||
**Database Schema:**
|
||||
```python
|
||||
class SyncEvent(AccountBaseModel):
|
||||
integration = ForeignKey(SiteIntegration)
|
||||
site = ForeignKey(Site)
|
||||
event_type = CharField(choices=['publish', 'sync', 'metadata_sync', 'error', 'webhook', 'test'])
|
||||
action = CharField(choices=['content_publish', 'status_update', 'metadata_update', ...])
|
||||
description = TextField()
|
||||
success = BooleanField()
|
||||
content_id = IntegerField(null=True)
|
||||
external_id = CharField(null=True)
|
||||
details = JSONField()
|
||||
error_message = TextField(null=True)
|
||||
duration_ms = IntegerField(null=True)
|
||||
created_at = DateTimeField()
|
||||
```
|
||||
|
||||
**Files Created:**
|
||||
- `backend/igny8_core/business/integration/models.py` (SyncEvent model added)
|
||||
- `backend/igny8_core/business/integration/migrations/0002_add_sync_event_model.py`
|
||||
|
||||
**Files Modified:**
|
||||
- `backend/igny8_core/modules/integration/views.py` (debug_status endpoint updated)
|
||||
- `backend/igny8_core/tasks/wordpress_publishing.py` (added event logging)
|
||||
|
||||
---
|
||||
|
||||
### ✅ Issue 5: Incomplete Field Publishing to WordPress
|
||||
|
||||
**Root Cause:**
|
||||
This was NOT actually broken. The existing code already sends ALL fields:
|
||||
- Categories, tags, images, SEO metadata, cluster/sector IDs
|
||||
|
||||
**Verification:**
|
||||
Reviewed the complete publishing flow:
|
||||
|
||||
1. **Celery Task** (`publish_content_to_wordpress`):
|
||||
- Sends: categories, tags, featured_image_url, gallery_images, seo_title, seo_description, primary_keyword, secondary_keywords, cluster_id, sector_id
|
||||
- Logs: Full payload summary with all fields
|
||||
|
||||
2. **WordPress REST Endpoint** (`publish_content_to_wordpress`):
|
||||
- Logs: All incoming fields for debugging
|
||||
- Validates: title, content_html, content_id
|
||||
|
||||
3. **WordPress Post Creation** (`igny8_create_wordpress_post_from_task`):
|
||||
- Processes: Categories → `wp_set_post_terms()`
|
||||
- Processes: Tags → `wp_set_post_terms()`
|
||||
- Processes: Featured image → `igny8_set_featured_image()`
|
||||
- Processes: SEO metadata → Multiple SEO plugins (Yoast, SEOPress, AIOSEO)
|
||||
- Processes: Gallery images → `igny8_set_gallery_images()`
|
||||
- Assigns: Cluster/sector taxonomy terms
|
||||
|
||||
**Conclusion:**
|
||||
All fields ARE being published. The WordPress plugin logs show complete field processing. No changes needed.
|
||||
|
||||
---
|
||||
|
||||
## Complete System Flow (After Fixes)
|
||||
|
||||
### Publishing Flow: IGNY8 → WordPress
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 1. User clicks "Publish" in Review page │
|
||||
│ frontend/src/pages/Writer/Review.tsx │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ POST /api/v1/writer/content/{id}/publish/
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 2. ContentViewSet.publish() - IMMEDIATE STATUS UPDATE │
|
||||
│ backend/igny8_core/modules/writer/views.py │
|
||||
│ - content.status = 'published' ✅ │
|
||||
│ - Queues Celery task │
|
||||
│ - Returns 202 ACCEPTED immediately │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ Celery task queued
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 3. publish_content_to_wordpress() Celery Task │
|
||||
│ backend/igny8_core/tasks/wordpress_publishing.py │
|
||||
│ - Prepares full payload (title, content, SEO, images, etc) │
|
||||
│ - Logs sync event (start) │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ POST {site_url}/wp-json/igny8/v1/publish-content/
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 4. WordPress REST Endpoint │
|
||||
│ igny8-wp-plugin/includes/class-igny8-rest-api.php │
|
||||
│ - Validates API key │
|
||||
│ - Logs all incoming fields │
|
||||
│ - Calls igny8_create_wordpress_post_from_task() │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 5. WordPress Post Creation │
|
||||
│ igny8-wp-plugin/sync/igny8-to-wp.php │
|
||||
│ - wp_insert_post() - Create post │
|
||||
│ - Assign categories/tags │
|
||||
│ - Set featured image │
|
||||
│ - Set SEO metadata (Yoast/SEOPress/AIOSEO) │
|
||||
│ - Assign cluster/sector taxonomies │
|
||||
│ - Store IGNY8 meta fields │
|
||||
│ - Send status webhook to IGNY8 ✅ NEW │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ Return post_id, post_url, post_status
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 6. Celery Task Receives Response │
|
||||
│ backend/igny8_core/tasks/wordpress_publishing.py │
|
||||
│ - content.external_id = post_id ✅ │
|
||||
│ - content.external_url = post_url ✅ │
|
||||
│ - content.metadata['wordpress_status'] = post_status ✅ NEW │
|
||||
│ - content.save() │
|
||||
│ - Log sync event (success) ✅ NEW │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Status Sync Flow: WordPress → IGNY8
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 1. WordPress Post Status Changes │
|
||||
│ (User edits post, changes status in WordPress) │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ WordPress hook: save_post, transition_post_status
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 2. igny8_sync_post_status_to_igny8() │
|
||||
│ igny8-wp-plugin/sync/post-sync.php │
|
||||
│ - Check if IGNY8-managed post │
|
||||
│ - Get content_id from post meta │
|
||||
│ - Call igny8_send_status_webhook() ✅ NEW │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│ POST /api/v1/integration/webhooks/wordpress/status/
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 3. wordpress_status_webhook() │
|
||||
│ backend/igny8_core/modules/integration/webhooks.py ✅ NEW │
|
||||
│ - Validate API key │
|
||||
│ - Find Content by content_id │
|
||||
│ - Update content.metadata['wordpress_status'] │
|
||||
│ - Update content.status (if publish/draft change) │
|
||||
│ - Log sync event │
|
||||
└────────────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 4. Frontend Published Page Auto-Refreshes │
|
||||
│ - WP Status column shows updated status ✅ │
|
||||
│ - Debug Status page shows real-time event ✅ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files Modified Summary
|
||||
|
||||
### Backend (Django)
|
||||
|
||||
1. **backend/igny8_core/business/integration/models.py**
|
||||
- ✅ Added `SyncEvent` model for event logging
|
||||
|
||||
2. **backend/igny8_core/tasks/wordpress_publishing.py**
|
||||
- ✅ Added `wordpress_status` field update in Content model
|
||||
- ✅ Added SyncEvent logging for publish, error, and webhook events
|
||||
- ✅ Added duration tracking
|
||||
|
||||
3. **backend/igny8_core/modules/integration/views.py**
|
||||
- ✅ Updated `debug_status()` endpoint to fetch real SyncEvent records
|
||||
|
||||
4. **backend/igny8_core/modules/integration/webhooks.py** (NEW)
|
||||
- ✅ Created `wordpress_status_webhook()` endpoint
|
||||
- ✅ Created `wordpress_metadata_webhook()` endpoint
|
||||
|
||||
5. **backend/igny8_core/modules/integration/urls.py**
|
||||
- ✅ Added webhook URL routes
|
||||
|
||||
6. **backend/igny8_core/business/integration/migrations/0002_add_sync_event_model.py** (NEW)
|
||||
- ✅ Database migration for SyncEvent model
|
||||
|
||||
### WordPress Plugin
|
||||
|
||||
7. **igny8-wp-plugin/sync/igny8-to-wp.php**
|
||||
- ✅ Added `igny8_send_status_webhook()` function
|
||||
- ✅ Added webhook call after post creation
|
||||
|
||||
8. **igny8-wp-plugin/sync/post-sync.php**
|
||||
- ✅ Added webhook call after status sync
|
||||
|
||||
---
|
||||
|
||||
## Migration Instructions
|
||||
|
||||
### 1. Apply Database Migration
|
||||
|
||||
```bash
|
||||
cd /data/app/igny8/backend
|
||||
source .venv/bin/activate
|
||||
python manage.py migrate integration
|
||||
```
|
||||
|
||||
This will create the `igny8_sync_events` table.
|
||||
|
||||
### 2. Restart Services
|
||||
|
||||
```bash
|
||||
# Restart Django server
|
||||
sudo systemctl restart igny8-backend
|
||||
|
||||
# Restart Celery worker (to pick up new task code)
|
||||
sudo systemctl restart igny8-celery-worker
|
||||
|
||||
# If Celery is running in Docker, restart container:
|
||||
docker-compose restart celery
|
||||
```
|
||||
|
||||
### 3. Update WordPress Plugin
|
||||
|
||||
The WordPress plugin files have been updated. If you deployed via version control:
|
||||
|
||||
```bash
|
||||
cd /data/app/igny8/igny8-wp-plugin
|
||||
git pull
|
||||
# OR manually copy updated files to WordPress plugins directory
|
||||
```
|
||||
|
||||
No WordPress plugin settings changes required - the webhook uses the existing API key.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
### ✅ Test 1: Content Publishing
|
||||
1. Go to Review page
|
||||
2. Click "Publish" on a content item
|
||||
3. **Expected:** Status changes to "Published" immediately
|
||||
4. **Expected:** Within 5-10 seconds, WordPress post is created
|
||||
5. **Expected:** `external_id` and `external_url` are populated
|
||||
|
||||
### ✅ Test 2: WP Status Column on Published Page
|
||||
1. Go to Published page
|
||||
2. Look at WP Status column
|
||||
3. **Expected:** Shows "Published" (green badge) for published content
|
||||
4. **Expected:** Shows "Not Published" (gray badge) if not yet published to WP
|
||||
|
||||
### ✅ Test 3: Debug Status Page - Real-Time Events
|
||||
1. Go to Settings → Debug Status
|
||||
2. Select a site with WordPress integration
|
||||
3. **Expected:** See list of recent sync events with:
|
||||
- Event type (publish, sync, webhook, error)
|
||||
- Description
|
||||
- Timestamp
|
||||
- Success/failure status
|
||||
- Content ID, WordPress post ID
|
||||
4. Publish new content
|
||||
5. **Expected:** New event appears in the list within seconds
|
||||
|
||||
### ✅ Test 4: WordPress Status Sync Back to IGNY8
|
||||
1. Publish content from IGNY8
|
||||
2. Go to WordPress admin
|
||||
3. Change post status (draft → publish, or publish → draft)
|
||||
4. **Expected:** Within 5 seconds, IGNY8 Published page reflects the change
|
||||
5. **Expected:** Debug Status page shows webhook event
|
||||
|
||||
### ✅ Test 5: Complete Field Publishing
|
||||
1. Create content with:
|
||||
- Categories
|
||||
- Tags
|
||||
- Featured image
|
||||
- Gallery images
|
||||
- SEO title & description
|
||||
- Primary & secondary keywords
|
||||
2. Publish to WordPress
|
||||
3. **Expected:** All fields appear in WordPress post:
|
||||
- Categories assigned
|
||||
- Tags assigned
|
||||
- Featured image set
|
||||
- SEO metadata in Yoast/SEOPress/AIOSEO
|
||||
- IGNY8 custom fields stored
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: SyncEvent table doesn't exist
|
||||
**Solution:** Run migration: `python manage.py migrate integration`
|
||||
|
||||
### Issue: Webhook not being called from WordPress
|
||||
**Solution:**
|
||||
1. Check WordPress error log for "IGNY8: Status webhook" messages
|
||||
2. Verify API key is set in WordPress settings
|
||||
3. Check WordPress can reach IGNY8 backend (firewall, DNS)
|
||||
|
||||
### Issue: Debug status shows no events
|
||||
**Solution:**
|
||||
1. Verify migration was applied
|
||||
2. Publish test content to generate events
|
||||
3. Check `igny8_sync_events` table has records
|
||||
|
||||
### Issue: WP Status still not updating
|
||||
**Solution:**
|
||||
1. Check Content.metadata field has `wordpress_status` key
|
||||
2. Verify Celery worker is running with updated code
|
||||
3. Check webhook endpoint is accessible: `POST /api/v1/integration/webhooks/wordpress/status/`
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints Added
|
||||
|
||||
### Webhook Endpoints (NEW)
|
||||
|
||||
#### POST /api/v1/integration/webhooks/wordpress/status/
|
||||
Receives WordPress post status updates
|
||||
|
||||
**Headers:**
|
||||
- `X-IGNY8-API-KEY`: WordPress site API key
|
||||
|
||||
**Body:**
|
||||
```json
|
||||
{
|
||||
"post_id": 123,
|
||||
"content_id": 456,
|
||||
"post_status": "publish",
|
||||
"post_url": "https://example.com/post-title/",
|
||||
"post_title": "Post Title",
|
||||
"site_url": "https://example.com"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"content_id": 456,
|
||||
"status": "published",
|
||||
"wordpress_status": "publish",
|
||||
"external_id": "123",
|
||||
"external_url": "https://example.com/post-title/"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### POST /api/v1/integration/webhooks/wordpress/metadata/
|
||||
Receives WordPress metadata updates (categories, tags, author, etc.)
|
||||
|
||||
**Headers:**
|
||||
- `X-IGNY8-API-KEY`: WordPress site API key
|
||||
|
||||
**Body:**
|
||||
```json
|
||||
{
|
||||
"post_id": 123,
|
||||
"content_id": 456,
|
||||
"site_url": "https://example.com",
|
||||
"metadata": {
|
||||
"categories": ["Tech", "News"],
|
||||
"tags": ["AI", "Machine Learning"],
|
||||
"author": {"id": 1, "name": "Admin"},
|
||||
"modified_date": "2025-11-30T12:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Debug Status Endpoint (UPDATED)
|
||||
|
||||
#### GET /api/v1/integration/integrations/{id}/debug-status/
|
||||
Now returns real SyncEvent records instead of placeholder data
|
||||
|
||||
**Query Parameters:**
|
||||
- `include_events`: boolean (default: true) - Include sync events
|
||||
- `event_limit`: integer (default: 50) - Number of events to return
|
||||
- `include_validation`: boolean (default: false) - Include validation matrix
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"health": {
|
||||
"api_status": "healthy",
|
||||
"plugin_active": true,
|
||||
"sync_healthy": true,
|
||||
"last_sync": "2025-11-30T12:00:00Z"
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"id": 123,
|
||||
"type": "publish",
|
||||
"action": "content_publish",
|
||||
"description": "Published content 'Sample Post' to WordPress",
|
||||
"timestamp": "2025-11-30T12:00:00Z",
|
||||
"success": true,
|
||||
"content_id": 456,
|
||||
"external_id": "789",
|
||||
"duration_ms": 1250,
|
||||
"details": {
|
||||
"post_url": "https://example.com/sample-post/",
|
||||
"wordpress_status": "publish",
|
||||
"categories": ["Tech"],
|
||||
"tags": ["AI", "ML"]
|
||||
}
|
||||
}
|
||||
],
|
||||
"events_count": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Impact
|
||||
|
||||
### Backend
|
||||
- **SyncEvent logging:** ~5-10ms per event (non-blocking)
|
||||
- **Webhook processing:** ~50-100ms per webhook (async)
|
||||
- **Database:** New table with indexes, minimal impact
|
||||
|
||||
### WordPress
|
||||
- **Webhook sending:** Non-blocking (async), no user-facing delay
|
||||
- **Post creation:** ~100-200ms additional for webhook call
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### Webhook Authentication
|
||||
- Webhooks use the same API key as WordPress integration
|
||||
- API key verified against `SiteIntegration.credentials_json['api_key']`
|
||||
- Webhook endpoints have no throttling (AllowAny) but require valid API key
|
||||
- Mismatched API key returns 401 Unauthorized
|
||||
|
||||
### Data Validation
|
||||
- All webhook payloads validated for required fields
|
||||
- Content ID existence checked before update
|
||||
- Integration verification ensures webhook is from correct site
|
||||
|
||||
---
|
||||
|
||||
## Summary of All Fixes
|
||||
|
||||
| Issue | Status | Fix Description |
|
||||
|-------|--------|----------------|
|
||||
| Content status not changing to 'published' | ✅ ALREADY FIXED | Status changes immediately on publish button click |
|
||||
| WP Status not updating in IGNY8 | ✅ FIXED | Added wordpress_status to Content.metadata + webhooks |
|
||||
| Status changes in WP not syncing back | ✅ FIXED | Created webhook endpoints + WordPress webhook calls |
|
||||
| Debug status page showing no events | ✅ FIXED | Created SyncEvent model + real-time event logging |
|
||||
| Incomplete field publishing | ✅ VERIFIED | All fields already being sent and processed correctly |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps (Post-Deployment)
|
||||
|
||||
1. **Monitor sync events** in Debug Status page
|
||||
2. **Check Celery worker logs** for any errors during publishing
|
||||
3. **Verify WordPress error logs** for webhook send confirmation
|
||||
4. **Test edge cases:**
|
||||
- Publishing content with no categories/tags
|
||||
- Publishing content with very long titles
|
||||
- Changing status multiple times rapidly
|
||||
5. **Performance monitoring:**
|
||||
- Monitor `igny8_sync_events` table size
|
||||
- Consider adding cleanup job for old events (>30 days)
|
||||
|
||||
---
|
||||
|
||||
**All issues have been diagnosed and fixed. The system is now fully functional with real-time sync event monitoring!** 🎉
|
||||
@@ -1,275 +0,0 @@
|
||||
# WordPress Integration Fixes - Implementation Complete
|
||||
**Date:** December 1, 2025
|
||||
**Status:** ✅ IMPLEMENTED
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
All critical WordPress integration issues have been fixed:
|
||||
|
||||
### ✅ Fix #1: Status Sync Logic (COMPLETED)
|
||||
**Issue:** Content stayed in "review" after WordPress publish
|
||||
**Fix:** Updated webhook handler to always sync status when WordPress status differs
|
||||
|
||||
**File Changed:** `backend/igny8_core/modules/integration/webhooks.py`
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
# Only updates if WordPress status changed from draft to publish
|
||||
if post_status == 'publish' and old_status != 'published':
|
||||
content.status = 'published'
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# FIXED: Always update status when WordPress status differs
|
||||
if content.status != igny8_status:
|
||||
content.status = igny8_status
|
||||
logger.info(f"[wordpress_status_webhook] Status updated: {old_status} → {content.status}")
|
||||
```
|
||||
|
||||
**Result:** Content status now correctly changes from 'review' → 'published' when WordPress publishes the post.
|
||||
|
||||
---
|
||||
|
||||
### ✅ Fix #2: Sync Button Republishes Everything (COMPLETED)
|
||||
**Issue:** "Sync Now" button republished all content instead of just fetching metadata
|
||||
**Fix:** Created new SyncMetadataService that only fetches WordPress structure
|
||||
|
||||
**Files Changed:**
|
||||
- `backend/igny8_core/business/integration/services/sync_metadata_service.py` (NEW)
|
||||
- `backend/igny8_core/modules/integration/views.py`
|
||||
- `frontend/src/services/integration.api.ts`
|
||||
- `frontend/src/pages/Sites/Settings.tsx`
|
||||
|
||||
**New Service:**
|
||||
```python
|
||||
class SyncMetadataService:
|
||||
def sync_wordpress_structure(self, integration):
|
||||
"""
|
||||
Fetch WordPress site structure (post types, taxonomies, counts).
|
||||
Does NOT publish or sync any content.
|
||||
"""
|
||||
# Calls: /wp-json/igny8/v1/site-metadata/
|
||||
# Returns: post_types, taxonomies, counts
|
||||
# Updates: integration.last_sync_at, integration.sync_status
|
||||
```
|
||||
|
||||
**Backend Changes:**
|
||||
```python
|
||||
# IntegrationViewSet.sync() now supports direction='metadata'
|
||||
if direction == 'metadata':
|
||||
metadata_service = SyncMetadataService()
|
||||
result = metadata_service.sync_wordpress_structure(integration)
|
||||
else:
|
||||
# Full content sync (legacy behavior)
|
||||
sync_service = SyncService()
|
||||
result = sync_service.sync(integration, direction=direction)
|
||||
```
|
||||
|
||||
**Frontend Changes:**
|
||||
```typescript
|
||||
// Changed default sync type to 'metadata'
|
||||
async syncIntegration(
|
||||
integrationId: number,
|
||||
syncType: 'metadata' | 'incremental' | 'full' = 'metadata'
|
||||
)
|
||||
|
||||
// Settings.tsx now calls with 'metadata'
|
||||
const res = await integrationApi.syncIntegration(wordPressIntegration.id, 'metadata');
|
||||
```
|
||||
|
||||
**Result:** "Sync Now" button now only fetches WordPress structure, completes in <5 seconds, and does NOT republish content.
|
||||
|
||||
---
|
||||
|
||||
### ✅ Fix #3: Improved Sync Status UI (COMPLETED)
|
||||
**Issue:** Poor visual feedback for sync status and last sync time
|
||||
**Fix:** Enhanced Content Types tab with status indicators, spinners, and better layout
|
||||
|
||||
**File Changed:** `frontend/src/pages/Sites/Settings.tsx`
|
||||
|
||||
**New UI Features:**
|
||||
1. **Status Indicator Badge:**
|
||||
- Green dot + "Synced" when sync_status = 'success'
|
||||
- Red dot + "Failed" when sync_status = 'failed'
|
||||
- Yellow dot + "Pending" when sync_status = 'pending'
|
||||
|
||||
2. **Last Sync Time Display:**
|
||||
- Shows relative time (e.g., "2m ago", "1h ago")
|
||||
- Updates when sync completes
|
||||
|
||||
3. **Improved Sync Button:**
|
||||
- Shows spinner icon during sync
|
||||
- Text changes: "Sync Structure" → "Syncing..."
|
||||
- Refresh icon when idle
|
||||
|
||||
4. **Better Empty State:**
|
||||
- Large icon when no content types
|
||||
- Helpful message: "Click 'Sync Structure' to fetch WordPress content types"
|
||||
|
||||
5. **Loading State:**
|
||||
- Spinner animation
|
||||
- "Loading content types..." message
|
||||
|
||||
**Result:** Much clearer visual feedback for users, professional UI/UX.
|
||||
|
||||
---
|
||||
|
||||
### ✅ Fix #4: Diagnostic Logging for Categories/Tags/Images (COMPLETED)
|
||||
**Issue:** Categories, tags, and images not appearing in WordPress posts
|
||||
**Fix:** Added comprehensive logging to identify exact issue
|
||||
|
||||
**File Changed:** `igny8-wp-plugin/includes/class-igny8-rest-api.php`
|
||||
|
||||
**Added Logging:**
|
||||
```php
|
||||
// Log raw request body
|
||||
$raw_body = $request->get_body();
|
||||
error_log('========== RAW REQUEST BODY ==========');
|
||||
error_log($raw_body);
|
||||
error_log('======================================');
|
||||
|
||||
// Log parsed JSON
|
||||
$content_data = $request->get_json_params();
|
||||
error_log('========== PARSED JSON DATA ==========');
|
||||
error_log(print_r($content_data, true));
|
||||
error_log('======================================');
|
||||
```
|
||||
|
||||
**Result:** WordPress plugin now logs both raw and parsed data, making it easy to identify if fields are lost during transmission or parsing.
|
||||
|
||||
---
|
||||
|
||||
## Testing Instructions
|
||||
|
||||
### Test #1: Status Sync
|
||||
1. Create new content in Writer (status: 'draft' or 'review')
|
||||
2. Click "Publish to WordPress"
|
||||
3. Wait for WordPress to create post
|
||||
4. Verify in IGNY8: Content status should change to 'published'
|
||||
5. Check Content list: "WP Status" column should show published badge
|
||||
|
||||
### Test #2: Sync Button
|
||||
1. Go to: Settings → Integrations → WordPress → Content Types
|
||||
2. Click "Sync Structure" button
|
||||
3. Verify:
|
||||
- Button shows spinner and "Syncing..." text
|
||||
- Completes in < 5 seconds
|
||||
- No duplicate posts created in WordPress
|
||||
- Content Types table updates with fresh counts
|
||||
- Status indicator shows green "Synced"
|
||||
- Last sync time updates
|
||||
|
||||
### Test #3: Improved UI
|
||||
1. Navigate to Site Settings → Content Types tab
|
||||
2. Verify UI elements:
|
||||
- Sync status indicator (green/red/yellow dot + text)
|
||||
- Last sync time displayed (e.g., "2m ago")
|
||||
- Sync button has refresh icon
|
||||
- Spinner shows during sync
|
||||
- Empty state message if no data
|
||||
|
||||
### Test #4: Categories/Tags/Images Diagnostic
|
||||
1. Publish content with categories, tags, and images
|
||||
2. Check WordPress plugin logs (wp-content/debug.log or error_log)
|
||||
3. Verify logs show:
|
||||
- Raw request body
|
||||
- Parsed JSON data
|
||||
- All fields (categories, tags, images) visible in logs
|
||||
4. If fields are missing in parsed JSON but present in raw body, we've identified the parsing issue
|
||||
|
||||
---
|
||||
|
||||
## Next Steps for Categories/Tags/Images Issue
|
||||
|
||||
**Current Status:** Diagnostic logging added, ready to identify issue
|
||||
|
||||
**Action Required:**
|
||||
1. Publish content from IGNY8 to WordPress
|
||||
2. Check WordPress plugin logs
|
||||
3. If fields are present in raw body but missing in parsed JSON:
|
||||
- Issue is WordPress REST API JSON parsing
|
||||
- Solution: Use `$request->get_body()` and manually parse with `json_decode()`
|
||||
4. If fields are missing in raw body:
|
||||
- Issue is in IGNY8 backend payload building
|
||||
- Solution: Fix payload construction in `wordpress_publishing.py`
|
||||
|
||||
**Likely Issue:** WordPress REST API may be filtering/sanitizing certain fields. The diagnostic logs will confirm this.
|
||||
|
||||
**Quick Fix (if parsing is the issue):**
|
||||
```php
|
||||
// Replace get_json_params() with manual parsing
|
||||
$raw_body = $request->get_body();
|
||||
$content_data = json_decode($raw_body, true);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
### Backend (Python/Django)
|
||||
1. `backend/igny8_core/modules/integration/webhooks.py` - Fixed status sync logic
|
||||
2. `backend/igny8_core/business/integration/services/sync_metadata_service.py` - NEW file
|
||||
3. `backend/igny8_core/modules/integration/views.py` - Added metadata sync support
|
||||
|
||||
### Frontend (TypeScript/React)
|
||||
4. `frontend/src/services/integration.api.ts` - Changed sync API signature
|
||||
5. `frontend/src/pages/Sites/Settings.tsx` - Improved UI and changed sync call
|
||||
|
||||
### WordPress Plugin (PHP)
|
||||
6. `igny8-wp-plugin/includes/class-igny8-rest-api.php` - Added diagnostic logging
|
||||
|
||||
### Documentation
|
||||
7. `docs/WORDPRESS-INTEGRATION-REFACTOR-PLAN-2025-12-01.md` - Complete refactor plan
|
||||
8. `docs/WORDPRESS-INTEGRATION-FIXES-IMPLEMENTATION-2025-12-01.md` - This file
|
||||
|
||||
---
|
||||
|
||||
## Performance Improvements
|
||||
|
||||
**Before:**
|
||||
- Sync button: 30+ seconds, republishes 100 items
|
||||
- Status sync: Not working (stays 'review')
|
||||
- UI: Minimal feedback, no status indicators
|
||||
|
||||
**After:**
|
||||
- Sync button: < 5 seconds, only fetches metadata
|
||||
- Status sync: Works correctly (review → published)
|
||||
- UI: Professional with status indicators, spinners, clear feedback
|
||||
|
||||
---
|
||||
|
||||
## Backend Services Status
|
||||
|
||||
✅ Backend restarted successfully
|
||||
✅ Celery worker restarted successfully
|
||||
✅ New SyncMetadataService loaded
|
||||
✅ Webhook handler updated
|
||||
✅ All services healthy
|
||||
|
||||
---
|
||||
|
||||
## Summary of Fixes
|
||||
|
||||
| Issue | Status | Impact |
|
||||
|-------|--------|--------|
|
||||
| Status sync broken | ✅ FIXED | Content now changes from 'review' → 'published' |
|
||||
| Sync republishes everything | ✅ FIXED | Now only fetches metadata, 6x faster |
|
||||
| Poor sync UI feedback | ✅ FIXED | Professional status indicators and feedback |
|
||||
| Categories/tags/images missing | 🔍 DIAGNOSTIC ADDED | Logs will identify exact issue |
|
||||
|
||||
---
|
||||
|
||||
## Total Implementation Time
|
||||
|
||||
- Fix #1 (Status Sync): 15 minutes
|
||||
- Fix #2 (Sync Service): 45 minutes
|
||||
- Fix #3 (UI Improvements): 30 minutes
|
||||
- Fix #4 (Diagnostic Logging): 15 minutes
|
||||
- Testing & Documentation: 15 minutes
|
||||
|
||||
**Total: 2 hours** (vs. estimated 8-11 hours in plan)
|
||||
|
||||
All core fixes implemented and working. Categories/tags/images diagnostic logging ready for next publish operation.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,334 +0,0 @@
|
||||
# WordPress Publishing Complete Field Mapping
|
||||
|
||||
**Last Updated**: 2025-12-01
|
||||
**Purpose**: Complete documentation of IGNY8 → WordPress publishing workflow with all field mappings
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Publishing Workflow Overview
|
||||
|
||||
```
|
||||
Frontend (Review.tsx)
|
||||
↓ POST /v1/publisher/publish/
|
||||
Backend (PublisherViewSet)
|
||||
↓ PublisherService.publish()
|
||||
WordPressAdapter._publish_via_api_key()
|
||||
↓ POST {site_url}/wp-json/igny8/v1/publish
|
||||
WordPress Plugin (class-igny8-rest-api.php)
|
||||
↓ publish_content_to_wordpress()
|
||||
WordPress Plugin (igny8-to-wp.php)
|
||||
↓ igny8_create_wordpress_post_from_task()
|
||||
WordPress Core
|
||||
↓ wp_insert_post() + post_meta
|
||||
WordPress Database (wp_posts, wp_postmeta, wp_term_relationships)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Complete Field Mapping Table
|
||||
|
||||
| # | IGNY8 Field | DB Column | Backend Publisher | WordPress Plugin Endpoint | WordPress Plugin Function | WordPress Field | Status | Notes |
|
||||
|---|-------------|-----------|-------------------|---------------------------|--------------------------|----------------|--------|-------|
|
||||
| 1 | **id** | `id` | `content_id` | `content_id` (required) | `_igny8_content_id` (post_meta) | `wp_postmeta` | ✅ Published | Tracking field |
|
||||
| 2 | **title** | `title` | `title` | `title` (required) | `post_title` | `wp_posts.post_title` | ✅ Published | Core field |
|
||||
| 3 | **content_html** | `content_html` | `content_html` | `content_html` (required) | `post_content` | `wp_posts.post_content` | ✅ Published | Core field, sanitized with `wp_kses_post()` |
|
||||
| 4 | **meta_title** | `meta_title` | `seo_title` | `seo_title` OR `meta_title` | `_yoast_wpseo_title`, `_seopress_titles_title`, `_aioseo_title`, `_igny8_meta_title` | `wp_postmeta` (SEO plugins) | ✅ Published | SEO title for Yoast/SEOPress/AIOSEO |
|
||||
| 5 | **meta_description** | `meta_description` | `seo_description` | `seo_description` OR `meta_description` | `_yoast_wpseo_metadesc`, `_seopress_titles_desc`, `_aioseo_description`, `_igny8_meta_description` | `wp_postmeta` (SEO plugins) | ✅ Published | SEO description for plugins |
|
||||
| 6 | **primary_keyword** | `primary_keyword` | `primary_keyword` | `primary_keyword` | Added to tags | `wp_terms` (post_tag) | ✅ Published | Auto-added as tag if not exists |
|
||||
| 7 | **secondary_keywords** | `secondary_keywords` | `secondary_keywords` (JSON array) | `secondary_keywords` (JSON array) | Added to tags | `wp_terms` (post_tag) | ✅ Published | Each keyword becomes a tag |
|
||||
| 8 | **taxonomy_terms (category)** | `taxonomy_terms` (M2M) | `categories` (array of names) | `categories` (array) | `category_ids` → `wp_set_post_terms()` | `wp_term_relationships` (category) | ✅ Published | Retrieved via `taxonomy_terms.filter(taxonomy_type='category')` |
|
||||
| 9 | **taxonomy_terms (tag)** | `taxonomy_terms` (M2M) | `tags` (array of names) | `tags` (array) | `tag_ids` → `wp_set_post_terms()` | `wp_term_relationships` (post_tag) | ✅ Published | Retrieved via `taxonomy_terms.filter(taxonomy_type='tag')` |
|
||||
| 10 | **cluster** | `cluster_id` (FK) | `cluster_id`, `categories` (fallback) | `cluster_id` | `_igny8_cluster_id` (post_meta), `igny8_clusters` (taxonomy) | `wp_postmeta`, `wp_term_relationships` | ✅ Published | Used as category fallback if no taxonomy_terms exist |
|
||||
| 11 | **sector** | `sector_id` (FK) | `sector_id` | `sector_id` | `_igny8_sector_id` (post_meta), `igny8_sectors` (taxonomy) | `wp_postmeta`, `wp_term_relationships` | ✅ Published | Custom taxonomy |
|
||||
| 12 | **Images (featured)** | `Images` model (FK) | `featured_image_url` | `featured_image_url` OR `featured_image` | `_thumbnail_id` (post_meta) | `wp_postmeta`, `wp_posts` (attachment) | ✅ Published | Via `igny8_set_featured_image()` - downloads/attaches image |
|
||||
| 13 | **Images (gallery)** | `Images` model (FK) | `gallery_images` (array) | `gallery_images` (array) | Gallery processing | `wp_postmeta`, `wp_posts` (attachments) | ✅ Published | Via `igny8_set_gallery_images()` - downloads/attaches images |
|
||||
| 14 | **status** | `status` | `status` | `status` | `post_status` | `wp_posts.post_status` | ✅ Published | Mapped via `igny8_map_igny8_status_to_wp()` (draft/published/review→publish/draft) |
|
||||
| 15 | **published_at** | `published_at` | NOT SENT | `published_at` (optional) | `post_date`, `post_date_gmt` | `wp_posts.post_date`, `wp_posts.post_date_gmt` | ⚠️ Available | Not currently sent by backend |
|
||||
| 16 | **excerpt** | N/A | `excerpt` (generated from content_html) | `excerpt` | `post_excerpt` | `wp_posts.post_excerpt` | ✅ Published | Auto-generated: first 150 chars of stripped HTML |
|
||||
| 17 | **content_type** | `content_type` | `content_type` (optional) | `content_type` (optional) | `_igny8_content_type` (post_meta) | `wp_postmeta` | ✅ Published | Tracking field (e.g., "blog_post", "product_page") |
|
||||
| 18 | **content_structure** | `content_structure` | `content_structure` (optional) | `content_structure` (optional) | `_igny8_content_structure` (post_meta) | `wp_postmeta` | ✅ Published | Tracking field |
|
||||
| 19 | **source** | `source` | NOT SENT | `source` (optional) | `_igny8_source` (post_meta) | `wp_postmeta` | ⚠️ Available | Not currently sent by backend |
|
||||
| 20 | **author** | `user_id` (FK) | NOT SENT | `author` (optional) | `post_author` | `wp_posts.post_author` | ⚠️ Available | Not sent - WordPress uses fallback author mapping |
|
||||
| 21 | **word_count** | `word_count` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | Could be sent as post_meta |
|
||||
| 22 | **optimization_scores** | `optimization_scores` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | JSON field - could be sent as post_meta |
|
||||
| 23 | **metadata** | `metadata` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | JSON field - could be sent as post_meta |
|
||||
| 24 | **internal_links** | `internal_links` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | JSON field - could be sent as post_meta |
|
||||
| 25 | **external_id** | `external_id` | NOT SENT (return value) | N/A | `post_id` (return) | `wp_posts.ID` | ✅ Return Value | WordPress returns this to IGNY8, saved to content.external_id |
|
||||
| 26 | **external_url** | `external_url` | NOT SENT (return value) | N/A | `post_url` (return) | `get_permalink()` | ✅ Return Value | WordPress returns this to IGNY8, saved to content.external_url |
|
||||
| 27 | **external_metadata** | `external_metadata` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | JSON field - not sent |
|
||||
| 28 | **task_id** | N/A | NOT SENT | `task_id` (optional) | `_igny8_task_id` (post_meta) | `wp_postmeta` | ⚠️ Available | Not sent by backend, but WordPress accepts it |
|
||||
| 29 | **brief** | `brief` | NOT SENT | `brief` (optional) | Used for excerpt fallback | `wp_posts.post_excerpt` | ⚠️ Available | Not sent, but would be used as excerpt if sent |
|
||||
| 30 | **slug** | `slug` | NOT SENT | N/A | N/A | `wp_posts.post_name` | ❌ Not Published | WordPress auto-generates slug |
|
||||
| 31 | **ai_content_brief** | `ai_content_brief` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | IGNY8-internal field |
|
||||
| 32 | **ai_response_raw** | `ai_response_raw` | NOT SENT | N/A | N/A | N/A | ❌ Not Published | IGNY8-internal field |
|
||||
|
||||
---
|
||||
|
||||
## 📋 Status Legend
|
||||
|
||||
- ✅ **Published** - Field is actively sent and saved to WordPress
|
||||
- ⚠️ **Available** - Field is accepted by WordPress but not currently sent by IGNY8 backend
|
||||
- ❌ **Not Published** - Field is not sent and not used in publishing workflow
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Authentication & Endpoints
|
||||
|
||||
### Backend API Endpoint
|
||||
```
|
||||
POST /v1/publisher/publish/
|
||||
Authorization: Token <user_token>
|
||||
|
||||
Request Body:
|
||||
{
|
||||
"destination_id": 123,
|
||||
"content_ids": [456]
|
||||
}
|
||||
```
|
||||
|
||||
### WordPress Plugin Endpoint
|
||||
```
|
||||
POST {site_url}/wp-json/igny8/v1/publish
|
||||
Headers:
|
||||
X-IGNY8-API-KEY: <api_key>
|
||||
Content-Type: application/json
|
||||
|
||||
Request Body:
|
||||
{
|
||||
"content_id": 456,
|
||||
"title": "...",
|
||||
"content_html": "...",
|
||||
"seo_title": "...",
|
||||
"seo_description": "...",
|
||||
"categories": ["Category 1", "Category 2"],
|
||||
"tags": ["tag1", "tag2", "keyword"],
|
||||
"featured_image_url": "https://...",
|
||||
"gallery_images": [{"url": "...", "alt": "...", "caption": "..."}],
|
||||
"status": "publish",
|
||||
"primary_keyword": "...",
|
||||
"secondary_keywords": ["...", "..."],
|
||||
"cluster_id": 789,
|
||||
"sector_id": 101,
|
||||
"content_type": "blog_post",
|
||||
"content_structure": "..."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ WordPress Database Mapping
|
||||
|
||||
### Core Post Data (`wp_posts`)
|
||||
- `post_title` ← title
|
||||
- `post_content` ← content_html (sanitized)
|
||||
- `post_excerpt` ← excerpt (auto-generated)
|
||||
- `post_status` ← status (mapped: draft→draft, published→publish, review→publish)
|
||||
- `post_type` ← resolved from content_type (default: "post")
|
||||
- `post_author` ← mapped from WordPress author settings
|
||||
- `post_date` ← published_at (if provided)
|
||||
- `post_date_gmt` ← published_at (converted to GMT)
|
||||
|
||||
### Post Meta (`wp_postmeta`)
|
||||
- `_igny8_content_id` ← content_id
|
||||
- `_igny8_cluster_id` ← cluster_id
|
||||
- `_igny8_sector_id` ← sector_id
|
||||
- `_igny8_content_type` ← content_type
|
||||
- `_igny8_content_structure` ← content_structure
|
||||
- `_igny8_meta_title` ← meta_title
|
||||
- `_igny8_meta_description` ← meta_description
|
||||
- `_yoast_wpseo_title` ← meta_title (Yoast SEO)
|
||||
- `_yoast_wpseo_metadesc` ← meta_description (Yoast SEO)
|
||||
- `_seopress_titles_title` ← meta_title (SEOPress)
|
||||
- `_seopress_titles_desc` ← meta_description (SEOPress)
|
||||
- `_aioseo_title` ← meta_title (AIOSEO)
|
||||
- `_aioseo_description` ← meta_description (AIOSEO)
|
||||
- `_thumbnail_id` ← featured image attachment ID
|
||||
|
||||
### Taxonomies (`wp_term_relationships`, `wp_terms`)
|
||||
- `category` ← categories array + cluster.name (fallback)
|
||||
- `post_tag` ← tags array + primary_keyword + secondary_keywords
|
||||
- `igny8_clusters` ← cluster_id (custom taxonomy)
|
||||
- `igny8_sectors` ← sector_id (custom taxonomy)
|
||||
|
||||
### Attachments (`wp_posts` type=attachment)
|
||||
- Featured image ← downloaded from featured_image_url
|
||||
- Gallery images ← downloaded from gallery_images[].url
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Backend Code Locations
|
||||
|
||||
### Publisher Service
|
||||
**File**: `backend/igny8_core/business/publishing/services/publisher_service.py`
|
||||
- `PublisherService._publish_to_destination()` - Main orchestration
|
||||
- Updates `content.external_id`, `content.external_url`, `content.status='published'` after success
|
||||
|
||||
### WordPress Adapter
|
||||
**File**: `backend/igny8_core/business/publishing/services/adapters/wordpress_adapter.py`
|
||||
- `WordPressAdapter.publish()` - Entry point
|
||||
- `WordPressAdapter._publish_via_api_key()` - Prepares payload and sends to WordPress
|
||||
- **Line 147-160**: Extracts optional fields (meta_title, meta_description, keywords, cluster, sector)
|
||||
- **Line 162-178**: Gets categories from `taxonomy_terms.filter(taxonomy_type='category')`
|
||||
- **Line 180-212**: Gets tags from `taxonomy_terms.filter(taxonomy_type='tag')` + primary_keyword + secondary_keywords
|
||||
- **Line 214-253**: Gets images (featured + gallery) from `Images` model
|
||||
|
||||
### Publisher ViewSet
|
||||
**File**: `backend/igny8_core/modules/writer/views.py`
|
||||
- `PublisherViewSet.publish()` - API endpoint handler
|
||||
|
||||
---
|
||||
|
||||
## 🎯 WordPress Plugin Code Locations
|
||||
|
||||
### REST API Endpoint
|
||||
**File**: `igny8-wp-plugin/includes/class-igny8-rest-api.php`
|
||||
- **Line 90**: Registers `/wp-json/igny8/v1/publish` endpoint
|
||||
- **Line 490-631**: `publish_content_to_wordpress()` - Validates request, logs data, calls creation function
|
||||
|
||||
### Post Creation Function
|
||||
**File**: `igny8-wp-plugin/sync/igny8-to-wp.php`
|
||||
- **Line 73-300+**: `igny8_create_wordpress_post_from_task()` - Main creation function
|
||||
- **Line 106**: Prepares `content_html` (accepts `content_html` or legacy `content`)
|
||||
- **Line 112**: Maps author via `igny8_map_content_author()`
|
||||
- **Line 118-131**: Creates post data array with title, content, excerpt, status, type, author
|
||||
- **Line 133-138**: Sets publication date if provided
|
||||
- **Line 141-175**: Adds IGNY8 metadata (_igny8_content_id, _igny8_cluster_id, etc.)
|
||||
- **Line 178**: Creates post with `wp_insert_post()`
|
||||
- **Line 186**: Imports SEO metadata via `igny8_import_seo_metadata()`
|
||||
- **Line 189**: Imports featured image via `igny8_import_featured_image()`
|
||||
- **Line 192**: Imports taxonomies via `igny8_import_taxonomies()`
|
||||
- **Line 195**: Imports/processes content images via `igny8_import_content_images()`
|
||||
- **Line 209-238**: Processes and assigns categories via `igny8_process_categories()`
|
||||
- **Line 241-258**: Processes and assigns tags via `igny8_process_tags()`
|
||||
- **Line 261-273**: Sets featured image from URL
|
||||
- **Line 274-299**: Sets SEO meta fields for Yoast/SEOPress/AIOSEO
|
||||
|
||||
---
|
||||
|
||||
## 📈 Field Usage Statistics
|
||||
|
||||
### Currently Published: 19 fields
|
||||
- Core content: title, content_html, excerpt, status
|
||||
- SEO: meta_title, meta_description, primary_keyword, secondary_keywords
|
||||
- Taxonomies: categories (via taxonomy_terms), tags (via taxonomy_terms), cluster (fallback), sector
|
||||
- Images: featured_image_url, gallery_images
|
||||
- Tracking: content_id, cluster_id, sector_id, content_type, content_structure
|
||||
|
||||
### Available But Unused: 4 fields
|
||||
- published_at (WordPress accepts, backend doesn't send)
|
||||
- source (WordPress accepts, backend doesn't send)
|
||||
- author (WordPress has fallback mapping, backend doesn't send)
|
||||
- task_id (WordPress accepts, backend doesn't send)
|
||||
|
||||
### Not Published: 9 fields
|
||||
- word_count, optimization_scores, metadata, internal_links
|
||||
- slug (WordPress auto-generates)
|
||||
- ai_content_brief, ai_response_raw (IGNY8-internal)
|
||||
- external_id, external_url (return values from WordPress)
|
||||
- external_metadata (not sent)
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Recent Enhancements (2025-12-01)
|
||||
|
||||
### Content Status Update
|
||||
After successful WordPress publish, IGNY8 backend now updates:
|
||||
```python
|
||||
content.status = 'published'
|
||||
content.save()
|
||||
```
|
||||
**File**: `publisher_service.py` line 184
|
||||
|
||||
### Error Handling Improvements
|
||||
Frontend now shows proper error messages instead of "undefined":
|
||||
```typescript
|
||||
errorMessage = "Publishing failed"
|
||||
```
|
||||
**File**: `frontend/src/pages/Writer/Review.tsx`
|
||||
|
||||
### Taxonomy Integration Fixed
|
||||
Categories and tags now properly extracted from `ContentTaxonomy` M2M relationship:
|
||||
```python
|
||||
categories = [term.name for term in content.taxonomy_terms.filter(taxonomy_type='category')]
|
||||
tags = [term.name for term in content.taxonomy_terms.filter(taxonomy_type='tag')]
|
||||
```
|
||||
**File**: `wordpress_adapter.py` lines 162-178, 180-212
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Verification & Testing
|
||||
|
||||
### Check WordPress Logs
|
||||
WordPress plugin logs to `/wp-content/uploads/igny8-logs/`:
|
||||
```bash
|
||||
tail -f /path/to/wordpress/wp-content/uploads/igny8-logs/igny8-YYYY-MM-DD.log
|
||||
```
|
||||
|
||||
### Check IGNY8 Celery Logs
|
||||
```bash
|
||||
docker logs -f igny8_celery_worker
|
||||
```
|
||||
|
||||
### Verify Published Post
|
||||
After publishing, IGNY8 saves:
|
||||
- `content.external_id` = WordPress post_id
|
||||
- `content.external_url` = WordPress post URL
|
||||
- `content.status` = 'published'
|
||||
|
||||
### Database Queries
|
||||
|
||||
**IGNY8 Backend**:
|
||||
```sql
|
||||
-- Check content with taxonomies
|
||||
SELECT c.id, c.title, c.status, c.external_id, c.external_url,
|
||||
COUNT(DISTINCT ctr.id) as taxonomy_count
|
||||
FROM igny8_content c
|
||||
LEFT JOIN igny8_content_taxonomy_relations ctr ON c.id = ctr.content_id
|
||||
WHERE c.id = 456
|
||||
GROUP BY c.id;
|
||||
|
||||
-- Check taxonomy terms for content
|
||||
SELECT ct.name, ct.taxonomy_type, ct.slug
|
||||
FROM igny8_content_taxonomy_terms ct
|
||||
JOIN igny8_content_taxonomy_relations ctr ON ct.id = ctr.taxonomy_id
|
||||
WHERE ctr.content_id = 456;
|
||||
```
|
||||
|
||||
**WordPress**:
|
||||
```sql
|
||||
-- Check post and meta
|
||||
SELECT p.ID, p.post_title, p.post_status, pm.meta_key, pm.meta_value
|
||||
FROM wp_posts p
|
||||
LEFT JOIN wp_postmeta pm ON p.ID = pm.post_id
|
||||
WHERE pm.meta_key = '_igny8_content_id' AND pm.meta_value = '456';
|
||||
|
||||
-- Check categories and tags
|
||||
SELECT t.name, tt.taxonomy
|
||||
FROM wp_terms t
|
||||
JOIN wp_term_taxonomy tt ON t.term_id = tt.term_id
|
||||
JOIN wp_term_relationships tr ON tt.term_taxonomy_id = tr.term_taxonomy_id
|
||||
WHERE tr.object_id = <post_id>;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💡 Recommendations
|
||||
|
||||
### Potentially Useful Fields to Publish
|
||||
1. **published_at** - Allow scheduled publishing
|
||||
2. **author** - Map IGNY8 user to WordPress author
|
||||
3. **word_count** - Store as post_meta for analytics
|
||||
4. **optimization_scores** - Store as post_meta for content quality tracking
|
||||
|
||||
### Fields to Keep Internal
|
||||
- `ai_content_brief`, `ai_response_raw` - IGNY8-specific, no value in WordPress
|
||||
- `external_metadata` - Already used for other destinations (Shopify, Sites)
|
||||
- `metadata` - Generic field, may contain IGNY8-specific data
|
||||
|
||||
---
|
||||
|
||||
## 📚 Related Documentation
|
||||
- [WordPress Integration Fixes 2025-11-30](./WORDPRESS-INTEGRATION-FIXES-2025-11-30.md)
|
||||
- [WordPress Integration Fixes Implementation 2025-12-01](./WORDPRESS-INTEGRATION-FIXES-IMPLEMENTATION-2025-12-01.md)
|
||||
- [WordPress Bidirectional Sync Reference](./04-WORDPRESS-BIDIRECTIONAL-SYNC-REFERENCE.md)
|
||||
- [WordPress Plugin API Integration Guide](./03-WORDPRESS-PLUGIN-API-INTEGRATION-GUIDE.md)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,744 +0,0 @@
|
||||
# IGNY8 WordPress Plugin - Complete Refactor Plan
|
||||
|
||||
**Created**: 2025-12-01
|
||||
**Scope**: Simplify to one-way publishing only, remove all automatic sync, fix broken features
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Refactor Goals
|
||||
|
||||
1. **One-Way Publishing Only**: IGNY8 → WordPress, no bidirectional sync
|
||||
2. **Immediate Response**: WordPress returns all IDs right after post creation
|
||||
3. **Clean UI**: Proper meta boxes for keywords, SEO, and tracking data
|
||||
4. **Pure Taxonomies**: Cluster/sector as taxonomies only (remove post_meta duplication)
|
||||
5. **Working Images**: Fix gallery image saving
|
||||
6. **User Control**: Draft vs publish setting in WP admin
|
||||
7. **Remove Complexity**: Delete all automatic sync, cron jobs, hooks
|
||||
|
||||
---
|
||||
|
||||
## 📋 Refactor Tasks
|
||||
|
||||
### Phase 1: Remove Automatic Sync (Clean Up)
|
||||
|
||||
#### Task 1.1: Delete Sync Hooks
|
||||
**File**: `sync/hooks.php`
|
||||
**Action**: Delete entire file
|
||||
**Reason**: All automatic sync hooks removed
|
||||
|
||||
**Files to remove:**
|
||||
```
|
||||
sync/hooks.php (DELETE)
|
||||
sync/post-sync.php (DELETE - bidirectional sync)
|
||||
sync/taxonomy-sync.php (DELETE - bidirectional sync)
|
||||
```
|
||||
|
||||
**Code to remove from other files:**
|
||||
- Remove `require_once 'sync/hooks.php'` from main plugin file
|
||||
- Remove all cron job registrations
|
||||
- Remove all `save_post`, `publish_post`, `transition_post_status` hooks
|
||||
|
||||
---
|
||||
|
||||
#### Task 1.2: Remove Brief Meta Box
|
||||
**File**: `admin/class-post-meta-boxes.php`
|
||||
**Changes**:
|
||||
```php
|
||||
// REMOVE these lines from add_meta_boxes() method:
|
||||
add_meta_box(
|
||||
'igny8-planner-brief',
|
||||
__('IGNY8 Planner Brief', 'igny8-bridge'),
|
||||
array($this, 'render_planner_brief_box'),
|
||||
$post_type,
|
||||
'side',
|
||||
'default'
|
||||
);
|
||||
|
||||
// REMOVE entire method:
|
||||
public function render_planner_brief_box($post) { ... }
|
||||
|
||||
// REMOVE AJAX handlers:
|
||||
add_action('wp_ajax_igny8_fetch_planner_brief', array($this, 'fetch_planner_brief'));
|
||||
add_action('wp_ajax_igny8_refresh_planner_task', array($this, 'refresh_planner_task'));
|
||||
|
||||
// REMOVE methods:
|
||||
public function fetch_planner_brief() { ... }
|
||||
public function refresh_planner_task() { ... }
|
||||
```
|
||||
|
||||
**Reason**: No brief data exists in IGNY8
|
||||
|
||||
---
|
||||
|
||||
#### Task 1.3: Clean Up task_id References
|
||||
**Investigation needed**: Determine if `_igny8_task_id` is:
|
||||
- Writer task (remove completely)
|
||||
- Celery task for async operations (keep for tracking)
|
||||
|
||||
**Action**: If writer task, remove all references to `_igny8_task_id`
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Fix Core Publishing
|
||||
|
||||
#### Task 2.1: Fix Gallery Images Function
|
||||
**File**: `sync/igny8-to-wp.php`
|
||||
**Current**: Line 290 calls `igny8_set_gallery_images()` but function is named `igny8_set_image_gallery()`
|
||||
|
||||
**Fix**:
|
||||
```php
|
||||
// Option 1: Rename function call
|
||||
if (!empty($content_data['gallery_images'])) {
|
||||
Igny8_Logger::info("{$log_prefix} STEP: Setting gallery with " . count($content_data['gallery_images']) . " images");
|
||||
igny8_set_image_gallery($post_id, $content_data['gallery_images']); // Changed from igny8_set_gallery_images
|
||||
}
|
||||
|
||||
// OR Option 2: Add alias function
|
||||
function igny8_set_gallery_images($post_id, $gallery_images) {
|
||||
return igny8_set_image_gallery($post_id, $gallery_images);
|
||||
}
|
||||
```
|
||||
|
||||
**Test**: Verify gallery images are saved to `_igny8_gallery_images` post_meta
|
||||
|
||||
---
|
||||
|
||||
#### Task 2.2: Fix Cluster/Sector Storage
|
||||
**File**: `sync/igny8-to-wp.php`
|
||||
**Current**: Lines 141-175 save cluster_id and sector_id as post_meta
|
||||
|
||||
**Remove these lines**:
|
||||
```php
|
||||
// REMOVE (lines ~163-175):
|
||||
if (!empty($content_data['cluster_id'])) {
|
||||
$post_data['meta_input']['_igny8_cluster_id'] = $content_data['cluster_id'];
|
||||
}
|
||||
|
||||
if (!empty($content_data['sector_id'])) {
|
||||
$post_data['meta_input']['_igny8_sector_id'] = $content_data['sector_id'];
|
||||
}
|
||||
```
|
||||
|
||||
**Keep only taxonomy assignment** (lines ~195-230):
|
||||
```php
|
||||
// KEEP: This correctly assigns taxonomies
|
||||
if (!empty($content_data['cluster_id'])) {
|
||||
$cluster_terms = get_terms(array(
|
||||
'taxonomy' => 'igny8_clusters',
|
||||
'meta_key' => '_igny8_cluster_id',
|
||||
'meta_value' => $content_data['cluster_id'],
|
||||
'hide_empty' => false
|
||||
));
|
||||
|
||||
if (!is_wp_error($cluster_terms) && !empty($cluster_terms)) {
|
||||
wp_set_post_terms($post_id, array($cluster_terms[0]->term_id), 'igny8_clusters');
|
||||
}
|
||||
}
|
||||
// Same for sector...
|
||||
```
|
||||
|
||||
**Issue**: This searches for terms by meta_key, but terms need to exist first!
|
||||
|
||||
**Better approach**:
|
||||
```php
|
||||
if (!empty($content_data['cluster_id'])) {
|
||||
// Get cluster name from IGNY8 (need to send cluster_name in payload)
|
||||
$cluster_name = $content_data['cluster_name'] ?? '';
|
||||
|
||||
if (!empty($cluster_name)) {
|
||||
$term = wp_insert_term($cluster_name, 'igny8_clusters', array(
|
||||
'slug' => sanitize_title($cluster_name)
|
||||
));
|
||||
|
||||
if (!is_wp_error($term)) {
|
||||
// Store IGNY8 cluster_id as term meta for future lookups
|
||||
update_term_meta($term['term_id'], '_igny8_cluster_id', $content_data['cluster_id']);
|
||||
wp_set_post_terms($post_id, array($term['term_id']), 'igny8_clusters');
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Backend change needed**: `wordpress_adapter.py` must send `cluster_name` and `sector_name` in payload
|
||||
|
||||
---
|
||||
|
||||
#### Task 2.3: Add Draft/Publish Setting
|
||||
**File**: `admin/settings.php`
|
||||
**Add new setting field**:
|
||||
|
||||
```php
|
||||
// In settings registration (around line ~100):
|
||||
add_settings_field(
|
||||
'igny8_default_post_status',
|
||||
__('Default Post Status', 'igny8-bridge'),
|
||||
'igny8_render_default_post_status_field',
|
||||
'igny8-settings',
|
||||
'igny8_settings_section'
|
||||
);
|
||||
|
||||
// Add field renderer:
|
||||
function igny8_render_default_post_status_field() {
|
||||
$status = get_option('igny8_default_post_status', 'draft');
|
||||
?>
|
||||
<fieldset>
|
||||
<label>
|
||||
<input type="radio" name="igny8_default_post_status" value="draft" <?php checked($status, 'draft'); ?>>
|
||||
<?php _e('Draft - Save as draft for review', 'igny8-bridge'); ?>
|
||||
</label><br>
|
||||
<label>
|
||||
<input type="radio" name="igny8_default_post_status" value="publish" <?php checked($status, 'publish'); ?>>
|
||||
<?php _e('Publish - Publish immediately', 'igny8-bridge'); ?>
|
||||
</label>
|
||||
<p class="description">
|
||||
<?php _e('Choose whether content from IGNY8 should be published immediately or saved as draft.', 'igny8-bridge'); ?>
|
||||
</p>
|
||||
</fieldset>
|
||||
<?php
|
||||
}
|
||||
```
|
||||
|
||||
**File**: `sync/igny8-to-wp.php`
|
||||
**Use settings in post creation** (line ~122):
|
||||
|
||||
```php
|
||||
// OLD:
|
||||
'post_status' => igny8_map_igny8_status_to_wp($content_data['status'] ?? 'draft'),
|
||||
|
||||
// NEW:
|
||||
'post_status' => get_option('igny8_default_post_status', 'draft'),
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Task 2.4: Return All Term IDs Immediately
|
||||
**File**: `includes/class-igny8-rest-api.php`
|
||||
**Modify `publish_content_to_wordpress()` return** (around line 615):
|
||||
|
||||
```php
|
||||
// After post creation (line ~605), collect term IDs:
|
||||
$term_ids = array(
|
||||
'categories' => array(),
|
||||
'tags' => array(),
|
||||
'igny8_clusters' => array(),
|
||||
'igny8_sectors' => array()
|
||||
);
|
||||
|
||||
// Get assigned category IDs
|
||||
$category_terms = wp_get_post_terms($post_id, 'category', array('fields' => 'ids'));
|
||||
if (!is_wp_error($category_terms)) {
|
||||
$term_ids['categories'] = $category_terms;
|
||||
}
|
||||
|
||||
// Get assigned tag IDs
|
||||
$tag_terms = wp_get_post_terms($post_id, 'post_tag', array('fields' => 'ids'));
|
||||
if (!is_wp_error($tag_terms)) {
|
||||
$term_ids['tags'] = $tag_terms;
|
||||
}
|
||||
|
||||
// Get assigned cluster IDs
|
||||
$cluster_terms = wp_get_post_terms($post_id, 'igny8_clusters', array('fields' => 'ids'));
|
||||
if (!is_wp_error($cluster_terms)) {
|
||||
$term_ids['igny8_clusters'] = $cluster_terms;
|
||||
}
|
||||
|
||||
// Get assigned sector IDs
|
||||
$sector_terms = wp_get_post_terms($post_id, 'igny8_sectors', array('fields' => 'ids'));
|
||||
if (!is_wp_error($sector_terms)) {
|
||||
$term_ids['igny8_sectors'] = $sector_terms;
|
||||
}
|
||||
|
||||
// Return enhanced response:
|
||||
return $this->build_unified_response(
|
||||
true,
|
||||
array(
|
||||
'post_id' => $post_id,
|
||||
'post_url' => get_permalink($post_id),
|
||||
'post_status' => get_post_status($post_id),
|
||||
'content_id' => $content_id,
|
||||
'task_id' => $task_id,
|
||||
'term_ids' => $term_ids // NEW
|
||||
),
|
||||
'Content successfully published to WordPress',
|
||||
null,
|
||||
null,
|
||||
201
|
||||
);
|
||||
```
|
||||
|
||||
**Backend change needed**: `wordpress_adapter.py` must capture and save `term_ids` from response
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Add Custom Meta Boxes
|
||||
|
||||
#### Task 3.1: Add IGNY8 Keywords Meta Box
|
||||
**File**: `admin/class-post-meta-boxes.php`
|
||||
**Add meta box registration**:
|
||||
|
||||
```php
|
||||
public function add_meta_boxes() {
|
||||
$post_types = array('post', 'page', 'product');
|
||||
|
||||
foreach ($post_types as $post_type) {
|
||||
// NEW: IGNY8 Keywords
|
||||
add_meta_box(
|
||||
'igny8-keywords',
|
||||
__('IGNY8 Keywords', 'igny8-bridge'),
|
||||
array($this, 'render_keywords_box'),
|
||||
$post_type,
|
||||
'side',
|
||||
'high'
|
||||
);
|
||||
|
||||
// NEW: IGNY8 SEO
|
||||
add_meta_box(
|
||||
'igny8-seo',
|
||||
__('IGNY8 SEO', 'igny8-bridge'),
|
||||
array($this, 'render_seo_box'),
|
||||
$post_type,
|
||||
'normal',
|
||||
'high'
|
||||
);
|
||||
|
||||
// NEW: IGNY8 Sync Data (read-only)
|
||||
add_meta_box(
|
||||
'igny8-sync-data',
|
||||
__('IGNY8 Sync Data', 'igny8-bridge'),
|
||||
array($this, 'render_sync_data_box'),
|
||||
$post_type,
|
||||
'side',
|
||||
'low'
|
||||
);
|
||||
|
||||
// KEEP: IGNY8 Optimizer (existing)
|
||||
add_meta_box(
|
||||
'igny8-optimizer',
|
||||
__('IGNY8 Optimizer', 'igny8-bridge'),
|
||||
array($this, 'render_optimizer_box'),
|
||||
$post_type,
|
||||
'side',
|
||||
'default'
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Add render methods**:
|
||||
|
||||
```php
|
||||
/**
|
||||
* Render Keywords meta box
|
||||
*/
|
||||
public function render_keywords_box($post) {
|
||||
$primary_keyword = get_post_meta($post->ID, '_igny8_primary_keyword', true);
|
||||
$secondary_keywords = get_post_meta($post->ID, '_igny8_secondary_keywords', true);
|
||||
|
||||
// Decode JSON if needed
|
||||
if (is_string($secondary_keywords)) {
|
||||
$secondary_keywords = json_decode($secondary_keywords, true);
|
||||
}
|
||||
if (!is_array($secondary_keywords)) {
|
||||
$secondary_keywords = array();
|
||||
}
|
||||
|
||||
wp_nonce_field('igny8_keywords_nonce', 'igny8_keywords_nonce');
|
||||
?>
|
||||
<div class="igny8-keywords-box">
|
||||
<p>
|
||||
<label for="igny8_primary_keyword">
|
||||
<strong><?php _e('Primary Keyword', 'igny8-bridge'); ?></strong>
|
||||
</label>
|
||||
<input type="text"
|
||||
id="igny8_primary_keyword"
|
||||
name="igny8_primary_keyword"
|
||||
value="<?php echo esc_attr($primary_keyword); ?>"
|
||||
class="widefat"
|
||||
placeholder="<?php _e('Enter primary keyword', 'igny8-bridge'); ?>">
|
||||
</p>
|
||||
|
||||
<p>
|
||||
<label for="igny8_secondary_keywords">
|
||||
<strong><?php _e('Secondary Keywords', 'igny8-bridge'); ?></strong>
|
||||
</label>
|
||||
<input type="text"
|
||||
id="igny8_secondary_keywords"
|
||||
name="igny8_secondary_keywords"
|
||||
value="<?php echo esc_attr(implode(', ', $secondary_keywords)); ?>"
|
||||
class="widefat"
|
||||
placeholder="<?php _e('keyword1, keyword2, keyword3', 'igny8-bridge'); ?>">
|
||||
<span class="description"><?php _e('Separate keywords with commas', 'igny8-bridge'); ?></span>
|
||||
</p>
|
||||
|
||||
<?php if (!empty($primary_keyword) || !empty($secondary_keywords)) : ?>
|
||||
<p class="description">
|
||||
<em><?php _e('✅ These keywords were set by IGNY8', 'igny8-bridge'); ?></em>
|
||||
</p>
|
||||
<?php endif; ?>
|
||||
</div>
|
||||
<?php
|
||||
}
|
||||
|
||||
/**
|
||||
* Render SEO meta box
|
||||
*/
|
||||
public function render_seo_box($post) {
|
||||
$meta_title = get_post_meta($post->ID, '_igny8_meta_title', true);
|
||||
$meta_description = get_post_meta($post->ID, '_igny8_meta_description', true);
|
||||
|
||||
wp_nonce_field('igny8_seo_nonce', 'igny8_seo_nonce');
|
||||
?>
|
||||
<div class="igny8-seo-box">
|
||||
<p>
|
||||
<label for="igny8_meta_title">
|
||||
<strong><?php _e('SEO Title', 'igny8-bridge'); ?></strong>
|
||||
</label>
|
||||
<input type="text"
|
||||
id="igny8_meta_title"
|
||||
name="igny8_meta_title"
|
||||
value="<?php echo esc_attr($meta_title); ?>"
|
||||
class="widefat"
|
||||
placeholder="<?php _e('Enter SEO title', 'igny8-bridge'); ?>">
|
||||
<span class="description"><?php _e('Recommended: 50-60 characters', 'igny8-bridge'); ?></span>
|
||||
</p>
|
||||
|
||||
<p>
|
||||
<label for="igny8_meta_description">
|
||||
<strong><?php _e('SEO Meta Description', 'igny8-bridge'); ?></strong>
|
||||
</label>
|
||||
<textarea id="igny8_meta_description"
|
||||
name="igny8_meta_description"
|
||||
rows="3"
|
||||
class="widefat"
|
||||
placeholder="<?php _e('Enter meta description', 'igny8-bridge'); ?>"><?php echo esc_textarea($meta_description); ?></textarea>
|
||||
<span class="description"><?php _e('Recommended: 150-160 characters', 'igny8-bridge'); ?></span>
|
||||
</p>
|
||||
|
||||
<?php if (!empty($meta_title) || !empty($meta_description)) : ?>
|
||||
<p class="description">
|
||||
<em><?php _e('✅ These SEO fields were set by IGNY8', 'igny8-bridge'); ?></em>
|
||||
</p>
|
||||
<?php endif; ?>
|
||||
</div>
|
||||
<?php
|
||||
}
|
||||
|
||||
/**
|
||||
* Render Sync Data meta box (read-only)
|
||||
*/
|
||||
public function render_sync_data_box($post) {
|
||||
$content_id = get_post_meta($post->ID, '_igny8_content_id', true);
|
||||
$content_type = get_post_meta($post->ID, '_igny8_content_type', true);
|
||||
$content_structure = get_post_meta($post->ID, '_igny8_content_structure', true);
|
||||
$cluster_id = get_post_meta($post->ID, '_igny8_cluster_id', true);
|
||||
$sector_id = get_post_meta($post->ID, '_igny8_sector_id', true);
|
||||
|
||||
?>
|
||||
<div class="igny8-sync-data-box">
|
||||
<table class="widefat striped">
|
||||
<tbody>
|
||||
<?php if ($content_id) : ?>
|
||||
<tr>
|
||||
<td><strong><?php _e('Content ID', 'igny8-bridge'); ?></strong></td>
|
||||
<td><code><?php echo esc_html($content_id); ?></code></td>
|
||||
</tr>
|
||||
<?php endif; ?>
|
||||
|
||||
<?php if ($content_type) : ?>
|
||||
<tr>
|
||||
<td><strong><?php _e('Content Type', 'igny8-bridge'); ?></strong></td>
|
||||
<td><?php echo esc_html($content_type); ?></td>
|
||||
</tr>
|
||||
<?php endif; ?>
|
||||
|
||||
<?php if ($content_structure) : ?>
|
||||
<tr>
|
||||
<td><strong><?php _e('Structure', 'igny8-bridge'); ?></strong></td>
|
||||
<td><?php echo esc_html($content_structure); ?></td>
|
||||
</tr>
|
||||
<?php endif; ?>
|
||||
|
||||
<?php if ($cluster_id) : ?>
|
||||
<tr>
|
||||
<td><strong><?php _e('Cluster ID', 'igny8-bridge'); ?></strong></td>
|
||||
<td><code><?php echo esc_html($cluster_id); ?></code></td>
|
||||
</tr>
|
||||
<?php endif; ?>
|
||||
|
||||
<?php if ($sector_id) : ?>
|
||||
<tr>
|
||||
<td><strong><?php _e('Sector ID', 'igny8-bridge'); ?></strong></td>
|
||||
<td><code><?php echo esc_html($sector_id); ?></code></td>
|
||||
</tr>
|
||||
<?php endif; ?>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<?php if ($content_id) : ?>
|
||||
<p class="description">
|
||||
<em><?php _e('This post was published from IGNY8', 'igny8-bridge'); ?></em>
|
||||
</p>
|
||||
<?php else : ?>
|
||||
<p class="description">
|
||||
<em><?php _e('This post was not created by IGNY8', 'igny8-bridge'); ?></em>
|
||||
</p>
|
||||
<?php endif; ?>
|
||||
</div>
|
||||
<?php
|
||||
}
|
||||
```
|
||||
|
||||
**Add save handlers**:
|
||||
|
||||
```php
|
||||
/**
|
||||
* Save keywords when post is saved
|
||||
*/
|
||||
public function save_keywords($post_id) {
|
||||
// Check nonce
|
||||
if (!isset($_POST['igny8_keywords_nonce']) || !wp_verify_nonce($_POST['igny8_keywords_nonce'], 'igny8_keywords_nonce')) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Check autosave
|
||||
if (defined('DOING_AUTOSAVE') && DOING_AUTOSAVE) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Check permissions
|
||||
if (!current_user_can('edit_post', $post_id)) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Save primary keyword
|
||||
if (isset($_POST['igny8_primary_keyword'])) {
|
||||
update_post_meta($post_id, '_igny8_primary_keyword', sanitize_text_field($_POST['igny8_primary_keyword']));
|
||||
}
|
||||
|
||||
// Save secondary keywords
|
||||
if (isset($_POST['igny8_secondary_keywords'])) {
|
||||
$keywords = sanitize_text_field($_POST['igny8_secondary_keywords']);
|
||||
$keywords_array = array_map('trim', explode(',', $keywords));
|
||||
$keywords_array = array_filter($keywords_array); // Remove empty
|
||||
update_post_meta($post_id, '_igny8_secondary_keywords', json_encode($keywords_array));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Save SEO fields when post is saved
|
||||
*/
|
||||
public function save_seo($post_id) {
|
||||
// Check nonce
|
||||
if (!isset($_POST['igny8_seo_nonce']) || !wp_verify_nonce($_POST['igny8_seo_nonce'], 'igny8_seo_nonce')) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Check autosave
|
||||
if (defined('DOING_AUTOSAVE') && DOING_AUTOSAVE) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Check permissions
|
||||
if (!current_user_can('edit_post', $post_id)) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Save meta title
|
||||
if (isset($_POST['igny8_meta_title'])) {
|
||||
$meta_title = sanitize_text_field($_POST['igny8_meta_title']);
|
||||
update_post_meta($post_id, '_igny8_meta_title', $meta_title);
|
||||
|
||||
// Also update SEO plugin fields
|
||||
update_post_meta($post_id, '_yoast_wpseo_title', $meta_title);
|
||||
update_post_meta($post_id, '_seopress_titles_title', $meta_title);
|
||||
update_post_meta($post_id, '_aioseo_title', $meta_title);
|
||||
}
|
||||
|
||||
// Save meta description
|
||||
if (isset($_POST['igny8_meta_description'])) {
|
||||
$meta_description = sanitize_textarea_field($_POST['igny8_meta_description']);
|
||||
update_post_meta($post_id, '_igny8_meta_description', $meta_description);
|
||||
|
||||
// Also update SEO plugin fields
|
||||
update_post_meta($post_id, '_yoast_wpseo_metadesc', $meta_description);
|
||||
update_post_meta($post_id, '_seopress_titles_desc', $meta_description);
|
||||
update_post_meta($post_id, '_aioseo_description', $meta_description);
|
||||
}
|
||||
}
|
||||
|
||||
// Register save handlers in constructor:
|
||||
public function __construct() {
|
||||
add_action('add_meta_boxes', array($this, 'add_meta_boxes'));
|
||||
add_action('save_post', array($this, 'save_keywords'), 10, 1);
|
||||
add_action('save_post', array($this, 'save_seo'), 10, 1);
|
||||
add_action('admin_enqueue_scripts', array($this, 'enqueue_scripts'));
|
||||
|
||||
// Keep optimizer AJAX handler
|
||||
add_action('wp_ajax_igny8_create_optimizer_job', array($this, 'create_optimizer_job'));
|
||||
add_action('wp_ajax_igny8_get_optimizer_status', array($this, 'get_optimizer_status'));
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Backend Changes (IGNY8 Django)
|
||||
|
||||
#### Task 4.1: Send cluster_name and sector_name
|
||||
**File**: `backend/igny8_core/business/publishing/services/adapters/wordpress_adapter.py`
|
||||
**Add to payload** (around line 172):
|
||||
|
||||
```python
|
||||
# Current:
|
||||
if hasattr(content, 'cluster') and content.cluster:
|
||||
content_data['cluster_id'] = content.cluster.id
|
||||
optional_fields.append('cluster_id')
|
||||
|
||||
# NEW:
|
||||
if hasattr(content, 'cluster') and content.cluster:
|
||||
content_data['cluster_id'] = content.cluster.id
|
||||
content_data['cluster_name'] = content.cluster.name # ADD THIS
|
||||
optional_fields.append('cluster_id')
|
||||
|
||||
if hasattr(content, 'sector') and content.sector:
|
||||
content_data['sector_id'] = content.sector.id
|
||||
content_data['sector_name'] = content.sector.name # ADD THIS
|
||||
optional_fields.append('sector_id')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Task 4.2: Capture and Save term_ids from Response
|
||||
**File**: `backend/igny8_core/business/publishing/services/adapters/wordpress_adapter.py`
|
||||
**Modify response handling** (around line 300):
|
||||
|
||||
```python
|
||||
if response.status_code == 201:
|
||||
wp_data = response.json().get('data', {})
|
||||
logger.info(f"[WordPressAdapter._publish_via_api_key] ✅ Success! WordPress post created: post_id={wp_data.get('post_id')}, url={wp_data.get('post_url')}")
|
||||
|
||||
# NEW: Extract term_ids
|
||||
term_ids = wp_data.get('term_ids', {})
|
||||
|
||||
return {
|
||||
'success': True,
|
||||
'external_id': str(wp_data.get('post_id')),
|
||||
'url': wp_data.get('post_url'),
|
||||
'published_at': datetime.now(),
|
||||
'metadata': {
|
||||
'post_id': wp_data.get('post_id'),
|
||||
'status': destination_config.get('status', 'publish'),
|
||||
'term_ids': term_ids # NEW: Save term mappings
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Optional**: Store term_ids in `content.external_metadata` JSON field for future reference
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ File Structure After Refactor
|
||||
|
||||
```
|
||||
igny8-wp-plugin/
|
||||
├── igny8-bridge.php (main plugin file)
|
||||
├── uninstall.php
|
||||
├── admin/
|
||||
│ ├── class-admin.php
|
||||
│ ├── class-admin-columns.php
|
||||
│ ├── class-post-meta-boxes.php ✅ UPDATED (new meta boxes, remove brief)
|
||||
│ ├── settings.php ✅ UPDATED (add draft/publish setting)
|
||||
│ └── assets/
|
||||
├── includes/
|
||||
│ ├── class-igny8-api.php
|
||||
│ ├── class-igny8-rest-api.php ✅ UPDATED (return term_ids)
|
||||
│ ├── class-igny8-logger.php
|
||||
│ ├── class-igny8-webhooks.php (KEEP for future)
|
||||
│ └── functions.php ✅ UPDATED (ensure taxonomies registered)
|
||||
├── sync/
|
||||
│ ├── igny8-to-wp.php ✅ UPDATED (fix gallery, remove post_meta for cluster/sector)
|
||||
│ ├── hooks.php ❌ DELETE
|
||||
│ ├── post-sync.php ❌ DELETE
|
||||
│ └── taxonomy-sync.php ❌ DELETE
|
||||
├── data/ (keep for site collection/link graph)
|
||||
└── tests/ (keep for testing)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Testing Checklist
|
||||
|
||||
### After Refactor, Test:
|
||||
|
||||
1. **Publish from IGNY8**:
|
||||
- ✅ Post created in WordPress
|
||||
- ✅ Title, content, excerpt correct
|
||||
- ✅ Categories assigned correctly
|
||||
- ✅ Tags assigned correctly
|
||||
- ✅ Featured image downloaded and set
|
||||
- ✅ Gallery images downloaded and saved to post_meta
|
||||
- ✅ Cluster assigned as taxonomy (no post_meta)
|
||||
- ✅ Sector assigned as taxonomy (no post_meta)
|
||||
- ✅ Primary keyword saved to `_igny8_primary_keyword`
|
||||
- ✅ Secondary keywords saved to `_igny8_secondary_keywords` (JSON)
|
||||
- ✅ Meta title saved to SEO plugin fields
|
||||
- ✅ Meta description saved to SEO plugin fields
|
||||
|
||||
2. **WordPress Response**:
|
||||
- ✅ Returns post_id immediately
|
||||
- ✅ Returns post_url immediately
|
||||
- ✅ Returns term_ids for all taxonomies
|
||||
- ✅ IGNY8 backend saves external_id, external_url, status='published'
|
||||
|
||||
3. **WordPress Editor UI**:
|
||||
- ✅ "IGNY8 Keywords" meta box shows primary + secondary keywords (editable)
|
||||
- ✅ "IGNY8 SEO" meta box shows meta_title + meta_description (editable)
|
||||
- ✅ "IGNY8 Sync Data" meta box shows all tracking fields (read-only)
|
||||
- ✅ NO "IGNY8 Planner Brief" meta box
|
||||
- ✅ Cluster and sector show in taxonomy sidebars
|
||||
- ✅ Editing keywords/SEO saves correctly
|
||||
|
||||
4. **WordPress Settings**:
|
||||
- ✅ "Default Post Status" option exists (draft/publish radio)
|
||||
- ✅ Changing setting affects next publish
|
||||
|
||||
5. **No Automatic Sync**:
|
||||
- ✅ Editing post in WordPress does NOT trigger API call to IGNY8
|
||||
- ✅ Publishing post in WordPress does NOT trigger sync
|
||||
- ✅ No cron jobs running
|
||||
|
||||
---
|
||||
|
||||
## 📝 Summary of Changes
|
||||
|
||||
### Removed:
|
||||
- ❌ All automatic sync hooks (save_post, publish_post, etc.)
|
||||
- ❌ Bidirectional sync files (post-sync.php, taxonomy-sync.php, hooks.php)
|
||||
- ❌ Brief meta box (no data in IGNY8)
|
||||
- ❌ Cron jobs for sync
|
||||
- ❌ Post_meta storage for cluster_id and sector_id
|
||||
|
||||
### Fixed:
|
||||
- ✅ Gallery images function name (`igny8_set_gallery_images` → `igny8_set_image_gallery`)
|
||||
- ✅ Cluster/sector stored ONLY as taxonomies (with term_meta for IGNY8 ID mapping)
|
||||
|
||||
### Added:
|
||||
- ✅ "IGNY8 Keywords" meta box (primary_keyword, secondary_keywords)
|
||||
- ✅ "IGNY8 SEO" meta box (meta_title, meta_description)
|
||||
- ✅ "IGNY8 Sync Data" meta box (read-only tracking fields)
|
||||
- ✅ WP admin setting: "Default Post Status" (draft/publish)
|
||||
- ✅ WordPress returns term_ids in publish response
|
||||
- ✅ Backend sends cluster_name and sector_name for taxonomy creation
|
||||
- ✅ Backend saves term_ids from WordPress response
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Implementation Order
|
||||
|
||||
1. **Phase 1** (Clean up): Remove sync files, hooks, brief meta box
|
||||
2. **Phase 2** (Fix core): Gallery images, cluster/sector, draft setting, response enhancement
|
||||
3. **Phase 3** (Add UI): New meta boxes for keywords, SEO, sync data
|
||||
4. **Phase 4** (Backend): Update WordPress adapter to send names and capture term_ids
|
||||
|
||||
**Estimated time**: 4-6 hours
|
||||
**Risk level**: Low (mostly removing code, fixing bugs, adding UI)
|
||||
**Testing requirement**: High (verify all fields save correctly)
|
||||
@@ -1,253 +0,0 @@
|
||||
# WordPress Publishing - Simplified Field Reference
|
||||
|
||||
**Last Updated**: 2025-12-01
|
||||
**Purpose**: Simple reference for IGNY8 → WordPress one-way publishing
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Publishing Flow (One-Way Only)
|
||||
|
||||
```
|
||||
IGNY8 Review.tsx
|
||||
↓ User clicks "Publish to WordPress"
|
||||
↓ POST /v1/publisher/publish/
|
||||
Backend WordPressAdapter
|
||||
↓ POST {site_url}/wp-json/igny8/v1/publish (with all content data)
|
||||
WordPress Plugin
|
||||
↓ wp_insert_post() + taxonomies + images + meta
|
||||
↓ IMMEDIATE RETURN: {post_id, post_url, term_ids}
|
||||
IGNY8 Backend
|
||||
↓ Saves: external_id, external_url, status='published'
|
||||
✅ DONE - No automatic sync, no bidirectional updates
|
||||
```
|
||||
|
||||
**Publishing Timing**: Immediate (synchronous) - WordPress returns data right after post creation
|
||||
|
||||
---
|
||||
|
||||
## 📊 Published Fields Reference
|
||||
|
||||
| Field | IGNY8 → WordPress | WordPress Stores As | Notes |
|
||||
|-------|-------------------|---------------------|-------|
|
||||
| **CORE CONTENT** |
|
||||
| title | `title` | `wp_posts.post_title` | Required |
|
||||
| content_html | `content_html` | `wp_posts.post_content` | Required, sanitized |
|
||||
| status | `status` | `wp_posts.post_status` | User-configurable: draft or publish (WP admin setting) |
|
||||
| **SEO FIELDS** (Need Meta Boxes) |
|
||||
| meta_title | `seo_title` OR `meta_title` | `_yoast_wpseo_title`, `_seopress_titles_title`, `_aioseo_title`, `_igny8_meta_title` | **⚠️ No UI in WP editor yet** |
|
||||
| meta_description | `seo_description` OR `meta_description` | `_yoast_wpseo_metadesc`, `_seopress_titles_desc`, `_aioseo_description`, `_igny8_meta_description` | **⚠️ No UI in WP editor yet** |
|
||||
| **KEYWORDS** (Need Meta Boxes) |
|
||||
| primary_keyword | `primary_keyword` | `_igny8_primary_keyword` (post_meta) + added as tag | **⚠️ Should be custom field, not just tag** |
|
||||
| secondary_keywords | `secondary_keywords` (JSON array) | `_igny8_secondary_keywords` (post_meta) + each added as tag | **⚠️ Should be custom field, not just tag** |
|
||||
| **TAXONOMIES** |
|
||||
| categories (from taxonomy_terms) | `categories` (array of names) | `wp_term_relationships` (category taxonomy) | ✅ Working |
|
||||
| tags (from taxonomy_terms) | `tags` (array of names) | `wp_term_relationships` (post_tag taxonomy) | ✅ Working |
|
||||
| **CUSTOM TAXONOMIES** (Currently Broken) |
|
||||
| cluster | `cluster_id` | ~~`_igny8_cluster_id` (post_meta)~~ + `igny8_clusters` (taxonomy) | **⚠️ Currently saved as post_meta, should ONLY be taxonomy** |
|
||||
| sector | `sector_id` | ~~`_igny8_sector_id` (post_meta)~~ + `igny8_sectors` (taxonomy) | **⚠️ Currently saved as post_meta, should ONLY be taxonomy** |
|
||||
| **IMAGES** |
|
||||
| featured_image | `featured_image_url` | `_thumbnail_id` (post_meta) → attachment | ✅ Working |
|
||||
| gallery_images | `gallery_images` (array) | `_igny8_gallery_images`, `_product_image_gallery`, `_gallery_images` | **❌ BROKEN - Not saving** |
|
||||
| **TRACKING FIELDS** (Show in Meta Box) |
|
||||
| content_id | `content_id` | `_igny8_content_id` (post_meta) | ✅ Saved, **⚠️ Not visible in editor** |
|
||||
| content_type | `content_type` | `_igny8_content_type` (post_meta) | ✅ Saved, **⚠️ Not visible in editor** |
|
||||
| content_structure | `content_structure` | `_igny8_content_structure` (post_meta) | ✅ Saved, **⚠️ Not visible in editor** |
|
||||
| **AUTO-GENERATED** |
|
||||
| excerpt | Generated from content_html (first 150 chars) | `wp_posts.post_excerpt` | ✅ Working |
|
||||
| **RETURN VALUES** (WordPress → IGNY8) |
|
||||
| - | `post_id` | Returned immediately | Saved to `content.external_id` |
|
||||
| - | `post_url` | Returned immediately | Saved to `content.external_url` |
|
||||
| - | `category_term_ids` | Returned immediately | **⚠️ Need to return for mapping** |
|
||||
| - | `tag_term_ids` | Returned immediately | **⚠️ Need to return for mapping** |
|
||||
| - | `cluster_term_id` | Returned immediately | **⚠️ Need to return for mapping** |
|
||||
| - | `sector_term_id` | Returned immediately | **⚠️ Need to return for mapping** |
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Required Fixes
|
||||
|
||||
### 1. **Add Meta Boxes in WordPress Post Editor**
|
||||
**Missing UI for these 4 fields:**
|
||||
- `primary_keyword` - Custom field (text input)
|
||||
- `secondary_keywords` - Custom field (tag-style input)
|
||||
- `meta_title` - SEO field (text input)
|
||||
- `meta_description` - SEO field (textarea)
|
||||
|
||||
**Action**: Create new meta boxes in `class-post-meta-boxes.php`
|
||||
|
||||
---
|
||||
|
||||
### 2. **Fix Cluster/Sector as Pure Taxonomies**
|
||||
**Current**: Saved as both post_meta AND taxonomy
|
||||
**Required**: Save ONLY as taxonomy, remove post_meta storage
|
||||
|
||||
**Code locations to fix:**
|
||||
- `sync/igny8-to-wp.php` lines 141-175 (remove `_igny8_cluster_id`, `_igny8_sector_id` post_meta)
|
||||
- Keep only `wp_set_post_terms()` for `igny8_clusters` and `igny8_sectors` taxonomies
|
||||
|
||||
---
|
||||
|
||||
### 3. **Fix Gallery Images Not Saving**
|
||||
**Current**: Function `igny8_set_gallery_images()` doesn't exist, only `igny8_set_image_gallery()` exists
|
||||
**Issue**: Called at line 290 but function is named differently
|
||||
|
||||
**Action**: Fix function name or create alias
|
||||
|
||||
---
|
||||
|
||||
### 4. **Add Draft/Publish Setting in WP Admin**
|
||||
**Location**: `admin/settings.php`
|
||||
**Add option**: "Default status for IGNY8 content" → Radio: Draft / Publish
|
||||
**Use in**: `igny8_create_wordpress_post_from_task()` to override `post_status`
|
||||
|
||||
---
|
||||
|
||||
### 5. **Show All IGNY8 Meta in Post Editor**
|
||||
**Create meta box displaying:**
|
||||
- Content ID: `_igny8_content_id`
|
||||
- Content Type: `_igny8_content_type`
|
||||
- Content Structure: `_igny8_content_structure`
|
||||
- Cluster ID: `_igny8_cluster_id` (until removed)
|
||||
- Sector ID: `_igny8_sector_id` (until removed)
|
||||
- All other `_igny8_*` meta fields
|
||||
|
||||
**Action**: Add "IGNY8 Sync Data" meta box (read-only)
|
||||
|
||||
---
|
||||
|
||||
### 6. **Immediate Response with All IDs**
|
||||
**Current return**:
|
||||
```json
|
||||
{
|
||||
"post_id": 123,
|
||||
"post_url": "https://...",
|
||||
"post_status": "draft"
|
||||
}
|
||||
```
|
||||
|
||||
**Required return**:
|
||||
```json
|
||||
{
|
||||
"post_id": 123,
|
||||
"post_url": "https://...",
|
||||
"post_status": "draft",
|
||||
"term_ids": {
|
||||
"categories": [45, 67],
|
||||
"tags": [12, 34, 56],
|
||||
"igny8_clusters": [89],
|
||||
"igny8_sectors": [101]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Action**: Modify `publish_content_to_wordpress()` to collect and return all term IDs
|
||||
|
||||
---
|
||||
|
||||
### 7. **Remove All Automatic Sync Code**
|
||||
**Remove these files/functions:**
|
||||
- `sync/hooks.php` - All `save_post`, `publish_post`, cron hooks
|
||||
- `sync/post-sync.php` - All WordPress → IGNY8 sync functions
|
||||
- `sync/taxonomy-sync.php` - Bidirectional taxonomy sync
|
||||
- Cron jobs: `igny8_sync_post_statuses`, `igny8_sync_from_igny8`, etc.
|
||||
|
||||
**Keep only:**
|
||||
- One-way publish: IGNY8 → WordPress via `/wp-json/igny8/v1/publish`
|
||||
- Immediate response after post creation
|
||||
|
||||
---
|
||||
|
||||
### 8. **Remove Brief Meta Box**
|
||||
**Current**: `class-post-meta-boxes.php` has "IGNY8 Planner Brief" meta box
|
||||
**Issue**: No `brief` data exists in IGNY8
|
||||
**Action**: Remove entire brief meta box and related AJAX handlers
|
||||
|
||||
---
|
||||
|
||||
### 9. **Clean Up task_id Usage**
|
||||
**Current**: `_igny8_task_id` stored in post_meta
|
||||
**Question**: Is this Celery task_id or writer task_id?
|
||||
**Action**: If writer task, remove. If Celery, keep for tracking async operations.
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ WordPress Database After Publishing
|
||||
|
||||
### Post Data
|
||||
```
|
||||
wp_posts:
|
||||
- post_title (from title)
|
||||
- post_content (from content_html)
|
||||
- post_excerpt (auto-generated)
|
||||
- post_status (draft or publish, from WP admin setting)
|
||||
- post_author (mapped)
|
||||
```
|
||||
|
||||
### Post Meta
|
||||
```
|
||||
wp_postmeta:
|
||||
- _igny8_content_id (tracking)
|
||||
- _igny8_content_type (tracking)
|
||||
- _igny8_content_structure (tracking)
|
||||
- _igny8_primary_keyword (NEW - custom field)
|
||||
- _igny8_secondary_keywords (NEW - custom field, JSON)
|
||||
- _igny8_meta_title (SEO)
|
||||
- _igny8_meta_description (SEO)
|
||||
- _yoast_wpseo_title (SEO plugin compatibility)
|
||||
- _yoast_wpseo_metadesc (SEO plugin compatibility)
|
||||
- _thumbnail_id (featured image)
|
||||
- _igny8_gallery_images (gallery attachment IDs)
|
||||
```
|
||||
|
||||
### Taxonomies
|
||||
```
|
||||
wp_term_relationships:
|
||||
- category (standard WP categories)
|
||||
- post_tag (standard WP tags)
|
||||
- igny8_clusters (custom taxonomy)
|
||||
- igny8_sectors (custom taxonomy)
|
||||
```
|
||||
|
||||
### Attachments
|
||||
```
|
||||
wp_posts (type=attachment):
|
||||
- Featured image (downloaded from featured_image_url)
|
||||
- Gallery images (downloaded from gallery_images[].url)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Code Locations
|
||||
|
||||
### Backend (IGNY8)
|
||||
- **Publisher**: `backend/igny8_core/business/publishing/services/adapters/wordpress_adapter.py`
|
||||
- **Line 147-253**: Prepares payload with all fields
|
||||
|
||||
### WordPress Plugin
|
||||
- **Endpoint**: `includes/class-igny8-rest-api.php` line 490
|
||||
- **Post Creation**: `sync/igny8-to-wp.php` line 73
|
||||
- **Taxonomies**: `includes/functions.php` line 465 (registration)
|
||||
- **Meta Boxes**: `admin/class-post-meta-boxes.php`
|
||||
- **Settings**: `admin/settings.php`
|
||||
|
||||
---
|
||||
|
||||
## ✅ What Works Now
|
||||
- ✅ Post creation with title, content, excerpt
|
||||
- ✅ Categories and tags from taxonomy_terms
|
||||
- ✅ Featured image download and attachment
|
||||
- ✅ SEO meta saved to plugin fields
|
||||
- ✅ Cluster/sector taxonomies registered
|
||||
|
||||
## ❌ What's Broken
|
||||
- ❌ Gallery images not saving (function name mismatch)
|
||||
- ❌ No UI for primary_keyword, secondary_keywords, meta_title, meta_description
|
||||
- ❌ Cluster/sector saved as both post_meta AND taxonomy (should be taxonomy only)
|
||||
- ❌ No visibility of IGNY8 tracking fields in WP editor
|
||||
- ❌ Brief meta box exists but has no data
|
||||
- ❌ Automatic sync hooks causing unnecessary API calls
|
||||
|
||||
## ⚠️ What's Missing
|
||||
- ⚠️ User can't choose draft vs publish in WP admin settings
|
||||
- ⚠️ WordPress doesn't return term_ids for mapping
|
||||
- ⚠️ Keywords stored as tags only, not as custom fields for editing
|
||||
@@ -1,437 +0,0 @@
|
||||
# IGNY8 WordPress Plugin Refactor - IMPLEMENTATION COMPLETE
|
||||
|
||||
**Date**: 2025-12-01
|
||||
**Status**: ✅ 95% Complete (1 manual task remaining)
|
||||
|
||||
---
|
||||
|
||||
## ✅ Completed Changes
|
||||
|
||||
### Phase 1: Remove Automatic Sync ✅
|
||||
|
||||
**Files Modified:**
|
||||
- `igny8-bridge.php` - Updated to v1.1.0, removed sync file includes, removed cron scheduling
|
||||
|
||||
**Changes:**
|
||||
```php
|
||||
// REMOVED: sync/hooks.php, sync/post-sync.php, sync/taxonomy-sync.php includes
|
||||
// REMOVED: igny8_schedule_cron_jobs() and igny8_unschedule_cron_jobs()
|
||||
// ADDED: Default post status option initialization
|
||||
```
|
||||
|
||||
**Files to Delete Manually:**
|
||||
- `sync/hooks.php` (no longer included)
|
||||
- `sync/post-sync.php` (no longer included)
|
||||
- `sync/taxonomy-sync.php` (no longer included)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Core Publishing Fixes ✅
|
||||
|
||||
#### 2.1 Gallery Images Function Fix ✅
|
||||
**File**: `sync/igny8-to-wp.php` line 290
|
||||
```php
|
||||
// FIXED: Function call mismatch
|
||||
igny8_set_image_gallery($post_id, $content_data['gallery_images']);
|
||||
// Was: igny8_set_gallery_images() (doesn't exist)
|
||||
```
|
||||
|
||||
#### 2.2 Cluster/Sector as Pure Taxonomies ✅
|
||||
**File**: `sync/igny8-to-wp.php` lines 141-230
|
||||
|
||||
**Removed:**
|
||||
- `_igny8_cluster_id` post_meta storage
|
||||
- `_igny8_sector_id` post_meta storage
|
||||
|
||||
**Added:**
|
||||
- Auto-create cluster terms with `cluster_name` from IGNY8
|
||||
- Auto-create sector terms with `sector_name` from IGNY8
|
||||
- Store IGNY8 IDs in term_meta (`_igny8_cluster_id`, `_igny8_sector_id`)
|
||||
- Term lookups by IGNY8 ID for future publishes
|
||||
|
||||
#### 2.3 Default Post Status Setting ✅
|
||||
**Files:**
|
||||
- `admin/class-admin.php` - Added `igny8_default_post_status` setting
|
||||
- `admin/settings.php` - Added UI (radio buttons: Draft/Publish)
|
||||
- `sync/igny8-to-wp.php` - Uses `get_option('igny8_default_post_status', 'draft')`
|
||||
|
||||
**UI Location**: Settings → Automation Settings → "Default Post Status for IGNY8 Content"
|
||||
|
||||
#### 2.4 Keywords Saved as Custom Fields ✅
|
||||
**File**: `sync/igny8-to-wp.php` lines 141-175
|
||||
|
||||
**Added:**
|
||||
- `_igny8_primary_keyword` post_meta
|
||||
- `_igny8_secondary_keywords` post_meta (JSON)
|
||||
|
||||
These are now editable in meta boxes (see Phase 3).
|
||||
|
||||
#### 2.5 Return Term IDs Immediately ✅
|
||||
**Files:**
|
||||
- `sync/igny8-to-wp.php` - Collects all term IDs after post creation, returns array
|
||||
- `includes/class-igny8-rest-api.php` - Handles new return format, includes term_ids in response
|
||||
|
||||
**Response Format:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"post_id": 123,
|
||||
"post_url": "https://...",
|
||||
"post_status": "draft",
|
||||
"content_id": 456,
|
||||
"term_ids": {
|
||||
"categories": [45, 67],
|
||||
"tags": [12, 34, 56],
|
||||
"igny8_clusters": [89],
|
||||
"igny8_sectors": [101]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 2.6 Removed Automatic Task Updates ✅
|
||||
**File**: `sync/igny8-to-wp.php`
|
||||
|
||||
**Removed:**
|
||||
- Automatic IGNY8 task update after post creation
|
||||
- Webhook sending
|
||||
- All bidirectional sync code
|
||||
|
||||
**Result**: WordPress only responds with post data, no automatic callbacks to IGNY8.
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Meta Boxes (MANUAL TASK REQUIRED) ⚠️
|
||||
|
||||
**File Prepared**: New version of `admin/class-post-meta-boxes.php` created
|
||||
|
||||
**Manual Steps Required:**
|
||||
1. Backup current file: `cp admin/class-post-meta-boxes.php admin/class-post-meta-boxes.php.bak`
|
||||
2. Replace with new version (provided separately)
|
||||
|
||||
**New Meta Boxes:**
|
||||
1. **IGNY8 Keywords** (side, high priority)
|
||||
- Primary Keyword (text input)
|
||||
- Secondary Keywords (comma-separated input)
|
||||
- Saves to `_igny8_primary_keyword` and `_igny8_secondary_keywords`
|
||||
|
||||
2. **IGNY8 SEO** (normal, high priority)
|
||||
- SEO Title (text input, 50-60 chars recommended)
|
||||
- Meta Description (textarea, 150-160 chars recommended)
|
||||
- Saves to `_igny8_meta_title`, `_igny8_meta_description`
|
||||
- Also updates Yoast, SEOPress, AIOSEO fields
|
||||
|
||||
3. **IGNY8 Sync Data** (side, low priority - read-only)
|
||||
- Content ID
|
||||
- Content Type
|
||||
- Content Structure
|
||||
- Displays "Published from IGNY8" or "Not created by IGNY8"
|
||||
|
||||
**Removed:**
|
||||
- IGNY8 Planner Brief meta box (no data in IGNY8)
|
||||
- Related AJAX handlers
|
||||
|
||||
**Kept:**
|
||||
- IGNY8 Optimizer meta box (existing functionality)
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Backend (Django) Updates ✅
|
||||
|
||||
#### 4.1 Send Cluster/Sector Names ✅
|
||||
**File**: `backend/igny8_core/business/publishing/services/adapters/wordpress_adapter.py`
|
||||
|
||||
**Added to Payload:**
|
||||
```python
|
||||
content_data['cluster_name'] = content.cluster.name # NEW
|
||||
content_data['sector_name'] = content.sector.name # NEW
|
||||
```
|
||||
|
||||
WordPress now receives names to create taxonomy terms if they don't exist.
|
||||
|
||||
#### 4.2 Capture Term IDs ✅
|
||||
**Files:**
|
||||
- `wordpress_adapter.py` - Extracts `term_ids` from WordPress response
|
||||
- `publisher_service.py` - Saves `term_ids` to `content.external_metadata['wordpress_term_ids']`
|
||||
|
||||
**Saved Data Structure:**
|
||||
```python
|
||||
content.external_metadata = {
|
||||
'wordpress_term_ids': {
|
||||
'categories': [45, 67],
|
||||
'tags': [12, 34, 56],
|
||||
'igny8_clusters': [89],
|
||||
'igny8_sectors': [101]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ Files Modified Summary
|
||||
|
||||
### WordPress Plugin (PHP)
|
||||
1. ✅ `igny8-bridge.php` - Removed sync, updated version to 1.1.0
|
||||
2. ✅ `admin/class-admin.php` - Added post_status setting registration
|
||||
3. ✅ `admin/settings.php` - Added UI for post_status setting
|
||||
4. ⚠️ `admin/class-post-meta-boxes.php` - **NEEDS MANUAL REPLACEMENT**
|
||||
5. ✅ `sync/igny8-to-wp.php` - Gallery fix, cluster/sector taxonomies, keywords, term IDs
|
||||
6. ✅ `includes/class-igny8-rest-api.php` - Return term_ids in response
|
||||
|
||||
### Django Backend (Python)
|
||||
1. ✅ `wordpress_adapter.py` - Send cluster_name, sector_name, capture term_ids
|
||||
2. ✅ `publisher_service.py` - Save term_ids to external_metadata
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing Required
|
||||
|
||||
### WordPress Side
|
||||
1. ✅ Test publishing from IGNY8 → WordPress
|
||||
2. ✅ Verify post created with correct status (draft/publish based on setting)
|
||||
3. ✅ Check categories assigned correctly
|
||||
4. ✅ Check tags assigned correctly
|
||||
5. ✅ Verify featured image downloaded
|
||||
6. ⚠️ Test gallery images saved to `_igny8_gallery_images` post_meta
|
||||
7. ✅ Verify cluster taxonomy assigned (not post_meta)
|
||||
8. ✅ Verify sector taxonomy assigned (not post_meta)
|
||||
9. ⚠️ Check Keywords meta box shows primary + secondary keywords
|
||||
10. ⚠️ Check SEO meta box shows meta_title + meta_description
|
||||
11. ⚠️ Check Sync Data meta box shows tracking fields
|
||||
12. ✅ Verify WordPress returns term_ids in response
|
||||
|
||||
### Django/IGNY8 Side
|
||||
1. ✅ Verify `content.external_id` saved (WordPress post_id)
|
||||
2. ✅ Verify `content.external_url` saved (WordPress post URL)
|
||||
3. ✅ Verify `content.status` = 'published' after success
|
||||
4. ✅ Verify `content.external_metadata['wordpress_term_ids']` saved
|
||||
|
||||
### WordPress Admin Settings
|
||||
1. ✅ Go to Settings → IGNY8 Bridge
|
||||
2. ✅ Find "Default Post Status for IGNY8 Content"
|
||||
3. ✅ Test changing Draft ↔ Publish
|
||||
4. ✅ Verify next publish uses selected status
|
||||
|
||||
---
|
||||
|
||||
## 📋 Manual Task Checklist
|
||||
|
||||
### Task 1: Replace Meta Boxes File
|
||||
```bash
|
||||
cd /data/app/igny8/igny8-wp-plugin/admin
|
||||
cp class-post-meta-boxes.php class-post-meta-boxes.php.bak
|
||||
# Then replace with new version
|
||||
```
|
||||
|
||||
**New file location**: [File provided separately - 500+ lines]
|
||||
|
||||
**What it does:**
|
||||
- Removes brief meta box
|
||||
- Adds Keywords meta box (primary + secondary)
|
||||
- Adds SEO meta box (meta_title + meta_description)
|
||||
- Adds Sync Data meta box (read-only tracking fields)
|
||||
- Keeps Optimizer meta box
|
||||
|
||||
### Task 2: Delete Obsolete Sync Files
|
||||
```bash
|
||||
cd /data/app/igny8/igny8-wp-plugin/sync
|
||||
rm hooks.php post-sync.php taxonomy-sync.php
|
||||
```
|
||||
|
||||
These files are no longer included in `igny8-bridge.php`.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Deployment Steps
|
||||
|
||||
### 1. Deploy Django Backend
|
||||
```bash
|
||||
cd /data/app/igny8/backend
|
||||
docker exec igny8_backend python manage.py check
|
||||
docker restart igny8_backend
|
||||
docker restart igny8_celery_worker
|
||||
```
|
||||
|
||||
### 2. Deploy WordPress Plugin
|
||||
```bash
|
||||
# SSH to WordPress server
|
||||
cd /path/to/wordpress/wp-content/plugins/igny8-wp-plugin
|
||||
|
||||
# Backup current version
|
||||
tar -czf igny8-wp-plugin-backup-$(date +%Y%m%d).tar.gz igny8-wp-plugin/
|
||||
|
||||
# Upload modified files
|
||||
# - igny8-bridge.php
|
||||
# - admin/class-admin.php
|
||||
# - admin/settings.php
|
||||
# - admin/class-post-meta-boxes.php (NEW VERSION)
|
||||
# - sync/igny8-to-wp.php
|
||||
# - includes/class-igny8-rest-api.php
|
||||
|
||||
# Delete obsolete files
|
||||
rm sync/hooks.php sync/post-sync.php sync/taxonomy-sync.php
|
||||
|
||||
# Verify plugin
|
||||
wp plugin list
|
||||
wp plugin activate igny8-bridge
|
||||
```
|
||||
|
||||
### 3. Configure WordPress Settings
|
||||
1. Go to **Settings → IGNY8 Bridge**
|
||||
2. Under **Automation Settings**, find **"Default Post Status for IGNY8 Content"**
|
||||
3. Choose **Draft** (recommended) or **Publish**
|
||||
4. Click **"Save Automation Settings"**
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Verification Commands
|
||||
|
||||
### Check WordPress Response
|
||||
```bash
|
||||
# Publish content from IGNY8 and check logs
|
||||
docker logs -f igny8_backend | grep "WordPress"
|
||||
```
|
||||
|
||||
Look for:
|
||||
```
|
||||
WordPress response: status=201
|
||||
Term IDs received: {'categories': [...], 'tags': [...], ...}
|
||||
Saved term_ids to external_metadata
|
||||
```
|
||||
|
||||
### Check WordPress Post Meta
|
||||
```sql
|
||||
-- In WordPress database
|
||||
SELECT post_id, meta_key, meta_value
|
||||
FROM wp_postmeta
|
||||
WHERE post_id = <post_id>
|
||||
AND meta_key LIKE '_igny8%';
|
||||
```
|
||||
|
||||
Should show:
|
||||
- `_igny8_content_id`
|
||||
- `_igny8_primary_keyword`
|
||||
- `_igny8_secondary_keywords`
|
||||
- `_igny8_meta_title`
|
||||
- `_igny8_meta_description`
|
||||
- `_igny8_gallery_images` (if gallery exists)
|
||||
|
||||
### Check Taxonomies
|
||||
```sql
|
||||
-- Check cluster/sector assigned as taxonomies (not post_meta)
|
||||
SELECT t.name, tt.taxonomy
|
||||
FROM wp_terms t
|
||||
JOIN wp_term_taxonomy tt ON t.term_id = tt.term_id
|
||||
JOIN wp_term_relationships tr ON tt.term_taxonomy_id = tr.term_taxonomy_id
|
||||
WHERE tr.object_id = <post_id>
|
||||
AND tt.taxonomy IN ('igny8_clusters', 'igny8_sectors');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 What Changed - Summary
|
||||
|
||||
### Removed ❌
|
||||
- All automatic sync hooks (save_post, publish_post, etc.)
|
||||
- All cron jobs (sync_post_statuses, sync_from_igny8, etc.)
|
||||
- Bidirectional sync files (hooks.php, post-sync.php, taxonomy-sync.php)
|
||||
- Brief meta box (no data in IGNY8)
|
||||
- Cluster/sector as post_meta (now pure taxonomies)
|
||||
- Automatic IGNY8 task updates after post creation
|
||||
|
||||
### Added ✅
|
||||
- Default post status setting (draft/publish) in WP admin
|
||||
- Keywords meta box (primary + secondary)
|
||||
- SEO meta box (meta_title + meta_description)
|
||||
- Sync Data meta box (read-only tracking)
|
||||
- Cluster_name and sector_name in publish payload
|
||||
- Term IDs returned from WordPress
|
||||
- Term IDs saved to content.external_metadata
|
||||
- Automatic term creation if cluster/sector don't exist in WordPress
|
||||
|
||||
### Fixed ✅
|
||||
- Gallery images function call (igny8_set_image_gallery)
|
||||
- Cluster/sector stored ONLY as taxonomies (with term_meta for ID mapping)
|
||||
- Keywords stored as custom fields (editable in meta boxes)
|
||||
- Content status updates to 'published' after successful WordPress publish
|
||||
|
||||
---
|
||||
|
||||
## 🎯 End Result
|
||||
|
||||
### Publishing Flow (One-Way Only)
|
||||
```
|
||||
IGNY8 Review.tsx
|
||||
↓ User clicks "Publish to WordPress"
|
||||
↓ POST /v1/publisher/publish/
|
||||
Backend WordPressAdapter
|
||||
↓ POST {site_url}/wp-json/igny8/v1/publish
|
||||
↓ Sends: title, content, SEO, keywords, categories, tags, cluster_name, sector_name, images
|
||||
WordPress Plugin
|
||||
↓ wp_insert_post() + taxonomies + images + meta
|
||||
↓ Creates cluster/sector terms if don't exist
|
||||
↓ IMMEDIATE RETURN: {post_id, post_url, term_ids}
|
||||
IGNY8 Backend
|
||||
↓ Saves: external_id, external_url, status='published', external_metadata['wordpress_term_ids']
|
||||
✅ DONE - No automatic sync, no callbacks
|
||||
```
|
||||
|
||||
### WordPress Editor Experience
|
||||
1. Published content shows 4 meta boxes:
|
||||
- **IGNY8 Keywords** - Edit primary/secondary keywords
|
||||
- **IGNY8 SEO** - Edit SEO title/description
|
||||
- **IGNY8 Sync Data** - View tracking info (read-only)
|
||||
- **IGNY8 Optimizer** - Run optimization jobs
|
||||
|
||||
2. Taxonomies in sidebar:
|
||||
- Categories (standard)
|
||||
- Tags (standard)
|
||||
- IGNY8 Clusters (custom)
|
||||
- IGNY8 Sectors (custom)
|
||||
|
||||
3. Featured image and gallery in Media section
|
||||
|
||||
---
|
||||
|
||||
## 🆘 Troubleshooting
|
||||
|
||||
### Issue: Gallery images not saving
|
||||
**Check**: `igny8_set_image_gallery()` function exists in `sync/igny8-to-wp.php` (line ~780)
|
||||
**Verify**: `_igny8_gallery_images` post_meta contains array of attachment IDs
|
||||
|
||||
### Issue: Cluster/sector not showing as taxonomy
|
||||
**Check**: Terms registered in `includes/functions.php` line 465
|
||||
**Verify**: `igny8_clusters` and `igny8_sectors` taxonomies exist
|
||||
**Run**: `wp taxonomy list` to confirm
|
||||
|
||||
### Issue: Keywords meta box not showing
|
||||
**Action**: Replace `admin/class-post-meta-boxes.php` with new version (manual task)
|
||||
|
||||
### Issue: Term IDs not saving to IGNY8
|
||||
**Check**: WordPress response includes `term_ids` field
|
||||
**Check**: Django logs show "Saved term_ids to external_metadata"
|
||||
**Verify**: `content.external_metadata` column is JSON type in database
|
||||
|
||||
---
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
- Plugin version updated to **1.1.0**
|
||||
- All changes are backward compatible (old posts unaffected)
|
||||
- No database migrations required
|
||||
- Existing meta boxes (Optimizer) preserved
|
||||
- SEO plugin compatibility maintained (Yoast, SEOPress, AIOSEO)
|
||||
|
||||
---
|
||||
|
||||
## ✅ Completion Status
|
||||
|
||||
- **Phase 1**: ✅ 100% Complete
|
||||
- **Phase 2**: ✅ 100% Complete
|
||||
- **Phase 3**: ⚠️ 95% Complete (1 manual file replacement)
|
||||
- **Phase 4**: ✅ 100% Complete
|
||||
|
||||
**Overall**: ✅ 95% Complete
|
||||
|
||||
**Remaining**: Replace `admin/class-post-meta-boxes.php` manually
|
||||
Reference in New Issue
Block a user