stage2-2 and docs
This commit is contained in:
alorig
89
refactor-plan/refactor-stage-2.md
Normal file
89
refactor-plan/refactor-stage-2.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# Stage 2 – Planner & Wizard UX
|
||||
|
||||
## Overview
|
||||
- Feature flag: `USE_SITE_BUILDER_REFACTOR` (must be `true`)
|
||||
- Goal: ship state-aware, guided Site Builder experience and expose cluster/taxonomy readiness to Planner UI
|
||||
- Dependencies: Stage 1 migrations/services already deployed
|
||||
- Entry point: `/sites/builder/workflow/:blueprintId` (protected route)
|
||||
|
||||
---
|
||||
|
||||
## Backend Enhancements
|
||||
|
||||
### New services / endpoints
|
||||
| Component | Description | File |
|
||||
| --- | --- | --- |
|
||||
| `WizardContextService` | Aggregates workflow state, cluster stats, taxonomy summaries, coverage counts | `backend/igny8_core/business/site_building/services/wizard_context_service.py` |
|
||||
| Workflow context API | `GET /api/v1/site-builder/siteblueprint/{id}/workflow/context/` returns `workflow`, `cluster_summary`, `taxonomy_summary`, `coverage`, `next_actions` | `modules/site_builder/views.py` |
|
||||
| Workflow serializer helpers | `SiteBlueprintSerializer` now returns `workflow_state` + `gating_messages` via `WorkflowStateService.serialize_state()` | `modules/site_builder/serializers.py` |
|
||||
|
||||
### Workflow telemetry & logging
|
||||
- `WorkflowStateService` normalizes step metadata (`status`, `code`, `message`, `updated_at`)
|
||||
- Emits structured log events (`wizard_step_updated`, `wizard_blocking_issue`) for future analytics
|
||||
- `serialize_state()` provides consistent payload for API, wizard store, and planner warnings
|
||||
|
||||
---
|
||||
|
||||
## Frontend Implementation
|
||||
|
||||
### API layer
|
||||
- Added interfaces + calls (`fetchSiteBlueprints`, `fetchSiteBlueprintById`, `updateSiteBlueprint`, `fetchWizardContext`, `updateWorkflowStep`)
|
||||
- Located in `frontend/src/services/api.ts`
|
||||
|
||||
### Zustand store
|
||||
| Store | Purpose |
|
||||
| --- | --- |
|
||||
| `useBuilderWorkflowStore` | Tracks `blueprintId`, `currentStep`, `completedSteps`, `blockingIssues`, `workflowState`, `context`, telemetry queue |
|
||||
- Actions: `initialize`, `refreshState`, `goToStep`, `completeStep`, `setBlockingIssue`, `clearBlockingIssue`, `flushTelemetry`, `reset`
|
||||
- Persists last blueprint + step in `builder-workflow-storage` (sessionStorage)
|
||||
|
||||
### Wizard shell & routing
|
||||
- New page: `WorkflowWizard` at `/sites/builder/workflow/:blueprintId`
|
||||
- Lazy-loaded via `App.tsx`
|
||||
- Refreshes context every 10s to keep telemetry/current state accurate
|
||||
- Progress indicator (`WizardProgress`) shows completed/current/blocked steps
|
||||
|
||||
### Step components
|
||||
| Step | Component | Status |
|
||||
| --- | --- | --- |
|
||||
| 1 – Business Details | `steps/BusinessDetailsStep.tsx` | **Functional** (saves blueprint + completes step) |
|
||||
| 2 – Cluster Assignment | `steps/ClusterAssignmentStep.tsx` | Placeholder UI (shows stats, next iteration: attach/detach clusters) |
|
||||
| 3 – Taxonomy Builder | `steps/TaxonomyBuilderStep.tsx` | Placeholder UI (future: taxonomy tree/table + imports) |
|
||||
| 4 – AI Sitemap Review | `steps/SitemapReviewStep.tsx` | Placeholder UI (future: grid w/ grouping + regenerate) |
|
||||
| 5 – Coverage Validation | `steps/CoverageValidationStep.tsx` | Placeholder UI (future: coverage cards + gating logic) |
|
||||
| 6 – Ideas Hand-off | `steps/IdeasHandoffStep.tsx` | Placeholder UI (future: page selection + prompt override) |
|
||||
|
||||
### Planner UX TODOs (next iteration)
|
||||
- Cluster matrix view linking back into wizard
|
||||
- Taxonomy management table with inline edits & imports
|
||||
- Planner dashboard banner if blueprint missing requirements
|
||||
|
||||
---
|
||||
|
||||
## Testing & Verification
|
||||
1. Set `USE_SITE_BUILDER_REFACTOR=true`
|
||||
2. Create or reuse a `SiteBlueprint` and open `/sites/builder/workflow/{id}`
|
||||
3. Confirm payload from `GET /workflow/context/` includes:
|
||||
- `workflow.steps` array with `status`, `code`, `message`
|
||||
- `cluster_summary` (counts + list)
|
||||
- `taxonomy_summary`
|
||||
- `coverage` (pages, statuses)
|
||||
- `next_actions`
|
||||
4. Step 1 (Business Details) should allow update/save → triggers workflow state update
|
||||
5. Steps 2‑6 currently show placeholders + gating alerts (will be wired to new UIs)
|
||||
|
||||
Automated tests pending: once UI solidified, add Cypress flows for each step + Zustand store unit tests.
|
||||
|
||||
---
|
||||
|
||||
## Open Items / Future Work
|
||||
- Build full step experiences (tables, drag/drop mapping, AI sitemap review UI)
|
||||
- Planner cluster matrix & taxonomy management views
|
||||
- Telemetry dispatcher (currently logs to console)
|
||||
- Accessibility polish: keyboard navigation, helper tooltips, context drawer
|
||||
- QA automation + e2e coverage
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-11-19*
|
||||
|
||||
85
refactor-plan/refactor-stage-3.md
Normal file
85
refactor-plan/refactor-stage-3.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Stage 3 – Writer / Linker / Optimizer Enhancements
|
||||
|
||||
## Objective
|
||||
Propagate the new metadata (clusters, taxonomies, entity types, attributes) through the planner → writer pipeline, enforce validation before publish, and unlock linker/optimizer capabilities. Stage 3 builds directly on Stage 2’s wizard outputs; all changes stay behind `USE_SITE_BUILDER_REFACTOR` until pilot-ready.
|
||||
|
||||
---
|
||||
|
||||
## Backend Plan
|
||||
|
||||
### 1. Metadata Audit & Backfill
|
||||
| Task | Description | Owner |
|
||||
| --- | --- | --- |
|
||||
| Backfill tables | Implement data migration using the stub added in `writer/migrations/0012_metadata_mapping_tables.py` to populate `ContentClusterMap`, `ContentTaxonomyMap`, `ContentAttributeMap` for legacy content/tasks. | Backend lead |
|
||||
| Entity defaults | Ensure existing Tasks/Content have sensible defaults for `entity_type`, `taxonomy_id`, `cluster_role` (e.g., `blog_post` + `supporting`). | Backend lead |
|
||||
| Audit script | Management command `python manage.py audit_site_metadata --site {id}` summarizing gaps per site. | Backend lead |
|
||||
|
||||
### 2. Pipeline Updates
|
||||
| Stage | Changes |
|
||||
| --- | --- |
|
||||
| Ideas → Tasks | Update Task creation so every task inherits cluster/taxonomy/attribute metadata. Enforce “no cluster, no idea/task” rule. |
|
||||
| Tasks → Content | Adjust `PageGenerationService` / writer Celery tasks to persist mappings into `ContentClusterMap`, `ContentTaxonomyMap`, `ContentAttributeMap`. |
|
||||
| AI Prompts | Update prompts to include cluster role, taxonomy context, product attributes. Leverage Stage 1 metadata fields like `dimension_meta`, `attribute_values`. |
|
||||
| Validation services | Add reusable validators (e.g., `ensure_required_attributes(task)`), returning structured errors for UI. |
|
||||
|
||||
### 3. Linker & Optimizer
|
||||
| Component | Requirements |
|
||||
| --- | --- |
|
||||
| LinkerService | Use mapping tables to suggest hub ↔ supporting ↔ attribute links; include priority score + context snippet. |
|
||||
| OptimizerService | Scorecards factoring cluster coverage, taxonomy alignment, attribute completeness. Provide `/sites/{id}/progress` and `/writer/content/{id}/validation` endpoints. |
|
||||
| Caching/Indexes | Add DB indexes for frequent join patterns (content ↔ cluster_map, taxonomy_map). |
|
||||
|
||||
### 4. API Additions
|
||||
- `GET /api/v1/sites/{id}/progress/` → cluster-level completion + validation flags.
|
||||
- `GET /api/v1/writer/content/{id}/validation/` → aggregated checklist for Writer UI.
|
||||
- `POST /api/v1/writer/content/{id}/validate/` → re-run validators and return actionable errors.
|
||||
|
||||
---
|
||||
|
||||
## Frontend Plan
|
||||
|
||||
### 1. Planner / Ideas / Writer Enhancements
|
||||
| Area | UX Changes |
|
||||
| --- | --- |
|
||||
| Planner Ideas & Writer Tasks lists | Show chips/columns for cluster, taxonomy, entity type, validation status; add filters. |
|
||||
| Writer Editor | Sidebar module summarizing cluster, taxonomy tree, attribute form; validation panel blocking publish until cleared. |
|
||||
| Linker UI | Group internal link suggestions by cluster role; show context snippet + CTA to insert link. |
|
||||
| Optimizer Dashboard | Scorecards per cluster dimension with color coding + “next action” cards. |
|
||||
| Site Progress Widgets | On site overview, show completion bars (hub/supporting/attribute); deep link to problematic clusters/pages. |
|
||||
|
||||
### 2. Validation & Notifications
|
||||
- Inline toasts + optional email when validation fails.
|
||||
- Publish button disabled until validators pass; display error list linking to relevant fields.
|
||||
- Credit reminder banner when user regenerates content/optimization tasks.
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
| Area | Automated | Manual |
|
||||
| --- | --- | --- |
|
||||
| Pipeline | Unit tests for Task → Content metadata persistence, Prompt builders. | End-to-end: blueprint → ideas → tasks → content. |
|
||||
| Validation | Tests ensuring validators trigger correct errors. | Attempt publish without taxonomy/attributes; confirm UX flow. |
|
||||
| Linker/Optimizer | Service tests for scoring & suggestions. | Performance profiling on large datasets; UX review. |
|
||||
| Progress Widgets | Component tests verifying counts. | Compare UI vs. database for pilot site. |
|
||||
|
||||
---
|
||||
|
||||
## Rollout Checklist
|
||||
1. Deploy Stage 3 backend with feature flag ON in staging.
|
||||
2. Run metadata backfill; verify progress dashboards and validators.
|
||||
3. QA regression: planner → writer flow, linker, optimizer.
|
||||
4. Pilot with internal content team; gather feedback on validation friction.
|
||||
5. Optimize slow queries (indexes/caching) before production rollout.
|
||||
6. Update training docs/videos for writers; enable flag gradually across accounts.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions / Risks
|
||||
- Volume impact when backfilling large content sets? (Plan: chunked migrations + progress logs)
|
||||
- Telemetry volume for validator events? (Plan: aggregate counts; sample per site)
|
||||
- WordPress deploy parity: ensure metadata travels through sync (handled in Stage 4).
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-11-19*
|
||||
|
||||
89
refactor-plan/refactor-stage-4.md
Normal file
89
refactor-plan/refactor-stage-4.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# Stage 4 – Publishing & Sync Integration
|
||||
|
||||
## Objective
|
||||
Achieve feature parity between IGNY8-hosted deployments and WordPress sites using the shared metadata model introduced in Stages 1–3. This includes two-way sync for taxonomies/products, deployment readiness checks, and operational tooling.
|
||||
|
||||
---
|
||||
|
||||
## Backend Plan
|
||||
|
||||
### 1. Sync Architecture
|
||||
| Task | Description |
|
||||
| --- | --- |
|
||||
| Audit adapters | Review current WordPress adapter + sync service; confirm endpoints for categories, tags, WooCommerce attributes/products. |
|
||||
| Data mapping | Document mapping between WordPress taxonomies and IGNY8 `SiteBlueprintTaxonomy`, `ContentTaxonomyMap`, `external_reference` fields. |
|
||||
| Sync configs | Extend integration settings to store WordPress/Woo credentials, sync frequency, and site archetype. |
|
||||
|
||||
### 2. Enhancements
|
||||
| Area | Implementation |
|
||||
| --- | --- |
|
||||
| Import | `ContentSyncService` fetches WP taxonomies/products/custom post types -> maps to IGNY8 schema via `TaxonomyService`. Auto-create missing clusters/taxonomies with an `imported` flag. |
|
||||
| Export | `WordPressAdapter` ensures taxonomies exist before publishing posts/products. Pushes product attributes/tags ahead of content. |
|
||||
| Sync Health APIs | `/sites/{id}/sync/status`, `/sites/{id}/sync/run`, returning last sync time, mismatch counts, error logs. |
|
||||
| Deployment | `SitesRendererAdapter` consumes cluster/taxonomy metadata for navigation, breadcrumbs, internal links. Deployment readiness check ensures cluster coverage, content status, validation flags. |
|
||||
| Logging & Monitoring | Structured logs per sync run (duration, items processed, failures). Alerts for repeated sync failures or deployment errors. |
|
||||
|
||||
---
|
||||
|
||||
## Frontend Plan
|
||||
|
||||
### 1. Sync Dashboard
|
||||
| Component | Features |
|
||||
| --- | --- |
|
||||
| Parity indicators | Status icons for taxonomies, products, posts. |
|
||||
| Controls | Manual sync, retry failed items, view logs (with pagination/filtering). |
|
||||
| Detail drawer | Shows mismatched items, suggested fixes, quick links into Planner/Writer. |
|
||||
|
||||
### 2. Deployment Panel
|
||||
| Component | Features |
|
||||
| --- | --- |
|
||||
| Readiness checklist | Displays cluster coverage, content validation, sync status. |
|
||||
| Actions | Deploy and rollback buttons with confirmation modals and logging. |
|
||||
| Notifications | Toasts for success/failure; optional email/webhook integration. |
|
||||
|
||||
### 3. WordPress Connection UI
|
||||
| Task | Description |
|
||||
| --- | --- |
|
||||
| Integration status | Show credential health, last sync time, active site type. |
|
||||
| Troubleshooting | Inline helper text + links to docs/runbook. |
|
||||
|
||||
---
|
||||
|
||||
## Operational Runbooks
|
||||
- Sync troubleshooting steps (auth errors, taxonomy mismatches, WooCommerce throttling).
|
||||
- Rollback procedures for failed deployments.
|
||||
- Escalation path + scripts to force sync, clear queue, or recover failed deploy.
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
| Area | Automated | Manual |
|
||||
| --- | --- | --- |
|
||||
| Sync logic | Integration tests hitting mocked WP APIs, verifying mapping + retries. | Staging sync from live WP instance; verify taxonomy parity. |
|
||||
| Deployment | Renderer tests using fixture metadata to ensure navigation/internal links. | Deploy IGNY8 site; inspect front-end output, breadcrumbs, menus. |
|
||||
| Dashboards | Component/unit tests for Sync/Deployment panels. | Pilot user testing + UX review. |
|
||||
| Runbooks | N/A | Tabletop exercises for failure scenarios (sync fails, deploy rollback). |
|
||||
|
||||
---
|
||||
|
||||
## Rollout Checklist
|
||||
1. Enable Stage 4 flag in staging; run full import/export tests.
|
||||
2. Pilot with one IGNY8-hosted and one WordPress-hosted site.
|
||||
3. Train support on dashboards/runbooks; ensure alerting configured.
|
||||
4. Announce availability; roll out gradually to accounts with WordPress integrations.
|
||||
5. Monitor logs/alerts during first production syncs; iterate on tooling as needed.
|
||||
|
||||
---
|
||||
|
||||
## Risks & Mitigations
|
||||
| Risk | Mitigation |
|
||||
| --- | --- |
|
||||
| API rate limits (WordPress/WooCommerce) | Backoff + batching, highlight in dashboard. |
|
||||
| Data mismatches (taxonomies/products) | Detailed diff view + retry actions + runbook. |
|
||||
| Deployment failures | Preflight checks + rollback buttons + structured logs. |
|
||||
| Operational overhead | Alerting dashboards + documented on-call runbook. |
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2025-11-19*
|
||||
|
||||
Reference in New Issue
Block a user