salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b5cc262f049729c4693b9b06db7335c1bf4b402f
igny8/refactor-plan/UNIFIED_SITE_BUILDER_IMPLEMENTATION_PLAN.md
IGNY8 VPS (Salman) 142077ce85 1
2025-11-19 14:03:45 +00:00

18 KiB
Raw Blame History

IGNY8 Refactor Plan – Stage Implementation Guide

This document complements SITE_BUILDER_PLANNER_ARCHITECTURE_PLAN.md by outlining how to deliver the refactor in four controlled stages. Each stage lists scope, tasks, exits, and testing so workstreams can move independently without breaking user flows.

Stage 1 – Data & Services Foundation

Objective: Build all backend scaffolding (schema + services) required for the guided site builder, richer planner, and downstream metadata.

1. Sequenced Plan of Work

  1. Preparation
    • Freeze any pending migrations; branch off refactor-stage1.
    • Create feature flags (USE_SITE_BUILDER_REFACTOR) in settings/env files.
    • Draft ERD updates and share with backend team for approval.
  2. Schema Implementation
    • Site Builder tables
      • SiteBlueprintCluster: fields for cluster_id, role, coverage_status, metadata.
      • SiteBlueprintTaxonomy: taxonomy type enum, cluster_id, optional external reference.
      • WorkflowState: blueprint FK, current_step, step_status, blocking_reason, timestamps.
    • Writer tables
      • ContentClusterMap, ContentTaxonomyMap, ContentAttributeMap with FK constraints + indexes for performant lookups.
    • Run migrations locally → staging; capture SQL diff in change log.
  3. Service Layer
    • Implement WorkflowStateService with methods:
      • initialize(site_blueprint) – create default states.
      • update_step(step, status, meta) – persists transitions and emits audit logs.
      • validate_step(step) – raises descriptive errors for UI.
    • Implement TaxonomyService:
      • CRUD handlers.
      • Import adapters (WordPress categories/tags, WooCommerce product attributes).
      • Helper to map taxonomy ↔ cluster(s).
    • Update ClusteringService prompts to emit:
      • Topic cluster suggestions.
      • Attribute/feature recommendations per keyword set.
      • Taxonomy hints (category/tag names).
  4. API/Serializer Updates
    • Planner serializers return context_type, dimension_meta.
    • Writer serializers include entity_type, taxonomy associations, validation errors.
    • Site builder endpoints expose workflow state + gating booleans.
  5. Validation Hooks
    • Add reusable validators (e.g., ensure_clusters_attached(site_blueprint)).
    • Integrate validators into existing actions (generate_structure, create_tasks_from_blueprint).
  6. Internal Tooling & Docs
    • Update Postman collection / API reference for new endpoints + payloads.
    • Write developer doc describing new models + services.
    • Set up monitoring dashboards (Grafana/NewRelic) for:
      • Migration duration.
      • Celery task errors referencing new services.

2. Owners & Dependencies

  • Backend Lead: owns migrations, services, validator integration.
  • Data/DBA: reviews migration plan, handles staging rollout.
  • QA: prepares regression scripts for planner/writer.
  • Docs: updates master-docs (backend + AI framework) with new tables/services.
  • Dependencies: PostgreSQL 15 features already available; no external API changes needed.

3. Deliverables & Exit Criteria

  • ✅ Migrations applied to staging + rollback verified.
  • ✅ WorkflowStateService, TaxonomyService, ClusteringService updates merged with unit tests.
  • ✅ Planner/Writer legacy flows pass smoke tests (imports, clustering, idea creation, content generation).
  • ✅ API reference + architecture docs updated.
  • ✅ Feature flag defaults to OFF (no user-facing change yet).

4. Testing Matrix

Area Automated Manual
Migrations pytest migration test + Django fake migrations Dry run on staging, rollback rehearsal
Services Unit tests for new service methods Postman calls to workflow/taxonomy endpoints
Validators Serializer/endpoint tests expecting 4xx when requirements unmet Trigger via API (e.g., attempt sitemap gen without clusters)
Regression Existing planner/writer test suite Smoke test: keyword import → cluster → idea → task → content

5. Rollout Checklist

  1. Merge migrations + services behind feature flag.
  2. Deploy to staging, run migration + regression script.
  3. Monitor Celery/DB logs for anomalies.
  4. After 24h stability, promote to production (still gated by flag).
  5. Announce availability to frontend team to begin Stage 2 work.

Stage 2 – Planner + Wizard UX

Objective: Ship the state-aware, self-guided planner + site builder experience on top of Stage 1 services.

1. Sequenced Plan of Work

  1. Planning & Design
    • UX team finalizes wizard wireframes + microcopy for each step, including helper text/tooltips.
    • Define telemetry schema (events: wizard_step_started, wizard_step_completed, wizard_blocking_issue).
    • Review backend responses needed per step (cluster metrics, taxonomy suggestions, validation errors).
  2. Backend Enhancements
    • Extend planner endpoints to return:
      • Cluster stats (keyword count, projected hubs/support pages).
      • Suggested taxonomies/attributes from Stage 1 dimension_meta.
      • Validation error payloads structured for UI (“missing clusters”, “taxonomy incomplete”).
    • Add audit logging to WorkflowStateService.update_step for troubleshooting.
  3. Frontend Infrastructure
    • Build builderWorkflowStore (Zustand):
      • State: currentStep, completedSteps, blockingIssues, workflowState, loading.
      • Actions: goToStep, completeStep, refreshState, setBlockingIssue.
      • Persistence: sessionStorage/localStorage to resume across refreshes.
    • Hook store into new /site-builder/workflow/{blueprintId} endpoints.
  4. Wizard Step Implementation
    • Step 1 – Business Details
      • Components for site type selection, hosting detection (show WP integration status), brand inputs.
      • Save actions call blueprint PATCH + workflow state update.
    • Step 2 – Cluster Assignment
      • Table/list of clusters with filters (intent, volume, context type).
      • Coverage meter (keywords covered vs. target).
      • “Add cluster” action hitting /blueprints/{id}/clusters/attach.
      • Blocking logic: enforce minimum cluster count/keyword count per site type.
    • Step 3 – Taxonomy Builder
      • Tree/table for categories, tags, product attributes, service groups.
      • Import buttons (WordPress/WooCommerce) calling TaxonomyService endpoints.
      • Mapping UI linking taxonomy items to clusters (drag/drop or select dropdown).
    • Step 4 – AI Sitemap Review
      • Grid grouped by cluster/taxonomy.
      • Visual checklist (Hub page, Supporting pages count).
      • Ability to edit page titles/types, reorder, remove.
      • “Regenerate” button to re-run AI structure (with warning on credit usage).
    • Step 5 – Coverage Validation
      • Summary cards showing clusters covered/uncovered, taxonomy gaps, attribute coverage.
      • CTA to jump back to specific step if gaps remain.
    • Step 6 – Ideas Hand-off
      • Select pages to push to Planner Ideas.
      • Secondary prompt textarea; show preview of what data AI receives.
      • Confirmation modal summarizing cluster/page counts.
  5. Planner Module Enhancements
    • Cluster matrix view showing clusters vs. taxonomy/attribute coverage, clickable to drill into wizard step.
    • Taxonomy management table (search, filters, bulk edits).
    • Inline warnings (e.g., banner on Planner home when blueprint still missing requirements).
  6. UX Guardrails
    • Breadcrumb/progress component with checkmarks.
    • Disabled Next button shows tooltip listing unmet requirements.
    • Contextual helper drawer per step (links to docs, best practices).

2. Owners & Dependencies

  • Frontend Lead: wizard implementation + planner UI updates.
  • Backend Support: ensure required data returned, logging in place.
  • Design/UX: final visuals, helper content, accessibility review.
  • Docs/Enablement: in-app help, video walkthroughs.
  • Dependency: Stage 1 feature flag must be AVAILABLE with APIs stable.

3. Deliverables & Exit Criteria

  • ✅ Wizard completes end-to-end (feature-flagged) for both IGNY8-hosted and WP-linked sites.
  • ✅ Telemetry events firing and visible in analytics dashboard.
  • ✅ QA scripts covering success/failure states executed, accessibility issues resolved.
  • ✅ Documentation (knowledge base, tooltips) updated.
  • ✅ Rollout plan ready (progressive enablement per account/site type).

4. Testing Matrix

Area Automated Manual
Wizard Flow Cypress tests for each step, gating logic, resume Scenario walkthroughs (new site, WP import), cross-browser testing
Planner Views Jest/unit tests for new components, store logic Performance check on cluster matrix, UX review
Telemetry Unit tests ensuring events dispatch Verify dashboards receiving events
Accessibility Lint checks (aria labels, keyboard nav) Screen reader + keyboard-only validation

5. Rollout Checklist

  1. Feature flag ON for internal testers → gather feedback.
  2. Fix critical issues, re-run automated suite.
  3. Enable for pilot accounts (one IGNY8 host, one WP host).
  4. Monitor telemetry/feedback; ensure support team ready.
  5. Gradually roll out to all accounts; announce in release notes + in-app banner.

Stage 3 – Writer / Linker / Optimizer Enhancements

Objective: Propagate the new metadata through content creation and optimization, ensuring validation before publish.

1. Sequenced Plan of Work

  1. Metadata Audit & Defaults
    • Verify Stage 1 migrations populated defaults for legacy content/tasks.
    • Script to backfill entity_type, taxonomy_id, cluster_role for existing records (with audit log).
  2. Backend Pipeline Updates
    • Ideas → Tasks: update services so every task inherits cluster/taxonomy/attribute metadata.
    • Tasks → Content: adjust ContentPipelineService to store mappings in ContentClusterMap, etc.
    • Prompt Updates: AI prompts include cluster role, taxonomy context, product attributes to produce richer output.
    • Validation Services: create reusable validators (e.g., ensure_required_attributes(task)) returning human-friendly errors.
  3. Linker & Optimizer Enhancements
    • Linker service leverages mapping tables to suggest hub/supporting/attribute links; adds priority scoring.
    • Optimizer scoring factors:
      • Cluster coverage (are all hubs/supporting pages present?).
      • Taxonomy alignment (page categorized correctly).
      • Attribute completeness (product/service specs filled).
    • New APIs:
      • GET /sites/{id}/progress (cluster-level completion + validation status).
      • GET /writer/content/{id}/validation (aggregated checklist for UI).
  4. Frontend Enhancements
    • Planner Ideas / Writer Tasks lists
      • Columns/chips for cluster, taxonomy, attributes.
      • Filters for entity type & validation status.
    • Writer Editor
      • Sidebar module with cluster summary, taxonomy tree, attribute form.
      • Validation panel showing remaining requirements; publish button disabled until cleared.
    • Linker UI
      • Group internal link suggestions by cluster role; show context snippet, CTA to insert.
    • Optimizer Dashboard
      • Scorecards per cluster dimension with color coding + “next action” cards.
    • Site Progress Widgets
      • On site overview, show bars for hubs/supporting/attribute completion; link to problematic clusters.
  5. Notifications & Workflow
    • Inline toast + optional email when validation fails or content ready for review.
    • Credit reminder banner when user tries to regenerate content/optimization tasks.

2. Owners & Dependencies

  • Backend Lead: pipeline + services, validation APIs.
  • Frontend Lead: writer/linker/optimizer UI, progress widgets.
  • AI/Prompt Specialist: adjust prompts/templates and verify outputs.
  • QA: regression scripts for writer workflow, linker, optimizer.
  • Dependencies: Stage 2 wizard must deliver cluster/taxonomy data; Stage 1 mapping tables in place.

3. Deliverables & Exit Criteria

  • ✅ All new tasks/content carry entity/taxonomy metadata; legacy content backfilled.
  • ✅ Publish action blocked when required metadata missing; errors surfaced clearly.
  • ✅ Linker/optimizer screens powered by new data with acceptable performance.
  • ✅ Site progress dashboards live for pilot sites with feedback loop established.

4. Testing Matrix

Area Automated Manual
Pipeline Unit tests on service functions, serializer coverage End-to-end run: blueprint → ideas → tasks → content
Validation Tests ensuring validators trigger proper errors Attempt publish without taxonomy/attributes; confirm UI flow
Linker/Optimizer Service tests for scoring & suggestions Performance profiling on large datasets; UX review
Progress Widgets Component/unit tests Compare counts vs. database for pilot site

5. Rollout Checklist

  1. Deploy to staging with feature flag ON; run regression suite.
  2. Backfill metadata for existing content in staging; verify no issues.
  3. Pilot with internal content team; gather feedback on validation friction.
  4. Optimize slow queries (add indexes/caching) before production rollout.
  5. Update training docs/videos for writers; enable flag gradually across accounts.

Stage 4 – Publishing & Sync Integration

Objective: Achieve feature parity between IGNY8-hosted deployments and WordPress sites, using the shared metadata model.

1. Sequenced Plan of Work

  1. Sync Architecture Review
    • Audit existing WordPress adapter + sync service.
    • Confirm required endpoints (categories, tags, WooCommerce attributes/products) and authentication flows.
    • Document data mapping between WP taxonomies and new IGNY8 taxonomy tables.
  2. Backend Enhancements
    • Import
      • Extend ContentSyncService to fetch WP taxonomies, products, custom post types.
      • Map external IDs to SiteBlueprintTaxonomy, ContentTaxonomyMap, storing external_reference.
      • Auto-create missing clusters/taxonomies with flag indicating “imported”.
    • Export
      • Update WordPressAdapter to create/update taxonomies before publishing posts/products.
      • Ensure product attributes/tags pushed before product content.
    • Sync Health
      • New endpoints: /sites/{id}/sync/status, /sites/{id}/sync/run.
      • Track last sync time, mismatch counts, error logs.
    • Deployment
      • SitesRendererAdapter consumes cluster/taxonomy metadata to build navigation, breadcrumbs, internal links.
      • Deployment readiness check service summarizing cluster coverage, content publish status, validation flags.
    • Logging/Monitoring
      • Structured logs for sync runs (duration, items processed, failures).
      • Alerts for repeated sync failures or deployment errors.
  3. Frontend Enhancements
    • Sync Dashboard
      • Parity indicators (taxonomies, products, posts) with status icons.
      • Buttons for manual sync, retry failed items, view logs.
      • Detail drawer showing mismatched items and suggested fixes.
    • Deployment Panel
      • Readiness checklist (clusters covered, content statuses, validation passes).
      • Deploy + rollback buttons with confirmation modals.
      • Toast/notification system for deploy/sync success/failure.
    • WordPress Connection UI
      • Surface integration status, credentials check, last sync info.
  4. Operational Runbooks
    • Document sync troubleshooting steps, rollback procedures, and escalation path.
    • Provide support scripts for forcing sync, clearing queue, recovering failed deployment.

2. Owners & Dependencies

  • Backend Lead: sync/import/export logic, deployment readiness service.
  • Frontend Lead: sync dashboard, deployment UI.
  • Infra/DevOps: monitoring, alerting, runbooks.
  • QA: end-to-end publish tests IGNY8↔WP.
  • Dependencies: Stage 3 metadata stable; WordPress credentials for pilot sites available.

3. Deliverables & Exit Criteria

  • ✅ End-to-end publishing works for blog/ecommerce/company templates on IGNY8 hosting and WordPress (posts/products/services).
  • ✅ Sync dashboard shows green status for pilot accounts; manual retry resolves mismatches.
  • ✅ Deployment readiness checks enforced; logs + rollback procedures documented.
  • ✅ Release notes + training material published.

4. Testing Matrix

Area Automated Manual
Sync Logic Integration tests hitting mocked WP APIs Staging sync from live WP instance; verify taxonomy parity
Deployment Renderer tests verifying metadata usage Deploy IGNY8 site, inspect navigation/internal links
Dashboards Component/unit tests Pilot user testing of sync dashboard + deployment flow
Runbooks N/A Tabletop exercises for failure scenarios (sync fails, deploy rollback)

5. Rollout Checklist

  1. Enable Stage 4 flag in staging; run full sync/import tests.
  2. Pilot with one IGNY8-hosted and one WP-hosted site; gather feedback.
  3. Train support team on new dashboards/runbooks.
  4. Announce availability; roll out gradually to accounts with WordPress integrations.
  5. Monitor logs/alerts closely during first production syncs; iterate on tooling as needed.

Stage Coordination Notes

  • Feature flags: introduce per-stage flags to roll out gradually.
  • Monitoring: add dashboards for workflow step failures, sync errors, credit usage spikes.
  • Documentation cadence: update master-docs + part2-dev/planning after each stage completes.
  • Entry/exit reviews: hold short readiness reviews before moving to the next stage to ensure dependencies satisfied.

This plan ensures each layer stabilizes before the next, keeping users on a guided, predictable experience throughout the refactor.

Reference in New Issue View Git Blame Copy Permalink