cleanup
This commit is contained in:
alorig
219
active-workflow-docs/MIGRATION_RESET_PLAN.md
Normal file
219
active-workflow-docs/MIGRATION_RESET_PLAN.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# Complete Migration Reset Plan - Backup, Recreate, Restore
|
||||
|
||||
## Overview
|
||||
This plan will:
|
||||
1. Backup all database data
|
||||
2. Remove WorkflowState from migrations
|
||||
3. Delete all migration files
|
||||
4. Generate fresh 0001_initial.py migrations
|
||||
5. Recreate database
|
||||
6. Restore data
|
||||
|
||||
## Database Detection
|
||||
- **Primary**: PostgreSQL (from DATABASE_URL or DB_* env vars)
|
||||
- **Fallback**: SQLite (if no PostgreSQL config)
|
||||
- **Detection**: Check DATABASE_URL or DB_ENGINE env vars
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Execution Plan
|
||||
|
||||
### PHASE 1: PREPARATION & BACKUP
|
||||
|
||||
#### Step 1.1: Detect Database Type
|
||||
- Read `settings.py` to determine database engine
|
||||
- Check environment variables: `DATABASE_URL`, `DB_ENGINE`, `DB_HOST`, etc.
|
||||
- Determine if PostgreSQL or SQLite
|
||||
|
||||
#### Step 1.2: Create Backup Script
|
||||
**For PostgreSQL:**
|
||||
```bash
|
||||
# Backup script: backup_db.sh
|
||||
pg_dump -h $DB_HOST -U $DB_USER -d $DB_NAME -F c -f backup_$(date +%Y%m%d_%H%M%S).dump
|
||||
```
|
||||
|
||||
**For SQLite:**
|
||||
```bash
|
||||
# Backup script: backup_db.sh
|
||||
cp $DB_PATH backup_$(date +%Y%m%d_%H%M%S).sqlite3
|
||||
```
|
||||
|
||||
#### Step 1.3: Execute Backup
|
||||
- Run backup script
|
||||
- Verify backup file created
|
||||
- Store backup path for restore
|
||||
|
||||
---
|
||||
|
||||
### PHASE 2: MIGRATION CLEANUP
|
||||
|
||||
#### Step 2.1: Remove WorkflowState from Migration 0003
|
||||
**File**: `backend/igny8_core/business/site_building/migrations/0003_workflow_and_taxonomies.py`
|
||||
|
||||
**Actions:**
|
||||
- Remove `CreateModel` for WorkflowState (lines 37-59)
|
||||
- Remove 3 `AddIndex` operations for WorkflowState (lines 98-109)
|
||||
- Keep: SiteBlueprintCluster, SiteBlueprintTaxonomy, and their indexes
|
||||
|
||||
#### Step 2.2: Identify All Apps with Migrations
|
||||
Apps to process:
|
||||
1. `igny8_core.auth` (label: `igny8_core_auth`)
|
||||
2. `igny8_core.modules.planner` (label: `planner`)
|
||||
3. `igny8_core.modules.writer` (label: `writer`)
|
||||
4. `igny8_core.modules.system` (label: `system`)
|
||||
5. `igny8_core.modules.billing` (label: `billing`)
|
||||
6. `igny8_core.business.site_building` (label: `site_building`)
|
||||
7. `igny8_core.business.optimization` (label: `optimization`)
|
||||
8. `igny8_core.business.publishing` (label: `publishing`)
|
||||
9. `igny8_core.business.integration` (label: `integration`)
|
||||
10. `igny8_core.modules.automation` (label: `automation`)
|
||||
11. `igny8_core.business.automation` (label: `automation`)
|
||||
|
||||
#### Step 2.3: Delete All Migration Files
|
||||
**For each app:**
|
||||
- List all files in `*/migrations/` directory
|
||||
- Delete all `*.py` files EXCEPT `__init__.py`
|
||||
- Keep `__pycache__/` directory (will be regenerated)
|
||||
|
||||
**Files to delete:**
|
||||
- `0001_initial.py`
|
||||
- `0002_*.py`
|
||||
- `0003_*.py`
|
||||
- ... (all numbered migrations)
|
||||
|
||||
---
|
||||
|
||||
### PHASE 3: GENERATE NEW MIGRATIONS
|
||||
|
||||
#### Step 3.1: Generate Fresh Migrations
|
||||
**Command:**
|
||||
```bash
|
||||
cd backend
|
||||
python manage.py makemigrations
|
||||
```
|
||||
|
||||
This will create new `0001_initial.py` for each app based on current models.
|
||||
|
||||
#### Step 3.2: Verify New Migrations
|
||||
- Check each app has new `0001_initial.py`
|
||||
- Verify WorkflowState is NOT in site_building migration
|
||||
- Verify all other models are present
|
||||
|
||||
---
|
||||
|
||||
### PHASE 4: DATABASE RECREATION
|
||||
|
||||
#### Step 4.1: Drop Database
|
||||
**For PostgreSQL:**
|
||||
```bash
|
||||
# Connect to postgres database (not app database)
|
||||
psql -h $DB_HOST -U postgres -c "DROP DATABASE IF EXISTS $DB_NAME;"
|
||||
psql -h $DB_HOST -U postgres -c "CREATE DATABASE $DB_NAME OWNER $DB_USER;"
|
||||
```
|
||||
|
||||
**For SQLite:**
|
||||
```bash
|
||||
rm $DB_PATH # Delete SQLite file
|
||||
```
|
||||
|
||||
#### Step 4.2: Run New Migrations
|
||||
```bash
|
||||
cd backend
|
||||
python manage.py migrate
|
||||
```
|
||||
|
||||
This creates all tables with new structure (no WorkflowState table).
|
||||
|
||||
---
|
||||
|
||||
### PHASE 5: DATA RESTORATION
|
||||
|
||||
#### Step 5.1: Restore Data
|
||||
**For PostgreSQL:**
|
||||
```bash
|
||||
pg_restore -h $DB_HOST -U $DB_USER -d $DB_NAME -c backup_*.dump
|
||||
```
|
||||
|
||||
**For SQLite:**
|
||||
```bash
|
||||
# SQLite backup is just a copy, so:
|
||||
cp backup_*.sqlite3 $DB_PATH
|
||||
```
|
||||
|
||||
#### Step 5.2: Handle WorkflowState Data
|
||||
**Important**: The backup contains WorkflowState table data, but new schema doesn't have it.
|
||||
|
||||
**Options:**
|
||||
- **Option A**: Restore will fail on WorkflowState table - ignore error (table doesn't exist)
|
||||
- **Option B**: Use `pg_restore --exclude-table=igny8_site_blueprint_workflow_states` to skip it
|
||||
- **Option C**: Manually edit backup to remove WorkflowState data before restore
|
||||
|
||||
**Recommended**: Option B - exclude WorkflowState table during restore
|
||||
|
||||
#### Step 5.3: Verify Data Restored
|
||||
- Check row counts match original
|
||||
- Verify critical tables have data
|
||||
- Test application functionality
|
||||
|
||||
---
|
||||
|
||||
## Execution Script Structure
|
||||
|
||||
```python
|
||||
# reset_migrations.py
|
||||
1. detect_database_type()
|
||||
2. backup_database()
|
||||
3. remove_workflowstate_from_migration()
|
||||
4. delete_all_migration_files()
|
||||
5. generate_new_migrations()
|
||||
6. drop_and_recreate_database()
|
||||
7. run_migrations()
|
||||
8. restore_data(exclude_workflowstate=True)
|
||||
9. verify_restoration()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Safety Checks
|
||||
|
||||
### Before Starting:
|
||||
- [ ] Verify database connection works
|
||||
- [ ] Check disk space for backup
|
||||
- [ ] Confirm backup location is writable
|
||||
- [ ] Document current migration state
|
||||
|
||||
### After Each Phase:
|
||||
- [ ] Verify backup file exists and is valid
|
||||
- [ ] Confirm migrations deleted correctly
|
||||
- [ ] Verify new migrations generated
|
||||
- [ ] Check database recreated successfully
|
||||
- [ ] Validate data restored
|
||||
|
||||
---
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If something goes wrong:
|
||||
1. **Stop immediately**
|
||||
2. **Restore from backup**: Use backup file to restore original database
|
||||
3. **Restore migrations**: Git checkout original migration files
|
||||
4. **Verify**: Test application works
|
||||
|
||||
---
|
||||
|
||||
## Estimated Time
|
||||
- Backup: 1-5 minutes (depends on data size)
|
||||
- Migration cleanup: 2-3 minutes
|
||||
- Generate migrations: 1-2 minutes
|
||||
- Database recreation: 1-2 minutes
|
||||
- Data restoration: 2-10 minutes (depends on data size)
|
||||
- **Total**: ~10-25 minutes
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
- WorkflowState table data will be lost (intentional)
|
||||
- All other data preserved
|
||||
- Migration history reset to clean state
|
||||
- All apps start with fresh 0001_initial.py
|
||||
|
||||
171
active-workflow-docs/SITE_BUILDER_WIZARD_REMOVAL_REFERENCE.md
Normal file
171
active-workflow-docs/SITE_BUILDER_WIZARD_REMOVAL_REFERENCE.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# Site Builder Wizard Removal - Reference Document
|
||||
|
||||
**Date:** 2025-11-20
|
||||
**Status:** Complete
|
||||
|
||||
---
|
||||
|
||||
## What Was Deleted
|
||||
|
||||
### Frontend Files
|
||||
- `/frontend/src/pages/Sites/Builder/Wizard.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/Preview.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/Blueprints.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/BusinessDetailsStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/BriefStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/ObjectivesStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/StyleStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/ClusterAssignmentStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/TaxonomyBuilderStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/SitemapReviewStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/CoverageValidationStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/steps/IdeasHandoffStep.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/components/WizardProgress.tsx`
|
||||
- `/frontend/src/pages/Sites/Builder/components/HelperDrawer.tsx`
|
||||
- `/frontend/src/store/builderStore.ts`
|
||||
|
||||
### Backend Files
|
||||
- `/backend/igny8_core/business/site_building/services/workflow_state_service.py`
|
||||
- `/backend/igny8_core/business/site_building/services/wizard_context_service.py`
|
||||
- `/backend/igny8_core/business/site_building/services/validators.py`
|
||||
|
||||
### Backend Model
|
||||
- `WorkflowState` model removed from `models.py`
|
||||
|
||||
### Routes Removed
|
||||
- `/sites/builder` (Wizard route)
|
||||
- `/sites/builder/preview` (Preview route)
|
||||
- `/sites/blueprints` (Blueprints route)
|
||||
|
||||
### Database
|
||||
- Table: `igny8_site_blueprint_workflow_states` (dropped)
|
||||
|
||||
---
|
||||
|
||||
## What's Missing from Workflow Now
|
||||
|
||||
**PHASE 2: SITE BUILDER WIZARD** - Completely removed
|
||||
|
||||
The 6-step wizard process is gone:
|
||||
1. ❌ Business Details Step
|
||||
2. ❌ Cluster Assignment Step
|
||||
3. ❌ Taxonomy Builder Step
|
||||
4. ❌ AI Sitemap Review Step
|
||||
5. ❌ Coverage Validation Step
|
||||
6. ❌ Ideas Hand-off Step
|
||||
|
||||
**Impact:** Users can no longer use the guided wizard to build sites. Site blueprints must be created/managed through other means.
|
||||
|
||||
---
|
||||
|
||||
## What Changed in Workflow
|
||||
|
||||
### Before (Workflow A)
|
||||
```
|
||||
PLANNING → WIZARD (6 steps) → WRITER → OPTIMIZE → PUBLISH
|
||||
```
|
||||
|
||||
### After (Workflow A)
|
||||
```
|
||||
PLANNING → WRITER → OPTIMIZE → PUBLISH
|
||||
```
|
||||
|
||||
**Direct Path:** Keywords/Clusters → Ideas → Tasks → Content (no wizard intermediary)
|
||||
|
||||
---
|
||||
|
||||
## Migration & Database Changes
|
||||
|
||||
### Migrations Reset
|
||||
- **Deleted:** 101 migration files (all numbered migrations)
|
||||
- **Created:** Fresh `0001_initial.py` for all 11 apps
|
||||
- **Removed:** WorkflowState from `0003_workflow_and_taxonomies.py` before deletion
|
||||
|
||||
### Database Actions
|
||||
1. **Backup:** `backup_postgres_20251120_232816.sql` (646KB)
|
||||
2. **Dropped:** Database recreated
|
||||
3. **Migrations Applied:** 34 new migrations
|
||||
4. **Data Restored:** All data preserved (except WorkflowState)
|
||||
5. **WorkflowState Table:** Dropped from database
|
||||
|
||||
### Migration State
|
||||
- All apps now have single `0001_initial.py`
|
||||
- No migration history (clean slate)
|
||||
- Database structure matches current models
|
||||
|
||||
---
|
||||
|
||||
## What's Still Kept (Site Builder Tables & Columns)
|
||||
|
||||
### Models Still Active
|
||||
1. **SiteBlueprint** (`igny8_site_blueprints`)
|
||||
2. **PageBlueprint** (`igny8_page_blueprints`)
|
||||
3. **SiteBlueprintCluster** (`igny8_site_blueprint_clusters`)
|
||||
4. **SiteBlueprintTaxonomy** (`igny8_site_blueprint_taxonomies`)
|
||||
5. **BusinessType** (`igny8_site_builder_business_types`)
|
||||
6. **AudienceProfile** (`igny8_site_builder_audience_profiles`)
|
||||
7. **BrandPersonality** (`igny8_site_builder_brand_personalities`)
|
||||
8. **HeroImageryDirection** (`igny8_site_builder_hero_imagery`)
|
||||
|
||||
### Complete Table Structure
|
||||
|
||||
#### `igny8_site_blueprints`
|
||||
- id, name, description, config_json, structure_json
|
||||
- status, hosting_type, version, deployed_version
|
||||
- tenant_id, site_id, sector_id
|
||||
- created_at, updated_at
|
||||
|
||||
#### `igny8_page_blueprints`
|
||||
- id, slug, title, type, blocks_json, status, order
|
||||
- site_blueprint_id, tenant_id, site_id, sector_id
|
||||
- created_at, updated_at
|
||||
|
||||
#### `igny8_site_blueprint_clusters`
|
||||
- id, role, coverage_status, metadata
|
||||
- site_blueprint_id, cluster_id, tenant_id, site_id, sector_id
|
||||
- created_at, updated_at
|
||||
|
||||
#### `igny8_site_blueprint_taxonomies`
|
||||
- id, name, slug, taxonomy_type, description, metadata, external_reference
|
||||
- site_blueprint_id, tenant_id, site_id, sector_id
|
||||
- created_at, updated_at
|
||||
|
||||
#### `igny8_site_blueprint_taxonomies_clusters` (M2M)
|
||||
- id, siteblueprinttaxonomy_id, clusters_id
|
||||
|
||||
#### Metadata Tables (Still Active)
|
||||
- `igny8_site_builder_business_types`
|
||||
- `igny8_site_builder_audience_profiles`
|
||||
- `igny8_site_builder_brand_personalities`
|
||||
- `igny8_site_builder_hero_imagery`
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints Still Available
|
||||
|
||||
- `GET/POST /api/v1/site-builder/blueprints/`
|
||||
- `GET/POST /api/v1/site-builder/pages/`
|
||||
- `POST /api/v1/site-builder/blueprints/{id}/generate_structure/`
|
||||
- `POST /api/v1/site-builder/blueprints/{id}/generate_all_pages/`
|
||||
- `POST /api/v1/site-builder/blueprints/{id}/clusters/attach`
|
||||
- `POST /api/v1/site-builder/blueprints/{id}/taxonomies/`
|
||||
- `GET /api/v1/site-builder/metadata/`
|
||||
|
||||
---
|
||||
|
||||
## Services Still Active
|
||||
|
||||
- `StructureGenerationService` - AI structure generation
|
||||
- `PageGenerationService` - Page content generation
|
||||
- `TaxonomyService` - Taxonomy management
|
||||
- `SiteBuilderFileService` - File management
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Removed:** Wizard UI, WorkflowState model/service, 6-step guided process
|
||||
**Kept:** All Site Builder models, tables, API endpoints, services
|
||||
**Changed:** Workflow now bypasses wizard, goes directly Planning → Writer
|
||||
**Database:** Clean migration state, WorkflowState table removed, all other data preserved
|
||||
|
||||
Reference in New Issue
Block a user