igny8/MIGRATION_RESET_PLAN.md

Complete Migration Reset Plan - Backup, Recreate, Restore

Overview

This plan will:

1. Backup all database data
2. Remove WorkflowState from migrations
3. Delete all migration files
4. Generate fresh 0001_initial.py migrations
5. Recreate database
6. Restore data

Database Detection

• Primary: PostgreSQL (from DATABASE_URL or DB_* env vars)
• Fallback: SQLite (if no PostgreSQL config)
• Detection: Check DATABASE_URL or DB_ENGINE env vars

---

Step-by-Step Execution Plan

PHASE 1: PREPARATION & BACKUP

Step 1.1: Detect Database Type

• Read settings.py to determine database engine
• Check environment variables: DATABASE_URL, DB_ENGINE, DB_HOST, etc.
• Determine if PostgreSQL or SQLite

Step 1.2: Create Backup Script

For PostgreSQL:

# Backup script: backup_db.sh
pg_dump -h $DB_HOST -U $DB_USER -d $DB_NAME -F c -f backup_$(date +%Y%m%d_%H%M%S).dump

For SQLite:

# Backup script: backup_db.sh
cp $DB_PATH backup_$(date +%Y%m%d_%H%M%S).sqlite3

Step 1.3: Execute Backup

• Run backup script
• Verify backup file created
• Store backup path for restore

---

PHASE 2: MIGRATION CLEANUP

Step 2.1: Remove WorkflowState from Migration 0003

File: backend/igny8_core/business/site_building/migrations/0003_workflow_and_taxonomies.py

Actions:

• Remove CreateModel for WorkflowState (lines 37-59)
• Remove 3 AddIndex operations for WorkflowState (lines 98-109)
• Keep: SiteBlueprintCluster, SiteBlueprintTaxonomy, and their indexes

Step 2.2: Identify All Apps with Migrations

Apps to process:

1. igny8_core.auth (label: igny8_core_auth)
2. igny8_core.modules.planner (label: planner)
3. igny8_core.modules.writer (label: writer)
4. igny8_core.modules.system (label: system)
5. igny8_core.modules.billing (label: billing)
6. igny8_core.business.site_building (label: site_building)
7. igny8_core.business.optimization (label: optimization)
8. igny8_core.business.publishing (label: publishing)
9. igny8_core.business.integration (label: integration)
10. igny8_core.modules.automation (label: automation)
11. igny8_core.business.automation (label: automation)

Step 2.3: Delete All Migration Files

For each app:

• List all files in */migrations/ directory
• Delete all *.py files EXCEPT __init__.py
• Keep __pycache__/ directory (will be regenerated)

Files to delete:

• 0001_initial.py
• 0002_*.py
• 0003_*.py
• ... (all numbered migrations)

---

PHASE 3: GENERATE NEW MIGRATIONS

Step 3.1: Generate Fresh Migrations

Command:

cd backend
python manage.py makemigrations

This will create new 0001_initial.py for each app based on current models.

Step 3.2: Verify New Migrations

• Check each app has new 0001_initial.py
• Verify WorkflowState is NOT in site_building migration
• Verify all other models are present

---

PHASE 4: DATABASE RECREATION

Step 4.1: Drop Database

For PostgreSQL:

# Connect to postgres database (not app database)
psql -h $DB_HOST -U postgres -c "DROP DATABASE IF EXISTS $DB_NAME;"
psql -h $DB_HOST -U postgres -c "CREATE DATABASE $DB_NAME OWNER $DB_USER;"

For SQLite:

rm $DB_PATH  # Delete SQLite file

Step 4.2: Run New Migrations

cd backend
python manage.py migrate

This creates all tables with new structure (no WorkflowState table).

---

PHASE 5: DATA RESTORATION

Step 5.1: Restore Data

For PostgreSQL:

pg_restore -h $DB_HOST -U $DB_USER -d $DB_NAME -c backup_*.dump

For SQLite:

# SQLite backup is just a copy, so:
cp backup_*.sqlite3 $DB_PATH

Step 5.2: Handle WorkflowState Data

Important: The backup contains WorkflowState table data, but new schema doesn't have it.

Options:

• Option A: Restore will fail on WorkflowState table - ignore error (table doesn't exist)
• Option B: Use pg_restore --exclude-table=igny8_site_blueprint_workflow_states to skip it
• Option C: Manually edit backup to remove WorkflowState data before restore

Recommended: Option B - exclude WorkflowState table during restore

Step 5.3: Verify Data Restored

• Check row counts match original
• Verify critical tables have data
• Test application functionality

---

Execution Script Structure

# reset_migrations.py
1. detect_database_type()
2. backup_database()
3. remove_workflowstate_from_migration()
4. delete_all_migration_files()
5. generate_new_migrations()
6. drop_and_recreate_database()
7. run_migrations()
8. restore_data(exclude_workflowstate=True)
9. verify_restoration()

---

Safety Checks

Before Starting:

• Verify database connection works
• Check disk space for backup
• Confirm backup location is writable
• Document current migration state

After Each Phase:

• Verify backup file exists and is valid
• Confirm migrations deleted correctly
• Verify new migrations generated
• Check database recreated successfully
• Validate data restored

---

Rollback Plan

If something goes wrong:

1. Stop immediately
2. Restore from backup: Use backup file to restore original database
3. Restore migrations: Git checkout original migration files
4. Verify: Test application works

---

Estimated Time

• Backup: 1-5 minutes (depends on data size)
• Migration cleanup: 2-3 minutes
• Generate migrations: 1-2 minutes
• Database recreation: 1-2 minutes
• Data restoration: 2-10 minutes (depends on data size)
• Total: ~10-25 minutes

---

Notes

• WorkflowState table data will be lost (intentional)
• All other data preserved
• Migration history reset to clean state
• All apps start with fresh 0001_initial.py