salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

reorg docs

Browse Source
This commit is contained in:
IGNY8 VPS (Salman)
2026-01-10 02:42:31 +00:00
parent 346d3f0531
commit ce66dadc00
5 changed files with 196 additions and 220 deletions
Download Patch File Download Diff File Expand all files Collapse all files

662
docs/90-REFERENCE/BACKUP-AND-CLEANUP-GUIDE.md Normal file
View File

@@ -0,0 +1,662 @@
# Phase 6: Data Backup & Cleanup Guide
**Version:** 1.0
**Created:** January 9, 2026
**Purpose:** Pre-V1.0 Launch Database Preparation
---
## 📋 Table of Contents
1. [Overview](#overview)
2. [What Was Created](#what-was-created)
3. [When to Use](#when-to-use)
4. [Pre-Execution Checklist](#pre-execution-checklist)
5. [Command 1: Export System Config](#command-1-export-system-config)
6. [Command 2: Cleanup User Data](#command-2-cleanup-user-data)
7. [Complete Workflow](#complete-workflow)
8. [Safety Measures](#safety-measures)
9. [Rollback Procedures](#rollback-procedures)
10. [FAQ](#faq)
---
## 📖 Overview
Phase 6 provides two Django management commands to safely prepare your IGNY8 database for V1.0 production launch:
1. **Export System Configuration** - Backs up all system settings to JSON files
2. **Cleanup User Data** - Removes all test/development user data while preserving system configuration
### Why These Commands?
- **Clean Start**: Launch V1.0 with a pristine database
- **Configuration Preservation**: Keep all your carefully configured settings
- **Safety First**: Multiple safety checks and dry-run options
- **Audit Trail**: Complete metadata and logging
---
## 🛠️ What Was Created
### File Locations
```
backend/igny8_core/management/commands/
├── export_system_config.py # System configuration backup
└── cleanup_user_data.py # User data cleanup
```
### Command 1: `export_system_config.py`
**Purpose**: Exports all system configuration to JSON files for backup and version control.
**What it exports:**
- ✅ Subscription Plans (Starter, Growth, Scale)
- ✅ Credit Cost Configurations
- ✅ AI Model Settings (OpenAI, Anthropic, etc.)
- ✅ Global Integration Settings
- ✅ Industries and Sectors
- ✅ Seed Keywords (reference data)
- ✅ Author Profiles
- ✅ AI Prompts and Variables
**What it creates:**
- Individual JSON files for each data type
- `export_metadata.json` with timestamp and statistics
- Organized folder structure in `backups/config/`
### Command 2: `cleanup_user_data.py`
**Purpose**: Safely removes all user-generated test data before production launch.
**What it deletes:**
- 🗑️ Sites and Site Settings
- 🗑️ Keywords, Clusters, Ideas
- 🗑️ Tasks, Content, Images
- 🗑️ Publishing Records
- 🗑️ WordPress Sync Events
- 🗑️ Credit Transactions and Usage Logs
- 🗑️ Automation Runs
- 🗑️ Notifications
- 🗑️ Orders
**What it preserves:**
- ✅ User Accounts (admin users)
- ✅ System Configuration (all settings from export)
- ✅ Plans and Pricing
- ✅ AI Models and Prompts
- ✅ Industries and Sectors
---
## ⏰ When to Use
### Correct Timing
✅ **Use these commands when:**
- You're preparing for V1.0 production launch
- You've completed all testing and configuration
- You want to start production with clean data
- All system settings (Plans, AI models, prompts) are finalized
❌ **Do NOT use these commands when:**
- You're still in active development
- You haven't backed up your configurations
- You're unsure about your system settings
- You're in production with live users
### Recommended Timeline
```
Day -7: Final configuration review
Day -5: Export system config (first backup)
Day -3: Test commands in staging
Day -2: Export system config (final backup)
Day -1: Cleanup user data in staging
Day 0: Launch day - cleanup in production
```
---
## ✅ Pre-Execution Checklist
Before running ANY Phase 6 command, complete this checklist:
### Environment Verification
- [ ] Confirm you're in the correct environment (staging vs production)
- [ ] Check `ENVIRONMENT` setting in Django settings
- [ ] Verify database connection is correct
- [ ] Ensure you have full database backup
### System State
- [ ] All Plans configured and tested
- [ ] All AI prompts finalized
- [ ] All credit costs verified
- [ ] All industries/sectors populated
- [ ] Seed keywords imported
### Safety Backups
- [ ] Full database dump exists
- [ ] Previous export exists (if available)
- [ ] Media files backed up
- [ ] Environment variables documented
### Access & Permissions
- [ ] You have Django shell access
- [ ] You have database backup access
- [ ] You have rollback permissions
- [ ] Stakeholders notified
---
## 📤 Command 1: Export System Config
### Basic Usage
```bash
cd /data/app/igny8/backend
python manage.py export_system_config
```
### With Custom Output Directory
```bash
python manage.py export_system_config --output-dir=/path/to/backup
```
### Step-by-Step Execution
#### Step 1: Navigate to Backend
```bash
cd /data/app/igny8/backend
```
#### Step 2: Run Export
```bash
python manage.py export_system_config --output-dir=../backups/config/$(date +%Y%m%d)
```
#### Step 3: Verify Output
```bash
ls -la ../backups/config/$(date +%Y%m%d)/
```
Expected output:
```
plans.json # Subscription plans
credit_costs.json # Credit cost configurations
ai_models.json # AI model settings
global_integrations.json # Integration settings
industries.json # Industry master data
sectors.json # Sector master data
seed_keywords.json # Reference keywords
author_profiles.json # Writing style profiles
prompts.json # AI prompts
prompt_variables.json # Prompt variables
export_metadata.json # Export timestamp & stats
```
#### Step 4: Verify Data
Check one of the exports:
```bash
cat ../backups/config/$(date +%Y%m%d)/plans.json | head -20
```
#### Step 5: Commit to Version Control
```bash
cd /data/app/igny8
git add backups/config/
git commit -m "Backup: V1.0 system configuration export"
git push
```
### What The Output Looks Like
```
Exporting system configuration to: /data/app/igny8/backups/config/20260109
✓ Exported 3 Subscription Plans → plans.json
✓ Exported 12 Credit Cost Configurations → credit_costs.json
✓ Exported 4 AI Model Configurations → ai_models.json
✓ Exported 1 Global Integration Settings → global_integrations.json
✓ Exported 15 Industries → industries.json
✓ Exported 47 Sectors → sectors.json
✓ Exported 523 Seed Keywords → seed_keywords.json
✓ Exported 3 Author Profiles → author_profiles.json
✓ Exported 8 AI Prompts → prompts.json
✓ Exported 12 Prompt Variables → prompt_variables.json
✓ Metadata saved to export_metadata.json
======================================================================
System Configuration Export Complete!
Successful: 10 exports
Failed: 0 exports
Location: /data/app/igny8/backups/config/20260109
======================================================================
```
### Troubleshooting
**Problem**: "No module named 'django'"
```bash
# Solution: Activate virtual environment or use Docker
docker-compose exec backend python manage.py export_system_config
```
**Problem**: "Permission denied" when writing files
```bash
# Solution: Check directory permissions
mkdir -p ../backups/config
chmod 755 ../backups/config
```
**Problem**: Empty JSON files
```bash
# Solution: Verify data exists in database
python manage.py shell
>>> from igny8_core.modules.billing.models import Plan
>>> Plan.objects.count()
```
---
## 🗑️ Command 2: Cleanup User Data
### ⚠️ CRITICAL WARNING
**THIS COMMAND PERMANENTLY DELETES DATA**
- Cannot be undone without database restore
- Removes ALL user-generated content
- Should ONLY be run before production launch
- ALWAYS run `--dry-run` first
### Safety Features
1. **Dry-Run Mode**: Preview deletions without actually deleting
2. **Confirmation Prompt**: Must type "DELETE ALL DATA" to proceed
3. **Production Protection**: Blocked in production environment (unless explicitly allowed)
4. **Transaction Safety**: All deletions in atomic transaction
5. **Detailed Logging**: Shows exactly what was deleted
### Usage: Dry Run (Always First!)
```bash
cd /data/app/igny8/backend
python manage.py cleanup_user_data --dry-run
```
### Dry Run Output Example
```
======================================================================
DRY RUN - No data will be deleted
======================================================================
✓ Would delete 1,234 Notifications
✓ Would delete 5,678 Credit Usage Logs
✓ Would delete 456 Credit Transactions
✓ Would delete 23 Orders
✓ Would delete 8,901 WordPress Sync Events
✓ Would delete 234 Publishing Records
✓ Would delete 45 Automation Runs
✓ Would delete 3,456 Images
✓ Would delete 2,345 Content
✓ Would delete 4,567 Tasks
✓ Would delete 5,678 Content Ideas
✓ Would delete 1,234 Clusters
✓ Would delete 9,876 Keywords
✓ Would delete 12 Sites
→ Keeping 3 Users (not deleted)
Total records to delete: 43,739
======================================================================
To proceed with actual deletion, run:
python manage.py cleanup_user_data --confirm
======================================================================
```
### Usage: Actual Cleanup
```bash
python manage.py cleanup_user_data --confirm
```
**You will be prompted:**
```
======================================================================
⚠️ DELETING ALL USER DATA - THIS CANNOT BE UNDONE!
======================================================================
Type "DELETE ALL DATA" to proceed:
```
**Type exactly:** `DELETE ALL DATA`
### Actual Cleanup Output
```
Proceeding with deletion...
✓ Deleted 1,234 Notifications
✓ Deleted 5,678 Credit Usage Logs
✓ Deleted 456 Credit Transactions
✓ Deleted 23 Orders
✓ Deleted 8,901 WordPress Sync Events
✓ Deleted 234 Publishing Records
✓ Deleted 45 Automation Runs
✓ Deleted 3,456 Images
✓ Deleted 2,345 Content
✓ Deleted 4,567 Tasks
✓ Deleted 5,678 Content Ideas
✓ Deleted 1,234 Clusters
✓ Deleted 9,876 Keywords
✓ Deleted 12 Sites
======================================================================
User Data Cleanup Complete!
Total records deleted: 43,739
Failed deletions: 0
======================================================================
```
### Production Environment Protection
If you try to run cleanup in production:
```
⚠️ BLOCKED: Cannot run cleanup in PRODUCTION environment!
To allow this, temporarily set ENVIRONMENT to "staging" in settings.
```
To override (ONLY if absolutely necessary):
```python
# In settings.py - TEMPORARY
ENVIRONMENT = 'staging' # Change back after cleanup!
```
---
## 🔄 Complete Workflow
### Full Pre-Launch Procedure
```bash
# ========================================
# STEP 1: FULL DATABASE BACKUP
# ========================================
cd /data/app/igny8/backend
pg_dump -h localhost -U postgres igny8_db > ../backups/$(date +%Y%m%d)_pre_v1_full_backup.sql
# Verify backup exists and has content
ls -lh ../backups/$(date +%Y%m%d)_pre_v1_full_backup.sql
head -50 ../backups/$(date +%Y%m%d)_pre_v1_full_backup.sql
# ========================================
# STEP 2: EXPORT SYSTEM CONFIGURATION
# ========================================
python manage.py export_system_config --output-dir=../backups/config/$(date +%Y%m%d)
# Verify exports
ls -la ../backups/config/$(date +%Y%m%d)/
# Review critical configs
cat ../backups/config/$(date +%Y%m%d)/plans.json | python -m json.tool | head -30
cat ../backups/config/$(date +%Y%m%d)/credit_costs.json | python -m json.tool | head -30
# ========================================
# STEP 3: COMMIT CONFIGS TO GIT
# ========================================
cd /data/app/igny8
git add backups/config/
git commit -m "Pre-V1.0: System configuration backup $(date +%Y%m%d)"
git push
# ========================================
# STEP 4: BACKUP MEDIA FILES
# ========================================
cd /data/app/igny8
tar -czf backups/$(date +%Y%m%d)_media_backup.tar.gz backend/media/
# ========================================
# STEP 5: DRY RUN CLEANUP (REVIEW CAREFULLY)
# ========================================
cd backend
python manage.py cleanup_user_data --dry-run
# Review the counts - make sure they're expected
# ========================================
# STEP 6: ACTUAL CLEANUP (POINT OF NO RETURN)
# ========================================
python manage.py cleanup_user_data --confirm
# Type: DELETE ALL DATA
# ========================================
# STEP 7: VERIFY CLEANUP
# ========================================
python manage.py shell << 'EOF'
from igny8_core.auth.models import Site, CustomUser
from igny8_core.business.planning.models import Keywords
from igny8_core.modules.billing.models import Plan
print(f"Sites: {Site.objects.count()} (should be 0)")
print(f"Keywords: {Keywords.objects.count()} (should be 0)")
print(f"Users: {CustomUser.objects.count()} (admins preserved)")
print(f"Plans: {Plan.objects.count()} (should have your plans)")
EOF
# ========================================
# STEP 8: TEST APPLICATION
# ========================================
python manage.py runserver 0.0.0.0:8000 &
# Visit app and verify:
# - Can login as admin
# - Dashboard loads (empty state)
# - Plans visible in settings
# - Can create new user account
# ========================================
# STEP 9: TAG RELEASE
# ========================================
cd /data/app/igny8
git tag -a v1.0.0-clean -m "V1.0.0 - Clean database ready for launch"
git push origin v1.0.0-clean
```
---
## 🛡️ Safety Measures
### Built-in Protections
1. **Atomic Transactions**: All deletions in single transaction - all or nothing
2. **Production Check**: Requires explicit override in production
3. **Confirmation Prompt**: Must type exact phrase
4. **Dry Run**: See exactly what will be deleted
5. **Detailed Logging**: Know what was deleted and any failures
### Manual Safety Checklist
Before running cleanup:
- [ ] **Full database backup** exists and verified
- [ ] **System config export** completed and committed to git
- [ ] **Media files** backed up
- [ ] **Dry run reviewed** and counts are expected
- [ ] **Stakeholders notified** of pending cleanup
- [ ] **Rollback plan** documented and tested
- [ ] **Off-hours execution** scheduled (if production)
- [ ] **Monitoring ready** to catch any issues
### Additional Recommendations
1. **Staging First**: Always test in staging environment first
2. **Screenshot Evidence**: Take screenshots of dry-run output
3. **Communication**: Notify team before and after
4. **Timing**: Run during low-traffic hours
5. **Verification**: Test application immediately after
---
## 🔙 Rollback Procedures
### If Something Goes Wrong
#### During Cleanup (Transaction Failed)
No action needed - atomic transaction will automatically rollback.
#### After Cleanup (Need to Restore)
```bash
# OPTION 1: Restore from PostgreSQL backup
cd /data/app/igny8
psql -U postgres -d igny8_db < backups/20260109_pre_v1_full_backup.sql
# OPTION 2: Restore specific tables (if partial restore needed)
pg_restore -U postgres -d igny8_db -t "specific_table" backups/20260109_pre_v1_full_backup.sql
# OPTION 3: Restore from Docker backup (if using Docker)
docker-compose exec -T db psql -U postgres igny8_db < backups/20260109_pre_v1_full_backup.sql
```
#### Restore Media Files
```bash
cd /data/app/igny8
tar -xzf backups/20260109_media_backup.tar.gz -C backend/
```
#### Verify Restore
```bash
cd backend
python manage.py shell << 'EOF'
from igny8_core.auth.models import Site
from igny8_core.business.planning.models import Keywords
print(f"Sites restored: {Site.objects.count()}")
print(f"Keywords restored: {Keywords.objects.count()}")
EOF
```
---
## ❓ FAQ
### Q: Can I run these commands multiple times?
**A:**
- **Export Config**: Yes, safe to run multiple times. Creates timestamped backups.
- **Cleanup**: Yes, but after first cleanup there's nothing left to delete (idempotent).
### Q: What if I only want to delete some data?
**A:** These commands are all-or-nothing by design for safety. To delete specific data, use Django admin or write a custom management command.
### Q: Can I restore individual items from the export?
**A:** Yes! The JSON files use Django's standard serialization format. Use `python manage.py loaddata <file>.json` to restore.
### Q: Will this affect my development environment?
**A:** Only if you run it there. These commands work on whatever database your Django settings point to.
### Q: How long does cleanup take?
**A:** Depends on data volume. Typical ranges:
- Small (< 10k records): 1-5 seconds
- Medium (10k-100k): 5-30 seconds
- Large (> 100k): 30-120 seconds
### Q: What if cleanup fails halfway?
**A:** Can't happen - it's wrapped in an atomic transaction. Either everything deletes or nothing does.
### Q: Do I need to stop the application?
**A:** Recommended but not required. Stopping the app prevents race conditions during cleanup.
### Q: Can I schedule these as cron jobs?
**A:**
- **Export**: Yes, great for automated backups
- **Cleanup**: No, should only be run manually with explicit confirmation
### Q: What about Django migrations?
**A:** Cleanup only deletes data, not schema. All tables and migrations remain intact.
### Q: How do I know if my system config is complete?
**A:** Run the export and review the counts in `export_metadata.json`. Compare with your documentation.
---
## 📞 Support
### If You Need Help
1. **Check this guide** thoroughly first
2. **Review error messages** carefully
3. **Test in staging** before production
4. **Contact team** if unsure about any step
### Emergency Contacts
- **Database Issues**: DBA team
- **Application Issues**: Backend team
- **Configuration Questions**: System admin
- **Rollback Needed**: All hands on deck!
---
## ✅ Success Criteria
After completing Phase 6, you should have:
- ✅ Multiple timestamped config exports in `backups/config/`
- ✅ Full database SQL backup in `backups/`
- ✅ Media files backup in `backups/`
- ✅ Zero user-generated data in database
- ✅ All system configurations intact
- ✅ Application starts and loads empty state
- ✅ Admin can log in
- ✅ New users can sign up
- ✅ Plans visible and functional
- ✅ Git tag created for v1.0.0-clean
---
**Document Version:** 1.0
**Last Updated:** January 9, 2026
**Next Review:** After V1.0 Launch
---
*This guide is part of the IGNY8 Pre-Launch Preparation (Phase 6)*