flower celery
This commit is contained in:
IGNY8 VPS (Salman)
199
docs/BACKUP-SYSTEM.md
Normal file
199
docs/BACKUP-SYSTEM.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# IGNY8 Application Packaging & Backup System
|
||||
|
||||
## Overview
|
||||
|
||||
This system creates portable, clean packages of the IGNY8 application containing only source code (no dependencies or generated artifacts).
|
||||
|
||||
## Components
|
||||
|
||||
### 1. Packaging Script
|
||||
**Location**: `/data/app/igny8/scripts/package_app.sh`
|
||||
|
||||
**What it does**:
|
||||
- Creates compressed archive of application source code
|
||||
- Excludes: `node_modules/`, `.venv/`, `dist/`, `*.sql`, `*.log`, build artifacts
|
||||
- Includes: Source code, Docker configs, documentation
|
||||
- Generates SHA256 checksum for verification
|
||||
- Auto-cleans packages older than 7 days
|
||||
- Package size: ~44 MB (compressed)
|
||||
|
||||
**Manual Usage**:
|
||||
```bash
|
||||
/data/app/igny8/scripts/package_app.sh
|
||||
```
|
||||
|
||||
### 2. Automated Backup Tasks
|
||||
**Location**: `/data/app/igny8/backend/igny8_core/tasks/backup.py`
|
||||
|
||||
**Celery Tasks**:
|
||||
|
||||
1. **`backup.create_daily_app_package`**
|
||||
- Runs daily at 00:00 UTC
|
||||
- Creates full application package
|
||||
- Logs success/failure
|
||||
- Returns status dict
|
||||
|
||||
2. **`backup.cleanup_old_packages`**
|
||||
- Runs weekly (Monday at 01:00 UTC)
|
||||
- Removes packages older than 30 days
|
||||
- Keeps recent backups safe
|
||||
|
||||
### 3. Celery Beat Schedule
|
||||
**Location**: `/data/app/igny8/backend/igny8_core/celery.py`
|
||||
|
||||
Configured schedules:
|
||||
```python
|
||||
'daily-app-package-backup': {
|
||||
'task': 'backup.create_daily_app_package',
|
||||
'schedule': crontab(hour=0, minute=0), # Daily at midnight
|
||||
},
|
||||
'weekly-package-cleanup': {
|
||||
'task': 'backup.cleanup_old_packages',
|
||||
'schedule': crontab(hour=1, minute=0, day_of_week=1), # Monday 01:00
|
||||
},
|
||||
```
|
||||
|
||||
## Backup Storage
|
||||
|
||||
**Location**: `/data/backups/igny8/`
|
||||
|
||||
**Files created**:
|
||||
```
|
||||
/data/backups/igny8/
|
||||
├── igny8-app-20251210_161527.tar.gz # Compressed package
|
||||
├── igny8-app-20251210_161527.tar.gz.sha256 # Checksum file
|
||||
├── igny8-app-20251211_000000.tar.gz
|
||||
├── igny8-app-20251211_000000.tar.gz.sha256
|
||||
└── ... (7-30 days of backups retained)
|
||||
```
|
||||
|
||||
## Package Contents
|
||||
|
||||
### ✅ INCLUDED (Source code only - ~56 MB uncompressed):
|
||||
- `backend/igny8_core/` - Django application
|
||||
- `backend/manage.py`, `requirements.txt`
|
||||
- `frontend/src/`, `frontend/public/`
|
||||
- `frontend/package.json`, config files
|
||||
- `docs/`, `README.md`
|
||||
- Dockerfiles, docker-compose files
|
||||
|
||||
### ❌ EXCLUDED (Auto-generated - will be rebuilt):
|
||||
- `backend/.venv/` - Python virtual environment
|
||||
- `frontend/node_modules/` - NPM dependencies
|
||||
- `frontend/dist/` - Build output
|
||||
- `*.sql`, `*.sqlite3` - Database files
|
||||
- `logs/` - Runtime logs
|
||||
- `celerybeat-schedule` - Celery state
|
||||
- `staticfiles/` - Collected static files
|
||||
- `__pycache__/`, `*.pyc` - Python cache
|
||||
|
||||
## Restoration Process
|
||||
|
||||
To restore from a package on a new server:
|
||||
|
||||
```bash
|
||||
# 1. Extract package
|
||||
cd /data/app
|
||||
tar -xzf /path/to/igny8-app-YYYYMMDD_HHMMSS.tar.gz
|
||||
|
||||
# 2. Verify checksum (optional but recommended)
|
||||
sha256sum -c igny8-app-YYYYMMDD_HHMMSS.tar.gz.sha256
|
||||
|
||||
# 3. Follow INSTALLATION.md to:
|
||||
# - Build Docker images (installs all dependencies)
|
||||
# - Start services
|
||||
# - Run migrations
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
**Check backup status**:
|
||||
```bash
|
||||
# List recent backups
|
||||
ls -lh /data/backups/igny8/
|
||||
|
||||
# View Celery Beat schedule
|
||||
docker exec igny8_celery_beat celery -A igny8_core inspect scheduled
|
||||
|
||||
# View Celery logs
|
||||
docker logs igny8_celery_beat
|
||||
docker logs igny8_celery_worker
|
||||
```
|
||||
|
||||
**Verify backup worked**:
|
||||
```bash
|
||||
# Check if backup ran
|
||||
ls -lt /data/backups/igny8/ | head -5
|
||||
|
||||
# Verify package integrity
|
||||
cd /data/backups/igny8/
|
||||
sha256sum -c igny8-app-*.tar.gz.sha256
|
||||
```
|
||||
|
||||
## Retention Policy
|
||||
|
||||
- **Daily backups**: Kept for 7 days in packaging script
|
||||
- **Long-term backups**: Kept for 30 days by cleanup task
|
||||
- **Manual backups**: Never auto-deleted (run script manually with different output dir)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Backup didn't run at midnight**:
|
||||
```bash
|
||||
# Check Celery Beat is running
|
||||
docker ps | grep celery_beat
|
||||
|
||||
# Check Beat logs
|
||||
docker logs igny8_celery_beat
|
||||
|
||||
# Manually trigger backup
|
||||
docker exec igny8_celery_worker celery -A igny8_core call backup.create_daily_app_package
|
||||
```
|
||||
|
||||
**Insufficient disk space**:
|
||||
```bash
|
||||
# Check disk usage
|
||||
df -h /data
|
||||
|
||||
# Clean old backups manually
|
||||
find /data/backups/igny8/ -name "*.tar.gz" -mtime +30 -delete
|
||||
```
|
||||
|
||||
**Package too large**:
|
||||
- Check no large files accidentally included
|
||||
- Verify `.git` folder is excluded
|
||||
- Ensure `node_modules`, `.venv` excluded
|
||||
|
||||
## Security Notes
|
||||
|
||||
1. **Packages contain source code** - store securely
|
||||
2. **No secrets in package** - `.env` is excluded
|
||||
3. **Checksum verification** - always verify SHA256 before restoring
|
||||
4. **Backup the backups** - Consider copying to remote storage
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Action | Command |
|
||||
|--------|---------|
|
||||
| Create backup now | `/data/app/igny8/scripts/package_app.sh` |
|
||||
| List backups | `ls -lh /data/backups/igny8/` |
|
||||
| Verify package | `sha256sum -c package.tar.gz.sha256` |
|
||||
| Extract package | `tar -xzf package.tar.gz` |
|
||||
| Check schedule | `docker exec igny8_celery_beat celery -A igny8_core inspect scheduled` |
|
||||
| View logs | `docker logs igny8_celery_beat` |
|
||||
|
||||
## Configuration
|
||||
|
||||
To change backup schedule, edit `/data/app/igny8/backend/igny8_core/celery.py`:
|
||||
|
||||
```python
|
||||
'daily-app-package-backup': {
|
||||
'task': 'backup.create_daily_app_package',
|
||||
'schedule': crontab(hour=0, minute=0), # Change time here
|
||||
},
|
||||
```
|
||||
|
||||
Then restart Celery:
|
||||
```bash
|
||||
docker restart igny8_celery_beat igny8_celery_worker
|
||||
```
|
||||
Reference in New Issue
Block a user