Files
igny8/multi-tenancy/faulty-docs-with issues/FINAL-IMPLEMENTATION-REQUIREMENTS.md
IGNY8 VPS (Salman) c54db6c2d9 reorg
2025-12-08 20:15:09 +00:00

690 lines
20 KiB
Markdown

# Final Implementation Requirements & Constraints
## Complete Specification - Ready for Implementation
**Status:** Complete specification, ready to begin
**Critical Issues:** 4 major + original gaps
**Implementation Time:** 7-10 days
---
## Summary of All Requirements
This document consolidates:
1. Original tenancy gaps (from audits)
2. Free trial signup simplification
3. Four critical new issues discovered
4. Current database state context
---
## CRITICAL ISSUE A: Plan Allocation & Credits Must Be Strict
### Problem
- Inconsistent plan fallback logic in old code
- Some accounts created with 0 credits despite plan having credits
- Enterprise plan being auto-assigned (should never happen)
- Multiple fallback paths causing confusion
### Strict Rules (NO EXCEPTIONS)
#### Rule A1: Free Trial Signup
```python
# /signup route ALWAYS assigns:
plan_slug = "free-trial" # First choice
if not exists:
plan_slug = "free" # ONLY fallback
# NEVER assign: starter, growth, scale, enterprise automatically
```
#### Rule A2: Credit Seeding (MANDATORY)
```python
# On account creation, ALWAYS:
account.credits = plan.get_effective_credits_per_month()
account.status = 'trial' # For free-trial/free plans
# Log transaction:
CreditTransaction.create(
account=account,
transaction_type='subscription',
amount=credits,
description='Initial credits from {plan.name}',
metadata={'registration': True, 'plan_slug': plan.slug}
)
```
#### Rule A3: Enterprise Plan Protection
```python
# Enterprise plan (slug='enterprise') must NEVER be auto-assigned
# Only Developer/Admin can manually assign enterprise
# Check in serializer:
if plan.slug == 'enterprise' and not user.is_developer():
raise ValidationError("Enterprise plan requires manual assignment")
```
#### Rule A4: Paid Plan Assignment
```python
# Paid plans (starter, growth, scale) can ONLY be assigned:
# 1. From /account/upgrade endpoint (inside app)
# 2. After payment confirmation
# NEVER during initial /signup
```
### Implementation Location
- **File:** [`backend/igny8_core/auth/serializers.py:276`](backend/igny8_core/auth/serializers.py:276)
- **Changes:** Already applied, but needs enterprise protection added
---
## CRITICAL ISSUE B: Subscription Date Accuracy
### Problem
- Trial accounts have missing or incorrect period dates
- Bank transfer activation doesn't set proper subscription periods
- No clear rule for date calculation
### Strict Rules (ZERO AMBIGUITY)
#### Rule B1: Free Trial Signup
```python
from django.utils import timezone
from datetime import timedelta
# Constants
TRIAL_DAYS = 14 # or 30, must be defined
# On registration:
now = timezone.now()
subscription = Subscription.objects.create(
account=account,
status='trialing',
payment_method='trial', # or None
current_period_start=now,
current_period_end=now + timedelta(days=TRIAL_DAYS),
cancel_at_period_end=False
)
account.status = 'trial'
```
#### Rule B2: Bank Transfer Activation
```python
# When admin confirms payment:
now = timezone.now()
# For monthly plan:
if plan.billing_cycle == 'monthly':
period_end = now + timedelta(days=30)
elif plan.billing_cycle == 'annual':
period_end = now + timedelta(days=365)
subscription.payment_method = 'bank_transfer'
subscription.external_payment_id = payment_ref
subscription.status = 'active'
subscription.current_period_start = now
subscription.current_period_end = period_end
subscription.save()
account.status = 'active'
account.credits = plan.get_effective_credits_per_month()
account.save()
```
#### Rule B3: Subscription Renewal
```python
# On renewal (manual or webhook):
previous_end = subscription.current_period_end
# Set new period (NO GAP, NO OVERLAP)
subscription.current_period_start = previous_end
if plan.billing_cycle == 'monthly':
subscription.current_period_end = previous_end + timedelta(days=30)
elif plan.billing_cycle == 'annual':
subscription.current_period_end = previous_end + timedelta(days=365)
# Reset credits
account.credits = plan.get_effective_credits_per_month()
account.save()
subscription.save()
```
### Implementation Location
- **File:** `backend/igny8_core/business/billing/views.py` (bank transfer endpoint)
- **File:** [`backend/igny8_core/auth/serializers.py:276`](backend/igny8_core/auth/serializers.py:276) (registration)
---
## CRITICAL ISSUE C: Superuser Session Contamination
### Problem
**CRITICAL SECURITY ISSUE:**
- New regular users sometimes logged in as superuser
- Frontend picks up admin/developer session from same browser
- Catastrophic for tenancy isolation
### Root Cause
**Session auth + JWT auth coexistence:**
- Admin logs into Django admin → Session cookie created
- Regular user visits frontend → Browser sends session cookie
- Backend authenticates as admin instead of JWT user
- Frontend suddenly has superuser access
### Strict Fix (MANDATORY)
#### Fix C1: Disable Session Auth for API Routes
**File:** [`backend/igny8_core/api/authentication.py`](backend/igny8_core/api/authentication.py)
```python
# ViewSets should ONLY use:
authentication_classes = [JWTAuthentication] # NO CSRFExemptSessionAuthentication
# Exception: Admin panel can use session
# But /api/* routes must be JWT-only
```
#### Fix C2: Middleware Superuser Detection
**File:** [`backend/igny8_core/auth/middleware.py:25`](backend/igny8_core/auth/middleware.py:25)
Add after account validation:
```python
def process_request(self, request):
# ... existing code ...
# CRITICAL: Detect superuser on non-admin routes
if not request.path.startswith('/admin/'):
if hasattr(request, 'user') and request.user and request.user.is_superuser:
# Non-admin route but superuser authenticated
# This should ONLY happen for JWT with developer role
auth_header = request.META.get('HTTP_AUTHORIZATION', '')
if not auth_header.startswith('Bearer '):
# Superuser via session, not JWT - BLOCK IT
from django.contrib.auth import logout
logout(request)
return JsonResponse({
'success': False,
'error': 'Session authentication not allowed for API routes. Please use JWT.'
}, status=403)
```
#### Fix C3: Frontend Explicit Logout on Register
**File:** [`frontend/src/store/authStore.ts:120`](frontend/src/store/authStore.ts:120)
Before registration:
```typescript
register: async (registerData) => {
// Clear any existing sessions first
try {
await fetch(`${API_BASE_URL}/v1/auth/logout/`, {
method: 'POST',
credentials: 'include' // Clear session cookies
});
} catch (e) {
// Ignore errors, just ensure clean state
}
set({ loading: true });
// ... rest of registration ...
}
```
#### Fix C4: Frontend Clear All Auth on Logout
```typescript
logout: () => {
// Clear cookies
document.cookie.split(";").forEach(c => {
document.cookie = c.trim().split("=")[0] + "=;expires=Thu, 01 Jan 1970 00:00:00 UTC;path=/";
});
// Clear localStorage
localStorage.clear();
// Clear state
set({ user: null, token: null, refreshToken: null, isAuthenticated: false, loading: false });
},
```
### Implementation Priority
🔥 **CRITICAL** - Fix before any production deployment
---
## CRITICAL ISSUE D: Docker Build Cache Causing Router Errors
### Problem
**Symptoms:**
- `useLocation() may be used only in the context of a <Router> component`
- `useNavigate` similar errors
- Errors appear in: Planner, Writer, Sites modules and subpages
- **Resolved by removing containers and rebuilding WITHOUT code change**
### Root Cause
**Not a code issue - Docker build cache issue**
- Stale node_modules cached in Docker layers
- Stale build artifacts from previous versions
- React Router hydration mismatch between cached and new code
### Strict Fix
#### Fix D1: Frontend Dockerfile - No Build Cache
**File:** `frontend/Dockerfile.dev`
Ensure these lines:
```dockerfile
# Copy package files
COPY package*.json ./
# Clean install (no cache)
RUN npm ci --only=production=false
# Remove any cached builds
RUN rm -rf dist/ .vite/ node_modules/.vite/
# Copy source
COPY . .
```
#### Fix D2: Docker Compose - No Volume Cache for node_modules
**File:** [`docker-compose.app.yml:77`](docker-compose.app.yml:77)
Current:
```yaml
volumes:
- /data/app/igny8/frontend:/app:rw
```
Change to:
```yaml
volumes:
- /data/app/igny8/frontend:/app:rw
# Exclude node_modules from volume mount to prevent cache issues
- /app/node_modules
```
#### Fix D3: Build Script - Force Clean Build
**File:** `frontend/rebuild.sh` (create this)
```bash
#!/bin/bash
# Force clean frontend rebuild
echo "Removing old containers..."
docker rm -f igny8_frontend igny8_marketing_dev igny8_sites
echo "Removing old images..."
docker rmi -f igny8-frontend-dev:latest igny8-marketing-dev:latest igny8-sites-dev:latest
echo "Rebuilding without cache..."
cd /data/app/igny8/frontend
docker build --no-cache -t igny8-frontend-dev:latest -f Dockerfile.dev .
docker build --no-cache -t igny8-marketing-dev:latest -f Dockerfile.marketing.dev .
cd /data/app/igny8/sites
docker build --no-cache -t igny8-sites-dev:latest -f Dockerfile.dev .
echo "Restarting containers..."
cd /data/app/igny8
docker compose -f docker-compose.app.yml up -d igny8_frontend igny8_marketing_dev igny8_sites
echo "Done! Frontend rebuilt fresh."
```
#### Fix D4: Deployment Best Practice
```bash
# After git push, ALWAYS do:
docker compose -f docker-compose.app.yml down
docker compose -f docker-compose.app.yml build --no-cache
docker compose -f docker-compose.app.yml up -d
# This ensures no stale cache
```
### Why This Fixes Router Errors
- Fresh node_modules every build
- No stale React Router components
- No hydration mismatches
- Clean build artifacts
---
## Updated Implementation Plan with All Issues
### Phase 0: Pre-Implementation Checklist ✅
- [x] Analyze database state
- [x] Document all relationships
- [x] Identify all gaps
- [x] Create free trial code changes
- [x] Document all 4 critical issues
### Phase 1: Free Trial Signup (Day 1)
**Actions:**
1. ✅ Update RegisterSerializer (already done)
2. ✅ Update SignUpForm (already done)
3. ⏳ Create free-trial plan: `docker exec igny8_backend python manage.py create_free_trial_plan`
4. ✅ Add enterprise plan protection
5. ✅ Create Subscription with correct trial dates
6. Test signup flow
**Critical Constraints:**
- ✅ Must assign free-trial or free ONLY
- ✅ Must seed credits from plan
- ✅ Must create Subscription with trial dates
- ✅ Must log CreditTransaction
### Phase 2: Superuser Session Fix (Day 1 - CRITICAL)
**Actions:**
1. Remove CSRFExemptSessionAuthentication from API ViewSets
2. Add middleware superuser detection
3. Add frontend logout before register
4. Add frontend cookie clearing on logout
5. Test: Regular user cannot access superuser session
**Critical Constraints:**
- 🔥 API routes must be JWT-only
- 🔥 Superuser on API route without JWT = logout
- 🔥 Registration clears old sessions first
### Phase 3: Docker Build Cache Fix (Day 1 - CRITICAL)
**Actions:**
1. Update frontend Dockerfile to use `npm ci`
2. Add node_modules volume exclusion
3. Create rebuild.sh script
4. Document deployment procedure
5. Test: Router errors don't occur after rebuild
**Critical Constraints:**
- 🔥 Always use `--no-cache` for frontend builds
- 🔥 Exclude node_modules from volume mounts
- 🔥 Clean rebuild after every git deployment
### Phase 4: Payment Method Fields (Day 2)
**Actions:**
1. Create migration 0007
2. Add payment_method to Account<br>
3. Add payment_method, external_payment_id to Subscription
4. Make stripe_subscription_id nullable
5. Add 'pending_payment' status
6. Run migration
### Phase 5: Subscription Date Accuracy (Day 2-3)
**Actions:**
1. Update RegisterSerializer to create Subscription with trial dates
2. Update bank transfer endpoint with strict date rules
3. Add renewal logic with correct date transitions
4. Test all date transitions
**Critical Constraints:**
- ✅ Trial: current_period_end = now + TRIAL_DAYS
- ✅ Activation: current_period_end = now + billing_cycle
- ✅ Renewal: current_period_start = previous_end (NO GAP)
### Phase 6: Account Validation Helper (Day 3)
**Actions:**
1. Create validate_account_and_plan() in auth/utils.py
2. Update middleware to use helper
3. Update API key authentication to use helper
4. Test validation blocks suspended/cancelled accounts
### Phase 7: Throttling Fix (Day 4)
**Actions:**
1. Remove blanket authenticated bypass
2. Add get_cache_key() for per-account throttling
3. Test throttling enforces limits per account
### Phase 8: Bank Transfer Endpoint (Day 4-5)
**Actions:**
1. Create BillingViewSet
2. Add confirm_bank_transfer endpoint
3. Add URL routes
4. Test payment confirmation flow
### Phase 9: Comprehensive Tests (Day 6)
**Actions:**
1. Test free trial signup
2. Test credit seeding
3. Test subscription dates
4. Test superuser isolation
5. Test API key validation
6. Test throttling
7. Test bank transfer
### Phase 10: Documentation & Verification (Day 7)
**Actions:**
1. Update all documentation
2. Run full system test
3. Verify all flows
4. Deploy to production
---
## Critical Constraints Summary
### A. Plan & Credits (STRICT)
```
✅ free-trial → free (fallback) → ERROR (nothing else)
✅ Credits always seeded on registration
✅ CreditTransaction always logged
❌ Never auto-assign enterprise
❌ Never allow 0 credits after registration
```
### B. Subscription Dates (PRECISE)
```
✅ Trial: start=now, end=now+14days
✅ Activation: start=now, end=now+billing_cycle
✅ Renewal: start=previous_end, end=start+billing_cycle
❌ No gaps between periods
❌ No overlapping periods
```
### C. Superuser Isolation (SECURITY)
```
✅ API routes: JWT auth ONLY
✅ Superuser on /api/* without JWT → logout + error
✅ Registration clears existing sessions
✅ Logout clears all cookies and localStorage
❌ Never allow session auth for API
❌ Never allow superuser contamination
```
### D. Docker Build (STABILITY)
```
✅ Use npm ci (not npm install)
✅ Exclude node_modules from volume mounts
✅ Always build with --no-cache after git push
✅ Removing containers + rebuild fixes router errors
❌ Don't cache build artifacts between deployments
```
---
## Verification Matrix
### Test 1: Free Trial Signup
```bash
# Prerequisites: free-trial plan exists, code deployed
# Action: Visit /signup, fill form, submit
# Expected:
# - Account created with status='trial'
# - Credits = 2000 (or plan.included_credits)
# - Subscription created with trial dates
# - CreditTransaction logged
# - Redirect to /sites
# - User can immediately use app
# Database check:
docker exec igny8_backend python manage.py shell -c "
from igny8_core.auth.models import User;
u = User.objects.latest('id');
assert u.account.status == 'trial';
assert u.account.credits > 0;
assert u.account.plan.slug in ['free-trial', 'free'];
print('✅ Free trial signup working')
"
```
### Test 2: Superuser Isolation
```bash
# Prerequisites: Regular user account, admin logged into /admin
# Action: Login as regular user in frontend
# Expected:
# - User sees only their account
# - User does NOT have superuser privileges
# - API calls use JWT, not session
# Test:
# Inspect frontend network tab
# All API calls must have: Authorization: Bearer <jwt_token>
# No sessionid cookies sent to /api/*
```
### Test 3: Docker Build Stability
```bash
# Action: Deploy code, rebuild containers
cd /data/app/igny8
docker compose -f docker-compose.app.yml down
docker build --no-cache -t igny8-frontend-dev:latest -f frontend/Dockerfile.dev frontend/
docker compose -f docker-compose.app.yml up -d
# Expected:
# - No useLocation errors
# - No useNavigate errors
# - Planner, Writer, Sites pages load correctly
# - Router context available everywhere
```
### Test 4: Subscription Dates
```bash
# Action: Confirm bank transfer for trial account
curl -X POST /api/v1/billing/confirm-bank-transfer/ \
-H "Authorization: Bearer <admin_jwt>" \
-d '{
"account_id": 123,
"external_payment_id": "BT-001",
"amount": "29.99",
"payer_name": "Test User"
}'
# Expected:
# - subscription.status = 'active'
# - subscription.current_period_start = now
# - subscription.current_period_end = now + 30 days
# - account.status = 'active'
# - account.credits = plan monthly credits
```
---
## Implementation Order (Revised)
### Day 1 (CRITICAL)
1. ✅ Free trial signup (code changes done, need to create plan)
2. 🔥 Superuser session fix (MUST FIX)
3. 🔥 Docker build cache fix (MUST FIX)
### Day 2
4. Payment method fields migration
5. Subscription date accuracy updates
### Day 3
6. Account validation helper
7. API key authentication fix
### Day 4
8. Throttling fix
9. Bank transfer endpoint
### Day 5-6
10. Comprehensive tests
### Day 7
11. Documentation
12. Deployment
13. Verification
---
## Rollback Plan (If Any Issue Occurs)
### Database Rollback
```bash
docker exec igny8_backend python manage.py migrate igny8_core_auth 0006_soft_delete_and_retention
```
### Code Rollback
```bash
git revert <commit_hash>
docker compose -f docker-compose.app.yml down
docker compose -f docker-compose.app.yml up -d
```
### Emergency Disable Feature Flags
Add to settings.py:
```python
# Emergency feature flags
TENANCY_ENABLE_FREE_TRIAL = False # Fall back to old signup
TENANCY_VALIDATE_API_KEY = False # Disable validation temporarily
TENANCY_STRICT_JWT_ONLY = False # Allow session auth temporarily
```
---
## Success Criteria (ALL must pass)
- ✅ Signup creates account with correct credits
- ✅ Subscription has accurate start/end dates
- ✅ Regular users NEVER get superuser access
- ✅ Router errors don't appear after container rebuild
- ✅ API key validates account status
- ✅ Throttling enforces per-account limits
- ✅ Bank transfer confirmation works
- ✅ All tests passing (>80% coverage)
- ✅ Zero authentication bypasses
- ✅ Zero credit seeding failures
---
## Files Reference
### Analysis Documents (This Folder)
1. **CURRENT-STATE-CONTEXT.md** - Database state from Docker query
2. **IMPLEMENTATION-SUMMARY.md** - Context gathering summary
3. **FINAL-IMPLEMENTATION-REQUIREMENTS.md** (this file) - Complete spec
4. **FINAL-IMPLEMENTATION-PLAN-COMPLETE.md** - Detailed phase guide
5. **FREE-TRIAL-SIGNUP-FIX.md** - Signup flow specifics
6. **COMPLETE-IMPLEMENTATION-PLAN.md** - Original gap analysis
7. **Final_Flow_Tenancy.md** - Target flow specifications
8. **Tenancy_Audit_Report.md** - Audit findings
9. **audit_fixes.md** - Previous recommendations
10. **tenancy-implementation-plan.md** - Original plan
### Code Changes Made (Review Before Deploy)
1. `backend/igny8_core/auth/serializers.py` - Free trial registration
2. `frontend/src/components/auth/SignUpForm.tsx` - Simplified signup
3. `backend/igny8_core/auth/management/commands/create_free_trial_plan.py` - Plan creation
### Code Changes Needed (Not Yet Made)
1. Middleware - Superuser detection
2. Authentication - Remove session auth from API
3. Frontend authStore - Clear sessions before register
4. Dockerfile - No-cache build
5. docker-compose.app.yml - Exclude node_modules volume
6. All Phase 4-10 changes from FINAL-IMPLEMENTATION-PLAN-COMPLETE.md
---
## Hand-off Instructions
**To implement this system:**
1. **Review code changes** in serializer and frontend
2. **Start with Day 1 critical fixes:**
- Create free-trial plan
- Fix superuser session contamination
- Fix Docker build caching
3. **Then proceed** through Phase 4-10
4. **Use** `FINAL-IMPLEMENTATION-PLAN-COMPLETE.md` as step-by-step guide
5. **Reference** `CURRENT-STATE-CONTEXT.md` for what exists in DB
**All specifications are complete, accurate, and ready for implementation.**