Phase 3 - credts, usage, plans app pages #Migrations

This commit is contained in:
IGNY8 VPS (Salman)
2026-01-06 21:28:13 +00:00
parent cb8e747387
commit 9ca048fb9d
37 changed files with 9328 additions and 1149 deletions

View File

@@ -0,0 +1,653 @@
# Final Credits & Limits Implementation Plan
**Created:** January 5, 2026
**Status:** ✅ COMPLETE (Pending UAT & Production Deployment)
**Last Updated:** January 5, 2026
**Verified Against:** Backend codebase, Frontend codebase, Current implementation
---
## Executive Summary
After comprehensive codebase review, this document presents the verified and refined implementation plan for simplifying the credits and limits system.
### Key Finding: Credit Flow is CORRECT ✅
The current credit architecture is **sound and logical**:
```
┌─────────────────────────────────────────────────────────────────┐
│ CREDIT FLOW (VERIFIED) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Plan.included_credits = Monthly allocation (e.g., 10,000) │
│ ↓ (Added on subscription renewal/approval) │
│ Account.credits = Current balance (real-time, decremented) │
│ ↓ (Decremented on each AI operation) │
│ CreditTransaction = Log of all credit changes │
│ CreditUsageLog = Detailed operation tracking │
│ │
│ THESE ARE NOT PARALLEL - They serve different purposes: │
│ • Plan.included_credits = "How many credits per month" │
│ • Account.credits = "How many credits you have RIGHT NOW" │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Where credits are added to Account.credits:**
1. `billing/views.py:445-465` - When manual payment is approved
2. `payment_service.py:219-249` - When credit package purchased
3. `credit_service.py:413-445` - Generic `add_credits()` method
---
## Part 1: What To KEEP (Verified Working)
### 1.1 Credit System (NO CHANGES NEEDED)
| Component | Location | Status |
|-----------|----------|--------|
| `Account.credits` | `auth/models.py:83` | ✅ Keep - Real-time balance |
| `Plan.included_credits` | `auth/models.py:253` | ✅ Keep - Monthly allocation |
| `CreditService` | `billing/services/credit_service.py` | ✅ Keep - All methods working |
| `CreditTransaction` | `billing/models.py:26-48` | ✅ Keep - Audit trail |
| `CreditUsageLog` | `billing/models.py:90-130` | ✅ Keep - Operation tracking |
### 1.2 Hard Limits to KEEP (4 only)
| Limit | Plan Field | Current Location | Status |
|-------|------------|------------------|--------|
| Sites | `max_sites` | `auth/models.py:207` | ✅ Keep |
| Users | `max_users` | `auth/models.py:204` | ✅ Keep |
| Keywords | `max_keywords` | `auth/models.py:214` | ✅ Keep |
| **Ahrefs Queries** | `max_ahrefs_queries` | **NEW** | Add |
---
## Part 2: What To REMOVE (Verified Unused)
### 2.1 Remove From Plan Model
These fields create confusing "double limiting" - credits already control consumption:
```python
# auth/models.py - REMOVE these fields (lines 220-251)
max_clusters = ... # Remove - no real use
max_content_ideas = ... # Remove - credits handle this
max_content_words = ... # Remove - credits handle this
max_images_basic = ... # Remove - credits handle this
max_images_premium = ... # Remove - credits handle this
max_image_prompts = ... # Remove - credits handle this
```
**Why remove?** User confusion: "I have credits but can't generate because I hit my word limit?"
### 2.2 Remove From Account Model
```python
# auth/models.py - REMOVE these fields (lines 108-114)
usage_content_ideas = ... # Remove - tracked in CreditUsageLog
usage_content_words = ... # Remove - tracked in CreditUsageLog
usage_images_basic = ... # Remove - tracked in CreditUsageLog
usage_images_premium = ... # Remove - tracked in CreditUsageLog
usage_image_prompts = ... # Remove - tracked in CreditUsageLog
```
### 2.3 Update LimitService
```python
# limit_service.py - Update HARD_LIMIT_MAPPINGS
HARD_LIMIT_MAPPINGS = {
'sites': {...}, # Keep
'users': {...}, # Keep
'keywords': {...}, # Keep
# 'clusters': {...}, # REMOVE
}
# limit_service.py - Update MONTHLY_LIMIT_MAPPINGS
MONTHLY_LIMIT_MAPPINGS = {
'ahrefs_queries': { # NEW - only monthly limit
'plan_field': 'max_ahrefs_queries',
'usage_field': 'usage_ahrefs_queries',
'display_name': 'Keyword Research Queries',
},
# REMOVE all others (content_ideas, content_words, images_*, image_prompts)
}
```
---
## Part 3: Database Migration
### Migration Script
```python
# migrations/0XXX_simplify_credits_limits.py
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('igny8_core_auth', '0XXX_previous'), # Update with actual
]
operations = [
# STEP 1: Add new Ahrefs fields
migrations.AddField(
model_name='plan',
name='max_ahrefs_queries',
field=models.IntegerField(
default=0,
help_text='Monthly Ahrefs keyword research queries (0 = disabled)'
),
),
migrations.AddField(
model_name='account',
name='usage_ahrefs_queries',
field=models.IntegerField(
default=0,
help_text='Ahrefs queries used this month'
),
),
# STEP 2: Remove unused Plan fields
migrations.RemoveField(model_name='plan', name='max_clusters'),
migrations.RemoveField(model_name='plan', name='max_content_ideas'),
migrations.RemoveField(model_name='plan', name='max_content_words'),
migrations.RemoveField(model_name='plan', name='max_images_basic'),
migrations.RemoveField(model_name='plan', name='max_images_premium'),
migrations.RemoveField(model_name='plan', name='max_image_prompts'),
# STEP 3: Remove unused Account fields
migrations.RemoveField(model_name='account', name='usage_content_ideas'),
migrations.RemoveField(model_name='account', name='usage_content_words'),
migrations.RemoveField(model_name='account', name='usage_images_basic'),
migrations.RemoveField(model_name='account', name='usage_images_premium'),
migrations.RemoveField(model_name='account', name='usage_image_prompts'),
]
```
---
## Part 4: Frontend Changes
### 4.1 Plans & Billing Page (Financial Focus)
**File:** `frontend/src/pages/account/PlansAndBillingPage.tsx`
**Current (881 lines):** Shows credit usage charts, limit bars = DUPLICATE of Usage page
**Target:** Pure financial focus
- Tab 1: Current Plan → Name, price, renewal date, "View Usage" link
- Tab 2: Upgrade Plan → Pricing table (already good)
- Tab 3: Billing History → Invoices, payment methods
**Remove from Current Plan tab:**
- Usage charts (move to Usage page)
- Credit consumption breakdown (move to Usage page)
- Limit progress bars (move to Usage page)
### 4.2 Usage Analytics Page (Consumption Tracking)
**File:** `frontend/src/pages/account/UsageAnalyticsPage.tsx`
**Current (266 lines):** Basic tabs with some usage data
**Target:** Comprehensive usage tracking
```
Tab 1: Overview (NEW)
├── Quick Stats Cards: Credits Balance, Sites, Users, Keywords
├── Period Selector: 7 days | 30 days | 90 days
└── Key Metrics for Selected Period
Tab 2: Your Limits (SIMPLIFIED)
└── Progress Bars for ONLY 4 limits:
├── Sites: 2 / 5
├── Users: 2 / 3
├── Keywords: 847 / 1,000
└── Keyword Research Queries: 23 / 50 this month
Tab 3: Credit Insights (NEW)
├── Credits by Site (pie chart)
├── Credits by Action Type (bar chart)
├── Credits by Image Quality (basic vs premium)
├── Credits by Automation (manual vs automated)
└── Timeline Chart (line graph over time)
Tab 4: Activity Log (renamed from "API Activity")
└── Detailed transaction history (existing functionality)
```
### 4.3 CreditBalance Interface Update
**File:** `frontend/src/services/billing.api.ts`
```typescript
// Current (correct - no changes needed)
export interface CreditBalance {
credits: number; // Account.credits (current balance)
plan_credits_per_month: number; // Plan.included_credits
credits_used_this_month: number; // Sum of CreditUsageLog this month
credits_remaining: number; // = credits (same as current balance)
}
```
### 4.4 UsageSummary Interface Update
**File:** `frontend/src/services/billing.api.ts` (add new interface)
```typescript
export interface UsageSummary {
// Hard limits (4 only)
hard_limits: {
sites: { current: number; limit: number; display_name: string };
users: { current: number; limit: number; display_name: string };
keywords: { current: number; limit: number; display_name: string };
ahrefs_queries: { current: number; limit: number; display_name: string };
};
// Credits
credits: {
balance: number;
plan_allocation: number;
used_this_month: number;
};
// Period info
period: {
start: string;
end: string;
days_remaining: number;
};
}
```
---
## Part 5: Keyword Research Feature
### 5.1 Two-Option Structure
```
┌───────────────────────────────────────────────────────────────────┐
│ KEYWORD RESEARCH PAGE │
├───────────────────────────────────────────────────────────────────┤
│ │
│ [Browse IGNY8 Database] [Research with Ahrefs - 42/50 left] │
│ │
├───────────────────────────────────────────────────────────────────┤
│ Option 1: Browse Pre-Researched Keywords (FREE) │
│ • Global IGNY8 keyword database │
│ • Pre-analyzed metrics from our research │
│ • Free to browse and add │
│ • Counts toward max_keywords when added │
│ │
├───────────────────────────────────────────────────────────────────┤
│ Option 2: Research with Ahrefs (LIMITED) │
│ • Live Ahrefs API queries │
│ • Fresh, custom keyword data │
│ • Monthly limit: 42 / 50 queries remaining │
│ • ⚠️ Each search uses 1 query from monthly limit │
│ │
└───────────────────────────────────────────────────────────────────┘
```
### 5.2 Plan Tiers for Ahrefs ✅ CONFIGURED
| Plan | max_ahrefs_queries | Description |
|------|-------------------|-------------|
| Free | 0 | Browse only (no live Ahrefs) |
| Starter | 50 | 50 queries/month |
| Growth | 200 | 200 queries/month |
| Scale | 500 | 500 queries/month |
---
## Part 6: Enforcement Points ✅ COMPLETED
### 6.1 Backend Validation Checklist ✅
| Check Point | Location | Status |
|-------------|----------|--------|
| **Keywords** | | |
| Manual creation | `planner/views.py:perform_create` | ✅ Implemented |
| Bulk import | `planner/views.py:import_keywords` | ✅ Implemented |
| **Sites** | | |
| Site creation | `defaults_service.py` | ✅ Implemented |
| **Users** | | |
| User invite | `account_views.py:team_invite` | ✅ Implemented |
| **Credits** | | |
| All AI ops | `ai/engine.py` | ✅ Pre-flight checks exist |
### 6.2 Frontend Pre-Check ✅ IMPLEMENTED
Components created:
- `frontend/src/components/billing/InsufficientCreditsModal.tsx` - Modal with upgrade/buy options
- `frontend/src/utils/creditCheck.ts` - Pre-flight credit check utility
```typescript
// Usage example
import { checkCreditsBeforeOperation } from '@/utils/creditCheck';
import { useInsufficientCreditsModal } from '@/components/billing/InsufficientCreditsModal';
const { showModal } = useInsufficientCreditsModal();
const check = await checkCreditsBeforeOperation(estimatedCost);
if (!check.hasEnoughCredits) {
showModal({
requiredCredits: check.requiredCredits,
availableCredits: check.availableCredits,
});
return;
}
```
---
## Part 7: Implementation Timeline
### Week 1: Backend Foundation ✅ COMPLETED
- [x] Create database migration (add Ahrefs fields, remove unused)
- [x] Update Plan model (remove 6 fields, add 1)
- [x] Update Account model (remove 5 fields, add 1)
- [x] Update LimitService (simplify mappings)
- [x] Run migration on dev/staging
- [x] Update PlanSerializer (fix removed field references)
- [x] Update management commands (create_aws_admin_tenant, get_account_limits)
### Week 2: Backend Enforcement ✅ COMPLETED
- [x] Add keyword limit checks at all entry points (perform_create, import_keywords)
- [ ] Create Ahrefs query endpoint with limit check (deferred - Ahrefs not yet integrated)
- [x] Update usage summary endpoint
- [x] Add site limit check (defaults_service.py)
- [x] Add user limit check (account_views.py - team invite)
- [x] Update API serializers
- [x] AI pre-flight credit checks (already in ai/engine.py)
### Week 3: Frontend - Plans & Billing ✅ COMPLETED
- [x] Update billing.api.ts interfaces (Plan, UsageSummary)
- [x] Update UsageLimitsPanel.tsx (4 limits only)
- [x] Update PlansAndBillingPage.tsx
- [x] Update pricing-table component
- [x] Update pricingHelpers.ts
- [x] Update Plans.tsx, SignUp.tsx, SignUpFormUnified.tsx
### Week 4: Frontend - Usage Analytics ✅ COMPLETED
- [x] UsageAnalyticsPage uses UsageLimitsPanel (already updated)
- [x] Your Limits tab shows 4 limits only
- [x] Create Credit Insights tab with charts
- [x] Overview quick stats visible on all tabs
### Week 5: Testing & Documentation ✅ COMPLETED
- [ ] Run full test suite (pending - manual testing done)
- [x] Update API documentation (docs/10-MODULES/BILLING.md)
- [x] Update user documentation (docs/40-WORKFLOWS/CREDIT-SYSTEM.md)
- [ ] UAT testing (pending)
- [ ] Production deployment (pending)
---
## Part 8: Verification Checklist
### 8.1 Docker Verification Commands
```bash
# Enter Django shell
docker exec -it igny8_backend python manage.py shell
# Verify Plan model changes
from igny8_core.auth.models import Plan
plan = Plan.objects.first()
print(f"max_sites: {plan.max_sites}")
print(f"max_users: {plan.max_users}")
print(f"max_keywords: {plan.max_keywords}")
print(f"max_ahrefs_queries: {plan.max_ahrefs_queries}")
# These should ERROR after migration:
# print(f"max_clusters: {plan.max_clusters}") # Should fail
# Verify Account model changes
from igny8_core.auth.models import Account
account = Account.objects.first()
print(f"credits: {account.credits}")
print(f"usage_ahrefs_queries: {account.usage_ahrefs_queries}")
# These should ERROR after migration:
# print(f"usage_content_ideas: {account.usage_content_ideas}") # Should fail
# Verify LimitService
from igny8_core.business.billing.services.limit_service import LimitService
print(f"Hard limits: {list(LimitService.HARD_LIMIT_MAPPINGS.keys())}")
# Should be: ['sites', 'users', 'keywords']
print(f"Monthly limits: {list(LimitService.MONTHLY_LIMIT_MAPPINGS.keys())}")
# Should be: ['ahrefs_queries']
# Verify Credit Flow
from igny8_core.business.billing.services.credit_service import CreditService
print("Credit calculation for content:")
credits = CreditService.calculate_credits_from_tokens('content_generation', 1000, 500)
print(f" 1000 input + 500 output tokens = {credits} credits")
```
### 8.2 Django Admin Verification
**Check in browser at `/admin/`:**
1. **Accounts → Account**
- [ ] `credits` field visible
- [ ] `usage_ahrefs_queries` field visible
- [ ] `usage_content_ideas` NOT visible (removed)
2. **Accounts → Plan**
- [ ] `max_sites`, `max_users`, `max_keywords` visible
- [ ] `max_ahrefs_queries` visible
- [ ] `max_content_ideas`, `max_content_words` NOT visible (removed)
- [ ] `included_credits` visible
3. **Billing → Credit Transaction**
- [ ] Shows all credit additions/deductions
- [ ] `balance_after` shows running total
4. **Billing → Credit Usage Log**
- [ ] Shows all AI operations
- [ ] `operation_type`, `credits_used`, `tokens_input`, `tokens_output` visible
### 8.3 Frontend Verification
**Plans & Billing Page (`/account/billing`):**
- [ ] Current Plan tab shows: Plan name, price, renewal date
- [ ] Current Plan tab does NOT show usage charts
- [ ] "View Usage Details" link works
- [ ] Upgrade tab shows pricing table
- [ ] Billing History shows invoices
**Usage Analytics Page (`/account/usage`):**
- [ ] Overview tab shows quick stats
- [ ] Your Limits tab shows ONLY 4 limits
- [ ] Credit Insights tab shows breakdown charts
- [ ] Activity Log tab shows transaction history
**Header Credit Display:**
- [ ] Shows current credit balance
- [ ] Updates after AI operations
---
## Part 9: Error Messages (User-Friendly)
### Before (Technical)
```
HTTP 402 - HardLimitExceededError: max_keywords limit reached
```
### After (User-Friendly)
```
┌─────────────────────────────────────────────────────────┐
│ ⚠️ Keyword Limit Reached │
├─────────────────────────────────────────────────────────┤
│ │
│ You've reached your keyword limit. │
│ │
│ Current: 1,000 keywords │
│ Your plan allows: 1,000 keywords │
│ │
│ To add more keywords: │
│ • Delete unused keywords │
│ • Upgrade your plan │
│ │
│ [Manage Keywords] [Upgrade Plan] [Cancel] │
│ │
└─────────────────────────────────────────────────────────┘
```
---
## Part 10: Risk Mitigation
| Risk | Mitigation |
|------|------------|
| Migration breaks production | Test on staging with prod data clone first |
| Users lose tracked usage | Keep CreditUsageLog (detailed tracking continues) |
| Ahrefs costs spike | Monthly limit enforced server-side |
| Credit confusion | Clear documentation + help tooltips |
| Rollback needed | Keep migration reversible (add back fields) |
---
## Appendix A: Files to Modify
### Backend Files
| File | Changes |
|------|---------|
| `backend/igny8_core/auth/models.py` | Remove 11 fields, add 2 fields |
| `backend/igny8_core/business/billing/services/limit_service.py` | Simplify mappings |
| `backend/igny8_core/business/billing/serializers.py` | Update serializers |
| `backend/igny8_core/modules/billing/views.py` | Update usage summary |
| `backend/igny8_core/admin/` | Update admin panels |
| `backend/migrations/` | New migration file |
### Frontend Files
| File | Changes |
|------|---------|
| `frontend/src/pages/account/PlansAndBillingPage.tsx` | Simplify (remove usage) |
| `frontend/src/pages/account/UsageAnalyticsPage.tsx` | Add new tabs |
| `frontend/src/services/billing.api.ts` | Update interfaces |
| `frontend/src/components/billing/UsageLimitsPanel.tsx` | Show 4 limits only |
### Documentation Files
| File | Changes |
|------|---------|
| `docs/10-MODULES/BILLING.md` | Update limits documentation |
| `docs/40-WORKFLOWS/CREDIT-SYSTEM.md` | Update credit flow docs |
---
## Conclusion
The credit system architecture is **fundamentally correct**:
- `Plan.included_credits` defines monthly allocation
- `Account.credits` tracks real-time balance
- Credits are added on subscription renewal/payment approval
- Credits are deducted on each AI operation
**What's broken:**
- Too many unused limits causing user confusion
- Duplicate data displayed across pages
- Monthly limits (content_words, images, etc.) that duplicate what credits already control
**The fix:**
- Simplify to 4 hard limits + credits
- Clear page separation (financial vs consumption)
- Better UX with multi-dimensional credit insights
---
**Document Version:** 1.0
**Last Updated:** January 5, 2026
**Verified By:** Codebase review of backend and frontend
##############################################
## Organized Implementation Changes Plan
### 1. **Credit Consumption Widget (Rename & Enhance)**
**Current:** "Where Credits Go" with just a pie chart
**New:** "Credit Consumption" with pie chart + detailed table
| Operation | Credits Used | Items Created |
|-----------|-------------|---------------|
| Clustering | 150 | 12 clusters |
| Ideas | 200 | 45 ideas |
| Content | 1,200 | 24 articles |
| Image Prompts | 50 | 30 prompts |
| Images (Basic) | 100 | 100 images |
| Images (Quality) | 250 | 50 images |
| Images (Premium) | 450 | 30 images |
- Pie chart shows credits consumed per operation
- Table shows both **credits** AND **output count**
- Learn from Planner/Writer footer workflow completion widgets
---
### 2. **New Usage Logs Page** (`/account/usage/logs`)
**Purpose:** Detailed, filterable, paginated log of all AI operations
**Layout:** Same as Planner/Writer table pages (consistent template)
**Table Columns:**
| Date | Operation | Details | Credits | Cost (USD) |
|------|-----------|---------|-------|--------|---------|------------|
| Jan 5, 2:30pm | Content Writing | "SEO Guide for..." | 35 | $0.042 |
| Jan 5, 2:15pm | Image Generation | Premium quality | 15 | $0.18 |
**Features:**
- Filters: Operation type, date range, site
- Pagination
- USD cost calculated from token pricing (from AIModelConfig)
- Link to related content where applicable
- Export option
---
### 3. **Usage Dashboard - Multi-Dimensional Widgets**
**Remove:**
- ❌ "Operations" summary card (single metrics, not useful)
- ❌ "Recent Activity" card (move to Usage Logs page)
**Add/Enhance:**
-**Credit Consumption** (pie + table as described above)
-**Usage Trend** - Line chart showing daily/weekly credit usage
-**Credits by Site** - If multiple sites, show distribution
-**Image Quality Breakdown** - Basic vs Quality vs Premium usage
-**Top Operations** - Which operations consume most credits
-**Quick link** to Usage Logs page for detailed view
---
### 4. **Data Sources Needed**
From backend, I'll need to check:
- `CreditUsageLog` - Has operation_type, credits_used, model_used, tokens_in, tokens_out
- `AIModelConfig` - Has cost_per_1k_input, cost_per_1k_output for USD calculation
- Aggregations for: clusters created, ideas generated, articles written, images by tier
---
### Summary of Deliverables
| Item | Type | Description |
|------|------|-------------|
| 1 | Widget Enhancement | "Credit Consumption" - pie chart + table with credits & output counts |
| 2 | New Page | `/account/usage/logs` - Filterable, paginated AI operation logs with USD costs |
| 3 | Dashboard Update | Remove weak widgets, add multi-dimensional data-rich charts |
| 4 | Sidebar | Add "Usage Logs" as sub-item under Usage Dashboard |
---
**Does this organization look correct?** Should I proceed with implementation, or would you like to adjust any part of this plan?