pre-launch-final mods-docs

2025-12-11 07:20:21 +00:00
parent 20fdd3b295
commit a736bc3d34
8 changed files with 5464 additions and 0 deletions
--- a/docs/PRE-LAUNCH/ITEM-2-CREDITS-BILLING-PRICING.md
+++ b/docs/PRE-LAUNCH/ITEM-2-CREDITS-BILLING-PRICING.md
@@ -0,0 +1,628 @@
+# Item 2: Credits, Billing, Pricing Logic, and Usage Limits
+
+**Priority:** Critical  
+**Target:** Production Launch  
+**Last Updated:** December 11, 2025
+
+---
+
+## Overview
+
+Define and implement a comprehensive credit cost system, plan-based usage limits, and billing logic for all AI operations. This includes setting credit costs per function, establishing plan tiers with limits, and implementing enforcement mechanisms across backend and frontend.
+
+---
+
+## Current Implementation Analysis
+
+### Credit System Architecture
+
+**Location:** `backend/igny8_core/business/billing/`
+
+#### Credit Models
+
+| Model | Purpose | Key Fields |
+|-------|---------|------------|
+| **CreditTransaction** | Tracks all credit additions/deductions | `transaction_type`, `amount`, `balance_after`, `description` |
+| **CreditUsageLog** | Detailed log per AI operation | `operation_type`, `credits_used`, `cost_usd`, `model_used`, `tokens_input`, `tokens_output` |
+| **CreditCostConfig** | Admin-configurable credit costs | `operation_type`, `credits_cost`, `unit`, `display_name` |
+
+**Credit Transaction Types:**
+- `purchase` - Credit purchase
+- `subscription` - Monthly subscription renewal
+- `refund` - Credit refund
+- `deduction` - Usage deduction (AI operations)
+- `adjustment` - Manual admin adjustment
+
+#### Credit Service
+
+**Location:** `backend/igny8_core/business/billing/services/credit_service.py`
+
+**Methods:**
+- `get_credit_cost(operation_type, amount)` - Calculate cost for operation
+- `check_credits(account, operation_type, amount)` - Validate sufficient credits
+- `deduct_credits(account, amount, operation_type, ...)` - Deduct and log
+- `deduct_credits_for_operation(...)` - Convenience method with auto-calculation
+
+**Logic:**
+1. Checks database `CreditCostConfig` first
+2. Falls back to hardcoded `CREDIT_COSTS` constants
+3. Applies unit-based calculation (per 100 words, per image, etc.)
+4. Validates sufficient balance before deduction
+5. Creates both `CreditTransaction` and `CreditUsageLog` records
+
+---
+
+### Current Credit Costs
+
+**Location:** `backend/igny8_core/business/billing/constants.py`
+
+| Operation | Current Cost | Unit | Notes |
+|-----------|--------------|------|-------|
+| `clustering` | 10 credits | per request | Clusters all submitted keywords |
+| `idea_generation` | 15 credits | per request | Ideas for one cluster |
+| `content_generation` | 1 credit | per 100 words | Word-count based |
+| `image_prompt_extraction` | 2 credits | per content | Extract prompts from content |
+| `image_generation` | 5 credits | per image | Generate single image |
+| `linking` | 8 credits | per content | Internal linking (NEW) |
+| `optimization` | 1 credit | per 200 words | Content optimization (NEW) |
+| `site_structure_generation` | 50 credits | per site | Site blueprint (Phase 7) |
+| `site_page_generation` | 20 credits | per page | Page generation (Phase 7) |
+
+**Legacy Aliases:**
+- `ideas` → `idea_generation`
+- `content` → 3 credits fixed (legacy)
+- `images` → `image_generation`
+- `reparse` → 1 credit
+
+**Issues with Current Costs:**
+1. **Not optimized for profitability** - Costs may not reflect actual AI provider costs
+2. **Arbitrary values** - No clear formula based on model costs, processing time, or value
+3. **Inconsistent granularity** - Some per-request, some per-word, some per-item
+4. **No differentiation by quality** - Same cost regardless of model quality (GPT-4 vs GPT-3.5)
+
+---
+
+### Plan Model and Limits
+
+**Location:** `backend/igny8_core/auth/models.py` - `Plan` model
+
+#### Current Plan Structure
+
+| Field | Purpose | Current State |
+|-------|---------|---------------|
+| `name`, `slug` | Plan identification | ✅ Implemented |
+| `price`, `billing_cycle` | Pricing | ✅ Monthly/Annual support |
+| `included_credits` | Monthly credit allocation | ✅ Implemented |
+| `extra_credit_price` | Per-credit overage cost | ✅ Default $0.01 |
+| `allow_credit_topup` | Can buy more credits | ✅ Boolean flag |
+| `auto_credit_topup_threshold` | Auto-buy trigger | ✅ Optional |
+| `auto_credit_topup_amount` | Auto-buy amount | ✅ Optional |
+| `max_users` | Users per account | ✅ Implemented |
+| `max_sites` | Sites per account | ✅ Implemented |
+| `max_industries` | Industries/sectors limit | ✅ Optional |
+| `max_author_profiles` | Writing styles limit | ✅ Default 5 |
+
+**What's MISSING:**
+- ❌ Max keywords limit
+- ❌ Max clusters limit
+- ❌ Max ideas limit
+- ❌ Max content pieces limit
+- ❌ Max images limit
+- ❌ Max tasks in queue limit
+- ❌ Daily/monthly usage caps (beyond credits)
+- ❌ Per-user vs per-account limits distinction
+
+---
+
+### Account Credit Balance
+
+**Location:** `backend/igny8_core/auth/models.py` - `Account` model
+
+**Field:** `credits` (IntegerField with MinValueValidator(0))
+
+**Current Behavior:**
+- Credits deducted on AI operation completion
+- Credit balance checked before operation starts
+- `InsufficientCreditsError` raised if balance < required
+
+**No Implementation For:**
+- Credit expiration dates
+- Credit rollover rules (monthly vs annual)
+- Negative balance prevention (hard stop vs warning)
+- Credit reserve for pending operations
+
+---
+
+## Pricing Plan Requirements
+
+### Recommended Plan Tiers
+
+Based on industry standards and target market:
+
+| Plan | Monthly Price | Annual Price | Included Credits | Target User |
+|------|---------------|--------------|------------------|-------------|
+| **Free** | $0 | $0 | 50 | Trial users, hobbyists |
+| **Starter** | $29 | $299 (15% off) | 500 | Solo creators, small blogs |
+| **Growth** | $99 | $1,019 (15% off) | 2,000 | Growing sites, agencies |
+| **Pro** | $299 | $3,077 (15% off) | 7,500 | Power users, large agencies |
+| **Enterprise** | Custom | Custom | Custom | Enterprise clients |
+
+**Free Plan Considerations:**
+- Should be marked `is_internal=True` to hide from public pricing
+- Limits should be strict enough to encourage upgrade
+- Should not include advanced features (automation, API access)
+
+---
+
+### Usage Limits Per Plan
+
+**Proposed Limits** (to be finalized):
+
+| Limit Type | Free | Starter | Growth | Pro | Enterprise |
+|------------|------|---------|--------|-----|------------|
+| **Monthly Credits** | 50 | 500 | 2,000 | 7,500 | Custom |
+| **Max Users** | 1 | 2 | 5 | 15 | Unlimited |
+| **Max Sites** | 1 | 3 | 10 | 50 | Unlimited |
+| **Max Keywords (saved)** | 100 | 1,000 | 5,000 | 25,000 | Unlimited |
+| **Max Clusters** | 20 | 100 | 500 | 2,500 | Unlimited |
+| **Max Ideas (saved)** | 50 | 500 | 2,500 | 12,500 | Unlimited |
+| **Max Content Pieces** | 25 | 250 | 1,250 | 6,250 | Unlimited |
+| **Max Images** | 25 | 250 | 1,250 | 6,250 | Unlimited |
+| **Max Queue Size** | 5 | 20 | 50 | 200 | Unlimited |
+| **Automation Enabled** | ❌ | ❌ | ✅ | ✅ | ✅ |
+| **API Access** | ❌ | ❌ | ✅ | ✅ | ✅ |
+| **Priority Support** | ❌ | ❌ | ❌ | ✅ | ✅ |
+
+**Notes:**
+- "Unlimited" means no hard limit, but still subject to fair use policy
+- Limits apply per account (across all sites in account)
+- Deleted items don't count toward limits (soft-delete system)
+
+---
+
+### Credit Cost Optimization Strategy
+
+**Goal:** Define credit costs that:
+1. Cover AI provider costs + margin
+2. Are competitive with market rates
+3. Encourage usage without abuse
+4. Scale predictably with usage
+
+#### Recommended Credit Cost Revisions
+
+**Analysis Required:**
+- [ ] Calculate actual AI provider costs per operation (OpenAI, Runware, etc.)
+- [ ] Add 30-50% margin for infrastructure, support, and profit
+- [ ] Compare with competitor pricing (Jasper, Copy.ai, Writesonic)
+- [ ] Test with sample use cases to ensure plan value
+
+**Proposed Adjustments** (pending analysis):
+
+| Operation | Current | Proposed | Reasoning |
+|-----------|---------|----------|-----------|
+| Clustering | 10 | **8** | Lower barrier for discovery phase |
+| Idea Generation | 15 | **12** | Encourage ideation before writing |
+| Content (100 words) | 1 | **1.5** | Reflect actual GPT-4 costs |
+| Image Prompts | 2 | **3** | More complex extraction logic |
+| Image Generation | 5 | **6** | Runware/DALL-E costs increasing |
+| Optimization | 1 per 200 words | **0.5 per 100 words** | Encourage optimization usage |
+
+**Variable Costs by Model Quality:**
+- **Option:** Charge more for GPT-4 vs GPT-3.5, DALL-E 3 vs DALL-E 2
+- **Implementation:** Add `model_tier` multiplier in `get_credit_cost()`
+
+---
+
+## Required Implementation
+
+### A. Expand Plan Model with Usage Limits
+
+**File:** `backend/igny8_core/auth/models.py` - `Plan` model
+
+**Add Fields:**
+
+```python
+# Content Creation Limits (NULL = unlimited)
+max_keywords = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum keywords saved per account"
+)
+max_clusters = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum clusters per account"
+)
+max_ideas = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum content ideas saved per account"
+)
+max_content = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum content pieces per account"
+)
+max_images = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum images per account"
+)
+
+# Queue and Rate Limits
+max_queue_size = models.IntegerField(
+    default=10,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum concurrent items in queue"
+)
+max_daily_ai_requests = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum AI requests per day (prevents abuse)"
+)
+max_monthly_content_generated = models.IntegerField(
+    null=True, blank=True,
+    validators=[MinValueValidator(1)],
+    help_text="Maximum content pieces generated per month"
+)
+
+# Feature Access Flags
+allow_automation = models.BooleanField(
+    default=False,
+    help_text="Enable automation wizard"
+)
+allow_api_access = models.BooleanField(
+    default=False,
+    help_text="Enable API access"
+)
+allow_bulk_operations = models.BooleanField(
+    default=True,
+    help_text="Enable bulk actions (delete, export, etc.)"
+)
+```
+
+**Migration:** Create Django migration to add these fields with default NULL values
+
+---
+
+### B. Create Limit Enforcement Service
+
+**File:** `backend/igny8_core/business/billing/services/limit_service.py` (NEW)
+
+**Service Class:** `LimitService`
+
+**Methods to Implement:**
+
+| Method | Purpose | Returns |
+|--------|---------|---------|
+| `check_keyword_limit(account)` | Check if can add more keywords | `bool` or raises `LimitExceededError` |
+| `check_cluster_limit(account)` | Check if can add more clusters | `bool` or raises `LimitExceededError` |
+| `check_idea_limit(account)` | Check if can add more ideas | `bool` or raises `LimitExceededError` |
+| `check_content_limit(account)` | Check if can add more content | `bool` or raises `LimitExceededError` |
+| `check_image_limit(account)` | Check if can add more images | `bool` or raises `LimitExceededError` |
+| `check_queue_limit(account)` | Check queue capacity | `bool` or raises `LimitExceededError` |
+| `check_daily_request_limit(account)` | Check daily AI request quota | `bool` or raises `LimitExceededError` |
+| `get_usage_stats(account)` | Get current usage counts | `dict` with all counters |
+| `get_limit_stats(account)` | Get limits and remaining capacity | `dict` with limits |
+
+**Implementation Logic:**
+
+```python
+def check_keyword_limit(account):
+    plan = account.plan
+    if plan.max_keywords is None:
+        return True  # Unlimited
+    
+    current_count = Keywords.objects.filter(
+        account=account,
+        deleted_at__isnull=True  # Exclude soft-deleted
+    ).count()
+    
+    if current_count >= plan.max_keywords:
+        raise LimitExceededError(
+            f"Keyword limit reached ({plan.max_keywords}). Upgrade your plan."
+        )
+    
+    return True
+```
+
+**Exception:** `LimitExceededError` (inherit from `BillingException`)
+
+---
+
+### C. Integrate Limit Checks in API Views
+
+**Files to Update:**
+
+- `backend/igny8_core/modules/planner/views.py` - KeywordsViewSet
+- `backend/igny8_core/modules/planner/views.py` - ClustersViewSet
+- `backend/igny8_core/modules/planner/views.py` - ContentIdeasViewSet
+- `backend/igny8_core/modules/writer/views.py` - TasksViewSet
+- `backend/igny8_core/modules/writer/views.py` - ContentViewSet
+
+**Integration Points:**
+
+| ViewSet | Action | Check to Add |
+|---------|--------|--------------|
+| KeywordsViewSet | `create()` | `LimitService.check_keyword_limit(account)` |
+| KeywordsViewSet | `bulk_create()` | Check limit with proposed count |
+| ClustersViewSet | `create()` | `LimitService.check_cluster_limit(account)` |
+| ContentIdeasViewSet | `create()` | `LimitService.check_idea_limit(account)` |
+| TasksViewSet | `create()` | `LimitService.check_content_limit(account)` + `check_queue_limit()` |
+| ContentViewSet | `create()` | `LimitService.check_content_limit(account)` |
+
+**Example Integration:**
+
+```python
+def create(self, request, *args, **kwargs):
+    account = request.user.account
+    
+    # Check limit before creating
+    try:
+        LimitService.check_keyword_limit(account)
+    except LimitExceededError as e:
+        return Response({
+            'error': str(e),
+            'error_code': 'LIMIT_EXCEEDED',
+            'upgrade_url': '/pricing'
+        }, status=403)
+    
+    # Proceed with creation
+    return super().create(request, *args, **kwargs)
+```
+
+---
+
+### D. Add Usage Tracking and Counter Cache
+
+**Optimization:** Instead of counting records on every request, cache counts
+
+**Implementation Options:**
+
+#### Option 1: Add Counter Fields to Account Model
+
+```python
+# Add to Account model
+keyword_count = models.IntegerField(default=0)
+cluster_count = models.IntegerField(default=0)
+idea_count = models.IntegerField(default=0)
+content_count = models.IntegerField(default=0)
+image_count = models.IntegerField(default=0)
+```
+
+**Update counters in signals:**
+- `post_save` signal: increment counter
+- `post_delete` signal: decrement counter
+- Periodic reconciliation task to fix drift
+
+#### Option 2: Cache Usage Stats (Recommended)
+
+Use Django cache with 5-minute TTL:
+
+```python
+def get_cached_usage_stats(account):
+    cache_key = f'usage_stats_{account.id}'
+    stats = cache.get(cache_key)
+    
+    if stats is None:
+        stats = {
+            'keywords': Keywords.objects.filter(account=account, deleted_at__isnull=True).count(),
+            'clusters': Clusters.objects.filter(account=account, deleted_at__isnull=True).count(),
+            # ... etc
+        }
+        cache.set(cache_key, stats, 300)  # 5 minutes
+    
+    return stats
+```
+
+**Invalidate cache on:**
+- Create operations
+- Delete operations
+- Soft-delete operations
+
+---
+
+### E. Frontend Limit Display
+
+#### 1. Usage Dashboard Widget
+
+**Location:** `frontend/src/components/dashboard/UsageLimitsWidget.tsx` (NEW)
+
+**Display:**
+- Current usage vs limit for each resource
+- Progress bars with color coding:
+  - Green: < 70% used
+  - Yellow: 70-90% used
+  - Red: > 90% used
+- "Upgrade Plan" button when approaching limits
+
+**Example UI:**
+
+```
+Usage & Limits
+━━━━━━━━━━━━━━━━━━━━━━━━━━━
+Keywords: 750 / 1,000 ████████░░ 75%
+Clusters: 45 / 100    ████░░░░░░ 45%
+Ideas: 380 / 500      ███████░░░ 76%
+Content: 120 / 250    ████░░░░░░ 48%
+
+[Upgrade Plan]
+```
+
+#### 2. Inline Warnings
+
+**Show warnings when approaching limits:**
+
+- At 80%: Yellow badge "Approaching limit"
+- At 90%: Orange warning "Near limit - Upgrade recommended"
+- At 100%: Red error "Limit reached - Upgrade required"
+
+**Display in:**
+- Header metrics
+- Page headers
+- Before bulk operations
+- In forms (disable submit if limit reached)
+
+#### 3. Create/Import Dialogs
+
+**Add limit check before showing form:**
+
+```typescript
+const handleCreateKeyword = () => {
+  const stats = usageStats; // from API
+  const limit = account.plan.max_keywords;
+  
+  if (limit && stats.keywords >= limit) {
+    toast.error('Keyword limit reached. Upgrade your plan.');
+    navigate('/settings/billing');
+    return;
+  }
+  
+  setShowCreateModal(true);
+};
+```
+
+#### 4. Upgrade Prompts
+
+**When limit error occurs:**
+- Show modal with:
+  - Current plan
+  - Current limit
+  - Recommended plan
+  - Benefits of upgrading
+  - "Upgrade Now" CTA
+
+---
+
+### F. Credit Cost Configuration UI (Admin)
+
+**Location:** Django Admin or custom Admin Panel page
+
+**Feature:** Allow superusers to edit credit costs without code changes
+
+**Admin Interface:**
+- List all operations with current costs
+- Edit cost, unit, and display name
+- Track change history (previous_cost field)
+- Enable/disable operations
+- Preview impact on sample use cases
+
+**Models Used:**
+- `CreditCostConfig` - Admin-editable costs
+- Falls back to `CREDIT_COSTS` constants if not configured
+
+---
+
+## Testing Requirements
+
+### Limit Enforcement Tests
+
+| Test Case | Expected Result |
+|-----------|-----------------|
+| Create keyword at limit | Error: "Keyword limit reached" |
+| Create keyword below limit | Success |
+| Create 10 keywords via bulk import at limit | Error with count blocked |
+| Delete keyword then create | Success (count decremented) |
+| Soft-delete keyword then restore | Counts update correctly |
+| Upgrade plan mid-session | New limits apply immediately |
+
+### Credit Deduction Tests
+
+| Test Case | Expected Result |
+|-----------|-----------------|
+| Generate content with sufficient credits | Content created, credits deducted |
+| Generate content with insufficient credits | Error: "Insufficient credits" |
+| Generate content at exact credit balance | Success, balance = 0 |
+| Generate multiple items in queue | Each deducts credits sequentially |
+| Credit deduction failure mid-operation | Transaction rolled back, no partial deduction |
+
+### Plan Limit Tests
+
+| Plan | Test Case | Expected Result |
+|------|-----------|-----------------|
+| Free | Create 101st keyword (limit: 100) | Blocked |
+| Starter | Create 6 queue items (limit: 5) | Blocked |
+| Growth | Enable automation | Success (has access) |
+| Pro | Create unlimited keywords | Success (no limit) |
+| Enterprise | All operations | No limits enforced |
+
+---
+
+## Pricing Page Updates
+
+**Location:** `frontend/src/pages/marketing/Pricing.tsx`
+
+### Required Elements
+
+1. **Plan Comparison Table**
+   - All tiers side-by-side
+   - Feature checkmarks
+   - Highlight "Most Popular" plan
+   - Monthly/Annual toggle with savings badge
+
+2. **Usage Limits Display**
+   - Show key limits per plan
+   - Use "Unlimited" label for null limits
+   - Tooltip explanations for complex limits
+
+3. **Credit System Explanation**
+   - What credits are
+   - How they're consumed
+   - How to buy more
+   - Credit rollover rules
+
+4. **FAQ Section**
+   - "What happens when I run out of credits?"
+   - "Can I change plans mid-month?"
+   - "Do unused credits roll over?"
+   - "What's included in Enterprise?"
+
+5. **Calculator Widget** (Optional)
+   - Estimate monthly usage
+   - Recommend plan based on needs
+   - Show credit consumption breakdown
+
+---
+
+## Success Metrics
+
+- ✅ All AI operations enforce credit checks
+- ✅ All create operations enforce limit checks
+- ✅ Credit costs reflect actual provider costs + margin
+- ✅ Plans are competitively priced
+- ✅ Usage dashboard shows accurate counts
+- ✅ Limit warnings prevent user frustration
+- ✅ Upgrade flow is clear and frictionless
+- ✅ Admin can adjust costs without code changes
+- ✅ All tests pass
+
+---
+
+## Related Files Reference
+
+### Backend
+- `backend/igny8_core/auth/models.py` - Account, Plan models
+- `backend/igny8_core/business/billing/models.py` - Credit models
+- `backend/igny8_core/business/billing/constants.py` - Credit costs
+- `backend/igny8_core/business/billing/services/credit_service.py` - Credit logic
+- `backend/igny8_core/business/billing/services/limit_service.py` - **NEW** Limit enforcement
+- `backend/igny8_core/modules/planner/views.py` - Planner API views
+- `backend/igny8_core/modules/writer/views.py` - Writer API views
+
+### Frontend
+- `frontend/src/components/dashboard/UsageLimitsWidget.tsx` - **NEW** Usage display
+- `frontend/src/pages/marketing/Pricing.tsx` - Pricing page
+- `frontend/src/pages/Planner/*.tsx` - Planner pages (add limit checks)
+- `frontend/src/pages/Writer/*.tsx` - Writer pages (add limit checks)
+- `frontend/src/services/api.ts` - API service (handle limit errors)
+
+---
+
+## Notes
+
+- Limits should be enforced at API level, not just UI level
+- Consider "soft limits" with warnings vs "hard limits" with blocks
+- Credit expiration and rollover rules need business decision
+- Enterprise pricing needs custom quote system
+- Monitor actual usage patterns to optimize costs and limits
+- A/B test different pricing tiers to maximize conversion