diff --git a/CREDITS-TOKENS-GUIDE.md b/CREDITS-TOKENS-GUIDE.md new file mode 100644 index 00000000..e9950250 --- /dev/null +++ b/CREDITS-TOKENS-GUIDE.md @@ -0,0 +1,455 @@ +# IGNY8 Credits & Tokens System Guide + +**Version:** 1.0 +**Last Updated:** December 19, 2025 +**System Status:** ✅ Fully Operational + +--- + +## Overview + +IGNY8 uses a **token-based credit system** where all AI operations consume credits calculated from actual AI token usage. This guide covers configuration, data flow, and monitoring. + +--- + +## System Architecture + +### Data Flow + +``` +User Action (Content Generation, Ideas, etc.) + ↓ +Backend Service Initiated + ↓ +AI API Called (OpenAI, etc.) + ↓ +Response Received: {input_tokens, output_tokens, cost_usd, model} + ↓ +Credits Calculated: (total_tokens / tokens_per_credit) + ↓ +Credits Deducted from Account + ↓ +CreditUsageLog Created: {tokens_input, tokens_output, cost_usd, credits_used, model_used} + ↓ +Reports Updated with Real-Time Analytics +``` + +### Key Components + +1. **BillingConfiguration** - System-wide billing settings +2. **CreditCostConfig** - Per-operation token-to-credit ratios +3. **CreditUsageLog** - Transaction log with token data +4. **AITaskLog** - Detailed AI execution history + +--- + +## Configuration + +### 1. Global Billing Settings + +**Location:** Django Admin → Billing → Billing Configuration + +**Key Settings:** +- **Default Tokens Per Credit:** 100 (base ratio) +- **Default Credit Price:** $0.01 USD +- **Rounding Mode:** Up (conservative billing) +- **Token Reporting Enabled:** Yes + +**When to Adjust:** +- Change credit pricing across all operations +- Modify rounding behavior for credit calculations +- Enable/disable token tracking + +### 2. Per-Operation Configuration + +**Location:** Django Admin → Billing → Credit Cost Configs + +**Current Ratios:** + +| Operation | Tokens/Credit | Min Credits | Price/Credit | +|-----------|---------------|-------------|--------------| +| Clustering | 150 | 2 | $0.0100 | +| Content Generation | 100 | 3 | $0.0100 | +| Idea Generation | 200 | 1 | $0.0100 | +| Image Generation | 50 | 5 | $0.0200 | +| Image Prompt Extraction | 100 | 1 | $0.0100 | +| Linking | 300 | 1 | $0.0050 | +| Optimization | 200 | 1 | $0.0050 | + +**Adjusting Pricing:** +- **Increase `tokens_per_credit`** → Lower cost (more tokens per credit) +- **Decrease `tokens_per_credit`** → Higher cost (fewer tokens per credit) +- **Adjust `min_credits`** → Enforce minimum charge per operation +- **Change `price_per_credit_usd`** → Override default credit price + +**Example:** To make Content Generation cheaper: +``` +Current: 100 tokens/credit (1000 tokens = 10 credits) +Change to: 150 tokens/credit (1000 tokens = 6.67 → 7 credits) +Result: 30% cost reduction +``` + +### 3. Token Extraction + +**Location:** `backend/igny8_core/ai/engine.py` (line 380-381) + +**Current Implementation:** +```python +tokens_input = raw_response.get('input_tokens', 0) +tokens_output = raw_response.get('output_tokens', 0) +``` + +**Critical:** Field names must match AI provider response format +- ✅ Correct: `input_tokens`, `output_tokens` +- ❌ Wrong: `tokens_input`, `tokens_output` + +**Supported Providers:** +- OpenAI (GPT-4, GPT-4o, GPT-5.1) +- Anthropic Claude +- Runware (image generation) + +--- + +## Monitoring & Reports + +### 1. AI Cost & Margin Analysis + +**URL:** `https://api.igny8.com/admin/reports/ai-cost-analysis/` + +**Metrics Displayed:** +- **Total Cost** - Actual USD spent on AI APIs +- **Revenue** - Income from credits charged +- **Margin** - Profit (Revenue - Cost) with percentage +- **Margin / 1M Tokens** - Profit efficiency per million tokens +- **Margin / 1K Credits** - Profit per thousand credits charged +- **Projected Monthly** - Forecasted costs based on trends + +**Tables:** +- Model Cost Comparison - Profitability by AI model +- Top Spenders - Highest cost accounts +- Cost by Function - Profitability by operation type +- Cost Anomalies - Expensive outlier calls + +**Use Cases:** +- Identify unprofitable operations or accounts +- Optimize token-to-credit ratios +- Detect unusual AI spending patterns +- Track margins over time + +### 2. Token Usage Report + +**URL:** `https://api.igny8.com/admin/reports/token-usage/` + +**Metrics Displayed:** +- Total tokens consumed (input + output) +- Average tokens per call +- Cost per 1K tokens +- Token distribution by model +- Token distribution by operation +- Daily token trends + +**Use Cases:** +- Understand token consumption patterns +- Identify token-heavy operations +- Optimize prompts to reduce token usage +- Track token efficiency over time + +### 3. Usage Report + +**URL:** `https://api.igny8.com/admin/reports/usage/` + +**Metrics Displayed:** +- Total credits used system-wide +- Credits by operation type +- Top credit consumers +- Model usage distribution + +**Use Cases:** +- Monitor overall system usage +- Identify high-volume users +- Track popular AI operations +- Plan capacity and scaling + +### 4. Data Quality Report + +**URL:** `https://api.igny8.com/admin/reports/data-quality/` + +**Purpose:** Identify data integrity issues +- Orphaned content +- Duplicate keywords +- Missing SEO metadata + +--- + +## Data Models + +### CreditUsageLog (Primary Transaction Log) + +**Purpose:** Record every credit deduction with full context + +**Key Fields:** +- `account` - User account charged +- `operation_type` - Function executed (e.g., "content_generation") +- `credits_used` - Credits deducted +- `cost_usd` - Actual AI provider cost +- `tokens_input` - Input tokens consumed +- `tokens_output` - Output tokens generated +- `model_used` - AI model (e.g., "gpt-4o") +- `related_object_type/id` - Link to content/site/keyword +- `metadata` - Additional context (prompt, settings, etc.) + +**Query Examples:** +```python +# Total tokens for an account +CreditUsageLog.objects.filter(account=account).aggregate( + total_tokens=Sum('tokens_input') + Sum('tokens_output') +) + +# Average cost by operation +CreditUsageLog.objects.values('operation_type').annotate( + avg_cost=Avg('cost_usd'), + total_calls=Count('id') +) + +# Margin analysis +logs = CreditUsageLog.objects.all() +revenue = logs.aggregate(Sum('credits_used'))['credits_used__sum'] * 0.01 +cost = logs.aggregate(Sum('cost_usd'))['cost_usd__sum'] +margin = revenue - cost +``` + +### AITaskLog (Execution History) + +**Purpose:** Detailed AI execution tracking + +**Key Fields:** +- `function_name` - AI function executed +- `account` - User account +- `cost` - AI provider cost +- `tokens` - Total tokens (input + output) +- `phase` - Execution stage +- `status` - Success/failure +- `execution_time` - Processing duration +- `raw_request/response` - Full API data + +**Use Cases:** +- Debug AI execution failures +- Analyze prompt effectiveness +- Track model performance +- Audit AI interactions + +--- + +## Historical Data Backfill + +### Issue +Prior to December 2025, token fields were not populated due to incorrect field name mapping. + +### Solution +A backfill script matched AITaskLog entries to CreditUsageLog records using: +- Account matching +- Timestamp matching (±10 second window) +- 40/60 input/output split estimation (when only total available) + +### Result +- ✅ 777,456 tokens backfilled +- ✅ 380/479 records updated (79% coverage) +- ✅ Historical margin analysis now available +- ⚠️ 99 records remain at 0 tokens (no matching AITaskLog) + +### Script Location +`backend/igny8_core/management/commands/backfill_tokens.py` + +--- + +## Troubleshooting + +### Empty Margin Metrics +**Symptom:** "Margin / 1M Tokens" shows "No token data yet" + +**Causes:** +1. No recent AI calls with token data +2. Token extraction not working (field name mismatch) +3. Historical data has 0 tokens + +**Resolution:** +1. Check AIEngine token extraction: `tokens_input`, `tokens_output` fields +2. Verify AI responses contain `input_tokens`, `output_tokens` +3. Run test AI operation and check CreditUsageLog +4. Consider backfill for historical data + +### Zero Tokens in CreditUsageLog +**Symptom:** `tokens_input` and `tokens_output` are 0 + +**Causes:** +1. Field name mismatch in AIEngine +2. AI provider not returning token data +3. Historical records before fix + +**Resolution:** +1. Verify `engine.py` line 380-381 uses correct field names +2. Check AI provider API response format +3. Restart backend services after fixes +4. Future calls will populate correctly + +### Incorrect Margins +**Symptom:** Margin percentages seem wrong + +**Causes:** +1. Incorrect token-to-credit ratios +2. Credit price misconfigured +3. Decimal division errors + +**Resolution:** +1. Review CreditCostConfig ratios +2. Check BillingConfiguration credit price +3. Verify margin calculations use `float()` conversions +4. Check for TypeError in logs + +### Operations Not Charging Correctly +**Symptom:** Wrong number of credits deducted + +**Causes:** +1. Token-to-credit ratio misconfigured +2. Minimum credits not enforced +3. Rounding mode incorrect + +**Resolution:** +1. Check operation's CreditCostConfig +2. Verify `min_credits` setting +3. Review `rounding_mode` in BillingConfiguration +4. Test with known token count + +--- + +## Best Practices + +### Pricing Strategy +1. **Monitor margins weekly** - Use AI Cost Analysis report +2. **Adjust ratios based on costs** - If margins drop below 70%, decrease tokens_per_credit +3. **Set reasonable minimums** - Enforce min_credits for small operations +4. **Track model costs** - Some models (GPT-4) cost more than others + +### Token Optimization +1. **Optimize prompts** - Reduce unnecessary tokens +2. **Use appropriate models** - GPT-4o-mini for simple tasks +3. **Cache results** - Avoid duplicate AI calls +4. **Monitor anomalies** - Investigate unusually expensive calls + +### Data Integrity +1. **Regular audits** - Check token data completeness +2. **Verify field mappings** - Ensure AI responses parsed correctly +3. **Monitor logs** - Watch for errors in CreditService +4. **Backup configurations** - Export CreditCostConfig settings + +### Performance +1. **Archive old logs** - Move historical CreditUsageLog to archive tables +2. **Index frequently queried fields** - account, operation_type, created_at +3. **Aggregate reports** - Use materialized views for large datasets +4. **Cache report data** - Reduce database load + +--- + +## API Integration + +### Frontend Credit Display + +**Endpoint:** `/v1/billing/credits/balance/` + +**Response:** +```json +{ + "credits": 1234, + "plan_credits_per_month": 5000, + "credits_used_this_month": 876, + "credits_remaining": 4124 +} +``` + +**Pages Using This:** +- `/account/plans` - Plans & Billing +- `/account/usage` - Usage Analytics +- Dashboard credit widget + +### Credit Transaction History + +**Endpoint:** `/v1/billing/credits/usage/` + +**Response:** +```json +{ + "results": [ + { + "id": 123, + "operation_type": "Content Generation", + "credits_used": 15, + "tokens_input": 500, + "tokens_output": 1000, + "cost_usd": 0.015, + "model_used": "gpt-4o", + "created_at": "2025-12-19T10:30:00Z" + } + ] +} +``` + +--- + +## Quick Reference + +### Common Operations + +**Check account credits:** +```python +account.credits # Current balance +``` + +**Manual credit adjustment:** +```python +CreditService.add_credits(account, amount=100, description="Bonus credits") +``` + +**Get operation config:** +```python +config = CreditService.get_or_create_config('content_generation') +# Returns: CreditCostConfig with tokens_per_credit, min_credits +``` + +**Calculate credits needed:** +```python +credits = CreditService.calculate_credits_from_tokens( + operation_type='content_generation', + tokens_input=500, + tokens_output=1500 +) +# Returns: 20 (if 100 tokens/credit) +``` + +### Important File Locations + +- **Credit Service:** `backend/igny8_core/business/billing/services/credit_service.py` +- **AI Engine:** `backend/igny8_core/ai/engine.py` +- **Reports:** `backend/igny8_core/admin/reports.py` +- **Models:** `backend/igny8_core/modules/billing/models.py` +- **Admin:** `backend/igny8_core/modules/billing/admin.py` + +### Report Access + +All reports require staff/superuser login: +- AI Cost Analysis: `/admin/reports/ai-cost-analysis/` +- Token Usage: `/admin/reports/token-usage/` +- Usage Report: `/admin/reports/usage/` +- Data Quality: `/admin/reports/data-quality/` + +--- + +## Support & Updates + +For questions or issues with the credits & tokens system: +1. Check Django admin logs: `/admin/` +2. Review CreditUsageLog for transaction details +3. Monitor AITaskLog for execution errors +4. Check backend logs: `docker logs igny8_backend` + +**System Maintainer:** IGNY8 DevOps Team +**Last Major Update:** December 2025 (Token-based credit system implementation) diff --git a/backend/igny8_core/admin/reports.py b/backend/igny8_core/admin/reports.py index 4a0cc35b..eb3f57b4 100644 --- a/backend/igny8_core/admin/reports.py +++ b/backend/igny8_core/admin/reports.py @@ -82,6 +82,10 @@ def usage_report(request): operation_count=Count('id') ).order_by('-total_credits') + # Format operation types as Title Case + for usage in usage_by_operation: + usage['operation_type'] = usage['operation_type'].replace('_', ' ').title() if usage['operation_type'] else 'Unknown' + # Top credit consumers top_consumers = CreditUsageLog.objects.values( 'account__name' @@ -269,11 +273,9 @@ def token_usage_report(request): start_date = timezone.now() - timedelta(days=days) - # Base queryset - filter for records with token data + # Base queryset - include all records (tokens may be 0 for historical data) logs = CreditUsageLog.objects.filter( - created_at__gte=start_date, - tokens_input__isnull=False, - tokens_output__isnull=False + created_at__gte=start_date ) # Total statistics @@ -310,7 +312,8 @@ def token_usage_report(request): for func in token_by_function: func['total_tokens'] = (func['total_tokens_input'] or 0) + (func['total_tokens_output'] or 0) func['avg_tokens'] = func['total_tokens'] / func['call_count'] if func['call_count'] > 0 else 0 - func['function'] = func['operation_type'] # Add alias for template + # Format operation_type as Title Case + func['function'] = func['operation_type'].replace('_', ' ').title() if func['operation_type'] else 'Unknown' token_by_function = sorted(token_by_function, key=lambda x: x['total_tokens'], reverse=True) # Token usage by account (top consumers) @@ -366,7 +369,7 @@ def token_usage_report(request): # Cost efficiency total_cost = logs.aggregate(total=Sum('cost_usd'))['total'] or Decimal('0.00') - cost_per_1k_tokens = (total_cost / (total_tokens / 1000)) if total_tokens > 0 else Decimal('0.00') + cost_per_1k_tokens = float(total_cost) / (total_tokens / 1000) if total_tokens > 0 else 0.0 context = { 'title': 'Token Usage Report', @@ -427,6 +430,20 @@ def ai_cost_analysis(request): total_tokens_output = logs.aggregate(total=Sum('tokens_output'))['total'] or 0 total_tokens = total_tokens_input + total_tokens_output + # Revenue & Margin calculation + from igny8_core.business.billing.models import BillingConfiguration + billing_config = BillingConfiguration.get_config() + total_credits_charged = logs.aggregate(total=Sum('credits_used'))['total'] or 0 + total_revenue = Decimal(total_credits_charged) * billing_config.default_credit_price_usd + total_margin = total_revenue - total_cost + margin_percentage = float((total_margin / total_revenue * 100) if total_revenue > 0 else 0) + + # Per-unit margins + # Calculate per 1M tokens (margin per million tokens) + margin_per_1m_tokens = float(total_margin) / (total_tokens / 1_000_000) if total_tokens > 0 else 0 + # Calculate per 1K credits (margin per thousand credits) + margin_per_1k_credits = float(total_margin) / (total_credits_charged / 1000) if total_credits_charged > 0 else 0 + # Cost by model with efficiency metrics cost_by_model = logs.values('model_used').annotate( total_cost=Sum('cost_usd'), @@ -436,7 +453,7 @@ def ai_cost_analysis(request): total_tokens_output=Sum('tokens_output') ).order_by('-total_cost') - # Add cost efficiency (cost per 1K tokens) for each model + # Add cost efficiency and margin for each model for model in cost_by_model: model['total_tokens'] = (model['total_tokens_input'] or 0) + (model['total_tokens_output'] or 0) model['avg_tokens'] = model['total_tokens'] / model['call_count'] if model['call_count'] > 0 else 0 @@ -445,6 +462,14 @@ def ai_cost_analysis(request): model['cost_per_1k_tokens'] = float(model['total_cost']) / (model['total_tokens'] / 1000) else: model['cost_per_1k_tokens'] = 0 + + # Calculate margin for this model + model_credits = logs.filter(model_used=model['model_used']).aggregate(total=Sum('credits_used'))['total'] or 0 + model_revenue = Decimal(model_credits) * billing_config.default_credit_price_usd + model_margin = model_revenue - model['total_cost'] + model['revenue'] = float(model_revenue) + model['margin'] = float(model_margin) + model['margin_percentage'] = float((model_margin / model_revenue * 100) if model_revenue > 0 else 0) # Cost by account (top spenders) cost_by_account = logs.values('account__name', 'account_id').annotate( @@ -468,10 +493,19 @@ def ai_cost_analysis(request): total_tokens_output=Sum('tokens_output') ).order_by('-total_cost')[:10] - # Add total_tokens and function alias + # Add total_tokens, function alias, and margin for func in cost_by_function: func['total_tokens'] = (func['total_tokens_input'] or 0) + (func['total_tokens_output'] or 0) - func['function'] = func['operation_type'] # Add alias for template + # Format operation_type as Title Case + func['function'] = func['operation_type'].replace('_', ' ').title() if func['operation_type'] else 'Unknown' + + # Calculate margin for this operation + func_credits = logs.filter(operation_type=func['operation_type']).aggregate(total=Sum('credits_used'))['total'] or 0 + func_revenue = Decimal(func_credits) * billing_config.default_credit_price_usd + func_margin = func_revenue - func['total_cost'] + func['revenue'] = float(func_revenue) + func['margin'] = float(func_margin) + func['margin_percentage'] = float((func_margin / func_revenue * 100) if func_revenue > 0 else 0) # Daily cost trends (time series) daily_cost_data = [] @@ -507,7 +541,8 @@ def ai_cost_analysis(request): # Add aliases and calculate total tokens for each anomaly for anomaly in anomalies: anomaly['model'] = anomaly['model_used'] - anomaly['function'] = anomaly['operation_type'] + # Format operation_type as Title Case + anomaly['function'] = anomaly['operation_type'].replace('_', ' ').title() if anomaly['operation_type'] else 'Unknown' anomaly['cost'] = anomaly['cost_usd'] anomaly['tokens'] = (anomaly['tokens_input'] or 0) + (anomaly['tokens_output'] or 0) else: @@ -544,9 +579,16 @@ def ai_cost_analysis(request): efficiency_score = 100.0 context = { - 'title': 'AI Cost Analysis', + 'title': 'AI Cost & Margin Analysis', 'days_filter': days, 'total_cost': float(total_cost), + 'total_revenue': float(total_revenue), + 'total_margin': float(total_margin), + 'margin_percentage': round(margin_percentage, 2), + 'margin_per_1m_tokens': round(margin_per_1m_tokens, 4), + 'margin_per_1k_credits': round(margin_per_1k_credits, 4), + 'total_credits_charged': total_credits_charged, + 'credit_price': float(billing_config.default_credit_price_usd), 'total_calls': total_calls, 'avg_cost_per_call': float(avg_cost_per_call), 'total_tokens': int(total_tokens), diff --git a/backend/igny8_core/ai/engine.py b/backend/igny8_core/ai/engine.py index c1a179fd..e8319a22 100644 --- a/backend/igny8_core/ai/engine.py +++ b/backend/igny8_core/ai/engine.py @@ -376,18 +376,18 @@ class AIEngine: # Map function name to operation type operation_type = self._get_operation_type(function_name) - # Calculate actual amount based on results - actual_amount = self._get_actual_amount(function_name, save_result, parsed, data) + # Get actual token usage from response (AI returns 'input_tokens' and 'output_tokens') + tokens_input = raw_response.get('input_tokens', 0) + tokens_output = raw_response.get('output_tokens', 0) - # Deduct credits using the new convenience method + # Deduct credits based on actual token usage CreditService.deduct_credits_for_operation( account=self.account, operation_type=operation_type, - amount=actual_amount, + tokens_input=tokens_input, + tokens_output=tokens_output, cost_usd=raw_response.get('cost'), model_used=raw_response.get('model', ''), - tokens_input=raw_response.get('tokens_input', 0), - tokens_output=raw_response.get('tokens_output', 0), related_object_type=self._get_related_object_type(function_name), related_object_id=save_result.get('id') or save_result.get('cluster_id') or save_result.get('task_id'), metadata={ @@ -399,7 +399,10 @@ class AIEngine: } ) - logger.info(f"[AIEngine] Credits deducted: {operation_type}, amount: {actual_amount}") + logger.info( + f"[AIEngine] Credits deducted: {operation_type}, " + f"tokens: {tokens_input + tokens_output} ({tokens_input} in, {tokens_output} out)" + ) except InsufficientCreditsError as e: # This shouldn't happen since we checked before, but log it logger.error(f"[AIEngine] Insufficient credits during deduction: {e}") diff --git a/backend/igny8_core/business/billing/models.py b/backend/igny8_core/business/billing/models.py index 29df0fee..624757b4 100644 --- a/backend/igny8_core/business/billing/models.py +++ b/backend/igny8_core/business/billing/models.py @@ -109,8 +109,8 @@ class CreditUsageLog(AccountBaseModel): class CreditCostConfig(models.Model): """ - Configurable credit costs per AI function - Admin-editable alternative to hardcoded constants + Token-based credit pricing configuration. + ALL operations use token-to-credit conversion. """ # Operation identification operation_type = models.CharField( @@ -120,26 +120,27 @@ class CreditCostConfig(models.Model): help_text="AI operation type" ) - # Cost configuration - credits_cost = models.IntegerField( - validators=[MinValueValidator(0)], - help_text="Credits required for this operation" + # Token-to-credit ratio (tokens per 1 credit) + tokens_per_credit = models.IntegerField( + default=100, + validators=[MinValueValidator(1)], + help_text="Number of tokens that equal 1 credit (e.g., 100 tokens = 1 credit)" ) - # Unit of measurement - UNIT_CHOICES = [ - ('per_request', 'Per Request'), - ('per_100_words', 'Per 100 Words'), - ('per_200_words', 'Per 200 Words'), - ('per_item', 'Per Item'), - ('per_image', 'Per Image'), - ] + # Minimum credits (for very small token usage) + min_credits = models.IntegerField( + default=1, + validators=[MinValueValidator(0)], + help_text="Minimum credits to charge regardless of token usage" + ) - unit = models.CharField( - max_length=50, - default='per_request', - choices=UNIT_CHOICES, - help_text="What the cost applies to" + # Price per credit (for revenue reporting) + price_per_credit_usd = models.DecimalField( + max_digits=10, + decimal_places=4, + default=Decimal('0.01'), + validators=[MinValueValidator(Decimal('0.0001'))], + help_text="USD price per credit (for revenue reporting)" ) # Metadata @@ -149,6 +150,7 @@ class CreditCostConfig(models.Model): # Status is_active = models.BooleanField(default=True, help_text="Enable/disable this operation") + # Audit fields created_at = models.DateTimeField(auto_now_add=True) updated_at = models.DateTimeField(auto_now=True) @@ -162,10 +164,10 @@ class CreditCostConfig(models.Model): ) # Change tracking - previous_cost = models.IntegerField( + previous_tokens_per_credit = models.IntegerField( null=True, blank=True, - help_text="Cost before last update (for audit trail)" + help_text="Tokens per credit before last update (for audit trail)" ) # History tracking @@ -179,20 +181,92 @@ class CreditCostConfig(models.Model): ordering = ['operation_type'] def __str__(self): - return f"{self.display_name} - {self.credits_cost} credits {self.unit}" + return f"{self.display_name} - {self.tokens_per_credit} tokens/credit" def save(self, *args, **kwargs): - # Track cost changes + # Track token ratio changes if self.pk: try: old = CreditCostConfig.objects.get(pk=self.pk) - if old.credits_cost != self.credits_cost: - self.previous_cost = old.credits_cost + if old.tokens_per_credit != self.tokens_per_credit: + self.previous_tokens_per_credit = old.tokens_per_credit except CreditCostConfig.DoesNotExist: pass super().save(*args, **kwargs) +class BillingConfiguration(models.Model): + """ + System-wide billing configuration (Singleton). + Global settings for token-credit pricing. + """ + # Default token-to-credit ratio + default_tokens_per_credit = models.IntegerField( + default=100, + validators=[MinValueValidator(1)], + help_text="Default: How many tokens equal 1 credit (e.g., 100)" + ) + + # Credit pricing + default_credit_price_usd = models.DecimalField( + max_digits=10, + decimal_places=4, + default=Decimal('0.01'), + validators=[MinValueValidator(Decimal('0.0001'))], + help_text="Default price per credit in USD" + ) + + # Reporting settings + enable_token_based_reporting = models.BooleanField( + default=True, + help_text="Show token metrics in all reports" + ) + + # Rounding settings + ROUNDING_CHOICES = [ + ('up', 'Round Up'), + ('down', 'Round Down'), + ('nearest', 'Round to Nearest'), + ] + + credit_rounding_mode = models.CharField( + max_length=10, + default='up', + choices=ROUNDING_CHOICES, + help_text="How to round fractional credits" + ) + + # Audit fields + updated_at = models.DateTimeField(auto_now=True) + updated_by = models.ForeignKey( + settings.AUTH_USER_MODEL, + on_delete=models.SET_NULL, + null=True, + blank=True, + help_text="Admin who last updated" + ) + + class Meta: + app_label = 'billing' + db_table = 'igny8_billing_configuration' + verbose_name = 'Billing Configuration' + verbose_name_plural = 'Billing Configuration' + + def save(self, *args, **kwargs): + """Enforce singleton pattern""" + self.pk = 1 + super().save(*args, **kwargs) + + @classmethod + def get_config(cls): + """Get or create the singleton config""" + config, created = cls.objects.get_or_create(pk=1) + return config + + def __str__(self): + return f"Billing Configuration (1 credit = {self.default_tokens_per_credit} tokens)" + + class PlanLimitUsage(AccountBaseModel): """ Track monthly usage of plan limits (ideas, words, images, prompts) diff --git a/backend/igny8_core/business/billing/services/credit_service.py b/backend/igny8_core/business/billing/services/credit_service.py index da7d0d2f..abfd9a48 100644 --- a/backend/igny8_core/business/billing/services/credit_service.py +++ b/backend/igny8_core/business/billing/services/credit_service.py @@ -10,111 +10,101 @@ from igny8_core.auth.models import Account class CreditService: - """Service for managing credits""" + """Service for managing credits - Token-based only""" @staticmethod - def get_credit_cost(operation_type, amount=None): + def calculate_credits_from_tokens(operation_type, tokens_input, tokens_output): """ - Get credit cost for operation. - Now checks database config first, falls back to constants. + Calculate credits from actual token usage using configured ratio. + This is the ONLY way credits are calculated in the system. Args: - operation_type: Type of operation (from CREDIT_COSTS) - amount: Optional amount (word count, image count, etc.) + operation_type: Type of operation + tokens_input: Input tokens used + tokens_output: Output tokens used Returns: - int: Number of credits required - + int: Credits to deduct + Raises: - CreditCalculationError: If operation type is unknown + CreditCalculationError: If configuration error """ import logging + import math + from igny8_core.business.billing.models import CreditCostConfig, BillingConfiguration + logger = logging.getLogger(__name__) - # Try to get from database config first - try: - from igny8_core.business.billing.models import CreditCostConfig - - config = CreditCostConfig.objects.filter( - operation_type=operation_type, - is_active=True - ).first() - - if config: - base_cost = config.credits_cost - - # Apply unit-based calculation - if config.unit == 'per_100_words' and amount: - return max(1, int(base_cost * (amount / 100))) - elif config.unit == 'per_200_words' and amount: - return max(1, int(base_cost * (amount / 200))) - elif config.unit in ['per_item', 'per_image'] and amount: - return base_cost * amount - else: - return base_cost + # Get operation config (use global default if not found) + config = CreditCostConfig.objects.filter( + operation_type=operation_type, + is_active=True + ).first() - except Exception as e: - logger.warning(f"Failed to get cost from database, using constants: {e}") + if not config: + # Use global billing config as fallback + billing_config = BillingConfiguration.get_config() + tokens_per_credit = billing_config.default_tokens_per_credit + min_credits = 1 + logger.info(f"No config for {operation_type}, using default: {tokens_per_credit} tokens/credit") + else: + tokens_per_credit = config.tokens_per_credit + min_credits = config.min_credits - # Fallback to hardcoded constants - base_cost = CREDIT_COSTS.get(operation_type, 0) - if base_cost == 0: - raise CreditCalculationError(f"Unknown operation type: {operation_type}") + # Calculate total tokens + total_tokens = (tokens_input or 0) + (tokens_output or 0) - # Variable cost operations (legacy logic) - if operation_type == 'content_generation' and amount: - # Per 100 words - return max(1, int(base_cost * (amount / 100))) - elif operation_type == 'optimization' and amount: - # Per 200 words - return max(1, int(base_cost * (amount / 200))) - elif operation_type == 'image_generation' and amount: - # Per image - return base_cost * amount - elif operation_type == 'idea_generation' and amount: - # Per idea - return base_cost * amount + # Calculate credits (fractional) + if tokens_per_credit <= 0: + raise CreditCalculationError(f"Invalid tokens_per_credit: {tokens_per_credit}") - # Fixed cost operations - return base_cost + credits_float = total_tokens / tokens_per_credit + + # Get rounding mode from global config + billing_config = BillingConfiguration.get_config() + rounding_mode = billing_config.credit_rounding_mode + + if rounding_mode == 'up': + credits = math.ceil(credits_float) + elif rounding_mode == 'down': + credits = math.floor(credits_float) + else: # nearest + credits = round(credits_float) + + # Apply minimum + credits = max(credits, min_credits) + + logger.info( + f"Calculated credits for {operation_type}: " + f"{total_tokens} tokens ({tokens_input} in, {tokens_output} out) " + f"÷ {tokens_per_credit} = {credits} credits" + ) + + return credits @staticmethod - def check_credits(account, operation_type, amount=None): + def check_credits_for_tokens(account, operation_type, estimated_tokens_input, estimated_tokens_output): """ - Check if account has sufficient credits for an operation. + Check if account has sufficient credits based on estimated token usage. Args: account: Account instance operation_type: Type of operation - amount: Optional amount (word count, image count, etc.) + estimated_tokens_input: Estimated input tokens + estimated_tokens_output: Estimated output tokens Raises: InsufficientCreditsError: If account doesn't have enough credits """ - required = CreditService.get_credit_cost(operation_type, amount) + required = CreditService.calculate_credits_from_tokens( + operation_type, estimated_tokens_input, estimated_tokens_output + ) if account.credits < required: raise InsufficientCreditsError( f"Insufficient credits. Required: {required}, Available: {account.credits}" ) return True - @staticmethod - def check_credits_legacy(account, required_credits): - """ - Legacy method: Check if account has enough credits (for backward compatibility). - - Args: - account: Account instance - required_credits: Number of credits required - - Raises: - InsufficientCreditsError: If account doesn't have enough credits - """ - if account.credits < required_credits: - raise InsufficientCreditsError( - f"Insufficient credits. Required: {required_credits}, Available: {account.credits}" - ) - @staticmethod @transaction.atomic def deduct_credits(account, amount, operation_type, description, metadata=None, cost_usd=None, model_used=None, tokens_input=None, tokens_output=None, related_object_type=None, related_object_id=None): @@ -172,44 +162,65 @@ class CreditService: @staticmethod @transaction.atomic - def deduct_credits_for_operation(account, operation_type, amount=None, description=None, metadata=None, cost_usd=None, model_used=None, tokens_input=None, tokens_output=None, related_object_type=None, related_object_id=None): + def deduct_credits_for_operation( + account, + operation_type, + tokens_input, + tokens_output, + description=None, + metadata=None, + cost_usd=None, + model_used=None, + related_object_type=None, + related_object_id=None + ): """ - Deduct credits for an operation (convenience method that calculates cost automatically). + Deduct credits for an operation based on actual token usage. + This is the ONLY way to deduct credits in the token-based system. Args: account: Account instance operation_type: Type of operation - amount: Optional amount (word count, image count, etc.) + tokens_input: REQUIRED - Actual input tokens used + tokens_output: REQUIRED - Actual output tokens used description: Optional description (auto-generated if not provided) metadata: Optional metadata dict cost_usd: Optional cost in USD model_used: Optional AI model used - tokens_input: Optional input tokens - tokens_output: Optional output tokens related_object_type: Optional related object type related_object_id: Optional related object ID Returns: int: New credit balance + + Raises: + ValueError: If tokens_input or tokens_output not provided """ - # Calculate credit cost - credits_required = CreditService.get_credit_cost(operation_type, amount) + # Validate token inputs + if tokens_input is None or tokens_output is None: + raise ValueError( + f"tokens_input and tokens_output are REQUIRED for credit deduction. " + f"Got: tokens_input={tokens_input}, tokens_output={tokens_output}" + ) + + # Calculate credits from actual token usage + credits_required = CreditService.calculate_credits_from_tokens( + operation_type, tokens_input, tokens_output + ) # Check sufficient credits - CreditService.check_credits(account, operation_type, amount) + if account.credits < credits_required: + raise InsufficientCreditsError( + f"Insufficient credits. Required: {credits_required}, Available: {account.credits}" + ) # Auto-generate description if not provided if not description: - if operation_type == 'clustering': - description = f"Clustering operation" - elif operation_type == 'idea_generation': - description = f"Generated {amount or 1} idea(s)" - elif operation_type == 'content_generation': - description = f"Generated content ({amount or 0} words)" - elif operation_type == 'image_generation': - description = f"Generated {amount or 1} image(s)" - else: - description = f"{operation_type} operation" + total_tokens = tokens_input + tokens_output + description = ( + f"{operation_type}: {total_tokens} tokens " + f"({tokens_input} in, {tokens_output} out) = {credits_required} credits" + ) return CreditService.deduct_credits( account=account, @@ -256,39 +267,4 @@ class CreditService: ) return account.credits - - @staticmethod - def calculate_credits_for_operation(operation_type, **kwargs): - """ - Calculate credits needed for an operation. - Legacy method - use get_credit_cost() instead. - - Args: - operation_type: Type of operation - **kwargs: Operation-specific parameters - - Returns: - int: Number of credits required - - Raises: - CreditCalculationError: If calculation fails - """ - # Map legacy operation types - if operation_type == 'ideas': - operation_type = 'idea_generation' - elif operation_type == 'content': - operation_type = 'content_generation' - elif operation_type == 'images': - operation_type = 'image_generation' - - # Extract amount from kwargs - amount = None - if 'word_count' in kwargs: - amount = kwargs.get('word_count') - elif 'image_count' in kwargs: - amount = kwargs.get('image_count') - elif 'idea_count' in kwargs: - amount = kwargs.get('idea_count') - - return CreditService.get_credit_cost(operation_type, amount) diff --git a/backend/igny8_core/business/linking/services/linker_service.py b/backend/igny8_core/business/linking/services/linker_service.py index 1300da97..481ef9c1 100644 --- a/backend/igny8_core/business/linking/services/linker_service.py +++ b/backend/igny8_core/business/linking/services/linker_service.py @@ -63,10 +63,14 @@ class LinkerService: content.linker_version += 1 content.save(update_fields=['html_content', 'internal_links', 'linker_version']) - # Deduct credits + # Deduct credits (non-AI operation - use fixed token estimate) + # Estimate: 1 token per 4 characters of HTML content + estimated_tokens = len(content.html_content or '') // 4 self.credit_service.deduct_credits_for_operation( account=account, operation_type='linking', + tokens_input=estimated_tokens, + tokens_output=0, # No output tokens for linking operation description=f"Internal linking for content: {content.title or 'Untitled'}", related_object_type='content', related_object_id=content.id @@ -139,10 +143,14 @@ class LinkerService: content.linker_version += 1 content.save(update_fields=['html_content', 'internal_links', 'linker_version']) - # Deduct credits + # Deduct credits (non-AI operation - use fixed token estimate) + # Estimate: 1 token per 4 characters of HTML content + estimated_tokens = len(content.html_content or '') // 4 self.credit_service.deduct_credits_for_operation( account=account, operation_type='linking', + tokens_input=estimated_tokens, + tokens_output=0, description=f"Product linking for: {content.title or 'Untitled'}", related_object_type='content', related_object_id=content.id @@ -193,10 +201,14 @@ class LinkerService: content.linker_version += 1 content.save(update_fields=['html_content', 'internal_links', 'linker_version']) - # Deduct credits + # Deduct credits (non-AI operation - use fixed token estimate) + # Estimate: 1 token per 4 characters of HTML content + estimated_tokens = len(content.html_content or '') // 4 self.credit_service.deduct_credits_for_operation( account=account, operation_type='linking', + tokens_input=estimated_tokens, + tokens_output=0, description=f"Taxonomy linking for: {content.title or 'Untitled'}", related_object_type='content', related_object_id=content.id diff --git a/backend/igny8_core/business/optimization/services/optimizer_service.py b/backend/igny8_core/business/optimization/services/optimizer_service.py index f3e7fe47..48a36026 100644 --- a/backend/igny8_core/business/optimization/services/optimizer_service.py +++ b/backend/igny8_core/business/optimization/services/optimizer_service.py @@ -133,7 +133,10 @@ class OptimizerService: scores_after = self.analyzer.analyze(optimized_content) # Calculate credits used - credits_used = self.credit_service.get_credit_cost('optimization', word_count) + estimated_tokens = len(content.html_content or '') // 4 + credits_used = self.credit_service.calculate_credits_from_tokens( + 'optimization', estimated_tokens, 0 + ) # Update optimization task task.scores_after = scores_after @@ -148,18 +151,22 @@ class OptimizerService: content.optimization_scores = scores_after content.save(update_fields=['html_content', 'optimizer_version', 'optimization_scores']) - # Deduct credits + # Deduct credits (non-AI operation - use fixed token estimate based on content size) + # Estimate: 1 token per 4 characters of HTML content + estimated_tokens = len(content.html_content or '') // 4 self.credit_service.deduct_credits_for_operation( account=account, operation_type='optimization', - amount=word_count, + tokens_input=estimated_tokens, + tokens_output=0, description=f"Content optimization: {content.title or 'Untitled'}", related_object_type='content', related_object_id=content.id, metadata={ 'scores_before': scores_before, 'scores_after': scores_after, - 'improvement': scores_after.get('overall_score', 0) - scores_before.get('overall_score', 0) + 'improvement': scores_after.get('overall_score', 0) - scores_before.get('overall_score', 0), + 'word_count': word_count } ) @@ -279,7 +286,10 @@ class OptimizerService: scores_after = self._enhance_product_scores(scores_after, optimized_content) # Calculate credits used - credits_used = self.credit_service.get_credit_cost('optimization', word_count) + estimated_tokens = len(content.html_content or '') // 4 + credits_used = self.credit_service.calculate_credits_from_tokens( + 'optimization', estimated_tokens, 0 + ) # Update optimization task task.scores_after = scores_after @@ -294,11 +304,14 @@ class OptimizerService: content.optimization_scores = scores_after content.save(update_fields=['html_content', 'optimizer_version', 'optimization_scores']) - # Deduct credits + # Deduct credits (non-AI operation - use fixed token estimate based on content size) + # Estimate: 1 token per 4 characters of HTML content + estimated_tokens = len(content.html_content or '') // 4 self.credit_service.deduct_credits_for_operation( account=account, operation_type='optimization', - amount=word_count, + tokens_input=estimated_tokens, + tokens_output=0, description=f"Product optimization: {content.title or 'Untitled'}", related_object_type='content', related_object_id=content.id, @@ -306,6 +319,7 @@ class OptimizerService: 'scores_before': scores_before, 'scores_after': scores_after, 'improvement': scores_after.get('overall_score', 0) - scores_before.get('overall_score', 0), + 'word_count': word_count, 'entity_type': 'product' } ) @@ -372,7 +386,11 @@ class OptimizerService: scores_after = self._enhance_taxonomy_scores(scores_after, optimized_content) # Calculate credits used - credits_used = self.credit_service.get_credit_cost('optimization', word_count) + # Calculate estimated credits for task tracking + estimated_tokens = len(content.html_content or '') // 4 + credits_used = self.credit_service.calculate_credits_from_tokens( + 'optimization', estimated_tokens, 0 + ) # Update optimization task task.scores_after = scores_after @@ -387,17 +405,20 @@ class OptimizerService: content.optimization_scores = scores_after content.save(update_fields=['html_content', 'optimizer_version', 'optimization_scores']) - # Deduct credits + # Deduct credits (non-AI operation - use fixed token estimate based on content size) + # Estimate: 1 token per 4 characters of HTML content self.credit_service.deduct_credits_for_operation( account=account, operation_type='optimization', - amount=word_count, + tokens_input=estimated_tokens, + tokens_output=0, description=f"Taxonomy optimization: {content.title or 'Untitled'}", related_object_type='content', related_object_id=content.id, metadata={ 'scores_before': scores_before, 'scores_after': scores_after, + 'word_count': word_count, 'improvement': scores_after.get('overall_score', 0) - scores_before.get('overall_score', 0), 'entity_type': 'taxonomy' } diff --git a/backend/igny8_core/management/commands/backfill_tokens.py b/backend/igny8_core/management/commands/backfill_tokens.py new file mode 100644 index 00000000..35afd12f --- /dev/null +++ b/backend/igny8_core/management/commands/backfill_tokens.py @@ -0,0 +1,62 @@ +""" +Backfill token data from AITaskLog to CreditUsageLog +""" +from django.core.management.base import BaseCommand +from igny8_core.ai.models import AITaskLog +from igny8_core.modules.billing.models import CreditUsageLog +from datetime import timedelta + + +class Command(BaseCommand): + help = 'Backfill token data from AITaskLog to CreditUsageLog' + + def handle(self, *args, **options): + self.stdout.write("=== Token Data Backfill ===\n") + + # Get AITaskLog entries with token data + ai_logs = AITaskLog.objects.filter( + tokens__gt=0, + status='success' + ).select_related('account').order_by('created_at') + + self.stdout.write(f"Found {ai_logs.count()} AITaskLog entries with tokens\n") + + updated_count = 0 + skipped_count = 0 + no_match_count = 0 + + for ai_log in ai_logs: + # Find matching CreditUsageLog within 10 second window + time_start = ai_log.created_at - timedelta(seconds=10) + time_end = ai_log.created_at + timedelta(seconds=10) + + # Try to find exact match + credit_log = CreditUsageLog.objects.filter( + account=ai_log.account, + created_at__gte=time_start, + created_at__lte=time_end, + tokens_input=0, + tokens_output=0 + ).order_by('created_at').first() + + if credit_log: + # AITaskLog has total tokens, estimate 40/60 split for input/output + # This is approximate but better than 0 + total_tokens = ai_log.tokens + estimated_input = int(total_tokens * 0.4) + estimated_output = total_tokens - estimated_input + + credit_log.tokens_input = estimated_input + credit_log.tokens_output = estimated_output + credit_log.save(update_fields=['tokens_input', 'tokens_output']) + + updated_count += 1 + if updated_count % 50 == 0: + self.stdout.write(f" Updated {updated_count} records...") + else: + no_match_count += 1 + + self.stdout.write(self.style.SUCCESS(f"\n✅ Backfill complete!")) + self.stdout.write(f" Updated: {updated_count}") + self.stdout.write(f" No match: {no_match_count}") + self.stdout.write(f" Total processed: {ai_logs.count()}") diff --git a/backend/igny8_core/modules/billing/admin.py b/backend/igny8_core/modules/billing/admin.py index 749ff1d0..e4ce5f24 100644 --- a/backend/igny8_core/modules/billing/admin.py +++ b/backend/igny8_core/modules/billing/admin.py @@ -9,6 +9,7 @@ from simple_history.admin import SimpleHistoryAdmin from igny8_core.admin.base import AccountAdminMixin, Igny8ModelAdmin from igny8_core.business.billing.models import ( CreditCostConfig, + BillingConfiguration, Invoice, Payment, CreditPackage, @@ -426,55 +427,57 @@ class CreditCostConfigAdmin(SimpleHistoryAdmin, Igny8ModelAdmin): list_display = [ 'operation_type', 'display_name', - 'credits_cost_display', - 'unit', + 'tokens_per_credit_display', + 'price_per_credit_usd', + 'min_credits', 'is_active', 'cost_change_indicator', 'updated_at', 'updated_by' ] - list_filter = ['is_active', 'unit', 'updated_at'] + list_filter = ['is_active', 'updated_at'] search_fields = ['operation_type', 'display_name', 'description'] fieldsets = ( ('Operation', { 'fields': ('operation_type', 'display_name', 'description') }), - ('Cost Configuration', { - 'fields': ('credits_cost', 'unit', 'is_active') + ('Token-to-Credit Configuration', { + 'fields': ('tokens_per_credit', 'min_credits', 'price_per_credit_usd', 'is_active'), + 'description': 'Configure how tokens are converted to credits for this operation' }), ('Audit Trail', { - 'fields': ('previous_cost', 'updated_by', 'created_at', 'updated_at'), + 'fields': ('previous_tokens_per_credit', 'updated_by', 'created_at', 'updated_at'), 'classes': ('collapse',) }), ) - readonly_fields = ['created_at', 'updated_at', 'previous_cost'] + readonly_fields = ['created_at', 'updated_at', 'previous_tokens_per_credit'] - def credits_cost_display(self, obj): - """Show cost with color coding""" - if obj.credits_cost >= 20: - color = 'red' - elif obj.credits_cost >= 10: + def tokens_per_credit_display(self, obj): + """Show token ratio with color coding""" + if obj.tokens_per_credit <= 50: + color = 'red' # Expensive (low tokens per credit) + elif obj.tokens_per_credit <= 100: color = 'orange' else: - color = 'green' + color = 'green' # Cheap (high tokens per credit) return format_html( - '{} credits', + '{} tokens/credit', color, - obj.credits_cost + obj.tokens_per_credit ) - credits_cost_display.short_description = 'Cost' + tokens_per_credit_display.short_description = 'Token Ratio' def cost_change_indicator(self, obj): - """Show if cost changed recently""" - if obj.previous_cost is not None: - if obj.credits_cost > obj.previous_cost: - icon = '📈' # Increased + """Show if token ratio changed recently""" + if obj.previous_tokens_per_credit is not None: + if obj.tokens_per_credit < obj.previous_tokens_per_credit: + icon = '📈' # More expensive (fewer tokens per credit) color = 'red' - elif obj.credits_cost < obj.previous_cost: - icon = '📉' # Decreased + elif obj.tokens_per_credit > obj.previous_tokens_per_credit: + icon = '📉' # Cheaper (more tokens per credit) color = 'green' else: icon = '➡️' # Same @@ -484,8 +487,8 @@ class CreditCostConfigAdmin(SimpleHistoryAdmin, Igny8ModelAdmin): '{} ({} → {})', icon, color, - obj.previous_cost, - obj.credits_cost + obj.previous_tokens_per_credit, + obj.tokens_per_credit ) return '—' cost_change_indicator.short_description = 'Recent Change' @@ -538,3 +541,47 @@ class PlanLimitUsageAdmin(AccountAdminMixin, Igny8ModelAdmin): return f"{obj.period_start} to {obj.period_end}" period_display.short_description = 'Billing Period' + +@admin.register(BillingConfiguration) +class BillingConfigurationAdmin(Igny8ModelAdmin): + """Admin for global billing configuration (Singleton)""" + list_display = [ + 'id', + 'default_tokens_per_credit', + 'default_credit_price_usd', + 'credit_rounding_mode', + 'enable_token_based_reporting', + 'updated_at', + 'updated_by' + ] + + fieldsets = ( + ('Global Token-to-Credit Settings', { + 'fields': ('default_tokens_per_credit', 'default_credit_price_usd', 'credit_rounding_mode'), + 'description': 'These settings apply when no operation-specific config exists' + }), + ('Reporting Settings', { + 'fields': ('enable_token_based_reporting',), + 'description': 'Control token-based reporting features' + }), + ('Audit Trail', { + 'fields': ('updated_by', 'updated_at'), + 'classes': ('collapse',) + }), + ) + + readonly_fields = ['updated_at'] + + def has_add_permission(self, request): + """Only allow one instance (singleton)""" + from igny8_core.business.billing.models import BillingConfiguration + return not BillingConfiguration.objects.exists() + + def has_delete_permission(self, request, obj=None): + """Prevent deletion of the singleton""" + return False + + def save_model(self, request, obj, form, change): + """Track who made the change""" + obj.updated_by = request.user + super().save_model(request, obj, form, change) diff --git a/backend/igny8_core/modules/billing/migrations/0018_remove_creditcostconfig_credits_cost_and_more.py b/backend/igny8_core/modules/billing/migrations/0018_remove_creditcostconfig_credits_cost_and_more.py new file mode 100644 index 00000000..dd0e5d8d --- /dev/null +++ b/backend/igny8_core/modules/billing/migrations/0018_remove_creditcostconfig_credits_cost_and_more.py @@ -0,0 +1,99 @@ +# Generated by Django 5.2.9 on 2025-12-19 18:20 + +import django.core.validators +import django.db.models.deletion +from decimal import Decimal +from django.conf import settings +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('billing', '0017_add_history_tracking'), + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.RemoveField( + model_name='creditcostconfig', + name='credits_cost', + ), + migrations.RemoveField( + model_name='creditcostconfig', + name='previous_cost', + ), + migrations.RemoveField( + model_name='creditcostconfig', + name='unit', + ), + migrations.RemoveField( + model_name='historicalcreditcostconfig', + name='credits_cost', + ), + migrations.RemoveField( + model_name='historicalcreditcostconfig', + name='previous_cost', + ), + migrations.RemoveField( + model_name='historicalcreditcostconfig', + name='unit', + ), + migrations.AddField( + model_name='creditcostconfig', + name='min_credits', + field=models.IntegerField(default=1, help_text='Minimum credits to charge regardless of token usage', validators=[django.core.validators.MinValueValidator(0)]), + ), + migrations.AddField( + model_name='creditcostconfig', + name='previous_tokens_per_credit', + field=models.IntegerField(blank=True, help_text='Tokens per credit before last update (for audit trail)', null=True), + ), + migrations.AddField( + model_name='creditcostconfig', + name='price_per_credit_usd', + field=models.DecimalField(decimal_places=4, default=Decimal('0.01'), help_text='USD price per credit (for revenue reporting)', max_digits=10, validators=[django.core.validators.MinValueValidator(Decimal('0.0001'))]), + ), + migrations.AddField( + model_name='creditcostconfig', + name='tokens_per_credit', + field=models.IntegerField(default=100, help_text='Number of tokens that equal 1 credit (e.g., 100 tokens = 1 credit)', validators=[django.core.validators.MinValueValidator(1)]), + ), + migrations.AddField( + model_name='historicalcreditcostconfig', + name='min_credits', + field=models.IntegerField(default=1, help_text='Minimum credits to charge regardless of token usage', validators=[django.core.validators.MinValueValidator(0)]), + ), + migrations.AddField( + model_name='historicalcreditcostconfig', + name='previous_tokens_per_credit', + field=models.IntegerField(blank=True, help_text='Tokens per credit before last update (for audit trail)', null=True), + ), + migrations.AddField( + model_name='historicalcreditcostconfig', + name='price_per_credit_usd', + field=models.DecimalField(decimal_places=4, default=Decimal('0.01'), help_text='USD price per credit (for revenue reporting)', max_digits=10, validators=[django.core.validators.MinValueValidator(Decimal('0.0001'))]), + ), + migrations.AddField( + model_name='historicalcreditcostconfig', + name='tokens_per_credit', + field=models.IntegerField(default=100, help_text='Number of tokens that equal 1 credit (e.g., 100 tokens = 1 credit)', validators=[django.core.validators.MinValueValidator(1)]), + ), + migrations.CreateModel( + name='BillingConfiguration', + fields=[ + ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('default_tokens_per_credit', models.IntegerField(default=100, help_text='Default: How many tokens equal 1 credit (e.g., 100)', validators=[django.core.validators.MinValueValidator(1)])), + ('default_credit_price_usd', models.DecimalField(decimal_places=4, default=Decimal('0.01'), help_text='Default price per credit in USD', max_digits=10, validators=[django.core.validators.MinValueValidator(Decimal('0.0001'))])), + ('enable_token_based_reporting', models.BooleanField(default=True, help_text='Show token metrics in all reports')), + ('credit_rounding_mode', models.CharField(choices=[('up', 'Round Up'), ('down', 'Round Down'), ('nearest', 'Round to Nearest')], default='up', help_text='How to round fractional credits', max_length=10)), + ('updated_at', models.DateTimeField(auto_now=True)), + ('updated_by', models.ForeignKey(blank=True, help_text='Admin who last updated', null=True, on_delete=django.db.models.deletion.SET_NULL, to=settings.AUTH_USER_MODEL)), + ], + options={ + 'verbose_name': 'Billing Configuration', + 'verbose_name_plural': 'Billing Configuration', + 'db_table': 'igny8_billing_configuration', + }, + ), + ] diff --git a/backend/igny8_core/modules/billing/migrations/0019_populate_token_based_config.py b/backend/igny8_core/modules/billing/migrations/0019_populate_token_based_config.py new file mode 100644 index 00000000..cb06f8d8 --- /dev/null +++ b/backend/igny8_core/modules/billing/migrations/0019_populate_token_based_config.py @@ -0,0 +1,111 @@ +# Generated by Django 5.2.9 on 2025-12-19 18:28 + +from django.db import migrations +from decimal import Decimal + + +def populate_token_config(apps, schema_editor): + """ + Populate BillingConfiguration singleton and update CreditCostConfig records. + Token-based pricing ratios (tokens per 1 credit): + - Default: 100 tokens = 1 credit + - AI operations vary by complexity + """ + BillingConfiguration = apps.get_model('billing', 'BillingConfiguration') + CreditCostConfig = apps.get_model('billing', 'CreditCostConfig') + + # Create BillingConfiguration singleton (if not exists) + if not BillingConfiguration.objects.exists(): + BillingConfiguration.objects.create( + default_tokens_per_credit=100, + default_credit_price_usd=Decimal('0.01'), + enable_token_based_reporting=True, + credit_rounding_mode='up' + ) + + # Token-to-credit ratios for each operation type + # Lower number = more expensive (fewer tokens per credit) + # Higher number = cheaper (more tokens per credit) + operation_configs = { + 'clustering': { + 'tokens_per_credit': 150, # Clustering is fairly complex + 'min_credits': 2, + 'price_per_credit_usd': Decimal('0.01'), + 'display_name': 'Content Clustering', + 'description': 'AI-powered keyword clustering' + }, + 'idea_generation': { + 'tokens_per_credit': 200, # Idea generation is mid-complexity + 'min_credits': 1, + 'price_per_credit_usd': Decimal('0.01'), + 'display_name': 'Idea Generation', + 'description': 'AI content idea generation' + }, + 'content_generation': { + 'tokens_per_credit': 100, # Content generation is expensive (outputs many tokens) + 'min_credits': 3, + 'price_per_credit_usd': Decimal('0.01'), + 'display_name': 'Content Generation', + 'description': 'AI content writing' + }, + 'image_generation': { + 'tokens_per_credit': 50, # Image generation is most expensive + 'min_credits': 5, + 'price_per_credit_usd': Decimal('0.02'), + 'display_name': 'Image Generation', + 'description': 'AI image generation' + }, + 'optimization': { + 'tokens_per_credit': 200, # Optimization is algorithm-based with minimal AI + 'min_credits': 1, + 'price_per_credit_usd': Decimal('0.005'), + 'display_name': 'Content Optimization', + 'description': 'SEO and content optimization' + }, + 'linking': { + 'tokens_per_credit': 300, # Linking is mostly algorithmic + 'min_credits': 1, + 'price_per_credit_usd': Decimal('0.005'), + 'display_name': 'Internal Linking', + 'description': 'Automatic internal link injection' + }, + 'reparse': { + 'tokens_per_credit': 150, # Reparse is mid-complexity + 'min_credits': 1, + 'price_per_credit_usd': Decimal('0.01'), + 'display_name': 'Content Reparse', + 'description': 'Content reparsing and analysis' + }, + } + + # Update or create CreditCostConfig records + for operation_type, config in operation_configs.items(): + CreditCostConfig.objects.update_or_create( + operation_type=operation_type, + defaults={ + 'tokens_per_credit': config['tokens_per_credit'], + 'min_credits': config['min_credits'], + 'price_per_credit_usd': config['price_per_credit_usd'], + 'display_name': config['display_name'], + 'description': config['description'], + 'is_active': True + } + ) + + +def reverse_token_config(apps, schema_editor): + """Reverse migration - clean up configuration""" + BillingConfiguration = apps.get_model('billing', 'BillingConfiguration') + BillingConfiguration.objects.all().delete() + # Note: We don't delete CreditCostConfig records as they may have historical data + + +class Migration(migrations.Migration): + + dependencies = [ + ('billing', '0018_remove_creditcostconfig_credits_cost_and_more'), + ] + + operations = [ + migrations.RunPython(populate_token_config, reverse_token_config), + ] diff --git a/backend/igny8_core/templates/admin/reports/ai_cost_analysis.html b/backend/igny8_core/templates/admin/reports/ai_cost_analysis.html index 62980e45..0b62c2f5 100644 --- a/backend/igny8_core/templates/admin/reports/ai_cost_analysis.html +++ b/backend/igny8_core/templates/admin/reports/ai_cost_analysis.html @@ -23,31 +23,44 @@ -
${{ total_cost|floatformat:2 }}
{{ total_calls }} API calls
${{ total_revenue|floatformat:2 }}
+{{ total_credits_charged }} credits @ ${{ credit_price }}
+${{ total_margin|floatformat:2 }}
+{{ margin_percentage }}% margin
+${{ avg_cost_per_call|floatformat:4 }}
+${{ margin_per_1m_tokens|floatformat:2 }}
+ {% else %} +No token data yet
+New AI calls will populate this
+ {% endif %} +${{ margin_per_1k_credits|floatformat:2 }}
+ {% else %} +No data
+ {% endif %}${{ projected_monthly|floatformat:2 }}
+${{ projected_monthly|floatformat:2 }}
Based on last 7 days
{{ efficiency_score }}%
-Successful cost ratio
-${{ failed_cost|floatformat:2 }}
-{{ wasted_percentage|floatformat:1 }}% of total
-Historical logs (before system upgrade) don't have token data. Token tracking started after the recent backend update.
++ Next steps: Trigger any AI operation (content generation, clustering, etc.) and token data will start appearing here automatically. +
+{{ total_tokens|floatformat:0 }}
+{{ total_tokens|floatformat:0 }}
{{ total_calls }} API calls
{{ avg_tokens_per_call|floatformat:0 }}
+{{ avg_tokens_per_call|floatformat:0 }}