igny8/docs/CREDITS-TOKENS-GUIDE.md

IGNY8 Credits & Tokens System Guide

Version: 1.0
Last Updated: December 19, 2025
System Status: ✅ Fully Operational

---

Overview

IGNY8 uses a token-based credit system where all AI operations consume credits calculated from actual AI token usage. This guide covers configuration, data flow, and monitoring.

---

System Architecture

Data Flow

User Action (Content Generation, Ideas, etc.)
    ↓
Backend Service Initiated
    ↓
AI API Called (OpenAI, etc.)
    ↓
Response Received: {input_tokens, output_tokens, cost_usd, model}
    ↓
Credits Calculated: (total_tokens / tokens_per_credit)
    ↓
Credits Deducted from Account
    ↓
CreditUsageLog Created: {tokens_input, tokens_output, cost_usd, credits_used, model_used}
    ↓
Reports Updated with Real-Time Analytics

Key Components

1. BillingConfiguration - System-wide billing settings
2. CreditCostConfig - Per-operation token-to-credit ratios
3. CreditUsageLog - Transaction log with token data
4. AITaskLog - Detailed AI execution history

---

Configuration

1. Global Billing Settings

Location: Django Admin → Billing → Billing Configuration

Key Settings:

• Default Tokens Per Credit: 100 (base ratio)
• Default Credit Price: $0.01 USD
• Rounding Mode: Up (conservative billing)
• Token Reporting Enabled: Yes

When to Adjust:

• Change credit pricing across all operations
• Modify rounding behavior for credit calculations
• Enable/disable token tracking

2. Per-Operation Configuration

Location: Django Admin → Billing → Credit Cost Configs

Current Ratios:

Operation	Tokens/Credit	Min Credits	Price/Credit
Clustering	150	2	$0.0100
Content Generation	100	3	$0.0100
Idea Generation	200	1	$0.0100
Image Generation	50	5	$0.0200
Image Prompt Extraction	100	1	$0.0100
Linking	300	1	$0.0050
Optimization	200	1	$0.0050

Adjusting Pricing:

• Increase tokens_per_credit → Lower cost (more tokens per credit)
• Decrease tokens_per_credit → Higher cost (fewer tokens per credit)
• Adjust min_credits → Enforce minimum charge per operation
• Change price_per_credit_usd → Override default credit price

Example: To make Content Generation cheaper:

Current: 100 tokens/credit (1000 tokens = 10 credits)
Change to: 150 tokens/credit (1000 tokens = 6.67 → 7 credits)
Result: 30% cost reduction

3. Token Extraction

Location: backend/igny8_core/ai/engine.py (line 380-381)

Current Implementation:

tokens_input = raw_response.get('input_tokens', 0)
tokens_output = raw_response.get('output_tokens', 0)

Critical: Field names must match AI provider response format

• ✅ Correct: input_tokens, output_tokens
• ❌ Wrong: tokens_input, tokens_output

Supported Providers:

• OpenAI (GPT-4, GPT-4o, GPT-5.1)
• Anthropic Claude
• Runware (image generation)

---

Monitoring & Reports

1. AI Cost & Margin Analysis

URL: https://api.igny8.com/admin/reports/ai-cost-analysis/

Metrics Displayed:

• Total Cost - Actual USD spent on AI APIs
• Revenue - Income from credits charged
• Margin - Profit (Revenue - Cost) with percentage
• Margin / 1M Tokens - Profit efficiency per million tokens
• Margin / 1K Credits - Profit per thousand credits charged
• Projected Monthly - Forecasted costs based on trends

Tables:

• Model Cost Comparison - Profitability by AI model
• Top Spenders - Highest cost accounts
• Cost by Function - Profitability by operation type
• Cost Anomalies - Expensive outlier calls

Use Cases:

• Identify unprofitable operations or accounts
• Optimize token-to-credit ratios
• Detect unusual AI spending patterns
• Track margins over time

2. Token Usage Report

URL: https://api.igny8.com/admin/reports/token-usage/

Metrics Displayed:

• Total tokens consumed (input + output)
• Average tokens per call
• Cost per 1K tokens
• Token distribution by model
• Token distribution by operation
• Daily token trends

Use Cases:

• Understand token consumption patterns
• Identify token-heavy operations
• Optimize prompts to reduce token usage
• Track token efficiency over time

3. Usage Report

URL: https://api.igny8.com/admin/reports/usage/

Metrics Displayed:

• Total credits used system-wide
• Credits by operation type
• Top credit consumers
• Model usage distribution

Use Cases:

• Monitor overall system usage
• Identify high-volume users
• Track popular AI operations
• Plan capacity and scaling

4. Data Quality Report

URL: https://api.igny8.com/admin/reports/data-quality/

Purpose: Identify data integrity issues

• Orphaned content
• Duplicate keywords
• Missing SEO metadata

---

Data Models

CreditUsageLog (Primary Transaction Log)

Purpose: Record every credit deduction with full context

Key Fields:

• account - User account charged
• operation_type - Function executed (e.g., "content_generation")
• credits_used - Credits deducted
• cost_usd - Actual AI provider cost
• tokens_input - Input tokens consumed
• tokens_output - Output tokens generated
• model_used - AI model (e.g., "gpt-4o")
• related_object_type/id - Link to content/site/keyword
• metadata - Additional context (prompt, settings, etc.)

Query Examples:

# Total tokens for an account
CreditUsageLog.objects.filter(account=account).aggregate(
    total_tokens=Sum('tokens_input') + Sum('tokens_output')
)

# Average cost by operation
CreditUsageLog.objects.values('operation_type').annotate(
    avg_cost=Avg('cost_usd'),
    total_calls=Count('id')
)

# Margin analysis
logs = CreditUsageLog.objects.all()
revenue = logs.aggregate(Sum('credits_used'))['credits_used__sum'] * 0.01
cost = logs.aggregate(Sum('cost_usd'))['cost_usd__sum']
margin = revenue - cost

AITaskLog (Execution History)

Purpose: Detailed AI execution tracking

Key Fields:

• function_name - AI function executed
• account - User account
• cost - AI provider cost
• tokens - Total tokens (input + output)
• phase - Execution stage
• status - Success/failure
• execution_time - Processing duration
• raw_request/response - Full API data

Use Cases:

• Debug AI execution failures
• Analyze prompt effectiveness
• Track model performance
• Audit AI interactions

---

Historical Data Backfill

Issue

Prior to December 2025, token fields were not populated due to incorrect field name mapping.

Solution

A backfill script matched AITaskLog entries to CreditUsageLog records using:

• Account matching
• Timestamp matching (±10 second window)
• 40/60 input/output split estimation (when only total available)

Result

• ✅ 777,456 tokens backfilled
• ✅ 380/479 records updated (79% coverage)
• ✅ Historical margin analysis now available
• ⚠️ 99 records remain at 0 tokens (no matching AITaskLog)

Script Location

backend/igny8_core/management/commands/backfill_tokens.py

---

Troubleshooting

Empty Margin Metrics

Symptom: "Margin / 1M Tokens" shows "No token data yet"

Causes:

1. No recent AI calls with token data
2. Token extraction not working (field name mismatch)
3. Historical data has 0 tokens

Resolution:

1. Check AIEngine token extraction: tokens_input, tokens_output fields
2. Verify AI responses contain input_tokens, output_tokens
3. Run test AI operation and check CreditUsageLog
4. Consider backfill for historical data

Zero Tokens in CreditUsageLog

Symptom: tokens_input and tokens_output are 0

Causes:

1. Field name mismatch in AIEngine
2. AI provider not returning token data
3. Historical records before fix

Resolution:

1. Verify engine.py line 380-381 uses correct field names
2. Check AI provider API response format
3. Restart backend services after fixes
4. Future calls will populate correctly

Incorrect Margins

Symptom: Margin percentages seem wrong

Causes:

1. Incorrect token-to-credit ratios
2. Credit price misconfigured
3. Decimal division errors

Resolution:

1. Review CreditCostConfig ratios
2. Check BillingConfiguration credit price
3. Verify margin calculations use float() conversions
4. Check for TypeError in logs

Operations Not Charging Correctly

Symptom: Wrong number of credits deducted

Causes:

1. Token-to-credit ratio misconfigured
2. Minimum credits not enforced
3. Rounding mode incorrect

Resolution:

1. Check operation's CreditCostConfig
2. Verify min_credits setting
3. Review rounding_mode in BillingConfiguration
4. Test with known token count

---

Best Practices

Pricing Strategy

1. Monitor margins weekly - Use AI Cost Analysis report
2. Adjust ratios based on costs - If margins drop below 70%, decrease tokens_per_credit
3. Set reasonable minimums - Enforce min_credits for small operations
4. Track model costs - Some models (GPT-4) cost more than others

Token Optimization

1. Optimize prompts - Reduce unnecessary tokens
2. Use appropriate models - GPT-4o-mini for simple tasks
3. Cache results - Avoid duplicate AI calls
4. Monitor anomalies - Investigate unusually expensive calls

Data Integrity

1. Regular audits - Check token data completeness
2. Verify field mappings - Ensure AI responses parsed correctly
3. Monitor logs - Watch for errors in CreditService
4. Backup configurations - Export CreditCostConfig settings

Performance

1. Archive old logs - Move historical CreditUsageLog to archive tables
2. Index frequently queried fields - account, operation_type, created_at
3. Aggregate reports - Use materialized views for large datasets
4. Cache report data - Reduce database load

---

API Integration

Frontend Credit Display

Endpoint: /v1/billing/credits/balance/

Response:

{
  "credits": 1234,
  "plan_credits_per_month": 5000,
  "credits_used_this_month": 876,
  "credits_remaining": 4124
}

Pages Using This:

• /account/plans - Plans & Billing
• /account/usage - Usage Analytics
• Dashboard credit widget

Credit Transaction History

Endpoint: /v1/billing/credits/usage/

Response:

{
  "results": [
    {
      "id": 123,
      "operation_type": "Content Generation",
      "credits_used": 15,
      "tokens_input": 500,
      "tokens_output": 1000,
      "cost_usd": 0.015,
      "model_used": "gpt-4o",
      "created_at": "2025-12-19T10:30:00Z"
    }
  ]
}

---

Quick Reference

Common Operations

Check account credits:

account.credits  # Current balance

Manual credit adjustment:

CreditService.add_credits(account, amount=100, description="Bonus credits")

Get operation config:

config = CreditService.get_or_create_config('content_generation')
# Returns: CreditCostConfig with tokens_per_credit, min_credits

Calculate credits needed:

credits = CreditService.calculate_credits_from_tokens(
    operation_type='content_generation',
    tokens_input=500,
    tokens_output=1500
)
# Returns: 20 (if 100 tokens/credit)

Important File Locations

• Credit Service: backend/igny8_core/business/billing/services/credit_service.py
• AI Engine: backend/igny8_core/ai/engine.py
• Reports: backend/igny8_core/admin/reports.py
• Models: backend/igny8_core/modules/billing/models.py
• Admin: backend/igny8_core/modules/billing/admin.py

Report Access

All reports require staff/superuser login:

• AI Cost Analysis: /admin/reports/ai-cost-analysis/
• Token Usage: /admin/reports/token-usage/
• Usage Report: /admin/reports/usage/
• Data Quality: /admin/reports/data-quality/

---

Support & Updates

For questions or issues with the credits & tokens system:

1. Check Django admin logs: /admin/
2. Review CreditUsageLog for transaction details
3. Monitor AITaskLog for execution errors
4. Check backend logs: docker logs igny8_backend

System Maintainer: IGNY8 DevOps Team
Last Major Update: December 2025 (Token-based credit system implementation)