salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

final docs for final audit implemenation

Browse Source
This commit is contained in:
IGNY8 VPS (Salman)
2025-12-27 00:34:22 +00:00
parent 277ef5c81d
commit 7a9fa8fd8f
12 changed files with 2565 additions and 1590 deletions
Download Patch File Download Diff File Expand all files Collapse all files

308
to-do-s/part2/PRICING-SIMPLIFICATION.md Normal file
View File

@@ -0,0 +1,308 @@
# IGNY8 Pricing System - Final Simplified Plan
**Date:** December 26, 2025
**Status:** APPROVED DIRECTION
---
## 1. Current System Assessment ✅
The existing token-based credit system is **correct and safe**:
- ✅ `CreditCostConfig` with `tokens_per_credit` ratio per operation
- ✅ `min_credits` per operation (floor protection)
- ✅ Actual tokens calculated after AI call (accurate margin)
- ✅ Model-aware pricing (different models = different costs)
**Keep this system. Just simplify the limits and UI.**
---
## 2. Actual Cost Analysis (From Production Data)
### Per-Article Cost Breakdown (GPT-4.1 pricing)
| Stage | Input Tokens | Output Tokens | Actual Cost |
|-------|--------------|---------------|-------------|
| Clustering (per article share*) | ~400 | ~400 | **$0.005** |
| Idea Generation (per article share*) | ~900 | ~1000 | **$0.012** |
| Content Writing | 2500 | 3500 | **$0.041** |
| Image Prompt Extraction | 500 | 1100 | **$0.012** |
| **Text Subtotal** | | | **$0.07** |
| Images (3 avg @ $0.04) | | | **$0.12** |
| **TOTAL PER ARTICLE** | | | **$0.19** |
*With 6 images (max): **$0.31 per article**
### Target Markup: 10-15x
| Scenario | Our Cost | 10x Price | 15x Price |
|----------|----------|-----------|-----------|
| Article (3 images) | $0.19 | $1.90 | $2.85 |
| Article (6 images) | $0.31 | $3.10 | $4.65 |
---
## 3. Simplified Credit Pricing (10x Markup)
### 3.1 Credit Cost per Operation
Using **1 credit = $0.01** and **10x markup**:
| Operation | Actual Cost | Target Price | Credits |
|-----------|-------------|--------------|---------|
| Clustering | $0.005 | $0.05 | **5** |
| Idea Generation | $0.012 | $0.12 | **10** |
| Content Writing | $0.041 | $0.41 | **40** |
| Image Prompts | $0.012 | $0.12 | **10** |
| Image (per image) | $0.04 | $0.40 | **40** |
| Linking | $0.003 | $0.03 | **3** |
| Optimization | $0.005 | $0.05 | **5** |
### 3.2 Full Article Workflow Cost
| Workflow | Calculation | Total Credits |
|----------|-------------|---------------|
| Text only | 5 + 10 + 40 + 10 = 65 | **65 credits** |
| + 3 images | 65 + (3 × 40) = 185 | **185 credits** |
| + 6 images | 65 + (6 × 40) = 305 | **305 credits** |
| + linking + optimization | +3 +5 = +8 | **+8 credits** |
**Typical article: ~200 credits = $2.00** (at 10x margin)
---
## 4. Simplified Plan Structure
### 4.1 Remove All Limits Except Keywords
**DELETE:**
- ❌ max_content_ideas
- ❌ max_content_words
- ❌ max_images_basic
- ❌ max_images_premium
- ❌ max_image_prompts
- ❌ max_clusters (soft limit via credits)
- ❌ All usage tracking fields
**KEEP:**
- ✅ Credits (token-based, monthly allocation)
- ✅ max_keywords (storage limit, defines plan tier)
- ✅ max_sites, max_users (account management)
### 4.2 New Plan Tiers
| Plan | Price | Credits/Month | Keywords | Sites | Users | ~Articles |
|------|-------|---------------|----------|-------|-------|-----------|
| **Free** | $0 | 200 | 100 | 1 | 1 | ~1 |
| **Starter** | $29 | 3,000 | 1,000 | 3 | 3 | ~15 |
| **Growth** | $79 | 10,000 | 5,000 | 10 | 10 | ~50 |
| **Scale** | $199 | 30,000 | Unlimited | ∞ | ∞ | ~150 |
**Simple value proposition:**
- Starter: ~15 full articles/month for $29 (~$2/article)
- Growth: ~50 full articles/month for $79 (~$1.60/article)
- Scale: ~150 full articles/month for $199 (~$1.33/article)
### 4.3 Credit Top-Up Pricing
| Package | Credits | Price | Per Credit |
|---------|---------|-------|------------|
| Small | 500 | $7 | $0.014 |
| Medium | 2,000 | $25 | $0.0125 |
| Large | 5,000 | $55 | $0.011 |
| Bulk | 15,000 | $150 | $0.01 |
---
## 5. Content Length Feature (New)
### 5.1 Add Word Count Selection
**Content Settings (Site-level default):**
```
Target Article Length: [Short ~800] [Medium ~1500] [Long ~2500] [Custom: ___]
```
**Idea Generation (per-idea):**
- Include target word count in idea brief
- AI considers length when generating outline
**Content Generation (per-content):**
- Pass word count to prompt
- Affects token usage (longer = more credits)
### 5.2 Credit Adjustment for Length
| Length | Words | Token Estimate | Credits |
|--------|-------|----------------|---------|
| Short | ~800 | 2000 | 25 |
| Medium | ~1500 | 3500 | 40 |
| Long | ~2500 | 5500 | 60 |
| Extra Long | ~4000 | 8000 | 90 |
**User sees:** "Short article: ~150 credits, Long article: ~250 credits"
---
## 6. Implementation Plan
### Phase 1: Backend Updates (Day 1-2)
**6.1 Update CreditCostConfig values:**
```python
# New fixed credit costs (min_credits = display value)
CREDIT_CONFIGS = {
'clustering': {'tokens_per_credit': 160, 'min_credits': 5},
'idea_generation': {'tokens_per_credit': 190, 'min_credits': 10},
'content_generation': {'tokens_per_credit': 150, 'min_credits': 25}, # Base for short
'image_prompt_extraction': {'tokens_per_credit': 160, 'min_credits': 10},
'image_generation': {'tokens_per_credit': 1, 'min_credits': 40}, # Fixed per image
'linking': {'tokens_per_credit': 300, 'min_credits': 3},
'optimization': {'tokens_per_credit': 200, 'min_credits': 5},
}
```
**6.2 Remove monthly limit fields:**
```python
# Plan model - DELETE these fields
- max_content_ideas
- max_content_words
- max_images_basic
- max_images_premium
- max_image_prompts
# Account model - DELETE these fields
- usage_content_ideas
- usage_content_words
- usage_images_basic
- usage_images_premium
- usage_image_prompts
- usage_period_start
- usage_period_end
```
**6.3 Remove LimitService monthly methods:**
- Delete `check_monthly_limit()`
- Delete `increment_usage()`
- Delete monthly reset task
- Keep `check_hard_limit()` for sites/users/keywords
### Phase 2: Content Length Feature (Day 2-3)
**6.4 Add ContentSettings.target_word_count:**
```python
class ContentSettings(models.Model):
# ... existing fields ...
target_word_count = models.IntegerField(
default=1500,
choices=[(800, 'Short'), (1500, 'Medium'), (2500, 'Long'), (4000, 'Extra Long')],
help_text="Default target word count for articles"
)
```
**6.5 Update AI prompts:**
- `generate_ideas.py` - Include word count in idea brief
- `generate_content.py` - Pass word count target
### Phase 3: Frontend Updates (Day 3-4)
**6.6 Update Content Settings page:**
- Add word count selector
**6.7 Update Usage page:**
- Show credits only (remove monthly limit bars)
- Show keywords used vs limit
- Show simple cost reference
**6.8 Fix/Replace CreditCostsPanel.tsx:**
- Show fixed credit costs (not token-based confusion)
- Simple table: "Content: 40 credits, Image: 40 credits"
**6.9 Update Pricing page:**
- New plan structure
- "~X articles/month" messaging
### Phase 4: Migration & Cleanup (Day 4-5)
**6.10 Database migration:**
- Remove deprecated fields
- Update existing plans to new structure
**6.11 Clean up dead code:**
- Remove LimitService monthly tracking
- Remove usage increment calls in services
---
## 7. User Experience (After Simplification)
### What User Sees:
**Dashboard:**
```
Credits: 2,847 / 3,000 remaining
Keywords: 234 / 1,000
```
**Before generating content:**
```
Estimated cost: ~185 credits
- Content (medium): 40 credits
- Image prompts: 10 credits
- 3 images: 120 credits
- Clustering share: ~5 credits
- Ideas share: ~10 credits
```
**Content Settings:**
```
Article Length: [Short] [Medium ✓] [Long] [Extra Long]
Images per Article: [3 ▼]
```
### What Admin Sees:
**Plans admin:**
- Credits/month
- Keywords limit
- Sites/Users limits
- Price
**No more:**
- 5 different monthly limit fields
- Usage tracking complexity
- Monthly reset management
---
## 8. Summary
| Before | After |
|--------|-------|
| Credits + 5 monthly limits | Credits only |
| 9 limit fields per plan | 3 limit fields (credits, keywords, sites) |
| 5 usage tracking fields | 1 field (credits balance) |
| Complex monthly resets | Simple credit allocation |
| Unpredictable costs | Clear cost estimates |
| No length control | Word count selection |
**Result:**
- ✅ Safe margins (token-based calculation preserved)
- ✅ Simple for user (one number to track)
- ✅ Simple for admin (fewer fields to manage)
- ✅ Predictable (show estimated costs upfront)
- ✅ Flexible (manual or automation workflows work same way)
---
## 9. Immediate Actions
1. **Update CreditCostConfig** in database with new values
2. **Add target_word_count** to ContentSettings model
3. **Update prompts** to include word count
4. **Remove monthly limit checks** from services
5. **Update frontend** Usage page and CreditCostsPanel
6. **Migration** to remove deprecated fields

406
to-do-s/part2/dashboard_mods.md Normal file
View File

@@ -0,0 +1,406 @@
# Section 1: Dashboard - Audit & Action Plan
**Date:** December 27, 2025
**Status:** Finalized for Implementation (To be done LAST after all other sections complete)
**Scope:** Main dashboard page - metrics, workflow visualization, quick actions
---
## 1.1 Current Functionality
**Route:** `/`
**Files:** `pages/Dashboard/Home.tsx`, `components/dashboard/*`
### What Dashboard Currently Shows
- **Workflow Progress:** 6-step pipeline visualization (Sites → Keywords → Clusters → Ideas → Content → Published)
- **Quick Actions:** 5 navigation shortcuts
- **Key Metrics:** 4 cards (Keywords, Articles, Images, Completion %)
- **Credit Usage:** Monthly allowance and usage bar
- **Workflow Modules Guide:** 8 info cards explaining modules
- **Onboarding:** Site creation wizard for new users
---
## 1.2 Critical Gaps
| # | Issue | Impact | Priority |
|---|-------|--------|----------|
| 1 | No aggregated API endpoint | Performance - makes 6+ sequential API calls with 120ms delays | 🔴 Critical |
| 2 | Published content count incorrect | Data accuracy - cannot distinguish published vs draft | 🔴 Critical |
| 3 | Usage Summary is hardcoded | Misleading - shows fake "547 credits / $0.34" data | 🔴 Critical |
| 4 | Recent Activity is hardcoded | Misleading - static mock activity that never updates | 🔴 Critical |
| 5 | No real-time updates | Stale data - only refreshes on manual action | High |
---
## 1.3 Missing Professional Features
| # | Feature | Why Important | Priority |
|---|---------|---------------|----------|
| 6 | Needs Attention section | Users don't know what requires action | High |
| 7 | Recent content activity | No real list of recently created/published content | High |
| 8 | Error/warning alerts | No indication of failed syncs, low credits, config issues | High |
| 9 | Pipeline queue depth | No visibility into items waiting at each stage | Medium |
| 10 | Automation status | No run status, last run time, or items processed | Medium |
| 11 | Site health/sync status | No WordPress sync health indicator | Medium |
---
## 1.4 Workflow Issues
| # | Issue | Priority |
|---|-------|----------|
| 12 | Quick Actions don't adapt to user state | Medium - shows same 5 actions regardless of workflow stage |
| 13 | Workflow Completion % is misleading | Medium - formula doesn't reflect real content-from-keywords ratio |
| 14 | Modules Guide not dismissible | Low - 8 large cards always shown, no way to hide |
| 15 | Chart widget code exists but unused | Low - dead code, no trend visualization |
---
## 1.5 Issues to Address
### Critical (Must Fix)
| # | Issue | Action |
|---|-------|--------|
| 1 | No aggregated API endpoint | Create `/v1/dashboard/summary/` endpoint that returns all dashboard data in single call |
| 2 | Published content count incorrect | Fix query to separate published vs draft status |
| 3 | Usage Summary is hardcoded | Replace with real billing data from billingStore |
| 4 | Recent Activity is hardcoded | Implement real activity log OR remove section entirely |
### High Priority
| # | Issue | Action |
|---|-------|--------|
| 5 | No real-time updates | Add polling or websocket for live updates |
| 6 | No "Needs Attention" section | Add widget showing: pending reviews, failed syncs, low credits, incomplete setup |
| 7 | No recent content activity | Show real list of recently created/updated content |
| 8 | No error/warning alerts | Display alerts for: failed WordPress syncs, approaching limits, config issues |
### Medium Priority
| # | Issue | Action |
|---|-------|--------|
| 9 | No pipeline queue depth | Show count of items at each workflow stage |
| 10 | No automation status | Display: last run time, items processed, next scheduled run |
| 11 | No site health indicator | Show WordPress connection status per site |
| 12 | Quick Actions don't adapt | Make contextual based on user's workflow state |
| 13 | Completion % misleading | Revise formula to reflect actual workflow progress |
### Low Priority
| # | Issue | Action |
|---|-------|--------|
| 14 | Modules Guide not dismissible | Add dismiss/hide option, remember preference |
| 15 | Dead chart widget code | Either implement trend visualization or remove dead code |
---
## 1.6 Dashboard Revamp Specification
### New Dashboard Layout
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ DASHBOARD │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ NEEDS ATTENTION (if any) │ │
│ │ ⚠ 3 content pieces pending review [View →] │ │
│ │ ⚠ WordPress sync failed for Site A [Retry] [Fix] │ │
│ │ ⚠ Credit usage at 85% [Upgrade →] │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Keywords │ │ Articles │ │ Images │ │ Completion │ │
│ │ 156 │ │ 43 │ │ 127 │ │ 68% │ │
│ │ +12 this mo │ │ +8 this mo │ │ +24 this mo │ │ ↑ from 52% │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ WORKFLOW PIPELINE │ │
│ │ Sites(2) → Keywords(156) → Clusters(23) → Ideas(67) → Content(43) → Published(38) │
│ │ ✓ ✓ ✓ 12 pending 5 pending │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────┐ ┌────────────────────────────────────┐ │
│ │ CONTENT USAGE │ │ QUICK ACTIONS │ │
│ │ ████████████░░░░ 68/100 │ │ [+ Add Keywords] │ │
│ │ 68 used this month │ │ [Generate Content] ← contextual │ │
│ │ Resets in 12 days │ │ [Review Drafts] (5) │ │
│ └────────────────────────────┘ │ [View Published] │ │
│ └────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ RECENT ACTIVITY │ │
│ │ • Generated 5 articles from "Product Reviews" cluster 2 hours ago │ │
│ │ • Published "Best Running Shoes 2025" to Site A 3 hours ago │ │
│ │ • Created 12 ideas from keyword clustering Yesterday │ │
│ │ • Automation run completed: 8 articles generated Yesterday │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ AUTOMATION STATUS │ │
│ │ Status: ● Active Last run: 2 hours ago Next: Tomorrow 9:00 AM │ │
│ │ Last run: Processed 15 keywords → 3 clusters → 12 ideas → 8 articles │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Dashboard Components
| Component | Purpose | Data Source |
|-----------|---------|-------------|
| Needs Attention | Actionable alerts requiring user action | Aggregated from all modules |
| Key Metrics | Overview counts | Aggregated API endpoint |
| Workflow Pipeline | Visual progress through stages | Aggregated API endpoint |
| Content Usage | Credit/content piece usage | billingStore (single source of truth) |
| Quick Actions | Contextual navigation shortcuts | Based on workflow state |
| Recent Activity | Real activity log | Activity tracking system |
| Automation Status | Automation module summary | Automation service |
---
## 1.7 Aggregated API Endpoint
### Endpoint: `GET /v1/dashboard/summary/`
**Response Structure:**
```json
{
"metrics": {
"keywords_total": 156,
"keywords_this_month": 12,
"articles_total": 43,
"articles_this_month": 8,
"articles_draft": 5,
"articles_published": 38,
"images_total": 127,
"images_this_month": 24,
"completion_percentage": 68
},
"pipeline": {
"sites": 2,
"keywords": 156,
"clusters": 23,
"ideas": 67,
"ideas_pending": 12,
"content": 43,
"content_pending": 5,
"published": 38
},
"usage": {
"credits_used": 68,
"credits_total": 100,
"reset_days": 12
},
"alerts": [
{
"type": "pending_review",
"message": "3 content pieces pending review",
"action_url": "/writer/review",
"action_label": "View"
},
{
"type": "sync_failed",
"message": "WordPress sync failed for Site A",
"action_url": "/sites/1/settings",
"action_label": "Fix"
}
],
"recent_activity": [
{
"type": "content_generated",
"message": "Generated 5 articles from 'Product Reviews' cluster",
"timestamp": "2025-12-27T10:30:00Z"
}
],
"automation": {
"status": "active",
"last_run": "2025-12-27T08:00:00Z",
"next_run": "2025-12-28T09:00:00Z",
"last_run_summary": "15 keywords → 3 clusters → 12 ideas → 8 articles"
}
}
```
---
## 1.8 "Needs Attention" Widget Specification
### Alert Types
| Type | Trigger | Message | Action |
|------|---------|---------|--------|
| `pending_review` | Content in review status > 0 | "X content pieces pending review" | Link to Writer > Review |
| `sync_failed` | WordPress sync error | "WordPress sync failed for [Site]" | Link to Site Settings |
| `low_credits` | Usage > 80% | "Credit usage at X%" | Link to Upgrade |
| `credits_exhausted` | Usage = 100% | "Monthly content limit reached" | Link to Upgrade |
| `setup_incomplete` | Setup checklist incomplete | "Complete your setup" | Link to incomplete item |
| `automation_failed` | Last automation run had errors | "Automation encountered errors" | Link to Automation |
### Display Rules
- Show maximum 3-4 alerts at a time
- Prioritize by severity: errors > warnings > info
- Dismissible alerts (remember dismissal for session)
- Empty state: Hide section entirely when no alerts
---
## 1.9 Quick Actions - Contextual Logic
### Logic for Displaying Actions
```
IF no sites exist:
→ "Create Your First Site"
ELSE IF no keywords:
→ "Add Keywords"
ELSE IF keywords but no clusters:
→ "Cluster Keywords"
ELSE IF clusters but no ideas:
→ "Generate Ideas"
ELSE IF ideas but no content:
→ "Generate Content"
ELSE IF content in draft:
→ "Review Drafts (X)"
ELSE IF content in review:
→ "Publish Content (X)"
ALWAYS show:
→ "View Published" (if any published)
→ "Run Automation" (if automation configured)
```
### Default Actions (Always Available)
- Add Keywords
- View Planner
- View Writer
- Settings
---
## 1.10 Implementation Notes
### Data Flow
1. **On Dashboard Load:**
- Single API call to `/v1/dashboard/summary/`
- Populate all widgets from response
- No sequential calls, no delays
2. **Real-time Updates (Optional):**
- Poll every 60 seconds for updates
- OR use WebSocket for push updates
- Update only changed data
3. **Credit Usage:**
- Use billingStore as single source of truth
- Dashboard reads from store, doesn't fetch separately
### Performance Requirements
- Dashboard should load in < 2 seconds
- Single API call instead of 6+ sequential calls
- Lazy load Modules Guide (below fold)
- Cache dashboard data for 30 seconds
---
## Summary
### Total Issues
| Category | Critical | High | Medium | Low | Total |
|----------|----------|------|--------|-----|-------|
| Data Issues | 4 | 0 | 0 | 0 | 4 |
| Missing Features | 0 | 4 | 4 | 0 | 8 |
| UX Issues | 0 | 0 | 2 | 2 | 4 |
| **TOTAL** | **4** | **4** | **6** | **2** | **16** |
---
### Critical Items (Must Fix)
1. Create aggregated dashboard API endpoint
2. Fix published content count (distinguish published vs draft)
3. Replace hardcoded usage summary with real data
4. Replace hardcoded recent activity with real data OR remove
---
### High Priority Items
1. Add "Needs Attention" widget
2. Implement real recent activity log
3. Add error/warning alerts display
4. Add real-time or polling updates
---
### Medium Priority Items
1. Show pipeline queue depth (items at each stage)
2. Add automation status display
3. Add site health/sync indicator
4. Make Quick Actions contextual
5. Fix completion percentage formula
6. Integrate Setup Completion Checklist
---
### Files to Create
| File | Purpose |
|------|---------|
| `components/dashboard/NeedsAttention.tsx` | Alerts widget |
| `components/dashboard/AutomationStatus.tsx` | Automation summary widget |
| `components/dashboard/RecentActivity.tsx` | Real activity log |
| Backend: `api/views/dashboard.py` | Aggregated endpoint |
---
### Files to Modify
| File | Changes |
|------|---------|
| `pages/Dashboard/Home.tsx` | Complete revamp with new layout |
| `components/dashboard/CreditBalanceWidget.tsx` | Connect to billingStore single source |
| `components/dashboard/WorkflowProgress.tsx` | Add queue depth indicators |
| `components/dashboard/QuickActions.tsx` | Add contextual logic |
| `components/dashboard/KeyMetrics.tsx` | Connect to aggregated API |
---
### Backend Work Required
| Area | Work |
|------|------|
| Aggregated API | Create `/v1/dashboard/summary/` endpoint |
| Activity Logging | Ensure all actions are logged for activity feed |
| Alert Aggregation | Collect alerts from all modules |
| Pipeline Counts | Query for items at each workflow stage |
---
### Dependencies
**Dashboard should be implemented LAST because it depends on:**
- Section 2 (SETUP): Setup Completion Checklist
- Section 3 (WORKFLOW): Pipeline data, activity logging
- Section 4 (ACCOUNT): Credit balance single source of truth
- Section 6 (Navigation): Consistent navigation patterns
---