salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enhance architecture plan with new site integration model and credit-based usage system

Browse Source 
- Introduced a new Site Integration API structure to support multiple platforms (WordPress, Shopify).
- Added credit-based usage system, removing previous plan limits and implementing a credit cost structure for various operations.
- Updated Site model to include new fields for integration and hosting types.
- Enhanced publishing workflow to support multi-destination publishing and integration adapters.
This commit is contained in:
IGNY8 VPS (Salman)
2025-11-16 16:31:24 +00:00
parent bca5229c61
commit 4561f73afb
3 changed files with 982 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

296
IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md
View File

@@ -187,6 +187,10 @@ backend/igny8_core/
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── integration/ # Site Integration API (NEW)
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── automation/ # Automation API (NEW)
│ │ ├── views.py
│ │ ├── serializers.py
@@ -216,7 +220,8 @@ backend/igny8_core/
│ │
│ ├── external/ # External integrations
│ │ ├── wordpress.py # WordPress client
│ │ └── sites_renderer.py # Sites renderer client (NEW)
│ │ ├── sites_renderer.py # Sites renderer client (NEW)
│ │ └── shopify.py # Shopify client (Future)
│ │
│ ├── storage/ # Storage abstractions
│ │ └── file_storage.py
@@ -250,10 +255,11 @@ backend/igny8_core/
| `ContentIdeas` | `modules/planner/models.py` | `domain/planning/models.py` | Add: `entity_type` support |
| `InternalLinks` | - | `domain/linking/models.py` | NEW: `source_id`, `target_id`, `anchor`, `position`, `link_type` |
| `OptimizationTask` | - | `domain/optimization/models.py` | NEW: `content_id`, `type`, `target_keyword`, `scores_before`, `scores_after`, `html_before`, `html_after` |
| `SiteBlueprint` | - | `domain/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status` |
| `SiteBlueprint` | - | `domain/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status`, `hosting_type` |
| `PageBlueprint` | - | `domain/site_building/models.py` | NEW: `site_blueprint`, `slug`, `type`, `blocks_json`, `status` |
| `PublishingRecord` | - | `domain/publishing/models.py` | NEW: `content_id`, `destination`, `status`, `external_id`, `published_at` |
| `DeploymentRecord` | - | `domain/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at` |
| `SiteIntegration` | - | `domain/integration/models.py` | NEW: `site`, `platform`, `platform_type`, `config_json`, `credentials`, `is_active`, `sync_enabled` |
| `PublishingRecord` | - | `domain/publishing/models.py` | NEW: `content_id`, `destination`, `destination_type`, `status`, `external_id`, `published_at`, `sync_status` |
| `DeploymentRecord` | - | `domain/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at`, `deployment_type` |
| `AutomationRule` | - | `domain/automation/models.py` | NEW: `name`, `trigger`, `conditions`, `actions`, `schedule`, `is_active` |
| `ScheduledTask` | - | `domain/automation/models.py` | NEW: `automation_rule`, `scheduled_at`, `status`, `executed_at` |
@@ -666,35 +672,94 @@ docker-data/
---
## 10. LIMITS & BOUNDARIES
## 10. CREDIT-BASED USAGE SYSTEM
### 10.1 Plan Limits (Existing)
### 10.1 Core Principle
| Limit | Model Field | Enforcement Location | Per |
|-------|-------------|---------------------|-----|
| **Max Keywords** | `Plan.max_keywords` | `domain/planning/services/clustering_service.py` | Account |
| **Max Clusters** | `Plan.max_clusters` | `domain/planning/services/clustering_service.py` | Account |
| **Max Content Ideas** | `Plan.max_content_ideas` | `domain/planning/services/ideas_service.py` | Account |
| **Daily Content Tasks** | `Plan.daily_content_tasks` | `domain/content/services/content_generation_service.py` | Site |
| **Daily AI Requests** | `Plan.daily_ai_requests` | `infrastructure/ai/engine.py` | Site |
| **Monthly Word Count** | `Plan.monthly_word_count_limit` | `domain/content/services/content_generation_service.py` | Site |
| **Daily Image Generation** | `Plan.daily_image_generation_limit` | `infrastructure/ai/functions/generate_images.py` | Site |
| **Monthly Image Count** | `Plan.monthly_image_count` | `infrastructure/ai/functions/generate_images.py` | Site |
| **Max Sites** | `Plan.max_sites` | `core/auth/models.py` (Account.save) | Account |
| **Max Users** | `Plan.max_users` | `core/auth/models.py` (Account.save) | Account |
**Credit-Only Model**: All features are unlimited. Only credits restrict usage.
### 10.2 New Limits for Phase 2
- ✅ All features unlocked for paid users
- ✅ No plan-based limits (keywords, clusters, tasks, etc.)
- ✅ Only credits control usage
- ✅ Operational limits only (50 keywords/request, 6 images/request)
| Limit | Model Field | Enforcement Location | Per |
|-------|-------------|---------------------|-----|
| **Max Sites Built** | `Plan.max_sites_built` (NEW) | `domain/site_building/services/` | Account |
| **Daily Site Generations** | `Plan.daily_site_generations` (NEW) | `domain/site_building/services/` | Site |
| **Storage Quota** | `Plan.storage_quota_gb` (NEW) | `infrastructure/storage/file_storage.py` | Site |
| **Max Automation Rules** | `Plan.max_automation_rules` (NEW) | `domain/automation/services/` | Account |
| **Daily Automation Executions** | `Plan.daily_automation_executions` (NEW) | `domain/automation/services/` | Site |
| **Max Concurrent Tasks** | `Plan.max_concurrent_tasks` (NEW) | Celery queue config | Site |
### 10.2 Plan Model (Simplified)
### 10.3 Admin Boundaries
| Field | Type | Purpose |
|-------|------|---------|
| `monthly_credits` | Integer | Credits added per month |
| `support_level` | CharField | Support tier (basic, priority, enterprise) |
| `billing_cycle` | CharField | 'monthly' or 'annual' |
| `price` | Decimal | Subscription price |
| `features` | JSONField | Feature flags (for future use) |
**REMOVED FIELDS** (no longer needed):
- `max_keywords`, `max_clusters`, `max_content_ideas`
- `daily_content_tasks`, `monthly_word_count_limit`
- `daily_image_generation_limit`, `monthly_image_count`
- All other usage limits
### 10.3 Credit Cost Structure
| Operation | Credit Cost | Type | Notes |
|-----------|-------------|------|-------|
| **Clustering** | 10 credits | Fixed | Per clustering request |
| **Idea Generation** | 15 credits | Fixed | Per cluster → ideas request |
| **Content Generation** | 1 credit per 100 words | Variable | Based on word count |
| **Image Generation** | 5 credits | Fixed | Per image (user sees batch) |
| **Image Prompt Extraction** | 2 credits | Fixed | Per content piece |
| **Linking** | 8 credits | Fixed | Per content piece (NEW) |
| **Optimization** | 1 credit per 200 words | Variable | Per content piece (NEW) |
| **Site Structure Generation** | 50 credits | Fixed | Per site blueprint (NEW) |
| **Site Page Generation** | 20 credits | Fixed | Per page (NEW) |
### 10.4 Credit Enforcement
| Location | Check | Implementation |
|----------|-------|----------------|
| **AI Engine** | Before AI call | `infrastructure/ai/engine.py` - Check credits, deduct before request |
| **Content Generation** | Before generation | `domain/content/services/content_generation_service.py` |
| **Image Generation** | Before generation | `infrastructure/ai/functions/generate_images.py` |
| **Linking** | Before linking | `domain/linking/services/linker_service.py` (NEW) |
| **Optimization** | Before optimization | `domain/optimization/services/optimizer_service.py` (NEW) |
| **Site Building** | Before structure gen | `domain/site_building/services/structure_generation_service.py` (NEW) |
### 10.5 Credit Logging
| Log Type | Fields | Purpose |
|----------|--------|---------|
| **CreditUsageLog** | `account`, `operation_type`, `credits_used`, `related_object_type`, `related_object_id`, `metadata` | Track all credit consumption |
| **CreditTransaction** | `account`, `transaction_type`, `amount`, `description` | Track credit additions/deductions |
**Usage Tracking**:
- Total credits used (account level)
- Credits per operation type (clustering, content, images, etc.)
- Credits per site (site level breakdown)
- Credits per user (user level breakdown)
- Historical usage trends
### 10.6 Operational Limits (Technical Constraints)
| Limit | Value | Type | Reason |
|-------|-------|-------|--------|
| **Keywords per request** | 50 | Technical | API payload size, processing time |
| **Images per request** | 6 | Technical | Queue management (user sees as batch) |
| **Images per AI call** | 1 | Technical | Image API limitation (internal) |
These are **NOT** business limits - they're technical constraints for request processing.
### 10.7 Credit System Features
| Feature | Implementation | Location |
|---------|----------------|----------|
| **Credit Check** | Before every AI operation | `domain/billing/services/credit_service.py` |
| **Credit Deduction** | After successful operation | `domain/billing/services/credit_service.py` |
| **Credit Top-up** | On-demand purchase | `modules/billing/views.py` |
| **Monthly Replenishment** | Celery Beat task | `infrastructure/messaging/automation_tasks.py` |
| **Low Credit Warning** | When < 10% remaining | Frontend + Email notification |
| **Usage Dashboard** | Show credits by operation type | `frontend/src/pages/Billing/Usage.tsx` |
### 10.8 Admin Boundaries
| Boundary | Implementation | Location |
|----------|----------------|----------|
@@ -705,7 +770,7 @@ docker-data/
| **Credit Management** | Admin-only actions | `modules/billing/views.py` |
| **Automation Management** | Role-based (Owner/Admin) | `modules/automation/views.py` |
### 10.4 Multi-Tenancy Boundaries
### 10.9 Multi-Tenancy Boundaries
| Boundary | Implementation | Location |
|----------|----------------|----------|
@@ -718,6 +783,179 @@ docker-data/
---
## 11. SITE INTEGRATION & MULTI-DESTINATION PUBLISHING
### 11.1 Site Integration Model
| Field | Type | Purpose |
|-------|------|---------|
| `site` | ForeignKey(Site) | Links to Site model |
| `platform` | CharField | Platform name ('wordpress', 'shopify', 'custom') |
| `platform_type` | CharField | Type: 'cms', 'ecommerce', 'custom_api' |
| `config_json` | JSONField | Platform-specific configuration |
| `credentials` | EncryptedField | Encrypted API keys, tokens, passwords |
| `is_active` | Boolean | Enable/disable integration |
| `sync_enabled` | Boolean | Enable two-way sync |
| `last_sync_at` | DateTime | Last successful sync |
| `sync_status` | CharField | 'success', 'failed', 'pending' |
**Purpose**:
- Sites can be NEW (built via Site Builder) or EXISTING (already integrated)
- One site can have multiple integrations (WordPress + Shopify)
- Integration credentials stored securely
- Two-way sync support
### 11.2 Site Types & Hosting
| Site Type | Description | Integration | Hosting |
|-----------|-------------|-------------|---------|
| **NEW Site (IGNY8 Built)** | Created via Site Builder | None initially | IGNY8 Sites renderer |
| **EXISTING WordPress** | Already has WordPress site | WordPress integration | WordPress hosting |
| **EXISTING Shopify** | Already has Shopify store | Shopify integration | Shopify hosting |
| **EXISTING Custom** | Custom platform/API | Custom integration | External hosting |
| **Hybrid** | Built in IGNY8, also integrated | Multiple integrations | IGNY8 + External |
### 11.3 Site Builder Structure
```
Site Builder Flow:
1. User creates SiteBlueprint (via wizard)
2. SiteBlueprint.hosting_type = 'igny8_sites' | 'wordpress' | 'shopify' | 'multi'
3. If 'wordpress' or 'shopify':
- User must configure SiteIntegration first
- Site Builder generates content compatible with platform
4. If 'igny8_sites':
- Site deployed to Sites renderer
5. If 'multi':
- Site can publish to multiple destinations
```
### 11.4 Multi-Destination Publishing
| Content Type | Can Publish To | Implementation |
|--------------|----------------|----------------|
| **Blog Posts** | WordPress, IGNY8 Sites, Shopify Blog | PublisherService routes to adapters |
| **Pages** | WordPress, IGNY8 Sites, Shopify Pages | PublisherService routes to adapters |
| **Products** | Shopify, WooCommerce (WordPress), IGNY8 Sites | Product adapter per platform |
| **Categories** | WordPress, Shopify Collections, IGNY8 Sites | Taxonomy adapter per platform |
| **Site Blueprint** | IGNY8 Sites, WordPress (via plugin), Shopify (future) | DeploymentService routes to adapters |
**Publishing Flow:**
```python
# Single content can publish to multiple destinations
publisher_service.publish(
content=content,
destinations=['wordpress', 'igny8_sites', 'shopify'] # Multiple destinations
)
```
### 11.5 Integration Adapters
| Adapter | Platform | Purpose | Implementation |
|---------|----------|---------|-----------------|
| **WordPressAdapter** | WordPress | Publish content, sync status | Uses WordPress REST API |
| **SitesRendererAdapter** | IGNY8 Sites | Deploy site blueprints | Uploads to Sites container |
| **ShopifyAdapter** | Shopify | Publish products, pages, blog | Uses Shopify Admin API |
| **BaseAdapter** | Abstract | Common interface | All adapters inherit |
### 11.6 Site Integration Service
```python
# domain/integration/services/integration_service.py
class IntegrationService:
def create_integration(self, site, platform, config, credentials):
"""Create new site integration"""
def test_integration(self, integration):
"""Test integration connection"""
def sync_site_data(self, integration, sync_type='full'):
"""Sync site data from external platform"""
# Full: All posts/products
# Incremental: Only changes since last sync
def get_available_platforms(self):
"""Get list of supported platforms"""
return ['wordpress', 'shopify', 'custom_api']
```
### 11.7 Site Model Extensions
| Field | Type | Purpose |
|-------|------|---------|
| `site_type` | CharField | 'new', 'existing', 'hybrid' |
| `hosting_type` | CharField | 'igny8_sites', 'wordpress', 'shopify', 'custom', 'multi' |
| `integrations` | ManyToMany(SiteIntegration) | Multiple integrations per site |
**Current Site Model** (needs extension):
- Already has: `wp_url`, `wp_username`, `wp_app_password` (WordPress specific)
- **NEW**: Move to `SiteIntegration` model for flexibility
- **NEW**: Add `site_type` and `hosting_type` fields
### 11.8 Publishing Workflow
```
Content/Site Publishing Flow:
1. User selects content/site to publish
2. System checks Site.integrations (available destinations)
3. User selects destinations (can be multiple)
4. PublisherService routes to appropriate adapters:
- WordPressAdapter.publish()
- SitesRendererAdapter.deploy()
- ShopifyAdapter.publish()
5. PublishingRecord created for each destination
6. Status tracked per destination
7. Two-way sync enabled if integration.sync_enabled = True
```
### 11.9 Implementation Files
| Component | File | Purpose |
|-----------|------|---------|
| **SiteIntegration Model** | `domain/integration/models.py` | Store integration configs |
| **IntegrationService** | `domain/integration/services/integration_service.py` | Manage integrations |
| **SyncService** | `domain/integration/services/sync_service.py` | Handle two-way sync |
| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | WordPress publishing |
| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | IGNY8 Sites deployment |
| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Shopify publishing (future) |
| **Integration API** | `modules/integration/views.py` | CRUD for integrations |
| **Integration UI** | `frontend/src/pages/Settings/Integrations.tsx` | Manage integrations |
### 11.10 Site Builder Integration Flow
```
Site Builder → Publishing Flow:
1. User builds site via Site Builder wizard
2. SiteBlueprint created with hosting_type
3. If hosting_type = 'wordpress' or 'shopify':
a. User must configure SiteIntegration
b. Site Builder generates platform-compatible structure
4. User clicks "Deploy"
5. DeploymentService:
- If 'igny8_sites': Deploy to Sites renderer
- If 'wordpress': Publish via WordPressAdapter
- If 'shopify': Publish via ShopifyAdapter
- If 'multi': Publish to all configured destinations
6. DeploymentRecord created
7. Site available on selected platforms
```
### 11.11 Content Type Support Per Platform
| Content Type | WordPress | IGNY8 Sites | Shopify | Notes |
|--------------|-----------|-------------|---------|-------|
| **Blog Posts** | ✅ | ✅ | ✅ (Blog) | All platforms support |
| **Pages** | ✅ | ✅ | ✅ (Pages) | All platforms support |
| **Products** | ✅ (WooCommerce) | ✅ | ✅ | Product adapter needed |
| **Product Categories** | ✅ (WooCommerce) | ✅ | ✅ (Collections) | Taxonomy mapping |
| **Product Attributes** | ✅ (WooCommerce) | ✅ | ✅ (Variants) | Attribute mapping |
| **Service Pages** | ✅ | ✅ | ❌ | Custom content type |
| **Taxonomy Terms** | ✅ | ✅ | ❌ | Blog-specific |
---
## SUMMARY
### Key Architectural Decisions