igny8/v2/V2-Execution-Docs/01G-sag-health-monitoring.md

# IGNY8 Phase 1: SAG Health Monitoring (Doc 01G)

> **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14

**Document ID:** 01G
**Module:** SAG Health Monitoring
**Phase:** Phase 1 - Core Implementation
**Last Updated:** 2026-03-23
**Status:** Build Specification

---

## 1. Current State

### Existing Infrastructure
- **SAGBlueprint model** (01A) includes `sag_health_score` and `last_health_check` fields (nullable)
- **Cluster model** (01C) tracks `status` (planned/in-progress/complete) and content completion % by phase
- **Blueprint wizard** (01D) creates initial blueprint structure
- **Gap analysis pipeline** (01E) identifies content gaps by cluster
- **Attribute/taxonomy system** (01F) maps products and content to taxonomy terms

### Current Gaps
- No health score calculation logic
- No automated health monitoring or weekly checks
- No evolution trigger detection system
- No health trend tracking/history
- No dashboard health widget
- No blueprint versioning system
- No recommendation generation engine

### Dependencies
- Celery + Celery Beat (for weekly scheduling)
- Existing SAGBlueprint, Cluster, Product, BlogPost, HubPage models
- Existing attribute/taxonomy system
- Existing internal linking structure

---

## 2. What to Build

### 2.1 Health Score Calculation Engine
**File:** `sag/services/health_service.py`
**Primary Function:** `calculate_health_score(blueprint) → Float`

Comprehensive scoring system (0-100) with 5 weighted components:

#### Component 1: Content Completeness (30 points max)
- **Hub Pages (10 points):** Score = (published_hubs / defined_hubs) × 10
  - Count hub pages with status="published" for blueprint
  - Divide by expected hubs (one per hub cluster type)

- **Supporting Blog Content (10 points):** Score = max(0, (published_blogs / recommended_blogs) × 10) if ratio ≥ 0.6
  - Count blog posts linked to blueprint clusters
  - Expected recommendation = clusters.count × target_blogs_per_cluster (e.g., 3)
  - Award 10 points only if ≥60% of recommended published

- **Attribute Term Pages (10 points):** Score = max(0, (attributes_with_pages / total_attributes) × 10) if ratio ≥ 0.75
  - Count attributes with dedicated term pages (status="published")
  - Award 10 points only if ≥75% coverage
  - 0 points if below threshold

#### Component 2: Cluster Coverage (25 points max)
- **Baseline Score:** 5 points per cluster in "complete" status (up to 5 clusters = 25 points)
  - Cluster is "complete" when all planned phases have content deliverables published

- **Partial Completion Penalty:** For clusters in "in-progress", apply proportional score
  - Score = (completed_phases / planned_phases) × 5

- **Planned Clusters Penalty:** Clusters in "planned" status = 0 points, potential -5 penalty if > 50% of clusters are unstarted

#### Component 3: Internal Linking (20 points max)
- **Hub-to-Category Links (5 points):** All hub pages linked from respective category landing pages
  - Check: hub.linked_from_category_page == True
  - Score = (linked_hubs / total_hubs) × 5

- **Blog-Hub Bidirectional Links (5 points):** Blog posts link to hubs AND hubs reference blogs
  - Count blogs linked to/from hubs for the blueprint
  - Score = (linked_blogs / total_blogs) × 5 (minimum 50% to earn points)

- **Attribute Term Page Interlinks (5 points):** Attribute pages link to related pages
  - Check internal link graph for attribute term pages
  - Score = (interlinked_attributes / total_attributes) × 5 (minimum 70% to earn points)

- **Navigation Consistency (5 points):** Breadcrumb trails and navigation menus consistent across cluster pages
  - Audit: all pages in cluster have proper breadcrumb structure
  - Score = 5 if ≥90% of pages consistent, else (consistency_ratio × 5)

#### Component 4: Taxonomy Alignment (15 points max)
- **Primary Attribute Tagging (5 points):** Products tagged with primary (main) attributes
  - Score = (products_with_primary_attr / total_products) × 5 (minimum 70% to earn points)

- **Secondary Attribute Content Tags (5 points):** Content tagged with secondary attributes
  - Count blog posts and pages with secondary attribute tags
  - Score = (tagged_content / total_content) × 5 (minimum 50% to earn points)

- **Taxonomy Term Coverage (5 points):** All defined taxonomy terms have associated content
  - Score = (terms_with_content / total_terms) × 5

- **Orphaned Content Penalty:** -5 points per cluster if > 10% of content is untagged
  - Affects final score but never goes below 0

#### Component 5: Keyword Coverage (10 points max)
- **Keyword-to-Content Mapping (7 points):** Keywords have published content in target clusters
  - Count keywords with ≥1 published content piece
  - Score = (keywords_with_content / total_keywords) × 7 (minimum 50% to earn points)

- **Keyword Cluster Distribution (3 points):** Cluster has average 2-3 keywords minimum
  - Calculate avg keywords per cluster: total_keywords / cluster_count
  - 3 points if avg ≥ 2.5
  - 1.5 points if avg ≥ 2.0
  - 0 points if avg < 2.0

### 2.2 Score Interpretation Guide

| Score Range | Status | Meaning |
|---|---|---|
| 80-100 | Excellent | Blueprint well-implemented, comprehensive content coverage |
| 60-79 | Good | Core structure complete, some gaps in content or linking |
| 40-59 | Fair | Partial implementation, significant gaps remain |
| 0-39 | Poor | Blueprint not yet implemented, foundational work needed |

---

## 3. Data Models & APIs

### 3.1 Data Models

#### SAGHealthSnapshot (New Model)
```python
class SAGHealthSnapshot(models.Model):
    """Weekly health check snapshot for trend tracking."""

    blueprint = models.ForeignKey(SAGBlueprint, on_delete=models.CASCADE, related_name='health_snapshots')
    score = models.FloatField()  # 0-100

    # Component breakdown
    completeness_score = models.FloatField()  # 0-30
    cluster_coverage_score = models.FloatField()  # 0-25
    internal_linking_score = models.FloatField()  # 0-20
    taxonomy_alignment_score = models.FloatField()  # 0-15
    keyword_coverage_score = models.FloatField()  # 0-10

    # Cluster status snapshot
    cluster_statuses = models.JSONField(
        default=dict,
        help_text="{ 'complete': count, 'in_progress': count, 'planned': count }"
    )

    # Issues/recommendations at time of check
    recommendations = models.JSONField(
        default=list,
        help_text="[{ 'type': 'create_hub', 'cluster_id': X, 'priority': 'high' }, ...]"
    )

    # Health alerts
    alerts = models.JSONField(
        default=list,
        help_text="[{ 'cluster_id': X, 'issue': 'unstarted', 'severity': 'high' }, ...]"
    )

    created_at = models.DateTimeField(auto_now_add=True)

    class Meta:
        ordering = ['-created_at']
        indexes = [
            models.Index(fields=['blueprint', '-created_at']),
        ]

    def __str__(self):
        return f"Health snapshot for {self.blueprint.name} - {self.score}/100 ({self.created_at.date()})"
```

#### Update SAGBlueprint Model
```python
# Add to existing SAGBlueprint model:
sag_health_score = models.FloatField(null=True, blank=True, default=None)
last_health_check = models.DateTimeField(null=True, blank=True)
blueprint_version = models.PositiveIntegerField(default=1)
version_status = models.CharField(
    max_length=20,
    choices=[('active', 'Active'), ('archived', 'Archived'), ('draft', 'Draft')],
    default='active'
)
parent_version = models.ForeignKey(
    'self',
    null=True,
    blank=True,
    on_delete=models.SET_NULL,
    related_name='child_versions',
    help_text="Links to previous version if this is a new version"
)
```

### 3.2 API Endpoints

#### Endpoint 1: Trigger Health Check
```
POST /api/v1/sag/blueprints/{id}/health-check/

Request Body:
{
  "force_recalculate": true  # Optional, bypasses cache
}

Response:
{
  "status": "success",
  "blueprint_id": 123,
  "health_score": 72.5,
  "component_scores": {
    "completeness": 25,
    "cluster_coverage": 18,
    "internal_linking": 16,
    "taxonomy_alignment": 12,
    "keyword_coverage": 1.5
  },
  "cluster_statuses": {
    "complete": 2,
    "in_progress": 3,
    "planned": 1
  },
  "recommendations": [
    { "type": "create_hub", "cluster_id": 5, "priority": "high" },
    { "type": "create_attribute_pages", "count": 8, "priority": "medium" },
    ...
  ],
  "checked_at": "2026-03-23T10:30:00Z"
}
```

#### Endpoint 2: Get Health History (Trend Data)
```
GET /api/v1/sag/blueprints/{id}/health-history/?weeks=4

Query Parameters:
  weeks: 1-52 (default: 4)

Response:
{
  "blueprint_id": 123,
  "trend_data": [
    {
      "week": "2026-03-09",
      "score": 68.2,
      "component_scores": { ... },
      "cluster_statuses": { ... }
    },
    {
      "week": "2026-03-16",
      "score": 70.1,
      "component_scores": { ... },
      "cluster_statuses": { ... }
    },
    {
      "week": "2026-03-23",
      "score": 72.5,
      "component_scores": { ... },
      "cluster_statuses": { ... }
    }
  ],
  "current_score": 72.5,
  "trend": "improving",  # "improving" / "stable" / "declining"
  "trend_percent": 6.3  # % change from first to last
}
```

#### Endpoint 3: Get Recommendations
```
GET /api/v1/sag/blueprints/{id}/recommendations/?priority=high

Query Parameters:
  priority: all | high | medium | low (default: all)

Response:
{
  "blueprint_id": 123,
  "total_recommendations": 8,
  "recommendations": [
    {
      "id": "rec_001",
      "type": "create_hub",
      "cluster_id": 5,
      "cluster_name": "Cluster Name",
      "description": "Create hub page for 'Cluster Name' cluster",
      "priority": "high",
      "impact_score": 10,
      "estimated_effort": "2-4 hours",
      "action_url": "/dashboard/blueprint/123/cluster/5/create-hub"
    },
    {
      "id": "rec_002",
      "type": "create_attribute_pages",
      "count": 8,
      "attributes": ["Attr1", "Attr2", ...],
      "description": "Create term pages for 8 attributes missing pages",
      "priority": "medium",
      "impact_score": 10,
      "estimated_effort": "4-6 hours"
    },
    ...
  ]
}
```

#### Endpoint 4: Trigger Evolution Check
```
POST /api/v1/sag/blueprints/{id}/evolve/

Request Body: {} (no required fields)

Response:
{
  "blueprint_id": 123,
  "evolution_detected": true,
  "triggers": [
    {
      "type": "new_product_category",
      "description": "8 new products detected in category not in blueprint",
      "data": {
        "category": "New Category",
        "product_count": 8,
        "suggested_cluster": "New Cluster Name",
        "existing_attributes": ["Attr1", "Attr2"]
      }
    },
    {
      "type": "keyword_gap",
      "description": "5 new keywords added not mapping to any cluster",
      "data": {
        "keywords": ["kw1", "kw2", "kw3", "kw4", "kw5"],
        "suggestions": ["new_cluster" | "existing_cluster_X" | "orphan"]
      }
    },
    {
      "type": "new_sector",
      "description": "New sector 'SectorName' added",
      "data": {
        "sector_name": "SectorName",
        "new_attributes": ["AttrX", "AttrY", "AttrZ"],
        "merge_impact": "8 new clusters likely"
      }
    }
  ],
  "recommendation": "Consider updating blueprint to v2",
  "suggested_version_name": "Blueprint v2 (with NewSector)",
  "user_action_required": true
}
```

#### Endpoint 5: Create New Blueprint Version
```
POST /api/v1/sag/blueprints/{id}/create-version/

Request Body:
{
  "from_evolution_triggers": true,  # Use evolution suggestions
  "trigger_ids": ["trigger_001", "trigger_002"],  # Which triggers to include
  "version_name": "Blueprint v2",
  "description": "Updated with new sector and product categories"
}

Response:
{
  "status": "success",
  "new_blueprint_version": 2,
  "new_blueprint_id": 124,
  "version_status": "draft",
  "message": "Blueprint v2 created as draft. Review and confirm to activate.",
  "review_url": "/dashboard/blueprint/124/review-version",
  "changes": {
    "clusters_added": 2,
    "clusters_modified": 3,
    "clusters_unchanged": 5,
    "new_attributes": 4
  }
}
```

#### Endpoint 6: List Blueprint Versions
```
GET /api/v1/sag/blueprints/{id}/versions/

Response:
{
  "blueprint_name": "Main Blueprint",
  "total_versions": 3,
  "versions": [
    {
      "version_number": 2,
      "status": "active",
      "blueprint_id": 124,
      "created_at": "2026-03-20T14:00:00Z",
      "activated_at": "2026-03-20T15:30:00Z",
      "created_by": "User Name",
      "description": "Updated with new sector",
      "cluster_count": 10,
      "health_score": 74.2
    },
    {
      "version_number": 1,
      "status": "archived",
      "blueprint_id": 123,
      "created_at": "2026-02-15T10:00:00Z",
      "activated_at": "2026-02-15T10:30:00Z",
      "archived_at": "2026-03-20T15:30:00Z",
      "created_by": "User Name",
      "description": "Initial blueprint",
      "cluster_count": 8,
      "health_score": 68.5
    }
  ]
}
```

---

## 4. Implementation Steps

### Phase 1: Health Score Calculation (Week 1-2)

#### Step 1.1: Create Health Service Module
**File:** `sag/services/health_service.py`

Create a comprehensive HealthScoreCalculator class:

```python
class HealthScoreCalculator:
    """Calculates SAG health score across 5 components."""

    def __init__(self, blueprint):
        self.blueprint = blueprint
        self.clusters = blueprint.clusters.all()
        self.products = blueprint.site.products.all()
        self.blogs = blueprint.site.blog_posts.all()
        self.hubs = blueprint.site.hub_pages.filter(blueprint=blueprint)
        self.attributes = blueprint.site.attributes.all()
        self.keywords = blueprint.keywords.all()

    def calculate_health_score(self):
        """Main method returning total score 0-100."""
        completeness = self.calculate_completeness()
        cluster_coverage = self.calculate_cluster_coverage()
        internal_linking = self.calculate_internal_linking()
        taxonomy_alignment = self.calculate_taxonomy_alignment()
        keyword_coverage = self.calculate_keyword_coverage()

        total = completeness + cluster_coverage + internal_linking + taxonomy_alignment + keyword_coverage

        return {
            'total_score': min(100, total),
            'components': {
                'completeness': completeness,
                'cluster_coverage': cluster_coverage,
                'internal_linking': internal_linking,
                'taxonomy_alignment': taxonomy_alignment,
                'keyword_coverage': keyword_coverage
            },
            'interpretation': self.interpret_score(total)
        }

    def calculate_completeness(self) -> float:
        """Component 1: Content Completeness (max 30)."""
        hub_score = self._hub_page_score()
        blog_score = self._blog_content_score()
        attribute_score = self._attribute_page_score()
        return hub_score + blog_score + attribute_score

    def calculate_cluster_coverage(self) -> float:
        """Component 2: Cluster Coverage (max 25)."""
        # Implementation details...
        pass

    def calculate_internal_linking(self) -> float:
        """Component 3: Internal Linking (max 20)."""
        # Implementation details...
        pass

    def calculate_taxonomy_alignment(self) -> float:
        """Component 4: Taxonomy Alignment (max 15)."""
        # Implementation details...
        pass

    def calculate_keyword_coverage(self) -> float:
        """Component 5: Keyword Coverage (max 10)."""
        # Implementation details...
        pass

    def interpret_score(self, score: float) -> str:
        """Interpret health score."""
        if score >= 80:
            return "Excellent"
        elif score >= 60:
            return "Good"
        elif score >= 40:
            return "Fair"
        else:
            return "Poor"
```

**Key Methods to Implement:**
- `_hub_page_score()` - Count published hubs vs expected
- `_blog_content_score()` - Validate ≥60% of recommended blogs published
- `_attribute_page_score()` - Validate ≥75% of attributes have term pages
- `_complete_clusters_count()` - Count clusters with all phases complete
- `_cluster_interlink_audit()` - Verify bidirectional hub-blog links
- `_taxonomy_coverage_check()` - Verify products/content tagged appropriately
- `_keyword_distribution_analysis()` - Calculate avg keywords per cluster

#### Step 1.2: Create SAGHealthSnapshot Model
**File:** `sag/models.py`

Add `SAGHealthSnapshot` model as specified in section 3.1.

#### Step 1.3: Update SAGBlueprint Model
**File:** `sag/models.py`

Add fields to SAGBlueprint:
- `sag_health_score` (FloatField)
- `last_health_check` (DateTimeField)
- `blueprint_version` (PositiveIntegerField)
- `version_status` (CharField with choices)
- `parent_version` (ForeignKey)

Create database migration.

#### Step 1.4: Create Unit Tests for Health Calculation
**File:** `tests/sag/test_health_service.py`

Test each scoring component independently:
- Test hub page scoring
- Test cluster coverage calculation
- Test internal linking validation
- Test taxonomy alignment checks
- Test keyword distribution analysis

Minimum coverage: 85% of health_service.py

#### Step 1.5: Validation & Edge Cases
Ensure robust handling:
- Empty blueprints (all scores = 0)
- Blueprints with only planned clusters
- Orphaned content (no taxonomy tags)
- Broken internal links
- Missing keyword assignments

---

### Phase 2: Weekly Automation & API Endpoints (Week 2-3)

#### Step 2.1: Create Celery Tasks
**File:** `sag/tasks.py`

```python
from celery import shared_task

@shared_task
def run_blueprint_health_check(site_id: int):
    """
    Weekly health check task.
    1. Calculate health score
    2. Identify underperforming clusters
    3. Generate recommendations
    4. Check taxonomy coverage
    5. Verify internal link integrity
    6. Store health snapshot
    7. Generate report
    """
    # Implementation...
    pass

@shared_task
def check_blueprint_evolution_triggers(site_id: int):
    """
    Check for evolution triggers.
    1. New products in undefined categories
    2. Keywords not mapping to clusters
    3. New sectors added
    4. Flag for user review
    """
    # Implementation...
    pass
```

**Task Details:**
- `run_blueprint_health_check`:
  1. Fetch active blueprint for site
  2. Call `HealthScoreCalculator.calculate_health_score()`
  3. Identify clusters with score < 60%
  4. Generate alert list (type, cluster_id, severity)
  5. Generate recommendations (create_hub, add_blogs, etc.)
  6. Query for unassigned products/pages
  7. Run internal link validation (check for 404s, broken refs)
  8. Create SAGHealthSnapshot record
  9. Generate email report (optional)

- `check_blueprint_evolution_triggers`:
  1. Query products with attribute values not in blueprint
  2. Analyze new keywords vs cluster keyword mappings
  3. Check for new site sectors
  4. Return list of evolution candidates
  5. Trigger notification to user

#### Step 2.2: Configure Celery Beat Schedule
**File:** `igny8_core/celery.py`

```python
CELERY_BEAT_SCHEDULE = {
    'weekly-blueprint-health-check': {
        'task': 'sag.tasks.run_blueprint_health_check',
        'schedule': crontab(hour=2, minute=0, day_of_week='mon'),  # Monday 2 AM UTC
        'kwargs': {'site_id': 1}  # Parametrize per site
    },
    'weekly-evolution-trigger-check': {
        'task': 'sag.tasks.check_blueprint_evolution_triggers',
        'schedule': crontab(hour=3, minute=0, day_of_week='mon'),  # Monday 3 AM UTC
        'kwargs': {'site_id': 1}
    }
}
```

#### Step 2.3: Create API Serializers
**File:** `sag/serializers.py`

```python
class HealthComponentScoresSerializer(serializers.Serializer):
    completeness = serializers.FloatField()
    cluster_coverage = serializers.FloatField()
    internal_linking = serializers.FloatField()
    taxonomy_alignment = serializers.FloatField()
    keyword_coverage = serializers.FloatField()

class HealthCheckResponseSerializer(serializers.Serializer):
    blueprint_id = serializers.IntegerField()
    health_score = serializers.FloatField()
    component_scores = HealthComponentScoresSerializer()
    cluster_statuses = serializers.JSONField()
    recommendations = serializers.JSONField()
    checked_at = serializers.DateTimeField()
```

#### Step 2.4: Create API Views
**File:** `sag/views.py`

Implement ViewSets for health endpoints:

```python
class SAGBlueprintHealthViewSet(viewsets.ViewSet):
    """Health monitoring endpoints for SAG blueprints."""

    @action(detail=True, methods=['post'])
    def health_check(self, request, pk=None):
        """Trigger health score calculation."""
        # Implementation...
        pass

    @action(detail=True, methods=['get'])
    def health_history(self, request, pk=None):
        """Get health trend data."""
        # Implementation...
        pass

    @action(detail=True, methods=['get'])
    def recommendations(self, request, pk=None):
        """Get action recommendations."""
        # Implementation...
        pass

    @action(detail=True, methods=['post'])
    def evolve(self, request, pk=None):
        """Check for evolution triggers."""
        # Implementation...
        pass

    @action(detail=True, methods=['post'])
    def create_version(self, request, pk=None):
        """Create new blueprint version from evolution."""
        # Implementation...
        pass

    @action(detail=True, methods=['get'])
    def versions(self, request, pk=None):
        """List all blueprint versions."""
        # Implementation...
        pass
```

#### Step 2.5: URL Configuration
**File:** `sag/urls.py`

```python
router = DefaultRouter()
router.register(r'blueprints', SAGBlueprintHealthViewSet, basename='blueprint-health')

urlpatterns = [
    path('api/v1/sag/', include(router.urls)),
]
```

#### Step 2.6: Create API Tests
**File:** `tests/sag/test_health_api.py`

Test all endpoint scenarios:
- POST /health-check/ returns correct score
- GET /health-history/ returns 4-week trend
- GET /recommendations/ filters by priority
- POST /evolve/ detects new categories/keywords
- POST /create-version/ creates draft blueprint
- GET /versions/ lists all versions with correct status

---

### Phase 3: Evolution Triggers & Blueprint Versioning (Week 3-4)

#### Step 3.1: Create Evolution Detection Service
**File:** `sag/services/evolution_service.py`

```python
class EvolutionDetector:
    """Detects when blueprint needs updating."""

    def detect_new_product_categories(self, blueprint):
        """Detect products with undefined attribute values."""
        # Compare product attributes to blueprint attribute set
        # Return list of new values found
        pass

    def detect_keyword_gaps(self, blueprint):
        """Detect keywords not mapping to any cluster."""
        # Compare site keywords to blueprint cluster keywords
        # Return unmatched keywords
        pass

    def detect_new_sectors(self, blueprint):
        """Detect new sectors added to multi-sector site."""
        # Check site.sectors vs blueprint.sectors
        # Return new sectors
        pass

    def generate_evolution_report(self, blueprint):
        """Compile all evolution triggers into single report."""
        # Return dict with triggers, recommendations, version bump
        pass
```

#### Step 3.2: Implement Blueprint Versioning Logic
**File:** `sag/services/versioning_service.py`

```python
class BlueprintVersionManager:
    """Manages blueprint versions and activation."""

    def create_new_version(self, current_blueprint, from_evolution=False, triggers=None):
        """
        Create v2 blueprint from current version.
        - Copy clusters, keywords, attributes
        - Apply evolution changes
        - Set status='draft'
        - Link parent_version to current
        """
        pass

    def activate_version(self, draft_blueprint):
        """
        Promote draft to active.
        - Set status='active'
        - Archive old version (status='archived')
        - Update site.active_blueprint
        """
        pass

    def compare_versions(self, v1, v2):
        """Return diff of clusters/keywords/attributes between versions."""
        pass

    def rollback_version(self, blueprint_id, target_version):
        """Revert to previous version (archival only, no delete)."""
        pass
```

#### Step 3.3: Create Evolution API Handlers
Update `sag/views.py` to handle:
- POST /blueprints/{id}/evolve/
  - Detect all evolution triggers
  - Return triggers + recommendations
  - Flag if user action needed
- POST /blueprints/{id}/create-version/
  - Accept trigger selection
  - Create new version
  - Return draft for review

#### Step 3.4: Create Evolution Tests
**File:** `tests/sag/test_evolution_service.py`

Test scenarios:
- New product category detection
- Keyword gap detection
- New sector detection
- Evolution report generation
- Blueprint version creation
- Version activation & archival

---

### Phase 4: Dashboard Widget & Frontend (Week 4)

#### Step 4.1: Create Dashboard Widget Component
**File:** `frontend/src/components/SAGHealthWidget.tsx`

Display:
```
┌─────────────────────────────────────┐
│  SAG Health Score: 72.5 / 100       │
│  ████████████░░░░░░░░░░░░░░░░░░░░░  │
│  Status: GOOD  ↗ Trending Up (+4.2) │
├─────────────────────────────────────┤
│ Component Breakdown:                 │
│ ├─ Completeness:       25/30 ✓      │
│ ├─ Cluster Coverage:   18/25 ◐      │
│ ├─ Internal Linking:   16/20 ◐      │
│ ├─ Taxonomy Align:     12/15 ◐      │
│ └─ Keyword Coverage:    1.5/10 ✗    │
├─────────────────────────────────────┤
│ Cluster Status:                      │
│ ├─ Complete:    2 clusters          │
│ ├─ In Progress: 3 clusters          │
│ └─ Planned:     1 cluster           │
├─────────────────────────────────────┤
│ Next Recommended Actions:            │
│ 1. Create hub page for Cluster 5    │
│    Impact: HIGH | Effort: 2-4h      │
│                                      │
│ 2. Add internal links (8 blogs)     │
│    Impact: MEDIUM | Effort: 1-2h    │
│                                      │
│ 3. Create 8 attribute term pages    │
│    Impact: MEDIUM | Effort: 4-6h    │
├─────────────────────────────────────┤
│ [View Blueprint] [Update Blueprint]  │
│ [View Execution Plan] [History]      │
└─────────────────────────────────────┘
```

**Implementation Details:**
- Fetch health score via API
- Display as progress bar + numeric score
- Show component breakdown with visual indicators
- Show cluster status pie/bar chart
- Render recommendations list (top 5, sorted by impact)
- Include action buttons with links to relevant pages
- Add 4-week trend chart

#### Step 4.2: Create Health History Chart
**File:** `frontend/src/components/SAGHealthChart.tsx`

Line chart showing:
- X-axis: Last 4 weeks (Monday to Monday)
- Y-axis: Score 0-100
- Line: Overall health trend
- Stacked area: Component breakdown (optional)
- Hover: Show detailed scores for week

#### Step 4.3: Create Recommendations Page
**File:** `frontend/src/pages/SAGRecommendations.tsx`

Page showing:
- All recommendations (20+)
- Filter by type (create_hub, add_blogs, link_pages, etc.)
- Filter by priority (high/medium/low)
- Sort by impact or effort
- Action buttons to implement each recommendation
- Status tracking (completed/in-progress/pending)

#### Step 4.4: Create Blueprint Version History Page
**File:** `frontend/src/pages/BlueprintVersionHistory.tsx`

Display:
- Timeline of all versions
- Version number + name + description
- Status badge (active/archived/draft)
- Created by + date
- Health score snapshot
- Diff view vs previous version
- Activate/rollback buttons for archived versions

#### Step 4.5: Create Evolution Trigger Review Page
**File:** `frontend/src/pages/EvolutionTriggerReview.tsx`

Display detected triggers:
- New product categories (with suggestion)
- Keyword gaps (with cluster mapping suggestions)
- New sectors (with impact analysis)
- Action: Accept → create v2 / Reject → ignore
- Preview new blueprint structure before creation

#### Step 4.6: Frontend Tests
**File:** `frontend/src/__tests__/SAGHealthWidget.test.tsx`

Test:
- Widget renders with health score
- Trend indicator shows correctly
- Component bars display with correct proportions
- Recommendations list populated
- Chart renders with trend data
- Action buttons navigate correctly

---

### Phase 5: Integration & Validation (Week 4-5)

#### Step 5.1: End-to-End Testing
Test complete workflow:
1. Create blueprint via wizard
2. Trigger health check
3. Verify score calculation
4. Review recommendations
5. Implement recommendation (e.g., create hub page)
6. Re-trigger health check
7. Verify score improved
8. Check Celery task runs on schedule
9. Verify email report generated

#### Step 5.2: Performance Optimization
- Cache health scores (expires in 24h unless force_recalculate)
- Index frequently-queried fields (blueprint_id, created_at in SAGHealthSnapshot)
- Optimize cluster completion query (use select_related/prefetch_related)
- Batch process evolution checks for multiple sites

#### Step 5.3: Documentation
- Health score calculation algorithm (markdown)
- API endpoint documentation (Swagger/OpenAPI)
- Celery task documentation
- Frontend widget integration guide
- User guide: interpreting health scores & recommendations

#### Step 5.4: Staging Deployment
- Deploy to staging environment
- Run full test suite
- Perform load test (simulate 1000 blueprints, health check)
- Manual QA: test all UI flows
- Get stakeholder sign-off

---

## 5. Acceptance Criteria

### 5.1 Health Score Calculation
- [ ] Health score calculated across all 5 components
- [ ] Score matches specification (0-100 range, component weights)
- [ ] Handles edge cases (empty blueprint, orphaned content)
- [ ] Score interpretation (Excellent/Good/Fair/Poor) correct
- [ ] Unit test coverage ≥85% for health_service.py
- [ ] Score deterministic (same input → same output)
- [ ] Performance: calculation < 2 seconds for typical blueprint

### 5.2 Data Persistence
- [ ] SAGHealthSnapshot model created and migrated
- [ ] SAGBlueprint model updated with versioning fields
- [ ] Health snapshots stored weekly via Celery task
- [ ] Historical data retained (no automatic deletion)
- [ ] Database indexes created for performance

### 5.3 Weekly Automation
- [ ] Celery Beat schedule configured (Monday 2 AM UTC)
- [ ] run_blueprint_health_check task completes successfully
- [ ] SAGHealthSnapshot created after each run
- [ ] Recommendations generated and stored
- [ ] Evolution triggers detected and flagged
- [ ] Task handles errors gracefully (no task failure on partial data)
- [ ] Task logs action for audit trail

### 5.4 API Endpoints
- [ ] POST /health-check/ returns health score + breakdown
- [ ] GET /health-history/ returns 4-week trend data
- [ ] GET /recommendations/ returns prioritized action list
- [ ] POST /evolve/ detects evolution triggers
- [ ] POST /create-version/ creates draft blueprint version
- [ ] GET /versions/ lists all versions with status
- [ ] All endpoints require authentication (blueprint owner/admin)
- [ ] All endpoints return proper error messages (400/403/404)
- [ ] API test coverage ≥90%

### 5.5 Blueprint Versioning
- [ ] New versions created with incremented version number
- [ ] Version status transitions: draft → active → archived
- [ ] Parent version link maintains version history
- [ ] Version activation archives previous active version
- [ ] Historical versions preserved (not deleted)
- [ ] Version diff shows cluster/keyword changes

### 5.6 Evolution Triggers
- [ ] New product category detected when products have undefined attributes
- [ ] Keyword gaps detected when keywords don't map to clusters
- [ ] New sector detection triggers blueprint merge analysis
- [ ] User notified of evolution opportunities
- [ ] User can confirm/reject evolution triggers

### 5.7 Dashboard Widget
- [ ] Widget displays on site dashboard
- [ ] Health score shown as number + progress bar
- [ ] Trend indicator shows improving/stable/declining
- [ ] Component breakdown bars show all 5 components
- [ ] Cluster status counts correct
- [ ] Top 5 recommendations displayed
- [ ] Action buttons navigate to relevant pages
- [ ] Widget responsive (mobile/tablet/desktop)
- [ ] Widget performance: renders < 1 second

### 5.8 Frontend Charts & Pages
- [ ] Health history chart shows 4-week trend
- [ ] Recommendations page filters by type/priority/effort
- [ ] Version history page shows timeline of all versions
- [ ] Evolution trigger review page shows detected triggers
- [ ] All pages require blueprint ownership for access
- [ ] All pages have proper error handling (empty states, loading states)

### 5.9 Integration
- [ ] Health score integrates with blueprint creation workflow
- [ ] Health score integrates with execution plan (doc 01E)
- [ ] Recommendations integrate with gap analysis (doc 01E)
- [ ] Evolution triggers trigger version control flow
- [ ] All cross-references to other docs (01A, 01C, 01D, 01E, 01F) working

### 5.10 Documentation & Deployment
- [ ] API documentation complete (Swagger/OpenAPI)
- [ ] Health score calculation algorithm documented
- [ ] Celery task configuration documented
- [ ] Frontend widget integration guide documented
- [ ] User guide: interpreting health & recommendations
- [ ] Deployment checklist completed
- [ ] Staging environment tested & approved

---

## 6. Claude Code Instructions

### For Developers Implementing This Feature

#### 6.1 Getting Started
1. **Read the specifications** in sections 2-4 above carefully
2. **Review cross-references** to docs 01A (models), 01C (clusters), 01D (wizard), 01E (pipeline), 01F (attributes)
3. **Set up development environment** with Celery, Redis for task queue
4. **Create feature branch** from main: `git checkout -b feature/01G-health-monitoring`

#### 6.2 Implementation Order
Follow Phase 1 → Phase 5 sequentially. Do not skip phases.

**Week 1-2:** Health score calculation
- Implement HealthScoreCalculator class
- Create comprehensive unit tests
- Validate edge cases

**Week 2-3:** Automation & API
- Build Celery tasks
- Create API endpoints & tests
- Configure Celery Beat schedule

**Week 3-4:** Evolution & Versioning
- Implement EvolutionDetector
- Implement BlueprintVersionManager
- Create versioning UI

**Week 4:** Dashboard & Frontend
- Build health widget
- Create recommendation & history pages
- Implement charts

**Week 5:** Integration & Validation
- End-to-end testing
- Performance optimization
- Staging deployment

#### 6.3 Key Implementation Notes

**Health Score Calculation:**
- Each component must be independently testable
- Use Python's `decimal.Decimal` for precise calculations
- Cache calculated scores (expires 24h unless force_recalculate)
- Document scoring logic in code comments

**Celery Tasks:**
- Use shared_task decorator for easy registration
- Handle missing/null data gracefully
- Log all actions for auditing
- Test with Celery test client (not live broker)

**API Design:**
- Follow DRF best practices (serializers, viewsets, routers)
- Use pagination for list endpoints (default: 20 items)
- Return JSON consistently across all endpoints
- Include `checked_at` timestamp in responses

**Frontend:**
- Use React hooks (no class components)
- Fetch health data via custom hook `useHealthScore(blueprintId)`
- Implement loading/error states
- Use TypeScript for type safety

**Database:**
- Create migrations incrementally (don't squash)
- Add indexes for frequently-queried fields
- Use bulk_create for batch operations
- Document schema changes in migration files

**Testing:**
- Unit tests for all service methods
- Integration tests for API endpoints
- Functional tests for Celery tasks
- E2E tests for critical workflows
- Target coverage: ≥85% across backend, ≥80% across frontend

#### 6.4 Common Pitfalls to Avoid

1. **Circular imports:** Import models/services carefully; use TYPE_CHECKING if needed
2. **N+1 queries:** Use select_related/prefetch_related in views
3. **Task timeouts:** Set reasonable timeouts for Celery tasks (default: 30 min)
4. **Floating point errors:** Use Decimal for score calculations
5. **Missing error handling:** Wrap API calls in try/except; return proper HTTP status
6. **Stale health scores:** Always verify last_health_check is recent (< 7 days)

#### 6.5 Code Style & Standards

- Follow PEP 8 for Python
- Follow ESLint rules for TypeScript/React
- Comment complex calculations (especially health score components)
- Use meaningful variable names (not `x`, `y`, `temp`)
- Docstrings on all public methods
- Type hints on all function signatures

#### 6.6 Debugging Tips

**Health score seems wrong:**
- Manually run HealthScoreCalculator with test blueprint
- Verify each component score independently
- Check database queries (use django-debug-toolbar)
- Print intermediate calculations in unit tests

**Celery task not running:**
- Verify Celery Beat schedule is configured
- Check task is registered (`celery -A igny8_core inspect active_queues`)
- Check for errors in Celery worker logs
- Test task manually via shell: `run_blueprint_health_check.delay(site_id=1)`

**API endpoint returning 500:**
- Check Django error logs for traceback
- Test endpoint with curl/Postman first
- Verify authentication & permissions
- Check serializer validation errors

**Frontend chart not rendering:**
- Inspect network tab for API response
- Verify response format matches serializer schema
- Check console for TypeScript/runtime errors
- Test with mock data first

#### 6.7 Version Control Workflow

```bash
# Create feature branch
git checkout -b feature/01G-health-monitoring

# Commit frequently (one commit per step)
git commit -m "01G: Implement HealthScoreCalculator and unit tests"
git commit -m "01G: Create SAGHealthSnapshot model and migration"
git commit -m "01G: Build health check API endpoints"

# Push for review
git push origin feature/01G-health-monitoring

# Create pull request with:
# - Link to this spec (01G)
# - Summary of changes by phase
# - Test coverage report
# - Performance metrics
# - Screenshots of UI changes
```

#### 6.8 Code Review Checklist

Before submitting PR, verify:
- [ ] All acceptance criteria (section 5) met
- [ ] Test coverage ≥85% for backend, ≥80% for frontend
- [ ] No console errors/warnings (frontend)
- [ ] No Django deprecation warnings (backend)
- [ ] Database migrations created & tested
- [ ] Celery tasks tested with test client
- [ ] API documentation updated
- [ ] User guide/documentation complete
- [ ] No hardcoded values (use settings)
- [ ] Performance acceptable (< 2s for health check)
- [ ] Error handling comprehensive
- [ ] Security review: no SQL injection, XSS, CSRF issues

#### 6.9 Documentation to Create/Update

Create/update these files:
1. **API Documentation:** `docs/api/sag-health-monitoring.md`
2. **Health Score Algorithm:** `docs/architecture/health-score-calculation.md`
3. **Celery Tasks:** `docs/architecture/celery-tasks.md`
4. **Frontend Integration:** `docs/frontend/sag-health-widget.md`
5. **User Guide:** `docs/user-guides/understanding-health-scores.md`
6. **Database Schema:** `docs/database/health-snapshot-schema.md`

#### 6.10 Staging & Deployment

**Pre-staging checklist:**
- [ ] All tests passing locally
- [ ] Code review approved
- [ ] Feature branch merged to develop
- [ ] Database migrations tested
- [ ] Celery Beat schedule configured
- [ ] Environment variables set (cache TTL, task timeouts, etc.)

**Staging deployment:**
1. Deploy code to staging server
2. Run migrations: `python manage.py migrate sag`
3. Start Celery worker: `celery -A igny8_core worker -l info`
4. Start Celery Beat: `celery -A igny8_core beat -l info`
5. Run smoke tests against staging API
6. Manual QA: test all UI flows
7. Monitor logs for errors
8. Get sign-off from product & QA

**Production rollout:**
- Blue-green deployment (run both versions simultaneously)
- Monitor health check CPU/memory usage
- Monitor Celery task queue depth
- Monitor API response times
- Have rollback plan ready (revert to previous version)

---

## 7. Related Documents

- **01A:** SAG Blueprint Data Model & Initialization
- **01C:** Cluster Architecture & Completeness Tracking
- **01D:** Blueprint Creation Wizard
- **01E:** Content Gap Analysis & Recommendations Pipeline
- **01F:** Attribute Taxonomy & Mapping System

---

## 8. Appendix: Scoring Formula Reference

### Quick Formula Reference

```
TOTAL_SCORE = COMPLETENESS + CLUSTER_COVERAGE + INTERNAL_LINKING + TAXONOMY + KEYWORDS

COMPLETENESS (max 30)
  = HUBS (0-10) + BLOGS (0-10) + ATTRIBUTES (0-10)

CLUSTER_COVERAGE (max 25)
  = COMPLETE_CLUSTERS * 5 (up to 5 clusters)
  + PARTIAL_CLUSTERS * (completion_ratio * 5)
  - PLANNED_PENALTY (if >50% unstarted)

INTERNAL_LINKING (max 20)
  = HUB_CATEGORY (0-5) + BLOG_HUB_LINKS (0-5)
    + ATTRIBUTE_INTERLINKS (0-5) + NAV_CONSISTENCY (0-5)

TAXONOMY_ALIGNMENT (max 15)
  = PRIMARY_TAGS (0-5) + SECONDARY_TAGS (0-5)
    + TERM_COVERAGE (0-5) - ORPHANED_PENALTY

KEYWORD_COVERAGE (max 10)
  = KEYWORD_CONTENT_MAPPING (0-7) + DISTRIBUTION (0-3)
```

### Minimum Thresholds for Points

| Component | Metric | Threshold | Points |
|---|---|---|---|
| Completeness | Hubs | % of defined | 10 |
| Completeness | Blogs | ≥60% recommended | 10 |
| Completeness | Attributes | ≥75% have pages | 10 |
| Cluster Coverage | Complete clusters | per cluster | 5 each |
| Linking | Hub-category | % linked | 5 |
| Linking | Blog-hub | ≥50% | 5 |
| Linking | Attribute interlinks | ≥70% | 5 |
| Linking | Navigation | ≥90% consistent | 5 |
| Taxonomy | Primary tags | ≥70% | 5 |
| Taxonomy | Secondary tags | ≥50% | 5 |
| Taxonomy | Term coverage | % with content | 5 |
| Keywords | With content | ≥50% | 7 |
| Keywords | Distribution | avg ≥2.5/cluster | 3 |

---

**Document Version:** 1.0
**Last Reviewed:** 2026-03-23
**Next Review Date:** After Phase 1 Implementation
**Owner:** SAG Development Team
**Status:** Ready for Implementation