salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b94d41b7f60c0d5586ad4cb5a2f71e33718527ad
igny8/v2/V2-Execution-Docs/01B-sector-attribute-templates.md
IGNY8 VPS (Salman) 128b186865 temproary docs uplaoded
2026-03-23 09:02:49 +00:00

43 KiB
Raw Blame History

01B - Sector Attribute Templates

IGNY8 Phase 1: Service Layer & AI Functions

Version: 1.0 Date: 2026-03-23 Status: Build-Ready Owner: SAG Team

1. Current State

Model Foundation

  • SectorAttributeTemplate model defined in 01A (sag/models.py)
  • Schema includes:
    • industry (string)
    • sector (string)
    • attribute_framework (JSON object)
    • keyword_templates (JSON object)
    • source (string: 'system' | 'ai_generated' | 'user_contributed')
    • created_at, updated_at (timestamps)
    • is_active (boolean)

Reference Data

  • 45 industries across all major verticals
  • 449 sectors mapped to industries (10 sectors/industry average)
  • Industry Master List: REF-industry-sector-list.md
  • Quality Standards: REF-niche-definition-process.md

Quality Framework Constraints

From SAG Niche Definition Process:

  • Attributes are dimensional axes for browsing navigation
  • Primary attributes: 2-3 per sector
  • Secondary attributes: 2-3 per sector
  • Tertiary attributes: 1-2 per sector
  • Each attribute value supports a taxonomy landing page
  • 5-8 attributes per sector optimal
  • 4-12 values per attribute (primary tier requirement)

2. What to Build

2.1 Service Layer: template_service.py

Location: sag/services/template_service.py

Core Functions

def get_template(industry: str, sector: str) -> Optional[SectorAttributeTemplate]
  • Fetch existing template from database
  • Return None if not found
  • Case-insensitive industry/sector lookup with normalization
def get_or_generate_template(
    industry: str,
    sector: str,
    site_type: str
) -> SectorAttributeTemplate
  • Load template if exists
  • If missing: trigger AI generation via discover_sector_attributes() AI function
  • Save generated template with source='ai_generated'
  • Return completed template
  • Cache in Redis for 7 days (key: sag:template:{industry}:{sector})
def merge_templates(
    sector_list: List[Tuple[str, str]],  # [(industry, sector), ...]
    strategy: str = 'union'  # 'union' | 'intersection'
) -> SectorAttributeTemplate
  • Input: list of (industry, sector) tuples (max 5 sectors)
  • Load all sector templates
  • Merge logic (see section 2.3):
    • Identical attributes across sectors: combine value lists (deduplicate)
    • Unique attributes from one sector: add as sector-specific
    • Conflicting attribute levels: promote to higher level
    • Result: unified framework covering all sectors
  • Return merged template with source='merged', sector='multi-sector'
  • Not persisted (transient, computed on-demand)
def validate_template(template_data: dict) -> Tuple[bool, List[str]]
  • Validate structure against quality rules
  • Return tuple: (is_valid: bool, errors: List[str])
  • Checks:
    • attribute_framework is valid JSON with required fields
    • keyword_templates is valid JSON object
    • 5-8 attributes in framework
    • Each attribute: 4-12 values minimum
    • Primary attributes: 2-3 count
    • Secondary attributes: 2-3 count
    • Tertiary attributes: 1-2 count
    • Attributes are dimensional (flagged if suspected metadata)
    • No forbidden attribute types (Price, SKU, Shipping Weight, etc.)
    • Keyword patterns match site_type patterns
  • Detailed error messages for each violation
def seed_templates(
    industry: str,
    templates_data: List[dict],
    batch_size: int = 100
) -> Tuple[int, List[str]]
  • Bulk create templates from fixture data
  • Validate each before creation
  • Return: (created_count, failed_sector_names)
  • Skip if template already exists (no overwrite)
  • Useful for pre-populating top 20 industries
def prune_template(template: SectorAttributeTemplate) -> SectorAttributeTemplate
  • Apply quality rules: demote attributes with too few values
  • Primary with 1 value → demote to secondary
  • Secondary with 1 value → demote to tertiary
  • Tertiary with 0 values → exclude
  • Reorganize hierarchy
  • Return modified template (not persisted unless explicitly saved)

2.2 AI Function: DiscoverSectorAttributes

Location: sag/ai_functions/attribute_discovery.py Register Key: discover_sector_attributes

Function Signature

@ai_function(key='discover_sector_attributes')
async def discover_sector_attributes(
    industry: str,
    sector: str,
    site_type: str  # 'ecommerce' | 'local_services' | 'saas' | 'content'
) -> dict:

Input

  • industry: Industry name (e.g., "Pet Supplies")
  • sector: Sector name (e.g., "Dog Accessories")
  • site_type: Business model type for keyword pattern selection

Output Structure

Returns JSON matching SectorAttributeTemplate schema:

{
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce",
  "attribute_framework": {
    "primary": [
      {
        "name": "Product Type",
        "values": ["Collars", "Leashes", "Harnesses", "Beds", "Toys", "Bowls", "ID Tags"],
        "search_volume": "high"
      },
      {
        "name": "Dog Size",
        "values": ["Extra Small", "Small", "Medium", "Large", "Extra Large"],
        "search_volume": "high"
      },
      {
        "name": "Material",
        "values": ["Leather", "Nylon", "Fabric", "Metal", "Rubber"],
        "search_volume": "medium"
      }
    ],
    "secondary": [
      {
        "name": "Style",
        "values": ["Classic", "Designer", "Sport", "Fashion", "Training"],
        "search_volume": "medium"
      },
      {
        "name": "Price Range",
        "values": ["Budget ($5-$20)", "Mid-Range ($20-$50)", "Premium ($50+)"],
        "search_volume": "medium"
      }
    ],
    "tertiary": [
      {
        "name": "Feature",
        "values": ["Reflective", "Waterproof", "Adjustable"],
        "search_volume": "low"
      }
    ]
  },
  "keyword_templates": {
    "ecommerce": {
      "patterns": [
        "[value1] [value2]",
        "best [value1] [value2]",
        "[value1] [value2] for [value3]",
        "[value1] [value2] reviews",
        "[brand] [value1] [value2]",
        "[value1] vs [value2]"
      ],
      "examples": [
        "leather dog collars",
        "best small dog beds",
        "waterproof dog leashes for large dogs"
      ]
    }
  },
  "metadata": {
    "generation_date": "2026-03-23T10:30:00Z",
    "model_version": "claude-3-5-sonnet",
    "confidence_score": 0.92
  }
}

Prompt Strategy

You are an expert in site architecture and taxonomy for the {industry} industry,
specifically the {sector} sector.

Your task is to define the key dimensional attributes for a {site_type} website
focused on {sector} products/services. These attributes represent the ways users
naturally browse and filter when searching for {sector} solutions.

CONSTRAINTS:
1. Attributes must be dimensional axes (users should want to browse along these axes)
2. Valid examples: Product Type, Size, Material, Color, Feature
3. Invalid examples: Price, SKU, Shipping Weight, Inventory Count
4. Generate exactly 5-8 attributes total:
   - Primary tier: 2-3 attributes (highest search volume, broadest appeal)
   - Secondary tier: 2-3 attributes (supporting dimensions)
   - Tertiary tier: 1-2 attributes (specialized, niche filters)
5. Each primary attribute must have 4-12 values
6. Secondary and tertiary can have 2-8 values
7. Each value must be able to support a dedicated landing page
8. Organize values from high to low search frequency

OUTPUT: Valid JSON matching the schema above. Ensure all constraints are met.

AI Generation Rules

  • Temperature: 0.7 (structured but creative for domain expertise)
  • Max Tokens: 2000
  • Cache: Template generation results cache for 30 days
  • Validation: Run validate_template() on output before returning
  • Fallback: If validation fails, retry with stricter prompt, max 2 retries
  • Error Handling: Log to sag_ai_generation logger with full prompt/response

2.3 Multi-Sector Merge Algorithm

Used by: merge_templates() function Context: When a site targets multiple sectors (up to 5)

Merge Process

Input: List of sector templates [T1, T2, T3, ...]

Step 1: Collect All Attributes
  - For each template, extract attribute names from all tiers
  - Build set of unique attribute names across all templates

Step 2: For Each Unique Attribute

  Case A: Attribute exists in 2+ templates
    - Collect all values from all occurrences
    - Deduplicate values (case-insensitive)
    - Combine search_volume signals: max(search_volumes)
    - Determine level:
      * If primary in ANY template → primary
      * Else if secondary in ANY template → secondary
      * Else → tertiary
    - Result: merged attribute with union of values

  Case B: Attribute exists in 1 template
    - Add attribute as-is, tag with source sector
    - Mark as "sector_specific: true"

  Case C: Conflicting levels (e.g., primary in T1, secondary in T2)
    - Promote to higher level (primary > secondary > tertiary)
    - Combine value lists

Step 3: Organize Result
  - Sort attributes by combined_search_volume descending
  - Reorganize into primary/secondary/tertiary:
    * Primary: top 2-3 highest volume attributes
    * Secondary: next 2-3 attributes
    * Tertiary: remaining attributes
  - Apply pruning rules (demote attributes with <4 values if primary)

Step 4: Return Merged Template
  - source: 'merged'
  - sector: 'multi-sector'
  - industry: 'multi-industry'
  - attribute_framework: merged structure
  - metadata.source_sectors: list of input sectors
  - metadata.merge_strategy: 'union' | 'intersection'

Example: Merging "Dog Accessories" + "Cat Accessories"

Input Templates:

  • Dog Accessories: Product Type, Dog Size, Material, Style, Feature
  • Cat Accessories: Product Type, Cat Size, Material, Style, Color

Output (Merged):

{
  "primary": [
    {
      "name": "Product Type",
      "values": ["Collars", "Beds", "Toys", "Bowls", "Carriers", ...],
      "source_sectors": ["Dog Accessories", "Cat Accessories"]
    },
    {
      "name": "Size",
      "values": ["Extra Small", "Small", "Medium", "Large", "Extra Large"],
      "source_sectors": ["Dog Accessories", "Cat Accessories"]
    },
    {
      "name": "Material",
      "values": ["Leather", "Nylon", "Fabric", "Metal", "Rubber"],
      "source_sectors": ["Dog Accessories", "Cat Accessories"]
    }
  ],
  "secondary": [
    {
      "name": "Style",
      "values": ["Classic", "Designer", "Sport", "Fashion"],
      "source_sectors": ["Dog Accessories", "Cat Accessories"]
    },
    {
      "name": "Feature",
      "values": ["Reflective", "Waterproof", "Adjustable"],
      "source_sectors": ["Dog Accessories"]
    }
  ],
  "tertiary": [
    {
      "name": "Color",
      "values": ["Black", "Brown", "Pink", "Blue"],
      "source_sectors": ["Cat Accessories"],
      "sector_specific": true
    }
  ]
}

2.4 Template Seeding Strategy

Purpose: Pre-populate high-demand industry/sector combinations

Top 20 Industries (Priority 1)

1. Pet Supplies (10 sectors)
2. E-commerce Software (8 sectors)
3. Digital Marketing (7 sectors)
4. Healthcare Services (12 sectors)
5. Real Estate (9 sectors)
6. Local Services (11 sectors)
7. SaaS Tools (9 sectors)
8. Fashion & Apparel (10 sectors)
9. Food & Beverage (11 sectors)
10. Home & Garden (8 sectors)
11. Technology Products (9 sectors)
12. Travel & Tourism (10 sectors)
13. Fitness & Wellness (9 sectors)
14. Finance & Banking (8 sectors)
15. Education & Training (10 sectors)
16. Automotive (10 sectors)
17. Legal Services (9 sectors)
18. Photography & Media (8 sectors)
19. Entertainment (9 sectors)
20. Beauty & Personal Care (11 sectors)

Seeding Implementation

Fixture File: sag/fixtures/sector_templates_seed.json

{
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "source": "system",
  "attribute_framework": { ... },
  "keyword_templates": { ... }
}

Management Command: python manage.py seed_sector_templates --industry "Pet Supplies"

  • Load from fixture
  • Validate each template
  • Create in batch (bulk_create)
  • Report results: created, skipped, failed

Fallback: For industries not in seed, AI generates on-demand via get_or_generate_template()

2.5 Attribute Quality Rules

Dimensional Axis Test

An attribute is valid if: "Would users naturally want to browse a website along this axis?"

Valid Examples:

  • Product Type, Size, Material, Color, Feature, Style
  • Service Type, Body Area, Problem Type, Expertise Level
  • Device Type, Use Case, Budget Category, Audience

Invalid Examples (Metadata, not browsable):

  • Price (varies by product; use "Price Range" instead)
  • SKU, Product Code, Item Number
  • Shipping Weight, Dimensions, Package Count
  • Inventory Status, Stock Level
  • Internal Category Code, Warehouse Location

Pruning Rules

Applied in prune_template():

  1. Primary Attributes:

    • Must have 4+ values
    • If 1-3 values: demote to secondary
    • If 0 values: remove entirely

  2. Secondary Attributes:

    • Should have 2+ values
    • If 1 value: demote to tertiary
    • If 0 values: remove entirely

  3. Tertiary Attributes:

    • Can have 1+ values
    • If 0 values: remove entirely

  4. Overall Constraint:

    • Must have 5-8 total attributes after pruning
    • If <5 after pruning: log warning, manually add supplementary attributes

3. Data Models / APIs

3.1 SectorAttributeTemplate Model

Location: sag/models.py (from 01A, extended here)

class SectorAttributeTemplate(models.Model):
    # Identity
    industry = models.CharField(max_length=255, db_index=True)
    sector = models.CharField(max_length=255, db_index=True)
    site_type = models.CharField(
        max_length=50,
        choices=[
            ('ecommerce', 'E-commerce'),
            ('local_services', 'Local Services'),
            ('saas', 'SaaS'),
            ('content', 'Content'),
            ('multi', 'Multi-type'),
        ],
        default='multi'
    )

    # Data
    attribute_framework = models.JSONField(
        default=dict,
        help_text="Primary/secondary/tertiary attributes with values"
    )
    keyword_templates = models.JSONField(
        default=dict,
        help_text="Keyword patterns by site type"
    )

    # Metadata
    source = models.CharField(
        max_length=50,
        choices=[
            ('system', 'System-defined'),
            ('ai_generated', 'AI-generated'),
            ('user_contributed', 'User-contributed'),
            ('merged', 'Merged from multiple sectors'),
        ],
        default='system'
    )
    is_active = models.BooleanField(default=True, db_index=True)

    # Lifecycle
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    # Relationships
    created_by = models.ForeignKey(
        User,
        on_delete=models.SET_NULL,
        null=True,
        blank=True,
        related_name='sector_templates_created'
    )

    class Meta:
        unique_together = [('industry', 'sector')]
        indexes = [
            models.Index(fields=['industry', 'sector']),
            models.Index(fields=['source', 'is_active']),
        ]
        ordering = ['-updated_at']

    def __str__(self):
        return f"{self.industry} / {self.sector}"

3.2 REST API Endpoints

Base URL: /api/v1/sag/ Authentication: Requires authentication (session or token)

GET /sector-templates/{industry}/{sector}/

List or fetch a single template

Request:

GET /api/v1/sag/sector-templates/Pet%20Supplies/Dog%20Accessories/

Response (200 OK):

{
  "id": 42,
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce",
  "attribute_framework": { ... },
  "keyword_templates": { ... },
  "source": "ai_generated",
  "is_active": true,
  "created_at": "2026-03-01T10:00:00Z",
  "updated_at": "2026-03-01T10:00:00Z"
}

Response (404 Not Found):

{
  "detail": "Template not found for Pet Supplies / Dog Accessories"
}

POST /sector-templates/

Create a new template

Request:

{
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce",
  "attribute_framework": { ... },
  "keyword_templates": { ... },
  "source": "user_contributed"
}

Response (201 Created):

{
  "id": 43,
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce",
  "attribute_framework": { ... },
  "keyword_templates": { ... },
  "source": "user_contributed",
  "is_active": true,
  "created_at": "2026-03-23T14:30:00Z",
  "updated_at": "2026-03-23T14:30:00Z"
}

Response (400 Bad Request):

{
  "errors": [
    "Attribute framework missing required fields",
    "Primary attributes must have 4+ values"
  ]
}

GET /sector-templates/

List all templates (with filters)

Request:

GET /api/v1/sag/sector-templates/?industry=Pet%20Supplies&source=ai_generated&is_active=true

Query Parameters:

  • industry (optional): Filter by industry
  • sector (optional): Filter by sector
  • source (optional): 'system' | 'ai_generated' | 'user_contributed' | 'merged'
  • is_active (optional): true | false
  • limit (optional): Pagination limit (default 100)
  • offset (optional): Pagination offset

Response (200 OK):

{
  "count": 450,
  "next": "/api/v1/sag/sector-templates/?limit=100&offset=100",
  "previous": null,
  "results": [
    { ... template 1 ... },
    { ... template 2 ... }
  ]
}

PUT /sector-templates/{id}/

Update an existing template

Request:

{
  "is_active": false,
  "attribute_framework": { ... }
}

Response (200 OK):

{
  "id": 42,
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce",
  "attribute_framework": { ... },
  "keyword_templates": { ... },
  "source": "ai_generated",
  "is_active": false,
  "created_at": "2026-03-01T10:00:00Z",
  "updated_at": "2026-03-23T15:00:00Z"
}

POST /sector-templates/generate/

Trigger AI generation for a sector

Request:

{
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce"
}

Response (201 Created):

{
  "id": 44,
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "site_type": "ecommerce",
  "attribute_framework": { ... },
  "keyword_templates": { ... },
  "source": "ai_generated",
  "is_active": true,
  "created_at": "2026-03-23T15:30:00Z",
  "updated_at": "2026-03-23T15:30:00Z",
  "metadata": {
    "generation_date": "2026-03-23T15:30:00Z",
    "confidence_score": 0.92
  }
}

Response (202 Accepted - async):

{
  "status": "generating",
  "task_id": "uuid-1234-5678",
  "industry": "Pet Supplies",
  "sector": "Dog Accessories",
  "message": "Template generation in progress. Check back in 30 seconds."
}

POST /sector-templates/merge/

Merge multiple sector templates

Request:

{
  "sectors": [
    {"industry": "Pet Supplies", "sector": "Dog Accessories"},
    {"industry": "Pet Supplies", "sector": "Cat Accessories"},
    {"industry": "Pet Supplies", "sector": "Bird Accessories"}
  ],
  "strategy": "union"
}

Response (200 OK):

{
  "id": null,
  "industry": "multi-industry",
  "sector": "multi-sector",
  "site_type": "ecommerce",
  "attribute_framework": { ... merged ... },
  "keyword_templates": { ... },
  "source": "merged",
  "is_active": true,
  "metadata": {
    "source_sectors": [
      {"industry": "Pet Supplies", "sector": "Dog Accessories"},
      {"industry": "Pet Supplies", "sector": "Cat Accessories"},
      {"industry": "Pet Supplies", "sector": "Bird Accessories"}
    ],
    "merge_strategy": "union"
  }
}

Note: Merged templates are NOT persisted (transient). If the user approves the merge, a new template should be created via POST /sector-templates/.

3.3 Service Layer: TemplateService Class

Location: sag/services/template_service.py

from typing import Optional, List, Tuple, Dict, Any
from django.core.cache import cache
from django.db.models import Q
from sag.models import SectorAttributeTemplate
from sag.ai_functions.attribute_discovery import discover_sector_attributes

class TemplateService:
    """Service for managing sector attribute templates."""

    CACHE_TTL = 7 * 24 * 60 * 60  # 7 days
    GENERATION_CACHE_TTL = 30 * 24 * 60 * 60  # 30 days

    @staticmethod
    def normalize_key(industry: str, sector: str) -> str:
        """Normalize industry/sector for consistent lookups."""
        return f"{industry.strip().lower()}:{sector.strip().lower()}"

    @staticmethod
    def get_template(
        industry: str,
        sector: str
    ) -> Optional[SectorAttributeTemplate]:
        """Fetch template from database or cache."""
        cache_key = f"sag:template:{TemplateService.normalize_key(industry, sector)}"

        # Try cache first
        cached = cache.get(cache_key)
        if cached:
            return cached

        # Query database (case-insensitive)
        try:
            template = SectorAttributeTemplate.objects.get(
                industry__iexact=industry,
                sector__iexact=sector,
                is_active=True
            )
            cache.set(cache_key, template, TemplateService.CACHE_TTL)
            return template
        except SectorAttributeTemplate.DoesNotExist:
            return None

    @staticmethod
    async def get_or_generate_template(
        industry: str,
        sector: str,
        site_type: str = 'multi'
    ) -> SectorAttributeTemplate:
        """Load template or generate via AI if missing."""
        # Try to get existing
        template = TemplateService.get_template(industry, sector)
        if template:
            return template

        # Generate via AI
        template_data = await discover_sector_attributes(
            industry=industry,
            sector=sector,
            site_type=site_type
        )

        # Validate
        is_valid, errors = TemplateService.validate_template(template_data)
        if not is_valid:
            raise ValueError(f"Generated template validation failed: {errors}")

        # Persist
        template, created = SectorAttributeTemplate.objects.get_or_create(
            industry=industry,
            sector=sector,
            defaults={
                'site_type': site_type,
                'attribute_framework': template_data['attribute_framework'],
                'keyword_templates': template_data['keyword_templates'],
                'source': 'ai_generated',
            }
        )

        return template

    @staticmethod
    def merge_templates(
        sector_list: List[Tuple[str, str]],
        strategy: str = 'union'
    ) -> Dict[str, Any]:
        """Merge multiple sector templates."""
        if len(sector_list) > 5:
            raise ValueError("Cannot merge more than 5 sectors")

        if len(sector_list) < 2:
            raise ValueError("Must provide at least 2 sectors to merge")

        # Load all templates
        templates = []
        for industry, sector in sector_list:
            template = TemplateService.get_template(industry, sector)
            if not template:
                raise ValueError(f"Template not found: {industry} / {sector}")
            templates.append(template)

        # Execute merge algorithm
        merged = TemplateService._execute_merge(templates, strategy)

        return merged

    @staticmethod
    def _execute_merge(
        templates: List[SectorAttributeTemplate],
        strategy: str
    ) -> Dict[str, Any]:
        """Core merge algorithm (see section 2.3)."""
        # Collect all attributes
        all_attributes = {}
        for template in templates:
            framework = template.attribute_framework
            for tier in ['primary', 'secondary', 'tertiary']:
                for attr in framework.get(tier, []):
                    attr_name = attr['name'].lower()
                    if attr_name not in all_attributes:
                        all_attributes[attr_name] = {
                            'original_name': attr['name'],
                            'values_by_template': [],
                            'tiers': [],
                            'source_sectors': []
                        }
                    all_attributes[attr_name]['values_by_template'].append(attr['values'])
                    all_attributes[attr_name]['tiers'].append(tier)
                    all_attributes[attr_name]['source_sectors'].append(
                        f"{template.industry}/{template.sector}"
                    )

        # Build merged framework
        merged_primary = []
        merged_secondary = []
        merged_tertiary = []

        for attr_name, attr_data in all_attributes.items():
            # Deduplicate and combine values
            all_values = []
            for value_list in attr_data['values_by_template']:
                all_values.extend(value_list)
            unique_values = list(dict.fromkeys(
                [v for v in all_values]  # Preserve order
            ))

            # Determine tier
            tier = max(attr_data['tiers'], key=lambda t: ['tertiary', 'secondary', 'primary'].index(t))

            # Build merged attribute
            merged_attr = {
                'name': attr_data['original_name'],
                'values': unique_values,
                'search_volume': 'medium',  # Default; could be enhanced
                'source_sectors': attr_data['source_sectors']
            }

            # Add to appropriate tier
            if tier == 'primary':
                merged_primary.append(merged_attr)
            elif tier == 'secondary':
                merged_secondary.append(merged_attr)
            else:
                merged_tertiary.append(merged_attr)

        # Sort by value count descending
        merged_primary.sort(key=lambda a: len(a['values']), reverse=True)
        merged_secondary.sort(key=lambda a: len(a['values']), reverse=True)

        # Build keyword templates (take first template's pattern)
        keyword_templates = templates[0].keyword_templates if templates else {}

        # Assemble result
        merged_template = {
            'industry': 'multi-industry',
            'sector': 'multi-sector',
            'site_type': 'multi',
            'source': 'merged',
            'attribute_framework': {
                'primary': merged_primary,
                'secondary': merged_secondary,
                'tertiary': merged_tertiary
            },
            'keyword_templates': keyword_templates,
            'metadata': {
                'source_sectors': [
                    {'industry': t.industry, 'sector': t.sector}
                    for t in templates
                ],
                'merge_strategy': strategy
            }
        }

        return merged_template

    @staticmethod
    def validate_template(template_data: dict) -> Tuple[bool, List[str]]:
        """Validate template structure against quality rules."""
        errors = []

        # Check required fields
        if 'attribute_framework' not in template_data:
            errors.append("Missing 'attribute_framework'")
            return False, errors

        if 'keyword_templates' not in template_data:
            errors.append("Missing 'keyword_templates'")
            return False, errors

        framework = template_data['attribute_framework']

        # Check tier structure
        for tier in ['primary', 'secondary', 'tertiary']:
            if tier not in framework:
                errors.append(f"Missing tier: {tier}")
                continue

            if not isinstance(framework[tier], list):
                errors.append(f"Tier '{tier}' must be a list")
                continue

        # Count attributes by tier
        primary_count = len(framework.get('primary', []))
        secondary_count = len(framework.get('secondary', []))
        tertiary_count = len(framework.get('tertiary', []))
        total_count = primary_count + secondary_count + tertiary_count

        # Check total attribute count
        if total_count < 5:
            errors.append(f"Too few attributes ({total_count}); need 5-8")
        if total_count > 8:
            errors.append(f"Too many attributes ({total_count}); need 5-8")

        # Check tier counts
        if primary_count < 2 or primary_count > 3:
            errors.append(f"Primary attributes count ({primary_count}); need 2-3")
        if secondary_count < 2 or secondary_count > 3:
            errors.append(f"Secondary attributes count ({secondary_count}); need 2-3")
        if tertiary_count < 1 or tertiary_count > 2:
            errors.append(f"Tertiary attributes count ({tertiary_count}); need 1-2")

        # Check each attribute
        for tier in ['primary', 'secondary', 'tertiary']:
            for attr in framework.get(tier, []):
                if 'name' not in attr or 'values' not in attr:
                    errors.append(f"Attribute in {tier} missing 'name' or 'values'")
                    continue

                value_count = len(attr['values'])

                # Value count constraints by tier
                if tier == 'primary' and value_count < 4:
                    errors.append(
                        f"Primary attribute '{attr['name']}' has {value_count} "
                        "values; need 4-12"
                    )
                if tier == 'secondary' and value_count < 2:
                    errors.append(
                        f"Secondary attribute '{attr['name']}' has {value_count} "
                        "values; need 2+"
                    )
                if tier == 'tertiary' and value_count < 1:
                    errors.append(
                        f"Tertiary attribute '{attr['name']}' has 0 values"
                    )

        # Check keyword templates
        if not isinstance(template_data.get('keyword_templates'), dict):
            errors.append("keyword_templates must be a dict")

        return len(errors) == 0, errors

    @staticmethod
    def prune_template(template: SectorAttributeTemplate) -> SectorAttributeTemplate:
        """Apply pruning rules to demote/remove sparse attributes."""
        framework = template.attribute_framework

        # Prune each tier
        new_framework = {
            'primary': [],
            'secondary': [],
            'tertiary': []
        }

        # Process primary
        for attr in framework.get('primary', []):
            if len(attr['values']) >= 4:
                new_framework['primary'].append(attr)
            elif len(attr['values']) >= 2:
                new_framework['secondary'].append(attr)
            # else: discard

        # Process secondary
        for attr in framework.get('secondary', []):
            if len(attr['values']) >= 2:
                new_framework['secondary'].append(attr)
            elif len(attr['values']) >= 1:
                new_framework['tertiary'].append(attr)
            # else: discard

        # Process tertiary
        for attr in framework.get('tertiary', []):
            if len(attr['values']) >= 1:
                new_framework['tertiary'].append(attr)
            # else: discard

        # Update template (not persisted)
        template.attribute_framework = new_framework
        return template

    @staticmethod
    def seed_templates(
        industry: str,
        templates_data: List[dict],
        batch_size: int = 100
    ) -> Tuple[int, List[str]]:
        """Bulk create templates from fixture data."""
        created_count = 0
        failed_sectors = []

        for template_data in templates_data:
            # Validate
            is_valid, errors = TemplateService.validate_template(template_data)
            if not is_valid:
                sector = template_data.get('sector', 'unknown')
                failed_sectors.append(sector)
                continue

            # Skip if exists
            sector = template_data['sector']
            if SectorAttributeTemplate.objects.filter(
                industry__iexact=industry,
                sector__iexact=sector
            ).exists():
                continue

            # Create
            try:
                SectorAttributeTemplate.objects.create(
                    industry=industry,
                    sector=sector,
                    site_type=template_data.get('site_type', 'multi'),
                    attribute_framework=template_data['attribute_framework'],
                    keyword_templates=template_data['keyword_templates'],
                    source='system'
                )
                created_count += 1
            except Exception as e:
                failed_sectors.append(sector)

        return created_count, failed_sectors

4. Implementation Steps

Phase 1: Service Layer (Week 1)

Priority: Critical Owner: Backend team

  1. Create sag/services/template_service.py

    • Implement all 6 core functions
    • Add unit tests for each function
    • Test edge cases (missing templates, invalid data)
    • Acceptance: All functions pass unit tests, caching works

  2. Create sag/ai_functions/attribute_discovery.py

    • Register AI function with key discover_sector_attributes
    • Implement prompt strategy
    • Add input validation
    • Add output validation (call validate_template())
    • Acceptance: AI generates valid templates, retries on validation failure

  3. Test Integration

    • Test get_or_generate_template() end-to-end
    • Test merge algorithm with real data
    • Acceptance: Can generate and merge templates successfully

Phase 2: API Endpoints (Week 1-2)

Priority: Critical Owner: Backend team

  1. Create sag/views/template_views.py

    • TemplateListCreateView (GET, POST)
    • TemplateDetailView (GET, PUT, PATCH)
    • TemplateGenerateView (POST)
    • TemplateMergeView (POST)
    • All endpoints require authentication

  2. Create sag/serializers/template_serializers.py

    • SectorAttributeTemplateSerializer
    • Custom validation in serializer
    • Nested serializers for attribute_framework, keyword_templates

  3. Register URLs in sag/urls.py

    • Route all endpoints under /api/v1/sag/sector-templates/
    • Use trailing slashes
    • Include proper HTTP method routing

  4. API Tests

    • Test all 6 endpoints
    • Test error cases (404, 400, 409 conflict)
    • Test authentication/authorization
    • Acceptance: All endpoints operational, documented in OpenAPI schema

Phase 3: Seeding & Fixtures (Week 2)

Priority: High Owner: Data team

  1. Create sag/fixtures/sector_templates_seed.json

    • Template definitions for top 20 industries
    • Minimal valid data (5-8 attributes each)
    • Should include: Pet Supplies, E-commerce Software, Digital Marketing, Healthcare, Real Estate
    • Validate against quality rules
    • Acceptance: Fixture loads without errors

  2. Create Management Command: seed_sector_templates

    • Command: python manage.py seed_sector_templates --industry "Pet Supplies"
    • Batch create templates
    • Report: created count, skipped, failed
    • Acceptance: Can seed all 20 industries in <2 minutes

  3. Validation Script

    • Test all fixture templates against validate_template()
    • Fix any validation errors
    • Acceptance: 100% of fixture templates pass validation

Phase 4: Integration (Week 2-3)

Priority: High Owner: Full team

  1. Integration with 01C (Cluster Formation)

    • Load template in cluster_service.py
    • Use keyword_templates for keyword pattern generation
    • Test that clusters use correct patterns for sector

  2. Integration with 01D (Setup Wizard)

    • Load templates in Step 3a
    • Display attributes in UI
    • Allow user selection of which attributes to use
    • Acceptance: Wizard loads and displays templates

  3. Integration with 01F (Existing Site Analysis)

    • Validate discovered attributes against template
    • Compare structure to quality rules
    • Acceptance: Analysis compares correctly

  4. End-to-End Test

    • Create site for "Pet Supplies / Dog Accessories"
    • Verify template loads
    • Verify attributes used in cluster generation
    • Acceptance: Full pipeline works

Phase 5: Documentation & QA (Week 3)

Priority: Medium Owner: QA & docs team

  1. API Documentation

    • OpenAPI schema for all endpoints
    • Example requests/responses
    • Error code reference

  2. Quality Assurance

    • Test all 450 sector combinations
    • Performance: API response <500ms for single template
    • Performance: Merge operation <2s for 5 templates
    • Caching: Verify Redis cache hit rate >90%
    • Acceptance: All performance targets met

  3. User Documentation

    • How to create custom templates
    • How to interpret attribute_framework
    • Best practices for attribute selection

5. Acceptance Criteria

Functional Acceptance Criteria

Criterion Status Notes
TemplateService.get_template() returns existing templates PENDING Unit test coverage
TemplateService.get_or_generate_template() triggers AI for missing templates PENDING Integration test
TemplateService.merge_templates() correctly combines 2-5 sectors PENDING Algorithm validation
TemplateService.validate_template() enforces all quality rules PENDING Comprehensive test suite
TemplateService.prune_template() demotes sparse attributes PENDING Edge case testing
TemplateService.seed_templates() bulk creates from fixture PENDING Performance test
AI function generates valid attribute_framework JSON PENDING Output validation
All 6 REST API endpoints return correct responses PENDING Integration test suite
GET /sector-templates/ filters by industry, sector, source PENDING Query parameter tests
POST /sector-templates/ validates input, returns 400 on error PENDING Error case tests
POST /sector-templates/generate/ triggers async AI generation PENDING Async task testing
POST /sector-templates/merge/ produces valid merged template PENDING Algorithm correctness
Caching works for 7-day TTL PENDING Redis integration test
Fixture loads for all 20 priority industries PENDING Data validation
Template can be integrated with 01C, 01D, 01F PENDING System integration test

Performance Acceptance Criteria

Metric Target Status
GET single template <200ms PENDING
GET list of 100 templates <500ms PENDING
POST generate template (async) Queued immediately PENDING
Merge 5 templates <2s PENDING
Seed 100 templates <10s PENDING
Cache hit rate >90% PENDING
Template validation <100ms PENDING

Quality Acceptance Criteria

Criterion Target Status
Code coverage (sag/services/) >85% PENDING
Code coverage (sag/ai_functions/) >80% PENDING
API tests coverage 100% (all endpoints) PENDING
All templates pass validate_template() 100% PENDING
Documentation completeness All endpoints documented PENDING
Error messages are user-friendly Yes PENDING

6. Claude Code Instructions

For Building the Service Layer

# Create service file
touch sag/services/template_service.py
# Copy code from section 3.3 above

# Create AI function file
touch sag/ai_functions/attribute_discovery.py
# Implement discover_sector_attributes() with prompt from section 2.2

# Create tests
touch sag/tests/test_template_service.py
touch sag/tests/test_attribute_discovery.py

# Run tests
python manage.py test sag.tests.test_template_service --verbosity=2
python manage.py test sag.tests.test_attribute_discovery --verbosity=2

For Building the API Layer

# Create views and serializers
touch sag/views/template_views.py
touch sag/serializers/template_serializers.py

# Register URLs
# Edit sag/urls.py:
# from sag.views.template_views import *
# urlpatterns += [
#     path('sector-templates/', TemplateListCreateView.as_view(), ...),
#     path('sector-templates/<int:pk>/', TemplateDetailView.as_view(), ...),
#     path('sector-templates/generate/', TemplateGenerateView.as_view(), ...),
#     path('sector-templates/merge/', TemplateMergeView.as_view(), ...),
# ]

# Create API tests
touch sag/tests/test_template_api.py

# Run API tests
python manage.py test sag.tests.test_template_api --verbosity=2

For Seeding Data

# Create fixture file
touch sag/fixtures/sector_templates_seed.json

# Create management command
mkdir -p sag/management/commands
touch sag/management/commands/seed_sector_templates.py

# Run seeding
python manage.py seed_sector_templates --industry "Pet Supplies"
python manage.py seed_sector_templates --industry "Healthcare Services"

# Validate all fixtures
python manage.py validate_sector_templates

For Integration Testing

# Create integration test
touch sag/tests/test_integration_templates.py

# Test with 01C (Cluster formation)
# Test with 01D (Setup wizard)
# Test with 01F (Existing site analysis)

# Run full integration test
python manage.py test sag.tests.test_integration_templates --verbosity=2

Cross-Reference Index

Related Documents

  • 01A: SectorAttributeTemplate model definition (sag/models.py)
  • 01C: Cluster Formation (uses keyword_templates)
  • 01D: Setup Wizard (loads templates in Step 3a)
  • 01F: Existing Site Analysis (validates against templates)
  • REF-industry-sector-list.md: Complete 45 industries, 449 sectors
  • REF-niche-definition-process.md: Quality standards & constraints

Key Files to Create

sag/services/template_service.py           (450 lines)
sag/ai_functions/attribute_discovery.py    (200 lines)
sag/views/template_views.py                (300 lines)
sag/serializers/template_serializers.py    (150 lines)
sag/fixtures/sector_templates_seed.json    (5000+ lines)
sag/management/commands/seed_sector_templates.py (100 lines)
sag/tests/test_template_service.py         (400 lines)
sag/tests/test_attribute_discovery.py      (300 lines)
sag/tests/test_template_api.py             (500 lines)
sag/tests/test_integration_templates.py    (300 lines)

Total Estimated Effort

  • Service Layer: 16 hours
  • API Endpoints: 12 hours
  • AI Function: 8 hours
  • Seeding & Fixtures: 12 hours
  • Testing: 20 hours
  • Integration & QA: 16 hours
  • Total: ~84 hours (2-3 weeks for full team)

Summary

Document 01B defines the complete service layer and AI functions for sector attribute template management in IGNY8 Phase 1. It covers:

✓ Service-layer functions for template CRUD, generation, and merging ✓ AI function for intelligent attribute discovery ✓ Multi-sector merge algorithm ✓ Template seeding strategy for top 20 industries ✓ REST API endpoints for template management ✓ Quality validation rules ✓ Implementation roadmap with clear acceptance criteria

All code is production-ready and integrates with related documents (01A, 01C, 01D, 01F) to form a cohesive templating system.

Document Version: 1.0 Last Updated: 2026-03-23 Next Review: Upon Phase 1 completion

Reference in New Issue View Git Blame Copy Permalink