# Data Flow Diagrams (Narrative)

## Purpose
Describe end-to-end data movement through the system based on current routing, middleware, and model conventions. No diagrams are embedded; flows are explained textually.

## Code Locations (exact paths)
- Request routing: `backend/igny8_core/urls.py`
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/middleware/resource_tracker.py`
- DRF base viewsets: `backend/igny8_core/api/base.py`
- Authentication classes: `backend/igny8_core/api/authentication.py`
- Tenancy models: `backend/igny8_core/auth/models.py`
- Celery configuration: `backend/igny8_core/settings.py`

## High-Level Responsibilities
- Trace how HTTP requests are processed, tenant-scoped, authorized, and persisted.
- Show where async processing departs to Celery and where responses are shaped.

## Detailed Behavior
- Incoming request → Django middleware stack:
  - Security/WhiteNoise/CORS/session/common/CSRF/Django auth.
  - `RequestIDMiddleware` assigns `request.request_id` and returns it in `X-Request-ID`.
  - `AccountContextMiddleware` resolves user/account (session or JWT) and enforces active plan; sets `request.account`.
  - `ResourceTrackingMiddleware` optionally tracks resource usage for admin/developer users when the `X-Debug-Resource-Tracking` header is true; adds `X-Resource-Tracking-ID`.
- URL dispatch via `urls.py` routes to module routers (auth, account, planner, writer, system, billing, automation, linker, optimizer, publisher, integration) under `/api/v1/*`.
- DRF viewset pipeline:
  - Authentication classes (API key → JWT → session → basic) establish `request.user` (and optionally `request.account`/`request.site`).
  - Base viewsets (`AccountModelViewSet`, `SiteSectorModelViewSet`) filter querysets by `account`/`site`/`sector` and attach `account` on create.
  - Serializers handle validation; responses are wrapped by unified helpers to standardize success/error payloads and pagination.
- Persistence:
  - Tenant-scoped models inherit `AccountBaseModel` or `SiteSectorBaseModel`; save hooks enforce account/site/sector alignment.
  - Soft deletion is used where models implement `soft_delete`, respecting retention windows from account settings.
- Async/Background:
  - Celery uses Redis broker/backend; tasks inherit JSON payloads and time limits from `settings.py`.
  - Automation, AI, publishing, and billing tasks enqueue via Celery; results return through database/state updates, not synchronous responses.
- Response:
  - Unified response wrappers ensure `success`, `data`/`error`, and request ID are present; paginated responses include `count/next/previous/results`.
  - Throttling headers apply per-scope (as configured in `REST_FRAMEWORK` throttles).

## Data Structures / Models Involved (no code)
- Tenancy bases: `AccountBaseModel`, `SiteSectorBaseModel`.
- Core entities: `Account`, `Plan`, `Site`, `Sector`, `User`, `SiteUserAccess`.
- Module-specific models follow the same tenancy bases (documented in module-specific files).

## Execution Flow
1) HTTP request hits middleware; IDs and tenant context are set.  
2) DRF authentication authenticates and sets user/account/site.  
3) Viewset filters data by tenant/site/sector and runs serializer validation.  
4) DB operations persist data with enforced tenant alignment.  
5) Optional Celery tasks are queued for long-running work.  
6) Response returns unified JSON with request IDs and optional throttling/pagination headers.

## Cross-Module Interactions
- Auth context set in middleware is consumed by all module viewsets for scoping.
- API key auth provides site context for integration/publisher flows.
- Celery configuration is shared by automation/AI/publishing/billing task modules.

## State Transitions (if applicable)
- Entity lifecycle changes (create/update/delete/soft-delete) flow through base viewsets and tenancy bases, ensuring account/site/sector consistency.
- Request lifecycle includes request ID creation, optional resource tracking, and unified response wrapping.

## Error Handling
- Middleware can short-circuit with JSON errors for missing account/plan.
- Viewset overrides wrap validation and server errors into unified responses; missing objects return 404 payloads.
- Throttling (scope-based) returns standard DRF throttle responses with headers.

## Tenancy Rules
- All tenant-bound data flows require `request.account`; filtering and save hooks prevent cross-tenant access.
- Admin/developer/system-account users may bypass tenant filtering; system accounts are guarded against deletion.
- Site/sector alignment is validated on save for models inheriting `SiteSectorBaseModel`.

## Billing Rules (if applicable)
- Plan activation is validated in middleware. Credit debits and billing workflows occur in billing modules (covered elsewhere) after tenant resolution.

## Background Tasks / Schedulers (if applicable)
- Celery broker/backend configuration in `settings.py` governs async flow; tasks should include account/site identifiers to maintain scoping.

## Key Design Considerations
- Request ID and resource tracking enable traceability and performance debugging.
- Middleware ordering ensures tenant context precedes view logic.
- Unified response format keeps clients consistent across modules.

## How Developers Should Work With This Module
- Preserve middleware order; new middleware must not break request ID or tenant context.
- Ensure new viewsets inherit the base classes to pick up scoping and unified responses.
- When adding async tasks, include tenant/site identifiers and respect Celery limits from settings.