# Automation Logging

## Purpose
Explain how automation runs are logged, where artifacts are written, and how logs are retrieved via API.

## Code Locations (exact paths)
- Logger implementation: `backend/igny8_core/business/automation/services/automation_logger.py`
- Service usage: `backend/igny8_core/business/automation/services/automation_service.py`
- API exposure: `backend/igny8_core/business/automation/views.py` (`logs` action)
- Run metadata: `backend/igny8_core/business/automation/models.py`

## High-Level Responsibilities
- Create per-run directories and log files for automation runs (primary and optional shared mirror).
- Record stage-level start/progress/complete events and structured traces.
- Provide tail retrieval of logs through an API endpoint.

## Detailed Behavior
- `AutomationLogger`:
  - On `start_run`: generates `run_id` (`run_YYYYMMDD_HHMMSS_<trigger>`), creates run directory under `base_log_dir` (default `/data/app/logs/automation`), and optionally mirrors to `shared_log_dir` when configured. Initializes `automation_run.log` with run metadata and writes a JSONL trace entry (`run_trace.jsonl`).
  - `log_stage_start`: appends to main log and writes a stage-specific log file `stage_<n>.log`; mirrors to shared dir if configured; writes a trace event (`stage_start` with pending count).
  - `log_stage_progress`: appends progress lines to main and stage logs (and mirrored copies), and writes a `stage_progress` trace event.
  - `log_stage_complete`: writes completion info with processed count, time elapsed, credits used to main/stage logs; emits trace (`stage_complete`).
  - `log_stage_error`: writes error lines to logs and emits an error trace.
  - `append_trace`: helper used by the service for per-item events; appends JSONL entries.
  - `get_activity_log`: reads and returns the tail of `automation_run.log` (used by API).
- `AutomationService` uses the logger at every stage boundary and significant progress point; also logs warnings/errors and mirrors events via the logger’s trace helpers.
- Stage result data is stored in `AutomationRun.stage_n_result` JSON fields alongside logs, providing structured state in the DB.

## Data Structures / Models Involved (no code)
- `AutomationRun`: stores per-stage result JSON and run status used to correlate with log files.

## Execution Flow
- Run start → `start_run` creates directories/files → each stage emits start/progress/complete/error logs and traces → API `logs` endpoint reads `automation_run.log` tail for a given `run_id`.

## Cross-Module Interactions
- Logging uses the automation run/account/site context; no external modules write to these files. AI stages provide progress data; billing is inferred from AI task logs but not logged here.

## State Transitions
- Log files accumulate throughout a run; traces document run lifecycle events. Completion or failure is also reflected in `AutomationRun` status/results.

## Error Handling
- Logger operations are best-effort; exceptions during file/trace writes are swallowed to avoid breaking the run.
- API `logs` returns 404 if the run does not exist; otherwise returns the requested number of lines.

## Tenancy Rules
- Run directories are keyed by account/site/run; API access is constrained by site ownership through the viewset query for `AutomationRun`.

## Billing Rules
- None directly; credit use is recorded in stage result JSON and inferred from AI task logs, not in the file logs.

## Background Tasks / Schedulers
- Logging occurs inside Celery-executed stages; no separate scheduled logging tasks.

## Key Design Considerations
- File-based logs per run make multi-stage processes auditable and debuggable without DB bloat.
- Optional shared log dir supports centralized aggregation when multiple containers run workers.
- JSONL traces allow machine parsing for UI status feeds while text logs support human debugging.

## How Developers Should Work With This Module
- When adding stage steps, emit progress via `AutomationLogger` to keep observability consistent.
- Keep log writes best-effort; do not raise from logger helpers.
- Ensure any new API log endpoints read from the same per-run directory structure and remain site/account scoped.