Quentin McClellan

Public project · v0.2.0

SRE Log Pipeline: structured incident evidence from noisy operational logs.

SRE Log Pipeline is a Python CLI that distills raw operational logs into structured JSONL incident evidence. It normalizes events, filters by benchmark window, produces incident artifacts and markdown reports, and exposes output to StarkGrid and AI clients through defined adapters.

Review proof Pipeline architecture Operator Evidence MCP View on GitHub

Problem

Operational logs are noisy, unstructured, and difficult to share across tooling boundaries.

Incident review and postmortem workflows require structured evidence: normalized events, a defined time window, bounded results, and a format that can be consumed by downstream tools without manual cleanup. SRE Log Pipeline was built to turn raw JSONL log streams into reviewable, shareable incident evidence artifacts.

Pipeline architecture

Normalization, filtering, distillation, and structured output.

Log ingestion Reads raw JSONL log streams. Handles batch processing and per-file streaming. No runtime dependencies.
Benchmark-window filtering Timezone-aware timestamp filtering by benchmark window. Tested for fractional metadata and edge cases.
Incident distillation Produces structured incident evidence conforming to incident-evidence.json schema v1.0. Captures record counts, metrics, and metadata.
JSONL summary Summarizes processed log batches into structured JSONL output for further analysis or storage.
Markdown report Generates human-readable markdown incident reports from distilled evidence. Supports postmortem and on-call review workflows.
StarkGrid adapter Translates distilled incident evidence into StarkGrid-compatible format. Enables ingestion of pipeline output into the StarkGrid evidence platform.
Storage Handles artifact persistence for distilled evidence files. Structured output directory conventions.
Runtime validation Jenkinsfile, Docker Compose, and smoke test suite validate end-to-end pipeline behavior in CI.

Proof evidence

v0.2.0 with incident schema v1.0, 8 test files, and a StarkGrid adapter.

SRE Log Pipeline reached v0.2.0 with a stable incident evidence schema (v1.0), 8 test files covering all major pipeline behaviors, and a StarkGrid adapter that makes the pipeline's output consumable by the StarkGrid evidence platform.

The pipeline was scaffolded from Forge's python-worker template, demonstrating the template's structural discipline: the scaffolded project reached a tagged v0.2.0 release with zero runtime dependencies and a full Jenkinsfile.

Test coverage

  • test_batch.py — batch processing behavior
  • test_benchmark_window.py — timezone-aware window filtering
  • test_incident_distillation.py — incident schema output
  • test_jsonl_summary.py — JSONL summary output
  • test_markdown_report.py — markdown report generation
  • test_starkgrid_adapter.py — StarkGrid format translation
  • test_storage.py — artifact persistence behavior
  • test_smoke.py — end-to-end smoke validation

Supporting project · Operator Evidence MCP · v0.1.1

Read-only incident evidence access for AI clients.

Operator Evidence MCP is a read-only Model Context Protocol server that exposes bounded incident evidence and operational summaries to AI clients. It was built to allow AI tools to query distilled incident evidence without granting write access, execution authority, or access to private infrastructure details.

The server reads SRE Log Pipeline output: evidence sources, incident runs, summaries, trace samples, and individual incidents. All responses are bounded to prevent context overflow. Path confinement and slug validation prevent traversal outside the evidence directory.

Security boundary

  • All tools are read-only. No write, create, delete, or execute tools exist in the contract.
  • Slug and path validation on all source and run_id parameters — tested in test_store.py.
  • Results are bounded: limit parameters and truncation prevent unbounded response growth.
  • Version reported via MCP serverInfo — tested in test_incident_store.py.

Current limitations

What SRE Log Pipeline does not claim.