Solution — Detection as code
Detection as code that treats rules like the production software they are
Sigma rules in a git repo, pull-request review, backtests on 30-90 days of historical events, staged shadow deployment, and effectiveness telemetry annotated back onto every rule.
Sigma
Native rule format — portable across data platforms
30-90d
Historical replay window on every pull request
Shadow
Deploy tier measures live volume before analysts see the rule
MITRE
ATT&CK coverage diff fails PRs that regress technique coverage
The detection lifecycle, run like a software delivery pipeline
Every rule is a file. Every change is a commit. Every commit is reviewed, tested against real historical events, shadow-deployed, and measured in production — with the results annotated back onto the rule so the next change starts with the truth.
Sigma-native rule repository
Detections live as Sigma YAML in a git repo — one file per rule, with metadata for MITRE ATT&CK technique, data source, author, severity, and reference links. The repo is the source of truth; the platform pulls from it, never the other way around.
PR-based review with codeowners
Every rule change goes through a pull request. CODEOWNERS routes threat-hunt rules, cloud rules, and identity rules to the right reviewer. Approvals, discussion, and merge history are the audit trail no SIEM console gives you.
Backtest harness on historical events
Before merge, the CI harness replays 30 to 90 days of production telemetry against the proposed rule and reports match count, entity spread, and estimated alert volume — so tuning happens before analysts get paged, not after.
CI checks for syntax and coverage regressions
Sigma lint, field-mapping validation against the platform's schema, and a MITRE ATT&CK coverage diff run on every PR. Removing a rule that was the only coverage for a technique fails the build with a named regression.
Staged deployment pipeline
Merged rules deploy to a shadow tier first — they fire into a silent stream that measures true volume and entity spread on live data. Promotion to the production alert queue is a second, gated step with automatic rollback on volume blow-out.
Effectiveness telemetry on top
Each production rule reports fire count, true-positive rate from analyst dispositions, mean time to close, and suppression coverage — surfaced back into the repo as annotations so the next PR sees the operating reality of what it is changing.
SIEM console editing vs detection-as-code
| SIEM console rule editing | ManySignal detection-as-code |
|---|---|
| Rules edited directly in the SIEM console with no diff or review | Every rule change is a git commit reviewed via pull request |
| No test before deploy — production is the test environment | Backtest replays 30-90 days of historical events before merge |
| Nobody knows who wrote a rule, when, or why | git blame answers the question in one command |
| Removing a rule silently deletes coverage for a MITRE technique | CI fails the PR with a named ATT&CK coverage regression |
| A noisy rule pages the on-call before anyone notices | Shadow deployment measures live volume before promotion |
| Rule effectiveness is a spreadsheet someone updates quarterly | Fire count, TP rate, and MTTR annotate the rule in the repo |
What a detection PR looks like
- Sigma YAML diff with MITRE ATT&CK technique metadata
- Sigma lint pass and field-mapping validation against the platform schema
- Backtest replay results — match count and entity spread over 90 days
- Estimated alert volume at current triage capacity
- MITRE ATT&CK coverage diff — techniques added, retained, regressed
- CODEOWNERS-driven review routing to the right detection engineer
- Shadow deployment plan with volume-blow-out rollback threshold
- Effectiveness annotations from the last 30 days of related rules
Detection as code — buyer questions
What is detection-as-code?
Detection-as-code is the practice of managing security detection rules the same way software engineering teams manage application code: version-controlled in git, changed via pull request, tested with unit and integration tests, deployed through CI/CD with staged rollout, and monitored for effectiveness in production. The goal is to give detection engineering the same reliability, reviewability, and rollback capability that modern software delivery already has.
Why does editing rules in the SIEM console fail at scale?
Console editing has no diff, no review, no test, no rollback, and no history beyond a per-rule audit stamp. A single fat-fingered field name can drop coverage for a whole technique and nobody notices until an incident. Detection-as-code fixes each of those failures directly — pull-request review catches the fat-finger, CI backtesting catches the volume blow-out, and git history answers every 'who changed this, when, and why' question in seconds.
What rule format does ManySignal use?
ManySignal is Sigma-native. Rules are Sigma YAML in a git repository, with metadata for MITRE ATT&CK technique, data source, severity, author, and reference links. Sigma keeps rules portable across data platforms and lets teams share community rule packs without a translation step. The platform also supports correlation rules and identity-graph rules as first-class file types in the same repo.
How does the backtest harness work?
When a rule is added or modified in a pull request, the CI harness replays 30 to 90 days of production telemetry against the proposed rule and reports match count, distinct entity spread, and estimated alert volume at the current triage capacity. Reviewers see the actual operating characteristics before the rule ever fires against a live event, so tuning happens on the PR instead of during an alert storm.
What does staged deployment look like?
Merged rules deploy to a shadow tier first. In shadow, the rule fires into a silent stream that measures true volume, entity spread, and correlation with existing detections on live data — no analyst gets paged. A separate, gated promotion step moves the rule into the production alert queue, and an automatic volume-blow-out check rolls the promotion back if fire rate exceeds the projection by a configurable multiple.
See the detection repo and CI pipeline
We will walk through the ManySignal detection repo, a live PR with backtest output, and the shadow-to-production promotion flow — bring one of your own noisy rules and we'll rewrite it on the call.