Detection engineering vs rule writing
Rule writing produces a condition. Detection engineering produces a condition, a test suite, an ATT&CK mapping, a triage question set, a response playbook, and a git history showing every change and why it was made. The difference matters when an auditor asks how you detected an incident or when a false positive costs an analyst two hours.
Detection-as-code means detections live in git, are reviewed like code, and are tested before they reach production. The YAML format enforces that every detection ships with its metadata — severity, domain, ATT&CK technique, triage questions — not as an afterthought but as a compile-time requirement.
Detection types: streaming, correlation, behavioural
Streaming detections fire on single events: a specific event class, field match, and threshold. They are deterministic, fast, and easy to test. Use them for high-signal atomic indicators: new admin account created, MFA disabled, public S3 bucket policy applied.
Correlation detections fire on sequences or aggregations across multiple events in a time window. Use them for multi-stage attack patterns: T1078 (Valid Accounts) followed by T1059 (Command and Scripting Interpreter) within 30 minutes on the same host.
Behavioural detections fire when an entity's current behaviour deviates significantly from its own historical baseline. They require the entity graph and are the only type that can detect novel techniques without a known indicator. Use them for lateral movement, data exfiltration volume anomalies, and insider threat patterns.
# detections/cloud/mfa_disabled.yaml
key: cloud.mfa_disabled
domain: identity
severity: high
type: streaming
where:
class_uid: 3005 # API Activity
operation: "DisableMFA"
outcome: success
attck:
tactic: TA0006 # Credential Access
techniques: [T1556]
triage_question_set: cloud.mfa_disabled.questions
response_playbook: playbooks/identity/mfa_disabled.md
stage: activeWriting testable detections
Every detection needs at least three tests: a true-positive fixture (an event that should fire), a true-negative fixture (a similar event that should not), and a boundary fixture (an event on the exact threshold). Tests live in tests/ alongside the detection YAML and run in CI on every PR.
Fixtures are synthetic events in OCSF JSON. They do not require production data — they exercise the detection logic in isolation. A detection that can't be tested with a synthetic fixture is under-specified.
Backtesting runs the detection logic against a 30-day production event sample and reports how many times it would have fired, how many of those would have been true positives (based on existing case verdicts), and the expected FP rate. A detection with an FP rate above 20% does not promote to staging.
The promotion pipeline
The promotion pipeline has four stages: draft (local development), staging (runs in production data, recommend-only), active (runs in production, fires real cases), and archived. Promotion from staging to active requires: CI green, backtest FP rate below threshold, peer review, and SecOps lead approval.
Every promotion is a signed commit. The commit message includes the backtest summary, the reviewer, and the effective date. This is the audit trail that answers 'when did this detection go live and who approved it?'
Coverage gap analysis with MITRE ATT&CK
Map every active detection to its ATT&CK technique. Generate a heatmap monthly. Techniques with no coverage are explicit risks — prioritise new detection development against your most likely threat actors' known TTPs.
ManySignal's coverage report generates this heatmap automatically from the detection catalogue. Export it as a PDF for board-level risk reporting or as JSON for integration with threat-modelling tools.
Focus coverage gap work on techniques relevant to your industry vertical. A healthcare org should prioritise T1078 (Valid Accounts) and T1190 (Exploit Public-Facing Application) over techniques used primarily in ICS attacks.
Tuning: reducing FPs without losing TPs
Every FP is a tuning opportunity. When an analyst closes a case as false positive, the case timeline captures the rationale. Use those rationales to add exclusions, raise thresholds, or add a resolver to the question set that would have caught the false context.
Never tune by raising the severity threshold. That suppresses visibility. Tune by adding specificity: an additional field match, an entity-graph condition, or a baseline check that distinguishes the FP from the TP.
Key takeaways
- Detection engineering produces a complete artefact: condition, tests, ATT&CK mapping, question set, playbook.
- Three detection types: streaming (atomic), correlation (sequence), behavioural (baseline anomaly).
- Every detection needs three tests: TP fixture, TN fixture, boundary fixture.
- Promotion requires green CI, acceptable backtest FP rate, peer review, and lead approval.
- Monthly ATT&CK coverage heatmaps drive detection development priorities.
- Tune FPs by adding specificity — never by suppressing severity.