Writing detections once and meaning it
Why I moved my detection rules out of the SIEM and into source control, and what it cost me to do it.
Why I moved my detection rules out of the SIEM and into source control, and what it cost me to do it.
The first detection I ever shipped to production lived inside a Splunk search bar. I tuned it for a week, got it right, and then a month later someone asked me to port it to Sentinel. I had to read my own SPL, work out what it actually meant, and rewrite the logic in KQL from scratch. By the time I was done I was no longer sure the two rules caught the same thing. They probably did not.
That experience is the whole reason I built the detection-as-code pipeline. A rule that lives in one SIEM is stranded there. Every port is a fresh chance to introduce drift, and once two SIEMs drift apart your coverage stops being a fact and becomes a guess. I wanted coverage to be a number I could compute, not a number I had to defend in a meeting.
The fix is to write the rule once, in Sigma, and compile it. The logic lives in one YAML file. Splunk SPL, Elastic ES|QL, and Sentinel KQL are all generated from that one source by pySigma. When I change the rule I change one file, regenerate three queries, and the diff shows up in a pull request like any other code change.
The part that surprised me was how much the format forced me to think clearly. You cannot hand-wave a Sigma rule the way you can fudge a search bar. The logsource has to be right or the conversion picks the wrong pipeline. The selection has to be specific or you get a rule that technically converts and then matches everything. The discipline is annoying at first and then it becomes the point.
There is one failure mode in detection work that quietly wrecks coverage: the rule that compiles cleanly and produces an empty query on the target backend. It passes review. It sits in the SIEM. It never fires, and nobody notices until an incident proves it was dead the whole time.
My CI workflow lints every rule with sigma check, then compiles all of them to all three backends and fails the build on any conversion error. That catches the empty-query problem before merge. It is not glamorous. It is the single most valuable thing the pipeline does.
A green CI run tells me the rule is well formed. It does not tell me the rule actually detects the technique it claims to. Those are different questions and I kept conflating them early on.
So the real validation happens elsewhere. In the companion purple-team lab, Atomic Red Team fires each technique against an instrumented endpoint, and the matching detection has to fire in Wazuh before I promote the rule. CI checks the grammar. The lab checks the meaning. A rule that passes one and fails the other is not done.
Ten rules across credential access, execution, persistence, and defense evasion went through that gate. The LSASS access rule was the one I rewrote the most, because the naive version that keys on access mask 0x1010 alone is a false-positive machine. Tuning it to the masks real dumping tools request, and excluding the legitimate readers, took more iterations than I expected. That is the work. The pipeline just made the work reviewable.
If I were starting a detection program from zero tomorrow, this is the first thing I would stand up. Not the SIEM. The pipeline that keeps the SIEM honest.