Four gates before merge, and why they run apart
A CI pipeline that fans security checks out into separate jobs, so a red build tells you which gate tripped instead of handing you one long log to scroll.
A CI pipeline that fans security checks out into separate jobs, so a red build tells you which gate tripped instead of handing you one long log to scroll.
There is a version of a secure pipeline that is one giant job: run every scanner, dump everything into one log, and color the build red if anything trips. I built that version first. It works, technically. The problem shows up the moment a build fails and someone has to figure out why. You scroll. You search the log for the word error. You find three of them and none is the one that mattered.
So I split it. The pipeline now runs the checks as separate jobs, and a red build names the gate that failed. SAST is its own job. The secret scan is its own job. The dependency audit is its own job. When the build goes red, the failing job is the answer, not the start of a search.
lint runs first as a cheap fail-fast, because there is no point spending Semgrep minutes on code that does not even pass ruff. Once lint is green, the three security scans fan out in parallel: Semgrep for SAST, gitleaks for committed secrets, pip-audit for known-vulnerable dependencies. test waits on all three. The SOC notification runs last with if: always(), so a failure gets reported, not just a clean run.
That always() is deliberate. The whole reason to tell the SOC about CI is the failures. A pipeline that only reports green is a pipeline that hides exactly the events worth hearing about.
Semgrep emits SARIF, and the pipeline uploads it so findings land in the repo's Security tab and as inline PR annotations. This matters more than it sounds. An engineer reviewing a pull request sees the finding next to the offending line, in the place they are already looking. A finding buried in a job log is a finding nobody reads.
I also wrote four custom Semgrep rules for cases the standard packs miss in this codebase: Flask debug=True, subprocess with shell=True on a non-literal argument, jwt.decode with verification disabled, and a binding to all interfaces. None is exotic. All four are the kind of thing that slips through review when everyone is tired.
The last job posts the run outcome to a Shuffle webhook: repo, commit SHA, actor, status, link back to the run. In the lab that webhook feeds my SOC automation lab, so a failed security gate opens a TheHive case the same way a Wazuh alert does. A blocked merge becomes a thing the SOC can see, not a private matter between an engineer and a red checkmark.
If the webhook secret is unset the job no-ops instead of failing the build, which is the right default. Security tooling that breaks the build when its optional integration is misconfigured teaches people to rip the tooling out.
The pipeline that proves the source is clean is half the chain of custody. The other half, proving the artifact you ship is the one you built, is a separate problem, and it is the one I picked up next.