This post was originally published on The Data Source on July 5th, 2023, my monthly newsletter covering the top innovation in data infrastructure, engineering and developer-first tooling. Subscribe here and never miss an issue!
Security has always been top of mind for infrastructure leaders and 2023 is no different. Cyber threats are becoming more complex and sophisticated, with an increasing number of attack vectors. These threats can infiltrate various data sources, networks, applications, endpoints, and cloud infrastructure.
With the increasing adoption of cloud services and hybrid infrastructure, there is a strong push to enhance security measures with smarter and more robust tools. From my conversations with security practitioners, I find that there is an increased focus on leveraging AI and machine learning to develop techniques that can automatically learn and adapt to emerging threats. And, a growing appetite for adopting automation and orchestration frameworks to reduce manual effort in detecting and mitigating threats that traditional tools struggle to handle.
What has been the most interesting to me is learning all about how companies today are securing their most important assets. While the concept of detection engineering is not new, I found that companies such as Palantir, Datadog, Brex, and others have all established a core practice in codifying detection policies to improve security posture and enable early detection of malicious activities. This practice is called Detection as Code.
Detection as Code aligns security with DevOps, treating security detection and monitoring workflows as code. It empowers security practitioners to define, implement, and manage their detection capabilities through code, automating deployment and configuration for seamless operation.
By implementing detection as code, security teams borrow from core software engineering principles like Continuous Integration & Deployment (CI/CD) version control, change management, software testing and more. These principles have revolutionized the way software is built, tested and deployed. When applied to the security world, teams are empowered to leverage similar workflows to manage their detection rules. Instead of creating, improving and sharing detection content through disconnected consoles and interfaces, security analysts can leverage tools that fit the GitHub Flow model for software development.
One of the most intriguing findings from successful implementations of Detection as Code (see how it’s been done at Palantir and Brex) is how well it complements internal incident management strategies. From vulnerability management to threat intelligence to incident response, Detection as Code enables teams to leverage outputs like alerts, logs, and behavioral analytics to prioritize and triage security incidents. For example, detection rules for account takeovers, credential resets and more have been crucial in the identification of potential threats. This approach enables security teams to establish scalable and repeatable ways in which detection content for detection and response is written, tested and deployed.
Security Information and Event Management (SIEM) and Security Operations Center (SOC) are critical components of the security stack. Today, things are changing with detection engineering: It's shaking up traditional SIEM and SOC processes, making them more efficient at preventing and resolving security issues.
In the past, SIEM was all about passively looking at logs, generating alerts and reacting to incidents as they came up, while SOC involved lots of manual work around orchestrating people, processes and tools for the purpose of investigating cyber threats. But now, detection engineering enables SIEM solutions to be more proactive in their approach. Instead of relying on static rules and log data, teams leverage advanced analytics to get real-time visibility into their security posture. And SOC is getting a makeover too, thanks to automation and orchestration frameworks. By enabling teams to develop and deploy detection rules and algorithms as code, teams are better equipped to manage, and continuously iterate on their log management, alert prioritization and incident response workflows.
Other solutions include Expel, Anvilogic, Sekoia, and SIEM Rules.
Building the Threat Detection Ecosystem at Brex by Julie Agnes Sparks
“We build our detections with a layered approach — they exist first as a collection of signals that represent an individual event or action which then are aggregated together to demonstrate a technique or attacker behavior. Since signals represent behaviors of interest, they are also useful outside of detection engineering and can be applied to incident response, threat hunting, and threat intelligence functions. Analysts can review signals to reveal key events that were taken, pull out event level behavioral patterns, or assist in security investigations. By building signals on one specific behavior and linking multiple signals together to model an attack, Detection & Response teams can create more flexible alerts.”
Table stakes for Detection Engineering by Zack Allen
“What has changed is that these experts need a solid basis in software engineering principles. You can't scale all those detections and deploy them in a modern environment, manage sprints (yes, this is software engineering :)), or write unit, integration, and regression tests without lots of bodies or automation. I can reliably say my boss would rather hear that I can scale the problem away with software than with hiring more people. Lastly, and I think this is the next step in the evolution of security research to detection engineering: we all must improve the explainability, and thus impact, of our rules, and statistics is how you do it. You can't reliably create, improve, deprecate or justify your detections to your sales teams, internal leadership, or customers without a background in statistics. This does not mean you need a graduate degree. Still, I think if security engineers and researchers spent some time looking at concepts like sampling bias and error, confusion matrices, precision, and recall, they could better understand how rules perform under certain conditions and spot errors much earlier before a rule hits production.”
Lessons Learned in Detection Engineering by Ryan McGeehan
“Hunting should be efficient, as should any on-call time being spent diving into an alert. Instead of burying yourself in browser tabs for different investigative tools, many stronger detection teams are integrating their tools in a single place. The problem set of detection is a complicated integration challenge. So why don’t more detection teams focus on integration testing? This is an incredibly interesting space in intrusion detection, to treat detection the same way you’d treat a build pipeline supported by CI/CD platforms like Jenkins.”
Security practitioners and startup builders, if this is an area of interest to you, please reach out to chat!
