Sky’s the Limit: Stratus Red Team for Azure
Contents
Sky’s the Limit: Stratus Red Team for Azure#
2022-06-23
Trust, but Verify#
As a purple teamer, I aim to bring both red and blue closer together. Purple teamers need to demonstrate relevant attacks in ways that are consumable and actionable for defenders. This often involves asking the following questions:
Can we observe the attack through our sensing systems?
Can we validate that the relevant telemetry is useful for detection or response purposes?
Did we detect what we expected?
Can we capture the salient features of the attack to bootstrap novel detection logic?
Can we identify the cause of a missed detection?
It is rarely sufficient to run through a simulated attack just once. Re-running attacks is necessary to validate that any changes made actually improve security posture. Every software developer will tell you that even seemingly small changes to a program or its infrastructure can result in unforeseen and adverse consequences. This is especially true for “the cloud”, which is always in a state of flux.
Due diligence for defenders means more than just answering can we detect this threat?, but also routinely asking can we still detect this threat?
I firmly believe that defenders will need to adopt some of the testing practices from modern software development to keep up with the pace of change in our technology stack. As such, we will need easily reproducible automated attacks that are suitable as test cases. While projects like Atomic Red Team offer automated attack test suites for the endpoint, fewer open source tools exist to do the same for the cloud.
What is Stratus Red Team?#
In January 2022, a passionate team at Datadog released the Stratus Red Team (stratus) project. Stratus Red Team allows the community to test threat detection systems through automated attacks against cloud infrastructure deployed in their own environments. The official documentation highlights similarities and differences between stratus and earlier projects, such Atomic Red Team, Pacu, and CloudGoat.
Stratus Red Team comes with a test suite of automated attacks that are mapped against the techniques on MITRE’s ATT&CK Framework. stratus is opinionated about the kinds of automated attacks included in the public test suite, so each is granular and actionable as a test case. Check out the Attack Techniques Reference in the documentation to see the breadth of options for cloud providers and attack techniques.
Stratus Red Team provides both attack and infrastructure automation. It ensures that properly configured cloud infrastructure is deployed in a given environment to act as the “victim” of the automated attack. This allows defenders to observe the automated attack using their actual threat detection stack, but without the risk of interferring with production/mission-critical systems.
Under the hood, Stratus Red Team uses terraform to manage the deployment of test infrastructure. It includes a terraform module for each automated attack, all of which are packaged into a compiled, single-file Golang executable. The combination of terraform and Golang makes stratus portable across a variety of test runners and cloud providers.
The automated attacks in stratus are implemented using a simple interface: each has a detonate function that takes the output from the corresponding terraform module as its arguments. The logic that performs attack actions against the terraform-managed test infrastructure in the detonate functions is usually written with the official Golang SDK for the relevant cloud service.
After the attack is completed, stratus can automatically destroy the victim infrastructure. Similar to terraform, Stratus Red Team shares the concept of a provider, which is an abstraction to handle configuration and authentication for a given cloud service. The v1.x releases of stratus shipped with dozens of automated attacks against AWS and Kubernetes.
If you are interested in a more detailed walkthrough of the architecture for Stratus Red Team, please read this blog post from the maintainer @christophetd.
Testing with Stratus Red Team#
My colleagues and I started using Stratus Red Team within a few weeks of its initial release. We wanted to use stratus to challenge assumptions about our detection coverage and validate the behavior of our detection systems.
To correlate the telemetry generated by the automated attack, my colleague wrote a simple bash script wrapped the stratus command-line interface. This script was responsible for creating a unique IAM role session name and recording the start and end times for each detonation in a JSON file. It also copied out the terraform state files between the warmup and detonate steps, so we could associate the randomly-generated identifiers assigned to ephemeral infrastructure with each detonation.
I wrote Jupyter Notebooks to parse the JSON files created by the bash script and generate search queries to find related telemetry in an XDR database. For each detonation, my Jupyter Notebook created an XDR investigation containing the relevant events and alerts from our search results. We then opened a ticket with the respective team and linked the corresponding XDR investigation for their review. Lastly, if we knew specific detectors that should fire on the automated attack, we could use the Jupyter Notebooks to assert that the expected alerts were present in our investigation.
Beyond Pass or Fail#
There is often much to learn from running automated attacks regardless if the test case passes or fails.
The automated attacks in Stratus Red Team helped highlight where our detection systems and logic were working properly. But even when stratus validated our assumptions, it was still useful to capture the relevant telemetry in XDR investigations. The XDR investigations we created from the automated attacks made for excellent training opportunities for analysts. Upon manual review, human analysts found novel ways to detect the automated attack that the original detection logic did not previously cover.
While the term failure has a negative connotation in everyday life, failure is hugely positive in the context of software testing and validation. A failing test that is fixed before shipping to production can prevent undue harm to ourselves and those we have a duty to protect.
Testing helps us identify problems before they become costly. The process of writing and fixing test cases provides an opportunity to learn something new about the complex detection systems we rely on.
We did find a missed detection during our early experiments with Stratus Red Team. Engineers from the responsible team were quickly able to identify the cause of the problem after reviewing the XDR investigation we created for the automated attack. But most importantly, the engineers asked “how can we re-run this attack so we know if the problem happens again in the future?” This was the moment when I realized the potential of automated attack testing tools like Stratus Red Team. We could easily answer how to reproduce the attack using the portable stratus binary from their existing CI/CD pipelines.
Everyone wins when there is a convenient and effective test case to guarantee we won’t make the same mistake twice. I believe that purple teams will be most successful when we can measure our work in terms of the automated attack test cases that we produce and the problems we identify with those test cases over time.
Contributing the Azure Provider#
My purple team successfully used Stratus Red Team to answer two key questions: “can we detect this?” and “can we still detect this?” And I bet others will find it useful too. It is only fair that we found a way to contribute something back to the project and the community.
Most of my customers are heavily integrated with Microsoft cloud services, but the v1.x release of stratus did not come with a provider for Azure. It was obvious that an Azure provider for stratus would be useful and an investment in the future that pays dividends quickly. It was also a great opportunity to write some Golang, which is not my comfort zone after a decade of Python development.
I am super proud to share that my pull requests adding support for Azure and two attack techniques (azure.execution.vm-run-command and azure.execution.vm-custom-script-extension) for Stratus Red Team have been merged into main. And better yet, this small contribution coincided with the milestone of the project’s v2.0 release.
I hope others in the community will benefit from this new functionality. I encourge everyone to expand upon the repertoire of attack techniques in Stratus Red Team.