Test Driven Security in the DevOps pipeline

The myth of attackers breaking through layers of firewalls or decoding encryption with their smartphones makes for great movies, but poor real world examples. In the majority of cases, attackers go for easy targets: web frameworks with security vulnerabilities, out of date systems, administration pages open to the Internet with guessable passwords or security credentials mistakenly leaked in open source code are all popular candidates. The goal of Test Driven Security is to take care of the baseline: apply elementary sets of controls on applications and infrastructures, and test them continuously, directly inside the DevOps deployment pipeline.




A baseline of security controls defines the minimal requirements applications should match before being deployed to production. The controls are simple and specific, such as:

- All websites must implement a Content Security Policy

- Form submission must require CSRF tokens, unless explicitely whitelisted

- SSH Root login must require sudo on all systems

- The rules in firewalls and security groups must be tested at every deployment

- HTTP traffic is prohibited, HTTPS endpoints must use Mozilla's modern guidelines

- Outdated and vulnerable dependencies must be upgraded




The list of security best practices is established by the security team with the help of developers and operators to make sure everyone agrees on their value. A list of baseline requirements can be assembled quickly by collecting those best practices and adding some common sense. The controls themselves are simple and do not require particular expertise, the difficulty comes from testing and implementing them everywhere and all the time.




This is where Test Driven Security comes in. TDS is a similar approach to Test Driven Development (TDD) which recommends developers to write tests that represent the desired behavior first, then write the code that implements the tests. TDS proposes to write security tests first, thus representing the expected state, and then implement the controls that pass the tests.




The TDS approach brings several benefits:


1. Writing tests forces security engineer to clarify and document expectations. Engineers can build products with the full knowledge of the required controls rather than catching up post-implementation.


2. Controls must be small and very specific units which are easy to tests. Vague requirements such as “encrypt network communication” are avoided, instead we will prefer the explicit “enforce HTTPS with ciphers X, Y and Z or all traffic”, which clearly states what is expected.


3. Re-usability of the tests across products is high, as most products and services share the same base infrastructure. Once a set of baseline tests is written, the security team can focus on more complex tasks.


4. Security regressions are caught in real-time, prior to deployment, rather than periodically during manual reviews.




Julien Vehent
Firefox Operations Security Lead, Mozilla
Julien leads the Firefox Operations Security team at Mozilla, tasked with defining, implementing and operating the security of Firefox's backend services and release engineering infrastructure. Julien's background is in web applications security, services architecture

-

Managed by the official OWASP Media Project https://www.owasp.org/index.php/OWASP...