Michael BoganAn Overview of DevSecOps and How to Automate It
With the proliferation of agile product development models, industry experts from all levels have come to appreciate the value of incremental releases. However, there is also an expectation that each release cycle will maintain and improve the reliability and security of the product being delivered.
As a developer or engineer, your challenge is to implement security best practices without slowing down development or delaying your release dates. This article will illustrate several ways to include security practices in your development lifecycle to prevent critical issues later, and without slowing you down.
I’ll use Heroku Flow as an example flow to show how these security practices (or DevSecOps) can be integrated into your CI/CD practice, though the practices can be used in almost any common scenario.
What is DevSecOps?
DevSecOps is the philosophy of integrating security best practices early in the product development process. With DevSecOps, security is not treated as an isolated process or separate feature, but rather as an integral part of your development lifecycle. Automation helps you identify and fix security problems early, ideally before merging the application code to the main branch of the code repository.
Some examples of DevSecOps practices include scanning repositories for security vulnerabilities, early threat modeling, security design reviews, static code analysis, and code reviews.
Enter Heroku Flow
Heroku Flow provides a comprehensive CI/CD solution for Heroku-based applications. It seamlessly ties together several services (Heroku Pipelines, Review Apps, Heroku CI, and GitHub integrations) in a single view, giving engineers greater visibility to each code release— from a pull request to the production drop.
For a visual check out this Heroku Flow workflow from initial commit to production.
As the image visual shows, automated tests will run in Heroku CI when pull requests are created. Heroku CI is a cloud continuous integration tool from Heroku; it can either automatically detect the language and run default commands (e.g. npm test) or can be configured via the app.json file. CI results are available in both the pull request details in GitHub and the Heroku interface.
For a successful CI build, create a new review of the application and deploy it to a new temporary Heroku environment using Review Apps. The new environment link is available in GitHub pull request view, allowing engineers to easily check CI results or run any manual tests immediately.
After merging the pull request, the new review of the application is available in pre-production environments using Heroku Pipelines. Then, the review can be promoted to production.
Note that while some pieces of Heroku flow are included with a free account (namely pipelines and review apps) some features do incur a fee (Heroku CI).
How to Automate DevSecOps with Heroku Flow
As a comprehensive CI/CD solution integrated with GitHub, Heroku Flow offers several ways to automate your DevSecOps practices. Let's explore three common examples below:
- Upgrading dependencies with security vulnerabilities safely
- Identifying security bugs early
- Preventing unauthorized components or libraries
Safely Upgrade Dependencies with Security Vulnerabilities
You probably already know that you should upgrade dependencies with known vulnerabilities. It can be pretty time-consuming to identify and update those dependencies. Thankfully, you can automate the majority of this work.
GitHub provides a dependency vulnerability scanner, also known as Dependabot, which can be enabled per repository in GitHub’s Security settings. By default, it will add warnings to the GitHub interface when identifying a dependency with a known vulnerability.
While it’s a useful feature, it still requires you to check the warnings and manually create pull requests to upgrade the affected dependencies and create a fixed version. Fortunately, there's a beta feature in Dependabot that automatically creates pull requests to fix known vulnerabilities.
To enable this feature, simply add a .github/dependabot.yml file to your repository:
# Basic dependabot.yml file for JavaScript application
# check docs for other dependencies
version: 2
updates:
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "daily"
Dependabot will create pull requests with the suggested fixes, adding the GitHub code owners as default reviewers. The Dependabot documentation covers all available options.
Pull request created by Dependabot to address a known vulnerability
While the pull request will upgrade the affected library version, it's still important to verify that the application will work as expected after the upgrade.
The pull request raised by Dependabot will run CI tests and will be deployed to a new Heroku environment. Both versions are accessible from the GitHub interface.
Checks from Pull Request view in GitHub
After merging the pull request, the pipeline will run CI tests and deploy it to pre-production environments. Then, it can be promoted to production.
Heroku Pipeline view
Setting up Dependabot and Heroku Flow will automate most of the manual work required to address security vulnerabilities in libraries and dependencies.
Identify Security Bugs Early
Naturally, the ideal time to catch security bugs is before deploying to production. Many different tools can run static code analysis and identify problematic code before merging the code to the main branch.
As an example, let’s consider a simple Node.js application. Developers commonly use ESlint to enforce consistent coding styles and to catch common problems. Enabling ESlint-plugin-security will also identify common security bugs:
.eslintrc
...
"plugins": [
"security"
],
"extends": [
"plugin:security/recommended"
]
...
To ensure Eslint executes during CI, the app.json file is editable and points to a file in the repository:
app.json
{
"environments": {
"test": {
"scripts": {
"test": "bash ./ci.sh"
}
}
}
}
In this custom script file, you can run any desired command:
ci.sh
#!/bin/bash
set -eux
#### Running unit tests
npm test
#### Searching for security problems
npm run lint
If the lint fails, the build will be marked unsuccessful, and the deployment will not continue.
While ESlint-plugin-security is specific for JavaScript, most mature languages have static code analysis tools, like the popular Brakeman for Ruby, or Find-Sec-Bugs for Java.
Although the CI snippets in this article were shown in bash scripts, the Heroku CI supports multiple languages.
Prevent Unauthorized Components or Libraries
Some organizations heavily emphasize controlling application deployment complexity and implementing centralized controls for all applications. For example, these controls may prevent the use of Redis add-on or a specific JavaScript library.
All Heroku components for an application are defined in the app.json file as code. This opens up the possibility for pre-deployment checks. Infrastructure engineers can create a centralized script to prevent specific components from being deployed, and ensure all applications pass the same checks.
For example, let's consider the centralized script infrastructure-checks.sh shown below. It’s currently available in a public git repository "mygithubaccount/infrastructure-scripts". For this tutorial, let's say your goal is to prevent all Heroku add-ons from deploying.
infrastructure-scripts.sh
#!/bin/bash
set -eux
ADDONS=$(cat app.json | jq '.addons')
# Prevents all addons
if [[ "$ADDONS" != "null" ]]; then
echo "Add-ons are not allowed"
exit 1
fi
Inside the infrastructure script, you can add any desired number of checks to exclude specific addons, check environment variables, prevent certain instance types from being created, and even check for specific libraries that shouldn't be used. In short, you can do anything necessary to maintain the consistency of all environments.
For each Heroku application, the CI can be configured to download and execute infrastructure-scripts.sh from the centralized repository_: _
app.json
{
"environments": {
"test": {
"scripts": {
"test": "bash ./ci.sh"
}
}
}
}
ci.sh
#!/bin/bash
set -eux
INFRA_SCRIPTS_REPOSITORY="mygithubaccount/infrastructure-scripts"
wget https://raw.githubusercontent.com/${INFRA_SCRIPTS_REPOSITORY}/master/infrastructure-checks.sh
bash ./infrastructure-checks.sh
# Add as well all commands to run all other tests
...
The central infrastructure repository can also be private, but authentication will be required when downloading the script file.
Conclusion
Hopefully, you’ve now seen some practical examples of implementing security controls as part of your CI/CD pipeline using Heroku Flow. You should be able to implement similar controls in other CI/CD solutions as well, but those may not have tight integration with GitHub and Heroku.
More and more organizations are realizing, security shouldn’t be an afterthought, but rather part of a continuous improvement process. Implementing security controls and fixes as code in a minimally intrusive way will help you deliver code reliably and safely without slowing down delivery speed. It also ensures customers or end-users are largely protected from potential security breaches.
As part of DevSecOps, automation also means catching security vulnerabilities isn’t a reactive process where scanners and audit processes find security loopholes in live systems, but rather a proactive approach.
