In my previous blog, I explored secret scanning in CI pipelines using Gitleaks and pre-commit hooks. Beyond the risk of exposing secrets in repositories and pipelines, vulnerabilities can also be introduced into an application through third-party libraries and dependencies.

The first step in addressing and managing these vulnerabilities is to accurately collect, categorize, and present them in a readable format.

A powerful tool for managing security vulnerabilities is DefectDojo, which helps streamline the vulnerability management process.

"DefectDojo is a security orchestration and vulnerability management platform. DefectDojo allows you to manage your application security program, maintain product and application information, triage vulnerabilities, and push findings to systems like JIRA and Slack. DefectDojo enriches and refines vulnerability data using a number of heuristic algorithms that improve the more you use the platform."
— Source: DefectDojo DockerHub page

Overview

The process involves utilizing a Python script (upload.py) that interfaces with the DefectDojo API to import scan results from various security tools such as Gitleaks, Semgrep, and NJSSCAN. By integrating this script into GitLab CI/CD pipeline from here, you can automate the process of uploading scan results whenever a new scan is performed.

Demo Server

To simplify the process of setting up a DefectDojo instance, you can try out the demo server at demo.defectdojo.org. Log in with:

Username: admin
Password: 1Defectdojo@demo#appsec

Please note that the demo is publicly accessible and regularly reset. Do not put sensitive data in the demo.

When managing security vulnerabilities in an organization, it's crucial to have a structured approach to tracking, categorizing, and remediating issues. DefectDojo provides this structure through a set of well-defined concepts and terms that facilitate the organization and management of security-related activities.

Key Terms and Concepts in DefectDojo

Report: After testing, DefectDojo enables you to generate detailed reports on the findings. These reports are essential for communicating security issues to stakeholders.

Product: This represents the application or system you're securing. Each product can have different versions or components, allowing you to keep track of the security status of multiple versions.

Product Type: Products are grouped into categories called product types. For example, you might categorize products as web applications, APIs, or microservices. This helps in reporting and filtering across multiple similar products.

Engagement: DefectDojo organizes security assessments through engagements. Each engagement represents a security testing activity—such as a penetration test or vulnerability scan—on a specific product.

Test: Tests are actions carried out within an engagement, like running a security tool or performing manual code reviews. Each engagement can contain multiple tests.

Finding: Findings are the vulnerabilities or issues discovered during a test. These are tracked, prioritized, and assigned for remediation, and DefectDojo's heuristic algorithms help refine and enrich this data for better decision-making.

Endpoint: Products often contain multiple IP addresses, URLs, or domains that need testing. Endpoints allow you to track which specific parts of your product were tested.

Risk Acceptance: Not all vulnerabilities can or need to be immediately fixed. In cases where the risk is acknowledged but deemed acceptable, you can mark it as a risk acceptance, ensuring proper documentation.

By understanding and utilizing these concepts, you can better manage your application security program and streamline your vulnerability management process using DefectDojo.

The Python Script

Firstly, generate API key as shown below:

This API Key will be used in the Python script to automate scan report upload.

Here’s a breakdown of the upload.py script:

import requests
import sys

file_name = sys.argv[1]
scan_type = ''

if file_name == 'gitleaks.json':
    scan_type = 'Gitleaks Scan'
elif file_name == 'njsscan.sarif':
    scan_type = 'SARIF'
elif file_name == 'semgrep.json':
    scan_type = 'Semgrep JSON Report'

headers = {
    'Authorization': 'Token 548afd6fab3bea9794a41b31da0e9404f733e222'
}

url = 'https://demo.defectdojo.org/api/v2/import-scan/'

data = {
    'active': True,
    'verified': True,
    'scan_type': scan_type,
    'minimum_severity': 'Low',
    'engagement': 19
}

files = {
    'file': open(file_name, 'rb')
}

response = requests.post(url, headers=headers, data=data, files=files)

if response.status_code == 201:
    print('Scan results imported successfully')
else:
    print(f'Failed to import scan results: {response.content}')

Key Components

File Handling: The script accepts a filename as an argument, determining the type of scan based on the file extension.
API Integration: It sends a POST request to the DefectDojo API with the necessary headers and data, including the scan type and engagement ID.
Response Handling: It checks the response status to confirm whether the upload was successful.

Security Considerations

Ensure that the API token used in the script has the necessary permissions and is kept secure. Avoid hardcoding sensitive information; consider using environment variables or secret management tools.

GitLab CI/CD Integration

To automate the execution of this script, you can create a .gitlab-ci.yml file that defines the stages of your CI/CD pipeline. Here’s an example configuration:

variables:
  IMAGE_NAME: sirlawdin/demo-app
  IMAGE_TAG: juice-shop-1.1

stages:
  - cache
  - test
  - build

create_cache:
  image: node:18-bullseye
  stage: cache
  script:
    - yarn install --ignore-engines
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules/
      - yarn.lock
      - .yarn
    policy: pull-push

yarn_test:
  image: node:18-bullseye
  stage: test
  script:
    - yarn install
    - yarn test
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules/
      - yarn.lock
      - .yarn
    policy: pull-push

gitleaks:
  stage: test
  image:
    name: ghcr.io/gitleaks/gitleaks:latest
    entrypoint: [""]
  script:
    - gitleaks detect --source . --verbose --format json --report-path gitleaks-report.json
  allow_failure: true
  artifacts:
    when: always
    paths:
      - gitleaks-report.json

njsscan:
  stage: test
  image: python:3.11-slim
  before_script:
    - pip install --upgrade njsscan
  script:
    - njsscan --exit-warning . --sarif -o njsscan.sarif
  allow_failure: true
  artifacts:
    when: always
    paths:
      - njsscan.sarif

semgrep:
  stage: test
  image: returntocorp/semgrep:latest
  variables:
    SEMGREP_RULES: p/javascript
  script:
    - semgrep ci --json --output semgrep.json
  allow_failure: true
  artifacts:
    when: always
    paths:
      - semgrep.json

upload_reports:
  stage: test
  image: python
  needs: ["gitleaks", "njsscan", "semgrep"]
  when: always
  before_script:
    - pip3 install requests
  script:
    - python3 upload-reports.py gitleaks.json
    - python3 upload-reports.py njsscan.sarif
    - python3 upload-reports.py semgrep.json


build_image:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  before_script:
    - echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
  script:
    - docker build -t $IMAGE_NAME:$IMAGE_TAG .
    - docker push $IMAGE_NAME:$IMAGE_TAG

include:
  - remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Jobs/Secret-Detection.latest.gitlab-ci.yml'

Explanation of the CI/CD Configuration

Stages: Define a stage named security_scan.
Script: Call the upload.py script for each scan result file generated during the CI process.

Conclusion

By automating the upload of security scan results to DefectDojo, you can streamline your vulnerability management workflow. This integration not only saves time but also ensures that security issues are promptly addressed. Implementing this solution within your CI/CD pipeline will enhance the overall security posture. To learn more about DefectDojo, you can use the documentation.

Automate Uploading Security Scan Results to DefectDojo