How to integrate OWASP ZAP in Gitlab CI/CD pipeline.

In this article you will learn how to automate your secure deployment by integrating OWASP ZAP in your GitLab CI/CD pipeline. Doing so for GitHub is largely similar with GitHub’s actions.

Cybersecurity is becoming a priority on a worldwide scale. Both the US (NIST SSDF and Memorandum M-22-18) and the EU (NIS2) have recently introduced legislations to aim at high common level of cybersecurity. Meanwhile, the application security state-of-the-practice in large and small firms is far from ideal (to put it mildly). The attack sophistication and absolute numbers are increasing. The community is playing a whack-a-mole game hoping tooling will solve everything. The security teams on the other hand have already realized we are headed towards a train crash. Security teams want less and not more tools as tooling pushes them down the rabbit hole with thousands of alerts. Paradoxically, we have known the systematic solution to the security challenge. We (the actual security experts) name it a secure software development lifecycle or an AppSec programme.

There are many AppSec programmes out there (NIST SSDF, Microsoft SDLC, BSIMM, etc) with OWASP SAMM being by far the simplest one to start with. The cornerstones of any AppSec programme are People, Process, Knowledge, Tools, Training and Risk. Thus, tooling is just one wheel in the whole machine. To make tooling efficient we should start from the process and fit the tools in the puzzle, rather than trying to reverse engineer the puzzle from the tools. 

In this blog, we present a small part of our AppSec programme, namely secure deploy. We provide a high-level overview of how the process is structured and how we’ve intergrated the tool in our process.

What is ZAP?

ZAP is free, highly customizable and flexible open source web application tool with a great community behind it. Before getting into how we integrated it into our GitLab deployment pipeline, we highly recommend you to watch the ZAP introduction videos.

In simple terms, ZAP is a toolbox you can leverage to search for vulnerabilities in your web application both manually and in an automated fashion. It’s functionality is abundant, but in this blog we will be focusing on running a basic automated scan and integrating it into a GitLab pipeline.

Integrating the OWASP ZAP Automation Framework Baseline scan into your Gitlab pipeline

1. 1. 1. Setup the Context

         We’ll be using the vulnerable Broken Crystals Application to get things rolling. You can obviously use your own application as well. We’ve set up a user with an email ‘admin@admin.com’ and a password: ‘admin’ so we’ll use that to setup the ZAP context. Double click on the ‘Default Context’  from the context, chose the ‘Authentication’ menu and fill the settings as show below.

         

         After that chose the ‘Users’ menu and click on the ‘Add’ button to add our user details and fill our ‘admin@admin.com’ user details.

         

          

      2. Run the First Automation Plan

         Now that we have our context set up, let’s get cracking on the Automation Plan. We won’t be focusing on how to run an automated scan via the GUI, but there’s a great in depth explanation on how to do that here. Just make sure you have the following settings set up for your ‘Spider’ job:

         

         
         

         

         And for your ‘Spider Ajax’ job:

         

         
         

         

          

      3. Export the Automation Plan

         After running it successfully, we need to export our automation framework configuration file. This can be done by clicking on the “Save Plan As…” option as shown below, which will save your automation plan as a yaml file.

         

         You can read more about what the produced configuration file contains here.

    

   1. 1. Testing the Automation Plan with ZAP’s Command Line Tool

         The next step is to pull the ZAP docker image:

         docker pull owasp/zap2docker-stable
         

         and after that all we need to do is go to the directory where we downloaded our automation.yaml configuration file and run the command:

         docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap.sh -cmd -autorun wrk/borken_crystals.yaml
         

         The result should be something along the lines of:

         Job spider finished
         Job spiderAjax started
         Job spiderAjax found 0 URLs
         Job spiderAjax test of type stats failed: At least 100 URLs found [0 < 100]
         Job spiderAjax finished
         Job passiveScan-wait started
         Job passiveScan-wait finished
         Job report started
         Job report failed to generate report: Cannot create directory '/Users/codific/2022-12-29-ZAP-Report-brokencrystals.com'.
         Job report finished
         Automation plan failures:
         Job report failed to generate report: Cannot create directory '/Users/codific/2022-12-29-ZAP-Report-brokencrystals.com'

    

   1. Adding OWASP ZAP Automation Plan to GitLab’s CI/CD

      Before we move onto adding the script to the CI/CD, you should replace the occurrences of some configurable strings in your broken_crystals.yaml file:

      env:
      ...
        contexts:
      ...
          users:
            - name: "admin"
              credentials:
                password: "PASSWORD"
                username: "USERNAME"
      

      And replace the report task config, found at the end of the file, with the following:

      ...
      - parameters:
          template: "traditional-html"
          reportDir: "CI_PROJECT_DIR"
          reportFile: "ZAP_REPORT"
          reportTitle: "ZAP Baseline Scan Report"
          reportDescription: ""
          displayReport: false
        risks:
          - "info"
          - "low"
          - "medium"
          - "high"
        confidences:
          - "falsepositive"
          - "low"
          - "medium"
          - "high"
          - "confirmed"
        sections:
          - "instancecount"
          - "alertdetails"
          - "alertcount"
        name: "report"
        type: "report"
      - parameters:
          template: "traditional-md"
          reportDir: "CI_PROJECT_DIR"
          reportFile: "ZAP_ALERT_REPORT"
          reportTitle: "ZAP Scanning Report"
          reportDescription: "This is report that should contain only alerts that should\
            \ fail the pipeline and be revisited by a team member"
          displayReport: false
        risks:
        - "low"
        - "medium"
        - "high"
        confidences:
        - "low"
        - "medium"
        - "high"   
        - "confirmed"
        sections:
        - "alertdetails"
        - "alertcount"
        name: "alert-report"
        type: "report"
      

      This allows us to move the credentials to secure gitlab variables and it also generates an HTML and a Markdown report which we will use to determine whether the scan completed successfully (didn’t find any scary errors). Move the configuration file inside your repository in  a new directory called ‘zap’.

      Next, create a script in your project root directory called zap_automation_baseline_scan.sh with the following content:

      #!/bin/bash
      
      sed -i "s/PASSWORD/$PASSWORD/g" zap/broken_crystals.yaml
      sed -i "s/USERNAME/$USERNAME/g" zap/broken_crystals.yaml
      sed -i "s/ZAP_REPORT/$ZAP_REPORT/g" zap/broken_crystals.yaml
      
      # NOTE:
      # We need this because the variable contains / in it.
      # More info here https://unix.stackexchange.com/questions/255789/is-there-a-way-to-prevent-sed-from-interpreting-the-replacement-string/255869#255869
      ESCAPED_CI_PROJECT_DIR=$(sed -e 's/[&\\/]/\\&/g; s/$/\\/' -e '$s/\\$//' <<<"$CI_PROJECT_DIR")
      sed -i "s/CI_PROJECT_DIR/$ESCAPED_CI_PROJECT_DIR/g" zap/broken_crystals.yaml
      
      sed -i "s/ZAP_ALERT_REPORT/$ZAP_ALERT_REPORT/g" zap/broken_crystals.yaml
      
      /zap/zap.sh -cmd -autorun $CI_PROJECT_DIR/zap/broken_crystals.yaml
      
      returnCode=0
      if grep -q "Instances" alert-report.md; then
        head -n 20 $ZAP_ALERT_REPORT.md
        echo "DAST RESULT: There are some vulnerabilities that ZAP has found (those visible here may not be the only ones). See the detailed report for more information." 
        returnCode=1
      fi
      
      exit $returnCode
      

      After that add the following to your .gitlab-ci.yaml file:

      zap-automation-baseline-scan:
        stage: test
        image:
          name: owasp/zap2docker-stable:latest
          entrypoint: [""]
        variables:
          CI_PROJECT_DIR: path_to_project_dir
          ZAP_ALERT_REPORT: alert-report
          ZAP_REPORT: baselinescan
        script:
          - bash zap_automation_baseline_scan.sh
        only:
          refs:
          - branches
        artifacts:
          when: always
          expire_in: 1 week
          paths:
            - ${CI_PROJECT_DIR}/${ZAP_REPORT}.html
            - ${CI_PROJECT_DIR}/${ZAP_ALERT_REPORT}.md
      

      And voila! Running this gitlab job will cause it to fail if our script has found the ‘Instances’ heading in the Markdown report file generated by the scan. We also have a prettier HTML version of the report which our colleagues could review for more information on what exactly the issues were.

Integrating the OWASP ZAP Full Scan into a GitLab Pipeline

Alongside the “baseline scan”, which we run daily, we also use a “full scan” which is aggressive and slow. That’s why we run it on a weekly basis. The setup is similar.

Create a zap_full_scan.sh script with the following content:

#!/bin/bash
# NOTE:
# We need this because the variable contains / in it.
# More info here https://unix.stackexchange.com/questions/255789/is-there-a-way-to-prevent-sed-from-interpreting-the-replacement-string/255869#255869
ESCAPED_CI_PROJECT_DIR=$(sed -e 's/[&\\/]/\\&/g; s/$/\\/' -e '$s/\\$//' <<<"$CI_PROJECT_DIR")
sed -i "s/CI_PROJECT_DIR/$ESCAPED_CI_PROJECT_DIR/g" zap/zap_config.conf

ESCAPED_TARGET=$(sed -e 's/[&\\/]/\\&/g; s/$/\\/' -e '$s/\\$//' <<<"$TARGET")
sed -i "s/TARGET/$ESCAPED_TARGET/g" zap/context.context

/zap/zap-full-scan.py -d \
                      -t https://brokencrystals.com \
                      -r $CI_PROJECT_DIR/$ZAP_REPORT.html \
                      -w $CI_PROJECT_DIR/$ZAP_ALERT_REPORT.md \
                      -n $CI_PROJECT_DIR/zap/context.context \
                      -U $USERNAME

returnCode=0
if grep -q "Instances" $CI_PROJECT_DIR/$ZAP_ALERT_REPORT.md; then
  head -n 20 $CI_PROJECT_DIR/$ZAP_ALERT_REPORT.md
  echo "DAST RESULT: There are some vulnerabilities that ZAP has found (those visible here may not be the only ones). See the detailed report for more  information."
  returnCode=1
fi

exit $returnCode

And add the following to your .gitlab-ci.yaml file:

zap-full-scan:
  stage: test
  image: 
    name: owasp/zap2docker-stable:latest
    entrypoint: [""]
  variables:
    CI_PROJECT_DIR: path_to_project_dir
    ZAP_ALERT_REPORT: alert_report
    ZAP_REPORT: full_scan
  script:
    - bash zap_full_scan.sh
  only:
    refs:
      - branches
  artifacts:
    when: always
    expire_in: 1 week
    paths:
      - ${CI_PROJECT_DIR}/${ZAP_REPORT}.html
      - ${CI_PROJECT_DIR}/${ZAP_ALERT_REPORT}.md

We don’t recommend using the full scan towards production applications as it may cause corrupted database, stored XSS, etc. At the end of the scan the report that is generated is similar to the one generated by the “baseline” scan, with the only difference that it might have found different vulnerabilites due to the nature of the scan.

What does the Codific team build with OWASP ZAP and Gitlab?

Codific is a team of security software engineers that leverage privacy by design principles to build secure cloud solutions. We build applications in different verticals such as HR-tech, Ed-Tech and Med-Tech. Secure collaboration and secure sharing are at the core of our solutions.

Videolab is used by top universities, academies and hospitals to put the care in healthcare. Communication skills, empathy and other soft skills are trained by sharing patient interview recordings for feedback.

SARA is used by top HR-Consultants to deliver team assessments, psychometric tests, 360 degree feedback, cultural analysis and other analytical HR tools.

SAMMY Is a Software Assurance Maturity Model management tool. It enables companies to formulate and implement a security assurance program tuned to the risks they are facing. That way other companies can help us build a simple and safe digital future. Obviously our AppSec program and SAMMY itself is built on top of it.

We believe in collaboration and open innovation, we would love to hear about your projects an see how we can contribute in developing secure software and privacy by design architecture. Contact us.