Integrating NG SAST into the Bitbucket Merge Request Workflow

This article shows you how to integrate ShiftLeft CORE's NG SAST into your Bitbucket Merge Request Workflow for automated code analysis whenever you create a new PR.

Background

Integrating NG SAST into your Bitbucket Merge Request workflow is done using the merge checks (please note that merge checks are a premium Bitbucket feature).

Merge checks allow you to recommend or require specific conditions that must be met before someone can merge changes into your master branch.

We assume that you're working with a Bitbucket repo utilizing Bitbucket's CI Pipelines.

Prerequisites

To facilitate communication between ShiftLeft and Bitbucket, please create the required environment variables and app password as described in the following two sections.

ShiftLeft Variables

Create the following environment variables so that Bitbucket can communicate and use ShiftLeft:

VariableValue
SHIFTLEFT_API_TOKENYour ShiftLeft Public API Token
SHIFTLEFT_ACCESS_TOKENYour Access Token

You can find your API token by going to Dashboard > Account Settings.

When running in a production environment, we recommend that you use a CI token as the access token. You can create your CI token in the ShiftLeft Dashboard.

Please note that the presence of any set environment variables will override those in a configuration file.

Bitbucket App Password

You will need to create one more environment variable for your Bitbucket app password. If you haven't already do so, create an app password to grant access to your Bitbucket account.

Bitbucket App Password

Save this password in an environment variable called APP_PASSWORD.

Step 1: Mandate Successful Merge Checks

Using an administrative account, you can edit your project properties and enforce branch protection rules. To do so, go to Repository settings and launch the branch protection settings and check the Prevent a merge with unresolved merge checks option.

BitBucket branch protection settings

Click Save to proceed.

Step 2: Configure the Bitbucket Pipelines Configuration File and Include Code Analysis

The bitbucket-pipelines.yml file controls the jobs executed by Bitbucket's pipelines, and this is where you can insert the instructions needed to run NG SAST.

You can use Bitbucket's web-based UI to edit your configuration file.

The following snippet demonstrates what you need to add to your config file to build and automate code analysis for a Java app using Bitbucket merge checks and CI/CD pipelines:

image: maven:3.3.9-jdk-8
pipelines:
pull-requests:
'**':
- step:
name: Build the code
caches:
- maven
script: # Modify the commands below to build your repository.
- mvn clean package
artifacts:
- target/<path-to-built-app> #replace with your path
- sl-analysis.sh
- shiftleft.yml
- step:
name: shiftleft code analysis
image: shiftleft/inspect
script:
- sh sl-analysis.sh

Note that the configuration file calls a script called sl-analysis.sh. You will need to add the file, which includes the code shown below, to the root of your repository (be sure to replace the <path-to-your-app> variable under the Analyze your code section):

#!/bin/sh
echo "Got merge request $BITBUCKET_PR_ID for branch $BITBUCKET_BRANCH"
# Install NG SAST
curl https://www.shiftleft.io/download/sl-latest-linux-x64.tar.gz > /tmp/sl.tar.gz && tar -C /usr/local/bin -xzf /tmp/sl.tar.gz
APP_NAME="$BITBUCKET_REPO_SLUG-BB"
echo $APP_NAME
# Analyze your code
sl analyze --version-id "$BITBUCKET_COMMIT" --tag branch="$BITBUCKET_BRANCH" --app "$APP_NAME" --java --cpg --wait target/<path-to-your-app>.war
# Run the build rules check
URL="https://www.shiftleft.io/findingsSummary/$APP_NAME?apps=$APP_NAME&isApp=1"
BUILDRULECHECK=$(sl check-analysis --app "$APP_NAME" --branch "$BITBUCKET_BRANCH")
# Set up comment body for the merge request
COMMENT_BODY='{"raw":""}'
COMMENT_BODY=$(echo "$COMMENT_BODY" | jq '.raw += "## NG SAST Analysis Findings \n "')
NEW_FINDINGS=$(curl -H "Authorization: Bearer $SHIFTLEFT_API_TOKEN" "https://www.shiftleft.io/api/v4/orgs/$SHIFTLEFT_ORG_ID/apps/$APP_NAME/scans/compare?source=tag.branch=$BITBUCKET_BRANCH&target=tag.branch=$BITBUCKET_BRANCH" | jq -c -r '.response.common | .? | .[] | "* [ID " + .id + "](https://www.shiftleft.io/findingDetail/" + .app + "/" + .id + "): " + "["+.severity+"] " + .title')
echo $NEW_FINDINGS
COMMENT_BODY=$(echo "$COMMENT_BODY" | jq ".raw += \"### New findings \n \n \"")
COMMENT_BODY=$(echo "$COMMENT_BODY" | jq ".raw += \"$NEW_FINDINGS \n \n \"")
echo "COMMENT_BODY: $COMMENT_BODY"
if [ -n "$BUILDRULECHECK" ]; then
PR_COMMENT="Build rule failed, click here for vulnerability list - $URL\n\n"
echo $PR_COMMENT
curl -XPOST "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_FULL_NAME/pullrequests/$BITBUCKET_PR_ID/comments" \
-u "$BITBUCKET_WORKSPACE:$APP_PASSWORD" \
-H "Content-Type: application/json" \
-d "{\"content\": $COMMENT_BODY}"
exit 1
else
PR_COMMENT="Build rule succeeded, click here for vulnerability list! - $URL\n\n"
echo $PR_COMMENT
curl -XPOST "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_FULL_NAME/pullrequests/$BITBUCKET_PR_ID/comments" \
-u "$BITBUCKET_WORKSPACE:$APP_PASSWORD" \
-H "Content-Type: application/json" \
-d "{\"content\": $COMMENT_BODY}"
exit 0
fi

Please note that the script above includes a section that runs build rules; we will cover build rules in step 3 of this article.

Step 3: Configure Your Build Rules

NG SAST allows you to create build rules, which define the security approval conditions that must be met before a merge request can be merged.

You can define build rules in a configuration file. By default, the name and location of your configuration file in the root of your repository should be shiftleft.yml. The file should be formatted as follows:

build_rules:
- id: build-rule-identifier
severity:
- SEVERITY_MEDIUM_IMPACT
- SEVERITY_HIGH_IMPACT
type:
- SQL Injection
- Sensitive Data Leak
owasp_category:
- a1-injection
threshold: 10
- id: another-build-rule
severity:
- SEVERITY_LOW_IMPACT
threshold: 100

Step 4: Push Your Files to Bitbucket

Be sure to commit and push the files you've just created to Bitbucket (alternatively, if you created any of these files using Bitbucket's web UI, pull your changes to make sure that everything is in sync):

  • bitbucket-pipelines.yml
  • shiftleft.yml
  • sl-analysis.sh

Step 5: Testing the Workflow

At this point, you can test your workflow. Bitbucket will automatically trigger your pipelines whenever your push changes to your repository and create a Merge Request to run NG SAST, analyze your code, and indicate whether the merge should fail or not based on the build rules you define.

For example, if one of your build rules is violated, you will see the following result in your build logs:

Bitbucket build logs

Bitbucket will report that a build rule failed:

Bitbucket merge check approval

You will also see a list of the findings as a comment on your Merge Request:

NG SAST findings