Integrating NG SAST into the Bitbucket Merge Request Workflow

This article shows you how to integrate ShiftLeft CORE's NG SAST into your Bitbucket Merge Request Workflow for automated code analysis whenever you create a new PR.

Background

Integrating NG SAST into your Bitbucket Merge Request workflow is done using the merge checks (please note that merge checks are a premium Bitbucket feature).

Merge checks allow you to recommend or require specific conditions that must be met before someone can merge changes into your master branch.

We assume that you're working with a Bitbucket repo utilizing Bitbucket's CI Pipelines.

Prerequisites

To facilitate communication between ShiftLeft and Bitbucket, please create the required environment variables and app password as described in the following two sections.

ShiftLeft Variables

Create the following environment variables so that Bitbucket can communicate and use ShiftLeft:

Variable	Value
SHIFTLEFT_API_TOKEN	Your ShiftLeft Public API Token
SHIFTLEFT_ACCESS_TOKEN	Your Access Token

You can find your API token by going to Dashboard > Account Settings.

When running in a production environment, we recommend that you use a CI token as the access token. You can create your CI token in the ShiftLeft Dashboard.

Please note that the presence of any set environment variables will override those in a configuration file.

Bitbucket App Password

You will need to create one more environment variable for your Bitbucket app password. If you haven't already do so, create an app password to grant access to your Bitbucket account.

Save this password in an environment variable called APP_PASSWORD.

Step 1: Mandate Successful Merge Checks

Using an administrative account, you can edit your project properties and enforce branch protection rules. To do so, go to Repository settings and launch the branch protection settings and check the Prevent a merge with unresolved merge checks option.

Click Save to proceed.

Step 2: Configure the Bitbucket Pipelines Configuration File and Include Code Analysis

The bitbucket-pipelines.yml file controls the jobs executed by Bitbucket's pipelines, and this is where you can insert the instructions needed to run NG SAST.

You can use Bitbucket's web-based UI to edit your configuration file.

The following snippet demonstrates what you need to add to your config file to build and automate code analysis for a Java app using Bitbucket merge checks and CI/CD pipelines:

image: maven:3.3.9-jdk-8

pipelines:

pull-requests:

'**':

- step:

name: Build the code

caches:

- maven

script: # Modify the commands below to build your repository.

- mvn clean package

artifacts:

- target/<path-to-built-app> #replace with your path

- sl-analysis.sh

- shiftleft.yml

- step:

name: shiftleft code analysis

image: shiftleft/inspect

script:

- sh sl-analysis.sh

Note that the configuration file calls a script called sl-analysis.sh. You will need to add the file, which includes the code shown below, to the root of your repository (be sure to replace the <path-to-your-app> variable under the Analyze your code section):

#!/bin/sh

echo "Got merge request $BITBUCKET_PR_ID for branch $BITBUCKET_BRANCH"

# Install NG SAST

curl https://www.shiftleft.io/download/sl-latest-linux-x64.tar.gz > /tmp/sl.tar.gz && tar -C /usr/local/bin -xzf /tmp/sl.tar.gz

APP_NAME="$BITBUCKET_REPO_SLUG-BB"

echo $APP_NAME

# Analyze your code

sl analyze --version-id "$BITBUCKET_COMMIT" --tag branch="$BITBUCKET_BRANCH" --app "$APP_NAME" --java --cpg --wait target/<path-to-your-app>.war

# Run the build rules check

URL="https://www.shiftleft.io/findingsSummary/$APP_NAME?apps=$APP_NAME&isApp=1"

BUILDRULECHECK=$(sl check-analysis --app "$APP_NAME" --branch "$BITBUCKET_BRANCH")

# Set up comment body for the merge request

COMMENT_BODY='{"raw":""}'

COMMENT_BODY=$(echo "$COMMENT_BODY" | jq '.raw += "## NG SAST Analysis Findings \n "')

NEW_FINDINGS=$(curl -H "Authorization: Bearer $SHIFTLEFT_API_TOKEN" "https://www.shiftleft.io/api/v4/orgs/$SHIFTLEFT_ORG_ID/apps/$APP_NAME/scans/compare?source=tag.branch=$BITBUCKET_BRANCH&target=tag.branch=$BITBUCKET_BRANCH" | jq -c -r '.response.common | .? | .[] | "* [ID " + .id + "](https://www.shiftleft.io/findingDetail/" + .app + "/" + .id + "): " + "["+.severity+"] " + .title')

echo $NEW_FINDINGS

COMMENT_BODY=$(echo "$COMMENT_BODY" | jq ".raw += \"### New findings \n \n \"")

COMMENT_BODY=$(echo "$COMMENT_BODY" | jq ".raw += \"$NEW_FINDINGS \n \n \"")

echo "COMMENT_BODY: $COMMENT_BODY"

if [ -n "$BUILDRULECHECK" ]; then

PR_COMMENT="Build rule failed, click here for vulnerability list - $URL\n\n"

echo $PR_COMMENT

curl -XPOST "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_FULL_NAME/pullrequests/$BITBUCKET_PR_ID/comments" \

-u "$BITBUCKET_WORKSPACE:$APP_PASSWORD" \

-H "Content-Type: application/json" \

-d "{\"content\": $COMMENT_BODY}"

exit 1

else

PR_COMMENT="Build rule succeeded, click here for vulnerability list! - $URL\n\n"

echo $PR_COMMENT

curl -XPOST "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_FULL_NAME/pullrequests/$BITBUCKET_PR_ID/comments" \

-u "$BITBUCKET_WORKSPACE:$APP_PASSWORD" \

-H "Content-Type: application/json" \

-d "{\"content\": $COMMENT_BODY}"

exit 0

fi

Please note that the script above includes a section that runs build rules; we will cover build rules in step 3 of this article.

Step 3: Configure Your Build Rules

NG SAST allows you to create build rules, which define the security approval conditions that must be met before a merge request can be merged.

You can define build rules in a configuration file. By default, the name and location of your configuration file in the root of your repository should be shiftleft.yml. The file should be formatted as follows:

build_rules:

- id: build-rule-identifier

severity:

- SEVERITY_MEDIUM_IMPACT

- SEVERITY_HIGH_IMPACT

type:

- SQL Injection

- Sensitive Data Leak

owasp_category:

- a1-injection

threshold: 10

- id: another-build-rule

severity:

- SEVERITY_LOW_IMPACT

threshold: 100

Step 4: Push Your Files to Bitbucket

Be sure to commit and push the files you've just created to Bitbucket (alternatively, if you created any of these files using Bitbucket's web UI, pull your changes to make sure that everything is in sync):

• bitbucket-pipelines.yml
• shiftleft.yml
• sl-analysis.sh

Step 5: Testing the Workflow

At this point, you can test your workflow. Bitbucket will automatically trigger your pipelines whenever your push changes to your repository and create a Merge Request to run NG SAST, analyze your code, and indicate whether the merge should fail or not based on the build rules you define.

For example, if one of your build rules is violated, you will see the following result in your build logs:

Bitbucket will report that a build rule failed:

You will also see a list of the findings as a comment on your Merge Request: