Integrate NG SAST into the Bitbucket merge request workflow

This article shows you how to integrate ShiftLeft CORE's NG SAST into your Bitbucket pull request workflow for automated code analysis whenever you create a new pull request.

Background

Integrating NG SAST into your Bitbucket Merge Request workflow is done using the merge checks (please note that merge checks are a premium Bitbucket feature).

Merge checks allow you to recommend or require specific conditions that must be met before someone can merge changes into your master branch.

We assume that you're working with a Bitbucket repo utilizing Bitbucket's CI Pipelines.

Prerequisites

We recommend creating the following environment variables and app password to facilitate communication between ShiftLeft and Bitbucket, to ensure that the scan can access your application correctly, and (if desired) to print the results of your scan to your merge request.

Please note that the presence of any set environment variables will override those in your configuration files (if present).

VariableDescription
APP_PASSWORDThe app password you've set to ensure that ShiftLeft can communicate with Bitbucket
SHIFTLEFT_APP_NAMEThe name of your application
SHIFTLEFT_ACCESS_TOKENSet its value as your CI token -- you can create your CI token in the ShiftLeft Dashboard.
SHIFTLEFT_APP_PATHSpecify the application path ShiftLeft should use to identify scan's build target or project directory

Step 1: Mandate successful merge checks

You can edit your project properties and enforce branch protection rules using an administrative account. To do so:

  1. Go to Repository settings and launch the branch protection page.
  2. Click Add a branch permission.
  3. In the pop-up window, ensure that By name or pattern is selected. Provide an * to apply this rule to all branches. Then, check the Prevent a merge with unresolved merge checks option.

Click Save to proceed.

Step 2: Configure the Bitbucket Pipelines configuration file and include code analysis

The bitbucket-pipelines.yml file controls the jobs executed by Bitbucket's pipelines, and this is where you can insert the instructions needed to run NG SAST.

You can use Bitbucket's web-based UI to edit your configuration file.

The following snippet demonstrates what you need to add to your config file to build and automate code analysis for a Java app using Bitbucket merge checks and CI/CD pipelines (if your apps are using different architectures, e.g., you're working with a C# or a JavaScript app, you will need to change the images used for the pipeline):

image: maven:3.3.9-jdk-8
pipelines:
pull-requests:
'**':
- step:
name: Build the code
caches:
- maven
script: # Modify the commands below to build your repository.
- mvn clean package
artifacts:
- target/<path-to-built-app> #replace with your path
- sl-analysis.sh
- shiftleft.yml
- step:
name: Run ShiftLeft CORE
image: shiftleft/inspect
script:
- sh sl-analysis.sh

Note that the configuration file calls a script called sl-analysis.sh. You will need to add the file, which includes the code shown below, to the root of your repository (be sure to replace the <path-to-your-app> variable under the Analyze your code section):

#!/bin/sh
echo "Got merge request $BITBUCKET_PR_ID for branch $BITBUCKET_BRANCH"
# Review script environment variables and set defaults
if [ ! -n "$SHIFTLEFT_APP_NAME" ]; then
SHIFTLEFT_APP_NAME="YOUR_APP_NAME"
fi
if [ ! -n "$SHIFTLEFT_APP_PATH" ]; then
echo "Missing Environment Variable: \$SHIFTLEFT_APP_PATH"
exit 1
fi
echo "BITBUCKET_COMMIT= \"$BITBUCKET_COMMIT\""
echo "BITBUCKET_BRANCH= \"$BITBUCKET_BRANCH\""
echo "BITBUCKET_PR_ID= \"$BITBUCKET_PR_ID\""
echo "BITBUCKET_REPO_FULL_NAME=\"$BITBUCKET_REPO_FULL_NAME\""
echo "BITBUCKET_REPO_SLUG= \"$BITBUCKET_REPO_SLUG\""
echo "BITBUCKET_WORKSPACE= \"$BITBUCKET_WORKSPACE\""
echo "SHIFTLEFT_APP_NAME= \"$SHIFTLEFT_APP_NAME\""
echo "SHIFTLEFT_APP_PATH= \"$SHIFTLEFT_APP_PATH\""
# Install NG SAST
curl https://cdn.shiftleft.io/download/sl > /usr/local/bin/sl && chmod a+rx /usr/local/bin/sl
# Analyze your code
sl analyze \
--app "$SHIFTLEFT_APP_NAME" \
--version-id "$BITBUCKET_COMMIT" \
--tag branch="$BITBUCKET_BRANCH" \
--java
--wait \
"$SHIFTLEFT_APP_PATH"
# Check if this is running in a pull request
if [ -n "$BITBUCKET_PR_ID" ]; then
echo "Pull request [$BITBUCKET_PR_ID] issued for branch [$BITBUCKET_BRANCH]"
# Run check-analysis and save report to /tmp/check-analysis.md
echo "Starting sl check-analysis..."
sl check-analysis \
--app "$SHIFTLEFT_APP_NAME" \
--config ~/shiftleft.yml \
--report \
--report-file /tmp/check-analysis.md \
--source "tag.branch=master" \
--target "tag.branch=$BITBUCKET_BRANCH"
BUILDRULECHECK=$?
CHECK_ANALYSIS_OUTPUT=$(cat /tmp/check-analysis.md)
COMMENT_BODY=$(jq -n --arg body "$CHECK_ANALYSIS_OUTPUT" '{raw: $body}')
echo "BUILDRULECHECK= \"$BUILDRULECHECK\""
echo "CHECK_ANALYSIS_OUTPUT= \"$CHECK_ANALYSIS_OUTPUT\""
echo "COMMENT_BODY= \"$COMMENT_BODY\""
# Post report as merge request comment
echo "Posting ShiftLeft Check-Analysis Results to Bitbucket PR Comments..."
curl "https://api.bitbucket.org/2.0/repositories/$BITBUCKET_WORKSPACE/$BITBUCKET_REPO_SLUG/pullrequests/$BITBUCKET_PR_ID/comments" \
--verbose \
-u "$BITBUCKET_WORKSPACE:$APP_PASSWORD_ALL" \
-H "Content-Type: application/json" \
-d "{\"content\": $COMMENT_BODY}"
echo ""
if [ "$BUILDRULECHECK" -eq "1" ]; then
PR_COMMENT="Build rule(s) failed..."
echo $PR_COMMENT
exit 1
else
PR_COMMENT="Build rule(s) passed..."
echo $PR_COMMENT
exit 0
fi
fi

Please note that the script above includes a section that runs build rules; we will cover build rules in step 3.

Step 3: Configure your build rules

NG SAST allows you to create build rules, which define the security approval conditions that must be met before merging a merge request.

You can define build rules in a configuration file. By default, the name and location of your configuration file in the root of your repository should be shiftleft.yml. The file should be formatted as follows:

build_rules:
- id: build-rule-identifier
severity:
- SEVERITY_MEDIUM_IMPACT
- SEVERITY_HIGH_IMPACT
type:
- SQL Injection
- Sensitive Data Leak
owasp_category:
- a1-injection
threshold: 10
- id: another-build-rule
severity:
- SEVERITY_LOW_IMPACT
threshold: 100

Step 4: Push your files to Bitbucket

Be sure to commit and push the files you've just created to Bitbucket (alternatively, if you created any of these files using Bitbucket's web UI, pull your changes to make sure that everything is in sync):

  • bitbucket-pipelines.yml
  • shiftleft.yml
  • sl-analysis.sh

Step 5: Testing the workflow

At this point, you can test your workflow. Bitbucket will automatically trigger your pipelines whenever your push changes to your repository and create a pull request to run NG SAST, analyze your code, and indicate whether the merge should fail or not based on the build rules you define.

For example, if one of your build rules is violated, you will see the following result in your build logs:

Bitbucket build logs

Bitbucket will report that a build rule failed:

Bitbucket merge check approval

You will also see a list of the findings as a comment on your Merge Request:

NG SAST findings