Scan Multi-Language Repositories in GitHub

This article will show you how to integrate ShiftLeft CORE into your GitHub Pull Request (PR) workflow for automated code analysis using GitHub Actions if your application utilizes multiple programming languages.

Step 1: Create your GitHub secret

GitHub's secrets are encrypted environment variables that protect information while making them available for use in GitHub Actions workflows. They are specific to your GitHub repository. You will use secrets to provide an access token that lets GitHub interact with ShiftLeft.

To create a secret specific to your GitHub repository, go to Settings > Secrets. Click New Secret. You will need to create a secret called SHIFTLEFT_ACCESS_TOKEN to store your ShiftLeft CI config token.

You can create your CI token in the ShiftLeft Dashboard.

GitHub Secrets

Step 2: Create your GitHub Action and define its workflow

GitHub Actions offers you workflow automation functionality. You can use this to automatically run NG SAST (e.g., when you create a new Pull Request).

To create a new GitHub Action for your repository, click Actions. If this is your first time setting up a GitHub Action, click set up a workflow yourself near the top-left; otherwise, click New workflow, then select set up a workflow yourself.

GitHub Actions starter workflows

You will be redirected to a YAML editing window. Rename the file (if desired), and provide the following script to invoke NG SAST.

---
# This workflow integrates ShiftLeft with GitHub
name: ShiftLeft CORE
on:
pull_request:
workflow_dispatch:
jobs:
#########################################################
# Scan dynamic languages, such as JavaScript/TypeScript,
# Python, and Go.
#########################################################
ShiftLeft-Source-Analysis:
runs-on: ubuntu-latest
# Use the shiftleft/core image, which supports multiple
# languages, including JavaScript/TypeScript, Python, and Go
container:
image: shiftleft/core:latest
strategy:
fail-fast: false
matrix:
language: [go, js, python]
steps:
- uses: actions/checkout@v2
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
id: extract_branch
# Perform any app specific build or restore here
- name: App specific build
if: ${{ matrix.language == 'python' }}
run: |
pip install -r python/requirements.txt
# Run ShiftLeft for the source languages
- name: Scan source languages
if: ${{ matrix.language == 'js' || matrix.language == 'python' }}
run: |
sl analyze --tag app.group=multi-vuln-app --app app-${{ matrix.language }} --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --${{ matrix.language }} .
env:
SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
# For Go, the last argument used must be identical to the one used with `go build`
- name: Scan go application
if: ${{ matrix.language == 'go' }}
run: |
cd go && sl analyze --tag app.group=multi-vuln-app --app app-${{ matrix.language }} --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --${{ matrix.language }} ./...
env:
SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
##############################################
# Scan Java applications
##############################################
Java-Analysis:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Setup Java JDK
uses: actions/setup-java@v1.4.3
with:
java-version: 1.8
- name: Download ShiftLeft cli
run: |
curl https://cdn.shiftleft.io/download/sl > ${GITHUB_WORKSPACE}/sl && chmod a+rx ${GITHUB_WORKSPACE}/sl
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
id: extract_branch
# Try to auto build Maven, Gradle, and/or sbt projects
- name: Build and Analyze
run: |
TARGET_DIR=target
BUILT=0
POM_COUNT=$(find . -maxdepth 1 -type f -name "pom.xml" -not -path '*/\.git/*' | wc -l | tr -d " ")
GRADLE_COUNT=$(find . -maxdepth 1 -type f -name "build.gradle" -not -path '*/\.git/*' | wc -l | tr -d " ")
SBT_COUNT=$(find . -maxdepth 1 -type f -name "build.sbt" -not -path '*/\.git/*' | wc -l | tr -d " ")
if [ "$POM_COUNT" != "0" ]; then
mvn compile package
BUILT=1
elif [ "$GRADLE_COUNT" != "0" ]; then
gradle jar
#./gradlew jar
TARGET_DIR=build
BUILT=1
elif [ "$SBT_COUNT" != "0" ]; then
echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo apt-key add
sudo apt update -y
sudo apt-get install sbt -y
sbt package
BUILT=1
fi
if [ "$BUILT" = "1" ] && [ -d "$TARGET_DIR" ]; then
jar cvf app.jar -C $TARGET_DIR .
${GITHUB_WORKSPACE}/sl analyze --wait --tag app.group=multi-vuln-app --app app-java --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --vcs-prefix-correction "*=src/main/java" --java app.jar
else
echo "Unable to build the project automatically. Please follow the instructions in our documentation to setup this project - https://docs.shiftleft.io/ngsast/analyzing-applications/java"
fi
env:
SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
working-directory: java
##############################################
# Check for violations for each language
##############################################
Build-Rules:
runs-on: ubuntu-latest
needs: [ShiftLeft-Source-Analysis, Java-Analysis]
strategy:
fail-fast: false
matrix:
language: [go, js, python, java]
steps:
- uses: actions/checkout@v2
- name: Download ShiftLeft CLI
run: |
curl https://cdn.shiftleft.io/download/sl > ${GITHUB_WORKSPACE}/sl && chmod a+rx ${GITHUB_WORKSPACE}/sl
- name: Extract branch name
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
id: extract_branch
- name: Validate Build Rules per app
if: ${{ github.event_name == 'pull_request' }}
run: ${GITHUB_WORKSPACE}/sl check-analysis --v2 --config shiftleft.yml --app app-${{ matrix.language }} \
--branch "${{ github.head_ref || steps.extract_branch.outputs.branch }}" --report \
--github-pr-number=${{github.event.number}} --github-pr-user=${{ github.repository_owner }} \
--github-pr-repo=${{ github.event.repository.name }} --github-token=${{ secrets.GITHUB_TOKEN }}
env:
SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
GitHub Actions Configuration

When done, click Start commit and follow the prompts to commit the file to your repo.

Commit configuration

You'll see your newly configured workflow listed under the repository's Actions.

GitHub Actions

Step 3: Test Your Workflow

At this point, you're done with the configuration steps. You can check whether you successfully set up the GitHub Action by triggering the workflow (e.g., by creating a Pull Request).

New PR

You can click Status for additional details about the workflow's progress:

Workflow Progress

When done, you can see a summary of NG SAST's results on the PR:

Completed Check

You can get full details regarding the analysis from the ShiftLeft Dashboard.