Scan Multi-Language Repositories in GitHub

This article will show you how to integrate ShiftLeft CORE into your GitHub Pull Request (PR) workflow for automated code analysis using GitHub Actions if your application utilizes multiple programming languages.

Step 1: Create your GitHub secret

GitHub's secrets are encrypted environment variables that protect information while making them available for use in GitHub Actions workflows. They are specific to your GitHub repository. You will use secrets to provide an access token that lets GitHub interact with ShiftLeft.

To create a secret specific to your GitHub repository, go to Settings > Secrets. Click New Secret. You will need to create a secret called SHIFTLEFT_ACCESS_TOKEN to store your ShiftLeft CI config token.

You can create your CI token in the ShiftLeft Dashboard.

Step 2: Create your GitHub Action and define its workflow

GitHub Actions offers you workflow automation functionality. You can use this to automatically run NG SAST (e.g., when you create a new Pull Request).

To create a new GitHub Action for your repository, click Actions. If this is your first time setting up a GitHub Action, click set up a workflow yourself near the top-left; otherwise, click New workflow, then select set up a workflow yourself.

You will be redirected to a YAML editing window. Rename the file (if desired), and provide the following script to invoke NG SAST.

---

# This workflow integrates ShiftLeft with GitHub

name: ShiftLeft CORE

on:

pull_request:

workflow_dispatch:

jobs:

#########################################################

# Scan dynamic languages, such as JavaScript/TypeScript,

# Python, and Go.

#########################################################

ShiftLeft-Source-Analysis:

runs-on: ubuntu-latest

# Use the shiftleft/core image, which supports multiple

# languages, including JavaScript/TypeScript, Python, and Go

container:

image: shiftleft/core:latest

strategy:

fail-fast: false

matrix:

language: [go, js, python]

steps:

- uses: actions/checkout@v2

- name: Extract branch name

shell: bash

run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"

id: extract_branch

# Perform any app specific build or restore here

- name: App specific build

if: ${{ matrix.language == 'python' }}

run: |

pip install -r python/requirements.txt

# Run ShiftLeft for the source languages

- name: Scan source languages

if: ${{ matrix.language == 'js' || matrix.language == 'python' }}

run: |

sl analyze --tag app.group=multi-vuln-app --app app-${{ matrix.language }} --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --${{ matrix.language }} .

env:

SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}

# For Go, the last argument used must be identical to the one used with `go build`

- name: Scan go application

if: ${{ matrix.language == 'go' }}

run: |

cd go && sl analyze --tag app.group=multi-vuln-app --app app-${{ matrix.language }} --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --${{ matrix.language }} ./...

env:

SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}

##############################################

# Scan Java applications

##############################################

Java-Analysis:

runs-on: ubuntu-latest

steps:

- uses: actions/checkout@v2

- name: Setup Java JDK

uses: actions/setup-java@v1.4.3

with:

java-version: 1.8

- name: Download ShiftLeft cli

run: |

curl https://cdn.shiftleft.io/download/sl > ${GITHUB_WORKSPACE}/sl && chmod a+rx ${GITHUB_WORKSPACE}/sl

- name: Extract branch name

shell: bash

run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"

id: extract_branch

# Try to auto build Maven, Gradle, and/or sbt projects

- name: Build and Analyze

run: |

TARGET_DIR=target

BUILT=0

POM_COUNT=$(find . -maxdepth 1 -type f -name "pom.xml" -not -path '*/\.git/*' | wc -l | tr -d " ")

GRADLE_COUNT=$(find . -maxdepth 1 -type f -name "build.gradle" -not -path '*/\.git/*' | wc -l | tr -d " ")

SBT_COUNT=$(find . -maxdepth 1 -type f -name "build.sbt" -not -path '*/\.git/*' | wc -l | tr -d " ")

if [ "$POM_COUNT" != "0" ]; then

mvn compile package

BUILT=1

elif [ "$GRADLE_COUNT" != "0" ]; then

gradle jar

#./gradlew jar

TARGET_DIR=build

BUILT=1

elif [ "$SBT_COUNT" != "0" ]; then

echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list

curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo apt-key add

sudo apt update -y

sudo apt-get install sbt -y

sbt package

BUILT=1

fi

if [ "$BUILT" = "1" ] && [ -d "$TARGET_DIR" ]; then

jar cvf app.jar -C $TARGET_DIR .

${GITHUB_WORKSPACE}/sl analyze --wait --tag app.group=multi-vuln-app --app app-java --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --vcs-prefix-correction "*=src/main/java" --java app.jar

else

echo "Unable to build the project automatically. Please follow the instructions in our documentation to setup this project - https://docs.shiftleft.io/ngsast/analyzing-applications/java"

fi

env:

SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}

working-directory: java

##############################################

# Check for violations for each language

##############################################

Build-Rules:

runs-on: ubuntu-latest

needs: [ShiftLeft-Source-Analysis, Java-Analysis]

strategy:

fail-fast: false

matrix:

language: [go, js, python, java]

steps:

- uses: actions/checkout@v2

- name: Download ShiftLeft CLI

run: |

curl https://cdn.shiftleft.io/download/sl > ${GITHUB_WORKSPACE}/sl && chmod a+rx ${GITHUB_WORKSPACE}/sl

- name: Extract branch name

shell: bash

run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"

id: extract_branch

- name: Validate Build Rules per app

if: ${{ github.event_name == 'pull_request' }}

run: ${GITHUB_WORKSPACE}/sl check-analysis --v2 --config shiftleft.yml --app app-${{ matrix.language }} \

--branch "${{ github.head_ref || steps.extract_branch.outputs.branch }}" --report \

--github-pr-number=${{github.event.number}} --github-pr-user=${{ github.repository_owner }} \

--github-pr-repo=${{ github.event.repository.name }} --github-token=${{ secrets.GITHUB_TOKEN }}

env:

SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}

When done, click Start commit and follow the prompts to commit the file to your repo.

You'll see your newly configured workflow listed under the repository's Actions.

Step 3: Test Your Workflow

At this point, you're done with the configuration steps. You can check whether you successfully set up the GitHub Action by triggering the workflow (e.g., by creating a Pull Request).

You can click Status for additional details about the workflow's progress:

When done, you can see a summary of NG SAST's results on the PR:

You can get full details regarding the analysis from the ShiftLeft Dashboard.