Scan Multi-Language Repositories in GitHub

note

preZero can now analyze multilanguage applications with a single CLI command. Please refer to our Multilanguage documentation to learn more about this feature.

This article will show you how to integrate Qwiet preZero into your GitHub Pull Request (PR) workflow for automated code analysis using GitHub Actions if your application utilizes multiple programming languages.

Step 1: Create your GitHub secret

GitHub's secrets are encrypted environment variables that protect information while making them available for use in GitHub Actions workflows. They are specific to your GitHub repository. You will use secrets to provide an access token that lets GitHub interact with Qwiet.

To create a secret specific to your GitHub repository, go to Settings > Secrets. Click New Secret. You will need to create a secret called SHIFTLEFT_ACCESS_TOKEN to store your Qwiet CI config token.

You can create your CI token in the Qwiet Dashboard.

Step 2: Create your GitHub Action and define its workflow

GitHub Actions offers you workflow automation functionality. You can use this to automatically run preZero (e.g., when you create a new Pull Request).

To create a new GitHub Action for your repository, click Actions. If this is your first time setting up a GitHub Action, click set up a workflow yourself near the top-left; otherwise, click New workflow, then select set up a workflow yourself.

You will be redirected to a YAML editing window. Rename the file (if desired), and provide the following script to invoke preZero.

---
# This workflow integrates Qwiet with GitHub
name: Qwiet preZero

on:
  pull_request:
  workflow_dispatch:

jobs:
  #########################################################
  # Scan dynamic languages, such as JavaScript/TypeScript,
  # Python, and Go.
  #########################################################
  Qwiet-Source-Analysis:
    runs-on: ubuntu-latest
    # Use the shiftleft/core image, which supports multiple
    # languages, including JavaScript/TypeScript, Python, and Go
    container:
      image: shiftleft/core:latest
    strategy:
      fail-fast: false
      matrix:
        language: [go, js, python]
    steps:
    - uses: actions/checkout@v2
    - name: Extract branch name
      shell: bash
      run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
      id: extract_branch
    # Perform any app specific build or restore here
    - name: App specific build
      if: ${{ matrix.language == 'python' }}
      run: |
        pip install -r python/requirements.txt
    # Run Qwiet for the source languages
    - name: Scan source languages
      if: ${{ matrix.language == 'js' || matrix.language == 'python' }}
      run: |
        sl analyze --tag app.group=multi-vuln-app --app app-${{ matrix.language }} --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --${{ matrix.language }} .
      env:
        SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
    # For Go, the last argument used must be identical to the one used with `go build`
    - name: Scan go application
      if: ${{ matrix.language == 'go' }}
      run: |
        cd go && sl analyze --tag app.group=multi-vuln-app --app app-${{ matrix.language }} --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --${{ matrix.language }} ./...
      env:
        SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
  ##############################################
  # Scan Java applications
  ##############################################
  Java-Analysis:
    runs-on: ubuntu-latest    
    steps:
    - uses: actions/checkout@v2
    - name: Setup Java JDK
      uses: actions/setup-java@v1.4.3
      with:
        java-version: 1.8
    - name: Download Qwiet cli
      run: |
        curl https://cdn.shiftleft.io/download/sl > ${GITHUB_WORKSPACE}/sl && chmod a+rx ${GITHUB_WORKSPACE}/sl
    - name: Extract branch name
      shell: bash
      run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
      id: extract_branch
    # Try to auto build Maven, Gradle, and/or sbt projects
    - name: Build and Analyze
      run: |
        TARGET_DIR=target
        BUILT=0
        POM_COUNT=$(find . -maxdepth 1 -type f -name "pom.xml" -not -path '*/\.git/*' | wc -l | tr -d " ")
        GRADLE_COUNT=$(find . -maxdepth 1 -type f -name "build.gradle" -not -path '*/\.git/*' | wc -l | tr -d " ")
        SBT_COUNT=$(find . -maxdepth 1 -type f -name "build.sbt" -not -path '*/\.git/*' | wc -l | tr -d " ")
        if [ "$POM_COUNT" != "0" ]; then
          mvn compile package
          BUILT=1
        elif [ "$GRADLE_COUNT" != "0" ]; then
          gradle jar
          #./gradlew jar
          TARGET_DIR=build
          BUILT=1
        elif [ "$SBT_COUNT" != "0" ]; then
          echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list
          curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo apt-key add
          sudo apt update -y
          sudo apt-get install sbt -y
          sbt package
          BUILT=1
        fi
        if [ "$BUILT" = "1" ] && [ -d "$TARGET_DIR" ]; then
          jar cvf app.jar -C $TARGET_DIR .
          ${GITHUB_WORKSPACE}/sl analyze --wait --tag app.group=multi-vuln-app --app app-java --tag branch=${{ github.head_ref || steps.extract_branch.outputs.branch }} --vcs-prefix-correction "*=src/main/java" --java app.jar
        else
          echo "Unable to build the project automatically. Please follow the instructions in our documentation to setup this project - https://docs.shiftleft.io/sast/analyzing-applications/java"  
        fi
      env:
        SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}
      working-directory: java
  ##############################################
  # Check for violations for each language
  ##############################################
  Build-Rules:
    runs-on: ubuntu-latest
    needs: [Qwiet-Source-Analysis, Java-Analysis]
    strategy:
      fail-fast: false
      matrix:
        language: [go, js, python, java]
    steps:
    - uses: actions/checkout@v2
    - name: Download Qwiet CLI
      run: |
        curl https://cdn.shiftleft.io/download/sl > ${GITHUB_WORKSPACE}/sl && chmod a+rx ${GITHUB_WORKSPACE}/sl
    - name: Extract branch name
      shell: bash
      run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
      id: extract_branch
    - name: Validate Build Rules per app
      if: ${{ github.event_name == 'pull_request' }}
      run: ${GITHUB_WORKSPACE}/sl check-analysis --v2 --config shiftleft.yml --app app-${{ matrix.language }} \
            --branch "${{ github.head_ref || steps.extract_branch.outputs.branch }}" --report \
            --github-pr-number=${{github.event.number}} --github-pr-user=${{ github.repository_owner }} \
            --github-pr-repo=${{ github.event.repository.name }}  --github-token=${{ secrets.GITHUB_TOKEN }}
      env:
        SHIFTLEFT_ACCESS_TOKEN: ${{ secrets.SHIFTLEFT_ACCESS_TOKEN }}

When done, click Start commit and follow the prompts to commit the file to your repo.

You'll see your newly configured workflow listed under the repository's Actions.

Step 3: Test Your Workflow

At this point, you're done with the configuration steps. You can check whether you successfully set up the GitHub Action by triggering the workflow (e.g., by creating a Pull Request).

You can click Status for additional details about the workflow's progress:

When done, you can see a summary of preZero's results on the PR:

You can get full details regarding the analysis from the Qwiet Dashboard.