sl check-analysis

The Check Analysis feature allows you to compare your analysis results against a set of build rules you've defined. You can also use its reporting functionality to return a Markdown file of your code analysis results.

API Token

Because sl check-analysis calls the ShiftLeft API, you'll need to create and set a process environment variable called SHIFTLEFT_API_TOKEN with your API token.

You can obtain your API token from the Dashboard's Account Settings page under Public API Token (Beta).

Usage

To run: sl check-analysis [command options]

Command Options

OptionEnvironment VariableDescription
--app <name> or -a <name>The name of the application analyzed by ShiftLeft
--branch <name>The tag for the application branch analyzed
--config <filepath>The YAML configuration file containing your build rules (default: ./shiftleft.yml)
--create-jira-issuesSHIFTLEFT_CHECK_ANALYSIS_CREATE_JIRA_ISSUESSend findings that are flagged by build rules to Jira, where each finding is logged as a Jira issue. Requires a configured Jira integration
--source <scan id / branch name>The source application; used in conjunction with --target to identify differences in fixed and newly introduced vulnerabilities
--target <scan id / branch name>The target application; used in conjunction with --source to identify differences in fixed and newly introduced vulnerabilities
--template <filepath>Path to a file containing the template for the check analysis report
--no-build-rulesSkip build rules and only show the report
--reportEnable check analysis to return a plain text report of analysis results instead of checking build rules
--github-api-base-urlGITHUB_API_BASE_URLThe base URL for your GitHub Enterprise API Server
--github-pr-user <name>SHIFTLEFT_GH_PR_REPO_OWNER=<name>The GitHub PR User who owns the repo
--github-pr-repo <name>SHIFTLEFT_GH_PR_REPO_NAME=<name>The GitHub repo name where the PR is open
--github-pr-number <number>SHIFTLEFT_GH_PR_NUMBER=<number>The GitHub pull request number
--github-token <token>GITHUB_TOKEN=<token>A token necessary to submit comments to GitHub
--reportEnables sl check-analysis to return a plain text report of analysis results instead of running build rules
--report-file <filepath>Indicates the generated report should be saved as a file. Must be used in conjunction with the --report flag

When providing values for --source or --target, you can provide either the scan ID (e.g., scan.4) or the branch name (e.g., tag.branch=master for your master branch). For example, if your app is myApp, you might provide the scan ID or the branch name as follows:

  • Using the scan ID: sl check-analysis --app myApp --config ~/shiftleft.yml --target scan.4
  • Using the branch name: sl check-analysis --app myApp --config ~/shiftleft.yml --target tag.branch=master

Comparing the Current and Previous Scans

If you set --source=previous, you can compare the latest scan in a branch with the same branch's previous scan. To use this feature, you must set target to a branch. For example:

sl check-analysis --source=previous --target=tag.branch=pr-branch --config ~/shiftleft.yml --app shiftleft-demo

Reporting Summary of Findings

If you would like a report summarizing the findings obtained by NG SAST, you can append the --report flag to sl check-analysis.

Run:

sl check-analysis --branch=master --app <AppName> --report

to obtain the following pieces of information:

  • Total Number of Findings
  • Number of Critical Findings
  • Number of Moderate Findings
  • Number of Informational Findings

If you provide both the --source and the --target, then you will also get the following counts:

  • The number of new findings introduced in the --target version
  • The number of fixed findings that were present in --source but not --target
  • The number of common findings between the two versions

The command used to obtain this additional information is similar to this example:

sl check-analysis --source scan.1 --target scan.2 --app <AppName> --report

You can print the report without running build rules using the --no-build-rules flag:

sl check-analysis --source scan.1 --target scan.2 --app <AppName> --report --no-build-rules

If you'd like the results exported to file, use the --report-file flag. For example, the following generates a file named report.md to the tmp folder:

sl analyze ... --report --report-file /tmp/report.md

Output

NG SAST uses the following default template to display your results:

![shiftleft logo](https://www.shiftleft.io/static/images/ShiftLeft_logo_white.svg)
## Summary
ShiftLeft NextGen analysis found {total_count} vulnerabilities in this PR
| Severity | Count |
|:-----------|:---------|
| Critical | {critical_count} |
| Moderate | {moderate_count} |
| Info | {info_count} |
| Category | Count |
|:-----------|:---------|
| New | {new_count} |
| Fixed | {fixed_count} |
| Common | {common_count} |
Failed build rule ID "{failed_rule_id}": {failed_total} matched vulnerabilities (configured threshold is {failure_threshold})

The following is what the output looks rendered:

Rendered Report

Custom Templates

You can use your own template, as long as you use the keywords indicated in the default template (e.g., {critical_count}, {moderate_count}, {failed_rule_id}).

To do so, add the --template <path-to-template-file> flag to sl check-analysis.

Include Comparison Summary Information in Your GitHub Pull Request

You can enable sl check-analysis to post NG SAST results as a comment to your GitHub pull request.

SL Findings on the PR

To do so, you will need to provide the following information, either directly in your sl check analysis command or as an environment variable:

Variablesl check analysis flagEnvironment Variable
The user/org that owns the repo--github-pr-userSHIFTLEFT_GH_PR_REPO_OWNER
The name of the repository--github-pr-repoSHIFTLEFT_GH_PR_REPO_NAME
The pull request number--github-pr-numberSHIFTLEFT_GH_PR_NUMBER
Your GitHub access token--github-tokenGITHUB_TOKEN
note

See GitHub's instructions on creating a personal access token if you don't already have one. During this process, you will be asked to set the scopes of the token; the only mandatory scopes for this process are those related to repo.

For example, if you want the counts of critical, moderate, and informational findings, use this command:

sl check-analysis --branch=master --app <AppName> --report --github-pr-user=myUserName --github-pr-repo=myGitHubRepo --github-pr-number=1

If, however, you also want information gained by comparing two scans, use:

sl check-analysis --source scan.1 --target scan.2 --app <AppName> --report --github-pr-user=myUserName --github-pr-repo=myGitHubRepo --github-pr-number=1

For GitHub Enterprise Server Users

If you're working with a GitHub Enterprise API server (on-prem or in a dedicated cloud), there's an additional step you need to take to facilitate communication with ShiftLeft. You must provide the base URL ShiftLeft should use with the --github-api-base-url flag. This is required for tasks like posting comments to a pull request using sl check-analysis.

--github-api-base-url https://github.example.com/

This URL typically ends with /api/v3/.

As an example, the full command you'd use to post comments to pull requests via a GitHub Enterprise server is:

sl check-analysis --app YOUR_APP --source scan.1 --target scan.2 --github-pr-user YOUR_GH_USER --github-pr-repo YOUR_REPO --github-pr-number YOUR_PR_NUMBER --github-token YOUR_GH_TOKEN --github-api-base-url https://github.example.com/api/v3 --report

Bypassing TLS Security Checks

Occasionally, the GitHub server uses a TLS certificate that ShiftLeft can't verify. If this happens, you can bypass the TLS security checks on the HTTPS connection to the GitHub Enterprise server, forcing sl check-analysis to accept any TLS certificate with any hostname that the server presents. However, we do not recommend that you use this flag; bypassing TLS security checks makes you susceptible to machine-in-the-middle attacks, so use the flag only as a last resort:

--github-api-force-insecure-tls

For example:

sl check-analysis --app YOUR_APP --source scan.1 --target scan.2 --github-pr-user YOUR_GH_USER --github-pr-repo YOUR_REPO --github-pr-number YOUR_PR_NUMBER --github-token YOUR_GH_TOKEN --github-api-force-insecure-tls --github-api-base-url https://github.example.com/api/v3 --report