sl check-analysis

sl check-analysis v1 has been deprecated. Please use sl check-analysis v2; see migrations for additional information.

The check-analysis feature allows you to compare your results against a set of build rules. You can also use its reporting functionality to return a Markdown file of your results.

Authentication

Because sl check-analysis calls the ShiftLeft API, you'll need to create and set an environment variable called SHIFTLEFT_ACCESS_TOKEN. You can obtain your access token from the Dashboard's Account Settings page under Access Token.

Usage

To run: sl check-analysis [command options]

Command options

For easy reuse in future analyses, you can store some command-related information as environment variables (we've provided the specific environment variable below). Note that the values for options set via environment variables override those set in a configuration file (which, in turn, are overridden by those specified via command-line flags).

OptionEnvironment variableDescription
--app <name> or -a <name>The name of the application analyzed by ShiftLeft
--branch <name>The tag for the application branch analyzed
--config <filepath>The YAML configuration file containing your build rules (default: ./shiftleft.yml)
--create-jira-issuesSHIFTLEFT_CHECK_ANALYSIS_CREATE_JIRA_ISSUESSend findings that are flagged by build rules to Jira, where each finding is logged as a Jira issue. Requires a configured Jira integration
--github-api-base-urlGITHUB_API_BASE_URLThe base URL for your GitHub Enterprise API Server
--github-pr-user <name>SHIFTLEFT_GH_PR_REPO_OWNER=<name>The GitHub PR User who owns the repo
--github-pr-repo <name>SHIFTLEFT_GH_PR_REPO_NAME=<name>The GitHub repo name where the PR is open
--github-pr-number <number>SHIFTLEFT_GH_PR_NUMBER=<number>The GitHub pull request number
--github-token <token>GITHUB_TOKEN=<token>A token necessary to submit comments to GitHub
--no-build-rulesSkip build rules and only show the report
--reportEnables sl check-analysis to return a plain text report of analysis results instead of running build rules
--report-file <filepath>Indicates the generated report should be saved as a file. Must be used in conjunction with the --report flag
--source <scan id / branch name>The source application; used in conjunction with --target to identify differences in fixed and newly introduced vulnerabilities
--target <scan id / branch name>The target application; used in conjunction with --source to identify differences in fixed and newly introduced vulnerabilities
--timeout <timeoutValue>Timeout to wait for the command to finish. Default: 10s
--template <filepath>Path to a file containing the template for the check analysis report

When providing values for --source or --target, you can provide either the scan ID (e.g., scan.4) or the branch name (e.g., tag.branch=master for your master branch). For example, if your app is myApp, you might provide the scan ID or the branch name as follows:

  • Using the scan ID: sl check-analysis --app myApp --config ~/shiftleft.yml --target scan.4
  • Using the branch name: sl check-analysis --app myApp --config ~/shiftleft.yml --target tag.branch=master

Comparing the current and previous scans

If you set --source=previous, you can compare the latest scan in a branch with the same branch's previous scan. You must set target to a branch to use this feature. For example:

sl check-analysis --source=previous --target=tag.branch=pr-branch --config ~/shiftleft.yml --app shiftleft-demo

Summary of findings

If you would like a report summarizing the findings obtained by NG SAST, you can append the --report flag to sl check-analysis.

Run:

sl check-analysis --branch=master --app <AppName> --report

to obtain the following pieces of information:

  • Total Number of Findings
  • Number of Critical Findings
  • Number of Moderate Findings
  • Number of Informational Findings

If you provide both the --source and the --target, then you will also get the following counts:

  • The number of new findings introduced in the --target version
  • The number of fixed findings that were present in --source but not --target
  • The number of common findings between the two versions

The command used to obtain this additional information is similar to this example:

sl check-analysis --source scan.1 --target scan.2 --app <AppName> --report

You can print the report without running build rules using the --no-build-rules flag:

sl check-analysis --source scan.1 --target scan.2 --app <AppName> --report --no-build-rules

Use the'-- report-file' flag if you'd like the results exported to file. For example, the following generates a file named report.md to the tmp folder:

sl check-analysis ... --report --report-file /tmp/report.md

Output

NG SAST uses the following default template to display your results:

![shiftleft logo](https://app.shiftleft.io/static/images/ShiftLeft_logo_white.svg)
## Summary
ShiftLeft NextGen analysis found {total_count} vulnerabilities in this PR
| Severity | Count |
|:-----------|:---------|
| Critical | {critical_count} |
| Moderate | {moderate_count} |
| Info | {info_count} |
| Category | Count |
|:-----------|:---------|
| New | {new_count} |
| Fixed | {fixed_count} |
| Common | {common_count} |
Failed build rule ID "{failed_rule_id}": {failed_total} matched vulnerabilities (configured threshold is {failure_threshold})

The following is what the output looks rendered:

Rendered Report

Custom templates

You can use your own template, as long as you use the keywords indicated in the default template (e.g., {critical_count}, {moderate_count}, {failed_rule_id}).

To do so, add the --template <path-to-template-file> flag to sl check-analysis.

Include comparison summary information in your GitHub pull request

You can enable sl check-analysis to post NG SAST results as a comment to your GitHub pull request.

SL Findings on the PR

To do so, you will need to provide the following information, either directly in your sl check analysis command or as an environment variable:

Variablesl check analysis flagEnvironment variable
The user/org that owns the repo--github-pr-userSHIFTLEFT_GH_PR_REPO_OWNER
The name of the repository--github-pr-repoSHIFTLEFT_GH_PR_REPO_NAME
The pull request number--github-pr-numberSHIFTLEFT_GH_PR_NUMBER
Your GitHub access token--github-tokenGITHUB_TOKEN
note

See GitHub's instructions on creating a personal access token if you don't already have one. During this process, you will be asked to set the scopes of the token; the only mandatory scopes for this process are those related to repo.

For example, if you want the counts of critical, moderate, and informational findings, use this command:

sl check-analysis --branch=master --app <AppName> --report --github-pr-user=myUserName --github-pr-repo=myGitHubRepo --github-pr-number=1

If, however, you also want information gained by comparing two scans, use:

sl check-analysis --source scan.1 --target scan.2 --app <AppName> --report --github-pr-user=myUserName --github-pr-repo=myGitHubRepo --github-pr-number=1

For GitHub Enterprise Server users

If you're working with a GitHub Enterprise API server (on-prem or in a dedicated cloud), there's an additional step you need to take to facilitate communication with ShiftLeft. You must provide the base URL ShiftLeft should use with the --github-api-base-url flag. This is required for tasks like posting comments to a pull request using sl check-analysis.

--github-api-base-url https://github.example.com/

This URL typically ends with /api/v3/.

As an example, the entire command you'd use to post comments to pull requests via a GitHub Enterprise server is:

sl check-analysis --app YOUR_APP --source scan.1 --target scan.2 --github-pr-user YOUR_GH_USER --github-pr-repo YOUR_REPO --github-pr-number YOUR_PR_NUMBER --github-token YOUR_GH_TOKEN --github-api-base-url https://github.example.com/api/v3 --report

Bypassing TLS security checks

For some deployments, the GitHub server uses a TLS certificate that ShiftLeft can't verify. If this happens, you can bypass the TLS security checks on the HTTPS connection to the GitHub Enterprise server, forcing sl check-analysis to accept any TLS certificate with any server's hostname. However, we do not recommend that you use this flag; bypassing TLS security checks makes you susceptible to machine-in-the-middle attacks, so use the flag only as a last resort:

--github-api-force-insecure-tls

For example:

sl check-analysis --app YOUR_APP --source scan.1 --target scan.2 --github-pr-user YOUR_GH_USER --github-pr-repo YOUR_REPO --github-pr-number YOUR_PR_NUMBER --github-token YOUR_GH_TOKEN --github-api-force-insecure-tls --github-api-base-url https://github.example.com/api/v3 --report