sl check-analysis v2

The check-analysis feature allows you to compare your analysis results against a set of build rules you've defined and create a report of your code analysis results.

Authentication

Because sl check-analysis calls the Qwiet API, you'll need to create and set an environment variable called SHIFTLEFT_ACCESS_TOKEN. You can obtain your access token from the Dashboard's Account Settings page under Access Token.

Usage

To run: sl check-analysis --v2 [command options]

Command options

For easy reuse in future analyses, you can store some command-related information as environment variables (we've provided the specific environment variable below). Note that the values for options set via environment variables override those set in a configuration file (which, in turn, are overridden by those specified via command-line flags).

Option	Environment variable	Description
--app <name> or -a <name>	SHIFTLEFT_APP	Required. The name of the application analyzed by Qwiet
--branch <branch name>		The tag for the application branch analyzed; a shortcut for --source=tag.branch=<branch name>. If you provide this parameter, but there's no prior scan of the branch, Qwiet preZero will ignore this value
--config <filepath>		Required. The YAML configuration file containing your build rules (default: ./shiftleft.yml)
--create-jira-issues	SHIFTLEFT_CHECK_ANALYSIS_CREATE_JIRA_ISSUES	Send findings that are flagged by build rules to Jira, where each finding is logged as a Jira issue. Requires a configured Jira integration
--github-api-base-url	GITHUB_API_BASE_URL	The base URL for your GitHub Enterprise API Server
--github-pr-user <name>	SHIFTLEFT_GH_PR_REPO_OWNER=<name>	The GitHub PR User who owns the repo
--github-pr-repo <name>	SHIFTLEFT_GH_PR_REPO_NAME=<name>	The GitHub repo name where the PR is open
--github-pr-number <number>	SHIFTLEFT_GH_PR_NUMBER=<number>	The GitHub pull request number
--github-token <token>	GITHUB_TOKEN=<token>	A token necessary to submit comments to GitHub
--no-build-rules		Skip build rules and only show the report
--max-findings		The maximum number of findings to return when --no-build-rules is set. This is the same as the rule option num_findings in a build rules .yml file. Default: 5
--report-file <filepath>		Indicates that the generated report should be saved as a file. Cannot be combined with the --github-* options.
--source <scan id / branch name>		The source scan; used in conjunction with --target to identify differences in fixed and newly introduced vulnerabilities. If there is a scan for a specific branch, you should include a source for comparison purposes
--target <scan id / branch name>		The target scan; used in conjunction with --source to identify differences in fixed and newly introduced vulnerabilities
--template <filepath>		Path to a file containing the custom template for the check analysis report
--dump-template		Prints the template to be used by check-analysis for its report
--timeout		Requested timeout in seconds (e.g., 60s). This is the amount of time Qwiet will wait for the command to finish. Default: 10s

Sources and targets

When providing values for --source or --target, you can provide either the scan ID (e.g., scan.4) or the branch name (e.g., tag.branch=master for your master branch). For example, if your app is myApp, you might provide the scan ID or the branch name as follows:

• Using the scan ID: sl check-analysis --app myApp --config ~/shiftleft.yml --target scan.4
• Using the branch name: sl check-analysis --app myApp --config ~/shiftleft.yml --target tag.branch=master

Comparing the current and previous scans

If you set --source=scan.previous, you can compare the latest scan with the previous scan. For example:

sl check-analysis --source=scan.previous --config ~/shiftleft.yml --app shiftleft-demo

You can also compare the latest scan with the previous scan for a specific branch. For example:

sl check-analysis --source=scan.previous --target=tag.branch=pr-branch --config ~/shiftleft.yml --app shiftleft-demo

Comparing a specific scan with the previous scan

Similarly, you can also compare a specific target scan with the previous scan. For instance, the example below compares scan 2 with the previous scan (1).

sl check-analysis --source=scan.previous --target=scan.2 --config ~/shiftleft.yml --app shiftleft-demo

Summary of findings

sl check-analysis will output a report summarizing the findings obtained by preZero.

This report will include:

• Whether the report is for a single scan or a comparison of two scans and a link to the dashboard page for that scan/comparison. When comparing two scans, Qwiet only considers findings present in the target that are not present in the source (and not marked as ignored or resolved).

• The following information for each rule:

  • A list of the findings that matched the rule (by default, Qwiet includes five findings, but you can change this using the num_findings rule option). The IDs for these findings link to the appropriate Dashboard page with more details.

  • Summaries of the distribution of those findings by severity, and (where appropriate) the finding type, finding category, language, and OWASP category.

Sample report

Qwiet will print a report of its findings to your GitHub pull request:

Custom templates

You can use a custom template for your report to define the information that Qwiet includes and how the data is presented.

Once you've defined your custom template, add the --template <path-to-template-file> flag to sl check-analysis.

Rules failures and error codes

If your build rules fail, Qwiet will return an error code of 2 (an error code of 1 indicates all other types of failure, e.g., network issues, incorrect template, etc.).