sl check-analysis v2

The check-analysis feature allows you to compare your analysis results against a set of build rules you've defined and create a report of your code analysis results.

Authentication

Because sl check-analysis calls the Qwiet API, you'll need to create and set an environment variable called SHIFTLEFT_ACCESS_TOKEN. You can obtain your access token from the Dashboard's Account Settings page under Access Token.

Usage

To run: sl check-analysis --v2 [command options]

Command options

For easy reuse in future analyses, you can store some command-related information as environment variables (we've provided the specific environment variable below). Note that the values for options set via environment variables override those set in a configuration file (which, in turn, are overridden by those specified via command-line flags).

Option	Environment variable	Description
--app <name> or -a <name>	SHIFTLEFT_APP	Required. The name of the application analyzed by Qwiet
--branch <branch name>		The tag for the application branch analyzed; a shortcut for --source=tag.branch=<branch name>. If you provide this parameter, but there's no prior scan of the branch, Qwiet AI by Harness will ignore this value
--config <filepath>		Required. The YAML configuration file containing your build rules (default: ./shiftleft.yml)
--create-jira-issues	SHIFTLEFT_CHECK_ANALYSIS_CREATE_JIRA_ISSUES	Send findings that are flagged by build rules to Jira, where each finding is logged as a Jira issue. Requires a configured Jira integration
--github-api-base-url	GITHUB_API_BASE_URL	The base URL for your GitHub Enterprise API Server
--github-pr-user <name>	SHIFTLEFT_GH_PR_REPO_OWNER=<name>	The GitHub PR User who owns the repo
--github-pr-repo <name>	SHIFTLEFT_GH_PR_REPO_NAME=<name>	The GitHub repo name where the PR is open
--github-pr-number <number>	SHIFTLEFT_GH_PR_NUMBER=<number>	The GitHub pull request number
--github-token <token>	GITHUB_TOKEN=<token>	A token necessary to submit comments to GitHub
--no-build-rules		Skip build rules and only show the report
--max-findings		The maximum number of findings to return when --no-build-rules is set. This is the same as the rule option num_findings in a build rules .yml file. Default: 5
--multilanguage		If true, the project names and scan IDs refer to multilanguage projects and scans. Overrides multilanguage option in config file. Can be set to false explicitly (--multilanguage=false)
--report-file <filepath>		Indicates that the generated report should be saved as a file. Cannot be combined with the --github-* options.
--source <scan id / branch name>		The source scan; used in conjunction with --target to identify differences in fixed and newly introduced vulnerabilities. If there is a scan for a specific branch, you should include a source for comparison purposes
--target <scan id / branch name>		The target scan; used in conjunction with --source to identify differences in fixed and newly introduced vulnerabilities
--template <filepath>		Path to a file containing the custom template for the check analysis report
--dump-template		Prints the template to be used by check-analysis for its report
--timeout		Requested timeout in seconds (e.g., 60s). This is the amount of time Qwiet will wait for the command to finish. Default: 10s
--wait or -w	SHIFTLEFT_CHECK_ANALYSIS_WAIT	Wait for the analysis of the target scan to finish before applying build rules. Requires version 2 build rules and multilanguage (polyglot) scans. See Waiting for analysis to complete
--wait-timeout	SHIFTLEFT_CHECK_ANALYSIS_WAIT_TIMEOUT	How long --wait waits for the analysis to finish (e.g., 30m). Default: 30m
--wait-grace	SHIFTLEFT_CHECK_ANALYSIS_WAIT_GRACE	When no explicit --target scan.<N> is given, how long --wait waits for a new scan to appear before deciding there is nothing to wait for (e.g., 60s). Default: 1m

Sources and targets

When providing values for --source or --target, you can provide either the scan ID (e.g., scan.4) or the branch name (e.g., tag.branch=master for your master branch). For example, if your app is myApp, you might provide the scan ID or the branch name as follows:

• Using the scan ID: sl check-analysis --app myApp --config ~/shiftleft.yml --target scan.4
• Using the branch name: sl check-analysis --app myApp --config ~/shiftleft.yml --target tag.branch=master

Comparing the current and previous scans

If you set --source=scan.previous, you can compare the latest scan with the previous scan. For example:

sl check-analysis --source=scan.previous --config ~/shiftleft.yml --app shiftleft-demo

You can also compare the latest scan with the previous scan for a specific branch. For example:

sl check-analysis --source=scan.previous --target=tag.branch=pr-branch --config ~/shiftleft.yml --app shiftleft-demo

Comparing a specific scan with the previous scan

Similarly, you can also compare a specific target scan with the previous scan. For instance, the example below compares scan 2 with the previous scan (1).

sl check-analysis --source=scan.previous --target=scan.2 --config ~/shiftleft.yml --app shiftleft-demo

Multilanguage applications

When targeting multilanguage applications, provide the --multilanguage option and make sure the scan ID is a polyglot scan ID. For example:

sl check-analysis --v2 --app qwiet-js-demo --target scan.4 \
    --no-build-rules --report --multilanguage

A polyglot scan ID is the scan ID for a multilanguage application, and it might differ from individual applications (compound element) scans IDs if certain languages or platforms have been analyzed and others haven't. The polyglot scan ID is the same scan ID shown on the Application page on the Qwiet AI by Harness UI. It's also displayed as part of the output for the sl analyze command, as well as the structured output.

Multi-language analysis: enabled, main language "JSSRC", selected languages [SECRETS], secrets enabled
This will be multilanguage scan 4 of compound "qwiet-js-demo".

Here's an example that uses sl analyze structured output to get the polyglot scan ID:

sl analyze --wait --structured-output --structured-output-file-path \
    sl-analyze-output.json --app qwiet-js-demo --jssrc .

SCAN_ID=$(jq '.polyglot_scan_id' sl-analyze-output.json)

sl check-analysis --v2 --app qwiet-js-demo --target scan.${SCAN_ID} \
    --config ./default_rules.yml --report --multilanguage

You should see a message similar to this:

 Checking analysis of application `qwiet-js-demo` against 3 build rules. 

Checking findings on multilanguage scan 4.
   - Target comprises scans qwiet-js-demo 4, qwiet-js-demo_element_JSSRC 2.

Waiting for analysis to complete

Use --wait when check-analysis runs immediately after sl analyze (e.g. without --wait) and you want build rules applied only after the target scan's analysis has finished. This is especially useful in CI/CD pipelines for multilanguage applications, where different language components may finish at different times.

--wait applies only to version 2 build rules with multilanguage (polyglot) scans. If you use version 1 build rules (without --v2 or version: 2 in your configuration file), the CLI prints a warning to stderr and ignores the flag. If multilanguage is not enabled, the command exits with an error.

While waiting, the CLI polls the Qwiet API about every 10 seconds until every component of the target scan has reached a terminal state (no longer in progress). If any component failed, timed out, or did not succeed, --wait continues unless --strict has been provided, in which case it exits with an error and build rules are not applied.

Targeting a specific scan

When you provide an explicit polyglot scan ID (for example, --target scan.4), --wait polls that scan until its analysis completes or the wait budget runs out:

sl check-analysis --v2 --app qwiet-js-demo --target scan.4 \
    --config ./shiftleft.yml --multilanguage --wait

Progress messages are written to stderr, not to the check-analysis report (which may be a file or a GitHub pull request comment).

Targeting a branch

When you target a branch instead of a specific scan ID (for example, --target tag.branch=main or --branch main), there is no scan ID to pin at the start of the run. In that case, --wait first spends up to --wait-grace (default: 1m) watching for a new scan to appear:

• If a scan is already visible and still in progress, --wait follows it immediately.
• If the newest scan has already finished, it may be from a prior pipeline run. The CLI waits up to --wait-grace for a newer scan to appear.
• If no newer scan appears within the grace period, --wait exits successfully without waiting further — there is nothing new to wait for.
• If a newer scan appears, --wait then waits up to --wait-timeout (default: 30m) for that scan's analysis to complete.

sl check-analysis --v2 --app qwiet-js-demo --target tag.branch=main \
    --config ./shiftleft.yml --multilanguage --wait

Timeouts and --strict

If the analysis is still in progress when --wait-timeout elapses, the CLI prints a warning and continues with check-analysis against whatever scan the server resolves as the target. To treat a wait timeout as a fatal error instead, pass the global --strict flag or set strict: true in your configuration file.

Client errors from the API other than 404 Not Found abort the wait immediately. A 404, server errors (5xx), and transient network errors are retried on the next poll.

Summary of findings

sl check-analysis will output a report summarizing the findings obtained by Qwiet AI by Harness.

This report will include:

• Whether the report is for a single scan or a comparison of two scans and a link to the dashboard page for that scan/comparison. When comparing two scans, Qwiet only considers findings present in the target that are not present in the source (and not marked as ignored or resolved).

• The following information for each rule:

  • A list of the findings that matched the rule (by default, Qwiet includes five findings, but you can change this using the num_findings rule option). The IDs for these findings link to the appropriate Dashboard page with more details.

  • Summaries of the distribution of those findings by severity, and (where appropriate) the finding type, finding category, language, and OWASP category.

Sample report

Qwiet will print a report of its findings to your GitHub pull request:

Custom templates

You can use a custom template for your report to define the information that Qwiet includes and how the data is presented.

Once you've defined your custom template, add the --template <path-to-template-file> flag to sl check-analysis.

Rules failures and error codes

If your build rules fail, Qwiet will return an error code of 2 (an error code of 1 indicates all other types of failure, e.g., network issues, incorrect template, etc.).