Java
This article shows you how to analyze your applications written in Java using preZero. It assumes that you have already set up and authenticated with Qwiet.
Requirements
See Prerequisites for more information.
Analyzing your Java application
To analyze your compiled Java application, run:
sl analyze --app <name> --vcs-prefix-correction "*=src/main/java" --java [<path-to-JAR/WAR>]
| Parameter | Description |
|---|---|
--app <name> | The name of the application to be analyzed (maximum length: 100 characters) |
--vcs-prefix-correction "*=src/main/java" | The file path modification so that Qwiet's source code view works correctly. Note: Qwiet only supports the use of one correction, so this approach may not work for multi-module projects |
--java | The flag identifying the application's language |
<path> | The location of the application's .jar or .war file to be analyzed |
See the CLI reference for additional sl analyze options.
If you're using a templating framework like JavaServer Pages (JSP), the templates are included in the .jar / .war file analyzed.
SCA
To identify open-source vulnerabilities, Qwiet preZero automatically searches for build manifests in the project path you provided when running sl analyze. However, depending on how your project repo is structured, you may need to provide --oss-project-dir <project-path> so that Qwiet preZero can identify where your dependencies are located.
Additional parameters
The analysis accepts additional parameters after a double hyphen --. Such parameters, if valid, are passed directly to the Java plugin.
Combining multiple artifacts for analysis
You can combine multiple .jar files for analysis by preZero as follows:
sl analyze --app YOUR_APP --vcs-prefix-correction "*=src/main/java" --dep dependency.jar --java main.jar
Note that you can include --dep as many times as needed:
sl analyze --app YOUR_APP --vcs-prefix-correction "*=src/main/java" --dep dependency1.jar --dep dependency2.jar --dep dependency3.jar --java main.jar
Tagging results with your branch name
To include the branch name in your preZero results, allowing you to distinguish one set of results from another, add the following to your invocation of Qwiet:
sl analyze --tag branch=$(git symbolic-ref --short HEAD)
If you're working in a GitHub environment (e.g., GitHub Actions), you can also use --tag branch=${{ github.head_ref }} to populate your branch name.
If you don't provide a branch name, but Qwiet detects one available in your environment, it will use that name.
Memory
When running code analysis, we recommend using a heap size that includes an additional 20% to ensure sufficient physical memory on your server for other requirements (e.g., Java).
We recommend setting a process environment variable called SHIFTLEFT_JAVA_OPTS (e.g., SHIFTLEFT_JAVA_OPTS="-Xmx10g") that allows for the running of Qwiet to set the heap memory required for your particular application.
Source code view
The Qwiet Dashboard's findings list can include URLs that, when used, will direct you to the specific source code lines where the vulnerability occurs.
However, to leverage Qwiet's source code view with Java applications, you must augment the source code filepaths by passing --vcs-prefix-correction flag to sl analyze (e.g., --vcs-prefix-correction "*=src/main/java"). preZero uses the byte code, not the source code, for analysis, so you'll need to provide additional information about your file path structure to ensure that preZero generates the links properly.
Troubleshooting
If you have any issues scanning your project, please see our general troubleshooting page, as well as our Java-specific suggestions that follow.
Java requirements
We require Java 1.8 for code analysis. If Java or JRE is unavailable, the Qwiet CLI will automatically download and install JRE 1.8 to the $HOME/.shiftleft directory. You can confirm the JRE download in the verbose log:
[TRACE] Looking for prefix:(jre-) suffix:(.tar.gz)
[TRACE] removed 0 files
[TRACE] extracting /home/vsts/.shiftleft/jre172685618
[TRACE] extracting /home/vsts/.shiftleft/jre172685618/lib
[TRACE] extracting /home/vsts/.shiftleft/jre172685618/lib/resources.jar
[TRACE] extracting /home/vsts/.shiftleft/jre172685618/lib/logging.properties
Analysis time outs
If your analysis times out, there are two things to try:
- Adjusting the memory allocated to the scan
- Tuning your whitelist or blacklist
Adjusting the memory allocated to the scan
First, set the SL_CPG_OPTS environment variable as follows:
export SL_CPG_OPTS="-J-Xms2g -J-Xmx7g"
Set -Xmx to the max RAM available on the workstation running Qwiet. For large projects (e.g., those with over 500,000 lines of code), we ask for a minimum of 16 GB RAM.
After you set SL_CPG_OPTS, re-run the analysis and check the verbose log to make sure that the change is taking effect (ignore duplicate -Xms values):
[TRACE] args [/home/vsts/.shiftleft/jre/bin/java -Xms256m -Xms2g -Xmx6g -jar /home/vsts/.shiftleft/libplugin-latest.jar analyze /tmp/shiftleft-638314335/libplugin-state524922089 --]
...
[TRACE] args [/home/vsts/.shiftleft/jre/bin/java -Xms256m -Xms2g -Xmx6g -jar /home/vsts/.shiftleft/libplugin-latest.jar generate /tmp/shiftleft-638314335/libplugin-state524922089 --]
The libplugin analyze and generate steps should have memory-related settings.
At the end of the log, you can also see memory usage information; in the example below, the analysis used 24 GB RAM:
2020-12-21 15:19:19.793 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 18287 / 22905
2020-12-21 15:19:24.795 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 19912 / 22905
2020-12-21 15:19:29.861 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 2190 / 23290
2020-12-21 15:19:34.864 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 3299 / 23290
2020-12-21 15:19:39.866 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 4307 / 23290
2020-12-21 15:19:44.868 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 5508 / 23290
2020-12-21 15:19:49.875 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 6794 / 23290
2020-12-21 15:19:54.877 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 8082 / 23290
2020-12-21 15:19:59.879 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 10082 / 23290
2020-12-21 15:20:04.881 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 11237 / 23290
Tuning your whitelist or blacklist
If setting the SL_CPG_OPTS environment variable isn't fixing the problem, then you can tune the project's whitelist and blacklist.
Building a CPG is a resource-intensive process. Qwiet can analyze many frameworks and libraries, but it helps to focus on specific frameworks or namespaces. For example, few are interested in viewing static analysis reports for open source (OSS) libraries; thus, OSS libraries are pre-annotated and blacklisted by default.
However, your app may use libraries that haven't yet been blacklisted. For example, you might be using internal libraries common to your applications. Instead of reporting the same vulnerabilities in these libraries across all of your applications, it may make sense to blacklist these so that your findings list is more focused and actionable for your developers.
You can see if your project will benefit from whitelisting by looking for DEBUG Not replacing in your logs:
2020-12-21 15:17:00.583 [main] DEBUG N2020-12-21 15:17:00.682 [main] DEBUG Not replacing: /tmp/java2cpg_class3799848926187001016/io/netty/util/concurrent/GlobalEventExecutor$1.class
If you see 500+ occurrences of such a message, your project is a good candidate.
To manipulate your whitelist/blacklist, please contact Qwiet, making sure to provide:
- Your org id
- Your app name
- Your verbose logs
Qwiet will inform you about the amount of time required for this work.
VCS prefix corrections
You must modify the VCS URL that preZero sets by default for Java apps. To do this, include the following flag with sl analyze:
--vcs-prefix-correction "*=/src/main/java"
Similarly, for Scala projects, use:
--vcs-prefix-correction "*=/src/main/scala"
preZero currently cannot support multi-module repos where the src directory contains multiple separate modules. In such cases, we recommend using the API and applying a transformation script to fix the source code path.