Java

This article shows you how to analyze your applications written in Java using NG SAST. It assumes that you have already set up and authenticated with ShiftLeft.

Requirements

  • NG SAST supports the analysis of applications written in Java 7, 8, 9, 10, 11, 14, 15, and 17.
  • Your build environment must have at least 16 GB of memory available.
  • For SCA: To identify open-source vulnerabilities in Java/Scala applications, ShiftLeft CORE requires one of the following package formats: Maven (pom.xml), Gradle (build.gradle, .kts), Scala (SBT)

NG SAST utilizes Java SE Runtime Environment 8; if necessary, you can install using sl update jre.

Building your application

NG SAST's code analysis is performed on the compiled application bytecode (not on the source code). As such, you must build your application before analyzing it with NG SAST.

Some build tools you might consider include Maven, Gradle, sbt, etc.

Analyzing your Java application

To analyze your Java application, run:

sl analyze --app <name> --vcs-prefix-correction "*=src/main/java" --java [<path-to-JAR/WAR>]
ParameterDescription
--app <name>The name of the application to be analyzed
--vcs-prefix-correction "*=src/main/java"The file path modification so that ShiftLeft's source code view works correctly. Note: ShiftLeft only supports the use of one correction, so this approach may not work for multi-module projects
--javaThe flag identifying the application's language
<path>The location of the application's .jar or .war file to be analyzed

See the CLI reference for additional sl analyze options.

If you're using a templating framework like JavaServer Pages (JSP), the templates are included in the .jar / .war file analyzed.

SCA

To identify open-source vulnerabilities, ShiftLeft CORE automatically searches for build manifests in the project path you provided when running sl analyze. However, depending on how your project repo is structured, you may need to provide --oss-project-dir <project-path> so that ShiftLeft CORE can identify where your dependencies are located.

Combining multiple artifacts for analysis

You can combine multiple .JAR files for analysis by NG SAST as follows:

sl analyze --app YOUR_APP --vcs-prefix-correction "*=src/main/java" --java first.jar --dep second.jar

Note that you can include --dep as many times as needed:

sl analyze --app YOUR_APP --vcs-prefix-correction "*=src/main/java" --java first.jar --dep second.jar --dep third.jar --dep fourth.jar

Tagging results with your branch name

To include the branch name in your NG SAST results, allowing you to distinguish one set of results from another, add the following to your invocation of ShiftLeft:

sl analyze --tag branch=`git symbolic-ref --short HEAD`

If you're working in a GitHub environment (e.g., GitHub Actions), you can also use --tag branch=${{ github.head_ref }} to populate your branch name.

If you don't provide a branch name, but ShiftLeft detects one available in your environment, it will use that name.

Memory

When running code analysis, we recommend using a heap size that includes an additional 20% to ensure sufficient physical memory on your server for other requirements (e.g., Java).

We recommend setting a process environment variable called SHIFTLEFT_JAVA_OPTS (e.g., SHIFTLEFT_JAVA_OPTS="-Xmx10g") that allows for the running of ShiftLeft to set the heap memory required for your particular application.

Source code view

The ShiftLeft Dashboard's findings list can include URLs that, when used, will direct you to the specific source code lines where the vulnerability occurs.

However, to leverage ShiftLeft's source code view with Java applications, you must augment the source code filepaths by passing --vcs-prefix-correction flag to sl analyze (e.g., --vcs-prefix-correction "*=src/main/java"). NG SAST uses the byte code, not the source code, for analysis, so you'll need to provide additional information about your file path structure to ensure that NG SAST generates the links properly.

Troubleshooting

If you have any issues scanning your project, please see our general troubleshooting page, as well as our Java-specific suggestions that follow.

Java requirements

We require Java 1.8 for code analysis. If Java or JRE is unavailable, the ShiftLeft CLI will automatically download and install JRE 1.8 to the $HOME/.shiftleft directory. You can confirm the JRE download in the verbose log:

[TRACE] Looking for prefix:(jre-) suffix:(.tar.gz)
[TRACE] removed 0 files
[TRACE] extracting /home/vsts/.shiftleft/jre172685618
[TRACE] extracting /home/vsts/.shiftleft/jre172685618/lib
[TRACE] extracting /home/vsts/.shiftleft/jre172685618/lib/resources.jar
[TRACE] extracting /home/vsts/.shiftleft/jre172685618/lib/logging.properties

Analysis time outs

If your analysis times out, there are two things to try:

  1. Adjusting the memory allocated to the scan
  2. Tuning your whitelist or blacklist

Adjusting the memory allocated to the scan

First, set the SL_CPG_OPTS environment variable as follows:

export SL_CPG_OPTS="-J-Xms2g -J-Xmx7g"

Set -Xmx to the max RAM available on the workstation running ShiftLeft. For large projects (e.g., those with over 500,000 lines of code), we ask for a minimum of 16 GB RAM.

After you set SL_CPG_OPTS, re-run the analysis and check the verbose log to make sure that the change is taking effect (ignore duplicate -Xms values):

[TRACE] args [/home/vsts/.shiftleft/jre/bin/java -Xms256m -Xms2g -Xmx6g -jar /home/vsts/.shiftleft/libplugin-latest.jar analyze /tmp/shiftleft-638314335/libplugin-state524922089 --]
...
[TRACE] args [/home/vsts/.shiftleft/jre/bin/java -Xms256m -Xms2g -Xmx6g -jar /home/vsts/.shiftleft/libplugin-latest.jar generate /tmp/shiftleft-638314335/libplugin-state524922089 --]

The libplugin analyze and generate steps should have memory-related settings.

At the end of the log, you can also see memory usage information; in the example below, the analysis used 24 GB RAM:

2020-12-21 15:19:19.793 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 18287 / 22905
2020-12-21 15:19:24.795 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 19912 / 22905
2020-12-21 15:19:29.861 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 2190 / 23290
2020-12-21 15:19:34.864 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 3299 / 23290
2020-12-21 15:19:39.866 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 4307 / 23290
2020-12-21 15:19:44.868 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 5508 / 23290
2020-12-21 15:19:49.875 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 6794 / 23290
2020-12-21 15:19:54.877 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 8082 / 23290
2020-12-21 15:19:59.879 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 10082 / 23290
2020-12-21 15:20:04.881 [java2cpg-jvm-monitor] DEBUG Memory used/committed (MB): 11237 / 23290

Tuning your whitelist or blacklist

If setting the SL_CPG_OPTS environment variable isn't fixing the problem, then you can tune the project's whitelist and blacklist.

Building a CPG is a resource-intensive process. ShiftLeft can analyze many frameworks and libraries, but it helps to focus on specific frameworks or namespaces. For example, few are interested in viewing static analysis reports for open source (OSS) libraries; thus, OSS libraries are pre-annotated and blacklisted by default.

However, your app may use libraries that haven't yet been blacklisted. For example, you might be using internal libraries common to your applications. Instead of reporting the same vulnerabilities in these libraries across all of your applications, it may make sense to blacklist these so that your findings list is more focused and actionable for your developers.

You can see if your project will benefit from whitelisting by looking for DEBUG Not replacing in your logs:

2020-12-21 15:17:00.583 [main] DEBUG N2020-12-21 15:17:00.682 [main] DEBUG Not replacing: /tmp/java2cpg_class3799848926187001016/io/netty/util/concurrent/GlobalEventExecutor$1.class

If you see 500+ occurrences of such a message, your project is a good candidate.

To manipulate your whitelist/blacklist, please contact ShiftLeft, making sure to provide:

  • Your org id
  • Your app name
  • Your verbose logs

ShiftLeft will inform you about the amount of time required for this work.

VCS prefix corrections

You must modify the VCS URL that NG SAST sets by default for Java apps. To do this, include the following flag with sl analyze:

--vcs-prefix-correction "*=/src/main/java"

Similarly, for Scala projects, use:

--vcs-prefix-correction "*=/src/main/scala"

NG SAST currently cannot support multi-module repos where the src directory contains multiple separate modules. In such cases, we recommend using the API and applying a transformation script to fix the source code path.