MopFuzzer

MopFuzzer is a tool for automated testing of the JDK, utilizing various optimization evoking mutators and profile data-based guidance to rigorously test JVM JIT compilers by maximizing optimization interactions. Our artifact includes the source code of MopFuzzer, the mutation seed files required during the execution of MopFuzzer, and a README file describing the installation process and the bugs discovered by the tool.

The Designed Optimization Evoking Mutators

In this work, we designed 13 Optimization Evoking Mutators, which aim to trigger primarily the following optimization behaviors respectively: LoopUnrolling, LockElimination, LockCoarsening, Inlining, DeReflection, LoopPeeling, LoopUnswitching, Deoptimization, AutoboxElimination, RedundantStoreElimination, AlgebraicSimplification, EscapeAnalysis, DeadCodeElimination.

The remaining mutators can be found at the Supplementary Material of the paper.

Usage of MopFuzzer

Step 1: Prerequisites

MopFuzzer needs the debug build of JVM, so users should download the source code of JVM and set the debug flag. Here we take the OpenJDK (mainline) as an example.

# git clone https://github.com/openjdk/jdk.git
# cd jdk
# bash configure --enable-debug
# make images JOBS=40

Step 2: Run the tool

To use MopFuzzer, users should specify the target test case, the path of the debug build of JVM (at least one), We will check whether the outputs of these JDKs are consistant (if perform differential testing). The following is Parameters of running MopFuzzer.

usage: help [-jdk <arg>] [-project_path <arg>] [-target_case <arg>] [-line_number <arg>] [-is_use_jtreg <arg>] [-max_iter <arg>] [-?] [-enable_profile_guide <arg>]

 -?,--help                     Print this help message
 --project_path <arg>          source code root path. e.g., /home/user/benchmark/jtreg17. Necessary.
 --target_case <arg>           target java file. e.g., a.b.c denotes a/b/c.java. Necessary.
 --use_gc_options              use gc options? e.g., true. Default is false.
 --use_jit_options             use JIT options? e.g., true. Default is false.
 --enable_profile_guide <arg>  enable profile guidance? 
                               Recommend false for faster testing. Default is
                               false.
 --is_use_jtreg <arg>          use Jtreg? e.g., true. Default is false
 --jdk <arg>                   the jdk directory, path to bin/. At least one JDK. Necessary
                               We will check whether the outputs of these JDKs are consistent. 
 --line_number <arg>           line number of target file to be mutated. e.g., 10. Default is null.
 --max_iter <arg>              max iteration times.. e.g., 100. Default is 50

Example Command:

# differential testing of two target jdks
path/to/java17/bin/java -jar MopFuzzer.jar --project_path benchmarks/JavaFuzzer/tests/  --target_case Test0001 --jdk path/to/targetJDK1/bin/,path/to/targetJDK2/bin/ --enable_profile_guide true

# Testing a single jdk using jtreg seeds
path/to/java17/bin/java -jar MopFuzzer.jar --project_path benchmarks/jtreg17/ --target_case compiler.codegen.TestBooleanVect --jdk path/to/targetJDK/bin/ --enable_profile_guide true

Show Cases of Detected Bugs

synchronized void write(char[] data, int offset, int length) throws IOException {
    while (--length >= 0) {
        synchronized (new Test8267042()) {
            int var1 = 0;
            while (var1 < 100000) {
                if (var1 == 50000) {
                    getZeroOnStack(offset);
                }
                var1++;
            }
        }
        // original code
        // getZeroOnStack(offset);
        write(data[offset++]);
    }
}

The cause of this bug lies in improper operations during the Just-In-Time (JIT) compiler's escape analysis and On-Stack Replacement (OSR) transformation.

This bug involves optimization processes including escape analysis and lock optimization. Specifically, JVM incorrectly identifies certain objects as non-escaping during escape analysis, leading to incorrect handling of these objects' lock states in the On-Stack Replacement (OSR) transformation, which triggers a crash.

public class Test {

    public static final int CHUNK = 10000;
    public static ArrayList<Object> arr = new ArrayList<>();

    public static void main(String[] args) {
        while (true) {
            synchronized (Test.class) {
                synchronized (new Test()) {
                    synchronized (Test.class) {
                        arr.add(new byte[CHUNK]);
                    }
                }
            }
        }
    }
}

The cause of this bug is improper handling of nested synchronized locks during the deoptimization process. The bug involves optimization processes related to deoptimization and the maintenance of synchronized locks.

JVM attempts to eliminate nested locks and locks on non-escaping allocations in the C2 compiler optimization behavior, but during deoptimization, it handles the unlocking and relocking sequence improperly (starting from the inner frame instead of the outer frame), leading to a crash during the deoptimization process.

public class Test {
    public static final int N = 400;

    public static long instanceCount = 3L;
    public static int[] iArrFld = new int[N];

    public void mainTest(String[] strArr1) {
        for (int i0 = 0; i0 < N; i0++) {
            Test.iArrFld[i0] = -1;
        }

        for (int i14 = 343; i14 > 21; --i14) {
            Test.instanceCount += (i14 ^ Test.instanceCount);
        }

        for (int i1 = 0; i1 < N; ++i1) {
            for (int var6 = 0; var6 < 10000; ++var6) {
                if (var6 == 4) {
                }
            }

            int[] tmp = new int[N];
            for (int i2 = 0; i2 < N; i2++) {
                tmp[i2] = 9;
            }
            Test.iArrFld = tmp;

            for (int i18 = 3; i18 < 63; i18++) {
                Test.iArrFld[i18 - 1] <<= (int) Test.instanceCount;
            }
        }

        long sum = 0;
        for (int i : Test.iArrFld) {
            sum += i;
        }
        System.out.println("sum: " + sum);
    }

    public static void main(String[] strArr) {
        Test _instance = new Test();
        _instance.mainTest(strArr);
    }
}

The cause of this bug is the improper handling of shift operation codes by the JVM during automatic SIMD (Single Instruction, Multiple Data) optimization, leading to behavior inconsistent with Java semantics. This bug involves the optimization process of automatic SIMD optimization. Specifically, the JVM improperly handled the VSHL (Vector Shift Left) operation code during the execution of automatic SIMD optimization.

Profile Flags

JVM Flags and the Corresponding Regular Expression Rules for Profiling Optimization Behaviors

Confirmed Bugs

Since unconfirmed bugs cannot be shown in Java Bug System(JBS), we only show the bugs that are confirmed by developers.

OpenJDK Bugs

OpenJ9 Bugs