Compiler Fuzzer Tests

fuzzer-tests-* bundle introduces applications/fuzzer suite of jtregs. This suite contains the bundles with pregenerated JavaFuzzer tests, plus jtreg wrappers for them. These tests are verified to complete in reasonable amount of time in both compiled and interpreted modes: tens of seconds and tens of minutes, respectively. The tests verify workload output with golden output generated by interpreter during bundle creation.


After mounting, it can be used as the regular OpenJDK suite. Note that the bundle reuses infrastructure code from the JDK tree, so it has to be mounted in the OpenJDK tree. Choose the bundle relevant to the JDK version. If there is no exact JDK version bundle, try the adjacent versions.


$ cd [openjdk-root]
$ mkdir test/hotspot/jtreg/applications/fuzzer/
$ mount fuzzer-tests-*.sfs test/hotspot/jtreg/applications/fuzzer/
$ make run-test TEST=applications/fuzzer/ TEST_OPTS="JAVA_OPTIONS=..."

JDK 8 is more complicated, because it needs the test suite alias in TEST.groups:


$ cd [openjdk-root]
$ mkdir hotspot/test/fuzzer/
$ mount fuzzer-tests-*.sfs hotspot/test/fuzzer/
$ echo "hotspot_fuzzer = fuzzer/" >> hotspot/test/TEST.groups
$ make test TEST=hotspot_fuzzer

Allocate a fair share of time to run these tests. It is a good idea to run them on heavily parallel machine if you want "real" times to stay reasonable. On TR 3970X, 100K tests in default configuration complete in:


real   70m15.056s
user 2267m33.454s
sys   174m22.460s

/fuzzer/

File Name  ↓ File Size  ↓ Date  ↓ 
--
375.3 MiB2021-Jun-25 07:21
375.3 MiB2021-Jun-25 07:21
375.3 MiB2021-Jun-25 07:21
37.5 MiB2021-Jun-24 18:28
37.5 MiB2021-Jun-24 18:28
37.5 MiB2021-Jun-24 18:28
3.8 MiB2021-Jun-24 16:15
3.8 MiB2021-Jun-24 16:15
3.8 MiB2021-Jun-24 16:15
666 B2021-Oct-24 17:08