< prev index next >

doc/testing.md

Print this page

        

@@ -10,14 +10,15 @@
 The main target `test` uses the jdk-image as the tested product. There is
 also an alternate target `exploded-test` that uses the exploded image
 instead. Not all tests will run successfully on the exploded image, but using
 this target can greatly improve rebuild times for certain workflows.
 
-Previously, `make test` was used to invoke an old system for running tests, and
+Previously, `make test` was used invoke an old system for running test, and
 `make run-test` was used for the new test framework. For backward compatibility
 with scripts and muscle memory, `run-test` (and variants like
-`exploded-run-test` or `run-test-tier1`) are kept as aliases.
+`exploded-run-test` or `run-test-tier1`) are kept as aliases. The old system
+can still be accessed for some time using `cd test && make`.
 
 Some example command-lines:
 
     $ make test-tier1
     $ make test-jdk_lang JTREG="JOBS=8"

@@ -37,12 +38,12 @@
 containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME`
 environment variable to point to the JTReg home before running `configure`.)
 
 To be able to run microbenchmarks, `configure` needs to know where to find
 the JMH dependency. Use `--with-jmh=<path to JMH jars>` to point to a directory
-containing the core JMH and transitive dependencies. The recommended dependencies
-can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which
+containing the core JMH and transitive dependencies. The recommended dependencies 
+can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which 
 `--with-jmh=build/jmh/jars` should work.
 
 ## Test selection
 
 All functionality is available using the `test` make target. In this use case,

@@ -199,65 +200,20 @@
 proper quoting of command line arguments through.)
 
 As far as possible, the names of the keywords have been standardized between
 test suites.
 
-### General keywords (TEST_OPTS)
-
-Some keywords are valid across different test suites. If you want to run
-tests from multiple test suites, or just don't want to care which test suite specific
-control variable to use, then you can use the general TEST_OPTS control variable.
-
-There are also some keywords that applies globally to the test runner system,
-not to any specific test suites. These are also available as TEST_OPTS keywords.
-
-#### JOBS
-
-Currently only applies to JTReg.
-
-#### TIMEOUT_FACTOR
-
-Currently only applies to JTReg.
-
-#### VM_OPTIONS
-
-Applies to JTReg, GTest and Micro.
-
-#### JAVA_OPTIONS
-
-Applies to JTReg, GTest and Micro.
-
-#### AOT_MODULES
-
-Applies to JTReg and GTest.
-
-#### JCOV
-
-This keywords applies globally to the test runner system. If set to `true`, it
-enables JCov coverage reporting for all tests run. To be useful, the JDK under
-test must be run with a JDK built with JCov instrumentation (`configure
---with-jcov=<path to directory containing lib/jcov.jar>`, `make jcov-image`).
-
-The simplest way to run tests with JCov coverage report is to use the special
-target `jcov-test` instead of `test`, e.g. `make jcov-test TEST=jdk_lang`. This
-will make sure the JCov image is built, and that JCov reporting is enabled.
-
-The JCov report is stored in `build/$BUILD/test-results/jcov-output`.
-
-Please note that running with JCov reporting can be very memory intensive.
-
 ### JTReg keywords
 
 #### JOBS
 The test concurrency (`-concurrency`).
 
 Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to
-JOBS, except for Hotspot, where the default is *number of CPU cores/2* (for
-sparc, if more than 16 cpus, then *number of CPU cores/5*, otherwise *number of
-CPU cores/4*), but never more than *memory size in GB/2*.
+JOBS, except for Hotspot, where the default is *number of CPU cores/2*, but
+never more than 12.
 
-#### TIMEOUT_FACTOR
+#### TIMEOUT
 The timeout factor (`-timeoutFactor`).
 
 Defaults to 4.
 
 #### TEST_MODE

@@ -287,28 +243,10 @@
 Limit memory consumption for JTReg test framework and VM under test. Set to 0
 to disable the limits.
 
 Defaults to 512m, except for hotspot, where it defaults to 0 (no limit).
 
-#### KEYWORDS
-
-JTReg kewords sent to JTReg using `-k`. Please be careful in making sure that
-spaces and special characters (like `!`) are properly quoted. To avoid some
-issues, the special value `%20` can be used instead of space.
-
-#### EXTRA_PROBLEM_LISTS
-
-Use additional problem lists file or files, in addition to the default
-ProblemList.txt located at the JTReg test roots.
-
-If multiple file names are specified, they should be separated by space (or, to
-help avoid quoting issues, the special value `%20`).
-
-The file names should be either absolute, or relative to the JTReg test root of
-the tests to be run.
-
-
 #### OPTIONS
 Additional options to the JTReg test framework.
 
 Use `JTREG="OPTIONS=--help all"` to see all available JTReg options.
 

@@ -316,16 +254,10 @@
 Additional Java options to JTReg (`-javaoption`).
 
 #### VM_OPTIONS
 Additional VM options to JTReg (`-vmoption`).
 
-#### AOT_MODULES
-
-Generate AOT modules before testing for the specified module, or set of
-modules. If multiple modules are specified, they should be separated by space
-(or, to help avoid quoting issues, the special value `%20`).
-
 ### Gtest keywords
 
 #### REPEAT
 The number of times to repeat the tests (`--gtest_repeat`).
 

@@ -336,16 +268,10 @@
 #### OPTIONS
 Additional options to the Gtest test framework.
 
 Use `GTEST="OPTIONS=--help"` to see all available Gtest options.
 
-#### AOT_MODULES
-
-Generate AOT modules before testing for the specified module, or set of
-modules. If multiple modules are specified, they should be separated by space
-(or, to help avoid quoting issues, the special value `%20`).
-
 ### Microbenchmark keywords
 
 #### FORK
 Override the number of benchmark forks to spawn. Same as specifying `-f <num>`.
 

@@ -371,36 +297,10 @@
 Additional VM arguments to provide to forked off VMs. Same as `-jvmArgs <args>`
 
 #### OPTIONS
 Additional arguments to send to JMH.
 
-## Notes for Specific Tests
-
-### Docker Tests
-
-Docker tests with default parameters may fail on systems with glibc versions not
-compatible with the one used in the default docker image (e.g., Oracle Linux 7.6 for x86).
-For example, they pass on Ubuntu 16.04 but fail on Ubuntu 18.04 if run like this on x86:
-
-    $ make test TEST="jtreg:test/hotspot/jtreg/containers/docker"
-
-To run these tests correctly, additional parameters for the correct docker image are
-required on Ubuntu 18.04 by using `JAVA_OPTIONS`.
-
-    $ make test TEST="jtreg:test/hotspot/jtreg/containers/docker" JTREG="JAVA_OPTIONS=-Djdk.test.docker.image.name=ubuntu -Djdk.test.docker.image.version=latest"
-
-### Non-US locale
-
-If your locale is non-US, some tests are likely to fail. To work around this you can
-set the locale to US. On Unix platforms simply setting `LANG="en_US"` in the
-environment before running tests should work. On Windows, setting
-`JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US"` helps for most, but not all test cases.
-For example:
-
-    $ export LANG="en_US" && make test TEST=...
-    $ make test JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US" TEST=...
-
 ---
 # Override some definitions in the global css file that are not optimal for
 # this document.
 header-includes:
  - '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>'
< prev index next >