< prev index next >

doc/testing.md

Print this page

        

*** 10,23 **** The main target `test` uses the jdk-image as the tested product. There is also an alternate target `exploded-test` that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows. ! Previously, `make test` was used to invoke an old system for running tests, and `make run-test` was used for the new test framework. For backward compatibility with scripts and muscle memory, `run-test` (and variants like ! `exploded-run-test` or `run-test-tier1`) are kept as aliases. Some example command-lines: $ make test-tier1 $ make test-jdk_lang JTREG="JOBS=8" --- 10,24 ---- The main target `test` uses the jdk-image as the tested product. There is also an alternate target `exploded-test` that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows. ! Previously, `make test` was used invoke an old system for running test, and `make run-test` was used for the new test framework. For backward compatibility with scripts and muscle memory, `run-test` (and variants like ! `exploded-run-test` or `run-test-tier1`) are kept as aliases. The old system ! can still be accessed for some time using `cd test && make`. Some example command-lines: $ make test-tier1 $ make test-jdk_lang JTREG="JOBS=8"
*** 37,48 **** containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME` environment variable to point to the JTReg home before running `configure`.) To be able to run microbenchmarks, `configure` needs to know where to find the JMH dependency. Use `--with-jmh=<path to JMH jars>` to point to a directory ! containing the core JMH and transitive dependencies. The recommended dependencies ! can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which `--with-jmh=build/jmh/jars` should work. ## Test selection All functionality is available using the `test` make target. In this use case, --- 38,49 ---- containing `lib/jtreg.jar` etc. (An alternative is to set the `JT_HOME` environment variable to point to the JTReg home before running `configure`.) To be able to run microbenchmarks, `configure` needs to know where to find the JMH dependency. Use `--with-jmh=<path to JMH jars>` to point to a directory ! containing the core JMH and transitive dependencies. The recommended dependencies ! can be retrieved by running `sh make/devkit/createJMHBundle.sh`, after which `--with-jmh=build/jmh/jars` should work. ## Test selection All functionality is available using the `test` make target. In this use case,
*** 199,263 **** proper quoting of command line arguments through.) As far as possible, the names of the keywords have been standardized between test suites. - ### General keywords (TEST_OPTS) - - Some keywords are valid across different test suites. If you want to run - tests from multiple test suites, or just don't want to care which test suite specific - control variable to use, then you can use the general TEST_OPTS control variable. - - There are also some keywords that applies globally to the test runner system, - not to any specific test suites. These are also available as TEST_OPTS keywords. - - #### JOBS - - Currently only applies to JTReg. - - #### TIMEOUT_FACTOR - - Currently only applies to JTReg. - - #### VM_OPTIONS - - Applies to JTReg, GTest and Micro. - - #### JAVA_OPTIONS - - Applies to JTReg, GTest and Micro. - - #### AOT_MODULES - - Applies to JTReg and GTest. - - #### JCOV - - This keywords applies globally to the test runner system. If set to `true`, it - enables JCov coverage reporting for all tests run. To be useful, the JDK under - test must be run with a JDK built with JCov instrumentation (`configure - --with-jcov=<path to directory containing lib/jcov.jar>`, `make jcov-image`). - - The simplest way to run tests with JCov coverage report is to use the special - target `jcov-test` instead of `test`, e.g. `make jcov-test TEST=jdk_lang`. This - will make sure the JCov image is built, and that JCov reporting is enabled. - - The JCov report is stored in `build/$BUILD/test-results/jcov-output`. - - Please note that running with JCov reporting can be very memory intensive. - ### JTReg keywords #### JOBS The test concurrency (`-concurrency`). Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to ! JOBS, except for Hotspot, where the default is *number of CPU cores/2* (for ! sparc, if more than 16 cpus, then *number of CPU cores/5*, otherwise *number of ! CPU cores/4*), but never more than *memory size in GB/2*. ! #### TIMEOUT_FACTOR The timeout factor (`-timeoutFactor`). Defaults to 4. #### TEST_MODE --- 200,219 ---- proper quoting of command line arguments through.) As far as possible, the names of the keywords have been standardized between test suites. ### JTReg keywords #### JOBS The test concurrency (`-concurrency`). Defaults to TEST_JOBS (if set by `--with-test-jobs=`), otherwise it defaults to ! JOBS, except for Hotspot, where the default is *number of CPU cores/2*, but ! never more than 12. ! #### TIMEOUT The timeout factor (`-timeoutFactor`). Defaults to 4. #### TEST_MODE
*** 287,314 **** Limit memory consumption for JTReg test framework and VM under test. Set to 0 to disable the limits. Defaults to 512m, except for hotspot, where it defaults to 0 (no limit). - #### KEYWORDS - - JTReg kewords sent to JTReg using `-k`. Please be careful in making sure that - spaces and special characters (like `!`) are properly quoted. To avoid some - issues, the special value `%20` can be used instead of space. - - #### EXTRA_PROBLEM_LISTS - - Use additional problem lists file or files, in addition to the default - ProblemList.txt located at the JTReg test roots. - - If multiple file names are specified, they should be separated by space (or, to - help avoid quoting issues, the special value `%20`). - - The file names should be either absolute, or relative to the JTReg test root of - the tests to be run. - - #### OPTIONS Additional options to the JTReg test framework. Use `JTREG="OPTIONS=--help all"` to see all available JTReg options. --- 243,252 ----
*** 316,331 **** Additional Java options to JTReg (`-javaoption`). #### VM_OPTIONS Additional VM options to JTReg (`-vmoption`). - #### AOT_MODULES - - Generate AOT modules before testing for the specified module, or set of - modules. If multiple modules are specified, they should be separated by space - (or, to help avoid quoting issues, the special value `%20`). - ### Gtest keywords #### REPEAT The number of times to repeat the tests (`--gtest_repeat`). --- 254,263 ----
*** 336,351 **** #### OPTIONS Additional options to the Gtest test framework. Use `GTEST="OPTIONS=--help"` to see all available Gtest options. - #### AOT_MODULES - - Generate AOT modules before testing for the specified module, or set of - modules. If multiple modules are specified, they should be separated by space - (or, to help avoid quoting issues, the special value `%20`). - ### Microbenchmark keywords #### FORK Override the number of benchmark forks to spawn. Same as specifying `-f <num>`. --- 268,277 ----
*** 371,406 **** Additional VM arguments to provide to forked off VMs. Same as `-jvmArgs <args>` #### OPTIONS Additional arguments to send to JMH. - ## Notes for Specific Tests - - ### Docker Tests - - Docker tests with default parameters may fail on systems with glibc versions not - compatible with the one used in the default docker image (e.g., Oracle Linux 7.6 for x86). - For example, they pass on Ubuntu 16.04 but fail on Ubuntu 18.04 if run like this on x86: - - $ make test TEST="jtreg:test/hotspot/jtreg/containers/docker" - - To run these tests correctly, additional parameters for the correct docker image are - required on Ubuntu 18.04 by using `JAVA_OPTIONS`. - - $ make test TEST="jtreg:test/hotspot/jtreg/containers/docker" JTREG="JAVA_OPTIONS=-Djdk.test.docker.image.name=ubuntu -Djdk.test.docker.image.version=latest" - - ### Non-US locale - - If your locale is non-US, some tests are likely to fail. To work around this you can - set the locale to US. On Unix platforms simply setting `LANG="en_US"` in the - environment before running tests should work. On Windows, setting - `JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US"` helps for most, but not all test cases. - For example: - - $ export LANG="en_US" && make test TEST=... - $ make test JTREG="VM_OPTIONS=-Duser.language=en -Duser.country=US" TEST=... - --- # Override some definitions in the global css file that are not optimal for # this document. header-includes: - '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>' --- 297,306 ----
< prev index next >