1 # Building HAT
2
3 ----
4 * [Contents](hat-00.md)
5 * Build Babylon and HAT
6 * [Quick Install](hat-01-quick-install.md)
7 * [Building Babylon with jtreg](hat-01-02-building-babylon.md)
8 * [Building HAT with jtreg](hat-01-03-building-hat.md)
9 * [Enabling the NVIDIA CUDA Backend](hat-01-05-building-hat-for-cuda.md)
10 * [Testing Framework](hat-02-testing-framework.md)
11 * [Running Examples](hat-03-examples.md)
12 * [HAT Programming Model](hat-03-programming-model.md)
13 * Interface Mapping
14 * [Interface Mapping Overview](hat-04-01-interface-mapping.md)
15 * [Cascade Interface Mapping](hat-04-02-cascade-interface-mapping.md)
16 * Development
17 * [Project Layout](hat-01-01-project-layout.md)
18 * Implementation Details
19 * [Walkthrough Of Accelerator.compute()](hat-accelerator-compute.md)
20 * [How we minimize buffer transfers](hat-minimizing-buffer-transfers.md)
21 * [Running HAT with Docker on NVIDIA GPUs](hat-07-docker-build-nvidia.md)
22 ---
23
24 # Building HAT with Script
25
26 We initially used `maven` and `cmake` to build HAT. If you feel more comfortable
27 with maven consider [building with maven and cmake](hat-01-03-building-hat-with-maven.md)
28 but it is possible that maven support will be removed if the `Script` approach takes off.
29
30 ## Dependencies
31
32 Before we start to build HAT we will need `cmake` and `jextract` installed.
33
34 You can download `jextract` from [here](https://jdk.java.net/jextract/)
35
36 Use `sudo apt` on Linux or `brew install`.
37
38 ```bash
39 sudo apt install cmake
40
41 ```
42
43 ```bash
44 brew install cmake
45 ```
46
47 You will also need a Babylon JDK built (the one we built [here](hat-01-02-building-babylon.md))
48
49 ## Setting your PATH variable
50
51 To build HAT we will need `JAVA_HOME` to point to our prebuilt babylon jdk
52
53 I suggest you also create a `JEXTRACT_HOME` var to point to the location where you placed JEXTRACT. The jextract EA builds can be found at https://jdk.java.net/jextract/.
54
55 In my case
56 ```
57 export JEXTRACT_HOME=/Users/me/jextract-22
58 ```
59
60 Make sure also that `cmake` in in your PATH
61
62 ## ./env.bash
63
64 Thankfully just sourcing the top level `env.bash` script should then be able to set up your `PATH` for you.
65
66 It should detect the arch type (AARCH64 or X86_46) and
67 select the correct relative parent dir for your BABYLON_JDK and inject that dir in your `PATH`.
68
69 It should also add jextract to your PATH (based on the value you set above for `JEXTRACT_HOME`)
70
71
72 ```bash
73 cd hat
74 export JEXTRACT_HOME=/Users/me/jextract-22
75 . ./env.bash
76 echo ${JAVA_HOME}
77 /Users/me/github/babylon/hat/../build/macosx-aarch64-server-release/jdk
78 echo ${PATH}
79 /Users/me/github/babylon/hat/../build/macosx-aarch64-server-release/jdk/bin:/Users/me/jextract-22/bin:/usr/local/bin:......
80 ```
81
82 ## Building using bld
83
84 To build hat artifacts (hat jar + backends and examples)
85 ```bash
86 java @hat/bld
87 ```
88
89 This places build artifacts in the `build` and `stages` dirs
90
91 ```bash
92 cd hat
93 . ./env.bash
94 java @hat/bld
95 ls build
96 hat-1.0.jar hat-example-heal-1.0.jar libptx_backend.dylib
97 hat-backend-ffi-cuda-1.0.jar hat-example-mandel-1.0.jar libspirv_backend.dylib
98 hat-backend-ffi-mock-1.0.jar hat-example-squares-1.0.jar mock_info
99 hat-backend-ffi-opencl-1.0.jar hat-example-view-1.0.jar opencl_info
100 hat-backend-ffi-ptx-1.0.jar hat-example-violajones-1.0.jar ptx_info
101 hat-backend-ffi-spirv-1.0.jar libmock_backend.dylib spirv_info
102 hat-example-experiments-1.0.jar libopencl_backend.dylib
103 ls stage
104 opencl_jextracted opengl_jextracted
105 ```
106
107 `bld` relies on cmake to build native code for backends, so if cmake finds OpenCL libs/headers, you will see libopencl_backend (.so or .dylib) in the build dir, if cmake finds CUDA you will see libcuda_backend(.so or .dylib)
108
109 We have another script called `sanity` which will check all .md/.java/.cpp/.h for tabs, lines that end with whitespace
110 or files without appropriate licence headers
111
112 This is run using
113
114 ```bash
115 java @hat/sanity
116 ```
117
118 ## Running an example
119
120 To run a HAT example we can run from the artifacts in `build` dir
121
122 ```bash
123 ${JAVA_HOME}/bin/java \
124 --add-modules jdk.incubator.code --enable-preview --enable-native-access=ALL-UNNAMED \
125 --class-path build/hat-core-1.0.jar:build/hat-example-shared-1.0.jar:build/hat-backend-ffi-shared-1.0.jar:build/hat-backend-ffi-opencl-1.0.jar:build/hat-example-mandel-1.0.jar \
126 --add-exports=java.base/jdk.internal=ALL-UNNAMED \
127 -Djava.library.path=build\
128 mandel.Main
129 ```
130
131 The `hat/run.java` script can also be used which simply needs the backend
132 name `ffi-opencl|ffi-java|ffi-cuda|ffi-ptx|ffi-mock` and the package name `mandel`
133
134 ```bash
135 java @hat/run ffi-opencl mandel
136 ```
137
138 If you pass `headless` as the first arg
139
140 ```bash
141 java @hat/run headless ffi-opencl mandel
142 ```
143
144 This sets `-Dheadless=true` and passes '--headless' to the example. Some examples can use this to avoid launching UI.
145
146
147 # More Bld info
148 `hat/Script.java` is an evolving set of static methods and types required (so far.. ;) )
149 to be able to build HAT, hat backends and examples via the `bld` script
150
151 We rely on java's ability to launch java source directly (without needing to javac first)
152
153 * [JEP 458: Launch Multi-File Source-Code Program](https://openjdk.org/jeps/458)
154 * [JEP 330: Launch Single-File Source-Code Programs](https://openjdk.org/jeps/330)
155
156 The `hat/bld.java` script (really java source) can be run like this
157
158 ```bash
159 java --add-modules jdk.incubator.code --enable-preview --source 26 hat/bld.java
160 ```
161
162 In our case the magic is under the `hat`subdir
163
164 We also have a handy `hat/XXXX` which allows us to avoid specifying common args `--enable-preview --source 26` eash time we launch a script
165
166 ```
167 hat
168 ├── hat
169 | ├── Script.java
170 | ├── sanity (the args for sanity.java) "--enable-preview --source 26 sanity"
171 | |-- sanity.java (the script)
172 | ├── run (the args for sanity.java) "--enable-preview --source 26 hatrun"
173 | |-- run.java (the script)
174 | ├── bld (the args for bld.java) "--enable-preview --source 26 bld"
175 | ├── bld.java (the script)
176
177 ```
178
179 For example
180 ```bash
181 java @hat/bld
182 ```
183
184 Is just a shortcut for
185 ```bash
186 java --add-modules jdk.incubator.code --enable-preview --source 26 hat/bld.java
187 ```
188
189 ----
190 ### HAT runtime environment variable.
191
192 During ffi-backend development we added some useful flags to pass to native code to allow us to trace calls, inject logging, control options.
193
194 These should be considered just development flags.
195
196 At runtime the ffi-backends all communicate from the java side to the native side via a 32 bit 'config' int.
197
198 The Java side class [hat.backend.ffi.Config](https://github.com/openjdk/babylon/blob/code-reflection/hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java)
199
200 Is initialized from an environment variable `HAT` at runtime
201
202 So for example when we launch
203 ```bash
204 java @hat/run ffi-opencl heal
205 ```
206
207 We can pass config info to `HAT` via either `-DHAT=xxx` or via the `HAT` ENV variable.
208
209 So for example to get HAT to dump the text form of the kernel code models.
210
211 ```
212 HAT=INFO,SHOW_KERNEL_MODEL java @hat/run ffi-opencl heal
213 ```
214
215 Or to show generated opencl or cuda code
216 ```
217 HAT=INFO,SHOW_CODE java @hat/run ffi-opencl heal
218 ```
219
220 This is particularly useful for selecting PLATFORM + DEVICE if you have multiple GPU devices.
221
222 Here we select DEVICE 0 on PLATFORM 0 (actually the default)
223 ```
224 HAT=INFO,PLATFORM:0,DEVICE:0,SHOW_CODE ...
225 ```
226 or for DEVICE 1 on PLATFORM 1
227 ```
228 HAT=INFO,PLATFORM:1,DEVICE:1,SHOW_CODE ...
229 ```
230
231 No the platform and device id's are 4 bits. This probably works for development, but we will need a more expressive way of capturing this via the accelerator selection.
232
233 This just allows us to test without code changes.
234
235 To keep he java code and the native code in sync. The `main` method at the end of
236 [hat.backend.ffi.Config](https://github.com/openjdk/babylon/blob/code-reflection/hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java)
237 is actually used to create the C99 header for Cuda and OpenCL ffi-backends.
238
239 So whenever we change the Java config class we should run the main method to generate the header. This is not really robust, (but proved way better than trying to remember for all backends) but you need to know, if you add move config bits to the Java side.
240
241 The Main method uses a variant of the code builders used for C99 style code (CUDA/OpenCL) to generate the config.h header.
242
243 This is how we keep the ffi based backends in sync
244
245 Some more useful `config bits`
246
247 ```
248 HAT=PTX java @hat/run ffi-cuda ....
249 ```
250 Sends PTX generated by our prototype PTX generator to the backend (for CUDA) rather than C99 code.
251
252 ```
253 HAT=INFO java @hat/run ....
254 ```
255
256 Will dump all of the `config bits` from the native side.
257
258 At present this yields
259 ```
260 native minimizeCopies 0
261 native trace 0
262 native profile 0
263 native showCode 0
264 native showKernelModel 0
265 native showComputeModel 0
266 native info 1
267 native traceCopies 0
268 native traceSkippedCopies 0
269 native traceEnqueues 0
270 native traceCalls 0
271 native showWhy 0
272 native showState 0
273 native ptx 0
274 native interpret 0
275 ```
276 Generally the flags all represent single bits (except PLATFORM:n and DEVICE:n) and are set using comma separated uppercase+underscore string forms.
277
278
279 So to experiment with `minimizeCopies` and to `showCode` we set `HAT=MINIMIZE_COPIES,SHOW_CODE`