1 # Building HAT
2
3 ----
4
5 * [Contents](hat-00.md)
6 * House Keeping
7 * [Project Layout](hat-01-01-project-layout.md)
8 * [Building Babylon](hat-01-02-building-babylon.md)
9 * [Building HAT](hat-01-03-building-hat.md)
10 * [Enabling the CUDA Backend](hat-01-05-building-hat-for-cuda.md)
11 * Programming Model
12 * [Programming Model](hat-03-programming-model.md)
13 * Interface Mapping
14 * [Interface Mapping Overview](hat-04-01-interface-mapping.md)
15 * [Cascade Interface Mapping](hat-04-02-cascade-interface-mapping.md)
16 * Implementation Detail
17 * [Walkthrough Of Accelerator.compute()](hat-accelerator-compute.md)
18 * [How we minimize buffer transfers](hat-minimizing-buffer-transfers.md)
19
20 ---
21
22 # Building HAT with Script
23
24 We initially used maven and cmake to build hat. If you feel more comfortable
25 with maven consider [building with maven and cmake](hat-01-03-building-hat-with-maven.md)
26 but it is possible that maven support will be removed if the `Script` approach takes off.
27
28 ## Dependencies
29
30 Before we start to build HAT we will need `cmake` and `jextract` installed.
31
32 You can download jextract from [here](https://jdk.java.net/jextract/)
33
34 Use `sudo apt` on Linux or `brew install`.
35
36 ```bash
37 sudo apt install cmake
38
39 ```
40
41 ```bash
42 brew install cmake
43 ```
44
45
46 You will also need a Babylon JDK built (the one we built [here](hat-01-02-building-babylon.md))
47
48
49 ## Setting your PATH variable
50
51 To build HAT we will need `JAVA_HOME` to point to our prebuilt babylon jdk
52
53 I suggest you also create a `JEXTRACT_HOME` var to point to the location where you placed JEXTRACT)
54
55 In my case
56 ```
57 export JEXTRACT_HOME=/Users/me/jextract-22
58 ```
59
60 Make sure also that `cmake` in in your PATH
61
62 ## ./env.bash
63
64 Thankfully just sourcing the top level `env.bash` script should then be able to set up your PATH for you.
65
66 It should detect the arch type (AARCH64 or X86_46) and
67 select the correct relative parent dir for your BABYLON_JDK and inject that dir in your PATH.
68
69 It should also add jextract to your PATH (based on the value you set above for JEXTRACT_HOME)
70
71
72
73 ```bash
74 cd hat
75 export JEXTRACT_HOME=/Users/me/jextract-22
76 . ./env.bash
77 echo ${JAVA_HOME}
78 /Users/me/github/babylon/hat/../build/macosx-aarch64-server-release/jdk
79 echo ${PATH}
80 /Users/me/github/babylon/hat/../build/macosx-aarch64-server-release/jdk/bin:/Users/me/jextract-22/bin:/usr/local/bin:......
81 ```
82
83 ## Building using bld
84
85 To build hat artifacts (hat jar + backends and examples)
86 ```bash
87 java @hat/bld
88 ```
89
90 This places build artifacts in the `build` and `stages` dirs
91
92 ```bash
93 cd hat
94 . ./env.bash
95 java @hat/bld
96 ls build
97 hat-1.0.jar hat-example-heal-1.0.jar libptx_backend.dylib
98 hat-backend-ffi-cuda-1.0.jar hat-example-mandel-1.0.jar libspirv_backend.dylib
99 hat-backend-ffi-mock-1.0.jar hat-example-squares-1.0.jar mock_info
100 hat-backend-ffi-opencl-1.0.jar hat-example-view-1.0.jar opencl_info
101 hat-backend-ffi-ptx-1.0.jar hat-example-violajones-1.0.jar ptx_info
102 hat-backend-ffi-spirv-1.0.jar libmock_backend.dylib spirv_info
103 hat-example-experiments-1.0.jar libopencl_backend.dylib
104 ls stage
105 opencl_jextracted opengl_jextracted
106 ```
107
108 `bld` relies on cmake to build native code for backends, so if cmake finds OpenCL libs/headers, you will see libopencl_backend (.so or .dylib) in the build dir, if cmake finds CUDA you will see libcuda_backend(.so or .dylib)
109
110 We have another script called `sanity` which will check all .md/.java/.cpp/.h for tabs, lines that end with whitespace
111 or files without appropriate licence headers
112
113 This is run using
114
115 ```
116 java @hat/sanity
117 ```
118
119
120 ## Running an example
121
122 To run a HAT example we can run from the artifacts in `build` dir
123
124 ```bash
125 ${JAVA_HOME}/bin/java \
126 --add-modules jdk.incubator.code --enable-preview --enable-native-access=ALL-UNNAMED \
127 --class-path build/core-1.0.jar:build/hat-backend-ffi-shared-1.0.jar:build/hat-backend-ffi-opencl-1.0.jar:build/hat-example-mandel-1.0.jar \
128 --add-exports=java.base/jdk.internal=ALL-UNNAMED \
129 -Djava.library.path=build\
130 mandel.Main
131 ```
132
133 The `hat/run.java` script can also be used which simply needs the backend
134 name `ffi-opencl|ffi-java|ffi-cuda|ffi-ptx|ffi-mock` and the package name `mandel`
135
136 ```bash
137 java @hat/run ffi-opencl mandel
138 ```
139
140 If you pass `headless` as the first arg
141
142 ```bash
143 java @hat/run headless ffi-opencl mandel
144 ```
145
146 This sets `-Dheadless=true` and passes '--headless' to the example. Some examples can use this to avoid launching UI.
147
148
149 # More Bld info
150 `hat/Script.java` is an evolving set of static methods and types required (so far.. ;) )
151 to be able to build HAT, hat backends and examples via the `bld` script
152
153 We rely on java's ability to launch java source directly (without needing to javac first)
154
155 * [JEP 458: Launch Multi-File Source-Code Program](https://openjdk.org/jeps/458)
156 * [JEP 330: Launch Single-File Source-Code Programs](https://openjdk.org/jeps/330)
157
158 The `hat/bld.java` script (really java source) can be run like this
159
160 ```bash
161 java --add-modules jdk.incubator.code --enable-preview --source 26 hat/bld.java
162 ```
163
164 In our case the magic is under the `hat`subdir
165
166 We also have a handy `hat/XXXX` which allows us to avoid specifying common args `--enable-preview --source 26` eash time we launch a script
167
168 ```
169 hat
170 ├── hat
171 | ├── Script.java
172 | ├── sanity (the args for sanity.java) "--enable-preview --source 26 sanity"
173 | |-- sanity.java (the script)
174 | ├── run (the args for sanity.java) "--enable-preview --source 26 hatrun"
175 | |-- run.java (the script)
176 | ├── bld (the args for bld.java) "--enable-preview --source 26 bld"
177 | ├── bld.java (the script)
178
179 ```
180
181 For example
182 ```bash
183 java @hat/bld
184 ```
185
186 Is just a shortcut for
187 ```bash
188 java --add-modules jdk.incubator.code --enable-preview --source 26 hat/bld.java
189 ```
190
191
192 ----
193 ### HAT runtime environment variable.
194
195 During ffi-backend development we added some useful flags to pass to native code to allow us to trace calls, inject logging, control options.
196
197 These should be considered just development flags.
198
199 At runtime the ffi-backends all communicate from the java side to the native side via a 32 bit 'config' int.
200
201 The Java side class [hat.backend.ffi.Config](https://github.com/openjdk/babylon/blob/code-reflection/hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java)
202
203 Is initialized from an environment variable `HAT` at runtime
204
205 So for example when we launch
206 ```bash
207 java @hat/run ffi-opencl heal
208 ```
209
210 We can pass config info to `HAT` via either `-DHAT=xxx` or via the `HAT` ENV variable.
211
212 So for example to get HAT to dump the text form of the kernel code models.
213
214 ```
215 HAT=INFO,SHOW_KERNEL_MODEL java @hat/run ffi-opencl heal
216 ```
217
218 Or to show generated opencl or cuda code
219 ```
220 HAT=INFO,SHOW_CODE java @hat/run ffi-opencl heal
221 ```
222
223 This is particularly useful for selecting PLATFORM + DEVICE if you have multiple GPU devices.
224
225 Here we select DEVICE 0 on PLATFORM 0 (actually the default)
226 ```
227 HAT=INFO,PLATFORM:0,DEVICE:0,SHOW_CODE ...
228 ```
229 or for DEVICE 1 on PLATFORM 1
230 ```
231 HAT=INFO,PLATFORM:1,DEVICE:1,SHOW_CODE ...
232 ```
233
234 No the platform and device id's are 4 bits. This probably works for development, but we will need a more expressive way of capturing this via the accelerator selection.
235
236 This just allows us to test without code changes.
237
238 To keep he java code and the native code in sync. The `main` method at the end of
239 [hat.backend.ffi.Config](https://github.com/openjdk/babylon/blob/code-reflection/hat/backends/ffi/shared/src/main/java/hat/backend/ffi/Config.java)
240 is actually used to create the C99 header for Cuda and OpenCL ffi-backends.
241
242 So whenever we change the Java config class we should run the main method to generate the header. This is not really robust, (but proved way better than trying to remember for all backends) but you need to know, if you add move config bits to the Java side.
243
244 The Main method uses a variant of the code builders used for C99 style code (CUDA/OpenCL) to generate the config.h header.
245
246 This is how we keep the ffi based backends in sync
247
248 Some more useful `config bits`
249
250 ```
251 HAT=PTX java @hat/run ffi-cuda ....
252 ```
253 Sends PTX generated by our prototype PTX generator to the backend (for CUDA) rather than C99 code.
254
255 ```
256 HAT=INFO java @hat/run ....
257 ```
258
259 Will dump all of the `config bits` from the native side.
260
261 At present this yields
262 ```
263 native minimizeCopies 0
264 native trace 0
265 native profile 0
266 native showCode 0
267 native showKernelModel 0
268 native showComputeModel 0
269 native info 1
270 native traceCopies 0
271 native traceSkippedCopies 0
272 native traceEnqueues 0
273 native traceCalls 0
274 native showWhy 0
275 native showState 0
276 native ptx 0
277 native interpret 0
278 ```
279 Generally the flags all represent single bits (except PLATFORM:n and DEVICE:n) and are set using comma separated uppercase+underscore string forms.
280
281
282 So to experiment with `minimizeCopies` and to `showCode` we set `HAT=MINIMIZE_COPIES,SHOW_CODE`