New hat/docs/Build/nvidia-notes.md

1 
25
26 # Building HAT (NVidia Notes)
27 [Back to Index ../](../index.md)
28
29 This guide provides step-by-step instructions to configure Babylon/HAT for CUDA-based GPU acceleration.
30
31 ### Prerequisites
32
33 To run Babylon/HAT with the CUDA backend, ensure the following components are properly installed:
34
35 1. **The NVIDIA GPU Driver:** Download and install the latest appropriate driver for your operating system from the [NVIDIA Drivers page](https://www.nvidia.com/en-us/drivers/)
36 2. **The CUDA Toolkit (SDK):** Download the matching CUDA Toolkit from the [NVIDIA CUDA Downloads page](https://developer.nvidia.com/cuda-downloads).
37
38 The CUDA Toolkit version must be compatible with your installed NVIDIA driver.
39 For example, as of June 2025, the stable NVIDIA driver for Linux is `570.169`, which supports CUDA Toolkit version `12.8`.
40
41 Always verify compatibility before installation to prevent runtime errors:
42 - You can find previous CUDA Toolkit versions on the [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-toolkit-archive)
43 - Review supported CUDA versions and PTX ISA implementations in the [NVIDIA Parallel Thread Execution documentation](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#release-notes).
44
45
46 ### Troubleshooting: Unsupported CUDA/PTX Versions
47
48 If Babylon/HAT runs with an incompatible CUDA/PTX version, you may encounter an error similar to:
49
50 ```bash
51 cuModuleLoadDataEx CUDA error = 222 CUDA_ERROR_UNSUPPORTED_PTX_VERSION
52 /<path/to/babylon>/hat/backends/ffi/cuda/src/main/native/cpp/cuda_backend.cpp line 220
53 ```
54
55 If this is your case, you can:
56 - **update your GPU driver**.
57 - **or downgrade your CUDA version.**
58
59 ### Building HAT for the CUDA backend
60
61 If the NVIDIA driver and the CUDA Toolkit SDK are installed, the HAT build process will automatically compile
62 all sources to dispatch with the CUDA backend.
63
64 ```bash
65 mvn clean package
66 ```
67
68 ### Run HAT examples with the CUDA Backend
69 You can enable the CUDA backend by using the `ffi-cuda` option from HAT.
70 For example, to run the Matrix Multiplication example:
71
72 ```bash
73 java @.ffi-cuda-example matmul.Main --kernel=1D
74 ```
75
76 ### CUDA source compiler selection (NVRTC, NVCC)
77
78 The CUDA backend compiles generated CUDA source with `nvrtc` by default. To
79 compile with `NVCC` instead, set `HAT=CUDA_NVCC`:
80
81 ```bash
82 HAT=CUDA_NVCC java @.ffi-cuda-example matmul.Main --kernel=1D
83 ```
84
85 There are two flags to select the compiler:
86
87 - **`HAT=CUDA_NVRTC`:** compile in-process with NVRTC. SIMT kernels are emitted
88 as PTX, and Tile kernels as cuda_tile IR. This is the default option.
89 - **`HAT=CUDA_NVCC`:** compile with the `nvcc` executable. SIMT kernels are
90 emitted as PTX, and Tile kernels as cubin.
91
92 SIMT PTX and nvcc Tile cubin need no extra setup. Loading Tile IR in-process
93 registers signal handlers. Those handlers process the signals first and resend
94 them with a polluted signal state, which breaks JVM signal handling. NVRTC emits
95 Tile IR only, so `libjsig` is required. Preload JDK `libjsig` so the JVM signal
96 handlers process each signal first and then dispatch to those handlers via
97 chained handlers:
98
99 ```bash
100 LD_PRELOAD=$JAVA_HOME/lib/libjsig.so HAT=CUDA_NVRTC \
101 java @.ffi-cuda-test hat.test.TestTileAPI
102 ```
103
104 **IMPORTANT:** If the `libjsig` is not preloaded, HAT warns once and compiles Tile kernels with
105 `nvcc` instead.
106
107 When NVRTC is selected, HAT loads `libnvrtc.so` from the CUDA Toolkit library
108 directory recorded at build time, then from the default library search path. To
109 use a specific library, set `HAT_CUDA_NVRTC_LIBRARY` to its path or
110 loader-visible name. If that library cannot be loaded, HAT exits:
111
112 ```bash
113 HAT=CUDA_NVRTC HAT_CUDA_NVRTC_LIBRARY=/path/to/libnvrtc.so java \
114 @.ffi-cuda-example matmul.Main --kernel=1D
115 ```