1 # HAT's Tile Programming Model implementation
2 [Back to Index ../](../index.md)
3
4 ## Introduction
5
6 A tile model is a high-level programming abstraction that facilitates expressing
7 array-oriented algorithms across hardware accelerators, including GPUs.
8
9 In contrast to more conventional GPU programming models, such as OpenCL and
10 CUDA, in which programmers often organize work explicitly around threads and
11 thread blocks under a SIMT (Single Instruction Multiple Thread) model, a
12 tile-based approach describes operations on block of data (tiles). Then, an
13 underlying compiler is free to map logical blocks of data to hardware resources
14 more efficiently.
15
16 HAT provides an initial implementation of a Tile Programming Model based on
17 the [Triton programming language](https://triton-lang.org/main/index.html) and
18 inspired by the
19 NVIDIA [Tile programming model](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/writing-tile-kernels.html).
20
21 While this prototype has been inspired by the aforementioned programmin models,
22 the HAT Tile programming model implementation does not necessarily follow the
23 same parallel constructs. Instead, HAT is free to evolve to incorporate those
24 ideas and make them available for Java programs, not only for GPUs, but also for
25 other hardware.
26
27 ## Disclaimer
28
29 HAT's tile programming model implementation is a work in progress that
30 demonstrates how tile programming can be integrated into Java through code
31 reflection.
32
33 Currently, HAT only provides an implementation for CUDA, and maps the tile
34 programs expressed with Java to CUDA Tile C++. To be able to run Tile programs
35 on NVIDIA hardware, developers must have a GPU >= Ampere (Blackwell
36 recommended), and use the NVIDIA driver or later. See
37 full [list of requirements below](#requirements).
38
39 The tile implementation in HAT does not include an OpenCL, or CPU
40 implementations. However, it is in our plans to extend support with both models
41 (by mapping to OpenCL devices, and providing a Java implementation).
42
43 ## Requirements
44
45 - NVIDIA GPU Graphics Card, Ampere or later (Blackwell recommended).
46 - NVIDIA Driver `610.57.04` or later.
47 - CUDA SDK: `13.3` or later.
48 - [Babylon build for Java](../Build/babylon.md).
49
50 ## Installation
51
52 If dependencies are satisfied, the build is identical to upstream `HAT`.
53
54 ```bash
55 mvn clean package
56 ```
57
58 ## Example and Execution
59
60 ```java
61 // Tile kernel to be offloaded and accelerator on the GPU
62 @Reflect
63 public static void vectorAddTile(TensorF32 inputA,
64 TensorF32 inputB,
65 TensorF32 output,
66 final int tileSize) {
67
68 // Access the thread-block id
69 final var pid = TileContext.BIDX();
70
71 // Load the tiles from the input tensors
72 var tileA = TileContext.load(inputA, pid, tileSize);
73 var tileB = TileContext.load(inputB, pid, tileSize);
74
75 // Perform tile addition
76 var result = Tile.add(tileA, tileB);
77
78 // Store the result into the output tensor
79 TileContext.store(output, pid, result);
80 }
81
82 // Method dispatch to invoke a tile kernel
83 @Reflect
84 public static void vectorAddTile(ComputeContext computeContext,
85 TensorF32 inputA,
86 TensorF32 inputB,
87 TensorF32 output,
88 final int tileSize) {
89 // invoke to dispatch tile method
90 computeContext.dispatchTile(
91 NDRange.of1D(inputA.m(), tileSize), // 1D-Range Tile
92 () -> vectorAddTile(inputA, inputB, output, tileSize)); // Invoke the Tile Kernel
93 }
94
95 public void run() {
96 var accelerator = new Accelerator(MethodHandles.lookup(), Backend.FIRST);
97
98 final int size = Math.powExact(2, 16);
99 final int tileSize = 64;
100
101 TensorF32 inputA = TensorF32.create(accelerator, size);
102 TensorF32 inputB = TensorF32.create(accelerator, size);
103 TensorF32 result = TensorF32.create(accelerator, size);
104
105 // Fill data
106 Random r = new Random(19);
107 for (int i = 0; i < size; i++) {
108 inputA.array(i, r.nextFloat());
109 inputB.array(i, r.nextFloat());
110 }
111
112 accelerator.compute((@Reflect Compute) computeContext ->
113 vectorAddTile(computeContext, inputA, inputB, result, tileSize));
114 }
115 ```
116
117 Run vector addition:
118
119 ```bash
120 java @.ffi-cuda-test hat.test.TestTileAPI#test_hat_tile_01
121 ```
122
123
124 ## Limitations
125
126 - Current implementation in HAT implements a few Tile operations (`mma`, `add`,
127 `sub`, `min`, etc.). More operations are planned.
128 - Current implementations only maps to CUDA Tile C++. Future versions will
129 extend with Java and OpenCL implementations.
130