1
2 # Compute Analysis or Runtime tracing
3
4 ----
5
6 * [Contents](hat-00.md)
7 * House Keeping
8 * [Project Layout](hat-01-01-project-layout.md)
9 * [Building Babylon](hat-01-02-building-babylon.md)
10 * [Maven and CMake](hat-01-03-maven-cmake.md)
11 * Programming Model
12 * [Programming Model](hat-03-programming-model.md)
13 * Interface Mapping
14 * [Interface Mapping Overview](hat-04-01-interface-mapping.md)
15 * [Cascade Interface Mapping](hat-04-02-cascade-interface-mapping.md)
16 * Implementation Detail
17 * [Walkthrough Of Accelerator.compute()](hat-accelerator-compute.md)
18
19 ----
20
21 # Compute Analysis or Runtime tracing
22
23 HAT does not dictate how a backend chooses to optimize execution, but does
24 provide the tools (Babylon's Code Models) and some helpers which the Backend is encouraged
25 use.
26
27 The ComputeContext contains all the information that the backend needs, but does not
28 include any 'policy' for minimizing data movements.
29
30 Our assumption is that backend can use various tools to deduce the most efficient execution strategy.
31
32 ## Some possible strategies..
33
34 ### Copy data every time 'just in case' (JIC execution ;) )
35 Just naiively execute the code as described in Compute graph. So the backend will copy each buffer to the device, execute the kernel and copy the data back again.
36
37 ### Use kernel knowledge to minimise data movement
38 Execute the code described in the Compute Graph, but use knowledge extracted from kernel models
39 to only copy to device buffers that the kernel is going to read, and only copy back from the device
40 buffers that the kernel has written to.
41
42 ### Use Compute knowledge and kernel knowledge to further minimise data movement
43 Use knowledge extracted from the compute reachable graph and the kernel
44 graphs to determine whether Java has mutated buffers between kernel dispatches
45 and only copy data to the device that we know the Java code has mutated.
46
47 This last strategy is ideal
48
49 We can achieve this using static analysis of the compute and kernel models or by being
50 involved in the execution process at runtime.
51
52 #### Static analysis
53
54 #### Runtime Tracking
55
56 * Dynamical
57 1. We 'close over' the call/dispatch graph from the entrypoint to all kernels and collect the kernels reachable from the entrypoint and all methods reachable from methods reachable by kernels.
58 2. We essentially end up with a graph of codemodels 'rooted' at the entrypoint
59 3. For each kernel we also determine how the kernel accesses it's 'MemorySegment` parameters, for each MemorySegment parameters we keep a side table of whther the kernel reads or writes to the segment. We keep this infomation in a side map.
60
61 This resulting 'ComputeClosure' (tree of codemodels and relevant side tables) is made available to the accelerator to coordinate execution.
62
63 Note that our very simple Compute::compute method neither expresses the movement of the MemorySegment to a device, or the retrieval of the data from a device when the kernel has executed.
64
65 Our assumption is that given the ComputeClosure we can deduce such movements.
66
67 There are many ways to achieve this. One way would be by static analysis.
68
69 Given the Compute::compute entrypoint it is easy to determine that we are always (no conditional or loops) passing (making available
70 might be a better term) a memory segment to a kernel (Compute::kernel) and this kernel only mutates the `MemorySegment`.
71
72 So from simple static analysis we could choose to inject one or more calls into the model representing the need for the accelerator to move data to the devices and/ord back from the device, after the kernel dispatch.
73
74 This modified model, would look like we had presented it with this code.
75
76 ```java
77 void compute(Accelerator accelerator, MemorySegment memorySegment, int len) {
78 Accelerator.Range range = accelerator.range(len);
79 accelerator.run(Compute::kernel, range, memorySegment);
80 accelerator.injectedCopyFromDevice(memorySegment);
81 }
82 ```
83
84 Note the ```injectedCopyFromDevice()``` call.
85
86 Because the kernel does not read the `MemorySegment` we only need inject the code to request a move back from the device.
87
88 To do this requires HAT to analyse the kernel(s) and inject appropriate code into
89 the Compute::compute method to inform the vendor backend when it should perform such moves.
90
91 Another strategy would be to not rely on static analysis but to inject code to trace 'actual' mutations of the MemorySegments and use these flags to guard against unnecessary copies
92
93 ```java
94 void compute(Accelerator accelerator, MemorySegment memorySegment, int len) {
95 boolean injectedMemorySegmentIsDirty = false;
96 Accelerator.Range range = accelerator.range(len);
97 if (injectedMemorySegmentIsDirty){
98 accelerator.injectedCopyToDevice(memorySegment);
99 }
100 accelerator.run(Compute::kernel, range, memorySegment);
101 injectedMemorySegmentIsDirty = true; // based on Compute::kernel sidetable
102 if (injectedMemorySegmentIsDirty) {
103 accelerator.injectedCopyFromDevice(memorySegment);
104 }
105 }
106 ```
107
108
109 Whether this code mutation generates Java bytecode and executes (or interprets) on the JVM or whether the
110 CodeModels for the closure are handed over to a backend which reifies the kernel code and the
111 logic for dispatch is not defined.
112
113 The code model for the compute will be mutated to inject the appropriate nodes to achieve the goal
114
115 It is possible that some vendors may just take the original code model and analyse themselves.
116
117 Clearly this is a trivial compute closure. Lets discuss the required kernel analysis
118 and proposed pseudo code.
119
120 ## Copying data based on kernel MemorySegment analysis
121
122 Above we showed that we should be able to determine whether a kernel mutates or accesses any of
123 it's Kernel MemorySegment parameters.
124
125 We determined above that the kernel only called set() so we need
126 not copy the data to the device.
127
128 The following example shows a kernel which reads and mutates a memorysegment
129 ```java
130 static class Compute {
131 @CodeReflection public static
132 void doubleup(Accelerator.NDRange ndrange, MemorySegment memorySegment) {
133 int temp = memorySegment.get(JAVA_INT, ndrange.id.x);
134 memorySegment.set(JAVA_INT, temp*2);
135 }
136
137 @CodeReflection public static
138 void compute(Accelerator accelerator, MemorySegment memorySegment, int len) {
139 Accelerator.Range range = accelerator.range(len);
140 accelerator.run(Compute::doubleup, range, memorySegment);
141 }
142 }
143 ```
144 Here our analysis needs to determine that the kernel reads and writes to the segment (it does)
145 so the generated compute model would equate to
146
147 ```java
148 void compute(Accelerator accelerator, MemorySegment memorySegment, int len) {
149 Accelerator.Range range = accelerator.range(len);
150 accelerator.copyToDevice(memorySegment); // injected via Babylon
151 accelerator.run(Compute::doubleup, range, memorySegment);
152 accelerator.copyFromDevice(memorySegment); // injected via Babylon
153 }
154 ```
155 So far the deductions are fairly trivial
156
157 Consider
158 ```java
159 @CodeReflection public static
160 void compute(Accelerator accelerator, MemorySegment memorySegment, int len, int count) {
161 Accelerator.Range range = accelerator.range(len);
162 for (int i=0; i<count; i++) {
163 accelerator.run(Compute::doubleup, range, memorySegment);
164 }
165 }
166 ```
167
168 Here HAT should deduce that the java side is merely looping over the kernel dispatch
169 and has no interest in the memorysegment between dispatches.
170
171 So the new model need only copy in once (before the fist kernel) and out once (prior to return)
172
173 ```java
174 @CodeReflection public static
175 void compute(Accelerator accelerator, MemorySegment memorySegment, int len, int count) {
176 Accelerator.Range range = accelerator.range(len);
177 accelerator.copyToDevice(memorySegment); // injected via Babylon
178 for (int i=0; i<count; i++) {
179 accelerator.run(Compute::doubleup, range, memorySegment);
180 }
181 accelerator.copyFromDevice(memorySegment); // injected via Babylon
182 }
183 ```
184
185 Things get slightly more interesting when we do indeed access the memory segment
186 from the Java code inside the loop.
187
188 ```java
189 @CodeReflection public static
190 void compute(Accelerator accelerator, MemorySegment memorySegment, int len, int count) {
191 Accelerator.Range range = accelerator.range(len);
192 for (int i=0; i<count; i++) {
193 accelerator.run(Compute::doubleup, range, memorySegment);
194 int slot0 = memorySegment.get(INTVALUE, 0);
195 System.out.println("slot0 ", slot0);
196 }
197 }
198 ```
199 Now we expect babylon to inject a read inside the loop to make the data available java side
200
201 ```java
202 @CodeReflection public static
203 void compute(Accelerator accelerator, MemorySegment memorySegment, int len, int count) {
204 Accelerator.Range range = accelerator.range(len);
205 accelerator.copyToDevice(memorySegment); // injected via Babylon
206 for (int i=0; i<count; i++) {
207 accelerator.run(Compute::doubleup, range, memorySegment);
208 accelerator.copyFromDevice(memorySegment); // injected via Babylon
209 int slot0 = memorySegment.get(INTVALUE, 0);
210 System.out.println("slot0 ", slot0);
211 }
212
213 }
214 ```
215
216 Note that in this case we are only accessing 0th int from the segment so a possible
217 optimization might be to allow the vendor to only copy back this one element....
218 ```java
219 @CodeReflection public static
220 void compute(Accelerator accelerator, MemorySegment memorySegment, int len, int count) {
221 Accelerator.Range range = accelerator.range(len);
222 accelerator.copyToDevice(memorySegment); // injected via Babylon
223 for (int i=0; i<count; i++) {
224 accelerator.run(Compute::doubleup, range, memorySegment);
225 if (i+1==count){// injected
226 accelerator.copyFromDevice(memorySegment); // injected
227 }else {
228 accelerator.copyFromDevice(memorySegment, 1); // injected via Babylon
229 }
230 int slot0 = memorySegment.get(INTVALUE, 0);
231 System.out.println("slot0 ", slot0);
232 }
233
234 }
235 ```
236
237 Again HAT will merely mutate the code model of the compute method,
238 the vendor may choose to interpret bytecode, generate bytecode and execute
239 or take complete control and execute the model in native code.
240
241 So within HAT we must find all set/get calls on MemorySegments and trace them back to kernel parameters.
242
243 We should allow aliasing of memory segments... but in the short term we may well throw an exception when we see such aliasing
244
245
246 ```java
247 @CodeReflection public static
248 void doubleup(Accelerator.NDRange ndrange, MemorySegment memorySegment) {
249 MemorySegment alias = memorySegment;
250 alias.set(JAVA_INT, ndrange.id.x, alias.get(JAVA_INT, ndrange.id.x)*2);
251 }
252 ```
253
254 ## Weed warning #1
255
256 We could find common kernel errors when analyzing
257
258 This code is probably wrong, as it is racey writing to 0th element
259
260 ```java
261 void doubleup(Accelerator.NDRange ndrange, MemorySegment memorySegment) {
262 MemorySegment alias = memorySegment;
263 alias.set(JAVA_INT, 0, alias.get(JAVA_INT, ndrange.id.x)*2);
264 }
265 ```
266
267 By allowing a 'lint' like plugin mechanism for code model it would be easy to find.
268 If we ever find a constant index in set(...., <constant> ) we are probably in a world of hurt.
269 Unless the set is included in some conditional which itself is dependant on a value extracted from a memory segment.
270
271 ```java
272 void doubleup(Accelerator.NDRange ndrange, MemorySegment memorySegment) {
273 MemorySegment alias = memorySegment;
274 if (????){
275 alias.set(JAVA_INT, 0, alias.get(JAVA_INT, ndrange.id.x) * 2);
276 }
277 }
278 ```
279
280 There are a lot opportunities for catching such bugs.
281
282
283 ## Flipping Generations
284
285 Many algorithms require us to process data from generations. Consider
286 Convolutions or Game Of Life style problems where we have an image or game state and
287 we need to calculate the result of applying rules to cells in the image or game.
288
289 It is important that when we process the next generation (either in parallel or sequentially) we
290 must ensure that we only use prev generation data to generate next generation data.
291
292 ```
293 [ ][ ][*][ ][ ] [ ][ ][ ][ ][ ]
294 [ ][ ][*][ ][ ] [ ][*][*][*][ ]
295 [ ][ ][*][ ][ ] -> [ ][ ][ ][ ][ ]
296 [ ][ ][ ][ ][ ] [ ][ ][ ][ ][ ]
297
298 ```
299
300 This usually requires us to hold two copies, and applying the kernel to one input set
301 which writes to the output.
302
303 In the case of the Game Of Life we may well use the output as the next input...
304
305 ```java
306 @CodeReflection void conway(Accelerator.NDRange ndrange,
307 MemorySegment in, MemorySegment out, int width, int height) {
308 int cx = ndrange.id.x % ndrange.id.maxx;
309 int cy = ndrange.id.x / ndrange.id.maxx;
310
311 int sum = 0;
312 for (int dx = -1; dx < 2; dy++) {
313 for (int dy = -1; dy < 2; dy++) {
314 if (dx != 0 || dy != 0) {
315 int x = cx + dx;
316 int y = cy + dy;
317 if (x >= 0 && x < widh && y >= 0 && y < height) {
318 sum += in.get(INT, x * width + h);
319 }
320 }
321 }
322 }
323 result = GOLRules(sum, in.get(INT, ndrange.id.x));
324 out.set(INT, ndrange.id.x);
325
326 }
327 ```
328
329 In this case the assumption is that the compute layer will swap the buffers for alternate passes
330
331 ```java
332 import java.lang.foreign.MemorySegment;
333
334 @CodeReflection
335 void compute(Accelerator accelerator, MemorySegment gameState,
336 int width, int height, int maxGenerations) {
337 MemorySegment s1 = gameState;
338 MemorySegment s2 = allocateGameState(width, height);
339 for (int generation = 0; generation < maxGenerations; generation++){
340 MemorySegment from = generation%2==0?s1?s2;
341 MemorySegment to = generation%2==1?s1?s2;
342 accelerator.run(Compute::conway, from, to, range, width, height);
343 }
344 if (maxGenerations%2==1){ // ?
345 gameState.copyFrom(s2);
346 }
347 }
348 ```
349
350 This common pattern includes some aliasing of MemorySegments that we need to untangle.
351
352 HAT needs to be able to track the aliases to determine the minimal number of copies.
353 ```java
354 import java.lang.foreign.MemorySegment;
355
356 @CodeReflection
357 void compute(Accelerator accelerator, MemorySegment gameState, int width, int height, int maxGenerations,
358 DisplaySAM displaySAM) {
359 MemorySegment s1 = gameState;
360 MemorySegment s2 = allocateGameState(width, height);
361
362 for (int generation = 0; generation < maxGenerations; generation++){
363 MemorySegment from = generation%2==0?s1?s2;
364 MemorySegment to = generation%2==1?s1?s2;
365 if (generation == 0) { /// injected
366 accerator.copyToDevice(from); // injected
367 } // injected
368 accelerator.run(Compute::conway, from, to, range, width, height, 1000);
369 if (generation == maxGenerations-1){ // injected
370 accerator.copyFromDevice(to); //injected
371 } //injected
372 }
373 if (maxGenerations%2==1){ // ?
374 gameState.copyFrom(s2);
375 }
376
377 }
378 ```
379
380 ```java
381 import java.lang.foreign.MemorySegment;
382
383 @CodeReflection
384 void compute(Accelerator accelerator, MemorySegment gameState, int width, int height,
385 int maxGenerations,
386 DisplaySAM displaySAM) {
387 MemorySegment s1 = gameState;
388 MemorySegment s2 = allocateGameState(width, height);
389
390 for (int generation = 0; generation < maxGenerations; generation++){
391 MemorySegment from = generation%2==0?s1?s2;
392 MemorySegment to = generation%2==1?s1?s2;
393 accelerator.run(Compute::conway, from, to, range, width, height,1000);
394 displaySAM.display(s2,width, height);
395 }
396 if (maxGenerations%2==1){ // ?
397 gameState.copyFrom(to);
398 }
399 }
400 ```
401
402
403
404 ### Example babylon transform to track buffer mutations.
405
406 One goal of hat was to automate the movement of buffers from Java to device.
407
408 One strategy employed by `NativeBackends` might be to track 'ifaceMappedSegment' accesses and inject tracking data into the compute method.
409
410 Here is a transformation for that
411
412 ```java
413 static FuncOpWrapper injectBufferTracking(ComputeClosure.ResolvedMethodCall resolvedMethodCall) {
414 FuncOpWrapper original = resolvedMethodCall.funcOpWrapper();
415 var transformed = original.transformInvokes((builder, invoke) -> {
416 if (invoke.isIfaceBufferMethod()) { // void array(long idx, T value) or T array(long idx)
417 // Get the first parameter (computeClosure)
418 CopyContext cc = builder.context();
419 Value computeClosure = cc.getValue(original.parameter(0));
420 // Get the buffer receiver value in the output model
421 Value receiver = cc.getValue(invoke.operand(0)); // The buffer we are mutatibg or accessing
422 if (invoke.isIfaceMutator()) {
423 // inject computeContext.preMutate(buffer);
424 builder.op(CoreOps.invoke(ComputeClosure.M_CC_PRE_MUTATE, computeClosure, receiver));
425 builder.op(invoke.op());
426 // inject computeContext.postMutate(buffer);
427 builder.op(CoreOps.invoke(ComputeClosure.M_CC_POST_MUTATE, computeClosure, receiver));
428 } else if ( invoke.isIfaceAccessor()) {
429 // inject computeContext.preAccess(buffer);
430 builder.op(CoreOps.invoke(ComputeClosure.M_CC_PRE_ACCESS, computeClosure, receiver));
431 builder.op(invoke.op());
432 // inject computeContext.postAccess(buffer);
433 builder.op(CoreOps.invoke(ComputeClosure.M_CC_POST_ACCESS, computeClosure, receiver));
434 } else {
435 builder.op(invoke.op());
436 }
437 }else{
438 builder.op(invoke.op());
439 }
440 return builder;
441 }
442 );
443 transformed.op().writeTo(System.out);
444 resolvedMethodCall.funcOpWrapper(transformed);
445 return transformed;
446 }
447 ```
448
449 So in our `OpenCLBackend` for example
450 ```java
451 public void mutateIfNeeded(ComputeClosure.MethodCall methodCall) {
452 injectBufferTracking(entrypoint);
453 }
454
455 @Override
456 public void computeContextClosed(ComputeContext computeContext){
457 var codeBuilder = new OpenCLKernelBuilder();
458 C99Code kernelCode = createKernelCode(computeContext, codeBuilder);
459 System.out.println(codeBuilder);
460 }
461 ```
462 I hacked the Mandle example. So the compute accessed and mutated it's arrays.
463
464 ```java
465 @CodeReflection
466 static float doubleit(float f) {
467 return f * 2;
468 }
469
470 @CodeReflection
471 static float scaleUp(float f) {
472 return doubleit(f);
473 }
474
475 @CodeReflection
476 static public void compute(final ComputeContext computeContext, S32Array2D s32Array2D, float x, float y, float scale) {
477 scale = scaleUp(scale);
478 var range = computeContext.accelerator.range(s32Array2D.size());
479 int i = s32Array2D.get(10,10);
480 s32Array2D.set(10,10,i);
481 computeContext.dispatchKernel(MandelCompute::kernel, range, s32Array2D, pallette, x, y, scale);
482 }
483 ```
484 So here is the transformation being applied to the above compute
485
486 BEFORE (note the !'s indicating accesses through ifacebuffers)
487 ```
488 func @"compute" (%0 : hat.ComputeContext, %1 : hat.buffer.S32Array2D, %2 : float, %3 : float, %4 : float)void -> {
489 %5 : Var<hat.ComputeContext> = var %0 @"computeContext";
490 %6 : Var<hat.buffer.S32Array2D> = var %1 @"s32Array2D";
491 %7 : Var<float> = var %2 @"x";
492 %8 : Var<float> = var %3 @"y";
493 %9 : Var<float> = var %4 @"scale";
494 %10 : float = var.load %9;
495 %11 : float = invoke %10 @"mandel.Main::scaleUp(float)float";
496 var.store %9 %11;
497 %12 : hat.ComputeContext = var.load %5;
498 %13 : hat.Accelerator = field.load %12 @"hat.ComputeContext::accelerator()hat.Accelerator";
499 %14 : hat.buffer.S32Array2D = var.load %6;
500 ! %15 : int = invoke %14 @"hat.buffer.S32Array2D::size()int";
501 %16 : hat.NDRange = invoke %13 %15 @"hat.Accelerator::range(int)hat.NDRange";
502 %17 : Var<hat.NDRange> = var %16 @"range";
503 %18 : hat.buffer.S32Array2D = var.load %6;
504 %19 : int = constant @"10";
505 %20 : int = constant @"10";
506 ! %21 : int = invoke %18 %19 %20 @"hat.buffer.S32Array2D::get(int, int)int";
507 %22 : Var<int> = var %21 @"i";
508 %23 : hat.buffer.S32Array2D = var.load %6;
509 %24 : int = constant @"10";
510 %25 : int = constant @"10";
511 %26 : int = var.load %22;
512 ! invoke %23 %24 %25 %26 @"hat.buffer.S32Array2D::set(int, int, int)void";
513 %27 : hat.ComputeContext = var.load %5;
514 ...
515 ```
516 AFTER
517 ```
518 func @"compute" (%0 : hat.ComputeContext, %1 : hat.buffer.S32Array2D, %2 : float, %3 : float, %4 : float)void -> {
519 %5 : Var<hat.ComputeContext> = var %0 @"computeContext";
520 %6 : Var<hat.buffer.S32Array2D> = var %1 @"s32Array2D";
521 %7 : Var<float> = var %2 @"x";
522 %8 : Var<float> = var %3 @"y";
523 %9 : Var<float> = var %4 @"scale";
524 %10 : float = var.load %9;
525 %11 : float = invoke %10 @"mandel.Main::scaleUp(float)float";
526 var.store %9 %11;
527 %12 : hat.ComputeContext = var.load %5;
528 %13 : hat.Accelerator = field.load %12 @"hat.ComputeContext::accelerator()hat.Accelerator";
529 %14 : hat.buffer.S32Array2D = var.load %6;
530 invoke %0 %14 @"hat.ComputeClosure::preAccess(hat.buffer.Buffer)void";
531 ! %15 : int = invoke %14 @"hat.buffer.S32Array2D::size()int";
532 invoke %0 %14 @"hat.ComputeClosure::postAccess(hat.buffer.Buffer)void";
533 %16 : hat.NDRange = invoke %13 %15 @"hat.Accelerator::range(int)hat.NDRange";
534 %17 : Var<hat.NDRange> = var %16 @"range";
535 %18 : hat.buffer.S32Array2D = var.load %6;
536 %19 : int = constant @"10";
537 %20 : int = constant @"10";
538 invoke %0 %18 @"hat.ComputeClosure::preAccess(hat.buffer.Buffer)void";
539 ! %21 : int = invoke %18 %19 %20 @"hat.buffer.S32Array2D::get(int, int)int";
540 invoke %0 %18 @"hat.ComputeClosure::postAccess(hat.buffer.Buffer)void";
541 %22 : Var<int> = var %21 @"i";
542 %23 : hat.buffer.S32Array2D = var.load %6;
543 %24 : int = constant @"10";
544 %25 : int = constant @"10";
545 %26 : int = var.load %22;
546 invoke %0 %23 @"hat.ComputeClosure::preMutate(hat.buffer.Buffer)void";
547 ! invoke %23 %24 %25 %26 @"hat.buffer.S32Array2D::set(int, int, int)void";
548 invoke %0 %23 @"hat.ComputeClosure::postMutate(hat.buffer.Buffer)void";
549 %27 : hat.ComputeContext = var.load %5;
550 ```
551 And here at runtime the ComputeClosure is reporting accesses when executing via the interpreter after the injected calls.
552
553 ```
554 ComputeClosure.preAccess S32Array2D[width()=1024, height()=1024, array()=int[1048576]]
555 ComputeClosure.postAccess S32Array2D[width()=1024, height()=1024, array()=int[1048576]]
556 ComputeClosure.preAccess S32Array2D[width()=1024, height()=1024, array()=int[1048576]]
557 ComputeClosure.postAccess S32Array2D[width()=1024, height()=1024, array()=int[1048576]]
558 ComputeClosure.preMutate S32Array2D[width()=1024, height()=1024, array()=int[1048576]]
559 ComputeClosure.postMutate S32Array2D[width()=1024, height()=1024, array()=int[1048576]]
560 ```
561 ## Why inject this info?
562 So the idea is that the ComputeContext would maintain sets of dirty buffers, one set for `gpuDirty` and one set for `javaDirty`.
563
564 We have the code for kernel models. So we know which kernel accesses, mutates or accesses AND mutates particular parameters.
565
566 So when the ComputeContext receives `preAccess(x)` or `preMutate(x)` the ComputeContext would determine if `x` is in the `gpuDirty` set.
567 If so it would delegate to the backend to copy the GPU data back from device into the memory segment (assuming the memory is not coherent!)
568 before removing the buffer from `gpuDirty` set and returning.
569
570 Now the Java access to the segment sees the latest buffer.
571
572 After `postMutate(x)` it will place the buffer in `javaDirty` set.
573
574 When a kernel dispatch comes along, the parameters to the kernel are all checked against the `javaDirty` set.
575 If the parameter is 'accessed' by the kernel. The backend will copy the segment to device. Remove the parameter
576 from the `javaDirty` set and then invoke the kernel.
577 When the kernel completes (lets assume synchronous for a moment) all parameters are checked again, and if the parameter
578 is known to be mutated by the kernel the parameter is added to the 'gpuDirty' set.
579
580 This way we don't have to force the developer to request data movements.
581
582 BTW if kernel requests are async ;) then the ComputeContext maintains a map of buffer to kernel. So `preAccess(x)` or `preMutate(x)` calls
583 can wait on the kernel that is due to 'dirty' the buffer to complete.
584
585 ### Marking hat buffers directly.
586
587
588
589
590
591
592
593
594