1 /*
  2  *  Copyright (c) 2019, 2021, Oracle and/or its affiliates. All rights reserved.
  3  *  DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  4  *
  5  *  This code is free software; you can redistribute it and/or modify it
  6  *  under the terms of the GNU General Public License version 2 only, as
  7  *  published by the Free Software Foundation.  Oracle designates this
  8  *  particular file as subject to the "Classpath" exception as provided
  9  *  by Oracle in the LICENSE file that accompanied this code.
 10  *
 11  *  This code is distributed in the hope that it will be useful, but WITHOUT
 12  *  ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 13  *  FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
 14  *  version 2 for more details (a copy is included in the LICENSE file that
 15  *  accompanied this code).
 16  *
 17  *  You should have received a copy of the GNU General Public License version
 18  *  2 along with this work; if not, write to the Free Software Foundation,
 19  *  Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
 20  *
 21  *   Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
 22  *  or visit www.oracle.com if you need additional information or have any
 23  *  questions.
 24  *
 25  */
 26 
 27 /**
 28  * <p> Classes to support low-level and efficient foreign memory/function access, directly from Java.
 29  *
 30  * <h2>Foreign memory access</h2>
 31  *
 32  * <p>
 33  * The main abstractions introduced to support foreign memory access is {@link jdk.incubator.foreign.MemorySegment}, which
 34  * models a contiguous memory region, which can reside either inside or outside the Java heap.
 35  * A memory segment represents the main access coordinate of a memory access var handle, which can be obtained
 36  * using the combinator methods defined in the {@link jdk.incubator.foreign.MemoryHandles} class; a set of
 37  * common dereference and copy operations is provided also by the {@link jdk.incubator.foreign.MemorySegment} class, which can
 38  * be useful for simple, non-structured access. Finally, the {@link jdk.incubator.foreign.MemoryLayout} class
 39  * hierarchy enables description of <em>memory layouts</em> and basic operations such as computing the size in bytes of a given
 40  * layout, obtain its alignment requirements, and so on. Memory layouts also provide an alternate, more abstract way, to produce
 41  * memory access var handles, e.g. using <a href="MemoryLayout.html#layout-paths"><em>layout paths</em></a>.
 42  *
 43  * For example, to allocate an off-heap memory region big enough to hold 10 values of the primitive type {@code int}, and fill it with values
 44  * ranging from {@code 0} to {@code 9}, we can use the following code:
 45  *
 46  * <pre>{@code
 47 MemorySegment segment = MemorySegment.allocateNative(10 * 4, ResourceScope.newImplicitScope());
 48 for (int i = 0 ; i < 10 ; i++) {
 49    segment.setAtIndex(ValueLayout.JAVA_INT, i, 42);
 50 }
 51  * }</pre>
 52  *
 53  * This code creates a <em>native</em> memory segment, that is, a memory segment backed by
 54  * off-heap memory; the size of the segment is 40 bytes, enough to store 10 values of the primitive type {@code int}.
 55  * Inside a loop, we then initialize the contents of the memory segment using the
 56  * {@link jdk.incubator.foreign.MemorySegment#setAtIndex(ValueLayout.OfInt, long, int)} dereference method. Note how
 57  * the dereference method accepts a {@linkplain jdk.incubator.foreign.ValueLayout value layout},
 58  * which specifies the size, alignment constraints, byte order as well
 59  * as the Java type ({@code int}, in this case) associated with the dereference operation. More specifically,
 60  * if we view the memory segment as a set of 10 adjacent slots, {@code s[i]}, where {@code 0 <= i < 10},
 61  * where the size of each slot is exactly 4 bytes, the initialization logic above will set each slot
 62  * so that {@code s[i] = i}, again where {@code 0 <= i < 10}.
 63  *
 64  * <h3><a id="deallocation"></a>Deterministic deallocation</h3>
 65  *
 66  * When writing code that manipulates memory segments, especially if backed by memory which resides outside the Java heap, it is
 67  * often crucial that the resources associated with a memory segment are released when the segment is no longer in use,
 68  * and in a timely fashion. For this reason, there might be cases where waiting for the garbage collector to determine that a segment
 69  * is <a href="../../../java/lang/ref/package.html#reachability">unreachable</a> is not optimal.
 70  * Clients that operate under these assumptions might want to programmatically release the memory associated
 71  * with a memory segment. This can be done, using the {@link jdk.incubator.foreign.ResourceScope} abstraction, as shown below:
 72  *
 73  * <pre>{@code
 74 try (ResourceScope scope = ResourceScope.newConfinedScope()) {
 75     MemorySegment segment = MemorySegment.allocateNative(10 * 4, scope);
 76     for (int i = 0 ; i < 10 ; i++) {
 77         segment.setAtIndex(ValueLayout.JAVA_INT, i, 42);
 78     }
 79 }
 80  * }</pre>
 81  *
 82  * This example is almost identical to the prior one; this time we first create a so called <em>resource scope</em>,
 83  * which is used to <em>bind</em> the life-cycle of the segment created immediately afterwards. Note the use of the
 84  * <em>try-with-resources</em> construct: this idiom ensures that all the memory resources associated with the segment will be released
 85  * at the end of the block, according to the semantics described in Section {@jls 14.20.3} of <cite>The Java Language Specification</cite>.
 86  *
 87  * <h3><a id="safety"></a>Safety</h3>
 88  *
 89  * This API provides strong safety guarantees when it comes to memory access. First, when dereferencing a memory segment,
 90  * the access coordinates are validated (upon access), to make sure that access does not occur at an address which resides
 91  * <em>outside</em> the boundaries of the memory segment used by the dereference operation. We call this guarantee <em>spatial safety</em>;
 92  * in other words, access to memory segments is bounds-checked, in the same way as array access is, as described in
 93  * Section {@jls 15.10.4} of <cite>The Java Language Specification</cite>.
 94  * <p>
 95  * Since memory segments can be closed (see above), segments are also validated (upon access) to make sure that
 96  * the resource scope associated with the segment being accessed has not been closed prematurely.
 97  * We call this guarantee <em>temporal safety</em>. Together, spatial and temporal safety ensure that each memory access
 98  * operation either succeeds - and accesses a valid memory location - or fails.
 99  *
100  * <h2>Foreign function access</h2>
101  * The key abstractions introduced to support foreign function access are {@link jdk.incubator.foreign.SymbolLookup},
102  * {@link jdk.incubator.foreign.MemoryAddress} and {@link jdk.incubator.foreign.CLinker}.
103  * The first is used to lookup symbols inside native libraries; the second is used to model native addresses (more on that later),
104  * while the third provides linking capabilities which allows modelling foreign functions as {@link java.lang.invoke.MethodHandle} instances,
105  * so that clients can perform foreign function calls directly in Java, without the need for intermediate layers of native
106  * code (as it's the case with the <a href="{@docRoot}/../specs/jni/index.html">Java Native Interface (JNI)</a>).
107  * <p>
108  * For example, to compute the length of a string using the C standard library function {@code strlen} on a Linux x64 platform,
109  * we can use the following code:
110  *
111  * <pre>{@code
112       var linker = CLinker.systemCLinker();
113       MethodHandle strlen = linker.downcallHandle(
114         linker.lookup("strlen").get(),
115         FunctionDescriptor.of(ValueLayout.JAVA_LONG, ValueLayout.ADDRESS)
116       );
117 
118       try (var scope = ResourceScope.newConfinedScope()) {
119          var cString = MemorySegment.allocateNative(5 + 1, scope);
120          cString.setUtf8String("Hello");
121          long len = (long)strlen.invoke(cString); // 5
122       }
123  * }</pre>
124  *
125  * Here, we obtain a {@linkplain jdk.incubator.foreign.CLinker#systemCLinker() linker instance} and we use it
126  * to {@linkplain jdk.incubator.foreign.CLinker#lookup(java.lang.String) lookup} the {@code strlen} symbol in the
127  * standard C library; a <em>downcall method handle</em> targeting said symbol is subsequently
128  * {@linkplain jdk.incubator.foreign.CLinker#downcallHandle(jdk.incubator.foreign.FunctionDescriptor) obtained}.
129  * To complete the linking successfully, we must provide a {@link jdk.incubator.foreign.FunctionDescriptor} instance,
130  * describing the signature of the {@code strlen} function.
131  * From this information, the linker will uniquely determine the sequence of steps which will turn
132  * the method handle invocation (here performed using {@link java.lang.invoke.MethodHandle#invoke(java.lang.Object...)})
133  * into a foreign function call, according to the rules specified by the platform C ABI.
134  * The {@link jdk.incubator.foreign.MemorySegment} class also provides many useful methods for
135  * interacting with native code, such as converting Java strings into native strings and back
136  * (see {@link jdk.incubator.foreign.MemorySegment#setUtf8String(long, java.lang.String)} and
137  * {@link jdk.incubator.foreign.MemorySegment#getUtf8String(long)}, respectively), as demonstrated in the above example.
138  *
139  * <h3>Foreign addresses</h3>
140  *
141  * When a memory segment is created from Java code, the segment properties (spatial bounds, temporal bounds and confinement)
142  * are fully known at segment creation. But when interacting with native libraries, clients will often receive <em>raw</em> pointers;
143  * such pointers have no spatial bounds (example: does the C type {@code char*} refer to a single {@code char} value,
144  * or an array of {@code char} values, of given size?), no notion of temporal bounds, nor thread-confinement.
145  * <p>
146  * Raw pointers are modelled using the {@link jdk.incubator.foreign.MemoryAddress} class. When clients receive a
147  * memory address instance from a foreign function call, they can perform memory dereference on it directly,
148  * using one of the many <em>unsafe</em> dereference methods provided
149  * (see {@link jdk.incubator.foreign.MemoryAddress#get(jdk.incubator.foreign.ValueLayout.OfInt, long)}):
150  *
151  * <pre>{@code
152 ...
153 MemoryAddress addr = ... //obtain address from native code
154 int x = addr.get(ValueLayout.JAVA_INT, 0);
155  * }</pre>
156  *
157  * Alternatively, the client can create a memory segment <em>unsafely</em>, using the
158  * {@link jdk.incubator.foreign.MemorySegment#ofAddressNative(jdk.incubator.foreign.MemoryAddress, long, jdk.incubator.foreign.ResourceScope)} factory.
159  * This allows the client to inject extra knowledge about spatial bounds which might, for instance, be available in the documentation of the foreign function
160  * which produced the native address. Here is how an unsafe segment can be created from a native address:
161  *
162  * <pre>{@code
163 ResourceScope scope = ... // initialize a resource scope object
164 MemoryAddress addr = ... //obtain address from native code
165 MemorySegment segment = MemorySegment.ofAddressNative(addr, 4, scope); // segment is 4 bytes long
166 int x = segment.get(ValueLayout.JAVA_INT, 0);
167  * }</pre>
168  *
169  * <h3>Upcalls</h3>
170  * The {@link jdk.incubator.foreign.CLinker} interface also allows to turn an existing method handle (which might point
171  * to a Java method) into a memory address, so that Java code can effectively be passed to other foreign functions.
172  * For instance, we can write a method that compares two integer values, as follows:
173  *
174  * <pre>{@code
175 class IntComparator {
176     static int intCompare(MemoryAddress addr1, MemoryAddress addr2) {
177         return addr1.get(ValueLayout.JAVA_INT, 0) - addr2.get(ValueLayout.JAVA_INT, 0);
178     }
179 }
180  * }</pre>
181  *
182  * The above method dereferences two memory addresses containing an integer value, and performs a simple comparison
183  * by returning the difference between such values. We can then obtain a method handle which targets the above static
184  * method, as follows:
185  *
186  * <pre>{@code
187 MethodHandle intCompareHandle = MethodHandles.lookup().findStatic(IntComparator.class,
188                                                    "intCompare",
189                                                    MethodType.methodType(int.class, MemoryAddress.class, MemoryAddress.class));
190  * }</pre>
191  *
192  * Now that we have a method handle instance, we can link it into a fresh native memory address, using the {@link jdk.incubator.foreign.CLinker} interface, as follows:
193  *
194  * <pre>{@code
195 ResourceScope scope = ...
196 Addressable comparFunc = CLinker.systemCLinker().upcallStub(
197      intCompareHandle,
198      FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.ADDRESS, ValueLayout.ADDRESS),
199      scope
200 );
201  * }</pre>
202  *
203  * As before, we need to provide a {@link jdk.incubator.foreign.FunctionDescriptor} instance describing the signature
204  * of the function pointer we want to create; this descriptor allows the linker to determine the
205  * sequence of steps which allow foreign code to call the stub for {@code intCompareHandle} according to the rules specified
206  * by the platform C ABI. The lifecycle of the stub returned by
207  * {@link jdk.incubator.foreign.CLinker#upcallStub(java.lang.invoke.MethodHandle, jdk.incubator.foreign.FunctionDescriptor, jdk.incubator.foreign.ResourceScope)}
208  * is tied to the {@linkplain jdk.incubator.foreign.ResourceScope resource scope} parameter passed to that method. This
209  * is made available by the {@link jdk.incubator.foreign.NativeSymbol} instance returned by that method.
210  *
211  * <a id="restricted"></a>
212  * <h2>Restricted methods</h2>
213  * Some methods in this package are considered <em>restricted</em>. Restricted methods are typically used to bind native
214  * foreign data and/or functions to first-class Java API elements which can then be used directly by clients. For instance
215  * the restricted method {@link jdk.incubator.foreign.MemorySegment#ofAddressNative(jdk.incubator.foreign.MemoryAddress, long, jdk.incubator.foreign.ResourceScope)}
216  * can be used to create a fresh segment with given spatial bounds out of a native address.
217  * <p>
218  * Binding foreign data and/or functions is generally unsafe and, if done incorrectly, can result in VM crashes, or memory corruption when the bound Java API element is accessed.
219  * For instance, in the case of {@link jdk.incubator.foreign.MemorySegment#ofAddressNative(jdk.incubator.foreign.MemoryAddress, long, jdk.incubator.foreign.ResourceScope)},
220  * if the provided spatial bounds are incorrect, a client of the segment returned by that method might crash the VM, or corrupt
221  * memory when attempting to dereference said segment. For these reasons, it is crucial for code that calls a restricted method
222  * to never pass arguments that might cause incorrect binding of foreign data and/or functions to a Java API.
223  * <p>
224  * Access to restricted methods is <em>disabled</em> by default; to enable restricted methods, the command line option
225  * {@code --enable-native-access} must mention the name of the caller's module.
226  */
227 package jdk.incubator.foreign;