1 /*
  2  *  Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
  3  *  DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  4  *
  5  *  This code is free software; you can redistribute it and/or modify it
  6  *  under the terms of the GNU General Public License version 2 only, as
  7  *  published by the Free Software Foundation.  Oracle designates this
  8  *  particular file as subject to the "Classpath" exception as provided
  9  *  by Oracle in the LICENSE file that accompanied this code.
 10  *
 11  *  This code is distributed in the hope that it will be useful, but WITHOUT
 12  *  ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 13  *  FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
 14  *  version 2 for more details (a copy is included in the LICENSE file that
 15  *  accompanied this code).
 16  *
 17  *  You should have received a copy of the GNU General Public License version
 18  *  2 along with this work; if not, write to the Free Software Foundation,
 19  *  Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
 20  *
 21  *   Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
 22  *  or visit www.oracle.com if you need additional information or have any
 23  *  questions.
 24  *
 25  */
 26 
 27 /**
 28  * <p> Classes to support low-level and efficient foreign memory/function access, directly from Java.
 29  *
 30  * <h2>Foreign memory access</h2>
 31  *
 32  * <p>
 33  * The key abstractions introduced to support foreign memory access are {@link jdk.incubator.foreign.MemorySegment} and {@link jdk.incubator.foreign.MemoryAddress}.
 34  * The first models a contiguous memory region, which can reside either inside or outside the Java heap; the latter models an address - which also can
 35  * reside either inside or outside the Java heap (and can sometimes be expressed as an offset into a given segment).
 36  * A memory segment represents the main access coordinate of a memory access var handle, which can be obtained
 37  * using the combinator methods defined in the {@link jdk.incubator.foreign.MemoryHandles} class; a set of
 38  * common dereference operations is provided also by the {@link jdk.incubator.foreign.MemoryAccess} class, which can
 39  * be useful for simple, non-structured access. Finally, the {@link jdk.incubator.foreign.MemoryLayout} class
 40  * hierarchy enables description of <em>memory layouts</em> and basic operations such as computing the size in bytes of a given
 41  * layout, obtain its alignment requirements, and so on. Memory layouts also provide an alternate, more abstract way, to produce
 42  * memory access var handles, e.g. using <a href="MemoryLayout.html#layout-paths"><em>layout paths</em></a>.
 43  *
 44  * For example, to allocate an off-heap memory region big enough to hold 10 values of the primitive type {@code int}, and fill it with values
 45  * ranging from {@code 0} to {@code 9}, we can use the following code:
 46  *
 47  * <pre>{@code
 48 MemorySegment segment = MemorySegment.allocateNative(10 * 4, ResourceScope.newImplicitScope());
 49 for (int i = 0 ; i < 10 ; i++) {
 50    MemoryAccess.setIntAtIndex(segment, i, 42);
 51 }
 52  * }</pre>
 53  *
 54  * Here create a <em>native</em> memory segment, that is, a memory segment backed by
 55  * off-heap memory; the size of the segment is 40 bytes, enough to store 10 values of the primitive type {@code int}.
 56  * Inside a loop, we then initialize the contents of the memory segment using the
 57  * {@link jdk.incubator.foreign.MemoryAccess#setIntAtIndex(jdk.incubator.foreign.MemorySegment, long, int)} helper method;
 58  * more specifically, if we view the memory segment as a set of 10 adjacent slots,
 59  * {@code s[i]}, where {@code 0 <= i < 10}, where the size of each slot is exactly 4 bytes, the initialization logic above will set each slot
 60  * so that {@code s[i] = i}, again where {@code 0 <= i < 10}.
 61  *
 62  * <h3><a id="deallocation"></a>Deterministic deallocation</h3>
 63  *
 64  * When writing code that manipulates memory segments, especially if backed by memory which resides outside the Java heap, it is
 65  * often crucial that the resources associated with a memory segment are released when the segment is no longer in use,
 66  * and in a timely fashion. For this reason, there might be cases where waiting for the garbage collector to determine that a segment
 67  * is <a href="../../../java/lang/ref/package.html#reachability">unreachable</a> is not optimal.
 68  * Clients that operate under these assumptions might want to programmatically release the memory associated
 69  * with a memory segment. This can be done, using the {@link jdk.incubator.foreign.ResourceScope} abstraction, as shown below:
 70  *
 71  * <pre>{@code
 72 try (ResourceScope scope = ResourceScope.newConfinedScope()) {
 73     MemorySegment segment = MemorySegment.allocateNative(10 * 4, scope);
 74     for (int i = 0 ; i < 10 ; i++) {
 75         MemoryAccess.setIntAtIndex(segment, i, 42);
 76     }
 77 }
 78  * }</pre>
 79  *
 80  * This example is almost identical to the prior one; this time we first create a so called <em>resource scope</em>,
 81  * which is used to <em>bind</em> the life-cycle of the segment created immediately afterwards. Note the use of the
 82  * <em>try-with-resources</em> construct: this idiom ensures that all the memory resources associated with the segment will be released
 83  * at the end of the block, according to the semantics described in Section {@jls 14.20.3} of <cite>The Java Language Specification</cite>.
 84  *
 85  * <h3><a id="safety"></a>Safety</h3>
 86  *
 87  * This API provides strong safety guarantees when it comes to memory access. First, when dereferencing a memory segment,
 88  * the access coordinates are validated (upon access), to make sure that access does not occur at an address which resides
 89  * <em>outside</em> the boundaries of the memory segment used by the dereference operation. We call this guarantee <em>spatial safety</em>;
 90  * in other words, access to memory segments is bounds-checked, in the same way as array access is, as described in
 91  * Section {@jls 15.10.4} of <cite>The Java Language Specification</cite>.
 92  * <p>
 93  * Since memory segments can be closed (see above), segments are also validated (upon access) to make sure that
 94  * the resource scope associated with the segment being accessed has not been closed prematurely.
 95  * We call this guarantee <em>temporal safety</em>. Together, spatial and temporal safety ensure that each memory access
 96  * operation either succeeds - and accesses a valid memory location - or fails.
 97  *
 98  * <h2>Foreign function access</h2>
 99  * The key abstractions introduced to support foreign function access are {@link jdk.incubator.foreign.SymbolLookup} and {@link jdk.incubator.foreign.CLinker}.
100  * The former is used to lookup symbols inside native libraries; the latter
101  * provides linking capabilities which allow to model foreign functions as {@link java.lang.invoke.MethodHandle} instances,
102  * so that clients can perform foreign function calls directly in Java, without the need for intermediate layers of native
103  * code (as it's the case with the <a href="{@docRoot}/../specs/jni/index.html">Java Native Interface (JNI)</a>).
104  * <p>
105  * For example, to compute the length of a string using the C standard library function {@code strlen} on a Linux x64 platform,
106  * we can use the following code:
107  *
108  * <pre>{@code
109       MethodHandle strlen = CLinker.getInstance().downcallHandle(
110         CLinker.systemLookup().lookup("strlen").get(),
111         MethodType.methodType(long.class, MemoryAddress.class),
112         FunctionDescriptor.of(CLinker.C_LONG, CLinker.C_POINTER)
113       );
114 
115       try (var scope = ResourceScope.newConfinedScope()) {
116          var cString = CLinker.toCString("Hello", scope);
117          long len = (long)strlen.invokeExact(cString.address()); // 5
118       }
119  * }</pre>
120  *
121  * Here, we lookup the {@code strlen} symbol in the {@linkplain jdk.incubator.foreign.CLinker#systemLookup() system lookup}.
122  * Then, we obtain a linker instance (see {@link jdk.incubator.foreign.CLinker#getInstance()}) and we use it to
123  * obtain a method handle which targets the {@code strlen} library symbol. To complete the linking successfully,
124  * we must provide (i) a {@link java.lang.invoke.MethodType} instance, describing the type of the resulting method handle
125  * and (ii) a {@link jdk.incubator.foreign.FunctionDescriptor} instance, describing the signature of the {@code strlen}
126  * function. From this information, the linker will uniquely determine the sequence of steps which will turn
127  * the method handle invocation (here performed using {@link java.lang.invoke.MethodHandle#invokeExact(java.lang.Object...)})
128  * into a foreign function call, according to the rules specified by the platform C ABI. The {@link jdk.incubator.foreign.CLinker}
129  * class also provides many useful methods for interacting with native code, such as converting Java strings into
130  * native strings and viceversa (see {@link jdk.incubator.foreign.CLinker#toCString(java.lang.String, ResourceScope)} and
131  * {@link jdk.incubator.foreign.CLinker#toJavaString(jdk.incubator.foreign.MemorySegment)}, respectively), as
132  * demonstrated in the above example.
133  *
134  * <h3>Foreign addresses</h3>
135  *
136  * When a memory segment is created from Java code, the segment properties (spatial bounds, temporal bounds and confinement)
137  * are fully known at segment creation. But when interacting with native libraries, clients will often receive <em>raw</em> pointers;
138  * such pointers have no spatial bounds (example: does the C type {@code char*} refer to a single {@code char} value,
139  * or an array of {@code char} values, of given size?), no notion of temporal bounds, nor thread-confinement.
140  * <p>
141  * When clients receive a {@link jdk.incubator.foreign.MemoryAddress} instance from a foreign function call, it might be
142  * necessary to obtain a {@link jdk.incubator.foreign.MemorySegment} instance to dereference the memory pointed to by that address.
143  * To do that, clients can proceed in three different ways, described below.
144  * <p>
145  * First, if the memory address is known to belong to a segment the client already owns, a <em>rebase</em> operation can be performed;
146  * in other words, the client can ask the address what its offset relative to a given segment is, and, then, proceed to dereference
147  * the original segment accordingly, as follows:
148  *
149  * <pre>{@code
150 MemorySegment segment = MemorySegment.allocateNative(100, scope);
151 ...
152 MemoryAddress addr = ... //obtain address from native code
153 int x = MemoryAccess.getIntAtOffset(segment, addr.segmentOffset(segment));
154  * }</pre>
155  *
156  * Secondly, if the client does <em>not</em> have a segment which contains a given memory address, it can create one <em>unsafely</em>,
157  * using the {@link jdk.incubator.foreign.MemoryAddress#asSegment(long, ResourceScope)} factory. This allows the client to
158  * inject extra knowledge about spatial bounds which might, for instance, be available in the documentation of the foreign function
159  * which produced the native address. Here is how an unsafe segment can be created from a native address:
160  *
161  * <pre>{@code
162 ResourceScope scope = ... // initialize a resource scope object
163 MemoryAddress addr = ... //obtain address from native code
164 MemorySegment segment = addr.asSegment(4, scope); // segment is 4 bytes long
165 int x = MemoryAccess.getInt(segment);
166  * }</pre>
167  *
168  * Alternatively, the client can fall back to use the so called <em>everything</em> segment - that is, a primordial segment
169  * which covers the entire native heap. This segment can be obtained by calling the {@link jdk.incubator.foreign.MemorySegment#globalNativeSegment()}
170  * method, so that dereference can happen without the need of creating any additional segment instances:
171  *
172  * <pre>{@code
173 MemoryAddress addr = ... //obtain address from native code
174 int x = MemoryAccess.getIntAtOffset(MemorySegment.globalNativeSegment(), addr.toRawLongValue());
175  * }</pre>
176  *
177  * <h3>Upcalls</h3>
178  * The {@link jdk.incubator.foreign.CLinker} interface also allows to turn an existing method handle (which might point
179  * to a Java method) into a native memory address (see {@link jdk.incubator.foreign.MemoryAddress}), so that Java code
180  * can effectively be passed to other foreign functions. For instance, we can write a method that compares two
181  * integer values, as follows:
182  *
183  * <pre>{@code
184 class IntComparator {
185     static int intCompare(MemoryAddress addr1, MemoryAddress addr2) {
186         return MemoryAccess.getIntAtOffset(MemorySegment.globalNativeSegment(), addr1.toRawLongValue()) -
187                MemoryAccess.getIntAtOffset(MemorySegment.globalNativeSegment(), addr2.toRawLongValue());
188     }
189 }
190  * }</pre>
191  *
192  * The above method dereferences two memory addresses containing an integer value, and performs a simple comparison
193  * by returning the difference between such values. We can then obtain a method handle which targets the above static
194  * method, as follows:
195  *
196  * <pre>{@code
197 MethodHandle intCompareHandle = MethodHandles.lookup().findStatic(IntComparator.class,
198                                                    "intCompare",
199                                                    MethodType.methodType(int.class, MemoryAddress.class, MemoryAddress.class));
200  * }</pre>
201  *
202  * Now that we have a method handle instance, we can link it into a fresh native memory address, using the {@link jdk.incubator.foreign.CLinker} interface, as follows:
203  *
204  * <pre>{@code
205 ResourceScope scope = ...
206 MemoryAddress comparFunc = CLinker.getInstance().upcallStub(
207      intCompareHandle,
208      FunctionDescriptor.of(C_INT, C_POINTER, C_POINTER),
209      scope
210 );
211  * }</pre>
212  *
213  * As before, we need to provide a {@link jdk.incubator.foreign.FunctionDescriptor} instance describing the signature
214  * of the function pointer we want to create; as before, this, coupled with the method handle type, uniquely determines the
215  * sequence of steps which will allow foreign code to call {@code intCompareHandle} according to the rules specified
216  * by the platform C ABI. The lifecycle of the memory address returned by
217  * {@link jdk.incubator.foreign.CLinker#upcallStub(java.lang.invoke.MethodHandle, jdk.incubator.foreign.FunctionDescriptor, jdk.incubator.foreign.ResourceScope)}
218  * is tied to the {@linkplain jdk.incubator.foreign.ResourceScope resource scope} parameter passed to that method.
219  *
220  * <a id="restricted"></a>
221  * <h2>Restricted methods</h2>
222  * Some methods in this package are considered <em>restricted</em>. Restricted methods are typically used to bind native
223  * foreign data and/or functions to first-class Java API elements which can then be used directly by clients. For instance
224  * the restricted method {@link jdk.incubator.foreign.MemoryAddress#asSegment(long, ResourceScope)} can be used to create
225  * a fresh segment with given spatial bounds out of a native address.
226  * <p>
227  * Binding foreign data and/or functions is generally unsafe and, if done incorrectly, can result in VM crashes, or memory corruption when the bound Java API element is accessed.
228  * For instance, in the case of {@link jdk.incubator.foreign.MemoryAddress#asSegment(long, ResourceScope)}, if the provided
229  * spatial bounds are incorrect, a client of the segment returned by that method might crash the VM, or corrupt
230  * memory when attempting to dereference said segment. For these reasons, it is crucial for code that calls a restricted method
231  * to never pass arguments that might cause incorrect binding of foreign data and/or functions to a Java API.
232  * <p>
233  * Access to restricted methods is <em>disabled</em> by default; to enable restricted methods, the command line option
234  * {@code --enable-native-access} must mention the name of the caller's module.
235  */
236 package jdk.incubator.foreign;