I am fully aware that I am doing is:
1) Unsafe, that I have no more guarantees, type safety, and that the JVM could therefore crash
2) That I could do similar operations using ByteBuffers or JNI
3) That Unsafe is an internal class and could well disappear.
I am doing this for experimentation purposes only and am aware of the consequences.
With this in mind, I am trying to extract the data from an array using Unsafe and reflection:
I first find the field offset of the array:
public long findFieldOffset(Event event) {
try {
Class cl = event.getClass();
Field data_field = cl.getDeclaredField("data");
data_field.setAccessible(true);
long offset = unsafe.objectFieldOffset(data_field);
return offset;
} catch (NoSuchFieldException e) {
e.printStackTrace();
}
return 0;
}
I also extract the base location of the array:
int base = unsafe.arrayBaseOffset(byte[].class);
I subsequently try to extract the array from the class Event, and copy it to a buffer (byte_offset is the result of findFieldOffset).
In the code below, the first part is just a test function and prints out the correct string, whilst the second part extracts what should be the array, but when testing returns garbage unicode values:
/* Testing */
active_buffer.getBuffer().position(1);
active_buffer.getBuffer().put(event.getData());
active_buffer.getBuffer().position(1);
active_buffer.getBuffer().get(tuple, 0, (int)tuple_size);
System.out.println("Test1: " + new String(tuple)) ;
/* Test1 prints out the correct string */
unsafe.copyMemory( (Object) event, byte_offset + base, (Object) null, active_buffer.getAddress() + 1, tuple_size);
active_buffer.getBuffer().position(1);
active_buffer.getBuffer().get(tuple, 0, (int)tuple_size);
System.out.println("Test2: " + new String(tuple));
/* Garbage unicode values gets printed*/
Can anyone see anything wrong with this code?
As data is a field, it is either a primitive or a reference, in either case you cannot copy it somewhere and treat it as a byte[]
If data is a byte[] you need to treat this as the true object, not the Event.
Related
I have been using the JNA library to get all visible window handles in Windows. I need to do something similar in macOS using JNA.
Here is code to get all window handles in Windows:
public static List<HWND> findAll() {
final List<HWND> windows = new LinkedList<>();
User32.INSTANCE.EnumWindows(new User32.WNDENUMPROC() {
#Override
public boolean callback(HWND hWnd, Pointer arg) {
if (User32.INSTANCE.IsWindowVisible(hWnd)) {
windows.add(hWnd);
}
return true;
}
}, null);
return windows;
}
What is the equivalent code in macOS?
You'll need to map portions of the Core Graphics Framework. You can list windows using the CGWindowListCopyWindowInfo() function.
To load the framework you'll need to map a CoreGraphics interface extending JNA's Library class, and map the function you need:
public interface CoreGraphics extends Library {
CoreGraphics INSTANCE = Native.load("CoreGraphics", CoreGraphics.class);
CFArrayRef CGWindowListCopyWindowInfo(int option, int relativeToWindow);
}
The CFArrayRef type is already mapped in JNA in the CoreFoundation class. Pick the appropriate Window List Option (probably kCGWindowListOptionAll = 0). If you already had a window number you could use relative reerences, otherwise you'll use kCGNullWindowID (0) for the second parameter. Calling it from your code should be simple:
CFArrayRef windowInfo = CoreGraphics.INSTANCE.CGWindowListCopyWindowInfo(0, 0);
That will give you an array of CFDictionaryRef objects representing the windows. You can iterate the array and then use further methods in the CFDictionaryRef class to explore these dictionary objects: you'll create a CFString for the keys. A list of required keys is documented here and optional keys are here. The constant strings match the variable name.
This should get you a CFNumberRef for each window number (the "handle" equivalent):
// Set up keys for dictionary lookup
CFStringRef kCGWindowNumber = CFStringRef.createCFString("kCGWindowNumber");
CFStringRef kCGWindowOwnerPID = CFStringRef.createCFString("kCGWindowOwnerPID");
// Note: the Quartz name is rarely used
CFStringRef kCGWindowName = CFStringRef.createCFString("kCGWindowName");
CFStringRef kCGWindowOwnerName = CFStringRef.createCFString("kCGWindowOwnerName");
// Iterate the array
int numWindows = windowInfo.getCount();
for (int i = 0; i < numWindows; i++) {
// For each array element, get the dictionary
Pointer result = windowInfo.getValueAtIndex(i);
CFDictionaryRef windowRef = new CFDictionaryRef(result);
// Now get information from the dictionary.
// Get a pointer to the result, in this case a CFNumber
result = windowRef.getValue(kCGWindowNumber);
// "Cast" the pointer to the appropriate type
CFNumberRef windowNumber = new CFNumberRef(result);
// CoreFoundation.INSTANCE.CFNumberGetType(windowNumber)
// --> 4 = kCFNumberSInt64Type, signed 64 bit so use getLong()
// Get a pointer to the result, in this case a CFNumber
result = windowRef.getValue(kCGWindowOwnerPID);
// "Cast" the pointer to the appropriate type
CFNumberRef windowOwnerPID = new CFNumberRef(result);
// CoreFoundation.INSTANCE.CFNumberGetType(windowOwnerPID)
// --> 4 = kCFNumberSInt64Type, signed 64 bit so use getLong()
// Get a pointer to the result, in this case a CFString
result = windowRef.getValue(kCGWindowName);
// "Cast" the pointer to the appropriate type
// Optional key, check for null
String windowName = result == null ? "" : new CFStringRef(result).stringValue();
// Get a pointer to the result, in this case a CFString
result = windowRef.getValue(kCGWindowOwnerName);
// "Cast" the pointer to the appropriate type
// Optional key, check for null
String windowOwnerName = result == null ? "" : new CFStringRef(result).stringValue();
// ... look up other keys if needed ...
// use ProcessHandle with the PID to get start time
// Output or add to List, etc.
System.out.println(windowNumber.longValue()
+ " (" + windowOwnerName + ", pid="
+ windowOwnerPID.longValue()
+ "): " + windowName);
}
// CF references from "Copy" or "Create" must be released
// release the created key references
kCGWindowNumber.release();
kCGWindowOwnerPID.release();
kCGWindowName.release();
kCGWindowOwnerName.release();
// release the array
windowInfo.release();
This question already has answers here:
Returning a Renderscript struct from a Renderscript kernel
(2 answers)
Closed 6 years ago.
I have a problem. I would like to retrieve a struct from a renderscript kernel. What I wanted was that I would get an input a struct element... I would modify it and then return it modified. But there is no such a way in the reflected layer. I tryied to manually deserialize the data from the buffer but I am not even able to copy the buffer to a ByteBuffer because the Allocation has validation in the copyTo on a type so I have no idea what am I supposed to do...
RenderScript supports custom elements. To create one, declare a custom typedef struct like the following one, inside a RS script:
typedef struct MyElement {
int x;
int y;
bool simpleBool;
} MyElement_t;
After the build process, a ScriptField_MyElement Java class will appear, mirroring the RS struct. You will be able to use this class to create a custom Allocation that uses your own Element:
// Declares a new Allocation, based upon the custom struct Element
Element myElement = ScriptField_MyElement.createElement(mRS);
Allocation myElementsAllocation = Allocation.createSized(mRS, myElement, 5);
// Or
Allocation myElementsAllocation = ScriptField_MyElement.create1D(mRS, sizeX).getAllocation();
You can find an example of this process inside the CustomElementExample sample project.
Also, inside the SurfaceRenderExample sample project you can see how a custom element can be used to model a mathematical structure (in this case a particle, falling with some acceleration).
Inside RenderScript scripts:
To get a custom element from an allocation:
MyElement_t el = * (MyElement_t *) rsGetElementAt(aIn, index);
To change a custom element member:
el.x = 10;
To set a custom element in an allocation:
rsSetElementAt(myAlloc, (void *)&el);
Reference: RenderScript: parallel computing on Android, the easy way
Edit:
For now, there is no direct way to copy a custom struct element to the Java side.
The CustomStructElementCopyToJava sample project provides an example of the process.
Short explanation of the example
Note: the following process is EXPERIMENTAL and not performant at all! If you plan to heavily use this process, please use the Android NDK to access the allocation.
Also, in future versions of the Android SDK, this code may break because it relies on Java reflection; some normally hidden methods can change without any notice in the Android SDK.
Let's assume using the following custom struct element:
typedef struct Point {
int x;
int y;
} Point_t;
When looking at the generated code of the struct (which can be seen, in Android Studio, by pressing CTRL+B while focusing on a ScriptField_Point element on the Java side), the following elements can be seen:
public static Element createElement(RenderScript rs) {
Element.Builder eb = new Element.Builder(rs);
eb.add(Element.I32(rs), "x");
eb.add(Element.I32(rs), "y");
return eb.create();
}
You can map the contents of the custom struct in a hacky way:
1) Define the destination byte array:
byte destinationArray[] = new byte[allocationGrayPointOrdered.getBytesSize()];
2) Use Java reflection to access the hidden Allocation.copyTo method:
private static Method getCopyToWithoutValidationMethod(){
// private void copyTo(Object array, Element.DataType dt, int arrayLen)
Method allocationHiddenCopyToMethod = null;
try {
allocationHiddenCopyToMethod = Allocation.class.getDeclaredMethod("copyTo", Object.class, Element.DataType.class, int.class);
allocationHiddenCopyToMethod.setAccessible(true);
} catch (NoSuchMethodException e) {
throw new RuntimeException("Could not find allocationHiddenCopyToMethod");
}
return allocationHiddenCopyToMethod;
}
3) Perform the copy:
// Gets reflected method
Method copyToWithoutValidationMethod = getCopyToWithoutValidationMethod();
// Tries to copy contents
try {
copyToWithoutValidationMethod.invoke(allocationGrayPointOrdered, destinationArray,
Element.DataType.UNSIGNED_8, destinationArray.length);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
} catch (InvocationTargetException e) {
throw new RuntimeException(e);
}
4) Once the array is filled with source data, it is then possible to map its content to a human-readable struct.
// Defines the destination array
ScriptField_Point.Item mappedItems[][] = new ScriptField_Point.Item[sizeX][sizeY];
// Wraps array contents
ByteBuffer byteBuffer = ByteBuffer.wrap(destinationArray);
// Sets byte order to be Android-like
byteBuffer.order(ByteOrder.LITTLE_ENDIAN);
// Iterates on every column and row
for (int x = 0; x < sizeX; x++) {
for (int y = 0; y < sizeY; y++) {
// Allocates a new item
ScriptField_Point.Item currentItem = new ScriptField_Point.Item();
// Calculate the offset in the source array
int currentOffset = (x + y * sizeX) * ScriptField_Point.Item.sizeof;
// Gets data from the byte array
currentItem.x = byteBuffer.getInt(currentOffset);
currentItem.y = byteBuffer.getInt(currentOffset + 4);
mappedItems[x][y] = currentItem;
}
}
For the complete explanation, please refer to the book.
Given an object byte[], when we want to operate with such object often we need pieces of it. In my particular example i get byte[] from wire where first 4 bytes describe lenght of the message then another 4 bytes the type of the message (an integer that maps to concrete protobuf class) then remaining byte[] is actual content of the message... like this
length|type|content
in order to parse this message i have to pass content part to specific class which knows how to parse an instance from it... the problem is that often there are no methods provided so that you could specify from where to where parser shall read the array...
So what we end up doing is copying remaining chuks of that array, which is not effective...
As far as i know in java it is not possible to create another byte[] reference that actually refers to some original bigger byte[] array with just 2 indexes (this was approach with String that led to memory leaks)...
I wonder how do we solve situations like this? I suppose giving up on protobuf just because it does not provide some parseFrom(byte[], int, int) does not make sence... protobuf is just an example, anything could lack that api...
So does this force us to write inefficient code or there is something that can be done? (appart from adding that method)...
Normally you would tackle this kind of thing with streams.
A stream is an abstraction for reading just what you need to process the current block of data. So you can read the correct number of bytes into a byte array and pass it to your parse function.
You ask 'So does this force us to write inefficient code or there is something that can be done?'
Usually you get your data in the form of a stream and then using the technique demonstrated below will be more performant because you skip making one copy. (Two copies instead of three; once by the OS and once by you. You skip making a copy of the total byte array before you start parsing.) If you actually start out with a byte[] but it is constructed by yourself then you may want to change to constructing an object such as { int length, int type, byte[] contentBytes } instead and pass contentBytes to your parse function.
If you really, really have to start out with byte[] then the below technique is just a more convenient way to parse it, it would not be more performant.
So suppose you got a buffer of bytes from somewhere and you want to read the contents of that buffer. First you convert it to a stream:
private static List<Content> read(byte[] buffer) {
try {
ByteArrayInputStream bytesStream = new ByteArrayInputStream(buffer);
return read(bytesStream);
} catch (IOException e) {
e.printStackTrace();
}
}
The above function wraps the byte array with a stream and passes it to the function that does the actual reading.
If you can start out from a stream then obviously you can skip the above step and just pass that stream into the below function directly:
private static List<Content> read(InputStream bytesStream) throws IOException {
List<Content> results = new ArrayList<Content>();
try {
// read the content...
Content content1 = readContent(bytesStream);
results.add(content1);
// I don't know if there's more than one content block but assuming
// that there is, you can just continue reading the stream...
//
// If it's a fixed number of content blocks then just read them one
// after the other... Otherwise make this a loop
Content content2 = readContent(bytesStream);
results.add(content2);
} finally {
bytesStream.close();
}
return results;
}
Since your byte-array contains content you will want to read Content blocks from the stream. Since you have a length and a type field, I am assuming that you have different kinds of content blocks. The next function reads the length and type and passes the processing of the content bytes on to the proper class depending on the read type:
private static Content readContent(InputStream stream) throws IOException {
final int CONTENT_TYPE_A = 10;
final int CONTENT_TYPE_B = 11;
// wrap the InputStream in a DataInputStream because the latter has
// convenience functions to convert bytes to integers, etc.
// Note that DataInputStream handles the stream in a BigEndian way,
// so check that your bytes are in the same byte order. If not you'll
// have to find another stream reader that can convert to ints from
// LittleEndian byte order.
DataInputStream data = new DataInputStream(stream);
int length = data.readInt();
int type = data.readInt();
// I'm assuming that above length field was the number of bytes for the
// content. So, read length number of bytes into a buffer and pass that
// to your `parseFrom(byte[])` function
byte[] contentBytes = new byte[length];
int readCount = data.read(contentBytes, 0, contentBytes.length);
if (readCount < contentBytes.length)
throw new IOException("Unexpected end of stream");
switch (type) {
case CONTENT_TYPE_A:
return ContentTypeA.parseFrom(contentBytes);
case CONTENT_TYPE_B:
return ContentTypeB.parseFrom(contentBytes);
default:
throw new UnsupportedOperationException();
}
}
I have made up the below Content classes. I don't know what protobuf is but it can apparently convert from a byte array to an actual object with its parseFrom(byte[]) function, so take this as pseudocode:
class Content {
// common functionality
}
class ContentTypeA extends Content {
public static ContentTypeA parseFrom(byte[] contentBytes) {
return null; // do the actual parsing of a type A content
}
}
class ContentTypeB extends Content {
public static ContentTypeB parseFrom(byte[] contentBytes) {
return null; // do the actual parsing of a type B content
}
}
In Java, Array is not just section of memory - it is an object, that have some additional fields (at least - length). So you cannot link to part of array - you should:
Use array-copy functions or
Implement and use some algorithm that uses only part of byte array.
The concern seems that there is no way to create a view over an array (e.g., an array equivalent of List#subList()). A workaround might be making your parsing methods take in the reference to the entire array and two indices (or an index and a length) to specify the sub-array the method should work on.
This would not prevent the methods from reading or modifying sections of the array they should not touch. Perhaps an ByteArrayView class could be made to add a little bit of safety if this is a concern:
public class ByteArrayView {
private final byte[] array;
private final int start;
private final int length;
public ByteArrayView(byte[] array, int start, int length) { ... }
public byte[] get(int index) {
if (index < 0 || index >= length) {
throw new ArrayOutOfBoundsExceptionOrSomeOtherRelevantException();
}
return array[start + index];
}
}
But if, on the other hand, performance is a concern, then a method call to get() for fetching each byte is probably undesirable.
The code is for illustration; it's not tested or anything.
EDIT
On a second reading of my own answer, I realized that I should point this out: having a ByteArrayView will copy each byte you read from the original array -- just byte by byte rather than as a chunk. It would be inadequate for the OP's concerns.
I'm creating a StringBuilder to collect strings that I periodically flush to a server. If the flush fails, I want to keep the strings to try again next time, although in the mean time I might get additional strings to send which must be added to the StringBuilder.
What I want to know is what the most efficient way to do this would be, as this is being done in an Android app where battery usage and thus CPU usage is a big concern. Does calling StringBuilder's toString() function store the resulting string it returns internally so that a subsequent call doesn't have to do the work of copying all the original strings over? Or if the call fails, should I create a new StringBuilder initialized with the return value from toString()?
Here is the OpenJDK source code for StringBuilder:
public String toString() {
// Create a copy, don't share the array
return new String(value, 0, count);
}
The source for the String constructor with those parameters is:
public String(char value[], int offset, int count) {
if (offset < 0) {
throw new StringIndexOutOfBoundsException(offset);
}
if (count < 0) {
throw new StringIndexOutOfBoundsException(count);
}
// Note: offset or count might be near -1>>>1.
if (offset > value.length - count) {
throw new StringIndexOutOfBoundsException(offset + count);
}
this.offset = 0;
this.count = count;
this.value = Arrays.copyOfRange(value, offset, offset+count);
}
So yes, it does create a new String everytime, and yes, it makes a copy of the char[] everytime.
It's important to note that this is one implementation of toString, and another implementation may obviously be different.
It would be an implementation detail. Since java strings are immutable a correct impl can choose to share or create new strings from StringBuilder.toString() even if it's not needed.
As everyone says, you can test to see if this is indeed a real performance issue for you. If it is one (clunky) workaround is to wrap StringBuilder and cache the resulting string. You can use a dirty flag to indicate the content was modified.
StringBuilder.toString API says that a new String object is allocated and initialized to contain the character sequence currently represented by this object.
I can't seem to find the answer anywhere, I'm trying to obtain a socket in Java, and hand over its file descriptor number so that I can use it in a C binary (the fd would be as argument).
I've obtained the FileDescriptor using reflection... but can't access the actual number anywhere.
I know other people have suggested JNI, but I'd like to keep it within Java if possible (and couldn't fully figure out how to do it)
In Java 7, you can cast a SocketInputStream to a FileInputStream, and call getFD() to get the FileDescriptor object.
Then you can use reflection to access the FileDescriptor object's private int fd field. (You use the Class.getDeclaredField(...) method to get the Field, call Field.setAccessible(true), and then get the field's value using Field.getInt(...).)
Beware that you may be making your code platform dependent by doing this. There are no guarantees that the particular private field will be present in older ... or forth-coming versions of Java, or in implementations of Java done by other vendors / suppliers.
Stephen C's answer addresses how to get a FileDescriptor, but here's a method to the file descriptor number from that object. On Windows, FileDescriptor uses long handle instead of int fd internally, so this method first checks if handle is used and returns that if so, otherwise it falls back to returning fd. I haven't tested this with sockets as OP is using, but I imagine Windows JVMs still use handle.
public static long fileno(FileDescriptor fd) throws IOException {
try {
if (fd.valid()) {
// windows builds use long handle
long fileno = getFileDescriptorField(fd, "handle", false);
if (fileno != -1) {
return fileno;
}
// unix builds use int fd
return getFileDescriptorField(fd, "fd", true);
}
} catch (IllegalAccessException e) {
throw new IOException("unable to access handle/fd fields in FileDescriptor", e);
} catch (NoSuchFieldException e) {
throw new IOException("FileDescriptor in this JVM lacks handle/fd fields", e);
}
return -1;
}
private static long getFileDescriptorField(FileDescriptor fd, String fieldName, boolean isInt) throws NoSuchFieldException, IllegalAccessException {
Field field = FileDescriptor.class.getDeclaredField(fieldName);
field.setAccessible(true);
long value = isInt ? field.getInt(fd) : field.getLong(fd);
field.setAccessible(false);
return value;
}