JNA getting back a byte array - java

I have a library developed in C that successfully decompresses an LZMA encoded file. The signature for the file is a pointer to the data, the compressed, decompressed, and an 'out' field for the error code. The return value is a pointer to the array (or null if it fails).
The function in C looks similar to this:
char* decompressLZMA(const char *lzmaData, int compressedSize, int uncompressedSize, t_status_codes *statusCode);
I've tried using pointers and memory from other examples but they are not working.
How do I properly pass a byte array and pointer in to get data back?
This is my interface:
public interface LZMALibrary extends Library {
Memory lzma_uncompress(Memory lzma_data, int compressed_size, int uncompressed_size, Pointer status_code);
}

It appears that I wanted to make a pointer of the 'Memory' class instead. The solution that is working for me right now is to create Pointer objects, and then the library will fill up the pointers, and I get them back and handle it appropriately.
My interface turned to:
public interface LZMALibrary extends Library {
Pointer lzma_uncompress(Pointer lzma_data, int compressed_size, int uncompressed_size, Pointer status_code);
}
From there I am able to write in the data:
Pointer ptr = new Memory(lzmaData.length);
ptr.write(0, lzmaData, 0, lzmaData.length);
I also need the pointer that will be written to:
Pointer errorStatus = new Memory(4);
Then I can call the function to get a pointer back, and read that pointer if it's not null:
Pointer p = lzmaLib.lzma_uncompress(ptr, lzmaData.length, DECOMPRESSED_LENGTH, errorStatus); // DECOMPRESSED_LENGTH is a constant.
if (p != null) {
byte[] decompressedData = p.getByteArray(0, DECOMPRESSED_LENGTH);
System.out.println(new String(decompressedData, "ASCII")); // My data is ASCII text.
}
if (errorStatus != null) {
int errorStatusCode = errorStatus.getInt(0);
System.out.println("Error code: " + errorStatusCode);
}
This appears to have solved my problem. I am very new to JNA so hopefully I am not missing anything.
If there's any possible errors I might run into, please feel free to post on how to correct it.

Related

How to send a string from c++ app to a message only window in java (LPARAM to String conversion)

I am trying to send a String to a message-only window that I have created in java using JNA.
I am using SendMessage to send a message. (I know that it is not recommended to use FindWindow every time I send a message, but this is just for this example here)
SendMessage(FindWindow(NULL,"jnaWnd"), WM_USER + 10, 0, (LPARAM)L"Test");
On the Java (JNA) side, I had to use this interface for a proper WindowProc implementation
Besides that, I also made my way through creating a message-only hwnd on the java side. But this is not relevant for this problem.
Anyway, this is the implementation of the interface.
#Override
public LRESULT callback(HWND hwnd, int uMsg, WPARAM wParam, LPARAM lParam) {
switch (uMsg) {
case 1034: { //1034 is WM_USER + 10
//These lines doesn't work.... :(
//java.lang.UnsupportedOperationException: This pointer is opaque
String s = lParam.toPointer().getString(0);
// java.lang.Error: Invalid memory access
String s = lParam.toPointer().getWideString(0);
return new LRESULT(0);
}
default:
return JNA.USER32.INSTANCE.DefWindowProc(hwnd, uMsg, wParam, lParam);
}
}
Everything works fine, and I do receive the messages.
Working with Integers works fine, and conversion is also easy.
But I have no clue how to parse the String out of the LPARAM value.
JNA is very well known for converting types on the fly by using the java equivalent type in the methods.
The first problem is that I use this interface, and therefore I need to stick to the params defined in the interface.
The second problem is that I also want to work with Integer values as well, which means that I cannot just replace LPARAM lParam with String lParam
Even tho I could try to dig deeper on the JNA side to create my own interface and such...
I just wanted to ask here before I move forward on this topic.
Maybe there is a simple way to get the String value out of the LPARAM
Any tips are appreciated.
EDIT_1:
I tried to use the WM_COPYDATA method, but this still leads to an invalid memory access exception...
This is what I have tried so far:
I defined a new struct named STRMSG struct on both sides;
Java:
public class STRMSG extends Structure{
public STRMSG() {
super();
}
public STRMSG(Pointer p) {
super(p);
read();
}
public String message;
protected List<String> getFieldOrder() {
return Arrays.asList("message");
}
}
C++:
typedef struct STRMSG {
LPCSTR message;
};
And here is the new SendMessage procedure:
STRMSG msg;
msg.message = "Some message";
COPYDATASTRUCT cds;
cds.dwData = 10;
cds.cbData = sizeof(STRMSG);
cds.lpData = &msg;
SendMessage(FindWindow(NULL, L"jnaWnd"), WM_COPYDATA, 0, (LPARAM)&cds);
I also added the following case in my callback method on the java side:
case WinUser.WM_COPYDATA: {
COPYDATASTRUCT cds = new COPYDATASTRUCT(new Pointer(lParam.longValue()));
ULONG_PTR uMsg1 = cds.dwData;
Pointer lParam1 = cds.lpData;
int wParam1 = cds.cbData;
switch(uMsg1.intValue()){
case 10: {
STRMSG msg = new STRMSG(lParam1); // Error: Invalid memory access here
Logger.debug(msg.message);
}
}
return new LRESULT(0);
}
The invalid memory error happens anyway :(...
If the message-only window exists in a different process, then you can't send a string via a raw pointer across process boundaries in the manner you are trying. The sent pointer is only valid in the context of the sending process, it will not point at valid memory which exists in the context of the target process. So, you must marshal the string data instead (ie, actually copy the characters into the memory of the target process), such as with WM_COPYDATA (or any other IPC mechanism, such as sockets, pipes, ActiveX/COM, etc). You can't just send a raw pointer, as you are trying to do (unless you use VirtualAllocEx() to allocate the pointed-at memory in the target process, so the raw pointer is valid in that process. But don't go this way in this situation, there are better ways).
In your EDIT, your WM_COPYDATA is suffering from the exact same problem as your orignal code. Your sender is still sending a raw pointer as-is, just wrapped inside of a STRMSG struct, and then your receiver is trying to use the received pointer to access memory which does not exist in the receiving process. You didn't marshal the string data at all. Get rid of STRMSG and point the COPYDATASTRUCT directly at the string data so WM_COPYDATA will copy the actual characters.
Also, Pointer.getString() likely won't work in this situation, as the string being sent is a wchar_t string, so you should use Pointer.getWideString() instead.
With all of that said, try something more like this instead:
const UINT myStringId = RegisterWindowMessage(L"MyStringDataID");
...
HWND hwnd = FindWindow(NULL, L"jnaWnd");
if ((hwnd != NULL) && (myStringId != 0))
{
std::wstring msg = L"Test";
COPYDATASTRUCT cds;
cds.dwData = myStringId;
cds.cbData = (msg.size() + 1) * sizeof(wchar_t);
cds.lpData = const_cast<wchar_t*>(msg.c_str());
SendMessage(hwnd, WM_COPYDATA, 0, reinterpret_cast<LPARAM>(&cds));
}
final int myStringId = JNA.USER32.INSTANCE.RegisterWindowMessage("MyStringDataID");
...
#Override
public LRESULT callback(HWND hwnd, int uMsg, WPARAM wParam, LPARAM lParam) {
switch (uMsg) {
case WinUser.WM_COPYDATA: {
COPYDATASTRUCT cds = new COPYDATASTRUCT(new Pointer(lParam.longValue()));
if ((myStringId != 0) && (cds.dwData.intValue() == myStringId)) {
String s = cds.lpData.getWideString(0);
Logger.debug(s);
return new LRESULT(0);
}
break;
}
}
return JNA.USER32.INSTANCE.DefWindowProc(hwnd, uMsg, wParam, lParam);
}

JNA - How to turn `char ***devices` PointerByReferences to `char **devices` PointerByReference

Here is the c header file:
idevice_error_t idevice_get_device_list(char ***devices, int *count);
idevice_error_t idevice_device_list_free(char **devices);
Here is the JNA that JNAerator generated for me:
int idevice_get_device_list(PointerByReference devices, IntBuffer count);
int idevice_device_list_free(PointerByReference devices);
Here is how I'm using it. It's in Kotlin, but it's equivalent to what you'd do in Java too:
fun getDeviceList(): List<String> {
val deviceUuidsPbr = PointerByReference()
val deviceSizeBuffer = IntBuffer.wrap(IntArray(1))
val resultInt = LibIMobileDeviceLibrary.INSTANCE.idevice_get_device_list(deviceUuidsPbr, deviceSizeBuffer)
val size = deviceSizeBuffer.get()
logger.v {"getDeviceList $resultInt, Size: $size" }
val stringArrayP = deviceUuidsPbr.value
val devices = stringArrayP
.getStringArray(0, size)
.toList()
logger.v { "Devices: $devices" }
LibIMobileDeviceLibrary.INSTANCE.idevice_device_list_free(deviceUuidsPbr)
return devices
}
Everything blows up when I free memory:
LibIMobileDeviceLibrary.INSTANCE.idevice_device_list_free(deviceUuidsPbr)
The free wants a char **devices, but I'm submitting a char ***devices. How should I get the PointerByReference into the correct format?
A PointerByReference is just a pointer (to a pointer). And the two C functions being mapped simply have an extra level of indirection.
Consider this function definition:
device_get_device_list(char ***devices, int *count);
idevice_device_list_free(char **devices);
now consider foo defined to be *devices and the original functions simplify to:
device_get_device_list(char **foo, int *count);
idevice_device_list_free(char *foo);
And if you define bar as *foo you get:
device_get_device_list(char *bar, int *count);
idevice_device_list_free(char bar);
So you cannot pass the PointerByReference you are receiving from device_get_device_list (*bar) directly to idevice_device_list_free (which needs bar); you pass its pointed-to value (which happens to be a pointer to another pointer, but that doesn't matter.)
In Java you would simply change the argument in your idevice_device_list_free() call from deviceUuidsPbr to deviceUuidsPbr.getValue().
I am not a Kotlin user but it would seem based on your other code that you need deviceUuidsPbr.value there.

How to retrieve a struct from Renderscript kernel [duplicate]

This question already has answers here:
Returning a Renderscript struct from a Renderscript kernel
(2 answers)
Closed 6 years ago.
I have a problem. I would like to retrieve a struct from a renderscript kernel. What I wanted was that I would get an input a struct element... I would modify it and then return it modified. But there is no such a way in the reflected layer. I tryied to manually deserialize the data from the buffer but I am not even able to copy the buffer to a ByteBuffer because the Allocation has validation in the copyTo on a type so I have no idea what am I supposed to do...
RenderScript supports custom elements. To create one, declare a custom typedef struct like the following one, inside a RS script:
typedef struct MyElement {
int x;
int y;
bool simpleBool;
} MyElement_t;
After the build process, a ScriptField_MyElement Java class will appear, mirroring the RS struct. You will be able to use this class to create a custom Allocation that uses your own Element:
// Declares a new Allocation, based upon the custom struct Element
Element myElement = ScriptField_MyElement.createElement(mRS);
Allocation myElementsAllocation = Allocation.createSized(mRS, myElement, 5);
// Or
Allocation myElementsAllocation = ScriptField_MyElement.create1D(mRS, sizeX).getAllocation();
You can find an example of this process inside the CustomElementExample sample project.
Also, inside the SurfaceRenderExample sample project you can see how a custom element can be used to model a mathematical structure (in this case a particle, falling with some acceleration).
Inside RenderScript scripts:
To get a custom element from an allocation:
MyElement_t el = * (MyElement_t *) rsGetElementAt(aIn, index);
To change a custom element member:
el.x = 10;
To set a custom element in an allocation:
rsSetElementAt(myAlloc, (void *)&el);
Reference: RenderScript: parallel computing on Android, the easy way
Edit:
For now, there is no direct way to copy a custom struct element to the Java side.
The CustomStructElementCopyToJava sample project provides an example of the process.
Short explanation of the example
Note: the following process is EXPERIMENTAL and not performant at all! If you plan to heavily use this process, please use the Android NDK to access the allocation.
Also, in future versions of the Android SDK, this code may break because it relies on Java reflection; some normally hidden methods can change without any notice in the Android SDK.
Let's assume using the following custom struct element:
typedef struct Point {
int x;
int y;
} Point_t;
When looking at the generated code of the struct (which can be seen, in Android Studio, by pressing CTRL+B while focusing on a ScriptField_Point element on the Java side), the following elements can be seen:
public static Element createElement(RenderScript rs) {
Element.Builder eb = new Element.Builder(rs);
eb.add(Element.I32(rs), "x");
eb.add(Element.I32(rs), "y");
return eb.create();
}
You can map the contents of the custom struct in a hacky way:
1) Define the destination byte array:
byte destinationArray[] = new byte[allocationGrayPointOrdered.getBytesSize()];
2) Use Java reflection to access the hidden Allocation.copyTo method:
private static Method getCopyToWithoutValidationMethod(){
// private void copyTo(Object array, Element.DataType dt, int arrayLen)
Method allocationHiddenCopyToMethod = null;
try {
allocationHiddenCopyToMethod = Allocation.class.getDeclaredMethod("copyTo", Object.class, Element.DataType.class, int.class);
allocationHiddenCopyToMethod.setAccessible(true);
} catch (NoSuchMethodException e) {
throw new RuntimeException("Could not find allocationHiddenCopyToMethod");
}
return allocationHiddenCopyToMethod;
}
3) Perform the copy:
// Gets reflected method
Method copyToWithoutValidationMethod = getCopyToWithoutValidationMethod();
// Tries to copy contents
try {
copyToWithoutValidationMethod.invoke(allocationGrayPointOrdered, destinationArray,
Element.DataType.UNSIGNED_8, destinationArray.length);
} catch (IllegalAccessException e) {
throw new RuntimeException(e);
} catch (InvocationTargetException e) {
throw new RuntimeException(e);
}
4) Once the array is filled with source data, it is then possible to map its content to a human-readable struct.
// Defines the destination array
ScriptField_Point.Item mappedItems[][] = new ScriptField_Point.Item[sizeX][sizeY];
// Wraps array contents
ByteBuffer byteBuffer = ByteBuffer.wrap(destinationArray);
// Sets byte order to be Android-like
byteBuffer.order(ByteOrder.LITTLE_ENDIAN);
// Iterates on every column and row
for (int x = 0; x < sizeX; x++) {
for (int y = 0; y < sizeY; y++) {
// Allocates a new item
ScriptField_Point.Item currentItem = new ScriptField_Point.Item();
// Calculate the offset in the source array
int currentOffset = (x + y * sizeX) * ScriptField_Point.Item.sizeof;
// Gets data from the byte array
currentItem.x = byteBuffer.getInt(currentOffset);
currentItem.y = byteBuffer.getInt(currentOffset + 4);
mappedItems[x][y] = currentItem;
}
}
For the complete explanation, please refer to the book.

JNA: Set struct pointer to NULL

I am starting to use JNA to communicate with a device on the RS485 interface of a computer. Suprisingly to me I came to good results very quickly. But now I am stuck by a simple problem. The library I use accepts a pointer to a pointer of struct. The actual signature is
func(Struct1 **, Struct2 **, Struct3 *, Struct4 *, long)
Now to indicate the size of the first parameter the library expects the last pointer to be a NULL pointer. This is what fails. Following code is what I tried so far:
Struct1.ByReference[] s = (Struct1.ByReference[]) new Struct1.ByReference().toArray(size);
int pos = 0;
// ...
// for loop to set the s[pos] struture values
for(pos = 0; pos < size - 1; pos++)
// ...
// Now set the last array element to a null pointer to indicate end-of-list
s[pos].getPointer().setPointer(0, null);// Following does not work: results in zero memoried structure
s[pos] = null; // Following does not work wither: NullPointerException at com.sun.jna.Structure.autoWrite
EDIT 1
s[pos] = new Struct1.ByReference(Pointer.NULL); // results in zero memoried structure as well
EDIT 2
According to technomage's question. If I were to write C code it would probably look something like that:
Struct1 **s = malloc(n * sizeof(Struct1*));
for(int i=0; i<n; i++)
{
if(i == n -1)
{
s[i] = NULL;
}
else
{
s[i] = malloc(sizeof(Struct1));
s[i].bla = value;
....
}
}
But be warned: I am not very skilled in C/C++. I consider Java to be my domain.
Has anyone had a similar problem? Maybe I am just not seeing the wood for the trees...
Thanks in advance.
Structures in JNA are pointers, so what you really need here is a pointer to a (pointer to a) Structure, which is a PointerByReference -- in your case, an array of them.
Given the code example above, you'll create your array of Structures, one less than n:
Struct1[] struct1Array = new Struct1[n-1];
This only allocates the Java memory for the array.
Next you'll instantiate and write the changes you make to native memory:
for (int i = 0; i < n-1; i++) {
struct1Array[i] = new Struct1();
struct1Array[i].bla = value;
struct1Array[i].write();
}
The new Struct1() allocates native side memory for these structures. It's possible to use the Structure.toArray() method to do this as well; I'm intentionally doing this a bit more manual and low-level to try to make clear what's happening.
Then you'll create a corresponding PointerByReference array to hold the pointers to these structures. You'll add an extra element for the null:
PointerByReference[] pbrArray = new PointerByReference[n];
Again, this is only java-side allocation. And then you fill it with pointers to the pointers to the structure, obtained from the Structure.getPointer() method:
for (int i = 0; i < n-1; i++) {
pbrArray[i] = new PointerByReference(struct1Array[i].getPointer());
}
pbrArray[n - 1] = new PointerByReference(Pointer.NULL);
The new PointerByReference() here allocates the native side memory for the pointer itself, which points to the native-side structure you allocated earlier.
From how I understand your initial question, you will pass this PointerByReference array to your function, which presumably updates your structures.
Since you created the two arrays in this fashion, you can keep track of their correspondence by array index. You may have to iterate through the structure array and read() the native memory into the Java-side structure to do further processing with it. Typically when you work directly with Structures being passed to methods they autowrite and autoread, but when using a PointerByReference to indirectly reference the Structure, JNA isn't as friendly.
As an alternative to tracking the two arrays by corresponding indices, you could "forget" the initial Structure assignment and recover it later using the PointerByReference.getValue() method on your array to recover a pointer to the memory for the structure, and then instantiate a new structure using that Pointer in its constructor (e.g. new Struct1(pbr.getValue()) which calls super() with that pointer).

Getting text data from C++ using JNI through std::ostream into Java

I have a class in C++ which takes an std::ostream as an argument in order to continuously output text (trace information). I need to get this text over to the Java side as efficiently as possible. What's the best way to do this? I was thinking of using a direct buffer, but another method would be to take all the function calls across to Java and do all the processing there, but it seems that I'd need a lot of JNI calls.
If an example could be shown of the exact implementation method, it would be very helpful, or if some code exists already to do this (perhaps part of another project). Another help would be to connect it up directly to a standard Java streaming construct, such that the entire implementation was completely transparent to the developer.
(Edit: I found Sharing output streams through a JNI interface which seems to be a duplicate, but not really of much help -- he didn't seem to find the answer he was looking for)
The std::ostream class requires a std::streambuf object for its output. This is used by the fstream and stringstream classes, which use the features of ostream by providing a custom implementation of the streambuf class.
So you can write your own std::streambuf implementation with an overwritten overflow method, buffer the incomming chars in an internal stringbuffer. Every x calls or on eof/newline generate an java-string and call the print method of your java PrintStream.
An incomplete example class:
class JavaStreamBuff : std::streambuf
{
std::stringstream buff;
int size;
jobject handle;
JNIEnv* env
//Ctor takes env pointer for the working thread and java.io.PrintStream
JavaStreamBuff(JNIEnv* env, jobject jobject printStream, int buffsize = 50)
{
handle = env->NewGlobalRef(printStream);
this->env = env;
this->size = size;
}
//This method is the central output of the streambuf class, every charakter goes here
int overflow(int in)
{
if(in == eof || buff.size() == size)
{
std::string blub = buff.str();
jstring do = //magic here, convert form current locale unicode then to java string
jMethodId id = env->(env->GetObjectClass(handle),"print","(java.lang.String)V");
env->callVoidMethod(id,handle,do);
buff.str("");
}
else
{buff<<in;}
}
virtual ~JavaStreamBuff()
{
env->DeleteGlobalRef(handle);
}
}
Missing:
Multithread support (the env pointer is only valid for the jvm thread)
Error handling (checking for java exceptions thrown)
Testing(written within the last 70 min)
Native java method to set the printstream.
On the java side you need a class to convert the PrintStream to a BufferedReader.
There have to be some bugs there, haven't spend enough time to work on them.
The class requires all access to be from the thread it was created in.
Hope this helps
Note
I got it to work with visual studio but I can't get it to work with g++, will try to debug that later.
Edit
Seems that I should have looked for a more official tutorial on this bevore posting my answer, the MSDN page on this topic derives the stringbuffer in a different way.
Sorry for posting this without testing it better :-(.
A small correction to the code above in a more or less unrelated point: Just implement InputStream with a custom class and push byte[] arrays instead of Strings from c++.
The InputStream has a small interface and a BufferedReader should do most of the work.
Last update on this one, since im unable to get it to work on linux, even with the comments on the std::streambuf class stating that only overflow has to be overwritten.
This implementation pushes the raw strings into an inputstream, which can be read from by an other thread. Since I am too stupid to get the debugger working its untested, again.
//The c++ class
class JavaStreamBuf :public std::streambuf
{
std::vector<char> buff;
unsigned int size;
jobject handle;
JNIEnv* env;
public:
//Ctor takes env pointer for the working thread and java.io.PrintStream
JavaStreamBuf(JNIEnv* env, jobject cppstream, unsigned int buffsize = 50)
{
handle = env->NewGlobalRef(cppstream);
this->env = env;
this->size = size;
this->setbuf(0,0);
}
//This method is the central output of the streambuf class, every charakter goes here
virtual int_type overflow(int_type in = traits_type::eof()){
if(in == std::ios::traits_type::eof() || buff.size() == size)
{
this->std::streambuf::overflow(in);
if(in != EOF)
buff.push_back(in);
jbyteArray o = env->NewByteArray(buff.size());
env->SetByteArrayRegion(o,0,buff.size(),(jbyte*)&buff[0]);
jmethodID id = env->GetMethodID(env->GetObjectClass(handle),"push","([B)V");
env->CallVoidMethod(handle,id,o);
if(in == EOF)
env->CallVoidMethod(handle,id,NULL);
buff.clear();
}
else
{
buff.push_back(in);
}
return in;
}
virtual ~JavaStreamBuf()
{
overflow();
env->DeleteGlobalRef(handle);
}
//The java class
/**
*
*/
package jx;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InterruptedIOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
/**
* #author josefx
*
*/
public class CPPStream extends InputStream {
List<Byte> data = new ArrayList<Byte>();
int off = 0;
private boolean endflag = false;
public void push(byte[] d)
{
synchronized(data)
{
if(d == null)
{
this.endflag = true;
}
else
{
for(int i = 0; i < d.length;++i)
{
data.add(d[i]);
}
}
}
}
#Override
public int read() throws IOException
{
synchronized(data)
{
while(data.isEmpty()&&!endflag)
{
try {
data.wait();
} catch (InterruptedException e) {
throw new InterruptedIOException();
}
}
}
if(endflag)return -1;
else return data.remove(0);
}
}
Sorry for wasting so much space^^(and time :-().
It sounds as though the deliverable here is a subclass of ostream. The immediate question I'd want to be clear about is, will this class be responsible for buffering data until Java calls into it to retrieve, or is it expected to immediately (synchronously?) call via JNI to pass it on? That will be the strongest guide to how the code will shape up.
If you can reasonably expect the text to appear as a series of lines, I'd think about presenting them to Java in one line per call: this seems a fair compromise between the number of JNI calls and not unduly delaying the passing on of the text.
On the Java side I think you're looking at creating a Reader so that clients can pick up the text via a familiar interface, or perhaps a subclass of BufferedReader.

Categories

Resources