I am using JNA to call a function from a C DLL :
extern _declspec( dllexport )
int ReadCP(IN OUT unsigned char* Id, IN OUT unsigned int* Size);
In Java I am using an interface for JNA with this method :
int ReadCP(byte[] id, IntByReference size);
I load the DLL successfully and call the method that way :
byte[] id= new byte[10];
IntByReference size = new IntByReference();
IMimicDLL demo = (IMimicDLL) Native.loadLibrary("MyLib", IMimicDLL.class);
size.setValue(10);
//....
while(true){
demo.ReadCP(id, size);
//...
}
The first time in the loop id has a correct value, but it keeps the same value even if the logic should change it. What can be the problem? Is that something to do with pointers?
Your mapping of id is wrong: you cannot pass a primitive array as an argument via JNA.
You should change your interface to use a Pointer:
int ReadCP(Pointer id, IntByReference size);
Then you would allocate native-side memory for id:
Pointer id = new Memory(10);
After passing and retrieving id from the function you would then fetch the byte array from the native memory:
byte[] idByteArray = id.getByteArray(0, 10);
There are other get*() methods for Pointer, such as getString(), that may or may not be more applicable to the ultimate type of the Id field that you're trying to fetch.
As far as the value updating once but not after repeated calls, this sounds like at some point the system is taking a "snapshot" of the current hardware state and you must find a way to refresh that snapshot. Troubleshooting steps would include:
Clear out the data in the array/pointer and see if it's repopulated from the C-side DLL (the problem is not in your JNA it's in usage of the DLL).
Check your size variable throughout the process to make sure it's remaining the value of 10 you expect. It's possible that when you remove the card it may return 0, and then if you try to read a new value (of length 0) you're not overwriting the old array past index 0.
Alternate which card is used first.
Alternate the order of starting the program, loading, and swapping out the cards to collect data on which step of the process seems to cause the value to stick.
Investigate the DLL for methods to "refresh" or "reload" a snapshot of the hardware state.
Try unloading and reloading the DLL in between loops.
Most of these steps are outside of the scope of your question, on using JNA, and would require you to provide more information about the DLL being used for us to help further.
here the business login in the while loop
while(true){
try {
Thread.sleep(1500);
} catch (InterruptedException e1)
{
e1.printStackTrace();
}
if(demo.cardPresent() == 0 && read == false){
demo.ReadCP(id, size);
try {
System.out.println(" -- id : " + new String(id.getByteArray(0, 10),"UTF-8"));
read = true;
continue;
} catch (Exception ex) {
ex.printStackTrace();
}
}else if(demo.cardPresent() != 0){
read = false;
}
Related
Here is a version of implementation of function Atomic::cmpxchg used for CAS:
jbyte Atomic::cmpxchg(jbyte exchange_value, volatile jbyte* dest, jbyte compare_value) {
assert(sizeof(jbyte) == 1, "assumption.");
uintptr_t dest_addr = (uintptr_t)dest;
// look here
uintptr_t offset = dest_addr % sizeof(jint);
volatile jint* dest_int = (volatile jint*)(dest_addr - offset);
jint cur = *dest_int;
jbyte* cur_as_bytes = (jbyte*)(&cur);
jint new_val = cur;
jbyte* new_val_as_bytes = (jbyte*)(&new_val);
// ... and here
new_val_as_bytes[offset] = exchange_value;
while (cur_as_bytes[offset] == compare_value) {
jint res = cmpxchg(new_val, dest_int, cur);
if (res == cur) break;
cur = res;
new_val = cur;
new_val_as_bytes[offset] = exchange_value;
}
return cur_as_bytes[offset];
}
In the code above, I want to know what the use of offset actually is. I think we could simply and directly comapre cur_as_bytes and compare_value, without any offset. So why do we need it and how does it work? Is it for alignment? Thanks.
Yes, it is for alignment. The posted code implements a single-byte compare-exchange, using an already existing int-based compare-exchange.
This gives a few problems that the code needs to solve:
The int-based compare-exchange is restricted to reading int-aligned values, which means that you have to work out which of the (4?) bytes of the int you actually want to change. After all, the other bytes in the int must be unaffected
When you then actually do the compare-exchange, it is only a failure if the single byte you are trying to alter has been changed behind your back. If any of the other bytes in the int have changed, then that is only a failure for the int-cmpxchg, but not a failure for the byte-cmpxchg
The part before the while-loop handles the first part of that, by creating an int-aligned pointer that the int-value can be read from, and then setting up the "int we expect to see" and "int we want to change to" values.
The loop then handles the second part, where the algorithm attempts the int-cmpxchg, and then retries any failures as long as it is one of the other bytes that have been changed from expected.
RasEnumConnections function which realized in JNA is returning incomplete data.
What wrong? This is my code:
public static void main(String[] args) {
Connected();
}
private static void Connected () {
boolean state = false;
ArrayList<String> connectedNames = new ArrayList<>();
IntByReference lpcb = new IntByReference(0);
IntByReference lpcConnections = new IntByReference(0);
Rasapi32.INSTANCE.RasEnumConnections(null, lpcb,lpcConnections);
WinRas.RASCONN conn = new WinRas.RASCONN();
conn.dwSize = lpcb.getValue();
WinRas.RASCONN[] connArray;
if(lpcConnections.getValue() > 0)
connArray = (WinRas.RASCONN[])conn.toArray(lpcConnections.getValue());
else
connArray = (WinRas.RASCONN[])conn.toArray(1);
System.out.println("lpcb: " + lpcb.getValue() + " lpcConnections: " + lpcConnections.getValue() + " RASCONN Size: " + conn.dwSize);
int error = Rasapi32.INSTANCE.RasEnumConnections(connArray, lpcb,lpcConnections);
if(error == WinError.ERROR_SUCCESS) {
System.out.println("Entry name: " + Native.toString(connArray[0].szEntryName)
+ " Guid string: " + connArray[0].guidEntry.toGuidString());
System.out.println(connArray[0].guidEntry.Data1);
System.out.println(connArray[0].guidEntry.Data2);
System.out.println(connArray[0].guidEntry.Data3);
}
else System.out.println("Error: " + error);
WinRas.RASENTRY.ByReference entry = getPhoneBookEntry("test1");
if(entry != null) {
System.out.println("test1 guid: "+ entry.guidId.toGuidString());
System.out.println(entry.guidId.Data1);
System.out.println(entry.guidId.Data2);
System.out.println(entry.guidId.Data3);
}
else System.out.println("Error: " + Native.getLastError());
}
}
Char array szEntryName contains only 3 last chars of connection name. (Connection name is "test1")
As I've noted in the comments, the debug output gives you a strong hint at what's happening. The missing "t" and "e" characters appear as 0x74 and 0x65 in the midst of what JNA expects to be a 64-bit pointer. The logical conclusion is that Windows is returning a 32-bit pointer followed by the string, 4 bytes earlier than JNA expected.
RasEnumConnections states a few things regarding the buffer you are passing as connArray:
On input, an application must set the dwSize member of the first
RASCONN structure in the buffer to sizeof(RASCONN) in order to
identify the version of the structure being passed.
In your sample code above you are leaving this value the same as the value from the initial return. This is specifying the "wrong" version of the structure. Instead, you should set the dwSize member to the size you want in your JNA structure:
conn.dwSize = conn.size();
Actually, the constructor for RASCONN sets this for you! So you actually don't have to do this. But in your code sample above, you are overwriting what was pre-set; just delete your conn.dwSize line.
Note that since you are now requesting a (4-bytes per array element) larger buffer by definining the structure size, you also need to pass the increased size in the (second) RasEnumConnections() call. It's set as the number of elements times the (smaller) structure size, but you should reset to the number of elements times the (larger) size like this:
lpcb.setValue(conn.size() * lpcConnections.getValue());
prior to fetching the full array. Otherwise you'll get the error 632 (Incorrect Structure Size).
For reference (or perhaps a suitable replacement for your own code), take a look at the code as implemented in the getRasConnection(String connName) method in JNA's Rasapi32Util.java class.
I had this "java.lang.OutOfMemoryError: Java heap space " and I read and understand that I can increase my memory using -Xmx1024m. But I think in my code I can change something to this error does not happen anymore.
First, this is the image from VisualVM about my memory :
In the image you can see that the object "Pedidos" is not so big and I have the another object "Enderecos" that have more and less the same size but is not complete because I have the error before the object is completed.
The point is :
I have 2 classes that search for a big csv file ( 400.000 values each ), I will show the code. I tried to use Garbage Collector, set variables as null, but is not working, can anyone help me please? Here is the Code from the class "Pedidos", the class "Enderecos" is the same and my project is just calling this 2 classes.
// all Imports
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.Reader;
import java.util.ArrayList;
import javax.swing.JOptionPane;
import Objetos.Pedido;
// CLASS
public class GerenciadorPedido{
// ArrayList I will add all the "Pedidos" Objects
ArrayList<Pedido> listaPedidos = new ArrayList<Pedido>();
// Int that I need to use the values correctly
int helper;
// I create this global because I didnt want to create a new String everytime the for is running (trying to use less memory)
String Campo[];
String Linha;
String newLinha;
public ArrayList<Pedido> getListaPedidos() throws IOException {
// Here I change the "\" and "/" to be accepted be the FILE (the csv address)
String Enderecotemp = System.getProperty("user.dir"), Endereco = "";
char a;
for (int i = 0; i < Enderecotemp.length(); i++) {
a = Enderecotemp.charAt(i);
if (a == '\\') a = '/';
Endereco = Endereco + String.valueOf(a);
}
Endereco = Endereco + "/Pedido.csv";
// Open the CSV File and the reader to read it
File NovoArquivo = new File(Endereco);
Reader FileLer = null;
// Try to read the File
try
{
FileLer = new FileReader(NovoArquivo);
}
catch(FileNotFoundException e) {
JOptionPane.showMessageDialog(null, "Erro, fale com o Vini <Arquivo de Pedido Não Encontrado>");
}
// Read the File
BufferedReader Lendo = new BufferedReader(FileLer);
try
{
// Do for each line of the csv
while (Lendo.ready()) {
// Read the line and replace the caracteres ( needed to funcionality works )
Linha = Lendo.readLine();
newLinha = Linha.replaceAll("\"", "");
newLinha = newLinha.replaceAll(",,", ", , ");
newLinha = newLinha.replaceAll(",,", ", , ");
newLinha = newLinha + " ";
// Create Campo[x] for each value between ","
Campo = newLinha.split(",");
// Object
Pedido pedido = new Pedido();
helper = 0;
// Just to complete the object with the right values if the Campo.length have 15, 16, 17, 18 or 19 of size.
switch (Campo.length) {
case 15: pedido.setAddress1(Campo[9]);
break;
case 16: pedido.setAddress1(Campo[9] + Campo[10]);
helper = 1;
break;
case 17: pedido.setAddress1(Campo[9] + Campo[10] + Campo[11]);
helper = 2;
break;
case 18: pedido.setAddress1(Campo[9] + Campo[10] + Campo[11] + Campo[12]);
helper = 3;
break;
case 19: pedido.setAddress1(Campo[9] + Campo[10] + Campo[11] + Campo[12] + Campo[13]);
helper = 4;
break;
}
// Complete the Object
pedido.setOrder(Campo[0]);
pedido.setOrderValue(Float.parseFloat(Campo[1]));
pedido.setOrderPv(Float.parseFloat(Campo[2]));
pedido.setCombinedOrderFlag(Campo[3]);
pedido.setCombineOrder(Campo[4]);
pedido.setOrderType(Campo[5]);
pedido.setOrderShipped(Campo[6]);
pedido.setOrderCancelled(Campo[7]);
pedido.setTransactionType(Campo[8]);
pedido.setAddress2(Campo[10 + helper]);
pedido.setAddress3(Campo[11 + helper]);
pedido.setPost(Campo[12 + helper]);
pedido.setCity(Campo[13 + helper]);
pedido.setState(Campo[14 + helper]);
// Add the object in the ArrayList
listaPedidos.add(pedido);
// Set everything to null to start again
Campo = null;
Linha = null;
newLinha = null;
}
}
catch(IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally
{
// Close the file and run garbage collector to try to clear the trash
Lendo.close();
FileLer.close();
System.gc();
}
// return the ArrayList.
return listaPedidos;
}
}
The project runs this class, but when the project try to run the another ( the same as this one, changing just names and the csv ), I have the memory Error. I don't know how can I clear this char[] and String that is to big as can you see on the image. Any new Ideas ? Is really impossible without increase the memory ?
As is being discussed in the comments already, the main factor is your program places everything in memory at the same time. That design will inherently limit the size of the files you can process.
The way garbage collection works is that only garbage is collected. Any object that is referenced by another is not garbage. So, starting with the "root" objects (anything declared as static or local variables currently on the stack), follow the references. Your GerenciadorPedido instance is surely referenced from main(). It references a list listaPedidos. That list references (many) instances of Pedido each of which references many String instances. Those objects will all remain in memory while they are reachable through the list.
The way to design your program so it doesn't have a limit on the size of the file it can process is to eliminate the list entirely. Don't read the entire file and return a list (or other collection). Instead implement an Iterator. Read one line from the CSV file, create the Pedido, return it. When the program is finished with that one, then read the next line and create the next Pedido. Then you will have only one of these objects in memory at any given time.
Some additional notes regarding your current algorithm:
every String object references a char[] internally that contains the characters
ArrayList has very poor memory usage characteristics when adding to a large list. Since it is backed by an array, in order to grow to add the new element, it must create an entirely new array larger than the current one then copy all of the references. During the process it will use double the memory that it needs. This also becomes increasingly slower the larger the list is.
One solution is to tell the ArrayList how large you will need it to be so you can avoid resizing. This is only applicable if you actually know how large you will need it to be. If you need 100 elements: new ArrayList<>(100).
Another solution is to use a different data structure. A LinkedList is better for adding elements one at a time because it does not need to allocate and copy an entire array.
Each call to .replaceAll() will create a new char[] for the new String object. Since you then orphan the previous String object, it will get garbage collected. Just be aware of this need for allocation.
Each string concatenation (eg newLinha + " " or Campo[9] + Campo[10]) will create a new StringBuilder object, append the two strings, then create a new String object. This, again, can have an impact when repeated for large amounts of data.
You should, in general, never need to call System.gc(). It is okay to call it, but the system will perform garbage collection whenever memory is needed.
One addtional note: your approach to parsing the CSV will fail when the data contains characters you aren't expecting. In particular if any of the fields were to contain a comma. I recommend using an existing CSV parsing library for a simple solution to correctly handling the entire definition of CSV. (I have successful experience using opencsv)
My final goal is to obtain the icon of a HWND in Java with help of JNA library. Everything works fine except of one important thing: I need the size of the icon for further processing steps in Java.
It seems that I cannot request the size. I always obtain the size 0x0. What am I doing wrong? The basic code example looks like the following. Most API function templates were not part of JNA. So, I had to define them on my own.
final long hicon = ExtUser32.INSTANCE.SendMessageA(hwnd, ExtUser32.WM_GETICON, ExtUser32.ICON_BIG, 0);
final Pointer hIcon = new Pointer(hicon);
final ICONINFO info = new ICONINFO();
final BITMAP bmp = new BITMAP();
final SIZE size = new SIZE();
System.out.println(ExtUser32.INSTANCE.GetIconInfo(hIcon, info));
System.out.println(info);
System.out.println(ExtGdi32.INSTANCE.GetBitmapDimensionEx(info.hbmColor, size));
System.out.println(size);
if (info.hbmColor != null)
{
final int nWrittenBytes = ExtGdi32.INSTANCE.GetObjectA(info.hbmColor, bmp.size(), bmp.getPointer());
System.out.println(nWrittenBytes);
System.out.println(bmp);
}
The sysouts print this:
true
ICONINFO(auto-allocated#0x5b72b4f0 (32 bytes)) {
WinDef$BOOL fIcon#0=1
WinDef$DWORD xHotspot#4=16
WinDef$DWORD yHotspot#8=16
WinDef$HBITMAP hbmMask#10=native#0xffffffffb00515e8 (com.sun.jna.platform.win32.WinDef$HBITMAP#b00515e7)
WinDef$HBITMAP hbmColor#18=native#0xffffffffa50515c8 (com.sun.jna.platform.win32.WinDef$HBITMAP#a50515c7)
}
true
WinUser$SIZE(auto-allocated#0x652a3000 (8 bytes)) {
int cx#0=0
int cy#4=0
}
32
BITMAP(auto-allocated#0x5b72b5b0 (32 bytes)) {
WinDef$LONG bmType#0=0
WinDef$LONG bmWidth#4=0
WinDef$LONG bmHeight#8=0
WinDef$LONG bmWidthBytes#c=0
WinDef$WORD bmPlanes#10=0
WinDef$WORD bmBitsPixel#12=0
WinDef$LPVOID bmBits#18=0
}
The request of ICONINFO structure seems to be correct. But if I try to request the dimension for the set hbmColor structure component by Gdi32.GetBitmapDimensionEx() then the structure keeps initialized with zeros. This approach via hbmColor or hbmMask was suggested by:
How to determine the size of an icon from a HICON?
UPDATE 1
Error tracing added!
As the sysouts indicate (true), the concerning function invocations didn't fail.
UPDATE 2
Further observation: In Java, these recreated structure types are intialized with zeros after instantiation. I set the initial values of the structure components in SIZE and BITMAP to a value that deviates from zero. GetBitmapDimensionEx sets it back to zero. But GetObjectA doesn't modify the structure! The function's return result indicates that bytes were written but that's not true!
...
size.cx = 1;
size.cy = 2;
bmp.bmType.setValue(1);
bmp.bmWidth.setValue(2);
bmp.bmHeight.setValue(3);
bmp.bmWidthBytes.setValue(4);
bmp.bmPlanes.setValue(5);
bmp.bmBitsPixel.setValue(6);
bmp.bmBits.setValue(7);
System.out.println(ExtGdi32.INSTANCE.GetBitmapDimensionEx(info.hbmColor, size));
System.out.println(size);
if (info.hbmColor != null)
{
final int nWrittenBytes = ExtGdi32.INSTANCE.GetObjectA(info.hbmColor, bmp.size(), bmp.getPointer());
System.out.println(nWrittenBytes);
System.out.println(bmp);
}
Results:
true
WinUser$SIZE(auto-allocated#0x64fbcb20 (8 bytes)) {
int cx#0=0
int cy#4=0
}
32
BITMAP(auto-allocated#0x64fb91f0 (32 bytes)) {
WinDef$LONG bmType#0=1
WinDef$LONG bmWidth#4=2
WinDef$LONG bmHeight#8=3
WinDef$LONG bmWidthBytes#c=4
WinDef$WORD bmPlanes#10=5
WinDef$WORD bmBitsPixel#12=6
WinDef$LPVOID bmBits#18=7
}
I would have added this as a comment but my reputation is too low:
You are not showing your BITMAP or GetObjectA definitions so I'm guessing but
in your line:
final int nWrittenBytes = ExtGdi32.INSTANCE.GetObjectA(info.hbmColor, bmp.size(), bmp.getPointer());
you fail to call 'bmp.read()' afterwards.
If you look at the javadoc for Struture.getPointer()
https://jna.java.net/javadoc/com/sun/jna/Structure.html
you see that you are responsible for calling Structure.write() and Structure.read() before and after making the call to native method that uses a pointer obtained with getPointer(). In your case the write is superfluous but it is a good practice.
To understand why this is necessary consider that your BITMAP/bmp object is a Java object living in Java heap where it can get moved around during garbage collection. Hence the getPointer() cannot return the real address of the 'real' object. Instead it returns a pointer to a separate fixed (non movable) chunk of memory in the native heap (which chunk JNA allocates and associates with your Java object. Now your getObjectA() routine will write its stuff to that memory but JNA or anyone on the java side cannot have a clue that this is what happens. So you need to call the read() to tell JNA to copy the native side stuff to the Java object.
If this is a 32-bit application only:
Your BITMAP structure is incorrect. Here's a simple C program that prints the expected offsets of all the fields of BITMAP, and its total size:
// 24 january 2015
#include <windows.h>
#include <stdio.h>
#include <stddef.h>
int main(void)
{
printf("%d\n", sizeof (BITMAP));
#define O(f) printf("%s %x\n", #f, offsetof(BITMAP, f))
O(bmType);
O(bmWidth);
O(bmHeight);
O(bmWidthBytes);
O(bmPlanes);
O(bmBitsPixel);
O(bmBits);
return 0;
}
And here's what I get (in wine, compiled as a 32-bit program with MinGW-w64):
24
bmType 0
bmWidth 4
bmHeight 8
bmWidthBytes c
bmPlanes 10
bmBitsPixel 12
bmBits 14
Notice that your Java output above has a different size for BITMAP and a different offset for bmBits.
(The BITMAP structure is correct for 64-bit applications.)
General:
Why GetObject() is indicating success is beyond me.
I don't know JNA at all so I don't know what to do about this. Does JNA not provide a WinGDI$BITMAP?
I am using a library libfprint on ubuntu nd I am trying to call a function through my java code.
API_EXPORTED struct fp_img *fpi_img_new(size_t length)
{
struct fp_img *img = g_malloc(sizeof(*img) + length);
memset(img, 0, sizeof(*img));
fp_dbg("length=%zd", length);
img->length = length;
return img;
}
I am passing integer value 5 from my java code to this function. When I try to execute above function I got following errors:
GLib-ERROR **: /build/buildd/glib2.0-2.30.0/./glib/gmem.c:170: failed to allocate 3077591024 bytes
I have tried same code on 2 different ubuntu machine but the error remains the same. I dont know why it is trying to allocate so many bytes for 24+5 length.
Could anyone suggest me any solution?
The source code clearly states:
/* structs that applications are not allowed to peek into */
(...)
struct fp_img;
So, I'm not sure what you did in order to even compile something that needs the size of struct fp_img: you're not supposed to be able to do that, since the structure declaration is opaque.
It look like you get a pointer instead of a size_t.
Try to change your definition to:
API_EXPORTED struct fp_img *fpi_img_new(size_t * length);
You then need to derefenrece it:
API_EXPORTED struct fp_img *fpi_img_new(size_t * length)
{
struct fp_img *img = g_malloc(sizeof(*img) + *length);
memset(img, 0, sizeof(*img));
fp_dbg("length=%zd", *length);
img->length = *length;
return img;
}
Note: It seems that 3077591024 is on the stack (0x125807FE) it is highly platform dependent so don't quote me on that.
What happens if you change
sizeof(*img)
with
sizeof(struct fp_img)
? I'm thinking that this could be your problem because *img is not initialised to anything at the time you are calling g_malloc().
ERROR: struct fp_img *img = g_malloc(sizeof(*img) + length);
*img is new created, so you can not use it like "sizeof(*img)",
you could write like this: "sizeof(struct fp_img)"