I want to access to a char[] from JNI code, avoiding if possible a copy. For now, here is what I do :
void my_function(JNIEnv *env, jcharArray array) {
jsize len = env->GetArrayLength(array);
if (env->ExceptionCheck() == JNI_TRUE)
return;
jchar *chars = env->GetPrimitiveArrayCritical(array);
// process chars and len
env->ReleasePrimitiveArrayCritical(array, chars, JNI_ABORT);
}
According to JNI documentation, GetPrimitiveArrayCritical may return NULL when an OOM exception was "thrown". However they do not tell explicitly if ReleasePrimitiveArrayCritical should be still called in that situation.
Should I do
jchar *chars = env->GetPrimitiveArrayCritical(array);
if (chars == NULL)
return;
or
jchar *chars = env->GetPrimitiveArrayCritical(array);
if (chars != NULL) {
// process chars
}
env->ReleasePrimitiveArrayCritical(array, chars, JNI_ABORT);
ReleasePrimitiveArrayCritical(…, NULL, …) will fail. But I must confess that I never experienced GetPrimitiveArrayCritical() to return NULL. It's a pity there is no way to ask for the bytes conditionally, i.e. no JNI function GetPrimitiveArrayCriticalNoCopyOrFail()!
My understanding of docs is that, there is nothing to release, if you don't get your "chars". By calling release, you don't release your "array", just elements associated with the array. It frees "chars" if necessary (only if it is a copy)
Releasing "array" is necessary if you are the owner of it. In this case:
{code}
int len = 10;
jcharArray array = env->NewCharArray(len);
if (!array) return;
jchar* chars = env->GetCharArrayElements(array, NULL);
if (!chars) { env->DeleteLocalRef(array); return; }
{... do something...}
env->ReleaseCharArrayElements(array, chars, JNI_ABORT);
env->DeleteLocalRef(array);
Also, I found this sample of production code which explicitly calls to ReleasePrimitiveArrayCritical only if the returned pointer is not null.
https://code.google.com/p/chromium/codesearch#chromium/src/third_party/libjpeg_turbo/turbojpeg-jni.c&q=GetPrimitiveArrayCritical&sq=package:chromium&l=365
Looking at openjdk code, it seems that the call to ReleasePrimitiveArrayCritical is necessary to unlock the GC (GC_locker::unlock_critical) that was locked in the GetPrimitiveArrayCritical call.
Seems also that java array and native array arguments are ignored, and that GetPrimitiveArrayCritical never returns NULL anyway.
This is valid for openjdk 7 only though.
Related
I am reading BufferedInputStream's source code in Java.I noticed there's a private Field:BUF_OFFSET.enter image description here
private static final long BUF_OFFSET
= U.objectFieldOffset(BufferedInputStream.class, "buf");
I tried to check it out,then I found the method:objectFieldOffset1.It is a native method.So I download the JDK source code(jdk-17 +35) from github.https://github.com/openjdk/jdk/tree/jdk-17%2B35
Finally I found the jlong find_field_offset method.
I can basically understand the code except the JavaFieldStreamenter image description here
static jlong find_field_offset(jclass clazz, jstring name, TRAPS) {
assert(clazz != NULL, "clazz must not be NULL");
assert(name != NULL, "name must not be NULL");
ResourceMark rm(THREAD);
char *utf_name = java_lang_String::as_utf8_string(JNIHandles::resolve_non_null(name));
InstanceKlass* k = InstanceKlass::cast(java_lang_Class::as_Klass(JNIHandles::resolve_non_null(clazz)));
jint offset = -1;
for (JavaFieldStream fs(k); !fs.done(); fs.next()) {
Symbol *name = fs.name();
if (name->equals(utf_name)) {
offset = fs.offset();
break;
}
}
if (offset < 0) {
THROW_0(vmSymbols::java_lang_InternalError());
}
return field_offset_from_byte_offset(offset);
}
I tried to search in google,but nothing found.I tried to find it in Java offical document,nothing either.
Looks like JavaFieldStream is a iterator?
Looks like JavaFieldStream is a iterator?
Correct.
It is declared in "jdk17u/src/hotspot/share/oops/fieldStreams.hpp"
The comments in the file say (for FieldStreamBase):
// The is the base class for iteration over the fields array
// describing the declared fields in the class. Several subclasses
// are provided depending on the kind of iteration required. The
// JavaFieldStream is for iterating over regular Java fields and it
// generally the preferred iterator. InternalFieldStream only
// iterates over fields that have been injected by the JVM.
// AllFieldStream exposes all fields and should only be used in rare
// cases.
Hint: find jdk17u -type f | xargs grep JavaFieldStream | less
I am starting to use JNA to communicate with a device on the RS485 interface of a computer. Suprisingly to me I came to good results very quickly. But now I am stuck by a simple problem. The library I use accepts a pointer to a pointer of struct. The actual signature is
func(Struct1 **, Struct2 **, Struct3 *, Struct4 *, long)
Now to indicate the size of the first parameter the library expects the last pointer to be a NULL pointer. This is what fails. Following code is what I tried so far:
Struct1.ByReference[] s = (Struct1.ByReference[]) new Struct1.ByReference().toArray(size);
int pos = 0;
// ...
// for loop to set the s[pos] struture values
for(pos = 0; pos < size - 1; pos++)
// ...
// Now set the last array element to a null pointer to indicate end-of-list
s[pos].getPointer().setPointer(0, null);// Following does not work: results in zero memoried structure
s[pos] = null; // Following does not work wither: NullPointerException at com.sun.jna.Structure.autoWrite
EDIT 1
s[pos] = new Struct1.ByReference(Pointer.NULL); // results in zero memoried structure as well
EDIT 2
According to technomage's question. If I were to write C code it would probably look something like that:
Struct1 **s = malloc(n * sizeof(Struct1*));
for(int i=0; i<n; i++)
{
if(i == n -1)
{
s[i] = NULL;
}
else
{
s[i] = malloc(sizeof(Struct1));
s[i].bla = value;
....
}
}
But be warned: I am not very skilled in C/C++. I consider Java to be my domain.
Has anyone had a similar problem? Maybe I am just not seeing the wood for the trees...
Thanks in advance.
Structures in JNA are pointers, so what you really need here is a pointer to a (pointer to a) Structure, which is a PointerByReference -- in your case, an array of them.
Given the code example above, you'll create your array of Structures, one less than n:
Struct1[] struct1Array = new Struct1[n-1];
This only allocates the Java memory for the array.
Next you'll instantiate and write the changes you make to native memory:
for (int i = 0; i < n-1; i++) {
struct1Array[i] = new Struct1();
struct1Array[i].bla = value;
struct1Array[i].write();
}
The new Struct1() allocates native side memory for these structures. It's possible to use the Structure.toArray() method to do this as well; I'm intentionally doing this a bit more manual and low-level to try to make clear what's happening.
Then you'll create a corresponding PointerByReference array to hold the pointers to these structures. You'll add an extra element for the null:
PointerByReference[] pbrArray = new PointerByReference[n];
Again, this is only java-side allocation. And then you fill it with pointers to the pointers to the structure, obtained from the Structure.getPointer() method:
for (int i = 0; i < n-1; i++) {
pbrArray[i] = new PointerByReference(struct1Array[i].getPointer());
}
pbrArray[n - 1] = new PointerByReference(Pointer.NULL);
The new PointerByReference() here allocates the native side memory for the pointer itself, which points to the native-side structure you allocated earlier.
From how I understand your initial question, you will pass this PointerByReference array to your function, which presumably updates your structures.
Since you created the two arrays in this fashion, you can keep track of their correspondence by array index. You may have to iterate through the structure array and read() the native memory into the Java-side structure to do further processing with it. Typically when you work directly with Structures being passed to methods they autowrite and autoread, but when using a PointerByReference to indirectly reference the Structure, JNA isn't as friendly.
As an alternative to tracking the two arrays by corresponding indices, you could "forget" the initial Structure assignment and recover it later using the PointerByReference.getValue() method on your array to recover a pointer to the memory for the structure, and then instantiate a new structure using that Pointer in its constructor (e.g. new Struct1(pbr.getValue()) which calls super() with that pointer).
I'm using SWIG 2.0.10 in Ubuntu to call C++ code in Java.
My C++ code is:
//ImgPro.h:
#include <vector>
typedef struct _bin
{
char* name;
float value;
} Bin;
typedef struct imgprops
{
std::vector<Bin> color;
int width;
int height;
char *print;
} ImageProperties;
class ImgPro
{
public:
ImgPro();
ImageProperties *processImage(char* imagePath);
};
The processImage function definition is:
ImageProperties* ImgPro::processImage(char *imagePath)
{
ImageProperties* imgProp = new ImageProperties();
imgProp->width = 200;
imgProp->height = 200;
char* fp = new char(5);
strcpy(fp, "abc!");
imgProp->print = fp;
Bin outputBin1;
char *name1 = new char(strlen("red")+1);
strcpy(name1, "red");
outputBin1.name = name1;
outputBin1.value = 0.125;
Bin outputBin2;
char *name2 = new char(strlen("blue")+1);
strcpy(name2, "blue");
outputBin2.name = name1;
outputBin2.value = 0.27;
vector<Bin> tempVec;
tempVec.push_back(outputBin1);
tempVec.push_back(outputBin2);
imgProp->color = tempVec;
return imgProp;
}
So, to generate the jni code using swig, i've used the following swig file (note: the vector.i file was created using this example ) :
%module CBIR
// to handle char** has String_Array in Java
%include <various.i>
%include "vector.i"
%{
#include "ImgPro.h"
%}
// to handle char** has String_Array in Java
%apply char **STRING_ARRAY { char ** };
// memory release
%extend imgprops {
~imgprops(){
if($self != NULL)
{
// releasing print element
if($self->print != NULL)
delete[] $self->print;
// releasing vector elements
for(uint x = 0; x < $self->color.size(); x++)
{
Bin currentBin = $self->color[x];
if(currentBin.name != NULL)
delete[] currentBin.name;
}
// releasing stuct Pointer
delete $self;
}
}
}
%include "ImgPro.h"
%template(BinVec) std::vector<Bin>;
And this generates in the swig_wrap file the next function:
SWIGINTERN void delete_imgprops(imgprops *self){
if(self != NULL)
{
// releasing print element
if(self->print != NULL)
delete[] self->print;
// releasing vector elements
for(uint x = 0; x < self->color.size(); x++)
{
Bin currentBin = self->color[x];
if(currentBin.name != NULL)
delete[] currentBin.name;
}
// releasing stuct Pointer
delete self;
}
}
which is called in the delete ImageProperties c++ function.
However, running the following code in Java never releases the memory (calling the function delete_imgprops) allocated in C++:
ImgPro imgObject = new ImgPro();
ImageProperties propObject = imgObject.processImage("imagem123-jpg");
int width = propObject.getWidth();
int height = propObject.getHeight();
String fingerPrint = propObject.getPrint();
propObject.delete();
imgObject.delete();
So, after analyzing the code flow, i found the reason why the memory isn't released. The ImageProperties.Java file generated by SWIG contains, among others, the delete function:
public synchronized void delete() {
if (swigCPtr != 0) {
if (swigCMemOwn) {
swigCMemOwn = false;
CBIRJNI.delete_ImageProperties(swigCPtr);
}
swigCPtr = 0;
}
}
The line "CBIRJNI.delete_ImageProperties(swigCPtr);" is never called because the var swigCMemOwn is always false.
I understand that because the Java side doesn't alloc memory so it also doesn't release it, so what can i do to ensure that java releases memory without any modification on java files generated by swig?
The solution that i found to release the memory is to comment the if(swigCMemOwn) test on delete() function, but i don't think that it's the best way of do it!
Thanks, Sérgio
You could check out the
%newobject
directive (in swig 2.0) with a factory method. It tells swig that a particular function will return a new object and that the java proxy class should be responsible for cleaning up the c++ memory as well. The generated code will set swigCMemOwn to true resulting in the c++ destructor being called.
If you do call the delete method yourself, that's fine - you just have to change your style of programming to think of the swig'ed object as something like a file handle or a db connection. You call close() on those objects when you're done because you don't want to wait for java's GC to kick in at some unknown later point and collect this expensive resource- you want to manage it manually.
But you also obviously must remember to exercise good code discipline to make sure you don't use the java object anywhere after calling delete()
You should never call delete() manually. If you correctly implemented C++ destructor (or wherever you memory is freed) the memory will be released as soon as the Java wrapper object is released, SWIG wrapper code then will call appropriate method automatically. Read more at SWIG doc for Java.
I have found the clone() method of Object in Java:
protected native Object clone() throws CloneNotSupportedException;
Is the source of this method available? Maybe in OpenJDK?
From jdk/src/share/native/java/lang/Object.c
static JNINativeMethod methods[] = {
{"hashCode", "()I", (void *)&JVM_IHashCode},
{"wait", "(J)V", (void *)&JVM_MonitorWait},
{"notify", "()V", (void *)&JVM_MonitorNotify},
{"notifyAll", "()V", (void *)&JVM_MonitorNotifyAll},
{"clone", "()Ljava/lang/Object;", (void *)&JVM_Clone},
};
Meaning its a function pointer(probably done so they could implement platform-specific native code)
doing a grep for JVM_Clone produces, among other things:
(from hotspot/src/share/vm/prims/jvm.cpp)
JVM_ENTRY(jobject, JVM_Clone(JNIEnv* env, jobject handle))
JVMWrapper("JVM_Clone");
Handle obj(THREAD, JNIHandles::resolve_non_null(handle));
const KlassHandle klass (THREAD, obj->klass());
JvmtiVMObjectAllocEventCollector oam;
#ifdef ASSERT
// Just checking that the cloneable flag is set correct
if (obj->is_javaArray()) {
guarantee(klass->is_cloneable(), "all arrays are cloneable");
} else {
guarantee(obj->is_instance(), "should be instanceOop");
bool cloneable = klass->is_subtype_of(SystemDictionary::Cloneable_klass());
guarantee(cloneable == klass->is_cloneable(), "incorrect cloneable flag");
}
#endif
// Check if class of obj supports the Cloneable interface.
// All arrays are considered to be cloneable (See JLS 20.1.5)
if (!klass->is_cloneable()) {
ResourceMark rm(THREAD);
THROW_MSG_0(vmSymbols::java_lang_CloneNotSupportedException(), klass->external_name());
}
// Make shallow object copy
const int size = obj->size();
oop new_obj = NULL;
if (obj->is_javaArray()) {
const int length = ((arrayOop)obj())->length();
new_obj = CollectedHeap::array_allocate(klass, size, length, CHECK_NULL);
} else {
new_obj = CollectedHeap::obj_allocate(klass, size, CHECK_NULL);
}
// 4839641 (4840070): We must do an oop-atomic copy, because if another thread
// is modifying a reference field in the clonee, a non-oop-atomic copy might
// be suspended in the middle of copying the pointer and end up with parts
// of two different pointers in the field. Subsequent dereferences will crash.
// 4846409: an oop-copy of objects with long or double fields or arrays of same
// won't copy the longs/doubles atomically in 32-bit vm's, so we copy jlongs instead
// of oops. We know objects are aligned on a minimum of an jlong boundary.
// The same is true of StubRoutines::object_copy and the various oop_copy
// variants, and of the code generated by the inline_native_clone intrinsic.
assert(MinObjAlignmentInBytes >= BytesPerLong, "objects misaligned");
Copy::conjoint_jlongs_atomic((jlong*)obj(), (jlong*)new_obj,
(size_t)align_object_size(size) / HeapWordsPerLong);
// Clear the header
new_obj->init_mark();
// Store check (mark entire object and let gc sort it out)
BarrierSet* bs = Universe::heap()->barrier_set();
assert(bs->has_write_region_opt(), "Barrier set does not have write_region");
bs->write_region(MemRegion((HeapWord*)new_obj, size));
// Caution: this involves a java upcall, so the clone should be
// "gc-robust" by this stage.
if (klass->has_finalizer()) {
assert(obj->is_instance(), "should be instanceOop");
new_obj = instanceKlass::register_finalizer(instanceOop(new_obj), CHECK_NULL);
}
return JNIHandles::make_local(env, oop(new_obj));
JVM_END
I found these files, but all they do is import the function from elsewhere. I can't find the actual source (try asking a Hotspot dev).
Object.c
jvm.h
Edit: Here's a link to the actual source online (Thanks user439407 for finding which file it was in)
Given this C++ Code:
void LoadData(char** myVar)
{
std:: string str("[Really Long String Here]");
unsigned int size = str.length() + 1;
*myVar = new char[size];
strncpy(*myVar, str.c_str(), size);
}
And this JNA Java:
Pointer myVar = new Memory(Pointer.SIZE);
this.Lib.LoadData(myVar);
this.someVar = myVar.getPointer(0).getString(0);
I'm having memory leaks, as I understand it, getPointer(0) should create a pointer object that should be released on finalize(), but it seems to not be.
Am I missing something? This seems up to spec... and I can run the function above with no leaks in C++ fine.
I call the Java code in a loop to test the leak, I've tried putting in pauses, and manually calling the GC, also it'll bloat to gigabytes rather quickly this way.
I've been banging my head against this for a few days now and it sucks to get hung up on something so trivial as attempting to free memory.As far as I can tell I can only manually free memory in Java if I have the address, but I can't see how I'd get that.
Edit:
Nevermind, I don't even think there is a way to do manually free through JNA without extending it...
Add this function to the C++ library...
void FreeData(char** myVar)
{
delete [] *myVar;
}
And then make this the JNA code
Pointer myVar = new Memory(Pointer.SIZE);
this.Lib.LoadData(myVar);
this.someVar = myVar.getPointer(0).getString(0);
this.Lib.FreeData(myVar);
This way you allocate and delete the memory in C++.
Allocate in the caller, not the callee.
For example:
int LoadData(char* buf, int maxlen) {
std:: string str("[Really Long String Here]");
strncpy(buf, str.c_str(), maxlen);
if (str.length() < maxlen)
return str.length();
return maxlen;
}
Then when you call from Java, pass in a byte[] of the appropriate size. Note that this implementation is potentially very inefficient, but the idea is that you don't generally want to be allocating memory in one context and deallocating it in another.
Instead of
myVar = new char[size]
use
*myVar = malloc(size);
strncpy(*myVar, str.c_str(), size);
Arrays need to be deleted like:
delete [] *myVar;
JNA prolly doesn't know to do that.