Java: load a library that depends on other libs - java

I want to load my own native libraries in my java application. Those native libraries depend upon third-party libraries (which may or may not be present when my application is installed on the client computer).
Inside my java application, I ask the user to specify the location of dependent libs. Once I have this information, I am using it to update the "LD_LIBRARY_PATH" environment variable using JNI code. The following is the code snippet that I am using to change the "LD_LIBRARY_PATH" environment variable.
Java code
public static final int setEnv(String key, String value) {
if (key == null) {
throw new NullPointerException("key cannot be null");
}
if (value == null) {
throw new NullPointerException("value cannot be null");
}
return nativeSetEnv(key, value);
}
public static final native int nativeSetEnv(String key, String value);
Jni code (C)
JNIEXPORT jint JNICALL Java_Test_nativeSetEnv(JNIEnv *env, jclass cls, jstring key, jstring value) {
const char *nativeKey = NULL;
const char *nativeValue = NULL;
nativeKey = (*env)->GetStringUTFChars(env, key, NULL);
nativeValue = (*env)->GetStringUTFChars(env, value, NULL);
int result = setenv(nativeKey, nativeValue, 1);
return (jint) result;
}
I also have corresponding native methods to fetch the environment variable.
I can successfully update the LD_LIBRARY_PATH (this assertion is based on the output of C routine getenv().
I am still not able to load my native library. The dependent third-party libraries are still not detected.
Any help/pointers are appreciated. I am using Linux 64 bit.
Edit:
I wrote a SSCE (in C) to test if dynamic loader is working. Here is the SSCE
#include
#include
#include
#include
int main(int argc, const char* const argv[]) {
const char* const dependentLibPath = "...:";
const char* const sharedLibrary = "...";
char *newLibPath = NULL;
char *originalLibPath = NULL;
int l1, l2, result;
void* handle = NULL;
originalLibPath = getenv("LD_LIBRARY_PATH");
fprintf(stdout,"\nOriginal library path =%s\n",originalLibPath);
l1 = strlen(originalLibPath);
l2 = strlen(dependentLibPath);
newLibPath = (char *)malloc((l1+l2)*sizeof(char));
strcpy(newLibPath,dependentLibPath);
strcat(newLibPath,originalLibPath);
fprintf(stdout,"\nNew library path =%s\n",newLibPath);
result = setenv("LD_LIBRARY_PATH", newLibPath, 1);
if(result!=0) {
fprintf(stderr,"\nEnvironment could not be updated\n");
exit(1);
}
newLibPath = getenv("LD_LIBRARY_PATH");
fprintf(stdout,"\nNew library path from the env =%s\n",newLibPath);
handle = dlopen(sharedLibrary, RTLD_NOW);
if(handle==NULL) {
fprintf(stderr,"\nCould not load the shared library: %s\n",dlerror());
exit(1);
}
fprintf(stdout,"\n The shared library was successfully loaded.\n");
result = dlclose(handle);
if(result!=0) {
fprintf(stderr,"\nCould not unload the shared library: %s\n",dlerror());
exit(1);
}
return 0;
}
The C code also does not work. Apparently, the dynamic loader is not rereading the LD_LIBRARY_PATH environment variable. I need to figure out how to force the dynamic loader to re-read the LD_LIBRARY_PATH environment variable.

See the accepted answer here:
Changing LD_LIBRARY_PATH at runtime for ctypes
In other words, what you're trying to do isn't possible. You'll need to launch a new process with an updated LD_LIBRARY_PATH (e.g., use ProcessBuilder and update environment() to concatenate the necessary directory)

This is a hack used to manipulate JVM's library path programmatically. NOTE: it relies on internals of ClassLoader implementation so it might not work on all JVMs/versions.
String currentPath = System.getProperty("java.library.path");
System.setProperty( "java.library.path", currentPath + ":/path/to/my/libs" );
// this forces JVM to reload "java.library.path" property
Field fieldSysPath = ClassLoader.class.getDeclaredField( "sys_paths" );
fieldSysPath.setAccessible( true );
fieldSysPath.set( null, null );
This code uses UNIX-style file path separators ('/') and library path separator (':'). For cross-platform way of doing this use System Properties to get system-specific separators: http://download.oracle.com/javase/tutorial/essential/environment/sysprop.html

I have successfully implemented something similar for CollabNet Subversion Edge, which depends on the SIGAR libraries across ALL Operating Systems (we support Windows/Linux/Sparc both 32 bits and 64 bits)...
Subversion Edge is a web application that helps one managing Subversion repositories through a web console and uses SIGAR to the SIGAR libraries helps us provide users data values directly from the OS... You need to update the value of the property "java.library.path" at runtime. (https://ctf.open.collab.net/integration/viewvc/viewvc.cgi/trunk/console/grails-app/services/com/collabnet/svnedge/console/OperatingSystemService.groovy?revision=1890&root=svnedge&system=exsy1005&view=markup Note that the URL is a Groovy code, but I have modified it to a Java here)...
The following example is the implementation in URL above... (On Windows, your user will be required to restart the machine if he/she has downloaded the libraries after or downloaded them using your application)... The "java.library.path" will update the user's path "usr_paths" instead of System path "sys_paths" (permissions exception might be raised depending on the OS when using the latter).
133/**
134 * Updates the java.library.path at run-time.
135 * #param libraryDirPath
136 */
137 public void addDirToJavaLibraryPathAtRuntime(String libraryDirPath)
138 throws Exception {
139 try {
140 Field field = ClassLoader.class.getDeclaredField("usr_paths");
141 field.setAccessible(true);
142 String[] paths = (String[])field.get(null);
143 for (int i = 0; i < paths.length; i++) {
144 if (libraryDirPath.equals(paths[i])) {
145 return;
146 }
147 }
148 String[] tmp = new String[paths.length+1];
149 System.arraycopy(paths,0,tmp,0,paths.length);
150 tmp[paths.length] = libraryDirPath;
151 field.set(null,tmp);
152 String javaLib = "java.library.path";
153 System.setProperty(javaLib, System.getProperty(javaLib) +
154 File.pathSeparator + libraryDirPath);
155
156 } catch (IllegalAccessException e) {
157 throw new IOException("Failed to get permissions to set " +
158 "library path to " + libraryDirPath);
159 } catch (NoSuchFieldException e) {
160 throw new IOException("Failed to get field handle to set " +
161 "library path to " + libraryDirPath);
162 }
163 }
The Bootstrap services (Groovy on Grails application) class of the console runs a service and executes it with the full path to the library directory... UNiX-based servers do not need to restart the server to get the libraries, but Windows boxes do need a server restart after the installation. In your case, you would be calling this as follows:
String appHomePath = "/YOUR/PATH/HERE/TO/YOUR/LIBRARY/DIRECTORY";
String yourLib = new File(appHomePath, "SUBDIRECTORY/").getCanonicalPath();
124 try {
125 addDirToJavaLibraryPathAtRuntime(yourLib);
126 } catch (Exception e) {
127 log.error("Error adding the MY Libraries at " + yourLib + " " +
128 "java.library.path: " + e.message);
129 }
For each OS you ship your application, just make sure to provide a matching version of the libraries for the specific platform (32bit-Linux, 64bit-Windows, etc...).

Related

Connecting with a third party C++ header file using JNA

I have got a third party dll file with header like below.
class MyClass
{
public:
MyClass() {};
virtual ~MyClass() {};
int Double(int x);
};
I cannot connect with the dll file using JNA (as I cannot extern here). So I have created a wrapper dll. From that wrapper dll , I am trying to connect with the third party dll. wrapper dll header file (Sample.h)
extern "C" {
int __declspec(dllexport) Double(int x);
}
wrapper dll's cpp file
typedef int (__cdecl *MYPROC)(int);
int func(int x)
{
HINSTANCE hinstLib;
MYPROC ProcAdd;
BOOL fFreeResult, fRunTimeLinkSuccess = FALSE;
int result;
printf("inside the function");
// Get a handle to the DLL module.
hinstLib = LoadLibrary(TEXT("example_dll.dll"));
// If the handle is valid, try to get the function address.
if (hinstLib != NULL)
{
ProcAdd = (MYPROC) GetProcAddress(hinstLib, "Double");
// If the function address is valid, call the function.
if (NULL != ProcAdd)
{
fRunTimeLinkSuccess = TRUE;
result = ProcAdd(x);
printf("Result: %d", result);
} else {
printf("function not found");
}
// Free the DLL module.
fFreeResult = FreeLibrary(hinstLib);
}
// If unable to call the DLL function, use an alternative.
if (! fRunTimeLinkSuccess)
printf("Message printed from executable\n");
return result;
}
Using a C program(compiled by g++) I can connect with both the dll and get the data.
But from a Java program(using JNA), I can connect with the first dll but first dll cannot connect with the second dll.
Could anybody please enlighten me how to fix the problem? Of if there are any other way to fix it?

R - Connecting R and java using Rserve

I have build an application connecting R and java using the Rserve package.
In that, i am getting the error as "evaluation successful but object is too big to transport". i have tried increasing the send buffer size value in Rconnection class also. but that doesn't seem to work.
The object size which is being transported is 4 MB
here is the code from the R connection file
public void setSendBufferSize(long sbs) throws RserveException {
if (!connected || rt == null) {
throw new RserveException(this, "Not connected");
}
try {
RPacket rp = rt.request(RTalk.CMD_setBufferSize, (int) sbs);
System.out.println("rp is send buffer "+rp);
if (rp != null && rp.isOk()) {
System.out.println("in if " + rp);
return;
}
} catch (Exception e) {
e.printStackTrace();
LogOut.log.error("Exception caught" + e);
}
//throw new RserveException(this,"setSendBufferSize failed",rp);
}
The full java class is available here :Rconnection.java
Instead of RServe, you can use JRI, that is shipped with rJava package.
In my opinion JRI is better than RServe, because instead of creating a separate process it uses native calls to integrate Java and R.
With JRI you don't have to worry about ports, connections, watchdogs, etc... The calls to R are done using an operating system library (libjri).
The methods are pretty similar to RServe, and you can still use REXP objects.
Here is an example:
public void testMeanFunction() {
// just making sure we have the right version of everything
if (!Rengine.versionCheck()) {
System.err.println("** Version mismatch - Java files don't match library version.");
fail(String.format("Invalid versions. Rengine must have the same version of native library. Rengine version: %d. RNI library version: %d", Rengine.getVersion(), Rengine.rniGetVersion()));
}
// Enables debug traces
Rengine.DEBUG = 1;
System.out.println("Creating Rengine (with arguments)");
// 1) we pass the arguments from the command line
// 2) we won't use the main loop at first, we'll start it later
// (that's the "false" as second argument)
// 3) no callback class will be used
engine = REngine.engineForClass("org.rosuda.REngine.JRI.JRIEngine", new String[] { "--no-save" }, null, false);
System.out.println("Rengine created...");
engine.parseAndEval("rVector=c(1,2,3,4,5)");
REXP result = engine.parseAndEval("meanVal=mean(rVector)");
// generic vectors are RVector to accomodate names
assertThat(result.asDouble()).isEqualTo(3.0);
}
I have a demo project that exposes a REST API and calls R functions using this package.
Take a look at: https://github.com/jfcorugedo/RJavaServer

Slow service response Times : Java SecureRandom & /dev/random [duplicate]

This question already has answers here:
How to deal with a slow SecureRandom generator?
(17 answers)
Closed 2 years ago.
I am trying to debug a few slow responses served by an app deployed on Tomcat.
Right now I am focussing on SecureRandom and /dev/random (some of the other probable causes have been investigated and ruled out).
The pattern is as follows:
The first call takes exactly 30.0xy seconds after Tomcat restart (even if the request arrives 4 minutes after the Startup)
Later, some calls take exactly 15.0pq seconds (there was no specific pattern that I could establish, pq being the time approximate time taken in TP99)
The service call involves encryption and decryption (AES/ECB/PKCS5Padding).
Is it possible that SecureRandom init/repopulating is leading to this?
(Although, there is a log written in catalina.log that says "Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [28,760] milliseconds.")
Also, in order to check whether /dev/random or /dev/urandom is being used, I used the test from this question. To my surprise, I didn't see reads from either of them unlike the way it happens in the linked question.
These are the last few lines from the strace log:
3561 lstat("/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/jsse.jar", {st_mode=S_IFREG|0644, st_size=258525, ...}) = 0
3561 open("/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/jsse.jar", O_RDONLY) = 6
3561 stat("/dev/random", {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 8), ...}) = 0
3561 stat("/dev/urandom", {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
3561 open("/dev/random", O_RDONLY) = 7
3561 open("/dev/urandom", O_RDONLY) = 8
3561 unlink("/tmp/hsperfdata_xxxx/3560") = 0
What is then being used for seeding SecureRandom?
fyi, java -version
java version "1.6.0_32"
OpenJDK Runtime Environment (IcedTea6 1.13.4) (rhel-7.1.13.4.el6_5-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
I could not check your OpenJDK concrete version, but I could check jdk6-b33.
SecureRandom uses SeedGenerator to get the seed bytes
public byte[] engineGenerateSeed(int numBytes) {
byte[] b = new byte[numBytes];
SeedGenerator.generateSeed(b);
return b;
}
SeedGenerator gets the seedSource (String) from SunEntries
String egdSource = SunEntries.getSeedSource();
SunEntries tries to get the source from the system property java.security.egd first, if is not found then tries to get the property securerandom.source from the java.security properties file, if the property is not found returns a blank string.
// name of the *System* property, takes precedence over PROP_RNDSOURCE
private final static String PROP_EGD = "java.security.egd";
// name of the *Security* property
private final static String PROP_RNDSOURCE = "securerandom.source";
final static String URL_DEV_RANDOM = "file:/dev/random";
final static String URL_DEV_URANDOM = "file:/dev/urandom";
private static final String seedSource;
static {
seedSource = AccessController.doPrivileged(
new PrivilegedAction<String>() {
public String run() {
String egdSource = System.getProperty(PROP_EGD, "");
if (egdSource.length() != 0) {
return egdSource;
}
egdSource = Security.getProperty(PROP_RNDSOURCE);
if (egdSource == null) {
return "";
}
return egdSource;
}
});
}
the SeedGenerator check this value to initialize the instance
// Static instance is created at link time
private static SeedGenerator instance;
private static final Debug debug = Debug.getInstance("provider");
final static String URL_DEV_RANDOM = SunEntries.URL_DEV_RANDOM;
final static String URL_DEV_URANDOM = SunEntries.URL_DEV_URANDOM;
// Static initializer to hook in selected or best performing generator
static {
String egdSource = SunEntries.getSeedSource();
// Try the URL specifying the source
// e.g. file:/dev/random
//
// The URL file:/dev/random or file:/dev/urandom is used to indicate
// the SeedGenerator using OS support, if available.
// On Windows, the causes MS CryptoAPI to be used.
// On Solaris and Linux, this is the identical to using
// URLSeedGenerator to read from /dev/random
if (egdSource.equals(URL_DEV_RANDOM) || egdSource.equals(URL_DEV_URANDOM)) {
try {
instance = new NativeSeedGenerator();
if (debug != null) {
debug.println("Using operating system seed generator");
}
} catch (IOException e) {
if (debug != null) {
debug.println("Failed to use operating system seed "
+ "generator: " + e.toString());
}
}
} else if (egdSource.length() != 0) {
try {
instance = new URLSeedGenerator(egdSource);
if (debug != null) {
debug.println("Using URL seed generator reading from "
+ egdSource);
}
} catch (IOException e) {
if (debug != null)
debug.println("Failed to create seed generator with "
+ egdSource + ": " + e.toString());
}
}
// Fall back to ThreadedSeedGenerator
if (instance == null) {
if (debug != null) {
debug.println("Using default threaded seed generator");
}
instance = new ThreadedSeedGenerator();
}
}
if the source is
final static String URL_DEV_RANDOM = "file:/dev/random";
or
final static String URL_DEV_URANDOM = "file:/dev/urandom"
uses the NativeSeedGenerator, on Windows tries to use the native CryptoAPI on Linux the class simply extends the SeedGenerator.URLSeedGenerator
package sun.security.provider;
import java.io.IOException;
/**
* Native seed generator for Unix systems. Inherit everything from
* URLSeedGenerator.
*
*/
class NativeSeedGenerator extends SeedGenerator.URLSeedGenerator {
NativeSeedGenerator() throws IOException {
super();
}
}
and call to the superclass constructor who loads /dev/random by default
URLSeedGenerator() throws IOException {
this(SeedGenerator.URL_DEV_RANDOM);
}
so, OpenJDK uses /dev/random by default until you do not set another value in the system property java.security.egd or in the property securerandom.source of security properties file.
If you want to see the read results using strace you can change the command line and add the trace=open,read expression
sudo strace -o a.strace -f -e trace=open,read java class
the you can see something like this (I did the test with Oracle JDK 6)
13225 open("/dev/random", O_RDONLY) = 8
13225 read(8, "#", 1) = 1
13225 read(3, "PK\3\4\n\0\0\0\0\0RyzB\36\320\267\325u\4\0\0u\4\0\0 \0\0\0", 30) = 30
....
....
The Tomcat Wiki section for faster startup suggest using a non-blocking entropy source like /dev/urandom if you are experiencing delays during startup
More info: https://wiki.apache.org/tomcat/HowTo/FasterStartUp#Entropy_Source
Hope this helps.
The problem is not SecureRandom per se but that /dev/random blocks if it doesn't have enough data. You can use urandom instead but that might not be a good idea if you need cryptographically strong random seeds.
On headless Linux systems you can install the haveged daemon. This keeps /dev/random topped up with enough data so that calls don't have to wait for the required entropy to be generated.
I've done this on a Debian Aws instance and watched SecureRandom generateBytes calls drop from 25 seconds to sub millisecond (Openjdk 1.7 something, can't remember specifically what version).

All possible values os.arch in 32bit JRE and in 64bit Jre

I need latest compilation of all possible values of the os.arch property in JRE 1.6 on Linux,Solaris and Windows.
If possible please Quote the source of your findings.
I need this values to select resources in my JNLP file. Basically I need to assign different JVM memory based on whether the JRE is 32bit or 64bit.
Waiting for your answer.
Thanks
The best place where you can look for this it's in the own jdk.
Looking on java.lang.System you can see that the properties are initialized in initializeSystemClass method using initProperties method which relies on native code using JNI:
private static native Properties initProperties(Properties props);
/**
* Initialize the system class. Called after thread initialization.
*/
private static void initializeSystemClass() {
// VM might invoke JNU_NewStringPlatform() to set those encoding
// sensitive properties (user.home, user.name, boot.class.path, etc.)
// during "props" initialization, in which it may need access, via
// System.getProperty(), to the related system encoding property that
// have been initialized (put into "props") at early stage of the
// initialization. So make sure the "props" is available at the
// very beginning of the initialization and all system properties to
// be put into it directly.
props = new Properties();
initProperties(props); // initialized by the VM
...
...
}
If you check the source of this native code called from initProperties for the different platforms you can see the possible values for os.arch system property. So do it step by step:
First look at System.c to see the JNI method called from java.lang.System.initProperties. From System.c
JNIEXPORT jobject JNICALL
Java_java_lang_System_initProperties(JNIEnv *env, jclass cla, jobject props)
{
char buf[128];
java_props_t *sprops = GetJavaProperties(env);
jmethodID putID = (*env)->GetMethodID(env,
(*env)->GetObjectClass(env, props),
"put",
"(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;");
if (sprops == NULL || putID == NULL ) return NULL;
PUTPROP(props, "java.specification.version",
JDK_MAJOR_VERSION "." JDK_MINOR_VERSION);
PUTPROP(props, "java.specification.name",
"Java Platform API Specification");
PUTPROP(props, "java.specification.vendor", "Sun Microsystems Inc.");
PUTPROP(props, "java.version", RELEASE);
PUTPROP(props, "java.vendor", VENDOR);
PUTPROP(props, "java.vendor.url", VENDOR_URL);
PUTPROP(props, "java.vendor.url.bug", VENDOR_URL_BUG);
...
/* os properties */
PUTPROP(props, "os.name", sprops->os_name);
PUTPROP(props, "os.version", sprops->os_version);
// HERE IS THE `os.arch` PROPERTY :)
PUTPROP(props, "os.arch", sprops->os_arch);
So as you can see the os.arch comes from PUTPROP(props, "os.arch", sprops->os_arch); and the sprops it's achieved using java_props_t *sprops = GetJavaProperties(env);. so lets look at GetJavaProperties(env), this method it's defined in java_props.h as:
java_props_t *GetJavaProperties(JNIEnv *env);
And the implementation seems that depends on OS.
So finally looking a specific implementation for GetJavaProperties;
in Windows the possible values which this property can take are ia64, amd64, x86, or unknown. You can see from java_props_md.c file :
#if _M_IA64
sprops.os_arch = "ia64";
#elif _M_AMD64
sprops.os_arch = "amd64";
#elif _X86_
sprops.os_arch = "x86";
#else
sprops.os_arch = "unknown";
#endif
For Solaris seems more complicated since the property value in the native code comes from a Macro defined in the java_props_md.c specific for solaris as:
sprops.os_arch = ARCHPROPNAME;
And this Macro it's defined in the follow Makefile as:
OTHER_CPPFLAGS += -DARCHPROPNAME='"$(ARCHPROP)"'
So it looks like this comes from the environment, where it's compiled (sorry I'm not a C expert, I'm just guessing however maybe I can guide you a bit).
In the Linux folder in src/linux/native/ there is no java_props_md.c so I suppose that in this case take the same source as solaris (I'm guessing again...).
NOTE: I use the 1.6 version to get this values, however new values can will be added in newest java versions, so check your required version.
Hope it helps,
I ran into the same problem in 2019. Especially with regard to arm processors.
After trying it out, the raspberry pi 2 (ARMv7) seems to simply return the string arm.
The raspberry pi 3 (ARMv8) returns aarch64.
x86 64-bit desktops and servers return amd64.
Hope this helps someone.
You can also write some code like below to find out os and its archi.
import java.util.HashMap;
import java.util.Map;
import org.apache.commons.lang.SystemUtils;
public class PlatformDetection {
private String os;
private String arch;
public static String OS_WINDOWS = "windows";
public static String OS_OSX = "osx";
public static String OS_SOLARIS = "solaris";
public static String OS_LINUX = "linux";
public static String ARCH_PPC = "ppc";
public static String ARCH_X86_32 = "x86_32";
public static String ARCH_X86_64 = "x86_64";
public PlatformDetection() {
// resolve OS
if (SystemUtils.IS_OS_WINDOWS) {
this.os = OS_WINDOWS;
} else if (SystemUtils.IS_OS_MAC_OSX) {
this.os = OS_OSX;
} else if (SystemUtils.IS_OS_SOLARIS) {
this.os = OS_SOLARIS;
} else if (SystemUtils.IS_OS_LINUX) {
this.os = OS_LINUX;
} else {
throw new IllegalArgumentException("Unknown operating system " + SystemUtils.OS_NAME);
}
// resolve architecture
Map<String, String> archMap = new HashMap<String, String>();
archMap.put("x86", ARCH_X86_32);
archMap.put("i386", ARCH_X86_32);
archMap.put("i486", ARCH_X86_32);
archMap.put("i586", ARCH_X86_32);
archMap.put("i686", ARCH_X86_32);
archMap.put("x86_64", ARCH_X86_64);
archMap.put("amd64", ARCH_X86_64);
archMap.put("powerpc", ARCH_PPC);
this.arch = archMap.get(SystemUtils.OS_ARCH);
if (this.arch == null) {
throw new IllegalArgumentException("Unknown architecture " + SystemUtils.OS_ARCH);
}
}
public String getOs() {
return os;
}
public String getArch() {
return arch;
}
public void setArch(String arch) {
this.arch = arch;
}
public void setOs(String os) {
this.os = os;
}
public String toString() {
return os + "_" + arch;
}
}
Refer below Links
https://github.com/trustin/os-maven-plugin/blob/master/src/main/java/kr/motd/maven/os/Detector.java
https://github.com/rachelxqy/EligibilityCriteriaModeling/blob/57001f6d86084f074f4ca6aaff157e93ef6abf95/src/main/java/edu/mayo/bmi/medtagger/ml/util/PlatformDetection.java

Issues with SHA1 hash implementation in Android

I have two small snippets for calculating SHA1.
One is very fast but it seems that it isn't correct and the other is very slow but correct.
I think the FileInputStream conversion to ByteArrayInputStream is the problem.
Fast version:
MessageDigest md = MessageDigest.getInstance("SHA1");
FileInputStream fis = new FileInputStream("path/to/file.exe");
ByteArrayInputStream byteArrayInputStream =
new ByteArrayInputStream(fis.toString().getBytes());
DigestInputStream dis = new DigestInputStream(byteArrayInputStream, md);
BufferedInputStream bis = new BufferedInputStream(fis);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int ch;
while ((ch = dis.read()) != -1) {
byteArrayOutputStream.write(ch);
}
byte[] newInput = byteArrayOutputStream.toByteArray();
System.out.println("in digest : " +
byteArray2Hex(dis.getMessageDigest().digest()));
byteArrayOutputStream = new ByteArrayOutputStream();
DigestOutputStream digestOutputStream =
new DigestOutputStream(byteArrayOutputStream, md);
digestOutputStream.write(newInput);
System.out.println("out digest: " +
byteArray2Hex(digestOutputStream.getMessageDigest().digest()));
System.out.println("length: " +
new String(
byteArray2Hex(digestOutputStream.getMessageDigest().digest())).length());
digestOutputStream.close();
byteArrayOutputStream.close();
dis.close();
Slow version:
MessageDigest algorithm = MessageDigest.getInstance("SHA1");
FileInputStream fis = new FileInputStream("path/to/file.exe");
BufferedInputStream bis = new BufferedInputStream(fis);
DigestInputStream dis = new DigestInputStream(bis, algorithm);
// read the file and update the hash calculation
while (dis.read() != -1);
// get the hash value as byte array
byte[] hash = algorithm.digest();
Conversion method:
private static String byteArray2Hex(byte[] hash) {
Formatter formatter = new Formatter();
for (byte b : hash) {
formatter.format("%02x", b);
}
return formatter.toString();
}
I hope there is another possibility to get it running because I need the performance.
I used a high performance c++ implementation which I load with JNI.
For more details write a comment, please.
EDIT:
Requirements for JNI is the Android NDK. For Windows is needed in addition cygwin or something similar.
If you decided for cygwin, I give you some little instructions how to get it working with the NDK:
Download the setup.exe from cygwin and execute it.
Click on Next and choice Install from Internet confirm with Next.
The next two steps adjust the settings as desired and as always click Next.
Select your internet connection and the same procedure as in the final stages.
A download page will catch the eye select it or take just a download page, which is in your country. There is nothing more to say.
We need the packages make and gcc-g++. You can find them using the search in the left upper corner, click on the Skip til a version is displayed and the first field is selected. Do that what we have always done after a selection.
You will get the information, that there are dependencies, which must be resolved. It is usually not necessary to do it yourself and confirm it.
The download and installation started.
If you need you can create shortcuts otherwise click on exceptional Finish.
Download the zip file and extract the NDK to a non space containing path.
You can start now cygwin.
Navigate to the NDK. The path /cydrive gives you all available drives f.e. cd /cygdrive/d navigates to the drive with the letter D.
In the root folder of the NDK you can execute the file ndk-build with ./ndk-build. There should be an error occurs like Android NDK: Could not find application project directory !.
You have to navigate in an Android project to execute the command. So let's start with a project.
Before we can start with the project search for a C/C++ implementation of the hash algorithm. I took the code from this site CSHA1.
You should edit the source code for your requirements.
Now we can start with JNI.
You create a folder called jni in your Android project. It contains all native source files and the Android.mk (more about that file later), too.
Copy your downloaded (and edited) source files in that folder.
My java package is called de.dhbw.file.sha1, so I named my source files similar to find them easily.
Android.mk:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_LDLIBS := -llog
# How the lib is called?
LOCAL_MODULE := SHA1Calc
# Which is your main SOURCE(!) file?
LOCAL_SRC_FILES := de_dhbw_file_sha1_SHA1Calc.cpp
include $(BUILD_SHARED_LIBRARY)
Java code:
I used the AsyncTask with a ProgressDialog to give the user some feedback about the action.
package de.dhbw.file.sha1;
// TODO: Add imports
public class SHA1HashFileAsyncTask extends AsyncTask<String, Integer, String> {
// [...]
static {
// loads a native library
System.loadLibrary("SHA1Calc");
}
// [...]
// native is the indicator for native written methods
protected native void calcFileSha1(String filePath);
protected native int getProgress();
protected native void unlockMutex();
protected native String getHash();
// [...]
}
Native code (C++):
Remember accessing variables inside native code or other way around using threads needs synchronizing or you will get a segmentation fault soon!
For JNI usage you have to add #include <jni.h>.
For logging insert following include #include <android/log.h>.
Now you can log with __android_log_print(ANDROID_LOG_DEBUG, DEBUG_TAG, "Version [%s]", "19");.
The first argument is the type of message and the second the causing library.
You can see I had a version number in my code. It is very helpful because sometimes the apk builder doesn't use the new native libraries. Troubleshooting can be extremely shortened, if the wrong version is online.
The naming conventions in the native code are a little bit crasier: Java_[package name]_[class name]_[method name].
The first to arguments are always given, but depending on the application you should distinguish:
func(JNIEnv * env, jobject jobj) -> JNI call is an instance method
func(JNIEnv * env, jclass jclazz) -> JNI call is a static method
The header for the method calcFileSha1(...):
JNIEXPORT void JNICALL Java_de_dhbw_file_sha1_SHA1HashFileAsyncTask_calcFileSha1(JNIEnv * env, jobject jobj, jstring file)
The JDK delivers the binary javah.exe, which generates the header file for the native code. The usage is very simple, simply call it with the full qualified class:
javah de.dhbw.file.sha1.SHA1HashFileAsyncTask
In my case I have to give the bootclasspath additionally, because I use Android classes:
javah -bootclasspath <path_to_the_used_android_api> de.dhbw.file.sha1.SHA1HashFileAsyncTask
That would be the generated file:
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class de_dhbw_file_sha1_SHA1HashFileAsyncTask */
#ifndef _Included_de_dhbw_file_sha1_SHA1HashFileAsyncTask
#define _Included_de_dhbw_file_sha1_SHA1HashFileAsyncTask
#ifdef __cplusplus
extern "C" {
#endif
#undef de_dhbw_file_sha1_SHA1HashFileAsyncTask_ERROR_CODE
#define de_dhbw_file_sha1_SHA1HashFileAsyncTask_ERROR_CODE -1L
#undef de_dhbw_file_sha1_SHA1HashFileAsyncTask_PROGRESS_CODE
#define de_dhbw_file_sha1_SHA1HashFileAsyncTask_PROGRESS_CODE 1L
/*
* Class: de_dhbw_file_sha1_SHA1HashFileAsyncTask
* Method: calcFileSha1
* Signature: (Ljava/lang/String;)V
*/
JNIEXPORT void JNICALL Java_de_dhbw_file_sha1_SHA1HashFileAsyncTask_calcFileSha1
(JNIEnv *, jobject, jstring);
/*
* Class: de_dhbw_file_sha1_SHA1HashFileAsyncTask
* Method: getProgress
* Signature: ()I
*/
JNIEXPORT jint JNICALL Java_de_dhbw_file_sha1_SHA1HashFileAsyncTask_getProgress
(JNIEnv *, jobject);
/*
* Class: de_dhbw_file_sha1_SHA1HashFileAsyncTask
* Method: unlockMutex
* Signature: ()V
*/
JNIEXPORT void JNICALL Java_de_dhbw_file_sha1_SHA1HashFileAsyncTask_unlockMutex
(JNIEnv *, jobject);
/*
* Class: de_dhbw_file_sha1_SHA1HashFileAsyncTask
* Method: getHash
* Signature: ()Ljava/lang/String;
*/
JNIEXPORT jstring JNICALL Java_de_dhbw_file_sha1_SHA1HashFileAsyncTask_getHash
(JNIEnv *, jobject);
#ifdef __cplusplus
}
#endif
#endif
You can change the file without further notice. But do not use javah again!
Class and methods
To get a class instance you can use jclass clz = callEnv->FindClass(CALL_CLASS);. In this case is CALL_CLASS the full qualified path to the class de/dhbw/file/sha1/SHA1HashFileAsyncTask.
To find a method you need the JNIEnv and an instance of the class:
jmethodID midSet = callEnv->GetMethodID(callClass, "setFileSize", "(J)V");
The first argument is the instance of the class, the second the name of the method and the third is the signature of the method.
The signature you can get with the from JDK given binary javap.exe. Simply call it with the full qualified path of the class f.e. javap -s de.dhbw.file.sha1.SHA1HashFileAsyncTask.
You will get an result like:
Compiled from "SHA1HashFileAsyncTask.java"
public class de.dhbw.file.sha1.SHA1HashFileAsyncTask extends android.os.AsyncTas
k<java.lang.String, java.lang.Integer, java.lang.String> {
[...]
static {};
Signature: ()V
public de.dhbw.file.sha1.SHA1HashFileAsyncTask(android.content.Context, de.dhb
w.file.sha1.SHA1HashFileAsyncTask$SHA1AsyncTaskListener);
Signature: (Landroid/content/Context;Lde/dhbw/file/sha1/SHA1HashFileAsyncTas
k$SHA1AsyncTaskListener;)V
protected native void calcFileSha1(java.lang.String);
Signature: (Ljava/lang/String;)V
protected native int getProgress();
Signature: ()I
protected native void unlockMutex();
Signature: ()V
protected native java.lang.String getHash();
Signature: ()Ljava/lang/String;
[...]
public void setFileSize(long);
Signature: (J)V
[...]
}
If the method is found the variable is not equal 0.
Calling the method is very easy:
callEnv->CallVoidMethod(callObj, midSet, size);
The first argument is the given jobject from the "main" method and I think the others are clear.
Remember that you can call from native code although private methods of the class, because the native code is part of it!
Strings
The given string would be converted with following code:
jboolean jbol;
const char *fileName = env->GetStringUTFChars(file, &jbol);
And the other way:
TCHAR* szReport = new TCHAR;
jstring result = callEnv->NewStringUTF(szReport);
It can be every char* variable.
Exceptions
Can be thrown with the JNIEnv:
callEnv->ThrowNew(callEnv->FindClass("java/lang/Exception"),
"Hash generation failed");
You can also check if there is an exception occurred also with JNIEnv:
if (callEnv->ExceptionOccurred()) {
callEnv->ExceptionDescribe();
callEnv->ExceptionClear();
}
Specifications
Java Native Interface Specifications
Build/Clean
Build
After we have created all files and filled them with content, we can build it.
Open cygwin, navigate to the project root and execute from there the ndk-build, which is in the NDK root.
This start the compile, if it is success you will get an output like that:
$ /cygdrive/d/android-ndk-r5c/ndk-build
Compile++ thumb : SHA1Calc <= SHA1Calc.cpp
SharedLibrary : libSHA1Calc.so
Install : libSHA1Calc.so => libs/armeabi/libSHA1Calc.so
If there is any error, you will get the typical output from the compiler.
Clean
Open cygwin, switch in your Android project and execute the command /cygdrive/d/android-ndk-r5c/ndk-build clean.
Build apk
After you have build the native libraries you can build your project. I've found clean, it is advantageous to use the eclipse feature clean project.
Debugging
Debugging of java code isn't different as before.
The debugging of c++ code will follow in the next time.
Do this:
MessageDigest md = MessageDigest.getInstance("SHA1");
InputStream in = new FileInputStream("hereyourinputfilename");
byte[] buf = new byte[8192];
for (;;) {
int len = in.read(buf);
if (len < 0)
break;
md.update(buf, 0, len);
}
in.close();
byte[] hash = md.digest();
Performance comes from handling data by blocks. An 8 kB buffer, as here, ought to be blocky enough. You do not have to use a BufferedInputStream since the 8 kB buffer also serves as I/O buffer.
The reason the fast one is fast and incorrect is (I think) that it is not hashing the file contents!
FileInputStream fis = new FileInputStream("C:/Users/Ich/Downloads/srware_iron.exe");
ByteArrayInputStream byteArrayInputStream =
new ByteArrayInputStream(fis.toString().getBytes());
The fis.toString() call does not read the contents of the file. Rather it gives you a string that (I suspect) looks something like this:
"java.io.FileInputStream#xxxxxxxx"
which you are then proceeding to calculate the SHA1 hash for. FileInputStream and its superclasses do not override Object::toString ...
The simple way to read the entire contents of an InputStream to a byte[] is to use an Apache Commons I/O helper method - IOUtils.toByteArray(InputStream).
public void computeSHAHash(String path)// path to your file
{
String SHAHash = null;
try
{
MessageDigest md = MessageDigest.getInstance("SHA1");
InputStream in = new FileInputStream(path);
byte[] buf = new byte[8192];
int len = -1;
while((len = in.read(buf)) > 0)
{
md.update(buf, 0, len);
}
in.close();
byte[] data = md.digest();
try
{
SHAHash = convertToHex(data);
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
} catch (NoSuchAlgorithmException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Toast.makeToast(getApplicationContext(),"Generated Hash ="+SHAHash,Toast.LENGTH_SHORT).show();
}
private static String convertToHex(byte[] data) throws java.io.IOException
{
StringBuffer sb = new StringBuffer();
String hex = null;
hex = Base64.encodeToString(data, 0, data.length, NO_OPTIONS);
sb.append(hex);
return sb.toString();
}

Categories

Resources