Develop Reference

Why am I obtaining different results when running curl command from Terminal and in Java?

I have learned lots of suggestions to run curl in Java or its derivatives. For example, curl command in Java, using curl command in Java, etc.
Also, I have figured out how to fetch the metadata of a given resource using DOI. From this instruction, I am very interested in running this curl command using a small snippet in Java to handle the result.
Let's give an example. The URL is http://dx.doi.org/10.1016/j.immuni.2015.09.001.
Running curl command from a terminal
curl -LH "Accept: application/x-bibtex" http://dx.doi.org/10.1016/j.immuni.2015.09.001
The output looks like
#article{Biswas_2015,
doi = {10.1016/j.immuni.2015.09.001},
url = {https://doi.org/10.1016%2Fj.immuni.2015.09.001},
year = 2015,
month = {sep},
publisher = {Elsevier {BV}},
volume = {43},
number = {3},
pages = {435--449},
author = {Subhra~K. Biswas},
title = {Metabolic Reprogramming of Immune Cells in Cancer Progression},
journal = {Immunity}
Running this curl command in Groovy
Recycling some codes sharing on this site, I have written the process as below.
Map result = [:]
String command = "curl -LH 'Accept: application/x-bibtex' http://dx.doi.org/10.1016/j.immuni.2015.09.001"
Process process = Runtime.getRuntime().exec(command)
InputStream stream = process.getInputStream()
result.put("data", stream.text)
process.destroy()
What I obtain is the whole page in HTML rather than a BibTeX formatted form as what is my expectation.
The question is: what am I doing wrong here? Are there any of you that have experienced with that issue?

Using exec is not a shell - you can't and don't have to quote for
a shell, that is not there. Further exec(String) uses by default
a string tokenizer (which basically splits at whitespace) to make it
particularly useless for any slightly advanced usecase.
You are most likely always better off to use the version that accepts
a string array for the command (+ args).
What you where effectively calling looked like this (note, that the
command gets split at whitespace -- so I used \' to make my shell
ignore that):
# curl -LH \'Accept: application/x-bibtex\' http://dx.doi.org/10.1016/j.immuni.2015.09.001
curl: (6) Could not resolve host: application
... HTML ...
The shortest route using groovy looks like this (note that exec also
has a version for passing in an array of strings):
groovy:000> ["curl", "-LH", "Accept: application/x-bibtex", "http://dx.doi.org/10.1016/j.immuni.2015.09.001"].execute().text
===> #article{Biswas_2015,
9doi = {10.1016/j.immuni.2015.09.001},
9url = {https://doi.org/10.1016%2Fj.immuni.2015.09.001},
9year = 2015,
9month = {sep},
9publisher = {Elsevier {BV}},
9volume = {43},
9number = {3},
9pages = {435--449},
9author = {Subhra~K. Biswas},
9title = {Metabolic Reprogramming of Immune Cells in Cancer Progression},
9journal = {Immunity}
}
If you need "shell-isms", then use ["sh", "-c", command] instead.

Invoke PHP code from Java (Scala) and get result

This seems to have been asked in several places and has been marked as "closed" and "off-topic". However, people seem to have this problem constantly
invoking a php method from java (closed)
Calling PHP from Java (closed)
How can I run PHP code within a Java application? (closed)
This answer in the last question partly answers this but does not clarify how to read the outputs.
I finally found the answer to the question:
How do I run a PHP program from within Java and obtain its output?
To give more context, someone has given me a PHP file containing code for some method foo that returns a string. How do we invoke this from JVM?
Searching on Google is not helpful as all articles I found don't explain how to call PHP from Java but rather Java from PHP.
The answer below explains how to do this using the PHP/Java bridge.
The answer is in Scala but would be easy to read for Java programmers.

Code created from this SO answer and this example:
package javaphp
import javax.script.ScriptEngineManager
import php.java.bridge._
import php.java.script._
import php.java.servlet._
object JVM{ // shared object for PHP/JVM communication
var out = ""
def put(s:String) = {
out = s
}
}
object Test extends App {
val engine = (new ScriptEngineManager).getEngineByExtension("php")
val oldCode = """
<?php
function foo() {
return 'hello';
// some code that returns string
}
?>
"""
val newCode = """
<?php
$ans = foo();
java('javaphp.JVM')->put($ans);
?>
"""+oldCode
// below evaluates and returns
JVM.out = "" //reset shared output
engine.eval(newCode)
println("output is : "+JVM.out) // prints hello
}
To run this file:
Install PHP, Scala and set path correctly. Then create a file php.scala with the above code. Then run:
scalac php.scala
and
scala javaphp.Test

Base64 encode script for user data (AWS SDK for Java API)

How do I turn what was in a bash script file into a base64 encoded string that will work with Amazon's Java API? This code expresses what I'm trying to do but results in an exception:
...Caused by: com.amazonaws.AmazonServiceException: Invalid BASE64 encoding of user data (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue...
String startupUserData = "#!/bin/bash cd /home/ubuntu/myTestDir; mvn test -PmyBuild";
startupUserData = org.apache.commons.codec.binary.Base64.encodeBase64String(startupUserData.getBytes());
runRqst.withImageId(_computerAmi)
.withInstanceType(instanceSize)
.withMinCount(hwRequest.numHwComputers)
// .withMaxCount(Utils.MAX_EC2_INSTANCES_AT_A_TIME) // NOT THIS
.withMaxCount(hwRequest.numHwComputers)
.withKeyName(_keyName)
.withSecurityGroups(_securityGroup)
.withUserData(startupUserData);
ec2.runInstances(runRqst);
Edit:
com.amazonaws.util.Base64.encodeAsString() quells the exception but the script still doesn't execute. How should the string be formatted? Should there be carriage returns?

Had problem to use answers above. My solution was that I imported
import com.amazonaws.util.Base64;
and then
String initScript = "sudo -i apt-get install docker.io";
RunInstancesRequest run_request = new RunInstancesRequest()
.withImageId(ami_id)
.withInstanceType(InstanceType.T2Micro)
.withMaxCount(1)
.withMinCount(1)
.withUserData(Base64.encodeAsString(initScript.getBytes()));

For encoding you need to use Encoding class provided by Amazon.
i.e amazon.webservices.common.Encoding
It contains API EncodeBase64() which returns you encoded data.
public static String EncodeBase64(byte[] rawData) {
return Base64.encodeBytes(rawData);
}

com.amazonaws.util.Base64.encodeAsString works for me. I use \n to separate multi-lin. You can also verify it at /var/lib/cloud/instance/user-data.txt the same http://169.254.169.254/user-data returns.

Run GraphicsMagick compare command using im4java + gm4java

I am trying to do a GraphicsMagick compare using im4java and gm4java. The GraphicsMagick command I'm using is like this:
gm compare -maximum-error 0 -metric MAE -highlight-style xor -hilight-color red -file C:/output/diffFile.pdf C:/input/file1.pdf C:/input/file2.pdf
I'm trying to translate that into Java. I know that im4java was originally built for ImageMagick and their commands may differ. Is it possible to run the above compare using im4java plus gm4java?
I've tried this:
SimpleGMService service = new SimpleGMService();
service.setGMPath("C:/path/to/graphicsMagick/gm.exe");
try
{
GMConnection connection = service.getConnection();
try {
GMBatchCommand command = new GMBatchCommand(service, "compare");
// create the operation, add images and operators/options
IMOperation op = new IMOperation();
op.metric("MAE");
op.addRawArgs("-file C:/output/diffFile.pdf");
op.addImage();
op.addImage();
ArrayListOutputConsumer output = new ArrayListOutputConsumer();
command.setOutputConsumer(output);
//debug
command.createScript("C:/output/myscript.bat",op);
command.run(op, "C:/input/file1.pdf", "C:/input/file2.pdf");
....
The above gives me the error:
org.im4java.core.CommandException: compare: Unrecognized option (-file C:/output/diffFile.pdf)

You can attain this by using im4java alone or im4java+gm4java. What gm4java gives you is the performance when you need to process a large number of images.
The problem you are getting is due to the improper use of the addRawArgs() method. Each argument you have in your command line needs to be added as individual arguments instead of all together in one string.
Try:
op.addRawArgs("-file", "C:/output/diffFile.pdf");

Reading BSTR in JAVA through JACOB

I am trying to access a device through a COM object in a JAVA interface.
The particular call (as described by the manufacturer) is:
Name: ScanUSB
Parameters: [out] VARIANT* serialNumbers
Use: serialNumbers is a pointer to a VARIANT containing an array of BSTR.
The exact call doesn't matter, but I need to feed it a BSTR array through the Java interface. A VB demo for the COM interface simply does this with the commandlm_Co1.ScanUSB(snNum), with Dim snNum As Object = Nothing. The Items in snNum are then displayed in a dropdown menu.
I am trying to do this with JACOB, as I have had the most luck with communication. This is more or less what I am using:
import com.jacob.com.Variant;
import com.jacob.com.Dispatch;
public class JunoReader {
public JunoReader() {
Dispatch oOphirLink = new Dispatch("clsid:{--the id of the device here--}");
Variant snNum = new Variant();
Variant testing = Dispatch.call(oOphirLink,"ScanUSB", snNum);
println(testing);
}
}
When I run this, everything compiles properly and I can confirm that I am communicating with the device, but all I get back is the null Variant that I put in.
My question is: How can I feed a BSTR array to a COM object through a JAVA interface?
Thanks!

So, one month and zero free time later, I have an answer to my own question and that answer is "Use Python".
Python allows users to access COM objects with the comtypes module, effectively with a single command:
from comtypes.client import CreateObject
target_guid = CreateObject("{TARGET-COM-CLSID-HERE}")
This allows python to talk with whatever the COM object is, so there was none of the trouble that Java was giving me. The single line: target_guid.ScanUSB() produced (u'717610',), the serial number of the target device. Notice that Python has no trouble reading the Unicode produced by the COM object.
The second trick involves Python's COM server, generated with the win32com module. Because the COM servers can be registered to Windows, I don't have to worry about where the dll is located. All I need is the clsid. To build the python COM server, I followed the instructions for a "quick start to server side COM and Python". So what does it all look like?
Python code:
class OphirPyCOM:
_reg_clsid_ = "{PYTHON-COM-CLSID-HERE}"
_reg_desc_ = "Python COM server"
_reg_progid_ = "Python COM server"
_public_methods_ = [ 'Hello',
'ConnectOphir',
'ScanUSB', """More methods here"""
]
_public_attrs_ = ["""Some public attributes"""]
_readonly_attrs_ = ["""some read-only attributes"""]
def __init__(self):
"""some variables declared here"""
def Hello(self, who):
"""Verifies a connection"""
return "{PYTHON-COM-CLSID-HERE}" + str(who)
def ConnectOphir(self,clsid):
"""Connects to the target COM Object"""
from comtypes.client import CreateObject
self.target_guid = CreateObject(clsid)
return "OphirLMMeasurement object created."
def ScanUSB(self):
"""Communicates with the target device"""
self.connected_inst = self.target_guid.ScanUSB()
for i in range(0,len(self.connected_inst)):
self.inst_list.append(str(self.connected_inst[i]))
return self.inst_list
if __name__ == "__main__":
# use 'python com.py' to register the COM server
# use 'python com.py --unregister' to unregister it
print "Registering COM server..."
import win32com.server.register
win32com.server.register.UseCommandLine(OphirPyCOM)
We can run that with the command line and get it registered. Then we take the values over to Java with JACOB:
import com.jacob.com.Variant;
import com.jacob.com.Dispatch;
public class JunoReader {
String pyClsid = "{PYTHON-COM-CLSID-HERE}"; // This is where your python COM clsid goes
String opClsid = "{TARGET-COM-CLSID-HERE}"; // This is where your ultimate target clsid goes
public JunoReader() {
_pyClsid = "clsid:" + pyClsid
// This finds the COM object location:
Dispatch oOphirLink = new Dispatch(_pyClsid);
}
public String HandShake() {
String _talkBack = "is connected.";
Variant _handShake = Dispatch.call(oOphirLink,"Hello",_talkBack); // I am trying to look for the Juno...
return (_handShake.toString());
}
public String ConnectOphir() {
Variant _connectOphir = Dispatch.call(oOphirLink,"ConnectOphir", opClsid); // Connect to the target COM object
return (_connectOphir.toString());
}
public String ScanUSB() {
Variant _serialNumberList = Dispatch.call(oOphirLink,"ScanUSB"); // This scans the USB ports for devices
return (_serialNumberList.toString());
}
}
calling JunoReader.ScanUSB() produces: 717610, exactly like it is supposed to. Implementing the subsequent methods of the manufacturer dll allowed me to read data from this device into a Java applet. Problem solved.
A caveat: you may need to unregister the Python COM every time you change the change the source file and then re-register with a different clsid. Java was having a tough time establishing a connection to the updated code unless I used a new clsid every time I changed the file.
Why did I spend all this time typing this up? Because most of the advice related to reading native data types into Java involved some version of JNI and JNA. I spent weeks trying to get the tutorials to compile and didn't make any progress. On the other hand, I thought of this approach yesterday and can now communicate with my device through Java. The Python COM server provided a simple, straightforward way to interface Java with native applications. No UnsatisfiedLinkErrors, no Can't find librarys, no Classpath and JAVA_HOME issues. I didn't need to learn C or C++ and all the Python tutorials worked as described with no necessary modification.
To summarize, if you are having trouble reading native data types into Java, just set up a Python COM server and let Python's dynamic typing do the work for you.