Develop Reference

Which encoding for ProcessBuilder parameters

Using ProcessBuilder, I need to be able to send non-ASCII parameters to another Java program.
In this case, a program Abc needs to send e.g. Arabic characters to Def program through the parameters. I have control of Abc code, but not of Def.
Using the normal way of ProcessBuilder without any playing with the encoding, it was mentioned here, it is not possible. Def recieves question marks "?????".
However, I am able to get some result, but different encodings can be used for different scenarios.
E.g. I am trying all encodings to send to the recipient, and comparing the result of what is expected.
Windows, IntelliJ console:
Default charset: UTF-8
Found charsets: windows-1252, windows-1254 and windows-1258
Windows, command prompt:
Default charset: windows-1252
Found charsets: CESU-8 and UTF-8
Ubuntu, command prompt:
Default charset: ISO-8859-1
Found charsets: ISO-2022-CN, ISO-2022-KR, ISO-8859-1, ISO-8859-15, ISO-8859-9, x-IBM1129, x-ISO-2022-CN-CNS and x-ISO-2022-CN-GB
My question is: how to programmatically know which correct encoding to use, since I need to have something universal?
In other words, what is the relation between the default charset and the found ones?
public class Abc {
private static final Path PATH = Paths.get("."); // With maven: ./target/classes
public static void main(String[] args) throws Exception {
var string = "hello أحمد";
var bytes = string.getBytes();
System.out.println("Original string: " + string);
System.out.println("Default charset: " + Charset.defaultCharset());
for (var c : Charset.availableCharsets().values()) {
var newString = new String(bytes, c);
var process = new ProcessBuilder().command("java", "-cp",
PATH.toAbsolutePath().toString(),
"Def", newString).start();
process.waitFor();
var output = asString(process.getInputStream());
if (output.contains(string)) {
System.out.println("Found " + c + " " + output);
}
}
}
private static String asString(InputStream is) throws IOException {
try (var reader = new BufferedReader(new InputStreamReader(is))) {
var builder = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
if (builder.length() != 0) {
builder.append(System.lineSeparator());
}
builder.append(line);
}
return builder.toString();
}
}
}
public class Def {
public static void main(String[] args) {
System.out.println(args[0]);
}
}

Under the hood, what's actually being passed around is bytes, not chars. Normally, you'd expect the java method that ends up turning characters into bytes to have an overload that lets you specify charset, but, for whatever reason, it does not exist here.
How it should work is thusly:
You pass a string to ProcessBuilder
PB will turn that string into bytes using Charset.defaultCharset() (why? Because PB is all about making the OS do things, and the default charset reflects the OS's preferred charset).
These bytes are then fed to the process.
The process starts up. If it is java, and we're talking the args in psv main(String[] args), the same is done in reverse: Java takes the bytes and turns them back to characters via Charset.defaultCharset(), again.
This does show an immediate issue: If the default charset is not capable of representing a certain character, then in theory you are out of luck.
That would strongly suggest that using java to fire up java.exe should ordinarily mean you can pass whatever you want (unless the characters involved aren't representable in the system's charset).
Your code is odd. In particular, this line is the problem:
var bytes = string.getBytes();
This is short for string.getBytes(Charset.defaultCharset()). So now you have your bytes in the provided charset.
var newString = new String(bytes, c);
and now you're taking those bytes and turning them into a string using a completely different charset. I'm not sure what you're trying to accomplish with this. Pure gobbledygook would come out.
In other words, what is the relation between the default charset and the found ones?
What do you mean by 'found ones'? The string "Found charsets" appears nowhere in your code. If you mean: What Charset.availableCharsets() returns - there is no relationship at all. availableCharsets isn't relevant for ProcessBuilder.

One possibility is to convert your String to Unicode sequences string and then pass it to another process and there convert it back to a regular String. String of Unicode sequences will always contain ASCI characters only. Here is how it may look like:
String encoded = StringUnicodeEncoderDecoder.encodeStringToUnicodeSequence("hello أحمد"));
The result will be that String encode will hold this value:
"\u0068\u0065\u006c\u006c\u006f\u0020\u0623\u062d\u0645\u062f"
This String you can safely pass to another process. In that other process, you can do the following:
String originalString = StringUnicodeEncoderDecoder.decodeUnicodeSequenceToString(encodedString);
And the result will be that originalString will now hold this value:
"hello أحمد"
Class StringUnicodeEncoderDecoder could be found in an Open Source library called MgntUtils. You can get this library as Maven Artifact or get it on Github (including source code and JavaDoc). JavaDoc online is available here
This library and this particular feature is used and well tested by multiple users.
Disclamer: This library is written by me

Elevate Java application while running

A nasty problem popped out with my software. I am making a program that interacts with another existing software (a game). User has reported that he runs the game with administrator privileges and under that circumstances, my program stops working for him.
Short investigation revealed that some people really need to run the game under administrator account and some don't. It would be great if my program would be able to detect this and warn user if the game is running under administrator account:
If the user clicks "Elevate", I'd like to ask windows to elevate the java.exe running my jar file and invoke the typical UAC dialog.
Obviously, this time the question would not be about java updater but JRE
My question is: Is this possible? Can windows elevate my java.exe instance's privilege? Does java have a way to do it? Or can I use command line command?
I want to avoid restarting the program (though it wouldn't probably be such a big deal).
Edit:
If you look in the comments, you'll see that there's no avoiding the restart of an application - process can only start elevated, not become elevated. This kinda shifts the question, unfortunately. Basically, it now sounds more like: "How to restart my application with admin rights?". Unless, of course, there's a trick like two java.exe sharing one jar...

If still of interest: In Windows 7 my JavaElevator works. It elevates a running Java process when used in the main method of the Java application. Simply add -elevate as last program parameter and use the elevator in the main method.
The elevator class:
package test;
import com.sun.jna.Native;
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.Kernel32Util;
import com.sun.jna.platform.win32.ShellAPI;
import com.sun.jna.platform.win32.WinDef;
/**
* Elevates a Java process to administrator rights if requested.
*/
public class JavaElevator {
/** The program argument indicating the need of being elevated */
private static final String ELEVATE_ARG = "-elevate";
/**
* If requested, elevates the Java process started with the given arguments to administrator level.
*
* #param args The Java program arguments
* #return The cleaned program arguments
*/
public static String[] elevate(String[] args) {
String[] result = args;
// Check for elevation marker.
boolean elevate = false;
if (args.length > 0) {
elevate = args[args.length - 1].equals(ELEVATE_ARG);
}
if (elevate) {
// Get the command and remove the elevation marker.
String command = System.getProperty("sun.java.command");
command = command.replace(ELEVATE_ARG, "");
// Get class path and default java home.
String classPath = System.getProperty("java.class.path");
String javaHome = System.getProperty("java.home");
String vm = javaHome + "\\bin\\java.exe";
// Check for alternate VM for elevation. Full path to the VM may be passed with: -Delevation.vm=...
if (System.getProperties().contains("elevation.vm")) {
vm = System.getProperty("elevation.vm");
}
String parameters = "-cp " + classPath;
parameters += " " + command;
Shell32.INSTANCE.ShellExecute(null, "runas", vm, parameters, null, 0);
int lastError = Kernel32.INSTANCE.GetLastError();
if (lastError != 0) {
String errorMessage = Kernel32Util.formatMessageFromLastErrorCode(lastError);
errorMessage += "\n vm: " + vm;
errorMessage += "\n parameters: " + parameters;
throw new IllegalStateException("Error performing elevation: " + lastError + ": " + errorMessage);
}
System.exit(0);
}
return result;
}
}
Usage in the main method of the Java application:
public static void main(String[] args) {
String[] args1 = JavaElevator.elevate(args);
if (args1.length > 0) {
// Continue as intended.
...
I know, this is a very basic implementation - sufficient for one of my daily hiccups: Starting an elevated process from Eclipse. But maybe it points someone in some dicrection...

As has been pointed in comments, sadly the Java (or any other process) cannot be elevated while running. While in the case of JWM, it could be theoretically possible to move whole program context from normal user java.exe to elevated one, I don't think it's possible. I hope some day someone will come and tell me I'm wrong.
Surprisingly, even with restart in place, this was a tricky task that took me a while to figure out.
The non java part
First, how do we exactly run a program elevated from command line? There's an answer and you can see it's not simple. But we can break it to this VBS script:
Set UAC = CreateObject("Shell.Application")
UAC.ShellExecute "program name", "command line parameters", "working directory", "runas", 1
Soon, it also turns out that we won't have any success running java.exe from VBS script. In the end, I decided to run a helper batch file. Finally, here (answer to question in the last link) we have a complete set of two scripts which really run the given .jar file elevated. Here's improved version that allows quick testing by drag'n'dropping the Jar file on it:
' Require first command line parameter
if WScript.Arguments.Count = 0 then
MsgBox("Jar file name required.")
WScript.Quit 1
end if
' Get the script location, the directorry where it's running
Set objShell = CreateObject("Wscript.Shell")
strPath = Wscript.ScriptFullName
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.GetFile(strPath)
strFolder = objFSO.GetParentFolderName(objFile)
'MsgBox(strFolder)
' Create the object that serves as runnable something
Set UAC = CreateObject("Shell.Application")
' Args:
' path to executable to run
' command line parameters - first parameter of this file, which is the jar file name
' working directory (this doesn't work but I use it nevertheless)
' runas command which invokes elevation
' 0 means do not show the window. Normally, you show the window, but not this console window
' which just blinks and disappears anyway
UAC.ShellExecute "run-normally.bat", WScript.Arguments(0), strFolder, "runas", 0
WScript.Quit 0
The Java part
Java part is more straightforward. What we need to do is to open new process and execute the prepared scripts in it.
/**
* Start this very jar file elevated on Windows. It is strongly recommended to close any existing IO
* before calling this method and avoid writing anything more to files. The new instance of this same
* program will be started and simultaneous write/write or read/write would cause errors.
* #throws FileNotFoundException if the helper vbs script was not found
* #throws IOException if there was another failure inboking VBS script
*/
public void StartWithAdminRights() throws FileNotFoundException, IOException {
//The path to the helper script. This scripts takes 1 argument which is a Jar file full path
File runAsAdmin = new File("run-as-admin.vbs");;
//Our
String jarPath;
//System.out.println("Current relative path is: " + s);
try {
jarPath = "\""+new File(Main.class.getProtectionDomain().getCodeSource().getLocation().toURI().getPath()).getAbsolutePath()+"\"";
} catch (URISyntaxException ex) {
throw new FileNotFoundException("Could not fetch the path to the current jar file. Got this URISyntax exception:"+ex);
}
//If the jar path was created but doesn't contain .jar, we're (most likely) not running from jar
//typically this happens when running the program from IDE
//These 4 lines just serve as a fallback in testing, should be deleted in production
//code and replaced with another FileNotFoundException
if(!jarPath.contains(".jar")) {
Path currentRelativePath = Paths.get("");
jarPath = "\""+currentRelativePath.toAbsolutePath().toString()+"\\AutoClient.jar\"";
}
//Now we check if the path to vbs script exists, if it does we execute it
if(runAsAdmin.exists()) {
String command = "cscript \""+runAsAdmin.getAbsolutePath()+"\" "+jarPath;
System.out.println("Executing '"+command+"'");
//Note that .exec is asynchronous
//After it starts, you must terminate your program ASAP, or you'll have 2 instances running
Runtime.getRuntime().exec(command);
}
else
throw new FileNotFoundException("The VBSScript used for elevation not found at "+runAsAdmin.getAbsolutePath());
}

This is my version. It creates a VBScript script, then executes it. This only works if the program that is being run is in a jar file, so you will have to run your IDE as administrator to actually test your program.
public static void relaunchAsAdmin() throws IOException {
relaunchAsAdmin(ThisClass.class); //Change ThisClass to the class that this method is in
}
public static void relaunchAsAdmin(Class<?> clazz) throws IOException {
if(isCurrentProcessElevated()) {
return;
}
final String dir = System.getProperty("java.io.tmpdir");
final File script = new File(dir, "relaunchAsAdmin" + System.nanoTime() +
".vbs");
try {
script.createNewFile();
OutputStreamWriter osw = new OutputStreamWriter(new FileOutputStream(script));
osw.append("Set s=CreateObject(\"Shell.Application\")" + ln + "s.ShellExecute \"" +
System.getProperty("java.home") + "\\bin\\java.exe" + "\",\"-jar \"\"" +
new File(clazz.getProtectionDomain().getCodeSource(
).getLocation().toURI()).getAbsolutePath() + "\"\"\",,\"runas\",0" +
ln + "x=createObject(\"scripting.fileSystemObject\").deleteFile(" +
"WScript.scriptfullname)");
osw.close();
if(System.getenv("processor_architecture").equals("x86")) {
Runtime.getRuntime().exec("C:\\Windows\\System32\\wscript.exe \"" +
script.getAbsolutePath() + "\"");
} else {
Runtime.getRuntime().exec("C:\\Windows\\SysWoW64\\wscript.exe \"" +
script.getAbsolutePath() + "\"");
}
} catch(URISyntaxException e) {
e.printStackTrace();
}
Runtime.getRuntime().exit(0);
}
Note that it is a bit messy. I have been using this method before, so it has been line wrapped to 100 characters (except the comment I wrote for this answer). The
isCurrentProcessElevated()
method will have to be implemented in one way or another. You could try using JNI, or you could use a pure Java method, such as writing in the Program Files or System32 directory and seeing if it failed.
Obviously, this solution will only work on Windows. I never needed to elevate on Linux or Mac systems (mainly because I don't have any Mac systems, and I don't use Linux - I just play with it).

command fails from exec() but works on terminal

I'm trying to convert pdf to txt by using Java. I've tried Apache PDFBox but, for some weird reason, it doesn't convert the whole document. For this reason I decided to use pdftotext by executing a Runtime.getRuntime().exec() call. The problem is that, while on my terminal pdftotext works flawlessly, the exec() call gives me error code 1 (sometimes even 99).
Here's the call:
pdftotext "/home/www-data/CANEFS_TEST/Hello/ciao.pdf" "/tmp/ciao.pdf.txt"
Here's the code
private static File callPDF2Text(File input,File output){
assert input.exists();
assert Utils.getExtension(input).equalsIgnoreCase("pdf");
assert Utils.getExtension(output).equalsIgnoreCase("txt") : output.getAbsoluteFile().toString();
Process p=null;
try {
System.out.println(String.format(
PDF2TXT_COMMAND,
input.getAbsolutePath(),
output.getAbsolutePath()));
p=Runtime.getRuntime().exec(String.format(
PDF2TXT_COMMAND,
input.getAbsolutePath(),
output.getAbsolutePath()));
p.waitFor();
if (p.exitValue()!=0){
throw new RuntimeException("exit value for pdftotext is "+p.exitValue());
}
} catch (Exception e) {
throw new RuntimeException(e);
}
return output;
}
Here's PDF2TXT_COMMAND string definition:
public static final String PDFTXT_COMMAND="pdftotext \"%s\" \"%s\"";
I know that usually these kinds of errors are caused by the permission setup. So, here 's the output of ls -l command on the Hello folder:
ls -l /home/www-data/CANEFS_TEST/Hello/
total 136
-rwxrwxr-- 1 www-data www-data 136041 mar 27 16:31 ciao.pdf
Also, note that the user creating the process is koldar, which is in the group www-data itself.
Thank you for your time and patience!

Don't use " in your format string... These chars are specially parsed by the shell and you don't use a shell to launch the command...
I can suggest you to use exec(String []) not exec(String) so that you will be able to separate each arg of your command:
String []command = new String[3];
command[0] = "pdftotext";
command[1] = input.getAbsolutePath();
command[2] = output.getAbsolutePath();
Runtime.getRuntime().exec(command);
That should work. If it doesn't, that may be a question of access rights on dir.

issue with running a perl script from java

I'm trying to run a Perl script file from java code but it's not working with me. I modified the Perl script and put the arguments in it instead of passing them via java code. The script works fine when running it from the command line but it's not working inside java code, always prints "wrong"!!. I wrote another Perl script (test.pl) and it's working but the desired script doesn't?? I'm working in netbeans7.3.1 (ubuntu).
Here is my code:
package program;
import java.io.*;
//import java.lang.ProcessBuilder;
/**
*
* #author seed
*/
public class Program {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws IOException,Exception {
File input = new File("//home//seed//Downloads//MADA-3.2//sample");
FileOutputStream out = new FileOutputStream(input);
PrintWriter p = new PrintWriter(out);
String s = "قصدنا في هذا القول ذكر";
p.println(s);
p.close();
Process pro = Runtime.getRuntime().exec("perl /home/seed/Downloads/MADA+TOKAN.pl");
pro.waitFor();
if(pro.exitValue() == 0)
{
System.out.println("Command Successful");
}
else{
System.out.print("wrong");}
// TODO code application logic here
}
}

My guess is that some kind of string/path conversion issue.
I see utf8 strings in your code, maybe the path is converted to something.
The filename (MADA+TOKAN.pl) contain special char, it would be better MADAplusTOKAN.pl.
Also your string in script and in question are not the same: (MADA 3.2 != MADA-3.2)
perl MADA+TOKAN.pl config=/home/seed/Downloads/mada/MADA-3.2/config files/template.madaconfig file=/home/seed/Downloads/mada/MADA 3.2/inputfile
vs
perl MADA+TOKAN.pl config=/home/seed/Downloads/MADA-3.2/config-files/template.madaconfig file=/home/seed/Downloads/MADA-3.2/sample

It sounds like it is finding your perl script and executing it, since test.perl and MADA.perl run OK.
It does sound like the arguments being passed in to the perl script are not what was expected. Can you modify the perl script to echo all its input parameters to a file?

Java: How to detect (and change?) encoding of System.console?

I have a program which runs on a console and its Umlauts and other special characters are being output as ?'s on Macs. Here's a simple test program:
public static void main( String[] args ) {
System.out.println("höhößüä");
System.console().printf( "höhößüä" );
}
On a default Mac console (with default UTF-8 encoding), this prints:
h?h????
h?h????
But after manually setting the Mac terminal's encoding to "Mac OS Roman", it correctly printed
höhößüä
höhößüä
Note that on Windows systems using System.console() works:
h÷h÷▀³õ
höhößüä
So how do I make my program...rolleyes..."run everywhere"?

Try the following command-line argument when starting your application:
-Dfile.encoding=utf-8
This changes the default encoding of the JVM for I/O operations.
You can also try:
System.setOut(new PrintStream(System.out, true, "utf-8"));

Epaga: have a look right here. You can set the output encoding in a printstream - just have to determine or be absolutely sure about which is being set.
import java.io.PrintStream;
import java.io.UnsupportedEncodingException;
public class Test {
public static void main (String[] argv) throws UnsupportedEncodingException {
String unicodeMessage =
"\u7686\u3055\u3093\u3001\u3053\u3093\u306b\u3061\u306f";
PrintStream out = new PrintStream(System.out, true, "UTF-8");
out.println(unicodeMessage);
}
}
To determine the console encoding you could use the system command "locale" and parse the output which - on a german UTF-8 system looks like:
LANG="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_CTYPE="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_ALL=