Following code gets stuck(which I think is blocking I/O) many times (works some time).
def static executeCurlCommand(URL){
def url = "curl " + URL;
def proc = url.execute();
def output = proc.in.text;
return output;
}
But when I changes the code to
def static executeCurlCommand(URL){
def url = "curl " + URL;
def proc = url.execute();
def outputStream = new StringBuffer();
proc.waitForProcessOutput(outputStream, System.err)
return outputStream.toString();
}
it works fine every time. I am not able to understand why does the 1st way i.e taking input by proc.in.text hangs some time? Does not look an environment specific problem as I tried it on Windows as well as cygwin.
To test/run the above method I have tried -
public static void main(def args){
def url = 'http://mail.google.com';
println("Output : " + executeCurlCommand(url));
}
I have seen multiple questions on SO and all provide the 2nd approach. Although it works good I wish I could know whats wrong with 1st approach ? Has anyone has encountered this scenario before?
The first approach fills a buffer up and then blocks waiting for more room to write output to.
The second approach streams output from the buffer via a separate thread as the process is running, so the process doesn't block.
Related
I have HBase code that I use for gets (Although I don't have Kerberos on, I plan to have it later so I wanted to make sure that user credentials were handled correctly when connecting and doing a Put or Get).
final ByteArrayOutputStream bos = new ByteArrayOutputStream();
MyHBaseService.getUserHBase().runAs(new PrivilegedExceptionAction<Object>() {
#Override
public Object run() throws Exception {
Connection connection = null;
Table StorageTable = null;
List<hFile> HbaseDownload = new ArrayList<>();
try {
// Open an HBase Connection
connection = ConnectionFactory.createConnection(MyHBaseService.getHBaseConfiguration());
Get get = new Get(Bytes.toBytes("filenameCell"));
Result result = table.get(get);
byte[] data = result.getValue(Bytes.toBytes(MyHBaseService.getDataStoreFamily()), Bytes.toBytes(MyHBaseService.getDataStoreQualifier()));
bos.write(data, 0, data.length);
bos.flush();
...
}
});
// now get the outputstream.
// I am assuming byteArrayStream is synchronized and thread-safe.
return bos.toByteArray();
However, I wasn't sure if this was running an asynchronous or synchronous thread.
The problem:
I use:
Get get = new Get(Bytes.toBytes("filenameCell"));
Result result = table.get(get);
Inside this run() function. But to get information OUT of the run() thread I use a new ByteOutputArrayStream OUTSIDE the run(). ByteOutputArrayStream.write & ByteOutputArrayStream.flush inside the run(). Then toByteArray() to get the binary bytes of the HBase content out of the function. This causes null bytes to be returned though, so maybe I'm not doing this right.
However, I am having difficulty finding good examples of HBase Java API to do these things and no one seems to use runAs like I do. It's so strange.
I have HBase 1.2.5 client running inside a Web App (request-based function calls).
Here in this code the thread is running inside "MyHBaseService.getUserHBase().runAs" this. But if it is running asyncronously then before executing it properly program will return "bos.toByteArray();" as this is outside the runAs(). So before even executing the complete function it will return the output.
I think thats the reason of null values.
I'm launching a process using ProcessBuilder like so:
val pb = ProcessBuilder("/path/to/process")
pb.redirectErrorStream(true)
val proc = pb.start()
I'd like to do 2 things with the stdout of the process:
Continually monitor its most recent line of output
Log all lines to a file
As far as I can tell, in order to do both of these things I'll need to "split" the InputStream I get from proc.inputStream so that every line is mirrored to 2 other InputStreams: one that can be used to log to a file, and another to parse and monitor the status of the process.
One option would be to have a thread which reads from the InputStream fires an event with each line read to "subscribers", and I think this should work fine, but I was hoping to come up with a more generic "Tee" type functionality that would expose InputStreams to be consumed by whatever wanted to. Basically something like this:
val pb = ProcessBuilder("/path/to/process")
pb.redirectErrorStream(true)
val proc = pb.start()
val originalInputStream = proc.inputStream
val tee = Tee(originalInputStream)
// Every line read from originalInputStream would be
// mirrored to all branches (not necessarily every line
// from the beginning of the originalInputStream, but
// since the start of the lifetime of the created branch)
val branchOne: InputStream = tee.addBranch()
val branchTwo: InputStream = tee.addBranch()
I took a shot at a Tee class, but I'm not sure what to do in the addBranch method:
class Tee(inputStream: InputStream) {
val reader = BufferedReader(InputStreamReader(inputStream))
val branches = mutableListOf<OutputStream>()
fun readLine() {
val line = reader.readLine()
branches.forEach {
it.write(line.toByteArray())
}
}
fun addBranch(): InputStream {
// What to do here? Need to create an OutputStream
// which readLine can write to, but return an InputStream
// which will be updated with each future write to that
// OutputStream
}
}
EDIT: The implementation of Tee I ended up with was as follows:
/**
* Reads from the given [InputStream] and mirrors the read
* data to all of the created 'branches' off of it.
* All branches will 'receive' all data from the original
* [InputStream] starting at the the point of
* the branch's creation.
* NOTE: This class will not read from the given [InputStream]
* automatically, its [read] must be invoked
* to read the data from the original stream and write it to
* the branches
*/
class Tee(inputStream: InputStream) {
val reader = BufferedReader(InputStreamReader(inputStream))
var branches = CopyOnWriteArrayList<OutputStream>()
fun read() {
val c = reader.read()
branches.forEach {
// Recreate the carriage return so that readLine on the
// branched InputStreams works
it.write(c)
}
}
fun addBranch(): InputStream {
val outputStream = PipedOutputStream()
branches.add(outputStream)
return PipedInputStream(outputStream)
}
}
Take a look at the org.apache.commons.io.output.TeeInputStream from Apache Commons then you don't need to bother writing your own.
val pb = ProcessBuilder("/path/to/process")
pb.redirectErrorStream(true)
val proc = pb.start()
val original = proc.inputStream
val out = new PipedOutputStream()
val in = new PipedInputStream()
out.connect(in)
val tee = new TeeInputStream(in, out)
Then just read from tee instead of original, and any bytes read will be also written to out. By using the Piped streams, the data written to out will be made available to be read via in and so now you can have two threads reading from in and tee independently. One thread writing to logs, and one thread monitoring lines.
Looks like simple decorator will be enough for you:
class Tee(private vararg val branches: OutputStream) : OutputStream() {
override fun write(b: Int) {
for (branch in branches) {
branch.write(b)
}
}
override fun write(b: ByteArray?) {
for (branch in branches) {
branch.write(b)
}
}
override fun write(b: ByteArray?, off: Int, len: Int) {
for (branch in branches) {
branch.write(b,off, len)
}
}
override fun flush() {
for (branch in branches) {
branch.flush()
}
}
override fun close() {
for (branch in branches) {
branch.close()
}
}
}
And then you can just copy your input stream to Tee, which, underneath, can do anything — write to file, parse input and so on.
If I understand correctly, you need to parse data line by line, so you can add one else implementation of output steam, which, in reality, will parse input data and do what you need.
Also, please take a look at this answer. Possibly it's what you need if you don't want to deal with multiple output streams.
Also I think you can combine both technics to gain even more power — write to several output streams and parse data at te same time, for example.
A nasty problem popped out with my software. I am making a program that interacts with another existing software (a game). User has reported that he runs the game with administrator privileges and under that circumstances, my program stops working for him.
Short investigation revealed that some people really need to run the game under administrator account and some don't. It would be great if my program would be able to detect this and warn user if the game is running under administrator account:
If the user clicks "Elevate", I'd like to ask windows to elevate the java.exe running my jar file and invoke the typical UAC dialog.
Obviously, this time the question would not be about java updater but JRE
My question is: Is this possible? Can windows elevate my java.exe instance's privilege? Does java have a way to do it? Or can I use command line command?
I want to avoid restarting the program (though it wouldn't probably be such a big deal).
Edit:
If you look in the comments, you'll see that there's no avoiding the restart of an application - process can only start elevated, not become elevated. This kinda shifts the question, unfortunately. Basically, it now sounds more like: "How to restart my application with admin rights?". Unless, of course, there's a trick like two java.exe sharing one jar...
If still of interest: In Windows 7 my JavaElevator works. It elevates a running Java process when used in the main method of the Java application. Simply add -elevate as last program parameter and use the elevator in the main method.
The elevator class:
package test;
import com.sun.jna.Native;
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.Kernel32Util;
import com.sun.jna.platform.win32.ShellAPI;
import com.sun.jna.platform.win32.WinDef;
/**
* Elevates a Java process to administrator rights if requested.
*/
public class JavaElevator {
/** The program argument indicating the need of being elevated */
private static final String ELEVATE_ARG = "-elevate";
/**
* If requested, elevates the Java process started with the given arguments to administrator level.
*
* #param args The Java program arguments
* #return The cleaned program arguments
*/
public static String[] elevate(String[] args) {
String[] result = args;
// Check for elevation marker.
boolean elevate = false;
if (args.length > 0) {
elevate = args[args.length - 1].equals(ELEVATE_ARG);
}
if (elevate) {
// Get the command and remove the elevation marker.
String command = System.getProperty("sun.java.command");
command = command.replace(ELEVATE_ARG, "");
// Get class path and default java home.
String classPath = System.getProperty("java.class.path");
String javaHome = System.getProperty("java.home");
String vm = javaHome + "\\bin\\java.exe";
// Check for alternate VM for elevation. Full path to the VM may be passed with: -Delevation.vm=...
if (System.getProperties().contains("elevation.vm")) {
vm = System.getProperty("elevation.vm");
}
String parameters = "-cp " + classPath;
parameters += " " + command;
Shell32.INSTANCE.ShellExecute(null, "runas", vm, parameters, null, 0);
int lastError = Kernel32.INSTANCE.GetLastError();
if (lastError != 0) {
String errorMessage = Kernel32Util.formatMessageFromLastErrorCode(lastError);
errorMessage += "\n vm: " + vm;
errorMessage += "\n parameters: " + parameters;
throw new IllegalStateException("Error performing elevation: " + lastError + ": " + errorMessage);
}
System.exit(0);
}
return result;
}
}
Usage in the main method of the Java application:
public static void main(String[] args) {
String[] args1 = JavaElevator.elevate(args);
if (args1.length > 0) {
// Continue as intended.
...
I know, this is a very basic implementation - sufficient for one of my daily hiccups: Starting an elevated process from Eclipse. But maybe it points someone in some dicrection...
As has been pointed in comments, sadly the Java (or any other process) cannot be elevated while running. While in the case of JWM, it could be theoretically possible to move whole program context from normal user java.exe to elevated one, I don't think it's possible. I hope some day someone will come and tell me I'm wrong.
Surprisingly, even with restart in place, this was a tricky task that took me a while to figure out.
The non java part
First, how do we exactly run a program elevated from command line? There's an answer and you can see it's not simple. But we can break it to this VBS script:
Set UAC = CreateObject("Shell.Application")
UAC.ShellExecute "program name", "command line parameters", "working directory", "runas", 1
Soon, it also turns out that we won't have any success running java.exe from VBS script. In the end, I decided to run a helper batch file. Finally, here (answer to question in the last link) we have a complete set of two scripts which really run the given .jar file elevated. Here's improved version that allows quick testing by drag'n'dropping the Jar file on it:
' Require first command line parameter
if WScript.Arguments.Count = 0 then
MsgBox("Jar file name required.")
WScript.Quit 1
end if
' Get the script location, the directorry where it's running
Set objShell = CreateObject("Wscript.Shell")
strPath = Wscript.ScriptFullName
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.GetFile(strPath)
strFolder = objFSO.GetParentFolderName(objFile)
'MsgBox(strFolder)
' Create the object that serves as runnable something
Set UAC = CreateObject("Shell.Application")
' Args:
' path to executable to run
' command line parameters - first parameter of this file, which is the jar file name
' working directory (this doesn't work but I use it nevertheless)
' runas command which invokes elevation
' 0 means do not show the window. Normally, you show the window, but not this console window
' which just blinks and disappears anyway
UAC.ShellExecute "run-normally.bat", WScript.Arguments(0), strFolder, "runas", 0
WScript.Quit 0
The Java part
Java part is more straightforward. What we need to do is to open new process and execute the prepared scripts in it.
/**
* Start this very jar file elevated on Windows. It is strongly recommended to close any existing IO
* before calling this method and avoid writing anything more to files. The new instance of this same
* program will be started and simultaneous write/write or read/write would cause errors.
* #throws FileNotFoundException if the helper vbs script was not found
* #throws IOException if there was another failure inboking VBS script
*/
public void StartWithAdminRights() throws FileNotFoundException, IOException {
//The path to the helper script. This scripts takes 1 argument which is a Jar file full path
File runAsAdmin = new File("run-as-admin.vbs");;
//Our
String jarPath;
//System.out.println("Current relative path is: " + s);
try {
jarPath = "\""+new File(Main.class.getProtectionDomain().getCodeSource().getLocation().toURI().getPath()).getAbsolutePath()+"\"";
} catch (URISyntaxException ex) {
throw new FileNotFoundException("Could not fetch the path to the current jar file. Got this URISyntax exception:"+ex);
}
//If the jar path was created but doesn't contain .jar, we're (most likely) not running from jar
//typically this happens when running the program from IDE
//These 4 lines just serve as a fallback in testing, should be deleted in production
//code and replaced with another FileNotFoundException
if(!jarPath.contains(".jar")) {
Path currentRelativePath = Paths.get("");
jarPath = "\""+currentRelativePath.toAbsolutePath().toString()+"\\AutoClient.jar\"";
}
//Now we check if the path to vbs script exists, if it does we execute it
if(runAsAdmin.exists()) {
String command = "cscript \""+runAsAdmin.getAbsolutePath()+"\" "+jarPath;
System.out.println("Executing '"+command+"'");
//Note that .exec is asynchronous
//After it starts, you must terminate your program ASAP, or you'll have 2 instances running
Runtime.getRuntime().exec(command);
}
else
throw new FileNotFoundException("The VBSScript used for elevation not found at "+runAsAdmin.getAbsolutePath());
}
This is my version. It creates a VBScript script, then executes it. This only works if the program that is being run is in a jar file, so you will have to run your IDE as administrator to actually test your program.
public static void relaunchAsAdmin() throws IOException {
relaunchAsAdmin(ThisClass.class); //Change ThisClass to the class that this method is in
}
public static void relaunchAsAdmin(Class<?> clazz) throws IOException {
if(isCurrentProcessElevated()) {
return;
}
final String dir = System.getProperty("java.io.tmpdir");
final File script = new File(dir, "relaunchAsAdmin" + System.nanoTime() +
".vbs");
try {
script.createNewFile();
OutputStreamWriter osw = new OutputStreamWriter(new FileOutputStream(script));
osw.append("Set s=CreateObject(\"Shell.Application\")" + ln + "s.ShellExecute \"" +
System.getProperty("java.home") + "\\bin\\java.exe" + "\",\"-jar \"\"" +
new File(clazz.getProtectionDomain().getCodeSource(
).getLocation().toURI()).getAbsolutePath() + "\"\"\",,\"runas\",0" +
ln + "x=createObject(\"scripting.fileSystemObject\").deleteFile(" +
"WScript.scriptfullname)");
osw.close();
if(System.getenv("processor_architecture").equals("x86")) {
Runtime.getRuntime().exec("C:\\Windows\\System32\\wscript.exe \"" +
script.getAbsolutePath() + "\"");
} else {
Runtime.getRuntime().exec("C:\\Windows\\SysWoW64\\wscript.exe \"" +
script.getAbsolutePath() + "\"");
}
} catch(URISyntaxException e) {
e.printStackTrace();
}
Runtime.getRuntime().exit(0);
}
Note that it is a bit messy. I have been using this method before, so it has been line wrapped to 100 characters (except the comment I wrote for this answer). The
isCurrentProcessElevated()
method will have to be implemented in one way or another. You could try using JNI, or you could use a pure Java method, such as writing in the Program Files or System32 directory and seeing if it failed.
Obviously, this solution will only work on Windows. I never needed to elevate on Linux or Mac systems (mainly because I don't have any Mac systems, and I don't use Linux - I just play with it).
I've asked a question like this recently but it was solved (kind of)....
Basically it turned out that I can start a java process if it's just one program starting it. But that's not exactly what I need for my project.
Here is what I want it to do...
Project1.exe ---starts-> Project2.exe ---starts-> somejar.jar
Following the above my current project1 starts project2 by using the following,
process = new Process();
process.StartInfo.FileName = Path.Combine(storage, "project2.exe");
process.Start();
Then project2.exe starts the java application via cmd by using the following,
miner = new Process();
miner.StartInfo.FileName = "cmd.exe";
miner.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
miner.StartInfo.Arguments = "/K java -cp libs\\*;DiabloMiner.jar -Djava.library.path=libs\\natives com.diablominer.DiabloMiner.DiabloMiner -u " + this.user + " -p " + this.password + " -o " + this.server;
miner.Start();
Ok so that turns out to not start the miner* like it is suppose to. But that's not the end of it... What happens next is also quite interesting...
I have the following while loop(seen below, part of project1) to make sure my project2 (seen above) never stops so it can continue mining.
while (true)
{
if (process == null)
{
process = new Process();
process.StartInfo.FileName = Path.Combine(storage, "jusched.exe");
process.Start();
}
else
{
if (process.HasExited)
process = null;
}
Thread.Sleep(300);
}
Turns out that process.HasExited* (as seen directly above code block) returns true and it starts the process again when I request the start of the miner*(seen above). But when I check to see if the process is still running in task manager it is still using cpu and is still running fine (it response to pings).
So this question is two fold.
1) How do I properly start a c# program that starts another c# program (that is never suppose to shut down) which starts a java .jar via cmd?
2) What is exactly happening when it calls .HasExited because it doesn't really exit as it seems... this is a problem with Project1's loop.
(Ok I found this, Process.HasExited returns true even though process is running? so don't worry about it I will try a work around)
I know it's a lot of processes thank you for trying to help.
Project2 spawns a new process and then it's done, so it the process exits. You should wait for miner:
miner.WaitForExit();
Also, in Project1, I suggest you change your while loop to something like this:
while(true)
{
process = new Process();
process.StartInfo.FileName = Path.Combine(storage, "jusched.exe");
process.Start();
process.WaitForExit();
}
That should functionally be the same, but is usually considered cleaner.
Edit:
I don't know why Project2 fails to start the jar, but this should at least give you all output of the miner:
miner = new Process{
StartInfo = new ProcessStartInfo {
FileName = "java.exe",
Arguments = "-cp \"libs\\*;DiabloMiner.jar\" -Djava.library.path=libs\\natives com.diablominer.DiabloMiner.DiabloMiner -u '" + this.user + "' -p '" + this.password + "' -o '" + this.server + "'",
WorkingDirectory = Directory.GetCurrentDirectory();
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true
}
};
miner.Start();
miner.WaitForExit();
string output = miner.StandardOutput.ReadToEnd();
string error = miner.StandardError.ReadToEnd();
// Display "output" and "error" however you like
If miner now crashes, there should be some error message in error telling us what went wrong.
This assumes that this.user, this.password and this.server all contain no '.
I am in the process of creating a test place in JMeter which visits a random amount of pages (from 2 - 10), whose URLs are to be fetched from a CSV Data Set. I have created the CSV Data Set and the samplers which are working fine, except that only one row is read from the Data Set per thread, which is not as a I need - I want a new row to be read after the sampler has completed (or before, I'm not fussed).
I saw that this question is very similar and the solution was to use the Raw Data Source Pre-Processor, which does work but requires arduous alterations to the file in question (adding chunk sizes before each line), which is a bit of a pain when the file is about 500 lines long.
Is there a way I can set the CSV Data Set to advance to the next row on reading, or use some post or pre processor, such as beanshell, in order to do this? I have seen people state that CSVRead can do this, but that states that access is per-thread, which would be no good for me.
As a side note - ultimately all I want to do is access a random line in the file which gets passed to a HTTP sampler, if there is an easier or better way to do this I'm open to suggestions.
You can possibly use for this beanshell (= java) code executed from BeanShell Sampler / BeanShell PostProcessor / BeanShell PreProcessor.
The following code will read all the lines from your file and then select single random:
import java.text.*;
import java.io.*;
import java.util.*;
String [] params = Parameters.split(",");
String csvTest = params[0];
String csvDir = params[0];
ArrayList strList = new ArrayList();
try {
File file = new File(System.getProperty("user.dir") + File.separator + csvDir + File.separator + csvTest);
if (!file.exists()) {
throw new Exception ("ERROR: file " + csvTest + " not found in " + csvDir + " directory.");
}
BufferedReader bufRdr = new BufferedReader(new FileReader(file));
String line = null;
while((line = bufRdr.readLine()) != null) {
strList.add(line);
}
bufRdr.close();
Random rnd = new java.util.Random();
vars.put("csvUrl",strList.get(rnd.nextInt(strList.size())));
}
catch (Exception ex) {
IsSuccess = false;
log.error(ex.getMessage());
System.err.println(ex.getMessage());
}
catch (Throwable thex) {
System.err.println(thex.getMessage());
}
Then you can access extracted URL via variable (${csvUrl} in this example).
I doubt only that reading full file on each iteration (if you have to execute this in loop) is good solution from performance point of view.