we used the following codes to merge 2 files with same header and columns:
p = Runtime.getRuntime().exec(new String[]{"bash", "-c", "tail -n +2 " + file1 + " >> " + file2});
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine())!= null) {}
However, we have weird issue that one file content is missing after merging and it doesn't have any exception or failures from the logs, we have hundreds of such merges everyday and the code never have issues before. File size is between 1-2G(we have much bigger files without merge issues) and we don't have space issues when it runs either.
Does anyone have any clue on how this could happen?
Related
I´ve run into problem. I want to convert video using ffmpeg but it gives me no output
public void convert(String inputFile, String outputFile, String ... optionalParams) {
ProcessBuilder processBuilder = new ProcessBuilder("ffmpeg", "-i", "\"" + inputFile.trim() +"\"", "\""+ outputFile.trim() + "\"");
DownloadRecord downloadRecord = table.getItems().get(0);
downloadRecord.setStatus("Downloading");
// Try to execute process
try {
// Set the working directory
processBuilder.directory(new File(workingDirectory));
//Start the process
Process process = processBuilder.start();
// Read the output from cmd
BufferedReader r = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedReader ra = new BufferedReader(new InputStreamReader(process.getErrorStream()));
String line;
String errline;
while ((line = r.readLine()) != null) {
System.out.println(line);
}
while ((errline = ra.readLine()) != null) {
System.out.println(errline);
}
process.waitFor();
System.out.println("the end");
} catch(IOException | InterruptedException e) {
System.out.println(e.toString());
}
}
I've been searching on stackoverflow and find some solutions, none worked. What I tried and figured out so far
No output or error output
I tried to remove backslashes from ProcessBuilder, it
also gives me no output
I tried to let the program running, but it never finishes
I tried to use full path to the ffmpeg, no changes
I tried to run the video, no error
I am using
Netbeans IDE so I tried clean and rebuild project, no change
process also never finishes
I would like from it an output. Does someone know what I am doing wrong here ?
I fixed it by reinstalling the ffmpeg. Just went ffmpeg website downloaded newest version, replaced files in folder and it works
Edit:
It just works for files with less thatn 2 mins for some reason, more thatn 2 mins files are behaving like this
I start converting, it will not convert entirely until program runs. After I exit the program it will finish. It´s strange behaviour.
In the following program am giving name as "don" so the command will search activedirectory
with all the names starting with don (like donald etc). But the line2 variable becomes null after the assignment from reader object and it never goes into the loop. What am i doing wrong? FYI: the command works when i give it on the command line.
try {
Process p = Runtime.getRuntime().exec(
"dsquery user -name " + name + "* -limit 200|dsget user -samid -display");
p.waitFor();
BufferedReader reader = new BufferedReader(
new InputStreamReader(p.getInputStream()));
String line2 = reader.readLine();
HashMap<String,String> hmap = new HashMap<String,String>();
while (line2 != null) {
line2 = line2.trim();
if (line2.startsWith("dsget")||line2.startsWith("samid")) {
continue;
}
String[] arr = line2.split(" ",1);
hmap.put(arr[0].toLowerCase(),arr[1].toLowerCase());
line2 = reader.readLine();
}
reader.close();
line2 = reader.readLine();
}
If I am not mistaken, the pipe (or redirection) requires to launch the programs with cmd.exe.
Something like:
Process p = Runtime.getRuntime().exec("cmd /c dsquery user -name " + name + "* -limit 200|dsget user -samid -display");
I can see at least some possible problems:
1) as PhiLho wrote: pipe and redirection is done by the shell (sh, bash,... or cmd.exe on Windows). You must handle it in the Java code or run your commands in a shell.
2) after calling waitFor() the Thread is blocked until the process terminates, the process only terminates if you "consume" it's InputStream. This is not happening since waitFor() is still waiting... Better to read and process the InputStream in an additional Thread (or call waitFor after reading the InputStream).
3) reading after closing (2 last lines) should throw an Exception.
Reading the ErrorStream could help find some errors, and checking the return of waitFor is also indicated.
EDIT:
actually there should be some Exceptions being throw by that code.
Are the Exceptions being reported (printStackTrace) or just ignored?
i need to fetch the nth line of a txt file using shell script.
my text file is like
abc
xyz
i need to fetch the 2nd line and store it in a variable
i've tried all combinations using commands like :
sed
awk
head
tail
cat
... etc
problem is, when the script is called from the terminal, all these commands work fine.
but when i call the same shell script, from my java file, these commands do not work.
I expect, it has something to do with the non-interactive shell.
Please help
PS : using read command i'm able to store the first line in a variable.
read -r i<edit.txt
here , "i" is the variable and edit.txt is my txt file.
but i cant figure out, how to get the second line.
thanks in advance
edit :
ALso the script exits, when i use these "non-working" commands, And none of the remaining commands is executed.
already tried commands :
i=`awk 'N==2' edit.txt`
i=$(tail -n 1 edit.txt)
i=$(cat edit.txt | awk 'N==2')
i=$(grep "x" edit.txt)
java code:
try
{
ProcessBuilder pb = new ProcessBuilder("./myScript.sh",someParam);
pb.environment().put("PATH", "OtherPath");
Process p = pb.start();
InputStreamReader isr = new InputStreamReader(p.getInputStream());
BufferedReader br = new BufferedReader(isr);
String line ;
while((line = br.readLine()) != null)
System.out.println(line);
int exitVal = p.waitFor();
}catch(Exception e)
{ e.printStackTrace(); }
}
myscript.sh
read -r i<edit.txt
echo "session is : "$i #this prints abc, as required.
resFile=$(echo `sed -n '2p' edit.txt`) #this ans other similar commands donot do anything.
echo "file path is : "$resFile
An efficient way to print nth line from a file (especially suited for large files):
sed '2q;d' file
This sed command quits just after printing 2nd line rather than reading file till the end.
To store this in a variable:
line=$(sed '2q;d' file)
OR using a variable for line #:
n=2
line=$(sed $n'q;d' file)
UPDATE:
Java Code:
try {
ProcessBuilder pb = new ProcessBuilder("/bin/bash", "/full/path/of/myScript.sh" );
Process pr = pb.start();
InputStreamReader isr = new InputStreamReader(pr.getInputStream());
BufferedReader br = new BufferedReader(isr);
String line;
while((line = br.readLine()) != null)
System.out.println(line);
int exitVal = pr.waitFor();
System.out.println("exitVal: " + exitVal);
} catch(Exception e) { e.printStackTrace(); }
Shell script:
f=$(dirname $0)/edit.txt
read -r i < "$f"
echo "session is: $i"
echo -n "file path is: "
sed '2q;d' "$f"
Try this:
tail -n+X file.txt | head -1
where X is your line number:
tail -n+4 file.txt | head -1
for the 4th line.
I would like to read and process the output of tcpdump line by line as connections come in. So far I am using the Process class. I also want to run as a non-root user, so I have configured sudo:
user1 ALL= NOPASSWD:/usr/sbin/sudo
I am running on RHEL 6.4.
My Java code looks like this:
String[] tcpdumpCmd = {"/usr/bin/sudo", "-n", "/usr/sbin/tcpdump", "-i", "eth0", "port 8561"};
Process p = new ProcessBuilder(tcpdumpCmd).start();
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
tcpdumpOut = null;
tcpdumpLineCnt = 0;
while ((tcpdumpOut = in.readLine()) != null ) {
System.out.println(tcpdumpOut);
tcpdumpLineCnt++;
}
System.out.println("Output lines: " + tcpdumpLineCnt+"\nCommand exit code: " + p.exitValue());
The command works fine, so it is obvious to me what is happening because when I Ctrl-C I get the output printed. So it doing a sort of "buffering" until I interrupt it (at which point the remaining code isn't executed, so I don't get to the RC part.)
I assume that either my problem is in how I handle the output in my while loop, or it just cannot be done in this way. So I kindly ask if anyone has any advice for me.
I'm trying to periodically send post requests including a pic to a site. It works for the first times but then stucks. What am I doing wrong?
p = r.exec("curl --form api_key=<key> --form api_secret=<secret> "
+ "--form upload=#record.jpg -m 20 "
+ "http://api.face.com/faces/detect.json");
BufferedReader br =
new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "", text = "";
while ((line = br.readLine()) != null) {
text += line + System.getProperty("line.separator");
}
System.out.println(text);
My guess is it printing an error you cannot see. (I suggest you use ProcessBuilder to redirect error messages to std out) or the process is hanging.
Try flushing and closing the bufferedreader before doing a new request. I never used curl, but could it be that you have to wait until the process is done before running a new one?