Develop Reference

Why jTextArea.setText() method works after it should?

Sorry guys maybe it can be a silly question but really i couldn't find any similar situation like this.
Here is my code:
private void startHashingButtonActionPerformed(java.awt.event.ActionEvent evt) {
consoleArea.setText( myFile.getName() + " started to be hashing! It can take few minutes, please wait.."); //20:05
try {
BufferedReader reader = new BufferedReader(new FileReader(myFile));
myHash = new HashOA(300000);
try {
while(reader.readLine() != null){
myHash.hash(reader.readLine());
}
consoleArea.append("\n" + myFile.getName() + " is successfully hashed!!");
} catch (IOException ex) {
Logger.getLogger(MainScreen.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (FileNotFoundException ex) {
Logger.getLogger(MainScreen.class.getName()).log(Level.SEVERE, null, ex);
}
}
I expect that in consoleArea(TextArea) there should be "file.txt started to be hashing! It can take few minutes, please wait.." written and after that the hashing process(that while(reader.readLine() != null) loop) should be started. But when i run the program and click on "startHashingButton" it first finishes hashing process and later it writes on console(jTextArea) --> "file.txt started to be hashing! It can take few minutes, please wait..", "file.txt is successfully hashed!!"
I'm working on a large text file and it takes a while to hash it. That's why i want to tell user he/she should wait a bit.
Why the working queue differs from my code order ?
Note: The only thing that came to my mind is to use thread, could it solve the problem?

Note: The only thing that came to my mind is to use thread, could it solve the problem?
Yes, that is correct. Code executed in an event listener is invoked on the Event Dispatch Thread. The GUI can't be repainted until all the code is finished executing. So a long running task prevents the GUI from repainting itself.
Read the section from the Swing tutorial on Concurrency in Swing for more information. Maybe a SwingWorker will be a better solution than creating your own Thread.

Use SwingWorker to implement a worker thread.
Do all the processing in doInBackground method, you can add your following code in doInBackground. Before that you can you will set the text in your console area. And once your file is hashed, you can implement done() method and set the appropriate message in your console area. To give you an idea, it will look something like this
#Override
protected Integer doInBackground() throws Exception {
try {
BufferedReader reader = new BufferedReader(new FileReader(myFile));
myHash = new HashOA(300000);
try {
while(reader.readLine() != null){
myHash.hash(reader.readLine());
}
return null;
} catch (IOException ex) {
Logger.getLogger(MainScreen.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (FileNotFoundException ex) {
Logger.getLogger(MainScreen.class.getName()).log(Level.SEVERE, null, ex);
}
}
#Override
protected void done() {
consoleArea.append("\n" + myFile.getName() + " is successfully hashed!!");
}
Refer this for more clarity : How do I make my SwingWorker example work properly?

Trouble sending multiple values to an external process with apache commons-exec

I'm trying to write a little application that would automate the use of an external application which is cisco any connect mobility client. It provides a command line tools that you can use to connect to your VPN.
I want to run this command line tools from my java application using apache commons-exec library and be able to read his output to send needed information.
I already searched on the net to find "how to communicate" with an external application but the only post I found was this article : Trouble providing multiple input to a Command using Apache Commons Exec and extracting output where it just says "hey I found the solutions", but I don't understand how he did it.
When I start the process, I run a function that read the input like this :
Thread T = new Thread() {
public void run() {
String line;
try {
line = processOutput.readLine();
while (line != null) {
System.out.println(line);
if(line.contains("VPN-Password")){
sendMessage(processInput, "1");
}
if(line.contains("Please enter your username and password")){
sendMessage(processInput, "username");
}
line = processOutput.readLine();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
T.start();
the function send message just run a thread to write in the process inputstream then flush it.
Thread T = new Thread() {
public void run() {
try {
os.write((message+"\n").getBytes());
os.flush();
System.out.println("SENT : "+message);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
T.start();
As you can see I check the output to send a message to the process depending on it (basicly to answer questions). However, when it comes to the "Please enter...", I got this exception
java.io.IOException: Read end dead
My issue is that I can't find how to "communicate" with the process by reading his output and sending it messages depending on what it tells me.
Can you help me ?
Thanks for reading.

Multiple Threads downloading same file from sftp server

I have a system that, when files of a certain type are found, I download, encode, and upload them in a separate thread.
while(true) {
for(SftpClient c : clients) {
try {
filenames = c.list("*.wav", "_rdy_");
} catch (SftpException e) {
e.printStackTrace();
}
if(filenames.size() > 0) {
//AudioThread run() method handles the download, encode, and upload
AudioThread at = new AudioThread(filenames);
at.setNode(c.getNode());
Thread t = new Thread(at);
t.start();
}
}
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
The run method from AudioThread
public void run() {
System.out.println("Running...");
this.buildAsteriskMapping();
this.connectToSFTP();
ac = new AudioConvert();
this.connectToS3();
String downloadDir = "_rough/" + getNode() + "/" + Time.getYYYYMMDDDate() + "/";
String encodeDir = "_completed" + getNode() + "/" + Time.getYYYYMMDDDate() + "/";
String uploadDir = getNode() + "/" + Time.getYYYYMMDDDate() + "/";
System.out.println("Downloading...");
try {
sftp.get(filenames, downloadDir);
} catch (SftpException e) {
//download failed
System.out.println("DL Failed...");
e.printStackTrace();
}
System.out.println("Encoding...");
try {
ac.encodeWavToMP3(filenames, downloadDir, encodeDir);
} catch (IllegalArgumentException | EncoderException e) {
System.out.println("En Failed...");
e.printStackTrace();
}
System.out.println("Uploading...");
try {
s3.upload(filenames, encodeDir, uploadDir);
} catch (AmazonClientException e) {
System.out.println("Up Failed...");
e.printStackTrace();
}
}
The download method:
public void get(ArrayList<String> src, String dest) throws SftpException {
for(String file : src) {
System.out.println(dest + file);
channel.get(file, dest + file);
}
}
The encode method:
public void encodeWavToMP3(ArrayList<String> filenames, String downloadDir, String encodeDir) throws IllegalArgumentException, EncoderException {
for(String f : filenames) {
File wav = new File(downloadDir + f);
File mp3 = new File(encodeDir + wav.getName().replace(".wav", ".mp3"));
encoder.encode(wav, mp3, attrs);
}
}
The upload method:
public void upload(ArrayList<String> filenames, String encodeDir, String uploadDir) throws AmazonClientException, AmazonServiceException {
for(String f : filenames) {
s3.putObject(new PutObjectRequest(bucketName, uploadDir, new File(encodeDir + f)));
}
}
The issue is I keep downloading the same files (or about the same files) for every thread. I want to add a variable for each client that holds the files that are being downloaded but I don't know how to remove the lists/filenames from this variable. What would be a solution? My boss would also like to only allow x amount of threads to run.

It's kind of hard to see the problem, as the code that actually does the download is missing :P
However, I would use some kind of ExecutorService instead.
Basically, I would add each download request to the service (wrapped in a "DownloadTask" with a reference to the file to be downloaded and any other relevant information it might need to get the file) and let the service take care of the rest.
The download tasks could be coded to take into account existing files as you see fit.
Depending on your requirements, this could be a single thread or multi-threaded service. It could also allow you to place upload quests in it as well.
Check out the Executors trail for more info
The general idea is to use a kind of producer/consumer pattern. You would have (at least) a thread that would look up all the files to be downloaded and for each file, you would add it to the executor service. After the file has been downloaded, I would queue and upload request into the same service.
This way, you avoid all the mess with synchronization and thread management :D
You could use the same idea with the scan tasks, for each client, you could a task to a separate service

There is a problem in your code where you instantiate AudioThread in a while loop.
Note that after you create a thread and do a t.start(), all downloading, encoding and uploading happens asynchronously. Therefore, after you start the thread the loop continuous to do another call to c.list(...) while the first thread you created is still processing the first set of files. Most probably the same set of files is returned in the succeeding c.list() calls since you specified a file pattern in the call and there is no code which marks which files are currently being processed.
My suggestion:
Use Executors.newFixedThreadPool(int nThreads) as mentioned in previous post. And specify the number of threads to the number of processors in your machine. Do this before your while loop.
For each filename you retrieved from ftp s.list(), create a Callable class and call ExecutorService.invokeAll(Collection<Callable<T>> tasks). The code in the Callable you will create is your AudioThread code. Modify AudioThread code to only process one file at at time (if possible), this way you are doing downloads,uploads, encoding in parallel for each file.
Add code which marks which files were already processed. I would suggest adding a code which renames the files you have processed to a different name to avoid getting returned in the next c.list() call.
Call ExecutorService.shutdown(...) after your while loop block

Eclipse MessageConsole seems blocked - need Asynchronous output

I want write the output of a program to the console, but in someway the Console seems blocked/the Console doesn't even show before the function has finished.
Here is some sample code, which shows the exact same behaviour:
public void startConsole() throws IOException {
long start = System.currentTimeMillis();
MessageConsole console = new MessageConsole("TestConsole", null);
ConsolePlugin.getDefault().getConsoleManager().addConsoles(new IConsole[] {console});
ConsolePlugin.getDefault().getConsoleManager().showConsoleView(console);
MessageConsoleStream stream = console.newMessageStream();
stream.setActivateOnWrite(true);
stream.println("Start: " + (System.currentTimeMillis()-start));
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
stream.println("End: " + (System.currentTimeMillis()-start));
}
What I get: The function runs for about a second and I see the output ("Start: 0\nEnd:1000")
What I want: I launch the function, and see the first output ("Start: 0"), one second later, I want "End: 1000" to be added to the Console.
How do I achieve this?

You will need to run the method in a new thread.
Define a new class that implements java.lang.Runnable with
its run() method does the things that have been shown above.
Then, create an instance of java.lang.Thread with the Runnable
object that you defined. Kick off the job by invoking the
thread by calling start() method on the thread instance.

Java Gridgain application starts to fail after 1 day of stress testing

So I have a an application which is running on top of gridgain and does so quite successfully for about 12-24 hours of stress testing before it starts to act funny. After this period of time the application will suddenly start replying to all queries with the exception java.nio.channels.ClosedByInterruptException (full stack trace is at http://pastie.org/664717
The method that is failing from is (edited to use #stephenc feedback)
public static com.vlc.edge.FileChannel createChannel(final File file) {
FileChannel channel = null;
try {
channel = new FileInputStream(file).getChannel();
channel.position(0);
final com.vlc.edge.FileChannel fileChannel = new FileChannelImpl(channel);
channel = null;
return fileChannel;
} catch (FileNotFoundException e) {
throw new VlcRuntimeException("Failed to open file: " + file, e);
} catch (IOException e) {
throw new VlcRuntimeException(e);
} finally {
if (channel != null) {
try {
channel.close();
} catch (IOException e){
// noop
LOGGER.error("There was a problem closing the file: " + file);
}
}
}
}
and the calling function correctly closes the object
private void fillContactBuffer(final File signFile) {
contactBuffer = ByteBuffer.allocate((int) signFile.length());
final FileChannel channel = FileUtils.createChannel(signFile);
try {
channel.read(contactBuffer);
} finally {
channel.close();
}
contactBuffer.rewind();
}
The application basically serves as a distributed file parser so it does a lot of these types of operations (will typically open about 10 such channels per query per node). It seems that after a certain period it stops being able to open files and I'm at a loss to explain why this could be happening and would greatly appreciate any one who can tell me what could be causing this and how I could go about tracking it down and fixing it. If it is possibly related to file handle exhaustion, I'd love to hear any tips for finding out for sure... i.e. querying the JVM while it's running or using linux command line tools to find out more information about what handles are currently open.
update: I've been using command line tools to interrogate the output of lsof and haven't been able to see any evidence that file handles are being held open... each node in the grid has a very stable profile of openned files which I can see changing as the above code is executed... but it always returns to a stable number of open files.
Related to this question: Freeing java file handles

There are a couple of scenarios where file handles might not be being closed:
There might be some other code that opens files.
There might be some other bit of code that calls createChannel(...) and doesn't call fillContactBuffer(...)
If channel.position(0) throws an exception, the channel won't be closed. The fix is to rearrange the code so that the following statements are inside the try block.
channel.position(0);
return new FileChannelImpl(channel);
EDIT: Looking at the stack trace, it seems that the two methods are in different code-bases. I'd point the finger of blame at the createChannel method. It is potentially leaky, even if it is not the source of your problems. It needs an in internal finally clause to make sure that the channel is closed in the event of an exception.
Something like this should do the trick. Note that you need to make sure that the finally block does not closes the channel on success!
public static com.vlc.edge.FileChannel createChannel(final File file) {
final FileChannel channel = null;
try {
channel = new FileInputStream(file).getChannel();
channel.position(0);
FileChannel res = new FileChannelImpl(channel);
channel = null;
return res;
} catch (FileNotFoundException e) {
throw new VlcRuntimeException("Failed to open file: " + file, e);
} catch (IOException e) {
throw new VlcRuntimeException(e);
} finally {
if (channel != null) {
try {
channel.close();
} catch (...) {
...
}
}
}
}
FOLLOWUP much later
Given that file handle leakage has been eliminated as a possible cause, my next theory would be that the server side is actually interrupting its own threads using Thread.interrupt(). Some low-level I/O calls respond to an interrupt by throwing an exception, and the root exception being thrown here looks like one such exception.
This doesn't explain why this is happening, but at a wild guess I'd say that it was the server-side framework trying to resolve an overload or deadlock problem.