Develop Reference

Editing a file using async threads in Java

I'm a small java developer currently working on a discord bot that I made in Java. one of the features of my bot is to simply have a leveling system whenever anyone sends a message (and other conditions but this is irrelevant for the problem I'm encountering).
Whenever someone sends a message an event is fired and a thread is created to compute how much exp the user should gain. and eventually, the function to edit the storage file is called.
which works fine when called sparsely. but if two threads try to write on the file at once, the file usually gets deleted or truncated. either of these two cases being undesired behavior
I then tried to make a queuing system that worked for over 24h but still failed once so it is more stable in a way. I only know the basics of how threads work so I may've skipped over an important thing that causes the problem
the function looks like this
Thread editingThread = null;
public boolean editThreadStarted = false;
HashMap<String, String> queue = new HashMap<>();
public final boolean editParameter(String key, String value) {
queue.put(key, value);
if(!editThreadStarted) {
editingThread = new Thread(new Runnable() {
#Override
public void run() {
while(queue.keySet().size() > 0) {
String key = (String) queue.keySet().toArray()[0];
String value = queue.get(key);
File inputFile = getFile();
File tempFile = new File(getFile().getName() + ".temp");
try {
tempFile.createNewFile();
} catch (IOException e) {
DemiConsole.error("Failed to create temp file");
handleTrace(e);
continue;
}
//System.out.println("tempFile.isFile = " + tempFile.isFile());
try (BufferedReader reader = new BufferedReader(new FileReader(inputFile)); BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile))){
String currentLine;
while((currentLine = reader.readLine()) != null) {
String trimmedLine = currentLine.trim();
if(trimmedLine.startsWith(key)) {
writer.write(key + ":" + value + System.getProperty("line.separator"));
continue;
}
writer.write(currentLine + System.getProperty("line.separator"));
}
writer.close();
reader.close();
inputFile.delete();
tempFile.renameTo(inputFile);
} catch (IOException e) {
DemiConsole.error("Caught an IO exception while attempting to edit parameter ("+key+") in file ("+getFile().getName()+"), returning false");
handleTrace(e);
continue;
}
queue.remove(key);
}
editThreadStarted = false;
}
});
editThreadStarted = true;
editingThread.start();
}
return true;
}
getFile() returns the file the function is meant to write to
the file format is
memberid1:expamount
memberid2:expamount
memberid3:expamount
memberid4:expamount
the way the editing works is by creating a temporary file to which i will write all of the original file's data line by line, checking if the memberid matches with what i want to edit, if it does, then instead of writing the original file's line, i will write the new edited line with the new expamount instead, before continuing on with the rest of the lines. Once that is done, the original file is deleted and the temporary file is renamed to the original file, replacing it.
This function will always be called asynchronously so making the whole thing synchronous is not an option.
Thanks in advance
Edit(1) :
I've been suggested to use semaphores and after digging a little into it (i never heard of semaphores before) it seems to be a really good option and would remove the need for a queue, simply aquire in the beginning and release at the end, nothing more required!

I ended up using semaphores as per user207421's suggestions and it seems to work perfectly
I simply put delays between each line write to artificially make the task longer and make it easier to have multiple threads trying to write at once, and they all wait for their turns!
Thanks

JavaFX - SetText() to TextArea does not execute until the program has completed [duplicate]

This question already has an answer here:
JavaFX 2 StringProperty does not update field until enclosing method returns
(1 answer)
Closed 7 years ago.
I am currently learning JavaFX. I am building an app that takes a file as input, connects to an external server, and returns the parsed results to a MySQL database. The core functionality is working fine, but I would like to have JavaFX update to the user, so that they know they are waiting to connect to the server etc. However, when the method is invoked, the text updates are not visible until the method has completed execution - I would like the updates to occur in real time. The offending method is below:
public void doBlast(){
//this line is not updated until completion of the program - not straight away
status.setText("Connecting to NCBI servers..NCBI" + "\n");
//Biojava API details extracted as it is not relevant
BufferedReader bufferedFileReader = null;
Scanner bufferedScanner = null;
BufferedWriter bufferedWriterXML = null;
String rid = null;
int count = 0;
try {
//this.getFasta() represents the input file
bufferedScanner
= new Scanner(new BufferedReader(new FileReader (this.getFasta())));
//this.getTempXML() is a temporary file for temporary storage of xml output from the server
bufferedWriterXML
= new BufferedWriter (new FileWriter (this.getTempXML(), true));
String line, nextline, query, xmlLine;
//Loops for a specified number of times
while (count < alignmentLimit) {
this.setSequence_id(bufferedScanner.nextLine());
//newline here
bufferedScanner.nextLine();
//this line from the file is used to search a database in the server
query = bufferedScanner.nextLine();
//this line is not updated until completion of the program - not straight away
status.setText("Connecting to NCBI servers.." + "\n");
//below service connects to server
rid = service.sendAlignmentRequest(query, props);
//results
InputStream in = service.getAlignmentResults(rid, outputProps);
//this line is not updated until completion of the program - not straight away
status.setText("Connected to NCBI." + "\n");
bufferedFileReader = new BufferedReader(new InputStreamReader(in));
xmlLine = bufferedFileReader.readLine();
while (xmlLine != null) {
bufferedWriterXML.write("\n" + xmlLine);
xmlLine = bufferedFileReader.readLine();
// have to flush here so that it prints to file;
// the file can then be deleted each loop
bufferedWriterXML.flush();
}
count++;
//this method removes specific strings
this.removeAll("<?xml", "<!DOCTYPE", this.getTempXML(), this.getXML());
//this method fills a database with the data from server
this.xml2mysql();
//file cleaning
this.eraseData(this.getTempXML());
this.eraseData(this.getXML());
bufferedScanner.nextLine();
//this line is not updated until completion of the program - not straight away
numberOfRecords.setText(Integer.toString(count));
}
}
catch (Exception anException){
anException.printStackTrace();
}
finally {
try {
bufferedWriterXML.close();
bufferedScanner.close();
bufferedFileReader.close();
}
catch (Exception anException){
System.out.println("Error: " + anException);
}
}
}
The setText() methods use a FXML-defined fx:id variable and the above method is in a controller.
Much appreciated if anyone can help me out. I have searched everywhere for a solution.
Thanks.

You should look at using a task or service which will place the loading of the file on a background thread.
You can hook into the methods called during the execution:
From Oracle documentation:
Inside the call method, you can use the updateProgress, updateMessage,
updateTitle methods, which update the values of the corresponding
properties on the JavaFX Application thread. However, if the task was
canceled, a return value from the call method is ignored
If you perform the long process on the JavaFX Thread, the UI will be blocked and appear frozen.

Can't read previously written JSON data (Unterminated String error) in Java

I'm using the twitter4j package for an information retrieval class and have collected some tweets. However, for the next part of the assignment, I am to use Lucene to index on the tweets. In order to do this, my thought was to save the tweets as JSON Strings to a file and then reread them when needed. However, I'm running into an error.
When the file is written, I can see the entire JSON object just fine. The total object is quite large (2500 characters). However, when reading back from the file, I get a Unterminated string at xxxx error. I am using the TwitterObjectFactory methods to both write and read the string. Here is a sample code:
Writing:
public void onStatus(Status status) {
try{
String jsonString = TwitterObjectFactory.getRawJSON(status);
output.write(jsonString+"\n");
numTweets++;
if(numTweets > 10){
synchronized(lock){
lock.notify();
}
}
}
catch(IOException e){
e.printStackTrace();
}
}
Reading:
Scanner input = new Scanner(file);
while(input.hasNext()){
Status status = TwitterObjectFactory.createStatus(input.nextLine());
System.out.println(status.getUser().getScreenName());
}
This works only some of the time. If I run the program multiple times and get many tweets, the program almost always crashes after 2-3 tweets have been read from the file, always with the same error. If you'd like to replicate the code, you can follow this example. I've added a synchronized block in order to close the stream after 10 tweets, but it's not necessary to replicate the error.
Can someone explain what is happening? My guess is that there's something wrong with the way I'm encoding the JSON into the file. I'm using BufferedWriter wrapping an OutputStreamWriter in order to output in UTF-8 format.
Edit: I do close the stream. Here's the bottom snippet of the code:
twitterStream.addListener(listener);
twitterStream.sample("en");
try{
synchronized(lock){
lock.wait();
}
}
catch(InterruptedException e){
e.printStackTrace();
}
twitterStream.clearListeners();
twitterStream.cleanUp();
twitterStream.shutdown();
output.close();

You probably need to flush your output, before you notify the reader. Otherwise parts of your String will stay in the buffer.
public void onStatus(Status status) {
try{
String jsonString = TwitterObjectFactory.getRawJSON(status);
output.write(jsonString+"\n");
output.flush();
numTweets++;
if(numTweets > 10){
synchronized(lock){
lock.notify();
}
}
}
catch(IOException e){
e.printStackTrace();
}
}

I don't see the code where you properly close the BufferedWriter. If you don't close it manually before the first program ends, then data might remain in the internal buffer and never written to the file.
You can also try to open the file in a text editor and look at the contents. Tools like http://codebeautify.org/jsonviewer or http://jsonlint.com/ allow you to validate/beautify the contents to see errors.
Lastly, try BufferedReader( new InputStreamReader( new FileInputStream(file), "UTF-8" ) ). Maybe non-ASCII characters in the input are confusing Scanner.

What value is there after we read values from InputStream

I wanted to know that when we read something from InputStream after that what value is there in stream? Is it deleted or same value is retained?
For eg I have one code which reads something from InputStream and prints on the screen. Here is the code:
char[] data = new char[1024];
while (!isInterrupted()) {
try {
in.read(data); // Step 1
} catch (IOException e) {
continue;
}
msg = new String(data);
System.out.println(msg); // Step 2
}
So when first time step 1 reads something and step 2 prints it, it is okay. But when again it is coming to Step 1 and IF sender has NOT sent anything then what Step 1 will do? Will it wait till it gets new value (or sender sends anything) or it will read the same value and print the same value? Or it will throw an Exception? Please help.

The javadoc says:
This method blocks until input data is available, end of file is detected, or an exception is thrown.
So, unless the sender closes the stream, it will block until data is available, as per the documentation.

BufferedReader never ready (Socket programming in Java)

I have socket already declared socket like this:
serverAddr = InetAddress.getByName(this.ip);
socket = new Socket(serverAddr, port);
out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())), true);
however, the following doesn't work. in.ready() always returns false and if removed the program will freeze at String message = in.readLine();
private void receive() {
try {
InputStreamReader isr = new InputStreamReader(socket.getInputStream());
System.out.println(isr.getEncoding());
BufferedReader in = new BufferedReader(isr);
if (in.ready()) {
String message = in.readLine();
if (message != null) {
if (listener != null) {
listener.receiveMessage(ip, message);
} else {
print("Client recieved: " + message);//
}
}
}
in.close();
} catch (Exception e) {
print("Error with input stream: " + e);
disconnect();
}
}
How could i solve this?
EDIT:
This is how sending looks like in my server class:
out.println(message);
out.flush();
This happens in a loop whenever i've put something in message. out is closed after this loop.

You shouldn't be using ready() like this. The javadoc says this:
"Returns: True if the next read() is guaranteed not to block for input, false otherwise. Note that returning false does not guarantee that the next read will block. "
Your code is implicitly assuming that ready() -> false means that the next read will block. In actual fact, it means the next read might or might not block.
As #EJP says ... just do the read call.
What could i do to prevent a block though? The client will be unable to send anything if it's blocked
If blocking in read is a problem for your application, either use a separate thread to do the reading, or change your code to use NIO channel selectors.

Just remove the in.ready() test. It isn't helping you. readLine() will block until there is data available. What else were you planning to do if no data has arrived yet?

There are 3 things that come to my mind:
You are re-opening the input stream in every receive call, and wrapping it into a BufferedReader. This might read more than a single line into the buffer, and after finishing (closing it), the remaining buffered bytes will no longer be available for subsequent receive calls
Did you think about using an own thread for reading the server messages? There it won't harm if it is blocked
I have experienced some problems when closing one side of a socket after writing data, and immediately closing it. Sometimes not all of the data was received by the other side, despite flush() and close() calls. Maybe this is also an issue in your situation
Edit:
Smiply keeping the in reference outside of the receive method will not fully solve your problem. You should use a while loop for reading all buffered messages and call the listener for everyone, e.g.:
if (in.ready()) {
String message;
while ((message = in.readLine()) != null) {
// ...
}
}
But watch out as the last line might be a partially read message (e.g. 3 and 1/2 messages were buffered). If this is an issue, you could read the messages char-by-char for determining when a line ends, and use a PushbackReader for putting back incomplete messages.

You may need to call out.flush() to flush anything in BufferedWriter