Develop Reference

Issue with retrieving String from another class that uses a thread

I have a class that calls a method from another class which uses a thread as it is somewhat a intensive task. The Thread is responsible for looking through a dictionary to find a matching word. When a word is found it should/does set a local variable in that class. I can see that it is successfully setting this String because it say's so in the log. However, whenever I try and retrieve this String from another class and set the TextView to the value of this String, Nothing happens.
I'm using the Thread because a lot of frames were being skipped. However, when I don't use the thread it works as it is suppose to(Minus the frames being skipped).
Here is the method with the thread:
public String checkLetters() {
new Thread(new Runnable() {
#Override
public void run() {
//Finding the directory on SD Card
File sdcard = Environment.getExternalStorageDirectory();
//Retrieve the text file
File file = new File(sdcard,"NewEnglishDictionary.txt");
try {
BufferedReader bufferRead = new BufferedReader(new FileReader(file),24396);
String line; //= "";
//While no word found keep looping
while (wholeWordFound == false ) {
line = bufferRead.readLine();
Log.d("ResolveWord", "Current Line: " + line);
wordReturned = workOutWord(line);
setWord(wordReturned);
}
String value = getWord().toString().toLowerCase();
Log.d("Value of setWord: ", " equals: "+ value);
bufferRead.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
return wordReturned;
}
And calling to retrieve the variable that has been set (according to the log)
tv_WordFound.setText(fixAnagram.getWord());
And the method which is suppose to return it:
public String getWord() {
Log.d("Test", "getWord: " + wordReturned);
return wordReturned;
}
Is there something I'm missing with threads?
Cheers for any help. Logcat itself gives me no clue to where the error lies.

you need to using callback or interface or asynctask for this case. Because your Thread inside checkLetters method will end after your checkLetters. That means if you call getWord() immediately after calling checkLetters, you can only get the previous checking result.

The Memory model of Java does not guarantee, that values set by one thread are immediately visible by another thread.
To guarantee this, you either must declare the variable as "volatile" or have a synchronized involved (e.g. have the getter and setter method with the keyword synchronized).
e.g.
private volatile String word;
or
public synchronized String getWord() { return word; }
public synchronized void setWord(String w) { word = w; }

(Java) How would you write a program that counts the number of times it runs and stores that number?

Say the user runs SomeProgram.java to calculate a bunch of stuff. One of the things they want to keep track of is how many times this program has been run and output the current run number. This is how far I got but it resets each time.
public class SomeProgram
{
public volatile int counter = 1;
public int getNextRun()
{
return counter++;
}
//calculates a bunch of variable that get output to user
public static void main(String args[])
{
SomeProgram variable = new SomeProgram();
runNumber = variable.getNextRun();
System.out.println(runNumber + "Some bunch of calculations");
}
}
Can someone explain why this got downvoted?

Whenever the user stops running your program, you're going to lose any variables stored in memory, so you're going to have to store that value somewhere else. The easiest solution would be to store it in a local file.
If your business needs to know this number, you can have the program call home to a webserver every time it starts up - this prevents the user from modifying the file on their computer - but is far more complicated to set up, and some users might not appreciate this unexpected behavior.

Complete implementation which stores updated counter in a file, invoke it whenever you want a counter to increment (i.e. when program starts). When a file doesn't exist, it is created. This method returns updated counter or 0 if there was some IOException.
public static int updateCounter() {
String counterFileName = "counter.txt";
int counter = 0;
File counterFile = new File(counterFileName);
if (counterFile.isFile()) {
try (BufferedReader reader = new BufferedReader(new FileReader(counterFileName))) {
counter = Integer.parseInt(reader.readLine());
} catch (IOException e) {
e.printStackTrace();
return 0;
}
}
try (FileWriter writer = new FileWriter(counterFileName)) {
writer.write(String.valueOf(++counter));
} catch (IOException e) {
e.printStackTrace();
return 0;
}
return counter;
}

Writing to the local file is not a good idea. You'll have to implement locking mechanism on your local file, otherwise you'll suffer of race conditions in case of simultaneous start of several program instances.
Alternative idea is to log each run into a persistent storage. So if you write each run's date and time to the db, you'll be able to calculate number of runs for arbitrary time interval.
Actual implementation depends on your requirements

You can use a Properties file:
public void loadProperties(String fileName)
{
Properties props = new Properties();
InputStream is = null;
// First try loading from the current directory
try {
File f = new File(fileName);
is = new FileInputStream( f );
}catch ( Exception e ) {
is = null;
e.printStackTrace();
}
try {
if ( is == null ) {
// Try loading from classpath
is = getClass().getResourceAsStream("example.properties");
}
// Try loading properties from the file (if found)
props.load( is );
String counter1 = props.getProperty("COUNTER_RUN");
String counter2 = props.getProperty("COUNTER_OUTPUT");
counterRun = Integer.parseInt(counter1);
counterOutput = = Integer.parseInt(counter2);
}catch ( Exception e ) {
e.printStackTrace();
}
}
public void saveProperties(String fileName) {
try {
Properties props = new Properties();
props.setProperty("COUNTER_RUN", ""+counterRun );
props.setProperty("COUNTER_OUTPUT", ""+counterOutput );
File f = new File(fileName);
OutputStream out = new FileOutputStream( f );
props.store(out, "Config params");
} catch (Exception e ) { e.printStackTrace(); }
}
counterRun and counterOutput are global vars
File example.properties
#Config paramns
#Tue May 03 14:17:35 COT 2016
COUNTER_RUN=241
COUNTER_OUTPUT=123

Running threads in round robin fashion in java

I am new to Multithreading and synchronization in java. I am trying to achieve a task in which i am given 5 files, each file will be read by one particular thread. Every thread should read one line from file then forward execution to next thread and so on. When all 5 threads read the first line, then again start from thread 1 running line no. 2 of file 1 and so on.
Thread ReadThread1 = new Thread(new ReadFile(0));
Thread ReadThread2 = new Thread(new ReadFile(1));
Thread ReadThread3 = new Thread(new ReadFile(2));
Thread ReadThread4 = new Thread(new ReadFile(3));
Thread ReadThread5 = new Thread(new ReadFile(4));
// starting all the threads
ReadThread1.start();
ReadThread2.start();
ReadThread3.start();
ReadThread4.start();
ReadThread5.start();
and in ReadFile (which implements Runnable, in the run method, i am trying to synchronize on bufferreader object.
BufferedReader br = null;
String sCurrentLine;
String filename="Source/"+files[fileno];
br = new BufferedReader(new FileReader(filename));
synchronized(br)
{
while ((sCurrentLine = br.readLine()) != null) {
int f=fileno+1;
System.out.print("File No."+f);
System.out.println("-->"+sCurrentLine);
br.notifyAll();
// some thing needs to be dine here i guess
}}
Need Help

Though this is not an ideal scenario for using multi-threading but as this is assignment I am putting one solution that works. The threads will execute sequentially and there are few point to note:
Current thread cannot move ahead to read the line in the file until and unless its immediately previous thread is done as they are supposed to read in round-robin fashion.
After current thread is done reading the line it must notify the other thread else that thread will wait forever.
I have tested this code with some files in temp package and it was able to read the lines in round robin fashion. I believe Phaser can also be used to solve this problem.
public class FileReaderRoundRobinNew {
public Object[] locks;
private static class LinePrinterJob implements Runnable {
private final Object currentLock;
private final Object nextLock;
BufferedReader bufferedReader = null;
public LinePrinterJob(String fileToRead, Object currentLock, Object nextLock) {
this.currentLock = currentLock;
this.nextLock = nextLock;
try {
this.bufferedReader = new BufferedReader(new FileReader(fileToRead));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
#Override
public void run() {
/*
* Few points to be noted:
* 1. Current thread cannot move ahead to read the line in the file until and unless its immediately previous thread is done as they are supposed to read in round-robin fashion.
* 2. After current thread is done reading the line it must notify the other thread else that thread will wait forever.
* */
String currentLine;
synchronized(currentLock) {
try {
while ( (currentLine = bufferedReader.readLine()) != null) {
try {
currentLock.wait();
System.out.println(currentLine);
}
catch(InterruptedException e) {}
synchronized(nextLock) {
nextLock.notify();
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
synchronized(nextLock) {
nextLock.notify(); /// Ensures all threads exit at the end
}
}
}
public FileReaderRoundRobinNew(int numberOfFilesToRead) {
locks = new Object[numberOfFilesToRead];
int i;
String fileLocation = "src/temp/";
//Initialize lock instances in array.
for(i = 0; i < numberOfFilesToRead; ++i) locks[i] = new Object();
//Create threads
int j;
for(j=0; j<(numberOfFilesToRead-1); j++ ){
Thread linePrinterThread = new Thread(new LinePrinterJob(fileLocation + "Temp" + j,locks[j],locks[j+1]));
linePrinterThread.start();
}
Thread lastLinePrinterThread = new Thread(new LinePrinterJob(fileLocation + "Temp" + j,locks[numberOfFilesToRead-1],locks[0]));
lastLinePrinterThread.start();
}
public void startPrinting() {
synchronized (locks[0]) {
locks[0].notify();
}
}
public static void main(String[] args) {
FileReaderRoundRobinNew fileReaderRoundRobin = new FileReaderRoundRobinNew(4);
fileReaderRoundRobin.startPrinting();
}
}
If the only objective is to read the files in round-robin fashion and not strictly in same order then we can also use Phaser. In this case the order in which files are read is not always same, for example if we have four files (F1, F2, F3 and F4) then in first phase it can read them as F1-F2-F3-F4 but in next one it can read them as F2-F1-F4-F3. I am still putting this solution for sake of completion.
public class FileReaderRoundRobinUsingPhaser {
final List<Runnable> tasks = new ArrayList<>();
final int numberOfLinesToRead;
private static class LinePrinterJob implements Runnable {
private BufferedReader bufferedReader;
public LinePrinterJob(BufferedReader bufferedReader) {
this.bufferedReader = bufferedReader;
}
#Override
public void run() {
String currentLine;
try {
currentLine = bufferedReader.readLine();
System.out.println(currentLine);
} catch (IOException e) {
e.printStackTrace();
}
}
}
public FileReaderRoundRobinUsingPhaser(int numberOfFilesToRead, int numberOfLinesToRead) {
this.numberOfLinesToRead = numberOfLinesToRead;
String fileLocation = "src/temp/";
for(int j=0; j<(numberOfFilesToRead-1); j++ ){
try {
tasks.add(new LinePrinterJob(new BufferedReader(new FileReader(fileLocation + "Temp" + j))));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
public void startPrinting( ) {
final Phaser phaser = new Phaser(1){
#Override
protected boolean onAdvance(int phase, int registeredParties) {
System.out.println("Phase Number: " + phase +" Registeres parties: " + getRegisteredParties() + " Arrived: " + getArrivedParties());
return ( phase >= numberOfLinesToRead || registeredParties == 0);
}
};
for(Runnable task : tasks) {
phaser.register();
new Thread(() -> {
do {
phaser.arriveAndAwaitAdvance();
task.run();
} while(!phaser.isTerminated());
}).start();
}
phaser.arriveAndDeregister();
}
public static void main(String[] args) {
FileReaderRoundRobinUsingPhaser fileReaderRoundRobin = new FileReaderRoundRobinUsingPhaser(4, 4);
fileReaderRoundRobin.startPrinting();
// Files will be accessed in round robin fashion but not exactly in same order always. For example it can read 4 files as 1234 then 1342 or 1243 etc.
}
}
The above example can be modified as per exact requirement. Here the constructor of FileReaderRoundRobinUsingPhaser takes the number of files and number of lines to read from each file. Also the boundary conditions need to be taken into consideration.

You are missing many parts of the puzzle:
you attempt to synchronize on an object local to each thread. This can have no effect and the JVM may even remove the whole locking operation;
you execute notifyAll without a matching wait;
the missing wait must be at the top of the run method, not at the bottom as you indicate.
Altogether, I'm afraid that fixing your code at this point is beyond the scope of one StackOverflow answer. My suggestion is to first familiarize yourself with the core concepts: the semantics of locks in Java, how they interoperate with wait and notify, and the precise semantics of those methods. An Oracle tutorial on the subject would be a nice start.

Read the 30Million user id's one by one from the big file

I am trying to read a very big file using Java. That big file will have data like this, meaning each line will have an user id.
149905320
1165665384
66969324
886633368
1145241312
286585320
1008665352
And in that big file there will be around 30Million user id's. Now I am trying to read all the user id's one by one from that big file only once. Meaning each user id should be selected only once from that big file. For example, if I have 30Million user id's then it should print 30 Million user id only once with the use of Multithreading code.
Below is the code I have which is a multithreaded code running with 10 threads but with the below program, I am not able to make sure that each user id is selected only once.
public class ReadingFile {
public static void main(String[] args) {
// create thread pool with given size
ExecutorService service = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
service.submit(new FileTask());
}
}
}
class FileTask implements Runnable {
#Override
public void run() {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader("D:/abc.txt"));
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
//do things with line
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Can anybody help me with this? What wrong I am doing? And what is the fastest way to do this?

You really can't improve on having one thread reading the file sequentially, assuming that you haven't done anything like stripe the file across multiple disks. With one thread, you do one seek and then one long sequential read; with multiple threads you're going to have the threads causing multiple seeks as each gains control of the disk head.
Edit: This is a way to parallelize the line processing while still using serial I/O to read the lines. It uses a BlockingQueue to communicate between threads; the FileTask adds lines to the queue, and the CPUTask reads them and processes them. This is a thread-safe data structure, so no need to add any synchronization to it. You're using put(E e) to add strings to the queue, so if the queue is full (it can hold up to 200 strings, as defined in the declaration in ReadingFile) the FileTask blocks until space frees up; likewise you're using take() to remove items from the queue, so the CPUTask will block until an item is available.
public class ReadingFile {
public static void main(String[] args) {
final int threadCount = 10;
// BlockingQueue with a capacity of 200
BlockingQueue<String> queue = new ArrayBlockingQueue<>(200);
// create thread pool with given size
ExecutorService service = Executors.newFixedThreadPool(threadCount);
for (int i = 0; i < (threadCount - 1); i++) {
service.submit(new CPUTask(queue));
}
// Wait til FileTask completes
service.submit(new FileTask(queue)).get();
service.shutdownNow(); // interrupt CPUTasks
// Wait til CPUTasks terminate
service.awaitTermination(365, TimeUnit.DAYS);
}
}
class FileTask implements Runnable {
private final BlockingQueue<String> queue;
public FileTask(BlockingQueue<String> queue) {
this.queue = queue;
}
#Override
public void run() {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader("D:/abc.txt"));
String line;
while ((line = br.readLine()) != null) {
// block if the queue is full
queue.put(line);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
class CPUTask implements Runnable {
private final BlockingQueue<String> queue;
public CPUTask(BlockingQueue<String> queue) {
this.queue = queue;
}
#Override
public void run() {
String line;
while(true) {
try {
// block if the queue is empty
line = queue.take();
// do things with line
} catch (InterruptedException ex) {
break; // FileTask has completed
}
}
// poll() returns null if the queue is empty
while((line = queue.poll()) != null) {
// do things with line;
}
}
}

We are talking about an average of a 315 MB file with lines separated by new line. I presume this easily fits into memory. It is implied that there is no particular order in the user names that has to be conserved. So I would recommend the following algorithm:
Get the file length
Copy each a 10th of the file into a byte buffer (binary copy should be fast)
Start a thread for processing each of these buffers
Each thread processes all lines in his area except the first and last one.
Each thread must return the first and last partitial line in its data when done,
the “last” of each thread must be recombined with the “first” one of the one working on the next file block because you may have cut through a line. And these tokens must then be processed afterwards.

Fork Join API introduced in 1.7 is a great fit for this use case. Check out http://docs.oracle.com/javase/tutorial/essential/concurrency/forkjoin.html. If you search, you are going to find lots of examples out there.

how to read files in multi process, multi threaded environment

I was successful in reading a file while using multi-process environment using file locking
and in case of multithreaded(singleprocess) i used a queue filled it with file names, opened a thread separately, read from it and then waited till the entire reading was over, after which i used to rename them. In this way i used to read files in multithreaded(in a batch).
Now, i want to read the files in a directory using both multiprocess and multithreads. I tried merging my two approaches but that didn't fare well. log showed a lot of files were showing FileNotFound exception(because their names were changed), some were never read (because thread died), sometimes locks were not released.
///////////////////////////////////////////////////////////////////////
//file filter inner class
class myfilter implements FileFilter{
#Override
public boolean accept(File pathname) {
// TODO Auto-generated method stub
Pattern pat = Pattern.compile("email[0-9]+$");
Matcher mat = pat.matcher(pathname.toString());
if(mat.find()) {
return true;
}
return false;
}
}
/////////////////////////////////////////////////////////////////////////
myfilter filter = new myfilter();
File alreadyread[] = new File[5];
Thread t[] = new Thread[5];
fileread filer[] = new fileread[5];
File file[] = directory.listFiles(filter);
FileChannel filechannel[] = new FileChannel[5];
FileLock lock[] = new FileLock[5];
tuple_json = new ArrayList();
//System.out.println("ayush");
while(true) {
//declare a queue
ConcurrentLinkedQueue filequeue = new ConcurrentLinkedQueue();
//addfilenames to queue and their renamed file names
try{
if(file.length!=0) {
//System.out.println(file.length);
for(int i=0;i<5 && i<file.length;i++) {
System.out.println("acquiring lock on file " + file[i].toString());
try{
filechannel[i] = new RandomAccessFile(file[i], "rw").getChannel();
lock[i] = filechannel[i].tryLock();
}
catch(Exception e) {
file[i] = null;
lock[i] = null;
System.out.println("cannot acquire lock");
}
if(lock[i]!=null){
System.out.println("lock acquired on file " + file[i].toString());
filequeue.add(file[i]);
alreadyread[i] = new File(file[i].toString() + "read");
System.out.println(file[i].toString() + "-----" + times);
}
else{
System.out.println("else condition of acquiring lock");
file[i] = null;
}
System.out.println("-----------------------------------");
}
//starting the thread to read the files
for(int i=0;i<5 && i<file.length && lock[i]!=null && file[i]!=null;i++){
filer[i] = new fileread(filequeue.toArray()[i].toString());
t[i] = new Thread(filer[i]);
System.out.println("starting a thread to read file" + file[i].toString());
t[i].start();
}
//read the text
for(int i=0;i<5 && i<file.length && lock[i]!=null && file[i]!=null;i++) {
try {
System.out.println("waiting to read " + file[i].toString() + " to be read completely");
t[i].join();
System.out.println(file[i] + " was read completetly");
//System.out.println(filer[i].getText());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
//file has been read Now rename the file
for(int i=0;i<5 && i<file.length && lock[i]!=null && file[i]!=null;i++){
if(lock[i]!=null){
System.out.println("renaming file " + file[i].toString());
file[i].renameTo(alreadyread[i]);
System.out.println("releasing lock on file " + file[i].toString());
lock[i].release();
}
}
//rest of the processing
/////////////////////////////////////////////////////////////////////////////////////////////////////
Fileread class
class fileread implements Runnable{
//String loc = "/home/ayusun/workspace/Eclipse/fileread/bin";
String fileloc;
BufferedReader br;
String text = "";
public fileread(String filename) {
this.fileloc = filename;
}
#Override
public void run() {
try {
br = new BufferedReader(new FileReader(fileloc));
System.out.println("started reading file" + fileloc);
String currline;
while((( currline = br.readLine())!=null)){
if(text == "")
text += currline;
else
text += "\n" + currline;
}
System.out.println("Read" + fileloc + " completely");
br.close();
} catch ( IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public String getText() {
return text;
}
}
I would like to know, if there is nay other approach that i can adopt.

If you want to create exclusive access to a file, you cannot use file locking, as on most OSes file locking is advisory, not mandatory.
I'd suggest creating a common lock directory for all your processes; in this lock directory, you would create a directory per file you want to lock, right before you open a file.
The big advantage is that directory creation, unlike file creation, is atomic; as such, you can use Files.createDirectory() (or File's .mkdir() if you still use Java6 but then don't forget to check the return code) to grab a lock on the files you read. If this fails, you know someone else is using the file.
Of course, when you're done with a file, don't forget to remove the lock directory matching this file... (in a finally block)
(note: with Java 7 you can use Files.newBufferedReader(); there is even Files.readAllLines())

If you need to process a large number of files using multiple threads, you should probably first distribute the specific files to each thread before it starts.
For example, if you only want to process files whose names start with email and are followed by some digits, you could create 10 threads. The first thread would look for files with names starting with email0, the second thread could handle email1, etc.
This of course would be efficient only if the numbers are evenly distributed.
Another way could be do have the main thread run through and collect all filenames to deal with. It could then divide the files across the number of available threads, and pass each thread an array of those file names.
There could be other ways of dividing the system load which are relevant to your situation.