Develop Reference

Java: Parsing a CSV file intermittently fails to read 170K lines in File

I have a CSV file containing customer info, one customer each row.
The CSV file has a size of about 170,000 lines.
The app first parsed the whole file line by line and saved each line as a Customer object into an ArrayList. It implied that the size of the list would also be in the order of 170k.
The code is like the below:
final class CustomerInfoLineProcessor implements LineProcessor<CustomerInfo> {
...
#Override
public boolean processLine(final String line) {
parseLine(line);
return true;
}
private void parseLine(final String line) {
try {
if (!line.trim().isEmpty()) {
//do job
}
} catch (final RuntimeException e) {
handleLineError(e.getClass().getName() + ": " + e.getMessage(), e, lineStatus);
}
}
...
}
It was found intermittently that the parsing process ended abnormally in the middle. No errors or runtime exceptions were thrown. The whole process was also not stopped. The app kept doing further jobs based on whatever inside the ArrayList.
In the beginning, I thought there might be some invisible characters hidden somewhere in the file, which caused the process quit early. But the possibility was excluded after the same file was tested without any problem by the same app on my test machine.
The second guess was: the memory setting -Xmx256m was too small, thus I changed it to an even smaller one, -Xmx128m. The app immediately threw an OutOfMemoryError, and the app was terminated automatically. It implied that the memory usage of -Xmx256m seemed not to be an issue.
Any other reasons I have not yet thought about?

Here is the problem found.
* the client's app ftp the csv file to us in a specified folder every morning;
* then the file_sync app started parsing the cvs file;
* sometimes the cvs file's ftp transferring was not complete while the file_sync app was kicked started. It caused the problem.
Thus the solution is to make sure the csv file is not being opened by another process before starting the file_sync app.

Java Application with Single Instance per User

currently I am struggling with the problem of a single instance JavaFX application, packed into an .exe using install4j. The application should run on a Windows terminal server and every user should only be able to run one instance of it. Meaning, Alice and Bob may use separate instances of the application but Alice may only have one instance open.
Writing a lock file with the process id is not a viable option, since the application is targed at Java 8, which has no consistent possibility to retrieve the process id. Opening a socket is also not a desirable solution, as there can be many instances on the same host. Moreover I suppose admins would not be that happy if some application randomly opened sockets on their server...
As I am using install4j to pack the application, I toggled the 'single instance only' feature which seems to run well when connected via a full RDP session. However, the application may be deployed using the RemoteApp feature which in some way circumvents install4j's checking mechanism, allowing one instance to be launched in a RDP session and another by using the RemoteApp.
This leads me to two questions:
How does the install4j check work? (I was not able to find any details...)
What would be the best solution to ensure a single instance per user at all times? (And also be failsafe, e.g. recover from JVM crashes)
Regarding the possibility of FileLock: as different operating system may handle file locks differently, can it be assured that the file lock is exclusively acquired by one JVM instance on the whole system?

Sockets will be a bit problematic if you want the application to run concurrently under different users.
The option of using an NIO FileLock is possible. You create the file under the user's directory so that another user can have his own lock file. The key thing to do here is to still try to acquire the file lock if the file exists already, by attempting to delete it before recreating it. This way if the application crashes and the file is still there, you will still be able to acquire a lock on it. Remember that the OS should release all locks, open file handles and system resources when a process terminates.
Something like this:
public ExclusiveApplicationLock
throws Exception {
private final File file;
private final FileChannel channel;
private final FileLock lock;
private ExclusiveApplicationLock() {
String homeDir = System.getProperty("user.home");
file = new File(homeDir + "/.myapp", app.lock");
if (file.exists()) {
file.delete();
}
channel = new RandomAccessFile(file, "rw").getChannel();
lock = channel.tryLock();
if (lock == null) {
channel.close();
throw new RuntimeException("Application already running.");
}
Runtime.getRuntime().addShutdownHook(new Thread(() -> releaseLock());
}
private void releaseLock() {
try {
if (lock != null) {
lock.release();
channel.close();
file.delete();
}
}
catch (Exception ex) {
throw new RuntimeException("Unable to release application process lock", ex);
}
}
}
Another alternative is to use a library that does this for you like Junique. I haven't tried it myself but you could have a go. It seems very old but I guess there isn't much that needs to change in something like this, nothing much changed in NIO since Java 1.4.
http://www.sauronsoftware.it/projects/junique/
It is on Maven Central though so you can import it easily.
https://mvnrepository.com/artifact/it.sauronsoftware/junique/1.0.4
If you look at the code you will see that it does the same thing with file locks:
https://github.com/poolborges/it.sauronsoftware.junique/blob/master/src/main/java/it/sauronsoftware/junique/JUnique.java

As for 1: On Windows, install4j launchers create a semaphore with the CreateSemaphore function in the Windows API. You can check the name of the semaphore by executing the launcher from the command line with the
/create-i4j-log
argument.

I faced the same issue, and solved it by using a FileLock like the other answer.
In my case, the arguments that are passed to the launched processes needed to be forwarded to the first process. For this, I used a named pipe, which includes the username in its name. The first process creates the named pipe at \.\pipe\app_$USER. If the same exe is is started by the the same user, it is detected by the FileLock, and the agruments are passed through the named pipe.

showing error in java

I want to create an application that shows a user how many times he opened or used the software. For this I have created the code below. But it is not showing correct output: when I run the application first it is showing 1 and then the second time I run it it is also showing 1.
public Founder() {
initComponents();
int c=0;
c++;
jLabel1.setText(""+c);
return;
}

I’m unsure whether I’m helping you or giving you a load of new problems and unanswered questions. The following will store the count of times the class Founder has been constructed in a file called useCount.txt in the program’s working directory (probably the root binary directory, where your .class files are stored). Next time you run the program, it will read the count from the file, add 1 and write the new value back to the file.
static final Path counterFile = FileSystems.getDefault().getPath("useCount.txt");
public Founder() throws IOException {
initComponents();
// read use count from file
int useCount;
if (Files.exists(counterFile)) {
List<String> line = Files.readAllLines(counterFile);
if (line.size() == 1) { // one line in file as expected
useCount = Integer.parseInt(line.get(0));
} else { // not the right file, ignore lines from it
useCount = 0;
}
} else { // program has never run before
useCount = 0;
}
useCount++;
jLabel1.setText(String.valueOf(useCount));
// write new use count back to file
Files.write(counterFile, Arrays.asList(String.valueOf(useCount)));
}
It’s not the most elegant nor robust solution, but it may get you started. If you run the program on another computer, it will not find the file and will start counting over from 0.

When you are running your code the first time, the data related to it will be stored in your system's RAM. Then when you close your application, all the data related to it will be deleted from the RAM (for simplicity let's just assume it will be deleted, although in reality it is a little different).
Now when you are opening your application second time, new data will be stored in the RAM. This new data contains the starting state of your code. So the value of c is set to 0 (c=0).
If you want to remember the data, you have to store it in the permanent storage (your system hard drive for example). But I think you are a beginner. These concepts are pretty advanced. You should do some basic programming practice before trying such things.

Here you need to store it on permanent basic.
Refer properties class to store data permanently: https://docs.oracle.com/javase/7/docs/api/java/util/Properties.html
You can also use data files ex. *.txt, *.csv
Serialization also provide a way for persistent storage.
You can create a class that implements Serializable with a field for each piece of data you want to store. Then you can write the entire class out to a file, and you can read it back in later.Learn about serialization here:https://www.tutorialspoint.com/java/java_serialization.htm

Java : one lock per method per object

I'm building a webcrawler and it has 2 main feature wich are both executed as threads :
-The fetcher (crawl a website and separate links from files store both of them into the database).
-The downloader (download files based on their url returned by the fetcher).
I've an object WebSite wich include everything I want to know about a website. Now I want to manipulate my database to change the status of a link from waiting to fetching then to fetched. The same goes for files from waiting to downloading then to downloaded.
To prevent a Fetcher to fetch a link that has been chosen by another fetcher I've done this function inside my WebSite object :
public synchronized String[] getNextLink(){
//Return the next link from database that has visited set to 0 then change it to -1 to say that it's in-use.
}
And I've done the same for my Downloaders with this function :
public synchronized String getNextFile(){
//Return the next file from database that has downloaded set to 0 then change it to -1 to say that it's downloading
}
Both method are inside my WebSite object since if 2 Fetchers are working with different websites they cannot Select the same row inside my database (same goes for downloaders). But both function can be called at the same time because Fetchers never select a file and Downloaders never select a link.
Now synchronized is using a single lock (per object) so both of my methods cannot be called at the same time. Is there another keyword to use one lock per method per object ? Or do I need to code it ?

Instead of applying the synchronized keyword to whole methods, which implicitly uses this as a lock-object, you can use two independent lock objects (and any object can be used as a lock-object in Java) within the methods. Each lock object will be independent of others:
private final Object fetcherMutex = new Object();
private final Object downloaderMutex = new Object();
public String[] getNextLink(){
synchronized (fetcherMutex) { /* ... */ }
}
public String[] getNextFile(){
synchronized (downloaderMutex) { /* ... */ }
}

Create worker thread which performs specific task in the background

As per My Project,
Data has been fetched from database through a query,
There is an Iterator on result set and data has been added continuously to this result set.
By iterating over Iterator object results are added to ArrayList.
Once we got all the entries (more than 200000) then writing it to a file.
But as it is using more heap space of jvm ,I need to use a worker thread which runs in back ground and writes the data to the file.
As I am new to multi threading ,
I thought of using Executor service by creating fixed thread pool of 1 thread and whenever entries reaches the count of 50000 ,then submit those entries to executor to append them to file.
please suggest me if this approach is fine or do I need to follow any other approach.

I don't think you need a ThreadPool in order to handle single thread. You can do it by creating a single thread(pseudo code):
List<Entry> list = new ArraList<Entry>(); // class member that will hold the entries from Result set. I Assume entry as `Entry` here
....
void addEntry(Entry entry){
list.add(entry);
if(list.size() >= 20000){
//assign current list to a temp list inorder to reinitialze the list for next set of entries.
final List tempList = list;// tempList has 20000 entries!
list = new ArraList<Entry>();// list is reinitialized
// initiate a thread to write tempList to file
Thread t = new Thread(new Runnable(){
public void run() {
// stuff that will write `tempList` to file
}});
t.start();// start thread for writing.It will be run in background and
//the calling thread (from where you called `addEntry()` )will continue to add new entries to reinitialized list
}//end of if condition
}
Note: You mentioned about the heap space - even if we use thread it still uses heap.

Executing the process in a thread will free up the main thread to do other stuff.
It will not solve your heap space problem.
The heap space problem is caused by the number of entries returned from the query. You could change your query to return only a set number of rows. Process that and do the query again starting from the last row that you processed.
If you are using MS SQL, there is already an answer here on how to split your queries.
Row offset in SQL Server

You don't need to fetch all 20000 entries before writing them to file, unless they have some dependencies to each other.
In the simplest case you can write the entries directly to file as you're fetching them, making it unnecessary to have large amounts of heap.
An advanced version of that is the producer-consumer pattern, which you can then adjust to get different speed/memory use characteristics.

Created worker thread which process entries in the beckground.Starting this thread before fetching entries and stopping it when finished fetching all entries,
public class WriteToOutputFile implements Runnable{
BlockingQueue<entry> queue;
File file;
volatile boolean processentries;
WriteToOutputFile(BlockingQueue queue,File file){
this.queue = queue;
this.file = file;
this.processentries= tue;
}
#override
public void run(){
while(processentries && !queue.isEmpty()){
entry = queue.take();
if (entry== lastentry)break;
//logic to write entries to file
} }
public void stop(){
processentries = false;
queue.put(lastentry);
}
}