Develop Reference

Reading Large file A, Search Records matching records from file B and write file C in java

I have two files assume its already sorted.
This is just example data, in real ill have around 30-40 Millions of records each file Size 7-10 GB file as row length is big, and fixed.
It's a simple text file, once searched record is found. ill do some update and write to file.
File A may contain 0 or more records of matching ID from File B
Motive is to complete this processing in least amount of time possible.
I am able to do but its time taking process...
Suggestions are welcome.
File A
1000000001,A
1000000002,B
1000000002,C
1000000002,D
1000000002,D
1000000003,E
1000000004,E
1000000004,E
1000000004,E
1000000004,E
1000000005,E
1000000006,A
1000000007,A
1000000008,B
1000000009,B
1000000010,C
1000000011,C
1000000012,C
File B
1000000002
1000000004
1000000006
1000000008
1000000010
1000000012
1000000014
1000000016
1000000018\
// Not working as of now. due to logic is wrong.
private static void readAndWriteFile() {
System.out.println("Read Write File Started.");
long time = System.currentTimeMillis();
try(
BufferedReader in = new BufferedReader(new FileReader(Commons.ROOT_PATH+"input.txt"));
BufferedReader search = new BufferedReader(new FileReader(Commons.ROOT_PATH+"search.txt"));
FileWriter myWriter = new FileWriter(Commons.ROOT_PATH+"output.txt");
) {
String inLine = in.readLine();
String searchLine = search.readLine();
boolean isLoopEnd = true;
while(isLoopEnd) {
if(searchLine == null || inLine == null) {
isLoopEnd = false;
break;
}
if(searchLine.substring(0, 10).equalsIgnoreCase(inLine.substring(0,10))) {
System.out.println("Record Found - " + inLine.substring(0, 10) + " | " + searchLine.substring(0, 10) );
myWriter.write(inLine + System.lineSeparator());
inLine = in.readLine();
}else {
inLine = in.readLine();
}
}
in.close();
myWriter.close();
search.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Read and Write to File done in - " + (System.currentTimeMillis() - time));
}

My suggestion would be to use a database. As said in this answer. Using txt files has a big disadvantage over DBs. Mostly because of the lack of indexes and the other points mentioned in the answer.
So what I would do, is create a Database (there are lots of good ones out there such as MySQL, PostgreSQL, etc). Create the tables that are needed, and read the file afterward. Insert each line of the file into the DB and use the db to search and update them.
Maybe this would not be an answer to your concrete question on
Motive is to complete this processing in the least amount of time possible.
But this would be a worthy suggestion. Good luck.

With this approach I am able to process 50M Records in 150 Second on i-3, 4GB Ram and SSD Hardrive.
private static void readAndWriteFile() {
System.out.println("Read Write File Started.");
long time = System.currentTimeMillis();
try(
BufferedReader in = new BufferedReader(new FileReader(Commons.ROOT_PATH+"input.txt"));
BufferedReader search = new BufferedReader(new FileReader(Commons.ROOT_PATH+"search.txt"));
FileWriter myWriter = new FileWriter(Commons.ROOT_PATH+"output.txt");
) {
String inLine = in.readLine();
String searchLine = search.readLine();
boolean isLoopEnd = true;
while(isLoopEnd) {
if(searchLine == null || inLine == null) {
isLoopEnd = false;
break;
}
// Since file is already sorted, i was looking for the //ans i found here..
long seachInt = Long.parseLong(searchLineSubString);
long inInt = Long.parseLong(inputLineSubString);
if(searchLine.substring(0, 10).equalsIgnoreCase(inLine.substring(0,10))) {
System.out.println("Record Found - " + inLine.substring(0, 10) + " | " + searchLine.substring(0, 10) );
myWriter.write(inLine + System.lineSeparator());
}
// Which pointer to move..
if(seachInt < inInt) {
searchLine = search.readLine();
}else {
inLine = in.readLine();
}
}
in.close();
myWriter.close();
search.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Read and Write to File done in - " + (System.currentTimeMillis() - time));
}

Create a shortcut on desktop when run the JAR for the first time [duplicate]

This question already has answers here:
How to run Java programs by clicking on their icon on Windows?
(12 answers)
Closed 6 years ago.
I've searched a lot about this, but I did't find it in Java.
I want to create a shortcut on desktop when run the JAR for the first time without using plugins. I wanna a native way to generate a shortcut when the first time.
I wanna create a txt file just to know whether it's the first time, but the difficult is generate the shortcut.

you can do this by creating a vb script and execute it by java:
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
public class Shourtcut {
public static void main(String[] args) {
String path = System.getProperty("user.dir") + System.getProperty("file.separator") + "shourtcut.jar";
path = '"' + path + '"';
try {
File file = File.createTempFile("shortcut_geni", ".vbs");
file.deleteOnExit();
try (FileWriter fw = new java.io.FileWriter(file)) {
String vbs = "Set oWS = WScript.CreateObject(\"WScript.Shell\") \n"
+ "sLinkFile = oWS.ExpandEnvironmentStrings(\"%HOMEDRIVE%%HOMEPATH%\\Desktop\\jar_file_name.lnk\")\n"
+ "Set oLink = oWS.CreateShortcut(sLinkFile)\n "
+ "oLink.TargetPath = oWS.ExpandEnvironmentStrings(" + path + ")\n"
+ "oLink.Save \n";
fw.write(vbs);
}
Process p = Runtime.getRuntime().exec("wscript " + file.getPath());
p.waitFor();
} catch (IOException | InterruptedException e) {
System.out.println("" + e);
}
}
}

Writing data to a txt file file using java classes

I am trying to understand why my code is not writing the output to the textfile as I expect it to work. My program takes a filename as a command line argument, and prints some text to the file as well as the screen. It is a bit more complicated since it uses classes and objects to demonstrate how objects work. Can anyone help decipher why it is not writing to the file? Here's my code:-
public class Mamoonp3test {
public static void main(String[] args) throws Exception {
//Create array of 10 guitar (Mamoonp3) objects
final int NUMBER_OF_INSTANCES = 10;
Mamoonp3[] objectNames = new Mamoonp3[NUMBER_OF_INSTANCES];
try
{
String fileName = new String(args[0]);
for(int i=0; i<NUMBER_OF_INSTANCES; i++) {
objectNames[i] = new Mamoonp3(FileName);
System.out.println("This is guitar number: " + i);
objectNames[i].tuneGuitar();
objectNames[i].playGuitar();
objectNames[i].displayAcronym();
objectNames[i].stopGuitar();
System.out.println("---------------------------");
}
}
catch (Exception e)
{
System.out.println("please provide an input file");
System.out.println("Usage: java Mamoonp3test filename.txt");
}
}
}
import java.io.*;
public class Mamoonp3 {
final int NUMBER_OF_STRINGS = 6;
char[] stringNames = {'E','A','D','G','B','E'};
int[] stringNumbers = {6,5,4,3,2,1};
String[] stringPitch = {"Sixth","Fifth","Fourth","Third","Second","First"};
boolean isTuned;
boolean isPlaying;
String stringAcronym = new String("Even After Dinner Giant Boys Eat");
//create a PrintWriter for output
PrintWriter output;
public Mamoonp3(String fileName) throws Exception{
isTuned = false;
isPlaying = false;
// create target file
File targetFile = new File(fileName);
//create a PrintWriter for output
output = new PrintWriter(targetFile);
}
public void tuneGuitar() {
System.out.println("The guitar is now tuned.");
for (int i=0; i<NUMBER_OF_STRINGS; i++) {
System.out.println(stringNames[i] + " is string number " + stringNumbers[i] + " and ranked " + stringPitch[i] + " in pitch");
output.print(stringNames[i] + " is string number " + stringNumbers[i] + " and ranked " + stringPitch[i] + " in pitch");
output.close();
}
}
public void playGuitar() {
System.out.println("The guitar is now playing.");
output.print("The guitar is now playing.");
output.close();
}
public void stopGuitar() {
System.out.println("The guitar is now stoped.");
output.print("The guitar is now stoped.");
output.close();
}
public void displayAcronym() {
System.out.println("Always remember your string names!");
System.out.println("Heres a reminder: " + stringAcronym);
output.print("Always remember your string names!");
output.print("Heres a reminder: " + stringAcronym);
output.close();
}
}

You're setting the File of an object that you then do nothing with, that you're not writing with,
Mamoonp3 newObject = new Mamoonp3(fileName);
... and not setting the File in objects that you try to write with. Check which constructors you are using: every Manoop3 object created in the for loop. To see that this is so, check which constructors you're using
I suggest that you change your approach entirely.
Get all file input and output out of your Mamoonp3 class.
Instead, that class should concern itself with representing the state of the musical instrument, and nothing else.
Give the class a decent toString() override method.
I & O should go elsewhere in a separate class of its own.
Give your I&O class a method that allows you to pass Mamoonp3 objects into it so that they can be written.
As an aside, you almost never would use new String(anything). Just use args[0].
Always close your PrintWriter when you are done writing. This is likely causing your error.
Edit
Possibly another way to solve this:
Create a PrintWriter object in the main method.
Give your Manoop3 class a PrintWriter field and a constructor that takes this PrintWriter and sets its field with it.
Write with the PrintWriter in Manoop3, but don't close it.
Then close the PrintWriter in the main method when all Manoop3 objects have completed their use of it.

Open file to view its content [duplicate]

This question already has answers here:
How to open a file with the default associated program
(4 answers)
Closed 9 years ago.
I have a files list. Lets say it looks:
String[] lst = new String[] {
"C:\\Folder\\file.txt",
"C:\\Another folder\\another file.pdf"
};
I need some method to open these files with default program for them, lets say "file.txt" with Notepad, "another file.pdf" with AdobeReader and so on.
Does anyone knows how?

There is a method to do this:
java.awt.Desktop.getDesktop().open(file);
JavaDoc:
Launches the associated application to open the file.
If the specified file is a directory, the file manager of the current platform is launched to open it.

The Desktop class allows a Java application to launch associated applications registered on the native desktop to handle a URI or a file.

If you are using J2SE 1.4 o Java SE 5, the best option is:
for(int i = 0; i < lst.length; i++) {
String path = lst[i];
if (path.indexOf(' ') > 0) {
// Path with spaces
Runtime.getRuntime().exec("explorer \"" + lst[i] + "\"");
} else {
// Path without spaces
Runtime.getRuntime().exec("explorer " + lst[i]);
}
}

Just make sure the file is in the right location, and this should work fine.
try
{
File dir = new File(System.getenv("APPDATA"), "data");
if (!dir.exists()) dir.mkdirs();
File file = new File(dir"file.txt");
if (!file.exists()) System.out.println("File doesn't exist");
else Desktop.getDesktop().open(file);
} catch (Exception e)
{
e.printStackTrace();
}

I didn't know you have a String array now. So, this one uses regex to process the file list in the format you specified before. Ignore if not required.
If the file list is huge and you would prefer that the files open one by one cmd works great. If you want them to open all at once use explorer. Works only on Windows but then on almost all JVM versions. So, there's a trade-off to consider here.
public class FilesOpenWith {
static String listOfFiles = "{\"C:\\Setup.log\", \"C:\\Users\\XYZ\\Documents\\Downloads\\A B C.pdf\"}";
public static void main(String[] args) {
if (args != null && args.length == 1) {
if (args[0].matches("{\"[^\"]+\"(,\\s?\"[^\"]+\")*}")) {
listOfFiles = args[0];
} else {
usage();
return;
}
}
openFiles();
}
private static void openFiles() {
Matcher m = Pattern.compile("\"([^\"]+)\"").matcher(listOfFiles);
while (m.find()) {
try {
Runtime.getRuntime().exec("cmd /c \"" + m.group(1) + "\"");
// Runtime.getRuntime().exec("explorer \"" + m.group(1) + "\"");
} catch (IOException e) {
System.out.println("Bad Input: " + e.getMessage());
e.printStackTrace(System.err);
}
}
}
private static void usage() {
System.out.println("Input filelist format = {\"file1\", \"file2\", ...}");
}
}

Java writing to a text file not working properly

The Java application that I support is logging some details in a flat file. the problem I face some times is that, the entry is very low compared to the previous day. This entry is most essential because our reports are generated based on the file. I went thro code for writing I couldn't figure out any issues. the method which is writing is sync method.
Any suggestions? I can also provide the code for you is you may need?
public synchronized void log (String connID, String hotline, String callerType,
String cli, String lastMenu, String lastInput,
String status, String reason)
{
//String absoluteFP = LOG_LOC + ls + this.getFilename();
//PrintWriter pw = this.getPrintWriter(absoluteFP, true, true);
try
{
pw.print (this.getDateTime ()+ ","+connID +","+hotline+","+callerType+","+ cli+"," + lastMenu + "," + lastInput + "," + status + "," + reason);
//end 1006
pw.print (ls);
pw.flush ();
//pw.close();
}
catch (Exception e)
{
e.printStackTrace ();
return;
}
}
private synchronized PrintWriter getPrintWriter (String absoluteFileName,
boolean append, boolean autoFlush)
{
try
{
//set absolute filepath
File folder = new File (absoluteFileName).getParentFile ();//2009-01-23
File f = new File (absoluteFileName);
if (!folder.exists ())//2009-01-23
{
//System.out.println ("Call Detailed Record folder NOT FOUND! Creating a new);
folder.mkdirs ();
//System.out.println ("Configure log folder");
this.setHiddenFile (LOG_LOC);//set tmp directory to hidden folder
if (!f.exists ())
{
//System.out.println ("Creating a new Call Detailed Record...");//2009-01-23
f.createNewFile ();//2009-01-23
}
}
else
{
if (!f.exists ())
{
//System.out.println ("Creating a new Call Detailed Record...");//2009-01-23
f.createNewFile ();//2009-01-23
}
}
FileOutputStream tempFOS = new FileOutputStream (absoluteFileName, append);
if (tempFOS != null)
{
return new PrintWriter (tempFOS, autoFlush);
}
else
{
return null;
}
}
catch (Exception ex)
{
ex.printStackTrace ();
return null;
}
}
/**
* Set the given absolute file path as a hidden file.
* #param absoluteFile String
*/
private void setHiddenFile (String absoluteFile)
{
//set hidden file
//2009-01-22, KC
Runtime rt = Runtime.getRuntime ();
absoluteFile = absoluteFile.substring (0, absoluteFile.length () - 1);//2009-01-23
try
{
System.out.println (rt.exec ("attrib +H " + "\"" + absoluteFile + "\"").getInputStream ().toString ());
}
catch (IOException e)
{
e.printStackTrace ();
}
}
private String getDateTime ()
{
//2011-076-09, KC-format up to milliseconds to prevent duplicate PK in CDR table.
//return DateUtils.now ("yyyy/MM/dd HH:mm:ss");
return DateUtils.now ("yyyy/MM/dd HH:mm:ss:SSS");
//end 0609
}
private String getFilename ()
{
///return "CDR_" + port + ".dat";//2010-10-01
return port + ".dat";//2010-10-01
}
public void closePW ()
{
if (pw != null)
{
pw.close ();
}
}

You've created a FileOutputStream, but aren't closing that stream. Close that stream and try again. That might be causing the problem.
Messages are getting logged sometime because the garbage collector kicks in at some intervals and closes the FileOutStream. This then allows messages to be logged again. You're getting the unreachable error since you have a return statement in both the if & else blocks. You'll have to take the PrintWriter and FileOutStreamWriter out of the getPrintWriter put it where you usually call the getPrintWriter(). Then you'll be able to close the streams correctly. getPrintWriter should only ensure file exists, so rename it to ensureFileExistance

If you can use Apache Common IO, try this:
public synchronized void log(String connID, String hotline, String callerType,
String cli, String lastMenu, String lastInput,
String status, String reason) {
String absoluteFP = LOG_LOC + ls + this.getFilename();
File file = new File(absoluteFP);
String message = this.getDateTime() + "," + connID + "," + hotline + "," + callerType + "," + cli + "," + lastMenu + "," + lastInput + "," + status + "," + reason;
try {
// note that you must explicitly add new line character if you want the line to end with newline
FileUtils.write(file, message + "\n", "UTF-8", true);
} catch (IOException ex) {
ex.printStackTrace ();
}
}
In Common IO 2.1, you can append a file that you are writting to. You can now get rid of the closePW and getPrintwriter and since the log method is synchronized, the file can be written one at a time from the same object. However, if you try to write the same file from different object at the same time, you will end up having overwritting problem.
Also, Common IO create the missing parent folder for you automatically. There is no need to explicitly check and create the folder.