Develop Reference

Duplicate word frequencies issues in Java [duplicate]

This question already has an answer here:
Duplicate word frequencies problem in text file in Java [closed]
(1 answer)
Closed 1 year ago.
[I am new to Java and Stackoverflow. My last question was closed. I have added a complete code this time. thanks] I have a large txt file of 4GB (vocab.txt). It contains plain Bangla(unicode) words. Each word is in newline with its frequency(equal sign in between). Such as,
আমার=5
তুমি=3
সে=4
আমার=3 //duplicate of 1st word of with different frequency
করিম=8
সে=7 //duplicate of 3rd word of with different frequency
As you can see, it has same words multiple times with different frequencies. How to keep only a single word (instead of multiple duplicates) and with summation of all frequencies of the duplicate words. Such as, the file above would be like (output.txt),
আমার=8 //5+3
তুমি=3
সে=11 //4+7
করিম=8
I have used HashMap to solve the problem. But I think I made some mistakes somewhere. It runs and shows the exact data to output file without changing anything.
package data_correction;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.OutputStreamWriter;
import java.util.*;
import java.awt.Toolkit;
public class Main {
public static void main(String args[]) throws Exception {
FileInputStream inputStream = null;
Scanner sc = null;
String path="C:\\DATA\\vocab.txt";
FileOutputStream fos = new FileOutputStream("C:\\DATA\\output.txt",true);
BufferedWriter bufferedWriter = new BufferedWriter(
new OutputStreamWriter(fos,"UTF-8"));
try {
System.out.println("Started!!");
inputStream = new FileInputStream(path);
sc = new Scanner(inputStream, "UTF-8");
while (sc.hasNextLine()) {
String line = sc.nextLine();
line = line.trim();
String [] arr = line.split("=");
Map<String, Integer> map = new HashMap<>();
if (!map.containsKey(arr[0])){
map.put(arr[0],Integer.parseInt(arr[1]));
}
else{
map.put(arr[0], map.get(arr[0]) + Integer.parseInt(arr[1]));
}
for(Map.Entry<String, Integer> each : map.entrySet()){
bufferedWriter.write(each.getKey()+"="+each.getValue()+"\n");
}
}
bufferedWriter.close();
if (sc.ioException() != null) {
throw sc.ioException();
}
} finally {
if (inputStream != null) {
inputStream.close();
}
if (sc != null) {
sc.close();
}
}
System.out.print("FINISH");
Toolkit.getDefaultToolkit().beep();
}
}
Thanks for your time.

This should do what you want with some mor eJava magic:
public static void main(String[] args) throws Exception {
String separator = "=";
Map<String, Integer> map = new HashMap<>();
try (Stream<String> vocabs = Files.lines(new File("test.txt").toPath(), StandardCharsets.UTF_8)) {
vocabs.forEach(
vocab -> {
String[] pair = vocab.split(separator);
int value = Integer.valueOf(pair[1]);
String key = pair[0];
if (map.containsKey(key)) {
map.put(key, map.get(key) + value);
} else {
map.put(key, value);
}
}
);
}
System.out.println(map);
}
For test.txt take the correct file path. Pay attention that the map is kept in memory, so this is maybe not the best approach. If necessary replace the map with a e.g. database backed approach.

How to replace a line with something? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
So my program needs to overwrite e.g line 5 in a file. Just the 5th line, keep the others.
We don't know what is the content of line 5.
But I have no idea how to do it, can't found anything about how to do this with BufferedWriter and FileWriter.
I can't write there a code, because.. I just don't know how to do it.:/

A sample solution could look like this
package teststuff;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
public class Btest {
public static void main(String args[])
{
try
{
File file = new File("test.txt");
BufferedReader reader = new BufferedReader(new FileReader(file));
String line = "", oldtext = "", fivthLine = "";
int x=0;
while((line = reader.readLine()) != null)
{
oldtext += line + "\r\n";
if(x == 4)
{
fivthLine = line;
}
x++;
}
reader.close();
String newtext = oldtext.replaceAll(fivthLine, "blah blah blah");
FileWriter writer = new FileWriter("test.txt");
writer.write(newtext);writer.close();
}
catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}
Note that this is a combination of what Emmanuel wrote and this
It will also replace whats written in the 5th line everywhere on the file, so that another line containing the same content of line 5 will also be overwritten with
"blah blah blah"

by first you can start looking for "How to count lines on a file" like this i found
https://www.geeksforgeeks.org/counting-number-lines-words-characters-paragraphs-text-file-using-java/
Then add counter++ each time you pass a line, when (counter == 5)
then do whatever you need to do..

This is a very simple example of replacing a given line in a file:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.nio.file.Files;
import java.nio.file.Paths;
public class LineReplaceTest
{
public static void main(String... args)
throws Exception
{
int lineToReplace = 5;
String replacementText = "This is a different line";
Path input = Paths.get("input.txt");
Path output = Paths.get("output.txt");
// Use try-with-resources to ensure our readers & writers are closed
try (BufferedReader reader = Files.newBufferedReader(input);
BufferedWriter writer = Files.newBufferedWriter(output)) {
String line;
int lineNumber = 0;
// While there is a line to read from input...
while ((line = reader.readLine()) != null) {
lineNumber++;
// Write out either the line from the input file or our replacement line
if (lineNumber == lineToReplace) {
writer.write(replacementText);
} else {
writer.write(line);
}
writer.newLine();
}
}
// Once we're done, overwrite the input file
Files.move(output, input);
}
}
It ignores several important things line error handling and platform-specific newline handling, but it should at least get you started.

read words from txt file and sort them [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I am trying to read test.txt and sort it alphabeticly in order and print it to new file sort.txt. Test.txt has exactly 1 word in each line exp. :
mouse
car
noun
swat
Cant figure it out what am i missing.
MANAGE TO FIND OUT WHAT I WAS MISSING!It wrote to file but it gaves me first like 10000 empty lines then starts to output corectly.
Now to the second part.
My second question and its not related to the code is: I have to sort words by length and aplhabeticly. I was thinking first to order them alhabeticly then by length. You think that would work? Any ideas? When done all words needs to be in txt file.
package test;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class sort {
public static void main(String[] args) throws Exception {
String inputFile = "test.txt";
String outputFile = "sort.txt";
FileReader fileReader = new FileReader(inputFile);
BufferedReader bufferedReader = new BufferedReader(fileReader);
String inputLine;
List<String> lineList = new ArrayList<String>();
while ((inputLine = bufferedReader.readLine()) != null) {
lineList.add(inputLine);
}
fileReader.close();
Collections.sort(lineList);
FileWriter fileWriter = new FileWriter(outputFile);
PrintWriter out = new PrintWriter(fileWriter);
for (String outputLine : lineList) {
out.println(outputLine);
}
out.flush();
out.close();
fileWriter.close();
}
}

In Linux u can do cat test.txt | sort > test1.txt from command utility. it's very simple. output will be stored in test1.txt
In java also u can use sort method for string array as below:
Read file to a string array first
///
Path filePath = new File("fileName").toPath();
Charset charset = Charset.defaultCharset();
List<String> stringList = Files.readAllLines(filePath, charset);
String[] strNames = stringList.toArray(new String[]{});
Then use the below method on string array strNames, sort method to sort the array
//String array
String[] strNames = new String[]{"John", "alex", "Chris", "williams", "Mark", "Bob"};
/*
* To sort String array in java, use Arrays.sort method.
* Sort method is a static method. *
*/
//sort String array using sort method
Arrays.sort(strNames);
System.out.println("String array sorted (case sensitive)");
//print sorted elements
for(int i=0; i < strNames.length; i++){
System.out.println(strNames[i]);
}
/*
* Please note that, by default Arrays.sort method sorts the Strings
* in case sensitive manner.
*
* To sort an array of Strings irrespective of case, use
* Arrays.sort(String[] strArray, String.CASE_INSENSITIVE_ORDER) method instead.
*/

Manage to make it work. It sorts words first alphabeticly and second by length. Thanks for all your replies.
package test;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
public class sort {
public static class MyComparator implements Comparator<String>{
#Override
public int compare(String o1, String o2) {
if (o1.length() > o2.length()) {
return 1;
} else if (o1.length() < o2.length()) {
return -1;
}
return o1.compareTo(o2);
}
}
public static void main(String[] args) throws Exception {
String inputFile = "test.txt";
String outputFile = "sort.txt";
FileReader fileReader = new FileReader(inputFile);
BufferedReader bufferedReader = new BufferedReader(fileReader);
String inputLine;
List<String> lineList = new ArrayList<String>();
while ((inputLine = bufferedReader.readLine()) != null) {
lineList.add(inputLine);
}
fileReader.close();
Collections.sort(lineList,String.CASE_INSENSITIVE_ORDER);
FileWriter fileWriter = new FileWriter(outputFile);
PrintWriter out = new PrintWriter(fileWriter);
for (String outputLine : lineList) {
out.println(outputLine);
}
Collections.sort(lineList, new MyComparator());
FileWriter Fw = new FileWriter(outputFile);
PrintWriter pw = new PrintWriter(fileWriter);
for (String outputLine : lineList) {
out.println(outputLine);
}
out.flush();
out.close();
fileWriter.close();
}
}

Implement a Comparator and pass it to Collections.sort(list,comparator).
In your case use Collections.sort(lineList, new MyComparator()); in place of Collections.sort(lineList);
class MyComparator implements Comparator<String> {
public int compare(String s1, String s2) {
if (s1.length() > s2.length()) {
return 1;
} else if (s1.length() < s2.length()) {
return -1;
} else {
return s1.compareTo(s2);
}
}
}
A comparison function, which imposes a total ordering on some collection of objects. Comparators can be passed to a sort method (such as Collections.sort or Arrays.sort) to allow precise control over the sort order. Comparators can also be used to control the order of certain data structures (such as sorted sets or sorted maps), or to provide an ordering for collections of objects that don't have a natural ordering.
Learn more from java docs

Java Program to write text file, using separator, and in first line write record number [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I am trying to write in file, I need to write total number of the record in first line, and in while loop write all other line, everything working good but, in first line need to write total number of the record how can i do that, Please help me!! Thanks!!
Here is my code:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
public class headerline {
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
File folderall = new File("FilesIn1");
File[] BFFileall = folderall.listFiles();
for (final File file : BFFileall) {
String str = file.getName();
BufferedReader br = null;
BufferedWriter lbwp = null;
BufferedWriter lb = null;
try {
int lbwpcount = 1;
int lbcount = 1;
String reprintbwletterbwpca = (str);
lbwp = new BufferedWriter(new FileWriter(reprintbwletterbwpca));
lbwp.write("Total line number: " + lbwpcount);
String reprintbwletterbwpcalb = (str);
lb = new BufferedWriter(new FileWriter(reprintbwletterbwpcalb));
lb.write("Total line number: " + lbwpcount);
br = new BufferedReader(new FileReader(file));
String line;
line = br.readLine();
while ((line = br.readLine()) != null) {
String[] actionID = line.split("|");
String actionid = actionID[2];
String custnumber = actionID[3];
lbwp.write("ActionID: " + actionid + ",CustomerNumber: " + custnumber + "\r\n");
lbwpcount++;
lb.write("ActionID: " + actionid + ",CustomerNumber: " + custnumber + "\r\n");
lbcount++;
}
lbwp.close();
lb.close();
} catch(Exception e1) {
e1.printStackTrace();
}
}
}
}
suppose file has 1201 lines, it should print
Total line number: 1200, in first line.
than
"ActionID: " + actionid + ",CustomerNumber: " + custnumber
..........
suppose other file has 1451 lines, it should print
Total line number: 1450, in first line.
than
"ActionID: " + actionid + ",CustomerNumber: " + custnumber
..........
I have no idea how can i do that, please help me!! can i write first line as last after finish while loop??
Thanks in advanced!!

Simply use java.nio.file package. It has class Files which has a method readAllLines(...). This will read all lines and add it to a List. Simply use List.size() to get number of lines, and write it to another file as you wanted :-)
Try this program, this will let you know the number of lines in the file :
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.util.List;
public class ReadAndWriteFile {
private Path actualPath;
private Path sourcePath;
private BufferedReader reader;
private BufferedWriter writer;
private List<String> lines;
public ReadAndWriteFile() {
sourcePath = Paths.get("xanadu.txt");
//sourcePath = sourcePath.toAbsolutePath();
actualPath = Paths.get("xanadu_new.txt");
//targetPath = actualPath.toAbsolutePath();
Charset charset = Charset.forName("US-ASCII");
try {
lines = Files.readAllLines(sourcePath, charset);
System.out.println("Number of Lines : " + lines.size());
reader = Files.newBufferedReader(sourcePath, charset);
writer = Files.newBufferedWriter(actualPath, charset);
String message = "Total Line Number : " + lines.size();
writer.write(String.format("%s%n", message));
for (String line : lines) {
System.out.println(line);
writer.write(String.format("%s%n", line));
}
reader.close();
writer.close();
} catch(IOException ioe) {
ioe.printStackTrace();
}
}
public static void main(String[] args) {
new ReadAndWriteFile();
}
}
Text File (xanadu.txt) Contents :
In Xanadu did Kubla Khan
A stately pleasure-dome decree:
Where Alph, the sacred river, ran
Through caverns measureless to man
Down to a sunless sea.

You can use Apache Commons FileUtils.readLines(). This returns you all lines in your original file as a list, from which you can get the size.
Otherwise you would have to read all the lines manually to count them first and then write them to the file.

Here's a high level explanation:
Loop through all the lines and store it in a List<Record>.
Write the total number of elements in your List<Record> as the first line (this will be total number of records)
In a loop, write the record data to the rest of the file
Since you need the total first, that means you've gotta loop once to find out what the total is. Then loop again to write the records.

You cannot know the total line number until you have read all the lines of the file.
Some options available to you are:
Read the file twice. The first time, simply keep a count the number of lines. Print it. Then read the file a second time, and write our the records like you are doing at the moment.
Hold the records in memory temporarily. As you read the file, keep the content in a collection (e.g. ArrayList) and write it out at the end. You can use list.size() to get the number of records. You could use use Apache Commons FileUtils.readLines() to do this, if you don't mind introducing a JAR dependency. EDIT: Or write your own method (see below).
Prepend the record count afterwards. Write out the file then as you are doing then prepend the record count to the file.
Challenge the requirements. Ask if it is really necessary to output the record count at the start? Could it be omitted? Could it be at the end? Etc.
Here's some sample code for option 2 - a method for reading the file into a list that you can manipulate:
public static List<String> readLines(File file) throws IOException {
List<String> lines = new ArrayList<String>();
BufferedReader br = new BufferedReader(new FileReader(file));
try {
String line;
while ((line = br.readLine()) != null) {
lines.add(line);
}
} finally {
br.close();
}
return lines;
}
You use it like this:
File file = new File("example.txt");
List<String> lines = readLines(file);
int lineCount = lines.size();
// TODO: Write out the line count
for (String line : lines) {
// TODO: Process the line
}

How to find certain words in a text file, then find numbers in Java?

I have the following text file (answers.txt):
Problem A: 23|47|32|20
Problem B: 40|50|30|45
Problem C: 5|8|11|14
Problem D: 20|23|25|30
What I need is something that will read the problem that I tell it(Problem A, Problem B), then read the numbers after it, which are separated by the lines, and print it out like this:
Answers for Problem A: a.23 b.47 c.32 d.20
Does anyone know how this can be done? I've been stuck on it for a while.

Read the lines one by one, split the lines at " " first. The you will get an array with three parts "Problem", "A:" and "23|47|32|20". Then split the third part at "|" so you will get a second array with four parts "23,"47","32","20".
Combine all to get the output you want.
If you want info on how to read lines from a file, or spilt strings then there are billions of tutorials online on how to do that so I wont go into detail on how its done. IM sure you can find them.

Check out this code!
It assumes that you have such file format:
Problem A:
23|7|32|20
Problem B:
40|50|30|45
Problem C:
5|8|11|14
Problem D:
20|23|25|30
because you wrote "numbers after it, which are separated by the lines"
import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Scanner;
public class Demo {
public static void main(String[] args) throws FileNotFoundException {
Scanner sc = new Scanner(new File("answers.txt"));
List<String> dataList = new ArrayList<String>();
while(sc.hasNextLine()){
dataList.add(sc.nextLine());
}
System.out.println(dataList);
Map<String,String> map = new HashMap<String,String>();
for(int i=0;i<dataList.size();i=i+2){
map.put(dataList.get(i),dataList.get(i+1));
}
for(Entry<String,String> en:map.entrySet()){
System.out.println(en.getKey()+" : "+en.getValue());
}
String problemC = map.get("Problem C:");
String splitted[] = problemC.split("\\|");
System.out.println("Get me problem C: "+String.format("a:%s, b:%s, c:%s, d:%s",splitted[0],splitted[1],splitted[2],splitted[3]));
}
}

Hope this helps!
public static void main(String args[])
{
BufferedReader br = new BufferedReader(new FileReader(new File("answers.txt")));
String lineRead = null;
String problem = "Problem A";//Get this from user Input
List<String> numberData = new ArrayList<String>();
while((lineRead = br.readLine())!=null)
{
if(lineRead.contains(problem))
{
StringTokenizer st = new StringTokenizer(lineRead,":");
String problemPart = st.nextToken();
String numbersPart = st.nextToken();
st = new StringTokenizer(lineRead,"|");
while(st.hasMoreTokens())
{
String number = st.nextToken();
System.out.println("Number is: " + number);
numberData.add(number);
}
break;
}
}
System.out.println("Answers for " + problem + " : " + numberData );
}

Read the lines one by one, split the lines with :. The you will get an array with two parts "Problem A:" and "23|47|32|20". Then split the second part at "|" so you will get a second array with four parts "23,"47","32","20".
Combining all this you will get the output you want.
Cheers!

Use java.util.Scanner and you can filter the integers in the file.
Scanner s = new Scanner (new File ("answers.txt")).useDelimiter("\\s+");
while (s.hasNext()) {
if (s.hasNextInt()) { // check if next token is integer
System.out.print(s.nextInt());
} else {
s.next(); // else read the next token
}
}

Do you know how to read line by line ? If not , chect it How to read a large text file line by line in java?
To sub your string data there have many ways to do. You can sub as you wish. Here for my code..
String data = yourReader.readLine();
String problem = data.substring("Problem".length(), data.indexOf(":"));
System.err.println("Problem is " + problem);
data = data.substring(data.indexOf(":") + 2, data.length());
String[] temp = data.split("\\|");
for (String result : temp) {
System.out.println(result);
}

Assuming there are always four possible answers as in your Example:
// read complete file in fileAsString
String regex = "^(Problem \\w+): (\\d+)\\|(\\d+)\\|(\\d+)\\|(\\d+)$";
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(fileAsString);
//and so on, read all the Problems using matcher.find() and matcher.group(int) to get the parts
// put in a Map maybe?
// output the one you want...

I might suggest creating a simple data type for the purpose of organization:
public class ProblemAnswer {
private final String problem;
private final String[] answers;
public ProblemAnswer(String problem, String[] answers) {
this.problem = problem;
this.answers = new String[answers.length];
for (int i = 0; i < answers.length; i++) {
this.answers[i] = answers[i];
}
}
public String getProblem() {
return this.problem;
}
public String[] getAnswers() {
return this.answers;
}
public String getA() {
return this.answers[0];
}
public String getB() {
return this.answers[1];
}
public String getC() {
return this.answers[2];
}
public String getD() {
return this.answers[3];
}
}
Then the reading from the text file would look something like this:
public void read() {
Scanner s = new Scanner("answers.txt");
ArrayList<String> lines = new ArrayList<String>();
while (s.hasNext()) {
lines.add(s.nextLine());//first separate by line
}
ProblemAnswer[] answerKey = new ProblemAnswer[lines.size()];
for (int i = 0; i < lines.size(); i++) {
String[] divide = lines.get(i).split(": "); //0 is the problem name, 1 is the list
//of answers
String[] answers = divide[1].split("|"); //an array of the answers to a given
//question
answerKey[i] = new ProblemAnswer(divide[0], answers); //add a new ProblemAnswer
//object to the key
}
}
Now that leaves you with an answer key with ProblemAnswer objects which is easily checked
with a simple .equals() comparison on the getProblem() method, and whatever index is matched, you have all the answers neatly arranged right within that same object.