Develop Reference

How do I count word occurrences in a csv file?

I have a CSV file that I need to read and display the number of occurrences of each word, the application should only count words that have more than one letter and not alphanumerical also turned to lowercase.
This is what I have right now and I'm stuck at this and have no ideas where to go from this.
public static void countWordNumber() throws IOException, CsvException
String pathFile1 = "src/main/resources/Documents/Example.csv"
{
CSVReader reader = new CSVReaderBuilder(new FileReader(pathFile1)).withSkipLines(1).build();
Map<String, Integer> frequency = new HashMap<>();
String[] line;
while ((line = reader.readNext()) != null) {
String words = line[1];
words = words.replaceAll("\\p{Punct}", " ").trim();
words = words.replaceAll("\\s{2}", " ");
words = words.toLowerCase();
if (frequency.containsKey(words)) {
frequency.put(words, frequency.get(words) + 1);
} else {
frequency.put(words, 0);
}
}
}
I am trying to read the second index in the array list of the csv, which is line[1] , This is where the text of the document is located.
I have replaced all punctuation with spaces and trimmed it, also if there are more than 2 spaces I have replaced those with 1 and made it lowercase.
The output I am trying to achieve is:
Title of document: XXXX
Word: is, Value: 3
EDIT: This is an example of my input file.
title,text,date
exampleTitle,This is is is an example example, April 2022

Your solution does not look that bad. But for initialization i would replace
frequency.put(words, 0);
with
frequency.put(words, 1);
Since I am mising your Input file i created a dummy that works fine.
Map<String, Integer> frequency = new HashMap<>();
List<String> csvSimulation = new ArrayList<String>();
csvSimulation.add("test");
csvSimulation.add( "marvin");
csvSimulation.add("aaaaa");
csvSimulation.add("nothing");
csvSimulation.add("test");
csvSimulation.add("test");
csvSimulation.add("aaaaa");
csvSimulation.add("stackoverflow");
csvSimulation.add("test");
csvSimulation.add("bread");
Iterator<String> iterator = csvSimulation.iterator();
while(iterator.hasNext()){
String words = iterator.next();
words = words.toLowerCase();
if (frequency.containsKey(words)) {
frequency.put(words, frequency.get(words) + 1);
} else {
frequency.put(words, 1);
}
}
System.out.println(frequency);
Are you sure that accessing line[1] in an loop while iteration is correct? The correct reading of the input seems to be the problem for me. Without seeing your CSV file i however cant help you any further.
EDIT:
with the provided csv data an adjustemt to your Code like this would solve your Problem
.....
.....
while ((line = reader.readNext()) != null) {
String words = line[1];
words = words.replaceAll("\\p{Punct}", " ").trim();
words = words.replaceAll("\\s{2}", " ");
words = words.toLowerCase();
String[] singleWords = words.split(" ");
for(int i = 0 ; i < singleWords.length; i++) {
String currentWord = singleWords[i];
if (frequency.containsKey(currentWord)) {
frequency.put(currentWord, frequency.get(currentWord) + 1);
} else {
frequency.put(currentWord, 1);
}
}
}
System.out.println("Word: is, Value: " + frequency.get("is"));

You can use a regex match to verify that words fits your criteria before adding it to your HashMap, like so:
if (words.matches("[a-z]{2,}"))
[a-z] specifies only lowercase alpha chars
{2,} specifies "Minimum of 2 occurrences, maximum of <undefined>"
Though, given you're converting punctuation to spaces, it sounds like you could have multiple words in line[1]. If you want to gather counts of multiple words across multiple lines, then you may want to split words on the space char, like so:
for (String word : words.split(" ")) {
if (word.matches("[a-z]{2,}")) {
// Then use your code for checking if frequency contains the term,
// but use `word` instead of `words`
}
}

Just another twist on things:
Since it's been established (by OP) that the CSV file consists of Title, Text, Date data, it can be assumed that every data line of that file is delimited with the typical comma (,) and, that each line of that CSV data file (other than the Header line) can potentially contain a different Title.
Yet, the established desired output (by OP) is:
Title of document: exampleTitle
Word: is, Value: 3
Word: example, Value: 2
Let's change this output so that things are wee more pleasing to the eye:
-------------------------------
Title of document: exampleTitle
-------------------------------
Word Value
=====================
an 1
is 3
example 2
this 1
=====================
Based on this information, it only seems logical that because each file data line contains a Title we need to process and store word occurrences from column 2 only for that data line. With each word being processed we need to maintain the origin of that word so that we know what Title it came from rather than just carrying out a simple occurrence count of all column 2 words in all rows (file lines). This if course then means that the KEY for the Map which would be a word must also hold the origin of the word. This isn't a big deal but a little more thought will be needed when it comes time to pull relevant data from the Map so to display it properly in the Console Window or, to use for other purposes within the application. What we could do is utilize a List Interface of Map, for example:
List<Map<String, Integer>> mapsList = new ArrayList<>();
By doing this we can place each file line processed into a Map and then add that map to a List named mapsList.
The provided text file contents example leaves little to be desired, never the less, it does help to some extent in the sense that, it comforts me to know that, yes, there is a Header line in the CSV data file and the fact that the typical comma is used as the delimiter in the file .... and that's all. So more questions come to mind:
Can more than one file data line contain the same Title (Origin)?
If "yes", then what do you want to do with the word
occurrences for the other line(s)?
Do you want to add them to the first established
Title?
Or do you want to start a new modified Title? (must be a different title name in this case)
Will the second column (the text column) ever possibly contain a
comma delimiter? Commas are very common in text of reasonable
length.
If "yes", is the text in column 2 enclosed in quotation marks?
How long can the text in column 2 possibly get to? (just curious - it's actually irrelevant).
Will there ever be different CSV files to get Word Occurrences from
which contain more than three columns?
Will there ever be a time where Word Occurrences will need to be
derived from more than one column on any CSV file data line?
The method provided below (in the runnable application) named getWordOccurrencesFromCSV() is flexible enough to basically cover all the questions above. This method also makes use of two other helper methods named getWordOccurrences() and combineSameOrigins() to get the job done. These helper methods can be used on their own for other situations if so desired although the combineSameOrigins() method is uniquely designed for the getWordOccurrencesFromCSV() method. The startApp() method gets the ball rolling and displays the generated List of Maps into the console Window.
Here is the runnable code (be sure to read ALL the comments in code):
package so_demo_hybridize;
public class SO_Demo_hybridize {
private final java.util.Scanner userInput = new java.util.Scanner(System.in);
public static void main(String[] args) {
// Started this way to avoid the need for statics.
new SO_Demo_hybridize().startApp(args);
}
private void startApp(String[] args) {
String ls = System.lineSeparator();
String filePath = "DemoCSV.csv"; //"DemoCSV.csv";
/* Retrieve the neccessary data from the supplied CSV
file and place it into a List of Maps:
getWordOccurrencesFromCSV() Parameters Info:
--------------------------------------------
filePath: Path and file name of the CSV file.
"," : The delimiter used in the CSV file.
1 : The literal data column which will hold the Origin String.
2 : The literal data column which will hold text of words to get occurrences from.
1 : The minimum number of occurrences needed to save. */
java.util.List<java.util.Map<String, Integer>> mapsList
= getWordOccurrencesFromCSV(filePath, ",", 1, new int[]{2}, 1);
/* Display what is desired from the gathered file data now
contained within the Maps held within the 'mapsList' List.
Now that you have this List of Maps, you can do whatever
and display whatever you like with the data. */
System.out.println("Word Occurrences In CSV File" + ls
+ "============================" + ls);
for (java.util.Map<String, Integer> maps : mapsList) {
String mapTitle = "";
int cnt = 0;
for (java.util.Map.Entry<String, Integer> entry : maps.entrySet()) {
/* Because the Origin is attached to the Map Key (a word)
we need to split it off. Note the special delimiter. */
String[] keyParts = entry.getKey().split(":\\|:");
String wordOrigin = keyParts[0];
String word = keyParts[1];
if (mapTitle.isEmpty()) {
mapTitle = "Title of document: " + wordOrigin;
// The Title underline...
String underLine = String.join("", java.util.Collections.nCopies(mapTitle.length(), "-"));
System.out.println(underLine + ls + mapTitle + ls + underLine);
// Disaplay a Header and underline
String mapHeader = "Words Values" + ls
+ "=====================";
System.out.println(mapHeader);
cnt++;
}
System.out.println(String.format("%-15s%-6s", word, entry.getValue()));
}
if (cnt > 0) {
// The underline for the Word Occurences table displayed.
System.out.println("=====================");
System.out.println();
}
}
}
/**
* Retrieves and returns a List of Word Occurrences Maps ({#code Map<String,
* Integer>}) from each CSV data line from the specified column. The each
* word is considered the KEY and the number of Occurrences of that word
* would be VALUE. Each KEY in each Map is also prefixed with the Origin
* String of that word delimited with ":|:". READ THIS DOCUMENT IN FULL!
*
* #param csvFilePath (String) The full path and file name of the
* CSV file to process.<br>
*
* #param csvDelimiter (String) The delimiter used within the CSV
* File. This can be and single character
* string including the whitespace. Although
* not mandatory, adding whitespaces to your
* CSV Delimiter argument should be
* discouraged.<br>
*
* #param originFromColumn (Integer) The The CSV File line data column
* literal number where the Origin for the
* evaluated occurrences will be related to. By
* <b><i>literal</i></b> we mean the actual
* column number, <u>not</u> the column Index
* Value. Whatever literal column number is
* supplied, the data within that column should
* be Unique to all other lines within the CSV
* data file. In most CSV files, records in that
* file (each line) usually contains one column
* that contains a Unique ID of some sort which
* identifies that line as a separate record. It
* would be this column which would make a good
* candidate to use as the <b><i>origin</i></b>
* for the <i>Word Occurrences</i> about to be
* generated from the line (record) column
* specified from the argument supplied to the
* 'occurrencesFromColumn' parameter.<br><br>
*
* If null is supplied to this parameter then
* Column one (1) (index 0) of every data line
* will be assumed to contain the Origin data
* string.<br>
*
* #param occurrencesFromColumn (int[] Array) Because this method can gather
* Word Occurrences from 1 <b><i>or
* more</i></b> columns within any given CSV
* file data line, the literal column number(s)
* must be supplied within an <b>int[]</b> array.
* The objective of this method is to obviously
* collect Word Occurrences contained within
* text that is mounted within at least one
* column of any CSV file data line, therefore,
* the literal column number(s) from where the
* Words to process are located need to be suppled.
* By <b><i>literal</i></b> we mean the actual
* column number, <u>not</u> the column Index
* Value. The desired column number (or column
* numbers) can be supplied to this parameter
* in this fashion: <b><i>new int[]{3}</i></b>
* OR <b><i>new int[]{3,5,6}</i></b>.<br><br>
* All words processed, regardless of what
* columns they come from, will all fall under
* the same Origin String.<br><br>
*
* Null <b><u>can not</u></b> be supplied as an
* argument to this parameter.<br>
*
* #param minWordCountRequired (Integer) If any integer value less than 2
* is supplied then all words within the
* supplied Input String will be placed into
* the map regardless of how many there are. If
* however you only want words where there are
* two (2) or more within the Input String then
* supply 2 as an argument to this parameter.
* If you only want words where there are three
* (3) or more within the Input String then
* supply 3 as an argument to this parameter...
* and so on.<br><br>
*
* If null is supplied to this parameter then a
* default of one (1) will be assumed.<br>
*
* #param options (Optional - Two parameters both boolean):<pre>
*
* noDuplicateOrigins - Optional - Default is true. By default, duplicate
* Origins are not permitted. This means that no two
* Maps within the List can contain the same Origin
* for word occurrences. Obviously, it is possible
* for a CSV file to contain data lines which holds
* duplicate Origins but in this case the words in
* the duplicate Origin are added to the Map within
* the List which already contains that Origin and
* if any word in the new duplicate Origin Map is
* found to be in the already stored Map with the
* original Origin then the occurrences count for
* the word in the new Map is added to the same word
* occurrences count of the already stored Map.
*
* If boolean <b>false</b> is optionally supplied to this
* parameter then a duplicate Map with the duplicate
* Origin is added to the List of Maps.
*
* Null can be supplied to this optional parameter.
* You can not just supply a blank comma.
*
* noNumerics - Optional - Default is false. By default, this
* method considers numbers (either integer or
* floating point) as words therefore this
* parameter would be considered as always
* false. If however you don't want then
* occurrences of numbers to be placed into the
* returned Map then you can optionally supply
* an argument of boolean true here.
*
* If null or a boolean value is supplied to this
* optional parameter then null or a boolean value
* <u>must</u> be supplied to the noDuplicateOrigins
* parameter.</pre>
*
* #return ({#code List<Map<String, Integer>>})
*/
#SuppressWarnings("CallToPrintStackTrace")
public java.util.List<java.util.Map<String, Integer>> getWordOccurrencesFromCSV(
String csvFilePath, String csvDelimiter, Integer originFromColumn,
int[] occurrencesFromColumn, Integer minWordCountRequired, Boolean... options) {
String ls = System.lineSeparator();
// Handle invalid arguments to this method...
if (!new java.io.File(csvFilePath).exists()) {
throw new IllegalArgumentException(ls + "getWordOccurrencesFromCSV() "
+ "Method Error! The file indicated below can not be found!" + ls
+ csvFilePath + ls);
}
else if (csvFilePath == null || csvFilePath.isEmpty()) {
throw new IllegalArgumentException(ls + "getWordOccurrencesFromCSV() "
+ "Method Error! The csvFilePath parameter can not be supplied "
+ "null or a null string!" + ls);
}
else if (csvDelimiter == null || csvDelimiter.isEmpty()) {
throw new IllegalArgumentException(ls + "getWordOccurrencesFromCSV() "
+ "Method Error! The csvDelimiter parameter can not be supplied "
+ "null or a null string!" + ls);
}
if (originFromColumn == null || originFromColumn < 1) {
originFromColumn = 1;
}
for (int i = 0; i < occurrencesFromColumn.length; i++) {
if (occurrencesFromColumn[i] == originFromColumn) {
throw new IllegalArgumentException(ls + "getWordOccurrencesFromCSV() "
+ "Method Error! The 'occurrencesFromColumn' argument ("
+ occurrencesFromColumn[i] + ")" + ls + "can not be the same column "
+ "as the 'originFromColumn' argument (" + originFromColumn
+ ")!" + ls);
}
else if (occurrencesFromColumn[i] < 1) {
throw new IllegalArgumentException(ls + "getWordOccurrencesFromCSV() "
+ "Method Error! The argument for the occurrencesFromColumn "
+ "parameter can not be less than 1!" + ls);
}
}
if (minWordCountRequired == null || minWordCountRequired < 2) {
minWordCountRequired = 1;
}
final int minWrdCnt = minWordCountRequired;
// Take care of the Optional Parameters
boolean noDuplicateOrigins = true;
boolean noNumerics = false;
if (options != null && options.length > 0) {
if (options[0] != null && options.length >= 1) {
noDuplicateOrigins = options[0];
}
if (options[1] != null && options.length >= 2) {
noNumerics = options[1];
}
}
java.util.List<java.util.Map<String, Integer>> mapsList = new java.util.ArrayList<>();
// 'Try With Resources' is used here to auto-close file and free resources.
try (java.util.Scanner reader = new java.util.Scanner(new java.io.FileReader(csvFilePath))) {
String line = reader.nextLine(); // Skip the Header line (first line in file).
String origin = "", date = "";
java.util.Map<String, Integer> map;
while (reader.hasNextLine()) {
line = reader.nextLine().trim();
// Skip blank lines (if any)
if (line.isEmpty()) {
continue;
}
// Get columnar data from data line
// If there are no quotation marks in data line.
String regex = "\\s*\\" + csvDelimiter + "\\s*";
/* If there are quotation marks in data line and they are
actually balanced. If they're not balanced the we obviously
use the regular expression above. The regex below ignores
the supplied delimiter contained between quotation marks. */
if (line.contains("\"") && line.replaceAll("[^\"]", "").length() % 2 == 0) {
regex = "\\s*\\" + csvDelimiter + "\\s*(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)";
}
String[] csvColumnStrings = line.split(regex);
// Acquire the Origin String
origin = csvColumnStrings[originFromColumn - 1];
// Get the Word Occurrences from the provided column number(s)...
for (int i = 0; i < occurrencesFromColumn.length; i++) {
/* Acquire the String to get Word Occurrences from
and remove any punctuation characters from it. */
line = csvColumnStrings[occurrencesFromColumn[i] - 1];
line = line.replaceAll("\\p{Punct}", "").replaceAll("\\s+", " ").toLowerCase();
// Get Word Occurrences...
map = getWordOccurrences(origin, line, noNumerics);
/* Has same Origin been processed before?
If so, do we add this one to the original? */
if (noDuplicateOrigins || i > 0) {
if (combineSameOrigins(mapsList, map) > 0) {
continue;
}
}
mapsList.add(map);
}
}
}
catch (java.io.FileNotFoundException ex) {
ex.printStackTrace();
}
/* Remove any words from all the Maps within the List that
does not meet our occurrences minimum count argument
supplied to the 'minWordCountRequired' parameter.
(Java8+ needed) */
for (java.util.Map<String, Integer> mapInList : mapsList) {
mapInList.entrySet().removeIf(e -> e.getValue() < minWrdCnt);
}
return mapsList; // Return the generated List of Maps
}
/**
* This method will go through all Maps contained within the supplied List of
* Maps and see if the Origin String within the supplied Map already exists
* within a Listed Map. If it does then those words within the Supplied Map
* are added to the Map within the List is they don't already exist there.
* If any words from the Supplied Map does exist within the Listed Map then
* only the count values from those words are summed to the words within the
* Listed Map.<br>
*
* #param list ({#code List of Map<String, Integer>}) The List Interface which
* contains all the Maps of Word Occurrences or different Origins.<br>
*
* #param suppliedMap ({#code Map<String, Integer> Map}) The Map to check
* against all Maps contained within the List of Maps for duplicate Origins.
*
* #return (int) The number of words added to any Map contained within the
* List which contains the same Origin.
*/
public int combineSameOrigins(java.util.List<java.util.Map<String, Integer>> list,
java.util.Map<String, Integer> suppliedMap) {
int wrdCnt = 0;
String newOrigin = suppliedMap.keySet().stream().findFirst().get().split(":\\|:")[0].trim();
String originInListedMap;
for (java.util.Map<String, Integer> mapInList : list) {
originInListedMap = mapInList.keySet().stream().findFirst().get().split(":\\|:")[0].trim();
if (originInListedMap.equals(newOrigin)) {
wrdCnt++;
for (java.util.Map.Entry<String, Integer> suppliedMapEntry : suppliedMap.entrySet()) {
String key = suppliedMapEntry.getKey();
int value = suppliedMapEntry.getValue();
boolean haveIt = false;
for (java.util.Map.Entry<String, Integer> mapInListEntry : mapInList.entrySet()) {
if (mapInListEntry.getKey().equals(key)) {
haveIt = true;
mapInListEntry.setValue(mapInListEntry.getValue() + value);
break;
}
}
if (!haveIt) {
mapInList.put(key, value);
}
}
}
}
return wrdCnt;
}
/**
* Find the Duplicate Words In a String And Count the Number Of Occurrences
* for each of those words. This method will fill and return a Map of all
* the words within the supplied string (as Key) and the number of
* occurrences for each word (as Value).<br><br>
* <p>
* <b>Example to read the returned Map and display in console:</b><pre>
* {#code for (java.util.Map.Entry<String, Integer> entry : map.entrySet()) {
* System.out.println(String.format("%-12s%-4s", entry.getKey(), entry.getValue()));
* } }</pre>
*
* #param origin (String) The UNIQUE origin String of what the word is
* related to. This can be anything as long as this same
* origin string is applied to all the related words of
* the same Input String. A unique ID of some sort or a
* title string of some kind would work fine for
* this.<br>
*
* #param inputString (String) The string to process for word
* occurrences.<br>
*
* #param noNumerics (Optional - Default - false) By default, this method
* considers numbers (either integer or floating point)
* as words therefore this parameter would be considered
* as always false. If however you don't want the
* occurrences of numbers to be placed into the returned
* Map then you can optionally supply an argument of
* boolean true here.<br>
*
* #return ({#code java.util.Map<String, Integer>}) Consisting of individual
* words found within the supplied String as KEY and the number of
* occurrences as VALUE.
*/
public static java.util.Map<String, Integer> getWordOccurrences(String origin,
String inputString, Boolean... noNumerics) {
boolean allowNumbersAsWords = true;
if (noNumerics.length > 0) {
if (noNumerics[0] != null) {
allowNumbersAsWords = !noNumerics[0];
}
}
// Use this to have Words in Ascending order.
java.util.TreeMap<String, Integer> map = new java.util.TreeMap<>();
// Use this to have Words in the order of Insertion (when they're added).
//java.util.LinkedHashMap<String, Integer> map = new java.util.LinkedHashMap<>();
// Use this to have Words in a 'who cares' order.
//java.util.Map<String, Integer> map = new java.util.HashMap<>();
String[] words = inputString.replaceAll("[^A-Za-z0-9' ]", "")
.replaceAll("\\s+", " ").trim().split("\\s+");
for (int i = 0; i < words.length; i++) {
String w = words[i];
if (!allowNumbersAsWords && w.matches("-?\\d+(\\.\\d+)?")) {
continue;
}
if (map.containsKey(origin + ":|:" + w)) {
int cnt = map.get(origin + ":|:" + w);
map.put(origin + ":|:" + w, ++cnt);
}
else {
map.put(origin + ":|:" + w, 1);
}
}
return map;
}
}

Replace all black-labeled items if not part of the white-labeled items in text

I need to replace all occurrences from text with *, which are
available in the black-labeled list and
bordered by (spaces or "," or "." or start or end of text) and
not part of any white-labeled item
Example:
["is", "panter"] // black-labeled
["pink panter", "blue panter"] // white-labeled
"This is pink panter." -> "This * pink panter."
"This is black panter." -> "This * black *."
String input = ...;
List<String> whiteLabelded = ...;
List<String> blackLabelded = ...;
String enhencedText = enhenceText(input, whiteLabelded, blackLabelded);
My try:
Map<String, String> whiteLabeldedMap = assignUniquePlaceHolder(whiteLabelded); // like ####1, ####2, ####3
String output = input;
whiteLabeldedMap.forEach((key, value) -> output = output.replace(key, value)); // replace white labeled strings
blackLabelded.forEach(key -> output = output.replace(key, "*")); // replace black labeled strings
whiteLabeldedMap.forEach((key, value) -> output = output.replace(value, key)); // return white labeled strings
Is it possible to implement better?

A first version that roughly does what you want, very inefficiently, but hey, it needs to be correct before it's fast. Adapt to your needs and improve performance if needed.
import java.util.Arrays;
import java.util.List;
public class BlackWhiteLists {
/**
* Replace words in black lists by * unless they are in whitelist
* #param input the input
* #param white the white list
* #param black the black list
* #return the replaced string
*/
public static String enhanceText(String input, List<String> white, List<String> black) {
String result=input;
for (String bs:black) {
// find text to remove
int ix=result.indexOf(bs);
while (ix>-1) {
boolean ok=true;
// find if text we want to keep overlaps
for (String ws:white) {
int ix2=input.indexOf(ws);
while (ok && ix2>-1) {
// overlaps
if (ix>=ix2 && ix<ix2+ws.length() ) {
ok=false;
}
// search for next instance of white
ix2=input.indexOf(ws,ix2+ws.length());
}
}
// no overlap, we replace
if (ok && isWord(result,ix,bs.length())) {
result=result.substring(0,ix)+"*"+result.substring(ix+bs.length());
// search for next instance of black
ix=result.indexOf(bs,ix+1);
} else {
// search for next instance of black
ix=result.indexOf(bs,ix+bs.length());
}
}
}
return result;
}
/**
* Is the text at the specific place in the input a word
* #param input the input string
* #param ix the start index
* #param length the text length
* #return true if the text is not preceded or followed by another letter
*/
public static boolean isWord(String input, int ix, int length) {
if (ix>0) {
if (Character.isLetter(input.charAt(ix-1))){
return false;
}
}
if (ix+length<input.length()) {
if (Character.isLetter(input.charAt(ix+length))){
return false;
}
}
return true;
}
public static void main(String[] args) {
List<String> white=Arrays.asList("pink panter", "blue panter");
List<String> black=Arrays.asList("is", "panter");
System.out.println(enhanceText("This is pink panter.",white,black));
System.out.println(enhanceText("This is black panter.",white,black));
}
}

I am not sure if this is of help. Anyway i assume you want to solve this task with some hints? So here are some hints:
A text can be considered an array of characters (assuming it is one long string). First letter has index 0 and the last character has index "length-1".
1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0
T h i s i s b l a c k p a n t e r .
Knowing this you can start the comparisons:
for a black labeled string (here: panter) see if it is found in the string
if the string is not found -> good. nothing to do
if the string is found write down the index where it starts (java string substring gives information about this) and how long it is (eg start index 14, length 6 for given example "panter")
now do for all white labeled items
find if it matches the string and if it does: find where the start index is and how long the match is (eg match at position 8 and length is 12 for above example black panter)
(java string object has methods for this) if the location overlaps with the location from step 2. than the string is white labeled and will remain unchanged. Otherwise it has to be replaced (again the java string object has methods for his)

Replace java giving 2 strings

Well as you can read, when i use the replace function in my code it prints out:
Hey, I'm.
Hey, (name).
When it should only print out Hey, (name).
And i dont understand why. Here is the code:
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
package fiestaaburrida;
import java.util.Scanner;
/**
*
* #author xBaco
*/
public class FiestaAburrida {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
// TODO code application logic here
Scanner teclado = new Scanner(System.in);
int times = teclado.nextInt();
int index = 0;
while (index < times){
String greeting = teclado.next();
String newgreeting = greeting.replace("I'm ","");
System.out.println("Hey, "+newgreeting+".");
}
}
}

Scanner.next() will read your input up to the next delimeter, which by default is a space. Because of this, you are getting two inputs, I'm Joe, rather than the one input I'm Joe.
If you want to take in an entire line at once, you should use Scanner.nextLine(). (Though you should watch out for this quirk which will affect you because of your first nextInt.)

It is because teclado.next(); fetches the next value in the console that is separated by a space. You want to use teclado.nextLine();. Although this is not a complete fix. If you were to follow up with this approach and enter "I'm Jake", than the program would print "Hey, ." followed by "Hey, Jake". This is because you are using teclado.nextInt();, which works, but it will not cause Scanner#nextLine() to read "I'm Jake" immediately. Thus you must also replace nextInt(); with nextLine(); and parse it:
public static void main(String[] args) {
Scanner teclado = new Scanner(System.in);
int times = Integer.parseInt(teclado.nextLine());
int index = 0;
while (index < times){
String greeting = teclado.nextLine();
String newgreeting = greeting.replace("I'm ","");
System.out.println("Hey, " + newgreeting + ".");
}
}

Java's String.replace method find one string and replace the one.
So, I recommend you use String.replaceAll
// ....
while (index < times){
String greeting = teclado.next();
String newgreeting = greeting.replaceAll("I'm ","");
System.out.println("Hey, "+newgreeting+".");
}
// ....

Java spell checker using hash tables

I don't want any codes. I really want to learn the logic myself but I need pointing to the right direction. Pseudocode is fine. I basically need to create a spell checker using hash tables as my primary data structure. I know it may not be the best data structure for the job but that it what i was tasked to do. The words with correct spellings will come from a text file. Please guide me on how to approach the problem.
The way I'm thinking of doing it:
I'm guessing I need to create a ADT class that takes the string words.
I need a main class that reads the dictionary text file and takes a sentence inputted by a user. This class then scans that string of words then places each word into an ArrayList by noting the spaces in between the words. A boolean method will then pass each word in the Arraylist to the class that will handle misspellings and return if the word is valid or false.
I believe I need to create a class that generates the misspellings from the word list and stores them into the hash table? There will be a boolean method that takes a string parameter that checks in the table if the word is valid and return true or false.
In generating the misspellings, the key concepts I will have to look out for will be:
(Take for example the word: "Hello")
Missing characters. E.g. "Ello", "Helo"
Jumbled version of the word. E.g. "ehllo", "helol"
Phonetic misspelling. E.g. "fello" ('f' for 'h')
How can I improve on this thinking?
EDIT! This is what I came up with using HashSet
/**
* The main program that loads the correct dictionary spellings
* and takes input to be analyzed from user.
* #author Catherine Austria
*/
public class SpellChecker {
private static String stringInput; // input to check;
private static String[] checkThis; // the stringInput turned array of words to check.
public static HashSet dictionary; // the dictionary used
/**
* Main method.
* #param args Argh!
*/
public static void main(String[] args) {
setup();
}//end of main
/**
* This method loads the dictionary and initiates the checks for errors in a scanned input.
*/
public static void setup(){
int tableSIZE=59000;
dictionary = new HashSet(tableSIZE);
try {
//System.out.print(System.getProperty("user.dir"));//just to find user's working directory;
// I combined FileReader into the BufferReader statement
//the file is located in edu.frostburg.cosc310
BufferedReader bufferedReader = new BufferedReader(new FileReader("./dictionary.txt"));
String line = null; // notes one line at a time
while((line = bufferedReader.readLine()) != null) {
dictionary.add(line);//add dictinary word in
}
prompt();
bufferedReader.close(); //close file
}
catch(FileNotFoundException ex) {
ex.printStackTrace();//print error
}
catch(IOException ex) {
ex.printStackTrace();//print error
}
}//end of setUp
/**
* Just a prompt for auto generated tests or manual input test.
*/
public static void prompt(){
System.out.println("Type a number from below: ");
System.out.println("1. Auto Generate Test\t2.Manual Input\t3.Exit");
Scanner theLine = new Scanner(System.in);
int choice = theLine.nextInt(); // for manual input
if(choice==1) autoTest();
else if(choice==2) startwInput();
else if (choice==3) System.exit(0);
else System.out.println("Invalid Input. Exiting.");
}
/**
* Manual input of sentence or words.
*/
public static void startwInput(){
//printDictionary(bufferedReader); // print dictionary
System.out.println("Spell Checker by C. Austria\nPlease enter text to check: ");
Scanner theLine = new Scanner(System.in);
stringInput = theLine.nextLine(); // for manual input
System.out.print("\nYou have entered this text: "+stringInput+"\nInitiating Check...");
/*------------------------------------------------------------------------------------------------------------*/
//final long startTime = System.currentTimeMillis(); //speed test
WordFinder grammarNazi = new WordFinder(); //instance of MisSpell
splitString(removePunctuation(stringInput));//turn String line to String[]
grammarNazi.initialCheck(checkThis);
//final long endTime = System.currentTimeMillis();
//System.out.println("Total execution time: " + (endTime - startTime) );
}//end of startwInput
/**
* Generates a testing case.
*/
public static void autoTest(){
System.out.println("Spell Checker by C. Austria\nThis sentence is being tested:\nThe dog foud my hom. And m ct hisse xdgfchv!## ");
WordFinder grammarNazi = new WordFinder(); //instance of MisSpell
splitString(removePunctuation("The dog foud my hom. And m ct hisse xdgfchv!## "));//turn String line to String[]
grammarNazi.initialCheck(checkThis);
}//end of autoTest
/**
* This method prints the entire dictionary.
* Was used in testing.
* #param bufferedReader the dictionary file
*/
public static void printDictionary(BufferedReader bufferedReader){
String line = null; // notes one line at a time
try{
while((line = bufferedReader.readLine()) != null) {
System.out.println(line);
}
}catch(FileNotFoundException ex) {
ex.printStackTrace();//print error
}
catch(IOException ex) {
ex.printStackTrace();//print error
}
}//end of printDictionary
/**
* This methods splits the passed String and puts them into a String[]
* #param sentence The sentence that needs editing.
*/
public static void splitString(String sentence){
// split the sentence in between " " aka spaces
checkThis = sentence.split(" ");
}//end of splitString
/**
* This method removes the punctuation and capitalization from a string.
* #param sentence The sentence that needs editing.
* #return the edited sentence.
*/
public static String removePunctuation(String sentence){
String newSentence; // the new sentence
//remove evil punctuation and convert the whole line to lowercase
newSentence = sentence.toLowerCase().replaceAll("[^a-zA-Z\\s]", "").replaceAll("\\s+", " ");
return newSentence;
}//end of removePunctuation
}
This class checks for misspellings
public class WordFinder extends SpellChecker{
private int wordsLength;//length of String[] to check
private List<String> wrongWords = new ArrayList<String>();//stores incorrect words
/**
* This methods checks the String[] for spelling errors.
* Hashes each index in the String[] to see if it is in the dictionary HashSet
* #param words String list of misspelled words to check
*/
public void initialCheck(String[] words){
wordsLength=words.length;
System.out.println();
for(int i=0;i<wordsLength;i++){
//System.out.println("What I'm checking: "+words[i]); //test only
if(!dictionary.contains(words[i])) wrongWords.add(words[i]);
} //end for
//manualWordLookup(); //for testing dictionary only
if (!wrongWords.isEmpty()) {
System.out.println("Mistakes have been made!");
printIncorrect();
} //end if
if (wrongWords.isEmpty()) {
System.out.println("\n\nMove along. End of Program.");
} //end if
}//end of initialCheck
/**
* This method that prints the incorrect words in a String[] being checked and generates suggestions.
*/
public void printIncorrect(){//delete this guy
System.out.print("These words [ ");
for (String wrongWord : wrongWords) {
System.out.print(wrongWord + " ");
}//end of for
System.out.println("]seems incorrect.\n");
suggest();
}//end of printIncorrect
/**
* This method gives suggestions to the user based on the wrong words she/he misspelled.
*/
public void suggest(){
MisSpell test = new MisSpell();
while(!wrongWords.isEmpty()&&test.possibilities.size()<=5){
String wordCheck=wrongWords.remove(0);
test.generateMispellings(wordCheck);
//if the possibilities size is greater than 0 then print suggestions
if(test.possibilities.size()>=0) test.print(test.possibilities);
}//end of while
}//end of suggest
/*ENTERING TEST ZONE*/
/**
* This allows a tester to look thorough the dictionary for words if they are valid; and for testing only.
*/
public void manualWordLookup(){
System.out.print("Enter 'ext' to exit.\n\n");
Scanner line = new Scanner(System.in);
String look=line.nextLine();
do{
if(dictionary.contains(look)) System.out.print(look+" is valid\n");
else System.out.print(look+" is invalid\n");
look=line.nextLine();
}while (!look.equals("ext"));
}//end of manualWordLookup
}
/**
* This is the main class responsible for generating misspellings.
* #author Catherine Austria
*/
public class MisSpell extends SpellChecker{
public List<String> possibilities = new ArrayList<String>();//stores possible suggestions
private List<String> tempHolder = new ArrayList<String>(); //telps for the transposition method
private int Ldistance=0; // the distance related to the two words
private String wrongWord;// the original wrong word.
/**
* Execute methods that make misspellings.
* #param wordCheck the word being checked.
*/
public void generateMispellings(String wordCheck){
wrongWord=wordCheck;
try{
concatFL(wordCheck);
concatLL(wordCheck);
replaceFL(wordCheck);
replaceLL(wordCheck);
deleteFL(wordCheck);
deleteLL(wordCheck);
pluralize(wordCheck);
transposition(wordCheck);
}catch(StringIndexOutOfBoundsException e){
System.out.println();
}catch(ArrayIndexOutOfBoundsException e){
System.out.println();
}
}
/**
* This method concats the word behind each of the alphabet letters and checks if it is in the dictionary.
* FL for first letter
* #param word the word being manipulated.
*/
public void concatFL(String word){
char cur; // current character
String tempWord=""; // stores temp made up word
for(int i=97;i<123;i++){
cur=(char)i;//assign ASCII from index i value
tempWord+=cur;
//if the word is in the dictionary then add it to the possibilities list
tempWord=tempWord.concat(word); //add passed String to end of tempWord
checkDict(tempWord); //check to see if in dictionary
tempWord="";//reset temp word to contain nothing
}//end of for
}//end of concatFL
/**
* This concatenates the alphabet letters behind each of the word and checks if it is in the dictionary. LL for last letter.
* #param word the word being manipulated.
*/
public void concatLL(String word){
char cur; // current character
String tempWord=""; // stores temp made up word
for(int i=123;i>97;i--){
cur=(char)i;//assign ASCII from index i value
tempWord=tempWord.concat(word); //add passed String to end of tempWord
tempWord+=cur;
//if the word is in the dictionary then add it to the possibilities list
checkDict(tempWord);
tempWord="";//reset temp word to contain nothing
}//end of for
}//end of concatLL
/**
* This method replaces the first letter (FL) of a word with alphabet letters.
* #param word the word being manipulated.
*/
public void replaceFL(String word){
char cur; // current character
String tempWord=""; // stores temp made up word
for(int i=97;i<123;i++){
cur=(char)i;//assign ASCII from index i value
tempWord=cur+word.substring(1,word.length()); //add the ascii of i ad the substring of the word from index 1 till the word's last index
checkDict(tempWord);
tempWord="";//reset temp word to contain nothing
}//end of for
}//end of replaceFL
/**
* This method replaces the last letter (LL) of a word with alphabet letters
* #param word the word being manipulated.
*/
public void replaceLL(String word){
char cur; // current character
String tempWord=""; // stores temp made up word
for(int i=97;i<123;i++){
cur=(char)i;//assign ASCII from index i value
tempWord=word.substring(0,word.length()-1)+cur; //add the ascii of i ad the substring of the word from index 1 till the word's last index
checkDict(tempWord);
tempWord="";//reset temp word to contain nothing
}//end of for
}//end of replaceLL
/**
* This deletes first letter and sees if it is in dictionary
* #param word the word being manipulated.
*/
public void deleteFL(String word){
String tempWord=word.substring(1,word.length()-1); // stores temp made up word
checkDict(tempWord);
//print(possibilities);
}//end of deleteFL
/**
* This deletes last letter and sees if it is in dictionary
* #param word the word being manipulated.
*/
public void deleteLL(String word){
String tempWord=word.substring(0,word.length()-1); // stores temp made up word
checkDict(tempWord);
//print(possibilities);
}//end of deleteLL
/**
* This method pluralizes a word input
* #param word the word being manipulated.
*/
public void pluralize(String word){
String tempWord=word+"s";
checkDict(tempWord);
}//end of pluralize
/**
* It's purpose is to check a word if it is in the dictionary.
* If it is, then add it to the possibilities list.
* #param word the word being checked.
*/
public void checkDict(String word){
if(dictionary.contains(word)){//check to see if tempWord is in dictionary
//if the tempWord IS in the dictionary, then check if it is in the possibilities list
//then if tempWord IS NOT in the list, then add tempWord to list
if(!possibilities.contains(word)) possibilities.add(word);
}
}//end of checkDict
/**
* This method transposes letters of a word into different places.
* Not the best implementation. This guy was my last minute addition.
* #param word the word being manipulated.
*/
public void transposition(String word){
wrongWord=word;
int wordLen=word.length();
String[] mixer = new String[wordLen]; //String[] length of the passed word
//make word into String[]
for(int i=0;i<wordLen;i++){
mixer [i]=word.substring(i,i+1);
}
shift(mixer);
}//end of transposition
/**
* This method takes a string[] list then shifts the value in between
* the elements in the list and checks if in dictionary, adds if so.
* I agree that this is probably the brute force implementation.
* #param mixer the String array being shifted around.
*/
public void shift(String[] mixer){
System.out.println();
String wordValue="";
for(int i=0;i<=tempHolder.size();i++){
resetHelper(tempHolder);//reset the helper
transposeHelper(mixer);//fill tempHolder
String wordFirstValue=tempHolder.remove(i);//remove value at index in tempHolder
for(int j=0;j<tempHolder.size();j++){
int inttemp=0;
String temp;
while(inttemp<j){
temp=tempHolder.remove(inttemp);
tempHolder.add(temp);
wordValue+=wordFirstValue+printWord(tempHolder);
inttemp++;
if(dictionary.contains(wordValue)) if(!possibilities.contains(wordValue)) possibilities.add(wordValue);
wordValue="";
}//end of while
}//end of for
}//end for
}//end of shift
/**
* This method fills a list tempHolder with contents from String[]
* #param wordMix the String array being shifted around.
*/
public void transposeHelper(String[] wordMix){
for(int i=0;i<wordMix.length;i++){
tempHolder.add(wordMix[i]);
}
}//end of transposeHelper
/**
* This resets a list
* #param thisList removes the content of a list
*/
public void resetHelper(List<String> thisList){
while(!thisList.isEmpty()) thisList.remove(0); //while list is not empty, remove first value
}//end of resetHelper
/**
* This method prints out a list
* #param listPrint the list to print out.
*/
public void print(List<String> listPrint){
if (possibilities.isEmpty()) {
System.out.print("Can't seem to find any related words for "+wrongWord);
return;
}
System.out.println("Maybe you meant these for "+wrongWord+": ");
System.out.printf("%s", listPrint);
resetHelper(possibilities);
}//end of print
/**
* This returns a String word version of a list
* #param listPrint the list to make into a word.
* #return the generated word version of a list.
*/
public String printWord(List<String> listPrint){
Object[] suggests = listPrint.toArray();
String theWord="";
for(Object word: suggests){//form listPrint elements into a word
theWord+=word;
}
return theWord;
}//end of printWord
}

Instead of generating all possible misspellings of the words in your dictionary and adding them to the hash table, consider performing all possible changes (that you already suggested) to the user-entered words, and checking to see if those words are in the dictionary file.

It sounds like what you want is a quick way to verify that a word is spelled correctly, or to find the correct spelling. If this is what your trying to do you can use a HashMap<String,String> (i.e. a hash table with String keys and String values). Whenever you find a word in your dictionary you enter a key for it with a null value indicating that the word is not to be changed (i.e. a correct spelling). You can then compute and add keys for possible misspellings and give the correct word for the value.
You'd have to devise a way to do this very carefully, because if your dictionary has two similar words "clot" and "colt" computed misspellings of one may replace the correct spelling (or misspellings) of the other. Once your done you can look up a word to see if it is in the dictionary, if it is a misspelling of a dictionary word (and which word), or if it is not found at all.
I believe this is a bad design though, because your table has to be exponentially larger than your (I assume, already quite large) dictionary. And because you spent a lot of time calculating many misspelling for every word in the dictionary (very big overhead if you only check a few lines which may contain a few of these words). Given a only a little liberty I would opt for a HashSet<String> (which is a hash table but without values) filled only with dictionary words. This is allows you to check quickly if a word is in the dictionary or not.
You can dynamically compute other ways to spell words when you encounter ones not in your dictionary. If your doing this for only a line or two it should not be slow at all (certainly faster than computing alternatives for everything in your dictionary). But if you wanted to this for every for a whole file you may want to keep a much smaller HashMap<String,String> separate from your dictionary to store any corrections you find since the author may misspell the word the same way in future. Checking this before computing alternatives keeps you from duplicating your efforts several times over.

How do I read in a text file as a string and extract just a single character? "0"

How do I go about asking for a string input, then extracting the 0 character?
I was using "fstream.next()" and it would input whatever, not just a string.
I just need a character, which I can later use in a loop where the program only accepts char inputs of T, D, and E.
Then the program reads the double. Then the program calls the instance method with the parameters (char, double). The instance method later does it's thing and saves the inputs. The program later loops back and does it all over again.
Currently I am receiving an error "java.util.InputMismatchException". If you could offer suggestions, I'd greatly appreciate it! If clarification is needed, please let me know. Thank you in advance! :)
Here is my code:
/** holds answer of whether user has more inputs or not */
String answer;
/** holds the results when calling inLetter() */
char letter;
/** holds the results when checking for types/strings in txt file */
String inType = "";
/** holds double results when searching line by line the text file */
double inAmount = 0;
/** holds the results when calling inAmount() */
double amount;
/** initiates infile to null */
File infile = null;
/** count system for how many valid lines were read and used */
int count = 0;
/** calls description method */
description();
/** initiates new Object */
GironEventClass newInput = new GironEventClass();
try{
/** defines new variable linked to .dat file */
infile = new File("GironEvent.dat");
/** calls fstream scanner - for use on file */
Scanner fstream = new Scanner(infile);
/** calls while loop. As long as there is a line, the loop continues */
while(fstream.hasNext())
{
/** inputs first string in line of file to variable inType */
inType = fstream.nextLine();
char a_char = inType.charAt(0);
/** inputs first int in line of file to variable inAmount */
inAmount = fstream.nextDouble();
try{
/** calls instance method with two parameters */
newInput.donations(a_char, inAmount);
/** count ticket increases */
count+=1;
}
catch(IllegalArgumentException a){
/** prints out error if exception is caught*/
System.out.println("Just caught an illegal argument exception. ");
}
}
/** closes stream between program and fiel */
fstream.close();
/** outputs line count, and totals per type */
System.out.println("\n" + count + " number of valid lines were read!\n");
System.out.println("Total Sales: " + newInput.getSale());
System.out.println("Donations: " + newInput.getDonated());
System.out.println("Expenses: " + newInput.getExpenses());
}
catch (FileNotFoundException e){
/** Outputs error if file cannot be opened. */
System.out.println("\nGironEvent.dat could not be opened. ");
}
do{
/** loop asks user if there is more that needs to be added to the totals. N exits loop. */
System.out.print("\nAre there any more items to add that were not in the text file? (Type 'Y' or 'N'): ");
answer = keyboard.next();
if (("Y".equals(answer)) || ("y".equals(answer)))
{
letter = inLetter();
amount = inAmount();
newInput.donations(letter, amount);
}
}while (("Y".equals(answer)) || ("y".equals(answer)));
/** calls instance method to display totals in a fancy manner */
newInput.display();
}
/** inLetter - displays types to user. Asks user to input type. Converts input into uppercase letter.
*
* #return resultTwo - Uppercase form of result. Result is the type the user inputted.
*/
public static char inLetter(){
Scanner keyboard = new Scanner(System.in);
String result;
String resultTwo;
System.out.println("T = Tiket Sales");
System.out.println("D = Donations");
System.out.println("E = Expenses");
System.out.print("Please input an identifier ");
result = keyboard.nextLine();
resultTwo = result.toUpperCase();
return resultTwo;
}
/** inAmount - asks user for amount. Tests that amount isn't a negative number or a zero.
*
* #return result - the amount the user inputed. regardless if it was the first or tenth time.
*/
public static double inAmount(){
Scanner keyboard = new Scanner(System.in);
double result;
System.out.println("Please input an amount ");
result = keyboard.nextDouble();
if(result <= 0){
System.out.print("Please input a positive and non-zero amount ");
result = keyboard.nextDouble();
}
return result;
}
/** description - displays a description of what the program does
* void.
*/
public static void description(){
System.out.println("The program will ask you what amount is being spent on what.");
System.out.println(" ex: expenses, ticket sales, and profts.");
System.out.println("This program will help determine whether the event generated or lost money.");
}
I need assistance with the following block:
/** inputs first string in line of file to variable inType */
inType = fstream.nextLine();
char a_char = inType.charAt(0);
/** inputs first int in line of file to variable inAmount */
inAmount = fstream.nextDouble();

Your fstream is having a combination of String and Double.
So when you use fstream.nextDouble() it throws java.util.InputMismatchException
Reference: http://docs.oracle.com/javase/7/docs/api/java/util/Scanner.html#nextDouble()
You can first check whether the next character is double or not with the method hasNextDouble()
Reference:
http://docs.oracle.com/javase/7/docs/api/java/util/Scanner.html#hasNextDouble()