Develop Reference

How to parse "Sección" to "Sección"? (string accutes encoding issue)

I have a string with this value "Sección"
I need to parse it to UTF-8, so the string gets transformed to "Sección"
I tried with line = new String(line.getBytes("UTF-8"), "UTF-8"); but this does not work.
Edit
I'm reading the string with this method:
public static String loadLine(InputStream is) {
if (is == null)
return null;
final short TAM_LINE = 256;
String line;
char[] buffer = new char[TAM_LINE];
short i;
int ch;
try {
line = "";
i = 0;
do {
ch = is.read();
if ((ch != '\n') && (ch != -1)) {
buffer[i++] = (char)(ch & 0xFF);
if (i >= TAM_LINE) {
line += new String(buffer, 0, i);
i = 0;
}
}
} while ((ch != '\n') && (ch != -1));
// Si no hemos llegado a leer ning�n caracter, devolvemos null
if (ch == -1 && i == 0)
return null;
// A�adimos el �ltimo trozo de l�nea le�do
line += new String(buffer, 0, i);
} catch (IOException e) {
e.printStackTrace();
return null;
}
return line;
}

The character ó is encoded as 0xc3 0xb3 in UTF-8. It appears that whichever program read that UTF-8-encoded string in the first place read it assuming the wrong encoding, for example windows-1252, where 0xc3 encodes à and 0xb3 encodes ³.
In your case, your edit shows that (as far as I can tell, I don't know Java), you're reading the input byte by byte, building the string one character at a time, one from each byte. This is not a good idea if the encoding UTF-8 uses multiple bytes to encode certain characters such as ó.
You should read the input into a bytes array first, then build a String using the correct encoding:
line = new String(byteArray, "UTF-8")

Analysing text that is read from a file in Java

I am writing a program that analyses text that is input and calculates the average letter count.
I have it working for when text is entered by the user, but I cannot get it to work for when the text is read from a file.
public static void ReadFromFile() throws IOException {
String filename = "name.txt";
String str = new String(new char[100]);
try{
FileReader fr = new FileReader(filename);
BufferedReader br = new BufferedReader(fr);
while ((str = br.readLine()) !=null) {
System.out.println(str + "\n");
}
br.close();
} catch(IOException e) {
System.out.println("File not found");
}
while (c != str.length()) {
if (str.charAt(c) >= 'a' && str.charAt(c) <= 'z') {
count[str.charAt(c) - 'a']++;
}
c++;
}
for (c = 0; c<26;c++){
if (count[c] !=0){
double num = count[c];
double denom = str.length();
double average = num / denom;
System.out.print((char) (c + 'a') + " occurs " + count[c] + " times in the entered string. Average is ");
System.out.println(average);
}}}}
I keep getting an error that says
Exception in thread "main" java.lang.NullPointerException
at MainMenu.ReadFromFile(MainMenu.java:79)
at MainMenu.main(MainMenu.java:25)
Any help would be greatly appreciated
Thanks

You are reading the entire file in, line by line and printing it to System.out
After you have finished that, you then try to process the data, but you've already finished reading in the file, hence str is 'null' and your program dies.

Earlier in your code, you have this loop:
while ((str = br.readLine()) != null) {
System.out.println(str + "\n");
}
This will read lines from your file while str is not null.
That means that when it stops, str is null.
Since you don't give str a non-null value after that, you get an NPE when you try to run methods on str later in the code:
while (c != str.length()) { // str is null here!
if (str.charAt(c) >= 'a' && str.charAt(c) <= 'z') {
count[str.charAt(c) - 'a']++;
}
c++;
}
My guess from looking at your code is that you want to collect information about each line in the file. To do this, instead of just printing each line, move your counting logic inside the loop which is reading the file:
// read each line from the file
while ((str = br.readLine()) != null) {
System.out.println(str + "\n");
// collect character data for this line
while (c != str.length()) {
if (str.charAt(c) >= 'a' && str.charAt(c) <= 'z') {
count[str.charAt(c) - 'a']++;
}
c++;
}
}
Then process the count array later in the code as you are already doing.

Why is my bufferedReader reading `java.lang.String;#70177ecd` from a text file? [duplicate]

This question already has answers here:
printing arrays
(6 answers)
Closed 8 years ago.
I have a bufferedReader which is being used to read lines of text from a text file, however, the only line of text that the bufferedReader is reading is [Ljava.lang.String;#70177ecd my code is below:
private static final String dirToWriteFile = System.getProperty("user.dir") + "/VirtualATM.txt"; //File path for main user detail store.
FileInputStream fin = new FileInputStream(dirToWriteFile);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(fin));
String lineFromFile = null;
lineFromFile = bufferedReader.readLine();
String [] lineFromFileToCheck = lineFromFile.split("\\s+");
do{
for(i = 0; i < lineFromFileToCheck.length; i++){
System.out.println(lineFromFileToCheck);
/************* Error because "[Ljava.lang.String;#70177ecd" is being read from file ******************/
if(lineFromFileToCheck[i].matches(cardNumberStr)){
accountNoFound = true;
}
if(accountNoFound == false & i > 10){
break;
}
}
if(accountNoFound == true){
for(j = 0; j < lineFromFileToCheck.length; j++){
if(lineFromFileToCheck[j].matches(pinToCheck)){
pinNoFound = true; //if pin matches exactly what is inputted by user
break;
}
}
lineFromFile = bufferedReader.readLine();
if (lineFromFile != null) {
lineFromFileToCheck = lineFromFile.split("\\s+");
}
}
if(pinNoFound == false & lineFromFile == null){
JOptionPane.showMessageDialog(null, "Error! Invalid input length/type. Try again...", "Error", JOptionPane.ERROR_MESSAGE);
}
if(pinNoFound == true & accountNoFound == true & lineFromFile == null){
break;
}
}while(accountNoFound == false | pinNoFound == false | lineFromFile != null);
fin.close();
bufferedReader.close();
If anyone could suggest what is wrong with the reader and what exactly [Ljava.lang.String;#70177ecd means and what causes it be returned by the BufferedReader that would be great!

lineFromFileToCheck is a String array.you need to call lineFromFileToCheck[i] to print a String in it
change this
for(i = 0; i < lineFromFileToCheck.length; i++){
System.out.println(lineFromFileToCheck);
to this
for(i = 0; i < lineFromFileToCheck.length; i++){
System.out.println(lineFromFileToCheck[i]);

It's not error nor does buffered reader read it wrong. You are trying to print array which doesn't have toString implemented. If you need clarity on the same use:
System.out.println(Arrays.toString(lineFromFileToCheck));//your splitted line that you read from file
or Just print line and not the array as below:
System.out.println(lineFromFile);

That most probably means that it's receiving an array and using deepToString method.
From the JavaDoc:
Returns a string representation of the "deep contents" of the specified array.
So all you have to do is read from the specific element in your for-loop. Just do -
System.out.println(lineFromFileToCheck);

Java: reading utf-8 file page by page using FileInputStream

I need some code that will allow me to read one page at a time from a UTF-8 file.
I've used the code;
File fileDir = new File("DIRECTORY OF FILE");
BufferedReader in = new BufferedReader(new InputStreamReader(new FileInputStream(fileDir), "UTF8"));
String str;
while ((str = in.readLine()) != null) {
System.out.println(str);
}
in.close();
}
After surrounding it with a try catch block it runs but outputs the entire file!
Is there a way to amend this code to just display ONE PAGE of text at a time?
The file is in UTF-8 format and after viewing it in notepad++, i can see the file contains FF characters to denote the next page.

You will need to look for the form feed character by comparing to 0x0C.
For example:
char c = in.read();
while ( c != -1 ) {
if ( c == 0x0C ) {
// form feed
} else {
// handle displayable character
}
c = in.read();
}
EDIT added an example of using a Scanner, as suggested by Boris
Scanner s = new Scanner(new File("a.txt")).useDelimiter("\u000C");
while ( s.hasNext() ) {
String str = s.next();
System.out.println( str );
}

If the file is valid UTF-8, that is, the pages are split by U+00FF, aka (char) 0xFF, aka "\u00FF", 'ÿ', then a buffered reader can do. If it is a byte 0xFF there would be a problem, as UTF-8 may use a byte 0xFF.
int soughtPageno = ...; // Counted from 0
int currentPageno = 0;
try (BufferedReader in = new BufferedReader(new InputStreamReader(
new FileInputStream(fileDir), StandardCharsets.UTF_8))) {
String str;
while ((str = in.readLine()) != null && currentPageno <= soughtPageno) {
for (int pos = str.indexOf('\u00FF'; pos >= 0; )) {
if (currentPageno == soughtPageno) {
System.out.println(str.substring(0, pos);
++currentPageno;
break;
}
++currentPageno;
str = str.substring(pos + 1);
}
if (currentPageno == soughtPageno) {
System.out.println(str);
}
}
}
For a byte 0xFF (wrong, hacked UTF-8) use a wrapping InputStream between FileInputStream and the reader:
class PageInputStream implements InputStream {
InputStream in;
int pageno = 0;
boolean eof = false;
PageInputSTream(InputStream in, int pageno) {
this.in = in;
this.pageno = pageno;
}
int read() throws IOException {
if (eof) {
return -1;
}
while (pageno > 0) {
int c = in.read();
if (c == 0xFF) {
--pageno;
} else if (c == -1) {
eof = true;
in.close();
return -1;
}
}
int c = in.read();
if (c == 0xFF) {
c = -1;
eof = true;
in.close();
}
return c;
}
Take this as an example, a bit more work is to be done.

You can use a Regex to detect form-feed (page break) characters. Try something like this:
File fileDir = new File("DIRECTORY OF FILE");
BufferedReader in = new BufferedReader(new InputStreamReader(new FileInputStream(fileDir), "UTF8"));
String str;
Regex pageBreak = new Regex("(^.*)(\f)(.*$)")
while ((str = in.readLine()) != null) {
Match match = pageBreak.Match(str);
bool pageBreakFound = match.Success;
if(pageBreakFound){
String textBeforeLineBreak = match.Groups[1].Value;
//Group[2] will contain the form feed character
//Group[3] will contain the text after the form feed character
//Do whatever logic you want now that you know you hit a page boundary
}
System.out.println(str);
}
in.close();
The parenthesis around portions of the Regex denote capture groups, which get recorded in the Match object. The \f matches on the form feed character.
Edited Apologies, for some reason I read C# instead of Java, but the core concept is the same. Here's the Regex documentation for Java: http://docs.oracle.com/javase/tutorial/essential/regex/

String.split() will keep original char array inside

I've noticed that Java String will reuse char array inside it to avoid creating new char array for a new String instance in method such as subString(). There are several unpublish constructors in String for this purpose, accepting a char array and two int as range to construct a String instance.
But until today I found that split will also reuse the char arr of original String instance. Now I read a loooooong line from a file, split it with "," and cut a very limit column for real usage. Because every part of the line secretly holding the reference of the looooong char array, I got an OOO very soon.
here is example code:
ArrayList<String> test = new ArrayList<String>(3000000);
BufferedReader origReader = new BufferedReader(new FileReader(new File(
"G:\\filewithlongline.txt")));
String line = origReader.readLine();
int i = 0;
while ((line = origReader.readLine()) != null) {
String name = line.split(',')[0];
test.add(name);
i++;
if (i % 100000 == 0) {
System.out.println(name);
}
}
System.out.println(test.size());
Is there any standard method in JDK to make sure that every String instance that spitted is a "real deep copy" not "shallow copy"?
Now I am using a very ugly workaround to force creating a new String instance:
ArrayList<String> test = new ArrayList<String>(3000000);
BufferedReader origReader = new BufferedReader(new FileReader(new File(
"G:\\filewithlongline.txt")));
String line = origReader.readLine();
int i = 0;
while ((line = origReader.readLine()) != null) {
String name = line.split(',')[0]+" ".trim(); // force creating a String instance
test.add(name);
i++;
if (i % 100000 == 0) {
System.out.println(name);
}
}
System.out.println(test.size());

The simplest approach is to create a new String directly. This is one of the rare cases where its a good idea.
String name = new String(line.split(",")[0]); // note the use of ","
An alternative is to parse the file yourself.
do {
StringBuilder name = new StringBuilder();
int ch;
while((ch = origReader.read()) >= 0 && ch != ',' && ch >= ' ') {
name.append((char) ch);
}
test.add(name.toString());
} while(origReader.readLine() != null);

String has a copy constructor you can use for this purpose.
final String name = new String(line.substring(0, line.indexOf(',')));
... or, as Peter suggested, just only read until the ,.
final StringBuilder buf = new StringBuilder();
do {
int ch;
while ((ch = origReader.read()) >= 0 && ch != ',') {
buf.append((char) ch);
}
test.add(buf.toString());
buf.setLength(0);
} while (origReader.readLine() != null);