Develop Reference

Can't read from binary file - read some lines in UTF-8, some in binary

I have this code:
import java.io.*;
import java.nio.charset.StandardCharsets;
public class Main {
public static void main(String[] args) {
zero("zero.out");
System.out.println(zeroRead("zero.out"));
}
public static String zeroRead(String name) {
try (FileInputStream fos = new FileInputStream(name);
BufferedInputStream bos = new BufferedInputStream(fos);
DataInputStream dos = new DataInputStream(bos)) {
StringBuffer inputLine = new StringBuffer();
String tmp;
String s = "";
while ((tmp = dos.readLine()) != null) {
inputLine.append(tmp);
System.out.println(tmp);
}
dos.close();
return s;
}
catch (IOException e) {
e.printStackTrace();
}
return null;
}
public static void zero(String name) {
File file = new File(name);
String text = "König" + "\t";
try (FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(fos);
DataOutputStream dos = new DataOutputStream(bos)) {
dos.write(text.getBytes(StandardCharsets.UTF_8));
dos.writeInt(50);
} catch (IOException e) {
e.printStackTrace();
}
}
}
zero() method writes data into file: the string is written in UTF-8, while the number is written in binary. zeroRead() read the data from file.
The file looks like this after zero() is executed:
This is what zeroRead() returns:
How do I read the real data König\t50 from the file?

DataInputStream's readLine method has javadoc that is almost yelling that it doesn't want to be used. You should heed this javadoc: That method is bad and you should not use it. It doesn't do charset encoding.
Your file format is impossible as stated: You have no idea when to stop reading the string and start reading the binary numbers. However, the way you've described things, it sounds like the string is terminated by a newline, so, the \n character.
There is no easy 'just make this filter-reader and call .nextLine on it available, as they tend to buffer. You can try this:
InputStreamReader isr = new InputStreamReader(bos, StandardCharsets.UTF_8);
However, basic readers do not have a readLine method, and if you wrap this in a BufferedReader, it may read past the end (the 'buffer' in that name is not just there for kicks). You'd have to handroll a method that fetches one character at a time, appending them to a stringbuilder, ending on a newline:
StringBuilder out = new StringBuilder();
for (int c = isr.read(); c != -1 && c != '\n'; c = isr.read())
out.append((char) c);
String line = out.toString();
will get the job done and won't read 'past' the newline and gobble up your binary number.

Decompress large binary files

I have a function to decompress large zip files using the below method. They are times where I run into OutOfMemoryError error because the file is just too large. Is there a way I can optimize my code? I have read something about breaking the file into smaller parts that can fit into memory and decompress but I don't know how to do that. Any help or suggestion is appreciated.
private static String decompress(String s){
String pathOfFile = null;
try(BufferedReader reader = new BufferedReader(new InputStreamReader(new GZIPInputStream(new FileInputStream(s)), Charset.defaultCharset()))){
File file = new File(s);
FileOutputStream fos = new FileOutputStream(file);
String line;
while((line = reader.readLine()) != null){
fos.write(line.getBytes());
fos.flush();
}
pathOfFile = file.getAbsolutePath();
} catch (IOException e) {
e.printStackTrace();
}
return pathOfFile;
}
The stacktrace:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.Arrays.copyOf(Arrays.java:3689)
at java.base/java.util.ArrayList.grow(ArrayList.java:237)
at java.base/java.util.ArrayList.ensureCapacity(ArrayList.java:217)

Don't use Reader classes because you don't need to write output file character by character or line by line. You should read and write byte by byte with InputStream.transferTo() method:
try(var in = new GZIPInputStream(new FileInputStream(inFile));
var out = new FileOutputStream(outFile)) {
in.transferTo(out);
}
Also you probably don't need to call flush() explicitly, doing it after every line is wasteful.

Java fileinput (read and write) to a file looping thru each line

I have this method that access a exisitng file, loop thru each line and replace (string to string) a certain line if the condition is met:
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.InputStreamReader;
private void UpdateConfig() {
try {
FileInputStream fstream = new FileInputStream("c:\\user\\config.properties");
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine;
while ((strLine = br.readLine()) != null) {
if (strLine.contains("FTPDate=2014/07/01 00:59:00")) {
System.out.println("FILE " + strLine);
strLine.replace("FTPDate=2014/07/01 00:59:00", "FTPDate=2014/09/10 00:00:00");
//strLine.replace("((19|20)\\d\\d/(0?[1-9]|1[012])/(0?[1-9]|[12][0-9]|3[01])) ([2][0-3]|[0-1][0-9]|[1-9]):[0-5][0-9]:([0-5][0-9]|[6][0])", "2014/09/10 00:00:00");
System.out.println("FILE " + strLine);
}
}
in.close();
} catch (Exception e) {
}
}
In the sysout it seems its being replaced:
FILE FTPDateTejas=2014/07/01 00:59:00
FILE FTPDateTejas=2014/09/10 00:00:00
But when I check the file, the date still stays the same. Am I missing something? anyone knows what I missed out? thank you

When you are doing:
strLine = br.readLine() it loads the next line from the BufferedReader into memory. This means that you have your data on disk and in memory and that those two are not linked to each other in any way. When doing modifications on strLine I believe you have in your code:
strLine = strLine.replace("FTPDate=2014/07/01 00:59:00", "FTPDate=2014/09/10 00:00:00");
As replace doesn't modify the contents of the objects on which it is being called but returns a new String objects (Strings are immutable). So what that does it creates a new object but does not modify your on disk data (as I said, it's not linked to it any more!).
You could think "ok then how do I link those two and override the file in place?". Well Java does provide random file access as described in the doc but the only thing you can do with it is modify characters at a certain position, you cannot insert things in the middle. So what you would have to do is read the rest of your file, make your modification and then append that rest of the file, yes you need to shift things in case your new string with which you are substituting would be shorter/longer than what you are replacing.
That's why an easier solution would be to:
open a new file to write to
write line by line to it (the strings after the replace)
delete the old file and rename the new file
Without copying the file the code would look something like this:
private void UpdateConfig() {
File fstream = new File("c:\\user\\config.properties");
File file = new File("c:\\user\\config.properties-new");
try {
file.createNewFile();
} catch (IOException e) {
// handle
}
try (FileReader in = new FileReader(fstream);
FileWriter fw = new FileWriter(file.getAbsoluteFile())) {
try (BufferedReader br = new BufferedReader(in);
BufferedWriter bw = new BufferedWriter(fw)) {
String strLine;
while ((strLine = br.readLine()) != null) {
if (strLine.contains("FTPDate=2014/07/01 00:59:00")) {
System.out.println("FILE " + strLine);
strLine = strLine.replace("FTPDate=2014/07/01 00:59:00",
"FTPDate=2014/09/10 00:00:00");
//strLine.replace("((19|20)\\d\\d/(0?[1-9]|1[012])/(0?[1-9]|[12][0-9]|3[01])) ([2][0-3]|[0-1][0-9]|[1-9]):[0-5][0-9]:([0-5][0-9]|[6][0])", "2014/09/10 00:00:00");
bw.write(strLine);
System.out.println("FILE " + strLine);
}
}
}
// copy files here
} catch (IOException e) {
// handle
}
}
There might be some logical/syntactic problems as I was writing in in a plain text editor. I modified the code a bit to use Java 7's try-with-resources, which is a cleaner way of closing resources than what you were doing - in your code when an exception would be thrown the stream might not had been closed.

Main.class.getResource() and jTextArea

When I read a file from the jar file and want to put it in in a jTextArea, it shows me crypted symbols, not the true content.
What I am doing:
public File loadReadme() {
URL url = Main.class.getResource("/readme.txt");
File file = null;
try {
JarURLConnection connection = (JarURLConnection) url
.openConnection();
file = new File(connection.getJarFileURL().toURI());
if (file.exists()) {
this.readme = file;
System.out.println("all ok!");
}
} catch (Exception e) {
System.out.println("not ok");
}
return file;
}
And then i read the file:
public ArrayList<String> readFileToArray(File file) {
ArrayList<String> array = new ArrayList<String>();
BufferedReader br = null;
try {
String sCurrentLine;
br = new BufferedReader(new FileReader(file));
while ((sCurrentLine = br.readLine()) != null) {
String test = sCurrentLine;
array.add(test);
}
} catch (IOException e) {
System.out.println("not diese!");
} finally {
try {
if (br != null)
br.close();
} catch (IOException ex) {
}
}
return array;
}
Now, i put all lines from the ArrayList in the jTextArea, that showes me things like that:
PK����?����^��S?��3��� z_��
%�Q Tl?7��+�;�
�fK� �N��:k�����]�Xk,������U"�����q��\����%�Q#4x�|[���o� S{��:�aG�*s g�'.}���n�X����5��q���hpu�H���W�9���h2��Q����#���#7(�#����F!��~��?����j�?\xA�/�Rr.�v�l�PK�bv�=
The textfiled contains:
SELECTION:
----------
By clicking the CTRL Key and the left mouse button you go in the selection mode.
Now, by moving the mouse, you paint a rectangle on the map.
DOWNLOAD:
---------
By clicking on the download button, you start the download.
The default location for the tiles to download is: <your home>
I am sure that the file exists!
Does anyone know what the problem is? Is my "getResource" correct?

Based on the output, I'm suspecting your code actually reads the JAR file itself (since it starts with PK). Why not use the following code to read the text file:
Main.class.getResourceAsStream("/readme.txt")
That would give you an InputStream to the text file without doing the hassle of opening the JAR file, etc.
You can then pass the InputStream object to the readFileToArray method (instead of the File object) and use
br = new BufferedReader(new InputStreamReader(inputStream));
The rest of your code should not need any change.

This seems to be an encoding problem. FileReader doesn't allow you to specify that. Try using
br = new BufferedReader(new InputStreamReader(new FileInputStream(file), yourEncoding));

You seem to be making far too much work for yourself here. You start by calling getResource, which gives you a URL to the readme.txt entry inside your JAR file, but then you take that URL, determine the JAR file that it is pointing inside, then open that JAR file with a FileInputStream and read the whole JAR file.
You can instead simply call .openStream() on the original URL that getResource returned, and this will give you an InputStream from which you can read the content of readme.txt
br = new BufferedReader(new InputStreamReader(url.openStream(), "UTF-8"));
(if readme.txt is not encoded in UTF-8 then change that parameter as appropriate)

Android FileReader from R.raw.file

Really newbie question:
I have a .csv file that I need to read. I've put it in the raw folder. For convenience, Im' using the http://opencsv.sourceforge.net/ library for reading the file. The library provides this method for creating a CSVReader object:
CSVReader reader = new CSVReader(new FileReader("yourfile.csv"));
But I don0't get how to point this constructor to my file, since the file in Android is usually referenced like R.raw.file rather than a String address to the file.
Any help would be greatly appreciated.

You want to do something like this -
public void readCSVFromRawResource(Context context)
{
//this requires there to be a dictionary.csv file in the raw directory
//in this case you can swap in whatever you want
InputStream inputStream = getResources().openRawResource(R.raw.dictionary);
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
try
{
String word;//word
int primaryKey = 0;//primary key
Map dictionaryHash = new HashMap();
while ((word = reader.readLine()) != null)
{
if(word.length() < 7)
{
dictionaryHash.put(primaryKey,word );
primaryKey++;
if(primaryKey % 1000 == 0)
Log.v("Percent load completed ", " " + primaryKey);
}
}
//write the dictionary to a file
File file = new File(DICTIONARY_FILE_NAME);
BufferedOutputStream fos = new BufferedOutputStream(new FileOutputStream(DICTIONARY_FILE_NAME));
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(dictionaryHash);
oos.flush();
oos.close();
Log.v("alldone","done");
}
catch (Exception ex) {
// handle exception
Log.v(ex.getMessage(), "message");
}
finally
{
try
{
inputStream.close();
}
catch (IOException e) {
// handle exception
Log.v(e.getMessage(), "message");
}
}
}

You can use this solution to get a String from the raw resource:
Android read text raw resource file
then use StringReader instead of FileReader with the CSVReader constructor.