Ok so I'm learning to write and read binary file in java and this is the method I get suggested everywhere I google
Here's the weighting class
public Writer(String fileName, String text) throws IOException {
ObjectOutputStream output = null;
try{
output = new ObjectOutputStream(new FileOutputStream(fileName, true));
} catch (FileNotFoundException e) {
System.out.println("File not found!");
System.exit(0);
} catch (IOException e) {
System.out.println("IO Exception!!");
System.exit(0);
}//THE TEXT HERE IS "test"
output.writeChars(text);
output.close();
System.out.println("Successful writing!");
}
Here's the reading Class
public Reader(String fileName) throws IOException {
ObjectInputStream in = null;
try {
in = new ObjectInputStream(new FileInputStream(fileName));
} catch (FileNotFoundException e) {
System.out.println("File Not found!");
System.exit(0);
} catch (IOException e) {
System.out.println("IO Exception!!");
System.exit(0);
}
int i;
while ((i = in.read()) != -1){
System.out.print((char) i);
}
in.close();
}
but then my output is t e s t "There are squares in between each char"
For binary, non-text, files DataInputStream/DataOutputStream are more clear.
try (FileOutputStream fos = new FileOutputStream("test.bin");
DataOutputStream dos = new DataOutputStream(fos)) {
dos.writeUTF8("La projekto celas ŝtopi breĉojn en Vikipedio");
dos.writeInt(42);
dos.writeDouble(Math.PI);
}
try (FileInputStream fis = new FileInputStream("test.bin");
DataInputStream dis = new DataInputStream(fis)) {
String s = dis.readUTF8(); // "La projekto celas ŝtopi breĉojn en Vikipedio"
int n = dis.readInt(); // 42
double pi = dis.readDouble() // Math.PI
}
writeUTF8 writes a length, and the an UTF-8 encoded string. A Unicode format, so any script may be written. One may mix Japanese, Greek, emoticons and Bulgarian.
Related
Question at the bottom
I'm using netty to transfer a file to another server.
I limit my file-chunks to 1024*64 bytes (64KB) because of the WebSocket protocol. The following method is a local example what will happen to the file:
public static void rechunck(File file1, File file2) {
FileInputStream is = null;
FileOutputStream os = null;
try {
byte[] buf = new byte[1024*64];
is = new FileInputStream(file1);
os = new FileOutputStream(file2);
while(is.read(buf) > 0) {
os.write(buf);
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
if(is != null && os != null) {
is.close();
os.close();
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
}
The file is loaded by the InputStream into a ByteBuffer and directly written to the OutputStream.
The content of the file cannot change while this process.
To get the md5-hashes of the file I've wrote the following method:
public static String checksum(File file) {
InputStream is = null;
try {
is = new FileInputStream(file);
MessageDigest digest = MessageDigest.getInstance("MD5");
byte[] buffer = new byte[8192];
int read = 0;
while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
return new BigInteger(1, digest.digest()).toString(16);
} catch(IOException | NoSuchAlgorithmException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
is.close();
} catch(IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
return null;
}
So: just in theory it should return the same hash, shouldn't it? The problem is that it returns two different hashes that do not differ with every run.. file size stays the same and the content either.
When I run the method once for in: file-1, out: file-2 and again with in: file-2 and out: file-3 the hashes of file-2 and file-3 are the same! This means the method will properly change the file every time the same way.
1. 58a4a9fbe349a9e0af172f9cf3e6050a
2. 7b3f343fa1b8c4e1160add4c48322373
3. 7b3f343fa1b8c4e1160add4c48322373
Here is a little test that compares all buffers if they are equivalent. Test is positive. So there aren't any differences.
File file1 = new File("controller/templates/Example.zip");
File file2 = new File("controller/templates2/Example.zip");
try {
byte[] buf1 = new byte[1024*64];
byte[] buf2 = new byte[1024*64];
FileInputStream is1 = new FileInputStream(file1);
FileInputStream is2 = new FileInputStream(file2);
boolean run = true;
while(run) {
int read1 = is1.read(buf1), read2 = is2.read(buf2);
String result1 = Arrays.toString(buf1), result2 = Arrays.toString(buf2);
boolean test = result1.equals(result2);
System.out.println("1: " + result1);
System.out.println("2: " + result2);
System.out.println("--- TEST RESULT: " + test + " ----------------------------------------------------");
if(!(read1 > 0 && read2 > 0) || !test) run = false;
}
} catch (IOException e) {
e.printStackTrace();
}
Question: Can you help me chunking the file without changing the hash?
while(is.read(buf) > 0) {
os.write(buf);
}
The read() method with the array argument will return the number of files read from the stream. When the file doesn't end exactly as a multiple of the byte array length, this return value will be smaller than the byte array length because you reached the file end.
However your os.write(buf); call will write the whole byte array to the stream, including the remaining bytes after the file end. This means the written file gets bigger in the end, therefore the hash changed.
Interestingly you didn't make the mistake when you updated the message digest:
while((read = is.read(buffer)) > 0) {
digest.update(buffer, 0, read);
}
You just have to do the same when you "rechunk" your files.
Your rechunk method has a bug in it. Since you have a fixed buffer in there, your file is split into ByteArray-parts. but the last part of the file can be smaller than the buffer, which is why you write too many bytes in the new file. and that's why you do not have the same checksum anymore. the error can be fixed like this:
public static void rechunck(File file1, File file2) {
FileInputStream is = null;
FileOutputStream os = null;
try {
byte[] buf = new byte[1024*64];
is = new FileInputStream(file1);
os = new FileOutputStream(file2);
int length;
while((length = is.read(buf)) > 0) {
os.write(buf, 0, length);
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
} finally {
try {
if(is != null)
is.close();
if(os != null)
os.close();
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
}
Due to the length variable, the write method knows that until byte x of the byte array, only the file is off, then there are still old bytes in it that no longer belong to the file.
I have written an downloader which should be used to download text files, as well as images. So I download the files as binaries. Many of the downloads work very well, but some parts of the text files and many image files are corrupted. The errors occur always at the same files and at the same places (as long as I can tell when analysing the text files). I used this code for downloading:
public File downloadFile(HttpURLConnection connection) {
return writeFileDataToFile(getFileData(connection));
}
//downloads the data of the file and returns the content as string
private List<Byte> getFileData(HttpURLConnection connection) {
List<Byte> fileData = new ArrayList<>();
try (InputStream input = connection.getInputStream()) {
byte[] fileChunk = new byte[8*1024];
int bytesRead;
do {
bytesRead = input.read(fileChunk);
if (bytesRead != -1) {
fileData.addAll(Bytes.asList(fileChunk));
fileChunk = new byte[8*1024];
}
} while (bytesRead != -1);
return fileData;
} catch (IOException e) {
System.out.println("Receiving file at " + url.toString() + " failed");
System.exit(1);
return null; //shouldn't be reached
}
}
//writes data to the file
private File writeFileDataToFile(List<Byte> fileData) {
if (!this.file.exists()) {
try {
this.file.getParentFile().mkdirs();
this.file.createNewFile();
} catch (IOException e) {
System.out.println("Error while creating file at " + file.getPath());
System.exit(1);
}
}
try (OutputStream output = new FileOutputStream(file)) {
output.write(Bytes.toArray(fileData));
return file;
} catch (IOException e) {
System.out.println("Error while accessing file at " + file.getPath());
System.exit(1);
return null;
}
}
I could suggest you to not pass through List of Byte, since you create a list of Byte from an array, to get it back to an array of Byte, which is not really efficient.
Moreover you wrongly assume the chunk size (not necesseraly 8192 bytes).
Why don't you just do something as:
private File writeFileDataToFile(HttpURLConnection connection) {
if (!this.file.exists()) {
try {
this.file.getParentFile().mkdirs();
//this.file.createNewFile(); // not needed, will be created at FileOutputStream
} catch (IOException e) {
System.out.println("Error while creating file at " + file.getPath());
//System.exit(1);
// instead do a throw of error or return null
throw new YourException(message);
}
}
OutputStream output = null;
InputStream input = null;
try {
output = new FileOutputStream(file):
input = connection.getInputStream();
byte[] fileChunk = new byte[8*1024];
int bytesRead;
while ((bytesRead = input.read(fileChunk )) != -1) {
output.write(fileChunk , 0, bytesRead);
}
return file;
} catch (IOException e) {
System.out.println("Receiving file at " + url.toString() + " failed");
// System.exit(1); // you should avoid such exit
// instead do a throw of error or return null
throw new YourException(message);
} finally {
if (input != null) {
try {
input.close();
} catch (Execption e2) {} // ignore
}
if (output != null) {
try {
output.close();
} catch (Execption e2) {} // ignore
}
}
}
The failure was adding the whole fileChunk Array to file data, even if it wasn't completely filled by the read operation.
Fix:
//downloads the data of the file and returns the content as string
private List<Byte> getFileData(HttpURLConnection connection) {
List<Byte> fileData = new ArrayList<>();
try (InputStream input = connection.getInputStream()) {
byte[] fileChunk = new byte[8*1024];
int bytesRead;
do {
bytesRead = input.read(fileChunk);
if (bytesRead != -1) {
fileData.addAll(Bytes.asList(Arrays.copyOf(fileChunk, bytesRead)));
}
} while (bytesRead != -1);
return fileData;
} catch (IOException e) {
System.out.println("Receiving file at " + url.toString() + " failed");
System.exit(1);
return null; //shouldn't be reached
}
}
Where the relevant change is changing
if (bytesRead != -1) {
fileData.addAll(Bytes.asList(fileChunk));
fileChunk = new byte[8*1024];
}
into
if (bytesRead != -1) {
fileData.addAll(Bytes.asList(Arrays.copyOf(fileChunk, bytesRead)));
}
I've split a mp3 file of 10 MB size into 10 parts of 1 MB each in mp3 format on my Android device, each file plays successfully by the player but while reading the data of all the 10 files and writing it to a single file the total size of the new file is more than 17 MB and the file doesn't play itself. Following is the code:
CODE FOR FILE SPLIT :
File file = new File(Environment.getExternalStorageDirectory()
+ "/MusicFile.mp3");
try {
FileInputStream fis = new FileInputStream(file);
FileOutputStream fos = null;
int size = 1048576; // 1 MB of data
byte buffer[] = new byte[size];
int count = 0;
int i = 0;
while (true) {
i = fis.read(buffer, 0, size);
if (i == -1) {
break;
}
File filename = getSplitFileName("split_" + count);
fos = new FileOutputStream(filename);
fos.write(buffer, 0, i);
++count;
}
fis.close();
fos.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
} catch (Exception e2) {
e2.printStackTrace();
}
CODE FOR FILE JOIN :
File folder = new File(cacheDirSplit.getAbsolutePath());
File files[] = folder.listFiles();
BufferedReader bufReader = null;
BufferedWriter bufWriter = null;
if (files.length > 1) {
try {
File fileName = getJoinedFileName("NewMusicFile");
String data;
for (int i = 0; i < files.length; i++) {
long dataSize = 0;
bufReader = new BufferedReader(new FileReader(
files[i]));
bufWriter = new BufferedWriter(new FileWriter(
fileName, true));
while ((data = bufReader.readLine()) != null) {
bufWriter.write(data);
dataSize = dataSize + data.getBytes().length;
}
Log.i("TAG", "File : " + files[i] + "size ==> "
+ dataSize);
}
bufReader.close();
bufWriter.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
What i do not understand is that while reading each file is read as 1.7MB as printed by the LOGCAT output but on the device when i check the splitted file is of 1MB only. Is there anything wrong with the code or is there some other thing I'm missing? Thanks in advance.
You cannot use readLine() on the content of an mp3 file. readLine() is for text files only. And if the ten were really playable and real mp3 files you had to strip the header first as Onur explained.
I have a TCP server that accepts data and saves it to a text file. It then uses that text file to create an image and sends it back to the client. Every couple of hours I will get a NullPointerException that gets thrown to every client that connects after that. I am not sure how to go about debugging this as I cannot replicate it on my own.
Does anyone have any debugging practices to help me figure out why this is becoming a problem?
The server running is running Ubuntu 12.04 i386 with 2 gigs of RAM. My initial suspicion is that something is not getting closed properly and creating issues but everything should be getting closed as far as I can tell.
ServerSocket echoServer = null;
Socket clientSocket = null;
try {
echoServer = new ServerSocket(xxx);
} catch (IOException e) {
System.out.println(e);
}
while(true)
{
InputStream is = null;
FileOutputStream fos = null;
BufferedOutputStream bos = null;
int bufferSize = 0;
FileInputStream fis = null;
BufferedInputStream bis = null;
BufferedOutputStream out = null;
try {
//Receieve text file
is = null;
fos = null;
bos = null;
bufferSize = 0;
String uid = createUid();
try {
clientSocket = echoServer.accept();
clientSocket.setKeepAlive(true);
clientSocket.setSoTimeout(10000);
System.out.println("Client accepted from: " + clientSocket.getInetAddress());
} catch (IOException ex) {
System.out.println("Can't accept client connection. ");
}
try {
is = clientSocket.getInputStream();
bufferSize = clientSocket.getReceiveBufferSize();
System.out.println("Buffer size: " + bufferSize);
} catch (IOException ex) {
System.out.println("Can't get socket input stream. ");
}
try {
fos = new FileOutputStream("/my/diretory/" + uid + ".txt");
bos = new BufferedOutputStream(fos);
} catch (FileNotFoundException ex) {
System.out.println("File not found. ");
}
byte[] bytes = new byte[bufferSize];
int count;
while ((count = is.read(bytes)) > 0) {
bos.write(bytes, 0, count);
System.out.println("Receiving... " + count);
}
System.out.println("Done receiving text file");
bos.flush();
bos.close();
fos.close();
//image
String[] command = new String[3];
command[0] = "python";
command[1] = "imagecreationfile.py";
command[2] = uid;
System.out.println("Starting python script");
Boolean success = startScript(command);
if(success)
{
System.out.println("Script completed successfully");
//Send image here
String image = "/my/directory/" + uid + ".png";
File imageFile = new File(image);
long length = imageFile.length();
if (length > Integer.MAX_VALUE) {
System.out.println("File is too large.");
}
bytes = new byte[(int) length];
fis = new FileInputStream(imageFile);
bis = new BufferedInputStream(fis);
out = new BufferedOutputStream(clientSocket.getOutputStream());
count = 0;
while ((count = bis.read(bytes)) > 0) {
out.write(bytes, 0, count);
System.out.println("Writing... " + count);
}
out.flush();
out.close();
fis.close();
bis.close();
}
else
{
System.out.println("Script failed");
}
System.out.println("Closing connection");
is.close();
clientSocket.close();
} catch (Exception e) {
System.out.println(e); //This is where the exception is being caught
}
if(!clientSocket.isClosed())
{
try {
clientSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
try {
if(is != null)
is.close();
if(fos != null)
fos.close();
if(bos != null)
bos.close();
if(fis != null)
fis.close();
if(bis != null)
bis.close();
if(out != null)
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Maybe exception was thrown in one of your try-catch scope.
And the next try-catch scope found null variables.
for example
//Receieve text file
is = null;
fos = null;
bos = null;
bufferSize = 0;
String uid = createUid();
try {
clientSocket = echoServer.accept();
clientSocket.setKeepAlive(true);
clientSocket.setSoTimeout(10000);
System.out.println("Client accepted from: " + clientSocket.getInetAddress());
} catch (IOException ex) {
System.out.println("Can't accept client connection. ");
}
try {
is = clientSocket.getInputStream();
bufferSize = clientSocket.getReceiveBufferSize();
System.out.println("Buffer size: " + bufferSize);
} catch (IOException ex) {
System.out.println("Can't get socket input stream. ");
}
if IOException was thrown in "clientSocket = echoServer.accept();" , it will print "Can't accept client connection. ".
When, "is = clientSocket.getInputStream();" executed, it will throw NullPointer because "clientSocket" was not initialized properly.
My suggestion, dont break a sequenced statement in different try-catch scope until it necessary.
Hi my java program is supposed to read in and display a .txt file the user enters when prompted, convert the integers in the file to an output .dat file, then read in that .dat file and display the numbers again. When I run my program it displays the contents of the file, and creates the .dat file, but dosn't read it in again. My code is below. What do I need to do?
public class InputFile
{
public static void main(String [] args)
{
BufferedReader inputStream = null;
System.out.print("Enter file name (with .txt extension): ");
Scanner keys = new Scanner(System.in);
String inFileName = keys.next();
try
{
inputStream = new BufferedReader (new FileReader(inFileName));
System.out.println("The file " + inFileName + " contains the following lines:");
String inFileString = inputStream.readLine();
while(inFileString != null)
{
System.out.println(inFileString);
inFileString = inputStream.readLine();
}
inputStream.close();
}
catch(FileNotFoundException e)
{
System.out.println(inFileName + " not found! Try Again.");
}
catch(IOException e)
{
System.out.println(e.getMessage());
}
String fileName = "numbers.dat";
try
{
ObjectOutputStream outputStream = new ObjectOutputStream(new FileOutputStream(fileName));
int anInt = 0;
while(anInt >=0);
{
anInt = Integer.parseInt(inputStream.readLine());
outputStream.writeInt(anInt);
}
outputStream.close();
}
catch(FileNotFoundException e)
{
System.out.println("Problem opening file.");
}
catch(IOException e)
{
System.out.println("Problem with output to the file.");
}
try
{
ObjectInputStream inputStream2 = new ObjectInputStream(new FileInputStream(fileName));
System.out.println("The file being read yields:");
int anInteger = inputStream2.readInt();
while(anInteger >= 0)
{
System.out.println(anInteger);
anInteger = inputStream2.readInt();
}
inputStream2.close();
}
catch(FileNotFoundException e)
{
System.out.println("Problem with opening the file.");
}
catch(EOFException e)
{
System.out.println("Problem reading the file.");
}
catch(IOException e)
{
System.out.println("There was a problem reading the file.");
}
}
}
There's a mistype (or at least I suppose it was a mistype) hard to spot that makes your second loop infinite.
(...)
try
{
ObjectOutputStream outputStream = new ObjectOutputStream(new FileOutputStream(fileName));
int anInt = 0;
while(anInt >=0); <=====
{
anInt = Integer.parseInt(inputStream.readLine());
outputStream.writeInt(anInt);
}
outputStream.close();
}
Remove this ';' after the while and I guess it'll run normally.
you are not writing to the output stream because by that time the inputStream has been exhausted and is closed.
create a collection to store the elements from the first file.
String inFileName = keys.next();
Collection<String> lines = new ArrayList<String>();
...
System.out.println(inFileString);
lines.add(inFileString);
...
for(String line : lines){
...
outputStream.write(Integer.parseInt(line));
...
}