I'm trying read a file(doesn't matter the extension) and write after this, but when I do it, the output file is different from the input.
my code is the next:
OutputStream outputStream = null;
FileReader fr = new FileReader("rute\\inputfile.PNG");
BufferedReader br = new BufferedReader(fr);
String line;
while ((line= br.readLine()) != null) {
content += line;
}
byte[] toBytes= content.getBytes();
InputStream inputStream = new ByteArrayInputStream(toBytes);
try {
outputStream = new FileOutputStream(new File("rute\\output.PNG"));
int read = 0;
byte[] bytes = new byte[1024];
while ((read = inputStream.read(bytes)) != -1) {
outputStream.write(bytes, 0, read);
}
outputStream.close();
} catch (Exception e) {
e.printStackTrace();
}
inputStream.close();
If you ask me why convert into bytes and write from this form, is because I need do something with the data, and I need this conversion.
If you tell me that i cant load an image on a String, yes I can do something like that:
File fil = ~~~~;
FileInputStream fis = null;
fis = new FileInputStream(fil);
byte[] bytess = IOUtils.toByteArray(fis);
But I dont want do it by this way because if I want load big files, the heap size is not enough an this could be solved by the "line per line" read.
Thanks for your answers
I will recommend read this question before. Since you are reading binary data into a String you are changing the encoding of that data. So the output will be different.
Best approach is read binary files as byte arrays. But I will depend which type of transformation/edition/changes you need to do with them.
UPDATE
And, of course, you are editing your content before writing
while ((line= br.readLine()) != null) {
content += line + "\n";
}
so the your output file will be different always.
UPDATE 2
Since the question/problem is how to read a big binary file, google is usually your friend.
Or you can check this other question
Related
Currently I have a source file which has base64 encoded data (20 mb in size approx). I want to read from this file, decode the data and write to a .TIF output file. However I don't want to decode all 20MB data at once. I want to read a specific number of characters/bytes from the source file, decode it and write to destination file. I understand that the size of the data I read from the source file has to be in multiples of 4 or else it can't be decoded?
Below is my current code where I decode it all at once
public write Output(File file){
BufferedReader br = new BufferedReader (new Filereader(file));
String builder sb = new StringBuilder ();
String line=BR.readLine();
While(line!=null){
....
//Read line by line and append to sb
}
byte[] decoded = Base64.getMimeDecoder().decode(SB.toString());
File outputFile = new File ("output.tif")
OutputStream out = new BufferedOutputStream(new FileOutputStream(outputFile));
out.write(decoded);
out.flush();
}
How can I read specific number of characters from source file and decode and then write to output file so I don't have to load everything in memory?
Here is a simple method to demonstrate doing this, by wrapping the Base64 Decoder around an input stream and reading into an appropriately sized byte array.
public static void readBase64File(File inputFile, File outputFile, int chunkSize) throws IOException {
FileInputStream fin = new FileInputStream(inputFile);
FileOutputStream fout = new FileOutputStream(outputFile);
InputStream base64Stream = Base64.getMimeDecoder().wrap(fin);
byte[] chunk = new byte[chunkSize];
int read;
while ((read = base64Stream.read(chunk)) != -1) {
fout.write(chunk, 0, read);
}
fin.close();
fout.close();
}
I have the following code:
BlobDomain blobDomain = null;
OutputStream out = null;
try {
blobDomain = new BlobDomain();
out = blobDomain.getBinaryOutputStream();
byte[] buffer = new byte[8192];
int bytesRead = 0;
while ((bytesRead = in.read(buffer, 0, 8192)) != -1) {
out.write(buffer, 0, bytesRead);
String line = (new String(buffer));
fullText += line;
}
} catch (Exception e) {
//do nothing
}finally{
if (out != null)
try {
out.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
when i print the fullText what i see for larger files is that end part of the text is added again to the fullText. So full text has some lines repeated in the end. any suggestions on what is wrong here?
The reason that you are getting this is that you are writing the entire buffer every time to your String. Thus, when you reach the end of the file you may not have read exactly the amount of bytes that your buffer is sized at. The old data is still in the buffer and will also be written to your String.
One option to solve this may be to write your data to a String first and then to write your String to the output stream. This should also be faster than adding to a String after each read.
Save inputStream to String:
java.util.Scanner s = new java.util.Scanner(in).useDelimiter("\\A");
fullText = s.hasNext() ? s.next() : "";
Write String to output stream:
out.write(fullText.getBytes());
If you want to keep you code as-is then perform a substring on the buffer and retrieve only the amount of bytes read. For example:
String line = (new String(buffer.substring(0,bytesRead));
I'm looking for a way to switch between reading bytes (as byte[]) and reading lines of Strings from a file. I know that a byte[] can be obtained form a file through a FileInputStream, and a String can be obtained through a BufferedReader, but using both of them at the same time is proving problematic. I know how long the section of bytes are. String encoding can be kept constant from when I write the file. The filetype is a custom one that is still in development, so I can change how I write data to it.
How can I read Strings and byte[]s from the same file in java?
Read as bytes. When you have read a sequence of bytes that you know should be a string, place those bytes in an array, put the array inside a ByteArrayInputStream and use that as the underlying InputStream for a Reader to get the bytes as characters, then read those characters to produce a String.
For the later parts of this process see the related SO question on how to create a String from an InputStream.
Read the file as Strings using a BufferedReader then use String.getBytes().
Why not try this:
BufferedReader bufferedReader = null;
try {
bufferedReader = new BufferedReader(new FileReader("testing.txt"));
String line = bufferedReader.readLine();
while(line != null){
byte[] b = line.getBytes();
}
} finally {
if(bufferedReader!=null){
bufferedReader.close();
}
}
or
FileInputStream in = null;
BufferedReader bufferedReader = null;
try {
bufferedReader = new BufferedReader(new FileReader("xanadu.txt"));
String line = bufferedReader.readLine();
while(line != null){
//read your line
}
in = new FileInputStream("xanadu.txt");
int c;
while ((c = in.read()) != -1) {
//read your bytes (c)
}
} finally {
if (in != null) {
in.close();
}
if(bufferedReader!=null){
bufferedReader.close();
}
}
Read everything as bytes from the buffered input stream, and convert string sections into String's using constructor that accepts the byte array:
String string = new String(bytes, offset, length, "US-ASCII");
Depending on how the data are actually encoded, you may need to use "UTF-8" or something else as the name of the charset.
I'm attempting to output a text file to the console with Java. I was wondering what is the most efficient way of doing so?
I've researched several methods however, it's difficult to discern which is the least performance impacted solution.
Outputting a text file to the console would involve reading in each line in the file, then writing it to the console.
Is it better to use:
Buffered Reader with a FileReader, reading in lines and doing a bunch of system.out.println calls?
BufferedReader in = new BufferedReader(new FileReader("C:\\logs\\"));
while (in.readLine() != null) {
System.out.println(blah blah blah);
}
in.close();
Scanner reading each line in the file and doing system.print calls?
while (scanner.hasNextLine()) {
System.out.println(blah blah blah);
}
Thanks.
If all you want to do is print the contents of a file (and don't want to print the next int/double/etc.) to the console then a BufferedReader is fine.
Your code as it is won't produce the result you're after, though. Try this instead:
BufferedReader in = new BufferedReader(new FileReader("C:\\logs\\log001.txt"));
String line = in.readLine();
while(line != null)
{
System.out.println(line);
line = in.readLine();
}
in.close();
I wouldn't get too hung up about it, though because it's more likely that the main bottleneck will be the ability of your console to print the information that Java is sending it.
If you're not interested in the character based data the text file is containing, just stream it "raw" as bytes.
InputStream input = new BufferedInputStream(new FileInputStream("C:/logs.txt"));
byte[] buffer = new byte[8192];
try {
for (int length = 0; (length = input.read(buffer)) != -1;) {
System.out.write(buffer, 0, length);
}
} finally {
input.close();
}
This saves the cost of unnecessarily massaging between bytes and characters and also scanning and splitting on newlines and appending them once again.
As to the performance, you may find this article interesting. According the article, a FileChannel with a 256K byte array which is read through a wrapped ByteBuffer and written directly from the byte array is the fastest way.
FileInputStream input = new FileInputStream("C:/logs.txt");
FileChannel channel = input.getChannel();
byte[] buffer = new byte[256 * 1024];
ByteBuffer byteBuffer = ByteBuffer.wrap(buffer);
try {
for (int length = 0; (length = channel.read(byteBuffer)) != -1;) {
System.out.write(buffer, 0, length);
byteBuffer.clear();
}
} finally {
input.close();
}
If it's a relatively small file, a one-line Java 7+ way to do this is:
System.out.println(new String(Files.readAllBytes(Paths.get("logs.txt"))));
See https://docs.oracle.com/javase/7/docs/api/java/nio/file/package-summary.html for more details.
Cheers!
If all you want is most efficiently dump the file contents to the console with no processing in-between, converting the data into characters and finding line breaks is unnecessary overhead. Instead, you can just read blocks of bytes from the file and write then straight out to System.out:
package toconsole;
import java.io.BufferedInputStream;
import java.io.FileInputStream;
public class Main {
public static void main(String[] args) {
BufferedInputStream bis = null;
byte[] buffer = new byte[8192];
int bytesRead = 0;
try {
bis = new BufferedInputStream(new FileInputStream(args[0]));
while ((bytesRead = bis.read(buffer)) != -1) {
System.out.write(buffer, /* start */ 0, /* length */ bytesRead);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try { bis.close(); } catch (Exception e) { /* meh */ }
}
}
}
In case you haven't come across this kind of idiom before, the statement in the while condition both assigns the result of bis.read to bytesRead and then compares it to -1. So we keep reading bytes into the buffer until we are told that we're at the end of the file. And we use bytesRead in System.out.write to make sure we write only the bytes we've just read, as we can't assume all files are a multiple of 8 kB long!
FileInputStream input = new FileInputStream("D:\\Java\\output.txt");
FileChannel channel = input.getChannel();
byte[] buffer = new byte[256 * 1024];
ByteBuffer byteBuffer = ByteBuffer.wrap(buffer);
try {
for (int length = 0; (length = channel.read(byteBuffer)) != -1;) {
System.out.write(buffer, 0, length);
byteBuffer.clear();
}
} finally {
input.close();
}
Path temp = Files.move
(Paths.get("D:\\\\Java\\\\output.txt"),
Paths.get("E:\\find\\output.txt"));
if(temp != null)
{
System.out.println("File renamed and moved successfully");
}
else
{
System.out.println("Failed to move the file");
}
}
For Java 11 you could use more convenient approach:
Files.copy(Path.of("file.txt"), System.out);
Or for more faster output:
var out = new BufferedOutputStream(System.out);
Files.copy(Path.of("file.txt"), out);
out.flush();
I'm having an issue reading from a java input stream. I have a buffer of size 1024, and an input stream of size 29k-31k. I read the inputStream in a loop, but I only get 29 bytes for the first read, 39 for the second read, and nothing after that. The same behavior repeats for different InputStreams. (I'm writing the data to an output stream but I don't see how this can affect the first read)
int bytesRead = 0;
byte[] byteBuf = new byte[1024];
OutputStream fileStream = FileUtil.openFileForWrite(saveTo);
bytesRead = reader.read(byteBuf);
while(bytesRead!=-1){
fileStream.write(byteBuf, 0, bytesRead);
bytesRead = reader.read(byteBuf);
}
What am I missing?
Any help is appreciated :)
Where are you getting the input stream from? How do you know that it's 29K-31K?
Your code looks reasonable to me, although I generally structure the loop slightly different to avoid the duplication of the read call.
Have you tried using readline() instead of read()?
Path file = ...;
InputStream in = null;
try {
in = file.newInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
} catch (IOException x) {
System.err.println(x);
} finally {
if (in != null) in.close();
}