Develop Reference

Write a number to FileOutputStream after an offset

I want to write a number to the FileOutputStream after a certain offset. What I have wrote is the following:
String t = Integer.toString(num);
output.write(t.getBytes(), OFFSET, t.length());
Is this a good way to do this, or this there a better way?

This is a classic XY problem. What you want to do is write to a file at a specific offset. Specifying 'FileOutputStream' isn't part of the problem, it is part of your incorrect solution. You should be using RandomAccessFile for this task.

You're not using the FileOutputStream class correctly. The first parameter is the array you want to write on the file. The second one is the offset from that array that you want to start writing from. The third one is the length of what you want to write. So, in your case, if you use an offset which is different from 0, you'll get an out of bounds exception.
If you want to write that number inside a file using FileOutputStream, you can read that file (using, for example, a FileInputStream), read the amount of bytes you want to read and write them into the output stream, then write that number using something akin to:
output.write(t.getBytes(),0,t.length);
And finally reading the rest of the inputstream and writing it into the outputstream.
Best of luck.

I found this solution that corresponds to the #sirs comment :
final RandomAccessFile raf = new RandomAccessFile(file, "rw");
raf.setLength(offset);
raf.seek(offset);
FileOutputStream fos = new FileOutputStream(raf.getFD()) {
#Override
public void close() throws IOException {
super.close();
raf.close();
}
};
fos.write(myInt);

Reading from input stream

I am developing a library for my Android application where I am appending extra info to the end of a PNG image. I am creating a DataOutputStream variable and writing extra info to the end of it to use when I open the PNG and convert it to a Bitmap using a DataInputStream. I added a marker to distinguish when the image code finishes and my extra info starts.
The extra data is correctly being appended after the marker. The problem is reading ONLY the PNG data of the DataInputStream to convert it into a Bitmap. All of the DataInputStream is being read (even if I add a large amount of placeholder bytes before the marker).
The implementation I am using to read the PNG portion of the stream is:
Bitmap image = BitmapFactory.decodeStream(inputStream);
I am wondering if there is another way I should be implementing this to stop reading the stream after the PNG data bytes.
If there isn't a better way, the route I would be taking is copying the input stream into an array. I would then read all of the data until I reach the marker.

You can create a wrapper InputStream that would then report EOF before reading the entire stream. This lets you avoid having to read the whole stream into a byte array.
class MarkerInputStream extends FilterInputStream {
MarkerInputStream(InputStream in) {
super(in);
}
#Override
public int read() throws IOException {
if (isAtMarker()) {
return -1;
}
// may need to read from a cache depending on what isAtMarker method does.
return super.read();
}
private boolean isAtMarker() {
// logic for determining when you're at the end of the image portion
return false;
}
}

Sending images through a socket using qt and read it using java

I'm trying to send an image upload in a Qt server trough the socket and visualize it in a client created using Java. Until now I have only transferred strings to communicate on both sides, and tried different examples for sending images but with no results.
The code I used to transfer the image in qt is:
QImage image;
image.load("../punton.png");
qDebug()<<"Image loaded";
QByteArray ban; // Construct a QByteArray object
QBuffer buffer(&ban); // Construct a QBuffer object using the QbyteArray
image.save(&buffer, "PNG"); // Save the QImage data into the QBuffer
socket->write(ban);
In the other end the code to read in Java is:
BufferedInputStream in = new BufferedInputStream(socket.getInputStream(),1);
File f = new File("C:\\Users\\CLOUDMOTO\\Desktop\\JAVA\\image.png");
System.out.println("Receiving...");
FileOutputStream fout = new FileOutputStream(f);
byte[] by = new byte[1];
for(int len; (len = in.read(by)) > 0;){
fout.write(by, 0, len);
System.out.println("Done!");
}
The process in Java gets stuck until I close the Qt server and after that the file generated is corrupt.
I'll appreciate any help because it's neccessary for me to do this and I'm new to programming with both languages.
Also I've used the following commands that and the receiving process now ends and show a message, but the file is corrupt.
socket->write(ban+"-1");
socket->close(); in qt.
And in java:
System.out.println(by);
String received = new String(by, 0, by.length, "ISO8859_1");
System.out.println(received);
System.out.println("Done!");

You cannot transport file over socket in such simple way. You are not giving the receiver any clue, what number of bytes is coming. Read javadoc for InputStream.read() carefully. Your receiver is in endless loop because it is waiting for next byte until the stream is closed. So you have partially fixed that by calling socket->close() at the sender side. Ideally, you need to write the length of ban into the socket before the buffer, read that length at receiver side and then receive only that amount of bytes. Also flush and close the receiver stream before trying to read the received file.
I have absolutely no idea what you wanted to achieve with socket->write(ban+"-1"). Your logged output starts with %PNG which is correct. I can see there "-1" at the end, which means that you added characters to the image binary file, hence you corrupted it. Why so?
And no, 1x1 PNG does not have size of 1 byte. It does not have even 4 bytes (red,green,blue,alpha). PNG needs some things like header and control checksum. Have a look at the size of the file on filesystem. This is your required by size.

Checking if a stream is a zip file

We have a requirement to determine whether an incoming InputStream is a reference to an zip file or zip data. We do not have reference to the underlying source of the stream. We aim to copy the contents of this stream into an OutputStream directed at an alternate location.
I tried reading the stream using ZipInputStream and extracting a ZipEntry. The ZipEntry is null if the stream is a regular file - as expected - however, in checking for a ZipEntry I loose the initial couple of bytes from the stream. Hence, by the time I know that the stream is a regular stream, I have already lost initial data from the stream.
Any thoughts around how to check if the InputStream is an archive without data loss would be helpful.
Thanks.

Assuming your original inputstream is not buffered, I would try wrapping the original stream in a BufferedInputStream, before wrapping that in a ZipInputStream to check. You can use "mark" and "reset" in the BufferedInputStream to return to the initial position in the stream, after your check.

This is how I did it.
Using mark/reset to restore the stream if the GZIPInputStream detects incorrect zip format (throws the ZipException).
/**
* Wraps the input stream with GZIPInputStream if needed.
* #param inputStream
* #return
* #throws IOException
*/
private InputStream wrapIfZip(InputStream inputStream) throws IOException {
if (!inputStream.markSupported()) {
inputStream = new BufferedInputStream(inputStream);
}
inputStream.mark(1000);
try {
return new GZIPInputStream(inputStream);
} catch (ZipException e) {
inputStream.reset();
return inputStream;
}
}

You can check first bytes of stream for ZIP local header signature (PK 0x03 0x04), that would be enough for most cases. If you need more precision, you should take last ~100 bytes and check for central directory locator fields.

You have described a java.io.PushbackInputStream - in addition to read(), it has an unread(byte[]) which allows you push them bck to the front of the stream, and to re-read() them again.
It's in java.io since JDK1.0 (though I admit I haven't seen a use for it until today).

It sounds a bit like a hack, but you could implement a proxy java.io.InputStream to sit between ZipInputStream and the stream you originally passed to ZipInputStream's constructor. Your proxy would stream to a buffer until you know whether it's a ZIP file or not. If not, then the buffer saves your day.

how to write a file without allocating the whole byte array into memory?

This is a newbie question, I know. Can you guys help?
I'm talking about big files, of course, above 100MB. I'm imagining some kind of loop, but I don't know what to use. Chunked stream?
One thins is for certain: I don't want something like this (pseudocode):
File file = new File(existing_file_path);
byte[] theWholeFile = new byte[file.length()]; //this allocates the whole thing into memory
File out = new File(new_file_path);
out.write(theWholeFile);
To be more specific, I have to re-write a applet that downloads a base64 encoded file and decodes it to the "normal" file. Because it's made with byte arrays, it holds twice the file size in memory: one base64 encoded and the other one decoded. My question is not about base64. It's about saving memory.
Can you point me in the right direction?
Thanks!

From the question, it appears that you are reading the base64 encoded contents of a file into an array, decoding it into another array before finally saving it.
This is a bit of an overhead when considering memory. Especially given the fact that Base64 encoding is in use. It can be made a bit more efficient by:
Reading the contents of the file using a FileInputStream, preferably decorated with a BufferedInputStream.
Decoding on the fly. Base64 encoded characters can be read in groups of 4 characters, to be decoded on the fly.
Writing the output to the file, using a FileOutputStream, again preferably decorated with a BufferedOutputStream. This write operation can also be done after every single decode operation.
The buffering of read and write operations is done to prevent frequent IO access. You could use a buffer size that is appropriate to your application's load; usually the buffer size is chosen to be some power of two, because such a number does not have an "impedance mismatch" with the physical disk buffer.

Perhaps a FileInputStream on the file, reading off fixed length chunks, doing your transformation and writing them to a FileOutputStream?

Perhaps a BufferedReader? Javadoc: http://download-llnw.oracle.com/javase/1.4.2/docs/api/java/io/BufferedReader.html

Use this base64 encoder/decoder, which will wrap your file input stream and handle the decoding on the fly:
InputStream input = new Base64.InputStream(new FileInputStream("in.txt"));
OutputStream output = new FileOutputStream("out.txt");
try {
byte[] buffer = new byte[1024];
int readOffset = 0;
while(input.available() > 0) {
int bytesRead = input.read(buffer, readOffset, buffer.length);
readOffset += bytesRead;
output.write(buffer, 0, bytesRead);
}
} finally {
input.close();
output.close();
}

You can use org.apache.commons.io.FileUtils. This util class provides other options too beside what you are looking for. For example:
FileUtils.copyFile(final File srcFile, final File destFile)
FileUtils.copyFile(final File input, final OutputStream output)
FileUtils.copyFileToDirectory(final File srcFile, final File destDir)
And so on.. Also you can follow this tut.