Develop Reference

Java: Replace part of file without writing the entire file again

Is it possible to replace part of a files content, without rewriting the entire file to the disk.
Say that i have a very large file of several gigabytes, how to i replace the bytes from, lets say position 100 to 200 without rewriting the entire file?
As an added bonus, i need a solution that does not use any features never than java 1.4.

If you're positive that you're going to be writing exactly the same number of bytes, you can use a RandomAccessFile to accomplish this (available since Java 1.0). Just open the file, seek to wherever you need to be, and overwrite those bytes with whatever your new data is.
RandomAccessFile f = new RandomAccessFile(new File("C:\\test\\huge.txt"), "rw");
f.seek(100); // Seek ahead
f.write("here is some new stuff".getBytes())
You can also read from the file at arbitrary points in the same fashion, in case you don't know exactly how much data you need to replace (e.g. so you can pad/truncate whatever you're writing to avoid doing something awful by accident).

sending serialization file via sockets in java

System.out.println("Java is awesome!");
Pardon my enthusiasm; I just can't believe how powerful Java is, what with its ability to not only save objects (and load them), but also with its main purpose, to send them over a network. This is exactly what I must do, for I am conducting a beta-test. In this beta-test, I have given the testers a version of the game that saves the data as Objects in a location most people don't know about (we are the enlightened ones hahaha). This would work fine and dandy, except that it isn't meant for long-term persistence. But, I could collect their record.ser and counter.bin files (the latter tells me how many Objects are in record.ser) via some client/server interaction with sockets (which I know nothing about, until I started reading about it, but I still feel clueless). Most of the examples I have seen online (this one for example: http://uisurumadushanka89.blogspot.com/2010/08/send-file-via-sockets-in-java.html ) were sending the File as a stream of bytes, namely some ObjectOutputStream and ObjectInputStream. This is exactly what my current version of the game is using to save/load GameData.
Sorry for this long-winded intro, but do you know what I would have to do (steps-wise, so I can UNDERSTAND) to actually send the whole file. Would I have to reconstruct the file byte-by-byte (or Object-by-Object)?

Its pretty simple, actually. Just make your objects serializable, and create an ObjectOutputStream and ObjectInputStream that are connected to whatever underlying stream you have, say FileInputStream, etc. Then just write() whatever object you want to the stream and read it on the other side.
Heres an example for you.
For sockets it will be something like
ObjectOutputStream objectOut = new ObjectOutputStream(serverSocket.getOutputStream());
ObjectInputStream objectIn = new ObjectInputStream(clientSocket.getInputStream());

Java Serialization is an immensely powerful protocol. java.io.ObjectOutputStream and java.io.ObjectInputStream are the higher level classes which of course are wrapped with the lower level classes such as FileInputStream and FileOutputStream. My question is why do you wish to read the file byte by byte when the entire file can be read in Objects.
Here is a good explanation of the procedure.
http://www.tutorialspoint.com/java/java_serialization.html

Stream chaining in Java

Is it bad style to keep the references to streams "further down" a filter chain, and use those lower level streams again, or even to swap one type of stream for another? For example:
OutputStream os = new FileOutputStream("file");
PrintWriter pw = new PrintWriter(os);
pw.print("print writer stream");
pw.flush();
pw = null;
DataOutputStream dos = new DataOutputStream(os);
dos.writeBytes("dos writer stream");
dos.flush();
dos = null;
os.close();
If so, what are the alternatives if I need to use the functionality of both streams, e.g. if I want to write a few lines of text to a stream, followed by binary data, or vice versa?

This can be done in some cases, but it's error-prone. You need to be careful about buffers and stuff like the stream headers of ObjectOutputStream.
if I want to write a few lines of text to a stream, followed by binary
data, or vice versa?
For this, all you need to know is that you can convert text to binary data and back but always need to specify an encoding. However, it is also error-prone because people tend to use the API methods that use the platform default encoding, and of course you're basically implementing a parser for a custom binary file format - lots of things can go wrong there.
All in all, if you're creating a file format, especially when mixing text and binary data, it's best to use an existing framework like Google protocol buffers

If you have to do it, then you have to do it. So if you're dealing with an external dependency that you don't have control over, you just have to do it.
I think the bad style is the fact that you would need to do it. If you had to send binary data across sometimes, and text across at others, it would probably be best to have some kind of message object and send the object itself over the wire with Serialization. The data overhead isn't too much if structured properly.

I don't see why not. I mean, the implementations of the various stream classes should protect you from writing invalid data. So long as you're reading it back the same way, and your code is otherwise understandable, I don't see why that would be a problem.
Style doesn't always mean you have to do it the way you've seen others do it. So long as it's logical, and someone reading the code would see what (and why) you're doing it without you needing to write a bunch of comments, then I don't see what the issue is.

Since you're flushing between, it's probably fine. But it might be cleaner to use one OutputStream and just use os.write(string.getBytes()); to write the strings.

How do I write/read to the beginning of a text file?

EDIT
This is my file reader, can I make this read it from bottom to up seeing how difficult it is to make it write from bottom to up.
BufferedReader mainChat = new BufferedReader(new FileReader("./messages/messages.txt"));
String str;
while ((str = mainChat.readLine()) != null)
{
System.out.println(str);
}
mainChat.close();
OR (old question)
How can I make it put the next String at the beginning of the file and then insert an new line(to shift the other lines down)?
FileWriter chatBuffer = new FileWriter("./messages/messages.txt",true);
BufferedWriter mainChat = new BufferedWriter(chatBuffer);
mainChat.write(message);
mainChat.newLine();
mainChat.flush();
mainChat.close();

Someone could correct me, but I'm pretty sure in most operating systems, there is no option but to read the whole file in, then write it back again.
I suppose the main reason is that, in most modern OSs, all files on the disc start at the beginning of a boundary. The problem is, you cannot tell the file allocation table that your file starts earlier than that point.
Therefore, all the later bytes in the file have to be rewritten. I don't know of any OS routines that do this in one step.
So, I would use a BufferedReader to store whole file into a Vector or StringBuffer, then write it all back with the prepended string first.
--
Edit
A way that would save memory for larger files, reading #Saury's randomaccessfile suggestion, would be:
file has N bytes to start with
we want to add on "hello world"
open the file for append
append 11 spaces
i=N
loop {
go back to byte i
read a byte
move to byte i+11
write that byte back
i--
} until i==0
then move to byte 0
write "hello world"
voila

Use FileUtils from Apache Common IO to simplify this if you can. However, it still needs to read the whole file in so it will be slow for large files.
List<String> newList = Arrays.asList("3");
File file = new File("./messages/messages.txt");
newList.addAll(FileUtils.readLines(file));
FileUtils.writeLines(file, newList);
FileUtils also have read/write methods that take care of encoding.

Use RandomAccessFile to read/write the file in reverse order. See following links for more details.
http://www.java2s.com/Code/Java/File-Input-Output/UseRandomAccessFiletoreverseafile.htm
http://download.oracle.com/javase/1.5.0/docs/api/java/io/RandomAccessFile.html

As was suggested here pre-pending to a file is rather difficult and is indeed linked to how files are stored on the hard drive. The operation is not naturally available from the OS so you will have to make it yourself and most obvious answers to this involve reading the whole file and writing it again. this may be fine for you but will incur important costs and could be a bottleneck for your application performance.
Appending would be the natural choice but this would, as far as I understand, make reading the file unnatural.
There are many ways you could tackle this depending on the specificities of your situation.
If writing this file is not time critical in your application and the file does not grow too big you could bite the bullet and read the whole file, prepend the information and write it again. apache's common-io's FileUtils will be of help here simpifying the operation where you can read the file as a list of strings, prepend the new lines to the list and write the list again.
If writing is time critical but have control over the reading or the file. That is, if the file is to be read by another of your programs. you could load the file in a list of lines and reverse the list. Again FileUtils from the common-io library and helper functions in the Collections class in the standard JDK should do the trick nicely.
If writing is time critical but the file is intended to be read through a normal text editor you could create a small class or program that would read the file and write it in another file with the preferred order.

(Java) File redirection (both ways) within Runtime.exec?

I want to execute this command:
/ceplinux_work3/myName/opt/myCompany/ourProduct/bin/EXECUTE_THIS -p cepamd64linux.myCompany.com:19021/ws1/project_name < /ceplinux_work3/myName/stressting/Publisher/uploadable/00000.bin >> /ceplinux_work3/myName/stressting/Publisher/stats/ws1.project_name.19021/2011-07-22T12-45-20_PID-2237/out.up
But it doesn't work because EXECUTE_THIS requires an input file via redirect, and simply passing this command to Runtime.exec doesn't work.
Side note: I searched all over on how to solve this before coming here to ask. There are many questions/articles on the web regarding Runtime.exec and Input/Output redirect. However, I cannot find any that deal with passing a file to a command and outputting the result to another file. Plus, I am totally unfamiliar with Input/Output streams, so I have a hard time putting all the info out there together for my specific situation.
That said, any help is much appreciated.
P.S. If there are multiple ways to do this, I prefer whatever is fastest in terms of throughput.
Edit: As discussed in my last question, I CANNOT change this to a bash call because the program must wait for this process to finish before proceeding.

Unless you are sending a file name to the standard input of the process, there is no distinction of whether the data came from a file or from any other data source.
You need to write to the OutputStream given by Process.getOutputStream(). The data you write to it you can read in from a file using a FileInputStream.
Putting that together might look something like this:
Process proc = Runtime.getRuntime().exec("...");
OutputStream standardInputOfChildProcess = proc.getOutputStream();
InputStream dataFromFile = new FileInputStream("theFileWithTheData.dat");
byte[] buff = new byte[1024];
for ( int count = -1; (count = dataFromFile.read(buff)) != -1; ) {
standardInputOfChildProcess.write(buff, 0, count);
}
I've left out a lot of details, this is just to get the gist of it. You'll want to safely close things, might want to consider buffering and you need to worry about the pitfalls of Runtime.exec().
Edit
Writing the output to a file is similar. Obtain a FileOutputStream pointing to the output file and write the data you read from Process.getInputStream() to that OutputStream. The major caveat here is that you must do this operation in a second thread, since accessing two blocking streams from the same thread will lead to deadlock (see the article above).