I am trying to send zipped bytes to another server and then have that server receive them and write out the zipped filed. When I do the zipping and writing on the same server it works great. The local version looks something like this:
ZipOutputStream zout = new ZipOutputStream(FileOutputStream);
zout.write(byteBuffer, 0, len);
zout.flush()
FileOutputStream.flush();
zout.close();
The cross server implementation creates a bad output though. The sending code is: (magic string tells the server it has received all of the data.
ZipOutputStream zout = new ZipOutputStream(out);
ZipEntry entry = new ZipEntry(fileName);
zout.putNextEntry(entry);
System.out.println("sending zipped bytes...");
zout.write(inputBuffer, contentBegin, len);
zout.flush();
zout.closeEntry();
out.flush();
byte[] magicStringData = "--------MagicStringCSE283Miami".getBytes("US-ASCII");
out.write(magicStringData, 0, magicStringData.length);
out.flush();
System.out.println("Done writing file and sending zipped bytes.");
Thread.sleep(10000);
zout.close();
clntSock.close(); // Close the socket. We are done with this client!
The receiving code looks like this:
System.out.println("receiving zipped bytes...");
byte[] inputBuffer = new byte[BUF_SIZE];
int total2 = 0, count = 0;
while(count != -1) { // read from origin's buffer into byteBuffer until origin is out of data
count = inFromCompression.read(inputBuffer, total2, BUF_SIZE - total - 1);
String msg = new String(inputBuffer, total2, count, "US-ASCII");
total2 += count;
if(msg.contains("-------MagicString")){
System.out.println("full message received...");
break;
}
}
String inputString = new String(inputBuffer, 0, total2, "US-ASCII");
int contentEnd = inputString.indexOf("--------MagicString");
FileOutputStream fout2 = new FileOutputStream(outputFileName + ".zip");
fout2.write(inputBuffer, 0, contentEnd);
fout2.flush();
fout2.close();
System.out.println("Done writing zipped bytes.");
//Thread.sleep(10000);
//socketToCompression.close();
Any ideas? I am thinking it might be something like I am not closing the ZipOutputStream before sending the magic string that signals the end of the data, but every time I call zout.close() immediately after flushing zout it closes the entire socket.
Get rid of the magic string and just send and receive the actual data. You're presently throwing away any buffer that contains the magic string, including whatever ZIP data may have been before it.
You don't need a ByteArrayOutputStream.
Related
Using Java, I am trying to send some file data over a DatagramSocket. I need to read a file in 1000-byte chunks and send them over as packets. My code:
reads a file into a byte array wrapped in a byte buffer
places the data in a packet and sends it
has the receiver open the packet and re-write the contents to a new file.
I am having a problem with writing the byte array back to a file. Please see my code below.
Client/Sender:
byte[] data = new byte[1000];
ByteBuffer b = ByteBuffer.wrap(data);
DatagramPacket pkt;
File file = new File(sourceFile);
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
CRC32 crc = new CRC32();
while(true){
b.clear();
b.putLong(0); // I need to put the checksum at the beginning for easy retrieval
bytesRead = bis.read(data);
if(bytesRead==-1) { break; }
crc.reset();
crc.update(data, 8, data.length-8);
long chksum = crc.getValue();
b.rewind();
b.putLong(chksum);
pkt = new DatagramPacket(data, 1000, addr); // addr is valid, works fine
sk.send(pkt);
}
bis.close();
fis.close();
Server/Receiver:
DatagramSocket sk = new DatagramSocket(port);
File destfile = new File("hello.txt");
FileOutputStream fos = new FileOutputStream(destfile);
BufferedOutputStream bos = new BufferedOutputStream(fos);
PrintStream ps = new PrintStream(fos);
byte[] data = new byte[1000];
DatagramPacket pkt = new DatagramPacket(data, data.length);
ByteBuffer b = ByteBuffer.wrap(data);
CRC32 crc = new CRC32();
while(true) {
pkt.setLength(data.length);
sk.receive(pkt);
b.rewind();
// compare checksum, print error if checksum is different
// if checksum is the same:
bos.write(data); // Where the problem seems to be occurring.
// send acknowledgement packet.
}
bos.close();
fos.close();
Here, I am mainly having issues with writing the file back. With a small text file that says Hello World!, I get a strange output that says vˇ]rld!. Also, the input file is only 12 bytes but the file that the receiver creates is 1KB.
I think my issue is dealing with a byte buffer - I've written a program that copies files using file streams and buffered streams, which worked well. But I'm confused with how streams work in this sort of situation, and I would really appreciate any help. Thanks!
In the sender's data[] you overwrite the text, which was read from the file by the crc! You have to read the text in a position after the long. When correcting this in the Sender, it works:
//int bytesRead = bis.read(data); --old code
int bytesRead=bis.read(data,8,data.length-8);
Furthermore you send 1000 bytes, so will receive 1000 bytes, which will go into the destfile.
BTW: you do not check the crc in the server.... so why send it ?
I am trying to create a simple server-client-program where the user can upload and download files. I have got the Sockets and Streams to work, and I can upload a file to the server. But whenever one file has been uploaded the Server-side seems to get stuck in the loop that reads the streams and forwards it to the Server-file.
Server Code:
InputStream in = clientSocket.getInputStream();
String filePath = "......."
+ op[1];
System.out.println(op[0] + ": " + filePath);
FileOutputStream out = new FileOutputStream(filePath);
byte[] bytes = new byte[16*1024];
int count;
while ((count = in.read(bytes)) > 0) {
out.write(bytes, 0, count);
}
Client Code:
String filePath = "...."
+ path;
System.out.println("Attempting: " + filePath);
dos = new DataOutputStream(serverSocket.getOutputStream());
fis = new FileInputStream(filePath);
byte[] buffer = new byte[4096];
while (fis.read(buffer) > 0) {
dos.write(buffer);
}
dos.flush();
fis.close();
The problem is that the program gets stuck at the while-loop, so the Server can not perform anything else. There are no errors or anything...
You never close the stream on the client side. Add dos.close() after dos.flush()!
When I run my client server - which is connecting and I try to send a file it doesn't won't send the whole file which is pulling errors else where, it gets about halfway through and constantly stops at the same part. This set up works when running the server-client on the same machine so I am completely confused
Server --->
// output (a DataOutputstream) is set up elsewhere and messages are sent and received properly
output.writeInt((int)file.length());
// send file
byte [] mybytearray = new byte [(int)file.length()];
FileInputStream fis = new FileInputStream(file);
BufferedInputStream bis = new BufferedInputStream(fis);
bis.read(mybytearray,0,mybytearray.length);
System.out.println("Sending " + file + "(" + mybytearray.length + " bytes)");
output.write(mybytearray,0,mybytearray.length);
output.flush();
System.out.println("Done.");
Client --->
// input (a DataInputstream) is set up elsewhere and messages are sent and received properly
String FILE_TO_RECEIVED = "Load_From.xml";
File file = new File(FILE_TO_RECEIVED);
int FILE_SIZE = input.readInt();
if(FILE_SIZE!=0){
// receive file
System.out.println("received file size : " + FILE_SIZE);
byte [] mybytearray = new byte [FILE_SIZE];
FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(fos);
int bytesRead = input.read(mybytearray, 0, mybytearray.length);
bos.write(mybytearray, 0, bytesRead);
You are only reading part of the data, both when reading the file and when reading the socket. The read method returns the number of bytes read, and you need a loop to read everything. For example,
int read = 0, offset = 0;
while ((read = bis.read(mybytearray, offset, mybytearray.length - offset) != -1) {
offset += read;
}
Or you can use classes from the standard library, for example DataInputStream has a readFully method.
DataInputStream dis = new DataInputStream(fis);
dis.readFully(mybytearray);
Please see the documentation of read() at: http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read(byte[],%20int,%20int).
In particular:
An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer.
This means that if the length returned is not what you expect and you haven't encountered the end of file yet, you should call read() again (in a loop).
I'm writing a custom protocol. I have a command name's put and the code looks as follows.
if(commandString.equals("PUT")){
File f = new File(currentFolder, "test.txt");
if(!f.exists())
f.createNewFile();
FileOutputStream fout = new FileOutputStream(f);
long size = 150;
long count = 0;
int bufferLeng = 0;
byte[] buffer = new byte[512];
while((bufferLeng = dis.read(buffer))>0) //dis is a data input stream.
{
count =+ bufferLeng;
fout.write(buffer);
}
System.out.println("All Good");
fout.flush();
fout.close();
}
This command gets sent to the server by the client as follows pWriter.println("PUT");. Now I run this and it does create the file test.txt but then is freezes and the server does not display the All Good message. Why would this be and what is a easy fix?
The server and the client works!!
Thank you
The server waits for the client to close the socket. This would transmit the end-of-file which would cause dis.read() to return -1.
Usually, this isn't what you want. The solution is to send the file size before the data and then read exactly this amount of data.
Make sure your client calls socket.flush() after writing the last byte of file data or the data might get stuck in a buffer which would also cause the server to hang.
Perhaps eliminate dis.read(buffer) and use the following.
if(commandString.equals("PUT")){
File f = new File(currentFolder, "test.txt");
if(!f.exists())
f.createNewFile();
FileOutputStream fout = new FileOutputStream(f);
long size = 150;
long count = 0;
byte[] buffer = new byte[512];
int bufferLeng = buffer.length;
while(count < size && bufferLeng>0) //dis is a data input stream.
{
fout.write(buffer);
count =+ bufferLeng;
}
System.out.println("All Good");
fout.flush();
fout.close();
}
This should work
In TCP i am receiving media stream from an IP camera as RAW. According to there advise, i need to write that as file. And then i can play it with media player such as VLC.
But when i write this to a file, and play with media players it never play corrupted.
After comparing the original file i see my Java writing it in wrong characters. And there sample file shows different. What or how do i fix such file writing issue, here is how i am writing it:
byte[] buf=new byte[1024];
int bytes_read = 0;
try {
bytes_read = sock.getInputStream().read(buf, 0, buf.length);
String data = new String(buf, 0, bytes_read);
System.err.println("DATA: " + bytes_read + " bytes, data=" +data);
BufferedWriter out = new BufferedWriter(
new FileWriter("capture.ogg", true));
out.write(data);
out.close();
} catch (IOException e) {
e.printStackTrace(System.err);
}
You shouldn't use Readers, Writers and Strings for binary data. Stick with InputStreams and OutputStreams.
I.e., change
BufferedWriter -> BufferedOutputStream,
FileWriter -> FileOutputStream
and instead of String, just use a byte[].
If you're dealing with sockets, I must advice you to look into the NIO package though.
You're doing it right... at least until the part where you turn your byte[] into a String:
That step only really makes sense if your byte[] represents textual data in the first place! Which it doesn't!
Whenever you handle binary data or don't actually care what the data represents you must avoid using String/Reader/Writer to handle that data. Instead do use byte[]/InputStream/OutputStream.
Also, you must read from the socket in a loop, because nothing guarantees that you've read everything:
byte[] buf=new byte[1024];
int bytes_read;
OutputStream out = new FileOutputStream("capture.ogg", true);
InputStream in = sock.getInputStream();
while ((bytes_read = in.read(buf)) != -1) {
out.write(buf, 0, bytes_read);
}
out.close();
The way you have it written limits the output file to a maximum size of 1024 bytes. Try a loop:
try {
byte[] buf = new byte[1024];
int bytes_read = 0;
InputStream in = sock.getInputStream();
FileOutputStream out = new FileOutputStream(new File("capture.ogg"));
do {
bytes_read = in.read(buf, 0, buf.length);
System.out.println("Just Read: " + bytes_read + " bytes");
if (bytes_read < 0) {
/* Handle EOF however you want */
}
if (bytes_read > 0)
out.write(buf, 0, bytes_read);
} while (bytes_read >= 0);
out.close();
} catch (IOException e) {
e.printStackTrace(System.err);
}