I'm attempting to encode a String in a client using GZIPOutputStream then decoding the String in a server using GZIPOutputStream.
The client's side code (after the initial socket connection establishment) is:
// ... Establishing connection, getting a socket object.
// ... Now proceeding to send data using that socket:
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
String message = "Hello World!";
ByteArrayOutputStream out = new ByteArrayOutputStream();
GZIPOutputStream gzip = new GZIPOutputStream(out);
gzip.write(message);
gzip.close();
String encMessage = out.toString();
out.writeInt(encMessage.getBytes().length);
out.write(encMessage.getBytes());
out.flush();
And the server's side code (again, after establishing a connection):
DataInputStream input = new DataInputStream(socket.getInputStream());
int length = input.readInt();
byte[] buffer = new byte[length];
input.readFully(buffer);
GZIPInputStream gz = new GZIPInputStream(new ByteArrayInputStream(buffer));
BufferedReader r = new BufferedReader(new InputStreamReader(gz));
String s = "";
String line;
while ((line = r.readLine()) != null)
{
s += line;
}
I checked and the buffer length (i.e., the coded message's size) is passed correctly, so the right number of bytes is transferred.
However, I'm getting this:
java.util.zip.ZipException: invalid code lengths set
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:117)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:122)
at parsing.ReceiveResponsesTest$TestReceiver.run(ReceiveResponsesTest.java:147)
at java.lang.Thread.run(Thread.java:745)
Any ideas?
Thanks in advance for any assistance!
You're calling toString() on the ByteArrayOutputStream - that is incorrect, and it opens up all kinds of character encoding problems that are probably biting you here. You need to call toByteArray instead:
byte[] encMessage = out.toByteArray();
out.writeInt(encMessage.length);
out.write(encMessage);
Detail:
if you use toString(), Java will encode your bytes in your platform default character encoding. That could be some Windows codepage, UTF-8, or whatnot.
However not all characters can be encoded properly, and some will be replaced by an alternative character - a question mark perhaps. Without knowing the details, it's hard to tell.
But in any case, encoding the byte array to a String, and then decoding it to a byte array again when you write it out, is very likely to change the data in the byte array. And there is not need to do it, you can just get the byte array straight away as shown in the code above.
Why on earth are you indulging in all this complication? You can reduce it all to this:
GZIPOutputStream gzip = new GZIPOutputStream(socket.getOutputStream());
DataOutputStream out = new DataOutputStream(gzip);
String message = "Hello World!";
out.writeUTF(message);
out.close();
// ...
GZIPInputStream gz = new GZIPInputStream(new ByteArrayInputStream(socket.getInputStream()));
DataInputStream input = new DataInputStream(gz);
String line = input.readUTF();
I further note that your code doesn't actually compile. I would further note that unless the messages are several orders of magnitude larger, there is no benefit to the GZipping.
Related
I receive data from Bluetooth BLE as follow in 250 bytes chunk at most.
onDataReceived(byte[] data) {
my_readline(); // <-- how could I implement this
}
the data are string but chuncked. so what is the proper way to detect lines from incomming byte arrays. or it would be good as well if it is possible to convert received data to inputStream as well.
You could do it with an ByteArrayInputStream and turn that into an BufferedReader. It is not very clean altough it should work.
InputStream inputStream = new ByteArrayInputStream(data);
BufferedReader buffReader = new BufferedReader(new InputStreamReader(inputStream));
You can convert byte array to String,
String inputStr = new String(data, "UTF-8");
And if you want to convert to InputStream,
InputStream dataInputStream = new ByteArrayInputStream(data);
i need to read an Base64 encoded array of bytes from an inputstream.
I have to stop reading when I reach a \n character in the decoded string, i cannot find an efficient way to do this.
Now i ended with something like this but it does not work as intended because it's too easy it catches an exception and messes all up...
byte buffer[] = new byte[2048];
byte results[] = new byte[2048];
String totalResult = null;
try {
int bytes_read = is.read(buffer);
while (buffer[bytes_read - 1] != '\n') {
bytes_read = is.read(buffer);
String app = new String(buffer, 0, bytes_read);
totalResult += app;
}
String response = Base64.getDecoder().decode(totalResult).toString();
Any idea? The input Stream does not close, so i need to get data from it and separated by '\n'
Rather than reinventing the wheel, consider using (for example) org.apache.commons.codec.binary.Base64InputStream from the Commons Codec project and a BufferedReader (JavaDoc) to wrap your InputStream like so:
try(BufferedReader reader = new BufferedReader(new InputStreamReader(new Base64InputStream(is)))) {
String response = reader.readLine();
...
}
Notes:
Try with resources will automatically close the reader when you're done.
The Base64InputStream will decode Base64 encoded characters on the fly
BufferedReader.readLine()considers \n, \r or \r\n to be line separators for the purpose of determining the end of a line.
I am sure other libraries exist that will facilitate the on-the-fly decoding, or you could write a simple InputStreamWrapper yourself.
I have a problem with sending large string through socket from server to android client.
String is about 10MB.
Code for writing data to socket is this:
int socketTimeout = 200;
socket = new Socket(client.getHost(), client.getPort());
socket.setSoTimeout(socketTimeout);
OutputStreamWriter oos=new OutputStreamWriter(socket.getOutputStream());
String d = data.getData().toString() + "\n";
oos.write(d);
oos.flush();
Code for reading data from socket is this:
Socket s = params[0];
InputStream is = null;
try {
is = s.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[32768];
while ((nRead = is.read(data, 0, data.length)) != -1) {
baos.write(data, 0, nRead);
}
return new String(baos.toByteArray());
}
So problem comes at line where I'm reading from inputStream where I get OutOfMemoryException. I tried using different examples of reading string from stream. I tried with BufferedInputStream, InputStreamReader, IOUtils, StringBuilder, BufferedReader ..etc. and all of them give me OutOfMemory exception when the string is large. I tested with smaller data something around 100K and it works perfectly.
Exception that I get on server-side is "Connection is reset by peer, socket write error."
You can read byte by byte in the client and write to a File byte by byte, in that way you are not holding the whole string in memory.
And then of course read that file by tokens or lines, not the whole string at once
By storing it in a ByteArrayOutputStream (or similar), you are coming up against the maximum heap size for the JVM in Android. This is a different size depending on the device. See: Android heap size on different phones/devices and OS versions
As has already been suggested, you should consider using a file stream to write the received data to disk.
i am developing an android application wherein i have to send a frame in jpeg format allocated to a BufferedArrayOutputStream (baos variable in code). I convert this baos into a byte array to write into the socket.
On the server side i would like to reconstruct the image in jpeg format. If i write the data received in a variable to a '.jpg' file on the server, on opening the file, it gives an error like "file starting with ffx0 not jpeg format". I think this is because the string variable in python writes the data in the file as a hex string.
The client code is as follows :-
Bitmap memoryImage = Bitmap.createBitmap(rgb, previewSize.width,previewSize.height,Bitmap.Config.ARGB_8888);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
if(memoryImage.compress(CompressFormat.JPEG,100, baos)){
try {
if(count==0){
byte [] Finalbaos = baos.toByteArray();
int tempLen = Finalbaos.length;
Log.v("Client","ImageBytes :"+ tempLen);
String dataMeta = Integer.toString(tempLen);
Log.v("Client","Integer Size :"+ dataMeta.length());
PrintWriter tempOut = new PrintWriter(socket.getOutputStream());
if(tempOut!=null){
tempOut.write(dataMeta);
Log.v("Client","data size sent");
tempOut.flush();
}
DataInputStream in = new DataInputStream(socket.getInputStream());
if(in!=null){
Log.v("Client","read buffer created");
String xyz = in.readLine();
String temp = "recvd";
Log.v("Client",xyz);
if(xyz.equals(temp)){
OutputStream out = socket.getOutputStream();
out.write(Finalbaos,0,tempLen);
out.flush();
Log.d("Client", "Client sent message");
}
}
server code:
import socket,thread
import string
import array
host=""
port=54321
s=socket.socket()
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind((host,port))
s.listen(5)
conn,address=s.accept()
data=""
mylen=0
dataRecv=0
file1 = open("myfile.jpg","w")
totalLength=""
length=conn.recv(1024)
conn.send("recvd")
mylen=int(length)
while dataRecv<mylen:
newData=""
newData=conn.recv(1)
if not newData:
break
data+=newData
dataRecv+=len(newData)
result= array.array('B',data.decode("hex"))
file1.write(result)
file1.close()
print len(data)
conn.close()
s.close()
can anyone let me know how to reconstruct the frame on server either in python or C++
mylen=len(length) doesn't give you the length you're trying to send. it gives you how many bytes were read in the previsous recv. So you get the wrong lenght there.
on your client side, you use String xyz = in.readLine(); which will block until a newline character is encountered. but you never send a '\n' on the server side, instead you go waiting for a response from the client. so you have a deadlock there.
you use data.decode("hex") on your recieved data. unless you do the equivalend of data.encode("hex") in java on the other side, that won't work. it should give you an error if the string is not a valid hex-representation of a binary string.
result is an array.array, which you write to file. file1.write expects a string as argument, it gives you an error if you pass your result object.
so i can't even see why your code works at all, and why there's anything at all in your file.
I am writing a byte array from a socket client as:
byte[] arr = { 49, 49, 49, 49, 49};
InetAddress address = InetAddress.getByName(host);
connection = new Socket(address, port);
out = new ObjectOutputStream(connection.getOutputStream());
out.flush();
At receiving end, on the server I have:
byte[] msgType = new byte[5];
in = new BufferedInputStream(connection.getInputStream());
int bytesRead = in.read(msgType, 0, 5);
System.out.println("msg rcvd: " + msgType.toString());
In the output I get weird string:
server: waiting for connection
server: connection received from localhost
server: Connection Successful
bytes read: 5
msg rcvd: ��w
How can make sure I get same bytes as I sent from my client?
I'm not sure exactly what are trying to print out, but I can tell you that msgType.toString() will not print the contents of the byte array.
Here is a link I found to a method which will print out the byte array in a more meaningful fashion.
You're getting the same bytes, it's just a matter of how you interpret them. If you want to see the bytes as a String use this instead:
System.out.println("msg rcvd: " + new String(msgType, "UTF-8"));
Be careful that the bytes you're dealing have the right encoding though (I assumed UTF-8 here). Since you're already ObjectOutputStream though, you could just use its writeUTF() on the server side and ObjectInputStream.readUTF() on the client side.
If you use an ObjectOutputStream on one side, you'll have to use an ObjectInputStream at the other side.
In your case, a simple OutputStream (maybe buffered) and .write() and .read() will do.
But for printing, don't use byte[].toString(), use Arrays.toString() if you want to have a formatted output.
Edit: I just see you are not even writing your array on the sending side. So you are actually only reading the ObjectOutputStream header.
From the comment:
I am handling server side and I am told that I would be send a byte array. How
do I receive and print that byte array ? byte array in this case is bytes of text/strings
This sounds like the server sends something like simply the Strings encoded in some encoding, like ASCII, UTF-8 or ISO-8859-1. If so, on the receiving end you can use something like this:
String encoding = "UTF-8";
BufferedReader in =
new BufferedReader(new InputStreamReader(connection.getInputStream(),
encoding));
String line;
while((line = in.readLine()) != null) {
System.out.println(line);
}
Of course, make sure the encoding is the same encoding as actually used on the sending side.
The corresponding sending code could be something like this:
String encoding = "UTF-8";
Writer w =
new BufferedWriter(new OutputStreamWriter(connection.getOutputStream(),
encoding));
w.write("Hello World!\n");
w.write("And another line.\n");
w.flush();