I found this article about simple proxy server implemented in JAVA:
http://www.java2s.com/Code/Java/Network-Protocol/Asimpleproxyserver.htm
The code simply gets some stream from the client, after sends it to the server and after it gets stream from the server and sends the response to the client. What I would like to do is to compress this streams before it is sent and decompress after it is received.
I found the class GZIPInputStream but I'm not sure how to use it and what I found on internet didn't help me. I either didn't understand that so much or it was not a good solution for me.
My idea is too that but I'm not sure if its ok:
final InputStream streamFromClient = client.getInputStream();
final OutputStream streamToClient = client.getOutputStream();
final InputStream streamFromServer = server.getInputStream();
final OutputStream streamToServer = server.getOutputStream();
InputStream gzipStream = new GZIPInputStream(streamFromClient );
try
{
while ((bytesRead = gzipStream.read(request)) != -1)
{
streamToServer.write(request, 0, bytesRead);
streamToServer.flush();
}
}
catch (Exception e) {
System.out.println(e);
}
Now the data sent to the server should be compressed before sending (but I'm not sure if it's a correct solution). IS IT?
Now imagine the server sends me the compressed data.
So this stream:
final InputStream streamFromServer = server.getInputStream();
is compressed.
How can I decompress it and write to the
final OutputStream streamToClient = client.getOutputStream();
Thanks for the help, guys!
Read the javadoc of these streams : http://download.oracle.com/javase/6/docs/api/java/util/zip/GZIPInputStream.html and http://download.oracle.com/javase/6/docs/api/java/util/zip/GZIPOutputStream.html.
GZIPOutputStream compresses the bytes you write into it before sending them to the wrapped output stream. GZIPInputStream reads compressed bytes from the wrapped stream and returns uncompressed bytes.
So, if you want to send compressed bytes to anyone, you must write to a GZIPOutputStream. But of course, this will only work if the receiving end knows it and decompresses the bytes it receives.
Similarly, if you want to read compressed bytes, you need to read them from a GZIPInputSTream. But of course, it'll only work if the bytes are indeed compressed using the same algorithm by the sending end.
Related
Beginner in java, I try to decompress an HTTP response in Gzip format. Roughly, I have a bufferReader which allows me to read lines of http response from a socket. Thanks to that, I parse the http header and if it specifies that the body is in gzip format then I have to decompress it. Here is the code which I use:
DataInputStream response = new DataInputStream(clientSideSocket.getInputStream());
BufferedReader buffer = new BufferedReader(new InputStreamReader(response))
header = parseHTTPHeader(buffer); // return a map<String,String> with header options
StringBuilder SBresponseBody = new StringBuilder();
String responseBody = new String();
String line;
while((line = buffer.readLine())!= null) // extract the body as if was a string...
SBresponseBody.append(line);
responseBody = SBresponseBody.toString();
if (header.get("Content-Encoding").contains("gzip"))
responseBody = unzip(responseBody); // function I try to construct
My attempt for the unzip function is as follows:
private String unzip(String body) throws IOException {
String responseBody = "";
byte[] readBuffer = new byte[5000];
GZIPInputStream gzip = new GZIPInputStream (new ByteArrayInputStream(body.getBytes());
int read = gzip.read(readBuffer,0,readBuffer.length);
gzip.close();
byte[] result = Arrays.copyOf(readBuffer, read);
responseBody = new String(result, "UTF-8");
return responseBody;
}
I get an error in the GZIPInputStream: not GZIP format (because gzip header is not found in body).
Here are my thoughts:
• Is body.toByte() wrong since it has been read by a bufferReader as a character string and therefore converting it back to byte[] makes no sense since it has already been interpreted in the wrong way? Or do I reconvert Sting body to byte[] in the wrong way?
• Do I have to build a GZIP header myself using the information provided in the HTTP header and adding it to the String body ?
• Do I need to create another InputStream from my socket.getInputStream() to read the information byte by byte, or is it tricky since there is already a buffer "connected" to this socket?
Roughly, I have a bufferReader which allows me to read lines of http response from a socket.
You've handrolled a HTTP client.
This is not a good thing; HTTP is considerably more complicated than you think it is. gzip is just one of about 10,000 things you need to think about. There's HTTP/2.0, Spdy, http3, chunked transfer encoding, TLS, redirects, mime packing, and so much more to think about.
So, if you want to write an actual HTTP client, you need about 100x this code and a ton of domain knowledge, because the actual specs of the HTTP protocol, while handy, don't really tell the story. The de-facto protocol you're implementing is 'whatever servers connected to the internet tend to send' and what they tend to send is tightly wound up with 'whatever commonly used browsers tend to get right', which is almost, but not quite, what that spec document says. This is one of those cases where pragmatics and implementations are the 'real spec', and the actual spec is merely attempting to document reality.
That's a long way around to say: Your mistake is trying to handroll a HTTP client. Don't do that. Use OkHttp or the http client introduced in jdk11 in the core libraries.
But, I know what I want!
Your code is loaded up with bugs, though.
DataInputStream response = new DataInputStream(clientSideSocket.getInputStream());
DataInputStream is useless here. Remove that wrapper.
BufferedReader buffer = new BufferedReader(new InputStreamReader(response))
Missing semi-colon. Also, this is broken - this will convert the bytes flowing over the wire to characters using 'platform default encoding' which is wrong, you need to look at the Content-Type header.
responseBody = unzip(responseBody)
You cannot do this. Your major misunderstanding is that you appear to think that there is no difference between a bunch of bytes, and a sequence of characters.
That's wrong. Once you stored bytes into chars, you cannot unzip it anymore.
The fix is to check for the gzip header FIRST, then wrap your inputstream through GZipStream.
I send some data to server I use httpURLConnection and DataOutputStream.
DataOutputStream outputStream = new DataOutputStream(httpURLConnection.getOutputStream());
outputStream.write(data);
outputStream.flush();
outputStream.close();
Is there a way to check if i have sent all bytes?
I mean there is something like getContentLength() but it return -1 so i guess it contains only a length which i set in header. i have to be sure if i sent e.g. 500/500 bytes(not only 480/500 bytes). so is there a way to check it? or all i can do is valid data at server side and send response?
I am sending MultiPart content to my remote server to store it in filesystem. For this I am using Java TCP/IP protocol. For avoiding network bandwidth and TCP Input / Output buffer memory , I am sending the data in GZIP compressed format. But , I cannot decompress the data received from the client. I got Unexpected end of ZLIB input stream Exception. Its due to the server is receiving data in chunks.
Java Code
Client
OutputStream out = new GZIPOutputStream(sock.getOutputStream());
byte[] dataToSend = FileUtil.readFile(new File("/Users/bharathi/Downloads/programming_in_go.pdf"));
out.write(dataToSend);
Server
out = new FileOutputStream("/Users/bharathi/Documents/request_trace.log");
InputStream in = new GZIPInputStream(clntSocket.getInputStream());
int totalBytesRead = 0;
int bytesRead;
byte[] buffer = new byte[BUFFER_SIZE];
while ((bytesRead = in.read(buffer)) != -1)
{
out.write(buffer , 0 , bytesRead);
totalBytesRead += bytesRead;
}
Is there any solution to send the data in GZIP compressed format in Socket?
GZIPOutputStream generates a GZIP file format, meaning that the other end has to receive the complete stream (which is a file) before it can process it, this is the reason for your error.
If you are looking to actually do a stream based data transfer, drop gzip, and go for zlib, I believe Zlib compression Using Deflate and Inflate classes in Java answers how to do this.
Try adding:
out.flush();
sock.shutdownOutput();
to your client code.
This is my code
URL url = new URL("http://172.16.32.160:8080/epramaan/loginotp");
URLConnection connection1 = url.openConnection();
connection1.setDoOutput(true);
ObjectOutputStream out=new ObjectOutputStream(connection1.getOutputStream());
out.writeObject(send);
out.flush();
out.close();
ObjectInputStream in = new ObjectInputStream(connection1.getInputStream());
String output=(String)in.readObject();
in.close();
//Rest of the code
Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?
I assume that by stop execution you mean block.
Just noticed that you are using readObject and not read. Please elaborate what kind of data you are reading/writing and why are u using object streams ?
As you mentioned you are using String, I would suggest to use method readFully(byte[] buf). This method blocks till all the bytes are read. Once you have the byte array, a String can be created from this byte array.
You can use InputStream.read(byte[]) for reading the entire byte array to memory (you can get the array length from the HTTP Content-Length header) and use URLConnection.setReadTimeout() for timing out if you are blocking for too long.
From the byte array you can construct your object, constructing your ObjectInputStream over a ByteArrayInputStream
Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?
Not precisely. Opening the InputStream doesn't block anything, and doesn't even cause the request headers to be sent. However, reading from the InputStream will do both of those things.
I suspect that the real cause of your problems is that you are getting an error response from the server that is something other than a serialized object; e.g. it could be a generic HTML error page from the server. Naturally, attempting to deserialize this fails.
The correct procedure is:
Create the URLConnection object.
Set any request headers you need to.
Connect it (or skip this ... it will happen implicitly).
Open and write to the OutputStream.
Close the OutputStream.
Use getResponseCode() to see if the request succeeded or failed.
If it succeeded, call getInputStream() and read and process the response.
If it failed, call getErrorStream() and process the error output.
I am using the official Dropbox API for Java.
So far, everything works smoothly. Authentication via oauth works and so do other functions (like directory listings).
Now, I tried to upload a file like this:
InputStream is = getInputStream();
byte[] bytes = is2Bytes(is); // Gets all bytes "behind" the stream
int len = bytes.length;
api.putFileOverwrite(path, is, len, null);
Now, when I do this call, my application hangs for about 15 seconds and then I get an exception thrown that Dropbox server did not respond.
So, first I asked Dropbox support if there was something wrong with their server. There isn't.
Then, I played around with the parameters of the putFileOverwrite method and I found out that if I set len=0 manually, the server responds and creates a 0 byte file with the correct file name.
As another test, I manually entered the value len=100 (the original file has 250KB so that should be ok). Again, the server does NOT respond.
So, what's wrong?
That is not weird at all. Since you use your self-made method is2Bytes, the steam is empty, because you read all the bytes to count them. The proper way of doing this would be either knowing how many bytes you are going to send or using the build-in method for sending a file.
public HttpResponse putFile(String root, String dbPath, File localFile)
Very weird. I was able to work around this by re-creating a new InputStream from the byte array and send that to Dropbox:
InputStream is = getInputStream();
byte[] bytes = is2Bytes(is); // Gets all bytes "behind" the stream
int len = bytes.length;
ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
api.putFileOverwrite(path, bis, len, null);