This is my code
URL url = new URL("http://172.16.32.160:8080/epramaan/loginotp");
URLConnection connection1 = url.openConnection();
connection1.setDoOutput(true);
ObjectOutputStream out=new ObjectOutputStream(connection1.getOutputStream());
out.writeObject(send);
out.flush();
out.close();
ObjectInputStream in = new ObjectInputStream(connection1.getInputStream());
String output=(String)in.readObject();
in.close();
//Rest of the code
Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?
I assume that by stop execution you mean block.
Just noticed that you are using readObject and not read. Please elaborate what kind of data you are reading/writing and why are u using object streams ?
As you mentioned you are using String, I would suggest to use method readFully(byte[] buf). This method blocks till all the bytes are read. Once you have the byte array, a String can be created from this byte array.
You can use InputStream.read(byte[]) for reading the entire byte array to memory (you can get the array length from the HTTP Content-Length header) and use URLConnection.setReadTimeout() for timing out if you are blocking for too long.
From the byte array you can construct your object, constructing your ObjectInputStream over a ByteArrayInputStream
Once the OutputStream writes data to the stream, will the object InputStream stop execution till the response is received?
Not precisely. Opening the InputStream doesn't block anything, and doesn't even cause the request headers to be sent. However, reading from the InputStream will do both of those things.
I suspect that the real cause of your problems is that you are getting an error response from the server that is something other than a serialized object; e.g. it could be a generic HTML error page from the server. Naturally, attempting to deserialize this fails.
The correct procedure is:
Create the URLConnection object.
Set any request headers you need to.
Connect it (or skip this ... it will happen implicitly).
Open and write to the OutputStream.
Close the OutputStream.
Use getResponseCode() to see if the request succeeded or failed.
If it succeeded, call getInputStream() and read and process the response.
If it failed, call getErrorStream() and process the error output.
Related
Consider the following code.
try {
httpURLConnection = (HttpURLConnection) new URL(strings[0]).openConnection();
httpURLConnection.setConnectTimeout(Config.HTTP_CONNECTION_TIMEOUT);
httpURLConnection.setReadTimeout(Config.HTTP_CONNECTION_TIMEOUT);
httpURLConnection.connect();
responseCode = httpURLConnection.getResponseCode();
httpURLConnection.getHeaderFields();
}
finally {
httpURLConnection.disconnect();
}
The issue is even when I don't use the InputStream to read the response, in my Internet/Wifi connection logs I can see the response-body. What I want is simply to check a field in the header and based upon that field I will continue reading the InputStream.
My questions are these:
Is it correct behavior for the connected stream to automatically download all/partial file even before a BufferedInputStream is created and read from?
If yes, then is it possible to stop the file download until an InputStream is used to read the response?
If not then is there something I am doing wrong or missing?
The response includes both the header and the body, the server does not stop for the client to acknowledge the headers before sending the body.
At the time the client is able to read the response code from the headers, a part of the body has already been sent, the size of which depends on the network latency, buffering, ....
The current implementation of HttpURLConnection.getResponseCode() even use getInputStream() to ensure that the connection is in the correct state.
The client can choose to ignore the body, but it's usually not recommended, because it may prevent a persistent connection to be reused.
I am not sure about Android but since Java 6, a background thread is automatically used to read the remaining data.
If If-Modified-Since is not an option, why not use a HEAD request ? :
The HTTP HEAD method requests the headers that are returned if the
specified resource would be requested with an HTTP GET method. Such a
request can be done before deciding to download a large resource to
save bandwidth, for example.
I am using HttpsURLConnection to call a server and return the response returned from the HttpsURLConnection from my servlet. I am copying the response from HttpssURLConnection to HttpServletresponse using streams, copying bytes from the httpconnection response input stream to the response's output stream, checking the end by seeing if read returns < 0.
Following is the code for copying the response. The variable response is of type HttpServletResponse and the variable httpCon is of type HttpsURLConnection.
InputStream responseStream = httpCon.getInputStream();
if (responseStream != null)
{
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
int len;
while ((len = responseStream.read(buffer)) >= 0)
{
os.write(buffer, 0, len);
}
os.flush();
os.close();
}
On the client side, I am using python requests library to read the response.
What I am seeing that if I use the curl to test my servlet, I am getting the proper response json, response = u'{"key":"value"}'.
If i read it from the requests python, it is putting some extra characters in the response , the response looks like the following
response = u'b0\r\n{"key":"value"}\r\n0\r\n\r\n'
Both the strings are unicode. But the second one has extra characters.
Same resonse if I try from curl/Postman restclient, I am able to get it properly. But from python requests, it is not working. I tried another livetest library in python, with that also, it is not working and the response has same characters. I also tried to change the accept-encoding header but it did not have any effect.
Because of this, I am not able to parse the json.
I don't want to change the client to parse this kind of string.
Can I change something on the server so that it will work correctly?
Did the response contain the below header "Transfer-Encoding: chunked"?
The response should be in Chunked transfer encoding
https://en.wikipedia.org/wiki/Chunked_transfer_encoding.
In this case, you get \r\n0\r\n\r\n at the end of the response is as expected since it is terminating symbol of this encoding. I guest curl/Postman just help us to handle Chunked transfer encoding, so you can't find these chunked symbols.
I have been going over the following tutorial and came across this code which I do not understand the purpose of:
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write( data );
wr.flush();
I don't know what is the purpose of the above code and where is it writing this data to...
From what I could gather, the documentation states that it converts character to bytes... but then it writes it to some where... not sure why this is.
It is basically used for turning a character stream into a byte stream.
The byte streams and character streams are incompatible for linking as the first one operates on 8-bit ASCII characters and the other on 16-bit Unicode characters. To link them explicitly, two classes exist in java.io package, InputStreamReader and OutputStreamWriter.
InputStreamReader links a byte stream, with the character stream BufferedReader (on reading-side) .
Whereas with the OutputStreamWriter, the characters of 2-bytes are encoded (converted) into bytes of 1-byte (InputStreamReader does it other way – bytes to characters).
For a Java program to interact with a server-side process it simply must be able to write to a URL, thus providing data to the server. It can do this by following these steps:
1.Create a URL.
2.Retrieve the URLConnection object.
3.Set output capability on the URLConnection.
4.Open a connection to the resource.
5.Get an output stream from the connection.
6.Write to the output stream.
7.Close the output stream.
Now in the snippet you provided ,
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
creates an output stream on the connection and opens an
OutputStreamWriter on it--Step 5 If the URL does support output, then
this method returns an output stream that is connected to the input
stream of the URL on the server side — the client's output is the
server's input.If the URL does not support output, getOutputStream
method throws an UnknownServiceException.
And
wr.write( data );
wr.close();
wr.flush();
It writes the required information to the output stream and closes the
stream. The data written to the output stream on the client side is
the input on the server side.
It's writing it to the output stream of the URLConnection - which is basically used for the body of an HTTP request (assuming it's an HTTP URL, of course).
I'm using the Apache HttpClient 4.2.3 in my application. We store the response of an HTTP call like so:
HttpResponse httpResponse = (DefaultHttpClient)httpClient.execute(httpRequest);
The response body is an InputStream in the 4.x API:
InputStream responseStream = httpResponse.getEntity().getContent();
My problem is I need to read the response body as a string and as a byte[] at various points in the application. But the InputStream used by Apache is an EofSensorInputStream, which means once I reach the stream EOF, it gets closed. Is there anyway I can get the string and byte[] representations multiple times and not close the stream?
I've already tried wrapping the byte array in a new ByteArrayInputStream and setting that as the request body, but it doesn't work since my response body can reach a few gigs. I've also tried this, but I noticed the original response stream still gets closed.
Any pointers would be welcome.
EDIT: On a related note, it would be also be great if I could find the length of the InputStream either without consuming the stream or by reversing the consumption.
1 . I think you have somewhat conflicting requirements:
a)
it doesn't work since my response body can reach a few gigs
b)
Is there anyway I can get the string and byte[] representations multiple times and not close the stream
If you do not have enough memory this is not possible.
Btw, another way to get the response as bytes is EntityUtils.byte[] toByteArray(final HttpEntity entity).
Do you really need N-gigs String? What are you going to do with it?
2 .
it would be also be great if I could find the length of the InputStream
httpResponse.getEntity().getContentLength()
3 . Since the response does not fit into the memory I would suggest to save it into a file (or temp file). Then set up InputStream on that file, and then read it as many times as you need.
Question on HttpResponse object in servlets. Can the contents of a HttpResponse be only read once?
If so do I need to user a filter and some form of "javax.servlet.http.HttpServletResponseWrapper" in order to read the content of a HttpResponse object as I need to read its content to retrieve XML/JSON from the response? At the moment Im getting the below exception when I go to read the HttpResponse object.
Content has been consumed
at org.apache.http.entity.BasicHttpEntity.getContent(BasicHttpEntity.java:84)
Thanks,
John
This is not a problem in the server/servlet side. It's a problem in the client side. The servlet doesn't send HttpServletResponse object to the client or something, it just sends a byte stream only once. You just need to read it only once into a reuseable object such as a byte[] or String, depending on the actual content and and then reuse/copy exactly this object in the remnant of the code.
InputStream input = httpResponse.getEntity().getContent();
ByteArrayOutputStream output = new ByteArrayOutputStream(); // Or some file?
IOUtils.copy(input, output);
byte[] content = output.toByteArray();
// Now you can reuse content as many times as you want.
Do you want to read the content of the response or request? Usually we write the content of the response and do not read it, unless you have an special case here.