I'm a newbie to http and Apache's HttpComponents API.
I need to process a streaming response of an http request using Apache's HttpComponents, while there may be additional http requests made on that connection. For example, a POST request is initially made to http://mystreams.net, which later follows with additional requests, while throughout I have to listen and process the streaming response. I need to hold the same initial connection I made.
How can I do that? I was able to create a simple HttpClient and do a simple HttpPost request and then process the response entity that is non-streaming, but how do I hold on to it when it continues to stream data and at the same time make new requests to the same address using the same context (ie cookies)?
Is your streaming data coming back as a single HTTP response? If so, you won't be able to receive other responses on that connection until it is done. But you could take the cookies from that response (while it is still streaming the entity to you) and use them to make other requests on another connection.
HttpEntity entity = httpclient.execute(httpget).getEntity();
InputStream is = entity.getContent()
when calling the stream, use a new Thread, and make subsequent requests in the main thread (or, better, in separate thread for reach)
Also check here
Related
In order to send a string data to the server once, I do as below:
Make “HttpURLConnection” to my URL address and open it
Set the required headers
for the connection I Set setDoOutput to True
new a DataOutputStream from my connection and finally write my string data to it.
HttpURLConnection myConn = (HttpURLConnection);
myUrl.openConnection();
myConn.setRequestProperty("Accept", "application/json, text/plain, */*");
myConn.setDoOutput(true);
DataOutputStream my_output = new DataOutputStream(myConn.getOutputStream());
my_output.write(myData.getBytes("UTF-8"));
But what to do if I want to send exactly the same data with same URl and headers multiple times?
Can I write to it multiple times?(I mean that is it possible to use the last line of code multiple times?) Or should I repeat the above steps and try it with a new connection?
And if yes should I wait for some second or millisecond before sending the next one?
I also searched for some other alternatives such as “HttpClient” Http API and making synchronous Http request which as far as I got can help me setting the headers only once.
At the end, I appreciate your help and support and any other alternatives would be welcome.
Thanks a million.
I understand that the question has be answered in the comments, but I am leaving this here so that future viewers can see it.
An HTTP request contains 3 main parts:
Request Line: Method, Path, Protocol
Headers: Key-Pairs
Body: Data
Running my_output.write() will just add bytes to the body until my_output.flush() has been executed. Flushing the stream will write the data to the server.
Because HTTP requests are usually closed by the server once all data has been sent/received, whether or not you create a new connection or just add on to the body depends on your intentions. Typically, clients will create a new connection for each request because each response should be handled individually, rather than sending a repetitive body. This will vary though because some servers choose to hold a connection (such as WebSockets).
If you are open to external libraries, you may find this chart insightful:
AsyncHttpClient would be a good fit for your intentions.
Alternatively, you can use cURL by running a terminal command with Runtime.getRuntime().exec(). More information about using cURL with POST Requests can be found here. While cURL is efficient, you have to depend on the fact that your OS supports the command (though usually all devices that can run Java have this command).
Is it necessary to read request data before calling the close method of response inside a http server in java?
I have a http server written by sun's http server. When I receive a http request, I just simply do the following steps:
write my text to response body
close the response
Does it matter to read the request body before closing the response?
Note that I have too many http requests on my server. (10000 call/sec.)
My benchmark says if request body does not have too much data, there is no big difference between two methods. I didn't get any timeout and other type of exceptions using both methods.
This is the time benchmark result for 1M request using 100 concurrent threads (10000 request per thread) on both methods:
Don't read request body: 210,581ms
Read request body: 206,599ms
I'm implementing a http client app with Netty. The task is to send the same request to several endpoints and collect the answers for further processing. Many such requests can be sent concurrently to the same endpoints. The problem is to match responses received from one endpoint to requests.
One possible solution is to create a new handler (and pipeline) for each request as describe here https://stackoverflow.com/a/7905761/4926576. In this case request can be mapped to handler and handler can store the response. But this means creating new connection for each request which will degrade the performance.
I also don't want to change the protocol and include request ids in requests/responses only for the purpose of matching.
If both the client and server are respecting HTTP pipelining semantics, then the server must respond to requests in order. Therefore, for each connection, you can maintain a queue of requests. Each new request goes on the back of the queue and each response pops it's matching request from the front of the queue.
In the event of connection failure the queue also provides you with the list of requests that were sent but for which you have not received a response. You can then take appropriate error correcting action for each request.
Is it possible to send HTTP POST request to a webserver and retrieve just headers of response or read just few bytes of the body, so the rest won't be downloaded at all (so it won't consume traffic)? If yes, how?
I know that there is a HEAD method for this, but I need to achieve it by POST method .. well, I am not sure if I need the POST method, I just need to post the data. Maybe if the webserver isn't secured well enough (it doesn't check what method it's used - it's just directly access the post data), is it possible to send "post data" by HEAD request?
There is no built-in HTTP mechanism for this, and HTTP HEAD requests do not allow content in the body. If however you are the one writing the server code then anything is possible.
If this is the case, I would suggest a URL parameter that triggers this behavior. For example:
POST /myURL - This would return the whole response
POST /myURL?body=minimal - Returns the reduced size response that you are looking for.
And you would have to code your server method to construct and return the appropriate response based on the URL parameter.
Is it possible to make GET & POST requests in Java or another language such that you don't care about what is returned?
As in just sending the requests but not wanting to receive any responses?
Whether you care about the response or not, it will be sent. The HTTP protocol specifications say that it must be.
If you don't care about the response, your client could just close the connection immediately after sending the request. But the chances are that you do want to know that the request was processed (i.e. the response status) even if you don't want to look at the contents of the response message.
So maybe you could send the request and request body, and read the response status and then close the connection without reading the response body. However, this has a downside. It means that you can't reuse the HTTP connection to make further requests. The next request to the same server has to open a new connection.
You could use anynchronous HTTP requests if you don't care about the responses (that way your worker thread will not have to wait for the response to come back). See http://www.javaworld.com/javaworld/jw-03-2008/jw-03-asynchhttp.html for some details on Asynchronous/Synchronous HTTP queries in Java. Then you can control if the anychronous thread does or does not handle the response (or any failure flagged on the communication) - as long as there were no TCP level failures on the request the connection will still be opened.
You can't control whether or not the server returns a response. Your code is free to ignore any response it receives.
It's pretty hard to not get responses because they're part of the HTTP protocol. but you can certainly ignore the responses.