I am using HttpURLConnection for making POST requests. I observe always the same behaviour during tests:
first request runs very fast (miliseconds)
all following requests take one second + some miliseconds
So something is causing 1 second delay. What can it be? The delay is happening exactly in HttpURLConnection#getInputStream().
I replaced the implementation with HttpClient - then everything is OK, no second delays (so it is not the server fault). But I really don't want to use any external dependency, so I would like to fix the HttpURLConnection thing... Any ideas?
Below current implementation. I tried some tips from stackoverflow (adding headers to the request), but with no success.
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-Length", ""
+ (body == null ? 0 : body.length));
// Send post request
con.setDoOutput(true);
OutputStream wr = con.getOutputStream();
if (body != null) {
wr.write(body);
}
wr.flush();
wr.close();
BufferedReader rd = new BufferedReader(new InputStreamReader(
con.getInputStream()));
String line;
String result = "";
while ((line = rd.readLine()) != null) {
result += line;
}
rd.close();
con.disconnect();
return result;
PS: It is about jse, not android.
You're never closing the input stream from the connection - that may mean that the connection isn't eligible for pooling, and on the next attempt it's waiting for up to a second to see if the previous request's connection will become eligible for pooling.
At the very least, it would be a good idea to close the stream. Use a try-with-resources block if you're using Java 7 - and ditto for the writer.
As an aside, I suggest you explicitly state the encoding you expect when reading - or use a higher-level library which detects that automatically based on headers.
Related
I want to commit text file "demo2.txt" to bitbucket server using rest API. I can upload the same file using Postman but it's not working with Java code. As shown in the below code I want to send string object "str" as the body. Can someone help me here to upload the file on the bitbucket server? Also Please let me know if there is any other way to do this.
URL url = new URL("https://api.bitbucket.org/2.0/repositories/{team name}/{repository name}/src");
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
httpCon.setRequestProperty("X-Requested-with", "Curl");
httpCon.setDoOutput(true);
httpCon.setDoInput(true);
httpCon.setRequestProperty("Connection", "Keep-Alive");
httpCon.setRequestProperty("Content-Type", "multipart/form-data; boundary="+boundary);
httpCon.setRequestProperty("Accept", "application/x-www-form-urlencoded");
httpCon.setRequestProperty("Authorization", basicauth);
httpCon.setRequestMethod("POST");
String str =
"{"
+ "\"-F\":\"File3=#/D:/log/demo2.txt\" "
+ "}";
try {
OutputStream output = httpCon.getOutputStream();
output.write(str.getBytes());
output.close();
} catch(Exception e){
System.out.println(e.getMessage());
}
int responseCode = httpCon.getResponseCode();
String inputLine;
StringBuffer response = new StringBuffer();
if (responseCode == HttpURLConnection.HTTP_OK || responseCode == HttpURLConnection.HTTP_CREATED){
BufferedReader in = new BufferedReader(new .
InputStreamReader(httpCon.getInputStream()));
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
List<String> message = new ArrayList<>();
message.add(response.toString());
}
If this is all of your code, then your problem may be as simple as the fact that you're not making any sort of call to finalize the request...to tell HttpURLConnection that you're done forming the request and want it to complete. There are two things you can do to help this:
close the output stream when you're done writing to it. You're generally supposed to do this. Here, you can call output.close(). Better still, since you have a try/catch block already anyway, use a "try with resources" construct to make sure that the stream is closed no matter what happens (assuming you're using a newer version of Java that supports this).
make some sort of call to query the response to the request. It may
be that the request is not being fully sent until you do this. Try
calling httpCon.getResponseCode() at the bottom of your code.
Given that you have provided no information as to what "it's not working with Java code" means, this may be useful information but not the ultimate solution to your problem. Your code does look good other than exhibiting these omissions.
I'm trying to proceed the JIRA transitions by java coding, most of the time it works, but the jira rest api call sometimes return the below error: (Actually with this error, the transition is processed)
java.io.IOException: Server returned HTTP response code: 400 for URL: http://testingSite/jira/rest/api/latest/issue/ABC-123/transitions
Also, there are some case the rest api call do not return errors but the transition is not proceed.
Here is my coding, most of the time it works, so it ruined my days to figure out what is happening.
try {
String authkey = "YWRtaW46cGFzc3dvcmQ=";
URL url = new URL("http://testingSite/jira/rest/api/latest/issue/ABC-123/transitions");
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setDoOutput(true);
connection.setRequestProperty("Authorization", "Basic " + authkey);
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "application/json;charset=UTF-8");
OutputStream os = connection.getOutputStream();
String data = "{\"transition\": {\"id\": \"71\"}}";
os.write(data.toString().getBytes("UTF-8"));
os.close();
content = connection.getInputStream();
in = new InputStreamReader(content);
br = new BufferedReader(in);
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
} catch (Exception e) {
e.printStackTrace();
}
The error comes from this line:
content = connection.getInputStream();
I expect there is not exception and all transition is processed but the behaviour is quite strange for me.
Behaviour 1 : Server returned HTTP response code : 400 but the transition is processed
Behaviour 2 : Server do not return any error but the transition is not processed
So I was looking for reference documentation here. And its stated that
POST: 400 - If there is no transition specified.
You have you transition id hardcoded and maybe for that type of issue it has different transition id or something similar. Try to call
GET /rest/api/2/issue/{issueIdOrKey}/transitions?{transitionId}
to verify you transition is actually there.
I cannot comprehend why doesn't the following code does not put a packet onto wire (confirmed via wireshark). It is a fairly standard method of sending an HTTP POST request, as I believe. I don't intend to read anything just POST.
private void sendRequest() throws IOException {
String params = "param=value";
URL url = new URL(otherUrl.toString());
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setDoOutput(true);
con.setDoInput(true); //setting this to `false` does not help
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "text/plain");
con.setRequestProperty("Content-Length", "" + Integer.toString(params.getBytes().length));
con.setRequestProperty("Accept", "text/plain");
con.setUseCaches(false);
con.connect();
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(params);
wr.flush();
wr.close();
//Logger.getLogger("log").info("URL: "+url+", response: "+con.getResponseCode());
con.disconnect();
}
What happens is... actually nothing, unless I try to read anything. For example by uncommenting the above log line which reads the response code. Trying to read a response via con.getInputStream(); also works. There is no movement of packets. When I uncomment the getResponseCode, I can see that http POST is sent, and then 200 OK is sent back. The order is proper. I.e. I don't get some wild response before sending POST. Everything else looks exactly the same (I can attach wireshark screenshots if needed.). In the debugger the code executes (i.e. does not block anywhere).
I don't understand under what circumstances this can be happening. I belive it should be possible, to send a POST request with con.setDoInput(false);. Currently it doesn't send anything or fails (when trying to execute con.getResponseCode()) with an exception because I obviously promised I won't read anything.
It might be relevant, that before sendRequest I do request some data from the same site, but I trust I close everything properly. I.e:
public static String getData(String urlAddress) throws MalformedURLException, IOException {
URL url = new URL(urlAddress);
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setDoOutput(false);
InputStream in = con.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder data = new StringBuilder();
String line;
while((line = reader.readLine()) != null) {
data.append(line);
}
reader.close();
in.close();
con.getResponseCode();
con.disconnect();
return data.toString();
}
The server for url in both cases is the same, port also, so I believe it is possible to use the same socket for communication. The above code works and retrieves the data properly.
I am not sure, maybe I don't clean something, and it gets cached, so with out an explicit read the POST gets delayed. There is no other traffic on the socket.
Unless you're using fixed-length or chunked transfer mode, HttpURLConnection will buffer all your output until you call getInputStream() or getResponseCode(), so that it can send a correct Content-length header.
If you call getResponseCode() you should have a look at its value.
This question already has answers here:
Why does LogCat not show the full message?
(2 answers)
Closed 8 years ago.
I work with HttpUrlConnection in my App and in my common Java Test and I implemented a method and that Method (common for both of them, so, identical!!!) behaves in Android case in another way.
Both of them can right receive an identical response from Server but in Java Test I can show this response while in Android App is chunked to 3200 Chars.
That's my Code
private String sendPost() throws Exception{
String url = "http://www.something.com/my_page.jsp?";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
String urlParameters ="param1=val1¶m2=val2";
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
wr.writeBytes(urlParameters);
wr.flush();
wr.close();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// return result
Log.i("TAG", "sendPost:: Response length : " + response.length()); // <- This line returns the same length!!!
return response.toString();
}
All I can get of this object con from Class HttpUrlConnection like ContentLength, ContentType, etc is the same in both of these cases, therefore I suspect, there must be an intern Setting/Parameter of String/StringBuffer in Android, which distinguishes these case but I don't know what. readLine reads the same or at least the same number of chars cause the length of response is the same in both of cases.
If you could say me, what is wrong, I'd be very thankful.
Kind Regards
I can't understand your description of the symptoms; i.e. why you think that something is being truncated.
However, I can assure you that it is NOT due to a limit on the length of String or StringBuffer.
Those two classes do have a limit, but it is 2**31 (i.e. >2 billion) characters. You will typically get an OutOfMemoryError before your buffer gets that big.
We are using HttpURLConnection API to invoke a REST API to the same provider often (kind of an aggregation usecase). We want to keep a pool of 5 connections always open to the provider host (always the same IP).
What is the proper solution? Here is what we tried:
System.setProperty("http.maxConnections", 5); // set globally only once
...
// everytime we need a connection, we use the following
HttpURLConnection conn = (HttpURLConnection) (new URL(url)).openConnection();
conn.setRequestMethod("GET");
conn.setDoInput(true);
conn.setDoOutput(false);
conn.setUseCaches(true);
...
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
...
At this point we read the input stream until the BufferedReader returns no more bytes. What do we do after that point if we want to reuse the underlying connection to the provider? We were under the impression that if the input stream is completely read, the connection is then added back to the pool.
It's been working for several weeks this way, but today it stopped working producing this exception: java.net.SocketException: Too many open files
We found numerous sockets in the CLOSE_WAIT state like this (by running lsof):
java 1814 root 97u IPv6 844702 TCP colinux:58517->123.123.254.205:www (CLOSE_WAIT)
Won't either conn.getInputStream().close() or conn.disconnect() completely close the connection and remove it from the pool?
We had this problem also on Java 5 and our solution is to switch to Apache HttpClient with pooled connection manager.
The keepalive implementation of Sun's URL handler for HTTP is very buggy. There is no maintenance thread to close idle connections.
Another bigger problem with keepalive is that you need to delete responses. Otherwise, the connection will be orphaned also. Most people don't handle error stream correctly. Please see my answer to this question for an example on how to read error responses correctly,
HttpURLConnection.getResponseCode() returns -1 on second invocation
From here:
The current implementation doesn't buffer the response body. Which means that the application has to finish reading the response body or call close() to abandon the rest of the response body, in order for that connection to be reused. Furthermore, current implementation will not try block-reading when cleaning up the connection, meaning if the whole response body is not available, the connection will not be reused.
I read this as if your solution should work, but that you are also free to call close and the connection will still be reused.
The reference cited by disown was what really helped.
We know Apache HttpClient is better, but that would require another jar and we might use this code in an applet.
Calling HttpURLConnection.connect() was unnecessary. I'm not sure if it prevents connection reuse, but we took it out. It is safe to close the stream, but calling disconnect() on the connection will prevent reuse. Also, setting sun.net.http.errorstream.enableBuffering=true helps.
Here is what we ended up using:
System.setProperty("http.maxConnections", String.valueOf(CONST.CONNECTION_LIMIT));
System.setProperty("sun.net.http.errorstream.enableBuffering", "true");
...
int responseCode = -1;
HttpURLConnection conn = null;
BufferedReader reader = null;
try {
conn = (HttpURLConnection) (new URL(url)).openConnection();
conn.setRequestProperty("Accept-Encoding", "gzip");
// this blocks until the connection responds
InputStream in = new GZIPInputStream(conn.getInputStream());
reader = new BufferedReader(new InputStreamReader(in));
StringBuffer sb = new StringBuffer();
char[] buff = new char[CONST.HTTP_BUFFER_SIZE];
int cnt;
while((cnt = reader.read(buff)) > 0) sb.append(buff, 0, cnt);
reader.close();
responseCode = conn.getResponseCode();
if(responseCode != HttpURLConnection.HTTP_OK) throw new IOException("abnormal HTTP response code:"+responseCode);
return sb.toString();
} catch(IOException e) {
// consume error stream, otherwise, connection won't be reused
if(conn != null) {
try {
InputStream in = ((HttpURLConnection)conn).getErrorStream();
in.close();
if(reader != null) reader.close();
} catch(IOException ex) {
log.fine(ex);
}
}
// log exception
String rc = (responseCode == -1) ? "unknown" : ""+responseCode;
log.severe("Error for HttpUtil.httpGet("+url+")\nServer returned an HTTP response code of '"+rc+"'");
log.severe(e);
}