Develop Reference

URLEncoder mangling my JSON

I am trying to make a REST call from an Android app, and I seem to be having trouble getting the proper data to the server. The node side of things is getting queries like the following:
{ '{"search_term":"eese"}': '' }
The sending code from the app is:
OutputStream out = urlConnection.getOutputStream();
String body = r.getJSONString();
String body_encoded = URLEncoder.encode(body, "UTF-8");
out.write(body_encoded.getBytes("UTF-8"));
out.close();
If I step through the above code, I see that body is
"{"search_term":"eese"}"
And advice?

It seems my issue arose from an incorrect Content-Type.
I added
urlConnection.setRequestProperty("Content-Type", "application/JSON");
and the server started getting the correctly formatted items.

Invoking a 'REST' service which have query parameters in the URL

I have to invoke a GET on a service which returns text/xml.
The endpoint is something like this:
http://service.com/rest.asp?param1=34&param2=88&param3=foo
When I hit this url directly on a browser (or some UI tool), all's good. I get a response.
Now, I am trying to use CXF WebClient to fetch the result using a piece of code like this:
String path = "rest.asp?param1=34&param2=88&param3=foo";
webClient.path(path)
.type(MediaType.APPLICATION_JSON)
.accept(MediaType.TEXT_XML_TYPE)
.get(Response.class);
I was debugging the code and found that the request being sent was url encoded which appears something like this:
http://service.com/rest.asp%3Fparam1=34%26param2=88%26param3=foo
Now, the problem is the server doesn't seem to understand this request with encoded stuff. It throws a 404. Hitting this encoded url on the browser also results in a 404.
What should I do to be able to get a response successfully (or not let the WebClient encode the url)?

Specify the parameters using the query method:
String path = "rest.asp";
webClient.path(path)
.type(MediaType.APPLICATION_JSON)
.accept(MediaType.TEXT_XML_TYPE)
.query("param1","34")
.query("param2","88")
.query("param3","foo")
.get(Response.class);

You will need to encode your URL. You can do it with the URLEncoder class as shown below:
Please replace your line
String path = "rest.asp?param1=34&param2=88&param3=foo";
with
String path = URLEncoder.encode("rest.asp?param1=34&param2=88&param3=foo");

Getting "java.net.ProtocolException: Server redirected too many times" Error

I'm making a simple URL request with code like this:
URL url = new URL(webpage);
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
But on that last line, I'm getting the "redirected too many times error". If my "webpage" var is, say, google.com then it works fine, but when I try to use my servlet's URL then it fails. It seems I can adjust the number of times it follows the redirects (default is 20) with this:
System.setProperty("http.maxRedirects", "100");
But when I crank it up to, say, 100 it definitely takes longer to throw the error so I know it is trying. However, the URL to my servlet works fine in (any) browser and using the "persist" option in firebug it seems to only be redirecting once.
A bit more info on my servlet ... it is running in tomcat and fronted by apache using 'mod-proxy-ajp'. Also of note, it is using form authentication so any URL you enter should redirect you to the login page. As I said, this works correctly in all browsers, but for some reason the redirect isn't working with the URLConnection in Java 6.
Thanks for reading ... ideas?

It's apparently redirecting in an infinite loop because you don't maintain the user session. The session is usually backed by a cookie. You need to create a CookieManager before you use URLConnection.
// First set the default cookie manager.
CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));
// All the following subsequent URLConnections will use the same cookie manager.
URLConnection connection = new URL(url).openConnection();
// ...
connection = new URL(url).openConnection();
// ...
connection = new URL(url).openConnection();
// ...
See also:
Using java.net.URLConnection to fire and handle HTTP requests

Duse, I have add this lines:
java.net.CookieManager cm = new java.net.CookieManager();
java.net.CookieHandler.setDefault(cm);
See this example:
java.net.CookieManager cm = new java.net.CookieManager();
java.net.CookieHandler.setDefault(cm);
String buf="";
dk = new DAKABrowser(input.getText());
try {
URL url = new URL(dk.toURL(input.getText()));
DataInputStream dis = new DataInputStream(url.openStream());
String inputLine;
while ((inputLine = dis.readLine()) != null) {
buf+=inputLine;
output.append(inputLine+"\n");
}
dis.close();
}
catch (MalformedURLException me) {
System.out.println("MalformedURLException: " + me);
}
catch (IOException ioe) {
System.out.println("IOException: " + ioe);
}
titulo.setText(dk.getTitle(buf));

I was using Jenkins on Tomcat6 on a unix environment and got this bug. For some reason, upgrading to Java7 solved it. I'd be interested to know exactly why that fixed it.

I had faced the same problem and it took considerable amount of time to understand the problem.
So to summarize the problem was in mismatch of headers.
Consider below being my Resource
#GET
#Path("booksMasterData")
#Produces(Array(core.MediaType.APPLICATION_JSON))
def booksMasterData(#QueryParam("stockStatus") stockStatus : String): Response = {
// some logic here to get the books and send it back
}
And here is client code, which was trying to connect to my above resource
ClientResponse clientResponse = restClient.resource("http://localhost:8080/booksService").path("rest").path("catalogue").path("booksMasterData").accept("application/boks-master-data+json").get(ClientResponse.class);
And the error was coming on exactly above line.
What was the problem?
My Resource was using
"application/json"
in
#Produces annotation
and my client was using
accept("application/boks-master-data+json")
and this was the problem.
It took me long to find out this as the error was no where related. Break through was when I tried to access my resource in postman with
Accept-> "application/json" header
it worked fine, however with
Accept-> "application/boks-master-data+json" header
it doesnt.
And again, even Postman was not giving me proper error. The error was too generic. Please see the below image for reference.

HttpURLConnection and encoded characters in the URL

I have a code like this:
URL url = new URL("http://foo.com/?param=paj%E9");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
...
However, it seems like the openConnection is supressing the "%E9" part of the url, and the server ends up receiving a request http://foo.com?param=paj
Am I forgetting to apply any different setting for this to work properly?
Thanks!
EDIT: The url "http://foo.com/?param=paj%E9" is already encoded (from http://foo.com/?param=pajé), and this should be the request the server should receive. If I try to access http://foo.com/?param=paj%E9 straight from the browser, it works as expected. If I URLEncode "paj%E9", I'll be double-encoding the parameter, and the server would see "paj%E9" instead "pajé" upon decoding the value.
I'm actually trying to build a proxy, and therefore I receive the urls already encoded. The problem is that whenever I pass such an encoded parameter to be requested using HttpURLConnection, it simply ignores the encoded part (like %E9).

You need to use java.net.URI class to encode your URL instead of handle it on your own. Chek this:
HTTP URL Address Encoding in Java

You can use the following code
URLEncoder.encode("中文", "utf-8")

java.io.IOException: Server returns HTTP response code 505

I have HTML based queries in my code and one specific kind seems to give rise to IOExceptions upon receiving 505 response from the server. I have looked up the 505 response along with other people who seemed to have similar problems. Apparently 505 stands for HTTP version mismatch, but when I copy the same query URL to any browser (tried firefox, seamonkey and Opera) there seems to be no problem. One of the posts I read suggested that the browsers might automatically handle the version mismatch problem..
I have tried to dig in deeper by using the nice developer tool that comes with Opera, and it looks like there is no mismatch in versions (I believe Java uses HTTP 1.1) and a nice 200 OK response is received. Why do I experience problems when the same query goes through my Java code?
private InputStream openURL(String urlName) throws IOException{
URL url = new URL(urlName);
URLConnection urlConnection = url.openConnection();
return urlConnection.getInputStream();
}
sample link: http://www.uniprot.org/uniprot/?query=mnemonic%3aNUGM_HUMAN&format=tab&columns=id,entry%20name,reviewed,organism,length

There has been some issues in Tomcat with URLs containing space in it. To fix the problem, you need to encode your url with URLEncoder.
Example (notice the space):
String url="http://example.org/test test2/index.html";
String encodedURL=java.net.URLEncoder.encode(url,"UTF-8");
System.out.println(encodedURL); //outputs http%3A%2F%2Fexample.org%2Ftest+test2%2Findex.html

AS a developer at www.uniprot.org I have the advantage of being able to look in the request logs. In the last year according to the logs we have not send a 505 response code. In any case our servers do understand http 1 requests as well as the default http1.1 (though you might not get the results that you expect).
That makes me suspect there was either some kind of data corruption on the way. Or you where affected by a hardware failure (lately we have had some trouble with a switch and a whole datacentre ;). In any case if you ever have questions or problems with uniprot.org please contact help#uniprot.org then we can see if we can help/fix the problem.
Your code snippet seems normal and should work.
Regards,
Jerven Bolleman

Are you behind a proxy? This code works for me and prints out the same text I see through a browser.
final URL url = new URL("http://www.uniprot.org/uniprot/?query=mnemonic%3aNUGM_HUMAN&format=tab&columns=id,entry%20name,reviewed,organism,length");
final URLConnection conn = url.openConnection();
final InputStream is = conn.getInputStream();
System.out.println(IOUtils.toString(is));
conn is an instance of HttpURLConnection

from the API documentation for the URL class:
The URL class does not itself encode or decode any URL components
[...]. It is the responsibility of the caller to encode any fields,
which need to be escaped prior to calling URL, and also to decode any
escaped fields, that are returned from URL.
so if you have any spaces in your url-str encode it before calling new URL(url-str)

#posdef I was having same HTTP error code 505 problem. When I pasted URL that I was using in Java code in Firefox, Chrome it worked. But through code was giving IOException. But at last I came to know that in url string there were brackets '(' and ')', by removing them it worked so it seems I needed URLEncodeing same like browsers.