Develop Reference

URL encoding of "&"

I am currently facing an issue with sending some encoded characters. My main aim is to send the text using a POST https request. The catch is that the back-end is not quite the best one, so special letters (such as æ) I have to send in a special (custom) way.
Giving a simple example, I have the text hjælp. The letter æ should become &aelig in order the back-end to understand that it's this specific letter.
My url looks like this:
https://example.com/back-end/sendText?user=admin&text=hj&aeliglp
Obviously, this wouldn't work, because the back-end would see 3 parameter keys: user, text and aeliglp.
Of course, in code, my url is an actual URL object. However, if I use URLEncoder.encode(value, "utf-8"); it would turn my & into %26 and the %26 itself to %2526.
On wikipedia I read about this:
Because the percent ("%") character serves as the indicator for
percent-encoded octets, it must be percent-encoded as "%25" for that
octet to be used as data within a URI.
Nevertheless, I must send it with a %26, but without encoding it to %2526. That is because I cannot change or ask for a change on the back-end.
In order to send the POST I use the most basic way:
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(); //url is my URL object
conn.setRequestMethod("POST");
int result = conn.getResponseCode();
Is there any way I can create an URL object without encoding it automatically?

How to correctly redirect a URL which does NOT start with HTTP or HTTPS protocols?

I am overwriting the HttpServletResponseWrapper.sendRedirect() method. Normally, a redirect url will start with http or https. But we do encounter some URLs like this:
//www.google.com.
This URL works when you assign this url to window.location in js. However, it would fail if we try to redirect this URL. Because it will always consider it as a relative path.
Do you know how to correctly redirect a URL like this?

You could rely on the following.
http://docs.oracle.com/javaee/1.2.1/api/javax/servlet/ServletRequest.html#getScheme()
This will tell you whether it's http/https

For handling relative URLs (i.e. ones which don't specify schema or host), you need to copy the missing parts from the ServletRequest that triggered the processing. The Java class URL has a helper method for that:
URL requestURL = new URL( request.getRequestURL() );
URL redirectURL = new URL( requestURL, "//www.google.com" );

Referring to this question, link without protocol would use the current protocol by default.
Thus you may simply use the protocol of current page.

Invoking a 'REST' service which have query parameters in the URL

I have to invoke a GET on a service which returns text/xml.
The endpoint is something like this:
http://service.com/rest.asp?param1=34&param2=88&param3=foo
When I hit this url directly on a browser (or some UI tool), all's good. I get a response.
Now, I am trying to use CXF WebClient to fetch the result using a piece of code like this:
String path = "rest.asp?param1=34&param2=88&param3=foo";
webClient.path(path)
.type(MediaType.APPLICATION_JSON)
.accept(MediaType.TEXT_XML_TYPE)
.get(Response.class);
I was debugging the code and found that the request being sent was url encoded which appears something like this:
http://service.com/rest.asp%3Fparam1=34%26param2=88%26param3=foo
Now, the problem is the server doesn't seem to understand this request with encoded stuff. It throws a 404. Hitting this encoded url on the browser also results in a 404.
What should I do to be able to get a response successfully (or not let the WebClient encode the url)?

Specify the parameters using the query method:
String path = "rest.asp";
webClient.path(path)
.type(MediaType.APPLICATION_JSON)
.accept(MediaType.TEXT_XML_TYPE)
.query("param1","34")
.query("param2","88")
.query("param3","foo")
.get(Response.class);

You will need to encode your URL. You can do it with the URLEncoder class as shown below:
Please replace your line
String path = "rest.asp?param1=34&param2=88&param3=foo";
with
String path = URLEncoder.encode("rest.asp?param1=34&param2=88&param3=foo");

How to encode a URL that I want to pass as a query string

I'm trying to send a URL as paramter of a query string like this example:
http://localhost.com/myapp.jsp?pathToFileURL=http://192.168.0.1/my_file.pdf
What I did is I used encode URL to encode the path before sending it to the server, problem is im getting a "400 Invalid URI: noSlash" because of this.
From what I read the problem is the tomcat security and that I should add a parameter to the tomcat startup
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
But I can't modify the parameters of the tomcat, so is it possible to do it other way?
Thanks

You can do URLSafebase64 encoding at the client side and URLSafebase64 decoding at the server side.
Check URLEncoder class for more details:
http://docs.oracle.com/javase/1.5.0/docs/api/java/net/URLEncoder.html
You can test manually before coding using any of the online URL Encoder/Decoder. Just google for "URL Encoder/Decoder"

Complete stab in the dark but you could try escaping the slashes with backslashes or you could try replacing them with %2F which is the URL encoded version of forward slash.
Hope this helps.

Base64 the URL then on the receiving end base64 decode to get the original URL without any alteration

java.io.IOException: Server returns HTTP response code 505

I have HTML based queries in my code and one specific kind seems to give rise to IOExceptions upon receiving 505 response from the server. I have looked up the 505 response along with other people who seemed to have similar problems. Apparently 505 stands for HTTP version mismatch, but when I copy the same query URL to any browser (tried firefox, seamonkey and Opera) there seems to be no problem. One of the posts I read suggested that the browsers might automatically handle the version mismatch problem..
I have tried to dig in deeper by using the nice developer tool that comes with Opera, and it looks like there is no mismatch in versions (I believe Java uses HTTP 1.1) and a nice 200 OK response is received. Why do I experience problems when the same query goes through my Java code?
private InputStream openURL(String urlName) throws IOException{
URL url = new URL(urlName);
URLConnection urlConnection = url.openConnection();
return urlConnection.getInputStream();
}
sample link: http://www.uniprot.org/uniprot/?query=mnemonic%3aNUGM_HUMAN&format=tab&columns=id,entry%20name,reviewed,organism,length

There has been some issues in Tomcat with URLs containing space in it. To fix the problem, you need to encode your url with URLEncoder.
Example (notice the space):
String url="http://example.org/test test2/index.html";
String encodedURL=java.net.URLEncoder.encode(url,"UTF-8");
System.out.println(encodedURL); //outputs http%3A%2F%2Fexample.org%2Ftest+test2%2Findex.html

AS a developer at www.uniprot.org I have the advantage of being able to look in the request logs. In the last year according to the logs we have not send a 505 response code. In any case our servers do understand http 1 requests as well as the default http1.1 (though you might not get the results that you expect).
That makes me suspect there was either some kind of data corruption on the way. Or you where affected by a hardware failure (lately we have had some trouble with a switch and a whole datacentre ;). In any case if you ever have questions or problems with uniprot.org please contact help#uniprot.org then we can see if we can help/fix the problem.
Your code snippet seems normal and should work.
Regards,
Jerven Bolleman

Are you behind a proxy? This code works for me and prints out the same text I see through a browser.
final URL url = new URL("http://www.uniprot.org/uniprot/?query=mnemonic%3aNUGM_HUMAN&format=tab&columns=id,entry%20name,reviewed,organism,length");
final URLConnection conn = url.openConnection();
final InputStream is = conn.getInputStream();
System.out.println(IOUtils.toString(is));
conn is an instance of HttpURLConnection

from the API documentation for the URL class:
The URL class does not itself encode or decode any URL components
[...]. It is the responsibility of the caller to encode any fields,
which need to be escaped prior to calling URL, and also to decode any
escaped fields, that are returned from URL.
so if you have any spaces in your url-str encode it before calling new URL(url-str)

#posdef I was having same HTTP error code 505 problem. When I pasted URL that I was using in Java code in Firefox, Chrome it worked. But through code was giving IOException. But at last I came to know that in url string there were brackets '(' and ')', by removing them it worked so it seems I needed URLEncodeing same like browsers.