Develop Reference

Java - how to build an URI using a query string that is already escaped?

LATER EDIT: not same problem as the suggested answer. In my case, I need to build the URI, relying on the fact that the original query string is not modified.
I have a String (coming from a request query String) that is already correctly escaped, like param1=%2Ffolder1%2Ffolder2%2Ffolder%26name%202&param2=9481dxcv234.
The decoded value of param1 is /folder1/folder2/folder&name 2. Obviously, I cannot unescape that String (because of the & char in this value)...
I need to build an URI which has that original string as query value.
I tried using org.apache.http.client.utils.URIBuilder but could not get it to work: if I provide the original String to the URI(... constructor, the resulting URL is double-escaped, like param1=%252Ffolder1%252Ffolder2%252Ffolder%2526name%25202&param2=9481dxcv234.
Can I somehow do what I need ? To build an URI by passing the query string already escaped and leave it unchanged ?
Thanks.

I think, the simplest way is unescape it first.
Then you can work with url as usualy.

You could use org.springframework.web.util.UriComponentsBuilder:
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url);
UriComponents components = builder.build(true); //EDIT: pass true when the URI was fully encoded already.
MultiValueMap<String, String> parameters = components.getQueryParams();
parameters.get("param1") //"%2Ffolder1%2Ffolder2%2Ffolder%26name%202"
You can then URLDecode to get what you need.
Edit:
Since you appear to be using Apache HttpClient,
List<NameValuePair> params = org.apache.http.client.utils.URLEncodedUtils.parse(new URI(url), Charset.forName("UTF-8"));
params.get("param1") //"/folder1/folder2/folder&name 2"

Play framework WS url spaces

I have a problem calling WS.url() in play framework 2.3.3 with url containing spaces. All other characters all url encoded automatically but not spaces. When i try to change all spaces to "%20", WS convert it to "%2520" because of "%" character. With spaces i've got java.net.URISyntaxException: Illegal character in query. How can i handle this ?
part of the URL's query String:
&input=/mnt/mp3/music/folder/01 - 23.mp3
The code looks like this:
Promise<JsonNode> jsonPromise = WS.url(url).setAuth("", "cube", WSAuthScheme.BASIC).get().map(
new Function<WSResponse, JsonNode>() {
public JsonNode apply(WSResponse response) {
System.out.println(response.getBody());
JsonNode json = response.asJson();
return json;
}
}
);

You should "build" your URL based on the way java.net.URL(which Play! uses for it's WS) does it. WS.url() follows the same logic.
The use of URLEncoder/Decoder is recommended only for form data.
From JavaDoc:
"Note, the java.net.URI class does perform escaping of its component
fields in certain circumstances. The recommended way to manage the
encoding and decoding of URLs is to use java.net.URI, and to convert
between these two classes using toURI() and URI.toURL(). The
URLEncoder and URLDecoder classes can also be used, but only for HTML
form encoding, which is not the same as the encoding scheme defined
in RFC2396."
So, the solution is to use THIS:
WS.url(baseURL).setQueryString(yourQueryString);
Where:
baseURL is your scheme + host + path etc.
yourQueryString is... well, your query String, but WITHOUT the ?: input=/mnt/mp3/music/folder/01 - 23.mp3
Or, if you want to use a more flexible, programmatic approach, THIS:
WS.url(baseURL).setQueryParameter(param, value);
Where:
param is the parameter's name in the query String
value is the value of the parameter
If you want multiple parameters with values in your query you need to chain them by adding another .setQueryParameter(...). This implies that this approach is not very accomodating for complex, multi-parameter query Strings.
Cheers!

If you check the console you will find that the exception is : java.net.URISyntaxException: Illegal character in path at index ...
That's because play Java api uses java.net.URL (as you can see here in line 47).
You can use java.net.URLEncoder to encode your URL
WS.url("http://" + java.net.URLEncoder.encode("google.com/test me", "UTF-8"))
UPDATE
If you want an RFC 2396 compliant method you can do this :
java.net.URI u = new java.net.URI(null, null, "http://google.com/test me",null);
System.out.println("encoded url " + u.toASCIIString());

How to convert URL toURI when there are unwise characters?

I've got URL object with path containing unwise characters (RFC 2396) in my case it is "|" (pipe) character.
Now I need to safely convert that to URI, but URL.toURI() throws an exception.
I've read URL documentation but this part is for me confusing:
The URL class does not itself encode or decode any URL components
according to the escaping mechanism defined in RFC2396. It is the
responsibility of the caller to encode any fields, which need to be
escaped prior to calling URL, and also to decode any escaped fields,
that are returned from URL. Furthermore, because URL has no knowledge
of URL escaping, it does not recognize equivalence between the encoded
or decoded form of the same URL.
So how should I do it? What is the pattern here to encode this characters during conversion? Do I need create encoded copy of my URL object?

OK, I come up with something like this:
URI uri = new URI(url.getProtocol(),
null /*userInfo*/,
url.getHost(),
url.getPort(),
(url.getPath()==null)?null:URLDecoder.decode(url.getPath(), "UTF-8"),
(url.getQuery()==null)?null:URLDecoder.decode(url.getQuery(), "UTF-8"),
null /*fragment*/);
Looks like it works, here is an example. Can some one confirm that this is proper solution?
Edit: initial solution had some problems when there was a query so I've fixed it.

Use URL encoding?
From your example, you currently have:
URL url = new URL("http", "google.com", 8080, "/crapy|path with-unwise_characters.jpg");
Instead, I would use:
String path = "/crapy|path with-unwise_characters.jpg"
URL url = new URL("http", "google.com", 8080, URLEncoder.encode(path, "UTF-8"));
This should work and handle all unwise characters in the path as per the standard URL encoding.

HTTPClient 4 has an object for that org.apache.http.client.utils.URIBuilder:
URIBuilder builder =
new URIBuilder()
.setScheme(url.getProtocol())
.setHost(url.getHost())
.setPort(url.getPort())
.setUserInfo(url.getUserInfo())
.setPath(url.getPath())
.setQuery(url.getQuery());
URI uri = builder.build();
return uri;

Encode URL parameters using Java

I want to encode part of URL paramter in JAVA
http://statebuild-dev.com/iit-title/size.xml?id=(102T OR 140T)
to
http://statebuild-dev.com/iit-title/size.xml?id=(102%20OR%20140)
have tried using URI to encode but it also encodes ? which I do not want. In the URL I want to encode the part after '='
URI uri = new URI("http",
"statebuild-dev.com/iit-title", "/size.xml?id=(102 OR 140)", null);
//URL url = uri.toURL();
System.out.println(uri.toString());
System.out.println(url1);
Thank You

you want to use URLEncoder to encode each query parameter before adding to the url, e.g.:
String encodedValue = URLEncoder.encode("(102 OR 140)", "UTF-8");

This answer has a good discussion of encoding the various parts of a URI/URL. You're on the right track, but your specific problem is that you have the various parts of the URI wrong. You need to use the multi-part constructor that takes an authority, path, query, and fragment:
URI uri = new URI("http", "statebuild-dev.com", "/iit-title/size.xml", "id=(102 or 104)", null);
System.out.println(uri.toString());
System.out.println(uri.toASCIIString());

You used the wrong constructor.
Try this:
URI uri = new URI("http","statebuild-dev.com", "/iit-title/size.xml", "id=(102 OR 140)", null);
See also java.net.URLEncoder

The java.net.URI class can help; in the documentation of URL you find
Note, the URI class does perform escaping of its component fields in certain circumstances. The recommended way to manage the encoding and decoding of URLs is to use URI
Check this thread.

How to encode URL to avoid special characters in Java? [duplicate]

This question already has answers here:
HTTP URL Address Encoding in Java
(24 answers)
Closed 5 years ago.
i need java code to encode URL to avoid special characters such as spaces and % and & ...etc

URL construction is tricky because different parts of the URL have different rules for what characters are allowed: for example, the plus sign is reserved in the query component of a URL because it represents a space, but in the path component of the URL, a plus sign has no special meaning and spaces are encoded as "%20".
RFC 2396 explains (in section 2.4.2) that a complete URL is always in its encoded form: you take the strings for the individual components (scheme, authority, path, etc.), encode each according to its own rules, and then combine them into the complete URL string. Trying to build a complete unencoded URL string and then encode it separately leads to subtle bugs, like spaces in the path being incorrectly changed to plus signs (which an RFC-compliant server will interpret as real plus signs, not encoded spaces).
In Java, the correct way to build a URL is with the URI class. Use one of the multi-argument constructors that takes the URL components as separate strings, and it'll escape each component correctly according to that component's rules. The toASCIIString() method gives you a properly-escaped and encoded string that you can send to a server. To decode a URL, construct a URI object using the single-string constructor and then use the accessor methods (such as getPath()) to retrieve the decoded components.
Don't use the URLEncoder class! Despite the name, that class actually does HTML form encoding, not URL encoding. It's not correct to concatenate unencoded strings to make an "unencoded" URL and then pass it through a URLEncoder. Doing so will result in problems (particularly the aforementioned one regarding spaces and plus signs in the path).

I also spent quite some time with this issue, so that's my solution:
String urlString2Decode = "http://www.test.com/äüö/path with blanks/";
String decodedURL = URLDecoder.decode(urlString2Decode, "UTF-8");
URL url = new URL(decodedURL);
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
String decodedURLAsString = uri.toASCIIString();

If you don't want to do it manually use Apache Commons - Codec library. The class you are looking at is: org.apache.commons.codec.net.URLCodec
String final url = "http://www.google.com?...."
String final urlSafe = org.apache.commons.codec.net.URLCodec.encode(url);

Here is my solution which is pretty easy:
Instead of encoding the url itself i encoded the parameters that I was passing because the parameter was user input and the user could input any unexpected string of special characters so this worked for me fine :)
String review="User input"; /*USER INPUT AS STRING THAT WILL BE PASSED AS PARAMTER TO URL*/
try {
review = URLEncoder.encode(review,"utf-8");
review = review.replace(" " , "+");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
String URL = "www.test.com/test.php"+"?user_review="+review;

I would echo what Wyzard wrote but add that:
for query parameters, HTML encoding is often exactly what the server is expecting; outside these, it is correct that URLEncoder should not be used
the most recent URI spec is RFC 3986, so you should refer to that as a primary source
I wrote a blog post a while back about this subject: Java: safe character handling and URL building