Develop Reference

How to encode only a part of URL

I have incomplete URL's which I am redirecting (don't have the full URL) like
a.jsp?id=269101|14000
and
b.jsp?action=in&id=239394|2000&inmethod=
I wanted to encode the pipe "|" char only, so I started with java.net.URI class but it asks for complete url.So I used URLEncoder but it encodes the entire url.
I know I can look for | in url and encode it directly but what would be the best approach?

Using String.replace():
String myUrl = "b.jsp?action=in&id=239394|2000&inmethod=";
myUrl = myUrl.replace("|","%7C");

You need to use the URLEncoder on each query parameter value that needs to be encoded.
String url = "b.jsp?action=in" +
"&id=" + URLEncoder.encode("239394|2000", StandardCharsets.UTF_8) +
"&inmethod=";
System.out.println(url); // prints: b.jsp?action=in&id=239394%7C2000&inmethod=

Using the URLEncoder is the correct way to go. However you should do the encoding before you create your full url. using it on your full url will cause all special URL characters to be encoded. Which is not what you want here
Change your code to something like this
String url = "a.jsp?id=" + URLEncoder.encode("269101|14000",StandardCharsets.UTF_8);

Play framework WS url spaces

I have a problem calling WS.url() in play framework 2.3.3 with url containing spaces. All other characters all url encoded automatically but not spaces. When i try to change all spaces to "%20", WS convert it to "%2520" because of "%" character. With spaces i've got java.net.URISyntaxException: Illegal character in query. How can i handle this ?
part of the URL's query String:
&input=/mnt/mp3/music/folder/01 - 23.mp3
The code looks like this:
Promise<JsonNode> jsonPromise = WS.url(url).setAuth("", "cube", WSAuthScheme.BASIC).get().map(
new Function<WSResponse, JsonNode>() {
public JsonNode apply(WSResponse response) {
System.out.println(response.getBody());
JsonNode json = response.asJson();
return json;
}
}
);

You should "build" your URL based on the way java.net.URL(which Play! uses for it's WS) does it. WS.url() follows the same logic.
The use of URLEncoder/Decoder is recommended only for form data.
From JavaDoc:
"Note, the java.net.URI class does perform escaping of its component
fields in certain circumstances. The recommended way to manage the
encoding and decoding of URLs is to use java.net.URI, and to convert
between these two classes using toURI() and URI.toURL(). The
URLEncoder and URLDecoder classes can also be used, but only for HTML
form encoding, which is not the same as the encoding scheme defined
in RFC2396."
So, the solution is to use THIS:
WS.url(baseURL).setQueryString(yourQueryString);
Where:
baseURL is your scheme + host + path etc.
yourQueryString is... well, your query String, but WITHOUT the ?: input=/mnt/mp3/music/folder/01 - 23.mp3
Or, if you want to use a more flexible, programmatic approach, THIS:
WS.url(baseURL).setQueryParameter(param, value);
Where:
param is the parameter's name in the query String
value is the value of the parameter
If you want multiple parameters with values in your query you need to chain them by adding another .setQueryParameter(...). This implies that this approach is not very accomodating for complex, multi-parameter query Strings.
Cheers!

If you check the console you will find that the exception is : java.net.URISyntaxException: Illegal character in path at index ...
That's because play Java api uses java.net.URL (as you can see here in line 47).
You can use java.net.URLEncoder to encode your URL
WS.url("http://" + java.net.URLEncoder.encode("google.com/test me", "UTF-8"))
UPDATE
If you want an RFC 2396 compliant method you can do this :
java.net.URI u = new java.net.URI(null, null, "http://google.com/test me",null);
System.out.println("encoded url " + u.toASCIIString());

Encoding URL query parameters in Java

How does one encode query parameters to go on a url in Java? I know, this seems like an obvious and already asked question.
There are two subtleties I'm not sure of:
Should spaces be encoded on the url as "+" or as "%20"? In chrome if I type in "http://google.com/foo=?bar me" chrome changes it to be encoded with %20
Is it necessary/correct to encode colons ":" as %3B? Chrome doesn't.
Notes:
java.net.URLEncoder.encode doesn't seem to work, it seems to be for encoding data to be form submitted. For example, it encodes space as + instead of %20, and encodes colon which isn't necessary.
java.net.URI doesn't encode query parameters

java.net.URLEncoder.encode(String s, String encoding) can help too. It follows the HTML form encoding application/x-www-form-urlencoded.
URLEncoder.encode(query, "UTF-8");
On the other hand, Percent-encoding (also known as URL encoding) encodes space with %20. Colon is a reserved character, so : will still remain a colon, after encoding.

Unfortunately, URLEncoder.encode() does not produce valid percent-encoding (as specified in RFC 3986).
URLEncoder.encode() encodes everything just fine, except space is encoded to "+". All the Java URI encoders that I could find only expose public methods to encode the query, fragment, path parts etc. - but don't expose the "raw" encoding. This is unfortunate as fragment and query are allowed to encode space to +, so we don't want to use them. Path is encoded properly but is "normalized" first so we can't use it for 'generic' encoding either.
Best solution I could come up with:
return URLEncoder.encode(raw, "UTF-8").replaceAll("\\+", "%20");
If replaceAll() is too slow for you, I guess the alternative is to roll your own encoder...
EDIT: I had this code in here first which doesn't encode "?", "&", "=" properly:
//don't use - doesn't properly encode "?", "&", "="
new URI(null, null, null, raw, null).toString().substring(1);

EDIT: URIUtil is no longer available in more recent versions, better answer at Java - encode URL or by Mr. Sindi in this thread.
URIUtil of Apache httpclient is really useful, although there are some alternatives
URIUtil.encodeQuery(url);
For example, it encodes space as "+" instead of "%20"
Both are perfectly valid in the right context. Although if you really preferred you could issue a string replace.

It is not necessary to encode a colon as %3B in the query, although doing so is not illegal.
URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
query = *( pchar / "/" / "?" )
pchar = unreserved / pct-encoded / sub-delims / ":" / "#"
unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
pct-encoded = "%" HEXDIG HEXDIG
sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="
It also seems that only percent-encoded spaces are valid, as I doubt that space is an ALPHA or a DIGIT
look to the URI specification for more details.

The built in Java URLEncoder is doing what it's supposed to, and you should use it.
A "+" or "%20" are both valid replacements for a space character in a URL. Either one will work.
A ":" should be encoded, as it's a separator character. i.e. http://foo or ftp://bar. The fact that a particular browser can handle it when it's not encoded doesn't make it correct. You should encode them.
As a matter of good practice, be sure to use the method that takes a character encoding parameter. UTF-8 is generally used there, but you should supply it explicitly.
URLEncoder.encode(yourUrl, "UTF-8");

I just want to add anther way to resolve this problem.
If your project depends on spring web, you can use their utils.
import org.springframework.web.util.UriUtils
import java.nio.charset.StandardCharsets
UriUtils.encode('vip:104534049:5', StandardCharsets.UTF_8)
Output:
vip%3A104534049%3A5

String param="2019-07-18 19:29:37";
param="%27"+param.trim().replace(" ", "%20")+"%27";
I observed in case of Datetime (Timestamp)
URLEncoder.encode(param,"UTF-8") does not work.

The white space character " " is converted into a + sign when using URLEncoder.encode. This is opposite to other programming languages like JavaScript which encodes the space character into %20. But it is completely valid as the spaces in query string parameters are represented by +, and not %20. The %20 is generally used to represent spaces in URI itself (the URL part before ?).

if you have only space problem in url. I have used below code and it work fine
String url;
URL myUrl = new URL(url.replace(" ","%20"));
example : url is
www.xyz.com?para=hello sir
then output of muUrl is
www.xyz.com?para=hello%20sir

What is the most efficient way to format UTF-8 strings in java?

I am doing the following:
String url = String.format(WEBSERVICE_WITH_CITYSTATE, cityName, stateName);
String urlUtf8 = new String(url.getBytes(), "UTF8");
Log.d(TAG, "URL: [" + urlUtf8 + "]");
Reader reader = WebService.queryApi(url);
The output that I am looking for is essentially to get the city name with blanks (e.g., "Overland Park") to be formatted as Overland%20Park.
Is it this the best way?

Assuming you are actually wanting to encode your string for use in a URL (ie, "Overland Park" can also be formatted as "Overland+Park") you want URLEncoder.encode(url, "UTF-8"). Other unsafe characters will be converted to the %xx format you are asking for.

The simple answer is to use URLEncoder.encode(...) as stated by #Recurse. However, if part or all of the URL has already been encoded, then this can lead to double encoding. For example:
http://foo.com/pages/Hello%20There
or
http://foo.com/query?keyword=what%3f
Another concern with URLEncoder.encode(...) is that it doesn't understand that certain characters should be escaped in some contexts and not others. So for example, a '?' in a query parameter should be escaped, but the '?' that marks the start of the "query part" should not be escaped.
I think that safer way to add missing escapes would be the following:
String safeURI = new URI(url).toASCIIString();
However, I haven't tested this ...

Escaping & in a URL

I am using jsps and in my url I have a value for a variable like say "L & T". Now when I try to retrieve the value for it by using request.getParameter I get only "L". It recognizes "&" as a separator and thus it is not getting considered as a whole string.
How do I solve this problem?

java.net.URLEncoder.encode("L & T", "utf8")
this outputs the URL-encoded, which is fine as a GET parameter:
L+%26+T

A literal ampersand in a URL should be encoded as: %26
// Your URL
http://www.example.com?a=l&t
// Encoded
http://www.example.com?a=l%26t

You need to "URL encode" the parameters to avoid this problem. The format of the URL query string is:
...?<name>=<value>&<name>=<value>&<etc>
All <name>s and <value>s need to be URL encoded, which basically means transforming all the characters that could be interpreted wrongly (like the &) into %-escaped values. See this page for more information:
http://www.w3schools.com/TAGS/ref_urlencode.asp
If you're generating the problem URL with Java, you use this method:
String str = URLEncoder.encode(input, "UTF-8");
Generating the URL elsewhere (some templates or JS or raw markup), you need to fix the problem at the source.

You can use UriUtils#encode(String source, String encoding) from Spring Web. This utility class also provides means for encoding only some parts of the URL, like UriUtils#encodePath.