Develop Reference

Encoding query parameters in URL using Java with valid charset

I am trying to understand what is the difference and importance of different charsets available while encoding and decoding text.
I have a scenario, where I want to call a RestAPI. The RestAPI has a base URL, for ex: https://myrestapiurl.com. Now to perform a GET request, the URL is formed by appending the id of the entity that I want to fetch, like: https://myrestapiurl.com('id')
id : It has no limitations on valid characters!
I have encountered an id: باقی ریسورس , So before calling the RestAPI, I need to encode it. Using Java's URLEncoder, I tried the following:
String s ="باقی ریسورس";
String encodedID = URLEncoder.encode(s, StandardCharsets.UTF_8.name() )
Using the encodedID, I try to make a request using PostMan. The request fails with 404 or 400 when I use different charset. It only succeeds when I encode using ISO_8859_1 as follows:
String encodedID = URLEncoder.encode(s, StandardCharsets.ISO_8859_1.name());
String URL = "https://myrestapiurl.com('" + encodedID + "')";
This works fine, through code as well as PostMan. My question is:
How can I decide which charset to use before encoding? Or should I have fallbacks? That is if it fails with UTF_8 then try with UTF_16 etc etc...but this is very in-efficient. In case if the entity actually doesn't exist, then, these tries would be overhead
Also, when I visit https://www.w3schools.com/tags/ref_urlencode.ASP and enter the text to be encoded, it provides the valid encoded string with ISO_8859_1 , how does it manage to do so?
How can this be done in Java without using any other extra libraries like apache? We don't have choice to add extra dependencies!

How to URL-encode the the whole xml value of a query param using Spring's rest template?

I am working on a Spring Boot application
I need to make a request to an external service, old and ill-conceived. The request take the form of a HTTP GET (or POST) call, but the payload, an xml content, need to be passed as a query parameter. For example,
GET http://ill-service.com/plain.cgi?XML_DATA=<request attribute="attributeValue"><content contentAttribute="plain"/></request>
Of course, the value of query param XML_DATA need to be URL encoded, and normally, the RestTemplate of Spring boot work good on that, following RFC 3986 (see http://www.ietf.org/rfc/rfc3986.txt).
Except that, as allowed by this RFC, '/' and '=' character are left in the param value, giving me the following query :
GET http://ill-service.com/plain.cgi?XML_DATA=%3Crequest%20attribute=%22attributeValue%22%3E%3Ccontent%20contentAttribute=%22plain%22/%3E%3C/request%3E
In a perfect wold, this would be good, but do you remember when I said that the service I am trying to call is ill-conceived ? In another world, it needs to have the full content of XML_DATA URL-encoded. In another words, it needs the following query:
GET http://ill-service.com/plain.cgi?XML_DATA=%3Crequest%20attribute%3D%22attributeValue%22%3E%3Ccontent%20contentAttribute%3D%22plain%22%2F%3E%3C%2Frequest%3E%0A
I am quite lost on how to instruct the rest template or the UriComponentBuilder I am using to do so. Any help would be greatly appreciated

Probably u can use spring's UriUtils class

Use java.net.URLEncoder to encode your XML payload first and then append the encoded payload.

Following the suggestion of Vasif, and some information about UriComponentBuilder I found the following solutions :
String xmlContent = "<request attribute="attributeValue"><content contentAttribute="plain"/></request>";
URI uri = UriComponentsBuilder.fromHttpUrl("http://ill-service.com/plain.cgi")
//This part set the query param as a full encoded value, not as query value encoded
.queryParam("XML_DATA", UriUtils.encode(xmlContent, "UTF-8"))
//The build(true) indicate to the builder that the Uri is already encoded
.build(true).toUri();
String responseStr = restTemplate.getForObject(uri ,String.class)

How to send JSON as a parameter in url using Spring RestTemplate?

I am trying to achieve same thing as this: How to use query parameter represented as JSON with Spring RestTemplate?, sending JSON string as a URL parameter in restTemplate.exchange().
The accepted answer mentions that sending JSON object as request parameter is generally not a good idea since you will probably face problem with curly brackets inside JSON. That is exactly what is happening when I am trying to make a GET call to an API. Since this is an API from another system, I cannot ask them to change the format and will have to call the GET endpoint, passing JSON as parameter. How can I achieve this in restTemplate.exchange() call?
Note: The mentioned related question does not guide on how to overcome this problem and I do not have enough reputation to comment on it to ask the author of the answer.

Answering my own question. While it is a bad idea to pass JSON like this in a query/url parameter, there is a workaround as suggested here: https://jira.spring.io/browse/SPR-9220?focusedCommentId=76760&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-76760.
Replicating the code here in case this link goes dead:
String url = "http://localhost:8983/solr/select?wt=json&indent=true&fl=*&q=*:*&fq={!geofilt}&sfield=venue_location&pt=28.0674,-80.5595&d=25";
URI uri = UriComponentsBuilder.fromUriString(url).build().encode().toUri();
System.out.println(uri);
// http://localhost:8983/solr/select?wt=json&indent=true&fl=*&q=*:*&fq=%7B!geofilt%7D&sfield=venue_location&pt=28.0674,-80.5595&d=25
Basically, instead of passing url having JSON query/url parameters as a string, pass it as a URI. Then call exchange method as before, but with URI instead of String:
restTemplate.exchange(uri, HttpMethod.GET, requestEntity, String.class)

If this is 3rd party API and you cannot control or change JSON processing on backend side - there is no solution. Even if you will encode with URLEncoder - there is no guarantee that API backend would process such request correctly.

You can use URLEncoder class to encode the URL in exchange method, e.g.:
String url = "http://www.yoursite.com/api?param={\"some_key\":\"some_value\"}";
System.out.println(URLEncoder.encode(url, StandardCharsets.UTF_8.name()));
This will encode the characters (like braces and double quotes) and server then will decode it back to json.

Android parse special characters in json response

In my Android application I get JSON response string from a PHP url. from the response I get some hotel names with apostrophe, I get &#039 character instead of apostrophe. How can I parse the hotel with special characters in android? I can see the apostrophe in the browser but could not see in android logcat.
I have tried jresponse = URLEncoder.encode(jresponse,"UTF-8"); but I could not get apostrophe for hotel name.
This is the one of the hotel name in the response.
I see the following in browser.
{"id":12747,
"name":"Oscar's",
....
}
But in the logcat:
id 12747
name Oscar's

Use the decoder instead of encoder. URLDecoder.decode(jresponse,"UTF-8")

Use ISO-8859-2 when you create the URLEncodedEntity that you send off. You can set this as a parameter in the constructor.
Without a specified charset, you are probably sending the data in UTF-8/UTF-16 (most common) which the server is interpreting in a different way.
EDIT: It looks like ISO-8859-2 doesn't support ñ. You may have to change something server-side. http://en.wikipedia.org/wiki/ISO/IEC_8859-2

You can try Html class. eg :-
jresponse = Html.fromHtml(jresponse);

Java api to encode URL query parameter values?

I used URLEncoder but this seems URLEncoder does not cover everything of url query parameter encoding (escaped). I searched and someone used URLEncoder to encode whole url. For my case I iust want to encode (escape) url query parameter values.
Here are some code I am used
String url = "http://myserver.com?x=" + URLEncoder.encode(x_value, "UTF-8");
Anyone has any ideas? Thanks.