Develop Reference

How to call GET api with query params having special chars{&,(,),'} using spring rest template

Below was the code used to encode uri having query params using UriComponentsBuilder
String uri = "http://hostname/api/items"
// api expected with params --> http://hostname/api/items?filter=IN('123') and id eq '123_&123'
restTemplate.exchange(UriComponentsBuilder.fromUriString(uri).queryParam("filter","IN('123') and id eq '123_&123'").encode().toUriString(), HttpMethod.GET, request, Response_Entity.class)
When above code is called, somehow at api side, i was getting 2 query params with keys -->filter & 123
How to handle it correctly using ?

try encoding query param by using URLEncoder.
String param = "IN('123') and id eq '123_&123'";
String encodedParam = URLEncoder.encode(param, Charset.defaultCharset()));
restTemplate.exchange(UriComponentsBuilder.fromUriString(uri).queryParam("filter",encodedParam).toUriString(), httpMethod, httpEntity, Some_Entity.class)
https://www.baeldung.com/java-url-encoding-decoding

Somehow query params are encoded and at api side, by default these are retrieved correctly after decoding, if i use toURI() of UriComponentsBuilder
Same was not working if i convert it to string using toUriString
Below is the code which worked for me.
URI uri = UriComponentsBuilder.fromUriString(uri)
.queryParam("filter",encodedParam)
.encode()
.build()
.toUri();
restTemplate.exchange(uri, HttpMethod.GET, request, Response_Entity.class)

how to get the value from HttpMethodParams

At client side I use the following code:
HashMap<String, String> paramMap = new HashMap<>();
paramMap.put("userId", "1579533296");
paramMap.put("identity", "352225199101195515");
paramMap.put("phoneNum", "15959177178");
HttpClient client = new HttpClient();
PostMethod method = new PostMethod("http://localhost:8088/requestTest");
HttpMethodParams p = new HttpMethodParams();
for (Map.Entry<String, String> entry : paramMap.entrySet()) {
p.setParameter(entry.getKey(), entry.getValue());
}
method.setParams(p);
client.executeMethod(method);
And the code of my server-side is like this:
#RequestMapping("/requestTest")
public void requestTest(HttpServletRequest request) throws IOException {
String userId = request.getParameter("userId");
String identity= request.getParameter("identity");
String phoneNum= request.getParameter("phoneNum");
System.out.println(userId+identity+phoneNum);
}
but I got the null value of userId,identity,and phoneNum,so how can I get the value of them? I know I can use method.setParameter(key,value) to set the parameter at client-side and use getParameter(key) to get the parameter value, but I just curious if there any way to get the value at server-side set by HttpMethodParams.

I think , you are getting confused between user defined parameters set in HttpServletRequest and HttpMethodParams .
As per JavaDoc of - HttpMethodParams ,
This class represents a collection of HTTP protocol parameters
applicable to HTTP methods.
These are predefined parameters specific to that HTTP method (see this)and has nothing to do with - HttpServletRequest parameters.
Request parameters need to be set as illustrated here
You have to also note that all these classes (HttpClient, PostMethod, HttpMethodParams etc ) that you are using on client side are from Apache to just be a convenient way to generate and call a HTTP end point but eventually what you will have on server side is a HttpServletRequest and there system is not Apache HttpClient specific.
So all you got on server side is to extract a named header or headers using - getHeaders() , getIntHeader() , getHeaderNames() , getDateHeader() , getProtocol() etc . Server side is standardized so you shouldn't see anything like - HttpMethodParams there.

You have to send your parameters using HttpServletRequest.
HttpMethodParams represent a collection of HTTP protocol parameters applicable to HTTP methods. List of Http method parameter can be found here.
But if you want to send it forcibly by HttpMethodParams you can set the JSON representation of your parameter in one of the variables of HttpMethodParameter and retrieve its value using that variable name.
Sample Code:
HttpMethodParams p = new HttpMethodParams();
p.setCredentialCharset("{userId":1579533296}");
//for loop not required
//your code
Now you can parse that JSON using ObjectMapper and get your required value.
Sample Code:
HttpMethodParams p = new HttpMethodParams();
JSONObject jsonObj = new JSONObject(p.getCredentialCharset());
jsonObj.get("userdId");
Note: This may work but not the recommended way.

Spring's UriComponentsBuilder.queryParam issue

I have recently switched to Spring for consuming REST API calls hosted by ServiceNow.
I am building my URI as below:
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(baseUrl.toString());
logger.info("URI before Query Param: " + builder.build().encode().toUri());
builder.queryParam("sysparm_limit", "2000000");
builder.queryParam("sysparm_offset", "0");
builder.queryParam("sysparm_exclude_reference_link", "true");
//this line is the issue because there is a = sign here
builder.queryParam("sysparm_query=user_name", snUser.getUser_name());
logger.info("URI after Query Param: " + builder.build().encode().toUri());
The output of this code is:
INFO: URI before Query Param: https://sandbox.service-now.com/api/now/v1/table/sys_user
INFO: URI after Query Param: https://sandbox.service-now.com/api/now/v1/table/sys_user?sysparm_limit=2000000&sysparm_offset=0&sysparm_exclude_reference_link=true&sysparm_query%3Duser_name=AX0011
The problem is with the final builder.queryParam. I am getting output as this:
sysparm_query%3Duser_name=AX0011
but what I want is:
sysparm_query=user_name=AX0011
So that eventually the final URI looks like this:
INFO: URI after Query Param: https://sandbox.service-now.com/api/now/v1/table/sys_user?sysparm_limit=2000000&sysparm_offset=0&sysparm_exclude_reference_link=true&sysparm_query=user_name=Z001NR6
So I tried replacing,
builder.queryParam("sysparm_query=user_name", snUser.getUser_name());
by:
builder.query("sysparm_query=user_name=" + snUser.getUser_name());
which changed the original output from:
INFO: URI after Query Param: https://sandbox.service-now.com/api/now/v1/table/sys_user?sysparm_limit=2000000&sysparm_offset=0&sysparm_exclude_reference_link=true&sysparm_query%3Duser_name=Z001NR6
to:
INFO: URI after Query Param: https://sandbox.service-now.com/api/now/v1/table/sys_user?sysparm_limit=2000000&sysparm_offset=0&sysparm_exclude_reference_link=true&sysparm_query=user_name%3DZ001NR6
Notice how sysparm_query%3Duser_name=Z001NR6 changed to sysparm_query=user_name%3DZ001NR6
Is ther anyway to see a = instead of %3D in the output?

The param looks quite strange - however - you can add it manually using the UriComponentsBuilder#query method:
UriComponentsBuilder builder = UriComponentsBuilder
.fromHttpUrl("https://example.com/api/")
.queryParam("param1", "12345")
.queryParam("param2", "abc")
.query("query=username=JOE");
System.out.println(builder.build().toString());
// produces https://example.com/api/?param1=12345&param2=abc&query=username=JOE
System.out.println(builder.build().encode().toString());
// produces https://example.com/api/?param1=12345&param2=abc&query=username%3DJOE
Manual concatenation:
UriComponentsBuilder builder = UriComponentsBuilder
.fromHttpUrl("https://example.com/api/")
.queryParam("param1", "12345")
.queryParam("param2", "abc");
// the parameter has to be properly url-encoded manually (not shown here)
String uri = builder.build().encode().toString() + "&query=username=JOE";
System.out.println(uri);
// produces: https://example.com/api/?param1=12345&param2=abc&query=username=JOE

The query component of a URL is frequently used to carry information in key=value pairs; you could think of this as a Map<String, String>. In this case, = and & are special characters that delimit these pairs, and they must be encoded when they form part of the key or the value to ensure that anything reading the query string this way is able to parse it properly.
In your case, how you use the builder depends on how you would want to retrieve your data later on. There are two options:
// Building the URL:
builder.queryParam("sysparm_query=user_name", snUser.getUser_name());
// URL contains ...&sysparm_query%3Duser_name=AX0011
// Reading the parsed query map:
Map<String, String> query = ...
String data = query.get("sysparm_query=user_name");
// value is AX0011
Or
// Building the URL:
builder.queryParam("sysparm_query", "user_name=" + snUser.getUser_name());
// URL contains ...&sysparm_query=user_name%3DAX0011
// Reading the parsed query map:
Map<String, String> query = ...
String value = query.get("sysparm_query");
// value is user_name=AX0011
In a correctly encoded URL, one of the = will always be encoded as %3D. Using a UriComponentsBuilder ensures that your URLs will be correctly encoded and that anything reading your URLs will be able to do so properly without data loss.

Correct use case of String parameter in SetQuery function of SolrQuery?

I have q
queryString = "select?wt=json&rows=0&indent=true&facet=true&q=*:*&facet=true&facet.field=outcome_type"
If queried like :
http://x.x.x.x:8983/solr/abc/queryString
it works. here abc is a core.
Now I would like to execute it programmatically, and using the following approach :
SolrQuery query = new SolrQuery();
query.setQuery(queryString);
QueryResponse resp = server.query(query);
here queryString as defined above, but it return the following error :
Exception in thread "main"
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
undefined field text
What I am missing here ? Or I need to build the query by set functions ?

I see few problems in your tentative.
You should not pass entire query string with the setQuery method. For almost each parameter available in query string there is a corresponding method in SolrQuery class.
SolrQuery does not support json format, SolrJ only supports the javabin and xml formats, I suggest to not specify any wt parameter.
So, you should use setQuery method only for q parameter:
query.setQuery("*:*");
For remaining parameters, the easiest way is use add method:
query.add("rows", "0"); // instead of setRows(0)
query.add("indent", "true");
query.add("facet", "true"); // ... setFacet(true)
query.add("facet.field", "outcome_type"); // ... addFacetField("outcome_type")
Hope this helps

I have used following approach to execute the query and it worked:
SolrQuery query = new SolrQuery();
query.setQuery(queryString);
query.setFacet(true);
query.set("wt", "json");
query.set("indent",true);
query.setRows(0);
query.addFacetField("outcome_type");
QueryResponse resp = server.query(query);

How to parse and decode URI in Java to URI components?

I am trying to find a method that would parse an URL, decoded it and returned the decoded components in an unambiguous way.
URLDecoder isn't a right fit, because it may return ambiguous String, e.g.
URLDecoder.decode("http://www.google.com?q=abc%26def", "UTF-8")
returns:
http://www.google.com?q=abc&def
So the information about escaped & is lost.
I'd like to have something like:
DecodedUrlComponents cmp = GreatURLDecoder.decode(url);
Map<String, List<String>> decodedQuery = cmp.getQuery();
decodedQuery.get("q").get(0); //returns "abc&def"
How do I accomplish that?
EDIT:
Thanks for the responses, but my question was a bit different: I would like to get decoded components in an unambiguous way, so neither of the following does what I need:
new URI("http://www.google.com?q=abc%26def").getRawQuery() returns encoded query: q=abc%26def
new URI("http://www.google.com?q=abc%26def").getQuery() returns ambiguous value: q=abc&def
URLDecoder.decode("http://www.google.com?q=abc%26def", "UTF-8") returns ambiguous value: http://www.google.com?q=abc&def
org.springframework.web.util.UriComponentsBuilder.fromUriString("http://www.google.com?q=abc%26def").build(true).getQueryParams() - close, but still not what I want, because it returns a map of encoded params: {q=[abc%26def]}

With spring framework (org.springframework.web.util) you can do the following:
URI uri = <your_uri_here>;
UriComponentsBuilder uriComponentsBuilder = UriComponentsBuilder.fromUri(uri);
UriComponents uriComponents = uriComponentsBuilder.build();
String path = uriComponents.getPath();
MultiValueMap<String, String> queryParams = uriComponents.getQueryParams(); //etc.

You could for example use an implementation of javax.ws.rs.core.UriInfo. One example would be org.jboss.resteasy.spi.ResteasyUriInfo. If you're using maven you only need to add the following to your pom.xml:
<dependency>
<groupId>org.jboss.resteasy</groupId>
<artifactId>resteasy-jaxrs</artifactId>
<version>3.0.6.Final</version>
</dependency>
Then the following code should do what you want:
UriInfo ui = new ResteasyUriInfo(new URI("http://www.google.com?q=abc%26def"));
List<String> qValues = ui.getQueryParameters().get("q");
for (String q : qValues) {
System.out.println(q);
}

Use the following:
String url = "http://www.google.com?test=34%3fg";
URL testUrl = new java.net.URL(url);
System.out.println(testUrl.getQuery());
Should print test=34%3fg.

URLDecoder does not split your URL into components, it simply translates the String representation thereof to a specific format, as hinted by it's JavaDoc and its signature, which returns a String. As others mentioned, you should just construct a URL object from your string, which exposes all the functionality you need. See here.

Generate a java.net.URL from your URL-String and then use mwthods like url.getQuery(), url.getProtocol(), url.getHost() etc. - it's all there.