In my Rest API it should be possible to retrieve data which is inside a bounding box. Because the bounding box has four coordinates I want to design the GET requests in such way, that they accept the bounding box as JSON. Therefore I need to be able to send and document JSON strings as URL parameter.
The test itself works, but I can not document these requests with Spring RestDocs (1.0.0.RC1). I reproduced the problem with a simpler method. See below:
#Test public void ping_username() throws Exception
{
String query = "name={\"user\":\"Müller\"}";
String encodedQuery = URLEncoder.encode(query, "UTF-8");
mockMvc.perform(get(URI.create("/ping?" + encodedQuery)))
.andExpect(status().isOk())
.andDo(document("ping_username"));
}
When I remove .andDo(document("ping_username")) the test passes.
Stacktrace:
java.lang.IllegalArgumentException: Illegal character in query at index 32: http://localhost:8080/ping?name={"user":"Müller"}
at java.net.URI.create(URI.java:852)
at org.springframework.restdocs.mockmvc.MockMvcOperationRequestFactory.createOperationRequest(MockMvcOperationRequestFactory.java:79)
at org.springframework.restdocs.mockmvc.RestDocumentationResultHandler.handle(RestDocumentationResultHandler.java:93)
at org.springframework.test.web.servlet.MockMvc$1.andDo(MockMvc.java:158)
at application.rest.RestApiTest.ping_username(RestApiTest.java:65)
After I received the suggestion to encode the URL I tried it, but the problem remains.
The String which is used to create the URI in my test is now /ping?name%3D%7B%22user%22%3A%22M%C3%BCller%22%7D.
I checked the class MockMvcOperationRequestFactory which appears in the stacktrace, and in line 79 the following code is executed:
URI.create(getRequestUri(mockRequest)
+ (StringUtils.hasText(queryString) ? "?" + queryString : ""))
The problem here is that a not encoded String is used (in my case http://localhost:8080/ping?name={"user":"Müller"}) and the creation of the URI fails.
Remark:
Andy Wilkinson's answer is the solution for the problem. Although I think that David Sinfield is right and JSONs should be avoided in the URL to keep it simple. For my bounding box I will use a comma separated string, as it is used in WMS 1.1: BBOX=x1,y1,x2,y2
You haven't mentioned the version of Spring REST Docs that you're using, but I would guess that the problem is with URIUtil. I can't tell for certain as I can't see where URIUtil is from.
Anyway, using the JDK's URLEncoder works for me with Spring REST Docs 1.0.0.RC1:
String query = "name={\"user\":\"Müller\"}";
String encodedQuery = URLEncoder.encode(query, "UTF-8");
mockMvc.perform(get(URI.create("/baz?" + encodedQuery)))
.andExpect(status().isOk())
.andDo(document("ping_username"));
You can then use URLDecoder.decode on the server side to get the original JSON:
URLDecoder.decode(request.getQueryString(), "UTF-8")
The problem is that URIs have to be encoded as ACII. And ü is not a valid ASCII character, so it must be escaped in the url with % escaping.
If you are using Tomcat, you can use URIEncoding="UTF-8" in the Connector element of the server.xml, to configure UTF-8 escaping as default. If you do this, ü will be automatically converted to %C3%BC, which is the ASCII representation of the \uC3BC Unicode code-point (which represents ü).
Edit: It seems that I have missed the exact point of the error, but it is still the same error. Curly braces are invalid in a URI. Only the following characters are acceptable according to RFC 3986:
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]#!$&'()*+,;=%
So these must be escaped too.
Related
I have a problem calling WS.url() in play framework 2.3.3 with url containing spaces. All other characters all url encoded automatically but not spaces. When i try to change all spaces to "%20", WS convert it to "%2520" because of "%" character. With spaces i've got java.net.URISyntaxException: Illegal character in query. How can i handle this ?
part of the URL's query String:
&input=/mnt/mp3/music/folder/01 - 23.mp3
The code looks like this:
Promise<JsonNode> jsonPromise = WS.url(url).setAuth("", "cube", WSAuthScheme.BASIC).get().map(
new Function<WSResponse, JsonNode>() {
public JsonNode apply(WSResponse response) {
System.out.println(response.getBody());
JsonNode json = response.asJson();
return json;
}
}
);
You should "build" your URL based on the way java.net.URL(which Play! uses for it's WS) does it. WS.url() follows the same logic.
The use of URLEncoder/Decoder is recommended only for form data.
From JavaDoc:
"Note, the java.net.URI class does perform escaping of its component
fields in certain circumstances. The recommended way to manage the
encoding and decoding of URLs is to use java.net.URI, and to convert
between these two classes using toURI() and URI.toURL(). The
URLEncoder and URLDecoder classes can also be used, but only for HTML
form encoding, which is not the same as the encoding scheme defined
in RFC2396."
So, the solution is to use THIS:
WS.url(baseURL).setQueryString(yourQueryString);
Where:
baseURL is your scheme + host + path etc.
yourQueryString is... well, your query String, but WITHOUT the ?: input=/mnt/mp3/music/folder/01 - 23.mp3
Or, if you want to use a more flexible, programmatic approach, THIS:
WS.url(baseURL).setQueryParameter(param, value);
Where:
param is the parameter's name in the query String
value is the value of the parameter
If you want multiple parameters with values in your query you need to chain them by adding another .setQueryParameter(...). This implies that this approach is not very accomodating for complex, multi-parameter query Strings.
Cheers!
If you check the console you will find that the exception is : java.net.URISyntaxException: Illegal character in path at index ...
That's because play Java api uses java.net.URL (as you can see here in line 47).
You can use java.net.URLEncoder to encode your URL
WS.url("http://" + java.net.URLEncoder.encode("google.com/test me", "UTF-8"))
UPDATE
If you want an RFC 2396 compliant method you can do this :
java.net.URI u = new java.net.URI(null, null, "http://google.com/test me",null);
System.out.println("encoded url " + u.toASCIIString());
I'm trying to get an url parameter in jee.
So I have this kind of url :
http://MySite/MySite.jsp?page=recherche&msg=toto
First i tried with : request.getParameter("msg").toString();
it works well but if I try to search "c++" , the method "getParameter()" returns "c" and not "c++" and i understand.
So I tried another thing. I get the current URL and parse it to get the value of the message :
String msg[]= request.getQueryString().split("msg=");
message=msg[1].toString();
It works now for the research "c++" but now I can't search accent. What can I do ?
EDIT 1
I encode the message in the url
String urlString=Utils.encodeUrl(request.getParameter("msg"));
so for the URL : http://MySite/MySite.jsp?page=recherche&msg=c++
i have this encoded URL : http://MySite/MySite.jsp?page=recherche&msg=c%2B%2B
And when i need it, i decode the message of the URL
String decodedUrl = URLDecoder.decode(url, "ISO-8859-1");
Thanks everybody
Anything you send via "get" method goes as part of the url, which needs to be urlencoded to be valid in case it contains at least one of the reserved characters. So, any character will need to be encoded before sending.
In order to send c++, you would have to send c%2B%2B. That would be interpreted properly at the server side.
Here some reference you can check:
http://www.blooberry.com/indexdot/html/topics/urlencoding.htm
Now the question is, how and where do you generate your URL? According to the language, you will need to use the proper method to encode your strings.
if I try to search "c++" , the method "getParameter()" returns "c" and not "c++"
Query parameters are treated as application/x-www-form-urlencoded, so a + character in the URL means a space character in the parameter value. If you want to send a + character then it needs to be encoded in the URL as %2B:
http://MySite/MySite.jsp?page=recherche&msg=c%2B%2B
The same applies to accented characters, they need to be escaped as the bytes of their UTF-8 representation, so été would need to be:
msg=%C3%A9t%C3%A9
(é being Unicode character U+00E9, which is C3 A9 in UTF-8).
In short, it's not the fault of this code, it's the fault of whatever component is responsible for constructing the URL on the client side.
Call your URL with
msg=c%2B%2B
+ in a URL mean 'space'. It needs to be escaped.
You need to escape special characters when passing them as URL parameters. Since + means space and & means and another parameter, these cannot be used as parameter values.
See this other S.O. question.
You may want to use the Apache HTTP client library to help you with the URL encoding/decoding. The URIUtil class has what you need.
Something like this should work:
String rawParam = request.getParameter("msg");
String msgParam = URIUtil.decode(rawParam);
Your example indicates that the data is not being properly encoded on the client side. See this JavaScript question.
I have a servlet running on tomcat 6 which should be called as follows:
http://<server>/Address/Details?summary="Acme & co"
However: when I iterate through the parameters in the servlet code:
//...
while (paramNames.hasMoreElements()) {
paramName = (String) paramNames.nextElement();
if (paramName.equals("summary")) {
summary = request.getParameter(paramName).toString();
}
}
//...
the value of summary is "Acme ".
I assume tomcat ignores the quotes - so it sees "& co" as a second parameter (albeit improperly formed: there's no =...).
So: is there any way to avoid this? I want the value of summary to be "Acme & co". I tried replacing '&' in the URL with & but that doesn't work (presumably because it's decoded back to a straight '&' before the params are parsed out).
Thanks.
Use http://<server>/Address/Details?summary="Acme %26 co". Because in URL special http symbol(e.g. &,/, //) does not work as parameters.
Are you encoding and decoding the URL with URLEncode ? If so, can you check what the input and output of those are ? Seems like one of the special characters is not being properly encoded/decoded
Try %26 for the &
Try your parameter like
summary="Acme & co"
& is part reserved characters. Refer RFC2396 section
2.2. Reserved Characters.
how to encode URL to avoid special characters in java
Characters allowed in GET parameter
HTTP URL - allowed characters in parameter names
http://illegalargumentexception.blogspot.in/2009/12/java-safe-character-handling-and-url.html
I have a request,In Browser address bar enter:
http://localhost:8888/cmens-tops-outwear/t-b-f-a-c-s-fLoose-p-g-e-i-o.htm?'"--></style></script><script>netsparker(0x0000E1)</script>=
Tomcat6.0.35 i have set URIEncoding="UTF-8"
Use request.getQueryString() in servlet:
if chrome,i get
'%22--%3E%3C/style%3E%3C/script%3E%3Cscript%3Enetsparker(0x0000E1)%3C/script%3E=
if ie,I get
'"--></style></script><script>netsparker(0x0000E1)</script>=
Why?
Additional
I want to get request.getQueryString() to create a uri
URI uri = URI.create(url)
if ie:
java.net.URISyntaxException: Illegal character in query at index 36: /cmens/t-b-f-a-c-s-f-p-g-e-i-o.htm?'"--></style></script><script>netsparker(0x0000E1)</script>
at java.net.URI$Parser.fail(URI.java:2809)
at java.net.URI$Parser.checkChars(URI.java:2982)
at java.net.URI$Parser.parseHierarchical(URI.java:3072)
at java.net.URI$Parser.parse(URI.java:3024)
at java.net.URI.<init>(URI.java:578)
at java.net.URI.create(URI.java:840)
How to determine the queryString whether has be encoded?
The HttpServletRequest#getQueryString() is per definition undecoded. See also the javadoc (emphasis mine):
Returns:
a String containing the query string or null if the URL contains no query string. The value is not decoded by the container.
Basically, you need to URL-decode it yourself if you'd like to parse it manually instead of using getParameterXxx() methods for some reason (which implicitly decodes the parameters!).
String decodedQueryString = URLDecoder.decode(request.getQueryString(), "UTF-8");
As to why Chrome sends it encoded while IE not, that's because Chrome is doing a better job of handling HTTP requests the safe/proper way. This is beyond your control. Just always URL-decode the query string yourself if you intend to parse it manually for some reason. The URIEncoding="UTF-8" configuration has only effect on getParameterXxx() methods during GET requests.
The Chrome version is URLEncoded while the IE string is decoded.
Use this tool to compare the URLEncoded and decoded versions: http://meyerweb.com/eric/tools/dencoder/
Chrome uses the URL encoding way, but IE is using strings.
For example: " is %22 in URL encoding.
< is %3E
and > is %3C
Chrome is doing it the "right way" but IE just can't do as all the others.
You can find complete list of URL characters here: http://www.w3schools.com/tags/ref_urlencode.asp
Chrome sends the url encoded. Try decoding the query string using
URLDecoder.decode(queryString, "UTF-8");
As stated by the javadoc, the query string is not decoded by the container:
returns a String containing the query string or null if the URL contains no query string. The value is not decoded by the container.
javadoc
I am using jsps and in my url I have a value for a variable like say "L & T". Now when I try to retrieve the value for it by using request.getParameter I get only "L". It recognizes "&" as a separator and thus it is not getting considered as a whole string.
How do I solve this problem?
java.net.URLEncoder.encode("L & T", "utf8")
this outputs the URL-encoded, which is fine as a GET parameter:
L+%26+T
A literal ampersand in a URL should be encoded as: %26
// Your URL
http://www.example.com?a=l&t
// Encoded
http://www.example.com?a=l%26t
You need to "URL encode" the parameters to avoid this problem. The format of the URL query string is:
...?<name>=<value>&<name>=<value>&<etc>
All <name>s and <value>s need to be URL encoded, which basically means transforming all the characters that could be interpreted wrongly (like the &) into %-escaped values. See this page for more information:
http://www.w3schools.com/TAGS/ref_urlencode.asp
If you're generating the problem URL with Java, you use this method:
String str = URLEncoder.encode(input, "UTF-8");
Generating the URL elsewhere (some templates or JS or raw markup), you need to fix the problem at the source.
You can use UriUtils#encode(String source, String encoding) from Spring Web. This utility class also provides means for encoding only some parts of the URL, like UriUtils#encodePath.