I am connecting to a third party tool that uses brackets inside their url parameters. I'm proxying someone else parameters, so I'm not building these parameters up and don't want to have to parse them exactly.
I've tried a basic encoding of parameters, this fails due to the third party application not knowing how to parse the encoding, it tries to read the encoded values directly as far as I can tell.
I realize this is not exactly how resttemplate is designed to work, but everywhere else in our code uses restTemplate and I don't want to bring in a new service simply for a basic proxy.
Is there any way to make resttemplate allow the brackets through without trying to do substitution on them?
You can use escape URL codes for special characters in the URL. e.g. If your URL is http://domainURL/{url-part}/rest-of-the-url then you could refer to it as http://domainURL/%7Burl-part%7D/rest-of-the-url.
Below is the code snippet for the reference:
LoginDetails loginDetails = restTemplate.getForObject(restServiceURL +
"%7Btest-paranthesis%7D/" + userUUID, LoginDetails.class);
For the above code the URL is http://localhost:8087/userLogin{test-paranthesis}/842063819010
Related
I am working on a Spring Boot application
I need to make a request to an external service, old and ill-conceived. The request take the form of a HTTP GET (or POST) call, but the payload, an xml content, need to be passed as a query parameter. For example,
GET http://ill-service.com/plain.cgi?XML_DATA=<request attribute="attributeValue"><content contentAttribute="plain"/></request>
Of course, the value of query param XML_DATA need to be URL encoded, and normally, the RestTemplate of Spring boot work good on that, following RFC 3986 (see http://www.ietf.org/rfc/rfc3986.txt).
Except that, as allowed by this RFC, '/' and '=' character are left in the param value, giving me the following query :
GET http://ill-service.com/plain.cgi?XML_DATA=%3Crequest%20attribute=%22attributeValue%22%3E%3Ccontent%20contentAttribute=%22plain%22/%3E%3C/request%3E
In a perfect wold, this would be good, but do you remember when I said that the service I am trying to call is ill-conceived ? In another world, it needs to have the full content of XML_DATA URL-encoded. In another words, it needs the following query:
GET http://ill-service.com/plain.cgi?XML_DATA=%3Crequest%20attribute%3D%22attributeValue%22%3E%3Ccontent%20contentAttribute%3D%22plain%22%2F%3E%3C%2Frequest%3E%0A
I am quite lost on how to instruct the rest template or the UriComponentBuilder I am using to do so. Any help would be greatly appreciated
Probably u can use spring's UriUtils class
Use java.net.URLEncoder to encode your XML payload first and then append the encoded payload.
Following the suggestion of Vasif, and some information about UriComponentBuilder I found the following solutions :
String xmlContent = "<request attribute="attributeValue"><content contentAttribute="plain"/></request>";
URI uri = UriComponentsBuilder.fromHttpUrl("http://ill-service.com/plain.cgi")
//This part set the query param as a full encoded value, not as query value encoded
.queryParam("XML_DATA", UriUtils.encode(xmlContent, "UTF-8"))
//The build(true) indicate to the builder that the Uri is already encoded
.build(true).toUri();
String responseStr = restTemplate.getForObject(uri ,String.class)
I am trying to achieve same thing as this: How to use query parameter represented as JSON with Spring RestTemplate?, sending JSON string as a URL parameter in restTemplate.exchange().
The accepted answer mentions that sending JSON object as request parameter is generally not a good idea since you will probably face problem with curly brackets inside JSON. That is exactly what is happening when I am trying to make a GET call to an API. Since this is an API from another system, I cannot ask them to change the format and will have to call the GET endpoint, passing JSON as parameter. How can I achieve this in restTemplate.exchange() call?
Note: The mentioned related question does not guide on how to overcome this problem and I do not have enough reputation to comment on it to ask the author of the answer.
Answering my own question. While it is a bad idea to pass JSON like this in a query/url parameter, there is a workaround as suggested here: https://jira.spring.io/browse/SPR-9220?focusedCommentId=76760&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-76760.
Replicating the code here in case this link goes dead:
String url = "http://localhost:8983/solr/select?wt=json&indent=true&fl=*&q=*:*&fq={!geofilt}&sfield=venue_location&pt=28.0674,-80.5595&d=25";
URI uri = UriComponentsBuilder.fromUriString(url).build().encode().toUri();
System.out.println(uri);
// http://localhost:8983/solr/select?wt=json&indent=true&fl=*&q=*:*&fq=%7B!geofilt%7D&sfield=venue_location&pt=28.0674,-80.5595&d=25
Basically, instead of passing url having JSON query/url parameters as a string, pass it as a URI. Then call exchange method as before, but with URI instead of String:
restTemplate.exchange(uri, HttpMethod.GET, requestEntity, String.class)
If this is 3rd party API and you cannot control or change JSON processing on backend side - there is no solution. Even if you will encode with URLEncoder - there is no guarantee that API backend would process such request correctly.
You can use URLEncoder class to encode the URL in exchange method, e.g.:
String url = "http://www.yoursite.com/api?param={\"some_key\":\"some_value\"}";
System.out.println(URLEncoder.encode(url, StandardCharsets.UTF_8.name()));
This will encode the characters (like braces and double quotes) and server then will decode it back to json.
I'm struggling with handling special character in query parameter value while working with Rest Assured.
In url (as given below), I have to pass the value which is separated with pipe symbol '|'. I encoded symbol with value %7C however service call doesn't not give matching response instead returns default response.
http://localhost:8080/api/abc?Id=7325860%7CXYZ
Interesting part is same url works fine with any browser rest client or other java based solution.
REST Assured performs URL encoding for query parameters by default. You can easily disable it though:
given().urlEncodingEnabled(false).when().get("http://localhost:8080/api/abc?Id=7325860%7CXYZ");
See documentation for more info.
Is there a clean and spec-conformant way to define a custom URL scheme that acts as an adapter on the resource returned by another URL?
I have already defined a custom URL protocol which returns a decrypted representation of a local file. So, for instance, in my code,
decrypted-file:///path/to/file
transparently decrypts the file you would get from file:///path/to/file. However, this only works for local files. No fun! I am hoping that the URL specification allows a clean way that I could generalize this by defining a new URL scheme as a kind of adapter on existing URLs.
For example, could I instead define a custom URL scheme decrypted: that could be used as an adapter that prefixes another absolute URL that retrieved a resource? Then I could just do
decrypted:file:///path/to/file
or decrypted:http://server/path/to/file or decrypted:ftp://server/path/to/file or whatever. This would make my decrypted: protocol composable with all existing URL schemes that do file retrieval.
Java does something similar with the jar: URL scheme but from my reading of RFC 3986 it seems like this Java technology violates the URL spec. The embedded URL is not properly byte-encoded, so any /, ?, or # delimiters in the embedded URL should officially be treated as segment delimiters in the embedding URL (even if that's not what JarURLConnection does). I want to stay within the specs.
Is there a nice and correct way to do this? Or is the only option to byte-encode the entire embedded URL (i.e., decrypted:file%3A%2F%2F%2Fpath%2Fto%2Ffile, which is not so nice)?
Is what I'm suggesting (URL adapters) done anywhere else? Or is there a deeper reason why this is misguided?
There's no built-in adaptor in Cocoa, but writing your own using NSURLProtocol is pretty straightforward for most uses. Given an arbitrary URL, encoding it like so seems simplest:
myscheme:<originalurl>
For example:
myscheme:http://example.com/path
At its simplest, NSURL only actually cares if the string you pass in is a valid URI, which the above is. Yes, there is then extra URL support layered on top, based around RFC 1808 etc. but that's not essential.
All that's required to be a valid URI is a colon to indicate the scheme, and no invalid characters (basically, ASCII without spaces).
You can then use the -resourceSpecifier method to retrieve the original URL and work with that.
I want a user to be able to submit a url, and then display that url to other users as a link.
If I naively redisplay what the user submitted, I leave myself open to urls like
http://somesite.com' ><script>[any javacscript in here]</script>
that when I redisplay it to other users will do something nasty, or at least something that makes me look unprofessional for not preventing it.
Is there a library, preferably in java, that will clean a url so that it retains all valid urls but weeds out any exploits/tomfoolery?
Thanks!
URLs having ' in are perfectly valid. If you are outputting them to an HTML document without escaping, then the problem lies in your lack of HTML-escaping, not in the input checking. You need to ensure that you are calling an HTML encoding method every time you output any variable text (including URLs) into an HTML document.
Java does not have a built-in HTML encoder (poor show!) but most web libraries do (take your pick, or write it yourself with a few string replaces). If you use JSTL tags, you get escapeXml to do it for free by default:
ok
Whilst your main problem is HTML-escaping, it is still potentially beneficial to validate that an input URL is valid to catch mistakes - you can do that by parsing it with new URL(...) and seeing if you get a MalformedURLException.
You should also check that the URL begins with a known-good protocol such as http:// or https://. This will prevent anyone using dangerous URL protocols like javascript: which can lead to cross-site-scripting as easily as HTML-injection can.
I think what you are looking for is output encoding. Have a look at OWASP ESAPI which is tried and tested way to perform encoding in Java.
Also, just a suggestion, if you want to check if a user is submitting malicious URL, you can check that against Google malware database. You can use SafeBrowing API for that.
You can use apache validator URLValidator
UrlValidator urlValidator = new UrlValidator(schemes);
if (urlValidator.isValid("http://somesite.com")) {
//valid
}