public Response getCustomerByName(
#PathParam("customerName") String customerName)
Problem :
I am passing customerName as : stack overflow (URL is encoded as : stack%20overflow). I want to receive as decoded string (stack overflow, without %20) in my java code.
What I tried :
This works perfectly fine, but I felt it is not more generic way of doing it.
URLDecoder.decode(customerName, "UTF-8");
Require more generic solution :
I want to do the similar changes in rest of the APIs as well, so using URLDecoder in each API is burden . Is there any common practice which I can follow to impose this decoding at application level? (#PathParam is already decoded when I receive the request)
It shall be auto "Decoded" and you don't need explicit decoding using URLDecoder.decode(customerName, "UTF-8");
As mentioned in javadoc of PathParam javadoc:
The value is URL decoded unless this is disabled using the Encoded annotation.
I just verified below and it works as per javadoc (in weblogic server)
#GET
#Produces(value = { "text/plain"})
#Path("{customerName}")
public Response getCustomerByName(#PathParam("customerName") String customerName) {
System.out.println(customerName);
return Response.ok().entity(customerName).type("text/plain").build();
}
Related
I've hit quite an issue with query parameters in Spring RestTemplate.
When I pass a URL to .exchange as a String, then the URL is used as given, and the query parameters reach the other end (Spring #GetMapping endpoint's #RequestParam) encoded.
When I pass it as URI(url) (there's an overload), the query param values are unescaped / unencoded.
This was my original code. The value of myTest intended to be transferered is a+b.
val url = "http://localhost:$port/foo?myTest=a%2Bb"
val requestEntity = RequestEntity<Any>(Headers.bearerAuth(token), GET, URI(url))
val response = restTemplateTrustStore.exchange(requestEntity, String::class.java)
This was unescaping the URL and the value of myTest came to the #RequestParam as a+b, so after "manual urlDecoding", a b. So I rewrote to:
val requestEntity = RequestEntity<Any>(Headers.bearerAuth(spokeToken), null, null)
val response = restTemplateTrustStore.exchange(url, GET, requestEntity, String::class.java)
That does not decode the URL, and it arrives to #RequestParam as a%2Bb, so after "manual urlDecoding", a+b.
Why is this different? The URI#toString() displays the URL properly. So I attribute this to Spring's handling of URI() vs. String.
Also. I would expect #RequestParam to decode the value for me, but it seems it does not. Or if it does, then the question "shifts left" - I am URL-decoding an already URL-decoded value, and Spring is encoding it second time on the sending side when using String url but not encoding when using URI(url). I'll check soon which one it is.
Note that the URI/URL encoding is quite complicated topic full of misconceptions, so I might be following one. But AFAIK, the pluses in query parameters mean spaces, and an actual plus should be encoded as %xy. Originally I was using URIBuilder, but that was not handling this properly and the result was an invalid URL being sent if the param name or value included the "unprotected" characters.
My RestTemplate factory is quite basic for this test case:
#Bean fun restTemplate(builder: RestTemplateBuilder) = builder.build()
I have also tried one which creates a basic RestTemplate without the default goodies:
restTemplateBuilder
.requestFactory(HttpComponentsClientHttpRequestFactory::class.java)
.build()
But the result is the same.
Can someone enlighten me on what's Spring's idea on this, or what am I doing wrongly?
Edit: I found org/springframework/web/util/DefaultUriTemplateHandler.java which does this:
protected UriComponents expandAndEncode(UriComponentsBuilder builder, Map<String, ?> uriVariables) {
if (!isStrictEncoding()) {
return builder.buildAndExpand(uriVariables).encode();
}
else {
Map<String, ?> encodedUriVars = UriUtils.encodeUriVariables(uriVariables);
return builder.buildAndExpand(encodedUriVars);
}
}
Which is similar to what I was previously trying to do with the URI - putting it together using UriBuilder, but resorted to doing it by my code as it could not escape one character and not escape another; the value decoded on the #RequestParam side was half-wrong.
I am working on a project in that java script as front-end and java as back-end are used, my problem is that I want to pass some string using restangular calls to my back-end resources. If passing parameter have space between the string then I got 500 (Server Error) before reaching to back-end resource side.
Lets take example :
At Java Script : RESTangular call
var myRestCall= Restangular.all('myRoot/myMethod/'+myLocalPath+'/'+folderName);
restPSTFolders.getList().then(function(listPSTFolders){
//my stuff
});
At Java Resource :
#ApiOperation(value = "My Method",
notes = "Returns My Method list",
responseContainer = "List",
response = List.class)
#Path("/myMethod/{myLocalPath}/{folderName}")
#GET
#Transactional
#Timed
public List myMethod(#PathParam("myLocalPath") String myLocalPath, #PathParam("folderName") String sFolderName) {
//my stuff
}
In my example myLocalPath parameter can have spaces and special characters in the string as it can be any :
C:\MY DRIVE\My Path One\My Path
D:\My favorite
To pass this to back-end class from RESTangular call, I need to replace all spaces with some character, it work for me, but I am not thinking its a good way to encode the special character and space with any character because the replacing character can also be a part of existing path then at back-end on again replacing the character, might change the path.
EDIT : If I passed the parameter as json object:
var parameterJsonPath = {};
parameterJsonPath={"myLocalPath": pathValue};
var myRestCall= Restangular.all('myRoot/myMethod/'+parameterJsonPath+'/'+folderName);
Does not make any sense as I got : ../myRoot/myMethod/%5Bobject%20Object%5D Failed to load resource: the server responded with a status of 500 (Server Error)
And making the parameterJsonPath to JSON.stringify(parameterJsonPath); will pass this as string that of no use for me.
Thus in all, is there any good way to encode the special character of string in js, so at back-end side I could decode that string using same key that were used while encoding?
I am currently properly escaping my filters, either using Spring LDAP Filter clases, or by going through LdapEncoder.filterEncode().
At the same time, I am using WireShark to capture packets being exchanged between my local machine and the LDAP server.
And I seem to have a problem. Even if I properly escape values (which I have confirmed through debugging), they come out unescaped through the network. I have also confirmed (through debugging) that the value stays encoded all the way until it enters javax.naming.InitialContext.
Here is an example (note that I am using Spring LDAP 1.3.0, and that these happen on both Oracle JDK 6u45 and Oracle JDK 7u45).
In my own code, on the service layer, the call being made is:
String lMailAddress = (String) ldapTemplate.searchForObject("", new EqualsFilter(ldapUserSearchFilterAttribute, principal).encode(), new ContextMapper() {
#Override
public Object mapFromContext(Object ctx) {
DirContextAdapter lContext = (DirContextAdapter) ctx;
return lContext.getStringAttribute("mail");
}});
At this point, I can confirm that the String returned by the encode() method on the filter is "(sAMAccountName=boi\2a)"
The last point I can debug the code is the following one (starts at line 229 of org.springframework.ldap.core.LdapTemplate):
SearchExecutor se = new SearchExecutor() {
public NamingEnumeration executeSearch(DirContext ctx) throws javax.naming.NamingException {
return ctx.search(base, filter, controls);
}
};
When executeSearch() is later invoked, I can also verify that the filter String contains "(sAMAccountName=boi\2a)".
I cannot debug any further, since I do not have the source code to javax,naming.* or com.sun.jndi.ldap.* (since com.sun.jndi.ldap.LdapCtx is being invoked).
However, as soon as the call returns from executeSearch(), WireShark informs me that an LDAP packet containing a searchRequest with the filter "(sAMAccountName=boi*)" has been transmitted (the * is no longer escaped).
I have used similar encoding and used different methods of LdapTemplate that yielded the result I was expecting (I saw the encoded filter being transmitted in WireShark), but I cannot explain why, in the case I just exposed, the value gets decoded before being transmitted.
Please help me understanding the situation. Hpoefully, I am the one who does not properly understand the LDAP protocol here.
Thanks.
Disclaimer: I have posted the same question to Spring LDAP forums.
TL/DR: Why is com.sun.jndi.ldap.LdapCtx decoding LDAP encoded filters (like \2a to *) before transmitting them to the LDAP server?
Update: Tried and observed the same behavior with IBM's J9 JDK7.
Although I'm not familiar with Spring LDAP, it doesn't sound like there's necessarily a reason to be concerned. LDAP filters aren't transmitted as clear text, but rather in a binary encoding, and there is no need for escaping in this mechanism (nor would it be correct to do so).
Let's take "(sAMAccountName=boi*)" as an example. As written, this filter is a substring filter with a subInitial component of "boi". As you point out, if you want it to be an equality filter rather than a substring filter, then the string representation would have to be "(sAMAccountName=boi\2a)". However, the binary encodings for these filters don't use any escaping, but instead use an ASN.1 BER type to differentiate between substring and equality filters.
If you want "(sAMAccountName=boi*)" as a substring filter, then the encoded representation would be:
a417040e73414d4163636f756e744e616d6530058003626f69
On the other hand, if you want "(sAMAccountName=boi\2a)" as an equality filter, the encoding would be:
a316040e73414d4163636f756e744e616d650404626f692a
The full explanation of the encoding isn't something I want to get into, but the "a4" at the beginning of the first one indicates that it's a substring filter, whereas the "a3" at the beginning of the second indicates that it's an equality filter.
You should be able to verify the actual bytes sent in WireShark. It may well be that WireShark doesn't properly escape the filter when generating the string representation, but that would be an issue with WireShark itself. The directory server only gets the binary representation, and it's hard to believe that an LDAP server would misinterpret that.
OWASP suggest to encode strings for searches:
public static final String escapeLDAPSearchFilter(String filter) {
StringBuffer sb = new StringBuffer(); // If using JDK >= 1.5 consider using StringBuilder
for (int i = 0; i < filter.length(); i++) {
char curChar = filter.charAt(i);
switch (curChar) {
case '\\':
sb.append("\\5c");
break;
case '*':
sb.append("\\2a");
break;
case '(':
sb.append("\\28");
break;
case ')':
sb.append("\\29");
break;
case '\u0000':
sb.append("\\00");
break;
default:
sb.append(curChar);
}
}
return sb.toString();
}
DN strings are escaped different. See the link below.
https://www.owasp.org/index.php/Preventing_LDAP_Injection_in_Java
The best way is to use parameterized filter search method, thus the parameter will be properly encoded.
See https://docs.oracle.com/javase/jndi/tutorial/ldap/search/search.html
// Perform the search
NamingEnumeration answer = ctx.search("ou=NewHires",
"(&(mySpecialKey={0}) (cn=*{1}))", // Filter expression
new Object[]{key, name}, // Filter arguments
null); // Default search controls
I have my application url which I am sending the the end user on their emails.
Now that url contains the 'username' field, which can contains '#' character.
For e.g. link which sent to the end user :
http://localhost:8080/my-app/someaction/activateuser/abc#def.com/somedata/
Now whenever user clicks on above link, its throwing following exception :
java.lang.IllegalArgumentException
Input string 'abc#def.com' is not valid; the character '#' at position 4 is not valid.
at org.apache.tapestry5.internal.services.URLEncoderImpl.decode(URLEncoderImpl.java:144)
at $URLEncoder_137022607d9.decode($URLEncoder_137022607d9.java)
at org.apache.tapestry5.internal.services.ContextPathEncoderImpl.decodePath(ContextPathEncoderImpl.java:92)
at $ContextPathEncoder_137022607cd.decodePath($ContextPathEncoder_137022607cd.java)
at org.apache.tapestry5.internal.services.ComponentEventLinkEncoderImpl.checkIfPage(ComponentEventLinkEncoderImpl.java:328)
at org.apache.tapestry5.internal.services.ComponentEventLinkEncoderImpl.decodePageRenderRequest(ComponentEventLinkEncoderImpl.java:307)
at org.apache.tapestry5.internal.services.linktransform.LinkTransformerInterceptor.decodePageRenderRequest(LinkTransformerInterceptor.java:68)
at $ComponentEventLinkEncoder_137022607c1.decodePageRenderRequest($ComponentEventLinkEncoder_137022607c1.java)
at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:41)
at $Dispatcher_137022607c2.dispatch($Dispatcher_137022607c2.java)
at $Dispatcher_137022607bd.dispatch($Dispatcher_137022607bd.java)
at org.apache.tapestry5.services.TapestryModule$RequestHandlerTerminator.service(TapestryModule.java:321)
at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26)
Is there any way to handle such scenario, like encoding/decoding the urls ?
You cannot have an # in the url, because it's a reserved character (the specific RFC is RFC 3986).
You can use the URLEncoder class to encode the url to an acceptable value
As MiniBill has already answered, that can't work, and as Howard has added, Tapestry has its own encoder for URLs. This means that the easiest way for you to get a URL in the format that Tapestry can read is to have Tapestry create it, and then pass it to the component that sends your emails:
#Inject
private LinkSource linkSource;
#OnEvent(...)
void sendActivationEmail() {
final Link activationLink = this.createUserActivationLink(email, otherStuff);
this.activationEmailSender.sendWithActivationLink(email, activationLink);
}
private Link createUserActivationLink(String email, String otherStuff) {
return linkSource.createPageRenderLink(
"someaction/activateuser", false, email, otherStuff);
}
I was able to solve the problem by encoding my string to Base64, and unpacking on Tapestry Java side. My strings were of UTF-8 encoded characters.
I modified the Base64 encoder from this answer: https://stackoverflow.com/a/40392850/5339857
function b64EncodeUnicode(str) {
return btoa(encodeURIComponent(str).replace(/%([0-9A-F]{2})/g, function(match, p1) {
return String.fromCharCode('0x' + p1);
})).replace(/\=+$/, '');
}
(just added the .replace in the end, to remove padding =s that Tapestry doesn't like)
And in the Java side the decoding was a breeze: (this example is of an ajax click from javascript - where the Base64 encoding happens)
#OnEvent(value = "clickAjax")
Object clickAjax(String parameter) {
somePagePropetry = new String(java.util.Base64.getDecoder().decode(parameter));
return this;
}
I want to send a URI as the value of a query/matrix parameter. Before I can append it to an existing URI, I need to encode it according to RFC 2396. For example, given the input:
http://google.com/resource?key=value1 & value2
I expect the output:
http%3a%2f%2fgoogle.com%2fresource%3fkey%3dvalue1%2520%26%2520value2
Neither java.net.URLEncoder nor java.net.URI will generate the right output. URLEncoder is meant for HTML form encoding which is not the same as RFC 2396. URI has no mechanism for encoding a single value at a time so it has no way of knowing that value1 and value2 are part of the same key.
Jersey's UriBuilder encodes URI components using application/x-www-form-urlencoded and RFC 3986 as needed. According to the Javadoc
Builder methods perform contextual encoding of characters not permitted in the corresponding URI component following the rules of the application/x-www-form-urlencoded media type for query parameters and RFC 3986 for all other components. Note that only characters not permitted in a particular component are subject to encoding so, e.g., a path supplied to one of the path methods may contain matrix parameters or multiple path segments since the separators are legal characters and will not be encoded. Percent encoded values are also recognized where allowed and will not be double encoded.
You could also use Spring's UriUtils
I don't have enough reputation to comment on answers, but I just wanted to note that downloading the JSR-311 api by itself will not work. You need to download the reference implementation (jersey).
Only downloading the api from the JSR page will give you a ClassNotFoundException when the api tries to look for an implementation at runtime.
I wrote my own, it's short, super simple, and you can copy it if you like:
http://www.dmurph.com/2011/01/java-uri-encoder/
It seems that CharEscapers from Google GData-java-client has what you want. It has uriPathEscaper method, uriQueryStringEscaper, and generic uriEscaper. (All return Escaper object which does actual escaping). Apache License.
I think that the URI class is the one that you are looking for.
Mmhh I know you've already discarded URLEncoder, but despite of what the docs say, I decided to give it a try.
You said:
For example, given an input:
http://google.com/resource?key=value
I expect the output:
http%3a%2f%2fgoogle.com%2fresource%3fkey%3dvalue
So:
C:\oreyes\samples\java\URL>type URLEncodeSample.java
import java.net.*;
public class URLEncodeSample {
public static void main( String [] args ) throws Throwable {
System.out.println( URLEncoder.encode( args[0], "UTF-8" ));
}
}
C:\oreyes\samples\java\URL>javac URLEncodeSample.java
C:\oreyes\samples\java\URL>java URLEncodeSample "http://google.com/resource?key=value"
http%3A%2F%2Fgoogle.com%2Fresource%3Fkey%3Dvalue
As expected.
What would be the problem with this?