We are using Apache Axis 2 in our product. We have exposed some services to the client. Client is hitting the service with UTF-8 encoding of both request parameters key and values. But the values are receiving to the serverend as it is. Consider the below example.
Client: https://abc.xyz.com/service?key=value%20value
Service(server end) : value will be same like value%20value but we are expecting the "value value".
We want to handle the special characters (/,\, space, # , #, &)
Please help if I am missing anything?
Related
I'm trying to create a GET call in Java on an API with a path variable containing an illegal character.
GET http://api.example.com/key/foo<bar
The path contains the character <. The HTTP client encodes it as %3C but I need to keep it as <, because API server don't decode it.
I have to do this call, and the API provider doesn't want to change anything to be more compliant.
Calling the GET request with illegal character works with Postman.
I'm trying to do it in Java now.
I've tried with Feign, HttpClient, WebClient and OkHttp, I don't find a way to do it.
Do you know a Java Http client that will allow me to do it ? Or the proper encoding option for the clients above ?
I've tried to create a RequestInterceptor with Feign to update uri, but it works only for some characters, as defined in RFC 6570, but not for <, which is re-replaced with %3C after my replacement.
Thanks
I have a spring boot application with a GET service.
#RequestMapping(value = "/abc/track/{type}", method = RequestMethod.GET)
public void DummFunc(
#RequestParam(value="subs", required = false) String sub,
, HttpServletResponse response) {}
value for subs is an encoded value.
If I pass following as value to parameter subs
{%22endpoint%22:%22https://def.abc.com/tyu/send/eD3vpGNQW28:APA91bHOo8rYrV0xccdQz3okjZJG-QGrJX8LG6ahJnEUpMNfGedqi3hJxQsJx_8BMbH6oDjaSPPEXqzNWchWrGSkhuTkSdemikkys1U22Ipd7MsRw0owWbw89V2fslIVAJ6G5lkyvYuQ%22,%22expirationTime%22:null,%22keys%22:{%22p256dh%22:%22BK0pktn50CMsTQtqwdPlKlJtdFs0LFeXX14T1zgoz6QWnvSTp5dxtChnUP5P1JX0TsjcopbdPKyN31HfABMUyic%22,%22auth%22:%22qbO_z0vGao9wD-E5g8VU-A%22}}
It fails to capture the request and control does not come inside of the function.
If we instead pass as value to parameter subs:
%7B%22endpoint%22:%22https://def.abc.com/tyu/send/dX5q5eV7hFQ:APA91bHib-0QXrMzjatcvTR_uaIeeJK8lf6GmXUC9Jxv0Oxth-BzD4GmWnd4-YpDZv8qSFZ0eSg9mB2YkRvkc5ezdXW5KeaHjuQZfdyDxyBXjJgE-25Xbtlk37pdm8vfLk20k0k_VxW9%22,%22expirationTime%22:null,%22keys%22:%7B%22p256dh%22:%22BCCvcBLpRqp4u0auP688_MUJLsAiwWlQSn5kpyl8YVsEo_J-KpSdnhCmVIE_BhDXBcFYflPK52hqhYf3EaOCyuY%22,%22auth%22:%22iKuW_ESkCZnubWcQu_JK8w%22%7D%7D
It works fine.
Why is this happening? What's wrong with first encoding?
Since server is not able to handle the request, it returns 400. I need to capture such requests and then handle them by encoding them properly. What can be way forward?
I am new to Spring boot/Spring and Java itself. Would be great if I can get some insight.
Also, I can decode both of them online here without any issues- https://www.urldecoder.org/
Edit: Basically, the request that has issue getting through has { and } instead of %7Band %7D.
My question is instead of application failing with 400 bad request,how do I capture such requests in my app, encode them properly and then process them.
This is not related to Java nor the Spring itself but the HTML URL Encoding Reference. URLs can only be sent over the Internet using the ASCII character set.
The unsafe characters are defined in the beginning of RFC-1738 and here is the list:
" < > # % { } | \ ^ ~ [ ] ` including the blank/empty space.
Aside from those, there are also reserved characters where belong the following ones and are used to distinguish the parameters, the key-value representation, the port etc.
; / ? : # = &
The unsafe characters you have used are { and } which are equal to %7B and %7D.
Essentially, you should not be concerned about the data the client sends you in the way you describe. The server must demand the correct form and URL passed. Although, the browsers and REST clients encode those characters automatically, sending them programmatically might cause errors. The only two available solutions in Spring I am aware of is through registering the CharacterEncodingFilter bean (already answered) or the Spring-Boot configuration:
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
You need to enable the encoding first and force on HTTP requests and responses.
spring-boot is very much concerned about security. Adding double quote / single quotes or either escaping won't work I guess.
Please go through : https://www.rfc-editor.org/rfc/rfc1738
I think you should try the manual encoding { to %7B and } to %7D
Unsafe:
Characters can be unsafe for a number of reasons. The space
character is unsafe because significant spaces may disappear and
insignificant spaces may be introduced when URLs are transcribed or
typeset or subjected to the treatment of word-processing programs.
The characters "<" and ">" are unsafe because they are used as the
delimiters around URLs in free text; the quote mark (""") is used to
delimit URLs in some systems. The character "#" is unsafe and should
always be encoded because it is used in World Wide Web and in other
systems to delimit a URL from a fragment/anchor identifier that might
follow it. The character "%" is unsafe because it is used for
encodings of other characters. Other characters are unsafe because
gateways and other transport agents are known to sometimes modify
such characters. These characters are "{", "}", "|", "", "^", "~",
"[", "]", and "`".
All unsafe characters must always be encoded within a URL. For
example, the character "#" must be encoded within URLs even in
systems that do not normally deal with fragment or anchor
identifiers, so that if the URL is copied into another system that
does use them, it will not be necessary to change the URL encoding.
Generally this issue comes when the APP server like Undertow, tomcat, Jboss which is hosting your Springboot application does not allow some special characters, which can very from Server to server. This is done to secure URLs and no one can send some special characters to the server to compromise with its functionality.
If still want to allow special characters so that those can reach to controller in Springboot application, you need to allow this configuration in APP server. e.g. this is the configuration required in Undertow server to allow special characters in URL:
#Configuration
public class HttpConfig {
#Bean
public UndertowServletWebServerFactory servletWebServerFactory() {
UndertowServletWebServerFactory factory = new UndertowServletWebServerFactory();
factory.addBuilderCustomizers((UndertowBuilderCustomizer) builder ->
builder.setServerOption(UndertowOptions.ALLOW_UNESCAPED_CHARACTERS_IN_URL, Boolean.TRUE));
return factory;
}
}
In order to make springboot work well with UTF-8 encoding/decoding of rest URLs, add this configuration in application.properties file, for springboot version 2.6:
server.servlet.encoding.charset=UTF-8
server.servlet.encoding.enabled=true
server.servlet.encoding.force=true
server.servlet.encoding.force-request=true
server.servlet.encoding.force-response=true
Best approach is to encode special characters in the client application which is calling this rest URL to comply with security principles
Due to Tomcat upgraded their security in URL param.
Please find possible solution
https://github.com/bohnman/squiggly/issues/42#issuecomment-386658525
I am having a RESTful Java Web Service that accepts a long string with '#' in between.
When I am trying to send the string to the method while calling, the string is getting split on '#' and I can retrieve the [0] value alone.
Before sending the message is intact, but after using this..
req.open("GET","https://localhost:8443/registername/resources/registerName/"+"My#Name", true);
req.send();
is the problem.
These are the first few lines in the Web Service...
#GET
#Path("/{message}")
public String validateName(#PathParam("message") String message) throws Exception{
System.out.println(message);
...}
And, it displays "My" alone.
Can anyone please help me on why this is happening? Thanks!
In URLs, a # sign indicates a "named anchor," something that local javascript, and it is not sent to the remote server, so when you have the URL:
https://localhost:8443/registername/resources/registerName/My#Name
Name isn't sent to the server. You need to use a different split character.
See What is it when a link has a pound "#" sign in it or http://www.hypergurl.com/anchors.html for more information.
HTTP post or get will not read anything after #,
Do a URLEncode before doing POST or GET.
I don't think the part after the # is sent up with the GET request.
I'm having a little problem. I'm building a small server in java, based on jetty websockets implementations.
The clients are the browsers and I send information using the websockets javascript api.
Everything works great until I send those special characters such as : ă Ț î ș ê ñ ü
So here is the problem. Client 1 sends a message to the server with one of this characters. Server prints the message and then send the message to client 2.
Client 2 receives the message and prints the message on a browser html page and works great The characters are showed correctly.
The problem is when I wanna print the String on the server site. Instead of ă is shows me the ? char. This is causing me problems because I want to insert the text in a database(mysql- with ut8 encoding enabled)
So.. what seems to be problem. The text that is send from the browser is not UT8 encoded? or the jetty websocket implementation is not receiving String in utf8 encoding??
Thanks
Here's a function I use to HTML-encode all special characters in a string (but not html itself (like < or >)). If you apply it before sending a string to the server, everybody should see the same and you can store it in a database table:
function toHtmlEncoded(string){
return string.replace(/[\u0080-\uC350]/g,
function(a) {return '&#'+a.charCodeAt(0)+';';}
);
}
First read this http://kunststube.net/encoding/
Then check everywhere you've converted bytes into Strings (or the reverse). Common places to make a mistake include calling getBytes() on a String without specifying an encoding. Other pitfalls include not setting the encoding in the database connection string.
I'm using the client's browser to submit HTTP request.
For report generation the securityToken is submitted as POST, for report download the same token needs to be submitted by the user browser, this time using GET.
What encoding would you recommend for the securityToken which actually represents encrypted data.
I've tried BASE64 but this fails because the standard can include the "+" character which gets translated in HTTP GET to ' ' (blank space).
Then I tried URL Encoding, but this fails because for HTTP POST stuff such as %3d are transmitted without translation but when browser does HTTP GET with the data %3d is converted to '='.
What encoding would you recommend, to allow safe transmission over HTTP POST & GET without data being misinterpreted.
The environment is Java, Tomcat.
Thank you,
Maxim.
Hex string.
Apache commons-codec has a Hex class that provides this functionality.
It will look like this:
http://youraddress.com/context/servlet?param=ac7432be432b21
Well, you can keep the Base64 and use this solution:
Code for decoding/encoding a modified base64 URL