I have a spring boot application with a GET service.
#RequestMapping(value = "/abc/track/{type}", method = RequestMethod.GET)
public void DummFunc(
#RequestParam(value="subs", required = false) String sub,
, HttpServletResponse response) {}
value for subs is an encoded value.
If I pass following as value to parameter subs
{%22endpoint%22:%22https://def.abc.com/tyu/send/eD3vpGNQW28:APA91bHOo8rYrV0xccdQz3okjZJG-QGrJX8LG6ahJnEUpMNfGedqi3hJxQsJx_8BMbH6oDjaSPPEXqzNWchWrGSkhuTkSdemikkys1U22Ipd7MsRw0owWbw89V2fslIVAJ6G5lkyvYuQ%22,%22expirationTime%22:null,%22keys%22:{%22p256dh%22:%22BK0pktn50CMsTQtqwdPlKlJtdFs0LFeXX14T1zgoz6QWnvSTp5dxtChnUP5P1JX0TsjcopbdPKyN31HfABMUyic%22,%22auth%22:%22qbO_z0vGao9wD-E5g8VU-A%22}}
It fails to capture the request and control does not come inside of the function.
If we instead pass as value to parameter subs:
%7B%22endpoint%22:%22https://def.abc.com/tyu/send/dX5q5eV7hFQ:APA91bHib-0QXrMzjatcvTR_uaIeeJK8lf6GmXUC9Jxv0Oxth-BzD4GmWnd4-YpDZv8qSFZ0eSg9mB2YkRvkc5ezdXW5KeaHjuQZfdyDxyBXjJgE-25Xbtlk37pdm8vfLk20k0k_VxW9%22,%22expirationTime%22:null,%22keys%22:%7B%22p256dh%22:%22BCCvcBLpRqp4u0auP688_MUJLsAiwWlQSn5kpyl8YVsEo_J-KpSdnhCmVIE_BhDXBcFYflPK52hqhYf3EaOCyuY%22,%22auth%22:%22iKuW_ESkCZnubWcQu_JK8w%22%7D%7D
It works fine.
Why is this happening? What's wrong with first encoding?
Since server is not able to handle the request, it returns 400. I need to capture such requests and then handle them by encoding them properly. What can be way forward?
I am new to Spring boot/Spring and Java itself. Would be great if I can get some insight.
Also, I can decode both of them online here without any issues- https://www.urldecoder.org/
Edit: Basically, the request that has issue getting through has { and } instead of %7Band %7D.
My question is instead of application failing with 400 bad request,how do I capture such requests in my app, encode them properly and then process them.
This is not related to Java nor the Spring itself but the HTML URL Encoding Reference. URLs can only be sent over the Internet using the ASCII character set.
The unsafe characters are defined in the beginning of RFC-1738 and here is the list:
" < > # % { } | \ ^ ~ [ ] ` including the blank/empty space.
Aside from those, there are also reserved characters where belong the following ones and are used to distinguish the parameters, the key-value representation, the port etc.
; / ? : # = &
The unsafe characters you have used are { and } which are equal to %7B and %7D.
Essentially, you should not be concerned about the data the client sends you in the way you describe. The server must demand the correct form and URL passed. Although, the browsers and REST clients encode those characters automatically, sending them programmatically might cause errors. The only two available solutions in Spring I am aware of is through registering the CharacterEncodingFilter bean (already answered) or the Spring-Boot configuration:
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
You need to enable the encoding first and force on HTTP requests and responses.
spring-boot is very much concerned about security. Adding double quote / single quotes or either escaping won't work I guess.
Please go through : https://www.rfc-editor.org/rfc/rfc1738
I think you should try the manual encoding { to %7B and } to %7D
Unsafe:
Characters can be unsafe for a number of reasons. The space
character is unsafe because significant spaces may disappear and
insignificant spaces may be introduced when URLs are transcribed or
typeset or subjected to the treatment of word-processing programs.
The characters "<" and ">" are unsafe because they are used as the
delimiters around URLs in free text; the quote mark (""") is used to
delimit URLs in some systems. The character "#" is unsafe and should
always be encoded because it is used in World Wide Web and in other
systems to delimit a URL from a fragment/anchor identifier that might
follow it. The character "%" is unsafe because it is used for
encodings of other characters. Other characters are unsafe because
gateways and other transport agents are known to sometimes modify
such characters. These characters are "{", "}", "|", "", "^", "~",
"[", "]", and "`".
All unsafe characters must always be encoded within a URL. For
example, the character "#" must be encoded within URLs even in
systems that do not normally deal with fragment or anchor
identifiers, so that if the URL is copied into another system that
does use them, it will not be necessary to change the URL encoding.
Generally this issue comes when the APP server like Undertow, tomcat, Jboss which is hosting your Springboot application does not allow some special characters, which can very from Server to server. This is done to secure URLs and no one can send some special characters to the server to compromise with its functionality.
If still want to allow special characters so that those can reach to controller in Springboot application, you need to allow this configuration in APP server. e.g. this is the configuration required in Undertow server to allow special characters in URL:
#Configuration
public class HttpConfig {
#Bean
public UndertowServletWebServerFactory servletWebServerFactory() {
UndertowServletWebServerFactory factory = new UndertowServletWebServerFactory();
factory.addBuilderCustomizers((UndertowBuilderCustomizer) builder ->
builder.setServerOption(UndertowOptions.ALLOW_UNESCAPED_CHARACTERS_IN_URL, Boolean.TRUE));
return factory;
}
}
In order to make springboot work well with UTF-8 encoding/decoding of rest URLs, add this configuration in application.properties file, for springboot version 2.6:
server.servlet.encoding.charset=UTF-8
server.servlet.encoding.enabled=true
server.servlet.encoding.force=true
server.servlet.encoding.force-request=true
server.servlet.encoding.force-response=true
Best approach is to encode special characters in the client application which is calling this rest URL to comply with security principles
Due to Tomcat upgraded their security in URL param.
Please find possible solution
https://github.com/bohnman/squiggly/issues/42#issuecomment-386658525
Related
I'm trying to create a GET call in Java on an API with a path variable containing an illegal character.
GET http://api.example.com/key/foo<bar
The path contains the character <. The HTTP client encodes it as %3C but I need to keep it as <, because API server don't decode it.
I have to do this call, and the API provider doesn't want to change anything to be more compliant.
Calling the GET request with illegal character works with Postman.
I'm trying to do it in Java now.
I've tried with Feign, HttpClient, WebClient and OkHttp, I don't find a way to do it.
Do you know a Java Http client that will allow me to do it ? Or the proper encoding option for the clients above ?
I've tried to create a RequestInterceptor with Feign to update uri, but it works only for some characters, as defined in RFC 6570, but not for <, which is re-replaced with %3C after my replacement.
Thanks
We are using Apache Axis 2 in our product. We have exposed some services to the client. Client is hitting the service with UTF-8 encoding of both request parameters key and values. But the values are receiving to the serverend as it is. Consider the below example.
Client: https://abc.xyz.com/service?key=value%20value
Service(server end) : value will be same like value%20value but we are expecting the "value value".
We want to handle the special characters (/,\, space, # , #, &)
Please help if I am missing anything?
I am trying to maintain some logs of a couple of Javascript objects in my webapp. The easiest way to log them would be to stringify them and put them on a jersey path as a string.
My logger works fine with regular strings but gives Error 400: The request sent by the client was syntactically incorrect when I pass a JSON stringified object. There are two things that I can't explain and are going wrong with my code:
Everything seems to be working fine on my development server but not on the server where I am deploying it. I develop on a Mac / Homebrew / Tomcat enviroment and deploy on a CentOS server.
Even on the CentOS server, logging works fine when I pass a simple one word strings as message but passing a JSON string throws up the error.
My Logger code looks like this:
#PUT
#Path("logEvent/{fn_event}/{fn_message}")
public void logEvent(#PathParam("fn_event") String event,
#PathParam("fn_message") String message)
throws Exception {
:
:
:
}
I have tried investigating catalina logs but it doesn't tell anything. Access logs give no more information than specifying "Error 400".
This may happen if you don't escape quotes in your JSON string. Try to escape it with \"
Json String may be having "Spaces" etc etc.
So when you call http://yourserver/logEvent/oneword/onewordMessage it may work fine but when you call http://yourserver/logEvent/oneword with space and with & and so many things/or one message with " and not " etc
Then in second case, your path may be incorrectly encoded. Form Encode your json stream, and then pass it as path.
Better move to Post, and pass the stream as Body of Request. Not sure why you will prefer using "entire" json file as path of your service
I'm having a little problem. I'm building a small server in java, based on jetty websockets implementations.
The clients are the browsers and I send information using the websockets javascript api.
Everything works great until I send those special characters such as : ă Ț î ș ê ñ ü
So here is the problem. Client 1 sends a message to the server with one of this characters. Server prints the message and then send the message to client 2.
Client 2 receives the message and prints the message on a browser html page and works great The characters are showed correctly.
The problem is when I wanna print the String on the server site. Instead of ă is shows me the ? char. This is causing me problems because I want to insert the text in a database(mysql- with ut8 encoding enabled)
So.. what seems to be problem. The text that is send from the browser is not UT8 encoded? or the jetty websocket implementation is not receiving String in utf8 encoding??
Thanks
Here's a function I use to HTML-encode all special characters in a string (but not html itself (like < or >)). If you apply it before sending a string to the server, everybody should see the same and you can store it in a database table:
function toHtmlEncoded(string){
return string.replace(/[\u0080-\uC350]/g,
function(a) {return '&#'+a.charCodeAt(0)+';';}
);
}
First read this http://kunststube.net/encoding/
Then check everywhere you've converted bytes into Strings (or the reverse). Common places to make a mistake include calling getBytes() on a String without specifying an encoding. Other pitfalls include not setting the encoding in the database connection string.
I have a custom proxy servlet that has to deal with URL-s that contain special characters (e.g. ; , . / in their) in their path. This is because it is a RESTful application that has ugly path params by design. (Don't comment it as it is not mine.)
My client, (actually wget, because browsers tend to show unescaped the URL) send a request to this URL:
http://localhost:8080/MyApplication/proxy/foo/ugly%3Apart%2Fcomes%3Bhere/children
//note: %2F = '/', %3A = ':', %3B = ';'
In my servlet (mapped to /proxy/*) when I try to forward the GET request, I am unable to reconstruct it because HttpRequest.getPathInfo() returns me the URL unescaped:
http://localhost:8080/MyApplication/proxy/foo/ugly:part/comes;here/children
And therefore the information of which /s and ;s were originally escaped or unescaped is lost. And that makes a difference for me, for example ; makes my URL a so called matrix URL, see http://www.w3.org/DesignIssues/MatrixURIs.html, or all the REST path parameters get shifted by slashes.
Actually I found this issue on a Glassfish server, so I'm not sure if different application servers treat this differently or not. I found only this in the Servlet API:
getPathInfo() Returns any extra path information associated with the
URL the client sent when it made this request.
How could I get the original, unescaped request URL that was sent by the client?
Have a look at HttpServletRequest's getRequestURI() and getRequestURL() methods.
If you need to remove context and servlet mappings, look at getContextPath() and getServletPath().