We have two parts in our app:
Server - provide REST services
Client - consume them via Spring restTemplate
In addition to the HTTP status our server returns an HTTP body with JSON that describe error in detail.
So, I've added custom Error handler to restTemplate to treat some error coded as non errors - it helps parse HTTP body very well.
But I get an exception via parsing of the HTTP body in the case of an HTTP/1.1 401 Unauthorized.
All other error codes are handled fine(400, 402, etc. )
We are using plain server logic that sends HTTP response in the case of an error, no special rules for different types of an error:
writeErrorToResponse(int status, String errMsg, HttpServletResponse resp) throws IOException {
response.setStatus(status);
response.setContentType(MediaType.APPLICATION_JSON_VALUE);
String message = String.format("{\"error\":\"%s\"}", StringUtils.escapeJson(errMsg));
resp.getWriter().println(message);
}
But on client only HTTP/1.1 401 throws exception - "java.net.HttpRetryException: cannot retry due to server authentication, in streaming mode"
I've made some debugging and see that the cause of the problem is code in SimpleClientHttpResponse:
HttpURLConnection.getInputStream()
Tracing with Fiddler have these next responses:
Message is parsed correct on the client:
HTTP/1.1 402 Payment Required
X-Powered-By: Servlet/3.0
Content-Type: application/json
Content-Language: en-GB
Content-Length: 55
Connection: Close
Date: Sat, 25 May 2013 10:10:44 GMT
Server: WebSphere Application Server/8.0
{"error":"I cant find that user. Please try again."}
And message that is cause of exception:
HTTP/1.1 401 Unauthorized
X-Powered-By: Servlet/3.0
Content-Type: application/json
Content-Language: en-GB
Content-Length: 55
Date: Sat, 25 May 2013 11:00:21 GMT
Server: WebSphere Application Server/8.0
{"error":"I cant find that user. Please try again."}
What could be the cause of java.net.HttpRetryException in this situation?
In addition: Some times ago this mechanism worked fine. But since we have changed a lot of code in app.
I faced the same issue when using SimpleClientHttpRequestFactory. Solved it by setting
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setOutputStreaming(false);
return requestFactory;
The problem is due to chunking and subsequent retry mechanism incase of authentication.
You can also disable chunks using HttpClientPolicy
This issue is reported in Spring Framework.
Reference:
https://jira.spring.io/browse/SPR-9367
Copying from Reference:
It is very difficult (impossible) to handle a 401 response in the RestTemplate with default settings. In fact it is possible, but you have to supply an error handler and a request factory. The error handler was obvious, but the problem is that the default request factory uses java.net which can throw HttpRetryException when you try to look at the status code of the response (despite it being obviously available). The solution is to use HttpComponentsClientHttpRequestFactory. E.g.
template.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
template.setErrorHandler(new DefaultResponseErrorHandler() {
public boolean hasError(ClientHttpResponse response) throws IOException {
HttpStatus statusCode = response.getStatusCode();
return statusCode.series() == HttpStatus.Series.SERVER_ERROR;
}
});
HttpComponentsClientHttpRequestFactory requires below dependency. Add below dependency in your POM file:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
I was having the exact same issue using Spring Boot 2.2.4 and a TestRestTemplate. It only failed on 401's. Simply adding the latest version of Http Client (4.5.11) in a test scope fixed the problem. No further configuration was necessary.
I think passing a configured HttpComponentsClientHttpRequestFactory to your RestTemplate would help.
Here you might find the solution on how to do it.
The reason that I used such development solution was to handle the response messages in the body of Http error responses like 401, 404, and so on, and also to let the application be deployed on real-server side environment.
This exception has a quite unexpected impact on Spring OAuth2 token retrieval. For this retrieval the standard implementation uses RestTemplate which is configured with SimpleClientHttpRequestFactory where output streaming set to true. If the 4XX authentication error happens , the standard ResponseErrorHandler tries to read the output stream and receives, of course, the discussed exception, but decides to silently consume it, ignore.
Sometimes, however, OAuth2 authentication server writes important information into the output/error stream which must be read and analyzed.
To make this information available, upon the creation of OAuth2RestTemplate, its AccessTokenProvider should be set to a custom one, let's name it ErrorStreamAwareAccessTokenProvider:
oauth2RestTemplate.setAccessTokenProvider(new ErrorStreamAwareAccessTokenProvider());
and this custom AccessTokenProvider should overwrite, e.g. getRestTemplate method and tune the RestTemplate correspondingly, as the accepted answers advise. For example:
class ErrorStreamAwareAccessTokenProvider extends OAuth2AccessTokenSupport{
// your exact superclass may vary depending on grant type and other things
protected RestOperations getRestTemplate() {
RestTemplate template = super.getRestTemplate();
template.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
// ... also set ResponseErrorHandler if needed
return template;
}
}
When this is done, the standard ResponseErrorHandler could read the output/error stream or, if you want, for example, to throw a custom exception which has the retrieved information stored in it, you could set your custom ResponseErrorHandler in the above overwritten method.
Related
For context, I have been working on a springboot application with a ReactJS frontend. The ReactJS frontend connects to the springboot application, and the application makes calls to another server to get the requested data to display on the front end. I had gotten to a point where I was able to make a post request with a JSON payload, get the requested data in the form of another JSON payload, and display it to the frontend with no issues.
I had to implement a change, and add one more variable to the object that was being returned to the front end. And now the browser is telling me that:
Access to fetch at [url] been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
All that is different is the addition of the data to the object being returned, and the fact I am making another call to the server where the data is stored.
Here is the Response header when I dont make the additional call, and I get back the JSON payload
HTTP/1.1 200
Vary: Origin
Vary: Access-Control-Request-Method
Vary: Access-Control-Request-Headers
Access-Control-Allow-Origin: *
Set-Cookie: [I erased the cookie after pasting]
Content-Type: text/plain;charset=ISO-8859-1
Content-Length: 83363
Date: Thu, 09 Jul 2020 18:05:40 GMT
Keep-Alive: timeout=60
Here is the Response Header when I make the additional call
HTTP/1.1 500
Connection: close
Date: Thu, 09 Jul 2020 18:13:24 GMT
Ive debugged the backend, and no errors are thrown on the server side, and nothing happens while processing the data. I have all of a sudden just started getting the issue with CORS.
The following is my post request that has been working so far
fetch(url, {
headers: {
'Content-Type': 'text/plain',
'Accept': '*/*',
'Accept-Encoding': 'gzip, deflate, br',
},
method: 'post',
body: JSON.stringify(jsonData)
})
Im sure I have enabled everything I had to do on the Springboot side for CORS considering It works all the time except for when trying to get the extra data. If there is something I am missing I would like to know, and if this issue can be clarified by reading documentation on springboot and cors please point me in that direction.
Note:
I have also tried adding
#Configuration
#EnableWebMvc
public class SpringBootConfiguration implements WebMvcConfigurer{
#Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**");
}
}
to my application, but that does not seem to work either. This is all still fairly new to me, so I am unsure what is going wrong.
Thank you.
The header was too large now with the added data it seems. So all I had to do was increase the max-http-header-size in my application.properties. I can now get the full JSON payload with zero issues. I was too focused on the error on the frontend side that I neglected to go deep enough into the backend to see what was happening.
I added in my application.properties
server.max-http-header-size = 48000
I didnt actually set it to 48000, its much lower than that, I found the answer here
How to set max-http-header-size in spring boot 2.x application
I had issues with CORS andthe snippet you gave was something I had tried but it ended up breaking some other endpoints for unknown reasons. You can add #CrossOrigin(origins = "URL", allowedHeaders = "*") to your controllers.
Such as
#Controller
#CrossOrigin(origins = "localhost", allowedHeaders = "*")
public class Controller
In my Ruby on Rails application I tried to upload an image through the POSTMAN REST client in Base64 format. When I POST the image I am getting a 406 Not Acceptable Response. When I checked my database, the image was there and was successfully saved.
What is the reason for this error, is there anything I need to specify in my header?
My request:
URL --- http://localhost:3000/exercises.json
Header:
Content-Type - application/json
Raw data:
{
"exercise": {
"subbodypart_ids": [
"1",
"2"
],
"name": "Exercise14"
},
"image_file_name": "Pressurebar Above.jpg",
"image":"******base64 Format*******"
}
Your operation did not fail.
Your backend service is saying that the response type it is returning is not provided in the Accept HTTP header in your Client request.
Ref: http://en.wikipedia.org/wiki/List_of_HTTP_header_fields
Find out the response (content type) returned by Service.
Provide this (content type) in your request Accept header.
http://en.wikipedia.org/wiki/HTTP_status_code -> 406
406 Not Acceptable
The resource identified by the request is only capable of generating response entities which have content characteristics not
acceptable according to the accept headers sent in the request.
406 happens when the server cannot respond with the accept-header specified in the request.
In your case it seems application/json for the response may not be acceptable to the server.
You mentioned you're using Ruby on Rails as a backend. You didn't post the code for the relevant method, but my guess is that it looks something like this:
def create
post = Post.create params[:post]
respond_to do |format|
format.json { render :json => post }
end
end
Change it to:
def create
post = Post.create params[:post])
render :json => post
end
And it will solve your problem. It worked for me :)
"Sometimes" this can mean that the server had an internal error, and wanted to respond with an error message (ex: 500 with JSON payload) but since the request headers didn't say it accepted JSON, it returns a 406 instead. Go figure. (in this case: spring boot webapp).
In which case, your operation did fail. But the failure message was obscured by another.
You can also receive a 406 response when invalid cookies are stored or referenced in the browser - for example, when running a Rails server in Dev mode locally.
If you happened to run two different projects on the same port, the browser might reference a cookie from a different localhost session.
This has happened to me...tripped me up for a minute. Looking in browser > Developer Mode > Network showed it.
const request = require('request');
const headers = {
'Accept': '*/*',
'User-Agent': 'request',
};
const options = {
url: "https://example.com/users/6",
headers: headers
};
request.get(options, (error, response, body) => {
console.log(response.body);
});
Changing header to Accept: */* resolved my issue and make sure you don't have any other Accept Header
In my case, I added:
Content-Type: application/x-www-form-urlencoded
solved my problem completely.
If you are using 'request.js' you might use the following:
var options = {
url: 'localhost',
method: 'GET',
headers:{
Accept: '*/*'
}
}
request(options, function (error, response, body) {
...
})
In my case for a API in .NET-Core, the api is set to work with XML (by default is set to response with JSON), so I add this annotation in my Controller :
[Produces("application/xml")]
public class MyController : ControllerBase {...}
Thank you for putting me on the path !
It could also be due to a firewall blocking the request. In my case the request payload contained string properties - "like %abc%" and ampersand symbol "&" - which caused the firewall to think it is a security risk (eg. a sql injection attack) and it blocked the request. Note here the request does not actually go to the server but is returned at the firewall level itself.
In my case, there were no application server logs generated so I knew that the request did not actually reach the server and was blocked before that. The logs that helped me were Web application firewall (WAF) logs.
I'm pretty new to Java and REST/REST assured.
I'm trying to create a POST Request with "Transfer-Encoding: chunked" set (via a Header) but I get the exception “org.apache.http.ProtocolException: Transfer-encoding header already present”.
The code I'm using is similar to the following .....
#Test
......
given()
.headers(uses a method that sets the required headers, including "Transfer-encoding")
.body("testdata".getBytes())
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.log().all()
.expect()
.statusCode(HttpStatus.SC_OK)
.post();
but I get the exception "org.apache.http.ProtocolException: Transfer-encoding header already present”.
Does anyone have an idea why i'm getting the exception / how i can resolve it??
Thanks in advance.
Yesterday I able to use octate stream
given().urlEncodingEnabled(false)
.config(RestAssured.config()
.encoderConfig(new EncoderConfig().appendDefaultContentCharsetToContentTypeIfUndefined(false)
.encodeContentTypeAs("application/octet-stream", ContentType.TEXT)))
You do not need to add Transfer-encoding as REST Assure framework does it for you.
You need urlEncodingEnabled as it automatically encodes, while need encodeContentTypeAs as internally framework using serializer.
This might helped other who may faced similar problem in future.
I want to access a REST service from my Dropwizard app using a Jersey client. Same technique works fine in different places, but in the present case the service always returns a 500 although I know the URL, etc. are correct. Since I can't modify the service I want to log my requests to verify that they're fine (because I suspect my entity is not properly serialized).
Therefore I registered a logger on the client like this:
restClient.register(new LoggingFilter(Logger.getAnonymousLogger(), true));
But it just never logs my entity. I always get this:
INFO [2015-07-01 14:12:47,433] unknown.jul.logger: 1 * Sending client request on thread dw-admin-34
1 > POST [My URL which is correct]
1 > Accept: application/json
1 > Accept-Encoding: gzip
1 > Content-Type: application/json
?
Does it mean it doesn't recognize any entity?
This is how I build the client:
this.restClient = new JerseyClientBuilder(dropwizardEnvironment)
.using(MyConfiguration.jerseyClient) // new JerseyClientConfiguration() in Dropwizard config class
.build("my client name");
And here's how I make the request:
Response r = restClient.target(address)
.request(MediaType.APPLICATION_JSON)
.acceptEncoding("gzip")
.post(Entity.json(entity)); //omitting entity. It's very simple and I checked that its values are fine
I'm pretty new to all of this, so probably this is just some stupid noob mistake. Any ideas why the request body is not properly logged?
The problem was that the Jersey client gzipped the body by default. After disabling taht it logs and works like a charm.
I have pretty simple JAX-RS WS:
#Stateless
#Path("/foo")
public class FooFacadeBean {
#GET
#Produces(MediaType.TEXT_HTML)
public String performFooCall(#Context HttpServletRequest request, #Context HttpServletResponse response) {
response.setStatus(500);
return "Hello";
}
}
After deployment I execute: curl -v localhost:8080/foo,
the result was:
About to connect() to localhost port 8080
Trying 127.0.0.1...
connected
Connected to localhost (127.0.0.1) port 8080
GET /foo HTTP/1.1
User-Agent: curl/7.26.0
Host: localhost:8080
Accept: */
HTTP/1.1 200 OK
X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2 Java/Sun Microsystems Inc./1.6)
Server: GlassFish Server Open Source Edition 3.1.2
Content-Type: text/html
Transfer-Encoding: chunked
Date: Thu, 26 Jul 2012 13:52:10 GMT
Hello
Connection to host localhost left intact
Closing connection
As you can see status code in HTTP hasn't changed at all, despite of it was set manually.
Why does this happen? I've spent a lot of hours googling without any results.
Such workaround as returning Response object, something like this:
Response.status(404).entity("Hello").build();
works perfectly!
Glassfish uses JERSEY as JAX-RS implementation. I use embedded variant of Glassfish.
Basically, that's not how you should use Jersey.
Here's what's happening:
The method executes, and you interact with the ServletResponse and set the status code to 500. But since you don't write any data, the response isn't committed.
Then the method returns a String value. Since Jersey has no way of knowing if you've interacted with the ServletResponse or not, it behaves normally.
A String method that returns implies a 200 response code (void implies 204, etc). Jersey tries to set the response code to the default, ultimately by calling setStatus(200) on the response instance. Since the response hasn't been committed yet, this isn't a problem, and the response of 500 is changed to 200.
The HttpServletResponse is committed and is sent back to the client.
I'm assuming that what you want to do is return a different status code, but in an exceptional case only. The way to do that is to return a String in the normal case, and then throw a WebApplicationException in the exceptional case.
#GET
#Produces(MediaType.TEXT_HTML)
public String performFooCall() {
if (isNormal()) {
return "Hello";
}
throw new WebApplicationException(Response.status(500).entity("Hello").build());
}
Alternatively, you can throw a standard exception, and control how it is rendered as a response with a separate ExceptionMapper.
Jersey building a new response so setting the response code in your code, is override later.
You have to build a new response in your code and put you string in the response (and change your return value for that), or you can create a filter that will work later on in the filter chaing and change the response status.
look here for more details