Develop Reference

getEntity() returns null, readEntity(InputStream.class) loads everything to memory

I have problem downloading file (getting the input stream) from server (quarkus implementation) in client (quarkus implementation)
I am using 2.9.2.Final version of Quarkus for both client and server,
dependency - quarkus-rest-client-reactive-jackson
I have a server which returns InputStream inside Response:
#Override
public Response downloadFileById(String id) {
var inputStream = repo.downloadFileByFileId(id);
return Response.ok(inputStream, MediaType.APPLICATION_OCTET_STREAM)
.header("Content-Disposition", "attachment; filename = \"" + fileName + "\"")
.header("Content-Length", repo.getFileLengthByFileById(id))
.build();
}
On client side, I've tried these variations:
This works but loads everything into memory. It uses ByteArrayInputStream inside ClientSendRequestHandler.java from package org.jboss.resteasy.reactive.client.handlers; (see line 340 in https://github.com/quarkusio/quarkus/blob/b0019c087880f9fd1371776b8c23c1b49129dcb3/independent-projects/resteasy-reactive/client/runtime/src/main/java/org/jboss/resteasy/reactive/client/handlers/ClientSendRequestHandler.java)
var stream = service.downloadFileById(id)
.await()
.indefinitely()
.readEntity(InputStream.class);
I expected this to work but it returns null
var stream = service
.downloadFileById(id)
.await()
.indefinitely()
.getEntity();
As I am going through following method from the ClientSendRequestHandler.java
private void attachSentHandlers(Future<HttpClientResponse> sent,
HttpClientRequest httpClientRequest,
RestClientRequestContext requestContext)
I am wondering, is it not supported?
I see cases for Multipart data, File, and an else branch
Not sure what am I missing.

Getting java.io.IOException: Server returned HTTP response code: 400 for URL: when using a url which return 400 status code

I am trying to perform a get request using Groovy using the below code:
String url = "url of endpoint"
def responseXml = new XmlSlurper().parse(url)
If the endpoint returns status as 200 then everything works good but there is one case where we have to validate the error response like below and status returned is 400:
<errors>
<error>One of the following parameters is required: xyz, abc.</error>
<error>One of the following parameters is required: xyz, mno.</error>
</errors>
In this case parse method throws :
java.io.IOException: Server returned HTTP response code: 400 for URL: "actual endpoint throwing error"
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1900)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:646)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:150)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:831)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:796)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:142)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:644)
at groovy.util.XmlSlurper.parse(XmlSlurper.java:205)
at groovy.util.XmlSlurper.parse(XmlSlurper.java:271)
Can anyone pls suggest how to handle if server give error message by throwing 400 status code?

In the question since we are getting 400 status code for GET request. So in built XmlSlurper().parse(URI) method does not work as it throw io.Exception.
Groovy also support HTTP methods for api request and response and the below worked for me:
def getReponseBody(endpoint) {
URL url = new URL(endpoint)
HttpURLConnection get = (HttpURLConnection)url.openConnection()
get.setRequestMethod("GET")
def getRC = get.getResponseCode()
BufferedReader br = new BufferedReader(new InputStreamReader(get.getErrorStream()))
StringBuffer xmlObject = new StringBuffer()
def eachLine
while((eachLine = br.readLine()) !=null){
xmlObject.append(eachLine)
}
get.disconnect()
return new XmlSlurper().parseText(xmlObject.toString())
}

Getting the response text from the HttpURLConnection class rather than implicitly through XmlSlurper allows you much more flexibility in handling unsuccessful responses. Try something like this:
def connection = new URL('https://your.url/goes.here').openConnection()
def content = { ->
try {
connection.content as String
} catch (e) {
connection.responseMessage
}
}()
if (content) {
def responseXml = new XmlSlurper().parseText(content)
doStuffWithResponseXml(responseXml)
}
Even better would be to use an actual full-featured HTTP client, like the Spring Framework's HttpClient or RestTemplate classes.

You should check the return code and than obtain the error stream from http request instance in case of an error. The problem itself has nothing to do with JsonSlurper, as no instance of "input stream" is returned from http request instance if service returns not successfull return codes (400, 401, 500 etc.) POST example can be seen below:
http= new URL("yourUrl").openConnection() as HttpURLConnection
http.setRequestMethod('POST')
http.setDoOutput(true)
http.setRequestProperty("Content-Type", 'application/json')
http.setRequestProperty("Accept", 'application/json')
http.setRequestProperty("Authorization", "Bearer $yourTokenVariable")
http.outputStream.write(data.getBytes("UTF-8"))
http.connect()
if(http.getResponseCode() != 200 && http.getResponseCode() != 201){
throw new InvalidInputException("There was an error: " + http.getErrorStream().getText("UTF-8"))
} else {
//You can take input stream here
}

How to set JSON payload on AmazoneWebServiceRequest

I am using AmazonHttpClient & AmazonWebServiceRequest to make http requests to API gateway from android app. The requestBody is a JSON String and DefaultRequest only seem to accept InputStream as content.
final AmazonWebServiceRequest awsRequest = new AmazonWebServiceRequest() {};
final Request request = new DefaultRequest(awsRequest, UtilConstants.API_GATEWAY_SERVICE_NAME);
request.setEndpoint(uri);
request.setHttpMethod(requestType);
request.addHeader(HttpHeader.CONTENT_TYPE, "application/json");
final InputStream stream = new ByteArrayInputStream(requestBody.getBytes(StandardCharsets.UTF_8));
request.setContent(stream);
When I make the request I get following exception
Caused by: com.amazonaws.AmazonClientException: Unable to execute HTTP request: expected 0 bytes but received 38

adding HttpHeader.CONTENT_LENGTH header solved the issue.

Java Jersey ClientResponse returns wrong Status from getClientResponseStatus

I feel like I am missing something here. I have a filter which prints out my server's returned information and I report that I am returning the correct response (403). I wrote a JUnit test to verify this logic and many times I am reporting 200 instead of 403. The weird part is that my server logs still show that I sent a 403. Is there some known bug in Jersey 1.17 that I am not aware of and I need to upgrade to resolve? I am not really in a position to upgrade at this point in time so I am hoping there is some bug in my logic. Here is my test case.
#Test
public void testIdsOwnedBySomeoneElse()
{
final Login user1Cred = Logins.getInstance().getLogin(Logins.LoginType.User1);
final Login user2Cred = Logins.getInstance().getLogin(Logins.LoginType.User2);
final ServiceEndpointAuthenticated authUser1 = LoginHelper.Login(user1Cred);
final ServiceEndpointAuthenticated authUser2 = LoginHelper.Login(user2Cred);
// Create generic entry owned by user 1
BigInteger user1Id = null;
{
final Object payload = endpoint.CreateEntity(authUser1.getUserId());
final ClientResponse response = endpoint.Post(authUser1, payload);
assertTrue(Status.OK == response.getClientResponseStatus());
final byte[] data = Utilities.getBytes(response.getEntityInputStream());
user1Id = endpoint.getEntityId(data);
}
// Using User2, try to use that id from user1!
{
// test 1
final MyEndpointWrapper endpoint = new MyEndpointWrapper(user1Id, validId);
final Object payload = endpoint.CreateEntity(authUser2.getUserId());
final ClientResponse response = endpoint.Post(authUser2, payload);
final Status status = response.getClientResponseStatus();
System.out.println("Returned status = " + status);
if (status != Status.FORBIDDEN)
{
byte[] data = Utilities.getBytes(response.getEntityInputStream());
String toString = null;
try
{
toString = new String(data, "UTF-8");
}
catch (UnsupportedEncodingException e)
{
}
System.out.println("data: " + toString);
}
assertEquals("Status " + status + " is not forbidden!", Status.FORBIDDEN, status);
}
{
// test 2
final MyEndpointWrapper endpoint = new MyEndpointWrapper(validId, user1Id);
final Object payload = endpoint.CreateEntity(authUser2.getUserId());
final ClientResponse response = endpoint.Post(authUser2, payload);
final Status status = response.getClientResponseStatus();
System.out.println("Returned status = " + status);
if (status != Status.FORBIDDEN)
{
int i = 9;
}
assertEquals("Status " + status + " is not forbidden!", Status.FORBIDDEN, status);
}
// Go ahead and delete this data for cleanup
assertTrue(Status.OK == endpoint.Delete(authUser1, user1Id).getClientResponseStatus());
}
My generic code first logs into our server for the creds. These creds are "attached" to the WebResource and it attaches the proper headers automatically when I build my request. I first create an entity, post it, and store the returned id to be used by another user. I create another endpointwrapper which references that violation id and I attempt to post with that id. The server logs:
INFO: RESPONSE: 403 http://myendpoint MediaType:(application/json) Payload: 232 MyErrorMessage
I can even print this message out (as shown above)! The part I dont understand is that getClientResponseStatus returned to me OK. Why?
My Post code looks like:
#Override
public ClientResponse Post(ServiceEndpointAuthenticated endpoint, Object entity)
{
MyUploadData uploadData = (MyUploadData)entity;
return endpoint.getResourceBuilder("/myendpoint")
.accept(MediaTypeExt.APPLICATION_JSON)
.type(MediaTypeExt.APPLICATION_JSON)
.post(ClientResponse.class, gson.toJson(uploadData));
}
[UPDATE]
I ran wire capture and actually do see 200 being sent back! This does appear to be something inside of Jersey Server. Here is what I see:
When working:
Request: 1099 17.021219000 127.0.0.1 127.0.0.1 HTTP 2214 POST /myEndpoint HTTP/1.1 (application/json)
Response: 1153 17.042535000 127.0.0.1 127.0.0.1 HTTP 628 HTTP/1.1 400 Bad Request (application/json)
When not working:
Request: 1161 17.044313000 127.0.0.1 127.0.0.1 HTTP 250 POST /myEndpoint HTTP/1.1 (application/json)
Response: 1217 17.066059000 127.0.0.1 127.0.0.1 HTTP 412 HTTP/1.1 200 OK
When it works I see my normal headers in the response (eg: Access-Control-*, Pragma no cache, etc). When it doesn't work I dont see any of my headers but I do see "Transfer-Encoding: chunked" and my response is my error message but my response code is 200. I added an explicit Trace statement in the server right before I sent my response to ensure I am sending the right Status and I am.
I am okay with allowing chunked transfer but I am not really okay with losing my desired http response.

Incase anyone else encounters something similar. After digging around I finally found the problem. We have a heartbeat on some of our endpoints. Some endpoints can take longer than expected time. To ensure the client doesn't disconnect prematurely we have a component which attaches to the ServletOutputStream. This sends a space to the client to keep the connection alive.
When an error is thrown (caught by our new exception remapper), this keep-alive component was not being shutdown properly. This caused Jersey to switch into chunked mode. Ensuring the keep-alive component was shutdown properly resolved the problem.

Why am I getting chunked stream ended unexpectedly when I use HttpClient?

The problem only seems to occur with "large" files I'm trying to post.
My code looks like this:
PostMethod method = new PostMethod(url);
File input = new File(filePathname);
RequestEntity entity = new FileRequestEntity(input, "text/xml; charset=ISO-8859-1");
method.setRequestEntity(entity);
method.setRequestHeader("Content-Disposition", "attachment; filename=xyzzy")
HttpClient client = new HttpClient();
Credentials defaultcreds = new UsernamePasswordCredentials("userid", "pw");
client.getState().setCredentials(new AuthScope("hostname", port, AuthScope.ANY_REALM), defaultcreds);
try {
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
throw new Exception("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody();
return new String(responseBody);
}
catch (HttpException e) {
System.err.println("Fatal protocol violation: " + e.getMessage());
throw e;
}
catch (IOException e) {
System.err.println("Fatal transport error: " + e.getMessage());
throw e;
}
finally {
// Release the connection.
method.releaseConnection();
}
The exception text looks like this:
Fatal transport error: chunked stream ended unexpectedly
Exception in thread "main" java.io.IOException: chunked stream ended unexpectedly
at org.apache.commons.httpclient.ChunkedInputStream.getChunkSizeFromInputStream(ChunkedInputStream.java:252)
at org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputStream.java:221)
at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:176)
at java.io.FilterInputStream.read(FilterInputStream.java:127)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:108)
at java.io.FilterInputStream.read(FilterInputStream.java:101)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:127)
at org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:690)
I get a similar exception whether I use getResponseBody() or getResponseBodyAsStream().
I shouldn't be getting much data back, but I am posting over 200mb of data.

I am able to make this problem go away by altering the length of the file name value specified in the PostMethod's requestHeader. I had been including an encoded version of the full file pathname in the request header. Through trial and error I found that success or failure of the file I was "posting" seemed to depend on the folder it was in. A long folder file pathname wasn't working while a short one, albeit with the same file, was. So I eliminated the path name from the request header and only started including the file name and I'm no longer seeing the problem.

May be old , Can save some time.....
I got this error where Server is in Python and Clinet is Java.
1st -
Error from Java Client "Error while sending data over http java.io.IOException: CRLF expected at end of chunk: 79/82
java.io.IOException: CRLF expected at end of chunk: 79/82"
2nd -
Error from Java Clinet "Error while sending data over http java.io.IOException: chunked stream ended unexpectedly
java.io.IOException: chunked stream ended unexpectedly"
Both the errors got resolved by changing the ok response with chunked stream size
One with issues -
HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nServer: Jetty(6.1.26)\r\n\r\nDE\r\n
Resolved with -
HTTP/1.1 200 OK\r\nContent-Length: 20000\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nServer: Jetty(6.1.26)\r\n\r\n229\r\n
Note = nDE is replaced with n229