Develop Reference

How to wait for the HTTP request to get completed in java

I am writing Java code where i am downloading the file from a server and i have to copy the file in my local system when the file download is complete.
I am using the below code:-
OkHttpClient.Builder builder = new OkHttpClient.Builder();
OkHttpClient client = builder.readTimeout(600, TimeUnit.SECONDS).writeTimeout(600, TimeUnit.SECONDS)
.connectTimeout(600, TimeUnit.SECONDS).build();
Request downloadRequest = new Request.Builder().url(url + fileName).addHeader("cache-control", "no-cache")
.addHeader("Authorization", token).build();
try {
Response downloadResponse = client.newCall(downloadRequest).execute();
System.out.println(downloadResponse.message());
System.out.println("got response from blob " + downloadResponse.isSuccessful() + " " + fileName);
return downloadResponse;
} catch (IOException e1) {
e1.printStackTrace();
}
return null;
}
But the request is made asynchronously and before the request is completed then response is returned which is incomplete. Can anyone please help me how can i make a request and wait till the response is completed.
Any help is highly appreciated!

Looks like you're returning the response object (not the response body content).
try something like:
return downloadedResponse.body().string()
My experience with HttpClient is such that the headers return first. The content doesn't necessarily come across the wire unless/until you consume it.

To make a synchronous GET request we need to build a Request object based on a URL and make a Call. After its execution we get back an instance of Response:
#Test
public void whenGetRequest_thenCorrect() throws IOException {
Request request = new Request.Builder()
.url(BASE_URL + "/date")
.build();
Call call = client.newCall(request);
Response response = call.execute();
assertThat(response.code(), equalTo(200));
}

You are already using synchronous method calling.
client.newCall(downloadRequest).execute();
This is a synchronous way of requesting URL. If you want to do the aysynchronous call you need to use enqueue method of Call class.
call.enqueue(new Callback() {
public void onResponse(Call call, Response response)
throws IOException {
// ...
}
public void onFailure(Call call, IOException e) {
fail();
}
});
I think problem is somewhere else. Kindly give more details why you are suspecting the current one as an asynchronous call so that we can do RCA.

Sparkjava unable to put data into the response body

I'm trying to a client (Android) talk to our server which is using sparkjava, but running into the issue that when the client is trying to parse the body of the response response.data with Volley, it's getting that the response body is empty. The client is sending a JsonRequestObject , which will throw a JSONException if the response's body is empty.
Here is our sparkJava controller:
public static String doThis(Request request, Response response) {
response.type("application/json");
// If the request fails validations, then return a 400
if (request.failsValidations()) {
response.status(HTTP_BAD_REQUEST); // 400
response.header("Error", "Bad request");
} else {
response.status(HTTP_SUCCESS); // 200
// Put the response into the data
String responseData = "{//someJson}"
response.header("data", responseData);
response.body(data);
}
return "";
}
I'm setting the same data in the header and body, but when I look at the response received on the client, the data is only in the header and not in the body. So I was thinking that Spark's response.body() method isn't really putting the data into the response being sent back.
Is the way response.body is represented in sparkjava different from how volley views it? Or is there another way to put data into the response body from spark?

You can do it in two ways:
Using a JSON Object
public static Object doThis(Request request, Response response) {
response.type("application/json");
JSONObject jo = new JSONObject();
jo.put("data", "someData");
return jo;
}
Using a string formatted as JSON
public static Object doThis(Request request, Response response) {
response.type("application/json");
return "{\"data\":\"someData\"}";
}
The first one is better IMO because you can modify the JSON object in a much more convenient way (but you have to import org.json.JSONObject).
Then, on the client side, you should treat the data as JSON data type. Hope it helps.

How to serve huge file over streaming rest?

I have 2 Spring Web applications: Application1 and Application2. In Application1, I have an endpoint at "http://application1/getbigcsv" that uses streaming in order to serve a gigantic 150MB CSV file back to the user if they hit that URL.
I dont want users to hit Application1 directly, but hit Application2 instead.
If I have the following method in my controller in Application2
#RequestMapping(value = "/large.csv", method = GET, produces = "text/csv")
#ResponseStatus(value = HttpStatus.OK)
public String streamLargeCSV() {
// Make an HTTP Request to http://application1/getbigcsv
// Return its response
}
My worry is the above is not doing "streaming" whereas Application1 is doing streaming. Is there some way I can make sure that the application2 will be serving back the same data from application1's rest endpoint in a streaming fashion? Or is the method above actually returning things in a "Streaming" method already because Application1 is serving its endpoint as streaming?

First of all: you can but not with that method signature.
Unfortunately, you have not shown how you produce that CSV file in app1, whether this is truly streaming. Let's assume it is.
You signature will look like this:
#RequestMapping(value = "/large.csv", method = GET, produces = "text/csv")
#ResponseStatus(value = HttpStatus.OK)
public void streamLargeCSV(OutputStream out) {
// Make an HTTP Request to http://application1/getbigcsv
// Return its response
}
Now we have to grab the input stream from app1 first. Use Apache HttpClient to get your HttpEntity. This entity has a writeTo(OutputStream) method which will receive your out parameter. It will block until all bytes are consumed/streamed. When you are done, free all HttpClient resources.
Complete code:
#RequestMapping(value = "/large.csv", method = GET, produces = "text/csv")
#ResponseStatus(value = HttpStatus.OK)
public void streamLargeCSV(OutputStream out) {
// Make an HTTP Request to http://application1/getbigcsv
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://application1/getbigcsv");
CloseableHttpResponse response = httpclient.execute(httpGet);
try {
HttpEntity entity = response.getEntity();
// Return its response
entity.writeTo(out);
} finally {
response.close();
}
}
Here is my real world example. Start reading from "Interesting to say what I have achieved in particular with this:"

In java.ws.rs.core package you have classes: StreamingOutput and ResponseBuilder.
Not sure if it will help you, but you may try.
Example:
#Produces("application/octet-stream")
public Response doThings () {
...
StreamingOutput so;
try {
so = new StreamingOutput() {
public void write(OutputStream output) {
…
}
};
} catch (Exception e) {
...
}
ResponseBuilder response = Response.ok(so);
response.header("Content-Type", ... + ";charset=utf-8");
return response.build();
}

Change your methods return type to ResponseEntity<?> and return as following:
#GetMapping("/download")
public ResponseEntity<?> fetchActivities(
#RequestParam("filename") String filename) {
String string = "some large text"
InputStream is = new ByteArrayInputStream(string.getBytest());
HttpHeaders headers = new HttpHeaders();
headers.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=large.txt");
headers.add(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_OCTET_STREAM_VALUE);
return ResponseEntity.ok().headers(headers).body(new InputStreamResource(is));
}

Streaming bytes via HTTP PUT with JAX-RS

I have a workflow that involves doing a HTTP POST from my Java client to my web server. The body of the post has a specification object. I then pass that on from my webserver to Apache ZooKeeper (which runs in its own process on the server) that runs a big hairy calculation. I am struggling with figuring out how to send back the bytes to my webserver in streaming fashion. I need it to stream back because I have a HTTP GET request on my webserver from my Java client that is waiting to stream back the bytes. I cannot wait for the whole calculation to finish, I want bytes sent as soon as possible back to the client.
Most the examples online for JAX-RS that do a HTTP PUT from the client side and on the webserver side don't have examples for streaming code. I'll post what I have so far, but it doesn't work.
Here is my ZooKeeper Java code, which calls a JAX-RS client-side PUT. I am really unsure of how to do this, I have never tried streaming data with JAX-RS.
final Client client = ClientBuilder.newClient();
final WebTarget createImageTarget = client.target("groups/{imageGroupUuid:" + Regex.UUID + "}");
StreamingOutput imageResponse = createImageTarget.request(MediaType.APPLICATION_OCTET_STREAM).put(Entity.entity(createRandomImageDataBytes(imageConfigurationObject), MediaType.APPLICATION_OCTET_STREAM), StreamingOutput.class);
Here is my webserver code which handles the HTTP PUT. It is just a stub because I have no confidence in my client side HTTP PUT.
#PUT
#PATH("groups/{uuid:" + Regex.UUID + "}")
#Consumes(MediaType.APPLICATION_OCTET_STREAM)
public void updateData(StreamingOutput streamingOutput)
{
}

Try something like this:
#GET
#Produces(MediaType.TEXT_PLAIN)
#Path("/{arg}")
public Response get(#PathParam("arg") {
//get your data based on "arg"
StreamingOutput stream = new StreamingOutput() {
#Override
public void write(OutputStream os) throws IOException, WebApplicationException {
Writer writer = new BufferedWriter(new OutputStreamWriter(os));
for (org.neo4j.graphdb.Path path : paths) {
writer.write(path.toString() + "\n");
}
writer.flush();
}
};
return Response.ok(stream).build();
}
#PUT
#Consumes("application/octet-stream")
public Response putFile(#Context HttpServletRequest request,
#PathParam("fileId") long fileId,
InputStream fileInputStream) throws Throwable {
// Do something with the fileInputStream
// etc
}

Use GZIP, JSON responses and JQuery

However, I want to compress my responses with GZIP wheren possible. I tried using the Compression filter code available for free download in the headfirst site. It works great for html, images, css and javascript.
I post the filter next. It checks if GZIP is an accepted encoding and it adds gzip as Content-Encoding. See: wrappedResp.setHeader("Content-Encoding", "gzip");
public class CompressionFilter implements Filter {
private ServletContext ctx;
private FilterConfig cfg;
/**
* The init method saves the config object and a quick reference to the
* servlet context object (for logging purposes).
*/
public void init(FilterConfig cfg)
throws ServletException {
this.cfg = cfg;
ctx = cfg.getServletContext();
//ctx.log(cfg.getFilterName() + " initialized.");
}
/**
* The heart of this filter wraps the response object with a Decorator
* that wraps the output stream with a compression I/O stream.
* Compression of the output stream is only performed if and only if
* the client includes an Accept-Encoding header (specifically, for gzip).
*/
public void doFilter(ServletRequest req,
ServletResponse resp,
FilterChain fc)
throws IOException, ServletException {
HttpServletRequest request = (HttpServletRequest) req;
HttpServletResponse response = (HttpServletResponse) resp;
// Dose the client accept GZIP compression?
String valid_encodings = request.getHeader("Accept-Encoding");
if ( (valid_encodings != null) && (valid_encodings.indexOf("gzip") > -1) ) {
// Then wrap the response object with a compression wrapper
// We'll look at this class in a minute.
CompressionResponseWrapper wrappedResp = new CompressionResponseWrapper(response);
// Declare that the response content is being GZIP encoded.
wrappedResp.setHeader("Content-Encoding", "gzip");
// Chain to the next component (thus processing the request)
fc.doFilter(request, wrappedResp);
// A GZIP compression stream must be "finished" which also
// flushes the GZIP stream buffer which sends all of its
// data to the original response stream.
GZIPOutputStream gzos = wrappedResp.getGZIPOutputStream();
gzos.finish();
// The container handles the rest of the work.
//ctx.log(cfg.getFilterName() + ": finished the request.");
} else {
fc.doFilter(request, response);
//ctx.log(cfg.getFilterName() + ": no encoding performed.");
}
}
public void destroy() {
// nulling out my instance variables
cfg = null;
ctx = null;
}
}
I was using the next code to send JSON responses in Struts web application.
public ActionForward get(ActionMapping mapping,
ActionForm form,
HttpServletRequest request,
HttpServletResponse response) {
JSONObject json = // Do some logic here
RequestUtils.populateWithJSON(response, json);
return null;
}
public static void populateWithJSON(HttpServletResponse response,JSONObject json) {
if(json!=null) {
response.setContentType("text/x-json;charset=UTF-8");
response.setHeader("Cache-Control", "no-cache");
try {
response.getWriter().write(json.toString());
} catch (IOException e) {
throw new ApplicationException("IOException in populateWithJSON", e);
}
}
}
It works fine without compression but if I compress JSON responses, I can not see my JSON objects anymore. I handle JSON Ajax calls with JQuery with code snippets as follows:
$.post(url,parameters, function(json) {
// Do some DOM manipulation with the data contained in the JSON Object
}, "json");
If I see the response with Firebug it is empty.
Should I refractor my compression filter to skip compression in JSON responses? or there is a workaround to this?
For me, it looks like JQuery does not recognize the response as JSON because I am adding the Gzip compression.

If I see the response with Firebug it
is empty.
There's your clue - it's not a JQuery problem, it's server-side. (I'm afraid I can't help you with that, other than to suggest you stop looking at the client-side)
There's no problem gzipping ajax responses - if you can't see the response in Firebug, then JQuery can't see it either.

you have to add one more header "content-encoding: gzip" if you are compressing it.

Have you tried with an explicit java-based client to ensure it's a problem with jQuery or browser? If java client fails, something is wrong with server response.
But I am guessing that whereas browser can deal with uncompression with direct requests, this is perhaps not applied to Ajax calls.
It's an interesting question, I hope we'll get a more definitive answer. :)