Develop Reference

File upload handling and buffering internals with Jersey API

How does Jersey handle receiving files from a client (e.g. web application)?
I've looked around for a longer while and cannot seem to get the answers I need.
Let's say that I have an exposed Jersey endpoint that consumes a multipart data form with a file that looks pretty much like this:
#POST
public Response upload(#FormDataParam("file") InputStream inputStream,
#FormDataParam("file") FormDataContentDisposition fileDetails { ... }
I've noticed that if I try to upload the file, the endpoint is not called until the uploading is finished. Does Jersey attempt to read and buffer (memory? disk?) an entire file before handling it for further processing? Does it mean that the inputStream source and size is already known when the processing of endpoint logic starts (because it was already read)?
And finally, is it possible to handle it in a "as comes" manner, without waiting with further actions for whole file to be uploaded first?

File Upload error: The request sent by the client was syntactically incorrect with jersey 2.x and swagger 1.5

I am trying to create RESUfull web services with swagger documentation. After i googled and found the below mentioned two dependencies are the latest and both works fine.
<dependency>
<groupId>com.wordnik</groupId>
<artifactId>swagger-jersey2-jaxrs_2.10</artifactId>
<version>1.3.12</version>
</dependency>
OR
<dependency>
<groupId>io.swagger</groupId>
<artifactId>swagger-jersey2-jaxrs</artifactId>
<version>1.5.13</version>
</dependency>
I have created API for GET and POST request. and I am able to get the responses for both GET and POST.
Now my new requirement is to create file upload POST API. So i fowlloed the stranded coding proacties,
1) added MultipartFeature in web.xml from glassfish api
org.glassfish.jersey.media.multipart.MultiPartFeature
2) my POST API
**import org.glassfish.jersey.media.multipart.FormDataContentDisposition;
import org.glassfish.jersey.media.multipart.FormDataParam;**
#POST
#Path("/pdfupload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Produces(MediaType.APPLICATION_XHTML_XML)
#ApiImplicitParams({#ApiImplicitParam(dataType="file", paramType="formData", name="file")})
public Response fileUpload(#ApiParam(value="PDF File Upload", required=true) #FormDataParam("file") InputStream pdfInputStream,
#FormDataParam("file") FormDataContentDisposition fileDetails) {
String uploadSuccessful = null;
if (null != pdfInputStream) {
uploadSuccessful = "API successfully called "+ fileDetails.getFileName();
}
return Response.ok(uploadSuccessful).build();
}
I am facing two issue with this API now,
First, I am not getting choose file button in swagger documentation and I tried different approach to get the button #ApiImplicitParams.
Secondly, I am getting "The request sent by the client was syntactically incorrect" error response(400), I am not sure what is missing here.
Could you please someone help me this.
Appreciate your help.

Spring: JSON data and file in the same request

I know how to create endpoints that are handling files using MediaType.MULTIPART_FORM_DATA and #FormDataParam("file") FormDataBodyPart bodyPart, but I was wondering if I can also have JSON data along that request? Something like:
#POST
#Path("somepath")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFileAndJSON(#RequestBody SomeModel someModel,
#FormDataParam("file") FormDataBodyPart bodyPart) {
return null;
}
At the moment if I add some JSON data on the "raw" tab on the following Postman request I'm getting HTTP 415 Unsupported Media Type probably because I specified that I consume MULTIPART_FORM_DATA but I'm also using #RequestBody which is looking for JSON content which is APPLICATION_JSON. So how can I have JSON data and a file handled in the same request? I know that it's possible to do that in two requests, I just want to do it in one if possible?

Why are you using both Spring and Jersey annotations? You should stick to using the annotations meant for the framework. Since you are using Jersey, should stick to its its annotations.
So here are the things to consider about your current code and environment.
There can't be two separate bodies. With your code, that's what it appears you expect to happen.
You can though put the JSON as part of the multi-part body. For that you should also annotate the SomeModel with the Jersey #FormDataParam
#POST
#Path("somepath")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFileAndJSON(
#FormDataParam("model") SomeModel someModel,
#FormDataParam("file") FormDataBodyPart bodyPart) {
}
In the Jersey configuration, you need to make sure to register the MultiPartFeature. If you don't the body won't be able to be deserialized, and you will get exceptions and error responses.
Now the Postman problem. You can see similar problem here. The problem was that the Content-Type was not set for the JSON body part. For example the body might look something like
--AaB03x
Content-Disposition: form-data; name="model"
{"some":"model", "data":"blah"}
--AaB03x
Content-Disposition: form-data; name="file"; filename="file1.txt"
Content-Type: text/plain
... contents of file1.txt ...
--AaB03x--
You can actually see the body, if you hit the Preview button in Postman. The problem is that there is no Content-Type for the "model" part, as you can see in the "file" part. This happens because you can't set individual parts' Content-Type in Postman. The one that you will see will be discovered from the file extension. For example a .txt file will make Postman set the Content-Type to text/plain and a .png file to image/png.
If you look in the link above, I proposed maybe you could use a .json file instead of typing in the data. Of course that was just a theory. I didn't actually test it.
In any case, the Content-Type must be set in order for Jersey to be able to know to deserialize it as JSON. If the .json file extension theory doesn't pan out, then you can use a different client, like cURL, which I showed an example in the link, or you can use the Jersey client to test, as seen here.
Don't set the Content-Type header to multipart/form-data in Postman. It sets it for you when you use the form-data. I just saw a post where someone said there is bug when you set the header. Can't find the post now, and not something I've confirmed, but I'd just leave it out.
UPDATE
So the OP was able to find a way to set the Content-Type: application/json to the "model" part. But it is sometimes the case where with a Javascript client, you are not able to set it. So there will be no Content-Type. If this is the case, Jersey will not be able to deserialize the JSON, as it has no idea that it is actually JSON being sent. If you absolutely can't or have no idea how to set the Content-Type for individual parts, you could resort to doing the following.
#POST
#Path("somepath")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFileAndJSON(#FormDataParam("model") FormDataBodyPart jsonPart,
#FormDataParam("file") FormDataBodyPart bodyPart) {
jsonPart.setMediaType(MediaType.APPLICATION_JSON_TYPE);
SomeModel model = jsonPart.getValueAs(SomeModel.class);
}

Yes, you can get that as multipart form data.
you get like this in angularjs:
$scope.uploadFile = function () {
var file = $scope.selectedFile[0];
$scope.upload = $upload.upload({
url: 'api/upload',
method: 'POST',
data: angular.toJson($scope.model),
file: file
}).progress(function (evt) {
$scope.uploadProgress = parseInt(100.0 * evt.loaded / evt.total, 10);
}).success(function (data) {
//do something
});
};
$scope.onFileSelect = function ($files) {
$scope.uploadProgress = 0;
$scope.selectedFile = $files;
};
public Response uploadFileAndJSON(#RequestParam("data") String data,
#MultiPartFile("file")File file) {
you can data as form data and convert it
like you want to your object using Gson jar.
return null;
}
Have a look at it for angularjs code:
Angularjs how to upload multipart form data and a file?
https://puspendu.wordpress.com/2012/08/23/restful-webservice-file-upload-with-jersey/

How to get rid of WebKitFormBoundary in uploaded file

I'm implementing a file upload in a web application.
The front-end is written in angularJS and uses the angular-file-upload package (the one that can be found at this link https://github.com/nervgh/angular-file-upload).
The back-end is Java / Jersey Web Services.
My Question is:
The uploaded file contains a WebKitFormBoundary header and footer, like this:
------WebKitFormBoundarylqskdjlqksdjl
Content-Disposition: form-data; name="upload"; filename="foo.bar"
Content-Type: multipart/form-data
Therefore, I'm not sure whether I'm uploading a file or a request. And of course, my back-end application considers that the uploaded files are corrupted and would not display them unless those lines are removed (for now manually).
Bottom line is : how do I get rid of that header and footer in the uploaded file?
Here are some code samples.
Front-End
Once again: angularJS angular-file-upload
item.headers = {
'Content-Disposition': 'attachment; filename="' + item.file.name + '"',
'Content-Type': 'multipart/form-data'
};
Back-End
and Java / Jersey
#POST
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Path("someurl/{fileName}")
public Artifact uploadArtifact(InputStream uploadedStream, #PathParam("fileName") String fileName) throws Exception;
Note
I'm wondering if the Content-Disposition: attachment in my angularJS part could be what's messing it up?
And that it should rather be Content-Disposition: form-data?
Thx in advance!

You need to place #FormDataParam annotation in order to properly handle boundary.
#POST
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.MULTIPART_FORM_DATA)
#Path("someurl/{fileName}")
public Artifact uploadArtifact(
#FormDataParam("file") InputStream uploadedStream,
#FormDataParam("file") FormDataContentDisposition fileDetails,
#PathParam("fileName") String fileName) throws Exception;

Cannot Compress Java Server Response Sent To JSP

I am having trouble in returning compressed response (GZip) from my Java Servlet, to a JSP.
Flow :
Request comes to java servlet
Process the request, and create a JSON object, with the response
Convert the JSON object to string
Compress the response string with GZip
The compressed response string is set as attribute in the request object and control passed to JSP
In the JSP, the response string (compressed) is printed on screen
Precautions :
Request object has "Accepting-Encoding" set with "gzip"
Response header has "Content-Encoding" set to "gzip"
Response content type is set to "application/json"
Response character encoding is set to "ISO-8859-1"
Result :
Firefox shows "Content Encoding Error"
Chrome shows "Error 330 (net::ERR_CONTENT_DECODING_FAILED): Unknown error."
Can anyone help point me out, in the right direction please?

The compressed response string is set as attribute in the request object and control passed to JSP
You shouldn't have forwarded a JSON response to a JSP. You should have printed the JSON plain to the response and have the JavaScript/Ajax code in your JSP Android app to call the URL of the servlet which returns the JSON. See also How to use Servlets and Ajax?.
As to the GZIP compression, you shouldn't do it yourself. Let the server do itself.
Fix your code to remove all manual attempts to compress the response, it should end up to basically look like this:
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String json = createItSomehow();
response.setContentType("application/json");
response.setCharacterEncoding("UTF-8");
response.getWriter().write(json);
}
That's all, if you let your Android app call the URL of the servlet, it'll retrieve the JSON string.
Finally edit the server configuration to turn on automatic GZIP compression. In case of for example Tomcat, that would be a matter of adding compression="on" to the <Connector> element in Tomcat's /conf/server.xml file:
<Connector ... compression="on">
As per the documentation, the compressable mime types defaults to text/html,text/xml,text/plain. You can configure this to add application/json.
<Connector ... compression="on" compressableMimeType="text/html,text/xml,text/plain,application/json">
Unrelated to the concrete problem, the response character encoding must be set to UTF-8 which is as per the JSON specification.

JSPs are for rendering textual data to the client. GZIP is binary data, even if it is compressed text underneath.
I suggest using a GZIP servlet filter to compress your data on the fly, instead of doing it programmatically in your business logic.
See this prior question for how to get hold of one off-the shelf: Which compression (is GZIP the most popular) servlet filter would you suggest?
Failing that, then write your own servlet filter that does the same thing.