I want to send a big object (lets say 4G) over http.
We have a custom serializer that writes the object to an OutputStream. At the moment we write the object to disk and use that file for the inputstream which is used for the request.
Something like these lines:
private static Response sendObject(Object bigObject) throws IOException {
File tempFile = File.createTempFile("x", "y");
OutputStream out = new FileOutputStream(tempFile);
CustomSerializer.serialize(bigObject, out);
out.close();
WebTarget resource = service.path("data");
FormDataMultiPart multiPartEntity = new FormDataMultiPart();
InputStream inputStream = new FileInputStream(tempFile);
StreamDataBodyPart streamBodyPart = new StreamDataBodyPart(
"data",
inputStream,
"data",
MediaType.APPLICATION_OCTET_STREAM_TYPE);
MultiPart multiPart = multiPartEntity.bodyPart(streamBodyPart);
return resource.request(MediaType.APPLICATION_JSON_TYPE)
.post(Entity.entity(multiPart, multiPart.getMediaType()));
}
We save some memory because we don't serialize to a byte array in memory. Thats nice. But could I save the memory without writing to disk.
Could you write directly to the input stream without rewriting the CustomSerializer?
Could you write directly to the input stream while it reads into the request?
-
It is a little hard to explain, But I think I am after something like this pseudo code:
private static Response sendObject(Object bigObject) throws IOException {
WebTarget resource = service.path("data");
FormDataMultiPart multiPartEntity = new FormDataMultiPart();
// A type of stream I don't know if exist
OutputStream outIn = new OutputInputStream() {
public void openInputStream() {
CustomSerializer.serialize(bigObject, this);
}
};
StreamDataBodyPart streamBodyPart = new StreamDataBodyPart(
"data",
outIn.getInputStream(),
"data",
MediaType.APPLICATION_OCTET_STREAM_TYPE);
MultiPart multiPart = multiPartEntity.bodyPart(streamBodyPart);
return resource.request(MediaType.APPLICATION_JSON_TYPE)
.post(Entity.entity(multiPart, multiPart.getMediaType()));
}
You could use a StreamingOutput and use your CustomSerializer to write to the provided OutputStream
StreamingOutput entity = new StreamingOutput() {
#Override
public void write(OutputStream out)
throws IOException, WebApplicationException {
CustomSerializer.serialize(bigObject, out);
}
};
The write() method will be called by Jersey, giving you a chance to write directly to response entity stream.
Then just use a FormDataBodyPart
BodyPart bigPart = new FormDataBodyPart(
"data", entity, MediaType.APPLICATION_OCTET_STREAM_TYPE);
MultiPart multiPart = new FormDataMultiPart().bodyPart(bigPart);
Related
I have a Rest enpoint that accept a list of MultipartFile like this:
#CrossOrigin(origins = "*", methods = {RequestMethod.POST})
#PostMapping(value = "/files", consumes = MediaType.MULTIPART_FORM_DATA_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody
ResponseEntity<UploadFileResponse> uploadDocuments(#RequestPart("files") List<MultipartFile> files) throws InfoNotFoundException, IOException {
log.info("Upload Documents controller");
if (files == null || files.isEmpty()) {
throw new RuntimeException("You must select at least one file for uploading");
}
UploadFileResponse uploadFileResponse = service.uploadFiles(files);
ResponseEntity<UploadFileResponse> response = new ResponseEntity<UploadFileResponse>(uploadFileResponse, HttpStatus.OK);
return response;
}
This is calling a service that has call a method inside to save the information in a ObjectStorage from oracle, like this:
public void upload(MultipartFile file) throws Exception {
HSAClientConfiguration conf = new HSAClientConfiguration();
UploadObject app = new UploadObject();
String fileName = "oci/oci_api_key.pem";
InputStream is = app.getFileFromResourceAsStream(fileName);
String result = IOUtils.toString(is, StandardCharsets.UTF_8);
log.info("Authenticating...");
AuthenticationDetailsProvider authenticationDetailsProvider =
SimpleAuthenticationDetailsProvider.builder()
.tenantId(conf.getTenantId())
.userId(conf.getUserId())
.fingerprint(conf.getFingerprint())
.privateKeySupplier(new StringPrivateKeySupplier(result))
.build();
ObjectStorage client = new ObjectStorageClient(authenticationDetailsProvider);
client.setRegion(Region.EU_FRANKFURT_1);
//Construccion del nombre del archivo.
String fecha = DateUtil.convertToYYYYMM(Timestamp.from(Instant.now()));
String objectName = fecha + "/" + file.getOriginalFilename();
log.info("Loading the file to Object Storage with name: " + objectName);
//Convertir el fichero para pasar en el putObject
InputStream inputStream = file.getInputStream();
log.info("Creating the source object to send");
PutObjectRequest putObjectRequest =
PutObjectRequest.builder()
.namespaceName(conf.getNameSpace())
.bucketName(conf.getBucket())
.objectName(objectName)
.contentLength(file.getSize())
.putObjectBody(inputStream)
.build();
client.putObject(putObjectRequest);
PutObjectResponse putObjectResponse = client.putObject(putObjectRequest);
System.out.println("Response: " + putObjectRequest);
}
If instead of Multipart, I pass to this code the Inputstream of a file in the system, it will be save correctly. But when I use the MultipartFile, I recevie an error from the RestClient like this:
com.oracle.bmc.http.internal.RestClient : Error calling available on the stream to get the available number of bytes
Look like this is produce in the clases from oracle that try to serialize the object. I don't know why. Does anyone know how to serialize a InputStream or deal with this?
Thanks
It was a stupid problem with the response that try to serialize the InputStream. If you remove PutObjectResponse putObjectResponse = client.putObject(putObjectRequest); The code works smooth.
Enjoy it!
Resteasy docs does not explain who is responsible for closing streams passed to MultipartFormDataOutput. Let's consider the following example:
WebTarget target = ClientBuilder.newClient().target("url");
MultipartFormDataOutput formData = new MultipartFormDataOutput();
FileInputStream fis1 = new FileInputStream(new File("/path/to/image1"));
FileInputStream fis2 = new FileInputStream(new File("/path/to/image2"));
formData.addFormData("image", fis1, MediaType.APPLICATION_OCTET_STREAM_TYPE);
formData.addFormData("image", fis2, MediaType.APPLICATION_OCTET_STREAM_TYPE);
Entity<MultipartFormDataOutput> entity = Entity.entity(formData, MediaType.MULTIPART_FORM_DATA);
Response response = target.request().post(entity);
Will the fis1 and fis2 be closed by the resteasy or the user should take care of closing these streams?
I would suggest to use try-with-resource to be sure that they will be closed.
WebTarget target = ClientBuilder.newClient().target("url");
MultipartFormDataOutput formData = new MultipartFormDataOutput();
try(FileInputStream fis1 = new FileInputStream(new File("/path/to/image1")));
FileInputStream fis2 = new FileInputStream(new File("/path/to/image2")))
{
formData.addFormData("image", fis1, MediaType.APPLICATION_OCTET_STREAM_TYPE);
formData.addFormData("image", fis2, MediaType.APPLICATION_OCTET_STREAM_TYPE);
Entity<MultipartFormDataOutput> entity = Entity.entity(formData,
MediaType.MULTIPART_FORM_DATA);
Response response = target.request().post(entity);
}
So I can answer my question myself, hope someone will benefit from this.
Resteasy will close the passed stream. In my case, the InputStreamProvider will take care of closing the FileInputStream.
public void writeTo(InputStream inputStream, Class<?> type, Type genericType, Annotation[] annotations, MediaType mediaType, MultivaluedMap<String, Object> httpHeaders, OutputStream entityStream) throws IOException
{
LogMessages.LOGGER.debugf("Provider : %s, Method : writeTo", getClass().getName());
try
{
int c = inputStream.read();
if (c == -1)
{
httpHeaders.putSingle(HttpHeaderNames.CONTENT_LENGTH, Integer.toString(0));
entityStream.write(new byte[0]); // fix RESTEASY-204
return;
}
else
entityStream.write(c);
ProviderHelper.writeTo(inputStream, entityStream);
}
finally
{
inputStream.close();
}
}
I have a Get API by which can download PDF. Using Spring rest template I am able to get content but when I am creating PDF file it's creating a blank pdf.
I am using byte[] to create a new file.
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_PDF));
HttpEntity<String> entity = new HttpEntity<>(headers);
ResponseEntity<String> result =
restTemplate.exchange(uri, HttpMethod.GET, entity, String.class);
String content = response.getBody();
byte[] bytes = content.getBytes();
Files.write(Paths.get("/home/123.pdf"), bytes, StandardOpenOption.CREATE );
Please suggest me anyway to do it, Finally my objective
to upload in S3.
Your request headers should also include MediaType.APPLICATION_OCTET_STREAM
This response object will return a byte array- which will be your pdf.
So, the complete example would be something like this-
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_PDF, MediaType.APPLICATION_OCTET_STREAM));
HttpEntity<String> entity = new HttpEntity<>(headers);
ResponseEntity<byte[]> result =
restTemplate.exchange(uri, HttpMethod.GET, entity, byte[].class);
byte[] content = result.getBody();
Files.write(Paths.get("/home/123.pdf"), content, StandardOpenOption.CREATE );
Hope this helps.
You can Use PDFbox(or itext) API, which is a great way of parsing and creating PDF. It will allow you to place your text etc,
That said
public static void writeToFIle(InputStream uploadedInputStream, String uploadedFileLocation) {
try {
OutputStream out = new FileOutputStream(new File(uploadedFileLocation));
int read = 0;
byte[] bytes = new byte[1024];
out = new FileOutputStream(new File(uploadedFileLocation));
while ((read = uploadedInputStream.read(bytes)) != -1) {
out.write(bytes, 0, read);
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
should be enough to create a PDF, if its blank check if the value arn't null
I personnaly use jersey to link client and server so i cant really tell you if your method to get the pdf works
I have a Servlet which makes a request to my Rest API, and I want it to return the API Response content to the final user through the HttpServletResponse.
The content is actually a .xls file to download which I put in the Response with the StreamingOutput Object.
How can I do that ? I can't cast the Response into a HttpServletResponse
Rest API method :
#GET
#Produces( MediaType.APPLICATION_JSON )
#Path("bla")
public Response getTopicByName() {
final Workbook wb = new HSSFWorkbook();
StreamingOutput stream = new StreamingOutput() {
#Override
public void write(OutputStream output) throws IOException, WebApplicationException {
wb.write(output);
}
};
responseBuilder = responseBuilder.entity(stream);
responseBuilder = responseBuilder.status(Response.Status.OK);
responseBuilder = responseBuilder.header("Content-Disposition", "attachment; filename=" + device + ".xls");
return responseBuilder.build();
}
Servlet POST method :
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
Client client = ClientBuilder.newClient();
WebTarget target = client.target(url);
Response res = target. request().get();
if (res.getStatus() == 200) {
// how to put res stream into response stream ?
ServletOutputStream stream = response.getOutputStream();
}
client.close();
}
EDIT :
I tried TedTrippin method and after finding out the way to recover an InputStream from the Response, it worked well.
But I keep getting corrupted xls files. And it is quite annoying. I don't get those corrupted files when I make the request directly from the browser.
Got any clues where it comes from ?
POST method :
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
Client client = ClientBuilder.newClient();
WebTarget target = client.target(url + param + format);
Response res = target.request().get();
if (res.getStatus() == 200) {
response.setHeader("Content-Disposition", "attachment; filename=test.xls");
InputStream in = res.readEntity(InputStream.class);
ServletOutputStream out = response.getOutputStream();
byte[] buffer = new byte[1024];
while (in.read(buffer) >= 0) {
out.write(buffer);
}
out.flush();
}
client.close();
}
Simplest way is to read the response stream and write it straight to the response output stream. Either use a library function from IOUtils or Guava or pure java...
try (InputStream in = ...;
OutputStream out = ...) {
byte[] buffer = new byte[1024];
while (in.read(buffer) >= 0)
out.write(buffer);
} catch (IOException ex) {
...
}
A nicer (depending on your view) way would be to read/save the response as a temporary file then you could return that or write it to the output stream.
Third approach would be to create a pipe, but I don't think that would be applicable here.
I´m trying to open a pdf that I have created using iText library in my browser, but it fails.
This is the code I´m using to send to browser
File file = new File(path);
try{
//InputStream stream=blob.getBinaryStream();
InputStream streamEntrada = new FileInputStream(file);
//ServletOutputStream fileOutputStream = response.getOutputStream();
PrintWriter print = response.getWriter();
int ibit = 256;
while ((ibit) >= 0)
{
ibit = streamEntrada.read();
print.write(ibit);
}
response.setContentType("application/text");
response.setHeader("Content-Disposition", "attachment;filename="+name);
response.setHeader("Pragma", "cache");
response.setHeader("Cache-control", "private, max-age=0");
streamEntrada.close();
print.close();
return null;
}
catch(Exception e){
return null;
}
}
I tried with FileOutputStream but isn´t works. I´m desperate.
Thank you.
Now, I´m trying this way, but it doesn´t work:
public class MovilInfoAction extends DownloadAction{
protected StreamInfo getStreamInfo(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception {
//Here the creation of the PDF
//Storing data
PdfData dPdf = pdf.drawPDF(terminal);
String path = dPdf.getPath();//Path
String name = dPdf.getName()+".pdf";//Pdf´s name
String contentType = "application/pdf";//ContentType
response.setContentType(contentType);
response.setHeader("Content-Disposition","attachment; filename="+name);
response.setHeader("Cache-control", "private, max-age=0");
response.setHeader("Content-Disposition", "inline");
File file = new File(path);
byte[] pdfBytes = es.vodafone.framework.utils.Utils.getBytesFromFile(file);
return new ByteArrayStreamInfo(contentType, pdfBytes);
}
protected class ByteArrayStreamInfo implements StreamInfo {
protected String contentType;
protected byte[] bytes;
public ByteArrayStreamInfo(String contentType, byte[] bytes) {
this.contentType = contentType;
this.bytes = bytes;
}
public String getContentType() {
return contentType;
}
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(bytes);
}
}
}
You specify the mimetype as application/text, when it should be application/pdf.
You should set the Header and ContentType before you write the data.
And set the Content Type to application/pdf.
change
response.setContentType("application/text");
to
response.setContentType("application/pdf");
and if you want your pdf to open in browser then make following change
response.setHeader("Content-Disposition", "inline");
Put the filename in double quote "
response.setHeader("Content-Disposition","attachment; filename=\"" + attachmentName + "\"");
Android Default Browser requires GET Request. It does not understand POST Request and hence cannot download the attachment. You can send a GET request as by sending GET request, it resolved my problem. Android browser generates a GET request on its own and sends it back to server. The response received after second request will be considered final by the browser even if GET request is sent on first time by the servlet.