I have a Rest enpoint that accept a list of MultipartFile like this:
#CrossOrigin(origins = "*", methods = {RequestMethod.POST})
#PostMapping(value = "/files", consumes = MediaType.MULTIPART_FORM_DATA_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
public #ResponseBody
ResponseEntity<UploadFileResponse> uploadDocuments(#RequestPart("files") List<MultipartFile> files) throws InfoNotFoundException, IOException {
log.info("Upload Documents controller");
if (files == null || files.isEmpty()) {
throw new RuntimeException("You must select at least one file for uploading");
}
UploadFileResponse uploadFileResponse = service.uploadFiles(files);
ResponseEntity<UploadFileResponse> response = new ResponseEntity<UploadFileResponse>(uploadFileResponse, HttpStatus.OK);
return response;
}
This is calling a service that has call a method inside to save the information in a ObjectStorage from oracle, like this:
public void upload(MultipartFile file) throws Exception {
HSAClientConfiguration conf = new HSAClientConfiguration();
UploadObject app = new UploadObject();
String fileName = "oci/oci_api_key.pem";
InputStream is = app.getFileFromResourceAsStream(fileName);
String result = IOUtils.toString(is, StandardCharsets.UTF_8);
log.info("Authenticating...");
AuthenticationDetailsProvider authenticationDetailsProvider =
SimpleAuthenticationDetailsProvider.builder()
.tenantId(conf.getTenantId())
.userId(conf.getUserId())
.fingerprint(conf.getFingerprint())
.privateKeySupplier(new StringPrivateKeySupplier(result))
.build();
ObjectStorage client = new ObjectStorageClient(authenticationDetailsProvider);
client.setRegion(Region.EU_FRANKFURT_1);
//Construccion del nombre del archivo.
String fecha = DateUtil.convertToYYYYMM(Timestamp.from(Instant.now()));
String objectName = fecha + "/" + file.getOriginalFilename();
log.info("Loading the file to Object Storage with name: " + objectName);
//Convertir el fichero para pasar en el putObject
InputStream inputStream = file.getInputStream();
log.info("Creating the source object to send");
PutObjectRequest putObjectRequest =
PutObjectRequest.builder()
.namespaceName(conf.getNameSpace())
.bucketName(conf.getBucket())
.objectName(objectName)
.contentLength(file.getSize())
.putObjectBody(inputStream)
.build();
client.putObject(putObjectRequest);
PutObjectResponse putObjectResponse = client.putObject(putObjectRequest);
System.out.println("Response: " + putObjectRequest);
}
If instead of Multipart, I pass to this code the Inputstream of a file in the system, it will be save correctly. But when I use the MultipartFile, I recevie an error from the RestClient like this:
com.oracle.bmc.http.internal.RestClient : Error calling available on the stream to get the available number of bytes
Look like this is produce in the clases from oracle that try to serialize the object. I don't know why. Does anyone know how to serialize a InputStream or deal with this?
Thanks
It was a stupid problem with the response that try to serialize the InputStream. If you remove PutObjectResponse putObjectResponse = client.putObject(putObjectRequest); The code works smooth.
Enjoy it!
Related
I have a controller in gateway microservice that accepts the MultipartFile and resends to the service behind it
#PostMapping
public ResponseEntity upload(#ApiParam(name = "file", value = "File", required = true) MultipartFile file)
throws BaseException {
if (Objects.isNull(file)){
throw new CheckFieldException("file", MultipartFile.class);
}
if (megabyte * maxFileSize - file.getSize() < 0){
return ResponseEntity.accepted().body(new DocumentResponseDTO(false, "File size exceeds " + maxFileSize + "MB"));
}
DiscoveryConfig.CashTracking config = discoveryConfig.getCashTracking();
UriComponents uriStatementUpload = UriComponentsBuilder.newInstance().scheme(config.getScheme())
.host(config.getHost()).port(config.getPort()).path(config.getExcelNominalOperationsPath()).build(true);
try {
HttpEntity<byte[]> fileEntity = new HttpEntity(file.getBytes());
ResponseEntity<DocumentResponseDTO> entity = restTemplate.postForEntity(uriStatementUpload.toUri(), fileEntity, DocumentResponseDTO.class);
return entity;
} catch (HttpClientErrorException e) {
return ResponseEntity.status(e.getStatusCode()).body(e.getResponseBodyAsString());
} catch (IOException e) {
return ResponseEntity.status(500).body("IOException while getting bytes stream from file");
}
}
and in CashTracking service there is also file upload like that:
#PostMapping(value = "/upload")
public ResponseEntity uploadExcelNominalOperationsFile(#ApiParam(name = "file", value = "File", required = true) MultipartFile file) throws IOException {
try (InputStream is = file.getInputStream()) {
log.info("Processing incoming Excel file with nominal operations");
Workbook workbook = new XSSFWorkbook(is);
log.info("Processing workbook");
Sheet sheet = workbook.getSheetAt(0);
log.info("Processing the first sheet");
List<NominalOperationVO> nominalOperationVOs = new ArrayList<>();
List<String> fileHeaders = new ArrayList<>();
And when the file is actually uploaded to the gateway service, the service behind it starts processing the file upload, but the MultipartFile file is null. I have explicitly put it in the Entity I have sent to the service behind the gateway, the question, what I'm doing wrong if it is null? If I do upload to that microservice directly, it process the request correctly.
The main stuff I was missing was putting the Http headers per specific multipart form's parts. They should be identical to what has been sent to the gateway service.
public ResponseEntity upload(#ApiParam(name = "file", value = "Файл", required = true) MultipartFile file)
throws BaseException {
if (Objects.isNull(file)){
throw new CheckFieldException("file", MultipartFile.class);
}
if (megabyte * maxFileSize - file.getSize() < 0){
return ResponseEntity.accepted().body(new DocumentResponseDTO(false, "File size exceeds " + maxFileSize + "MB"));
}
DiscoveryConfig.CashTracking config = discoveryConfig.getCashTracking();
UriComponents uriStatementUpload = UriComponentsBuilder.newInstance().scheme(config.getScheme())
.host(config.getHost()).port(config.getPort()).path(config.getExcelNominalOperationsPath()).build(true);
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
MultipartBodyBuilder multipartBodyBuilder = new MultipartBodyBuilder();
//here is the really needed stuff with 2 headers
Resource resource = new ByteArrayResource(file.getBytes());
multipartBodyBuilder.part("file", resource)
.header("Content-Type",file.getContentType())
.header("Content-Disposition","form-data; name=\"file\"; filename=\""+file.getOriginalFilename()+"\"");
// multipart/form-data request body
MultiValueMap<String, HttpEntity<?>> body = multipartBodyBuilder.build();
HttpEntity<MultiValueMap<String, HttpEntity<?>>> requestEntity
= new HttpEntity<>(body, headers);
ResponseEntity<DocumentResponseDTO> entity = restTemplate.postForEntity(uriStatementUpload.toUri(), requestEntity, DocumentResponseDTO.class);
return entity;
} catch (HttpClientErrorException e) {
return ResponseEntity.status(e.getStatusCode()).body(e.getResponseBodyAsString());
} catch (IOException e) {
return ResponseEntity.status(500).body("IOException while getting bytes stream from file");
}
}
I have a folder in S3 bucket that contains json files. I am using Spring Boot. The user provides folder name and specific file(json file) name.
public ResponseEntity<?> downloading(String folderName, String fileName) throws IOException {
S3Object s3Object = s3client
.getObject(new GetObjectRequest(s3BucketName, folderName + fileName));
if (s3Object.getKey().length() > 0) {
//enables the user to dowload json file
//return the object that can be dowloaded, status code
return new ResponseEntity<>( HttpStatus.OK);
}
else{
//return error message and status code
return new ResponseEntity<>( HttpStatus.NOT_FOUND);
}
}
I have referred the answer from here.
So, I have added just a part of the code that will help you :
public ResponseEntity<byte[]> downloading(String folderName, String fileName) {
S3Object s3Object = s3client
.getObject(new GetObjectRequest(s3BucketName, folderName + fileName));
if (s3Object.getKey().length() > 0) {
S3ObjectInputStream input = s3Object.getObjectContent();
byte[] bytes = IOUtils.toByteArray(input);
String file = URLEncoder.encode(s3Object.getKey(), "UTF-8").replaceAll("\\+", "%20");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headers.setContentLength(bytes.length);
headers.setContentDispositionFormData("attachment", file);
return new ResponseEntity<byte[]>(bytes, HttpStatus.OK);
}
return new ResponseEntity<byte[]>(null, HttpStatus.NOT_FOUND);
}
Thanks. :)
Requirement:
I need to create a Rest API which can allows to download a file as well as a JSON response.
I already have 2 different APIs to solve the purpose, but now I need to merge these APIs to a single one.
public ResponseEntity<InputStreamResource> downloadFile1(
#RequestParam(defaultValue = DEFAULT_FILE_NAME) String fileName) throws IOException {
MediaType mediaType = MediaTypeUtils.getMediaTypeForFileName(this.servletContext, fileName);
System.out.println("fileName: " + fileName);
System.out.println("mediaType: " + mediaType);
File file = new File(DIRECTORY + "/" + fileName);
InputStreamResource resource = new InputStreamResource(new FileInputStream(file));
return ResponseEntity.ok()
// Content-Disposition
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=" + file.getName())
// Content-Type
.contentType(mediaType)
// Contet-Length
.contentLength(file.length()) //
.body(resource);
}
Above is the existing code that only return a file to download but I need a json response as well.
You need to return Multipart content. See for example
https://github.com/juazugas/spring-boot-multipart/blob/master/src/main/java/com/example/demo/server/MultiEndpoint.java
The code
#GET
#Produces("multipart/mixed")
public MultipartBody getMulti2(#QueryParam("name") String name) {
List<Attachment> attachments = new LinkedList<>();
attachments.add(new Attachment("root", "application/json", service.getEntity(name)));
attachments.add(new Attachment("image", "application/octet-stream", service.getEntityData(name)));
return new MultipartBody(attachments, true);
}
Server Code :
#POST
#Path("reportDownload")
#Consumes(MediaType.APPLICATION_JSON)
public Response generateReport(QueryData queryData) {
File file = new File("report.xlsx") // large file
StreamingOutput stream = new FileStreamingOutput(file) ;
return Response.ok(stream, MediaType.APPLICATION_OCTET_STREAM)
.header("filename" , file.getName())
.build();
}
Client Code :
Using the following code I'm able to download files upto some limit. Getting out of memory heap error for large files.
final String uri = buildUri("/reportGenerate/reportDownload");
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(read_timeout);
factory.setConnectTimeout(connection_timeout);
RestTemplate restTemplate = new RestTemplate(factory);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
List<MediaType> mediaTypeList = new ArrayList<>();
mediaTypeList.add(MediaType.APPLICATION_OCTET_STREAM);
headers.setAccept(mediaTypeList);
HttpEntity entity = new HttpEntity(queryData, headers);
ResponseEntity<byte[]> data = restTemplate.exchange(uri, HttpMethod.POST, entity, byte[].class);
HttpHeaders responseHeader = data.getHeaders();
String fileName = (String) responseHeader.get("filename").get(0);
String downloadFolder = ApplicationConfig.REPORT_DOWNLOAD_FOLDER.getValue();
if (data.getStatusCode() == HttpStatus.OK) {
FileOutputStream fos = null;
File toFile = null;
try {
toFile = new File(downloadFolder + File.separator + fileName);
fos = new FileOutputStream(toFile);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
IOUtils.write(data.getBody(), bos);
bos.writeTo(fos);
} catch (Exception e) {
convertReportException(e);
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException ex) {
convertReportException(ex);
}
}
return toFile;
}
}
How to use stream for download larger files.
Here is how I do it with a ResponseExtractor. Based on hints from this Spring Jira issue.
RestTemplate restTemplate // = ...;
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<Void> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("some/path");
Files.copy(response.getBody(), path);
return null;
};
restTemplate.execute(URI.create("www.something.com"), HttpMethod.GET, requestCallback, responseExtractor);
update
Here is what RestTemplate does behind the scenes on postForObject and friends (inline comments from me):
#Override
public <T> T postForObject(String url, Object request, Class<T> responseType, Map<String, ?> uriVariables)
throws RestClientException {
// From RequestCallback's javadoc:
// Callback interface for code that operates on a ClientHttpRequest.
// Allows to manipulate the request headers, and write to the request body.
//
// Used internally by the RestTemplate, but also useful for application code.
RequestCallback requestCallback = httpEntityCallback(request, responseType);
// HttpMessageConverterExtractor checks the response type header and requested
// responseType class to select the proper message converter to handle the response.
// It also implements ResponseExtractor.
HttpMessageConverterExtractor<T> responseExtractor =
new HttpMessageConverterExtractor<T>(responseType, getMessageConverters(), logger);
return execute(url, HttpMethod.POST, requestCallback, responseExtractor, uriVariables);
}
/**
* Returns a request callback implementation that writes the given object to the
* request stream.
*/
protected <T> RequestCallback httpEntityCallback(Object requestBody, Type responseType) {
return new HttpEntityRequestCallback(requestBody, responseType);
}
Note: This is essentially a duplicate of my answer at https://stackoverflow.com/a/38664475/1030527 but I can't mark the questions as duplicate since neither this one or that one have upvoted answers.
At the client that you mentioned
don't store the file in memory to download a large via the RestTemplate, it can cause the Java heap exception.
it should be stored on disk.
Here is some code sample to download a large file via the RestTemplate
#GetMapping("largeFile")
public ResponseEntity<InputStreamResource> downloadLargeFile(
#RequestParam("fileName") String fileName
) throws IOException {
RestTemplate restTemplate = new RestTemplate();
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<InputStreamResource> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("tmp/" + fileName);
Files.copy(response.getBody(), path, StandardCopyOption.REPLACE_EXISTING);
return new InputStreamResource(new FileInputStream(String.format("tmp/%s", fileName)));
};
InputStreamResource response = restTemplate.execute(
String.format("http://%s:%s/file/largeFileRestTemplate?fileName=%s", host, "9091", fileName),
HttpMethod.GET,
requestCallback,
responseExtractor
);
return ResponseEntity
.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, String.format("attachment; filename=%s", fileName))
.body(response);
}
I have the following code which I use for posting a file to a service and it works fine.
The only problem I have, is that I have to write a temporary file to get a FileSystemResource for posting the object with the restTemplate
Is there anyway I can adapt the following code so that I dont have to write a temporary file?
public String postNewIcon2(Integer fileId, MultipartFile multiPartfile) {
LOG.info("Entered postNewIcon");
Map<String, Object> params = getParamsWithAppKey();
params.put("fileId", fileId);
String result = null;
File tempFile = null;
try {
String originalFileNameAndExtension = multiPartfile.getOriginalFilename();
String tempFileName = "c:\\temp\\image";
String tempFileExtensionPlusDot = ".png";
tempFile = File.createTempFile(tempFileName, tempFileExtensionPlusDot);
multiPartfile.transferTo(tempFile);
FileSystemResource fileSystemResource = new FileSystemResource(tempFile);
// URL Parameters
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("file", fileSystemResource);
// Post
result = restTemplate.postForObject(getFullURLAppKey(URL_POST_NEW_ICON), parts, String.class, params);
} catch (RestClientException restClientException) {
System.out.println(restClientException);
} catch (IOException ioException) {
System.out.println(ioException);
} finally {
if (tempFile != null) {
boolean deleteTempFileResult = tempFile.delete();
LOG.info("deleteTempFileResult: {}", deleteTempFileResult);
}
}
return result;
}
Thank you
Answer with help with Kresimir Nesek and this link
Sending Multipart File as POST parameters with RestTemplate requests
The following code did the trick - no need for a temporary file now
MultiValueMap<String, Object> map = new LinkedMultiValueMap<String, Object>();
final String filename="somefile.txt";
map.add("name", filename);
map.add("filename", filename);
ByteArrayResource contentsAsResource = new ByteArrayResource(content.getBytes("UTF-8")){
#Override
public String getFilename(){
return filename;
}
};
map.add("file", contentsAsResource);
String result = restTemplate.postForObject(urlForFacade, map, String.class);
MultipartFile needs to have some temp location.
Please try this code, to get physical file:
private File getTempFile(MultipartFile attachment){
CommonsMultipartFile commonsMultipartFile = (CommonsMultipartFile) attachment;
DiskFileItem diskFileItem = (DiskFileItem) commonsMultipartFile.getFileItem();
return diskFileItem.getStoreLocation();
}