Develop Reference

API response to file

I'm receiving this response from an API and I'm trying to convert it to a file in Java, but when the file is created it says that the file is corrupted.
Here's the code that i'm using.
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(serverUrl)
.queryParam("fileID", fileID);
HttpEntity<?> entity = new HttpEntity<>(headers);
ResponseEntity<?> response = restTemplate.exchange(
builder.toUriString(),
HttpMethod.POST,
entity,
String.class);
System.out.println("Bytes de response body: " + response.getBody());
saveFile("C:\\Users\\xxx\\Documents\\" + fileName,body);
}
private static void saveFile(String path, String octetStream) throws IOException{
try(FileOutputStream fos = new FileOutputStream(path)){
fos.write(octetStream.getBytes());
}
Any input will be appreciated.
Thanks.

Answer:
I changed the response entity type from String to byte []
ResponseEntity<byte[]> response = restTemplate.exchange(
builder.toUriString(),
HttpMethod.POST,
entity,
byte[].class);

how to downlod file using resttemplate

String uri = "URL";
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new ByteArrayHttpMessageConverter());
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
HttpEntity<String> entity = new HttpEntity<String>(headers);
ResponseEntity<byte[]> response = restTemplate.exchange(uri, HttpMethod.GET, entity, byte[].class, "1");
I get response like-> <200,[B#c505096,[Pragma:"no-cache", Content-Disposition:"inline; filename=935436242330664960_pratikpopo.txt", Expires:"0", Cache-Control:"no-cache, no-store, must-revalidate", Content-Type:"application/octet-stream", Content-Length:"18", Date:"Mon, 31 Jan 2022 04:16:56 GMT"]> I want to downlod (935436242330664960_pratikpopo.txt) this file. Is there any way to download this file

{
URL link = new URL("your full url http://........");
InputStream in = new BufferedInputStream(link.openStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int n = 0;
while (-1!=(n=in.read(buf)))
{
out.write(buf, 0, n);
}
out.close();
in.close();
byte[] response = out.toByteArray();
FileOutputStream fos = new FileOutputStream("C:/download-file/YourFileName.txt");
fos.write(response);
fos.close();
}

#RequestMapping(value = "/download3/{path}", method = RequestMethod.GET)
public String downloadFile4( #RequestParam("file_name") String path) throws IOException
{
String uri = "File Path-URL(location wehere your file is stored on server)";
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new ByteArrayHttpMessageConverter());
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
HttpEntity<String> entity = new HttpEntity<String>(headers);
ResponseEntity<byte[]> response = restTemplate.exchange(uri, HttpMethod.GET, entity, byte[].class, "1");
URL furl = new URL(uri);
ReadableByteChannel rbc = Channels.newChannel(furl.openStream());
FileOutputStream fos = new FileOutputStream("C:/download-file/"+ response.getHeaders().getContentDisposition().getFilename());
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
return "file downloaded";
}

InputStream / RestTemplate taking forever to copy file

I have this block of code that copies a file to a remote server using an InputStream:
final InputStream fis = new FileInputStream(new File(CACHED_CLONE_FILE_NAME));
final RequestCallback requestCallback = new RequestCallback()
{
public void doWithRequest(final ClientHttpRequest request)
{
try {
request.getHeaders().add("Content-type", "application/octet-stream");
OutputStream procOutput = request.getBody();
BufferedOutputStream bos = new BufferedOutputStream(procOutput);
IOUtils.copy(fis, bos);
}
catch (IOException ex) {
}
}
};
final RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new StringHttpMessageConverter());
restTemplate.getMessageConverters().add(new MappingJacksonHttpMessageConverter());
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
final HttpMessageConverterExtractor<String> responseExtractor = new HttpMessageConverterExtractor<String>(String.class, restTemplate.getMessageConverters());
restTemplate.execute(urlToPost, HttpMethod.POST, requestCallback, responseExtractor);
It works, except that for a small file (4MB) it takes hours to copy it over. Why is this occurring, and what can I do about the performance?
This code is being used on an Android device, and there are no network problems.

File upload of CommonsMultipartFile using RestTemplate is failing

I'm trying to consume a web service that accepts a CommonsMultipartFile in the request. So, I created an HTTP client using Spring's RestTemplate. Below is the method that takes in URI and a MultipartFile as parameters. I'm trying to pass this file to the web service in the form of ByteArrayResource.
public String upload(String uri, MultipartFile file) throws IOException {
logger.info("URI: " + uri);
ByteArrayResource fileAsResource = new ByteArrayResource(file.getBytes()) {
#Override
public String getFilename() {
return file.getOriginalFilename();
}
};
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("file", fileAsResource);
parts.add("fileName", file.getOriginalFilename());
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<MultiValueMap<String, Object>>(parts, httpHeaders);
ResponseEntity<String> responseEntity = rest.exchange(uri, HttpMethod.POST, requestEntity, String.class);
this.setStatus(responseEntity.getStatusCode());
return responseEntity.getBody();
}
This is how I'm creating a CommonsMultipartFile:
private MultipartFile getCommonsMultipartFile() throws FileNotFoundException, IOException {
File file = new File("C:\\Dummy_Test.txt");
DiskFileItemFactory factory = new DiskFileItemFactory();
FileItem fileItem = factory.createItem( "file", "multipart/form-data", false, "Dummy_Test.txt" );
IOUtils.copy(new FileInputStream(file), fileItem.getOutputStream());
MultipartFile commonsMultipartFile = new CommonsMultipartFile(fileItem);
return commonsMultipartFile;
}
But whenever I run this client to hit the web service I keep getting this error.
org.springframework.web.client.ResourceAccessException: I/O error: resource loaded from byte array cannot be resolved to absolute file path; nested exception is java.io.FileNotFoundException: resource loaded from byte array cannot be resolved to absolute file path
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:453)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:401)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:377)
at com.attikala.service.UploaderService.upload(UploaderService.java:118)
at com.attikala.service.UploaderService.main(UploaderService.java:55)
Caused by: java.io.FileNotFoundException: resource loaded from byte array cannot be resolved to absolute file path
at org.springframework.core.io.AbstractResource.getFile(AbstractResource.java:107)
at org.springframework.core.io.AbstractResource.contentLength(AbstractResource.java:116)
at org.springframework.http.converter.ResourceHttpMessageConverter.getContentLength(ResourceHttpMessageConverter.java:99)
at org.springframework.http.converter.ResourceHttpMessageConverter.write(ResourceHttpMessageConverter.java:81)
at org.springframework.http.converter.ResourceHttpMessageConverter.write(ResourceHttpMessageConverter.java:1)
at org.springframework.http.converter.FormHttpMessageConverter.writePart(FormHttpMessageConverter.java:288)
at org.springframework.http.converter.FormHttpMessageConverter.writeParts(FormHttpMessageConverter.java:252)
at org.springframework.http.converter.FormHttpMessageConverter.writeMultipart(FormHttpMessageConverter.java:242)
at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:194)
at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:1)
at org.springframework.web.client.RestTemplate$HttpEntityRequestCallback.doWithRequest(RestTemplate.java:588)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:436)
... 4 more
Can someone help me in figuring out what's happening here?
Note: If I use the below code to upload the file, it works perfectly fine.
public String upload(String uri) {
LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
FileSystemResource value = new FileSystemResource(new File("C:\\Dummy_Test.txt"));
map.add("file", value);
map.add("fileName", "Dummy_Test.txt");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<>(map, headers);
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String> responseEntity = restTemplate.exchange(uri, HttpMethod.POST, requestEntity, String.class);
return responseEntity.getBody();
}
So, I'm thinking, do I need to provide absolute path always for the file I'm uploading? I know I'm missing something here. Don't know what.
Thanks.

Finally, I found what's happening.
Here I go -
when this statement
ResponseEntity<String> responseEntity = rest.exchange(uri, HttpMethod.POST, requestEntity, String.class);
gets executed, behind the scenes it's trying to extract the file of type java.io.File from the MultipartFile passed, and then get the file length. But MultipartFile is not of that type and as a result it was throwing an exception.
To fix that I had to also override contentLength() method when creating an instance of ByteArrayResource. Ta-da!
ByteArrayResource fileAsResource = new ByteArrayResource(file.getBytes()) {
#Override
public String getFilename() {
return file.getOriginalFilename();
}
#Override
public long contentLength() throws IOException {
return file.getSize();
}
};
Hope this helps if anyone runs into the same problem.

The accepted answer did not work. In my case I had to override the getFile(), see my solution below.
registry.addResourceHandler("/my-app/user-tracking-script.js")
.setCachePeriod(0)
.resourceChain(false)
.addResolver(new ResourceResolver() {
#Override
public Resource resolveResource(final HttpServletRequest request, final String requestPath, final List<? extends Resource> locations, final ResourceResolverChain chain) {
try {
final HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
final HttpEntity<String> entity = new HttpEntity<>(headers);
// RELEVANT PART BELOW
final ResponseEntity<byte[]> response = restTemplate.exchange(userTrackingScript, HttpMethod.GET, entity, byte[].class);
final ByteArrayResource fileAsResource = new ByteArrayResource(response.getBody()) {
#Override
public File getFile() throws IOException {
final File tempFile = File.createTempFile("user-tracking", ".js");
try (final FileOutputStream out = new FileOutputStream(tempFile)) {
IOUtils.copy(new ByteArrayInputStream(response.getBody()), out);
}
return tempFile;
}
};
return fileAsResource;
} catch (final Exception e) {
log.error("Could not download user-tracking-script.js for URL: {}", userTrackingScript, e);
}
return null;
}
#Override
public String resolveUrlPath(final String resourcePath, final List<? extends Resource> locations, final ResourceResolverChain chain) {
log.error("Unexpected call to resolveUrlPath by {}", resourcePath);
return null;
}
})
;

Download large file through Spring rest template

Server Code :
#POST
#Path("reportDownload")
#Consumes(MediaType.APPLICATION_JSON)
public Response generateReport(QueryData queryData) {
File file = new File("report.xlsx") // large file
StreamingOutput stream = new FileStreamingOutput(file) ;
return Response.ok(stream, MediaType.APPLICATION_OCTET_STREAM)
.header("filename" , file.getName())
.build();
}
Client Code :
Using the following code I'm able to download files upto some limit. Getting out of memory heap error for large files.
final String uri = buildUri("/reportGenerate/reportDownload");
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(read_timeout);
factory.setConnectTimeout(connection_timeout);
RestTemplate restTemplate = new RestTemplate(factory);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
List<MediaType> mediaTypeList = new ArrayList<>();
mediaTypeList.add(MediaType.APPLICATION_OCTET_STREAM);
headers.setAccept(mediaTypeList);
HttpEntity entity = new HttpEntity(queryData, headers);
ResponseEntity<byte[]> data = restTemplate.exchange(uri, HttpMethod.POST, entity, byte[].class);
HttpHeaders responseHeader = data.getHeaders();
String fileName = (String) responseHeader.get("filename").get(0);
String downloadFolder = ApplicationConfig.REPORT_DOWNLOAD_FOLDER.getValue();
if (data.getStatusCode() == HttpStatus.OK) {
FileOutputStream fos = null;
File toFile = null;
try {
toFile = new File(downloadFolder + File.separator + fileName);
fos = new FileOutputStream(toFile);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
IOUtils.write(data.getBody(), bos);
bos.writeTo(fos);
} catch (Exception e) {
convertReportException(e);
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException ex) {
convertReportException(ex);
}
}
return toFile;
}
}
How to use stream for download larger files.

Here is how I do it with a ResponseExtractor. Based on hints from this Spring Jira issue.
RestTemplate restTemplate // = ...;
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<Void> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("some/path");
Files.copy(response.getBody(), path);
return null;
};
restTemplate.execute(URI.create("www.something.com"), HttpMethod.GET, requestCallback, responseExtractor);
update
Here is what RestTemplate does behind the scenes on postForObject and friends (inline comments from me):
#Override
public <T> T postForObject(String url, Object request, Class<T> responseType, Map<String, ?> uriVariables)
throws RestClientException {
// From RequestCallback's javadoc:
// Callback interface for code that operates on a ClientHttpRequest.
// Allows to manipulate the request headers, and write to the request body.
//
// Used internally by the RestTemplate, but also useful for application code.
RequestCallback requestCallback = httpEntityCallback(request, responseType);
// HttpMessageConverterExtractor checks the response type header and requested
// responseType class to select the proper message converter to handle the response.
// It also implements ResponseExtractor.
HttpMessageConverterExtractor<T> responseExtractor =
new HttpMessageConverterExtractor<T>(responseType, getMessageConverters(), logger);
return execute(url, HttpMethod.POST, requestCallback, responseExtractor, uriVariables);
}
/**
* Returns a request callback implementation that writes the given object to the
* request stream.
*/
protected <T> RequestCallback httpEntityCallback(Object requestBody, Type responseType) {
return new HttpEntityRequestCallback(requestBody, responseType);
}
Note: This is essentially a duplicate of my answer at https://stackoverflow.com/a/38664475/1030527 but I can't mark the questions as duplicate since neither this one or that one have upvoted answers.

At the client that you mentioned
don't store the file in memory to download a large via the RestTemplate, it can cause the Java heap exception.
it should be stored on disk.
Here is some code sample to download a large file via the RestTemplate
#GetMapping("largeFile")
public ResponseEntity<InputStreamResource> downloadLargeFile(
#RequestParam("fileName") String fileName
) throws IOException {
RestTemplate restTemplate = new RestTemplate();
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<InputStreamResource> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("tmp/" + fileName);
Files.copy(response.getBody(), path, StandardCopyOption.REPLACE_EXISTING);
return new InputStreamResource(new FileInputStream(String.format("tmp/%s", fileName)));
};
InputStreamResource response = restTemplate.execute(
String.format("http://%s:%s/file/largeFileRestTemplate?fileName=%s", host, "9091", fileName),
HttpMethod.GET,
requestCallback,
responseExtractor
);
return ResponseEntity
.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, String.format("attachment; filename=%s", fileName))
.body(response);
}