I'm receiving this response from an API and I'm trying to convert it to a file in Java, but when the file is created it says that the file is corrupted.
Here's the code that i'm using.
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(serverUrl)
.queryParam("fileID", fileID);
HttpEntity<?> entity = new HttpEntity<>(headers);
ResponseEntity<?> response = restTemplate.exchange(
builder.toUriString(),
HttpMethod.POST,
entity,
String.class);
System.out.println("Bytes de response body: " + response.getBody());
saveFile("C:\\Users\\xxx\\Documents\\" + fileName,body);
}
private static void saveFile(String path, String octetStream) throws IOException{
try(FileOutputStream fos = new FileOutputStream(path)){
fos.write(octetStream.getBytes());
}
Any input will be appreciated.
Thanks.
Answer:
I changed the response entity type from String to byte []
ResponseEntity<byte[]> response = restTemplate.exchange(
builder.toUriString(),
HttpMethod.POST,
entity,
byte[].class);
Related
I am new to resttemplate and trying to understand how to invoke below API using restTemplate
public ResponseEntity<String> upload(#ResquestPart("file") MultipartFIle file,
#RequestParam("path") String path){
//businness logic
}
I am trying to call above as below however it fails on "java.io.filenotfoundexception multipartfile resource [ABC.txt]cannot be resolved to absolute file path"
public void uploadFile() {
Path path = Paths.get("C:/ABC.txt");
byte[] content = null;
try{
content = Files.readAllBytes(path); // All file is read in content variable
} catch(final IOException e){
}
MultipartFile file = new MockMultipartFile("ABC.txt",content);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
headers.setContentType("Accept", Mediatype.APPLICATION_JSON_VALUE);
headers.setContentType("Content-type", Mediatype.APPLICATION_JSON_VALUE);
MultiValueMap<String, Object> obj = new LinkedMultiValueMap<String, Object>();
obj.add("file", file);
obj.add9("path", "/opt/apps");
HttpEntity<?> requestEntity = new HttpEntity<>(obj, headers);
String result = getRestTemplate().postForEntity("url", requestEntity, String.class);
}
I'm trying to consume a web service that accepts a CommonsMultipartFile in the request. So, I created an HTTP client using Spring's RestTemplate. Below is the method that takes in URI and a MultipartFile as parameters. I'm trying to pass this file to the web service in the form of ByteArrayResource.
public String upload(String uri, MultipartFile file) throws IOException {
logger.info("URI: " + uri);
ByteArrayResource fileAsResource = new ByteArrayResource(file.getBytes()) {
#Override
public String getFilename() {
return file.getOriginalFilename();
}
};
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("file", fileAsResource);
parts.add("fileName", file.getOriginalFilename());
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<MultiValueMap<String, Object>>(parts, httpHeaders);
ResponseEntity<String> responseEntity = rest.exchange(uri, HttpMethod.POST, requestEntity, String.class);
this.setStatus(responseEntity.getStatusCode());
return responseEntity.getBody();
}
This is how I'm creating a CommonsMultipartFile:
private MultipartFile getCommonsMultipartFile() throws FileNotFoundException, IOException {
File file = new File("C:\\Dummy_Test.txt");
DiskFileItemFactory factory = new DiskFileItemFactory();
FileItem fileItem = factory.createItem( "file", "multipart/form-data", false, "Dummy_Test.txt" );
IOUtils.copy(new FileInputStream(file), fileItem.getOutputStream());
MultipartFile commonsMultipartFile = new CommonsMultipartFile(fileItem);
return commonsMultipartFile;
}
But whenever I run this client to hit the web service I keep getting this error.
org.springframework.web.client.ResourceAccessException: I/O error: resource loaded from byte array cannot be resolved to absolute file path; nested exception is java.io.FileNotFoundException: resource loaded from byte array cannot be resolved to absolute file path
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:453)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:401)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:377)
at com.attikala.service.UploaderService.upload(UploaderService.java:118)
at com.attikala.service.UploaderService.main(UploaderService.java:55)
Caused by: java.io.FileNotFoundException: resource loaded from byte array cannot be resolved to absolute file path
at org.springframework.core.io.AbstractResource.getFile(AbstractResource.java:107)
at org.springframework.core.io.AbstractResource.contentLength(AbstractResource.java:116)
at org.springframework.http.converter.ResourceHttpMessageConverter.getContentLength(ResourceHttpMessageConverter.java:99)
at org.springframework.http.converter.ResourceHttpMessageConverter.write(ResourceHttpMessageConverter.java:81)
at org.springframework.http.converter.ResourceHttpMessageConverter.write(ResourceHttpMessageConverter.java:1)
at org.springframework.http.converter.FormHttpMessageConverter.writePart(FormHttpMessageConverter.java:288)
at org.springframework.http.converter.FormHttpMessageConverter.writeParts(FormHttpMessageConverter.java:252)
at org.springframework.http.converter.FormHttpMessageConverter.writeMultipart(FormHttpMessageConverter.java:242)
at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:194)
at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:1)
at org.springframework.web.client.RestTemplate$HttpEntityRequestCallback.doWithRequest(RestTemplate.java:588)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:436)
... 4 more
Can someone help me in figuring out what's happening here?
Note: If I use the below code to upload the file, it works perfectly fine.
public String upload(String uri) {
LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
FileSystemResource value = new FileSystemResource(new File("C:\\Dummy_Test.txt"));
map.add("file", value);
map.add("fileName", "Dummy_Test.txt");
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<>(map, headers);
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String> responseEntity = restTemplate.exchange(uri, HttpMethod.POST, requestEntity, String.class);
return responseEntity.getBody();
}
So, I'm thinking, do I need to provide absolute path always for the file I'm uploading? I know I'm missing something here. Don't know what.
Thanks.
Finally, I found what's happening.
Here I go -
when this statement
ResponseEntity<String> responseEntity = rest.exchange(uri, HttpMethod.POST, requestEntity, String.class);
gets executed, behind the scenes it's trying to extract the file of type java.io.File from the MultipartFile passed, and then get the file length. But MultipartFile is not of that type and as a result it was throwing an exception.
To fix that I had to also override contentLength() method when creating an instance of ByteArrayResource. Ta-da!
ByteArrayResource fileAsResource = new ByteArrayResource(file.getBytes()) {
#Override
public String getFilename() {
return file.getOriginalFilename();
}
#Override
public long contentLength() throws IOException {
return file.getSize();
}
};
Hope this helps if anyone runs into the same problem.
The accepted answer did not work. In my case I had to override the getFile(), see my solution below.
registry.addResourceHandler("/my-app/user-tracking-script.js")
.setCachePeriod(0)
.resourceChain(false)
.addResolver(new ResourceResolver() {
#Override
public Resource resolveResource(final HttpServletRequest request, final String requestPath, final List<? extends Resource> locations, final ResourceResolverChain chain) {
try {
final HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
final HttpEntity<String> entity = new HttpEntity<>(headers);
// RELEVANT PART BELOW
final ResponseEntity<byte[]> response = restTemplate.exchange(userTrackingScript, HttpMethod.GET, entity, byte[].class);
final ByteArrayResource fileAsResource = new ByteArrayResource(response.getBody()) {
#Override
public File getFile() throws IOException {
final File tempFile = File.createTempFile("user-tracking", ".js");
try (final FileOutputStream out = new FileOutputStream(tempFile)) {
IOUtils.copy(new ByteArrayInputStream(response.getBody()), out);
}
return tempFile;
}
};
return fileAsResource;
} catch (final Exception e) {
log.error("Could not download user-tracking-script.js for URL: {}", userTrackingScript, e);
}
return null;
}
#Override
public String resolveUrlPath(final String resourcePath, final List<? extends Resource> locations, final ResourceResolverChain chain) {
log.error("Unexpected call to resolveUrlPath by {}", resourcePath);
return null;
}
})
;
I am trying to get push notifications from a resource on Google Drive to my server. I have been looking at this example:
https://developers.google.com/drive/v3/web/push
And I have tried translating that to Java into something like this:
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
MultiValueMap<String, String> map = new LinkedMultiValueMap<String, String>();
map.add("id", "36d00d08-000d-4723-91bc-a1a6ec302e59");
map.add("type", "web_hook");
map.add("address", "https://mydomain.appspot.com/rest/drive");
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<MultiValueMap<String, String>>(map, headers);
ResponseEntity<String> response = restTemplate.postForEntity(uri, request, String.class);
I have previously been using Googles libs for Drive to access files. In those cases I didn't need to create the request in such a "manual" way. I have used the class GoogleAuthorizationCodeFlow with a token to authorize my requests. I'm not sure how I should do that with RestTemplate. I am guessing that I need to do something like:
headers.set("Authorization", X);
What should X be here? Is that even the right way to approach authorization?
Edit:
Here is my attempt by reading a secret. The result is HTTP 401:
#Override
public String startListening() throws IOException {
final String fileId = "omitted";
String uri = "https://www.googleapis.com/drive/v3/files/" + fileId + "/watch";
HttpHeaders headers = getHeaders(getSecret());
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<MultiValueMap<String, String>>(getProperties(), headers);
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String> response = restTemplate.postForEntity(uri, request, String.class);
return response.getStatusCode() + " " + response.getBody() + " " + response.getHeaders();
}
private static HttpHeaders getHeaders(String theString) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.set("Authorization", "Bearer " + theString);
return headers;
}
private static MultiValueMap<String, String> getProperties() {
MultiValueMap<String, String> map = new LinkedMultiValueMap<String, String>();
map.add("id", "some uid");
map.add("type", "web_hook");
map.add("address", "https://mydomain.appspot.com/rest/drive");
return map;
}
private static String getSecret() throws IOException {
InputStream in =
ConcreteDriveListenerFactory.class.getResourceAsStream("/drive_secret.json");
StringWriter writer = new StringWriter();
IOUtils.copy(in, writer, "UTF-8");
return writer.toString();
}
As #DalmTo has mentioned, X is for token. With regard to sample POST request for Drive API try this code snippet from this SO thread. It also uses a POST method.
public static void main(String argv[]) throws Exception {
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost post = new HttpPost(
"https://www.googleapis.com/drive/v2/files");
post.addHeader("Content-Type", "application/json");
post.addHeader("Authorization",
"Bearer XXXXXXXXXXXXXXXXXXXXXXXXX");
JsonObject jsonObject = new JsonObject();
jsonObject.addProperty("title", "Test folder");
jsonObject
.addProperty("mimeType", "application/vnd.google-apps.folder");
post.setEntity(new StringEntity(jsonObject.toString()));
httpClient.execute(post);
}
Is there a way to set the httpEntiy in the restTemplate.execute Method? I have to put the Authorization in the header, so thats why I can not exclude it. As a ResponseEntity I get a InputStreamResource.
This is working without HttpEntiy set:
File responseFile = restTemplate.execute(
uriComponents.toUri(),
HttpMethod.GET, null,
new ResponseExtractor<File>() {
#Override
public File extractData(ClientHttpResponse response) throws IOException {
File serverFile = fileProcessHelper.createFile(pathToFile);
BufferedOutputStream stream = new BufferedOutputStream(
new FileOutputStream(serverFile));
byte[] bytes = IOUtils.toByteArray(response.getBody());
stream.write(bytes);
stream.close();
return serverFile;
}
});
This is NOT working. Error is: java.io.IOException: stream is closed
ResponseEntity<InputStreamResource> responseEntity = restTemplate.exchange(
uriComponents.toUri(),
HttpMethod.GET, requestEntity,
InputStreamResource.class);
InputStreamResource stream = new InputStreamResource(responseEntity.getBody().getInputStream());
HttpHeaders respHeaders = new HttpHeaders();
respHeaders.setContentLength(stream.contentLength());
response.setHeader("Content-Disposition", "attachment; filename=" + stream.getFilename());
return ResponseEntity.ok().headers(respHeaders).body(stream);
Or is there a way to reopen the inputstreamresource?
Thanks in advance!
Ok. I found a solution:
in the RquestCallback you can set the headers:
RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(ClientHttpRequest request) throws IOException {
byte[] plainCredsBytes = plainCreds.getBytes();
byte[] base64CredsBytes = Base64.encodeBase64(plainCredsBytes);
String base64Creds = new String(base64CredsBytes);
request.getHeaders().set("Authorization", "Basic " + base64Creds);
}
};
Server Code :
#POST
#Path("reportDownload")
#Consumes(MediaType.APPLICATION_JSON)
public Response generateReport(QueryData queryData) {
File file = new File("report.xlsx") // large file
StreamingOutput stream = new FileStreamingOutput(file) ;
return Response.ok(stream, MediaType.APPLICATION_OCTET_STREAM)
.header("filename" , file.getName())
.build();
}
Client Code :
Using the following code I'm able to download files upto some limit. Getting out of memory heap error for large files.
final String uri = buildUri("/reportGenerate/reportDownload");
HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
factory.setReadTimeout(read_timeout);
factory.setConnectTimeout(connection_timeout);
RestTemplate restTemplate = new RestTemplate(factory);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
List<MediaType> mediaTypeList = new ArrayList<>();
mediaTypeList.add(MediaType.APPLICATION_OCTET_STREAM);
headers.setAccept(mediaTypeList);
HttpEntity entity = new HttpEntity(queryData, headers);
ResponseEntity<byte[]> data = restTemplate.exchange(uri, HttpMethod.POST, entity, byte[].class);
HttpHeaders responseHeader = data.getHeaders();
String fileName = (String) responseHeader.get("filename").get(0);
String downloadFolder = ApplicationConfig.REPORT_DOWNLOAD_FOLDER.getValue();
if (data.getStatusCode() == HttpStatus.OK) {
FileOutputStream fos = null;
File toFile = null;
try {
toFile = new File(downloadFolder + File.separator + fileName);
fos = new FileOutputStream(toFile);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
IOUtils.write(data.getBody(), bos);
bos.writeTo(fos);
} catch (Exception e) {
convertReportException(e);
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException ex) {
convertReportException(ex);
}
}
return toFile;
}
}
How to use stream for download larger files.
Here is how I do it with a ResponseExtractor. Based on hints from this Spring Jira issue.
RestTemplate restTemplate // = ...;
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<Void> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("some/path");
Files.copy(response.getBody(), path);
return null;
};
restTemplate.execute(URI.create("www.something.com"), HttpMethod.GET, requestCallback, responseExtractor);
update
Here is what RestTemplate does behind the scenes on postForObject and friends (inline comments from me):
#Override
public <T> T postForObject(String url, Object request, Class<T> responseType, Map<String, ?> uriVariables)
throws RestClientException {
// From RequestCallback's javadoc:
// Callback interface for code that operates on a ClientHttpRequest.
// Allows to manipulate the request headers, and write to the request body.
//
// Used internally by the RestTemplate, but also useful for application code.
RequestCallback requestCallback = httpEntityCallback(request, responseType);
// HttpMessageConverterExtractor checks the response type header and requested
// responseType class to select the proper message converter to handle the response.
// It also implements ResponseExtractor.
HttpMessageConverterExtractor<T> responseExtractor =
new HttpMessageConverterExtractor<T>(responseType, getMessageConverters(), logger);
return execute(url, HttpMethod.POST, requestCallback, responseExtractor, uriVariables);
}
/**
* Returns a request callback implementation that writes the given object to the
* request stream.
*/
protected <T> RequestCallback httpEntityCallback(Object requestBody, Type responseType) {
return new HttpEntityRequestCallback(requestBody, responseType);
}
Note: This is essentially a duplicate of my answer at https://stackoverflow.com/a/38664475/1030527 but I can't mark the questions as duplicate since neither this one or that one have upvoted answers.
At the client that you mentioned
don't store the file in memory to download a large via the RestTemplate, it can cause the Java heap exception.
it should be stored on disk.
Here is some code sample to download a large file via the RestTemplate
#GetMapping("largeFile")
public ResponseEntity<InputStreamResource> downloadLargeFile(
#RequestParam("fileName") String fileName
) throws IOException {
RestTemplate restTemplate = new RestTemplate();
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<InputStreamResource> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("tmp/" + fileName);
Files.copy(response.getBody(), path, StandardCopyOption.REPLACE_EXISTING);
return new InputStreamResource(new FileInputStream(String.format("tmp/%s", fileName)));
};
InputStreamResource response = restTemplate.execute(
String.format("http://%s:%s/file/largeFileRestTemplate?fileName=%s", host, "9091", fileName),
HttpMethod.GET,
requestCallback,
responseExtractor
);
return ResponseEntity
.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, String.format("attachment; filename=%s", fileName))
.body(response);
}