I'm trying to use the Spring Reactive WebClient to upload a file to a spring controller. The controller is really simple and looks like this:
#PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<String> uploadFile(
#RequestParam("multipartFile") MultipartFile multipartFile,
#RequestParam Map<String, Object> entityRequest
) {
entityRequest.entrySet().forEach(System.out::println);
System.out.println(multipartFile);
return ResponseEntity.ok("OK");
}
When I use this controller with cURL everything works fine
curl -X POST http://localhost:8080/upload -H 'content-type: multipart/form-data;' -F fileName=test.txt -F randomKey=randomValue -F multipartFile=#document.pdf
The multipartFile goes to the correct parameter and the other parameters go in to the Map.
When I try to do the same from the WebClient I get stuck. My code looks like this:
WebClient client = WebClient.builder().baseUrl("http://localhost:8080").build();
MultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
map.set("multipartFile", new ByteArrayResource(Files.readAllBytes(Paths.get("/path/to/my/document.pdf"))));
map.set("fileName", "test.txt");
map.set("randomKey", "randomValue");
String result = client.post()
.uri("/upload")
.contentType(MediaType.MULTIPART_FORM_DATA)
.syncBody(map)
.exchange()
.flatMap(response -> response.bodyToMono(String.class))
.flux()
.blockFirst();
System.out.println("RESULT: " + result);
This results in an 400-error
{
"timestamp":1510228507230,
"status":400,
"error":"Bad Request",
"message":"Required request part 'multipartFile' is not present",
"path":"/upload"
}
Does anyone know how to solve this issue?
So i found a solution myself. Turns out that Spring really needs the Content-Disposition header to include a filename for a upload to be serialized to a MultipartFile in the Controller.
To do this i had to create a subclass of ByteArrayResource that supports setting the filename
public class MultiPartResource extends ByteArrayResource {
private String filename;
public MultiPartResource(byte[] byteArray) {
super(byteArray);
}
public MultiPartResource(byte[] byteArray, String filename) {
super(byteArray);
this.filename = filename;
}
#Nullable
#Override
public String getFilename() {
return filename;
}
public void setFilename(String filename) {
this.filename = filename;
}
}
Which can then be used in the client with this code
WebClient client = WebClient.builder().baseUrl("http://localhost:8080").build();
MultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
map.set("fileName", "test.txt");
map.set("randomKey", "randomValue");
ByteArrayResource resource = new MultiPartResource(Files.readAllBytes(Paths.get("/path/to/my/document.pdf")), "document.pdf");
String result = client.post()
.uri("/upload")
.contentType(MediaType.MULTIPART_FORM_DATA)
.body(BodyInserters.fromMultipartData(map))
.exchange()
.flatMap(response -> response.bodyToMono(String.class))
.flux()
.blockFirst();
System.out.println("RESULT: " + result);
You need include a filename to file part to upload success, in combination with asyncPart() to avoid buffering all file content, then you can write the code like this:
WebClient client = WebClient.builder().baseUrl("http://localhost:8080").build();
Mono<String> result = client.post()
.uri("/upload")
.contentType(MediaType.MULTIPART_FORM_DATA)
.body((outputMessage, context) ->
Mono.defer(() -> {
MultipartBodyBuilder bodyBuilder = new MultipartBodyBuilder();
Flux<DataBuffer> data = DataBufferUtils.read(
Paths.get("/tmp/file.csv"), outputMessage.bufferFactory(), 4096);
bodyBuilder.asyncPart("file", data, DataBuffer.class)
.filename("filename.csv");
return BodyInserters.fromMultipartData(bodyBuilder.build())
.insert(outputMessage, context);
}))
.exchange()
.flatMap(response -> response.bodyToMono(String.class));
System.out.println("RESULT: " + result.block());
Easier way to provide the Content-Disposition
MultipartBodyBuilder builder = new MultipartBodyBuilder();
String header = String.format("form-data; name=%s; filename=%s", "paramName", "fileName.pdf");
builder.part("paramName", new ByteArrayResource(<file in byte array>)).header("Content-Disposition", header);
// in the request use
webClient.post().body(BodyInserters.fromMultipartData(builder.build()))
Using a ByteArrayResource in this case is not efficient, as the whole file content will be loaded in memory.
Using a UrlResource with the "file:" prefix or a ClassPathResource should solve both issues.
UrlResource resource = new UrlResource("file:///path/to/my/document.pdf");
Related
I want to receive Multipart/form-data from a client (frontend for example). And then stream file content of form-data to another backend service.
For now i can read the whole file and pass it somewhere via byte[] (base64 string) like this:
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<ResponseType> upload(#RequestPart("document") FilePart document,
#RequestPart("stringParam") String stringParam) {
return service.upload(document, stringParam);
}
// Casually convert to single byte array...
private Mono<byte[]> convertFilePartToByteArray(FilePart filePart) {
return Mono.from(filePart
.content()
.map(dataBuffer -> {
byte[] bytes = new byte[dataBuffer.readableByteCount()];
dataBuffer.read(bytes);
DataBufferUtils.release(dataBuffer);
return bytes;
}));
}
There're a few problems with this approach:
I don't want to read the whole file into memory;
Array size in limited to Integer.MAX_VALUE;
Array encodes as base64 String, which takes extra memory;
Since i put the whole array in Mono - "spring.codec.max-in-memory-size" must be bigger than array size.
I've already tried sending file via asyncPart of WebClientBuilder:
MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.asyncPart("document", document.content(), DataBuffer.class);
But i'm getting an error:
java.lang.IllegalStateException: argument type mismatch
Method [public reactor.core.publisher.Mono<> upload(**org.springframework.http.codec.multipart.FilePart**,java.lang.String)] with argument values:
[0] [type=**org.springframework.http.codec.multipart.DefaultParts$DefaultFormFieldPart**]
UPD: full code, which generates error
// External controller for client.
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/v2")
public Mono<DocumentUploadResponse> uploadV2(#RequestPart("document") FilePart document,
#RequestPart("stringParam") String stringParam) {
MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.asyncPart("document", document.content(), DataBuffer.class);
builder.part("stringParam", stringParam);
WebClient webClient = webClientBuilder.build();
return webClient.post()
.uri("URL_TO_ANOTHER_SERVICE")
.contentType(MediaType.MULTIPART_FORM_DATA)
.body(BodyInserters.fromMultipartData(builder.build()))
.retrieve()
.bodyToMono(FileMetaDto.class)
.map(DocumentUploadResponse::new);
}
// Internal service controller.
#PostMapping(path = "/upload/v2", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<FileMetaDto> upload(#RequestPart("document") FilePart document,
#RequestPart("stringParam") String stringParam) {
return ...;
}
Looks like i was managed to stream file, working code below:
// External controller for client.
#PostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/v2")
public Mono<DocumentUploadResponse> uploadV2(#RequestPart("document") FilePart document,
#RequestPart("stringParam") String stringParam) {
MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.asyncPart("document", document.content(), DataBuffer.class).filename(document.filename());
builder.part("stringParam", stringParam);
WebClient webClient = webClientBuilder.build();
return webClient.post()
.uri("URL_TO_ANOTHER_SERVICE")
.contentType(MediaType.MULTIPART_FORM_DATA)
.body(BodyInserters.fromMultipartData(builder.build()))
.retrieve()
.bodyToMono(FileMetaDto.class)
.map(DocumentUploadResponse::new);
}
// Internal service controller.
#PostMapping(path = "/upload/v2", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Mono<FileMetaDto> upload(#RequestPart("document") FilePart document,
#RequestPart("stringParam") String stringParam) {
return ...;
}
In the original question code i've been missing:
builder.asyncPart("document", document.content(), DataBuffer.class).filename(document.filename());
I have two servers, where server A is sending files to server B. On server B i have an endpoint which recieves files of given category. This is signature of endpoint i need to send files to:
#PostMapping("/uploadMultipleFiles/{projectId}")
public List<UploadFileResponseDts> uploadMultipleFiles(#RequestParam("files") MultipartFile[] files, #RequestParam("categoryId") Long categoryId, #PathVariable("projectId") Long projectId) {
return uploadMulitpleFiles(files, categoryId, projectId);
}
Now im having trouble creating such request with RestTemplate exchange. On Server A i'm trying to send files one by one (endpoint on server B needs to accept multipart array since its used somewhere else as well). This is what i tried:
public Optional<String> uploadFile(File file, Long projectId) throws Exception {
String authToken = getAccessToken();
String projectFileUploadEndpoint = fileUploadEndpoint + SEPARATOR + projectId;
FileInputStream input = new FileInputStream(file);
byte[] bytes = IOUtils.toByteArray(input);
MultiValueMap<String, Object> parts =
new LinkedMultiValueMap<>();
parts.add("files", new ByteArrayResource(bytes));
parts.add("categoryId", 0L);
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
headers.set("Authorization", authToken);
HttpEntity<MultiValueMap<String, Object>> requestEntity =
new HttpEntity<>(parts, headers);
ResponseEntity<String> response =
restTemplate.exchange(projectFileUploadEndpoint ,
HttpMethod.POST, requestEntity, String.class);
return Optional.empty();
}
this almost works, the only issue is that on server B files from #RequestParam("files") MultipartFile[] files is always null. I assume that the issue is with the way im assembling parts in on my server A, however i cant find my mistake. Could you point it out?
in your controller all is fine, the problem is about your client code.
Basically you should use FileSystemResource object instead of ByteArrayResource.
The basic motivation is about file metadata. Using FileSystemResource spring is able to rebuild the required MultipartFile info, therefore the your code may be like below
#Test
public void contextLoads()
{
File file = new File("test.txt");
FileSystemResource fileSystemResource = new FileSystemResource(file);
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<>();
parts.add("categoryId", 0L);
parts.add("files", fileSystemResource);
String serviceUrl = "http://localhost:8080/uploadMultipleFiles/0";
RequestEntity<MultiValueMap<String, Object>> requestEntity = post(fromPath(serviceUrl).build().toUri())
.contentType(MediaType.MULTIPART_FORM_DATA)
.body(parts);
RestTemplate restTemplate = new RestTemplate();
restTemplate.setErrorHandler(new ResponseErrorHandler()
{
#Override
public boolean hasError(ClientHttpResponse response) throws IOException
{
return false;
}
#Override
public void handleError(ClientHttpResponse response) throws IOException
{
System.out.println(response);
}
});
ResponseEntity<String> response = restTemplate.exchange(serviceUrl, HttpMethod.POST, requestEntity, String.class);
assertThat(200, Is.is(response.getStatusCode()));
}
I hope that this test case can help you to solve the problem
How to upload multiple files using Webflux?
I send request with content type: multipart/form-data and body contains one part which value is a set of files.
To process single file I do it as follow:
Mono<MultiValueMap<String, Part> body = request.body(toMultipartData());
body.flatMap(map -> FilePart part = (FilePart) map.toSingleValueMap().get("file"));
But how to done it for multiple files?
PS. Is there another way to upload a set of files in webflux ?
I already found some solutions.
Let's suppose that we send an http POST request with an parameter files which contains our files.
Note responses are arbitrary
RestController with RequestPart
#PostMapping("/upload")
public Mono<String> process(#RequestPart("files") Flux<FilePart> filePartFlux) {
return filePartFlux.flatMap(it -> it.transferTo(Paths.get("/tmp/" + it.filename())))
.then(Mono.just("OK"));
}
RestController with ModelAttribute
#PostMapping("/upload-model")
public Mono<String> processModel(#ModelAttribute Model model) {
model.getFiles().forEach(it -> it.transferTo(Paths.get("/tmp/" + it.filename())));
return Mono.just("OK");
}
class Model {
private List<FilePart> files;
//getters and setters
}
Functional way with HandlerFunction
public Mono<ServerResponse> upload(ServerRequest request) {
Mono<String> then = request.multipartData().map(it -> it.get("files"))
.flatMapMany(Flux::fromIterable)
.cast(FilePart.class)
.flatMap(it -> it.transferTo(Paths.get("/tmp/" + it.filename())))
.then(Mono.just("OK"));
return ServerResponse.ok().body(then, String.class);
}
You can iterate hashmap with Flux and return Flux
Flux.fromIterable(hashMap.entrySet())
.map(o -> hashmap.get(o));
and it will be send as an array with filepart
the key is use toParts instead of toMultipartData, which is more simpler. Here is the example that works with RouterFunctions.
private Mono<ServerResponse> working2(final ServerRequest request) {
final Flux<Void> voidFlux = request.body(BodyExtractors.toParts())
.cast(FilePart.class)
.flatMap(filePart -> {
final String extension = FilenameUtils.getExtension(filePart.filename());
final String baseName = FilenameUtils.getBaseName(filePart.filename());
final String format = LocalDateTime.now().format(DateTimeFormatter.BASIC_ISO_DATE);
final Path path = Path.of("/tmp", String.format("%s-%s.%s", baseName, format, extension));
return filePart.transferTo(path);
});
return ServerResponse
.ok()
.contentType(APPLICATION_JSON_UTF8)
.body(voidFlux, Void.class);
}
希望对你有帮助
#PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public JSON fileUpload(#RequestPart FilePart file)throws Exception{
OSS ossClient = new OSSClientBuilder().build(APPConfig.ENDPOINT, APPConfig.ALI_ACCESSKEYID, APPConfig.ALI_ACCESSSECRET);
File f = null;
String url;
try {
String suffix = file.filename();
String fileName = "images/" + file.filename();
Path path = Files.createTempFile("tempimg", suffix.substring(1, suffix.length()));
file.transferTo(path);
f = path.toFile();
ossClient.putObject(APPConfig.BUCKETNAME, fileName, new FileInputStream(f));
Date expiration = new Date(System.currentTimeMillis() + 3600L * 1000 * 24 * 365 * 10);
url = ossClient.generatePresignedUrl(APPConfig.BUCKETNAME, fileName, expiration).toString();
}finally {
f.delete();
ossClient.shutdown();
}
return JSONUtils.successResposeData(url);
}
Following is the working code for uploading multiple files using WebFlux.
#RequestMapping(value = "upload", method = RequestMethod.POST)
Mono<Object> upload(#RequestBody Flux<Part> parts) {
return parts.log().collectList().map(mparts -> {
return mparts.stream().map(mmp -> {
if (mmp instanceof FilePart) {
FilePart fp = (FilePart) mmp;
fp.transferTo(new File("c:/hello/"+fp.filename()));
} else {
// process the other non file parts
}
return mmp instanceof FilePart ? mmp.name() + ":" + ((FilePart) mmp).filename() : mmp.name();
}).collect(Collectors.joining(",", "[", "]"));
});
};
I want to be able to download a file from a legacy service through a middle-layer Spring Web service. The problem currently is that I am returning the contents of the file and not the file itself.
I've used FileSystemResource before, but I do not want to do this, since I want Spring to only redirect and not create any files on the server itself.
Here is the method:
#Override
public byte[] downloadReport(String type, String code) throws Exception {
final String usernamePassword = jasperReportsServerUsername + ":" + jasperReportsServerPassword;
final String credentialsEncrypted = Base64.getEncoder().encodeToString((usernamePassword).getBytes("UTF-8"));
final HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.add("Accept", MediaType.APPLICATION_JSON_VALUE);
httpHeaders.add("Authorization", "Basic " + credentialsEncrypted);
httpHeaders.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
final HttpEntity httpEntity = new HttpEntity(httpHeaders);
final String fullUrl = downloadUrl + type + "?code=" + code;
return restTemplate.exchange(fullUrl, HttpMethod.GET, httpEntity, byte[].class, "1").getBody();
}
Turns out I was missing this annotation parameter in my *Controller class:
produces = MediaType.APPLICATION_OCTET_STREAM_VALUE
The whole method of the controller should look like this:
#RequestMapping(value = "/download/{type}/{id}", method = RequestMethod.GET, produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<?> downloadReport(#PathVariable String type, #PathVariable String id) throws Exception {
return new ResponseEntity<>(reportService.downloadReport(type, id), HttpStatus.OK);
}
I have a controller which returns file stream using ResponseEntity class.
But I'm not sure if the resource is closed after finished the method.
#RequestMapping(value = "/VMS-49001/playlist/{listName:.+}")
#ResponseBody
public ResponseEntity<?> playlist(HttpServletRequest request, HttpServletResponse response,
#PathVariable String listName) throws IOException {
String hlsPath = getHLSPath(request.getParameter("dt"), listName, OtuEnum.URLType.HLS);
Path filePath = Paths.get(hlsPath);
if (filePath.toFile().exists()) {
Path fileNamePath = filePath.getFileName();
String fileName = "";
if (fileNamePath != null) {
fileName = fileNamePath.toString();
}
HttpHeaders headers = new HttpHeaders();
headers.setContentDispositionFormData(fileName, fileName);
return ResponseEntity.ok().contentLength(filePath.toFile().length())
.contentType(MediaType.parseMediaType("application/vnd.apple.mpegurl")).headers(headers)
.body(new InputStreamResource(Files.newInputStream(filePath)));
} else {
String errorMsg = "404 file not found";
return ResponseEntity.status(HttpStatus.NOT_FOUND)
.contentType(MediaType.parseMediaType("text/html"))
.body(errorMsg);
}
}
if you see below code fragment, Files.newInputStream(filePath) implements Closeable, so it should be closed after use but I can't find the code closing it. :
return ResponseEntity.ok()
.contentLength(filePath.toFile().length())
.contentType(MediaType.parseMediaType("application/vnd.apple.mpegurl"))
.headers(headers)
.body(new InputStreamResource(Files.newInputStream(filePath)));
To response file stream, is it good to serve the file with this code? Or is there any better approach?
With Spring 4.1 your approach will work there is no issue in it.
Here below is another approach in case if you want to look :
#RequestMapping(value = "/VMS-49001/playlist/{listName:.+}")
public ResponseEntity<byte[]> testphoto() throws IOException {
InputStream in = servletContext.getResourceAsStream("/images/no_image.jpg");
final HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.parseMediaType("application/vnd.apple.mpegurl"));
headers.setContentDispositionFormData(fileName, fileName);
return new ResponseEntity<byte[]>(IOUtils.toByteArray(in), headers, HttpStatus.CREATED);
}