Develop Reference

How to pass complex object with file data via REST

I need to download files from one source and pass this data to another application via rest. The file types are: .txt .csv and .zip for now. The file size could be up to 500Mb - 1 Gb.
What is the optimal way to do it. Should I convert java File objects to byte array at first? Will the Multipart content type be the most appropriate one for this purpose?
I stacked a bit because the class I am going to transfer can contain different file types.
There is not code needed from your side just to have a clue how to handle it in a better way! ;)
The class below to transfer is:
public class FileEventsRequest {
private File originalFile;
private int rowCount;
private String md5;
private String cobDate;
private File controlFile; }

I worked on a task almost identical to the one you describe fairly recently!
To download the file I used -
#PutMapping("/{originalFileName}")
public ResponseEntity<ImmutableDocument> send(#PathVariable String originalFileName, InputStream payload) {
LOG.info("Receiving: {}", originalFileName);
sendPayload(payload, originalFileName);
return ResponseEntity.ok().build();
}
I chose InputStream as it's generally a bad idea to handle massive files 1GB in memory with a byte array, you could potentially blow the stack!
As for sending that file on to the target -
#Autowired
private RestTemplate sendTemplate;
private ResponseEntity<Void> sendPayload(final InputStream payload, final String originalFileName) throws IOException {
// You can send any other bits of information you need on the headers too
HttpHeaders headers = new HttpHeaders();
headers.put("originalFileName", originalFileName);
headers.setContentType(asMediaType(MimeType.valueOf({desired mimetype})));
headers.setAccept(singletonList(APPLICATION_OCTET_STREAM));
HttpEntity<Resource> requestEntity = new HttpEntity<>(new InputStreamResource(payload), headers);
UriComponentsBuilder builder = fromUriString({someurl});
UriComponents uriComponents = builder.build().encode();
return sendTemplate.exchange(uriComponents.toUri(), HttpMethod.POST, requestEntity, Void.class);
}
}
I also used my own Configuration class for the ResTemplate with a message converter for sending the binary InputStream -
#Configuration
public class RestClientConfiguration {
#Bean
public RestTemplate sendTemplate(ClientHttpRequestFactory clientHttpRequestFactory) {
return new RestTemplateBuilder()
.requestFactory(() -> clientHttpRequestFactory)
.messageConverters(new ResourceHttpMessageConverter())
.build();
}
}

Reading resources from src/main/resources/.. in spring-MVC

I have a question that makes my head ache.
First, my project structure looks like below.
I made a controller, which returns image(*.png) file to the appropriate request.
The code of controller is written below.
#Controller
public class ImageController {
#GetMapping(value = "/ImageStore.do", produces = MediaType.IMAGE_PNG_VALUE)
public #ResponseBody byte[] getStoreImage(HttpServletRequest request) throws IOException {
String image_name = request.getParameter("image_name");
Resource resource = null;
try {
resource = new ClassPathResource("/images/stores/" + image_name);
if(resource == null) {
throw new NullPointerException();
}
} catch(NullPointerException e) {
resource = new ClassPathResource("/images/stores/noimage.png");
}
InputStream inputStream = resource.getInputStream();
return IOUtils.toByteArray(inputStream);
}
}
Q1. I added try-catch phrase to send noimage.png if the request parameter is wrong, or if the filename of request parameter image_name does not exist. But it doesn't seem to work, and it gives me log saying
class path resource [images/stores/noima.png] cannot be opened because it does not exist
(If you need to know the full stack trace, I will comment below.)
Q2. I have 2 image files, hello.png and noimage.png in the folder /resources/images/stores/. I can read noimage.png correctly, but if I make request localhost:8080/ImageStore.do?image_name=hello.png, then it makes an error, making the same log in Q1.

There's no reason to think that the constructor would result in a null value.
The exception you are getting is likely from the getInputStream method, which is documented to throw
FileNotFoundException - if the underlying resource doesn't exist
IOException - if the content stream could not be opened
A slight adjustment might help
#Controller
public class ImageController {
#GetMapping(value = "/ImageStore.do", produces = MediaType.IMAGE_PNG_VALUE)
public #ResponseBody byte[] getStoreImage(HttpServletRequest request) throws IOException {
InputStream is = null;
try {
String image_name = request.getParameter("image_name");
is = new ClassPathResource("/images/stores/" + image_name).getInputStream();
} catch(FileNotFoundException e) {
is = new ClassPathResource("/images/stores/noimage.png").getInputStream();
}
return IOUtils.toByteArray(is);
}
}
You should include the stack trace, and exception message, which might assist understanding your second query, but I would check that the file really does exist, with the exact name you're using.

java.lang.IllegalStateException: InputStream has already been read - do not use InputStreamResource if a stream needs to be read multiple times

I am trying to read a file from aws s3 bucket and set it as resource inside my spring batch reader class. When I test the application on aws lambda function I got below error. any suggestion experts?
Caused by: java.lang.IllegalStateException: InputStream has already been read - do not use InputStreamResource if a stream needs to be read multiple times
at org.springframework.core.io.InputStreamResource.getInputStream(InputStreamResource.java:97) ~[task/:na]
at org.springframework.batch.item.file.DefaultBufferedReaderFactory.create(DefaultBufferedReaderFactory.java:34) ~[task/:na]
at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:266) ~[task/:na]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:146) ~[task/:na]
Class to read from s3 bucket
#Service
public class S3BucketProcessing {
private static final AmazonS3 s3 = AmazonS3ClientBuilder.standard().build();
public InputStreamResource readFile() throws IOException{
String bucketName = "mybuckey";
String key = "File.txt";
S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));
return new InputStreamResource(object.getObjectContent());
}
Spring batch reader class
#Component
public class MyReader extends FlatFileItemReader<MyEntity> {
MyLineMapper mapper;
MyTokenizer tokenizer;
S3BucketProcessing s3BucketProcessing;
#Autowired
public MyReader(MyTokenizer tokenizer, MyLineMapper mapper, S3BucketProcessing s3BucketProcessing) throws Exception{
LOG.info("CardCustomerNotificationReader constructor");
this.mapper = mapper;
this.tokenizer = tokenizer;
this.s3BucketProcessing= s3BucketProcessing;
this.setResource(s3BucketProcessing.readFile());
mapper.setLineTokenizer(tokenizer);
this.setLineMapper(mapper);
}
}

The docs suggest using ByteArrayResource to cache the content in memory, rather than InputStreamResource.
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/core/io/InputStreamResource.html
Just change the returns part like this:
//As suggested by berzerk
byte[] content = IOUtils.toByteArray(object.getObjectContent());
//Then
return new ByteArrayResource( content );

Instead of returning InputStreamResource , you shud return content of the stream may be byte[ ].
byte[] content = IOUtils.toByteArray(object.getObjectContent());
return content ;

So I had to send in JSON/XML object as Output Stream. I was using the InputStreamResource and was getting the same error as OP.
Here's the solution that worked for me.
#Override
public Resource dataExportForFieldExtractorModel() {
ObjectMapper xmlMapper = new XmlMapper().enable(SerializationFeature.INDENT_OUTPUT);
byte[] data;
Resource resource = null;
try {
data = xmlMapper.writerWithDefaultPrettyPrinter().writeValueAsBytes(new DataExportResponse());
resource = new ByteArrayResource(data);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
return resource;
}
As the resource needs to be byte[], I changed resource = new InputStreamResource(new ByteArrayInputStream(data)); to
resource = new ByteArrayResource(data);.

#FormParameter data becomes null after reading and setting the same data in ContainerRequestContext entityStream

I have implemented filter and I have called getEntityStream of ContainerRequestContext and set the exact value back by using setEntitystream. If i use this filter then #FormParameter data becomes null and if i don't use filter then everything will be fine (as I am not calling getEntityStream) and i have to use filter to capture request data.
Note: I am getting form params from MultivaluedMap formParams but not from #FormParameter.
Environment :- Rest Easy API with Jboss Wildfly 8 server.
#Provider
#Priority(Priorities.LOGGING)
public class CustomLoggingFilter implements ContainerRequestFilter, ContainerResponseFilter{
final static Logger log = Logger.getLogger(CustomLoggingFilter.class);
#Context
private ResourceInfo resourceInfo;
#Override
public void filter(ContainerRequestContext requestContext)
throws IOException {
MDC.put("start-time", String.valueOf(System.currentTimeMillis()));
String entityParameter = readEntityStream(requestContext);
log.info("Entity Parameter :"+entityParameter);
}
private String readEntityStream(ContainerRequestContext requestContext){
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
final InputStream inputStream = requestContext.getEntityStream();
final StringBuilder builder = new StringBuilder();
int read=0;
final byte[] data = new byte[4096];
try {
while ((read = inputStream.read(data)) != -1) {
outStream.write(data, 0, read);
}
} catch (IOException e) {
e.printStackTrace();
}
byte[] requestEntity = outStream.toByteArray();
if (requestEntity.length == 0) {
builder.append("");
} else {
builder.append(new String(requestEntity));
}
requestContext.setEntityStream(new ByteArrayInputStream(requestEntity) );
return builder.toString();
}
return null;
}
}
class customResource
{
//// This code is not working
#POST
#Path("voiceCallBack")
#ApiOperation(value = "Voice call back from Twilio")
public void voiceCallback(#FormParam("param") String param)
{
log.info("param:" + param);
}
// This code is working
#POST
#Path("voiceCallBackMap")
#ApiOperation(value = "Voice call back from Twilio")
public void voiceCallbackMap(final MultivaluedMap<String, String> formParams)
{
String param = formParams.getFirst("param");
}
}
please suggest me solution & Thanks in Advance.

I found during run time that instance of the entity stream (from http request) is of type org.apache.catalina.connector.CoyoteInputStream (I am using jboss-as-7.1.1.Final). But we are setting entity stream with the instance of java.io.ByteArrayInputStream. So Resteasy is unable to bind individual formparmeters.
There are two solutions for this you can use any one of them :
Use this approach How to read JBoss Resteasy's servlet request twice while maintaing #FormParam binding?
Get form parameters like this:
#POST
#Path("voiceCallBackMap")
#ApiOperation(value = "Voice call back from Twilio")
public void voiceCallbackMap(final MultivaluedMap<String, String> formParams)
{
String param = formParams.getFirst("param");
}

Receiving customized response and file from REST web services

I'm trying to create some REST web services with Java in order to send data, do calculations on the server, and return the result. In a first stage I send and receive information as an excel file (in the future I prefer to use XML or JSON).
Well, after a lots of hours trying it, and reading lots of posts, it seems I'm very close to achieve it, but I don't know how to obtain the final response of the server.
I have a service like this:
#GET
#Path("/test")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getFile(#QueryParam("IDfile") String IDfile) {
if(IDfile.trim().length() == 0 || IDfile == null) {
return Response.status(Response.Status.BAD_REQUEST).entity("IDfile cannot be blank").build();
}
String uploadedFileLocation = "C:\\FilesWebservice\\" + IDfile;
Boolean sortida = false;
try {
prova prueba = new prova();
sortida = prueba.prova(uploadedFileLocation); //this creates an xls file as response
} catch (Exception ex) {
System.out.println("error" + ex.toString());
Logger.getLogger(ServiceResource.class.getName()).log(Level.SEVERE, null, ex);
}
if (sortida) {
File file = new File("C:\\FilesWebservice\\out\\prediction.xls"); // the File path you want to serve.
return Response.ok(file, MediaType.APPLICATION_OCTET_STREAM)
.header("Content-Disposition", "attachment; filename=\"" + file.getName() + "\"" ) //optional
.build();
} else
return Response.status(500).entity("It was unable to calculate (Ask God for the reason)").build();
}
It works OK, if I send a GET through the browser I receive the file in my downloads folder, but I need to consume the service with another application. Thus, I'm developing a client with Netbeans, and then, NB created automatic code according to my web service. In this case I have:
public <T> T getFile(Class<T> responseType, String IDfile) throws ClientErrorException {
WebTarget resource = webTarget;
if (IDfile != null) {
resource = resource.queryParam("IDfile", IDfile);
}
resource = resource.path("test");
Builder builder = resource.request(MediaType.APPLICATION_OCTET_STREAM_TYPE);
Invocation invocation = builder.buildGet();
return resource.request(MediaType.APPLICATION_OCTET_STREAM_TYPE).get(responseType);
}
Maybe I added some lines, I can't remember now. Anyway, the service returns a status code, a customised message and the file as attachment. I want to read at least the status code and obviously save the file, but I don't know how can I do it.
I tried to do:
MyJerseyClientAlgA client = new MyJerseyClientAlgA("192.168.1.30");
Object response = client.getFile(Response.class, "3cphkhfu.xls");
but it was unsuccessful to extract the information I need from 'response'.
Any help or ideas would be appreciated.
Many thanks in advance
EDIT:
Thanks #LutzHorn for your reply. I'm not sure if I understand well your proposal, I'll do some tests and if I find a solution I'll post under my question. Anyway, I generated again the automatic code for consuming the REST service, that is:
public <T> T getFile(Class<T> responseType, String IDfile) throws ClientErrorException {
WebTarget resource = webTarget;
if (IDfile != null) {
resource = resource.queryParam("IDfile", IDfile);
}
resource = resource.path("test");
return resource.get(responseType);
}
but I have an error in the last line, it indicates:
cannot find symbol
symbol: method get(Class)
so I changed this line for
return resource.request(MediaType.APPLICATION_OCTET_STREAM_TYPE).get(responseType);
but I'm not sure if this is right.

Well, after some hours searching and testing, this piece of code works. I don't know if it is the best solution, but it does exactly what I want: extract the status and save the file returned by the web service.
public void getFile(String IDfile) throws ClientErrorException {
WebTarget resource = webTarget;
if (IDfile != null) {
resource = resource.queryParam("IDfile", IDfile);
}
resource = resource.path("test");
Invocation inv = resource.request(MediaType.APPLICATION_OCTET_STREAM_TYPE).buildGet();
Response rp = inv.invoke();
InputStream attachment = null;
try {
if (rp.getStatus() == 200) {
attachment = rp.readEntity(InputStream.class); //This method can be invoked only once unless you buffer the response...
ReadableByteChannel rbc = Channels.newChannel(attachment); //website.openStream()
FileOutputStream fos = new FileOutputStream("C://FilesWebservice/solution.xls");
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
} else {
System.out.println(rp.getStatus());
}
} catch ( Exception ex) {
ex.printStackTrace();
} finally {
rp.close();
}
}