Can we provide both Json response as well as a csv file as attachment in a Rest Service?
I have written a code like below, but I am also sure its not going to work.
ResponseBuilder responseBuilder = null;
responseBuilder = Response.status(200).type(MediaType.APPLICATION_JSON)
.entity(parseOrganizations(getOrganizationsResponseMashery(limit, offset)));
responseBuilder.type(MediaType.TEXT_PLAIN).entity(file).header("Content-Disposition", "attachment; filename=Organizations.csv");
return responseBuilder.build();
The second setter for entity with file, basically over writes the json content that I had inserted earlier as entity. So please suggest.
Yes, that's right, an HTTP response should be of a single type. If you are telling you return JSON, then the client will be expecting a JSON object, not a file. And similarly, if you say you return a file, it will be expecting a file.
The client will be taking an action based on the return type stated in the response headers (Eg: Mapping a JSON object to a class instance, etc.), so it is important this is unambiguous.
In the case of springboot, it appears the last call to the type method overwrites an previous one.
Related
I have a Java REST endpoint which should return a multipart response with a json document and a pdf file. I chose "form-data" and not "mixed" because I want to assign a name to the parts in case I have multiple different json or file parts in the future.
The problem: In the output, the parts seem to be sorted depending on the key I specify, which makes sense, since formDataMap of MultipartFormDataOutput is a HashMap. If the json part is first, everything works as it should. BUT if the file part is sorted first, then the json part is missing in the output, as well as the form-data boundary at the end.
I feel I've tried every possible variation by switching out media types, response types, annotations, trying different pdf files, etc but nothing seems to help.
Does anyone have an idea how to explain this behavior? Any help is welcome, thanks!
#POST
#Path("test")
#Consumes(MediaType.APPLICATION_JSON)
#Produces("multipart/form-data")
public Response test(MyObject o)
{
// process input and get the byte arrays for the json document and pdf file
var output = new MultipartFormDataOutput();
output.addFormData("foo.json", new ByteArrayInputStream(myJsonByteArray), MediaType.APPLICATION_JSON_TYPE.withCharset("UTF-8"), "foo.json");
output.addFormData("document.pdf", new ByteArrayInputStream(myPdfByteArray), MediaType.valueOf("application/pdf"), "document.pdf");
return Response.ok(output).build();
}
I'm on Quarkus 1.13.7 with Resteasy 4.5.12
Not sure if this helps, but try wrapping MultipartFormDataOutput with GenericEntity before returning.
GenericEntity<MultipartFormDataOutput> outputGen = new GenericEntity<MultipartFormDataOutput>(output) {};
return Response.ok(outputGen).build();
I have come to the understanding that knowing this is indicative of a lack of knowledge of how REST-like APIs work, and if someone can provide me a reference where I can learn the background behind this question, I would appreciate it. In the meantime, though, I would also appreciate help answering this question!
I have a java application that posts files from the local filesystem to an API. My goal is to instead of having millions of files sitting on the volume with all of their file handles, I want to leave the files in a .tar.gz file, and then in memory pull them out of archive and POST them without writing them to disk. I know that I can write them to disk, POST them, and then delete them, but I view that option as a last resort.
So here's code that works to POST a file that exists in the file system, not in an archive
public CloseableHttpResponse submit (File file) throws IOException {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost post = new HttpPost(API_LOCATION + API_BASE);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addBinaryBody("files", file, ContentType.APPLICATION_OCTET_STREAM, null);
HttpEntity multipartEntity = builder.build();
post.setEntity(multipartEntity);
CloseableHttpResponse response = client.execute(post);
System.out.println("response: " + IOUtils.toString(response.getEntity().getContent(),"UTF-8"));
client.close();
return response;
}
I get back a JSON response from my particular API that looks like this
response: {"data":[<bunch of json>]}
I've put the same file into a .tar.gz archive and have used apache commons compress to unzip the file and pull out each file as a TarArchiveEntry, and I've tested that it works properly by writing the text file to disk and opening it manually outside of java - I am definitely getting the entry into memory correctly. I tried changing the entity attached to the POST to a ByteArrayEntity and converting the archive entry to a byte stream, but the API insists it will only accept a multipart entity. So looking at the API for MultipartEntityBuilder.addBinaryBody it appears I'm left with two options: I can either post a byte array or an InputStream. I've tried both and I can't get either to work - I'll post my example code for the byte array approach, but I can't figure out how to convert the tar archive to an InputStream - at least not without converting it to a byte array first, which seems sorta silly at that point.
public CloseableHttpResponse submit (byte[] xmlBytes) throws IOException {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost post = new HttpPost(API_LOCATION + API_BASE);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addBinaryBody("files", xmlBytes, ContentType.APPLICATION_OCTET_STREAM, null);
HttpEntity multipartEntity = builder.build();
post.setEntity(multipartEntity);
CloseableHttpResponse response = client.execute(post);
System.out.println("response: " + IOUtils.toString(response.getEntity().getContent(),"UTF-8"));
System.out.println(response.getStatusLine().getStatusCode());
client.close();
return response;
}
I believe the code is identical with the exception of the data type of the input parameter. Here is my empty response, which comes with a status code 207:
response: {"data":[]}
So here is my real question: Can any API that accept files also accept a file in the form of a byte stream or byte array? Can the API tell the difference, and what is really happening when I POST a file? Does the API have to be specifically configured to accept this file in the form of a byte stream or a byte array? A link to a reference along with a short explanation would be highly appreciated - I really need to learn this stuff and understand it well.
Is there some easy to correct mistake that I'm making? Am I using the wrong Content-Type or something? I'm not even sure what the meaning of the third argument to MultipartEntityBuilder.build is (the one I've left null).
Any help is appreciated, thank you very much!
It appears that an API that accepts a file doesn't care if it comes from a file object or a byte array. Per JB Nizet:
You're passing null as the file name. When passing a File as argument, the actual name of the File is used if you passed null as file name. That doesn't happen obviously if you pass a bute array. So specify a non-null file name as last argument. That can only be found out by reading the javadoc and the source code of MultipartEntityBuilder. It's open source: use that as an advantage.
In this specific case, adding a random string as the last argument of the build method fixes the problem and the API accepts the byte array as a file.
I am trying to achieve same thing as this: How to use query parameter represented as JSON with Spring RestTemplate?, sending JSON string as a URL parameter in restTemplate.exchange().
The accepted answer mentions that sending JSON object as request parameter is generally not a good idea since you will probably face problem with curly brackets inside JSON. That is exactly what is happening when I am trying to make a GET call to an API. Since this is an API from another system, I cannot ask them to change the format and will have to call the GET endpoint, passing JSON as parameter. How can I achieve this in restTemplate.exchange() call?
Note: The mentioned related question does not guide on how to overcome this problem and I do not have enough reputation to comment on it to ask the author of the answer.
Answering my own question. While it is a bad idea to pass JSON like this in a query/url parameter, there is a workaround as suggested here: https://jira.spring.io/browse/SPR-9220?focusedCommentId=76760&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-76760.
Replicating the code here in case this link goes dead:
String url = "http://localhost:8983/solr/select?wt=json&indent=true&fl=*&q=*:*&fq={!geofilt}&sfield=venue_location&pt=28.0674,-80.5595&d=25";
URI uri = UriComponentsBuilder.fromUriString(url).build().encode().toUri();
System.out.println(uri);
// http://localhost:8983/solr/select?wt=json&indent=true&fl=*&q=*:*&fq=%7B!geofilt%7D&sfield=venue_location&pt=28.0674,-80.5595&d=25
Basically, instead of passing url having JSON query/url parameters as a string, pass it as a URI. Then call exchange method as before, but with URI instead of String:
restTemplate.exchange(uri, HttpMethod.GET, requestEntity, String.class)
If this is 3rd party API and you cannot control or change JSON processing on backend side - there is no solution. Even if you will encode with URLEncoder - there is no guarantee that API backend would process such request correctly.
You can use URLEncoder class to encode the URL in exchange method, e.g.:
String url = "http://www.yoursite.com/api?param={\"some_key\":\"some_value\"}";
System.out.println(URLEncoder.encode(url, StandardCharsets.UTF_8.name()));
This will encode the characters (like braces and double quotes) and server then will decode it back to json.
I have a JSON that looks more or less like this:
{"id":"id","date":"date","csvdata":"csvdata".....}
where csvdata property is a big amount of data in JSON format too.
I was trying to POST this JSON using AJAX in Play! Framework 1.4.x so I sended just like that, but when I receive the data in the server side, the csvdata looks like [object Object] and stores it in my db.
My first thought to solve this was to send the csvdata json in string format to store it like a longtext, but when I try to do this, my request fails with the following error:
413 (Request Entity Too Large)
And Play's console show me this message:
Number of request parameters 3623 is higher than maximum of 1000, aborting. Can be configured using 'http.maxParams'
I also tried to add http.maxParams=5000 in application.conf but the only result is that Play's console says nothing and in my database this field is stored as null.
Can anyone help me, or maybe suggest another solution to my problem?
Thanks you so much in advance.
Is it possible that you sent "csvdata" as an array, not a string? Each element in the array would be a separate parameter. I have sent 100KB strings using AJAX and not run into the http.maxParams limit. You can check the contents of the request body using your browser's developer tools.
If your csvdata originates as a file on the client's machine, then the easiest way to send it is as a File. Your controller action would look like:
public static void upload(String id, Date date, File csv) {
...
}
When Play! binds a parameter to the File type, it writes the contents of the parameter to a temporary file which you can read in. (This avoids running out of memory if a large file is uploaded.) The File parameter type was designed for a normal form submit, but I have used it in AJAX when the browser supported some HTML5 features (File API and Form Data).
I am trying to send a JSON array in the request body of an HTTP POST endpoint which returns a CSV file.
var config = {
foo: ['bar']
}
$http.post(url, config).success(...).error(...);
The Java class representing this config input looks like this:
#JsonProperty List<String> foo;
The above $http.post works fine, except that I need to open a download prompt for the CSV file. I need to support IE8 so I am constructing a form similar to this gist.
var form = $("<form></form>").attr("action", url)
.attr("method", "post");
form.append($("<input/>").attr("type", "hidden")
.attr("name", "foo[]")
.attr("value", "bar"));
form.appendTo($("body"))
.submit()
.remove();
The above code does not seem to send a properly formatted JSON array for Jackson to deserialize. According to Chrome dev tools, the $http.post version sends foo: ["bar"] but the form version sends foo[]: "bar".
How can I POST the JSON array I want using HTML forms?
There is no equivalent as far as I know. You need to POST the HTML form using JavaScript by constructing (and later sending in the request) a JSON payload with the values in the HTML form (kind of weird but hey, you are sending data!).
Or just POST the form and build a JSON on the server side.