I have written a service that uses Jersey. In the last class that returns a Response object, I have:
Stream<Revision> revisionss = dataSourceHandler.getRevisions( .... );
StreamingOutput stream = os -> objectHistoryWorker.revisionsTransfer(revisions, os);
return Response.ok(stream).build();
This gets a Stream of Revision objects, converts the stream into a StreamingOutput, and then sends it out in the Response.
I'm trying to write an integration test to test this, and I want to see what contents are actually inside of the Response. In other words, I want to know information such as
How many Revision objects exist
Does a Revision object contain the correct information
The issue I'm having is that it is an OutboundJaxrsResponse, and the readEntity() method is not supported for it.
It has methods that will return whether it passed or not (i.e. status code 200), but I can't seem to figure out a way to actually read the contents of the Response.
Is there a way to get that information?
(The expected response content will be in Json format)
The class org.glassfish.jersey.message.internal.OutboundJaxrsResponse does support getEntity().
How to get the response contents as a Java String. From this, I'm sure you can work out how to read it as Json or XML ....
private String entity( Response response ) throws WebApplicationException, IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
StreamingOutput output = (StreamingOutput) response.getEntity();
output.write( baos );
return baos.toString( "UTF-8" );
}
Related
I am trying to upload a Flux object into azure blob storage, but I'm not sure how to send a Flux pojo using BlobAsyncClient. BlobAsyncClient has upload methods that take Flux or BinaryData but I have no luck trying to convert CombinedResponse to BYteBuffer or BinaryData. Does anyone have any suggestions or know how to upload a flux object to blob storage?
You will need an asynch blob container client:
#Bean("blobServiceClient")
BlobContainerAsyncClient blobServiceClient(ClientSecretCredential azureClientCredentials, String storageAccount, String containerName) {
BlobServiceClientBuilder blobServiceClientBuilder = new BlobServiceClientBuilder();
return blobServiceClientBuilder
.endpoint(format("https://%s.blob.core.windows.net/", storageAccount))
.credential(azureClientCredentials)
.buildAsyncClient()
.getBlobContainerAsyncClient(containerName);
}
And in your code you can use it to get a client, and save your Flux to it:
Flux<ByteBuffer> content = getContent();
blobServiceClient.getBlobAsyncClient(id)
.upload(content, new ParallelTransferOptions(), true);
I get that the getContent() step is the part you are struggling with. You can save either a BinaryData object or a Flux<ByteBuffer> stream.
To turn your object into a BinaryData object, use the static helper method:
BinaryData foo = BinaryData.fromObject(myObject);
BinaryData is meant for exactly what the name says: binary data. For example the content of an image file.
If you want to turn it into a ByteBuffer, keep in mind that you're trying to turn an object into a stream of data here. You will probably want to use some standardized way of doing that, so it can be reliably reversed, so rather than a stream of bytes that may break if you ever load the data in a different client, or even just a different version of the same, we usually save a json or xml representation of the object.
My go-to tool for this is Jackson:
byte[] myBytes = new ObjectMapper().writeValueAsBytes(myObject);
var myByteBuffer = ByteBuffer.wrap(myBytes);
And return it as a Flux:
Flux<ByteBuffer> myFlux = Flux.just(myByteBuffer);
By the way, Azure uses a JSON serializer under the hood in the BinaryData.fromObject() method. From the JavaDoc:
Creates an instance of BinaryData by serializing the Object using the default JsonSerializer.
Note: This method first looks for a JsonSerializerProvider
implementation on the classpath. If no implementation is found, a
default Jackson-based implementation will be used to serialize the object
Can we provide both Json response as well as a csv file as attachment in a Rest Service?
I have written a code like below, but I am also sure its not going to work.
ResponseBuilder responseBuilder = null;
responseBuilder = Response.status(200).type(MediaType.APPLICATION_JSON)
.entity(parseOrganizations(getOrganizationsResponseMashery(limit, offset)));
responseBuilder.type(MediaType.TEXT_PLAIN).entity(file).header("Content-Disposition", "attachment; filename=Organizations.csv");
return responseBuilder.build();
The second setter for entity with file, basically over writes the json content that I had inserted earlier as entity. So please suggest.
Yes, that's right, an HTTP response should be of a single type. If you are telling you return JSON, then the client will be expecting a JSON object, not a file. And similarly, if you say you return a file, it will be expecting a file.
The client will be taking an action based on the return type stated in the response headers (Eg: Mapping a JSON object to a class instance, etc.), so it is important this is unambiguous.
In the case of springboot, it appears the last call to the type method overwrites an previous one.
I have come to the understanding that knowing this is indicative of a lack of knowledge of how REST-like APIs work, and if someone can provide me a reference where I can learn the background behind this question, I would appreciate it. In the meantime, though, I would also appreciate help answering this question!
I have a java application that posts files from the local filesystem to an API. My goal is to instead of having millions of files sitting on the volume with all of their file handles, I want to leave the files in a .tar.gz file, and then in memory pull them out of archive and POST them without writing them to disk. I know that I can write them to disk, POST them, and then delete them, but I view that option as a last resort.
So here's code that works to POST a file that exists in the file system, not in an archive
public CloseableHttpResponse submit (File file) throws IOException {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost post = new HttpPost(API_LOCATION + API_BASE);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addBinaryBody("files", file, ContentType.APPLICATION_OCTET_STREAM, null);
HttpEntity multipartEntity = builder.build();
post.setEntity(multipartEntity);
CloseableHttpResponse response = client.execute(post);
System.out.println("response: " + IOUtils.toString(response.getEntity().getContent(),"UTF-8"));
client.close();
return response;
}
I get back a JSON response from my particular API that looks like this
response: {"data":[<bunch of json>]}
I've put the same file into a .tar.gz archive and have used apache commons compress to unzip the file and pull out each file as a TarArchiveEntry, and I've tested that it works properly by writing the text file to disk and opening it manually outside of java - I am definitely getting the entry into memory correctly. I tried changing the entity attached to the POST to a ByteArrayEntity and converting the archive entry to a byte stream, but the API insists it will only accept a multipart entity. So looking at the API for MultipartEntityBuilder.addBinaryBody it appears I'm left with two options: I can either post a byte array or an InputStream. I've tried both and I can't get either to work - I'll post my example code for the byte array approach, but I can't figure out how to convert the tar archive to an InputStream - at least not without converting it to a byte array first, which seems sorta silly at that point.
public CloseableHttpResponse submit (byte[] xmlBytes) throws IOException {
CloseableHttpClient client = HttpClients.createDefault();
HttpPost post = new HttpPost(API_LOCATION + API_BASE);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addBinaryBody("files", xmlBytes, ContentType.APPLICATION_OCTET_STREAM, null);
HttpEntity multipartEntity = builder.build();
post.setEntity(multipartEntity);
CloseableHttpResponse response = client.execute(post);
System.out.println("response: " + IOUtils.toString(response.getEntity().getContent(),"UTF-8"));
System.out.println(response.getStatusLine().getStatusCode());
client.close();
return response;
}
I believe the code is identical with the exception of the data type of the input parameter. Here is my empty response, which comes with a status code 207:
response: {"data":[]}
So here is my real question: Can any API that accept files also accept a file in the form of a byte stream or byte array? Can the API tell the difference, and what is really happening when I POST a file? Does the API have to be specifically configured to accept this file in the form of a byte stream or a byte array? A link to a reference along with a short explanation would be highly appreciated - I really need to learn this stuff and understand it well.
Is there some easy to correct mistake that I'm making? Am I using the wrong Content-Type or something? I'm not even sure what the meaning of the third argument to MultipartEntityBuilder.build is (the one I've left null).
Any help is appreciated, thank you very much!
It appears that an API that accepts a file doesn't care if it comes from a file object or a byte array. Per JB Nizet:
You're passing null as the file name. When passing a File as argument, the actual name of the File is used if you passed null as file name. That doesn't happen obviously if you pass a bute array. So specify a non-null file name as last argument. That can only be found out by reading the javadoc and the source code of MultipartEntityBuilder. It's open source: use that as an advantage.
In this specific case, adding a random string as the last argument of the build method fixes the problem and the API accepts the byte array as a file.
Seriously.
I've been scratching around trying to find the answer to this conundrum for a while.
The request size is too large if the String is encoded, and the company won't take Base64 anyway. They actually want the binary code, but in JSON. Can anyone shed any light on how they think that other people might do this? Currently I'm processing it like this;
String addressProof = null;
if (proofRequired)
{
Part filePart = request.getPart("proof_of_address");
addressFileName = getSubmittedFileName(filePart);
InputStream fileContent = filePart.getInputStream();
final byte[] bytes = IOUtils.toByteArray(fileContent);
addressProof = new String(bytes);
//byte[] bytes64 = Base64.encodeBase64(fileBytes);
//addressProof = new String(fileBytes);
fileContent.close();
}
Am I being dim, or is this whole request, just a little bit flawed.
Many thanks.
You can send it (or receive) as a hex string. See how-to-convert-a-byte-array-to-a-hex-string-in-java.
Example output would be (if enclosed by a JSON object):
{
"content": "C5192E4190E54F5985ED09C6CD0D4BCC"
}
or just plain hex string: "C5192E4190E54F5985ED09C6CD0D4BCC"
You don't have to write it (or read) all at once. You can open two streams (in and out) and then stream the data. From file to response output stream or from request input stream to file.
Sorry but I am not sure if You want to send the bytes or receive them.
I have a web service capable of returning PDF files in two ways:
RAW: The file is simply included in the response body. For example:
HTTP/1.1 200 OK
Content-Type: application/pdf
<file_contents>
JSON: The file is encoded (Base 64) and served as a JSON with the following structure:
HTTP/1.1 200 OK
Content-Type: application/json
{
"base64": <file_contents_base64>
}
I want to be able to consume both services on Android / Java by using the following architecture:
// Get response body input stream (OUT OF THE SCOPE OF THIS QUESTION)
InputStream bodyStream = getResponseBodyInputStream();
// Get PDF file contents input stream from body stream
InputStream fileStream = getPDFFileContentsInputStream(bodyStream);
// Write stream to a local file (OUT OF THE SCOPE OF THIS QUESTION)
saveToFile(fileStream);
For the first case (RAW response), the response body will the file itself. This means that the getPDFFileContentsInputStream(InputStream) method implementation is trivial:
#NonNull InputStream getPDFFileContentsInputStream(#NonNull InputStream bodyStream) {
// Return the input
return bodyStream;
}
The question is: how to implement the getPDFFileContentsInputStream(InputStream) method for the second case (JSON response)?
You can use any json parser (like Jackson or Gson), and then use Base64InputStream from apache-commons codec.
EDIT: You can obtain an input stream from string using ByteArrayInputStream, i.e.
InputStream stream = new ByteArrayInputStream(exampleString.getBytes(StandardCharsets.UTF_8));
as stated here.
EDIT 2: This will cause 2 pass over the data, and if the file is big, you might have memory problems. To solve it, you can use Jackson and parse the content yourself like this example instead of obtaining the whole object through reflection. you can wrap original input stream in another one, say ExtractingInputStream, and this will skip the data in the underlying input stream until the encoded part. Then you can wrap this ExtractingInputStream instance in a Base64InputStream. A simple algorithm to skip unnecessary parts would be like this: In the constructor of ExtractingInputStream, skip until you have read three quotation marks. In read method, return what underlying stream returns except return -1 if the underlying stream returns quotation mark, which corresponds to the end of base 64 encoded data.