I implemented the Message interface to include some headers for use with a HeaderValueRouter on server side.
Within one VM this works (tested using a filter between two endpoints).
But if I send the message through HttpOutboundGatway my fields get stripped (not included in the HttpRequest). And therefor the routing information is lost on server side.
Am I not supposed to manipulate headers?
public class TaskMessage implements Message<String> {
private MessageHeaders headers;
private String payload;
public TaskMessage(String taskId, String boxId, String payload) {
super();
this.taskId = taskId;
this.boxId = boxId;
this.payload = payload;
StringMessage sm = new StringMessage(payload);
Set<String> keySet = sm.getHeaders().keySet();
HashMap<String, Object> map = new HashMap<String, Object>();
for (String key : keySet) {
map.put(key, sm.getHeaders().get(key));
}
map.put("taskId", taskId);
map.put("boxId", boxId);
headers = new MessageHeaders(map);
}
#Override
public MessageHeaders getHeaders() {
return headers;
}
#Override
public String getPayload() {
return payload;
}
}
EDIT:
The version is 1.0.3
The part of my configuration is:
<si:inbound-channel-adapter ref="jdbcInputAdapter" method="fetchData" channel="msgChannel">
<si:poller max-messages-per-poll="1">
<si:interval-trigger interval="5000" />
</si:poller>
</si:inbound-channel-adapter>
<http:outbound-gateway id="httpChannelAdapter" auto-startup="true" request-timeout="1000" request-channel="msgChannel" reply-channel="replyChannel" default-url="http://localhost:8080/taskserver/gateway"/>
The version you are using does not support (custom) header serialization. The solution would be to craft a request that contains all the information needed and pass it along as the payload. The new REST based http support in version 2.0.x does support header mapping and also exposes extension points for converting messages (including headers).
As a side note, it is quite uncommon to have to implement a custom Message, so instead of doing that I'd create a message using MessageBuilder
MessageBuilder.withPayload("foo").setHeader("taskId", "someTaskId").build();
In general not all headers can be transferred with all protocols, so if you want to use a distributed system it is usually more flexible to pack all information you need to send into the payload.
Related
I have a java controller which have to send me some text data and different byte arrays. So I am building n multipart request and writing it to stream from HttpServletResponse.
Now my problem is how to parse the response at client side and extract the multiple parts.
SERVER CODE SNIPPET:-
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
// Prepare payload
builder.addBinaryBody("document1", file);
builder.addBinaryBody("document2", file2);
builder.addPart("stringData", new StringBody(jsonData, ContentType.TEXT_PLAIN));
// Set to request body
HttpEntity entity = builder.build();
postRequest.setEntity(entity);
CLIENT CODE SNIPPET:-
HttpPost httpPost = new HttpPost(finalUrl);
StringEntity entity = new StringEntity(json);
httpPost.setEntity(entity);
httpPost.setHeader("Content-type", APPLICATION_JSON_TYPE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
CloseableHttpResponse response = httpClient.execute(httpPost);
InputStream in = new BufferedInputStream(response.getEntity().getContent());
I checked CloseableHttpResponse and HttpEntity but none of them is providing method to parse multipart request.
EDIT 1:
This is my sample response I am receiving at client side stream:-
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="numeric"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
01010110
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="stringmessage"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding:8bit
testmessage
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="binarydata"; filename="file1"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
HI, THIS IS MY BINARY DATA
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="ending"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
ending
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe--
I have finally got a workaround for it.
I will be using javax mail MimeMultipart.
Below is a code snipped for the solution:-
ByteArrayDataSource datasource = new ByteArrayDataSource(in, "multipart/form-data");
MimeMultipart multipart = new MimeMultipart(datasource);
int count = multipart.getCount();
log.debug("count " + count);
for (int i = 0; i < count; i++) {
BodyPart bodyPart = multipart.getBodyPart(i);
if (bodyPart.isMimeType("text/plain")) {
log.info("text/plain " + bodyPart.getContentType());
processTextData(bodyPart.getContent());
} else if (bodyPart.isMimeType("application/octet-stream")) {
log.info("application/octet-stream " + bodyPart.getContentType());
processBinaryData(bodyPart.getInputStream()));
} else {
log.warn("default " + bodyPart.getContentType());
}
}
Please let me know if anybody else have any standard solution.
TL;DR
If you know what HttpMessageConverters are, then skip the "In General" part.
If your client is reactive, than you most likely don't have a big problem. Go to the part named "Reactive Client (Spring Webflux)" for details.
If your client is non-reactive i.e. you're using Spring MVC and RestTemplate then the last section is for you. In short, it is not possible out of the box and you need to write custom code.
In General
When we want to read multipart data, then we are at the serialization/marshalling layer of our application. This is basically the same layer as when we transform a JSON or an XML document to a POJO via Jackson for example. What I want to emphasize here is that the logic for parsing multipart data should not take place in a service but rather much earlier.
Our hook to transform multipart data comes as early, as to when an HTTP response enters our application in form of an HttpInputMessage. Out of the box, Spring provides a set of HttpMessageConverters, that are able to transform our HTTP response to an object with which we can work. For example, the MappingJackson2HttpMessageConverter is used to read and write all request/responses that have the MediaType "application/Json".
If the application is reactive, then Spring uses HttpMessageReader
and HttpMessageWriter instead of HttpMessageConverters. They save the same purpose.
The following two sections show how to read (download) a multipart response via the two different paradigms.
Reactive Client (Spring Webflux)
This would be the easiest use case and the only thing we need, is already available in Spring Webflux out of the box.
The class MultipartHttpMessageReader would be doing all the heavy lifting. In case it does not behave exactly how you need it to, you can easily extend it and overwrite the methods to your liking. Your custom Reader can then be registered as a bean like so:
#Configuration
public class MultipartMessageConverterConfiguration {
#Bean
public CodecCustomizer myCustomMultipartHttpMessageWriter() {
return configurer -> configurer.customCodecs()
.register(new MyCustomMultipartHttpMessageWriter());
}
}
Non-Reactive Client (Spring MVC/RestTemplate)
If you have a "classic" application that uses RestTemplate to communicate via HTTP, then you need to rely on the aforementioned HttpMessageConverters. Unfortunately, the MessageConverter that is responsible for reading multipart data, does not support reading/downloading data:
Implementation of HttpMessageConverter to read and write 'normal' HTML forms and also to write (but not read) multipart data (e.g. file uploads)
Source: FormHttpMessageConverter Documentation
So what we need to do, is write our own MessageConverter, which is able to download multipart data. An easy way to do that, would be to make use of the DefaultPartHttpMessageReader that is internally used by MultipartHttpMessageReader. We don't even need Webflux for that, as it is already shipped with spring-web.
First let us define 2 classes in which we save the several parts the we read:
public class MyCustomPart {
public MyCustomPart(byte[] content, String filename, MediaType contentType) {
//assign to corresponding member variables; here omitted
}
}
/**
* Basically a container for a list of objects of the class above.
*/
public class MyCustomMultiParts {
public MyCustomMultiParts(List<MyCustomPart> parts){
//assign to corresponding member variable; here omitted
}
}
Later on, you can always take each Part and convert it to whatever is appropriate. The MyCustomPart represents a single block of your multipart data response. The MyCustomMultiParts represent the whole multipart data.
Now we come to the meaty stuff:
public class CustomMultipartHttpMessageConverter implements HttpMessageConverter<MyCustomMultiParts> {
private final List<MediaType> supportedMediaTypes = new ArrayList<>();
private final DefaultPartHttpMessageReader defaultPartHttpMessageReader;
public CustomMultipartHttpMessageConverter() {
this.supportedMediaTypes.add(MediaType.MULTIPART_FORM_DATA);
this.defaultPartHttpMessageReader = new DefaultPartHttpMessageReader();
}
#Override
public boolean canRead(final Class<?> clazz, #Nullable final MediaType mediaType) {
if (!MyCustomMultiParts.class.isAssignableFrom(clazz)) {
return false;
}
if (mediaType == null) {
return true;
}
for (final MediaType supportedMediaType : getSupportedMediaTypes()) {
if (supportedMediaType.includes(mediaType) && mediaType.getParameter("boundary") != null) {
return true;
}
}
return false;
}
/**
* This wraps the input message into a "reactive" input message, that the reactive DefaultPartHttpMessageReader uses.
*/
private ReactiveHttpInputMessage wrapHttpInputMessage(final HttpInputMessage message) {
return new ReactiveHttpInputMessage() {
#Override
public HttpHeaders getHeaders() {
return message.getHeaders();
}
#SneakyThrows //Part of lombok. Just use a try catch block if you're not using it
#Override
public Flux<DataBuffer> getBody() {
final DefaultDataBuffer wrappedBody = new DefaultDataBufferFactory()
.wrap(message.getBody().readAllBytes());
return Flux.just(wrappedBody);
}
};
}
#Override
public MyCustomMultiParts read(#Nullable final Class<? extends MyCustomMultiParts> clazz,
final HttpInputMessage message) throws IOException, HttpMessageNotReadableException {
final ReactiveHttpInputMessage wrappedMessage = wrapHttpInputMessage(message);
final ResolvableType resolvableType = ResolvableType.forClass(byte[].class); //plays no role
List<Part> rawParts = defaultPartHttpMessageReader.read(resolvableType, wrappedMessage, Map.of())//
.buffer()//
.blockFirst();
//You can check here whether the result exists or just continue
final List<MyCustomPart> customParts = rawParts.stream()// Now we convert to our customPart
.map(part -> {
//Part consists of a DataBuffer, we make a byte[] so we can convert it to whatever we want later
final byte[] content = Optional.ofNullable(part.content().blockFirst())//
.map(DataBuffer::asByteBuffer)//
.map(ByteBuffer::array)//
.orElse(new byte[]{});
final HttpHeaders headers = part.headers();
final String filename = headers.getContentDisposition().getFilename();
final MediaType contentType = headers.getContentType();
return new MyCustomPart(content, filename, contentType);
}).collect(Collectors.toList());
return new MyCustomMultiParts(customParts);
}
#Override
public void write(final MyCustomMultiParts parts, final MediaType contentType,
final HttpOutputMessage outputMessage) {
// we're just interested in reading
throw new UnsupportedOperationException();
}
#Override
public boolean canWrite(final Class<?> clazz, final MediaType mediaType) {
// we're just interested in reading
return false;
}
#Override
public List<MediaType> getSupportedMediaTypes() {
return this.supportedMediaTypes;
}
}
From here on, you should know better what to do with your "CustomPart". Whether it is a JSON, a bitmap or a PDF. From the byte array you can convert it into anything.
Now if you want to test it, you only have to add your CustomConverter to a RestTemplate and then "await" the MyCustomMultiParts that we defined:
// This could also be inside your #Bean definition of RestTemplate of course
final RestTemplate restTemplate = new RestTemplate();
final List<HttpMessageConverter<?>> messageConverters = restTemplate.getMessageConverters();
messageConverters.add(new CustomMultipartHttpMessageConverter());
String url = "http://server.of.choice:8080/whatever-endpoint-that-sends-multiparts/";
final HttpHeaders headers = new HttpHeaders();
headers.setAccept(List.of(MediaType.MULTIPART_FORM_DATA));
final HttpEntity<Void> requestEntity = new HttpEntity<>(headers);
//here we await our MyCustomMultiParts
final MyCustomMultiParts entity = restTemplate.exchange(url, GET, requestEntity, MyCustomMultiParts.class);
Mime4j from Apache is one way to parse the responses from client-side. Its a common practice to use a tool like this.
You can refer this link - http://www.programcreek.com/java-api-examples/index.php?api=org.apache.james.mime4j.MimeException
You can download the jar from this link -
http://james.apache.org/download.cgi#Apache_Mime4J
This is related to an existing spring boot question raised by me(Request Body is not properly encoded and hidden when using spring form encoder in Feign Client).
According to this question, we can add either content type in headers or add during request mapping itself as consumes.
So what I did was added content type in headers in the client configuration class
public class EmailClientConfiguration {
#Bean
public RequestInterceptor requestInterceptor(Account<Account> account) {
return template -> {
template.header("Content-Type", "application/x-www-form-urlencoded");
};
}
#Bean
public OkHttpClient client() {
return new OkHttpClient();
}
#Bean
Logger.Level feignLoggerLevel() {
return Logger.Level.FULL;
}
#Bean
public Decoder feignDecoder() {
return new JacksonDecoder();
}
#Bean
public Encoder feignFormEncoder () {
return new SpringFormEncoder(new JacksonEncoder());
}
}
and I see in the headers the content type is correctly set as application/x-www-form-urlencoded when the request is sent. But the request body is still sent in json format and also not hidden.
Request Body:
Map<String, String> requestBody = new HashMap<>();
requestBody.put("username", "xyz");
requestBody.put("email", "xyz#gmail.com");
requestBody.put("key", "xxx");
Request Body received in server end:
{"{\n \"key\" : \"xxx\",\n \"email\" : \"xyz#gmail.com\",\n \"username\" : \"xyz\"\n}"
When I add consumes in my request mapping as application/x-www-form-urlencoded
#FeignClient(name = "email", url = "localhost:3000",
configuration = EmailClientConfiguration.class)
public interface EmailClient {
#PostMapping(value = "/email/send", consumes = "application/x-www-form-urlencoded")
ResponseDto sendEmail(#RequestBody Map<String, String> requestBody);
}
it works fine(request body is hidden in server end and also properly encoded). And when I removed the header in the configuration class and adding only consumes works fine without no issues but the vice versa has this problem.
I searched in internet for this and couldn't find any answer.
Feign encodes the request body and parameters before passing the request to any RequestInterceptor (and rightly so). If you do not declare consumes = "application/x-www-form-urlencoded", SprinFormEncoder doesn't know that you're trying to send form data, so it delegates serialization to the inner JacksonEncoder which only does JSON (see for yourself by printing template.body() before setting the header).
Handling such a well-supported header in the interceptor doesn't seem like a good idea, when you already have consumes. If you insist on doing so, you have to provide your own encoder which doesn't rely on the header value and always outputs form-urlencoded data.
I have a java controller which have to send me some text data and different byte arrays. So I am building n multipart request and writing it to stream from HttpServletResponse.
Now my problem is how to parse the response at client side and extract the multiple parts.
SERVER CODE SNIPPET:-
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
// Prepare payload
builder.addBinaryBody("document1", file);
builder.addBinaryBody("document2", file2);
builder.addPart("stringData", new StringBody(jsonData, ContentType.TEXT_PLAIN));
// Set to request body
HttpEntity entity = builder.build();
postRequest.setEntity(entity);
CLIENT CODE SNIPPET:-
HttpPost httpPost = new HttpPost(finalUrl);
StringEntity entity = new StringEntity(json);
httpPost.setEntity(entity);
httpPost.setHeader("Content-type", APPLICATION_JSON_TYPE);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
CloseableHttpResponse response = httpClient.execute(httpPost);
InputStream in = new BufferedInputStream(response.getEntity().getContent());
I checked CloseableHttpResponse and HttpEntity but none of them is providing method to parse multipart request.
EDIT 1:
This is my sample response I am receiving at client side stream:-
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="numeric"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
01010110
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="stringmessage"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding:8bit
testmessage
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="binarydata"; filename="file1"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
HI, THIS IS MY BINARY DATA
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe
Content-Disposition: form-data; name="ending"
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
ending
--bvRi5oZum37DUldtLgQGSbc5RRVZxKpjZMO4SYDe--
I have finally got a workaround for it.
I will be using javax mail MimeMultipart.
Below is a code snipped for the solution:-
ByteArrayDataSource datasource = new ByteArrayDataSource(in, "multipart/form-data");
MimeMultipart multipart = new MimeMultipart(datasource);
int count = multipart.getCount();
log.debug("count " + count);
for (int i = 0; i < count; i++) {
BodyPart bodyPart = multipart.getBodyPart(i);
if (bodyPart.isMimeType("text/plain")) {
log.info("text/plain " + bodyPart.getContentType());
processTextData(bodyPart.getContent());
} else if (bodyPart.isMimeType("application/octet-stream")) {
log.info("application/octet-stream " + bodyPart.getContentType());
processBinaryData(bodyPart.getInputStream()));
} else {
log.warn("default " + bodyPart.getContentType());
}
}
Please let me know if anybody else have any standard solution.
TL;DR
If you know what HttpMessageConverters are, then skip the "In General" part.
If your client is reactive, than you most likely don't have a big problem. Go to the part named "Reactive Client (Spring Webflux)" for details.
If your client is non-reactive i.e. you're using Spring MVC and RestTemplate then the last section is for you. In short, it is not possible out of the box and you need to write custom code.
In General
When we want to read multipart data, then we are at the serialization/marshalling layer of our application. This is basically the same layer as when we transform a JSON or an XML document to a POJO via Jackson for example. What I want to emphasize here is that the logic for parsing multipart data should not take place in a service but rather much earlier.
Our hook to transform multipart data comes as early, as to when an HTTP response enters our application in form of an HttpInputMessage. Out of the box, Spring provides a set of HttpMessageConverters, that are able to transform our HTTP response to an object with which we can work. For example, the MappingJackson2HttpMessageConverter is used to read and write all request/responses that have the MediaType "application/Json".
If the application is reactive, then Spring uses HttpMessageReader
and HttpMessageWriter instead of HttpMessageConverters. They save the same purpose.
The following two sections show how to read (download) a multipart response via the two different paradigms.
Reactive Client (Spring Webflux)
This would be the easiest use case and the only thing we need, is already available in Spring Webflux out of the box.
The class MultipartHttpMessageReader would be doing all the heavy lifting. In case it does not behave exactly how you need it to, you can easily extend it and overwrite the methods to your liking. Your custom Reader can then be registered as a bean like so:
#Configuration
public class MultipartMessageConverterConfiguration {
#Bean
public CodecCustomizer myCustomMultipartHttpMessageWriter() {
return configurer -> configurer.customCodecs()
.register(new MyCustomMultipartHttpMessageWriter());
}
}
Non-Reactive Client (Spring MVC/RestTemplate)
If you have a "classic" application that uses RestTemplate to communicate via HTTP, then you need to rely on the aforementioned HttpMessageConverters. Unfortunately, the MessageConverter that is responsible for reading multipart data, does not support reading/downloading data:
Implementation of HttpMessageConverter to read and write 'normal' HTML forms and also to write (but not read) multipart data (e.g. file uploads)
Source: FormHttpMessageConverter Documentation
So what we need to do, is write our own MessageConverter, which is able to download multipart data. An easy way to do that, would be to make use of the DefaultPartHttpMessageReader that is internally used by MultipartHttpMessageReader. We don't even need Webflux for that, as it is already shipped with spring-web.
First let us define 2 classes in which we save the several parts the we read:
public class MyCustomPart {
public MyCustomPart(byte[] content, String filename, MediaType contentType) {
//assign to corresponding member variables; here omitted
}
}
/**
* Basically a container for a list of objects of the class above.
*/
public class MyCustomMultiParts {
public MyCustomMultiParts(List<MyCustomPart> parts){
//assign to corresponding member variable; here omitted
}
}
Later on, you can always take each Part and convert it to whatever is appropriate. The MyCustomPart represents a single block of your multipart data response. The MyCustomMultiParts represent the whole multipart data.
Now we come to the meaty stuff:
public class CustomMultipartHttpMessageConverter implements HttpMessageConverter<MyCustomMultiParts> {
private final List<MediaType> supportedMediaTypes = new ArrayList<>();
private final DefaultPartHttpMessageReader defaultPartHttpMessageReader;
public CustomMultipartHttpMessageConverter() {
this.supportedMediaTypes.add(MediaType.MULTIPART_FORM_DATA);
this.defaultPartHttpMessageReader = new DefaultPartHttpMessageReader();
}
#Override
public boolean canRead(final Class<?> clazz, #Nullable final MediaType mediaType) {
if (!MyCustomMultiParts.class.isAssignableFrom(clazz)) {
return false;
}
if (mediaType == null) {
return true;
}
for (final MediaType supportedMediaType : getSupportedMediaTypes()) {
if (supportedMediaType.includes(mediaType) && mediaType.getParameter("boundary") != null) {
return true;
}
}
return false;
}
/**
* This wraps the input message into a "reactive" input message, that the reactive DefaultPartHttpMessageReader uses.
*/
private ReactiveHttpInputMessage wrapHttpInputMessage(final HttpInputMessage message) {
return new ReactiveHttpInputMessage() {
#Override
public HttpHeaders getHeaders() {
return message.getHeaders();
}
#SneakyThrows //Part of lombok. Just use a try catch block if you're not using it
#Override
public Flux<DataBuffer> getBody() {
final DefaultDataBuffer wrappedBody = new DefaultDataBufferFactory()
.wrap(message.getBody().readAllBytes());
return Flux.just(wrappedBody);
}
};
}
#Override
public MyCustomMultiParts read(#Nullable final Class<? extends MyCustomMultiParts> clazz,
final HttpInputMessage message) throws IOException, HttpMessageNotReadableException {
final ReactiveHttpInputMessage wrappedMessage = wrapHttpInputMessage(message);
final ResolvableType resolvableType = ResolvableType.forClass(byte[].class); //plays no role
List<Part> rawParts = defaultPartHttpMessageReader.read(resolvableType, wrappedMessage, Map.of())//
.buffer()//
.blockFirst();
//You can check here whether the result exists or just continue
final List<MyCustomPart> customParts = rawParts.stream()// Now we convert to our customPart
.map(part -> {
//Part consists of a DataBuffer, we make a byte[] so we can convert it to whatever we want later
final byte[] content = Optional.ofNullable(part.content().blockFirst())//
.map(DataBuffer::asByteBuffer)//
.map(ByteBuffer::array)//
.orElse(new byte[]{});
final HttpHeaders headers = part.headers();
final String filename = headers.getContentDisposition().getFilename();
final MediaType contentType = headers.getContentType();
return new MyCustomPart(content, filename, contentType);
}).collect(Collectors.toList());
return new MyCustomMultiParts(customParts);
}
#Override
public void write(final MyCustomMultiParts parts, final MediaType contentType,
final HttpOutputMessage outputMessage) {
// we're just interested in reading
throw new UnsupportedOperationException();
}
#Override
public boolean canWrite(final Class<?> clazz, final MediaType mediaType) {
// we're just interested in reading
return false;
}
#Override
public List<MediaType> getSupportedMediaTypes() {
return this.supportedMediaTypes;
}
}
From here on, you should know better what to do with your "CustomPart". Whether it is a JSON, a bitmap or a PDF. From the byte array you can convert it into anything.
Now if you want to test it, you only have to add your CustomConverter to a RestTemplate and then "await" the MyCustomMultiParts that we defined:
// This could also be inside your #Bean definition of RestTemplate of course
final RestTemplate restTemplate = new RestTemplate();
final List<HttpMessageConverter<?>> messageConverters = restTemplate.getMessageConverters();
messageConverters.add(new CustomMultipartHttpMessageConverter());
String url = "http://server.of.choice:8080/whatever-endpoint-that-sends-multiparts/";
final HttpHeaders headers = new HttpHeaders();
headers.setAccept(List.of(MediaType.MULTIPART_FORM_DATA));
final HttpEntity<Void> requestEntity = new HttpEntity<>(headers);
//here we await our MyCustomMultiParts
final MyCustomMultiParts entity = restTemplate.exchange(url, GET, requestEntity, MyCustomMultiParts.class);
Mime4j from Apache is one way to parse the responses from client-side. Its a common practice to use a tool like this.
You can refer this link - http://www.programcreek.com/java-api-examples/index.php?api=org.apache.james.mime4j.MimeException
You can download the jar from this link -
http://james.apache.org/download.cgi#Apache_Mime4J
I am adding some headers in my-transformer:
public Message<?> transform(final Message<?> message) {
List<Item> items = doStuff(message);
final MessageBuilder<?> messageBuilder = MessageBuilder
.withPayload(message.getPayload())
.copyHeadersIfAbsent(message.getHeaders());
for (final Item item : items) {
messageBuilder.setHeader(item.getHeaderName(), item.getValue());
}
return messageBuilder.build();
}
And I wrote an integration test to confirm that my header is present on the output channel:
public static class HeaderTest extends TransformerTest {
#Test
public void test() throws Exception {
channels.input().send(new GenericMessage<>(TransformerTest.EXAMPLE_PAYLOAD));
final Message<?> out = this.collector.forChannel(this.channels.output()).poll(10, TimeUnit.SECONDS);
assertThat(out, HeaderMatcher.hasHeader("header-test", notNullValue()));
}
}
But, when I created a stream like:
http --port=1234 | my-transformer | log --expression=toString()
and sent the same EXAMPLE_PAYLOAD I received the following message in the logs log: GenericMessage [payload=..., headers={kafka_offset=0, id=f0a0727c-9351-274c-58b3-edee9ccbf6ce, kafka_receivedPartitionId=0, contentType=text/plain;charset=UTF-8, kafka_receivedTopic=myTopic.my-transformer, timestamp=1485171448947}].
Why isn't my header-test in the message headers?
-- EDIT --
So if I understood correctly I am supposed to do something like:
public class MyTransformer implements Transformer {
private final EmbeddedHeadersMessageConverter converter = new EmbeddedHeadersMessageConverter();
#Override
public Message<?> transform(final Message<?> message) {
List<Item> items = doStuff(message);
final MessageBuilder<byte[]> messageBuilder = MessageBuilder
.withPayload(((String) message.getPayload()).getBytes())
.copyHeadersIfAbsent(message.getHeaders());
final int itemsSize = items.size();
final String[] headerNames = new String[itemsSize];
for (int i = 0; i < itemsSize; i++) {
final Item item = items.get(i);
messageBuilder.setHeader(item.getHeaderName(), item.getValue());
headerNames[i] = item.getHeaderName();
}
final Message<byte[]> msg = messageBuilder.build();
final byte[] rawMessageWithEmbeddedHeaders;
try {
rawMessageWithEmbeddedHeaders = converter.embedHeaders(new MessageValues(msg), headerNames);
} catch (final Exception e) {
throw new HeaderEmbeddingException(String.format("Cannot embed headers from '%s' into message: %s", items, msg), e);
}
return new GenericMessage<>(rawMessageWithEmbeddedHeaders);
}
}
with spring.cloud.stream.bindings.output.producer.headerMode=raw set in application.properties and then convert the message payload on the receiving side? Or can I somehow make the receiving side automatically convert the message payload?
You don't say whether you are using Spring XD or Spring Cloud DataFlow, but the solution is similar in each case.
Since kafka has no native support for headers, we have to embed them in the message payload. Since we don't want to transport unnecessary headers, you have to opt-in for the headers you want transported by setting the header names in servers.yml for Spring XD or application.yml (or .properties) for a Spring Cloud Stream app.
EDIT
Unfortunately, there is no support for patterns. One option would be to use the EmbeddedHeadersMessageConverter yourself, and set the kafka mode to raw (on your transformer's output destination). Raw mode means the binder won't embed headers.
That way, the next app (without mode raw) should be able to decode the headers as if they had been encoded by the binder in your transformer. Javadocs here.
You are limited to 255 headers.
I've been debugging this for three hours, I still cannot explain why my custom headers (registered via a client request filter) are not sent.
The client is configured as such (full source here):
private WebTarget webTarget(String host, String appId, String appKey) {
return newClient(clientConfiguration(appId, appKey))
.target(host + "/rest");
}
private Configuration clientConfiguration(String appId, String appKey) {
ClientConfig config = new ClientConfig();
config.register(requestFilter(appId, appKey));
return config;
}
private ClientRequestFilter requestFilter(String appId, String appKey) {
return new VidalRequestFilter(apiCredentials(appId, appKey));
}
The filter is as follows:
public class VidalRequestFilter implements ClientRequestFilter {
private final ApiCredentials credentials;
public VidalRequestFilter(ApiCredentials credentials) {
this.credentials = credentials;
}
#Override
public void filter(ClientRequestContext requestContext) throws IOException {
MultivaluedMap<String, Object> headers = requestContext.getHeaders();
headers.add(ACCEPT, APPLICATION_ATOM_XML_TYPE);
headers.add("app_id", credentials.getApplicationId());
headers.add("app_key", credentials.getApplicationKey());
}
}
And the call is like:
String response = webTarget
.path("api/packages")
.request()
.get()
.readEntity(String.class);
All I get is 403 forbidden, because the specific endpoint I am calling is protected (the auth is performed with the custom headers defined above).
The weirdest thing is that, while I'm debugging, I see that sun.net.www.MessageHeader is properly invoked during the request write (i.e. the instance is valued as such: sun.net.www.MessageHeader#14f9390f7 pairs: {GET /rest/api/packages HTTP/1.1: null}{Accept: application/atom+xml}{app_id: XXX}{app_key: YYY}{User-Agent: Jersey/2.22.1 (HttpUrlConnection 1.8.0_45)}{Host: ZZZ}{Connection: keep-alive}.
However, I have the confirmation that neither our API server, nor its reverse proxy received GET requests with the required auth headers (a first HEAD request seems to be OK, though).
I know for sure the credentials are good 'cause the equivalent curl command just works!
I tried the straightforward approach to set headers directly when defining the call without any success.
What am I missing?