I am evaluating using Apache AVRO for my Jersey REST services. I am using Springboot with Jersey REST.
Currently I am accepting JSON as input which are converted to Java Pojos using the Jackson object mapper.
I have looked in different places but I cannot find any example that is using Apache AVRO with a Jersey end point.
I have found this Github repository (https://github.com/FasterXML/jackson-dataformats-binary/) which has Apache AVRO plugin.
I still cannot find any good example as how to integrate this. Has anyone used Apache AVRO with Jersey? If yes, is there any example I can use?
To start , two things need to happen:
You need to develop a custom ObjectMapper after the fashion of the Avro schema format
You need to supply that custom ObjectMapper to Jersey.
That should look something like this:
#Provider
public class AvroMapperProvider implements ContextResolver<ObjectMapper> {
final AvroMapper avroMapper = new AvroMapper();
#Override
public ObjectMapper getContext(Class<?> type) {
return avroMapper;
}
}
Configure your application to use Jackson as the message handler:
public class MyApplication extends ResourceConfig {
public MyApplication() {
super(JacksonFeature.class,AvroMapperProvider.class);
}
}
Alternatively, you can implement a custom MessageBodyReader and MessageBodyWriter that allows you to directly process the payloads on the way in and out:
public class AvroMessageReader implements MessageBodyReader<Person> {
AvroSchema schema;
final AvroMapper avroMapper = new AvroMapper();
public AvroMessageReader(){
schema = avroMapper.schemaFor(Person.class); //generates an Avro schema from the POJO class.
}
#Override
public boolean isReadable(Class<?> type, Type type1, Annotation[] antns, MediaType mt) {
return type == Person.class; //determines that this reader can handle the Person class.
}
#Override
public Person readFrom(Class<Person> type, Type type1, Annotation[] antns, MediaType mt, MultivaluedMap<String, String> mm, InputStream in) throws IOException, WebApplicationException {
return avroMapper.reader(schema).readValue(in);
}
}
Here, we generate an avro schema from a hypothetical Person class. The JAX-RS runtime will select this reader based on the response from isReadable.
You can then inject the MessageBodyWorkers component into your service implementation class:
#Path("app")
public static class BodyReaderTest{
#Context
private MessageBodyWorkers workers;
#POST
#Produces("avro/binary")
#Consumes("avro/binary")
public String processMessage() {
workers.getMessageBodyReader(Person.class, Person.class, new Annotation[]{}, MediaType.APPLICATION_JSON_TYPE);
}
}
To answer your last comment: Setting the mime type on your handler to the recommended avro/binary ought to do it.
There is a comprehensive demo (which I wrote) on how to use avro in a JAX-RS REST service at. The JAX-RS message body readers and writers for avro are implemented at and they do support avro binary, json, idiomatic json, csv where applicable. They do provide full support for schema evolution and projections (via the http accept header). There is a list of articles that explain in more detail the demonstrated concepts at. Also this demo project runs live on GKE at, you can browse at the openapi at. Avro is being used in the project for everything, for logs, for metrics, for profiling.
Related
I'm using Jackson 2.7.5 with Jersey 2.25.1. I'm trying to fix existing production code that is now failing with "UnrecognizedPropertyException: Unrecognized field" when it gets an unexpected field in the JSON input.
In researching this, I found several old posts (5+ years) suggesting various fixes that were very different from my current code. I didn't pay much attention to these, because they were for old versions of Jackson/Jersey. And more recent suggestions, including Jersey's own documentation (https://jersey.github.io/documentation/latest/media.html#json.jackson), look very similar to what I already have in place. In fact, to my eyes, it looks like my existing code is already following the current practice. However, Jersey seems to be ignoring my custom ObjectMapper setting of...
DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false
I'm pretty sure that setting should fix the error, but Jersey seems to be using a default ObjectMapper instead of my custom settings.
First, here is the dependency information, which I believe matches what is shown in the Jersey documentation (https://jersey.github.io/documentation/latest/media.html#json.jackson).
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
<version>${jersey.version}</version>
</dependency>
Here is the call that is returning the error:
// this will throw an exception if it can't convert the string to the class
PropSearchResponse propResponse = null;
try {
propResponse = getResponse.readEntity(PropSearchResponse.class);
} catch(final ProcessingException e) {
throw new ProcessResultException(Code.FAILED, "failed to map from prop response", e);
}
Here is the original code for my custom ObjectMapper:
#Provider()
#Produces(value = MediaType.APPLICATION_JSON)
public class OutMapperProvider implements ContextResolver<ObjectMapper> {
private final ObjectMapper mapper;
public OutMapperProvider() {
mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
mapper.configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, true);
mapper.setSerializationInclusion(Include.NON_NULL);
mapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssZ"));
}
#Override()
public ObjectMapper getContext(final Class<?> type) {
return mapper;
}
}
Here is the example from the Jersey documentation:
#Provider
public class MyObjectMapperProvider implements ContextResolver<ObjectMapper> {
final ObjectMapper defaultObjectMapper;
public MyObjectMapperProvider() {
defaultObjectMapper = createDefaultMapper();
}
#Override
public ObjectMapper getContext(Class<?> type) {
return defaultObjectMapper;
}
private static ObjectMapper createDefaultMapper() {
final ObjectMapper result = new ObjectMapper();
result.configure(Feature.INDENT_OUTPUT, true);
return result;
}
}
I have tried the Jersey example (changing the names to match mine, of course) as well as several other examples I found online. The Jersey example does the customization after the #Override. Most other examples do it before the #Override, but they all seem substantially similar to each other and to my existing code. But it doesn't seem to make any difference. No matter what I have tried, the custom configuration is ignored and Jersey calls a default ObjectMapper, which fails on unexpected JSON fields.
Disclaimer: This is my first experience with both Jersey and Jackson. I don't have a good understanding of the underlying mechanism yet. I'm just trying to follow the patterns of the examples.
Update: I believe the code above is basically correct. But Paul's comment below says that I need to register the custom ObjectMapper. I have tried reproducing several examples I have found on the web (Example 4.2 at https://docs.huihoo.com/jersey/2.13/deployment.html#environmenmt.appmodel, for example), but without success. For my current attempt, I have tried adding a new MyApplication class to an existing config package (com.dmx.repl.config) using Jersey's ResourceConfig. The code is below. But still, it is not working.
Edit: Ignore this code, it didn't work. See solution below.
package com.dmx.repl.config;
import org.glassfish.jersey.server.ResourceConfig;
import com.dmx.repl.commons.OutMapperProvider;
/**
*
* #author Greg
* #version 1.0
*/
// Attempt to register custom ObjectMapper
public class MyApplication extends ResourceConfig {
public MyApplication() {
// I've tried both of these.
//register(OutMapperProvider.class);
packages("com.dmx.repl.commons");
}
}
It's working now. Jersey is now recognizing the custom ObjectMapper, which is configured to ignore unknown JSON fields with "FAIL_ON_UNKNOWN_PROPERTIES, false".
The ObjectMapper code above is correct. The problem (as suggested by Paul in the comments) was that the client had not registered the custom ObjectMapper. This was fixed very simply, by adding the following line to the client setup method, following client setup with ClientBuilder.
this.client.register(OutMapperProvider.class);
I have an API with the following return type:
class Example {
private Complex1 complex1;
private Complex2 complex2;
}
Complex1 and Complex2:
class Complex1 {
private String test1;
private String test2;
}
class Complex2 {
private String test3;
private String test4;
}
Now I would like to make use of standard serialization for Complex1 but add custom serialization for Complex2.
I tried to add a Provider for Complex2. But JAX-RS seems to be not aware of it since it is not the actual return type. If I add a Provider for Example JAX-RS makes use of my custom Provider. But having a custom Provider for Example has the drawback that I have to add logic for Complex1 too although standard serialization would be OK for Complex1.
In this example it would be OK, to do serialization for Complex1 too, but in my scenario Complex1 is huge and would like to avoid implementing serialization logic for Complex1.
Do you have any advice?
So it won't work like that. A single JSON provider is used for the entire serialization. It already knows how to serialize the entire object. There is just no way for Jackson to know that a different provider is needed mid-serialization (especially because Jackson doesn't even have any knowledge of providers, that is a JAX-RS concept).
At the Jackson level though, we can tell Jackson how to serialize with the use JsonSerializers. You can have a look at this article for writing custom serializers. Once you have the serilizer, then you can annotate Complex2 class with the custom serializer
#JsonSerialize(using = Complex2Serializer.class)
public class Complex2 {}
How can I make my object mapper work in situation when there is another object mapper defined in jar from dependencies ?
I'm trying to use Swagger with Jersey 2 which is being run under Jetty. The problem is that as soon as I add Swagger JAX-RX jar into classpath my object mapper is not discovered therefore I lose custom serialization of my objects.
Here is how my object mapper defined
#Provider
public class ObjectMapperProvider implements ContextResolver<ObjectMapper> {
}
I've posted issue to Swagger's maintainers where you could read details.
After hours of debugging in internals of Jersey I found that Swagger's own object mapper com.wordnik.swagger.jaxrs.json.JacksonJsonProvider calls super.setMapper(commonMapper) that sets non-null value to ProviderBase._mapperConfig._mapper. Later when http request handler attempts to serialize instance of my class call ends up in ProviderBase.locateMapper which has following body
public MAPPER locateMapper(Class<?> type, MediaType mediaType)
{
// First: were we configured with a specific instance?
MAPPER m = _mapperConfig.getConfiguredMapper();
if (m == null) {
// If not, maybe we can get one configured via context?
m = _locateMapperViaProvider(type, mediaType);
if (m == null) {
// If not, let's get the fallback default instance
m = _mapperConfig.getDefaultMapper();
}
}
return m;
}
in correct code-flow _mapperConfig.getConfiguredMapper() returns null which subsequently causes invocation of _locateMapperViaProvider which finds my custom mapper. With Swagger it defaults to com.fasterxml.jackson.jaxrs.json.JsonMapperConfigurator and my custom json serializers never get invoked.
I created small project which reproduces this problem here.
How would you guys suggest to fix this ? I could probably specify deserializer on each property of type TTLocalDate but it'll pollute the code :(
As noted by fehguy in the issue report, using the latest Swagger release and using the SwaggerSerializers should fix this issue. Where previously the Swagger JacksonJsonProvider would be used for all serializions, the SwaggerSerializers is only used for the Swagger model object
public class SwaggerSerializers implements MessageBodyWriter<Swagger> {
#Override
public boolean isWriteable(Class type, Type genericType, Annotation[] annotations,
MediaType mediaType) {
return Swagger.class.isAssignableFrom(type);
}
Consider the following interface/object hierarchy in a spring project:
public interface MyInterface {
//method defenitions
}
#Component
#Scope(SCOPE_PROTOTYPE)
public class MyClass implements MyInterface {
//method implementations
}
I use MyClass in a controller method where it is read from the requests body:
#RequestMapping(method = POST, value = "/posturi", consumes = "application/json")
public void createEntity(#RequestBody MyClass myClass) {
//handle request
}
The jackson library is used to read json data and convert it to a java object.
I would like to change the type of the parameter in the controller method from MyClass to MyInterface. This does not seem to work since the interface can't be instantiated with the new operator. But it could be created like this:
MyInterface instance = applicationContext.getBean(MyInterface.class);
Is it possible to make spring/jackson instantiate the object this way? I would like to do this so that my controller does not need to be aware of what implementation is used.
It should be possible with Converters. See documentation http://static.springsource.org/spring/docs/3.2.x/spring-framework-reference/html/validation.html. Question is, how do you know which class you return by converter? Rather rethink your design to use POJOs in input.
I have solved this now and the concept is quite simple but the implementation can be a bit tricky. As I understand it, you can annotate any type with #RequestBody as long as you provide a HttpMessageConverter that can convert from a http request to your desired type.
So the solution is:
Implement a HttpMessageConverter
Configure spring so that your HttpMessageConverter is used.
The second part can be a bit tricky. This is because spring adds a bunch of default HttpMessageConverter that can handle common types such as strings, integers, dates and I want these to continue to function as usual. Another problem is that if jackson is on the path, spring also adds a MappingJackson2HttpMessageConverter for generic json handling such as converting to concrete objects, maps and so on. Spring will use the first HttpMessageConverter it finds that claims to be able to convert to your type. The MappingJackson2HttpMessageConverter claims to be able to do so for my objects, but it is not able to, so it fails and the request fails. This could be considered a bug...
The chain that I wanted was:
Springs default HttpMessageConverters.
My own HttpMessageConverter
The MappingJackson2HttpMessageConverter
I found two ways to acheive this. First, you can declare this explicitly through xml.
<mvc:annotation-driven>
<mvc:message-converters>
<!-- All converters in specific order here -->
</mvc:message-converters>
</mvc:annotation-driven>
The downside of this is that if the default HttpMessageConverter chain changes in later releases, it will not change for your configuration.
Another way to do it is to programatically insert your own HttpMessageConverter before the MappingJackson2HttpMessageConverter.
#Configuration
public class MyConfiguration {
#Autowired
private RequestMappingHandlerAdapter adapter;
#Autowired
private MyHttpMessageConverter myHttpMessageConverter;
#PostConstruct
private void modify() {
List<HttpMessageConverter<?>> messageConverters = adapter.getMessageConverters();
int insertLocation = messageConverters.size() - 1;
for (int i = 0; i < messageConverters.size(); i++) {
Object messageConverter = messageConverters.get(i);
if (messageConverter instanceof MappingJackson2HttpMessageConverter) {
insertLocation = i;
}
}
messageConverters.add(insertLocation, myHttpMessageConverter);
}
}
The second alternative will continue to use the "default configuration" even if it changes in later releases. I consider it a bit hacky and not at all elegant but the reason I think it is a valid soulution is that there seems to be flaws in the MappingJackson2HttpMessageConverter claiming to be able to convert to types it cannot convert to. And also that you cannot explicitly add a HttpMessageConverter to a specific position in the chain.
For now I am going with the second option but how you do is up to you...
Say I have a data class in a JAX-RS 1 environment (RestEasy 2 with the Jackson provider) like this:
class Foo {
int id;
String name;
Bar bar;
...
}
with Bar being:
class Bar {
int one;
String two;
}
Now I want to have Bar serialized in a special way (perhaps depending on the media type that was requested (or depending the phase of the moon), I would write a MessageBodyWriter<Bar>
#Provider
#Produces("application/json")
public class BarWriter implements MessageBodyWriter<Bar> {
...
}
which works very well if Bar is requested on its own like in
#GET #Path("bar")
public Bar getBar() { return new Bar(...); }
But when I request Foo as in
#GET #Path("foo")
public Foo getFoo() { return new Foo(...); }
the message body writer is ignored.
Now what I want is that this MessageBodyWriter is also used when I return Foo or a List<Bar>
I think the latter can be achieved by just writing a custom MessageBodyWriter for the List case, but for the former case I can't write a message body writer for all my application classes that contain a Bar field.
Any ideas on how to solve this? I was also trying to use a Jackson serializer on the Bar instance, but it looks like this is not even registered by RestEasy (and then, I think that way is too fragile anyway).
Unfortunately, this is not how message body writers work. The JAX-RS implementation will locate a writer, to be used in serialization, based on the type being returned from your resource method. So in your case, with a custom writer defined for Bar, with this resource method:
#GET #Path("bar")
public Bar getBar() { return new Bar(...); }
the JAX-RS provider will serialize Bar using your custom writer. However for this resource method:
#GET #Path("foo")
public Foo getFoo() { return new Foo(...); }
you do not have a custom writer defined, and serialization will be handled by the first matching (default) provider that can handle the combination of return class and content-type. A key thing to remember is that, unlike typical JSON and XML serialization libraries, JAX-RS entity providers are not recursive. Aka, for a given object A being returned in a resource method, the provider will attempt to locate a custom writer only for A, and not for any of the types included in A as variables.
Since you are using Jackson though, why not just define a custom serializer for your Bar class? That will handle pretty much every scenario you described:
public class BarSerializer extends JsonSerializer<Bar> {
#Override
public void serialize(final Bar value, final JsonGenerator jgen,
final SerializerProvider provider) throws IOException,
JsonProcessingException {
jgen.writeStartObject();
jgen.writeFieldName("myBar");
jgen.writeString(value.getTwo());
jgen.writeEndObject();
}
}
You tell Jackson to use this custom serializer thusly:
#JsonSerialize(using=BarSerializer.class)
class Bar {
int one;
String two;
}
Lastly, don't forget that if you anticipate getting JSON back in the same form as you serialized, that you will also need a custom JsonDeserializer.
To get it to work, you need the jackson-mapper and jackson-jaxrs jars in your classpath (and probably the jackson-core one as well).
The JAX-RS runtime will only look up one MessageBodyWriter for the object returned by the resource method (see sectioin "4.2.2 Message Body Writer" in the specification), then that single MessageBodyWriter has complete control over the serialization of the entire object graph to be returned to the client.
In order to implement the behavior you wanted, you would need a custom MessageBodyWriter per media type, that is willing to delegate the serialization of a part of the object graph to another writer whenever it encounters a specific type in the graph, and then resume its own logic. Obtaining the delegate writer for the specific type wouldn't be a big problem (inject a javax.ws.rs.ext.Providers and call getMessageBodyWriter()), but I don't think the existing xml/json/etc serializers are implemented with such kind of extensions in mind, so I guess you couldn't just relay on them. Reimplementing an xml marshaller just for this is not an attractive option either.
Refer the below post for writing custom message body writer for your Java object's serialization.
http://h2labz.blogspot.in/2014/12/marshalling-java-to-json-in-jax-rs.html