Develop Reference

Feign client body type serialization failure with Jackson encoder

I'm working on implementing a Spring service and client and would like to use OpenFeign for the client. The client will be deployed with legacy applications that do not want to incur a dependency on Spring, so I'm using OpenFeign directly instead of via Spring Cloud.
I've run into an issue with the Jackson encoder and the Body type. It seems that the Jackson encoder cannot serialize an interface implementation to an interface type for the client method. e.g. if my client method is createFoo(Foo interface) where Foo is an interface calling the method with createFoo((FooImpl)fooImpl) where FooImpl implements the Foo interface then I get an encoder exception.
I've created an MCCE Gradle project demonstrating the issue here
The client definition is this:
public interface FooClient {
#RequestLine("POST /submit")
#Headers("Content-Type: application/json")
Response createFoo(Foo foo);
#RequestLine("POST /submit")
#Headers("Content-Type: application/json")
Response createFooImpl(FooImpl foo);
interface Foo { int id(); }
record FooImpl(int id) implements Foo { }
}
And the failing test demonstrating the issue is this:
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
class FooClientTest {
#LocalServerPort int port;
#Test
public void clientTest() {
final FooClient lClient = Feign.builder()
.encoder(new JacksonEncoder(List.of(
// Possibly this would be necessary with the original encoder implementation.
// new FooModule()
)))
.target(FooClient.class, String.format("http://localhost:%s", port));
Response response = lClient.createFooImpl(new FooImpl(10));
assertThat(response.status()).isEqualTo(404);
response = lClient.createFoo(new FooImpl(10));
assertThat(response.status()).isEqualTo(404); // <<===== This fails with the exception below.
}
public static class FooModule extends SimpleModule {
{
addAbstractTypeMapping(Foo.class, FooImpl.class);
}
}
}
The exception is:
feign.codec.EncodeException: No serializer found for class
codes.asm.feign.mcce.client.FooClient$FooImpl
and no properties discovered to create
BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
This issue was introduced in this commit. It seems to have somehow removed Jackson's ability to map the encoder for the interface to the implementation by explicitly calling for the interface encoder.
JavaType javaType = mapper.getTypeFactory().constructType(bodyType);
template.body(mapper.writerFor(javaType).writeValueAsBytes(object), Util.UTF_8);
Based on some experiments I think the original code would work fine, potentially with some configuration of the encoder via a Module.
As demonstrated in the test, I can work around the issue by typing the client method with the interface implementation, but this is undesirable for a number of reasons in my context.

I've figured out a workaround, but it's quite ugly. Create a Module and add the following serializer. I expect this to be extremely brittle and will likely just abandon the interface and go with a concrete record definition as a DTO.
addSerializer(new JsonSerializer<Foo>() {
#Override
public Class<Foo> handledType() {
return Foo.class;
}
/**
* This is an ugly hack to work around this: https://github.com/OpenFeign/feign/issues/1608
* Alternative would be to just make Foo a concrete record
* instead of an interface. That may be better.
*/
#Override
public void serialize(Foo value, JsonGenerator gen,
SerializerProvider serializers) throws IOException {
gen.writeStartObject();
final Method[] methods = Foo.class.getMethods();
for (Method method : methods) {
try {
final Object result = method.invoke(value);
gen.writePOJOField(method.getName(), result);
} catch (IllegalAccessException | InvocationTargetException e) {
throw new IllegalArgumentException(String.format("Class %s has method %s which is not an accessible no argument getter", value.getClass(), method.getName()));
}
}
gen.writeEndObject();
}
});

Use JsonIgnore for an attribute in an other class

I meet an issue with a class contained in a library that I use.
This issue comes when I want deserialize it.
Indeed, this class has a method names "getCopy" which returns a new instance of himself which contains this same method and call it still a StackOverFlowException on the following cycle :
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:723)
public class Object {
...
ObjectAttribute objectAttribute;
...
public ObjectAttribute getObjectAttribute(){
return this.objectAttribute
}
...
}
public class ObjectAttribute{
...
public ObjectAttribute getCopy{
return copy(this) //return a new instance of himself
}
...
}
Is there a way to ignore the method getCopy() like #JsonIgnoreAttribute("objectProperty.copy")?

For this specific use case, when you have a class in a third party library that you are not able to modify, Jackson provides the Mix-in annotations.
The idea behind this concept is to provide a class that indicates how the serialization of another class should be accomplished.
For instance, consider the following mix-in class definition for your use case:
public abstract class ObjectAttributeMixIn{
// You need to provide definitions for every property you need
// to serialize, and the proper constructor if necessary
...
// Ignore the getCopy method
#JsonIgnore
public abstract ObjectAttribute getCopy();
...
}
You can use the full set of Jackson annotations in the mix-in definitions.
Then, associate the mix-in with the ObjectAttribute class. You can use the instance of ObjectMapper you are using for serialization for this purpose:
objectMapper.addMixInAnnotations(ObjectAttribute.class, ObjectAttributeMixIn.class);
Yon can also register a custom module instead; please, see the relevant documentation.

for ignore method getCopy, just enough rename this method , e.g copy
every method start with get then serialized ,e.g if method name is getSomething then serialized to something: (return value by method))
so if you change method name to copy or copyInstance or every name without start by get then method not serialized

You can override JsonSerializer and do specific logic for class
public class CustomSerializerForC extends JsonSerializer<C> {
#Override
public Class<C> handledType() {
return C.class;
}
#Override
public void serialize(C c, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException, JsonProcessingException {
String upperCase = c.getValue().toUpperCase();
jsonGenerator.writeString(upperCase);
}
}
And use Serializer in moudle used in ObjectMapper:
SimpleModule module = new SimpleModule("MyCustomModule", new Version(1, 0, 0, null));
module.addSerializer(new CustomSerializerForC());
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(module);

There are 2 ways I see how to figure out your issue:
Write custom deserializer for you specific class and register it in Jackson mapper.
Tune up global Jackson mapper to ignore class getters in auto-detection and use only fields.
Please try 2 way with following config:
ObjectMapper mapper = new ObjectMapper();
mapper.setVisibility(JsonMethod.ALL, Visibility.NONE);
mapper.setVisibility(JsonMethod.FIELD, Visibility.ANY);
If you decide to move forward with 1 way, please write here if you need help.

You can register serializer and choose the fields you would like
/**
* We can not change source code so we are adding serializer for a specific type.
*
*/
public static class JsonSpecificTypeSerializer extends JsonSerializer<SpecificType> {
#Override
public void serialize(SpecificType t, JsonGenerator jsonGen, SerializerProvider serializerProvider) throws IOException {
jsonGen.writeStartObject();
jsonGen.writeFieldName("field1");
jsonGen.writeNumber(t.getield1());
.......
jsonGen.writeEndObject();
}
}
/**
* Customize jackson.
*
* adding configuration to jackson without overriding spring boot default conf.
*/
#Bean
public Jackson2ObjectMapperBuilderCustomizer customizeJackson() {
return jacksonObjectMapperBuilder -> {
jacksonObjectMapperBuilder.serializerByType(SpecificType.class,
new JsonSpecificTypeSerializer());
};
}

Intercept Deserialization in Jackson

I want to hook into Jackson's deserialization to optionally deserialize a different JSON document than the one provided. That seems like a really weird use case so let me explain.
I am using the Amazon SQS Extended client to put messages that are too large for SQS on S3 instead and a message that looks like this through SQS
["com.amazon.sqs.javamessaging.MessageS3Pointer",{"s3BucketName":"my-bucket","s3Key":"f5a0fa29-7f9c-4852-8bbb-53697799efe2"}]
An elastic beanstalk worker is listening to the other end of that which means that those messages are POSTed to a Jersey endpoint my application maintains. Since those messages are POSTed instead of using a SQS receiveMessage call the extended client will not fetch the message from S3 itself.
I was thinking it would be pretty clever to make a custom JsonDeserializer that would look at the message to see if it was an S3 pointer, download that file, and deserialize it. Otherwise, just deserialize the provided message. However, that isn't working out quite as smoothly as I hoped.
Here is what I have so far:
public class SQSS3Deserializer<T> extends JsonDeserializer<T> {
private static final String s3PointerHeader = "com.amazon.sqs.javamessaging.MessageS3Pointer";
private Class<T> type;
private ObjectMapper mapper = new ObjectMapper();
public SQSS3Deserializer() {
super();
type = getParameterizedTypeArgument();
}
#Override
public T deserialize(JsonParser jp, DeserializationContext dc) throws IOException, JsonProcessingException {
if (jp.isExpectedStartArrayToken()) {
jp.nextToken();
if (s3PointerHeader.equals(jp.getValueAsString())) {
jp.nextToken();
S3Pointer p = jp.readValueAs(S3Pointer.class);
return mapper.readValue(S3Utils.getInputStream(p.s3BucketName, p.s3Key), type);
}
}
return jp.readValueAs(type);
}
#SuppressWarnings("unchecked")
protected Class<T> getParameterizedTypeArgument() {
return (Class<T>) ((ParameterizedType) getClass().getGenericSuperclass()).getActualTypeArguments()[0];
}
static private class S3Pointer {
public String s3BucketName;
public String s3Key;
}
}
For each POJO I want to deserialize I'll have to create an empty subclass with the correct generic specialization, for example:
public class POJOS3Deserializer extends SQSS3Deserializer<POJO> {}
I also will need to add the JsonDeserializer annotation to the class
#JsonDeserialize(using=POJOS3Deserializer.class)
public class POJO { ... }
However, doing it this way causes a stack overflow error because it will continually reenter my deserializer when it calls JsonParser.readValueAs() since readValueAs looks at the JsonDeserialize annotation.
So, I have two questions:
How do I change this to keep this fairly generic and still have Jackson do most of the heavy lifting of parsing while avoiding that recursive call?
Is there a way to remove the need to derive from SQSS3Deserializer for each POJO I want to deserialize this way?
Thanks

jax-ws web-service tier to the hibernate-based data provider

A data provider (java, hibernate) has an API for accessing instances of JPA-annotated classes. A web-service (jax-ws) is exposing the API to the network clients. A bit of a problem I'm thinking to solve is that a client of the data provider cannot be easily reconfigured to either use the provider directly or over the web-service. The reason is that for any persistent class there is a definition of this class in jax-ws client code and in data provider code, they are identical structurally but are different classes in Java. The obvious solution of putting generated classes to the same namespace as the original classes and setting up a class path in such a way that generated classes are always ignored doesn't seem ot be a clean one.
Has anyone solved this or knows a better way?

One way I've solved this in a similar problem is to work with interfaces and use reflection to build proxy objects that wrap the real underlying object. Something like:
interface IEntity
{
void setFoo(String foo);
String getFoo();
}
class WSEntity
{/* code generated by jax-ws */
}
class DataEntity
{ /* code generated by java, hibernate, .. */
}
class WSEntityInvocationHandler implements InvocationHandler
{
private final WSEntity entity;
public WSEntityInvocationHandler(WSEntity entity)
{
this.entity = entity;
}
public Object invoke(Object proxy,
Method method, Object[] args) throws Throwable
{
// this is a simplified version
Method m = entity.getClass().getMethod(method.getName(), params);
return m.invoke(entity, args);
}
}
static void example()
{
InvocationHandler handler = new WSEntityInvocationHandler(entity);
IEntity ie = (IEntity) Proxy
.newProxyInstance(IEntity.class.getClassLoader(),
new Class[]{IEntity.class},
handler);
}
Basically all your app would need to do, is decide which "invocation handler" to use, e.g.
InvocationHandler handler = new WSEntityInvocationHandler(entity);
or
InvocationHandler handler = new DataEntityInvocationHandler(entity);

How do I add a type to GWT's Serialization Policy whitelist?

GWT's serializer has limited java.io.Serializable support, but for security reasons there is a whitelist of types it supports. The documentation I've found, for example this FAQ entry says that any types you want to serialize "must be included in the serialization policy whitelist", and that the list is generated at compile time, but doesn't explain how the compiler decides what goes on the whitelist.
The generated list contains a number of types that are part of the standard library, such as java.lang.String and java.util.HashMap. I get an error when trying to serialize java.sql.Date, which implements the Serializable interface, but is not on the whitelist. How can I add this type to the list?

There's a workaround: define a new Dummy class with member fields of all the types that you want to be included in serialization. Then add a method to your RPC interface:
Dummy dummy(Dummy d);
The implementation is just this:
Dummy dummy(Dummy d) { return d; }
And the async interface will have this:
void dummy(Dummy d, AsyncCallback< Dummy> callback);
The GWT compiler will pick this up, and because the Dummy class references those types, it will include them in the white list.
Example Dummy class:
public class Dummy implements IsSerializable {
private java.sql.Date d;
}

Any specific types that you include in your service interface and any types that they reference will be automatically whitelisted, as long as they implement java.io.Serializable, eg:
public String getStringForDates(ArrayList<java.util.Date> dates);
Will result in ArrayList and Date both being included on the whitelist.
It gets trickier if you try and use java.lang.Object instead of specific types:
public Object getObjectForString(String str);
Because the compiler doesn't know what to whitelist. In that case if the objects are not referenced anywhere in your service interface, you have to mark them explicitly with the IsSerializable interface, otherwise it won't let you pass them through the RPC mechanism.

The whitelist is generated by the GWT compiler and contains all the entries that are designated by the IsSerializable marker interface.
To add a type to the list you just need to make sure that the class implements the IsSerializable interface.
Additionally for serialization to work correctly the class must have a default no arg constructor (constructor can be private if needed). Also if the class is an inner it must be marked as static.

IMHO the simpliest way to access whitelist programmatically is to create a class similar to this:
public class SerializableWhitelist implements IsSerializable {
String[] dummy1;
SomeOtherThingsIWishToSerialize dummy2;
}
Then include it in the .client package and reference from the RPC service (so it gets analyzed by the compiler).
I couldn't find a better way to enable tranfer of unparameterized maps, which is obviously what you sometimes need in order to create more generic services...

The whitelist is generated by the gwt compiler and contains all the entries that are designated by the IsSerializable marker interface.
To add a type to the list you just need to make sure that the class implements the IsSerializable interface.
-- Andrej
This is probably the easiest solution.
The only thing to remember with this is that all the classes that you want to serialize should have "public, no-argument" constructor, and (depending upon requirements) setter methods for the member fields.

to ensure the desired result delete all war/<app>/gwt/*.gwt.rpc

To anyone who will have the same question and doesn't find previous answers satisfactory...
I'm using GWT with GWTController, since I'm using Spring, which I modified as described in this message. The message explains how to modify GrailsRemoteServiceServlet, but GWTController calls RPC.decodeRequest() and RPC.encodeResponseForSuccess() in the same way.
This is the final version of GWTController I'm using:
/**
* Used to instantiate GWT server in Spring context.
*
* Original version from this tutorial.
*
* ...fixed to work as explained in this tutorial.
*
* ...and then fixed to use StandardSerializationPolicy as explained in
* this message to allow
* using Serializable instead of IsSerializable in model.
*/
public class GWTController extends RemoteServiceServlet implements Controller, ServletContextAware {
// Instance fields
private RemoteService remoteService;
private Class<? extends RemoteService> remoteServiceClass;
private ServletContext servletContext;
// Public methods
/**
* Call GWT's RemoteService doPost() method and return null.
*
* #param request
* The current HTTP request
* #param response
* The current HTTP response
* #return A ModelAndView to render, or null if handled directly
* #throws Exception
* In case of errors
*/
public ModelAndView handleRequest(HttpServletRequest request, HttpServletResponse response) throws Exception {
doPost(request, response);
return null; // response handled by GWT RPC over XmlHttpRequest
}
/**
* Process the RPC request encoded into the payload string and return a string that encodes either the method return
* or an exception thrown by it.
*
* #param payload
* The RPC payload
*/
public String processCall(String payload) throws SerializationException {
try {
RPCRequest rpcRequest = RPC.decodeRequest(payload, this.remoteServiceClass, this);
// delegate work to the spring injected service
return RPC.invokeAndEncodeResponse(this.remoteService, rpcRequest.getMethod(), rpcRequest.getParameters(), rpcRequest.getSerializationPolicy());
} catch (IncompatibleRemoteServiceException e) {
return RPC.encodeResponseForFailure(null, e);
}
}
/**
* Setter for Spring injection of the GWT RemoteService object.
*
* #param RemoteService
* The GWT RemoteService implementation that will be delegated to by the {#code GWTController}.
*/
public void setRemoteService(RemoteService remoteService) {
this.remoteService = remoteService;
this.remoteServiceClass = this.remoteService.getClass();
}
#Override
public ServletContext getServletContext() {
return servletContext;
}
public void setServletContext(ServletContext servletContext) {
this.servletContext = servletContext;
}
}

I found that just putting it in the client package or using it in a dummy service interface was not sufficient as it seemed the system optimized it away.
I found it easiest to create a class that derived from one of the types already used in the service interface and stick it in the client package. Nothing else needed.
public class GWTSerializableTypes extends SomeTypeInServiceInterface implements IsSerializable {
Long l;
Double d;
private GWTSerializableTypes() {}
}

I had this problem but ended up tracing the problem back to a line of code in my Serializable object:
Logger.getLogger(this.getClass().getCanonicalName()).log(Level.INFO, "Foo");
There were no other complaints before the exception gets caught in:
#Override
protected void serialize(Object instance, String typeSignature)
throws SerializationException {
assert (instance != null);
Class<?> clazz = getClassForSerialization(instance);
try {
serializationPolicy.validateSerialize(clazz);
} catch (SerializationException e) {
throw new SerializationException(e.getMessage() + ": instance = " + instance);
}
serializeImpl(instance, clazz);
}
And the business end of the stack trace is:
com.google.gwt.user.client.rpc.SerializationException: Type 'net.your.class' was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded. For security purposes, this type will not be serialized.: instance = net.your.class#9c7edce
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serialize(ServerSerializationStreamWriter.java:619)