Develop Reference

Dynamic Kafka Consumer with AOP

I have got few dynamic Kafka consumers (based upon the department id, etc..) and you can find the code below.
Basically, I wanted to log the time taken for each onMessage() method call and so I have created a #LogExecutionTime method level custom annotation and added it for onMessage() method .
But my logExecutionTime() of LogExecutionTimeAspect never gets called even though my onMessage() is being invoked whenever there is a message on to the topic and everything else works fine.
Could you please help on what am I missing LogExecutionTimeAspect class so that it starts working?
LogExecutionTime:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
public #interface LogExecutionTime {
}
LogExecutionTimeAspect class:
#Aspect
#Component
public class LogExecutionTimeAspect {
#Around("within(com.myproject..*) && #annotation(LogExecutionTime)")
public Object logExecutionTime(ProceedingJoinPoint joinPoint) throws Throwable {
long startTime = System.currentTimeMillis();
Object object = joinPoint.proceed();
long endTime = System.currentTimeMillis();
System.out.println(" Time taken by Listener ::"+(endTime-startTime)+"ms");
return object;
}
}
DepartmentsMessageConsumer class:
#Component
public class DepartmentsMessageConsumer implements MessageListener {
#Value(value = "${spring.kafka.bootstrap-servers}" )
private String bootstrapAddress;
#PostConstruct
public void init() {
Map<String, Object> consumerProperties = new HashMap<>();
consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
bootstrapAddress);
consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "DEPT_ID_HERE");
ContainerProperties containerProperties =
new ContainerProperties("com.myproj.depts.topic");
containerProperties.setMessageListener(this);
DefaultKafkaConsumerFactory<String, Greeting> consumerFactory =
new DefaultKafkaConsumerFactory<>(consumerProperties,
new StringDeserializer(),
new JsonDeserializer<>(Department.class));
ConcurrentMessageListenerContainer container =
new ConcurrentMessageListenerContainer<>(consumerFactory,
containerProperties);
container.start();
}
#Override
#LogExecutionTime
public void onMessage(Object message) {
ConsumerRecord record = (ConsumerRecord) message;
Department department = (Department)record.value();
System.out.println(" department :: "+department);
}
}
ApplicationLauncher class:
#SpringBootApplication
#EnableKafka
#EnableAspectJAutoProxy
#ComponentScan(basePackages = { "com.myproject" })
public class ApplicationLauncher extends SpringBootServletInitializer {
public static void main(String[] args) {
SpringApplication.run(ApplicationLauncher.class, args);
}
}
EDIT:
I have tried #EnableAspectJAutoProxy(exposeProxy=true), but did not work.

You should consider to turn on this option on the #EnableAspectJAutoProxy:
/**
* Indicate that the proxy should be exposed by the AOP framework as a {#code ThreadLocal}
* for retrieval via the {#link org.springframework.aop.framework.AopContext} class.
* Off by default, i.e. no guarantees that {#code AopContext} access will work.
* #since 4.3.1
*/
boolean exposeProxy() default false;
On the other hand there is something like this, which is going to be better than AOP:
/**
* A plugin interface that allows you to intercept (and possibly mutate) records received by the consumer. A primary use-case
* is for third-party components to hook into the consumer applications for custom monitoring, logging, etc.
*
* <p>
* This class will get consumer config properties via <code>configure()</code> method, including clientId assigned
* by KafkaConsumer if not specified in the consumer config. The interceptor implementation needs to be aware that it will be
* sharing consumer config namespace with other interceptors and serializers, and ensure that there are no conflicts.
* <p>
* Exceptions thrown by ConsumerInterceptor methods will be caught, logged, but not propagated further. As a result, if
* the user configures the interceptor with the wrong key and value type parameters, the consumer will not throw an exception,
* just log the errors.
* <p>
* ConsumerInterceptor callbacks are called from the same thread that invokes {#link org.apache.kafka.clients.consumer.KafkaConsumer#poll(long)}.
* <p>
* Implement {#link org.apache.kafka.common.ClusterResourceListener} to receive cluster metadata once it's available. Please see the class documentation for ClusterResourceListener for more information.
*/
public interface ConsumerInterceptor<K, V> extends Configurable {
UPDATE
#EnableAspectJAutoProxy(exposeProxy=true) did not work and I know that I could use interceptor, but I wanted to make it working with AOP.
Then I suggest you to consider to separate a DepartmentsMessageConsumer and the ConcurrentMessageListenerContainer. I mean move that ConcurrentMessageListenerContainer into the separate #Configuration class. The ApplicationLauncher is a good candidate. Make it as a #Bean and dependent on your DepartmentsMessageConsumer for injection. The point is that you need to give an AOP a chance to instrument your DepartmentsMessageConsumer, but with the #PostConstruct, that's too early to instantiate and start consumption from Kafka.

Spring how to get beans in a factory

I have the following code :
public interface CreatorFactory<E extends Vehicle> {
public VehicleType<E> getVehicle();
public boolean supports(String game);
}
public abstract AbstractVehicleFactory<E extends Vehicle> implements CreatorFactory {
public VehicleType<E> getVehicle() {
// do some generic init
getVehicle();
}
public abstract getVehicle();
public abstract boolean supports(String game);
}
and i have multiple factories, for car, truck..etc..
#Component
public CarFactory extends AbstractVehicleFactory<Car> {
/// implemented methods
}
#Component
public TruckFactory extends AbstractVehicleFactory<Truck> {
/// implemented methods
}
What I would like to do is pull the implemented factories into a seperate class as a list, but im not sure how generics works in this case... I know in spring you can get all beans of a specific type... would this still work?...
With erasure, i guess the generic types would be removed .. ??

Firstly, I think there is maybe no need to get a list of beans. And you just want get the exact bean which has declared with generics type.
In BeanFactory interface in Spring framework, there is a method to use for your requirement:
public interface BeanFactory {
/**
* Return the bean instance that uniquely matches the given object type, if any.
* #param requiredType type the bean must match; can be an interface or superclass.
* {#code null} is disallowed.
* <p>This method goes into {#link ListableBeanFactory} by-type lookup territory
* but may also be translated into a conventional by-name lookup based on the name
* of the given type. For more extensive retrieval operations across sets of beans,
* use {#link ListableBeanFactory} and/or {#link BeanFactoryUtils}.
* #return an instance of the single bean matching the required type
* #throws NoSuchBeanDefinitionException if there is not exactly one matching bean found
* #since 3.0
* #see ListableBeanFactory
*/
<T> T getBean(Class<T> requiredType) throws BeansException;
}
You can use code like:
Car carFactory = applicationContext.getBean( CarFactory.class );
Trunk trunkFactory = applicationContext.getBean( TrunkFactory.class );
or just see #Qualifier annotation for injection automaticly.
#Component("carFactory")
public CarFactory extends AbstractVehicleFactory<Car> {
/// implemented methods
}
#Component("truckFactory ")
public TruckFactory extends AbstractVehicleFactory<Truck> {
/// implemented methods
}
In client side code:
#Qualifier("carFactory")
#Autowired
private CarFactory carFactory ;
#Qualifier("truckFactory")
#Autowired
private TruckFactory TruckFactory;

Looks like you need:
#Autowired
List<AbstractVehicleFactory> abstractVehicleFactories;

How to use ServletScopes.scopeRequest() and ServletScopes.continueRequest()?

How is one supposed to use ServletScopes.scopeRequest()?
How do I get a reference to a #RequestScoped object inside the Callable?
What's the point of seedMap? Is it meant to override the default binding?
What's the difference between this method and ServletScopes.continueRequest()?

Answering my own question:
ServletScopes.scopeRequest() runs a Callable in a new request scope. Be careful not to reference objects across different scopes, otherwise you'll end up with threading issues such as trying to use a database connection that has already been closed by another request. static or top-level classes are your friend here.
You inject the Callable before passing it into ServletScopes.scopeRequest(). For this reason, you must be careful what fields your Callable contains. More on this below.
seedMap allows you to inject non-scoped objects into the scope. This is dangerous so be careful with what you inject.
ServletScopes.continueRequest() is similar except that it runs inside an existing request scope. It takes a snapshot of the current HTTP scope and wraps it in a Callable. The original HTTP request completes (you return some response from the server) but then complete the actual operation asynchronously in a separate thread. When the Callable is invoked at some later time (in that separate thread) it will have access to the original HttpServletRequest but not the HTTP response or session.
So, what's the best way to do this?
If you don't need to pass user-objects into the Callable: Inject the Callable outside the request scope, and pass it into ServletScopes.scopeRequest(). The Callable may only reference Provider<Foo> instead of Foo, otherwise you'll end up with instances injected outside of the request scope.
If you need to pass user-objects into the Callable, read on.
Say you have a method that inserts names into a database. There are two ways for us to pass the name into the Callable.
Approach 1: Pass user-objects using a child module:
Define InsertName, a Callable that inserts into the database:
#RequestScoped
private static class InsertName implements Callable<Boolean>
{
private final String name;
private final Connection connection;
#Inject
public InsertName(#Named("name") String name, Connection connection)
{
this.name = name;
this.connection = connection;
}
#Override
public Boolean call()
{
try
{
boolean nameAlreadyExists = ...;
if (!nameAlreadyExists)
{
// insert the name
return true;
}
return false;
}
finally
{
connection.close();
}
}
}
Bind all user-objects in a child module and scope the callable using RequestInjector.scopeRequest():
requestInjector.scopeRequest(InsertName.class, new AbstractModule()
{
#Override
protected void configure()
{
bind(String.class).annotatedWith(Names.named("name")).toInstance("John");
}
})
We instantiate a RequestInjector outside the request and it, in turn, injects a second Callable inside the request. The second Callable can reference Foo directly (no need for Providers) because it's injected inside the request scope.
import com.google.common.base.Preconditions;
import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Key;
import com.google.inject.Module;
import com.google.inject.servlet.ServletScopes;
import java.util.Collections;
import java.util.Map;
import java.util.concurrent.Callable;
/**
* Injects a Callable into a non-HTTP request scope.
* <p/>
* #author Gili Tzabari
*/
public final class RequestInjector
{
private final Map<Key<?>, Object> seedMap = Collections.emptyMap();
private final Injector injector;
/**
* Creates a new RequestInjector.
*/
#Inject
private RequestInjector(Injector injector)
{
this.injector = injector;
}
/**
* Scopes a Callable in a non-HTTP request scope.
* <p/>
* #param <V> the type of object returned by the Callable
* #param callable the class to inject and execute in the request scope
* #param modules additional modules to install into the request scope
* #return a wrapper that invokes delegate in the request scope
*/
public <V> Callable<V> scopeRequest(final Class<? extends Callable<V>> callable,
final Module... modules)
{
Preconditions.checkNotNull(callable, "callable may not be null");
return ServletScopes.scopeRequest(new Callable<V>()
{
#Override
public V call() throws Exception
{
return injector.createChildInjector(modules).getInstance(callable).call();
}
}, seedMap);
}
}
Approach 2: Inject a Callable outside the request that references Provider<Foo>. The call() method can then get() the actual values inside the request scope. The object objects are passed in by way of a seedMap (I personally find this approach counter-intuitive):
Define InsertName, a Callable that inserts into the database. Notice that unlike Approach 1, we must use Providers:
#RequestScoped
private static class InsertName implements Callable<Boolean>
{
private final Provider<String> name;
private final Provider<Connection> connection;
#Inject
public InsertName(#Named("name") Provider<String> name, Provider<Connection> connection)
{
this.name = name;
this.connection = connection;
}
#Override
public Boolean call()
{
try
{
boolean nameAlreadyExists = ...;
if (!nameAlreadyExists)
{
// insert the name
return true;
}
return false;
}
finally
{
connection.close();
}
}
}
Create bogus bindings for the types you want to pass in. If you don't you will get: No implementation for String annotated with #com.google.inject.name.Named(value=name) was bound. https://stackoverflow.com/a/9014552/14731 explains why this is needed.
Populate the seedMap with the desired values:
ImmutableMap<Key<?>, Object> seedMap = ImmutableMap.<Key<?>, Object>of(Key.get(String.class, Names.named("name")), "john");
Invoke ServletScopes.scopeRequest():
ServletScopes.scopeRequest(injector.getInstance(InsertName.class), seedMap);

Spring AOP pointcut that matches annotation on interface

I have a service class implemented in Java 6 / Spring 3 that needs an annotation to restrict access by role.
I have defined an annotation called RequiredPermission that has as its value attribute one or more values from an enum called OperationType:
public #interface RequiredPermission {
/**
* One or more {#link OperationType}s that map to the permissions required
* to execute this method.
*
* #return
*/
OperationType[] value();}
public enum OperationType {
TYPE1,
TYPE2;
}
package com.mycompany.myservice;
public interface MyService{
#RequiredPermission(OperationType.TYPE1)
void myMethod( MyParameterObject obj );
}
package com.mycompany.myserviceimpl;
public class MyServiceImpl implements MyService{
public myMethod( MyParameterObject obj ){
// do stuff here
}
}
I also have the following aspect definition:
/**
* Security advice around methods that are annotated with
* {#link RequiredPermission}.
*
* #param pjp
* #param param
* #param requiredPermission
* #return
* #throws Throwable
*/
#Around(value = "execution(public *"
+ " com.mycompany.myserviceimpl.*(..))"
+ " && args(param)" + // parameter object
" && #annotation( requiredPermission )" // permission annotation
, argNames = "param,requiredPermission")
public Object processRequest(final ProceedingJoinPoint pjp,
final MyParameterObject param,
final RequiredPermission requiredPermission) throws Throwable {
if(userService.userHasRoles(param.getUsername(),requiredPermission.values()){
return pjp.proceed();
}else{
throw new SorryButYouAreNotAllowedToDoThatException(
param.getUsername(),requiredPermission.value());
}
}
The parameter object contains a user name and I want to look up the required role for the user before allowing access to the method.
When I put the annotation on the method in MyServiceImpl, everything works just fine, the pointcut is matched and the aspect kicks in. However, I believe the annotation is part of the service contract and should be published with the interface in a separate API package. And obviously, I would not like to put the annotation on both service definition and implementation (DRY).
I know there are cases in Spring AOP where aspects are triggered by annotations one interface methods (e.g. Transactional). Is there a special syntax here or is it just plain impossible out of the box.
PS: I have not posted my spring config, as it seems to be working just fine. And no, those are neither my original class nor method names.
PPS: Actually, here is the relevant part of my spring config:
<aop:aspectj-autoproxy proxy-target-class="false" />
<bean class="com.mycompany.aspect.MyAspect">
<property name="userService" ref="userService" />
</bean>

If I understand you correct, you want a pointcut that finds all methods in classes that extends MyService and is annotated and with the preferred arguments.
I propose that you replace:
execution(public * com.mycompany.myserviceimpl.*(..))
with:
execution(public * com.mycompany.myservice.MyService+.*(..))
The plus sign is used if you want a joinpoint to match the MyService class or a class that extends it.
I hope it helps!

Espen, your code works only for one class:
execution(public * com.mycompany.myservice.MyService+.*(..))
but what if I want this behaviour for all services in *com.mycompany.services.** package?

How do I add a type to GWT's Serialization Policy whitelist?

GWT's serializer has limited java.io.Serializable support, but for security reasons there is a whitelist of types it supports. The documentation I've found, for example this FAQ entry says that any types you want to serialize "must be included in the serialization policy whitelist", and that the list is generated at compile time, but doesn't explain how the compiler decides what goes on the whitelist.
The generated list contains a number of types that are part of the standard library, such as java.lang.String and java.util.HashMap. I get an error when trying to serialize java.sql.Date, which implements the Serializable interface, but is not on the whitelist. How can I add this type to the list?

There's a workaround: define a new Dummy class with member fields of all the types that you want to be included in serialization. Then add a method to your RPC interface:
Dummy dummy(Dummy d);
The implementation is just this:
Dummy dummy(Dummy d) { return d; }
And the async interface will have this:
void dummy(Dummy d, AsyncCallback< Dummy> callback);
The GWT compiler will pick this up, and because the Dummy class references those types, it will include them in the white list.
Example Dummy class:
public class Dummy implements IsSerializable {
private java.sql.Date d;
}

Any specific types that you include in your service interface and any types that they reference will be automatically whitelisted, as long as they implement java.io.Serializable, eg:
public String getStringForDates(ArrayList<java.util.Date> dates);
Will result in ArrayList and Date both being included on the whitelist.
It gets trickier if you try and use java.lang.Object instead of specific types:
public Object getObjectForString(String str);
Because the compiler doesn't know what to whitelist. In that case if the objects are not referenced anywhere in your service interface, you have to mark them explicitly with the IsSerializable interface, otherwise it won't let you pass them through the RPC mechanism.

The whitelist is generated by the GWT compiler and contains all the entries that are designated by the IsSerializable marker interface.
To add a type to the list you just need to make sure that the class implements the IsSerializable interface.
Additionally for serialization to work correctly the class must have a default no arg constructor (constructor can be private if needed). Also if the class is an inner it must be marked as static.

IMHO the simpliest way to access whitelist programmatically is to create a class similar to this:
public class SerializableWhitelist implements IsSerializable {
String[] dummy1;
SomeOtherThingsIWishToSerialize dummy2;
}
Then include it in the .client package and reference from the RPC service (so it gets analyzed by the compiler).
I couldn't find a better way to enable tranfer of unparameterized maps, which is obviously what you sometimes need in order to create more generic services...

The whitelist is generated by the gwt compiler and contains all the entries that are designated by the IsSerializable marker interface.
To add a type to the list you just need to make sure that the class implements the IsSerializable interface.
-- Andrej
This is probably the easiest solution.
The only thing to remember with this is that all the classes that you want to serialize should have "public, no-argument" constructor, and (depending upon requirements) setter methods for the member fields.

to ensure the desired result delete all war/<app>/gwt/*.gwt.rpc

To anyone who will have the same question and doesn't find previous answers satisfactory...
I'm using GWT with GWTController, since I'm using Spring, which I modified as described in this message. The message explains how to modify GrailsRemoteServiceServlet, but GWTController calls RPC.decodeRequest() and RPC.encodeResponseForSuccess() in the same way.
This is the final version of GWTController I'm using:
/**
* Used to instantiate GWT server in Spring context.
*
* Original version from this tutorial.
*
* ...fixed to work as explained in this tutorial.
*
* ...and then fixed to use StandardSerializationPolicy as explained in
* this message to allow
* using Serializable instead of IsSerializable in model.
*/
public class GWTController extends RemoteServiceServlet implements Controller, ServletContextAware {
// Instance fields
private RemoteService remoteService;
private Class<? extends RemoteService> remoteServiceClass;
private ServletContext servletContext;
// Public methods
/**
* Call GWT's RemoteService doPost() method and return null.
*
* #param request
* The current HTTP request
* #param response
* The current HTTP response
* #return A ModelAndView to render, or null if handled directly
* #throws Exception
* In case of errors
*/
public ModelAndView handleRequest(HttpServletRequest request, HttpServletResponse response) throws Exception {
doPost(request, response);
return null; // response handled by GWT RPC over XmlHttpRequest
}
/**
* Process the RPC request encoded into the payload string and return a string that encodes either the method return
* or an exception thrown by it.
*
* #param payload
* The RPC payload
*/
public String processCall(String payload) throws SerializationException {
try {
RPCRequest rpcRequest = RPC.decodeRequest(payload, this.remoteServiceClass, this);
// delegate work to the spring injected service
return RPC.invokeAndEncodeResponse(this.remoteService, rpcRequest.getMethod(), rpcRequest.getParameters(), rpcRequest.getSerializationPolicy());
} catch (IncompatibleRemoteServiceException e) {
return RPC.encodeResponseForFailure(null, e);
}
}
/**
* Setter for Spring injection of the GWT RemoteService object.
*
* #param RemoteService
* The GWT RemoteService implementation that will be delegated to by the {#code GWTController}.
*/
public void setRemoteService(RemoteService remoteService) {
this.remoteService = remoteService;
this.remoteServiceClass = this.remoteService.getClass();
}
#Override
public ServletContext getServletContext() {
return servletContext;
}
public void setServletContext(ServletContext servletContext) {
this.servletContext = servletContext;
}
}

I found that just putting it in the client package or using it in a dummy service interface was not sufficient as it seemed the system optimized it away.
I found it easiest to create a class that derived from one of the types already used in the service interface and stick it in the client package. Nothing else needed.
public class GWTSerializableTypes extends SomeTypeInServiceInterface implements IsSerializable {
Long l;
Double d;
private GWTSerializableTypes() {}
}

I had this problem but ended up tracing the problem back to a line of code in my Serializable object:
Logger.getLogger(this.getClass().getCanonicalName()).log(Level.INFO, "Foo");
There were no other complaints before the exception gets caught in:
#Override
protected void serialize(Object instance, String typeSignature)
throws SerializationException {
assert (instance != null);
Class<?> clazz = getClassForSerialization(instance);
try {
serializationPolicy.validateSerialize(clazz);
} catch (SerializationException e) {
throw new SerializationException(e.getMessage() + ": instance = " + instance);
}
serializeImpl(instance, clazz);
}
And the business end of the stack trace is:
com.google.gwt.user.client.rpc.SerializationException: Type 'net.your.class' was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded. For security purposes, this type will not be serialized.: instance = net.your.class#9c7edce
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serialize(ServerSerializationStreamWriter.java:619)