Develop Reference

Apache Camel: access Route after polling

I'm using Camel JPA endpoints to poll a database and copy the data to a second one.
To not poll duplicates, I'm planning to save the highest ID of the copied data and only poll data with an ID higher than that one.
To save a few database writes, I want to write back the highest ID after the current polling / copying run is over, not for every single data element. I can access the element (and its ID) in the Camel Route class:
private Long startId = 0L;
private Long lastId = 0L;
from("jpa://Data").routeId("dataRoute")
.onCompletion().onCompleteOnly().process(ex -> {
if (lastId > startId) {
startId = lastId;
logger.info("New highest ID: {}", startId);
}
}).end()
.process(ex -> {
Data data = ex.getIn().getBody(Data.class);
lastId = data.getId();
NewData newData = (NewData) convertData(data);
ex.getMessage().setBody(newData);
}).to("jpa://NewData")
Now I want to save startId after the current polling is over. To do so, I overrode the PollingConsumerPollStrategy with my custom one where I want to access lastId inside the commit method (which gets executed exactly when I want it to, after the current polling is complete).
However, I can't access the route there. I tried via the route ID:
#Override
public void commit(Consumer consumer, Endpoint endpoint, int polledMessages) {
var route = (MyRoute) endpoint.getCamelContext().getRoute("dataRoute");
var lastId = route.getLastId();
log.debug("LastID: {}", lastId);
}
However I'm getting a class cast exception: DefaultRoute to MyRoute. Is it something about handing the ID to my route?

I would do it a bit differently.
Instead of using RouteBuilder instance vars for storing startId and lastId, you may also put these values as GlobalOptions (which is basically a map of key-value pairs) of current CamelContext.
This way, you can easily obtain their value using:
public void commit(Consumer consumer, Endpoint endpoint, int polledMessages) {
String lastId = endpoint.getCamelContext().getGlobalOption​("lastId");
}
It is also (theoretically) a better implementation because it also supports potential concurrent executions, as the id are shared for all instances running in the context.

Unique ID number in static method through concurrency JAVA

I want to get unique ID from my domain object (table) ID each time a method is called. So that ID's do not repeat. I have a function that returns unique ID.
public static Long generateID (Short company)
throws Exception
{
IDDAO iDDAO = SpringApplicationContext.getBean (IDDAO.class);
ID iD = iDDAO.findByNaturalKey (new IDNatKey (company);
if (iD != null)
{
// Check if ID has reached limit, then reset the ID to the first ID
if (iD.getLatestIDno ().longValue () == iD.getLastIDno ().longValue ())
{
iD.setLatestIDno (iD.getFrstIDno ());
}
// Get next ID
iD.setLatestIDno (iD.getLatestIDno () + 1);
// update database with latest id
iDDAO.update (iD);
return iD.getLatestIDno ();
}
}
The issue is that if access the application from two machines and press button from UI to generate ID exactly at the same time, there are sometimes duplicate IDs returned from this method
e.g.
Long ID = TestClass.generateID (123);
This gives me duplicate sometimes.
I made the method like this
public static synchronized Long generateID (Short company)
throws Exception
so that only one thread can go in this function at a time, but the duplicate issue is still there.
I do not want to use database sequences as I do not want gaps in the ID sequences if the transaction rolls back, in that case sequence will be incremented still, which I do not want.Gaps at the middle are OK but not at end. E.g we have 1, 2 and 3 as IDs and 2 rolls back, that is OK. But if 3 rolls back, we should get 3 again when another user comes, in case of sequence, it will give 4
Please help me tell what I am doing incorrect ? static synchronized will still cause other threads to go inside this function at same time ? I have many other static (but not synchronized) functions in the class. Will this cause issue with them too if I make it static synchronized ?
Thanks
Aiden

You can use java.util.UUID. it will generate a universal uniqueId.

Keep 2 unique IDs:
a db-provided, internal transaction ID, created by an autoincrement every time a new transaction is built. Gaps may appear if transactions are rolled back.
a pretty, gap-less "ticket ID", assigned only once the transaction commits successfully.
Assign both from the DB - it is best to keep all shared state there, as the DB will guarantee ACID, while Java concurrency is far trickier to get right.

In that case I think you try the below :
synchronized(this){if (iD != null)
{
// Check if ID has reached limit, then reset the ID to the first ID
if (iD.getLatestIDno ().longValue () == iD.getLastIDno ().longValue ())
{
iD.setLatestIDno (iD.getFrstIDno ());
}
// Get next ID
iD.setLatestIDno (iD.getLatestIDno () + 1);
// update database with latest id
iDDAO.update (iD);
return iD.getLatestIDno ();
}}

How add messages (integer values) from multiple agents in another agent using JADE

I have the following requirement in JADE.
I have to create multiple agents whose task is to generate random numbers. Another receiver agent collects all the random numbers generated and sums them up to make a decision.
This is the sender agent class extends tickerbehaviour and its tick onTick method is as follows
protected void onTick()
{
ACLMessage msg_LoadDetails = new ACLMessage(ACLMessage.INFORM);
msg_LoadDetails.addReceiver(new AID("LoadCoordinator", AID.ISLOCALNAME));
msg_LoadDetails.setContent(String.valueOf(getLoad()));
LoadConv.send(msg_LoadDetails);
//load = (int)(Math.random()*1000+1);
}
The receiver class extend cyclic behaviour and its action method is as follows
public void action()
{
ACLMessage msg_IncomingLoadDetails = LoadCoordinator.receive();
if(msg_IncomingLoadDetails!=null)
totalLoad = Integer.parseInt(msg_IncomingLoadDetails.getContent());
if(totalLoad>500)
{actioncommand = "off";}
else
{actioncommand = "on";}
System.out.println("The current load is:" +totalLoad+ "; the load will be switched " +actioncommand);
block();
}
The issue here is that the received values are just for one agent which i create (from console). I want to receive values for all created agents. Does this require me create an array of agents? How do i read values from many agent messages? Could some one please help with the code to do so as I am new to JAVA and JADE?

I tested your code and receiver agent gets message from all sender agents.
I have some questions and remarks:
How often sender agents should send their number? Only once, or cyclically, after given time?
Your receiver agent doesn't sum received values, he always compares last received value.
Better use this structure for logic, after receiving message:
if (msg_IncomingLoadDetails != null) {
// your logic
} else {
block();
}
it can help to prevent some problems like NullPointerException

Data commit issue in multithreading

I am new to Java and Hibernate.
I have implemented a functionality where I generate request nos. based on already saved request no. This is done by finding the maximum request no. and incrementing it by 1,and then again save i it to database.
However I am facing issues with multithreading. When two threads access my code at the same time both generate same request no. My code is already synchronized. Please suggest some solution.
synchronized (this.getClass()) {
System.out.println("start");
certRequest.setRequestNbr(generateRequestNumber(certInsuranceRequestAddRq.getAccountInfo().getAccountNumberId()));
reqId = Utils.getUniqueId();
certRequest.setRequestId(reqId);
ItemIdInfo itemIdInfo = new ItemIdInfo();
itemIdInfo.setInsurerId(certRequest.getRequestId());
certRequest.setItemIdInfo(itemIdInfo);
dao.insert(certRequest);
addAccountRel();
System.out.println("end");
}
Following is the output showing my synchronization:
start
end
start
end
Is it some Hibernate issue.
Does the use of transactional attribute in Spring affects the code commit in my Case?
I am using the following Transactional Attribute:
#Transactional(readOnly = false, propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
EDIT: code for generateRequestNumber() shown in chat room.
public String generateRequestNumber(String accNumber) throws Exception {
String requestNumber = null;
if (accNumber != null) {
String SQL_QUERY = "select CERTREQUEST.requestNbr from CertRequest as CERTREQUEST, "
+ "CertActObjRel as certActObjRel where certActObjRel.certificateObjkeyId=CERTREQUEST.requestId "
+ " and certActObjRel.certObjTypeCd=:certObjTypeCd "
+ " and certActObjRel.certAccountId=:accNumber ";
String[] parameterNames = {"certObjTypeCd", "accNumber"};
Object[] parameterVaues = new Object[]
{
Constants.REQUEST_RELATION_CODE, accNumber
};
List<?> resultSet = dao.executeNamedQuery(SQL_QUERY,
parameterNames, parameterVaues);
// List<?> resultSet = dao.retrieveTableData(SQL_QUERY);
if (resultSet != null && resultSet.size() > 0) {
requestNumber = (String) resultSet.get(0);
}
int maxRequestNumber = -1;
if (requestNumber != null && requestNumber.length() > 0) {
maxRequestNumber = maxValue(resultSet.toArray());
requestNumber = Integer.toString(maxRequestNumber + 1);
} else {
requestNumber = Integer.toString(1);
}
System.out.println("inside function request number" + requestNumber);
return requestNumber;
}
return null;
}

Don't synchronize on the Class instance obtained via getClass(). It can have some strange side effects. See https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=43647087
For example use:
synchronize(this) {
// synchronized code
}
or
private synchronized void myMethod() {
// synchronized code
}
To synchronize on the object instance.
Or do:
private static final Object lock = new Object();
private void myMethod() {
synchronize(lock) {
// synchronized code
}
}
Like #diwakar suggested. This uses a constant field to synchronize on to guarantee that this code is synchronizing on the same lock.
EDIT: Based on information from chat, you are using a SELECT to get the maximum requestNumber and increasing the value in your code. Then this value is set on the CertRequest which is then persisted in the database via a DAO. If this persist action is not committed (e.g. by making the method #Transactional or some other means) then another thread will still see the old requestNumber value. So you could solve this by making the code transactional (how depends on which frameworks you use etc.). But I agree with #VA31's answer which states that you should use a database sequence for this instead of incrementing the value in code. Instead of a sequence you could also consider using an auto-incement field in CertRequest, something like:
#GeneratedValue(strategy=GenerationType.AUTO)
private int requestNumber;
For getting the next value from a sequence you can look at this question.

You mentioned this information in your question.
I have implemented a functionality where I generate request nos. based on already saved request no. This is done by finding the maximum request no. and incrementing it by 1,and then again save i it to database.
On a first look, it seems the problem caused by multi appserver code. Threads are synchronised inside one JVM(appserver). If you are using more than one appserver then you have to do it differently using more robust approach by using server to server communication or by batch allocation of request no to each appserver.
But, if you are using only one appserver and multiple threads accessing the same code then you can put a lock on the instance of the class rather then the class itself.
synchronized(this) {
lastName = name;
nameCount++;
}
Or you can use the locks private to the class instance
private Object lock = new Object();
.
.
synchronized(lock) {
System.out.println("start");
certRequest.setRequestNbr(generateRequestNumber(certInsuranceRequestAddRq.getAccountInfo().getAccountNumberId()));
reqId = Utils.getUniqueId();
certRequest.setRequestId(reqId);
ItemIdInfo itemIdInfo = new ItemIdInfo();
itemIdInfo.setInsurerId(certRequest.getRequestId());
certRequest.setItemIdInfo(itemIdInfo);
dao.insert(certRequest);
addAccountRel();
System.out.println("end");
}
But make sure that your DB is updated by the new sequence no before the next thread is accessing it to get new one.

It is a good practice to generate "the request number (Unique Id)" by using the DATABASE SEQUENCE so that you don't need to synchronize your Service/DAO methods.

First thing:
Why are you getting the thread inside the method. I is not required here.
Also, one thing;
Can you try like this once:
final static Object lock = new Object();
synchronized (lock)
{
.....
}
what I feel is that object what you are calling is different so try this once.

Netty: Add and Remove FrameDecoder dynamically from a Pipeline, Protocol encapsulation

I've been working with Netty 3.3.1-Final for 3 weeks now.
My Protocol has 3 steps and each step needs a different FrameDecoder:
Read arguments
Transfer some data
Mutual close of the data pipe
I've been through a lot of "blocking" issues that I could not understand. It finally appears to me, reading the org.jboss.netty.example.portunification example that I had some buffer issue when trying to dynamically change my FrameDecoder: the buffer of one FrameDecoder was (probably) not empty when changing for the next one...
Is there a way to do that easily in Netty? Do I have to change my Protocol? Do I need to write one big FrameDecoder and manage a state?
If so, how to avoid code duplication between different protocols with common sub parts (for instance "reading arguments")?
Today I came to the idea of a FrameDecoderUnifier (code below) with the purpose of a way to hot add and remove some FrameDecoder, what do you think?
Thanks for your help!
Renaud
----------- FrameDecoderUnifier class --------------
/**
* This FrameDecoder is able to forward the unused bytes from one decoder to the next one. It provides
* a safe way to replace a FrameDecoder inside a Pipeline.
* It is not safe to just add and remove FrameDecoder dynamically from a Pipeline because there is a risk
* of unread bytes inside the buffer of the FrameDecoder you wan't to remove.
*/
public class FrameDecoderUnifier extends FrameDecoder {
private final Method frameDecoderDecodeMethod;
volatile boolean skip = false;
LastFrameEventHandler eventHandler;
LinkedList<Entry> entries;
Entry entry = null;
public FrameDecoderUnifier(LastFrameEventHandler eventHandler) {
this.eventHandler = eventHandler;
this.entries = new LinkedList<Entry>();
try {
this.frameDecoderDecodeMethod = FrameDecoder.class.getMethod("decode", ChannelHandlerContext.class, Channel.class, ChannelBuffer.class);
} catch (NoSuchMethodException ex) {
throw new RuntimeException(ex);
} catch (SecurityException ex) {
throw new RuntimeException(ex);
}
}
public void addLast(FrameDecoder decoder, LastFrameIdentifier identifier) {
entries.addLast(new Entry(decoder, identifier));
}
private Object callDecode(FrameDecoder decoder, ChannelHandlerContext ctx, Channel channel, ChannelBuffer buffer) throws Exception {
return frameDecoderDecodeMethod.invoke(decoder, ctx, channel, buffer);
}
#Override
protected Object decode(ChannelHandlerContext ctx, Channel channel, ChannelBuffer buffer) throws Exception {
if (entry == null && !entries.isEmpty()) {
entry = entries.getFirst();
}
if (entry == null) {
return buffer; //No framing, no decoding
}
//Perform the decode operation
Object obj = callDecode(entry.getDecoder(), ctx, channel, buffer);
if (obj != null && entry.getIdentifier().isLastFrame(obj)) {
//Fire event
eventHandler.lastObjectDecoded(entry.getDecoder(), obj);
entry = null;
}
return obj;
}
/**
* You can use this interface to take some action when the current decoder is changed for the next one.
* This can be useful to change some upper Handler in the pipeline.
*/
public interface LastFrameEventHandler {
public void lastObjectDecoded(FrameDecoder decoder, Object obj);
}
public interface LastFrameIdentifier {
/**
* True if after this frame, we should disable this decoder.
* #param obj
* #return
*/
public abstract boolean isLastFrame(Object decodedObj);
}
private class Entry {
FrameDecoder decoder;
LastFrameIdentifier identifier;
public Entry(FrameDecoder decoder, LastFrameIdentifier identifier) {
this.decoder = decoder;
this.identifier = identifier;
}
public FrameDecoder getDecoder() {
return decoder;
}
public LastFrameIdentifier getIdentifier() {
return identifier;
}
}
}

I have had similar problems, in that removing a frame decoder from a pipeline does not seem to prevent it from being called, and there isn't an obvious way to make the decoder to behave as if it wasn't in the chain: Netty insists that the decode() reads at least one byte so you can't simply return the incoming ChannelBuffer, whereas returning null stops the processing of incoming data until the next packet arrives, stalling the protocol decoding process.
Firstly: the Netty 3.7 docs for FrameDecoder does in fact has a section "Replacing a decoder with another decoder in a pipeline". It says:
It is not possible to achieve this simply by calling
ChannelPipeline#replace()
Instead, it suggests passing the data on by returning an array wrapping the decoded first packet and the rest of the data received.
return new Object[] { firstMessage, buf.readBytes(buf.readableBytes()) };
Importantly, "unfolding" must have been enabled prior to this, but this part is easy to miss and isn't explained. The best clue I could find was Netty issue 132, which evidently gave rise to the "unfold" flag on FrameDecoders. If true, the decoder will unpack such arrays into objects in a way which is transparent to downstream handlers. A peep at the source code seems to confirm this is what "unfolding" means.
Secondly: there seems to be an even simpler way, since the example also shows how to pass data on down the pipeline unchanged. For example, after doing its job, my sync packet FrameDecoder sets an internal flag and removes itself from the pipeline, returning the decoded object as normal. Any subsequent invocations when the flag is set then simply pass the data on like so:
protected Object decode(ChannelHandlerContext ctx,
Channel channel, ChannelBuffer cbuf) throws Exception {
// Close the door on more than one sync packet being decoded
if (m_received) {
// Pass on the data to the next handler in the pipeline.
// Note we can't just return cbuf as-is, we must drain it
// and return a new one. Otherwise Netty will detect that
// no bytes were read and throw an IllegalStateException.
return cbuf.readBytes(cbuf.readableBytes());
}
// Handle the framing
ChannelBuffer decoded = (ChannelBuffer) super.decode(ctx, channel, cbuf);
if (decoded == null) {
return null;
}
// Remove ourselves from the pipeline now
ctx.getPipeline().remove(this);
m_received = true;
// Can we assume an array backed ChannelBuffer?
// I have only hints that we can't, so let's copy the bytes out.
byte[] sequence = new byte[magicSequence.length];
decoded.readBytes(sequence);
// We got the magic sequence? Return the appropriate SyncMsg
return new SyncMsg(Arrays.equals(sequence, magicSequence));
}
A decoder derived from LengthFieldBasedFrameDecoder remains downstream and handles all subsequent data framing. Works for me, so far.

I think, having a frame decoder which switch internal decoders based on some state and dynamically adding/removing upper layer handlers should be avoided because
Difficult to understand/debug the code
Handlers are not having well defined responsibilities (That's why you are removing/adding handlers right? One handler should handle one or more (related) types of protocol messages, not many handlers same type of messages)
Ideally frame decoder only extract the protocol frame, not to decode the frame based on state (here frame decoder can have internal chain of decoders to decoder the frame and fire a MessageEvent with decoded message, above handlers can react to decoded messages).
UPDATE: Here I have considered a protocol where each messages can have a unique tag/identifier with end of the message is clearly marked (for example Tag Length Value frame format)