My questions is as follows:
I have a service, which queries the DB and retrieves some records. Then it updates an external system using that information and in the end in updates back the DB.
Using spring trascations and weblogic jta transaction manager i was able with sample code below not to loose any messages in case
No records are retrived. (these are mandatory for the External System)
External System Error
Failed to update the DB
So in all aboce cases the JMS Listener puts the errror message back to the queue.
My question, is there any better way, using Spring with all its goodies, to manage that? The sample code below throws explicilty a RuntimeException which i dont think is a good design...
Please for your comments.
EDIT:
The queue is being polled by the submissionListener MDP that its configuration is shown below. After the message is consumed it invokes the registerDocument() of the service. (another Spring bean). That service invokes 2 times the DAO and the external system.
Check out Spring's documentation on JmsTemplate and Message Driven POJOs for patterns in the core Spring framework.
Spring Integration models higher-level abstractions related to message-orientated patterns.
Related
We recently had a very unexpected issue in PROD which we manage to reproduce but we still don't have a clear explanation of what is causing it and more importantly how to fix it.
Our system is a vendor workflow framework that offers us hooks to add our custom behavior. It is running on WebSphere Application Server and it is a mix of Camel and Spring and EJB. Initially it was pure EJB but a few years ago the vendor added Camel with a view of getting rid of application server.
Now the problem:
The start of processing is a Camel route which is transacted with PROPAGATION_REQUIRED attribute. Part of the processing there is a local call to an EJB method. There is not explicit setting for any of EJB methods exposed apart from being Container managed. This means the EJB call should get the implicit value of REQUIRED. Part of the EJB call there is data change happening. The reference to the EJB is obtained using some code like below:
String repositoryName = MBLLookupUtil.getInstance().getRepositoryName();
RepositoryInstanceFacadeHome facadeHome = MBLLookupUtil.getInstance().getRepositoryInstanceFacadeHome(
repositoryName);
RepositoryInstanceFacade facade = facadeHome.create();
// Here is when the data change happens:
facade.doSomeWork();
The application logs show clearly that the EJB processing is happening in the same thread as the client. After making the call there is more work in the calling client when a Timeout exception happens.
We expect that both data change on client side and that changed as part of the EJB call will be roll backed, however the data changed by EJB call stays committed while the client side (Camel) gets roll backed.
In the spring application context we use WebSphereUowTransactionManager which is the IBM recommended way of configuring transaction manager:
<bean id="txManager" class="org.springframework.transaction.jta.WebSphereUowTransactionManager"/>
Any hints about what could cause this or how we could further investigate would be great appreciated. Thank you in advance.
UPDATE:
As advised in tomgeraghty3's comment I made a code change to log the value of TransactionSynchronizationManager.isActualTransactionActive() and the result was true
UPDATE 2:
We found out where the issue was by spending hours of analyzing decompiled vendor code. The transaction propagation was configured in an application-context.xml file but it was also overwritten straight in the POJO code with an #Transactional(propagation=REQUIRED_NEW) annotation. We proved that was the problem by removing the POJO class from the vendor jar and adding one with no annotation with the same package in one of ours. Very dirty just to prove our suspicion.
Now our bigger challenge would be to persuade the vendor address the issue. Hope it is just a bug and not an impacting change.
I am working on design where the requirement is as follows
The application would be built using spring boot
As part of the requirement we would have to build some orchestration where we would end up calling multiple SOAP and REST services and we would perform aggregation, transformation etc.
When an exception/error occurs during the orchestration, we need to capture and persist the state and retry it from the same point where the error occurred.
For example, Lets say there is a Java method in which we have multiple calls to external components say 1 soap call and 2 rest calls i.e.
String doOrchestration(InputObject obj){
Line 1: Object obj = soap call[ input is XML format];
Line 2: Object obj1 = rest1 call [the input is obj in JSON format]
Line 3: Object obj3 = rest2 call[the input is obj1 in JOSN format]
//do something
return str
}
The error occurred on Line 2. We need to capture/persist the state and continue the execution from the Line 2 later.
Is there any workflow library, orchestration patterns and/or framework that we can integrate with spring/Spring-boot
Thanks,
well, one tecnology that i use to handle this kind of problems is apache camel, whith camel you can set routes,for instance in your requirement, you can build the main route who will orchest other routes such as yours endpoints, if one endpoint has an exception you can call the route who will persist that and call again the route of the endpoint, or something like that.
http://camel.apache.org/getting-started.html
edit: camel has a lot of potencial transforming messages such as json, xml, objects.
i have a very simple project on github with spring boot and camel if you want to check it.
https://github.com/chill-hub/spring_boot_camel_base_archetype
The Apache Camel version spring like is spring-integration at
https://spring.io/projects/spring-integration
Apache Camel and Spring Integration implements enterprise integration pattern called EIP. Please look at https://www.enterpriseintegrationpatterns.com/ first.
It is basically design patterns for workflow and orchestration !
You have an integration (starter) with spring boot made by spring named spring-boot-starter-integration.
You should probably look at Java Spring Config dsl reference at https://github.com/spring-projects/spring-integration-java-dsl/wiki/spring-integration-java-dsl-reference
which will show you a nice overview, with the more modern approach (xml, and annotation starts to disapear).
With what you need to do you will probably need https://docs.spring.io/spring-integration/reference/html/ws.html which is the web extension for spring integration.
Be careful it is a very elegant solution but with lot of abstraction and can be difficult at the beginning. Keep in mind EIP defines building block and vocabulary to build workflows, Apache Camel and Spring Integration are frameworks that implements these building blocks and offer a threading model abstraction (also queue persistant abstraction), and both have several extensions to adapt easily these building blocks to real word (web, ldap, databases, ...)
Both frameworks are good with pros and cons, but you will probably prefer spring integration in a spring ecosysteme team, same conventions than other spring modules !
Again theses framework are really elegant but abstraction has always a cost!
Disclaimer : I was a trainer for spring integration course - Official
I am trying to make a queue with activemq and spring boot using this link and it looks fine. What I am unable to do is to make this queue persistent after application goes down. I think that SimpleJmsListenerContainerFactory should be durable to achieve that but when I set factory.setSubscriptionDurable(true) and factory.setClientId("someid") I am unable to receive messages any more. I would be greatfull for any suggestions.
I guess you are embedding the broker in your application. While this is ok for integration tests and proof of concepts, you should consider having a broker somewhere in your infrastructure and connect to it. If you choose that, refer to the ActiveMQ documentation and you should be fine.
If you insist on embedding it, you need to provide a brokerUrl that enables message persistence.
Having said that, it looks like you misunderstand durable subscriber and message persistence. The latter can be achieved by having a broker that actually stores the content of the queue somewhere so that if the broker is stopped and restarted, it can restore the content of its queue. The former is to be able to receive a message even if the listener is not active at a period of time.
you can enable persistence of messages using ActiveMQConnectionFactory.
as mentioned in the spring boot link you provided, this ActiveMQConnectionFactory gets created automatically by spring boot.so you can have this bean in your application configuration created manually and you can set various property as well.
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=true");
Here is the link http://activemq.apache.org/how-do-i-embed-a-broker-inside-a-connection.html
I have a general question about Apache Camel. I wasn't able to find whether the aggregator is transacted. If it is transacted, how the transactions are implemented and how fast the aggregation is?
Sending the messages into the aggregator can run in a transaction.
You would need a persistent store with the aggregator to let the outgoing messages act as a transaction. See the documentation about persistence
http://camel.apache.org/aggregator2
For example there is a JDBC based and HawtDB (file based) persistent support out of the box. Its pluggable as you can also build your custom.
Camel in Action book chapter 8 and 9 convers this in much more details.
We've got a Spring based web application that makes use of Hibernate to load/store its entities to the underlying database.
Since it's a backend application we not only want to allow our UI but also 3rd party tools to manually initiate DB transactions. That's why the callers need to
Call a StartTransaction method and in return get an ID that they can refer to
Do all DB relevant calls (e. g. creating, modifying, deleting) by referring to this ID to make clear which operations belong to the started transaction
Call the CommitTransaction method to signal to our backend that the transaction can be committed now (or in the negative case RollbackTransaction will be called)
So keeping in mind, that all database handling will be done internally by the Java persistence annotations, how can we open the transaction management to our UI that behaves like a 3rd party application that has no direct access to the backend entities but deals with data transfer objects only?
From the Spring Reference: Programmatic transaction management
I think this can be done but would be a royal pain to implement/verify. You would basically require a transaction manager which is not bounded by "per-thread-transaction" definition but spans across multiple invocations for the same client.
JTA + Stateful session beans might be something you would want to have a look at.
Why don't you build services around your 'back end application' for example a SOAP interface or a REST interface.
With this strategy you can manage your transaction in the backend