I know about the 3 phase commit and SAGA pattern(doing transactions async in a distributed system using MQ). But it would involve using an MQ which makes me think of approach being taken mostly in SOA using ESB. Is there any way in which consistency can be achieved in between transactions in microservices using SpringCloud's inbuilt functionality.
Related
I have two micro services. One is in spring boot and other is using simple jdbc apis to perform database operation. I have to perform database operation using these two services.
(i.e.
Insert using jdbc service,
Insert using spring-boot service,
Insert using jdbc service).
These operation should follow acid property.
I have tried saga pattern using axon framework, it is working fine but I want to do it by using 2pc protocol. I have tried 2pc protocol in jdbc service but it is working only for transactions happened only in this service. I also used atomikos framework in spring service, it is working for this service only.
Is there any way to co-ordinate javax.transaction and springframework.transaction ?
For the scenario described in the question, you should try using Oracle Transaction Manager for Microservices (MicroTx). It is a free product that comes with a transaction manager and client library for microservices written in Java and node.js. With this, you can create XA (2PC) transactions involving multiple microservices. Oracle MicroTx - https://www.oracle.com/database/transaction-manager-for-microservices
I have several apps developed using spring boot. Some apps call another apps, that in time call other apps, it is getting hard to manage and scale. I need to be able to distribute them in a network and also combine the apps in different 'flows' with minimun changes to the apps.
Ideally I would like to wrap the apps and abstract them into components that have N inputs and M outputs. At boot time I would use some configuration to wire the inputs and outputs to real kafka topic queues.
For instance, input A to an app can come from several kafka topic queues, and the output B from the same app can go to other set of kafka topic queues.
I would like to be able to change the queues without having to recompile the apps, also no extra network hops to send/receive from/to multiple queues, this should in the same process and multi threaded.
Does anybody knows if something similar exists already? Can spring integration do this? Apache Camel? Or am I better off writing it myself?
See Spring for Apache Kafka. There is also a spring-integration-kafka extension that sits on top of spring-kafka.
The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with #KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.
In the introduction page of spring cloud it is said that
Spring Cloud provides tools for developers to quickly build some of
the common patterns in distributed systems (e.g. configuration
management, service discovery, circuit breakers, intelligent routing,
micro-proxy, control bus, one-time tokens, global locks, leadership
election, distributed sessions, cluster state)
In terms of microservices and distributed systems, can someone explain what intelligent routing is ?
Intelligent routing in Spring Cloud refers to the Zuul project (see the Spring Cloud Netflix guide). It uses reverse proxies: when you define a resource location in your public API, Zuul basically re-routes calls to these locations to your microservices. In effect, you can hide the locations and compositions of your microservices without changing your API.
I am building a spring cloud-based microservice ML pipeline.
I have a data ingestion service that (currently) takes in data from SQL, this data needs to be used by the prediction service.
The general consensus is that writes should use async message-based communication using kafka/rabbitmq.
What I am not sure about is how do I orchestrate these services?
Should I use an API gateway that invokes ingestion that starts the pipeline?
Typically you would build a service with rest endpoints (Spring Boot) to ingest the data. This service can then be deployed multiple times behind a api gateway (Zuul, Spring Cloud) that takes care about routing. This is the default spring cloud microservices setup. The ingest service can then convert the data and produce it to a RabbitMQ or Kafka. I recommend using Spring Cloud Stream for the interaction with the queue, it's abstraction on top of RabbitMQ and Kafka, which can be configured using starters/binders.
Spring Cloud Dataflow is a declarative approach for orchestration of your queues and also takes care of deployment on several cloud services/platforms. This can also be used but might add extra complexity to your use case.
I have two spring boot apps running in the same local network and they need to communicate with each other. An obvious answer is to leverage REST API and make http calls, but I would like to use Spring Integration project for this purpose.
That said, I have several architectural questions regarding that:
Should I setup a standalone messaging framework (e.g. Rabbit MQ) or embedded should also work (e.g. messaging will be embedded to one of the two apps).
If standalone, what messaging framework should I choose: ActiveMQ, RabbitMQ or something else?
Welcome to the Messaging Microservices world!
You go right way, but forget the embedded middleware if you are going to production. Especially when your application will be distributed geographically.
So, right you need some Message Broker and that should be definitely external one.
It's really your choice which one is better for your purpose. For example you can even take into account Apache Kafka or Redis.
If we talk here about Spring Integration it might be better for you to consider to use our new product - Spring Cloud Stream.
With that you just have your applications as Spring Boot Microservices which are able to connect to the external middleware transparently for the application. You just deal with message channels in the application!