I am an inexperienced Java developer trying to wrap my head around some fundamental middleware/SOA concepts and technologies, specifically:
Service-Oriented Architecture (SOA)
Message-Oriented Middleware (MOM)
Message Queue
Apache Camel
Mule
EJBs
Endpoints & Routes
Service Bus/ESB
JMS
After looking each of these up online/on Wikipedia, I was able to get (for the most part) decent definitions for each of these. What I am not understanding is how all of these technologies/concepts work together on the backend to provide a 2nd/business tier solution.
Can someone please give an example of an architecture that would use all of these technologies/concepts, and explain what role each of them play in the overall solution? Once I see a working example I'm sure it will help me connect most of the dots.
Edit: Since I added the bounty, I've had several answers that suggest reading books. Although I appreciate all feedback here, I simply can't part ways with 300 reputation points for an answer that, essentially, boils down to "RTM" (especially when I'm flat broke and can't afford the manual!) To reiterate, the bounty and definitive answer will go to someone who can hit all of these bullets in a meaningful, practical example. This does not have to be a middleware compendium!!! Just a paragraph or two that shows how all these can be used together in harmony to produce a Java business-tier solution. Thanks again.
SOA main principles: Build systems as set of services where each service is
Coarse-grained
Interoperable
Loosely coupled
A company offers a lot of business services (coarse-grained) developed over many years and exposed to the users (human or other systems) in some form. There are more chances that each of these features have been designed and developed not keeping the above three principles in mind. Moreover, each of those features might be running on disparate heterogeneous platforms, using different technologies etc.
What if you want to integrate these disparate features thus creating new solutions (For e.g. Amazon store front is a new service composed of their catalog service, shopping cart service etc)?
You have two choices:
Building the new feature from scratch keeping the 3 principles in mind. But it is a very costly endeavor, and one that’s almost never successful.
An effective and less risky alternative is to assemble/compose it from existing, proven (well tested) services.
Option 2 is where ESBs can help with their support for routing, transformation, monitoring etc. Apache Camel, Mule are open-source ESB's. Endpoints & Routes are the terminology used in EIP (Enterprise Integration Patterns) that these ESB's implement. ESB's can take help of MOM's (Message-Oriented-Middleware) when they want to route/integrate services that are running on heterogeneous platforms (For e.g. the catalog service might be running on a mainframe system but the shopping cart is implemented using stateful EJBs running in a Java Application server). Message queue is a concept in MOM that acts a temporary storage of the message between the sender and receiver. This temporary storage provides lot of benefits like asynchronous delivery, guaranteed delivery etc. There are several different MOM vendors like IBM (WebSphere MQ), open-source ActiveMQ etc. We can use JMS to keep your code independent of the vendor.
I tried to relate all the concepts with an example. I also tried to keep it short. Please
ask follow up questions to gain more understanding.
MOM is not a requirement to implement SOA. For e.g. if all of your services are exposed over SOAP via HTTP then you don't need a MOM in this case.
Java classes/example for every tech. might not be possible in single post because what you asked is evolution industry went through in last decade and still evolving. So, what happened over last decade can not be covered in one post. However, its good to understand how it went through this phase and why new technology stack required and what kind of problem it solves.
EJBs
Enterprise Java Beans serverside component architecture. It enables rapid and simplified development of
1) distributed(where multiple application server talks to each other, server components(e.g. service calling other service hosted on different server).
2) transactional - persistance bean (DB TXNs), most important part of any simple/web/distributed application. Easy development e.g configuration base. Write XML which takes care of the transaction e.g. when to commit, when to roll back(on exceptions) etc. JPA Java Persistance APIs provide object relationship maping. Such as your table row is mapped to your java object through the xml configuration.
3) secure - authentication(uid/pwd) and authorization(role based- who is logged in user and what all task he can do?).
This looks good at one point to develope any enterprise application however it has got some disadvantages e.g. its very heavy (all jars included in it.). Classes used as bean should confirm to EJB standards (classes should have implemented certain interface for EJB engine to understand which type of bean it is).
To overcome such scenarioes, there are many alternatives are available in industry for EJBs e.g Hibrnate does same things such as OR mapping, TXN handling same provided by persistance bean in EJB. Spring, light weight framework and simplifies business logic (you can use your already build class which need not to implement any interface, checked exceptions or extends some of the mandatory abstract classes).
Now a days, most of the compnies realy on the light weight frame work such as Spring, Hibernate, IBatis, Axis-2.
Service-Oriented Architecture (SOA)
Service Oriented Architecture is an answer to the platform independence at the enterprise level.
OR
To integrate your application faster, to communicate between different application server.
Just think that you want to implement solution where you are providing option for hotel booking all over the world. Your requirement is to check availability of rooms in those hotels. Now, it means that you need to interact with multiple hotel applications at a time. It's not necessary that every hotel is using same standard or their application(server, programming language) may be deployed on the different application servers. At a same time, its not practical to write different applications which can talk to all different type of the application server. We need some standard based solution which can solve this problem. It's possible through the Web services.
It is possible because web services are sending message in the SOAP(Simple Object Access Protocol), based on the XML. XML is used to interchange data across any language, platform or network protocol.
Web services can be classified SOAP Based and REST.
SOAP based service JAX-RPC and JAX-WS (http://www.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc/index.html)
Web services can be developed
contract first - first write WSDL.
code first - first write code.
Now, lets talk how to start for the web services practically.
Simplest web service or hello world(JAXWS) can be written as follows:-
http://svn.apache.org/viewvc/cxf/trunk/distribution/src/main/release/samples/java_first_jaxws/
Message-Oriented Middleware (MOM)
JMS
Message Queue (Point-to-Point)
MOM is required to over come disadvantages of request-response style communication. Server need to be alive when client sends response. Client wait for response till server executes and respond back.
Request response - application will fail if server or client is down.
MOM - Either of the end point is not required to be on time you send the request message for the processing.
MOM is concept and JMS is specification on this concept. Many vendors have implementation of this specification e.g IBM has MQ, OpenJMS open source implementation, EMS from Tibco etc.
JMS specification has majorly two patterns. Pub/sub and ponin-to-point.
Pub/sub is topic, your application want to publish certain information to all the interested parties. e.g. dash board. (Stock application want to notify certain message to all the registered listeners).
Point-to-Point communication is done through message queue.
Business use case- think you have application e.g customer request to the customer care. Other side you have multiple customer care representatives and other side customers sometimes more than customer care representatives, at a time one and only one representative will get the request to be processed and he/she will not get next request until finishes the task. (Same one queue multiple windows and which ever window is free will process the request). You can think of other complexity in this e.g what if one of the node is failed, request not processed and particular type of request needs to be process by particular node. etc.
Produce code:- http://docs.oracle.com/javaee/1.4/tutorial/examples/jms/simple/src/SimpleProducer.java
Consumer synchronous code:- (POJO classes)
http://docs.oracle.com/javaee/1.4/tutorial/examples/jms/simple/src/SimpleSynchConsumer.java
http://www.java2s.com/Code/Java/J2EE/ThisexampleisasimpleJMSclientapplication.htm
Consume asynchronous code:- (Spring by example - reads message from destination till program will not be stopped.)
http://www.springbyexample.org/examples/simple-spring-jms-listener-config.html
Though, its just basic there are many aspects to be covered in this MOM e.g. what's fail-over mechanism, what is selector, durable message, message acknowledgement modes etc...
Service Bus/ESB
Endpoints & Routes
Apache Camel
Mule
Now, lets say you have adopted SOA and MOM long back and you have bunch of services which talks to each others to accomplish enterprise wide task. Imagine to manage logic such as multiple destination whom should be redirected from where will be very cumbersome. Some people call this application logic. Service buses will be used to reduce application logic and focus more on the business logic(functionality provided by app).
In Simple words, consider End point as URL exposed on server. You will use this url/end point to invoke your service.
e.g.
http://localhost:8888/Context/MyService?wsdl
in code:-
String endpointAddress = "http://localhost:8080/jaxws/services/hello_world?wsdl";
// Add a port to the Service
service.addPort(PORT_NAME, SOAPBinding.SOAP11HTTP_BINDING, endpointAddress);
HelloWorld hw = service.getPort(HelloWorld.class);
System.out.println(hw.sayHi("World"));
Routes
When service bus receives particular message, it will route it through no of services/broker destinations such as queue/topics. This path is known as route.
E.g your stock application has got some input by analyst, it will be processed through the application/web component and then result will be published to all the interested/registered members for particular stock update.
Apache Camel and Muel http://camel.apache.org/how-does-camel-compare-to-mule.html
provides solution for the enterprise integration.
Enterprise Integration Patterns can help you understand how everything fits together.
[update:] Your follow-up question on another answer made me realise that you're confused about specific products. That's partly because software in practice tends to map to more than one concept and partly because different companies argue that they provide "everything", when really they don't.
The ESBs are toolkits / libraries that let you connect everything together. They are neither the services themselves, nor the messaging implementations, but the goo that fills the odd little gaps in-between. If you were writing everything from scratch you might not even need one, because what they are best at is fixing the mismatch between a whole pile of different technologies, and if you are starting from scratch you can avoid that mess.
The services are, well, the services. You might use some EJBs when implementing one (I only mention this because for some reason you include them in your question).
The messaging middleware is software that gets messages from A to B. That's extremely useful, but also complex, and everyone and their brother has invented their own. So you need some abstraction that lets you avoid lock-in. That can be an ESB or, if you are all-Java then it can be JMS. But even when you are all-Java with JMS you may still want to use an ESB because they are libraries of all the bits of Java code you would still need to write (random bits of routing logic, message reformatting, etc etc).
Hope that helps. My original answer is more about the abstract patterns that you build with these tools - when you're wiring things together the same problems come up again and again.
Endpoints & Routes: where the information comes from and goes to.
Message Queue is one type of endpoint. The other type is a Message Topic.
An endpoint is a 'logical name for a thing', e.g. PRICE.MSFT, that is used by a publisher or consumer application to get things from or send to. Topics deliver information to everyone subscribed (one-to-one or one-to-many), queues deliver messages to the first one that gets it (usually one-to-one). Forget about Queues, everything can also be done with Topics and topics have several advantages.
Message-Oriented Middleware (MOM): the software infrastructure that delivers information between topics or queus. It is 'message-oriented' not 'packet oriented' as TCP is. So each information blob is delivered in one, hopefully self-describing, message. The implementation of your MOM then gives you an API where you can do things like msg.get("bid")
JMS and AMQP are examples of a MOM specifications.
MOM implementations are the real products that implement these specs:
TIBCO EMS, Websphere MQ, MSMQ, Solace, and many, many others
Apache Camel - very interesting approach on how to configure workflows in this MOM world. But a more advanced concept.
Service-Oriented Architecture (SOA), Service Bus/ESB are just new buzzwords words for what used to be called EAI (Enterprise Application Integration). They are recommendations on how to use 'MOM's and a way to pay high-priced consultants.
What 'ESB' adds to a MOM is the idea of thinking of your publishers as 'services' providing a service. In other words: don't think too much about what one consumer wants right now. There might be 5 consumers in the future and that publisher should provide a service, not 'create information that consumer A wants'. (It'll become clearer once your architecture has grown to 5+ applications). Also you should have some common object model, maybe in XML to make things simple between applications.
Mule - one form of ESB, but its' not exactly main-stream. In 5 years, most middleware action might have been moved to AMQP or something else entirely.
EJBs: Sun's idea of sophisticated Java classes that run in a container. Supposed to make application development easier. But in many cases it made things more complex. A better alternative would be 'Spring' - but EJB's are about something else (not just MOM). Its more about how to develop bigger applications (see IoC pattern).
If you are looking on where to start: I would recommend learning about JMS (all other MOM's are simliar and JMS is the basis of EJB's/ Mule, ...) and, unless you have super-high performance requirements, consider messages to be a TextMessage containing XML. Most tools are available in that area. Or even simpler but less sophisticated, a MapMessage with key/value pairs.
Taking all of your requirements and packaging them into a query, I came across an excellent case study that should meet your needs:
Service Oriented Architecture: An Integration Blueprint
I went ahead and fulltext searched the book using Amazon's "Search Inside This Book" feature. It covers all of the integration cases you've discussed, appears to be thorough, and steps you through the entire design and implementation process.
I'm embarrassed to state I haven't read through this myself, but I highly recommend using the same tools I did to see if it fits your needs before investing in a copy. It seems more thorough, more complete, and more helpful than simply foisting you on a whole lot of incomplete documentation or spooling out content into an answer here.
You mix a lot of different concepts and technologies with different abstraction levels. But all of your concepts have something to do with (enterprise) application integration. I will try to comment your definitions:
Service-Oriented Architecture (SOA)
SOA provides a set of principles and methodologies to integrate existing applications as loosely coupled units. From the Enterprise Integration Patterns (see below): "SOAs blur the line between integration and distributed aplications".
Service Bus/ESB
The ESB is a main concept of SOA to reduce the dependencies within the applications in a SOA. Instead of a lot of dependencies between the applications each application is connected to the ESB.
Message-Oriented Middleware (MOM)
MOM is a infrastructure for sending and receiving messages between distributed systems. This is used to integrate applications. MOM was the golden hammer before the SOA hype came up. Since both are useful, big integration suites provides both ESB and MOM (or use MOM inside their ESB).
Message Queue
A message queue is just a technical detail aspect in MOM architecture. When message sending/receiving is decoupled, message are stored in queues until the recipient is ready.
Apache Camel
When the book Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions hit the market, some software solutions have been created which provides implementation for the patterns in this book. Apache Camel is one of them. Camel is also a part of Apache ServiceMix which is also an open source ESB. FuseSource and Talend are packaging Apache ServiceMix, Apache Camel and Apache Active MQ (MOM) to bundles with commercial support.
Mule
Mule is also an open source ESB and integration platform.
EJBs
From Wikipedia: Enterprise JavaBeans (EJB) is a managed, server-side component architecture for modular construction of enterprise applications. This means EJB is a component within an application and has primary nothing to do with integrating applications.
Endpoints & Routes
When you work with Apache Camel you are designing routes between endpoints, see a tutorial. In short, message are entering/leaving your system via endpoints and are processed in a flow defined by a route.
JMS
JMS or Java Message Service is a Message Oriented Middleware (MOM) with an standardized Java API.
Enterprise Application Integration (EAI) is key to connecting business applications with heterogeneous systems. Over the years, architects of integration solutions have invented their own blend of patterns in a variety of ways. But most of these architectures have similarities, initiating a set of widely accepted standards in architecting integration patterns. Most of these standards are described in the Enterprise Integration Patterns Catalog available at: http://www.eaipatterns.com/toc.html.
WSO2 ESB
WSO2 Enterprise Service Bus (ESB) 4.7.0 documentation! WSO2 ESB is a fast, lightweight, 100% open source, and user-friendly ESB distributed under the Apache Software License v2.0. WSO2 ESB allows system administrators and developers to conveniently configure message routing, mediation, transformation, logging, task scheduling, failover, load balancing, and more. It supports the most commonly used Enterprise Integration Patterns (EIPs) and enables transport switching, eventing, rule-based mediation, and priority-based mediation for advanced integration requirements. The ESB runtime is designed to be completely asynchronous, non-blocking, and streaming based on the Apache Synapse mediation engine.
Related
I'm reading some documentation about the micro-services architecture (through this link for example) and I was wondering what is exactly a service in this case.
In IT, everything could be called a service:
- a SPRING REST application launched through the java command like:
java -jar build/libs/gs-rest-service-0.1.0.jar
It could also be a classes corresponding to the business layer in a DDD
It could be simply something related to the domain studied, like providing something to somebody
and many others... (android background running services etc...)
But in microservices, what does it mean? And what kind of technologies / tools are used to create a "service running by himself" in the Java EE stack for example? It's only related to webservices?
Exactly, that's the beauty of microservices model! You can start thinking about microservices when you design your maven multi-module project, for example. Low coupling, clear separation of concerns, may be even asynchronous communication. When you feel more confident you extract them in into apps and run in a one host, next step - run in different hosts. It's up to you to decide how exactly they should be deployed, it's related to goals you want to achieve (fault-tolerance vs low latency, etc.) and DevOps resources you have (because more separation you have more maintenance you need).
Regarding Java EE stack - nothing specific, just usual jar or war file running using java -jar or application servers like Tomcat.
Another direction is to use tools like Docker + CoreOs / kubernetes / ..., Mesos + Marathon, etc., but they are suitable for any languages / frameworks in microservices.
Edit:
Microservices can use a combination of synchronous (REST, SOAP) and asynchronous protocols (messaging queues like ActiveMQ, RabbitMQ, etc). It's up to you to decide how to combine them. My example: labs.bench.co/2014/12/10/microservices-at-bench-intro
Previous answers are great.
Microservices architecture is just a functional decomposition design.
I suggest you to read this blog post : Microservice Design Patterns
From a technical point of view, there is a a lot of tools like Docker (to run each microservice as a linux container) and Kubernetes to orchestrate them as a service (here is a Kubernetes sample).
My own definition:
A microservice is a stand-alone, decoupled component that handles a single business concern, and is consumable from other services.
Others might agree or disagree, and there is a lot of interesting discussion on this topic that make it a great study point for software engineers.
From a technical standpoint:
You can create microservices in almost any technology: Java EE, Java + Spring, Python, Rails, Grails, Node.js and so forth. From what I have seen, it seems most commonly applied in the domain of web apps and back-end service-oriented ecosystems. In the article you reference, the NetFlix model is a very interesting thing to study, because you can see all the elements of a microservice architecture in depth: service discovery, circuit-breaking, monitoring, dynamic configuration, and so on.
Some things you might want to check out, if you are Java-oriented:
Spring Cloud allows you to use some of these same NetFlix components with a minimum of hand-coding: http://cloud.spring.io/spring-cloud-netflix/
An actual operational example on github (not mine, but I have used it in my own learning on the topic): https://github.com/ewolff/microservice
From a conceptual point of view, your question hints at a notorious microservice design dilemma. There is not necessarily a "correct" level of granularity for a microservice. The idea is to choose a level of granularity that has meaning within your business domain. If you implement microservices at a very low level of granularity, (e.g. the CRUD level), then you will almost certainly end up with very chatty services and you will probably have to build more meaningful composite services over top. If you choose too high a level of granularity, you could end up with a more monolithic application which may require refactoring into microservice-sized pieces later.
I would start with Your last question - It's only related to webservices?
That's debatable. I would say, NO. It's related to webservice (but not only to it.)
Martin fowler describes microservices as a small subset of SOA, after all microservices are services, and SOA is a very generic and broad term.
Below are some of the important aspects of Microservices:
Each service (or a set of few) should have it's own data store.
Services are organized around the business needs or functionality.
Each service is independent so they can be implemented in any language. Leads to polyglot programming culture in team.
Service can take request from client or from other services as well.
They are usually event driven and asynchronous so scaling becomes easier.
Services are dumb as they only do one thing (but they should be self sufficient to monitor themselves)
They can be helpful in continuous deployment or delivery as implement to deploy cycle is really small.
They are very small so there is not much of network overhead in deploying them. So they can be deployed across a cluster of nodes in few minutes.
Also, I want to stress that, above are NOT only true about microservices. Companies like google, netflix, and Amazon have been doing similar thing even before the term was coined.
Service to microservice is Java is to JavaScript. Don't think about it that way. Instead microservice can be thought of as the following:
A small problem domain.
Built and deployed by itself.
Runs in its own process.
Integrates via well-known interfaces.
Owns its own data storage.
A Microservice is basically a self contained process that provides a unique and single business capability. We don't create web Microservice, business logic Microservice, or datebase Microservice.
Why Microservice?
Microservice make our system loosely coupled, i.e. if we need to update, repair, or replace a Microservice, we don't need to rebuild our entire application, just swap out the part that needs it.
To built each Microservice can use different languages and tools. Microservices communicate with well defined interface
The communication should be stateless for scalability(copies of Microservice) and reliability(one copy fail other copy can serve), the most common methods for communication between Microservices are HTTP and messaging.
Each Microservice should have it's own datastore.
Small team capable to work on design, web development, coding, database admin and operations.
source
Microservices is a software architectural style that require functional decomposition of an application.
Usually, it involves a monolithic application is broken down into multiple smaller services, each deployed in its own archive, and then composed as a single application using standard lightweight communication, such as REST over HTTP or some async communication (of course, at some point micro services are written from scratch).
The term “micro” in microservices is no indication of the line of code in the service, it only indicates the scope is limited to a single functionality.
Each service is fully autonomous and full-stack. Thus changing a service implementation has no impact to other services as they communicate using well-defined interfaces. There are several advantages of such an application, but its not a free lunch and requires a significant effort in NoOps.
It's important to focus on that that each service must have the properties of:
Single purpose — each service should focus on one single purpose and do it well.
Loose coupling — services know little about each other. A change to one service should not require changing the others. Communication between services should happen only through public service interfaces.
High cohesion — each service encapsulates all related behaviors and data together. If we need to build a new feature, all the changes should be localized to just one single service.
I have built an application in OSGi which offers a REST API for reading values from different data sources. Then I have several modules installed in my application exposing those data sources. For example one module exposes readings from a database, another exposes from a website, another reads from a TCP/IP device, etc...
For example, if I do this:
http://--------/listAllDataSources
I would get something like this if I had 3 modules installed:
{
dataSources : [ 0, 1, 2 ]
}
Then if I do this:
http://--------/read/1
I would get a reading from the data source number 1.
{
currentValue : "44.5"
}
So my architecture consists of:
[ External Apps]
|
|
*Web*
|
|
[ REST API ]
[M1][M2]....[Mn]
Is this considered a SOA?
Also, I do not use any service broker as you can see. Does this makes my app not a SOA?
Thanks!
I think you're mixing concepts here, PedroD. You can read in detail what the term SOA refers to here. (I'm intentionally not linking to the Wikipedia article, which is absolutely terrible and misleading).
Your application is built to expose functionality, so it can be said that it can be a part of SOA but it is not a SOA by itself.
SOA refers to an architectural style that supports service orientation. It helps an organisation integrate and develop its systems in a service oriented way. SOA is a paradigm much broader than what you have and it greatly exceeds the context of a single application - e.g. it establishes internal enterprise guidelines, governance processes, building of service inventories across multiple business domains and orchestration of services in order to enable the organisation to implement business processes very fast, thus reducing time to market.
Of course SOA has levels of evolution. You may not still have processes in place or you may not use a messaging broker, and it can be said you're still working towards a fully service-oriented architecture. But in order to achieve SOA you must have at the very minimum have some kind of middleware tier, so you can avoid point-to-point integration and do at least simple service compositions.
I'd say that your appliation is built with integration in mind, with service orientation in mind, but it is not SOA by itself.
Note: If you want to internally organise your application in a style that borrows many concepts from SOA you might want to check out Microservice Architecture. Actually to me, your architecture seems more similar to Microservices.
SOA is an abstract concept which similarly to 3-tier architecture can be applied on different levels.
If you consider "External Apps" as a part of your system then these apps are service consumers and "[Rest API] with [M1]..[Mn]" is a service provider. Communication is over http. This is SOA.
If scope of your system is only "[Rest API] with [M1]..[Mn]" then you still have consumer - [Rest API] ( which is also provider of external service), and providers [M1]..[Mn].
Each of this module is a service so I would consider this design as a very simple SOA. Service broker job is done by OSGI runtime, and communication protocol can be just Java method calls - however it can be simply replaced by ECF to distribute modules across different JVMs and then the communication protocol depends on what is configured in ECF.
My app requires bidirectional continuous communication with a high volume of clients (which are java agents installed on user machines) in addition it includes a spring mvc webapp which provides a standard jsp UI to manage these agents.
I’ve only looked at the basics of java akka (no time to learn scala for this project). But it seems like a good choice to handle the high volume of client agents. I’ve looked at akka spring integration module and akka-spring-java examples and using akka on the spring side seems pretty straight forward.
I thought using akka remoting with the client agent side might also be a good idea, the agent which will likely be embedded in another app basically runs a thread needs watch various processes in the user’s jvm and communicates with services on server. Using location transparency would simplify the architecture conceptually and possibly be more efficient.
This article suggests this may not be the correct approach
Peer-to-Peer vs. Client-Server
The alternative to using remoting would to use camel websockets which seem to be associated with the akka spring integration module.
What would be the best direction to take in the context of my app given it's tech stack?
You probably don't want to use remoting for a server-clients situation. Remoting gives both sides the same rights and privileges. It was designed for clusters and peer-to-peer.
Take a look at Akka I/O. It gives you asynchronous actors on both sides, but fits the server-client use case better. You wouldn't have to worry about threads and processes.
Also remember, that even when using Akka with Java you need the Scala library as a dependency.
Where, when and why did you use ESB in a PHP-project?
Where, when and why do you think that it would make sense to use ESB in a PHP-project?
Does ESB (and ESB-facilitators like Mule) do provide any capability PHP and native LAMP-technologies are lacking?
Edit
My motivation for this question is stemming from my assumption that you actually never really need Mule. Mule will facilitate communication with external services which you could handle without mule. At the end of the day also Mule will create costs and overhead. So my question is steering at having somebody tell me about scenarios where you really benefit of ESB and tools like Mule or to second my guess with solid knowledge.
Edit 2
regarding Houcem's reply to my comment to his post ... what would be a native LAMP-answer to ESB/Mule?
Edit 3
Seems like Tuxedo might be a more PHP-native alternative to Mule/ESB. Somebody got experience using this tool?
ESB could be used in different ways :
Providing asynchronous processing : Example : If you have a web site that does a lot of emailing .. and sending an email takes a lot of time which may block the execution of your page : you may use an ESB to send the email data to Mule and route it to an Email outbound channel, this way you can say that you have implemented a mail message queue. Another form of asynchronous processing : using mule to execute php scripts (using command line) in a non blocking way.
Integration with java applications : You can send Messages to mule using php and implement some java business using mule API in java, php messages will be received by your java business components. This is used in big web sites which does a lot of complex processing and need flexible and powerful language like java.
What you need to know : ESB should be used like a Bus which means collecting data from heterogenous environments in a standard form (Mule messages) .. do business logic and then output data (after routing) to different environment
There is no native php integration with Mule in PHP world. To do it you should use web services (SOAP)
An ESB is a general solution to a scalability problem; a problem in managing the overhead, cost and complexity of a large number of application interfaces. I wrote a short article about the rationale for ESB/EAI solutions at http://psicom.com.au/solutions/eai
Admittedly, most PHP sites are small scale and would usually find it difficult to justify the administrative and technological overhead of an ESB. But it is now quite feasible to meet all of a business's application needs with OSS PHP products, and there is also increasing cost pressure on organisations, so I expect there will be increasing numbers of PHP shops that will start to feel the kinds of growing pains that I wrote about. This may make them re-evaluate their application integration issues, and an ESB is a good solution to that problem.
As far as I know there are no ESB products developed in PHP, and I don't expect to see this in the near future. But FWIW, many ESB products provide bindings for PHP and other OSS platforms, so the platform that the ESB runs on isn't crucial.
The ESB (Enterprise Service Bus) is a kind of backbone for the integration of the several heterogeneous applications of the enterprise, that may originate from different vendors, technologies, and even be redundant.
The fact it seems more linked to Java world than PHP or any other language is that usually big companies information systems are made of :
Mix of opensource development and software editors tools (including ERPs). Development are often made using Java EE in order to rely on Java EE stacks (and its vendors IBM, Oracle, ...)
Full Microsoft (no need of ESB, Microsoft provide EAI/ESB like tools)
PHP is most used for web application (even for big companies, but web oriented).
An ESB is a big cost, and is only usefull/needed when the number of interconnected application is raising.
When you have only few connections (between Java, PHP or whatever), you can handle it at a network level, using DNS, and at application level using configuration keys and performing protocol exchanges and small business integration for each point to point connection.
A potential use case for a PHP application would be an internet site of a travel agency quering multiple flight/train/hotels companies. And even in a such case, it won't be mad to develop a full cross quering system since it is the core of the business for such a site/company.
I would recommend Windows Azure Service Bus, which provides a PHP SDK here
https://github.com/WindowsAzure/azure-sdk-for-php
service bus is great but maintaining one is not. Windows Azure Service Bus solved all the maintenance overhead for you, and it is PHP compatible. You can even communicate with apps that are written in Java, C#, VS C++ from PHP easily.
Where can I find some information on the uses and benefits of an enterprise service bus (ESB)?
I am looking for information about:
the kinds of problems and ESB helps to solve
the alternatives to an ESB - and the tradeoffs in selecting between them
what you need to do as a developer to build ESB-compatible systems
I'm looking for a finer level of detail than just Wikipedia or online marketing brochures from vendors. Ideally, some example code would help to clarify what's involved in taking advantage of an ESB. Information from a .NET or Java perspective would be the most useful.
Thanks.
I'd suggest To ESB or not to ESB to start with, written by the creator of Mule.
ESB's are a good way to implement Enterprise Integration Patterns.
Kinds of problems that an ESB helps to solve
You have a number of protocols you'd like to normalize to a single protocol (e.g. FTP, email, SOAP, XMPP, etc. to a messaging system) e.g. ActiveMQ. This lets you decouple the implementation of services from the protocol.
You want a consistent way to hook services into this architecture so that they can listen for messages, process messages and generate messages (Message Endpoints, Channel Adapters etc.).
You may want a managed container to deploy these various components into (e.g. ServiceMix, Mule)
You may want a number of prebuilt components and adapters into various protocols (e.g. ServiceMix, Mule and Camel have a lot of pre-built components).
You may require long running workflows. Business Process Management is often something that is provided alongside an ESB (Apache ODE plugs into a number of Open Source ESBs).
Alternatives to an ESB
The alternatives really depend on the problem that you're trying to solve.
To provide distributed services, people often use application servers exposing services via some point to point RPC protocol (like EJBs over RMI or Web Services over HTTP). So, rather than putting a message onto a 'bus', a client directly calls a server.
To respond to specific protocols, you could just build a client that responds to that protocol for example writing an application that listens for emails using JavaMail or one that listens to XMPP using Smack. If your problem is constrained to one or two protocols it may not be worth bringing in a full ESB.
What you need to do as a developer to build ESB-compatible systems
This will depend on the ESB you select, although given that most of the good ones are designed to call out into all sorts of protocols as well as host POJOs, there isn't much you should need to do build ESB compatible systems. It is worth trying to make your code asynchronous.
For examples, Apache Camel probably has the most succinct configuration, here's a tutorial.
Three key advantages:
A bus provides a way for end points to connect to each other without having to directly talk to each other. It simplifies the communications for the end points as they only have to conform to a standard communication interface, the bus. (This is with any technical bus, not just ESBs)
An ESB provides a single place to get some key end point metrics: frequency, availability, and performance.
An ESB tends to provide more than one communication interface. However, a developer only needs to choose the easiest one to get and receive the data from the bus.
However, make sure those will provide business value for your situation. Having an ESB is adding yet another complexity to your enterprise. Ideally you shouldn't choose this based on a few applications, but the entire organization. There should be only one production ESB cluster for the organization.
Alternatives:
Just connect things to each other directly especially if the communication protocols are the same. This is good for simple application clusters and does not require too much thinking. However, as your number of applications grow, maintaining the interconnections becomes difficult.
Another alternative is an MQ implementation. This will provide you with a way of pushing data around without having complex interconnections, but then you are forced to use only one communication channel. Fortunately for Java, they have the JMS standard.
Practicality:
I have stated the possible alternatives. They may seem lousy at first, but it is not to say you cannot start that way. I personally write things to talk to the remote directly without going through an ESB to make sure it works without worrying too much about integration issues.
If you don't have an ESB, I suggest you try Mule for development and WebSphere ESB for test and production. I tend to use two products that supposedly follow standards to make sure we keep the vendors honest and to make sure your developers are conforming to standards preventing inadvertent vendor lock-in.
In the end, just answer the following question: is the time adding the bit of complexity to simplify other complexities your enterprise worth the cost in the long run?
In addition to the sites that were already mentioned. You should read this article on "Don't use an ESB unless you absolutely have to". It was written by the CTO of MuleSource, one of the most popular open source ESB's available. Not exactly an answer to your question, more of making a point to ask yourself "Do I need an ESB"?
There is a decent 3-part series over at IBM regarding ESB that's pretty concept oriented and vendor agnostic (for the most part). I have found lots of good stuff on ESB by poking around IBM's site. There is also some decent info and videos and stuff over at the BizTalk site.
Check out this Hanselminutes podcast. It answers a few questions that you should really ask yourself before implementing a service bus.
An enterprise service bus (ESB) is a software architecture for middleware that provides fundamental services for more complex architectures. For example, an ESB incorporates the features required to implement a service-oriented architecture (SOA). In a general sense, an ESB can be thought of as a mechanism that manages access to applications and services (especially legacy versions) to present a single, simple, and consistent interface to end-users via Web- or forms-based client-side front ends.
In essence, ESB does for distributed heterogeneous back end services and applications and distributed heterogenous front-end users and information consumers what middleware is really supposed to do: hide complexity, simplify access, allow developers to use generic, canonical forms of query, access and interaction, handling the complex details in the background. The key to ESB's appeal, and possibly also its future success, lies in its ability to support incremental service and application integration as driven by business requirements, not as governed by available technology.
http://searchsoa.techtarget.com/definition/enterprise-service-bus
WSO2 Enterprise Service Bus(Product)
WSO2 Enterprise Service Bus (ESB) 4.7.0 documentation! WSO2 ESB is a fast, lightweight, 100% open source, and user-friendly ESB distributed under the Apache Software License v2.0. WSO2 ESB allows system administrators and developers to conveniently configure message routing, mediation, transformation, logging, task scheduling, failover, load balancing, and more. It supports the most commonly used Enterprise Integration Patterns (EIPs) and enables transport switching, eventing, rule-based mediation, and priority-based mediation for advanced integration requirements. The ESB runtime is designed to be completely asynchronous, non-blocking, and streaming based on the Apache Synapse mediation engine.
WSO2 ESB is developed on top of the revolutionary WSO2 Carbon platform, an OSGi-based framework that provides seamless modularity to your SOA via componentization. It includes many features and optional components (add-ons) you can install in the ESB, and you can easily remove features not required in your environment, thereby allowing you to fully customize and tailor WSO2 ESB to meet your exact SOA needs.
Architecture
Application infrastructure on the enterprises may be inherently complex, comprising hundreds of applications with completely different semantics. Some of these applications are custom built, some are acquired from third parties, and some can be a combination of both and can be operating in different system environments.
Integration among these heterogeneous applications is vital to the enterprise. Different services may be using different data formats and communication protocols. Physical locations of services can change arbitrarily. All these constraints mean your applications are still tightly coupled together. An ESB can be used to loosen these couplings between different services and service consumers.
WSO2 ESB is a full-fledged, enterprise-ready ESB. It is built on the Apache Synapse project, which is built using the Apache Axis2 project. All the components are built as OSGi bundles.
Take a look at my presentation "Spoilt for Choice - How to choose the right ESB".
I explain when to use an ESB, Integration Suite, or just an Integration Framework (such as Apache Camel). I also discuss the differences between open source and proprietary ESBs.
there is zero reason to use an ESB. Don't do it. Needless complexity. Why go through an intermediary when you can go direct? The ESB folks will tell you point to point is bad, yet somehow point to point to and from the ESB is good.
The first question you need to ask yourself is why do you need an ESB?
ESB is usually used in Event SOA distributed architectures, which seem to be a hot buzzword nowadays. Before you jump into ESB let me remind you of Martin's Fowler First Law of distributing Systems:
http://martinfowler.com/bliki/FirstLaw.html
"My First Law of Distributed Object Design: Don't distribute your objects (From P of EAA).
The relevant chapter is available online."
When you're build a new system, the most important aspect is that it is future proof, that means easy scalability and maintainability. If you build your system around the concept of loosed services with static defined contract distributed in a networked environment, you can "hide" the architecture you want for that particular service, because the interfaces are still there.
ESB is close related to asyn messaging systems, so before you start jumping into that kind of implementation, know that an architecture does not have to be homogeneous, that is all services be implemented the same way, don´t start the biggest mistake which is distributing your system from the start, you should only distribute as you need to scale, no before hand. What you need to make sure though, is that your services should be able to be easily distributed should the need arise, without breaking any contracts which would mean, changes to clients of that service.
As for the benefits of ESB, they are the same as SOA, ESB adds the context of asyn messages (events) operations.
First let me explain SOA. It it about building an architecture as a set of reusable software modules exposed as “Services” with well defined interfaces. The services facilitate loose coupling and abstracts its implementation details from the clients.
SOA might end-up messy if each component called on services directly. Therefore it has following common issues.
How do you find which services are used & which are not?
How do you find the clients using a particular service?
How do you roll out updates to a service or expose new versions to existing services without disruption?
How do you support backwards compatibility for older clients invoking older service interfaces
How do you perform logging, auditing, security enforcement etc across all services for internal / external traffic?
The ESB is the solution to above issues. ESB…
Helps brings in order
Can strictly enforce corporate policies
e.g. Security, throttling, auditing etc applied consistently
Virtualizes service endpoints
Facilitate versioning, flexible updates, HA / Load Balancing etc.
Perform routing, mediation, transformation, etc
Some sample use cases can be found here. Note that they are from AdroitLogic's developer site and strictly coupled with UltraESB, AdroitLogic's ESB.