I am considering to develop an web application with Spring and OSGi. It seems like they fit together nicely. What are the options for clustering and loadbalancing such an app and what are the pros and cons of each?
Neither OSGi nor Spring were created to solve problems such as high-availability, clustering or load-balancing. You could, of course, build a clustered and load-balanced system using Spring and OSGi, but you are probably going to need something else as well, such as a way to detect and communicate node failures and load levels.
Since you are building a web application, most likely you will be using one of the many application servers. Good AS products provide clustering for you. Some also provide load balancing. You can also achieve load balancing through a completely independent setup of your own, using Apache for example to front your main application servers.
If you are really bent on creating your own solution, I have seen JGroups being used in multiple products to provide the necessary infrastructure to build a clustering and/or load-balancing solution. Some of the distributed in-memory caching products use JGroups, for example.
Talking about distributed caches, products such as Ehcache can help with scaling and load-balancing problems.
Related
I have been testing out microservices lately using springboot to create microservice projects. The more i understand about the setup the more questions i am confronted with.
How are all the microservices that are running, managed? How do developers manage, deploy or update microservices via a central location?
When deploying multiple instances of a microservice, do you leave the port to be decided during runtime, or should it be predefined?
I am sure there will be much more questions popping up later.
Links used:
http://www.springboottutorial.com/creating-microservices-with-spring-boot-part-1-getting-started
https://fernandoabcampos.wordpress.com/2016/02/04/microservice-architecture-step-by-step-tutorial/
Thanks in advance.
Microservices do tend to go out of control sooner than later. With so many services floating around, you need to think of deployment and monitoring strategies ahead of time.
Both of these are not an easy problem, but you have quite a few tools available at your disposal.
Start with CI/CD. Search around it and you will find a way around. One option is to make use of Jenkins for Blue/Green deployments
In this case jenkins will be one central place where you manage your deployments (but this is just an example, we do have quite a lot of tools build around this that may help you better based on your needs)
Other part of this problem lies in when where you tend to deploy stuff? Different cloud providers have their own specific ways of handling microservices and it depends on your host really. But one alternative is to make use containers.
If you go with raw containers like dockers directly you will have to take care of mapping ports (if they are deployed on same host machine) but then you can use abstraction on top of this like if you are on AWS then you can consider ECS or docker swarms or I personally prefer Kubernetes. You do not need to worry about the ports on which they are and can directly talk to your service over a load balancer. There is lot that is missing in here and you really need to pick one such tool and dig deep, but there are options out there for you to explore.
Next is monitoring, if you are going with kubernetes, you do get lot of monitoring tools out of the box that will help you access the service logs query them etc. But you also need to make sure that from development perspective you do provide for correlations id's, api metrics, response times, because you will need them to debug issues when its comes to microservices specially one related with latencies. If you are not on kubernetes you can still get all these features added but individually, like ELK stack for log monitoring (as you do not want to go to each service to check for logs), zipkin for tracking , API gateway and loadbalancers for service discovery and talking to containers.
Hope this helps you get started.
you can start with the following:
Monitoring :
Start with spring-boot-admin and prometheus.
https://github.com/codecentric/spring-boot-admin
Deployment:
Start with docker and docker-compose and move to kubernetes.
Few examples for docker compose:
https://github.com/jinternals/spring-cloud-stream
https://github.com/jinternals/spring-micrometer-demo
There are container services/container management systems available for example Amazon ECS, Azure container services, Kubernetes etc which take care of automated deployment by centralised repositories like Amazon ECR etc, automated scale up/down of microservice instances, take advantage of dynamic port allocations to run multiple instance of same service on a single instance/host and also give you a centralised dashboard to monitor resource usage and infrastructure events.
You can make use of any one to get answers to all of your questions as all of them provide most of the functionalities needed for managing your microservices.
We have a quite large monolithic app (java/spring) and we are considering splitting it up to microservices and using spring-cloud to utilize existing solution for some common problems (discovery, redundancy etc.). Currently we run one instance (with different modules) per client.
Some of our clients are small and one VPS handles it and others are larger and might use multiple servers.
The problem is that this "pack" of microservices should be isolated for each environment - they might be slightly different.
As I am reading through resources about Cloud Foundry - which looks really great - it seems that it would be best to run an cloud foundry instance per client and I am afraid that that is overkill and quite a lot of work to get one client running (which I would like to automate as much as possible).
Ideal Solution
BEGIN
We provide servers with heterogenous OS, possible containers (VM/docker/jail/...) with restrictions where they may rur and finally services with restrictions in which containers they may run.
When creating new environment I just provide list of services to run in it and the Solution creates containers, deploys services in them and sets up communication channels (message broker) between them.
It should also handle upgrades, monitoring, etc.
END
What approach would you recommend? Or please could you share your experience from building similar thing?
Thanks
You could provide each customer with their own space in a single CF instance where all the microservices are deployed.
I'm relatively new to Java EE and have already began to hear about the many different types of systems that can be clustered:
Virtual Machines (i.e. "that appliance is a cluster of VMs...")
Application servers, such as Tomcat, JBoss or GlassFish (i.e. "We're running clustered JBoss...")
Clustering APIs like Terracotta
Databases, like Oracle ("clustered database")
Cloud applications ("A cloud is basically a cluster...")
Wikipedia defines "clustering" as:
A computer cluster consists of a set of loosely connected computers that work together so that in many respects they can be viewed as a single system.
I'm wondering how clustering works for each of these "cluster types/methods" (mentioned above) and how they relate to one another.
For instance, if one could benefit from having a clustered application, he/she would probably put them on a clustered app server and then throw a cluster manager into the mix (again, like Terracotta).
But because the phrase "clustering" seems to be used in vague/ambiguous ways, I'm not seeing how each of these ties into the others ones, or if they even do. Thanks in advance to any brave StackOverflowers out there who can help me make sense of this interwoven terminology!
To me, clustering implies a number of qualities to a system but it boils down to fault tolerance -- server, networking, and data persistence. There are both loosely and tightly coupled systems and all flavors in between. Tightly coupled systems have the clustering performed at level close to the hardware. Many of the old clustering systems were more tightly coupled with the applications often not recognizing that they were clustered.
Loosely coupled systems are the norm these days with a large degree of the fault tolerance accomplished at a software level entirely. Systems in the cluster only share network connectivity to be able to accomplish fault tolerance. Usually there are some specialized load balancers which route requests to the various cluster servers using specialized hardware (sometimes just software) to accomplish this.
All of the examples you mentioned have some sort of "clustering". It is going to take a very long answer to describe the details about how each of the architectures accomplish this. For me, the differences are what comes "for free" when you use the architecture, and how much work you will have to do to get it to work optimally.
How you mix and match the solutions you've mentioned depends on what your architecture looks like and your requirements. You can have a Terracotta store for local high speed persistence and the cloud for the rest. You can use Glassfish as your application server and utilize Terracotta as your persistence layer.
Here are my thoughts about the technologies you listed:
Cloud applications ("A cloud is basically a cluster...")
Cloud applications are the easiest to work with obviously. Your only job from an architecture standpoint is to pick a good cluster provider. Certainly Amazon and Google will do it "right" in terms of fault tolerance and data integrity. There are many other players that probably do it "good enough" and are cheaper. You program to their APIs which come with their own set of limitations and expenses. One problem with cloud applications is that it most likely will be very hard to switch to a new one. Again, you might have some [large] portion of your application running on cloud servers and have some local systems for your higher latency requirements. The trend is to put most production functions in the cloud or at least start that way until you get too big or need some services they can't provide.
Clustering APIs like Terracotta
Databases, like Oracle ("clustered database")
JBoss
These 3 systems provide their own clustering capabilities. They may require you to do a lot of machine and service layer configurations to get the system running well in a production environment. I hear good things about Terracotta which is a distributed persistence layer. I've used Jgroups a lot which is under Jboss and it can be tricky to get running right but Jboss may also have some good default configurations/documentation. Oracle is most likely going to be hardest to cluster right. DBA's make a lot of money tweaking Oracle configurations.
Virtual Machines (i.e. "that appliance is a cluster of VMs...")
Application servers, such as Tomcat, GlassFish
These are the most amorphous to define in terms of clustering. Some VMs are considered "clustered" in that they share networking hardware and power backplanes but are really not clusters when compared to cloud computing certainly. As mentioned, there are some clustered hardware solutions that are very custom and will require a lot of specific domain knowledge to get running well.
I have very little experience with application servers such as Tomcat and Glassfish. We have our own clustering software on top of Jgroups and run Jetty entirely. Application servers are not, in themselves, "clustered" but packages such as Jboss and Terracotta run on top of them to provide clustering and they have internal projects which have clustering software written for them.
Hope some of this helps.
Here's a quick whack at it. How you cluster depends on what your goals are. Here are some thoughts that also tie in to GlassFish.
A cluster enables multiple instances to be managed as one since they share a common configuration. If you make a change to a configuration, such as defining a new resource, then all instances that belong to a cluster inherit that change. Deploying an application to a cluster deploys it to all instances of that cluster.
A cluster provides service availability. If one instance fails, deployed applications are still available on other instances.
A cluster can offer session availability. If an instance dies while a user has items in their shopping cart, then another instance can take ownership of handling that user's session such that the shopping cart contents are still there. The user never knows a backend server has failed.
With GlassFish, HTTP session state can be managed by GlassFish (built-in), delegated to a coherence grid, or the application can manage state itself (using terracotta, database, etc). The benefit of using the built-in capability is that it works out of the box and has gone through stress testing, QA, etc. The benefit of externalizing is that you can potentially get better scalability since you decouple session management and application logic. Externalizing lets the JVM focus on executing business logic, and uses less HEAP space since backup sessions exist elsewhere. Oracle has tested / QA'd externalizing to the Coherence Grid, and is a formal feature of the commercial Oracle GlassFish Server. If you roll your own via database, then you need to manage & QA itH yourself.
We are considering utilizing distributed OSGi in our enterprise environment.
We would have the following setup:
10 to 100 OSGi containers on many hosts offer various services.
Many of these services are provided by more than one container.
Some of these services may require being consistent across all
containers (same version deployed).
What is the proper way to manage bundles' lifecycle (install, start, update, stop, uninstall) across all containers?
Several requirements:
As there may be so many containers, all of them should be handled
together; i.e. when I am about to update a bundle, a single command
should update all containers where that bundle already present.
Commands must be repeatable: first perform a command on test systems, and then repeat the exact same command on production system once testing is complete.
I appreciate any suggestion regarding the above question.
Best regards,
Marton
You might want to have a look at more "managed" solutions made for cloud-like environments: Apache ACE or its bigger brother Amdatu.
Apache ACE turns a single OSGi Containers into managed containers whose state can be controlled from a single administration point.
Amdatu is a more complete framework that includes ACE for provisioning but adds horizontal functionality.
In terms of managing large numbers of bundles, look at Karaf features - they greatly simplify handling largely suites of bundles.
For the distributed side of things, take a look at the Karaf subproject Cellar, it's clustering Karaf using HazelCast (and it installs in Karaf via the features mechanism).
If you are serious about enterprise ready - meaning "robust" - distributed OSGi runtimes - then have a look at the Paremus Service Fabric. We've been doing this since 2005 :)
The Service Fabric's provisioning / management architecture is extremely scalable (>> 1,000's of containers), responsive and stable! Several years development and commercial runtime experience underpinning this product. The Service Fabric architecture supports multiple concurrent distributed OSGi based applications. The Service Fabric's architecture is OBR centric; our lead engineer was responsible recent OSGi Alliance OBR specification activities.
The Service Fabric message backplane leverages the DDS messaging protocol - which IMO is the natural partner for distributed OSGi. The Paremus RSA (remote service implementation) is a clean room implementation of the standard - which is highly robust, light weight and allows dynamic plug ability of protocol and distribution providers. The default discovery provider - is again DDS.
Finally - and Service Fabric works out of the box.
In Gyrex we use p2 to distribute bundles across different nodes of a cluster. It's possible to control which bundles should be provisioned to which node by using the node tags and LDAP filter expressions (common in OSGi). For example, it's possible to install (and activate) web bundles only on web nodes.
Support for distributed services is not available out-of-the box. ECF needs to be integrated manually.
http://en.wikipedia.org/wiki/Eureka_Streams
http://www.eurekastreams.org/
Eureka Streams uses a shared nothing architecture. It uses Memcached, Apache Maven, PostgreSQL and Hibernate. It uses Shindig for OpenSocial.
It makes use of Java Message Service (JMS), Java Persistence API (JPA), Lucene and Google Web Toolkit (GWT). It makes use of the Apache JServ Protocol (AJP), OAuth and Representational State Transfer (REST).
The tech sounds solid as hell, and it was developed by Lockheed Martin, but I just want to be sure before I commit to anything that this "enterprise" solution will function as fantastically as it sounds outside of a single-node intranet environment.
Thoughts?
Edit: As stated in the title, I'm specifically concerned with scalability and security.
We built Eureka to scale to enterprise sized populations. We're currently deployed on about 40,000 employees and are soon going to have to scale to over 100,000. We've run performance tests with these points in mind.
To scale to something Facebook sized we'd probably have to start using something like Cassandra. That said, we've made the architecture robust enough to be able to support switching out data sources, so if this ever had to be done, it wouldn't be a rewrite.
As for security we use Spring Security with a few default authentication mechanisms including a straight Forms auth provider, ldap and kerberos. You can create new authentication mechanisms pretty easily.
The OpenSocial container part of Eureka Streams has an OAuth implementation allowing OpenSocial applications to connect to external resources using OAuth 1.0a.
Feel free to post in the Google Groups discussion with any specific technical or other questions:
http://groups.google.com/group/eureka-streams-dev
To answer your question about scalability, we build our front-end data objects asynchronously and store them in the memcache distributed in-memory caching system. Search is distributed too in that each web node has a fully copy of the search engine. Every search is performed locally, and as you create more web front-ends, you're also scaling search.