I'm looking into hosting a standard Java web app on AWS and the new Elastic Beanstalk (http://aws.amazon.com/elasticbeanstalk/) seems to have most of what we want. The one thing I can't figure out is how to do distributed caching. It seems that AWS doesn't allow multicast discovery of new nodes, so I'm not sure how new nodes started by the auto-scaling process should be integrated into an existing distributed cache. Any suggestions / best practices appreciated.
Update: Ideally this would be a cache local to each application server instance. Best case scenario would be a hibernate level 2 cache config for something like ehcache or terracota.
Another route to go down today (after 2011-08-23) is to use Amazon ElastiCache which is protocol-compliant with Memcached and runs in the cloud for you. Makes it easy to put things into an in-memory cache.
Here is some of my thought:
Suppose you have distributed cacheing instance such as memcached running in some EC2 instances and you use the Elastic IP from AWS to map to these instances. As Elastic IP is sort of static IP address, now you can pre-configure your new web app instances to locate the memcached istanced through the memcached IP
During auto-scaling, now it is possible to locate your memcached servers.
If you want your cache on every instance separatly, I would recommend to use Multicontainer Docker Environments for EB, as a way to facilitate settign your app, and your casching layer on every node. Elastic Beanstalk part will work just like before on normal dedicated platform, some reconfiguration may be needed there if you are using private libs, etc. depending on your app details. But if you want your caching layers on separate nodes to talk to each other it may be not easy to achive...
Related
In cloud foundry all apps are accessible via cloud foundry load balancers.
Each load balancer has an url under which to call the app.
Even though there is a hidden way (e.g. X-CF-APP-INSTANCE),
it is not meant to call an app instance directly.
Eureka is highly available and partition tolerant, but it lacks of consistency (CAP theorem).
To overcome the staleness of registry data, Netflix uses client side load balancing (Ribbon).
(see: https://github.com/Netflix/eureka/wiki/Eureka-2.0-Architecture-Overview)
Because an app in cloud foundry is called via its load balancer, it registers itself with the
address of the load balancer at Eureka. As stated before, the app instance does not even have an address.
To make it more visible, let's say there are two instances of an app A (a1 and a2) that register themself at Eureka.
Eureka now has two entries for the app A but both having the same address assigned.
Now when Ribbon takes place to overcome the consistency problem of Eureka, it is very likely that a retry is directed to the same instance as the first try.
Failover using Ribbon is therefore not working with Eureka in cloud foundry.
The fact that all instances of an app are assigned to the same address in Eureka, makes things complicated in many situations.
Even the replication of Eureka we could solve with a workaround only. Turbine needs to be fed in push mode etc.
We are thinking about enhancing Eureka so that it sets the X-CF-APP-INSTANCE header.
Now before doing that, we wanted to know whether someone knows an easier way to make Eureka really work on cloud foundry?
This question is realated to: Deploying Eureka/ribbon code to Cloud Foundry
I think Eureka, and any other discovery service, has no place in a PaaS like CloudFoundry. Although it sounds appealing to enhance Eureka to support the X-CF-APP-INSTANCE header, you would also need to enhance the client part (Ribbon) to take advantage of that information, and add that header to each request.
Well, it's 9 months later, maybe you have done that in the meantime? Or you follow an alternative solution path?
Anyway, there's an additional app to app integration option in the meantime, the container to container networking. And even here, the CloudFoundry dev team decided to provide their own discovery mechanism.
My project is looking to deploy a new j2ee application to Amazon's cloud. ElasticBeanstalk supports Tomcat apps, which seems perfect. Are there any particular design considerations to keep in mind when writing said app that might differ from just a standalone tomcat on a server?
For example, I understand that the server is meant to scale automatically. Is this like a cluster? Our application framework tends to like to stick state in the HttpSession, is that a problem? Or when it says it scales automatically, does that just mean memory and CPU?
Automatic scaling on AWS is done via adding more servers, not adding more CPU/RAM. You can add more CPU/RAM manually, but it requires shutting down the server for a minute to make the change, and then configuring any software running on the server to take advantage of the added RAM, so that's not the way automatic scaling is done.
Elastic Beanstalk is basically a management interface for Amazon EC2 servers, Elastic Load Balancers and Auto Scaling Groups. It sets all that up for you and provides a convenient way of deploying new versions of your application easily. Elastic Beanstalk will create EC2 servers behind an Elastic Load Balancer and use an Auto Scaling configuration to add more servers as your application load increases. It handles adding the servers to the load balancer when they are ready to receive traffic, and removing them from the load balancer and deleting the extra servers when they are no longer needed.
For your Java application running on Tomcat you have a few options to handle horizontal scaling well. You can enable sticky sessions on the Load Balancer so that all requests from a specific user will go to the same server, thus keeping the HttpSession tied to the user. The main problem with this is that if a server is removed from the pool you may lose some HttpSessions and cause any users that were "stuck" to that server to be logged out of your application. The solution to this is to configure your Tomcat instances to store sessions in a shared location. There are Tomcat session store implementations out there that work with AWS services like ElastiCache (Redis) and DynamoDB. I would recommend using one of those, probably the Redis implementation if you aren't already familiar with DynamoDB.
Another consideration for moving a Java application to AWS is that you cannot use any tools or libraries that rely on multi-cast. You may not be using multi-cast for anything, but in my experience every Java app I've had to migrate to AWS relied on multi-cast for clustering and I had to modify it to use a different clustering method.
Also, for a successful migration to AWS I suggest you read up a bit on VPCs, private IP versus public IP, and Security Groups. A solid understanding of those topics is key to setting up your network so that your web servers can communicate with your DB and cache servers in a secure and performant manner.
I have a typical stateless Java application which provides a REST API and performs updates (CRUD) in a Postgresql Database.
However the number of clients is growing and I feel the need to
Increase redundancy, so that if one fails another takes place
For this I will probably need a load balancer?
Increase response speed by not flooding the network and the CPU of just one server (however how will the load balancer not get flooded?)
Maybe I will need to distribute the Database?
I want to be able to update my app seamlessly (I have seen a thingy called kubernetes doing this): Kill each redundant node one by one and immediately replace it with an updated version
My app also stores some image files, which grow fast in disk size, I need to be able to distribute them
All of this must be backup-able
This is the diagram of what I have now (both Java app and DB are on the same server):
What is the best/correct way of scaling this?
Thanks!
Web Servers:
Run your app on multiple servers, behind a load balancer. Use AWS Elastic Beanstalk or roll your own solution with EC2 + Autoscaling Groups + ELB.
You mentioned a concern about "flooding" of the load balancer, but if you use Amazon's Elastic Load Balancer service it will scale automatically to handle whatever traffic you get so that you don't need to worry about this concern.
Database Servers:
Move your database to RDS and enable multi-az fail-over. This will create a hot-standby server that your database will automatically fail-over to if there are issues with your primary server. Optionally add read replicas to scale-out your database capacity.
Start caching your database queries in Redis if you aren't already. There are plugins out there to do this with Hibernate fairly easily. This will take a huge load off your database servers if your app performs the same queries regularly. Use AWS ElastiCache or RedisLabs for your Redis server(s).
Images:
Stop storing your image files on your web servers! That creates lots of scalability issues. Move those to S3 and serve them directly from S3. S3 gives you unlimited storage space, automated backups, and the ability to serve the images directly from S3 which reduces the load on your web servers.
Deployments:
There are so many solutions here that it just becomes a question about which method someone prefers. If you use Elastic Beanstalk then it provides a solution for deployments. If you don't use EB, then there are hundreds of solutions to pick from. I'd recommend designing your environment first, then choosing an automated deployment solution that will work with the environment you have designed.
Backups:
If you do this right you shouldn't have much on your web servers to backup. With Elastic Beanstalk all you will need in order to rebuild your web servers is the code and configuration files you have checked into Git. If you end up having to backup EC2 servers you will want to look into EBS snapshots.
For database backups, RDS will perform a daily backup automatically. If you want backups outside RDS you can schedule those yourself using pg_dump with a cron job.
For images, you can enable S3 versioning and multi-region replication.
CDN:
You didn't mention this, but you should look into a CDN. This will allow your application to be served faster while reducing the load on your servers. AWS provides the CloudFront CDN, and I would also recommend looking at CloudFlare.
I have built a mobile application that needs to connect to my SpringBoot-WebApp which in turn has a MongoDB and some other things in the background.
I want to deploy this WebApp at Amazon AWS, but I am overwhelmed by all the possibilities. So far, I have just created it as a .jar, and ran it that way, and it worked fine at my other server.
Now, for traffic reasons and such, we want to move it to AWS. I have found out, that I need to create a .war instead of a .jar, which is not a problem. I then learned to upload this .war to Elastic Beanstalk. However, my application needs to connect to a MongoDB. I have logged on to AWS via SSH and installed MongoDB there and created the database, but it does not seem like this is the right way to do it.
It'd greatly appreciate if anyone could give me a hint on how to do this as I am very confused.
Thanks and best regards!
It isn't clear if you are doing this, but don't run MongoDB on Elastic Beanstalk. The Elastic Beanstalk server you have it installed on may be automatically deleted by AWS. In general you do not want to manually install anything on Elastic Beanstalk as it is a managed environment where servers may be automatically created or deleted based on server load.
Amazon doesn't provide a MongoDB service directly, so you either need to install and manage MongoDB on an EC2 instance (or fleet of instances) yourself, or use a third party MongoDB service that runs on AWS. You could use something like MongoLab which provides a MongoDB service that runs on AWS. This allows your network traffic between your web servers and database servers to stay within the AWS network, which you will want for both performance and security reasons.
If you use MongoLab just make sure you choose to create your database in the same AWS region that you are deploying your application to. Also, I wouldn't recommend their free sandbox databases for any sort of critical production application.
If you decide to install and manage MongoDB on AWS yourself, here is some documentation from Amazon, and some from MongoDB.
I've an app which is deployed on to a cluster with 2 jvms. The web application has cache implemented using Mbeans and the cache runs on each jvm. the cache is refreshed with a request pattern */refresh. The problem is that when the request goes through ODR, it routes it to only one server and the cache for only one server is refreshed. How do I solve this problem? Cache replication? I think it might be lot of work to implement cache replication. Any other solutions? Websphere api's ?
if I get the current instance of the application, I'm thinking of using AdminClient to get the clusters and then call the request on all the nodes on which the application is installed except for the current instance.
The Websphere way to do this is to use the DynaCache feature with DRS. The DynaCache is a kind of a hashmap, which can be distributed across the DRS cluster members. The dynacache has an API, DistributedMap, which extends the java.util.Map.
There are also a lot of configuration (Through AdminConsole and cachespec.xml) and monitoring possibilities (PMI with TPV).
Technical overview:
http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/index.jsp?topic=%2Fliaag%2Fcache%2Fpubwasdynacachoverview.htm
DistributedMap API
http://pic.dhe.ibm.com/infocenter/adiehelp/v5r1m1/index.jsp?topic=%2Fcom.ibm.wasee.doc%2Finfo%2Fee%2Fjavadoc%2Fee%2Fcom%2Fibm%2Fwebsphere%2Fcache%2FDistributedMap.html
A good article from developerworks
http://www.ibm.com/developerworks/websphere/library/techarticles/0906_salvarinov/0906_salvarinov.html
The crude way we did something similar was to directly hit each Web Container on its own port. If you're able to reach them, that is.