I have a web application (spring based war) which is deployed in a Tomcat webserver. This web application is served by several server instances each running an instance of Tomcat. I intend to cache some data on a Redis datastore and all application instances contact this datastore to read data. As a pre-step I want some of the data to be cached up in Redis when the application starts.
If I do it via the web-app, all of the nodes will try to initialize the cache. Making one of the instances leader is one of the options, are there any better solutions to this?
Restarting: Indicates : Stop tomcat and then start it again. It could be done for several reasons: deploying a new version of the web app / server (machine) restart / new server being added to the pool. It's unlikely that all of the tomcat instances would be started at the same time but some of them may be started at the near same time.
Cache server is independent of the web-app, but in case it also crashed and the data was lost. I will maintain the "last read" TS in the Cache as well.
You have at least two options here:
Either you warm your cache from inside the application
Or you do it outside
Either approach has its own properties and from here on you can derive a solution.
Warming inside the application
You have inside your application all infrastructure to perform cache warming. If cache warming is idempotent, then it could be fine to have it done by all your instances. If you don't want to deal all of your applications with cache warming, then you need to make a decision:
What will other instances do while the cache is warming?
I have two answers to that question:
Ignore and continue startup
Wait until the cache is warmed
In both cases, you can use Redis CAS (Compare-and-swap) with timeouts to create an expiring, distributed lock so the fastest starting instance will do warming and release the lock once it is done.
You could tackle that challenge also with a process that you deploy the first instance to your servers which perform cache warming. You wait until that application has finished its job and then you're good to go for the other instances.
Warming outside the application
With cache warming outside the application, you don't run into concurrency issues, but that requires some effort on the operating and development side. You need someone (or some process) that runs the cache warming before your application start/deployment. You also need to build that piece of code that will access your data and put it into the cache.
Building leader patterns would also work but require additional code/components. If you can, keep it simple.
HTH, Mark
Related
Is it considered bad practice to use separated, local cache for each node in distributed microservice application? I've heard that in monolithic application it's OK to use local EHCache as 2nd level cache provider for Hibernate, but in distributed environment it's common practice to use distributed caches, such as Memcached, Redis or Hazelcast. What are the consequences of using separated cache for each node?
"There are only two hard problems in Computer Science:cache invalidation and naming things."-- Phil Karlton
The main problem with local cache in app-server is that it makes cache invalidation much more hard that it was before.
Each time a resource change, it has to be invalidated (and updated) on all the local caches. This would require a system that knows about all the cache servers running at any point of time. This system would have to be informed about all updates so that it can co-ordinate the data invalidation on all servers. It will also have to take care of retries, handling failed servers, etc.
If your application server has it's own local cache, you will have to solve these problems yourselves using a separate system or in the application code. A distributed caching system, would have solved those problems for you. You can make an update call and on success have a guarantee of data consistency (or eventual consistency).
It is separation of concerns. With a separate cache cluster, the caching logic and the associated problems are handled at one place. The same cluster can be reused for multiple applications easily, rather than redoing the same for each application you develop.
Another minor disadvantage is that you would have to warm up the cache each time you spawn a new server, if you don't want a performance degradation. This would lead to longer time to spawn servers.
Here we can do one more thing is to use message broker for cache invalidation.
Use kafka or any other queue to catch packets and invalidate them.
Please note: if the cache systems mentioned in this question work so completely differently from one another that an answer to this question is nearly-impossible, then I would simplify this question down to anything that is just JCache (JSR107) compliant.
The major players in the distributed cache game, for Java at least, are EhCache, Hazelcast and Infinispan.
First of all, my understanding of a distributed cache is that it is a cache that lives inside a running JVM process, but that is constantly synchronizing its in-memory contents across other multiple JVM processes running elsewhere. Hence Process 1 (P1) is running on Machine 1 (M1), P2 is running on M2 and P3 is running on M3. An instance of the same distributed cache is running on all 3 processes, but they somehow all know about each other and are able to keep their caches synchronized with one another.
I believe EhCache accomplishes this inter-process synchrony via JGroups. Not sure what the others are using.
Furthermore, my understanding is that these configurations are limiting because, for each node/instance/process, you have to configure it and tell it about the other nodes/instances/processes in the system, so they can all sync their caches with one another. Something like this:
<cacheConfig>
<peers>
<instance uri="myapp01:12345" />
<instance uri="myapp02:12345" />
<instance uri="myapp03:12345" />
</peers>
</cacheConfig>
So to begin with, if anything I have stated is incorrect or is mislead, please begin by correcting me!
Assuming I'm more or less on track, then I'm confused how distributed caches could possibly work in an elastic/cloud environment where nodes are regulated by auto-scalers. One minute, load is peaking and there are 50 VMs serving your app. Hence, you would need 50 "peer instances" defined in your config. Then the next minute, load dwindles to a crawl and you only need 2 or 3 load balanced nodes. Since the number of "peer instances" is always changing, there's no way to configure your system properly in a static config file.
So I ask: How do distributed caches work on the cloud if there are never a static number of processes/instances running?
One way to handle that problem is to have an external (almost static) caching cluster which holds the data and your application (or the frontend servers) are using clients to connect to the cluster. You can still scale the caching clusters up and down to your needs but most of the time you'll need less nodes in the caching cluster than you'll need frontend servers.
I m new to app engine. I m using Memcache and running it in eclipse. And every time i run the app the value of memcache resets. And i dont have the old values
MemcacheService memCache= MemcacheServiceFactory.getMemcacheService();
memCache.setErrorHandler(ErrorHandlers.getConsistentLogAndContinue(Level.INFO));
If you mean that you are running the gae development environment locally starting it from inside Eclipse, then what are experiencing is expected behavior. The memcached instance gets started and stopped every time you restart the development environment. It is a memory only caching mechanism? What do you expect it to do?
In GAE development-mode memcached is embedded into your process, so when your process terminates you loose your memcached instance as well.
However, when you deploy on your Goggle GAE account, the memcached instance is independent from your app. instance and cache values can survive your app instance restarts...but memcached instance itsself might be restarted by GAE and its not guaranteed to hold cache values for long time.
Also, you cannot use memcached as a system of record...since its not a persistent cache. If you want more reliable memcached implementation you can use Couchbase that has clustered 24/7 memcached compatible implementation for enterprise level use. It can be used as system of record...e.g. its a "NoSQL DB" ... but its not offered in GAE.
Can Tomcat tell mod_jk's load balancer to use another worker when overloaded? If so, how?
I use Kodo JDO for my persistence layer of a multi-tenant application and a big factor in memory usage is the schema information loaded per database. Every organization who uses my application has their own database. The application does not support clustering or session serialization. Retrofitting it to support those is out of scope for this project.
Each instance of Tomcat is limited to 4GB of heap to keep the garbage collector from choking. No single instance of Tomcat has enough memory to load the schema of all the databases needed for all organizations.
Tomcat's load balancing is typically done by session. Doing balancing by session can lead to one instance loading more schemas than its memory can hold and it will die a slow and painful death ending in a GC Overhead Limit.
My current workaround is to use multiple explicitly separate instances with separate host names. Each organization is given access to one of these host names.
One option would be for a "full" Tomcat to signal that it can't serve the user.
Can Tomcat tell mod_jk's load balancer to use another worker when overloaded? If so, how?
Interesting question (+1). If you find an answer, do pass it on :)
The only way I know of doing this is via a load balancer.
Another cool feature of using a load balancer is that you can setup http health checks, so if anyone of your instances stops responds, it will move on to the next healthy server.
We have several Java web-applications that need to be deployed on the same machine, over tomcat. The web-applications are not related to each other. Some of them do intensive I/O and CPU operations and consume much memory.
Under the above conditions, which approach is recommended - having a single tomcat with multiple webapps, or multiple tomcats each running a single webapp ?
If all webapps are deployed on the same tomcat, is there a way to guarantee minimum resources per webapp ? I.e. minimum amount of memory, number of threads, etc.
Thanks,
Arnon.
What we did at our company is we run 1 application per instance of Tomcat. We originally started with multiple instances and it occurred occasionally that one application would affect the other, especially if you had to restart the Tomcat instance.
One thing that might be worth evaluating is Spring's TC Server.
http://www.springsource.com/developer/tcserver
Similar to #tjg184 's experience. I would recommend running a tomcat per application instance. If you have a decent config and process management system, the incremental cost is not all that high and it gives you the best isolation possible without separate vm's for each tomcat instance. You could start with a single tomcat and some solid monitoring and then see if you need to move to one tomcat per app.