The application I'm developing uses simple HashMaps as cache for certain objects that come from the DB. It's far from ideal, but the amount of data for these chached lists is really small (less than 100) and does not change often. This solution provides minimal overhead. When an item in one of these cached lists changes, its value is replaced in the HashMap.
We're nearing the launch date on production for this application. To provide a reasonably scalable solution, we've come with a load-balancing solution. The balancer switches between several Wildfly-nodes, which each hold the entire application, except for the DB.
The issue now is that when an cached item changes, it's only updated in one of the nodes. The change is not applied to the cache in other nodes. Possible solutions are:
Disable the caching. Not an option.
Use a cache server like Ehcache Server. In this way there would be one cache for all nodes. The problem however would be too much overhead due to REST calls.
A additional web service in every node. This web service would keep track of all load-balanced nodes. When a cached value changes in a node, the node would signal other nodes to evict their caches.
An off-the-shelf solution like Ehcache with signalling features. Does this exist?
My question is: Are there products that offer the last solution (free and with open license, commercially usable)? If not, I would implement the third solution. Are there any risks/mistakes I would have to look out for?
Risks/mistakes: Of course one major thing is data consistency. When caching data from a database I'll usually make sure I make use of transactions when updating. Usually I use a pattern like this:
begin transaction
invalidate cache entries in the transaction
update database
commit transaction
In case of a cache miss during the update happens the read needs to wait until the transaction is committed.
For your use case the typical choice is a clustered or distributed cache, like: HazelCast, Infinispan, Apache Ignite. However, somehow this seams really to heavy in your use case.
An alternative is to implement an own mechanism to publish invalidation events to all nodes. Still this is no easy task, since you may want to make sure that every node received the message, but also be fault tolerant if one nodes goes down at the same time. So you probably want to use a proper library for that, e.g. JGroups or the various MQ products.
I implemented it without JGroups or other signaling libraries. Each node has a REST endpoint to evict the cache. When a node starts up, it registers itself in a DB table with its IP, domain and a token. When it shuts down it removes its record.
When an object is updated in a node, the node evicts its cache and starts several threads that send a REST call (with its token and an object type) to all other nodes using Unirest, which in turn check the token and evict their caches. When an error is thrown, the called node is removed from the list.
It should be improved in terms of security and fault tolerance. The removal of nodes is really pessimistic now. Only after several failed attempts the node should be removed. For now, this simple solution does the job!
Related
One downside to distributed caching is that every cache query (hit or miss) is a network request which will obviously never be as fast as a local in memory cache. Often this is a worthy tradeoff to avoid cache duplication, data inconsistencies, and cache size constraints. In my particular case, it's only data inconsistency that I'm concerned about. The size of the cached data is fairly small and the number of application servers is small enough that the additional load on the database to populate the duplicated caches wouldn't be a big deal. I'd really like to have the speed (and lower complexity) of a local cache, but my data set does get updated by the same application around 50 times per day. That means that each of these application servers would need to know to invalidate their local caches when these writes occurred.
A simple approach would be to have a database table with a column for a cache key and a timestamp for when the data was last updated. The application could query this table to determine if it needs to expire it's local cache. Yes, this is a network request as well, but it would be much faster than transporting the entire cached data set over the network. Before I go and build something custom, is there an existing caching product for Java/Spring that can be configured to work this way? Is there a gotcha I'm not thinking about? Note that this isn't data that has to be transactionally consistent. If the application servers were out of sync by a few seconds, it wouldn't be a problem.
I don't know of any implementation that queries the database in the way you specify. What does exist are solutions where changes in local caches are distributed among the members in a group. JBossCache is an example where you also have the option to only distribute invalidation of objects. This might be the closest to what you are after.
https://access.redhat.com/documentation/en-us/jboss_enterprise_application_platform/4.3/html/cache_frequently_asked_questions/tree_cache#a19
JBossCache is not a spring component as such, but you create and use a cache as a spring bean should not be a problem.
We are building an java application which will use embedded Neo4j for graph traversal. Below are the reasons why we want to use embedded version instead of centralized server
This app is not a data owner. Data will be ingested on it through other app. Keeping data locally will help us in doing quick calculation and hence it will improve our api sla.
Since data foot print is small we don't want to maintain centralized server which will incur additional cost and maintenance.
No need for additional cache
Now this architecture bring two challenges. First How to update data in all instance of embedded Neo4j application at same time. Second how to make sure that all instance are in sync i.e using same version of data.
We thought of using Kafka to solve first problem. Idea is to have kafka listener with different groupid(to ensure all get updates) in all instance . Whenever there is update, event will be posted in kafka. All instance will listen for event and will perform the update operation.
However we still don't have any solid design to solve second problem. For various reason one of the instance can miss the event (it's consumer is down). One of the way is to keep checking latest version by calling api of data owner app. If version is behind replay the events.But this brings additional complexity of maintaining the event logs of all updates. Do you guys think if it can be done in a better and simpler way?
Kafka consumers are extremely consistent and reliable once you have them configured properly, so there shouldn't be any reason for them to miss messages, unless there's an infrastructure problem, in which case any solution you architect will have problems. If the Kafka cluster is healthy (e.g. at least one of the copies of the data is available, and at least quorum zookeepers are up and running), then your consumers should receive every single message from the topics they're subscribed to. The consumer will handle the retries/reconnecting itself, as long as your timeout/retry configurations are sane. The default configs in the latest kafka versions are adequate 99% of the time.
Separately, you can add a separate thread, for example, that is constantly checking what the latest offset is per topic/partitions, and compare it to what the consumer has last received, and maybe issue an alert/warning if there is a discrepancy. In my experience, and with Kafka's reliability, it should be unnecessary, but it can give you peace of mind, and shouldn't be too difficult to add.
Let say i have a array of memcache server, the memcache client will make sure the the cache entry is only on a single memcache server and all client will always ask that server for the cache entry... right ?
Now Consider two scenarios:
[1] web-server's are getting lots of different request(different urls) then the cache entry will be distributed among the memcache server and request will fan out to memcache cluster.
In this case the memcache strategy to keep single cache entry on a single server works.
[2] web-server's are getting lots of request for the same resource then all request from the web-server will land on a single memcache server which is not desired.
What i am looking for is the distributed cache in which:
[1] Each web-server can specify which cache node to use to cache stuff.
[2] If any web-server invalidate a cache then the cache server should invalidate it from all caching nodes.
Can memcache fulfill this usecase ?
PS: I dont have ton of resouces to cache , but i have small number of resource with a lots of traffic asking for a single resource at once.
Memcache is a great distributed cache. To understand where the value is stored, it's a good idea to think of the memcache cluster as a hashmap, with each memcached process being precisely one pigeon hole in the hashmap (of course each memcached is also an 'inner' hashmap, but that's not important for this point). For example, the memcache client determines the memcache node using this pseudocode:
index = hash(key) mod len(servers)
value = servers[index].get(key)
This is how the client can always find the correct server. It also highlights how important the hash function is, and how keys are generated - a bad hash function might not uniformly distribute keys over the different servers…. The default hash function should work well in almost any practical situation, though.
Now you bring up in issue [2] the condition where the requests for resources are non-random, specifically favouring one or a few servers. If this is the case, it is true that the respective nodes are probably going to get a lot more requests, but this is relative. In my experience, memcache will be able to handle a vastly higher number of requests per second than your web server. It easily handles 100's of thousands of requests per second on old hardware. So, unless you have 10-100x more web servers than memcache servers, you are unlikely to have issues. Even then, you could probably resolve the issue by upgrading the individual nodes to have more CPUs or more powerful CPUs.
But let us assume the worst case - you can still achieve this with memcache by:
Install each memcache as a single server (i.e. not as a distributed cache)
In your web server, you are now responsible for managing the connections to each of these servers
You are also responsible for determining which memcached process to pass each key/value to, achieving goal 1
If a web server detects a cache invalidation, it should loop over the servers invalidating the cache on each, thereby achieving goal 2
I personally have reservations about this - you are, by specification, disabling the distributed aspect of your cache, and the distribution is a key feature and benefit of the service. Also, your application code would start to need to know about the individual cache servers to be able to treat each differently which is undesirable architecturally and introduces a large number of new configuration points.
The idea of any distributed cache is to remove the ownership of the location(*) from the client. Because of this, distributed caches and DB do not allow the client to specify the server where the data is written.
In summary, unless your system is expecting 100,000k or more requests per second, it's doubtful that you will this specific problem in practice. If you do, scale the hardware. If that doesn't work, then you're going to be writing your own distribution logic, duplication, flushing and management layer over memcache. And I'd only do that if really, really necessary. There's an old saying in software development:
There are only two hard things in Computer Science: cache invalidation
and naming things.
--Phil Karlton
(*) Some distributed caches duplicate entries to improve performance and (additionally) resilience if a server fails, so data may be on multiple servers at the same time
We use eclipselink and weblogic
We have two websphere clusters, with 2 servers in each
Right now an app in 1 cluster uses rmi to do cache-coordination to keep 2 of those servers in synch
When we add a new app in the new cluster to the mix, we will have to synch the cache 2 clusters
How do I achieve this?
Can I still use jpa cache co ordination? using rmi? jms?
should I look into using coherence as l2 cache?
I dont need highly scale-able grid configurations. All I need to make sure is that cache has no stale data
Nothing is a sure thing to prevent stale data, so I hope you are using a form of optimistic locking where needed. You will have to evaluate what is the better solution for your 4 server architecture, but RMI, JMS and even just turning off the second level cache where stale data cannot be tolerated are valid options and would work. I recommend setting up simple tests that match your use cases, the expected load and evaluate if the network traffic and overhead of having to merge and maintain changes on the second level caches out weighs the cost of removing the second level cache. For highly volitile entities, that tipping point might come sooner, in which case you might have more benifit by disabling the shared cache for that entity.
In my experience, JMS has been easier to configure for cache coordination, as it is a central point all servers can connect to, where as RMI requires each server to maintain connections to every other server.
Sorry, wall of text; there's a summary at the bottom.
I am prototyping a Java application that will run on multiple servers. Every instance has an embedded Infinispan cache and the caches are configured to form a cluster in replication mode. The cache entries are loaded from an external system only - there is no need for actively adding entries using cache.put(key, value).
For that purpose, I implemented a custom CacheLoader. Loading values on-demand is working, but these entries are not replicated to other active cluster nodes. For test reasons, I tried adding entries with 'put' - these are replicated immediately.
The user guide pointed me to properties that affect cluster behavior when nodes are joining/leaving the cluster, or during writes, like fetchPersistentState, shared and fetchInMemoryState. The latter is useful in my case, since new nodes joining the cluster should receive the current state. And this initial synchronzation during startup is even fetching the entries loaded by the cache loader.
fetchPersistentState caused errors because my cache loader does not implement AdvancedCacheLoader - but since the advanced methods do not seem to be called after an invocation of 'load' I do not think that correctly implementing that interface would solve my problem.
I have also read about the ClusterLoader implementation that
consults other members in the cluster for values
but a roundtrip to the other nodes would increase response times while processing requests.
The rationale behind trying to load the value exactly once is that calls to the shared external system are considered to be rather expensive, so that the increased cluster-overhead created by replication messages should still be less problematic than loading the values on every node.
In order to have some kind of isolated test and code samples for this I forked the infinispan-quickstart/clustered-cache example on Github and adapted it to my needs:
https://github.com/flpa/infinispan-quickstart/tree/master/clustered-cache
The cache is backed by a CacheLoader now and nodes are periodically fetching/putting values to demonstrate how 'put' values are replicated but values fetched from the loader are not.
To sum this up:
Is it possible to configure an Infinispan cluster in replication mode that is populated entirely by cache loader lookups and replicates the results of those lookups on all nodes?
EDIT: I accidentally deleted the Github fork, so I recreated it from scratch including only the relevant 'clustered-cache' folder and adapted the link above.
I am afraid that out-of-the-box configuration options are not available. However, you can register listener on each node for the #CacheEntryLoaded event and in this listener execute putAsync(). Be sure to include flags SKIP_CACHE_LOAD, SKIP_CACHE_STORE and IGNORE_RETURN_VALUES to get the best performance.