Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I have to cache the oracle database change notifications and query results and use it to update another table in another oracle database. Answers to my previous question suggested me to use cache and I am not sure if there is a inbuilt cache mechanism in java or I have to use some external framework/library to achieve this?
Note: I am using oracle jdbc to fetch the results.
For example: If a user updates the table manually through sql developer or through command prompt then the resulting change-notifications coming from oracle database should be cached.
There is no built-in caching mechanism in the Java JDBC API per se.
The Oracle JDBC driver (apparently) supports client-side caching of result sets, but I don't see how that helps:
It caches resultsets, not inserts and deletes.
It only caches queries / resultsets done by the current client JVM, not anything done from other client JVMs ... or other kinds of client.
AFAIK, there is no public API for checking what has been cached, and you would need that if you were to use the cache for mirroring updates.
In fact, I don't think that caching is the right approach at all. Caches have finite size, and throw out (evict) old entries. A cache is (typically) not aware that something else is depending on entries NOT being evicted until they have been mirrored. Hence, I would expect a cache-based solution to loose updates if they happen faster than they can be mirrored.
I think that the best approach would be to set up Oracle database replication on the server side. There are reams of Oracle documentation on this ... and I'm not an expert.
Client-side replication using "ha-jdbc" (or similar) would work in some cases. However, all of the database clients would need to use the library, and that is precluded in your use-case.
Finally, it may be possible to implement a replicator that uses Oracle database update notifications. However it looks complicated and potentially expensive. (I'd expect Oracle's native replication to be more performant.)
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
Background:
I'm working on a web-based application built in Spring MVC and Angular. we have a help desk module where agents use to work for customer care. The application is deployed on a single server. we have a ticket locking mechanism i-e when an agent opens a ticket to start working on it ticket get locked to that agent so that other agents may not work on the same ticket at the same time. as soon as the agent closes the ticket its available for other agents to open and update if needed. For locking ticket to avoid too much DB calls we have implemented ConcurrentHashMap so that everyone gets updated for the locking ticket using the same map this is working absolutely fine.
Issue:
Now the application is deployed on two different servers and this ConcurrentHashMap is not working as MAP is maintained by each server. If a user is locking a ticket using Node-1 and if 2nd user's request goes to node-2, this approach is not going to work. To avoid this situation we are planning to change the flow so that we may avoid such issues. Parallelly, we don't want to save this locking details directly to DB to avoid DB IO as it a very frequent usage area of application.
Options
After doing some R&D I got the following options that we can implement, keeping the persistence in mind.
We can implement the In-Memory table concept using MSSQL or Redis
RabbitMQ
We can implement an API that will be deployed on a single node and both of our servers will use that to maintain locking tickets but we still have two problems with this calling API would be time taking and 2nd it's not persisting the data, if the server will get restarted we will lose the data.
Can anyone advise me on which approach should be good for the above case and how to implement it. I just need a startup.
thanks in advance.
I think your real problem is this:
For locking ticket to avoid too much DB calls [ you decided not to use the database ].
IMO, that was a mistake. A database call to acquire a "lock" on a ticket is unlikely to result in too many database calls.
In analyzing this, you need to consider how often someone will want to start working on a ticket, and how often it is likely to fail because someone is already working on the ticket. I don't know your use-case details, but I would be very surprised if the latter event happens more often than once per second.
If your database cannot sustain one "small" database operation per second (worst case!) for locking, then it won't be able to sustain the larger transactions involved in creating tickets, agents updating them, user reading them, and so on.
So suggestions are:
Work out what the actual database load for ticket locking will be ... relative to all of the other things that the database needs to do.
If it is small, just go back to the database for ticket locking. Keep it simple!
If it is large; either:
Scale up or scale out the existing database; e.g. use sharding. It seems likely that you will need to do this anyway. That should give you the "headroom" to use the existing database for locking as well.
Create a separate database server for the locking. It is unlikely that it will need to be big, and I can't envisage that it needs to be very fast. (See below!!)
Use one of your proposed solutions.
But my main advice is to AVOID the trap of premature optimization. You seem to be designing for bottlenecks that you think will exist without any clear evidence for this. For example:
"We can implement an API that will be deployed on a single node and both of our servers will use that to maintain locking tickets but we still have [the problem] with this calling API would be time taking ..."
Unless the time taken is multiple seconds, this is unlikely to be a real problem. The best strategy is to implement the system first the simple way and then measure performance to see 1) whether optimization effort is warranted and 2) where the real bottlenecks are in the complete system.
In your case, I doubt that the users will care if it takes (say) 1 second versus 2 seconds to be told that someone else is already working on a ticket.
Finally, wouldn't it be simpler to use an existing off-the-shelf ticketing system? There are many of them out there. Commercial products, open source, hosted, etcetera. (OK it is probably too late for this, because it sounds like you are committed to implementing your own ticketting system from scratch. But it may not be too late to reconsider your strategy.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I've been reading more about the Java cache and been wondering if one could extend it with a nosql system so that java apps across multiple systems can have a common cache; ie 10 app servers all running the same app on a distributed architecture behind a proxy could share a common java cache stored on a nosql system like memcached.
Does anyone know of anything like this?
The Java Cache API only defines a technology-agnostic API to access a cache from a Java application.
The main implementations are EHCache, JBoss TreeCache and a few other, but I fail to see why it wouldn't be possible to implement an adapter to use MongoDB, memcached or any other NoSQL database.
So technically, yes, it should be doable. But the API is still relatively new, so maybe those adapter will be implemented soon by your favorite database provider.
See here for the existing implementations : JSR 107 implementations
This is certainly a viable architecture approach. You might want to check out this article that provide a bit more details on how distributed caching can help distributing Java application:
http://www.cacheonix.org/articles/How_to_Distribute_Java_Application_on_Multiple_JVMs.htm
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I am building a web service using the Dropwizard framework (version 0.7.0). It involves executing some read-only queries to the database, manipulating the result set and then returning that data set. I am using MySQL as a database engine. Since I am new to this framework, I want to know which option I should choose: Hibernate or JDBI.
I've used both of these. I've used Hibernate with GORM in Grails as well as in a traditional Spring app and I've used JDBI in Dropwizard.
I have really enjoyed the simplicity of JDBI and here are a couple of reasons why I prefer it over Hibernate.
I know exactly what SQL is going to be executed to acquire the data I'm requesting. With Hibernate, you can sometimes have to do a lot of messing around with HQL and configuring your objects to what you intended to have returned. You ultimately resort to SQL, but then have the difficultly of properly mapping your results back to your domain objects, or you give up and allow hibernate to fetch them one by one.
I don't need to worry about lazy/eager fetching and how that is going to affect my query time on large data sets.
Mappings aren't complicated because you manage them on your own and you don't have to rely on getting the right combinations of annotations and optimizations.
For your case in particular, it sounds like you'd want something lightweight because you don't have a lot of use cases and that would definitely be JDBI over Hibernate in my opinion.
Really, both of these solutions are just "lock-in".
If you want to go with a persisted model type interface, write your code against JPA (if you are sure it's only going to back to a relational database) or JDO (if you might want to back to relational and other-type databases, like the no-SQL movement). This is because with either of these solutions, when problems occur you can switch persistence providers without rewriting the bulk of your code.
If you want to go with a procedural persistence model (dealing with SQL queries directly and such), then go with JDBi or perhaps even JDBC. JDBi provides a very nice abstraction over JDBC; however, there are cases where you want the lower level access (for performance reasons, of the kind were you are tuning the queries and database in concert). Again JDBC is a standard such that you can swap out one database for another with some ease; however, the SQL itself won't be as easy to swap out.
To amend the SQL swap out problems, I recommend using sets of property files to hold the queries, and then a Resource loader type mechanisim to bind the SQL for the right database to the code. It isn't 100% foolproof; but it does get you a bit further.
Now, if you ask me what I'd use, I highly recommend JDO.
if you have very few work upon database then use JDBI else go for Hibernate as it is very strong and provide many additional features to your persistence logic.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
We have a big MS-SQL database with millions of records. There are java services which interacts with DB for data retrieval. The application deals with lot of LIKE, IN & JOIN clauses. This leads to higher CPU usage & longer response time during service call for java services.
Based on our analysis, we identified 4 big tables in DB which occupies lot of space. The decision is to add one more DB in a different server and allocate enough memory. Then move those 4 huge tables to a separate one from existing DB.
Can anyone please suggest will this idea help for DB optimization , making full use of two databases or any other helpful techniques?
Has the existing server maxed out its CPUs? Often times CPU is not so much the issue but I/O capacity is (network, disks). You might be able to provide more I/O resources on your existing server and then distribute your temp db, tx logs, and potentially the four tables, across separate disks.
Splitting your schema across two servers will not be transparent to your Java clients. Also, if there are referential integrity constraints between those tables and the rest of the schema then you cannot do it unless you drop the constraints.
One alternative could be to use transcactional replication to replicate your complete schema across to another server. You could put both instances behind a load-balancer. Or you move all read-only processes to the replication.
Replications have their own downsides though. E.g. schema changes become harder because not all types of changes can be replicated. A stuck tx replication is a pain to resolve.
Before you start with any of this, make sure you have the appropriate indexes. The view sys.dm_db_missing_index_details can help you with that. Check out the MSDN article to see how to query the view.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am getting started working on a Java project I inherited from my predecessor. I am new to SQL, but SQLite is used to store various different types of data in this system. Previously, I always would store this type of information in internal Java data structures (i.e. Hashmap of usernames, etc.)
My question is why is SQL considered a better alternative?
SQL solves a number of problems that are not addressed by native programming languages. These may not be important for a particular application. But, here are some examples.
SQL can process data both from disk and in memory, transparently. It has the notion of a table, which is persistent and stored on disk, but the processing can all be in memory (if the table is already loaded into the page cache).
SQL seemlessly handles data as it gets larger, managing both memory and disk.
SQL handles security and authorization. New methods do not have to be invented.
SQL ensures atomicity of transactions, so you don't have to worry about partial updates to the system in the event of a failure.
SQL seemlessly enables multiple clients to access the same data, with the database taking care of concurrency issues.
SQL can readily handle multiple different types of entities and the processing needed to combine them.
Although not applicable to SQLite, SQL often takes advantage of multiple processors and multiple disks -- transparently to the application.
However, I must emphasize, this doesn't mean that all data in all applications should be stored in a database (although I do lean in that direction). You may have an application where an external data store simply isn't necessary.
An external database such as SQLite provides persistence and allows you to share data among multiple instances of your application.