Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
We have a big MS-SQL database with millions of records. There are java services which interacts with DB for data retrieval. The application deals with lot of LIKE, IN & JOIN clauses. This leads to higher CPU usage & longer response time during service call for java services.
Based on our analysis, we identified 4 big tables in DB which occupies lot of space. The decision is to add one more DB in a different server and allocate enough memory. Then move those 4 huge tables to a separate one from existing DB.
Can anyone please suggest will this idea help for DB optimization , making full use of two databases or any other helpful techniques?
Has the existing server maxed out its CPUs? Often times CPU is not so much the issue but I/O capacity is (network, disks). You might be able to provide more I/O resources on your existing server and then distribute your temp db, tx logs, and potentially the four tables, across separate disks.
Splitting your schema across two servers will not be transparent to your Java clients. Also, if there are referential integrity constraints between those tables and the rest of the schema then you cannot do it unless you drop the constraints.
One alternative could be to use transcactional replication to replicate your complete schema across to another server. You could put both instances behind a load-balancer. Or you move all read-only processes to the replication.
Replications have their own downsides though. E.g. schema changes become harder because not all types of changes can be replicated. A stuck tx replication is a pain to resolve.
Before you start with any of this, make sure you have the appropriate indexes. The view sys.dm_db_missing_index_details can help you with that. Check out the MSDN article to see how to query the view.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I am new to microservices architecture, but I have learned that a key advantage of using such architecture is scaling.
Say we have one microservice instance and it is connected to one Mysql database.
Now we need to scale it up to be 3 instances instead of one, does that mean that each instance will have its own database? or it will be 3 instances and one database?
Any information is appreciated.
Usually you would like to have a single database backing a microservice. Otherwise you are just creating multiple instances of the same service. Challenges there are to avoid locking tables and/or rows so the services scaling is worth.
Classic SQL database are hard to scale horizontally (typically the do not support multiple write replicas and depends on a single node for writing). This also represents a challenge in case your services are geographically distributed. There are multiple ways to reduce those effects (using no SQL databases, messages queues) but which matches best depends on each context.
Just in case, sharing database amongst several microservices is definitely an anti-pattern. That is using the database as integration point.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I already have a blog application which is built on Spring-Boot as a monolith.
There are 2 entities.
user
post
And the mapping is one to many.
A single user can create multiple blog posts.
How can i recreate the same functionality as separate microservice applications.
So far on researching through internet, what i see people saying is create database per service etc.
Suppose if i create 2 services say
UserService(which has separate DB and CRUD operations associated with it)
PostService(which has separate DB and CRUD operations associated with it)
How can i make communications in between them.
In the monolith app the POST entity has createdBy mapped to User.
But how this works in Microservices architecture?
Can any one please help me how can i design such architecture?
First list out the reasons why you want to break it in micro-services. To make it more scalable (in following scenarios for example).
Post comments becomes slow, and during this period registration of
new Users to remain unaffected.
Very few users upload/download files, and you want general users who simply view comments and post comments to be unaffected, while upload/download files may remain
slow.
Answers of the above question and analyzing,priotizing other NFR's shall help to determine how and what to break.
Additionally,
Post service only needs to validate whether the user is a valid logged in user.(Correct?)
User Service does not really need to communicate with post service at all.
Further you might want to decouple other minor features as well. Which in turn talk to each other which can be authenticated via other means like(Certificates, etc.) as they will be internal and updating some stats(user ranking), aggregates data etc.
The system might also have a lot of smaller hidden features, which might or might not have to do anything with Post Service at all, which can be separated in terms of different micro-services(like video/file/picture/any binary content upload/download) also and prioritized based on computation power needed, hit frequency and business priority.
After breaking it in to micro-services, you need to run some stress tests (based on current load) to know which services needs replication and which not and needs a automatic load balancing. Writing stress load first before breaking can also help to understand which features need to be move out of the monolith first.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I have to cache the oracle database change notifications and query results and use it to update another table in another oracle database. Answers to my previous question suggested me to use cache and I am not sure if there is a inbuilt cache mechanism in java or I have to use some external framework/library to achieve this?
Note: I am using oracle jdbc to fetch the results.
For example: If a user updates the table manually through sql developer or through command prompt then the resulting change-notifications coming from oracle database should be cached.
There is no built-in caching mechanism in the Java JDBC API per se.
The Oracle JDBC driver (apparently) supports client-side caching of result sets, but I don't see how that helps:
It caches resultsets, not inserts and deletes.
It only caches queries / resultsets done by the current client JVM, not anything done from other client JVMs ... or other kinds of client.
AFAIK, there is no public API for checking what has been cached, and you would need that if you were to use the cache for mirroring updates.
In fact, I don't think that caching is the right approach at all. Caches have finite size, and throw out (evict) old entries. A cache is (typically) not aware that something else is depending on entries NOT being evicted until they have been mirrored. Hence, I would expect a cache-based solution to loose updates if they happen faster than they can be mirrored.
I think that the best approach would be to set up Oracle database replication on the server side. There are reams of Oracle documentation on this ... and I'm not an expert.
Client-side replication using "ha-jdbc" (or similar) would work in some cases. However, all of the database clients would need to use the library, and that is precluded in your use-case.
Finally, it may be possible to implement a replicator that uses Oracle database update notifications. However it looks complicated and potentially expensive. (I'd expect Oracle's native replication to be more performant.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am getting started working on a Java project I inherited from my predecessor. I am new to SQL, but SQLite is used to store various different types of data in this system. Previously, I always would store this type of information in internal Java data structures (i.e. Hashmap of usernames, etc.)
My question is why is SQL considered a better alternative?
SQL solves a number of problems that are not addressed by native programming languages. These may not be important for a particular application. But, here are some examples.
SQL can process data both from disk and in memory, transparently. It has the notion of a table, which is persistent and stored on disk, but the processing can all be in memory (if the table is already loaded into the page cache).
SQL seemlessly handles data as it gets larger, managing both memory and disk.
SQL handles security and authorization. New methods do not have to be invented.
SQL ensures atomicity of transactions, so you don't have to worry about partial updates to the system in the event of a failure.
SQL seemlessly enables multiple clients to access the same data, with the database taking care of concurrency issues.
SQL can readily handle multiple different types of entities and the processing needed to combine them.
Although not applicable to SQLite, SQL often takes advantage of multiple processors and multiple disks -- transparently to the application.
However, I must emphasize, this doesn't mean that all data in all applications should be stored in a database (although I do lean in that direction). You may have an application where an external data store simply isn't necessary.
An external database such as SQLite provides persistence and allows you to share data among multiple instances of your application.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I was wondering what people prefer when fields like 'created' and 'modified' have to be changed? Is it prefered to do this in the application code or use triggers for that??
Both are valid, but my experience is that it's more common to do it in code. I tend to put it in the application code.
Often you want to be able to cross-reference these timestamps with various logging information. Doing it in code ensures that the timestamps line up. Otherwise, you add an extra maintenance load to always ensure that the system time on your db-servers and app-servers are in sync
Logic today tends to be more in the application than in the database. Stored procedures and the likes are less and less utilized, especially in systems being built from scratch. Therefore, less and less notice is taken of the database so putting logic in triggers tend to get forgotten over time and it makes it harder to debug your application as logic will pass across process (and technology) boundaries.
Sometimes triggers don't fire. Many people will claim that they are fool-proof, but I've seen them fail more than once. And it has forever tainted my opinion of them.
Proper design and usage of AOP and framework support will let you do this fairly unobtrusively and with minimal copy/paste boiler-plate code.
Doing it via triggers would have been preferred if every database supported triggers on Insert and Update. But that is not the case with all the databases. Your Java code should be database agnostic as much as possible and code should not assume dependency on particular database(s) hence it is better to have some layer/interceptor between your application code and database to do this for you. That way you can avoid repetitive code and keep business logic clean from these update calls.