I have a java application, using JVM as session storage. But recently when a certain number of users exceed. The application goes down. JVM is running out of memory.
I want to add new application server also want to use load balancer but as the session is JVM dependent, I can not share it with other application server.
It would be great if I can use one JVM instance dedicatedly for the JVM session and access it via multiple application server.How I can do that?
I am using Java Spring in the project. Is my plan ok to accommodate lot of users requests?
Thanks in advance.
There is a 3rd Party Application called Terracotta. i Tried it and work fine for Spring Application.
You can find the Configuration details from below link.
http://www.terracotta.org/documentation/4.1/terracotta-server-array/introduction
Put a Comment if need any help.
First make sure you know what the cause of out of memory is. If it is really related to having many sessions, you may want to change the way sessions are managed. Instead of keeping session in memory, you could save it into database. In that approach you would reduce memory and also after adding other machines session wouldn't be tied to any of them.
It sound like you are holding (large amounts of) session data in memory ... for performance reasons.
Is my plan ok to accommodate lot of users requests?
Ultimately you will run out of:
physical memory to hold all of the session data in one JVM, or
CPU and I/O bandwidth to satisfy the requests for session data from the other application servers, and/or
CPU resources for simply managing the data. (Hint: the time taken to do a full GC is proportional to the total amount on reachable data.)
If your architecture uses a single JVM for all session data, you will eventually hit a wall. That suggests that you should make it possible to replicate that part of your system. However, it is not possible to suggest the best way to do that ... without a much deeper analysis of your application ... and its real need for scalability.
Bottom line: there are no simple one-size-fits-all solutions for scalability.
Related
I wonder if there is any possibility to split the users of Java application running under Tomcat by server resources?
Problem description
We have an application written in Java and running under the control of Tomcat server. Sometimes users could possibly do some actions leading to 100% charge of server during a long period of time. That requires some limitation of server resources per user to disable him to make a server crash.
For a moment the only idea I've come up with is to containerize all the application in Docker and launch a separate resource-limited container for each user. It looks like missing an easier solution.
How do you intend to split Server resources?
Memory is shared across the JVM and you cannot limit given memory for a certain Thread. Spawning a new process it the only way to further limit memory in an easy and maintainable way.
If you want to avoid that, you would probably have to rework your memory intensive method.
A question came up at work today:
Imagine that I have a lot of concurrent users accessing my website, each with their own data stored in user sessions. Considering that I have a limited amount of memory available in my JVM, what happens, or what is supposed to happen, when the concurrent active sessions reach a point where the container JVM runs out of memory?
I tried to find something in the servlet spec about what should happen but there is nothing definitive in there, it just says that the developer will have access to the session objects, etc. This makes me think that it must be provider specific.
Taking Tomcat as an example, does the webserver just start throwing OutOfMemoryExceptions? Or is it more clever about the way it deals with this, for example paging the sessions off to another cache? Or another way all together?
The servlet specification does not say anything about memory considerations, so you'll get no help there.
Unless configured specifically to do otherwise, Tomcat will allow you to use-up all available memory with your HttpSession objects (realy, their contents) and ultimately the JVM will start throwing OutOfMemoryErrors, potentially taking your server down (though the JVM will continue to run, many things will behave ... unpredictably and unfortunately).
If a single request starts using a lot of memory in local variables, etc., the request-processing thread will suffer an OutOfMemoryError and stop processing the current request. (I believe in this case, the request-processing thread will actually be recycled by the Tomcat request-processing thread pool). The garbage collector will likely run shortly thereafter and re-claim the memory used by that request and your server will stabilize.
On the other hand, if you use-up a lot of memory and store those objects into the user's HttpSession, then the GC cannot free any memory and your server will continuously suffer from OutOfMemoryErrors. Though Tomcat will expire sessions on a specified schedule (the default is after 30 minutes of inactivity), the session-cleansing thread may encounter an OutOfMemoryError during operation and therefore fail to perform its duty, compounding the whole situation (because, effectively, HttpSessions will never expire).
There are several ways to mitigate the above unfortunate scenario. Which one makes sense to you is up to your requirements and environment.
Increase your heap size. This will obviously only get you so far. If you are filling-up a 16GiB heap with HttpSession and related objects, then you are reaching the limits of commodity hardware and you can't simply buy-a-bigger-box.
Reduce the session expiration time (default is 30 minutes). If a session isn't explicitly terminated (say, through a logout function), then the HttpSession object and all its contents stick around until the expiration interval has passed. If you have lots of data in lots of abandoned sessions, reducing the session expiration time may give you some breathing room.
Stop putting so much data into the user's HttpSession. While this may seem like an obnoxious suggestion ("stop doing that"), it is honestly a valid one: do you really need to store so much stuff in the session itself? How about using a data store of some kind (relational database, non-relational database like Cassandra, webcache, etc.)? Maybe store the data in a file on the disk? While these suggestions will certainly limit your ability to access your data quickly, they are certainly better options than having your server come crashing down under the weight of so much stuff in your session.
Use Tomcat's PersistentManager, which is a session manager capable of swapping active (but idle) sessions to some external storage (file-based and JDBC-based storage mechanisms are available by default). This can get you a long way while you figure out some other place to put all that data you are shoving into the user's session.
All but #4 are applicable to any servlet container. #4 should be available to JBoss users, as JBoss uses Tomcat internally. For other containers, you may find that a similar capability exists.
Generally this is vendor dependent. It is, however, frequent that containers can serialize sessions to disk at its discretion (if the session data can be serialized).
If the container can do this, it can push out inactive sessions when needed and restore them again when needed, using memory better and thereby postponing any OutOfMemoryExceptions.
I'm new here and I'm not that very good in CPU consumption and Multi Threading. But I was wondering why my web app is consuming too much of the CPU process? What my program does is update values in the background so that users don't have to wait for the processing of the data and will only need to fetch it upon request. The updating processes are scheduled tasks using executor library that fires off 8 threads every 5 seconds to update my data.
Now I'm wondering why my application is consuming too much of the CPU. Is it because of bad code or is it because of a low spec server? (2 cores with 2 database and 1 major application running with my web app)
Thank you very much for your help.
You need to profile your application to find out where the CPU is actually being consumed. Java has some basic profiling methods built in, or if your environment permits it, you could run the built in "hprof" compiler:
java -Xrunhprof ...
(In reality, you probably want to set some extra options: Google "hprof" for more details.)
The latter is easier in principle, but I mention the possibility of adding your own profiling routine because it's more flexible and you can do it e.g. in a Servlet environment where running another profiler is more cumbersome.
Paulo,
It is not possible for someone here to say whether the problem is that your code is inefficient or the server is under spec. It could be either or both of those, or something else.
You are going to need to do some research of your own:
Profile the code. This will allow you to identify where your webapp is spending most of its time.
Look at the OS-level stats that are available to you. This might tell you that the real problem is memory usage or disk I/O.
Look at the performance of the back-end database. Is it using a lot of CPU?
Once you have identified the area(s) where the CPU is being used, you need to figure out the real cause of the problem is and work out how to fix it. And once you've got a potential fix implemented, you can rerun your profiling, etc to see it has helped.
Can Tomcat tell mod_jk's load balancer to use another worker when overloaded? If so, how?
I use Kodo JDO for my persistence layer of a multi-tenant application and a big factor in memory usage is the schema information loaded per database. Every organization who uses my application has their own database. The application does not support clustering or session serialization. Retrofitting it to support those is out of scope for this project.
Each instance of Tomcat is limited to 4GB of heap to keep the garbage collector from choking. No single instance of Tomcat has enough memory to load the schema of all the databases needed for all organizations.
Tomcat's load balancing is typically done by session. Doing balancing by session can lead to one instance loading more schemas than its memory can hold and it will die a slow and painful death ending in a GC Overhead Limit.
My current workaround is to use multiple explicitly separate instances with separate host names. Each organization is given access to one of these host names.
One option would be for a "full" Tomcat to signal that it can't serve the user.
Can Tomcat tell mod_jk's load balancer to use another worker when overloaded? If so, how?
Interesting question (+1). If you find an answer, do pass it on :)
The only way I know of doing this is via a load balancer.
Another cool feature of using a load balancer is that you can setup http health checks, so if anyone of your instances stops responds, it will move on to the next healthy server.
I'm developing a website, that hopefully, will be accessed by more than a million people at same time. It still has only 70k users, and it's already lagging while uploading a file, or just opening pages and stuff..
I use SQLServer, tomcat and apache http server.
i've tried using another tomcat to manage the access to database, but i'm facing another problem, it has to share the same space of the other tomcat to save the uploaded files. and it causes a huge delay uploading..
what can i do to make my website faster?
The website is developed with JSF with richfaces and Java and Hibernate.
Scaling is hard.
For some operations scaling is impossible. Even the greatest (Google, Facebook, Amazon) are not free to choose their features; what is offered is often a compromise between "what would be cool" and "what will scale".
The question "how to make it faster" is unanswerable without profiling your application.
Making any decisions without considering the former point is PLAIN STUPID AND MIGHT PUT YOU IN EVEN WORSE SITUATION.
The traditional way of identifying of bottlenecks is to think separately about:
a) memory (does the system swap?)
b) cpu (are cpus really busy, or just waiting for the database?)
c) IO (usually that includes the database and bandwidth)
Depending on where is your problem, totally contradicting things will help. For example if you have plenty of memory and low bandwith, switch JSF to save state on server. This will use more memory, but make requests shorter. On the other hand if bandwitdth is not a problem and memory is, then do the opposite: switch JSF to keep state on the client. This will help to conserve memory (although in this case matters are more complicated: if tomcats in your cluster try to share session data, then saving state on server becomes an IO problem).
You say that the problem is with uploading files. To help, we would need to know: where do you save them? to DB? to filesystem? Are they short or long? How are they processed? Are there any patterns on the usage of the uploaded files (like: "new files are used most of the time")? and probably even more questions would pop up after these had been answered.
For your own sake: close this question. You will get plenty of well-intentioned, and yet misguiding answers like 'drop JSF', 'cluster everything', 'add memory', 'move to GAE or Amazon EC', 'go with a NoSQL database', 'do everything asynchronously, use a message queue', 'do everything on the client with ajax', 'drop ajax, it makes too many requests and kills server'. All of this is meaningless unless you profile, profile, profile, measure, measure, measure - FIRST. And then give as a better defined question.
If you access a db, consider to use EJB + CMP. Then follow the following model:
cluster your application server (e.g. GlassFish) for load balancing
keep all service calls of one single request in one single node (by only calling local services)