We are trying to move one of our web-services (Java) to the cloud from a development server, here are the details:
There is a PHP front-end, connecting to a Java-based web-service that is connected to a MySQL database (all requests to the database are sent from the web-service, the php part is communicating with the Java back-end only, no direct connection to the database).
Start Point
Dev Server - CentOS (cPanel), 765MB-1.5GB RAM, 4CPU, Tomcat 7
*the software is running fast, no speed issues, logs show normal CPU and memory usage
Scenario #1
PHP front-end on Elastic Beanstalk and Java web-service with database on Elastic Beanstalk
*the software is about 80% slower, logs show normal CPU and memory usage
Scenario #2
PHP front-end on VPS (same company/location with Jelastic) and Java web-service with database on Jelastic
*the software is about 70% slower, logs show normal CPU and memory usage
Scenario #3
PHP front-end on VPS, Java web-service with database on Elastic Beanstalk and Jelastic (swithing)
*the software is about 70-80% slower, logs show normal CPU and memory usage on both cloud environments
What I figured out, no matter where the PHP front-end is located, that will load fast, nothing to search here.
As soon as the Java back-end is moved from the VPS to the cloud (doesn't matter if Amazon or Jelastic), the whole software slows down extremely. Based on the logs and since we tried with two providers, this doesn't seem like a resource issue.
It cannot be a connection issue since we tried to have the PHP and Java in the same environment (Scenario #1).
It is either the Java web-service slowing down extremely (for unknown reason as logs show low resource usage) or it could be the connection between the Java application and the database (I doubt since in the first scenario, all three components are on Amazon, same environment, location).
Anyone ever had such an issue before? Any ideas? Thank you!
(note, I have zero experience with cloud hosting)
It might be related to specific parameters in configuration files, mostly for DB. Please double check that they are the same in each test.
Also it is not clear how you measure performance and what "slower" exactly means. And you have not specified size of resources on Jelastic and EB. Please double check that resources are equal as well.
For high performance Java cloud backend, you can try Jelastic implementation by Elastx - see the performance research that CloudSpectator did on them (they also used Amazon and Rackspace cloud in the study): http://blog.jelastic.com/wp-content/uploads/2013/09/Elastx-Fueld-by-SolidFire-9-5-13+Jelastic.pdf
Also, I do not know who your current Jelastic provider is, but if you contact them by clicking Help / Contact Support in Jelastic dashboard, I am sure that they will be happy to troubleshoot the issue! If this does not help - please ping me offline.
What you are measuring is CPU an memory. Since both give normal results, and your application is communicating over the network, I'd suspect network latency to be the culprit. Next thing to look into would be for example disk I/O performance, which can slow down your application like having a handbrake.
Related
We have a Cloud Foundry (Java) application running on IBM Bluemix and we are looking for a way of health check for it. We mainly would like to monitor memory usage (both CF instance memory and JVM heap). We know that Auto-Scaling can do a similar thing but we think it keeps memory usages for recent 2 hours. (Please correct us if we are misunderstanding.) We prefer to monitor memory usage at least recent 24 hours. Any suggestions or comments must be appreciated. Thank you.
From a platform standpoint, you don't have a lot of options:
You can configure an HTTP-based health check for your app. Instead of just monitoring the port your application is listening to, this will actually send an HTTP request and check that it gets a valid response. If it does not, then your application will get automatically restarted by the platform. This does not keep track of any of the metrics that you listed. It's just purely a check to determine if your application is still alive.
Ex: cf set-health-check my-super-cool-app http --endpoint /health
https://docs.cloudfoundry.org/devguide/deploy-apps/healthchecks.html
You can connect to the firehose and pull metrics. This will include the container metrics of CPU, RAM & Disk usage. The firehose is just a method to obtain this information though, the whole problem of storage and pretty graphs is one that you'd still need to solve.
The firehose plugin is the best example of doing this: https://github.com/cloudfoundry-community/firehose-plugin
Beyond the platform itself, you might want to look at an APM (application performance monitoring) tool. I'm not going to list examples here. There are many which you can find with a quick Internet search. Some even integrate nicely with the Java buildpack, those are listed here.
This is probably the solution you want as it will give you all sorts of metrics, including the ones you mentioned above.
Hope that helps!
Issue at hand:
I have a java application which is taking twice as long to run on DEV and QA Servers than on my local machine. When running the job on Dev and QA I’m getting times around 1:45 – 2:30 (hh:MM) compared to my local which is getting about 0:45 – 1:10. I’m trying to determine what could be causing slow performance of a java application on servers.
What I have looked into so far none providing an answer:
Testing with same maxheap size
Observing stress on cpu. Dev is idle about 75% of the time when running the batch application, so I don’t think this is an issue.
Observing ram on Dev. Dev has more than enough memory to provide the JVM the specified maxHeap (128mb). If I understand correctly the available memory of the machine doesn’t matter as long as the MaxHeap size can be provided by the machine correct?
Ensuring the version of java isn’t causing the issue.
Set logging level same: “INFO”
Processor: servers has 2.67GHz processor my local only has 2.19GHz
Additional Information.
Server OS: Linux
Local Computer OS: Windows
Single threaded Java application.
Application is reading and writing to text files and also has calls
to a database(hibernate c3p0). These are my most/only expensive operations
I’ve scoured dozens of sites to determine a root cause but I haven’t been able to nail down what is causing the issue any help will be much appreciated.
Application is reading and writing to text files and also has calls to a database(hibernate c3p0). These are my most/only expensive operations
Most likely your server has slower access to the database it is using. e.g. if you were to run the database on the same machine it can be a lot faster than across a network. I would look at the time it takes to perform some simple hibernate operations locally and on your server. If performance is a concern I suggest looking at removing hibernate or even your database and your program can run 10x - 100x faster.
I’m trying to determine what could be causing slow performance of a java application on servers
The Server I was testing on happened to be in the DMZ (outside the network). While the database and my local computer (when working in the office) are inside the network. This was the case I failed to evaluate.
I am deveoloping a web application with Hibernate, JPA, Spring and Struts2. When I run the application for a few hours in my web server (VPS Tomcat) the OS send a SIGKILL to tomcat because of the memory usage. My server has 288Mb, tomcat gets killed when it reaches 200Mb aprox. Someone has told me that I need more memory but my application is small and doesn´t have too much traffic, it is not in production yet. I am using postgresql and my database is about 150Mb, it has many images. I have tried to use a memory profiler with netbeans, but the IDE becomes to slow and I have not been able to find anything.
I'll appreciate any help.
Do you close properly your connections in a finally block ?
It's hard to reply without the code with only theses informations
i have used JProfiler and yourkit but i am not satisfied with output for actual performance tuning and memory usage currently we have been switched to java melody. This not only help performance optimization in dev but also in production system. Java melody is very easy to integrate and configure and in production you can enable or disable by just updating web.xml
I'm developing a test app with GWT+Java-AppEngine and the deploys are so heavy and slow.
I read about minimize permutations or parallel compilation of GWT, but my internet connection is not so good and I think that I'm uploading heavy files to App Engine Server.
How can I optimize this? Can I check where is the bottleneck?
The reason that I need several deployments is because I'm using Google API's through OAuth and I can't set localhost as a callback (I do?).
I am not entirely sure about your scenario, so I will try to guess your intentions.
For development purposes, you really should be working on the local server, it comes with all the API's and stubs for things like user login and what not. That is instantaneous. Once you are happy with your local app and it is time to upload, then if AppEngine overlord decides to take time due to AppSize/Slow Connection/Service outage/random act of diety, there is little one can do.
Considering that one doesn't deploy every hour, I think your time would be better spent on the app, instead of tuning the upload time.
I am assuming you are already following http://code.google.com/appengine/docs/java/gettingstarted/uploading.html
I personally have dabbled with appengine but the python version, and it may take a few mins but once upload is complete, you are good to go.
maybe you could get your local machine a dyndns hostname and make it accessible from the internet ? –
I think what Bastian meant was as follows (assuming Dev server can actually serve domains - i am not sure about that)
Have your domain host (example.com) maintain an 'A' record pointing to your development machine IP address [hence when you do example.com, your dev machine responds as the server]
This means if you setup DNS records to point to ghs.google.com or whatever, you will have to change them (DNS records take a while to propagate depending on host)
Once you are happy, and you want to test on google, you still have to 'upload' before you can try it on appspot.com and ofcourse change DNS entries again so example.com works off google servers.
Too much work in my opinion. Better to use dev server in local machine.
Have a break while you are uploading. Have a KitKat to kill time :)
Will a vps with 360 megs of ram running Linux be able to support a single user developing a java web application that uses Spring, Hibernate, and MySQL for the database? The server will be for development only so the application will not have more then one or two concurrent users.
edit:
By development I mean a server I can deploy and test on. The actual coding will be done on windows, but I want a Linux server to test on as well.
This could work ok, but it depends a lot on your application setup. If you cache a lot, your appserver caching page content, Hibernate caching query results/objects or MySQL caching query results you probably will need more RAM. So if your content is big it might not fit, otherwise it might just fit. If you have absulutely no option of increasing the amount of memory if you find out you need more I would certainly not recommend this setup.
But maybe more to the point: What is your target platform? I would say that your server should match that.
Just for linux testing it probably is easier to either get a cheap pc or run it inside a virtual machine on your development machine (assuming you've got plenty of ram on that one).
Depends on what you're running for your IDE. If you're using Eclipse, you're going to want somewhere around 1Gb of RAM (Eclipse is a memory hog...and slow as all hell if you don't have enough).
If you're using a more efficient (memory wise) IDE, then you should be good to go with that setup for development.
UPDATE
Since no coding is going to happen on the box...you should be just fine with that box to do your testing. Enjoy!
Short answer - I don't think you will have any problems with the amount of ram. I've deployed a rails app to a 256MB VPS and it worked great for development.