We have a java application hosted at m.large ec2 server which in future requires high number of computations to be done, To be precise it is expected to have 10 million computations per day and each unit computation requires around 100ms, computation is of type of multiple processing at java code and then dumping to db, not like Write Once, Read Many. Which is the best way to balance the load? Options we have been going across is to instantiate ec2 servers by script as the load increases, but before we implement it, we want to have a solid suggestions from experts. Kindly provide any suggestions.
This may not be the right answer for you but certainly worth a consideration. You should define the metric and threshold for your application.
instantiate ec2 servers by script as the load increases
In above statement, you need to define what do you mean by "load"? what is the threshold that load should cross in order to say the load has "increased"
Once you have this information handy, check whether you can monitor these metrics and thresholds using AWS cloudwatch. If yes, then you can create an autscalling group where the autscalling will spin-up a new instance as soon as it told by the cloudwatch alarm that "the load has increased".
If you find that cloudwatch does not support your metrics, then write your own custom metric and have cloudwatch monitor it. Click here for custom cloudwatch metrics documentations.
Once you have your custom metric, then again integrate autoscalling and cloudwatch alarms to manage creation of new EC2 instances when your load increases.
In short, explore AWS cloudwatch alarms and AWS autoscalling.
This whole process assumes that you do have a fully-automated way of creating an EC2 instance along with your software stack. You can create a pre-baked AMI with your application stack OR you can install the application stack on the fly using tools like Opscode Chef.
Use a Platform-as-a-Service (PaaS) instead of manually creating machines and scaling them. This is more flexible for long-term use, and requires less scripting. There are some PaaS recommendations here: Looking for PaaS providers recommendations
Disclaimer: I work for Gigaspaces, developer of the Cloudify open-source PaaS stack
Related
I've been playing around with a SpringBoot application, the Micrometer façade, the Statsd implementation for Micrometer and the AWS OpenTelemetry distro deployed on ECS/Fargate. So far, I've been able to export many different metrics (JVM, tomcat, data source, etc) to CloudWatch, adding the cluster name, the service name and the task ID as dimensions.
My problem now is that I don't know how to handle that information. In a production deployment I may have more than one container and I may need to scale them out/in. This makes impossible (or at least I don't know how to do it) to create a dashboard as I need to select the task IDs up front. Another problem is that there is no way to add a filter in the dashboard that just shows the list of available task IDs so I can select the one I want to monitor at that moment to remove the noise from the other ones. Something QuickSight can do.
Am I better off just moving to something like Prometheus/Grafana for this? How do people handle monitoring of containers, specially Java applications?
AWS gives you the option to alarm you based on ECS metrics but at the service level (so I guess either based on the average or max of CPU usage for example) but that isn't enough when you have a workload that is not evenly spread across your instances. Is alerting not possible when at the container level (something like alert me when the service is at 60% CPU or a single container is at 80% for example)?
We have a Cloud Foundry (Java) application running on IBM Bluemix and we are looking for a way of health check for it. We mainly would like to monitor memory usage (both CF instance memory and JVM heap). We know that Auto-Scaling can do a similar thing but we think it keeps memory usages for recent 2 hours. (Please correct us if we are misunderstanding.) We prefer to monitor memory usage at least recent 24 hours. Any suggestions or comments must be appreciated. Thank you.
From a platform standpoint, you don't have a lot of options:
You can configure an HTTP-based health check for your app. Instead of just monitoring the port your application is listening to, this will actually send an HTTP request and check that it gets a valid response. If it does not, then your application will get automatically restarted by the platform. This does not keep track of any of the metrics that you listed. It's just purely a check to determine if your application is still alive.
Ex: cf set-health-check my-super-cool-app http --endpoint /health
https://docs.cloudfoundry.org/devguide/deploy-apps/healthchecks.html
You can connect to the firehose and pull metrics. This will include the container metrics of CPU, RAM & Disk usage. The firehose is just a method to obtain this information though, the whole problem of storage and pretty graphs is one that you'd still need to solve.
The firehose plugin is the best example of doing this: https://github.com/cloudfoundry-community/firehose-plugin
Beyond the platform itself, you might want to look at an APM (application performance monitoring) tool. I'm not going to list examples here. There are many which you can find with a quick Internet search. Some even integrate nicely with the Java buildpack, those are listed here.
This is probably the solution you want as it will give you all sorts of metrics, including the ones you mentioned above.
Hope that helps!
I am working as a developer on a batch processing solution, how it works is that we split a big file and process it across jvms. So we have 4 processor jvms which take a chunk of file and process it and 1 gateway jvm job of gateway jvm is to split the file into no. of processor jvms i.e. 4 and send a rest request which is consumed by processor jvms, rest request has all the details the file location it has to pick the file from and some other details
Now if i want to add another processor jvm without any downtime is there any way we can do it. Currently we are maintaining the urls for 4 jvms in a property file is there any better way to do it ? which provided me the ability to add more jvms without restarting any component
You can consider setting up a load balancer and putting your JVM(s) behind it. The load balancer would be responsible for distributing the incoming requests to the JVMs.
This way you can scale up or scale down your JVM depending on the work load. Also, if one of the JVMs are not working, other part of your system need not care about it anymore.
Not sure what is your use case and tech stack you are following. But it seems that you need to have distributed system with auto-scaling and dynamic provisioning capabilities. Have you considered Hadoop or Spark clusters or Akka?
If you can not use any of it, then solution is to maintain list of JVMs in some datastore (lets say in a table); its dynamic data meaning one can add/remove/update JVMs. Then you need a resource manager who can decide whether to spin up a new JVM based on load or any other conditional logic. This resource manager needs to monitor entire system. Also, whenever you create a task or chunk or slice data then distribute it using message queues such as ApacheMQ, ActiveMQ. You can also consider Kafka for complex use cases. Now a days, application servers such as websphere (Libery profile), weblogic also provide auto-scaling capability. So, if you are already using any of such application server then you can think of making use of that capability. I hope this helps.
I'm looking for a java api way to do this, not an asadmin / CLI command.
Although I've looked around here: http://glassfish.java.net/nonav/docs/v3/api/
I haven't seen it yet.
This configuration com.sun.grizzly.http.StatsThreadPool is how the min and max number of threads are configured (you can see this in the admin/console/web page). However, I want a real-time "read" of the current number of free threads (I'd settle for being able to calculate max - currently used)
My goal is to create a filter that implements this sort of "I'm too busy" algorithm, based on request thread usage.
Naturally, there are other measures you could use, but this measure is appropriate for my application.
I would suggest Monitoring GlassFish using AMX.
AMX is an API that exposes all of the Enterprise Server configuration,
monitoring and JSR 77 MBeans as easy-to-use client-side dynamic
proxies implementing the AMX interfaces.`
You can query the Monitoring MBeans using this API. See http://docs.oracle.com/cd/E19575-01/821-0181/beapd/index.html
A program I am working on takes forever to complete (~3days, everytime).
Is there some place on the internet where I can leave the code, some robot might run it for me and I can come back and collect the results? Some online judge that offers this capability?
[I am not talking abt optimisations here.]
You may need to go to something like this:
Amazon Elastic Compute Cloud (Amazon
EC2) is a web service that provides
resizable compute capacity in the
cloud. It is designed to make
web-scale computing easier for
developers.
http://aws.amazon.com/ec2/
If you really cannot run it on your own machines, you can run in on Amazon EC2 cloud. You would need to maintain a virtual machine, but Amazon provides some preconfigured settings.
The pricing starts with $0.085/hour (~$6 for three days). The actual price is determined by the duration of use and your CPU needs. Higher CPU capability is more expensive.