Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
I'm executing a long running batch process in IBM WebSphere Application Server v8.5.5.1, which is getting data from a ECM repository, and afterwards converting it to PDF document. It runs about 20k of small documents, which compose the whole batch.
Right now I'm getting about 20 docs/sec, but the System is only about 45-55% cpu usage, so something is preventing me to get it to work at full throttle
I'm running in Windows 2012 R2 Standard on an HP Proliant DL385p Gen8 (32GB RAM, 2 x AMD Opteron 6272 each with 16 2.1GHz cores )
All resources are locally stored, so almost no network traffic should be bothering.
I've also tried to write the PDF output documents to a RAMDISK, but there's no improvement at all.
Any ideas of where should I peek to let this process use the whole power of my server?
Thanks!!
PS: Please see attached reference image
CPU Usage graph
I think the actual issue is that you're fully using one CPU but not the other.
The good news is that you can probably easily multithread your application to use both processors; just set up a task queue and play around with the number of worker threads you have until you achieve 100% usage.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 4 years ago.
Improve this question
There's very little information about the /tmp/ folder which App Engine can use to write files. https://cloud.google.com/appengine/docs/standard/java/runtime-java8#Java_The_sandbox
The main question is if this is isolated per instance? And if an instance saves a file, starts a push queue, will the push queue be ran by the same instance and able to read the file?
Thanks
The /tmp directory actually exists in memory, so it is local to each GAE instance. From the doc you quoted:
Files in /tmp will consume the memory allocated to your instance.
Typically the execution of a push queue task is not guaranteed to happen on the same instance that enqueued the task.
This guarantee can only exist in a very specific, rather not typical case: you use manual scaling with exactly one instance running and that instance both enqueues the task and (later) processes it.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 4 years ago.
Improve this question
My application setup: 4 docker containers setup by docker-compose: myapplication (tomcat, jersey application), kafka, postgres, zookeeper.
I need to know the CPU usage of tomcat container.
1, I used JavaMeloday to get % System CPU 93.36, more details.
2, I used docker stats <docker id> to get that CPU % is 356% at the same time with % System CPU 93.36 by JavaMelody.
In my machine, there are 8 cores. why the numbers in 1 and 2 are different? How to explain them?
Thanks
As mentioned in moby/moby issue 26711:
800% is correct here, which is a standard notation for telling that 8 cores are 100% in use.
So 356% means the docker container processes (not just the java Tomcat) uses almost 4 of the 8 cores.
As opposed to Javamelody, used in "A Step-by-Step Guide to Tomcat Performance Monitoring", which monitors the Java application through JMX, and reports more: from "Java – monitoring cpu and system load of multi-threaded application via OperatingSystemMXBean", that value is an aggregate for all cores).
The issue is: docker stats and javamelody don't monitor the activity at exactly the same time, and could report (and aggregate) different activity peaks.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
I have a java program that write some temporary files in the temp directory. The temp directory is on a SSD. The write operation is normally very fast. But now with some specific sample data it is very, very slow and the CPU of the Antimalware Service Executable is high (30% - 35%). The write speed is approx. 50 KB/s.
If I set a breakpoint on the write loop the CPU of the Antimalware go to 0%. If I continue then the CPU of the Antimalware go to high. I can repeat this multiple time.
It look like the Antimalware is scanning my temporary data intensively. Why occur this and how can I prevent this?
To prevent scanning you can exclude files from being scanned by file name, directory name, file extension of the associated process. To do so, open “Windows Security Essential” or “Windows defender” (type it in the Start Menu).
Then go to Settings -> Excluded files and locations.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
I am working on Traveling Salesman Problem algorithm. When I execute the program it takes 3-4 hours to finish (not a big surprise for TSP). However, my Task manager tells that CPU is used only by 5% and ram by 27%. Is it possible to add more CPU resource for program execution, without modifying the code (I am not allowed to modify it)?
When you run the java program out of eclipse, it runs as an application on its own and not inside eclipse. There is no way to limit or allow more resources inside java. The only way to do it is:
From a Virtual Machine
When your code can utilise only a limited number of cores and you run it on a multi core machine, in this case modifying the code would be able to utilise more resources.
Since for you none of the above apply - no it's not possible for you.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
We have a website which generates MB/TB of data which needs to be mined. What technologies should we use to process terra bytes of data in real time ? Hadoop , Cassandra are good for batch processing; but not for real time.
Real-time; means process the data as it is happening and show reports on that.
Any ideas or suggestions ?
Have you looked into the Storm project? It's used by Twitter. It's like real-time Hadoop.
We use it for one of our stream processing project. It's awesome. Documentation, development, deployment, scalability awesome. We recently ran a 20K message/sec with processing (storing in Cassandra, modifying and broadcasting, calculating mean), it worked reliably and like magic. Definitely worth giving a shot. The mailing list is very friendly, I rarely had to use it to ask a question.
You can process TBs of data with the same technologies as you can process 1 MB of data, but it will take longer.
I don't see how you intend to use the data in "real time" and I suspect you mean real world.
If you mean quickly, then you need to summarise the data for human consumption. You can only present to the user kilo-bytes or mega-bytes of information at once.
Using memory mapped files can make this more efficient if you have a need to load the data all at once. This can be used to process tens of millions of records per second.
Check this page: http://hadoop.apache.org/
There are listed related frameworks/libraries to work with large amount of data in distributed environment.