On my linux server, any java program (even the "Hello world") uses 100% of cpu and is very very slow. Id does not depend on the java version, I have tried different versions of openjdk and sun jdk, both behave the same. How could I fix that?
I have found the solution in this article http://blog.wpkg.org/2012/07/01/java-leap-second-bug-30-june-1-july-2012-fix/
There is a great chance, this is caused by the "leap second kernel bug". Firstly, check for the following in the dmesg
[10703552.860274] Clock: inserting leap second 23:59:60 UTC
To fix it, firstly stop the ntp client. On debian-like systems
/etc/init.d/ntp stop
Store the current time
date -s now
And test the java. If everything is working correctly, try to restart ntp service
/etc/init.d/ntp start
And test it again.
Related
I have a code running on Windows and Linux machine.
Message will be sent from windows to Linux machine and we include sent_time on message.
Linux machine will receive the message and we stamp receive_time on message.
Both the time are calculated using standard java API System.currentTimeMillis().
Problem here is, receive_time is ahead of sent_time which is not right as message was sent first and then received. This mostly looks to be clock sync issue between Windows and Linux machine.
How can we get the clock diff between Windows and Linux machine? (Not sure how to run program on both the machines and claculate time diff which is in millis)
if there is any sync issue, how can we get and add delta time receive_time ?
Please note that all the libraries are same on both machines.
You are trying to find time deviations, maybe even synchronize clocks across the network. This adds complexity because neither of the two machines know how much lag is in the network passing the data.
The NTP protocol mitigates all this. I'd use an existing implementation. But if you do not want to synchronize the system clock but still understand how it is done, check out the protocol documentation: http://www.ntp.org/rfc.html
We have a Java application running on Solaris which makes a connection to Oracle and checks the database for work to perform, and it runs just fine. We tried running the same code on a standalone Fedora system, its performance is good too. However, when we move it to its home on a Fedora VMWare virtual machine, it can take upwards of five minutes for the application to make the connection to the database. It ultimately DOES make the connection - it's just snail-slow. We suspect it's a configuration issue somewhere but can't find it. So far as we can tell, the two Fedora boxes have nearly identical configurations. Has anyone run into this problem before? If so, how did you get around it?
Thanks in advance for your help.
Mike Preston
Found it! When we are running under Solaris, we are running a 32-bit JVM with 32-bit extensions. We are executing through a Korn shell script and had an added -d64 flag to coerce 64-bit processing. On the Linux boxes we removed the -d64 flag from the shell script and everybody's happy. Thanks Alex for your thoughts and assistance.
Here is the solution which settled the issue...our headless development server was only occasionally getting any keyboard activity to fill the entropy pool (please read the article - I won't try to explain it here) and I assume it was blocking until there was enough "noise" to generate the requisite random numbers.Since there is only one other developer working on the system, it might take a couple of minutes to fill the buffer. Once the buffer was full it went ahead and executed the connection as expected. That also explains why we sometimes would see crisp performance followed by slow. In a nutshell, we added the string "-Djava.security.egd=file:///dev/urandom" to the Korn shell script between the call to java and the jar file name and now it works like a champ. Here's the full command string:
/usr/bin/java -Xms64m -Xmx1024m -Djava.security.egd=file:///dev/urandom -jar $1 $2 $PID
If you DO read the article, be sure to read the comments below. One of them is really funny!
I am running a simple script in Groovy on an Ubuntu 11.10 machine, which takes key/value pairs and adds them to a JDBM map in a loop. Every ~3 minutes the script hangs for a couple of minutes and then resumes. When I look at the resource monitor I see that there is no CPU or Memory activity and the process is in futex_wait_queue_me().
Please suggest means to overcome this, on a Windows machine by the way the application runs without the hangs.
Could this be an OS issue? (found many similar threads about similar futex_wait_queue_me() problems in Ubuntu0
Thanks
Please check the version of the kernel. I ran into a similar problem (java and other multithreaded applications) on Centos6 and upgrading the kernel to version 2.6.32-504.16.2.el6.x86_64 solved the issue.
See the centos bug report: https://bugs.centos.org/view.php?id=8703 which contains this pointer to an explanation of the problem:
https://github.com/torvalds/linux/commit/76835b0ebf8a7fe85beb03c75121419a7dec52f0 [^]
My stacktrace was:
cat /proc/23199/stack
[<ffffffff810b226a>] futex_wait_queue_me+0xba/0xf0
[<ffffffff810b33a0>] futex_wait+0x1c0/0x310
[<ffffffff810b4c91>] do_futex+0x121/0xae0
[<ffffffff810b56cb>] sys_futex+0x7b/0x170
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
For anyone interested I used those parameters when running java:
-Xms16384M -Xmx16384M
You can find additional GC optimization tips at http://randomlyrr.blogspot.it/2012/03/java-tuning-in-nutshell-part-1.html
I have a java application that runs on Tomcat (which runs as a service on Windows), the java process for which continues to eat up CPU before eventually requiring me to restart the Tomcat service.
First my setup:
Windows 2003 server
Tomcat 6, running as service using Wrapper
JDK: 1.6.0_20
I was seeing catch issues here and there leading up to yesterday. I had to restart midday yesterday, then at 2:30 this morning, then today I could barely restart the application and open jconsole to monitor it before it was hitting 99% CPU usage again. Through a combination of things I'm not quite sure of, it seems like I got the JVM to cycle itself and the app was hovering in the 10-30% CPU usage range for a couple hours. However, then it started to creep up again, finally going into its 99% CPU usage breakdown. I was also having trouble with high memory usage, but that has stayed fairly normal and steady since I so-called got the JVM to "cycle" (bad terminology perhaps, but this is really what it seemed to do - and in the wrapper log there was a dump of all the classes it was reloading after).
Then I was digging around some more and found a JRE 6 Update 24 installed on the server (I didn't install it as I do thorough testing with each java update - but maybe my server admin did the update). I attempted, but can't uninstall this. Thus, I get different versions when I do a java -version versus javac -version
java -version
java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)
javac -version
javac 1.6.0_20
Could this difference be causing a JVM conflict of sorts? JAVA_HOME and my PATH variables both point to the correct JDK installation.
Hoping for more stability, I decided to change my app to run on the previous JDK that was still installed - JDK 1.6.0_04. I changed the wrapper.conf, set env variables, cleaned and rebuilt, and started. This does seem more stable and has been up for about 4 hours. The CPU usage has climbed to the 90s, then it seems to clear itself out again.
I've done heapdumps then ran them through the Memory Analyzer in Eclipse (nothing new found there), I've used jconsole with jtop to look at threads - nothing jumps out, thus why I continue to be curious if it's a java/jvm issue. So, I know this is a long post - but I don't really know where to go from here. Any ideas?
(I've done exhaustive web searching on this and some articles have pointed to possibly a Quartz issue or Hibernate queries not flushing. Nothing has changed in the app since I started seeing the CPU issues, so I'm not sure where to start troubleshooting if it could indeed be linked to either.)
This isn't an easy problem. You are doing all of the basics to see if it something jumps out. It sounds like there is either a slow leak that builds up over time to the point where it can't operate. That sounds like GC is thrashing and app comes unresponsive. It could also be runaway background job(s) eating on the CPU and just doesn't complete, that might explain the long delay. You could try turning off any quartz to see if it stays up longer that might help lead you in a direction, or crank it up so it shows up sooner.
I know you've done some jconsole watching, but I think you need to revisit and watch your memory usage, the threads run time, how much time you're spending in GC, and watching what portions of memory are being eaten up (is it Eden, Tenure that's running out?).
I'd make sure you are writing out start and end messages for your background jobs running in Quartz. Then you can correlate when they start and finish with when this problem starts. Also will tell you if your jobs are finishing or not.
It's probably time to drop it into a profiler (instead of jconsole) so you can see where in the code it's spending time or what's blowing up memory. A real profiler will let you see all that data mashed up on your code and classes. My favorites is JProfiler, but YourKit is also good. You can get a 7-30 day trial so you'll have plenty of time to profile and figure your issue out without having to buy it.
Start this early in the morning so you'll hopefully see something by early night.
I just get the beach ball all day long (it's been doing nothing for hours). It's not taking CPU, not reading from disk, not using the network.
I'm using Java 1.6 on Mac OS X 10.5.4. It worked once, now even restarts of the computer won't help. Activity Monitor says it's "(Not Responding)". Only thing that I can do is kill -9 that sucker.
When I sample the process I see this:
mach_msg_trap 16620
read 831
semaphore_wait_trap 831
An acceptable answer that doesn't fix this would include a url for a decent free Oracle client for the Mac.
Edit:
#Mark Harrison sadly this happens every time I start it up, it's not an old connection. I'll like to avoid running Windows on my laptop. I'm giving some plugins for my IDE a whirl, but still no solution for me.
#Matthew Schinckel Navicat seems to only have a non-commercial Oracle product...I need a commercial friendly one (even if it costs money).
I get the same problem after there's been an active connection sitting idle for a while. I solve it by restarting sql developer every once in a while.
I also have Toad for Oracle running on a vmware XP session, and it works great. If you don't mind the money, try that.
The company Navicat has released an Oracle client for Mac (and they do a Windows version too).
It's not free, but I think you can get a 30 day demo.
Have you looked at http://www.aquafold.com/? They have a very JDBC/java Mac-friendly utility, Aqua Data Studio (ADS) that you can try for I think 30 days. It's not free, but...
Excellent support via Yahoo groups. VERY responsive re bugs or enhancement requests.
No affiliation with them - just a fan.
Squirrel is a nice database agonstic application development client. No Oracle specific features, but runs well on the mac
I use SQLDeveloper on the Mac and have had problems where it becomes unresponsive. Usually, I can fix this by going into the Activity Monitor and killing the process. However, this doesn't always work to end the process. When that happens, I go to the Terminal and find the process id and send it a SIGKILL and then the next time it will work correctly.
However, more importantly I evaluated SQLGrinder at one point. I didn't end up buying the software, largely because I have a Mac laptop and a windows desktop. Therefore, I more often use Toad on the windows desktop and it wasn't worth purchasing SQLGrinder for me.
Use RazorSQL. Do yourself a favor and spend the 60 bucks. It will pay for itself in the first hour or two of use. You may even be able to get 60 days for free out of it.
The latest version of SQL Developer is very good and I have experienced no problems with it on my Mac Pro. DB Solo 3 is also quite good.