Set priorities between mysql processes - java

I'm fighting with MySQL, Grails and Java.
I have the app Server, with Grails app running there.
I have another server with mysql db.
I have one Java app, that connects to the server to export data in csv files.
I have a lot of data (10Million of registers) in the DB, and every 15 min the Grails app is connecting to the DB, to check and save new info. Normal way to function.
My problem is that while the grails app is working, i want to execute a java app that exports a part of the information in the db. But my problem is that the process is reaaly slow, but only sometimes. I will explain:
If the Grails App is working, my Java app needs around 4 days to export all the data.The Java proccess takes around 0.3-0% of CPU.
If I turned off the Tomcat, and consequently all the connections to the DB (show processlist command on mySQL) takes around 40 minutes
to finish all the process. The Java proccess takes around 85% of
CPU.
I'm not sure about what is the problem, but i tried everything, and my problem is that i can't stop the grails app all the times that i want to export. Because of this, there is some way to give priorities to the between processes to be sure that my Java app is going to have the highest priority?
Thanks in advance for your answers,

This sounds like a database locking issue. I'd recommend getting a tool like innotop and taking a close look at what is happening in the database. In particular, I'd take a careful look at open tables, locks, and queries active when your grails app is running.

Related

swing Application is slow when I run it from network on server

I have a Swing Application on Java when I run the Jar file after clean and build from my local PC it runs very good and fast but when i move the App folder to the server and make a shurtcut of Jar file to some computers on the network the application runs very slow
I made my application using NetBeans IDE 8.0.2
the Java runtime is Java 8 update 66
thank you in advanced.
It could be that your application is accessing resources that are slow across the network ie. If you are reading s file it will be quick locally but slower access network. Also if you are reading data from a database and the query results are large and or take long to run, this will also be quicker locally than remote.
I would suggest breaking you application upj into "resources", then turn them all off PR don't use them and bring them back up one by one. That way you can find out what is causing the app to run slow
thank you for your replies
I tried to copy all the application folders to local machines and change the DB connection to DB file on the server
but the program still very slow
some one told me that the problem is SQLite and I need to change to SQL Server 2008 or something like it

Preventing multiple users from doing the same action

I have a swing desktop application that is installed on many desktops within a LAN. I have a mysql database that all of them talk to. At precisely 5 PM everyday, there is a thread that will wake up in each of these applications and try to back up files to a remote server. I would like to prevent all the desktop applications from doing the same thing.
The way I was thinking to do this was:
After waking up at 5PM , all the applications will try to write a row onto a MYSQL table. They will write the same information. Only 1 will succeed and the others will get a duplicate row exception. Whoever succeeds, then goes on to run the backup program.
My questions are:
Is this right way of doing things? Is there any better (easier) way?
I know we can do this using sockets as well. But I dont want to go down that route... too much of coding also I would need to ensure that all the systems can talk to each other first (ping)
Will mysql support such as a feature. My DB is INNO DB. So I am thinking it does. Typically I will have about 20-30 users in the LAN. Will this cause a huge overhead for the DB to handle.
If you could put an intermediate class in between the applications and the database that would queue up the results and allow them to proceed in an orderly manner you'd have it knocked.
It sounds like the applications all go directly against the database. You'll have to modify the applications to avoid this issue.
I have a lot of questions about the design:
Why are they all writing "the same row"? Aren't they writing information for their own individual instance?
Why would every one of them have exactly the same primary key? If there was an auto increment or timestamp you would't have this problem.
What's the isolation set to on the database connection? If it's set to SERIALIZABLE, you'll force each one to wait until the previous one is done, at the cost of performance.
Could you have them all write files to a common directory and pick them up later in an orderly way?
I'm just brainstorming now.
It seems you want to backup server data not client data.
I recommend to use a 3-tier architecture using Java EE.
You could use a Timer Service then to trigger the backup.
Though usually a backup program is an independent program e.g. started by a cron job on the server. But again: you'll need a server to do this properly, not just a shared folder.
Here is what I would suggest. Instead of having all clients wake up at the same time and trying to perform the backup, stagger the time at which they wake up.
So when a client wakes up
- It will check some table in your DB (MYSQL) to see if a back up job has completed or is running currently. If the job has completed, the client will go on with its normal duties. You can decide how to handle the case when the job is running.
- If the client finds that the back up job has not been run for the day, it will start the back up job. At the same time will modify the row to indicate that the back up job has started. Once the back up has completed the client will modify the table to indicate that the back up has completed.
This approach will prevent a spurt in network activity and can also provide a rudimentary form of failover. So if one client fails, another client at a later time can attempt the backup. (this is a bit more involved though. Basically it comes down to what a client should do when it sees that a back up job is on going).

Slow Startup for Java Web Start Application

I'm using the Netbeans IDE to develop a Java Web Start application that will launch from the web and then use the EclipseLink JPA to access a remote MySQL database. I'm using the Swing Application Framework to manage life cycle for my app.
When I launch the application from Netbeans it takes about 7 second for my application to load, but when I use the Netbeans IDE to create a Web Start distribution package (with the JAR and JNLP files) it takes about 60 seconds to launch. Also, the "verifying application"/"downloading application" progressbar window seems to run every time I launch the app even though a copy of it has already been cached.
From the users point of view, one first sees my splash screen for 1 to 2 seconds, then the "verifying application"/"downloading application" progressbar window for 5 to 20 seconds and then nothing for a good 40 seconds before the application launches.
The app code is written such that it should show itself BEFORE the JPA starts loading the persistence unit (so I doubt that's the problem), but I thought I'd mention it just in case.
Update: Method createEntityManagerFactory Slow With Web Start
Having looked into this further, I've found that the method createEntityManagerFactory--which is necessary for EclipseLink to connect to MySQL--takes aboutn 5 seconds to execute when I run the applicaiton from Netbeans or when I remotely log in to my server to launch the JNLP there, but when I run the application via the web the same line takes 35 seconds (hugely delaying startup). Interestingly, this time gets even worse as my internet connection speed gets worse. Below is a copy of the JNLP file I'm using.
Does anyone have any idea what may be causing such a delay?
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
MyApp
My App Vendor
My App Description
MyApp
I'm not sure if you solved this yet. But, I repeated your problem and fixed it. All I had to do was switch from Eclipselink to Toplink. That simple. Startups went from 1 to 2 minutes down to 5 seconds. I even traced the app and found the same hanging during the init of eclipselink.
Hope it helps.
First of all, in the Java Control Panel enable ALL logging. This will allow you to see exactly what Java WebStart is doing (but much of it doesn't make much sense unless you have access to the WebStart source).
My guess from your description is that you have one or more unreachable DNS servers in your network configuration AND you want to access a network resource that isn't available.

Memory usage in Google App Engine

I am a bit confused. I wrote a Java stand alone app and now I want to use GAE to
deploy it on the web and on the way also to learn about GAE.
In my application, I read data from file, store it in memory, process it, and then store the results in memory or file.
I understand that now I need to store the results in the GAE's data store, which is fine. So I can run my program independently on my computer, then write the results to file, and then use GAE to upload all the results to the data store, and then users can query it. However, is there a way that I can transfer the entire process into the GAE application? so the application reads data from file, do the processing (use the memory on the application server and not my computer - needs at least 4GB of RAM), and then when it's done (might take 1-2 hours), writes everything to the GAE data store? (so it's an internal "offline" process that no users are involved).
I'm a bit confused since Google don't mention anything about memory quota.
Thanks!
You will not be able to do your offline processing the way you are envisioning. There is a limit to how much memory your app can use, but that is not the main problem. All processing in app engine is done in request handlers. In other words, any action you want your app to do will be written as if it is handling a web request. Each of these handlers is limited to 30 seconds of running time. If your process tries to run longer, it will get shut down. App engine is optimized for serving web requests, not doing heavy computations.
All that being said, you may be able to break up your computational tasks into 30 second chunks and store intermediate results in the datastore or memcache. In that case you could use a cron job or task queue (both described in the app engine docs) to keep calling your processing handlers until the data crunching was done.
In summary, yes, it may be possible to do what you want, but it might not be worth the trouble. Look into other cloud solutions like Amazon's EC2 or Hadoop if you want to do computationally intensive things.

Java Applet starts up very slowly for some users?

[UPDATE: I forgot to add that this 30 sec. freezing problem only happens the first time I try to load a file from the server. Subsequent loads are very quick. Maybe some strange reverse DNS lookup? I am hosting on Google's appengine.]
I started a little project recently called http://www.chartle.net which is build around an applet.
Startup time is an important factor in the user's experience of an applet. I collect statistics and am shocked that I find often very long startup times (factor 50 to 100 higher then necessary)
The applet starts in 1-3 seconds depending on the speed of your computer and connection. Still for some users it takes up to 100 sec.
I have mixed results from my own tests. Mostly it is very fast but sometimes freezes the browser for a long time and the Java console doesn't tell me why. Best guess is, that it stalls when loading a saved chart.
Please help me figuring this out - best test by opening an already saved chart (click on one of the 'create' links at http://www.chartle.net/gallery)
Cheers,
Dieter
This is generic help rather than specific for your demo (which loaded fine for me in a few attempts).
Freezing applets
In the JDK bin directory there is a very handy programme called jstack. Refresh your browser window until it crashes and then run:
jstack *process_id*
This will give you the stack trace of any frozen Java process. If Java is not a separate process then you can use the browser's process (eg for Opera).
The following few problems were/are common for me:
I reccommend you use invokeLater rather than invokeAndWait on the init method (although you can't do this if you use start/stop methods)
Opera's custom java plugin acts very poorly...
Deadlocks caused by synch blocks and invokeAndWait's
Slow applets
Possibly the browser is fetching resources from the server, unable to use the jar file?
It may be that only the old plugin causes these problems. That means basically all people running on OSX and other users with Java prior to 1.6_update_10.
So, I would really appreciate people with such setups to watch their Java console and describe the first startup behaviour.
Cheers,
Dieter

Categories

Resources