Problem: I have a standalone Java app (henceforth known as "the agent") that runs as a service on internal company servers. It acts as a remote agent for some central servers. As the agent gets deployed in more places, managing them is getting more complicated. Specifically: pushing updates is painful because it's a fairly manual process, and getting access to the logs and other info about the environments where the agents are running is problematic, making debugging difficult. The servers under discussion are headless and unattended, meaning that this has to be a fully automated process with no manual intervention, hence Java Web Start isn't a viable solution.
Proposed solution: Make the agent phone home (to the central servers) periodically to provide agent status and check for updates.
I'm open to other suggested solutions to the problem, but I've already got a working prototype for the "status and self-updates" idea, which is what this question is focused on.
What I came up with is actually a separate project that acts as a wrapper for the agent. The wrapper periodically calls the central server via HTTP to check for an updated version of the agent. Upon finding an update, it downloads the new version, shuts down the running agent, and starts the new one. If that seems like an odd or roundabout solution, here are a few other considerations/constraints worth noting:
When the wrapper gets a new version of the agent, there may be new JAR dependencies, meaning class path changes, meaning I probably want to spawn a separate Java process instead of fiddling with ClassLoaders and running the risk of a permanent generation memory leak, which would require manual intervention--exactly what I'm trying to get away from. This is why I ended up with a separate, "wrapper" process to manage the agent updates in my prototype.
Some servers where the agents are deployed are resource-limited, so any solution needs to be low on CPU and memory usage. That makes me want a solution that doesn't involve spinning up a new JVM and is a stroke against having a separate wrapper process.
The agent is already deployed to both Windows and RHEL servers, so the solution must be cross-platform, though I wouldn't have a problem duplicating a reasonable amount of the process in batch and bash scripts to get things rolling.
Question: As stated, I want to know how to make a self-updating Java app. More specifically, are there any frameworks/libraries out there that would help me with this? Can someone with experience in this area give me some pointers?
If your application is OSGi based, you could let OSGi handle bundle updates for you. It is similar to the wrapper approach you suggest, in that the OSGi container itself is "the wrapper" and some of it won't be updated. Here's a discussion on this
Different solution: use (and pay for) install4j. Check out the auto-update features here
No need for wrapper (save memory) or java web start (adds more restrictions on your application), simply let a thread in you application check periodically for updates (e.g. from cloud) and download updates if available, then code these two calls in you application:
launch a shell script (.sh or .cmd) to update your artifacts and launch your application after few seconds pause in the script(to avoid having two instances of your application at the same time).
Terminate your application (first instance)
The script can overwrite needed artifacts and re-launch your application.
enjoy !
Have a look at Java Web Start.
It is technology that's been part of Java since... 1.5? maybe 1.4? and allows deployment and install of standalone Java-based apps through a web browswer. It also enables you to always run the latest app.
http://www.oracle.com/technetwork/java/javase/overview-137531.html
http://en.wikipedia.org/wiki/JNLP#Java_Network_Launching_Protocol_.28JNLP.29
also see this question: What's the best way to add a self-update feature to a Java Swing application?
It appears as though Webstart is the only built in way to do this at the moment.
Related
I would like to implement a java application (server application) that can download a new version (.jar file) from a given url, and then update itself at runtime.
What is the best way to do this and is it possible?
I guess that the application can download a new .jar file and start it. But how should I do the handover, e.g. know when the new application is started and then exit. Or is there a better way to do this?
The basic structure of a solution is as follows:
There is a main loop responsible for repeatedly loading the latest version of the app (if required) and launching it.
The application does its thing, but periodically checks the download URL. If it detects a new version it exits back to the launcher.
There are a number of ways you could implement this. For example:
The launcher could be a wrapper script or binary application that starts a new JVM to run the application from a JAR file that gets replaced.
The launcher could be a Java application that creates a classloader for the new JAR, loads an entrypoint class and calls some method on it. If you do it this way, you have to watch for classloader storage leaks, but that's not difficult. (You just need to make sure that no objects with classes loaded from the JAR are reachable after you relaunch.)
The advantages of the external wrapper approach are:
you only need one JAR,
you can replace the entire Java app,
any secondary threads created by the app, etc will go away without special shutdown logic, and
you can also deal with recovery from application crashes, etc.
The second approach requires two JARs, but has the following advantages:
the solution is pure Java and portable,
the changeover will be quicker, and
you can more easily retain state across the restart (modulo leakage issues).
The "best" way depends on your specific requirements.
It should also be noted that:
There are security risks with auto-updating. In general, if the server that provides the updates is compromised, or if the mechanisms for providing the updates are susceptible to attack, then auto-updating can lead to a compromise of the client(s).
Pushing a update to a client that cause damage to the client could have legal risks, and risks to your business' reputation.
If you can find a way to avoid reinventing the wheel, that would be good. See the other answers for suggestions.
I am currently developing a JAVA Linux Daemon and also had the need to implement an auto-update mechanism. I wanted to limit my application to one jar file, and came up with a simple solution:
Pack the updater application in the update itself.
Application: When the application detects a newer version it does the following:
Download update (Zipfile)
Extract Application and ApplicationUpdater (all in the zipfile)
Run updater
ApplicationUpdater: When the updater runs it does the following:
Stop the Application (in my case a daemon via init.d)
Copy the downloaded jar file to overwrite current Application
Start the Application
Cleanup.
Hope it helps someone.
I've recently created update4j which is fully compatible with Java 9's module system.
It will seamlessly start the new version without a restart.
This is a known problem and I recommend against reinventing a wheel - don't write your own hack, just use what other people have already done.
Two situations you need to consider:
App needs to be self-updatable and keep running even during update (server app, embedded apps). Go with OSGi: Bundles or Equinox p2.
App is a desktop app and has an installer. There are many installers with update option. Check installers list.
I've written a Java application that can load plugins at runtime and start using them immediately, inspired by a similar mechanism in jEdit. jEdit is open source so you have the option of looking to see how it works.
The solution uses a custom ClassLoader to load files from the jar. Once they're loaded you can invoke some method from the new jar that will act as its main method. Then the tricky part is making sure you get rid of all references to the old code so that it can be garbage collected. I'm not quite an expert on that part, I've made it work but it wasn't easy.
First way: use tomcat and it's deploy facilities.
Second way: to split application on two parts (functional and update) and let update part replace function part.
Third way: In your server appliction just download new version, then old version releases bound port, then old version runs new version (starts process), then old version sends a request on application port to the new version to delete old version, old version terminates and new version deletes old version. Like this:
This isn't necessarily the best way, but it might work for you.
You can write a bootstrap application (ala the World of Warcraft launcher, if you've played WoW). That bootstrap is responsible for checking for updates.
If an update is available, it will offer it to the user, handle the download, installation, etc.
If the application is up to date, it will allow the user to launch the application
Optionally, you can allow the user to launch the application, even if it isn't up to date
This way you don't have to worry about forcing an exit of your application.
If your application is web based, and if it is important that they have an up to date client, then you can also do version checks while the application runs. You can do them at intervals, while performing normal communication with the server (some or all calls), or both.
For a product I recently worked on, we did version checks upon launch (without a boot strapper app, but before the main window appeared), and during calls to the server. When the client was out of date, we relied on the user to quit manually, but forbid any action against the server.
Please note that I don't know if Java can invoke UI code before you bring up your main window. We were using C#/WPF.
If you build your application using Equinox plugins, you can use the P2 Provisioning System to get a ready-made solution to this problem. This will require the server to restart itself after an update.
I see a security problem when downloading a new jar (etc.), e.g., a man in the middle attack. You always have to sign your downloadable update.
On JAX2015, Adam Bien told about using JGit for updating the binaries.
Sadly I could not find any tutorials.
Source in German.
Adam Bien created the updater see here
I forked it here with some javaFX frontend. I am also working on an automatic signing.
I've created my first Play application. Which is the most suitable deployment method for production? Should i copy the whole project to the production server and run play start? or should i make a war out of my application and deploy in tomcat / jboss? Which is the most recommended way? Getting confused with it comparing to its rails type of behavior. Note that this is supposed to be a big data application and also it may server loaded requests later on. So we are thinking of scalability, availability, performance aspects too. This application is decided to be deployed in a cloud.
Thanks.
As others have stated, using the dist command is the easiest way to deploy Play for a one-off application. However, to elaborate, I have here some other options and my experience with them:
When I have an app that I update frequently, I usually install Play on the server and perform updates through Git. Doing so, after every update, I simply run play stop (to stop the running server), sometimes I then run play clean to clear out any potentially corrupted libraries or binaries, then I run play stage to ensure all prerequisites are present and to perform compilation, and then finally play start to run the server for the updated app. It seems like a lot, but it is easy to automate via a quick bash script.
Another method is to deploy Play behind a front-end web server such as Apache, Nginx, etc. This is mostly useful if you want to perform some sort of load balancing, but not required as Play comes bundled with its own server. Docs: http://www.playframework.com/documentation/2.1.1/HTTPServer
Creating a WAR archive using the play2war plugin is another way to deploy, but I wouldn't recommend it unless you are giving it to someone who already has a major infrastructure built upon these servlet containers you mentioned (as many large companies do). Using a servlet containers adds a level of complexity that Play is supposed to remove by nature (hence the integrated server). There are no notable performance gains that I am aware of using this method over the two previously described.
Of course, there is always the play dist which creates the package for you, which you upload to your server and run play start from there. This is probably the easiest option. Docs: http://www.playframework.com/documentation/2.1.1/ProductionDist
For performance and scalability, the Netty server in Play will function very adequately to exceptional for what you require. Here's a reputable link showing Netty with the fastest performance of all frameworks and a "stock" Play app as coming in somewhere in the middle of the field, but way ahead of Rails/Django in terms of performance: http://www.techempower.com/blog/2013/04/05/frameworks-round-2/.
Don't forget, you can always change your deployment architecture down the road to run behind a front-end server as described above if you need more load balancing and such for availability. That is a trivial change with Play. I still would not recommend the WAR deployment option unless, like I said, you already have a large installed base of servlet containers in use that someone is forcing you to serve your app with.
Scalability and performance also has a lot more to do with other factors as well, such as your use of caching, the database configuration, use of concurrency (which Play is good at) and the quality of the underlying hardware or cloud platform. For instance, Instagram and Pinterest serve millions of people every day on a Python/Django stack which has mediocre performance by all popular benchmarks. They mitigate that with lots of caching and high-performing databases (which is usually the bottleneck in large applications).
At the risk of making this answer too long, I'll just add one last thing. I, too, used to fret over performance and scalability, thinking I needed the most powerful stack and configuration around to run my apps. That just isn't the case any more unless you're talking like Google or Facebook scale where every algorithm has to be finely tuned as it will be bombarded a billion times every day. Hardware (or cloud) resources are cheap but developer/sysadmin time isn't. You should consider ease of use and maintainability for deployment of your app over raw performance comparisons, even though in the case of Play the best performing deployment configuration is arguably the easiest option as well.
You don't need to use Play's console for running application, it consumes some resources and it's main goal is fast launch while development stage.
The best option is using dist command as described in the doc. Thanks to this, you don't even need to install Play on the target machine, as dist creates ready to use stand-alone application containing all required elements (also build-in server, so you don't need to deploy it with WAR in any container).
If you planning to use a cloud you should also check offers ie. from Heroku, or CloudBees, which allows you to deploy your application just by... pushing changes via git repository, which is very comfortable way, check the documentation's home, scroll down to links: Deploying to... for more details.
I've been researching Apache's commons-daemon and it seems pretty cool: basically its an API as well as a library that helps register your JAR with the underlying OS so that it can be started and stopped like a daemon service. Additionally, it intercepts OS signals that would normally kill your app and instead gives you a chance to shutdown politely.
So it's got me wondering, if given the choice between deploying your business logic inside EJBs and wrapping them in a container like OGS or JBoss, why not just create a daemon JAR that listens on a port and responds to client requests?
Is it just the benefit of all the features/services that an app container provides out of the box (security, logging, etc.), or are there times when it would be favorable to choose a daemon over an app container/EJB solution?
Basically, what I'm asking if: when is it more appropriate to use an app container/EJB solution, and when is it more appropriate to use commons-daemon to help build a system-level service (in Java)?
Disclaimer: just interested in these two choices, I am aware that other solutions exist (web containers, ESBs, OSGi, etc.). But for the purposes of this question I am only interested in hearing the reasoning between app container or daemon solutions. Thanks in advance!
Why don't you look at it like System level (daemon) vs Application level (in container)?
This will give more or less clear distinction (especially if worked with Linux some time).
For Daemon:
has its own life cycle (you can start and stop it separately);
different privileges (could be run under different user);
use case is something like CRON, MailServer, synchronization and any system-level service.
For Container:
managed app (by some privileged user via Container console);
plenty of out-of-the-box features (which you'd already mentioned);
use case some general case business application.
Well the simple answer is yes, the app server (Glassfish or JBoss) give you plenty of nice things that you would have to implement or setup yourself in a plain Java SE app.
However it is not so black and white, and you can get a lot of the application server goodness with very little effort, I am in the process of writing a blog series on exactly this topic.
My reason for not using an app server, was that we had a project for a widely distributed software product, and we wanted to avoid having to patch and maintain thousands of application server instances!
However if your app will be running in one place, there is little reason to go Java SE.
I have a jar file which contains two Java classes. Using the javamail API I have developed these classes to read and edit my mail, then send to another mail id.
I am able to execute this through my standalone system via Eclipse. Now I want to host this jar file somewhere remotely so that it would fetch the data in real time and execute the job regularly. I have contacted couple of hosting sites and they are saying that they require a war file instead.
Does anyone have any suggestions to this problem?
To give you another point of view and to be constructive, I would go with embedding your jar into a war application and you get some things for free, the most important I think is that you gain a managed application lifecycle so with a standard web application context listener you can start and stop your program in a managed way. Besides you have more hosting options and it is less work.
Good luck with that.
As I don't know of any services specifically for plain execution of executables, your best bet is probably getting a cheap VPS. With some searching you can probably find one that would work for around $5 USD/month. For a single simple app you'd only need around 128MB of memory.
Pick one up, install Java (whatever Linux distro you get probably has OpenJDK in the repositories), copy your files over, and set up a cron job to run the executable at a set interval.
For easier administration, install something like webmin and use that to configure the cron job. The command would likely just be java -jar /path/to/my/App.jar, and you can use the web interface to configure the intervals for the command to be executed.
For an app like this, I would avoid anything related to a war file. You aren't making an application with a web interface (like a PHP app or some such) so it really wouldn't be appropriate. You would have to write some extra code to make it compatible with a container like Tomcat, and on top of that the memory requirements for running the application server would be a lot higher.
I need to develop a game server that will run periodically (e.g., triggered by a CRON job every five minutes or hour as appropriate). Once started up, the server will access all of the current game state (fetched through REST from the game's data servers (Stackmob, Parse or similar), do the processing of player actions, and then POST the results back to the data server. In other words, it will be doing a lot of HTTP requests, but does not itself necessarily need to be a web service.
I've been considering multiple ways of developing this.
I do not feel for setting up a server myself, so I need to find a service to run this on that permits the workflow I would like.
The game engine is Java, so something that works neatly with that.
Will need to GET and POST data files, so access to static files would be needed.
Most of the services that exist which provide something similar to what I require are directed at web services - which generally means that one needs to jump through some hoops to get things to work.
Google App Engine, for instance, would require that I implement this using backends (since the game server could potentially run for more than 60 seconds), and isn't particularly happy with the idea of static files.
Amazon EC2 would seem easier to develop on (again by building a web service frontend, of course), but there seems to be relatively poor support for CRON.
Generally speaking, it feels like I want to shoot some sparrows with a slingshot, but all the services are offering me cannons. Are there any alternative platforms/frameworks beyond the big two mentioned above that would be suitable for something like this?
You could try Heroku. They support Java. If you created a project that used a single worker dyno then the hosting would be free (see link).
The process would be running continuously, so you might want use a Timer for periodic execution. You could also use Quartz, but it might be overkill.
Edit:
Here's some links that might help get started:
Running non-web Java processes on Heroku
Heroku Java quickstart - this is for a web app ('web dyno') rather than a 'worker dyno', but it may help.
java.herokuapp.com has links to some example projects (again web apps rather than workers)
How about using EC2, but rather than putting the scheduler in the instance (which won't work because the instance can go away at any time), putting it in AWS? Like this guy:
http://alestic.com/2011/11/ec2-schedule-instance
Alternatively, if you manage your EC2 instances through Ylastic, it looks even easier:
http://blog.ylastic.com/scheduling-tasks-on-the-aws-cloud
Although you'll have to pay for Ylastic as well as EC2, i imagine.
I found a nifty way of writing something like this in Groovy with Maven. You can write a multithreaded Groovy script to pull the stats, do the updates, etc. and then have maven's assembly plugin assemble the whole thing into a self-contained, executable jar file that can be called by a CRON job. One nice thing about Groovy is that its syntax allows you to do this:
def google = "http://google.com".toURL().text
which will turn the string into a URL and handle all of the details of turning the URL into a HTTPURLConnection and getting the raw text.
You could develop the app as a standalone Java program first and then worry about where to deploy it later.
To develop the app you could write a simple Java program that uses HtmlUnit to talk to the external web services. The job could be internally started via Quartz. If you really wanted to start the job externally via CRON, you could have CRON run the app, passing in args. The app would then run and exit.
Alternatively, you could have the app always running and have cron run a bash script that triggers the job in some way.
Essentially, all you need to deploy is a Unix machine so you could use AWS.