I'm trying to figure out the optimum way to develop and release a fairly simple web application, and I'm running into several problems. I'll outline the decisions I've made, because somewhere I've clearly gone off the rails.. Hugely grateful for any help!
I have what I think is a fairly simple web application. It contains a couple of jsps that reference a couple of java beans, and the usual static html, js, css and images.
Decision 1) I wanted to have a clear and clean release procedure, such that I could develop on my local machine and then release reliably to a production machine. I therefore made the decision to package the application into a war file (including all the static resources), to minimize the separate bits and pieces I would need to release. So far so good?
Decision 2) I wanted things on my local machine to be as similar as possible to the production environment. So in my html, for example, I may have a reference to a static file such as http://static.foo.com/file . To keep this code working seamlessly on dev and prod, I decided to put static.foo.com in my /etc/hosts when developing locally, so that all the urls work correctly without changing anything.
Decision 3) I decided to use eclipse and maven to give me a best practice environment for administering and building my project.
So I have a nice tight set up now, except that:
Every time I want to change anything in development, like one line in an html file, I have to rebuild the entire project and then wait for tomcat to load the war before I can see if it's what I wanted. So my questions are:
1) Is there a way to connect up eclipse and tomcat so that I don't have to rebuild the war each time? ie tomcat is looking straight at my actual workspace to serve up the static files?
2)I think I'm maybe making things harder by using /etc/hosts to reflect production urls - is there a better way that doesn't involve manually changing over urls (relative urls are fine of course, but where you have many subdomains, say one for static files and one for dynamic, you have to write out the full path, surely?)
3) Is this really best practice?? How do people set things up so that they balance the requirement for an automated, all-encompassing build process on the one hand, and the speed and flexibility to be able to develop javascript and html and css quickly, as quickly as if one just pointed apache at the directory and developed live? What do people find works?
Many thanks!
Edit: Thanks all for your great responses! If I could mark them all right, I would.. This has really helped me out. What I'm hearing is that best practice is to conserve the structure of the webapp in development, and run it in as close an environment to production as possible. Seems like the differences between people are the extent to which people are prepared to hot deploy resources into the servlet container, circumventing the build process for a little extra speed or convenience. That makes sense. Thanks again.
This is much like what I have to do at work, although we use ant (for now?). Also, while I use an IDE (or two), I refuse to have one as part of my build process, EVER. Somebody needs to be able to understand and tune your build.
Is there a way to connect up eclipse
and tomcat so that I don't have to
rebuild the war each time?
1) I think you're relying too much on your IDE. Usually I have an Ant build.xml that has a couple of tasks: one is "build war" the other is "update jsps." Building the war compiles all the code, packages it, deploys it to Tomcat and restarts everything. Updating the jsps doesn't restart the server, it's just a straight copy from my local files to Tomcat's deployed instance. No restart necessary since they're JSPs. Takes about half a second.
where you have many subdomains, say
one for static files and one for
dynamic, you have to write out the
full path, surely?
2) No way, Jose. So you mean any time the server name changes, you have to recompile your code? If you need to support dynamic URLs you might just want to bite the bullet and take a look at a framework to do the heavy lifting for you. I'm partial to Stripes (which supports dynamic URL rewriting out-of-the-box)... there are others.
To answer #1, I would suggest the following:
Spend some time learning maven to build your .war without eclipse. It's not that hard with the proper archetype. See here for more details: http://maven.apache.org/guides/mini/guide-webapp.html
Maven can generate eclipse projects either through mvn eclipse:eclipse or by using the m2 plugin
For deployment to your local machine and to production, use the maven cargo plugin. http://cargo.codehaus.org/Maven2+plugin and http://blank.jasonwhaley.com/2010/03/automated-deployment-with-cargo-drive.html
To answer question #2, there's nothing wrong with modifying your /etc/hosts file to mimic production. Just have a quick script that lets you add/remove those entries and flushes your dns cache. I do exactly that all of the time. (be sure to make your browser clear its cache frequently also through its relevant setting).
To answer question #3) yes this is how you should be doing things. Each build should result in a single deployable artifact that you can deploy to any of your environments in one step. You need to make sure you can do this without your IDE and use the IDE only as a tool to help you during the development phase.
Others have already answered you, I'll just comment on this (this too long for a comment btw so I make it an answer):
Every time I want to change anything
in development, like one line in an
html file, I have to rebuild the
entire project and then wait for
tomcat to load the war before I can
see if it's what I wanted.
If you change one line in an html file, there's no need to rebuild the entire project.
Note that I always rebuild the full .war and redeploy my .war but this takes less than two seconds (less than one second to rezip the .war [that's really what a .war is, a zipped file] and less than one second to redeploy it) because:
you don't need to recompile your entire project when you simply change one line in an html file
Same when you change one .java file: you can simply recompile that one file and re-war.
I wrote my own Ant build file from scratch (no Maven here) and I've got several targets. I can force a "clean build", that shall re-compile everything but typically I'm simply repackaging and redeploying the .war
You can check it for yourself: build a .war, unzip it in, say, directory dir1, then modify one .html (or one .java/.class file) and build a new .war and unzip that new .war in, say, dir2.
Then compare dir1 and dir2: now fix your build process so that you can create that second .war without needing to recompile everything.
Changing one .html, .java, .jsp, .css, .js / whatever file and redeploying a new .war should be a matter of seconds (less than two seconds if you didn't throw the kitchen sink in your Webapp).
Note that on the very same project, another developer here prefers to "hot deploy" / replace the files directly in the exploded webapp (I prefer to redeploy a .war everytime and because my complete repackage/redeploy takes less than two seconds I'm fine with it that way).
You don't need to reconstruct war file if your project is an Dynamic Web App in Eclipse and configured Tomcat server properly. Follow the below instructions:
1) Check out the below of how to configure tomcat server with eclipse:
http://greatwebguy.com/programming/eclipse/make-eclipse-and-tomcat-play-nice-together/
2) Use relative paths for your application but not absolute paths.
3) If you follow the above 2 steps properly then you have a best environment for development.
during development, you should configure eclipse and tomcat so that no rebuild/redeloy is required. just modify html/css/jsp etc, save, and refresh browser to see the result.
but before deploying to production site, you should do a clean full build and test it carefully.
domains: they should be in a config file; dev and prod should have different config files.
Related
Typically on a Java project using Maven, you place client-side source files in src/main/webapp. This directory includes your html, css, scripts, images, etc.
However, many grunt projects tend to place these files on the root. It's as if Grunt was designed with the idea that your project is the client-side application, not part of a large server-side project, such as using Java with Spring.
Given a Java project using Maven, where would be the best place to put your web-related source files?
Do you place them in src/main/webapp?
Do you make another directory altogether, such as src/web, and then on a build, copy everything to src/main/webapp?
My goal is to make the client-side build tool as transparent as possible. I guess the ideal case is to simply work from src/main/webapp as I have been doing all along - this is pretty unobtrusive to the way my project is currently setup.
However, if I work from src/main/webapp, I know that I will need to distinguish between src and build directories somehow anyway. I'm sure my html files can stay where they are, but there's definitely going to be a conflict of interest here with javascript and css files, and maybe images too.
Does it make sense to literally have a 100% separate source folder from src/main/webapp? Is there a way to do continuous building/copying/syncing of the application as you modify files, from src/web to src/main/webapp? Will this be inconvenient and cause frustrations and problems? Is it slow?
I would like any advice on the subject. Thank you.
Option 1: Put UI resources in src/main/webapp:
This is a quick-n-dirty solution popular on Github. It helps to keep example projects small and concise. In this case, usually people put the package.json, bower.json, Gruntfile.js and .bowerrc in the maven project's root directory, where the .bowerrc says to install components into src/main/webapp/bower_components.
If you have a task that minifies/transforms resources, the transformed resources can go to a new directory like src/main/webapp/dist. Then use something like grunt-usemin to make your app use the resources in the dist directory.
If your application will use a security framework (like, say, Spring Security), you might want all your resources in src/main/webapp so that the security framework can regulate access to those resources. However you can still achieve this using Option 2 by having a grunt task that copies the necessary resources into src/main/webapp.
Pros
Common approach on GitHub. Keeps everything in one project.
Cons
The version of files getting served by your server is a copy of those in src/main/webapp, so changing a file in src/main/webapp isn't immediately reflected in your deployed app. To get hot reloading, you need to use something like grunt-contrib-* stuff.
Frontend and backend code is all mixed together. Harder for two different teams to work on the code base.
Option 2: Put UI resources in a separate project:
You can achieve better project organization by keeping the UI and backend in completely separate projects. In this case the UI and the maven project would likely be sibling directories.
Then when deploying to a server, you either:
deploy both projects separately (call them myproject-ui and myproject-services). The javascript in myproject-ui makes RESTful service calls to myproject-services/**.
Use a grunt task to copy necessary resources to src/main/webapp, then deploy your (one) project.
Pros
The backend and the frontend are separated as much as possible
The frontend maintains the workflow and directory structure found in most client-side example apps, as you mentioned
"Deploying" the UI is a simple matter of creating a symlink from your server's deployment directory to you UI source code. Changing any UI code is automatically reflected in the deployed app.
You can have several different UIs deployed simultaneously (maybe you have myproject-admin-ui and myproject-user-ui). You could hit either one simply by visiting
http://localhost:8080/myproject-admin-ui
http://localhost:8080/myproject-user-ui
I have created many hooks for Liferay with Eclipse Java EE IDE for Web Developers.
The problem is that when I create a hook is deployed inside the webapps folder.
My webapps folder tree:
There are many hooks. How can how can I move them into a new folder inside webapps?
in order to have something like this:
webapps
calendar-portlet
hooks
blogs-hook
breadcrumb-hook
dockbar-hook
login-hook
blogs-hook
sitepages-hook
truefalsetocheckbox-hook
userinfo-hook
wiki-hook
marketplace-portlet
my-theme
resources-importer-web
ROOT
Any help is greatly appreciated! Thank you so much!
I suppose you're speaking about Tomcat's (or whatever appserver you're using) webapp folder. You cannot merge different apps into one folder: it has to keep the structure on deploy.
So what you can do is to aggregate those hooks on source level (i.e., create only one hook project in Eclipse with the Liferay IDE so you can put everything there). So you could put them into the same plugin.
Btw I would discourage using several hooks, because they might occasionally overwrite each other's behaviour. Imagine you have some sort of property defined in a-hook and b-hook in a properties file. You cannot tell which one will be used after the deployment.
Adding a bit of reasoning to rlegendi's answer, here's what you might consider:
If you use Liferay's auto-deploy mechanics, e.g. through the plugins-sdk deployment, Liferay places the hooks in tomcat's webapps folder (in case you use tomcat) and you'd also have to override this.
You might be able to configure tomcat to monitor different directories, even on the same virtual host (I've only tried this for different virtual hosts and don't know if this will work properly with Liferay), but the auto-deploy problem would persist, you'd basically have to change the whole deployment logic, or deploy manually to tomcat as you'd do with "big iron" appservers.
If you're happy about the development-time separation between the hooks, but would like some better runtime overview, you might want to merge different hooks into one for deployment. However, you'd have to do this on your own.
But my final recommendation is to not care too much about what's in the webapps directory anyway - you shouldn't rely on directory listings of this directory for knowing what's running on your appserver. Rather use proper deployment techniques that monitor your implementations, and limit access for manual changes. E.g. deploy your applications scripted and install a system that alerts you should required components not be available and running.
I have a Swing/Java application that is being used for clients and has weekly updates.
The problem is the setup just lay out the classes and their respective directories and on the update I just update the classes.
I want to do a single jar containing all the classes, but I'm not sure how would I be able to update it...
Also some clients need some updates during the week where only one or two classes would be updated.
What is the right way of doing this ?
Im using Eclipse.
EDIT: I know how to create the jar, I just dont know how to dynamically update it.
I would suggest you look into Java WebStart which is designed to do exactly what you need.
You need to first create the finished deployment and then you can look into creating a JNLP file for the deployment, and a way to put all the files to a web server for the user to access. For now just let the user download the whole thing every time you update. When you get more experienced you can look into how you can make incremental updates.
I would strongly recommend including a build number or timestamp in the deployment paths so a jar file is not cached incorrectly in the client by accident.
The general way of doing this, even if only small changes were made, would be to repackage your JAR after each update and give that JAR to a user. The user would replace the existing JAR. How you produce your JAR is up to you. Many IDEs support it, you could write a shell script, use existing build systems like ant or maven or even make, whatever. (See edit below)
If your JAR is very large and deployment is cumbersome, you may be able to split your project into smaller subcomponents, each with their own JAR, and only redistribute the ones containing changes. That may or may not be a major refactoring for you, and it might not even be appropriate. For most small or average size projects, this is generally unnecessary.
As for deployment, there are also a zillion ways to make that easier on the user. You could just give them a JAR file. You could use e.g. install4j. You could use Java Web Start (although its kind of clunky). There are others.
Both install4j and JWS support automatically checking for updates. If you choose to support that feature, all you would need to do is update your distribution site, and users would receive updates automatically. That's also up to you.
But a short answer to your question is: If you have all of your classes packaged in a JAR, no matter how many classes change, you'll want to give the entire updated JAR to the user. The benefit that counters this cost is that a JAR is a nice, compressed, self-contained collection of your application/library's source, which makes management and deployment very convenient.
Edit: Responding to your edit where you specify that you are using Eclipse, Josh M gives you instructions in his comment on your answer. To add to his comment, to export a Runnable Jar you'll have to have a Run Configuration set up which, if you've been running your application in Eclipse already, you probably already have. If not you can create one in the Run menu.
Edit 2: Please refer to Thorbjørn Ravn Andersen's answer as well for some good JWS tips.
Our build/deploy process is very tedious, sufficiently manual and error-prone. Could you give proposals for improvement?
So let me describe our deployment strategy and build process.
We are developing system called Application Server (AS for short). It is essentially servlet-based web application hosted on JBoss Web server. AS can be installed in two "environments". Each environment is a directory with webapp's code. This directory is placed on network storage. Storage is mounted to several production servers where JBoss instances are installed. Directory is linked to JBoss' webapps directory. Thus all JBoss instances use the same code for environment. Configuration of JBoss is separate from environment and updated on per instance basis.
So we have two types of patches: webapp patches (for different environments) and configuration patches (for per instance configuration)
Patch is an executable file. In fact it is bash script with embedded binary rpm package. Installation is pretty straight-forward: you just execute file and optionally answer some questions. Important point is that the patch is not a system as a whole - it contains only some classes with fixes and/or scripts that modify configuration files. Classes are copied into WEB-INF/classes (AS is deployed as exploded directory).
The way we build those pathes is:
We take some previous patch files and copy them.
We modify content of patch. The most important part of it is RPM spec. There we change name of patch, change its prerequisite rpm packages and write down actual bash commands for backing up, copying and modifying files. This is one of the most annoying parts because we not always can get actual change-set. That is especially true for new complex features which are spanned among multiple change requests and commits. Also, writing those commands for change-set is tedious and error-prone.
For webapp patches we also modify spec for other environment. Usually they are identical excepting rpm package name.
We put all rpm related files to VCS
We modify build.xml by adding a couple of targets for building new patch. Modification is done by copypasting and editing.
We modify CruiseControl's config by copypasting project and changing ant targets in it
At last, we build a system
Also, I'm interested in any references on patch preparation and deployment practices, preferably for Java applications. I haven't succeed googling that.
The place I work had similar problems, but perhaps not as complex.
We responded by eliminating the concept of patch altogether. We stopped patching, and started simply installing the whole app (even if we do a just a small change).
We now have Cruise Control build complete install kits that happen to contain the build timestamp in the install-kit name. This is a Cruise Control build artifact.
Cruise Control autoinstalls them on a test server, and runs some automated smoke tests. We then run manual tests on the test server. Then we install the artifact on a staging, then production server.
Getting rid of patching caused some people to splutter, "isn't that wasteful if you're just changing a couple of things?" and "why would you overwrite all the software just to patch something?"
But the truth is that good source control, automated install-kit building, and one-step installation has saved us tons of time. It does take a few seconds longer to install, but we can do it far more repeatedly and with less developer labor.
I am working on a small team of web application developers. We edit JSPs in Eclipse on our own machines and then move them over to a shared application server to test the changes. I have an Ant script that will take ALL the JSPs on my machine and move them over to the application server, but will only overwrite JSPs if the ones on my machine are "newer". This works well most of the time, but not all of the time. Our update method doesn't preserve file change day/times, so it is possible that an Update on my machine will set the file day/time to now instead of when the file was actually last changed. If someone else worked on that file 1 hour ago (but hasn't committed the changes yet), then the older file on my PC will actually have a newer date. So when I run the Ant script it will overwrite their changes with an older file.
What I am looking for is an easy way to just move the file I am currently working on. Is there a way to specify the "current" file in an Ant script? Or an easy way to move the current file within Eclipse? Perhaps a good plugin to do this kind of stuff? I could go out to Windows Explorer to separately move the file, but I would much prefer to be able to do it from within Eclipse.
Add a target to your ant build file to copy a single jsp using a command line property definition as #matt b described.
Create a new external tool launch profile and use the "String Substitution Preferences" to pass in the reference to the active file in the editor (resource_name).
See Eclipse Help | Java Development User Guide | Reference | Preferences | Run/Debug | Launching | String Substitution
How would Ant know what file was "current"? It has no way of knowing.
You could pass the name of the file into your Ant script (by taking advantage of the fact that any arguments you pass into Ant with -D are automatically parameters in your script)...
ant -Dfile=myfile.jsp update
and have your script look something like...
<copy file=${myfile} todir="blah"/>
...but it would probably be a pain to constantly type in the name of the file on the commandline.
Honestly, the type of problem you've described are inevitable when you have multiple developers sharing an environment. I think that a better approach for you and your team long-term is to have each developer work/test on a local application server instance, before the code is promoted. This removes all headaches, bottlenecks, and scheduling trouble in sharing an app server with other people.
You should really use source control any time you have multiple people working on the same thing (well, you should use it any time regardless, but that's a different conversation). This way, when conflicts like this occur, the tool will know and make someone perform the merge so that no one's changes get lost. Then, the test server can run on a clean checkout from source, and each developer can also test the full application locally, because everyone's changes will be instantly available to them through the source repository.
I suggest you to use source control. I prefer Subversion. You can use CruiseControl to make the build automatically whenever someone commits new code.
The antrunner4e plugin is exactly what you are looking for -- see http://sourceforge.net/projects/antrunner4e/