At the very moment, i have 1 code base for all 7 client. This code is currently manually deploy. If i were to use Jenkins to deploy, any is there any documentation that points me on how to configure Maven/ant &Jenkins to solve the following 3 problems:
Each client has its own parameters and it is configure inside configuration files. Some is in text config properties, some has its very own parameter, some is inside XML and some is in a CSV. Hence, i maintain a separate folder for each client in SVN. Whenever i deploy, i make sure to copy the whole client configuration into the right path.
If new deployment, and since this is a console application, there is no web container to accept a war file and deflate. When i deploy a new whole application folder, and make sure the necessary open source jars are uploaded to the lib folder.
If existing upgrade deployment, i will only deploy the changed application jar, make sure to upload the new open source jars, any new folder(s) and keeping the existing folder untouch.
Item number 2 seems to me is a one time job. but i wonder if anything fancy in jenkins can make item 3 to behave like item 2 (eg: add but not replace)?
Jenkins is not a magic button. It has a lot of great plugins, including some aimed at various deployments, but when you have a set of custom requirements (and yours are custom), you've got to write your own scripts (bash/batch) to achieve that with a combination of plugins.
Related
I am unable to find another article that solves my problem but am happy to hear about one if you know the answer.
I have a RESTful service built in Java with Eclipse. It uses Spring and all of its bells and whistles.
After much work with configuration files, I am able to build the service into a war file, deploy it to my Tomcat webapps folder, and run it from standalone Tomcat. However, it still throws several errors on start when I try to start it from a service defined within Eclipse. The errors are the same errors that I was seeing before I got the config files correct for standalone Tomcat. The errors are about the inability to create the required beans because of references to properties that can't be found.
How did I get it running in Tomcat? I added a couple of .properties files to define properties needed by the beans that get launched at startup and then added a 'set CLASSPATH=...' line to the setenv.bat file in the Tomcat bin folder. This new line adds the properties files into the CLASSPATH. That seems to have fixed everything from standalone Tomcat.
So, my question is, "How do I make these same changes inside my Eclipse server?" I have added the properties folder with the property files under the config folder in Eclipse, just like it is in my native Tomcat folders, but I do not know how to modify the CLASSPATH string to specifically point to my .properties files (as I have done in the setenv.bat file for standalone Tomcat).
I think I understand the problem, but have no idea about the solution.
Thanks for listening.
Dave
Eclipse does not use any *.sh/*.bat files to startup Tomcat. In order to modify the way the server is started you have to open the configuration UI of the server by double-clicking on the server in the Servers view.
From there you will be able to modify the classpath of the system classloader through "Open launch configuration".
Another important setting is "Server path" which tells Eclipse, the value of $CATALINA_BASE. You can modify it only after removing all modules and cleaning the server. It is useful to set it to an easily accessible directory: this way you can verify directly that Eclipse didn't mess up your application deployment (sometimes it "forgets" to copy some libraries).
The "Configuration path" setting tells Eclipse where to find the files (but not subdirectories) that will be copied into $CATALINA_BASE/conf.
Remark: if your application requires you to add libraries to the top classloader, there is probably a problem in your project. The "missing" libraries should be added to WEB-INF/lib of your application instead: look into the "Deployment assembly" of your Eclipse project configuration.
I'm working with a dynamic web project in Eclipse and I'm planning on a Java JAX-RS RESTful back-end with a JavaScript single-page app front-end using a framework of the Angular/Durandal/Aurelia flavor. With that said, the typical way to deploy in the Java world is to bundle things up as a WAR file - which is essentially a JAR file. The trouble is, including the node_modules blows up the size of the WAR file considerably. On the other hand, I can execute 'npm install' after deployment. However, on my development machine, where I'm constantly deploying, that will take too much time. I would prefer if I can prepare the install directory on the web server with the 'npm install' modules and then deploy the WAR file on top of it. The trouble is, it seems the WAR file deployment enjoys wiping out folders if they are not contained in the WAR file.
I'm using GlassFish 4.1 application server. The ideal solution for me would be a way to 'cloak' directory in the WAR file by modifying the MANIFEST.MF file such that when it is expanded the cloaked directories are not overwritten. This would be the most parsimonious solution to my problem. However, I know of no cloaking manifest entries for JAR/WAR file manifest.
There may also be creative solutions arrived at using the 'npm link' command. Any suggestions are welcome.
Perhaps this, among other reasons, speaks to why once people gets started with npm on the client-side they start looking at node and express on the server-side. However, I'm not convinced they can't play nice together and I would like to keep the option of all the old school open source Java libraries at my disposal.
I know this question is almost two years old, but perhaps someone will still need an answer.
Put simply, you need to bundle your JavaScript. You should never be wrapping up your node_modules folder in a war, or even deploying it as-is to the server. Mainly because of exactly the issue you were having. It's... not the smallest.
In front-end development, you're expected to use a tool like webpack to gather up all your JS files into a single app.js file. This process will only take the actual files you directly require or import in your own JavaScript (and the files that those files require, etc), leaving out all the rest. Most importantly for this discussion, leaving out all your devDependencies!
Webpack will also bundle up files other than js. Importing your css files will tell webpack to also bundle those up, creating an app.css file alongside your app.js (though you will need to use an appropriate loader to tell Webpack what it means to import 'main.css').
Getting started is a fairly straightforward matter of adding a config file to your project, adding a new devDependency, and figuring out how to get your Java-based build tool to trigger the bundler. The frontend-maven-plugin, for instance, or the gradle-node-plugin.
These days, webpack and its ilk are even smarter. If your node_modules contains ES6 native modules, bundlers can perform tree-shaking on these files to only bundle the exports that are actually imported. This reduces the bundle size even more.
They can also pull out parts of the bundle into a separate file in order to create, say, a vendor.js file that contains the code for Angular, jQuery, etc. Or you can tell the bundler to treat those imports as external, meaning that they are assumed to have been included elsewhere in the web app. But this is all getting into more advanced features than you need at first. Just give webpack's getting started guide a go, see the difference it immediately makes to your war size, and go from there.
If you are using a nodejs build tool like Grunt (but probably not), then it's likely the devDependencies that's taking up so much space. If so, just copy your runtime dependencies out of node_modules.
If not: you don't have to deploy a .war; you can also deploy an 'exploded' directory. You could copy only changed files and touch .reload
Plus to mentioned above tools to pack NPM resources, let me also mention JNPM: https://github.com/OrienteerBAP/JNPM
It provides maven plugin (jnpm-maven-plugin) to download, filter and pack required NPM packages into your JAR/WAR. So in you case you should publish your client code as NPM package and then pack it into your WAR through this plugin.
Alright, so I have a web service that was created using an eclipse dynamic web project. It is currently shared on a CVS repository, but the versioning system used is irrelevant. At the moment, I have literally NEVER been able to pull this project out as is and get it working. It leads to countless errors that cannot be fixed. Every time I need to work on this webservice in a new machine I have to create an entirely new dynamic project, copy over the source files, add all the necessary libraries and make the deployment assembly work correctly again. After finally making it run I share the project as the same one, stop after a second, and then synchronize again (in a way tricking eclipse into thinking this was the shared project all along).
I feel like others must have run into this problem and found a way around it. So if you have a web service or any dynamic web project, what files do you share, and how do you successfully pull it from the repository and get it to run on another machine besides what I currently do now?
Your help is much appreciated,
-Asaf
Edit: After reading some of the responses I feel that this question is actually more specific to those who use WTP to create/test their web services. Just wanted to add the clarification.
Edit2: Let me also clarify that the other 20 or so projects not using WTP are shared just fine. I am able to pull and run them with no problem. Only web service projects are an issue.
In general, you want to check in everything that's not "derived" (generated or compiled - that's usually the contents of the bin directory or other place where your code is compiled/built into). For Eclipse Java projects, you want to include the .project, .classpath, .settings, and any other similar files that Web Tools might create for Dynamic Web projects. The Eclipse CVS client will ignore files marked as Derived so you shouldn't have to worry to much about it.
Without more detail about what kind of problems you've run into, it's not possible to guess what was causing them. My only guess is that perhaps you had different versions of Eclipse and/or the WTP (Web Tools Platform) plugins installed on the different machine. That's just a wild guess, but could explain some incompatibility when you check out the project from CVS.
Bottom line, checking in those .* files is the long recommended approach from Eclipse gurus. Maven can kind of change things, but you didn't mention it so I'm assuming you aren't using it.
I am primarily sharing my experience, may be you can find some help.
Conceptually speaking, the files which the IDE can generate itself while creating new project should not be pushed. I.e the IDE specific files should not be pushed. And everything which the IDE cannot generate on its own must be pushed.
Forexample in case of eclipse, following files should not be pushed:
.settings
build
.classpath
.project
For setting the project on new machine, first pull the files from server, and then create a project from IDE using pulled files.
EDIT: If your project has external jars/libraries, then you will have to add to the classpath manually. You could also push .classpath but that might give errors while creating a new project.
I think it's easiest to use a build system and let the IDE generate the project from your build system.
Eclipse, Netbeans, and Intellij are all pretty good at building projects from maven or ant build files. With this solution you have a simple build that is easy to setup in CI (Hudson, Bamboo, whatever) and you don't have any IDE specific files checked in. If my workspace is totally different than yours, with different versions, plugins, whatever, I'm not stuck with your project file and you're not stuck with mine. My IDE creates the project appropriate for my environment and your IDE does the same for yours.
Since you mentioned having to manually add libraries, I assume you are not using any build manager (like, maven or ant) besides ecplise.
For ecplise to handle the project properly you need the source files (*.java) in their respective directories, any resources bundled with the web service (e.g. services.xml), the ".project", ".classpath", ".settings", etc. files for eclipse. This should be enough for eclipse to generate anything else necessary to build the project.
Any files/directories that are generated by eclipse during the build process (e.g. target & bin directory, *.class, *.war) should not be checked in -- they will be generated when needed during the build.
I am thinking that, since you are adding the necessary 3rd-party jars manually, these libraries might reside in a different path between computers (e.g. if the path contains the username, it will not be transferable to another computer for a different user). To fix that you can set up the classpath using an eclipse classpath variable. In Preferences->Java->Build Path->Classpath Variables set up a varable linked to the "root" folder where the 3rd party jars a stored. Then add the libraries to the project using this new variable, not their full path. To make it work on someone else's computer, you would only need to set this classpath variable to have the build path point to the correct libraries.
It might be beneficial if you migrated your project from eclipse only to a build manager (e.g. maven) that takes care of many of these issues for you. Eclipse can build a project from the configuration of the build manager, making it easier to manage the project.
I have a web application. I'm configuring CI for it.
We didn't use a build tool (neither Ant nor Maven), and made builds by means of IDE.
Now I'm working on an Ant script which will be used by our build system.
There are several property files which should have different properties' values when the project is build on the build server but not on local machines.
What are the common approaches for handling such situation?
If we used Maven, I would probably go with using different profiles, but we have Ant.
One possible solution which I can see is creating a root folder for all property files sets, and storing each set in a separate subfolder (see the schema below).
/profiles
/dev1
prop1.properties
prop2.properties
/dev2
prop1.properties
prop2.properties
/build-server
prop1.properties
prop2.properties
/webapp
/WEB-INF
Then during the Ant build we can copy the correct set to the correct location. But it can be a problem to make builds by IDE as we used to do (because property files are not stored in their proper place under src folder any more).
Are there any other approaches?
If I understand you, you build separate ear/war/jars for each and every environment on your continuous build server. Is that correct?
I have two ways I handle this: The Smart Way, and the Dumb Way:
Smart Way
The smart way is to configure your application server (JBoss, Weblogic, etc.) to look for the desired property file external to the jar/ear/wars that are installed in the application server. This way, you build one set of jar/ear/wars, and it works everywhere. Plus, you do something very, very important: You cause the Finger O' Blame to point elsewhere.
If the properties files are packaged as part of the jar/ear/war, and something on the server changed, you'd get the blame because the your build was bad. Sure, you had no way of knowing the they changed the environment, but you did install that bad build file on the production server.
However, if the properties are stored outside of the build artifact, then it's the team responsible for configuring the servers that's at fault.
Actually, it's a lot easier for the team responsible for configuring the application server to handle the issue. They know what got changed, and they can update the properties file to reflect that change. Not only that, but you're only having to build and distribute a single set of artifacts. You don't have to worry if a new environment is setup, of if something in the old environment changes. In fact, changes can easily take place without forcing you to do a new release.
Dumb Way
We were able to do things the Smart Way at my last company. At my current company, we do things the Dumb Way. The properties are embedded in our build artifact, and there is no easy way to change it.
I divided each set of properties files by suffix instead of different directories (i.e build.properties.dev1, build.properties.dev2, etc). We placed properties files were put in a single directory.
When I do a build, I use the AntContrib <for> task to loop through the build process multiple times, each time with a different properties file. I then build an artifact for each and every set of properties files. I use the suffix on the properties file as folder names in my target directory where I store the built artifacts. That way, each build produces all the artifacts for all the environment.
That way, if the artifact worked in the dev environment, it will work in QA and Production. The only problem is that I'm storing 5 to 10 times the number of artifacts on my Jenkins server, so I need 5 to 10 times the disk space.
By the way, as long as I can define a <fileset> to find the properties files, I can use the <for> task, so you can use the different directories. And, you can use <basename> to pluck off the directory names.
I'm building an Android application (as an Eclipse project) that needs to access a web service. We will be sending the data from the service as JSON serialized classes, so we want to share some of the classes between the Android application and the server application. We are currently thinking that the way we need to do this is to structure our Git repository with 3 folders. One for the client, one for the server, and one for the shared library.
But at that point we couldn't figure out how to create the shared code. It looks like we can put all the .java files into a folder and then create a relative link to that folder from the other projects, but is that a good way to go about this?
The other possibility we found was to create another project in Eclipse and then include the library project in the client project. However we ran into a problem here. How can we make the library usable by both the server and Android? If I create a new Java project in Eclipse, I must select a JRE to build for, but Dalvik isn't an option, and even if it was how could I use the library with a desktop VM if the library was compiled for Dalvik?
I would use 3 projects:
server
shared
client (android)
and include shared as a project dependency in both server and client. If you use ant you'd drop the shared.jar in both server/lib and client/lib every time it's changed, and if you use maven it's a dependency (possibly with Ant + Ivy it's also a dependency). Consider Nexus as a repository location in that case.
You don't actually need to create a separate Eclipse project for the shared classes. You can just create a 'common source folder' outside of the other two projects' disk hierarchy. For both the server and client projects :in the Properties/Java Build Path/Source add a 'Link source' to the new folder. (Perhaps this is what you meant by 'relative link'). It's easy to add this common source folder to an Ant build file.
You could add the path to the classes/JARs to the server's runtime classpath.
Since it's unclear which one you're using, here's just a generic answer based on Apache Tomcat.
Open /conf/catalina.properties file.
Edit shared.loader entry to include the path to the package root of those classes or JAR file(s).
E.g.
shared.loader = /path/to/classes
or
shared.loader = /path/to/specific.jar
or
shared.loader = /path/to/*.jar
You can even specify multiple paths separated by a comma.