Semi-greenthumb here. I'm looking to download some Apache Commons and Google Guava libraries to use in Eclipse. Multiple Q&As (example, example) have said to download the library myself, and then either load it in Eclipse by path as a "User Library" that I can add manually to projects or go through an automated project management plugin like Maven. However, that leaves the question, where should I actually store the library on my system? (Mac OS)
Ideally, I want it in a directory that is common to all Mac/*NIX systems. However, this Q&A seems to suggest that doing so would be a bad idea, and this comment implies that I should keep a separate copy of the library within each project that uses it. This seems like it would be both a waste of space (for projects that use the same library version), as well as make linting Java files in a separate text editor a hassle due to libraries being stored within an Eclipse project's file structure rather than at the system level.
So where should I put 3rd party Java libraries?
I faced the same issue when I was maintaining my project dependencies in a manual way. It is difficult to have control over them, and sometimes updating a library can be a really painful experience if that update breaks a transitive dependency.
All this pain went away when I switched to Maven.
When you configure Maven, you can set the directory where these libraries will reside (common path is {user.dir}/.m2 } and every time a dependency is added to a project (via POM), then Maven will check if that library is already downloaded. If not, it will download it and store it for any future use (of the same version). It also resolves transitive dependencies for you, so you don't have to worry of breaking it when manually replacing a JAR.
This way you don't have to worry where the libraries are, your IDE will reference them automatically using the apropiate Maven plugin
I'm not saying you should use Maven, but if your problem is managing dependencies, then Maven (or any other dependency management system, eg: Gradle) may help you.
The comment you cited is pretty naïve in its approach. There are far too many build management tools to handle dependencies without having to deal with these minutiae.
If you decide on a tool such as Maven, your dependencies will be downloaded into a specified local repository (a directory on your filesystem), and all Mavenized applications can easily be configured to use those (shared) artifacts.
Most Java supported IDEs like Eclipse come with the option to initialize projects with Maven (or Gradle, as another example) and have sleek interfaces to easily edit their configuration files to specify which dependencies your projects will use.
I would strongly recommend either of those as opposed to manual JAR/artifact management, even for basic personal tinkering projects.
Related
I need to put some old java class library code that I have into a repo, from where others can check it out and build it. You know, like any public repo.
But, I'm not sure what the best way to do this is in the java world. In old-fashioned projects, we just used to supply the build scripts and a list of dependencies. You gathered or installed the dependencies separately before running the build scripts.
But these days for many languages, you have package managers and the like that pull from remote locations and your build scripts need to include dependency fetching.
Basically, I'm not familiar with how java libs and programs are packaged.
Should I include the (dependency) libs in the repo? And update them whenever a new version is out?
Does java now have a package manager that will pull in the latest versions of the dependencies?
Do I leave it upto the people checking out to download the libs themselves before they run the build scripts?
I'd prefer it if the solution didn't involve installing a huge package manager. Gradle wants to pull in like 150MB+ of stuff and as far as I am aware, it isn't ubiquitous on java deployments.
Thanks.
Use Maven. I believe these days it's the #1 "package manager" (not a term that's usually used to describe it, but quite apt) by a large margin. It's built into Netbeans, IntelliJ IDEA, and I believe Eclipse.
However, it won't just "pull the latest versions" of your dependencies, since your application may break. Only the versions you specify. Therefore, you should periodically update (and test) your code to reduce incompatibilities when someone tries to use your library in an application which directly or indirectly pulls newer versions of the same libs (and they get into a bit of "dll hell"), or reduce your use of third-party libraries in general.
You should also consider publishing your library in a compiled form to Maven Central so that using your library would be as easy as adding a dependency to the pom.xml. The problem that Maven solves, after all, is not so much making it easy to build your library (since just bundling the dependencies gets you most of the way), but making it easy to use your library.
I have a project where our CLASSPATH is getting out of hand. We have jars that depend on other jars, which in turn depend on even more jars, as well as a large collection of third-party jars. So each time we use one of these jars we need to construct a rather large CLASSPATH, and if downstream dependencies change it is no longer in sync.
One thought is to maybe write a tool a little like pkg-config to help with all of this. I did some searching and didn't find anything. Are there any tools like this that I've missed or do people have alternate solutions?
No, I don't want to use Maven. We already have a different build system in place.
If you aren't already using a build tool, I recommend starting with Maven. The main thing that turns people off Maven is that it has a standard project structure. Personally, I think that's a good thing.
If you want to be a little more free-form with your projects, there are the Maven Ant Tasks for Ant. I much prefer these to Ivy, but part of that is because I'm used to the Maven artifact naming conventions.
In either case (or if you use Ivy), look into a repository manager. Nexus is written by the people who maintain Maven, and I find it easier to use than Artifactory, which is another widely used system. One benefit for Artifactory is that they provide hosting (for a fee).
You may want to use maven with an IDE plugin like m2eclipse. All you have to do is type the class in your code and it will help you search for and bring in the right packages with their dependancies. It will also automatically configure the classpath for you. Here is a quick tutorial http://www.sonatype.com/people/2010/03/adding-dependencies-using-m2eclipse/
I have multiple java projects in a folder. Also there is a second folder with libraries, that might be used as build dependencies from the projects. The projects may also have dependencies to other Projects. What's the best approach to build all projects ?
In other words I want to build the projects without explicit telling their dependencies.I think the biggest problem is the dependecy between the projects.
There are multiple build systems that are available that you may use. Maven has a complete dependency system built into it. Almost all third party open source jars are directly accessible via the World Wide Maven repository system. Basically, you describe the jar you need (groupId, artifactId, and version) and Maven will automatically fetch it for you. Not only that, but Maven also will build your project without having to create a build file. Instead, you have to describe your project in a project object model (a pom.xml file) and Maven will download everything you need, including all compilers, etc.
Almost all new projects use Maven, but Maven has a few downsides:
Since you don't control a build process, it can sometimes feel like poking a prodding a black box to get the build to work the way you want.
Documentation can be scant -- especially if you're moving beyond basic Java compiles.
You usually have to arrange your project in a specific layout. For example, source files should go under src/main/java while JUnit tests are under src/test/java. You don't have to follow the recommended layout, but then you'd have to modify the pom.xml file this way and that to get your build to work. That defeats the whole purpose of the pom.xml in the first place.
If you already have another build system setup (like Ant), you lose everything. There's no easy way to move from Ant to Maven.
The other is called Ant with Ivy. Ivy uses Ant for building, but can access Maven's world wide repository system for third party dependencies. It's a great compromise if you already are heavily invested in Ant. I also find Ant with Ivy to be better documented than Maven (although that's not too difficult). There's an excellent chapter going over the basics of Ivy in Manning Publication's Ant in Action.
With either process, I would recommend that you build a company wide Maven repository using either Nexus or Artifactory. This way, any proprietary third party jars (like Oracle jars) can also be stored in your company wide Maven repository since they won't be in the standard World Wide Maven repository.
By the way, if this is a company wide effort, and you are moving multiple Ant projects into Ivy, I have an Ivy project I use in Github that makes things easier.
Oh, there's a third possibility called Gradle which I know nothing about. I also believe it can use the World Wide Maven repository. It's based on Groovy which is based on Java syntax, and that's about all I can say. Maybe others can fill you in on the details. The Gradle group contends it solves a lot of problems of both Ant/Ivy and Maven.
Whatever tool you use, if you have various projects interdependent, you need to be clear on the independent ones which will be built first before building the dependent projects. You need to have a clear dependency structure for your projects.
You can do this with Apache Ivy. You can lay out the locations for you common libraries, define published artifacts and inter-dependencies in an ivy.xml document in each project, and let a top-level Ant build with the Ivy tasks figure out what the build order should be based on those dependencies.
Quite new to maven here so let me explain first what I am trying to do:
We have certain JAR files which will not be added to the repo. This is because they are specific to Oracle ADF and are already placed on our application server. There is only 1 version to be used for all apps at anyone time. In order to compile though, we need to have these on the class path. There are a LOT of these JARS, so if we were to upgrade to a newer version of ADF, we would have to go into every application and redefine some pretty redundant dependencies. So again, my goal is to just add these JARs to the classpath, since we will control what version is actually used elsewhere.
So basically, I want to just add every JAR in a given network directory (of which devs do not have permission to modify) to maven's classpath for when it compiles. And without putting any of these JAR files in a repository. And of course, these JARs are not to be packaged into any EAR/WAR.
edit:
Amongst other reasons why I do not want to add these to the corporate repo is that:
These JARs are not used by anything else. There are a lot of them, uncommon and exclusive to Oracle.
There will only be one version of a given JAR used at anyone time. There will never be the case where Application A depends on 1.0 and Application B depends on 1.1. Both App A and B will depend on either 1.1 or 1.2 solely.
We are planning to maintain 100+ applications. That is a lot of pom.xml files, meaning anytime we upgrade Oracle ADF, if any dependency wasn't correctly specified (via human error) we will have to fix each mistake every time we edit those 100+ pom.xml files for an upgrade.
I see three options:
Put the dependencies in a repository (could be a file repository as described in this answer) and declare them with a scope provided.
Use the dirty system scope trick (i.e. declare the dependencies with a system scope and set the path to the jars in your file system.
Little variation of #2: create a jar with a MANIFEST.MF referencing all the jars (using a relative path) and declare a dependency on this almost empty jar with a system scope.
The clean way is option #1 but others would work too in your case. Option #3 seems be the closest to what you're looking for.
Update: To clarify option #3
Let's say you have a directory with a.jar and b.jar. Create a c.jar with a Class-Path entry in its META-INF/MANIFEST.MF listing other jars, something like this:
Class-Path: ./a.jar ./b.jar
Then declare a dependency in your POM on c (and only on c) with a system scope, other jars will become "visible" without having to explicitly list them in your POM (sure, you need to declare them in the manifest but this can be very easily scripted).
Although you explicitly stated you don't want them in the repository, your reasons are not justified. Here's my suggestion:
install these jars in your repostory
add them as maven dependencies, with <scope>provided</scope>. This means that they are provided by your runtime (the application server) and will not be included in your artifacts (war/ear)
Check this similar question
It is advisable that an organization that's using maven extensively has its own repository. You can see Nexus. Then you can install these jars in your repository and all developers will use them, rather than having the jars in each local repository only.
(The "ugliest" option would be not to use maven at all, put put the jars on a relative location and add them to the classpath of the project, submitting the classpath properties file (depending on the IDE))
if you are developing ADF (10g / 11g I guess) components, I suppose you'll be using JDeveloper as IDE. JDeveloper comes with a very rich Library Management Tool that allows you to define which libaries are required for compiling or which ones should be packaged for deployment. I I suppose you will already know how to add libraries to projects and indicate in the deployment profile which ones should be picked while packaging. If you want to keep your libraries out of maven, maybe this could be the best approach. Let´s say the libraries you refer too are the "Webcenter" ones, using this approach will guarantee you you have the adequate libraries as JDeveloper will come with the right version libraries.
Nevertheless, as you are using maven I would not recommend to keep some libraries out of control and maven repositories. I´d recommend choose between maven and Oracle JDeveloper library management. In our current project we are working with JDeveloper ADF 11g (and WebCenter) and we use maven, it simply make us library management easier. At the end of the day, we will have a big amount of third party libraries (say Apache, Spring, etc.) that are useful to be managed by maven and not so many Oracle libraries really required for compiling in the IDE (as you would only need the API ones and not their implementations). Our approach has been to add the Oracle libraries to our maven repository whenever they are required and let maven to control the whole dependency management.
As others say in their answers if you don´t want the dependencies to be included in any of your artifacts use <scope>provided</scope>. Once you configure your development environment you will be grateful maven does the work and you can (almost) forget about dependency management. To build the JDeveloper IDE files we are using the maven jdev plugin, so mvn jdev:jdev would build generate our project files and set up dependencies on libraries and among them to compile properly.
Updated:
Of course, you need to refer to ADF libraries in your pom files. In our project we just refer to the ones used on each application, say ADF Tag Libraries or a specific service, not the whole ADF/WebCenter stack. For this purpose use the "provided" scope. You can still let JDeveloper to manage your libraries, but we have found that it's simpler to either have a 100% JDeveloper libraries approach or a 100% maven approach. If you go with the maven approach it will take you some time to build your local repo at first, but once that's done it's very easy to maintain, and the whole cycle (development, build, test, packaging and deployment) will be simpler, having a more consistent configuration. It's true that in a future you'll have to update to later ADF versions, but as your repository structure will already be defined it should be something fast. For future upgrades I'd recommend to keep the ADF version as a property on the top pom, that will allow you to switch faster to a new version.
Perhaps the reason I stalled learning Java until now is because I HATE how Java handles external libraries. I'm stuck keeping them in one place, adding them individually, fixing problems with versioning and every time I move/rename them, and copying and writing the classpath over and over each time I release a Java application.
There has to be an elegant solution to all of this. I keep all of my libraries (regardless of task, platform, or other) in their own little folder inside a "lib" folder in my development folder, kind of like this:
Dev
-lib
+JS-jQuery
+Flex-Degrafa
-Java-Xerces
+Xerces-1.2.3
+More libraries
I can use either Netbeans or Eclipse for Java dev, but none of them provide a very streamlined (and not to mention idiot-proof) way of managing all of these.
A nudge in the right direction or an online article/tutorial on this would be greatly appreciated.
You can either use Ant + Ivy or Maven to manage your library dependencies.
If it is only dependency management you're after and you're happy with the rest of your build process, I would use Ivy, as it can unobtrusively manage your dependencies, leaving your existing build process intact. There is a plugin for Eclipse called IvyIDE that contributes your dependencies via a classpath container.
Maven 2 has a steeper learning curve but provides a much richer set of functionality for building your projects and Eclipse integration through m2eclipse or IAM.
Personally I use Maven as I have a large number of projects to work with and Maven is particularly suited to efficient development across lots of projects.
Have a look at the introductory documentation to see what works for you.
Ivy Tutorial
Maven Getting Started Guide
Netbeans 6.7.1's Maven support is quite good and comes out of the box with the IDE.
The Eclipse addon was frustrating enough that I gave Netbeans another try.
A third choice besides ChssPly76's options is to use Ant with the Maven Ant Tasks. I don't know if I'd call any of these solutions particularly "elegant," but they do spare you the need to manage your own lib/ directory and classpath variables.
If you're working on Linux you can install Java libraries with APT or RPM.
Otherwise, I normally check precompiled JARs into a lib directory in my project's version control repository and make sure the names of the JAR files include full version information. E.g. lib/foo-1.5.6.jar, not lib/foo.jar.
To avoid having to manually set the classpath before running your app, you can set the classpath in the Manifests of the JARs themselves to define the dependencies of each JAR file. The JVM will follow all the dependencies when loading classes.
Maven is often more trouble than it's worth, but the ability to open a maven project directly into IDEs such as IntelliJ is excellent. For example, IntelliJ will download all dependencies and have them available without having to run a build first, or an mvn command and then a project refresh. It also isn't necessary to re-generate the project every time a dependency is added. I work with a number of Eclipse developers who switched to IntelliJ for this alone.
However, one shortfall of Maven is that many libraries (or versions of libraries) are not available on public repositories. Therefore it is often necessary to set up a local repository such as archiva. In ant, it would just be a matter of adding it to the lib directory in the repository.
Maven can also attack when you need to do something that maven doesn't directly support via a plugin. What would normally be a few lines of ant can often turn into a morning's worth of work.
Finally, buildr is an excellent way of using Maven's dependency management and plugins, while also supporting ad-hoc tasks.