I would like to be able to determine what versions I am running of a dependency at runtime as well as the version of the web application itself.
Each web application I deploy is packaged with a pom.xml which I can read from, that part is trivial. The next part is parsing the pom without much effort.
As the web application is running, I want to be able to understand what version I am, and what versions my dependencies are.
Ideally, I would like to do something like:
MavenPom pom = new MavenPom(webApplicationPomInputStream);
pom.getVersion();
pom.getArtifactId();
pom.getGroupId();
for(Dependency dependency:pom.getDependencies())
{
dependency.getVersion();
dependency.getArtifactId();
dependency.getGroupId();
}
Should I just use XPath notation here, or is there a library I can call to do this type of thing?
After these posts, I am thinking the quickest/most reliable way is to generate a text file with the dependency tree in it: mvn dependency:tree. Then I will parse the text file, separate the groupId, artifactId, and version, and then determine the structure by the indentation level.
If I do that, can I export to XML instead of text? I can then use JAXB and easily parse that file without doing any/much work.
It is a hack, but looks promising.
Walter
I will just use the mvn dependency:tree plugin to generate a text file with the dependency tree. Then I will parse that in and create the dependency tree/graph from that. I will get the scope of the artifact, groupId, artifactId, version, and its parent.
I successfully implemented this type of lookup, it simply takes the dependency output, parses it and organizes dependencies simply using the indentation, nothing fancy. The artifact, group, version, and scope are easily parsed since the separator is a :.
Walter
Maven has of course such an API. Have a look at org.apache.maven.project.MavenProject. But, to be honest, I don't think it will be that easy to create a MavenProject instance. The source code will be helpful here, check for example MavenProjectTest or maybe the Maven Plugin API (actually, this task would be much, really much, simpler to achieve from a Mojo) for some guidance.
I'd suggest to search for or ask this question on the Maven Mailing Lists, org.apache.maven.dev would be appropriate here IMHO.
Related
To start off, I am still a student and trying to learn things, but I am quite stuck on how to get all the dependencies of each package from Maven. Initially, I got all the packages from https://libraries.io/, but without their dependencies, as I would like to construct a temporal graph in which I could display those dependencies using time. Libraries.io does not take time into account and therefore I only downloaded the packages.
Now, I'm finding it quite hard to download the dependencies of each package. Initially, I thought parsing the .pom files from this central repo https://repo1.maven.org/maven2/ is enough to get the dependencies for each package, but later on, I found out that there are also external dependencies that you can only get by resolving the .jar file (I think?). I do not really understand that, and I am trying to learn how could I get all the data in this case, as parsing does not seem to be entirely accurate.
Also, I should mention that I am trying to do this in golang, as it was a new language I wanted to try out and it seemed fitting for the task.
EDIT 1- For example, I would like to have this for each package from Maven:
{
"name": "react-dom",
"versions": {
"1.00": {
"timestamp": "06-05-2022T10:00:01",
"dependencies": {
"name": "^1.0.2"
"name": "^2.1.2"
}
}
}
}
EDIT 2: Yes, this is for research purposes. This is the description of the task https://imgur.com/a/D0LcbzF. Initially, I thought this research was not this complex, but after the first meeting, I was told that we basically have to do what https://libraries.io/ does, but make it accurate by adding a time component. From what I understood by the professor, what libraries.io does not take into account is an example like this:
Library A releases a version at time 1, named version A
v1.1
Library B, which depends on library A’s latest version at
the current time, is releasing a version at time 2. Therefore, library B depends on library A v1.1
Library A releases a version at time 3, named version A v1.2
Library C, which depends on library B, releases a version at time 4. Therefore, it also depends on library A.
Library C should depend on the latest version of library
A, which is A v1.2, even though library B depends on
version A v1.1.
So, to summarize, I have been tasked with getting the packages from https://libraries.io/api, and get the dependencies by resolving the pom files somehow (I cannot tell you more because I am very confused, I am not knowledgeable enough on this matter, I have just started)
Professor sent me this "WRT maven dependency listing: I was thinking something along the lines of mvn -DgroupId=junit -DartifactId=junit -Dversion=4.13.1 dependency:get but his only works for retrieving the pom/jar for downloading particular dependency into .m2", maybe that helps you understand something.
Afterward, after data is available, make a temporal graph that could suit the example above, and finally, see what measures we can find to see what are the most used software?
To me, and from what you have already said, this seems very out of reach and I am literally lost on what to do, as the professor does not really guide me in any way.
Some papers I found: https://www.researchgate.net/publication/335499638_The_Maven_Dependency_Graph_A_Temporal_Graph-Based_Representation_of_Maven_Central
The part of your comment: on the data to see what are the most used packages is data you don't have.
for that you need the download statistics of central repository where you don't have access to ...
If you would even have the download statistics of central repository it does not represent how much an artifact is being used because many companies are using repository managers which means an artifact is being downloaded exactly once but internally used a lot.
Furthermore if an artifacts is being downloaded does not really mean it's used. Some artifacts are downloaded based on transitive dependencies or just added in pom file but are not really used.
Technically you can download all artifacts or at least the pom files and analyse the dependencies but you lack the download statistics of central repository.
Also this is prevented because you can't download all the artifacts because you would being blocked from central repository.
The size of central is, an educated guess of mine ca. 5 TiB+
Another thing is that not only central repository does exist there are a lot of other Maven repositories available and being used.
Lets say there exists a library x.y.z-1.0, x.y.z-2.0, x.y.z-3.0 that uses a.b.c-1.0, a.b.c-2.0, a.b.c-3.0 respectively.
I am currently writing another library (com.mysdk) that uses x.y.z-2.0. But I have not check the compatability of my library with x.y.z-1.0 and x.y.z-3.0. Thus I am going to shade x.y.z-2.0 to com.mylibrary.shaded.x.y.z-2.0. However, after investigation, a.b.c-2.0 will not be shaded.
So the situation looks like the picture underneath.
My question comes in when the user of com.mysdk is also using a.b.c-1.0 at the same time. I could not make the project to use the specified a.b.c-1.0, instead it is using the a.b.c-2.0 in the sdk.
What is the best way to find the right dependency for a used class that are part of the maven-online-repository?
As far I see it is this approach:
lookup the import (e.g. org.whatever.X;) from your code at the maven-repository online (search.maven.org).
Pick one of the result list and include it in the dependency section of the POM.
Hope the chosen version and artifact of the dependency matches your requirements (compiling, runtime). If not try another artifact or version.
I'd like to share my way of doing it. What do you mean by "finding the ... for a used class that are part of the ..."? Do you mean that the dependancy is already used in somewhere else, or that you only know the package name that you may need?
I would first check which version I need for the current project.
If I'm working on a team project and someone has used the dependency in somewhere else, I would check their pom (to ensure we are using the same dependency).
Then I would look up the dependency in Maven repo and include it in my pom.
Hope this helps.
Essentially, yes this is what you have to do to obtain libraries/modules for your project.
Something that's helped me out though with this specific problem: versioning. You can set the versions you need for each of your dependencies with <properties> -> <gson.version>2.8.1</gson.version> (for example). That way, you can guarantee that your build matches with the reqs of the class or type of code you're trying to implement.
Maven doc ref: https://maven.apache.org/pom.html#Properties
XML being one of the reasons people move to Gradle, our project defines several pom packaged items. I assume that these are BOM's since they just used to pull in dependencies. So... how do you define a BOM in Gradle from a "best practices" / no XML perspective? I know that you can create a configuration that has dependencies and then just include it but I don't, for example, want "gradle build" to create a jar file for this sub-project as it would be pointless and slow down my build for no good reason. Am I on the right track or is there a better way and if so, what is it? Do I just turn off the jar creation somehow?
I think this is what I need. It's from right out of the Gradle User Guild in Chapter 52. Specifically it's Example 52.16. "Collections and arrays of dependencies".
List groovy = ["org.codehaus.groovy:groovy-all:2.4.4#jar",
"commons-cli:commons-cli:1.0#jar",
"org.apache.ant:ant:1.9.4#jar"]
List hibernate = ['org.hibernate:hibernate:3.0.5#jar',
'somegroup:someorg:1.0#jar']
dependencies {
runtime groovy, hibernate
}
I've just fought for a whole day with a strange maven problem:
I had a custom property called "deployment.name" that was never to resolved to what I configured for it, but rather the maven filtering mechanism always replaced it by the project's name.
I tried the goal "help:expressions" to find out whether this is a preconfigured property, but that goal only throws exceptions in m2eclipse. Google does not seem to know a pre-configured
property by that name.
The strangest bit: deployment.somethingelse works perfectly fine, so I ended up replacing ".name" with ".depname", then it works ;.)
The Maven Super POM defines the common configuration for all Maven projects. The values in that are accessible as properties (and , so that is where most of the properties you generally use come from (e.g. ${project.build.directory}), these are the pretty much the same as the output of help:expressions.
There is no deployment section in the super POM. The only thing I can think of is that the property is being set somewhere else, e.g. in a profile, or overridden by a plugin (though that seems unlikely). You could try running mvn help:effective-pom to see if the property is being set by a profile.
Are you able to post your POM? that might help diagnose it.
... I just ran help:effective-pom, and there is no trace of "deployment.name" in the output.
I can see all the other properties that I defined though (e.g. "deployment.depname").
Maybe "name" is a reserved attribute of some sort? Maybe debugging into m2eclipse will shed light on this riddle.