One of contractors at my current project has a weird habit to deploy fixes to the production as jars and not to push fixed code to the project's git repo. There are tens of maven projects which packs to a jars
Is there any practical way to detect and plot all discrepancies between compiled jars from production and code from a repo? Currently I can decompile a jar with IDEA and compare result with code. Decompiled code is not identical to original by design so it takes ages and possibly leads to errors.
To detect changed classes, I would compile the code in the codebase and make a class by class binary comparison between the class files you just created and the ones in production. This can be easily automated.
But the real solution is of course: Your company should have a rule that only builds from the build server (Jenkins, Bamboo, whatever) that draw from the official git repo can go to production. No exceptions.
The technical solution to your problem is called Reproducible_builds.
But i donot think that you need a technical solution for the "blaming-game" but an organisational solution like overwrite the production build with your own jar compilation and refuse to pay the contractor if the problem re-appears
Related
One of my colleague recently created a new project in eclipse ,committed and pushed with built-in git client.
After I cloned to my computer and opened with eclipse, I found eclipse creating .classpath file.
Isn't .classpath a crucial file for eclipse project(also .project) to find referenced jars?
I am very confused after googling, seeing all the discussions talking about ignoring them.
Aren't they crucial to Eclipse to work correct ?
Why are people ignoring them ?
What's the problem if I have them not ignored ?
These Eclipse-specific files make it easier to set up Eclipse and Visual Studio Code for a Java project.
In general, IDE and tool specific files (Eclipse, Jenkinsfile, GitHub workflows settings, etc.) should be shared as long as they are used and maintained. Otherwise, delete them.
Of course, if you use a different IDE than Eclipse and Visual Studio Code and do not use the Eclipse compiler in IntelliJ IDEA, these Eclipse-specific files are useless, but they do no harm. As long as you do not use functions like file or folder links (stored in the .project file), sharing these files does not lead to IDE lock-in.
In Maven and Gradle projects the .classpath file can be derived from the pom.xml or build.gradle file, but settings that cannot be derived like compiler settings (warnings and errors), formatter or save actions settings (which are stored in the project's .settings folder) should be shared so that everyone uses the same.
This also applies to the .project file, as it contains which natures the project has and which tooling is required. If something is missing, a dialog will ask if the missing plug-ins should be installed.
Eclipse puts these files into the project folder and not into the .metadata folder, because they are intended to be shared. But why there are people who do not share these files? Probably because of historical reasons. 15 or 20 years ago, there wasn't Git, Maven and Jenkins. In these days, a Java application was usually built on a developer's computer by manually exporting JARs or at best via some batch/shell scripts. This meant, making the same build on a different computer or even just with a different IDE or IDE version might led to a different result, causing a lot of trouble. IDE agnostic builds was the solution for this problem. Maybe that's why people today still think that everything have to be IDE agnostic and recommend to use Maven or Gradle. But these files are not shared to build a product to be shipped. Hopefully.
They're Eclipse specific, so they don't really belong to the project's source code. Developers might be using different IDEs, so Eclipse's .classpath would be useless for someone using IntelliJ IDEA for example.
Since the project most likely uses Maven / Gradle / some other build system, the IDE is capable of generating the classpath based on the pom.xml or build.gradle files, as you noticed. Only if there isn't a build system, and the project is IDE specific, it would be necessary to include those files to make sure the project keeps its meta-data. But that's an unlikely scenario in modern times and real life work situations.
It doesn't usually cause problems to include those (unless there are conflicting project specific configurations from different developers), but they're not necessary either. I don't include them since there's no advantage, and it keeps the root of the project cleaner.
That very much depends.
If your team decides that the eclipse project configuration files are the essential source of truth, and everybody else should be using them: then sure, these files should sit in your source code management repository.
But doing so leads to "IDE lock-in". Or worse, it leads to having multiple files containing the same information, as a future IntelliJ user might prefer to add .iml files on top of that. So any change to project definitions needs to happen twice now.
So, ideally, in 2020: use other tools as your base source of truth (like gradle definitions), and then tell your individual IDE to use that.
I was working on a project with a seriously large amount of classes that I want to compile to a jar. I know about entry-points and the manifest.txt and all the needed items inside my jar, my classes are all compiled and have the .class file and everything, but the problem is I will have to add all the class files to the final jar in compilation through a single line in Command Prompt. I was wondering and stumbled upon literally nothing in the internet if it could be done in an easier way because I will be updating my work constantly and have to recompile and re-jarify my work. I have heard of third party programs that will do the trick, but somebody on some website said that they could potentially be causing problems and stuff, so I dropped the idea quite quickly. Now that I am in a seriously tight spot though, I wish to hear opinions and suggestions on this. So to sum up:
I want a way to compile a big bunch of .class files in a single jar without typing all of them over and over again between compilations allowing me to save time and frustration.
I would prefer native stuff if this is even possible to do - e.g. the jar compiler of the JDK instead of anything third-party. If there is a way to do this using manifest or any other file in compile-time arguments, let me hear it.
Anyone who cares to suggest, discuss or give me a good reason why to or not to use third party applications for this will be most welcome.
Keep in mind that I work on Windows but my aplication will be cross-platfrom, so don't suggest as a main option some compile solution that will make a final file with a .exe extension (although if anyone knows how to do this, I would like to hear it in a comment as I wonder about this as well).
Thanks in advance and if you feel the need to ask me anything to help you reply, shoot away!
Have a look at this ant tutorial which shows how to write a simple build.xml which can compile and jar.
http://ant.apache.org/manual/tutorial-HelloWorldWithAnt.html
You can then adapt it for your own needs.
Note: ant is only suited for smaller projects like yours.
The solution to this, and related, issues, is to stop typing at the command line and use a build tool. The common tools here for Java builds are:
Apache Ant http://ant.apache.org
Apache Maven http://maven.apache.org
There are other less common ones. Both of these tools will provide you with what you need.
Just want to add some information about Ant and Maven.
In your case, you need to automate the build of your application. The basic solution would be some kind of script but it's not used at all. Nicer solution exist :
If you come from the idea of a script to automate your build, you can use a tool like Ant, it's a bit like make and such tool in the C world where you define the needed tasks for your build in a configuration file. The problem with such solution is that it allow you to define your own structure for your build and a new comer to your project may have some difficulties to understand the logic of the build.
The other approach is to describe what kind of build you want to do, organize your sources and resources as it is done in most cases (by following a convention in fact). For example, java sources are in src/main/java, tests are in src/test/java, config files are in src/main/resources, and so on. In the description of your build you will just say : this is a java project and I want to build a War web application and execute my tests using jUnit 4. The dependencies of my project are apache xerces and hibernate 4. Then, the tool will know what to do without the need to say how to do. This is the way maven do.
In short, in the Ant approch, you will say how to do what you want and in the Maven approach you will define what you want to do and the tool will know how by default.
You may also be interested in some kind of hybrid approache like the one provided by tools like Gradle.
For more information :
http://ant.apache.org/
http://maven.apache.org/
http://www.gradle.org/
Hope it helps
I love being organized, and I hate having all my java projects in eclipse right in the workspace. It makes it very cluttered, and I wanted a more organized approach, where I have folders in my workspace for different kinds of projects. However, folders can only be added into other folders, and not the workspace itself. Also, I wanted a way to be able to put classes into just any folder, and not just the src folder in a project. I love eclipse, but I really want the way to organize my programs to be a little more flexible. Does anyone know how I can do this?
I think what is organized to 1 person is not the same for another. I have found some of these techniques helpful maybe they will help you
Group projects to working sets so you do not see the full workspace
You can keep sources in different folders add those source folders in the build path
Use tools like Collabnet that integrates with your version control repository and issue management systems like JIRA and at given point in time you can view only the files that are relevant to the issue you are working with.
Know your keyboard shortcuts very well
For what I understand, you are formulating two expectations:
Have a clear directory structure for your projects, independent from the IDE;
Have a clear workspace organization, which reflects the underlying structure of your projects.
There is something you did not mention, which is the build facility for your project. I guess you are compiling and running your projects from within Eclipse? If not, or if you are willing to compile, test, deploy and run your project besides your Eclipse IDE, you can have a look to full-featured tool like Maven.
The point is that even if you do not want to build your project outside of the IDE, this tool will save you so much time that it is even worth to use it only to have a clear, proper, conventional and understandable directory structure as you seem to expect. On the other hand, the problem is that such a tool addresses issues that are far beyond your own requirements, which may render it overkill to use for the sole purpose exposed above.
Maven is a perfect tool for those who like to be organized. One of its core concept is convention over configuration.
This means, for instance, that if you follow Maven conventions for your projects, the simple command
mvn eclipse:eclipse
will generate for each of your them a corresponding Eclipse project, which you will be able to configure, generate and clean from outside Eclipse.
I'm trying to learn project automation and I'm using the book Pragmatic Project Automation as a guide. It's examples are in Java so I find it easier to follow it along in Java. I don't have any experience using Java or any of its IDEs. However, I've learned some C# using Visual Studio (although I'm still a beginner).
I'm having trouble understanding some parts of setting up a Java project. I'm using Netbeans IDE 7.0 on Windows 7. With Visual Studio, I have a solution with projects underneath. With Netbeans, I appear to just have a project with directories determining the rest of the structure (and the IDE to some degree)? The equivalent to Add Reference appears to be adding a source to the Classpath. There also seems to be a degree of separation between the compiler and the IDE. I'm currently in a situation where I can compile my project just fine while the IDE tells me I still have errors (and I assume this is because I have the project set up incorrectly).
I'm basically looking for analogies that will help me better understand Java project structure.
Lots of similarities between the two languages and IDEs. I spent many years in both. For starters the equivalent to 'add reference' in VS is adding a library or jar in netbeans. In respect to a reference - a jar is pretty much the same thing as a module or .dll in VS. A jar is a compiled reference. To add a reference just go to the project menu and then properties then to the libraries menu from there you can add either pre-assembled netbeans libraries, which are collections of .jar's, or a single .jar, or even a project. Once you add a reference you can import them into your class just like you would in C#.
Netbeans doesn't really have a 'solution' as VS does. You deal with individual projects. It does however have the capability to add a project as a reference so you don't have to continually re-build the references when you change something between multiple projects. It also has project groups to group similar projects.
Lastly Apache ANT is responsible for tying everything together in the background. Netbeans creates a build.xml and build-impl.xml file in the background to tell ANT how to assemble the project into a .jar.
There are my other things to cover but I thing this answers most of your questions. Does this help?
I can't speak for NetBeans, as I use Eclipse, but you are on the right track with classpath being roughly equivalent to references in the Visual Studio world. Libraries (usually .jar files) are placed on the classpath and must be there both at compile time and runtime (you specify the classpath to the compiler at compile time, and to the JVM at runtime). The classpath can contain many different entries, and they can be anywhere in the project structure (or outside of it entirely).
Java itself doesn't impose many restrictions on your project structure, although various IDEs and build tools do. The one thing that is a universal restriction in all Java environments is that source files (and class files) are placed in a directory named after the package name. So if your package name is com.test.something, then your source files will be in SRC_DIR/com/test/something, and your class files in OUT_DIR/com/test/something (note: SRC_DIR and OUT_DIR are not special variables; each IDE will have a different way to specify those directories).
Java libraires tend to heavily build-on one-another, so at some point, you'll find that the classpath entries are too many to manage manually. Before you get there, you'll want to take a look at Apache Maven or Apache Ivy which are dependency management tools. You'll need to understand how they work (either one, not both) and how to integrate them with your IDE. If you use Eclipse and Maven, m2eclipse offers fairly complete integration between the IDE and the dependency management tool.
With Netbeans, I appear to just have a
project with directories determining
the rest of the structure (and the IDE
to some degree)?
Visual Studio dictates a particular project layout and since the compiler is so tightly integrated into the IDE there's no real concept of a build script. In contrast, Java has no such structure (although certain 'best practices' have emerged such as having a 'src' directory for source files, 'lib' for libraries, 'test' for test source, etc.) and a build script is usually required to tell the compiler were to find source files and libraries, what artefacts to produce and a miscellany of other chores (running tests, deployment, creating code metrics and so forth).
In simple cases, the IDE will take care of this for you (if you follow the convention for that particular IDE) but ultimately you will probably want to take a look at a build tool to understand what's going on behind the scenes. Apache Ant and Apache Maven are both prominent offerings. Ant is very flexible whereas Maven attempts to dictate a common layout. I suggest you investigate both and see which suits.
There also seems to be a degree of
separation between the compiler and
the IDE. I'm currently in a situation
where I can compile my project just
fine while the IDE tells me I still
have errors
If your code compiles, it is correct. The IDE is simply acting in an advisory capacity (and will highlight issues beyond compiler errors, such as warning you of potential code mistakes or bad practice).
and I assume this is because I have
the project set up incorrectly
This is a possibility although, as stated above, there are many other explanations.
I am currently refactoring a large Java application. I have split up one of the central (Eclipse) projects into about 30 individual "components", however they are still heavily inter-dependent. In order to get a better idea of what depends on what I am looking for some way to graph the compile time dependencies.
All tools I have found so far are capable of graphing package or class dependencies or the dependencies between Eclipse plugins, however what I have in mind should just take a look at the classpath settings for each Eclipse project and build a coarser grained graph from that.
Later I will then go deeper, however right now this would just mean I would not be able to see the forest for all of the trees.
Check out JBoss Tattletale. It might not do all you ask but it's worth checking out. It's still relatively new though.
The tool will provide you with reports that can help you
Identify dependencies between JAR files
Find missing classes from the classpath
Spot if a class is located in multiple JAR files
Spot if the same JAR file is located in multiple locations
With a list of what each JAR file requires and provides
Verify the SerialVersionUID of a class
Find similar JAR files that have different version numbers
Find JAR files without a version number
Locate a class in a JAR file
Get the OSGi status of your project
Remove black listed API usage
Structure101 is capable of visualizing class and method JAR level dependencies in Jboss 5.
See the screenshot below or view it larger.
One tool that I believe would do what you want is Understand. It's not free, but you can download a free trial edition before investing any money into it.
Take a look at Dependency Finder
I am not sure if there is a(n Eclipse) classpath analysis tool.
May be Understand mentioned by MattK can help.
The closest I would pick amongst all the static code analysis tool referenced here would be JarAnalyzer (no graph though), able to detect "Physical dependencies" amongst jars.
Sounds like a use case for Degraph. It analyzes a bunch of class files and jar's, and visualizes the dependencies.
What makes it suitable for your usecase (I think) is the possibility to define arbitrary groups of classes to be bundled together. So you can reproduce your jar structure, seeing dependencies, especially cyclic dependencies.
You can unfold the groups to see their contained classes or collapse them to simplify the view.
For a quick impression what is possible, take a look at the Degraph Examples.
Example for Log4j:
JDeps is already included in the JDK, and shows JAR dependencies. For example:
jdeps -R -cp "my\jar\dir\*;my\other\jar\dir\*" my\classes\dir
Check out Class Dependency Analyzer (CDA): http://www.dependency-analyzer.org/
I have found it very useful for tidying up jars.
for the record (and for improving this knowledge base), I found Shrimp very helpful:
http://www.thechiselgroup.org/shrimp
Also, for easy dependency-checking, Byecycle is worth a try, but seems not to be updated anymore:
Byecycle
Both tools also offer Eclipse integration.