I forked a repository from Github that has a lot of packages and files, implementing all kind of algorithms, simulations and utility classes.
However, in my research I don't need all of these files/packages for my own simulation to work.
I would like to keep my forked project as minimal as possible, so I would like to keep only the necessary packages/files that are needed to compile my simulation.
I'm talking specifically about the IDE Eclipse. If I decide to "backtrack" all imports starting from my simulation file, I would definitely get lost because the original project is big.
On the other side, if I decide to "delete" a package and see if my simulation compiles, I would stay all week trying this out, and if I delete a needed file I would have to attach it again to my project which is troublesome.
Is there an automatic tool I can use to do this on Eclipse?
A simple option is to use a coverage checker to see what methods in what classes are used during execution, and delete the rest. And then revert anything that causes a compilation error.
This only works for code, not resources, though - and only if something like reflection isn’t used.
Related
I'm a beginning programmer and I apologize if my questions is trivial but I haven't found a sufficient answer to my problem.
I have a git repository called "toolbox" with some utility classes that I frequently use in other projects. Until now, I have manually copied those class files from this project in other projects whenever I needed them.
This if of course not a good way of doing it. I frequently add new features and fixes in whatever project I'm currently working on to these files. It makes version management a nightmare.
What I'd like to to is to import the toolbox-classes directly into the IntelliJ-Project(s) (which is also on the same git as the toolbox repo) and whenever I make a change to those files in the toolbox-repo I want the other projects to be able to automatically pull those changes as well.
If possible I'd also like to be able to share my toolbox-repo easily with others who might need those classes. But that is not a requirement. I'd just be nice to be able to do so.
I tried to use git submodule. It included the entire toolbox-repo in the target-repo but unfortunately I wasn't able to use the utility classes. I asked someone more experienced and they told me that I need to define "SourceSet" in the gradle.build but I wasn't able to configure that due to my lack of knowledge.
How can I include/import/use my utility classes from my toolbox-repo within other projects?
Thanks for any advice.
Background:
I'm working with (for me) a reasonably large codebase (eg: I've only got a few of the related projects checked out at the moment, and its > 11000 classes).
Build is ant, Tests are JUnit, CI is Jenkins.
Running all tests before checkin is not an option, it takes Jenkins hours. Even for some of the individual apps it can be 45 minutes.
There are some tests that don't reference individual methods based on reflection, and in some cases don't even directly reference the class of the tested methods, as they interrogate an aggregator class, and are aware of the patterns of pass-through methods in use here. As it's a big codebase, > 10 developers, and I'm not in charge, this is something I can not change for now.
What I want, is the ability to before check-in print out a list of all test classes that are two degrees away (Kevin-Bacon-wise) from any class in the git diff list. This way I can run them all and cut down on angry emails from Jenkins when something I missed eventually gets run and has an error.
The easiest way I can think of to achieve this is to code it myself with a Ruby script or similar, which allows me to account for some of the patterns we're using, but to do it I need to be able to query "which classes reference class X?"
I could parse .java or (easier) .class files to get this info, but I'd rather not :) Is there a way I can make Javac export it in a simple format as it compiles?
Is there a way I can make Javac export it in a simple format as it compiles?
AFAIK, no.
However, there are other ways to get a list of the dependencies:
How do I get a list of Java class dependencies for a main class?.
(Note however that you are unlikely to get a static tool to extract dependencies resulting from Class.forName(), etcetera. Also note that you cannot infer the complete set of dependencies from bytecode files because of the way that "compile time constants" are handled.)
It strikes me that there are a few problems here:
It sounds to me like your build, and indeed your project structure is monolithic. If you could restructure the code base into large-scale modules that build separately (according to their dependencies), and version controlled separately, then you only need to do a full build and run all unit tests when there is a change high up ... in a module that everything else depends on. (Can I suggest the "Maven" word. It really helps for a large codebase, and 11,000 classes is large.)
It sounds like you may be suffering from the "branches are hard" problem of classic VCS systems.
It sounds like you may need a beefier CI system. If you've got more cores and the build framework is right, you should be able to get faster CI builds. (And if you modularize so that you rebuild less ...)
I think it might be easier to address your slow build/test cycle that way rather than via extra (possibly bespoke) tooling to do dependency analysis.
But I recognize that it may not be up to you to make those decisions.
In my case there are two reason for doing that:
Sometimes people by mistake import classes which present in macbooks JDKs but absent in Linux. That causes build to fail on ci servers which are Linux based boxes. I doesn’t happen frequently, but when it does happened I'm thinking that there should be some smarter way to find out that earlier.
Unused imports trigger warning in IDE/code analysis. From time to time somebody need to spend time on cleaning up this stuff. Even if its just single right click in IDE you still need to commit your changes and make sure everything alright on build.
I'm curious if there is any way to find unused imports programmatically (lets say from unit test) and fail locally if there are any.
Maybe failing a build because of unused import sounds harsh, but if it saves time for team overall it makes sens to do so (would love to hear opinion on that as well).
UPDATE:
I followed yegor256 suggestion and incorporated Checkstyle task with initially small subset of Sun Code Conventions (unused imports is one of them) and made it break a build if violations found.
After one week of trial we've got zero unused imports in our codebase and surprisingly zero complaints about this rule (by the way, Checkstyle is really fast: analyzing ~100KLoc taking less than one second).
As for using IDE for such analysis: yes, it good choice, but having this kind of checks run as part of automated build is better.
What you're trying to do is called static code analysis. Checkstyle can help you. If you're using Maven, this plugin will do the automation for you: http://maven.apache.org/plugins/maven-checkstyle-plugin/
You can also take a look at qulice.com (I'm one of its developers), which integrates a few static analysis tools and pre-configures them (incl. Checkstyle, PMD, FindBugs).
If you are using eclipse IDE or IntelliJ IDEA, you can configure them to
1a. organize imports / remove unused imports on save or before commit (see cleanup preferences)
1b. switch the "unused imports" warning to an error (see error settings)
2a. configure a jre which does not include com.* stuff
2b. configure the warning of proprietary api usage from the jre to be an error
You might still want to check that on the build server, though. In this case the more complicated stuff like configuring CheckStyle would still be necessary.
I'm curious if there is any way to find unused imports programmatically (lets say from unit test) and fail build locally if there are any.
I use IntelliJ to organise imports, this removes all the unused imports. You can do this with one hot key from the top of you code base to correct all the imports. (It also has over 700 other types of static checks and fixes)
Maybe failing a build because of unused import sounds harsh, but if it saves time for team overall it makes sens to do so (would love to hear opinion on that as well).
I have IntelliJ check in code which formatted and with imports organised so the issue never arises in the first place. ;)
In Computer Science the name given to such a process of analyzing the code without executing is known as static code analysis.
Try using an IDE, I am using Eclipse, which marks all the Unused imports and Unused Variables or methods in with a Yellow color underline....
Aren't these unrelated questions? If you import classes only present in the local JDK, these imports are used (just unsatisfied). For either problem, I recommend solving it in the IDE so the problem will be detected when code is written, rather than prior to checkin (the earlier the detection, the easier the fix ...).
In eclipse, you could prevent unsatisfied imports with access rules, and automatically fix imports whenever a source file is saved by enabling the appropriate save action. If you check these settings into version control, you can easily share them with the team.
I see lot of comments in same way that use this IDE or that IDE. But all my friends try to understand the difference. Doing something programmatically is different and using IDE is different.
If I want a process to be programmatic then suggestion of IDE is not useful. It might be possible some one is asking this question because he is building complete process and this step is part of it. How opening IDE would help him on different machines and OS where CI is working?
I too building one tool on similar lines. I achieved it up to some level but it programmatically open IDE and close it automatically and fixes source code too. But opening same in Linux might be a question for me.
Understanding some one's view before answering is really very important.
I am new to maven and Jcodemodel. I am trying to create an archetype which will accept the wsdl and generate the code. I need to create java code for implementation class file by implementing the interface. I need to use JCodeModel to generate java code. But I am new to JCode. How to use Jcodemodel inside Maven archetype to generate java code?
Kindly, guide me
Maven will build your projects and automate some aspects of managing the builds and the project; but, it will not decide for you what the project does, or how to go about writing the project.
You might need to write Java code that (because you mentioned a WSDL) runs as a web service. That web service apparently should accept "something" and reply back with "something". It is a guess (this is a very vague question), that the something it should accept is some sort of description of a Java class, and the something it should reply will be either Java source code, or a compiled java class.
In either case, the project can be managed by Maven, meaning that Maven will compile, test, and package your project. In certain cases, it will also deploy it (if you configured Maven to do so).
Now if you want Maven to actually accept the WSDL and generate the code, then what Maven will build will be static, meaning that the "generated" code will not be able to change after Maven completes the build. You can; however, build the "next" version which might change. If this scenario sounds more like what you had in mind, then your "source" would be a static file, and the "built project" would be the source code or the object code corresponding to the source.
Without more direction, this is probably the best guide you are going to get. It is just too vague a question, covering too much ground to say much definitively. You also seem to lack a lot of knowledge in a lot key places simultaneously; perhaps the best solution is to identify what you will likely need to brush up on, order those by "what needs what" and start with the element that depends on nothing else.
The problem scienario is as follows (Note: this is not a cross-jar dependency issue, so tools like JarAnalyzer, ClassDep or Tattletale would not help. Thanks).
I have a big project which is compiled into 10 or more jar artifacts. All jars depend on each other and form a dependency hierarchy.
Whenever I need to modify one of the jars, I would check out the relevant source code and the source code for projects that depend on it. Modify the code, compile, repackage the jars. So far so good.
The problem is: I may forget to check one of the dependent projects, because inter-jar dependencies can be quite long, and may change with time. If this happens some jars may go "out-of-sync" and I will eventually get a NoSuchMethodException or a some other class incompatibility issue at run-time, which is what I want to avoid.
The only solution I can think of, the most straighforward one, is to check out all projects, and recompile the bunch. But this takes time, especially if I re-build it every small change. I do have a continuous integration server, that could do this for me, but it's shared with other developers, so seeing if the build breaks is not an option for me.
However, I do have all the jars so hypothetically it should be possible to verify jars which depend on the code that I modified have an inconsistency in method signature, class names, etc. But how could I perform such check?
Has anyone faced a similar problem before? If so, how did you solve it? Any tools or methodologies would be appreciated.
Let me know if you need clarification. Thanks.
EDIT:
I would like to clarify my question a little bit.
The ultimate goal of this task is to check that the changes that I have made will compile against the whole project. I am looking for a tool/technique that would aid me perform such check.
Consider this example:
You have 2 projects: A and B which are deployed as A.jar and B.jar respectively. A depends on B.
You wish to modify B, so you check it out and modify a method signature that A happens to depend on. You can compile B and run all tests by itself without any problems because B itself does not depend on anything. So you happily commit your changes.
In a few hours the complete project integration fails because A could not be compiled!
How do I avoid this?
The kind of tool I am looking for would retrieve A.jar and check that all dependencies in A on the new modified B are still fine. Like a potential compilation error that would happen if I were to recompile A and B sources together.
Another solution, as was suggested by many of you, is to set up a local continuous integration system that would recompile the whole project locally. I don't mind doing this, but I want to avoid doing it inside my workspace. On the other hand, if I check-out all sources to another temporary workspace, then I need to mirror my local changes to the temporary workspace.
This is quite a big issue in my team, as builds break very often because somebody forgot to check out (or open in Eclipse) the right set of projects. I tried persuading people to check-out source and recompile the bunch before commits, but not only it takes time, it needs running quite a few commands so most people just find it too troublesome to do. If the technique is not easy or automated, then it's unusable.
If you do not want to use your shared continuous integration server you should set up a local one on your developer machine where you perform the rebuild processes on change.
I know Jenkins - it is easy to setup (just start) on a local machine and I would advice to run it locally if no one is provided in the IT infrastructure that fits your needs.
Checking signatures is unfortunately not enough. Having the correct signatures does not mean it'll work. It's all about contracts and not just signatures. I mean what happens if the new version of a library has the same method signature, but accepts an ArrayList parameter now in reversed order? You will run into issues - sooner or later. I guess you maybe consider implementing tools like Ivy or Maven:
http://ant.apache.org/ivy/
http://maven.apache.org/
Yes it can be pain to implement it but once you have it it will "guard" your versions forever. You should never run into such an issue. But even those build tools are not 100% accurate. The only proper way of dealing with incompatible libraries, I know you won't like my answer, is extensive regression testing. For this you need bunch of testing tools. There are plenty of them out there: from very basic unit testing (JUnit) to database testing (JDBC Proxy) and UI testing frameworks like SWTBot (depends if your app is a web app or thick client).
Please note if your project gets really huge and you have large amount of dependencies you always not using all of the code there. Trying to check all interfaces and all signatures is way too much. Its not necessary to test it all when your code use lets say 30 % of the library code. What you need is to test what you really use. And this can be only done with extensive regression testing.
I have finally found a whole treasure box of answers at this post. Thanks for help, everyone!
The bounty goes to K. Claszen for the quickest and most input.
I'm also think that just setup local Jenkins is a best idea. What tool you use for build? Maybe you can improve you situation with switching to Maven as build tool? In more smart and don't recompile full project if you don't ask it directly. But switch to in can be HUGE paint in the neck - it hardly depends on how you project organized now...
And about VCS- exist Mercurial/SVN bridge - so you can use local Mercurial for you development ....
check this link: https://www.mercurial-scm.org/wiki/WorkingWithSubversion
There is a solution, jarjar, which allows to have different versions of the same library to be included multiple times in the dependency graph.
I use IntelliJ, not Eclipse, so maybe my answer is too IDE-specific. But in IntelliJ, I would simply include the modules from B into A, so that when I make changes to A, it breaks B immediately when compiling in the IDE. Modules can belong to multiple projects, so this is not anything like duplication, it's just adding references in the IDE to modules in other projects.