I'm working in Java development. I have recently come into a situation where I have to comply to coding standards: member and method ordering, naming conventions, modifier sequence. I am thinking about methods to either automate checking for compliance, or generate some sort of mechanism that does the reordering.
We're developing with Eclipse, but the technology would be open. One way it might work is to generate an external builder tool and add this to the projects. The disadvantage would be that it would automagically apply to all files, which could run into problems with legacy code, blowing up the error count to a degree where it is no longer a sensible metric of compliance. Also, it makes code reviews much more difficult, which is not wanted.
Another way would be some kind of parser with only informative capabilities. We could run a process inside Jenkins, and that certainly would work, but that would also mean that the code had already passed a review, which is usually a little late for a code compliance check.
Are there suggested or even easy methods to integrate such functionality into either IDE, the source control system (Mercurial) or even Jenkins? How is this enforced elsewhere?
I would not recommend doing such changes automatically. Even though most of the checkstyle/pmd complies are valid, it happens to me that I need to ignore some of the warnings/errors. Moreover - there is only very small pool of such easy issues. Most of the notifications require more complex operations and probably couldn't be done without human interaction.
I'm using Sonar integration. It contains many external checkers like PMD, CPD, Checkstyle, Findbugs and can integrate with some other useful tools like Cobertura (test coverage statistics). Its almost trivial to bind Sonar build to Jenkins build and trying to avoid major/critical issues might be considered as a good approach.
In developer environment I use Eclipse integration with findbugs. There is also some point of integration with sonar but it requires either submitting the code to server or running server locally, which I personally don't like. However after few cycles of polishing the code after code review in Sonar you will notice that you (and other team members) stick to most of the rules and checking reports on daily basis is enough.
One solution is to use JCSC / checkstyle or other tools that are command line friendly. Integrate this with your build process. Individual developer runs this on his branch.
Most tools integrate well with Jenkins (via plugins), which can be used as dash board
Further to #Jayan's answer, Jenkins has a CheckStyle plugin that will display the results of each CheckStyle run and let you set the build status depending on how many violations are found. So your setup steps would be:
Set up your CheckStyle rules to fit your coding standards
Add a step to your Jenkins build that runs CheckStyle
Add a post build step to publish the CheckStyle results.
Jayan mentions checkstyle, which is great for checking against coding standards.
I remember using Jalopy years ago for automated code formatting, it might suit your needs as well.
In all honesty though, I wouldn't reformat the code automatically. Using tools such as checkstyle to raise warnings is one thing. Taking control of a developer's source code is quite another, and most people find it terribly intrusive and unpleasant.
Also, a bug in a code checker will at worst generate incorrect warnings. A bug in a code beautifier might corrupt and destroy hours worth of work.
You can use JArchitect to check your best practice using CQLinq queries which is useful
to create easly your custom rules
JArchitect is free for open source contributors :http://www.jarchitect.com/JArchitectForOSS.aspx
Related
I am looking for a tool/plugin for JIRA/Eclipse that can,
Execute FindBugs over the code.
Based on generated report, provide a UI to mark bug or improvement in JIRA depending on the case. Note that the text should be copied directly from report to marked issue in JIRA.
Looked around on internet but could not find one. Please suggest if such a requirement is fulfilled by any other tool like PMD, etc.
Other answers have suggested that you should create a single task to deal with fixing FindBugs warnings. I'm not even sure that you should do that. The problems identified by FindBugs might not even be problems, and even if they are problems, might have little to no impact on the quality of the system under development. They really should be reviewed by a human instead of blindly added to a bug tracker as something to fix.
My suggestion would be to incorporate FindBugs (and any other static analysis tools) into your build process as well as your code review process. As part of your build process, it would generate reports after every build. As developers work on modules, they would be able to consult the FindBugs reports for those modules and address them as-needed. As part of the code review process, a developer (either the author or a reviewer) can manually launch FindBugs over the code under review, analyze the reports, and include specific issues as action items to handle coming out of the code review using your team's methodology.
That said, if you are using FindBugs 2.0, there appears to be a JIRA plugin to facilitate what you want to do. It appears to have limited support and must be built from source, but it might be something to look at. As I mentioned above, though, I wouldn't recommend it.
Does this really make sense? I mean, creating one JIRA issue for every FindBugs marker will take more time as fixing it immediately in most cases. Not to mention the huge amount of issues in which this will result. The developer will have to spend more time in closing JIRA issues than in fixing them.
I would create exactly one issue for this: "Fix FindBugs warnings before release".
I think I'd use one JIRA Task that says:
"Run FindBugs. Record the report somewhere. Fix the most obvious problems. Run FindBugs again and record the final report." That way everyone can see that work was done, and what work it was.
Storing information about each problem found by FindBugs in a JIRA issue would lead to a large number of issues, and many of them would quickly become outdated.
For the first part you can write a script in any language (I would have used python) to Execute FindBugs and manipulate the generated report in any way you wish and than implement the secod part by using the Jira remote API to create issues or a single issue as others suggested. To pick an API most suited check the manual for your Jira's version to see the full API capabilities.
Do we have any plugin/way in eclipse that forbids(show error/warning) while using some method. In my code, I want way to confirm that System.currentTimeMillis() / Calendar.getInstance() is not being called by anyone also if someone tries to use this, eclipse should show error.
Thanks
Ankush
You could write your own FindBugs plugin that relies on the FindBugs API to provide you with metadata about method invocations (among other opcodes found in the bytecode). You'll need to specifically implement a custom BytecodeScanningDetector that can verify if an operation involves the execution of a set of banned APIs (System.currentTimeMillis() and Calendar.getInstance()) in your case.
Here's a tutorial to get you started on writing a FindBugs plugin. You can use this plugin in Eclipse, but it is preferable to run it in a CI server as part of your build.
You may also find that other static analysis tools like PMD may have the same features, exposed using their own APIs.
Also, if you want to run this only as part of your build, you can delegate these checks to Sonar, which allows for easy construction of architectural rules like these. Note - I haven't tried this on methods, and so I'd warn that Sonar architectural constraints appear to be better suited when you want to ban entire class usages instead of method usages.
Eclipse is an IDE. It happens to include a Java compiler, but there are plenty of other Java compilers that run outside of Eclipse too. Eclipse shows errors returned by the Java compiler (the IDE itself isn't the one coming up with the errors).
Your best bet is to use Eclipse's ability to search for references within a project. It's a manual process, but it will find most references to what you want. Of course if someone uses reflection, all bets are off.
I've to audit the code-architecture quality and maintainability (in the end to be sure we have what we paid for) a Java EE web project based on JSF/CDI/EJB3.0/JPA (just to name some of the technologies involved).
This may not be the right place to ask but how do you deal with this kind of task?
Basically, I would proceed from coarse-grained to fine-grained, i.e. from the whole architecture to the java code.
Is it better to deal with each layer completely?
Should I spend more time on the low-level layers?
Do you assess the whole thing (build, deployment, test)?
At the lower physical/implementation level, what I like to do is adopt maven as a build tool, and then configure the extensive maven reporting, to produce a website full of various code metrics.
For starters there is the maven checkstyle plugin which can report on code conformance to a specified standard, consistency in coding standards has many obvious benefits, most projects would simply adopt one of the pre-configured standards e.g. sun or apache.
The findbugs plugin lists potential programming errors
There are a choice of code coverage reports, I've used cobertura. These show line for line in an application which parts are covered by unit tests. Maven supports unit tests in the build life-cycle, running them as part of the build. This has saved me a few times.
The PMD plugin identifies duplicated code, and highlights areas that may need refactoring.
Once this is setup and becomes part of the normal build cycle, it basically takes care of itself, and you won't have to worry about doing large bi-annual audits/catch-up.
Many of the reports have threshold limits that can be configured to fail the build if breached, i.e. more than n% checkstyle errors, cause a build failure.
Maven also promotes a modular approach to building applications, this results in smaller more understandable and re-usable modules, as well as separation of concerns, i.e. separate modules for presentation and persistence layers. The main benefit that maven provides is managing the inter-dependencies between the modules.
This doesn't help you very much at the higher-level architecture layers though, so a complementary approach will be required to cover that dimension.
See some sample reports at this link
http://maven.apache.org/plugins/maven-dependency-plugin/project-reports.html
To help in the code level audit and probably in project health too one software that can help is SONAR... it's very simple to setup just some maven commands, comes with a lot of proven code standards like code quality, reusability, bad practices measurements and so on...
it Runs on your project SVN or CVS and generate a website with graphics represent past and current status of the metrics it's creating, so you can navigate the project data and keep track of the improvements or faults.
It also uses all those maven and maven plugins listed in the other answer like cobertura, find bugs etc...
http://www.sonarsource.org/
Just download and point to your Repo.
In addition to the lower-level code metrics and static analysis already mentioned, I'd add a tool like Structure101 to help analyze higher-level structure and dependencies. It can also help in refactoring the same.
Identifying clusters of dependencies can help determine if the app was written with separation of concerns and modularity in mind, and can help identify potential pain points when considering extension or modification.
Be sure to break it down into areas of concern and address them individually. Areas I can think of to consider are:
Conformance to specified requirements (hopefully these are specific)
Performance/scalability
Code quality (including conventions you want followed)
Test coverage
Plugin/library licenses
It looks like others have addressed items 3 and 4. Since you're asking the question now (presumably after you've received the product) 1 and 2 will probably have to be manual process unless you have automated functionality tests already written (or want to automate tests so you can check future versions of what you bought). 5 is an item that is sometimes overlooked but can be VERY important. You probably don't want GPL code hooked in if you're going to be reselling this software. You need to review the licenses of every library included and decide which ones are compatible with your goals.
To Understand your architecture, you can try JavaDepend it gives the possibility to query code with CQL, like SQL for database, with more than 82 metrics and many interactive views to go deep inside your design, architecture and implementation.
I know there are some similar questions around in Stackoverflow, but they were either .Net related or didn't have any answer that helped us.
The case is as follows: with some friends we are starting an open source project. While setting the foundations of the hopefully successful project, a question arose: how to enforce the code conventions of the project?
The reasoning is that being an open source project, if people starts reformatting the code as they like, the patches will become cluttered with changes due to formatting that will hide the real "value" of the patch. So we want something that forces users to abide to a specific formatting, breaking the build if they don't.
The project is using Struts 2 + Spring + Hibernate, using Maven 2 (thinking on moving to Maven 3). We know we can use "CheckStyle" to test the Java files, but this leaves some questions open that hopefully someone can answer:
There is some tool to check the style of XML and SQL files, breaking the build if they don't abide to the rules?
There is some tool that automatically reformats the source files (Java, XML, SQL) to the desired convention? Can ti be integrated with Maven somehow?
We could find Jalopy for Java files, but we would prefer a free tool (as far as we know, the latest version is paid one). And we still couldn't find anything for SQL/XML.
UPDATE: just to make it clear: I'm not asking about PMD, Checkstyle or these tools. I'm asking for tools that:
Automatically format SQL, XML and/or Java files to a desired coding convention (4 spaces, etc)
Detect SQL and XML files with the wrong format and break a build because of that.
The standard tools I would use for java are
CheckStyle
FindBugs
PMD
All of these are capable of failing a Maven build when a certain threshold was reached.
But the question is of course: who runs those tools, and when?
My suggestion would be to get a Continuous Integration Server like hudson and configure it to run the maven checkstyle, findbugs and pmd goals (hudson has plugins to display graphs for all these tools), and you will always see who failed the build, because you can see who committed what between two builds.
It sounds to me like you're going along the route of creating an open source project and then giving commit access to all. I'd advise against this, whatever measures you try to put in place, someone stupid will come along at some point and sod the whole lot up.
Instead, I'd advise taking a different approach with commit access and handing it out only to people that prove they pay attention to the rules and restrictions you've put in place. They can generate patches, submit them to you and eventually if they're regular and trustworthy enough, you can hand out commit access.
One other point, something like Hudson may be of use here (it has a checkstyle plugin as well.) It won't prevent people committing crap code, but it can create a build and run checkstyle every so often, notifying people on a mailing list if someone screws up the code (either because it won't build or because someone's broken checkstyle rules.) Not foolproof, but this would enable you to keep an eye on things more easily.
Each of PMD, Checkstyle, Findbugs etc have Eclipse and other IDE plugins. You can also configure your IDE of choice to format according to the style you want, and then share this configuration (as a file) among members of the team.
I don't think you want your SCM server formatting files for you, otherwise after a check-in your local copy might be immediately out of date with whats in SCM. Everyone on the project would have to get used to doing an update immediately after a commit then, which sounds like a brittle process.
Instead, share your IDE configuration with the team (checking this config file in would be a good idea for what you want) and then fail the build if anyone violates the policy.
Yes, since eclipse, for example, has a build-in code-formatter, it could be possible that you may use the formatting-plugin to format all the java-code along a fixed configuration file, even without eclipse or any other GUI in a build process.
I'm the sole developer for an academic consortium headquartered at a university in the northeast. All of my development work involves internal tools, mostly in Java, so nothing that is released to the public. Right now, I feel like my development workflow is very "hobbyist" and is nothing like you would see at an experienced software development firm. I would be inclined to say that it doesn't really matter since I'm the only developer anyway, but it can't hurt to make some changes, if for no other reason than to make my job a little easier and get a few more technologies on my resume. Right now my workflow is something like this:
I do most of my development work in Eclipse on my laptop. Everything is saved locally on my laptop, and I don't use a VCS, nor do I really backup my code (except for occasionally emailing it to myself so I can see it on a different computer - yeah, I told you my development environment needs work).
When I'm done with a project and want to deploy it or if I just want to test it, I use the built-in Jar tool in Eclipse to make an executable .jar of my project. If I use external .jar libraries, I use the Fat-Jar plugin to include those .jars in my executable .jar.
After I create the .jar, I manually upload it to the server via SFTP and test it with something like java -jar MyProject.jar.
Oh yeah, did I mention that I don't unit test?
The most glaringly obvious problem that I'd like to fix first is my lack of source control. I like git because of it's distributed nature, but it doesn't seem to integrate with Eclipse well and I've heard that it doesn't work very well on Windows, which is my primary development OS. So, I'm leaning toward SVN, which I do have some experience with. I do have my own personal server, and I think I'll use that for my source control, because I'd rather be my own admin than have to deal with university bureaucracy. I had some trouble setting up SVN once before, but I'll give it another shot. Maybe I'll also install something like Trac or Redmine for bug-tracking, todo list, etc?
What about building and deployment? There has to be a better way than using Fat-Jar and manually uploading my jar to the server. I've heard about tools like Ant and Maven - do these apply to what I want to do? How can I get started using those?
I suppose I'd eventually like to integrate unit testing with JUnit too. Even though it probably should be, that is not my primary concern right now, because so far my applications aren't terribly complex. I'd really like to work on simplifying and streamlining my workflow right now, and then I'll ease into unit testing.
Sorry for the long question. I guess my question boils down to, for a sole developer, what tools and methodologies can/should I be using to not only make my job easier, but also just to expose myself to some technologies that would be expected requisite knowledge at a dedicated development house?
edit: Thanks for the great answers so far. I didn't mean to suggest that I wanted to make my workflow "enterprisey" just for the sake of doing it, but to make my job simpler and to get a few technologies under my belt that are typically used in enterprise development environments. That's all I meant by that.
It seems to me like you actually have a pretty good idea of what you need to do.
Using Subversion (or other VCS) is a must. Although it might be wise to setup a separate SVN repository for your work-related code rather than using a personal one.
You can integrate Subversion with Eclipse using a plugin like Subclipse, which I've found works pretty well.
I'd definitely use Ant or Maven - my preference is Ant because it's more flexible, and I think it would suit your development style more than Maven as well. But you might also want to look into Apache Ivy, which handles dependency-management.
Basically you set up an ant task which runs your compile, build and deployment steps - so that when you create a final JAR package you can be sure that it's been unit tested as that is part of your ant script. The best way to get started with ant is to look at some examples, and read through the manual.
As for unit testing - you can gradually build up with unit testing. I would recommend using JUnit in conjunction with a code coverage tool such as Cobertura (which is easy to set up) - it will help you to understand how much code your tests are covering and is an indicator about how effective your tests are.
It may also be worth your while setting up something like Trac - it's important to be able to keep track of bugs, and a wiki is surprisingly useful for documentation.
In other words, all of this sounds like you're on the right lines, you just need to get started using some of these tools!
If you're really set on distributed source control, I'd recommend you look at Bazaar. Its GIT-like distributed source control that's designed around performing very high quality merges. Out of the box it works on all platforms including Windows and they have a TortoiseBZR client.
Really though, any source control is better than none. If you're the sole developer then there's no need for anything more complex than SVN. Large companies and projects use SVN all the time with little issue.
As far as unit testing goes, you should make yourself familiar with JUnit. The fact that you're aware of unit testing and know you should be doing it is still several steps ahead of most ad-hoc developers.
Use version control. Period. SVN has great intergration with Eclipse and Windows. Get the TourtisSVN client for windows and use the subclipse plugin with Eclipse.
I would recomend getting an external HD or use one of your companies' servers for putting your repository on and do backups often. Subversion works great with deployment and upgrading as well. Just learn how to do it and you will never look back :)
As far as Unit Testing, some people would say that is the way to go but I have not found enough evidence to start the practice myself. If sombody on this question can convince me otherwise then please do!
Also, don't look to "enterprise" your workflow - look to make it better. Practices that work well with huge teams and corperations may not work well for you. I am pretty much an only developer myself and know the situation you are in. Just try everything and only keep what feels natural after a while.
But make sure to just try SVN! If your company has a LINUX server with apache see if you can set up your server there using DAV-SVN.
:)
I think you answered most of your own questions.
Source control: pick SVN - easy installation, great integration with Eclipse (subclipse).
Use Ant to build your project and deploy it (SCP/SFTP task)
Keep all your settings (Eclipse project settings, build xmls, etc.) in SVN.
Use Bugzilla to keep track of your bugs/issues/requests/ideas.
It would be VERY beneficial to start working with version control. Start now, don't delay! Git is moving REALLY fast, and there's already a TortoiseGit being developed. SVN is still a great standard to work with. And I have not worked with Mercurial, but that's another VCS that's worth looking into.
Other than that, I don't see why your workflow has to be enterprisey. It just has to be efficient and comfortable. That said, I think you should try working with a simple text editor and compiling from the command line. Most of the world's best programmers still use that instead of an IDE, and it will help you understand the processes underneath your favorite IDE.
Check out the Pragmatic Programmers' Pragmatic Starter Kit.
It schools you on the important basics of software development that universities/etc. seem to pass up, like version control, unit testing, and project automation (in that order), and does it in a very approachable manner.
It will give you a solid base to keep going from there.
Take a look # Matt Raible's Appfuse.
It incorporates Maven and Unit Testing.
http://raibledesigns.com/rd/tags/appfuse
Although you put it as the last thing, I think you should start using jUnit without delay.
The reason is, it's probably the easiest of the ambitions you've identified, and the tools are almost certainly already built into your Eclipse build.
Make a new folder in your project called 'jUnit'.
Let's say you have an Employee class, with setAnnualSalary() and getMonthlySalary() methods.
Right click on the jUunit folder, new -> "jUnit test case". This will make a new class. Call it TestEmployee. Eclipse generates the boilerplate stuff for you, as usual.
Add a void method with a name beginning with 'test':
public void testSalaryCalc() {
Employee emp = new Employee("John Doe");
emp.setAnnualSalary(12000);
assertEquals(1000,emp.getMonthlySalary());
}
Right click, "run as" -> "jUnit test". (the first time Eclipse may prompt you to do some setup for the project. Just do what it says.)
If Employee works properly, you'll see a green bar.
Sabotage the Employee class, run the test again, and you'll see a red bar, as well as output telling you what the failure was.
Congratulations: you are unit testing!
Right-clicking on the parent directory and choosing "Run as jUnit test" will run every Testcase class in the directory. Later you can incorporate jUnit into your build process, but don't worry about that for now.
Autocomplete will show you all the variations on assert() you can use. You can read up on it, and aim towards practices where you write the test cases before the implementation that passes them. But just doing the simple stuff above gets you big benefits.
Once you have a set up version control and a few unit-tests, I would consider a continous integration server (you wanted to be enterprisey, right?).
Even if you are and stay the sole developer, this might help you to uncover a few errors. Things you forgot to check in or the likes. A CI-server regularly checks out all your sources, does a clean build an runs all your tests. It also contacts you in the case of errors.
This gives you the guarantee that you (or any other person) is able to check out your code and build/run your projects.
I would recommend to take a look at Hudson
Like others have said, you already clearly know what you need to do. A VCS is a must, CI or bug-tracking may be overkill (for a single developer a spreadsheet might suffice for bug-tracking).
One thing that might benefit you greatly is keeping an organized product backlog. In solo development, I find keeping focused on the high-priority features and avoiding feature creep to be one of my biggest challenges. Keeping a backlog helps immensely. It doesn't have to be much more than a prioritized list of features with some notes about the scope of each. At my workplace, we keep this info in Trac, but here again, a spreadsheet may be all you need.
And I want to put in a plug for unit testing, particularly Test Driven Development (TDD). Kent Beck's book is a good place to start. I find that TDD helps keep me honest and focused on what I really need to do, particularly on a single-developer project without QA. Sometimes it seems like the code writes itself.
You got some really solid answers so short post adding a link to an article about Test Driven Development which is an agile practice that will look good on your CV.
TDD
If you are running a Windows Server where you want to put your SVN server, use Visual SVN as the server. It is super easy to setup and use, it supports both basic authentication and windows authentication. It is also free to use.
Eclipse has quite a few modules to integrate with a SVN server, so use one of those, or the already suggested Tortoise SVN.
All the previous comments covered almost everything you might ever need :-)
I want to add another approach on how to develop (the development workflow).
I suggest that you read the following article and although it's a git workflow you can use the same idea for any other tool you might be using.
http://nvie.com/posts/a-successful-git-branching-model/