Develop Reference

Java find where a class is used in the code - programmatically

I have a List of classes which I can iterate through. Using Java is there a way of finding out where these classes are used so that I can write it out to a report?
I know that I can find out using 'References' in Eclipse but there are too many to be able to do this manually. So I need to be able to do this programmatically. Can anyone give me any pointers please?
Edit:
This is static analysis and part of creating a bigger traceability report for non-technical people. I have comprehensive Javadocs but they are not 'friendly' and also work in the opposite direction to how I need the report. Javadocs start from package and work downwards, whereas I need to start a variable level and work upwards. If that makes any sense.

You could try to add a stacktrace dump somewhere in the class that isolates the specific case you are looking for.
public void someMethodInMyClass()
{
if (conditions_are_met_to_identify)
{
Thread.dumpStack();
}
// ... original code here
}

You may have to scan all the sources, and check the import statements. (Taking care of the * imports.. having to setup your scanner for both the fully Qualified class name and its packagename.*)
EDIT: It would be great to use the eclipse search engine for this. Perhaps here is the answer

Still another approach (probably not complete):
Search Google for 'java recursively list directories and files' and get source code that will recursively list all the *.java file path/names in a project.
For each file in the list:
1: See if the file path/name is in the list of fully qualified file names you are interested in. If so, record is path/name as a match.
2: Regardless if its a match or not, open the file and copy its content to a List collection. Iterate through the content list and see if the class name is present. If found, determine its path by seeing if its in the same package as the current file you are examining. If so, you have a match. If not, you need to extract the paths from the *.import statements, add it to the class name, and see if it exists in your recursive list of file path/names. If still not found, add it to a 'not found' list (including what line number it was found on) so you can manually see why it was not identified.
3: Add all matches to a 'found match' list. Examine the list to ensure it looks correct.

Not sure what you are trying to do, but in case you want to analyse code during runtime, I would use an out-of-the box profiler that shows you what is loaded and what allocated.
#Open source profilers: Open Source Java Profilers
On the other hand, if you want to do this yourself (During runtime) you can write your own custom profiler:
How to write a profiler?
You might also find this one useful (Although not exactly what you want):
How can I list all classes loaded in a specific class loader
http://docs.oracle.com/javase/7/docs/api/java/lang/instrument/Instrumentation.html
If what you are looking is just to examine your code base, there are really good tools out there as well.
#see http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis

How to script input for a Java program

I'm writing a Java program that requires its (technical) users to write scripts that it uses as input; it interprets these scripts into a series of actions and executes them. I am currently looking for the cleanest way to implement the script/configuration language. I was originally thinking of heading down the XML route, but the nature of the required input really is a procedural, linear flow of actions that need to be executed:
function move(Block b, Position p) {
// user-defined algorithm for moving block "b" to position "p"
}
Block a = getBlockA();
Position p = getPositionP();
move(a, p);
Etc. Please note: the above is an example only and does not constitute the exact syntax I am looking to achieve. I am still in the "30,000 ft view"-design phase, and don't know what my concreted scripting language will ultimately look like. I only provide this example to show that it is a flow/procedural script that the users must write, and that XML is probably not the best candidate for its implementation.
XML, perfect for hierarchial data, just doesn't feel like the best choice for such an implementation (although I could force it to work if need-be).
Not knowing a lick about DSLs, I've begun to read up on Groovy DSLs and they feel like a perfect match for what I need.
My uderstanding is that I could write, say, a Groovy (I'm stronger in Groovy than Scala, JRuby, etc.) DSL that would allow users to write scripts (.groovy files) that my program could then execute as input at runtime.
Is this correct, or am I misunderstanding the intent of DSLs altogether? If I am mistaken, does anybody have any suggestions for me? And if I am correct then how would a Java program read and execute a .groovy file (in other words, how would my program "consume" their script)?
Edit: I'm beginning to like ANTLR. Although I would love to roll up my sleeves and write a Groovy DSL, I don't want my users to be able to write any old Groovy program they want. I want my own "micro-language" and if users step outside of it I want the interpreter to invalidate the script. It's beginning to seem like Groovy/DSLs aren't the right choice, and maybe ANTLR could be the solution I need...?

I think you are on a really good path. Your users can write their files using your simple DSL and them you can run them by Evaling them at runtime. Your biggest challenge will be helping them to use the API of your DSL correctly. Unless they use an IDE this will be pretty tough.
Equivalent of eval() in Groovy

Yes, you can write a Groovy program that will accept a script as input and execute it. I recently wrote a BASIC DSL/interpreter in this way using groovy :
http://cartesianproduct.wordpress.com/binsic-is-not-sinclair-instruction-code/
(In the end it was more interpreter than DSL but that was to do with a peculiarity of Groovy that likely won't affect you - BASIC insists on UPPER CASE keywords which Groovy finds hard to parse - hence they have to be converted to lower case).
Groovy allows you to extend the script environment in various ways (eg injecting variables into the binding and transferring execution from the current script to a different, dynamically loaded script) which make this relatively simple.

Static code parser for Java source code to extract methods / comments

I'm looking for a parser that can extract methods from a java class (static source code -> .java file) and method signature, comments / documentation, variables of each of the methods. Preferably in Java programming language.
Could someone please advise?
Thanks.

You can use ASTParser by eclipse. Its super simple to use.
Find a quick standalone example here.

Here is what I do to extract the method signatures from a java file/s:
I use Sublime Text 2, to the file I want to get the signatures from and the do a find Ctrl+F with regular expression set for the following Regex I made (I tested it on my code and it works, I hope it will work for you too)
((synchronized +)?(public|private|protected) +(static [a-Z\[\]]+|[a-Z\[\]]+) [a-Z]+\([a-Z ,\[\]]*\)\n?[a-Z ,\t\n]*\{)
After Sublime Text 2 highlight my results I click on "Find All" then copy Ctrl+C, open a new tab Ctrl+N and paste Ctrl+V.
You will then see all your methods signatures.
I hope it helped.

If all you want is the exact text of each method, and the exact text of the variables inside methods, you could get by with a parser that produces a CST, walking the CST to find the right nodes, and then prettyprinting the found subtrees. ANTLR has a Java parser that would work for this. I don't know if it will capture comments. I think the main distribution of ANTLR is coded in Java.
You can likely do this more hackily, in Java, with a lexer for Java, implementing what amounts to a bad island parser that looks for the key phrases. ("After 'class', find '{' and print out everything you find up to the matching '}'" would give you all the methods and fields).
If you want more precise detail (e.g, you want to know the actual type of an argument rather than just its name, or where the type is actually defined) you'll need a parser with a full front end and name resolution. (ANTLR won't do this.) The Eclipse JDT certainly builds trees; it likely does name resolution. Our DMS Software Reengineering Toolkit with its Java Front End can provide everything necessary for this task, including comment capture and extraction. DMS isn't coded in Java.
You objected to Javadoc as being inadequate, because it doesn't give you the content of methods. Perhaps our Java Source Browser, which does give you that code, would serve better. It integrates name resolution data from our DMS/Java Front End to hyperlink JavaDoc-type information into browsable source text; all fields as well as local variables are explicitly indexed. The Source Browser isn't coded in Java, but then presumably you simply want to run it and scrape your result. Such scraping might be harder than it appears staring at the screen; there's a lot of HTML behind such a display.

Writing one file per group in Pig Latin

The Problem:
I have numerous files that contain Apache web server log entries. Those entries are not in date time order and are scattered across the files. I am trying to use Pig to read a day's worth of files, group and order the log entries by date time, then write them to files named for the day and hour of the entries it contains.
Setup:
Once I have imported my files, I am using Regex to get the date field, then I am truncating it to hour. This produces a set that has the record in one field, and the date truncated to hour in another. From here I am grouping on the date-hour field.
First Attempt:
My first thought was to use the STORE command while iterating through my groups using a FOREACH and quickly found out that is not cool with Pig.
Second Attempt:
My second try was to use the MultiStorage() method in the piggybank which worked great until I looked at the file. The problem is that MulitStorage wants to write all fields to the file, including the field I used to group on. What I really want is just the original record written to the file.
The Question:
So...am I using Pig for something it is not intended for, or is there a better way for me to approach this problem using Pig? Now that I have this question out there, I will work on a simple code example to further explain my problem. Once I have it, I will post it here. Thanks in advance.

Out of the box, Pig doesn't have a lot of functionality. It does the basic stuff, but more times than not I find myself having to write custom UDFs or load/store funcs to get form 95% of the way there to 100% of the way there. I usually find it worth it since just writing a small store function is a lot less Java than a whole MapReduce program.
Your second attempt is really close to what I would do. You should either copy/paste the source code for MultiStorage or use inheritance as a starting point. Then, modify the putNext method to strip out the group value, but still write to that file. Unfortunately, Tuple doesn't have a remove or delete method, so you'll have to rewrite the entire tuple. Or, if all you have is the original string, just pull that out and output that wrapped in a Tuple.
Some general documentation on writing Load/Store functions in case you need a bit more help: http://pig.apache.org/docs/r0.10.0/udf.html#load-store-functions

Can I automatically refactor an entire java project and rename uppercase method parameters to lowercase?

I'm working in a java project where a big part of the code was written with a formatting style that I don't like (and is also non standard), namely all method parameters are in uppercase (and also all local variables).
On IntellJ I am able to use "Analyze -> Inspect Code" and actually find all occurrences of uppercase method parameters (over 1000).
To fix one occurrence I can do "refactor > rename parameter" and it works fine (let's assume there is no overlapping).
Is there a way to automagically doing this refactor (e.g: rename method parameter starting with uppercase to same name starting with lowercase)?

Use a Source Parser
I think what you need to do is use a source code parser like javaparser to do this.
For every java source file, parse it to a CompilationUnit, create a Visitor, probably using ModifierVisitorAdapter as base class, and override (at least) visit(MethodDeclaration, arg). Then write the changed CompilationUnit to a new File and do a diff afterwards.
I would advise against changing the original source file, but creating a shadow file tree may me a good idea (e.g. old file: src/main/java/com/mycompany/MyClass.java, new file src/main/refactored/com/mycompany/MyClass.java, that way you can diff the entire directories).

I'd advise that you think about a few things before you do anything:
If this is a team effort, inform your team.
If this is for an employer, inform your boss.
If this is checked into a version control system, realize that you'll have diffs coming out the wazoo.
If it's not checked into a version control system, check it in.
Take a backup before you make any changes.
See if you have some tests to check before & after behavior hasn't changed.
This is a dangerous refactoring. Be careful.

I am not aware of any direct support for such refactoring out of the box in IDEs. As most IDEs would support name refactoring (which is regularly used). You may need to write some IDE plugin that could browse through source code (AST) and invoke rename refactoring behind the scene for such parameter names matching such format.

I have done a lot of such refactorings on a rather large scale of files, using TextPad or WildPad, and a bunch of reg-ex replace-all. Always worked for me!
I'm confident that if the code is first formatted using an IDE like Eclipse (if it is not properly formatted), and then a reg-ex involving the methods' signature (scope, return-type, name, bracket, arg list, bracket) can be devised, your job will be done in seconds with these tools. You might need more than one replace-all sets of reg-ex.
The only time-taking activity would be to come up with such a set of reg-ex.
Hope this helps!