Develop Reference

why is there an issue with scope of the variable?

Can someone please help identify what is the issue with the below code... and why is there a problem in compiling for the variable "i$"
Below is the code....
private void handlePendingFilesForPreviousCheckpoints(Map<Long, List<String>> pendingFilesPerCheckpoint) {
LOG.debug("Moving pending files to final location on restore.");
Set<Long> pastCheckpointIds = pendingFilesPerCheckpoint.keySet();
Iterator i$ = pastCheckpointIds.iterator();
while(i$.hasNext()) {
Long pastCheckpointId = (Long)i$.next();
Iterator i$ = ((List)pendingFilesPerCheckpoint.get(pastCheckpointId)).iterator();
while(i$.hasNext()) {
String filename = (String)i$.next();
Path finalPath = new Path(filename);
Path pendingPath = this.getPendingPathFor(finalPath);
try {
if(this.fs.exists(pendingPath)) {
LOG.debug("Restoring BucketingSink State: Moving pending file {} to final location after complete checkpoint {}.", pendingPath, pastCheckpointId);
this.fs.rename(pendingPath, finalPath);
}
} catch (IOException var10) {
LOG.error("Restoring BucketingSink State: Error while renaming pending file {} to final path {}: {}", new Object[]{pendingPath, finalPath, var10});
throw new RuntimeException("Error while renaming pending file " + pendingPath + " to final path " + finalPath, var10);
}
}
}
}
Even though the i$ is defined inside curly braces second time... it says, the variable i$ is already defined in the scope....
Can someone please help me fix this... and understand what is wrong with the variable i$ in the above code.
Thanks a lot.

The scope of a variable is the (entire) block in which it's defined, which of course includes all sub-blocks within the declaring block.
To fix the problem, use different variable names.

You commented this:
but wondering, why this is creating problem... as i copied this method from other class and that is working fine... but mine is creating problem.... can you please explain
OK what you actually did is that you decompiled a class and copied the decompiled code into your source code.
That approach often doesn't work. And yours is an example where it hasn't worked.
The design goals for a typical Java decompiler is to translate compiled code into something that a programmer can read. The decompiled code above satisfies this. A typical Java programmer reading the above will understand what the original code was trying to do.
However there are a couple of non-goals for a decompiler:
It is NOT a goal to reproduce the original source code. (It is mathematically impossible to do that!)
It is NOT a goal to produce code that is "best practice" in style. That is impractical for a number of reasons. (For a start, people can't agree on what best practice is!)
In is NOT a goal to produce code that is always compilable. It might be possible for bytecodes that haven't been obfuscated, but:
it is really hard to cover the edge cases,
it is impossible to prove that you have covered all of the edge cases
and the set of compilers whose code you need to support is growing all of the time.
What does this mean in practice?
In practice code that is generated by a decompiler is generally1 readable, and may be compilable, but:
you may need to correct compilation errors
the code is not as maintainable as the original source code2
the code is not suitable for use as an example of how to program.
1 - However, if the bytecodes have been obfuscated, all bets are off. The purpose of obfuscation is to make it difficult to decompile code and / or understand the decompiled code.
2 - For a start, when you compile code, the compiler discards all source code comments and (typically) the names of local variables. Then it transforms various constructs such as enhanced loops, inner classes and lambdas into simpler forms that the JVM supports. These transformations are not reversible.

A simple (but possibly not the best) solution is to just call the inner iterator something else like ir or something.
Ex. Change this line:
Iterator i$ =
((List)pendingFilesPerCheckpoint.get(pastCheckpointId)).iterator();
to something like this:
Iterator ir =
((List)pendingFilesPerCheckpoint.get(pastCheckpointId)).iterator();

parsing incomplete Java source code

In certain problem I need to parse a Java source code fragment that is potentially incomplete. For example, the code can refer to variables that are not defined in such fragment.
In that case, I would still like to parse such incomplete Java code, transform it to a convenient inspectable representation, and being able to generate source code from such abstract representation.
What is the right tool for this ? In this post I found suggestions to use Antlr, JavaCC or the Eclipse JDT.
However, I did not find any reference regarding dealing with incomplete Java source code fragments, hence this question (and in addition the linked question is more than two years old, so I am wondering if something new is on the map).
As an example, the code could be something like the following expression:
"myMethod(aVarName)"
In that case, I would like to be able to somehow detect that the variable aVarName is referenced in the code.

Uhm... This question does not have anything even vaguely like a simple answer. Any of the above parser technologies will allow you to do what you wish to do, if you write the correct grammar and manipulate the parser to do fallback parsing unknown token passover sort of things.
The least amount of work to get you where you're going is either to use ANTLR which has resumable parsing and comes with a reasonably complete java 7 grammar, or see what you can pull out of the eclipse JDT ( which is used for doing the error and intention notations and syntax highlighting in the eclipse IDE. )
Note that none of this stuff is easy -- you're writing klocs, not just importing a class and telling it to go.
At a certain point of incorrect/incompleteness all of these strategies will fail just because no computer ( or even person for that matter ) is able to discern what you mean unless you at least vaguely say it correctly.

Eclipse contains just that: a compiler that can cope with incomplete java code (basically, that was one reason for these guys to implement an own java-compiler. (See here for better explanation)
There are several tutorials that explain the ASTParser, here is one.

If you just want basic parsing - an undecorated AST - you can use existing Java parsers. But from your question I understand you're interested in deeper inspection of the partial code. First, be aware the problem you are trying to solve is far from simple, especially because partial code introduces a lot of ambiguities.
But there is an existing solution - I needed to solve a similar problem, and found that a nice fellow called Barthélémy Dagenais has worked on it, producing a paper and a pair of open-source tools - one based on Soot and the other (which is generally preferable) on Eclipse. I have used both and they work, though they have their own limitations - don't expect miracles.
Here's a direct link to a quick tutorial on how to start with the Eclipse-based tool.

I needed to solve a similar problem in my recent work. I have tried many tools, including Eclipse JDT ASTParser, python javalang and PPA. I'd like to share my experience. To sum up, they all can parse code fragment to some extent, but all failed to parse occasionally when the code fragment is too ambiguous.
Eclipse JDT ASTParser
Eclipse JDT ASTParser is the most powerful and widely-used tool. This is a code snippet to parse the method invocation node.
ASTParser parser = ASTParser.newParser(AST.JLS8);
parser.setResolveBindings(true);
parser.setKind(ASTParser.K_STATEMENTS);
parser.setBindingsRecovery(true);
Map options = JavaCore.getOptions();
parser.setCompilerOptions(options);
parser.setUnitName("test");
String src = "System.out.println(\"test\");";
String[] sources = { };
String[] classpath = {"C:/Users/chenzhi/AppData/Local/Programs/Java/jdk1.8.0_131"};
parser.setEnvironment(classpath, sources, new String[] { }, true);
parser.setSource(src.toCharArray());
final Block block = (Block) parser.createAST(null);
block.accept(new ASTVisitor() {
public boolean visit(MethodInvocation node) {
System.out.println(node);
return false;
}
});
You should pay attention to parser.setKind(ASTParser.K_STATEMENTS), this is setting the kind of constructs to be parsed from the source. ASTParser defines four kind (K_COMPILATION_UNIT, K_CLASS_BODY_DECLARATIONS, K_EXPRESSION, K_STATEMENTS), you can see this javadoc to understand the difference between them.
javalang
javalang is a simple python library. This is a code snippet to parse the method invocation node.
src = 'System.out.println("test");'
tokens = javalang.tokenizer.tokenize(code2)
parser = javalang.parser.Parser(tokens)
try:
ast = parser.parse_expression()
if type(ast) is javalang.tree.MethodInvocation:
print(ast)
except javalang.parser.JavaSyntaxError as err:
print("wrong syntax", err)
Pay attention to ast = parser.parse_expression(), just like the parser.setKind() function in Eclipse JDT ASTParser, you should set the proper parsing function or you will get the 'javalang.parser.JavaSyntaxError' exception. You can read the source code to figure out which function you should use.
PPA
Partial Program Analysis for Java (PPA) is a static analysis framework that transforms the source code of an incomplete Java program into a typed Abstract Syntax Tree. As #Oak said, this tool came from academy.
PPA comes as a set of Eclipse plug-ins which means it need to run with Eclipse. It has provided a headless way to run without displaying the Eclipse GUI or requiring any user input, but it is too heavy.
String src = "System.out.println(\"test\");";
ASTNode node = PPAUtil.getSnippet(src, new PPAOptions(), false);
// Walk through the compilation unit.
node.accept(new ASTVisitor() {
public boolean visit(MethodInvocation node) {
System.out.println(node);
return false;
}
});

Java source refactoring of 7000 references

I need to change the signature of a method used all over the codebase.
Specifically, the method void log(String) will take two additional arguments (Class c, String methodName), which need to be provided by the caller, depending on the method where it is called. I can't simply pass null or similar.
To give an idea of the scope, Eclipse found 7000 references to that method, so if I change it the whole project will go down. It will take weeks for me to fix it manually.
As far as I can tell Eclipse's refactoring plugin of Eclipse is not up to the task, but I really want to automate it.
So, how can I get the job done?

Great, I can copy a previous answer of mine and I just need to edit a tiny little bit:
I think what you need to do is use a source code parser like javaparser to do this.
For every java source file, parse it to a CompilationUnit, create a Visitor, probably using ModifierVisitor as base class, and override (at least) visit(MethodCallExpr, arg). Then write the changed CompilationUnit to a new File and do a diff afterwards.
I would advise against changing the original source file, but creating a shadow file tree may me a good idea (e.g. old file: src/main/java/com/mycompany/MyClass.java, new file src/main/refactored/com/mycompany/MyClass.java, that way you can diff the entire directories).

Eclipse is able to do that using Refactor -> Change Method signature and provide default values for the new parameters.
For the class parameter the defaultValue should be this.getClass() but you are right in your comment I don't know how to do for the method name parameter.

IntelliJ IDEA shouldn't have any trouble with this.
I'm not a Java expert, but something like this could work. It's not a perfect solution (it may even be a very bad solution), but it could get you started:
Change the method signature with IntelliJ's refactoring tools, and specify default values for the 2 new parameters:
c: self.getClass()
methodName: Thread.currentThread().getStackTrace()[1].getMethodName()
or better yet, simply specify null as the default values.

I think that there are several steps to dealing with this, as it is not just a technical issue but a 'situation':
Decline to do it in short order due to the risk.
Point out the issues caused by not using standard frameworks but reinventing the wheel (as Paul says).
Insist on using Log4j or equivalent if making the change.
Use Eclipse refactoring in sensible chunks to make the changes and deal with the varying defaults.
I have used Eclipse refactoring on quite large changes for fixing old smelly code - nowadays it is fairly robust.

Maybe I'm being naive, but why can't you just overload the method name?
void thing(paramA) {
thing(paramA, THE_DEFAULT_B, THE_DEFAULT_C)
}
void thing(paramA, paramB, paramC) {
// new method
}

Do you really need to change the calling code and the method signature? What I'm getting at is it looks like the added parameters are meant to give you the calling class and method to add to your log data. If the only requirement is just adding the calling class/method to the log data then Thread.currentThread().getStackTrace() should work. Once you have the StackTraceElement[] you can get the class name and method name for the caller.

If the lines you need replaced fall into a small number of categories, then what you need is Perl:
find -name '*.java' | xargs perl -pi -e 's/log\(([^,)]*?)\)/log(\1, "foo", "bar")/g'
I'm guessing that it wouldn't be too hard to hack together a script which would put the classname (derived from the filename) in as the second argument. Getting the method name in as the third argument is left as an exercise to the reader.

Try refactor using intellij. It has a feature called SSR (Structural Search and Replace). You can refer classes, method names, etc for a context. (seanizer's answer is more promising, I upvoted it)

I agree with Seanizer's answer that you want a tool that can parse Java. That's necessary but not sufficient; what you really want is a tool that can carry out a reliable mass-change.
To do this, you want a tool that can parse Java, can pattern match against the parsed code, install the replacement call, and spit out the answer without destroying the rest of the source code.
Our DMS Software Reengineering Toolkit can do all of this for a variety of languages, including Java. It parses complete java systems of source, builds abstract syntax trees (for the entire set of code).
DMS can apply pattern-directed, source-to-source transformations to achieve the desired change.
To achieve the OP's effect, he would apply the following program transformation:
rule replace_legacy_log(s:STRING): expression -> expression
" log(\s) " -> " log( \s, \class\(\), \method\(\) ) "
What this rule says is, find a call to log which has a single string argument, and replace it with a call to log with two more arguments determined by auxiliary functions class and method.
These functions determine the containing method name and containing class name for the AST node root where the rule finds a match.
The rule is written in "source form", but actually matches against the AST and replaces found ASTs with the modified AST.
To get back the modified source, you ask DMS to simply prettyprint (to make a nice layout) or fidelity print (if you want the layout of the old code preserved). DMS preserves comments, number radixes, etc.\
If the exisitng application has more than one defintion of the "log" function, you'll need to add a qualifier:
... if IsDesiredLog().
where IsDesiredLog uses DMS's symbol table and inheritance information to determine if the specific log refers to the definition of interest.

Il fact your problem is not to use a click'n'play engine that will allow you to replace all occurences of
log("some weird message");
by
log(this.getClass(), new Exception().getStackTrace()[1].getMethodName());
As it has few chances to work on various cases (like static methods, as an example).
I would tend to suggest you to take a look at spoon. This tool allows source code parsing and transformation, allowing you to achieve your operation in a -obviously code based- slow, but controlled operation.
However, you could alos consider transforming your actual method with one exploring stack trace to get information or, even better, internally use log4j and a log formatter that displays the correct information.

I would search and replace log( with log(#class, #methodname,
Then write a little script in any language (even java) to find the class name and the method names and to replace the #class and #method tokens...
Good luck

If the class and method name are required for "where did this log come from?" type data, then another option is to print out a stack trace in your log method. E.g.
public void log(String text)
{
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw, true);
new Throwable.printStackTrace(pw);
pw.flush();
sw.flush();
String stackTraceAsLog = sw.toString();
//do something with text and stackTraceAsLog
}

Refactoring advice and tools

I have some code that consists of a lot (several hundreds of LOC) of uggly conditionals i.e.
SomeClass someClass = null;
if("foo".equals(fooBar)) {
// do something possibly involving more if-else statments
// and possibly modify the someClass variable among others...
} else if("bar".equals(fooBar)) {
// Same as above but with some slight variations
} else if("baz".equals(fooBar)) {
// and yet again as above
}
//... lots of more else ifs
} else {
// and if nothing matches it is probably an error...
// so there is some error handling here
}
// Some code that acts on someClass
GenerateOutput(someClass);
Now I had the idea of refactoring this kind of code something along the lines of:
abstract class CheckPerform<S,T,Q> {
private CheckPerform<T> next;
CheckPerform(CheckPerform<T> next) {
this.next = next;
}
protected abstract T perform(S arg);
protected abstract boolean check(Q toCheck);
public T checkPerform(S arg, Q toCheck) {
if(check(toCheck)) {
return perform(arg);
}
// Check if this CheckPerform is the last in the chain...
return next == null ? null : next.checkPerform();
}
}
And for each if statment generate a subclass of CheckPerform e.g.
class CheckPerformFoo extends CheckPerform<SomeInput, SomeClass, String> {
CheckPerformFoo(CheckPerform<SomeInput, SomeClass, String> next) {
super(next);
}
protected boolean check(String toCheck) {
// same check as in the if-statment with "foo" above"
returs "foo".equals(toCheck);
}
protected SomeClass perform(SomeInput arg) {
// Perform same actions (as in the "foo" if-statment)
// and return a SomeClass instance (that is in the
// same state as in the "foo" if-statment)
}
}
I could then inject the diffrent CheckPerforms into eachother so that the same order of checks are made and the corresponding actions taken. And in the original class I would only need to inject one CheckPerform object. Is this a valid approach to this type of problem? The number of classes in my project is likely to explode, but atleast I will get more modular and testable code. Should I do this some other way?
Since these if-else-if-...-else-if-else statments are what I would call a recurring theme of the code base I would like to do this refactoring as automagically as possible. So what tools could I use to automate this?
a) Some customizable refactoring feature hidden somewhere in an IDE that I have missed (either in Eclipse or IDEA preferably)
b) Some external tool that can parse Java code and give me fine grained control of transformations
c) Should I hack it myself using Scala?
d) Should I manually go over each class and do the refactoring using the features I am familiar with in my IDE?
Ideally the output of the refactoring should also include some basic test code template that I can run (preferably also test cases for the original code that can be run on both new and old as a kind of regression test... but that I leave for later).
Thanks for any input and suggestions!

What you have described is the Chain of Responsibility Pattern and this sounds like it could be a good choice for your refactor. There could be some downsides to this.
Readability Because you are going to be injecting the the order of the CheckPerformers using spring or some such, this means that it is difficult to see what the code will actually do at first clance.
Maintainence If someone after you wants to add a new condition, as well as adding a whole new class they also have to edit some spring config. Choosing the correct place to add there new CheckPerformer could be difficult and error prone.
Many Classes Depending on how many conditions you have and how much repeated code within those conditions you could end up with a lot of new classes. Even though the long list of if else its very pretty, the logic it in one place, which again aids readability.

To answer the more general part of your question, I don't know of any tools for automatic refactoring beyond basic IDE support, but if you want to know what to look for to refactor have a look at the Refactoring catalog. The specific of your question are covered by replace conditional with Polymorphism and replace conditional with Visitor.

To me the easiest approach would involve a Map<String, Action>, i.e. mapping various strings to specific actions to perform. This way the lookup would be simpler and more performant than the manual comparison in your CheckPerform* classes, getting rid of much duplicated code.
The actions can be implemented similar to your design, as subclasses of a common interface, but it may be easier and more compact to use an enum with overridden method(s). You may see an example of this in an earlier answer of mine.
Unfortunately I don't know of any automatic refactoring which could help you much in this. Earlier when I did somewhat similar refactorings, I wrote unit tests and did the refactoring step-by-step, manually, using automated support at the level of Move Method et al. Of course since the unit tests were pretty similar to each other in their structure, I could reuse part of the code there.
Update
#Sebastien pointed out in his comment, that I missed the possible sub-ifs within the bigger if blocks. One can indeed use a hierarchy of maps to resolve this. However, if the hierarchy starts to be really complex with a lot of duplicated functionality, a further improvement might be to implement a DSL, to move the whole mapping out of code into a config file or DB. In its simplest form it might look something like
foo -> com.foo.bar.SomeClass.someMethod
biz -> com.foo.bar.SomeOtherClass.someOtherMethod
baz -> com.foo.bar.YetAnotherClass.someMethod
bar -> com.foo.bar.SomeOtherClass.someMethod
biz -> com.foo.bar.DifferentClass.aMethod
baz -> com.foo.bar.AndAnotherClass.anotherMethod
where the indented lines configure the sub-conditions for each bigger case.

Java Code Use Checker

I am working on a library where we want to determine how much of our library is being used. I.E. we want to know how many methods in our library are public, but never being called.
Goal:
Static Analysis
Determine how many lines of code call each public method in package A in the current project. If the number of calls is zero, the method should be reported as such.

I belive you are looking for this eclipse plugin --> UCDetector
From the documentation (pay notice to second bullet point)
Unnecessary (dead) code
Code where the visibility could be changed to protected, default or
private
Methods of fields, which can be final
On Larger scale, if you want to do Object Level Static Analysis, look at this tool from IBM -->Structural Analysis for Java. It is really helpful for object analysis of libraries, APIs, etc.

Not exactly what you are looking for, but:
Something similar be done with code coverage tools (like Cobertura). They do not do static inspection of the source code, but instrument the bytecode to gather metrics at runtime. Of course, you need to drive the application in a way that exercises all usage pattern, and might miss the rarer code paths.
On the static analysis front, maybe these tools can help you (the Apache project uses them to check for API compatibility for new releases, seems like that task is somewhat related to what you are trying to do):
Clirr is a tool that checks Java libraries for binary and source compatibility with older releases. Basically you give it two sets of jar files and Clirr dumps out a list of changes in the public api.
JDiff is a Javadoc doclet which generates an HTML report of all the packages, classes, constructors, methods, and fields which have been removed, added or changed in any way, including their documentation, when two APIs are compared.

Client use of reflective calls is one hole in static analysis to consider. As there's no way to know for sure that a particular method isn't being called via some bizarre reflection scheme. So, maybe a combination of runtime and static analysis might be best.

I don't think you are able to measure how "often" a class or a function is needed.
There are some simple questions:
What defines, if a usage statistic of your game library is "normal" or an "outlier"? Is it wrong to kill yourself in the game too often? You would use the "killScreen" class more frequently like a good gamer.
What defines "much"? Time or usage count? POJOs will consume rare time, but are used pretty frequently.
Conclusion:
I don't know what you are trying to accomplish.
If you want to display your code dependencies, there are other tools for doing this. If you're trying to measure your code execution, there are profiler or benchmarks for Java. If you are a statistic geek, you'll be happy with RapidMiner ;)
Good luck with that!

I would suggest JDepend shows you the dependencies between packages and classes, excellent to find cyclic dependencies!
http://clarkware.com/software/JDepend.html
(it has an eclipse plugin: http://andrei.gmxhome.de/jdepend4eclipse/
and also PMD for other metrics
http://pmd.sourceforge.net/

IntelliJ has a tool to detect methods, fields, class which can have more restricted modifiers. It also a has a quick fix to apply these changes which can save you a lot of work as well. If you don't want to pay for it, you can get the 30-day eval license which is more than enough time to change your code, its not something your should need to do very often.
BTW: IntelliJ has about 650 code inspections to improve code quality, about half has automatic fixes so I suggest spend a couple of day using it to refactor/tidy up your code.

Please take a look at Dead Code Detector. It claims to do just what you are looking for: finding unused code using static analysis.

Here's are a few lists of Java code coverage tools. I haven't used any of these personally, but it might get you started:
http://java-source.net/open-source/code-coverage
http://www.codecoveragetools.com/index.php/coverage-process/code-coverage-tools-java.html

Proguard may be an option too (http://proguard.sourceforge.net/):
"Some uses of ProGuard are:
...
Listing dead code, so it can be removed from the source code.
... "
See also http://proguard.sourceforge.net/manual/examples.html#deadcode

You could write your own utility for that (within an hours after reading this) using the ASM bytecode analysis library (http://asm.ow2.org). You'll need to implement a ClassVisitor and a MethodVisitor. You'll use a ClassReader to parse the class files in your library.
Your ClassVisitor's visitMethod(..) will be called for each declared method.
Your MethodVisitor's visitMethodInsn(..) will be called for each called method.
Maintain a Map to do the counting. The keys represent the methods (see below). Here's some code:
class MyClassVisitor {
// ...
public void visit(int version, int access, String name, ...) {
this.className = name;
}
public MethodVisitor visitMethod(int access, String name, String desc, ...):
String key = className + "." + name + "#" + desc;
if (!map.containsKey() {
map.put(key, 0);
}
return new MyMethodVisitor(map);
}
// ...
}
void class MyMethodVisitor {
// ...
public visitMethodInsn(int opcode, String name, String owner, String desc, ...) {
String key = owner + "." + name + "#" + desc;
if (!map.containsKey() {
map.put(key, 0);
}
map.put(key, map.get(key) + 1);
}
// ...
}
Basically that's it. Your're starting the show with something like this:
Map<String,Integer> map = new HashMap<String,Integer>();
for (File classFile : my library) {
InputStream input = new FileInputStream(classFile);
new ClassReader(input).accept(new MyClassVisitor(map), 0);
input.close();
}
for (Map.Entry<String,Integer> entry : map.entrySet()) {
if (entry.getValue() == 0) {
System.out.println("Unused method: " + entry.getKey());
}
}
Enjoy!