Does anyone have a complete implementation (possibly github or googlecode) for using an ANTLR grammar file and Java source code to analyze Java source. For example, I want to simply be able to count the number of variables, method, etc.
Also using a recent version of ANTLR.
I thought I'd take a crack at this over my lunch break. This may not completely solve your problem, but it might give you a place to start. The example assumes you're doing everything in the same directory.
Download the ANTLR source from GitHub. The pre-compiled "complete" JAR from the ANTLR site contains a known bug. The GitHub repo has the fix.
Extract the ANTLR tarball.
% tar xzf antlr-antlr3-release-3.4-150-g8312471.tar.gz
Build the ANTLR "complete" JAR.
% cd antlr-antlr3-8312471
% mvn -N install
% mvn -Dmaven.test.skip=true
% mvn -Dmaven.test.skip=true package assembly:assembly
% cd -
Download a Java grammar. There are others, but I know this one works.
Compile the grammar to Java source.
% mkdir com/habelitz/jsobjectizer/unmarshaller/antlrbridge/generated
% mv *.g com/habelitz/jsobjectizer/unmarshaller/antlrbridge/generated
% java -classpath antlr-antlr3-8312471/target/antlr-master-3.4.1-SNAPSHOT-completejar.jar org.antlr.Tool -o com/habelitz/jsobjectizer/unmarshaller/antlrbridge/generated Java.g
Compile the Java source.
% javac -classpath antlr-antlr3-8312471/target/antlr-master-3.4.1-SNAPSHOT-completejar.jar com/habelitz/jsobjectizer/unmarshaller/antlrbridge/generated/*.java
Add the following source file, Main.java.
import java.io.IOException;
import java.util.List;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import com.habelitz.jsobjectizer.unmarshaller.antlrbridge.generated.*;
public class Main {
public static void main(String... args) throws NoSuchFieldException, IllegalAccessException, IOException, RecognitionException {
JavaLexer lexer = new JavaLexer(new ANTLRFileStream(args[1], "UTF-8"));
JavaParser parser = new JavaParser(new CommonTokenStream(lexer));
CommonTree tree = (CommonTree)(parser.javaSource().getTree());
int type = ((Integer)(JavaParser.class.getDeclaredField(args[0]).get(null))).intValue();
System.out.println(count(tree, type));
}
private static int count(CommonTree tree, int type) {
int count = 0;
List children = tree.getChildren();
if (children != null) {
for (Object child : children) {
count += count((CommonTree)(child), type);
}
}
return ((tree.getType() != type) ? count : count + 1);
}
}
Compile.
% javac -classpath .:antlr-antlr3-8312471/target/antlr-master-3.4.1-SNAPSHOT-completejar.jar Main.java
Select a type of Java source that you want to count; for example, VAR_DECLARATOR, FUNCTION_METHOD_DECL, or VOID_METHOD_DECL.
% cat com/habelitz/jsobjectizer/unmarshaller/antlrbridge/generated/Java.tokens
Run on any file, including the recently created Main.java.
% java -classpath .:antlr-antlr3-8312471/target/antlr-master-3.4.1-SNAPSHOT-completejar.jar Main VAR_DECLARATOR Main.java
6
This is imperfect, of course. If you look closely, you may have noticed that the local variable of the enhanced for statement wasn't counted. For that, you'd need to use the type FOR_EACH, rather than VAR_DECLARATOR.
You'll need a good understanding of the elements of Java source, and be able to take reasonable guesses at how those match to the definitions of this particular grammar. You also won't be able to do counts of references. Declarations are easy, but counting uses of a field, for example, requires reference resolution. Does p.C.f refer to a static field f of a class C inside a package p, or does it refer to an instance field f of the object stored by a static field C of a class p? Basic parsers don't resolve references for languages as complex as Java, because the general case can be very difficult. If you want this level of control, you'll need to use a compiler (or something closer to it). The Eclipse compiler is a popular choice.
I should also mention that you have other options besides ANTLR. JavaCC is another parser generator. The static analysis tool PMD, which uses JavaCC as its parser generator, allows you to write custom rules that could be used for the kinds of counts you indicated.
Related
From this answer, I learnt that, every Groovy script compiles to a class that extends groovy.lang.Script class
Below is a test groovy script written for Jenkins pipeline in Jenkins editor.
node('worker_node'){
print "***1. DRY principle***"
def list1 = [1,2,3,4]
def list2 = [10,20,30,40]
def factor = 2
def applyFactor = {e -> e * factor}
print(list1.each(applyFactor))
print(list2.each(applyFactor))
print "***2. Higher order function***"
def foo = { value, f -> f(value *2) }
foo(3, {print "Value is $it"})
foo(3){
print "Value is $it"
}
}
How to compile this groovy script to see the class generated(source code)?
The class generated is bytecode, not source code. The source code is the Groovy script.
If you want to see something similar to what the equivalent Java source code would look like, use groovyc to compile the script as usual, and then use a Java decompiler to produce Java source (this question's answers lists a few).
That's subject to the usual caveats on decompiled code, of course. High-level information is lost in the process of compiling. Decompilers have to guess a bit to figure out the best way to represent what might have been in the original source. For instance, what was a for loop in the original code may end up being decompiled as a while loop instead.
groovy in jenkins pipeline is a Domain Specific Language.
It's not a plain groovy.
However if you remove node(){ } then it seems to be groovy in your case.
and you can run it in groovyconsole or compile to class with groovyc
just download a stable groovy binary and extract it.
if you have java7 or java8 on your computer - you can run groovyconsole and try your code there.
with Ctrl+T you can see the actual class code generated for your script.
I want Matlab program to call a java file, preferably with an example.
There are three cases to consider.
Java built-in libraries.
That is, anything described here. These items can simply be called directly. For example:
map = java.util.HashMap;
map.put(1,10);
map.put(2,30);
map.get(1) %returns 10
The only complication is the mapping Matlab performs between Matlab data types and Java data types. These mappings are described here (Matlab to Java) and here (Java to Matlab). (tl; dr: usually the mappings are as you would expect)
Precompiled *.jar files
You first need to add these to Matlab's java class path. You can do this dynamically (that is, per-Matlab session, with no required Matlab state), as follows:
javaaddpath('c:\full\path\to\compiledjarfile.jar')
You can also add these statically by editing the classpath.txt file. For more information use docsearch java class path.
Precompiled *.class files.
These are similar to *.jar file, except you need to add the directory containing the class file, rather than the class files themselves. For example:
javaaddpath('c:\full\path\to\directory\containing\class\files\')
%NOT THIS: javaaddpath('c:\full\path\to\directory\containing\class\files\classname.class')
Ok, I'll try to give a mini-example here. Either use the java functions right from the Matlab window as zellus suggests, or, if need permits, create your own java class. Here's an example:
package testMatlabInterface;
public class TestFunction
{
private double value;
public TestFunction()
{
value = 0;
}
public double Add(double v)
{
value += v;
return value;
}
}
Then turn it into a jar file. Assuming you put the file in a folder called testMatlabInterface, run this command at the command line:
jar cvf testMatlab.jar testMatlabInterface
Then, in Matlab, navigate to the directory where your testMatlab.jar file is located and run the command, import testMatlabInterface.* to import all the classes in the testMatlabInterface package. Then you may use the class like so:
>> methodsview testMatlabInterface.TestFunction
>> me = testMatlabInterface.TestFunction()
me =
testMatlabInterface.TestFunction#7e413c
>> me.Add(10)
ans =
10
>> me.Add(10)
ans =
20
>> me.Add(10)
ans =
30
Let me know if I can be of further assistance.
after having looked at many discussions like for example :
Install commons math library for java in Ubuntu
Correctly Importing Apache Commons Math Package
I am still stuck and I am not able to make the following code to work :
import org.apache.commons.math3.linear;
class linearAlgebraLearning{
public static void main(String[] args){
// Create a real matrix with two rows and three columns, using a factory
// method that selects the implementation class for us.
double[][] matrixData = { {1d,2d,3d}, {2d,5d,3d}};
RealMatrix m = MatrixUtils.createRealMatrix(matrixData);
// One more with three rows, two columns, this time instantiating the
// RealMatrix implementation class directly.
double[][] matrixData2 = { {1d,2d}, {2d,5d}, {1d, 7d}};
RealMatrix n = new Array2DRowRealMatrix(matrixData2);
// Note: The constructor copies the input double[][] array in both cases.
// Now multiply m by n
RealMatrix p = m.multiply(n);
System.out.println(p.getRowDimension()); // 2
System.out.println(p.getColumnDimension()); // 2
// Invert p, using LU decomposition
RealMatrix pInverse = new LUDecomposition(p).getSolver().getInverse();
}
}
So here are what I have done step by step.
First I installed Apache using
sudo apt-get install libcommons-math3-java
Then I have looked where commons-math3-java has been installed.
dpkg -L libcommons-math3-java
/.
/usr
/usr/share
/usr/share/maven-repo
/usr/share/maven-repo/org
/usr/share/maven-repo/org/apache
/usr/share/maven-repo/org/apache/commons
/usr/share/maven-repo/org/apache/commons/commons-math3
/usr/share/maven-repo/org/apache/commons/commons-math3/3.2
/usr/share/maven-repo/org/apache/commons/commons-math3/3.2/commons-math3-3.2.pom
/usr/share/maven-repo/org/apache/commons/commons-math3/debian
/usr/share/maven-repo/org/apache/commons/commons-math3/debian/commons-math3-debian.pom
/usr/share/doc
/usr/share/doc/libcommons-math3-java
/usr/share/doc/libcommons-math3-java/changelog.Debian.gz
/usr/share/doc/libcommons-math3-java/copyright
/usr/share/java
/usr/share/java/commons-math3.jar
/usr/share/maven-repo/org/apache/commons/commons-math3/3.2/commons-math3-3.2.jar
/usr/share/maven-repo/org/apache/commons/commons-math3/debian/commons-math3-debian.jar
/usr/share/java/commons-math3-3.2.jar
then I used ( as told in the Install commons math library for java in Ubuntu )
javac -cp .:/usr/share/java/commons-math3-3.2.jar linearAlgebraLearning.java
however I still an import error message :
linearAlgebraLearning.java:1: error: cannot find symbol
import org.apache.commons.math3.linear;
And additional errors since the compiler does not find the classes (like RealMatrix). I know that this kind question has been asked many times. People here might be tired of seeing this question... But I would be really happy if you could help me.
Ps : Because there is some bug with Eclipse on my linux distribution I am not using IDE and use gedit and the terminal.
I installed every libcommons package first:
sudo apt install libcommons\*
then set the classpath:
export CLASSPATH="/usr/share/java/commons-math3.jar:/usr/share/java/commons-lang3.jar"
Your java should pick it up automatically. I tested it with jshell and it was able to autocomplete/import BlockRealMatrix for example:
jshell> import org.apache.commons.math3.linear.BlockRealMatrix
jshell> BlockRealMatrix foo = new BlockRealMatrix(2, 2);
foo ==> BlockRealMatrix{{0.0,0.0},{0.0,0.0}}
OK, again having some problems with caliper.
I am now running on Linux, trying to use the beta snapshot. I am attempting to run Google's caliper via commandline using just the jar. (Beta snapshot)
I do not have access to maven on this machine, and installing it is out of the question. I would just like to use a jar and, maybe once this is working, I can write up a script or something.
Here is what I am doing:
1. Using small example Benchmark:
import com.google.caliper.Benchmark;
public class Tutorial {
public static class Benchmark1 {
#Benchmark void timeNanoTime(int reps) {
for (int i = 0; i < reps; i++) {
System.nanoTime();
}
}
}
}
2. Compile with javac -cp caliper-1.0-beta-SNAPSHOT-all.jar Tutorial.java
3. (Attempt to) run with
java -cp caliper-1.0-beta-SNAPSHOT-all.jar com.google.caliper.runner.CaliperMain Tutorial.Benchmark1, receive message Benchmark class not found: Tutorial.Benchmark1.
I've tried to work this out from bits and pieces of information from various sources but I am really having a heck of a time with this. I would appreciate any input.
I believe you really need no maven, this should work.
Your own class doesn't get found and I think it's a problem of your classpath. As they're usually more problem with nested classes try simply
java -cp caliper-1.0-beta-SNAPSHOT-all.jar com.google.caliper.runner.CaliperMain Tutorial
If the message changes to something like "class contains no benchmarks", then you'll know more. If you insists on using nested class, you may need to call Tutorial$Benchmark1 (unprobable, but possible; java class naming is sick).
Please try also
java -cp caliper-1.0-beta-SNAPSHOT-all.jar Tutorial.Benchmark1
to see if your class lies on the classpath (the message should change to something like "no main method").
See also this older post.
I want Matlab program to call a java file, preferably with an example.
There are three cases to consider.
Java built-in libraries.
That is, anything described here. These items can simply be called directly. For example:
map = java.util.HashMap;
map.put(1,10);
map.put(2,30);
map.get(1) %returns 10
The only complication is the mapping Matlab performs between Matlab data types and Java data types. These mappings are described here (Matlab to Java) and here (Java to Matlab). (tl; dr: usually the mappings are as you would expect)
Precompiled *.jar files
You first need to add these to Matlab's java class path. You can do this dynamically (that is, per-Matlab session, with no required Matlab state), as follows:
javaaddpath('c:\full\path\to\compiledjarfile.jar')
You can also add these statically by editing the classpath.txt file. For more information use docsearch java class path.
Precompiled *.class files.
These are similar to *.jar file, except you need to add the directory containing the class file, rather than the class files themselves. For example:
javaaddpath('c:\full\path\to\directory\containing\class\files\')
%NOT THIS: javaaddpath('c:\full\path\to\directory\containing\class\files\classname.class')
Ok, I'll try to give a mini-example here. Either use the java functions right from the Matlab window as zellus suggests, or, if need permits, create your own java class. Here's an example:
package testMatlabInterface;
public class TestFunction
{
private double value;
public TestFunction()
{
value = 0;
}
public double Add(double v)
{
value += v;
return value;
}
}
Then turn it into a jar file. Assuming you put the file in a folder called testMatlabInterface, run this command at the command line:
jar cvf testMatlab.jar testMatlabInterface
Then, in Matlab, navigate to the directory where your testMatlab.jar file is located and run the command, import testMatlabInterface.* to import all the classes in the testMatlabInterface package. Then you may use the class like so:
>> methodsview testMatlabInterface.TestFunction
>> me = testMatlabInterface.TestFunction()
me =
testMatlabInterface.TestFunction#7e413c
>> me.Add(10)
ans =
10
>> me.Add(10)
ans =
20
>> me.Add(10)
ans =
30
Let me know if I can be of further assistance.