This problem is sort of a continuation of How to write visitor classes for collections? - I tried that answer, but I find that the code works in eclipse, but has a null pointer problem in unix or windows. So now it looks like a different problem, so I created a new question.
I have uploaded the full code at https://sites.google.com/site/rogergdmn/ , Below is the summary.
Here are the details (the code is a variation of the LabeledExpr.g4 from the book) - I am trying to create an intermediate data structure by using the visitor classes. When I run in command line (in unix or windows), the line "why null e" is printed, but this line is not printed when I run in eclipse. How do I fix this bug?
This is the grammar:
prog: stat+ ;
stat: expr NEWLINE # printExpr
| NEWLINE # blank
;
expr: INT # int
;
These are the functions in EvalVisitor.java:
public Object visitInt(ExprParser.IntContext ctx) {
System.out.printf("visited----- 1\n");
int value = Integer.valueOf(ctx.INT().getText());
return new E(1, value);
}
public Object visitPrintExpr(ExprParser.PrintExprContext ctx) {
System.out.printf("visited----- 2\n");
E e = (E) visit(ctx.expr()); // evaluate the expr child
return new E(2, e);
}
public Object visitProg(ExprParser.ProgContext ctx) {
System.out.printf("visited----- 3\n");
List<ExprParser.StatContext> sL = ctx.stat();
List<E> eL = new ArrayList<E>();
for(ExprParser.StatContext s : sL) {
E e = (E) visit(s);
if(e==null) System.out.printf("why null e??\n");
eL.add(e);
}
return new E(7, eL);
}
This is the E class (only constructors present, to demonstrate the error):
public class E {
int typ;
int val;
E e1;
List<E> eL;
public E(int _typ, int _v) { typ = _typ; val = _v; }
public E(int _typ, E _e1) { typ = _typ; e1 = _e1; }
public E(int _typ, List<E> _eL) { typ = _typ; eL = _eL; }
}
The other codes are directly from the book example (http://pragprog.com/titles/tpantlr2/source_code , directory "starter").
BTW, a similar visitor code is shown at If/else statements in ANTLR using listeners and https://github.com/bkiers/Mu, and that code works fine for me. My code is pretty similar to that code, so I am not sure whats going wrong here.
You didn't override visitBlank, so any blank statement will return null from the visit method.
Edit: The difference between Eclipse and the other case is one of the following:
You could be using different input in the two cases. You'll need to examine the raw contents of the TokenStream to be sure.
You didn't include an explicit EOF at the end of your prog rule, so there is a possibility that your parser is ignoring some tokens of the input. To ensure all tokens are considered in all cases, add a reference to EOF at the end of the prog rule.
Related
I'm writing an app using iText 5G, I have seen a piece of code to doesn't allow that tables split at page end but that code was written in C#, I have converted almost of all the code to Java but I'm stuck in one line that I don't know the equivalent in Java.
The C# code:
public override IList<iTextSharp.text.IElement> End(iTextSharp.tool.xml.IWorkerContext ctx, iTextSharp.tool.xml.Tag tag, IList<iTextSharp.text.IElement> currentContent)
{
string keeprowstogetherattr = "keeprowstogether";
var retval = base.End(ctx, tag, currentContent);
if (tag.Attributes.ContainsKey(keeprowstogetherattr) && tag.Attributes[keeprowstogetherattr] == "true")
{
foreach (PdfPTable table in retval.OfType<PdfPTable>())
{
table.KeepRowsTogether(0);
}
}
return retval;
}
My Java code at the moment:
public List<Element> End(WorkerContext ctx, Tag tag, List<Element> currentContent)
{
{
String keeprowstogetherattr = "keeprowstogether";
List<Element> retval = super.end(ctx, tag, currentContent);
if (tag.getAttributes().containsKey(keeprowstogetherattr) && tag.getAttributes().get(keeprowstogetherattr) == "true");
{
for (PdfPTable table : retval.OfType<PdfPTable>())
{
table.keepRowsTogether(0);
}
}
return retval;
}
}
The line that I'm stuck:
PdfPTable table : retval.OfType<PdfPTable>()
I don't know the equivalent to C#'s OfType in Java.
Thanks.
EDIT: My min API is 19, so stream() doesn't work.
If you can't use the stream filters, the simplest way is probably to do an instanceof check inside the for loop:
for(Element e: retval) {
if(e instanceof PdfPTable) {
((PdfPTable) e).keepRowsTogether(0);
}
}
As stylistic notes, you should name your methods starting with lowercase letters (end), and that keeprowstogetherattr should be a constant field named KEEP_ROWS_TOGETHER_ATTR (or better yet, ATTR_KEEP_ROWS_TOGETHER) in both C# and Java.
I have started to use JSqlParser, i can parse a Where clause but i don't manage to go further with it.
JSqlParser github link
Indeed, i have tried to Override visit methods but don't understand how to reach my goal.
Let's say i have: (a=1 AND (b=2 OR (c=3 AND d=4))) OR e=2 as input and i would like as output:
-> a=1
-> AND
--> b=2
--> OR
---> c=3
---> AND
---> d=4
-> OR
-> e=5
My final goal is not a pretty print, but to understand how to know the current depth of the print at a given state. In order to dig in the tree as i want.
I have started a piece of code, parsing the where clause:
Expression expr = CCJSqlParserUtil.parseCondExpression("(a=1 AND (b=2 OR >(c=3 AND d=4))) OR e=2");
expr.accept(new ExpressionVisitorAdapter() {
#Override
public void visit(ExpressionClassToChoose expr) {
some code here...
}
});
Could you show me the way to start this kind of thing? Do i have to work with visit methods or am i wrong?
Thanks !
The visitor way is as always one way to go. If you want to get the parenthesis depth, you could stick with:
public void visit(Parenthesis parenthesis)
To get the ouput you want, is a little bit trickier. I implemented a simple example only taking ANDs and ORs into account. I do not use the ExpressionVisitorAdapter, but the ExpressionDeParser which is responsible for Expression printing. This way you can modify the generated output to your needs.
public static void main(String args[]) throws JSQLParserException {
Expression expr = CCJSqlParserUtil.parseCondExpression("(a=1 AND (b=2 OR (c=3 AND d=4))) OR e=2");
StringBuilder b = new StringBuilder();
expr.accept(new ExpressionDeParser(null, b) {
int depth = 0;
#Override
public void visit(Parenthesis parenthesis) {
if (parenthesis.isNot()) {
getBuffer().append("NOT");
}
depth++;
parenthesis.getExpression().accept(this);
depth--;
}
#Override
public void visit(OrExpression orExpression) {
visitBinaryExpr(orExpression, "OR");
}
#Override
public void visit(AndExpression andExpression) {
visitBinaryExpr(andExpression, "AND");
}
private void visitBinaryExpr(BinaryExpression expr, String operator) {
if (expr.isNot()) {
getBuffer().append("NOT");
}
if (!(expr.getLeftExpression() instanceof OrExpression)
&& !(expr.getLeftExpression() instanceof AndExpression)
&& !(expr.getLeftExpression() instanceof Parenthesis)) {
getBuffer().append(StringUtils.repeat("-", depth)).append(">");
}
expr.getLeftExpression().accept(this);
getBuffer().append("\n").append(StringUtils.repeat("-", depth)).append(">");
getBuffer().append(operator).append("\n");
if (!(expr.getRightExpression() instanceof OrExpression)
&& !(expr.getRightExpression() instanceof AndExpression)
&& !(expr.getRightExpression() instanceof Parenthesis)) {
getBuffer().append(StringUtils.repeat("-", depth)).append(">");
}
expr.getRightExpression().accept(this);
}
});
System.out.println(b);
}
As you can see, the parenthesis visitor changes the depth. The hard part is within visitBinaryExpression. The complex instanceof logic derives from using the And/OrExpression for output. Since for one text line multiple calls to visitBinaryExpression could happen, the indent has to be done from the most outer part.
If you would like to improve the printing of an JSqlParser parsed statement your updates should go to the deparsers.
I have a set of incoming records, that needs to be evaluated under a set of logical clauses defined and stored. An example logical clause be like :
Acct1 != 'Y' AND Acct2 > 1004 AND Acct3 >= 96 AND Acct4 < 1004 AND Acct5 = 99 AND ((Acct6 <= 9090 OR Acct7 IN (A1,A2,A6) AND Acct1 NOT IN (A3,A4)) AND Formatted LIKE 'LINUX' AND Acct9 NOT LIKE 'WINDOWS' AND (Acct10 = 'N' AND NOT Acct11 = 'N') AND EditableField BETWEEN (10 AND 20) )
My data input to the clause be like :
map.put(Acct1,"Y")
map.put(Acct2,1010)
map.put(Acct3,99)
map.put(Acct4,1015)
map.put(Acct5,99)
map.put(Acct6,9090)
map.put(Acct7,"A3")
map.put(Formatted,"LINUX_INST")
map.put(Updated,"LINUX_TMP")
map.put(Acct10,"Y")
map.put(Acct11,"N")
map.put(EditableFIeld,25)
I have to evaluate the incoming records populated into the map onto the clause defined above and print true or false based on the evaluation result.
The clause conditions and map values will be changed and executed as well.
I have the following conditional clauses to be evaluated:
!=
>
>=
<
=
<=
IN(
NOT IN(
LIKE(
NOT LIKE(
BETWEEN(
AND
OR
AND NOT
OR NOT
I have tried using grammar generators but I am told that it is not a recommended solution for our application hence I am looking for java code and I have this detailed example for reference to AND,OR,=.
resolving logical operations - AND, OR, looping conditions dynamically and looking for snippets to build on top of that if possible.
If you want to avoid a parser generator, consider using a StreamTokenizer to implement a recursive descent parser, with one method for each grammar rule.
For a subset of your grammar, this should look roughly like this (and should be straightforward to extend to your full grammar):
public class Parser {
public static Node parse(String expr) {
StreamTokenizer tokenizer =
new StreamTokenizer(new StringReader(expr));
tokenizer.nextToken();
Parser parser = new Parser(tokenizer);
Node result = parser.parseExpression();
if (tokenizer.ttype != StreamTokenizer.TT_EOF) {
throw new RuntimeException("EOF expected, got "
+ tokenizer.ttype + "/" + tokenizer.sval);
}
private StreamTokenizer tokenizer;
private Parser(StreamTokenizer tokenizer) {
this.tokenizer = tokenizer;
}
private Node parseExpression() {
Node left = parseAnd();
if (tokenizer.ttype == StreamTokenizer.TT_WORD
&& tokenizer.sval.equals("OR")) {
tokenizer.nextToken();
return new OperationNode(OperationNode.Type.OR,
left, parseExpression());
}
return left;
}
private Node parseAnd() {
Node left = parseRelational();
if (tokenizer.ttype == StreamTokenizer.TT_WORD
&& tokenizer.sval.equals("AND")) {
tokenizer.nextToken();
return new OperationNode(OperationNode.Type.AND,
left, parseAnd());
}
return left;
}
private Node parseRelational() {
Node left = parsePrimary();
OperationNode.Type type;
switch (tokenizer.ttype) {
case '<': type = OperationNode.Type.LESS; break;
case '=': type = OperationNode.Type.EQUAL; break;
case '>': type = OperationNode.Type.GREATER; break;
default:
return left;
}
tokenizer.nextToken();
return new OperationNode(type, left, parseRelational());
}
private Node parsePrimary() {
Node result;
if (tokenizer.ttype == '(') {
tokenizer.nextToken();
result = parseExpression();
if (tokenizer.ttype != ')') {
throw new RuntimeException(") expected, got "
+ tokenizer.ttype + "/" + tokenizer.sval);
}
} else if (tokenizer.ttype == '"' || tokenizer.ttype == '\'') {
result = new LiteralNode(tokenizer.sval);
} else if (tokenizer.ttype == TT_NUMBER) {
result = new LiteralNode(tokenizer.nval);
} else if (tokenizer.ttype == StreamTokenizer.TT_WORD) {
result = new FieldNode(tokenizer.sval);
} else {
throw new RuntimeException("Unrecognized token: "
+ tokenizer.ttype + "/" + tokenizer.sval);
}
tokenizer.nextToken();
return result;
}
}
This assumes a Node object hierarchy like this:
interface Node {
Object eval(Map<String,Object> data);
}
class FieldNode implements Node {
private String name;
FieldNode(String name) {
this.name = name;
}
public Object eval(Map<String,Object> data) {
return data.get(name);
}
}
class LiteralNode implements Node {
private Object value;
FieldNode(Object value) {
this.value = value;
}
public Object eval(Map<String,Object> data) {
return value;
}
}
class OperationNode implements Node {
enum Type {
AND, OR, LESS, GREATER, EQUALS
}
private Type type;
private Node leftChild;
private Node rightChild;
OperationNode(Type type, Node leftChild, Node rightChild) {
this.type = type;
this.leftChild = leftChild;
this.rightChild = rightChild;
}
public Object eval(Map<String,Object> data) {
Object left = leftChild.eval(data);
Object right = rightChild.eval(data);
switch (type) {
case AND: return ((Boolean) left) && ((Boolean) right);
case OR: return ((Boolean) left) || ((Boolean) right);
case LESS: return ((Comparable) left).compareTo(right) < 0;
case EQUALS: return left.equals(right);
case GREATE: return ((Comparable) left).compareTo(right) > 0;
default:
throw new RuntimeException("Invalid op: " + type);
}
}
To directly answer the question, a number of SO questions (e.g. 1, 2) describe the basics of writing a parser by hand, though in practice it is very unusual to write a parser manually outside of university compiler courses due to the boilerplate and exacting detail involved.
As discussed in the comments, it sounds like the main reason to avoid grammar generators is to avoid a dependency on outside libraries. However, when using a grammar generator (parser generator) like JavaCC (Java Compiler-Compiler), there are no JAR files or outside dependencies involved: The JavaCC binary converts a grammar specification into Java code, that can be run without involving any further libraries.
See this IBM tutorial, JoAnn Brereton's "Use JavaCC to build a user friendly boolean query language" (via archive.org) as an example, which incidentally involves a grammar for a search language not unlike yours.
Example inputs:
actor = "Christopher Reeve" and keyword=action and keyword=adventure
(actor = "Christopher Reeve" and keyword=action) or keyword=romance
actor = "Christopher Reeve" and (keyword=action or keyword=romance)
Grammar excerpts:
TOKEN :
{
<STRING : (["A"-"Z", "0"-"9"])+ >
<QUOTED_STRING: "\"" (~["\""])+ "\"" >
}
void queryTerm() :
{
}
{
(<TITLE> | <ACTOR> |
<DIRECTOR> | <KEYWORD>)
( <EQUALS> | <NOTEQUAL>)
( <STRING> | <QUOTED_STRING> )
|
<LPAREN> expression() <RPAREN>
}
Output files:
UQLParser.java
UQLParserConstants.java
UQLParserTokenManager.java
TokenMgrError.java
ParseException.java
Token.java
SimpleCharStream.java
This is one of several parser generators that you can consider; others, like yacc and bison, also generate standalone Java files without requiring outside libraries. If necessary, you can check the generated Java files directly into your repository, leaving the .jj compiler source file only if you need to adjust the syntax. (Though it would probably be better to compile freshly from source as part of your build process and avoid checking generated files into source control, this may better suit your constraints of a Java-only solution.)
I have an instance of this Java class accessible in my Javascript program
public class ContentProvider {
public Object c(int n) {
switch (n) {
case 1: return 1.1;
case 2: return 2.2;
case 3: return 3.3;
case 4: return "4";
case 5: return new java.util.Date();
}
return null;
}
}
This is the code inside main():
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine engine = mgr.getEngineByName("JavaScript");
engine.put("ctx", new ContentProvider());
res = engine.eval("ctx.c(1)");
System.out.printf("rhino:> %s (%s)%n"
, res
, res != null ? res.getClass().getName() : null
);
The simple expression ctx.c(1) prints:
rhino:> 1.1 (java.lang.Double)
Now here is what happens with ctx.c(1) + ctx.c(2):
rhino:> 1.12.2 (java.lang.String)
And finally (ctx.c(1) + ctx.c(2)) * ctx.c(3):
rhino:> nan (java.lang.Double)
Rhino is performing string concatenation instead of number arithmetics! The following program works as expected instead:
engine.put("a", 1.1);
engine.put("b", 2.2);
engine.put("c", 3.3);
res = engine.eval("(a + b) * c");
Outputs:
rhino:> 10,89 (java.lang.Double)
This is a strange feature of Rhino: a Java Number set with engine.put("one", new Double(1)) works as expected, while the result of a Java method depends on the return type declared by the method itself, which is read with the reflection API:
if it's a primitive, like double, it's converted to a Javascript number
otherwise it's handled like other host objects and the + means concatenation, either Object like in your sample as well as Double
You can configure this behavior with wrapFactory.setJavaPrimitiveWrap(false) on the WrapFactory in the current Context. This way the Rhino code can be kept in the bootstrap lines of your program and doesn't clutter ContentProvider (which I guess is some sort of configuration proxy)
From the live Javadoc of WrapFactory.isJavaPrimitiveWrap()
By default the method returns true to indicate that instances of
String, Number, Boolean and Character should be wrapped as any other
Java object and scripts can access any Java method available in these
objects
So you can set this flag to false to indicate that Java Number's should be converted to Javascript numbers. It takes just two lines of code
Context ctx = Context.enter();
ctx.getWrapFactory().setJavaPrimitiveWrap(false);
Here is the Gist with the full code I used to test
I created a value wrapper:
public static class JSValue extends sun.org.mozilla.javascript.internal.ScriptableObject
{
Object value;
public JSValue(Object value) {
this.value = value;
}
public String getClassName() {
return value != null? value.getClass().getName(): null;
}
#Override
public Object getDefaultValue(Class typeHint) {
if (typeHint == null || Number.class.isAssignableFrom(typeHint)) {
if (value instanceof Number)
return ((Number) value).doubleValue();
}
return toString();
}
#Override
public String toString() {
return value != null? value.toString(): null;
}
}
and an edit function:
public static class ContentProvider {
public Object c(int n) {
... return new JSValue(1.1);
Now the expression works as expected. Thanks all.
I have code of the following form:
class Test {
private final A t;
public Test() {
for ( ... : ... ) {
final A u = null;
}
t = new A();
}
private class A {}
}
Compiler says:
variable t might already have been assigned
Interestingly, if I perform any of the following changes to the loop it works out!
Change the loop's content to A u = null
Remove the loop (but keep final A u = null;)
Replace the foreach-style loop with a classic counting loop
What is going on here?
Note: I could not get the minimal example to cause the error so there is probably something wrong with the "environment" (about 1400 loc). I can not see what could disturb the initialisation of t, though, as t is written to nowhere else.
Fun fact: IntelliJ IDEA says "Variable 'u' can have 'final' modifier..." if I remove it.
I use javac 1.6.0_26.
Update: There you go, this example so so minimal:
import java.util.List;
class A {
private final boolean a;
public A() {
for ( final Object o : new Object[] {} ) {
final Object sh = null;
}
a = true;
}
class B {
private final Object b1;
private final Object b2;
B() {
b1 = null;
b2 = null;
}
}
}
Fails to compile on javac 1.6.0_26 but compiles on javac 1.7.0_02. So I guess I hit some wicked corner case of ... something?
Note that you can do any of
Remove any one member
Remove final inside the loop in A()
Replace the loop with a normal for loop, e.g. for ( int i=0; i<100; i++ ) { ... }
and it will compile.
If you have lots of code I would try this.
private final A t;
public Test() {
final int t = 1;
for ( ... ) {
final A u = null;
}
this.t = new A();
This will cause any code which "might" initialise t to fail (and show up in the compiler.
If you're constructor happen to call another constructor that doesn't itself set t, the compiler fails to understand that.
See here.
As the problem is fixed in Java 7, it is probably a bug in the Java 6 compiler.
It is my understanding that storing an object in a final var does not make your object immutable but its reference. That explain why when you remove the final keyword it works and as per removing the for-loop, i think you are accessing the object reference and not an instance.