I have to make a calculator in Java using AntLR .But when i try to calculate the square root using the command s 4 it shows me :no viable alternative at input 's4'.
I really need your help for this. I try everything and I dont know whats is wrong.
This is my grammar:
grammar Hello;
r : r SEMI r EOF
| r SEMI
| plus_op
| minus_op
| sqrt_op;
// match keyword hello followed by an identifier
ID : [a-z]+ ; // match lower-case identifiers
NUM : [0-9];
WS : [ \t\r\n]+ -> skip ; // skip spaces, tabs, newlines
ADD : '+';
MINUS : '-';
SEMI: ';';
SQRT: 's';
plus_token: NUM | ID;
minus_token: NUM | ID;
sqrt_token: NUM;
plus_op : plus_token ADD plus_token;
minus_op : minus_token MINUS minus_token;
sqrt_op: SQRT sqrt_token;
SQRT will never be matched as ID matches the same input. This is because ANTLR won't try to match the input with multiple rules in the lexer but rather uses the first found rule that can match the current input.
The problem should be solved if SQRT is being defined before ID in the grammar.
I have a Lexer Rule as follows:
PREFIX : [abcd]'_';
EXTRA : ('xyz' | 'XYZ' );
SUFFIX : [ab];
TCHAN : PREFIX EXTRA? DIGIT+ SUFFIX?;
and a parser rule:
tpin : TCHAN
;
In the exit_tpin() Listiner method, is there a syntax where I can extract the DIGIT component of the token? Right now I can get the ctx.TCHAN() element, but this is a string. I just want the digit portion of TCHAN.
Or should I remove TCHAN as a TOKEN and move that rule to be tpin (i.e)
tpin : PREFIX EXTRA? DIGIT+ SUFFIX?
Where I know how to extract DIGIT from the listener.
My guess is that by the time the TOKEN is presented to the parser it is too late to deconstruct it... but I was wondering if some ANTLR guru's out there knew of a technique.
If I re-write my TOKENIZER, there is a possiblity that TCHAN tokens will be missed for INT/ID tokens (I think thats why I ended up parsing as I do).
I can always do some regexp work in the listener method... but that seemed like bad form ... as I had the individual components earlier. I'm just lazy, and was wondering if a techniqe other than refactoring the parsing grammar was possible.
In The Definitive ANTLR Reference you can find examples of complex lexers where much of the work is done. But when learning ANTLR, I would advise to consider the lexer mostly for its splitting function of the input stream into small tokens. Then do the big work in the parser. In the present case I would do :
grammar Question;
/* extract digit */
question
: tpin EOF
;
tpin
// : PREFIX EXTRA? DIGIT+ SUFFIX?
// {System.out.println("The only useful information is " + $DIGIT.text);}
: PREFIX EXTRA? number SUFFIX?
{System.out.println("The only useful information is " + $number.text);}
;
number
: DIGIT+
;
PREFIX : [abcd]'_';
EXTRA : ('xyz' | 'XYZ' );
DIGIT : [0-9] ;
SUFFIX : [ab];
WS : [ \t\r\n]+ -> skip ;
Say the input is d_xyz123456b. With the first version
: PREFIX EXTRA? DIGIT+ SUFFIX?
you get
$ grun Question question -tokens data.txt
[#0,0:1='d_',<PREFIX>,1:0]
[#1,2:4='xyz',<EXTRA>,1:2]
[#2,5:5='1',<DIGIT>,1:5]
[#3,6:6='2',<DIGIT>,1:6]
[#4,7:7='3',<DIGIT>,1:7]
[#5,8:8='4',<DIGIT>,1:8]
[#6,9:9='5',<DIGIT>,1:9]
[#7,10:10='6',<DIGIT>,1:10]
[#8,11:11='b',<SUFFIX>,1:11]
[#9,13:12='<EOF>',<EOF>,2:0]
The only useful information is 6
Because the parsing of DIGIT+ translates to a loop which reuses DIGIT
setState(12);
_errHandler.sync(this);
_la = _input.LA(1);
do {
{
{
setState(11);
((TpinContext)_localctx).DIGIT = match(DIGIT);
}
}
setState(14);
_errHandler.sync(this);
_la = _input.LA(1);
} while ( _la==DIGIT );
and $DIGIT.text translates to ((TpinContext)_localctx).DIGIT.getText(), only the last digit is retained. That's why I define a subrule number
: PREFIX EXTRA? number SUFFIX?
which makes it easy to capture the value :
[#0,0:1='d_',<PREFIX>,1:0]
[#1,2:4='xyz',<EXTRA>,1:2]
[#2,5:5='1',<DIGIT>,1:5]
[#3,6:6='2',<DIGIT>,1:6]
[#4,7:7='3',<DIGIT>,1:7]
[#5,8:8='4',<DIGIT>,1:8]
[#6,9:9='5',<DIGIT>,1:9]
[#7,10:10='6',<DIGIT>,1:10]
[#8,11:11='b',<SUFFIX>,1:11]
[#9,13:12='<EOF>',<EOF>,2:0]
The only useful information is 123456
You can even make it simpler :
tpin
: PREFIX EXTRA? INT SUFFIX?
{System.out.println("The only useful information is " + $INT.text);}
;
PREFIX : [abcd]'_';
EXTRA : ('xyz' | 'XYZ' );
INT : [0-9]+ ;
SUFFIX : [ab];
WS : [ \t\r\n]+ -> skip ;
$ grun Question question -tokens data.txt
[#0,0:1='d_',<PREFIX>,1:0]
[#1,2:4='xyz',<EXTRA>,1:2]
[#2,5:10='123456',<INT>,1:5]
[#3,11:11='b',<SUFFIX>,1:11]
[#4,13:12='<EOF>',<EOF>,2:0]
The only useful information is 123456
In the listener you have a direct access to these values through the rule context TpinContext :
public static class TpinContext extends ParserRuleContext {
public Token INT;
public TerminalNode PREFIX() { return getToken(QuestionParser.PREFIX, 0); }
public TerminalNode INT() { return getToken(QuestionParser.INT, 0); }
public TerminalNode EXTRA() { return getToken(QuestionParser.EXTRA, 0); }
public TerminalNode SUFFIX() { return getToken(QuestionParser.SUFFIX, 0); }
I'm trying to create a grammar which parses a file line by line.
grammar Comp;
options
{
language = Java;
}
#header {
package analyseur;
import java.util.*;
import component.*;
}
#parser::members {
/** Line to write in the new java file */
public String line;
}
start
: objectRule {System.out.println("OBJ"); line = $objectRule.text;}
| anyString {System.out.println("ANY"); line = $anyString.text;}
;
objectRule : ObjectKeyword ID ;
anyString : ANY_STRING ;
ObjectKeyword : 'Object' ;
ID : [a-zA-Z]+ ;
ANY_STRING : (~'\n')+ ;
WhiteSpace : (' '|'\t') -> skip;
When I send the lexem 'Object o' to the grammar, the output is ANY instead of OBJ.
'Object o' => 'ANY' // I would like OBJ
I know the ANY_STRING is longer but I wrote lexer tokens in the order. What is the problem ?
Thank you very much for your help ! ;)
For lexer rules, the rule with the longest match wins, independent of rule ordering. If the match length is the same, then the first listed rule wins.
To make rule order meaningful, reduce the possible match length of the ANY_STRING rule to be the same or less than any key word or id:
ANY_STRING: ~( ' ' | '\n' | '\t' ) ; // also?: '\r' | '\f' | '_'
Update
To see what the lexer is actually doing, dump the token stream.
I have a grammar in Antlr to parse the format of a file I save. I broke the grammar down to the part that is not working and I hope someone can clarify. Here is the grammar:
grammar OptFile;
parseFile returns [java.util.List<java.util.List<java.util.List<String>>> list] :
{ list = new java.util.ArrayList<List<List<String>>>(); }
vc=VARIABLESCAPTION v=variables oc=OBJECTIVECAPTION o=objective
{ list.add($v.list); list.add($o.list); }
;
variables returns [java.util.List<java.util.List<String>> list] :
{ list = new java.util.ArrayList<List<String>>(); }
(v=variable { list.add($v.list); } )*
;
variable returns [java.util.List<String> list] :
{ list = new java.util.ArrayList<String>(); }
n=characters ';' t=characters ';' lb=characters ';' ub=characters ';'
{ list = new java.util.ArrayList(); list.add($n.string); list.add($t.string); list.add($lb.string); list.add($ub.string); }
;
objective returns [java.util.List<String> list] :
{ list = new java.util.ArrayList<String>(); }
t=characters ';' { list.add($t.string); }
(
'PIECEWISE;' pw=piecewisefunction { list.add($pw.string); }
| 'REGULAR;' rf=characters ';' { list.add($rf.string); }
);
piecewisefunction returns [String string] :
( characters ';' characters ';' characters ';' characters ';' )*
{ string = getText(); }
;
characters returns [String string] :
( ~(';') )* { string = getText(); }
;
VARIABLESCAPTION : '--Variables:--' ;
OBJECTIVECAPTION : '--ObjectiveFunction:--' ;
A valid input shall look like one this:
--Variables--x;INTEGER;0;INFTY;y;CONTINUOUS;-12;13;--ObjectiveFunction--MAX;13x^27+SIN(y);
or like this
--Variables--x;INTEGER;12;20;--ObjectiveFunction--MAX;x;12;x;16;0,5x;16;x;20;
After '--Variables--' can be arbitrary many variables with four fields each, after '--ObjectiveFunction--' is one field and then either one more field or arbitrary many "packs" of four fields.
Apparently, when compiling with Antlr, I get the following error:
warning(200): OptFile.g:26:37:
Decision can match input such as "OBJECTIVECAPTION {OBJECTIVECAPTION..VARIABLESCAPTION, 'PIECEWISE;'..'REGULAR;'} ';' 'PIECEWISE;' {OBJECTIVECAPTION..VARIABLESCAPTION, 'PIECEWISE;'..'REGULAR;'} ';' {OBJECTIVECAPTION..VARIABLESCAPTION, 'PIECEWISE;'..'REGULAR;'} ';' {OBJECTIVECAPTION..VARIABLESCAPTION, 'PIECEWISE;'..'REGULAR;'} ';' OBJECTIVECAPTION ';' 'PIECEWISE;'" using multiple alternatives: 1,2
As a result, alternative(s) 2 were disabled for that input
My questions now are:
How can the input even start with OBJECTIVECAPTION, as far as I understand, the input for my grammar has to start with VARIABLESCAPTION.
What do I need to change, to get this grammar running?
The error message may be a bit cryptic, but the problem is in production variables, it defines zero-or-more occurrences of variable. A variable can begin with the input shown in the error message, but variables can also be followed by the same input, that occurs in its invocation environment. Thus there is a problem deciding between continuation in variables (alternative 1) and completing it (alternative 2).
So the error message does not refer to the complete input, but to an input fragment that is going to be matched by variables. The line number shown should point you to the production that presents the problem.
For fixing it, you could introduce a delimiter for the list, such that it becomes clear when to stop collecting more occurrences of variable, e.g.
parseFile : VARIABLESCAPTION variables '.' OBJECTIVECAPTION objective ;
EDIT by Asker:
I tried the approach and it works great, but only if the dot that is used as seperation symbol is added to the list of characters that have to be ignored, i.e. the code line for characters has to be modified:
characters : ( ~(';' | '.') )*;
After that, it works just fine.
I'm missing some basic knowledge. Started playing around with ATLR today missing any source telling me how to do the following:
I'd like to parse a configuration file a program of mine currently reads in a very ugly way. Basically it looks like:
A [Data] [Data]
B [Data] [Data] [Data]
where A/B/... are objects with their associated data following (dynamic amount, only simple digits).
A grammar should not be that hard but how to use ANTLR now?
lexer only: A/B are tokens and I ask for the tokens he read. How to ask this and how to detect malformatted input?
lexer & parser: A/B are parser rules and... how to know the parser processed successfully A/B? The same object could appear multiple times in the file and I need to consider every single one. It's more like listing instances in the config file.
Edit:
My problem is not the grammer but how to get informed by parser/lexer what they actually found/parsed? Best would be: invoke a function upon recognition of a rule like recursive descent
ANTLR production rules can have return value(s) you can use to get the contents of your configuration file.
Here's a quick demo:
grammar T;
parse returns [java.util.Map<String, List<Integer>> map]
#init{$map = new java.util.HashMap<String, List<Integer>>();}
: (line {$map.put($line.key, $line.values);} )+ EOF
;
line returns [String key, List<Integer> values]
: Id numbers (NL | EOF)
{
$key = $Id.text;
$values = $numbers.list;
}
;
numbers returns [List<Integer> list]
#init{$list = new ArrayList<Integer>();}
: (Num {$list.add(Integer.parseInt($Num.text));} )+
;
Num : '0'..'9'+;
Id : ('a'..'z' | 'A'..'Z')+;
NL : '\r'? '\n' | '\r';
Space : (' ' | '\t')+ {skip();};
If you runt the class below:
import org.antlr.runtime.*;
import java.util.*;
public class Main {
public static void main(String[] args) throws Exception {
String input = "A 12 34\n" +
"B 5 6 7 8\n" +
"C 9";
TLexer lexer = new TLexer(new ANTLRStringStream(input));
TParser parser = new TParser(new CommonTokenStream(lexer));
Map<String, List<Integer>> values = parser.parse();
System.out.println(values);
}
}
the following will be printed to the console:
{A=[12, 34], B=[5, 6, 7, 8], C=[9]}
The grammar should be something like this (it's pseudocode not ANTLR):
FILE ::= STATEMENT ('\n' STATEMENT)*
STATEMENT ::= NAME ITEM*
ITEM = '[' \d+ ']'
NAME = \w+
If you are looking for way to execute code when something is parsed, you should either use actions or AST (look them up in the documentation).