My current project (name it IoTSuite) takes high-level specifications, parses them, and generates code in Java and Android. In this project, I have written ANTLR grammar to parse the high-level specification and I have used StringTemplate for the code generator.
However, due to nice editor support and syntax coloring features, I have used the xtext grammar(same as the ANTLR grammar, but it has been written in xText). Now, I perform the following three steps:
Step 1: I have written xtext grammar as show in the following image.
Step 2: Now, I generate the supporting artifacts from MyDsl.xtext grammar and then run the whole project as Eclipse Application, as shown in the following image.
Step 3: After this, Eclipse application runs that allows me to write high-level specifications, as shown in the following image.
Now, my requirement is - I would like to parse the high-level specification, as shown in the step3, using the ANTLR grammar written in IoTSuite project.
In brief, my requirement is to connect the high-level specification written in xtext with the ANTLR parser. How could I do it? Please let me know in case you need more clarity for the solution.
Xtext generates its own ANTLR grammar. This grammar is also instrumented with code to instantiate the AST for the grammar. The produced parser cannot be exchanged.
What you likely want to do is to invoke your own ANTLR parser when saving a resource. The IGenerator hook could be used to invoke your parser and generator.
Please consider also to use Xtend for code generation, that's much nicer than StringTemplate and integrates smoothly with Xtext infrastructure. Xtext's Statemachine Example shipped with the SDK shows how to do this (and to use the IGenerator interface).
Related
I was wondering, whether it is possible to take the antlr grammar (*.g) or the generated parsers (from this grammar) and use it in a separate project?
For this I was looking into the SysMLv2 (eclipse-based) project on github, where xtext was used in order to define the grammar of this new modelling language. The grammar and the generated parsers can be found here.
My first idea was just to take the grammar file (InternalAlf.g) and use antlr (i tried 3.5.0 and 3.5.2) in order to generate the parser + lexer. Doing this i end up with a bunch of error message that symbols were not found (the symbol in question: EObject).
Then since it is obviously an eclipse project i figured another naive solution would be to package the whole project as a jar and include it as library in mine. I tried to use eclipse for that (export -> excecutable jar). That option requires a MainClass, where i am not sure which one to take and which also lets me doubt this approach. Using the other export jar option, does not allow to add the necessary dependencies to my jar.
Anyone other proposals? Since the antlr grammar file is available, it should be (actually) quite easy to generate the parser, but i am not sure how to do this, since this grammar file has a bunch of dependecies. Or if I rephrase this question: how do i deal with this type of antlr grammar files (that have dependecies to java libraries). In typical antlr tutorials, I (as a newb in antlr and xtext) could not find the answer.
best regards
I looked at the grammar in that project. IT is HIGHLY specific to Xtext. (To the point that it’s a bit difficult to find the ANTLR grammar amongst all of the actions).
You might be able to use the ANTLR3 grammar to parse it and discard all of the actions, etc. that make it so tightly coupled to Xtext (being careful about any semantic predicates and dependencies they might have on those actions). Emphasis on the MIGHT here.
In short, it’s not going to be at all simple to generate a parser divorced from Xtext using this grammar.
If you were to elaborate on what you need to accomplish by not just using the Xtext SysMLv2, and feel a need to create a separate parser someone might be able to point you in an appropriate direction.
I have created an Ecore metamodel and generated an editor through EMF. To generate my output code I parse the resource set (XMI) when a diagram is created and then map these resources to my own Java classes. These Java classes are responsible for generating my output which is some javascript files based on few template files (.stg).
Everything works fine but my question is where do technologies such as Xtend/Acceleo come in? Do I need any of them? I know that they are template based but since I have my own templates that are rendered through my Java classes why would I need to use Xtend for example? I am totally confused.
Thank you so much
Xtend is a general purpose language. You could call it a "better Java". This is especially true for templating, because Xtend has nice template expressions. So if you want your generator code to be more readable, you could replace some Java classes with Xtend classes. You don't need to rewrite everything, Xtend interoperates seamlessly with Java. Just rewrite the parts that benefit most from Xtend's rich language features.
You are using the eclipse Enterprise Modelling Framework (EMF). This provides you with the ECore meta modell and can generate basic dialog based editors. Basically EMF brings your diagrams from the XMI file format to a standardized object model, similar to a DOM.
Probably you also use the Graphical Editor Framework (GEF) to visualize your diagrams. It takes your DOM and renders it to a graphical representation. Usually you can also modify the graphics using graphical user interface, which will then also change the contents of your DOM.
Parallel to the GEF there also exists a textual editor framework called xText. It brings your DOM to a textual representation. You can then use a text editor to change the textual representation. xText can parse this changed text and change the DOM accordingly. Additionally it generates a nice Eclipse editor with syntax high lightning, code folding, code completion and possibilities for extensions for custom code completion and refactoring.
xText can be augmented by XBase. XBase is a basic programming language that runs on the Java VM. You can then provide a mapping from your ECore artefacts to XBase concepts (e.g. which ECore object represents an if statement or a loop). XBase can then generate Java bytecode and hence convert your ECore model to runnable code.
XTend is a general purpose language. It is often used as a very advanced template engine to to convert your ECore model to text. In contrast to xText there is no way back to convert the generated text back to the ECore DOM. Of course you can use any template engine or even xText/xbase to generate your runnable code according to your needs.
do you know of any tool which creates an AST from a Java program or class and creates an XML representation (Collection or single XML document) from the AST?
kind regards,
Johannes
Not any tools directly, but http://www.antlr.org/ is the defacto tool for building ASTs from any general language. And there exists several grammar files for Java that you can repurpose for your own programs. So grab ANTLR, use the latest Java grammer, and write out the XML representation you want.
Our DMS Software Reengineering Toolkit with its Java Front End can do this directly. You ask DMS to parse the file, and produce an XML dump using a command line switch ++XML.
See What would an AST (abstract syntax tree) for an object-oriented programming language look like?.
As a general rule, we don't recommend this, for several reasons:
XML output for real files is really enormous, and takes a lot of time to write and read
Most people do this because they believe with an XML representation that just a little bit of XSLT will get them what they want
If you intend to modify the code, once you have the XML you pretty much can't regenerate it.
The machinery that DMS provides (attribute grammars, symbol tables, flow analyses, pattern matching and source-to-source transformations, source regeneration from the AST, is what you really want, and you get access to it by using DMS after the parsing step without exporting the XML ever
Does anyone knows a tool for Java (something like codedom for C#) that provides a way to generate Java code to a .java file?
EDIT:
I'm building a platform the main objective of which is to automate an operation. Giving some input, I want to generate code for an external tool. So it isn't generation on runtime. I want to generate and output that to an actual file.
JET maybe outdated (I didn't use it) JET Tutorial Part 1
More Plugins for Eclipse Plugins in Code Generation
EDIT:
Sorry I don't know codedom and what features this tool implies.
Standalone is Freemarker
and Velocity see also this example
I have had some success using ASM to both modify existing classes at the bytecode level or to generate completely new classes on the fly. The tutorial walks you through this in a very understandable fashion.
ASM like most such tools generates bytecode not source. The reason for this is if you want to dynamically generate and execute new code from with a program, historically it was not straight forward to invoke the Java compiler. Therefore it was generally easier to generate and use bytecode than source.
If you need to generate and run the code immediately within your program I recommend you use bytecode manipulation tool. If all you need is Java source, I would roll my own code generator that takes my input format and generates the code. You may want to look for a framework to help you with this but since a source file is just text, usually it is just as easy to create this yourself especially if you have a custom input format.
ABSE and AtomWeaver form a code generation and model-driven-development framework where you can easily implement what you want. ABSE is a new methodology where you compose your code generator from smaller bits (called Atoms) and AtomWaver is an straightforward IDE that lets you implement, manipulate and use your generator models.
It also allows non-programmers to build programs/configurations/whatever, made from already-built parts (Atoms you have previously prepared).
This project is just being publicly launched now, and an alpha version is being made available now. ABSE is open, and AtomWeaver is free for personal and commercial use.
Get more info here : http://www.abse.info (Disclaimer: I am the project lead)
What you could try is to use an existing grammar (e.g. from ANTLR) and build the AST. Then from the AST generate the code. That should be much more robust than simple templating. For something in the middle I suggest the (eye-opening) talk from Terence Parr about StringTemplate. (Sorry, don't have the link for the talk at hand)
I am not sure what you really need, but take a look at javassist. Is it the thing you are looking for?
I am working on a small text editor project and want to add basic syntax highlighting for a couple of languages (Java, XML..just to name a few). As a learning experience I wanted to add one of the popular or non popular Java lexer parser.
What project do you recommend. Antlr is probably the most well known, but it seems pretty complex and heavy.
Here are the option that I know of.
Antlr
Ragel (yes, it can generate Java source for processing input)
Do it yourself (I guess I could write a simple token parser and highlight the source code).
ANTLR or JavaCC would be the two I know. I'd recommend ANTLR first.
ANTLR may seem complex and heavy but you don't need to use all of the functionality that it includes; it's nicely layered. I'm a big fan of using it to develop parsers. For starters, you can use the excellent ANTLRWorks to visualize and test the grammars that you are creating. It's really nice to be able to watch it capture tokens, build parse trees and step through the process.
For your text editor project, I would check out filter grammars, which might suit your needs nicely. For filter grammars you don't need to specify the entire lexical structure of your language, only the parts that you care about (i.e. need to highlight, color or index) and you can always add in more until you can handle a whole language.
Google code has new project acacia-lex. Written by myself, it seems simple (so far) java lexer using javax annotations.
SableCC
Another interesting option (which I didn't try yet) would be Xtext, which uses Antlr but also includes tools for creating Eclipse editors for your language.
ANTLR is the way to go. I would not build it by hand. You'll also find if you look around on the ANTLR web site that grammars are available for Java, XML, etc.
Another option would be Xtext. It will not only generate a parser for your grammar, but also a complete editor with syntax coloring, error markers, content assist and outline view.
I've done it with JFlex before and was quite satisfied with it. But the language I was highlighting was simple enough that I didn't need a parser generator, so your mileage may vary.
JLex and CUP are decent lexer and parser generators, respectively. I'm currently using both to develop a simple scripting language for a project I'm working on.
I don't think that you need a lexer. all you need is first read the file extention to detect the language and then from a xml file which listed the language keywords easily find them and highlight them.