I'm writing a Java program that requires its (technical) users to write scripts that it uses as input; it interprets these scripts into a series of actions and executes them. I am currently looking for the cleanest way to implement the script/configuration language. I was originally thinking of heading down the XML route, but the nature of the required input really is a procedural, linear flow of actions that need to be executed:
function move(Block b, Position p) {
// user-defined algorithm for moving block "b" to position "p"
}
Block a = getBlockA();
Position p = getPositionP();
move(a, p);
Etc. Please note: the above is an example only and does not constitute the exact syntax I am looking to achieve. I am still in the "30,000 ft view"-design phase, and don't know what my concreted scripting language will ultimately look like. I only provide this example to show that it is a flow/procedural script that the users must write, and that XML is probably not the best candidate for its implementation.
XML, perfect for hierarchial data, just doesn't feel like the best choice for such an implementation (although I could force it to work if need-be).
Not knowing a lick about DSLs, I've begun to read up on Groovy DSLs and they feel like a perfect match for what I need.
My uderstanding is that I could write, say, a Groovy (I'm stronger in Groovy than Scala, JRuby, etc.) DSL that would allow users to write scripts (.groovy files) that my program could then execute as input at runtime.
Is this correct, or am I misunderstanding the intent of DSLs altogether? If I am mistaken, does anybody have any suggestions for me? And if I am correct then how would a Java program read and execute a .groovy file (in other words, how would my program "consume" their script)?
Edit: I'm beginning to like ANTLR. Although I would love to roll up my sleeves and write a Groovy DSL, I don't want my users to be able to write any old Groovy program they want. I want my own "micro-language" and if users step outside of it I want the interpreter to invalidate the script. It's beginning to seem like Groovy/DSLs aren't the right choice, and maybe ANTLR could be the solution I need...?
I think you are on a really good path. Your users can write their files using your simple DSL and them you can run them by Evaling them at runtime. Your biggest challenge will be helping them to use the API of your DSL correctly. Unless they use an IDE this will be pretty tough.
Equivalent of eval() in Groovy
Yes, you can write a Groovy program that will accept a script as input and execute it. I recently wrote a BASIC DSL/interpreter in this way using groovy :
http://cartesianproduct.wordpress.com/binsic-is-not-sinclair-instruction-code/
(In the end it was more interpreter than DSL but that was to do with a peculiarity of Groovy that likely won't affect you - BASIC insists on UPPER CASE keywords which Groovy finds hard to parse - hence they have to be converted to lower case).
Groovy allows you to extend the script environment in various ways (eg injecting variables into the binding and transferring execution from the current script to a different, dynamically loaded script) which make this relatively simple.
Related
I have a use-case where I want to enable users to write simple logic, and behind the scenes, convert this logic into a condition in the code.
For example, the user might write:
someFieldName > 10 AND otherFieldName is NULL
And I'd like that to generate the following code:
if (data["someFieldName"] > 10 && data["otherFieldName"] == null) {
// Do something
}
After doing some research, I saw that one of the options is using eval (by leveraging a JS engine), although it doesn't fit all use cases.
I also saw that it's possible to use tools like ANTLR, which seems a bit like overkill.
Are there any simple off-the-shelf products we can use for such purposes? Or would creating a simple parser ourselves be the best way to handle it?
Your use case can be adequately addressed by MVEL2.
There is no need for you to write a parser and AST with ANTLR or convert to Java code, just evaluate the expression with appropriate parameters.
In fact any Java expression language library would do. You could also look at JUEL.
However, looking at the expression, I would say it aligns more towards MVEL2.
Give both the libraries a try.
Scenario
I'm working with a Java model built from scratch in Eclipse. What's important in this model is that we save our output to MATLAB (.mat) files. I constantly add new features, which require new fields that in turn will have to be exported to the .mat file at every iteration. Upon restarting a crashed simulation, I might have to import the .mat file. To export or import my .mat file I use JMatIO.
For example, if I would add a new field rho_m (a simple double) to my class CModel, I have to add to my Save() method:
mlModel.setField("rho_m", new MLDouble(null, new double[] {rho_m}, 1));
And to my Load() method:
rho_m = ((MLDouble)mlModel.getField("rho_m")).getReal(0);
Note that even though rho_m is a double, it needs to be treated as a double[] in JMatIO. This probably has something to do with MATLAB being orientated towards matrices and matrix operations.
Problem
Instead of doing this manually (prone to errors, annoying to maintain) I would like to automate this procedure. Ideally, I would like my IDE to detect all the fields in CModel and write the code based on the field's name and type. Is there any way to do this in Java/Eclipse?
Ideas so far
I have no formal training in low-level programming languages (yes, Java is low-level to me) and am still relatively new to Java. I do have some experience with MATLAB. In MATLAB I think I could use eval() and fieldnames() in a for loop to do what I mentioned. My last resort is to copy-paste the Java code to MATLAB and from there generate the code using a huge, ugly script. Every time I want to make changes to the model I'd rerun the MATLAB script.
Besides that idea I've found terms like UML, but do not have the background knowledge to figure out if this is what I'm looking for or not.
Any help, even if it's just a small push in the right direction, is greatly appreciated. Let me know if I need to further clarify anything.
Looking at your scenario, you are doing model-driven code generation, that is, you have a model and want to get some code generated according to your current model. Therefore, you need a model-driven code generator.
I lead the ABSE/AtomWeaver project, so I'll outline what you can do to get what you want using AtomWeaver (There are however other solutions like MetaEdit+, XText or Eclipse's own GMT/EMF sub-system).
AtomWeaver is an IDE where you can build a model and generate code from that model. You can change your model as many times you want and hit the "Generate" button to get an updated version of your code. ABSE is the name of the modeling method.
We don't need to go into details, but essentially ABSE follows a "building-block" approach. You create a Template that represents a feature or concept of your model. Then, you can associate a mini-code generator just to that concept. You can then "instantiate" and combine those building blocks to quickly build your models. Variables increase the flexibility of your models.
You can also change your models, or add new features ("blocks") and generate again. The generators are built using the Lua programming language, a very simple language with C-Like syntax.
The best way to understand the ABSE development method and the AtomWeaver IDE is to download the IDE and see the samples or try the tutorials. And yes, you can use AtomWeaver for free.
I'm working on a Java based OSS app SqlHawk which as one of its features is to run upgrade sql scripts against a server.
Microsoft have made it a convention to split a script into batches with the GO statement, which is a good idea but just asking for false matches on the string.
At the moment I have a very rudimentary:
// split where GO on its own on a line
Pattern batchSplitter = Pattern.compile("^GO", Pattern.MULTILINE);
...
String[] splitSql = batchSplitter.split(definition);
...
which kind of works but is prone to being tripped up by things like quoted GO statements or indentation issues.
I think the only way to make this truly reliable is to have an SQL parser in the app, but I have no idea how to go about this, or whether that might actually end up being less reliable (especially given this tool supports multiple DBMSs).
What ways could I solve this problem? Code examples would be very helpful to me here.
Relevant sqlHawk code on github.
Currently using jtds to execute the batches found in the scripts.
GO is a client batch seperator command. You can replace it with ;. It should not be sent in your EXEC dynamic SQL.
USE master
GO --<----- client actually send the first batch to SQL and wait for a response
SELECT * from sys.databases
GO
Should be translated in
Application.Exec("USE master");
Application.Exec("SELECT * from sys.databases");
or you can write it this way:
Application.Exec("'USE master;SELECT * from sys.databases")
More about GO
http://msdn.microsoft.com/en-us/library/ms188037(v=sql.90).aspx
Ok, so this isn't going to be exactly what you want, but you might find it a start. I released SchemaEngine (which forms the core of most of my products) as open source here. In there, you will find C# code that does what you want very reliably (i.e. not tripping up with strings, comments etc etc). It also support the 'GO x' syntax to repeat a batch x times.
If you download that and have a look in /Atlantis.SchemaEngine/Helpers you'll find a class called BatchParser.cs which contains a method called ParseBatches - which does pretty much what it says on the tin.
I'm working in a java project where a big part of the code was written with a formatting style that I don't like (and is also non standard), namely all method parameters are in uppercase (and also all local variables).
On IntellJ I am able to use "Analyze -> Inspect Code" and actually find all occurrences of uppercase method parameters (over 1000).
To fix one occurrence I can do "refactor > rename parameter" and it works fine (let's assume there is no overlapping).
Is there a way to automagically doing this refactor (e.g: rename method parameter starting with uppercase to same name starting with lowercase)?
Use a Source Parser
I think what you need to do is use a source code parser like javaparser to do this.
For every java source file, parse it to a CompilationUnit, create a Visitor, probably using ModifierVisitorAdapter as base class, and override (at least) visit(MethodDeclaration, arg). Then write the changed CompilationUnit to a new File and do a diff afterwards.
I would advise against changing the original source file, but creating a shadow file tree may me a good idea (e.g. old file: src/main/java/com/mycompany/MyClass.java, new file src/main/refactored/com/mycompany/MyClass.java, that way you can diff the entire directories).
I'd advise that you think about a few things before you do anything:
If this is a team effort, inform your team.
If this is for an employer, inform your boss.
If this is checked into a version control system, realize that you'll have diffs coming out the wazoo.
If it's not checked into a version control system, check it in.
Take a backup before you make any changes.
See if you have some tests to check before & after behavior hasn't changed.
This is a dangerous refactoring. Be careful.
I am not aware of any direct support for such refactoring out of the box in IDEs. As most IDEs would support name refactoring (which is regularly used). You may need to write some IDE plugin that could browse through source code (AST) and invoke rename refactoring behind the scene for such parameter names matching such format.
I have done a lot of such refactorings on a rather large scale of files, using TextPad or WildPad, and a bunch of reg-ex replace-all. Always worked for me!
I'm confident that if the code is first formatted using an IDE like Eclipse (if it is not properly formatted), and then a reg-ex involving the methods' signature (scope, return-type, name, bracket, arg list, bracket) can be devised, your job will be done in seconds with these tools. You might need more than one replace-all sets of reg-ex.
The only time-taking activity would be to come up with such a set of reg-ex.
Hope this helps!
Ok, so coming from a background of mostly perl, and mostly writing dirty little apps to automate my tasks, I've read the pages about the evils of eval(), and I always use a hash (in perl). I'm currently working on a little project (mostly for me and a couple of other technical people at work), for creating "canned response" e-mails. To allow for additions, subtractions, edits, etc., I'd like to essentially describe the response form(s) in XML, and have my app parse the XML and create the response forms at runtime. I want to use Java (to integrate it into an existing Java tool that I created), and boiled down to a trivial example, what I'm trying to do is take some XML like:
<Form Name="first" Title="Title!">
<Label Name="before">Your Request:</Label>
<Textbox Name="input"/>
<Label Name="after">has been completed.</Label>
<Output>%before%%input%%after%</Output>
</Form>
<Form Name="second">
...
and from parsing that, I want to create a JFrame named first, which contains a JLabel named before with the obvious text, then a textbox, then another JLabel... you get the idea (I eventually want to use the output tag to control exactly how the response is formatted).
I can parse the XML, and get the element name and such, but I don't know how to instantiate the Objects with a name that is the value of a variable, effectively:
JFrame $(thisNode.getAttributes().getNamedItem("Name").getNodeValue()) = new JFrame(thisNode.getAttributes().getNamedItem("Title").getNodeValue());
I've read basically the whole first page of google results on java reflection, but I haven't come across anyone doing quite what I'm looking for (at least not that I could tell). Having basically zero experience with reflection, I'm curious if this is something that can be accomplished using it, or if I should take the same approach as I would in Perl, and create a HashMap or HashTable of Objects, and tie them to a entry in a Hash of JFrames. Or, I'm open to ideas that don't fall into those two categories. The Hash is sort of my stand-by answer, because I've done it in Perl plenty of times, and I'm sure I can make it work in Java, but if there's a feature (like reflection) that's made to do this task, then why not do it the way it was intended to be done?
What you're asking isn't possible in Java. It doesn't work that way and these sorts of tricks, which are common in dynamic languages, aren't the Java way. You can certainly do:
JFrame frame = JFrameBuilder.buildFromTemplate("frame.xml");
where you create a JFrameBuilder class that reads the XML and creates an object from it but the variable name can't be dynamic. You have to remember that there are two steps in Java.
Java source files are compiled into bytecode;
The bytecode is read by a Java interpreter (JVM) and executed.
What you want is essentially asking to execute code in step (1). Now annotations can do things in a compile step (like adding interfaces, implementing methods and so on) but local variable naming is not one of those things.
You could (not necessarily that you should) generate Java source based on your XML, compile the generated code, and finally, execute the compiled code. This could be more efficient if you saved the generated .class files and reused them instead of parsing the XML every time the program is run (it can check the timestamp on the XML and only generate and compile if it's been modified since the last code generation).