I have a filtering application written in Java that will allow me to filter companies based on their fundamentals (e.g. pretax profit, dividend yield etc.).
I have created a filtering engine that, at the moment is hard-coded to take filters that have been given at compile time.
What I want to do is open up a web service where I can pass in JSON to run the filtering using filters defined and passed in at runtime.
To do this I need to somehow convert strings into Java code.
So for example the following string in JSON:
current.preTaxProfit > previous.preTaxProfit
into Java code like this:
return current.getPreTaxProfit() > previous.getPreTaxProfit();
The main issue I am having is parsing mathematical strings such as:
current.preTaxProfit > (previous.preTaxProfit * 1.10)
Things could get very complex very quickly, especially when it comes to inner brackets and such, and adhering to BODMAS.
Are there any libraries out there specifically for parsing mathematical strings, or does anyone know any good resources that could help me?
For example, I have found this:
Javaassist: http://davidwinterfeldt.blogspot.co.uk/2009/02/genearting-bytecode.html
Does anyone have any experience of using Javaassist? In my architecture I pass in objects ith 'callback' methods implemented, so the ability to create entire classes and methods could useful.
Two possibilities:
the Java Scripting API:
the EL, Expression Language.
The scripting API might be most fast to get results from, but EL is nicer.
Consider using of an expression language, for instance JEXL.
If you put current and previous into the context, just evaluate:
current.preTaxProfit > (previous.preTaxProfit * 1.10)
Complete example:
// create context
JexlContext jc = new MapContext();
context.set("current", someObject);
context.set("previous", anotherObject);
// create expression
String jexlExp = "current.preTaxProfit > (previous.preTaxProfit * 1.10)";
Expression e = jexl.createExpression(jexlExp);
// evaluate
Object result = e.evaluate(jc);
Related
I wish to translate an OData query into a MongoDB query. I have the olingo-odata4 code and have isolated the calls necessary to take an input string e.g.
Name eq 'Buzz' and (amount gt 1.99 or size lt 10)
and run it through ExpressionParser with my own ExpressionVisitor implementation to intercept the various parse states like visitLiteral and visitBinaryOperator, etc. From there it is trivial to construct the equivalent MongoDB query. My main is essentially a trimmed down copy of the odata test drivers, including the mocking of the Edm class for startup:
ExpressionParser ep = new ExpressionParser(mock(Edm.class), odata);
My challenge is that I cannot seem to set up the environment properly for the parser to recognize fields, e.g. members. I would have hoped that Name eq 'Buzz' would cause the visitMember method to be called for Name but it does not; instead, I get a parse fail with the message: Property paths must follow a structured type.
So my overall question becomes: If the $filter argument to OData shows the expression syntax as Name eq 'Buzz' then what has to happen in the server implementation to recognize Name as a member?
The parser is unfortunately too closely coupled to the data model, IMHO. The test drivers I looked at were mostly {5 eq 5} and things like that so I was led astray. The property path test driver, however, cannot mock the EDM; it must construct a model in order to declare the type (string, etc.) of the fields. This is why visitMember Member class has the getType() method. But this means you cannot use the parser to just crack the syntax and dynamically perform actions on the results.
I ended up just creating a very small grammar in ANTLR to parse the OData query expressions and it was easy to convert to MongoDB query language.
The Flink doc gives a scala example for SVM algorithm:
val trainingDS: DataSet[LabeledVector] = env.readLibSVM(pathToTrainingFile)
val svm = SVM()
.setBlocks(10)
// Learn the SVM model
svm.fit(trainingDS)
How would the call to "fit" translate to java?
(The java expression "svm.fit(trainingDS)" gives a java compiler error:
"The method fit(DataSet, ParameterMap, FitOperation) in the type SVM is not applicable for the arguments (DataSet)"
)
At the moment, FlinkML can only be used with Scala. The reason is that FlinkML comes with a flexible pipeline mechanism which allows you to easily construct data analysis pipelines. These pipelines can consist of an arbitrary number of Estimators and a single trailing Predictor.
The implementation of this mechanism relies on type classes which encapsulate the program logic. The type classes are automatically retrieved and chained by declaring them as implicit values. Thus, the Scala compiler will make sure that the correct program logic is executed.
In theory, it is possible to construct these pipelines manually. However, it is a laborious process and, thus, I would recommend you to simply use Flink's Scala API if you want to interact with FlinkML.
This question already has answers here:
What are some valid uses for spEL (Spring Expression Language) ?
(2 answers)
Closed 7 years ago.
I am trying to learn Spring, so from here I got this example,
ExpressionParser parser = new SpelExpressionParser();
// evals to "Hello World"
String helloWorld = (String) parser.parseExpression("'Hello World'").getValue();
But I didn't understand it exact usage, 'cause I can better declare it as
String helloWorld = "Hello World";
So what is the exact use of above SpEL?
Where can I use it better way?
That particular SpEL is useless. It's useful when the expression that you pass to the parser comes from some external source, such as an annotation value or an XML attribute. A comprehensive explanation of ELs is way too broad for Stack Overflow, but I suggest looking at Thymeleaf for an example of how EL can be useful.
(Note also that it's much better to use getValue(String.class) instead of casting, both because it's safer and because you can take advantage of automatic type conversion.)
Spring Expression Language is used mainly in view layer like JSP, Thymeleaf. You can manipulate java object easily in view layer with SpEL. If you want to evaluate SpEL expression in java layer, you can use ExpressionParser to get the outcome.
Generally we want to keep view layer as much clean as possible(Don't want to evaluate complex logical operation). But sometimes little manipulation is must needed. In that case SpEL becomes handy.
I've got a bit of an interesting challenge
To the point:
I want to allow a user to enter an expression in a text field, and have that string treated as a python expression. There are a number of local variables I would like to make available to this expression.
I do have a solution though it will be cumbersome to implement. I was thinking of keeping a Python class source file, with a function that has a single %s in it. When the user enters his expression, we simply do a string format, and then call Jython's interpreter, to spit out something we can execute. There would have to be a number of variable declaration statements in front of that expression to make sure the variables we want to expose to the user for his expression.
So the user would be presented with a text field, he would enter
x1 + (3.5*x2) ** x3
and we would do our interpreting process to come up with an open delegate object. We then punch the values into this object from a map, and call execute, to get the result of the expression.
Any objections to using Jython, or should I be doing something other than modifying source code? I would like to think that some kind of mutable object akin to C#'s Expression object, where we could do something like
PythonExpression expr = new PythonExpression(userSuppliedText)
expr.setDefaultNamespace();
expr.loadLibraries("numPy", /*other libraries?*/);
//comes from somewhere else in the flow, but effectively we get
Map<String, Double> symbolValuesByName = new HashMap<>(){{
put("x1", 3.0);
put("x2", 20.0);
put("x3", 2.0);
}};
expr.loadSymbols(symbolValuesByName);
Runnable exprDelegate = expr.compile();
//sometime later
exprDelegate.run();
but, I'm hoping for a lot, and it looks like Jython is as good as it gets. Still, modifying source files and then passing them to an interpreter seems really heavy-handed.
Does that sound like a good approach? Do you guys have any other libraries you'd suggest?
Update: NumPy does not work with Jython
I should've discovered this one on my own.
So now my question shifts: Is there any way that from a single JVM process instance (meaning, without ever having to fork) I can compile and run some Python code?
If you simply want to parse the expressions, you ought to be able to put something together with a Java parser generator.
If you want to parse, error check and evaluate the expressions, then you will need a substantial subset of the functionality a full Python interpreter.
I'm not aware of a subset implementation.
If such a subset implementation exists, it is unclear that it would be any easier to embed / call than to use a full Python interpreter ... like Jython.
If the powers that be dictate that "thou shalt use python", then they need to pay for the extra work it is going to cause you ... and the next guy who is going to need to maintain a hybrid system across changes in requirements, and updates to the Java and Python / Jython ecosystems. Factor it into the project estimates.
The other approach would be to parse the full python expression grammar, but limit what your evalutor can handle ... based on what it actually required, and what is implementable in your project's time-frame. Limit the types supported and the operations on the types. Limit the built-in functions supported. Etcetera.
Assuming that you go down the Java calling Jython route, there is a lot of material on how to implement it here: http://www.jython.org/jythonbook/en/1.0/JythonAndJavaIntegration.html
I am trying to solve the following problem:
I have a Java app (not written by me) whose purpose is to take 2 report files (just comma separated table output), and compare each row and each column of the files to each other cell by cell - basically, for regression testing.
I would like to enhance it with the following functionality:
Let's say I made a software change that causes all the values in column C1 to increase by 100%.
When comparing specific column "C1", the software currently will report that 100% of values in C1 changed.
What I want to do is to be able to configure the regression tester to NOT simply compare "is C1 in row1 the same in both files", but instead apply a pre-configured comparison rule/formula to the field, e.g. "Make sure that C1 value in file#2 is 2 times bigger than C1 value in file #1". This way, not only will I suppress 100% of bogus regression errors that say every row's column C1 doesn't match, but ALSO catch any real errors where column C1 is not exactly 100% bigger using new software.
When coding this sort of functionality in Perl previously, the solution was very simple - simply code custom per-column comparator configuration into a Perl hash stored in the config file, with the hash keys being columns and hash values being Perl subroutines for comparing 2 values any complicated way I wanted.
Obviously, this approach will NOT work with Java since I can NOT write custom comparator logic in Java and have the Java differ evaluate/compile/execute those comparators during differ runtime.
This means that I need to come up with some domain specific interpretable language that my differ would interpret/evaluate/execute.
Since I'm not very familiar with Java ecosystem and libraries, I'm asking SO:
What would be a good solution for implementing this DSL for configurable comparator logic?
My requirements are:
The solution must be "free as in beer"
The solution must be "shrink wrapped". E.g. an existing library, that I can simply drop into my code, add the config file, and have it work.
Something which requires me to write my own BNF grammar and provides generic grammar parser for which I must write my own interpreter is NOT acceptable.
The solution must be reasonably flexible as far as data crunching and syntactically rich. E.g.:
you should be able to at the very least pass in - and reference/address from within the DSL - an entire row of data as a hash
it should have reasonably complete syntax; at the very least do basic string operations (concatenate, substring, ideally some level of regex matching and substitutions); basic arithmetic including ability to do abs(val1 - val2) > tolerance on floating point #s; and basic flow control/logic such as conditionals and ideally loops.
The solution should be reasonably fast, and must be scalable. E.g. comparing 100x100 size files should not take 10 minutes with 10-20 custom columns.
If it matters, the target environment is Java 1.6
There are multiple dynamic JVM programming languages that can be easily integrated in Java applications without much effort. I think it would be worth looking into Groovy and/or Scala.
Another possible option would be creating your own DSL using XText or XTend.
Whenever it comes to dynamic features in Java I come up with Janino, a charming runtime and in-memory compiler. In your case it would give you something similar to an eval(...) for plain Java, see: http://docs.codehaus.org/display/JANINO/Basic#Basic-expressionevaluator
The point here is that you don't have a DSL for your test configuration but you can use plain Java syntax to write custom expressions in your test configuration.
The only requirement that won't be fulfilled by the solution proposed below is that you can address a whole row from within the config file. The solution assumes that you write a Java test class that iterates through your test-data value-by-value (or better pair-by-pair) and uses your configured expressions to compare single values. So the dynamic part are the expressions, the static part is the iteration of your test-data.
However the code needed is very small and simple as shown below:
Config File (Java properties syntax, key is column name, value is test expression):
# custom expression for column C1
C1 = a == 2 * b
# custom expression for column C4
C4 = a == b ^ 2
# custom expression for column C47
C47 = Math.abs(a - b) < 1
Sketch for test code:
// read config file into Properties
Properties expressions = Properties.load(....);
// Compile expressions, this could also be done lazily
HashMap<String, ExpressionEvaluator> evals = new HashMap<String, ExpressionEvaluator>();
for (String column : expressions.stringPropertyNames()) {
String expression = expressions.getProperty(column);
ExpressionEvaluator eval = new ExpressionEvaluator(
expression, // expression
boolean.class, // expressionType
new String[] { "a", "b" }, // parameterNames
new Class[] { int.class, int.class } // parameterTypes, depends on your data
);
evals.put(column, eval);
}
// Now for every value pair (a, b) check if a custom expression is defined
// for the according column and if yes execute:
Boolean correct = (Boolean) theEvalForTheColumn.evaluate(
new Object[] { a, b } // parameterValues
);
if (!correct) throw Exception("Wrong values...");
As said on the Janino pages, performance for compiled expressions is pretty good (they are real java byte code), only the compilation will slow down the process. So it might be a problem if you have many custom expressions but it should scale well with an increasing number of values. hth.
No imbedded language is needed. Define your comparator as an interface.
You can load classes which define the interface at runtime using class.forName(name),
where name can be specified by command line arguments or any other convenient means.
your comparator class would look something like
class SpecialColumn3 implements ColumnCompare
{ boolean compare(String a,String b) {...}
}