Translate Flink scala into java - java

The Flink doc gives a scala example for SVM algorithm:
val trainingDS: DataSet[LabeledVector] = env.readLibSVM(pathToTrainingFile)
val svm = SVM()
.setBlocks(10)
// Learn the SVM model
svm.fit(trainingDS)
How would the call to "fit" translate to java?
(The java expression "svm.fit(trainingDS)" gives a java compiler error:
"The method fit(DataSet, ParameterMap, FitOperation) in the type SVM is not applicable for the arguments (DataSet)"
)

At the moment, FlinkML can only be used with Scala. The reason is that FlinkML comes with a flexible pipeline mechanism which allows you to easily construct data analysis pipelines. These pipelines can consist of an arbitrary number of Estimators and a single trailing Predictor.
The implementation of this mechanism relies on type classes which encapsulate the program logic. The type classes are automatically retrieved and chained by declaring them as implicit values. Thus, the Scala compiler will make sure that the correct program logic is executed.
In theory, it is possible to construct these pipelines manually. However, it is a laborious process and, thus, I would recommend you to simply use Flink's Scala API if you want to interact with FlinkML.

Related

Validate parameters passed into Constructors during compile time

In a huge project with tens of thousands of Java files there are a couple of Java classes where developers may pass in strings as parameters to a constructor class I had implemented
public byte[] getProductReport(List<String> products, Date from, Date to) {
// ... do some stuff before...
List<ReportParameterDto> reportParameters = new ArrayList<>();
reportParameters.add(new ReportParameterDto("From (YYYY.MM.DD)", ParameterType.DATE, from));
reportParameters.add(new ReportParameterDto("To_(YYYY.MM.DD)", ParameterType.DATE, to));
reportParameters.add(new ReportParameterDto("Products", ParameterType.SELECT, someList));
return ReportFromCRServerHelper.downloadReport("ProductReporot", reportParameters, ReportFormat.PDF);
}
If a developer uses wrong string values downloading a requested report (from a remote Report server) will fail during runtime.
In this example I would like to have some validation checking - during compilation - in order to avoid these errors before they are found by the customer.
I have API methods to obtain parameter values from a report which I hope to use
during compilation of the above method.
In my example the compilation should fail and throw an error highlighting how parameters should look instead:
"From (JJJJ-MM)" is invalid --> should be "From_(JJJJ-MM)"
"Products" is invalid --> should be "PRODUCT_LIST"
Can I detect these parameters (used in above ReportParameterDto constructors) through JAVAX annotation processing?
The few tutorials / blogs that I found dealt with validating parameters in method signatures, not the values passed into methods.
Or are there a more elegant tools available?
A compile-time tool like the Checker Framework can validate the arguments to a method or constructor. You annotate the parameter types to indicate the permitted values, and then when javac runs it issues a warning if an argument may not be compatible with the parameter.
If there is a limited number of possible values, then you can use the Fake Enum Checker to treat strings or integers as enumerated values. (Using a Java enum is also a good idea, but may not be possible or convenient because of other code that expects a string or integer.)
If the number of possible values is unlimited, then you can use the Constant Value Checker -- for example, to supply a regular expression that any constant string argument must satisfy.
You can also define your own compile-time validation if you want different functionality than is available in the checkers that are distributed with the Checker Framework.

How to pass functions as arguments from Dart/Flutter to Android native code?

I'd like to create a Java-based SDK that can be invoked from Flutter, the Flutter side needs to plug into the SDK by providing callbacks to be executed at different stages of the processing, ideally something like:
Future<Result> = doPayment(
amount: 100.00,
currency: 'USD',
onPasscodeEntry: () => _renderInputBox(),
onValidation: () => _doValidation()
);
the processing itself takes some time etc so a Future<PaymentResult> pattern makes sense, the application should react to some points of the processing, so it makes sense to pass in callbacks.
Problems:
how to pass functions to Java code? As far as I can tell all the Java-passable method arguments in Flutter are serializable-ish objects
(less critical/optional) how to return a POJO and handle it in Dart? Judging from the specs I should encode (within Java) the result in a ArrayList/HashMap or as a JSON string and then create the equivalent constructor in Dart that takes in List/Map ? This seems a bit unwieldy, so I'm asking if there's a simpler way, preferably without maintaining a Java & Dart variants of the same object
You can not pass a function.
You can create a map from a function name to a function reference and then pass the function name and look up the function in by name in the map and call it.
You also can't pass a POJO.
You need to serialize it to JSON (or some other supported format) and deserialize it on the other side of the channel. For that you need to have to POJO class implemented in Dart and in Java.
Packages like protobuf or flatbuffer allow to geneerate code for different languages based on an reduced language to specify the data structure.

Parsing a String to Java Code at Runtime

I have a filtering application written in Java that will allow me to filter companies based on their fundamentals (e.g. pretax profit, dividend yield etc.).
I have created a filtering engine that, at the moment is hard-coded to take filters that have been given at compile time.
What I want to do is open up a web service where I can pass in JSON to run the filtering using filters defined and passed in at runtime.
To do this I need to somehow convert strings into Java code.
So for example the following string in JSON:
current.preTaxProfit > previous.preTaxProfit
into Java code like this:
return current.getPreTaxProfit() > previous.getPreTaxProfit();
The main issue I am having is parsing mathematical strings such as:
current.preTaxProfit > (previous.preTaxProfit * 1.10)
Things could get very complex very quickly, especially when it comes to inner brackets and such, and adhering to BODMAS.
Are there any libraries out there specifically for parsing mathematical strings, or does anyone know any good resources that could help me?
For example, I have found this:
Javaassist: http://davidwinterfeldt.blogspot.co.uk/2009/02/genearting-bytecode.html
Does anyone have any experience of using Javaassist? In my architecture I pass in objects ith 'callback' methods implemented, so the ability to create entire classes and methods could useful.
Two possibilities:
the Java Scripting API:
the EL, Expression Language.
The scripting API might be most fast to get results from, but EL is nicer.
Consider using of an expression language, for instance JEXL.
If you put current and previous into the context, just evaluate:
current.preTaxProfit > (previous.preTaxProfit * 1.10)
Complete example:
// create context
JexlContext jc = new MapContext();
context.set("current", someObject);
context.set("previous", anotherObject);
// create expression
String jexlExp = "current.preTaxProfit > (previous.preTaxProfit * 1.10)";
Expression e = jexl.createExpression(jexlExp);
// evaluate
Object result = e.evaluate(jc);

parsing a Python expression from Java

I've got a bit of an interesting challenge
To the point:
I want to allow a user to enter an expression in a text field, and have that string treated as a python expression. There are a number of local variables I would like to make available to this expression.
I do have a solution though it will be cumbersome to implement. I was thinking of keeping a Python class source file, with a function that has a single %s in it. When the user enters his expression, we simply do a string format, and then call Jython's interpreter, to spit out something we can execute. There would have to be a number of variable declaration statements in front of that expression to make sure the variables we want to expose to the user for his expression.
So the user would be presented with a text field, he would enter
x1 + (3.5*x2) ** x3
and we would do our interpreting process to come up with an open delegate object. We then punch the values into this object from a map, and call execute, to get the result of the expression.
Any objections to using Jython, or should I be doing something other than modifying source code? I would like to think that some kind of mutable object akin to C#'s Expression object, where we could do something like
PythonExpression expr = new PythonExpression(userSuppliedText)
expr.setDefaultNamespace();
expr.loadLibraries("numPy", /*other libraries?*/);
//comes from somewhere else in the flow, but effectively we get
Map<String, Double> symbolValuesByName = new HashMap<>(){{
put("x1", 3.0);
put("x2", 20.0);
put("x3", 2.0);
}};
expr.loadSymbols(symbolValuesByName);
Runnable exprDelegate = expr.compile();
//sometime later
exprDelegate.run();
but, I'm hoping for a lot, and it looks like Jython is as good as it gets. Still, modifying source files and then passing them to an interpreter seems really heavy-handed.
Does that sound like a good approach? Do you guys have any other libraries you'd suggest?
Update: NumPy does not work with Jython
I should've discovered this one on my own.
So now my question shifts: Is there any way that from a single JVM process instance (meaning, without ever having to fork) I can compile and run some Python code?
If you simply want to parse the expressions, you ought to be able to put something together with a Java parser generator.
If you want to parse, error check and evaluate the expressions, then you will need a substantial subset of the functionality a full Python interpreter.
I'm not aware of a subset implementation.
If such a subset implementation exists, it is unclear that it would be any easier to embed / call than to use a full Python interpreter ... like Jython.
If the powers that be dictate that "thou shalt use python", then they need to pay for the extra work it is going to cause you ... and the next guy who is going to need to maintain a hybrid system across changes in requirements, and updates to the Java and Python / Jython ecosystems. Factor it into the project estimates.
The other approach would be to parse the full python expression grammar, but limit what your evalutor can handle ... based on what it actually required, and what is implementable in your project's time-frame. Limit the types supported and the operations on the types. Limit the built-in functions supported. Etcetera.
Assuming that you go down the Java calling Jython route, there is a lot of material on how to implement it here: http://www.jython.org/jythonbook/en/1.0/JythonAndJavaIntegration.html

Best sandboxed expression language for JVM

I want an expression language that runs on the JVM and includes support for
math expressions, including operator priority
string expressions, like substring, etc
supports named functions
this allows me to decorate and control exactly who and what functions can be executed.
read/write variables that are "typeless" / allow type conversion in a controlled manner.
does not allow arbitary java scriptlets.
it should not be possible to include constructs like new Someclass()
cannot execute arbitrary static or otherwise method
does not allow any OGNL like expressions.
I only want to functions I map to be available.
support for control constructs like if this then that is for the moment optional.
must be embeddable.
This previous stackoverflow question is similar, but:
does not really answer "how" or "what" as does the above,
allows java object expressions, throwing an exception from a SecurityManager to stop method execution, which is nasty and wrong.
java object like expressions should be an error at parse time.
jexel seem to be closest possible match, but License is a bit horrible (GPL/Commercial).
If you only want the scripts to output text, then Apache Velocity fit's your constraints quite well. It runs in an environment where it only has access to the objects you give it, but can do things like basic math.
The Apache license is a bit friendlier than GPL too.

Categories

Resources