Develop Reference

Using WEKA Filters in Java - Oversampling and Undersampling

I'm having an issue with finding out how to use WEKA filters in the java code. I've looked up help but it seems a little dated as I'm using WEKA 3.8.5 . I'm doing 3 test. Test 1: No Filter, Test 2: weka.filters.supervised.instance.SpreadSubsample -M 1.0 , and Test 3: weka.filters.supervised.instance.Resample -B 1.0 -Z 130.3.
If my research is correct I should import the filters like this. Now I'm lost on having "-M 1.0 " for SpreadSample(my under sampling Test) and "-B 1.0 -Z 130.3." for Resample(My oversampling test).
import weka.classifiers.trees.J48;
import weka.core.Instances;
import weka.filters.Filter;
import weka.filters.supervised.instance.Resample;
import weka.filters.supervised.instance.SpreadSubsample;
And I have Test 1(my no filter Test) coded below
import java.io.FileReader;
import java.util.Random;
import weka.classifiers.Evaluation;
import weka.classifiers.trees.J48;
import weka.core.Instances;
public class Fraud {
public static void main(String args[])
{
try {
// Creating J48 classifier for the tree
J48 j48Classifier = new J48();
// Setting the path for the dataset
String FraudDataset = "C:\\Users\\Owner\\Desktop\\CreditCard\\CreditCard.arff";
BufferedReader bufferedReader
= new BufferedReader(
new FileReader(FraudDataset));
// Creating the data set instances
Instances datasetInstances
= new Instances(bufferedReader);
datasetInstances.setClassIndex(
datasetInstances.numAttributes() - 1);
Evaluation evaluation
= new Evaluation(datasetInstances);
// Cross Validate Model. 10 Folds
evaluation.crossValidateModel(
j48Classifier, datasetInstances, 10,
new Random(1));
System.out.println(evaluation.toSummaryString(
"\nResults", false));
}
// Catching exceptions
catch (Exception e) {
System.out.println("Error Occured!!!! \n"
+ e.getMessage());
}
System.out.print("DT Successfully executed.");
}
}
The results of my code is:
Results
Correctly Classified Instances 284649 99.9445 %
Incorrectly Classified Instances 158 0.0555 %
Kappa statistic 0.8257
Mean absolute error 0.0008
Root mean squared error 0.0232
Relative absolute error 24.2995 %
Root relative squared error 55.9107 %
Total Number of Instances 284807
DT Successfully executed.
Does anyone have an idea on how I can add the filters and the settings I want for the filters to the code for Test 2 and 3? Any help will be appreciated. I will run the 3 tests multiple times and compare the results. I want to see what works best of the 3.

-M 1.0 and -B 1.0 -Z 130.3 are the options that you supply to the filters from the command-line. These filters implement the weka.core.OptionHandler interface, which offers the setOptions and getOptions methods.
For example, SpreadSubsample can be instantiated like this:
import weka.filters.supervised.instance.SpreadSubsample;
import weka.core.Utils;
...
SpreadSubsample spread = new SpreadSubsample();
// Utils.splitOptions generates an array from an option string
spread.setOptions(Utils.splitOptions("-M 1.0"));
// alternatively:
// spread.setOptions(new String[]{"-M", "1.0"});
In order to apply the filters, you should use the FilteredClassifier approach. E.g., for SpreadSubsample you would do something like this:
import weka.classifiers.meta.FilteredClassifier;
import weka.classifiers.trees.J48;
import weka.filters.supervised.instance.SpreadSubsample;
import weka.core.Utils;
...
// base classifier
J48 j48 = new J48();
// filter
SpreadSubsample spread = new SpreadSubsample();
spread.setOptions(Utils.splitOptions("-M 1.0"));
// meta-classifier
FilteredClassifier fc = new FilteredClassifier();
fc.setFilter(spread);
fc.setClassifier(j48);
And then evaluate the fc classifier object on your dataset.

Nashorn Abstract Syntax Tree Traversal

I am attempting to parse this Javascript via Nashorn:
function someFunction() { return b + 1 };
and navigate to all of the statements. This including statements inside the function.
The code below just prints:
"function {U%}someFunction = [] function {U%}someFunction()"
How do I "get inside" the function node to it's body "return b + 1"? I presume I need to traverse the tree with a visitor and get the child node?
I have been following the second answer to the following question:
Javascript parser for Java
import jdk.nashorn.internal.ir.Block;
import jdk.nashorn.internal.ir.FunctionNode;
import jdk.nashorn.internal.ir.Statement;
import jdk.nashorn.internal.parser.Parser;
import jdk.nashorn.internal.runtime.Context;
import jdk.nashorn.internal.runtime.ErrorManager;
import jdk.nashorn.internal.runtime.Source;
import jdk.nashorn.internal.runtime.options.Options;
import java.util.List;
public class Main {
public static void main(String[] args){
Options options = new Options("nashorn");
options.set("anon.functions", true);
options.set("parse.only", true);
options.set("scripting", true);
ErrorManager errors = new ErrorManager();
Context context = new Context(options, errors, Thread.currentThread().getContextClassLoader());
Source source = Source.sourceFor("test", "function someFunction() { return b + 1; } ");
Parser parser = new Parser(context.getEnv(), source, errors);
FunctionNode functionNode = parser.parse();
Block block = functionNode.getBody();
List<Statement> statements = block.getStatements();
for(Statement statement: statements){
System.out.println(statement);
}
}
}

Using private/internal implementation classes of nashorn engine is not a good idea. With security manager on, you'll get access exception. With jdk9 and beyond, you'll get module access error w/without security manager (as jdk.nashorn.internal.* packages not exported from nashorn module).
You've two options to parse javascript using nashorn:
Nashorn parser API ->https://docs.oracle.com/javase/9/docs/api/jdk/nashorn/api/tree/Parser.html
To use Parser API, you need to use jdk9+.
For jdk8, you can use parser.js
load("nashorn:parser.js");
and call "parse" function from script. This function returns a JSON object that represents AST of the script parsed.
See this sample: http://hg.openjdk.java.net/jdk8u/jdk8u-dev/nashorn/file/a6d0aec77286/samples/astviewer.js

Jython jlist text not showing in JFrame

I am writing a little stock quote application and when I compile the code the space for the text will be taken in the Jlist but no text will appear.
def loadStockDisplay(self):
self.display = JPanel()
self.display.setLayout(BorderLayout())
self.display.setBackground(Color.decode("#0A0A33"))
self.display.setBorder(BorderFactory.createMatteBorder(0,3,0,0,Color.decode("#8080E6")))
self.label = JLabel("Stocks")
self.label.setForeground(Color.decode("#FCFCFC"))
self.label.setFont(self.font)
self.display.add(self.label,BorderLayout.NORTH)
self.stocks = DefaultListModel();
self.items = JList(self.stocks)
self.items.setBackground(Color.decode("#0A0A33"))
self.items.setForeground(Color.decode("#FCFCFC"))
self.items.setFont(self.font)
self.items.setSelectionBackground(Color.decode("#0A0A33"))
self.items.setSelectionForeground(Color.decode("#FCFCFC"))
self.display.add(self.items, BorderLayout.CENTER)
self.frame.add(self.display,BorderLayout.EAST)
self.updateStocks()
def updateStocks(self):
companys = ["MSFT","SNDK","GOOGL","NOK","EMC","HPQ","IBM","EBAY","AAPL","AMZN"]
tempList = []
for company in companys:
Quote = web()
tempList.append(company + " " + str(Quote.getQuote(company)))
self.stocks.clear()
for item in tempList:
self.stocks.addElement(item)

Maybe there is a problem in the code that is not yet available in the question? Could you share the class and main method (if you have these)? There could also be an issue with fetching the quotes from the web.
If I add some code to make your program run, the list has items for each company (with a dummy quote):
from java.awt import BorderLayout, Color
from javax.swing import BorderFactory, DefaultListModel, JFrame, JLabel, JList, JPanel
class StocksTest:
def loadStockDisplay(self):
self.frame = JFrame('Stocks', defaultCloseOperation=JFrame.EXIT_ON_CLOSE, size=(300, 300), locationRelativeTo=None)
self.display = JPanel()
self.display.setLayout(BorderLayout())
self.display.setBackground(Color.decode("#0A0A33"))
self.display.setBorder(BorderFactory.createMatteBorder(0,3,0,0,Color.decode("#8080E6")))
self.label = JLabel("Stocks")
self.label.setForeground(Color.decode("#FCFCFC"))
self.label.setFont(self.frame.font)
self.display.add(self.label,BorderLayout.NORTH)
self.stocks = DefaultListModel();
self.items = JList(self.stocks)
self.items.setBackground(Color.decode("#0A0A33"))
self.items.setForeground(Color.decode("#FCFCFC"))
self.items.setFont(self.frame.font)
self.items.setSelectionBackground(Color.decode("#0A0A33"))
self.items.setSelectionForeground(Color.decode("#FCFCFC"))
self.display.add(self.items, BorderLayout.CENTER)
self.frame.add(self.display,BorderLayout.EAST)
self.frame.setVisible(True)
self.updateStocks()
def updateStocks(self):
companys = ["MSFT","SNDK","GOOGL","NOK","EMC","HPQ","IBM","EBAY","AAPL","AMZN"]
tempList = []
for company in companys:
#Quote = web()
#companyQuote = Quote.getQuote(company)
companyQuote = len(str(company)) * 314.15
tempList.append(company + " " + str(companyQuote))
self.stocks.clear()
for item in tempList:
self.stocks.addElement(item)
def main():
StocksTest().loadStockDisplay()
if __name__ == '__main__':
main()

How to parse data in Talend with Java (coming from a previously produced .txt file)?

I have a process in Talend which gets the search result of a page, saves the html and writes it into files, as seen here:
Initially I had a two step process with parsing out the date from the HTML files in Java. Here is the code: It works and writes it to a mysql database. Here is the code which basically does exactly that. (I'm a beginner, sorry for the lack of elegance)
package org.jsoup.examples;
import java.io.*;
import org.jsoup.*;
import org.jsoup.nodes.*;
import org.jsoup.select.Elements;
import java.io.IOException;
public class parse2 {
static parse2 parseIt2 = new parse2();
String companyName = "Platzhalter";
String jobTitle = "Platzhalter";
String location = "Platzhalter";
String timeAdded = "Platzhalter";
public static void main(String[] args) throws IOException {
parseIt2.getData();
}
//
public void getData() throws IOException {
Document document = Jsoup.parse(new File("C:/Talend/workspace/WEBCRAWLER/output/keywords_SOA.txt"), "utf-8");
Elements elements = document.select(".joblisting");
for (Element element : elements) {
// Parse Data into Elements
Elements jobTitleElement = element.select(".job_title span");
Elements companyNameElement = element.select(".company_name span[itemprop=name]");
Elements locationElement = element.select(".locality span[itemprop=addressLocality]");
Elements dateElement = element.select(".job_date_added [datetime]");
// Strip Data from unnecessary tags
String companyName = companyNameElement.text();
String jobTitle = jobTitleElement.text();
String location = locationElement.text();
String timeAdded = dateElement.attr("datetime");
System.out.println("Firma:\t"+ companyName + "\t" + jobTitle + "\t in:\t" + location + " \t Erstellt am \t" + timeAdded );
}
}
}
Now I want to do the process End-to-End in Talend, and I got assured this works.
I tried this (which looks quite shady to me):
Basically I put all imports in "advanced settings" and the code in the "basic settings" section. This importLibrary is thought to load the jsoup parsing library, as well as the mysql connect (i might to the connect with talend tools though).
Obviously this isn't working. I tried to strip the Base Code from classes and stuff and it was even worse. Can you help me how to get the generated .txt files parsed with Java here?
EDIT: Here is the Link to the talend Job http://www.share-online.biz/dl/8M5MD99NR1
EDIT2: I changed the code to the one I tried in JavaFlex. But it didn't work (the import part in the start part of the code, the rest in "body/main" and nothing in "end".

This is a problem related to Talend, in your code, use the complete method names including their packages. For your document parsing for example, you can use :
Document document = org.jsoup.Jsoup.parse(new File("C:/Talend/workspace/WEBCRAWLER/output/keywords_SOA.txt"), "utf-8");

ANTLR java test file can't create object of tree grammar

I am creating a parser using ANTLR 3.x that targets java. I have written both parser grammar (for creating Abstract Syntax Tree, AST) and Tree Grammar (for performing operations on AST). Finally, to test both grammar files, I have written a test file in Java.
Have a look at the below code,
protocol grammar
grammar protocol;
options {
language = Java;
output = AST;
}
tokens{ //imaginary tokens
PROT;
INITIALP;
PROC;
TRANSITIONS;
}
#header {
import twoprocess.Configuration;
package com.javadude.antlr3.x.tutorial;
}
#lexer::header {
package com.javadude.antlr3.x.tutorial;
}
/*
parser rules, in lowercase letters
*/
program
: declaration+
;
declaration
:protocol
|initialprocess
|process
|transitions
;
protocol
:'protocol' ID ';' -> ^(PROT ID)
;
initialprocess
:'pin' '=' INT ';' -> ^(INITIALP INT)
;
process
:'p' '=' INT ';' -> ^(PROC INT)
;
transitions
:'transitions' '=' INT ('(' INT ',' INT ')') + ';' -> ^(TRANSITIONS INT INT INT*)
;
/*
lexer rules (tokens), in upper case letters
*/
ID
: (('a'..'z' | 'A'..'Z'|'_')('a'..'z' | 'A'..'Z'|'0'..'9'|'_'))*;
INT
: ('0'..'9')+;
WHITESPACE
: ('\t' | ' ' | '\r' | '\n' | '\u000C')+ {$channel = HIDDEN;};
protocolWalker
grammar protocolWalker;
options {
language = Java;
//Error, eclipse can't access tokenVocab named protocol
tokenVocab = protocol; //import tokens from protocol.g i.e, from protocol.tokens file
ASTLabelType = CommonTree;
}
#header {
import twoprocess.Configuration;
package com.javadude.antlr3.x.tutorial;
}
program
: declaration+
;
declaration
:protocol
|initialprocess
|process
|transitions
;
protocol
:^(PROT ID)
{System.out.println("create protocol " +$ID.text);}
;
initialprocess
:^(INITIALP INT)
{System.out.println("");}
;
process
:^(PROC INT)
{System.out.println("");}
;
transitions
:^(TRANSITIONS INT INT INT*)
{System.out.println("");}
;
Protocoltest.java
package com.javadude.antlr3.x.tutorial;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.CommonTreeNodeStream;
public class Protocoltest {
/**
* #param args
*/
public static void main(String[] args) throws Exception {
//create input stream from standard input
ANTLRInputStream input = new ANTLRInputStream(System.in);
//create a lexer attached to that input stream
protocolLexer lexer = new protocolLexer(input);
//create a stream of tokens pulled from the lexer
CommonTokenStream tokens = new CommonTokenStream(lexer);
//create a pareser attached to teh token stream
protocolParser parser = new protocolParser(tokens);
//invoke the program rule in get return value
protocolParser.program_return r =parser.program();
CommonTree t = (CommonTree)r.getTree();
//output the extracted tree to the console
System.out.println(t.toStringTree());
//walk resulting tree; create treenode stream first
CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
//AST nodes have payloads that point into token stream
nodes.setTokenStream(tokens);
//create a tree walker attached to the nodes stream
//Error, can't create TreeGrammar object called walker
protocolWalker walker = new protocolWalker(nodes);
//invoke the start symbol, rule program
walker.program();
}
}
Problems:
In protocolWalker, I can't access the tokens (protocol.tokens)
//Error, eclipse can't access tokenVocab named protocol
tokenVocab = protocol; //import tokens from protocol.g i.e, from protocol.tokens file
In In protocolWalker, can I create the object of java class, called Configuration, in the action list?
protocol
:^(PROT ID)
{System.out.println("create protocol " +$ID.text);
Configuration conf = new Configuration();
}
;
In Protocoltest.java
//create a tree walker attached to the nodes stream
//Error, can't create TreeGrammar object called walker
protocolWalker walker = new protocolWalker(nodes);
Object of protocolWalker can't be created. I have seen in the examples and the tutorials that such object is created.

In protocolWalker, I can't access the tokens (protocol.tokens)...
It seems to be accessing protocol.tokens fine: changing tokenVocab to something else produces an error that it doesn't produce now. The problem with protocolWalker.g is that it's defined as a token parser (grammar protocolWalker) but it's being used like a tree parser. Defining the grammar as tree grammar protocolWalker took away the errors that I was seeing about the undefined tokens.
In protocolWalker, can I create the object of java class, called Configuration, in the action list?
Yes, you can. The normal Java programming caveats apply about importing the class and so on, but it's as available to you as code like System.out.println.
In Protocoltest.java ... Object of protocolWalker can't be created.
protocolWalker.g (as it is now) produces a token parser named protocolWalkerParser. When you change it to a tree grammar, it'll produce a tree parser named protocolWalker instead.
Thanks a lot for posting the whole grammars. That made answering the question much easier.

Thank you for your reply, that was a silly mistake.
Tokens problem and creating object of protocolWalker is resolved now but whenever, I change the grammar whether, protocol.g or protocolWalker.g, I had to write package name again(every time) in protocolParser.java and protocolWalker.java. I had the same problem with lexer file before but that was overcomed by the following declaration.
#header {
package com.javadude.antlr3.x.tutorial;
}
but I don't know how to overcome this problem?
Also, I have developed a GUI in Java using SWING where I have a textarea. In that text area,
user will write the input, like for my grammar user will write,
protocol test;
pin = 5;
p = 3;
transitions = 2(5,0) (5,1);
How can I process this input in Java Swing GUI and produce output there?
Moreover, if I give the following section of protocolWalker.g to
protocol
:^(PROT ID)
{
System.out.println("create protocol " +$ID.text);
Configuration conf = new Configuration();
conf.showConfiguration();
}
;
initialprocess
:^(INITIALP INT)
{System.out.println("create initial process (with state) ");}
;
process
:^(PROC INT)
{System.out.println("create processes ");}
;
and run the test file with the following input,
protocol test;
pin = 5;
p = 3;
transitions = 2(5,0) (5,1);
I get the following output
(PROT test) (INITIALP 5) (PROC 3) (TRANSITIONS 2 5 0 5 1)
create protocol test
why the second and the third println in the protocolWalker.g are not shown in the output?
Any thoughts/help?
Thank you once again.