I've used the Weka GUI for training and testing a file (making predictions), but can't do the same with the API. The error I'm getting says there's a different number of attributes in the train and test files. In the GUI, this can be solved by checking "Output predictions".
How to do something similar using the API? do you know of any samples out there?
import weka.classifiers.bayes.NaiveBayes;
import weka.classifiers.meta.FilteredClassifier;
import weka.classifiers.trees.J48;
import weka.core.Instances;
import weka.core.converters.ConverterUtils.DataSource;
import weka.filters.Filter;
import weka.filters.unsupervised.attribute.NominalToBinary;
import weka.filters.unsupervised.attribute.Remove;
public class WekaTutorial
{
public static void main(String[] args) throws Exception
{
DataSource trainSource = new DataSource("/tmp/classes - edited.arff"); // training
Instances trainData = trainSource.getDataSet();
DataSource testSource = new DataSource("/tmp/classes_testing.arff");
Instances testData = testSource.getDataSet();
if (trainData.classIndex() == -1)
{
trainData.setClassIndex(trainData.numAttributes() - 1);
}
if (testData.classIndex() == -1)
{
testData.setClassIndex(testData.numAttributes() - 1);
}
String[] options = weka.core.Utils.splitOptions("weka.filters.unsupervised.attribute.StringToWordVector -R first-last -W 1000 -prune-rate -1.0 -N 0 -stemmer weka.core.stemmers.NullStemmer -M 1 "
+ "-tokenizer \"weka.core.tokenizers.WordTokenizer -delimiters \" \\r\\n\\t.,;:\\\'\\\"()?!\"");
Remove remove = new Remove();
remove.setOptions(options);
remove.setInputFormat(trainData);
NominalToBinary filter = new NominalToBinary();
NaiveBayes nb = new NaiveBayes();
FilteredClassifier fc = new FilteredClassifier();
fc.setFilter(filter);
fc.setClassifier(nb);
// train and make predictions
fc.buildClassifier(trainData);
for (int i = 0; i < testData.numInstances(); i++)
{
double pred = fc.classifyInstance(testData.instance(i));
System.out.print("ID: " + testData.instance(i).value(0));
System.out.print(", actual: " + testData.classAttribute().value((int) testData.instance(i).classValue()));
System.out.println(", predicted: " + testData.classAttribute().value((int) pred));
}
}
}
Error:
Exception in thread "main" java.lang.IllegalArgumentException: Src and Dest differ in # of attributes: 2 != 17152
This was not an issue for the GUI.
You need to ensure that categories in train and test sets are compatible, try to
combine train and test sets
List item
preprocess them
save them as arff
open two empty files
copy the header from the top to line "#data"
copy in training set into first file and test set into second file
Related
I have a csv file containing 24231 rows. I would like to apply LOOCV based on the project name instead of the observations of the whole dataset.
So if my dataset contains information for 15 projects, I would like to have the training set based on 14 projects and the test set based on the other project.
I was relying on weka's API, is there anything that automates this process?
For non-numeric attributes, Weka allows you to retrieve the unique values via Attribute.numValues() (how many are there) and Attribute.value(int) (the -th value).
package weka;
import weka.core.Attribute;
import weka.core.Instance;
import weka.core.Instances;
import weka.core.converters.ConverterUtils;
public class LOOByValue {
/**
* 1st arg: ARFF file to load
* 2nd arg: 0-based index in ARFF to use for class
* 3rd arg: 0-based index in ARFF to use for LOO
*
* #param args the command-line arguments
* #throws Exception if loading/processing of data fails
*/
public static void main(String[] args) throws Exception {
// load data
Instances full = ConverterUtils.DataSource.read(args[0]);
full.setClassIndex(Integer.parseInt(args[1]));
int looCol = Integer.parseInt(args[2]);
Attribute looAtt = full.attribute(looCol);
if (looAtt.isNumeric())
throw new IllegalStateException("Attribute cannot be numeric!");
// iterate unique values to create train/test splits
for (int i = 0; i < looAtt.numValues(); i++) {
String value = looAtt.value(i);
System.out.println("\n" + (i+1) + "/" + full.attribute(looCol).numValues() + ": " + value);
Instances train = new Instances(full, full.numInstances());
Instances test = new Instances(full, full.numInstances());
for (int n = 0; n < full.numInstances(); n++) {
Instance inst = full.instance(n);
if (inst.stringValue(looCol).equals(value))
test.add((Instance) inst.copy());
else
train.add((Instance) inst.copy());
}
train.compactify();
test.compactify();
// TODO do something with the data
System.out.println("train size: " + train.numInstances());
System.out.println("test size: " + test.numInstances());
}
}
}
With Weka's anneal UCI dataset and the surface-quality for leave-one-out, you can generate something like this:
1/5: ?
train size: 654
test size: 244
2/5: D
train size: 843
test size: 55
3/5: E
train size: 588
test size: 310
4/5: F
train size: 838
test size: 60
5/5: G
train size: 669
test size: 229
I have a scheduled job and I want to get fixedRate dynamically but couldn't solve how to do it.
FixedRate gets value as milliseconds but I want to give time as hours. And I also tried read parameter from property file and multiply it but I did not work. How can I make this?
package com.ipera.communicationsuite.scheduleds;
import com.ipera.communicationsuite.models.FreeDbSize;
import com.ipera.communicationsuite.repositories.interfaces.IFreeDbSizeRepository;
import com.ipera.communicationsuite.repositories.interfaces.settings.IPropertiesRepository;
import com.ipera.communicationsuite.utilities.mail.SMTPConnection;
import lombok.AllArgsConstructor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.PropertySource;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
#Component
#AllArgsConstructor
#PropertySource("classpath:scheduled.properties")
public class KeepAlive {
private static Logger logger = LoggerFactory.getLogger(KeepAlive.class);
private IFreeDbSizeRepository freeDbSizeRepository;
private SMTPConnection smtpConnection;
private IPropertiesRepository propertiesRepository;
#Scheduled(fixedRateString ="${keepAlive.timer}")
public void keepAliveMailSender() {
StringBuilder content = new StringBuilder();
ArrayList<File> files = getDrivers();
List<FreeDbSize> list = freeDbSizeRepository.getFreeDbSize();
FreeDbSize dbDiskInfo = freeDbSizeRepository.dbDiskSize();
content.append("DB file size is: ").append(list.get(0).getType().equals("mdf") ? list.get(0).getFileSize() : list.get(1).getFileSize()).append(" MB\n")
.append("DB log size is: ").append(list.get(0).getType().equals("ldf") ? list.get(0).getFileSize() : list.get(1).getFileSize()).append(" MB\n");
propertiesRepository.updateByKey("DatabaseSize", list.get(0).getType().equals("mdf") ? list.get(0).getFileSize().toString() : list.get(1).getFileSize().toString());
propertiesRepository.updateByKey("DatabaseLogSize", list.get(0).getType().equals("ldf") ? list.get(0).getFileSize().toString() : list.get(1).getFileSize().toString());
propertiesRepository.updateByKey("FreeDiskSpaceForDb", dbDiskInfo.getFreeSpace().toString());
for (int i = 0; i < files.size(); i++) {
content.append("Free size for driver ").append(files.get(i)).append(" is ").append(files.get(i).getFreeSpace() / (1024 * 1024)).append(" MB\n");
propertiesRepository.createIfNotExistOrUpdate(("FreeSpaceInDisk".concat(Character.toString(files.get(i).toString().charAt(0)))), Long.toString(files.get(i).getFreeSpace() / (1024 * 1024)));
}
if (dbDiskInfo.getName().equals("-1")) {
content.append("This application has not permission to run query for calculate free size of disk.");
} else {
content.append("Free size of disk which contains Db is: ").append(dbDiskInfo.getFreeSpace());
}
smtpConnection.sendMail(content.toString(), "Server Is Up!!!", "fkalabalikoglu#iperasolutions.com", "", "", "", "");
logger.info("KeepAlive has runned.");
}
public ArrayList<File> getDrivers() {
ArrayList<File> list = new ArrayList<>();
File[] drives = File.listRoots();
if (drives != null && drives.length > 0) {
for (File aDrive : drives) {
list.add(aDrive);
}
}
return list;
}
}
And also my propery file is here:
keepAlive.timer=86400000
You could use SpEL in your annotation like:
#Scheduled(fixedRateString ="#{new Long('${keepAlive.timer}') * 1000 * 3600}")
to have expression evaluated. So keepAlive.timer would be the amount of hours.
But in my opinion it would be an ugly solution. I would rather put it in the properties as you have it now and just add a comment like:
# 24 hours is: 1000 * 3600 * 24
keepAlive.timer=86400000
Another way to use hours would be to use attribute cron that gives you more flexibility but might need some study before using:
In your code:
#Scheduled(cron = "${keepAlive.timer}")
and the cron expression in your properties - for example - like:
keepAlive.timer="*/60 00 21 * * ?"
This would run every day # 21.00
Note this "*/60" it should accept also "0" here but in my case it did not
package modifiedlines;
import edu.nyu.cs.javagit.api.DotGit;
import edu.nyu.cs.javagit.api.JavaGitConfiguration;
import edu.nyu.cs.javagit.api.JavaGitException;
import edu.nyu.cs.javagit.api.WorkingTree;
import edu.nyu.cs.javagit.api.commands.GitLogResponse;
import edu.nyu.cs.javagit.api.commands.GitStatus;
import edu.nyu.cs.javagit.api.commands.GitStatusOptions;
import edu.nyu.cs.javagit.api.commands.GitStatusResponse;
import java.io.File;
import java.io.IOException;
import java.util.List;
/**
*
* #
author aryan000
*/
public class UseGit {
private static File repositoryDirectory;
private static DotGit dotGit;
public static void main(String s[]) throws JavaGitException, IOException
{
File f = new File("C:\\Program Files\\Git\\bin\\git.exe");
if(!f.exists())
{
System.out.println("does not exist");
}
else
System.out.println("exists at " + f.getPath());
JavaGitConfiguration.setGitPath("C:\\Program Files\\Git\\bin");
System.out.println("Git version : " + JavaGitConfiguration.getGitVersion());
// repositoryDirectory = new File("/home/aryan000/Desktop/retrofit");
repositoryDirectory = new File("/home/aryan000/Desktop/changeprone/changeprone");
System.out.println("Git Repository Location : " + repositoryDirectory.getAbsolutePath());
//get the instance of the dotGit Object
dotGit = DotGit.getInstance(repositoryDirectory);
// System.out.println("checking what i have got ");
// GitLogResponse.CommitFile com ;
// com = (GitLogResponse.CommitFile) dotGit.getLog();
//
// System.out.println(com);
WorkingTree wt = dotGit.getWorkingTree();
File workingTreePath = wt.getPath();
GitStatus gitStatus = new GitStatus();
GitStatusResponse status = gitStatus.status(workingTreePath);
System.out.println("status is : " + status);
File anotherFileDir = new File("/home/aryan000/Desktop/retrofit/test.txt");
GitStatusOptions options = new GitStatusOptions();
options.setOptOnly(true);
status = gitStatus.status(workingTreePath);
System.out.println("status is : " + status);
System.out.println("----- Print log to see our commit -----");
for (GitLogResponse.Commit c : dotGit.getLog()) {
System.out.println("commit id is : " + c.getSha());
System.out.println(" commit message is : " + c.getMessage());
System.out.println(" author of the commit is : " + c.getAuthor());
System.out.println(" date modified is : " + c.getDateString());
System.out.println(" number of files changed is : " + c.getFiles());
List<GitLogResponse.CommitFile> store = c.getFiles();
if(store!=null)
System.out.println("the number of files changed is : " + store.size());
System.out.println("list of files changed is : " + c.getFilesChanged());
System.out.println("total number of additions : " + c.getLinesDeleted());
System.out.println("total number of merger : " + c.getMergeDetails());
}
// for(GitLogResponse.CommitFile c : dotGit.getLog())
}
}
Output is shown as :
Exception in thread "main" edu.nyu.cs.javagit.api.JavaGitException: 100002: Invalid path to git specified. { path=[C:\Program Files\Git\bin] }
at edu.nyu.cs.javagit.api.JavaGitConfiguration.setGitPath(JavaGitConfiguration.java:220)
at edu.nyu.cs.javagit.api.JavaGitConfiguration.setGitPath(JavaGitConfiguration.java:247)
at modifiedlines.UseGit.main(UseGit.java:40)
Caused by: edu.nyu.cs.javagit.api.JavaGitException: 100002: Invalid path to git specified.
at edu.nyu.cs.javagit.api.JavaGitConfiguration.determineGitVersion(JavaGitConfiguration.java:81)
at edu.nyu.cs.javagit.api.JavaGitConfiguration.setGitPath(JavaGitConfiguration.java:217)
... 2 more
Java Result: 1
My query is how to find git logs and the files changed due to a particular commit using a Java Program.
Can any 1 help me in this.
Please See : This code is working fine in Ubuntu i.e. no Path problem still I am unable to get the files changed during a commit. It is give me a List as a null.
This bug was opened a long time ago and just fixed it. Please give it a try , use master branch please, and let me know if it works for you.
I would like to perform a 10 fold cross validation on my data and I used the weka java programme. However, I encountered exception problems.
Here is the exceptions:
---Registering Weka Editors---
Trying to add database driver (JDBC): jdbc.idbDriver - Error, not in CLASSPATH?
Exception in thread "main" java.lang.IllegalArgumentException: No suitable converter found for ''!
at weka.core.converters.ConverterUtils$DataSource.<init>(ConverterUtils.java:137)
at weka.core.converters.ConverterUtils$DataSource.read(ConverterUtils.java:441)
at crossvalidationmultipleruns.CrossValidationMultipleRuns.main(CrossValidationMultipleRuns.java:45)
C:\Users\TomXavier\AppData\Local\NetBeans\Cache\8.1\executor-snippets\run.xml:53: Java returned: 1
BUILD FAILED (total time: 1 second)
Here is the programme I used:
import weka.core.Instances;
import weka.core.converters.ConverterUtils.DataSource;
import weka.core.Utils;
import weka.classifiers.Classifier;
import weka.classifiers.Evaluation;
import java.util.Random;
/**
* Performs a single run of cross-validation.
*
* Command-line parameters:
* <ul>
* <li>-t filename - the dataset to use</li>
* <li>-x int - the number of folds to use</li>
* <li>-s int - the seed for the random number generator</li>
* <li>-c int - the class index, "first" and "last" are accepted as well;
* "last" is used by default</li>
* <li>-W classifier - classname and options, enclosed by double quotes;
* the classifier to cross-validate</li>
* </ul>
*
* Example command-line:
* <pre>
* java CrossValidationSingleRun -t anneal.arff -c last -x 10 -s 1 -W "weka.classifiers.trees.J48 -C 0.25"
* </pre>
*
* #author FracPete (fracpete at waikato dot ac dot nz)
*/
public class CrossValidationSingleRun {
/**
* Performs the cross-validation. See Javadoc of class for information
* on command-line parameters.
*
* #param args the command-line parameters
* #throws Excecption if something goes wrong
*/
public static void main(String[] args) throws Exception {
// loads data and set class index
Instances data = DataSource.read(Utils.getOption("C:/Users/TomXavier/Documents/MATLAB/total_data.arff", args));
String clsIndex = Utils.getOption("first", args);
if (clsIndex.length() == 0)
clsIndex = "last";
if (clsIndex.equals("first"))
data.setClassIndex(0);
else if (clsIndex.equals("last"))
data.setClassIndex(data.numAttributes() - 1);
else
data.setClassIndex(Integer.parseInt(clsIndex) - 1);
// classifier
String[] tmpOptions;
String classname;
tmpOptions = Utils.splitOptions(Utils.getOption("weka.classifiers.trees.J48", args));
classname = tmpOptions[0];
tmpOptions[0] = "";
Classifier cls = (Classifier) Utils.forName(Classifier.class, classname, tmpOptions);
// other options
int seed = Integer.parseInt(Utils.getOption("1", args));
int folds = Integer.parseInt(Utils.getOption("10", args));
// randomize data
Random rand = new Random(seed);
Instances randData = new Instances(data);
randData.randomize(rand);
if (randData.classAttribute().isNominal())
randData.stratify(folds);
// perform cross-validation
Evaluation eval = new Evaluation(randData);
for (int n = 0; n < folds; n++) {
Instances train = randData.trainCV(folds, n);
Instances test = randData.testCV(folds, n);
// the above code is used by the StratifiedRemoveFolds filter, the
// code below by the Explorer/Experimenter:
// Instances train = randData.trainCV(folds, n, rand);
// build and evaluate classifier
Classifier clsCopy = Classifier.makeCopy(cls);
clsCopy.buildClassifier(train);
eval.evaluateModel(clsCopy, test);
}
// output evaluation
System.out.println();
System.out.println("=== Setup ===");
System.out.println("Classifier: " + cls.getClass().getName() + " " + Utils.joinOptions(cls.getOptions()));
System.out.println("Dataset: " + data.relationName());
System.out.println("Folds: " + folds);
System.out.println("Seed: " + seed);
System.out.println();
System.out.println(eval.toSummaryString("=== " + folds + "-fold Cross-validation ===", false));
}
}
Is there any solution for this problem?
Many thanks!
my problem is:
i have made a 3D model with texture in Cinema 4d ( similiar to this one : http://preview.turbosquid.com/Preview/2011/03/30__13_54_17/space%20shuttle%206.jpgeec17db2-6651-453c-9d27-ea1908e3c7dfLarge.jpg )
Now i want to export it to my jMonkeyEngine to set it up in my scene and to animate it.
I tried to export my model as a .obj-file and load it into my projekt (just the .obj-file).
The result is that i have no textures! What do i wrong?
package mygame;
import com.jme3.app.SimpleApplication;
import com.jme3.light.DirectionalLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.renderer.RenderManager;
import com.jme3.scene.Geometry;
import com.jme3.scene.Node;
import com.jme3.scene.Spatial;
import com.jme3.scene.shape.Box;
/**
* test
* #author normenhansen
*/
public class Main extends SimpleApplication {
public static void main(String[] args) {
Main app = new Main();
app.start();
}
#Override
public void simpleInitApp() {
//Modell laden
Spatial spaceShuttle =assetManager.loadModel("Models/test/space.obj");
//Skalieren
spaceShuttle.scale(0.005f, 0.005f, 0.005f);
//Szenenbaum erstellen
Node sceneNode = new Node("sceneNode");
Node geometryNode = new Node("geometryNode");
Node lightNode = new Node("lightNode");
sceneNode.attachChild(lightNode);
sceneNode.attachChild(geometryNode);
rootNode.attachChild(sceneNode);
//neue Elemente in den Baum Einfügen
geometryNode.attachChild(spaceShuttle);
DirectionalLight sun = new DirectionalLight();
sun.setDirection(new Vector3f(1,0,-2).normalizeLocal());
sun.setColor(ColorRGBA.White);
rootNode.addLight(sun);
}
#Override
public void simpleUpdate(float tpf) {
//TODO: add update code
}
#Override
public void simpleRender(RenderManager rm) {
//TODO: add render code
}
}
You're not doing anything wrong. C4D exports just the .obj, but no .mtl by default.
I know that's true for C4D R11.5 and R12, not sure about newer ones.
You'll can write a script to export the .mtl as well.
Here's a Python snippet for reference:
#save mtl
mcount = 0;
mtl = ''
for tag in op.GetTags():
if(tag.GetType() == 5616): #texture tag
mcount += 1
m = tag.GetMaterial()
mtl += 'newmtl '+clean(m.GetName())+'\n'
if(m[sy.MATERIAL_COLOR_COLOR]): mtl += 'Kd ' + str(m[sy.MATERIAL_COLOR_COLOR].x) + ' ' + str(m[sy.MATERIAL_COLOR_COLOR].y) + ' ' + str(m[sy.MATERIAL_COLOR_COLOR].z) + '\n'
if(m[sy.MATERIAL_SPECULAR_COLOR]): mtl += 'Ks ' + str(m[sy.MATERIAL_SPECULAR_COLOR].x) + ' ' + str(m[sy.MATERIAL_SPECULAR_COLOR].y) + ' ' + str(m[sy.MATERIAL_SPECULAR_COLOR].z) + '\n'
if(m[sy.MATERIAL_SPECULAR_BRIGHTNESS]): mtl += 'Ns ' + str(m[sy.MATERIAL_SPECULAR_BRIGHTNESS]) + '\n'
if(m[sy.MATERIAL_TRANSPARENCY_BRIGHTNESS]): mtl += 'd ' + str(m[sy.MATERIAL_TRANSPARENCY_BRIGHTNESS]) + '\n'
if(m[sy.MATERIAL_COLOR_SHADER]): mtl += 'map_Kd ' + str(m[sy.MATERIAL_COLOR_SHADER][sy.BITMAPSHADER_FILENAME]) + '\n'
if(m[sy.MATERIAL_TRANSPARENCY_SHADER]): mtl += 'map_d ' + str(m[sy.MATERIAL_COLOR_SHADER][sy.BITMAPSHADER_FILENAME]) + '\n'
if(m[sy.MATERIAL_BUMP_SHADER]): mtl += 'map_bump ' + str(m[sy.MATERIAL_BUMP_SHADER][sy.BITMAPSHADER_FILENAME]) + '\n'
mtl += 'illum 0\n\n\n'#TODO: setup the illumination, ambient and optical density
mtl = '# Material Count: '+str(mcount)+'\n'+mtl
file = open(mtlPath,'w')
file.write(mtl)
file.close()
It's part of this old script, but note this is with the old R11.5 C4D Python API, the syntax is a bit different now, so use updated documentation and the above more as a general direction on what properties to look for.
A 'codeless' alternative is to bring your model into a different 3D package which properly exports .obj (and .mtl) like Blender for example. You will need to find an intermediary format that will preserve the material data (you can try 3DS,Collada, FBX I think) but be aware of the difference in units and coordinate systems. Hopefully the model features you need are preserved in the file format you export from C4D and properly imported back into the other 3D package.