I am a beginner in java development field and still i am a learner of Java Programming. I wanted to see the output for the Support Vector Machine classifier on netbeans IDE. So i copied this attached piece of code and tried to run by using all the other required class and main method as well but i am getting Number format exception when i give a file containing input like 23,25,26,27 during the call of the method loadBinaryProblem() in main method and if i remove all the commas and replaced them with space ex: 23 25 26 27 then i am getting ArrayIndexOutOfBound exception instead of it. So anybody can help to get the output properly without any error.
package svmlearn;
import java.io.*;
import java.util.*;
/**
* Class representing an optimization problem (a data setting);
* taken from liblinear; "bias" excluded
* #author miafranc
*
*/
public class Problem {
/** The number of training data */
public int l;
/** The number of features (including the bias feature if bias >= 0) */
public int n;
/** Array containing the target values */
public int[] y;
/** Map of categories to allow various ID's to identify classes with. */
public CategoryMap<Integer> catmap;
/** Array of sparse feature nodes */
public FeatureNode[][] x;
public Problem() {
l = 0;
n = 0;
catmap = new CategoryMap<Integer>();
}
/**
* Loads a binary problem from file, i.e. having 2 classes.
* #param filename The filename containing the problem in LibSVM format.
*/
public void loadBinaryProblem(String filename) {
String row;
ArrayList<Integer> classes = new ArrayList<Integer>();
ArrayList<FeatureNode []> examples = new ArrayList<FeatureNode []>();
try {
BufferedReader r = new BufferedReader(new FileReader(filename));
while ((row = r.readLine()) != null) {
String [] elems = row.split(" ");
//Category:
Integer cat = Integer.parseInt(elems[0]);
catmap.addCategory(cat);
if (catmap.size() > 2) {
throw new IllegalArgumentException("only 2 classes allowed!");
}
classes.add(catmap.getNewCategoryOf(cat));
//Index/value pairs:
examples.add(parseRow(elems));
}
x = new FeatureNode[examples.size()][];
y = new int[examples.size()];
for (int i=0; i<examples.size(); i++) {
x[i] = examples.get(i);
y[i] = 2*classes.get(i)-1; //0,1 => -1,1
}
l = examples.size();
} catch (Exception e) {
System.out.println(e);
}
}
/**
* Parses a row from a LibSVM format file.
* #param row The already split row on spaces.
* #return The corresponding FeatureNode.
*/
public FeatureNode [] parseRow(String [] row) {
FeatureNode [] example = new FeatureNode[row.length-1];
int maxindex = 0;
for (int i=1; i<row.length; i++) {
String [] iv = row[i].split(":");
int index = Integer.parseInt(iv[0]);
if (index <= maxindex) {
throw new IllegalArgumentException("indices must be in increasing order!");
}
maxindex = index;
double value = Double.parseDouble(iv[1]);
example[i-1] = new FeatureNode(index, value);
}
if (n < maxindex)
n = maxindex;
return example;
}
}
i guess NumberformatExceptions comes from:
String [] elems = row.split(" "); //nothing done by "23,25,26,27"
//Category:
Integer cat = Integer.parseInt(elems[0]); //you are trying to parse "23,25,26,27"
ArrayIndexOutOfBound comes from:
String [] iv = row[i].split(":");//nothing done
...
double value = Double.parseDouble(iv[1]);//1 is out of bound
Related
I am struggling with a beginning Java class. I have to modify a program to replace a user-generated integer array with a Random() number generated double precision floating point array.
This is what I have so far. I think it is generating the correct dataset, but I can't get the PrintWriter section configured correctly.
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
package assign6array;
import java.io.PrintWriter;
import java.io.FileNotFoundException;
import java.util.Random;
/**
*
* #author matthew.neesley
*/
public class Assign6Array {
/**
* #param args the command line arguments
* #throws java.io.FileNotFoundException
*/
public static void main(String[] args) throws FileNotFoundException {
// TODO code application logic here
int[] array = new int[10];
int count = 0;
int numbers = 0;
Random rd = new Random(); // creating Random object
double[] arr = new double[10];
for (int i = 0; i < array.length; i++) {
arr[i] = rd.nextInt(); // storing random integers in an array
while (numbers!= -1 && count <= 9)
{
array[count] = numbers;
count++;
System.out.println(arr[i]); // printing each array element
PrintWriter writer = new PrintWriter(System.out);
printr.print(arr[i]);
}
}
}
}
You are creating PrintWriter like this
PrintWriter writer = new PrintWriter(System.out);
The variable is named writer, but you use it like this:
printr.print(arr[i]);
Using printr variable which doesn't exist. Do simply:
writer.print(arr[i]);
Also, you need PrintStream acts as a buffer, so you will have to flush it's contents so that they are written to the output stream.
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
import java.io.PrintWriter;
import java.io.FileNotFoundException;
import java.util.Random;
/**
* #author matthew.neesley
*/
public class Assign6Array {
/**
* #param args the command line arguments
* #throws java.io.FileNotFoundException
*/
public static void main(String[] args) throws FileNotFoundException {
// TODO code application logic here
int[] array = new int[10];
int count = 0;
int numbers = 0;
Random rd = new Random(); // creating Random object
double[] arr = new double[10];
for (int i = 0; i < array.length; i++) {
arr[i] = rd.nextInt(); // storing random integers in an array
PrintWriter writer = new PrintWriter(System.out);
while (numbers != -1 && count <= 9) {
array[count] = numbers;
count++;
System.out.println("abc " + arr[i]); // printing each array element
writer.println(arr[i]);
writer.flush();
}
writer.close();
}
}
}
This question already has answers here:
What is a NullPointerException, and how do I fix it?
(12 answers)
Closed 3 years ago.
I'm trying to print out the names of the employees and their department and location from a list of the names and id numbers and I keep getting the NullPointerException even though it prints all of the names and locations. It then stops the build and doesn't xecute the print department and print location methods.
I've tried re-doing the for loops and seeing if any one data point was the problem but it seems to happen if I do the loop for all of the Employee objects or if I just do one.
package homework5_parth_desai;
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;
/**
*
* #author Party Parth
*/
public class Homework5_Parth_Desai {
public static int emplIndex = -1;
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws FileNotFoundException {
File file = new File("acmeEgr.txt");
Scanner scan = new Scanner(file);
Employee[] emp = new Employee[50];
String s = "";
String t = "";
int r = 0;
while (scan.hasNextLine()) { //scans in file
emplIndex++;
emp[emplIndex] = new Employee();
if (scan.hasNextLine() == true) { //takes first line as first name, second as last naem and third as id number and tehn ccreates an object out of that
s = scan.nextLine();
}
if (scan.hasNextLine() == true) {
t = scan.nextLine();
}
if (scan.hasNextLine() == true) {
r = Integer.parseInt(scan.nextLine());
}
emp[emplIndex].Employee(s, t, r);
// TODO code application logic here
}
printAll(emp);
printDepartment("IT", emp);
printLocation("Auburn Hills", emp);
}
static void printAll(Employee[] ppl) {
for (int i = 0; i < ppl.length; i++) {
System.out.println(ppl[i].toString());
}
}
static void printDepartment(String title, Employee[] ppl) {
for (int i = 0; i < ppl.length; i++) {
if (title.equals(ppl[i].getDept())) {
System.out.println(ppl[i].getName() + " is in " + ppl[i].getLocation());
}
}
}
static void printLocation(String loc, Employee[] ppl) {
for (int i = 0; i < ppl.length; i++) {
if (loc.equals(ppl[i].getLocation())) {
System.out.println(ppl[i].getName() + " is in " + ppl[i].getDept());
}
}
}
}
Small exert of the .txt file:
Alexander
Seiber
10010
Zehua
Showalter
20010
Cassidy
Woodle
20030
Randall
Shaukat
10030
Pam
Korda
10020
Justin
Polito
20030
public static int emplIndex = -1;
Why is the index maintained as a static field? Don't do that.
Employee[] emp = new Employee[50];
The employee array has a fixed size of 50 elements, however
while (scan.hasNextLine()) {
this loop is based on the lines of the acmeEgr.txt file, which might be more than 50.
In that case, you'll get an ArrayOutOfBoundException first
emp[emplIndex] = new Employee();
or a NullPointerException after
emp[emplIndex].Employee(s, t, r);
Instead, if the lines are less then 50, this
for (int i = 0; i < ppl.length; i++) {
System.out.println(ppl[i].toString());
}
will still loop all the 50 elements, because
ppl.length = 50
Thus, this line
ppl[i].toString()
will throw a NullPointerException.
This is what happens if the elements are, for example, 40
System.out.println(ppl[0].toString());
System.out.println(ppl[1].toString());
System.out.println(ppl[2].toString());
System.out.println(ppl[3].toString());
...
System.out.println(ppl[40].toString()); // ppl[40] is null, NullPointerException!
ArrayList is a much easier array type to deal with. Try using it instead of a normal array, because then you don't have to deal with indexes.
I am testing a program I wrote for an assignment, and right now it's testing at 50% against test cases. I get the following errors when I submit, but I can't figure out why. I tried changing some pieces of code and retesting, but no matter what I do I get the same errors, so I must be completely missing what I should be looking for. Here is the CSV file from which the data is being pulled -
CSV Data File. Here are the errors I get:
Here is my code:
Main class:
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
package csc212hw04;
import java.io.File;
import java.util.Scanner;
/**
*
* #author Michal
*/
public class Main {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws Exception{
Scanner kb = new Scanner(System.in);
boolean flag = true;
System.out.println("Welcome to the Car Database");
System.out.println("Enter the size of the array:");
int size = Integer.parseInt(kb.nextLine());
CarDatabase db1 = new CarDatabase(size);
System.out.println("Enter the name of the input file:");
String userFile = new String(kb.nextLine());
db1.readFile(userFile);
while (flag) {
System.out.println("Enter make, mpg, weight, all, or quit:");
String command = kb.nextLine();
if (command.equals("make")) {
System.out.println("Enter the make:");
String make = kb.nextLine();
db1.displayMake(make);
} else if (command.equals("mpg")) {
System.out.println("Enter the mpg range:");
double mpgLow = Double.parseDouble(kb.nextLine());
double mpgHigh = Double.parseDouble(kb.nextLine());
db1.mpgRange(mpgLow, mpgHigh);
} else if (command.equals("weight")) {
System.out.println("Enter the weight range:");
double weightLow = Double.parseDouble(kb.next());
double weightHigh = Double.parseDouble(kb.next());
db1.weightRange(weightLow, weightHigh);
} else if (command.equals("all")) {
CarDatabase.displayAll();
} else if (command.equals("quit")) {
flag = false;
}
}
}
}
CarDatabase class:
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
package csc212hw04;
import java.util.Scanner;
import java.io.File;
import java.io.FileNotFoundException;
/**
*
* #author Michal
*/
public class CarDatabase {
private static Car[] database;
public CarDatabase(int s) {
this.database = new Car[s];
}
public boolean isFull() {
boolean full = true;
for (int i = 0; i < database.length; i++) {
if (database[i] == null) {
full = false;
}
}
return full;
}
public static void readFile(String f) throws FileNotFoundException {
File file = new File(f);
int lineNum = 0;
Scanner sc = new Scanner(file);
String csvSplitBy = ",";
while (sc.hasNextLine()) {
String line = sc.nextLine();
String[] carData = line.split(csvSplitBy);
String model = carData[0];
String make = carData[1];
double mpg = Double.parseDouble(carData[2]);
int weight = Integer.parseInt(carData[3]);
int year = Integer.parseInt(carData[4]);
database[lineNum] = new Car(model, make, mpg, weight, year);
lineNum++;
}
}
public static void displayMake(String m) {
for (int i = 0; i < database.length; i++) {
if (database[i].make.equals(m)) {
database[i].toString();
}
}
}
public static void mpgRange(double l, double h) {
for (int i = 0; i < database.length; i++) {
if (database[i].mpg >= l && database[i].mpg <= h) {
database[i].toString();
}
}
}
public static void weightRange(double l, double h) {
for (int i = 0; i < database.length; i++) {
if ((database[i].weight >= l) && (database[i].weight <= h)) {
database[i].toString();
}
}
}
public static void displayAll() {
for (int i = 0; i < database.length; i++) {
database[i].toString();
}
}
}
Car class:
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
package csc212hw04;
/**
*
* #author Michal
*/
public class Car {
public String model, make;
public double mpg;
public int weight, year;
public Car(String md, String mk, double umpg, int w, int y) {
model = md;
make = mk;
mpg = umpg;
weight = w;
year = y;
}
public String toString() {
return "Model:" + model + " Make:" + make + " mpg:" + " weight:" + " year:" + year;
}
}
Here is a sample from the CSV file if you cannot see it:
You are probably getting to the End of File and you try to split. The array then doesn't have any value. Make sure there is no blank line at the end of your database, or put a check in your readfile to make sure that if the carData.length == NUMBER OF DATA FIELDS
Update
You also should check to make sure that you don't pass the total number of database entries you have. So do:
while(sc.hasNextLine() && lineNum < database.length) {
ArrayIndexOutOfBoundsException is thrown in very specific circumstances:
Thrown to indicate that an array has been accessed with an illegal
index. The index is either negative or greater than or equal to the
size of the array.
In your code, you assume that each row in your csv has the same number of fields.
You're also assuming that the size of database array matches the number of cars in your file. Since you don't create your database array entries until you read them from the file - but you pre-initialize the size database array, you could end up reading past the end of your database if the number of records in the CSV is larger than the value you initialize for the database size.
I am new no ML and I hava strated using Deeplearning4j library. And I literaly got lost in the source code. How can i read training set with multiple labels, but not just 1? For example I wan't to teach lstm to classify texts in 4 classes. How can i read trainig dataset for that?
Thanks
Edit:
This is what my iterator's code looks like now. I hava got POJO class for vacancy, which contains only list of skill's ids and vacancy text. In each file for each train/test set 2 lines: one with ids (comma is the separator) and text. All set contains 4 skills, so net's outputs equals 5. I have trained word2vec model, so my iterator also uses that.
I use original code example for sentimenal analysis
My iterator:
package SkillsMiner;
import SkillsMiner.Entities.VacancyLightEntity;
import SkillsMiner.Utils.Reader;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.FilenameUtils;
import org.deeplearning4j.datasets.iterator.DataSetIterator;
import org.deeplearning4j.models.embeddings.wordvectors.WordVectors;
import org.deeplearning4j.text.tokenization.tokenizer.preprocessor.CommonPreprocessor;
import org.deeplearning4j.text.tokenization.tokenizerfactory.DefaultTokenizerFactory;
import org.deeplearning4j.text.tokenization.tokenizerfactory.TokenizerFactory;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.dataset.DataSet;
import org.nd4j.linalg.dataset.api.DataSetPreProcessor;
import org.nd4j.linalg.factory.Nd4j;
import org.nd4j.linalg.indexing.INDArrayIndex;
import org.nd4j.linalg.indexing.NDArrayIndex;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.NoSuchElementException;
/** This is a DataSetIterator that is specialized for the IMDB review dataset used in the Word2VecSentimentRNN example
* It takes either the train or test set data from this data set, plus a WordVectors object (typically the Google News
* 300 pretrained vectors from https://code.google.com/p/word2vec/) and generates training data sets.<br>
* Inputs/features: variable-length time series, where each word (with unknown words removed) is represented by
* its Word2Vec vector representation.<br>
* Labels/target: a single class (negative or positive), predicted at the final time step (word) of each review
*
* #author Alex Black
*/
public class SentimentExampleIterator implements DataSetIterator {
private final WordVectors wordVectors;
private final int batchSize;
private final int vectorSize;
private final int truncateLength;
private int cursor = 0;
private final File[] filePathes;
private final TokenizerFactory tokenizerFactory;
private int labelsCount = 4;
/**
* #param dataDirectory the directory of the IMDB review data set
* #param wordVectors WordVectors object
* #param batchSize Size of each minibatch for training
* #param truncateLength If reviews exceed
* #param train If true: return the training data. If false: return the testing data.
*/
public SentimentExampleIterator(String dataDirectory, WordVectors wordVectors, int batchSize, int truncateLength, boolean train) throws IOException {
this.batchSize = batchSize;
this.vectorSize = wordVectors.lookupTable().layerSize();
File p = new File(FilenameUtils.concat(dataDirectory, "learning/" + (train ? "train" : "test")) + "/");
filePathes = p.listFiles();
this.wordVectors = wordVectors;
this.truncateLength = truncateLength;
tokenizerFactory = new DefaultTokenizerFactory();
tokenizerFactory.setTokenPreProcessor(new CommonPreprocessor());
}
#Override
public DataSet next(int num) {
if (cursor >= filePathes.length) throw new NoSuchElementException();
try{
return nextDataSet(num);
}catch(IOException e){
throw new RuntimeException(e);
}
}
private DataSet nextDataSet(int num) throws IOException {
List<VacancyLightEntity> vacancies = new ArrayList<>(num);
boolean[] positive = new boolean[num];
for( int i=0; i<num && cursor<totalExamples(); i++ ){
String path = filePathes[cursor].getAbsolutePath();
vacancies.add(Reader.readVacancyFromFile(path));
cursor++;
}
//Second: tokenize vacancies and filter out unknown words
List<List<String>> allTokens = new ArrayList<>(vacancies.size());
int maxLength = 0;
for(VacancyLightEntity v : vacancies){
List<String> tokens = tokenizerFactory.create(v.getText()).getTokens();
List<String> tokensFiltered = new ArrayList<>();
for(String t : tokens ){
if(wordVectors.hasWord(t)) tokensFiltered.add(t);
}
allTokens.add(tokensFiltered);
maxLength = Math.max(maxLength,tokensFiltered.size());
}
//If longest review exceeds 'truncateLength': only take the first 'truncateLength' words
if(maxLength > truncateLength) maxLength = truncateLength;
//Create data for training
//Here: we have vacancies.size() examples of varying lengths
INDArray features = Nd4j.create(vacancies.size(), vectorSize, maxLength);
INDArray labels = Nd4j.create(vacancies.size(), labelsCount, maxLength); //Two labels: positive or negative
//Because we are dealing with vacancies of different lengths and only one output at the final time step: use padding arrays
//Mask arrays contain 1 if data is present at that time step for that example, or 0 if data is just padding
INDArray featuresMask = Nd4j.zeros(vacancies.size(), maxLength);
INDArray labelsMask = Nd4j.zeros(vacancies.size(), maxLength);
int[] temp = new int[2];
for( int i=0; i<vacancies.size(); i++ ){
List<String> tokens = allTokens.get(i);
temp[0] = i;
//Get word vectors for each word in review, and put them in the training data
for( int j=0; j<tokens.size() && j<maxLength; j++ ){
String token = tokens.get(j);
INDArray vector = wordVectors.getWordVectorMatrix(token);
features.put(new INDArrayIndex[]{NDArrayIndex.point(i), NDArrayIndex.all(), NDArrayIndex.point(j)}, vector);
temp[1] = j;
featuresMask.putScalar(temp, 1.0); //Word is present (not padding) for this example + time step -> 1.0 in features mask
}
int idx = (positive[i] ? 0 : 1);
int lastIdx = Math.min(tokens.size(),maxLength);
labels.putScalar(new int[]{i,idx,lastIdx-1},1.0); //Set label: [0,1] for negative, [1,0] for positive
labelsMask.putScalar(new int[]{i,lastIdx-1},1.0); //Specify that an output exists at the final time step for this example
}
return new DataSet(features,labels,featuresMask,labelsMask);
}
#Override
public int totalExamples() {
return filePathes.length;
}
#Override
public int inputColumns() {
return vectorSize;
}
#Override
public int totalOutcomes() {
return 2;
}
#Override
public void reset() {
cursor = 0;
}
#Override
public int batch() {
return batchSize;
}
#Override
public int cursor() {
return cursor;
}
#Override
public int numExamples() {
return totalExamples();
}
#Override
public void setPreProcessor(DataSetPreProcessor preProcessor) {
throw new UnsupportedOperationException();
}
#Override
public List<String> getLabels() {
return Arrays.asList("positive","negative");
}
#Override
public boolean hasNext() {
return cursor < numExamples();
}
#Override
public DataSet next() {
return next(batchSize);
}
#Override
public void remove() {
}
}
So I have a program that stores dates.
This is the function that counts the number of dates in the file.
public static int getLineCount() throws FileNotFoundException {
System.out.println("Line count called");
int datenumber = 0;
int attrnumber = 0;
try {
Scanner readLine = new Scanner(new FileReader(dates));
readLine.useDelimiter("<>");
while (readLine.hasNext()) {
readLine.next();
if (attrnumber == 3) {
datenumber++;
attrnumber = 0;
} else {
attrnumber++;
}
}
} catch (IOException e) {
System.out.print("Cannot read file!");
}
;
System.out.println("Line count is: " + datenumber);
return datenumber;
}
This is how the file looks
14<>7<>2014<>sdfsdf<>14<>4<>2016<>asdas<>
The Format is DAY<>MONTH<>YEAR<>NAME<>
Lets say I have 3 dates store and I want to delete the second date, how could I accomplish this ?
Could I use the same code here but instead once it gets to the position that I want to delete, it will somehow remove the data ?
I have
DAY1<>MONTH1<>YEAR1<>NAME1<>DAY2<>MONTH2<>YEAR2<>NAME2<>DAY3<>MONTH3<>YEAR3<>NAME3<>
I want
DAY1<>MONTH1<>YEAR1<>NAME1<>DAY3<>MONTH3<>YEAR3<>NAME3<>
public static void main(String args[]){
String str = "DAY1<>MONTH1<>YEAR1<>NAME1<>DAY2<>MONTH2<>YEAR2<>NAME2<>DAY3<>MONTH3<>YEAR3<>NAME3<>";
System.out.println(remove(str, "<>", 4, 2));
}
/**
*
* #param sentence your main sentence
* #param delimeter delimeter which sentence contains like <>
* #param range range of line, for this example as you can see each row contains 4 items, so its range is 4
* #param removed_range, whing range is removed, for example 2 range etc
* #return
*/
public static String remove(String sentence, String delimeter, int range, int removed_range){
String str= "";
ArrayList<String> items = new ArrayList<>();
if(sentence.contains(delimeter)){
items.addAll(Arrays.asList(sentence.split(delimeter)));
}
int cnt = 0;
for(String item : items){
if(cnt<range || cnt>= removed_range*range)
str+=item+delimeter;
cnt++;
}
return str;
}
result:
DAY1<>MONTH1<>YEAR1<>NAME1<>DAY3<>MONTH3<>YEAR3<>NAME3<>