Develop Reference

Apply LOOCV in java splitting with a specific condition

I have a csv file containing 24231 rows. I would like to apply LOOCV based on the project name instead of the observations of the whole dataset.
So if my dataset contains information for 15 projects, I would like to have the training set based on 14 projects and the test set based on the other project.
I was relying on weka's API, is there anything that automates this process?

For non-numeric attributes, Weka allows you to retrieve the unique values via Attribute.numValues() (how many are there) and Attribute.value(int) (the -th value).
package weka;
import weka.core.Attribute;
import weka.core.Instance;
import weka.core.Instances;
import weka.core.converters.ConverterUtils;
public class LOOByValue {
/**
* 1st arg: ARFF file to load
* 2nd arg: 0-based index in ARFF to use for class
* 3rd arg: 0-based index in ARFF to use for LOO
*
* #param args the command-line arguments
* #throws Exception if loading/processing of data fails
*/
public static void main(String[] args) throws Exception {
// load data
Instances full = ConverterUtils.DataSource.read(args[0]);
full.setClassIndex(Integer.parseInt(args[1]));
int looCol = Integer.parseInt(args[2]);
Attribute looAtt = full.attribute(looCol);
if (looAtt.isNumeric())
throw new IllegalStateException("Attribute cannot be numeric!");
// iterate unique values to create train/test splits
for (int i = 0; i < looAtt.numValues(); i++) {
String value = looAtt.value(i);
System.out.println("\n" + (i+1) + "/" + full.attribute(looCol).numValues() + ": " + value);
Instances train = new Instances(full, full.numInstances());
Instances test = new Instances(full, full.numInstances());
for (int n = 0; n < full.numInstances(); n++) {
Instance inst = full.instance(n);
if (inst.stringValue(looCol).equals(value))
test.add((Instance) inst.copy());
else
train.add((Instance) inst.copy());
}
train.compactify();
test.compactify();
// TODO do something with the data
System.out.println("train size: " + train.numInstances());
System.out.println("test size: " + test.numInstances());
}
}
}
With Weka's anneal UCI dataset and the surface-quality for leave-one-out, you can generate something like this:
1/5: ?
train size: 654
test size: 244
2/5: D
train size: 843
test size: 55
3/5: E
train size: 588
test size: 310
4/5: F
train size: 838
test size: 60
5/5: G
train size: 669
test size: 229

Is it possible to create a list in java using data from multiple text files

I have multiple text files that contains information about different programming languages popularity in different countries based off of google searches. I have one text file for each year from 2004 to 2015. I also have a text file that breaks this down into each week (called iot.txt) but this file does not include the country.
Example data from 2004.txt:
Region java c++ c# python JavaScript
Argentina 13 14 10 0 17
Australia 22 20 22 64 26
Austria 23 21 19 31 21
Belgium 20 14 17 34 25
Bolivia 25 0 0 0 0
etc
example from iot.txt:
Week java c++ c# python JavaScript
2004-01-04 - 2004-01-10 88 23 12 8 34
2004-01-11 - 2004-01-17 88 25 12 8 36
2004-01-18 - 2004-01-24 91 24 12 8 36
2004-01-25 - 2004-01-31 88 26 11 7 36
2004-02-01 - 2004-02-07 93 26 12 7 37
My problem is that i am trying to write code that will output the number of countries that have exhibited 0 interest in python.
This is my current code that I use to read the text files. But I'm not sure of the best way to tell the number of regions that have 0 interest in python across all the years 2004-2015. At first I thought the best way would be to create a list from all the text files not including iot.txt and then search that for any entries that have 0 interest in python but I have no idea how to do that.
Can anyone suggest a way to do this?
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.*;
public class Starter{
public static void main(String[] args) throws Exception {
BufferedReader fh =
new BufferedReader(new FileReader("iot.txt"));
//First line contains the language names
String s = fh.readLine();
List<String> langs =
new ArrayList<>(Arrays.asList(s.split("\t")));
langs.remove(0); //Throw away the first word - "week"
Map<String,HashMap<String,Integer>> iot = new TreeMap<>();
while ((s=fh.readLine())!=null)
{
String [] wrds = s.split("\t");
HashMap<String,Integer> interest = new HashMap<>();
for(int i=0;i<langs.size();i++)
interest.put(langs.get(i), Integer.parseInt(wrds[i+1]));
iot.put(wrds[0], interest);
}
fh.close();
HashMap<Integer,HashMap<String,HashMap<String,Integer>>>
regionsByYear = new HashMap<>();
for (int i=2004;i<2016;i++)
{
BufferedReader fh1 =
new BufferedReader(new FileReader(i+".txt"));
String s1 = fh1.readLine(); //Throw away the first line
HashMap<String,HashMap<String,Integer>> year = new HashMap<>();
while ((s1=fh1.readLine())!=null)
{
String [] wrds = s1.split("\t");
HashMap<String,Integer>langMap = new HashMap<>();
for(int j=1;j<wrds.length;j++){
langMap.put(langs.get(j-1), Integer.parseInt(wrds[j]));
}
year.put(wrds[0],langMap);
}
regionsByYear.put(i,year);
fh1.close();
}
}
}

Create a Map<String, Integer> using a HashMap and each time you find a new country while scanning the incoming data add it into the map country->0. Each time you find a usage of python increment the value.
At the end loop through the entrySet of the map and for each case where e.value() is zero output e.key().

Java text not formatting properly

I just can't seem to figure out how to get java to format my output properly. The code snipet for output is: System.out.format("\n %1s%20s%25s", intDf.format(i), df.format(monthlyPayment), df.format(totalPayment)); the output appears like this:
Interest Rate Monthly Payment Total Payment
5 188.71 11322.74
5.125 189.29 11357.13
5.25 189.86 11391.59
5.375 190.44 11426.11
5.5 191.01 11460.7
5.625 191.59 11495.35
5.75 192.17 11530.06
5.875 192.75 11564.84
6 193.33 11599.68
6.125 193.91 11634.59
6.25 194.49 11669.56
6.375 195.08 11704.59
6.5 195.66 11739.69
6.625 196.25 11774.85
6.75 196.83 11810.08
6.875 197.42 11845.37
7 198.01 11880.72
7.125 198.6 11916.14
7.25 199.19 11951.62
7.375 199.79 11987.16
7.5 200.38 12022.77
7.625 200.97 12058.44
7.75 201.57 12094.18
7.875 202.17 12129.97
8 202.76 12165.84
I want all the values to line up at the first letter in their corresponding line. Any help is appreciated, thanks!

Clearly your first column is more then one character wide. So when you say
System.out.format("\n %1s
That throws it off. The column name "Interest Rate" is 13 characters. So try
System.out.format("\n %14s
Edit
Based on your comment, I would use a DecimalFormat and a NumberFormat like
String[] headings = { "Interest Rate", "Monthly Payment",
"Total Payment" };
for (String heading : headings) {
System.out.print(heading);
System.out.print("\t");
}
System.out.println();
double[] interestRates = { 5, 5.125 };
double[] monthlyPayments = { 188.71, 189.29 };
double[] totalPayments = { 11322.74, 11357.13 };
NumberFormat interestFormat = new DecimalFormat("#.000'%'");
NumberFormat currencyFormat = NumberFormat.getCurrencyInstance();
for (int i = 0; i < interestRates.length; i++) {
System.out.printf("%-13s\t%-15s\t%-13s%n",
interestFormat.format(interestRates[i]),
currencyFormat.format(monthlyPayments[i]),
currencyFormat.format(totalPayments[i]));
}
Output is
Interest Rate Monthly Payment Total Payment
5.000% $188.71 $11,322.74
5.125% $189.29 $11,357.13

I changed it from
System.out.format("\n %1s%20s%25s", intDf.format(i), df.format(monthlyPayment), df.format(totalPayment));
to System.out.format("\n %-5s%20s%25s", intDf.format(i), df.format(monthlyPayment), df.format(totalPayment));
then changed the DecimalFormat to: "'$'0.00"

How to read the contents of (.bib) file format using Java

I need to read .bib file and insert it tags into an objects of bib-entries
the file is big (almost 4000 lines) , so my first question is what to use (bufferrReader or FileReader)
the general format is
#ARTICLE{orleans01DJ,
author = {Doug Orleans and Karl Lieberherr},
title = {{{DJ}: {Dynamic} Adaptive Programming in {Java}}},
journal = {Metalevel Architectures and Separation of Crosscutting Concerns 3rd
Int'l Conf. (Reflection 2001), {LNCS} 2192},
year = {2001},
pages = {73--80},
month = sep,
editor = {A. Yonezawa and S. Matsuoka},
owner = {Administrator},
publisher = {Springer-Verlag},
timestamp = {2009.03.09}
}
#ARTICLE{Ossher:1995:SOCR,
author = {Harold Ossher and Matthew Kaplan and William Harrison and Alexander
Katz},
title = {{Subject-Oriented Composition Rules}},
journal = {ACM SIG{\-}PLAN Notices},
year = {1995},
volume = {30},
pages = {235--250},
number = {10},
month = oct,
acknowledgement = {Nelson H. F. Beebe, University of Utah, Department of Mathematics,
110 LCB, 155 S 1400 E RM 233, Salt Lake City, UT 84112-0090, USA,
Tel: +1 801 581 5254, FAX: +1 801 581 4148, e-mail: \path|beebe#math.utah.edu|,
\path|beebe#acm.org|, \path|beebe#computer.org| (Internet), URL:
\path|http://www.math.utah.edu/~beebe/|},
bibdate = {Fri Apr 30 12:33:10 MDT 1999},
coden = {SINODQ},
issn = {0362-1340},
keywords = {ACM; object-oriented programming systems; OOPSLA; programming languages;
SIGPLAN},
owner = {Administrator},
timestamp = {2009.02.26}
}
As you can see , there are some entries that have more than line, entries that end with }
entries that end with }, or }},
Also , some entries have {..},{..}.. in the middle
so , i am a little bit confused on how to start reading this file and how to get these entries and manipulate them.
Any help will be highly appreciated.

We currently discuss different options at JabRef.
These are the current options:
JBibTeX
ANTLRv3 Grammar
JabRef's BibtexParser.java

unable to read modbus data using jamod 2( wimpi.modbus )

i am new to java serial port programming​,
I have trying to read data from the modbus slave device through modbus RTU over serial port from my java application.
I am using the Jamod java library to read modbus protocal.
In my case my application failed to receive entire modbus response from the device. please find my java coding and error log for your reference.
Any one can suggest me what may be the reason for the error.
<br/>
**ERROR**<br/>
Clear input: 02 c2 c1<br/>
Sent: 01 04 03 e8 00 05 b0 79 <br/>
Last request: 01 04 03 e8 00 05 b0 79<br/>
CRC Error in received frame: 0 bytes: <br/>
Response: 01 84 <br/>
net.wimpi.modbus.ModbusIOException: I/O exception - failed to read<br/>
at net.wimpi.modbus.io.ModbusRTUTransport.readResponse(ModbusRTUTransport.java:163)<br/>
at net.wimpi.modbus.io.ModbusSerialTransaction.execute(ModbusSerialTransaction.java:187)<br/>
at modbusnewapplication.ModbusConnection.main(ModbusConnection.java:8<br/>
Modbus Program <br/>
---------------<br/>
package modbusnewapplication;<br/>
import java.io.;<br/>
import javax.comm.;<br/>
import net.wimpi.modbus.ModbusCoupler;<br/>
import net.wimpi.modbus.io.ModbusSerialTransaction;<br/>
import net.wimpi.modbus.msg.ReadInputRegistersRequest;<br/>
import net.wimpi.modbus.msg.ReadInputRegistersResponse;<br/>
import net.wimpi.modbus.net.SerialConnection;<br/>
import net.wimpi.modbus.util.SerialParameters;<br/>
public class ModbusConnection {<br/>
public static void main(String[] args) {<br/>
// if (args.length < 4) {<br/>
// System.out.println("not enough args");<br/>
// System.exit(1);<br/>
// }else{<br/>
try {<br/>
System.out.println("Serial Port Connection");<br/><br/>
/* The important instances of the classes mentioned before */<br/>
SerialConnection con = null; //the connection<br/>
ModbusSerialTransaction trans = null; //the transaction<br/>
ReadInputRegistersRequest req = null; //the request<br/>
ReadInputRegistersResponse res = null; //the response<br/>
// **1 Variables for storing the parameters** <br/>
String portname= "COM1"; //the name of the serial port to be used<br/>
int unitid = 1; //the unit identifier we will be talking to<br/>
int ref = 1000; //the reference, where to start reading from<br/>
int count = 5; //the count of IR's to read<br/>
int repeat = 1; //a loop for repeating the transaction <br/>
boolean isopen = false;<br/><br/>
**// 2. Set master identifier**
// ModbusCoupler.createModbusCoupler(null);
// ModbusCoupler.getReference().setMaster(master); I added this in
// ModbusCoupler.getReference().setMaster(true);
// ModbusCoupler.getReference().setUnitID(1);
**// 3. Setup serial parameters**<br/>
SerialParameters params = new SerialParameters();<br/>
params.setPortName("COM1");<br/>
params.setBaudRate(9600);<br/>
params.setDatabits(8);<br/>
params.setParity("None");<br/>
params.setStopbits(1);<br/>
params.setEncoding("RTU");<br/>
params.setEcho(false);<br/>
System.setProperty("net.wimpi.modbus.debug", "true");<br/>
**// 4. Open the connection**<br/>
con = new SerialConnection(params);
System.out.println("Connection..." + con.toString());
con.open();
isopen = con.isOpen();<br/>
System.out.println("Serial port status..." + isopen);<br/>
**// 5. Prepare a request<br/>**
req = new ReadInputRegistersRequest(ref, count);<br/>
req.setUnitID(unitid);<br/>
req.setHeadless();<br/>
**// 6. Prepare a transaction<br/>**
trans = new ModbusSerialTransaction(con);<br/>
trans.setRequest(req);<br/>
**// 7. Execute the transaction repeat times<br/>**
int k = 0;<br/>
do { <br/>
trans.execute();<br/>
res = (ReadInputRegistersResponse) trans.getResponse();<br/>
for (int n = 0; n < res.getWordCount(); n++) {<br/>
System.out.println("Word " + n + "=" + res.getRegisterValue(n));<br/>
}<br/>
k++;<br/>
} while (k < repeat);<br/>
**// 8. Close the connection**<br/>
con.close();<br/>
} catch (Exception ex) {<br/>
ex.printStackTrace();<br/>
}<br/>
//}//else<br/>
}//main
}

You should add a method like Thread.sleep(500) between request and response method in jamod library. Maybe this kind of error is usually caused by the length of response data.
jamod library didn't consider the long size of response data. So, we need to request and wait until all data is received from the serial interface. Ff not, because all data is not received, CRC check will fail and cause an error.