Develop Reference

Input already created object via Scanner and do something with it

I want to know if it is possible to write an object name in an input(Scanner or maybe another way?) through the console and then do something with it.
package test;
import java.util.ArrayList;
import java.util.Scanner;
public class Test {
public static void main(String[] args) {
Scanner input = new Scanner(System.in);
Person p1 = new Person(1, "pedro");
Person p2 = new Person(2, "juan");
Person p3 = new Person(3, "diego");
ArrayList personList = new ArrayList();
personList.add(p1);
personList.add(p2);
//Something like this doesn't work :S
Person personToAdd = input.nextPerson();
personList.add(personToAdd);
}
}
for example in this case I wanna input the object name "p3" and then add it to the ArrayList, is there any way to do something like this?

List<Person> personList = new ArrayList<Person>();
personList.add(p1);
personList.add(p2);
// you need to get each field you want to save with scanner
// nextLine/nextInt and so on to set other field
// e.q newPerson.setAge(<value from scanner>)
String personName = input.nexLine();
Person newPerson = new Person(personName);
personList.add(newPerson);

How to get the field name of a primitive object inspector in java

I am trying to solve this problem for an UDF I am creating for hiveql environment.
public ObjectInspector initialize(ObjectInspector[] arguments)
throws UDFArgumentException {
if (arguments.length != 1) {
throw new UDFArgumentException("Usage : multiple_prop(primitive var) ");
}
// This will be an string
moi = (PrimitiveObjectInspector) arguments[0];
ArrayList structFieldNames = new ArrayList();
ArrayList structFieldObjectInspectors = new ArrayList();
structFieldNames.add("fields name"); <-- Issue is here
How could I do to get the field name in there? It can be easily done for structObjectInspectors, but how do we manage this in PrimitiveObjectInspectors?
Complete code would be this one
public class prop_step2 extends GenericUDF {
private PrimitiveObjectInspector moi;
#Override
public ObjectInspector initialize(ObjectInspector[] arguments)
throws UDFArgumentException {
if (arguments.length != 1) {
throw new UDFArgumentException("Usage : multiple_prop(primitive var) ");
}
// This will be an string
moi = (PrimitiveObjectInspector) arguments[0];
ArrayList structFieldNames = new ArrayList();
ArrayList structFieldObjectInspectors = new ArrayList();
// Change this to get the input variable name, and not the type name
structFieldNames.add(moi.getTypeName());<-- Change this to field name
structFieldObjectInspectors.add( PrimitiveObjectInspectorFactory.writableStringObjectInspector );
return ObjectInspectorFactory.getStandardStructObjectInspector(structFieldNames, structFieldObjectInspectors);
}
#Override
public Object evaluate(DeferredObject[] arguments) throws HiveException {
Object[] result;
result = new Object[1];
Text elem1 = new Text((String) moi.getPrimitiveJavaObject(arguments[0].get()));
result[0]= elem1;
return result;
}
#Override
public String getDisplayString(String[] children) {
return "stop";
}}
When this would be finished, i would like to call this udf from hive:
CREATE TEMPORARY FUNCTION step AS 'UDFpack.prop_step2';
select
step(bit) as sd
from my_table
And i would expect that if in an upper select i did this : sd.bit i would obtain the value of 'bit'.

It's simply not possible. The information passed to the UDF - the ObjectInspectors - do not contain their name. That's why you can see the output column names being changed to _col0, _col1 .. in the intermediary stages of a Hive explain plan. I am also quite annoyed by this and think this is an oversight by Hive.
A workaround would be to put your input into a struct and parse that.
i.e step(named_struct('bit',bit)) and then you can get the field name of the struct in your UDF. But it's not nearly as nice

How to pass csv mapped bean class to Dataset

I wrote code to read a csv file and map all the columns to a bean class.
Now, I'm trying to set these values to a Dataset and getting an issue.
7/08/30 16:33:58 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.IllegalArgumentException: object is not an instance of declaring class
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
If I try to set the values manually it works fine
public void run(String t, String u) throws FileNotFoundException {
JavaRDD<String> pairRDD = sparkContext.textFile("C:/temp/L1_result.csv");
JavaPairRDD<String,String> rowJavaRDD = pairRDD.mapToPair(new PairFunction<String, String, String>() {
public Tuple2<String,String> call(String rec) throws FileNotFoundException {
String[] tokens = rec.split(";");
String[] vals = new String[tokens.length];
for(int i= 0; i < tokens.length; i++){
vals[i] =tokens[i];
}
return new Tuple2<String, String>(tokens[0], tokens[1]);
}
});
ColumnPositionMappingStrategy cpm = new ColumnPositionMappingStrategy();
cpm.setType(funds.class);
String[] csvcolumns = new String[]{"portfolio_id", "portfolio_code"};
cpm.setColumnMapping(csvcolumns);
CSVReader csvReader = new CSVReader(new FileReader("C:/temp/L1_result.csv"));
CsvToBean csvtobean = new CsvToBean();
List csvDataList = csvtobean.parse(cpm, csvReader);
for (Object dataobject : csvDataList) {
funds fund = (funds) dataobject;
System.out.println("Portfolio:"+fund.getPortfolio_id()+ " code:"+fund.getPortfolio_code());
}
/* funds b0 = new funds();
b0.setK("k0");
b0.setSomething("sth0");
funds b1 = new funds();
b1.setK("k1");
b1.setSomething("sth1");
List<funds> data = new ArrayList<funds>();
data.add(b0);
data.add(b1);*/
System.out.println("Portfolio:" + rowJavaRDD.values());
//manual set works fine ///
// Dataset<Row> fundDf = SQLContext.createDataFrame(data, funds.class);
Dataset<Row> fundDf = SQLContext.createDataFrame(rowJavaRDD.values(), funds.class);
fundDf.printSchema();
fundDf.write().option("mergeschema", true).parquet("C:/test");
}
The line below is giving an issue: using rowJavaRDD.values():
Dataset<Row> fundDf = SQLContext.createDataFrame(rowJavaRDD.values(), funds.class);
what is the resolution to this? whatever values Im column mapping should be passed here, but how this needs to be done. Any idea really helps me.

Dataset fundDf = SQLContext.createDataFrame(csvDataList, funds.class);
Passing list worked!

Map array to object

I have an array defined as follows:
String [] source = {"26", "Tom", "foo", ...};
And a Person class:
public class Person{
private String age;
private String name;
private String print;
private String ......;//the same type and order and number of source
public Person() {
}
//full construtors
public Person(String age, String name, String print,String ....) {
this.age = age;
this.name = name;
this.print = print;
//....
}
/* setters & getters */
}
How can I map these values to a Personinstance?
this is my real coding
public static List<BasicalVo> readObject(String path) throws IOException, NoSuchMethodException {
InputStreamReader fReader = new InputStreamReader(new FileInputStream(path),"gb2312");
BufferedReader bufferedReader = new BufferedReader(fReader);
String currentLine;
String[] temp;
List<BasicalVo> basicalVoList= new ArrayList<BasicalVo>();
while ((currentLine = bufferedReader.readLine()) != null) {
temp = currentLine.split(",");//I get the Array
for (int i = 0; i < temp.length; i++) {
//I don't know hot to translate to BasicalVo .
BasicalVo vo = new BasicalVo();
basicalVoList.add(vo);
}
}
return basicalVoList;
}

If the source just contains one person's data then you can do this:
Person p = new Person(source[0], source[1], source[2] ...);
If the array is too short an ArrayIndexOutOfBoundsException will be thrown.

I.
If the array contains only one Person you only need to create the instance like this :
String[] source = new String[]{"26", "tom", "xx", "....."};
Person p = new Person(source[0], source[1], source[2], source[3],...);
Because you'll know how many parameters there is in the constructor and so you won't have an ArrayIndexOutOfBoundsException if the array is well-build
II.
Assuming you have only 3 attributes, you'll be able to do like this if the array is like this :
String[] source = new String[]{"26", "tom", "xx", "22", "john", "yy"};
ArrayList<Person> list = new ArrayList<>()
for (int i = 0; i < source.length; i += 3) {
list.add(new Person(source[i], source[i + 1], source[i + 2]));
}
III.
If you have multiple fields, you would better do like this :
public Person(String[]source) {
this.age = source[0];
this.name = source[1];
this.print = source[2];
//....
}
Because it wouldn't not surcharge the code you have in your loop which read from the data, and make it easier to do your stuff, and in fact this is not hard, because in every case if you have like 20 fields, you'll have to assignate these 20 attributs
IV.
Or last proposition with a factory method :
public static Person createPersoneFromArray(String[] array) {
Person p = new Person();
p.setAge(array[0]);
p.setName(array[1]);
//...
return p;
}
And in the main method :
Person p = Person.createPersoneFromArray(source);

you can also add another constructor to your BasicalVo class which takes a String[] as input :
public BasicalVo(String [] input) {
this.age = input[0];
this.name = input[1];
this.print = input[2];
//....
}
which you then can call in your main as follows without additional for loop
....
temp = currentLine.split(",");
BasicalVo vo = new BasicalVo(temp);
basicalVoList.add(vo);
....

You can use OpenCSV
CSVReader csvReader = new CSVReader(new FileReader("people.csv"),',');
ColumnPositionMappingStrategy mappingStrategy = new ColumnPositionMappingStrategy();
mappingStrategy.setType(Person.class);
String[] columns = new String[]{"age","name","print"};
mappingStrategy.setColumnMapping(columns);
CsvToBean ctb = new CsvToBean();
List personList = ctb.parse(mappingStrategy, csvReader);

In this specific case I think that your best option is to use reflection. Reflection are a set of classes and interfaces that allow you to call different methods at execution time. For instance:
String [] source = { "26", "tom", "xx", ... };
Constructor constructor = Person.class.getConstructors()[0]
constructor.newInstance(source)
Take into account that this example only works because you have only one constructor, and so Person.class.getConstructors()[0] returns the constructor you want. YOu can try to get the specific constructor with Person.class.getConstructors(Class<?>...), in that case you would need to pass as a parameter an array with the type of the arguments.

incompatible type of double array and properties string.split()

public static void main(String[] args)
{
String input="jack=susan,kathy,bryan;david=stephen,jack;murphy=bruce,simon,mary";
String[][] family = new String[50][50];
//assign family and children to data by ;
StringTokenizer p = new StringTokenizer (input,";");
int no_of_family = input.replaceAll("[^;]","").length();
no_of_family++;
System.out.println("family= "+no_of_family);
String[] data = new String[no_of_family];
int i=0;
while(p.hasMoreTokens())
{
data[i] = p.nextToken();
i++;
}
for (int j=0;j<no_of_family;j++)
{
family[j][0] = data[j].split("=")[0];
//assign child to data by commas
StringTokenizer v = new StringTokenizer (data[j],",");
int no_of_child = data[j].replaceAll("[^,]","").length();
no_of_child++;
System.out.println("data from input = "+data[j]);
for (int k=1;k<=no_of_child;k++)
{
family[j][k]= data[j].split("=")[1].split(",");
System.out.println(family[j][k]);
}
}
}
i have a list of family in input string and i seperate into a family and i wanna do it in double array family[i][j].
my goal is:
family[0][0]=1st father's name
family[0][1]=1st child name
family[0][2]=2nd child name and so on...
family[0][0]=jack
family[0][1]=susan
family[0][2]=kathy
family[0][3]=bryan
family[1][0]=david
family[1][1]=stephen
family[1][2]=jack
family[2][0]=murphy
family[2][1]=bruce
family[2][2]=simon
family[2][3]=mary
but i got the error as title: in compatible types
found:java.lang.String[]
required:java.lang.String
family[j][k]= data[j].split("=")[1].split(",");
what can i do?i need help
nyone know how to use StringTokenizer for this input?

Trying to understand why you can't just use split for your nested operation as well.
For example, something like this should work just fine
for (int j=0;j<no_of_family;j++)
{
String[] familySplit = data[j].split("=");
family[j][0] = familySplit[0];
String[] childrenSplit = familySplit[1].split(",");
for (int k=0;k<childrenSplit.length;k++)
{
family[j][k+1]= childrenSplit[k];
}
}

You are trying to assign an array of strings to a string. Maybe this will make it more clear?
String[] array = data.split("=")[1].split(",");
Now, if you want the first element of that array you can then do:
family[j][k] = array[0];

I always avoid to use arrays directly. They are hard to manipulate versus dynamic list. I implemented the solution using a Map of parent to a list of childrens Map<String, List<String>> (read Map<Parent, List<Children>>).
public static void main(String[] args) {
String input = "jack=susan,kathy,bryan;david=stephen,jack;murphy=bruce,simon,mary";
Map<String, List<String>> parents = new Hashtable<String, List<String>>();
for ( String family : input.split(";")) {
final String parent = family.split("=")[0];
final String allChildrens = family.split("=")[1];
List<String> childrens = new Vector<String>();
for (String children : allChildrens.split(",")) {
childrens.add(children);
}
parents.put(parent, childrens);
}
System.out.println(parents);
}
The output is this:
{jack=[susan, kathy, bryan], murphy=[bruce, simon, mary], david=[stephen, jack]}
With this method you can directory access to a parent using the map:
System.out.println(parents.get("jack"));
and this output:
[susan, kathy, bryan]