I am importing a csv file. I have a cell value something like 1.00E+13 while importing which has to be read as 10023000000000 (actual expanded value).
I am using OpenCSV.
Note: I achieved this by using apache POI for XLS and XLSX files, Since POI doesn't support CSV I am using OpenCSV.
POI's solution to above scenario is:
FormulaEvaluator objFormulaEvaluator = new HSSFFormulaEvaluator(hWorkbook);
HSSFCell cellValue = row.getCell(1); //input as 1.00E+13
objFormulaEvaluator.evaluate(cellValue); //result as 10023000000000
How do I achieve this in OpenCSV?
Thanks in advance.
Kenny's answer is dead on if you are parsing the Strings from the csvReader yourself. My only concern is that the csv file is strings so the input will have to be 1.0023E+13 not 1.00E+13.
All that said there is another way in opencsv and that is the CsvToBean class. You create a class that has the values you want with public getters and setters then the CsvToBean will handle the conversions for you as the passing test below showed.
public class CsvToBeanDoubleTest {
private static final double DOUBLE_NUMBER = 10023000000000d;
private static final String TEST_STRING = "name,orderNumber,doubleNum\n" +
"kyle,abc123456,10023000000000\n" +
"jimmy,def098765,1.0023E+13 ";
private CSVReader createReader() {
return createReader(TEST_STRING);
}
private CSVReader createReader(String testString) {
StringReader reader = new StringReader(testString);
return new CSVReader(reader);
}
private MockBean createMockBean(String name, String orderNumber, double num) {
MockBean mockBean = new MockBean();
mockBean.setName(name);
mockBean.setOrderNumber(orderNumber);
mockBean.setDoubleNum(num);
return mockBean;
}
#Test
public void parseBeanWithNoAnnotations() {
HeaderColumnNameMappingStrategy<MockBean> strategy = new HeaderColumnNameMappingStrategy<MockBean>();
strategy.setType(MockBean.class);
CsvToBean<MockBean> bean = new CsvToBean<MockBean>();
List<MockBean> beanList = bean.parse(strategy, createReader());
assertEquals(2, beanList.size());
assertTrue(beanList.contains(createMockBean("kyle", "abc123456", DOUBLE_NUMBER)));
assertTrue(beanList.contains(createMockBean("jimmy", "def098765", DOUBLE_NUMBER)));
}
}
Related
I have to map particular CSV column based on index with particular POJO attributes. Mapping will be based on a json file which will contain columnIndex and attribute name which means that for a particular columnIndex from csv file you have to map particular attribute from Pojo class.
Below is a sample of json file which shows column mapping strategy with Pojo attributes.
[{"index":0,"columnname":"date"},{"index":1,"columnname":"deviceAddress"},{"index":7,"columnname":"iPAddress"},{"index":3,"columnname":"userName"},{"index":10,"columnname":"group"},{"index":5,"columnname":"eventCategoryName"},{"index":6,"columnname":"message"}]
I have tried with OpenCSV library but the challenges which i faced with that I am not able to read partial column with it. As in above json you can see that we are skipping index 2 and 4 to read from CSV file. Below is the code with openCSV file.
public static List<BaseDataModel> readCSVFile(String filePath,List<String> columnListBasedOnIndex) {
List<BaseDataModel> csvDataModels = null;
File myFile = new File(filePath);
try (FileInputStream fis = new FileInputStream(myFile)) {
final ColumnPositionMappingStrategy<BaseDataModel> strategy = new ColumnPositionMappingStrategy<BaseDataModel>();
strategy.setType(BaseDataModel.class);
strategy.setColumnMapping(columnListBasedOnIndex.toArray(new String[0]));
final CsvToBeanBuilder<BaseDataModel> beanBuilder = new CsvToBeanBuilder<>(new InputStreamReader(fis));
beanBuilder.withMappingStrategy(strategy);
csvDataModels = beanBuilder.build().parse();
} catch (Exception e) {
e.printStackTrace();
}
}
List<ColumnIndexMapping> columnIndexMappingList = dataSourceModel.getColumnMappingStrategy();
List<String> columnNameList = columnIndexMappingList.stream().map(ColumnIndexMapping::getColumnname)
.collect(Collectors.toList());
List<BaseDataModel> DataModels = Utility
.readCSVFile(file.getAbsolutePath() + File.separator + fileName, columnNameList);
I have also tried with univocity but with this library how can i map csv with particular attributes. Below is the code -
CsvParserSettings settings = new CsvParserSettings();
settings.detectFormatAutomatically(); //detects the format
settings.getFormat().setLineSeparator("\n");
//extracts the headers from the input
settings.setHeaderExtractionEnabled(true);
settings.selectIndexes(0, 2); //rows will contain only values of columns at position 0 and 2
CsvRoutines routines = new CsvRoutines(settings); // Can also use TSV and Fixed-width routines
routines.parseAll(BaseDataModel.class, new File("/path/to/your.csv"));
List<String[]> rows = new CsvParser(settings).parseAll(new File("/path/to/your.csv"), "UTF-8");
Please have a look if someone can help me in this case.
Author of univocity-parsers here. You can define mappings to your class attributes in code instead of annotations. Something like this:
public class BaseDataModel {
private String a;
private int b;
private String c;
private Date d;
}
Then on your code, map the attributes to whatever column names you need:
ColumnMapper mapper = routines.getColumnMapper();
mapper.attributeToColumnName("a", "col1");
mapper.attributeToColumnName("b", "col2");
mapper.attributeToColumnName("c", "col3");
mapper.attributeToColumnName("d", "col4");
You can also use mapper.attributeToIndex("d", 3); to map attributes to a given column index.
Hope this helps.
I have a list of objects that are instances of a number of sub-classes of a base class. I've been trying to write these objects out together into one CSV file.
Each class contains the fields of the base class and adds a couple of extra fields of its own.
What I am trying to achieve is to write out a csv having the base class fields first and then the columns coming from the rest of the sub-classes. This of course means that the sub-classes that don't contain a particular column name should have that field empty.
I have tried achieving this using OpenCSV and SuperCSV but have not managed to configure them to do this. Looking at the libraries code I am pretty sure OpenCSV will not do this. Using SuperCSV with Dozer I got multiple classes to write in one file but I can't get the empty columns in place where a class is missing a particular column field.
I can obviously write my own custom CSV writer to achieve this but I was wondering if anyone could help me reach a solution based off an existing CSV writer library.
Edit: SuperCSV code added below per commenter's request
private static final String[] FIELD_MAPPING = new String[] { "documentNumber", "lineOfBusiness", "clientId", "childClass1Field", };
private static final String[] FIELD_MAPPING2 = new String[] { "documentNumber", "lineOfBusiness", "clientId", "childClass2Field1", "childClass2Field2"};
public static void writeWithCsvBeanWriter(PrintWriter writer, List<ParentClass> documents) throws Exception {
CsvDozerBeanWriter beanWriter = null;
try {
beanWriter = new CsvDozerBeanWriter(writer, CsvPreference.STANDARD_PREFERENCE);
final String[] header = new String[] { "documentNumber", "lineOfBusiness", "clientId", "childClass1Field", "childClass2Field1", "childClass2Field2"};
beanWriter.configureBeanMapping(ChildClass1.class, FIELD_MAPPING);
beanWriter.configureBeanMapping(ChildClass2.class, FIELD_MAPPING2);
final CellProcessor[] processors = new CellProcessor[] { new Optional(), new Optional(), new Optional(), new Optional() }
final CellProcessor[] processors2 = new CellProcessor[] { new Optional(), new Optional(), new Optional(), new Optional(), new Optional() }
beanWriter.writeHeader(header);
for (final ParentClass document : documents) {
if (document instanceof ChildClass1) {
beanWriter.write(document, processors);
} else {
beanWriter.write(document, processors2);
}
}
} finally {
if (beanWriter != null) {
beanWriter.close();
}
}
}
I am converting CSV files to a Java Bean. I need to maintain the comma inside a value which is enclosed in "".
Here is my code.
public static PPRCV convertContestToObj(String fileName) throws IOException {
PPRCV pprcvHandler = PPRCVFactory.getPPRCVTable(fileName);
CSVFormat csvFileFormat = CSVFormat.DEFAULT.newFormat(',').withEscape('"');
List<PPRCV> pprcvs = new ArrayList<>();
FileReader fileReader = new FileReader(fileName);
CSVParser csvFileParser = new CSVParser(fileReader, csvFileFormat);
List<CSVRecord> csvRecords = csvFileParser.getRecords();
for (CSVRecord csvRecord : csvRecords) {
pprcvs.add(pprcvHandler.populateDynamicDetails(csvRecord));
}
return pprcvHandler;
}
Sample CSV line:
7080001, XI, ProvinceX, TownX, BRGX, "SHOOL, BRGX", "0054A,0055A,0055B,0055C"
my DTO
private String precintCode;
private String regionName;
private String provinceName;
private String municipalityName;
private String districtName;
private String votingCenter;
private String precint;
My expected output should be
precintCode = "7080001"
regionName = "XI"
provinceName = "ProvinceX"
municipalityName = "TownX"
districtName = "BRGX"
votingCenter = "SCHOOL, BRGX"
precint = "0054A,0055A,0055B,0055C"
However actual output is this
precintCode = "7080001"
regionName = "XI"
provinceName = "ProvinceX"
municipalityName = "TownX"
districtName = "BRGX"
votingCenter = ""SCHOOL"
precint = " , BRGX,"0054A"
You need the withIgnoreSurroundingSpaces() optione here. All other settings could be remain DEFAULT.
final Reader in = new StringReader("7080001, XI, ProvinceX, TownX, BRGX, \"SHOOL, BRGX\", \"0054A,0055A,0055B,0055C\" ");
final CSVFormat csvFileFormat = CSVFormat.DEFAULT.withIgnoreSurroundingSpaces();
for (CSVRecord record: csvFileFormat.parse(in)) {
for (String field: record) {
System.out.println("\"" + field + "\"");
}
System.out.println();
}
The output is
"7080001"
"XI"
"ProvinceX"
"TownX"
"BRGX"
"SHOOL, BRGX"
"0054A,0055A,0055B,0055C"
I was able to do it using the withQuote function from the library.
CSVFormat.EXCEL.newFormat(',').withQuote('"')
Have you already tried using the CSVFormat.DEFAULT constant?-- it's for CSV files adhering to RFC 4180.
The following way worked for me:
CSVFormat.EXCEL.withQuote('"')
I have the below implementation.
csvReader = new CsvBeanReader(new InputStreamReader(stream), CsvPreference.STANDARD_PREFERENCE);
lastReadIdentity = (T) csvReader.read(Packages.class, Packages.COLS);
In my Packages.class
I have set my unitcount variable.
public String getUnitCount() {
return unitCount;
}
public void setUnitCount(String unitCount) {
this.unitCount = unitCount;
}
This works fine when it is taken as a string, but when taken as a integer, it throws the below exception. Please help
private int unitCount;
public int getUnitCount() {
return unitCount;
}
public void setUnitCount(int unitCount) {
this.unitCount = unitCount;
}
Exception:
org.supercsv.exception.SuperCsvReflectionException: unable to find method setUnitCount(java.lang.String) in class com.directv.sms.data.SubscriberPackages - check that the corresponding nameMapping element matches the field name in the bean, and the cell processor returns a type compatible with the field
context=null
at org.supercsv.util.ReflectionUtils.findSetter(ReflectionUtils.java:139)
at org.supercsv.util.MethodCache.getSetMethod(MethodCache.java:95)
I'm not sure about SuperCsv, but univocity-parsers should be able to handle this without a hitch, not to mention it is at least 3 times faster to parse your input.
Just annotate your class:
public class SubscriberPackages {
#Parsed(defaultNullRead = "0") // if the file contains nulls, then they will be converted to 0.
private int unitCount; // The attribute name will be matched against the column header in the file automatically.
}
To parse the CSV into beans:
// BeanListProcessor converts each parsed row to an instance of a given class, then stores each instance into a list.
BeanListProcessor<SubscriberPackages> rowProcessor = new BeanListProcessor<SubscriberPackages>(SubscriberPackages.class);
CsvParserSettings parserSettings = new CsvParserSettings(); //many options here, check the tutorial.
parserSettings.setRowProcessor(rowProcessor); //uses the bean processor to handle your input rows
parserSettings.setHeaderExtractionEnabled(true); // extracts header names from the input file.
CsvParser parser = new CsvParser(parserSettings); //creates a parser with your settings.
parser.parse(new FileReader(new File("/path/to/file.csv"))); //all rows parsed here go straight to the bean processor
// The BeanListProcessor provides a list of objects extracted from the input.
List<SubscriberPackages> beans = rowProcessor.getBeans();
Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).
Im using the FasterXML library to parse my CSV file. The CSV file has the column names in its first line. Unfortunately I need the columns to be renamed. I have a lambda function for this, where I can pass the red value from the csv file in and get the new value.
my code looks like this, but does not work.
CsvSchema csvSchema =CsvSchema.emptySchema().withHeader();
ArrayList<HashMap<String, String>> result = new ArrayList<HashMap<String, String>>();
MappingIterator<HashMap<String,String>> it = new CsvMapper().reader(HashMap.class)
.with(csvSchema )
.readValues(new File(fileName));
while (it.hasNext())
result.add(it.next());
System.out.println("changing the schema columns.");
for (int i=0; i < csvSchema.size();i++) {
String name = csvSchema.column(i).getName();
String newName = getNewName(name);
csvSchema.builder().renameColumn(i, newName);
}
csvSchema.rebuild();
when i try to print out the columns later, they are still the same as in the top line of my CSV file.
Additionally I noticed, that csvSchema.size() equals 0 - why?
You could instead use uniVocity-parsers for that. The following solution streams the input rows to the output so you don't need to load everything in memory to then write your data back with new headers. It will be much faster:
public static void main(String ... args) throws Exception{
Writer output = new StringWriter(); // use a FileWriter for your case
CsvWriterSettings writerSettings = new CsvWriterSettings(); //many options here - check the documentation
final CsvWriter writer = new CsvWriter(output, writerSettings);
CsvParserSettings parserSettings = new CsvParserSettings(); //many options here as well
parserSettings.setHeaderExtractionEnabled(true); // indicates the first row of the input are headers
parserSettings.setRowProcessor(new AbstractRowProcessor(){
public void processStarted(ParsingContext context) {
writer.writeHeaders("Column A", "Column B", "... etc");
}
public void rowProcessed(String[] row, ParsingContext context) {
writer.writeRow(row);
}
public void processEnded(ParsingContext context) {
writer.close();
}
});
CsvParser parser = new CsvParser(parserSettings);
Reader reader = new StringReader("A,B,C\n1,2,3\n4,5,6"); // use a FileReader for your case
parser.parse(reader); // all rows are parsed and submitted to the RowProcessor implementation of the parserSettings.
System.out.println(output.toString());
//nothing else to do. All resources are closed automatically in case of errors.
}
You can easily select the columns by using parserSettings.selectFields("B", "A") in case you want to reorder/eliminate columns.
Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).