Is there any way to write Java bean to Csv table format using Open Csv ?
What are the other libraries available to achieve this ?
uniVocity-parsers support for conversions to and from java beans is unmatched. Here's a simple example of a class:
public class TestBean {
// if the value parsed in the quantity column is "?" or "-", it will be replaced by null.
#NullString(nulls = {"?", "-"})
// if a value resolves to null, it will be converted to the String "0".
#Parsed(defaultNullRead = "0")
private Integer quantity
#Trim
#LowerCase
#Parsed(index = 4)
private String comments;
// you can also explicitly give the name of a column in the file.
#Parsed(field = "amount")
private BigDecimal value;
#Trim
#LowerCase
// values "no", "n" and "null" will be converted to false; values "yes" and "y" will be converted to true
#BooleanString(falseStrings = {"no", "n", "null"}, trueStrings = {"yes", "y"})
#Parsed
private Boolean pending;
}
Now, to write instances to a file, do this:
Collection<TestBean> beansToWrite = someMethodThatProducesTheObjectYouWant();
File output = new File("/path/to/output.csv");
new CsvRoutines().writeAll(beansToWrite, TestBean.class, output, Charset.forName("UTF-8"));
The library offers many configuration options and ways to achieve what you want. If you find yourself using the same annotations over and over again, just define a meta-annotation. For example, apply a replacement conversion over fields that contain the ` character, instead of declaring this in every single field:
#Parsed
#Replace(expression = "`", replacement = "")
public String fieldA;
#Parsed(field = "BB")
#Replace(expression = "`", replacement = "")
public String fieldB;
#Parsed(index = 4)
#Replace(expression = "`", replacement = "")
public String fieldC;
You can create a meta-annotatin like this:
#Retention(RetentionPolicy.RUNTIME)
#Inherited
#Target({ElementType.FIELD, ElementType.METHOD, ElementType.ANNOTATION_TYPE})
#Replace(expression = "`", replacement = "")
#Parsed
public #interface MyReplacement {
#Copy(to = Parsed.class)
String field() default "";
#Copy(to = Parsed.class, property = "index")
int myIndex() default -1;
And use it in your class like this:
#MyReplacement
public String fieldA;
#MyReplacement(field = "BB")
public String fieldB;
#MyReplacement(myIndex = 4)
public String fieldC;
}
I hope it helps.
Disclaimer: I'm the author of this library, it's open-source and free (Apache V2.0 license)
Related
I'm trying to read csv file that contains strings both quoted and not.
If string is quoted, it should save it's quote chars.
Beside that, if string contains comma, it should not be split.
I've tried multiple ways but nothing works as of now.
Current test data:
"field1 (with use of , we lose the other part)",some description
field2,"Dear %s, some text"
Getting 1st field of mapped bean
Expected result:
"field1 (with use of , we lose the other part)"
field2
Current result:
"field1 (with use of
field2
Here is the code:
public class CsvToBeanReaderTest {
#Test
void shouldIncludeDoubleQuotes() {
String testData =
"\"field1 (with use of , we lose the other part)\",some description\n"
+
"field2,\"Dear %s, some text\"";
RFC4180ParserBuilder rfc4180ParserBuilder = new RFC4180ParserBuilder();
rfc4180ParserBuilder.withQuoteChar(ICSVWriter.NO_QUOTE_CHARACTER);
ICSVParser rfc4180Parser = rfc4180ParserBuilder.build();
CSVReaderBuilder builder = new CSVReaderBuilder(new StringReader(testData));
CSVReader reader = builder
.withCSVParser(rfc4180Parser)
.build();
List<TestClass> result = new CsvToBeanBuilder<TestClass>(reader)
.withType(TestClass.class)
.withEscapeChar('\"')
.build()
.parse();
result.forEach(testClass -> System.out.println(testClass.getField1()));
}
private List<TestClass> readTestData(String testData) {
return new CsvToBeanBuilder<TestClass>(new StringReader(testData))
.withType(TestClass.class)
.withSeparator(',')
.withSkipLines(0)
.withIgnoreEmptyLine(true)
.build()
.parse();
}
public static final class TestClass {
#CsvBindByPosition(position = 0)
private String field1;
#CsvBindByPosition(position = 1)
private String description;
public String toCsvFormat() {
return String.join(",",
field1,
description);
}
public String getField1() {
return field1;
}
}
}
I've found out that if I comment or remove rfc4180ParserBuilder.withQuoteChar(ICSVWriter.NO_QUOTE_CHARACTER); the string will be parsed correctly, but I will lose the quote char which should not be lost. Is there any suggestions what can be done? (I would prefer not to switch on other csv libraries)
I'm parsing JSON object to Java object but some fields are null.
results
//Printed object after parsing, some fields are null
{host_ip='null', open=false, host_fqdn='null', nmap_results='null', cvss_vector='cvss2#av:n/ac:l/au:n/c:c/i:c/a:c', cvss=10.0, attackvector='n', list_
of_ports=null, ports___port='null', plugin_family='Windows', scanner_name='Local Scanner', operating_system='null', plugin_name='Microsoft Windows XP Unsupported
Installation Detection'
Usage of Gson library:
for (int v = 0; v < splunk_results_as_Array.size(); v++)
{
String vuln_as_string = splunk_results_as_Array.get(v).toString();
Splunk_data splunkdata = new Gson().fromJson(vuln_as_string, Splunk_data.class); // parsing happens here
System.out.print("\n Splunk : Splunk Java Object created \n" + splunkdata.toString());
splunkdata.ports_to_List();
splunkdata.setDataType(dataType);
list_of_Hosts.add(splunkdata);
}
Json object as string
"result":
{"cvss_vector":"cvss2#av:n/ac:l/au:n/c:c/i:c/a:c","host-ip":"XX.XX.XX.XX","plugin_family":"Windows","scanner_name":"Local Scanner","plugin_name":"Microsoft RDP RCE (CVE-2019-0708) (uncredentialed check)","hostname":"XX.XX.XX.XX","cvss":"10.0","attackvector":"n","ports{}.port":"3389"}
}
as you can see the fields : host-ip and ports{}.ports do have value.
The classed used:
public class Splunk_data
{
private String host_ip;
private boolean open;
private String host_fqdn;
private String nmap_results;
private String cvss_vector;
private double cvss;
private String attackvector;
private String ports_port;
private String plugin_family;
private String scanner_name;
private String operating_system;
private String plugin_name;
}
I think what caused the problem is host_ip doesnot match host-ip but I Java not possible to use '-' so I used '_'.
is there a way how to fix this?
To use different name for the variable than the actual key in JSON, you can use #Serialized annotation.
#Serialized("host-ip")
private String host_ip;
This will work and parse your host-ip as string.
I need to convert Strings of this format into an Array of objects.
[{name=Nancy Chapman, email=nchapman0#comcast.net}, {name=Jimmy Fisher, email=jfisher1#photobucket.com}]
Is there any easy way to convert this without having to do it completely manually?
UPDATE:
I am pulling these values from a custom SQL database (Amazon Athena). And the custom JDBC does not support getArray() so it looks like I need to manually parse the columns that contain an Array of Structs. It is unfortunately a limitation of the DB and I have no control over it. This is the format the SQL database returns when I call getString() on the column.
SQL Table Definition
id (int)
threadid (int)
senderemail (string)
sendername (string)
subject (string)
body (string)
recipients (array<struct<name:string,email:string>>)
ccrecipients (array<struct<name:string,email:string>>)
bccrecipients (array<struct<name:string,email:string>>)
attachments (array<binary>)
date (timestamp)
Java Objects
MessageObj
public class MessageObj {
private int id;
private int threadId;
private String senderEmail;
private String senderName;
private String subject;
private String body;
private List<RecipientObj> recipients;
private List<RecipientObj> ccRecipients;
private List<RecipientObj> bccRecipients;
private List<File> attachments;
private Calendar date;
}
RecipientObj
public class RecipientObj {
private String email;
private String name;
}
Parsing the data.
ResultSet rs = statement.executeQuery(sql);
while (rs.next()) {
// Retrieve table column.
int id = rs.getInt("id");
Integer threadId = rs.getInt("threadid");
String senderEmail = rs.getString("senderemail");
String senderName = rs.getString("sendername");
String subject = rs.getString("subject");
String body = rs.getString("body");
//How to convert recipients into ArrayList? rs.getArray("recipients") not supported.
//... Code here to add into an ArrayList of MessageObj.
}
Perhaps I'm misunderstanding your question but what you typed should work if you clean it up a bit so that it looks like this:
[{ name:"Nany Chapman", email:"nchapman0#comcast.net"},
{ name:"Nany Chapman", email:"nchapman0#comcast.net"}]
Rather than use the equal sign, use a colon and put quotation marks around your values.
I want a CSV format for Order objects. My Order Object will have order details, order line details and item details. Please find the java object below:
Order {
OrderNo, OrderName, Price,
OrderLine {
OrderLineNo, OrderLinePrice,
Item{
ItemNo, ItemName, Item Description
}
}
}
Can anyone please guide me to create csv format for this.
Have a POJO class for your Object for which you want to create CSV file and use java.io.FileWriter to write/append values in csv file. This Java-code-geek Link will help you with this.
If you are feeling adventurous, I'm building support for nested elements in CSV in uniVocity-parsers.
The 2.0.0-SNAPSHOT version supports parsing nested beans with annotations. We are planning to release the final version in a couple of weeks. Writing support has not been implemented yet, so that part you'll have to do manually (should be fairly easy with the current API).
Parsing this sort of structure is more complex, but the parser seems to be working fine for most cases. Have a look at that test case:
Input CSV:
1,Foo
Account,23234,HSBC,123433-000,HSBCAUS
Account,11234,HSBC,222343-130,HSBCCAD
2,BAR
Account,1234,CITI,213343-130,CITICAD
Note that the first column of each row identifies which bean will be read. As "Client" in the CSV matches the class name, you don't need to annotate
Pojos
enum ClientType {
PERSONAL(2),
BUSINESS(1);
int typeCode;
ClientType(int typeCode) {
this.typeCode = typeCode;
}
}
public static class Client {
#EnumOptions(customElement = "typeCode", selectors = { EnumSelector.CUSTOM_FIELD })
#Parsed(index = 0)
private ClientType type;
#Parsed(index = 1)
private String name;
#Nested(identityValue = "Account", identityIndex = 0, instanceOf = ArrayList.class, componentType = ClientAccount.class)
private List<ClientAccount> accounts;
}
public static class ClientAccount {
#Parsed(index = 1)
private BigDecimal balance;
#Parsed(index = 2)
private String bank;
#Parsed(index = 3)
private String number;
#Parsed(index = 4)
private String swift;
}
Code to parse the input
public void parseCsvToBeanWithList() {
final BeanListProcessor<Client> clientProcessor = new BeanListProcessor<Client>(Client.class);
CsvParserSettings settings = new CsvParserSettings();
settings.getFormat().setLineSeparator("\n");
settings.setRowProcessor(clientProcessor);
CsvParser parser = new CsvParser(settings);
parser.parse(new StringReader(CSV_INPUT));
List<Client> rows = clientProcessor.getBeans();
}
If you find any issue using the parser, please send update this issue
As a part of my course project we have been asked to annotate our code.
For the following code
import java.lang.annotations.*;
#Target({ElementType.LOCAL_VARIABLE})
#Retention(RetentionPolicy.RUNTIME)
public #interface DBAnnotation {
String variable () default "";
String table () default "";
String column () default "";
boolean isSource () default false;
}
public static void addFileToDB(String fileName, String fileLocation, int offerID){
#DBAnnotation (variable = "fileName", table = "files", column = "FileName", isSource = true)
#DBAnnotation (variable = "fileLocation", table = "files", column = "fileLocation", isSource = true)
String SQLFileSelect = "SELECT FileName FROM files WHERE OfferID = ? AND FileLocation = ?;";
.
.
.
}
I am getting the following error.
Duplicate annotation #File.DBAnnotation. Repeated annotations are allowed only at source level 1.8 or above
But if I change it to ...
public #interface DBAnnotation {
String[] variable () default "";
String table () default "";
String[] column () default "";
boolean[] isSource () default false;
}
.
.
.
#DBAnnotation (
variable = {"fileName","fileLocation"},
table = "files",
column = {"FileName","fileLocation"},
isSource = true)
then it does not give any error.
My concern here is, for the variable fileLocation, would te DBAnnotation be considered as
variable = "fileLocation", table = "files", column = "fileLocation",isSource = true
or will it be considered as
variable = "fileLocation", table = "", column = "fileLocation",isSource =
If you set it like this:
variable = {"fileName","fileLocation"},
table = "files",
column = {"FileName","fileLocation"},
isSource = true
then variable and column is going to be both values, since you defined them as an Array of String.
Important thing here is that what you do with a custom annotation is only up to you (what you do with it at runtime), so :
getAnnotation(DBAnnotation.class).variable(); // will return the String array with both values.