I am trying to parse a csv string like this
COL1,COL2,COL3
1,2,3
2,4,5
and map columns to a java object-
Class Person{
COL1,
COL2,
COL3;
}
Most of the libraries I found on google are for csv files but I am working with google app engine so can't write or read files. currently I am using split method but problems with this approach is
column that I am getting in csv string could vary as
COL1,COL3,COL2
don't want to use boiler plate code of splitting and getting each column.so what I need is list of column header and read all columns in a collection using header mapper. While iterating, map column value to a java object.
There are several question based on similar type of requirement but none of them helped me.
If anyone has done this before please could you share the idea? Thanks!
After searching and trying several libraries, I am able to solve it. I am sharing the code if anyone needs it later-
public class CSVParsing {
public void parseCSV() throws IOException {
List<Person> list = Lists.newArrayList();
String str = "COL1,COL2,COL3\n" +
"A,B,23\n" +
"S,H,20\n";
CsvSchema schema = CsvSchema.emptySchema().withHeader();
ObjectReader mapper = new CsvMapper().reader(Person.class).with(schema);
MappingIterator<Person> it = mapper.readValues(str);
while (it.hasNext()) {
list.add(it.next());
}
System.out.println("stored list is:" + (list != null ? list.toString() : null));
}}
Most of the libraries I found on google are for csv files but I am
working with google app engine so can't write or read files
You can read file (in project file system).
You can read and write file in blobstore, google cloud storage
Use a Tokenizer to split the string into objects then set them to the object.
//Split the string into tokens using a comma as the token seperator
StringTokenizer st = new StringTokenizer(lineFromFile, ",");
while (st.hasMoreTokens())
{
//Collect each item
st.nextElement();
}
//Set to object
Person p = new Person(item1, item2, item3);
If the columns can be reversed, you parse the header line, save it's values and and use it to decide which column each token falls under using, say, a Map
String columns[] = new String[3]; //Fill these with column names
Map<String,String> map = new HashMap<>();
int i=0;
while (st.hasMoreTokens())
{
//Collect each item
map.put(columns[i++], st.nextElement());
}
Then just, create the Person
Person p = new Person(map.get("COL1"), map.get("COL2"), map.get("COL3"));
Related
I'm not sure what format this object is in but can I parse the following invalid JSON object to Java class Pojo? I tried doing it using Jackson but since it's invalid, I was wondering if pojo class would work?
{
name: (sindey, crosby)
game: "Hockey"
type: athlete
}
The file would have multiple objects of this format
Geesh, don't recognise this format! If you want to use Jackson you could pre-process you data to wrap the values... perhaps a regex to catpure the groups and wrap the values in quotes something like (name|type):\s(.+) => $1: "$2"
I was wondering if pojo class would work?
Sure, you could make that work with a simple parser; plenty of room for improvement, but something like this would do it:
Record record = null;
var records = new LinkedList<>();
// I'm using a reader as example, but just provide lines any way you like
while ((line = reader.readLine()) != null) {
line = line.trim();
// may need to skip empty lines if you have them in your file...
if (line.equals("{")) {
record = new Record();
records.add(record);
} else {
// may need substrings if your data contains ":"
var tokens = line.split(":");
var field = tokens[0];
var value = tokens[1];
if (field.equals("name")) {
// perhaps shuffle the format to something nicer here..
record.setName(value);
}
/// same for game and type fields...
}
}
I am trying to implement a csv reader into my class in Java using eclipse. I keep getting an error for the add method "add(Person) in the type list is not applicable for the arguments (String[]). What am I doing wrong?
public static List<Person> readPersons(String fileName)
throws FileNotFoundException {
int count = 0;
List<Person[]> content = new ArrayList<>();
try(BufferedReader cv = new BufferedReader(new FileReader(fileName))){
String line = "";
while ((line = cv.readLine()) != null) {
content.add(line.split(","));
}
} catch (FileNotFoundException e) {
}
return content;
}
Also, how do I implement this FileNotFoundException extender? It is required in the program.
The line.split( "," ) method will return an array of strings.
What it does is: The original string line is split into an array of strings. In that array, every string is a substring of line which is separated by a comma.
For example, if line is "Peter,Smith,38", the following array of strings will be returned: [ "Peter", "Smith", "38" ].
But, since your List only can contain objects of the type Person, it cannot take the String[] array returned by line.split( "," ).
So assuming you have an Constructor for Person that looks like this: Person( String firstName, String secondName, int age ) you would have to change your while loop to something like this:
while ( ( line = cv.readLine( ) ) != null )
{
// Get the data from the CSV object
String[] csvData = line.split( "," );
// Create a Person object with the retrieved data
Person personFromCsvLine = new Person( csvData[0], csvData[1], Integer.parseInt( csvData[2] );
// Now you can add the person object to your list
content.add( personFromCsvLine );
}
The answer by be-ta is correct.
I would like to point out that for reading a CSV you might want to consider using an existing solution in your code, like:
Apache Commons CSV
Jackson CsvMapper
Example code
These libraries will help with quoted / unquoted columns and conversion of values to the proper datatypes.
I have a ArrayList<Metadata> and i want to know if there is a Java API for working with CSV files which has a write method which accepts a ArrayList<> as parameter similar to LinqToCsv in .Net. As i know OpenCSV is available but the CsvWriter class doesn't accept a collection.
My Metadata Class is
public class Metadata{
private String page;
private String document;
private String loan;
private String type;
}
ArrayList<Metadata> record = new ArrayList<Metadata>();
once i populate the record, i want to write each row into a csv file.
Please suggest.
Surely there'll be a heap of APIs that will do this for you, but why not do it yourself for such a simple case? It will save you a dependency, which is a good thing for any project of any size.
Create a toCsvRow() method in Metadata that joins the strings separated by a comma.
public String toCsvRow() {
return Stream.of(page, document, loan, type)
.map(value -> value.replaceAll("\"", "\"\""))
.map(value -> Stream.of("\"", ",").anyMatch(value::contains) ? "\"" + value + "\"" : value)
.collect(Collectors.joining(","));
}
Collect the result of this method for every Metadata object separated by a new line.
String recordAsCsv = record.stream()
.map(Metadata::toCsvRow)
.collect(Collectors.joining(System.getProperty("line.separator")));
EDIT
Should you not be so fortunate as to have Java 8 and the Stream API at your disposal, this would be almost as simple using a traditional List.
public String toCsvRow() {
String csvRow = "";
for (String value : Arrays.asList(page, document, loan, type)) {
String processed = value;
if (value.contains("\"") || value.contains(",")) {
processed = "\"" + value.replaceAll("\"", "\"\"") + "\"";
}
csvRow += "," + processed;
}
return csvRow.substring(1);
}
By using CSVWriter, you could convert the ArrayList to an array, and pass that to the writer .
csvWriter.writeNext(record.toArray(new String[record.size()]));
If you have an ArrayList of Objects (Metadata in your case) you would use the BeanToCSV instead of the CSVWriter.
You can look at the BeanToCSVTest in the opencsv source code for examples of how to use it.
I've written an Android application for my school that generates a HashMap which maps a course name to an ArrayList of available sections for that course (Strings). The map is generated using JSoup to connect to the school website and get all the current course information, parse and format it, and create the HashMap>().
It works. However, it literally takes around 5 minutes for the HashMap to generate on an Android device. I'm a relative novice in programming and I was wondering if there are other, more efficient ways of storing and dealing with such a large amount of data (The HashMap maps to around 800 ArrayLists, which in turn contain several Strings each). Ideally the data will be updated each time the application runs, so I'm not sure if writing to the internal storage would be efficient.
Any suggestions?
Thanks
Edit: Here's the method that creates the HashMap. It's a little convoluted but the website I'm pulling the data from wasn't easy to work with.
public HashMap<String, ArrayList<String>> generateCourseSectionMap()
{
ArrayList<String> store = new ArrayList<String>();
CourseLinks courses = new CourseLinks();
HashMap<String, String> courseLinks = courses.getCourseMap();
StringUtils util = new StringUtils();
HashMap<String, ArrayList<String>> map = new HashMap<String, ArrayList<String>>();
String sectionFormat = "((\\d){5};(\\d)+;(.*?) \\((.*?)\\);(.*?);(\\d)+ \\/ (\\d)+;(.*?);(TBA|Mo|Tu|We|Th|Fr|Sa|Su)+( (\\d){1,2}:(\\d){2}(AM|PM) - (\\d){1,2}:(\\d){2}(AM|PM))*?;(.*?));";
Document doc;
try
{
for (Map.Entry<String, String> entry : courseLinks.entrySet())
{
doc = Jsoup.connect(entry.getValue()).get();
Elements links = doc.select("*+tr>*:not(tr[class~=SectionTopic.*]>*):not(tr[class~=SectionTitle.*]>*)");
if (!links.isEmpty())
links.remove(0);
String build = "";
for (Element e : links)
{
String s = util.trim(e.text());
if (!s.isEmpty())
build = build + s + ";";
}
String rebuilt = rebuild(build);
store = util.toArrayList(rebuilt.split("BREAK"));
for (String d : store)
{
Pattern p = Pattern.compile(sectionFormat, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Matcher m = p.matcher(d);
String[] array = d.split(";");
String firstKey = d.substring(0, d.indexOf(";"));
ArrayList<String> sectionList = new ArrayList<String>();
while (m.find())
sectionList.add(array[0] + ";" + array[1] + ";" + m.group());
map.put(firstKey, sectionList);
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
return map;
}
First of all, this:
Pattern p = Pattern.compile(sectionFormat, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Compiling a pattern for each iteration in a for loop is suboptimal.
Compile it once at the beginning, and use the compiled pattern subsequently.
Also, this:
build = build + s + ";";
As build is a String, repeatedly concatenating it will create new strings in memory at each iteration.
Consider using a StringBuilder's append method instead.
That being said, these issues are not significant enough that they would slow down your process this much.
There isn't enough information right now for me to quickly notice further obvious problems, however it seems that depending on the number of links that are found and then the number of pages downloaded, most of the time might be spent in reading from the network and parsing HTML pages.
You might want to use a tool such as http://developer.android.com/tools/debugging/debugging-tracing.html to see what's happening.
In Python we can do this easily:
data = {'name':'Felix'}
s = 'Hello, %(name)s' % data
s
'Hello, Felix'
Is there a similar way in Java to implement the same thing?
PS:
Sorry for the unclear question. the use case is : we have a map which stores the key-values, the Template only need to specify a key in the map, then the value of the key will be in the place where the key is in the template.
AFAIK you can use String#format for this:
String name = "Felix";
String s = String.format("Hello, %s", name);
System.out.println(s);
This will print
Hello, Felix
More info about how to use the formatting of String#format can be found on java.util.Formatter syntax
You want String.format method.
String data = "Hello, %s";
String updated = String.format(data, "Felix");
If you want to replace only Strings with Strings then code from second part of my answer will be better
Java Formatter class doesn't support %(key)s form, but instead you can use %index$s where index is counted from 1 like in this example
System.out.format("%3$s, %2$s, %1s", "a", "b", "c");
// indexes 1 2 3
output:
c, b, a
So all you need to do is create some array that will contain values used in pattern and change key names to its corresponding indexes (increased by 1 since first index used by Formatter is written as 1$ not as 0$ like we would expect for arrays indexes).
Here is example of method that will do it for you
// I made this Pattern static and put it outside of method to compile it only once,
// also it will match every (xxx) that has % before it, but wont include %
static Pattern formatPattern = Pattern.compile("(?<=%)\\(([^)]+)\\)");
public static String format(String pattern, Map<String, ?> map) {
StringBuffer sb = new StringBuffer();
List<Object> valuesList = new ArrayList<>();
Matcher m = formatPattern.matcher(pattern);
while (m.find()) {
String key = m.group(1);//group 1 contains part inside parenthesis
Object value = map.get(key);
// If map doesn't contain key, value will be null.
// If you want to react somehow to null value like throw some
// Exception
// now is the good time.
if (valuesList.contains(value)) {
m.appendReplacement(sb, (valuesList.indexOf(value) + 1) + "\\$");
} else {
valuesList.add(value);
m.appendReplacement(sb, valuesList.size() + "\\$");
}
}
m.appendTail(sb);
return String.format(sb.toString(), valuesList.toArray());
}
usage
Map<String, Object> map = new HashMap<>();
map.put("name", "Felix");
map.put("age", 70);
String myPattern =
"Hi %(emptyKey)s! My name is %(name)s %(name)s and I am %(age)s years old";
System.out.println(format(myPattern, map));
output:
Hi null! My name is Felix Felix and I am 70 years old
As you can see you can use same key few times (in our case name) and if your map wont contain key used in your String pattern (like emptyKey) it will be replaced with null.
Above version was meant to let you set type of data like s d and so on, but if your data will always be replaced with Strings, then you can skip String.format(sb.toString(), valuesList.toArray()) and replace all your keys with values earlier.
Here is simpler version that will accept only map with <String,String> key-value relationship.
static Pattern stringsPattern = Pattern.compile("%\\(([^)]+)\\)s\\b");
public static String formatStrings(String pattern, Map<String, String> map) {
StringBuffer sb = new StringBuffer();
Matcher m = stringsPattern.matcher(pattern);
while (m.find()) {
// we can't use null as replacement so we need to convert it to String
// first. We can do it with String.valueOf method
m.appendReplacement(sb, String.valueOf(map.get(m.group(1))));
}
m.appendTail(sb);
return sb.toString();
}
Under this use case, you need a template engine like velocity or freemarker to use a Map-like data structure to render a string template, there is no builtin module in java to do that. like this(with velocity):
public static void main(String[] args) {
Context context = new VelocityContext();
context.put("appid", "9876543d1");
context.put("ds", "2013-09-11");
StringWriter sw = new StringWriter();
String template = "APPID is ${appid} and DS is ${ds}";
Velocity.evaluate(context, sw, "velocity", template);
System.out.println(sw.toString());
}
If you want more advanced techniques like i18n support, you can use the advanced Message Format features
ex:
in langage properties files you add the property 'template' wich is your message
template = At {2,time,short} on {2,date,long}, \
we detected {1,number,integer} spaceships on \
the planet {0}.
then you can format your valriables pass the arguments in an array:
Object[] messageArguments = {
"Mars",
new Integer(7),
new Date()
};
You call the formatter it this way:
MessageFormat formatter = new MessageFormat("");
formatter.setLocale(currentLocale);
formatter.applyPattern(messages.getString("template"));
String output = formatter.format(messageArguments);
the detailed example is here
http://docs.oracle.com/javase/tutorial/i18n/format/messageFormat.html