I have an ArrayList containing Movie objects which I produced from a List containing File objects using this method:
// returns a list containing movie objects with correct names
private static ArrayList<Movie> createMovieObjs(Collection<File> videoFiles) {
ArrayList<Movie> movieArrayList = new ArrayList<>();
Matcher matcher;
for (File file : videoFiles) {
matcher = Movie.NAME_PATTERN.matcher(file.getName());
while (matcher.find()) {
String movieName = matcher.group(1).replaceAll("\\.", " ");
Movie movie = new Movie(movieName);
if (!movieArrayList.contains(movie)) {
movieArrayList.add(movie);
}
}
}
return movieArrayList;
}
Everything works fine in above code, getting correct ArrayList.
Then I want to parse info for each Movie object in this ArrayList and set that info to that Movie object:
// want to parse genre, release year and imdbrating for every Movie object
for (Movie movie : movieArrayList) {
try {
movie.imdbParser();
} catch (IOException e) {
System.out.println("Parsing failed: " + e);
}
}
Here is Movie.imdbParser which uses Movie.createXmlLink (createXmlLink works fine on its own, so thas imdbParser - tested both):
private String createXmlLink() {
StringBuilder sb = new StringBuilder(XML_PART_ONE);
// need to replace spaces in movie names to "+" - api works that way
String namePassedToXml = this.title.replaceAll(" ", "+");
sb.append(namePassedToXml);
sb.append(XML_PART_TWO);
return sb.toString();
}
// parses IMDB page and sets releaseDate, genre and imdbRating in Movie objects
public void imdbParser() throws IOException {
String xmlLink = createXmlLink();
// using "new Url..." because my xml is on the web, not on my disk
Document doc = Jsoup.parse(new URL(xmlLink).openStream(), "UTF-8", "", Parser.xmlParser());
Element movieFromXml = doc.select("movie").first();
// using array to extract only last genre name - usually the most substantive one
String[] genreArray = movieFromXml.attr("genre").split(", ");
this.genre = genreArray[genreArray.length - 1];
this.imdbRating = Float.parseFloat(movieFromXml.attr("imdbRating"));
// using array to extract only year of release
String[] dateArray = movieFromXml.attr("released").split(" ");
this.releaseYear = Integer.parseInt(dateArray[2]);
}
Problems seems to be with accessing Movie objects, it doesn't create good XMLLink so when it tries to access genre in XML it throws NPE.
My error:
Exception in thread "main" java.lang.NullPointerException
at com.michal.Movie.imdbParser(Movie.java:79)
at com.michal.Main.main(Main.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Movie.java:79 is:
String[] genreArray = movieFromXml.attr("genre").split(", ");
and Main.java:52 is:
movie.imdbParser();
A lot of errors such as this are caused by the data not being as you expect. Often when dealing with a lot of data then sometimes it can be just a single record that does not "conform"!
You say that this line is causing you problems;
String[] genreArray = movieFromXml.attr("genre").split(", ");
So split the line down and debug your application to find the issue. If you can't debug then export the results to a log file.
Check that movieFromXml is not NULL.
Check that movieFromXml contains a "genre" attribute as you expect
Check that the data in the attribute can be split as you expect, and produces a valid output.
Often, it is good to view the data being retrieved prior to this call. I often find it useful to dump the data out to file and then load it in an external viewer to check that it is how I expect.
When you make a chain of calling method like this: movieFromXml.attr("genre").split(", "), you have to make sure that each of the preceding element is not null. In this case, you have to make sure that movieFromXml is not null and movieFromXml.attr("genre") is not null.
Related
I am doing a reading text file practice where I read and store the data into a string array object. One ArrayList data should have photo, title, website, and date. The text file looks like this:
photo:android_pie
title:Android Pie: Everything you need to know about Android 9
website:https://www.androidcentral.com/pie
date:20-08-2018
photo:oppo
title:OPPO Find X display
website:https://www.androidpit.com/oppo-find-x-display-review
date:25-08-2018
photo:android_pie2
title:Android 9 Pie: What Are App Actions & How To Use Them
website:https://www.androidheadlines.com/2018/08/android-9-pie-what-are-app-
actions-how-to-use-them.html
date:16-09-2018
I am trying to split and store them into a string array, which is an instance of my object class:
private List<ItemObjects> itemList;
and this is the constructor of my object class:
public ItemObjects(String photo, String name, String link, String date) {
this.photo = photo;
this.name = name;
this.link = link;
this.date = date;
}
I tried this but the ":" separator doesnt separate it like I want it to:
while ((sItems = bufferedReader.readLine()) != null) {
if (!sItems.equals("")) {
String[] tmpItemArr = sItems.split("\\:");
listViewItems.add(new ItemObjects(tmpItemArr[0], tmpItemArr[1], tmpItemArr[2], tmpItemArr[3]));
}
}
What is the best way of doing this? I have tried using for loop which stops at third line and adds the next one as new data. There are several online ways of doing it but some are very complicated and I am having trouble understanding it.
The problem is your understanding of using both the split function and the BufferReader.
By using readline function you are reading only one line so your split will split the first line only, you need to read the 4 first lines then add the item.
int count = 0;
String[] tmpItemArr = new String[4];
while ((sItems = bufferedReader.readLine()) != null) {
if (!sItems.equals("")) {
tmpItemArr[count] = sItems.split(":")[1];
count++;
if (count > 3) {
listViewItems.add(new ItemObjects(tmpItemArr[0], tmpItemArr[1], tmpItemArr[2], tmpItemArr[3]));
count = 0;
}
}
}
I'm on RavenDB 3.5.35183. I have a type:
import com.mysema.query.annotations.QueryEntity;
#QueryEntity
public class CountryLayerCount
{
public String countryName;
public int layerCount;
}
and the following query:
private int getCountryLayerCount(String countryName, IDocumentSession currentSession)
{
QCountryLayerCount countryLayerCountSurrogate = QCountryLayerCount.countryLayerCount;
IRavenQueryable<CountryLayerCount> levelDepthQuery = currentSession.query(CountryLayerCount.class, "CountryLayerCount/ByName").where(countryLayerCountSurrogate.countryName.eq(countryName));
CountryLayerCount countryLayerCount = new CountryLayerCount();
try (CloseableIterator<StreamResult<CountryLayerCount>> results = currentSession.advanced().stream(levelDepthQuery))
{
while(results.hasNext())
{
StreamResult<CountryLayerCount> srclc = results.next();
System.out.println(srclc.getKey());
CountryLayerCount clc = srclc.getDocument();
countryLayerCount = clc;
break;
}
}
catch(Exception e)
{
}
return countryLayerCount.layerCount;
}
The query executes successfully, and shows the correct ID for the document I'm retrieving (e.g. "CountryLayerCount/123"), but its data members are both null. The where clause also works fine, the country name is used to retrieve individual countries. This is so simple, but I can't see where I've gone wrong. The StreamResult contains the correct key, but getDocument() doesn't work - or, rather, it doesn't contain an object. The collection has string IDs.
In the db logger, I can see the request coming in:
Receive Request # 29: GET - geodata - http://localhost:8888/databases/geodata/streams/query/CountryLayerCount/ByName?&query=CountryName:Germany
Request # 29: GET - 22 ms - geodata - 200 - http://localhost:8888/databases/geodata/streams/query/CountryLayerCount/ByName?&query=CountryName:Germany
which, when plugged into the browser, correctly gives me:
{"Results":[{"countryName":"Germany","layerCount":5,"#metadata":{"Raven-Entity-Name":"CountryLayerCounts","Raven-Clr-Type":"DbUtilityFunctions.CountryLayerCount, DbUtilityFunctions","#id":"CountryLayerCounts/212","Temp-Index-Score":0.0,"Last-Modified":"2018-02-03T09:41:36.3165473Z","Raven-Last-Modified":"2018-02-03T09:41:36.3165473","#etag":"01000000-0000-008B-0000-0000000000D7","SerializedSizeOnDisk":164}}
]}
The index definition:
from country in docs.CountryLayerCounts
select new {
CountryName = country.countryName
}
AFAIK, one doesn't have to index all the fields of the object to retrieve it in its entirety, right ? In other words, I just need to index the field(s) to find the object, not all the fields I want to retrieve; at least that was my understanding...
Thanks !
The problem is related to incorrect casing.
For example:
try (IDocumentSession sesion = store.openSession()) {
CountryLayerCount c1 = new CountryLayerCount();
c1.layerCount = 5;
c1.countryName = "Germany";
sesion.store(c1);
sesion.saveChanges();
}
Is saved as:
{
"LayerCount": 5,
"CountryName": "Germany"
}
Please notice we use upper case letters in json for property names (this only applies to 3.X versions).
So in order to make it work, please update json properties names + edit your index:
from country in docs.CountryLayerCounts
select new {
CountryName = country.CountryName
}
Btw. If you have per country aggregation, then you can simply query using:
QCountryLayerCount countryLayerCountSurrogate =
QCountryLayerCount.countryLayerCount;
CountryLayerCount levelDepthQuery = currentSession
.query(CountryLayerCount.class, "CountryLayerCount/ByName")
.where(countryLayerCountSurrogate.countryName.eq(countryName))
.single();
Is it possible to parse a delimited file and find column datatypes? e.g
Delimited file:
Email,FirstName,DOB,Age,CreateDate
test#test1.com,Test User1,20/01/2001,24,23/02/2015 14:06:45
test#test2.com,Test User2,14/02/2001,24,23/02/2015 14:06:45
test#test3.com,Test User3,15/01/2001,24,23/02/2015 14:06:45
test#test4.com,Test User4,23/05/2001,24,23/02/2015 14:06:45
Output:
Email datatype: email
FirstName datatype: Text
DOB datatype: date
Age datatype: int
CreateDate datatype: Timestamp
The purpose of this is to read a delimited file and construct a table creation query on the fly and insert data into that table.
I tried using apache validator, I believe we need to parse the complete file in order to determine each column data type.
EDIT: The code that I've tried:
CSVReader csvReader = new CSVReader(new FileReader(fileName),',');
String[] row = null;
int[] colLength=(int[]) null;
int colCount = 0;
String[] colDataType = null;
String[] colHeaders = null;
String[] header = csvReader.readNext();
if (header != null) {
colCount = header.length;
}
colLength = new int[colCount];
colDataType = new String[colCount];
colHeaders = new String[colCount];
for (int i=0;i<colCount;i++){
colHeaders[i]=header[i];
}
int templength=0;
String tempType = null;
IntegerValidator intValidator = new IntegerValidator();
DateValidator dateValidator = new DateValidator();
TimeValidator timeValidator = new TimeValidator();
while((row = csvReader.readNext()) != null) {
for(int i=0;i<colCount;i++) {
templength = row[i].length();
colLength[i] = templength > colLength[i] ? templength : colLength[i];
if(colHeaders[i].equalsIgnoreCase("email")){
logger.info("Col "+i+" is Email");
} else if(intValidator.isValid(row[i])){
tempType="Integer";
logger.info("Col "+i+" is Integer");
} else if(timeValidator.isValid(row[i])){
tempType="Time";
logger.info("Col "+i+" is Time");
} else if(dateValidator.isValid(row[i])){
tempType="Date";
logger.info("Col "+i+" is Date");
} else {
tempType="Text";
logger.info("Col "+i+" is Text");
}
logger.info(row[i].length()+"");
}
Not sure if this is the best way of doing this, any pointers in the right direction would be of help
If you wish to write this yourself rather than use a third party library then probably the easiest mechanism is to define a regular expression for each data type and then check if all fields satisfy it. Here's some sample code to get you started (using Java 8).
public enum DataType {
DATETIME("dd/dd/dddd dd:dd:dd"),
DATE("dd/dd/dddd",
EMAIL("\\w+#\\w+"),
TEXT(".*");
private final Predicate<String> tester;
DateType(String regexp) {
tester = Pattern.compile(regexp).asPredicate();
}
public static Optional<DataType> getTypeOfField(String[] fieldValues) {
return Arrays.stream(values())
.filter(dt -> Arrays.stream(fieldValues).allMatch(dt.tester)
.findFirst();
}
}
Note that this relies on the order of the enum values (e.g. testing for datetime before date).
Yes it is possible and you do have to parse the entire file first. Have a set of rules for each data type. Iterate over every row in the column. Start of with every column having every data type and cancel of data types if a row in that column violates a rule of that data type. After iterating the column check what data type is left for the column. Eg. Lets say we have two data types integer and text... rules for integer... well it must only contain numbers 0-9 and may begin with '-'. Text can be anything.
Our column:
345
-1ab
123
The integer data type would be removed by the second row so it would be text. If row two was just -1 then you would be left with integer and text so it would be integer because text would never be removed as our rule says text can be anything... you dont have to check for text basically if you left with no other data type the answer is text. Hope this answers your question
I have slight similar kind of logic needed for my project. Searched lot but did not get right solution. For me i need to pass string object to the method that should return datatype of the obj. finally i found post from #sprinter, it looks similar to my logic but i need to pass string instead of string array.
Modified the code for my need and posted below.
public enum DataType {
DATE("dd/dd/dddd"),
EMAIL("#gmail"),
NUMBER("[0-9]+"),
STRING("^[A-Za-z0-9? ,_-]+$");
private final String regEx;
public String getRegEx() {
return regEx;
}
DataType(String regEx) {
this.regEx = regEx;
}
public static Optional<DataType> getTypeOfField(String str) {
return Arrays.stream(DataType.values())
.filter(dt -> {
return Pattern.compile(dt.getRegEx()).matcher(str).matches();
})
.findFirst();
}
}
For example:
Optional<DataType> dataType = getTypeOfField("Bharathiraja");
System.out.println(dataType);
System.out.println(dataType .get());
Output:
Optional[STRING]
STRING
Please note, regular exp pattern is vary based on requirements, so modify the pattern as per your need don't take as it is.
Happy Coding !
I want to use a text file to populate a linked list using a Java SE app. The text file has the following format:
firstnamelastname mobile home office
I want to insert these lines as nodes in the linked list! then i want to search for a specific node from the linked list that has been populated from the text file !
using firstnamelastname as a key i want to compare it with the nodes data (data is to to splitted as to compare with the "key") after finding the specific node i want to split that nodes data using split(" "); and show the resultant !!!
i just want to know about hint doing it all I will be very thankful in advance please help me out !!!
i have generated a java source code but its not working as it always give me the last nodes data so check the blunder I made if its totally wrong give any of your idea!
try {
String fullname;
String mobile;
String home;
String mobile2;
String office;
Node current = first;
while (current.data != key) {
String splitter[] = current.data.split(" ");
fullname = splitter[0];
mobile = splitter[1];
home = splitter[2];
mobile2 = splitter[3];
office = splitter[4];
if (fullname == null ? key == null : fullname.equals(key)) {
mobilefield.setText(mobile);
homefield.setText(home);
mobilefield2.setText(mobile2);
officefield.setText(office);
} else {
throw new FileNotFoundException(
"SORRY RECORD NOT LISTED IN DATABASE");
}
break;
}
} catch (Exception e) {
JOptionPane.showMessageDialog(this, e.getMessage()
+ "\nPLEASE TRY AGAIN !", "Search Error",
JOptionPane.ERROR_MESSAGE);
}
(key is as=firstname+lastname;)
I would use an ArrayList (needs less memory, has better performance for most operations)
Code looks ok for me, but I would not not split the current.data for every entry.
I would use a test like
if(current.data.startsWith(key)){
String splitter[]=current.data.split(" ");
...
}
and only if this is true I would split the current.data because only in this case you need mobile *home* office
i have 3 string list called Cname,cnamedb,Pname. These 3 lists has a result obtained by query the SQL database.cname and Pname results are getting from querying excel database. cnamedb result is getting from querying SQL database.
i declared and stored into array like this.
List Cname = new ArrayList();
List Pname =new ArrayList();
List cnamedb=new ArrayList();
Cname.add(rs.getString("Cname"));
Pname.add(rs.getString("Pname"));
cnamedb.add(res.getString("Cname"));
i tried like this
boolean hasCommonName = Cname.retainAll(Cnamedb);
if(hasCommonName){
out.println(Cname+"<br>");
out.println(hasCommonName);
}
boolean haspname=Pname.retainAll(Cnamedb);
if(haspname){
out.println(haspname);
}
This is just giving me a answer true for the first if statement but not printing the list elements. for the second if statements no results am getting.
i need to find out common elements exists in database and the excel, first i need to get common elements from Cname and Cnamedb and then from Pname and Cnamedb. how do i get the common elements.please provide me the code snippet.
the below given snippet worked for me.
List<String> commonToCnameAndCnamedb = new ArrayList<String>(Cname);
List<String> commonToPnameAndCnamedb = new ArrayList<String>(Pname);
boolean hasCommonName = commonToCnameAndCnamedb.retainAll(cnamedb);
if (hasCommonName) {
System.out.println(commonToCnameAndCnamedb + "<br>");
System.out.println(hasCommonName);
}
boolean haspname = commonToPnameAndCnamedb.retainAll(cnamedb);
if (haspname) {
System.out.println(commonToPnameAndCnamedb + "<br>");
System.out.println(haspname);
}