Develop Reference

Push data from a dataframe to a Map where values are List of objects in Scala

I am working on a Scala codebase and I have to implement a scenario which uses some data structure to populate information for further processing.
The gist of the problem is,
I have a dataframe studentDf which has the student marksheet information eg.
Name, ID, Subject, Details, Marks, isFail
Now there can be multiple records for the same Name-ID mapping. I have to reflect the scenario where if the student has failed in any subject, the details (and the corresponding record) will pop up in a resultDf. And if he has not failed in any subject (congratulations!!!) then we can populate any record corresponding to the Name-ID mapping.
Basically what I would do in Java 8 for this is,
Assuming I have List<StudentMarks> studentMarksList => list of all the marks of all the students.
Map<String, List<StudentMarks>> studentToMarkMapping = new HashMap<>();
studentMarksList.stream().foreach(studentMark->{
studentToMarkMapping.computeIfAbsent(studentMark.getName()+"_"+studentMark.getID, k => new ArrayList<>()).add(studentMark);
}
Set<Student> resultSet = new HashSet<>();
for(Map.Entry<String,List<StudentMarks>> studentToMark : studentToMarkMapping){
List<StudentMarks> studentMarks = studentToMark.getValue();
for(StudentMarks studentMark : studentMarks){
if(studentMark.getFailed() == true){//Return corresponding failed subject record
resultSet.add(studentMark);
break;
}
}
resultSet.add(studentMark.get(0)); // just add any subjectMark for student who has passed all subjects
}
In Scala to do the same, I was trying to load the data into a Mutable Map, but to populate multiple records for the same student into a list and then find out whether he has failed in any or not, I am getting stuck. I see the concept of using ListBuffer which is a mutable variant of a list, but I am confused how to use it. It is possible that we can do without Map as well, but I tried some other ways which didn't end up working.
If somebody can provide any help on this, would be great. Thanks a lot!!!

How to fetch specific attribute value from DynamoDB getItem in Java

I am using Dynamodb Item - getItem API to get records from DynamoDB table. But it returns Item object and I want to retrieve specific attribute value from the Item object. How can we do it in Java? I couldn't find references.
Table table = dynamoDB.getTable(tableName);
Item item = table.getItem(hashKeyFieldName, hashKeyFieldValue);
The item contains the following fields:
HashKey, TimeStamp, NumRetries
I want to get the specific NumRetries value from item above. Is it something that is possible? something like int numRetries = item.get("NumRetries");?

You should be able to do that with a Projection Expression:
GetItemSpec spec = new GetItemSpec().withPrimaryKey("primaryKey", primaryKey)
.withProjectionExpression("HashKey, TimeStamp, NumRetries");
Item outcome = table.getItem(spec);
A names map may be necessary.

You can use Projection Expressions to get certain attributes from an item but do keep in mind that using projection expressions does not reduce the usage and cost of RCUs that are used in retrieving the object.
Code example,
GetItemSpec spec = new GetItemSpec()
.withPrimaryKey("YourPrimaryKey", value)
.withProjectionExpression("NumRetries");
Item item = table.getItem(spec);
System.out.println(item.toJSONPretty());
More code examples can be found here.

Filter Object list with map key value , replace the Map value

I have a List of Objects
List<Person> personLst = [{"person:" {personName:AASH_01 , country :AUS, state :ADL, zip :null },
{personName:AASH_01 , country :AUS, state :MLB, zip :null}}]
and Map holds a key value and based on this need to apply the filter for the list.
Map<String, String> lstMap = new HashMap<>();
lstMap.put("ADL","12345")
Apply Filter Condition:
If the personLst.contains(ADL) this is a map key filter it and replace the zip with map value(12345).
Tried different stackoverflow question , did not help much on .

Just to be sure,
List<Person> personLst = [{"person:" {personName:AASH_01 , country :AUS, state :ADL, zip :null },{personName:AASH_01 , country :AUS, state :MLB, zip :null}}]
really refers to a list of valid java objects, right? Because in the current state those would be valid JavaScript Objects, but not valid Java objects.
Also, should a List<Person> really be called with personLst.contains(String)?
So, assuming these are valid Java objects with the correct getters and setters, and the correct method is intended, you should be able to use
personLst.forEach(p -> {
if(lstMap.containsKey(p.getState()))
p.setZip(lstMap.get(p.getState()));
});

Writing/appending data to a CSV file, column wise, in JAVA

I want to write/append data to a CSV file, column-by-column, in below fashion:
query1 query2 query3
data_item1 data_item7 data_item12
data_item2 data_item8 data_item13
data_item3 data_item9 data_item14
data_item4 data_item10
data_item5 data_item11
data_item6
I have the data in a hashMap, with the queryID (i.e. query1,query2) being the key and data_items for the
corresponding queries being the values.
The values(data_items for every query) are in a list.
Therefore, my hash map looks like this :
HashMap<String,List<String>> hm = new HashMap<String,List<String>>();
How can I write this data, column by column to a csv, as demonstrated above, using JAVA ?
I tried CSVWriter, but couldn't do it. Can anyone please help me out ?

csv files are mostly used to persist data structured like a table... meaning data with columns and rows that are in a close context.
In your example there seems to be only a very loose connection between query1, 2 and 3, and no connection horizontally between item 1,7 and 12, or 2, 8 and 13 and so on.
On top of that writing into files are usually facilitated along rows or lines. So you open your file write one line, and then another and so on.
So to write the data columnwise as you are asking, you have to either restructure your data in your code alrady to have all the data which is written into one line available on writing that line, or run through your csv file and it's lines several times, each time adding another item to a row. Of course the latter option is very time consuming and would not make much sense.
So i would suggest if there is really no connection between the data of the 3 queries, you either write your data into 3 different csv files: query1.csv, 2.csv and 3.csv.
Or, if you have a horizontal connection i.e. between item 1,7 and 12, and so on you write it into one csv file, organizing the data into rows and columns. Something like:
queryNo columnX columnY columnZ
1 item1 item2 item3
2 item7 item8 item9
3 item12 item13 item14
How to do that is well described in this thread: Java - Writing strings to a CSV file.
Other examples you can also find here https://mkyong.com/java/how-to-export-data-to-csv-file-java/

After days of tinkering around, I finally succeeded. Here is the implementation :
for(int k=0;k<maxRows;k++) {
List<String> rowValues = new ArrayList<String>();
for(int i=0;i<queryIdListArr.length;i++) {
subList = qValuesList.subList(i, i+1);
List<String> subList2 = subList.stream().flatMap(List::stream).collect(Collectors.toList());
if(subList2.size()<=k) {
rowValues.add("");
}else{
rowValues.add(subList2.get(k));
}
}
String[] rowValuesArr = new String[rowValues.size()];
rowValuesArr = rowValues.toArray(rowValuesArr);
// System.out.println(rowValues);
writer.writeNext(rowValuesArr);
}
maxRows : Size of the value list with max size. I have a list of values for each key. My hash map looks like this
HashMap<String,List<String>> hm = new HashMap<String,List<String>>();
queryIdListArr : List of all the values obtained from the hash map.
qValuesList : List of all the value lists.
List<List<String>> qValuesList = new ArrayList<List<String>>();
subList2 : sublist obtained from qValuesList using the below syntax :
qValuesList.subList(i, i+1);
rowValuesArr is an array that gets populated with the index wise value for each
value fetched from qValuesList.
The idea is to fetch all the values for each index from all the sublists and then write those values to the row. If for that index, no value is found, write a blank character.

find item from item name lookup api

I'm currently working on a project,
I need something that I can provide a name, then it can return what kind of item it is.
Say I have word starcraft,
then the API or some database can return something like game
or nba -> sports
or nike -> sports/shoe
or sadkljasd -> unknown
I saw something did this like months ago, but I can not recall.
I need something that has this kind functionality and data, and it does not have to be accurate
Anyone has any idea?
Thanks a lot

HashMap provides the kind of API you are looking for..
You can have a mapping from your item to their type.. In a Map, you store your mapping in the form of Key-Value Pair... If all your items are unique, it will be the best bet for you..
Here I will give a brief example of how a Map works.. Rest you can get from the link I have given..
Map<String, String> mapping = new HashMap<String, String>();
mapping.put("nba", "sports");
String type = mapping.get("nba");
System.out.println(type); // Will give you `sports`
And if you have multiple types for some items, you can have a Mapping from items to the list of their types: -
Map<String, ArrayList<String>> mapping =
new HashMap<String, ArrayList<String>>();

Use Collection.....
- Using Map will be appropriate at this stage...
Eg:
Map<String, ArrayList<String>> map = new HashMap<String,ArrayList<String>>();
Or
Map<String, String> map = new HashMap<String,String>();