I'm trying to create a plugin where I'm storing some Minecraft items' data along with some properties.
This is my YAML file's content:
rates:
- 391:
mul: 10000
store: 5000
- 392:
mul: 9000
store: 5000
So it's basically a list of maps of maps(I think so at least).
This is my JAVA code where I'm trying to access the key 'mul' of '391':
List<Map<?,?>> rates;
rates= getConfig().getMapList("rates");
for(Map<?,?> mp : rates){
Map <?,?> test = (Map<?,?>) mp.get("" + item);
player.sendMessage(test.toString());// HERE I get null pointer exception, and the following lines if this line wasn't there in the first place
player.sendMessage("Mul is: " + test.get("mul"));
player.sendMessage("Store is: " + test.get("store"));
}
As per suggested answer, here is my test code, where I still get NullPointerException:
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.Map;
import net.sourceforge.yamlbeans.YamlException;
import net.sourceforge.yamlbeans.YamlReader;
public class Test {
public static void main(String[] args) throws FileNotFoundException, YamlException{
YamlReader reader = new YamlReader(new FileReader("config.yml"));
Map map = (Map) reader.read();
Map itemMap = (Map) map.get("391");
System.out.println(itemMap.get("mul"));//This is where I get the exception now
System.out.println(itemMap.get("store"));
}
}
Parsing yaml by hand can be tedious and error prone. It might be easier to use a library like yamlbeans.
http://yamlbeans.sourceforge.net/
package com.jbirdvegas.q41267676;
import com.esotericsoftware.yamlbeans.YamlReader;
import java.io.StringReader;
import java.util.List;
import java.util.Map;
public class YamlExample {
public static void main(String[] args) throws Exception {
String yamlInput =
"rates:\n" +
"- 391:\n" +
" mul: 10000\n" +
" store: 5000\n" +
"- 392:\n" +
" mul: 9000\n" +
" store: 5000";
YamlReader reader = new YamlReader(new StringReader(yamlInput));
Map map = (Map) reader.read();
// rates is a list
List<Map> rates = (List<Map>) map.get("rates");
// each item in the list is a map
for (Map itemMap : rates) {
// each map contains a single map the key is [ 391|392 ]
itemMap.forEach((key, value) -> {
System.out.println("Key: " + key);
// the value in in this map is itself a map
Map embededMap = (Map) value;
// this map contains the values you want
System.out.println(embededMap.get("mul"));
System.out.println(embededMap.get("store"));
});
}
}
}
This prints:
Key: 391
10000
5000
Key: 392
9000
5000
This is a simple usecase but yamlbeans also provides GSON like reflective class model population if that would be better suited for your needs.
Your assumptions that that YAML is a list of maps is wrong. At the top level is is a map with a single key-value pair, the key of which is rates and the value is a sequence with two elements.
Each of those elements is mapping with a single key value pair, the key of which is a number (391, 392) and the value of which is a mapping with two key-value pairs each.
That there is a dash (-) at the left hand doesn't imply the that at the top level there is a sequence (there are no lists in a YAML file, that is construct in your programming language). If a sequence is a value for a specific key, those sequence elements can be at the same indentation level as the key, as in your YAML file.
Related
I'm trying to learn about Streams and Paths. I created a spellchecker in O(n) time and I'm trying to follow a guide that creates a spellchecker using Map / HashMap / Paths / and Streams.
I have created a Path to my word file and loaded it in. I'm having trouble understanding how to print this list off. I don't seem to have access to the list of words it creates when reading in from the Path. I believe I have to return a Stream of Strings, just having trouble figuring out how this exactly works.
The input file is a dictionary of words, around 56k words. One word per line.
Sample Output (one long stream of words)
aardvarkaardwolfaaronabackabacusabaftabaloneabandonabandonedabandonmentabandonsabaseabasedabasementabashabashedabateabatedabatementabatesabattoirabattoirsabb
Current Code
import java.io.*;
import java.util.*;
import java.util.Scanner;
import java.lang.*;
import java.nio.file.*;
import java.nio.file.Path;
import java.util.*;
import java.util.stream.*;
public class TheChecker {
private static Map<String,Integer> dict = new HashMap<>();
public static void main(String[] args) {
Path dictOpen = Paths.get("/Users/sam/IdeaProjects/wordSpell/src/newWords.txt");
try{
//try opening file
Spelling(dictOpen);
} catch(Exception e) {
System.out.println("Failed path");
}
//fails to find dict even though I load it into memory the step before.
for(String temp : dict) {
System.out.println(temp);
}
//This produces one giant string of all words combined on one line.
//Ideally I would like to use the known word method to find a single word
for(String temp : dict.keySet() {
System.out.println(temp);
}
}
private static void Spelling(Path dictionaryFile) throws Exception{
Stream.of(new String(Files.readAllBytes( dictionaryFile )).toLowerCase().replaceAll("[^a-z ]","").split(" ")).forEach( (word) ->{
dict.compute( word, (k,v) -> v == null ? 1 : v + 1 );
});
}
private static Stream<String> known(Stream<String> words){
return words.filter( (word) -> dict.containsKey(word) );
}
}
You have two problems:
1 - for iterate in maps use
for (Map.Entry<String, Integer> entry : dict.entrySet())
{
System.out.println(entry.getKey() + "/" + entry.getValue());
}
2 - Files.readAllBytes will read all file in one String. So if you have the words foo and bar , when you read with this function you will have : foobar.
Look for Files.readAllLines. http://www.java2s.com/Tutorials/Java/java.nio.file/Files/Java_Files_readAllLines_Path_path_Charset_cs_.htm
I am nearing the end of my assignment and one of the last thing that I have been instructed t do is to:
Use keySet() to retrieve the set of keys (the String part of the mapping). Loop over this set and print out the word and its count.
I have used the keySet() method to print out the number of keys within my HashMap, but what I have not done yet is find the length for each word within the HashMap and then print out the number of characters within each word. I am currently unsure as to how I should do this. I'm assuming I would use some time of for loop to iterate through the keySet() which I have already done and then use something like a .length() method to find out the length of each word and then print it out somehow?
Here is my relevant code so far:
Main class
package QuestionEditor;
import java.util.Set;
public class Main{
public static void main (String[] args) {
WordGroup secondWordGroup = new WordGroup ("When-you-play-play-hard-when-you-work-dont-play-at-all");
Set<String> set = secondWordGroup.getWordCountsMap().keySet();
System.out.println("set : " + set + "\n");
for(String key : set)
{
System.out.println(key);
}
}
}
WodGroup class
package QuestionEditor;
import java.util.HashMap;
public class WordGroup {
String word;
// Creates constructor which stores a string value in variable "word" and converts this into lower case using
// the lower case method.
public WordGroup(String aString) {
this.word = aString.toLowerCase();
}
public String[] getWordArray() {
String[] wordArray = word.split("-");
return wordArray;
}
public HashMap<String, Integer> getWordCountsMap() {
HashMap<String, Integer> myHashMap = new HashMap<String, Integer>();
for (String word : this.getWordArray()) {
if (myHashMap.keySet().contains(word)) {
myHashMap.put(word, myHashMap.get(word) + 1);
} else {
myHashMap.put(word, 1);
}
}
return myHashMap;
}
}
Any help on how to do this would be greatly appreciated, thanks.
UPDATE
So when my code compiles, I am getting the output:
Key: play has 3 counter
Key: all has 1 counter
Key: at has 1 counter
Key: work has 1 counter
Key: hard has 1 counter
Key: when has 2 counter
Key: you has 2 counter
Key: dont has 1 counter
But what I actually want it to do is to print out the amount of characters within each word. So for example, play would count 4 times, all would count 3 times, at would count 2 times etc. Any ideas on how to implement this?
The part that you are probably missing is: you can use the keys to then access your map values, like this:
Map<String, Integer> whateverMap = ... coming from somewhere
for (String key : whateverMap.keySet()) {
Integer countFromMap = whateverMap.get(key);
System.out.println("Key: " + key + " has " + countFromMap + " counter");
The above is meant as example to get you going, I didn't run it through a compiler.
My point here: there are various ways to iterate the elements you stored within a Map. You can use entrySet() to retrieve Entry objects; or you iterate the keys, and lookup values using each key.
You can use stream API from Java 8 to create a Map<String,Integer>
Map<String, Integer> stringIntegerMap = set.stream().collect(HashMap::new,
(hashMap, s) -> hashMap.put(s, s.length()), HashMap::putAll);
stringIntegerMap.forEach((key,value) ->System.out.println(key + " has length: "+key.length() + " and count: "+value));
The second parameter to collect function is an accumulator. You are accumulating a hasmap of your string from keyset and it's length
I have an XML which I'm converting to a java object using JAXB in the following way:
package IRCurves;
import java.io.File;
import java.util.HashMap;
import java.util.List;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;
public class XmlToObject {
public static void main(String[] args) {
try {
File file = new File("InterestRates_JPY_20160426.xml");
JAXBContext jaxbContext = JAXBContext.newInstance(InterestRateCurve.class);
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
InterestRateCurve ir= (InterestRateCurve) jaxbUnmarshaller.unmarshal(file);
System.out.println(ir.getEffectiveasof()+" "+ir.getCurrency()+" "+ir.getBaddayconvention());
System.out.println("Deposits:");
List<Deposits> list=ir.getDeposits();
for(Deposits ans:list) {
System.out.println(ans.getDaycountconvention()+" "+ans.getSnaptime()+" "+ans.getSpotdate());
System.out.println("Calenders:");
List<Calenders> list1=ans.getCalenders();
for(Calenders c:list1)
System.out.println(c.getCalender());
System.out.println("Curvepoint:");
List<Curvepoint> list2=ans.getCurvepoint();
for(Curvepoint curve:list2)
System.out.println(curve.getTenor()+" "+curve.getMaturitydate()+" "+curve.getParrate());
}
System.out.println("Swaps:");
List<Swaps> list3=ir.getSwaps();
for(Swaps swap:list3) {
System.out.println(swap.getFixeddaycountconvention()+" "+swap.getFloatingdaycountconvention()+" "+swap.getFixedpaymentfrequency()+" "+swap.getFloatingpaymentfrequency()+" "+swap.getSnaptime()+" "+swap.getSpotdate());
/*System.out.println("Calenders:");
List<Calenders> list1=swap.getCalenders();
for(Calenders c:list1)
System.out.println(c.getCalender());*/
System.out.println("Curvepoint:");
List<Curvepoint> list2=swap.getCurvepoint();
for(Curvepoint curve:list2)
System.out.println(curve.getTenor()+" "+curve.getMaturitydate()+" "+curve.getParrate());
}
} catch (JAXBException e) {
e.printStackTrace();
}
}
}
I want to store this into a Hash Map of the form HashMap<"Deposits_1M", 2016-06-29 -0.00029), where 1M is the value we get by doing curve.getTenor() and 2016-06-29 is one of the value we get by doing curve.getMaturitydate() & -.00029 we get by doing curve.getParrate(). so, basically I want the value we get from each iteration of for(Curvepoint curve:list2) for getTenor() to be as key along with the string "Deposits", and the value we get from curve.getMaturitydate() and curve.getParrate() to be as the value of one hash map entry.
How do I do that?
Java collections Maps (including HashMap) are parametric, so you need to include the key and value types in the declaration:
Map<String,String> resultsMap = new HashMap<>();
Then to create an entry, you use Map.put, so in your case I think this would be:
resultsMap.put(curve.getTenor(), curve.getMaturitydate()+" "+curve.getParrate())
Looking for a structure to have 1 key map to multiple values?
HashMap<String,List<Integer> results = new HashMap<String,List<Integer>();
// Does Key already exist?
List<Integer> temp = results.get(curve.getTenor());
if(temp != null){
//if it does add to object
temp.add(curve.getMaturitydate());
temp.add(curve.getParrate());
}else{
// if not make object and put in has map
List<Integer> nn = new ArrayList<Integer>();
nn.add(curve.getMaturitydate());
nn.add(curve.getParrate());
userList.put(curve.getTenor(), nn);
}
Then in your struct all values would be stored in ArrayList Position 0-1.
If you didn't want to split strings or store them as a concat. Also now if values change and store multiple answers. every 2 array indexes are a pair of values.
Say I have a input file as below
dept_id emp_id salary
1 13611 1234
2 13609 3245
3 13612 3251
2 13623 1232
1 13619 6574
3 13421 234
Now I want to find the average salary of each department. Like the following Hive query:
SELECT dept_id, avg(salary) FROM dept GROUP BY dept_id
This will return the output:
dept_id avg_sal
----------------
1 3904.0
2 2238.5
3 1742.5
Now, what I want to do is to generate the same output, but using the mapreduce framework. So how to write it? Thanks in advance!
IMPORTANT:
Before attempting to implement this, first try some basic examples in MapReduce, like implementing a word count program, to understand the logic and even before that, read a book or a tutorial about how MapReduce works.
The idea of aggregating stuff (like finding the average) is that you group by key (department id) in the map phase and then you reduce all the salaries of a specific department in the reduce phase.
In a more formalistic way:
MAP:
input:a line representing a salary record (i.e., dep_id, emp_id, salary)
output (key,value): (dep_id, salary)
REDUCE:
input (key, values): (dep_id, salaries:list of salary values having this dep_id)
output (key, value): (dep_id, avg(salaries))
This way, all the salaries that belong to the same department will be handled by the same reducer. All you have to do in the reducer, is find the average of the input values.
code----
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.FloatWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class AverageSalary {
public static class AvgMapper
extends Mapper<Object, Text, Text, FloatWritable>{
private Text dept_id = new Text();
private FloatWritable salary = new FloatWritable();
public void map(Object key, Text value, Context context
) throws IOException, InterruptedException {
String values[] = value.toString().split("\t");
dept_id.set(values[0]);
salary.set(Float.parseFloat(values[2]));
context.write(dept_id, salary);
}
}
public static class AvgReducer
extends Reducer<Text,FloatWritable,Text,FloatWritable> {
private FloatWritable result = new FloatWritable();
public void reduce(Text key, Iterable<FloatWritable> values,
Context context
) throws IOException, InterruptedException {
float sum = 0;
float count = 0;
for (FloatWritable val : values) {
sum += val.get();
count++;
}
result.set(sum/count);
context.write(key, result);
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "average salary");
job.setJarByClass(AverageSalary.class);
job.setMapperClass(AvgMapper.class);
job.setCombinerClass(AvgReducer.class);
job.setReducerClass(AvgReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(FloatWritable.class);
FileInputFormat.addInputPath(job, new Path("/home/kishore/Data/mapreduce.txt")); // input path
FileOutputFormat.setOutputPath(job, new Path("/home/kishore/Data/map3")); // output path
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
output
1 3904.0
2 2238.5
3 1742.5
If you have not gone through any training program yet, visit free videos on you tube by edureka for better understanding concepts : Map Reduce
Mapper
Mapper maps input key/value pairs to a set of intermediate key/value pairs.
Maps are the individual tasks that transform input records into intermediate records. The transformed intermediate records do not need to be of the same type as the input records. A given input pair may map to zero or many output pairs.
Reducer
Reducer reduces a set of intermediate values which share a key to a smaller set of values.
The number of reduces for the job is set by the user via Job.setNumReduceTasks(int).
Working example in Apache Hadoop website about : Word Count example
For your use case, simple use word count example would not be suffice.
You have to use Combiner and partitioners on Mapper since you are using Group by. Visit this video : Advanced Map reduce
So, I've been doing GSON for a while, but I just now ran into the issue of using JSON Maps, which as I understand it are basically just Key Value pairs where he value is a JSON object.
To give you an idea where I'm coming from, here's my JSON
{
"configs":[
{
"com.hp.sdn.adm.alert.impl.AlertManager":{
"trim.alert.age":{
"def_val":"14",
"desc":"Days an alert remains in storage (1 - 31)",
"val":"14"
},
"trim.enabled":{
"def_val":"true",
"desc":"Allow trim operation (true/false)",
"val":"true"
},
"trim.frequency":{
"def_val":"24",
"desc":"Frequency in hours of trim operations (8 - 168)",
"val":"24"
}
}
},
{
"com.hp.sdn.adm.auditlog.impl.AuditLogManager":{
"trim.auditlog.age":{
"def_val":"365",
"desc":"Days an audit log remains in storage (31 - 1870)",
"val":"365"
},
"trim.enabled":{
"def_val":"true",
"desc":"Allow trim operation (true/false)",
"val":"true"
},
"trim.frequency":{
"def_val":"24",
"desc":"Frequency in hours of trim operations (8 - 168)",
"val":"24"
}
}
}
]
}
All of those com.hp.sdn... things are dynamic, as in I won't know the key names until Runtime. I figured I can just use a HashMap for this and GSON would figure it out, but I'm not sure what I would name the field...
Here are my classes that I have so far
package com.wicomb.sdn.models.configs;
import java.util.HashMap;
import com.wicomb.sdn.types.Model;
public class ConfigResponse extends Model {
private ConfigGroup[] configs;
}
package com.wicomb.sdn.models.configs;
import com.wicomb.sdn.types.Model;
public class ConfigGroup extends Model {
private HashMap<String,Config> ????;
}
TL;DR How should I write the Java Class to let Gson know how to handle a Json property that I don't know the name of.. And lots of them.
You can feed Gson with a HashMap (or if children order is important a LinkedHashMap) than you iterate over the entries or the keys normally as you would do to any other map.
In the code below I use the following json as input:
{
"test": 123,
"nested":{
"nested-1": 1,
"nested-2": 2
}
}
And the code looks like these:
public void testGson() {
String input = "{\"test\": 123, \"nested\": {\"nested-1\": 1, \"nested-2\": 2}}";
LinkedHashMap<String, Object> json = new Gson().fromJson(input, LinkedHashMap.class);
// iterating
for(Map.Entry<String, Object> entry : json.entrySet()){
System.out.println(entry.getKey() + " -> " + entry.getValue());
}
// testing values
System.out.println(json.get("test")); // should be 123
Map<String, Object> nested = (Map<String, Object>) json.get("nested");
System.out.println(nested.get("nested-1")); // should be 1
System.out.println(nested.get("nested-2")); // should be 2
}