Im traslating a old enterprise App who uses C# with Linq queries to Java 8. I have some of those queries who I'm not able to reproduce using Lambdas as I dont know how C# works with those.
For example, in this Linq:
from register in registers
group register by register.muleID into groups
select new Petition
{
Data = new PetitionData
{
UUID = groups.Key
},
Registers = groups.ToList<AuditRegister>()
}).ToList<Petition>()
I undestand this as a GroupingBy on Java 8 Lambda, but what's the "select new PetitionData" inside of the query? I don't know how to code it in Java.
I have this at this moment:
Map<String, List<AuditRegister>> groupByMuleId =
registers.stream().collect(Collectors.groupingBy(AuditRegister::getMuleID));
Thank you and regards!
The select LINQ operation is similar to the map method of Stream in Java. They both transform each element of the sequence into something else.
collect(Collectors.groupingBy(AuditRegister::getMuleID)) returns a Map<String, List<AuditRegister>> as you know. But the groups variable in the C# version is an IEnumerable<IGrouping<string, AuditRegister>>. They are quite different data structures.
What you need is the entrySet method of Map. It turns the map into a Set<Map.Entry<String, List<AuditRegister>>>. Now, this data structure is more similar to IEnumerable<IGrouping<string, AuditRegister>>. This means that you can create a stream from the return value of entry, call map, and transform each element into a Petition.
groups.Key is simply x.getKey(), groups.ToList() is simply x.getValue(). It should be easy.
I suggest you to create a separate method to pass into the map method:
// you can probably came up with a more meaningful name
public static Petition mapEntryToPetition(Map.Entry<String, List<AuditRegister>> entry) {
Petition petition = new Petition();
PetitionData data = new PetitionData();
data.setUUID(entry.getKey());
petition.setData(data);
petition.setRegisters(entry.getValue());
return petition;
}
Related
I welcome methods in the API to easily create default initialisations.
For example in HashMaps. But why have they not been provided with Supplier Lambda methods? - Or am I missing an important step, or did I not learn the latest java Api versions?
Standard (Java8) version:
Map<String,List<Integer>> datas = new HashMap<>();
List<Integer> integersList = datas.getOrDefault( "somekey", new ArrayList<>() );
which would instantiate a new ArrayList anytime the code is executed - no matter if the new list is needed or not.
Desired Lambda supplier version:
Map<String,List<Integer>> datas = new HashMap<>();
List<Integer> integersList = datas.getOrDefault( "somekey", ()->new ArrayList() );
Would instantiate (or execute some instantiation code) only in case demanded key is not within the map.
The code of the getOrDefault()-Method could look something like this:
public V getOrDefault( K key, Supplier<V> supplier ) {
if ( !super.containsKey( key ) && supplier != null ) {
super.put( key, supplier.get() );
}
return super.get( key );
}
Why did they(?) not build it that way initially or added such functionality later on?
I guess there is even more examples where Lambda would solve an unnecessary code execution - not just Maps as shown with this example.
By the way: sorry for re-asking a question but I would not know how to exactly look for my question with different terms...
Be welcome to post helpful links.
Thanks for your shared knowledge :-)
What you are looking for exists since Java 8. Take a look at the javadoc of the HashMap and specifically the method Hashmap.computeIfAbsent. This method allows for adding new entries to the HashMap if none can be found using the key provided.
Examaple:
Map<Integer, String> map = new HashMap();
String created = map.computeIfAbsent(1, k -> "Test");
System.out.println(created);
The code above will trigger the HashMap to call the provided Function to add a new entry since it cannot find an existing one. It both returns the new entry and call the Hashmap.put method to add it.
I am trying to use map function on DataFrame in Spark using Java. I am following the documentation which says
map(scala.Function1 f, scala.reflect.ClassTag evidence$4)
Returns a new RDD by applying a function to all rows of this DataFrame.
While using the Function1 in map , I need to implement all the functions. I have seen some questions related to this , but the solution provided converts the DataFrame into RDD.
How can I use the map function in DataFrame without converting it into a RDD also what is the second parameter of map ie scala.reflect.ClassTag<R> evidence$4
I am using Java 7 and Spark 1.6.
I know your question is about Java 7 and Spark 1.6, but in Spark 2 (and obviously Java 8), you can have a map function as part of a class, so you do not need to manipulate Java lambdas.
The call would look like:
Dataset<String> dfMap = df.map(
new CountyFipsExtractorUsingMap(),
Encoders.STRING());
dfMap.show(5);
The class would look like:
/**
* Returns a substring of the values in the id2 column.
*
* #author jgp
*/
private final class CountyFipsExtractorUsingMap
implements MapFunction<Row, String> {
private static final long serialVersionUID = 26547L;
#Override
public String call(Row r) throws Exception {
String s = r.getAs("id2").toString().substring(2);
return s;
}
}
You can find more details in this example on GitHub.
I think map is not the right way to use on a DataFrame. Maybe you should have a look at the examples in the API
There they show how to operate on DataFrames
You can use the dataset directly, need not convert the read data to RDD, its unnecessary consumption of resource.
dataset.map(mapfuncton{...}, encoder); this should suffice your needs.
Because you don't give any specific problems, there're some common alternatives to map in DataFrame like select, selectExpr, withColumn. If the spark sql builtin functions can't fit your task, you can use UTF.
I have some data stored in a JPA Repository that I am trying to process. I would like to be able to use Java 8 Streams to do so, but can not figure out how to get the required information. This particular 'Entity' is actually only for recovery, so it holds items that would need to be processed after something like a power-fail/restart.
Using pre-Java 8 for-loops the code would look like:
List<MyEntity> deletes = myEntityJpaRepository.findByDeletes();
for (MyEntity item : deletes) {
String itemJson = item.getData();
// use a Jackson 'objectMapper' already setup to de-serialize
MyEventClass deleteEvent = objectMapper.readValue(itemJson, MyEventClass.class);
processDelete(deleteEvent, item.getId());
}
The problem arises from the two parameter method called at the very end. Using Streams, I believe I would do:
// deletes.stream()
// .map(i -> i.getData())
// .map(event -> objectMapper.readValue(event, MyEventClass.class))
// .forEach(??? can't get 'id' here to invoke 2 parameter method);
I have a solution (without Streams) that I can live with. However I would think this problem comes up a lot, thus my question is: IN GENERAL, is there a way using Streams to accomplish what I am trying to do?
Why not a Pair return on your map operation:
.map(i -> new Pair<>(i.getData(), i.getId()))
.map(pair -> new Pair<>(objectMapper.readValue(pair.getLeft(), MyEventClass.class), pair.getRight())
.forEach(p -> processDelete(pair.getLeft(), pair.getRight()))
I did not compile this, so there might be minor things to fix. But in general, you would need a Holder to pass your objects to the next stage in such a case. Either a Pair or some type or even a array.
Why not doing it simply this way?
deletes.forEach(item ->
processDelete(objectMapper.readValue(item.getData(), MyEventClass.class),
item.getId()));
This is a start at least, I guess it is dependent on why you want to use stream and how much you want to make it more functional
List<MyEntity> deletes = myEntityJpaRepository.findByDeletes();
deletes.stream().foreach(item -> {
String itemJson = item.getData();
// use a Jackson 'objectMapper' already setup to de-serialize
MyEventClass deleteEvent = objectMapper.readValue(itemJson, MyEventClass.class);
processDelete(deleteEvent, item.getId());
});
I am reading a simple JSON....
{"A":0,"B":0,"C":2,"D":0,"F":5}
into a map using JsonSlurper in Groovy...
Map gradeDistributon = jsonSlurper.parseText(jsonString)
But when iterating over this map with a closure..
gradeDistributon.each{ entry ->
println "From map got key ${entry.key}"
I am seeing the keys are not in the order they were in the original JSON, for example 'C' comes first. I think this is because Map does not maintain insertion order in Java. Is there a way I can keep the order of the original JSON?
If it means reading the JSON in a different way (instead of into a Map with JsonSlurper) then I am fine with that if you can show me how.
You can set JVM system property jdk.map.althashing.threshold to make JsonSlurper to use a LinkedHashMap instead of TreeMap as the internal Map implementation, e.g. -Djdk.map.althashing.threshold=512.
The reason is in source code of groovy.json.internal.LazyMap used by JsonSlurper.
private static final String JDK_MAP_ALTHASHING_SYSPROP = System.getProperty("jdk.map.althashing.threshold");
private void buildIfNeeded() {
if (map == null) {
/** added to avoid hash collision attack. */
if (Sys.is1_7OrLater() && JDK_MAP_ALTHASHING_SYSPROP != null) {
map = new LinkedHashMap<String, Object>(size, 0.01f);
} else {
map = new TreeMap<String, Object>();
}
}
}
Please note this solution should be used as a hack as it depends on Groovy's internal implementation details. So this behavior may change in future version of Groovy.
See my blog post for details.
So it was just a matter of sorting the keys after JsonSlurper built the Map, for that I just read into a TreeMap which sorts the keys by default..
TreeMap gradeDistributon = jsonSlurper.parseText(jsonString)
I can't reproduce your behaviour with groovy 2.4.5 but you can try using LinkedHashMap (allow to iterate over map keys maintaining the order in which the entries were inserted):
import groovy.json.*
def jsonText = '''
{"A":0,"B":0,"C":2,"D":0,"F":5,"G":7,"H":9}
'''
LinkedHashMap json = new JsonSlurper().parseText(jsonText)
json.each{ entry ->
println "${entry.key}"
}
NOTE: as stated by #XenoN the JsonSlurper() sort the json keys during the parsing process so independently of the input order (ie. {"H":0,"B":0,"A":2,"D":0,"G":5,"F":7,"C":9}) the output of JsonSlurper will be always: {"A":2,"B":0,"C":9,"D":0,"F":7,"G":5,"H":0}.
Using the LinkedHashMap instead of a HashMap we preserve the order given by JsonSlurper.
I run the same code on Groovy 2.4.x and on 3.0.x.
On 2.4 the order is preserved,but on 3.0 is sorted asc by default.
use the JsonSluperClassic().parse() instead it will preserve the order
I got the following variables
List<Pruefvorschrift> listP = new ArrayList<Pruefvorschrift>();
ObservableMap<TestDevice,List<Pruefvorschrift>> testDev = FXCollections.emptyObservableMap();
in one function i want to fill the testDev by using lambda expression
//first call REST service and get data
List<TestDevice> test_dev = call.getTestDevice("");
//now do a foreach to add each entry (as key) to the testDev ObservableMap with a empty List (as value)
test_dev.stream().forEach(td ->{
TestDevice t = td;
testDev.put(t, listP);
});
but all i get is a error
java.lang.UnsupportedOperationException at
java.util.AbstractMap.put(AbstractMap.java:209)
which obviously is this row
testDev.put(t, listP);
maybe i misunderstood the new stream api but i only want to fill the observable map with all the result of the call (key) and an empty List (value which will be modified later).
Any help? Thx
Whatever Map type is returned by FXCollections#emptyObservableMap
FXCollections.emptyObservableMap();
does not support the put method. You can't add anything to it. As the javadoc states
Creates and[sic] empty unmodifiable observable list.
This has nothing to do with lambda expressions or the Stream api.
just to complete here (Sotirios Delimanolis was absolute right and me so wrong :). My problem was solved by doing a correct job with the map itself
//create empty map
Map<TestDevice,List<Pruefvorschrift>> map = new HashMap<TestDevice,List<Pruefvorschrift>>();
//use this map to create the ObservableMap
ObservableMap<TestDevice,List<Pruefvorschrift>> testDev = FXCollections.observableMap(map);
And all works...Thx Sotirios