I have been trying to translate this
var winner =
node.Connections.Where(n => n.HQ != null).GroupBy(n
=>n.HQ)
.Select(g => new { Cantidate = g.Key, Count =
g.Count() })
.OrderByDescending(g => g.Count)
.First()
.Cantidate;
to Java, although I am not sure I can achieve this with streams. I would like for someone to explain to me exactly what that code does or to help me translate this to Java.
I have been looking up to this resource: https://github.com/mythz/java-linq-examples/blob/master/README.md
but I still cannot grasp what those lines of code do.
I understand first 3 lines, but the select gets me lost.
Thanks in advance
EDIT:
After trying some code from help here, I got this:
Map<Candidate,List<Candidate>> winnersByWinner = node.getConnections().stream()
.map(Node::getHQ)
.filter(Objects::nonNull)
.collect(Collectors.groupingBy(Function.identity()));
winner = winnersByWinner.entrySet().stream()
.map(e -> new AbstractMap.SimpleImmutableEntry<>(e.getKey(), e.getValue().size()))
.sorted(new Comparator<AbstractMap.SimpleImmutableEntry<Candidate, Integer>>() {
#Override
public int compare(AbstractMap.SimpleImmutableEntry<Candidate, Integer> o1, AbstractMap.SimpleImmutableEntry<Candidate, Integer> o2) {
Integer o1v = o1.getValue();
Integer o2v = o2.getValue();
if(o1v==o2v) {
Integer o1k = o1.getKey().getId();
Integer o2k = o2.getKey().getId();
return o2k.compareTo(o1k);
}
return o1v.compareTo(o2v);
}
})
//.reversed())
//.sorted(Comparator.comparingInt(AbstractMap.SimpleImmutableEntry::getValue).reversed())
.map(Map.Entry::getKey)
.findFirst()
.orElseGet(null);
Thanks to Novaterata.
This:
candidates.Select(c => nodes.Count(n => n.HQ == c));
translates to:
candidates.stream().map(c -> nodes.stream().map(Node::getHQ).filter(c::equals).count())
Thanks to Novaterata again.
My code works now quite ok, but I have to edit to make sure of one last thing that I may have translated badly:
nodes.Where(n => n.Committes.Any()
translated to:
nodes.stream().filter(n -> !n.Committes.isEmpty()).collect(Collectors.toList())
Is this correct?
This is assuming node is a type Node with a getHQ() property. This is my approximation of what you'd need to do. Using SimpleImmutableEntry inplace of the anonymous type. I'd create a simple private class to replace that for readability. This gist is that groupBy will result in a Map that you you need to turn back into a stream with .entrySet().stream() Map.Entry is very similar to KeyValuePair in C#. Hopefully the rest is self-explanatory. Any method or Class I've used here, you should look it up.
Map<HQ,List<HQ>> winnersByWinner = node.getConnection().stream()
.map(Node::getHQ)
.filter(Objects::nonNull)
.collect(Collectors.groupingBy(Function.identity()))
HQ winner = winndersByWinnder.entrySet().stream()
.map(e -> new AbstractMap.SimpleImmutableEntry<>(e.getKey(), e.getValue().size()))
.sorted(Comparator.comparingInt(Map.Entry::getValue).reversed())
.map(Map.Entry::getKey)
.findFirst()
.orElseGet(null);
Following is the explanation of the code.
var winner =node.Connections.Where(n => n.HQ != null) // this is checking the connection where HQ is not null
.GroupBy(n=>n.HQ) // grouping based on HQ
.Select(g => new { Cantidate = g.Key, Count = g.Count() }) // creating an enumerable anonymous type with cantidate , count
.OrderByDescending(g => g.Count) // sorting this enumerable type in type order based on the count
.First() // take the first element from this enumerable type
.Cantidate; // get the value of cantidate of that first element
This will be the equivalent SQL
(from n in connection
where( n.HQ != null)
GroupBy(n.HQ) into g
select new { Cantidate = g.key, Count = g.Count()}
orderby Count descending).First().Cantidate
Use the link which You have provide to convert it into java.
Related
I'm new to functional methods like reduce and fold in java 8 and Kotlin. I want to reduce -
List<List<List<Double>>> boundingPolygon = [[[-125.48845080566404,47.94508483691371],[-124.96110705566404,42.309040799653665],[-117.13884143066404,45.04173793121063],[-118.36931018066404,48.93624688577435],[-125.48845080566404,47.94508483691371]]];
to single string that represents one String concatenated by coordinates -
"-118.359053 33.931562,-118.372443 33.946939,-118.369053 33.951562,-118.337612 33.944342,-118.342012 33.944042,-118.359053 33.931562"
Trying to do -
val polygonCoordinates = boundingPolygon.first().reduce { acc, element ->
acc + "${element[0]} ${element[1]}"
}
This is not working.
The acc in your reduce operation is of type List<Double>, not String. Look at the reduce function signature and you should understand why. Here's what I propose to do what you want:
coords.first().joinToString(", ") { (x, y) -> "$x $y" }
I used destructuring to extract the first and second values from the coordinate lists. So this will only work with 2D coordinates.
Instead of Reduce, you can use flatMap. It will help you.
List<List<List<Double>>> boundingPolygon = List.of(List.of(List.of(-124.96110705566404, 42.309040799653665)
, List.of(-117.13884143066404, 45.04173793121063)
, List.of(118.36931018066404, 48.93624688577435)
));
var l = boundingPolygon.stream().flatMap(lists -> lists.stream().flatMap(doubles -> doubles.stream())).collect(Collectors.toList());
System.out.println(l);
It will print an output as below.
[-124.96110705566404, 42.309040799653665, -117.13884143066404, 45.04173793121063, 118.36931018066404, 48.93624688577435]
Try above code, this will help you.
Instead of reducing it, you should just add those into a StringBuilder, which is efficient when you are doing multiple operations (like concatenating a ton of Strings):
val boundingPolygon = listOf(
listOf(
listOf(-125.48845080566404, 47.94508483691371),
listOf(-124.96110705566404, 42.309040799653665),
listOf(-117.13884143066404, 45.04173793121063),
listOf(-118.36931018066404, 48.93624688577435),
listOf(-125.48845080566404, 47.94508483691371)
)
)
val sb = StringBuilder()
for (nestedList in boundingPolygon) {
for (innerNestedList in nestedList) {
sb.append(innerNestedList.joinToString(" "))
sb.append(',')
}
}
if (sb.isNotEmpty()) sb.deleteCharAt(sb.lastIndex)
println(sb)
// Output: -125.48845080566404 47.94508483691371,-124.96110705566404 42.309040799653665,-117.13884143066404 45.04173793121063,-118.36931018066404 48.93624688577435,-125.48845080566404 47.94508483691371
// val stringRepresentation = sb.toString() // for further use as String data-type
I am trying to filter a list of POJOs with two different predicates using a stream.
public class Toy {
public boolean isRed();
public boolean isBlue();
public boolean isGreen();
}
public class RedBlueExtravaganza {
public RedBlueExtravaganza(RedToy rt, BlueToy bt) {
//construct
}
}
// Wrappers around Toy with more verbose names
public class RedToy extends Toy { }
public class BlueToy extends Toy { }
public class GreenToy extends Toy { }
Basically, I want the first red and the first blue toy in the list of Toy objects.
List<Toy> toyList = Arrays.asList(greenToy1, redToy1, greenToy2, redToy2, blueToy1);
I want to write a stream that does the following:
RedBlueExtravaganza firstRedBlueList = toyList.stream()
// get first red but keep rest of list
// get first blue but keep rest of list
// discard rest of list
// map to a Tuple (or something) to distinguish the one red and one blue toy
// map to RedBlueExtravaganza
.findFirst()
.get();
log.info(firstRedBlueList); // now contains redToy1, blueToy1
Thanks for the help!
Here's a solution which traverses your list only once, giving you the first red and blue toys at the end. We can filter out all the other irrelevent colors and then create a map whose key is whether the toy is red and the value is the first toy matching the given criteria. Here's how it looks.
Map<Boolean, Toy> firstRedAndBlueToysMap = toyList.stream()
.filter(t -> t.isBlue() || t.isRed())
.collect(Collectors.toMap(Toy::isRed, t -> t, (a, b) -> a));
Toy firstRedToy = firstRedAndBlueToysMap.get(true);
Toy firstBlueToy = firstRedAndBlueToysMap.get(false);
And here's a one step approach to solve your problem.
RedBlueExtravaganza firstRedAndBlueToyPair = toyList.stream()
.filter(t -> t.isBlue() || t.isRed())
.collect(Collectors.collectingAndThen(Collectors.toMap(Toy::isRed,
t -> t, (a, b) -> a),
m -> new RedBlueExtravaganza(m.get(true), m.get(false))));
P.S. For this to work you need to have the following constructor in your RedBlueExtravaganza class contrary to the one you have provided above.
public RedBlueExtravaganza(Toy rt, Toy bt) {
if (!(rt instanceof RedToy) || !(bt instanceof BlueToy))
throw new IllegalArgumentException();
// remainder omitted.
}
I like this answers, similar solution can be with reduce and List as "Tuple (or something)"
List<Toy> reduceToList = toyList.stream()
.filter(t -> t.isBlue() || t.isRed())
.map(t -> Arrays.asList(t, t))
.reduce(Arrays.asList(null, null), (a, c) -> a.get(0) == null && c.get(0).isRed() ?
Arrays.asList(c.get(0), a.get(1)) : (a.get(1) == null && c.get(1).isBlue() ?
Arrays.asList(a.get(0), c.get(1)) : a)
);
If both values are not null then you can map to RedBlueExtravaganza
One way which first comes into my mind is to use 2 streams for finding first red and blue toy object, respectively. Then merge them into one stream by using Stream.concat() so that you can add more oprations for this.
Code snippet
RedBlueExtravaganza firstRedBlueList = Stream
.concat(
toyList.stream().filter(t -> t.isRed())
.findFirst()
.map(Collections::singletonList)
.orElseGet(Collections::emptyList)
.stream(),
toyList.stream().filter(t -> t.isBlue())
.findFirst()
.map(Collections::singletonList)
.orElseGet(Collections::emptyList)
.stream())
.map(x -> {
RedToy rt = new RedToy();
BlueToy bt = new BlueToy();
...
return new RedBlueExtravaganza(rt, bt);})
.findFirst()
.get();
I have a typed dataset of a custom class and use groupbykey method on it. You know that it results a KeyValueGroupedDataset. I want to filter this new dataset but there is no filter method for this type of dataset. So, My question is: How can I filter on this type of dataset? (Java solution is needed. spark version: 2.3.1).
sampleData:
"id":1,"fname":"Gale","lname":"Willmett","email":"gwillmett0#nhs.uk","gender":"Female"
"id":2,"fname":"Chantalle","lname":"Wilcher","email":"cwilcher1#blinklist.com","gender":"Female"
"id":3,"fname":"Polly","lname":"Grandisson","email":"pgrandisson2#linkedin.com","gender":"Female"
"id":3,"fname":"Moshe","lname":"Pink","email":"mpink3#twitter.com","gender":"Male"
"id":2,"fname":"Yorke","lname":"Ginnelly","email":"yginnelly4#apple.com","gender":"Male"
And What I did:
Dataset<Person> peopleDS = spark.read().format("parquet").load("\path").as(Encoders.bean(Person.class));
KeyValueGroupedDataset<String, Person> KVDS = peopleDS.groupByKey( (MapFunction<Person, String> ) f -> f.getGender() , Encoders.STRING());
//How Can I filter on KVDS's id field?
Update1 (use of flatMapGroups):
Dataset<Person> persons = KVDS.flatMapGroups((FlatMapGroupsFunction <String,Person,Person>) (f,k) -> (Iterator<Person>) k , Encoders.bean(Person.class));
Update2 (use of MapGroups)
Dataset<Person> peopleMap = KVDS.mapGroups((MapGroupsFunction <String,Person,Person>) (f,g) -> {
while (g.hasNext()) {
//What can I do here?
}
},Encoders.bean(Person.Class);
Update3 : I want to filter those groups that distinct of their ids is greater than 1. for example in below picture: I want just Female groups because distinct of their ids is greater that 1 (first field is id. Others are fname,lname,email and gender).
Update4: I did What I want with "RDD", but I want to do exactly this part of code with "Dataset":
List<Tuple2<String, Iterable<Person>>> f = PersonRDD
.mapToPair(s -> new Tuple2<>(s.getGender(), s)).groupByKey()
.filter(t -> ((Collection<Person>) t._2()).stream().mapToInt(e -> e.getId).distinct().count() > 1)
.collect();
Why don't you filter on id before grouping ? GroupByKey is an expensive action, it should be faster to filter first.
If you really want to group first, you may have to then use .flatMapGroups with identity function.
Not sure about java code but scala version would be something as follow:
peopleDS
.groupByKey(_.gender)
.mapGroups { case (gender, persons) => persons.filter(your condition) }
But again, you should filter first :). Specially since your ID field is already available before grouping.
Grouping is used for aggregation functions, you can find functions like "agg" in "KeyValueGroupedDataset" class. If you apply aggregation function for ex. "count", you will get "Dataset", and "filter" function will be available.
"groupBy" without aggregation function looks strange, other function, for ex. "distinct" can be used.
Filtering example with "FlatMapGroupsFunction":
.flatMapGroups(
(FlatMapGroupsFunction<String, Person, Person>) (f, k) -> {
List<Person> result = new ArrayList<>();
while (k.hasNext()) {
Person value = k.next();
// filter condition here
if (value != null) {
result.add(value);
}
}
return result.iterator();
},
Encoders.bean(Person.class))
Have two classes and two corresponding lists:
class Click {
long campaignId;
Date date;
}
class Campaign {
long campaignId;
Date start;
Date end;
String type;
}
List<Click> clicks = ..;
List<Campaign> campaigns = ..;
And want to find all Clicks in clicks that:
Have a corresponding Campaign in campaigns list, i.e., Campaign with the same campaignId AND
This Campaign has type = "prospective" AND
This Campaigns.start < click.date < Campaigns.end
So far I have the following implementation (which seems confusing and complex to me):
clicks.
stream().
filter(click -> campaigns.stream().anyMatch(
campaign -> campaign.getCampaignType().equals("prospecting") &&
campaign.getCampaignId().equals(click.getCampaignId()) &&
campaign.getStart().after(click.getDate()) &&
campaign.getEnd().before(click.getDate()))).
collect(toList());
I wonder if there is simpler solution for the problem.
Well, there is a very neat way to solve your problem IMO, original idea coming from Holger (I'll find the question and link it here).
You could define your method that does the checks (I've simplified it just a bit):
static boolean checkClick(List<Campaign> campaigns, Click click) {
return campaigns.stream().anyMatch(camp -> camp.getCampaignId()
== click.getCampaignId());
}
And define a function that binds the parameters:
public static <T, U> Predicate<U> bind(BiFunction<T, U, Boolean> f, T t) {
return u -> f.apply(t, u);
}
And the usage would be:
BiFunction<List<Campaign>, Click, Boolean> biFunction = YourClass::checkClick;
Predicate<Click> predicate = bind(biFunction, campaigns);
clicks.stream()
.filter(predicate::test)
.collect(Collectors.toList());
One thing that stands out is that your 2nd requirement has nothing to do with the matching, it's a condition on campaigns only. You'd have to test if this is any better for you:
clicks.stream()
.filter(click -> campaigns.stream()
.filter(camp -> "prospecting".equals(camp.type))
.anyMatch(camp ->
camp.campaignId == click.campaignId &&
camp.end.after(click.date) &&
camp.start.before(click.date)
)
)
.collect(Collectors.toList());
Otherwise, I have never seen a streams solution which does not involve streaming the 2nd collection inside the predicate of the 1st, so you can't do much better than what you did. In terms of readability, if it looks that confusing to you then create a method that test for the boolean condition and call it:
clicks.stream()
.filter(click -> campaigns.stream()
.filter(camp -> "pre".equals(camp.type))
.anyMatch(camp -> accept(camp, click))
)
.collect(Collectors.toList());
static boolean accept(Campaign camp, Click click) {
return camp.campaignId == click.campaignId &&
camp.end.after(click.date) &&
camp.start.before(click.date);
}
Finally, 2 unrelated suggestions:
Don't use the old Date class, instead use the new java.time API's LocalDate.
If Campaign's type can only have some predefined values (like "submitted", "prospecting", "accepted"...) then an enum would be a better fit than a general String.
My 2 cents:
Since there is no much boilerplate code in OP. So it may be not possible/necessary to reduce the lines/characters in the codes. we could rewrite it to make it a little more clearly:
Map<Long, List<Campaign>> map = campaigns.stream().filter(c -> c.type.equals("prospecting"))
.collect(Collectors.groupingBy(c -> c.campaignId));
clicks.stream().filter(k -> map.containsKey(k.campaignId))
.filter(k -> map.get(k.campaignId).stream().anyMatch(c -> c.start.before(k.date) && c.end.after(k.date)))
.collect(Collectors.toList());
The code is not much shorter than original code. but it will improve performance from O(nm) to O(n+m), as #Marco13 mentioned in the comments. if you want shorter, try StreamEx
Map<Long, List<Campaign>> map = StreamEx.of(campaigns)
.filter(c -> c.type.equals("prospecting")).groupingBy(c -> c.campaignId);
StreamEx.of(clicks).filter(k -> map.containsKey(k.campaignId))
.filter(k -> map.get(k.campaignId).stream().anyMatch(c -> c.start.after(k.date) && c.end.before(k.date)))
.toList();
public List<Click> findMatchingClicks(List<Campaign> cmps, List<Click> clicks) {
List<Campaign> cmpsProspective = cmps.stream().filter(cmp -> "prospective".equals(cmp.type)).collect(Collectors.toList());
return clicks.stream().filter(c -> matchesAnyCmp(c, cmpsProspective).collect(Collectors.toList());
}
public boolean matchesAnyCmp(Click click, List<Campaign> cmps) {
return cmps.stream().anyMatch(click -> cmp.start.before(click.date) && cmp.end.after(click.date));
}
Replace fields for getters, just wrote it quick.
I have a case where A has a list of B and B has some property i need.
pseudocode structure
class A
List<B> elements;
class B
Property property;
I have List of A. In order, to get property i should go through the double foreach loop to get the property i want. Like this:
String myFutureProp = null;
for (A a: list of A) {
for(B b: list of B) {
if("MY_PROPERTY".equals(b.getKey) {
myFutureProp = b.getValue();
}
}
}
I was thinking to get it more tasty using Stream API. I was looking forward with
forEach() solution:
final String[] myFutureProp = {null}
a.getElements()
.foreach(b -> b.getElements().stream()
.filter("MY_PROPERTY"::equals)
.forEach(prop -> myFutureProp[0] = (String)prop.getValue);
Then i'm taking myFutureProp[0] but it looks ugly to be honest. Is it any another solution in Stream API that i'm able to use?
I hope I got your class structure right.
You can use flatMap to get a Stream of all the Propertys of all the B instances, and return any property having the required key (if found) :
Optional<String> myFutureProp =
aList.stream()
.flatMap(a->a.getElements().stream().map(B::getProperty))
.filter(prop->prop.getKey().equals("MY_PROPERTY"))
.map(Property::getValue)
.findAny();
This should do a work -> get last matching value
final String value = bList.stream()
.flatMap(bElement -> bElement.elements().stream())
.map(aElement -> aElement.getProperty())
.filter(property -> property.getKey().equals("MY_PROPERTY"))
.map(property -> property.getValue())
.reduce((a,b) -> b).orElse(null);