I have a parallel stream because the task is really slow, I will paste the code below. The situation is this.
I have an arrayList, I need to do something with each object in that list (this is slow) and add the object to a temporal list, the process in the stream ends ok, I think, because I can see each object processed with logs.
When the stream ends, sometimes, the temporal list has n-1 objects or one as null.
Any idea?
With this sample code the errors are not happening, but the logic is the same but without the business logic.
public class SampleCode {
public List<SomeObject> example(List<SomeObject> someObjectList) {
List<SomeObject> someObjectListTemp = new ArrayList<>();
someObjectList.parallelStream().forEach(someObject -> {
List<ExtraData> extraDataList = getExtraData(someObject.getId());
if (extraDataList.isEmpty()) {
someObjectListTemp.add(someObject);
} else {
for (ExtraData extraData : extraDataList) {
SomeObject someObjectTemp = null;
someObjectTemp = (SomeObject) cloneObject(someObject);
if (extraData != null) {
someObjectTemp.setDate(extraData.getDate());
someObjectTemp.setData2(extraData.getData2());
}
if (someObjectTemp == null) {
System.out.println("Warning null object"); //I NEVER see this
}
someObjectListTemp.add(someObjectTemp);
System.out.println("Added object to list"); //I Always see this the same times as elements in original list
}
}
});
if (someObjectListTemp.size() < 3) {
System.out.println("Error: There should be at least 3 elements"); //Some times one object is missing in the list
}
for (SomeObject someObject : someObjectListTemp) {
if (someObject == null) {
System.out.println("Error: null element in list"); //Some times one object is null in the list
}
}
return someObjectListTemp;
}
Could you try to use the flatMap method instead of foreach?
flatMap takes a list of lists and put all their elements in a single list.
This way you do not use another ArrayList to store your temporary objects.
I feel that this might be the issue, because parallelStream is multi threading and ArrayList is not synchronised
List<SomeObject> someObjectListTemp = someObjectList.parallelStream()
.map(so -> processSomeObject(so)) // makes a stream of lists (Stream<List<SomeObject>>)
.flatMap(Collection::stream) // groups all the elements of all the lists in one stream (Stream<Someobject>)
.collect(Collectors.toList()); // transforms the stream into a list (List<SomeObject>)
And stick your code in a separate method processSomeObject which returns a list of SomeObject:
static List<SomeObject> processSomeObject(SomeObject someObject) {
List<ExtraData> extraDataList = getExtraData(someObject.getId());
List<SomeObject> someObjectListTemp = new ArrayList<>();
if (extraDataList.isEmpty()) {
someObjectListTemp.add(someObject);
} else {
for (ExtraData extraData : extraDataList) {
SomeObject someObjectTemp = (SomeObject) cloneObject(someObject);
if (extraData != null) {
someObjectTemp.setDate(extraData.getDate());
someObjectTemp.setData2(extraData.getData2());
}
someObjectListTemp.add(someObjectTemp);
System.out.println("Added object to list");
}
}
return someObjectListTemp;
}
A small example would be
public static void main(String[] args) {
List<Object> test = new ArrayList<>();
IntStream.range(0, 100000).parallel().forEach(i -> test.add(new Object()));
for(Object o : test) {
System.out.println(o.getClass());
}
}
i'ts because ArrayList is not threadsafe and the internal array gets screwed
Related
This is my forEach loop ,now I want to take the ouput for each row and return the same in an array any sort of help is appreciated .
as of now I have tried something like this , like suppose I am expecting 2object from the response , but I am recieving only one object which is the second object or last object ,
public Validation save(Multipart file){
ArrayList<Item> dta = new ArrayList<>();
IntStream.range(0, data.size()).forEach(rowNo-> {
try {
Item item saveData(rowNo+2,data.get(rowNo),vlidation);
ret.add(item);
});
return dta;
}
}
As some comments suggested you could solve this with streams, since you're already using them.
Simply map it and then call collect(Collectors.toList()) to get a list.
IntStream.range(0, data.size()).map(rowNo-> {
try {
return saveData(rowNo+2,data.get(rowNo),vlidation);
} catch(Exception ex) {
..
}
}).collect(Collectors.toList());
If you do not want to use streams you could loop and add the current item to a list:
ArrayList<Item> ret = new ArrayList<>;
IntStream.range(0, data.size()).forEach(rowNo-> {
try {
Item item = saveData(rowNo+2,data.get(rowNo),vlidation);
ret.add(item);
} catch(Exception ex) {
...
}
});
// ret is the list that you wanted
I would like to avoid the mutation of the input list of iterators tests by others. I only want others to run on a deep copy of tests.
How can this be achieved in Java?
Here is an example showing the effect of the mutation on tests. Both of the two parts are sorting the input. But the second part has nothing to be sorted since the mutation from the first part iterated the iterators to the end.
You can run the following example online here:
https://onlinegdb.com/NC4WzLzmt
import java.util.*;
public class ImmutableExample {
public static void main(String[] args) {
System.out.println("sort on demand");
List<Iterator<Integer>> mutableTests = Arrays.asList(
Arrays.asList(1, 2).iterator(),
Arrays.asList(0).iterator(),
Collections.emptyIterator()
);
List<Iterator<Integer>> tests = Collections.unmodifiableList(mutableTests);
MergingIterator mergingIterator = new MergingIterator(tests);
while (mergingIterator.hasNext()) {
System.out.println(mergingIterator.next());
}
System.out.println("sort all at once");
/* uncomment the following will see the same result:*/
// tests = Arrays.asList(
// Arrays.asList(1, 2).iterator(),
// Arrays.asList(0).iterator(),
// Collections.emptyIterator()
// );
MergeKSortedIterators sol = new MergeKSortedIterators();
Iterable<Integer> result = sol.mergeKSortedIterators(tests);
for (Integer num : result) {
System.out.println(num);
}
}
}
class PeekingIterator implements Iterator<Integer>, Comparable<PeekingIterator> {
Iterator<Integer> iterator;
Integer peekedElement;
boolean hasPeeked;
public PeekingIterator(Iterator<Integer> iterator) {
this.iterator = iterator;
}
public boolean hasNext() {
return hasPeeked || iterator.hasNext();
}
public Integer next() {
int nextElem = hasPeeked ? peekedElement : iterator.next();
hasPeeked = false;
return nextElem;
}
public Integer peek() {
peekedElement = hasPeeked ? peekedElement : iterator.next();
hasPeeked = true;
return peekedElement;
}
#Override
public int compareTo(PeekingIterator that) {
return this.peek() - that.peek();
}
}
class MergingIterator implements Iterator<Integer> {
Queue<PeekingIterator> minHeap;
public MergingIterator(List<Iterator<Integer>> iterators) {
// minHeap = new PriorityQueue<>((x, y) -> x.peek().compareTo(y.peek()));
minHeap = new PriorityQueue<>();
for (Iterator<Integer> iterator : iterators) {
if (iterator.hasNext()) {
minHeap.offer(new PeekingIterator(iterator));
}
}
}
public boolean hasNext() {
return !minHeap.isEmpty();
}
public Integer next() {
PeekingIterator nextIter = minHeap.poll();
Integer next = nextIter.next();
if (nextIter.hasNext()) {
minHeap.offer(nextIter);
}
return next;
}
}
class MergeKSortedIterators {
public Iterable<Integer> mergeKSortedIterators(List<Iterator<Integer>> iteratorList) {
List<Integer> result = new ArrayList<>();
if (iteratorList.isEmpty()) {
return result;
}
PriorityQueue<PeekingIterator> pq = new PriorityQueue<>();
for (Iterator<Integer> iterator : iteratorList) {
if (iterator.hasNext()) {
pq.add(new PeekingIterator(iterator));
}
}
while (!pq.isEmpty()) {
PeekingIterator curr = pq.poll();
// result.add(curr.peek());
// cannot use this one as hasNext() checks on `hasPeeked`
result.add(curr.next());
if (curr.hasNext()) {
pq.add(curr);
}
}
return result;
}
}
This question seems to be based on a misunderstanding ... or two.
How can I prevent mutation of a list of iterators?
You need to distinguish between the mutability of a list, and the mutability of the items in the list. I think you are actually asking about the latter. (And as such, the list is not really relevant to the question. As we shall see.)
I would like to avoid the mutation of the input list of iterators tests by others.
Again, you appear to be asking about the list, but I think you actually mean to ask about the iterators.
I only want others to run on a deep copy of tests.
This implies you want the iterators to be immutable.
Here's the problem:
An Iterator is an inherently stateful / mutable object. Indeed, there is no way to implement next() without mutating the iterator object.
Iterator objects are typically not deep copyable. They typically don't support clone() or public constructors, and they typically do not implement Serializable. (Indeed, if they were serializable, the semantics of serialize / deserialize would be problematic.)
So basically, your idea of a list of immutable iterators or a list that (somehow) produces deep copies of iterators is not practical.
You commented:
So List<Iterator<Integer>> tests = Collections.unmodifiableList(mutableTests); cannot produce an unmodifiable list for List<Iterator<Integer>>?
Well, yes it can. But that doesn't solve the problem. You need a list of unmodifiable iterators rather than an unmodifiable list of iterators.
Possible solutions:
You could just recreate the list of iterators from their base collections for each test run.
Use Iterable instead of Iterator. The collection types you are using all implement Iterable, and the third iterator could be created from an empty list.
List<Iterable<Integer>> tests = Arrays.asList(
Arrays.asList(1, 2),
Arrays.asList(0),
Collections.emptyList()
);
// to use them ...
for (Iterable<Integer> iterable : tests) {
Iterator<Integer> iterator = iterable.iterator();
// etc ...
}
If your iterators could not be recreated (for example, if you were iterating a source that couldn't be created or "rewound"), you could conceivably implement a caching iterator wrapper that remembered all of the elements in the iteration sequence and could either reset to the start of the sequence, or generate a new iterator to replay the sequence. (But that would be overkill here.)
Not sure how I can achieve this.
I have a object list, where it consists of multiple data example
ABC1231211
ABC1231111
ABC4562222
ABC4562456
Now I trying to seperate the list according to their code, which is 123 and 456, and add header and tailer to them. So my expected result would be
Head
ABC1231211
ABC1231111
Tail
Head2
ABC4562222
ABC4562456
Tail2
But the result I get is
Head
ABC1231211
Tail
Head
ABC1231111
Tail
Head2
ABC4562222
Tail2
Head2
ABC4562456
Tail2
Code
#Override
public List process(List<Detail> l) throws Exception {
for (Detail d : l) {
if (d.Code().equals("123")) {
list = generateS(d);
}
if (d.Code().equals("456")) {
list = generateR(d);
}
}
return list;
}
public List<String> generateS(Detail d) throws Exception {
try {
list.add(new HDR("Head").getHeader());
DetailL x = new DetailL();
x.setType(d.getType());
....
list.add(x.getDetail());
list.add(new TLR("Tail").getTailer());
} catch (Exception ex) {
throw new BatchException(DetailProcessor.class, ex);
}
return list;
}
Any help would be much appreciated
If you're using Java 8, you can use streams:
public void process(List<Detail> details) throws Exception {
Map<String, List<Detail>> byCode =
details.stream().collect(Collectors.groupingBy(Detail::getCode));
byCode.entrySet().stream().forEach(entry -> {
System.out.println(headerFromType(entry.getKey()));
entry.getValue().foreach(System.out::println);
System.out.println(tailFromType(entry.getKey()));
}
with headerFromType and tailFromType returning "Head"/"Head2" or "Tail"/"Tail2", depending on the given type.
You are generating a new head and tail for each element instead of adding to the already-generated list.
For each Detail, you should first check if the list exists, and if it doesn't, then call generateS or generateR as appropriate. If the list exists, you want to call e.g. sList.add(sList.size()-1, d.getDetail()). You'll of course want to replace the call d.getDetail() with the value that's supposed to go into the list or a method call that returns that value.
Then you probably want to use list.addAll(sList) to add the generated lists' contents to list.
Another solution is to generate the combined list on demand, and store the two lists separately. In that case, you would check if the corresponding list is null in the beginning of generateS or generateR, and initialize it if it is.
You create a new header and a new tail every time you call generateS or generateR but you should just create a new header once if you find a new code ( for example 123).
Solution: You collect your details into a list before you call generateS or generateR and put all the details from collected list into your DetailL.
Here is another implemetation that takes another approach:
private void go() {
List<String> list = new ArrayList<>();
list.add("ABC1231211");
list.add("ABC1231111");
list.add("ABC4562222");
list.add("ABC4562456");
String lastTag = null;
int startPos = 0;
for (int i = 0; i < list.size(); i++) {
String tag = list.get(i).substring(3, 6);
if (!tag.equals(lastTag) && lastTag != null) {
print(list.subList(startPos, i));
startPos = i;
}
lastTag = tag;
}
print(list.subList(startPos, list.size()));
}
private void print(List<String> list) {
System.out.println("Head");
for (String item : list) {
System.out.println(item);
}
System.out.println("Tail");
}
Simply "If you come accross an element with a different tag, print the previous sublist". (And print whatever is left at the end since that sublist's printout is not triggered by a new tag.)
I have a large json structure with nested values that I have converted into a list of objects to work with. I'd like to filter out all objects that don't contain a specific property value. Problem is though, that so far all I've come up with is a for loop within a for loop within a for loop (and that's given we know the json structure is only three nested levels). I only want to filter out the objects that do contain an integer (if it's null, it could be a parent containing something valid) or parents that are empty). If I try to stream with flattened - I can filter out all my objects and nested objects but won't I lose my structure?
quick eg.
public class ObjectA {
Integer id;
List<ObjectA> sublist;
}
List<ObjectA> fullList;
Set<Integer> keeptheseIntegers;
for (ObjectA obj : fullList) {
if (obj.getId() != null && !keeptheseIntegers.contains(obj.getId()){
fullList.remove(obj);
} else if (obj.getId() == null && obj.getSubList().size() > 0) {
for (ObjectA subObj : obj.getSubList()){
(same thing as above)
}
}
edit - I did realize later that the remove was not working properly and used iterator.remove. still same logical issue though
First: instead of manipulating your original structure (remove unwanted) I would collect the wanted items into an own list during the algorithm.
Second: Iterating through nested structures is a good candidate for the recursive pattern.
Third: Using java 8 I would implement it using streams and lambdas.
Something like this:
public class ObjectA
{
Integer id;
List<ObjectA> sublist;
}
private static final Set<Integer> keeptheseIntegers = new HashSet<>();
static
{
keeptheseIntegers.add( 1 );
}
private List<ObjectA> filter( List<ObjectA> list)
{
List<List<ObjectA>> subLists = new ArrayList<>();
List<ObjectA> result = list.stream()
// get all objects with sublist and add the sublist to interim subLists:
.peek( obj -> {
if ( obj.sublist == null )
{
// assert your assumption that Integer is assigned
if ( obj.id == null )
{
throw new IllegalArgumentException();
}
}
else
{
subLists.add( obj.sublist );
}
} )
// filter for the objects you want in result:
.filter( (obj -> obj.id != null && keeptheseIntegers.contains(obj.id)))
// and convert the resulting stream to a list:
.collect( Collectors.toList());
// call this method recusively for all found sublists:
subLists.forEach( i -> result.addAll(filter( i)) );
return result;
}
and somewher in your main program flow you call it:
...
List<ObjectA> fullList = new ArrayList<>();
List<ObjectA> objWithInt = filter( fullList);
// process the received list. Your original fullList is unchanged.
I have following code. In the first i tried to set values in the list called 'unavailable'.
Next, in the for each I have to produce a cycle on the list domainStr minus unavailable. How can i do it?
public Result execute(List<String> domainsStr) {
Result result = new Result();
try {
List<String> domains = domainService.findByNames(domainsStr);
result.setUnavailable(domains);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
for (String domain : domainsStr) {
......
}
return result;
}
public static class Result {
private List<String> unavailable = new ArrayList<>();
public List<String> getUnavailable() {
return unavailable;
}
public void setUnavailable(List<String> unavailable) {
this.unavailable = unavailable;
}
}
removeAll(Collection c) is the function which would be the most helpful to you. Having said that, this will work properly only if you have the equals method correctly defined for your Domain object. In this case it is a String so it doesnt matter. But, just to keep it in mind.
so just say, domainsStr.removeAll(result.getUnavailable());
Also, if the Result class is static, why the new object creation here?
Result result = new Result();
This result.setUnavailable(domains);
can be changed to
Result.setUnavailable(domains);
I have to to produce a cycle on the list domainStr minus unavailable.
If I understood correctly, I think you are looking for the removeAll method :
Removes from this list all of its elements that are contained in the
specified collection (optional operation).
domainsStr.removeAll(result.getUnavailable());
for (String domain : domainsStr) {
}
If you want to let domainsStr unchanged, you can create a temporary list and perfom these operations on it.
List<String> tempList = new ArrayList<String>(domainsStr);
tempList.removeAll(result.getUnavailable());
for(String domain : tempList){
.....
I put them into a tempt list so you don't lose the items in the domainsStr list.