Say I got 100 records from DB, I want to loop 10 times and perform some action.
and loop again and perform some action.. it continues until the last record is read.
The problem what i am seeing is
for (int number: numbers) {
add(number);
//after adding 10 items I want to complete a function and continue the loop
}
But here in Java, if use the above loop we can see it will iterate the complete list and comes out.
I know in older versions, we can iterate by counter like
for(int i=0; i<10;i++)
some thing like this.
My question is if forEach loop doesnot provide this flexibility, then why Sun Java introduced to a looping mechanism where it will iterate completely.
Trying to understand the logic of this design.
You can use a forEach or for-in with a nested conditional.
for(number: numbers){
if(number != (multOfTen)){
myFunction(number);
add(number);
}else{
add(number);
}
}
you will need to replace multOfTen w/ an expression that includes only multiples of ten.
One way to do this is to use regEx to check that the last digit is zero (so long as your using integers.)
As the name suggests, forEach will iterate over the whole collection. This is the same for JavaScript and several other languages. However you can abort / skip the iteration with a condition (which depends on the language implementation details).
There are difference between for and forEach loop. There are reason why java has these 2. forEach is enhance for loop. Both have their usage based on requirements.
for
This we can use for general purpose. This is totally based on indexes. If you want to play with data at particular index or want to perform some actions based on index of element, you should use for.
forEach
this is used with only collections and arrays. This iterate over whole collections at once. Means you can't have index of element while iterating it. This is used when you manipulate each data in list regardless whats its index. For example to print all element in a given list, instead of writing classical for loop
for (int i =0; i < list.length(); i++){
System.out.println(list(i));
}
we use forEach loop
list.forEach(e -> {
System.out.println(e);
});
this is more readable, easy to use and crisp.
because sometimes the question or the implementation you're doing using java doesn't need the index of the array "simply".
so instead of writing the whole for(int i=0;i<arr.length;i++) thing you can just use foreach and instead of arr[i] you use a simple variable name.
This is actually possible using the same Stream api but it's beyond the scope of forEach. What you want to do isn't possible by limiting yourself to forEach. The purpose of forEach is to execute some action without bias on all elements of an Iterable or Stream. What you can do is break down your objects into groups of 10 and then for each grouping of 10, do what you want which is add all to the underlying collection and then perform some other action which is what you want as well.
List<Integer> values = IntStream.range(0, 100)
.boxed()
.collect(Collectors.toList());
int batch = 10;
List<List<Integer>> groupsOfTen = IntStream.range(0, values.size() / batch + 1)
.map(index -> index * batch)
.mapToObj(index -> values.subList(index,
Math.min(index + batch, values.size())))
.collect(Collectors.toList());
groupsOfTen.forEach(myListOfTen -> myListOfTen.forEach(individual -> {
}));
Related
I have an arrayList with 30 elements. I'd like to create many sublists of 15 elements from this list. What's the efficient way of doing so?
Right now I clone the ArrayList and use remove(random) to do it, but I am sure this is too clumsy. What should I do instead?
Does Java have a "sample" function like in R?
Clarification: by sampling with no replacement I mean take at random 15 unique elements from the 30 available in the original list. Moreover I want to be able to do this repeatedly.
Use the Collections#shuffle method to shuffle your original list, and return a list with the first 15 elements.
Consider creating new list and adding random elements from current list instead of copying all elements and removing them.
Another way to do this is to create some kind of View on top of the current list.
Implement an Iterator interface that randomly generates index of element during next operation and retrieves element by index from current list.
No, Java does not have a sample function like in R. However, it is possible to write such a function:
// Samples n elements from original, and returns that list
public <T> static List<T> sample(List<T> original, int n) {
List<T> result = new ArrayList<T>(n);
for (int i = 0; i < original.size(); i++) {
if (result.size() == n)
return result;
if ((n - result.size()) >= (original.size() - i)) {
result.add(original.get(i));
} else if (Math.random() < ((double)n / original.size())) {
result.add(original.get(i));
}
}
return result;
}
This function iterates through original, and copies the current element to result based on a random number, unless we are near enough to the end of original to require copying all the remaining elements (the second if statement in the loop).
This is a basic combinatorics problem. You have 30 elements in your list, and you want to choose 15. If the order matters, you want a permutation, if it doesn't matter, you want a combination.
There are various Java combinatorics samples on the web, and they typically use combinadics. I don't know of any ready made Java libraries, but Apache Math Commons has binomial coefficient support to help you implement combinadics if you go that route. Once you have a sequence of 15 indices from 0 to 29, I'd suggest creating a read-only iterator that you can read the elements from. That way you won't have to create any new lists or copy any references.
I would like to know when I can use IntStream.range effectively. I have three reasons why I am not sure how useful IntStream.range is.
(Please think of start and end as integers.)
If I want an array, [start, start+1, ..., end-2, end-1], the code below is much faster.
int[] arr = new int[end - start];
int index = 0;
for(int i = start; i < end; i++)
arr[index++] = i;
This is probably because toArray() in IntStream.range(start, end).toArray() is very slow.
I use MersenneTwister to shuffle arrays. (I downloaded MersenneTwister class online.) I do not think there is a way to shuffle IntStream using MersenneTwister.
I do not think just getting int numbers from start to end-1 is useful. I can use for(int i = start; i < end; i++), which seems easier and not slow.
Could you tell me when I should choose IntStream.range?
There are several uses for IntStream.range.
One is to use the int values themselves:
IntStream.range(start, end).filter(i -> isPrime(i))....
Another is to do something N times:
IntStream.range(0, N).forEach(this::doSomething);
Your case (1) is to create an array filled with a range:
int[] arr = IntStream.range(start, end).toArray();
You say this is "very slow" but, like other respondents, I suspect your benchmark methodology. For small arrays there is indeed more overhead with stream setup, but this should be so small as to be unnoticeable. For large arrays the overhead should be negligible, as filling a large array is dominated by memory bandwidth.
Sometimes you need to fill an existing array. You can do that this way:
int[] arr = new int[end - start];
IntStream.range(0, end - start).forEach(i -> arr[i] = i + start);
There's a utility method Arrays.setAll that can do this even more concisely:
int[] arr = new int[end - start];
Arrays.setAll(arr, i -> i + start);
There is also Arrays.parallelSetAll which can fill an existing array in parallel. Internally, it simply uses an IntStream and calls parallel() on it. This should provide a speedup for large array on a multicore system.
I've found that a fair number of my answers on Stack Overflow involve using IntStream.range. You can search for them using these search criteria in the search box:
user:1441122 IntStream.range
One application of IntStream.range I find particularly useful is to operate on elements of an array, where the array indexes as well as the array's values participate in the computation. There's a whole class of problems like this.
For example, suppose you want to find the locations of increasing runs of numbers within an array. The result is an array of indexes into the first array, where each index points to the start of a run.
To compute this, observe that a run starts at a location where the value is less than the previous value. (A run also starts at location 0). Thus:
int[] arr = { 1, 3, 5, 7, 9, 2, 4, 6, 3, 5, 0 };
int[] runs = IntStream.range(0, arr.length)
.filter(i -> i == 0 || arr[i-1] > arr[i])
.toArray();
System.out.println(Arrays.toString(runs));
[0, 5, 8, 10]
Of course, you could do this with a for-loop, but I find that using IntStream is preferable in many cases. For example, it's easy to store an unknown number of results into an array using toArray(), whereas with a for-loop you have to handle copying and resizing, which distracts from the core logic of the loop.
Finally, it's much easier to run IntStream.range computations in parallel.
Here's an example:
public class Test {
public static void main(String[] args) {
System.out.println(sum(LongStream.of(40,2))); // call A
System.out.println(sum(LongStream.range(1,100_000_000))); //call B
}
public static long sum(LongStream in) {
return in.sum();
}
}
So, let's look at what sum() does: it counts the sum of an arbitrary stream of numbers. We call it in two different ways: once with an explicit list of numbers, and once with a range.
If you only had call A, you might be tempted to put the two numbers into an array and pass it to sum() but that's clearly not an option with call B (you'd run out of memory). Likewise you could just pass the start and end for call B, but then you couldn't support the case of call A.
So to sum it up, ranges are useful here because:
We need to pass them around between methods
The target method doesn't just work on ranges but any stream of numbers
But it only operates on individual numbers of the stream, reading them sequentially. (This is why shuffling with streams is a terrible idea in general.)
There is also the readability argument: code using streams can be much more concise than loops, and thus more readable, but I wanted to show an example where a solution relying on IntStreans is functionally superior too.
I used LongStream to emphasise the point, but the same goes for IntStream
And yes, for simple summing this may look like a bit of an overkill, but consider for example reservoir sampling
IntStream.range returns a range of integers as a stream so you can do stream processing over it.
like taking square of each element
IntStream.range(1, 10).map(i -> i * i);
Here are few differences that comes to my head between IntStream.range and traditional for loops :
IntStream are lazily evaluated, the pipeline is traversed when calling a terminal operation. For loops evaluate at each iteration.
IntStream will provides you some functions that are commonly applied to a range of ints such as sum and avg.
IntStream will allow you to code multiple operation over a range of int in a functional way which read more fluently - specially if you have a lot of operations.
So basically use IntStream when one or more of these differences are useful to you.
But please bear in mind that shuffling a Stream sound quite strange as a Stream is not a data structure and therefore it does not really make sense to shuffle it (in case you were planning on building a special IntSupplier). Shuffle the result instead.
As for the performance, while there may be a few overhead, you will still iterate N times in both case and should not really care more.
Basically, if you want Stream operations, you can use the range() method. For example, to use concurrency or want to use map() or reduce(). Then you are better off with IntStream.
For example:
IntStream.range(1, 5).parallel().forEach(i -> heavyOperation());
Or:
IntStream.range(1, 5).reduce(1, (x, y) -> x * y)
// > 24
You can achieve the second example also with a for-loop, but you need intermediate variables etc.
Also, if you want the first match for example, you can use findFirst() and cousins to stop consuming the rest of the Stream
It totally depends on the use case. However, the syntax and stream API adds lot of easy one liners which can definitely replace the conventional loops.
IntStream is really helpful and syntactic sugar in some cases,
IntStream.range(1, 101).sum();
IntStream.range(1, 101).average();
IntStream.range(1, 101).filter(i -> i % 2 == 0).count();
//... and so on
Whatever you can do with IntStream you can do with conventional loops. As one liner is more precise to understand and maintain.
Still for negative loops we can not use IntStream#range, it only works in positive increment. So following is not possible,
for(int i = 100; i > 1; i--) {
// Negative loop
}
Case 1 : Yes conventional loop is much faster in this case as toArray has a bit overhead.
Case 2 : I don't know anything about it, my apologies.
Case 3 : IntStream is not slow at all, IntStream.range and conventional loop are almost same in terms of performance.
See :
Java 8 nested loops with streams & performance
You could implement your Mersenne Twister as an Iterator and stream from that.
I am having 2 Lists and want to add them element by element. Like that:
Is there an easier way and probably much more well performing way than using a for loop to iterate over the first list and add it to the result list?
I appreciate your answer!
Depends on what kind of list and what kind of for loop.
Iterating over the elements (rather than indices) would almost certainly be plenty fast enough.
On the other hand, iterating over indices and repeatedly getting the element by index could work rather poorly for certain types of lists (e.g. a linked list).
My understanding is that you have List1 and List2 and that you want to find the best performing way to find result[index] = List1[index] + list2[index]
My main suggestion is that before you start optimising for performance is to measure whether you need to optimise at all. You can iterate through the lists as you said, something like:
for(int i = 0; i < listSize; i++)
{
result[i] = List1[i] + List2[i];
}
In most cases this is fine. See NPE's answer for a description of where this might be expensive, i.e. a linked list. Also see this answer and note that each step of the for loop is doing a get - on an array it is done in 1 step, but in a linked list it is done in as many steps at it takes to iterate to the element in the list.
Assuming a standard array, this is O(n) and (depending on array size) will be done so quickly that it will hardly result in a blip on your performance profiling.
As a twist, since the operations are completely independent, that is result[0] = List1[0] + List2[0] is independent of result[1] = List1[1] + List2[1], etc, you can run these operations in parallel. E.g. you could run the first half of the calculations (<= List.Size / 2) on one thread and the other half (> List.Size / 2) on another thread and expect the elapsed time to roughly halve (assuming at least 2 free CPUs). Now, the best number of threads to use depends on the size of your data, the number of CPUs, other operations happening at the same time and is normally best decided by testing and modeling under different conditions. All this adds complexity to your program, so my main recommendation is to start simple, then measure and then decide whether you need to optimise.
Looping is inevitable except you have a matrix API (e.g. OpenGL). You could implement a List<Integer> which is backed by the original Lists:
public class CalcList implements List<Integer> {
private List<Integer> l1, l2;
#Override
public int get(int index) {
return l1.get(index) + l2.get(index);
}
}
This avoids copy operations and moves the calculations at the end of your stack:
CalcList<Integer> results1 = new CalcList(list, list1);
CalcList<Integer> results2 = new CalcList(results1, list3);
// no calculation or memory allocated until now.
for (int result : results2) {
// here happens the calculation, still without unnecessary memory
}
This could give an advantage if the compiler is able to translate it into:
for (int i = 0; i < list1.size; i++) {
int result = list1[i] + list2[i] + list3[i] + …;
}
But I doubt that. You have to run a benchmark for your specific use case to find out if this implementation has an advantage.
Java doesn't come with a map style function, so the the way of doing this kind of operation is using a for loop.
Even if you use some other construct, the looping will be done anyway. An alternative is using the GPU for computations but this is not a default Java feature.
Also using arrays should be faster than operating with linked lists.
I'm writing a program in java where I need to slide the elements of the array and it should be performing as less as possible number of operations as it's inside a double loop and I'm working with length of array ranging from upto 10^8.
Example : A = {1,2,3,4,5,6}
Result : A = {2,3,4,5,6,1} for 1st time
A = {3,4,5,6,1,2} for 2nd time and so on..
Please feel free to suggest any other data structure or any modifications to the array!! Thank you guys!! :D
The simplest way to achieve that effect, is to do a "circular array"; that is, instead of moving the contents of the array, you can simply store the index that marks the beginning of the array.
To get the item at index i, you then do:
Type item = arr[(offset + i) % arr.length];
This way, you get the same properties as you have in an array, and you can perform any rotation in O(1).
In order to make this less of a hassle to use, you could make a simple wrapper class, that simply wraps an array, allowing easy rotation through this method. That way, the code could look clean, while you get efficient rotation.
In order to achieve an O(1) complexity, you could...
use a linked list
wrap your array with a class that stores the start position and let you access the array through "virtual" indexes (wrapped.acces(i) => array[(start + i) % array.length]
"double" your array and slice it in an appropriate way (so you don't have to change the surrounding code)
Otherwise, if you want to stick with your data structure, you need to pay O(n), no matter what.
I'd go with (2), because it is faster to both random access and linear access patterns (arrays have better data locality + O(1) random access complexity wrt O(n) of linked lists).
Use Collections.rotate(Arrays.asList(myArray), myDistance).
If you're not married to the idea of using arrays, then you could make use of the Collections.rotate() method.
List<Integer> list = new ArrayList<Integer>();
for (int i = 1; i <= 6; i++) {
list.add(i-1, new Integer(i));
}
int j = 0;
while (j < 100) {
Collections.rotate(list, -1);
System.out.print("{");
for (Integer integer : list) {
System.out.print(integer + ", ");
}
System.out.println("}");
}
Why do You have to rotate the table ?
Imagin that table is a circle and after that you can walk like this:
Object object = array[(offset + i) % array.length];
This give you O(1) on any access or rotation step;
You can use a simple List for that. If you do this sliding often, a queue would be the best choice.
The thing is, the array doesn't really change, you just start to read at position x and then continue to read at the start of the array length(array)-x elements. This is the fastest Variant.
I need to delete some objects from an ArrayList if they meet a condition and I'm wondering which way could be more efficient.
Here's the situation: I have a class that contains an ArrayList containing some other objects. I have to iterate over this ArrayList and delete all elements meeting a certain condition.
As far as I know, those would be my options to delete:
Create a new ArrayList and add the elements that doesn't meet the condition. After the iteration, swap from the old arraylist to the new one without the elements.
Create a new ArrayList and add the elements that meet the condition. After the iteration, use the removeAll() method passing the ArrayList with the objects to be deleted.
Is there a more efficient way to delete objects from an ArrayList?
You could iterate backwards and remove as you go through the ArrayList. This has the advantage of subsequent elements not needing to shift and is easier to program than moving forwards.
Another way: The Iterator has an optional remove()-method, that is implemented for ArrayList. You can use it while iterating.
I don't know though, which variant is the most performant, you should measure it.
starblue commented, that the complexity isn't good, and that's true (for removeAll() too), because ArrayList has to copy all elements, if in the middle is an element added or removed. For that cases should a LinkedList work better. But, as we all don't know your real use-cases the best is too measure all variants, to pick the best solution.
Most performant would, I guess, be using the listIterator method and do a reverse iteration:
for (ListIterator<E> iter = list.listIterator(list.size()); iter.hasPrevious();){
if (weWantToDelete(iter.previous())) iter.remove();
}
Edit: Much later, one might also want to add the Java 8 way of removing elements from a list (or any collection!) using a lambda or method reference. An in-place filter for collections, if you like:
list.removeIf(e -> e.isBad() && e.shouldGoAway());
This is probably the best way to clean up a collection. Since it uses internal iteration, the collection implementation could take shortcuts to make it as fast as possible (for ArrayLists, it could minimize the amount of copying needed).
Obviously, of the two methods you mention number 1 is more efficient, since it only needs to go through the list once, while with method number 2 the list has to be traversed two times (first to find the elements to remove, and them to remove them).
Actually, removing a list of elements from another list is likely an algorithm that's worse than O(n) so method 2 is even worse.
The iterator method:
List data = ...;
for (Iterator i = data.iterator(); i.hasNext(); ) {
Object element = i.next();
if (!(...)) {
i.remove();
}
}
First, I'd make sure that this really is a performance bottleneck, otherwise I'd go with the solution that is cleanest and most expressive.
If it IS a performance bottleneck, just try the different strategies and see what's the quickest. My bet is on creating a new ArrayList and puting the desired objects in that one, discarding the old ArrayList.
Unless you're positive that the issue you're facing is indeed a bottleneck, I would go for the readable
public ArrayList filterThings() {
ArrayList pileOfThings;
ArrayList filteredPileOfThings = new ArrayList();
for (Thing thingy : pileOfThings) {
if (thingy.property != 1) {
filteredPileOfThings.add(thingy);
}
}
return filteredPileOfThings;
}
There is a hidden cost in removing elements from an ArrayList. Each time you delete an element, you need to move the elements to fill the "hole". On average, this will take N / 2 assignments for a list with N elements.
So removing M elements from an N element ArrayList is O(M * N) on average. An O(N) solution involves creating a new list. For example.
List data = ...;
List newData = new ArrayList(data.size());
for (Iterator i = data.iterator(); i.hasNext(); ) {
Object element = i.next();
if ((...)) {
newData.add(element);
}
}
If N is large, my guess is that this approach will be faster than the remove approach for values of M as small as 3 or 4.
But it is important to create newList large enough to hold all elements in list to avoid copying the backing array when it is expanded.
int sizepuede= listaoptionVO.size();
for (int i = 0; i < sizepuede; i++) {
if(listaoptionVO.get(i).getDescripcionRuc()==null){
listaoptionVO.remove(listaoptionVO.get(i));
i--;
sizepuede--;
}
}
edit: added indentation
Maybe Iterator’s remove() method? The JDK’s default collection classes should all creator iterators that support this method.
I have found an alternative faster solution:
int j = 0;
for (Iterator i = list.listIterator(); i.hasNext(); ) {
j++;
if (campo.getNome().equals(key)) {
i.remove();
i = list.listIterator(j);
}
}
With an iterator you can handle always the element which comes to order, not a specified index. So, you should not be troubled with the above matter.
Iterator itr = list.iterator();
String strElement = "";
while(itr.hasNext()){
strElement = (String)itr.next();
if(strElement.equals("2"))
{
itr.remove();
}
Whilst this is counter intuitive this is the way that i sped up this operation by a huge amount.
Exactly what i was doing:
ArrayList < HashMap < String , String >> results; // This has been filled with a whole bunch of results
ArrayList < HashMap < String , String > > discard = findResultsToDiscard(results);
results.removeall(discard);
However the remove all method was taking upwards of 6 seconds (NOT including the method to get the discard results) to remove approximately 800 results from an array of 2000 (ish).
I tried the iterator method suggested by gustafc and others on this post.
This did speed up the operation slightly (down to about 4 seconds) however this was still not good enough. So i tried something risky...
ArrayList < HashMap < String, String>> results;
List < Integer > noIndex = getTheDiscardedIndexs(results);
for (int j = noIndex.size()-1; j >= 0; j-- ){
results.remove(noIndex.get(j).intValue());
}
whilst the getTheDiscardedIndexs save an array of index's rather then an array of HashMaps. This it turns out sped up removing objects much quicker ( about 0.1 of a second now) and will be more memory efficient as we dont need to create a large array of results to remove.
Hope this helps someone.
I'm good with Mnementh's recommentation.
Just one caveat though,
ConcurrentModificationException
Mind that you don't have more than one thread running. This exception could appear if more than one thread executes, and the threads are not well synchronized.