I wanted to know what changes should I be making to this code to reduce its execution time.
import java.util.*;
public class ExamPeekableQueueImpl <E extends Comparable<E>> implements ExamPeekableQueue<E> {
LinkedList<E> li = new LinkedList<E>();
public ExamPeekableQueueImpl(){
}
public void enqueue(E e){
if(li.isEmpty()){
li.add(0, e);
}
else
li.add(e);
}
public E dequeue(){
li.pollFirst();
return null;
}
public void printlist(){
System.out.println(li.toString());
}
public E peekMedian(){
int var = (((li.size())/2)+1);
Collections.sort(li);
//Integer var2 = li.get(var);
System.out.println("the median is:" + li.get(var-1));
return null;
}
public E peekMaximum(){
Collections.sort(li);
System.out.println("the maximum is:" + li.getLast());
return null;
}
public E peekMinimum(){
Collections.sort(li);
System.out.println("the minimum is:" + li.getFirst());
return null;
}
public int size(){
li.size();
return 0;
}
}
Also I wanted to know is whether for implementing queues, LinkedList is faster or ArrayList or any other data structure.
This is a loaded question.
What operation do you want to speed up? Right now, your peekMin/Max/Mid code is quite slow because you have to sort each time. However, your insert code is fast. If you want, you can maintain a sorted data structure internally. This would slow down your insert method, but speed up your peek methods. There is often a tradeoff of speed between operations like this. It's rare to be able to just speed up ALL the operations on some data structure, so you need to pick what operations you think are common, and optimize those.
It's about what operations you want to speed up.
Currently you have O(1) for insertion and O(nlogn) for getMin/getMax/getMedian. You can move the logn from the getters to the insertion part by using a sorted data structure. Or you can leave the insertion as it is and optimize getMin/getMax by doing a linear search through the list and just storing the minimal value. For getMedian there is nothing to do as you need a sorted set for that.
A further optimization would be to store the min/max and update the two values during each insertion step. This will not have any (non-constant) change on insertion and will reduce your getMin/getMax to O(1). (thanks to Tedil)
The same applies for getMedian, where you would keep a sorted list in parallel to your linked list. You can then simply pick the median out of the middle of that list. Of course this will change insertion time to O(logn) or worse (depending on the list you use) and will also double the amount of storage space. So it is a more expensive optimization than the one for getMin/getMax.
To answer your latter question. please take a look at this topic to learn about the differences between using arrays or linked list in structures.
Also, as an advice, when declaring structures, always use the interface as the type so that you can change the implementation without affecting the functionality.
It depends.
Actually, one thing most people answering/commenting don't realize is that Collections.sort gives about N performance on a nearly sorted list. (N log N is the performance if the list is not at all sorted.)
As others, have said, maybe you should be sorting when you change the list rather than read the list. Maybe you should be using an Ordered Set rather than list. (If you really need a list to hold multiple copies of an item, consider using a Ordered Map and making the number of occurrences the value.)
You can also consider making a an object that stores the Collection and keeps track of the min and max without sorting it. This would work well if finding medians is rare or if you can get away from needing the median.
In the peekMaximum() and peekMinimum() methods, instead of sorting the LinkedList you can directly use the methods Collections.max(li) and Collections.min(li).
I think that will save the time wasted to sort the list.
Related
Which Java Collection should I use if I have a lot of Objects with slowly changing keys?
Basically I have one ArrayList that has all the Objects that need sorting. That ArrayList is changed by an other Thread sometimes. To iterate over it I have written the ArrayList to another ArrayList with clear and then addAll until now, but I now realized that can also cause the ConcurrentModificationException, so I want to change that.
That new Collection (currently ArrayList) that I use for iteration needs to be sorted with a custom Comparator. The problem is that, before I make the next copy of the original, the keys will change and I will have to sort it multiple times. The order for these sorts is almost correct already usually. An insertion sort may be good here.
The big question now is:
Do I use TreeSet and call Collections.sort on it when the order changed
Do I use some kind of List (which kind? ArrayList? LinkedList?) and call Collections.sort when order changed
I think the correct answer for this question can only be given when considering that I have to copy an other unsorted Collection first (ArrayList, but not necessarily) and do that while the Data in the Collection is changed by an other Thread.
My thoughts on the solution are following:
Have the second Collection be a ArrayList and update it always with Collections.copy (making it the right size first, but it should usually be the right size already in my scenario (I realize this is not atomic and can cause issues)), then call Collections.sort on it as often as I want. Maybe implement insertion sort algorithm for every sort after the initial sort, because it should be faster then.
Use TreeSet for the second collection somehow. The problem is I don't know how I would iterate over the original threadsafe to add all the elements from there. Also I don't know how to sort the Set after that initialization efficiently. I saw someone use Collections.sort, but I don't know about efficiency there. Also remember it is almost sorted.
What you need to also keep in mind, I only need iterating over both, the original and the working copy Collection, no indexing is necessary.
I have made some Java Code to show the Problem now:
public static List<FloatWrapper> pending_additions = Collections.synchronizedList(new ArrayList<>());
public static List<FloatWrapper> pending_removals = Collections.synchronizedList(new ArrayList<>());
public static List<FloatWrapper> list = new ArrayList<>();
public static Random rnd = new Random();
public static void main(String[] args) {
// initialize array
for (int i = 0; i < 1000; i++) {
list.add(new FloatWrapper(i));
}
// the main loop (runs basically infinitly)
for (int runs_infinitly = 0; runs_infinitly < 100; runs_infinitly++) {
// apply pending changes
synchronized (pending_additions) {
list.addAll(pending_additions);
pending_additions.clear();
}
synchronized (pending_removals) {
list.removeAll(pending_removals);
pending_removals.clear();
}
// doing my stuff
for (int i = 0; i < 5; i++) {
// sort array with quicksort I believe, which is not the fastest
// for this scenario usually, except for the start when its completly unsorted
System.out.println("sorting");
Collections.sort(list);
// iterate very often doing different things
for (int j = 0; j < 1; j++) {
for (FloatWrapper num : list) {
// do something with num
System.out.print(num.number + " ");
}
System.out.println();
}
System.out.println("changing keys");
// change the values that are used for sorting
for (FloatWrapper num : list) {
num.number += (rnd.nextFloat() - 0.5f) * 0.2f;
}
}
}
}
public static class FloatWrapper implements Comparable<FloatWrapper> {
public float number;
public FloatWrapper(float number) {
this.number = number;
}
public int compareTo(FloatWrapper arg0) {
return Float.compare(number, arg0.number);
}
}
The Arrays pending_additions and pending_removals are the only ones written from an other Thread. They are my improvement since I wrote this post, so not the whole List needs to be copied and resorted.
My Question still stands, should I use a TreeSet to increase performance, should I do something else different? Basically I don't know about sorting a TreeSet efficiently. I could even imagine that the ArrayList with Collection.sort() is more efficient, but I don't know. Can someone explain that.
Also I'm using a custom Comparator which has even a little bit of maths in it so it is really benificial to optimize the process of sorting here
Your current implementation already leverages that the list is partially sorted:
The Javadoc for Collections.sort writes
This implementation defers to the List.sort(Comparator) method
and the Javadoc of that method says
This implementation is a stable, adaptive, iterative mergesort that requires far fewer than n lg(n) comparisons when the input array is partially sorted, while offering the performance of a traditional mergesort when the input array is randomly ordered. If the input array is nearly sorted, the implementation requires approximately n comparisons. Temporary storage requirements vary from a small constant for nearly sorted input arrays to n/2 object references for randomly ordered input arrays.
The implementation takes equal advantage of ascending and descending order in its input array, and can take advantage of ascending and descending order in different parts of the same input array. It is well-suited to merging two or more sorted arrays: simply concatenate the arrays and sort the resulting array.
The implementation was adapted from Tim Peters's list sort for Python ( TimSort). It uses techniques from Peter McIlroy's "Optimistic Sorting and Information Theoretic Complexity", in Proceedings of the Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, pp 467-474, January 1993.
Since you are iterating over the collection far more often than sorting it, the iterations (and the actions therein) are likely far more costly than the sort. That is, you are unlikely to get a significant improvement by further tuning the sort.
What you really need is not exactly clear, but I think you should be fine with maintaining a CopyOnWriteArrayList<T>, which you then iterate with
list.stream().sorted(yourComparator).forEach(yourAction);
The CopyOnWriteArrayList is thread-safe, ie if some other thread changes it while the iteration is going, you won't get a ConcurrentModificationException and continue iterating the list as it was when you started.
EDIT: Or, since you want to iterate several times:
List<FloatWrapper> sorted = list.stream().sorted().collect(toList());
for (int i = 0; i < 5; i++) { sorted.forEach(i -> doYourStuff(i)); }
I am working on my java application and i am concerned about performance speed especially because i have a lot of data structures like List, Map and so on.
I know that complexity of add(Object obj) method of LinkedList is O(1) (the main reason to use LinkedList) and the complexity of get(int index) method of ArrayList is O(1) (the main reason to use ArrayList).
I found the article below on internet :
Try to follow these rules while optimizing ArrayList performance of your code:
Add elements to the end of the list
Remove elements from the end too
Avoid contains, indexOf and remove(Object) methods
Even more avoid removeAll and retainAll methods
Use subList(int, int).clear() idiom to quickly clean a part of the list
Now what i need to know is what does it mean add elements to the end of the list because as far as i am concerned if we do not use index as parameter of the add method so in other words if we use add(Object obj) method then elements are always added to the end.
I have those two methods below and i am wondering if the performance of those methods when we will have 1000 files will be desirable or not . If not is there any way how to improve the performance
I have used profiler like jvm to measure speed performance but i am still not sure if this is the best performance i can achieve or not
public List<int[][]> AllSharesForFiles (int n , int k,List<File> file) {
sharesHolder = new LinkedList<int[][]>();
try{
for(int i=0;i<file.size();i++) {
byte[]secret = f.readingTheFile(file.get(i)); // We call the method which read the file
//method which evaluate the shares of each byte of the file
shar1= s.calculateThresholdScheme(secret, n,k,new RndGeneratorAllAlgorithm(new RandomGeneratorSHA1()),two);
sharesHolder.add(shar1);
}
} catch (IOException e) {
e.printStackTrace();
}
return sharesHolder;
}
/*Look for the possibilities to return a 2D array instead of one dimensional array
*
*/
public List<int[][]> AllSharesForEachFile (int n , int k,List<int [] []> f) {
sharesHolder1 = new LinkedList<int[][]>();
int s=f.size();
int l=f.get(1)[0].length;
for (int i = 0; i < n; i++) {
someValue=new int[s][l];
for(int j=0;j<f.size(); j++){
someValue[j]=f.get(j)[i];
}
sharesHolder1.add(someValue);
}
return sharesHolder1;
}
ArrayList is preferable to LinkedList for the vast majority of use cases. Have a look at this Q&A for a discussion on their performance difference.
In short, ArrayList have amortised complexity of O(1) for add(Object o). They also have the benefit of having a better cache locality since all the items are allocated in a single chunk of memory rather than being scattered all over the heap, which is quite costly since it requires extra indirection.
Said that, it's good that you worry about performance but if all you have to work with is a list of 1000 items you're probably going to see a little difference between the two implementations. Your list is not that big and you're also mixing operations on the list with I/O operations, which are surely going to dominate the execution time of your application.
Here is the pseudo-code I am using to remove an item from the linked-list:
public static void removeByID(LinkedList<Fruit> fruits, int fruitID) {
for(Fruit f : fruits) {
if (f.ID == fruitID) {
fruits.remove(f);
return;
}
}
}
I am thinking this is not very efficient as fruits.remove() will once again iterate over the list. Wondering if there is a better way to achieve this.
For a java.util.LinkedList, use the Iterator.
Iterator<Fruit> it = fruits.iterator();
while(it.hasNext()) {
if(it.next().ID == fruitID) {
it.remove();
break;
}
}
This will result in only a single traversal. The Iterator returned has access to the underlying link structure and can perform a removal without iterating.
The Iterator is implicitly used anyway when you use the for-each loop form. You'd just be retaining the reference to it so you can make use of its functionality.
You may also use listIterator for O(n) insertions.
Nope, not in terms of asymptotic complexity. That's the price you pay for using a LinkedList: removals require a traversal over the list. If you want something more efficient, you need to use a different data structure.
You're in fact doing two traversals here if you've got a singly linked list: the .remove() call needs to find the parent of the given node, which it can't do without another traversal.
If you need to access elements from a collection that have a unique attribute, it is better to use a HashMap instead, with that attribute as key.
Map<Integer, Fruit> fruits = new HashMap<Integer, Fruit>();
// ...
Fruit f = fruits.remove(fruitID);
As stated in the above linked lists generally need traversal for any operation. Avoiding multiple traversal can probably be done with an iterator. Although, if you are able to relate fruit to fruit.ID ahead of time you may be able to speed up your operations because you can a void the slow iterative look up. This will still require a different data structure, namely a Map (Hashmap probably).
Regarding to your post, using a HashMap appears to a good solution.
In addition, if we suppose that you need also to search a fruit using the fruitID into your set, HashMap will make the search time barely constant.
Regarding complexity, you can find additional information on Simple Notions article depending the data structure that you use.
Edit More detailed Description of the function:
I have a HashSet filled with a specific object named Symbol. A specific function iterates through all Symbols. The function specify the energy cost of every single Symbol and return this. I want to make a new Set/Map and save the Symbols in it, ordered by this energy cost.
The next algorithm takes the Symbol with the biggest energy cost and put it in another List which is limited in cases of energy cost. When this List is full, the algorithm ends.
What should I use to save the energy cost and the Symbol? I should be able to get the energy cost and the Symbol.
I thought about a SortedMap and order it through the guava.collection like here:
https://stackoverflow.com/a/3420912
But i don't know if this is the best solution? Maybe a Set would be better for this kind of problem?
If I have understood what you want to do, what you need is:
Ensure your Symbol class implements the Comparable interface.
Instead a Set (HashSet), use a SortedSet (TreeSet).
If a Set fulfills your needings, don't change to a Map, keep it simple ;). Maps are useful when you need random access.
If you want to extract maximum on each step, you should look at heap data structure.
In Java Collections class implementing heap is PriorityQueue.
If you want to extract maximum, you should provide your own comparator to constructor, like this:
PriorityQueue<Symbol> pq = new PriorityQueue<Symbol>(0, new Comparator<Symbol>(){
#Override
public int compare(Symbol s1, Symbol s2) {
if (s1.getLong() < s2.getLong()) {
return 1;
} else if (s1.getLong() > s2.getLong()) {
return -1;
} else {
return 0;
}
}
});
Take a look, how comparator works. It should return -1 if s1 is lesser than s2, but PriorityQueue extracts minimum value, not maximum, so we invert this behavior.
Of course, if you just want to iterate over data once and don't insert anything in it keeping maximum at the head, then #Pablo answer is better.
Need a collection of strings where elements inserted needed to be sorted and also non-duplicate, can be retrieved through index.
I can use TreeSet which removes duplicates and sorts everything in
order but cannot retrieve through index. for retrieving through
index, i can make ArrayList and addAll elements to it, but this
addAll takes lot of time.
or
I can use an ArrayList, insert required and then remove duplicates by some other method, then using Collections.sort method to sort elements.
But the thing is, all these take time, is there any straight-way to achieve this, a collection -sorted, non-duplicate, with O(1) random access by index.
There's a Data Type in the commons collection called SetUniqueList that I believe meetsyour needs perfectly. Check it out:
https://commons.apache.org/proper/commons-collections/apidocs/org/apache/commons/collections4/list/SetUniqueList.html
You can use the second idea:
I can use ArrayList,insert required and then remove duplicates by some
other method, then using Collections.sort method to sort elements.
but instead of removing the duplicates before the sort, you could sort the ArrayList first, then all duplicates are on consecutive positions and can be removed in a single pass afterwards.
At this point, both your methods have the same overall complexity: O(N*logN) and it's worth noting that you cannot obtain a sorted sequence faster than this anyway (without additional exploitation of some knowledge about the values).
The real problem here is that the OP has not told us the real problem. So lots of people guess at data structures and post answers without really thinking.
The real symptom, as the OP stated in a comment, is that it takes 700ms to put the strings in a TreeSet, and another 700 ms to copy that TreeSet into an ArrayList. Obviously, the program is not doing what the OP thinks it is, as the copy should take at most a few microseconds. In fact, the program below, running on my ancient Thinkpad, takes only 360ms to create 100,000 random strings, put them in a TreeSet, and copy that TreeSet into an ArrayList.
That said, the OP has selected an answer (twice). Perhaps if/when the OP decides to think about the real problem, this example of an SSCCE will be helpful. It's CW, so feel free to edit it.
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadMXBean;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.TreeSet;
public class Microbench
{
public static void main(String[] argv)
throws Exception
{
ThreadMXBean threadBean = ManagementFactory.getThreadMXBean();
long start = threadBean.getCurrentThreadCpuTime();
executeTest();
long finish = threadBean.getCurrentThreadCpuTime();
double elapsed = (finish - start) / 1000000.0;
System.out.println(String.format("elapsed time = %7.3f ms", elapsed));
}
private static List<String> executeTest()
{
String[] data = generateRandomStrings(100000);
TreeSet<String> set = new TreeSet<String>();
for (String s : data)
set.add(s);
return new ArrayList<String>(set);
}
private static String[] generateRandomStrings(int size)
{
Random rnd = new Random();
String[] result = new String[size];
for (int ii = 0 ; ii < size ; ii++)
result[ii] = String.valueOf(rnd.nextLong());
return result;
}
}
The performance depends on how frequently the elements are added and how frequently they will be accessed by index.
I can use TreeSet which removes duplicates and sorts everything in order but cannot retrieve through index. for retrieving through index, i can make arraylist and addall elements to it, but this addAll takes lot of time.
List.addAll(yourSortedSet) will take atleast O(n) time and space each time you want to access the SortedSet as List (i.e. by the index of element).
I can use ArrayList,insert required and then remove duplicates by some other method, then using Collections.sort method to sort elements.
sorting will certainly take More than O(n) each time you want a sorted view of your list.
One more solution
If you are not fetching by the index very often then it is more efficient to do it as follows:
Just store Strings in a SortedSet may be extend TreeSet and provide/implement your own get(int i) method where you iterate till the ith element and return that element. In the worst case, this will be O(n) otherwise much lesser. This way you are not performing any comparison or conversion or copying of Strings. No extra space is needed.
I am not sure, do you test map? I mean use your string as key in a TreeMap.
In a Map, it is a O(1) for a key to find its position(a hash value). And TreeMap's keySet will return a sorted set of keys in TreeMap.
Does this fit your requirement?
If you are bound to the List at the beginning and the end of the operation, convert it into a Set with the "copy" constructor (or addAll) after the elements are populated, this removes the duplicates. If you convert it into a TreeSet with an appropriate Comparator it'll even sort it. Than, you can convert it back into a List.
Use a Hashmap you will have solved problem with unique values and sort it by some of sorting methods. If it is possible use quicksort.
Maybe using LinkedList (which takes less memory than arraylist) with boolean method which determines if that element is already in the list and a QuickSort algorithm. All structures in java have to be somehow sorted and protected from duplicates I think, so everything takes time...
there is two ways to do that use LinkedMap where each element in map is unique or make your own extention of list and override method add
import java.util.ArrayList;
public class MyList<V> extends ArrayList<V>{
private static final long serialVersionUID = 5847609794342633994L;
public boolean add(V object) {
//make each object unique
if(contains(object)){
return false;
}
//you can make here ordering and after save it at position
//your ordering here
//using extended method add
super.add(yourposition,object);
}
}
I also faced the problem of finding element at a certain position in a TreeMap. I enhanced the tree with weights that allow accessing elements by index and finding elements at indexes.
The project is called indexed-tree-map https://github.com/geniot/indexed-tree-map . The implementation for finding index of an element or element at an index in a sorted map is not based on linear iteration but on a tree binary search. Updating weights of the tree is also based on vertical tree ascent. So no linear iterations.