Related
I am working on a sharding problem.
Imagine I have 10 lists.
Each list has a series of items that are independently sorted.
I want to get the Nth item as if all the lists were sorted together in one large list.
Do I need to sort the lists overall to get an item at a particular index?
I solved a similar but not equivalent problem where there is:
10 lists
Each list represents a range of items that are after the previous list.
here's the code to iterate through all the indexes of the lists:
/* code to iterate through all items in order
* threads refers to one of the lists */
int sizes[] = new int[threads.size()];
for (int i = 0 ; i < threads.size(); i++) {
sizes[i] = threads.get(i).data2.size();
}
int n = 0;
int thread = 0;
int size = threads.size();
int offset = 0;
long iterationStart = System.nanoTime();
while (thread < size) {
// System.out.println(String.format("%d %d", thread, offset + threads.get(thread).data.get(n)));
int current = offset + threads.get(thread).data.get(n);
n = n + 1;
if (n == sizes[thread]) {
offset += sizes[thread];
thread++;
n = 0;
}
}
long iterationEnd = System.nanoTime();
long iterationTime = iterationEnd - iterationStart;
Here's the code to lookup an item by index.
int lookupKey = 329131;
int current = lookupKey;
int currentThread = 0;
int total = 0;
while (current >= 0 && currentThread <= size - 1) {
int next = current - sizes[currentThread];
if (next >= 0) {
total += sizes[currentThread];
current -= sizes[currentThread];
currentThread++;
} else {
break;
}
}
long lookupEnd = System.nanoTime();
long lookupTime = lookupEnd - lookupStart;
System.out.println(String.format("%d %d",
currentThread,
total + threads.get(currentThread).data.get(current)));
I'm hoping there's some property of sorted collections that I can use to retrieve the Nth item in an overall sorted lists.
What I have in effect is multiple partial orders.
I have some other code that does a N way merge between multiple sorted lists. Is the fastest option to run this in a loop up to lookupIndex?
int size1 = threads.size();
int[] positions = new int[size1];
Arrays.fill(positions, 0);
PriorityQueue<Tuple> pq = new PriorityQueue<>(new Comparator<Tuple>() {
#Override
public int compare(Tuple o1, Tuple o2) {
return o1.value.compareTo(o2.value);
}
});
long startOrderedIteration = System.nanoTime();
for (ShardedTotalRandomOrder thread : threads) {
for (int i = 0; i < 10; i++) {
// System.out.println(thread.data2.get(i));
pq.add(thread.data2.get(i));
}
}
List<Integer> overall = new ArrayList<>();
while (!pq.isEmpty()) {
Tuple poll = pq.poll();
ArrayList<Tuple> data2 = threads.get(poll.thread).data2;
if (positions[poll.thread] < data2.size()) {
Tuple nextValue = data2.get(positions[poll.thread]++);
pq.offer(nextValue);
}
overall.add(poll.value);
// System.out.println(String.format("%d %d", poll.thread, poll.value));
}
System.out.println(overall);
long endOrderedIteration = System.nanoTime();
long orderedIterationTime = endOrderedIteration - startOrderedIteration;
You don't need to resort them. Since each list is already sorted you can merge them as follows. This uses a single method to merge two lists based on their relative values. Then it returns that list and feeds it back into the method to merge it with the next list.
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
public class Merging {
public static void main(String[] args) {
List<Integer> list1 = List.of(5,10,15,20,25,30,35,40,45,50);
List<Integer> list2 = List.of(2,4,6,8,10);
List<Integer> list3 = List.of(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20);
int nth = 10;
List<List<Integer>> lists = List.of(list1,list2,list3);
List<Integer> merged = lists.get(0);
for (int i = 1; i < lists.size(); i++) {
merged = mergeLists(merged, lists.get(i));
}
System.out.println(merged.get(nth));
}
prints
7
This works with any type that implements the Comparable interface.
It will loop until one list is exhausted or until both indices exceed the combined list size.
Once either list is finished, the other can be appended via the sublist.
public static <T extends Comparable<? super T>> List<T> mergeLists(List<T> list1, List<T> list2) {
List<T> merged = new ArrayList<>();
int i1 = 0;
int i2 = 0;
while (i1 + i2 < list1.size() + list2.size()) {
if (i1 >= list1.size()) {
merged.addAll(list2.subList(i2,list2.size()));
break;
}
if (i2 >= list2.size()) {
merged.addAll(list1.subList(i1,list1.size()));
break;
}
if(list1.get(i1).compareTo(list2.get(i2)) <= 0) {
merged.add(list1.get(i1++));
} else {
merged.add(list2.get(i2++));
}
}
return merged;
}
}
Here is a relatively efficient (linear with respect to the number of lists) algorithm that leverages some of the power of streams, but avoids a full list merge.
EDIT: To address shortcomings such as array length checking, array destruction, and readability I have improved this example. For better comparison, I have used the same integer test data as the other answer.
This virtual queue backed by the (presumably) immutable array will not mutate or otherwise
public class VirtualQueue<T> {
private List<T> list;
private int index=0;
public VirtualQueue(List<T> list) { this.list = list; }
public boolean hasMore() { return index < list.size(); }
public T pop() { return list.get(index++); }
public T peek() { return list.get(index);}
}
(I suspect that there is an easier way to do this with standard collections)
List<Integer> list1 = List.of(5,10,15,20,25,30,35,40,45,50);
List<Integer> list2 = List.of(2,4,6,8,10);
List<Integer> list3 = List.of(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20);
List<VirtualQueue<Integer>> listList = List.of(
new VirtualQueue<>(list1),
new VirtualQueue<>(list2),
new VirtualQueue<>(list3));
int n=10;
var value = IntStream.range(0,n)
.mapToObj(i -> listList.stream()
.filter(VirtualQueue::hasMore)
.min(Comparator.comparing(l -> l.peek()))
.get().pop())
.skip(n-1).findFirst().get();
//value is now the nth item in a hypothetical merged list.
Assuming that you have k sorted Lists, and you need to obtain the n from the aggregated List (but the merged list itself don't needed), then this problem can be solved in O(n * log k) time, and by using O(k) additional space.
Note:
If the code below looks too much involved, here's the rationale behind that. This solution is more performant than straightforward comparison of elements from each list which can be observed in this and this answers, which time complexity O(n * k) (opposed to O(n * log k)). A moderate additional complexity is a cost of the performance gain, and note that it's still maintainable.
In case you would need to materialize the merged sorted list (solution below is not doing this), you can simply combine the lists together and make use of the built-in Timsort algorithm implementation via List.sort(). Timsort is very good at spotting the sorted runs, therefore sorting a list that consists of sorted chunks would have a linear time complexity.
To address the problem in O(n * log k) time, we can maintain a PriorityQueue which would always have a size of k or less (therefore enqueue/dequeue operation would have a cost of O(log k)). At the beginning, the Queue should be initialized by adding the very first element from each List.
Then we need to perform n iteration (to find the target element). At each iteration step the Head element of the Queue should be removed, and the next element originated from the same list should be added to the Queue (i.e. if let's say the 7th element from the 3rd List appear to be the Head of the Queue, then after removing it, we need to enqueue the 8th element from the 3rd List).
In order to be able to track from which List each element is coming from and what was its index in the List, we can define a custom type:
public class ElementWrapper<V extends Comparable<V>> implements Comparable<ElementWrapper<V>> {
private V value;
private int listIndex;
private int elementIndex;
// all-args constructor, getters
#Override
public int compareTo(ElementWrapper<V> o) {
return value.compareTo(o.getValue());
}
}
And here's how this algorithm for finding the n-th element can be implemented. As I've said, the time complexity is O(n * log k), since we need n iteration step with each step having a cost of O(log k). Additional memory required only to maintain a Queue of k element.
public static <T extends Comparable<T>> T getNElement(List<List<T>> lists, int n) {
Queue<ElementWrapper<T>> queue = initializeWithFirstElements(lists);
T result = null;
int count = 1;
while (!queue.isEmpty()) {
ElementWrapper<T> current = queue.remove();
if (count == n) { // target index was reached
result = current.getValue();
break;
}
count++;
if (hasNext(current, lists)) {
addNext(current, lists, queue);
}
}
return result;
}
public static <T extends Comparable<T>> Queue<ElementWrapper<T>>
initializeWithFirstElements(List<List<T>> lists) {
Queue<ElementWrapper<T>> queue = new PriorityQueue<>();
for (int i = 0; i < lists.size(); i++) {
if (lists.get(i).isEmpty()) continue;
queue.add(new ElementWrapper<>(lists.get(i).get(0), i, 0));
}
return queue;
}
public static <T extends Comparable<T>> boolean
hasNext(ElementWrapper<T> current, List<List<T>> lists) {
return current.getElementIndex() + 1 < lists.get(current.getListIndex()).size();
}
public static <T extends Comparable<T>> void
addNext(ElementWrapper<T> current, List<List<T>> lists,
Queue<ElementWrapper<T>> queue) {
ElementWrapper<T> next = new ElementWrapper<>(
lists.get(current.getListIndex()).get(current.getElementIndex() + 1),
current.getListIndex(),
current.getElementIndex() + 1
);
queue.add(next);
}
Usage example:
public static void main(String[] args) {
List<List<Integer>> input =
List.of(List.of(1, 3), List.of(),
List.of(2, 6, 7), List.of(10), List.of(4, 5, 8, 9)
);
System.out.println(getNElement(input, 1));
System.out.println(getNElement(input, 3));
System.out.println(getNElement(input, 9));
}
Output:
1 // 1st
3 // 3rd
9 // 9th
Note: depending how do you want the n-th element to be indexed the count variable in the getNElement() method should be initialized accordingly, i.e. with 1 if you want to use 1-based indexes, and with 0 if you want n to be 0-based.
import java.util.*;
public class MyClass {
public static void main(String args[]) {
int[] arr = new int[]{2,3,4,5};
int[] min = new int[]{Integer.MAX_VALUE};
int tgt = 7;
List<Integer> lst = new ArrayList<>();
List<Integer> sol = new ArrayList<>();
recursion(arr, tgt, lst, min, sol);
for (int i = 0; i < sol.size(); i++) {
System.out.println(sol.get(i));
}
}
public void recursion(int[] nums, int tgt, List<Integer> lst, int[] minLen, List<Integer> sol) {
if (tgt < 0) {
return;
}
if (tgt == 0) {
if (lst.size() <= minLen[0]) {
minLen[0] = lst.size();
}
sol = lst;
return;
}
for (int i = 0; i < nums.length; i++) {
List<Integer> cpy = new ArrayList<>(lst);
cpy.add(nums[i]);
recursion(nums, tgt - nums[i], cpy, minLen, sol);
}
}
}
The basic logic of what I'm trying to do is, given an array of numbers and a target, I want to return the smallest list of numbers that can be summed up to the target (in this case I'd want to return either {3,4} or {5,2}. I call a recursive function that breaks down my problem until I reach a base case in which I return or do some work on my sol list. lst is there to build up my current list in that recursive path while sol is there to update whenever I find a new path that is the new minimum length. However, what I think is happening is that when sol updates and returns, it becomes reset so sol is empty when it returns to main. I thought the sol list was being added to the Heap and would persist across all the calls when updated (like the minLen array)? Or am I missing something. What would be a way to get around this while maintaining my logic (I don't want to return a value but would rather have some data structure that just updates and that I could just return from my main function).
When you have a code as follows:
public void recursion(int[] nums, int tgt, List<Integer> lst, int[] minLen, List<Integer> sol) {
//... removed not relevant code
sol = lst;
}
What you are really doing is just assigning sol variable to point to a different object. That doesn't have any affect on the sol variable that you passed as an argument in the main method. That's because java passes arguments by value. In this case value is a reference to the object.
In the recursion method that reference is copied into sol variable (parameter). So, when you assign it a different value (sol = lst), you only update value of this one variable, that is local to the function.
You have two solutions here:
Just add all values into a list that sol (passed as argument) points to.
(better) just return result from within a function and assign it to new variable in main method.
It' usually a bad idea to modify objects passed into the function (unless that's the job of the function - like sort function). It's (almost) always better to return result with a return statement.
I am trying to create a method that removes every Nth element from a List of unknown type (Wildcard), however every way I try doing it, it doesn't remove the specified elements, but I cannot figure out why. I have been struggling with this for two days now, so I am posting here as a last resort. I thank you in advance for any help.
The code I currently have is as follows:
public static void removeEveryNthElement(List<?> list, int n) {
//Set list equal to an ArrayList because List is immutable
list = new ArrayList<>(list);
//If n is negative or zero throw an exception
if(n <= 0) {
throw new IllegalArgumentException("Integer n needs to be a positive number.");
}
//If the list is null, throw an exception
if(list == null) {
throw new NullPointerException("The list must not be null.");
}
//Remove every nth element in the list
for(int i = 0; i < list.size(); i++) {
if(i % n == 0) {
list.remove(i);
}
}
The other way I have attempted is replacing the for loop with the following:
list.removeIf(i -> i % 3 == 0);
However, when I do it like this I receive the error that the % operator is undefined for the argument type.
I have also tried using a for loop to individually add each element from the list into another modifiable list, but no matter what I do, I am having no luck. If you could help me with this it would be greatly appreciated!
The most serious problem with your code is that removing an element at index i changes the index of all following elements and therefore your condition for removing elements (i % n) is wrong after removing the first element.
One way to get around the problem is iterating in reverse order:
for (int i = list.size()-1; i >= 0; i--) {
if (i % n == 0) {
list.remove(i);
}
}
Another way is to increment i not by one, but by n and adjust it for the removed element:
for (int i = 0; i < list.size(); i += n) {
list.remove(i);
i--;
}
and, since i--; followed by i += n; is the same as i += n-1;:
for (int i = 0; i < list.size(); i += n-1) {
list.remove(i);
}
An additional note: the check if (list == null) is useless after the statement list = new ArrayList<>(list);, since new ArrayList<>(list); already throws a NullPointerException if list is null
You need to keep in mind that creating new collection based on other collection is removing references to oryginal collection - values from collection are coppied to new one - any modification will not impact anything that is out of method scope. You'll need to pass the collection that supports deleting an object from itself or return new collection from method. Remember that the type does not define behaviour of object - its depending on the implementation that is compatible with class you cast to. Here is an example of what I said about the backend implementation (both variables are type List but the implementation is different).
Here is code when you want to do this 'in place':
public static void main(String[] args) {
List<Integer> list2 = new ArrayList<>();
list2.add(1);
list2.add(2);
list2.add(3);
list2.add(4);
removeEveryNthElement(list2, 3); // deleted number 3 because it is 3rd element
}
public static void removeEveryNthElement(List<?> list, int n) {
for (int i = 2; i < list.size(); i += 3) {
list.remove(i);
}
}
But I would like to recommend not to do any operation that is not transparent to programmer. It's better to read and understand bigger programms when you know that you pass value to method and 'it does something' then get value back because it got changed. For this example I use generics and streams:
public static void main(String[] args) {
List<Integer> list1 = Arrays.asList(1, 2, 3, 4);
list1 = removeEveryNthElement2(list1, 3); //deleted number 3
System.out.println();
}
public static <T> List<T> removeEveryNthElement2(List<T> list, int n) {
final Predicate<T> p = new Predicate<T>() {
int i = 0;
#Override
public boolean test(final T t) {
return ++i % n != 0;
}
};
return list.stream().filter(p).collect(Collectors.toList());
}
Brandon, let me first suggest that your method is side-effecting the list built in the calling method. This is allowed, but can lead to hard-to-understand bugs in more complicated code. Instead, try making a new list in your method, and assigned the returned value:
public class Remove {
public static void main(String[] args) {
List<String> list = Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h");
list = removeElements(list, 3);
System.out.println(list);
}
public static <T> List<T> removeElements(List<T> list, int n) {
List<T> newList = new ArrayList<T>();
for (int i = 0; i < list.size(); i++) {
if (i % n != 0) {
newList.add(list.get(i));
}
}
return newList;
}
}
As a result, it makes the method simpler, because we're no longer iterating over the list being modified.
Have a look at Side effect--what's this?
to read more about side-effecting.
I would like to create an array list such that when I print ArrayList.size(), I want it to display 100 or some other large number.
Scenario
It's for a program where I'm trying to find the minimum size of array list that fits a condition. So I want the initial value of the ArrayList to be large, so that it keeps getting over-written by a smaller combination.
Similarly to a program to find the smallest number where you initialize the variable smallest to a large number so that it gets over-written in a linear traversal.
EDIT : This is the question I am trying to solve
To find the minimum number of coins of denomination 3 and 5 to make change of 4. Yes, it should basically return that it's not possible.
I have done this correctly if the question is only to find the number of coins
public class FindCount
{
public static void main(String[] args)
{
int coins = fun(4);
if(coins == 999)
System.out.println("Not possible to make change with this denomination");
else
System.out.println(coins);
}
static int fun(int n)
{
if(n == 0)
return 0;
int ret1 = 999;
int ret2 = 999;
if(n >= 3)
{
int rec1 = fun(n-3);
if(rec1 != 999)
ret1 = 1+rec1;
}
if(n >= 5)
{
int rec2 = fun(n-5);
if(rec2 != 999)
ret2 = 1+rec2;
}
int totalret = min(ret1, ret2);
return totalret;
}
static int min(int a, int b)
{
return a<b?a:b;
}
}
Note that above, I have used a variable called totalret which, if 999, I can use it to declare that it's not possible to achieve this result.
However, if I'm asked to print the elements required instead of just finding count, the program goes wrong if there is any path where it's not possible to make change. This is because, here totalret is initialized to null and not to a large ArrayList of say 999 elements, as in the program above.
import java.util.ArrayList;
public class PrintSequence
{
static int[] options = {3,5};
public static void main(String[] args)
{
ArrayList<Integer> result = new ArrayList<Integer>();
result = fun(4);
for(int i = 0;i<result.size();i++)
System.out.print(result.get(i));
}
static ArrayList<Integer> fun(int n)
{
int smallest = 999;
ArrayList<Integer> totalret = null;
for(int i = 0;i<options.length;i++)
{
if(n-options[i] >= 0)
{
ArrayList<Integer> reci = fun(n-options[i]);
ArrayList<Integer> reti = new ArrayList<Integer>();
reti.addAll(reci);
reti.add(0,options[i]);
if(reti.size() < smallest)
{
totalret = reti;
smallest = reti.size();
}
}
}
if(totalret == null)
totalret = new ArrayList<Integer>();
return totalret;
}
}
Please suggest how I can edit the printing program to print that this is not possible.
One-liner to create an ArrayList of size 100:
new ArrayList<Object>(Arrays.asList(new Object[100]))
Update
Replace both Object with any suitable (non-primitive) type.
If you don't need it to be an ArrayList, but just a List that cannot change size, the following is good (showing for String as an example):
Arrays.asList(new String[100])
This might help you, though it is not a one liner.
Integer[] intArray = new Integer[100];
ArrayList arr = new ArrayList<Integer>(Arrays.asList(intArray));
int size = arr.size();
System.out.println(size);
Hmmm. If you have to use an ArrayList, the only way that I can think of initializing its size would be to fill it up with "dummy" values i.e. loop and add some value up to a certain Nth size. Would that be okay/feasible in your case?
Of course, if anyone else has a better solution, feel free to comment.
Hope this is can help you.
Integer[] is = new Integer[100 + new Random().nextInt(500)];
Arrays.fill(is, 0);
List<Integer> list = Arrays.asList(is);
i have 2 sequences, for instance s=aaba and ss=aa, and i want all the way ss is in s.
In this example:
[0,1], [0,3] and [1,3]
My code is below. It works fine, except for very long s with multiple ss. In that case i've got
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
(I already use java with -Xmx at the maximum I can…)
public static ArrayList<ArrayList<Integer>> getListIndex(String[] s, String[] ss, int is, int iss) {
ArrayList<ArrayList<Integer>> listOfListIndex = new ArrayList<ArrayList<Integer>>();
ArrayList<ArrayList<Integer>> listRec = new ArrayList<ArrayList<Integer>>();
ArrayList<Integer> listI = new ArrayList<Integer>();
if (iss<0||is<iss){
return listOfListIndex;
}
if (ss[iss].compareTo(s[is])==0){
//ss[iss] matches, search ss[0..iss-1] in s[0..is-1]
listRec = getListIndex(s,ss,is-1,iss-1);
//empty lists (iss=0 for instance)
if(listRec.size()==0){
listI = new ArrayList<Integer>();
listI.add(is);
listOfListIndex.add(listI);
}
else{
//adding to what we have already found
for (int i=0; i<listRec.size();i++){
listI = listRec.get(i);
listI.add(is);
listOfListIndex.add(listI);
}
}
}
//In all cases
//searching ss[0..iss] in s[0..is-1]
listRec = getListIndex(s,ss,is-1,iss);
for (int i=0; i<listRec.size();i++){
listI = listRec.get(i);
listOfListIndex.add(listI);
}
return listOfListIndex;
}
Is there anyway to do this more efficiently ?
I doubt the recursion is the problem (think of what the maximum recursion depth is). The algorithm can be efficiently implemented by collecting the indecies of each character of s in ss in TreeSets and then simply taking the .tailSet when needing to "advance" in the string.
import java.util.*;
public class Test {
public static Set<List<Integer>> solutions(List<TreeSet<Integer>> is, int n) {
TreeSet<Integer> ts = is.get(0);
Set<List<Integer>> sol = new HashSet<List<Integer>>();
for (int i : ts.tailSet(n+1)) {
if (is.size() == 1) {
List<Integer> l = new ArrayList<Integer>();
l.add(i);
sol.add(l);
} else
for (List<Integer> tail : solutions(is.subList(1, is.size()), i)) {
List<Integer> l = new ArrayList<Integer>();
l.add(i);
l.addAll(tail);
sol.add(l);
}
}
return sol;
}
public static void main(String[] args) {
String ss = "aaba";
String s = "aa";
List<TreeSet<Integer>> is = new ArrayList<TreeSet<Integer>>();
// Compute all indecies of each character.
for (int i = 0; i < s.length(); i++) {
TreeSet<Integer> indecies = new TreeSet<Integer>();
char c = s.charAt(i);
for (int j = 0; j < ss.length(); j++) {
if (ss.charAt(j) == c)
indecies.add(j);
}
is.add(indecies);
}
System.out.println(solutions(is, -1));
}
}
Output:
[[0, 1], [1, 3], [0, 3]]
ArrayList<Integer> is quite memory-inefficient due to the overhead of the wrapper class. Using TIntArrayList from GNU Trove will probably cut down your memory usage by a factor of 3 (or even more if you're running on a 64bit JVM).
Well, the basic problem is that your algorithm is recursive. Java doesn't do tail-call optimization, so every recursive call just adds to the stack until you overflow.
What you want to do is re-structure your algorithm to be iterable so you aren't adding to the stack. Think about putting a loop (with a termination test) as the outer-most element of your method instead.
Another way to look at this problem is to break it into two steps:
Capture the positions of all of the given character ('a' in your example) into a single set.
All you want here is the complete set of combinations between them. Remember that the equation for number of combinations of r things chosen from n different things is:
C(n,r) = n!/[r!(n-r)!]