Develop Reference

Generate random number only one time

I have a method who randomize a number between 1 and 5.
This method needs to stop when all 5 numbers are sorted too (isn't doing it now)
My actual code is:
public ArrayList<String> generated = new ArrayList<String>(); on top
And the method itself is:
public int RandomizeQuestion () {
// Question numbers
final int min = 1;
final int max = 5;
Random r = new Random();
int qran = r.nextInt((max - min) + 1) + min;
if (Collections.singletonList(generated).contains(qran)) {
RandomizeQuestion();
} else {
generated.add(String.valueOf(qran));
}
return qran;
}
But that happens is all time the random number appears as new and this number is added repeatedly in generator array.

Collections.singletonList(generated) returns a List<ArrayList<String>> containing generated as its only element. Obviously that won't contain qran. Also, you're not checking if the list is the expected length before attempting to add a new item. Something like this should work:
if (generated.size() < max - min + 1) {
if (generated.contains(String.valueOf(qran))) {
RandomizeQuestion();
} else {
generated.add(String.valueOf(qran));
}
}
But here's a much simpler and more efficient way to shuffle a sequence of values:
// generate a list of values between min and max
generated = IntStream.rangeClosed(min, max)
.map(String::valueOf)
.collect(Collectors.toCollection(ArrayList::new));
// shuffle them
Collections.shuffle(generated);

Why is Arrays.binarySearch not improving the performance compared to walking the array?

I gave a shot at solving the Hackerland Radio Transmitters programming challange.
To summarize, challenge goes as follows:
Hackerland is a one-dimensional city with n houses, where each house i is located at some xi on the x-axis. The Mayor wants to install radio transmitters on the roofs of the city's houses. Each transmitter has a range, k, meaning it can transmit a signal to all houses ≤ k units of distance away.
Given a map of Hackerland and the value of k, can you find the minimum number of transmitters needed to cover every house?
My implementation is as follows:
package biz.tugay;
import java.util.*;
public class HackerlandRadioTransmitters {
public static int minNumOfTransmitters(int[] houseLocations, int transmitterRange) {
// Sort and remove duplicates..
houseLocations = uniqueHouseLocationsSorted(houseLocations);
int towerCount = 0;
for (int nextHouseNotCovered = 0; nextHouseNotCovered < houseLocations.length; ) {
final int towerLocation = HackerlandRadioTransmitters.findNextTowerIndex(houseLocations, nextHouseNotCovered, transmitterRange);
towerCount++;
nextHouseNotCovered = HackerlandRadioTransmitters.nextHouseNotCoveredIndex(houseLocations, towerLocation, transmitterRange);
if (nextHouseNotCovered == -1) {
break;
}
}
return towerCount;
}
public static int findNextTowerIndex(final int[] houseLocations, final int houseNotCoveredIndex, final int transmitterRange) {
final int houseLocationWeWantToCover = houseLocations[houseNotCoveredIndex];
final int farthestHouseLocationAllowed = houseLocationWeWantToCover + transmitterRange;
int towerIndex = houseNotCoveredIndex;
int loop = 0;
while (true) {
loop++;
if (towerIndex == houseLocations.length - 1) {
break;
}
if (farthestHouseLocationAllowed >= houseLocations[towerIndex + 1]) {
towerIndex++;
continue;
}
break;
}
System.out.println("findNextTowerIndex looped : " + loop);
return towerIndex;
}
public static int nextHouseNotCoveredIndex(final int[] houseLocations, final int towerIndex, final int transmitterRange) {
final int towerCoversUntil = houseLocations[towerIndex] + transmitterRange;
int notCoveredHouseIndex = towerIndex + 1;
int loop = 0;
while (notCoveredHouseIndex < houseLocations.length) {
loop++;
final int locationOfHouseBeingChecked = houseLocations[notCoveredHouseIndex];
if (locationOfHouseBeingChecked > towerCoversUntil) {
break; // Tower does not cover the house anymore, break the loop..
}
notCoveredHouseIndex++;
}
if (notCoveredHouseIndex == houseLocations.length) {
notCoveredHouseIndex = -1;
}
System.out.println("nextHouseNotCoveredIndex looped : " + loop);
return notCoveredHouseIndex;
}
public static int[] uniqueHouseLocationsSorted(final int[] houseLocations) {
Arrays.sort(houseLocations);
final HashSet<Integer> integers = new HashSet<>();
final int[] houseLocationsUnique = new int[houseLocations.length];
int innerCounter = 0;
for (int houseLocation : houseLocations) {
if (integers.contains(houseLocation)) {
continue;
}
houseLocationsUnique[innerCounter] = houseLocation;
integers.add(houseLocationsUnique[innerCounter]);
innerCounter++;
}
return Arrays.copyOf(houseLocationsUnique, innerCounter);
}
}
I am pretty sure this implementation is correct. But please see the detail in the functions: findNextTowerIndex and nextHouseNotCoveredIndex: they walk the array one by one!
One of my tests is as follows:
static void test_01() throws FileNotFoundException {
final long start = System.currentTimeMillis();
final File file = new File("input.txt");
final Scanner scanner = new Scanner(file);
int[] houseLocations = new int[73382];
for (int counter = 0; counter < 73382; counter++) {
houseLocations[counter] = scanner.nextInt();
}
final int[] uniqueHouseLocationsSorted = HackerlandRadioTransmitters.uniqueHouseLocationsSorted(houseLocations);
final int minNumOfTransmitters = HackerlandRadioTransmitters.minNumOfTransmitters(uniqueHouseLocationsSorted, 73381);
assert minNumOfTransmitters == 1;
final long end = System.currentTimeMillis();
System.out.println("Took: " + (end - start) + " milliseconds..");
}
where input.txt can be downloaded from here. (It is not the most important detail in this question, but still..) So we have an array of 73382 houses, and I deliberately set the transmitter range so the methods I have loop a lot:
Here is a sample output from this test in my machine:
findNextTowerIndex looped : 38213
nextHouseNotCoveredIndex looped : 13785
Took: 359 milliseconds..
I also have this test, which does not assert anything, but just keeps time:
static void test_02() throws FileNotFoundException {
final long start = System.currentTimeMillis();
for (int i = 0; i < 400; i ++) {
final File file = new File("input.txt");
final Scanner scanner = new Scanner(file);
int[] houseLocations = new int[73382];
for (int counter = 0; counter < 73382; counter++) {
houseLocations[counter] = scanner.nextInt();
}
final int[] uniqueHouseLocationsSorted = HackerlandRadioTransmitters.uniqueHouseLocationsSorted(houseLocations);
final int transmitterRange = ThreadLocalRandom.current().nextInt(1, 70000);
final int minNumOfTransmitters = HackerlandRadioTransmitters.minNumOfTransmitters(uniqueHouseLocationsSorted, transmitterRange);
}
final long end = System.currentTimeMillis();
System.out.println("Took: " + (end - start) + " milliseconds..");
}
where I randomly create 400 transmitter ranges, and run the program 400 times.. I will get run times as follows in my machine..
Took: 20149 milliseconds..
So now, I said, why don 't I use binary search instead of walking the array and changed my implementations as follows:
public static int findNextTowerIndex(final int[] houseLocations, final int houseNotCoveredIndex, final int transmitterRange) {
final int houseLocationWeWantToCover = houseLocations[houseNotCoveredIndex];
final int farthestHouseLocationAllowed = houseLocationWeWantToCover + transmitterRange;
int nextTowerIndex = Arrays.binarySearch(houseLocations, 0, houseLocations.length, farthestHouseLocationAllowed);
if (nextTowerIndex < 0) {
nextTowerIndex = -nextTowerIndex;
nextTowerIndex = nextTowerIndex -2;
}
return nextTowerIndex;
}
public static int nextHouseNotCoveredIndex(final int[] houseLocations, final int towerIndex, final int transmitterRange) {
final int towerCoversUntil = houseLocations[towerIndex] + transmitterRange;
int nextHouseNotCoveredIndex = Arrays.binarySearch(houseLocations, 0, houseLocations.length, towerCoversUntil);
if (-nextHouseNotCoveredIndex > houseLocations.length) {
return -1;
}
if (nextHouseNotCoveredIndex < 0) {
nextHouseNotCoveredIndex = - (nextHouseNotCoveredIndex + 1);
return nextHouseNotCoveredIndex;
}
return nextHouseNotCoveredIndex + 1;
}
and I am expecting a great performance boost, as now I will at most loop for log(N) times, instead of O(N).. So test_01 outputs:
Took: 297 milliseconds..
Remember, it was Took: 359 milliseconds.. before. And for test_02:
Took: 18047 milliseconds..
So I always get values around 20 seconds with array walking implementation and 18 - 19 seconds for binary search implementation.
I was expecting a much better performance gain using Arrays.binarySearch but obviously it is not the case, why is this? What am I missing? Do I need an array with more than 73382 to see the benefit, or is it irrelevant?
Edit #01
After #huck_cussler 's comment, I tried doubling and tripling the dataset I have (with random numbers) and tried running test02 (of course with tripling the array sizes in the test itself..). For the linear implementation the times go like this:
Took: 18789 milliseconds..
Took: 34396 milliseconds..
Took: 53504 milliseconds..
For the binary search implementation, I got values as follows:
Took: 18644 milliseconds..
Took: 33831 milliseconds..
Took: 52886 milliseconds..

Your timing includes the retrieval of data from your hard drive. This could be taking the majority of your runtime. Omit the data load from your timing to get a more accurate comparison of your two approaches. Imagine if it takes up 18 seconds and you're comparing 18.644 vs 18.789 (0.77% improvement) instead of 0.644 vs 0.789 (18.38% improvement).
If you have a linear operation O(n), such as loading a binary structure, and you combine it with a binary search O(log n), you end up with O(n). If you trust Big O notation, then you should expect O(n + log n) to not be significantly different from O(2 * n) as they both reduce to O(n).
Also, a binary search may perform better or worse than a linear search depending on the density of houses between towers. Consider, say 1024 homes with a tower evenly dispersed every 4 homes. A linear search will step 4 times per tower, while a binary search will take log2(1024)=10 steps per tower.
One more thing... your minNumOfTransmitters method is sorting the already-sorted array passed into it from test_01 and test_02. That resorting step takes longer than your searches themselves, which further obscures the timing differences between your two search algorithms.
======
I created a small timing class to give a better picture of what's happening. I've removed the line of code from minNumOfTransmitters to prevent it from rerunning the sort, and added a boolean param to select whether to use your binary version. It totals the sum of times for 400 iterations, separating out each step. The results on my system illustrate that the load time dwarfs the sort time, which in turn dwarfs the solve time.
Load: 22.565s
Sort: 4.518s
Linear: 0.012s
Binary: 0.003s
It's easy to see how optimizing that last step doesn't make much difference in overall runtime.
private static class Timing {
public long load=0;
public long sort=0;
public long solve1=0;
public long solve2=0;
private String secs(long millis) {
return String.format("%3d.%03ds", millis/1000, millis%1000);
}
public String toString() {
return " Load: " + secs(load) + "\n Sort: " + secs(sort) + "\nLinear: " + secs(solve1) + "\nBinary: " + secs(solve2);
}
public void add(Timing timing) {
load+=timing.load;
sort+=timing.sort;
solve1+=timing.solve1;
solve2+=timing.solve2;
}
}
static Timing test_01() throws FileNotFoundException {
Timing timing=new Timing();
long start = System.currentTimeMillis();
final File file = new File("c:\\path\\to\\xnpwdiG3.txt");
final Scanner scanner = new Scanner(file);
int[] houseLocations = new int[73382];
for (int counter = 0; counter < 73382; counter++) {
houseLocations[counter] = scanner.nextInt();
}
timing.load+=System.currentTimeMillis()-start;
start=System.currentTimeMillis();
final int[] uniqueHouseLocationsSorted = HackerlandRadioTransmitters.uniqueHouseLocationsSorted(houseLocations);
timing.sort=System.currentTimeMillis()-start;
start=System.currentTimeMillis();
final int minNumOfTransmitters = HackerlandRadioTransmitters.minNumOfTransmitters(uniqueHouseLocationsSorted, 73381, false);
timing.solve1=System.currentTimeMillis()-start;
start=System.currentTimeMillis();
final int minNumOfTransmittersBin = HackerlandRadioTransmitters.minNumOfTransmitters(uniqueHouseLocationsSorted, 73381, true);
timing.solve2=System.currentTimeMillis()-start;
final long end = System.currentTimeMillis();
return timing;
}

In your time measurement you include operations that are much slower than array search. Namely filesystem I/O and array sorting.
I/O in general (reading/writing from filesystem, network communication) is by orders of magnitude slower than operations that involve only CPU and RAM access.
Let's rewrite your test in a way that does not read the file on every loop iteration:
static void test_02() throws FileNotFoundException {
final File file = new File("input.txt");
final Scanner scanner = new Scanner(file);
int[] houseLocations = new int[73382];
for (int counter = 0; counter < 73382; counter++) {
houseLocations[counter] = scanner.nextInt();
}
scanner.close();
final int rounds = 400;
final int[] uniqueHouseLocationsSorted = uniqueHouseLocationsSorted(houseLocations);
final int transmitterRange = 73381;
final long start = System.currentTimeMillis();
for (int i = 0; i < rounds; i++) {
final int minNumOfTransmitters = minNumOfTransmitters(uniqueHouseLocationsSorted, transmitterRange);
}
final long end = System.currentTimeMillis();
System.out.println("Took: " + (end - start) + " milliseconds..");
}
Notice in this version of the test the file is read only once and time measuring starts after that.
With the above, I get Took: 1700 milliseconds.. (more or less a few millis) for both the iterative version and the binary search. So we still can't see that binary search is faster. That's because almost all of that time goes into sorting the array 400 times.
Now let's remove the line that sorts the input array from the minNumOfTransmitters method. We sort the array (once) anyway at the beginning of the test.
Now we can see that things are much faster. After removing the line houseLocations = uniqueHouseLocationsSorted(houseLocations) from minNumOfTransmitters I get: Took: 68 milliseconds.. for the iterative version. Clearly, since this duration is already very small, we will not see a significant difference with the binary search version.
So let's increase the number of loop rounds to: 100000.
Now I get Took: 2121 milliseconds.. for the iterative version and Took: 36 milliseconds.. for the binary search version.
Because we now isolated what we measure and focus on the array searches, rather than including operations that are much slower, we can notice the big difference in performance (for the better) of binary search.
If you want to see how many times binary search enters its while loop, you can implement it yourself and add a counter:
private static int binarySearch0(int[] a, int fromIndex, int toIndex, int key) {
int low = fromIndex;
int high = toIndex - 1;
int loop = 0;
while (low <= high) {
loop++;
int mid = (low + high) >>> 1;
int midVal = a[mid];
if (midVal < key) {
low = mid + 1;
} else if (midVal > key) {
high = mid - 1;
} else {
return mid; // key found
}
}
System.out.println("binary search looped " + loop + " times");
return -(low + 1); // key not found.
}
The method is copied from the Arrays class in the JDK - I just added the loop counter and the println.
When the length of the array to search is 73382, the loop enters only 16 times.
That is exactly what we expect: log(73382) =~ 16.

I agree with other answers that the main issue with your tests is that they measure wrong things: IO and sorting. But I don't think suggested tests are good. My suggestion is following:
static void test_02() throws FileNotFoundException {
final File file = new File("43620487.txt");
final Scanner scanner = new Scanner(file);
int[] houseLocations = new int[73382];
for (int counter = 0; counter < 73382; counter++) {
houseLocations[counter] = scanner.nextInt();
}
final int[] uniqueHouseLocationsSorted = uniqueHouseLocationsSorted(houseLocations);
final Random random = new Random(0); // fixed seed to have the same sequences in all tests
long sum = 0;
// warm up
for (int i = 0; i < 100; i++) {
final int transmitterRange = random.nextInt(70000) + 1;
final int minNumOfTransmitters = minNumOfTransmitters(uniqueHouseLocationsSorted, transmitterRange);
sum += minNumOfTransmitters;
}
// actual measure
final long start = System.currentTimeMillis();
for (int i = 0; i < 4000; i++) {
final int transmitterRange = random.nextInt(70000) + 1;
final int minNumOfTransmitters = minNumOfTransmitters(uniqueHouseLocationsSorted, transmitterRange);
sum += minNumOfTransmitters;
}
final long end = System.currentTimeMillis();
System.out.println("Took: " + (end - start) + " milliseconds. Sum = " + sum);
}
Note also that I remove all System.out.println calls from findNextTowerIndex and nextHouseNotCoveredIndex and uniqueHouseLocationsSorted call from minNumOfTransmitters as they affect performance testing as well.
So what I think is important here:
Move all I/O and sorting out of the measurement loop
Perform some warm up outside of measurement
Use the same random sequence for all measurements
Don't dispose result of the calculation so JIT can't optimize that call out altogether
With such test I see about 10 times difference on my machine: around 80ms vs around 8ms.
And if you really want to do performance tests in Java you should consider using JMH aka Java Microbenchmark Harness

Agree with other answers, the IO time is most problem, and sort is second, the search is last time consumer.
And agree phatfingers's example, the binary search sometime is worst than linear search in your problem because totally linear search goes one loop for every element(n times compare) but binary search run for tower times (O(logn)*#tower)), one suggestion is that binary search not start from 0, but from current location
int nextTowerIndex = Arrays.binarySearch(houseLocations, houseNotCoveredIndex+1, houseLocations.length, arthestHouseLocationAllowed)
then it should O(logn)*#tower/2)
Even more, maybe you can calculate every tower cover how many houses avg then first compare avg houses then using binary search start from houseNotCoveredIndex + avg + 1, but not sure the performance is much better.
ps: sort and unique you can using TreeSet as
public static int[] uniqueHouseLocationsSorted(final int[] houseLocations) {
final Set<Integer> integers = new TreeSet<>();
for (int houseLocation : houseLocations) {
integers.add(houseLocation);
}
int[] unique = new int[integers.size()];
int i = 0;
for(Integer loc : integers){
unique[i] = loc;
i++;
}
return unique;
}

uniqueHouseLocationsSorted is not efficient, andy solution seems better, but I think this could improve the time spent (note that I did not test the code):
public static int[] uniqueHouseLocationsSorted(final int[] houseLocations) {
int size = houseLocations.length;
if (size == 0) return null; // you have to check for null later or maybe throw an exception here
Arrays.sort(houseLocations);
final int[] houseLocationsUnique = new int[size];
int previous = houseLocationsUnique[0] = houseLocations[0];
int innerCounter = 1;
for (int i = 1; i < size; i++) {
int houseLocation = houseLocations[i];
if (houseLocation == previous) continue; // since elements are sorted this is faster
previous = houseLocationsUnique[innerCounter++] = houseLocation;
}
return Arrays.copyOf(houseLocationsUnique, innerCounter);
}
Consider also using an Array list as copying the array takes time.

ProjectEuler solution returns incorrect answer

The problem I'm trying to solve comes from ProjectEuler.
Some integers have following property:
n + reverse(n) = a number consisting entirely of odd digits.
For example:
14: 14 + 41 = 55
Numbers starting or ending with 0 aren't allowed.
How many of these "reversible" numbers are there below 10^9?
The problem also gives a hint:
there are 120 such numbers below 1000.
I'm quite new to Java, and I tried to solve this problem by writing a program that checks all the numbers up to a billion, which is not the best way, I know, but I'm ok with that.
The problem is that my program gives out a wrong amount of numbers and I couldn't figure out why! (The code will most likely contain some ugly things, feel free to improve it in any way)
int result = 0;
boolean isOdd = true;
boolean hasNo0 = true;
public int reverseNumber(int r) //this method should be working
{ //guess the main problem is in the second method
int n = 0;
String m = "";
if (r % 10 == 0) { hasNo0 = false; }
while (r > 0){
n = r % 10;
m = String.valueOf(m+n);
r /= 10;
}
result = Integer.parseInt(m);
return result;
}
public void isSumOdd(int max)
{
int number = 1;
int sum = 0;
Sums reverseIt = new Sums();
int amount = 0;
while (number <= max)
{
sum = reverseIt.reverseNumber(number) + number;
while (sum > 0)
{
int x = sum % 10;
if (x % 2 == 0) { isOdd = false; }
sum /= 10;
}
if (isOdd && hasNo0) { amount++; }
number++;
isOdd = true;
hasNo0 = true;
}
System.out.println(amount);
}
Called by
Sums first = new Sums();
first.reversibleNumbers(1000000000);

The most important problem in your code is the following line:
sum = reverseIt.reverseNumber(number) + number;
in isSumOdd(int max) function. Here the reverseIt object is a new instance of Sums class. Since you are using Sums member data (the boolean variables) to signal some conditions when you use the new instance the value of these member variables is not copied to the current caller object. You have to change the line to:
sum = this.reverseNumber(number) + number;
and remove the Sums reverseIt = new Sums(); declaration and initialization.
Edit: Attempt to explain why there is no need to instantiate new object instance to call a method - I've found the following answer which explains the difference between a function and a (object)method: https://stackoverflow.com/a/155655/25429. IMO the explanation should be enough (you don't need a new object because the member method already has access to the member data in the object).

You overwrite odd check for given digit when checking the next one with this code: isOdd = false;. So in the outcome you check only whether the first digit is odd.
You should replace this line with
idOdd = idOdd && (x % 2 == 0);
BTW. You should be able to track down an error like this easily with simple unit tests, the practice I would recommend.

One of the key problems here is that your reverseNumber method does two things: check if the number has a zero and reverses the number. I understand that you want to ignore the result (or really, you have no result) if the number is a multiple of 10. Therefore, you have two approaches:
Only send numbers into reverseNumber if they are not a multiple of 10. This is called a precondition of the method, and is probably the easiest solution.
Have a way for your method to give back no result. This is a popular technique in an area of programming called "Functional Programming", and is usually implemented with a tool called a Monad. In Java, these are implemented with the Optional<> class. These allow your method (which always has to return something) to return an object that means "nothing at all". These will allow you to know if your method was unable or unwilling to give you a result for some reason (in this case, the number had a zero in it).

I think that separating functionnalities will transform the problem to be easier. Here is a solution for your problem. Perhaps it isn't the best but that gives a good result:
public static void main(final String [] args) {
int counter = 0;
for (int i = 0; i < 20; i++) {
final int reversNumber = reverseNumber(i);
final int sum = i + reversNumber;
if (hasNoZeros(i) && isOdd(sum)) {
counter++;
System.out.println("i: " + i);
System.out.println("r: " + reversNumber);
System.out.println("s: " + sum);
}
}
System.out.println(counter);
}
public static boolean hasNoZeros(final int i){
final String s = String.valueOf(i);
if (s.startsWith("0") || s.endsWith("0")) {
return false;
}
return true;
}
public static int reverseNumber(final int i){
final StringBuilder sb = new StringBuilder(String.valueOf(i));
return Integer.parseInt(sb.reverse().toString());
}
public static boolean isOdd(final int i){
for (final char s : String.valueOf(i).toCharArray()) {
if (Integer.parseInt(String.valueOf(s))%2 == 0) {
return false;
}
}
return true;
}
the output is:
i: 12
r: 21
s: 33
i: 14
r: 41
s: 55
i: 16
r: 61
s: 77
i: 18
r: 81
s: 99
4

Here is a quick working snippet:
class Prgm
{
public static void main(String args[])
{
int max=(int)Math.pow(10, 3); //change it to (10, 9) for 10^9
for(int i=1;i<=max;i++)
{
if(i%10==0)
continue;
String num=Integer.toString(i);
String reverseNum=new StringBuffer(num).reverse().toString();
String sum=(new Long(i+Long.parseLong(reverseNum))).toString();
if(sum.matches("^[13579]+$"))
System.out.println(i);
}
}
}
It prints 1 number(satisfying the condition) per line, wc is word count linux program used here to count number of lines
$javac Prgm.java
$java Prgm
...//Prgm outputs numbers 1 per line
$java Prgm | wc --lines
120

TreeSet search taking long time ,puzzle: to find lucky numbers

It actually is problem to find lucky number - those numbers whose sum of digits and sum of square of digits are prime. I have implemented Sieve of Eratosthenes. Now to optimize it further I commented my getDigitSum method, that I suppose was heavy and replaced with two hard-coded value , but it is still taking minutes to solve one test case. Here is a reference to actual problem asked
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Set;
import java.util.TreeSet;
public class Solution {
private static int[] getDigitSum(long num) {
long sum = 0;
long squareSum = 0;
for (long tempNum = num; tempNum > 0; tempNum = tempNum / 10) {
if (tempNum < 0) {
sum = sum + tempNum;
squareSum = squareSum + (tempNum * tempNum);
} else {
long temp = tempNum % 10;
sum = sum + temp;
squareSum = squareSum + (temp * temp);
}
}
int[] twosums = new int[2];
twosums[0] = Integer.parseInt(sum+"");
twosums[1] = Integer.parseInt(squareSum+"");
// System.out.println("sum Of digits: " + twoDoubles[0]);
// System.out.println("squareSum Of digits: " + twoDoubles[1]);
return twosums;
}
public static Set<Integer> getPrimeSet(int maxValue) {
boolean[] primeArray = new boolean[maxValue + 1];
for (int i = 2; i < primeArray.length; i++) {
primeArray[i] = true;
}
Set<Integer> primeSet = new TreeSet<Integer>();
for (int i = 2; i < maxValue; i++) {
if (primeArray[i]) {
primeSet.add(i);
markMutiplesAsComposite(primeArray, i);
}
}
return primeSet;
}
public static void markMutiplesAsComposite(boolean[] primeArray, int value) {
for (int i = 2; i*value < primeArray.length; i++) {
primeArray[i * value] = false;
}
}
public static void main(String args[]) throws NumberFormatException,
IOException {
// getDigitSum(80001001000l);
//System.out.println(getPrimeSet(1600));
Set set = getPrimeSet(1600);
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
int totalCases = Integer.parseInt(br.readLine());
for (int cases = 0; cases < totalCases; cases++) {
String[] str = br.readLine().split(" ");
long startRange = Long.parseLong(str[0]);
long endRange = Long.parseLong(str[1]);
int luckyCount = 0;
for (long num = startRange; num <= endRange; num++) {
int[] longArray = getDigitSum(num); \\this method was commented for testing purpose and was replaced with any two hardcoded values
if(set.contains(longArray[0]) && set.contains(longArray[1])){
luckyCount++;
}
}
System.out.println(luckyCount);
}
}
}
what I should use to cache the result so that it takes lesser amount of time to search, currently it takes huge no. of minutes to complete 10000 test cases with range 1 99999999999999(18 times 9 -the worst case) , even thought the search values have been hard-coded for testing purpose( 1600, 1501 ).

You need a different algorithm. Caching is not your problem.
If the range is large - and you can bet some will be - even a loop doing almost nothing would take a very long time. The end of the range is constrained to be no more than 1018, if I understand correctly. Suppose the start of the range is half that. Then you'd iterate over 5*1017 numbers. Say you have a 2.5 GHz CPU, so you have 2.5*109 clock cycles per second. If each iteration took one cycle, that'd be 2*108 CPU-seconds. A year has about 3.1*107 seconds, so the loop would take roughly six and a half years.
Attack the problem from the other side. The sum of the squares of the digits can be at most 18*92, that's 1458, a rather small number. The sum of the digits itself can be at most 18*9 = 162.
For the primes less than 162, find out all possible decompositions as the sum of at most 18 digits (ignoring 0). Discard those decompositions for which the sum of the squares is not prime. Not too many combinations are left. Then find out how many numbers within the specified range you can construct using each of the possible decompositions (filling with zeros if required).

There are few places in this implementation that can be improved. In order to to start attacking the issues i made few changes first to get an idea of the main problems:
made the total start cases be the value 1 and set the range to be a billion (1,000,000,000) to have a large amount of iterations. also I use the method "getDigitSum" but commented out the code that actually makes the sum of digits to see how the rest runs: following are the methods that were modified for an initial test run:
private static int[] getDigitSum(long num) {
long sum = 0;
long squareSum = 0;
// for (long tempNum = num; tempNum > 0; tempNum = tempNum / 10) {
// if (tempNum < 0) {
// sum = sum + tempNum;
// squareSum = squareSum + (tempNum * tempNum);
// } else {
// long temp = tempNum % 10;
// sum = sum + temp;
// squareSum = squareSum + (temp * temp);
//
// }
// }
int[] twosums = new int[2];
twosums[0] = Integer.parseInt(sum+"");
twosums[1] = Integer.parseInt(squareSum+"");
// System.out.println("sum Of digits: " + twoDoubles[0]);
// System.out.println("squareSum Of digits: " + twoDoubles[1]);
return twosums;
}
and
public static void main(String args[]) throws NumberFormatException,
IOException {
// getDigitSum(80001001000l);
//System.out.println(getPrimeSet(1600));
Set set = getPrimeSet(1600);
//BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
int totalCases = 1;
for (int cases = 0; cases < totalCases; cases++) {
//String[] str = br.readLine().split(" ");
long startRange = Long.parseLong("1");
long endRange = Long.parseLong("1000000000");
int luckyCount = 0;
for (long num = startRange; num <= endRange; num++) {
int[] longArray = getDigitSum(num); //this method was commented for testing purpose and was replaced with any two hardcoded values
if(set.contains(longArray[0]) && set.contains(longArray[1])){
luckyCount++;
}
}
System.out.println(luckyCount);
}
}
Running the code takes 5 minutes 8 seconds.
now we can start optimizing it step by step. I will now mention the various points in the implementation that can be optimized.
1- in the method getDigitSum(long num)
int[] twosums = new int[2];
twosums[0] = Integer.parseInt(sum+"");
twosums[1] = Integer.parseInt(squareSum+"");
the above is not good. on every call to this method, two String objects are created , e.g. (sum+"") , before they are parsed into an int. considering the method is called billion times in my test, that produces two billion String object creation operations. since you know that the value is an int (according to the math in there and based on the links you provided), it would be enough to use casting:
twosums[0] = (int)sum;
twosums[1] = (int)squareSum;
2- In the "Main" method, you have the following
for (long num = startRange; num <= endRange; num++) {
int[] longArray = getDigitSum(num); \\this method was commented for testing purpose and was replaced with any two hardcoded values
if(set.contains(longArray[0]) && set.contains(longArray[1])){
luckyCount++;
}
}
here there are few issues:
a- set.contains(longArray[0]) will create an Integer object (with autoboxing) because contains method requires an object. this is a big waste and is not necessary. in our example, billion Integer objects will be created. Also, usage of set, whether it is a treeset or hash set is not the best for our case.
what you are trying to do is to get a set that contains the prime numbers in the range 1 .. 1600. this way, to check if a number in the range is prime, you check if it is contained in the set. This is not good as there are billions of calls to the set contains method. instead, your boolean array that you made when filling the set can be used: to find if the number 1500 is prime, simply access the index 1500 in the array. this is much faster solution. since its only 1600 elements (1600 is greater than max sum of sqaures of digits of your worst case), the wasted memory for the false locations is not an issue compared to the gain in speed.
b- int[] longArray = getDigitSum(num);
an int array is being allocated and returned. that will happen billion times. in our case, we can define it once outside the loop and send it to the method where it gets filled. on billion iterations, this saved 7 seconds, not a big change by itslef. but if the test cases are repeated 1000 times as you plan, that is 7000 second.
therefore, after modifying the code to implement all of the above, here is what you will have:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Set;
import java.util.TreeSet;
public class Solution {
private static void getDigitSum(long num,int[] arr) {
long sum = 0;
long squareSum = 0;
// for (long tempNum = num; tempNum > 0; tempNum = tempNum / 10) {
// if (tempNum < 0) {
// sum = sum + tempNum;
// squareSum = squareSum + (tempNum * tempNum);
// } else {
// long temp = tempNum % 10;
// sum = sum + temp;
// squareSum = squareSum + (temp * temp);
//
// }
// }
arr[0] = (int)sum;
arr[1] = (int)squareSum;
// System.out.println("sum Of digits: " + twoDoubles[0]);
// System.out.println("squareSum Of digits: " + twoDoubles[1]);
}
public static boolean[] getPrimeSet(int maxValue) {
boolean[] primeArray = new boolean[maxValue + 1];
for (int i = 2; i < primeArray.length; i++) {
primeArray[i] = true;
}
for (int i = 2; i < maxValue; i++) {
if (primeArray[i]) {
markMutiplesAsComposite(primeArray, i);
}
}
return primeArray;
}
public static void markMutiplesAsComposite(boolean[] primeArray, int value) {
for (int i = 2; i*value < primeArray.length; i++) {
primeArray[i * value] = false;
}
}
public static void main(String args[]) throws NumberFormatException,
IOException {
// getDigitSum(80001001000l);
//System.out.println(getPrimeSet(1600));
boolean[] primeArray = getPrimeSet(1600);
//BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
int totalCases = 1;
for (int cases = 0; cases < totalCases; cases++) {
//String[] str = br.readLine().split(" ");
long startRange = Long.parseLong("1");
long endRange = Long.parseLong("1000000000");
int luckyCount = 0;
int[] longArray=new int[2];
for (long num = startRange; num <= endRange; num++) {
getDigitSum(num,longArray); //this method was commented for testing purpose and was replaced with any two hardcoded values
if(primeArray[longArray[0]] && primeArray[longArray[1]]){
luckyCount++;
}
}
System.out.println(luckyCount);
}
}
}
Running the code takes 4 seconds.
the billion iterations cost 4 seconds instead of 5 minutes 8 seconds, that is an improvement. the only issue left is the actual calculation of the sum of digits and sum of squares of digits. that code i commented out (as you can see in the code i posted). if you uncomment it, the runtime will take 6-7 minutes. and here, there is nothing to improve except if you find some mathematical way to have incremental calculation based on previous results.

How can I make this Java snippet more efficient?

In a computer contest, I was given a problem where I had to manipulate input data. The input has been split() into an array where data[0] is the number of repetitions. There can be up to 10^18 repetitions. My program returned Exception in thread "main" java.lang.OutOfMemoryError: Java heap space and I failed the contest.
Here's a piece of my code that's eating up memory and CPU:
long product[][]=new long[data[0]][2];
product[0][0]=data[1];
product[0][1]=data[2];
for(int a=1;a<data[0];a++){
product[a][0]=((data[5]*product[a-1][0] + data[6]) % data[3]) + 1; // Pi = ((A*Pi-1 + B) mod M) + 1 (for all i = 2..N)
product[a][1]=((data[7]*product[a-1][1] + data[8]) % data[4]) + 1; // Wi = ((C*Wi-1 + D) mod K) + 1 (for all i = 2..N)
}
Here's some of the input data:
980046644627629799 9 123456 18 10000000 831918484 451864686 840000324 650000765
972766173386786486 123 1 10000000 10000000 590000001 680000000 610000001 970000002
299896237124947938 681206 164538 2280874 981991 416793690 904023823 813682336 774801135
My program can only work up to about 7 or 8 digits, then it takes minutes to run. With 18 digits, it crashed almost as soon as I clicked "Run" in Eclipse.
I'm curious as to how is it possible to manipulate that much data on a normal computer. Please let me know if my question is unclear or you need more information. Thanks!

You can't have, and don't need, an array of such a huge length. You just need to track the most recent 2values. E.g., just have product1 and product2.
Also, consider testing if either number is a NaN after each iteration. If so, throw an Exception and give the iteration number.
Because once you get a NaN they will all be NaN. Except you are using long, so scratch that. "Nevermind". :-)

long product[][]=new long[data[0]][2];
This is the only line in the code you pasted that allocates memory. You allocate an array whose length will be data[0] in length! As data grows, so does the array. What is the formula you're trying to apply here?
The first input data you provide :
980046644627629799
is already too large to even declare an array for. Try creating a single dimension array with that as its length and see what happens....
Are you sure you don't just want a 1 x 2 matrix that you accumulate over? Explain your intended algorithm clearly and we can help you with a more optimal solution.

Let's put the numbers into perspective.
Memory: One long takes 8 bytes. 1018 longs take 16,000,000 terabytes. Way too much.
Time: 10,000,000 operations ≈ 1 second. 1018 steps ≈ 30 centuries. Also way too much.
You can solve the memory problem by realising that you only need the most recent values at any time, and that the entire array is redundant:
long currentP = data[1];
long currentW = data[2];
for (int a = 1; a < data[0]; a++)
{
currentP = ((data[5] * currentP + data[6]) % data[3]) + 1;
currentW = ((data[7] * currentW + data[8]) % data[4]) + 1;
}
The time problem is a bit trickier to solve. Since modulus is used, you can observe that the numbers must enter a cycle at some point. Once you find the cycle, you can predict what the value will be after n iterations without having to do each iteration manually.
The simplest method for finding cycles is to keep track of whether or not you visited each element, and then go through until you encounter an element you've seen before. In this situation, the amount of memory required is proportional to M and K (data[3] and data[4]). If they are too large, a more space-efficient cycle detection algorithm must be used.
Here is an example which finds the value for P:
public static void main(String[] args)
{
// value = (A * prevValue + B) % M + 1
final long NOT_SEEN = -1; // the code used for values not visited before
long[] data = { 980046644627629799L, 9, 123456, 18, 10000000, 831918484, 451864686, 840000324, 650000765 };
long N = data[0]; // the number of iterations
long S = data[1]; // the initial value of the sequence
long M = data[3]; // the modulus divisor
long A = data[5]; // muliply by this
long B = data[6]; // add this
int max = (int) Math.max(M, S); // all the numbers (except first) must be less than or equal to M
long[] seenTime = new long[max + 1]; // whether or not a value was seen and how many iterations it took
// initialize the values of 'seenTime' to 'not seen'
for (int i = 0; i < seenTime.length; i++)
{
seenTime[i] = NOT_SEEN;
}
// find the cycle
long count = 0;
long cycleValue = S; // the current value in the series
while (seenTime[(int)cycleValue] == NOT_SEEN)
{
seenTime[(int)cycleValue] = count;
cycleValue = (A * cycleValue + B) % M + 1;
count++;
}
long cycleLength = count - seenTime[(int)cycleValue];
long cycleOffset = seenTime[(int)cycleValue];
long result;
if (N < cycleOffset)
{
// Special case: requested iteration occurs before the cycle starts
// Straightforward simulation
long value = S;
for (long i = 0; i < N; i++)
{
value = (A * value + B) % M + 1;
}
result = value;
}
else
{
// Normal case: requested iteration occurs inside the cycle
// Simulate just the relevant part of one cycle
long positionInCycle = (N - cycleOffset) % cycleLength;
long value = cycleValue;
for (long i = 0; i < positionInCycle; i++)
{
value = (A * value + B) % M + 1;
}
result = value;
}
System.out.println(result);
}
I am only giving you the solution because it looks like the contest is over. The important lesson to learn from this is that you should always check the bounds to see whether your solution is practical before you start coding it up.