Develop Reference

Recursive algorithm stops working after certain depth

I was exploring Fork/Join framework and its possible speed benefits through factorial counting, when discovered that my sequential recursive algorithm breaks at a certain point. To be precise, when I try to count 46342! the result from RecursiveCounter is wrong, but before that value it is always right and is the same that result from ParallelCounter and LoopCounter. Does anyone have an idea why that may happen?
Here are the classes:
RecursiveCounter:
public class RecursiveCounter implements FactorialCounter, RangeFactorialCounter {
#Override
public BigInteger count(int number) {
return count(1, number);
}
#Override
public BigInteger count(int from, int to) {
int middle = (from + to) >> 1;
BigInteger left;
BigInteger right;
if (middle - from > 1)
left = count(from, middle);
else
left = new BigInteger(String.valueOf(from * middle));
if (to - (middle + 1) > 1)
right = count(middle + 1, to);
else
right = to == middle + 1 ? new BigInteger(String.valueOf(to)) : new BigInteger(String.valueOf((middle + 1) * to));
return left.multiply(right);
}
}
LoopCounter:
public class LoopCounter implements FactorialCounter, RangeFactorialCounter {
#Override
public BigInteger count(final int number) {
return count(1, number);
}
#Override
public BigInteger count(final int from, final int to) {
BigInteger result = new BigInteger("1");
for (int i = from; i < to + 1; i++) {
result = result.multiply(new BigInteger(String.valueOf(i)));
}
return result;
}
}
A RecursiveTask for ParallelCounter:
public class FactorialTask extends RecursiveTask<BigInteger> {
private static final int THRESHOLD = 1000;
private RangeFactorialCounter iterativeCounter = new LoopCounter();
private Integer firstVal;
private Integer lastVal;
public FactorialTask(Integer from, Integer to) {
super();
this.firstVal = from;
this.lastVal = to;
}
#Override
protected BigInteger compute() {
return count(firstVal, lastVal);
}
private BigInteger count(int from, int to) {
int middle = (from + to) >> 1;
if (to - from > THRESHOLD) {
List<FactorialTask> tasks = Arrays.asList(new FactorialTask(from, middle), new FactorialTask(middle + 1, to));
tasks.forEach(RecursiveTask::fork);
return tasks.stream()
.map(RecursiveTask::join)
.map(BigInteger.class::cast)
.reduce(new BigInteger("1"), BigInteger::multiply);
} else {
return (from != to) ? countSequential(from, to) : new BigInteger(String.valueOf(from));
}
}
private BigInteger countSequential(int from, int to) {
return iterativeCounter.count(from, to);
}
}

In RecursiveCounter, from * middle and (middle + 1) * to might overflow, you need use BigInteger to manipulate them:
...
left = BigInteger.valueOf(from).multiply(BigInteger.valueOf(middle));
...
right = to == middle + 1 ? BigInteger.valueOf(to) : BigInteger.valueOf(to).multiply(BigInteger.valueOf(middle + 1));
Then you can get the same result in RecursiveCounter and LoopCounter:
LoopCounter loopCounter = new LoopCounter();
RecursiveCounter recursiveCounter = new RecursiveCounter();
BigInteger loopResult = loopCounter.count(46342);
BigInteger recursiveResult = recursiveCounter.count(46342);
System.out.println(loopResult.equals(recursiveResult)); // true

This happens because of numeric overflow of an int, not because of recursive depth, which is nicely controlled by your algorithm, which needs O(log2n) stack frames for recursion.
The overflow happens here:
new BigInteger(String.valueOf((middle + 1) * to))
When to is high, this value can overflow int. Specifically, when middle approaches to in the second "leg" of recursive invocations, you multiply 46341 by 46342, which yields -2147432674 due to an overflow (demo).
You can fix this by using only BigInteger for "payload" multiplication, i.e.
BigInteger.valueOf(middle+1).multiply(BigInteger.valueOf(to))

What size does of stack does java -xss16M allocate 16 MegaBytes or 16Megabits

I am attempting to prove that I am using tail recursion properly by restricting the size allocated to the thread stack using the java -Xss160M command. I am writing a simple tail recursion algorithm for finding the factorial of a really big numbers:
/*
*
*my Program
*
*/
import java.math.BigInteger;
public class TailFactorial {
private BigInteger factorial(BigInteger n, BigInteger acc) {
if (n.equals(BigInteger.ONE)) return acc;
else return factorial(n.subtract(BigInteger.ONE), n.multiply(acc));
}
public static void main(String[] args) {
// TODO Auto-generated method stub
TailFactorial classs = new TailFactorial();
int num;
BigInteger bigNum = BigInteger.ONE;
boolean gTG = false;
String finalNumString = "";
String msg = "";
try {
num = Integer.parseInt(args[0]);
bigNum = BigInteger.valueOf(num);
msg = "fact("+String.valueOf(num)+") = ";
gTG = true;
}catch(Exception e) {
System.out.println("Could not get arguments, please make sure you are passing an integer");
gTG = false;
}
try {
if(gTG) {
BigInteger finalNum = classs.factorial(bigNum ,BigInteger.ONE);
System.out.println(String.valueOf(finalNum.bitCount()));
finalNumString = finalNum.toString();
//System.out.println( msg + finalNumString);
}
else {
System.out.print("Exiting");
}
}catch(StackOverflowError e) {
System.out.print("Not enough memory allocated to stack, try exicuting with java -Xss$bigger memory inserted here$ TrailFactorial");
}
catch(Exception e) {
System.out.print("Unrecoginzed error in finding factorial");
}
}
}
The code runs just fine, but I have to allocate 50M to the stack. When I run the current program, it displays the bit count of the result. When I find the factorial of one-hundred-thousand the bit count is 708,218, and I still have to use 50M. Can someone explain to me the meaning of the M? is it megabytes or bits? Am I not implementing tail-recursion properly?

As noted; Java doesn't support tail call recursion optimisation so for a large n value you will always run out of stack.
I suggest you use a simple loop if you want to avoid this.
public static BigInteger factorial(int n) {
BigInteger acc = BigInteger.ONE;
for (int i = 2; i <= n; i++)
acc = acc.multiply(BigInteger.valueOf(i));
return acc;
}
If you really, really want to use recursion, you can use divide and conquer
public static BigInteger factorial(int n) {
return product(1, n);
}
public static BigInteger product(int from, int to) {
if (from == to) return BigInteger.valueOf(from);
int mid = (from + to) >>> 1;
return product(from, mid).multiply(product(mid + 1, to));
}
This has a stack depth of log2(n) or a maximum of ~32.
BTW The following code
System.out.println(String.valueOf(finalNum.bitCount()));
finalNumString = finalNum.toString();
System.out.println( msg + finalNumString);
is the same as
System.out.println(finalNum.bitCount());
System.out.println(msg + finalNum);

Finding sum of digits of a large factorial

Problem
In the above problem, given a positive integer n it is intended to find the sum of all the digits in n!. So here is my java code for it:
public static void main (String[] args) throws java.lang.Exception
{
Scanner sc = new Scanner(System.in);
while(sc.hasNext())
{
int n = sc.nextInt();
BigInteger b = BigInteger.valueOf(1);
for(int i=2;i<=n;i++)
b=b.multiply(BigInteger.valueOf(i));
String s = b.toString();
int sum=0;
for(int i=0;i<s.length();i++)
sum+=(int)(s.charAt(i)-'0');
System.out.println(sum);
}
}
The limits for n is n<=1000. And it works flawlessly:
INPUT
5
60
100
1000
OUTPUT
3
288
648
10539
But the online judge is judging this as a wrong answer. Is there any problem in using the BigInteger class?
NOTE:I want the fault in the Implementation of the BigInteger class in my program, as this solution didn't exceed the time-limit but gave a wrong answer.

One of the initial conditions for your particular case is that Java 8 is used, so let's do that.
First, how to obtain the factorial for a given input number:
private static final BigInteger TWO = BigInteger.valueOf(2L);
private static BigInteger factorial(final BigInteger inputNumber)
{
BigInteger ret = BigInteger.ONE;
BigInteger b = TWO;
while (b.compareTo(inputNumber) <= 0) {
ret = ret.multiply(b);
b = b.add(BigInteger.ONE);
}
return ret;
}
Then, from that number, how to obtain the number of digits? We use base 10, we can therefore use the integer division and modulo operator:
private static int digitSum(final BigInteger someInteger)
{
int ret = 0;
BigInteger b = someInteger;
BigInteger[] modResult;
while (b.compareTo(BigInteger.ZERO) > 0) {
modResult = b.divideAndRemainter(BigInteger.TEN);
ret += modResult[1].intValueExact();
b = modResult[0];
}
return ret;
}
Now, plugging it in into a main program; you are supposed to read from a file, where inputs from that file are integers, one per line:
public final class FactDigitSum
{
// supposes that the two methods above are defined in this class
private static void printDigitSum(final BigInteger inputNumber)
{
final BigInteger factorial = factorial(inputNumber);
System.out.println(digitSum(factorial));
}
// The main program
public static void main(final String... args)
throws IOException
{
if (args.length == 0)
throw new IllegalArgumentException("No file as argument");
final Path path = Paths.get(args[0]);
try (
final Stream<String> stream = Files.lines(path);
) {
stream.map(BigInteger::new).forEach(FactDigitSum::printDigitSum);
}
}
}

Don't use the string. Calculate the value directly from the BigInteger.
This should give you the value. I'll leave I/O to you:
public class BigIntFacSum
{
private static int bigFacSum(final int n)
{
int sum = 0;
BigInteger fac = BigInteger.valueOf(2);
BigInteger num = BigInteger.valueOf(3);
for (int i = 3; i <= n; i++)
{
fac = fac.multiply(num);
num = num.add(BigInteger.ONE);
}
while (fac.compareTo(BigInteger.ZERO) > 0)
{
BigInteger[] quotRem = fac.divideAndRemainder(BigInteger.TEN);
fac = quotRem[0];
sum += quotRem[1].intValue();
}
return sum;
}
public static void main(String[] args)
{
System.out.println(bigFacSum(1000));
}
}
This is pretty fast in Java 7. It should be even faster in Java 8, as that uses Karatsuba and Burnikel-Ziegler optimizations for multiplication and division respectively (as far as I know, Java 7 doesn't).
For what it's worth: If the numbers get bigger, it is possible that the detour via string and then adding the digits in the string becomes faster. The naive way of using divideAndRemainder(BigInteger.TEN) in a loop does not work well for large numbers like 100000!. I tried this with my own implementation of BigInteger (in a different language), and the string detour was much faster for such huge nubmers. That is because the conversion to decimal is highly optimized with a divide-and-conquer algorithm and much faster than the naive loop. But only for relatively large numbers, i.e. well over 10000!. AFAIK, Java uses a similar algorithm, so the conversion to string should also be faster there, for similarly huge humbers.
I am not aware of a divide-and-conquer algorithm that counts the sum of the digits at the same time, although I don't see why it couldn't. The algorithm is not trivial though, and mine is not in Java.
But that only as an aside, in case you or anyone else may need this one day.

Java: Summing all digits of 2^1000

I'm trying to solve Project Euler problem #16, where I need to sum all the digits of 2^1000. I've gotten stuck dealing with such a big number. My program worked for any number below 10^16, but failed afterwards. This told me that my logic was correct. I went ahead and converted all variables and methods to BigDecimal, but now the program does not run properly. It compiles as it is and there is no error; it just does not terminate. Does anyone have an idea on where I went wrong here?
import java.math.BigDecimal;
import java.math.RoundingMode;
public class Powerdigitsum {
private static final BigDecimal one = new BigDecimal("1");
private static final BigDecimal ten = new BigDecimal("10");
private static BigDecimal sumofDigits(BigDecimal n){
BigDecimal sum = new BigDecimal("0");
while(n.compareTo(one) == 1 || n.compareTo(one) == 0){
sum.add(n.remainder(ten));
n.divide(ten);
n = n.setScale(0, RoundingMode.FLOOR);
}
return sum;
}
public static void main(String[] args) {
final double the_number = Math.pow(2,1000);
final double test = 15;
final BigDecimal two_to_the_thousandth_power = new BigDecimal(test);
System.out.println(sumofDigits(two_to_the_thousandth_power));
}
}

Just use BigInteger properly:
BigInteger a = new BigInteger("2").pow(1000);

The whole method is kinda wrong. See this:
private static BigInteger sumOfDigits(BigInteger n) {
BigInteger sum = BigInteger.ZERO;
while (n.compareTo(BigInteger.ZERO) == 1) {
sum = sum.add(n.remainder(ten));
n = n.divide(ten);
}
return sum;
}
You needed to compare to zero, not one. And you need to assign the values for BigIntegers and BigDecimals, their methods do nothing on their own, the instances of those classes are immutable.
For integers, it's generally better to use BigInteger. The decimal part (that gets there from dividing) is just thrown away.

final double the_number = Math.pow(2,1000);
This won't work because the_number is not large enought to take the result. You need to convert the pow call to BigInteger:
BigInteger result = new BigInteger("2").pow(1000);
But be aware.. this can take some time..

Don't use the BigDecimal(double) constructor: it is limited by the double primitive type, which cannot represent 2^1000.
You can use a BigInteger. Something along these lines should work (probably suboptimal, but...):
public static void main(final String... args)
{
// 2^1000
final BigInteger oneTo2000 = BigInteger.ONE.shiftLeft(1000);
BigInteger digitSum = BigInteger.ZERO;
// We don't want to split against the empty string, the first element would be ""
for (final String digit: oneTo2000.toString().split("(?<=.)"))
digitSum = digitSum.add(new BigInteger(digit));
System.out.println(digitSum);
}

public class SumofDigitsPow {
public static void main(String[] args) {
//2(2^1000)
String temp = BigInteger.ONE.shiftLeft(1000).toString();
int sum = 0;
for(int i=0;i<temp.length();i++){
sum+= temp.charAt(i) - '0';
}
System.out.println(Integer.toString(sum));
}
}
java.math.BigInteger.shiftLeft(int n) method returns a BigInteger whose value is (this << n),So you can get the answer by using BigInteger and LeftShift Method

import java.math.BigInteger;
public class Problem16 {
public static void main(String[] args) {
BigInteger number2 = new BigInteger("2");
BigInteger number3 = new BigInteger("0");
number3 =number2.pow(1000);
String str = number3.toString();
BigInteger sum = new BigInteger("0");
for(int i=0; i<str.length(); i++)
{
char c= str.charAt(i);
int value = Character.getNumericValue(c);
BigInteger value2 = new BigInteger(Integer.toString(value));
sum =sum.add(value2) ;
}
System.out.println(sum);
}
}

IF YOU THINK BIGINTEGER IS CHEATING AND/OR don't feel like using it/learning how to use it, this algorithm is the way to go.
Think about how you would calculate 2^1000 by hand. You'd start with 2^1 and multiply by two repeatedly. Now notice that the number of digits of powers of two increase by 1 for AT LEAST every 3 powers (could be after 4 powers like with 1024 to 8192). So make a jagged 2D array like this
int a[][]= new int[1000][];
for(int i=0;i<1000;i++)
{
a[i]= new int[1+(i/3)];
}
Then initialize a[0][0] to 2. After this, you want to write a for loop such that each row is filled from the rightmost spot. So make two variables "digit" and "carry". Digit is the number that you will input into the row you're working on, and the carry is the one you're going to take to the next calculation and add to the product of 2 and whatever digit you're multiplying it with. Be careful with the order you update digit and carry and reinitialize them to zero after every calculation. I think the hardest part is coming up with the limits for the for loop, so that it fits with the every 3 powers thing. You can make this simpler by just making a triangular jagged array that increments by one every row. I did it like this though. Here's my whole code.
import java.util.*;
public class ProjectEuler16
{
public static void main(String[] args)
{
long t1=System.currentTimeMillis();
ProjectEuler16 obj = new ProjectEuler16();
System.out.println(obj.bigNumHandler());
long t2= System.currentTimeMillis();
System.out.println(t2-t1);
}
int bigNumHandler()
{
int a[][] = new int[1000][];
for(int i=0;i<1000;i++)
{
a[i]= new int[1+(i/3)];
}
a[0][0]=2;
for(int i=1;i<1000;i++)
{
int carry=0;
int digit=0;
int f=0;
if(i%3==0)
{
f=1;
}
for(int j=a[i-1].length-1+f;j>=0;j--)
{
if(j==0&f==1)
{
a[i][0]=carry;
}
else
{
digit=((2*a[i-1][j-f])+carry)%10;
carry=((2*a[i-1][j-f])+carry)/10;
a[i][j]=digit;
}
}
}
int sum=0;
for(int k=0;k<a[999].length;k++)
{
sum=sum+a[999][k];
}
return sum;
}
}
Note that the last row lists the digits for 2^1000.I think you can figure out how to sum the digits. The program took about 5 seconds to come up with the answer.

solution::::
import java.math.BigInteger;
public class PR9 {
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
BigInteger zero=BigInteger.valueOf(0);
BigInteger ten=BigInteger.valueOf(10);
BigInteger sum=zero;
BigInteger a = new BigInteger("2").pow(1000);
while(a.compareTo(zero)>0){
sum=sum.add(a.mod(ten));
a=a.divide(ten);
}
System.out.println(sum);
}
}
output:::::
1366

import java.math.BigInteger;
public class P16 {
public static BigInteger digitSum(int n) {
BigInteger sum = BigInteger.ZERO;
BigInteger number = new BigInteger("2").pow(n);
while (number.compareTo(BigInteger.ZERO) == 1) {
BigInteger remainder = number.remainder(BigInteger.TEN);
sum = sum.add(remainder);
number = number.divide(BigInteger.TEN);
}
return sum;
}
public static void main(String[] args) {
final double START = System.nanoTime();
System.out.println(digitSum(Integer.parseInt(args[0])));
final double DURATION = System.nanoTime() - START;
System.out.println("Duration: " + DURATION / 1000000 + "ms.");
}
}
While there maybe a way of solving this problem without the use of BigIntegers, it is clear that they make the code run way faster.
Mine only took about 4ms to find an answer.

Providing an alternative to equals()?

Let's say I have a class called Number, and I intend to do a lot of equality comparisons of Number objects. I am concerned about the "overhead" (class comparison, etc...) of the generic Number::equals(Object o) method. In this case, is it useful to provide a method such as Number::isEqualTo(Number other) as an alternative to Number::equals(Object o)? Is this a common pattern? Or do JVMs currently optimize well enough that there is no advantage to doing this?
Here's a code example:
public class Number {
int _value;
Number(int value) {
_value = value;
}
#Override
public boolean equals(final Object o) {
if (o == this) return true;
if (o == null) return false;
if (o.getClass() != getClass()) return false;
return isEqualTo((Number)o);
}
public boolean isEqualTo(final Number other) {
return _value == other._value;
}
public static void main(String[] args) {
Number one = new Number(1);
Number two = new Number(2);
if (!one.isEqualTo(two)) {
System.out.println("fast comparison?");
}
if (!one.equals(two)) {
System.out.println("slow comparison?");
}
}
}

The two methods have different semantic:
equals has the semantic dictated by the Object::equals contract, while
isEqualTo has the semantic that applies exclusively to Number objects
Since the comparison is not apples-to-apples, it is fair that equals would require more CPU cycles. It is unlikely that you would notice the difference, however.
It is far more common for classes like yours to implement Comparable<T>. The semantic there calls for an ordering check, not just for an equality checks, but there is no requirement to take objects of unknown classes, letting you save CPU cycles.
You should have a good reason to provide an alternative to equality (e.g. a profiler run that points to equals(Object) as a bottleneck, a perceived improvement on readability due to the change, or achieving richer semantic due to adopting an interface that does more). Doing it for the sake of cutting a few CPU cycles would premature optimization.

A quick microbenchmark with the most unfavourable scenario (equals always calls isEqualTo) shows (in ms):
equals: 1014
isEqualTo: 1010
Bottom line: unless your program doesn't do anything else, this is not going to be a performance bottleneck and you should stick to the first principle of optimisation: profile first, then optimise what needs to be optimised.
Test code:
public class TestPerf {
private static int NUM_RUN;
private static List<Number> list = new ArrayList<>();
public static void main(String[] args) {
NUM_RUN = 100_000;
for (int i = 0; i < 10000; i++) {
list.add(new Number(i));
}
long sum = 0;
System.out.println("Warmup");
for (int i = 0; i < NUM_RUN; i++) {
sum += method1(17);
sum += method2(17);
}
System.gc();
System.out.println("Starting");
sum = 0;
long start = System.nanoTime();
for (int i = 0; i < NUM_RUN; i++) {
sum += method1(17);
}
long end = System.nanoTime();
System.out.println("equals: " + (end - start) / 1000000);
System.gc();
start = System.nanoTime();
for (int i = 0; i < NUM_RUN; i++) {
sum += method2(17);
}
end = System.nanoTime();
System.out.println("isEqualTo: " + (end - start) / 1000000);
System.out.println(sum);
}
private static int method1(int target) {
int sum = 0;
Number comparison = new Number(target);
for (Number n : list) {
if (n.equals(comparison)) sum++;
}
return sum;
}
private static int method2(int target) {
int sum = 0;
Number comparison = new Number(target);
for (Number n : list) {
if (n.isEqualTo(comparison)) sum++;
}
return sum;
}
public static class Number {
int _value;
Number(int value) {
_value = value;
}
#Override
public boolean equals(final Object o) {
if (o == this) return true;
if (o == null) return false;
if (o.getClass() != getClass()) return false;
return isEqualTo((Number) o);
}
public boolean isEqualTo(final Number other) {
return _value == other._value;
}
}
}

You may even provide an overload of equals itself: equals(Number). If you implement it very carefully (to be behaviorally indistinguishable from equals(Object)), you can achieve a minuscule speedup by avoiding a checked downcast in certain cases. Note that you are still going to have to check a.getClass() == b.getClass() so the difference is vanishingly small.

This depends on where you'd like to use a compare method.
Maybe you can use different implementations of a Comparator interface?
These can be used to eg. sort Lists.

xx.isEqualTo.yy is an comparison on "object" level. It simply checks whether this two object are referring the same object.
It is always better to write 'equations methods' for specific classes. For example in this case optimal comparison is simply ==.