Develop Reference

Text Justification Algorithm

This is a very famous problem in DP, Can somebody help to visualize the recursion part of it.How are the Permutations or Combinations will be generated.
problem reference.
https://www.geeksforgeeks.org/dynamic-programming-set-18-word-wrap/

Given the maximum line width as L, the idea to justify the Text T, is to consider all suffixes of the Text (consider words instead of characters for forming suffixes to be precise.)
Dynamic Programming is nothing but "Careful Brute-force".
If you consider the brute force approach, you need to do the following.
consider putting 1, 2, .. n words in the first line.
for each case described in case 1(say i words are put in line 1), consider cases of putting 1, 2, .. n -i words in the second line and then remaining words on third line and so on..
Instead lets just consider the problem to find out the cost of putting a word at the beginning of a line.
In general we can define DP(i) to be the cost for considering (i- 1)th word as the beginning of a Line.
How can we form a recurrence relation for DP(i)?
If jth word is the beginning of the next line, then the current line will contain words[i:j) (j exclusive) and the cost of jth word being the beginning of the next line will be DP(j).
Hence DP(i) = DP(j) + cost of putting words[i:j) in the current line
As we want to minimise the total cost, DP(i) can be defined as follows.
Recurrence relation:
DP(i) = min { DP(j) + cost of putting words[i:j in the current line }
for all j in [i+1, n]
Note j = n signify that no words are left to be put in the next line.
The base Case: DP(n) = 0 => at this point there is no word left to be written.
To summarise:
Subproblems: suffixes , words[:i]
Guess: Where to start the next line, # of choices n - i -> O(n)
Recurrence: DP(i) = min {DP(j) + cost of putting words[i:j) in the current line }
If we use memoization, the expression inside the curly brace should should take O(1) time, and the loop run O(n) times (# of choices times).
i Varies from n down to 0 => Hence total complexity is brought down to O(n^2).
Now even though we derived the minimum cost for justifying the text, we also need to solve the original problem by keeping track of the j value for chosen as minimum in the above expression, so that we can later use the same to print out the justified text. The idea is of keeping parent pointer.
Hope this helps you understand the solution. Below is the simple implementation of the above idea.
public class TextJustify {
class IntPair {
//The cost or badness
final int x;
//The index of word at the beginning of a line
final int y;
IntPair(int x, int y) {this.x=x;this.y=y;}
}
public List<String> fullJustify(String[] words, int L) {
IntPair[] memo = new IntPair[words.length + 1];
//Base case
memo[words.length] = new IntPair(0, 0);
for(int i = words.length - 1; i >= 0; i--) {
int score = Integer.MAX_VALUE;
int nextLineIndex = i + 1;
for(int j = i + 1; j <= words.length; j++) {
int badness = calcBadness(words, i, j, L);
if(badness < 0 || badness == Integer.MAX_VALUE) break;
int currScore = badness + memo[j].x;
if(currScore < 0 || currScore == Integer.MAX_VALUE) break;
if(score > currScore) {
score = currScore;
nextLineIndex = j;
}
}
memo[i] = new IntPair(score, nextLineIndex);
}
List<String> result = new ArrayList<>();
int i = 0;
while(i < words.length) {
String line = getLine(words, i, memo[i].y);
result.add(line);
i = memo[i].y;
}
return result;
}
private int calcBadness(String[] words, int start, int end, int width) {
int length = 0;
for(int i = start; i < end; i++) {
length += words[i].length();
if(length > width) return Integer.MAX_VALUE;
length++;
}
length--;
int temp = width - length;
return temp * temp;
}
private String getLine(String[] words, int start, int end) {
StringBuilder sb = new StringBuilder();
for(int i = start; i < end - 1; i++) {
sb.append(words[i] + " ");
}
sb.append(words[end - 1]);
return sb.toString();
}
}

Random strings with given length unit testing

I have a program that is generating pseudo random numbers(Only lowercase, uppercase and digits are allowed).
/**
*
* #return - returns a random digit (from 0 to 9)
*
*/
int randomDigits() {
return (int) (Math.random() * 10);
}
/**
*
* #return - returns a random lowercase (from "a" to "z")
*/
char randomLowerCase() {
return (char) ('a' + Math.random() * 26);
}
/**
*
* #return - returns a random uppercase (from "A" to "Z")
*/
char randomUpperCase() {
return (char) ('A' + Math.random() * 26);
}
/**
*
* #return - returns a random number between 1 and 3.
*
*/
char randomChoice() {
return (char) ((char) (Math.random() * 3) + 1);
}
/**
*
* #param length
* - the length of the random string.
* #return - returns a combined random string. All elements are builder side
* by side.
*
* We use the randomChoice method to get randomly upper, lower and
* digits.
*/
public String stringBuilder(int length) {
StringBuilder result = new StringBuilder();
int len = length;
for (int i = 0; i < len; i++) {
int ch = randomChoice();
if (ch == 1) {
result.append(randomDigits());
}
if (ch == 2) {
result.append(randomLowerCase());
}
if (ch == 3) {
result.append(randomUpperCase());
}
}
return result.toString();
}
How can i make a test for that code. I try to test the range for the digits (form 0 to 9)
int minRange = 0;
int maxRange = 0;
for (int i = 0; i < 100000; i++) {
int result = item.randomDigits();
if (result == 52) {
minRange++;
} else {
if (result == 19) {
maxRange++;
}
}
}
LOGGER.info("The min range in the digit is 0, and in the test appeared: {}", minRange);
LOGGER.info("The max range in the digit is 9, and in the test appeared: {}", maxRange);
But i cant find how to test the lower or upper?

Testing code which uses any randomness is tricky. There are two approaches you can take:
Your test can have sufficient iterations that it has a good chance to show any errors in your logic. For many cases iterating 1000 or 1000000 times and checking consistency of the answers is reasonable. This is your only option if you are also looking to check some required distribution across a range.
This might look something like:
for (int i = 0; i < 1000000; i++)
assertTrue(isValid(new RandomVal()));
If you want to check that all your characters appear at least once:
assertEquals(26 * 2 + 9, IntStream.range(0, 1000000)
.mapToObj(n -> stringBuilder(6))
.flatMap(String::chars)
.collect(Collectors.toSet())
.size());
This uses Java 8 and essentially adds every character (as an integer) to the set and then checks how large it is afterwards.
Using a mocking framework (such as Mockito) to check the result is the expected one for specific outputs from whatever you are using to generate randomness. This is the best way to test that you get the correct result boundary conditions (i.e. the generator returning results at each end of its range).
This might look something like:
Random mockRandom = mock(Random.class);
when(mockRandom.nextFloat()).thenReturn(0.0f);
assertTrue(isValid(new RandomVal(mockRandom));
when(mockRandom.nextFloat()).thenReturn(1.0f - Float.MIN_VALUE);
assertTrue(isValid(new RandomVal(mockRandom));
For completeness it's worth doing both of these.

If I understand your problem resolution is simple
int minRange = 999999; //improbable big value
int maxRange = -999999; //improbable low value
for (int i = 0; i < 100000; i++) {
int result = item.randomDigits();
minRange = Math.min(result, minRange);
maxRange = Math.max(result, maxRange);
}
Please try it.
If you don't like Math library you can of course do it without it
for (int i = 0; i < 100000; i++) {
int result = item.randomDigits();
if (result < minRange) {
minRange = result;
}
if (result > maxRange) {
maxRange = result;
}
}

Verifying correctness of FFT algorithm

Today I wrote an algorithm to compute the Fast Fourier Transform from a given array of points representing a discrete function. Now I'm trying to test it to see if it is working. I've tried about a dozen different input sets, and they seem to match up with examples I've found online. However, for my final test, I gave it the input of cos(i / 2), with i from 0 to 31, and I've gotten 3 different results based on which solver I use. My solution seems to be the least accurate:
Does this indicate a problem with my algorithm, or is it simply a result of the relatively small data set?
My code is below, in case it helps:
/**
* Slices the original array, starting with start, grabbing every stride elements.
* For example, slice(A, 3, 4, 5) would return elements 3, 8, 13, and 18 from array A.
* #param array The array to be sliced
* #param start The starting index
* #param newLength The length of the final array
* #param stride The spacing between elements to be selected
* #return A sliced copy of the input array
*/
public double[] slice(double[] array, int start, int newLength, int stride) {
double[] newArray = new double[newLength];
int count = 0;
for (int i = start; count < newLength && i < array.length; i += stride) {
newArray[count++] = array[i];
}
return newArray;
}
/**
* Calculates the fast fourier transform of the given function. The parameters are updated with the calculated values
* To ignore all imaginary output, leave imaginary null
* #param real An array representing the real part of a discrete-time function
* #param imaginary An array representing the imaginary part of a discrete-time function
* Pre: If imaginary is not null, the two arrays must be the same length, which must be a power of 2
*/
public void fft(double[] real, double[] imaginary) throws IllegalArgumentException {
if (real == null) {
throw new NullPointerException("Real array cannot be null");
}
int N = real.length;
// Make sure the length is a power of 2
if ((Math.log(N) / Math.log(2)) % 1 != 0) {
throw new IllegalArgumentException("The array length must be a power of 2");
}
if (imaginary != null && imaginary.length != N) {
throw new IllegalArgumentException("The two arrays must be the same length");
}
if (N == 1) {
return;
}
double[] even_re = slice(real, 0, N/2, 2);
double[] odd_re = slice(real, 1, N/2, 2);
double[] even_im = null;
double[] odd_im = null;
if (imaginary != null) {
even_im = slice(imaginary, 0, N/2, 2);
odd_im = slice(imaginary, 1, N/2, 2);
}
fft(even_re, even_im);
fft(odd_re, odd_im);
// F[k] = real[k] + imaginary[k]
// even odd
// F[k] = E[k] + O[k] * e^(-i*2*pi*k/N)
// F[k + N/2] = E[k] - O[k] * e^(-i*2*pi*k/N)
// Split complex arrays into component arrays:
// E[k] = er[k] + i*ei[k]
// O[k] = or[k] + i*oi[k]
// e^ix = cos(x) + i*sin(x)
// Let x = -2*pi*k/N
// F[k] = er[k] + i*ei[k] + (or[k] + i*oi[k])(cos(x) + i*sin(x))
// = er[k] + i*ei[k] + or[k]cos(x) + i*or[k]sin(x) + i*oi[k]cos(x) - oi[k]sin(x)
// = (er[k] + or[k]cos(x) - oi[k]sin(x)) + i*(ei[k] + or[k]sin(x) + oi[k]cos(x))
// { real } { imaginary }
// F[k + N/2] = (er[k] - or[k]cos(x) + oi[k]sin(x)) + i*(ei[k] - or[k]sin(x) - oi[k]cos(x))
// { real } { imaginary }
// Ignoring all imaginary parts (oi = 0):
// F[k] = er[k] + or[k]cos(x)
// F[k + N/2] = er[k] - or[k]cos(x)
for (int k = 0; k < N/2; ++k) {
double t = odd_re[k] * Math.cos(-2 * Math.PI * k/N);
real[k] = even_re[k] + t;
real[k + N/2] = even_re[k] - t;
if (imaginary != null) {
t = odd_im[k] * Math.sin(-2 * Math.PI * k/N);
real[k] -= t;
real[k + N/2] += t;
double t1 = odd_re[k] * Math.sin(-2 * Math.PI * k/N);
double t2 = odd_im[k] * Math.cos(-2 * Math.PI * k/N);
imaginary[k] = even_im[k] + t1 + t2;
imaginary[k + N/2] = even_im[k] - t1 - t2;
}
}
}

Validation
look here: slow DFT,iDFT at the end is mine slow implementation of DFT and iDFT they are tested and correct. I also used them for fast implementations validation in the past.
Your code
stop recursion is wrong (you forget to set the return element) mine looks like this:
if (n<=1) { if (n==1) { dst[0]=src[0]*2.0; dst[1]=src[1]*2.0; } return; }
so when your N==1 set the output element to Re=2.0*real[0], Im=2.0*imaginary[0] before return. Also I am a bit lost in your complex math (t,t1,t2) and to lazy to analyze.
Just to be sure here is mine fast implementation. It need too much things from class hierarchy so it will not be of another use for you then visual comparison to your code.
My Fast implementation (cc means complex output and input):
//---------------------------------------------------------------------------
void transform::DFFTcc(double *dst,double *src,int n)
{
if (n>N) init(n);
if (n<=1) { if (n==1) { dst[0]=src[0]*2.0; dst[1]=src[1]*2.0; } return; }
int i,j,n2=n>>1,q,dq=+N/n,mq=N-1;
// reorder even,odd (buterfly)
for (j=0,i=0;i<n+n;) { dst[j]=src[i]; i++; j++; dst[j]=src[i]; i+=3; j++; }
for ( i=2;i<n+n;) { dst[j]=src[i]; i++; j++; dst[j]=src[i]; i+=3; j++; }
// recursion
DFFTcc(src ,dst ,n2); // even
DFFTcc(src+n,dst+n,n2); // odd
// reorder and weight back (buterfly)
double a0,a1,b0,b1,a,b;
for (q=0,i=0,j=n;i<n;i+=2,j+=2,q=(q+dq)&mq)
{
a0=src[j ]; a1=+_cos[q];
b0=src[j+1]; b1=+_sin[q];
a=(a0*a1)-(b0*b1);
b=(a0*b1)+(a1*b0);
a0=src[i ]; a1=a;
b0=src[i+1]; b1=b;
dst[i ]=(a0+a1)*0.5;
dst[i+1]=(b0+b1)*0.5;
dst[j ]=(a0-a1)*0.5;
dst[j+1]=(b0-b1)*0.5;
}
}
//---------------------------------------------------------------------------
dst[] and src[] are not overlapping !!! so you cannot transform array to itself .
_cos and _sin are precomputed tables of cos and sin values (computed by init() function like this:
double a,da; int i;
da=2.0*M_PI/double(N);
for (a=0.0,i=0;i<N;i++,a+=da) { _cos[i]=cos(a); _sin[i]=sin(a); }
N is power of 2 (zero padded size of data set) (last n from init(n) call)
Just to be complete here is mine complex to complex slow version:
//---------------------------------------------------------------------------
void transform::DFTcc(double *dst,double *src,int n)
{
int i,j;
double a,b,a0,a1,_n,b0,b1,q,qq,dq;
dq=+2.0*M_PI/double(n); _n=2.0/double(n);
for (q=0.0,j=0;j<n;j++,q+=dq)
{
a=0.0; b=0.0;
for (qq=0.0,i=0;i<n;i++,qq+=q)
{
a0=src[i+i ]; a1=+cos(qq);
b0=src[i+i+1]; b1=+sin(qq);
a+=(a0*a1)-(b0*b1);
b+=(a0*b1)+(a1*b0);
}
dst[j+j ]=a*_n;
dst[j+j+1]=b*_n;
}
}
//---------------------------------------------------------------------------

I'd use something authoritative like Wolfram Alpha to verify.
If I evalute cos(i/2) for 0 <= i < 32, I get this array:
[1,0.878,0.540,0.071,-0.416,-0.801,-0.990,-0.936,-0.654,-0.211,0.284,0.709,0.960,0.977,0.754,0.347,-0.146,-0.602,-0.911,-0.997,-0.839,-0.476,0.004,0.483,0.844,0.998,0.907,0.595,0.137,-0.355,-0.760,-0.978]
If I give that as input to Wolfram Alpha's FFT function I get this result.
The plot that I get looks symmetric, which makes sense. The plot looks nothing like any of the ones you supplied.

re-arranging letters to produce a palindrome

I wrote this simple function (in Java) to re-arrange letters in a given String to produce a palindrome, or if it is not possible, just prints -1 and returns.
For some reason, I can't figure out why this is not working (because it does not pass the automated-grading scripts). I tested every case that I could think of, and it does pass.
Could anyone please provide some insights on this? Thanks!
/**
* Pseudo-code:
* (0) Compute the occurrences of each characters.
* (0')(Also remember how many groups of characters have an odd number of members
* (1) If the number remembered above is greater than 1
* (meaning there are more than one group of characters with an odd
* number of members),
* print -1 and return (no palindrome!)
* (2) Else, for each group of character
* - if the number of member is odd, save it to a var called 'left'
* - put a char from the group at the current position, and
* another one at postion [len - cur -1].
* (3) If a variable 'left' is defined, put it in the middle of the string
*
* #param wd
*/
private static void findPalin(String wd)
{
if (wd.isEmpty())
{
// Empty String is a palindrome itself!
System.out.println("");
return;
}
HashMap<Character, Integer> stats = new HashMap<Character, Integer>();
int len = wd.length();
int oddC = 0;
for (int n = 0; n < len; ++n)
{
Integer prv = stats.put(wd.charAt(n), 1);
if (prv != null)
{
if (prv % 2 == 0)
++oddC;
else
--oddC;
stats.put(wd.charAt(n), ++prv);
}
else
++oddC;
}
if (oddC > 1)
System.out.println(-1);
else
{
int pos = 0;
char ch[] = new char[len];
char left = '\0';
for (char theChar : stats.keySet())
{
Integer c = stats.get(theChar);
if (c % 2 != 0)
{
left = theChar;
--c;
}
while (c > 1)
{
ch[len - pos - 1] = ch[pos] = theChar;
++pos;
--c;
}
}
if (left != '\0')
ch[(len - 1) / 2] = left;
for (char tp : ch)
System.out.print(tp);
System.out.println();
}
}

Modifying Levenshtein Distance algorithm to not calculate all distances

I'm working on a fuzzy search implementation and as part of the implementation, we're using Apache's StringUtils.getLevenshteinDistance. At the moment, we're going for a specific maxmimum average response time for our fuzzy search. After various enhancements and with some profiling, the place where the most time is spent is calculating the Levenshtein distance. It takes up roughly 80-90% of the total time on search strings three letters or more.
Now, I know there are some limitations to what can be done here, but I've read on previous SO questions and on the Wikipedia link for LD that if one is willing limit the threshold to a set maximum distance, that could help curb the time spent on the algorithm, but I'm not sure how to do this exactly.
If we are only interested in the
distance if it is smaller than a
threshold k, then it suffices to
compute a diagonal stripe of width
2k+1 in the matrix. In this way, the
algorithm can be run in O(kl) time,
where l is the length of the shortest
string.[3]
Below you will see the original LH code from StringUtils. After that is my modification. I'm trying to basically calculate the distances of a set length from the i,j diagonal (so, in my example, two diagonals above and below the i,j diagonal). However, this can't be correct as I've done it. For example, on the highest diagonal, it's always going to choose the cell value directly above, which will be 0. If anyone could show me how to make this functional as I've described, or some general advice on how to make it so, it would be greatly appreciated.
public static int getLevenshteinDistance(String s, String t) {
if (s == null || t == null) {
throw new IllegalArgumentException("Strings must not be null");
}
int n = s.length(); // length of s
int m = t.length(); // length of t
if (n == 0) {
return m;
} else if (m == 0) {
return n;
}
if (n > m) {
// swap the input strings to consume less memory
String tmp = s;
s = t;
t = tmp;
n = m;
m = t.length();
}
int p[] = new int[n+1]; //'previous' cost array, horizontally
int d[] = new int[n+1]; // cost array, horizontally
int _d[]; //placeholder to assist in swapping p and d
// indexes into strings s and t
int i; // iterates through s
int j; // iterates through t
char t_j; // jth character of t
int cost; // cost
for (i = 0; i<=n; i++) {
p[i] = i;
}
for (j = 1; j<=m; j++) {
t_j = t.charAt(j-1);
d[0] = j;
for (i=1; i<=n; i++) {
cost = s.charAt(i-1)==t_j ? 0 : 1;
// minimum of cell to the left+1, to the top+1, diagonally left and up +cost
d[i] = Math.min(Math.min(d[i-1]+1, p[i]+1), p[i-1]+cost);
}
// copy current distance counts to 'previous row' distance counts
_d = p;
p = d;
d = _d;
}
// our last action in the above loop was to switch d and p, so p now
// actually has the most recent cost counts
return p[n];
}
My modifications (only to the for loops):
for (j = 1; j<=m; j++) {
t_j = t.charAt(j-1);
d[0] = j;
int k = Math.max(j-2, 1);
for (i = k; i <= Math.min(j+2, n); i++) {
cost = s.charAt(i-1)==t_j ? 0 : 1;
// minimum of cell to the left+1, to the top+1, diagonally left and up +cost
d[i] = Math.min(Math.min(d[i-1]+1, p[i]+1), p[i-1]+cost);
}
// copy current distance counts to 'previous row' distance counts
_d = p;
p = d;
d = _d;
}

The issue with implementing the window is dealing with the value to the left of the first entry and above the last entry in each row.
One way is to start the values you initially fill in at 1 instead of 0, then just ignore any 0s that you encounter. You'll have to subtract 1 from your final answer.
Another way is to fill the entries left of first and above last with high values so the minimum check will never pick them. That's the way I chose when I had to implement it the other day:
public static int levenshtein(String s, String t, int threshold) {
int slen = s.length();
int tlen = t.length();
// swap so the smaller string is t; this reduces the memory usage
// of our buffers
if(tlen > slen) {
String stmp = s;
s = t;
t = stmp;
int itmp = slen;
slen = tlen;
tlen = itmp;
}
// p is the previous and d is the current distance array; dtmp is used in swaps
int[] p = new int[tlen + 1];
int[] d = new int[tlen + 1];
int[] dtmp;
// the values necessary for our threshold are written; the ones after
// must be filled with large integers since the tailing member of the threshold
// window in the bottom array will run min across them
int n = 0;
for(; n < Math.min(p.length, threshold + 1); ++n)
p[n] = n;
Arrays.fill(p, n, p.length, Integer.MAX_VALUE);
Arrays.fill(d, Integer.MAX_VALUE);
// this is the core of the Levenshtein edit distance algorithm
// instead of actually building the matrix, two arrays are swapped back and forth
// the threshold limits the amount of entries that need to be computed if we're
// looking for a match within a set distance
for(int row = 1; row < s.length()+1; ++row) {
char schar = s.charAt(row-1);
d[0] = row;
// set up our threshold window
int min = Math.max(1, row - threshold);
int max = Math.min(d.length, row + threshold + 1);
// since we're reusing arrays, we need to be sure to wipe the value left of the
// starting index; we don't have to worry about the value above the ending index
// as the arrays were initially filled with large integers and we progress to the right
if(min > 1)
d[min-1] = Integer.MAX_VALUE;
for(int col = min; col < max; ++col) {
if(schar == t.charAt(col-1))
d[col] = p[col-1];
else
// min of: diagonal, left, up
d[col] = Math.min(p[col-1], Math.min(d[col-1], p[col])) + 1;
}
// swap our arrays
dtmp = p;
p = d;
d = dtmp;
}
if(p[tlen] == Integer.MAX_VALUE)
return -1;
return p[tlen];
}

I've written about Levenshtein automata, which are one way to do this sort of check in O(n) time before, here. The source code samples are in Python, but the explanations should be helpful, and the referenced papers provide more details.

According to "Gusfield, Dan (1997). Algorithms on strings, trees, and sequences: computer science and computational biology" (page 264) you should ignore zeros.

Here someone answers a very similar question:
Cite:
I've done it a number of times. The way I do it is with a recursive depth-first tree-walk of the game tree of possible changes. There is a budget k of changes, that I use to prune the tree. With that routine in hand, first I run it with k=0, then k=1, then k=2 until I either get a hit or I don't want to go any higher.
char* a = /* string 1 */;
char* b = /* string 2 */;
int na = strlen(a);
int nb = strlen(b);
bool walk(int ia, int ib, int k){
/* if the budget is exhausted, prune the search */
if (k < 0) return false;
/* if at end of both strings we have a match */
if (ia == na && ib == nb) return true;
/* if the first characters match, continue walking with no reduction in budget */
if (ia < na && ib < nb && a[ia] == b[ib] && walk(ia+1, ib+1, k)) return true;
/* if the first characters don't match, assume there is a 1-character replacement */
if (ia < na && ib < nb && a[ia] != b[ib] && walk(ia+1, ib+1, k-1)) return true;
/* try assuming there is an extra character in a */
if (ia < na && walk(ia+1, ib, k-1)) return true;
/* try assuming there is an extra character in b */
if (ib < nb && walk(ia, ib+1, k-1)) return true;
/* if none of those worked, I give up */
return false;
}
just the main part, more code in the original

I used the original code and places this just before the end of the j for loop:
if (p[n] > s.length() + 5)
break;
The +5 is arbitrary but for our purposes, if the distances is the query length plus five (or whatever number we settle upon), it doesn't really matter what is returned because we consider the match as simply being too different. It does cut down on things a bit. Still, pretty sure this isn't the idea that the Wiki statement was talking about, if anyone understands that better.

Apache Commons Lang 3.4 has this implementation:
/**
* <p>Find the Levenshtein distance between two Strings if it's less than or equal to a given
* threshold.</p>
*
* <p>This is the number of changes needed to change one String into
* another, where each change is a single character modification (deletion,
* insertion or substitution).</p>
*
* <p>This implementation follows from Algorithms on Strings, Trees and Sequences by Dan Gusfield
* and Chas Emerick's implementation of the Levenshtein distance algorithm from
* http://www.merriampark.com/ld.htm</p>
*
* <pre>
* StringUtils.getLevenshteinDistance(null, *, *) = IllegalArgumentException
* StringUtils.getLevenshteinDistance(*, null, *) = IllegalArgumentException
* StringUtils.getLevenshteinDistance(*, *, -1) = IllegalArgumentException
* StringUtils.getLevenshteinDistance("","", 0) = 0
* StringUtils.getLevenshteinDistance("aaapppp", "", 8) = 7
* StringUtils.getLevenshteinDistance("aaapppp", "", 7) = 7
* StringUtils.getLevenshteinDistance("aaapppp", "", 6)) = -1
* StringUtils.getLevenshteinDistance("elephant", "hippo", 7) = 7
* StringUtils.getLevenshteinDistance("elephant", "hippo", 6) = -1
* StringUtils.getLevenshteinDistance("hippo", "elephant", 7) = 7
* StringUtils.getLevenshteinDistance("hippo", "elephant", 6) = -1
* </pre>
*
* #param s the first String, must not be null
* #param t the second String, must not be null
* #param threshold the target threshold, must not be negative
* #return result distance, or {#code -1} if the distance would be greater than the threshold
* #throws IllegalArgumentException if either String input {#code null} or negative threshold
*/
public static int getLevenshteinDistance(CharSequence s, CharSequence t, final int threshold) {
if (s == null || t == null) {
throw new IllegalArgumentException("Strings must not be null");
}
if (threshold < 0) {
throw new IllegalArgumentException("Threshold must not be negative");
}
/*
This implementation only computes the distance if it's less than or equal to the
threshold value, returning -1 if it's greater. The advantage is performance: unbounded
distance is O(nm), but a bound of k allows us to reduce it to O(km) time by only
computing a diagonal stripe of width 2k + 1 of the cost table.
It is also possible to use this to compute the unbounded Levenshtein distance by starting
the threshold at 1 and doubling each time until the distance is found; this is O(dm), where
d is the distance.
One subtlety comes from needing to ignore entries on the border of our stripe
eg.
p[] = |#|#|#|*
d[] = *|#|#|#|
We must ignore the entry to the left of the leftmost member
We must ignore the entry above the rightmost member
Another subtlety comes from our stripe running off the matrix if the strings aren't
of the same size. Since string s is always swapped to be the shorter of the two,
the stripe will always run off to the upper right instead of the lower left of the matrix.
As a concrete example, suppose s is of length 5, t is of length 7, and our threshold is 1.
In this case we're going to walk a stripe of length 3. The matrix would look like so:
1 2 3 4 5
1 |#|#| | | |
2 |#|#|#| | |
3 | |#|#|#| |
4 | | |#|#|#|
5 | | | |#|#|
6 | | | | |#|
7 | | | | | |
Note how the stripe leads off the table as there is no possible way to turn a string of length 5
into one of length 7 in edit distance of 1.
Additionally, this implementation decreases memory usage by using two
single-dimensional arrays and swapping them back and forth instead of allocating
an entire n by m matrix. This requires a few minor changes, such as immediately returning
when it's detected that the stripe has run off the matrix and initially filling the arrays with
large values so that entries we don't compute are ignored.
See Algorithms on Strings, Trees and Sequences by Dan Gusfield for some discussion.
*/
int n = s.length(); // length of s
int m = t.length(); // length of t
// if one string is empty, the edit distance is necessarily the length of the other
if (n == 0) {
return m <= threshold ? m : -1;
} else if (m == 0) {
return n <= threshold ? n : -1;
}
if (n > m) {
// swap the two strings to consume less memory
final CharSequence tmp = s;
s = t;
t = tmp;
n = m;
m = t.length();
}
int p[] = new int[n + 1]; // 'previous' cost array, horizontally
int d[] = new int[n + 1]; // cost array, horizontally
int _d[]; // placeholder to assist in swapping p and d
// fill in starting table values
final int boundary = Math.min(n, threshold) + 1;
for (int i = 0; i < boundary; i++) {
p[i] = i;
}
// these fills ensure that the value above the rightmost entry of our
// stripe will be ignored in following loop iterations
Arrays.fill(p, boundary, p.length, Integer.MAX_VALUE);
Arrays.fill(d, Integer.MAX_VALUE);
// iterates through t
for (int j = 1; j <= m; j++) {
final char t_j = t.charAt(j - 1); // jth character of t
d[0] = j;
// compute stripe indices, constrain to array size
final int min = Math.max(1, j - threshold);
final int max = (j > Integer.MAX_VALUE - threshold) ? n : Math.min(n, j + threshold);
// the stripe may lead off of the table if s and t are of different sizes
if (min > max) {
return -1;
}
// ignore entry left of leftmost
if (min > 1) {
d[min - 1] = Integer.MAX_VALUE;
}
// iterates through [min, max] in s
for (int i = min; i <= max; i++) {
if (s.charAt(i - 1) == t_j) {
// diagonally left and up
d[i] = p[i - 1];
} else {
// 1 + minimum of cell to the left, to the top, diagonally left and up
d[i] = 1 + Math.min(Math.min(d[i - 1], p[i]), p[i - 1]);
}
}
// copy current distance counts to 'previous row' distance counts
_d = p;
p = d;
d = _d;
}
// if p[n] is greater than the threshold, there's no guarantee on it being the correct
// distance
if (p[n] <= threshold) {
return p[n];
}
return -1;
}