I have 256 binary matrices B(x,y) and I try to pool them in the group of 8 matrices. Considering a single pixel at a particular position (x,y), a set of binary values from matrices in the pool consisting of 8 matrices can be used to construct a binary code of 8 bits. The following formula illustrates that :
see the formula
The pool of 256 matrices in the group of 8 matrices will result in 32 matrices. I wrote the code in java and the code run properly, but I want to reduce the time complexity, where it takes about 30 seconds to get the result!
The size of each matrix is 300 X 100,what I can change or use to get the same result with much less time??
//index-Strat
int start = 0;
//index-end
int end = 7;
//pool evey eghit matrices togather, and produces 32 matrices
for (int j = 0; j < 32; j++)
{
//call the pooling function
Mat bit - 8 = new Mat();
bit - 8 = pool(start, end);
//add 8-bit matrix to pool array
p.add(bit - 8);
//increamt the indexs to pool the next 8 matrix
start = end + 1;
end = start + 7;
}
//---------------------------
Mat pool(int s, int e)
{
Mat bit - 8 = new Mat(binary.get(0).size(), binary.get(0).type());
//apply the Bit Plane
for (int i = 0; i < bit - 8. rows(); i++)
{
for (int j = 0; j < bit - 8. cols(); j++)
{
double[] sum = new double[1];
for (int k = 0; k < 8; k++)
{
double[] v = new double[1];
v = binary.get(s + k).get(i, j);
double new_value = v[0] * Math.pow(2, k);
sum[0] = sum[0] + new_value;
}
bit - 8. put(i, j, sum);
}
return bit - 8
}
I doubt that it fully explains your long computation time, but you can avoid a lot of object allocations in pool(int, int) by allocating the v and sum array outside the nested for loops, and using int[] instead:
int[] v = new int[1];
int[] sum = new int[1];
for (int i = 0; i < rows; ++i) {
for (int j = 0; j < cols; ++j) {
sum[0] = 0;
for (int k = 0; k < 8; ++k) {
binary.get(s + k).get(i, j, v);
sum[0] += v[0] << k;
}
bitMinus8.put(i, j, sum);
}
}
This reduces the number of allocated arrays from (rows*cols)*(8 + 2) to just 2.
Object creation is reasonably cheap, but avoiding repeated creation can speed things up.
Note that saying:
double[] v = new double[1];
v = <something>;
is redundant: you allocate a new array, and then immediately overwrite it, discarding the allocated array. You can simply write:
double[] v = <something>;
Looking at the docs, I believe that you can also avoid the explicit element-wise sum:
Mat bitMinus8 = new Mat(binary.get(0).size(), binary.get(0).type());
for (int k = 0; k < 8; ++k) {
Core.addWeighted(binary.get(s + k), 1 << k, bitMinus8, 1, 0, bitMinus8);
}
Related
I would like to calculate the distances between points in a three dimentional room that are stored in a 2d array. The points are generated randomly and we assume that for all of the points it is possible that they are located in a 3d space. The points would have three parameters, like shown here: [[3.0, 5.0, 8.0], [6.0, 5.0, 4.0], [3.0, 3.0, 9.0], [8.0, 0.0, 3.0]]
I have come so far I think, but I seem to get into the array wrongly. Would you have any suggestion? I am getting index out of bounds.
public class Points {
public static class points {
static String distanceBetweenPoints(int a) {
// int a = number of Points
int columns = 3;
// every point has three coordinates x,y,z
double[][] points = new double[a][columns];
// generating the points with random method:
for (int i = 0; i < a; i++) {
for (int j = 0; j < columns; j++) {
points[i][j] = (int) (Math.random() * 10);
}
}
double[][] distances = new double[points.length][points.length];
for (int i = 0; i < distances.length; i++) {
for (int j = 0, k = 0; j < distances.length; j++ )
for (int l = 0, m = 0; j < points[0].length; l++, m++) {
distances[i][j] = Math.sqrt((Math.pow((points[i][k] - points[l][k]),2)
+ Math.pow((points[j][1] - points[l][1]), 2)
+ Math.pow((points[j][2] - points[l][2]), 2)));
}
}
return Arrays.deepToString(distances);
}
public static void main(String[] args) {
System.out.println(distanceBetweenPoints(4));
}
}
}
Putting a simple
System.out.println( String.format( "i: %d, j: %d, k: %d, l: %d, m: %d", i, j, k, l, m) );
into your code would show you your error immediatley. In the fourth iteration of inner loop the l gets 4, which is bigger than the size of the points-array.
You have a mess which indices are declared, checked and incremented in which loop.
I'm trying to make a matrix arithmetic operation method using multidimensional arrays ([verybigrow][2]). I'm new at this, and I just can't find what I'm doing wrong. I'd really appreciate any help in telling me what it is.
try {
Stream<String> Matrix = Files.lines(Paths.get(file)).parallel();
String[][] DataSet = Matrix.map(mapping -> mapping.split(",")).toArray(String[][]::new);
Double[][] distanceTable = new Double[DataSet.length - 1][];
/* START WANT TO REPLACE THIS MATRIX CALCULATION WITH PARALLEL STREAM RATHER THAN USE TRADITIONAL ARRAY ARITHMETICS START */
for (int i = 0; i < distanceTable.length - 1; ++i) {
distanceTable[i] = new Double[i + 1];
for (int j = 0; j <= i; ++j) {
double distance = 0.0;
for (int k = 0; k < DataSet[i + 1].length; ++k) {
double difference = Double.parseDouble(DataSet[j][k]) - Double.parseDouble(DataSet[i + 1][k]);
distance += difference * difference;
}
distanceTable[i][j] = distance;
}
}
/* END WANT TO REPLACE THIS MATRIX CALCULATION WITH PARALLEL STREAM RATHER THAN USE TRADITIONAL ARRAY ARITHMETICS START */
} catch ( Exception except ){
System.out.println ( except );
}
I had rather not use libraries or anything like that, I'm mostly doing this to learn how it works. Thank you so much in advance. if you asking the data looks like :
4,53
5,63
10,59
9,77
13,49
The Output of data processing should look like this :
[101] <- ((4-5)^2) + ((53-63)^2)
[72, 41] <- ( ((4-10)^2) + ((53-59)^2) ), ( ((5,10)^2) + ((63-59)^2))
[601.0, 212.0, 325.0]
[97.0, 260.0, 109.0, 800.0]
[337.0, 100.0, 109.0, 80.0, 400.0]
I try to change matrixDistance with distanceTable. Try to move this code into different method so you can run it parallel
for(int i = 0; i < matrixDistance.length - 1; ++i) {
distanceTable[i] = new double[i + 1];
for(int j = 0; j <= i; ++j) {
double distance = 0.0;
for(int k = 0; k < DataSet[i+1].length; ++k) {
double difference = Double.parseDouble(DataSet[j][k]) - Double.parseDouble(DataSet[i+1][k]);
distance += difference * difference;
}
distanceTable[i][j] = distance;
}
}
I've created this example based on your question.
public void parallel(String file)
....
// parsing from csv into matrix 2d Double[][]
....
IntStream
.range(1, data.length - 1)
.parallel()
.forEach(i -> {
add(euclidian.euclidian(Arrays.copyOf(data, i+1)), i);
});
}
This is the mini version of your algorithm.
public Double[] euclidian(Double[][] data) {
Double[] result = new Double[data.length - 1];
for (int i = 0; i < result.length; i++) {
result[i] =
Math.pow(data[i][0] - data[data.length - 1][0], 2) +
Math.pow(data[i][1] - data[data.length - 1][1], 2);
}
return result;
}
And because of parallel execution, you need to add locking method for insert data into distanceTable.
private final Object lock = new Object();
Double[][] distanceTable;
void add(Double[] data, int index){
synchronized (lock) {
distanceTable[index - 1] = data;
}
}
I've tested it in my laptop, for 74 row in csv file the comparison is like this (ORI is using your code, PAR is using my approach):
java -jar target/stream-example-1.0-SNAPSHOT.jar test.csv
#####################
ORI read: 59 ms
ORI map: 71 ms
ORI time: 80 ms
#####################
PAR read: 0 ms
PAR map: 6 ms
PAR time: 11 ms
Hope it helps.
#Fahim Bagar answer example should run faster with big data sets, but you should improve your single thread code before making hasty decisions about timing metrics compared to parallel.
For example, removing wasteful Double.parseDouble is easy with code example provided by #Fahim Bagar swapping String[][] DataSet by Double[][] DataSet
//String[][] DataSet = Matrix.map(mapping -> mapping.split(",")).toArray(String[][]::new);
Double[][] DataSet = Matrix.map(row -> Arrays.stream(row.split(",")).map(Double::parseDouble).toArray(Double[]::new)).toArray(Double[][]::new);
Then take various array references for DataSet[i + 1] and DataSet[j] to local variables outside their loops:
for (int i = 0; i < distanceTable.length - 1; ++i) {
Double[] arriplus1 = new Double[i + 1];
Double[] iarr = DataSet[i + 1];
for (int j = 0; j <= i; ++j) {
double distance = 0.0;
Double[] jarr = DataSet[j];
for (int k = 0, sz = iarr.length; k < sz; ++k) {
double difference = jarr[k] - iarr[k];
distance += difference * difference;
}
arriplus1[j] = distance;
}
distanceTable[i] = arriplus1;
}
You can do same for #Fahim Bagar euclidian method
public Double[] euclidian(Double[][] data) {
Double[] result = new Double[data.length - 1];
Double[] dL1 = data[data.length - 1];
for (int i = 0; i < result.length; i++) {
Double[] di = data[i];
result[i] = Math.pow(di[0] - dL1[0], 2) + Math.pow(di[1] - dL1[1], 2);
}
return result;
}
After that, getting rid of Double and using double would speed up further / cut down on memory allocations.
On CSV rows 1048 I see these timings on the 10th run of each:
#####################
ORI read: 0 ms
ORI map: 4 ms
ORI time: 14 ms
#####################
PAR read: 0 ms
PAR map: 1 ms
PAR time: 10 ms
I am working on the LeetCode question Longest Substring Without Repeating Characters. But I got two different results between Run Code and Submit Solution. My c++ code is
class Solution {
public:
int lengthOfLongestSubstring(string s) {
int* a = new int[257];
int ans = 0;
int n = s.size();
for (int j = 0, i = 0; j < n; j++) {
i = i > a[s[j]] ? i : a[s[j]];
ans = ans > j - i +1 ? ans : j - i + 1;
a[s[j]] = j + 1;
}
return ans;
}
};
And two outputs are
I don't know what's wrong with my code. Besides, my c++ code is written by learning his website java answer
public class Solution {
public int lengthOfLongestSubstring(String s) {
int n = s.length(), ans = 0;
int[] index = new int[128]; // current index of character
// try to extend the range [i, j]
for (int j = 0, i = 0; j < n; j++) {
i = Math.max(index[s.charAt(j)], i);
ans = Math.max(ans, j - i + 1);
index[s.charAt(j)] = j + 1;
}
return ans;
}
}
for (int j = 0, i = 0; j < n; j++) {
i = i > a[s[j]] ? i : a[s[j]];
Since a is uninitialized, a[s[j]] is undefined behavior. You want
for (int i=0;i<257;i++)
a[i]=0;
or better a vector
vector<int> a(257,0);
Unlike Java, C++ does not zero heap memory for you.
int* a = new int[257];
What is the data inside your array a?
In Java, a[0], a[1], ... a[256] are all equal to zero. But in C++, a[0], a[1], ... a[256] contains random garbage from whatever data was previously at that memory address.
You have to zero the memory first:
std::fill_n(a, 257, 0);
Or, if you prefer memset:
std::memset(a, 0, sizeof(int) * 257);
EDIT: As pointed out by #It'scominghome, value-initialization (C++11) is also possible:
int* a = new int[257](); // will zero the array
I am trying to make a program that finds the maximum possible determinant of an n*n binary [0,1] matrix. The problem with my current code is that after 4x4, the numbers are too long for a 'long' to hold. The code below is what I use to generate the matrices.
for (int j = 0; j < Math.pow(2, Math.pow(dim, 2)); j++) { //for each possible matrix
int[][] matrix = new int[dim][dim];//make a 2d i*i matrix
long t = Long.valueOf(Long.toBinaryString(j)); //line 58
//format the string so its length = # of elements in the matrix
String format = "%0" + (dim * dim) + "d";
String s = String.format(format, t);
//fill matrix
int counter = 0;
for (int k = 0; k < dim; k++) {//for each row
for (int l = 0; l < dim; l++) {//for each column
matrix[k][l] = s.charAt(counter) - 48; //the -48 is for ascii conversions
counter++;
}// end l
}// end k -- matrix is filled
The error I get when I go over a 4*4:
Exception in thread "main" java.lang.NumberFormatException: For input string: "10000000000000000000"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:444)
at java.lang.Long.valueOf(Long.java:540)
at determinants.Determinants.findMaxBinDet(Determinants.java:58)
at determinants.Determinants.main(Determinants.java:38)
What else can I do to parse the number into binary?
It appears you want to create with all the combinations of 0 and 1 in the cells. A much simpler approach is
for (long j = 0, limit = 1<<(dim*dim); j < limit; j++) { //for each possible matrix
int[][] matrix = new int[dim][dim];//make a 2d i*i matrix
//fill matrix
for (int k = 0, counter = 0; k < dim; k++) {//for each row
for (int l = 0; l < dim; l++, counter++) {//for each column
matrix[k][l] = (j >>> counter) & 1;
}
}
This will only work up to 7x7 matrixes but since generating all 8x8 combination is like to take more than a lifetime, you need another approach for that.
I am having a really hard time creating a method to raise a matrix to the power. I tried using this
public static int powerMethod(int matrix, int power) {
int temp = matrix ;
for (int i = power; i == 1; i--)
temp = temp * matrix ;
return temp ;
but the return is WAYYY off. Only the first (1,1) matrix element is on point.
I tried using that method in a main like so
// Multiplying matrices
for (i = 0; i < row; i++)
{
for (j = 0; j < column; j++)
{
for (l = 0; l < row; l++)
{
sum += matrix[i][l] * matrix[l][j] ;
}
matrix[i][j] = sum ;
sum = 0 ;
}
}
// Solving Power of matrix
for (i = 0; i < row; i++) {
for (j = 0; j < column; j++)
matrixFinal[power][i][j] = Tools.powerMethod(matrix[i][j], power) ;
}
Where "power", "row", and "column" is an int that the user enters.
Any ideas how I can do this??
Thanks!!!
You have a lot of issues here.
First, your matrix squaring algorithm has a (common) error. You have:
for (i = 0; i < row; i++) {
for (j = 0; j < column; j++) {
for (l = 0; l < row; l++) {
sum += matrix[i][l] * matrix[l][j] ;
}
matrix[i][j] = sum ;
sum = 0 ;
}
}
However, you need to store the result in a temporary second matrix, because when you do matrix[i][j] = sum, it replaces the value at that position with the output, then later results end up being incorrect. Also I suggest initializing sum to 0 first, since it appears you declare it outside of this loop, and initializing it first protects you against any arbitrary value sum may have before going into the loop. Furthermore, it is not immediately clear what you mean by row and column -- make sure you are iterating over the entire matrix. E.g.:
int temp[][] = new int[matrix.length];
for (i = 0; i < matrix.length; i++) {
temp[i] = new int[matrix[i].length];
for (j = 0; j < matrix[i].length; j++) {
sum = 0 ;
for (l = 0; l < matrix.length; l++) {
sum += matrix[i][l] * matrix[l][j] ;
}
temp[i][j] = sum ;
}
}
// the result is now in 'temp', you could do this if you wanted:
matrix = temp;
Note that matrix.length and matrix[i].length are fairly interchangeable above if the matrix is square (which it must be, in order to be multiplied by itself).
Secondly, your multiplication squares a matrix. This means if you repeatedly apply it, you keep squaring the matrix every time, which means you will only be able to compute powers that are themselves powers of two.
Your third issue is your final bit doesn't make much sense:
for (i = 0; i < row; i++) {
for (j = 0; j < column; j++)
matrixFinal[power][i][j] = Tools.powerMethod(matrix[i][j], power) ;
}
It's not immediately clear what you are trying to do here. The final part seems to be trying to raise individual elements to a certain power. But this is not the same as raising a matrix to a power.
What you need to do is define a proper matrix multiplication method that can multiply two arbitrary matrices, e.g.:
int[][] multiplyMatrices (int[][] a, int[][] b) {
// compute and return a x b, similar to your existing multiplication
// algorithm, and of course taking into account the comments about
// the 'temp' output matrix above
}
Then computing a power becomes straightforward:
int[][] powerMatrix (int[][] a, int p) {
int[][] result = a;
for (int n = 1; n < p; ++ n)
result = multiplyMatrices(result, a);
return result;
}
Why not just use Math.pow?
import java.lang.Math;
Then you just have to do
matrixFinal[power][i][j] = (int) Math.pow(matrix[i][j],power); //might have to cast this to an int