I need to find a way to calculate w vector of SVM with rbf kernel. Refer to libsvm documentation, rbf kernel is defined as :
RBF: exp(-gamma*|u-v|^2)
I know u is a 1*n array (model.sv_coef)
and v is m*n matrix (model.SVs)
now, I don't know haw could I calculate euclidean distance of u and v ---> |u-v| (one of them is 1-d array and the other is 2-d one)
and after that how could I find a 1*n array which is a w array?
Thanks in advance.
It's not possible to perform the operation you dream of. One cannot calculate d(u, v) as in your case by use of euclidean distance. The dimensions do not match, when you define ANY type of vectors/matrices like you did in your question. u and v are data points, but not the results of some SV-estimation.
Make yourself clear what the dimensions of your vectors/matrices are. Consider using DTW (dynamic time warping) or the like, when you use vectors of different lengths. You can put DTW into the RBF kernel, at least this will give results on a empirical sound basis (but keep in mind DTW violates the triangle inequality).
One more thing: u and v are data points. You should familiarize yourself more with SVM.
Related
I have a large dataset (>500.000 elements) that contains the stress values (σ_xx, σ_yy, σ_zz, τ_xy, τ_yz, τ_xz) of FEM-Elements. These stress values are given in the global xyz-coordinate space of the model. I want to calculate the main axis stress values and directions from those. If you're not that familiar with the physics behind it, this means taking the symmetric matrix
| σ_xx τ_xy τ_xz |
| τ_xy σ_yy τ_yz |
| τ_xz τ_yz σ_zz |
and calculating its eigenvalues and eigenvectors. Calculating each set of eigenvalues and -vectors on its own is too slow. I'm looking for a library, an algorithm or something in Java that would allow me to do this as array calculations. As an example, in python/numpy I could just take all my 3x3-matrices, stack them along a third dimension to get a nx3x3-array, and pass that to np.linalg.eig(arr), and it automatically gives me an nx3-array for the three eigenvalues and an nx3x3-array for the three eigenvectors.
Things I tried:
nd4j has an Eigen-module for calculating eigenvalues and -vectors, but only supports a single square array at a time.
Calculate the characteristic polynomial and use cardanos formula to get the roots/eigenvalues - possible to do for the whole array at once, but I'm stuck now on how to get the corresponding eigenvectors. Is there maybe a general simple algorithm to get from those to the eigenvectors?
Looking for an analytical form of the eigenvalues and -vectors that can be calculated directly: It does exist, but just no.
You'll need to write a little code.
I'd create or use a Matrix class as a dependency and find methods to give you eigenvalues and eigenvectors. The ones you found in nd4j sound like great candidates. You might also consider the Linear Algebra For Java (LA4J) dependency.
Load the dataset into a List<Matrix>.
Use functional Java methods to apply a map to give you a List of eigenvalues as a vector per stress matrix and a List of eigenvectors as a matrix per stress matrix.
You can optimize this calculation to the greatest extent possible by applying the map function to a stream. Java will parallelize the calculation under the covers to leverage available cores to the greatest extent possible.
Follow-up: This is the way that worked best for me, as I can do all operations without iterating over every element. As stated above, I'm using Nd4j, which seems to be limited in its possibilities compared to numpy (or maybe I just didn't read the documentation thoroughly enough). The following method uses only basic array operations:
From the given stress values, calculate the eigenvalues using Cardano's formula. Only element wise instructions are needed to do that (add, sub, mul, div, pow). The result should be three vectors of size n, each containing one eigenvalue for all elements.
Use the formula given here to calculate the matrix S for each eigenvalue. Like step 1, this can obviously also be done using only element-wise operations with the stress value- and eigenvalue-vectors, in order to avoid specifiying some complicated instructions on which array to multiply according to which axis while keeping whatever other axis.
Take one column from S and normalize it to get a normalized eigenvector for the given eigenvalue.
Note that this method only works if you have a real symmetric matrix. You also should make sure to properly deal with cases where the same eigenvalue appears multiple times.
I have a 2X2 Matrix and am doing prop_test in R and then take the P value from the result.
Now, I want to do the same thing in Java i.e., computing the P value.
One way is to use https://commons.apache.org/proper/commons-math/javadocs/api-3.1/org/apache/commons/math3/stat/inference/ChiSquareTest.html and compute the P-value
But, chi-square test does not use a Yates' continuity correction which can result in differences between R's approach and ChiSquareTest approach.
I would like to know any other ways to get the P-Value using Yates' continuity correction
I'm trying to implement .3ds importer according to this documentation and I've approached the stage when I need to calculate vertex normals because .3ds files do not provide such. Here is the Java code:
/* Sctructure of vertex array is {x0, y0, z0, x1, y1, z1...}
*
* Basically, MathUtils.generateNormal_f(x0,y0,z0, x1,y1,z1, x2,y2,z2) is cross
* product between (x1-x0, y1-y0, z1-z0) and (x2-x0, y2-y0, z2-z0) */
normals = new float[this.vertex.length]; //every vertex has it's own normal
int n = 0;
for (int i=0; i<this.index.length; i++){
float[] Normal = MathUtils.generateNormal_f( //getting xyz coords of 1 normal
vertex[index[i]*3], vertex[index[i]*3+1], vertex[index[i]*3+2],
vertex[index[++i]*3], vertex[index[i]*3+1], vertex[index[i]*3+2],
vertex[index[++i]*3], vertex[index[i]*3+1], vertex[index[i]*3+2]);
normals[n++] = Normal[0];
normals[n++] = Normal[1];
normals[n++] = Normal[2];
}
Method MathUtils.generateNormal_f(...) tested and works fine. Result of this code can be seen below (first image). Just for example, in the second image, every normal of the model is the same and pointing towards the source of light.
Question is: how to calculate normals properly?
Your normals might be inverted.
I do not remember the 3ds format very well, but check if you can export and import the normals from the file instead of calculating them.
P.S. also do not use magic like this:
vertex[index[i]*3], vertex[index[i]*3+1], vertex[index[i]*3+2],
vertex[index[++i]*3], vertex[index[i]*3+1], vertex[index[i]*3+2],
vertex[index[++i]*3], vertex[index[i]*3+1], vertex[index[i]*3+2]
You will get different results based on the sequence of argument evaluation. better explicitly use [i], [i+1], [i+2] when calling calculate normal...
This information is correct as far as I know, and it's worked for me. For any 3 points A, B, and C on a plane, the normal, if we start at A, then B, and finally C, will be:
Where (B - A) and (C - B) each subtract two vectors, and the X sign represents finding the cross product of two vectors. The order of the points is quite important and determines our normal direction. If A, B, and C are organized in a counter-clockwise direction, than their normal will face outside the solid. If you want to know what a cross product is, then for any point P and Q, their cross product would be:
Another thing that is often done to the normal vector is that it is normalized. What this does is make the magnitude of the normal vector equal to 1 so that it is easier to work with. here's the equation:
Where the dot represents a dot product. If you don't know what a dot product is, allow me to illustrate by the following. For any points P and Q, their dot product, which is a scalar value, is:
Now that you have the surface normals, you can properly calculate the vertex normals for each vertex by averaging out the normals of any surface which shares that vertex. I don't have that formula on me, but I do know there are two approaches to find the vertex normal: weighted and non-weighted. A weighted approach involves calculating the area of each surface, while a non-weighted approach does not.
Hopefully, this information will help you. I leave the rest up to you or anyone else, as the remaining information is beyond my realm. Perhaps I'll come back and research some more on this question.
I have been working on latent semantic analysis lately. I have implemented it in java by making use of the Jama package.
Here is the code:
Matrix vtranspose ;
a = new Matrix(termdoc);
termdoc = a.getArray();
a = a.transpose() ;
SingularValueDecomposition sv =new SingularValueDecomposition(a) ;
u = sv.getU();
v = sv.getV();
s = sv.getS();
vtranspose = v.transpose() ; // we obtain this as a result of svd
uarray = u.getArray();
sarray = s.getArray();
varray = vtranspose.getArray();
if(semantics.maketerms.nodoc>50)
{
sarray_mod = new double[50][50];
uarray_mod = new double[uarray.length][50];
varray_mod = new double[50][varray.length];
move(sarray,50,50,sarray_mod);
move(uarray,uarray.length,50,uarray_mod);
move(varray,50,varray.length,varray_mod);
e = new Matrix(uarray_mod);
f = new Matrix(sarray_mod);
g = new Matrix(varray_mod);
Matrix temp =e.times(f);
result = temp.times(g);
}
else
{
Matrix temp = u.times(s);
result = temp.times(vtranspose);
}
result = result.transpose();
results = result.getArray() ;
return results ;
But how do we determine the number of dimensions? Is there a method to determine the number of dimensions to which the system should be reduced to obtain best results? What other parameters do we consider for effective performance of LSA?
Regarding the choice of the number of dimensions:
1) http://en.wikipedia.org/wiki/Latent_semantic_indexing:
Another challenge to LSI has been the alleged difficulty in
determining the optimal number of dimensions to use for performing the
SVD. As a general rule, fewer dimensions allow for broader comparisons
of the concepts contained in a collection of text, while a higher
number of dimensions enable more specific (or more relevant)
comparisons of concepts. The actual number of dimensions that can be
used is limited by the number of documents in the collection. Research
has demonstrated that around 300 dimensions will usually provide the
best results with moderate-sized document collections (hundreds of
thousands of documents) and perhaps 400 dimensions for larger document
collections (millions of documents). However, recent studies indicate
that 50-1000 dimensions are suitable depending on the size and nature
of the document collection.
Checking the amount of variance in the data after computing the SVD
can be used to determine the optimal number of dimensions to retain.
The variance contained in the data can be viewed by plotting the
singular values (S) in a scree plot. Some LSI practitioners select the
dimensionality associated with the knee of the curve as the cut-off
point for the number of dimensions to retain. Others argue that some
quantity of the variance must be retained, and the amount of variance
in the data should dictate the proper dimensionality to retain.
Seventy percent is often mentioned as the amount of variance in the
data that should be used to select the optimal dimensionality for
recomputing the SVD.
2) http://www.puffinwarellc.com/index.php/news-and-articles/articles/33-latent-semantic-analysis-tutorial.html?showall=1:
The trick in using SVD is in figuring out how many dimensions or
"concepts" to use when approximating the matrix. Too few dimensions
and important patterns are left out, too many and noise caused by
random word choices will creep back in.
The SVD algorithm is a little involved, but fortunately Python has a
library function that makes it simple to use. By adding the one line
method below to our LSA class, we can factor our matrix into 3 other
matrices. The U matrix gives us the coordinates of each word on our
“concept” space, the Vt matrix gives us the coordinates of each
document in our “concept” space, and the S matrix of singular values
gives us a clue as to how many dimensions or “concepts” we need to
include.
def calc(self): self.U, self.S, self.Vt = svd(self.A)
In order to
choose the right number of dimensions to use, we can make a histogram
of the square of the singular values. This graphs the importance each
singular value contributes to approximating our matrix. Here is the
histogram in our example.
For large collections of documents, the number of dimensions used is
in the 100 to 500 range. In our little example, since we want to graph
it, we’ll use 3 dimensions, throw out the first dimension, and graph
the second and third dimensions.
The reason we throw out the first dimension is interesting. For
documents, the first dimension correlates with the length of the
document. For words, it correlates with the number of times that word
has been used in all documents. If we had centered our matrix, by
subtracting the average column value from each column, then we would
use the first dimension. As an analogy, consider golf scores. We don’t
want to know the actual score, we want to know the score after
subtracting it from par. That tells us whether the player made a
birdie, bogie, etc.
3) Landauer, T.K., Foltz, P.W., Laham, D., (1998), 'Introduction to Latent Semantic
Analysis', Discourse Processes, 25, 259-284:
I'm working on AdaBoost implementation in Java.
It should have work for "double" coordinates on 2D 3D or 10D.
All I found for Java is for a binary data (0,1) and not for multi-dimensional space.
I'm currently looking for a way to represent the dimensions and to initialize the classifiers for boosting.
I'm looking for suggestions on how to represent the multidimensional space in Java, and how to initialize the classifiers to begin with.
The data is something in between [-15,+15]. And the target values are 1 or 2.
To use a boosted decision tree on spatial data, the typical approach is to try to find a "partition point" on some axis that minimizes the residual information in the two subtrees. To do this, you find some value along some axis (say, the x axis) and then split the data points into two groups - one group of points whose x coordinate is below that split point, and one group of points whose x coordinate is above that split point. That way, you convert the real-valued spatial data into 0/1 data - the 0 values are the ones below the split point, and the 1 values are the ones above the split point. The algorithm is thus identical to AdaBoost, except that when choosing the axis to split on, you also have to consider potential splitting points.
How about using JBoost, I think it's got what you're looking for.
Why don't you use a double[] array for each object? That is the common way of representing feature vectors in Java.