Develop Reference

How to find optimum combination for Cutting Stock Problem using Knapsack

EDIT (31-12-2019) - https://jonathan.overholt.org/projects/cutlist
Above is the link of the free project which is what exactly I am looking for. I am just looking for proper guidance so that I can make it work.
I am working on minimizing the wastage of aluminium extrusion cutting for Aluminium Sliding Window Fabricators and I am not able to figure out which Algorithm/Data Structure I should go with for the problem.
I have done basic research and found that the problem falls in Cutting Stock Problem (Also called One dimensional cutting problem), Linear Programming Problem, Greedy Algorithm. BUT I am unable to decide which one I should go with and also how to start with.
Brief about the problem :
Basically the window fabricators have 3 sizes of material options to purchase.
12 | 15 | 16 (IN FT)
Now the input would be the size of Window.
WIDTH x HEIGHT (IN FT)
1) 6 x 8 - 10 Windows
2) 9 x 3 - 20 Windows
The calculation is (2 x Width) + (2 x Height). So from the above sizes of window, they need extrusion as follow.
1) 6' (FT) Size Pieces -> 2 x 10 = 20
2) 8' (FT) Size Pieces -> 2 x 10 = 20
3) 9' (FT) Size Pieces -> 2 x 20 = 40
4) 3' (FT) Size Pieces -> 2 x 20 = 40
Using knapsack, we can find out the combination but it has restriction of having size to 1 only whereas here in my case I have 3 different sizes available out of which I would like to generate the best optimum combination for the cutting stock problem.
I would need help in how should I proceed for the above problem with respect to data structure and algorithm in Java or any other language. My knowledge in Maths is not up to the mark and that's why I am facing issue in implementing the logic in my project and like to get some help from the community.

There is no algorithm that will guarantee you the optimal solution other than brute-force checking all possible combinations. That's obviously not a good algorithm, at least not if you have large data sets.
You should have a look at heuristic search algorithms like Simulated Annealing, MCTS or the like. It should be no problem finding existing implementations of heuristic search engines that allow you to provide them with
an input set (random distribution of pieces on material),
a transition function (move pieces between materials),
and an evaluation function (the amount of waste)
and compute a near-optimal solution for you.

I'll echo the sentiment of the other answers: while there might be a "most correct" solution to this problem, what you're really looking for is a "correct enough" solution.
That said, I wrote up this little appendix to help make sense of the code in the project you referenced: cutlist generator design notes
Paraphrased:
Each iteration starts by creating a new instance of the longest stock and placing as many pieces into it as will fit. The stock is then reduced to the smallest stock that all of the selected pieces still fit into. This is all repeated until no pieces remain.
Another word of advice: make sure to clearly identify all of the assumptions you're building into the algorithm. Mine assumes that longer stock is cheaper per unit and that it's always preferred, but I've been asked for variations to optimize for the least number of cuts (bundle cutting) or to keep track of and prefer to use offcuts from previous runs first.
As always: Understand your customer's process very clearly before setting the assumptions.

Have you considered the simplex algorithm?
You have a minimization problem which can either be transformed into a maximization problem and then be solved by the algorithm or be solved by the dual simplex algorithm.
You'll find implementations on Google.

How to raise the accuracy? - Activity Recognition using continuous Hidden Markov Models (Jahmm)

I am a novice at HMMs but I have tried to build a code using Jahmm for the UCI Human Activity Recognition data set. The data set has 561 features and 7352 rows, and also includes the xyz inertial values of both the accelerometer and gyroscope, and it is mainly for recognizing 6 activities: Walking, Walking Upstairs, Walking Downstairs, Sitting, Standing, and Laying. So far, I have tried the following:
With the xyz inertial values:
For each of the 6 activities, I trained 6 HMMs for each axis (for
both accelerometer and gyroscope), only using the activity train
data for the corresponding HMM. For each activity also, I applied
equal weights on all axes' probabilities (that is, when applied to
test data), and added them all to get the total for each activity.
The maximum probability will be the one picked. (I had no luck on
this one. There are activities with super high accuracies at the
same time super low on others.) Note: I used "ObservationReal", 6
states (tried states 2-10, actually), and just uniformly divided
initial values for the HMM. I sometimes get NaN values for some of
the activities.
I also tried scaling (z-score) the data first in R, and then
applying the above method, but still to no avail.
I also tried coding the inertial values with "ObservationVector,"
but I couldn't figure out how to set the initial Opdfs (it says that
it has to be a positive definite matrix).
With the feature values:
I found that the feature set is just too large to run on Jahmm, so
with the scaled data (because I couldn't get any decent results with
the out-of-the-box data though it's normalized [-1,1]), I ran the
train and test data on R for PCA and correlation before I fed them
on my Jahmm code (which consists of six 6-state HMMs, each for every
activity, taking the maximum probability with test data), and the
results are still not so good. Particularly the Sitting activity,
which always gets around 20% accuracy. (The same parameters with the
"Note" above)
I ran randomForest with the same data on R (with mtry=8), and got
the importance values. I separated the locomotive and static
activities first with 119 variables, then classified the locomotive
activities (Walking, W. Upstairs, W. Downstairs) with 89 features
(based on RF importance values) and static activities (Sitting,
Standing, Laying) with 5 variables. Separating the locomotive and
static activities is easy (2 states, 100%) but this method, with
adjusted HMM parameters, I only gained 86% overall accuracy. (Used
3-state HMMs for the second level)
I trained one HMM for all activities, with 6 states (corresponding
to 1 activity, as I've read in one paper). But I couldn't figure out
how to use the Viterbi after that. It tells me the Viterbi needs
List<Observation O> test sequences, but I obviously have
List<List<ObservationReal>> for my test data.
I have also tried HMM packages in R:
depmixS4 - doesn't have viterbi, and I have no idea how to get the
posterior probabilities with the test data (it gives the probs with
the train data only); I've tried contacting the author of the
package and he tried helping me, but the code he told me to try
gives me errors (I have yet to email him back).
RHmm - works like a charm at first; trained only one 6-state HMM
with all train data, but produces nans, resulting to a bad viterbi
sequence with the test data.
According to what I've read about HMMs so far, these results are too low for HMM. Am I doing something wrong? Should I do more preprocessing before I use the said techniques? Is the data really too large for HMM/Jahmm? Am I overfitting it? I am stuck now, but I really have to do Activity Recognition and HMMs for my project. I would be so glad to get suggestions/feedback from people who have already tried Jahmm and R for continuous HMMs. I am also open to study other languages, if that would mean it would finally work.

I just stumbled upon your question while searching for a scalable Java library. It seems you did not train HMM properly. When I first used HMM, I was also not able to get the correct results. I have used R to train and test HMM, here are some suggestions that can be helpful to you.
Properly assign random initial states when initializing the states and observable probabilities. Here is the code snippet from R using HMM library.
library(HMM)
....
...
ranNum<-matrix(runif(numStates*numStates, 0.0001, 1.000),nrow=numStates,ncol=numStates)
transitionInit <- ranNum/rowSums(ranNum)
ranNum<-matrix(runif(numStates*numSymbols, 0.0001, 1.000),nrow=numStates,ncol=numSymbols)
emissionInit <- ranNum/rowSums(ranNum)
rowSums(emissionInit)
hmm = initHMM(c(1:numStates),symbols,transProbs=transitionInit,emissionProbs=emissionInit)
Try to chop your rows into short sequences. I used Sliding window technique to chop them and then remove the redundant ones to avoid retraining and to save time.
You can save memory by replacing a string observable by an integer or a symbol
I used the following to train HMM using BaumWelch and measured the logForwardProbabilties to determine the likelihood (not probability). You need to sum the loglikelihood of each state to get the final log likelihood of the sequence
bw = baumWelch(hmm,trainSet,maxIterations=numIterations, delta=1E-9, pseudoCount=1E-9)
logForwardProbabilities <- forward(bw$hmm,validationSet[cnt,])
vProbs<-sum(logForwardProbabilities[,seqSize])
This is a negative number, calculate it for each of the 6 HMMS you trained and then see whichever is the bigger would represent a sequence.
I hope this might help you or someone else; if it's not too late.

AIMA implementation bayesian networks

I would like to code bayesian networks in java to understand them better, and I have found some code of Artificial Intelligence A Modern Approach (3rd Edition), "AIMA"
Do you recommend I read the code there and adapt to a particular problem, or how do I start?
Could you please orient me where in how to use the code?
I found google has it here and here ,

I would say there is no need to look at existing code if you want to learn. You will probably learn more by doing it yourself.
A good start would be to write code that does the following:
Compute Condition Probabilities from Joint Probability table,
For example, from P(A,B,C) compute P(A|B)
Compute Joint Probability Table from complete set of Conditional Probabilities
For example, from P(A|B,C)*P(B)*P(C) compute P(A,B,C).
Given a DAG, compute if A is d-seperated from B
Do all of the above naively and then go back and try to make them efficient.
It should give you a good understanding of what Bayesian Networks are (conditional probability tables) and what they are used for (reasoning about probability).

Help with using FFT to determine frequency of an audio sample

I'm currently developing a percussion tutorial program. The program requires that I can determine what drum is being played, to do this I was going to analyse the frequency of the drum recording and see if the frequency is within a given range.
I have been using the Apache math commons implementation for FFT so far (http://commons.apache.org/math/) but my question is, once I preform the FFT, how do I use the array of results to calculate the frequencies contained in the signal?
Note: I have also tried experimenting with using Autocorrelation, but it didn't seem to work to well with sample from a drum kit
Any help or alternative suggestions of how to determine what drum is being hit would be greatly appreciated
Edit: Since writing this I've found a great online lesson on implementing FFT in java for Time/ frequency transformations Spectrum Analysis in Java

In the area of music information retrieval, people often use a related metric known as the mel-frequency cepstral coefficients (MFCCs).
For any N-sample segment of your signal, take the FFT. Those resulting N samples are transformed into a set of MFCCs containing, say, 12 elements (i.e., coefficients). This 12-element vector is used to classify the instrument, including which drum is used.
To do supervised classification, you can use something like a support vector machine (SVM). LIBSVM is a commonly used library that has Java compatibility (and many other languages). You train the SVM with these MFCCs and their corresponding instrument labels. Then, you test it by feeding a query MFCC vector, and it will tell you which instrument it is.
So the basic procedure, in summary:
Get FFT.
Get MFCCs from FFT.
Train SVM with MFCCs and instrument labels.
Query the SVM with MFCCs of the query signal.
Check for Java packages that do these things. (They must exist. I just don't know them.) Relatively, drum transcription is easier than most other instrument groups, so I am optimistic that this would work.
For further reading, there are a whole bunch of articles on drum transcription.

When I made a program using a DFT, I had it create an array of Frequencies and Amplitudes for each frequency. I could then find the largest amplitudes, and compare those to musical notes, getting a good grasp on what was played. If you know the approximate frequency of the drum, you should be able to do that.

Boosting my GA with Neural Networks and/or Reinforcement Learning

As I have mentioned in previous questions I am writing a maze solving application to help me learn about more theoretical CS subjects, after some trouble I've got a Genetic Algorithm working that can evolve a set of rules (handled by boolean values) in order to find a good solution through a maze.
That being said, the GA alone is okay, but I'd like to beef it up with a Neural Network, even though I have no real working knowledge of Neural Networks (no formal theoretical CS education). After doing a bit of reading on the subject I found that a Neural Network could be used to train a genome in order to improve results. Let's say I have a genome (group of genes), such as
1 0 0 1 0 1 0 1 0 1 1 1 0 0...
How could I use a Neural Network (I'm assuming MLP?) to train and improve my genome?
In addition to this as I know nothing about Neural Networks I've been looking into implementing some form of Reinforcement Learning, using my maze matrix (2 dimensional array), although I'm a bit stuck on what the following algorithm wants from me:
(from http://people.revoledu.com/kardi/tutorial/ReinforcementLearning/Q-Learning-Algorithm.htm)
1. Set parameter , and environment reward matrix R
2. Initialize matrix Q as zero matrix
3. For each episode:
* Select random initial state
* Do while not reach goal state
o Select one among all possible actions for the current state
o Using this possible action, consider to go to the next state
o Get maximum Q value of this next state based on all possible actions
o Compute
o Set the next state as the current state
End Do
End For
The big problem for me is implementing a reward matrix R and what a Q matrix exactly is, and getting the Q value. I use a multi-dimensional array for my maze and enum states for every move. How would this be used in a Q-Learning algorithm?
If someone could help out by explaining what I would need to do to implement the following, preferably in Java although C# would be nice too, possibly with some source code examples it'd be appreciated.

As noted in some comments, your question indeed involves a large set of background knowledge and topics that hardly can be eloquently covered on stackoverflow. However, what we can try here is suggest approaches to get around your problem.
First of all: what does your GA do? I see a set of binary values; what are they? I see them as either:
bad: a sequence of 'turn right' and 'turn left' instructions. Why is this bad? Because you're basically doing a random, brute-force attempt at solving your problem. You're not evolving a genotype: you're refining random guesses.
better: every gene (location in the genome) represents a feature that will be expressed in the phenotype. There should not be a 1-to-1 mapping between genome and phenotype!
Let me give you an example: in our brain there are 10^13ish neurons. But we have only around 10^9 genes (yes, it's not an exact value, bear with me for a second). What does this tell us? That our genotype does not encode every neuron. Our genome encodes the proteins that will then go and make the components of our body.
Hence, evolution works on the genotype directly by selecting features of the phenotype. If I were to have 6 fingers on each hand and if that would made me a better programmer, making me have more kids because I'm more successful in life, well, my genotype would then be selected by evolution because it contains the capability to give me a more fit body (yes, there is a pun there, given the average geekiness-to-reproducibily ratio of most people around here).
Now, think about your GA: what is that you are trying to accomplish? Are you sure that evolving rules would help? In other words -- how would you perform in a maze? What is the most successful thing that can help you: having a different body, or having a memory of the right path to get out? Perhaps you might want to reconsider your genotype and have it encode memorization abilities. Maybe encode in the genotype how much data can be stored, and how fast can your agents access it -- then measure fitness in terms of how fast they get out of the maze.
Another (weaker) approach could be to encode the rules that your agent uses to decide where to go. The take-home message is, encode features that, once expressed, can be selected by fitness.
Now, to the neural network issue. One thing to remember is that NNs are filters. They receive an input. perform operations on it, and return an output. What is this output? Maybe you just need to discriminate a true/false condition; for example, once you feed a maze map to a NN, it can tell you if you can get out from the maze or not. How would you do such a thing? You will need to encode the data properly.
This is the key point about NNs: your input data must be encoded properly. Usually people normalize it, maybe scale it, perhaps you can apply a sigma function to it to avoid values that are too large or too small; those are details that deal with error measures and performance. What you need to understand now is what a NN is, and what you cannot use it for.
To your problem now. You mentioned you want to use NNs as well: what about,
using a neural network to guide the agent, and
using a genetic algorithm to evolve the neural network parameters?
Rephrased like so:
let's suppose you have a robot: your NN is controlling the left and right wheel, and as input it receives the distance of the next wall and how much it has traveled so far (it's just an example)
you start by generating a random genotype
make the genotype into a phenotype: the first gene is the network sensitivity; the second gene encodes the learning ratio; the third gene.. so on and so forth
now that you have a neural network, run the simulation
see how it performs
generate a second random genotype, evolve second NN
see how this second individual performs
get the best individual, then either mutate its genotype or recombinate it with the loser
repeat
there is an excellent reading on the matter here: Inman Harvey Microbial GA.
I hope I did you some insight on such issues. NNs and GA are no silver bullet to solve all problems. In some they can do very much, in others they are just the wrong tool. It's (still!) up to us to get the best one, and to do so we must understand them well.
Have fun in it! It's great to know such things, makes everyday life a bit more entertaining :)

There is probably no 'maze gene' to find,
genetic algorithms are trying to setup a vector of properties and a 'filtering system' to decide by some kind of 'surival of the fittest' algorithm to find out which set of properties would do the best job.
The easiest way to find a way out of a maze is to move always left (or right) along a wall.
The Q-Algorithm seems to have a problem with local maxima this was workaround as I remember by kicking (adding random values to the matrix) if the results didn't improve.
EDIT: As mentioned above a backtracking algorithm suits this task better than GA or NN.
How to combine both algorithm is described here NeuroGen descibes how GA is used for training a NN.

Try using the free open source NerounDotNet C# library for your Neural networks instead of implementing it.
For Reinforcement Learning library, I am currently looking for one, especially for Dot NET framework..