Develop Reference

Comparing between two skeleton tracking on processing 3

I am currently doing my dissertation which would involve in having 2 people a professional athlete and an amateur. First with the image processing skeletonization I would like to record the professional athlete while performing the squat exercise , then when the amateur performs the exercise I want to be able to compare the professional skeleton with that of the amateur to see if it is properly formed.
Please I m open for any suggestions and opinions , Would gladly appreciate some help

Here lies your question:
properly formed.
What does properly performed actually mean ? How can this be quantified ?
Bare in mind I'm not an athletic/experienced in this field.
If I were given the task I would counter-intuitively go in the opposite direction:
moving away form Processing 3/kinect/computer. I would instead:
find a professional athlete
find a skilled with trainer with functional mobility training.
find an amateur (probably easiest)
Item 2 will be trickier.For example FMS seems to put a lot of emphasis on correct exercising and mobility (to enhance performance and reduce risk of injuries). I'm not sure if that's the only approach or the best. You might want to check opinions on Physical Fitness, consult with people studying/teaching exercise science, etc. Do check credentials as it feels like a field where everyone has an opinion/preference.
The idea is to understand how a professional educated trainer asses correct movement. Take note of how that works in the real world and try to systemise it.
What are the cues for a correct execution ?
is the key poses
the motion in between
how the skeletal and muscular system work together/ the weights/forces applied/etc.
Having a better understanding of how this works in the real world should lead you to things you can start quantifying/comparing numerically on a computer.
Try to make a checklist/score system manually using a pen and paper based on the information you gather. If this works you already have a system you can start programming.
The next step is acquiring the data.
This is probably where the kinect comes, but bare in mind:
the second version of the kinect is more precise than the first
there is a Kinect2 SDK wrapper for Processing 3: use that if you can (windows only). There is a way you can get libfreenect2 working with OpenNI on osx/linux and therefore with SimpleOpenNI in Processing, but it's not straight forward and you won't have the same precision on the skeleton tracking algorithm
use data that is as precise as possible:
you can get the accuracy of a tracked skeleton joint
use an environment that doesn't contain a complex background (makes it easy to segment users and detect/track skeletons with little change of mistaking it for something else). prefer artificial non-incandescent light (less of a problem with kinect v2, but still you want as little IR interference as possible).
comparing orientation matrices or joints on single poses might not be enough to get the full picture: how do you capture/quantify motion taking into account the things that the kinect can't easily see: muscles flexing/forces applied/moving centre of gravity/etc.
try to use a grid system that will make it simple to pair the digital values with real world measurements. Check out how people used to study motion in the past, for example Étienne-Jules Marey or Eadweard Muybridge
Motion capture by Étienne-Jules Marey
Motion study by Eadweard Muybridge (notice the grid)
It's a pretty full on project to get right involving bits of anatomy/physics/kinematics/etc.
Start with the research first:
how did people study this in the past ?
what are the current developments ?
how does it work in the real world (without computers) ?
Take your constraints into account:
what resources (people/gear/etc.) can you use ?
how much time do you have available ?
Given the above, what topic/section of the project can be realistically be tackled to get useful results.
Overall probably something along these lines:
background research
real world studies
comparison system has feature which can be measured both with kinect and by a person
record data (real world data + mobility comparison evalutation and kinect data + mobility comparison)
compare data
write evaluation of findings (how effective is the system? what are limitations ? what could be improved (future work) ? etc.)
In short be aware of the kinect limitations: skeleton tracking is probability based: it's not 100% accurate. use data that's as clean/correct as possible to begin with (make it easy to acquire good data if you can control the capture environment). From what a real trainer would track, what could you track with a kinect ? do a comparison of the intersecting measurements.

GPS data comparison after smoothing

I'm trying to compare multiple algorithms that are used to smooth GPS data. I'm wondering what should be the standard way to compare the results to see which one provides better smoothing.
I was thinking on a machine learning approach. To crate a car model based on a classifier and check on which tracks provides better behaviour.
For the guys who have more experience on this stuff, is this a good approach? Are there other ways to do this?

Generally, there is no universally valid way for comparing two datasets, since it completely depends on the applied/required quality criterion.
For your appoach
I was thinking on a machine learning approach. To crate a car model
based on a classifier and check on which tracks provides better
behaviour.
this means that you will need to define your term "better behavior" mathematically.
One possible quality criterion for your application is as follows (it consists of two parts that express opposing quality aspects):
First part (deviation from raw data): Compute the RMSE (root mean squared error) between the smoothed data and the raw data. This gives you a measure for the deviation of your smoothed track from the given raw coordinates. This means, that the error (RMSE) increases, if you are smoothing more. And it decreases if you are smoothing less.
Second part (track smoothness): Compute the mean absolute lateral acceleration that the car will experience along the track (second deviation). This will decrease if you are smoothing more, and it will increase if you are smoothing less. I.e., it behaves in contrary to the RMSE.
Result evaluation:
(1) Find a sequence of your data where you know that the underlying GPS track is a straight line or where the tracked object is not moving. Note, that for those tracks, the (lateral) acceleration is zero by definition(!).
For these, compute RMSE and mean absolute lateral acceleration.
The RMSE of appoaches that have (almost) zero acceleration results from measurement inaccuracies!
(2) Plot the results in a coordinate system with the RMSE on the x axis and the mean acceleration on the y axis.
(3) Pick all approaches that have an RMSE similar to what you found in step (1).
(4) From those approaches, pick the one(s) with the smallest acceleration. Those give you the smoothest track with an error explained through measurement inaccuracies!
(5) You're done :)

I have no experience on this topic but I have few things in mind that may help you.
You know it is a car. You know that the data is generated from a car so you can define a set of properties of a car. For example if a car is moving with speed above 50km than the angle of the corner should be at least 110 degrees. I am absolutely guessing with the values but if you do a little research i am sure you will be able to define such properties. Next thing you can do is to test how each approximation fits the car properties and choose the best one.
Raw data. I assume you are testing all methods on a part of given road. You can generate a "raw gps track" - a track that best fits the movement of a car. Google maps may help you to generate such track os some gps devise with higher accuracy. Than you measure the distance between each approximation and your generated track - the one with the min distance wins.

i think you easily match the coordinates after the address conversion.
because address have street,area and city. so you can easily match the different radius.
let try this link

Take a look at this paper that discusses comparing machine learning algorithms:
"Choosing between two learning algorithms
based on calibrated tests" available at:
http://www.cs.waikato.ac.nz/ml/publications/2003/bouckaert-calibrated-tests.pdf
Also check out this paper:
"Bayesian Comparison of Machine Learning Algorithms on Single and
Multiple Datasets" available at:
http://www.jmlr.org/proceedings/papers/v22/lacoste12/lacoste12.pdf
Note: It is noted from the question that you are looking into the best way to compare the results for machine learning algorithms and are not looking for additional machine learning algorithms that may implement this feature.

Machine Learning is not an well suited approach for that task, you would have to define what is good smoothing...
Principially your task cannot be solved by an algorithm that gives an general answer because every smoothing destroy the original data by some amount and adds invented positions, and different systems/humans that use the smoothed data react differently on that changed data.
The question is: What do you want to achieve with smoothing?
Why do you need smoothing? (have you forgotten to implement or enable a stand still filter that eliminates movement while the vehicle is standing still, which in GPS introduces jumping location during stand still?)
The GPS chip has already built in a (best possible?) real time smoothing using a Kalman filter, having on the one side more information than a post processed smotthing algo, on the other side it has less.
So next you have to ask yourself: do you compare post processing smooting algos or real time algos? (probably post processing) Comparing a real time smoothing algorithm with a post process smoothing algorithm is not fair.
Again: What do you expect from smoothed data: That they look somewhat fine, but unrealistic like photoshopped models for tv-advertisments?
What is good smoothing? near to real vehicle postion which nobody ever knows, or a curve whith low acceleration?
I would prefer an smoothing algorithm that produces the curve most near to the real (usually unknown) vehicle trajectory.
Or you might just think it should somehow look beautifull: In that case overlay the curves with different colors, display it on a satelitte image map, and let a team of humans (experts at least owning and driving an own car) decide what looks good and realistic.
We humans have the best multi purpose pattern matching algorithm built in.
Again why smooth?: for display in a map to please humans that look at that map?
or to use the smoothed tracks to feed other algorithms that have problems with the original data?
To please humans I have given an answer above.
To please other algorithms:
What they need? nearer positions? or better course value / direction between points.
What attributes do you want to smooth: only the latitude, longitude coordinates, or also the speed value, and course value?
I have much professional experience with GPS tracks, and recommend, to just remove every location under 7km/h and keep the rest as it is. In most cases there is no need for further smoothing.
Otherwise it gets expensive:
A possible solution:
1) You arrange a 2000€ Reference GPS receiver delivered with a magnetic vehicle roof antenna (E.g Company hemisphere 2000 GPS receiver) and use that as reference
2) You use a comnsumer GPS usually used for your task (smartphone, etc.)
Both mounted inside the car: drive some test tracks, in good conditions (highways) but more tracks at very bad: strong curves combined with big houses left and right. And through tunnel, a struight and a curved one, if you have one.
3) apply the smoothing algoritms to the consumer GPS tracks
4) compare the smoothed to the reference track, by matching two positions and finally calulate the (RMSE Root mean squared error)
Difficulties
matching two positions: Hopefully the time can be exactly matched which is usually not the case (0,5s offset possible).
Think what do you do when having an GPS outage.
Consider first to display a raw track and identify what kind of unsmoothed data is not suitable/ nice looking. (Probably later posting the pics here)

what about using the good old Kalman Filter!

How is userinput kept independent of FPS when using for example Java Slick 2D or .NET XNA framework?

I have written a few 2D games in the past using libraries such as LWJGL (with a Slick2D wrapper) and the XNA framework, but one thing i have never been able to grasp (or have the need to) is how the user input is kept constant, eq not dependent on FPS.
I'm looking for a more generic answer rather than framework specific. I understand it has something to do with time measured between frame updates ?
Thank you

I can't speak for some of those other frameworks, but I know that XNA basically lets you poll the current input state (are the buttons up or down?) whenever you like. You usually do it each frame.
What this means is, if your player happens to be a ninja and can hit keys faster than 60FPS, it is possible that they may hit a key (or mouse button) between pollings and you miss it. In practice it is almost never an issue.
If it does bother you, the solution to this problem is to hook the Windows message pump and receive keyboard up/down events.
For general gameplay it is really not worth the effort. Usually the only time where you really must capture every keystroke is when the user is typing text. So rather than capture key up/down events, you capture character events (WM_CHAR). This means you won't miss a keypress. But the more important problem that this solves is that it offloads key-to-character translation to Windows - allowing it to handle key-repeat, keyboard layout, shifted characters, etc, for you - allowing your game to behave like any other Windows application.
(Of course, if you can get away with just using the polling-based framework input stuff - go with that - it's much easier to implement and less platform-specific.)
The above only matters when you are detecting distinct key presses (eg: tap to fire this gun), as opposed to holding keys down (eg: accelerate this vehicle).
The alternate interpretation of your question is you are suggesting that a key may come up half way through a frame - how do you account for that, in a game with a discrete time-step?
Generally you don't worry about it. Just as 60 frames per second is fast enough to discretely calculate your game state and appear smooth and continuous to a human, it's fast enough to accept input.
But what happens if you're not running at 60FPS? If you're running at 30FPS (as you might on a mobile platform), then it can make your inputs - particularly analogue inputs - feel much smoother if you poll them at 60FPS. The easiest way to do this is to simply do two Updates for each Draw - if your Update is not too taxing on the CPU.

Can anyone help me create a simple machine learning algorithm for a 'pet' application

I'm creating an organism simulator for Android, so I guess the algorithm would ideally be in Java. I realize that there is a whole Stanford course on Machine Learning available on youtube, but I simply don't have the time to sit through the whole thing, and I think for my purposes the solution could be very simple.
The organism will be interacted with by the touchscreen primarily, or even if it's interacted with through the mic or accelerometer the inputs in the algorithm will mostly amount to coordinate positions for the different limbs. I think it will be inelegant to have a 'scolding' or 'rewarding' mechanism for random behaviors, so I would like to avoid that. So tracking general directions or patterns in movements and being able to repeat them when they have a high enough frequency would be the goal.
To be honest I'm not really sure how hard this is to accomplish, but I'd like to hear any feedback to know how much more I have to research before I can implement it.
EDIT: Is this a genetic algorithm? The problem is I have no idea how to measure a successful or non successful evolution.
EDIT 2: Okay, I'll try to add as much detail as possible. The application is still in concept stage at the moment, but I just wanted to know how difficult the algorithm would be to put it. So I'm building it in Processing, which is really just Java. The organism would be comprised of limbs that have a fixed distance between them, but are allowed to move independently from the center piece. The limbs move around freely and would find random points periodically to ease to. The organism would have a center appendage that has x and y coordinates as well, and each of the outer limbs would move in relation to that. The user could interact with the organism by manually moving the appendages or the center piece with drags on the touch screen. When the organism is being interacted with is where the algorithm would be used, because there's no point in learning from just random numbers. So I guess the algorithm would take the x and y coordinates of the center piece into consideration, and each appendage would have its own version of the algorithm that learns independently from the others. For instance, if the user continually dragged the organism to the right side of the touch screen, it might be more attracted to that place when it isn't being interacted with. I hope that clarifies a little bit.

I think that for your case, you should try to sit down and write down what are the variables that you can observe and what are the variables that you want to predict
Observable variables: position of the appendage, how many times a specific one is interacted with, for how long, ...
Variables you want to predict: which appendage will be interacted with next time, ...
Once you have the input variables and output variables, you can try to go through the list of standard machine learning algorithms. There are Weka(Java), Rapidminer, KNIME ... which are both libraries and standalone tools. Try to throw your problem at the available tools and see if you are doing better than chance.
If you are, tune its parameters. If you are not performing better than chance, you should ask your Data Mining/Machine Learning friends. They will know best what will work for your problem.
Other things that might affect your choice of algorithms:
Are there hidden states?
Are the variables independence?

The way I see it, all you'd need to do is have an array of, for example, the appendage coordinates, then just average them out and have it move towards that point on the screen

Best way to implement game playback?

I'm creating a grid based game in Java and I want to implement game recording and playback. I'm not sure how to do this, although I've considered 2 ideas:
Several times every second, I'd record the entire game state. To play it back, I write a renderer to read the states and try to create a visual representation. With this, however, I'd likely have a large save file, and any playback attempts would likely have noticeable lag.
I could also write every key press and mouse click into the save file. This would give me a smaller file, and could play back with less lag. However, the slightest error at the start of the game (For example, shooting 1 millisecond later) would result in a vastly different game state several minutes into the game.
What, then, is the best way to implement game playback?
Edit- I'm not sure exactly how deterministic my game is, so I'm not sure the entire game can be pieced together exactly by recording only keystrokes and mouse clicks.

A good playback mechanism is not something that can be simply added to a game without major difiiculties. The best would be do design the game infrastructure with it in mind. The command pattern can be used to achieve such a game infrastructure.
For example:
public interface Command{
void execute();
}
public class MoveRightCommand implements Command {
private Grid theGrid;
private Player thePlayer;
public MoveRightCommand(Player player, Grid grid){
this.theGrid = grid;
this.thePlayer = player;
}
public void execute(){
player.modifyPosition(0, 1, 0, 0);
}
}
And then the command can be pushed in an execution queue both when the user presses a keyboard button, moves the mouse or without a trigger with the playback mechanism. The command object can have a time-stamp value (relative to the beginning of the playback) for precise playback...

Shawn Hargreaves had a recent post on his blog about how they implemented replay in MotoGP. Goes over several different approaches and their pros and cons.
http://blogs.msdn.com/shawnhar/archive/2009/03/20/motogp-replays.aspx

Assuming that your game is deterministic, it might be sufficient if you recorded the inputs of the users (option 2). However, you would need to make sure that you are recognizing the correct and consistent times for these events, such as when it was recognized by the server. I'm not sure how you handle events in the grid.
My worry is that if you don't have a mechanism that can uniformly reference timed events, there might be a problem with the way your code handles distributed users.
Consider a game like Halo 3 on the XBOX 360 for example - each client records his view of the game, including server-based corrections.

Why not record several times a second and then compress your output, or perhaps do this:
recordInitialState();
...
runs 30 times a second:
recordChangeInState(previousState, currentState);
...
If you only record the change in state with a timestamp(and each change is small, and if there is no change, then record nothing), you should end up with reasonable file sizes.

There is no need to save everything in the scene for every frame. Save changes incrementally and use some good interpolation techniques. I would not really use a command pattern based approach, but rather make checks at a fixed rate for every game object and see if it has changed any attribute. If there is a change that change is recorded in some good encoding and the replay won't even become that big.

How you approach this will depend greatly on the language you are using for your game, but in general terms there are many approaches, depending on if you want to use a lot of storage or want some delay. It would be helpful if you could give some thoughts as to what sacrifices you are willing to make.
But, it would seem the best approach may be to just save the input from the user, as was mentioned, and either store the positions of all the actors/sprites in the game at the same time, which is as simple as just saving direction, velocity and tile x,y, or, if everything can be deterministic then ignore the actors/sprites as you can get their information.
How non-deterministic your game is would also be useful to give a better suggestion.
If there is a great deal of dynamic motion, such as a crash derby, then you may want to save information each frame, as you should be updating the players/actors at a certain framerate.

I would simply say that the best way to record a replay of a game depends entirely on the nature of the game. Being grid based isn't the issue; the issue is how predictable behaviour is following a state change, how often there are new inputs to the system, whether there is random data being injected at any point, etc, You can store an entire chess game just by recording each move in turn, but that wouldn't work for a first person shooter where there are no clear turns. You could store a first person shooter by noting the exact time of each input, but that won't work for an RPG where the result of an input might be modified by the result of a random dice roll. Even the seemingly foolproof idea of taking a snapshot as often as possible isn't good enough if important information appears instantaneously and doesn't persist in any capturable form.
Interestingly this is very similar to the problem you get with networking. How does one computer ensure that another computer is made aware of the game state, without having to send that entire game state at an impractically high frequency? The typical approach ends up being a bespoke mixture of event notifications and state updates, which is probably what you'll need here.

I did this once by borrowing an idea from video compression: keyframes and intermediate frames. Basically, every few seconds you save the complete state of the world. Then, once per game update, you save all the changes to the world state that have happened since the last game update. The details (how often do you save keyframes? What exactly counts as a 'change to the world state'?) will depend on what sort of game information you need to preserve.
In our case, the world consisted of many, many game objects, most of which were holding still at any given time, so this approach saved us a lot of time and memory in recording the positions of objects that weren't moving. In yours the tradeoffs might be different.