I would like to know if there is a way to align the RGB picture and the depth data of a Kinect V2 using the colour data as a starting point using Java, I am actually using java for Kinect as a wrapper and it does not seem to give me the possibility for that. Is there any way doing that?
Finally got around it by using #Spektre answer here, I got to play around with the formulas to make it work but it seem fine to me.
Rectified for my needs, it gives:
int alignx= (((x-512)<<8)/241)+Width;
int aligny= (((y-424)<<8)/240)+25+Height;
It works fine as long as the Kinect is at the same level that the object you want to target (ie: no pitch used).
I don't quite agree with Alex Acquier’s answer, it is not the right approach I feel. I have faced the same issue and I know I am doing this 8 months late, but in the interest of others who came here searching for the solution, I now present it here:
The thing is, you don't have to manually align the RGB and the Depth frames. There is a class already available that can do that for you, "IMultiSourceFrameReader". Using this as a source, you can sure to be making the point cloud in the right way.
Now this is okay if you just want to use the feeds. But if somewhere down your code, if you are going to be using some kind of a coordinate system and if you are going to be needing the coordinates of the RGB and the depth pixels, then you would expect them them to be the same, right? Because you are using the aligned coordinates after all, right? But you won't get the coordinates aligned till to use the "ICoordinateMapper" class. This class will make all the coordinates from all the different sensors, RGB and Infra, to align as well and will return the aligned coordinates.
Please refer to this source, it has been my go to source for Kinect V2 since a long time.
Related
I've not found anything here or on google. I'm looking for a way to identify shapes (circle, square, triangle and various other shapes) from a image file. Some examples:
You get the general idea. Not sure if BoofCV is the best choice here but it looks like it should be straightforward enough to use, but again I know nothing about it. I've looked at some of the examples and I though before I get in over my head (which is not hard to do some days), I thought I would ask if there is any info out there.
I'm taking a class on Knowledge Based AI solving Ravens Progressive Matrix problems and the final assignment will use strictly visual based images instead of the text files with attributes. We are not being graded on the visual since we only have a few weeks to work on this section of the project and we are encouraged to share this information. SOF has always been my go to source for information and I'm hoping someone out there might have some ideas on where to start with this...
Essentially what I want to do is detect the shapes (?? convert them into 2D geometry) and then make some assumptions about attributes such as size, fill, placement etc, create a text file with these attributes and then using that, send it through my existing code based that I wrote for my other projects to solve the problems.
Any suggestions????
There are a lot of ways you can do it. One way is to find the contour of the shape then fit a polygon to it or a oval. If you git a polygon to it and there are 4 sides with almost equal length then its a square. The contour can be found with binary blobs (my recommendation for the above images) or canny edge.
http://boofcv.org/index.php?title=Example_Fit_Polygon
http://boofcv.org/index.php?title=Example_Fit_Ellipse
I'm not totally sure this is in the right place, so let me know...
Just so I'm being totally transparent, this is part of a coursework for my University course. To that end, please don't post an answer but please DO give me hints & nudges in the right direction. I've been working on it for a few days without much success.
I've been tasked with converting a grayscale image into an RGB image. It's been suggested that we must segment the image and add colours to each segment with an algorithm. It'a also noted that we could develop algorithms for both the RGB & HSI colourspace to improve the visualisation.
My first thought was that we could segment the image using some threshold technique (on the grayscale intensity values) and then add colours to segments, but I'm not sure if this is right.
I'm programming in Java and have use of the OpenCV library.
Any thought / ideas / hints / suggestions appreciated :)
A very nice presentation describing different colorization algorithms
http://www.cs.unc.edu/~lazebnik/research/fall08/lec06_colorization.pdf
The basic idea is to match texture/luminance in source and target images and then transfer the color information. The better match you have, better would be your solution. However, matching intensity values in Lab space may be misleading as many pixels in the source image can have similar luminance values around them. To overcome this problem, segmenting the source image using texture information can prove helpful and then you can transfer color values of matching textures instead of luminance values.
Yesterday I tried to solve a Problem I had the entire Day and it is still unsolved. I searched for every combinations of words I could imagine to find solutions on Google etc. But without success. :(
The Problem consists of following idea:
I started programming GLES20 (Not GLES10!) on Android. There are many other ways how to compute matrixes and objects. So there aren't any methods like "Pop" or "push" Matrix.
I want to rotate a globe/sphere only by touching and moving my fingers. Touching functions etc works fine but the rotation itself never does. Everytime I first rotate by x-axis and then rotating by y-axis the rotation is still computed by local space axis of the object and Not two global axis. This happens whatever I do... :/
There are some people searching the solution for the Same Problem, but mostly GLES10 or completly different programming languages, never GLES20 and Java.
I will also post some Code parts later, when I get access to a Computer.
perhaps someone already understands what my Problem is.
Thank you so much! :)
Chrise
In OpenGL ES 2.0 there aren't any of the matrix functions and you're supposed to take care of your own matrices. So, for pushing and popping matrices what you need to do is when you're drawing you copy your matrix to a temporary one, do whatever transformation on it and pass the temporary matrix to the vertex shader.
i'm developing a software that compare images and i need to do it in a fast way! Actually i compare them using plain c but it's too slow.
I want to compare them using shaders and a couple of gl surfaces (textures), using c and not java, but this doesn't change the situation so much, and get back a list of changed parts, but i really don't know where to start.
Basically i want to use something like SIMD neon instruction to compare pixel colors to check for changes (well, i need to check only the first pixel fragment color, ex. only red ... these are photos so is unrealistic that it doesn't change) but instead to use neon instructions i want to use pixel shaders to do the comparison and get the list of changed part back
More, if it's possible, i want to use parallel comparison on the same image splitting it in blocks :)
Someone can give an hit?
note: i know that i can't output back a list of stuff, but, well, use a third texture as output is good anyway for me (if i put on the texture 2 ushorts that indicates x and y i'm ok and with an uint on the end of the texture that report the number of changed pixels)
OpenGL ES 1.1 doesn't have shaders, and the best route I can think of for what you want to do ends with a 50% reduction in colour precision. Issues are:
without extensions there's additive blending, but not subtractive. No problem, just upload the second of your textures with all colour values inverted.
OpenGL clamps output colours to the range [0, 1] and without extensions you're limited to one byte per channel. So you'd need to upload textures with 7bit colour channels to ensure you got the correct results within the 8bits coming back.
Shaders would allow a slightly circuitous route around that, because you can add or subtract or do whatever you want, and can split up the results. If you're sending two three channel 24bit images in to get a four channel 32bit image out, obviously there's enough space to fit in 9 bits per source channel, even though you're going to have to divide the data oddly and reconstruct it later.
In practice you're going to pay quite a lot for uploading and downloading images from the GPU, so NEON might be a better choice not just to avoid packing peculiarities. Assuming the Android kit supplies the same compiler intrinsics as the iPhone kit (likely, since they'll both include GCC), this page has a bit of an introduction showing how to convert an image to greyscale. So it's not exactly what you're looking for, but it's image processing in C using NEON so it should be a good start.
In both cases you're likely to end up with an image of the differences, rather than a simple count and list. A count is a concurrent operation, whatever way you think about it, so isn't
really something you'd do in GL or via NEON. You'd need to inspect the final image to work it out.
Here’s my task which I want to solve with as little effort as possible (preferrably with QT & C++ or Java): I want to use webcam video input to detect if there’s a (or more) crate(s) in front of the camera lens or not. The scene can change from "clear" to "there is a crate in front of the lens" and back while the cam feeds its video signal to my application. For prototype testing/ learning I have 2-3 images of the “empty” scene, and 2-3 images with one or more crates.
Do you know straightforward idea how to tackle this task? I found OpenCV, but isn't this framework too bulky for this simple task? I'm new to the field of computer vision. Is this generally a hard task or is it simple and robust to detect if there's an obstacle in front of the cam in live feeds? Your expert opinion is deeply appreciated!
Here's an approach I've heard of, which may yield some success:
Perform edge detection on your image to translate it into a black and white image, whereby edges are shown as black pixels.
Now create a histogram to record the frequency of black pixels in each vertical column of pixels in the image. The theory here is that a high frequency value in the histogram in or around one bucket is indicative of a vertical edge, which could be the edge of a crate.
You could also consider a second histogram to measure pixels on each row of the image.
Obviously this is a fairly simple approach and is highly dependent on "simple" input; i.e. plain boxes with "hard" edges against a blank background (preferable a background that contrasts heavily with the box).
You dont need a full-blown computer-vision library to detect if there is a crate or no crate in front of the camera. You can just take a snapshot and make a color-histogram (simple). To capture the snapshot take a look here:
http://msdn.microsoft.com/en-us/library/dd742882%28VS.85%29.aspx
Lots of variables here including any possible changes in ambient lighting and any other activity in the field of view. Look at implementing a Canny edge detector (which OpenCV has and also Intel Performance Primitives have as well) to look for the outline of the shape of interest. If you then kinda know where the box will be, you can perhaps sum pixels in the region of interest. If the box can appear anywhere in the field of view, this is more challenging.
This is not something you should start in Java. When I had this kind of problems I would start with Matlab (OpenCV library) or something similar, see if the solution would work there and then port it to Java.
To answer your question I did something similar by XOR-ing the 'reference' image (no crate in your case) with the current image then either work on the histogram (clustered pixels at right means large difference) or just sum the visible pixels and compare them with a threshold. XOR is not really precise but it is fast.
My point is, it took me 2hrs to install Scilab and the toolkits and write a proof of concept. It would have taken me two days in Java and if the first solution didn't work each additional algorithm (already done in Mat-/Scilab) another few hours. IMHO you are approaching the problem from the wrong angle.
If really Java/C++ are just some simple tools that don't matter then drop them and use Scilab or some other Matlab clone - prototyping and fine tuning would be much faster.
There are 2 parts involved in object detection. One is feature extraction, the other is similarity calculation. Some obvious features of the crate are geometry, edge, texture, etc...
So you can find some algorithms to extract these features from your crate image. Then comparing these features with your training sample images.