Using Java with OpenCV and Mask_RCNN - java

I've had a lot of experience with Java, and have already written my own utilities for processing imagery. I realise that there are many tutorials out there for Python, but I'm much more inclined to use Java as I would like to integrate some of the facilities of OpenCV 4.1 with my existing Java code.
One of my experiments is to try and obtain the masks of objects identified ion images using the coco dataset
// Give the textGraph and weight files for the model
private static final String TEXT_GRAPH = "models/mask_rcnn_inception_v2_coco_2018_01_28.pbtxt";
private static final String MODEL_WEIGHTS = "models/mask_rcnn_inception_v2_coco_2018_01_28/frozen_inference_graph.pb";
private static final String CLASSES_FILE = "models/mscoco_labels.names";
TextFileConsumer consumer = new TextFileConsumer();
File textFile = new File(CLASSES_FILE);
if (textFile.exists()){
BufferedReader fr1 = new BufferedReader(new FileReader(textFile));
fr1.lines().forEach(consumer::storeLines);
}
String[] CLASSES = consumer.lines.toArray(new String[0]);
Mat image = Imgcodecs.imread("images/bird.jpg");
Size size = image.size();
int cols = image.cols();
int rows = image.rows();
double h = size.height;
double w = size.width;
int hh = (int)size.height;
int ww = (int)size.width;
Mat blob = org.opencv.dnn.Dnn.blobFromImage(image, 1.0, new Size(w, h), Scalar.all(0), true, false);
// Load the network
Net net = org.opencv.dnn.Dnn.readNetFromTensorflow(MODEL_WEIGHTS, TEXT_GRAPH);
net.setPreferableBackend(org.opencv.dnn.Dnn.DNN_BACKEND_OPENCV);
net.setPreferableTarget(org.opencv.dnn.Dnn.DNN_TARGET_CPU);
net.setInput(blob);
ArrayList<String> outputlayers = new ArrayList<String>();
ArrayList<Mat> outputMats = new ArrayList<Mat>();
outputlayers.add("detection_out_final");
outputlayers.add("detection_masks");
net.forward(outputMats,outputlayers);
Mat numClasses = outputMats.get(0);
numClasses = numClasses.reshape(1, (int)numClasses.total() / 7);
for (int i = 0; i < numClasses.rows(); ++i) {
double confidence = numClasses.get(i, 2)[0];
//System.out.println(confidence);
if (confidence > 0.2) {
int classId = (int) numClasses.get(i, 1)[0];
String label = CLASSES[classId] + ": " + confidence;
System.out.println(label);
int left = (int)(numClasses.get(i, 3)[0] * cols);
int top = (int)(numClasses.get(i, 4)[0] * rows);
int right = (int)(numClasses.get(i, 5)[0] * cols);
int bottom = (int)(numClasses.get(i, 6)[0] * rows);
System.out.println(left + " " + top + " " + right + " " + bottom);
}
}
Mat numMasks = outputMats.get(1);
So far so good. My problem now is trying to obtain the segmentation shapes from numMasks Mat.
The size gives 90x100, total 2025000 and type -1*-1*CV_32FC1, isCont=true, isSubmat=true.
I know I have to reshape too, perhaps like this:
numMasks = numMasks.reshape(1, (int)numMasks.total() / 90);
Any help would be greatly appreciated.
Further edit
I think the masks are 15 x 15.
90 x 100 x 15 x 15 gives 2025000 so that gives me the total I saw earlier so I need to resize a mask and mix with original image.

Related

In android how do I count the number of lines in a image using openCV

This is what I tried.I got the output as a grayscale image in img2 of an Imageview object.
The problem is as lines.cols() considers everything as a line.
I want the count to be exactly the number of larger lines as shown in the 1stpic (I mean the lines that seperates the parking lot,in which the car can ocupy) My output image Can anyone guide me how to get the exact count of parking lines.I have used openCV version 2.4.I have been working on this for the past 2 days
public String getCount() {
Bitmap bitmap = BitmapFactory.decodeResource(getApplicationContext().getResources(), R.drawable.park);
mat = new Mat();
edges = new Mat();
Mat mRgba = new Mat(612, 816, CvType.CV_8UC1);
Mat lines = new Mat();
Utils.bitmapToMat(bitmap, mat);
Imgproc.Canny(mat, edges, 50, 90);
int threshold = 50;
int minLineSize = 20;
int lineGap = 20;
Imgproc.HoughLinesP(edges, lines, 1, Math.PI / 180, threshold, minLineSize, lineGap);
int count = lines.cols();
int coun= lines.rows();
System.out.println("count = " + count);
System.out.println("coun = " + coun);
String cou = String.valueOf(count);
for (int x = 0; x < lines.cols(); x++) {
double[] vec = lines.get(0, x);
double x1 = vec[0],
y1 = vec[1],
x2 = vec[2],
y2 = vec[3];
Point start = new Point(x1, y1);
Point end = new Point(x2, y2);
Core.line(mRgba, start, end, new Scalar(255, 0, 0), 3);
}
Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mRgba, bmp);
bitmap = bmp;
Drawable d = new BitmapDrawable(Resources.getSystem(), bitmap);
img2.setImageDrawable(d);
return cou;
}
You should modify one of the many answers regarding counting by OpenCV where kind of detection varies for every distinct case. You should make your own model for parking lines. Check some of these approaches/detectors Haar cascade classifier, latent SVM or Bag of Words.
You could also modify some of the answers that works for something else like this answer below for coins, only you should search for shape of parking lines instead of coins:
http://answers.opencv.org/question/36111/how-to-count-number-of-coins-in-android-opencv/

How to fit small cubes into a given volume and represent it graphically on a web page?

My query is to show programmatically, the fitting of many given non regular (but rectangular) cubes (i.e. boxes) of individually different sizes, inside a larger volume cube, such as a storage unit.
The mathematics part is understood. Like in Linear programming / linear algebra, we can add fit volume of all smaller cubes to find out the best fit for the volume of the larger cube.
The actual requirement is to show or allow this fitting graphically on a web-page, preferably in 3d. If possible, to allow user to interact with the fitting, i.e. shuffling the placement of the cubes, etc.
Also, since I am a Java developer by profession, Java or related languages / frameworks would be my choice. However, I can use any other technology / framework / language if the end results are met.
NB: Weight is also a concern (parameter). There is a maximum weight which can be stacked in any given storage unit.
Also, since storage units can be accessed without permission (by thieves), cost of the cubes stacked in one unit is also limited. The user may desire to fit cubes of higher cost in one unit which has higher security and vice versa.
Example: allow fitting many rectangular boxes containing household electronics in a given room. The boxes maybe of TVs, refrigerators, washing machines, dishwashers, playstations, xbox 360s, etc. The different dimensions of these boxes, is to give you an idea of what to expect while fitting to the limited volume.
If there is any FOSS library / project (or even non FOSS library or project) for the same, a pointer towards it would be welcome.
Disclaimer: Okay, I know it does not 100% answer your question and also the code it veeery old (as can be concluded from the old-fashioned CVS comments) and today I would not write it that way anymore. It does still run on Java 8, though, I tested it. But in addition to solving the little informatics challenge problem of water flowing through a 3D matrix of cuboids from top to bottom depending how "leaky" the matrix (symbolising some kind of Swiss cheese) is, it also uses some very simple 3D visualisation via Java 3D. Thus, you need to install Java 3D and put the corresponding libraries onto your classpath.
The 3D output looks something like this:
package vhs.bwinfo.cheese;
// $Id: Cuboid.java,v 1.1.2.1 2006/01/10 19:48:41 Robin Exp $
import javax.media.j3d.Appearance;
import javax.media.j3d.QuadArray;
import javax.media.j3d.Shape3D;
import javax.vecmath.Point3f;
import javax.vecmath.TexCoord2f;
import javax.vecmath.Vector3f;
public class Cuboid extends Shape3D {
private static final float POS = +0.5f;
private static final float NEG = -0.5f;
private static final Point3f[] POINTS = new Point3f[] {
new Point3f(NEG, NEG, NEG),
new Point3f(POS, NEG, NEG),
new Point3f(POS, NEG, POS),
new Point3f(NEG, NEG, POS),
new Point3f(NEG, POS, NEG),
new Point3f(POS, POS, NEG),
new Point3f(POS, POS, POS),
new Point3f(NEG, POS, POS)
};
private static final TexCoord2f[] TEX_COORDS = new TexCoord2f[] {
new TexCoord2f(0, 1),
new TexCoord2f(1, 1),
new TexCoord2f(1, 0),
new TexCoord2f(0, 0)
};
private static final int VERTEX_FORMAT =
QuadArray.COORDINATES |
QuadArray.NORMALS |
QuadArray.TEXTURE_COORDINATE_2;
public Cuboid(float scaleX, float scaleY, float scaleZ) {
Point3f[] points = new Point3f[8];
for (int i = 0; i < 8; i++)
points[i] = new Point3f(
POINTS[i].x * scaleX,
POINTS[i].y * scaleY,
POINTS[i].z * scaleZ
);
Point3f[] vertices = {
points[3], points[2], points[1], points[0], // bottom
points[4], points[5], points[6], points[7], // top
points[7], points[3], points[0], points[4], // left
points[6], points[5], points[1], points[2], // right
points[7], points[6], points[2], points[3], // front
points[5], points[4], points[0], points[1] // back
};
QuadArray geometry = new QuadArray(24, VERTEX_FORMAT);
geometry.setCoordinates(0, vertices);
for (int i = 0; i < 24; i++)
geometry.setTextureCoordinate(0, i, TEX_COORDS[i % 4]);
Vector3f normal = new Vector3f();
Vector3f v1 = new Vector3f();
Vector3f v2 = new Vector3f();
Point3f[] pts = new Point3f[4];
for (int i = 0; i < 4; i++)
pts[i] = new Point3f();
for (int face = 0; face < 6; face++) {
geometry.getCoordinates(face * 4, pts);
v1.sub(pts[0], pts[2]);
v2.sub(pts[1], pts[3]);
normal.cross(v1, v2);
normal.normalize();
for (int i = 0; i < 4; i++)
geometry.setNormal((face * 4 + i), normal);
}
setGeometry(geometry);
setAppearance(new Appearance());
}
public Cuboid(float scaleFactor) {
this(scaleFactor, scaleFactor, scaleFactor);
}
}
package vhs.bwinfo.cheese;
// $Id: LeakyCheese.java,v 1.2.2.2 2006/01/10 15:37:14 Robin Exp $
import com.sun.j3d.utils.applet.JMainFrame;
import javax.swing.*;
import java.util.Random;
import static java.lang.System.out;
public class LeakyCheese {
private int width = 20, height = 20, depth = 20;
private int numClasses = 100, samplesPerClass = 100;
private double pMin = 0, pMax = 1;
private double pDiff = pMax - pMin;
private double classSize = pDiff / numClasses;
private int[] stats;
enum CubeState {CHEESE, AIR, WATER}
final private CubeState[][][] cheese;
private static final Random RND = new Random();
public LeakyCheese(
int width, int height, int depth,
int numClasses, int samplesPerClass,
double pMin, double pMax
) {
this.width = width;
this.height = height;
this.depth = depth;
this.numClasses = numClasses;
this.samplesPerClass = samplesPerClass;
this.pMin = pMin;
this.pMax = pMax;
pDiff = pMax - pMin;
classSize = pDiff / numClasses;
cheese = new CubeState[width][height][depth];
}
public LeakyCheese(
int width, int height, int depth,
int numClasses, int samplesPerClass
) {
this(width, height, depth, numClasses, samplesPerClass, 0, 1);
}
public LeakyCheese() {
cheese = new CubeState[width][height][depth];
}
private boolean pourWater(int x, int y, int z) {
if (x < 0 || x >= width || y < 0 || y >= height || z < 0 || z >= depth)
return false;
if (cheese[x][y][z] != CubeState.AIR)
return false;
cheese[x][y][z] = CubeState.WATER;
boolean retVal = (y == 0);
retVal = pourWater(x + 1, y, z) || retVal;
retVal = pourWater(x - 1, y, z) || retVal;
retVal = pourWater(x, y + 1, z) || retVal;
retVal = pourWater(x, y - 1, z) || retVal;
retVal = pourWater(x, y, z + 1) || retVal;
retVal = pourWater(x, y, z - 1) || retVal;
return retVal;
}
private boolean isLeaky(double p) {
for (int x = 0; x < width; x++)
for (int y = 0; y < height; y++)
for (int z = 0; z < depth; z++)
cheese[x][y][z] = (RND.nextDouble() < p)
? CubeState.CHEESE
: CubeState.AIR;
boolean retVal = false;
for (int x = 0; x < width; x++)
for (int z = 0; z < depth; z++)
retVal = pourWater(x, height - 1, z) || retVal;
return retVal;
}
private void generateStats() {
if (stats != null)
return;
stats = new int[numClasses];
for (int i = 0; i < numClasses; i++) {
for (int j = 0; j < samplesPerClass; j++) {
double p = pMin + classSize * (RND.nextDouble() + i);
if (isLeaky(p))
stats[i]++;
}
}
}
public void printStats() {
generateStats();
out.println(
"p (cheese) | p (leaky)\n" +
"------------------+-----------"
);
for (int i = 0; i < numClasses; i++) {
out.println(
String.format(
"%1.5f..%1.5f | %1.5f",
pMin + classSize * i,
pMin + classSize * (i + 1),
(double) stats[i] / samplesPerClass
)
);
}
}
public static void main(String[] args) {
//new LeakyCheese().printStats();
//new LeakyCheese(40, 40, 40, 50, 100, 0.66, .71).printStats();
LeakyCheese cheeseBlock = new LeakyCheese(5, 20, 5, 20, 100);
//LeakyCheese cheeseBlock = new LeakyCheese(20, 20, 20, 20, 100);
while (!cheeseBlock.isLeaky(0.65))
;
out.println("*** required solution found - now rendering... ***");
JMainFrame f = new JMainFrame(new LeakyCheeseGUI(cheeseBlock.cheese), 512, 512);
f.setLocationRelativeTo(null);
f.setExtendedState(JFrame.MAXIMIZED_BOTH);
}
}
package vhs.bwinfo.cheese;
// $Id: LeakyCheeseGUI.java,v 1.1.2.1 2006/01/10 15:25:18 Robin Exp $
import com.sun.j3d.utils.applet.MainFrame;
import com.sun.j3d.utils.universe.SimpleUniverse;
import vhs.bwinfo.cheese.LeakyCheese.CubeState;
import javax.media.j3d.*;
import javax.vecmath.Point3d;
import javax.vecmath.Vector3f;
import java.applet.Applet;
import java.awt.*;
import java.util.Random;
public class LeakyCheeseGUI extends Applet {
static final long serialVersionUID = -8194627556699837928L;
public BranchGroup createSceneGraph(CubeState[][][] cheese) {
// Create the root of the branch graph
BranchGroup bgRoot = new BranchGroup();
// Composite of two rotations around different axes. The resulting
// TransformGroup is the parent of all our cheese cubes, because their
// orientation is identical. They only differ in their translation
// values and colours.
Transform3D tRotate = new Transform3D();
Transform3D tRotateTemp = new Transform3D();
tRotate.rotX(Math.PI / 8.0d);
tRotateTemp.rotY(Math.PI / -4.0d);
tRotate.mul(tRotateTemp);
TransformGroup tgRotate = new TransformGroup(tRotate);
bgRoot.addChild(tgRotate);
// Bounding sphere for rendering
BoundingSphere bounds =
new BoundingSphere(new Point3d(0.0, 0.0, 0.0), 100.0);
// Set background colour
// Note: Using Canvas3D.setBackground does not work, because it is an
// AWT method. Java 3D, though, gets its background colour from its
// background node (black, if not present).
Background background = new Background(0.5f, 0.5f, 0.5f);
background.setApplicationBounds(bounds);
bgRoot.addChild(background);
TransparencyAttributes transpAttr;
// Little cheese cubes
Appearance cheeseAppearance = new Appearance();
transpAttr =
new TransparencyAttributes(TransparencyAttributes.NICEST, 0.98f);
cheeseAppearance.setTransparencyAttributes(transpAttr);
cheeseAppearance.setColoringAttributes(
new ColoringAttributes(1, 1, 0, ColoringAttributes.NICEST));
PolygonAttributes pa = new PolygonAttributes();
//pa.setPolygonMode(PolygonAttributes.POLYGON_LINE);
pa.setCullFace(PolygonAttributes.CULL_NONE);
cheeseAppearance.setPolygonAttributes(pa);
// Little water cubes
Appearance waterAppearance = new Appearance();
transpAttr =
new TransparencyAttributes(TransparencyAttributes.NICEST, 0.85f);
waterAppearance.setTransparencyAttributes(transpAttr);
waterAppearance.setColoringAttributes(
new ColoringAttributes(0, 0, 1, ColoringAttributes.NICEST));
pa = new PolygonAttributes();
pa.setCullFace(PolygonAttributes.CULL_NONE);
waterAppearance.setPolygonAttributes(pa);
// Little air cubes
Appearance airAppearance = new Appearance();
transpAttr =
new TransparencyAttributes(TransparencyAttributes.NICEST, 0.95f);
airAppearance.setTransparencyAttributes(transpAttr);
airAppearance.setColoringAttributes(
new ColoringAttributes(1, 1, 1, ColoringAttributes.NICEST));
pa = new PolygonAttributes();
//pa.setPolygonMode(PolygonAttributes.POLYGON_LINE);
pa.setCullFace(PolygonAttributes.CULL_NONE);
airAppearance.setPolygonAttributes(pa);
// Water-coloured (i.e. blue) wire frame around cheese block, if leaky
Appearance waterWireFrameAppearance = new Appearance();
waterWireFrameAppearance.setColoringAttributes(
new ColoringAttributes(0, 0, 1, ColoringAttributes.NICEST));
pa = new PolygonAttributes();
pa.setPolygonMode(PolygonAttributes.POLYGON_LINE);
pa.setCullFace(PolygonAttributes.CULL_NONE);
waterWireFrameAppearance.setPolygonAttributes(pa);
// Cheese-coloured (i.e. yellow) wire frame around cheese block, if not leaky
Appearance cheeseWireFrameAppearance = new Appearance();
cheeseWireFrameAppearance.setColoringAttributes(
new ColoringAttributes(1, 1, 0, ColoringAttributes.NICEST));
pa = new PolygonAttributes();
pa.setPolygonMode(PolygonAttributes.POLYGON_LINE);
pa.setCullFace(PolygonAttributes.CULL_NONE);
cheeseWireFrameAppearance.setPolygonAttributes(pa);
// Absolute offsets for the cheese block to fit into the viewing canvas
final float xOffs = -0.8f;
final float yOffs = -0.55f;
final float zOffs = 0;
// Create all those little cubes ;-)
final int xSize = cheese.length;
final int ySize = cheese[0].length;
final int zSize = cheese[0][0].length;
final int maxSize = Math.max(xSize, Math.max(ySize, zSize));
final float xCenterOffs = 0.5f * (maxSize - xSize) / maxSize;
final float yCenterOffs = 0.5f * (maxSize - ySize) / maxSize;
final float zCenterOffs = -0.5f * (maxSize - zSize) / maxSize;
boolean isLeaky = false;
for (int x = 0; x < xSize; x++)
for (int y = 0; y < ySize; y++)
for (int z = 0; z < zSize; z++) {
Transform3D tTranslate = new Transform3D();
tTranslate.setTranslation(
new Vector3f(
xOffs + xCenterOffs + 1.0f * x / maxSize,
yOffs + yCenterOffs + 1.0f * y / maxSize,
zOffs + zCenterOffs - 1.0f * z / maxSize
)
);
TransformGroup tgTranslate = new TransformGroup(tTranslate);
tgRotate.addChild(tgTranslate);
Cuboid cube = new Cuboid(1.0f / maxSize);
switch (cheese[x][y][z]) {
case CHEESE:
cube.setAppearance(cheeseAppearance);
break;
case WATER:
cube.setAppearance(waterAppearance);
if (y == 0)
isLeaky = true;
break;
case AIR:
cube.setAppearance(airAppearance);
}
tgTranslate.addChild(cube);
}
// If cheese block is leaky, visualise it by drawing a water-coloured
// (i.e. blue) wire frame around it. Otherwise use a cheese-coloured
// (i.e. yellow) one.
Transform3D tTranslate = new Transform3D();
tTranslate.setTranslation(
new Vector3f(
xOffs + xCenterOffs + 0.5f * (xSize - 1) / maxSize,
yOffs + yCenterOffs + 0.5f * (ySize - 1) / maxSize,
zOffs + zCenterOffs - 0.5f * (zSize - 1) / maxSize
)
);
TransformGroup tgTranslate = new TransformGroup(tTranslate);
tgRotate.addChild(tgTranslate);
Cuboid cuboid = new Cuboid(
1.0f * xSize / maxSize,
1.0f * ySize / maxSize,
1.0f * zSize / maxSize
);
cuboid.setAppearance(isLeaky ? waterWireFrameAppearance : cheeseWireFrameAppearance);
tgTranslate.addChild(cuboid);
// Let Java 3D perform optimizations on this scene graph.
bgRoot.compile();
return bgRoot;
}
public LeakyCheeseGUI(CubeState[][][] cheese) {
// Create a simple scene and attach it to the virtual universe
GraphicsConfiguration graphCfg = SimpleUniverse.getPreferredConfiguration();
Canvas3D canvas = new Canvas3D(graphCfg);
setLayout(new BorderLayout());
add(canvas, "Center");
SimpleUniverse universe = new SimpleUniverse(canvas);
// This will move the ViewPlatform back a bit so the objects
// in the scene can be viewed.
universe.getViewingPlatform().setNominalViewingTransform();
universe.addBranchGraph(createSceneGraph(cheese));
}
public static void main(String[] args) {
final Random RND = new Random(System.currentTimeMillis());
CubeState[][][] testCheese = new CubeState[5][8][11];
for (int x = 0; x < 5; x++)
for (int y = 0; y < 8; y++)
for (int z = 0; z < 11; z++)
testCheese[x][y][z] = (RND.nextFloat() < 0.7f)
? CubeState.CHEESE
: (RND.nextBoolean() ? CubeState.WATER : CubeState.AIR);
// Applet can also run as a stand-alone application
new MainFrame(new LeakyCheeseGUI(testCheese), 512, 512);
}
}
You will probably want to use Javascript, and specifically WebGL. Javascript is the de facto language for interactive web pages, and WebGL is a Javascript API for rendering 2D and 3D scenes on an HTML5 canvas element. A solution using WebGL should be compatible with all major browsers. Programming even simple scenes in WebGL can be pretty involved though, so I'd recommend using a framework such as three.js to simplify things.
Here is an example of interactive, draggable cubes using three.js. Some of the key lines of code from that example are:
// create the cube
var object = new THREE.Mesh( geometry, new THREE.MeshLambertMaterial( { color: Math.random() * 0xffffff } ) );
// set coordinates, rotation, and scale of the cubes
object.position.x = ...
object.position.y = ...
object.position.z = ...
object.rotation.x = ...
object.rotation.y = ...
object.rotation.z = ...
object.scale.x = ...
object.scale.y = ...
object.scale.z = ...
// lighting stuff
object.castShadow = true;
object.receiveShadow = true;
// add to scene and list of objects
scene.add( object );
objects.push( object );
Again, the full, working example is found at this link (click view source on that page to view the code on github).

ESRI geometry compression algorithm in JAVA

I am writing ESRI geometry compression algorithm in JAVA, I am using this link.
Using instructions mentioned in provided link, this is my current code:
public static void main(String[] args) {
String dirPoints = "-118.356654545455,34.1146;-118.356436363636,34.1143272727273;-118.356418181818,34.1142363636364;-118.356490909091,34.1137636363636";
String compressedGeometry = "";
double xPointPrev = 0.0;
double yPointPrev = 0.0;
int coefficient = 55000;
String coefficient_32 = Integer.toString(coefficient, 32);
compressedGeometry = coefficient_32 + compressedGeometry;
String[] path_XY = dirPoints.split(";");
for (int i = 0, leni = path_XY.length; i < leni; i++) {
String[] xy = path_XY[i].split(",");
double pointX = Double.parseDouble(xy[0].trim());
double pointY = Double.parseDouble(xy[1].trim());
int xDifference = (int) Math.round(coefficient * (pointX - xPointPrev));
int yDifference = (int) Math.round(coefficient * (pointY - yPointPrev));
String xDifference_32 = Integer.toString(xDifference, 32);
compressedGeometry += xDifference_32;
String yDifference_32 = Integer.toString(yDifference, 32);
compressedGeometry += yDifference_32;
xPointPrev = pointX;
yPointPrev = pointY;
}
System.out.println(compressedGeometry);
}
Expected output: "+1lmo-66l1f+1p8af+c-f+1-5-4-q"
But I am getting this: "1lmo-66l1g1p8afc-f1-5-4-q"
What am I missing? Any help is highly appreciated.
Thanks,
According to Integer.toString()
"If the first argument is not negative, no sign character appears in the result."
Therefore I think you need to add the "+".
(And I think the f-g there is their typo).

Java: Why/what are these threads monitoring?

I have a multithreaded Java application which splits an image into 4 chunks, then 4 threads (I have a quad-core CPU) each work on an individual chunk of the image, converting it to grayscale.
I found that it was quite slow for some reason, so I used the NetBeans profiler and found that the threads were "monitoring" (waiting) quite a lot. For example,
(green = running, red = monitoring)
I experimented with different numbers of threads, e.g. 2, and found that this still happened (the only time it didn't happen was with 1 thread).
Inside the threads, I commented out bits of their code until I narrowed the "big delay" down to this statement:
newImage.setRGB(i,j,newColor.getRGB()); // Write the new value for that pixel
If this is commented out, the code runs MUCH (almost 5x) faster, and there is NO thread monitoring:
So why does that one line cause so much delay? Is it the Color library (alongside BufferedImage)? Right now I'm going to try and get an array of ints as the RGB values instead of using a Color object and see how that goes.
Here is the source code:
PixelsManipulation.java (main class):
public final class PixelsManipulation{
private static Sequential sequentialGrayscaler = new Sequential();
public static void main(String[] args) throws FileNotFoundException, IOException, InterruptedException {
File file = new File("src/pixelsmanipulation/hiresimage.jpg");
FileInputStream fis = new FileInputStream(file);
BufferedImage image = ImageIO.read(fis); //reading the image file
int rows = 2; // 2 rows and 2 cols will split the image into quarters
int cols = 2;
int chunks = rows * cols; // 4 chunks, one for each quarter of the image
int chunkWidth = image.getWidth() / cols; // determines the chunk width and height
int chunkHeight = image.getHeight() / rows;
int count = 0;
BufferedImage imgs[] = new BufferedImage[chunks]; // Array to hold image chunks
for (int x = 0; x < rows; x++) {
for (int y = 0; y < cols; y++) {
//Initialize the image array with image chunks
imgs[count] = new BufferedImage(chunkWidth, chunkHeight, image.getType());
// draws the image chunk
Graphics2D gr = imgs[count++].createGraphics(); // Actually create an image for us to use
gr.drawImage(image, 0, 0, chunkWidth, chunkHeight, chunkWidth * y, chunkHeight * x, chunkWidth * y + chunkWidth, chunkHeight * x + chunkHeight, null);
gr.dispose();
}
}
//writing mini images into image files
for (int i = 0; i < imgs.length; i++) {
ImageIO.write(imgs[i], "jpg", new File("img" + i + ".jpg"));
}
System.out.println("Mini images created");
// Start threads with their respective quarters (chunks) of the image to work on
// I have a quad-core machine, so I can only use 4 threads on my CPU
Parallel parallelGrayscaler = new Parallel("thread-1", imgs[0]);
Parallel parallelGrayscaler2 = new Parallel("thread-2", imgs[1]);
Parallel parallelGrayscaler3 = new Parallel("thread-3", imgs[2]);
Parallel parallelGrayscaler4 = new Parallel("thread-4", imgs[3]);
// Sequential:
long startTime = System.currentTimeMillis();
sequentialGrayscaler.ConvertToGrayscale(image);
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("Sequential code executed in " + elapsedTime + " ms.");
// Multithreaded (parallel):
startTime = System.currentTimeMillis();
parallelGrayscaler.start();
parallelGrayscaler2.start();
parallelGrayscaler3.start();
parallelGrayscaler4.start();
// Main waits for threads to finish so that the program doesn't "end" (i.e. stop measuring time) before the threads finish
parallelGrayscaler.join();
parallelGrayscaler2.join();
parallelGrayscaler3.join();
parallelGrayscaler4.join();
stopTime = System.currentTimeMillis();
elapsedTime = stopTime - startTime;
System.out.println("Multithreaded (parallel) code executed in " + elapsedTime + " ms.");
}
}
Parallel.java:
// Let each of the 4 threads work on a different quarter of the image
public class Parallel extends Thread{//implements Runnable{
private String threadName;
private static BufferedImage myImage; // Calling it "my" image because each thread will have its own unique quarter of the image to work on
private static int width, height; // Image params
Parallel(String name, BufferedImage image){
threadName = name;
System.out.println("Creating "+ threadName);
myImage = image;
width = myImage.getWidth();
height = myImage.getHeight();
}
public void run(){
System.out.println("Running " + threadName);
// Pixel by pixel (for our quarter of the image)
for (int j = 0; j < height; j++){
for (int i = 0; i < width; i++){
// Traversing the image and converting the RGB values (doing the same thing as the sequential code but on a smaller scale)
Color c = new Color(myImage.getRGB(i,j));
int red = (int)(c.getRed() * 0.299);
int green = (int)(c.getGreen() * 0.587);
int blue = (int)(c.getBlue() * 0.114);
Color newColor = new Color(red + green + blue, red + green + blue, red + green + blue);
myImage.setRGB(i,j,newColor.getRGB()); // Write the new value for that pixel
}
}
File output = new File("src/pixelsmanipulation/"+threadName+"grayscale.jpg"); // Put it in a "lower level" folder so we can see it in the project view
try {
ImageIO.write(newImage, "jpg", output);
} catch (IOException ex) {
Logger.getLogger(Parallel.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("Thread " + threadName + " exiting. ---");
}
}
I am a beginner at threading in Java (as well as using BufferedImage), just curious as to why it's so slow.
Why is Parallel.myImage static? This will cause all the threads to share the same image. That might explain why they are waiting on each other.

projecting Tango 3D point to screen Google Project Tango

Ptoject Tango provides a point cloud, how can you get the position in pixels of a 3D point in the point cloud in meters?
I tried using the projection matrix but I get very small values (0.5,1.3 etc) instead of say 1234,324 (in pixels).
I include the code I have tried
//Get the current rotation matrix
Matrix4 projMatrix = mRenderer.getCurrentCamera().getProjectionMatrix();
//Get all the points in the pointcloud and store them as 3D points
FloatBuffer pointsBuffer = mPointCloudManager.updateAndGetLatestPointCloudRenderBuffer().floatBuffer;
Vector3[] points3D = new Vector3[pointsBuffer.capacity()/3];
int j =0;
for (int i = 0; i < pointsBuffer.capacity() - 3; i = i + 3) {
points3D[j]= new Vector3(
pointsBuffer.get(i),
pointsBuffer.get(i+1),
pointsBuffer.get(i+2));
//Log.v("Points3d", "J: "+ j + " X: " +points3D[j].x + "\tY: "+ points3D[j].y +"\tZ: "+ points3D[j].z );
j++;
}
//Get the projection of the points in the screen.
Vector3[] points2D = new Vector3[points3D.length];
for(int i =0; i < points3D.length-1;i++)
{
Log.v("Points", "X: " +points3D[i].x + "\tY: "+ points3D[i].y +"\tZ: "+ points3D[i].z );
points2D[i] = points3D[i].multiply(projMatrix);
Log.v("Points", "pX: " +points2D[i].x + "\tpY: "+ points2D[i].y +"\tpZ: "+ points2D[i].z );
}
The example I'm using is the point cloud java which can be found here
https://github.com/googlesamples/tango-examples-java
UPDATE
TangoCameraIntrinsics ccIntrinsics = mTango.getCameraIntrinsics(TangoCameraIntrinsics.TANGO_CAMERA_COLOR);
double fx = ccIntrinsics.fx;
double fy = ccIntrinsics.fy;
double cx = ccIntrinsics.cx;
double cy = ccIntrinsics.cy;
double[][] projMatrix = new double[][] {
{fx, 0 , -cx},
{0, fy, -cy},
{0, 0, 1}
};
Then to compute the projected point I use
for(int i =0; i < points3D.length-1;i++)
{
double[][] point = new double[][] {
{points3D[i].x},
{points3D[i].y},
{points3D[i].z}
};
double [][] point2d = CustomMatrix.multiplyByMatrix(projMatrix, point);
points2D[i] = new Vector2(0,0);
if(point2d[2][0]!=0)
{
Log.v("temp point", "pX: " +point2d[0][0]/point2d[2][0]+" pY: " +point2d[1][0]/point2d[2][0] );
points2D[i] = new Vector2(point2d[0][0]/point2d[2][0],point2d[1][0]/point2d[2][0]);
}
}
But I think that the results are still not what is expected, I for instance get results like:
pX: -175.58042313027244 pY: -92.573740812066
Which to me looks not right.
UPDATE
Using color camera as suggested gives better results, but poitns are still negative
-1127.8086915171814 pY: -652.5887102192332
Would it be ok to just multiply them by -1?
You have to multiply 3D point with RGB camera's intrinsics matrix to obtain pixel coordinate. 3D points are in Depthcamera's frame. You get pixel coordinates by following method:
and
x and y are pixel coordinates.
And K is constructed with parameters using intrinsics function

Categories

Resources