Develop Reference

how do you draw a line in a pixel array

I like to have maximum control over the screen, so I have to control every pixel, and that has some pros and cons. one con is that I don't really have the help from any built-in functions. so I have no idea how to draw a line.
I've tried to make a function to handle line drawing but I just can't get it to work!
here's the code I used to draw the line
int startX;
int startY;
int deltaX = x1/x2;
int deltaY = y1/y2;
float deltaPixl = deltaX/deltaY;
for(int i=0;i<deltaY;i=i+1){
if(x1>x2){ startX = x2;}else{ startX=x1;}
if(y1>y2){ startY = y2;}else{ startY=y1;}
pixl(startX+i,round(startY+(deltaPixl*i)),0);
}
it uses a function called pixl so that it easily draw a pixel to the pixel array,
just to clarify why there's a function called pixl in the code.
and when I try to use this code, it doesn't crash, like processing usually does when it has an error!
it just doesn't work, instead just doing nothing!
I'd like some help on this subject, please.

You could get away with simply using PGraphics.
The idea is once you have a PGraphics instance you use dot notation to access the drawing functions used to (as long as they're called between .beginDraw() and .endDraw()).
Using noSmooth() you can get it looking pixel perfect.
Here's a basic sketch to illustrate the idea:
// disable anti-aliasing
noSmooth();
// create a PGraphics layer
PGraphics layer = createGraphics(25, 25);
// render a line
layer.beginDraw();
layer.line(0, 24, 24, 0);
layer.endDraw();
// render the line at 100%
image(layer, 0, 0);
// render the line scaled up
image(layer, 0, 0, width, height);
This should do for most cases. (It's only trickier cases with very small values and transparency that might give you headaches)
If for some reason you need a lot more control, you can you always implement your own method of rasterising. Once place you can probably start with is Bresenham's line algorithm
Regarding your code there are a few things that could go wrong:
float deltaPixl = deltaX/deltaY;: if deltaY is zero you'll run into an exception
you're doing integer division for deltaX and deltaY (potentially making it likely to get 0 for either of the values)
you should try a println() statement before the for loop with the start/end values to get a feel if that loop will actually execute or not. Additionally, within the for loop you can println(i) to see if you get the value you expect.
Overall I recommend checking Kevin Workman's How to Debug guide.
Additionally you could use lerp() to calculate linearly interpolated position between the line's start and end points. Pass each coordinate and a normalized (between 0.0, 1.0) value, where 0.0 = at the start point, 1.0 = at the end point and anything in between is on the line (e.g. 0.5 = 50% along the line).
Here's a basic example:
void drawLinePoints(int x1, int y1, int x2, int y2, int numberOfPoints){
// for each point
for(int i = 0; i < numberOfPoints; i++){
// map the counter to a normalized (0.0 to 1.0) value for lerp
// 0.0 = 0 % along the line, 0.5 = 50% along the line, 1.0 = 100% along the line
float t = map(i, 0, numberOfPoints, 0.0, 1.0);
// linearly interpolate between the start / end points (and snap to whole pixels (casting to integer type))
int x = (int)lerp(x1, x2, t);
int y = (int)lerp(y1, y2, t);
// render the point
point(x, y);
}
}
void setup(){
// render points are large squares
strokeWeight(6);
strokeCap(PROJECT);
}
void draw(){
// clear frame
background(255);
// calculate distance
float distance = dist(10, 10, mouseX, mouseY);
// map distance the number of points to illustrate interpolation (more points = continuous line)
int numPoints = (int)distance / 8;
// render points along the line
drawLinePoints(10, 10, mouseX, mouseY, numPoints);
}
For the sake of completeness here's the above snippet using the pixels[] instead:
void drawLinePoints(int x1, int y1, int x2, int y2, int numberOfPoints){
// for each point
for(int i = 0; i < numberOfPoints; i++){
// map the counter to a normalized (0.0 to 1.0) value for lerp
// 0.0 = 0 % along the line, 0.5 = 50% along the line, 1.0 = 100% along the line
float t = map(i, 0, numberOfPoints, 0.0, 1.0);
// linearly interpolate between the start / end points (and snap to whole pixels (casting to integer type))
int x = (int)lerp(x1, x2, t);
int y = (int)lerp(y1, y2, t);
// convert the x, y coordinate to pixels array index and render the point in black
pixels[x + (y * width)] = color(0);
}
}
void setup(){
noSmooth();
}
void draw(){
// clear frame
loadPixels();
java.util.Arrays.fill(pixels, color(255));
// calculate distance
float distance = dist(10, 10, mouseX, mouseY);
// map distance the number of points to illustrate interpolation (more points = continuous line)
int numPoints = (int)distance;
// render points along the line
drawLinePoints(10, 10, mouseX, mouseY, numPoints);
// update pixels
updatePixels();
}

I'm a bit late but I found a very simple method for line drawing to a pixel array on this website.
Here is a simple implementation I made in Monogame (btw sorry its not using processing - I have never used it):
public void drawLine(int x1, int y1, int x2, int y2)
{
//this will store the colour data of the canvas pixels
Color[] canvasData = new Color[canvas.Width * canvas.Height];
//store the pixel data of the canvas in canvasData
canvas.GetData<Color>(canvasData);
//drawing line starts here
int dx = x2 - x1;
int dy = y2 - y1;
for (int x = x1; x < x2; x++)
{
int y = y1 + dy * (x - x1) / dx;
//[y*canvas.Width+x] converts the 2d array index to a 1d array index
canvasData[y * canvas.Width + x] = Color.Black;
}
//line drawing ended
//setting the canvas' pixels to the modified pixels with the line
canvas.SetData<Color>(canvasData);
}

Android semantic segmentation post-processing is too slow

I'd really appreciate it if anyone can advise with a task I've been working without success for the last week.
I have semantic segmentation model (MobileNetV3 + Lightweight ASPP).Short info: input - 1024x1024, output - same size and 2 classes (bg and vehicle), so my output shape is (1, 1048576, 2). I'm not the mobile dev or java world guy, so I used a few complete andoid examples for image segmentation to test it:
the one from google: https://github.com/tensorflow/examples/tree/master/lite/examples/image_segmentation
and another one open-sourced: https://github.com/pillarpond/image-segmenter-android
I successfully converted it to tflite format and its inference time on OnePlus 7 with GPU enabled and 10 threads is between 105-140ms for such size. But here I run into a problem: general execution time in these two android examples or any you can find for semantic segmentation is about 1050-1300ms (which is less than 1FPS). The slower part of this pipeline is image post-processing (~900-1150ms). You can see that part in the Deeplab#segment method. Since I have only 1 class besides bg - I don't have this third loop, but everything else is untouched and still very slow. Output size is not small in comparison to other common mobile sizes like 128/226/512, but still. I think it shouldn't take so much time to process 1024x1024 matrix and draw rectangles in canvas on modern smartphones.
I tried different solutions, like splitting matrix manipulations into threads or creating all these objects like RectF and Recognition once before and just filling their attributes with new data inside nested loops, but I didn't succeed on either of them. On the desktop side I easily handle it with numpy and opencv and I don't even close to understanding how can I do the same in Android and will it even be efficient or not.
Here's code which I use in python:
CLASS_COLORS = [(0, 0, 0), (255, 255, 255)] # black for bg and white for mask
def get_image_array(image_input, width, height):
img = cv2.imread(image_input, 1)
img = cv2.resize(img, (width, height))
img = img.astype(np.float32)
img[:, :, 0] -= 128.0
img[:, :, 1] -= 128.0
img[:, :, 2] -= 128.0
img = img[:, :, ::-1]
return img
def get_segmentation_array(seg_arr, n_classes):
output_height = seg_arr.shape[0]
output_width = seg_arr.shape[1]
seg_img = np.zeros((output_height, output_width, 3))
for c in range(n_classes):
seg_arr_c = seg_arr[:, :] == c
seg_img[:, :, 0] += ((seg_arr_c)*(CLASS_COLORS[c][0])).astype('uint8')
seg_img[:, :, 1] += ((seg_arr_c)*(CLASS_COLORS[c][1])).astype('uint8')
seg_img[:, :, 2] += ((seg_arr_c)*(CLASS_COLORS[c][2])).astype('uint8')
return seg_img
interpreter = tf.lite.Interpreter(model_path=f"my_model.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
img_arr = get_image_array("input.png", 1024, 1024)
interpreter.set_tensor(input_details[0]['index'], np.array([x]))
interpreter.invoke()
output = interpreter.get_tensor(output_details[0]['index'])
output = output.reshape((1024, 1024, 2)).argmax(axis=2)
seg_img = get_segmentation_array(output, 2)
cv2.imwrite("output.png", seg_img)
Maybe there's anything powerful than the current solution for post-processing.
I would really appreciate any help with this. I'm sure there's anything that can improve post-processing and reduce its time to ~100ms, so I will have ~5FPS in general.

New Update. Thanks to Farmaker, I used a piece of code found in his repo from comment above and now pipeline looks like:
int channels = 3;
int n_classes = 2;
int float_byte_size = 4;
int width = model.inputWidth;
int height = model.inputHeight;
int[] intValues = new int[width * height];
ByteBuffer inputBuffer = ByteBuffer.allocateDirect(width * height * channels * float_byte_size).order(ByteOrder.nativeOrder());
ByteBuffer outputBuffer = ByteBuffer.allocateDirect(width * height * n_classes * float_byte_size).order(ByteOrder.nativeOrder());
Bitmap input = textureView.getBitmap(width, height);
input.getPixels(intValues, 0, width, 0, 0, height, height);
inputBuffer.rewind();
outputBuffer.rewind();
for (final int value: intValues) {
inputBuffer.putFloat(((value >> 16 & 0xff) - 128.0) / 1.0f);
inputBuffer.putFloat(((value >> 8 & 0xff) - 128.0) / 1.0f);
inputBuffer.putFloat(((value & 0xff) - 128.0) / 1.0f);
}
tfLite.run(inputBuffer, outputBuffer);
final Bitmap output = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
outputBuffer.flip();
int[] pixels = new int[width * height];
for (int i = 0; i < width * height; i++) {
float max = outputBuffer.getFloat();
float val = outputBuffer.getFloat();
int id = val > max ? 1 : 0;
pixels[i] = id == 0 ? 0x00000000 : 0x990000ff;
}
output.setPixels(pixels, 0, width, 0, 0, width, height);
resultView.setImageBitmap(resizeBitmap(output, resultView.getWidth(), resultView.getHeight()));
public static Bitmap resizeBitmap(Bitmap bm, int newWidth, int newHeight) {
int width = bm.getWidth();
int height = bm.getHeight();
float scaleWidth = ((float) newWidth) / width;
float scaleHeight = ((float) newHeight) / height;
// CREATE A MATRIX FOR THE MANIPULATION
Matrix matrix = new Matrix();
// RESIZE THE BIT MAP
matrix.postScale(scaleWidth, scaleHeight);
// "RECREATE" THE NEW BITMAP
Bitmap resizedBitmap = Bitmap.createBitmap(
bm, 0, 0, width, height, matrix, false);
bm.recycle();
return resizedBitmap;
}
Right now post-processing time is ~70-130ms, 95th is around 90ms, which alongside ~60ms of image pre-processing time, ~140ms inference time and around 30-40ms for other stuff with enabled GPU and 10 threads gives me general execution time around 330ms which is 3FPS! And this is for a large model for 1024x1024.
At this point, I'm more than satisfied and want to try different configurations for my model, including MobilenetV3 small as a backbone.

How to rotate the LinearGradient in a given Shape?

I try to find the way to rotate the LinearGradient object nested into e.g. Rectangle object, say:
Rectangle rect = new Rectangle(0, 0, 200, 200);
LinearGradient lg = new LinearGradient(0, 0, 100, 0, false, CycleMethod.REPEAT, new Stop[] {
new Stop(0, Color.BLACK);
new Stop(0.5, Color.WHITE);
new Stop(1, Color.BLACK);
});
rect.setFill(lg);
Now, I try to rotate this lg object, for example for 45 degrees to the left, but without rotating the whole rect. Is there any way to achieve that?

The first parameters that are given to the LinearGradient constructor are the coordinates of the start- and end point of the gradient axis, respectively. This means that you can achieve a "rotated" gradient simply by passing in an appropriately rotated axis.
In the simplest form, for the example that you described, you can use the following pattern:
double angleInRadians = Math.toRadians(45);
double length = 100;
double endX = Math.cos(angleInRadians) * length;
double endY = Math.sin(angleInRadians) * length;
LinearGradient lg = new LinearGradient(0, 0, endX, endY, ...);
This will result in a gradient rotated by 45 degrees.
The fixed values here will affect the final appearance of the gradient, together with the other parameters. Referring to your example, this gradient with the same "wave length" as before (namely 100), and start with the same color at the upper left corner (i.e. Color.BLACK will be at coordinates (0,0)).

Trig ratios can be used for a more flexible gradient angle. Please note: It does not implement repeat, hence add more stops in the gradient object.
private fun createGradient(width: Float, height: Float): LinearGradient {
val mode = TileMode.CLAMP
val angleInRadians = Math.toRadians(mAngle.toDouble())
val halfWidth = width / 2
val halfHeight = height / 2
val sinAngle = sin(angleInRadians)
val cosAngle = cos(angleInRadians)
val x0 = (halfWidth * (1 + sinAngle)).toFloat()
val y0 = (halfHeight * (1 - cosAngle)).toFloat()
val x1 = (halfWidth * (1 - sinAngle)).toFloat()
val y1 = (halfHeight * (1 + cosAngle)).toFloat()
return LinearGradient(x0, y0, x1, y1, mGradient, null, mode)
}

libgdx rectangle around actor

How does one draw a Rectangle around an Actor with a given Texture? The Rectangle should scale, move and rotate along with the Actor.
Current implementation:
shapeRenderer.begin(ShapeType.Line);
shapeRenderer.setColor(Color.RED);
Vector2 _BL = localToStageCoordinates(new Vector2(getX(), getY()));
Vector2 _BR = localToStageCoordinates(new Vector2(getX() + getWidth(), getY()));
Vector2 _TL = localToStageCoordinates(new Vector2(getX(), getY() + getHeight()));
Vector2 _TR = localToStageCoordinates(new Vector2(getX() + getWidth(), getY() + getHeight()));
float xmin = MyMathUtils.min(_BL.x, _BR.x, _TL.x, _TR.x);
float ymin = MyMathUtils.min(_BL.y, _BR.y, _TL.y, _TR.y);
float xmax = MyMathUtils.max(_BL.x, _BR.x, _TL.x, _TR.x);
float ymax = MyMathUtils.max(_BL.y, _BR.y, _TL.y, _TR.y);
shapeRenderer.rect(xmin, ymin, xmax-xmin, ymax-ymin);
shapeRenderer.setColor(Color.BLUE);
shapeRenderer.circle(_BL.x, _BL.y, 2);
shapeRenderer.circle(_BR.x, _BR.y, 2);
shapeRenderer.circle(_TL.x, _TL.y, 2);
shapeRenderer.circle(_TR.x, _TR.y, 2);
shapeRenderer.end();
The problem is that Rectangle doesn't fit well after moving the Actor. I think it has to do with a bad Coordinate Transformation.
I would like to do something like this:
Matix4 m = actor.computeTransform();
shapeRenderer.setTransformMatrix(m);
but the computeTransform() is available in Group (which is an Actor), but not in Actor itself.

Why are my VBOs slower than display lists?

I created two simple voxel engines, literally just chunks that hold cubes. For the first one, I use display lists and can render hundreds of chunks at 60 FPS no problem, despite the fact that the technology behind it is years old and deprecated by now. With my VBO version, I try to render 27 chunks and I suddenly drop to less than 50 FPS. What gives? I use shaders for my VBO version, but not for display list one. Without shaders for the VBO version, I still get the same FPS rate. I'll post some relevant code:
VBO
Initialization of chunk:
public void initGL() {
rand = new Random();
sizeX = (int) pos.getX() + CHUNKSIZE;
sizeY = (int) pos.getY() + CHUNKSIZE;
sizeZ = (int) pos.getZ() + CHUNKSIZE;
tiles = new byte[sizeX][sizeY][sizeZ];
vCoords = BufferUtils.createFloatBuffer(CHUNKSIZE * CHUNKSIZE * CHUNKSIZE * (3 * 4 * 6));
cCoords = BufferUtils.createFloatBuffer(CHUNKSIZE * CHUNKSIZE * CHUNKSIZE * (4 * 4 * 6));
createChunk();
verticeCount = CHUNKSIZE * CHUNKSIZE * CHUNKSIZE * (4 * 4 * 6);
vCoords.flip();
cCoords.flip();
vID = glGenBuffers();
glBindBuffer(GL_ARRAY_BUFFER, vID);
glBufferData(GL_ARRAY_BUFFER, vCoords, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, 0);
cID = glGenBuffers();
glBindBuffer(GL_ARRAY_BUFFER, cID);
glBufferData(GL_ARRAY_BUFFER, cCoords, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, 0);
}
private void createChunk() {
for (int x = (int) pos.getX(); x < sizeX; x++) {
for (int y = (int) pos.getY(); y < sizeY; y++) {
for (int z = (int) pos.getZ(); z < sizeZ; z++) {
if (rand.nextBoolean() == true) {
tiles[x][y][z] = Tile.Grass.getId();
} else {
tiles[x][y][z] = Tile.Void.getId();
}
vCoords.put(Shape.createCubeVertices(x, y, z, 1));
cCoords.put(Shape.getCubeColors(tiles[x][y][z]));
}
}
}
}
And then rendering:
public void render() {
glBindBuffer(GL_ARRAY_BUFFER, vID);
glVertexPointer(3, GL_FLOAT, 0, 0L);
glBindBuffer(GL_ARRAY_BUFFER, cID);
glColorPointer(4, GL_FLOAT, 0, 0L);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_COLOR_ARRAY);
shader.use();
glDrawArrays(GL_QUADS, 0, verticeCount);
shader.release();
glDisableClientState(GL_COLOR_ARRAY);
glDisableClientState(GL_VERTEX_ARRAY);
}
I know I use quads, and that's bad, but I'm also using quads for my display list engine. The shaders are very simple, all they do is take a color and apply it to the vertices, I won't even post them they are that simple.
Display List
Initialization:
public void init() {
rand = new Random();
opaqueID = glGenLists(1);
tiles = new byte[(int) lPosition.x][(int) lPosition.y][(int) lPosition.z];
genRandomWorld();
rebuild();
}
public void rebuild() {
glNewList(opaqueID, GL_COMPILE);
glBegin(GL_QUADS);
for (int x = (int) sPosition.x; x < (int) lPosition.x; x++) {
for (int y = (int) sPosition.y; y < (int) lPosition.y; y++) {
for (int z = (int) sPosition.z; z < (int) lPosition.z; z++) {
if (checkCubeHidden(x, y, z)) {
// check if tiles hidden. if not, add vertices to
// display list
if (type != 0) {
Tile.getTile(tiles[x][y][z]).getVertices(x, y, z, 1, spritesheet.getTextureCoordsX(tiles[x][y][z]), spritesheet.getTextureCoordsY(tiles[x][y][z]));
} else {
Tile.getTile(tiles[x][y][z]).getVertices(x, y, z, 1);
}
}
}
}
}
glEnd();
glEndList();
spritesheet.bind();
}
I should note that in my display list version, I only add in the visible cubes. So, that may be an unfair advantage, but it should not bring the VBO version down to that FPS with just 27 chunks versus 500 chunks for the display list version.
I render like this:
public void render() {
if (tiles.length != -1) {
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glCallList(opaqueID);
}
}
So, after all of that code, I really still wonder why my VBO version is just so darn slow? I do have a one dimensional list of chunks in my display list version for when I'm calling them to render, and a 3 dimensional one in my VBO version, but I think the JVM pretty much eliminates any lag with the extra dimensions. So, what am I doing wrong?

It is hard to answer such question without having an actual project and a profiler at hand, so these are theories:
You don't show your Display Lists generation code in detail, so I'm assuming you are doing something alike glColor(); glVertex3f(); in a loop (not that you declared color once and done with it).
Display List implementation is implementation-specific, but usually that is interleaved array of vertex properties, because that is much more friendly to a cache (all vertice props are tightly aligned by 16bytes instead of being spread by a size of array). On the other hand, VBO you use is coming in two non-interleaved chunks - Coordinates and Colors. This could cause excessive unfriendly cache usage (especially with big amounts of data).
As noted in comments:
try interleaving your position and colour data in a single buffer. That is the usual recommendation for static data as it gives better memory access patterns during rendering. – GuyRT`