Related
I have to write a program in Java that uses StdAudio and Picture to create a two-dimensional color visualization of a sound file while it is playing but I'm not really sure how to.
Can someone tell me everything that I need or tell me what I need to do to "convert" the sound file so that it's readable by Picture?
I could grab the samples from the sound file and return them as array of doubles, but then how would that even create an image? How could those values even sync with the image?
I have been playing around in eclipse just trying to figure out how this could possibly even work but my code just ends up being a whole mess.
private final static int SAMPLE_RATE = 44100;
private static int WIDTH = 500;
private static int HEIGHT = 100;
private static JFrame frame;
private static Picture pic;
public static void main(String[] args) throws IOException
{
pic = new Picture(WIDTH, HEIGHT); // <- blank black image
String audioFile = "SampleTest2.wav";
double[] audio = StdAudio.read(audioFile);
frame = new JFrame();
frame.setContentPane(pic.getJLabel());
frame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
frame.setTitle("Sound Visualization");
frame.setResizable(false);
frame.pack();
frame.setVisible(true);
for (int k = 0; k < audio.length; k++)
StdAudio.play(audio[k]);
for (int i = 0; i < pic.width(); i ++)
{
for (int j = 0; j < pic.height(); j++)
{
pic.set(i, j, toColor(audio[SAMPLE_RATE + i]));
//frame.setContentPane(pic.getJLabel());
}
frame.repaint();
}
}
private static Color toColor(double colVal)
{
int r = (int) (((colVal + 1) / 2) * 255);
int g = (int) (((colVal + 1) / 2) * 255);
int b = (int) (((colVal + 1) / 2) * 255);
return new Color(r, g, b);
}
To use StdAudio you need wav file with sampling rate of 44100. It means every second of this sound consists of 44100 values(samples). When you load such file with duration of 1 second using method double[] read(String filename) you will obtain an array with 44100 elements. Javadoc of that method tells us the values will be between -1.0 and +1.0. We can iterate over every sample, map values from -1..1 range to 0..255 range (because colors need values from 0 to 255) and paint each pixel with this color. For better effect let's not paint single pixel but a column of 100 pixels.
I'll create image of 500x100. It will display only 500 samples so it will represent 500/44100 = only 0,01 of a second. To create empty picture of that size use:
Picture p = new Picture(500, 100);
To paint separate pixels along the image use:
for (int i = 0; i < 500; i++) {
p.set(i, 0, color);
}
and to display this picture use:
p.show();
Next, to create a color we need 3 values: red, green and blue components. Here we have only one value so the resulting image will be a greyscale image because saturation of every component will be the same value new Color(value, value, value). To quickly convert a range from -1..1 to 0..255 use such formula: (int) (((d + 1) / 2) * 255)
I used the first sound file from this site:
http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/index-e.html and the image I obtained is:
The code I used is:
import java.awt.Color;
import java.io.IOException;
public class StackOverflow58899141 {
private static int IMAGE_WIDTH = 500;
private static int IMAGE_HEIGHT = 100;
static String filename = "O:\\1.wav";
public static void main(final String[] args) throws IOException {
// reading sound file to samples
double[] samples = StdAudio.read(filename);
// creating empty image
Picture p = new Picture(IMAGE_WIDTH, IMAGE_HEIGHT);
// filling image from left to right
for (int i = 0; i < IMAGE_WIDTH; i++) {
// filling image from top to bottom
for (int j = 0; j < IMAGE_HEIGHT; j++) {
// adding 44100 to skip 1s of silence at the beginning
p.set(i, j, doubleToColor(samples[44100 + i]));
}
}
p.show();
}
// convert number from range -1.0..1.0 to 0..255
private static Color doubleToColor(double d) {
int val = (int) (((d + 1) / 2) * 255);
return new Color(val, val, val);
}
}
Now you have a solid start to understand how it works. Although Picture class allows easy saving of an image it doesn't allow animating. To achieve that you'd need to create own JFrame and draw image and delay drawing each column of pixels to get the animation effect.
So I can load a 2D map using tiles using a text file, which is great and all. However, one issue I have met with this method is that I can't add objects/actors to my map since the file is a 2D grid. (The game is similar to games like zelda and pokemon.) I've tried creating an object layer so I can overlap images, but it doesn't seem to work for me. To give an example of what I want, have objects such as trees to be solid and on top of the background grass.
I am also looking for better methods to creating these tile based maps if you want to pitch some ideas to me.
**Note: I am about beginner/intermediate at Java.
Here is my constructor for the GameState class that calls the Map.
public GameState(Game game) {
super(game);
player = new Player(game, 0, 0, 64, 64);
map = new Map(game, "res/saves/save1.txt");
}
Here is the Map class (which works) that also calls the object (2nd) layer.
private int width, height;
public static int spawnX, spawnY;
private int[][] mapTiles;
MapObjects mapObjects;
Game game;
public Map(Game game, String path) {
this.game = game;
mapObjects = new MapObjects(game, "res/saves/save1_obj.txt", width, height);
loadMap(path);
}
private void loadMap(String path) {
String file = Utils.loadFileAsString(path);
//Token is which number it is out of the total
String[] tokens = file.split("\\s+");
//Sets what is what
width = Utils.parseInt(tokens[0]);
height = Utils.parseInt(tokens[1]);
spawnX = Utils.parseInt(tokens[2]);
spawnY = Utils.parseInt(tokens[3]);
mapTiles = new int[width][height];
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
// (x+y*width) : calculates the nth token (+4) : The 4 prior tokens before the graph
mapTiles[x][y] = Utils.parseInt(tokens[(x + y *width) + 4]);
}
}
}
public void render(Graphics g) {
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
//Only renders what is seen.
getMapTile(x, y).render(g, (int)(x*Tile.TILE_WIDTH-game.getCamera().getxOffset()), (int)(y*Tile.TILE_HEIGHT-game.getCamera().getyOffset()));
}
}
}
public void tick() {
}
//Gets the specific tile at specific coordinates.
private Tile getMapTile(int x, int y) {
Tile t = Tile.tiles[mapTiles[x][y]];
if(t == null) {
return Tile.grassTile;
}
return t;
}
And lastly, the object layer that doesn't work. It does not give an error, just the overlapping objects aren't visible. I've made sure to load the object layer before the Map layer, but that doesn't seem to be the issue.
private int width, height;
private int[][] objTiles;
Game game;
public MapObjects(Game game, String path, int width, int height) {
this.game = game;
loadObjects(path, width, height);
}
public void loadObjects(String path, int width, int height) {
this.width = width;
this.height = height;
String file = Utils.loadFileAsString(path);
String[] tokens = file.split("\\s+");
objTiles = new int[width][height];
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
objTiles[x][y] = Utils.parseInt(tokens[(x + y *width)]);
}
}
}
public void render(Graphics g) {
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
//Only renders what is seen.
getObjTile(x, y).render(g, (int)(x*Tile.TILE_WIDTH-game.getCamera().getxOffset()), (int)(y*Tile.TILE_HEIGHT-game.getCamera().getyOffset()));
}
}
}
public void tick() {
}
//Gets the specific object tile at specific coordinates.
private Tile getObjTile(int x, int y) {
Tile t = Tile.tiles[objTiles[x][y]];
if(t == null) {
return Tile.nothingTile;
}
return t;
}
We may need a bit more info from you.
Do you use a different "container/component" to draw map tiles than you do for your objects? Because if you render objects first then they will disappear as soon as you render the map. You should draw the map first, and then do objects like so:
public Map(Game game, String path) {
this.game = game;
//swapped the order of the lines below so the map loads first:
loadMap(path);
mapObjects = new MapObjects(game, "res/saves/save1_obj.txt", width, height);
}
From what you have said then this does not work either, however if you use the same component to draw your map and objects then one will always override the other, and something will always be missing. To fix this you need to crease two separate panes, one for the map, and a transparent one that sits on top of the map that you can use to render your objects.
See this illustration as an example:
You basically need to add a new transparent "content plane" similar to the way that glass pane shown above.
I am using code similar to Java - get pixel array from image to get low-level access to pixel data of a BMP image, along the lines of:
BufferedImage image = ImageIO.read(is);
DataBuffer buffer = image.getRaster().getDataBuffer();
byte[] rawPixels = ((DataBufferByte) buffer).getData();
The resulting array is laid bottom to top (ie. its first bytes are the beginning of the last image line), which makes sense considering that BMP files usually have the same layout.
I would like to hide this low-level detail from callers by flipping the lines in this situation. Is there a way I can query the pixels orientation/layout of the loaded BufferedImage?
I have checked the source code of the Java 7 BMPImageReader, and it does translate from bottom-up to top-down order while reading, as I expected it to do. The DataBuffers backing array will thus be in the normal top-down order. I cannot reproduce this behavior using Oracle Java 7 JRE on Windows.
The OP has verified that the problem was indeed in another part of the code, not posted as part of the question.
I think what is described just might be possible, using a special subclass of SampleModel that translates all incoming y-coordinates, but there's no standard method to query for orientation (all Rasters are assumed to be top-down).
Anyway, just for fun, I created some code, to test if it is at all possible. Below is a fully runnable example.
public class SampleModelOrientationTest {
public static void main(String[] args) {
BufferedImage image = new BufferedImage(16, 9, BufferedImage.TYPE_3BYTE_BGR);
WritableRaster raster = image.getRaster();
DataBuffer dataBuffer = raster.getDataBuffer();
SampleModel sampleModel = image.getSampleModel();
QueryingDataBuffer queryBuffer = new QueryingDataBuffer(dataBuffer, sampleModel.getWidth(), sampleModel.getNumDataElements());
sampleModel.getDataElements(0, 0, null, queryBuffer);
System.out.println(queryBuffer.getOrientation());
queryBuffer.resetOrientation();
SampleModel bottomUpSampleModel = new BottomUpSampleModel(sampleModel);
bottomUpSampleModel.getDataElements(0, 0, null, queryBuffer);
System.out.println(queryBuffer.getOrientation());
}
private static class QueryingDataBuffer extends DataBuffer {
enum Orientation {
Undefined,
TopDown,
BottomUp,
Unsupported
}
private final int width;
private final int numDataElements;
private Orientation orientation = Orientation.Undefined;
public QueryingDataBuffer(final DataBuffer dataBuffer, final int width, final int numDataElements) {
super(dataBuffer.getDataType(), dataBuffer.getSize());
this.width = width;
this.numDataElements = numDataElements;
}
#Override public int getElem(final int bank, final int i) {
if (bank == 0 && i < numDataElements && isOrientationUndefinedOrEqualTo(Orientation.TopDown)) {
orientation = Orientation.TopDown;
}
else if (bank == 0 && i >= (size - (width * numDataElements) - numDataElements) && isOrientationUndefinedOrEqualTo(Orientation.BottomUp)) {
orientation = Orientation.BottomUp;
}
else {
// TODO: Expand with more options as apropriate
orientation = Orientation.Unsupported;
}
return 0;
}
private boolean isOrientationUndefinedOrEqualTo(final Orientation orientation) {
return this.orientation == Orientation.Undefined || this.orientation == orientation;
}
#Override public void setElem(final int bank, final int i, final int val) {
}
public final void resetOrientation() {
orientation = Orientation.Undefined;
}
public final Orientation getOrientation() {
return orientation;
}
}
// TODO: This has to be generalized to be used for any BufferedImage type.
// I justy happen to know that 3BYTE_BGR uses PixelInterleavedSampleModel and has BGR order.
private static class BottomUpSampleModel extends PixelInterleavedSampleModel {
public BottomUpSampleModel(final SampleModel sampleModel) {
super(sampleModel.getDataType(), sampleModel.getWidth(), sampleModel.getHeight(),
sampleModel.getNumDataElements(), sampleModel.getNumDataElements() * sampleModel.getWidth(),
new int[] {2, 1, 0} // B, G, R
);
}
#Override public Object getDataElements(final int x, final int y, final Object obj, final DataBuffer data) {
return super.getDataElements(x, getHeight() - 1 - y, obj, data);
}
#Override public int getSample(final int x, final int y, final int b, final DataBuffer data) {
return super.getSample(x, getHeight() - 1 - y, b, data);
}
}
}
Backstory:
I'm trying to draw as many squares the the screen as possible using a single draw call. I'm using a custom glsl vertex shader that is specialized for 2D drawing, and that is supposed to be pulling position data for the vertices of the squares from a samplerBuffer. Since I don't need to worry about rotating or scaling the squares all I should need to do is load the position data into a buffer, bind a texture to that buffer, and then use the sampler to get each vertex's position in the shader. In order to get an index into the texture I store each elements index as the z-component of the vertices.
Everything seems to work really well for a thousand or so squares, but after that I start to get weird blinking. It sort of seems like it's not drawing all of the squares every draw step, or possibly not using all of the positions so that many of the squares are overlapping.
The weird thing is, that if I use drawElements instead of drawElementsMulti, the blinking goes away (but of course then all the squares are drawn as one single object, which I don't want)
One question I have is if my position data is limited to the max texture size, or the max texture buffer size. And if I am limited to the much smaller max texture size, how do I get around it? There's got to be a reason all of that texture buffer space is there, but I obviously don't get how to properly use it.
I'm also thinking maybe glMultiDrawElements is doing something I'm not accounting for with the sampler somehow. Idk, I'm really lost at this point, and yet..it works perfectly for smaller numbers of squares, so I must be doing something right.
[EDIT] Code had changed to reflect suggestions below (and for readability), but the problem persists.
Ok, so here's some code. First the vertex shader:
uniform mat3 projection;
attribute vec3 vertex;
uniform samplerBuffer positionSampler;
attribute vec4 vertex_color;
varying vec4 color;
float positionFetch(int index)
{
// I've tried texelFetch here as well, same effect
float value = texelFetchBuffer(positionSampler, index).r;
return value;
}
void main(void)
{
color = vec4(1, 1, 1, 1);
// use the z-component of the vertex to look up the position of this instance in the texture
vec3 real_position = vec3(vertex.x + positionFetch(int(vertex.z)*2), vertex.y + positionFetch(int(vertex.z)*2+1), 1);
gl_Position = vec4(projection * real_position, 1);
}
And now my GLRenderer, sorry there is so much code, I just really want to make sure there's enough info here to get an answer. This has really been driving me nuts, and examples for java seem to be hard to come by (maybe this code will help someone else on their quest):
public class GLRenderer extends GLCanvas implements GLEventListener, WindowListener
{
private static final long serialVersionUID = -8513201172428486833L;
private static final int bytesPerFloat = Float.SIZE / Byte.SIZE;
private static final int bytesPerShort = Short.SIZE / Byte.SIZE;
public float viewWidth, viewHeight;
public float screenWidth, screenHeight;
private FPSAnimator animator;
private boolean didInit = false;
JFrame the_frame;
SquareGeometry geometry;
// Thought power of 2 might be required, doesn't seem to make a difference
private static final int NUM_THINGS = 2*2*2*2*2*2*2*2*2*2*2*2*2*2;
float[] position = new float[NUM_THINGS*2];
// Shader attributes
private int shaderProgram, projectionAttribute, vertexAttribute, positionAttribute;
public static void main(String[] args)
{
new GLRenderer();
}
public GLRenderer()
{
// setup OpenGL Version 2
super(new GLCapabilities(GLProfile.get(GLProfile.GL2)));
addGLEventListener(this);
setSize(1800, 1000);
the_frame = new JFrame("Hello World");
the_frame.getContentPane().add(this);
the_frame.setSize(the_frame.getContentPane().getPreferredSize());
the_frame.setVisible(true);
the_frame.addWindowListener(this);
animator = new FPSAnimator(this, 60);
animator.start();
}
// Called by the drivers when the gl context is first made available
public void init(GLAutoDrawable d)
{
final GL2 gl = d.getGL().getGL2();
IntBuffer asd = IntBuffer.allocate(1);
gl.glGetIntegerv(GL2.GL_MAX_TEXTURE_BUFFER_SIZE, asd);
System.out.println(asd.get(0));
asd = IntBuffer.allocate(1);
gl.glGetIntegerv(GL2.GL_MAX_TEXTURE_SIZE, asd);
System.out.println(asd.get(0));
shaderProgram = ShaderLoader.compileProgram(gl, "default");
gl.glLinkProgram(shaderProgram);
_getShaderAttributes(gl);
gl.glUseProgram(shaderProgram);
_checkGLCapabilities(gl);
_initGLSettings(gl);
// Calculate batch of vertex data from dirt geometry
geometry = new SquareGeometry(.1f);
geometry.buildGeometry(viewWidth, viewHeight);
geometry.finalizeGeometry(NUM_THINGS);
geometry.vertexBufferID = _generateBufferID(gl);
_loadVertexBuffer(gl, geometry);
geometry.indexBufferID = _generateBufferID(gl);
_loadIndexBuffer(gl, geometry);
geometry.positionBufferID = _generateBufferID(gl);
// initialize buffer object
int size = NUM_THINGS * 2 * bytesPerFloat;
System.out.println(size);
IntBuffer bla = IntBuffer.allocate(1);
gl.glGenTextures(1, bla);
geometry.positionTextureID = bla.get(0);
gl.glUniform1i(positionAttribute, 0);
gl.glActiveTexture(GL2.GL_TEXTURE0);
gl.glBindTexture(GL2.GL_TEXTURE_BUFFER, geometry.positionTextureID);
gl.glBindBuffer(GL2.GL_TEXTURE_BUFFER, geometry.positionBufferID);
gl.glBufferData(GL2.GL_TEXTURE_BUFFER, size, null, GL2.GL_DYNAMIC_DRAW);
gl.glTexBuffer(GL2.GL_TEXTURE_BUFFER, GL2.GL_R32F, geometry.positionBufferID);
}
private void _initGLSettings(GL2 gl)
{
gl.glClearColor(0f, 0f, 0f, 1f);
}
private void _loadIndexBuffer(GL2 gl, SquareGeometry geometry)
{
gl.glBindBuffer(GL2.GL_ELEMENT_ARRAY_BUFFER, geometry.indexBufferID);
gl.glBufferData(GL2.GL_ELEMENT_ARRAY_BUFFER, bytesPerShort*NUM_THINGS*geometry.getNumPoints(), geometry.indexBuffer, GL2.GL_STATIC_DRAW);
}
private void _loadVertexBuffer(GL2 gl, SquareGeometry geometry)
{
int numBytes = geometry.getNumPoints() * 3 * bytesPerFloat * NUM_THINGS;
gl.glBindBuffer(GL2.GL_ARRAY_BUFFER, geometry.vertexBufferID);
gl.glBufferData(GL2.GL_ARRAY_BUFFER, numBytes, geometry.vertexBuffer, GL2.GL_STATIC_DRAW);
gl.glEnableVertexAttribArray(vertexAttribute);
gl.glVertexAttribPointer(vertexAttribute, 3, GL2.GL_FLOAT, false, 0, 0);
}
private int _generateBufferID(GL2 gl)
{
IntBuffer bufferIDBuffer = IntBuffer.allocate(1);
gl.glGenBuffers(1, bufferIDBuffer);
return bufferIDBuffer.get(0);
}
private void _checkGLCapabilities(GL2 gl)
{
// TODO: Respond to this information in a meaningful way.
boolean VBOsupported = gl.isFunctionAvailable("glGenBuffersARB") && gl.isFunctionAvailable("glBindBufferARB")
&& gl.isFunctionAvailable("glBufferDataARB") && gl.isFunctionAvailable("glDeleteBuffersARB");
System.out.println("VBO Supported: " + VBOsupported);
}
private void _getShaderAttributes(GL2 gl)
{
vertexAttribute = gl.glGetAttribLocation(shaderProgram, "vertex");
projectionAttribute = gl.glGetUniformLocation(shaderProgram, "projection");
positionAttribute = gl.glGetUniformLocation(shaderProgram, "positionSampler");
}
// Called by me on the first resize call, useful for things that can't be initialized until the screen size is known
public void viewInit(GL2 gl)
{
for(int i = 0; i < NUM_THINGS; i++)
{
position[i*2] = (float) (Math.random()*viewWidth);
position[i*2+1] = (float) (Math.random()*viewHeight);
}
gl.glUniformMatrix3fv(projectionAttribute, 1, false, Matrix.projection3f, 0);
// Load position data into a texture buffer
gl.glBindBuffer(GL2.GL_TEXTURE_BUFFER, geometry.positionBufferID);
ByteBuffer textureBuffer = gl.glMapBuffer(GL2.GL_TEXTURE_BUFFER, GL2.GL_WRITE_ONLY);
FloatBuffer textureFloatBuffer = textureBuffer.order(ByteOrder.nativeOrder()).asFloatBuffer();
for(int i = 0; i < position.length; i++)
{
textureFloatBuffer.put(position[i]);
}
gl.glUnmapBuffer(GL2.GL_TEXTURE_BUFFER);
gl.glBindBuffer(GL2.GL_TEXTURE_BUFFER, 0);
}
public void display(GLAutoDrawable d)
{
if (!didInit || geometry.vertexBufferID == 0)
{
return;
}
//long startDrawTime = System.currentTimeMillis();
final GL2 gl = d.getGL().getGL2();
gl.glClear(GL2.GL_COLOR_BUFFER_BIT | GL2.GL_DEPTH_BUFFER_BIT);
// If we were drawing any other buffers here we'd need to set this every time
// but instead we just leave them bound after initialization, saves a little render time
// No combination of these seems to fix the problem
//gl.glBindBuffer(GL2.GL_ARRAY_BUFFER, geometry.vertexBufferID);
//gl.glVertexAttribPointer(vertexAttribute, 3, GL2.GL_FLOAT, false, 0, 0);
//gl.glBindBuffer(GL2.GL_ELEMENT_ARRAY_BUFFER, geometry.indexBufferID);
gl.glBindBuffer(GL2.GL_TEXTURE_BUFFER, geometry.positionBufferID);
//gl.glActiveTexture(GL2.GL_TEXTURE0);
//gl.glTexBuffer(GL2.GL_TEXTURE_BUFFER, GL2.GL_R32F, geometry.positionBufferID);
_render(gl, geometry);
// Also tried these
//gl.glFlush();
//gl.glFinish();
}
public void _render(GL2 gl, SquareGeometry geometry)
{
gl.glMultiDrawElements(geometry.drawMode, geometry.countBuffer, GL2.GL_UNSIGNED_SHORT, geometry.offsetBuffer, NUM_THINGS);
// This one works, but isn't what I want
//gl.glDrawElements(GL2.GL_LINE_LOOP, count, GL2.GL_UNSIGNED_SHORT, 0);
}
public void reshape(GLAutoDrawable d, int x, int y, int width, int height)
{
final GL2 gl = d.getGL().getGL2();
gl.glViewport(0, 0, width, height);
float ratio = (float) height / width;
screenWidth = width;
screenHeight = height;
viewWidth = 100;
viewHeight = viewWidth * ratio;
Matrix.ortho3f(0, viewWidth, 0, viewHeight);
if (!didInit)
{
viewInit(gl);
didInit = true;
}
else
{
// respond to view size changing
}
}
}
The final bit is the SquareGeometry class which holds all the bufferIDs and vertex data, but also is responsible for filling the vertex buffer correctly so that each vertex's z component can function as an index into the position texture:
public class SquareGeometry
{
public float[] vertices = null;
ShortBuffer indexBuffer;
IntBuffer countBuffer;
PointerBuffer offsetBuffer;
FloatBuffer vertexBuffer;
public int vertexBufferID = 0;
public int indexBufferID = 0;
public int positionBufferID = 0;
public int positionTextureID = 0;
public int drawMode;
protected float width = 0;
protected float height = 0;
public SquareGeometry(float size)
{
width = size;
height = size;
}
public void buildGeometry(float viewWidth, float viewHeight)
{
vertices = new float[4 * 2];
vertices[0] = -width/2;
vertices[1] = -height/2;
vertices[2] = -width/2;
vertices[3] = height/2;
vertices[4] = width/2;
vertices[5] = height/2;
vertices[6] = width/2;
vertices[7] = -height/2;
drawMode = GL2.GL_POLYGON;
}
public void finalizeGeometry(int numInstances)
{
if(vertices == null) return;
int num_vertices = this.getNumPoints();
int total_num_vertices = numInstances * num_vertices;
// initialize vertex Buffer (# of coordinate values * 4 bytes per float)
ByteBuffer vbb = ByteBuffer.allocateDirect(total_num_vertices * 3 * Float.SIZE);
vbb.order(ByteOrder.nativeOrder());
vertexBuffer = vbb.asFloatBuffer();
for(int i = 0; i < numInstances; i++)
{
for(int v = 0; v < num_vertices; v++)
{
int vertex_index = v * 2;
vertexBuffer.put(vertices[vertex_index]);
vertexBuffer.put(vertices[vertex_index+1]);
vertexBuffer.put(i);
}
}
vertexBuffer.rewind();
// Create the indices
vbb = ByteBuffer.allocateDirect(total_num_vertices * Short.SIZE);
vbb.order(ByteOrder.nativeOrder());
indexBuffer = vbb.asShortBuffer();
for(int i = 0; i < total_num_vertices; i++)
{
indexBuffer.put((short) (i));
}
indexBuffer.rewind();
// Create the counts
vbb = ByteBuffer.allocateDirect(numInstances * Integer.SIZE);
vbb.order(ByteOrder.nativeOrder());
countBuffer = vbb.asIntBuffer();
for(int i = 0; i < numInstances; i++)
{
countBuffer.put(num_vertices);
}
countBuffer.rewind();
// create the offsets
offsetBuffer = PointerBuffer.allocateDirect(numInstances);
for(int i = 0; i < numInstances; i++)
{
offsetBuffer.put(num_vertices*i*2);
}
offsetBuffer.rewind();
}
public int getNumPoints()
{
return vertices.length/2;
}
}
Ok first things first, you are not setting gl_Color in the shader maybe that can be the issue here and you only lucky with small numbers. It is a varying, but do you also have fragment shader that picks up the value?
At no point do you ensure that NUM_THINGS*2 < GL_MAX_TEXTURE_SIZE. I don't know how FloatBuffer.put reacts; being Java probably / hopefully an exception.
Also you bind the positionBufferID buffer, then unbind it but never rebind it.
You create positionTextureID but never put any data there. This also what you put into the sampler positionSampler and try to access.
Yea well lots of issues but my gut tells me the last one may be the real issue here.
Alright, I've got it solved, though I'm still really not clear on what the original problem was. I fixed it by simplifying the drawing to use drawArrays instead of drawElements or multiDrawElements. I'm really not sure why I thought I needed them, as I really don't in this case. I'm pretty sure I was messing up a few things with the indexes and offsets.
Furthermore, as far as the proper way to bind the texture buffer, neither the code I have above, nor example found at the link I posted in a comment are correct at all.
If anyone is interested in the correct way to use the texture buffer like this I just did a pretty extensive write-up on it here http://zebadiah.me/?p=44. Thanks all for the help.
in order to implement some image analysis algorithms without having to worry too much on the data type (i.e. without having too much duplicate code), I'm setting up the visitor pattern for primitive arrays in Java.
In the example below, I've defined two types of visitors
a primitive type, where the signature of the visit method is visit(int, int double)
a generic type, where the signature of the visit method is visit(int, int Double).
Appart from this, both visitors do exactly the same operations. My idea was to try and measure the cost of boxing/unboxing.
So here is the full program
public class VisitorsBenchmark {
public interface Array2DGenericVisitor<TYPE, RET> {
void begin(int width, int height);
RET end();
void visit(int x, int y, TYPE value);
}
public interface Array2DPrimitiveVisitor<RET> {
void begin(final int width, final int height);
RET end();
void visit(final int x, final int y, final double value);
}
public static <RET>
RET
accept(final int width,
final int height,
final double[] data,
final Array2DGenericVisitor<Double, RET> visitor) {
final int size = width * height;
visitor.begin(width, height);
for (int i = 0, x = 0, y = 0; i < size; i++) {
visitor.visit(x, y, data[i]);
x++;
if (x == width) {
x = 0;
y++;
if (y == height) {
y = 0;
}
}
}
return visitor.end();
}
public static <RET> RET accept(final int width,
final int height,
final double[] data,
final Array2DPrimitiveVisitor<RET> visitor) {
final int size = width * height;
visitor.begin(width, height);
for (int i = 0, x = 0, y = 0; i < size; i++) {
visitor.visit(x, y, data[i]);
x++;
if (x == width) {
x = 0;
y++;
if (y == height) {
y = 0;
}
}
}
return visitor.end();
}
private static final Array2DGenericVisitor<Double, double[]> generic;
private static final Array2DPrimitiveVisitor<double[]> primitive;
static {
generic = new Array2DGenericVisitor<Double, double[]>() {
private double[] sum;
#Override
public void begin(final int width, final int height) {
final int length = (int) Math.ceil(Math.hypot(WIDTH, HEIGHT));
sum = new double[length];
}
#Override
public void visit(final int x, final int y, final Double value) {
final int r = (int) Math.round(Math.sqrt(x * x + y * y));
sum[r] += value;
}
#Override
public double[] end() {
return sum;
}
};
primitive = new Array2DPrimitiveVisitor<double[]>() {
private double[] sum;
#Override
public void begin(final int width, final int height) {
final int length = (int) Math.ceil(Math.hypot(WIDTH, HEIGHT));
sum = new double[length];
}
#Override
public void visit(final int x, final int y, final double value) {
final int r = (int) Math.round(Math.sqrt(x * x + y * y));
sum[r] += value;
}
#Override
public double[] end() {
return sum;
}
};
}
private static final int WIDTH = 300;
private static final int HEIGHT = 300;
private static final int NUM_ITERATIONS_PREHEATING = 10000;
private static final int NUM_ITERATIONS_BENCHMARKING = 10000;
public static void main(String[] args) {
final double[] data = new double[WIDTH * HEIGHT];
for (int i = 0; i < data.length; i++) {
data[i] = Math.random();
}
/*
* Pre-heating.
*/
for (int i = 0; i < NUM_ITERATIONS_PREHEATING; i++) {
accept(WIDTH, HEIGHT, data, generic);
}
for (int i = 0; i < NUM_ITERATIONS_PREHEATING; i++) {
accept(WIDTH, HEIGHT, data, primitive);
}
/*
* Benchmarking proper.
*/
double[] sumPrimitive = null;
double[] sumGeneric = null;
double aux = System.nanoTime();
for (int i = 0; i < NUM_ITERATIONS_BENCHMARKING; i++) {
sumGeneric = accept(WIDTH, HEIGHT, data, generic);
}
final double timeGeneric = System.nanoTime() - aux;
aux = System.nanoTime();
for (int i = 0; i < NUM_ITERATIONS_BENCHMARKING; i++) {
sumPrimitive = accept(WIDTH, HEIGHT, data, primitive);
}
final double timePrimitive = System.nanoTime() - aux;
System.out.println("prim = " + timePrimitive);
System.out.println("generic = " + timeGeneric);
System.out.println("generic / primitive = "
+ (timeGeneric / timePrimitive));
}
}
I know that the JIT is pretty clever, so I was not too surprised when both visitors turned out to perform equally well.
What is more surprising, is that the generic visitor seems to perform slightly faster than the primitive, which is unexpected. I know benchmarking can sometimes be difficult, so I must have done something wrong. Can you spot the error?
Thanks a lot for your help!!!
Sébastien
[EDIT] I've updated the code to account for a pre-heating phase (in order to let the JIT compiler do its work). This does not change the results, which are consistently below 1 (0.95 - 0.98).
I know benchmarking can sometimes be difficult, so I must have done something wrong. Can you spot the error?
I think that the problem is that your benchmarking does not take account of JVM warmup. Put the take the body of your main method and put it into another method. Then have your main method call that new method repeatedly in a loop. Finally, examine the results, and discard the first few that are distorted by JIT compilation and other warmup effects.
Small tips:
Do not use Math.random() to perform benchmarks as the results are non-deterministic. You need smth like new Random(xxx).
Always print the result of the operation. Mixing benchmark types in a single execution is bad practice as it can lead to different call site optimizations (not your case, though)
double aux = System.nanoTime(); -- not all longs fit into doubles - properly.
post the specification of the environment and the hardware you perform the benchmarks on
print 'staring test' while enabled printing the compilation -XX:-PrintCompilation and the garbage collection -verbosegc -XX:+PrintGCDetails - the GC can kick in during the 'wrong' test just enough to skew the results.
Edit:
I did check the generated assembler and none of them is the real reason. There is no allocation for Double.valueOf() as the method is inlined altogether and optimized away - it uses only the CPU registers. However w/o the hardware spec/JVM there is no real answer.
I found a JVM (1.6.0.26) where the generic version (Double) has better loop unroll(!), due to deeper analysis (obviously needed to EA the Double.valueOf()) and possibly constant folding of WIDTH/HEIGHT. Change the WIDTH/HEIGHT to some prime numbers and the results should differ.
The bottom line is: do not use microbenchmarks unless you know how the JVM optimizes and check the generated machine code.
Disclaimer: I am no JVM engineer
This is a totally "wild assed guess" but I think it has to do with copying bytes onto the stack. Passing a primitive double involves copying 8 bytes on the stack. Passing a Double only takes copying the pointer.