I want to start a deferred shading project with GLSL , Java & openGl
1. How does a deferred rendering pipeline works, does it render the scene for each image?
For example when I want to create a specular, blur and shadow texture, do I need to render the scene for each of these textures.
I've seen some code snippets and there where no multiple render loops.
2. What is a Geometry buffer and what does it do? Is it something like a storage for scene data that I can draw to a texture without rendering again?
To add something more specific so you might get started. You need FBOs with multiple attachments and a way for your shader to write to multiple FBO attachments. Google glDrawBuffers. Your FBO attachments also needs to be textures so the information can be passed to a shader. The FBO attachments should be the same size as the screen you are rendering to. There are many ways to approach this. Here is one example.
You need two FBOs
Geometry Buffer
1. Diffuse (GL_RGBA)
2. Normal Buffer (GL_RGB16F)
3. Position Buffer (GL_RGB32F)
4. Depth Buffer
Note that 3) is a huge waste since we can use the the depth buffer and the projection to reconstruct the position. This is a lot cheaper. Having the position buffer to begin with is a good start at least. Attack one problem at a time.
The 2) normal buffer can also be compressed more.
Light Accumulation Buffer
1. Light Buffer (GL_RGBA)
2. Depth Buffer
The depth buffer attachment in this FBO should be the same attachment as in the geometry buffer. We might not use this depth buffer information in this example, but you will need it sooner or later. It will always contain the depth information from the first stage.
How do we render this stuff?
We start by rendering our scene with very simple shaders. The purpose of these are mainly to fill the geometry buffer. We simply draw all our geometry with a very simple shader filling up the geometry buffer. For simplicity I use 120 shaders and no texture mapping (all though that is super trivial to add).
Vertex Shader :
#version 120
varying vec3 normal;
varying vec4 position;
void main( void )
{
normal = normalize(gl_NormalMatrix * gl_Normal);
position = gl_ModelViewMatrix * gl_Vertex;
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}
Fragment Shader :
#version 120
uniform vec4 objectColor; // Color of the object you are drawing
varying vec3 normal;
varying vec4 position;
void main( void )
{
// Use glDrawBuffers to configure multiple render targets
gl_FragData[0] = objectColor; // Diffuse
gl_FragData[1] = vec4(normalize(normals.xyz), 0.0); // normals
gl_FragData[2] = vec4(position.xyz, 0.0); // Position
}
We have now for example drawn 20 objects to out geometry buffer with different color. If we look at the diffuse buffer, it's a pretty dull image with plain colors (or plain textures without lighting), but we still have the view position, normal and depth of each single fragment. This will be valuable information for is in the next stage when doing the lighting.
Light Accumulation
Now we switch to our light accumulation buffer and it is time to do some light magic. For each single light we are going to draw to our light accumulation buffer with additive blending enabled. How you do this is not that important for the result as long as you cover all the fragments affected by the light. You can do this initially by drawing a full screen quad, but that is very costly. We will only cover point lights, but this more than sufficient to cover the simple lighting principle (simple point lights are extremely trivial to make). A simple way is to draw a cube or a low poly sphere (light volume) at the light position scaled by the light radius. This makes rendering tons of small lights way more efficient.. but don't worry about performance now. Fullscreen quad will do the trick just fine.
Now, the simple principle is :
Each fragment has a stored x,y,z position we simply get with a texture fetch
We pass in the position of the light
We pass in the radius of the light
We can know if the fragment is affected by the light simply by measuring the distance from the value in the position buffer and the light position
From there on it's pretty standard light calculations
Fragment shader :
(This shader works for anything. Light volumes, full screen quads.. whatever)
#version 120
uniform sampler2D diffuseBuffer;
uniform sampler2D positionBuffer;
uniform sampler2D normalBuffer;
uniform float lightRadius; // Radius of our point light
uniform vec3 lightPos; // Position of our point light
uniform vec4 lightColor; // Color of our light
uniform vec2 screensize; // screen resolution
void main()
{
// VU for the current fragment
vec2 uv = vec2(gl_FragCoord.x / screensize.x, gl_FragCoord.y / screensize.y);
// Read data from our gbuffer (sent in as textures)
vec4 diffuse_g = texture2D(diffuseBuffer, uv);
vec4 position_g = texture2D(positionBuffer, uv);
vec4 gnormal_g = texture2D(normalBuffer, uv);
// Distance from the light center and the current pixel
float distance = length(lightPos - position_g.xyz);
// If the fragment is NOT affecter by the light we discard it!
// PS : Don't kill me for using discard. This is for simplicity.
if(distance > lightRadius) discard;
// Calculate the intensity value this light will affect the fragment (Standard light stuff!)
... Use lightPos and position_g to calculate the light normal ..
... Do standard dot product of light normal and normal_g ...
... Just standard light stuff ...
// Super simple attenuation placeholder
float attenuation = 1.0 - (distance / lightRadius);
gl_FragColor = diffuse_g * lightColor * attenuation * <multiplier from light calculation>;
}
We repeat this for each light. The order the lights are rendered doesn't matter since the result will always be the same with additive blending. You can also do it much simpler by accumulating only light intensity. In theory you should already have the final lit result in the light accumulation buffer, but you might want to to additional adjustments.
Combine
You might want to adjust a few things. Ambient? Color correction? Fog? Other post processing stuff. You can combine the light accumulation buffer and the diffuse buffer with some adjustments. We kind of already did that in the light stage, but if you only saved light intensity, you will have to do a simple diffuse * light combine here.
Normally just a full screen quad that renders the final result to the screen.
More Stuff
As mentioned earlier we want to get rid of the position buffer. Use the depth buffer with your projection to reconstruct the position.
You don't need to use light volumes. Some prefer to simply render a quad large enough to cover the area on the screen.
The example above do not cover issues like how to define unique materials for each object. There are many resources and variants of gbuffer formats out there. Some prefer to save a material index in the alpha channel (in the diffuse buffer), then lookup up a row in a texture to get material properties.
Directional lights and other light types affecting the entire scene can easily be handled by rendering a full screen quad into the light accumulation buffer
Spot lights are also nice to have and also fairly easy to implement
We probably want more light properties
We might want some way to weight how the diffuse and light buffer is combined to support ambient and emissive
There are many ways to store normals in a more compact way. You can for example use spherical coordinates to remove one value. There are tons of articles about deferred lighting and gbuffer formats out there. Looking at the formats people are using can give you some ideas. Just make sure your gbuffer don't get too fat.
Reconstructing the view position using the linearized depth value and your projection is not that hard. You need to construct a vector using the projection constants. Multiply it with your depth value (between 0 and 1) to get the view position. There are several articles out there. It's just two lines of code.
There's probably a lot to pick in in this post, but hopefully it shows the general principle. None one the shaders have been compiled. This was just converted from 3.3 to 1.2 by memory.
There several approaches to light accumulation. You might want to reduce the number of draw calls making VBOs with 1000 cubes and cones to batch-draw everything. With more modern GL versions you can also use the geometry shader to calculate a quad that would cover the light area for each light. Probably the best way is to use compute shaders, but that requires GL 4.3. The advantage here is that you can iterate all the light information and do one single write. There are also pseudo-compute methods were you divide the screen into a rough grid and assign a light list to each cell. This can be done only with a fragment shader, but requires you to build the light lists on the CPU and sending in the data to the shader though UBOs.
The compute shader method is by far the simplest one to make. It removes a lot of the complexity in the older methods to keep track and organize everything. Simply iterate the lights and do one single write to the framebuffer.
1) Deferred shading involves separating rendering the geometry for a scene and basically everything else into separate passes.
For example when I want to create a specular, blur and shadow texture, do I need to render the scene for each of these textures.
For the shadow texture, probably (if you're using shadow mapping, this can't be avoided). But for everything else:
No, which is why deferred shading can be so useful. In a deferred pipeline you render the geometry once and save the color, normal, and 3d location (The Geometry Buffer) for each pixel. This can be achieved in a couple different ways, but the most common is to use Frame Buffer Objects (FBOs) with multiple render targets (MRTs). When using FBOs for deferred shading you render the geomotry in exactly the same way you would render normally, except that you bind the FBO, use multiple outputs in your fragment shader (one for each render target), and don't calculate any lighting. You can read more about FBOs and MRTs on the OpenGL website or through a quick google search. Then to light your scene you would read this data in a shader and use it to compute lighting just like you would normally. The easiest way to do this (but not the best way) is to render a full screen quad and sample color, normal, and location textures for your scene.
2) The geometry buffer is all of the data necessary for lighting and other shading that will be done on the scene. It is created during the geometry pass (the only time when geometry needs to be rendered) and is typically a set of textures. Each texture is used as a render target (See above about FBOs and MRTs) when rendering the geometry. You typically have one texture for color, one for normals, and one for 3d location. It can also contain more data (like parameters for lighting) if necessary. That gives you all the data you need for each pixel to be lit during a lighting pass.
Pseudo code could look like this:
for all geometry {
render to FBO
}
for all lights {
read FBO and do lighting
}
//... here you can read the FBO and use it for anything!
The basic idea of deferred rendering is to separate the process of transforming the geometry of the meshes into locations on the target framebuffer, and giving the pixels of the target framebuffer their final colour.
The first step is to render the geometry in a way, that each pixel of the framebuffer receives information about the original geometry, i.e. location in either world or eye space (eye space is preferred), the transformed tangent space (normal, tangent, binormal), and other attributes depending on what's later required. This is the "geometry buffer" (answering your 2. question as well).
With the geometry buffer at hand, the precomputed geometry→pixel mapping can be reused for several, similar processing steps. For example if you want to render 50 light sources, you have to process only the geometry 50 times (which equals to rendering 100 triangles, which is childs play for a modern GPU), where for each iteration other parameters are used (light position, direction, shadow buffers, etc.). This is in contrast to regular multipass rendering, where for each iteration the whole geometry needs to be reprocessed.
And of course each pass may be used to render a different kind of shading process (glow, blur, bokeh, halos, etc.)
Then for each iteration pass the results are merged together into a composite image.
Related
I'm implementing Warcraft/Age of Empires-style "Fog of War" by writing a Filter class and the appropriate JME material definition with vertex and fragment shaders.
I was able to figure that out very easily, I can now tint the entire screen for example.
But I'm now stuck again in computing where a given fragment is located in the world.
So, how can this be done?
Why I need this, is basically I have a texture (32x32) with which to darken the world at particular places based on the alpha channel of the texture.
0,0 in the texture would correspond to 0,0,0 in the world. Given a "world map or terrain" of size 100,100.
This doesn't answer the question on how to get the world position of a fragment in a shader used in Filters, but anyway:
It was explained to me that it would be wiser to implement fog of war for all the shaders used by objects in the game, becaues it's easier and a lot more easily extendable (one can toggle the FOW effect on individual objects or materials).
The answer is to add a new varying vec2 called "worldPos" for example and set it to worldPos = g_WorldMatrix * vec4(inPosition, 1.0); in the vertex shader.
That's all. This does not work for Filters though.
I'm trying to create a procedural animation engine for a simple 2D game, that would let me create nice looking animations out of a small number of images (similar to this approach, but for 2D: http://www.gdcvault.com/play/1020583/Animation-Bootcamp-An-Indie-Approach)
At the moment I have keyframes which hold data for different animation objects, the keyframes are arrays of floats representing the following:
translateX, translateY, scaleX, scaleY, rotation (degrees)
I'd like to add skewX, skewY, taperTop, and taperBottom to this list, but I'm having trouble properly rendering them.
This was my attempt at implementing a taper to the top of the sprite to give it a trapezoid shape:
float[] vert = sprite.getVertices();
vert[5] += 20; // top-left vertex x co-ordinate
vert[10] -= 20; // top-right vertex x co-ordinate
batch.draw(texture, vert, 0, vert.length);
Unfortunately this is producing some weird texture morphing.
I had a bit of a Google and a look around StackOverflow and found this, which appears to be the problem I'm having:
http://www.xyzw.us/~cass/qcoord/
However I don't understand the maths behind it (what are s, t, r and q?).
Can someone explain it a bit simpler?
Basically, the less a quad resembles a rectangle, the worse the appearance due to the effect of linearly interpolating the texture coordinates across the shape. The two triangles that make up the quad are stretched to different sizes, so linear interpolation make the seam very noticeable.
The texture coordinates of each vertex are linearly interpolated for each fragment that the fragment shader processes. Texture coordinates typically are stored with the size of the object already divided out, so the coordinates are in the range of 0-1, corresponding with the edges of the texture (and values outside this range are clamped or wrapped around). This is also typically how any 3D modeling program exports meshes.
With a trapezoid, we can limit the distortion by pre-multiplying the texture coordinates by the width and then post-dividing the width out of the texture coordinates after linear interpolation. This is like curving the diagonal between the two triangles such that its slope is more horizontal at the corner that is on the wider side of the trapezoid. Here's an image that helps illustrate it.
Texture coordinates are usually expressed as a 2D vector with components U and V, also known as S and T. But if you want to divide the size out of the components, you need one more component that you are going to divide by after interpolation, and this is called the Q component. (The P component would be used as the third position in the texture if you were looking up something in a 3D texture instead of a 2D texture).
Now here comes the hard part... libgdx's SpriteBatch doesn't support the extra vertex attribute necessary for the Q component. So you can either clone SpriteBatch and carefully go through and modify it to have an extra component in the texCoord attribute, or you can try to re-purpose the existing color attribute, although it's stored as an unsigned byte.
Regardless, you will need pre-width-divided texture coordinates. One way to simplify this is to, instead of using the actual size of the quad for the four vertices, get the ratio of the top and bottom widths of the trapezoid, so we can treat the top parts as width of 1 and therefore leave them alone.
float bottomWidth = taperBottom / taperTop;
Then you need to modify the TextureRegion's existing texture coordinates to pre-multiply them by the widths. We can leave the vertices on the top side of the trapezoid alone because of the above simplification, but the U and V coordinates of the two narrow-side vertices need to be multiplied by bottomWidth. You would need to recalculate them and put them into your vertex array every time you change the TextureRegion or one of the taper values.
In the vertex shader, you would need to pass the extra Q component to the fragment shader. In the fragment shader, we normally look up our texture color using the size-divided texture coordinates like this:
vec4 textureColor = texture2D(u_texture, v_texCoords);
but in our case we still need to divide by that Q component:
vec4 textureColor = texture2D(u_texture, v_texCoords.st / v_texCoords.q);
However, this causes a dependent texture read because we are modifying a vector before it is passed into the texture function. GLSL provides a function that automatically does the above (and I assume does not cause a dependent texture read):
vec4 textureColor = texture2DProj(u_texture, v_texCoords); //first two components automatically divided by last component
I'm trying to create a glow effect. I'm using OpenGL ES 3.0. I can create an outline of my object (a cube) using the stencil buffer, so I have no trouble with that. I can also render this outline to texture using a FBO, and achieve a blur effect using a box blur or Gaussian blur. However, the part that I'm have trouble with is in combining the resultant blurred glow outline (i.e the post-processed image) back with my original object, so that it lines the edges of the cube. I can blit the blurred texture onto the screen--but that copies the low-resolution blur over my high-resolution cube--meaning that I can no longer see my cube, just the blurred outline texture.
Question: How do I combine both into one image on the screen?
I want to make a composite image that consists of both images.
It's probably something simple, but I can't think of a way to do it. Pseudo code would be helpful.
Thanks
I think you want additive blending of those two images, correct?. Two approaches come to mind. Note that when using additive blend, you automatically get brighter colors as the values are summed.
1) If you have both the original cube and the blurred cube in textures, render a single screen-size quad with a fragment shader like this:
#version 300 es
precision mediump float;
uniform sampler2D originalTexture;
uniform sampler2D blurTexture;
in vec2 texCoord;
void main()
{
vec4 originalColor = texture(originalTexture, texCoord);
vec4 blurColor = texture(blurTexture, texCoord);
gl_FragColor = originalColor+blurColor;
}
The advantage with this approach is you can easily add multipliers to the colors to fine-tune the effect, e.g. tone down the glow if it's too much.
Alternatively, you can render your cube normally from geometry. In that case, just add the texture lookup from the blur texture to your fragment shader, and add the color values.
2) You could leverage the fixed-function blending. First render the original cube as you would normally, then set the blend mode:
glEnable(GL_BLEND);
glBlendEquation(GL_FUNC_ADD);
glBlendFunc(GL_ONE, GL_ONE);
Now, when you bind the blur texture and render to the same render target, those should overlap and give you your effect. This gives an equivalent result to the above shader. You can try using glBlendFunc(GL_ONE_MINUS_DST_COLOR, GL_ONE) too to reduce the result brightness. This approach should be faster than the shader approach.
I'm unsure if blending works with glBlitFramebuffer, but it's worth a shot I guess. Hope some of this works!
In my game, the world is made of cubes, but the cubes are divided into 5 parts: a tetrahedron and 4 corners. Each type of block has two colors. This is what a block might look like if one corner was cut, although each corner/face may have different colors from the rest.
The problem is, on the tetrahedral faces, I want the edges between the triangles to be seamless. So I can't use textures. (I could, but they would need to be high-res, and if I want to animate the colors (for example on water) this is not an option).
I've found these approaches:
Drawing each triangle on each tetrahedral face, then each square on each cubical face (using a VBO and all that stuff)
Too many polys! Lag ensues. And this was only rendering the tetrahedrals.
Using a fragment shader on world geometry
The math is simple: for each axis, find if the point is less than 0.5 within the cube and xor the results. This determines which color to use. I got lag, but I think my code is bad.
3D textures on world geometry
This seems to be the best option given how perfectly it matches my situation, but I really don't know.
Using instanced geometry with any of the above
I'm not sure about this one; I've read instancing can be slow on large scales. I would need 31 meshes, or more if I want to optimize for skipping hidden surfaces (which is probably unnecessary anyways).
Using a geometry shader
I've read geometry shaders don't perform well on large scales.
Which of these options would be the most efficient? I think using 3d and 2d textures might be the best option, but if I get lag I want to be sure it's because I'm using bad code not an inefficient approach.
Edit: Here's my shader code
#version 150 core
in vec4 pass_Position;
in vec4 pass_Color1;
in vec4 pass_Color2;
out vec4 out_Color;
void main(void) {
if ((mod(abs(pass_Position.x),1f)<=0.5f)^^(mod(abs(pass_Position.y),1f)<=0.5f)^^(mod(abs(pass_Position.z),1f)<=0.5f)) out_Color = pass_Color1;
else out_Color = pass_Color2;
}
The problem is, on the tetrahedral faces, I want the edges between the triangles to be seamless. So I can't use textures. (I could, but they would need to be high-res, and if I want to animate the colors (for example on water) this is not an option).
That's not necessarily the case. Remember that OpenGL doesn't see whole objects, but just individual triangles. So when rendering that cut face, it's in no way different to just render its flat, "fleshless" counterpart.
Any hard edge on the inner tetrahedron doesn't suffer from a texture crease as the geometrical edge is much stronger. So what I'd do is to have a separate 2D planar texture space aligned with the tetrahedral surfaces, which is shared by all faces coplanar to this (on a side note: applying this you could generate the texture coordinates using a vertex shader from the vertex position).
That being said: Simple 2D flat textures will eventually hit some limitations. Since you're effectively implementing a variant of an implicit surface tesselator (with the scalar field creating the surface being binary valued) it makes sense to think about procedural volumetric texture generation in the fragment shader.
What is the equivalent of gl_LightSource[0].position.xyz when writing in GLSL? In other words, how would I take a vec3 (or vec4) in GLSL and apply the same transformations to it that OpenGL applies to gl_LightSource[0].position? Is there a mat4 I could use?
You can do whatever you like. That's the nice thing about shaders. Define yourself a set of uniforms that describe a light. Its position may in any space you desire, you just have to somehow apply the right transformations.
Shaders are all about free choice, you can do whatever suits you best. Fixed function OpenGL does illumination calculations in eye/view space, i.e. after the modelview transform. To do this, it transforms the light position with the modelview, right when you call glLightfv(GL_LIGHT…, GL_POSITION, …). Then when drawing geometry, after the modelview transform has been applied as well, the illumination calculation is performed in view space.