I've got a question about some code that I found on the internet without further explanation.
It is about the calculation for normal mapping.
On my vertex shader, I got a uniform variable (vec3) of the light position. But the position of the light got premultiplied with the view matrix. So the light position is accessed relative to the camera.
The Java code looks like this: (note that I work with multiple lights, this has no influence on the calculation itself; super.loadVector() uploads a uniform vector).
public void loadLights(List<Light> lights, Matrix4f viewMatrix){
super.loadFloat(this.location_lightAmount, Math.min(RenderSettings.ENTITIES_MAX_LIGHTS,lights.size()));
for(int i=0;i<RenderSettings.ENTITIES_MAX_LIGHTS;i++){
if(i<lights.size()){
super.loadVector(location_lightPositionEyeSpace[i], getEyeSpacePosition(lights.get(i), viewMatrix));
super.loadVector(location_lightColor[i], lights.get(i).getColor());
super.loadVector(location_lightAttenuation[i], lights.get(i).getAttenuation());
}
}
}
public static Vector3f getEyeSpacePosition(Light light, Matrix4f viewMatrix){
Vector3f position = light.getAbsolutePosition();
Vector4f eyeSpacePos = new Vector4f(position.x,position.y, position.z, 1f);
Matrix4f.transform(viewMatrix, eyeSpacePos, eyeSpacePos);
return new Vector3f(eyeSpacePos);
}
This makes no sense to me because the camera position should have no influence on the calculation of the normals, right?
The next thing is following: When I render without normal mapping I transform the normal of each vertex by using the transformation matrix like this:
in vec3 normal;
out vec3 surfaceNormal;
uniform mat4 transformationMatrix;
void(main) {
//........
surfaceNormal = (transformationMatrix * vec4(normal,0.0)).xyz;
//........
}
However, this calculation is not applied in the normal mapping code that I found. Here, the normal (and the tangent) is multiplied with the viewMatrix and transformationMatrix like this:
mat4 modelViewMatrix = viewMatrix * transformationMatrix;
vec3 surfaceNormal = (modelViewMatrix * vec4(normal,0.0)).xyz;
The bitangent can be easily calculated with the surfaceNormal and the surfaceTangent calculated. The tangentSpaceMatrix is then used for transforming the light position. But why is the view matrix used for all the transformations?
The way that I understood normal mapping is the following: transform the whole system, so that the triangle (that is being rendered) lies flat on the ground (tangent space). Then the normal equals the RGB-value of the normal map.
(I know that this might sound a little bit confusing, I hope you understand what I mean).
Greetings, Finn
Related
I need to compute Normals for each Triangle Face (not for each vertex) using Opengl ES 2.0 in Android. But I can't pass attribute in the Fragment Shader directly.
Found one solution: Repeat vertices for each Triangle, and pass Triangle face Normal as an Attribute in Vertex Shader.
But I don't want to duplicate the vertices. I am drawing triangle using Vertex Indices.
So, a vertex is shared by more than one triangle, then how should I compute Triangle face Normals.
p.s. I am a newbie in opengl.
The easiest solution is just to duplicate vertices. The vertex shader is very rarely a bottleneck. I do not know your specific needs, however, there are cirumstances when duplicating a vertex is not a good solution. For example, if the mesh is skinned and animated that means a lot of computation happens in the vertex shader. Another case is when the mesh is animated in some weird way in the vertex shader and you have to recompute the normals. Clearly, you cannot compute per face normals in the vertex shader. You could do that in the geometry shader, but we do not have one in OpenGL ES 2.0. However, there is a simple solution - compute normals in fragment shader! So, if duplication of vertices does not work for you, here is the solution:
We will need an OpenGL extension - standard_derivatives, which is widely supported, but you will still need to check if it is supported on the device before running the code. To enable the extension, you will have to add the following line to the fragment shader before it's code:
#extension GL_OES_standard_derivatives : enable
We will need a varying variable for the vertex position in the world coordinates. It should be computed in the vertex shader and how it is done depends much on your shader. It is used for many needs, so you may already compute it in your vertex shader. So let's assume, that we have this line in the fragment shader:
varying vec3 positionWorld;
We will need a view matrix of the camera. It is possible that you are already passing one to the fragment shader. Let's assume that we have this uniform in the fragment shader:
uniform mat4 viewMatrix;
Now, we are going to compute the normal. First, we compute a normal in view space and then transform into the worldspace. To compute normal in the viewspace, we use derivative functions:
vec3 normalViewSpace = normalize(cross(dFdx(positionWorldSpace), dFdy(positionWorldSpace)));
Here a derivative of position is taken with respect to x and y coordinate in screen space. That means that we have two vectors that are in lay in the surface plane. To get normal to the surface we do a cross product. Sure, the result is not a unit vector, so we need also to normalize it.
The last step is to compute normal in the worldspace. View matrix applies a transformation from world space to view space. One could think that we need to compute the inverse of it since we need to go from view space to world space, but because view matrix is orthonormal, the transpose of that matrix is also its inverse, so the code will be:
vec3 normalWorldSpace = (vec4(normalViewSpace, 0.0) * viewMatrix).xyz;
To make life easier, we can wrap everything into a function:
vec3 ReconstructNormal(vec3 positionWorldSpace)
{
vec3 normalViewSpace = normalize(cross(dFdx(positionWorldSpace), dFdy(positionWorldSpace)));
vec3 normalWorldSpace = (vec4(normalViewSpace, 0.0) * viewMatrix).xyz;
return normalWorldSpace;
}
Now we have a reconstructed normal in world space. Below, just a simple example, why this can be very useful. Note, that since it uses WebGL, it is also pretty much OpenGL ES 2.0 compatible.
var container;
var camera, scene, renderer;
var mesh;
var uniforms;
var clock = new THREE.Clock();
init();
animate();
function init() {
container = document.getElementById('container');
camera = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 0.1, 100);
camera.position.z = 0.6;
camera.position.y = 0.2;
camera.rotation.x = -0.45;
scene = new THREE.Scene();
var boxGeometry = new THREE.PlaneGeometry(0.75, 0.75, 32, 32);
var heightMap = THREE.ImageUtils.loadTexture("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAFiUAABYlAUlSJPAAAAeeSURBVFhHTdfFq1VfFAfwfbzP7q5ndwcm2IqKgQoiOhQFQXAgDpxecObAP0ARdGJMdGDhwAILO7C7uzvv73wWHPltuLx7z9l7rfWNtc552ZQpUypt27ZN7dq1S/9f7969S1evXk116tRJtWrVSj9//kzfv39PX79+jfvdu3dPpVIp3b9/P3Xo0CH1798/tWrVKjVs2DBt3749vX//PnXr1i3VqFEj9jVq1Ci9efMm3blzJ7Vp0yY1adIk/fr1K2Xz5s2r2CjRy5cv42/r1q3Tnz9/0smTJ6MQh+vVq5du3LgRwSSsrq5OnTp1Sq9fv05fvnyJ38OHD0+7du1K+/bti8Jq166dfvz4EQU7v2TJkoh35cqVADN37txU1bJly0hm+VskaNasWXr+/HkEUe2AAQNSnz590ufPn1OLFi3Sx48f0+3btyO5fUePHo2CJ0yYkJo2bRpgqqqqIq4lzsaNG/8xaCkqGJDYjbdv36aaNWtGkKlTpwadEBTyPHz4MB0/fjw9efIknTp1Kn5bM2fODMqdlUjiFy9epAYNGsR9zCnYck0u+6KA5cuXVz59+hQXX716FbTWrVs3jR8/PnTt169f2rJlS9q2bVsEcNCi56NHj2IvyTAE+ePHj8Mv1u/fv0PSW7duhYwF01h2RsHZ4sWLw4SoVZ1No0ePjorpK8izZ8/+oejRo0dq37590P306dPYL6nCly5dGqyRD5uNGzf+J5Gk58+fj+Lu3bsXHsBsKa+6rHLJaevQgQMHIgGKbt68GYV8+/YtdBVM5c2bN093794NFnSE89euXYvvf//+Tbxlr2u6yHfxFaczLl26FEbNunTpUhk8eHAaMmRIOJSW6NUZUEvKB5IVa+DAgWncuHERvH79+oHGXmcZFgs+FpYktQ8b2OzVq1dI53wp17n84cOH0B/1c+bMiWIk6dixYxSjvzEjETTMh2IsYO/BgweRHAuoJomCJVq0aFHq2bNn9P/FixdDUrIxKtlKq1atKgtq8+HDhyOhBJs3b46gDEozruYBByWlJbSuW8ypKEEZbMyYMWnatGnxe/bs2SGj8zoIEKDFz9atW1dRoemltVRIQ0msQYMGxV9ONrlobpLpDguNCu7du3cwggVzBOUkVfzIkSMDGIDXr1+PYu0HvJQjL2sRSFUkuRvah0sVhTKGLFAzUWEwaEzBYujwjFiuQ01rPoAYesAUpJMUm61evboisCQuQLhnz55/wwJtClQxYxWTU3KLsxXgnsS6BpBiovouJqOPGjUq/GI/QwIYg0ilKmLCNWvWRKUYUIDrqtdeJqT29JAih6SYcV4y5yR2DgAsYKpz586xjyx8Ip4zfme5xhUobUIvhJI5LKilUnMAU5s2bUpHjhyJBJZkffv2TQsXLgxvSEhv0ulzC1rJ0C85mbW2XKU8cZluWocEivAdza6jmpkEKbrCb0uRCrAUp6hZs2bF044xtZ72dj3LstjP5L67RopS/oQr63HjGE2VSiUOqVYxKNVqdPRhKMbq2rVrJMUMUyoacmZ0HxMSug89YyteLuPcHvKGCaGlH00dKGZ10QkGEjQWlhiPhgzlGWLSKdp+Le2ae5gS234GFBsDpijmgCvllZXR5YJhxFyQCwYxrQREm6ECAZbco+HYsWNjJqDTnt27d8fEwwBAihfPuwPGMIExw4qpS/mXsocLFFpNVaq0UQIHtRYkPsxqL4Pmz5H4LjnjOnfs2LF/b0H+8g0wigGAAXnkwoULYdRSfrhMe4FpjWp6QsOx0Jpwng/QFAUKcOLEifjus3///kCGZtT7bi/UZCMxf/Ca/TqPjNnEiRMrKGIgzvRxoxjFWgxaBaLNfc91j9PipbUwnTZkLOMWYtrzku/YIqX9ZoHCmDJbuXJlxWFoVeqANXny5PiLdgEFGDFiRDp06FCg8c5g2Y9qUhTUkwPdEENrDRs2LJ4FJOEznUHebO3atRUDA21QM6MAPgynALoxFy/Q2D7f7ZWUhBZ2UKx4OpMBelLw06RJkyIPBrFw+vTpVMo1LqPDDQm9C+YvqoFSO27dujXt3bs3ng9aS1D7+EJxinAeEwrTFZ4dUGJAUQrUzszOI86KxV/ZggULKtB6WHC+YKqmt5dRSM+cORMIeUVy+/Uyr/CBYJiQLP9HJ15gCo8oBAOSkxEoIMQyM7INGzbEW7HFB2YCGplFN+hpKMkgqUDYcoa+06dPD13piQGv85KuX78+nhnFdFSgj9gex2YAk2b5W0s8jpkINTZLzNG+Q2mj32iGwBBCP5mgNVpNOoigO3fuXPQ4FgojAiGOLlHk0KFD4z0y/jFxiCksdKFZzxfz3JLQHkm9sDKVB5QgHL1ixYo4y9DenhQAGFnd12FAzJ8/P+IfPHgwnT17NpXyqstQ0Jx5zHSoTTeUFg8nlF++fDkQGiJ0J5l79kLpRVZg5739OiOGxM6ZJ87633HHjh0x9Er5y2PZTYZBPRSqdojZvFCqmAcgkojmPgaL/xfpumzZsvjN8SYqnwCEQei1pt9A7Ny5M4CZrlV0Qa8ukNwmfz1gFKFKukPqugQSksB0dI3RdMuMGTNCazJoNxK6rztIx/liGWgYr66uTv8BU33Si9zKcpYAAAAASUVORK5CYII=");
heightMap.wrapT = heightMap.wrapS = THREE.RepeatWrapping;
uniforms = {u_time: {type: "f", value: 0.0 }, u_heightMap: {type: "t",value:heightMap} };
var material = new THREE.ShaderMaterial({
uniforms: uniforms,
side: THREE.DoubleSide,
wireframe: false,
vertexShader: document.getElementById('vertexShader').textContent,
fragmentShader: document.getElementById('fragment_shader').textContent
});
mesh = new THREE.Mesh(boxGeometry, material);
mesh.rotation.x = 3.14 / 2.0;
scene.add(mesh);
renderer = new THREE.WebGLRenderer();
renderer.setClearColor( 0x000000, 1 );
container.appendChild(renderer.domElement);
onWindowResize();
window.addEventListener('resize', onWindowResize, false);
}
function onWindowResize(event) {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
}
function animate() {
requestAnimationFrame(animate);
render();
}
function render() {
var delta = clock.getDelta();
uniforms.u_time.value += delta;
//mesh.rotation.z += delta * 0.5;
renderer.render(scene, camera);
}
body { margin: 0px; overflow: hidden; }
<script src="https://threejs.org/build/three.min.js"></script>
<div id="container"></div>
<script id="fragment_shader" type="x-shader/x-fragment">
#extension GL_OES_standard_derivatives : enable
varying vec3 positionWorld; // position of vertex in world coordinates
vec3 ReconstructNormal(vec3 positionWorldSpace)
{
vec3 normalViewSpace = normalize(cross(dFdx(positionWorldSpace), dFdy(positionWorldSpace)));
vec3 normalWorldSpace = (vec4(normalViewSpace, 0.0) * viewMatrix).xyz;
return normalWorldSpace;
}
// Just some example of using a normal. Here we do a really simple shading
void main( void )
{
vec3 lightDir = normalize(vec3(1.0, 1.0, 1.0));
vec3 normal = ReconstructNormal(positionWorld);
float diffuse = max(dot(lightDir, normal), 0.0);
vec3 albedo = vec3(0.2, 0.4, 0.7);
gl_FragColor = vec4(albedo * diffuse, 1.0);
}
</script>
<script id="vertexShader" type="x-shader/x-vertex">
uniform lowp sampler2D u_heightMap;
uniform float u_time;
varying vec3 positionWorld;
// Example of vertex shader that moves vertices
void main()
{
vec3 pos = position;
vec2 offset1 = vec2(1.0, 0.5) * u_time * 0.01;
vec2 offset2 = vec2(0.5, 1.0) * u_time * 0.01;
float hight1 = texture2D(u_heightMap, uv + offset1).r * 0.02;
float hight2 = texture2D(u_heightMap, uv + offset2).r * 0.02;
pos.z += hight1 + hight2;
vec4 mvPosition = modelViewMatrix * vec4( pos, 1.0 );
positionWorld = mvPosition.xyz;
gl_Position = projectionMatrix * mvPosition;
}
</script>
I've been wondering how to efficiently implement the following image scale procedure in Java or Processing. When scaling out, the bounds of the image wrap around the screen edges.I'd like to apply the same at runtime to my Pixels() array in Processing. (to keep this Processing agnostic - Pixels() is nothing else than a method that returns all pixels on my current screen in an array).
(Note that this example was made in MaxMsp/Jitter using the jit.rota module, which appears to use a very efficient implementation).
unscaled
zoomed out
Can anyone help me out on how to get started? I assume it must be a combination of downscaling the image and creating adjactent copies of it - but this doesn't sound very efficient to me. the above example works perfectly on videos with even the most extreme settings.
One option I can think that will be fast is using a basic fragment shader.
Luckily you've got an example pretty close to what you need that ships with Processing via File > Examples > Topics > Shaders > Infinite Tiles
I won't be able to efficiently provide a decent start to finish guide, but
there's an exhaustive PShader tutorial on the Processing website if you're starting the from scratch.
A really rough gist of what you need:
shaders are programs that run really fast and parallelised on the GPU, split into two: vertex shaders (deal with 3d geometry mainly), fragment shaders (deal with "fragments"(what's about to become pixels on screen) mainly). You'll want to play with a fragment shader
The language is called GLSL and is a bit different(fewer types, stricter, simpler syntax), but not totally alien(similar C type of declaring variables, functions, conditions, loops, etc.)
if you want to make a variable from a GLSL program accessible in Processing you prefix it with the keyword uniform
use textureWrap(REPEAT) to wrap edges
to scale the image and wrap it you'll need to scale the texture sampling coordinates:
Here's what the InfiniteTiles scroller shader looks like:
//---------------------------------------------------------
// Display endless moving background using a tile texture.
// Contributed by martiSteiger
//---------------------------------------------------------
uniform float time;
uniform vec2 resolution;
uniform sampler2D tileImage;
#define TILES_COUNT_X 4.0
void main() {
vec2 pos = gl_FragCoord.xy - vec2(4.0 * time);
vec2 p = (resolution - TILES_COUNT_X * pos) / resolution.x;
vec3 col = texture2D (tileImage, p).xyz;
gl_FragColor = vec4 (col, 1.0);
}
You can simplify this a bit as you don't need to scrolling. Additionally, instead of subtracting, and multiplying(- TILES_COUNT_X * pos), you can simply multiply:
//---------------------------------------------------------
// Display endless moving background using a tile texture.
// Contributed by martiSteiger
//---------------------------------------------------------
uniform float scale;
uniform vec2 resolution;
uniform sampler2D tileImage;
void main() {
vec2 pos = gl_FragCoord.xy * vec2(scale);
vec2 p = (resolution - pos) / resolution.x;
vec3 col = texture2D (tileImage, p).xyz;
gl_FragColor = vec4 (col, 1.0);
}
Notice I've repurposed the time variable to become scale, therefore the Processing code accessing this uniform variable must also change:
//-------------------------------------------------------------
// Display endless moving background using a tile texture.
// Contributed by martiSteiger
//-------------------------------------------------------------
PImage tileTexture;
PShader tileShader;
void setup() {
size(640, 480, P2D);
textureWrap(REPEAT);
tileTexture = loadImage("penrose.jpg");
loadTileShader();
}
void loadTileShader() {
tileShader = loadShader("scroller.glsl");
tileShader.set("resolution", float(width), float(height));
tileShader.set("tileImage", tileTexture);
}
void draw() {
tileShader.set("scale", map(mouseX,0,width,-3.0,3.0));
shader(tileShader);
rect(0, 0, width, height);
}
Move the mouse to change scale:
Update You can play with a very similar shader here:
I effectively did come up with a solution - but will implement George's method next as the speed difference using shaders seems to be worth it!
public void scalePixels(double wRatio,double hRatio, PGraphics viewPort) {
viewPort.loadPixels();
int[] PixelsArrayNew = viewPort.pixels.clone();
double x_ratio = wRatio ;
double y_ratio = hRatio ;
double px, py ;
for (int i=0;i<viewPort.height;i++) {
for (int j=0;j<viewPort.width;j++) {
px = Math.floor(j%(wRatio*viewPort.width)/x_ratio) ;
py = Math.floor(i%(hRatio*viewPort.height)/y_ratio) ;
viewPort.pixels[(int)(i*viewPort.width)+j] = PixelsArrayNew[(int)((py*viewPort.width)+px)] ;
}
}
viewPort.updatePixels();
}
I have a scene containing a map generated from a heightmap. I can move this map around on the X and Y axis, where the map gets moved upwards or downwards such that the map geometry intersects the origin. There is a single light that hovers over the map, always 5 units above the height of the geometry at its x and y coordinate.
When the camera is located close to the origin of the scene, all the lighting behaves fine. The more I move away from it, however, the more triangles jump first quickly to bright white, after which they become black almost instantly. I have not been able to figure out what is causing this.
Here is an overview of the structure of the scene graph:
Container instances do not change the openGL state
The shader node activates the shader pair shown below.
The problem is also present when using the fixed function pipeline.
The scene is translated and rotated prior to rendering the scene graph
I'm fairly certain the problem lies with the construction of the scene, but I'm not entirely certain.
Here are some screenshots of the effects. First one where the light is close to the camera, and where the camera is located some distance away from the scene origin:
Note the light being shown as a sphere, marked by the red circle. Second, one where the light is away from the camera:
I am also drawing the normals for reference. The glow visible in this picture is always present, no matter where the light is located relative to the camera.
Here are my shaders. I'm fairly certain they work as they're supposed to, because they worked properly in ShaderMaker:
Vertex shader:
varying vec3 normal;
varying vec3 position;
void main( void )
{
gl_Position = ftransform();
gl_TexCoord[0] = gl_MultiTexCoord0;
normal = gl_NormalMatrix * gl_Normal;
position = ( gl_ModelViewMatrix * gl_Vertex ).xyz;
}
Fragment shader:
varying vec3 normal;
varying vec3 position;
vec4 lightSource(vec3 norm, vec3 view, gl_LightSourceParameters light)
{
vec3 lightVector = normalize(light.position.xyz - view);
vec3 reflection = normalize(lightVector - view.xyz);
float diffuseFactor = max(0, dot(norm, lightVector));
float specularDot = max(0, dot(norm, reflection));
float specularFactor = pow(specularDot, gl_FrontMaterial.shininess);
return
gl_FrontMaterial.ambient * light.ambient +
gl_FrontMaterial.diffuse * light.diffuse * diffuseFactor +
gl_FrontMaterial.specular * light.specular * specularFactor;
}
vec4 lighting()
{
// normal might be damaged by linear interpolation.
vec3 norm = normalize(normal);
return
gl_FrontMaterial.emission +
gl_FrontMaterial.ambient * gl_LightModel.ambient +
lightSource(norm, position, gl_LightSource[0]);
}
void main()
{
gl_FragColor = lighting();
}
The solution to the problem was that I was applying glFrustrum() on the modelView matrix. When I set the glMatrixMode to the projection matrix before calling glFrustrum instead, the scene rendered correctly.
I'm writing a small libGdx application that may be able to distort a face simulating hits (something similar to FaceSmash), but can't find what kind of filter should I apply, neither info about math algorithms to achieve something like a bulge effect.
I've started from SphereFilter on
http://www.jhlabs.com/
but definitely the effect is not what I'm looking for.
I'm aware of
Image Warping - Bulge Effect Algorithm
and
How to warp images in Android?
I've achieved to 'translate' the bulge effect mentioned on the first thread, but is relatively slow, and can't find the way to create bulges in a region, not the whole image.
The warp sample on Api demos (2nd thread) is really fast, and changes only part of the image, but is not exactly a bulge effect, and my image processing maths are far from understanding how could I modify the algorithm to change the effect.
I can feel I'm in the right direction, but I'm totally stuck. Any idea of how any of these algorithms could be modified in order to get 'local' bulges inside the image?
EDIT
Just found this thread.-
How can you apply distortions to a UIImage using OpenGL ES?
Not android, but looks promising. I'll give a try to OpenGL shaders and share (hopefully) a solution for my scenario.
Answering my own question, I managed to find a solution using OpenGL shaders based on GPUImage framework
The resulting fragment shader looks like this.-
#ifdef GL_ES
precision mediump float;
#endif
varying vec4 v_color;
varying vec2 v_texCoords;
uniform sampler2D u_texture;
uniform float aspectRatio;
uniform vec2 center;
uniform float radius;
uniform float scale;
void main()
{
vec2 textureCoordinateToUse = vec2(v_texCoords.x, (v_texCoords.y * aspectRatio + center.y - center.y * aspectRatio));
float dist = distance(center, textureCoordinateToUse);
textureCoordinateToUse = v_texCoords;
if (dist < radius)
{
textureCoordinateToUse -= center;
float percent = 1.0 - ((radius - dist) / radius) * scale;
percent = percent * percent;
textureCoordinateToUse = textureCoordinateToUse * percent;
textureCoordinateToUse += center;
}
gl_FragColor = texture2D(u_texture, textureCoordinateToUse);
}
Hope someone find it helpful.
EDIT: Changing the fragment shader so only one light per-run was possible solved my first problem. My second problem still stands.
thanks in advance for any help you can offer. I have been working on a deferred shading pipeline for my LWJGL rendering engine for a couple weeks now. I though I had managed to get everything to work as I would expected, but after distributing the program to a few people I know, I began encountering problems. I'll try to keep this as short as possible. Thanks for staying with me.
I'll start with the first of the two problems in my title. On my machine (AMD Phenom II 965 and Nvidia GTX 480) the final product of my renderer was exactly as expected. (I was going to post a link to an image, but seeing as I am a new user, I was unable to post more than 3 hyperlinks. But it is sufficient to say, it looked like it should have.)
This is exactly what I intended so I though the renderer was working fine. I sent it off to a friend (who was using a GT 440) and they had the same results.
Soon after this I gave a build of the engine to a friend of mine who has a laptop (with a GT 540M). This was what the renderer produced (ignore the FPS counter, it doesn't work):
http://i.imgur.com/DxxFEpy.png
Obviously this is not at all the results I expected. I experienced the same results on every other mobile graphics card I was able to test on. After more than a week of banging my head on my desk, I was able to narrow the problem down to the lighting pass, where glBlendFunc is called. My code is as follows:
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
List<Float[]>[] listArray = LightData.getInstance().updateLights();
List<Float[]> lightsColor = listArray[1];
List<Float[]> lightsPos = listArray[0];
viewMatrix.setViewMatrix(camera.getTranslation(), camera.getRotation());
glDisable(GL_DEPTH_TEST);
glUseProgram(0);
glCallList(quadList);
FBO.useShader();
FBO.passTextures(); //Just sets the shader uniform values to the correct textures
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glEnable(GL_BLEND);
glLoadIdentity();
glBlendFunc(GL_ONE, GL_ONE) ; //using GL_ONE and GL_ONE_MINUS_SRC_ALPHA have the same effect
for (int i = 0; i < lightsPos.size(); i++) {
glClear(GL_DEPTH_BUFFER_BIT);
int uniformLightPosition = glGetUniformLocation(FBO.getShaderID(), "uniformLightPosition");
int uniformLightColor = glGetUniformLocation(FBO.getShaderID(), "uniformLightColor");
int uniformViewMatrix = glGetUniformLocation(FBO.getShaderID(), "uniformViewMatrix");
int uniformAmbient = glGetUniformLocation(FBO.getShaderID(), "uniformAmbient");
glUniform1(uniformLightPosition, Tools.asFloatBuffer(lightsPos.get(i)));
glUniform1(uniformLightColor, Tools.asFloatBuffer(lightsColor.get(i)));
glUniform1f(uniformAmbient, 0.01f);
glUniformMatrix4(uniformViewMatrix, true, viewMatrix.asFloatBuffer());
glCallList(quadList); //is just a display list for a fullscreen quad (using GL_QUADS)
}
glDisable(GL_BLEND);
glEnable(GL_DEPTH_TEST);
This first thing that you might notice is the fact that I draw a quad and then clear the depth and color buffers. That will be a addressed in my next question, although I wouldn't be surprised if the problem in my next question was closely related to the one in this question. I am almost sure (99%) that the problem is in this method because when testing with an older build of the engine that only supported one light but was still using the deferred pipeline I was able to get perfect results on every computer I tested on. Once again, the renderer works on every desktop graphics card I have tested, but not on any laptop graphics cards. I have ruled out almost everything except this method. It might be worth noting that I have had no success using a texture with an internal format that is not GL_RGBA32f or GL_RGBA16f as a render target. Has anyone seen this before, or can anyone offer assistance? I would love if anyone even had any idea where to start looking for problems at this point, because I've got nothing. I've been complete unable to find a solution by myself.
Then moving on to the second problem and second question. In the beginning of the last code block I had a few line of codes that created a quad on the screen with no shader:
glDisable(GL_DEPTH_TEST);
glUseProgram(0);
glCallList(quadList);
FBO.useShader();
FBO.passTextures(); //Just sets the shader uniform values to the correct textures
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glEnable(GL_BLEND);
As far as I can tell, this code should do absolutely nothing. But when I remove the drawing of the quad, The window displays this:
http://i.imgur.com/mkMsP0F.png
I didn't know what else to call this besides "ghosting" because it is like a ghost image (at least that how it seems to me). When I rotate the MV matrix it distorts in the direction I rotate, and the first set of lights (I use an array of 7) light it, but then the rest light the actual model. I can't explain why this happens because the code that generates this image is exactly like the code above without the glCallList(quadList); which means the depth and color buffers are still cleared right before I enter the loop. I can't explain this problem at all. Does anyone have any idea what's wrong and how to fix it, or at least an idea about what's wrong?
EDIT I have found that this only happens to models that have texture coordinates. I don't know why.
EDIT It looks like when I limit the amount of lights allowed in each shader to 1, the ghosting gets much less noticeable, but is still there, So I assume this means that one run of the fragment shader results in these ghosts.
Thanks for ANY help anyone has with either of these two problems, it is greatly appreciated. If you have any questions for me just ask, although it might take me some time to get back to you, I'll try to get back as fast as possible.
EDIT Sory, I forgot my shader code:
GeometryPass vertex:
uniform sampler2D tex;
varying vec3 surfaceNormal;
varying vec3 varyingVertex;
void main() {
vec4 color = vec4(texture2D(tex, gl_TexCoord[1].st));
gl_FragColor = color;
}
GeometryPass fragment:
uniform sampler2D tex;
varying vec3 surfaceNormal;
varying vec3 varyingVertex;
uniform float materialValue;
uniform float specValue;
void main() {
vec4 color = vec4(texture2D(tex, gl_TexCoord[1].st)) ;//vec4(0.25,0.25,0.25,1);
vec4 vertex = vec4(varyingVertex, materialValue);
vec4 normal = vec4(surfaceNormal, specValue);
gl_FragData[0] = color;
gl_FragData[1] = vertex;
gl_FragData[2] = normal;
}
LightPass Phong vertex:
void main() {
gl_TexCoord[0] = gl_MultiTexCoord0;
gl_Position = gl_ModelViewMatrix * gl_Vertex;
}
LightPass Phong Fragment
uniform sampler2D location;
uniform sampler2D normal;
uniform sampler2D color;
uniform float uniformLightPosition[21];
uniform mat4 uniformViewMatrix;
uniform float uniformLightColor[28];
void main() {
vec4 color = texture2D(color, gl_TexCoord[0].st);
vec4 locationAndMat = texture2D(location, gl_TexCoord[0].st);
vec4 normalAndSpec = texture2D(normal, gl_TexCoord[0].st);
vec3 vertexPosition = locationAndMat.xyz;
vec3 surfaceNormal = normalAndSpec.xyz;
float spec = normalAndSpec.a;
float specA = locationAndMat.a;
vec4 lightColor[7];
int iterator = 0;
for (int i = 0; i<28; i = i+4) {
lightColor[iterator] = vec4(uniformLightColor[i], uniformLightColor[i+1], uniformLightColor[i+2], uniformLightColor[i+3]);
iterator = iterator + 1;
}
vec3 lightPos[7];
iterator = 0;
for (int i = 0; i<21; i = i+3) {
lightPos[iterator] = vec3(uniformLightPosition[i], uniformLightPosition[i+1], uniformLightPosition[i+2]);
lightPos[iterator] = (uniformViewMatrix * vec4(lightPos[iterator],1)).xyz ;
iterator = iterator + 1;
}
vec4 fragData[7];
vec4 endColor;
for (int i = 0; i<7 ; i++) {
if (lightColor[i] != vec4(0,0,0,0) && color != vec4(0,0,0,0)) {
vec3 lightDistance = lightPos[i]-vertexPosition;
float distance = pow((pow(lightDistance.x, 2) + pow(lightDistance.y, 2) + pow(lightDistance.z, 2)), 0.5);
if (distance < lightColor[i].a) {
float att = 1/((-3/800*(lightColor[i].a) + 0.225)*pow(distance, 2));
vec3 lightDirection = normalize(lightDistance);
float diffuseLightIntensity = max(0.0, dot(surfaceNormal, lightDirection));
fragData[i] += (vec4(diffuseLightIntensity,diffuseLightIntensity,diffuseLightIntensity,1));
vec3 reflectionDirection = normalize(reflect(-lightDirection, surfaceNormal));
float specular = max(0.0, dot(reflectionDirection, -normalize(vertexPosition)));
if (diffuseLightIntensity != 0) {
float fspecular = pow(specular, spec);
vec4 fspec = vec4(fspecular*specA, fspecular*specA, fspecular*specA,1);
fragData[i] += fspec;
}
fragData[i] *= lightColor[i];
fragData[i] *= 0.1;
fragData[i].a = 0;
fragData[i] *= att;
endColor += fragData[i];
}
}
}
gl_FragData[0] = endColor * color;
}
I solved the primary problem! This was good enough for me. It appears the problem I was having was that I had to many instructions per fragment shader (because of the for loop). When I adjusted the shader to only allow one light it worked as expected! I just use blending to take care of everything as I used to, but run the shader more. The downside is It requires more filtrate, but the upside is it works on older hardware and laptops.
I still can't figure out what's causing the ghosting, but it is less important to me, as I have a poor fix for that.