Graphics Programming I - Software Rasterization Part II PDF

GRAPHICS PROGRAMMING I SOFTWARE RASTERIZATION PART II Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad [TriangleList] Last week we rendered two triangles with different depths. Now we make a quad! The simplest quad would have 4 vertices and 6 indices, de...

GRAPHICS PROGRAMMING I SOFTWARE RASTERIZATION PART II Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad [TriangleList] Last week we rendered two triangles with different depths. Now we make a quad! The simplest quad would have 4 vertices and 6 indices, defining two triangles. Instead, we start off with 9 vertices and 24 indices (3 indices * 8 triangles). 𝑽𝟎 𝑽𝟏 𝑽𝟐 𝑉𝑒𝑟𝑡𝑖𝑐𝑒𝑠 𝑇𝑟𝑖𝑎𝑛𝑔𝑙𝑒𝑠 𝑽𝟎 = (-3, 3, -2) 𝑻𝟎 = (3, 0, 4) 𝑽1 = (0, 3, -2) 𝑻1 = (0, 1, 4) 𝑽2 = (3, 3, -2) 𝑻2 = (4, 1, 5) 𝑽3 = (-3, 0, -2) 𝑽𝟒 𝑻3 = (1, 2, 5) 𝑽𝟑 𝑽5 𝑽4 = (0, 0, -2) 𝑻4 = (6, 3, 7) 𝑽𝟓 = (3, 0, -2) 𝑻5 = (3, 4, 7) 𝑽6 = (-3, -3, -2) 𝑻6 = (7, 4, 8) 𝑽7 = (0, -3, -2) 𝑻7 = (4, 5, 8) 𝑽8 = (3, -3, -2) 𝑽6 𝑽7 𝑽8 Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad Using code from last week, with color white for all vertices, you should have this result. Moving the camera, you might encounter an issue: black lines between individual squares To fix this issue: check types & bounds which determine where a triangle ends. What if I told you… we can optimize triangle data even further. Graphics Programming 1 – Software Rasterization Part II  Rasterization: Quad [TriangleStrip] 𝑽𝟎 𝑽𝟏 𝑽𝟐 𝑻𝟎 = (3, 0, 4) A series of triangles where each triangle and the next have an edge in common is called a 𝑻1 = (0, 1, 4) triangle strip. 𝑻2 = (4, 1, 5) In the triangle list, each of these edges in common occurs twice: once per triangle. This 𝑽𝟑 𝑽𝟒 𝑽5 𝑻3 = (1, 2, 5) is duplicate information. 𝑻4 = (6, 3, 7) In memory these indices form one list: 𝑻5 = (3, 4, 7) (3,0,4,0,1,4,4,1,5,1,2,5,6,3,7,3,4,7,7,4,8,4,5,8) 𝑻6 = (7, 4, 8) 𝑻7 = (4, 5, 8) 𝑻𝟎 𝑻𝟏 𝑻𝟐 𝑻𝟑 𝑻𝟒 𝑻𝟓 𝑻𝟔 𝑻𝟕 𝑽6 𝑽7 𝑽8 Graphics Programming 1 – Software Rasterization Part II  Rasterization: Quad [TriangleStrip] How can we alter the list to deduplicate common edges? 𝑽𝟎 𝑽𝟏 𝑽𝟐 𝑽𝟎 𝑽𝟏 𝑽𝟐 𝑇𝑟𝑖𝑎𝑛𝑔𝑙𝑒𝑠 𝑇𝑟𝑖𝑎𝑛𝑔𝑙𝑒𝑠 𝑻𝟎 = (3, 0, 4) 𝑻𝟎 = (3, 0, 4) 𝑻1 = (0, 1, 4) 𝑻1 = (0, 4, 1) 𝑻2 = (4, 1, 5) 𝑻2 = (4, 1, 5) 𝑽𝟒 𝑻3 = (1, 2, 5) 𝑻3 = (1, 5, 2) 𝑽𝟑 𝑽5 𝑽𝟒 𝑽𝟑 𝑽5 Swapping the last two vertices of every other triangle aligns the common edges. (3,0,4, 0,1,4, 4,1,5, 1,2,5, 6,3,7, 3,4,7, 7,4,8, 4,5,8) (3,0,4, 0,4,1, 4,1,5, 1,5,2, 6,3,7, 3,7,4, 7,4,8, 4,8,5) Keep in mind that the reordered triangles are now counter-clockwise! Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad [TriangleStrip] Now we can remove duplicate vertices. 𝑽𝟎 𝑽𝟏 𝑽𝟐 (3,0,4, 0,4,1, 4,1,5, 1,5,2, 6,3,7, 3,7,4, 7,4,8, 4,8,5) (3,0,4, 0,4,1, 4,1,5, 1,5,2, 6,3,7, 3,7,4, 7,4,8, 4,8,5) 𝑽𝟒 (3, 0, 4, 1, 5, 2, 6, 3, 7, 4, 8, 5) 𝑽𝟑 𝑽5 For each triangle which has a common edge with the next we removed its last two vertices. 𝑽6 𝑽7 𝑽8 Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad [TriangleStrip] How to interpret this optimized list? 𝑽𝟎 𝑽𝟏 𝑽𝟐 𝑻𝟎 𝑻𝟐 𝑻𝟒 𝑻𝟔 (3, 0, 4, 1, 5, 2, 6, 3, 7, 4, 8, 5) 𝑻𝟏 𝑻𝟑 𝑻𝟓 𝑻𝟕 We read triangles by shifting one index at a time instead of three at a time. 𝑽𝟑 𝑽𝟒 𝑽5 Every other triangle, which has an odd index, is counter-clockwise. To make them clockwise again, we reverse the last two vertices when reading these. 𝑻1 = (0, 4, 1) 𝑻1 = (0, 1, 4) 𝑽6 𝑽7 𝑽8 Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad [TriangleStrip] There is still a gap between the first and second strip (going to the next “row” or “band”). How can we account for this given that we shift one index at a time? We can fix this by adding a degenerate triangle. A degenerate triangle is triangle with no surface area. Because it has no area it will not be rendered. It is there only to fix our index buffer loop. To create such a triangle, add the last vertex of the first strip and the first vertex of the second strip. (3, 0, 4, 1, 5, 2, 6, 3, 7, 4, 8, 5) 𝑻𝟎 𝑻𝟐 𝑫 𝑫 𝑻𝟒 𝑻𝟔 𝑽𝟐 and 𝑽𝟔 in this particular case. (3, 0, 4, 1, 5, 2, 2, 6, 6, 3, 7, 4, 8, 5) 𝑻𝟏 𝑻𝟑 𝑫 𝑫 𝑻𝟓 𝑻𝟕 Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad These two methods are usually supported (amongst others) by hardware accelerated rasterizers. The method used is defined by the Primitive Topology. The first method is called a TriangleList, while the latter is called a TriangleStrip. What to do: An enum for the Primitive Topology is already defined – a mesh can use one of these. (DataTypes.h, double check, it is possible your version contains a typo... ‘TriangeList TriangleList’) Change your index loop accordingly (not pixel loop!). Considering if it’s an odd or even triangle if using the triangle strip technique. Hint: odd or even? → modulo or bit masking Make two different index buffers to test both techniques. The vertex buffer doesn’t change! Use the buffers defined in the next slide. You’ll also have to update the VertexTransformationFunction. Graphics Programming 1 – Software Rasterization Part II Rasterization: Quad Graphics Programming 1 – Software Rasterization Part II Rasterization: Textures & Vertex Attributes Once you have both primitive topology techniques working, change your vertex struct by adding UV coordinates. The UV space is (typically) defined in [0,1] range. This data is usually provided in the mesh file but for now we add the coordinates manually to the quad. 𝑼 (0, 0) 𝑽𝟎 = (-3, 3, -2) – (0, 0) 𝑽1 = (0, 3, -2) – (.5, 0) 𝑽2 = (3, 3, -2) – (1, 0) 𝑽3 = (-3, 0, -2) – (0,.5) 𝑽 𝑽4 = (0, 0, -2) – (.5,.5) 𝑽𝟓 = (3, 0, -2) – (1,.5) 𝑽6 = (-3, -3, -2) – (0, 1) 𝑽7 = (0, -3, -2) – (.5, 1) 𝑽8 = (3, -3, -2) – (1, 1) (1, 1) Graphics Programming 1 – Software Rasterization Part II Rasterization: Textures & Vertex Attributes Now the quad is ready to be rendered using a texture. There are two things we need: Have a class that reads in a texture file. A way to sample the texture using UV coordinates. Start by reading in a texture file: Compared to the ray tracing framework, we added the necessary libraries to make your framework able to read in.png files. SDL2_image.lib was added to read in images. Complete the Texture Class functions. Texture::LoadFromFile (static) > Loads the image, Creates & Returns a Texture Object Texture::Sample > Samples a specific pixel from the SurfacePixels Graphics Programming 1 – Software Rasterization Part II Rasterization: Textures & Vertex Attributes Texture::LoadFromFile Hint: Once you have the SDL_Surface, you can access it just as the back buffer. See the constructor in Renderer.cpp how to do that. (use m_pSurfacePixel to store the pixel array) Texture::Sample, how to sample the correct pixel You can read a color from an SDL_Surface using SDL_GetRGB(…). One the parameters is a pointer to the pixel you want to read → figure out how to get the index from the UV coordinates. Hint: convert from [0,1] range to [0, 𝑤𝑖𝑑𝑡ℎ/ℎ𝑒𝑖𝑔ℎ𝑡 𝑟𝑎𝑛𝑔𝑒] for U and V. The SDL_Surface holds both the width and height of the texture (w and h). Make sure to use the correct pixel from the array based on your indexing calculation (2D indexing → 1D indexing). Hint: the U and V coordinate can be used for double arrays, but the pixels in the surface are defined in a single array. Find a way to convert from U and V coordinates to a single index → see how we reference a pixel in the back buffer The color you get is in the range [𝟎, 𝟐𝟓𝟓] → Uint8. Remap the color to [0,1] range for further calculations. Remember we multiple each component with 255 in the end when we write to the back buffer. Graphics Programming 1 – Software Rasterization Part II Rasterization: Textures & Vertex Attributes If finished the Texture class, load the given texture in your renderer (for now). When a pixel is inside our triangle, again interpolate the UV coordinates, just as you did with the color attribute. Use the interpolated UV coordinate to sample from the texture. Output the sampled color as your final color. If everything done right, you should get the following result: Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation Graphics Programming 1 – Software Rasterization Part II  Rasterization: Depth Interpolation What is going on? “An affine transformation is a geometric transformation that preserves lines and parallelism, but not necessarily Euclidean distances and angles.” We can see that interpolated points follow an affine transformation since lines and parallelism are preserved. “Perspective transformation is a nonlinear projection where three-dimensional objects are projected on a picture plane. This has the effect that distant objects appear smaller than nearer objects. It also means that parallel lines appear to intersect in the projected image.” Meanwhile the actual vertices follow a perspective transformation as expected. https://en.wikipedia.org/wiki/Affine_transformation https://en.wikipedia.org/wiki/3D_projection#Perspective_projection Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation On the right side we can see the desired result. Notice that parallel lines now intersect in a vanishing point. (Vertical lines are still parallel because the camera is in the xz-plane) 𝑉𝑎𝑛𝑖𝑠ℎ𝑖𝑛𝑔 𝑝𝑜𝑖𝑛𝑡 Why are interpolated points in our current implementation not following the perspective transformation as we would expect? Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation 1 𝑟𝑑 𝑽𝟎 3 Interpolated points are calculated in 2D space after projecting the triangle onto the view plane. 2 𝑷 𝑟𝑑 As we can see the ratio between the line segments has 3 𝑷−𝑽𝟎 𝑷′ −𝑽′𝟎 changed after the perspective divide: ≠. 𝑽𝟏 −𝑽𝟎 𝑽′𝟏 −𝑽′𝟎 But UV space was never perspective transformed, so we 𝑽𝟏 need to use the ratios from before the perspective divide 1 2 to interpolate! < 𝑟𝑑 > 𝑟𝑑 𝑽′ 3 3 𝑽′ This also means our depth buffer is NOT correct because 𝟎 𝟏 we also interpolate to calculate depth values! 𝑷′ Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation An analytical breakdown: 𝑃 − 𝑉0 = 0,4 − −4,2 = 4, 2 𝑉1 − 𝑉0 = 2,5 − −4,2 = 6, 3 ||𝑃 − 𝑉0 || = 𝟎. 𝟔𝟔𝟔 ||𝑉1 − 𝑉0 || 𝑃′ − 𝑉′0 = 0,1 − −2,1 = 2, 𝟎 𝑉′1 − 𝑉′0 = 0.4,1 − −2,1 = 2.4, 𝟎 ||𝑃′ − 𝑉′0 || = 𝟎. 𝟖𝟑𝟑 ||𝑉′1 − 𝑉′0 || Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation As you can see, the ratio is not preserved. If we want to use linear interpolation, we need to make sure we take this into account! This also means our depth buffer is NOT correct (remember we also linearly interpolate the values of the vertex while this is not the correct interpolation after projection: 𝒘′𝟎 ∗ 𝑽′ 𝟎 + 𝒘′ 𝟏 ∗ 𝑽′ 𝟏 + 𝒘′𝟐 ∗ 𝑽′ 𝟐 ! The fix is easy: Px = (-4*0,333 + (2*0,666) 𝑃 − 𝑉0 = 0,4 − −4,2 = 4, 2 =0 𝑉1 − 𝑉0 = 2,5 − −4,2 = 6, 2 Pz = (2*0,333) + (5*0,666) =4 ||𝑃 − 𝑉0 || = 𝟎. 𝟔𝟔𝟔 2 ∗ 0.333 + 5 ∗ 0.666 = 𝟒 ||𝑉1 − 𝑉0 || 𝑃′ − 𝑉′0 = 0,1 − −2,1 = 2, 𝟎 𝑉′1 − 𝑉′0 = 0.4,1 − −2,1 = 2.4, 𝟎 ||𝑃′ − 𝑉′0 || 𝟏 𝟏 = 𝟎. 𝟖𝟑𝟑 2 ∗ 0.167 + 5 ∗ 0.833 = 𝟒. 𝟓 ∗ 0.167 + ∗ 0.833 = 𝟎. 𝟐𝟓 ⇒ 𝟏/(𝟎. 𝟐𝟓) = 𝟒 ||𝑉′1 − 𝑉′0 || 𝟐 𝟓 Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation We calculate the correct interpolated depth by using the following formula (don’t forget to store this value in the depth buffer): 𝟏 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝟏 𝟏 𝟏 𝒘′𝟎 + 𝒘′𝟏 + 𝒘′ 𝑽𝟎,𝒛 𝑽𝟏,𝒛 𝑽𝟐,𝒛 𝟐 Divide each attribute by the original vertex depth and interpolate. Then take the correct depth into account by multiplying with the correct depth. 𝑽𝟎,𝒖𝒗 𝑽𝟏,𝒖𝒗 𝑽𝟐,𝒖𝒗 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝒘′𝟎 + 𝒘′𝟏 + 𝒘′𝟐 ∗ 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 𝑽𝟎,𝒛 𝑽𝟏,𝒛 𝑽𝟐,𝒛 Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation 1 𝑟𝑑 𝑽𝟎 3 We are not going to cover the mathematical proof, but we do verify for our example (in 2D). 𝑷 2 𝑟𝑑 𝟏 3 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝟏 𝟏 𝒘′𝟎 + 𝒘′ 𝑽𝟎,𝒛 𝑽𝟏,𝒛 𝟏 3 𝟏 𝑽𝟏 𝟏𝟓 𝟏 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = = = = 𝟐. 𝟓 1 4 𝟏 𝟒 𝟏 𝟏 𝟒 𝟐 𝟔 ∗ + ∗ + 5 𝟑 𝟓 𝟏. 𝟓 𝟓 𝟏𝟓 𝟏𝟓 𝑽′𝟎 5 𝑽′𝟏 𝑷′ 𝑽𝟎,𝒖𝒗 𝑽𝟏,𝒖𝒗 1.5 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝒘′𝟎 + 𝒘′𝟏 ∗ 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 𝑽𝟎,𝒛 𝑽𝟏,𝒛 𝟎 𝟒 𝟏 𝟏 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = ∗ + ∗ ∗ 𝟐. 𝟓 = 𝟎. 𝟑𝟑 … 𝟑 𝟓 𝟏. 𝟓 𝟓 Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation1 1 𝑟𝑑 𝑡ℎ 𝑽𝟎 3 5 We can also compare the previous (wrong) calculations with what we have now. 𝑷 2 𝑟𝑑 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅_𝒐𝒍𝒅 = 𝒘′𝟎 ∗ 𝑽𝟎,𝒛 + 𝒘′𝟏 ∗ 𝑽𝟏,𝒛 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝟏 3 𝟏 𝒘′ + 𝟏 4 𝑽𝟎,𝒛 𝟎 𝑽𝟏,𝒛 𝒘′𝟏 𝑡ℎ 5 𝟒 𝟏 𝟐𝟒 𝟑 𝟏 𝟏 𝟏𝟓 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅_𝒐𝒍𝒅 = ∗ 𝟑 + ∗ 𝟏. 𝟓 = + = 𝟐. 𝟕 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = = = = 𝟐. 𝟓 𝟓 𝟓 𝟏𝟎 𝟏𝟎 𝟏 𝟒 𝟏 𝟏 𝟒 𝟐 𝟔 ∗ + ∗ 𝟑 𝟓 𝟑 𝟓 𝟏𝟓 𝟏𝟓 + 𝑽𝟏 𝟐 3 1 4 ′ 𝑽𝟎,𝒖𝒗 𝑽𝟏,𝒖𝒗 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅_𝒘𝒓𝒐𝒏𝒈 = 𝑽𝟎,𝒖𝒗 ∗ 𝒘 + 𝑽𝟏,𝒖𝒗 ∗ 𝒘′𝟏 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝒘′𝟎 + 𝒘′𝟏 ∗ 𝒁𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 𝟎 𝑽𝟎,𝒛 𝑽𝟏,𝒛 𝑽′𝟎 5 5 𝑽′𝟏 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅_𝒘𝒓𝒐𝒏𝒈 = 𝟎 ∗ 𝟒 +𝟏∗ 𝟏 = 𝟏 𝑼𝑽𝒊𝒏𝒕𝒆𝒓𝒑𝒐𝒍𝒂𝒕𝒆𝒅 = 𝟎 𝟒 𝟏 𝟏 ∗ + ∗ ∗ 𝟐. 𝟓 = 𝟎. 𝟑𝟑 … 𝑷′ 𝟓 𝟓 𝟓 𝟑 𝟓 𝟑 𝟓 1,5 𝟐 Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation This needs to happen for every attributes we interpolate (color, uv, normal, etc.)! Do not forget to use the correct depth! Solving this issue creates an interesting side effect: The depth buffer is no longer linear! Vector3 position: x = screen space (x-coordinate of pixel) y = screen space (y-coordinate of pixel) z = interpolated camera space z-coordinate. https://developer.nvidia.com/content/depth-precision-visualized Graphics Programming 1 – Software Rasterization Part II Rasterization: Depth Interpolation Graphics Programming 1 – Software Rasterization Part II Rasterization : What to do? Add the option to rendering using two different primitive topologies: TriangleList TriangleStrip Implement the texture class, which makes you able to load an.png and sample from that texture using UV coordinates. Use correct depth interpolation, for every interpolated attribute! Fix the black lines between your triangles Graphics Programming 1 – Software Rasterization Part II GOOD LUCK! Graphics Programming 1 – Software Rasterization Part II