GP1_W10_DirectX_Pipeline PDF

GRAPHICS PROGRAMMING I HARDWARE RASTERIZATION PART I Graphics Programming 1 – Hardware Rasterization Part I DirectX: Introduction Previously we implemented a software rasterizer that runs on the CPU. Just as with the ray tracer, many calculations could be performed in parallel. If...

GRAPHICS PROGRAMMING I HARDWARE RASTERIZATION PART I Graphics Programming 1 – Hardware Rasterization Part I DirectX: Introduction Previously we implemented a software rasterizer that runs on the CPU. Just as with the ray tracer, many calculations could be performed in parallel. If only we had a lot of threads, that would accelerate calculations immensely. Hardware with many threads exists… Graphics Processing Units (GPUs)! For example: Nvidia 1080Ti has 28 Streaming Multiprocessors (SM). Each SM can have a maximum of 2048 threads in flight. Hence, 28*2048 = 57,344 concurrent threads running (max) ☺ Graphics Programming 1 – Hardware Rasterization Part I DirectX: Introduction How to communicate with the GPU? DirectX, OpenGl, Vulkan, Metal, LibGNM, etc. → 3D Rendering API’s CUDA, OpenCl, DirectCompute, etc. → GPGPU API’s They are Application Programming Interfaces (APIs) that simplify programming by hiding “nasty” hardware-software interfacing. USER-MODE KERNEL-MODE APPLICATION RENDER API HARDWARE DRIVER DRIVER Hardware interfacing is interesting, but very low-level! It requires thorough understanding of application programming interfaces, operating systems, drivers and hardware. https://nouveau.freedesktop.org/wiki/IntroductoryCourse/ https://github.com/torvalds/linux/tree/master/drivers/gpu/drm/nouveau Graphics Programming 1 – Hardware Rasterization Part I DirectX: Introduction We are going to use DirectX, which is only supported on Windows, but its concepts are similar for other APIs. DirectX is a collection of APIs for handling various multimedia tasks: DXGI – Manage low-level tasks including enumeration of hardware devices, presenting rendered frames to an output, controlling gamma and managing a full-screen transition. Direct2D – Drawing 2D Graphics Direct3D – Drawing 3D Graphics DirectXMath – Math Library supporting SIMD DirectCompute – GPGPU Computing DirectWrite – Text Rendering DirectX evolved over time and some of its APIs have since been deprecated: DirectInput – Input interface (alternative -> XInput) DirectSound – Audio (alternative -> FMOD) 1. https://alain.xyz/blog/comparison-of-modern-graphics-apis 2. https://docs.microsoft.com/en-us/windows/win32/directx 3. https://docs.microsoft.com/en-us/windows/win32/direct3ddxgi/d3d10-graphics-programming-guide-dxgi Graphics Programming 1 – Hardware Rasterization Part I DirectX: Introduction We are going to use DirectX 11 specifically. Why not use the more recent DirectX 12? Companies are the target audience for Microsoft. These companies tend to implement features themselves and want full control. Since DirectX 12 its users have more control, which also means you must implement your own command lists and buffers, resource management, etc. While this reduces the driver overhead and allows for more efficient resource utilization, it also results in more code, and requires increased understanding of hardware. Many DirectX API functionalities are discarded in the industry, preferring in-house implementations for: Math functionality Render state encapsulation through effects 3D model data … Luckily for us, there is a dedicated community that keeps ‘deprecated’ tools/API’s alive but be aware it may not be available when employed in the industry! We chose DirectX 11 because (on a beginner's level) it is not too difficult after programming your own software rasterizer. Many of its concepts are identical, but you will have to familiarize yourself with its API conventions, learn which function calls, are needed, use typical C++ techniques and learn how to handle data and resources. Time to explore the wonderful world of Graphics APIs ☺ Graphics Programming 1 – Hardware Rasterization Part I DirectX: Introduction All modern rendering APIs have a programmable rendering pipeline. Before we can render using DirectX 11, the final part of that pipeline must be initialized. We start with this initialization process and cover the full rendering pipeline afterwards. Time to browse the new starter project: Necessary dependencies: d3d11.lib → DirectX 11 API d3dcompiler → DirectX 11 shader compiler dxgi.lib → DXGI API dx11effects.lib → External effect framework, included with starter project Added the necessary includes linked to the libraries defined above. Added the DirectXEffect framework. This is not part of the official DirectX11 SDK, but it will prove very useful. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization We start by creating a private function in the renderer called InitializeDirectX() which is called once in the renderer’s constructor. DirectX 11 has two very important entities that need to be created when initializing: Device → ID3D11Device* : is responsible for creating resources (e.g. geometry, textures, etc.) on a display adapter (=GPU), as well as enumerating its capabilities. Device Context → ID3D11DeviceContext* : it contains the configuration of the rendering pipeline and generates rendering commands using the resources owned by a device. A device context is NOT thread safe! In game rendering usually one device and one context are sufficient. We can create a device and device context with the following code: 1. https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-devices-intro Graphics Programming 1 – Hardware Rasterization Part I  DirectX: Initialization When working with DirectX, it is recommended to make extensive use of MSDN and its documentation! When looking into a function: Pay attention to its parameters (input vs output). Read its parameter description, these often link to other pages further explaining concepts. After calling a function, check its result: HRESULT. Use search engines or large language models (but think critically)! ☺ Hone your problem-solving skills, there is plenty of educative material on the web. 1. https://docs.microsoft.com/en-us/windows/win32/seccrypto/common-hresult-values Graphics Programming 1 – Hardware Rasterization Part I  DirectX: Initialization Once we have a Device and Device Context, we need to create a Swap Chain. A swap chain contains at least two buffers: A screen or front buffer whose content is the output to be displayed. And one or multiple back buffers where renders are stored until they are ready to be swapped with the front buffer to be displayed. Why bother with multiple buffers? If we were to write directly to the screen (to a buffer that is not locked), we could introduce screen tearing as the buffer may be overwritten while the monitor is refreshing it display. To setup the swap chain, we need a DXGIFactory1. We use DXGI because: It has better performance and saves memory because it creates a swap chain suited to the GPU hardware. It also has access to some of the device functionality which further improves performance. 1. https://docs.microsoft.com/en-us/windows/win32/direct3d9/what-is-a-swap-chain- Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization With the DGXIFactory a swap chain can be created. In DirectX 11 you let the device create all the resources. To determine which resource is being created and with which settings, a matching descriptor needs to be filled in. Do not forget to initialize with { } first! 1. https://docs.microsoft.com/en-us/windows/win32/api/dxgi/ns-dxgi-dxgi_swap_chain_desc 2. https://docs.microsoft.com/en-us/windows/win32/direct3d10/d3d10-graphics-programming-guide-resources-data-conversion Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization When calling CreateSwapChain, a handle (HWND) to the window needs to be passed on. This handle comes from the OS and, in our case, is owned by SDL, which controls the window. Using the following code, we can get the window’s handle and create the swap chain. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization We also need a Depth Buffer to solve the visibility problem. This is another resource that needs to be created via the device, using a matching descriptor. Direct3D combines depth and stencil buffer. The latter is used to mask pixels in an image. This mask controls whether a pixel is drawn (before Z test), allowing to draw to the Render Target on a pixel-by-pixel basis. For now, we only use the depth buffer. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization The previous descriptor described how to create the resource for the depth-stencil buffer. Now we also need to describe the resource view. After which we create both the resource and matching resource view. Graphics Programming 1 – Hardware Rasterization Part I  DirectX: Initialization What is a Render Target? A render target creates resources for drawing and performs drawing operations. In DirectX, a render target consists of two important components: Render Target Buffer → ID3D11Resource Render Target View → ID3D11RenderTargetView DirectX has its own conventions on handling resources: ID3D11Resource: data that can be shared by multiple pipeline stages. Resource Views: determines how a resource is used (bound) in the pipeline ID3D11DepthStencilView: access the resource/texture during the depth-stencil testing. ID3D11RenderTargetView: access the resource/texture that is used as a render target. ID3D11ShaderResourceView: access the resource/texture as constant buffer, texture buffer, texture or sampler during rendering. ID3D11UnorderedAccessView: access an unordered resource using a pixel shader or a compute shader (no multisampling though). 1. https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-resources-intro?redirectedfrom=MSDN Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization Now that we have a depth buffer and back buffer (in our case, the render target), we must bind them to become the buffers used when rendering. Binding happens through resource views: we already have a view for the depth buffer, but not yet for the back buffer. We can retrieve the back buffer resource from the swap chain, and using this pointer, create a resource view for it. Using the two views, bind them as the active buffers during the Output Merger Stage. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization Finally, the last step is setting the viewport. The viewport defines where the content of the back buffer will be rendered on the screen. DirectX will use this viewport to transform the NDC to matching screen space coordinates. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization We have finished setting up the most basic DirectX pipeline. Now we can use the pipeline to render to the screen: 1. Clear buffers every time a new frame is to be rendered (clear value is different for each buffer!) 2. Render a new frame 3. Present the contents of the back buffer to the screen = swapping back with front buffer. These steps happen in the Render function of the renderer. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization Graphics Programming 1 – Hardware Rasterization Part I DirectX: Initialization We introduced resource leaks . This is just as bad as memory leaks! Whenever we create a resource via the device, we must release it once we are finished using that resource. Resources are released in reversed order (compared to creation order)! Some resources, such as the device context, require a few extra steps to release. We introduced 7 resource leaks to fix them! There is a hidden leak of the DXGI factory, fix that one too. Render Target View Render Target Buffer Depth Stencil View Depth Stencil Buffer Swap Chain Device Context Device DXGIFactory Graphics Programming 1 – Hardware Rasterization Part I Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Now that we have our DirectX 11 up and running, let’s have a look at what we call the Programmable Render Pipeline. You need to know this by heart!! ☺ Are you ready…? Sure…? Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Input Layout Vertices Indices Primitive Type Input Assembler (IA) Vertex Shader Input Assembler Stage Reads geometric data (vertices and indices) from memory. There are different primitive types e.g., triangle list and triangle strip. Based on the setting it will assemble all the data as the correct type. We cover over all four parts that define the assembly. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Input Layout Vertices Indices Primitive Type Input Assembler (IA) Vertex Shader Vertices: This is basically our vertex buffer. In DirectX you can create different types of buffers: Immutable Buffer: fixed data, filled once. GPU can only read it and CPU has no access at all. Dynamic Buffer: can be modified at runtime. Is accessible by both the GPU and CPU. Staging Buffer: resource that supports data transfer from the GPU to the CPU. What a vertex holds is up to us to decide. Like the software rasterizer, only a position is mandatory! Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Input Layout Vertices Indices Primitive Type Input Assembler (IA) Vertex Shader Input Layout: This describes the layout of the vertex you define. This is necessary so the GPU knows which data is available and how it should traverse the data buffer! When defining the layout pay extra attention to: Format ByteOffset Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Input Layout Vertices Indices Primitive Type Input Assembler (IA) Vertex Shader Indices: No changes compared to the software rasterizer. It defines the sequence of the vertices that make up the 3D model. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Input Layout Vertices Indices Primitive Type Input Assembler (IA) Vertex Shader Primitive Type: Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Input Assembler (IA) Vertex Shader Geometry Shader Vertex Shader: This is equivalent to the VertexTransformation function from the software rasterizer. Except it does not perform the perspective divide which now happens in the Rasterizer Stage. Its main purpose is to transform a vertex from model space to projection space (before the divide). A shader is a function that is executed on the GPU in parallel! In DirectX we define a shader in a.hlsl (High Level Shader Language) file. Vertex Shaders are also used for: Per Vertex Lighting Displacement Mapping → Snow Deformation Skinning (Animation) … Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Vertex Shader Geometry Shader Stream Rasterizer Output States Rasterizer Viewport Geometry Shader: This is an optional shader stage. We won’t use it this semester. You can use it for adding or removing geometry on the GPU. There are other optional stages like this, for example, tessellation stage. But we won’t pay attention to them for now as well. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Vertex Shader Geometry Shader Stream Rasterizer Output States Rasterizer Viewport Pixel Shader Rasterizer: This does what you’ve been doing the last couple of weeks. ☺ So, it does: Culling Clipping Homogenous/Perspective Divide Viewport Transformation Fragment Generation Attribute Interpolation Graphics Programming 1 – Hardware Rasterization Part I  DirectX: Graphics Pipeline Rasterizer Pixel Shader - Output Merger (OM) – Depth Test & Blending Pixel Shader: This is executed after the rasterizer and does what the PixelShading function does. Just as with the vertex shader, it is a function written in.hlsl that is executed on the GPU. For your information, internally the pixel shader is executed on 2x2 pixel tiles. This means these 4 pixels are calculated in parallel. This gives us some extra information (derivative quantities) for other techniques. Pixel can be clipped (discarded) by the HLSL clip function. Improves performance in some cases. Pixel can be occluded by another pixel fragment by the Depth Test in the Output Merger Stage. 1. https://docs.microsoft.com/en-us/windows/win32/direct3d11/pixel-shader-stage Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Depth-Stencil Pixel Shader Buffer - Output Merger (OM) – Depth-Stencil View Depth Test & Blending Rendertarget View Output Merger: Performs the Depth and Stencil Test after the pixel shader. Does this mean we perform unnecessary calculations in the pixel shader? No, there are hardware optimizations that perform early depth testing, etc. Not to worry! It can also use blending when we want to use transparency. More on that later. In the end it writes the result to the back buffer. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Graphics Programming 1 – Hardware Rasterization Part I  DirectX: Graphics Pipeline 𝐈𝐧𝐝𝐞𝐱 𝐕𝐞𝐫𝐭𝐞𝐱 𝐁𝐮𝐟𝐟𝐞𝐫 𝐁𝐮𝐟𝐟𝐞𝐫 Input Vertex Hull Domain Geometry Pixel Tesselator Rasterizer Assembler Shader Shader Shader Shader Shader workgroup workgroup Custom 𝐂𝐮𝐬𝐭𝐨𝐦 Amplication/Task Mesh Pixel Mesh Rasterizer 𝐃𝐚𝐭𝐚 Shader Shader Shader Generation OPTIONAL Graphics Programming 1 – Hardware Rasterization Part I DirectX: Graphics Pipeline Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Let’s start rendering! We need three things: Shader performing the Vertex Transformation and Pixel Shading. Create an effect class that represents our shader. We need this to create our input layout. Mesh representation, in other words: data buffers. We start by creating the shader… We use the Effects Framework, which means we are going to create.fx files instead of.hlsl. They are still written in HLSL, but they have some extra features which are going to prove useful later on. Create an.fx file called PosCol3D.fx, put it in a resources folder (just like the meshes and textures) and definitely DON’T add it to Visual Studio! Why? You don’t want to compile this file using the VS compiler but use the DirectX Shader Compiler (fxc.exe) instead! Open the file in your preferred text editor ☺ Graphics Programming 1 – Hardware Rasterization Part I  DirectX: Rendering We start by defining what a vertex looks like. Just like in C++, create a struct that matches the layout. It’s important that the vertex struct in C++ matches with the one in HLSL! Semantics can be used to reference GPU variables. Some are required though: SV_POSITION: contains the position of the vertex after the transformation and is required in the rasterizer. SV_TARGET: used to reference to which render target you want to render. You can bind multiple! 1. https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-semantics Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Once we have the structs defined, create our shader functions. Don’t forget to forward data from one stage to another. So you don’t you lose the information! Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Finally, because we are using the Effect Framework, we must define a technique. The technique is the “program” because it defines which shader functions to use for which stage. A technique can have multiple passes that run in a sequential order. Shader: function for a particular pipeline stage. Program: collection of shaders defining all stages of our pipeline. → this concept is deprecated Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Now we are ready to compile and load this effect file into memory! Create an effect class and add the following function that will compile and load the effect. In the constructor of the effect class, accept the pointer to the ID3D11Device and a path that determines which effect to load. In the constructor, load the effect using the function defined to the right, store the resulting pointer in a member of type ID3DX11Effect. Also get the technique and store it in a data member. We’ll need this later for the Input Layout. Write two getter functions that gives you access to the two data members. Don’t forget to release your resources in the destructor! Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering With this up and running, we can now create a 3D mesh representation. Create a class for the mesh representation. Also define a Vertex struct that has the same layout as the one defined in the shader. Make sure the constructor accepts the ID3D11Device*, a container that holds the raw vertex data and one that hold the raw index data, like your software rasterizer. In the constructor: Create an instance of the effect class you just created. This could be optimized, but that’s engine design. Don’t worry too much for now! Create the vertex layout using, again, a matching descriptor. Through the technique of the effect, create the input layout, using the vertex layout descriptor. Create the DirectX Vertex Buffer and Index Buffer, using the device and a descriptor ☺ Don’t forget to release the resources you’ve created through the device in the destructor. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Last thing we need to do is actual rendering ☺ Whenever you want to render you have to set all the correct data in the device context, defining the state of the render pipeline, and call “render”. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Put the code from the previous slide in a Render function, create an instance of the primitive and call render before the Present in the renderer. Which data should I use for the vertex buffer? Remember DirectX uses a left-handed coordinate system. (Winding Order...) If you’ve done everything correctly, you should get the following awesome result. Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering Graphics Programming 1 – Hardware Rasterization Part I DirectX: Rendering So, after this class you should really understand: Graphics Pipeline & (basic) Shaders Device vs Device Context Descriptors Resource vs Resource Views But what if you don’t get this result…. ☺ Graphics Programming 1 – Hardware Rasterization Part I VS Graphics Debugger DEMO GOOD LUCK! Graphics Programming 1 – Hardware Rasterization Part I