W02 C++ Libraries PDF

Week 02 – C++ Libraries Game Development / Software Engineering / 2024-2025 What is a library? Collection of precompiled code that can be reused in programs. Purpose: Reusability Modularity Adaptability In C++, there are three types of libraries: Stati...

Week 02 – C++ Libraries Game Development / Software Engineering / 2024-2025 What is a library? Collection of precompiled code that can be reused in programs. Purpose: Reusability Modularity Adaptability In C++, there are three types of libraries: Static (.lib,.a) Dynamic (.dll,.so) Header-only Game Development / Software Engineering / 2024-2025 What is a library? Static library: How does it work: code becomes part of executable during linking process. Advantages: Faster execution. No dependency on external files at runtime. Disadvantages: Larger executable file. Less flexibility for updates. Game Development / Software Engineering / 2024-2025 What is a library? Dynamic library: How does it work: code from library gets loaded at runtime. Advantages: Smaller executable. Library code shared among multiple processes/programs. Ability to update library code without need to recompile executable. → this is only true if function signatures and memory layouts didn’t change! Disadvantages: Dependency on external files (.dll file itself). Potential version conflicts. Game Development / Software Engineering / 2024-2025 What is a library? Header-only library: How does it work: code entirely defined in a header file, so no separate linking process required. Advantages: Easy to distribute. No additional linking required. Disadvantages: Potentially longer compiles times. Potential code bloat. No flexibility to update → executable needs to be recompiled! Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s start with making a static library in Visual Studio. First create an ‘blank solution’ (not ‘empty project’) in your desired folder. Game Development / Software Engineering / 2024-2025 Static library - Hands-on Now add the well-known console application to the solution. Make sure to add it to the current solution. Game Development / Software Engineering / 2024-2025 Static library - Hands-on Now let’s create a static library in the current solution. Game Development / Software Engineering / 2024-2025 Static library - Hands-on A lot of clutter was created. Let’s clean this up. Remove comments in main.cpp (Program) Remove all files (.h and.cpp) in the library (Library). We will add our own files. By default, in the library, Precompiled Headers (.pch) might be enabled (also see: stdafx). Precompiled Headers: Good to use for a large codebase where the header doesn’t change often. This reduces compilation times. If codebase has multiple ‘modules’ where common data or functionality, that doesn’t change often, can be shared between ‘modules’. Might require a manual recompilation when changed. Disable the use of precompiled headers for all configurations! Because we don’t care about Win32 configurations, we ideally also remove the x86 and Win32 configuration in the configuration manager. Configurations specify how projects are build and deployed. https://learn.microsoft.com/en-us/visualstudio/ide/understanding-build-configurations?view=vs-2022 Game Development / Software Engineering / 2024-2025 Static library - Hands-on Game Development / Software Engineering / 2024-2025 Static library - Hands-on Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s add a header file to our library called ‘Library.h’ and add a simple addition function. Build and have a look at your output directory. Our.lib is not being created even though the output is fine. Why? Game Development / Software Engineering / 2024-2025 Static library - Hands-on There is no implementation (.cpp) file so there is no code to compile inside this project. Move the implementation to a.cpp file and compile. Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s link the library to our program so we can start using the functionality. Set include directory for access to our header. Preferred a relative path from our Program folder. Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s link the library to our program so we can start using the functionality. Set library directory (same output directory) and specific library file by name. Certain Macros are provided for ease of use: https://learn.microsoft.com/en- us/cpp/build/reference/common-macros-for-build-commands-and-properties?view=msvc-170 Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s use the function in our main function of the Program. Game Development / Software Engineering / 2024-2025 Static library - Hands-on We don’t need Visual Studio’s interface in order to compile both the Program and the Library, and to link them. Let’s do this manually to understand what VS does for us. We can invoke the compiler command-line (CL), which uses the following syntax: https://learn.microsoft.com/en-us/cpp/build/reference/compiler-command-line-syntax?view=msvc-170 As you’ve seen in previous class, there are a lot of compiler options one can set. We won’t go over all options. You can find all options here: https://learn.microsoft.com/en- us/cpp/build/reference/compiler-options-listed-alphabetically?view=msvc-170 Let’s start by opening the compiler CL through Visual Studio, or by going to your VS install directory. E.g.: ‘C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.38.33130\bin\Hostx64\x64’ Game Development / Software Engineering / 2024-2025 Static library - Hands-on Game Development / Software Engineering / 2024-2025 Static library - Hands-on Make a new temporary folder for our output called ‘ManualBuild’ and put the cmd path to this folder using the command ‘cd’. Let’s start with compiling the library, as our program is dependent on the library, using the following commands: cl /c./../Library/Library.cpp lib /out:Library.lib Library.obj Game Development / Software Engineering / 2024-2025 Static library - Hands-on Can we already verify if the.lib file contains what we have/need? Yes, we can use dumpbin to display information about our library. You can find all options here: https://learn.microsoft.com/en-us/cpp/build/reference/dumpbin-options?view=msvc-170 Let’s output the symbols of our library using the following command: dumpbin /symbols Library.lib Game Development / Software Engineering / 2024-2025 Static library - Hands-on What does this mean? This is an actual of a function symbol. This will be used by the linker to resolve our references. Symbols are created based on a mangling (decoration) scheme. This depends on both the compiler and the Application Binary Interface (ABI). Some references: https://en.wikiversity.org/wiki/Visual_C%2B%2B_name_mangling https://github.com/gchatelet/gcc_cpp_mangling_documentation http://www.kegel.com/mangle.html https://itanium-cxx-abi.github.io/cxx-abi/abi.html Game Development / Software Engineering / 2024-2025 Static library - Hands-on What does this mean? For fun, let’s analyze our symbol: ?GetSum: function called GetSum (this can change based on your setup. For example when using a namespace we would get: ?GetSum@MyNamespace) @@: end of (function) name Y: Type Code → ‘Y’ == Unqualified FunctionCodeType (e.g. global function) A: Calling Convention → ‘A’ == __cdecl III: return type + input parameters → ‘I’ == unsigned int @: end of argument list Z: end of signature Game Development / Software Engineering / 2024-2025 Static library - Hands-on What are calling conventions? Calling conventions describe the interface of called code. It specifies the method that a compiler sets up to access a subroutine. So, what does it describe? Where parameters, return values and return addresses are placed (registers vs stack vs mix). In case we use registers, which registers must be reserved for the caller (known as: callee-saved registers). The order parameters are passed. How the task of setting up for and cleaning after a function call is divided between the caller and the callee. Where the previous frame pointer is stored. … Reference: https://learn.microsoft.com/nl-nl/cpp/cpp/calling-conventions?view=msvc- 170&viewFallbackFrom=vs-2017 Game Development / Software Engineering / 2024-2025  Static library - Hands-on The following calling conventions are supported by the Visual C/C++ compiler: Keyword Stack cleanup Parameter passing __cdecl caller Pushes parameters on the stack, in reverse order (right to left) __clrcall n/a Load parameters onto CLR expression stack in order (left to right) __stdcall callee Pushes parameters on the stack, in reverse order (right to left) __fastcall callee Stored in registers, then pushed on stack __thiscall callee Pushed on stack, this pointer stored in ECX __vectorcall callee Stored in registers, then pushed on stack in reverse order (right to left) By default, the Visual C/C++ compiler uses __cdecl. Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s manually build our program and link our library, in one command. cl./../Program/Program.cpp Library.lib /I./../Library Here we specify: Our library to use ‘Library.lib’ The include directory (/I) to our library files. We can also do it in two steps again: cl /c./../Program/Program.cpp /I./../Library link Program.obj Library.lib Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s do the first command again, but without specifying the.lib file: cl./../Program/Program.cpp /I./../Library Game Development / Software Engineering / 2024-2025 Static library - Hands-on Let’s do the first command again, but without specifying the include file now: cl./../Program/Program.cpp Library.lib Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on We will now switch to the next type of library, the dynamic library. Create a similar setup as before with Visual Studio: Blank Solution → remove x86 and Win32 configuration from configuration manager. Console Application → remove comments, a clean main function is preferred. Dynamic Library → remove all files, add blank Library.h and Library.cpp files, disable precompiled headers. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on As with the previous project, set for Debug configuration the: Additional Include Directories Additional Library Directories Additional Dependencies Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on When we build, there are issues: You might run into an issue where the Program is trying to compile before the Library. Though our Library depends on the Program. This is also true for static libraries. In Visual Studio, we can set the dependencies between projects. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on When we build, there are issues: No.lib was produced, only an.dll file. Why? When creating dynamic libraries in C++, we need to ‘flag’ everything we want to export. This can be functions, classes, structs, etc. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on We can export functions by adding the following class-storage attribute: __declspec(dllexport) More information can be found here: https://learn.microsoft.com/en-us/cpp/cpp/dllexport-dllimport?view=msvc-170 It is also possible to import when using a dynamic library. A good practice is to define a macro, for example ‘LIBRARY_API’, which based on a preprocessor define, either uses import or export. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on It is also possible to import when using a dynamic library. A good practice is to define a macro, for example ‘LIBRARY_API’, which based on a preprocessor define, either uses import or export. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Wait, did we just create and use a.lib file when creating a dynamic library? Yes ☺ When building a dynamic library, we get the following files:.dll file → contains the actual code and data for the exported functions and variables..lib file → contains information and references to exported symbols of the.dll..exp file → contains exported symbols. It helps the linker manage the export table (a lookup table in the.dll that points to addresses of exported functions). The.lib file in this situation is called an import library. It supplies the system with the information to load the.dll and locate the exported.dll functions when the program loads. In our project, we linked the.lib file, thus asking the linker to link our Library to our Program. This is called load-time dynamic linking. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on As mentioned earlier, we can also export an entire class. Inspect the symbols using: dumpbin /symbols Library.exp Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Constructor Move operator Copy operator Static member Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Another way to use a dynamic library is through run-time dynamic linking. Before showing you the technique, we must discuss function pointers. In C++, you have used pointers to point to memory of classes and/or POD types. You can also point to functions using function pointers! For testing, let’s define both a global function and a simple class with a similar class function which is const correct. We know we can just use functions as follows: Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Because a function resides in memory, just as with regular variables, we can have a pointer pointing to the address of the function. We define a function pointer as follows: return_type (*FunctionPointerName) (parameter type, …) For our example, that would be: Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on A common approach, for ease of use, is to define function pointers as ‘types’ using either typedef or using. These can be part of the main function, or any function, but often they are either global, part of a namespace or of a class. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on As you might notice, always having to provide the class instance is annoying. While function pointers do not allow storing the instance itself, the standard library has a container called std::function which allows you to do so using std::bind. More detailed information can be found here: https://en.cppreference.com/w/cpp/utility/functional/function Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Now that we know this, let us start using run-time dynamic linking instead. First, remove the interface library (.lib) linking in our Program properties, including the library directory. We keep the include directories and the build dependencies. Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Since we don’t link to the interface library, we will have to load the dynamic library ourselves and get a pointer to the function from the library. For now, define the function pointer type and change your main function as follows: Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on How do we load our dynamic library? We need to use OS specific functions and datatypes for this! On Windows we can use the functionality exposed via the libloaderapi.h header. Functions we are interested in for now: LoadLibraryEx → Loads specified module into the address space of the calling process. FreeLibrary → Frees loaded dynamic-link library module and, if necessary, decrements its reference count. GetProcAddress → Retrieves the address of an exported function or variable from the specified dynamic-link library. Link: https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/ Types we will encounter: HMODULE → Handle to a module. This is the base address of the module in memory. If not 16-bit Windows, HMODULE and HINSTANCE are the same thing. FARPROC → Pointer type but we can cast directly to our wanted type. Link: https://learn.microsoft.com/en-us/windows/win32/winprog/windows-data-types Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Each process (.exe) has its own address space. When a process loads a.dll, the functions of the.dll gets mapped into the calling process its address space at a certain base address. The OS ensures the addresses are resolved. The code segment of a.dll is shared across all processes that use it, but each process gets its own copy of the data segment. This ensures that different processes can call the same functions from the.dll, but while maintaining its own independent data. For each process using the.dll, the OS keeps track of how many processes use it (aka reference counting). When all processes free the loaded library, the OS will release the library. More info about OS address spaces and memory management: https://learn.microsoft.com/en-us/windows-hardware/drivers/gettingstarted/virtual-address-spaces https://learn.microsoft.com/en-us/windows/win32/memory/virtual-address-space Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on The shared code segment of a dynamic library is powerful. It also allows us to change the functionality of functions by just replacing the.dll without the needs to recompile the application. Be aware, if you change the memory layout of an exported struct or class, or you change the signature of an exported function, this might cause undefined behavior in your application! In this scenario you will have to make sure your executable is changed accordingly. Let’s change our code and load our library, find the appropriate function, use it and free the library. Hint: for the loading/unloading of our library, use the RAII (Resource Acquisition Is Initialization) paradigm to ensure we don’t forget to release reference counts! Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Our library loads fine, but our function cannot be found… Why? The GetSum(…) function is a global function, where the implementation is defined in another translation unit. In C++, when we want to export it, we need to flag it as extern. More information available here: https://learn.microsoft.com/en-us/cpp/cpp/extern- cpp?view=msvc-170 Even though we use extern, we still cannot find it… Why? Name mangeling! Let’s true to use the output we find through dumpbin. dumpbin /exports Library.dll Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on While this works, it is annoying to always keep both the function pointer and the symbol name in sync whenever we change the function signature. We can make this better by using: C-style name mangling using: Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on While this works, it is annoying to always keep both the function pointer and the symbol name in sync whenever we change the function signature. We can make this better by using: Module-Definition File (.def): Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on While this works, it is annoying to always keep both the function pointer and the symbol name in sync whenever we change the function signature. We can make this better by using: Module-Definition File (.def): Info: https://learn.microsoft.com/en-us/cpp/build/reference/module-definition-dot-def-files?view=msvc-170 Game Development / Software Engineering / 2024-2025 Dynamic library - Interoperability We can even use our C++ library from inside C# using interoperability! Game Development / Software Engineering / 2024-2025 Dynamic library - Interoperability Be aware: Calling conventions!! When we use C-style mangling, we rely on C, meaning we can’t export classes and other C++ specific features! Wrapper functions – e.g. constructors/destructors. Specialization – e.g. templates. Certain languages, like C#, don’t support (raw) pointers. There might be equivalents but be careful using them. For example, in C# we have IntPtr: https://learn.microsoft.com/en-us/dotnet/api/system.intptr?view=net-8.0 Data types might have different sizes or might not be supported. For example, a bool in C++ is 1 byte and in C# 4 bytes. Memory layouts of data structures. Game Development / Software Engineering / 2024-2025 Dynamic library - Interoperability Game Development / Software Engineering / 2024-2025 Dynamic library - Interoperability Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Last dynamic library specific method to discuss is the DllMain entry point. An optional entry point into a dynamic library. When the system starts or terminates a process or thread, it calls the entry-point function for each loaded dynamic library using the first thread of the process. The system also calls the entry-point function for a dynamic library, if provided, when it is loaded or unloaded using the LoadLibrary and FreeLibrary functions. More info: https://learn.microsoft.com/en-us/windows/win32/dlls/dllmain Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Game Development / Software Engineering / 2024-2025 Dynamic library - Hands-on Last dynamic library specific method to discuss is the DllMain entry point. An optional entry point into a dynamic library. When the system starts or terminates a process or thread, it calls the entry-point function for each loaded dynamic library using the first thread of the process. The system also calls the entry-point function for a dynamic library, if provided, when it is loaded or unloaded using the LoadLibrary and FreeLibrary functions. More info: https://learn.microsoft.com/en-us/windows/win32/dlls/dllmain Honorable mention, dynamic libraries support delayed loading, if specified with the linker option, though it has certain limitations. While it is used in game engines sometimes, it add extra complexity and often developers opt to manually load the library themselves for full control (as we did). But being aware it exists is important: https://learn.microsoft.com/en- us/cpp/build/reference/linker-support-for-delay-loaded-dlls?view=msvc-170 Game Development / Software Engineering / 2024-2025 Dynamic library - Shared Memory All data and pointers are process specific. This is also the case when loading an “instance” of a dynamic library. It is possible to create shared memory between processes and libraries. Shared memory with dynamic libraries in particular can be very powerful, though one must be cautious when doing so! When sharing memory, be aware of: Reading/Writing memory locations. Synchronization problems. Security considerations. Namespaces and memory scope (Global\ vs Local\) → important difference between.exe and.dll. References: https://learn.microsoft.com/en-us/windows/win32/memory/creating-named-shared-memory https://learn.microsoft.com/en-us/windows/win32/dlls/using-shared-memory-in-a-dynamic-link-library Game Development / Software Engineering / 2024-2025 Dynamic library - Shared Memory You can create a file-mapped object that will be shared among other processes using: CreateFileMappingA Important, the first process that load your.dll will create the object. Subsequent processes will open a handle to the same object when calling the function. This is reference counted, thus when the last application that closes the object will also destroy the object. In other words, this is not persistent memory after all processes are done with the object! The function has two variables for MaximumSize. We use the low-order DWORD for memory blobs that are smaller than 4 Gigabytes. Once we have the object, we need to map a view into the virtual address space using: MapViewOfFile All process that use this object will write to the same memory! Do not forget to unmap the view and close the object once you are done with it! When using a dynamic library, this mean every time you unload the library that uses a file-mapped object, it must unmap the view and close the object. Use functions: UnmapViewOfFile CloseHandle Game Development / Software Engineering / 2024-2025 Exercise - Shared Memory On Leho you can fine the assignment called ‘Exercise_LocalMultiplayerDemo_STUDENTS’. This is a non-graded exercise. See live demo for the wanted behavior. To do: Setup the MultiplayerFeature (.dll) so the LocalGame (.exe) can and will use it. You don’t need to change anything in the LocalGame. Finish the functionality of the MultiplayerFeature. You don’t need to change or add anything to the header, except code that might be required for exporting symbols. Optional: add the different ways of loading the library. You can have two projects as you don’t know how to use Cmake yet. Why: Practice dynamic library setup and usage. Practice coding skills ☺ Practice (shared) memory management. Game Development / Software Engineering / 2024-2025