FaberSanZ / The-Forge

The Forge Cross-Platform Rendering Framework PC, Linux, Ray Tracing, macOS / iOS, Android, XBOX, PS4, PS5, Switch, Stadia

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The Forge is a cross-platform rendering framework supporting

  • PC
    • Windows 10
      • with DirectX 12 / Vulkan 1.1
      • with DirectX Ray Tracing API
      • DirectX 11 Fallback Layer for Windows 7 support (not extensively tested)
    • Linux Ubuntu 18.04 LTS with Vulkan 1.1 and RTX Ray Tracing API
  • Android Pie with Vulkan 1.1
  • macOS / iOS / iPad OS with Metal 2.2
  • XBOX One / XBOX One X / XBOX Series X (only available for accredited developers on request)
  • PS4 / PS4 Pro (only available for accredited developers on request)
  • PS5 (only available for accredited developers on request)
  • Switch (only available for accredited developers on request)
  • Google Stadia (in development) (only available for accredited developers on request)

Particularly, the graphics layer of The Forge supports cross-platform

  • Descriptor management. A description is on this Wikipage
  • Multi-threaded and asynchronous resource loading
  • Shader reflection
  • Multi-threaded command buffer generation

The Forge can be used to provide the rendering layer for custom next-gen game engines. It is also meant to provide building blocks to write your own game engine. It is like a "lego" set that allows you to use pieces to build a game engine quickly. The "lego" High-Level Features supported on all platforms are at the moment:

  • Resource Loader as shown in 10_PixelProjectedReflections, capable to load textures, buffers and geometry data asynchronously
  • Lua Scripting System - currently used in 06_Playground to load models and textures and animate the camera and in several other unit tests to cycle through the options they offer during automatic testing.
  • Animation System based on Ozz Animation System
  • Consistent Math Library based on an extended version of Vectormath with NEON intrinsics for mobile platforms
  • Extended version of EASTL
  • Consistent Memory Managament:
  • Input system with Gestures for Touch devices based on an extended version of gainput
  • Fast Entity Component System based on our internally developed ECS
  • Cross-platform FileSystem C API, supporting disk-based files, memory streams, and files in zip archives
  • UI system based on imGui with a dedicated unit test extended for touch input devices
  • Shader Translator using a superset of HLSL as the shader language. There is a Wiki page on how to use the Shader Translator
  • Various implementations of high-end Graphics Effects as shown in the unit tests below

Please find a link and credits for all open-source packages used at the end of this readme.

Twitter Join the Discord channel at https://discord.gg/hJS54bz

Twitter Join the channel at https://twitter.com/TheForge_FX?lang=en

The Forge Interactive Inc. is a Khronos member

Build Status

  • Windows Build status
  • macOS Build Status

News

Release 1.47 - December 18th, 2020 - OpenGL ES 2.0 Android support | Device Reset Support | DRED / Breadcrumb support | Lua driven functional tests | DX11 refactor | YUV support through Vulkan

As the year winds slowly down, we finally found time to do another release. First of all, Happy Holidays and a happy new Year!

Happy Holidays and a happy new Year!

Most of us will take off over the Holiday season and spent time with their families. We should be back online in the middle of January 2021.

  • OpenGL ES 2.0: TF will run on probably several hundred million of mobile devices in the future. It will be the rendering layer of business application frameworks. For this usage case, we added OpenGL ES 2.0 support only for Android. The OpenGL ES 2.0 layer only supports unit tests 1, 5, 12 and 31 at the moment.
  • Device change / reset: we finally implemented all the code that can deal with device changes, device resets or device removed scenarios on all platforms. The underlying design was always there but it took us 3+ years to finally add the functionality :-) When you go into any of the *OSBase.* files you can find a snippet of code that looks like this:
if (pApp->mSettings.mResetGraphics) 
	{
		pApp->Unload();
		pApp->Load();
		pApp->mSettings.mResetGraphics = false;
	}
  • DRED / Breadcrumb support: to be able to better tell what the reason behind a removed device is, we implemented DRED support on PC with DirectX 12 and XBOX. We integrated this into the first functional test 01_Transformations. Here is a screenshot. Look for the "Simulate crash" button:

Image of the Transformations Unit test

Breadcrumb are user defined markers used to pinpoint which command has caused GPU to stall. In the Breadcrumb unit test, two markers get injected into the command list. Pressing the crash button would result in a GPU hang. In this situation, the first marker would be written before the draw command, but the second one would stall for the draw command to finish. Due to the infinite loop in the shader, the second marker won't be written, and we can reason that the draw command has caused the GPU to hang. We log the markers' information to verify this. Check out this link for more info: D3D12 Device Removed Extended Data (DRED)

  • More Lua Scripting support for all functional tests:
    • For the scripted testing of the Unit Tests, this layer provides automated function registration of the UI elements to Lua State.
    • Any UI elements added to the GUI will add a function or a pair of function(Getter/Setter) to the Lua state for using them in any script. Lua function name resolution will work like this:
    • UI Widget "label" name will be included in the function name as follows,
      • For Widget events: label name + "Event Name". e.g., Lua Function name for label - "Press", and event - OnEdited : "PressOnEdited"
      • For Widget modifiers such as ints / floats: "Set" and "Get" function pair will be added as a prefix to label name e.g., "X" variable will have "SetX" and "GetX" pair of functions.
  • After writing the scripts, you can let the layer know about the scripts using AddTestScripts() function call and run them on any frame by RunTestScript() defined in UIApp class. There are examples of these test scripts in most of the UTs showing how you can also add these scripts to UI and test them on runtime.

Here is how the current Lua support in the functional tests might look like:

Lua support

  • DX11 refactor: we re-wrote the DX11 run-time a few times. We ended up with the most straighforward version. This version only recently shipped in Hades along with the Vulkan run-time on PC.

  • YUV support: we have now YUV support for all our Vulkan API platforms PC, Linux, Android and Switch. There is a new functional test for YUV. It runs on all these platforms:

YUV unit test

  • Audio: we removed the audio functional test. It was the only test that was released unfinished and didn't run on all our platforms. Our customers show love for FMOD ... would make more sense to show an integration of that.

  • GitHub issues fixed:

    • #188 - typo - lowercase L in first DepthStencilClearFlags constant "ClEAR_DEPTH"
    • #186 - Ubuntu: Examples fail to build
    • #182 - Flickering on master when vsync is off
    • #176 - [08_GlftViewer] Application crash on missing resource

Numerous other fixes ...

Release 1.46 - October 1st, 2020 - Supergiant's Hades | Windows Management | AMD FX Stochastic SS Reflection

  • Supergiant's Hades we are working with Supergiant since 2014. One of the on-going challenges was that their run-time was written in C#. At the beginning of last year, we suggested to help them in building a new cross-platform game engine in C/C++ from scratch with The Forge. The project started in April 2019 and the first version of this new engine launched in May this year. Hades was then released for Microsoft Windows, macOS, and Nintendo Switch on September 17, 2020. The game can run on all platforms supported by The Forge.

Here is a screenshot of Hades running on Switch:

Supergiant Hades

Here is an article by Forbes about Hades being at the top of the Nintendo Switch Charts. Hades is also a technology showcase for Intel's integrated GPUs on macOS and Windows. The target group of the game seems to often own those GPUs.

  • Windows management: there is a new functional test named 32_Window that demonstrates windows management on Windows, Linux and macOS.
    • The window layout, position, and size are now driven by the client dimensions, meaning that the values that the client demands are the exact values the client area will be represented with, regardless of the window style. This allows for much greater flexibility and consistency, especially when working with a fullscreen window.
    • Multi-monitor support has also been improved significantly, offering smooth consistent transitions between client displays and guaranteeing correct window behavior and data retention. Media layer functionality has been expanded, allowing the client to control mouse positioning, mouse visibility, and mouse visual representation.
    • It is now possible to create independent mouse cursors to further customize the application.

Here are the screenshots:

Windows: Windows Management for Windows

macOS: Windows Management for macOS

Linux: Windows Management for Linux

  • Screen-Space reflections: we renamed the functional test "10_PixelProjectedReflections" to 10_ScreenSpaceReflections. You have now two choices: you can pick either Pixel Projected Reflections or AMD's FX Stochastic Screen Space Reflection. We just made AMD's FX code cross-platform. It runs now on Windows, Linux, macOS, Switch, PS and XBOX.

Here are the screenshots:

Windows final scene: AMD FX Stochastic Screen Space Reflections

Without denoising: AMD FX Stochastic Screen Space Reflections before denoise

With denoising: AMD FX Stochastic Screen Space Reflections before denoise

PS4: AMD FX Stochastic Screen Space Reflections on PS4

macOS: AMD FX Stochastic Screen Space Reflections on macOS

  • Resolved GitHub issues:
    • Issue #183 - VERTEX_ATTRIB_RATE_INSTANCE ignored on macOS 10.12, iOS 10.0

Release 1.45 - August 6th, 2020 - TressFX | File System Rewrite

  • TressFX: we upgraded TressFX a bit and retuned the lighting.

Here are the screenshots: Black hair

Blond hair

Brown hair

Red hair

  • File system: our old file system was designed more for tools or Windows applications than for games. It consumed more memory than the whole rendering system and used Windows file methods extensively. That is the opposite of what you want in a game. It took us now several months to correct the mistake and come up with a file system that is tailored towards games. That means that the interface changed substantially. Thanks to all those who pointed this out. Sometimes it takes a couple of iterations to land on a design that is efficient. If you look at the new interface there are still path related functions in there. They will be removed step-by-step. Please check out the new file system interface and let us know what you think.

  • Android Vulkan: validation layer is now supported

Release 1.44 - July 16th, 2020 - Android | Linux

  • Mobile Devices: DPI scaling is properly handled now so we shouldn't see messed up UI anymore on mobile devices

  • Android: the following Unit-tests are now included for Android:

    • 03_MultiThread 03_MultiThread
    • 04_ExecuteIndirect 04_ExecuteIndirect
    • 07_Tesselation 07_Tesselation
    • 10_PixelProjectedReflections 10_PixelProjectedReflections
    • 12_ZipFileSystem 12_ZipFileSystem
    • 13_UserInterface 13_UserInterface
    • 14_WaveIntrinsics 14_WaveIntrinsics
    • 15_Transparency 15_Transparency
  • gamepad support: tested with PS4 controller

  • sample size reduction

  • proper closing of apps with the back button

  • proper handling of vSync

  • .zip filesystem handling

  • shader compile #include directive support

  • overall stability improvements

  • improved swapchain creation process and proper handling of current frame index

  • Linux:

    • Window management is improved
    • Borderless fullscreen is supported
    • Implemented full screen toggle (usually alt-enter)
    • Cursor position is now correct
    • Camera movement with mouse now works properly
    • Resources are freed properly

Release 1.43 - May 22nd, 2020 - MTuner | macOS / iOS run-time

  • Filesystem: it turns out the file system is still confusing and not intuitive. It mixes up several concepts but is not consistent and somehow favors Windows folder naming conventions, that do not exist in most of our target platforms. We did a slight first step with this release. We need to make a deeper change with the next release.
  • DirectX 11: the DirectX 11 run-time gets a lot of mileage now. For one game it went now successfully through a test center. This release holds a wide range of changes especially for multi-threaded rendering.
  • MTuner : we are making another attempt on integrating MTuner into the framework. We need it to tune memory usage in some game titles. The current version only reliably supports Windows but we try to extend it to more platforms.
    • Integrated Milos Tosic’s MTuner SDK into the Windows 10 runtime of The Forge. Combined with mmgr, this addition will provide the following features:
      • Automatic generation of .MTuner capture file alongside existing .memleaks file.
      • In-depth analysis of the generated file using MTuner’s user-friendly UI app.
      • Clear and efficient highlighting of memory leaks and usage hotspots.
    • Support for additional platforms coming soon!

MTuner MTuner was integrated into the Windows 10 runtime of The Forge following a request for more in-depth memory profiling capabilities by one of the developers we support. It has been adapted to work closely with our framework and its existing memory tracking capabilities to provide a complete picture of a given application’s memory usage.

To use The Forge’s MTuner functionality, simply drag and drop the .MTuner file generated alongside your application’s executable into the MTuner host app, and you can immediately begin analyzing your program’s memory usage. The intuitive interface and exhaustive supply of allocation info contained in a single capture file makes it easy to identify usage patterns and hotspots, as well as tracking memory leaks down to the file and line number. The full documentation of MTuner can be found [here](link: https://milostosic.github.io/MTuner/).

Currently, this feature is only available on Windows 10, but support for additional platforms provided by The Forge is forthcoming. Here is a screenshot of an example capture done on our first Unit Test, 01_Transformations: MTuner

  • Multi-Threading system: especially for the Switch run-time we extended our multi-threading system to support a "preferred" core.
  • macOS / iOS run-time got another make over. This time we brought the overall architecture a bit closer to the rest of the rendering system and we are also working towards supporting lower end hardware like a 2015 MacBook Air and macOS 10.13.6. Those requirements were based on the Steam Hardware Survey.
  • there are now functions that help you calcuate the memory usage supported by all APIs: look for caculateMemoryUse / freeMemoryStats
  • Windows management got a bit more flexible by offering borderless windows and more style attributes
  • 17_EntityComponentSystem runs now better on AMD CPUs ... there were some inefficiencies in the unit test ...

Release 1.42 - April 15th, 2020 - macOS / iOS run-time

Most of us are working from home now due to the Covid-19 outbreak. We are all trying to balance life and work in new ways. Since the last release we made a thorough pass through the macOS / iOS run-time, so that it is easier to make macOS your main development environment for games. Unit-tests fixes:

  • Fixed wrong project ordering in XCode
  • Fixed build-time macOS / iOS warnings.
  • Fixed 03_Multithread until test not showing the appropriate charts for all threads
  • Fixed warnings in 06_MaterialPlayground due to wrong GLTF validation
  • Fixed visual artifacts in 08_GLTFViewer not producing correct normals due to wrong Metal shader.
  • Added missing projects to macOS / iOS workspace and removed unnecessary ones.
  • Fixed unit test 14_WaveIntrinsics macOS on AMD iMac. Implemented workaround for AMD driver issue on macOS.

Metal runtime fixes:

  • Fixed Metal issue handling barriers: scheduled barriers were being ignored, introducing visual artifacts due to read-write race condition.

Closed Issues:

  • #168 - 04_ExecuteInDirect poor performance on MacPro 2019, AMD Vega II, Catalina 10.15.3
  • #156 - Benchmark support?

Release 1.41 - March 5th, 2020 - Path Tracing Benchmark | CPU Cacheline alignment | Improved Profiler | D3D12 Memory Allocator

  • Based on request we are providing a Path Tracing Benchmark in 16_RayTracing. It allows you to compare the performance of three platforms:

    • Windows with DirectX 12 DXR
    • Windows with Vulkan RTX
    • Linux with Vulkan RTX

    We believe that every benchmarking tool should be open-source, so that everyone can see what the source code is doing. We will extend this benchmark to the non-public platforms we support to compare the PC performance with console performance. The benchmark comes with batch files for all three platforms. Each run generates a HTML output file from the microprofiler that is integrated in TF. The default number of iterations is 64 but you can adjust that. There is a Readme file in the 16_RayTracing folder that describes the options.

Windows DirectX 12 DXR, GeForce RTX 2070 Super, 3840x1600, NVIDIA Driver 441.99

Windows DXR output of Ray Tracing Benchmark

Windows Vulkan RTX, GeForce RTX 2070 Super, 3840x1600, NVIDIA Driver 441.99

Windows RTX output of Ray Tracing Benchmark

Linux Vulkan RTX, Geforce RTX 2060, 1920x1080, NVIDIA Driver 435.21

Linux RTX output of Ray Tracing Benchmark

We will adjust the output of the benchmark to what users request.

  • With this release we also aligned the whole renderer interface better to 64 byte CPU cache lines. We trimmed down all the structs substantially and removed many. This is a breaking change for the renderer interface and a major change to the whole code base.
  • DirectX 12
    • D3D12 Memory Allocator: we are using now AMD's D3D12 memory allocator for DirectX after having used the Vulkan equivalent for more than two years. We also extended it to support Multi-GPU.
    • We upgraded to the latest dxgi factory interface in DirectX 12
  • Microprofiler: because we need the microprofiler to offer the QA department help in reporting performance problems for some of the games that will be shipping with TF (and the benchmark mentioned above), we did another pass on its functionality and ease of use, especially on console platforms. The idea is that QA can quickly and easily store a screenshot or HTML file in a bug report. This is still work in progress and with every shipping game will probably be improved.
  • Now that GDC 2020 was postponed, we will also postpone our GDC related activities. The user meeting and our GDC talk will be postponed until the next GDC happens. If there is a need we can also do a user meeting in an online conference room or in Discord in a private area. Let us know.
  • Renamed CustomMiddleware to Custom-Middleware back ...

See the release notes from previous releases in the Release section.

PC Windows Requirements:

  1. Windows 10

  2. Drivers

  • AMD / NVIDIA / Intel - latest drivers
  1. Visual Studio 2017 with Windows SDK / DirectX (you need to get it via the Visual Studio Intaller)
  • Base version:

    • The minimum Windows 10 version is 1803.
    • The minimum SDK version is 1803 (10.0.17134.12).
  • To use Raytracing:

    • The minimum Windows 10 version is 1809.
    • The minimum SDK version is 1809 (10.0.17763.0).

https://developer.microsoft.com/en-us/windows/downloads/sdk-archive

  1. The Forge supports now as the min spec for the Vulkan SDK 1.1.82.0 and as the max spec 1.1.114

  2. The Forge is currently tested on

  • AMD 5x, VEGA GPUs (various)
  • NVIDIA GeForce 9x, 10x. 20x GPUs (various)
  • Intel Skull Canyon

macOS Requirements:

  1. macOS min spec. 10.13.6

  2. Xcode 11.0 (11A419c)

  3. The Forge is currently tested on the following macOS devices:

  • iMac with AMD RADEON 560 (Part No. MNDY2xx/A)
  • iMac with AMD RADEON 580 (Part No. MNED2xx/A)
  • MacBook Pro 13 inch (MacBookPro13,2)
  • Macbook Pro 13 inch (MacbookPro14,2)

In the moment we do not have access to an iMac Pro or Mac Pro. We can test those either with Team Viewer access or by getting them into the office and integrating them into our build system. We will not test any Hackintosh configuration.

iOS Requirements:

  1. iOS 13.1 beta 3 (17A5837a)

  2. XCode: see macOS

To run the unit tests, The Forge requires an iOS device with an A9 or higher CPU (see GPU Processors or see iOS_Family in this table iOS_GPUFamily3_v3). This is required to support the hardware tessellation unit test and the ExecuteIndirect unit test (requires indirect buffer support). The Visibility Buffer doesn't run on current iOS devices because the texture argument buffer on those devices is limited to 31 (see Metal Feature Set Table and look for the entry "Maximum number of entries in the texture argument table, per graphics or compute function") , while on macOS it is 128, which we need for the bindless texture array.

We are currently testing on

  • iPhone 7 (Model A1778)
  • iPhone Xs Max (Model MT5D2LL/A)

iPad OS Requirements:

  1. iPadOS 13.1 beta 3 (17A5837a)

  2. XCode: see macOS

We are currently testing on:

  • iPad (Model A1893)

PC Linux Requirements:

  1. Ubuntu 18.04 LTS Kernel Version: 4.15.0-20-generic

  2. GPU Drivers:

  1. Workspace file is provided for codelite 12.0.6

  2. Vulkan SDK Version 1.1.108: download the native Ubuntu Linux package for all the elements of the Vulkan SDK LunarG Vulkan SDK Packages for Ubuntu 16.04 and 18.04

  3. The Forge is currently tested on Ubuntu with the following GPUs:

  • AMD RADEON RX 480
  • AMD RADEON VEGA 56
  • NVIDIA GeForce 2070 RTX

Android Requirements:

  1. Android Phone with Android Pie (9.x) for Vulkan 1.1 support

  2. Visual Studio 2019 (Visual Studio 2017 works too but has a bug in the build module)

  3. Android API level 28 or higher

We can't use Android SDK < 26 due to

#if _ANDROID_API_ >= 26
int pthread_getname_np(pthread_t _pthread, char* _buf, size_t _n) _INTRODUCED_IN(26);
#endif / _ANDROID_API_ >= 26 /

At the moment, the Android run-time does not support the following unit tests due to -what we consider- driver bugs or lack of support:

  • 08_Procedural
  • 09a_HybridRayTracing
  • 16_RayTracing
  • 16a_SphereTracing
  • Visibility Buffer
  1. We are currently testing on

Install

  • For PC Windows run PRE_BUILD.bat. It will download and unzip the art assets and install the shader builder extension for Visual Studio 2017.

  • For Linux and Mac run PRE_BUILD.command. If its the first time checking out the forge make sure the PRE_BUILD.command has the correct executable flag by running the following command chmod +x PRE_BUILD.command

    It will only download and unzip required Art Assets (No plugins/extensions install).

Unit Tests

There are the following unit tests in The Forge:

1. Transformation

This unit test just shows a simple solar system. It is our "3D game Hello World" setup for cross-platform rendering.

Image of the Transformations Unit test

2. Compute

This unit test shows a Julia 4D fractal running in a compute shader. In the future this test will use several compute queues at once.

Image of the Compute Shader Unit test

3. Multi-Threaded Rendering

This unit test shows how to generate a large number of command buffers on all platforms supported by The Forge. This unit test is based on a demo by Intel called Stardust.

Image of the Multi-Threaded command buffer generation example

4. ExecuteIndirect

This unit test shows the difference in speed between Instanced Rendering, using ExecuteIndirect with CPU update of the indirect argument buffers and using ExecuteIndirect with GPU update of the indirect argument buffers. This unit test is based on the Asteroids example by Intel.

Image of the ExecuteIndirect Unit test Using ExecuteIndirect with GPU updates for the indirect argument buffers

Image of the ExecuteIndirect Unit test Using ExecuteIndirect with CPU updates for the indirect argument buffers

Image of the ExecuteIndirect Unit test Using Instanced Rendering

5. Font Rendering

This unit test shows the current state of our font rendering library that is based on several open-source libraries.

Image of the Font Rendering Unit test

6. Material Playground

This unit test shows a range of game related materials:

Hair: Many years ago in 2012 / 2013, we helped AMD and Crystal Dynamics with the development of TressFX for Tomb Raider. We also wrote an article about the implementation in GPU Pro 5 and gave a few joint presentations on conferences like FMX. At the end of last year we revisited TressFX. We took the current code in the GitHub repository, changed it a bit and ported it to The Forge. It now runs on PC with DirectX 12 / Vulkan, macOS and iOS with Metal 2 and on the XBOX One. We also created a few new hair assets so that we can showcase it. Here is a screenshot of our programmer art:

Hair on PC

Metal:

Material Playground Metal on PC

Wood:

Material Playground Wood on PC

7. Hardware Tessellation

This unit test showcases the rendering of grass with the help of hardware tessellation.

Image of the Hardware Tessellation Unit test

8. glTF Model Viewer

A cross-platform glTF model viewer that optimizes the vertex and index layout for the underlying platform and picks the right texture format for the underlying platform. We integrated Arseny Kapoulkine @zeuxcg excellent meshoptimizer and use the same PBR as used in the Material Playground unit test. This modelviewer can also utilize Binomials Basis Universal Texture Support as an option to load textures. Support was added to the Image class as a "new image format". So you can pick basis like you can pick DDS or KTX. For iOS / Android we go directly to ASTC because Basis doesn't support ASTC at the moment.

glTF model viewer running on iPad with 2048x1536 resolution

glTF model viewer

glTF model viewer

glTF model viewer running on Samsung Galaxy S10 with Vulkan with 1995x945 resolution

glTF model viewer

glTF model viewer

glTF model viewer running on Ubuntu AMD RX 480 with Vulkan with 1920x1080 resolution

glTF model viewer

glTF model viewer

9. Light and Shadow Playground

This unit test shows various shadow and lighting techniques that can be chosen from a drop down menu. There will be more in the future.

  • Exponential Shadow Map - this is based on Marco Salvi's @marcosalvi papers. This technique filters out the edge of the shadow map by approximating the shadow test using exponential function that involves three subjects: the depth value rendered by the light source, the actual depth value that is being tested against, and the constant value defined by the user to control the softness of the shadow
  • Adaptive Shadow Map with Parallax Correction Cache - this is based on the article "Parallax-Corrected Cached Shadow Maps" by Pavlo Turchyn in GPU Zen 2. It adaptively chooses which light source view to be used when rendering a shadow map based on a hiearchical grid structure. The grid structure is constantly updated depending on the user's point of view and it uses caching system that only renders uncovered part of the scene. The algorithm greatly reduce shadow aliasing that is normally found in traditional shadow map due to insufficient resolution. Pavlo Turchyn's paper from GPU Pro 2 added an additional improvement by implementing multi resolution filtering, a technique that approximates larger size PCF kernel using multiple mipmaps to achieve cheap soft shadow. He also describes how he integrated a Parallax Correction Cache to Adaptive Shadow Map, an algorithm that approximates moving sun's shadow on static scene without rendering tiles of shadow map every frame. The algorithm is generally used in an open world game to approximate the simulation of day & night’s shadow cycle more realistically without too much CPU/GPU cost.
  • Signed Distance Field Soft Shadow - this is based on Daniel Wright's Siggraph 2015 @EpicShaders presentation. To achieve real time SDF shadow, we store the distance to the nearest surface for every unique Meshes to a 3D volume texture atlas. The Mesh SDF is generated offline using triangle ray tracing, and half precision float 3D volume texture atlas is accurate enough to represent 3D meshes with SDF. The current implementation only supports rigid meshes and uniform transformations (non-uniform scale is not supported). An approximate cone intersection can be achieved by measuring the closest distance of a passed ray to an occluder which gives us a cheap soft shadow when using SDF.

To achieve high-performance, the playground runs on our signature rendering architecture called Triangle Visibility Buffer. The step that generates the SDF data also uses this architecture.

Click on the following screenshot to see a movie:

Signed Distance Field Soft Shadow Map

The following PC screenshots are taken on Windows 10 with a AMD RX550 GPU (driver 19.7.1) with a resolution of 1920x1080.

Exponential Shadow Maps:

Light and Shadow Playground - Exponential Shadow Map

Adaptive Shadow Map with Parallax Correction Cache

Adaptive Shadow Map with Parallax Correction Cache

Signed Distance Field Soft Shadow:

Signed Distance Field Soft Shadow Map

Signed Distance Field Soft Shadows - Debug Visualization

Signed Distance Field Soft Shadow Map

The following shots show Signed Distance Field Soft Shadows running on iMac with a AMD RADEON Pro 580

Signed Distance Field Soft Shadow Map

Signed Distance Field Soft Shadow Map

The following shots show Signed Distance Field Soft Shadows running on XBOX One:

Signed Distance Field Soft Shadow Map

Signed Distance Field Soft Shadow Map

Signed Distance Field Soft Shadow Map

Readme for Signed Distance Field Soft Shadow Maps:

To generate the SDF Mesh data you should select “Signed Distance Field” as the selected shadow type in the Light and Shadow Playground. There is a button called “Generate Missing SDF” and once its clicked, it shows a progress bar that represents the remaining SDF mesh objects utilized for SDF data generation. This process is multithreaded, so the user can still move around the scene while waiting for the SDF process to be finished. This is a long process and it could consume up to 8+ hours depending on your CPU specs. To check how many SDF objects there are presently in the scene, you can mark the checkbox "Visualize SDF Geometry On The Scene".

9a. Hybrid Ray-Traced Shadows

This unit test was build by Kostas Anagnostou @KostasAAA to show how to ray trace shadows without using a ray tracing API like DXR / RTX. It should run on all GPUs (not just NVIDIA RTX GPUs) and the expectation is that it should run comparable with a DXR / RTX based version even on a NVIDIA RTX GPU. That means the users of your game do not have to buy a NVIDIA RTX GPU to enjoy HRT shadows :-) Hybrid Ray Traced Shadows

10. Screen-Space Reflections

This test offers two choices: you can pick either Pixel Projected Reflections or AMD's FX Stochastic Screen Space Reflection. We just made AMD's FX code cross-platform. It runs now on Windows, Linux, macOS, Switch, PS and XBOX.

Here are the screenshots of AMD's FX Stochastic Screen Space Reflections:

Windows final scene: AMD FX Stochastic Screen Space Reflections

Without denoising: AMD FX Stochastic Screen Space Reflections before denoise

With denoising: AMD FX Stochastic Screen Space Reflections before denoise

PS4: AMD FX Stochastic Screen Space Reflections on PS4

macOS: AMD FX Stochastic Screen Space Reflections on macOS

In case you pick Pixel-Projected Reflections, the application features an implementation of the papers Optimized pixel-projected reflections for planar reflectors and IMPLEMENTATION OF OPTIMIZED PIXEL-PROJECTED REFLECTIONS FOR PLANAR REFLECTORS

Image of the Pixel-Projected Reflections Unit test

11. Multi-GPU (Driver support only on PC Windows)

This unit test shows a typical VR Multi-GPU configuration. One eye is rendered by one GPU and the other eye by the other one.

Image of the Multi-GPU Unit test

12. File System Test

This unit test showcases a cross-platform FileSystem C API, supporting disk-based files, memory streams, and files in zip archives. The API can be viewed in IFileSystem.h, and all of the example code has been updated to use the new API.

  • The API is based around Paths, where each Path represents an absolute, canonical path string on a particular file system. You can query information about the files at Paths, open files as FileStreams, and copy files between different Paths.
  • The concept of FileSystemRoots has been replaced by ResourceDirectorys. ResourceDirectorys are predefined directories where resources are expected to exist, and there are convenience functions to open files in resource directories. If your resources don’t exist within the default directory for a particular resource type, you can call fsSetPathForResourceDirectory to relocate the resource directory; see the unit tests for sample code on how to do this.

File System Unit Test

13. imGUI integration unit test

This unit test shows how the integration of imGui with a wide range of functionality.

Image of the imGui Integration in The Forge

14. Order-Independent Transparency unit test

This unit test compares various Order-Indpendent Transparency Methods. In the moment it shows:

15. Wave Intrinsics unit test

This unit test shows how to use the new wave intrinsics. Supporting Windows with DirectX 12 / Vulkan, Linux with Vulkan and macOS / iOS.

Image of the Wave Intrinsics unit test in The Forge

16. Path Tracer - Ray Tracing Unit Test

The new 16_Raytracing unit test shows a simple cross-platform path tracer. On iOS this path tracer requires A11 or higher. It is meant to be used in tools in the future and doesn't run in real-time. To support the new path tracer, the Metal raytracing backend has been overhauled to use a sort-and-dispatch based approach, enabling efficient support for multiple hit groups and miss shaders. The most significant limitation for raytracing on Metal is that only tail recursion is supported, which can be worked around using larger per-ray payloads and splitting up shaders into sub-shaders after each TraceRay call; see the Metal shaders used for 16_Raytracing for an example on how this can be done.

macOS 1920x1080 AMD Pro Vega 64

Path Tracer running on macOS

iOS iPhone X 812x375

Path Tracer running on macOS

Windows 10 1080p NVIDIA RTX 2080 with DXR Driver version 441.12

Path Tracer running on Windows DXR

Windows 10 1080p NVIDIA RTX 2080 with RTX Driver version 441.12

Path Tracer running on Windows RTX

Linux 1080p NVIDIA RTX 2060 with RTX Driver version 435

Path Tracer running on Linux RTX

16a. Sphere Tracing

This unit test was originally posted on ShaderToy by Inigo Quilez and Sopyer. It shows how a scene is ray marched with shadows, reflections and AO

Image of the Sphere Tracing  unit test in The Forge

17. Entity Component System Test

This unit test shows how to use the high-performance entity component system in The Forge. This unit test is based on a ECS system that we developed internally for tools.

Image of the Entity Component System unit test in The Forge

18. Sparse Virtual Textures

The Forge has now support for Sparse Virtual Textures on Windows and Linux with DirectX 12 / Vulkan. Sparse texture (also known as "virtual texture", “tiled texture”, or “mega-texture”) is a technique to load huge size (such as 16k x 16k or more) textures in GPU memory. It breaks an original texture down into small square or rectangular tiles to load only visible part of them.

The unit test 18_Virtual_Texture is using 7 sparse textures:

  • Mercury: 8192 x 4096
  • Venus: 8192 x 4096
  • Earth: 8192 x 4096
  • Moon: 16384 x 8192
  • Mars: 8192 x 4096
  • Jupiter: 4096 x 2048
  • Saturn: 4096 x 4096

There is a unit test that shows a solar system where you can approach planets with Sparse Virtual Textures attached and the resolution of the texture will increase when you approach.

Linux 1080p NVIDIA RTX 2060 with RTX Driver version 435

Sparse Virtual Texture on Linux Vulkan

Windows 10 1080p NVIDIA 1080 DirectX 12

Sparse Virtual Texture on Windows 10 DirectX 12

Windows 10 1080p NVIDIA 1080 Vulkan

Sparse Virtual Texture on Windows Vulkan

Sparse Virtual Texture on Windows Vulkan

21. Ozz Playback Animation

This unit test shows how to playback a clip on a rig.

Image of Playback Animation in The Forge

22. Ozz Playback Blending

This unit test shows how to blend multiple clips and play them back on a rig.

Image of Playback Blending in The Forge

23. Ozz Joint Attachment

This unit test shows how to attach an object to a rig which is being posed by an animation.

Image of Ozz Joint Attachment in The Forge

24. Ozz Partial Blending

This unit test shows how to blend clips having each only effect a certain portion of joints.

Image of Ozz Partial Blending in The Forge

25. Ozz Additive Blending

This unit test shows how to introduce an additive clip onto another clip and play the result on a rig.

Image of Ozz Additive Blending in The Forge

26. Ozz Baked Physics

This unit test shows how to use a scene of a physics interaction that has been baked into an animation and play it back on a rig.

Image of Ozz Baked Physics in The Forge

27. Ozz Multi Threading

This unit test shows how to animate multiple rigs simultaneously while using multi-threading for the animation updates.

Image of Ozz Multi Threading in The Forge

28. Ozz Skinning

This unit test shows how to use skinning with Ozz

Image of the Ozz Skinning unit test

29. Ozz Inverse Kinematic

This unit test shows how to use a Aim and a Two bone IK solvers

Aim IK Ozz Aim IK

Two Bone IK Ozz Two Bone IK

32. Windows Management

This test demonstrates windows management on Windows, Linux and macOS.

  • The window layout, position, and size are now driven by the client dimensions, meaning that the values that the client demands are the exact values the client area will be represented with, regardless of the window style. This allows for much greater flexibility and consistency, especially when working with a fullscreen window.
  • Multi-monitor support has also been improved significantly, offering smooth consistent transitions between client displays and guaranteeing correct window behavior and data retention. Media layer functionality has been expanded, allowing the client to control mouse positioning, mouse visibility, and mouse visual representation.
  • It is now possible to create independent mouse cursors to further customize the application.

Here are the screenshots:

Windows: Windows Management for Windows

macOS: Windows Management for macOS

Linux: Windows Management for Linux

Examples

There is an example implementation of the Triangle Visibility Buffer as covered in various conference talks. Here is a blog entry that details the implementation in The Forge.

Image of the Visibility Buffer

Tools

Below are screenshots and descriptions of some of the tools we integrated.

MTuner

MTuner MTuner was integrated into the Windows 10 runtime of The Forge following a request for more in-depth memory profiling capabilities by one of the developers we support. It has been adapted to work closely with our framework and its existing memory tracking capabilities to provide a complete picture of a given application’s memory usage.

To use The Forge’s MTuner functionality, simply drag and drop the .MTuner file generated alongside your application’s executable into the MTuner host app, and you can immediately begin analyzing your program’s memory usage. The intuitive interface and exhaustive supply of allocation info contained in a single capture file makes it easy to identify usage patterns and hotspots, as well as tracking memory leaks down to the file and line number. The full documentation of MTuner can be found [here](link: https://milostosic.github.io/MTuner/).

Currently, this feature is only available on Windows 10, but support for additional platforms provided by The Forge is forthcoming. Here is a screenshot of an example capture done on our first Unit Test, 01_Transformations: MTuner

Ray Tracing Benchmark

Based on request we are providing a Ray Tracing Benchmark in 16_RayTracing. It allows you to compare the performance of three platforms:

  • Windows with DirectX 12 DXR
  • Windows with Vulkan RTX
  • Linux with Vulkan RTX

We will extend this benchmark to the non-public platforms we support to compare the PC performance with console performance. The benchmark comes with batch files for all three platforms. Each run generates a HTML output file from the profiler that is integrated in TF. The default number of iterations is 64 but you can adjust that. There is a Readme file in the 16_RayTracing folder that describes the options.

Windows DirectX 12 DXR, GeForce RTX 2070 Super, 3840x1600, NVIDIA Driver 441.99

Windows DXR output of Ray Tracing Benchmark

Windows Vulkan RTX, GeForce RTX 2070 Super, 3840x1600, NVIDIA Driver 441.99

Windows RTX output of Ray Tracing Benchmark

Microprofiler

We integrated the Micro Profiler into our code base by replacing the proprietary UI with imGUI and simplified the usage. Now it is much more tightly and consistently integrated in our code base.

Here are screenshots of the Microprofiler running the Visibility Buffer on PC:

Microprofiler

Microprofiler

Microprofiler

Microprofiler

Here are screenshots of the Microprofiler running a unit test on iOS:

Microprofiler

Microprofiler

Microprofiler

Check out the Wikipage for an explanation on how to use it.

Shader Translator

We provide a shader translator, that translates one shader language -a superset of HLSL called Forge Shader Language (FLS) - to the target shader language of all our target platforms. That includes the console and mobile platforms as well. We expect this shader translator to be an easier to maintain solution for smaller game teams because it allows to add additional data to the shader source file with less effort. Such data could be for example a bucket classification or different shaders for different capability levels of the underlying platform, descriptor memory requirements or resource memory requirements in general, material info or just information to easier pre-compile pipelines. The actual shader compilation will be done by the native compiler of the target platform.

How to use the Shader Translator

Releases / Maintenance

Confetti will prepare releases when all the platforms are stable and running and push them to this GitHub repository. Up until a release, development will happen on internal servers. This is to sync up the console, mobile, macOS and PC versions of the source code.

Products

We would appreciate it if you could send us a link in case your product uses The Forge. Here are the ones we received so far:

Supergiant Games Hades

Supergiant's Hades we are working with Supergiant since 2014. One of the on-going challenges was that their run-time was written in C#. At the beginning of last year, we suggested to help them in building a new cross-platform game engine in C/C++ from scratch with The Forge. The project started in April 2019 and the first version of this new engine launched in May this year. Hades was then released for Microsoft Windows, macOS, and Nintendo Switch on September 17, 2020. The game can run on all platforms supported by The Forge.

Here is a screenshot of Hades running on Switch:

Supergiant Hades

Here is an article by Forbes about Hades being at the top of the Nintendo Switch Charts. Hades is also a technology showcase for Intel's integrated GPUs on macOS and Windows. The target group of the game seems to often own those GPUs.

Bethesda's Creation Engine

Bethesda based their rendering layer for their next-gen engine on The Forge. We helped integrate and optimize it.

Bethesda's Creation Engine

Here is more info about this game engine:

Todd Howard Teases Bethesda's New Game Engine Behind The Elder Scrolls 6 And Starfield

Bethesda's overhauling its engine for Starfield and The Elder Scrolls 6

StarVR One SDK

The Forge is used to build the StarVR One SDK:

StarVR

Torque 3D

The Forge will be used as the rendering framework in Torque 3D:

Torque 3D

Star Wars Galaxies Level Editor

SWB is an editor for the 2003 game 'Star Wars Galaxies' that can edit terrains, scenes, particles and import/export models via FBX. The editor uses an engine called 'atlas' that will be made open source in the future. It focuses on making efficient use of the new graphics APIs (with help from The-Forge!), ease-of-use and terrain rendering.

SWB Level Editor

Writing Guidelines

For contributions to The Forge we apply the following writing guidelines:

User Group Meetings

There will be a user group meeting during GDC. In case you want to organize a user group meeting in your country / town at any other point in time, we would like to support this. We could send an engineer for a talk.

Support for Education

In case your School / College / University uses The Forge for education, we would like to support this as well. We could send an engineer or help create material. So far the following schools use The Forge for teaching:

Breda University of Applied Sciences

        Contact:
        Jeremiah van Oosten 
        Monseigneur Hopmansstraat 1
        4817 JT Breda

Ontario Tech University

        Contact:
        Andrew Hogue
        Ontario Tech University
        SIRC 4th floor
        2000 Simcoe St N
        Oshawa, ON, L1H 7K4

Open-Source Libraries

The Forge utilizes the following Open-Source libraries:

About

The Forge Cross-Platform Rendering Framework PC, Linux, Ray Tracing, macOS / iOS, Android, XBOX, PS4, PS5, Switch, Stadia

License:Apache License 2.0


Languages

Language:C++ 63.4%Language:GLSL 13.7%Language:Metal 10.8%Language:C 4.6%Language:Objective-C++ 3.7%Language:Objective-C 1.3%Language:Python 0.9%Language:JavaScript 0.7%Language:C# 0.3%Language:Lua 0.2%Language:HLSL 0.2%Language:Makefile 0.1%Language:HTML 0.0%Language:Batchfile 0.0%Language:CSS 0.0%Language:Java 0.0%Language:Shell 0.0%Language:Pawn 0.0%