windows 21h2 gpu support
xiasun opened this issue · comments
Hi all, I was trying to run PixelStreaming on a Windows docker (Windows host and windows container).
My Windows version is 21h2, and cannot be downgraded. It seems that unreal docker does not have a 21h2 base version.
When I try to use 20h2 image, only hyper-v isolation mode is supported, and when I start a PixelStreaming exe in container, it throw error below:
[2022.08.02-11.40.40:592][ 0]LogD3D11RHI: Loaded GFSDK_Aftermath_Lib.x64.dll
[2022.08.02-11.40.40:593][ 0]LogD3D11RHI: D3D11 min allowed feature level: 11_0
[2022.08.02-11.40.40:593][ 0]LogD3D11RHI: D3D11 max allowed feature level: 11_0
[2022.08.02-11.40.40:593][ 0]LogD3D11RHI: D3D11 adapters:
[2022.08.02-11.40.40:596][ 0]LogD3D11RHI: 0. 'Microsoft Basic Render Driver' (Feature Level 11_0)
[2022.08.02-11.40.40:596][ 0]LogD3D11RHI: 0/0/5375 MB DedicatedVideo/DedicatedSystem/SharedSystem, Outputs:0, VendorId:0x1414
[2022.08.02-11.40.40:597][ 0]LogD3D11RHI: 1. 'Microsoft Basic Render Driver' (Feature Level 11_0)
[2022.08.02-11.40.40:597][ 0]LogD3D11RHI: 0/0/5375 MB DedicatedVideo/DedicatedSystem/SharedSystem, Outputs:0, VendorId:0x1414
[2022.08.02-11.40.40:597][ 0]LogD3D11RHI: Error: Failed to choose a D3D11 Adapter.
while running the same exe on host OS directly, the logging is like this
[2022.08.02-09.31.59:104][ 0]LogD3D11RHI: Creating new Direct3DDevice
[2022.08.02-09.31.59:104][ 0]LogD3D11RHI: GPU DeviceId: 0x1c03 (for the marketing name, search the web for "GPU Device Id")
[2022.08.02-09.31.59:104][ 0]LogWindows: EnumDisplayDevices:
[2022.08.02-09.31.59:104][ 0]LogWindows: 0. 'NVIDIA GeForce GTX 1060 6GB' (P:1 D:1)
[2022.08.02-09.31.59:106][ 0]LogWindows: 1. 'NVIDIA GeForce GTX 1060 6GB' (P:0 D:1)
[2022.08.02-09.31.59:107][ 0]LogWindows: 2. 'NVIDIA GeForce GTX 1060 6GB' (P:0 D:0)
[2022.08.02-09.31.59:108][ 0]LogWindows: 3. 'NVIDIA GeForce GTX 1060 6GB' (P:0 D:0)
[2022.08.02-09.31.59:109][ 0]LogWindows: DebugString: FoundDriverCount:4
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: Adapter Name: NVIDIA GeForce GTX 1060 6GB
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: Driver Version: 511.79 (internal:30.0.15.1179, unified:511.79)
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: Driver Date: 2-10-2022
[2022.08.02-09.31.59:109][ 0]LogRHI: Texture pool is 4236 MB (70% of 6052 MB)
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: Creating D3DDevice using adapter:
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: Description : NVIDIA GeForce GTX 1060 6GB
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: VendorId : 10de
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: DeviceId : 1c03
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: SubSysId : 11d71028
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: Revision : 00a1
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: DedicatedVideoMemory : 6345981952 bytes
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: DedicatedSystemMemory : 0 bytes
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: SharedSystemMemory : 17147652096 bytes
[2022.08.02-09.31.59:109][ 0]LogD3D11RHI: AdapterLuid : 0 46434
[2022.08.02-09.31.59:175][ 0]LogD3D11RHI: RHI has support for 64 bit atomics
[2022.08.02-09.31.59:175][ 0]LogD3D11RHI: Async texture creation enabled
[2022.08.02-09.31.59:175][ 0]LogD3D11RHI: Array index from any shader is supported
[2022.08.02-09.31.59:194][ 0]LogD3D11RHI: GPU Timing Frequency: 1000.000000 (Debug: 2 1)
[2022.08.02-09.31.59:264][ 0]LogRHI: GeForceNow SDK initialized: 1
[2022.08.02-09.31.59:265][ 0]LogRendererCore: Ray tracing is disabled. Reason: r.RayTracing=0.
According to the Hardware devices in containers, process isolation is needed to use d3d inside Windows container. And win10 21h2 does not support process isolation as mentioned in Windows container compability
Is there any way I can run PixelStreaming in my windows container on a Win10 21H2 host?
Many thanks!
Sadly, there's no way I know to do that right now. There are underlying mechanisms which I think (but have not tried) which could expose a GPU to a HyperV-isolated container, but they are not supported by the currently-released Docker Engine 20.10.
The upcoming Docker Engine 22.06 should support exposing a GPU to a HyperV-isolated container when using the (currently experimental) containerd backend, but I haven't tried it myself (I happen to be the one who implemented the syntax for this, but the actual implementation is in hcsshim), and because it's a different exposure mechanism, it might not work the same way; e.g., it might involve dedicating a GPU to the container, which is different from the process isolation mechanism demonstrated in the pixel streaming documentation, which simply exposes D3D access to the host's GPUs into the container, like any other process on the host.
For future reference, exposing a GPU into a HyperV-isolated container should look something like --device vpci-instance-id://PCIP\.....
I haven't tried this, but I suspect it relies on Discrete Device Assignment which is only supported on Windows Server. (I think the PowerShell command Get-VMHostAssignableDevice will list the supported devices present, and it'd the InstanceID field from that. Neither of my Windows 10 machines show any devices when I run that command, one has an AMD GPU, one has an Nvidia GPU.)
It's also possible that GPU Partitioning will work, but I literally only discovered that while researching this answer, and I don't know if the hcsshim code that drives this will support that. (Also, the existing guides I could find are all attaching this to a known VM, so if this is feasible it'll be low-level WMI calls to actually generate a GPU partition that we can pass into --device vpci-instance-id://... if this does work. (Also, it seems that GPU Partitioning is not formally supported on Windows Server so it's likely that the hcsshim haven't tried to support it, and it's possible that the underlying HCS system doesn't expose this level of control over its micro-VM at all.))
So all-up, this isn't currently feasible on any platform without pre-release or experimental code, and I suspect will never work on Windows client builds, which is the place where you're most-likely to need it due to lack of Process Isolation.
Sorry.
@TBBle thanks alot! I am now debugging my PixelStreaming-like app inside container on a machine with Win 11 21h2, to access d3d device. If everything is okay, I may try to deploy in on a Windows server later for production usages.