Giters

keylase / nvidia-patch

This patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

i still got an error with read-only file system in nvidia docker container

connor-tan opened this issue · comments

commented

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. using docker image: pytorch/pytorch 2.1.0-cuda12.1-cudnn8-runtime
  2. execute "docker run -itd --name test --runtime=nvidia -e NVIDIA_DRIVER_CAPABILITIES="all" pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime"
  3. execute docker-entrypoint.sh in container "test" when container start running
  4. execute patch-test.sh

Expected behavior
patched successfully

Output listings
cu->cuMemAlloc(&data, size) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
[AVHWDeviceContext @ 0x55fc0b49f5c0] cu->cuMemAlloc(&data, size) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
[hwupload @ 0x55fc0bd69780] Failed to allocate frame to upload to.
[vf#0:0 @ 0x55fc0b49d980] Error while filtering: Cannot allocate memory
Failed to inject frame into filter network: Cannot allocate memory
Error while filtering: Cannot allocate memory

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 22.04]
  • GPU model: [e.g. RTX 3080Ti]
  • Patch commit used: master branch ,The latest submission
  • Nvidia driver version: [e.g. 535.146.02]

Additional context
when i execute patch.sh,shows "./patch.sh: line 373: /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.535.146.02: Read-only file system"

commented

Can you make the docker filesystem write-able?

commented

Can you make the docker filesystem write-able?
this directory "/usr/lib/x86_64-linux-gnu/" is write-able

commented

You might have to build a custom image based on top of the cuda one you want to use. Please go through the README:

It is possible to use this patch with nvidia-docker containers, even if host machine hasn't patched drivers. See Dockerfile for example.

Essentially all you need to do during build is:

COPY the patch.sh and docker-entrypoint.sh files into your container.

Make sure docker-entrypoint.sh is invoked on container start.

docker-entrypoint.sh script does on-the-fly patching by means of manipulating dynamic linker to workaround read-only mount of Nvidia runtime. Finally it passes original docker command to shell, like if entrypoint was not restricted by ENTRYPOINT directive. So docker run --runtime=nvidia -it mycontainer echo 123 will print 123. Also it can be just invoked from your entrypoint script, if you have any.