rbonghi / jetson_stats

📊 Simple package for monitoring and control your NVIDIA Jetson [Orin, Xavier, Nano, TX] series

Home Page:https://rnext.it/jetson_stats

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

No GPU temperature when running jtop inside a docker container

livneros-greeneye opened this issue · comments

Describe the bug

While running jtop inside a docker container, the GPU temperature details are offline.

To Reproduce

Steps to reproduce the behavior:

from jtop import jtop
jetson = jtop()
jetson.start()
jetson.temperature["GPU"]

  1. Running the following commands on my host plots:
    > {'temp': -256.0, 'online': False}
  2. Running a docker container, as mentioned in the documentation:
    docker run --rm -it -v /run/jtop.sock:/run/jtop.sock rbonghi/jetson_stats:latest plots:
    > {'temp': 39.875, 'online': True}

The same issue happens while using jtop:

  1. On host:
    image
  2. Inside the docker container:
    image

Board

Software part of jetson-stats 4.2.2 - (c) 2023, Raffaello Bonghi
Model: Jetson AGX Orin as NX-16GB - Jetpack 5.1.1 [L4T 35.3.1]
NV Power Mode[0]: MAXN
Serial Number: [XXX Show with: jetson_release -s XXX]
Hardware:
 - 699-level Part Number: 699-13701-0005-500 M.0
 - P-Number: p3701-0005
 - Module: NVIDIA Jetson AGX Orin (64GB ram)
 - SoC: tegra23x
 - CUDA Arch BIN: 8.7
 - Codename: Concord
Platform:
 - Machine: aarch64
 - System: Linux
 - Distribution: Ubuntu 20.04 focal
 - Release: 5.10.104-tegra
 - Python: 3.8.10
jtop:
 - Version: 4.2.2
 - Service: Active
Libraries:
 - CUDA: 11.4.315
 - cuDNN: 8.6.0.166
 - TensorRT: 5.1.1
 - VPI: 2.2.7
 - Vulkan: 1.3.204
 - OpenCV: 4.5.4 - with CUDA: NO

I've encountered the same issue as well, but it's intermittent.
image

I have kept the detail by log.
image

It's weird. The value in actual file is minus.
image

Hi @balabala65535

I apologize for this huge delay in replying.
The GPU output you read from jtop is exactly what you notice from the NVIDIA output in devices/virtual/thermal.
With Jetpack 5 and Jetpack 6, when the GPU is not working also, the GPU output is equivalent to -256, which in jtop is "Offline".

The GPU output on your host and on your docker must be the same. The Jtop service reads this output one time and provides for every Jtop client the same value.

I updated jtop for the latest version of Jetpack 6.0

sudo pip3 install -U jetson-stats

I am closing this issue. If you encounter any new issues related to Docker, please feel free to reopen this issue.