Giters

ROCm / ROCm

AMD ROCm™ Software - GitHub Home

Home Page:https://rocm.docs.amd.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

LXC Ryzen 7 (5800H)

hkfuertes opened this issue · comments

commented

Hello, I'm trying to run ollama faster on an LXC container in my proxmox homelab. I have some questions, and maybe you could guide me through them...

  1. I'm using my APU, and my understanding is that I could use the iGPU to offload ollama onto ROCm, is that correct? my homelab its a minipc so no option to add a second card for VMs...
  2. On the same homelab I have Jellyfin using the APU (i believe) with VAAPI to transcode my videos (and as I only have one iGPU... I cannot exclusively use it in a vm, no passthrought, is this also correct?)
  3. So I installed mesa drivers on proxmox host and bind-mounted /dev/dri, /dev/kfd onto the ollama container. For this container I tried several things (ubuntu, debian, and most recently arch... the outcome is the same for all of them, but I will write down the steps for the last one...)

  • pacman -Syu libva-mesa-driver libva-utils rocm-hip-sdk rocm-opencl-sdk rocminfo

  •  #/opt/rocm/bin/rocminfo
 ROCk module is loaded
 hsa api call failure at: /usr/src/debug/rocminfo/rocminfo-rocm-6.0.0/rocminfo.cc:1219
 Call returned HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.
  1. Will I get any improvements using the iGPU (shared memory)? Because if the improvement is going to be zero compared to the CPU, I will not bother anymore 😄 .

Sorry that I write all down... Thank you in advance for the help that I could get!

commented

Following up:

[root@ollamarch ~]# /opt/rocm/bin/rocm-smi 
Exception caught: map::at
======================================== ROCm System Management Interface ========================================
================================================== Concise Info ==================================================
Device  [Model : Revision]    Temp    Power  Partitions      SCLK  MCLK     Fan  Perf  PwrCap       VRAM%  GPU%  
        Name (20 chars)       (Edge)  (Avg)  (Mem, Compute)                                                      
==================================================================================================================
0       [0x0123 : 0xc1]       34.0°C  11.0W  N/A, N/A        None  1200Mhz  0%   auto  Unsupported    1%   0%    
        Lucienne                                                                                                 
==================================================================================================================
============================================== End of ROCm SMI Log ===============================================

Does this means that the GPU is recognized (and usable) by the container?