Skip to main content

HW-Acceleration in LXC container

Vorbereitung Proxmox

Update Proxmox toauf latestdie neueste Version aktualisieren:
apt update && apt dist-upgrade -y

RebootNach afterdem theUpgrade upgradeneu starten:
reboot

InstallInstallation sudo,von git, gcc, make andund headerden filesHeader-Dateien:
apt install sudo git gcc make pve-headers-$(uname -r)

InstallAktuellen latestNVIDIA-Treiber NVIDIA driverinstallieren:
UsingHierfür listdie herefolgende asListe reference:als Referenz verwenden: GitHub - keylase/nvidia-patch

mkdir /opt/nvidia
cd /opt/nvidia
wget https://download.nvidia.com/XFree86/Linux-x86_64/418.56/<version>/NVIDIA-Linux-x86_64-<version>.run
chmod +x NVIDIA-Linux-x86_64-<version>.run
./NVIDIA-Linux-x86_64-<version>.run --no-questions --ui=none --disable-nouveau

DriverDer willTreiber createerstellt die Datei /etc/modprobe.d/nvidia-installer-disable-nouveau.conf andund disabledeaktiviert theden nouveauNouveau-Treiber. driver.Das Verifybitte thismit byder checking the contents of the createderstellten .confconf-Datei fileüberprüfen:

more /etc/modprobe.d/nvidia-installer-disable-nouveau.conf

#generated# generated by nvidia-installer
blacklist nouveau
options nouveau modeset=0

RebootProxmox tozum disableDeaktivieren nouveauder driversNouveau-Treiber neu starten:
reboot

RunFühre thedas NVIDIANVIDIA-Installationsprogramm installer,erneut whichaus, willum nowdie completeTreiberinstallation the driver install.abzuschließen:
/opt/nvidia/NVIDIA-Linux-x86_64-418.56.<version>.run --no-questions --ui=none

CheckKontrolliere, thatob der Befehl nvidia-smi worksrichtig nowausgeführt wird:
nvidia-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56525.60.11    Driver Version: 418.56525.60.11    CUDA Version: 10.112.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForceQuadro GTXP400         980 Ti  OffOn   | 00000000:43:05:00.0 Off |                  N/A |
| 23%34%   61C32C    P0P8    72WN/A /  275WN/A |      0MiB1MiB /  6075MiB2048MiB |      2%0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        GPUID   PID   Type   Process nameID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Create/updateErstellen the/ Aktualisieren der Datei modules.conf filefür forden bootBootvorgang:
nano /etc/modules-load.d/modules.conf

# /etc/modules-load.d/modules.conf
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

nvidia
nvidia_uvm

GenerateGeneriere thedas initramfsInitramfs imageImage withmit theder newneuen modules.confconf:
update-initramfs -u

CreateErstelle rulesRegeln tozum loadLaden theder driversTreiber onbeim bootBooten forsowohl bothfür NVIDIA andals nvidia_uvmauch für nvidia_uvm:
nano /etc/udev/rules.d/70-nvidia.rules

# /etc/udev/rules.d/70-nvidia.rules
# Create /nvidia0, /dev/nvidia1 … and /nvidiactl when nvidia module is loaded
KERNEL=="nvidia", RUN+="/bin/bash -c '/usr/bin/nvidia-smi -L'L && /bin/chmod 666 /dev/nvidia*'"
#
# Create the CUDA node when nvidia_uvm CUDA module is loaded
KERNEL=="nvidia_uvm", RUN+="/bin/bash -c '/usr/bin/nvidia-modprobe -c0 -u'u && /bin/chmod 0666 /dev/nvidia-uvm*'"

InstallInstalliere GitHub - NVIDIA/nvidia-persistenced:
git clone https://github.com/NVIDIA/nvidia-persistenced.git
cd nvidia-persistenced/init
./install.sh

Checking for common requirements...
  sed found in PATH?  Yes
  useradd found in PATH?  Yes
  userdel found in PATH?  Yes
  id found in PATH?  Yes
Common installation/uninstallation supported

Creating sample System V script... done.
Creating sample systemd service file... done.
Creating sample Upstart service file... done.

Checking for systemd requirements...
  /usr/lib/systemd/system directory exists?  No
  /etc/systemd/system directory exists?  Yes
  systemctl found in PATH?  Yes
systemd installation/uninstallation supported

Installation parameters:
  User  : nvidia-persistenced
  Group : nvidia-persistenced
  systemd service installation path : /etc/systemd/system

Adding user 'nvidia-persistenced' to group 'nvidia-persistenced'... done.
Installing sample systemd service nvidia-persistenced.service... done.
Enabling nvidia-persistenced.service... done.
Starting nvidia-persistenced.service... done.

systemd service successfully installed.

DoublePrüfen, checkob thatder theDienst serviceläuft isund runningaktiviert and enabledist:
systemctl status nvidia-persistenced

# generated by nvidia-installer
blacklist nouveau
options nouveau modeset=0
root@pve:~#  nano /etc/modules-load.d/modules.conf
root@pve:~# nano /etc/udev/rules.d/70-nvidia.rules
root@pve:~# systemctl status nvidia-persistenced
● nvidia-persistenced.service - NVIDIA Persistence Daemon
     Loaded: loaded (/etc/lib/systemd/system/nvidia-persistenced.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2019-04-042023-01-05 13:45:4411:03:33 CDT;CET; 38s1h 47min ago
    Process: 133561000 ExecStart=/usr/bin/nvidia-persistenced --user nvidia-persistenced (code=exited, status=0/SUCCESS)
   Main PID: 133621005 (nvidia-persiste)
      Tasks: 1 (limit: 19660)38358)
     Memory: 996.776.0K
        CPU: 262ms292ms
     CGroup: /system.slice/nvidia-persistenced.service
             └─133621005 /usr/bin/nvidia-persistenced --user nvidia-persistenced

AprJan 0405 13:45:4411:03:33 ripperpve systemd[1]: Starting NVIDIA Persistence Daemon...
AprJan 0405 13:45:4411:03:33 ripperpve nvidia-persistenced[13362]1005]: Started (13362)1005)
AprJan 0405 13:45:4411:03:33 ripperpve systemd[1]: Started NVIDIA Persistence Daemon.

RebootNeustart andund verifyÜberprüfung, allob NVIDIAalle devicesNVIDIA-Geräte comerichtig upangezeigt werden:
reboot
ls -l /dev/nv*

crw-rw-rw- 1 root root 195,   0 AprJan  45 13:4911:03 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 AprJan  45 13:4911:03 /dev/nvidiactl
crw-rw-rw- 1 root root 195, 254 AprJan  45 13:4911:03 /dev/nvidia-modeset
crw-rw-rw- 1 root root 235,507,   0 AprJan  45 13:4911:03 /dev/nvidia-uvm
crw-rw-rw- 1 root root 235,507,   1 AprJan  45 13:4911:03 /dev/nvidia-uvm-tools

PatchPatchen nvidiades driverNVIDIA toTreibers, removeum maxdie encodingBeschränkung sessionsder maximalen Sitzungen zu entfernen:
cd /opt/nvidia
git clone https://github.com/keylase/nvidia-patch.git
cd nvidia-patch
./patch.sh

Detected nvidia driver version: 418.56<version>
Attention! Backup not found. Copy current libnvcuvid.so to backup.
751706615c652c4725d48c2e0aaf53be1d9553d5  /opt/nvidia/libnvidia-encode-backup/libnvcuvid.so.418.56<version>
ee47ac207a3555adccad593dbcda47d8c93091c0  /usr/lib/x86_64-linux-gnu/libnvcuvid.so.418.56<version>
Patched!

InErstelle in Proxmox createeinen a newneuen LXC oroder editbearbeite thedie container conf fileContainer-Config (in mymeinem case,Fall containerContainer 100)105) andund addfüge thedie LXCLXC-Gruppe groupund andMounts mountsam toEnde the end of theder .confconf-Datei filehinzu:
nano /etc/pve/lxc/100.105.conf

lxc.cgroup.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup.cgroup2.devices.allow: c 235:507:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

Die

Notebeiden 1:obigen the two groups aboveGruppen (195 andund 235)507) comestammen fromaus theder Ausgabe von ls -l /dev/nv* output earlier

Note 2: In somemanchen casesFällen theändert groupsich fordie Gruppe für -uvm andund -uvm-tools willbeim changeNeustart. onWenn reboot.du Infeststellst, mydass casedas theypassiert, togglefüge betweenalle 235Gruppen, anddie 511.du Ifsiehst, youzur findallow-Liste thishinzu, happeningum addzu allverhindern, thedass groupsdu that you see occuring to the allow list to prevent needing to change thedie .confconf-Datei fileständig constantly.ändern Formusst.
Zum example, mine currently says:Beispiel:
lxc.cgroup.devices.allow: c 195:* rwm
lxc.cgroup.devices.allow: c 235:* rwm
lxc.cgroup.devices.allow: c 511:* rwm

StartStarte theden LXCLXC-Container containerund andlade downloadden thegleichen sameNVIDIA-Treiber NVIDIAin driverden toContainer the container.herunter:
RunFühre thedie installation,Installation butaus. doDas notInstallationsprogramm installfragt, the driver. The installer asks me if I want to installob libglvnd
because Iinstallieren havewerden ansoll, incompleteweil installation.die IInstallation haveunvollständig alwaysist. saidHier “dann bitte: "Don’t install libglvnd”libglvnd". Everything
elseAlles Iandere answerwird yesmit to"yes" andbeantwortet ignoreund theWarnungen warnings.
ignoriert.

mkdir /opt/nvidia
cd /opt/nvidia
wget https://download.nvidia.com/XFree86/Linux-x86_64/418.56/<version>/NVIDIA-Linux-x86_64-<version>.run
chmod +x NVIDIA-Linux-x86_64-<version>.run
./NVIDIA-Linux-x86_64-<version>.run --no-kernel-module

RunFühre nvidia-smi toaus, checkum thedie installation,Installation alsozu checküberprüfen. thatPrüfe theauch, Nvidiaob devicesdie arerichtigen present.NVIDIA-Geräte vorhanden sind:
nvidia-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56525.60.11    Driver Version: 418.56525.60.11    CUDA Version: 10.112.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForceQuadro GTX 980 TiP400         Off  | 00000000:43:05:00.0 Off |                  N/A |
| 0%34%   60C32C    P8    35WN/A /  275WN/A |      1MiB /  6075MiB2048MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        GPUID   PID   Type   Process nameID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

ls -l /dev/nv*

crw-rw-rw- 1 nobodyroot nogrouproot 195, 254 AprJan  45 19:2010:03 /dev/nvidia-modeset
crw-rw-rw- 1 nobodyroot nogrouproot 235,507,   0 AprJan  45 19:2010:03 /dev/nvidia-uvm
crw-rw-rw- 1 nobodyroot nogrouproot 235,507,   1 AprJan  45 19:2010:03 /dev/nvidia-uvm-tools
crw-rw-rw- 1 nobodyroot nogrouproot 195,   0 AprJan  45 19:2010:03 /dev/nvidia0
crw-rw-rw- 1 nobodyroot nogrouproot 195, 255 AprJan  45 19:2010:03 /dev/nvidiactl

LogLogge intodich in Jellyfin / Plex andein enableund hardwareaktiviere accelerationdie Hardwarebeschleunigung

Um

Note:zu Toüberprüfen, verifyob thatder yourMedienserver mediaeinen serverThread iszur kickingHardwarebeschleunigung offstartet, amuss hardware acceleration thread, you will need to callman nvidia-smi onauf thedem host,Host notaufrufen, fromnicht withinaus thedem lxcLXC-Container container.
heraus.

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.11    Driver Version: 525.60.11    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P400         On   | 00000000:05:00.0 Off |                  N/A |
| 34%   36C    P0    N/A /  N/A |    760MiB /  2048MiB |     86%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     49900      C   ...ib/jellyfin-ffmpeg/ffmpeg      756MiB |
+-----------------------------------------------------------------------------+
Back to top