ZooMass

I think that I have at last figured out what the problem was. When creating custom Docker networks, the Docker daemon default address pools would collide with the networking configuration on my Unraid host, and interrupt any network connections I had to the host. I think this is a manifestation of the problem described here: https://noted.lol/fixing-docker-network-problem/ and here: https://old.reddit.com/r/selfhosted/comments/1az6mqa/psa_adjust_your_docker_defaultaddresspool_size/ Those links suggest to configure the default address pools in the /etc/docker/daemon.json but this file is not persistent across reboots on Unraid, so instead I figured out the command-line equivalent, and edited /boot/config/docker.cfg to add DOCKER_OPTS="--default-address-pool base=172.16.0.0/12,size=24" I've only added this recently at my latest encounter with this issue, so I'm not sure that I'll never encounter this issue again. After trying repeatedly to recreate my Docker image and having multiple unpredictable lockups upon docker network create mynetwork calls, adding this parameter finally let me add all of my custom networks without locking up the server. I hope that nobody else has to deal with this, but if someone comes across this thread, I hope this solves it for you!

Adding a few more details: - I replaced my Unraid USB within the last year. It's branded by Micro Center, not a brand name but it's probably not some cheap random flash memory. - I don't have frequent read-writes to my USB. All the logs I'm aware of write to the cache or array. - Cache where system dataset lives are two Samsung NVMe SSDs, purchased new within last 2 years. SMART status is passing on both. - I just had this system freeze happen again while trying to recover from a previous one. It froze at the points when I was only running 4 containers and was running a docker network add command that hung, and then so did the system.

I've experienced this issue occasionally over several years across Unraid 6.11.x-6.12.14 versions where when I try to start a container (no particular pattern, it has happened from starting a container via Docker web GUI, via SSH docker run, and via docker compose up), suddenly all of Unraid becomes unresponsive and must be hard shutdown (physically kill power). When this happens, every subsequent Unraid boot also locks up the server (GUI, SSH connections) once Docker starts. Each time, before hanging indefinitely, the last syslog lines I get look like this: Dec 1 21:24:12 bananalord emhttpd: shcmd (151): /usr/local/sbin/mount_image '/mnt/cache/system/docker/' /var/lib/docker 256 Dec 1 21:24:12 bananalord emhttpd: shcmd (153): /etc/rc.d/rc.docker start Dec 1 21:24:12 bananalord root: starting dockerd ... Dec 1 21:24:13 bananalord avahi-daemon[7353]: Server startup complete. Host name is bananalord.local. Local service cookie is 2033039608. Dec 1 21:24:13 bananalord avahi-daemon[7353]: Service "bananalord" (/services/ssh.service) successfully established. Dec 1 21:24:13 bananalord avahi-daemon[7353]: Service "bananalord" (/services/smb.service) successfully established. Dec 1 21:24:13 bananalord avahi-daemon[7353]: Service "bananalord" (/services/sftp-ssh.service) successfully established. Dec 1 21:24:15 bananalord nmbd[7269]: [2024/12/01 21:24:15.878305, 2] ../../source3/nmbd/nmbd_elections.c:111(check_for_master_browser_fail) Dec 1 21:24:15 bananalord nmbd[7269]: check_for_master_browser_fail: Forcing election on workgroup WORKGROUP subnet 10.0.0.100 Dec 1 21:24:18 bananalord nmbd[7269]: [2024/12/01 21:24:18.881409, 2] ../../source3/nmbd/nmbd_elections.c:41(send_election_dgram) Dec 1 21:24:18 bananalord nmbd[7269]: send_election_dgram: Sending election packet for workgroup WORKGROUP on subnet 10.0.0.100 Dec 1 21:24:20 bananalord nmbd[7269]: [2024/12/01 21:24:20.883171, 2] ../../source3/nmbd/nmbd_elections.c:41(send_election_dgram) Dec 1 21:24:20 bananalord nmbd[7269]: send_election_dgram: Sending election packet for workgroup WORKGROUP on subnet 10.0.0.100 Dec 1 21:24:21 bananalord kernel: Bridge firewalling registered Dec 1 21:24:21 bananalord kernel: Initializing XFRM netlink socket The only solution (that I've begrudgingly become accustomed to doing) are to take the steps I've written in this post. Basically, I have to manually edit the /boot/config/docker.cfg on my boot USB on a separate machine to disable Docker before boot, then I have to delete my Docker directory /mnt/cache/system/docker and then re-enable Docker and re-download all of my containers via Community Apps > Previously Installed > Docker. I have about 120 containers so this takes 1-2 hours to download. For background on my Docker setup, I keep my Docker images on my cache drive, which is a 2-mirror ZFS zpool, with >1TB of unallocated space. I have had this hanging issue regardless of having a docker.img as an xfs volume, btrfs volume, or as a directory. I have most recently been using a directory with a ZFS overlay (so each image appears as a dataset). I have also tried using a zvol according to this guide. My Docker custom network type is ipvlan. What really frustrates me is the unpredictability. It doesn't seem to correspond to having too many containers, too many image layers, too much running on the server. Most recently it has happened when I try to start my Immich Docker Compose. Any help would be appreciated. Diagnostics attached. bananalord-diagnostics-20241201-2132.zip

Is there a way (environment variable or config setting) to change the port which the VNC server uses? Not the NoVNC client, specifically the actual VNC protocol port. 5901 often clashes with Unraid VMs which autostart to 5900 + 1 * however many VMs on VNC there are, which when they are running then prevent this container from starting. I can't just bind mount to another port because this container needs to run on network: host in order to detect NDI outputs on the actual (not internal Docker) network via mDNS.

Sorry for the delay. Ultimately I did have to reinstall Docker again. Thanks to the Community Applications tab Previous Apps it wasn't terrible to re-pull all of the Docker images, but it is sadly the best solution I had. This seems to happen more frequently with an Unraid boot USB that is older/has more writes, as I experience less of this issue after I moved my Unraid key to a new USB drive. Still, I worry about Docker freezing up my entire Unraid server again. I've also tried various Docker directory types, including ZFS native overlay, btrfs image, XFS image, and the latter two but virtualized in a Zpool. I still ran into the lockup issues and do blame it on the boot drive, though I'm not sure how. My older boot drive was not particularly old or worn, was not from an unreliable manufacturer, and did not have constant writes to it such as logs, so I'm really not sure why the drive degraded so quickly. It also makes me worry about my current Unraid boot USB. There is an arbitrary limit that you can transfer your Unraid key to a different drive UUID once a year, and to do it sooner you have to contact Limetech directly, I wish this wasn't necessary. Anyway, just spelling out the steps I take to quickly reinstall all Dockers: - Shut down the server - Unplug boot USB and mount the drive on another computer, preferably not Windows because of possible CLRF line ending issues conversions - Open the file UNRAID/config/docker.cfg and change docker_enabled="yes" to docker_enabled="no" - Unmount the USB and plug it back into the server - Reboot the server - Start array if it does not start automatically. Docker should not start automatically. - cd to the parent path of your Docker directory, for example if your Docker directory is /mnt/cache/system/docker then cd to /mnt/cache/system you should not go to the FUSE overlay /mnt/user/ - Delete the old Docker directory with rm -rf docker or move it away with mv docker docker.old - Go to Unraid web UI tab Settings, under System Settings click Docker. Confirm your Docker directory path does not exist (docker.old will not cause a conflict, just no dir named docker) Then set Enable Docker to Yes and Apply. - Docker is now enabled and there should be no images. Your container configs still live safely on your boot USB. - Go to Unraid web UI tab Apps and on the left side click Previous Apps > Docker. You should see a list of all of your previously installed apps. Select all of the apps that you want to download again (you can make one continuous selection between pages, just click the next page number and your selected apps should persist) and then on the top of the page click Install Selected Applications. - Wait a long time for Docker to pull down all of your app images again. When this is done go to the Unraid tab Docker and you should see your containers again. Confirm that your app configs persist and start the containers to confirm they work. - Delete /mnt/cache/system/docker.old if you chose to move your old Docker directory, as it is no longer necessary and takes up lots of space.

I am also suddenly experiencing complete system lockups as soon as the Docker service starts. I'm running Unraid 6.12.6 with Docker networks ipvlan. When I manually edit Unraid USB /config/docker.cfg to ENABLE_DOCKER="no" and reboot, the system is ok. As soon as I enable Docker, web UI hangs. Same behavior when VMs are enabled or disabled, and whether in safe mode or not. When I reboot I immediately manually start SSH tail streams for tail syslog, zpool iostat, and htop, all are ok and then freeze when Docker starts. Here is my syslog until it freezes: Jan 30 00:00:37 Tower root: Starting Samba: /usr/sbin/smbd -D Jan 30 00:00:37 Tower root: /usr/sbin/nmbd -D Jan 30 00:00:37 Tower root: /usr/sbin/wsdd2 -d -4 Jan 30 00:00:37 Tower root: /usr/sbin/winbindd -D Jan 30 00:00:37 Tower wsdd2[8123]: starting. Jan 30 00:00:37 Tower emhttpd: shcmd (162): /etc/rc.d/rc.avahidaemon restart Jan 30 00:00:37 Tower root: Stopping Avahi mDNS/DNS-SD Daemon: stopped Jan 30 00:00:37 Tower avahi-daemon[7498]: Got SIGTERM, quitting. Jan 30 00:00:37 Tower avahi-dnsconfd[7507]: read(): EOF Jan 30 00:00:37 Tower avahi-daemon[7498]: Leaving mDNS multicast group on interface br0.IPv4 with address 192.168.1.100. Jan 30 00:00:37 Tower avahi-daemon[7498]: avahi-daemon 0.8 exiting. Jan 30 00:00:37 Tower root: Starting Avahi mDNS/DNS-SD Daemon: /usr/sbin/avahi-daemon -D Jan 30 00:00:37 Tower avahi-daemon[8170]: Found user 'avahi' (UID 61) and group 'avahi' (GID 214). Jan 30 00:00:37 Tower avahi-daemon[8170]: Successfully dropped root privileges. Jan 30 00:00:37 Tower avahi-daemon[8170]: avahi-daemon 0.8 starting up. Jan 30 00:00:37 Tower avahi-daemon[8170]: Successfully called chroot(). Jan 30 00:00:37 Tower avahi-daemon[8170]: Successfully dropped remaining capabilities. Jan 30 00:00:37 Tower avahi-daemon[8170]: Loading service file /services/sftp-ssh.service. Jan 30 00:00:37 Tower avahi-daemon[8170]: Loading service file /services/smb.service. Jan 30 00:00:37 Tower avahi-daemon[8170]: Loading service file /services/ssh.service. Jan 30 00:00:37 Tower avahi-daemon[8170]: Joining mDNS multicast group on interface br0.IPv4 with address 192.168.1.100. Jan 30 00:00:37 Tower avahi-daemon[8170]: New relevant interface br0.IPv4 for mDNS. Jan 30 00:00:37 Tower avahi-daemon[8170]: Network interface enumeration completed. Jan 30 00:00:37 Tower avahi-daemon[8170]: Registering new address record for 192.168.1.100 on br0.IPv4. Jan 30 00:00:37 Tower emhttpd: shcmd (163): /etc/rc.d/rc.avahidnsconfd restart Jan 30 00:00:37 Tower root: Stopping Avahi mDNS/DNS-SD DNS Server Configuration Daemon: stopped Jan 30 00:00:37 Tower root: Starting Avahi mDNS/DNS-SD DNS Server Configuration Daemon: /usr/sbin/avahi-dnsconfd -D Jan 30 00:00:37 Tower avahi-dnsconfd[8179]: Successfully connected to Avahi daemon. Jan 30 00:00:37 Tower emhttpd: shcmd (171): /usr/local/sbin/mount_image '/mnt/cache/system/docker/docker-xfs.img' /var/lib/docker 128 Jan 30 00:00:37 Tower kernel: loop3: detected capacity change from 0 to 268435456 Jan 30 00:00:37 Tower kernel: XFS (loop3): Mounting V5 Filesystem Jan 30 00:00:37 Tower kernel: XFS (loop3): Ending clean mount Jan 30 00:00:37 Tower root: meta-data=/dev/loop3 isize=512 agcount=4, agsize=8388608 blks Jan 30 00:00:37 Tower root: = sectsz=512 attr=2, projid32bit=1 Jan 30 00:00:37 Tower root: = crc=1 finobt=1, sparse=1, rmapbt=0 Jan 30 00:00:37 Tower root: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 Jan 30 00:00:37 Tower root: data = bsize=4096 blocks=33554432, imaxpct=25 Jan 30 00:00:37 Tower root: = sunit=0 swidth=0 blks Jan 30 00:00:37 Tower root: naming =version 2 bsize=4096 ascii-ci=0, ftype=1 Jan 30 00:00:37 Tower root: log =internal log bsize=4096 blocks=16384, version=2 Jan 30 00:00:37 Tower root: = sectsz=512 sunit=0 blks, lazy-count=1 Jan 30 00:00:37 Tower root: realtime =none extsz=4096 blocks=0, rtextents=0 Jan 30 00:00:37 Tower emhttpd: shcmd (173): /etc/rc.d/rc.docker start Jan 30 00:00:37 Tower root: starting dockerd ... Jan 30 00:00:38 Tower avahi-daemon[8170]: Server startup complete. Host name is Tower.local. Local service cookie is 2005410125. Jan 30 00:00:39 Tower avahi-daemon[8170]: Service "Tower" (/services/ssh.service) successfully established. Jan 30 00:00:39 Tower avahi-daemon[8170]: Service "Tower" (/services/smb.service) successfully established. Jan 30 00:00:39 Tower avahi-daemon[8170]: Service "Tower" (/services/sftp-ssh.service) successfully established. Jan 30 00:00:41 Tower kernel: Bridge firewalling registered Jan 30 00:00:41 Tower kernel: Initializing XFRM netlink socket Some text anonymized. Diagnostics attached. This began after setting up Immich with Docker Compose Manager plugin and heavy workload on 3-way mirror importing files. I'm aware of disk1 ZFS specific file corruption that has been benign for months and I am rather certain is irrelevant. Have yet to run memtest, but RAM sticks are a couple months old out of the manufacturer box. Tower-diagnostics-20240130-0021.zip EDIT: My Docker image was most likely full. I resolved this by replacing my system/docker.img with a new vdisk double the size, and fresh pulling my template images. 2ND EDIT: As is the law of the universe, after an entire evening of re-pulling images, as I'm declaring victory and about to go to bed, the instant after I publish my edit, the full system halt returns at the exact same log line as above. My docker.img is absolutely not full. I manually isolate container CPUs to all except the first core (and its second thread) and this started after running a container with that cpusets change. There is something poisonous about Unraid 6.12.6 ZFS with an XFS vDisk Docker.img. It infuriates and terrifies me. Pray for me.

I am also experiencing this issue on Unraid 6.12.4 with Docker image data in an individual share on a ZFS disk. Cannot remove a container through CLI or Force Update through GUI, so the container is stuck. $ zfs version zfs-2.1.12-1 zfs-kmod-2.1.12-1 $ docker rm -f my-app Error response from daemon: container 3ed55f07dde27c39b475b232e8a06f248c19fc09f6464fbaf0276b8c81cab4ff: driver "zfs" failed to remove root filesystem: exit status 1: "/usr/sbin/zfs fs destroy -r cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888" => cannot open 'cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888': dataset does not exist $ zfs list | grep 503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888 cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888-init 136K 863G 91.4M legacy $ zfs unmount cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888 cannot open 'cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888': dataset does not exist $ zfs destroy cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888 cannot open 'cache/docker/503e6d29ad94faaa061257e4ab1c13c30cac283b17ad29d4edc2c5f283428888': dataset does not exist Some relevant GitHub issue discussions: 2015-09-07 moby/moby not exactly the same error but relevant, and I had the same one previously (nuked all Docker image data to solve) 2017-02-13 moby/moby 2019-10-24 moby/moby 2020-06-02 moby/moby 2021-12-13 moby/moby references above issue Based on the 2017-02-13 issue, I tried stopping Docker service and `rm /var/lib/docker` and restarting service, no change. The 2019-10-24 issue says that ZFS 2.2 may introduce a fix. The 2020-06-02 issue and Unraid user BVD recommend creating a zvol virtual disk with a non-ZFS filesystem inside. May have minor performance impact (another filesystem abstraction layer) and also limits the size of the docker.img (I changed to directory image data in the first place because I wanted no limit besides bare metal disk space). Hope that Unraid promptly upgrades to ZFS 2.2 when it is released. Attached diagnostics. tower-diagnostics-20230906-2033.zip

I have the same issue. If I don't type anything for 30 seconds, or if I change the focused window, the Unraid web terminal (web path https://unraid.example.com/webterminal/ttyd/ ) for a split second says Reconnected and the whole terminal session resets as if in a new window. I use Unraid 6.11.5, pfSense 2.7.0, and HAProxy package for pfSense 0.61_11. I use the Unbound DNS Resolver to resolve unraid.example.com to my pfSense IP. HAProxy frontend listens on pfSense port 80/443 on my LAN and serves a wildcard certificate for *.example.com (substituting my actual domain I own) that pfSense ACME signs with LetsEncrypt CA. Here are relevant parts of my HAProxy config file on pfSense at /var/etc/haproxy/haproxy.cfg frontend COM_EXAMPLE_LOCAL-merged bind 192.168.1.1:443 name 192.168.1.1:443 ssl crt-list /var/etc/haproxy/CO_BEREZIN_LOCAL.crt_list mode http log global option socket-stats option http-keep-alive timeout client 30000 acl aclcrt_COM_EXAMPLE_LOCAL var(txn.txnhost) -m reg -i ^([^\.]*)\.example\.com(:([0-9]){1,5})?$ acl unraid.example.com var(txn.txnhost) -m beg -i unraid.example.com use_backend Unraid_Web_UI_ipv4 if unraid.example.com backend Unraid_Web_UI_ipv4 mode http id 12345 log global timeout connect 30000 timeout server 30000 retries 3 source ipv4@ usesrc clientip server Unraid_UI 192.168.1.100:80 id 10110 check inter 1000 It seems to me that the web terminal simply hits the HAProxy timeout limit. Does any HAProxy guru have a handy ACL or other kind of config to make an exception of timeout just for the web terminal route?

Just pulled the latest image as of 2021-10-27, T-Rex miner version 0.24.2, it works again with no Nvidia warnings! Thank you for the quick rollback.

I do have --runtime=nvidia in my extra parameters. I I have the latest ptrfrll/nv-docker-trex:cuda11 as of 2021-10-25. Here is my full docker run command: /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker create --name='trex' --net='container:vpn' --privileged=true -e TZ="xxxxxxxxxx/xxxxxxxxxx" -e HOST_OS="Unraid" -e 'WALLET'='xxxxxxxxxx' -e 'SERVER'='stratum2+tcp://xxxxxxxxxx.ethash.xxxxxxxxxx.xxx:xxxxx' -e 'WORKER'='1080ti' -e 'ALGO'='ethash' -e 'NVIDIA_VISIBLE_DEVICES'='GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx' -e 'PASS'='xxxxxxxxxx' -e 'NVIDIA_DRIVER_CAPABILITIES'='all' -v '/mnt/user/appdata/trex':'/config':'rw' --runtime=nvidia 'ptrfrll/nv-docker-trex:cuda11' And here is my config.json { "ab-indexing" : false, "algo" : "ethash", "api-bind-http" : "0.0.0.0:4067", "api-bind-telnet" : "127.0.0.1:4068", "api-read-only" : false, "autoupdate" : false, "back-to-main-pool-sec" : 600, "coin" : "", "cpu-priority" : 2, "dag-build-mode" : "0", "devices" : "0", "exit-on-connection-lost" : false, "exit-on-cuda-error" : true, "exit-on-high-power" : 0, "extra-dag-epoch" : "-1", "fan" : "t:xx", "gpu-init-mode" : 0, "gpu-report-interval" : 30, "gpu-report-interval-s" : 0, "hashrate-avr" : 60, "hide-date" : false, "intensity" : "0", "keep-gpu-busy" : false, "kernel" : "0", "lhr-low-power" : false, "lhr-tune" : "0", "lock-cclock" : "0", "log-path" : "", "low-load" : "0", "monitoring-page" : { "graph_interval_sec" : 3600, "update_timeout_sec" : 10 }, "mt" : "0", "no-color" : false, "no-hashrate-report" : false, "no-nvml" : false, "no-strict-ssl" : false, "no-watchdog" : false, "pci-indexing" : false, "pl" : "xxxW", "pools" : [ { "pass" : "xxxxxxxxxx", "url" : "stratum2+tcp://xxxxxxxxxx.ethash.xxxxxxxxxx.xxx:xxxxx", "user" : "xxxxxxxxxx", "worker" : "1080ti" } ], "protocol-dump" : false, "reconnect-on-fail-shares" : 10, "retries" : 3, "retry-pause" : 10, "script-crash" : "", "script-epoch-change" : "", "script-exit" : "", "script-low-hash" : "", "script-start" : "", "send-stales" : false, "sharerate-avr" : 600, "temperature-color" : "67,77", "temperature-limit" : 0, "temperature-start" : 0, "time-limit" : 0, "timeout" : 300, "validate-shares" : false, "watchdog-exit-mode" : "", "worker" : "1080ti" }

Hi, I'm running ptrfrll/nv-docker-trex:cuda11 on Unraid 6.9.2 with Unraid Nvidia driver 495.29.05 with only a 1080 Ti, not stubbed. The container used to work fine, until I reinstalled it from CA using the same template I had before, same GPU ID. Now T-Rex repeatedly fails with this warning. 20211025 04:07:46 WARN: Can't load NVML library, dlopen(25): failed to load libnvidia-ml.so, libnvidia-ml.so: cannot open shared object file: No such file or directory 20211025 04:07:46 WARN: NVML error, code 12 20211025 04:07:46 WARN: Can't initialize NVML. GPU monitoring will be disabled. 20211025 04:07:47 20211025 04:07:47 NVIDIA Driver version N/A Any idea what might be causing this missing shared Nvidia library? I can run nvidia-smi just fine on my host. Tried rebooting.

Hi, my syslog gets spammed and 99% filled within minutes of booting up with millions of lines like this Jan 18 23:54:53 server kernel: vfio-pci 0000:09:00.0: BAR 3: can't reserve [mem 0xf0000000-0xf1ffffff 64bit pref] Jan 18 23:54:53 server kernel: vfio-pci 0000:09:00.0: BAR 3: can't reserve [mem 0xf0000000-0xf1ffffff 64bit pref] Jan 18 23:54:53 server kernel: vfio-pci 0000:09:00.0: BAR 3: can't reserve [mem 0xf0000000-0xf1ffffff 64bit pref] I am stubbing my graphics card with this plugin on unRAID 6.8.3. The address 09:00.0 is the device "VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)". HVM and IOMMU are enabled. PCIe ACS override is disabled. The graphics card passthrough (with dumped vbios rom) works in a VM, but on fixed 800x600 resolution (Nvidia drivers installed, Windows VM says there's a driver error code 43), but the VM logs say 2021-01-19T21:57:24.002296Z qemu-system-x86_64: -device vfio-pci,host=0000:09:00.0,id=hostdev0,bus=pci.0,addr=0x5,romfile=/mnt/disk5/isos/vbios/EVGA_GeForce_GTX_1070.vbios: Failed to mmap 0000:09:00.0 BAR 3. Performance may be slow Anybody seen this before? Can't find anything like it on the forum. EDIT: Found some more info. According to booting the server without the HDMI plugged in removed the spamming line. However, after plugging the HDMI back in and booting the VM, the VM logs are repeating lines like 2021-01-19T22:17:27.637837Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x101afe, 0x0,1) failed: Device or resource busy 2021-01-19T22:17:27.637849Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x101aff, 0x0,1) failed: Device or resource busy 2021-01-19T22:17:27.648663Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x4810, 0x1fef8c01,8) failed: Device or resource busy 2021-01-19T22:17:27.648690Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x4810, 0x1fef8c01,8) failed: Device or resource busy 2021-01-19T22:17:27.648784Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x102000, 0xabcdabcd,4) failed: Device or resource busy 2021-01-19T22:17:27.648798Z qemu-system-x86_64: vfio_region_write(0000:09:00.0:region3+0x102004, 0xabcdabcd,4) failed: Device or resource busy Windows device manager still says there are driver errors, and there are console-like artifacts horizontally across the screen, including a blinking cursor, on top of Windows. It seems like the unraid console and the Windows VM (or is it VFIO stubbing?) fight for the GPU. I have yet to try the recommendation in the above linked post to unbind the console at boot with the go script.

Bumping this post because I am dealing with the same issue. I have the same four containers with these template missing warnings, pointing to the same A75G templates, for which Apply Fix shows the same error. I have the following templates on my USB: $ ls -lhAF /boot/config/plugins/dockerMan/templates/templates/ | grep jitsi -rw------- 1 root root 4.3K Apr 25 2020 jitsi-jicofo.xml -rw------- 1 root root 4.0K Apr 25 2020 jitsi-jvb.xml -rw------- 1 root root 13K Apr 25 2020 jitsi-prosody.xml -rw------- 1 root root 7.2K Apr 25 2020 jitsi-web.xml $ ls -lhAF /boot/config/plugins/dockerMan/templates-user/ | grep jitsi -rw------- 1 root root 4066 Nov 10 10:36 my-jitsi_bridge.xml -rw------- 1 root root 4336 Nov 10 10:37 my-jitsi_focus.xml -rw------- 1 root root 7276 Nov 10 10:09 my-jitsi_web.xml -rw------- 1 root root 12837 Nov 10 10:35 my-jitsi_xmpp.xml I renamed my containers according to the filenames in the templates-user folder.

My question is very similar to this one. I have an arch-rtorrentvpn container (VPN disabled) using the network stack of a dedicated arch-privoxyvpn container using --net=container:vpn parameter. I am trying to set up port forwarding on the vpn container for rtorrent. On the arch-rtorrentvpn container, it just automatically acquires the forwarded port when the PIA endpoint being used supports it. I am aware of PIA's next-gen upgrades disabling port forwarding and I am primarily using their Israel, Romania, and CA Montreal servers. The arch-privoxyvpn container connects to those endpoints successfully, but it doesn't do the same automatic port forwarding that the arch-rtorrentvpn and arch-delugevpn containers do. Is there a setting to force this? I assume that the container supports it due to sharing the same container startup procedure across the binhex containers. Manually creating a STRICT_PORT_FORWARD variable in arch-privoxyvpn (like in the other two containers) has no effect. Even though I am using PIA, there is a log line that says: 2020-09-16 15:23:54,195 DEBG 'start-script' stdout output: [info] Application does not require port forwarding or VPN provider is != pia, skipping incoming port assignment Is using the ADDITIONAL_PORTS variable equivalent to just adding a new port to the template? Is the vpn_options variable just extra parameters for the /usr/bin/openvpn command?

+1 demand unit Help us SpaceInvaderOne, you are our only hope (not tagging you because I'm sure you're already annoyed with the last three @'s in this topic).

Having the same problem accessing web UI, I am using the manually created "docker network create container:vpn" and not "--net=container:vpn" extra parameter on unRAID 6.8.3 with Docker version 19.03.5.

I have been experiencing the same issue with Jackett and LazyLibrarian. There has been some discussion of this web UI issue over on binhex-privoxyvpn (I lay out my details there). For the record, a lot of people are using that container instead of binhex-delugevpn as a dedicated VPN container. Any ideas or advice would be useful!

Thank you for the quick response! My setup looks essentially the same as yours, with the VPN container named simply vpn, and unfortunately I still cannot access the web UI, just a 404. One thing I tried changing was that I changed the network from a custom public Docker network I have (to isolate from non-public-facing containers) to simply the bridge network like yours. Client container still receives the VPN IP, but I still can't access the web UI. I tried disabling my adblocker even though it should have no effect, and it in fact does not. The container is named jackettvpn because I modified my existing container, but that container's VPN is disabled.

Thank you for these very clear instructions! I was just looking for something like this after hitting my VPN device license limit, and SpaceInvader One released this timely video. Like a lot of you guys I wanted to use a dedicated container instead of binhex-delugevpn, and this binhex-privoxyvpn is perfect for the job. However, I'm unable to access the client container web UI. I've now tested with linuxserver/lazylibrarian (to hide libgen direct downloads) and linuxserver/jackett (migrating from dyonr/jackettvpn, but also tried with clean image). I'm on unRAID 6.8.3 and I've tried both "docker create network container:vpn" and "--net=container:vpn" extra parameters. (also, for the record, "docker run" complains when you set a custom network:container in the dropdown and also have translated ports, so be sure to remove ports at the same time you change the network). I've added the ports for the client containers (in my two test containers those 5299 and 9117 respectively) to the binhex-privoxyvpn container named vpn, restarted vpn, and rebuilt & restarted the client containers. Still can't reach container web UI on [host IP]:5299 or [host IP]:9117. In the client containers, I can curl ifconfig.io and I receive my VPN IP, so the container networking seems to work fine. The client web UI seems to be the only issue. I've seen a couple people in the comments on SpaceInvader One's video report the same issue. Has anyone else experienced this or fixed it? Would love to have this setup work out!

I'm having trouble using the "Tiered FFMPEG NVENC settings depending on resolution" plugin with ID "Tdarr_Plugin_d5d3_iiDrakeii_FFMPEG_NVENC_Tiered_MKV". It says it can't find my GPU. Command: /home/Tdarr/Tdarr/bundle/programs/server/assets/app/ffmpeg/ffmpeg42/ffmpeg -c:v h264_cuvid -i '/home/Tdarr/Media/Television/Stranger Things/Season 03/Stranger Things - S03E01 - Chapter One- Suzie, Do You Copy [HDTV-1080p].mkv' -map 0 -dn -c:v hevc_nvenc -pix_fmt p010le -rc:v vbr_hq -qmin 0 -cq:V 31 -b:v 2500k -maxrate:v 5000k -preset slow -rc-lookahead 32 -spatial_aq:v 1 -aq-strength:v 8 -a53cc 0 -c:a copy -c:s copy '/home/Tdarr/cache/Stranger Things - S03E01 - Chapter One- Suzie, Do You Copy [HDTV-1080p]-TdarrCacheFile-p1cwX-Dg.mkv' ffmpeg version N-95955-g12bbfc4 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1) configuration: --prefix=/home/z/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/z/ffmpeg_build/include --extra-ldflags=-L/home/z/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/z/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree libavutil 56. 36.101 / 56. 36.101 libavcodec 58. 64.101 / 58. 64.101 libavformat 58. 35.101 / 58. 35.101 libavdevice 58. 9.101 / 58. 9.101 libavfilter 7. 67.100 / 7. 67.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 libpostproc 55. 6.100 / 55. 6.100 Guessed Channel Layout for Input Stream #0.1 : 5.1 Input #0, matroska,webm, from '/home/Tdarr/Media/Television/Stranger Things/Season 03/Stranger Things - S03E01 - Chapter One- Suzie, Do You Copy [HDTV-1080p].mkv': Metadata: encoder : libebml v1.3.5 + libmatroska v1.4.8 creation_time : 2019-07-04T07:03:27.000000Z Duration: 00:50:33.63, start: 0.000000, bitrate: 7850 kb/s Chapter #0:0: start 306.015000, end 354.521000 Metadata: title : Intro start Chapter #0:1: start 354.521000, end 3033.632000 Metadata: title : Intro end Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default) Metadata: BPS-eng : 7205368 DURATION-eng : 00:50:33.573000000 NUMBER_OF_FRAMES-eng: 72733 NUMBER_OF_BYTES-eng: 2732251549 _STATISTICS_WRITING_APP-eng: mkvmerge v21.0.0 ('Tardigrades Will Inherit The Earth') 64-bit _STATISTICS_WRITING_DATE_UTC-eng: 2019-07-04 07:03:27 _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1, fltp (default) ... Stream #0:29 -> #0:29 (copy) Stream #0:30 -> #0:30 (copy) Stream #0:31 -> #0:31 (copy) Stream #0:32 -> #0:32 (copy) Press [q] to stop, [?] for help [hevc_nvenc @ 0x55aaaad84e40] Codec not supported [hevc_nvenc @ 0x55aaaad84e40] No capable devices found Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height Conversion failed! I have an EVGA GeForce GTX 760, obv an older card. nvidia-smi doesn't support it. Tue Mar 10 13:54:11 2020 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 440.59 Driver Version: 440.59 CUDA Version: 10.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 GeForce GTX 760 Off | 00000000:08:00.0 N/A | N/A | | 0% 35C P0 N/A / N/A | 0MiB / 1997MiB | N/A Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 Not Supported | +-----------------------------------------------------------------------------+ However my linuxserver/plex and linuxserver/emby containers do manage to use it for hardware transcoding. I made sure to set all the correct Docker template variables including --runtime=nvidia, NVIDIA_DRIVER_CAPABILITIES=all, NVIDIA_VISIBLE_DEVICES=<GPU ID>, I have Linuxserver Unraid Nvidia 6.8.3 installed. Any tips? I would really like to be able to transcode on the GPU, I've been brutally punishing my CPU for days slowly transcoding on Unmanic

That was it! Yes, I am running 6.6.7 because on 6.7+ my machine experienced the SQLite data corruption bug being investigated and 6.6.7 did not. Thanks for your help!

Just ran the container, tried to start the VM, got this error: internal error: process exited while connecting to monitor: 2019-10-27T19:31:11.597434Z qemu-system-x86_64: -machine pc-q35-3.1,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off: unsupported machine type Use -machine help to list supported machines I am using the --catalina OS flag. Only change I made to the container is that I changed the VM images location to /mnt/cache/domains, but that should be the same as /mnt/user/domains anyway. Anybody seen anything like this? My XML: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>MacinaboxCatalina</name> <uuid>51e88f1c-a2ab-43af-981f-80483d6600c8</uuid> <description>MacOS Catalina</description> <metadata> <vmtemplate xmlns="unraid" name="MacOS" icon="/mnt/user/domains/MacinaboxCatalina/icon/catalina.png" os="Catalina"/> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-3.1'>hvm</type> <loader readonly='yes' type='pflash'>/mnt/user/domains/MacinaboxCatalina/ovmf/OVMF_CODE.fd</loader> <nvram>/mnt/user/domains/MacinaboxCatalina/ovmf/OVMF_VARS.fd</nvram> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='2' threads='1'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='writeback'/> <source file='/mnt/user/domains/MacinaboxCatalina/Clover.qcow2'/> <target dev='hdc' bus='sata'/> <boot order='1'/> <address type='drive' controller='0' bus='0' target='0' unit='2'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/MacinaboxCatalina/Catalina-install.img'/> <target dev='hdd' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='3'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='writeback'/> <source file='/mnt/user/domains/MacinaboxCatalina/macos_disk.img'/> <target dev='hde' bus='sata'/> <address type='drive' controller='0' bus='0' target='0' unit='4'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:71:f5:95'/> <source bridge='br0'/> <model type='vmxnet3'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' websocket='-1' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-usb'/> <qemu:arg value='-device'/> <qemu:arg value='usb-kbd,bus=usb-bus.0'/> <qemu:arg value='-device'/> <qemu:arg value='isa-applesmc,osk=IAMZOOMASSANDIREMOVEDTHEMAGICWORDS'/> <qemu:arg value='-smbios'/> <qemu:arg value='type=2'/> <qemu:arg value='-cpu'/> <qemu:arg value='Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check'/> </qemu:commandline> </domain>

I'm trying to access the web UI through a reverse proxy, but when I go to https://mydomain.com/thelounge it just responds with a page that says "Cannot GET /thelounge/". I'm using update 15.05.19, the latest image as of posting this, and I made no changes to the default template except set the network to my public Docker network (where containers can resolve each other's hostnames). I made no changes to /config/config.js except set "reverseProxy: true,". I use jselage/nginxproxymanager container, here is my location block: location /thelounge { proxy_set_header Host $host; proxy_set_header X-Forwarded-Scheme $scheme; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-For $remote_addr; proxy_pass http://thelounge:9000; auth_request /auth-0; } It's a bit different from the block in https://thelounge.chat/docs/guides/reverse-proxies#nginx but I also tried adding the suggested lines and they made no difference. I also tried using this block from the linuxserver config even though I don't use the linuxserver/letsencrypt container, same thing. # thelounge does not require a base url setting location /thelounge { return 301 $scheme://$host/thelounge/; } location ^~ /thelounge/ { # enable the next two lines for http auth #auth_basic "Restricted"; #auth_basic_user_file /config/nginx/.htpasswd; # enable the next two lines for ldap auth, also customize and enable ldap.conf in the default conf #auth_request /auth; #error_page 401 =200 /login; include /config/nginx/proxy.conf; resolver 127.0.0.11 valid=30s; set $upstream_thelounge thelounge; rewrite /thelounge(.*) $1 break; proxy_pass http://$upstream_thelounge:9000; } Any other ideas?

This may be specific to Code Server itself and not this container, but how do we tell VSCode the subdirectory path it's located on in a reverse proxy?

Hey, that worked, thank so much! Silly of me not to try the line from the article itself.