JorgeB

Thanks, that helps. Only a few file locks are held, so shfs is not stuck on locks. Two shfs threads are reading continuously from a file on the cache pool, and most of the CPU time is the kernel checking Btrfs checksums for those reads. So shfs is passing on a heavy, nonstop read load, most likely from the docker image or appdata. When you have the problem again, please run this to see which file is being read: P=$(pgrep -f 'shfs /mnt/user -disks') for n in $(seq 40); do for t in /proc/$P/task/*; do set -- $(cat $t/syscall 2>/dev/null); [ "$1" = 17 ] && readlink /proc/$P/fd/$(($2)); done; sleep 0.5; done | sort | uniq -c > /boot/shfs-files.txt docker stats --no-stream >> /boot/shfs-files.txt Then post shfs-files.txt from your flash drive. After that, the Docker test from my last post is the most important one. Set Enable Docker to No in Settings > Docker and check whether shfs CPU drops. If it does, start the containers a few at a time. The BLOCK I/O column from docker stats should also help find the one doing all the reading. The NFS test is still worth doing, but it is less likely to matter now. Good call on backing up the cache first.

You are correct; I had looked at the btrfs info and device 2 is typically cache2, but sdc is cache1 according to the diags, so use cache1 (sdc) to reimport the pool Data Data Metadata Metadata System System Id Path single RAID1 RAID1 DUP RAID1 DUP Unallocated Total Slack -- --------- -------- --------- --------- --------- -------- -------- ----------- --------- ----- 1 missing - 392.00GiB 1.00GiB - 32.00MiB - 538.48GiB 931.51GiB - 2 /dev/sdc1 38.00GiB 392.00GiB 1.00GiB 2.00GiB 32.00MiB 64.00MiB 498.42GiB 931.51GiB - -- --------- -------- --------- --------- --------- -------- -------- ----------- --------- ----- Total 38.00GiB 392.00GiB 1.00GiB 1.00GiB 32.00MiB 32.00MiB 1.01TiB 1.82TiB 0.00B Used 36.31GiB 391.72GiB 437.39MiB 285.72MiB 48.00KiB 48.00KiB

Syslog reports the balance was canceled: Oct 23 08:39:34 Tower kernel: BTRFS info (device sdc1): found 351 extents, stage: move data extents Oct 23 08:39:34 Tower kernel: BTRFS info (device sdc1): balance: canceled Run it again, with the array running type btrfs balance start -f -dconvert=single,soft -mconvert=dup,soft /mnt/cache When that's done: btrfs device remove missing /mnt/cache

To confirm what Mainfrezzer found: I checked that IP directly, and it serves the Cloudflare origin certificate for makemkv.com, so it is MakeMKV's own server, not something suspicious. The MakeMKV app connects to it for its update check and to download its keys file. Your diagnostics also show the same containers, including MakeMKV, were already running before you updated, so the update itself is not the cause. Your server rebooted for the update and restarted all containers, which is why the alerts seemed to start then. Note that your Ripper container also includes MakeMKV, so either container can make this connection. If you want to stop the alerts, add an allow rule on the firewall for that IP. Or keep the containers stopped when you don't use them. Nothing in the diagnostics points to a compromise.

That error suggests the previous balance didn't finish; post new diags, please.

That IP is not one Unraid OS uses. It belongs to a Russian hosting company and resolves to a shared web hosting server, so it hosts many unrelated websites. Unraid's own servers (updates, license keys, Connect, Community Applications) are all behind Cloudflare. The update itself is very unlikely to be the cause, and the timing is most likely a coincidence, for example a container or plugin that was updated or restarted at the same time. Your firewall only sees the server's IP. Docker containers on bridge networks and VMs using NAT go out with that same IP, so the first step is to find out which process is making the connection. While the firewall is reporting it, open a terminal on the server and run: ss -tnp | grep 185.84.108.20 If this shows a process, it gives you its name and PID. To check whether that PID belongs to a container: cat /proc/<PID>/cgroup docker ps --no-trunc | grep <container id from the previous output> If ss shows nothing, the connection is probably coming from a container on a bridge network or from a VM. Check with: grep 185.84.108.20 /proc/net/nf_conntrack The src= address in the output shows where the connection starts. A 172.17.x.x type address is a Docker container, and your VM's IP points to that VM. Also check whether your firewall logs the DNS name or the SNI hostname for that connection. That tells you which website on the shared server is being contacted, and that usually explains the cause. If you still can't find the source, disable Docker and VMs in Settings and see if the alerts stop. If they stop, start the containers one at a time. If they don't stop, boot in safe mode to rule out plugins. Then please post your diagnostics (Tools > Diagnostics), ideally taken while the connections are happening, together with the output of the commands above.

It should have continued by itself, just wait until you get the cursor back. That means it's done.

That is normal, and since the pool is already using a single device, it's OK.

The diagnostics show the shfs process for /mnt/user using about two CPU cores the whole time since boot, with high system CPU. That is not normal, and I would like to see what it is doing. With the problem present, open a terminal and run: P=$(pgrep -f 'shfs /mnt/user -disks') top -H -b -n1 -p $P | head -30 > /boot/shfs-debug.txt grep -c FLOCK /proc/locks >> /boot/shfs-debug.txt for i in 1 2 3; do cat /proc/$P/task/*/stack >> /boot/shfs-debug.txt; echo ---- >> /boot/shfs-debug.txt; sleep 3; done Then post the shfs-debug.txt file from your flash drive. After that, please also try these tests and say whether shfs CPU goes down after each one: 1. Settings > Docker, set Enable Docker to No. If the CPU drops, start the containers again a few at a time to find which one causes it. 2. NFS is enabled but you are not exporting any share. Disable it in Settings > NFS, reboot, and check again. Separately, your cache pool has a more important problem. The syslog shows Btrfs "parent transid verify failed" errors on both copies of the metadata, so a scrub cannot fix them, and the docker image also shows corruption. Back up anything important on the cache now, and plan to reformat the pool and recreate the docker image. Both MX500 SSDs are also well past their rated endurance (189% and 253% used), so consider replacing them at the same time. When you recreate it, set the system share to use the cache only, with no secondary storage, and enable Settings > Global Share Settings > Permit exclusive shares. Do the same for appdata after its files on disk3 and disk4 are moved to the cache. The docker image and appdata will then bypass shfs completely, which is better for performance anyway.

Pool has a missing device, so the new writes created single profile chunks, this means you no longer can use the GUI since the pool is not fully redudntant. With the array running type btrfs balance start -f -dconvert=single,soft -mconvert=dup,soft /mnt/cache When that's done: btrfs device remove missing /mnt/cache Then stop the array and import the pool with the remaining device: on main click on the first device for that pool and then "remove pool" back on main, create a new pool with the same name and 1 slot assign the good pool device (cache2), leave the filesystem set to auto start the array to import the pool Post new diags, but if all look normal, you can then add another device if you like to mirror it again.

The plugin is gone now, but it left old container entries inside the Docker vDisk itself. Deleting containers_backup does not remove those. 7.2.4 hides them, which is why everything looks fine there. On 7.3.4 they are loaded again, so old versions of some containers (for example overseerr and paperless-ng) come back. Because they use the same names as the current containers, which ones show up can change after each reboot. Do this on 7.3.4. These entries are hidden on 7.2.4, so they cannot be removed there. 1. Remove the broken entries, as Kilrah suggested. You can do it in one command: grep -o 'RWLayer of container [0-9a-f]*' /var/log/syslog | awk '{print $4}' | sort -u | xargs docker container rm 2. Go to Settings → Docker, set Enable Docker to No, then set it back to Yes. 3. On the Docker page, remove any old containers you no longer use, for example overseerr and paperless-ng, if they still show up. 4. Reinstall any missing containers from Apps → Previous Apps. They use the same templates and appdata, so your settings and data are kept. 5. Reboot once and check that the same set of containers is there. If anything is still wrong after that, delete the Docker vDisk and recreate it as I described earlier. That shoudl clear everything for sure.

Something is missing here, a new config by itself will not touch any of the data disks.

Does it happen without Tailscale? Assuming you are local and can test.

Yes, the author should still see it.

The problem was that at some point you mounted the second cache SSD with the Unassigned Devices plugin while it was still part of the pool. That mounted the whole cache filesystem under /mnt/disks, and it was still mounted there, so Unraid could not remove the device or format the pool, because the devices are in use. The reboot resolved the issue.