JorgeB

Delete /config/plugins/community.applications.plg from the flash drive, reboot, and then go to APPS and try to install it.

This is not a system-wide Unraid OOM. It is a Docker OOM caused by one container exhausting its configured RAM + swap limit (RAM limit: 1.5 GB Swap limit: 1.5 GB), so maybe increase that; it appears to be Paperless-ngx

Create a stock install using the USB creator; see if that boots. If it still doesn't, Unraid is not the problem, but most likely the flash drive or a BIOS setting, if that boots, restore only the config folder from the backup, overwriting all files.

Try uninstalling the current version you have, then go to APPS and install again (assuming you are running 7.0.x or later)

You can try this (you may need to reconfigure some Connect options/settings if they were already set) Uninstall the Connect plugin first (Plugins tab) Then open a terminal window and type or copy/paste unraid-api stop mv /boot/config/plugins/dynamix.my.servers/ /boot/config/plugins/dynamix.my.servers.bak/ unraid-api start Then reinstall the Connect plugin and try again

I believe that warning always shows; it doesn't try to detect whether you are actually virtualizing Unraid or not.

Nothing obvious suggesting a disk failure. The server is under significant I/O load, so the rebuild appears to be heavily throttled in favor of normal array activity. Leave the rebuild running, but pause qBittorrent, media imports/scans, backups, and other array-writing workloads for 10–15 minutes and see whether the speed increases. If it doesn't, disable docker and VM services, reboot, let the rebuild run for 5 minutes or so and post new diags.

Try this: stop the array, unassign the device that is not part of the pool, start the array, stop the array, reassign that device to the pool, start the array, post new diags.

The diagnostics show that this is not starting as a Mellanox driver failure. The complete Helios connection drops: its USB hubs disconnect, followed by the Thunderbolt retimer and device. Only then do both Mellanox functions report that their PCI slot is unavailable. The enclosure reconnects a few seconds later, and the PCI bus is rescanned, but the Mellanox card does not recover because PCI reads still fail. The entire enclosure then disconnects and reconnects twice more. Rebooting fully restores the card. This also does not appear to be normal system sleep or USB autosuspend. There is no suspend event in the log, and USB autosuspend and PCIe port runtime power management are already disabled. ASPM L1 does remain enabled by the BIOS across the Thunderbolt path, however. I would suggest the following tests: 1. Temporarily use the onboard Realtek NIC for management so the server remains accessible when the Mellanox disappears. 2. Confirm whether the Helios power LED or fan cycles when the problem occurs. 3. Reseat the card, Thunderbolt cable and enclosure power connection. Test with the supplied short certified cable or another certified Thunderbolt cable. 4. Run the enclosure fan at its higher setting for comparison. 5. If possible, install the Mellanox card internally and run it for longer than the usual failure interval. If it remains stable internally, that would strongly point to the enclosure, cable or Thunderbolt path. 6. Check with MSI and OWC for BIOS, Thunderbolt NVM and Helios firmware updates. It would also be worth asking OWC whether the MCX4121A-ACAT is qualified for this enclosure. 7. If the BIOS provides an ASPM or Thunderbolt power-management option, disable it for one test. Adding pcie_aspm=off may not be sufficient because current Linux can leave the firmware’s existing ASPM configuration unchanged. USB Manager is installed, but its messages occur after the kernel reports the physical disconnect, so it does not currently look like the cause. A Safe Mode test without USB Manager/ USB-IP could still be tried later if the physical, firmware and internal-card tests are inconclusive.

Please post the diagnostics.

If you can boot in safe mode, it suggests a plugin problem, you can rename all .plg files to .bak in /config/plugins on the flash drive, confirm that it works, then start renaming one or a few at a time back to .plg and retest until you find the culprit.

There's already a bug report and it should be fixed for the next release. https://product.unraid.net/b/unraid-os-bugs/p/7-3-2-dashboard-nginx-log-spam-from-stale-docker-template-fallback-icons

Thanks for the outputs; this identifies the responsible container and lines up closely with the previous diagnostics. The earlier diagnostic contained 800+ Python processes owned by the same container account, using about 19 GiB in total. Your latest cgroup result attributed about 21 GiB to Beaver Habit Tracker, and stopping that container immediately released approximately the same amount. This therefore looks less like one Python process leaking 21 GiB and more like repeated Python healthcheck processes accumulating. Beaver Habit Tracker’s current image defines a Python healthcheck every 30 seconds. The previous Docker log also repeatedly reported that it could not drain container exec processes, which would explain why old healthchecks remained while new ones continued to start. The 2 GiB limit is a good way to protect the rest of the server, but monitor whether the container becomes unhealthy, is OOM-killed, or repeatedly restarts. While the system is healthy, post: docker inspect --format 'Image={{.Config.Image}} ImageID={{.Image}} Memory={{.HostConfig.Memory}} PidsLimit={{.HostConfig.PidsLimit}} Healthcheck={{json .Config.Healthcheck}}' Beaver-Habit-Tracker docker inspect --format 'OOMKilled={{.State.OOMKilled}} RestartCount={{.RestartCount}} Health={{json .State.Health}}' Beaver-Habit-Tracker docker top Beaver-Habit-Tracker -eo pid,ppid,state,rss,etime,comm,args If it starts growing again, also capture the container cgroup’s memory.stat, memory.events, pids.current, and pids.max before stopping it. As an A/B test, you could temporarily add --no-healthcheck to this container while leaving the application otherwise unchanged. If the process and memory growth stops, that would strongly confirm the healthcheck-exec mechanism. A PID limit based on the container’s normal process count may also contain this failure more directly, but neither limit explains the initial application or I/O stall.

One last thing you may try is to disconnect any devices that were ever used or tried to be used with that pool and retry the recovery options. It doesn't look like the issue here; but in some rare cases, another damaged device from the pool can interfere.

See here firts: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#findComment-819173

Lots of corruption on the docker image, try recreating it (but reboot after deleting it since there was also a busy error) https://docs.unraid.net/unraid-os/troubleshooting/common-issues/docker-troubleshooting/#re-creating-the-docker-image-file Then: https://docs.unraid.net/unraid-os/troubleshooting/common-issues/docker-troubleshooting/#reinstalling-docker-applications Also see below if you have any custom docker networks: https://docs.unraid.net/unraid-os/troubleshooting/common-issues/docker-troubleshooting/#restoring-docker-custom-networks

You can, but I don't expect that to help in this case. You can also try the recovery options here: https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#findComment-543490

Does that happen using a different flash drive with a stock install? No key needed.

Please post the diagnostics after array start (with 7.3.2)

Unraid should automatically have converted the pool to RAID 1 once you added the 2nd device, but you can post the diagnostics to confirm.

Would need previous diags to see the actual pool status; possibly it wasn't actually running as a mirror. The problem is that other than the missing device, the superblock on the remaining pool device is damaged, try this btrfs-select-super -s 1 /dev/sdg1 Then restart the array and post new diags.

You can install the key in Tools - Registration. Note that if you were using the flash drive for the key, it still needs to be connected; the other option is to transfer it to TPM, if available; the key cannot be tied to an internal boot pool.

Creating the internal boot drive should copy all the config files to the new internal pool, assuming it has enough space and it's working correctly. My recommendation would be to boot from the flash drive again, re-run the wizard. After you click to reboot look at the monitor output and confirm the rsync finishes copying the old config without any errors before the server reboots.

Any idea what happened to the missing device?

Mostly likely not, it just makes the logs very difficult to check. Not sure; don't remember seeing a similar issue before, an IPMI firmware update may help.