August 13, 2025Aug 13 If there's a better place to post this, please advise.Here's the backstory......to answer as many "why did/didn't you do ___?" type questions. I was a encountering a lot of random freezing overnight, every night, and figured it was hardware related. Over a period of several weeks, I cleaned the server with compressed airchecked/reseated all cablesreseated RAMran Memtest (passed)fsck'd my flash drivereformatted and rebuilt my flash drive with a backupreplaced my M.2 SATA port multiplier (9 drives, only 4 on-mobo SATA ports)ran SMART tests (all drives passed on the quick test, I'd get random freezing on most of the longer tests, so they would never complete)replaced two of the older spinning HDs (data drives) with new (refurb) larger ones, andreplaced my year-old SSD cache. All of the random freezing meant quite a lot of forced power cycles and unclean shutdowns. Parity check could not complete. Data rebuild after I replaced my two HDs could not complete. I finally just accepted that there would be some data loss, and had to go the New Config route. Again, this is while I was still searching for the true smoking gun; at this point I believed it was my two oldest HDs. Ultimately, replacing the SSD seems to have settled things down with regard to the freezing, so it could have been entirely the SSD, or the SSD + M.2 SATA port multiplier that was/were the culprit(s). In case it's relevant, of my HDs, disks 1 and 3 are original from before the trouble began; disks 2 and 4 are new.At this point, I have a partially working system (more details farther down):All my docker containers seem good to go. đđThe Plugins page doesn't display anything except "No plugins installed." âšī¸đThe VM manager won't even start. âšī¸đDiagnostics are attached. Any and all help is solicited and appreciated! I would really like my VMs back, and to be able to see my plugins.PluginsThey're there! I can see them in the CLI in /boot/config/plugins. In fact, on a fresh reboot, they show up on the Plugins page until the array starts; after that poof, they disappear. Plugins that are embedded into the Settings page are visible and function fine. If I go to Apps, some of them show as "installed," others don't. Reinstalling does nothing. I've included a screenshot that shows two web browsers displaying the blank page (Brave - shields down, and Falkon - ad-blocker disabled), and the CLI showing everything in its place.LibvirtI've looked around the forums and the web for help here, and none of the solutions have worked for me. I've rebooted more times than I can count, tried deleting the libvirt.img, starting the service manually (included in screenshot), and probably some other things I've since forgotten.My /etc/libvirt folder exists, but is empty. I also have /etc/libvirt- and have experimented with copying the contents of that folder to /etc/libvirt, but it didn't change anything. I also noticed some odd things with /var/log/libvirt/virtlogd.log and /var/log/libvirt/virtlockd.log - they didn't exist. I created them, tried to restart the service again, and got "No image mounted at /etc/libvirt." It feels like maybe there's a link that's missing, and my actual libvirt.img is set at /mnt/user/system/libvirt/libvirt.imgLibvirt experimentingroot@unraid:~# /etc/rc.d/rc.libvirt startrc.libvirt: Starting virtlock daemon...2025-08-13 14:59:27.786+0000: 1342789: info : libvirt version: 11.2.02025-08-13 14:59:27.786+0000: 1342789: info : hostname: unraid2025-08-13 14:59:27.786+0000: 1342789: error : main:906 : Can't load config file: Failed to open file '/etc/libvirt/virtlockd.conf': No such file or directory: /etc/libvirt/virtlockd.confrc.libvirt: virtlock daemon... Started.rc.libvirt: Starting virtlog daemon...2025-08-13 14:59:27.802+0000: 1342797: info : libvirt version: 11.2.02025-08-13 14:59:27.802+0000: 1342797: info : hostname: unraid2025-08-13 14:59:27.802+0000: 1342797: error : main:704 : Can't load config file: Failed to open file '/etc/libvirt/virtlogd.conf': No such file or directory: /etc/libvirt/virtlogd.confrc.libvirt: virtlog daemon... Started.rc.libvirt: Starting libvirt daemon...rc.libvirt: libvirt daemon... No image mounted at /etc/libvirt.root@unraid:~# ls /etc/libvirt/root@unraid:~# ls /etc/libvirt-/hooks/ network.conf qemu.conf virtchd.conf virtnetworkd.conf virtqemud.conflibvirt-admin.conf nwfilter/ secrets/ virtinterfaced.conf virtnodedevd.conf virtsecretd.conflibvirt.conf qemu/ storage/ virtlockd.conf virtnwfilterd.conf virtstoraged.conflibvirtd.conf qemu-lockd.conf virt-login-shell.conf virtlogd.conf virtproxyd.confroot@unraid:~# cp -r /etc/libvirt-/ /etc/libvirt/root@unraid:~# ls /etc/libvirt/libvirt-/root@unraid:~# rm -rf /etc/libvirt/libvirt-/root@unraid:~# /etc/rc.d/rc.libvirt startrc.libvirt: Starting virtlock daemon...2025-08-13 15:01:56.900+0000: 1366504: info : libvirt version: 11.2.02025-08-13 15:01:56.900+0000: 1366504: info : hostname: unraid2025-08-13 15:01:56.900+0000: 1366504: error : main:906 : Can't load config file: Failed to open file '/etc/libvirt/virtlockd.conf': No such file or directory: /etc/libvirt/virtlockd.confrc.libvirt: virtlock daemon... Started.rc.libvirt: Starting virtlog daemon...2025-08-13 15:01:56.914+0000: 1366516: info : libvirt version: 11.2.02025-08-13 15:01:56.914+0000: 1366516: info : hostname: unraid2025-08-13 15:01:56.914+0000: 1366516: error : main:704 : Can't load config file: Failed to open file '/etc/libvirt/virtlogd.conf': No such file or directory: /etc/libvirt/virtlogd.confrc.libvirt: virtlog daemon... Started.rc.libvirt: Starting libvirt daemon...rc.libvirt: libvirt daemon... No image mounted at /etc/libvirt.root@unraid:~# ls /etc/libvirt/root@unraid:~# ls /etc/libvirt-/hooks/ network.conf qemu.conf virtchd.conf virtnetworkd.conf virtqemud.conflibvirt-admin.conf nwfilter/ secrets/ virtinterfaced.conf virtnodedevd.conf virtsecretd.conflibvirt.conf qemu/ storage/ virtlockd.conf virtnwfilterd.conf virtstoraged.conflibvirtd.conf qemu-lockd.conf virt-login-shell.conf virtlogd.conf virtproxyd.confroot@unraid:~# cp -r /etc/libvirt-/* /etc/libvirt/root@unraid:~# ls /etc/libvirt/hooks/ network.conf qemu.conf virtchd.conf virtnetworkd.conf virtqemud.conflibvirt-admin.conf nwfilter/ secrets/ virtinterfaced.conf virtnodedevd.conf virtsecretd.conflibvirt.conf qemu/ storage/ virtlockd.conf virtnwfilterd.conf virtstoraged.conflibvirtd.conf qemu-lockd.conf virt-login-shell.conf virtlogd.conf virtproxyd.confroot@unraid:~# /etc/rc.d/rc.libvirt startrc.libvirt: Starting virtlock daemon...libvirt: error : failed to open /var/log/libvirt/virtlockd.log: No such file or directoryrc.libvirt: virtlock daemon... Started.rc.libvirt: Starting virtlog daemon...libvirt: error : failed to open /var/log/libvirt/virtlogd.log: No such file or directoryrc.libvirt: virtlog daemon... Started.rc.libvirt: Starting libvirt daemon...rc.libvirt: libvirt daemon... No image mounted at /etc/libvirt.root@unraid:~# root@unraid:~# cd /etc/libvirt/root@unraid:/etc/libvirt# lshooks/ network.conf qemu.conf virtchd.conf virtnetworkd.conf virtqemud.conflibvirt-admin.conf nwfilter/ secrets/ virtinterfaced.conf virtnodedevd.conf virtsecretd.conflibvirt.conf qemu/ storage/ virtlockd.conf virtnwfilterd.conf virtstoraged.conflibvirtd.conf qemu-lockd.conf virt-login-shell.conf virtlogd.conf virtproxyd.confroot@unraid:/etc/libvirt# rm -rf *root@unraid:/etc/libvirt# lsroot@unraid:/etc/libvirt# /etc/rc.d/rc.libvirt startrc.libvirt: Starting virtlock daemon...2025-08-13 15:04:36.271+0000: 1383507: info : libvirt version: 11.2.02025-08-13 15:04:36.271+0000: 1383507: info : hostname: unraid2025-08-13 15:04:36.271+0000: 1383507: error : main:906 : Can't load config file: Failed to open file '/etc/libvirt/virtlockd.conf': No such file or directory: /etc/libvirt/virtlockd.confrc.libvirt: virtlock daemon... Started.rc.libvirt: Starting virtlog daemon...2025-08-13 15:04:36.288+0000: 1383514: info : libvirt version: 11.2.02025-08-13 15:04:36.288+0000: 1383514: info : hostname: unraid2025-08-13 15:04:36.288+0000: 1383514: error : main:704 : Can't load config file: Failed to open file '/etc/libvirt/virtlogd.conf': No such file or directory: /etc/libvirt/virtlogd.confrc.libvirt: virtlog daemon... Started.rc.libvirt: Starting libvirt daemon...rc.libvirt: libvirt daemon... No image mounted at /etc/libvirt.root@unraid:/etc/libvirt# ls /etc/libvirt/root@unraid:/etc/libvirt# ls /etc/libvirt-/hooks/ network.conf qemu.conf virtchd.conf virtnetworkd.conf virtqemud.conflibvirt-admin.conf nwfilter/ secrets/ virtinterfaced.conf virtnodedevd.conf virtsecretd.conflibvirt.conf qemu/ storage/ virtlockd.conf virtnwfilterd.conf virtstoraged.conflibvirtd.conf qemu-lockd.conf virt-login-shell.conf virtlogd.conf virtproxyd.confroot@unraid:/etc/libvirt# cp -r /etc/libvirt-/* /etc/libvirt/root@unraid:/etc/libvirt# /etc/rc.d/rc.libvirt startrc.libvirt: Starting virtlock daemon...libvirt: error : failed to open /var/log/libvirt/virtlockd.log: No such file or directoryrc.libvirt: virtlock daemon... Started.rc.libvirt: Starting virtlog daemon...libvirt: error : failed to open /var/log/libvirt/virtlogd.log: No such file or directoryrc.libvirt: virtlog daemon... Started.rc.libvirt: Starting libvirt daemon...rc.libvirt: libvirt daemon... No image mounted at /etc/libvirt.root@unraid:/etc/libvirt# touch /var/log/libvirt/virtlockd.logtouch: cannot touch '/var/log/libvirt/virtlockd.log': No such file or directoryroot@unraid:/etc/libvirt# mkdir /var/log/libvirtroot@unraid:/etc/libvirt# touch /var/log/libvirt/virtlockd.logroot@unraid:/etc/libvirt# touch /var/log/libvirt/virtlogd.logroot@unraid:/etc/libvirt# /etc/rc.d/rc.libvirt startrc.libvirt: Starting virtlock daemon...rc.libvirt: virtlock daemon... Started.rc.libvirt: Starting virtlog daemon...rc.libvirt: virtlog daemon... Started.rc.libvirt: Starting libvirt daemon...rc.libvirt: libvirt daemon... No image mounted at /etc/libvirt.root@unraid:/etc/libvirt# ls /etc/libvirt-/libvirtlibvirt-admin.conf libvirt.conf libvirtd.conf root@unraid:/etc/libvirt# unraid-diagnostics-20250813-0950.zip
August 13, 2025Aug 13 Community Expert The syslog already rotated; reboot and post new diags after array start.
August 14, 2025Aug 14 Author Here's another diag, post reboot, just after array starts.unraid-diagnostics-20250814-0746.zip
August 14, 2025Aug 14 Community Expert 1 hour ago, molasses2 said:They are, though. After a full power cycle, in fact.They are not, syslog starts on Aug 9 04:40:16, and it had already rotated.And the last ones are the same, they also start on:Aug 12 04:40:16 unraid sSMTP[2163179]: Creating SSL connection to hostAug 12 04:40:16 unraid sSMTP[2163179]: SSL connection using TLS_AES_256_GCM_SHA384
August 14, 2025Aug 14 Author I don't see what you're seeing. I see Aug 12 in the syslog2.txt file, but syslog.txt shows Aug 14. I opened syslog.txt and attached it separately. I don't know what else to try.syslog2.txtsyslog.txt syslog.txt
August 14, 2025Aug 14 Community Expert Syslog2 and syslog1 should never exist on a fresh boot; those are only created when the syslog rotates. You are either not actually rebooting the server or posting the wrong diags, check the server uptime
August 14, 2025Aug 14 Author Maybe you've found something new that's also broken. As I mentioned, I have both rebooted and power cycled this server. Current uptime is 2h44m.
August 14, 2025Aug 14 Community Expert I don't see how that is possible, but stranger things have happened. Post the output fromls -l /var/log/sys*
August 14, 2025Aug 14 Author root@unraid:/boot/logs# ls -l /var/log/sys* -rw-r----- 1 root root  165482 Aug 14 11:33 /var/log/syslog -rw-r----- 1 root root 7185224 Aug 14 04:40 /var/log/syslog.1 -rw-r----- 1 root root 2568418 Aug 13 04:40 /var/log/syslog.2 root@unraid:/boot/logs#
August 14, 2025Aug 14 Community Expert That doesn't make any sense; those files are in RAM. They must disappear after a reboot, unless you have a script or something copying them back, syslog2 was last modified Aug 13th, that cannot be for a new boot from today.
August 14, 2025Aug 14 Community Expert If you don't find anything else, suggest redoing the flash drive, backup the current one first and then recreate it using the USB tool and just restore the bare minimum, like the key, super.dat and the pools folder for the assignments, also copy the docker user templates folder (\config\plugins\dockerMan\templates-user), if all works you can then reconfigure the server or try restoring a few config files at a time from the backup to see if you can find the culprit.
August 14, 2025Aug 14 Author I don't have any scripts, at least none that I wrote. I don't think any of my plugins are doing anything weird, but I don't really know how to check.Thank you for the suggestion. I've already recreated the flash drive with a full format, so I don't think it's corrupted. Can I just move everything in /boot/config to /boot/config-OLD, then create a new /boot/config and populate it with the bare minimum as you suggested, or is it a requirement that I actually reimage?
August 15, 2025Aug 15 Community Expert 13 hours ago, molasses2 said:Can I just move everything in /boot/config to /boot/config-OLD, then create a new /boot/config and populate it with the bare minimum as you suggested, or is it a requirement that I actually reimage?You can, but you also need the stock config files, not just the files I mentioned, so copy the config from a stock install zip, then restore those files.
August 15, 2025Aug 15 Author Thanks. I tried this yesterday evening with partial success. Libvirt works again, and I've built two VMs!No change with the plugins - they still show until the array begins to start, then disappear. I also checked the syslogs, and I still have 3 files - syslog, syslog1, and syslog2 - and the dates have changed, so they're definitely fresh, not just leftovers.I think I created the new /boot/config correctly, but possible I made a mistake. I will re-attempt tonight/tomorrow with a freshly created flash drive.I appreciate your help thus far!
August 19, 2025Aug 19 Author I'm pressing the pause button for a while. The issue with the plugins page still isn't resolved, but I am still experimenting as I have time.
August 23, 2025Aug 23 Author Solution RESOLVED!!! đAfter a LOT of effort and research, I finally learned that /var/log/pluginshas to exist, andhas to contain symlinks to each plugin located in /boot/config/pluginsI had to create the symlinks manually (see screenshot), but after doing that, I'm good to go.Thank you, JorgeB, for your assistance! Edited August 23, 2025Aug 23 by molasses2
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.