Jebberino84

I apologize for not immediately answering a couple weeks ago when you asked about which ones were persistant syslogs. I had a super busy weekend that week and things have only gotten more hectic here with other non-technical home issues that needed to be taken care of before we hosted for the holidays this year. Again, I do appreciate your help. I believe the logs for the previous session (2025-11-18_0157) might have been the one that should have covered the issue because the new logs that started 2025-11-08_2129 would have been created after the forced reboot, but unfortunately I'm not 100% sure since it's been so long and because as I mentioned in my post on November 7th, I'm pretty sure the syslog stops logging information when the server crashes with the crond error. I planned on waiting until the system crashed again with the same crond error before posting so we had multiple instances of data to help distinguish what was happening, but the next time it crashed was last night November 20th and when I rebooted I was met with an error at the top of my screen that said "your flash drive is corrupted or offline". Because of this Flash Drive Error and because I suspected this possible issue and made sure to purchase a legitimate San Disk USB and not use some I got off Amazon, I decided to replace the USB last night and forgot to look for any diags or logs. I have it back up and running on the new flash drive as of this morning, and if the system crashes again with the crond error, I'll copy any diags and syslogs I can and post again.

Unfortunately, you were right! This time it didn't even last a day and a half before the server started acting up again. Attached are the latest diags, syslogs, and a pic of what I'm seeing when connected directly to the Unraid server. Trying to log in does nothing but cause the login message to repeat and ask for credentials again. The Crond error is the same one I've been seeing since I first started posting here. Any help or guidance would be appreciated. To recap, I've replaced RAM, I've replaced the HBA, I've done smart tests, I've done mem tests, I've taken out the GPU and then put it back in, I've completely restored my NAS back to a fresh install of Unraid. Please let me know if you are able to find anything that might give a clue as to what is causing this. I'm in way over my head here. health-20251107-104301.log health-20251107-104413.log health-20251108-212322.log syslog_2025-11-08-2129 syslog-previous_2025-11-08-0157 jebsnas-diagnostics-20251108-2125.zip

Syslog server is running, but I'm not too confident anything will be shown in it since I included it in my past replies on this post, and it seems to quit logging when the system stops responding. I also included an extra script in my user scripts that will log the status of the networks to see if they are a contributing factor. I'll attach script below. I really am at a loss here since nothing seems to be logged or working when the server does this. I don't know what else to do that I haven't already tried. I've done drive tests, mem tests, changed out the LSI SAS HBA with a brand new one, etc. Nothing seems to log or provide a clue as to what the real underlying issue is. I've also updated BIOS so just about every piece of hardware has also been either tested, updated or replaced. For now, I guess I'll just wait until it crashes again and then I'll post Diags, Syslogs and anything else I can think of here to start over again. #!/bin/bash TS=$(date +"%Y%m%d-%H%M%S") OUT="/boot/logs/health-$TS.log" { echo "=== $TS ===" uptime date df -h df -i free -m ip -br addr # show docker network mode and running containers docker network ls docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Networks}}" # recent kernel and syslog lines dmesg | tail -n 100 tail -n 200 /var/log/syslog } >> "$OUT" 2>&1

Hi all, I waited a while to re-post here because I did not want to bump an old thread without substance. Over the past 4–5 months I moved my data to a temporary NAS, which is now serving as my backup NAS in case I run into future issues, and I completely restored my Unraid server from scratch. I rebuilt the machine about 3–3.5 months ago and have since moved the data back. Since that time I have seen the same issue recur: the Web GUI and the directly connected CLI become unresponsive. This has happened about 3–4 times, so roughly once per month. Each time, just like before, I perform a hard reboot and the server acts like nothing was wrong until about a month later when the same issue returns. This morning it happened again, and I finally turned to AI to comb through my diagnostics and logs to help me find an issue. To my surprise, it found something that makes me hopeful. According to ChatGPT, “Docker is creating a macvlan network on br0. Intermittent GUI lockups and total network loss on Unraid are very commonly caused by macvlan call traces when containers have their own IPs on the same LAN as the host. The system can stay up for days or weeks, then suddenly drop the web GUI and SSH. Reboot fixes it until the next trace.” It believes this because Nginx logged “open socket #... left in connection ... aborting” and, according to AI, “Given your symptoms and the macvlan line, the most likely root cause is macvlan call traces.” Based on that, I changed the Docker setting “Docker custom network type” from “macvlan” to “ipvlan.” So far this has not caused any issues with my Docker containers or other applications running on this Unraid server. Thanks again to JorgeB for your help a few months ago. I was completely lost myself, so even if we did not find the issue at the time, you gave me direction and made me feel like I was doing something to attempt to fix it and you may have kept me from going completely bald pulling my hair out! I see your posts on this forum often and I appreciate your help. Hopefully I will go another few months and recognize that it is not happening anymore, and then remember to post here again to let everyone know whether this fix did or did not work. If you do not see another post on this thread, it is likely because this VLAN change for Docker services did finally fix the issue and I just forgot to repost. If anyone else has a similar issue, feel free to try this out.

Hi, JorgeB! I did have the crond error happen again today. Attached below should be my Syslog, my diags, and a pic of the login screen showing that I am again unable to log into the server, even when physically attached via KVM. Looks like the logs stopped last night around 5:09 PM after some sort of nginx alert that caused an "abort". This seems to point as to why I am unable to reach the web GUI, but it doesn't tell me why I am then unable to log into the server, even via a direct KVM connection. The nginx error wouldn't bother me so much if it didn't cause my whole server to go down. Yesterday, the server had been up for a few days, so I felt confident starting to build the parity for a second parity disk. (disk WP026KC5) I figured that since the server was seemingly running fine again, and was able to complete it's own parity check, that this would be possible. I noticed that I was having the original "crond" error again this morning, and although I was unable to access the server to see shares or to login via Putty or KVM, I could see that the physical lights I have on my server that blink to show when disks were being accessed were blinking. I also noticed that I was able to access media via my plex server which runs on a docker container. I figured I'd let the computer keep running until I didn't see the lights blinking anymore, which would hopefully signify that the parity for the new drive was finally complete. The lights stopped blinking today a little after noon. I waited even longer and rebooted the server around 1:15 PM but unfortunately, the parity didn't take so I put the new disk back into unassigned devices. Please let me know if you guys can find what exactly is causing the issues with my server, and why it doesn't allow me to log in. Thanks once again for your time! jebsnas-diagnostics-20250625-1342.zip syslog-192.168.0.100.log

It figures that now I have someone else's attention looking at this, nothing is happening. I'm now on day 2 of not having to reboot the server when I wake up in the morning. It had been happening every night and even sometimes during the day too for the past ~2 weeks, and now all the sudden it's working fine. Sheesh! Just for the record, attached below is a pic of the parity check which finally completed. 0 errors and it took more than 22 hours! I'll make sure to update here the next time the server goes offline as long as it happens within the next 1-2 weeks. As a separate question, I'm not sure about the etiquette of this, but if it takes longer than 1-2 weeks for the server to drop offline by itself again, should I start a new thread or just continue commenting on this thread again?

Got it. The Syslog should be attached below. I should first mention that this is the first morning in a long time where I woke up and didn't have to reboot the server because it stopped working. I will continue to monitor and troubleshoot as I don't believe the problem just fixed itself, but this is a refreshing change of pace. Regarding the Syslog, Something I did see on the Syslog server that seemed interesting, is that there are a lot of logs for: JebsNAS kernel: br-336f8d0e3be0: port 1(veth6d2d15e) entered blocking state JebsNAS kernel: br-336f8d0e3be0: port 1(veth6d2d15e) entered disabled state This doesn't entirely make sense to me, and it doesn't really explain the issues I've been having. I believe this to be a completely unrelated issue to the original issue I've been having. I'll explain my logic below. Please let me know if (or more likely where) my logic is flawed. If there is a fix for this issue which I'm not seeing, please let me know as I'd still like to stop those errors. However, I'm not entirely convinced fixing these blocking/disabled networking errors will fix my original issue. I do have a PCIe network card installed on my machine, and I have 2 ethernet cords attached to my server from my L3 switch/router. I have one cable attached to the motherboard and another attached to the network card. I see how these could originally cause STP errors, however, I set these up so the motherboard (and thus my Unraid server) is on my 192.168.0.0/24 network, and the PCIe NIC is attached to a 192.168.5.0/24 network. The network card connection I use to passthrough traffic to my Windows VM running Blue Iris. I see how these could have had some issues with Spanning Tree Protocol when I first connected them, but now that they are on different networks, I don't believe this would be what's causing my issue. Furthermore when I look at the Network Settings, I can see that the (blocking/disabled state) issue doesn't seem to be about the two cords, it seems to be with the internal loopback address. The logs mention br-336f8d0e3be0, which network settings says is related to the 172.18.0.0/16 route. Knowing the loopback address was a possible issue, this made me think about my docker containers since they are all on a network I created for them and would constantly speak to each other. Most of the logs here seem to be related to the docker containers anyway. I've also posted a picture of my docker tab after expanding all of the container and IP port columns. In it you can see that the dockers all have separate port numbers and are on their own "dockermedianetwork" network so they can speak to each other. The only one that wasn't setup this way was the Plex docker. I left Plex with this original configuration when I added all the other docker apps a while back, but just in case, after posting this I will be changing Plex to also use the dockermedianetwork network too. I'll see if this causes any other issues with Plex access and/or with fixing the blocking/disabled state errors on my next reboot. As mentioned above, I'm not convinced my original issue would be fixed by stopping whatever is causing the blocking/disabled state issue. This is because the error I was getting in my original post for this thread did not only affect the network, but the error also was affecting the server itself. When I plug in via KVM, I am not able to log in and also unable to perform shutdown or powerdown or any other commands since I can't log in. I don't see how a network issue could do this if I was bypassing the network with KVM? Thanks again for looking. If you see anything else that might require my attention, please let me know! Also, I will post another Syslog and Diags when the server goes down again. syslog-192.168.0.100.log

Sorry it's taken me so long to respond here. My server is now doing something somewhat new. It stopped allowing me to connect via the web GUI early this morning, probably 4-5 hours ago. But at that time I was able to login via a KVM connected directly to the server. I was even able to display results for ifconfig. However, I let the server continue to run in this state, and my wife even watched a Plex movie which I run in Docker from this server. I also have a Blue Iris program running from a Windows 10 VM I have in this server, but I am unable to connect to the VM. I logged out of the server a few hours ago when the GUI went down, and I just tried logging in again but now that didn't work either. Since I am unable to log in, it also won't let me use powerdown, or shutdown commands so it seems I had to reboot the server manually again with a hard press of the power button. This is a similar issue to the one I originally reported because I am unable to connect via web GUI interface, I am able to ping the machine, and I am also unable to login or powerdown the machine, but it is different because I am not getting the repeating crond error I originally described, and I am also still able to use at least one of the docker containers. I figured I'd mention this in case the diags do show what's going on. Attached should be the diags after a reboot after losing GUI as requested above. Please let me know if there is anything else I can give or do to help track down this issue. Thanks again for your time and help! And finally as kind of a side note, I thought I already had the Syslog server settings enabled and setup correctly in my previous diags. I attached a photo here of my current settings, if this is incorrect, please let me know. jebsnas-diagnostics-20250621-1254.zip

Hello again! Attached are my diags after clicking "reset" button in the stats section and then rebooting after that finished. Thanks again for your time! Please let me know if there is anything else I can do to help track down this issue. jebsnas-diagnostics-20250620-1511.zip

Hi, JorgeB! Ahhh. This is where I misunderstood. Totally my fault. I thought by the term "pool" we were talking about the group of array devices since that is usually what I think of when I think of Unraid. I should have known we were talking about the "cache" pool since that's the only thing in Unraid referred to as a pool. Thank you for clearing this up for me. I really hate the way my brain works sometimes. Below is my result from scrubbing the cache pool. Thanks again for your help!

Hi, JorgeB! I apologize for my misunderstanding. Please help me clear this up and hopefully we can find where my logic is flawed. I respect your opinion and efforts and I see you post on here all the time helping people so I know you know what you're talking about. I believe this to be a flaw in my own understanding that I'm trying to get help with. I want to do what you're asking and I promise I am trying to do it. I believe by "scrub the pool and post the results from GUI" you are asking me to perform a parity check, and to then grab results of this parity check from the GUI and post them here. I believe that when Unraid performs and completes a parity check, it will say something like "completed with no errors" or something similar at the bottom of the "Main" tab where it gives information on the parity check, and this is what you want me to post? Please let me know if this is correct and I will attempt to post a picture or screen capture of this if I can get a parity check to complete. If this is not what you are looking for, please let me know exactly where in the GUI I should be able to find the information you are looking for. I promise, I am not just lazy and trying to not perform a parity check. I am just trying to explain why it hasn't been possible or easy for me to do so with the server going down all the time. The system started another parity check this morning when I had to restart the server again. In fact, I believe the parity check has been started on it's own almost every time my server has rebooted in the past couple of weeks because it's not being shut down gracefully. I will admit that many of the times the server was rebooted, I did stop the parity check because I didn't feel like it would do anything different than all of the other times it ran, but at least a few of the times it has rebooted (likely much more than a few) I let it continue to run the parity check until the server went down again. Because I have all 10TB drives in my server, it takes a long time to perform the parity check. I had to reboot my server again this morning because of the error described above caused it to stop working again last night before the parity check finished. The parity check has currently started again, and I will let all future parity checks continue and attempt to gather the result if it finishes before the server goes down again. In order to help in the meantime, I've also included a current screen capture of bottom of the main tab. As you can see from the picture attached, it currently says it will take ~11 hours to complete the parity check but my past experience tells me that it will likely take a bit longer. I believe the last times I saw it complete, it took closer to 13-16 hours. The issue I'm having with the parity checks in the past couple weeks, is they take so long to complete that many of the times it runs, my server stops working before the check completes. For instance, In the diagnostics I included below, you can see in the syslog-previous.txt log file that my server was rebooted yesterday at ~2:20 PM, and seemingly around 10:30 PM it went down again because that's where the log stops. I didn't reboot it again until this morning. I also want to note, that the parity-checks.log file does not include any line items from this instance (2:20PM - 10:30PM yesterday) when I rebooted the machine and let the check run. If I stop the parity check, it logs what it was able to do, but if it runs until the server goes down, nothing is logged about the parity check in the parity-checks.log file. This has been my experience so far, that the server can't complete the parity checks because they take longer to complete than the server stays online. The server also doesn't log when I do let the check run because it goes offline before the check finishes. I have also attached a copy of my parity-checks.log file found in /boot/config. In this file I believe there are a couple instances the parity check was able to complete in the past week before the server stopped working or I stopped the check. I believe the log for June 16 at 09:50:40 and the log for June 13 at 00:10:12 are times when the check was able to complete. I believe the "check P" log indicates they passed? If this is true, I'm hoping that this can serve as an indication that the parity check did complete after I began seeing this issue, and we can continue diagnosis and troubleshooting to figure out what is causing my server to stop working and send the crond errors described above. I want to re-iterate, please understand that I am not saying that I will not complete the parity check, I will still attempt to complete the parity check that is currently running after my reboot this morning, and I will try to capture the result before the server goes down again. I will also continue to let the parity checks run after all reboots from now on and see if I can capture one for you that completes. My last post was not an attempt to tell anyone that I refuse to do as you ask, it was only an attempt to explain the troubles I've been having and why I wasn't confident that it would be possible to complete the check and for me to capture the results in the GUI with the server going down all the time and the GUI being unresponsive. I also wanted to give as much other information that I believe might help in the meantime so that we cold continue troubleshooting while we waited for a successful parity check to complete, especially since I believe it may not be possible to get a successful parity check with the server going down all the time. Thank you again for your time! Please let me know if I am correct about the GUI results you're looking for, and if there is anything else we can do in the meantime while we wait for a parity check to complete that might rule out other possible issues. JorgeB, I really do appreciate everything you do for the community in these forums. Every time I have an Unraid issue and look things up myself I see threads littered with you're responses. Thank you for all that you do for this community. parity-checks.log jebsnas-diagnostics-20250620-0955.zip

Hi, JorgeB! None of the drives technically dropped offline. I was going to add a second parity drive which I decided would be a good idea to order when this issue became frequent. I installed the drive in the computer a few days ago when I believed I solved the issue, but every time I've tried to build the 2nd parity in the past week, the server stops working before the rebuild can complete. I did get a parity check to complete on the original drives once last week before I added the new drive, so I don't believe this is what's causing the issue. I believe what you might be seeing is the new drive, not a bad one which it might look like from the diags. I forgot to mention in my original post, but I did do a smart test on all the drives last week when this issue first started becoming a common occurrence, and also a memtest. Everything passed. This, along with the fact that I was able to complete a full parity check last week after the issue began, makes me think it's not a bad drive issue. If you see anything else that makes you believe a bad drive is the culprit, please feel free to let me know and I will then try to identify the bad drive and replace it since I already have a new one. I'd need another way to diagnose/identify this though since the smart tests seem to have passed. I have 2 drives currently connected which are "unassigned". One of them is the new parity drive, and the other is a purple HDD that I use for a Windows VM running Blue Iris from this server. I do have a picture of my monitor showing the memtest. I let it run through 3 times. The second 2 times took a very long time to complete. I also have pictures of the unraid terminal after attaching my monitor directly to the server. In the first you can see the "crond" error that keeps repeating. In the second you can see what happens when I try to login to the server when kvm is attached directly. The final pictures I have attached here show all the drives currently part of the array and their smart passing, and also the 2 unassigned drives which have passed smart tests.

Periodically (currently every day for the past week) my server GUI will stop working and I get a refused to connect error when I try to log in via web GUI. When this happens, I am able to ping the server through cmd from my windows desktop. I also tried "telnet MyServerIP 445" but this resulted in an error. I don't remember the exact wording of the error. I believed this might be a networking issue but I don't really believe this is the issue because 1. the GUI and Unraid server work fine for about a day before it stops working, and 2. When I viewed the server terminal with a KVM, I get a repeated error: "crond[2096]: exit status 135 from user root /usr/local/emhttp/plugins/dynamix/scritps/monitor &> /dev/null" that is repeated about every 30seconds-1minute Attempts to type in my username and password on terminal to sign into the machine without the network and GUI would just display: "-bash: /usr/share/bash-completion/bash_completion: Input/output error Unraid server OS version 7.1.3 IPv4 address: MyServerIP IPv6 address: not set" After displaying this message it would bring me back to the login prompt. I eventually decided I had to reboot the server when this happens. powerdown and shutdown commands do nothing. I've also attempted to do a short press of the power button for a slightly more graceful shutdown and then gave the server ~15 minutes to try to shut itself down, but the server doesn't seem to respond to this either. I eventually have to long press power button when this happens. I'm currently running Unraid 7.1.3 but this has been happening periodically since I was on unraid 6.x.x. I have an asus prim z370-a motherboard. I mention this because I've replaced a few usb sticks by now, and I'm not sure if it's a flash drive problem, or possibly something with the motherboard/BIOS. This same error has happened about 10 times in the past couple weeks and is now happening at least once a day. I found this line in the diagnostics -> logs -> syslog-previous.txt file after one of my reboots and the error seems to have happened right before it wen't offline... "Jun 17 07:34:00 JebsNAS kernel: BTRFS info (device sdf1): bdev /dev/sdf1 errs: wr 1792460, rd 783, flush 17277, corrupt 20433, gen 0" This makes me think it was a problem with the flash drive? Is "/dev/sdf1" the flash drive? In some of my old diagnostics I also had lots of errors from the NUT Community Applications program and my Cyberpower UPS is the only other thing connected to the server via USB (besides keyboard and mouse) so I eventually uninstalled that program to see if this changed anything, but it still did this error last night. As far as NUT program diagnosing went, I was seeing a repeated error in my logs about a pollonly flag. I remembered I got this once before a long time ago, and I added the pollonly flag to the .conf file, but it disappeared at some point. After adding this flag again, it disappeared by the next time the server was turned on so I got rid of NUT altogether to see if that would help. I should also mention here if it is the usb drive that's failing, it's a brand new flash drive which replaced my old one less than a week ago when I was trying to diagnose this and believed the old one was having issues. I've tried switching the OS flash drive to a different usb port when I still had the old drive, and I've switched with the new one too. I've been using Unraid now for about 3 years, and although I love what I'm able to do with it as far as a home NAS goes, and the unraid forum community has been a really great and helpful place to get information since most of my issues have been asked and answered before, but I also feel like I've had nothing but bad usb drive problems since I've set this machine up. This must be my 4th or 5th time replacing the flash drive in only 3 years! Attached should be my last few diagnostics files which I've been keeping every time I have to restart the server for about the past week. If anyone has any information that may help me keep this server running, please let me know. Thank you all for your time and effort throughout the years. jebsnas-diagnostics-20250608-1829.zip jebsnas-diagnostics-20250609-1917.zip jebsnas-diagnostics-20250614-1623.zip jebsnas-diagnostics-20250617-1149.zip jebsnas-diagnostics-20250617-1951.zip jebsnas-diagnostics-20250617-1954.zip jebsnas-diagnostics-20250618-0907.zip jebsnas-diagnostics-20250618-1828.zip jebsnas-diagnostics-20250619-0833.zip

Thank you for this! It led me to my issue! I do remember there being a flag at the top of the screen before I restarted, saying the server needed a "restart for VM updates to take effect" or something along those lines. It looks like what happened was the "PCIe ACS override" option in Settings -> VM Manager somehow changed from whatever setting I had it at before to "Disabled". This meant that the GPU I was using to passthrough to one of my VMs shared an IOMMU group with the LSI 9300-8i HBA. I know I had this setup properly in the past, and I don't believe I changed this setting on my own, so I'm not sure why/how it happened, but regardless, it seems to be fixed now. I appreciate your help, JorgeB!!

What happened: After a restart, my LSI 9300-8i HBA stopped showing disks. I've had the card for almost 3 years, and it seems to have just stopped working. After doing some diagnosing myself, I believe I've ruled out the hardware as being the issue, but I'm open to suggestions on further diagnosis if someone still believes it's a hardware issue. What I've tried so far to fix/diagnose: - I tried reseating the HBA in the PXIE slot and reseating the mini-SAS HD connectors and the SATA connectors. - I moved some disks from the HBA to directly connected to Motherboard SATA ports. The hard drives I connected to the motherboard showed up. This ruled out the drives/disks as the issue. - I purchased a brand new LSI 9300-8i and hooked it up along with switching out for the new cables but the drives still aren't showing up. - I can see the "Avago Technologies MPT SAS3 BIOS" HBA screen when booting the computer, and the green light (heartbeat LED) flashes on the HBA card which seems to show that it is capable of general activity so to me, the issue is likely not the HBA card(s). - I've read many posts here so far, but most seem to either be about the HBA never working in their initial setup, or the issues are fixed by purchasing a new card. Since I've done both of these things, I'm unfortunately unable to diagnose this issue without help. Fun request: If someone does find some helpful information in my diagnostics files or elsewhere, could you please include in your response where you found the information? This request is to hopefully help spread knowledge or "teach a man to fish". I attempted looking into the diagnostics files myself, but I have no idea where to start or what to look for. Some of the places I thought I might find information but haven't are: - tried looking in config\disk.cfg, logs\syslog.txt, logs\vfio-pci.txt, and others but didn't see anything related to the HBA. Thank you, everyone for your help here and with past issues so far. I've had few issues with UNRAID, but when I have had issues this community has been excellent in helping me find what I'm looking for. jebsnas-diagnostics-20250604-1422.zip

I logged onto other vms with the same user, but I guess it could have been something with this user on that VM? It's already deleted now, so no way to test it out further than logging onto other VMs.

I apologize. I guess I'm just really frustrated. I've tried everything I can think of and I don't understand what's happening. I'm nuking the disk and rebuilding the VM from scratch. Thanks for reading and posting and trying, JohnathanM!

Thank you for your comment on this thread, JonathanM. I'm not trying to be rude, (you have 16.1k comments so I believe you offer a lot of help on this forum and for that, I appreciate you) but I did mention in my first post that I already tried safe mode and unfortunately it made no difference. Something I did remembered now which I did not post about earlier is that I have other Windows 10 VMs on this device that started up just fine. This is the only VM that wouldn't restart after something went wrong with the whole server a few days ago. My last attempt was to do a windows 10 restore, (of which I only had 1 restore point available from 2 days ago which I don't remember making, so maybe this has something to do with why the whole server had issues but ???) and that didn't work either. So now, I am just going to delete the VM, reformat the hard drive, and start over. I give up. I will 100% be automating some sort of scheduled events to create more windows restore points, as well as snapshots of all my VMs, and Unraid flash drive backups, along with anything else I can think of to make sure this doesn't happen again. I probably should have done this earlier. Thanks again for anyone taking a look at this post. If anyone has any good theories on why or what happened, please feel free to comment as I may put off fixing this VM for a little bit.

Somewhat of an update, When I setup this VM, I passed through an entire hard drive which I dedicated to this VM. I was able to go into the BIOS of the Unraid server (ASUS Prime Z390-a Motherboard) and I changed the boot order so it would boot directly into windows. After doing this I was having the exact same error with the loading "welcome" screen after I enter in my credentials. To me, this seems to point towards an issue with windows and not Unraid or the VM setup. Knowing this, and after not getting any help here, I have decided to attempt finding other forums dedicated to windows 10 to see if anyone can help. I'm hoping there is a way to fix this without completely reformatting the drive and replacing the VM. I'll try to remember to post back here if I find a solution.

I'm still not entirely sure what happened, but I had an issue a few days ago and I lost some settings for my server. I was able to get the network back up and running, and then re-install Community Applications and all my dockers (Plex/Jellyfin/etc.) but now I've noticed some weird issues with one of my VMs. I run a Windows 10 VM on this server with Blue Iris on the Windows 10 VM. This entire server and this VM have been running fine for over a year, but it just started having issues a few days ago. When I try to run this VM it boots up, and I get past entering credentials, but while windows actually loads there is a "welcome" loading screen with the spinning dots. I've let the VM sit for over 6 hours and it never gets past this screen. I've attempted everything I can think of. Most forums here seem to indicate issues loading windows all-together and speak of the screen before you enter credentials. I am able to enter in my credentials, but after that it hangs up. At one point I did restart in safe mode (with internet), but that didn't do anything. Still got held up on the same screen. I've also tried not passing through the GPU, but that didn't help either. I've also tried updating the virtio drivers on unraid and on the VM settings. Nothing gets me past the windows "welcome" screen. I'm at a point now where if I don't figure this out, I believe I will have to scrap this VM and redo it from the beginning. Unfortunately, I did not create any snapshots so if I do this, it will have to be a completely new and fresh install. I'll also be looking into how to automate snapshots if I have to redo the whole VM. I'd like to not go that route though and I'm hoping someone will have some direction for me. This is my last resort to save the VM. I will include some logs to hopefully help. Unraid Diagnostics attached below. Logs from VM while it's stuck on loading screen: 2024-04-18 19:13:54.304+0000: starting up libvirt version: 8.7.0, qemu version: 7.2.0, kernel: 6.1.79-Unraid, hostname: JebsNAS LC_ALL=C \ PATH=/bin:/sbin:/usr/bin:/usr/sbin \ HOME=/var/lib/libvirt/qemu/domain-1-Win10BlueIris \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-Win10BlueIris/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-Win10BlueIris/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-Win10BlueIris/.config \ /usr/local/sbin/qemu \ -name guest=Win10BlueIris,debug-threads=on \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-Win10BlueIris/master-key.aes"}' \ -blockdev '{"driver":"file","filename":"/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ -blockdev '{"driver":"file","filename":"/etc/libvirt/qemu/nvram/018291b0-e041-9899-5b33-6243ef3ec348_VARS-pure-efi.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ -machine pc-i440fx-7.1,usb=off,dump-guest-core=off,mem-merge=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \ -accel kvm \ -cpu host,migratable=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=none,host-cache-info=on,l3-cache=off \ -m 8192 \ -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":8589934592}' \ -overcommit mem-lock=off \ -smp 10,sockets=1,dies=1,cores=5,threads=2 \ -uuid 018291b0-e041-9899-5b33-6243ef3ec348 \ -display none \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=36,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime \ -no-hpet \ -no-shutdown \ -boot strict=on \ -device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pci.0","addr":"0x7.0x7"}' \ -device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pci.0","multifunction":true,"addr":"0x7"}' \ -device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pci.0","addr":"0x7.0x1"}' \ -device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pci.0","addr":"0x7.0x2"}' \ -device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.0","addr":"0x3"}' \ -blockdev '{"driver":"host_device","filename":"/dev/disk/by-id/ata-WDC_WD84PURZ-85B2YY0_WD-CA37ANJK","node-name":"libvirt-3-storage","cache":{"direct":false,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-3-format","read-only":false,"cache":{"direct":false,"no-flush":false},"driver":"raw","file":"libvirt-3-storage"}' \ -device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x4","drive":"libvirt-3-format","id":"virtio-disk2","bootindex":1,"write-cache":"on","serial":"vdisk1"}' \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/Windows/Windows.iso","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"raw","file":"libvirt-2-storage"}' \ -device '{"driver":"ide-cd","bus":"ide.0","unit":0,"drive":"libvirt-2-format","id":"ide0-0-0","bootindex":2}' \ -blockdev '{"driver":"file","filename":"/mnt/user/isos/virtio-win-0.1.248-1.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"}' \ -device '{"driver":"ide-cd","bus":"ide.0","unit":1,"drive":"libvirt-1-format","id":"ide0-0-1"}' \ -netdev tap,fd=37,id=hostnet0 \ -device '{"driver":"virtio-net","netdev":"hostnet0","id":"net0","mac":"52:54:00:0a:95:60","bus":"pci.0","addr":"0x2"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -chardev socket,id=charchannel0,fd=35,server=on,wait=off \ -device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -device '{"driver":"vfio-pci","host":"0000:01:00.0","id":"hostdev0","bus":"pci.0","addr":"0x5"}' \ -device '{"driver":"vfio-pci","host":"0000:01:00.1","id":"hostdev1","bus":"pci.0","addr":"0x6"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on char device redirected to /dev/pts/0 (label charserial0) If any additional information is needed to help diagnose the issue, just let me know and I'll get on it ASAP. Thanks for taking a look! Unconfirmed 448532.crdownload

Thank you, JorgeB! I really don't think I ever would have guessed that was causing all the issues! I'm not 100% sure yet, but I believe the second NIC being configured was exactly what the issue was! I'm still downloading/installing community applications, and afterword I'll be re-installing my dockers, but right now it seems that I have a connection again and things are finally moving forward! I've marked your first comment as the solution. I'll also likely donate after I finish re-downloading and setting up the rest of the server. Thanks again!

I also just noticed that I spoke of Plex in this sentence, but I meant Unraid. When I speak of this server to my wife, I call it the Plex server because that is what she uses it for the most and what she recognizes it as. I doubt this did, but I apologize if this created any confusion.

My apologies. After posting all the other error messages and such, I should have in my last post. Here are the new Diagnostics. Unconfirmed 561677.crdownload

Thank you for suggesting this. I believed STP would just take over if the second NIC was still associated with the server and not a VM (which is what I planned on using it for) before I was able to setup and configure the VMs again. I did move the NIC to a different network and got rid of the second gateway, but this unfortunately did not fix my issue. Below is the error I get when I try to install the Community Applications plugin. I've let the server sit for more than 30 minutes after it stops on the last line but it never does anything else after executing the hook script. Not sure if this error message will help, but it does show that there is still an issue. plugin: installing: community.applications.plg Executing hook script: pre_plugin_checks plugin: downloading: community.applications.plg ... plugin: community.applications.plg download failure: Network failure Executing hook script: post_plugin_checks I am still able to ping my gateway, and other devices on my network, but I am unable to ping outside of my network as seen below. This is what doesn't make sense to me. I've obviously never set anything up like a firewall for this device, and I don't believe there is anything on my router that is administratively preventing the device from accessing the internet. If I am able to ping the router, it proves that there is a working connection to the router, and I am also able to access the outside internet from all other devices. The above solution I posted spoke of "wireguard" as being able to creep into the initial setup of the server's boot instructions and could somehow prevent the server from accessing the outside internet. Is there anything else that would have the ability to do this on Plex? root@JebsNAS:~# ping 192.168.0.1 PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data. 64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.165 ms 64 bytes from 192.168.0.1: icmp_seq=2 ttl=64 time=0.173 ms 64 bytes from 192.168.0.1: icmp_seq=3 ttl=64 time=0.351 ms 64 bytes from 192.168.0.1: icmp_seq=4 ttl=64 time=0.299 ms 64 bytes from 192.168.0.1: icmp_seq=5 ttl=64 time=0.351 ms 64 bytes from 192.168.0.1: icmp_seq=6 ttl=64 time=0.348 ms ^C --- 192.168.0.1 ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5123ms rtt min/avg/max/mdev = 0.165/0.281/0.351/0.081 ms root@JebsNAS:~# ping 192.168.0.100 PING 192.168.0.100 (192.168.0.100) 56(84) bytes of data. 64 bytes from 192.168.0.100: icmp_seq=1 ttl=64 time=0.016 ms 64 bytes from 192.168.0.100: icmp_seq=2 ttl=64 time=0.011 ms 64 bytes from 192.168.0.100: icmp_seq=3 ttl=64 time=0.017 ms 64 bytes from 192.168.0.100: icmp_seq=4 ttl=64 time=0.016 ms 64 bytes from 192.168.0.100: icmp_seq=5 ttl=64 time=0.014 ms 64 bytes from 192.168.0.100: icmp_seq=6 ttl=64 time=0.012 ms 64 bytes from 192.168.0.100: icmp_seq=7 ttl=64 time=0.025 ms 64 bytes from 192.168.0.100: icmp_seq=8 ttl=64 time=0.012 ms ^C --- 192.168.0.100 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 7148ms rtt min/avg/max/mdev = 0.011/0.015/0.025/0.004 ms root@JebsNAS:~# ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. From 192.168.5.100 icmp_seq=1 Destination Host Unreachable From 192.168.5.100 icmp_seq=2 Destination Host Unreachable From 192.168.5.100 icmp_seq=3 Destination Host Unreachable From 192.168.5.100 icmp_seq=4 Destination Host Unreachable From 192.168.5.100 icmp_seq=5 Destination Host Unreachable From 192.168.5.100 icmp_seq=6 Destination Host Unreachable ^C --- 8.8.8.8 ping statistics --- 8 packets transmitted, 0 received, +6 errors, 100% packet loss, time 7181ms pipe 4 root@JebsNAS:~# Thanks again for helping me with this. If you or anyone else has any recommendations, please let me know.

So, unfortunately, deleting and re-creating the docker.img did not work. I did find this thread which seemed to help other users, but I had absolutely nothing in my "wireguard" folder on my flash drive: I tried the above, after I noticed that my server is accessible from within my own LAN, and I can ping it, but I am unable to reach the outside internet from my server. I can ping my gateway/router and other devices on my LAN from the server, but nothing outside the LAN. I can also access the web GUI for my server from 2 different computers from my LAN which is why this took me so long to realize was an issue. This is really confusing me. I'm still at a loss and need to move on to other chores/errands for the day. If anyone has any advice on what could possibly be causing me all these issues, please post ideas/possible solutions. As of right now, I'm completely at a loss and frustrated. Thank you!