jmztaylor Posted May 22, 2022 Share Posted May 22, 2022 Upgraded to 6.10 yesterday and woke up this morning to docker and VMs offline. Trying to reboot the server but stuck at Retry unmounting disk share. tower-diagnostics-20220522-0934.zip Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 After awhile the server finally rebooted and everything seems to be up and running now without issue. Would like to find out the issue if possible. Quote Link to comment
itimpi Posted May 22, 2022 Share Posted May 22, 2022 You should enable the syslog server to get some log information that survives a reboot in case it happens again. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 2 minutes ago, itimpi said: You should enable the syslog server to get some log information that survives a reboot in case it happens again. diagnostics was ran before reboot. Quote Link to comment
itimpi Posted May 22, 2022 Share Posted May 22, 2022 Your syslog is full of crashes on macvlan. This can normally be fixed by upgrading to 6.10.1 and using iplan instead under docker. later you start getting btrfs errors on the loop2 device which is the one holding the docker image file so this is corrupt and will need recreating. Not sure if there are also problems at the btrfs level on the cache pool. Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 You should disable Intel VT-d ASAP, looks like the same issue that affects the HP Microserver Gen8, likely because they use the same NIC driver, more info below: https://forums.unraid.net/topic/123620-unraid-os-version-6100-available/page/8/#comment-1129501 Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 (edited) 44 minutes ago, itimpi said: Your syslog is full of crashes on macvlan. This can normally be fixed by upgrading to 6.10.1 and using iplan instead under docker. later you start getting btrfs errors on the loop2 device which is the one holding the docker image file so this is corrupt and will need recreating. Not sure if there are also problems at the btrfs level on the cache pool. Where is the setting for iplan? Also nuked the docker.img file and rebuilt from previous apps. 38 minutes ago, JorgeB said: You should disable Intel VT-d ASAP, looks like the same issue that affects the HP Microserver Gen8, likely because they use the same NIC driver, more info below: https://forums.unraid.net/topic/123620-unraid-os-version-6100-available/page/8/#comment-1129501 I don't specifically have VT-d in my BIOS but I am assuming its just the virtualization technology setting. Is there a bug tracker to follow this? I am also assuming VMs will not work till this is corrected? Edit: Also parity check is showing 2900 errors so far 3% into it. Is there a way to find out where they are to nuke those files. Array data does not concern me too much with loss. Edited May 22, 2022 by jmztaylor Quote Link to comment
itimpi Posted May 22, 2022 Share Posted May 22, 2022 55 minutes ago, jmztaylor said: Where is the setting for iplan? Under Settings->Docker with advanced view enabled (Docker Custom Network Type). I think the array needs to be stopped to change it. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 (edited) 1 minute ago, itimpi said: Under Settings->Focker with advanced view enabled (Docker Custom Network Type). I think the array needs to be stopped to change it. Ok missed the advanced view switch. Here is latest diags with 6.10.1 installed and iplan enabled. tower-diagnostics-20220522-1149.zip Edited May 22, 2022 by jmztaylor Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 1 hour ago, jmztaylor said: I don't specifically have VT-d in my BIOS It might be called a different thing, like Intel Virtualization technology or similar, alternatively add intel_iommu=off to syslinux.cfg append line, in either case check after booting that IOMMU is really disable, click on system info, top right of the GUI, you should not use the server until that's done. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 Just now, JorgeB said: It might be called a different thing, like Intel Virtualization technology or similar, alternatively add intel_iommu=off to syslinux.cfg append line, in either case check after booting that IOMMU is really disable, click on system info, top right of the GUI, you should not use the server until that's done. Yeah I disabled Intel Virtualization Technology. Assumed it was the same thing Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 It was still enable in the last diags posted. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 1 minute ago, JorgeB said: It was still enable in the last diags posted. It for sure is disabled and VMs are failing to start due to virtualization turned off. I have added that line to syslinux to be sure and rebooted. tower-diagnostics-20220522-1220.zip Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 Now it's disable, likely there are two different settings in the BIOS, the one you disabled is for VT-x, you can re-enable that and still run VMs as long an no hardware if being passed through (for that you need VT-d). 1 Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 1 minute ago, JorgeB said: Now it's disable, likely there are two different settings in the BIOS, the one you disabled is for VT-x, you can re-enable that and still run VMs as long an no hardware if being passed through (for that you need VT-d). What is the solution for the parity sync errors. If this issue causes corruption, what should be trusted? Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 You now should also run a correcting scrub on the pool. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 1 minute ago, JorgeB said: You now should also run a correcting scrub on the pool. Is there documentation for that? Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 Just now, jmztaylor said: If this issue causes corruption, what should be trusted? Without pre-existing checksums for the files not much you can do other that correct parity, any files that are corrupt in the pool will be listed in the syslog after the scrub. Quote Link to comment
JorgeB Posted May 22, 2022 Share Posted May 22, 2022 1 minute ago, jmztaylor said: Is there documentation for that? Click on cache on the main page then scroll down until the scrub section. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 1 minute ago, JorgeB said: Click on cache on the main page then scroll down until the scrub section. Didn't think my cache pool had an issue. Parity was showing sync errors on the array. Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 (edited) FWIW, I ran the scrub and this is what shows Edit: Ran repair right after and all corrupted files are plex phototranscoder files. So no big deal with those. tower-diagnostics-20220522-1332.zip Edited May 22, 2022 by jmztaylor Quote Link to comment
jmztaylor Posted May 22, 2022 Author Share Posted May 22, 2022 So the cache pool is returning no more errors but parity check on array is returning errors. Last run there was zero. So between the upgrade and this issue some corruption was introduced. What should be believed? That parity is still good and needs to do corrections or restart with blank parity and trust the data is good? Quote Link to comment
JorgeB Posted May 23, 2022 Share Posted May 23, 2022 You should run a correcting check now, you can then run a non correcting one after a couple of days to confirm no more issues. Quote Link to comment
jmztaylor Posted May 23, 2022 Author Share Posted May 23, 2022 7 hours ago, JorgeB said: You should run a correcting check now, you can then run a non correcting one after a couple of days to confirm no more issues. Is there a bug tracker for the iommu issue? Quote Link to comment
JorgeB Posted May 23, 2022 Share Posted May 23, 2022 Not for now, but I'll post an update in the announcements thread when there's new info, while this isn't an Unraid issue I know that LT is looking at it. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.