September 4, 20232 yr I have a pretty simple system. Upgrade went OK but when back it started a parity check? That's not cool and I am simply annoyed I have a second UnRaid that I am debating if I should upgrade given this experience
September 4, 20232 yr Community Expert 7 minutes ago, johnwhicker said: Upgrade went OK but when back it started a parity check? That's not cool and I am simply annoyed That implies the array shutdown before the reboot did not complete successfully. If you continue to get this when you reboot then you probably need to look into troubleshooting as described here.
September 4, 20232 yr Author 13 minutes ago, itimpi said: That implies the array shutdown before the reboot did not complete successfully. If you continue to get this when you reboot then you probably need to look into troubleshooting as described here. Well is doing parity check and can't touch it. Looking at my IPMI log file it was a forced shutdown? Say is collecting a diagnostic file Where can I find the diagnostic file so I can upload it here?
September 4, 20232 yr 1 hour ago, johnwhicker said: Well is doing parity check and can't touch it. Looking at my IPMI log file it was a forced shutdown? Say is collecting a diagnostic file Where can I find the diagnostic file so I can upload it here? /flash/logs
September 4, 20232 yr Author Again is absolutely SILLY that a system can't actually unmount the darn mounts and just reboot normally 🙂 I had nothing going on the background and this is a very basic, primitive system. Sep 4 08:54:23 NAS-UNRAID-2 emhttpd: Unmounting disks... Sep 4 08:54:23 NAS-UNRAID-2 emhttpd: shcmd (299477): umount /mnt/cache Sep 4 08:54:23 NAS-UNRAID-2 root: umount: /mnt/cache: target is busy. Sep 4 08:54:23 NAS-UNRAID-2 emhttpd: shcmd (299477): exit status: 32 Sep 4 08:54:23 NAS-UNRAID-2 emhttpd: Retry unmounting disk share(s)... Sep 4 08:54:28 NAS-UNRAID-2 emhttpd: Unmounting disks... Sep 4 08:54:28 NAS-UNRAID-2 emhttpd: shcmd (299478): umount /mnt/cache Sep 4 08:54:28 NAS-UNRAID-2 root: umount: /mnt/cache: target is busy. Sep 4 08:54:28 NAS-UNRAID-2 emhttpd: shcmd (299478): exit status: 32 Sep 4 08:54:28 NAS-UNRAID-2 emhttpd: Retry unmounting disk share(s)... Sep 4 08:54:29 NAS-UNRAID-2 root: Status of all loop devices Sep 4 08:54:29 NAS-UNRAID-2 root: /dev/loop1: [2049]:12 (/boot/previous/bzfirmware) Sep 4 08:54:29 NAS-UNRAID-2 root: /dev/loop2: [0043]:260 (/mnt/cache/system/docker/docker.img) Sep 4 08:54:29 NAS-UNRAID-2 root: /dev/loop0: [2049]:10 (/boot/previous/bzmodules) Sep 4 08:54:29 NAS-UNRAID-2 root: Active pids left on /mnt/* Sep 4 08:54:29 NAS-UNRAID-2 root: USER PID ACCESS COMMAND Sep 4 08:54:29 NAS-UNRAID-2 root: /mnt/addons: root kernel mount /mnt/addons Sep 4 08:54:29 NAS-UNRAID-2 root: /mnt/cache: root kernel mount /mnt/cache Sep 4 08:54:29 NAS-UNRAID-2 root: /mnt/disks: root kernel mount /mnt/disks Sep 4 08:54:29 NAS-UNRAID-2 root: /mnt/remotes: root kernel mount /mnt/remotes Sep 4 08:54:29 NAS-UNRAID-2 root: /mnt/rootshare: root kernel mount /mnt/rootshare Sep 4 08:54:29 NAS-UNRAID-2 root: Active pids left on /dev/md* Sep 4 08:54:29 NAS-UNRAID-2 root: Generating diagnostics... Edited September 4, 20232 yr by johnwhicker
September 4, 20232 yr 1 hour ago, johnwhicker said: Again is absolutely SILLY that a system can't actually unmount the darn mounts and just reboot normally 🙂 I had nothing going on the background and this is a very basic, primitive system. Sep 4 08:54:23 NAS-UNRAID-2 emhttpd: Unmounting disks... Sep 4 08:54:23 NAS-UNRAID-2 emhttpd: shcmd (299477): umount /mnt/cache Sep 4 08:54:23 NAS-UNRAID-2 root: umount: /mnt/cache: target is busy. Would you please post the full diagnostics? We need to see what happened before this for a clue as to what is keeping the drive busy
September 4, 20232 yr Author 9 minutes ago, ljm42 said: Would you please post the full diagnostics? We need to see what happened before this for a clue as to what is keeping the drive busy I sure can Sir. Please see attached nas-unraid-2-diagnostics-20230904-0854.zip
September 4, 20232 yr Community Expert Problem is with docker not stopping, but don't see why: Sep 4 08:53:30 NAS-UNRAID-2 root: stopping dockerd ... Sep 4 08:53:31 NAS-UNRAID-2 emhttpd: shcmd (299442): umount /var/lib/docker Sep 4 08:53:31 NAS-UNRAID-2 root: umount: /var/lib/docker: target is busy.
September 4, 20232 yr FWIW, this was an issue that happened when shutting down 6.12.3, it was not related to the upgrade to 6.12.4. However, if there is a bug here it was probably not solved by 6.12.4. The logs show that we issued the `rc.docker stop` command: Sep 4 08:53:21 NAS-UNRAID-2 emhttpd: shcmd (299441): /etc/rc.d/rc.docker stop Sep 4 08:53:30 NAS-UNRAID-2 root: stopping dockerd ... but when we try to unmount the volume, it fails: Sep 4 08:53:31 NAS-UNRAID-2 emhttpd: shcmd (299442): umount /var/lib/docker Sep 4 08:53:31 NAS-UNRAID-2 root: umount: /var/lib/docker: target is busy. Sep 4 08:53:31 NAS-UNRAID-2 emhttpd: shcmd (299442): exit status: 32 We may need to add more logging to figure out what the issue is. For now, I would suggest that you follow the instructions in this post when shutting down: https://forums.unraid.net/topic/144429-unraid-os-version-6124-available/#comment-1300996 i.e. 1) stop the array. 2) `umount /var/lib/docker` if needed, then 3) shutdown/restart. This will avoid any unclean shutdowns by ensuring the array is stopped before shutting down.
September 4, 20232 yr Author Not sure why this was moved to 6.12.3 Unclean Shutdown as I believe will be misleading for the folks here. This is a symptom of my upgrade from 6.12.3 to 6.12.4 and have nothing to do with an intentional shutdown or reboot. All this happened after the upgrade took place and is asking for a reboot and that is exactly what I did. Not sure where the confusion. I do have another UnRaid system due for 6.12.4 which I will not upgrade as I am concerned with the same behavior.
September 4, 20232 yr 31 minutes ago, ljm42 said: FWIW, this was an issue that happened when shutting down 6.12.3, it was not related to the upgrade to 6.12.4. The upgrade changes the files on the flash drive, it does not change the running system. So applying an upgrade will have no effect on whether the running system is able to shutdown properly.
September 4, 20232 yr Author 10 minutes ago, ljm42 said: The upgrade changes the files on the flash drive, it does not change the running system. So applying an upgrade will have no effect on whether the running system is able to shutdown properly. OK, so then how do we explain this unwanted behavior? Obviously when an upgrade is done and a reboot is required I would expect some checks and balances in the upgrade / reboot process? Can't just blindly upgrade and then blindly reboot, hoping that everything is gonna turn ok? I am extremely confused here. Do we need to worry every time we do a simple upgrade now? Very few dockers running in this system: Edited September 4, 20232 yr by johnwhicker
September 4, 20232 yr What I am saying is that the system likely would have had the same unclean shutdown if you had simply restarted it without doing an upgrade to 6.12.4, because the logs show that 6.12.3 issued the `rc.docker stop` command: Sep 4 08:53:21 NAS-UNRAID-2 emhttpd: shcmd (299441): /etc/rc.d/rc.docker stop Sep 4 08:53:30 NAS-UNRAID-2 root: stopping dockerd ... but when it later tried to unmount the volume, it failed: Sep 4 08:53:31 NAS-UNRAID-2 emhttpd: shcmd (299442): umount /var/lib/docker Sep 4 08:53:31 NAS-UNRAID-2 root: umount: /var/lib/docker: target is busy. Sep 4 08:53:31 NAS-UNRAID-2 emhttpd: shcmd (299442): exit status: 32 For now, I would suggest that you follow the instructions in this post when shutting down: https://forums.unraid.net/topic/144429-unraid-os-version-6124-available/#comment-1300996 i.e. 1) stop the array. 2) `umount /var/lib/docker` if needed, then 3) shutdown/restart. This will avoid any unclean shutdowns by ensuring the array is stopped before shutting down.
September 4, 20232 yr Author That is an easy fix and I wish I knew. My assumption was that is fixed in 6.12.3 but I guess is still not fixed. If the system is currently running 6.12.0 - 6.12.2, we're going to suggest that you stop the array at this point. If it gets stuck on "Retry unmounting shares", open a web terminal and type: umount /var/lib/docker The array should now stop successfully (This issue was resolved with 6.12.3)
September 5, 20232 yr I have this issue with 6.12.4, and a little research has shown it's an acknowledged issue with the "parity check tuning" plugin and it's being worked on right now. Edited September 5, 20232 yr by jonathanljs
September 5, 20232 yr Community Expert 27 minutes ago, johnwhicker said: I do not have "Parity Check Tuning" installed. In which case you will not get a spurious notification from the Parity Check Tuning plugin 😊
September 5, 20232 yr Today i did some changes in the Samba settings and therefore i stopped the array. When i start the array i got the unclean shutdown message again. I did no reboot! 6.12.4 & PCT plugin is installed. -faxxe
November 21, 20232 yr Hi folks, Unraid 6.12.5-rc1 makes more changes to Docker so *knocks on wood* it should not be possible for Docker containers to cause an unclean shutdown:
February 13, 20242 yr So Docker has still been holding the /var/lib/docker mount open for some folks. If you have been affected, please test Unraid 6.12.7-rc2 which does a "lazy unmount" of /var/lib/docker. This means instead of trying to unmount and giving up if it can't, it will wait until it is not in use and then automatically unmount. This should eliminate the need to ever have to manually run `umount /var/lib/docker` again.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.