Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Array stuck in stopping mode.

Featured Replies

Hi,

I needed to stop my array. For that I stopped all the docker containers and the docker service itself. Same with the VMs. Then hit the stop button. The array still did not go down.
Syslog shows:

Sep 20 16:06:04 nedio-server emhttpd: Unmounting disks...
Sep 20 16:06:04 nedio-server emhttpd: shcmd (39778678): /usr/sbin/zpool export -f cache-b
Sep 20 16:06:04 nedio-server root: cannot export 'cache-b': pool is busy
Sep 20 16:06:04 nedio-server emhttpd: shcmd (39778678): exit status: 1
Sep 20 16:06:04 nedio-server emhttpd: Retry unmounting disk share(s)...


I tried to check if there's anything holding the pool. Can't see anything.

On the Main tab - the array and the cache pools show online. Note: there is a small write (400-700 KB/s) that happens every few seconds on the cache-b.

Please advise what should I do. I want to stop the array in order to start it in maintenance mode, so that I can check the array xfs for errors and then rebuild one of the disks which is showing as x Disabled.

Please see attached the diagnostics.

nedio-server-diagnostics-20250920-1610.zip

Solved by JorgeB

These diags were taken after docker was stopped? Docker is still running. Open a web terminal and run:

/etc/rc.d/rc.docker stop

If that doesn't help, try:

/etc/rc.d/rc.docker force_stop

Also there was a bash shell open at the time these diags were taken, something as simple as a bash terminal cd'd to the share will block it from unmounting. Be sure to exit all Console/SSH/web terminals

  • Author

Yeah, docker was already stopped at the time of the diags taking (from the settings > docker > enable docker=No). The web-console was also closed (after diag extract).

Still I have run the command you suggested.

SRV:~# /etc/rc.d/rc.docker status
Docker daemon is not running.
SRV:~# /etc/rc.d/rc.docker stop
rc.docker: Stopping containers...
rc.docker: Stopping network...
rc.docker: Stopping Docker daemon...
rc.docker: Docker daemon...  Already stopped.
SRV:~# tail -f /var/log/syslog
Sep 20 21:22:35 nedio-server emhttpd: Unmounting disks...
Sep 20 21:22:35 nedio-server emhttpd: shcmd (39782356): /usr/sbin/zpool export -f cache-b
Sep 20 21:22:36 nedio-server root: cannot export 'cache-b': pool is busy
Sep 20 21:22:36 nedio-server emhttpd: shcmd (39782356): exit status: 1
Sep 20 21:22:36 nedio-server emhttpd: Retry unmounting disk share(s)...
Sep 20 21:22:41 nedio-server emhttpd: Unmounting disks...
Sep 20 21:22:41 nedio-server emhttpd: shcmd (39782357): /usr/sbin/zpool export -f cache-b
Sep 20 21:22:41 nedio-server root: cannot export 'cache-b': pool is busy
Sep 20 21:22:41 nedio-server emhttpd: shcmd (39782357): exit status: 1
Sep 20 21:22:41 nedio-server emhttpd: Retry unmounting disk share(s)...
SRV:~#

Isn't there a way to figure out what could be hugging the pool cache-b ?

Isn't there a way to figure out what could be hugging the pool cache-b ?

You can try this, but it doesn't catch everything

lsof | grep /mnt

Your previous diags showed a bash terminal was open, let's see if that is closed. If you run this from a web terminal there should only be one result (the one in use by the web terminal). If there are more then paste the output so we can kill them

ps -f | grep "bash" | grep -v "grep"
  • Author

Here's the outcome:

SRV:~# lsof | grep /mnt
SRV:~# ps -f | grep "bash" | grep -v "grep"
root     3081899 3081748  0 23:54 pts/0    00:00:00 /bin/bash --login
SRV:~#
  • Community Expert

I see you are using a docker folder with the native zfs driver. This is not recommended and is known to have performance and other issues due to the multitude of legacy datasets created, recommend recreating it using the overlay2 driver, or better yet, changing to a Docker image, since in my experience, Docker folders also can cause other strange issues.

https://docs.unraid.net/unraid-os/release-notes/7.0.0/#add-support-for-overlay2-storage-driver

  • Author

I had switched to docker folder as my docker image file would get corrupt several times (due to not being able to shutdown the system cleanly and having to hard restart).. I will look into re-creating it using overlay2 driver, thanks!

For now - any idea how do I proceed? Is it that the only option I have a hard restart again? Since my array is not ok, do I risk to get in trouble even more if I hard restart?

  • Community Expert
  • Solution

A hard restart may be the only option; other filesystems are already unmounted, and ZFS typically doesn't have issues with an unclean shutdown.

  • Author

Ok, hard restarted 🥶, started the array in maintenance mode (docker and VM are still off). Did the XFS check on disk 2 - no errors detected 🥳

Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - zero log...
        - scan filesystem freespace and inode maps...
        - found root inode chunk
Phase 3 - for each AG...
        - scan (but don't clear) agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
        - traversing filesystem ...
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
No modify flag set, skipping filesystem flush and exiting.

 
 No file system corruption detected.

The disc 2 still shows disabled. What are the next steps? I can't find the rebuilding-a-drive-onto-itself in the unraid documentation.. Did the procedure of restoring the disabled disc change? How should I proceed to fix the array?

  • Community Expert

If the emulated disk2 is mounting and the contents look correct:

  • stop array

  • unassign disk2

  • start array

  • stop array

  • reassign disk2

  • start array to begin the rebuild

  • Author

Thank you @JorgeB! I think the answers to both parts of the condition you mentioned are "Yes". But just to clarify, how/where do I see if the emulated disk 2 is mounting? It shows as disabled in the array with a mark.

  • Community Expert

Look at the used/free stats for it on Main, they will show if the disk is mounting. It will show "unmountable" if it's not, and if it's mounting, the data will be accessible under the shares and /mnt/disk2

  • Author

ok, my disk 2 was rebuilt. no errors so far 🤞.

Now I want to eliminate the potential reason of the cache-b holding back from array stop/clean reboot..

I have changed the docker settings to be overlay2. Did not enable docker yet.

Also I have moved everything from cache-b to array (using the mover).

Now, since there is nothing on the cache-b, I would expect to see 0 on the sizes in Main tab:

image.png

But I see some misterious 35 GB on it. Then I noticed that there are 344 snapshots (legacy) of the system share folder in the cache-b pool. (those grey/yellow/blue photo camera icons are confusing 😕)

image.png

I want to clean the cache-b. Can I just delete these snapshots? I need to get rid of those legacy snapshots and stop them being taken.

  • Community Expert

Those are created by the Docker folder with the native ZFS driver; you should be able to delete them all with zfs destroy -R cache-b/system

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.