July 9, 20232 yr Hi all, I'd appreciate some help. I have a ZFS pool that won't let go: Jul 9 01:50:16 Diamond emhttpd: Unmounting disks... Jul 9 01:50:16 Diamond emhttpd: shcmd (1774741): /usr/sbin/zpool export sata_top Jul 9 01:50:16 Diamond root: cannot unmount '/mnt/sata_top/system': pool or dataset is busy Jul 9 01:50:16 Diamond emhttpd: shcmd (1774741): exit status: 1 Jul 9 01:50:16 Diamond emhttpd: Retry unmounting disk share(s)... Jul 9 01:50:21 Diamond emhttpd: Unmounting disks... Jul 9 01:50:21 Diamond emhttpd: shcmd (1774742): /usr/sbin/zpool export sata_top Jul 9 01:50:21 Diamond root: cannot unmount '/mnt/sata_top/system': pool or dataset is busy Jul 9 01:50:21 Diamond emhttpd: shcmd (1774742): exit status: 1 Jul 9 01:50:21 Diamond emhttpd: Retry unmounting disk share(s)... Jul 9 01:50:26 Diamond emhttpd: Unmounting disks... Jul 9 01:50:26 Diamond emhttpd: shcmd (1774743): /usr/sbin/zpool export sata_top Jul 9 01:50:26 Diamond root: cannot unmount '/mnt/sata_top/system': pool or dataset is busy Jul 9 01:50:26 Diamond emhttpd: shcmd (1774743): exit status: 1 Jul 9 01:50:26 Diamond emhttpd: Retry unmounting disk share(s)... Jul 9 01:50:31 Diamond emhttpd: Unmounting disks... Jul 9 01:50:31 Diamond emhttpd: shcmd (1774744): /usr/sbin/zpool export sata_top Jul 9 01:50:31 Diamond root: cannot unmount '/mnt/sata_top/system': pool or dataset is busy Jul 9 01:50:31 Diamond emhttpd: shcmd (1774744): exit status: 1 Jul 9 01:50:31 Diamond emhttpd: Retry unmounting disk share(s)... I stopped all of my VMs, my dockers, then stopped the docker and VM service from the gui. The system share IS on that cache pool (sata_top). There's no other activity that I can find and several other posts that suggest finding the process and killing it using lsof (lsof | grep sata_top, etc) don't return with any results. I'm unsure what to do from here but would really like to not have to rebuild parity after a restart. Any other things i can try?
July 9, 20232 yr Author Dang... sorry, I read that a "powerdown -r" command would do what I thought was a clean shutdown so I tried it... and it's now rebuilding parity. Gonna be a long 24 hours. Running losetup now gives me the following (not that it helps now): NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC /dev/loop1 0 0 1 1 /boot/bzfirmware 0 512 /dev/loop2 0 0 1 0 /mnt/sata_top/system/docker/docker.img 0 512 /dev/loop0 0 0 1 1 /boot/bzmodules 0 512 I had just finished moving all of my data off of a different cache drive in preparation for restarting to install new drives, but was concerned about the system at that point so just restarted to minimize any other changes, hoping that the shutdown had been clean. When I was looking for the problem and going through other posts, I did try umounting the /dev/loop2 It said that there was no such device mounted. I didn't try it through the zfs unmount command.
July 11, 20232 yr On 7/9/2023 at 5:55 AM, JorgeB said: post the output of: losetup also having the same problem NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC /dev/loop1 0 0 1 1 /boot/bzfirmware 0 512 /dev/loop2 0 0 1 0 /mnt/cache/system/docker/docker.img 0 512 /dev/loop0 0 0 1 1 /boot/bzmodules 0 512
July 11, 20232 yr Community Expert Solution You can update to v6.12.3-rc2 to test, should be fixed, need to switch to the next branch.
July 11, 20232 yr NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC /dev/loop1 0 0 1 1 /boot/bzfirmware 0 512 /dev/loop0 0 0 1 1 /boot/bzmodules 0 512 /dev/loop3 0 0 1 0 /mnt/cache/system/libvirt/libvirt.img 0 512 Same Problem here, cant fix it, also tried the above, besides updating.
July 12, 20232 yr Community Expert 12 hours ago, Bluebirts1990 said: /dev/loop3 0 0 1 0 /mnt/cache/system/libvirt/libvirt.img In your case it's the libvirt image, this is not common, does it unmount if you type: umount /dev/loop3
July 15, 20232 yr I am having this same issue. I tried updating to the RC version Jorge suggested but unfortunately I still get a bunch of spam of this whenever I try to stop the array: Jul 15 10:28:28 vault emhttpd: Sync filesystems... Jul 15 10:28:28 vault emhttpd: shcmd (1362): sync Jul 15 10:28:28 vault emhttpd: shcmd (1363): /usr/sbin/zfs unmount -a Jul 15 10:28:28 vault emhttpd: shcmd (1364): umount /mnt/user0 Jul 15 10:28:28 vault emhttpd: shcmd (1365): rmdir /mnt/user0 Jul 15 10:28:28 vault emhttpd: shcmd (1366): umount /mnt/user Jul 15 10:28:28 vault root: umount: /mnt/user: target is busy. Jul 15 10:28:28 vault emhttpd: shcmd (1366): exit status: 32 Jul 15 10:28:28 vault emhttpd: shcmd (1367): rmdir /mnt/user Jul 15 10:28:28 vault root: rmdir: failed to remove '/mnt/user': Device or resource busy Jul 15 10:28:28 vault emhttpd: shcmd (1367): exit status: 1 Jul 15 10:28:28 vault emhttpd: shcmd (1369): /usr/local/sbin/update_cron Jul 15 10:28:28 vault emhttpd: Retry unmounting user share(s)... Jul 15 10:28:33 vault emhttpd: shcmd (1370): /usr/sbin/zfs unmount -a Jul 15 10:28:33 vault emhttpd: shcmd (1371): umount /mnt/user Jul 15 10:28:33 vault root: umount: /mnt/user: target is busy. Jul 15 10:28:33 vault emhttpd: shcmd (1371): exit status: 32 Jul 15 10:28:33 vault emhttpd: shcmd (1372): rmdir /mnt/user Jul 15 10:28:33 vault root: rmdir: failed to remove '/mnt/user': Device or resource busy Jul 15 10:28:33 vault emhttpd: shcmd (1372): exit status: 1 Jul 15 10:28:33 vault emhttpd: shcmd (1374): /usr/local/sbin/update_cron Jul 15 10:28:33 vault emhttpd: Retry unmounting user share(s)... Jul 15 10:28:38 vault emhttpd: shcmd (1375): /usr/sbin/zfs unmount -a Jul 15 10:28:38 vault emhttpd: shcmd (1376): umount /mnt/user Jul 15 10:28:38 vault root: umount: /mnt/user: target is busy. Jul 15 10:28:38 vault emhttpd: shcmd (1376): exit status: 32 Jul 15 10:28:38 vault emhttpd: shcmd (1377): rmdir /mnt/user and it just hangs forever, forcing me to have to reboot.
July 16, 20232 yr Community Expert 17 hours ago, jxi7eoiigzns7oaf7 said: I am having this same issue. Please post the complete diagnostics.
September 28, 20232 yr I figured out the problem eventually. It was a bad stick of RAM. I replaced it and have not had this issue since.
February 8, 20242 yr thanks mine was docker.img, unmounting it allowed the array to stop. root@larkbox:~# losetup NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC /dev/loop1 0 0 1 1 /boot/bzfirmware 0 512 /dev/loop2 0 0 1 0 /mnt/fastdata/system/docker/docker.img 0 512 /dev/loop0 0 0 1 1 /boot/bzmodules 0 512 root@larkbox:~# umount /dev/loop2
February 6, 20251 yr I am now having the same problem in 7.0.0 stable. Unraid is now trying for more than an hour to export my chache. Down are my full diagnostics. Output of losetup was: NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC /dev/loop1 0 0 1 1 /boot/bzmodules 0 512 /dev/loop0 0 0 1 1 /boot/bzfirmware 0 512 so there are no other loops to unmount than these from unraid. output of zpool status cache was: zfs get mounted cache pool: cache state: ONLINE scan: scrub repaired 0B in 00:03:03 with 0 errors on Sat Jan 25 22:03:47 2025 config: NAME STATE READ WRITE CKSUM cache ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 nvme1n1p1 ONLINE 0 0 0 nvme0n1p1 ONLINE 0 0 0 errors: No known data errors NAME PROPERTY VALUE SOURCE cache mounted no - What else can I do to stop the array correctly? The error occurs with every array stop. You can ignore the corrected faults of the parity in the syslog attached. It's caused by a hard shutdown a couple of days ago. diagnostics-20250206-1226.zip
February 6, 20251 yr Community Expert The problem appears to be docker, see if you can stop the array if you leave docker service disabled.
February 6, 20251 yr I got exactly the some problem like @Xerxes . LOSETUP looks identical and I already stopped docker service manually - but without success. The array will not stop
February 6, 20251 yr 1 hour ago, JorgeB said: The problem appears to be docker, see if you can stop the array if you leave docker service disabled. 28 minutes ago, independence said: I already stopped docker service manually At the moment docker and docker daemon is not running. docker info Client: Version: 27.0.3 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.15.1 Path: /usr/libexec/docker/cli-plugins/docker-buildx Server: ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? errors pretty printing info /etc/rc.d/rc.docker status Docker daemon is not running. How can I stop my array safely now? I don't want to perform another hard shutdown using the reset button. Before hitting the "Stop Array" button in the GUI, I manually stop all containers to make sure that no container is using the cache.
February 6, 20251 yr Community Expert If you cannot find out what is using it, you will need to force a reboot, then 3 hours ago, JorgeB said: see if you can stop the array if you leave docker service disabled.
February 6, 20251 yr for my end, I stopped all docker services manually and AFTER that I tried to stop the array without success.
February 6, 20251 yr Community Expert That's not what I'm asking, I'm asking to disable the docker service, start the array and then see if it stops.
February 6, 20251 yr And for the moment? Restart the server by a hard-reset? Edited February 6, 20251 yr by Xerxes
February 6, 20251 yr Community Expert 1 hour ago, independence said: stopped all docker services manually If by that you mean you have stopped each of your individual containers, that's not enough 1 hour ago, JorgeB said: That's not what I'm asking, I'm asking to disable the docker service, start the array and then see if it stops. Go to Settings - Docker and disable Docker service. You should probably do the same for VM Manager.
February 6, 20251 yr 3 minutes ago, trurl said: Go to Settings - Docker and disable Docker service. You should probably do the same for VM Manager. Did that. System log is still flooded with these errors Feb 6 17:56:59 nasgul emhttpd: Unmounting disks... Feb 6 17:56:59 nasgul emhttpd: shcmd (229694): /usr/sbin/zpool export -f cache Feb 6 17:56:59 nasgul root: cannot export 'cache': pool is busy Feb 6 17:56:59 nasgul emhttpd: shcmd (229694): exit status: 1 Feb 6 17:56:59 nasgul emhttpd: Retry unmounting disk share(s)...
February 6, 20251 yr 5 minutes ago, trurl said: Do you have a terminal open anywhere? No. To be sure that no extern processes were suspending my server I disconneced the lan-cable and pluged in a monitor.
February 6, 20251 yr Community Expert 3 hours ago, JorgeB said: If you cannot find out what is using it, you will need to force a reboot
February 7, 20251 yr 15 hours ago, JorgeB said: That's not what I'm asking, I'm asking to disable the docker service, start the array and then see if it stops. Ok, got it - I will try that later and give feedback 14 hours ago, trurl said: If by that you mean you have stopped each of your individual containers, that's not enough Go to Settings - Docker and disable Docker service. You should probably do the same for VM Manager. no, I did not stopped each individual container, I stopped the whole docker service in the settings. I will test it like @JorgeB said.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.