April 23, 201412 yr i'm having an issue with one of my disks (disk5). i am just recovering from getting a string of i/o errors after (stupidly) trying to hotswap a drive when the array was active. after restarting the array and swapping out a defective drive, i notice that one of the disks is constantly active in the array (i.e. while other drives are flashing and buzzing with activity, this drive's LED is solid and doesn't change). whenever i try to shut down the array, all the disks are successfully unmounted, except for this one. i get a string of errors in the syslog stating that the disk could not be unmounted. an excerpt from the syslog is shown to illustrate the point. Apr 23 15:38:19 Tower emhttp: shcmd (1008): /usr/local/sbin/emhttp_event stopping_svcs Apr 23 15:38:19 Tower emhttp_event: stopping_svcs Apr 23 15:38:19 Tower emhttp: Stop AVAHI... Apr 23 15:38:19 Tower emhttp: shcmd (1009): /etc/rc.d/rc.avahidaemon stop |& logger Apr 23 15:38:19 Tower logger: Stopping Avahi mDNS/DNS-SD Daemon: stopped Apr 23 15:38:19 Tower avahi-daemon[26280]: Got SIGTERM, quitting. Apr 23 15:38:19 Tower avahi-dnsconfd[26289]: read(): EOF Apr 23 15:38:19 Tower avahi-daemon[26280]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.1.10. Apr 23 15:38:19 Tower avahi-daemon[26280]: avahi-daemon 0.6.31 exiting. Apr 23 15:38:19 Tower kernel: mdcmd (44): nocheck Apr 23 15:38:19 Tower kernel: md: nocheck_array: check not active Apr 23 15:38:19 Tower emhttp: shcmd (1010): /etc/rc.d/rc.avahidnsconfd stop |& logger Apr 23 15:38:19 Tower logger: Stopping Avahi mDNS/DNS-SD DNS Server Configuration Daemon: stopped Apr 23 15:38:19 Tower emhttp: shcmd (1011): ps axc | grep -q rpc.mountd Apr 23 15:38:19 Tower emhttp: Stop NFS... Apr 23 15:38:19 Tower emhttp: shcmd (1012): /etc/rc.d/rc.nfsd stop |& logger Apr 23 15:38:19 Tower mountd[26272]: Caught signal 15, un-registering and exiting. Apr 23 15:38:20 Tower emhttp: Stop SMB... Apr 23 15:38:20 Tower emhttp: shcmd (1013): /etc/rc.d/rc.samba stop |& logger Apr 23 15:38:20 Tower emhttp: shcmd (1014): rm /etc/avahi/services/smb.service &> /dev/null Apr 23 15:38:20 Tower emhttp: Spinning up all drives... Apr 23 15:38:20 Tower emhttp: shcmd (1015): /usr/sbin/hdparm -S0 /dev/sde &> /dev/null Apr 23 15:38:20 Tower kernel: mdcmd (45): spinup 0 Apr 23 15:38:20 Tower kernel: mdcmd (46): spinup 1 Apr 23 15:38:20 Tower kernel: mdcmd (47): spinup 2 Apr 23 15:38:20 Tower kernel: mdcmd (48): spinup 3 Apr 23 15:38:20 Tower kernel: mdcmd (49): spinup 4 Apr 23 15:38:20 Tower kernel: mdcmd (50): spinup 5 Apr 23 15:38:20 Tower kernel: mdcmd (51): spinup 6 Apr 23 15:38:20 Tower kernel: nfsd: last server has exited, flushing export cache Apr 23 15:38:21 Tower emhttp: Sync filesystems... Apr 23 15:38:21 Tower emhttp: shcmd (1016): sync Apr 23 15:38:25 Tower emhttp: shcmd (1017): /usr/local/sbin/emhttp_event unmounting_disks Apr 23 15:38:25 Tower emhttp_event: unmounting_disks Apr 23 15:38:25 Tower cache_dirs: killing cache_dirs process 31790 Apr 23 15:38:45 Tower rc.unRAID[1229][1230]: Processing /etc/rc.d/rc.unRAID.d/ kill scripts. Apr 23 15:38:59 Tower emhttp: shcmd (1018): set -o pipefail ; umount /mnt/user |& logger Apr 23 15:38:59 Tower emhttp: shcmd (1019): rmdir /mnt/user |& logger Apr 23 15:38:59 Tower emhttp: shcmd (1020): set -o pipefail ; umount /mnt/user0 |& logger Apr 23 15:38:59 Tower shfs/user0: fuse_main exit: 0 Apr 23 15:38:59 Tower emhttp: shcmd (1021): rmdir /mnt/user0 |& logger Apr 23 15:38:59 Tower emhttp: shcmd (1022): crontab -c /etc/cron.d -d &> /dev/null Apr 23 15:38:59 Tower emhttp: Unmounting disks... Apr 23 15:38:59 Tower emhttp: shcmd (1023): umount /mnt/disk1 |& logger Apr 23 15:38:59 Tower shfs/user: fuse_main exit: 0 Apr 23 15:38:59 Tower emhttp: shcmd (1024): rmdir /mnt/disk1 |& logger Apr 23 15:38:59 Tower emhttp: shcmd (1025): umount /mnt/disk2 |& logger Apr 23 15:38:59 Tower emhttp: shcmd (1026): rmdir /mnt/disk2 |& logger Apr 23 15:38:59 Tower emhttp: shcmd (1027): umount /mnt/disk3 |& logger Apr 23 15:39:00 Tower emhttp: shcmd (1028): rmdir /mnt/disk3 |& logger Apr 23 15:39:00 Tower emhttp: shcmd (1029): umount /mnt/disk5 |& logger Apr 23 15:39:00 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:00 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:00 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:00 Tower emhttp: shcmd (1030): umount /mnt/disk6 |& logger Apr 23 15:39:00 Tower emhttp: shcmd (1031): rmdir /mnt/disk6 |& logger Apr 23 15:39:00 Tower emhttp: shcmd (1032): umount /mnt/cache |& logger Apr 23 15:39:00 Tower emhttp: shcmd (1033): rmdir /mnt/cache |& logger Apr 23 15:39:00 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:05 Tower emhttp: Unmounting disks... Apr 23 15:39:05 Tower emhttp: shcmd (1034): umount /mnt/disk5 |& logger Apr 23 15:39:05 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:05 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:05 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:05 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:10 Tower emhttp: Unmounting disks... Apr 23 15:39:10 Tower emhttp: shcmd (1035): umount /mnt/disk5 |& logger Apr 23 15:39:10 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:10 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:10 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:10 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:15 Tower emhttp: Unmounting disks... Apr 23 15:39:15 Tower emhttp: shcmd (1036): umount /mnt/disk5 |& logger Apr 23 15:39:15 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:15 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:15 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:15 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:20 Tower emhttp: Unmounting disks... Apr 23 15:39:20 Tower emhttp: shcmd (1037): umount /mnt/disk5 |& logger Apr 23 15:39:20 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:20 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:20 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:20 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:25 Tower emhttp: Unmounting disks... Apr 23 15:39:25 Tower emhttp: shcmd (1038): umount /mnt/disk5 |& logger Apr 23 15:39:25 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:25 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:25 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:25 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:30 Tower emhttp: Unmounting disks... Apr 23 15:39:30 Tower emhttp: shcmd (1039): umount /mnt/disk5 |& logger Apr 23 15:39:30 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:30 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:30 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:30 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:35 Tower emhttp: Unmounting disks... Apr 23 15:39:35 Tower emhttp: shcmd (1040): umount /mnt/disk5 |& logger Apr 23 15:39:35 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:35 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:35 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:35 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:40 Tower emhttp: Unmounting disks... Apr 23 15:39:40 Tower emhttp: shcmd (1041): umount /mnt/disk5 |& logger Apr 23 15:39:40 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:40 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:40 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:40 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:45 Tower emhttp: Unmounting disks... Apr 23 15:39:45 Tower emhttp: shcmd (1042): umount /mnt/disk5 |& logger Apr 23 15:39:45 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:45 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:45 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:45 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:50 Tower emhttp: Unmounting disks... Apr 23 15:39:50 Tower emhttp: shcmd (1043): umount /mnt/disk5 |& logger Apr 23 15:39:50 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:50 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:50 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:50 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:55 Tower emhttp: Unmounting disks... Apr 23 15:39:55 Tower emhttp: shcmd (1044): umount /mnt/disk5 |& logger Apr 23 15:39:55 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:39:55 Tower logger: (In some cases useful info about processes that use Apr 23 15:39:55 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:39:55 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:39:58 Tower in.telnetd[1526]: connect from 192.168.1.7 (192.168.1.7) Apr 23 15:40:00 Tower emhttp: Unmounting disks... Apr 23 15:40:00 Tower emhttp: shcmd (1045): umount /mnt/disk5 |& logger Apr 23 15:40:00 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:00 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:00 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:00 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:01 Tower login[1527]: ROOT LOGIN on '/dev/pts/0' from '192.168.1.7' Apr 23 15:40:05 Tower emhttp: Unmounting disks... Apr 23 15:40:05 Tower emhttp: shcmd (1046): umount /mnt/disk5 |& logger Apr 23 15:40:05 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:05 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:05 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:05 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:10 Tower emhttp: Unmounting disks... Apr 23 15:40:10 Tower emhttp: shcmd (1047): umount /mnt/disk5 |& logger Apr 23 15:40:10 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:10 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:10 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:10 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:15 Tower emhttp: Unmounting disks... Apr 23 15:40:15 Tower emhttp: shcmd (1048): umount /mnt/disk5 |& logger Apr 23 15:40:15 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:15 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:15 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:15 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:20 Tower emhttp: Unmounting disks... Apr 23 15:40:20 Tower emhttp: shcmd (1049): umount /mnt/disk5 |& logger Apr 23 15:40:20 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:20 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:20 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:20 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:25 Tower emhttp: Unmounting disks... Apr 23 15:40:25 Tower emhttp: shcmd (1050): umount /mnt/disk5 |& logger Apr 23 15:40:25 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:25 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:25 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:25 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:30 Tower emhttp: Unmounting disks... Apr 23 15:40:30 Tower emhttp: shcmd (1051): umount /mnt/disk5 |& logger Apr 23 15:40:30 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:30 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:30 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:30 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:35 Tower emhttp: Unmounting disks... Apr 23 15:40:35 Tower emhttp: shcmd (1052): umount /mnt/disk5 |& logger Apr 23 15:40:35 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:35 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:35 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:35 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:40 Tower emhttp: Unmounting disks... Apr 23 15:40:40 Tower emhttp: shcmd (1053): umount /mnt/disk5 |& logger Apr 23 15:40:40 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:40 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:40 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:40 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:45 Tower emhttp: Unmounting disks... Apr 23 15:40:45 Tower emhttp: shcmd (1054): umount /mnt/disk5 |& logger Apr 23 15:40:45 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:45 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:45 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:45 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:50 Tower emhttp: Unmounting disks... Apr 23 15:40:50 Tower emhttp: shcmd (1055): umount /mnt/disk5 |& logger Apr 23 15:40:50 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:50 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:50 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:50 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:40:55 Tower emhttp: Unmounting disks... Apr 23 15:40:55 Tower emhttp: shcmd (1056): umount /mnt/disk5 |& logger Apr 23 15:40:55 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:40:55 Tower logger: (In some cases useful info about processes that use Apr 23 15:40:55 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:40:55 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:41:00 Tower emhttp: Unmounting disks... Apr 23 15:41:00 Tower emhttp: shcmd (1057): umount /mnt/disk5 |& logger Apr 23 15:41:00 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:41:00 Tower logger: (In some cases useful info about processes that use Apr 23 15:41:00 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:41:00 Tower emhttp: Retry unmounting disk share(s)... Apr 23 15:41:05 Tower emhttp: Unmounting disks... Apr 23 15:41:05 Tower emhttp: shcmd (1058): umount /mnt/disk5 |& logger Apr 23 15:41:05 Tower logger: umount: /mnt/disk5: device is busy. Apr 23 15:41:05 Tower logger: (In some cases useful info about processes that use Apr 23 15:41:05 Tower logger: the device is found by lsof( or fuser(1)) Apr 23 15:41:05 Tower emhttp: Retry unmounting disk share(s)... can someone help me on how to troubleshoot this?
April 23, 201412 yr Author I should also note that if i try writing to and from /mnt/disk5 i can do that without a problem. not sure why the indicator for the drive is unrelenting and why stopping the array seems to be blocked by it.
April 24, 201412 yr Author I feel like i am talking to myself ... but I tried; lsof |grep /mnt/disk5 and I got the following response shfs 13742 root 4r REG 9,5 4314355750 174980 /mnt/disk5/movies/Les Miserables.mkv Does this mean that this movie file is somehow locking up my disk5? It's been several hours and it is still showing. Also what is shfs? none of my HTPC clients are accessing this file, so I am not sure why this file is in use? I'm hoping someone might give me some help... *** update: it just occurred to me that my crashplan engine is active. could shfs represent the process that enables crashplan to backup to the cloud?
April 24, 201412 yr POWERDOWN seems to be the standard for a clean shutdown. (search forums if you don't have it installed. I believe its also included as part of DYNAMIX, too.) It attempts to shutdown cleanly by ending all running processes, plugins, etc., and then dismounting the disks and shutting down power, etc... However, I seem to recall even it has trouble stopping some processes. Crashplan might very well be one of the ill-behaved that has to be stopped manually. You might search the forums for a discussion about stopping crashplan cleanly.
Archived
This topic is now archived and is closed to further replies.