April 29, 201016 yr I rebooted my server. I do not think the shutdown went so smoothly. There was an out of memory problem. I tried to shutdown as gracefully as possible. When I rebooted, and refreshed emhttp I saw all my drives say unformtatted. Since it was all the drives. I waited a minute, pressed STOP, Then pressed START. The drives came up. I looked at the syslog later on. I saw that the mount command was issued after emhttp was started. I believe it was taking a long time, I can only surmise that it was replaying the journal. I would have expected the message "Mounting" in this case, rather then Unformatted. But it's not something I normally do or look at this closely. Now in regards to the out of memory situation, I believe this to be my own issue. I was doing massive md5 and par2's on a very large directory. It must have used allot of ram as I got an OOM during the write of the par2 files. I do not believe this to be the problem of unRAID. It's how unRAID handled the startup after I rebooted. I have not tried to make it happen again.
April 29, 201016 yr Your case then had nothing to do with adding drives, correct? Do you recall any other error messages in the system log? It should not matter in the code how long it takes to mount.
April 30, 201016 yr Author Your case then had nothing to do with adding drives, correct? Do you recall any other error messages in the system log? It should not matter in the code how long it takes to mount. I did not add or remove any drives. There were no changes in the system layout what so ever. It just took a long time to mount initially. When I looked in the log all I saw was the series of mount commands.
April 30, 201016 yr Author Come to think of it.. I saved a syslog right away. I forgot about that. It's attached. After review, it's not that the mount took too long. It looks like the mounts failed. Apr 17 18:09:52 Atlas kernel: mdcmd (22): start STOPPED Apr 17 18:09:52 Atlas kernel: unraid: allocating 95308K for 2784 stripes (8 disks) Apr 17 18:09:52 Atlas kernel: md1: running, size: 976762552 blocks Apr 17 18:09:52 Atlas kernel: md2: running, size: 1465138552 blocks Apr 17 18:09:52 Atlas kernel: md3: running, size: 976762552 blocks Apr 17 18:09:52 Atlas kernel: md4: running, size: 976762552 blocks Apr 17 18:09:52 Atlas kernel: md5: running, size: 976762552 blocks Apr 17 18:09:52 Atlas kernel: md6: running, size: 976762552 blocks Apr 17 18:09:52 Atlas kernel: md7: running, size: 1465138552 blocks Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk6 Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk3 Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk7 Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk1 Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk5 Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk4 Apr 17 18:09:52 Atlas emhttp: shcmd (11): mkdir /mnt/disk2 Apr 17 18:09:52 Atlas emhttp: shcmd (13): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md6 /mnt/disk6 >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: shcmd (13): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md3 /mnt/disk3 >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: shcmd (14): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md5 /mnt/disk5 >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: shcmd (15): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md7 /mnt/disk7 >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: shcmd (16): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: shcmd (17): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 >/dev/null 2>&1 Apr 17 18:09:52 Atlas kernel: mdcmd (24): check Apr 17 18:09:52 Atlas kernel: md: recovery thread woken up ... Apr 17 18:09:52 Atlas emhttp: shcmd (18): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md4 /mnt/disk4 >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (18): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk7 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (19): rmdir /mnt/disk7 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (19): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk5 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (20): rmdir /mnt/disk5 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (20): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk2 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (21): rmdir /mnt/disk2 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (21): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk6 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (22): rmdir /mnt/disk6 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (22): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk3 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (23): rmdir /mnt/disk3 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (24): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk4 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (25): rmdir /mnt/disk4 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (25): exit status: 32 Apr 17 18:09:52 Atlas emhttp: disk1 mount error: 32 Apr 17 18:09:52 Atlas emhttp: shcmd (26): rmdir /mnt/disk1 Apr 17 18:09:52 Atlas emhttp: shcmd (32): rm /etc/samba/smb-shares.conf >/dev/null 2>&1 Apr 17 18:09:52 Atlas emhttp: _shcmd: shcmd (32): exit status: 1 Apr 17 18:09:52 Atlas emhttp: shcmd (33): cp /etc/exports- /etc/exports Apr 17 18:09:52 Atlas emhttp: shcmd (34): killall -HUP smbd Apr 17 18:09:52 Atlas emhttp: shcmd (35): /etc/rc.d/rc.nfsd restart | logger Apr 17 18:09:52 Atlas kernel: md: recovery thread has nothing to resync Apr 17 18:09:52 Atlas emhttp: shcmd (35): cp /var/spool/cron/crontabs/root- /var/spool/cron/crontabs/root Apr 17 18:09:52 Atlas emhttp: shcmd (36): echo '# Generated mover schedule:' >>/var/spool/cron/crontabs/root Apr 17 18:09:52 Atlas emhttp: shcmd (37): echo '40 3 * * * /usr/local/sbin/mover 2>&1 | logger' >>/var/spool/cron/crontabs/root Apr 17 18:09:52 Atlas emhttp: shcmd (38): crontab /var/spool/cron/crontabs/root Apr 17 18:09:53 Atlas S02-blockdev[2215]: Devices Online Apr 17 18:09:53 Atlas S02-sync-etc[2459]: Processing /boot/custom/etc/rc.d/S02-sync-etc Apr 17 18:09:53 Atlas S02-waitmnt_disk8[2647]: Processing /boot/custom/etc/rc.d/S02-waitmnt_disk8 Apr 17 18:09:53 Atlas S02-waitmnt_disk8[2647]: sh ..... Apr 17 18:16:43 Atlas emhttp: shcmd (40): /etc/rc.d/rc.samba stop | logger Apr 17 18:16:43 Atlas emhttp: shcmd (41): /etc/rc.d/rc.nfsd stop | logger Apr 17 18:16:43 Atlas mountd[2769]: Caught signal 15, un-registering and exiting. Apr 17 18:16:44 Atlas emhttp: Spinning up all drives... Apr 17 18:16:44 Atlas emhttp: shcmd (42): sync Apr 17 18:16:44 Atlas kernel: nfsd: last server has exited, flushing export cache Apr 17 18:16:44 Atlas kernel: mdcmd (70): spinup 0 Apr 17 18:16:44 Atlas kernel: mdcmd (71): spinup 1 Apr 17 18:16:44 Atlas kernel: mdcmd (72): spinup 2 Apr 17 18:16:44 Atlas kernel: mdcmd (73): spinup 3 Apr 17 18:16:44 Atlas kernel: mdcmd (74): spinup 4 Apr 17 18:16:44 Atlas kernel: mdcmd (75): spinup 5 Apr 17 18:16:44 Atlas kernel: mdcmd (76): spinup 6 Apr 17 18:16:44 Atlas kernel: mdcmd (77): spinup 7 Apr 17 18:16:44 Atlas kernel: mdcmd (78): stop Apr 17 18:16:44 Atlas kernel: md1: stopping Apr 17 18:16:44 Atlas kernel: md2: stopping Apr 17 18:16:44 Atlas kernel: md3: stopping Apr 17 18:16:44 Atlas kernel: md4: stopping Apr 17 18:16:45 Atlas kernel: md5: stopping Apr 17 18:16:45 Atlas kernel: md6: stopping Apr 17 18:16:45 Atlas kernel: md7: stopping Apr 17 18:16:45 Atlas emhttp: shcmd (43): rm /etc/samba/smb-shares.conf >/dev/null 2>&1 Apr 17 18:16:45 Atlas emhttp: shcmd (44): cp /etc/exports- /etc/exports Apr 17 18:16:45 Atlas emhttp: shcmd (45): /etc/rc.d/rc.samba start | logger Apr 17 18:16:45 Atlas logger: Starting Samba: /usr/sbin/nmbd -D Apr 17 18:16:45 Atlas logger: /usr/sbin/smbd -D Apr 17 18:16:45 Atlas emhttp: shcmd (46): /etc/rc.d/rc.nfsd start | logger Apr 17 18:16:53 Atlas emhttp: shcmd (47): /usr/local/sbin/set_ncq sdc 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (48): /usr/local/sbin/set_ncq sda 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (49): /usr/local/sbin/set_ncq sdb 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (50): /usr/local/sbin/set_ncq sde 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (51): /usr/local/sbin/set_ncq sdf 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (52): /usr/local/sbin/set_ncq sdg 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (53): /usr/local/sbin/set_ncq sdh 1 >/dev/null Apr 17 18:16:53 Atlas emhttp: shcmd (54): /usr/local/sbin/set_ncq sdi 1 >/dev/null Apr 17 18:16:53 Atlas kernel: mdcmd (85): start STOPPED Apr 17 18:16:53 Atlas kernel: unraid: allocating 95308K for 2784 stripes (8 disks) Apr 17 18:16:53 Atlas kernel: md1: running, size: 976762552 blocks Apr 17 18:16:53 Atlas kernel: md2: running, size: 1465138552 blocks Apr 17 18:16:53 Atlas kernel: md3: running, size: 976762552 blocks Apr 17 18:16:53 Atlas kernel: md4: running, size: 976762552 blocks Apr 17 18:16:53 Atlas kernel: md5: running, size: 976762552 blocks Apr 17 18:16:53 Atlas kernel: md6: running, size: 976762552 blocks Apr 17 18:16:53 Atlas kernel: md7: running, size: 1465138552 blocks Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk4 Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk5 Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk6 Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk3 Apr 17 18:16:53 Atlas kernel: mdcmd (87): check Apr 17 18:16:53 Atlas kernel: md: recovery thread woken up ... Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk1 Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk2 Apr 17 18:16:53 Atlas emhttp: shcmd (56): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md6 /mnt/disk6 >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (55): mkdir /mnt/disk7 Apr 17 18:16:53 Atlas emhttp: shcmd (57): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (58): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md5 /mnt/disk5 >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (59): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (60): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md7 /mnt/disk7 >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (61): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md3 /mnt/disk3 >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (62): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md4 /mnt/disk4 >/dev/null 2>&1 Apr 17 18:16:53 Atlas kernel: REISERFS (device md6): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md6): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md5): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md5): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md1): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md1): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md2): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md2): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md7): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md7): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md3): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md3): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md4): found reiserfs format "3.6" with standard journal Apr 17 18:16:53 Atlas kernel: REISERFS (device md4): using ordered data mode Apr 17 18:16:53 Atlas kernel: REISERFS (device md7): journal params: device md7, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md7): checking transaction log (md7) Apr 17 18:16:53 Atlas kernel: REISERFS (device md6): journal params: device md6, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md6): checking transaction log (md6) Apr 17 18:16:53 Atlas kernel: REISERFS (device md4): journal params: device md4, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md4): checking transaction log (md4) Apr 17 18:16:53 Atlas kernel: REISERFS (device md5): journal params: device md5, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md5): checking transaction log (md5) Apr 17 18:16:53 Atlas kernel: REISERFS (device md1): journal params: device md1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md1): checking transaction log (md1) Apr 17 18:16:53 Atlas kernel: REISERFS (device md3): journal params: device md3, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md3): checking transaction log (md3) Apr 17 18:16:53 Atlas kernel: REISERFS (device md2): journal params: device md2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 17 18:16:53 Atlas kernel: REISERFS (device md2): checking transaction log (md2) Apr 17 18:16:53 Atlas kernel: REISERFS (device md3): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: REISERFS (device md4): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: REISERFS (device md5): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: REISERFS (device md7): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: REISERFS (device md1): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: REISERFS (device md6): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: REISERFS (device md2): Using r5 hash to sort names Apr 17 18:16:53 Atlas kernel: md: recovery thread has nothing to resync Apr 17 18:16:53 Atlas emhttp: shcmd (69): rm /etc/samba/smb-shares.conf >/dev/null 2>&1 Apr 17 18:16:53 Atlas emhttp: shcmd (70): cp /etc/exports- /etc/exports Apr 17 18:16:53 Atlas emhttp: shcmd (71): mkdir /mnt/user Apr 17 18:16:53 Atlas emhttp: shcmd (72): /usr/local/sbin/shfs /mnt/user -o noatime,big_writes,allow_other,default_permissions syslog.zip
April 30, 201016 yr Error 32 is usually a broken pipe. Perhaps emhttp is not listening for the output of the mount commands and they fail because there is nowhere to write their output. Edit: Looking closer... the mount commands have their output sent to /dev/null... Might be useful to capture that.
April 30, 201016 yr Author Yep broken pipe. #define EPIPE 32 /* Broken pipe */ Hmm I see >/dev/null 2>&1 so that should work. Perhaps stdin is an issue? Should have seen this earlier. I thought it was just my system issue. Sorry Tom.
April 30, 201016 yr Yep broken pipe. #define EPIPE 32 /* Broken pipe */ Hmm I see >/dev/null 2>&1 so that should work. Perhaps stdin is an issue? Should have seen this earlier. I thought it was just my system issue. Sorry Tom. Newest syslog just posted showed both the new drive just cleared AND the prior drive failing to mount with the same error 32. Then a "Stop" of the array followed by a "Start" shows only the newly cleared drive failing with error 32 and the data drive mounting properly. After clear of drive: Apr 30 05:10:52 Tower emhttp: ... clearing 97% complete Apr 30 05:16:20 Tower emhttp: ... clearing 98% complete Apr 30 05:21:51 Tower emhttp: ... clearing 99% complete Apr 30 05:27:30 Tower emhttp: ... clearing 100% complete Apr 30 05:27:30 Tower emhttp: ... syncing Apr 30 05:27:32 Tower kernel: mdcmd (20): start PROTECTED_EXPANSION Apr 30 05:27:32 Tower kernel: unraid: allocating 18220K for 1280 stripes (3 disks) Apr 30 05:27:32 Tower kernel: md1: running, size: 1953514552 blocks Apr 30 05:27:32 Tower kernel: md2: running, size: 1953514552 blocks Apr 30 05:27:32 Tower kernel: mdcmd (23): spindown 2 Apr 30 05:27:32 Tower kernel: mdcmd (24): spindown 2 Apr 30 05:27:33 Tower emhttp: shcmd (33): mkdir /mnt/disk2 Apr 30 05:27:33 Tower emhttp: shcmd (33): mkdir /mnt/disk1 Apr 30 05:27:33 Tower emhttp: shcmd (34): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 >/dev/null 2>&1 Apr 30 05:27:33 Tower emhttp: shcmd (35): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 >/dev/null 2>&1 Apr 30 05:27:33 Tower emhttp: _shcmd: shcmd (35): exit status: 32 Apr 30 05:27:33 Tower emhttp: disk1 mount error: 32 Apr 30 05:27:33 Tower emhttp: shcmd (36): rmdir /mnt/disk1 Apr 30 05:27:33 Tower emhttp: _shcmd: shcmd (36): exit status: 32 Apr 30 05:27:33 Tower emhttp: disk2 mount error: 32 Apr 30 05:27:33 Tower emhttp: shcmd (37): rmdir /mnt/disk2 Apr 30 05:27:33 Tower kernel: mdcmd (25): check Apr 30 05:27:33 Tower kernel: md: recovery thread woken up ... after subsequent "Stop" and "Start" only the cleared drive exits with error 32. Apr 30 08:35:51 Tower kernel: mdcmd (1194): start STOPPED Apr 30 08:35:51 Tower kernel: unraid: allocating 18220K for 1280 stripes (3 disks) Apr 30 08:35:51 Tower kernel: md1: running, size: 1953514552 blocks Apr 30 08:35:51 Tower kernel: md2: running, size: 1953514552 blocks Apr 30 08:35:51 Tower emhttp: shcmd (53): mkdir /mnt/disk2 Apr 30 08:35:51 Tower emhttp: shcmd (53): mkdir /mnt/disk1 Apr 30 08:35:51 Tower emhttp: shcmd (54): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 >/dev/null 2>&1 Apr 30 08:35:51 Tower emhttp: shcmd (55): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 >/dev/null 2>&1 Apr 30 08:35:51 Tower kernel: mdcmd (1196): check Apr 30 08:35:51 Tower kernel: md: recovery thread woken up ... Apr 30 08:35:51 Tower kernel: md: recovery thread has nothing to resync Apr 30 08:35:51 Tower emhttp: _shcmd: shcmd (55): exit status: 32 Apr 30 08:35:51 Tower emhttp: disk2 mount error: 32 Apr 30 08:35:51 Tower emhttp: shcmd (56): rmdir /mnt/disk2 Apr 30 08:35:51 Tower kernel: REISERFS (device md1): found reiserfs format "3.6" with standard journal Apr 30 08:35:51 Tower kernel: REISERFS (device md1): using ordered data mode Apr 30 08:35:51 Tower kernel: REISERFS warning (device md2): sh-2021 reiserfs_fill_super: can not find reiserfs on md2 Apr 30 08:35:51 Tower kernel: REISERFS (device md1): journal params: device md1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Apr 30 08:35:51 Tower kernel: REISERFS (device md1): checking transaction log (md1) Apr 30 08:35:51 Tower kernel: REISERFS (device md1): Using r5 hash to sort names
April 30, 201016 yr Author Hey Joe, can you do the same mount command (without redirection) on a shell session. capture the output and post it (for the pre-cleared drive). I'm just curious.
April 30, 201016 yr Hey Joe, can you do the same mount command (without redirection) on a shell session. capture the output and post it (for the pre-cleared drive). I'm just curious. Sure: root@Tower:/boot# preclear_disk -t /dev/sdb Pre-Clear unRAID Disk ######################################################################## Device Model: ST31500341AS Serial Number: 9VS0G9D8 Firmware Version: CC1J User Capacity: 1,500,301,910,016 bytes Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders, total 2930277168 sectors Units = sectors of 1 * 512 = 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 63 2930277167 1465138552+ 0 Empty Partition 1 does not end on cylinder boundary. ######################################################################## ============================================================================ == == DISK /dev/sdb IS PRECLEARED == ============================================================================ root@Tower:/boot# mkdir /tmp/sdb root@Tower:/boot# mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/sdb1 /tmp/sdb mount: wrong fs type, bad option, bad superblock on /dev/sdb1, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so root@Tower:/boot# echo $? 32 Looks like exit status of 32 is not a broken pipe.... but "some other error"
April 30, 201016 yr The '32' is the exit status of the 'mount' command, which is a bitmap. '32' simply means 'mount error'. (see 'man mount'). I am about to post version '4.5.3-TEST' which will cause output from the mount command to get into syslog. Also will disable the format command (button is still there, it just doesn't actually issue the 'mkreiserfs' command).
April 30, 201016 yr If someone can make this happen 'at will' then yes, I would do that, but so far no one has contacted me via email that they can. So to make it easier for more people to try, what I want to do is publish a "4.5.3-TEST" version so that eventually someone will see this case again.
April 30, 201016 yr I will be putting whatever version is released on my test server and trying to break it. I got another drive the other day that should allow me to add/remove and generally mess with stuff to my hearts content. Just let me know if you want me to do anything in a specific order.
May 11, 201016 yr Author It happened again to me this morning. I was doing a copy last night from a new ripping machine. The copy kept stalling. I let it run over night. This morning the machine is locked up. I look on the screen and it has OOM errors with processes being killed. I power cycle, I come up. All disks show up as unformatted. There is a parity sync in process. May 11 05:00:04 Atlas kernel: mdcmd (22): start STOPPED May 11 05:00:04 Atlas kernel: unraid: allocating 95308K for 2784 stripes (8 disks) May 11 05:00:04 Atlas kernel: md1: running, size: 976762552 blocks May 11 05:00:04 Atlas kernel: md2: running, size: 1465138552 blocks May 11 05:00:04 Atlas kernel: md3: running, size: 976762552 blocks May 11 05:00:04 Atlas kernel: md4: running, size: 976762552 blocks May 11 05:00:04 Atlas kernel: md5: running, size: 976762552 blocks May 11 05:00:04 Atlas kernel: md6: running, size: 976762552 blocks May 11 05:00:04 Atlas kernel: md7: running, size: 1465138552 blocks May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk1 May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk7 May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk6 May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk4 May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk5 May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk3 May 11 05:00:04 Atlas emhttp: shcmd (11): mkdir /mnt/disk2 May 11 05:00:04 Atlas kernel: mdcmd (24): check May 11 05:00:04 Atlas kernel: md: recovery thread woken up ... May 11 05:00:04 Atlas kernel: md: recovery thread checking parity... May 11 05:00:04 Atlas emhttp: shcmd (12): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: shcmd (13): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md7 /mnt/disk7 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: shcmd (14): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md5 /mnt/disk5 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: shcmd (15): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md6 /mnt/disk6 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: shcmd (16): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md4 /mnt/disk4 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (16): exit status: 32 May 11 05:00:04 Atlas emhttp: disk7 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (17): rmdir /mnt/disk7 May 11 05:00:04 Atlas emhttp: shcmd (18): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: shcmd (19): mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md3 /mnt/disk3 >/dev/null 2>&1 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (19): exit status: 32 May 11 05:00:04 Atlas emhttp: disk1 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (20): rmdir /mnt/disk1 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (20): exit status: 32 May 11 05:00:04 Atlas emhttp: disk5 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (21): rmdir /mnt/disk5 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (21): exit status: 32 May 11 05:00:04 Atlas emhttp: disk6 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (22): rmdir /mnt/disk6 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (22): exit status: 32 May 11 05:00:04 Atlas emhttp: disk4 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (23): rmdir /mnt/disk4 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (26): exit status: 32 May 11 05:00:04 Atlas emhttp: disk3 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (27): rmdir /mnt/disk3 May 11 05:00:04 Atlas emhttp: _shcmd: shcmd (28): exit status: 32 May 11 05:00:04 Atlas emhttp: disk2 mount error: 32 May 11 05:00:04 Atlas emhttp: shcmd (29): rmdir /mnt/disk2 May 11 05:00:05 Atlas emhttp: shcmd (32): rm /etc/samba/smb-shares.conf >/dev/null 2>&1 May 11 05:00:05 Atlas emhttp: _shcmd: shcmd (32): exit status: 1 May 11 05:00:05 Atlas emhttp: shcmd (33): cp /etc/exports- /etc/exports May 11 05:00:05 Atlas emhttp: shcmd (34): killall -HUP smbd May 11 05:00:05 Atlas emhttp: shcmd (35): /etc/rc.d/rc.nfsd restart | logger May 11 05:00:05 Atlas kernel: md: using 2304k window, over a total of 1465138552 blocks. Manually I did a mount to see if there was an error. No errors on screen, but it took a long time and logged to the syslog. May 11 05:07:32 Atlas kernel: REISERFS (device md1): found reiserfs format "3.6" with standard journal May 11 05:07:32 Atlas kernel: REISERFS (device md1): using ordered data mode May 11 05:07:33 Atlas kernel: REISERFS (device md1): journal params: device md1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 May 11 05:07:33 Atlas kernel: REISERFS (device md1): checking transaction log (md1) May 11 05:13:57 Atlas kernel: REISERFS (device md1): replayed 682 transactions in 384 seconds May 11 05:13:57 Atlas kernel: REISERFS (device md1): Using r5 hash to sort names I did not add any disks. Just rebooted.
May 11, 201016 yr Author OK, I installed the -TEST version on my larger server. I rebooted, it would not come up. Something about cannot start console. I tried 3x then reverted. When I rebooted back to 4.5.3 every disk came up as unformatted again. I decided to install my own mount command with logging this time. I rebooted again. They did not come up as unformatted now. This is getting to be a pain. All I did was reboot and ALL the disks said unformatted. I did not do anything. Now that my mount wrapper is in place on my production system, maybe I'll catch it.
May 11, 201016 yr Is this happening on your Abit AB9 Pro based system. This is an aged board now and not everything can be blamed on software. If you think they are almost 9000 hours/year and if you run this server 24/7 you probably may encounter some hardware problems due to the aging components.(especially capacitors)
May 11, 201016 yr Author Is this happening on your Abit AB9 Pro based system. This is an aged board now and not everything can be blamed on software. If you think they are almost 9000 hours/year and if you run this server 24/7 you probably may encounter some hardware problems due to the aging components.(especially capacitors) Hardware is one thing... Having the management interface saying "Unformatted" is another. When I stop and start the management interface the drives mount fine.
May 12, 201016 yr Author Hmmm, The unformatted bug happened a number of times on my 2.6Ghz Core Duo 2 machine without changing the drives. All I did was reboot. Yet nothing I did on purpose caused it to happen on my 1ghz celeron M machine. seems like the race condition as mentioned is probably the issue.
Archived
This topic is now archived and is closed to further replies.