November 29, 201114 yr Normally I have 7 drives. 6 of them are WD Caviar Green and my cache is an old 300GB Maxtor of which I am not sure the speed. I have enough space to reduce the number of drives if needed, however I don't think I can until I get through the rebuild. I have a spare PSU sitting around, a LC-A350ATX and some standard -> sata power converters, but I don't have enough to do all the drives. I'm not sure exactly what you mean by the double vs. single 12 volt rail. Can you elaborate a little so I know what I need to replace my existing one with? Thank you!! You should be able to run 7 green drives off that supply. Have a look here for Power Supply info.
November 29, 201114 yr Author That PSU has no sata connectors and I have only enough adapters to run 5 drives. I will have to pick up a few more or a new PSU. I'll get right on it. Could the second rail be causing this? It's strange because it ran fine for 6+ months, through rebuilds, etc. Thanks!
November 29, 201114 yr Author I found a 6th adapter and hooked up the old PSU. The MB power slot is longer than the one on the PSU, but it seems to work ok. The rebuild is in progress (fingers crossed) and it is going faster than ever before (50+ MB/sec vs. a usual 10-15MB/sec). We shall see!
November 29, 201114 yr Author I was up until it hit 20% on the rebuild. When I woke up this morning the server was locked. Could a different PSU make that big of a difference in transfer speeds? Any other ideas on what could be wrong? The negative free space thing is still throwing me off. Thank you!!
November 29, 201114 yr I was up until it hit 20% on the rebuild. When I woke up this morning the server was locked. Could a different PSU make that big of a difference in transfer speeds? Any other ideas on what could be wrong? The negative free space thing is still throwing me off. Thank you!! You need to be running the "tail -f -n 100 /var/log/syslog" command while you are doing these rebuild attempts. That might help point to something.
November 29, 201114 yr Author Thank you again for the help. I will run it again when I get home this afternoon. On a side note, is there a MB on here that would work better than the one I have? What about CPU, etc? I am happy to buy components that help the system run better or faster. Newegg Tuesday Sale
November 30, 201114 yr Author I ran again with the syslog output. It locked within 1 hour of starting. Here is the result of the command: root@Tower:~# tail -f -n 100 /var/log/syslog Nov 29 20:15:53 Tower kernel: xor: automatically using best checksumming function: pIII_sse Nov 29 20:15:53 Tower emhttp: Spinning up all drives... Nov 29 20:15:53 Tower kernel: pIII_sse : 6684.000 MB/sec Nov 29 20:15:53 Tower kernel: xor: using function: pIII_sse (6684.000 MB/sec) Nov 29 20:15:53 Tower kernel: md: unRAID driver 1.1.1 installed Nov 29 20:15:53 Tower kernel: md: import disk0: [8,0] (sda) WDC WD20EARS-00M WD-WCAZA6004572 size: 1953514552 Nov 29 20:15:53 Tower kernel: md: import disk1: [8,96] (sdg) WDC WD20EADS-00W WD-WCAVY7301951 size: 1953514552 Nov 29 20:15:53 Tower kernel: md: import disk2: [8,16] (sdb) WDC WD20EARS-00J WD-WCAYY0019666 size: 1953514552 Nov 29 20:15:53 Tower kernel: md: import disk3: [8,80] (sdf) WDC WD20EARS-07M WD-WCAZA7893938 size: 1953514552 Nov 29 20:15:53 Tower kernel: md: import disk4: [8,64] (sde) WDC WD10EACS-00D WD-WCAU44784010 size: 976762552 Nov 29 20:15:53 Tower kernel: md: import disk5: [8,32] (sdc) WDC WD10EARS-00Y WD-WCAV56034780 size: 976762552 Nov 29 20:15:53 Tower kernel: mdcmd (1): set md_num_stripes 1280 Nov 29 20:15:53 Tower kernel: mdcmd (2): set md_write_limit 768 Nov 29 20:15:53 Tower kernel: mdcmd (3): set md_sync_window 288 Nov 29 20:15:53 Tower kernel: mdcmd (4): set spinup_group 0 2 Nov 29 20:15:53 Tower kernel: mdcmd (5): set spinup_group 1 1 Nov 29 20:15:53 Tower kernel: mdcmd (6): set spinup_group 2 0 Nov 29 20:15:53 Tower kernel: mdcmd (7): set spinup_group 3 0 Nov 29 20:15:53 Tower kernel: mdcmd (: set spinup_group 4 32 Nov 29 20:15:53 Tower kernel: mdcmd (9): set spinup_group 5 16 Nov 29 20:15:53 Tower kernel: mdcmd (10): spinup 0 Nov 29 20:15:53 Tower kernel: mdcmd (11): spinup 1 Nov 29 20:15:53 Tower kernel: mdcmd (12): spinup 2 Nov 29 20:15:53 Tower kernel: mdcmd (13): spinup 3 Nov 29 20:15:53 Tower kernel: mdcmd (14): spinup 4 Nov 29 20:15:53 Tower kernel: mdcmd (15): spinup 5 Nov 29 20:15:53 Tower emhttp: shcmd (4): /usr/local/sbin/set_ncq sda 1 >/dev/null Nov 29 20:15:53 Tower emhttp: shcmd (5): /usr/local/sbin/set_ncq sdg 1 >/dev/null Nov 29 20:15:53 Tower emhttp: shcmd (6): /usr/local/sbin/set_ncq sdb 1 >/dev/null Nov 29 20:15:53 Tower emhttp: shcmd (7): /usr/local/sbin/set_ncq sdf 1 >/dev/null Nov 29 20:15:53 Tower emhttp: shcmd (: /usr/local/sbin/set_ncq sde 1 >/dev/null Nov 29 20:15:53 Tower emhttp: shcmd (9): /usr/local/sbin/set_ncq sdc 1 >/dev/null Nov 29 20:15:53 Tower kernel: mdcmd (16): start STOPPED Nov 29 20:15:53 Tower kernel: unraid: allocating 33720K for 1280 stripes (6 disks) Nov 29 20:15:53 Tower kernel: md1: running, size: 1953514552 blocks Nov 29 20:15:53 Tower kernel: md2: running, size: 1953514552 blocks Nov 29 20:15:53 Tower kernel: md3: running, size: 1953514552 blocks Nov 29 20:15:53 Tower kernel: md4: running, size: 976762552 blocks Nov 29 20:15:53 Tower kernel: md5: running, size: 976762552 blocks Nov 29 20:15:53 Tower emhttp: shcmd (10): udevadm settle Nov 29 20:15:53 Tower emhttp: shcmd (11): mkdir /mnt/disk1 Nov 29 20:15:53 Tower emhttp: shcmd (11): mkdir /mnt/disk4 Nov 29 20:15:53 Tower emhttp: shcmd (11): mkdir /mnt/disk3 Nov 29 20:15:53 Tower emhttp: shcmd (11): mkdir /mnt/disk5 Nov 29 20:15:53 Tower emhttp: shcmd (11): mkdir /mnt/disk2 Nov 29 20:15:53 Tower emhttp: shcmd (12): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md5 /mnt/disk5 2>&1 | logger Nov 29 20:15:53 Tower emhttp: shcmd (13): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 2>&1 | logger Nov 29 20:15:53 Tower emhttp: shcmd (14): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md3 /mnt/disk3 2>&1 | logger Nov 29 20:15:53 Tower emhttp: shcmd (15): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 2>&1 | logger Nov 29 20:15:53 Tower kernel: mdcmd (17): check Nov 29 20:15:53 Tower kernel: md: recovery thread woken up ... Nov 29 20:15:53 Tower kernel: md: recovery thread rebuilding disk2 ... Nov 29 20:15:53 Tower emhttp: shcmd (16): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md4 /mnt/disk4 2>&1 | logger Nov 29 20:15:53 Tower kernel: md: using 1152k window, over a total of 1953514552 blocks. Nov 29 20:15:53 Tower kernel: REISERFS (device md5): found reiserfs format "3.6" with standard journal Nov 29 20:15:53 Tower kernel: REISERFS (device md5): using ordered data mode Nov 29 20:15:53 Tower kernel: REISERFS (device md3): found reiserfs format "3.6" with standard journal Nov 29 20:15:53 Tower kernel: REISERFS (device md3): using ordered data mode Nov 29 20:15:53 Tower kernel: REISERFS (device md1): found reiserfs format "3.6" with standard journal Nov 29 20:15:53 Tower kernel: REISERFS (device md1): using ordered data mode Nov 29 20:15:53 Tower kernel: REISERFS (device md2): found reiserfs format "3.6" with standard journal Nov 29 20:15:53 Tower kernel: REISERFS (device md2): using ordered data mode Nov 29 20:15:53 Tower kernel: REISERFS (device md4): found reiserfs format "3.6" with standard journal Nov 29 20:15:53 Tower kernel: REISERFS (device md4): using ordered data mode Nov 29 20:15:53 Tower kernel: REISERFS (device md3): journal params: device md3, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 29 20:15:53 Tower kernel: REISERFS (device md3): checking transaction log (md3) Nov 29 20:15:53 Tower kernel: REISERFS (device md5): journal params: device md5, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 29 20:15:53 Tower kernel: REISERFS (device md5): checking transaction log (md5) Nov 29 20:15:53 Tower kernel: REISERFS (device md1): journal params: device md1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 29 20:15:53 Tower kernel: REISERFS (device md1): checking transaction log (md1) Nov 29 20:15:53 Tower kernel: REISERFS (device md4): journal params: device md4, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 29 20:15:53 Tower kernel: REISERFS (device md4): checking transaction log (md4) Nov 29 20:15:53 Tower kernel: REISERFS (device md2): journal params: device md2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 29 20:15:53 Tower kernel: REISERFS (device md2): checking transaction log (md2) Nov 29 20:15:54 Tower kernel: REISERFS (device md4): replayed 2 transactions in 1 seconds Nov 29 20:15:54 Tower kernel: REISERFS (device md1): replayed 2 transactions in 1 seconds Nov 29 20:15:54 Tower kernel: REISERFS (device md5): replayed 2 transactions in 1 seconds Nov 29 20:15:54 Tower kernel: REISERFS (device md3): replayed 2 transactions in 1 seconds Nov 29 20:15:54 Tower kernel: REISERFS (device md1): Using r5 hash to sort names Nov 29 20:15:54 Tower kernel: REISERFS (device md4): Using r5 hash to sort names Nov 29 20:15:54 Tower kernel: REISERFS (device md5): Using r5 hash to sort names Nov 29 20:15:54 Tower kernel: REISERFS (device md3): Using r5 hash to sort names Nov 29 20:15:54 Tower kernel: REISERFS (device md2): replayed 2 transactions in 1 seconds Nov 29 20:15:54 Tower emhttp: shcmd (20): cp /var/spool/cron/crontabs/root- /var/spool/cron/crontabs/root Nov 29 20:15:54 Tower emhttp: shcmd (21): echo '# Generated mover schedule:' >>/var/spool/cron/crontabs/root Nov 29 20:15:54 Tower emhttp: shcmd (22): echo '40 3 * * * /usr/local/sbin/mover 2>&1 | logger' >>/var/spool/cron/crontabs/root Nov 29 20:15:54 Tower emhttp: shcmd (23): crontab /var/spool/cron/crontabs/root Nov 29 20:15:54 Tower kernel: REISERFS (device md2): Using r5 hash to sort names Nov 29 20:15:56 Tower emhttp: shcmd (25): rm /etc/samba/smb-shares.conf >/dev/null 2>&1 Nov 29 20:15:56 Tower emhttp: _shcmd: shcmd (25): exit status: 1 Nov 29 20:15:56 Tower emhttp: shcmd (26): cp /etc/exports- /etc/exports Nov 29 20:15:56 Tower emhttp: shcmd (27): mkdir /mnt/user Nov 29 20:15:56 Tower emhttp: shcmd (28): /usr/local/sbin/shfs /mnt/user -o noatime,big_writes,allow_other,default_permissions Nov 29 20:15:57 Tower emhttp: get_config_idx: fopen /boot/config/shares/incomplete.cfg: No such file or directory - assigning defaults Nov 29 20:15:57 Tower emhttp: shcmd (29): killall -HUP smbd Nov 29 20:15:57 Tower emhttp: shcmd (30): /etc/rc.d/rc.nfsd restart | logger Nov 29 20:17:13 Tower unmenu-status: Starting unmenu web-server Nov 29 20:17:13 Tower init: Re-reading inittab Nov 29 20:18:02 Tower in.telnetd[1857]: connect from 192.168.1.101 (192.168.1.101) Nov 29 20:18:06 Tower login[1858]: ROOT LOGIN on `pts/0' from `192.168.1.101' Nov 29 20:18:54 Tower in.telnetd[1871]: connect from 192.168.1.101 (192.168.1.101) Nov 29 20:19:00 Tower login[1872]: ROOT LOGIN on `pts/1' from `192.168.1.101' I also left open top in another session and this was the last screen before it died: top - 21:42:49 up 1:27, 2 users, load average: 1.00, 1.01, 0.95 Tasks: 83 total, 1 running, 82 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1033468k total, 209128k used, 824340k free, 560k buffers Swap: 0k total, 0k used, 0k free, 147968k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 20 0 700 304 264 S 0 0.0 0:02.34 init 2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd 3 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0 4 root 20 0 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0 5 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1 6 root 20 0 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1 7 root 20 0 0 0 0 S 0 0.0 0:00.00 events/0 8 root 20 0 0 0 0 S 0 0.0 0:00.32 events/1 9 root 20 0 0 0 0 S 0 0.0 0:00.01 khelper 14 root 20 0 0 0 0 S 0 0.0 0:00.00 async/mgr 116 root 20 0 0 0 0 S 0 0.0 0:00.00 sync_supers 118 root 20 0 0 0 0 S 0 0.0 0:00.00 bdi-default 120 root 20 0 0 0 0 S 0 0.0 0:00.00 kblockd/0 121 root 20 0 0 0 0 S 0 0.0 0:00.00 kblockd/1 122 root 20 0 0 0 0 S 0 0.0 0:00.00 kacpid 123 root 20 0 0 0 0 S 0 0.0 0:00.00 kacpi_notify 124 root 20 0 0 0 0 S 0 0.0 0:00.00 kacpi_hotplug 276 root 20 0 0 0 0 S 0 0.0 0:00.00 ata/0 277 root 20 0 0 0 0 S 0 0.0 0:00.00 ata/1 278 root 20 0 0 0 0 S 0 0.0 0:00.00 ata_aux 282 root 20 0 0 0 0 S 0 0.0 0:00.00 ksuspend_usbd 287 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd 290 root 20 0 0 0 0 S 0 0.0 0:00.00 kseriod 323 root 20 0 0 0 0 S 0 0.0 0:00.00 rpciod/0 324 root 20 0 0 0 0 S 0 0.0 0:00.00 rpciod/1 354 root 20 0 0 0 0 S 0 0.0 0:00.00 kswapd0 399 root 20 0 0 0 0 S 0 0.0 0:00.00 aio/0 400 root 20 0 0 0 0 S 0 0.0 0:00.00 aio/1 406 root 20 0 0 0 0 S 0 0.0 0:00.00 nfsiod 411 root 15 -5 0 0 0 S 0 0.0 0:00.00 kslowd000 412 root 15 -5 0 0 0 S 0 0.0 0:00.00 kslowd001 628 root 20 0 0 0 0 S 0 0.0 0:00.00 usbhid_resumer 668 root 16 -4 1828 760 480 S 0 0.1 0:00.09 udevd 865 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0 866 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_1 867 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_2 868 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_3 873 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_4 874 root 20 0 0 0 0 S 0 0.0 0:00.00 usb-storage 886 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_5 908 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_6 909 root 20 0 0 0 0 S 0 0.0 0:00.01 scsi_eh_7 911 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_8 912 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_9 1386 root 20 0 1688 596 504 S 0 0.1 0:00.02 syslogd 1390 root 20 0 1636 408 340 S 0 0.0 0:00.01 klogd 1429 root 20 0 1660 468 392 S 0 0.0 0:00.00 ifplugd 1471 bin 20 0 1752 476 392 S 0 0.0 0:00.00 rpc.portmap 1475 nobody 20 0 1800 696 608 S 0 0.1 0:00.00 rpc.statd 1485 root 20 0 1668 532 464 S 0 0.1 0:00.00 inetd 1495 root 20 0 1636 540 468 S 0 0.1 0:00.00 acpid 1502 root 20 0 1816 660 548 S 0 0.1 0:00.00 crond 1504 daemon 20 0 1668 380 304 S 0 0.0 0:00.00 atd 1509 root 20 0 8484 1868 1260 S 0 0.2 0:00.06 nmbd 1511 root 20 0 13596 3304 2548 S 0 0.3 0:00.02 smbd 1516 root 20 0 53780 1548 1160 S 0 0.1 0:00.08 emhttp 1520 root 20 0 13596 1692 936 S 0 0.2 0:00.00 smbd Thank you for the help.
December 1, 201114 yr Author Stopped the powerdown addon and unmenu. Those are the only add-ons I had running. My go script is now completely commented except for the web gui. Still locked. Here is the syslog. Looks the same. Linux 2.6.32.9-unRAID. root@Tower:~# tail -f -n 100 /var/log/syslog Nov 30 18:43:16 Tower emhttp: shcmd (2): modprobe -rw md-mod 2>&1 | logger Nov 30 18:43:16 Tower emhttp: shcmd (3): modprobe md-mod super=/boot/config/super.dat slots=8,0,8,96,8,16,8,80,8,64,8,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 2>&1 | logger Nov 30 18:43:16 Tower kernel: xor: automatically using best checksumming function: pIII_sse Nov 30 18:43:16 Tower kernel: pIII_sse : 6671.600 MB/sec Nov 30 18:43:16 Tower kernel: xor: using function: pIII_sse (6671.600 MB/sec) Nov 30 18:43:16 Tower kernel: md: unRAID driver 1.1.1 installed Nov 30 18:43:16 Tower kernel: md: import disk0: [8,0] (sda) WDC WD20EARS-00M WD-WCAZA6004572 size: 1953514552 Nov 30 18:43:16 Tower kernel: md: import disk1: [8,96] (sdg) WDC WD20EADS-00W WD-WCAVY7301951 size: 1953514552 Nov 30 18:43:16 Tower kernel: md: import disk2: [8,16] (sdb) WDC WD20EARS-00J WD-WCAYY0019666 size: 1953514552 Nov 30 18:43:16 Tower kernel: md: import disk3: [8,80] (sdf) WDC WD20EARS-07M WD-WCAZA7893938 size: 1953514552 Nov 30 18:43:16 Tower kernel: md: import disk4: [8,64] (sde) WDC WD10EACS-00D WD-WCAU44784010 size: 976762552 Nov 30 18:43:16 Tower kernel: md: import disk5: [8,32] (sdc) WDC WD10EARS-00Y WD-WCAV56034780 size: 976762552 Nov 30 18:43:16 Tower kernel: mdcmd (1): set md_num_stripes 1280 Nov 30 18:43:16 Tower kernel: mdcmd (2): set md_write_limit 768 Nov 30 18:43:16 Tower kernel: mdcmd (3): set md_sync_window 288 Nov 30 18:43:16 Tower kernel: mdcmd (4): set spinup_group 0 2 Nov 30 18:43:16 Tower kernel: mdcmd (5): set spinup_group 1 1 Nov 30 18:43:16 Tower kernel: mdcmd (6): set spinup_group 2 0 Nov 30 18:43:16 Tower kernel: mdcmd (7): set spinup_group 3 0 Nov 30 18:43:16 Tower kernel: mdcmd (: set spinup_group 4 32 Nov 30 18:43:16 Tower kernel: mdcmd (9): set spinup_group 5 16 Nov 30 18:43:16 Tower emhttp: Spinning up all drives... Nov 30 18:43:16 Tower kernel: mdcmd (10): spinup 0 Nov 30 18:43:16 Tower kernel: mdcmd (11): spinup 1 Nov 30 18:43:16 Tower kernel: mdcmd (12): spinup 2 Nov 30 18:43:16 Tower kernel: mdcmd (13): spinup 3 Nov 30 18:43:16 Tower kernel: mdcmd (14): spinup 4 Nov 30 18:43:16 Tower kernel: mdcmd (15): spinup 5 Nov 30 18:43:16 Tower emhttp: shcmd (4): /usr/local/sbin/set_ncq sda 1 >/dev/null Nov 30 18:43:16 Tower emhttp: shcmd (5): /usr/local/sbin/set_ncq sdg 1 >/dev/null Nov 30 18:43:16 Tower emhttp: shcmd (6): /usr/local/sbin/set_ncq sdb 1 >/dev/null Nov 30 18:43:16 Tower emhttp: shcmd (7): /usr/local/sbin/set_ncq sdf 1 >/dev/null Nov 30 18:43:16 Tower emhttp: shcmd (: /usr/local/sbin/set_ncq sde 1 >/dev/null Nov 30 18:43:16 Tower emhttp: shcmd (9): /usr/local/sbin/set_ncq sdc 1 >/dev/null Nov 30 18:43:16 Tower kernel: mdcmd (16): start STOPPED Nov 30 18:43:16 Tower kernel: unraid: allocating 33720K for 1280 stripes (6 disks) Nov 30 18:43:16 Tower kernel: md1: running, size: 1953514552 blocks Nov 30 18:43:16 Tower kernel: md2: running, size: 1953514552 blocks Nov 30 18:43:16 Tower kernel: md3: running, size: 1953514552 blocks Nov 30 18:43:16 Tower kernel: md4: running, size: 976762552 blocks Nov 30 18:43:16 Tower kernel: md5: running, size: 976762552 blocks Nov 30 18:43:16 Tower emhttp: shcmd (10): udevadm settle Nov 30 18:43:16 Tower emhttp: shcmd (11): mkdir /mnt/disk1 Nov 30 18:43:16 Tower emhttp: shcmd (11): mkdir /mnt/disk5 Nov 30 18:43:16 Tower emhttp: shcmd (11): mkdir /mnt/disk4 Nov 30 18:43:16 Tower emhttp: shcmd (11): mkdir /mnt/disk3 Nov 30 18:43:16 Tower emhttp: shcmd (11): mkdir /mnt/disk2 Nov 30 18:43:16 Tower emhttp: shcmd (12): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md5 /mnt/disk5 2>&1 | logger Nov 30 18:43:16 Tower emhttp: shcmd (13): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md1 /mnt/disk1 2>&1 | logger Nov 30 18:43:16 Tower emhttp: shcmd (14): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md3 /mnt/disk3 2>&1 | logger Nov 30 18:43:16 Tower emhttp: shcmd (15): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md2 /mnt/disk2 2>&1 | logger Nov 30 18:43:16 Tower kernel: mdcmd (17): check Nov 30 18:43:16 Tower kernel: md: recovery thread woken up ... Nov 30 18:43:16 Tower kernel: md: recovery thread rebuilding disk2 ... Nov 30 18:43:16 Tower emhttp: shcmd (16): set -o pipefail ; mount -t reiserfs -o noacl,nouser_xattr,noatime,nodiratime /dev/md4 /mnt/disk4 2>&1 | logger Nov 30 18:43:16 Tower kernel: md: using 1152k window, over a total of 1953514552 blocks. Nov 30 18:43:16 Tower kernel: REISERFS (device md5): found reiserfs format "3.6" with standard journal Nov 30 18:43:16 Tower kernel: REISERFS (device md5): using ordered data mode Nov 30 18:43:16 Tower kernel: REISERFS (device md3): found reiserfs format "3.6" with standard journal Nov 30 18:43:16 Tower kernel: REISERFS (device md3): using ordered data mode Nov 30 18:43:16 Tower kernel: REISERFS (device md1): found reiserfs format "3.6" with standard journal Nov 30 18:43:16 Tower kernel: REISERFS (device md1): using ordered data mode Nov 30 18:43:16 Tower kernel: REISERFS (device md2): found reiserfs format "3.6" with standard journal Nov 30 18:43:16 Tower kernel: REISERFS (device md2): using ordered data mode Nov 30 18:43:16 Tower kernel: REISERFS (device md4): found reiserfs format "3.6" with standard journal Nov 30 18:43:16 Tower kernel: REISERFS (device md4): using ordered data mode Nov 30 18:43:16 Tower kernel: REISERFS (device md5): journal params: device md5, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 30 18:43:16 Tower kernel: REISERFS (device md5): checking transaction log (md5) Nov 30 18:43:16 Tower kernel: REISERFS (device md3): journal params: device md3, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 30 18:43:16 Tower kernel: REISERFS (device md3): checking transaction log (md3) Nov 30 18:43:16 Tower kernel: REISERFS (device md1): journal params: device md1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 30 18:43:16 Tower kernel: REISERFS (device md1): checking transaction log (md1) Nov 30 18:43:16 Tower kernel: REISERFS (device md4): journal params: device md4, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 30 18:43:16 Tower kernel: REISERFS (device md4): checking transaction log (md4) Nov 30 18:43:16 Tower kernel: REISERFS (device md2): journal params: device md2, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30 Nov 30 18:43:16 Tower kernel: REISERFS (device md2): checking transaction log (md2) Nov 30 18:43:17 Tower kernel: REISERFS (device md5): replayed 2 transactions in 1 seconds Nov 30 18:43:17 Tower kernel: REISERFS (device md4): replayed 2 transactions in 1 seconds Nov 30 18:43:17 Tower kernel: REISERFS (device md1): replayed 2 transactions in 1 seconds Nov 30 18:43:17 Tower kernel: REISERFS (device md3): replayed 2 transactions in 1 seconds Nov 30 18:43:17 Tower kernel: REISERFS (device md5): Using r5 hash to sort names Nov 30 18:43:17 Tower kernel: REISERFS (device md4): Using r5 hash to sort names Nov 30 18:43:17 Tower kernel: REISERFS (device md1): Using r5 hash to sort names Nov 30 18:43:17 Tower kernel: REISERFS (device md3): Using r5 hash to sort names Nov 30 18:43:17 Tower kernel: REISERFS (device md2): replayed 2 transactions in 1 seconds Nov 30 18:43:17 Tower emhttp: shcmd (20): cp /var/spool/cron/crontabs/root- /var/spool/cron/crontabs/root Nov 30 18:43:17 Tower emhttp: shcmd (21): echo '# Generated mover schedule:' >>/var/spool/cron/crontabs/root Nov 30 18:43:17 Tower emhttp: shcmd (22): echo '40 3 * * * /usr/local/sbin/mover 2>&1 | logger' >>/var/spool/cron/crontabs/root Nov 30 18:43:17 Tower emhttp: shcmd (23): crontab /var/spool/cron/crontabs/root Nov 30 18:43:17 Tower kernel: REISERFS (device md2): Using r5 hash to sort names Nov 30 18:43:19 Tower emhttp: shcmd (25): rm /etc/samba/smb-shares.conf >/dev/null 2>&1 Nov 30 18:43:19 Tower emhttp: _shcmd: shcmd (25): exit status: 1 Nov 30 18:43:19 Tower emhttp: shcmd (26): cp /etc/exports- /etc/exports Nov 30 18:43:19 Tower emhttp: shcmd (27): mkdir /mnt/user Nov 30 18:43:19 Tower emhttp: shcmd (28): /usr/local/sbin/shfs /mnt/user -o noatime,big_writes,allow_other,default_permissions Nov 30 18:43:20 Tower emhttp: get_config_idx: fopen /boot/config/shares/incomplete.cfg: No such file or directory - assigning defaults Nov 30 18:43:20 Tower emhttp: shcmd (29): killall -HUP smbd Nov 30 18:43:20 Tower emhttp: shcmd (30): /etc/rc.d/rc.nfsd restart | logger Nov 30 19:26:44 Tower in.telnetd[1668]: connect from 192.168.1.101 (192.168.1.101) Nov 30 19:26:48 Tower login[1669]: ROOT LOGIN on `pts/0' from `192.168.1.101' Thanks much for the help!!
December 1, 201114 yr Use a spare flash drive or backup and format the one you have and install a clean copy of unRAID and see if the system works with a clean system install.
December 2, 201114 yr Author I backed up and reformatted the original flash drive. It boots fine with the fresh install. I'm not sure what to do next. I can re-add my devices (but only 3 unless I somehow fix the license) or I can leave it running to just see if it locks. Thanks!
December 3, 201114 yr Author It ran since my last post without freezing, but no strain was placed on the server either. I have not added drives or attempted a rebuild. How do I add an existing unraid configuration to a fresh unraid install? Thanks!
December 4, 201114 yr Author That did it. I copied the backup config dir onto the new install and it completed the drive rebuild. I will put everything back together and use the old PSU going forward. One thing remains. disk2 still shows negative available disk space. How do I fix this? Thanks!!
December 6, 201114 yr Author Just tried to use mc to move stuff from disk 3 to disk 5 and as soon as I clicked OK on the copy the monitor hooked up to the server showed: Kernel panic - not syncing: <4>REISERFS panic: vs-6030 check_internal_block_head: invalid item number level=3, nr_items=174, free_space=65424 rdkey Of course this froze the server and it is no longer responding to any form of input, and to top it off the lights on the keyboard are flashing. I'm also still stuck on the negative space issue. Thanks for the help!
December 6, 201114 yr Just tried to use mc to move stuff from disk 3 to disk 5 and as soon as I clicked OK on the copy the monitor hooked up to the server showed: Kernel panic - not syncing: <4>REISERFS panic: vs-6030 check_internal_block_head: invalid item number level=3, nr_items=174, free_space=65424 rdkey Of course this froze the server and it is no longer responding to any form of input, and to top it off the lights on the keyboard are flashing. I'm also still stuck on the negative space issue. Thanks for the help! You have some sort of disk issue. Can you remove the offending disk and see about copying data to another in the array? (or a couple if need be). I think you need to try and do that, run new config, and then see about removing that drive completely.
December 6, 201114 yr A rebuild won't fix the file system. See here: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems
December 7, 201114 yr Author When I rebooted after that lock disk3 was 'missing'. I reseated the drive and rebooted and it showed that disk3 required a rebuild. I swapped it out for a spare and started the rebuild. I will test the former disk3 overnight. It is amazing that I had two disks fail in 1 week, then none for the 3 weeks my server was down, attempting rebuilds every day, and now another one fail after the first successful rebuild/boot. I was one boot away from losing data?!? I will run the reiser disk util as soon as the data rebuild completes. I'd rather run it on a protected array. Thanks all!!!
December 7, 201114 yr Author The server locked after getting to about 5% on the rebuild again. :-| I will try the Reiser fs fix thing now and then attempt the rebuild again.
December 7, 201114 yr Author This is the last lines of the reiser fs check on disk2. bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (721) to the block (446171863), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (722) to the block (446171864), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (723) to the block (446171865), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (724) to the block (446171866), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (725) to the block (446171867), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (726) to the block (446171868), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (727) to the block (446171869), which is in tree already bad_indirect_item: block 87046405: The item (6348 6351 0x1 IND (1), len 2916, location 192 entry count 0, fsck need 0, format new) has the bad pointer (728) to the block (446171870), which is in tree already /114 (of 170-bad_node: vpf-10350: The block (446169089) is used more than once in the tree. the problem in the internal node occured (446169089), whole subtree is skipped / 32 (of 160\/ 1 (of 170|bad_node: vpf-10350: The block (446169146) is used more than once in the tree. the problem in the internal node occured (446169146), whole subtree is skipped / 33 (of 160// 4 (of 170/bad_indirect_item: block 298287174: The item (6352 6354 0x10f0a001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) has the bad pointer (0) to the block (298289152), which is in tree already / 34 (of 160|/ 91 (of 170\bad_node: vpf-10350: The block (446169251) is used more than once in the tree. the problem in the internal node occured (446169251), whole subtree is skipped / 35 (of 160|/ 11 (of 170\bad_node: vpf-10350: The block (325582939) is used more than once in the tree. the problem in the internal node occured (325582939), whole subtree is skipped / 36 (of 160|/ 32 (of 170|bad_indirect_item: block 244358114: The item (6367 6370 0x1d495001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) has the bad pointer (263) to the block (244478509), which is in tree already / 64 (of 170|bad_indirect_item: block 244358146: The item (6367 6370 0x25315001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) has the bad pointer (280) to the block (244510911), which is in tree already / 57 (of 160//161 (of 162-block 273481205: The level of the node (32981) is not correct, (1) expected the problem in the internal node occured (273481205), whole subtree is skipped /133 (of 160|/ 77 (of 165/bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (0) to the block (273481206), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (1) to the block (273481207), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (2) to the block (273481208), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (3) to the block (273481209), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (4) to the block (273481210), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (5) to the block (273481211), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (6) to the block (273481212), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (7) to the block (273481213), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer ( to the block (273481214), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (9) to the block (273481215), which is in tree already bad_indirect_item: block 95485953: The item (10876 13329 0x1 IND (1), len 44, location 3268 entry count 0, fsck need 0, format new) has the bad pointer (10) to the block (273481216), which is in tree already /135 (of 160// 45 (of 170-bad_path: The left delimiting key [15312 15313 0x1 DRCT (2)] of the node (298287105) must be equal to the first element's key [6352 6354 0x23a001 IND (1)] within th/ 46 (of 170\bad_path: The left delimiting key [15312 15314 0x0 SD (0)] of the node (298287106) must be equal to the first element's key [6352 6354 0x62e001 IND (1)] within the / 47 (of 170|bad_path: The left delimiting key [15312 15320 0x1 DRCT (2)] of the node (298287107) must be equal to the first element's key [6352 6354 0xa22001 IND (1)] within th/122 (of 170\bad_indirect_item: block 71354742: The item (15591 15592 0x1 IND (1), len 8, location 740 entry count 0, fsck need 0, format new) has the bad pointer (0) to the block (316966918), which is in tree already bad_indirect_item: block 71354742: The item (15591 15592 0x1 IND (1), len 8, location 740 entry count 0, fsck need 0, format new) has the bad pointer (1) to the block (316966919), which is in tree already /123 (of 170|bad_node: vpf-10350: The block (316964866) is used more than once in the tree. the problem in the internal node occured (316964866), whole subtree is skipped /136 (of 160// 21 (of 170-bad_indirect_item: block 284690194: The item (15828 15832 0x1 IND (1), len 8, location 788 entry count 0, fsck need 0, format new) has the bad pointer (0) to the block (444106922), which is in tree already bad_indirect_item: block 284690194: The item (15828 15832 0x1 IND (1), len 8, location 788 entry count 0, fsck need 0, format new) has the bad pointer (1) to the block (444106923), which is in tree already / 22 (of 170\bad_node: vpf-10350: The block (444104705) is used more than once in the tree. the problem in the internal node occured (444104705), whole subtree is skipped / 16 (of 16|/ 18 (of 169\/110 (of 170/bad_node: vpf-10350: The block (316964867) is used more than once in the tree. the problem in the internal node occured (316964867), whole subtree is skipped / 46 (of 169|/ 59 (of 137\bad_node: vpf-10350: The block (444104709) is used more than once in the tree. the problem in the internal node occured (444104709), whole subtree is skipped / 51 (of 169//160 (of 170/bad_node: vpf-10350: The block (444104719) is used more than once in the tree. the problem in the internal node occured (444104719), whole subtree is skipped /123 (of 169-/ 23 (of 170/bad_node: vpf-10350: The block (444104725) is used more than once in the tree. the problem in the internal node occured (444104725), whole subtree is skipped /152 (of 169|/ 27 (of 148\bad_indirect_item: block 364517672: The item (43988 43991 0x1 IND (1), len 8, location 1044 entry count 0, fsck need 0, format new) has the bad pointer (0) to the block (444112711), which is in tree already bad_indirect_item: block 364517672: The item (43988 43991 0x1 IND (1), len 8, location 1044 entry count 0, fsck need 0, format new) has the bad pointer (1) to the block (444112712), which is in tree already / 28 (of 148|bad_node: vpf-10350: The block (444104726) is used more than once in the tree. the problem in the internal node occured (444104726), whole subtree is skipped /168 (of 169-/115 (of 128/bad_node: vpf-10350: The block (444104728) is used more than once in the tree. the problem in the internal node occured (444104728), whole subtree is skipped finished Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs. Bad nodes were found, Semantic pass skipped 39 found corruptions can be fixed only when running with --rebuild-tree ########### reiserfsck finished at Tue Dec 6 20:37:01 2011 ########### root@Tower:~# According to http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems I should go ahead and run the --rebuild-tree option, so I'm going to do that now and measure the result. I will capture the full log and post it. Thank you!
December 7, 201114 yr Author Well that failed. The system locked during the --rebuild-tree. Tower login: root Password: Linux 2.6.32.9-unRAID. root@Tower:~# reiserfsck --rebuild-tree /dev/md2 reiserfsck 3.6.21 (2009 www.namesys.com) ************************************************************* ** Do not run the program with --rebuild-tree unless ** ** something is broken and MAKE A BACKUP before using it. ** ** If you have bad sectors on a drive it is usually a bad ** ** idea to continue using it. Then you probably should get ** ** a working hard drive, copy the file system from the bad ** ** drive to the good one -- dd_rescue is a good tool for ** ** that -- and only then run this program. ** ** If you are using the latest reiserfsprogs and it fails ** ** please email bug reports to [email protected], ** ** providing as much information as possible -- your ** ** hardware, kernel, patches, settings, all reiserfsck ** ** messages (including version), the reiserfsck logfile, ** ** check the syslog file for any related information. ** ** If you would like advice on using this program, support ** ** is available for $25 at www.namesys.com/support.html. ** ************************************************************* Will rebuild the filesystem (/dev/md2) tree Will put log info to 'stdout' Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes Replaying journal: Done. Reiserfs journal '/dev/md2' in blocks [18..8211]: 0 transactions replayed ########### reiserfsck --rebuild-tree started at Tue Dec 6 20:46:26 2011 ########### Pass 0: ####### Pass 0 ####### Loading on-disk bitmap .. ok, 179495289 blocks marked used Skipping 23115 blocks (super block, journal, bitmaps) 179472174 blocks will be read 0%....20% left 102664197, 26449 /seccccc Current situation: System locked while disk2 was undergoing --rebuild-tree System locked while disk3 was undergoing a disk rebuild I left it in the locked state currently in hopes that it recovers overnight, but I have little faith. Advice would be very appreciated.
December 8, 201114 yr Author Smart report for sda (cache drive) root@Tower:~# smartctl -a -d ata /dev/sda smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Maxtor DiamondMax 10 family (ATA/133 and SATA/150) Device Model: Maxtor 6L300S0 Serial Number: L61R52CG Firmware Version: BACE1G20 User Capacity: 300,089,646,592 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 7 ATA Standard is: ATA/ATAPI-7 T13 1532D revision 0 Local Time is: Wed Dec 7 17:44:21 2011 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (2283) seconds. Offline data collection capabilities: (0x5b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 118) minutes. SCT capabilities: (0x0021) SCT Status supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 3 Spin_Up_Time 0x0027 186 180 063 Pre-fail Always - 17402 4 Start_Stop_Count 0x0032 252 252 000 Old_age Always - 3263 5 Reallocated_Sector_Ct 0x0033 253 253 063 Pre-fail Always - 0 6 Read_Channel_Margin 0x0001 253 253 100 Pre-fail Offline - 0 7 Seek_Error_Rate 0x000a 253 252 000 Old_age Always - 0 8 Seek_Time_Performance 0x0027 249 237 187 Pre-fail Always - 54182 9 Power_On_Minutes 0x0032 193 193 000 Old_age Always - 131h+08m 10 Spin_Retry_Count 0x002b 253 249 157 Pre-fail Always - 0 11 Calibration_Retry_Count 0x002b 253 252 223 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 251 251 000 Old_age Always - 1078 192 Power-Off_Retract_Count 0x0032 253 253 000 Old_age Always - 0 193 Load_Cycle_Count 0x0032 253 253 000 Old_age Always - 0 194 Temperature_Celsius 0x0032 038 253 000 Old_age Always - 30 195 Hardware_ECC_Recovered 0x000a 253 252 000 Old_age Always - 3443 196 Reallocated_Event_Count 0x0008 253 253 000 Old_age Offline - 0 197 Current_Pending_Sector 0x0008 253 253 000 Old_age Offline - 0 198 Offline_Uncorrectable 0x0008 253 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0008 199 199 000 Old_age Offline - 0 200 Multi_Zone_Error_Rate 0x000a 253 252 000 Old_age Always - 0 201 Soft_Read_Error_Rate 0x000a 253 252 000 Old_age Always - 0 202 Data_Address_Mark_Errs 0x000a 253 252 000 Old_age Always - 0 203 Run_Out_Cancel 0x000b 253 252 180 Pre-fail Always - 1 204 Soft_ECC_Correction 0x000a 253 252 000 Old_age Always - 0 205 Thermal_Asperity_Rate 0x000a 253 252 000 Old_age Always - 0 207 Spin_High_Current 0x002a 253 249 000 Old_age Always - 0 208 Spin_Buzz 0x002a 253 252 000 Old_age Always - 0 209 Offline_Seek_Performnce 0x0024 240 240 000 Old_age Offline - 157 210 Unknown_Attribute 0x0032 253 252 000 Old_age Always - 0 211 Unknown_Attribute 0x0032 253 252 000 Old_age Always - 0 212 Unknown_Attribute 0x0032 253 252 000 Old_age Always - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. root@Tower:~# Smart report for sdb (parity) root@Tower:~# smartctl -a -d ata /dev/sdb smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD20EARS-00MVWB0 Serial Number: WD-WCAZA6004572 Firmware Version: 51.0AB51 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Wed Dec 7 17:44:56 2011 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (37260) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 194 190 051 Pre-fail Always - 2694 3 Spin_Up_Time 0x0027 244 169 021 Pre-fail Always - 2766 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 292 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 097 097 000 Old_age Always - 2546 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 65 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 35 193 Load_Cycle_Count 0x0032 199 199 000 Old_age Always - 3817 194 Temperature_Celsius 0x0022 123 110 000 Old_age Always - 27 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 199 000 Old_age Offline - 2 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. Smart report for sdc (disk2, drive which i tried to run --rebuild-tree on but the server locked in the process) root@Tower:~# smartctl -a -d ata /dev/sdc smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD20EARS-00J2GB0 Serial Number: WD-WCAYY0019666 Firmware Version: 80.00A80 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Wed Dec 7 17:45:03 2011 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (39600) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3031) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 172 153 021 Pre-fail Always - 8391 4 Start_Stop_Count 0x0032 099 099 000 Old_age Always - 1527 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 085 085 000 Old_age Always - 11373 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 105 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 50 193 Load_Cycle_Count 0x0032 195 195 000 Old_age Always - 16132 194 Temperature_Celsius 0x0022 122 089 000 Old_age Always - 30 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 196 000 Old_age Always - 92 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 11010 - # 2 Short offline Completed without error 00% 10981 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
December 8, 201114 yr Author Smart report for sdd (disk5) root@Tower:~# smartctl -a -d ata /dev/sdd smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD10EARS-00Y5B1 Serial Number: WD-WCAV56034780 Firmware Version: 80.00A80 User Capacity: 1,000,204,886,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Wed Dec 7 17:45:06 2011 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x84) Offline data collection activity was suspended by an interrupting command from host. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (18960) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 219) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3031) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 129 123 021 Pre-fail Always - 6541 4 Start_Stop_Count 0x0032 094 094 000 Old_age Always - 6025 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 14423 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 132 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 78 193 Load_Cycle_Count 0x0032 188 188 000 Old_age Always - 37387 194 Temperature_Celsius 0x0022 117 085 000 Old_age Always - 30 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. Smart report for sdf (disk4) root@Tower:~# smartctl -a -d ata /dev/sdf smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Model Family: Western Digital Caviar Green family Device Model: WDC WD10EACS-00D6B1 Serial Number: WD-WCAU44784010 Firmware Version: 01.01A01 User Capacity: 1,000,204,886,016 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Wed Dec 7 17:45:26 2011 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x84) Offline data collection activity was suspended by an interrupting command from host. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (22200) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x303f) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 168 156 021 Pre-fail Always - 6575 4 Start_Stop_Count 0x0032 093 093 000 Old_age Always - 7293 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0 9 Power_On_Hours 0x0032 068 068 000 Old_age Always - 23945 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 139 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 10 193 Load_Cycle_Count 0x0032 198 198 000 Old_age Always - 7293 194 Temperature_Celsius 0x0022 121 073 000 Old_age Always - 29 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 199 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. Smart report for sdg (disk3, drive which i tried to run data rebuild on but the server locked in the process) root@Tower:~# smartctl -a -d ata /dev/sdg smartctl 5.39.1 2010-01-28 r3054 [i486-slackware-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net === START OF INFORMATION SECTION === Device Model: WDC WD20EARX-00PASB0 Serial Number: WD-WCAZA8408153 Firmware Version: 51.0AB51 User Capacity: 2,000,398,934,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 8 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Wed Dec 7 17:45:29 2011 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x80) Offline data collection activity was never started. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (39960) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 255) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 100 253 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 175 174 021 Pre-fail Always - 6233 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 13 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 16 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 10 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 6 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 19 194 Temperature_Celsius 0x0022 121 117 000 Old_age Always - 29 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
December 8, 201114 yr Author Thank you much for all the help. The smart reports are above. A syslog is also here if needed. syslog.txt
Archived
This topic is now archived and is closed to further replies.