May 17, 201610 yr I was trying to shutdown my tower to install some rams and I was getting unraid the level of the node is not correct errors: May 17 20:55:27 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 23788 does not match to the expected one 1 May 17 20:55:27 Tower kernel: REISERFS error (device md2): vs-5150 search_by_key: invalid format found in block 95645568. Fsck? I didn't know what happened (as to why it wasn't shutting down), so I turned off the power. After turning it back on and seeing the console screen, it was printing loads of the error above. After waiting some time, I still got the errors, so I thought it was because the wires got loose, so I made sure everything was okay. I then started it back up,and the console is still printing the same errors. However after some time, everything else started - Plex, etc. Seeing that everything could start, it means that everything shuold be fine, but unfortunately it doesn't seem that way. My content from disk2 appears to be missing. I don't see them on Plex Server or via SMB (windows folder). However, viewing the file contents via the webgui seems ok (I didn't read the contents, but it was able to list the folders/files). I see the Main tab, and for disk2 I'm seeing some errors count. The drive has no reallocated sector count, no current pending sector (looks quite normal). So what is happening? I went to click on the Logs, and this is some of the stuff I see: May 17 23:52:03 Tower shfs/user: shfs_readdir: fstatat: Some Show Name.avi (13) Permission denied May 17 23:52:03 Tower shfs/user: shfs_readdir: readdir_r: /mnt/disk2/TV Shows/Some Show Name (13) Permission denied May 17 23:56:03 Tower shfs/user: shfs_readdir: fstatat: Lost (13) Permission denied May 17 23:56:03 Tower shfs/user: shfs_readdir: readdir_r: /mnt/disk2/TV Shows/Some Folder (13) Permission denied So apparently the data is there(?), but yet it is preventing me from accessing them, so I don't know if it's working or not. Parity is running now, so I don't know if it's able to read disk2 or if it's just setting all of disk2 as non readable and updating accordingly on the parity drive. Is it just some file corruption? What am I suppose to do to make things right? Or is the system now trying to fix itself?
May 17, 201610 yr Community Expert https://lime-technology.com/wiki/index.php/Check_Disk_Filesystems#Drives_formatted_with_ReiserFS_using_unRAID_v5_or_later Start with disk2 (md2)
May 17, 201610 yr Author I should cancel the parity check now? I'm also getting "May 18 01:07:11 Tower shfs/user: shfs_read: read: (5) Input/output error" in the logs now
May 17, 201610 yr Author Ok, I'll stop it then, because I cannot take the array offline and go into maintenance mode (greyed out) until the parity check has been stopped. I will report back how it goes tomorrow edit: I was verifying files (I had them on torrent) on Some Show Name and looks like it is no longer 100%. It's 96.1% now (it is a very big torrent). Looks like even though some of the data seems lost we (or rather the transmission user) can still read the files. I also did a short SMART test and it completed without error.
May 17, 201610 yr Author Seems like I'm having a problem stopping array like the first time. It's getting stuck at Stopping Docker... (it usually doesn't take more than a minute), however the logging.htm is not showing any errors yet. I assume it's not able to stop disk2? The webgui is also not responsive. Any ideas on how I should now proceed? I don't think a hard restart is a good idea, since it would just start up all the services/plugins again, and I have no way to stop the array. This is what the log is showing: May 18 01:29:07 Tower shfs/user: shfs_read: read: (5) Input/output error May 18 01:35:58 Tower emhttp: shcmd (53): /usr/local/sbin/set_ncq sdh 1 &> /dev/null May 18 01:35:58 Tower emhttp: shcmd (54): /usr/local/sbin/set_ncq sdb 1 &> /dev/null May 18 01:35:58 Tower emhttp: shcmd (55): /usr/local/sbin/set_ncq sdd 1 &> /dev/null May 18 01:35:58 Tower emhttp: shcmd (56): /usr/local/sbin/set_ncq sdf 1 &> /dev/null May 18 01:35:58 Tower emhttp: shcmd (57): /usr/local/sbin/set_ncq sdg 1 &> /dev/null May 18 01:35:58 Tower emhttp: shcmd (58): /usr/local/sbin/set_ncq sdc 1 &> /dev/null May 18 01:37:43 Tower emhttp: Stopping Docker... May 18 01:37:43 Tower logger: stopping docker ... May 18 01:37:44 Tower avahi-daemon[21360]: Withdrawing workstation service for vethb9609b1. May 18 01:37:44 Tower avahi-daemon[21360]: Withdrawing workstation service for veth978606e. May 18 01:37:44 Tower logger: fd024d56633e May 18 01:37:45 Tower logger: c31b3ba93c96 May 18 01:37:47 Tower logger: 3b3b75752239 May 18 01:37:47 Tower ntpd[1534]: Deleting interface #4 docker0, 172.17.42.1#123, interface stats: received=0, sent=0, dropped=0, active_time=13035 secs May 18 01:37:48 Tower logger: 9d44d2d3d777
May 18, 201610 yr Author Still need help entering maintenance mode, I can't do any drive check without entering into that mode. Any help would be great
May 18, 201610 yr Author Following instructions on forums, I did this to manually try to stop the array, but disk2 can't be stopped: killall smbd nmbd umount /mnt/disk* (where * is all my disks numbers) umount: /mnt/disk2: device is busy. (In some cases useful info about processes that use the device is found by lsof( or fuser(1)) I ran /usr/bin/fuser -mv /mnt/disk* /mnt/user/* and for disk2 I'm getting this: /mnt/disk2: root kernel mount /mnt/disk2 root 12505 f.... shfs
May 18, 201610 yr Author I just force shut down, rebooted with safe mode, and was able to stop it easily and go into maintenance mode. Ran the command, and this is the ending I get: the problem in the internal node occured (699286823),/ 21 (of 41)/144 (of 170)/ 33 (of 170)block 908214294: The level of the node (6008) is not correct, (1) expected the problem in the internal node occured (908214294),/145 (of 170)/ 92 (of 170)block 908214495: The level of the node (0) is not correct, (1) expected the problem in the internal node occured/ 33 (of 41)/115 (of 160)/167 (of 170)block 279183366: The level of the node (41175) is not correct, (1) expected the problem in the internal node occuredfinished Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs. Bad nodes were found, Semantic pass skipped 13 found corruptions can be fixed only when running with --rebuild-tree ########### reiserfsck finished at Wed May 18 21:42:49 2016 Should I go ahead with it? Also, I found that during boot the drive is giving me some errors, is this something to do with the disk data? Or will it be better if I just format the drive and use the parity drive to restore it? May 18 20:33:15 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:15 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:15 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:15 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:15 Tower kernel: ata2.00: cmd 25/00:40:b0:ff:4c/00:05:00:00:00/e0 tag 8 dma 688128 in May 18 20:33:15 Tower kernel: res 50/00:00:af:ff:4c/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:15 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:15 Tower kernel: ata2: hard resetting link May 18 20:33:15 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:15 Tower kernel: ata2.00: configured for UDMA/133 May 18 20:33:15 Tower kernel: ata2: EH complete May 18 20:33:20 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:20 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:20 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:20 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:20 Tower kernel: ata2.00: cmd 25/00:40:68:db:57/00:05:00:00:00/e0 tag 9 dma 688128 in May 18 20:33:20 Tower kernel: res 50/00:00:67:db:57/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:20 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:20 Tower kernel: ata2: hard resetting link May 18 20:33:20 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:20 Tower kernel: ata2.00: configured for UDMA/133 May 18 20:33:20 Tower kernel: ata2: EH complete May 18 20:33:21 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:21 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:21 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:21 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:21 Tower kernel: ata2.00: cmd 25/00:40:98:4d:5c/00:05:00:00:00/e0 tag 5 dma 688128 in May 18 20:33:21 Tower kernel: res 50/00:00:97:4d:5c/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:21 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:21 Tower kernel: ata2: hard resetting link May 18 20:33:21 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:21 Tower kernel: ata2.00: configured for UDMA/133 May 18 20:33:21 Tower kernel: ata2: EH complete May 18 20:33:22 Tower kernel: ata2.00: limiting speed to UDMA/100:PIO4 May 18 20:33:22 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:22 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:22 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:22 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:22 Tower kernel: ata2.00: cmd 25/00:40:58:33:5f/00:05:00:00:00/e0 tag 2 dma 688128 in May 18 20:33:22 Tower kernel: res 50/00:00:57:33:5f/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:22 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:22 Tower kernel: ata2: hard resetting link May 18 20:33:22 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:22 Tower kernel: ata2.00: configured for UDMA/100 May 18 20:33:22 Tower kernel: ata2: EH complete May 18 20:33:33 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:33 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:33 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:33 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:33 Tower kernel: ata2.00: cmd 25/00:08:40:64:61/00:03:00:00:00/e0 tag 29 dma 397312 in May 18 20:33:33 Tower kernel: res 50/00:00:3f:64:61/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:33 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:33 Tower kernel: ata2: hard resetting link May 18 20:33:34 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:34 Tower kernel: ata2.00: configured for UDMA/100 May 18 20:33:34 Tower kernel: ata2: EH complete May 18 20:33:37 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:37 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:37 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:37 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:37 Tower kernel: ata2.00: cmd 25/00:a8:08:99:6b/00:03:00:00:00/e0 tag 26 dma 479232 in May 18 20:33:37 Tower kernel: res 50/00:00:07:99:6b/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:37 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:37 Tower kernel: ata2: hard resetting link May 18 20:33:37 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:37 Tower kernel: ata2.00: configured for UDMA/100 May 18 20:33:37 Tower kernel: ata2: EH complete May 18 20:33:38 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:38 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:38 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:38 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:38 Tower kernel: ata2.00: cmd 25/00:a8:48:e6:6e/00:03:00:00:00/e0 tag 16 dma 479232 in May 18 20:33:38 Tower kernel: res 50/00:00:47:e6:6e/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:38 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:38 Tower kernel: ata2: hard resetting link May 18 20:33:38 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:38 Tower kernel: ata2.00: configured for UDMA/100 May 18 20:33:38 Tower kernel: ata2: EH complete May 18 20:33:39 Tower kernel: ata2.00: limiting speed to UDMA/33:PIO4 May 18 20:33:39 Tower kernel: ata2.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen May 18 20:33:39 Tower kernel: ata2.00: irq_stat 0x08000000, interface fatal error May 18 20:33:39 Tower kernel: ata2: SError: { UnrecovData 10B8B BadCRC } May 18 20:33:39 Tower kernel: ata2.00: failed command: READ DMA EXT May 18 20:33:39 Tower kernel: ata2.00: cmd 25/00:40:50:10:74/00:05:00:00:00/e0 tag 28 dma 688128 in May 18 20:33:39 Tower kernel: res 50/00:00:4f:10:74/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) May 18 20:33:39 Tower kernel: ata2.00: status: { DRDY } May 18 20:33:39 Tower kernel: ata2: hard resetting link May 18 20:33:40 Tower kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 18 20:33:40 Tower kernel: ata2.00: configured for UDMA/33 May 18 20:33:40 Tower kernel: ata2: EH complete
May 18, 201610 yr Community Expert ata2 may not be disk2, looks like a bad SATA cable, post complete syslog diagnostics.
May 18, 201610 yr Author Done, please see attached. Would appreciate if you could advise if I should go ahead with the command "reiserfsck --rebuild-tree /dev/md2" or if something else can be done first. Thank you! tower-diagnostics-20160518-2245.zip
May 18, 201610 yr Community Expert Also, I found that during boot the drive is giving me some errors, is this something to do with the disk data? Or will it be better if I just format the drive and use the parity drive to restore it? Whatever you do, do NOT format the disk or you will lose all its contents. Parity does not store data in the sense you seem to think, it just stores information to allow the combination of all the other disks plus parity to restore the contents of the physical sectors on a failed disk. If you mistakenly issued a format command then this would be an instruction to create an empty file system on the disk in question and parity would have been be updated to reflect this,
May 18, 201610 yr Community Expert ata2 is disk5, check/replace SATA cable, you may want to do the same for disk3, there's also an elevated number of UDMA_CRC errors. Before or after that run reiserfsck --rebuild-tree on disk2.
May 18, 201610 yr Author Thank you guys for the advice. Worst case I can replace disk2 with a brand new drive and let parity restore the disk2 data. I will check/change the sata cables for those 2 drives esp disk5 once I've fix disk2 and shutdown-ed. In the meantime I will be running "reiserfsck --rebuild-tree /dev/md2" and post the results here
May 18, 201610 yr Community Expert Thank you guys for the advice. Worst case I can replace disk2 with a brand new drive and let parity restore the disk2 data. I will check/change the sata cables for those 2 drives esp disk5 once I've fix disk2 and shutdown-ed. In the meantime I will be running "reiserfsck --rebuild-tree /dev/md2" and post the results here Assuming parity is in sync rebuilding disk2 to a new disk will end up with the same result, file system corruption included.
May 18, 201610 yr Author Ah I see, didn't know that..could be worth a try too though... I wonder how it become like this in the first place, is it because I force shutdown? To have 13 corruptions seems like a lot, and I see that others usually get the -fixable flag version instead Here's the current progress (it's very very slow, looks like I won't be able to shutdown my computer for a couple of days)
May 18, 201610 yr Community Expert Ah I see, didn't know that..could be worth a try too though... I wonder how it become like this in the first place, is it because I force shutdown? To have 13 corruptions seems like a lot, and I see that others usually get the -fixable flag version instead Here's the current progress (it's very very slow, looks like I won't be able to shutdown my computer for a couple of days) I am guessing, but I would expect the rebuild to take a similar amount of time to a parity check (although it can be less). One thing to be aware of is that you must not close the SSH window while the rebuild is in progress or it will be aborted. If you want to be able to close the window and leave the rebuild running, then you need to start it under the 'screen' application that allows applications to continue running even if their ssh session terminates (instructions on how to do this can be found in the wiki). Note that 'screen' is not included as a standard part of unRAID, so if you do want it installed then start by installing the Nerd Pack plugin - it offers 'screen' as one of the packages it can install.
May 19, 201610 yr Author Yea, I decided to leave it on, but was worried if Windows decide to restart itself or if the power went out, and the disk will be even more corrupted.. Here is an update, I have no idea how to interpret if things are going smoothly though
May 19, 201610 yr Author Hi, the process finished, it did 4 passes and showed some files/directories. The first thing I did was to check the logs, and the reading of disk2 during the operation gave a lot of disk2 read error in the sectors, I hope it isn't caused by the sata connection..this is an example (it's just repeating throughout the night/day): May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199600 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199608 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199616 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199624 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199632 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199640 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199648 May 19 02:03:01 Tower kernel: ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 May 19 02:03:01 Tower kernel: ata3.00: irq_stat 0x40000001 May 19 02:03:01 Tower kernel: ata3.00: failed command: READ DMA EXT May 19 02:03:01 Tower kernel: ata3.00: cmd 25/00:00:28:9e:a7/00:04:a8:00:00/e0 tag 9 dma 524288 in May 19 02:03:01 Tower kernel: res 51/40:7f:a8:9f:a7/00:02:a8:00:00/e0 Emask 0x9 (media error) May 19 02:03:01 Tower kernel: ata3.00: status: { DRDY ERR } May 19 02:03:01 Tower kernel: ata3.00: error: { UNC } May 19 02:03:01 Tower kernel: ata3.00: configured for UDMA/133 May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 Sense Key : 0x3 [current] [descriptor] May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 ASC=0x11 ASCQ=0x4 May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 CDB: opcode=0x88 88 00 00 00 00 00 a8 a7 9e 28 00 00 04 00 00 00 May 19 02:03:01 Tower kernel: blk_update_request: I/O error, dev sdd, sector 2829557672 May 19 02:03:01 Tower kernel: ata3: EH complete May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557608 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557616 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557624 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557632 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557640 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557648 Is ata3 referring to disk2? Actually how do I check that myself using the diagnostics? Do you guys think it's a good idea to a check again? If that gives me the ok then I will shutdown the tower, check/replace cables, and then check the lost+found folder that is supposedly generated after the rebuildtree.
May 19, 201610 yr Author I just ran a short test and it returns without errors. The 3 errors listed seems to be from before.. smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.1.18-unRAID] (local build) Copyright © 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: WDC WD40EFRX-68WT0N0 Serial Number: WD-WCC4EM0WN6HS LU WWN Device Id: 5 0014ee 20a945045 Firmware Version: 80.00A80 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-2 (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Thu May 19 20:55:48 2016 SGT SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (52680) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 527) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x703d) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 215 182 021 Pre-fail Always - 6250 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 255 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 083 083 000 Old_age Always - 12637 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 102 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 66 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 1287 194 Temperature_Celsius 0x0022 113 103 000 Old_age Always - 39 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0 SMART Error Log Version: 1 ATA Error Count: 3 CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 3 occurred at disk power-on lifetime: 12566 hours (523 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 78 00 f3 24 e1 Error: UNC 120 sectors at LBA = 0x0124f300 = 19198720 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 78 00 f3 24 e1 00 6d+00:36:58.947 READ DMA c8 00 80 f0 f0 24 e1 00 6d+00:36:58.947 READ DMA c8 00 08 10 66 22 e1 00 6d+00:36:58.924 READ DMA c8 00 60 58 41 23 e1 00 6d+00:36:58.882 READ DMA Error 2 occurred at disk power-on lifetime: 12566 hours (523 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 20 90 0a b5 eb Error: UNC 32 sectors at LBA = 0x0bb50a90 = 196414096 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 20 90 0a b5 eb 00 6d+00:35:47.124 READ DMA c8 00 c0 78 35 b5 eb 00 6d+00:35:47.117 READ DMA c8 00 98 48 1f b5 eb 00 6d+00:35:47.116 READ DMA c8 00 08 20 1f b5 eb 00 6d+00:35:47.112 READ DMA c8 00 30 d0 0a b5 eb 00 6d+00:35:47.111 READ DMA Error 1 occurred at disk power-on lifetime: 12566 hours (523 days + 14 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 30 c0 d8 be eb Error: UNC 48 sectors at LBA = 0x0bbed8c0 = 197056704 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 30 c0 d8 be eb 00 6d+00:34:17.937 READ DMA c8 00 28 60 cf be eb 00 6d+00:34:17.934 READ DMA c8 00 70 28 ca be eb 00 6d+00:34:17.932 READ DMA c8 00 28 d8 c8 be eb 00 6d+00:34:17.929 READ DMA c8 00 40 78 c4 be eb 00 6d+00:34:17.919 READ DMA SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 12637 - # 2 Short offline Completed without error 00% 12594 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
May 19, 201610 yr Community Expert There are no pending sectors but suspect it's a disk problem, I would do an extended SMART test.
May 19, 201610 yr Author Thanks, will post result once done! Then I'll start the check to make sure theres no need for rebuildtree again
May 19, 201610 yr Hi, the process finished, it did 4 passes and showed some files/directories. The first thing I did was to check the logs, and the reading of disk2 during the operation gave a lot of disk2 read error in the sectors, I hope it isn't caused by the sata connection..this is an example (it's just repeating throughout the night/day): May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199600 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199608 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199616 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199624 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199632 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199640 May 19 02:02:40 Tower kernel: md: disk2 read error, sector=2824199648 May 19 02:03:01 Tower kernel: ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 May 19 02:03:01 Tower kernel: ata3.00: irq_stat 0x40000001 May 19 02:03:01 Tower kernel: ata3.00: failed command: READ DMA EXT May 19 02:03:01 Tower kernel: ata3.00: cmd 25/00:00:28:9e:a7/00:04:a8:00:00/e0 tag 9 dma 524288 in May 19 02:03:01 Tower kernel: res 51/40:7f:a8:9f:a7/00:02:a8:00:00/e0 Emask 0x9 (media error) May 19 02:03:01 Tower kernel: ata3.00: status: { DRDY ERR } May 19 02:03:01 Tower kernel: ata3.00: error: { UNC } May 19 02:03:01 Tower kernel: ata3.00: configured for UDMA/133 May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 Sense Key : 0x3 [current] [descriptor] May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 ASC=0x11 ASCQ=0x4 May 19 02:03:01 Tower kernel: sd 3:0:0:0: [sdd] tag#9 CDB: opcode=0x88 88 00 00 00 00 00 a8 a7 9e 28 00 00 04 00 00 00 May 19 02:03:01 Tower kernel: blk_update_request: I/O error, dev sdd, sector 2829557672 May 19 02:03:01 Tower kernel: ata3: EH complete May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557608 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557616 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557624 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557632 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557640 May 19 02:03:01 Tower kernel: md: disk2 read error, sector=2829557648 Is ata3 referring to disk2? Actually how do I check that myself using the diagnostics? Do you guys think it's a good idea to a check again? If that gives me the ok then I will shutdown the tower, check/replace cables, and then check the lost+found folder that is supposedly generated after the rebuildtree. You can see which ata# corresponds with which disk # by clicking the very little hard drive icon on every disk in the main tab Sent from my LG-D852 using Tapatalk
May 19, 201610 yr Author Ah, I didn't know that was actually clickable! Nice to see that it's showing the related disk logs! But what if I didn't have access to the webgui and only have the diagnostics zip file? How would I go about deciphering it? Also, I forgot to post the ending of my rebuildtree, in case it's worth any insights: Looking for lost files: Flushing..finished, 1125 /sec Objects without names 113 Dirs linked to /lost+found: 10 Dirs without stat data found 1 Files linked to /lost+found 103 Pass 4 - finishedone 783135, 6265 /sec Deleted unreachable items 3970 Flushing..finished Syncing..finished ########### reiserfsck finished at Thu May 19 13:11:34 2016 ########### sounds like a lot of folders/files are deleted?
Archived
This topic is now archived and is closed to further replies.