Help Replacing Failed Disk


Recommended Posts

I had a 2TB WD EARS go bad on me last week.  Got the replacement through RMA, precleared and inserted into the array, rebuilt the disk, everything seemed normal

 

However, now the disk is showing as 17.59TB free in the gui.  Its disk 4 and all files from disk 4 are there when I browse the drive

 

How do I get this fixed?  Syslog is attached

 

thanks

 

Josh

syslog-2011-09-14.txt

Link to comment

I had a 2TB WD EARS go bad on me last week.  Got the replacement through RMA, precleared and inserted into the array, rebuilt the disk, everything seemed normal

 

However, now the disk is showing as 17.59TB free in the gui.  Its disk 4 and all files from disk 4 are there when I browse the drive

 

How do I get this fixed?  Syslog is attached

 

thanks

 

Josh

You are not the first to report an inconsistency in the reported free space in the latest betas.  You might just try rebooting and see if the reported size fixes itself.
Link to comment

so still have no idea what to do.  df command showing that disk 4 has used -15TB of its 2TB space

 

root@Tower:/etc/rc.d/unraid.d# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1              3907560   1270364   2637196  33% /boot
/dev/md11            1953454928 1922942796  30512132  99% /mnt/disk11
/dev/md5             976732736 943558840  33173896  97% /mnt/disk5
/dev/md18            1953454928 1900403088  53051840  98% /mnt/disk18
/dev/sdd1            1465093832 408558320 1056535512  28% /mnt/cache
/dev/md8             976732736 949529776  27202960  98% /mnt/disk8
/dev/md9             1953454928 1938895652  14559276 100% /mnt/disk9
/dev/md13            1953454928 1923419456  30035472  99% /mnt/disk13
/dev/md4             1953454928 -15226378888 17179833816   -  /mnt/disk4
/dev/md12            1953454928 1928291580  25163348  99% /mnt/disk12
/dev/md16            1953454928 1904103580  49351348  98% /mnt/disk16
/dev/md2             1953454928 1736181064 217273864  89% /mnt/disk2
/dev/md7             1953454928 1649254312 304200616  85% /mnt/disk7
/dev/md3             1953454928 1850980620 102474308  95% /mnt/disk3
/dev/md20            1953454928 1922858960  30595968  99% /mnt/disk20
/dev/md19            1953454928 1935262320  18192608 100% /mnt/disk19
/dev/md17            1953454928 1914129464  39325464  98% /mnt/disk17
/dev/md1             1953454928 1820623236 132831692  94% /mnt/disk1
/dev/md14            1953454928 1926885296  26569632  99% /mnt/disk14
/dev/md6             976732736 947299488  29433248  97% /mnt/disk6
/dev/md15            1953454928 1927515664  25939264  99% /mnt/disk15
/dev/md10            1953454928 1927366528  26088400  99% /mnt/disk10
shfs                 36138931984 17743122832 18395809152  50% /mnt/user0
shfs                 36138931984 17743122832 18395809152  50% /mnt/user
//192.168.0.3/Movies 8790531300 5069576552 3720954748  58% /mnt/Huey/Movies
//192.168.0.3/Misc   8790531300 5069576552 3720954748  58% /mnt/Huey/Misc

Link to comment

so still have no idea what to do.  df command showing that disk 4 has used -15TB of its 2TB space

 

root@Tower:/etc/rc.d/unraid.d# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1              3907560   1270364   2637196  33% /boot
/dev/md11            1953454928 1922942796  30512132  99% /mnt/disk11
/dev/md5             976732736 943558840  33173896  97% /mnt/disk5
/dev/md18            1953454928 1900403088  53051840  98% /mnt/disk18
/dev/sdd1            1465093832 408558320 1056535512  28% /mnt/cache
/dev/md8             976732736 949529776  27202960  98% /mnt/disk8
/dev/md9             1953454928 1938895652  14559276 100% /mnt/disk9
/dev/md13            1953454928 1923419456  30035472  99% /mnt/disk13
/dev/md4             1953454928 -15226378888 17179833816   -  /mnt/disk4
/dev/md12            1953454928 1928291580  25163348  99% /mnt/disk12
/dev/md16            1953454928 1904103580  49351348  98% /mnt/disk16
/dev/md2             1953454928 1736181064 217273864  89% /mnt/disk2
/dev/md7             1953454928 1649254312 304200616  85% /mnt/disk7
/dev/md3             1953454928 1850980620 102474308  95% /mnt/disk3
/dev/md20            1953454928 1922858960  30595968  99% /mnt/disk20
/dev/md19            1953454928 1935262320  18192608 100% /mnt/disk19
/dev/md17            1953454928 1914129464  39325464  98% /mnt/disk17
/dev/md1             1953454928 1820623236 132831692  94% /mnt/disk1
/dev/md14            1953454928 1926885296  26569632  99% /mnt/disk14
/dev/md6             976732736 947299488  29433248  97% /mnt/disk6
/dev/md15            1953454928 1927515664  25939264  99% /mnt/disk15
/dev/md10            1953454928 1927366528  26088400  99% /mnt/disk10
shfs                 36138931984 17743122832 18395809152  50% /mnt/user0
shfs                 36138931984 17743122832 18395809152  50% /mnt/user
//192.168.0.3/Movies 8790531300 5069576552 3720954748  58% /mnt/Huey/Movies
//192.168.0.3/Misc   8790531300 5069576552 3720954748  58% /mnt/Huey/Misc

Please report the issue to [email protected]

 

Obviously you cannot have negative space used.    It looks like a bug in the latest beta in reporting disk space used/free.

 

Joe L.

Link to comment

Will do, also just finished a non-correcting parity check and this was the result

 

Sep 16 11:43:54 Tower kernel: md: sync done. time=39315sec
Sep 16 11:43:54 Tower kernel: md: recovery thread sync completion status: 0
Sep 16 12:21:18 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 840 does not match to the expected one 1
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-5150 search_by_key: invalid format found in block 448294367. Fsck?
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [10 305 0x0 SD]
Sep 16 12:21:18 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 840 does not match to the expected one 1
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-5150 search_by_key: invalid format found in block 448294367. Fsck?
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [10 305 0x0 SD]

Link to comment

Will do, also just finished a non-correcting parity check and this was the result

 

Sep 16 11:43:54 Tower kernel: md: sync done. time=39315sec
Sep 16 11:43:54 Tower kernel: md: recovery thread sync completion status: 0
Sep 16 12:21:18 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 840 does not match to the expected one 1
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-5150 search_by_key: invalid format found in block 448294367. Fsck?
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [10 305 0x0 SD]
Sep 16 12:21:18 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 840 does not match to the expected one 1
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-5150 search_by_key: invalid format found in block 448294367. Fsck?
Sep 16 12:21:18 Tower kernel: REISERFS error (device md4): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [10 305 0x0 SD]

That is an entirely different issue.

 

You probably need to perform a file-system check of /dev/md4 as described here in the wiki:

http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems

Link to comment

Its telling me I should rebuild-tree, should I go ahead and do it?

 

root@Tower:~# reiserfsck --check /dev/md4
reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to [email protected], **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will read-only check consistency of the filesystem on /dev/md4
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Wed Sep 21 13:56:32 2011
###########
Replaying journal: Trans replayed: mountid 264, transid 174545, desc 1615, len 1, commit 1617, next trans offset 1600
Trans replayed: mountid 264, transid 174546, desc 1618, len 1, commit 1620, next trans offset 1603
Trans replayed: mountid 264, transid 174547, desc 1621, len 1, commit 1623, next trans offset 1606
Replaying journal: Done.
Reiserfs journal '/dev/md4' in blocks [18..8211]: 3 transactions replayed
Checking internal tree.. \/  1 (of  20|/ 49 (of 162|/138 (of 170-block 448207242: The level of the node (26129) is not correct, (1) expected
the problem in the internal node occured (448207242), whole subtree is skipped                                                                                                                                    / 50 (of 162\/ 16 (of  92\block 448255879: The level of the node (37605) is not correct, (1) expected
the problem in the internal node occured (448255879), whole subtree is skipped                                                                                                                                    / 51 (of 162|block 448288289: The level of the node (46881) is not correct, (2) expected
the problem in the internal node occured (448288289), whole subtree is skipped                                                                                                                                    /  6 (of  20// 72 (of 148// 24 (of  86/block 144853871: The level of the node (49067) is not correct, (1) expected
the problem in the internal node occured (144853871), whole subtree is skipped                                                                                                                                    / 83 (of 148\/130 (of 169/bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (239) to the block (144853869), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (241) to the block (144853872), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (242) to the block (144853873), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (243) to the block (144853874), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (244) to the block (144853875), which is in trefinished
Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs.
Bad nodes were found, Semantic pass skipped
4 found corruptions can be fixed only when running with --rebuild-tree
###########
reiserfsck finished at Wed Sep 21 14:10:46 2011
###########

Link to comment

Its telling me I should rebuild-tree, should I go ahead and do it?

 

root@Tower:~# reiserfsck --check /dev/md4
reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to [email protected], **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will read-only check consistency of the filesystem on /dev/md4
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
###########
reiserfsck --check started at Wed Sep 21 13:56:32 2011
###########
Replaying journal: Trans replayed: mountid 264, transid 174545, desc 1615, len 1, commit 1617, next trans offset 1600
Trans replayed: mountid 264, transid 174546, desc 1618, len 1, commit 1620, next trans offset 1603
Trans replayed: mountid 264, transid 174547, desc 1621, len 1, commit 1623, next trans offset 1606
Replaying journal: Done.
Reiserfs journal '/dev/md4' in blocks [18..8211]: 3 transactions replayed
Checking internal tree.. \/  1 (of  20|/ 49 (of 162|/138 (of 170-block 448207242: The level of the node (26129) is not correct, (1) expected
the problem in the internal node occured (448207242), whole subtree is skipped                                                                                                                                    / 50 (of 162\/ 16 (of  92\block 448255879: The level of the node (37605) is not correct, (1) expected
the problem in the internal node occured (448255879), whole subtree is skipped                                                                                                                                    / 51 (of 162|block 448288289: The level of the node (46881) is not correct, (2) expected
the problem in the internal node occured (448288289), whole subtree is skipped                                                                                                                                    /  6 (of  20// 72 (of 148// 24 (of  86/block 144853871: The level of the node (49067) is not correct, (1) expected
the problem in the internal node occured (144853871), whole subtree is skipped                                                                                                                                    / 83 (of 148\/130 (of 169/bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (239) to the block (144853869), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (241) to the block (144853872), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (242) to the block (144853873), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (243) to the block (144853874), which is in tree already
bad_indirect_item: block 134172764: The item (3983 39 0x1 IND (1), len 1132, location 192 entry count 0, fsck need 0, format new) has the bad pointer (244) to the block (144853875), which is in trefinished
Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs.
Bad nodes were found, Semantic pass skipped
4 found corruptions can be fixed only when running with --rebuild-tree
###########
reiserfsck finished at Wed Sep 21 14:10:46 2011
###########

yes
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.