[6.3.2] problem with reiserfsck 3.6.25

[sacretagent 0]

Hi Tom, Limetech Team,

 

me and a few other people have a problem with the current version of reiserfsck that is in unraid 6.3.2

 

while running --rebuild-tree after --check tells us to do so... it throws a segmentation fault as underneath

i had it with 2 different disks in 2 different servers on different positions on each disk

but on the same position when i moved the disk to the other server

 

see thread from me in the general support forum

 

root@R2D2:~# reiserfsck --rebuild-tree /dev/sdg1
reiserfsck 3.6.25

*************************************************************
** Do not  run  the  program  with  --rebuild-tree  unless **
** something is broken and MAKE A BACKUP  before using it. **
** If you have bad sectors on a drive  it is usually a bad **
** idea to continue using it. Then you probably should get **
** a working hard drive, copy the file system from the bad **
** drive  to the good one -- dd_rescue is  a good tool for **
** that -- and only then run this program.                 **
*************************************************************

Will rebuild the filesystem (/dev/sdg1) tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Replaying journal: Done.
Reiserfs journal '/dev/sdg1' in blocks [18..8211]: 0 transactions replayed
###########
reiserfsck --rebuild-tree started at Thu Mar  9 20:41:59 2017
###########

Pass 0:
####### Pass 0 #######
Loading on-disk bitmap .. ok, 241399810 blocks marked used
Skipping 15663 blocks (super block, journal, bitmaps) 241384147 blocks will be read
0%...Segmentation fault                              left 207493688, 22914 /sec
root@R2D2:~#

 

when we downgrade unraid to 6.2.4 which has reiserfsck 3.6.24 then it doesn't throw the segmentation fault and can we repair our drives

 

####### Pass 3a (lost+found pass) #########
Looking for lost directories:
Looking for lost files:7 /sec
rewrite_file: 2 items of file [10750 10751] moved to [10750 66]
vpf-10680: The file [10750 66] has the wrong block count in the StatData (18592) - corrected to (0)
vpf-10680: The file [96666 8433] has the wrong block count in the StatData (86526040) - corrected to (86526024)
Flushing..finished
        Objects without names 2
        Files linked to /lost+found 2
        Objects having used objectids: 1
                files fixed 1
Pass 4 - finished done 494032, 55 /sec
Flushing..finished
Syncing..finished
###########
reiserfsck finished at Fri Mar 10 07:23:55 2017
###########

 

 

would it be possible to check on this and solve the issue or downgrade reiserfsck in the next release?

 

 

Edited March 10, 2017 by sacretagent

[JorgeB 3852]

Tom, in case it helps, these are the other two similar cases I know of:

 

 

[limetech 2284]

Thanks for diagnosing this.  Hans still has a few years to go before possibly being let out of prison, so we'll take a look at this.

[limetech 2284]

On 3/9/2017 at 4:31 PM, sacretagent said:

while running --rebuild-tree after --check tells us to do so... it throws a segmentation fault as underneath

 

I need to see more of the info spit out as a result of the segfault.  It probably would be in your syslog, do you still have it?

[sacretagent 0]

Hi Tom,

 

above in the first post is all what it gives me ...

nothing more

let me see if i can find something in the logs but i doubt it as i did some unclean shutdowns and so on ....

i am kind of busy merging the 2 servers and so for the moment am running without parity ,....

that's why i was so pissed when reiserfsck which always worked in the past now suddenly refused to repair "minor" damages in the file system

 

anyway one of the disks is fully repaired and the other one i am over 80% checking now.. but i still have some bad blocks...

 

will see what i can do

 

[JorgeB 3852]

Another reiserfsck --rebuild-tree segfault here.

 

syslog entry:

 

Mar 18 21:29:14 unRAID kernel: reiserfsck[2828]: segfault at 8 ip 00002b412d20a980 sp 00007ffdfc89fc48 error 4 in libreiserfscore.so.0.0.0[2b412d1fa000+27000]

 

reiserfsck output:

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Replaying journal: Done.
Reiserfs journal '/dev/md1' in blocks [18..8211]: 0 transactions replayed
###########
reiserfsck --rebuild-tree started at Sat Mar 18 21:28:40 2017
###########

Pass 0:
####### Pass 0 #######
Loading on-disk bitmap .. ok, 226344267 blocks marked used
Skipping 30567 blocks (super block, journal, bitmaps) 226313700 blocks will be read
0%block 2523084: The number of items (1531) is incorrect, should be (1) - corrected
block 2523084: The free space (65269) is incorrect, should be (4045) - corrected
pass0: vpf-10110: block 2523084, item (0): Unknown item type found [4094427913 185720068 0xfdf30df6 ??? (15)] - deleted
Segmentation fault

 

 

[hgeorges 0]

Here are another two runs, on the dd image pulled from a 2TB HDD:

root@Tower:~# dd if=/dev/sdg1 of=/mnt/hdd5-reiserfs.dd bs=4096 conv=noerror
488378638+0 records in
488378638+0 records out
2000398901248 bytes (2.0 TB, 1.8 TiB) copied, 23448.1 s, 85.3 MB/s

root@Tower:~# reiserfsck --rebuild-tree --scan-whole-partition /mnt/hdd5-reiserfs.dd
reiserfsck 3.6.25

*************************************************************
** Do not  run  the  program  with  --rebuild-tree  unless **
** something is broken and MAKE A BACKUP  before using it. **
** If you have bad sectors on a drive  it is usually a bad **
** idea to continue using it. Then you probably should get **
** a working hard drive, copy the file system from the bad **
** drive  to the good one -- dd_rescue is  a good tool for **
** that -- and only then run this program.                 **
*************************************************************

Will rebuild the filesystem (/mnt/hdd5-reiserfs.dd) tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Replaying journal: Done.
Reiserfs journal '/mnt/hdd5-reiserfs.dd' in blocks [18..8211]: 0 transactions replayed
###########
reiserfsck --rebuild-tree started at Mon Mar 20 20:12:55 2017
###########

Pass 0:
####### Pass 0 #######
The whole partition (488378638 blocks) is to be scanned
Skipping 23115 blocks (super block, journal, bitmaps) 488355523 blocks will be read
0%....20%....40%..block 245253713: The number of items (29451) is incorrect, should be (1) - corrected
block 245253713: The free space (29797) is incorrect, should be (3536) - corrected
pass0: vpf-10110: block 245253713, item (0): Unknown item type found [33554688 184551936 0x4000000 (15)] - deleted
Segmentation fault
root@Tower:~#

And trying again:

root@Tower:/mnt# reiserfsck --rebuild-tree --scan-whole-partition hdd5-reiserfs.dd
reiserfsck 3.6.25

*************************************************************
** Do not  run  the  program  with  --rebuild-tree  unless **
** something is broken and MAKE A BACKUP  before using it. **
** If you have bad sectors on a drive  it is usually a bad **
** idea to continue using it. Then you probably should get **
** a working hard drive, copy the file system from the bad **
** drive  to the good one -- dd_rescue is  a good tool for **
** that -- and only then run this program.                 **
*************************************************************

Will rebuild the filesystem (hdd5-reiserfs.dd) tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Replaying journal: Done.
Reiserfs journal 'hdd5-reiserfs.dd' in blocks [18..8211]: 0 transactions replayed
###########
reiserfsck --rebuild-tree started at Mon Mar 20 22:31:37 2017
###########

Pass 0:
####### Pass 0 #######
The whole partition (488378638 blocks) is to be scanned
Skipping 23115 blocks (super block, journal, bitmaps) 488355523 blocks will be read
0%....20%....40%..block 245253713: The number of items (29451) is incorrect, should be (1) - corrected
block 245253713: The free space (29797) is incorrect, should be (3536) - corrected
pass0: vpf-10110: block 245253713, item (0): Unknown item type found [33554688 184551936 0x4000000 (15)] - deleted
Segmentation fault
root@Tower:/mnt#

 

Edited March 21, 2017 by hgeorges
grammar correction

[limetech 2284]

10 hours ago, hgeorges said:

Here are another two runs, on the dd image pulled from a 2TB HDD:

 

Well at least it doesn't appear to be destructive.

We have sent a report to the reiserfsprogs maintainer about this, so far no response.  We are preparing a 6.3.3 release which will have this program reverted back to the previous release.

[sacretagent 0]

Thanks Tom,

 

will make things easier when another disk would go bad

 

 

[hgeorges 0]

Hi,

any update on this particular problem. I'm sort-of in "limbo" - not sure how to best resolve the issue.

I started this procedure hoping I can undelete some files (pictures).

But it turns out that there are bigger problems lurking in the background.

fsck is not working reliably

fsck is reporting strange filesystem problems, which a cursory monthly scan of parity doesn't reveal

fsck is also modifying the image to the point that it is no longer mountable. (it was before running fsck). Probably crashing in the middle of the procedure did that.

 

Here is the output of a new run:

root@Tower:~# reiserfsck --rebuild-tree --scan-whole-partition /mnt/disk9/hd5.dd

reiserfsck 3.6.25

*************************************************************
** Do not  run  the  program  with  --rebuild-tree  unless **
** something is broken and MAKE A BACKUP  before using it. **
** If you have bad sectors on a drive  it is usually a bad **
** idea to continue using it. Then you probably should get **
** a working hard drive, copy the file system from the bad **
** drive  to the good one -- dd_rescue is  a good tool for **
** that -- and only then run this program.                 **
*************************************************************

Will rebuild the filesystem (/mnt/disk9/hd5.dd) tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Replaying journal: Done.
Reiserfs journal '/mnt/disk9/hd5.dd' in blocks [18..8211]: 0 transactions replayed
###########
reiserfsck --rebuild-tree started at Sun Mar 26 07:57:15 2017
###########

Pass 0:
####### Pass 0 #######
The whole partition (488378638 blocks) is to be scanned
Skipping 23115 blocks (super block, journal, bitmaps) 488355523 blocks will be read
0%....20%....40%..block 245253713: The number of items (29451) is incorrect, should be (1) - corrected
block 245253713: The free space (29797) is incorrect, should be (3536) - corrected
pass0: vpf-10110: block 245253713, item (0): Unknown item type found [33554688 184551936 0x4000000 (15)] - deleted
Segmentation fault
root@Tower:~# mount -o loop /mnt/disk9/hd5.dd /mnt/recovery                     mount: /dev/loop1: can't read superblock
root@Tower:~# ls  /mnt/disk9/hd5.dd
/mnt/disk9/hd5.dd
root@Tower:~#

 

What is the verdict?

I could rollback to one of the prior versions, but I would have to go through successive tries to find out which one has a reliable reiserfsck.

 

[hgeorges 0]

So... I went up in this thread, and apparently the answer might be to go back to unraid 6.2.4.

Is this the official stance too?

[Squid 3285]

 

Pretty much...  If only to run reiserfsck or until 6.3.3 is released

[limetech 2284]

Finishing up 6.3.3 release, should be very soon (yes I hate saying that, but am working on Sunday to get this done).

[hgeorges 0]

Thank you, Tom!

[JorgeB 3852]

I know you're going to go back to previous tools but you may want to inform the reiser guys that --rebuild-sb is also not working on the latest reiserfs-progs, noticed because of this thread where reiserfsck gives an error I never seen before.

 

If the superblock is not found or corrupt you get an unhelpful error instead of the suggestion to run --rebuild-sb, but even if you know that you need to rebuild it and use --rebuild-sb it still won't work and exit with same error, how to reproduce:

 

delete superblock on reiser disk with dd:

 

dd if=/dev/zero seek=192 of=/dev/sdd bs=512 count=1

 

run reiserfsck, output with v6.3.2:

reiserfsck /dev/sdd1
reiserfsck 3.6.25

Will read-only check consistency of the filesystem on /dev/sdd1
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Failed to open the device '/dev/sdd1': Unknown code er3k 127

output with v6.2.4:

reiserfsck /dev/sdd1
reiserfsck 3.6.24

Will read-only check consistency of the filesystem on /dev/sdd1
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes

reiserfs_open: the reiserfs superblock cannot be found on /dev/sdd1.
Failed to open the filesystem.

If the partition table has not been changed, and the partition is
valid  and  it really  contains  a reiserfs  partition,  then the
superblock  is corrupted and you need to run this utility with
--rebuild-sb.

 

We know the superblock is missing so try to rebuild it on v6.3.2, same error:

reiserfsck --rebuild-sb /dev/sdd1
reiserfsck 3.6.25

Will check superblock and rebuild it if needed
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Failed to open the device '/dev/sdd1': Unknown code er3k 127

On v6.2.4 it works as usual and it correctly rebuilds the superblock.

 

Edited March 30, 2017 by johnnie.black

[SSD 342]

Another sign that RFS is not getting the attention it needs to maintain it as a viable file system for important user data. Previously we had the bug where RFS caused data corruption. Now the tools are not being tested before release. Definitely recommend a move to XFS.

[Squid 3285]

4 hours ago, bjp999 said:

Another sign that RFS is not getting the attention it needs to maintain it as a viable file system for important user data. Previously we had the bug where RFS caused data corruption. Now the tools are not being tested before release. Definitely recommend a move to XFS.

Probably high time that unRaid actively prevents new disks being formatted at RFS.  (Would still have to allow already existing disks to be mounted though)

[RobJ 137]

Perhaps deprecate it first, for a few versions.  Stick a " [Not recommended] " next to it, in the appropriate places.  This would help Tom evaluate the feedback, how upset some might be without it.  Hopefully, there's no reaction at all.

[Squid 3285]

1 hour ago, RobJ said:

Perhaps deprecate it first, for a few versions.  Stick a " [Not recommended] " next to it, in the appropriate places.  This would help Tom evaluate the feedback, how upset some might be without it.  Hopefully, there's no reaction at all.

I'll issue a PR for that tonight.  I don't think he'd have any issues with that comment in the GUI as reiser keeps being a PITA for him.

[limetech 2284]

6 minutes ago, Squid said:

I'll issue a PR for that tonight.  I don't think he'd have any issues with that comment in the GUI as reiser keeps being a PITA for him.

 

Please hold off on that until after 6.3.3/6.4.0-rc1 is released since we are about to release.

 

I agree we should deprecate ReiserFS but here are the reasons we haven't taken an official stance on this yet (other than making xfs the default for array disks and btrfs the default for cache):

1. So far it really hasn't been much of an issue until this latest upgrade to reiserfsprogs.  Yes there was a big snafu a couple years ago with data corruption, but reiserfs hasn't been the only filesystem to ever have those kinds of regressions.  I thought the reiserfs maintainers learned their lesson with that one, but the differences between current reisersfsprogs and previous is that SUSE, for some damn reason, decided to "refactor" and put some common code into a library.  Why they decided to do this with legacy s/w is a total mystery to me.  But you all are right, it's time to dump this thing, but...
2. It doesn't feel right to tell everyone to not use a filesystem, and in fact recommend to migrate off, when we don't provide a tool for doing so.  The solution to this will come in 6.4 series where we also fix the "Unexplained Disk Spinups" issue.  To fix that issue we are generalizing the 'mover' to be able to move contents of any device to any other device or device set.  The implementation of this utility will fix the spinup bug but can also be used to depopulate a device.  A couple reasons to depopulate a device would be in preparation of removing from service, or in preparation for changing the filesystem on the device.

[SSD 342]

Sounds great! Thanks Tom!!

[RobJ 137]

6.3.3 is now out, with reiserfsprogs 3.6.24, so it's safe to run reiserfsck again!

[Alkali 0]

Just wanted to say Thanks for this thread, was running into all sorts of issues after I had to do a new config due to a bad SATA cable.  Upgraded to Unraid 6.3.5 and reiserfsck actually was able to correct the issues identified.

 

Cheers!

Edited January 10, 2018 by Alkali
Wrong Version listed