Good morning,

I am having trouble deleting several files on my server. It says the file is deleted but when I refresh the directory the file shows up again. I ran the permissions script in the utilities tab but this did not fix the issue. The only thing that indicates this is a permissions issue is that when I try to rename the files it tells me that I require permission from Tower\nobody which looking through the forums seems to indicate a rights permission issue.

Below is what the log is showing. It looks like some kind of file system corruption, is there a way to fix this? The files play fine. I am also fine with deleting these files as I re-ripped the episodes.

 

/usr/bin/tail -f /var/log/syslog

Dec 22 09:28:54 Tower emhttp_event: started

Dec 22 09:28:55 Tower avahi-daemon[1998]: Service "Tower" (/services/smb.service) successfully established.

Dec 22 09:29:43 Tower shfs/user: shfs_unlink: unlink: /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad .mkv (30) Read-only file system

Dec 22 09:29:43 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 49942 does not match to the expected one 1

Dec 22 09:29:43 Tower kernel: REISERFS error (device md2): vs-5150 search_by_key: invalid format found in block 548346361. Fsck?

Dec 22 09:29:43 Tower kernel: REISERFS (device md2): Remounting filesystem read-only

Dec 22 09:29:43 Tower shfs/user: shfs_unlink: unlink: /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E03 Brunch .mkv (30) Read-only file system

Dec 22 09:29:53 Tower shfs/user: shfs_rename: rename: /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad .mkv /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad 2 .mkv (30) Read-only file system

Dec 22 09:30:47 Tower shfs/user: shfs_rename: rename: /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad .mkv /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad 2 .mkv (30) Read-only file system

Dec 22 09:32:18 Tower kernel: mdcmd (47): clear

Dec 22 09:33:35 Tower shfs/user: shfs_rename: rename: /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad .mkv /mnt/disk2/TV Shows/How I met your Mother/Season 2/How I Met Your Mother S02E02 The Scorpion and the Toad 2 .mkv (30) Read-only file system

Please see Check Disk File systems, and follow the instructions very carefully.

Your file system is corrupted. That is why the syslog says it switched to readonly mode. You need to look into file system checks.

So in my case disk 2 is SDD , so the command would be

reiserfsck --fix-fixable /dev/sdd

 

All the data is backed up to another unRAID box, so I am not concerned about the files not being recoverable.

You want to be very careful and know if you should be using the unraid device instead of the raw device. If you use the /dev/sdX that will break parity!

 

All the wiki entries say to use /dev/mdX.

 

oh cool, good point, although at this point parity would have used the corrupt files?

oh cool, good point, although at this point parity would have used the corrupt files?

No - putting the array into maintenance mode and using /dev/md2 would correct the error and keep parity intact.   

 

You should avoid using the /dev/sd? type devices if at all possible as it breaks parity and so any new error on another disk while recovering this one will cause data loss..    Also note that if you use the raw devices you have to include the partition (e.g. /dev/sdd1) whereas this is handled for you with the md devices.

oh cool, good point, although at this point parity would have used the corrupt files?

If you fix the files you would want that fix to update the parity drive.

 

Also, the parity can still be used to rebuild one of the other disks which don't have file corruption. Or even rebuild the disk with file corruption which would be better than losing it all.

So did the file system corruption occur due to a bad server shutdown or does it mean the drive maybe having issues? I know the server had a few bad shutdowns and so I am thinking that may have been the cause but obviously it is a concern.

Once reiserfsck finished it recommended a --rebuild-tree which I am currently running but seems like it is going to take forever

So reiserfsck finally finished the rebuild tree and all the files look good now. I suppose my last question here is, should I run reiserfsck on the other disks in the array now? I would never have known there was an issue until I tried to delete these files.

So reiserfsck finally finished the rebuild tree and all the files look good now. I suppose my last question here is, should I run reiserfsck on the other disks in the array now? I would never have known there was an issue until I tried to delete these files.

 

Normally, you run it when you know it needs to be run, but it doesn't hurt to run it on all disks (but NOT the parity drive!) with the --check option (which is the default so doesn't need to be specified).

So in my case disk 2 is SDD , so the command would be

reiserfsck --fix-fixable /dev/sdd

 

All the data is backed up to another unRAID box, so I am not concerned about the files not being recoverable.

NO... completely wrong.  You'll cause damage to the file-system with that command.

 

you need to use /dev/md2 for disk2.

Luckily I ran --rebuild-tree on the MD2 and once that had corrected the issue I ran reiserfsck on the rest of the drives in the array where there were no other errors found.