February 22, 201412 yr I tried to delete a couple of files from the array but failed. /usr/bin/tail -f /var/log/syslog Feb 22 22:10:34 Tower kernel: REISERFS error (device md4): vs-5150 search_by_key: invalid format found in block 19732468. Fsck? Feb 22 22:10:34 Tower kernel: REISERFS error (device md4): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [1239 1244 0x0 SD] Feb 22 22:10:34 Tower kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 53100 does not match to the expected one 3 Feb 22 22:10:34 Tower kernel: REISERFS error (device md4): vs-5150 search_by_key: invalid format found in block 19732468. Fsck? Feb 22 22:10:34 Tower kernel: REISERFS error (device md4): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [1239 1243 0x0 SD] Feb 22 22:10:48 Tower shfs/user: shfs_unlink: unlink: /mnt/disk4/MOVIES/saab (2013)/saab (2013)-poster.jpg (30) Read-only file system Feb 22 22:10:48 Tower shfs/user: shfs_unlink: unlink: /mnt/disk4/MOVIES/saab (2013)/saab (2013).mp4 (30) Read-only file system Feb 22 22:10:48 Tower shfs/user: shfs_unlink: unlink: /mnt/disk4/MOVIES/saab (2013)/saab (2013).nfo (30) Read-only file system Feb 22 22:10:48 Tower shfs/user: shfs_unlink: unlink: /mnt/disk4/MOVIES/saab (2013)/saab (2013).srt (30) Read-only file system Feb 22 22:10:48 Tower shfs/user: shfs_unlink: unlink: /mnt/disk4/MOVIES/saab (2013)/saab (2013)-fanart.jpg (30) Read-only file system I had to stop the array and restart the array , then only I can delete files. After a couple of minutes, I can the same problem again with other files. I need to restart the array again. I am on Unraid 5.0.5
February 22, 201412 yr Author One more thing, each time I start the array I get this events in the log. /usr/bin/tail -f /var/log/syslog Feb 22 22:30:48 Tower emhttp: shcmd (539): exportfs -ra |& logger Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/Trance Videos does not support NFS export. Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/TV SHOWS does not support NFS export. Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/Music Videos does not support NFS export. Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/MOVIES SD does not support NFS export. Feb 22 22:30:48 Tower emhttp: shcmd (540): /usr/local/sbin/emhttp_event svcs_restarted Feb 22 22:30:48 Tower emhttp_event: svcs_restarted Feb 22 22:30:48 Tower emhttp: shcmd (541): /usr/local/sbin/emhttp_event started Feb 22 22:30:48 Tower emhttp_event: started Feb 22 22:30:49 Tower avahi-daemon[4142]: Service "Tower" (/services/smb.service) successfully established.
February 22, 201412 yr The reiserfs errors in the log indicate that there is file system corruption on the /dev/md4 drive. While such corruption is present then the disk is only available in read-only mode which can have unpredictable side-effects. The way to start attempting to recover is to put the array into maintenance mode and then from a telnet/console session run a reiserfsck using a command like: reiserfsck --fix-fixable /dev/md4 and then follow the onscreen prompts. If the reiserfsck finished with an error message then I would suggest report back here with the message and get guidance on the next step.
February 22, 201412 yr What itimpi said. Here's more info, should you need it: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems
February 22, 201412 yr Author Here is what I got from the telnet session. Tower login: root Linux 3.9.11p-unRAID. root@Tower:~# reiserfsck --fix-fixable /dev/md4 reiserfsck 3.6.24 Will check consistency of the filesystem on /dev/md4 and will fix what can be fixed without --rebuild-tree Will put log info to 'stdout' Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes ########### reiserfsck --fix-fixable started at Sun Feb 23 06:02:38 2014 ########### Replaying journal: Trans replayed: mountid 65, transid 16481, desc 4807, len 1, commit 4809, next trans offset 4792 Trans replayed: mountid 65, transid 16482, desc 4810, len 1, commit 4812, next t rans offset 4795 Trans replayed: mountid 65, transid 16483, desc 4813, len 32, commit 4846, next trans offset 4829 Replaying journal: Done. Reiserfs journal '/dev/md4' in blocks [18..8211]: 3 transactions replayed Checking internal tree.. \/ 2 (of 11// 83 (of 129-/ 57 (of 170\block 18841601: The level of the node (46388) is not correct, (1) expected the problem in the internal node occured/ 3 (of 11//123 (of 142/block 1973246 7: The level of the node (43804) is not correct, (2) expected the problem in the internal node occured (19732467), / 4 (of 11-block 1973246 8: The level of the node (53100) is not correct, (3) expected the problem in the internal node occured (19732468), whole subtrefinished Comparing bitmaps..vpf-10630: The on-disk and the correct bitmaps differs. Will be fixed later. Bad nodes were found, Semantic pass skipped 3 found corruptions can be fixed only when running with --rebuild-tree ########### root@Tower:~# Replaying journal: Trans replayed: mountid 65, transid 16481, desc 4807, len 1, commit 4809, next trans offset 4792 -bash: Replaying: command not found ########### root@Tower:~# Trans replayed: mountid 65, transid 16482, desc 4810, len 1, commi t 4812, next trans offset 4795 -bash: Trans: command not found root@Tower:~# Trans replayed: mountid 65, transid 16483, desc 4813, len 32, comm it 4846, next trans offset 4829 -bash: Trans: command not found root@Tower:~# Replaying journal: Done. -bash: Replaying: command not found root@Tower:~# Reiserfs journal '/dev/md4' in blocks [18..8211]: 3 transactions r eplayed -bash: Reiserfs: command not found root@Tower:~# Checking internal tree.. \/ 2 (of 11// 83 (of 129-/ 57 (of 170\b lock 18841601: The level of the node (46388) is not correct, (1) expected -bash: syntax error near unexpected token `(' root@Tower:~# the problem in the internal node occured/ 3 (of 11//123 (of 142 /block 19732467: The level of the node (43804) is not correct, (2) expected -bash: syntax error near unexpected token `(' root@Tower:~# the problem in the internal node occured (19732467), / 4 (of 11 -block 19732468: The level of the node (53100) is not correct, (3) expected -bash: syntax error near unexpected token `(' root@Tower:~# the problem in the internal node occured (19732468), whole subtre finished -bash: syntax error near unexpected token `(' root@Tower:~# Comparing bitmaps..vpf-10630: The on-disk and the correct bitmaps differs. Will be fixed later. -bash: Comparing: command not found root@Tower:~# Bad nodes were found, Semantic pass skipped -bash: Bad: command not found root@Tower:~# 3 found corruptions can be fixed only when running with --rebuild- tree -bash: 3: command not found
February 22, 201412 yr Run with rebuild tree. Starting with reiserfsck --fix-fixable is NOT advisable. Follow the instruction in the wiki carefully.
February 22, 201412 yr One more thing, each time I start the array I get this events in the log. /usr/bin/tail -f /var/log/syslog Feb 22 22:30:48 Tower emhttp: shcmd (539): exportfs -ra |& logger Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/Trance Videos does not support NFS export. Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/TV SHOWS does not support NFS export. Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/Music Videos does not support NFS export. Feb 22 22:30:48 Tower logger: exportfs: Warning: /mnt/user/MOVIES SD does not support NFS export. Feb 22 22:30:48 Tower emhttp: shcmd (540): /usr/local/sbin/emhttp_event svcs_restarted Feb 22 22:30:48 Tower emhttp_event: svcs_restarted Feb 22 22:30:48 Tower emhttp: shcmd (541): /usr/local/sbin/emhttp_event started Feb 22 22:30:48 Tower emhttp_event: started Feb 22 22:30:49 Tower avahi-daemon[4142]: Service "Tower" (/services/smb.service) successfully established. Only because no-one else commented on these I thought I would jump in. You can safely ignore these as a cause for concern. The problem is that NFS will not properly share folders with spaces in them. I had a similar issue with a share 'TV Shows'. I was still able to mount them via NFS though, so not a big deal. This only becomes difficult if you are trying to work with these folders in a shell (like Putty) as the spaces can cause a few headaches as you move around your drive. If you want to clear these up you can use the UnRAID gui to rename these shares to something without names. UnRAID will actually rename the folders on the drive for you as well (I renamed mine to TV). The only issue here is if you have clients referencing these shares you will need to update them with the new name. It looks like you have more serious issues going on which others are helping with, but I figured you might appreciate this information once everything else settles down.
February 23, 201412 yr Author I am currently doing the reiserfsck --rebuild-tree step now. After completing, if the disk4 is repaired, will the parity drive need to be rebuild to match the changes errors corrected on disk4? Or is the parity update automatically once I start the array afterwards ?
February 23, 201412 yr If you did the reisrefsck with the array in maintenance mode and against the /dev/md?? type device names then parity is automatically updated during the reiserfsck process. In fact it is even possible if you use the /dev/md?? type names to run the reiserfack against a disk that is missing/disabled as it is emulated, and then later replace the disabled disk and rebuild on to it. Having said that it would not do any harm to do a parity check after getting your drive back in case at any stage you managed to make a change that was not reflected by appropriate parity changes.
February 23, 201412 yr Or is the parity update automatically once I start the array afterwards ? The only time parity updates "automatically" is on each individual write to any disk in the array.
February 23, 201412 yr Author The reiserfsck rebuild tree finally completed but I forgot to capture the log. Anyway I rebooted the server and now I got a redball on the disk4. Shall I get a new harddisk for immediate replacement and see if I can recover data automatically?
February 23, 201412 yr Author Ahhhhh, rebooted the server into BIOS now and found out that the disk4 which was redballed is not detected in the BIOS now. Looks like its dead. Bad luck.
February 24, 201412 yr Author Today, I removed disk4 and attached it to a Windows machine using USB external cable. Successfully detected as USB HDD and formatted to NTFS. Removed the disk and reattached it to the server. Detected by the BIOS. Doing a preclear on UNRAID now. Hope all the data for this HDD can be rebuild after starting the array later on.
Archived
This topic is now archived and is closed to further replies.