airic499 Posted September 15, 2013 Share Posted September 15, 2013 Using 5.0 ver 16c. Had a disk red ball. No problem, always have a pre-cleared disk ready to go. Spun the array down, replaced disk, spin up, blue ball , re build array, re-do parity check no errors, no problems. Go to access files from share and the share is empty. Mandatory panic attack, start checking individual disks, and everything is there and fine. Check other shares, all fine, just this one. Ok several searches here have suggested re-setting the ownership, so did that. Nothing. Other threads parity re-check. Again nothing. Cold restart, well several times in the process, but hey one more time, nothing. Ok, maybe I can just re-copy some of the files into the share see if that helps, nope won't let me. No real specific error just nothing. Try to add a folder to the share and I get this error: Could not find this item This is no longer located in <%3 Null:OpText>. Verify the item's location and try again. Ok at least an error message, but no ideas what to do. Probably wasn't the wisest thing to do, but tried to delete the share but it wouldn't let me. No error, but acted like when you try to delete a non-empty share (which I presume is exactly what it is). The share shows some or all of files in cache. Checked cache drive as others have indicated and nothing there. Can I turn off cache on this share and re-boot? No real need for cache as this is strictly a backup directory. I can go earlier, but here is the pertinent part of the log in the .txt file. The share in question is the movie iso backup one that is throwing the permission errors shown below. Sep 15 09:24:23 Tower2 emhttp: shcmd (94): /usr/local/sbin/emhttp_event disks_mounted Sep 15 09:24:23 Tower2 emhttp_event: disks_mounted Sep 15 09:24:23 Tower2 emhttp: shcmd (95): :>/etc/samba/smb-shares.conf Sep 15 09:24:23 Tower2 emhttp: shcmd (96): cp /etc/exports- /etc/exports Sep 15 09:24:23 Tower2 emhttp: shcmd (97): cp /etc/netatalk/AppleVolumes.default- /etc/netatalk/AppleVolumes.default Sep 15 09:24:23 Tower2 emhttp: Restart SMB... Sep 15 09:24:23 Tower2 emhttp: shcmd (98): killall -HUP smbd Sep 15 09:24:23 Tower2 emhttp: shcmd (99): ps axc | grep -q rpc.mountd Sep 15 09:24:23 Tower2 emhttp: Restart NFS... Sep 15 09:24:23 Tower2 emhttp: shcmd (100): exportfs -ra |& logger Sep 15 09:24:23 Tower2 emhttp: Restart AFP... Sep 15 09:24:23 Tower2 emhttp: shcmd (101): killall -HUP afpd Sep 15 09:24:23 Tower2 emhttp: shcmd (102): /usr/local/sbin/emhttp_event svcs_restarted Sep 15 09:24:23 Tower2 emhttp_event: svcs_restarted Sep 15 09:25:34 Tower2 afpd[1814]: dsi_tcp_open: invalid header Sep 15 09:26:03 Tower2 shfs/user: shfs_readdir: fstatat: Drama (13) Permission denied Sep 15 09:26:03 Tower2 shfs/user: shfs_readdir: readdir_r: /mnt/disk10/Movie ISO Backup (13) Permission denied Sep 15 09:26:03 Tower2 shfs/user: shfs_readdir: fstatat: Drama (13) Permission denied Sep 15 09:26:03 Tower2 shfs/user: shfs_readdir: readdir_r: /mnt/disk10/Movie ISO Backup (13) Permission denied Sep 15 09:26:03 Tower2 kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 0 does not match to the expected one 3 Sep 15 09:26:03 Tower2 kernel: REISERFS error (device md10): vs-5150 search_by_key: invalid format found in block 359209748. Fsck? Sep 15 09:26:03 Tower2 kernel: REISERFS error (device md10): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [15288 15289 0x0 SD] Sep 15 09:26:03 Tower2 kernel: REISERFS warning: reiserfs-5090 is_tree_node: node level 0 does not match to the expected one 3 Sep 15 09:26:03 Tower2 kernel: REISERFS error (device md10): vs-5150 search_by_key: invalid format found in block 359209748. Fsck? Sep 15 09:26:03 Tower2 kernel: REISERFS error (device md10): vs-13070 reiserfs_read_locked_inode: i/o failure occurred trying to find stat data of [15288 15289 0x0 SD] Sep 15 09:30:35 Tower2 afpd[1862]: dsi_tcp_open: invalid header Sep 15 09:35:35 Tower2 afpd[1863]: dsi_tcp_open: invalid header Tower2_Log.txt Quote Link to comment
itimpi Posted September 15, 2013 Share Posted September 15, 2013 If there was any sort of file system corruption, then the rebuild would not have fixed it. You might want to to try running a reiserfsck check of the problem disk. Hopefully you have still go the disk you removed? I would hang on to that in case you need to try and get data off it. Quote Link to comment
airic499 Posted September 15, 2013 Author Share Posted September 15, 2013 Thanks, yes I have it. As noted, the problem disk is removed from the box. It red balled and I replaced. I can stick it on the separate box and run reiserfsck check, if I need to salvage the files. But, this is my backup server so not worried too much about losing files, just don't want to re-coy 12 TB if I don't have too. So far though, all the files appear to be fine, just not mounted into the share. So the question is how do I get them back into the share? Alternatives are, add them to a new share, and remove them from the original so I can delete, or just blow away the old share? Second thought, looking at MC to see if I can copy to a new share that I have created. But the question is I see disks and can drill down to folders using MC, but not shares. I assume I need to navigate someplace other than mnt? Suggestions if that is a good or bad idea and if good where to copy files to. Thanks all. Quote Link to comment
itimpi Posted September 15, 2013 Share Posted September 15, 2013 You can run raiserfsck on the drive that has just been rebuilt. Put the array into maintenance mode and then run reiserfsck against the /dev/md? device where ? corresponds to the disk number. This would be much faster than trying to copy the data around (although it can still take some time). Strangely enough you can even do this against a failed/missing drive as it is emulated using the parity drive. As long as you are using the md devices any corrections are still written correctly to the emulated drive although I would expect performance to be lower due to the emulation overheads. Quote Link to comment
airic499 Posted September 15, 2013 Author Share Posted September 15, 2013 Thanks, itimpi. I put the array into Maintenance Mode. Telnetted in and typed reiserfsck /dev/md10 where is 10 is the replaced red balled drive. Got an reiserfsck message and it said type Yes to continue. I got the following returned after various steps. Looking up this flag now for syntax to run. finished 3 found corruptions can be fixed only when running with --rebuild-tree ...update, running with --rebuild-tree. Running hopefully things will get better shows 30,000 seconds (some 500 hours) remaining and rapidly increasing not decreasing. Quote Link to comment
airic499 Posted September 15, 2013 Author Share Posted September 15, 2013 Still rapidly increasing in time although blocks are going down. I hope/assume it will rip through things a bit faster at some point? Otherwise it would be a lot faster just to copy to a new share and see if that fixes the bad data. Quote Link to comment
airic499 Posted September 16, 2013 Author Share Posted September 16, 2013 Completing the loop, kudos to itimpi. Reiserfsck did the trick. I was afraid it was going to take about a month to finish, but once it got through the bad sections it wrapped up in about 7 hours. Files are back in share. Quote Link to comment
Fireball3 Posted September 25, 2013 Share Posted September 25, 2013 THX for sharing the result. Maybe mark the thread "solved"? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.