October 16, 200718 yr hi guys, i got a bid of a problem on my hands. one evening i powered down as usual, nothing wrong with the array. that next morning i powered it up again, but disk1 showed up as unformatted and a parity sinc was in progress. i found some strange things in my syslog that i cant make heads of tails of. it seams to go wrong @ 10:55:48 does anyone have a solution, besides just format the drive? thx http://www.poulemania.nl/uplooi/temp/syslog.txt PS. why cant i upload a 34kb txt to the server? i get an error it cant upload contact admin.
October 16, 200718 yr hi guys, i got a bid of a problem on my hands. one evening i powered down as usual, nothing wrong with the array. that next morning i powered it up again, but disk1 showed up as unformatted and a parity sinc was in progress. i found some strange things in my syslog that i cant make heads of tails of. it seams to go wrong @ 10:55:48 does anyone have a solution, besides just format the drive? thx http://www.poulemania.nl/uplooi/temp/syslog.txt PS. why cant i upload a 34kb txt to the server? i get an error it cant upload contact admin. Don't re-format the disk. Unless the data on it is moved elsewhere first. Instead, try the "reiserfsck" command to attempt to fix any corruption that might exist on the drive's file-system.
October 16, 200718 yr A Wiki link for using reiserfsck on an unRAID drive is: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems. It mentions the --fix-fixable parameter, but there is also the --rebuild-tree parameter, which I suspect is what you will need. It will specify what you need after its first run. If you search the forum for reiserfsck, you will find a few other threads also. PS. why cant i upload a 34kb txt to the server? i get an error it cant upload contact admin. Attachments are not working currently. There are a couple of very worrisome issues here that I'd like to comment on, as they affect all of us with an unRAID system, and Tom will need to check this out. One is that unRAID called the drive 'Unformatted'. Corruption occurred in the syslog (in the 'md1: Removing' line) and in the reiserfs thread, and when it realized something was horribly wrong, it tried to clean up and quit, but invalid_op's appeared in the execution path and the kernel appears to take over and the thread terminates, with who knows what return code(s). We don't know why unRAID interpreted it to mean 'Unformatted', but Tom may want to have a look at the code here, to ensure that only a truly unformatted drive is so flagged. The second very worrisome issue is that it appears that the Reiser file system was not given enough time to finish its final housekeeping duties, before the server was powered off. Transactions were being replayed on all 3 drives, had finished a trivial 2 transactions on drive 2 and 3, and was working on node management on drive 1. Although the poster notes that "i powered down as usual, nothing wrong with the array", which is correct in that nothing would have appeared to be wrong, the Reiser system had not finished when the power went off. (I'm assuming he used the PowerDown button.) You really really really do not want btree management to be interrupted, even if 'journalized', and this appears to have happened. There are several... I'll write more when I get more time.
October 16, 200718 yr There are a couple of very worrisome issues here that I'd like to comment on, as they affect all of us with an unRAID system, and Tom will need to check this out. One is that unRAID called the drive 'Unformatted'. Corruption occurred in the syslog (in the 'md1: Removing' line) and in the reiserfs thread, and when it realized something was horribly wrong, it tried to clean up and quit, but invalid_op's appeared in the execution path and the kernel appears to take over and the thread terminates, with who knows what return code(s). We don't know why unRAID interpreted it to mean 'Unformatted', but Tom may want to have a look at the code here, to ensure that only a truly unformatted drive is so flagged. I have seen the unRaid software report drives as unformatted when they were unable to be affiliated with the "md" device for some reason. I have had this occur when I was shutting down the array and pressed the "refresh" button on the browser because I was impatient. At that point, all my drives appeared as unformatted even though they were perfectly fine. The management display was out-of-sync and confused. I agree, better analysis of the physical drive is needed before reporting a drive as unformatted.
October 16, 200718 yr I think the confusion stems from the actual word "Unformatted". What it really means in this context is "Not formatted as unRAID" expects. The difference is clear that unRAID shouldnt say a drive is unformatted when it actually is. At a minimum the use of this word should be abandoned for something clearer.
October 16, 200718 yr Just a disclaimer, especially to the veterans here, I tend to write for both veteran and newbie at the same time, and don't ever *intend* to be condescending. You may very well know more than me. It used to be, a format was a format, but then factory formatting arrived and the distinction between low-level formatting and file system installation, and partitions and MBRs and logical volumes, etc. I think Tom uses the word in the conventional sense of the state after low-level formatting, and partitioning, and before ReiserFS installation, but he adds a clearing step. He doesn't specifically test for cleared-but-unformatted state, as I found out once when I installed a new drive, waited for the clearing to finish, saw the Format button but wanted to change a cable or something first, so powered down and made the change, then booted and expected to see the array ready to format, but had to wait while it cleared all over again. Perhaps it would be better to hide the clearing step in the Format process. Formats used to be done that way anyway, as part of the search for bad sectors. I don't know Tom's code, but this and Joe's example almost seem like a Case and Default situation, as in Case: NoDrive, Case: GoodDriveButUnresponsive, Case: NoPartitioning, Case: GoodReiserFS, Case: ReiserInstalledButFaulty, Default: if none of the above, then call it Unformatted. As Joe indicated, 'Unformatted' should probably require specific confirmations first. In the case of this thread, perhaps a test could also be added to check that all of the reiser threads are still running and able to return some kind of Status_OK message. Another thought, from the syslog I'm not sure that the unRAID engine *knew* it was unformatted, as the unRAID server was in an unstable state, with ReiserFS in trouble, and possible stack and other memory corruption. It may have been the web management routine that decided to call it 'Unformatted', for lack of better state info, and made the decisions as to what options/actions to offer. that next morning i powered it up again, but disk1 showed up as unformatted and a parity sinc was in progress. X4n, I apologize if it seems I'm just talking behind your back, ignoring you. I would like to get more specifics as to what you saw, and the steps you took. Your statement above was probably an abbreviation of what happened, as it is not consistent with the syslog. According to the syslog, it finished booting at 10:55:48, then nothing happened for several minutes, then you Telnet'ed in from another station and copied the syslog. Normally when unRAID boots, if there's a major change in the array configuration, such as an invalid, missing, or unformatted disk, the array is Stopped, and appropriate options are shown on the Web management page. It does not automatically start a parity sync, until various actions have been taken. If you could elaborate, that would help, and any syslogs captured since might help. I would also like to ask you if you can remember what happened the night before, specifically what file operations involving the unRAID server immediately before it was powered down. This concerns my second worry, mentioned above, that the Reiser file system was not finished before power down. If you cannot remember any specific file operations (in particular, writes not reads, file saves or deletions or moves or renames), do you have automated backups or syncs from any of your stations to unRAID drive 1 that might have been in progress at the time of power down? ReiserFS is an advanced file system with better performance, data safety, and space management, but as in most things, the added complexity brings new kinds of problems. As I understand it, the file system is built on a btree system, which adds speed, but requires a little more maintenance. In addition, it actually stores some of the data WITHIN the tree, instead of just the links to the data. I personally find this exciting. It is similar to my own ideas for faster data access back when I was programming inverted btrees and bitstrings. But data stored only within the tree makes it definitely more scary if something goes wrong. Normally if an index goes bad, you delete it and build a new one, but in this case you cannot do that, because some of the data is embedded in the nodes of the tree. The journaling aspects of ReiserFS should help to recover from many problems, but there are some maintenance tasks like balancing or splitting or merging nodes that are very risky if interrupted. The Reiser file system is at least partly to blame here, because it should have anticipated every possible problem and data corruption, and recovered from it. But the file system should have been allowed to complete its operations, and that seems the fault of the Linux kernel.
October 16, 200718 yr hi guys, i got a bid of a problem on my hands. one evening i powered down as usual, nothing wrong with the array. that next morning i powered it up again, but disk1 showed up as unformatted and a parity sinc was in progress. i found some strange things in my syslog that i cant make heads of tails of. it seams to go wrong @ 10:55:48 does anyone have a solution, besides just format the drive? thx http://www.poulemania.nl/uplooi/temp/syslog.txt PS. why cant i upload a 34kb txt to the server? i get an error it cant upload contact admin. Are you using the 'Power down' button of the Management Utility, or some other method? The syslog shows telltale sign of a reiserfs corruption. As far as I'm aware, all such occurances have been fixed using 'reiserfsck'. I'll look into the attachment problem.
October 16, 200718 yr To Joe L., RobJ, and NASUser, Thank you for providing explanation and help. The term 'Unformatted' means that there was a mount error. Here's what happens: 1. The unraid driver is started, this results in devices /dev/md1, /dev/md2, ..., being created. Each of these devices corresponds to a physical data disk within the array. 2. An attempt to mount each 'md' device is made, e.g., 'mount -t reiserfs -o noatime,nodiratime /dev/md1 /mnt/disk1' 3. If the mount command succeeds, then obviously the disk is formatted with a reiserfs file system. If the mount does not succeed then the status is set to 'Unformatted'. Agreed, this is probably not the best way to handle this case. In X4n's case, the error in the syslog is a code error in reiserfs. I've spent some time trying to track this down and have posted to kernel mailing list, but have not received a satisfactory answer about it. I think it has to do with moving to 2.6 kernel when file system was created using 2.4 kernel. I'd try to ask mr. reiser directly, but he's "tied up" right now
October 17, 200718 yr I think possible bug might be http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.23.y.git;a=commit;h=2d3466a348a61c4d7f958ce80020eba17c09d7f7 or http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.23.y.git;a=commit;h=6f01046b35d940079822827498a7dd6d3eec8c6b After reading what has changed from 2.6.22.5 to 2.6.23.1 i think it would be nice to have unRAID with 2.6.23.1 kernel
Archived
This topic is now archived and is closed to further replies.