Linux file command

 

Thanks, having read the document you sent me to re: "Linux File command", I confess I'm not exactly sure what that might produce and/or how to execute it against an entire disk of data.  XFS_Repair has apparently reset every directory/sub-directory/named directory "inode" into a Numbered Directory (e.g. 8751517665) followed by changing its nlinks from e.g. "5 to 4" (297 Pages +/- worth)!

 

Since I have converted the entire disk to XFS, does XFS have a similar option to Reiserfs to view my data from perhaps a Windows or Mac based machine?  It might be somewhat easier to parse data/files/directories in a more "friendly" (or more easily understood) format. 

 

Obviously as a Linux or XFS noob I have absolutely no idea of what that means or how it affected the Data on my disk.

 

For a Linux/XFS Genius can these Lost+Found Data Files somehow be re-constituted into Directories with data?

 

Dave

I think I have seen mention of reading Linux filesystems in Windows with some utility. Google it or see tips in my sig to search the forum.

 

If you put the drives in another system and it modifies them in anyway, then you will invalidate parity.

Thanks Trurl, I've been searching on Google most of the day, but nothing seems to be able to "undo" the Lost+Found files.  I've even sent a note to Tom M.  While XFS may be better than Reiserfs, it seems much less user friendly for drive recovery.  I recall having a problem with one of my Pata drives years ago and was able to recover most of the data fairly easily with Reiserfs_check, then move it to another drive altogether.  From my review/research it seems that XFS_Repair seems to break the initial Directory/SubDirectory Header then everything is lost and placed into the numerical lists of Lost+Found.

 

Perhaps this is a Word of Warning to those changing Formats.  Reiserfs may not be as good, but its not that bad either especially for "idiots " like me trying to adjust their Servers!

 

Regards and Thanks Again for making the attempt to assist and to all others with suggestions.  If I find a solution aside from painstakingly looking through all 297 pages and manually restoring my Data, I'll be sure to Post It!

 

Dave

This whole episode reinforces the importance of BACKUPS 

 

But in the absence of backups, it also confirms something else I've been very critical of ... the "mad rush" to convert from Reiser to XFS => which I've noted many times is completely unnecessary for existing drives full of media.  [ http://lime-technology.com/forum/index.php?topic=37490.msg414717#msg414717 ]    I'd venture to say I've seen more data lost as a result of converting from RFS to XFS than for just about any other reason (i.e. drive failures) in the past few months.

 

Reiserfsck would almost certainly been FAR more successful in recovering the files you lost; and in addition could have likely recovered the originals on the old 1 & 2 TB drives you moved the data from, even though the program you used to move them had "deleted" them.    By the way, as I've noted many times [and outline in the persistent thread I wrote on backups:  http://lime-technology.com/forum/index.php?topic=31020.0 ]  it is not nearly as expensive as many think to maintain a good set of backups => you can simply use your older drives for that purpose ... i.e. if you'd simply kept the 1 & 2 TB drives intact instead of deleting the files from them, this would have been a minor annoyance instead of a major data loss situation.

 

This whole episode reinforces the importance of BACKUPS 

 

But in the absence of backups, it also confirms something else I've been very critical of ... the "mad rush" to convert from Reiser to XFS => which I've noted many times is completely unnecessary for existing drives full of media.  [ http://lime-technology.com/forum/index.php?topic=37490.msg414717#msg414717 ]    I'd venture to say I've seen more data lost as a result of converting from RFS to XFS than for just about any other reason (i.e. drive failures) in the past few months.

 

Reiserfsck would almost certainly been FAR more successful in recovering the files you lost; and in addition could have likely recovered the originals on the old 1 & 2 TB drives you moved the data from, even though the program you used to move them had "deleted" them.    By the way, as I've noted many times [and outline in the persistent thread I wrote on backups:  http://lime-technology.com/forum/index.php?topic=31020.0 ]  it is not nearly as expensive as many think to maintain a good set of backups => you can simply use your older drives for that purpose ... i.e. if you'd simply kept the 1 & 2 TB drives intact instead of deleting the files from them, this would have been a minor annoyance instead of a major data loss situation.

 

Gary, "to Quote the Raven, Nevermore!"  I certainly concur with your statement given my experience.  Perhaps the other issue is with Version 6 Plugins and Dockers requiring more horsepower, we find ourselves upgrading functioning Servers and in the process also upgrading our data repositories (drives).

 

As a further comment, I really appreciate Freddie's work on diskmv and consld8, they worked as described and wonderfully, but consld8 does move the data from the disk vs consolidating/copying the data to a new disk.  I've posted/sent a note to Freddie regarding my problem and your suggestion to maintain a backup of data after moving and/or consolidating it on a "new" disk regardless of format.

 

If you have any brilliant thoughts of how to REVERSE my error (ie go back in time  ) I would be most happy to use them!  Oh for a Tardis and Dr. Who!

 

Dave

Unfortunately my time machine broke a long time ago --- Just as I was ready to go back in time with a bunch of $$ and invest in all the x100 stocks of the last 40 years  :)

 

The one thing that likely would have helped has already been suggested by itimpi [New Config with "Parity is already valid" checked].  You discounted it because of the specific disk configuration that was in use ... HOWEVER =>  think CAREFULLY about exactly what you've done and just how it was accomplished.  A couple of questions ...

 

(a)  Was the utility you used to move your data run from within UnRAID?  If so, then parity would have been maintained during this process.

 

(b)  Has ANYTHING been written to ANY disk in the array other than the accidental writes to the 5TB data drive?  [The good thing about this is that the actual parity disk was assigned as a data disk during this process, so it was NOT written to.]

 

©  Do you know the EXACT configuration that you had before this all happened?    You don't actually need to know which disks were assigned to which slots => you just need to know which disk was parity (clearly you now do) and which disks were assigned as data disks.

 

=> If the answers to the above are Yes, No, and Yes, then there's hope.    You can indeed do what itimpi suggested and then you'll be able to rebuild your 5TB data disk.    That's as close as I can come to a time machine 

 

If you're going to do that, you want to do a couple of things.    First, do a New Config, but don't assign any disks.  Then go into the options and turn off Auto-Start (on the Disk Settings page).    Now assign the CORRECT parity disk and all of your data disks from the final configuration you had before, and check the "Parity is already valid" box.  Now Start the array -- and IMMEDIATELY cancel the parity check that starts.

 

Next, Stop the array and unassign the 5TB data drive (the one you know is corrupted);  then Start the array (so it shows as missing);  then Stop the array again and reassign it;  and then Start the array again -- and it will do the data rebuild.

 

IF the answers to the questions above were indeed Yes, No, and Yes (i.e. you didn't do anything you've forgotten about) then when the rebuild is completed you'll have all of your data back 

 

The real Bottom line, however, is that there's simply no substitute for backing up your data.

 

Was re-reading the thread and noted you used utilities that do indeed run within UnRAID, so the answer to my first question is clearly Yes.

 

So the real key to a complete recovery is the answer to the next two questions ... especially #2  [You could, in fact, actually try a couple different configurations if you're not certain about #3].

 

If you have ANY doubts about exactly what to do, be sure to ask => IF everything is okay for trying this, you need to do it exactly right or you'll likely lose the ability to do it.

 

Hi Gary, Thank you for the analysis and options.  Yes consld8 was run within UnRaid via command line and worked perfectly moving all the data to the 5TB Drive.  At that point I had a working Array with 2x1TB & 2x2TB drives plus the 5TB drive with my info.  I then stopped the Array and changed the formats on the 1&2 TB drives to XFS, started it up and formatted them and I think created an Array w/Parity? 

 

I then proceeded to create a new Config which is where the error occurred (both 5TB drives have almost identical ID's) and started the Array then immediately stopped it.  The remainder of the efforts have been performed singly.

 

If I understand your suggestion and Parity actually existed with the 2-1TB & 2TB Drives (without any data but reformatted in XFS) plus my 5TB drive, a proper sequence would perhaps allow it to be rebuilt.

 

Can you recommend the proper steps to try at this point?

 

Dave

Was re-reading the thread and noted you used utilities that do indeed run within UnRAID, so the answer to my first question is clearly Yes.

 

So the real key to a complete recovery is the answer to the next two questions ... especially #2  [You could, in fact, actually try a couple different configurations if you're not certain about #3].

 

If you have ANY doubts about exactly what to do, be sure to ask => IF everything is okay for trying this, you need to do it exactly right or you'll likely lose the ability to do it.

 

I think he should exclude any of the 5tb drives as they are larger than his original parity drive. That would cause a problem. So a rebuild without them should work.

Then after everything is restored to the original data drives he can replace the parity drive with a new 5tb parity drive. install his new 5tb data drives and rebuild again from the new 5tb parity drive.

 

 

I think he should exclude any of the 5tb drives as they are larger than his original parity drive. That would cause a problem. So a rebuild without them should work.

Then after everything is restored to the original data drives he can replace the parity drive with a new 5tb parity drive. install his new 5tb data drives and rebuild again from the new 5tb parity drive.

 

Thanks for the suggestion, but I don't think that will work as the 2TB Parity Drive (still intact & unchanged) used the 2x1TB's & 2x2TB's & 1x1.5TB all of which have been Re-Formatted to XFS.  If I understand Parity, it can rebuild 1 drive at a time by effectively adding/subtracting 0' & 1's and replacing the missing number odd/even.  Without the corresponding drives in place w/data in existence, I think that would never work - essentially trying to recreate all 5 drives+ at once.

 

I did have a good Parity (5TB Drive) with the 2x1TB Drives and 2x2TB Drives plus the 5TB Drive (now screwed up).  The question is "if" Parity was Valid after moving all the Data from the other drives to 1 of the 2TB drives (converted to XFS) and all of my Movies et. al to the 5TB Drive, then Re-formatting the 1 & 2 TB drives remaining.  They had no data before Re-formatting to XFS from Reiserfs but I don't know if the conversion would affect Parity?  My intent was to remove them from the Array which is where I screwed up!

 

Dave

Garycase/Itimpi/Trurl/Thestewman/et. al.,

 

I've given a lot of thought to your ideas and am trying a solution that only time will tell.  Before I screwed up I had created a "new" array w/o the 2x1TB drives.  It consisted of 2x2TB drives from before & my old Parity Drive (unchanged & unformatted) so when everything was complete, I had a 5TB Parity Drive, 2x2TB Drives and an Unformatted 2TB drive (old Parity) plus my 5TB data drive.

 

Not sure if this will work, but I followed Gary's suggestions and reconfigured the setup again, but replaced my 5TB (messed up) with another 5TB drive (never could get pre-cleared).  It showed a wrong config since it was not the same drive, so I did another reconfig and inserted the different 5TB drive.  I then started the array, stopped it, set the replacement drive no drive, started the Array - drive Missing, stopped it and started again with the Replacement Drive.

 

It's Rebuilding as I write this note, the question of course is whether it's the same as the messed up drive or has my Data at least partially intact been restored.  I absolutely don't trust the new Drive, but if it works at least temporarily, then I can move the data to one that is safe.

 

That's my latest Update, more after we see what transpires.

 

Dave

It's not exactly clear what you just did, but if you included all of the previous drives (2 x 1TB and 2 x 2TB) in the New Config so it exactly matched what you used to have, it should be correct.    As I noted before, you had to be CERTAIN you did this correctly ... but I'm not clear on exactly which drives you included.    Hopefully all is correct.    But even if it's not, if ALL you did was a New Config and then a rebuild, you could do it again as long as you don't write ANYTHING to the array ... so the only drive being written to is the one being rebuilt.

 

At the point you outlined here:  "I did have a good Parity (5TB Drive) with the 2x1TB Drives and 2x2TB Drives plus the 5TB Drive (now screwed up).  The question is "if" Parity was Valid after moving all the Data from the other drives to 1 of the 2TB drives (converted to XFS) and all of my Movies et. al to the 5TB Drive, then Re-formatting the 1 & 2 TB drives remaining.  They had no data before Re-formatting to XFS from Reiserfs but I don't know if the conversion would affect Parity?"

 

... you had a GOOD array with GOOD parity.    And if you did not write any additional data to those drives, and just did your New Config with the wrong drive assigned as parity, then the ONLY drive "messed up" was your 5TB drive that you incorrectly assigned as parity.

 

IF that's exactly what you did, it should in fact be doing a complete rebuild of your old 5TB data drive ... and you'll recover nearly ALL of your data.    The one very important thing is that you IMMEDIATELY stopped the parity check that starts when you Start the array after a New Config with the "Parity is already valid" box checked.    ANY parity "corrections" that get made during that check are actually messing up the ability to recover.  I REALLY wish LimeTech would provide a means to disable that check  (perhaps as a 2nd box option when you click the "already valid" box).

 

 

 

 

Hi Gary,

 

I did stop the the Parity Check, so we'll see what happens.  Once the rebuild is made do I need to stop the Array before it adds the drive back into the Array setup?

 

Since it's a different drive, I would like to be able to check it to see if I'm getting any of the old data, but don't want to jeopardize anything that's already been written.  Is there anyway to see if it's actually rebuilding my Data or do I have to wait until it's totally complete?

 

Dave

If you configured the array correctly, you won't have to do anything when the rebuild is done => you should simply be able to see the files on the disk with no problem.    If anything except that happens, STOP and don't do ANYTHING until you detail exactly what you're seeing and I have a chance to comment on it.

 

Hi Gary,

 

See the Pictures attached from overnight.  Apparently didn't work or disk4 (replacement) is "bad".  Note I had a problem with this disk prior to trying this as it would not Pre-Clear.

 

Dave

It appears you're having an issue with disk #3 as well.    If this has somehow been modified since the original config, you're simply out of luck -- you won't be able to rebuild.

 

The configuration however, is NOT what you had indicated it should have been.    You said there were 2 1TB drives, 2 2TB drives, a 5TB parity, and a 5TB data drive in the configuration when you moved the data around.  THAT is the configuration you should have set for this New Config.

 

As long as you have NOT made any changes to the drives involved, you can still do that.  If you have somehow written data to one of the 2TB drives then you're out of luck.    [but if that unmountable disk #3 is simply your old 2TB parity ... and was not one of the 2TB data drives that had been in the system, you should still be okay.]

 

The KEY is whether or not you've written ANYTHING to the array since the problem happened.    If the answer to that is No, then you should still be able to do the rebuild.

 

Just to be very clear ... if what I just said matches your original config, what you need to do is a New Config with the following assignments:

 

=>  The CORRECT 5TB parity drive set as parity

=>  The "other" 5TB drive (the one you messed up) set as a data drive

=>  The CORRECT set of 2 2TB drives assigned as data drives

=>  The CORRECT set of 2 1TB Drives assigned as data drives

 

Then you need to check the "Parity is already valid" box and Start the array -- IMMEDIATELY stopping the parity check that starts.

 

Then you Stop the array;  unassign the 5TB data drive; Start the array so it's shown as unassigned;  Stop the array and re-assign it; and then Start the array and let it rebuild.

 

If you have ANY doubts/questions, ask about them BEFORE you do anything.

 

Done w/2x1TB drives & 2x2TB drives plus the Replacement 5TB Drive rebuilding at this time.  Not exactly sure what I will get, but hopefully what you suggested. 

 

One of my 1 TB drives is still Reiserfs formatted even though the data had been moved to the 5TB drive.  So if this doesn't work perhaps your Reiserfs_check or may be able to "recover" some of my Movies/TV/Music though I'm not exactly sure where those folders all resided or were split across multiple other drives.

 

Now for a "backup ?".  Since I'm trying to rebuild/build a new Array and intend on replacing the smaller drives when/if I get my data back is there a way to use my "old Server" as a Backup?  If most of my Movies/TV/Music/Data were loaded on or in the old drives, the only issue would be new writes/data etc.  So can a Cache and /or External Drive being used as a Cache (for new data writes) send the same information over to two (2) UnRaid boxes at different times at night?  That way except for Parity, the data would exist in 2 locations on a daily basis.  Yes there is still the potential problem of not being off-site, but barring a major Flood/Fire/catastrophe it should serve to protect my information and work.

 

Dave

...  So can a Cache and /or External Drive being used as a Cache (for new data writes) send the same information over to two (2) UnRaid boxes at different times at night?...

This doesn't really make sense since a drive cannot be attached to more than one computer at a time, and it doesn't "send" anything unless a computer asks for it.

 

What you probably want is something like rsync where one computer sends the data to another computer over the network. Lots of stuff about this on the forum. I think there is even a docker. See search tips in my sig.

Just to be very clear ... if what I just said matches your original config, what you need to do is a New Config with the following assignments:

 

=>  The CORRECT 5TB parity drive set as parity

=>  The "other" 5TB drive (the one you messed up) set as a data drive

=>  The CORRECT set of 2 2TB drives assigned as data drives

=>  The CORRECT set of 2 1TB Drives assigned as data drives

 

Then you need to check the "Parity is already valid" box and Start the array -- IMMEDIATELY stopping the parity check that starts.

 

Then you Stop the array;  unassign the 5TB data drive; Start the array so it's shown as unassigned;  Stop the array and re-assign it; and then Start the array and let it rebuild.

 

If you have ANY doubts/questions, ask about them BEFORE you do anything.

 

Hi Gary,

 

Per your suggestion, I repeated the "Re-Build" again with the Replacement Drive.  It has apparently completed (see attachments) but is "unmountable".  Again I'm not sure if this is just a "bad" disk or everything is lost and can't be retrieved.

 

If you have any other suggestions, I'd be happy to give them a whirl.  Since the 1st 1TB drive is still formatted in Reiserfs, what tools would I use to perhaps recover some of those files?

 

edit - Is there anyway to check what has actually been written to disk5 since its unmountable?  That would at least let me see if there is anything useful?

Assuming that you indeed reconstructed the EXACT configuration I had told you to, then you're out of options with one minor exception.

 

If that's correct, then what apparently happened is that the automatic parity check that starts after a New Config with the "Trust Parity" option did too many "corrections" in the few seconds it ran to allow a successful reconstruction.    I've asked that this feature be changed precisely because of this issue => you're not the first person who's needed to reconstruct a disk due to some mistake ... and that automatic check really messes up the ability to do so.  [  http://lime-technology.com/forum/index.php?topic=43765.0 ]

 

... but it's not going to help you even if it's changed.

 

The one minor exception is that you MAY be able to run a Reiserfsck on the one disk you have that's still formatted with Reiser and recover the files that were on it.    Search the forum for detailed instructions on how to do that.

 

Whatever you do, be sure that going forward you maintain BACKUPS of your data !!    You can use the older (smaller) drives you're not going to use in the array anymore for this -- the 1TB and 2TB units.    Either set up a 2nd server with them and just do a sync between your primary and backup arrays;  or just use them as standalone backup drives ... copying everything you copy to the array to the "current" backup drive until it's full; then switching to the next one; etc.    If you do the latter, you can connect them with an external drive caddy [e.g. http://www.newegg.com/Product/Product.aspx?Item=N82E16817392022&cm_re=eSATA_dock-_-17-392-022-_-Product ] and then store them in "DriveBoxes" ... these work very nicely for this (I use them).    http://www.amazon.com/DriveBox-Anti-Static-Storage-3-5-inch-10-pack/dp/B004UALLPE

 

The one minor exception is that you MAY be able to run a Reiserfsck on the one disk you have that's still formatted with Reiser and recover the files that were on it.    Search the forum for detailed instructions on how to do that.

Even if the drive has been reformmatted with xfs, it's very likely that you can retrieve the reiserfs information using the reiser recovery tools and procedures. Reformatting only rewrites very small portions of the drive, the bulk of the data should still be retrievable. I tried to say that very early in the thread, but was completely ignored.

...  I tried to say that very early in the thread, but was completely ignored.

 

You weren't ignored at all 

 

You suggested it ...

 

... If they were reiserfs formatted, there is a good chance they can be fully rebuilt with a reiserfsck scan-whole-partition command.

 

... and the OP responded that they had all been converted to XFS:

 

... Sorry, but the drives were converted to XFS prior to removing most from the Array.

 

 

As it turns out, ONE of the drives had not, in fact, been converted => so there's a least a bit of hope for the data that was on that one.

 

Of course, if he could have done a successful rebuild of the 5TB drive it would have been FAR better, as he would have recovered ALL of his data => and THAT had to be attempted first, as once you do a Reiserfsck on the one RFS drive that remains, there would be NO chance of a rebuild.

 

At this point, a Reiserfsck on that one drive is the best option to at least recover SOME data.

 

Perhaps I could have been more clear. The current format, XFS or whatever, doesn't matter. If they were formatted with reiserfs when they were full of data, that data may be recoverable with the reiserfs tools.

...  I tried to say that very early in the thread, but was completely ignored.

You weren't ignored at all 

 

You suggested it ...

 

... If they were reiserfs formatted, there is a good chance they can be fully rebuilt with a reiserfsck scan-whole-partition command.

... and the OP responded that they had all been converted to XFS:

Converting to XFS doesn't matter, they "were reiserfs formatted" when they had the data in question on them. The recovery procedures still apply, no matter what the current format. I tried to say that and was shut down.

... The recovery procedures still apply, no matter what the current format. I tried to say that and was shut down.

 

r.e. "... I tried to say that and was shut down."  ==>  When??

 

Looking back through the thread, when the OP responded to your suggestion in this post:

 

Has anything been written to those drives since they were emptied? If they were reiserfs formatted, there is a good chance they can be fully rebuilt with a reiserfsck scan-whole-partition command.

 

Sorry, but the drives were converted to XFS prior to removing most from the Array.

 

All for naught!

 

... you NEVER replied pointing out that the reformat didn't matter until the post you made a couple hours ago (Reply #46).

 

One other key thing:  The drives haven't been modified since that time, so that is STILL an option !!

 

Archivist => Be sure to note that you may also be able to use Reiserfsck on the other drives as well ... you might just get VERY lucky with your missing data !!