I moved from IL to MI.

I moved the array with the flash drive still in it.

Skip ahead about a week and a half, I try to boot it up. No dice, USB drive is dead. None of my computers will recognize it.

I get a new USB drive, install the latest stable unraid on it (did not have a backup of my previous drive), get my Plus license sorted out so it can support the 5 drives I have in the system. Licensing is good to go.

 

I start up the system. Assign what I thought was the parity drive (sdb) to the parity drive slot. And assign the other 4 drives to the first four disks in the array.

 

Array starts up. Parity is invalid, 1st regular disk is unmountable (sdc). I choose to rebuild parity. Then I realize I shouldn't have done that since I have one drive that is not mountable. I think I swapped the parity for the 1st drive in the array (sdb & sdc). I tried unassigning those two drives and reassigning them to what I think is the correct position, but unraid tells me that's "wrong" on the 1st disk in the array.

 

So I wiped my USB drive and restored from a backup I created after I got my licensing figured out. I'm sitting here with the array down with sdc assigned to parity and sdb assigned to the 1st disk of the array. Should I try starting the array in this way and will it initiate a rebuild of disk 1 based on the parity I think is on sdc? (which is now assigned to parity)

Using those device letter designations isn't very useful, since they are not guaranteed to stay the same, especially if you make any disk changes. Unraid keeps track of the disk assignments using the serial numbers of each disk.

 

For future reference, you can always download a zipped copy of flash at Main - Boot Device - Flash - Flash Backup. As you can see, it is important to always have a current flash backup.

 

The biggest mistake here though was not asking for advice before doing anything. We would have told you to assign all disks as data and no disks as parity, then you could identify parity by the fact that it was unmountable.

 

I think there is a good chance you will have some problems even after getting the disks assigned correctly, but there isn't really any alternative at this point.

 

Were all the other disks mountable?

 

15 minutes ago, trurl said:

The biggest mistake here though was not asking for advice before doing anything.

Please don't do anything else without further instruction or you have a good chance of making things worse.

 

You will get further instructions after answering this question:

17 minutes ago, trurl said:

Were all the other disks mountable?

 

3 hours ago, trurl said:

Using those device letter designations isn't very useful, since they are not guaranteed to stay the same, especially if you make any disk changes. Unraid keeps track of the disk assignments using the serial numbers of each disk.

 

For future reference, you can always download a zipped copy of flash at Main - Boot Device - Flash - Flash Backup. As you can see, it is important to always have a current flash backup.

 

The biggest mistake here though was not asking for advice before doing anything. We would have told you to assign all disks as data and no disks as parity, then you could identify parity by the fact that it was unmountable.

 

I think there is a good chance you will have some problems even after getting the disks assigned correctly, but there isn't really any alternative at this point.

 

Were all the other disks mountable?

 

The other 3 drives (drives 2-4) were mounted and I was able to access the contents via the shares I had setup on my primary pc.

 

I have not done anything else since I posted my original message. 

 

Serial Numbers for the drives currently assigned to the array which has not yet been started are as follows:
Parity: WDC_WD80EFAX-68LHPNO_7SGG2ARC 8TB (sdc)

Disk 1: WDC_WD80EFAX-68LHPNO_7SGG00EC 8TB (sdb) (contained most data)

Disk 2: WDC_WD80EFAX-68LHPNO_7SGJVE9C 8TB (sde) (contained a small amount)

Disk 3: WDC_WD80EFAX-68LHPNO_7SGHU0VC 8TB (sdf) (contained a small amount)

Disk 4: WDC_WD10EZEX-08WN4A0_WD-WCC6Y0YXL4LJ 1TB (sdd) (was rather new and empty)

 

and I say how much data they held based on the fact that the one now assigned as sdc was not mountable. sdb, as now assigned, I'm assuming is partially overwritten with the parity data before I cancelled the parity rebuild.

Edited May 11, 20197 yr by thor79

OK. We can use the last 4 characters of the serial numbers if we need to clarify which disk we are talking about.

 

So is 2ARC the disk that you now believe to be parity? Or is it the one you had incorrectly assigned to parity?

2 hours ago, trurl said:

OK. We can use the last 4 characters of the serial numbers if we need to clarify which disk we are talking about.

 

So is 2ARC the disk that you now believe to be parity? Or is it the one you had incorrectly assigned to parity?

2ARC is the one I originally assigned to Disk 1, this is the one that was not mountable. Yes I believe 2ARC was the one that has my parity information. OOEC is the one that was originally assigned to parity.

Edited May 11, 20197 yr by thor79
clarifying a bit

Sorry life got in the way and nobody else picked this up while I was gone.

 

The assignments you want to make above are not the assignments Unraid has now, which is why it won't let you start the array. And it won't record the new assignments until the array is started.

 

But it is designed this way so people won't casually make changes to their disk assignments, possibly invalidating parity, or, as in your case, overwriting data with parity.

 

You have to jump through an extra hoop to get Unraid to allow you to make these changes. This is what New Config is all about.

 

New Config lets you assign the disks however you want, and after you have made those assignments, it will, by default, rebuild parity, since that is usually what is required to get valid parity when you have just assigned a bunch of disks from who knows where. There is, however, a checkbox that allows you to tell it parity is already valid.

 

What you want to do is get it to accept those assignments (New Config), but not rebuild parity, so you can use that parity to attempt a rebuild of disk1. There are 2 methods.

 

1.  Tell it to rebuild disk1 instead of rebuilding parity. This requires going to the command line to enter one command after making the assignments.
2. Tell it parity is already valid, then unassign and reassign disk1 to get it to rebuild disk1.

 

Method 1 is basically one step, whereas method 2 is a few steps, since at each step you must restart the array to get the assignments, unassignment, and reassignment to register. Both methods take about the same amount of time though, since in the end most of the time will be the actual rebuilding of disk1, which requires rewriting the entire disk from the parity calculation.

 

So I will let you think about that and decide which method you would prefer, and ask any questions you may have.

 

 

 

With method 2, it is actually possible to check the contents of the rebuilt disk1 before rebuilding it.

 

At the step where the disk is unassigned and the array started, that disk will be "emulated" from the parity calculation. This will allow you to see if the emulated disk will mount and even read the files on it. While emulated, Unraid will not be using the disk, it will be using the parity calculation from all the other disks.

 

And, the contents of the emulated disk is exactly what would be rebuilt when you reassign the disk and start the array.

 

8 hours ago, trurl said:

With method 2, it is actually possible to check the contents of the rebuilt disk1 before rebuilding it.

 

At the step where the disk is unassigned and the array started, that disk will be "emulated" from the parity calculation. This will allow you to see if the emulated disk will mount and even read the files on it. While emulated, Unraid will not be using the disk, it will be using the parity calculation from all the other disks.

 

And, the contents of the emulated disk is exactly what would be rebuilt when you reassign the disk and start the array.

 

Method 2 sounds good to me. Just preparing to head over to the house where the array is at. Just starting my day.

 

The Array was functioning just prior to being moved, so nothing should have happened to that parity data.

Edited May 12, 20197 yr by thor79

Let me know if you have any questions about any of this. Make sure that nothing writes anything to your array at least until the step when it begins to rebuild disk1, even better until it has completed the rebuild.

 

Go to Tools - New Config. Doesn't matter all that much what you choose from the dropdown. It is going to let you change any assignments you want anyway, the dropdown is just a way to get a starting place for making changes from.

 

After you make the assignments, but before you start the array, there will be a checkbox to tell it parity is already valid. Check the box and start the array. Then post a screenshot of Main - Array Devices.

 

8 minutes ago, trurl said:

Let me know if you have any questions about any of this. Be sure you don't write anything to your array at least until the step when it begins to rebuild disk1, even better until it has completed the rebuild.

 

Go to Tools - New Config. Doesn't matter all that much what you choose from the dropdown. It is going to let you change any assignments you want anyway, the dropdown is just a way to get a starting place for making changes from.

 

After you make the assignments, but before you start the array, there will be a checkbox to tell it parity is already valid. Check the box and start the array. Then post a screenshot of Main - Array Devices.

 

Here's a photo...best I could do, kind of new to the built in GUI which is what I'm using atm. Just easier for me since my network here is still in the works.

Process performed:

I had started the array in maintenance mode just to assign the drives.

I stopped the array.

I went to Tools->New Config, confirmed I wanted to do it. Clicked Apply

Assigned the drive assignments as I had outlined earlier, selected parity is valid checkbox.

Started the Array.

 

I'm guessing the reason why Disk 1 is unmountable currently is because of that short time it served as the parity disk before I shut down the parity rebuild. So if that's the case, that is expected.

OK.

 

Stop the array.

Unassign disk1.

Start the array with disk1 unassigned.

Post another screenshot.

 

12 minutes ago, trurl said:

OK.

 

Stop the array.

Unassign disk1.

Start the array with disk1 unassigned.

Post another screenshot.

 

Done.

 

Stopped the array.

Unassigned Disk 1

Started the array with Disk 1 unassigned.

 

Since no response recently, I went ahead with what I thought was the logical thing to do:

Stopped the array.

Assigned OOEC to Disk 1.

Started the Array.

 

It is now doing a Parity Sync/Data Rebuild...total size 8 TB. So it looks like it's rebuilding Drive 1 based on the parity.

If so, thank you for getting me as far as you did. I will backup my flash as soon as this completes.

 

Not touching the computer again until that completes. I do see that the disk contents are emulated, but I'm just going to leave it until it's done.

 

If the disk was showing as unmountable before the rebuild that will not be fixed by a rebuild.  You will need to run a file system check on the drive and if that reports corruption run the option to attempt to repair it.

Sorry for leaving this for a while, it's been a family sort of day.

 

 As itimpi said, repairing the filesystem may be required. In fact, your screenshot seems to say the emulated disk is unmountable. At that point you can either go ahead with the rebuild and then repair, or repair the emulated disk and then rebuild. Since you are rebuilding let it complete and we will see what is next.

 

I already mentioned the importance of having a flash backup and how to get one. I usually also mention the importance of having backups of any important and irreplaceable data. Many of us don't have the capacity to backup everything, but you absolutely must have another copy of anything important and irreplaceable. Parity is no substitute. Plenty of ways to lose data that don't involve a failed disk, including user error.

 

13 hours ago, trurl said:

Sorry for leaving this for a while, it's been a family sort of day.

 

 As itimpi said, repairing the filesystem may be required. In fact, your screenshot seems to say the emulated disk is unmountable. At that point you can either go ahead with the rebuild and then repair, or repair the emulated disk and then rebuild. Since you are rebuilding let it complete and we will see what is next.

 

I already mentioned the importance of having a flash backup and how to get one. I usually also mention the importance of having backups of any important and irreplaceable data. Many of us don't have the capacity to backup everything, but you absolutely must have another copy of anything important and irreplaceable. Parity is no substitute. Plenty of ways to lose data that don't involve a failed disk, including user error.

So the data-rebuild completed. Disk 1 was still unmountable. 

 

Following this page: https://wiki.unraid.net/Check_Disk_Filesystems

 

Stopped the Array

Started it in Maintenance Mode

Ran xfs-repair -v /dev/md1

It ran successfully in 25 seconds.

Stopped the Array

Started it up again in Regular Mode

 

Drive 1 now has a good xfs file system. No problems reported with the Array.

On the dashboard it Reports Disk1 as having 1.37 TB used, with a total of 1.41 TB used across the entire array.

 

But browsing Disk 1 only has a lost+found in it. It has only recovered a portion of the files as fragments. I guess now I have to try to recover what I can from those file fragments. Any tips? The wiki isn't particularly helpful...especially since the two links to "one user's story" are broken.

 

I know the importance of backups. That's why I normally have copies on my local drive (gaming rig), copies on my file server, and copies in the cloud for the irreplaceable data. The backup of the usb drive data got lost in the shuffle of my move. I'm in a tight spot in my life. Out of work, unable to pay bills, living off what my father provides. I just moved in with him. Paying for online storage is one of the last things I'm thinking about right now. I don't know what happened with the data on my gaming rig. I wish I could have a solid backed up everything. But I just can't afford it right now. I know parity isn't a backup (I don't count it as a backup, You don't have a backup if you don't have it in 3 places with one being off site), but I was hoping SOMETHING would go right with this move. Guess Not. The USB failure happened at pretty much the worst possible time.

 

Seems like there's still data stored on that Disk 1 as Unraid is reporting over a TB of data sitting on there....guess I'll try some data recovery tools to see if I can recover some files.

 

What I want to know is...did I do something wrong after the last instructions you gave @trurl?

Edited May 13, 20197 yr by thor79

7 minutes ago, thor79 said:

browsing Disk 1 only has a lost+found in it. It has only recovered a portion of the files as fragments. I guess now I have to try to recover what I can from those file fragments. Any tips? The wiki isn't particularly helpful...especially since the two links to "one user's story" are broken.

Since we moved the forum those links don't work as you have seen. The posts are probably in the forum somewhere but I don't have any suggestions for finding them.

 

Sorry no tips for dealing with lost+found. It is very tedious to go through if there is a lot of it.

 

13 minutes ago, thor79 said:

guess I'll try some data recovery tools to see if I can recover some files.

Some people have gotten good results with this:

 

https://www.ufsexplorer.com/

Your experience does make me wonder, though, if there might have been a different result had you used Method 1, so I am going to put this out there for @johnnie.black to comment on.

 

On 5/11/2019 at 11:10 PM, trurl said:

1.  Tell it to rebuild disk1 instead of rebuilding parity. This requires going to the command line to enter one command after making the assignments.
2. Tell it parity is already valid, then unassign and reassign disk1 to get it to rebuild disk1.

 

After you began the actual rebuild, though, it was to late to go back. I was hoping to see if the emulated disk was mountable before you did that.

1 minute ago, trurl said:

Your experience does make me wonder, though, if there might have been a different result had you used Method 1, so I am going to put this out there for @johnnie.black to comment on.

 

 

After you began the actual rebuild, though, it was to late to go back. I was hoping to see if the emulated disk was mountable before you did that.

Yeah sorry about that. I got impatient and figured skipping checking that would be fine. I'm sitting in the basement of my father's house with a temporary setup as I try to install the networking for the house. I kept checking for your reply, but after 3 hours I figured you went to bed or something and I didn't want to waste more time. Guess that was a bad move.

7 minutes ago, thor79 said:

I kept checking for your reply, but after 3 hours I figured you went to bed or something

No, we were celebrating Mother's Day here in the US Eastern Time Zone. I was out-of-town for several hours.

48 minutes ago, thor79 said:

But browsing Disk 1 only has a lost+found in it. It has only recovered a portion of the files as fragments. I guess now I have to try to recover what I can from those file fragments. Any tips? The wiki isn't particularly helpful...especially since the two links to "one user's story" are broken.

It can be useful to use the 'file' command on the files in lost+found which at least gives you the file types of the files present so that you can give them a valid extension.   It still leaves a large task ahead so you may still want to look at alternative ways of getting your files back.

26 minutes ago, trurl said:

Your experience does make me wonder, though, if there might have been a different result had you used Method 1, so I am going to put this out there for @johnnie.black to comment on.

If possible to do it method 1 is always the best option, method 2 should be OK in most cases but for best results array should be started in maintenance mode the first time, or there will always be some changes done to parity just by mounting the file systems, theses may or may not cause issues with the subsequent rebuild, but there's always a chance of filesystem corruption (and it's always fatal with for example btrfs because of transaction ids).