replaced disk shows"Unmountable: No file system" after rebuild completed.


buzzra

13 posts in this topic Last Reply

Recommended Posts

Hello everyone,

I found disk 4 in my array had failed and was being emulated. I stopped the array and powered it off. I replaced the failed disk with another one of the same size. I powered on the array, chose the new disk as a replacement for the bad drive, and started the array. I noticed the unmountable error, but ignored it because the array was rebuilding. After the rebuild the array is up, parity shows valid and last check completed with no errors. The problem is that drive 4 still shows "Unmountable: No file system", and I still have the option to format the unmountable disk 4. All drives, including 4, show green.  I haven't done anything else. I don't know what was on the bad disk, so I can't tell yet if anything has been lost. 

Have I lost that data? How should I proceed? 

From reading other forum posts, I think the next step would be to put the array in maintenance mode and do a filesystem check, but I wanted to come here and verify first. The disks are formatted with reiserfs. I have attached diagnostics. The array has not been rebooted since the rebuild.

 

Thanks for any help,

buzz 

la_drone-diagnostics-20191006-1644.zip

Link to post
21 minutes ago, buzzra said:

next step would be to put the array in maintenance mode and do a filesystem check,

correct

21 minutes ago, buzzra said:

The disks are formatted with reiserfs.

You really should consider after fixing the problem switching the file systems to instead be xfs

 

Link to post

Thanks Squid. Running the reiserfsck --check now. The replacement disk was not new, but had not had any known errors in it's previous life. It only had a blank partition on it. Was I just lucky enough to use a bad drive to replace a bad drive? That would be par for the course 🙄. Also, do you think I've lost any data at this point if the drive gets repaired? I don't see it being emulated.

 

I have some new larger drives and am planning to upgrade parity then use the new disks to upgrade to XFS. Just need to get one of those 'round tuits' 😉

 

buzz 

Edited by buzzra
Link to post

Well that didn't take as long as I expected. Ended saying I need to run with the --rebuild tree option.

Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs.
Bad nodes were found, Semantic pass skipped
24 found corruptions can be fixed only when running with --rebuild-tree
###########
reiserfsck finished at Sun Oct  6 16:27:48 2019
###########

Any recommendations before I run it?

 

Link to post

Well, I started the array after the reiserfsck --rebuild-tree completed. The array is up and shows healthy. BUT...

 

There are a CRAPLOAD of files in the lost +found folder. A lot have names and are in folders but most are just files with numbers for names. Pretty much no way to figure out what they were. So in essence most of the data on that drive was lost.

 

Most likely it is because I replaced a bad drive with a bad drive, but there were no errors with the drive in it's previous life. I have replaced drives before in Unraid with used drives I had, but never had this problem. 

 

I wish there was some notification during the rebuild that the disk couldn't be written to or something. 

 

Thanks for all the help @Squid

 

buzz

 

Edited by buzzra
thank Squid for help
Link to post

Thanks @itimpi!

it works great, but knowing the file type is not helping much. I guess I could rename with the correct type and then open and see what each is. But that would take YEARS with the thousands of files in the lost+found folder that have no identification. So still pretty much lost. It is very frustrating knowing that the total used space on that drive is almost exactly what the original was, but it is 97% in the lost+found with no identification of any kind.  

Link to post

If the data is of the sort where it's not very important right now, but you could see a future where you want to spend the time to dig through it, then pull the drive and replace it. Keep the original, then copy the usable files off the rebuilt copy in the array to other disks and format it XFS to start the migration process.

Link to post

@jonathanm That's probably what I'm gonna do. It was all media files. I can get them again. I still need to sync a library and see what went away. 

 

Thinking about this, If a drive goes away and is being emulated, can I still copy the data from just that drive? Was there an emulated diks4 share? I have never looked to see if that was true when I've had one being emulated. 

 

 

Link to post
4 minutes ago, buzzra said:

@jonathanm That's probably what I'm gonna do. It was all media files. I can get them again. I still need to sync a library and see what went away. 

 

Thinking about this, If a drive goes away and is being emulated, can I still copy the data from just that drive? Was there an emulated diks4 share? I have never looked to see if that was true when I've had one being emulated. 

 

 

If you had disk shares enabled, then yes, the disk share shows up emulated. The contents are emulated regardless of the status of disk shares though, they show up exactly like nothing happened, assuming the drive dropped cleanly and parity was currently accurate. If parity is out of sync when a disk drops, the contents will be corrupt.

 

Zero error parity checks are critical.

Link to post

Yep, parity was good.

 

So a precautionary step you could take would be to copy just the emulated disk's data somewhere else before you replace it, in case something like this happens during rebuild.  Probably not necessary most of the time, but it is something you could do if you wanted to be extra sure not to lose data. I'll have to remember that, but I probably wouldn't have done it this time anyway. 🙄

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.