2 SATA's fail at same under 6.9.2


Recommended Posts

Never had any problems with my 6 month old server.  Today I lost 1 of the parity drives and Disk 2 (both 14tb).   I am copying all my files as we speak onto my 2nd unraid server.  There is a red X on parity 1, and disk 2 says Unmountable: not mounted 

 

All of the drives were purchased in November/December 2021 - I don't know what happened.   I don't want to make the issue worse so i am asking for some assistance before i make a critical mistake.  Please let me know what to do next.

I have attached a screenshot of the drive array, not sure if this provides anything but let me know

 

image.thumb.png.b5f76607f0c44ba0dcc340c17a23f396.png

image.png

Link to comment

I'm just going to put a reminder here for some things we can look at later after you get your array fixed.

 

Why do you have 250G docker.img? Looks like 20G would be plenty for you. Have you had problems filling it?

 

And appdata, domains, system shares have files on the array.

Link to comment

Turl,

How do i check file sys on disk2, the drive is not showing up.  Never had any problems, this is all new for me to deal with.   Not sure why i 250g for docker.  I think someplace i read it was better to be too big then too small so i think that is the reason.   My main fear right now is many files are missing from the drive so i assume this was on disk2.  How do i get this back, i just want to copy it someplace safe (my other server).  

 

Initially, the server was froze up earlier today when i went to access it.   Couldnt get to any shares or anything so I restarted the PC and that is where i saw the disk problems.

Link to comment

Gorge, 😜

You check the filesystem on the emulated disk2 just the same as you would if the drive were not missing/disabled/emulated.

 

Looks like it was already in that state when you booted. Physical disk2 SMART looks OK.

 

Assuming repair of emulated filesystem goes well you will be able to access the files on the emulated disk2 and copy any important and irreplaceable files off the server. After repair you can rebuild disk2 and parity at the same time since you have dual parity.

 

If you have a spare to rebuild disk2 that would be best so you can keep original disk2 as is in case there are problems. But if repair goes well should be OK to rebuild on top.

 

Did you read the wiki link for check filesystem?

 

 

Link to comment
1 hour ago, GeorgeJetson20 said:

250g for docker.  I think someplace i read it was better to be too big then too small

Lots of bad advice floating around. You want it big enough, 20G looks like plenty for you as mentioned. If usage is growing you have something wrong and better to fix that than make docker.img huge.

 

Making it too big just keeps you from realizing you have problems for a while.

Link to comment

I did everything in disk filesystem.  stopped array.   Ran in Maint mode, xfs_repair -v  /dev/md2   (it ran and seems like it's ok) but disk2 is red X and 2nd parity is red X.   I still don't see all the data from disk2.   Do I spin it down (disk2) now???    All I have done is rebooted but disk2 is still not alive......   Not quite sure what to do.   

Link to comment

Running an xfs_repair on the emulated disk2 will not remove the disabled (red ‘x’) state.   Disabled status can only be removed by doing a rebuild.   However it can clear a disk showing as unmountable.

 

You did not mention whether the xfs_repair output indicated errors or if completed successfully.   If not have you restarted the array in Normal mode and does the emulated disk2 now mount?   If so is there a lost+found folder on it and if so how much content?   Posting new diagnostics might be a good idea.

Link to comment

Here are some screenshots of what it said.  It seems like the repair was ok.  I will do new diagnostics also.   YES -- LOTS OF FILES / FOLDERS IN LOST+FOUND --  But can't I just rebuild from the Parity Disk 1 -    (there are 4252 items) -

Snag_c097053.png.913f296a7231eeb353af01507100a218.png

I did what it said then ran it again and got the following at the end..

 

Snag_c09bc70.png.b012cf4ba7ca62dea8781b3d0dff1ece.pngtower-diagnostics-20220715-0018.zip

 

Here is the diag files.  After the xfs fix...

 

I REALLY APPRECIATE EVERYONES HELP WITH THIS..

 

Edited by GeorgeJetson20
Link to comment
8 hours ago, GeorgeJetson20 said:

can't I just rebuild from the Parity

  

The emulated disk is how the parity calculation already sees things. Rebuilding from parity will just write the contents of the emulated disk, which is exactly what you already have, the disk mounts but has lost+found, etc.

 

11 hours ago, trurl said:

If you have a spare to rebuild disk2 that would be best so you can keep original disk2 as is in case there are problems.

 

You can check the contents of the physical disk as an Unassigned Device to see if it is any better than the emulated disk contents.

Link to comment

lost+found is where things get put when repair can't figure them out. Typically they will be folders and files without their correct names, or any idea what folder they belong to. You can sometimes figure them out yourself but often not worth the trouble. The linux 'file' command might be able to give some clue what sort of contents a file has.

 

Emulated disk2 shows 5TB used, 

3 minutes ago, GeorgeJetson20 said:

next thing to do

6 minutes ago, trurl said:

check the contents of the physical disk as an Unassigned Device to see if it is any better

Stop the array, Unassign disk2, start the array with disk2 unassigned.

 

Then you can see if you can mount that disk with Unassigned Devices plugin to copy files from it and compare its contents with the emulated disk2 contents.

Link to comment
4 hours ago, trurl said:

Trurl,

why did have to be 5tb of data on that drive (lol)  - still in process of copying the lost and found.  80% can be identified inside the folders but it's a huge pia.     Any idea what could have happened that caused this or the file corruption?   I figured when it froze up and i rebooted, i did the damage there but never had any issues before this and only thing crazy running was docker for Unmanic but that is something i've used many times previously.   

 

When i stop array and remove disk2, its going to rebuild array at this point.  Then i should have both Parity drives back online?    Then after that occurs, i add disk2 back in and let it rebuild again?

 

Link to comment
42 minutes ago, GeorgeJetson20 said:

When i stop array and remove disk2, its going to rebuild array at this point.

No, both disk2 and parity will still be disabled. You have to reassign a disk to get it rebuilt.

 

What I am suggesting is unassigning disk2, try to mount it as an Unassigned Device, and see if the disk contents looks better than the emulated/repaired disk2. If so then you could copy its files, or just put it back in the array with New Config and rebuild both parity instead of rebuilding disk2.

Link to comment
43 minutes ago, GeorgeJetson20 said:

what could have happened

You should always get diagnostics before rebooting. Current syslog is in RAM like the rest of the OS, and that is the syslog that is included in diagnostics.

 

Everything else in diagnostics just tells us how things are currently, syslog might tell what happened earlier, but unless you have it saved somewhere with syslog server it doesn't survive reboot.

Link to comment

I get it, but when the system froze, i was not able to get the drive array, the web page, nothing.  Tried 2 different browsers.  The console was frozen also.      The dreaded rebuild of the array takes about 5-6 days but i really want to figure out what happened.  I will get a little piece of mind.  Going to do the unassign device and i will report back later

Link to comment
14 hours ago, GeorgeJetson20 said:

dreaded rebuild of the array takes about 5-6 days

The usual estimate is 2-3 hours per TB of parity unless you have controller bottlenecks or something is wrong.

 

Why do you have so many very large disks that are mostly empty? I recommend large disks, but not more than you need. Each additional disk requires more hardware, more power. Most importantly, each additional disk is an additional point of failure.

Link to comment

When i try to mount the Disk2 from unassigned, it says mounting for a SECOND then back to mount.   But i was able to copy all the lost+found and ID a good portion.   Why I have so much space.. Great question but got a Fractal Meshify 2 XL and wanted to use all the drive bays, lol.   I really just needed a big local storage/editing A LOT of 4k training videos and a very large sports collection/Concerts etc and before loading it up, i was giving it a running test for last 6 months to make sure the solution worked and became more trusting each month and got careless with some of the data as far as backups.

 

I only used new config one time so i hope i dont mess up but going to try to get both drives back today.   copied all the files off except some docker containers but have the whole appdata folder just in case.

Edited by GeorgeJetson20
Link to comment

I stopped array again, changed the disk assignment from disk 2 (still empty there) and put it in disk 14.  It's a doing clean disk now.  Is that what I should have done?   I looked at the new config but i didnt want to cause a problem so i skipped that for now.   Please advise what i should do.  Been up for hours in the middle of night getting the files from lost+found so not thinking clearly at all.

Link to comment
3 hours ago, GeorgeJetson20 said:

It's a doing clean disk now.  Is that what I should have done? 

No - what you have just done erases all its contents beyond any chance of revovery.

 

What was recommended was to instead try and mount it as an Unassigned Device as that might have mounted OK and you could then have copied files off it instead of trying to sort out the Lost+Found folder on the emulated disk.

Link to comment
11 minutes ago, itimpi said:

No - what you have just done erases all its contents beyond any chance of revovery.

 

What was recommended was to instead try and mount it as an Unassigned Device as that might have mounted OK and you could then have copied files off it instead of trying to sort out the Lost+Found folder on the emulated disk.   

I

I did do the unasssigned, see the previous message, but it wouldn't mount.  I clicked mount, it said mounting, but went right back to mount.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.