2 disks "unmountable wrong encryption key"

[tgggd86]

I left on vacation for 10 days and when I returned I noticed 2 of my disks in my array were disabled. I rebooted and when starting the array, 2 of my disks could not be mounted with the "unmountable wrong encryption key" error (these were the same disks that were disabled). I googled the error and found the below thread which hints at a memory error/issue. 

 

https://forums.unraid.net/topic/114191-solved-multiple-unmountable-disks-with-wrong-encryption-key-but-its-correct/

 

I've been running memtest 86+ on my server for 36 hours (7 passes) with 0 errors. So I assume memory is not the issue here.

 

I have dual parity so I'm assuming that I can just rebuild those 2 drives without any issues. But I want to make sure I prevent this issue from happening again and that I won't end up with data corruption if theirs another underling issue that hasn't revealed itself yet. So before I begin rebuilding my array looking for any advice/recomendations before I commit. Below are a couple things that I've noticed or done that may be a contributing factor.

 

- Upgraded my cache drive aprx. 3 weeks ago. Did not notice any issues beforehand and followed the "replace a cache drive" on the unraid wiki.

- Plex seemed to have "lost" many of my files located in various folders. Example: I had a playlist with 20 movies in it and it only remebered 2. 

      - I forced plex to rescan my folders and it began adding items that were already on the array as if they were "new".

- Log file when I first looked at it (before my initial restart) indicated a bunch of read errors on disks that were not disabled/emulated.

     - Those disk errors haven't popped up since restarting.

 

Thanks in advance!  Log files keep failing to upload. I'll try in a follow on post.

[tgggd86]

Diagnostics attached

tower-diagnostics-20211114-2029.zip

[JorgeB]

If it's not a RAM issue could be corrupt LUKS headers, do you have a backup of those?

 

In any case two simultaneously disable disks is usually not a disk problem, SMART looks OK for both and they are on a SASLP which is not recommended for Unraid and known to drop disks sometimes, so you might want to check if the actual disks mount with UD and if yes re-sync parity instead.

 

Also I wouldn't recommend encryption unless you absolutely needed it, it's just another layer of complexity, but if you keep using it make sure you backup the LUKS headers.

[tgggd86]

46 minutes ago, JorgeB said:

If it's not a RAM issue could be corrupt LUKS headers, do you have a backup of those?

 

In any case two simultaneously disable disks is usually not a disk problem, SMART looks OK for both and they are on a SASLP which is not recommended for Unraid and known to drop disks sometimes, so you might want to check if the actual disks mount with UD and if yes re-sync parity instead.

 

Also I wouldn't recommend encryption unless you absolutely needed it, it's just another layer of complexity, but if you keep using it make sure you backup the LUKS headers.

 

I do require encryption, but unfortunately did not back up my LUKS headers. Searched that and will do so once my array has been rebuilt.

 

If I can't mount my two drives due to encryption issues, I'm assuming I'll need to format them and then rebuild them using parity. I don't see how rebuilding parity is the best course of action. What am I missing? 

[JonathanM]

1 hour ago, tgggd86 said:

format them and then rebuild them using parity.

That will result in an empty fresh filesystem, that you will need to repopulate from your external backups. Is that what you intend to do?

[tgggd86]

8 hours ago, JonathanM said:

That will result in an empty fresh filesystem, that you will need to repopulate from your external backups. Is that what you intend to do?

 

That is not what I intend to do, but I do have a backup of my most critical data.

 

My understanding of dual parity, is that you can still recover data after two drives fail. Why is this situation different?

[JorgeB]

Because the problem is corruption, not failure, in this case looks like LUKS corruption, filesystem corruption can have similar results, i.e., parity won't help, but you can still try what I posted above, it should work.