Pulled disk: no reaction

[Ancan 0]

Hi all,
I'm trying out Unraid as a platform for my new NAS, and pulled a disk to see the impact on the array.

 

I'm bit surprised actually, because I pulled the disk 30 minutes ago, and so far nothing has happened except a popup about twenty or so read errors. You would think completely removing a disk from an array would cause some distress, but it's still green in the interface and I can browse the structure (but not download). What's the expected behavious here really? Because I thought I'd get a big red warning about a dead disk.

 

[Conmyster 3]

8 minutes ago, Ancan said:

Hi all,
I'm trying out Unraid as a platform for my new NAS, and pulled a disk to see the impact on the array.

 

I'm bit surprised actually, because I pulled the disk 30 minutes ago, and so far nothing has happened except a popup about twenty or so read errors. You would think completely removing a disk from an array would cause some distress, but it's still green in the interface and I can browse the structure (but not download). What's the expected behavious here really? Because I thought I'd get a big red warning about a dead disk.

 

I haven't tested myself, however when removing a disk without shutting array down etc. It should detect this and then emulate the storage using parity data.

 

I would expect it to also show a notification stating something along the lines of "disk not detected"

Edited September 13 by Conmyster

[Ancan 0]

14 minutes ago, Conmyster said:

I haven't tested myself, however when removing a disk without shutting array down etc. It should detect this and then emulate the storage using parity data.

Nothing yet. Except a "*" on the disk temperature and up to 32 errors now. Still green otherwise. By now I ideally should have had a mail in my inbox and be on my way to the store for a replacement. Hmm...

 

 

Edit: I *love* how you can just paste images from the clipboard into the forum here (Ok, I somehow get double attachments when I do it but it 10 x beats saving the image and uploading it)

Edited September 13 by Ancan
PS

[Conmyster 3]

27 minutes ago, Ancan said:

Nothing yet. Except a "*" on the disk temperature and up to 32 errors now. Still green otherwise. By now I ideally should have had a mail in my inbox and be on my way to the store for a replacement. Hmm...

 

 

Edit: I *love* how you can just paste images from the clipboard into the forum here (Ok, I somehow get double attachments when I do it but it 10 x beats saving the image and uploading it)

If you setup the notification system under Settings > Notification Settings

 

Then it should of notified you of the errors on the drive... I would need to test myself though

 

Edit: After pulling my disk 5, I got a notification within 5 seconds (Discord) This is what my main screen looks like too:

 

After this I stopped the array, unassigned the device, started the array. Then stopped the array and reassigned the device and started the array.

 

Data is now rebuilding fine.

Edited September 13 by Conmyster

[Ancan 0]

24 minutes ago, Conmyster said:

If you setup the notification system under Settings > Notification Settings

 

Then it should of notified you of the errors on the drive... I would need to test myself though

 

Edit: After pulling my disk 5, I got a notification within 5 seconds (Discord) This is what my main screen looks like too:

 

After this I stopped the array, unassigned the device, started the array. Then stopped the array and reassigned the device and started the array.

 

Data is now rebuilding fine.

 

Kudos on commitment for pulling a drive!

I've got notifications set to "browser" only while I'm testing, and haven't seen anything except the "errors" popup. Disk still green in the GUI, even though it's sitting here on my desk.

I should mention that my paritity is not fully synchronized yet, and was not when I pulled the disk. I'm being cruel I know. I validate enterprise storage installations as part of my job, and am a bit damaged by that probably.

 

Edited September 13 by Ancan
wording

[Conmyster 3]

17 minutes ago, Ancan said:

 

Kudos on commitment for pulling a drive!

I've got notifications set to "browser" only while I'm testing, and haven't seen anything except the "errors" popup. Disk still green in the GUI, even though it's sitting here on my desk.

I should mention that my paritity is not fully synchronized yet, and was not when I pulled the disk. I'm being cruel I know. I validate enterprise storage installations as part of my job, and am a bit damaged by that probably.

 

Ah okay, I'm unsure on how it would deal with loosing a drive during a parity sync. As I'm aware most single parity hardware raid (say raid 5) would just die. 

 

And I don't have a system to test that with myself.

[Ancan 0]

Parity is now green as well. Just as green as Toshiba disk on my desk here beside me. Perhaps I should post on the bug-forum instead?

 

 

I'll stop the array now, and replace the Toshiba with a new disk instead. Let's hope Unraid will let go of it then.

 

 

Edit: About time!!!

 

 

Edited September 13 by Ancan

[Conmyster 3]

8 minutes ago, Ancan said:

Parity is now green as well. Just as green as Toshiba disk on my desk here beside me. Perhaps I should post on the bug-forum instead?

 

 

I'll stop the array now, and replace the Toshiba with a new disk instead. Let's hope Unraid will let go of it then.

 

 

Edit: About time!!!

 

 

Ah so likely due to the parity sync running.

 

Although I don't think that should be the correct course of action by unraid....

[Ancan 0]

Just now, Conmyster said:

Ah so likely due to the parity sync running.

 

Although I don't think that should be the correct course of action by unraid....

 

No I think there's room for improvement here. I bit worried about the lack of notifications.

[Conmyster 3]

2 minutes ago, Ancan said:

 

No I think there's room for improvement here. I bit worried about the lack of notifications.

Yeh I'm unsure why it waited for the parity sync for it to display a notification about the removed disk..

 

Would need a more experienced user with unraid to see why...

[Squid 1864]

It would take an access to the drive read or write for it to know it's not there

Sent from my NSA monitored device

[jonathanm 648]

While this behaviour is a little unsettling, and possibly should be modified, I can see why it probably happened. Parity is built walking the disk sectors in order. Since the disk you pulled was only a 500GB, and you were building to 6TB of parity, the disk was already not being read because the parity build process was beyond the 500GB mark. Its contents (and all the other drives to that point) were already written to parity and being emulated by the rest of the drives.

 

I suspect if you had read file contents or written to that drive, it would have immediately failed and warned you, but since all you did was browse the TOC that was most likely in RAM, no activity was actually asked of the disk, so unraid didn't know it was missing.

 

Since one of the main selling points of unraid is the ability to keep some drives spun down if they are not needed, I think some effort has been put into not actively poking drives just to be sure they are there.

 

Unraid only fails a drive when a write to it errors out, so it's quite conceivable that a failed drive could hang out in the array for some time without being noticed. Regularly scheduled parity checks (typically monthly) are a way to be sure little used drives are still capable of participating in a rebuild should they be called into action.

[jonathanm 648]

2 minutes ago, Squid said:

It would take an access to the drive read or write for it to know it's not there

So you wait 3 hours to snipe my post?🤣

[Squid 1864]

beat you by a minute