[7.0.0-beta.2] Disk stays attached to Pool after its unassigned. Data loss

[meinereiner]

Hi !

Im not sure if this is a Bug, or something else....

I have a main data pool with 3 disks.
Parity - 18 TB WD
Disk1 - 18 TB WD
Disk2 - 18 TB WD

Because of a weird error, Disk1 got marked red and its contend emulated.
Didnt think too much about it,
started the Pool with the failed Disk1 unassigned.
Formated Disk1
Reassigned Disk1 to Pool
Started Parity Sync

Quote

text  error  warn  system  array  login  

Sep 11 15:34:05 schneuz kernel: sd 2:0:7:0: [sdh] 4394582016 4096-byte logical blocks: (18.0 TB/16.4 TiB)
Sep 11 15:34:05 schneuz kernel: sd 2:0:7:0: [sdh] Write Protect is off
Sep 11 15:34:05 schneuz kernel: sd 2:0:7:0: [sdh] Mode Sense: 46 00 10 08
Sep 11 15:34:05 schneuz kernel: sd 2:0:7:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA
Sep 11 15:34:05 schneuz kernel: sdh: sdh1
Sep 11 15:34:05 schneuz kernel: sd 2:0:7:0: [sdh] Attached SCSI disk
Sep 11 15:34:56 schneuz emhttpd: online: WDC_WD180EDGZ-11B9PA0_2GK4LAZU (sdh) 4096 4394582016
Sep 11 15:34:56 schneuz kernel: mdcmd (2): import 1 sdh 64 17578328012 0 WDC_WD180EDGZ-11B9PA0_2GK4LAZU
Sep 11 15:34:56 schneuz kernel: md: import disk1: (sdh) WDC_WD180EDGZ-11B9PA0_2GK4LAZU size: 17578328012 
Sep 11 15:34:57 schneuz emhttpd: read SMART /dev/sdh
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#873 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#873 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 26 08 00 00 00 3c 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549873216 op 0x1:(WRITE) flags 0x0 phys_seg 60 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#870 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#870 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 25 08 00 00 01 00 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549871168 op 0x1:(WRITE) flags 0x4000 phys_seg 256 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#828 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#828 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 26 44 00 00 00 3b 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549873696 op 0x1:(WRITE) flags 0x0 phys_seg 59 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#769 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#769 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 26 7f 00 00 00 3a 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549874168 op 0x1:(WRITE) flags 0x0 phys_seg 58 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#771 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#771 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 26 b9 00 00 00 29 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549874632 op 0x1:(WRITE) flags 0x0 phys_seg 41 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#773 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#773 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 26 e2 00 00 00 21 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549874960 op 0x1:(WRITE) flags 0x0 phys_seg 33 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#879 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#879 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8a 22 a3 00 00 00 3b 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1548817688 op 0x1:(WRITE) flags 0x0 phys_seg 59 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#775 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#775 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 27 03 00 00 00 60 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549875224 op 0x1:(WRITE) flags 0x0 phys_seg 96 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#876 UNKNOWN(0x2003) Result: hostbyte=0x05 driverbyte=DRIVER_OK cmd_age=10s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#876 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8a 21 fd 00 00 00 89 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1548816360 op 0x1:(WRITE) flags 0x0 phys_seg 137 prio class 0
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#784 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=DRIVER_OK cmd_age=0s
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#784 Sense Key : 0x5 [current] 
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#784 ASC=0x25 ASCQ=0x0 
Sep 13 05:44:39 schneuz kernel: sd 2:0:7:0: [sdh] tag#784 CDB: opcode=0x8a 8a 00 00 00 00 00 0b 8c 27 63 00 00 00 3f 00 00
Sep 13 05:44:39 schneuz kernel: I/O error, dev sdh, sector 1549875992 op 0x1:(WRITE) flags 0x0 phys_seg 63 prio class 0
Sep 13 05:44:41 schneuz kernel: sd 2:0:7:0: [sdh] Synchronizing SCSI cache
Sep 13 05:44:41 schneuz kernel: sd 2:0:7:0: [sdh] Synchronize Cache(10) failed: Result: hostbyte=0x01 driverbyte=DRIVER_OK
Sep 13 05:44:57 schneuz emhttpd: offline: WDC_WD180EDGZ-11B9PA0_2GK4LAZU (sdh) 4096 4394582016

** Press ANY KEY to close this window ** 

BUT 

Disk1 stayed somehow attached to the Pool.
So i formated it for real and wrote it to the Parity Disk
Whole Disk Data Lost.

FURTHER

Its reproducable.
I now removed Disk1 again, moved it to its own separate Pool.
But still if i copy Data to the Shares attached to the Pool, the Data is written to this Disk1, which isnt even attached to the Main Pool which contains the shares.
If i would Format it now again, it will probably delete Data again !?


Thanks in advance for guidance !
 

schneuz-diagnostics-20240914-2113.zip

[JonathanM]

31 minutes ago, meinereiner said:

Disk1 got marked red and its contend emulated.
Didnt think too much about it,
started the Pool with the failed Disk1 unassigned.
Formated Disk1

You formatted the emulated disk1, not the physical disk.

[meinereiner]

I formatted an unassigned device.

[JorgeB]

There's no point in formatting an unassigned disk and then re-assigning it to the array, since it will be re-written during the rebuild, to show the same as the emulated disk showed.

[meinereiner]

So it is expected behaviour that a Disk stays attached via hidden Links to a pool in the background, when its removed or unasigned or moved to an other/new pool.
And SMB shares access a Disk from an Pool it is not allowed to.
Im speechless, but OK now i know.

[JorgeB]

13 minutes ago, meinereiner said:

So it is expected behaviour that a Disk stays attached via hidden Links to a pool in the background, when its removed or unasigned or moved to an other/new pool.

I don't really understand what you mean, I just mentioned that there's no point in formatting and unassigned disk that is going to be added to the array, and the fact that you mentioned that, makes mew think that you may not fully understand how parity and emulated disks work, but please try to describe exactly what you did to see if I can follow.

[meinereiner]

2 problems, which i actually think are bugs are happening.

Problem 1:

I removed a Disk from the main Array (here where 3 disks)


to an new Pool


BUT
Unraid still thinks that this third Disk belongs to the Main Array



Here they are marked as "unprotected" which shouldnt be possible.
But it shows, because through some hidden Backlinks Unraid thinks the single unprotected Disk still belongs to the Main Array.

And Data also shows on the SMB Share. which shouldnt be possible,

 

[itimpi]

@meinereiner 

Your original post stated that you started the array with disk1 unassigned (which caused it to be emulated.

At that point you formatted disk1 which results in you formatting the emulated disk and updating parity to reflect this.    You would have gotten a warning before the formatting went ahead that this is NEVER part of standard recovery and is likely to lead to data loss, but apparently you continued anyway.

 

The next thing mentioned is formatting the physical drive while it was an unassigned device thus wiping its data as well.

 

it is possible a file recovery utility such as UFS explored might still be able to recover most of the data from the physical drive.

[JorgeB]

10 minutes ago, meinereiner said:

Unraid still thinks that this third Disk belongs to the Main Array

Shares are not automatically adjusted if you move disks around, they remain as they were, you need to go to the shares and correctly readjust the primary and secondary storage.

[meinereiner]

Problem 2

I started the Array with Disk 1 unassigned. So Disk 1 was emulated.

To make sure the Rebuild goes in the right direction, i removed the Partition from the now unassigned Device. 
Stopped the Array
Assigned Disk 1 again.
Started the Array and let the Rebuild do its thing.

No Formatting.... It was Partition Removing.... my Fault, sry 

[JorgeB]

Just now, meinereiner said:

To make sure the Rebuild goes in the right direction, i removed the Partition from the now unassigned Device. 

You don't need to do that, it won't make any difference.

[meinereiner]

2 minutes ago, JorgeB said:

Shares are not automatically adjusted if you move disks around, they remain as they were, you need to go to the shares and correctly readjust the primary and secondary storage.

and how ? the Config shows correct.

[JorgeB]

Just now, meinereiner said:

and how ? the Config shows correct.

Then why do you say

 

16 minutes ago, meinereiner said:

Unraid still thinks that this third Disk belongs to the Main Array

 

[meinereiner]

The Config shows correct.
But it clearly isnt, as told and shown

[JorgeB]

I'm not seeing anything wrong, why do you say disk3 is still included?

[meinereiner]

21 minutes ago, meinereiner said:

2 problems, which i actually think are bugs are happening.

Problem 1:

I removed a Disk from the main Array (here where 3 disks)


to an new Pool


BUT
Unraid still thinks that this third Disk belongs to the Main Array



Here they are marked as "unprotected" which shouldnt be possible.
But it shows, because through some hidden Backlinks Unraid thinks the single unprotected Disk still belongs to the Main Array.

And Data also shows on the SMB Share. which shouldnt be possible,

 

 

[JorgeB]

Click on "compute all" in that page and post a screenshot of the results

[meinereiner]

[JorgeB]

All shares still exist on the disk that now is a pool, single it's a single disk pool it will show as unprotected, you need to move/delete those shares from that disk, likely just the top-level folder.

 

Note that if cache is also a single device pool, it will also show unprotected for any shares that exist there.

 

 

[meinereiner]

Probably a good idea to remove the Partition to clear all the hidden Share Links.
Meanwhile killing a chicken and painting a Pentagram with Chickenblood so Cthulhu may help to not loose Main Array Data while removing Partitions 🤣
Thx for your Time, have nice Sunday

[itimpi]

6 minutes ago, meinereiner said:

Probably a good idea to remove the Partition to clear all the hidden Share Links.

Not quite sure what you mean by this.   Removing the partition would not normally have any effect - but then I do not understand what you mean by “hidden Shard links”.

 

[meinereiner]

And Removing the partition fixed it:

[JorgeB]

That's not a link, it means that folder existed on that device, shares are just top level folders, removing the partition will fixe it since by doing that you need to re-format the device, and thus deleting any existing folders.