Unassigned Devices Preclear - a utility to preclear disks before adding them to the array

[dlandon]

19 hours ago, Top Gun said:

IS IT ANOTHER BAD DRIVE OR NOT ?   Thank you.

You have disk errors:

May  1 08:59:29 ENIGMA kernel: sd 15:0:4:0: attempting task abort!scmd(0x0000000086a55ffe), outstanding for 30047 ms & timeout 30000 ms
May  1 08:59:29 ENIGMA kernel: sd 15:0:4:0: [sdg] tag#1958 CDB: opcode=0x12 12 00 00 00 24 00
May  1 08:59:29 ENIGMA kernel: scsi target15:0:4: handle(0x000d), sas_address(0x4433221106000000), phy(6)
May  1 08:59:29 ENIGMA kernel: scsi target15:0:4: enclosure logical id(0x500605b0061b36c0), slot(1) 
May  1 08:59:29 ENIGMA kernel: sd 15:0:4:0: task abort: SUCCESS scmd(0x0000000086a55ffe)
May  1 08:59:29 ENIGMA kernel: sd 15:0:4:0: Power-on or device reset occurred
### [PREVIOUS LINE REPEATED 1 TIMES] ###
May  1 08:59:38 ENIGMA kernel: sd 15:0:4:0: attempting task abort!scmd(0x0000000086a55ffe), outstanding for 7072 ms & timeout 7000 ms
May  1 08:59:38 ENIGMA kernel: sd 15:0:4:0: [sdg] tag#1880 CDB: opcode=0x12 12 01 00 00 fe 00
May  1 08:59:38 ENIGMA kernel: scsi target15:0:4: handle(0x000d), sas_address(0x4433221106000000), phy(6)
May  1 08:59:38 ENIGMA kernel: scsi target15:0:4: enclosure logical id(0x500605b0061b36c0), slot(1) 
May  1 08:59:39 ENIGMA kernel: sd 15:0:4:0: task abort: SUCCESS scmd(0x0000000086a55ffe)
May  1 08:59:39 ENIGMA kernel: sd 15:0:4:0: Power-on or device reset occurred

and the SMART report:

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   079   064   044    -    87489136
  3 Spin_Up_Time            PO----   097   097   000    -    0
  4 Start_Stop_Count        -O--CK   100   100   020    -    3
  5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
  7 Seek_Error_Rate         POSR--   070   061   045    -    10455677

I'd say yes.

[Espressomatic]

Just updated UD to 05.01 and here we go again - pre-clear never gets beyond "starting" on any of 5 drives tested.

 

Quote

May  2 11:59:56 TVHD kernel: sd 3:0:6:0: [sdp] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
May  2 11:59:56 TVHD kernel: sd 3:0:6:0: [sdp] Write Protect is off
May  2 11:59:56 TVHD kernel: sd 3:0:6:0: [sdp] Mode Sense: 7f 00 10 08
May  2 11:59:56 TVHD kernel: sd 3:0:6:0: [sdp] Write cache: enabled, read cache: enabled, supports DPO and FUA
May  2 11:59:56 TVHD kernel: sdp: sdp1
May  2 11:59:56 TVHD kernel: sd 3:0:6:0: [sdp] Attached SCSI disk
May  2 12:01:46 TVHD emhttpd: ST2000DL003-9VT166_5YD1CCLE (sdp) 512 3907029168
May  2 12:01:46 TVHD emhttpd: read SMART /dev/sdp
May  2 12:03:49 TVHD unassigned.devices: Removing all partitions from disk '/dev/sdp'.

** Press ANY KEY to close this window **

 

Uninstalling and re-installing fixed it again. Something's seems wrong with the update process maybe?

Edited May 2 by Espressomatic

[dlandon]

40 minutes ago, Espressomatic said:

Just updated UD to 05.01 and here we go again - pre-clear never gets beyond "starting" on any of 5 drives tested.

 

 

Uninstalling and re-installing fixed it again. Something's seems wrong with the update process maybe?

When you uninstall UD, you also uninstall preclear and have to re-install both,  Not sure which one causes the issue.

 

Not sure which one you think you uninstalled and re-installed.  I need more information in order to be able to help you.

 

The log snippet is not helpful either.  It shows UD was asked to clear the disk and nothing else.

[Gico]

On 4/25/2024 at 2:11 PM, dlandon said:

I suppose it is possible you have a bad disk.

It's not. When I put it in my main machine as parity 2 and it went through a successful Parity Sync process.

Edited May 2 by Gico

[Espressomatic]

On 5/2/2024 at 1:09 PM, dlandon said:

When you uninstall UD, you also uninstall preclear and have to re-install both,  Not sure which one causes the issue.

Not sure which one you think you uninstalled and re-installed.  I need more information in order to be able to help you.

Unassigned Devices uninstalled. Both installed again (as you mentioned, because UDP is automatically removed).

 

Quote

The log snippet is not helpful either.  It shows UD was asked to clear the disk and nothing else.

 

I understand that, but I don't know where else to find/see any relevant information. If you have any suggestions (for next time) I'll make sure to try and get better diagnostic info. It'll be time to add another drive sooner rather than later, so I shouldn't so soon forget about this subject.

[foreseeable-concertina5279]

I attached an image, but have 3 disks connected via an external JBOD enclosure, and they all seem to have the same identification number, serial for all is (20170331000C3). This means I cannot start multiple concurrent preclears, as it seems to think all are active when I start on a single disk. 

 

Any ideas on ways to get around this? 

[dlandon]

2 hours ago, foreseeable-concertina5279 said:

Any ideas on ways to get around this?

Looks like a bug in that preclear thinks the serial number is the same for all three even though they are not.  You'll have to just preclear them one at a time by removing the other two disks.

[dlandon]

4 hours ago, foreseeable-concertina5279 said:

Any ideas on ways to get around this? 

Can you post ud_diagnostics?  Go to a command line and do 'ud_diagnostics' then post the /boot/logs/ud_diagnostics.zip.

[foreseeable-concertina5279]

8 hours ago, dlandon said:

Can you post ud_diagnostics?  Go to a command line and do 'ud_diagnostics' then post the /boot/logs/ud_diagnostics.zip.

Thanks for taking a look! DM'd it to you. 

[dlandon]

11 hours ago, foreseeable-concertina5279 said:

Thanks for taking a look! DM'd it to you. 

I see the problem.  I'll need to see what can be done about it.

[foreseeable-concertina5279]

On 5/15/2024 at 3:08 AM, dlandon said:

I see the problem.  I'll need to see what can be done about it.

Don't mean to be a bother, but any update here? They're 10TB disks so would like to do in parallel to save time. Otherwise I'm looking at a few days per disk.

 

Could I just start a bunch of tmux sessions and trigger the commands manually using the disk path?

[Kilrah]

If you continued doing them one by one since you posted that they should be done by now

 

Doing 3 at a time will most likely be bottlenecked by USB at least part of the time.

Edited yesterday at 07:37 AM by Kilrah

[foreseeable-concertina5279]

1 hour ago, Kilrah said:

If you continued doing them one by one since you posted that they should be done by now

 

Doing 3 at a time will most likely be bottlenecked by USB at least part of the time.

 

Haha I wish lol. Have had one going since I initially posted, its running the final cycle now. Its a USB 3.1 enclosure and they're HDDs, so USB shouldn't be a bottleneck. ~600 MBps is 4.8 Gbps

[dlandon]

5 hours ago, foreseeable-concertina5279 said:

Don't mean to be a bother, but any update here? They're 10TB disks so would like to do in parallel to save time. Otherwise I'm looking at a few days per disk.

 

Could I just start a bunch of tmux sessions and trigger the commands manually using the disk path?

In looking at the issue, a fix would be quite troublesome beause when the plugin is updated, the tracking of any existing preclear operations would potentially be lost and the preclear page would not be able to update the preclear operation  This also affects UD tracking the preclear operation.  I'm actually considering not doing anything here for the following reasons:

• As I mentioned, the tracking of any running preclears would be affected.
• This only seems to happen on USB bays and USB is really not robust enough for preclearing, especially multiple drives.

You can get around this problem you are having by installing the binhex preclear docker.  It will allow you to preclear by device.

[foreseeable-concertina5279]

11 hours ago, dlandon said:

In looking at the issue, a fix would be quite troublesome beause when the plugin is updated, the tracking of any existing preclear operations would potentially be lost and the preclear page would not be able to update the preclear operation  This also affects UD tracking the preclear operation.  I'm actually considering not doing anything here for the following reasons:

• As I mentioned, the tracking of any running preclears would be affected.
• This only seems to happen on USB bays and USB is really not robust enough for preclearing, especially multiple drives.

You can get around this problem you are having by installing the binhex preclear docker.  It will allow you to preclear by device.

Fair enough, using the docker now! Honestly probably a better solution anyways, thanks for the suggestion. Didn't realize they ran the same script.