Unassigned Devices Preclear - a utility to preclear disks before adding them to the array


dlandon

Recommended Posts

On 2/5/2024 at 3:06 PM, nick5429 said:

These messages are not generated for me when the plugin is "installed" - only on full reboot.

Update to the latest UD Preclear and test.  I can only do limited testing because I don't have a drive that gives a double quote in the family string.

Link to comment
On 2/7/2024 at 8:59 PM, dlandon said:

Update to the latest UD Preclear and test.  I can only do limited testing because I don't have a drive that gives a double quote in the family string.

Updated, and get no errors on boot with the new version. Haven't testing anything beyond that. But thanks!

Link to comment

Hi guys, first of all thanks for your amazing job DLandon.

I've a problem trying to preclear a bunch of disk. I used the plugin for the first disk and everything went as planned. now I tried already 2 more disk but the preclear is stuck at "starting" also after several hours. I tried to stop, reboot, format / clear the partition but nothing, I can't find a way to make it works.

 

any idea?

 

Swsn4nb.png

 

 

hSMiwcb.png

Edited by playmake
Link to comment

Hi guys,

first of all thanks the amazing plugin DLandon.

I found something strange while pre-clearing my SAS drives. I have never been able to use the resume option after reboot and today tried to figure out why.

Although I can resume my pre-clear and resume it from the web interface (see attached screenshots), it never gives me the option to resume after a reboot.

Reading up on this forum post I checked /var/log/preclear.log, which states that the pre-clear has been paused.

 

root@Unraid:/var/log/preclear# tail -f preclear.log 
Feb 10 10:12:07 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --notify 4 --frequency 4 --cycles 1 --skip-preread --skip-postread --no-prompt /dev/sds
Feb 10 10:12:08 preclear_disk_PBGK10SX_26298: Preclear Disk Version: 1.0.29
Feb 10 10:12:08 preclear_disk_PBGK10SX_26298: Disk size: 4000787030016
Feb 10 10:12:08 preclear_disk_PBGK10SX_26298: Disk blocks: 7814037168
Feb 10 10:12:08 preclear_disk_PBGK10SX_26298: Blocks (512 bytes): 7814037168
Feb 10 10:12:08 preclear_disk_PBGK10SX_26298: Block size: 512
Feb 10 10:12:08 preclear_disk_PBGK10SX_26298: Start sector: 0
Feb 10 10:12:10 preclear_disk_PBGK10SX_26298: Zeroing: zeroing the disk started 1 of 5 retries...
Feb 10 10:12:10 preclear_disk_PBGK10SX_26298: Zeroing: emptying the MBR.
Feb 10 11:45:09 preclear_disk_PBGK10SX_26298: Zeroing: progress - 25% zeroed @ 172 MB/s
Feb 10 11:45:54 Preclear paused on all devices
Feb 10 11:45:55 preclear_disk_PBGK10SX_26298: Pause requested
Feb 10 11:45:56 preclear_disk_PBGK10SX_26298: Paused
Feb 10 11:46:55 Preclear resumed on sds
Feb 10 11:46:56 preclear_disk_PBGK10SX_26298: Resumed
Feb 10 11:49:11 Preclear paused on all devices
Feb 10 11:49:11 preclear_disk_PBGK10SX_26298: Pause requested
Feb 10 11:49:11 preclear_disk_PBGK10SX_26298: Paused

 

Looking at the /boot/preclear_reports directory I can see a resume file has been made.

root@Unraid:/boot/preclear_reports# ls -lat
total 56
-rw-------  1 root root  599 Feb 10 11:49 5000cca03b1ef89c.resume

 

Content of that file looks good.

root@Unraid:/boot/preclear_reports# cat 5000cca03b1ef89c.resume 
# parsed arguments
format_html=''
no_prompt='y'
read_stress='y'
write_size=''
notify_channel='4'
skip_postread='y'
notify_freq='4'
skip_preread='y'
write_disk_mbr=''
cycles='1'
read_blocks=''
erase_preclear='n'
verify_zeroed=''
read_size=''
verify_disk_mbr=''
short_test=''
erase_disk='n'

# current operation
current_op='zero'
current_pos='1040277569536'
current_timer='5752'
current_cycle='1'

# previous operations
preread_average=''
preread_speed=''
write_average=''
write_speed=''
postread_average=''
postread_speed=''

# current elapsed time
main_elapsed_time='5762'
cycle_elapsed_time='5761'

 

Progress also shows that the pre-clear is paused.

 

####################################################################################################
#                          Unraid Server Preclear of disk 5000cca03b1ef89c                         #
#                            Cycle 1 of 1, partition start on sector 64.                           #
#                                                                                                  #
#   Step 1 of 3 - Zeroing in progress: (26% Done) PAUSED                                           #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#   ** PAUSED                                                                                      #
#                                                                                                  #
####################################################################################################
#       Cycle elapsed time: 1:36:02 | Total elapsed time: 1:36:03                                  #
####################################################################################################

 

When I tried another drive (in this case a SATA drive), resuming did work after reboot. But I noticed that for this drive, the name in the log file and the name of the resume file where the same. Which was not the case with the SAS drive as shown above.

root@Unraid:/boot/preclear_reports# tail -f /var/log/preclear/preclear.log 
Feb 10 12:08:06 Preclear script invoked as: /usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh  --notify 4 --frequency 4 --cycles 1 --skip-preread --skip-postread --no-prompt /dev/sdc
Feb 10 12:08:07 preclear_disk_Z1D22906_22098: Preclear Disk Version: 1.0.29
Feb 10 12:08:07 preclear_disk_Z1D22906_22098: Disk size: 1000204886016
Feb 10 12:08:07 preclear_disk_Z1D22906_22098: Disk blocks: 244190646
Feb 10 12:08:07 preclear_disk_Z1D22906_22098: Blocks (512 bytes): 1953525168
Feb 10 12:08:07 preclear_disk_Z1D22906_22098: Block size: 4096
Feb 10 12:08:07 preclear_disk_Z1D22906_22098: Start sector: 0
Feb 10 12:08:12 preclear_disk_Z1D22906_22098: Zeroing: zeroing the disk started 1 of 5 retries...
Feb 10 12:08:12 preclear_disk_Z1D22906_22098: Zeroing: emptying the MBR.
Feb 10 12:27:14 preclear_disk_Z1D22906_22098: Zeroing: progress - 25% zeroed @ 193 MB/s
Feb 10 12:27:43 Preclear paused on all devices
Feb 10 12:27:44 preclear_disk_Z1D22906_22098: Pause requested
Feb 10 12:27:45 preclear_disk_Z1D22906_22098: Paused



root@Unraid:/boot/preclear_reports# ls -lat
total 64
drwx------  2 root root 8192 Feb 10 12:27 ./
-rw-------  1 root root  598 Feb 10 12:27 Z1D22906.resume

After renaming the resume file to the name used in the logfile of the SAS drive. Resuming after reboot also works for the SAS drive.

 cp 5000cca03b1ef89c.resume PBGK10SX.resume

preclear running.png

preclear pause.png

preclear resumed.png

Edited by MvHeezik
Typo
Link to comment
  • 2 weeks later...

I just purchased a "used" Samsung SSD for use on my Dell R820. When I plug it in, the server sees it, Preclear states "Cannot Preclear". The logs show this:

 

{
    "sdm": {
        "ID_SCSI_SERIAL": "P0NA0H912234",
        "ID_SERIAL": "35002538a069d5590",
        "ID_SERIAL_SHORT": "5002538a069d5590",
        "ID_VENDOR": "SAMSUNG",
        "SERIAL_SHORT": "P0NA0H912234",
        "SERIAL": "35002538a069d5590",
        "BUS": "scsi",
        "FSTYPE": "",
        "SMART": "-d auto",
        "FAMILY": "SAMSUNG P1633N19 EMC1920",
        "MODEL": "SAMSUNG P1633N19 EMC1920 - Rev. EQPC",
        "FIRMWARE": "n\/a",
        "SIZE": 0,
        "SIZE_H": "0 B",
        "DEVICE": "\/dev\/sdm",
        "PARTS": [],
        "NAME": "sdm",
        "NAME_H": "sdm",
        "RUNNING": false,
        "TIMESTAMP": 0,
        "TEMP": "*",
        "SSD": true,
        "PRECLEAR": false
    }
}

 

Unraid is showing this:

Feb 21 14:48:57 Unraid-1 kernel: sd 1:0:13:0: [sdm] Synchronizing SCSI cache
Feb 21 14:48:57 Unraid-1 kernel: sd 1:0:13:0: [sdm] Synchronize Cache(10) failed: Result: hostbyte=0x01 driverbyte=DRIVER_OK
Feb 21 14:49:02 Unraid-1 kernel: sd 1:0:14:0: [sdm] Spinning up disk...
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Unsupported sector size 520.
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] 0 512-byte logical blocks: (0 B/0 B)
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] 4160-byte physical blocks
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Write Protect is off
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Mode Sense: b7 00 10 08
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Write cache: enabled, read cache: enabled, supports DPO and FUA
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Preferred minimum I/O size 8192 bytes
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Optimal transfer size 8192 bytes
Feb 21 14:49:29 Unraid-1 kernel: sd 1:0:14:0: [sdm] Attached SCSI disk
Feb 21 14:49:32 Unraid-1 emhttpd: device /dev/sdm has size zero

 

I stopped the array, and still no joy. Did I happen to get a bad drive, or is there a way to "wipe" the drive and make Unraid look at it again?

 

v/r

Jon

Link to comment
36 minutes ago, irishjd said:

Good point… Instead I am going try to “clear” the disk by wiping the MBR via a dd command. If it works, I will post the actual dd command here in case someone else ever has an issue like this.

UD should be able to clear the disk.

Link to comment

I think the disk might be bad. I ran the blkdiscard command on it and didnt have any errors. However when I go to MAIN-UD i see the drive listed (sdm), but it says its size is 0. If I try to add it to my Cache Pool Devices, it is not listed as available.

Link to comment
1 minute ago, irishjd said:

I think the disk might be bad. I ran the blkdiscard command on it and didnt have any errors. However when I go to MAIN-UD i see the drive listed (sdm), but it says its size is 0. If I try to add it to my Cache Pool Devices, it is not listed as available.

There have been some issues with nvme disks not seating properly.  If this is an nvme disk, power down, remove the device and re-install.  See if that fixes the issue.

Link to comment
  • 2 weeks later...

Anyone ever seen a blank dashboard page during preclear?  I started a preclear on a new 8TB drive two days ago.  When I login to the server the main Dashboard page is blank.  I can navigate to all other tabs just fine.  But the main Dashboard is blank.  The only thing that has changed on my server is starting the preclear.   

 

Capture.thumb.PNG.4d4bdfc848dd44e50e9669b52bde51f8.PNG

Link to comment
On 2/10/2024 at 1:33 AM, playmake said:

Hi guys, first of all thanks for your amazing job DLandon.

I've a problem trying to preclear a bunch of disk. I used the plugin for the first disk and everything went as planned. now I tried already 2 more disk but the preclear is stuck at "starting" also after several hours. I tried to stop, reboot, format / clear the partition but nothing, I can't find a way to make it works.

 

any idea?

 

Swsn4nb.png

 

 

hSMiwcb.png

 I had the same issue. After preclearing one drive, I installed two more but was unable to preclear them. They showed "Starting..." forever. The log showed the initial script invoke line but nothing after that.

I don't know the cause but I was finally able to get started by uninstalling and reinstalling the plugin.

Link to comment
52 minutes ago, treos33 said:

I don't know the cause but I was finally able to get started by uninstalling and reinstalling the plugin.

Next time try clicking on the double arrows on the preclear page and see if that doesn't fix preclear.

Link to comment
  • 2 weeks later...

I have been using the Preclear Tool on a couple of disks now which some success.  It's nice tool to determine which disk may be EOL. I usually print out a copy of the preclear preview log upon completion and include with the disk.

 

I'm just wondering if the preclear logs are available on the system *after* the disk has been unmounted?

Link to comment
2 hours ago, Pablo808 said:

I have been using the Preclear Tool on a couple of disks now which some success.  It's nice tool to determine which disk may be EOL. I usually print out a copy of the preclear preview log upon completion and include with the disk.

 

I'm just wondering if the preclear logs are available on the system *after* the disk has been unmounted?

Look in /flash/preclear_reports/.

  • Thanks 1
Link to comment

Hi- Trying to get the UD preclear to work, but running into issues. I was able to install the UD preclear plugin and it does show up when I go to Tools>Disk Utilities>Preclear Disk. I clicked on the drive shown (which is attached to my tower running Unraid 6.12.8 via a USB 3.0 enclosure). I select Start Preclear text, and configure the pop up window as shown in the screenshot.

 

583543907_Screenshot2024-03-18at17-11-04DLVTOWER_Preclear.thumb.png.31b21d55a8811f161abcf6bec0b21bff.png

 

1635768755_Screenshot2024-03-18at17-12-02DLVTOWER_Preclear.thumb.png.006c6e4444c1f0e76cc331eea9500d11.png

 

After that, it appears to stall as the status of the preclear never changes from Starting

 

1224429034_Screenshot2024-03-18at17-14-00DLVTOWER_Preclear.thumb.png.1e5f9fabc77af3bbbca27ab07bd13d35.png

 

Any pointers would be appreciated.

 

Thanks.

Link to comment
On 3/18/2024 at 7:15 PM, propman07 said:

I clicked on the drive shown (which is attached to my tower running Unraid 6.12.8 via a USB 3.0 enclosure).

USB is not generally robust enough to do reliable preclears.

Link to comment
5 hours ago, dlandon said:

USB is not generally robust enough to do reliable preclears.

 

Thanks for the reply. Does that imply that a device attached via unassigned devices as an example is not reliable to read/write data from/to?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.