Unassigned Devices Preclear - a utility to preclear disks before adding them to the array


dlandon

Recommended Posts

1 hour ago, dlandon said:

Possibly, but you should check your cable.

after 4hrs, and not even getting through 1% of the pre-read, I killed that pre clear and added another drive in that spot. The pre clear on that is moving along at about 270 MB/s, with no errors, so I think the drive is just toast. I will try ite again after the current pre clear is finished.

Link to comment
  • 2 weeks later...
1 hour ago, Kilrah said:

Is the drive clear of all partitions? You cropped tight so can't see. If not clear them first.

 

1 hour ago, dlandon said:

Looks like UD sees a partition on the device.  Enable destructive mode and clear the disk (red 'X') next to the serial number.

 

I just removed it from the Parity, and reconfigured unraid to forget about it.

I have destructive mode on - but unfortunately no X is appearing for this disk:

1576134648_Screenshot2023-05-16220234.png.21f2b353ae128f3de9b02bff5c2f2311.png

 

EDIT: Just out of the blue the X appeared and I was able to clear the disk.

I'm now preclearing it (since the rebuilt of the Parity is still in progress).
Let's see when the parity is terminated if I'll be able to add it to the array 🙂 

 

Edited by TDA
Link to comment
3 hours ago, TDA said:

Let's see when the parity is terminated if I'll be able to add it to the array 🙂 

It’s up to you but there isn’t any requirement to preclear. Many people (including me) use preclear to test new drives. Since this drive isn’t new you might want to skip the preclear and just go ahead and add it. UnRAID will clear the drive as part of the process. Should save you a few hours.

 

Edit: AFTER PARITY REBUILD IS COMPLETED.

Edited by wgstarks
Link to comment

Hello,
I run into an issue where the script errors out and only the first command runs in the background (like pre-read or erase).
As far as I can determine its not the scripts fault but something with the filesystem/config.

See the log below where i want to erase a replaced hdd. The script finishes immediately due to the error seen. In the background the dd command which writes the zeros is running and when its done the part after "--> RESULT: .... " gets added to the log.


PRECLEAR LOG:

#   Step 1 of 1 - Erasing in progress ...                                                          #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
#                                                                                                  #
####################################################################################################
#       Cycle elapsed time: 0:00:00 | Total elapsed time: 0:00:00                                  #
####################################################################################################

####################################################################################################
#   S.M.A.R.T. Status (device type: default)                                                       #
#                                                                                                  #
#   ATTRIBUTE                   INITIAL STATUS                                                     #
#   Reallocated_Sector_Ct       0       -                                                          #
#   Power_On_Hours              47335   -                                                          #
#   Temperature_Celsius         29      -                                                          #
#   Reallocated_Event_Count     0       -                                                          #
#   Current_Pending_Sector      0       -                                                          #
#   Offline_Uncorrectable       0       -                                                          #
#   UDMA_CRC_Error_Count        83      -                                                          #
#                                                                                                  #
#                                                                                                  #
####################################################################################################
#                                                                                                  #
####################################################################################################
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 569: 0 * 100 /
        0 : division by 0 (error token is "0 ")
--> ATTENTION: Please take a look into the SMART report above for drive health issues.

--> RESULT: Erase Finished Successfully!.



31: /tmp/.preclear/sdb/dd_output_complete: No such file or directory
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 536: /tmp/.prec
lear/sdb/dd_output_complete: No such file or directory
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 531: /tmp/.prec
lear/sdb/dd_output_complete: No such file or directory
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 536: /tmp/.prec
lear/sdb/dd_output_complete: No such file or directory
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 531: /tmp/.prec
lear/sdb/dd_output_complete: No such file or directory
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 536: /tmp/.prec
lear/sdb/dd_output_complete: No such file or directory
/usr/local/emhttp/plugins/unassigned.devices.preclear/scripts/preclear_disk.sh: line 538: /tmp/.prec
lear/sdb/dd_output: No such file or directory



The error in line 569 is in the root_free_space() function, where it checks if enough space is available.
But unfortunately the is none, but I don't know why.

the line where it gets the 0 is     " avail=$(df --output=avail / | tail -n +2 )"
 

root@Prime:~# df / 
Filesystem     1K-blocks  Used Available Use% Mounted on
rootfs                 0     0         0    - /


I found that maybe a nfs share could mess things up, but its not enabled.

I restarted and checked with array stopped , but still the same.

I don't know where to continue. Any help/hint is appreciated :)

I'm running up to date unRaid (6.11.5) with a i think normal/standart setup (1 parity, 3 disks, 2 ssd)






 

Link to comment

Can you also post the preclear logs?  I've made a change so the divide by zero will not happen, but we need to find out why there is no available space.  Something is clobbering the rootfs.

 

Do the following:

  • Post the output of the 'ls -la /' command.
  • Reboot in safe mode.
  • Post the results of the 'df /' command.
  • If the command shows expected results, you'll need to find out which plugin, VM, or Docker Container is messing with rootfs.  You may have a misconfigured path.
Link to comment
root@Prime:/# ls -la /
total 8
drwxr-xr-x  21 root root    0 May 17 16:01 ./
drwxr-xr-x  21 root root    0 May 17 16:01 ../
drwxr-xr-x   3 root root    0 May 17 09:13 .config/
drwxr-xr-x   2 root root    0 May 17 09:13 bin/
drwx------  11 root root 8192 Jan  1  1970 boot/
drwxr-xr-x  17 root root 3560 May 17 09:17 dev/
drwxr-xr-x  59 root root    0 May 17 09:17 etc/
drwxr-xr-x   2 root root    0 Nov 20 22:27 home/
drwxr-xr-x   2 root root    0 May 17 09:12 hugetlbfs/
lrwxrwxrwx   1 root root   10 Nov 20 22:27 init -> /sbin/init*
drwxrwxrwx   8 root root    0 Mar 18  2020 lib/
drwxr-xr-x   7 root root    0 Nov 20 22:27 lib64/
drwxr-xr-x  13 root root    0 May 17 09:17 mnt/
drwx--x--x   3 root root    0 May 17 09:17 opt/
dr-xr-xr-x 418 root root    0 May 17 09:12 proc/
drwx--x---   3 root root    0 May 17 09:36 root/
drwxr-xr-x  18 root root 1000 May 17 09:38 run/
drwxr-xr-x   2 root root    0 May 17 09:13 sbin/
dr-xr-xr-x  13 root root    0 May 17 09:12 sys/
drwxrwxrwt  15 root root    0 May 17 17:38 tmp/
drwxr-xr-x  14 root root    0 Mar 29 17:05 usr/
drwxr-xr-x  15 root root    0 Jan  9  2017 var/

 

 

in safe mode

root@Prime:~# df /
Filesystem     1K-blocks   Used Available Use% Mounted on
rootfs           7990928 954976   7035952  12% /

 

 

so there has to be something messing with a plugin or docker container (no vm setup).

 

I disabled auto start of the array and turned of safe mode

 

root@Prime:~# df /
Filesystem     1K-blocks  Used Available Use% Mounted on
rootfs                 0     0         0    - /

 


since the array is offline... am I right to assume that no docker container is messing around and that i should look through my plugins?

 

Link to comment
1 minute ago, TDA said:

The disk is the only one which is not encrypted.

It will be formatted using the default filesystem set on Settings -> Disk settings, if it's still empty you can re-format with encryption, but now need to change the fs for that disk, by clicking on it with the array stopped.

Link to comment

Sorry to ask a noob question but how do I install this plugin while the array is stopped? I can't start it until I preclear my drive and while trying to install this via Apps it says:

Requirements not met

The requirements for this upgrade have not been met on your system

 

I am using unraidOS 6.9.2.

 

Edit: just noticed the requirements and it says Unassigned Devices plugin required. Installed and all good now.

Edited by CyberMew
Fixed my noob issue.
Link to comment

image.png.4fb76f05b323b6d1a36dbf3de2999ce6.png

 

For some reason I can't seem to satisfy this notification. For the last 5-6 months even after clicking the "x" on the notification, it inevitably comes back within a few hours if not sooner.

 

This last UD update I rebooted prior to updating, then again after updating, and yet, here we are!

Link to comment

When UD is installed, it applies a protective mount on the mount points it uses.  This only allows 1MB of data on the mountpoints.  If a Docker Containr or VM has an incorrect mapping to one of thee mount points, the mount point fills up and prevents any more writes so the ramfs is not filled.  The banner that you see is because UD detected something already mounted at one of it's mount points and wants a reboot to clear up the mount.

 

Do this:

  • Make sure all devices are unmounted in UD - disks, remote shares, and iso mounts.
  • Go to a command prompt and do an ls on '/mnt/didks/', /mnt/remotes/', /mnt/rootshare/', and '/mnt/addons/'.

If you see anything at any of the mount points, it must be removed so UD can complete an update.  The exception is '/mnt/addons/' that is intended for users to mount their unique stuff.  If you are mounting something there, be sure it does it's mount after UD has installed.

  • Like 1
Link to comment
16 hours ago, dlandon said:
  • Go to a command prompt and do an ls on '/mnt/didks/', /mnt/remotes/', /mnt/rootshare/', and '/mnt/addons/'.

If you see anything at any of the mount points, it must be removed so UD can complete an update.  The exception is '/mnt/addons/' that is intended for users to mount their unique stuff.  If you are mounting something there, be sure it does it's mount after UD has installed.

@dlandon

 

I shut down the array and ran the ls command as instructed.

 

The only item that's still "mounted" is my ZFS pool under /mnt/disks/

 

Do I need to unmount my ZFS pool as well? If so, do you happen to know the right commands to unmount and remount it? If not, no worries, I'll track down the original instructions I followed to set it up.

 

Also, once this is done do I just need to reboot? Or do I have to force the plugin to re-install followed by restart?

 

-K

Edited by v3life
Link to comment
1 hour ago, v3life said:

@dlandon

 

I shut down the array and ran the ls command as instructed.

 

The only item that's still "mounted" is my ZFS pool under /mnt/disks/

 

Do I need to unmount my ZFS pool as well? If so, do you happen to know the right commands to unmount and remount it? If not, no worries, I'll track down the original instructions I followed to set it up.

 

Also, once this is done do I just need to reboot? Or do I have to force the plugin to re-install followed by restart?

 

-K

Several issues with what you are doing:

  • A mount point has been created for these kinds of mounts.  Use /mnt/addons/.  It used to be recommended to use /mnt/disks/ so FCP wouldn't flag a problem with your mounts.  That's why UD creates the /mnt/addons/.  So users have a place for special mounts that do't interfere with UD.  UD adds some protection and FCP will not flag anything there as a problem.
  • Your ZFS mount needs to happen after the array is running and UD has been installed on a reboot.  Updates of UD won't flag the issue because the protection mount has already been applied at boot.
Link to comment
10 hours ago, dlandon said:

Several issues with what you are doing:

  • A mount point has been created for these kinds of mounts.  Use /mnt/addons/.  It used to be recommended to use /mnt/disks/ so FCP wouldn't flag a problem with your mounts.  That's why UD creates the /mnt/addons/.  So users have a place for special mounts that do't interfere with UD.  UD adds some protection and FCP will not flag anything there as a problem.
  • Your ZFS mount needs to happen after the array is running and UD has been installed on a reboot.  Updates of UD won't flag the issue because the protection mount has already been applied at boot.

 

Understood. I don't believe /mnt/addons/ was around when I set up ZFS back in '19... I appreciate the heads up. Once I prepare the pool for export it'll unmount and give me a window between preparing and upgrading to get the UD issue resolved once and for all.

 

Thank you for the support!

 

-K

Link to comment

I cleared my drive yesterday and I got this:

#                            Cycle 1 of 1, partition start on sector 64.                           #
#                                                                                                  #
#   Step 1 of 5 - Pre-read verification:                  [23:22:44 @ 213 MB/s] SUCCESS            #
#   Step 2 of 5 - Zeroing the disk:                       [23:23:36 @ 213 MB/s] SUCCESS            #
#   Step 3 of 5 - Writing Unraid's Preclear signature:                          SUCCESS            #
#   Step 4 of 5 - Verifying Unraid's Preclear signature:                        SUCCESS            #
#   Step 5 of 5 - Post-Read verification:                                          FAIL            #

 

Not sure if it happened while I knocked into the server case (doubt so though). How can I run Step 5 again without going through Step 1-4?

And if Step 5 fails again, should I run the whole preclear again? Or will it be grounds for an RMA (don't know what to write for a return)? FWIW I start in safe mode so there are no other activities going on  to interfere (drives are not mounted). Thanks for any advice in advance..

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.