[Support] binhex - Preclear


binhex

Recommended Posts

Trying to figure out the mail options. Would it be something like this?
preclear_binhex.sh -A -f -M 4 [email protected] /dev/sdX

And not sure what I need to do for this?

The -m, -M options requires that a valid working mail command is installed.
 
Ok I will take a look into that, for now let's ignore the mail setup, just want to confirm a good oreclear first. Can you try running it with the -A flag as per my previous post.

Sent from my CLT-L09 using Tapatalk

Link to comment
40 hours 37 minutes 3 seconds later...
========================================================================1.18== invoked as: /usr/local/bin/preclear_binhex.sh -f /dev/sdc== ST8000NM0055-1RM112   ZA1FS9VW== Disk /dev/sdc has been successfully precleared== with a starting sector of 1 == Ran 1 cycle==== Using :Read block size = 1000448 Bytes== Last Cycle's Pre Read Time  : 14:00:01 (158 MB/s)== Last Cycle's Zeroing time   : 12:18:43 (180 MB/s)== Last Cycle's Post Read Time : 14:17:20 (155 MB/s)== Last Cycle's Total Time     : 40:37:03==== Total Elapsed Time 40:37:03==== Disk Start Temperature: 31C==== Current Disk Temperature: 32C, ==============================================================================** Changed attributes in files: /tmp/smart_start_sdc  /tmp/smart_finish_sdc               ATTRIBUTE   NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS      RAW_VALUE     Raw_Read_Error_Rate =    83      82           44        ok          217697608         Seek_Error_Rate =    77      76           45        ok          55541351        Spin_Retry_Count =   100     100           97        near_thresh 0        End-to-End_Error =   100     100           99        near_thresh 0 Airflow_Temperature_Cel =    68      69           40        ok          32      G-Sense_Error_Rate =    99     100            0        ok          2432     Temperature_Celsius =    32      31            0        ok          32  Hardware_ECC_Recovered =    83      82            0        ok          217697608No SMART attributes are FAILING_NOW0 sectors were pending re-allocation before the start of the preclear.0 sectors were pending re-allocation after pre-read in cycle 1 of 1.0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.0 sectors are pending re-allocation at the end of the preclear,   the number of sectors pending re-allocation did not change.0 sectors had been re-allocated before the start of the preclear.0 sectors are re-allocated at the end of the preclear,   the number of sectors re-allocated did not change. ============================================================================

Also:

parted -lModel: ATA ST8000NM0055-1RM (scsi)Disk /dev/sdc: 8002GBSector size (logical/physical): 512B/4096BPartition Table: msdosDisk Flags: Number  Start  End  Size  Type  File system  Flags

So it does recognize that it's an 8TB drive. The partition table of "msdos" is a little disconcerting, the others are showing gpt. The others also show partitions under the table listing at the bottom and a file system of "xfs". I would presume at this point that these differences are due to the fact that the disk hasn't been formatted yet.
 
Any thoughts, comments or concerns, or should I just shove it in the array now with a variety of lessons learned?

The good news is as you can see the drive is healthy, yay! The bad news is I think if you added it as it is right now I think unraid would see it as a2TB drive due to the incorrect tabke type of msdos.

So if your are still game then please can you also try it with the -A option and we will see what happens.

I completely understand if you just want to switch back to the plugin for now and get that drive added, choice is yours.

Sent from my CLT-L09 using Tapatalk

Link to comment
2 hours ago, binhex said:

So if your are still game then please can you also try it with the -A option and we will see what happens.

We'll see what happens...

 

The drive does not show up in Unassigned Devices - is that to be expected? I was going to look there to see what it thought of the drive, but no love.

Link to comment
We'll see what happens...
 
The drive does not show up in Unassigned Devices - is that to be expected? I was going to look there to see what it thought of the drive, but no love.
It's possible unassigned devices has detected that the precleared drive has a array signature on the drive and thus it ignores the drive, it's a guess though as I don't know the inner workings of unassigned devices.

Sent from my CLT-L09 using Tapatalk

Link to comment
4 hours ago, binhex said:

 
Ok I will take a look into that, for now let's ignore the mail setup, just want to confirm a good oreclear first. Can you try running it with the -A flag as per my previous post.

Sent from my CLT-L09 using Tapatalk
 

Started it yesterday evening with -A and -f. Pre-read completed successfully. Currently at 2% on zeroing. This is on an 8TB drive.

  • Like 1
Link to comment

Interesting observation.  I have both Binhex_Preclear and Binhex_Krusader installed.  If either one is 'Started', the other one will error out when you attempt to start it with this message:

 

image.png.68830db9b32b36b2ce297fd3c0cdfbd4.png

 

'Stopping' the first one will allow the other one to start.  Apparently, you can not have two running instances of noVNC running at the same time.  

 

 

Link to comment
Interesting observation.  I have both Binhex_Preclear and Binhex_Krusader installed.  If either one is 'Started', the other one will error out when you attempt to start it with this message:
 
image.png.68830db9b32b36b2ce297fd3c0cdfbd4.png
 
'Stopping' the first one will allow the other one to start.  Apparently, you can not have two running instances of noVNC running at the same time.  
 
 
Yes you can check host port

Sent from my CLT-L09 using Tapatalk

Link to comment

@binhex

Tried starting another cycle using -M or -m flags. Wouldn't work. Just shows the help menu when I try to enter the command. Guessing because there is no unmenu mail package maybe??? I know you want to keep this docker very basic without the bells and whistles, but I really think notifications would be a basic feature IMHO. Just a thought.

Link to comment
@binhex

Tried starting another cycle using -M or -m flags. Wouldn't work. Just shows the help menu when I try to enter the command. Guessing because there is no unmenu mail package maybe??? I know you want to keep this docker very basic without the bells and whistles, but I really think notifications would be a basic feature IMHO. Just a thought.

I will take a look at notifications, hopefully I will be able to pass through the mail client to the container

 

Did you by any chance run parted -l to confirm the partition was the correct size and type?

 

Sent from my CLT-L09 using Tapatalk

 

 

 

Link to comment

Even before the post-read completed, Unassigned Devices is reporting:

image.png.e8b541af79f10e5e63dd4bdfa93f89b7.png

I'd take that as a positive sign!  :D

 

It seems that the `-A` parameter made the difference.

 

This time it was 40:44:43. Sooooooo much slower. ;)

========================================================================1.18
== invoked as: /usr/local/bin/preclear_binhex.sh -f -A /dev/sdc
== ST8000NM0055-1RM112   ZA1FS9VW
== Disk /dev/sdc has been successfully precleared
== with a starting sector of 1 
== Ran 1 cycle
==
== Using :Read block size = 1000448 Bytes
== Last Cycle's Pre Read Time  : 14:02:00 (158 MB/s)
== Last Cycle's Zeroing time   : 12:17:22 (180 MB/s)
== Last Cycle's Post Read Time : 14:24:22 (154 MB/s)
== Last Cycle's Total Time     : 40:44:43
==
== Total Elapsed Time 40:44:43
==
== Disk Start Temperature: 31C
==
== Current Disk Temperature: 32C, 
==
============================================================================
** Changed attributes in files: /tmp/smart_start_sdc  /tmp/smart_finish_sdc
                ATTRIBUTE   NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS      RAW_VALUE
      Raw_Read_Error_Rate =    66      83           44        near_thresh 3626020
          Seek_Error_Rate =    79      78           45        ok          74098720
         Spin_Retry_Count =   100     100           97        near_thresh 0
         End-to-End_Error =   100     100           99        near_thresh 0
  Airflow_Temperature_Cel =    68      69           40        ok          32
      Temperature_Celsius =    32      31            0        ok          32
   Hardware_ECC_Recovered =    66      83            0        ok          3626020
 No SMART attributes are FAILING_NOW

 0 sectors were pending re-allocation before the start of the preclear.
 0 sectors were pending re-allocation after pre-read in cycle 1 of 1.
 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.
 0 sectors are pending re-allocation at the end of the preclear,
    the number of sectors pending re-allocation did not change.
 0 sectors had been re-allocated before the start of the preclear.
 0 sectors are re-allocated at the end of the preclear,
    the number of sectors re-allocated did not change. 
============================================================================

All looks good to me, but  what does "near_thresh" mean:

      Raw_Read_Error_Rate =    66      83           44        near_thresh 3626020
         Spin_Retry_Count =   100     100           97        near_thresh 0
         End-to-End_Error =   100     100           99        near_thresh 0

And out would these counts be near threshold this early in the disk's life? Is that something to be concerned about? They're the exact same numbers as in the report from the 1st preclear. (Well, the first one to complete...)

 

Actually, I just realized that the "Raw_Read_Error_Rate" was OK after the first run and is now "near_thresh" after this 2nd run. I've added it to the call-out. Is that something to be concerned about?

Edited by FreeMan
Link to comment
12 hours ago, FreeMan said:
Even before the post-read completed, Unassigned Devices is reporting:
image.png.e8b541af79f10e5e63dd4bdfa93f89b7.png
I'd take that as a positive sign! 
 
It seems that the `-A` parameter made the difference.
 
This time it was 40:44:43. Sooooooo much slower.

========================================================================1.18== invoked as: /usr/local/bin/preclear_binhex.sh -f -A /dev/sdc== ST8000NM0055-1RM112   ZA1FS9VW== Disk /dev/sdc has been successfully precleared== with a starting sector of 1 == Ran 1 cycle==== Using :Read block size = 1000448 Bytes== Last Cycle's Pre Read Time  : 14:02:00 (158 MB/s)== Last Cycle's Zeroing time   : 12:17:22 (180 MB/s)== Last Cycle's Post Read Time : 14:24:22 (154 MB/s)== Last Cycle's Total Time     : 40:44:43==== Total Elapsed Time 40:44:43==== Disk Start Temperature: 31C==== Current Disk Temperature: 32C, ==============================================================================** Changed attributes in files: /tmp/smart_start_sdc  /tmp/smart_finish_sdc               ATTRIBUTE   NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS      RAW_VALUE     Raw_Read_Error_Rate =    66      83           44        near_thresh 3626020         Seek_Error_Rate =    79      78           45        ok          74098720        Spin_Retry_Count =   100     100           97        near_thresh 0        End-to-End_Error =   100     100           99        near_thresh 0 Airflow_Temperature_Cel =    68      69           40        ok          32     Temperature_Celsius =    32      31            0        ok          32  Hardware_ECC_Recovered =    66      83            0        ok          3626020No SMART attributes are FAILING_NOW0 sectors were pending re-allocation before the start of the preclear.0 sectors were pending re-allocation after pre-read in cycle 1 of 1.0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.0 sectors are pending re-allocation at the end of the preclear,   the number of sectors pending re-allocation did not change.0 sectors had been re-allocated before the start of the preclear.0 sectors are re-allocated at the end of the preclear,   the number of sectors re-allocated did not change. ============================================================================
 

All looks good to me, but  what does "near_thresh" mean:


      Raw_Read_Error_Rate =    66      83           44        near_thresh 3626020        Spin_Retry_Count =   100     100           97        near_thresh 0        End-to-End_Error =   100     100           99        near_thresh 0
 

And out would these counts be near threshold this early in the disk's life? Is that something to be concerned about? They're the exact same numbers as in the report from the 1st preclear. (Well, the first one to complete...)
 
Actually, I just realized that the "Raw_Read_Error_Rate" was OK after the first run and is now "near_thresh" after this 2nd run. I've added it to the call-out. Is that something to be concerned about?

Ignore thos values, it's a Seagate issue with smart, the one to watch are reallocated, pending sector and crc error count.

 

edit - see here http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html

and also:-


Sent from my CLT-L09 using Tapatalk

Edited by binhex
  • Thanks 1
Link to comment
On 9/29/2019 at 6:29 PM, wgstarks said:

I know you want to keep this docker very basic without the bells and whistles, but I really think notifications would be a basic feature IMHO. Just a thought.

good progress on this, im going to test my changes with a kicked off preclear on my (tortured) 250GB drive, if all good then i will release a new version with working notifications sometime thursday i would guess.

  • Thanks 1
Link to comment
11 hours ago, FreeMan said:

I'm going to say that the new docker is a success! The only thing you may want to recommend is using the -A flag on larger (i.e. currently normal sized) drives.

So i was expecting the -A flag to work, as i have been specifying this flag myself whenever i have been preclearing drives using this preclear script (this is prior to my mods mind you). and it always works, even for drives larger than 2TB.

 

So reading up on the whole sector boundary fiasco i have found out the following:-

 

If your drive uses Advanced Format (generally used for drives larger than 2TB) and your partition does NOT start on a physical sector boundary that aligns ( a multiple of 8, e.g. 64) then you will get a write performance hit for this, taken from a WD whitepaper:-
 

Quote

When misalignment occurs, a logical 4K block will reside on two physical sectors. In this case, a single read or write of a 4K block will result in a read/write of two physical sectors. The impact of a “read” is minimal, whereas a single write will cause two “Read-ModifyWrites” to occur, potentially impacting performance.


So unless im missing something here (a real possibility hehe) then setting the start sector boundary to '1', as the preclear plugin i THINK does by default, will mean a write penalty, as its not a multiple of 8 and thus not aligned (see whitepaper quote above). I have read that the performance hit with unaligned GPT partitions is 'minimal', so maybe not of too much concern.

 

For this script going forward i am going to set it to use a start sector boundary of 64, which will result in it being aligned, i.e. flag -A will be the default.

 

link to WD whitepaper:-

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/white-paper/white-paper-advanced-format.pdf

Edited by binhex
Link to comment

OK email notification looks good, so the image is currently building, once its ready please pull down. if you want to use email notification then you will need to add in the following additional volume mounts (new users AFTER 2nd Oct 2019 will NOT have to do this), please copy the host and container paths EXACTLY as they are in the screenshots below:-

 

image.png.18d46f24f24b917a4c96e1d6bc28f0c0.png

 

and also another additional volume mount:-

 

image.png.3f58d02ed5d07f0cac0085ee91b70d13.png

 

then run the preclear script with the -M option to get notification of progress via email.

 

NOTE as long as you have setup email notification on unraid then you will NOT need to specify any email configuration or recipient email address(es).

 

Edited by binhex
Link to comment
4 hours ago, binhex said:

For this script going forward i am going to set it to use a start sector boundary of 64, which will result in it being aligned, i.e. flag -A will be the default.

Thanks for digging into it - this seems like a good option.

 

To the uninformed and/or uninitiated, "start on sector boundary 64" instead of "start on sector boundary 1" screams "OhMerGersh I'm not getting full use of my disk!!!". Considering it's an 8TB drive, even if that is true, it's a minor loss at worst...

 

11 minutes ago, binhex said:

OK email notification looks good, so the image is currently building, once its ready please pull down

This sounds really handy and I'll be sure to use it for my next preclear! I know you're not after bells and whistles, so let's just call this "a random glitter that worked its way in, cause that's what glitter does". :)

Edited by FreeMan
  • Like 1
Link to comment
4 hours ago, binhex said:

For this script going forward i am going to set it to use a start sector boundary of 64, which will result in it being aligned, i.e. flag -A will be the default.

I’m a little dense sometimes, so want to be sure I’m understanding this correctly. Does this mean we no longer need to set the -A flag when we run the script?

Link to comment
7 minutes ago, wgstarks said:

I’m a little dense sometimes, so want to be sure I’m understanding this correctly. Does this mean we no longer need to set the -A flag when we run the script?

correct, the default is now start sector of 64, see the FAQ notes section at the bottom ive added in the logic the script now uses.

Link to comment
1 hour ago, binhex said:

OK email notification looks good, so the image is currently building, once its ready please pull down. if you want to use email notification then you will need to add in the following additional volume mounts (new users AFTER 2nd Oct 2019 will NOT have to do this), please copy the host and container paths EXACTLY as they are in the screenshots below:-

 

image.png.18d46f24f24b917a4c96e1d6bc28f0c0.png

 

and also another additional volume mount:-

 

image.png.3f58d02ed5d07f0cac0085ee91b70d13.png

 

then run the preclear script with the -M option to get notification of progress via email.

 

NOTE as long as you have setup email notification on unraid then you will NOT need to specify any email configuration or recipient email address(es).

 

Would you please setup your GUI (temporarily) for the Black type on White background so that, when we print out these instructions, they are legible.   Settings   >>>  Display Settings  >>>  Dynamix color theme:  change to 'white'.  

Link to comment
55 minutes ago, Frank1940 said:

Would you please setup your GUI (temporarily) for the Black type on White background so that, when we print out these instructions, they are legible.   Settings   >>>  Display Settings  >>>  Dynamix color theme:  change to 'white'.  

easier to just post it as code :-)

 

create two additional paths:-

 

host path /boot/config/plugins/dynamix/dynamix.cfg
container path /unraid/config/plugins/dynamix/dynamix.cfg
access mode read only
host path /etc/ssmtp/ssmtp.conf
container path /unraid/ssmtp/ssmtp.conf
access mode read only

FYI what this does is it re-uses the existing unraid host email configuration, so you dont need to define it again, cool eh.

  • Thanks 1
Link to comment

Notifications appear to be working great as well as the new default start sector.

Running /usr/local/bin/preclear_binhex.sh -f -M 4 /dev/sdm.

 

================================================================== 1.19
=                unRAID server Pre-Clear disk /dev/sdm
=               cycle 1 of 1, partition start on sector 64 
= Disk Pre-Read in progress: 0% complete
Pre Read Started on /dev/sdm 0% complete.  
(  0  of  8,001,563,222,016  bytes read ) 
Disk Temperature: 34C, 
Using Block size of  1,000,448  Bytes
Next report at 25%

 

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.