binhex Posted September 28, 2019 Author Share Posted September 28, 2019 Trying to figure out the mail options. Would it be something like this?preclear_binhex.sh -A -f -M 4 [email protected] /dev/sdX And not sure what I need to do for this? The -m, -M options requires that a valid working mail command is installed. Ok I will take a look into that, for now let's ignore the mail setup, just want to confirm a good oreclear first. Can you try running it with the -A flag as per my previous post.Sent from my CLT-L09 using Tapatalk Quote Link to comment
binhex Posted September 28, 2019 Author Share Posted September 28, 2019 40 hours 37 minutes 3 seconds later...========================================================================1.18== invoked as: /usr/local/bin/preclear_binhex.sh -f /dev/sdc== ST8000NM0055-1RM112 ZA1FS9VW== Disk /dev/sdc has been successfully precleared== with a starting sector of 1 == Ran 1 cycle==== Using :Read block size = 1000448 Bytes== Last Cycle's Pre Read Time : 14:00:01 (158 MB/s)== Last Cycle's Zeroing time : 12:18:43 (180 MB/s)== Last Cycle's Post Read Time : 14:17:20 (155 MB/s)== Last Cycle's Total Time : 40:37:03==== Total Elapsed Time 40:37:03==== Disk Start Temperature: 31C==== Current Disk Temperature: 32C, ==============================================================================** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 83 82 44 ok 217697608 Seek_Error_Rate = 77 76 45 ok 55541351 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 68 69 40 ok 32 G-Sense_Error_Rate = 99 100 0 ok 2432 Temperature_Celsius = 32 31 0 ok 32 Hardware_ECC_Recovered = 83 82 0 ok 217697608No SMART attributes are FAILING_NOW0 sectors were pending re-allocation before the start of the preclear.0 sectors were pending re-allocation after pre-read in cycle 1 of 1.0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change.0 sectors had been re-allocated before the start of the preclear.0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ Also:parted -lModel: ATA ST8000NM0055-1RM (scsi)Disk /dev/sdc: 8002GBSector size (logical/physical): 512B/4096BPartition Table: msdosDisk Flags: Number Start End Size Type File system Flags So it does recognize that it's an 8TB drive. The partition table of "msdos" is a little disconcerting, the others are showing gpt. The others also show partitions under the table listing at the bottom and a file system of "xfs". I would presume at this point that these differences are due to the fact that the disk hasn't been formatted yet. Any thoughts, comments or concerns, or should I just shove it in the array now with a variety of lessons learned?The good news is as you can see the drive is healthy, yay! The bad news is I think if you added it as it is right now I think unraid would see it as a2TB drive due to the incorrect tabke type of msdos.So if your are still game then please can you also try it with the -A option and we will see what happens.I completely understand if you just want to switch back to the plugin for now and get that drive added, choice is yours.Sent from my CLT-L09 using Tapatalk Quote Link to comment
FreeMan Posted September 28, 2019 Share Posted September 28, 2019 2 hours ago, binhex said: So if your are still game then please can you also try it with the -A option and we will see what happens. We'll see what happens... The drive does not show up in Unassigned Devices - is that to be expected? I was going to look there to see what it thought of the drive, but no love. Quote Link to comment
binhex Posted September 28, 2019 Author Share Posted September 28, 2019 We'll see what happens... The drive does not show up in Unassigned Devices - is that to be expected? I was going to look there to see what it thought of the drive, but no love.It's possible unassigned devices has detected that the precleared drive has a array signature on the drive and thus it ignores the drive, it's a guess though as I don't know the inner workings of unassigned devices.Sent from my CLT-L09 using Tapatalk Quote Link to comment
wgstarks Posted September 28, 2019 Share Posted September 28, 2019 4 hours ago, binhex said: Ok I will take a look into that, for now let's ignore the mail setup, just want to confirm a good oreclear first. Can you try running it with the -A flag as per my previous post. Sent from my CLT-L09 using Tapatalk Started it yesterday evening with -A and -f. Pre-read completed successfully. Currently at 2% on zeroing. This is on an 8TB drive. 1 Quote Link to comment
Frank1940 Posted September 29, 2019 Share Posted September 29, 2019 Interesting observation. I have both Binhex_Preclear and Binhex_Krusader installed. If either one is 'Started', the other one will error out when you attempt to start it with this message: 'Stopping' the first one will allow the other one to start. Apparently, you can not have two running instances of noVNC running at the same time. Quote Link to comment
binhex Posted September 29, 2019 Author Share Posted September 29, 2019 Interesting observation. I have both Binhex_Preclear and Binhex_Krusader installed. If either one is 'Started', the other one will error out when you attempt to start it with this message: 'Stopping' the first one will allow the other one to start. Apparently, you can not have two running instances of noVNC running at the same time. Yes you can check host portSent from my CLT-L09 using Tapatalk Quote Link to comment
Frank1940 Posted September 29, 2019 Share Posted September 29, 2019 1 hour ago, binhex said: Yes you can check host port I assume this is a setting somewhere. I did some looking on my own and found this: Is this where I would make this change? Would changing the 'Host Port' to (say) 6081 work? Or am I completely off base? Quote Link to comment
binhex Posted September 29, 2019 Author Share Posted September 29, 2019 I assume this is a setting somewhere. I did some looking on my own and found this: Is this where I would make this change? Would changing the 'Host Port' to (say) 6081 work? Or am I completely off base?Yes that would do itSent from my CLT-L09 using Tapatalk Quote Link to comment
wgstarks Posted September 29, 2019 Share Posted September 29, 2019 Just finished first preclear using preclear_binhex.sh -A -f /dev/sdX. UD is showing the drive as precleared. Looks like it worked properly using the -A flag. Quote Link to comment
wgstarks Posted September 29, 2019 Share Posted September 29, 2019 @binhex Tried starting another cycle using -M or -m flags. Wouldn't work. Just shows the help menu when I try to enter the command. Guessing because there is no unmenu mail package maybe??? I know you want to keep this docker very basic without the bells and whistles, but I really think notifications would be a basic feature IMHO. Just a thought. Quote Link to comment
binhex Posted September 29, 2019 Author Share Posted September 29, 2019 @binhex Tried starting another cycle using -M or -m flags. Wouldn't work. Just shows the help menu when I try to enter the command. Guessing because there is no unmenu mail package maybe??? I know you want to keep this docker very basic without the bells and whistles, but I really think notifications would be a basic feature IMHO. Just a thought.I will take a look at notifications, hopefully I will be able to pass through the mail client to the container Did you by any chance run parted -l to confirm the partition was the correct size and type? Sent from my CLT-L09 using Tapatalk Quote Link to comment
FreeMan Posted September 30, 2019 Share Posted September 30, 2019 (edited) Even before the post-read completed, Unassigned Devices is reporting: I'd take that as a positive sign! It seems that the `-A` parameter made the difference. This time it was 40:44:43. Sooooooo much slower. ========================================================================1.18 == invoked as: /usr/local/bin/preclear_binhex.sh -f -A /dev/sdc == ST8000NM0055-1RM112 ZA1FS9VW == Disk /dev/sdc has been successfully precleared == with a starting sector of 1 == Ran 1 cycle == == Using :Read block size = 1000448 Bytes == Last Cycle's Pre Read Time : 14:02:00 (158 MB/s) == Last Cycle's Zeroing time : 12:17:22 (180 MB/s) == Last Cycle's Post Read Time : 14:24:22 (154 MB/s) == Last Cycle's Total Time : 40:44:43 == == Total Elapsed Time 40:44:43 == == Disk Start Temperature: 31C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 66 83 44 near_thresh 3626020 Seek_Error_Rate = 79 78 45 ok 74098720 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 68 69 40 ok 32 Temperature_Celsius = 32 31 0 ok 32 Hardware_ECC_Recovered = 66 83 0 ok 3626020 No SMART attributes are FAILING_NOW 0 sectors were pending re-allocation before the start of the preclear. 0 sectors were pending re-allocation after pre-read in cycle 1 of 1. 0 sectors were pending re-allocation after zero of disk in cycle 1 of 1. 0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change. 0 sectors had been re-allocated before the start of the preclear. 0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ All looks good to me, but what does "near_thresh" mean: Raw_Read_Error_Rate = 66 83 44 near_thresh 3626020 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 And out would these counts be near threshold this early in the disk's life? Is that something to be concerned about? They're the exact same numbers as in the report from the 1st preclear. (Well, the first one to complete...) Actually, I just realized that the "Raw_Read_Error_Rate" was OK after the first run and is now "near_thresh" after this 2nd run. I've added it to the call-out. Is that something to be concerned about? Edited September 30, 2019 by FreeMan Quote Link to comment
binhex Posted October 1, 2019 Author Share Posted October 1, 2019 (edited) 12 hours ago, FreeMan said: Even before the post-read completed, Unassigned Devices is reporting: I'd take that as a positive sign! [emoji3] It seems that the `-A` parameter made the difference. This time it was 40:44:43. Sooooooo much slower. [emoji6] ========================================================================1.18== invoked as: /usr/local/bin/preclear_binhex.sh -f -A /dev/sdc== ST8000NM0055-1RM112 ZA1FS9VW== Disk /dev/sdc has been successfully precleared== with a starting sector of 1 == Ran 1 cycle==== Using :Read block size = 1000448 Bytes== Last Cycle's Pre Read Time : 14:02:00 (158 MB/s)== Last Cycle's Zeroing time : 12:17:22 (180 MB/s)== Last Cycle's Post Read Time : 14:24:22 (154 MB/s)== Last Cycle's Total Time : 40:44:43==== Total Elapsed Time 40:44:43==== Disk Start Temperature: 31C==== Current Disk Temperature: 32C, ==============================================================================** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 66 83 44 near_thresh 3626020 Seek_Error_Rate = 79 78 45 ok 74098720 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 Airflow_Temperature_Cel = 68 69 40 ok 32 Temperature_Celsius = 32 31 0 ok 32 Hardware_ECC_Recovered = 66 83 0 ok 3626020No SMART attributes are FAILING_NOW0 sectors were pending re-allocation before the start of the preclear.0 sectors were pending re-allocation after pre-read in cycle 1 of 1.0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.0 sectors are pending re-allocation at the end of the preclear, the number of sectors pending re-allocation did not change.0 sectors had been re-allocated before the start of the preclear.0 sectors are re-allocated at the end of the preclear, the number of sectors re-allocated did not change. ============================================================================ All looks good to me, but what does "near_thresh" mean: Raw_Read_Error_Rate = 66 83 44 near_thresh 3626020 Spin_Retry_Count = 100 100 97 near_thresh 0 End-to-End_Error = 100 100 99 near_thresh 0 And out would these counts be near threshold this early in the disk's life? Is that something to be concerned about? They're the exact same numbers as in the report from the 1st preclear. (Well, the first one to complete...) Actually, I just realized that the "Raw_Read_Error_Rate" was OK after the first run and is now "near_thresh" after this 2nd run. I've added it to the call-out. Is that something to be concerned about? Ignore thos values, it's a Seagate issue with smart, the one to watch are reallocated, pending sector and crc error count. edit - see here http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html and also:- Sent from my CLT-L09 using Tapatalk Edited October 1, 2019 by binhex 1 Quote Link to comment
binhex Posted October 1, 2019 Author Share Posted October 1, 2019 On 9/29/2019 at 6:29 PM, wgstarks said: I know you want to keep this docker very basic without the bells and whistles, but I really think notifications would be a basic feature IMHO. Just a thought. good progress on this, im going to test my changes with a kicked off preclear on my (tortured) 250GB drive, if all good then i will release a new version with working notifications sometime thursday i would guess. 1 Quote Link to comment
FreeMan Posted October 1, 2019 Share Posted October 1, 2019 15 hours ago, binhex said: edit - see here http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html and also:- Awesome! Thanks for the links and the reassurances. I'm going to say that the new docker is a success! The only thing you may want to recommend is using the -A flag on larger (i.e. currently normal sized) drives. Quote Link to comment
binhex Posted October 2, 2019 Author Share Posted October 2, 2019 (edited) 11 hours ago, FreeMan said: I'm going to say that the new docker is a success! The only thing you may want to recommend is using the -A flag on larger (i.e. currently normal sized) drives. So i was expecting the -A flag to work, as i have been specifying this flag myself whenever i have been preclearing drives using this preclear script (this is prior to my mods mind you). and it always works, even for drives larger than 2TB. So reading up on the whole sector boundary fiasco i have found out the following:- If your drive uses Advanced Format (generally used for drives larger than 2TB) and your partition does NOT start on a physical sector boundary that aligns ( a multiple of 8, e.g. 64) then you will get a write performance hit for this, taken from a WD whitepaper:- Quote When misalignment occurs, a logical 4K block will reside on two physical sectors. In this case, a single read or write of a 4K block will result in a read/write of two physical sectors. The impact of a “read” is minimal, whereas a single write will cause two “Read-ModifyWrites” to occur, potentially impacting performance. So unless im missing something here (a real possibility hehe) then setting the start sector boundary to '1', as the preclear plugin i THINK does by default, will mean a write penalty, as its not a multiple of 8 and thus not aligned (see whitepaper quote above). I have read that the performance hit with unaligned GPT partitions is 'minimal', so maybe not of too much concern. For this script going forward i am going to set it to use a start sector boundary of 64, which will result in it being aligned, i.e. flag -A will be the default. link to WD whitepaper:- https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/white-paper/white-paper-advanced-format.pdf Edited October 2, 2019 by binhex Quote Link to comment
binhex Posted October 2, 2019 Author Share Posted October 2, 2019 (edited) OK email notification looks good, so the image is currently building, once its ready please pull down. if you want to use email notification then you will need to add in the following additional volume mounts (new users AFTER 2nd Oct 2019 will NOT have to do this), please copy the host and container paths EXACTLY as they are in the screenshots below:- and also another additional volume mount:- then run the preclear script with the -M option to get notification of progress via email. NOTE as long as you have setup email notification on unraid then you will NOT need to specify any email configuration or recipient email address(es). Edited October 2, 2019 by binhex Quote Link to comment
wgstarks Posted October 2, 2019 Share Posted October 2, 2019 4 minutes ago, binhex said: email notification looks good, Great.👍 I’ve got an old disk I can test this on tonight. 1 Quote Link to comment
FreeMan Posted October 2, 2019 Share Posted October 2, 2019 (edited) 4 hours ago, binhex said: For this script going forward i am going to set it to use a start sector boundary of 64, which will result in it being aligned, i.e. flag -A will be the default. Thanks for digging into it - this seems like a good option. To the uninformed and/or uninitiated, "start on sector boundary 64" instead of "start on sector boundary 1" screams "OhMerGersh I'm not getting full use of my disk!!!". Considering it's an 8TB drive, even if that is true, it's a minor loss at worst... 11 minutes ago, binhex said: OK email notification looks good, so the image is currently building, once its ready please pull down This sounds really handy and I'll be sure to use it for my next preclear! I know you're not after bells and whistles, so let's just call this "a random glitter that worked its way in, cause that's what glitter does". Edited October 2, 2019 by FreeMan 1 Quote Link to comment
wgstarks Posted October 2, 2019 Share Posted October 2, 2019 4 hours ago, binhex said: For this script going forward i am going to set it to use a start sector boundary of 64, which will result in it being aligned, i.e. flag -A will be the default. I’m a little dense sometimes, so want to be sure I’m understanding this correctly. Does this mean we no longer need to set the -A flag when we run the script? Quote Link to comment
binhex Posted October 2, 2019 Author Share Posted October 2, 2019 7 minutes ago, wgstarks said: I’m a little dense sometimes, so want to be sure I’m understanding this correctly. Does this mean we no longer need to set the -A flag when we run the script? correct, the default is now start sector of 64, see the FAQ notes section at the bottom ive added in the logic the script now uses. Quote Link to comment
Frank1940 Posted October 2, 2019 Share Posted October 2, 2019 1 hour ago, binhex said: OK email notification looks good, so the image is currently building, once its ready please pull down. if you want to use email notification then you will need to add in the following additional volume mounts (new users AFTER 2nd Oct 2019 will NOT have to do this), please copy the host and container paths EXACTLY as they are in the screenshots below:- and also another additional volume mount:- then run the preclear script with the -M option to get notification of progress via email. NOTE as long as you have setup email notification on unraid then you will NOT need to specify any email configuration or recipient email address(es). Would you please setup your GUI (temporarily) for the Black type on White background so that, when we print out these instructions, they are legible. Settings >>> Display Settings >>> Dynamix color theme: change to 'white'. Quote Link to comment
binhex Posted October 2, 2019 Author Share Posted October 2, 2019 55 minutes ago, Frank1940 said: Would you please setup your GUI (temporarily) for the Black type on White background so that, when we print out these instructions, they are legible. Settings >>> Display Settings >>> Dynamix color theme: change to 'white'. easier to just post it as code :-) create two additional paths:- host path /boot/config/plugins/dynamix/dynamix.cfg container path /unraid/config/plugins/dynamix/dynamix.cfg access mode read only host path /etc/ssmtp/ssmtp.conf container path /unraid/ssmtp/ssmtp.conf access mode read only FYI what this does is it re-uses the existing unraid host email configuration, so you dont need to define it again, cool eh. 1 Quote Link to comment
wgstarks Posted October 2, 2019 Share Posted October 2, 2019 Notifications appear to be working great as well as the new default start sector. Running /usr/local/bin/preclear_binhex.sh -f -M 4 /dev/sdm. ================================================================== 1.19 = unRAID server Pre-Clear disk /dev/sdm = cycle 1 of 1, partition start on sector 64 = Disk Pre-Read in progress: 0% complete Pre Read Started on /dev/sdm 0% complete. ( 0 of 8,001,563,222,016 bytes read ) Disk Temperature: 34C, Using Block size of 1,000,448 Bytes Next report at 25% 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.