26 minutes ago, binhex said:

 

edit - sorry to be clear here, im talking about faster preclear script has the requirement for strings, no idea if thats the case with this threads version of the script.

link to original script (not patched):- https://lime-technology.com/forums/topic/30921-unofficial-faster-preclear/
 

heads up guys it (script linked above) also now needs patching due to 'scripts' command not included in unraid 6.5.1 (yes this is getting ugly), ive patched my own copy with a replacement of 'strings' (2 occurrences) with 'cat -v' which is actually a lot faster.

 

i have a copy of the fast preclear script, patched for sfdisk, dd and strings, so it SHOULD (not tested) be compatible with unraid 6.5.1 now, if anybody wants to give it a whirl feel free, but as i mentioned it is currently untested (plan to do some testing in the next few days).

 

https://raw.githubusercontent.com/binhex/scripts/master/shell/unraid/system/preclear_bjp.sh

 

I am trying it now. One difference I noticed is that the starting preread speed is different, although this could be inherent to the scripts. 

The gfjardim script started at 175MB/s and the bjp666 one started at 133MB/s

3 hours ago, hobbie1009 said:

...  I attempted just to add one of the drives to the array due to the above noted issues w/ the plugin for my ver of unraid 6.4.x. I get a "unmountable no file system" error for the disk I had attempted preclear on...  The array says it is missing that disk now but I never wrote to it or ran a parity check with it so it seems like it would be fine to leave as is.

 

Thanks

 

Justin

 

You seem to have left out some important details in your description. You can't get directly from adding a disk to "unmountable no file system". If you add a disk to a new slot and that disk hasn't been marked as cleared then unRAID is going to clear it for you and then offer to format it. Even if it was mistakenly marked as clear by the failed preclear unRAID would still offer to format it. Maybe if you had taken a closer look you would have noticed that it was telling you to check a box to format the disk when the array was started. Or did you do a New Config to add the disk to the array?

 

Definitely NOT fine to leave as is. In order to remove the disk from the array you will have to New Config without it and resync parity.

 

*EDIT*

Another possibility would be to assign it back to the slot it is missing from and then check the box to format it. Any time you check the box to format make very sure that only the disk you want to format is unmountable. You should definitely do a parity check afterwards since it isn't completely clear what you did to get to this state.

18 minutes ago, papnikol said:

I am trying it now. One difference I noticed is that the starting preread speed is different, although this could be inherent to the scripts. 

The gfjardim script started at 175MB/s and the bjp666 one started at 133MB/s

 

improvements in the script by i would assume by gfjardim, i would like to hear the outcome, can you please post your result (good or bad) in this thread:-

 

https://lime-technology.com/forums/topic/30921-unofficial-faster-preclear/

 

49 minutes ago, binhex said:

Heads up guys, this script (faster preclear by bjp666) also now needs patching due to 'scripts' command not included in unraid 6.5.1 (yes this is getting ugly), ive patched my own copy with a replacement of 'strings' (2 occurrences) with 'cat -v' which is actually a lot faster.

 

i have a copy of the fast preclear script, patched for sfdisk, dd and strings, so it SHOULD (not tested) be compatible with unraid 6.5.1 now, if anybody wants to give it a whirl feel free, but as i mentioned it is currently untested (plan to do some testing in the next few days).

 

https://raw.githubusercontent.com/binhex/scripts/master/shell/unraid/system/preclear_bjp.sh

 

Please keep in mind this is probably not as refined as gfjardim's preclear plugin and script (see here) so i would advise people to use his script over this - having said that any results using this patched version (good or bad) are welcome.

Also running the script now. Will keep you posted.

4 hours ago, trurl said:

 

You seem to have left out some important details in your description. You can't get directly from adding a disk to "unmountable no file system". If you add a disk to a new slot and that disk hasn't been marked as cleared then unRAID is going to clear it for you and then offer to format it. Even if it was mistakenly marked as clear by the failed preclear unRAID would still offer to format it. Maybe if you had taken a closer look you would have noticed that it was telling you to check a box to format the disk when the array was started. Or did you do a New Config to add the disk to the array?

 

Definitely NOT fine to leave as is. In order to remove the disk from the array you will have to New Config without it and resync parity.

 

*EDIT*

Another possibility would be to assign it back to the slot it is missing from and then check the box to format it. Any time you check the box to format make very sure that only the disk you want to format is unmountable. You should definitely do a parity check afterwards since it isn't completely clear what you did to get to this state.

I apologize for the lack of clarity.  I did not see the format option at the time, had my browser a little scrunched.  The option was down at the bottom.  I stopped the array and replaced the drive into the slot I was attempting before.  Once that was done I selected the format option.  After completing the format its added the drive but the orange triangle lists it as "contents emulated".  The parity sync is running now.

 

Thanks again for replying.

.

6 hours ago, binhex said:

 

improvements in the script by i would assume by gfjardim, i would like to hear the outcome, can you please post your result (good or bad) in this thread:-

 

https://lime-technology.com/forums/topic/30921-unofficial-faster-preclear/

 

 

Will do. It will be some time as it is an 8tb disk. I will post the results in the thread you mentioned

21 hours ago, rvoosterhout said:

Also running the script now. Will keep you posted.

I'm at post-reading now but I noticed my speed dropped to 65MB/s. Is that normal?

5 hours ago, rvoosterhout said:

I'm at post-reading now but I noticed my speed dropped to 65MB/s. Is that normal?

Yes, a slow down is normal - the actual speed will depend on the drive, but I have some older drives that go down to below 65MB/s towards the end of the disk.  By convention disk drves access from the outer cylinders first.  Since they are physically longer they have more sectors and since the rotation speed is fixed the data rate starts high and then reduces as the heads are moved in to the higher numbered cylinders.

1 hour ago, S80_UK said:

Yes, a slow down is normal - the actual speed will depend on the drive, but I have some older drives that go down to below 65MB/s towards the end of the disk.  By convention disk drves access from the outer cylinders first.  Since they are physically longer they have more sectors and since the rotation speed is fixed the data rate starts high and then reduces as the heads are moved in to the higher numbered cylinders.

Ah, great explanation, thanks!

1 hour ago, rvoosterhout said:

Ah, great explanation, thanks!

Off topic - it's about 30 years since I was designing hard disk controller hardware and writing test software for it.  In those days you had to know how it all worked, otherwise it didn't work at all.

 

18 hours ago, rvoosterhout said:

I'm at post-reading now but I noticed my speed dropped to 65MB/s. Is that normal?

 

13 hours ago, S80_UK said:

Yes, a slow down is normal - the actual speed will depend on the drive, but I have some older drives that go down to below 65MB/s towards the end of the disk.  By convention disk drves access from the outer cylinders first.  Since they are physically longer they have more sectors and since the rotation speed is fixed the data rate starts high and then reduces as the heads are moved in to the higher numbered cylinders.

 

I think 65MB/s might be normal at the end of the disk, but is that the case for rvoosterhout?

Because I am also preclearing an 8TB Seagate archive and although speeds for preread and zeroing were normal, I am getting a postread speed of 57 MB/s but I am only at 8% (I dont remember if I clicked the fast post-read option, still too slow).

 

EDIT: I might be misunderstanding something because the web gui shows that:

but the log (pressing the eye icon) shows that:

 

 

Edited May 2, 20188 yr by papnikol

8 minutes ago, papnikol said:

 

 

 

I think 65MB/s might be normal at the end of the disk, but is that the case for rvoosterhout?

Because I am also preclearing an 8TB Seagate archive and although speeds for preread and zeroing were normal, I am getting a postread speed of 57 MB/s but I am only at 8% (I dont remember if I clicked the fast post-read option, still too slow).

 

EDIT: I might be misunderstanding something because the web gui shows that:

but the log (pressing the eye icon) shows that:

 

 

Yeah I also noticed that the gui and cli are different speeds. Mine was also in the beginning of the post read.

Yeah I also noticed that the gui and cli are different speeds. Mine was also in the beginning of the post read.

Joe L. script reads the disk in chunks. It reads, in a loop, n bytes and sum them up to verify it's zeroed. The speed you guys see on the cli is the read speed; in the gui you can see the whole operation speed (read + sum + verify zero sum).

Enviado de meu SM-G955F usando Tapatalk

7 hours ago, gfjardim said:

Joe L. script reads the disk in chunks. It reads, in a loop, n bytes and sum them up to verify it's zeroed. The speed you guys see on the cli is the read speed; in the gui you can see the whole operation speed (read + sum + verify zero sum).

Enviado de meu SM-G955F usando Tapatalk
 

Thanks for the clarification gfjardim. I did not expect that, tbh, since my respective times are ~180Mb/s and ~60Mb/s. That means that the (sum + verify zero sum) portion roughly takes twice the time than the reading portion ...

does anyone know which previous version worked?

Edited May 3, 20188 yr by Syco54645

48 minutes ago, Syco54645 said:

does anyone know which previous version worked?

 

On 4/30/2018 at 6:14 PM, binhex said:

Heads up guys, this script (faster preclear by bjp666) also now needs patching due to 'scripts' command not included in unraid 6.5.1 (yes this is getting ugly), ive patched my own copy with a replacement of 'strings' (2 occurrences) with 'cat -v' which is actually a lot faster.

 

i have a copy of the fast preclear script, patched for sfdisk, dd and strings, so it SHOULD (not tested) be compatible with unraid 6.5.1 now, if anybody wants to give it a whirl feel free, but as i mentioned it is currently untested (plan to do some testing in the next few days).

 

https://raw.githubusercontent.com/binhex/scripts/master/shell/unraid/system/preclear_bjp.sh

 

Please keep in mind this is probably not as refined as gfjardim's preclear plugin and script (see here) so i would advise people to use his script over this - having said that any results using this patched version (good or bad) are welcome.

This script is clearing successful over here. 

7 minutes ago, rvoosterhout said:

 

This script is clearing successful over here. 

But binhex suggests using gfjardim's atm. Isn't that not working though, or is there a patch for that?

I've started looking through the forums after the second Post-Read fail, and am having trouble piecing together what's happening and what's working.

Edited May 3, 20188 yr by Ryonez

5 minutes ago, Ryonez said:

But you suggest using gfjardim's atm. Isn't that not working though, or is there a patch for that?

I've started looking through the forums after the second Post-Read fail, and am having trouble piecing together what's happening and what's working.

I have no idea. I'm using the BJP scipt, which is working fine on my unraid systems (latest os). Are you using the BJP script mentioned above?

 

Ah i get it now. Binhex is talking about gfjardim's plugin, not the script. If you would like to use the pluging, you can use the gfjardim's in combination with the script. I'm using the CLI, so I don't use any pluging for my preclearing

Edited May 3, 20188 yr by rvoosterhout

1 minute ago, rvoosterhout said:

I have no idea. I'm using the BJP scipt, which is working fine on my unraid systems (latest os). Are you using the BJP script mentioned above?

No, I've been using gfjardim's. Not sure where to install and how use BJP's.

Okay, I've had a quick look at the script, I'm just worried what values I should use for the starting sector.

Short explanations:

 

- Copy the scipt to the root of your usb thumb-drive

- Open up a terminal screen (you can do this on your main screen of unraid)

- enter: cd /boot

you are now in your flashdrive.

- enter: ls -l

This wil show you a list of files and directories

- verify that you preclear scipt is in that list

- enter preclear_bjp.sh -l

this wil show you a list of disk that are able to be preacleared

- Check if the drive you want to clear is pressent in the list (you can check the name of the drive in the main screen of unraid, under Unassigned devices), for example mine is: SAMSUNG_HD204UI_S2H7J1AZA06529

- write down you hard drive ID (/dev/SDX the last letter is your drive) in my case this would be /dev/sdd.

- enter: preclear_bjp.sh /dev/sdX

- Type Yes to start the pre clear. Make absolutelty sure that you chose the right  drive, as this will overwrite your data.

 

Quick explanation, please anybody correct me if i'm wrong, but like this it's working here.

Edited May 3, 20188 yr by rvoosterhout
Typo

6 minutes ago, Ryonez said:

Okay, I've had a quick look at the script, I'm just worried what values I should use for the starting sector.

I'm not setting any, been running the script fine without.

7 minutes ago, rvoosterhout said:

Short explanations:

 

- Copy the scipt to the root of your usb thumb-drive

- Open up a terminal screen (you can do this on your main screen of unraid)

- enter: cd /boot

you are now in your flashdrive.

- enter: ls -l

This wil show you a list of files and directories

- verify that you preclear scipt is in that list

- enter preclear_bjp.sh -l

this wil show you a list of disk that are able to be preacleared

- Check if the drive you want to clear is pressent in the list (you can check the name of the drive in the main screen of unraid, under Unassigned devices), for example mine is: SAMSUNG_HD204UI_S2H7J1AZA06529

- write down you hard drive ID (/dev/SDX the last letter is your drive) in my case this would be /dev/sdd.

- enter: preclear_bjp.sh /dev/sdX

- Type Yes to start the pre clear. Make absolutelty sure that you chose the wrong drive, as this will overwrite your data.

 

Quick explanation, please anybody correct me if i'm wrong, but like this it's working here.

Great, so I don't need to specify the starting sector. I'll be using the -W flag to skip the pre-read, the drive has completed that twice successfully with the other script. I am looking at using the -f flag for fast post-read, as the normal post-read is designed to test the hdd as well.

This drive has already gone through two full reads and two, mostly full? writes with gfjardim's script. I should be alright with the fast post-read, yes?

17 minutes ago, Ryonez said:

Great, so I don't need to specify the starting sector. I'll be using the -W flag to skip the pre-read, the drive has completed that twice successfully with the other script. I am looking at using the -f flag for fast post-read, as the normal post-read is designed to test the hdd as well.

This drive has already gone through two full reads and two, mostly full? writes with gfjardim's script. I should be alright with the fast post-read, yes?

Yeah you could. I'm not skipping those though. I have the time  and better safe than sorry.

Edited May 3, 20188 yr by rvoosterhout

Just now, rvoosterhout said:

Yeah you could. I'm not skipping those though. I have the time  and better safe than sorry.

Not sure how it's unsafe. The drive has completed two full reads successfully. So no bad sectors. The writing worked, thought the other script seems to missed someplace as part of the the script, not the hdd, which caused the post-read to fail.

The reason the ppre-read takes so long is to add stress to the drive to see if it fails isn't it? The hdd has already had two full-reads, and two full writes, so I'm not sure where the risk is in skipping the pre-read and speeding up the post-read.

At this point I don't see how increasing the time would increase the safety.