Preclear plugin


Recommended Posts

59 minutes ago, kbbeer said:

Sent! Thanks mate

Hi, apparently it's not a problem with your drive, it's a problem in the script itself. I've being doing some tests to prevent the server to become out of memory if a sector read of a disk fail. If the dd program get's stuck, it starts to output massive amounts of errors to the RAM file system. We want it to try and retry to read a damaged sector to force it to become a pending sector, but not indefinitely. In this case, the script tries to figure if the program got stuck on a sector for too much time, kill it and retry for 5 times.

 

Apparently the problem is the script failing in updating the progress of the read/write operation, and  kill the program before it reaches it's end. I'm testing a new code right now, and it will take some time to proper test the fix.

You can revert to a prior working version by running this command in the command line:

wget https://github.com/gfjardim/unRAID-plugins/raw/master/archive/preclear.disk-2020.11.18.txz
installpkg preclear.disk-2020.11.18.txz 
rm preclear.disk-2020.11.18.txz

 

I'll release a working version as soon as I proper test it.

Link to comment
4 hours ago, gfjardim said:

Hi, apparently it's not a problem with your drive, it's a problem in the script itself. I've being doing some tests to prevent the server to become out of memory if a sector read of a disk fail. If the dd program get's stuck, it starts to output massive amounts of errors to the RAM file system. We want it to try and retry to read a damaged sector to force it to become a pending sector, but not indefinitely. In this case, the script tries to figure if the program got stuck on a sector for too much time, kill it and retry for 5 times.

 

Apparently the problem is the script failing in updating the progress of the read/write operation, and  kill the program before it reaches it's end. I'm testing a new code right now, and it will take some time to proper test the fix.

You can revert to a prior working version by running this command in the command line:


wget https://github.com/gfjardim/unRAID-plugins/raw/master/archive/preclear.disk-2020.11.18.txz
installpkg preclear.disk-2020.11.18.txz 
rm preclear.disk-2020.11.18.txz

 

I'll release a working version as soon as I proper test it.

I am in the middle of a preclear on a new 8TB drive, do I need to stop?  Or only if it has problems?

Link to comment
9 hours ago, kbbeer said:

Thanks a lot for your response and continuing to maintain the plugin. To confirm, after performing the commands above should the version number in the plugins gui be updated?

 

Updated - I ran the command above and I'm getting a permission denied error to start the script. Any idea what I did wrong?

Ops, you need this command too:

 

chmod -R +x /usr/local/emhttp/plugins/preclear.disk/script

 

Link to comment
17 hours ago, gfjardim said:

It's a racing condition, so it may or not affect you. You should continue to run your preclear and only downgrade if if fail.

Started preclear #2 on a WD Red Plus 8TB.

 

Does the SATA link speed affect how long the preclear will take?  I.e. 3 Gb/s vs 6 Gb/s.

 

It takes the 8TB drive 14.5hrs just to do the pre-read part, and 14hrs to zero, etc.

 

Can you add a feature that you can go back to the running preclear and tell it to do another round of preclear when done current preclear?  I.e thr current preclear is going to finish in the middle of the night.

Link to comment
4 hours ago, Paul_Ber said:

Started preclear #2 on a WD Red Plus 8TB.

 

Does the SATA link speed affect how long the preclear will take?  I.e. 3 Gb/s vs 6 Gb/s.

 

It takes the 8TB drive 14.5hrs just to do the pre-read part, and 14hrs to zero, etc.

 

Can you add a feature that you can go back to the running preclear and tell it to do another round of preclear when done current preclear?  I.e thr current preclear is going to finish in the middle of the night.

1) probably not, but your speed is on par with others I've seen on reports for that disk size.

 

2) it's a difficult feature to implement, but there is a dirty hack you can do. Stop your preclear session, go to <Flash>/config/plugins/preclear.disk/ and find the file named <serial of the disk>. resume . Change the value of the variable >cycles< accordingly. Go to the webgui and click to start a preclear, and then choose to resume the preclear session.

Link to comment

I love UPS delivery, only way I knew my RMA replacement WD60EFRX hard drive arrived was a text message.  They never rang the door bell.

 

So to the question, it feels cold, I.e. it must of been in the back of UPS van all day.   Should I let it warm up 1st before run my 3 rounds of Preclear.  My thought is yes.

Edited by Paul_Ber
Spelling
Link to comment

Hello.

 

So I have four 4TB drives preclearing right now.  They are used drives that I intend to use for a backup server, so not a huge deal as I got them for $50.

 

One of the four keeps failing around 90% or so.  It gives me a "dd process hung at ###" then kills the process according to the logs.  Is the drive bad?

 

The reseller wiped the SMART data so I don't know much about the overall health of the drive.

 

Also; I ended up canceling the preclear for the other three drives after they had finished one cycle.  Now I can't preclear them because the icon is missing.  I wanted to run them for a couple more cycles as well.  How do I do that?

Edited by aidenpryde
Link to comment
Hello.
 
So I have four 4TB drives preclearing right now.  They are used drives that I intend to use for a backup server, so not a huge deal as I got them for $50.
 
One of the four keeps failing around 90% or so.  It gives me a "dd process hung at ###" then kills the process according to the logs.  Is the drive bad?
 
The reseller wiped the SMART data so I don't know much about the overall health of the drive.
 
Also; I ended up canceling the preclear for the other three drives after they had finished one cycle.  Now I can't preclear them because the icon is missing.  I wanted to run them for a couple more cycles as well.  How do I do that?
There was a bug introduced recently that lead to the exact behavior you're describing. Please update your pugin and start again.

If you can't find the icon on Unassigned Devices, take a look at the Preclear Plug-in in the Tools menu.

Enviado de meu SM-G985F usando o Tapatalk

  • Like 1
Link to comment
Just now, gfjardim said:

There was a bug introduced recently that lead to the exact behavior you're describing. Please update your pugin and start again.

If you can't find the icon on Unassigned Devices, take a look at the Preclear Plug-in in the Tools menu.

Enviado de meu SM-G985F usando o Tapatalk
 

Yeah I read about that, and upgraded the plugin while they were still preclearing, then the drive failed again, so I thought I better cancel and start again because maybe the plugin was in memory still.  Restarted the server and the icon disappeared, but thanks, now I can find it under tools!

Link to comment

Did a defaults preclear of a new drive, with pre and post reads on. When I came back to it the status show 'An error occured check the log' but I can't see anything wrong here.  Ideas? Am I missing something? Ok to still put in array?

 

Dec 30 16:29:35 ur kernel: sd 7:0:0:0: [sde] 7814037168 512-byte logical blocks: (4.00 TB/3.64 TiB)
Dec 30 16:29:35 ur kernel: sd 7:0:0:0: [sde] 4096-byte physical blocks
Dec 30 16:29:35 ur kernel: sd 7:0:0:0: [sde] Write Protect is off
Dec 30 16:29:35 ur kernel: sd 7:0:0:0: [sde] Mode Sense: 00 3a 00 00
Dec 30 16:29:35 ur kernel: sd 7:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 30 16:29:35 ur kernel: sd 7:0:0:0: [sde] Attached SCSI disk
Dec 30 16:29:43 ur emhttpd: WDC_ (sde) 512 7814037168
Dec 30 16:33:16 ur preclear_disk_WD-[12024]: Command: /usr/local/emhttp/plugins/preclear.disk/script/preclear_disk.sh --cycles 1 --no-prompt /dev/sde
Dec 30 16:33:18 ur preclear_disk_WD-[12024]: Pre-Read: dd if=/dev/sde of=/dev/null bs=2097152 skip=0 count=4000787030016 conv=notrunc,noerror iflag=nocache,count_bytes,skip_bytes
Dec 31 00:37:47 ur preclear_disk_WD-[12024]: Zeroing: dd if=/dev/zero of=/dev/sde bs=2097152 seek=2097152 count=4000784932864 conv=notrunc iflag=count_bytes,nocache,fullblock oflag=seek_bytes

 

Link to comment
4 hours ago, John_M said:

As well as the flashing display in the UD area, a successfully pre-cleared disk no longer shows "pre-cleared" as the FS type (it displays as "-" instead). Other than these minor display issues the plugin seems to be working well at the moment.

I have a fix for the file system not showing 'precleared' in UD.  I also implemented the temperature display and reads and writes during a preclear.

 

@gfjardim I see the problem with the blinking of the preclear status line in UD.  UD does a refresh of the page every 3 seconds.  You are updating the status periodically like when UD did not refresh itself.  They are clashing.  The best bet is to let UD do the page update and preclear just provides the information.

 

Also take a look at the /usr/local/state/devs.ini file in 6.9 RC2.  Unraid is providing disk information for Unassigned Devices.  I use this for disk spin, reads, writes, and temperature information.  I don't query the disk.  UD keeps this file current when a hot plug event occurs.

  • Like 1
  • Thanks 1
Link to comment
I have a fix for the file system not showing 'precleared' in UD.  I also implemented the temperature display and reads and writes during a preclear.
 
@gfjardim I see the problem with the blinking of the preclear status line in UD.  UD does a refresh of the page every 3 seconds.  You are updating the status periodically like when UD did not refresh itself.  They are clashing.  The best bet is to let UD do the page update and preclear just provides the information.
 
Also take a look at the /usr/local/state/devs.ini file in 6.9 RC2.  Unraid is providing disk information for Unassigned Devices.  I use this for disk spin, reads, writes, and temperature information.  I don't query the disk.  UD keeps this file current when a hot plug event occurs.
Hi Dan, I'm on a vacation this week, but will take a look soon as possible. Thanks for tracking the problem and sorry for the delay.

Enviado de meu SM-G985F usando o Tapatalk

Link to comment
On 1/1/2021 at 9:15 PM, dlandon said:

No problem.  

Dan, just took a look into the problem and apparently it was easier to fix than I though. Just committed a version 2021.01.03 of Preclear Plugin, please let me know if it works correctly for you.

Again, thanks and sorry for the delay.

Link to comment
21 minutes ago, gfjardim said:

Dan, just took a look into the problem and apparently it was easier to fix than I though. Just committed a version 2021.01.03 of Preclear Plugin, please let me know if it works correctly for you.

Again, thanks and sorry for the delay.

Looks good.

Link to comment
  • 2 weeks later...

Apologies if this is covered elsewhere (as I'm almost sure it must be) but I've searched the topic and googled the Web and not found the answer. Which probably means it's bleedin' obvious, hence my apologies.


I want to preclear a 16TB drive that's currently one of three unassigned devices. One of them is the DOM of the QNAP box hosting UnRAID. The other is an external USB hard drive.

 

The preclear UI seems to be threatening to preclear all three of these devices. How do I ensure it only preclears the 16TB drive? (Yes, I can unplug the external USB drive, but the DOM is glued in place and I'm happy to leave it there.)

 

SpaceInvader One's excellent (as always) video on the subject doesn't answer my question as he has only a single drive in his unassigned devices list.

 

I'm running UnRAID 6.9.0 rc2

 

LATER THAT SAME EVENING

 

OK, rather than just delete this post and pretend it never happened, I'll let my ignorance shine out for all to see and add some notes that may help some others running into this same (imagined) problem. 

 

There's no problem once you understand that 1) only UNMOUNTED unassigned devices are candidates for preclearing and 2) the "Start Preclear" in blue at the far right end of  each device's listing on the preclear app page IS A LINK. Clicking this link only starts the preclear on that particular drive. It's not the case, as I previously thought, that drives with this blue message appearing at the end of their tuples are being lined up as candidates for automatic preclearing.

 

Once you've clicked the link you get a dialogue box confirming that this is the drive you want to preclear. If you've hit the wrong link, you can back out at this point.

 

There is an option, as I understand it, to the left of the set of icons above the unassigned drive list, for preclearing multiple drives, either in sequence or simultaneously.

 

One thing I had guessed right: that the solution was bleedin' obvious.

 

-- 

Chris

Edited by bidmead
additional thoughts
Link to comment

Currently swapping out some old drives on my server for newer larger ones. I am intending to sell my old drives and have in the past done a three loop pre-clear on any drives I sell to give them a stress test to hopefully give the buyer a little peace of mind. 

 

I am currently pre-clearing six drives simultaneously and all is going well. Apart from one drive that can't seem to get past the first zeroing. 

 

I've attached the log. It gets to the end of the zeroing loop. The dd process hangs, gets killed and starts from the beginning of the zeroing again. A dodgy drive perhaps? The pre-clearing isn't failing and SMART doesn't show any issues. 

preclear_disk_WD-WMC4N1495101_9736.txt

Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.