Preclear plugin


Recommended Posts

1 minute ago, CHBMB said:

sed is also a complete and utter b***h to use in my opinion.

I agree 100%.  As I said, I was introduced to it and quickly decided that I would never attempt to use it on my own.  But when someone puts a command together (as JoeL did in this case), I will use it with joy as it can do thousands of individual edits in a file with a single command.  (That is the reason I decided never to use it on my own.  A slight error in formatting can reek havoc in a file...)

Link to comment

See the attached screen shot.  Is it normal for two identical drives being pre-cleared at the same time to run at different speeds?  I've been watching and the 2nd drive has been anywhere from 2 MB/s to 15 MB/s slower than the first.  They were started within a minute of each other and it looks like the pre-read took 11 minutes longer for the 2nd drive.  Also there are a lot more reads on the faster drive than the slower.

 

preclear.thumb.JPG.cb52cf04a71b2dc78e2bc802581a17c6.JPG

Edited by TODDLT
Link to comment
1 minute ago, TODDLT said:

See the attached screen shot.  Is it normal for two identical drives being pre-cleared at the same time to run at different speeds?  I've been watching and the 2nd drive has been anywhere from 2 MB/s to 15 MB/s slower than the first.  They were started within a minute of each other and it looks like the pre-read took 11 minutes longer for the 2nd drive.

 

preclear.thumb.JPG.cb52cf04a71b2dc78e2bc802581a17c6.JPG

 

I've seen this before if that helps.  I didn't worry too much about it and my results were ok.

Link to comment

Hey there again.

My first pre clear on dev/sdd went though, skipped PreRead, 1 time Zeroing and PostReaed. At PostRead I started the PreClear on my second disk (dev/sde), PreRead, 3 times Zeroing and PostRead. That one went good  till about 30-40% of the first Zeroing process. Then at 22:32 o clock, I stopped my array because I wanted to add the cleared, first disk (dev/sdd) and the GUI went down again. I wasn't able to get a last diagnostics file but captured the syslog in FCPsyslog_tail.txt maybe this could help to evaluate the problem.

 

FCPsyslog_tail.txt

thebeast-diagnostics-20170514-2232.zip

Link to comment
20 hours ago, TODDLT said:
On 5/14/2017 at 7:09 AM, Frank1940 said:

There are a couple of Mod's that HAVE to be applied to JoeL's script for it to run under 6.?.  JoeL. knows that the script needs work but it has not been done.  (Unfortunately, since all of the old direct-links-to posts on this new board are now broke (they worked for the first few weeks), I can't point you to them.  I would suggest that you use the default script that is included with the current plugin and see if it works.

 

Thanks I'll dig around and see what I can find.  Odd that to started up the one time fine and now won't again.

I found the post from Joe L. to fix his script:

 

  

 

  • Upvote 1
Link to comment
On 15.5.2017 at 0:27 AM, itimpi said:

I believe there is an issue with the GUI if you are running a preclear when you try to stop the array.  

The GUI issue doesn't only occur when I stop the array but also when using the GUI in general at pre clear. 

 

I also notices that when the GUI crashes, the TimeMachine Backup Directory also doesn't work anymore.

 

On 15.5.2017 at 0:27 AM, itimpi said:

When the preclear finishes the GUI comes back to life.

You where right. Yesterday the GUI crashed at the first zeroing process, now it's done and the GUI is back, while first postread.

 

Link to comment

So having a strange problem. I have been trying to preclear a disk I had laying around. It starts normally, but a couple hours in it just disappears from the interface. No sign of it in the unassigned disk section of the dashboard, nor on the pre clear plugin page. Just gone. If I reboot the server, it's back. Not really sure what to check. I had just erased and zeroed the whole disk on my desktop machine prior to trying the pre clear. The syslog (attached) seems to have an error that indicates the drive size goes to zero ?!? Not sure what that means. Near the end of the log you'll see various entries for [sdn] that's the disk.

 

tower-diagnostics-20170522-2349.zip

Edited by ratmice
Link to comment
9 hours ago, ratmice said:

So having a strange problem. I have been trying to preclear a disk I had laying around. It starts normally, but a couple hours in it just disappears from the interface. No sign of it in the unassigned disk section of the dashboard, nor on the pre clear plugin page. Just gone. If I reboot the server, it's back. Not really sure what to check. I had just erased and zeroed the whole disk on my desktop machine prior to trying the pre clear. The syslog (attached) seems to have an error that indicates the drive size goes to zero ?!? Not sure what that means. Near the end of the log you'll see various entries for [sdn] that's the disk.

 

tower-diagnostics-20170522-2349.zip

Do you think that these two might be related?

 

 

Link to comment
2 minutes ago, Frank1940 said:

Do you think that these two might be related?

 

 

 

So, I can't tell if your comment is an honest question, or a snarky retort, so I'll assume the former:

 

Well, that's part of what I'm trying to figure out. I have pre cleared multiple drives over the past few months with no problem. A different drive is pre clearing, happily, right now. There do seem to be some warnings/errors in the syslog and in my limited knowledge they do not seem to be related to SMART status. I'm just trying to get some definitive info about that DB plugin. As I understand it it just sets up a cron job to update the SMART DB, seems far-fetched to see how that would affect pre clearing in the manner I'm seeing, but that's why i'm asking people way smarter than I.

 

I considered it (the DB plugin) being an issue after I had started a new pre clear. Once it gets through a full cycle i will remove the other plugin and try the wonky drive again. In the mean time I thought I'd ask about it.

 

 

Link to comment

It was an honest question.  (Since you did not say what Fix Common Problems was complaining about.)  However in this post, you have provided another bit of useful information in that another drive is working fine and previous drives have worked fine.  At this point, I would be thinking more along the lines that there is some problem with that drive.  If it is new or in warranty, I would get it replaced.  (If neither of these applied, I, personally, would deep-six the drive!) 

Link to comment
15 minutes ago, Frank1940 said:

It was an honest question.  (Since you did not say what Fix Common Problems was complaining about.)  However in this post, you have provided another bit of useful information in that another drive is working fine and previous drives have worked fine.  At this point, I would be thinking more along the lines that there is some problem with that drive.  If it is new or in warranty, I would get it replaced.  (If neither of these applied, I, personally, would deep-six the drive!) 

 

I kinda think that drive is not long for this world, too. Unfortunately it is an older drive :-(  I thought it interesting that it kinda just disappears without any trace. Here's hoping that someone can shed some light on the info in the syslog. 

 

p.s. 'Fix Common Problems' complains that it doesn't know about 'dynamix.smart.drivedb.update' so it MAY be incompatible.

 

p.p.s Thanks for not taking umbrage to me being unsure of the motives behind your 1st reply. So many things can be misconstrued without social interaction context.

Link to comment
2 hours ago, ratmice said:

p.s. 'Fix Common Problems' complains that it doesn't know about 'dynamix.smart.drivedb.update' so it MAY be incompatible.

 

IIRC dynamix smart drive db update was released when there was an issue with the smartdb then currently present in unRaid being way out of date.  Since then, LT has been updating the smartDB with every release, so the plugin shouldn't be needed anymore (hence why its not available in CA which is why FCP yells about it).

  • Upvote 1
Link to comment
20 minutes ago, trurl said:

Turbo Write isn't a plugin, and your question doesn't really make much sense. Did you bother to read the linked post?

 

What do you mean by is not a plugin? I installed it from CA and it was labeled as a plugin. 

 

Yes I did read the link, I understand what it does. I just wanted to know if it helped speed up the preclear process that's it. Thanks anyways

Edited by jrdnlc
Link to comment
1 minute ago, jrdnlc said:

 

What do you mean by is not a plugin? I installed it from CA and it was labeled as a plugin. 

The plugin leverages the built-in functionality of turbo mode aka reconstruct write available under disk settings to enable / disable turbo write mode depending upon the number of hard disks currently spinning.  Main difference is that enabling turbo write via disk settings means you're always using turbo mode while the plugin bounces back and forth.  Plugin is basically used to take advantage of when all drives are already spinning, but when some drives are spun down to drop down to the slower mode so that you save money on your hydro bill.

 

Trurl may not have been aware of its existence.

Link to comment
3 minutes ago, jrdnlc said:

 

What do you mean by is not a plugin? I installed it from CA and it was labeled as a plugin. 

Sorry, I had forgotten about the Auto Turbo Write plugin. Still recommend you read the linked post to get a better understanding of what turbo write actually is.

Link to comment
5 hours ago, Squid said:

IIRC dynamix smart drive db update was released when there was an issue with the smartdb then currently present in unRaid being way out of date.  Since then, LT has been updating the smartDB with every release, so the plugin shouldn't be needed anymore (hence why its not available in CA which is why FCP yells about it).

 

That's sorta what I had assumed, but always good to get corroboration. Thanks.

Link to comment

Guys, Just got my server put together over the weekend and started preclearing Saturday. I have 3x 8TB WD Reds. I'm running unRAID 6.3.4 with a i3-6100, supermicro x11ssm mobo & 8gb of ecc ram. The only things I have installed are the preclear plugin and unassigned devices. I decided to do 3 passes of preclear and started them all at the same time. Everything went fine (except a network issue with a cable) and the first preclear finished. The 2nd round started and Tuesday morning all three drives started the zeroing process. I checked Tuesday night and everything seemed to be working. I didn't receive any emails that it had progressed to the next stage. So I logged into the gui last night and the preclear time was still where it showed Tuesday. I didn't have a lot of time so I figured it might update at a later point. Got home earlier and it's still showing the same progress as before.

 

I tried to pull up the log for preclear but nothing loads. I can download the log but the last entry just shows the zeroing starting Tuesday. I did an ssh connection with putty and typed in preclear. It shows the same stats as the gui. I downloaded the diagnostics but didn't see any errors but I'm not 100% sure what to look for. I will attach that zip.

 

So my question is did preclear hang on me? Should I stop all 3 preclears and restart the server? If I do will my progress pick up where it left off or will I have to start over? Could this be an issue due to only having 8gb of ram but trying 3 preclears at once?

tower-diagnostics-20170525-1747.zip

 

Edit: I stopped the preclear and powered down the server. I started a preclear on all 3 drives again and it all went fine. All 3 passed with no problem. 

Edited by ozone614
Updated after restart of preclear
Link to comment

Hi,

 

I am also having issues with the preclear script stopping. I am using the most recent version 2017.03.31 with scripts gfjardim 0.8.5-beta and bjp999 - 1.15b. Unassigned devices is installed and up-to-date with 2017.05.21.

 

The progress simply stalls, no values change anymore and the disk access LED indicates that there is not activity on the disk. The GUI/server remains responsive. It's happening on different disks, controllers and with both scripts. Starting the script via the preclear plugin or unassigned devices makes not difference.

 

It happens in the second or third cycle. E.g. my last try stalled at 95% during post-read in the second cycle. I did not observe any changes in memory or CPU usage. So I dont think the process has an issue with resources.

 

Stopping and restarting the preclear does not work. The script says: "Starting..." and doesnt do anything. Instead it displays the following error message:

 

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 2 -J /dev/sdq
root@SS:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 2 -J /dev/sdq
sfdisk: invalid option -- 'R'

 

The script seems to continue to run which is also indicated by this line in the log:

May 26 10:54:29 SS preclear.disk: Pausing preclear of disk 'sdm'

Which occurred after the pre-clear stalled and I stopped the array.

 

I didn't see anything else bluntly obvious in the log but close to the drive the following warning occurs:

May 24 10:46:35 SS kernel: sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1

 

To fix this a shutdown is required. A reboot is not sufficient.

 

I initially thought that this points to a hardware issue. But I tried different controllers on my Supermicro X9SCM-F  and 

AOC-SAS2LP-MV8 with the same results and then I saw others also reporting the script stalling so I thought I share this.


I trust that the script still performs its tasks properly, so I just start another run to get to my expected cycles. So it's not a real problem for me, more an annoyance.

 

Best regards,

 

Thomas 

Link to comment

I have precleared a drive, and it ended successfully, but it doesn't show as precleared in Unassigned Devices list.

I precleared it again and got the same result.

Any thoughts? How can I check if it is indeed has the preclear signature?

Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.