Preclear plugin


Recommended Posts

Hi gfjardim,

 

First thank you for the script/plugin, been using them for a while now since unRAID 6.0, and using Joe L.'s pre-clear script long before.  You've made the process such a breeze now.

 

Quick question, and I'm sure it's been asked before, but I just don't have the time to go through the thread. (Perhaps edit(ing) the first post with updates, and a FAQ?, something I think needs to be done a lot more on these forums, or move to a better forum software...  but I digress)

 

My issue is I'm not getting any S.M.A.R.T. reporting in neither the status view while pre-clear is running, or at the end in my reports.  The box is just blank.  I'm sure it's a quick fix, and other than that everything is working well.

 

Thanks again!

 

S.M.A.R.T. status becomes unavailable if the script can't detect your controller properly. Do you mind sharing your server configuration and which controller is hosting your disk?

 

I think this has been mentioned a couple of times before in this thread, that some new drives doesn't have SMART enabled by default and therefor need it enabled before starting the script. After adding the drive to the array, SMART is available.

I have had this on a few new drives when using your plugin and script. It might have been johnnie.black that mentioned this.

 

I question that this is my issue, because I can run full SMART status requests from the unRAID web GUI, and I get full SMART status reporting through the unRAID

GUI as well.

 

I did try adding the drive(s) to the array and back, but same result.

 

Feel like I was left at the buss stop.

Any advice?

Link to comment

Remove the plugin and reinstall it again.

 

Finally got through a massive issue with emhttp hanging and the server not shutting down. I've removed & reinstalled the plug in.

 

Going to let the preclear run with nothing else (dockers) running on the server. Hoping I get through it this time.

 

Thanks!

Link to comment

I redownloaded the original preclear_disk.zip, extracted preclear_disk.sh from it and compared SHA256 fingerprints from the fresh copy v the copy I'm actually using, and they match.

 

The original preclear_disk.sh has to be patched in order to work with unRAID 6.2

 

Just to be clear....by "patched" do we mean the following:

 

Patched = download the original preclear script by Joe L (here) and place that preclear script in "/boot/config/plugins/preclear.disk/"?

 

 

Link to comment

I redownloaded the original preclear_disk.zip, extracted preclear_disk.sh from it and compared SHA256 fingerprints from the fresh copy v the copy I'm actually using, and they match.

 

The original preclear_disk.sh has to be patched in order to work with unRAID 6.2

 

Just to be clear....by "patched" do we mean the following:

 

Patched = download the original preclear script by Joe L (here) and place that preclear script in "/boot/config/plugins/preclear.disk/"?

 

 

Here is the info on the patch for Joe L's original preclear script http://lime-technology.com/forum/index.php?topic=13054.msg464653#msg464653.

 

According to gfjardim, his script is 6.2 compatible out of the box. No patch required. https://lime-technology.com/forum/index.php?topic=39985.msg504465#msg504465.

Link to comment

I redownloaded the original preclear_disk.zip, extracted preclear_disk.sh from it and compared SHA256 fingerprints from the fresh copy v the copy I'm actually using, and they match.

 

The original preclear_disk.sh has to be patched in order to work with unRAID 6.2

 

Just to be clear....by "patched" do we mean the following:

 

Patched = download the original preclear script by Joe L (here) and place that preclear script in "/boot/config/plugins/preclear.disk/"?

 

 

Here is the info on the patch for Joe L's original preclear script http://lime-technology.com/forum/index.php?topic=13054.msg464653#msg464653.

 

According to gfjardim, his script is 6.2 compatible out of the box. No patch required. https://lime-technology.com/forum/index.php?topic=39985.msg504465#msg504465.

 

Thank you.

 

I tried several times but had trouble with gfjardim's script on v6.2.1.  I installed, uninstalled, re-installed, etc, many times but had issues with my syslog filling up, webGUI becoming unresponsive and the zeroing (write cycle) was ~ 5MB/s.

 

Then I had tried to run Joe L's original preclear script in v6.2 via telnet and couldn't the preclear script to run, but.... with the "sed..." command modification Joe L. lists at the 1st link you provided, it works via telnet, although the pre-read cycle does seem slow...

 

It seems preclear isn't playing well with v6.2 for me...

 

FYI - here is the syslog detail from the preclear operation....this keeps repeating over and over while preclear runs filling up the syslog.  I ended up canceling the preclear I had running in telnet.

 

Oct 13 22:41:08 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 7b 4b 04 d8 00 00 01 00 00 00
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(11)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(114688)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(16384), sc->result(0x00000000)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:09 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 2f 00 00 00 01 00 00 00
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(9)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(131072)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(0), sc->result(0x00000000)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:10 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 32 00 00 00 01 00 00 00
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(9)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(130560)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(512), sc->result(0x00000000)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 42 00 00 00 01 00 00 00
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(11)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(130560)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(512), sc->result(0x00000000)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 43 00 00 00 01 00 00 00
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(11)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(130560)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(512), sc->result(0x00000000)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)

Link to comment

Hi,

 

I was just wondering if the preclear signature gets deleted when formatting a precleared drive using the Unassigned Devices plugin afterwards?

 

So is it safe to preclear a drive, format it with UD and just assign it to the array when needed some day?

 

Or should I just leave it precleared and let unRAID format the drive when adding it to the array?

Link to comment

I was just wondering if the preclear signature gets deleted when formatting a precleared drive using the Unassigned Devices plugin afterwards?

Yes, formatting a disk removes the 'clear' signature.

 

So is it safe to preclear a drive, format it with UD and just assign it to the array when needed some day?
Not a good idea unless you actually want to use it via UD in the meantime for storing data.    If you have formatted the drive in UD, then when you later try to add it to the parity protected array, unRAID will go through its clear process wiping existing contents and then on completing the clear expect you to format it from within unRAID.

 

Or should I just leave it precleared and let unRAID format the drive when adding it to the array?
this would be the recommended approach in most cases.    The format from from within unRAID only takes moments, it is the long 'clear' step that is avoided by having it pre-cleared.
Link to comment

Alright, thanks for clarifying

I always pipe in on this sort of thing to elaborate. A formatted disk is obviously NOT clear because it has had an empty filesystem written to it. An empty filesystem contains the metadata of the filesystem and a clear disk just has all zeros in every byte. The point of a clear disk is all zeros has no impact on parity. The metadata of an empty filesystem does impact parity, which is why unRAID updates parity when you format a disk you have added to the array.

 

Format means "write an empty filesystem to this disk". That is what it has always meant in every operating system you have ever used. If the disk is in the parity array, unRAID treats this write to this array disk just like it does any other write, by updating parity.

 

If you try to add a formatted disk to the parity array unRAID will want to clear it so parity will remain valid.

Link to comment

Alright, thanks for clarifying

I always pipe in on this sort of thing to elaborate. A formatted disk is obviously NOT clear because it has had an empty filesystem written to it. An empty filesystem contains the metadata of the filesystem and a clear disk just has all zeros in every byte. The point of a clear disk is all zeros has no impact on parity. The metadata of an empty filesystem does impact parity, which is why unRAID updates parity when you format a disk you have added to the array.

 

Format means "write an empty filesystem to this disk". That is what it has always meant in every operating system you have ever used. If the disk is in the parity array, unRAID treats this write to this array disk just like it does any other write, by updating parity.

 

If you try to add a formatted disk to the parity array unRAID will want to clear it so parity will remain valid.

 

thanks for the explanation. I understand now

Link to comment

It seems preclear isn't playing well with v6.2 for me...

 

FYI - here is the syslog detail from the preclear operation....this keeps repeating over and over while preclear runs filling up the syslog.  I ended up canceling the preclear I had running in telnet.

 

Oct 13 22:41:08 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 7b 4b 04 d8 00 00 01 00 00 00
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(11)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(114688)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(16384), sc->result(0x00000000)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:08 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:09 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 2f 00 00 00 01 00 00 00
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(9)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(131072)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(0), sc->result(0x00000000)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:09 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:10 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 32 00 00 00 01 00 00 00
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(9)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(130560)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(512), sc->result(0x00000000)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:10 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 42 00 00 00 01 00 00 00
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(11)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(130560)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(512), sc->result(0x00000000)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: sd 1:0:11:0: [sdm] tag#0 CDB: opcode=0x88 88 00 00 00 00 00 00 00 43 00 00 00 01 00 00 00
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	sas_address(0x4433221105000000), phy(5)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	enclosure_logical_id(0x500605b002b71570),slot(6)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	handle(0x0014), ioc_status(success)(0x0000), smid(11)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	request_len(131072), underflow(131072), resid(130560)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	tag(65535), transfer_count(512), sc->result(0x00000000)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	scsi_status(check condition)(0x02), scsi_state(autosense valid )(0x01)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: 	[sense_key,asc,ascq]: [0x06,0x29,0x00], count(18)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)
Oct 13 22:41:11 Artoo-Detoo kernel: mpt2sas_cm0: log_info(0x31120312): originator(PL), code(0x12), sub_code(0x0312)

 

The mpt2sas modules (and similar variants) are notorious for horrible error reporting.  This is not particular to an unRAID version, or even Preclear (any version or script).  I would reconnect the drive off the SAS card, so you can get intelligible error messaging, find out what's wrong.  (Doesn't necessarily mean something is wrong with the drive, but there may be.)

Link to comment

I have a problem with one server in that it is running out of log space. My investigation reveals a rapidly growing /var/log/preclear.disk.log file. My other servers don't have this file at all, but they don't have any unassigned devices either. Whenever the Main window of the GUI is open, or when I go to Tools -> Preclear Disks I get four entries added to the aforementioned log file every ten seconds:

 

Thu Oct 20 20:50:12 BST 2016: PHP Warning: syntax error, unexpected $end, expecting TC_DOLLAR_CURLY or TC_QUOTED_STRING or '"' in /var/state/preclear.disk/state.ini on line 8

in /usr/local/emhttp/plugins/preclear.disk/Preclear.php on line 273

Thu Oct 20 20:50:12 BST 2016: PHP Warning: array_key_exists() expects parameter 2 to be array, boolean given in /usr/local/emhttp/plugins/preclear.disk/Preclear.php on line 275

Thu Oct 20 20:50:12 BST 2016: PHP Warning: syntax error, unexpected $end, expecting TC_DOLLAR_CURLY or TC_QUOTED_STRING or '"' in /var/state/preclear.disk/state.ini on line 8

in /usr/local/emhttp/plugins/preclear.disk/Preclear.php on line 273

Thu Oct 20 20:50:12 BST 2016: PHP Warning: array_key_exists() expects parameter 2 to be array, boolean given in /usr/local/emhttp/plugins/preclear.disk/Preclear.php on line 275

 

If I navigate away to some other page, such as the Dashboard, the log file stops growing until I switch back to either the Main page or the Preclear Disk page.

 

I have two (unassigned) disks in the affected server, both mounted using the Unassigned Devices plugin.

 

The problem seems to be associated with the file called /var/state/preclear.disk/state.ini mentioned in the error messages. This file also only exists on the affected server and is absent from my others.

 

root@Lapulapu:~# cat /var/state/preclear.disk/state.ini 

[/dev/disk/by-id/ata-TOSHIBA_MQ01ABB200_44NSTB4DT]
ID_MODEL = "TOSHIBA_MQ01ABB200"
ID_SERIAL_SHORT = "44NSTB4DT"
FAMILY = "Toshiba 2.5" HDD MQ01ABB..."
MODEL = "TOSHIBA MQ01ABB200"
FIRMWARE = "AY000U"
SIZE = 2000398934016root@Lapulapu:~#

 

This is obviously a reference to my /dev/sdl (see attached screenshots). Does anyone know what the problem is and how to fix it? Should there be a newline at the end of the state.ini file?

 

EDIT: unRAID version 6.2.1, plugin version 2016.09.27 (both latest stable)

Main_-_Unassigned_Devices.png.4b1c3b793e704c8544aba6d0c0abdba6.png

Preclear_Disk.png.6a32c4e00a77195242c8a3863e2733b7.png

Link to comment

What version unRAID and plugin?

 

Apologies. I've amended my post to include that information.

 

EDIT: I should emphasise that the system is idle. I'm not running any preclears and haven't done for some months. I'm pretty convinced that the state.ini file is the cause of the problem. It looks wrong, with the extra newline at the top and the missing one at the bottom (line 8, as mentioned in the error message). And why is only /dev/sdl included? I would expect /dev/sdm to be there as well. What creates and maintains this file? Presumably the plugin itself does, given its pathname.

 

Link to comment

Hi gfjardim,

 

First thank you for the script/plugin, been using them for a while now since unRAID 6.0, and using Joe L.'s pre-clear script long before.  You've made the process such a breeze now.

 

Quick question, and I'm sure it's been asked before, but I just don't have the time to go through the thread. (Perhaps edit(ing) the first post with updates, and a FAQ?, something I think needs to be done a lot more on these forums, or move to a better forum software...  but I digress)

 

My issue is I'm not getting any S.M.A.R.T. reporting in neither the status view while pre-clear is running, or at the end in my reports.  The box is just blank.  I'm sure it's a quick fix, and other than that everything is working well.

 

Thanks again!

 

S.M.A.R.T. status becomes unavailable if the script can't detect your controller properly. Do you mind sharing your server configuration and which controller is hosting your disk?

 

I think this has been mentioned a couple of times before in this thread, that some new drives doesn't have SMART enabled by default and therefor need it enabled before starting the script. After adding the drive to the array, SMART is available.

I have had this on a few new drives when using your plugin and script. It might have been johnnie.black that mentioned this.

 

I question that this is my issue, because I can run full SMART status requests from the unRAID web GUI, and I get full SMART status reporting through the unRAID

GUI as well.

 

I did try adding the drive(s) to the array and back, but same result.

 

Feel like I was left at the buss stop.

Any advice?

 

Hi gfjardim,

 

First thank you for the script/plugin, been using them for a while now since unRAID 6.0, and using Joe L.'s pre-clear script long before.  You've made the process such a breeze now.

 

Quick question, and I'm sure it's been asked before, but I just don't have the time to go through the thread. (Perhaps edit(ing) the first post with updates, and a FAQ?, something I think needs to be done a lot more on these forums, or move to a better forum software...  but I digress)

 

My issue is I'm not getting any S.M.A.R.T. reporting in neither the status view while pre-clear is running, or at the end in my reports.  The box is just blank.  I'm sure it's a quick fix, and other than that everything is working well.

 

Thanks again!

 

S.M.A.R.T. status becomes unavailable if the script can't detect your controller properly. Do you mind sharing your server configuration and which controller is hosting your disk?

 

But of course,

 

It's happened on two different controllers now, first the on board Intel C602 chipset controller. 

(The board I use also comes with 4 on board Marvell SE923, Though I've not tried on those SATA headers as there's already some known issues in unRAID with the Marvell chipsets. )

 

The second, and now currently running pre-clear, an Intel RS2WC080 RAID Controller SAS/SATA MD2 6GB/s PCIe 2.0x8 flashed to the  LSI SAS2008 IR/IT firmware.

 

Fun part is if I look at the drives through the unRAID web GUI, all S.M.A.R.T. status is reported, and correct.

 

I'm moving on from this plugin.  Hopefully the information here will help someone in the future should they have the same issues.

(Should they resolve it, please send me a PM  ;D )

...

...

If of course, they can ever find it...

Tom at the least should consider using a more up to date, and friendly forum software.

 

Anyway.

 

I've found: Every configuration results in the same scenario of no S.M.A.R.T. status results being shown during or after the pre-clear. Including blank S.M.A.R.T. status in the final report saved on the flash drive.

 

unRAID: 6.1.9

Pre-Clear Plugin: 2016.09.27

 

Tried 8 brand new drives from various manufacturers including Western Digital, HGST, Toshiba, Hitachi, and Samsung.

(I use Pre-Clear a lot as a way of testing new drives before they go into a client's PC/Workstation/Server)

Drives were tested in AND out of a 5 in 3-5.25" iStar hot-swap bay.  They were connected directly to the motherboard as well as to the Raid card in both scenarios, (see previous message for configuration details of RAID controller, and on-board chipset).

 

Saarg's mention of having to add to the array first to activate S.M.A.R.T. status is outside the question.  It would be counter intuitive to add a drive to the array, wait the hours for it to clear, wait for a parity check because of a newly added drive, then remove the drive, wait for a parity check, pre-clear the drive, wait for pre-clear, add the cleared drive to the array, and again wait for a parity check..... Greatly defeats the purpose.

(Waiting for Parity checks in between is exaggerated, but not unreasonable with a server holding data not wanting to be lost.)

Null and voided anyway, as after trying the suggestion, led to the same results.

 

 

After having to read through 18 pages, (such the normal process when looking for any guidance with unRAID troubleshooting.)

it was found that aptalca was the one having "similar" issues to what I am experiencing.

I say "Similar" because all of the drives report S.M.A.R.T. status within my unRAID GUI, and all show S.M.A.R.T. status as "Enabled".

 

FEATURE REQUEST for beta version:

 

Some newly added drives have smart disabled. In that state if the preclear script (beta) is run, there are no smart attributes retrieved. It kind of defeats the purpose of running the preclear to stress test the drives as one cannot compare the values before and after.

 

Could you modify the script to make sure that smart is enabled prior to start and enable it if it wasn't?

 

I currently do the following to enable it manually but sometimes forget or not realize:

smartctl --smart=on /dev/sdX

 

Thanks

 

However after using his code via putty session, my drives don't magically start showing the S.M.A.R.T. status as his do... 

They actually disappear from the list of drives with in the Pre-Clear Plugin, at which point I have to reboot the server to have the plugin list the drives again.

 

I would like to expand upon his statement, that there is no "Pre-Clear Start" report generated.  Maybe I'm missing something, or perhaps this is also another bug I am experiencing, but Having a Start and End report is quite valuable, I would think?

 

I frankly just don't have anymore time to put into this.  I'll continue to initiate Pre-Clear Via Putty. ... Bummer.

Link to comment

Preclear just doesn't seem to like me much lately...

 

I've launched a preclear on a drive that, admittedly, I've had other trouble with.

 

This time, I've started the preclear with the drive attached to a USB3.0 -> SATA adapter, and the drive is outside the case. I started the preclear on 10/21 at 06:41 local, and it seems to have hung after 7 hours of processing at 20% complete (first attachment). I kept seeing that it was running at 32MB/s and thought that was insanely slow, but didn't register that it was also only reporting 7 hours of processing time when it's been nearly 36 hours since I launched the preclear, therefore, I've just left it running while doing other stuff.

 

The log reports that it couldn't allocate memory and, therefore, there was a script error (second attachment). I've got 8GB of RAM in the box, but I do have a bunch of dockers running (see .sig).

 

A) It would be nice if the script could be made a bit more robust to capture these errors and not just hang. Easier said than done, I understand.

B) Is is possible to copy text from the log window? Firefox just doesn't seem to want to let me do that, hence the screen cap instead.

C) What do I need to do now? I presume that I need to just stop the preclear (though I had issues with that previously and ended up having to power off the machine with a long-press of the power button, and would like to avoid that again if possible).

 

Do I simply not have enough RAM to support the configuration I'm attempting to run?

I readily acknowledge that there could simply be issues with the drive, having had a bit of a smoke (see first link), but it does seem to have passed SMART testing. I'd like to make a rather thorough test of the drive before giving up on it, since it's passed HGST's tests, they won't RMA it, even through it's still under warranty.

unRAID_preclear_hung_2.PNG.a5799ed32249cf5cd6435512bff9de47.PNG

unraid_preclear_message_3.PNG.e2e3e76729502f87ed5a4541178b734e.PNG

Link to comment

I've been trying to find out why I'm having this problem. There are two error messages that each get added to the /var/log/preclear.disk.log file twice every ten seconds.

 

The first is

 

Thu Oct 20 20:50:12 BST 2016: PHP Warning: syntax error, unexpected $end, expecting TC_DOLLAR_CURLY or TC_QUOTED_STRING or '"' in /var/state/preclear.disk/state.ini on line 8

in /usr/local/emhttp/plugins/preclear.disk/Preclear.php on line 273

 

Line 273 of the Preclear.php file is

 

  $state = is_file($state_file) ? @parse_ini_file($state_file, true) : array();

 

and it looks as though it's failing when it tries to parse the state.ini file. My state.ini file (listed in my original post) appears to be truncated, but since I don't know how that file is created and maintained I can't figure out why.

 

The second error message is

 

Thu Oct 20 20:50:12 BST 2016: PHP Warning: array_key_exists() expects parameter 2 to be array, boolean given in /usr/local/emhttp/plugins/preclear.disk/Preclear.php on line 275

 

and the relevant line of Preclear.php is

 

  if (array_key_exists($device, $state) && ! $reload)

 

which is failing because the type of the parameter

state

returned by the previous line of code is Boolean, where an array is expected.

 

I'm not a PHP coder so I don't really understand enough to be able to work out exactly what's going on, but if I shut down and remove my two unassigned disks the problem goes away because then neither the /var/state/preclear.disk/state.ini file nor the /var/log/preclear.disk.log file is created. If I put them back in again the /var/log/preclear.disk.log file grows steadily and I'll be out of log space in a couple of weeks. If the plugin gave a single error message and then exited gracefully I could live with it for the time being and possibly troubleshoot some more but as it is I'm going to have to uninstall it.

 

Is there a bug in the plugin code or is there something about my two unassigned disks that causes the state.ini file to be truncated?

 

I took the opportunity to update to unRAID 6.2.2 (and I'm using the latest 2016.09.27 version of the plugin) but, as I expected, it has had no effect on this problem.

 

Thanks to anyone who can shed any light.

 

EDIT: I tried uninstalling and reinstalling, after first making sure all my other plugins were up to date, to no avail.

 

EDIT2: I haven't been able to resolve the problem so I've uninstalled the plugin again for now. If anyone has any suggestions or if the plugin is updated I'll reinstall and try again.

 

Link to comment

Preclear just doesn't seem to like me much lately...

 

I've launched a preclear on a drive that, admittedly, I've had other trouble with.

 

This time, I've started the preclear with the drive attached to a USB3.0 -> SATA adapter, and the drive is outside the case. I started the preclear on 10/21 at 06:41 local, and it seems to have hung after 7 hours of processing at 20% complete (first attachment). I kept seeing that it was running at 32MB/s and thought that was insanely slow, but didn't register that it was also only reporting 7 hours of processing time when it's been nearly 36 hours since I launched the preclear, therefore, I've just left it running while doing other stuff.

 

The log reports that it couldn't allocate memory and, therefore, there was a script error (second attachment). I've got 8GB of RAM in the box, but I do have a bunch of dockers running (see .sig).

 

A) It would be nice if the script could be made a bit more robust to capture these errors and not just hang. Easier said than done, I understand.

B) Is is possible to copy text from the log window? Firefox just doesn't seem to want to let me do that, hence the screen cap instead.

C) What do I need to do now? I presume that I need to just stop the preclear (though I had issues with that previously and ended up having to power off the machine with a long-press of the power button, and would like to avoid that again if possible).

 

Do I simply not have enough RAM to support the configuration I'm attempting to run?

I readily acknowledge that there could simply be issues with the drive, having had a bit of a smoke (see first link), but it does seem to have passed SMART testing. I'd like to make a rather thorough test of the drive before giving up on it, since it's passed HGST's tests, they won't RMA it, even through it's still under warranty.

 

This is also the second time that the log has been full of this:

tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"
tput: unknown terminal "screen"

That's only part of it, but that's the only thing in the log.

 

And, I cannot get the preclear to stop, clicking on the x doesn't do anything but force a page reload.

Link to comment

I don't know if it's got anything to do with it, but the 6.2.2 release notification came while this preclear was running. It was either the 6.2 or 6.2.1 release notification that showed up when my last preclear hung, too.

 

Maybe there's something in the update detection/notification routine that's breaking the preclear? I dunno, just a possibility.

 

I did reboot the server (with no issues this time), installed all updated updates, and my preclear is running along just fine, now.

Link to comment
  • Squid featured, unfeatured and pinned this topic
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.