Preclear plugin


Recommended Posts

Just noticed this in the log. 

Mar 30 09:29:40 MiniVault kernel: ACPI Error: Method parse/execution failed \_SB.PCI0.SAT0.PRT0._GTF, AE_NOT_FOUND (20170728/psparse-550)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20170728/psargs-364)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: Method parse/execution failed \_SB.PCI0.SAT0.PRT3._GTF, AE_NOT_FOUND (20170728/psparse-550)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20170728/psargs-364)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: Method parse/execution failed \_SB.PCI0.SAT0.PRT0._GTF, AE_NOT_FOUND (20170728/psparse-550)
Mar 30 09:29:40 MiniVault kernel: ata4.00: configured for UDMA/133
Mar 30 09:29:40 MiniVault kernel: ata4: EH complete
Mar 30 09:29:40 MiniVault kernel: ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20170728/psargs-364)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: Method parse/execution failed \_SB.PCI0.SAT0.PRT2._GTF, AE_NOT_FOUND (20170728/psparse-550)
Mar 30 09:29:40 MiniVault kernel: ata1.00: configured for UDMA/133
Mar 30 09:29:40 MiniVault kernel: ata1: EH complete
Mar 30 09:29:40 MiniVault kernel: ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20170728/psargs-364)
Mar 30 09:29:40 MiniVault kernel: ACPI Error: Method parse/execution failed \_SB.PCI0.SAT0.PRT2._GTF, AE_NOT_FOUND (20170728/psparse-550)
Mar 30 09:29:40 MiniVault kernel: ata3.00: configured for UDMA/133
Mar 30 09:29:40 MiniVault kernel: ata3: EH complete
Mar 30 09:29:41 MiniVault kernel: ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20170728/psargs-364)
Mar 30 09:29:41 MiniVault kernel: ACPI Error: Method parse/execution failed \_SB.PCI0.SAT0.PRT1._GTF, AE_NOT_FOUND (20170728/psparse-550)
Mar 30 09:29:41 MiniVault kernel: ata2.00: configured for UDMA/133
Mar 30 09:29:41 MiniVault kernel: ata2: EH complete

 

Link to comment

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon

and @Squid have been working behind the scenes to try and clean up, and obviously @Frank1940 has put a great deal of work in here.  There are many others too.

 

The unsung heroes imho....

 

This has highlighted how fragile things are, and those of us that maintain plugins/containers have to maintain these things and I feel we owe a responsibility to do so, or at least state that life has taken over as PhAzE did when he wound things down.

Edited by CHBMB
  • Like 5
  • Upvote 3
Link to comment
17 hours ago, CHBMB said:

I think Thanks should be directed at all those members that have been dealing with the mess the last few months. @dlandon

and @Squid have been working behind the scenes to try and clean up, and obviously @Frank1940 has put a great deal of work in here.  There are many others too.

 

The unsung heroes imho....

 

This has highlighted how fragile things are, and those of us that maintain plugins/containers have to maintain these things and I feel we owe a responsibility to do so, or at least state that life has taken over as PhAzE did when he wound things down.

 

For sure, many thanks and kudos to @dlandon @Squid @Frank1940 and those behind the scenes that worked on this!  

Link to comment
On 3/30/2018 at 10:58 AM, Frank1940 said:

It can also be caused by tying all of the SATA cables together to make the inside of the case look 'neat'!  (Tying cables together can cross-talk between the cables if the cables are not shielded and 99%+ of all cables today are not!)   

 

This is what I was just thinking; maybe all cables are tied together or are running too closely to other components or power wires.

 

I miss read the original post and thought it was one CRC on one drive, not one on all four.  You could have not so great SATA cables or a marginal controller on the motherboard.  

 

Once you you get the array built I would run two or three parity checks to make sure the numbers don’t rise.

 

Also, if the SATA power cable terminator is not so great or poorly connected, or if you don’t have a large enough power supply you can sometimes get these error.

 

I once racked up thousands of CRC’s on four drives connected to the same 4-way power splitter... and discovered my SATA powered splitter was not plugged in all the was to the power supply SATA power connector.  Unplug and replug and problem was gone.  Many years later no CRC has never increased.

 

craigr

Link to comment
On 05/02/2017 at 3:57 PM, gfjardim said:

Another functionality I added is "Erase and Clear the Disk". This is a "partial" fulfillment of dgaschk's request, because it won't intercalate between random and zero data writing. Instead it will erase the disk with random data and then write zeroes on it.

 

 

Can you tell me what this does as opposed to just clearing? Is it a more secure method of erasing a disk because it firstly overwrites all the sectors before setting all bits to zero?

Link to comment
21 hours ago, craigr said:

 

This is what I was just thinking; maybe all cables are tied together or are running too closely to other components or power wires.

 

I miss read the original post and thought it was one CRC on one drive, not one on all four.  You could have not so great SATA cables or a marginal controller on the motherboard.  

 

Once you you get the array built I would run two or three parity checks to make sure the numbers don’t rise.

 

Also, if the SATA power cable terminator is not so great or poorly connected, or if you don’t have a large enough power supply you can sometimes get these error.

 

I once racked up thousands of CRC’s on four drives connected to the same 4-way power splitter... and discovered my SATA powered splitter was not plugged in all the was to the power supply SATA power connector.  Unplug and replug and problem was gone.  Many years later no CRC has never increased.

 

craigr

 

It's a small form factor ITX build, so yes, cable management is tight with SATA and power cables in close proximity of one another.  I used a SATA power cable adapter as it was easier to route than the PSU SATA cable, but I can remove it and reroute things.  I also ordered shorter SATA data cables to help with the rats nest.  I'm using an EVGA G3 650w power supply.  

 

The preclear completed successfully and no additional errors were found; however, here are the UDMA CRC error counts:

  • Parity - 2
  • Disk 1 - 3
  • Disk 2 - 2
  • Disk 3 -1

After 2 Parity checks, the CRC errors remained the same... no other errors found.    

Link to comment
On 3/30/2018 at 11:01 AM, dchamb said:

 

My monthly parity check is scheduled for tomorrow so I should know before the weekend is over. Thanks!

Monthly parity check shows no errors! 

 

Event: unRAID Parity check

Subject: Notice [TOWER] - Parity check finished (0 errors)

Description: Duration: 23 hours, 49 minutes, 14 seconds. Average speed: 139.9 MB/s

Importance: normal

 

Plus that 12TB WD Gold drive has improved the elapsed time of the parity check. I've been very happy with those drives.

 

Dale

Link to comment
1 hour ago, luisv said:

 

It's a small form factor ITX build, so yes, cable management is tight with SATA and power cables in close proximity of one another.  I used a SATA power cable adapter as it was easier to route than the PSU SATA cable, but I can remove it and reroute things.  I also ordered shorter SATA data cables to help with the rats nest.  I'm using an EVGA G3 650w power supply.  

 

The preclear completed successfully and no additional errors were found; however, here are the UDMA CRC error counts:

  • Parity - 2
  • Disk 1 - 3
  • Disk 2 - 2
  • Disk 3 -1

After 2 Parity checks, the CRC errors remained the same... no other errors found.    

Avoid running cables parallel to each other.  Having them cross over each other isn’t usually an issue, but if run side by side the EM fields can cause errors.

 

craigr

Link to comment
On 3/29/2018 at 4:54 PM, gfjardim said:

 

 

Sorry for the delay, guys. I'm very short on time so I had to decrease the level of support dedicated to this plugin. If someone could verify if the script is working fine with v6.5 I do appreciate that.

 

Results of attempted preclear on a WD ST8000AS0002 using plugin version 2018.03.29 (default script) on unRAID 6.3.5

 

Seems to have stalled on step 5 post read.  Log and diagnostics attached.

RAID-01-preclear.disk-20180401-1354.zip

raid-01-diagnostics-20180401-1357.zip

Link to comment
59 minutes ago, dabl said:

 

Results of attempted preclear on a WD ST8000AS0002 using plugin version 2018.03.29 (default script) on unRAID 6.3.5

 

Seems to have stalled on step 5 post read.  Log and diagnostics attached.

RAID-01-preclear.disk-20180401-1354.zip

raid-01-diagnostics-20180401-1357.zip

 

 

Your hard drive has tons of these errors on the syslog:

 

Quote

Mar 31 23:14:52 raid-01 kernel: sd 6:0:0:0: attempting task abort! scmd(ffff880084335b00)
Mar 31 23:14:52 raid-01 kernel: sd 6:0:0:0: tag#29 CDB: opcode=0x0 00 00 00 00 00 00
Mar 31 23:14:52 raid-01 kernel: scsi target6:0:0: handle(0x0009), sas_address(0x4433221102000000), phy(2)
Mar 31 23:14:52 raid-01 kernel: scsi target6:0:0: enclosure_logical_id(0x500605b0026aa190), slot(1)
Mar 31 23:14:52 raid-01 kernel: sd 6:0:0:0: task abort: SUCCESS scmd(ffff880084335b00)

 

Don't know what that error mean, but apparently those aborts are interfering with the preclear operation.

Link to comment
3 hours ago, trurl said:

Just make sure they aren't too short. If a cable barely reaches then it is going to have forces acting on it which can affect the connection.

 

Current cables are 18" long, going with 8 - 10" instead, but yes, I'll make sure there's no strain on the connectors.  Thanks!

Link to comment
17 minutes ago, gfjardim said:

 

 

Your hard drive has tons of these errors on the syslog:

 

 

Don't know what that error mean, but apparently those aborts are interfering with the preclear operation.

 

Ah, thanks looking at that now.  I can't tell from the entries if it's clear the errors are definitely related to the drive in question but of course I don't doubt they are.

 

I ran both a short and extended SMART self-test on the drive with no errors prior to running preclear.

 

I wonder if it might possible be controller rather than drive related.

Link to comment

Y

1 hour ago, dabl said:

 

Ah, thanks looking at that now.  I can't tell from the entries if it's clear the errors are definitely related to the drive in question but of course I don't doubt they are.

 

I ran both a short and extended SMART self-test on the drive with no errors prior to running preclear.

 

I wonder if it might possible be controller rather than drive related.

 

It's the drive in question, scsi target6:0:0 is disk sdg.

 

Yep, maybe its a controller issue.

Link to comment
3 hours ago, gfjardim said:

 

 

Your hard drive has tons of these errors on the syslog:

 

 

Don't know what that error mean, but apparently those aborts are interfering with the preclear operation.

 

I noticed before all these errors you excerpted the first sign of a problem in the syslog was well before this and appears to be related to the preclear plugin.

 

Perhaps these PHP warnings are non fatal and not directly responsible but it does look suspicious.

Mar 31 09:47:51 raid-01 rc.diskinfo[9321]: PHP Warning: Missing argument 2 for force_reload() in /etc/rc.d/rc.diskinfo on line 691
Mar 31 09:47:51 raid-01 rc.diskinfo[9321]: SIGHUP received, forcing refresh of disks info.
Mar 31 09:47:54 raid-01 rc.diskinfo[9321]: PHP Warning: Missing argument 2 for force_reload() in /etc/rc.d/rc.diskinfo on line 691
Mar 31 09:47:54 raid-01 rc.diskinfo[9321]: SIGHUP received, forcing refresh of disks info.

Directly after these lines the first attempting task abort! entry appears which then repeats for the next several hours.

Mar 31 09:49:08 raid-01 kernel: sd 6:0:0:0: attempting task abort! scmd(ffff8801100fd080)
Mar 31 09:49:08 raid-01 kernel: sd 6:0:0:0: tag#0 CDB: opcode=0x0 00 00 00 00 00 00
Mar 31 09:49:08 raid-01 kernel: scsi target6:0:0: handle(0x0009), sas_address(0x4433221102000000), phy(2)
Mar 31 09:49:08 raid-01 kernel: scsi target6:0:0: enclosure_logical_id(0x500605b0026aa190), slot(1)
Mar 31 09:49:08 raid-01 kernel: sd 6:0:0:0: task abort: SUCCESS scmd(ffff8801100fd080)
Mar 31 09:49:25 raid-01 autofan: Highest disk temp is 31C, adjusting fan speed from: 105 (41% @ 1454rpm) to: 135 (52% @ 1795rpm)
Mar 31 09:50:32 raid-01 kernel: sd 6:0:0:0: attempting task abort! scmd(ffff88007b757980)

I powered down, moved the drive to another slot and resumed the previous preclear session (nice feature!).  So far so good.  I'll probably run another full preclear in one or both slots to try and determine if that's the problem.

Link to comment

So I'm a bit confused. If I want to use this updated plugin with 6.5, do I still need to download either the Joe L or bjp999 script and then patch it with the Frank1940 link a couple pages back? The community apps page says the script is not included.

edit: I'm also given the option of a gfjardim beta script. Has this been patched already?

Edited by mfort312
Link to comment
6 hours ago, mfort312 said:

So I'm a bit confused. If I want to use this updated plugin with 6.5, do I still need to download either the Joe L or bjp999 script and then patch it with the Frank1940 link a couple pages back? The community apps page says the script is not included.

edit: I'm also given the option of a gfjardim beta script. Has this been patched already?

The gfjardim beta script does not require patching. If you want to use either of the others they probably will.

Link to comment

Hi everyone, for some reason pre-clear is using up most of my log space. Has anyone experienced this?

 

PS. I'm running version 6.3.5. Not updated yet since I wasn't so confident on stability and such.

root@unRAID:~# df -h /var/log
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           384M  356M   29M  93% /var/log
root@unRAID:~# du -sm /var/log/*
1       /var/log/PhAzE-Logs
1       /var/log/apcupsd.events
1       /var/log/apcupsd.events.1
1       /var/log/apcupsd.events.2
1       /var/log/apcupsd.events.3
1       /var/log/apcupsd.events.4
0       /var/log/btmp
0       /var/log/btmp.1
0       /var/log/cron
0       /var/log/debug
1       /var/log/dmesg
2       /var/log/docker.log
1       /var/log/faillog
1       /var/log/lastlog
0       /var/log/libvirt
0       /var/log/maillog
0       /var/log/messages
0       /var/log/nfsd
3       /var/log/packages
0       /var/log/plugins
351     /var/log/preclear.disk.log
1       /var/log/removed_packages
1       /var/log/removed_scripts
0       /var/log/samba
1       /var/log/scripts
0       /var/log/secure
0       /var/log/setup
0       /var/log/spooler
1       /var/log/syslog
2       /var/log/syslog.1
1       /var/log/wtmp

This is what I can see when I do a tail, its adding it every 10 seconds or something:


 

Thu Apr 5 17:51:32 EDT 2018: get_content Finished: 0
Thu Apr 5 17:51:43 EDT 2018: Starting get_content: 0
Thu Apr 5 17:51:43 EDT 2018:
Disks:
+ /dev/sdd => /dev/disk/by-id/ata-WDC_WD30EFRX-68AX9N0_WD-WMC1T2574503
+ /dev/sdc => /dev/disk/by-id/ata-WDC_WD30EFRX-68AX9N0_WD-WMC1T2277054
+ /dev/sdb => /dev/disk/by-id/ata-WDC_WD30EFRX-68AX9N0_WD-WMC1T2737524
+ /dev/sdf => /dev/disk/by-id/ata-WDC_WD60EFRX-68L0BN1_WD-WX11DC57SFX7
+ /dev/sdg => /dev/disk/by-id/ata-WDC_WD50EFRX-68L0BN1_WD-WXB1HB4KUF1J
+ /dev/sde => /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N0303087
+ /dev/sdh => /dev/disk/by-id/ata-Samsung_SSD_850_EVO_250GB_S21NNXBGA75793K
+ /dev/sda => /dev/disk/by-id/usb-Kingston_DT_Micro_1C6F654E4910BD30C95403FF-0:0
Thu Apr 5 17:51:43 EDT 2018:
unRAID Serials:
+ 0951-168A-4910-BD30C95403FF
+ WDC_WD60EFRX-68L0BN1_WD-WX11DC57SFX7
+ WDC_WD30EFRX-68AX9N0_WD-WMC1T2277054
+ WDC_WD30EFRX-68AX9N0_WD-WMC1T2574503
+ WDC_WD30EFRX-68EUZN0_WD-WCC4N0303087
+ WDC_WD30EFRX-68AX9N0_WD-WMC1T2737524
+ WDC_WD50EFRX-68L0BN1_WD-WXB1HB4KUF1J
+ Samsung_SSD_850_EVO_250GB_S21NNXBGA75793K
+ Kingston_DT_Micro_1C6F654E4910BD30C95403FF-0:0
Thu Apr 5 17:51:43 EDT 2018:
unRAID Disks:
+ /dev/disk/by-id/ata-WDC_WD60EFRX-68L0BN1_WD-WX11DC57SFX7
+ /dev/disk/by-id/ata-WDC_WD30EFRX-68AX9N0_WD-WMC1T2277054
+ /dev/disk/by-id/ata-WDC_WD30EFRX-68AX9N0_WD-WMC1T2574503
+ /dev/disk/by-id/ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N0303087
+ /dev/disk/by-id/ata-WDC_WD30EFRX-68AX9N0_WD-WMC1T2737524
+ /dev/disk/by-id/ata-WDC_WD50EFRX-68L0BN1_WD-WXB1HB4KUF1J
+ /dev/disk/by-id/ata-Samsung_SSD_850_EVO_250GB_S21NNXBGA75793K
+ /dev/disk/by-id/usb-Kingston_DT_Micro_1C6F654E4910BD30C95403FF-0:0
Thu Apr 5 17:51:43 EDT 2018: benchmark: get_unasigned_disks() took 0.004354s.
Thu Apr 5 17:51:43 EDT 2018: benchmark: get_all_disks_info() took 0.004442s.

 

Edited by Necrotic
Link to comment
On 4/5/2018 at 5:53 PM, Necrotic said:

Hi everyone, for some reason pre-clear is using up most of my log space. Has anyone experienced this?

 

PS. I'm running version 6.3.5. Not updated yet since I wasn't so confident on stability and such.

 

 

For what it's worth below are my results running unRAID Server OS 6.3.5 with preclear 2018.03.29

 

Per above previous posts these results are after I ran 1 preclear cycle that stalled, then resumed the session which ran to completion.

 

root@raid-01:~# df -h /var/log
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           128M  3.3M  125M   3% /var/log

 

root@raid-01:~# du -h /var/log
0       /var/log/setup/tmp
0       /var/log/setup
232K    /var/log/scripts
0       /var/log/samba/cores/winbindd
0       /var/log/samba/cores/smbd
0       /var/log/samba/cores/nmbd
0       /var/log/samba/cores
0       /var/log/samba
0       /var/log/removed_scripts
4.0K    /var/log/removed_packages
0       /var/log/plugins
2.3M    /var/log/packages
0       /var/log/nfsd
0       /var/log/libvirt/uml
0       /var/log/libvirt/lxc
0       /var/log/libvirt/qemu
0       /var/log/libvirt
3.3M    /var/log

Link to comment
Just now, dabl said:

 

For what it's worth below are my results running unRAID Server OS 6.3.5 with preclear 2018.03.29

 

Per above previous posts these results are after I ran 1 preclear cycle that stalled, then resumed the session which ran to completion.

 

 

No I mean I get that spam without running pre-clear. Its just constantly adding to my log all the time.

Link to comment
On 4/1/2018 at 6:34 PM, dabl said:

 

I noticed before all these errors you excerpted the first sign of a problem in the syslog was well before this and appears to be related to the preclear plugin.

 

Perhaps these PHP warnings are non fatal and not directly responsible but it does look suspicious.


Mar 31 09:47:51 raid-01 rc.diskinfo[9321]: PHP Warning: Missing argument 2 for force_reload() in /etc/rc.d/rc.diskinfo on line 691
Mar 31 09:47:51 raid-01 rc.diskinfo[9321]: SIGHUP received, forcing refresh of disks info.
Mar 31 09:47:54 raid-01 rc.diskinfo[9321]: PHP Warning: Missing argument 2 for force_reload() in /etc/rc.d/rc.diskinfo on line 691
Mar 31 09:47:54 raid-01 rc.diskinfo[9321]: SIGHUP received, forcing refresh of disks info.

 

 

So gfjardim, can you please comment on these these PHP warnings?  

 

Is this something you can fix?

 

 

Link to comment
6 minutes ago, Necrotic said:

 

No I mean I get that spam without running pre-clear. Its just constantly adding to my log all the time.

 

Ok, good to clarify that. 

 

Perhaps try an unraid start/stop and/or reboot and if it persists then try a preclear plugin uninstall/re-install?

Link to comment
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.