parity check 17 days?

[BDPM]

Parity check 1 time a month. They used to run about 12 to 15 hrs.  Now its saying 17 days to complete.

 

I dont even know where to begin looking. Any suggestions would be greatly appreciated 

server-diagnostics-20200908-0431.zip

[ChatNoir]

Is there something doing some heavy r/w operations on the array ?

Logically, this would slow down the parity check.

[BDPM]

35 minutes ago, ChatNoir said:

Is there something doing some heavy r/w operations on the array ?

Logically, this would slow down the parity check.

Honestly, i dont know.

 

Mover runs on a schedule I think. The daily data backup runs and is completed as normal.  Weekly appdata backup isnt running. 

 

It has run before in 12hr. I dunno what has been changed.

[ChatNoir]

Can you post your diagnostics in your next post ? (tools / diagnostics)

 

Maybe someone can see if there is something special happening.

[JorgeB]

Nothing jumps out, is it still going slow? Maybe a disk getting some slow sector zones, you could run the speedtest docker to see if the disks are performing normally.

[BDPM]

27 minutes ago, JorgeB said:

Nothing jumps out, is it still going slow? Maybe a disk getting some slow sector zones, you could run the speedtest docker to see if the disks are performing normally.

yes.  Still horribly slow.

 

Ill try the speed test docker

 

Total size:6 TB

Elapsed time:10 minutes

Current position:2.60 GB (0.0 %)

Estimated speed:3.9 MB/sec

Estimated finish:17 days, 14 hours, 29 minutes

Sync errors corrected:0

 

I tried turning off all dockers and the vm that was running.  It helped.  Got up to around 9MB/sec LOL.. I canceled the check.  Restarted the server.  Started check again.  Above is where I stand now.

 

 

[BDPM]

Thanks all..

 

Diskspeed docker revealed problem below.

 

So it looks like disk 3, my backup drive, is failing.  That sucks, but I have critical data remotely backed up as well as locally.  Non critical data will get backed up during the daily backup.

 

My question now is, should I just replace the drive and let backup scripts do their job, or replace the drive and re-silver it?

 

 

Edited September 8, 2020 by BDPM

[JorgeB]

1 minute ago, BDPM said:

So it looks like disk 3, my backup drive, is failing. 

Yep.

[JorgeB]

2 minutes ago, BDPM said:

or replace the drive and re-silver it?

Replace as soon as you can.

[BDPM]

2 minutes ago, JorgeB said:

Replace as soon as you can.

Drive will be here in a day or two.

 

Resilver or let backup scripts do it?  
 

Disk 3 is only for backup purposes.  

[JorgeB]

You'll need to rebuilt it anyway, not sure what script you're using, but you can decide if it's worth running it also.

[BDPM]

3 hours ago, JorgeB said:

You'll need to rebuilt it anyway, not sure what script you're using, but you can decide if it's worth running it also.

Custom Backup scripts.  I will re-silver and then run the scripts.

 

Thanks

[trurl]

Not familiar with that use of the word resilver, but I guess it isn't a typo since it was used repeatedly. @JorgeB seems to interpret it as "rebuild". Do I have a language barrier?

[Gragorg]

1 hour ago, trurl said:

Not familiar with that use of the word resilver, but I guess it isn't a typo since it was used repeatedly. @JorgeB seems to interpret it as "rebuild". Do I have a language barrier?

Just an uncommon term for rebuilding.  Goes back to Antique mirrors as they go "tarnished" or damaged you could restore them by re-silvering them.

[BDPM]

1 hour ago, trurl said:

Not familiar with that use of the word resilver, but I guess it isn't a typo since it was used repeatedly. @JorgeB seems to interpret it as "rebuild". Do I have a language barrier?

I think i picked up the term resilver around ZFS or Freenas days.  But yes, rebuild, same thing. 

Edited September 8, 2020 by BDPM

[trurl]

As an amateur astronomer I am familiar with resilvering mirrors.

[JorgeB]

Yeah, resilvering is what zfs uses when rebuilding/repairing a disk, why I assumed he meant rebuilding.

 

9 hours ago, trurl said:

As an amateur astronomer

I love astronomy, I wish I had better conditions in my home for that, I did get a telescope some years ago, but basically only looked at the moon and a couple of planets, still worth it seeing Jupiter's moons and Saturn's rings.