Issue with parity sync process.

[NetPlex]

I configured my array with parity2 16 TB disks with data disks  4 16 TB  and 9 14 TB disks.

2 14 TB disks as cache. When I started the array, Parity sync started with 291 days which ridiculous. 

 

I'm afraid that I bought the pro license unnecessarily. I was using Truenas which was so good even though its free.

 

So any help on this will be appriciated. 

[Kilrah]

Need to know more about your setup. Parity build will read from all data disks at the same time (and write to both parity ones), you need an appropriate hardware setup to not run into bandwidth bottlenecks.

 

Using HDDs as cache makes little sense, that's a job for SSDs.

Edited July 27, 2022 by Kilrah

[NetPlex]

Thanks for quick response. 

 

I know the its no sense using HDD for cache. My thought to make this cache is to transfer the data from my old Trunas server to new unraid server. If I try to write the data to parity enabled array. Its too slow like 40 MBps. So thought that I can 1st transfer the data to cache with 10 GB network port with some 350 Mbps and transfer the data to array.

 

My Setup contains AMD Ryzen 9 3950x with 64 GB 3600 DDR4 RAM and Nvidia Quadro P4000 GPU.

 

SCSI Raid Card ASR-78165 (6 GBPS) in HBA mode.

 

 

[Squid]

You'd need to post your diagnostics

[NetPlex]

Jul 27 14:01:18 NetPlex avahi-daemon[10089]: New relevant interface virbr0.IPv4 for mDNS.
Jul 27 14:01:18 NetPlex avahi-daemon[10089]: Registering new address record for 192.168.122.1 on virbr0.IPv4.
Jul 27 14:01:18 NetPlex dnsmasq[10842]: started, version 2.86 cachesize 150
Jul 27 14:01:18 NetPlex dnsmasq[10842]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
Jul 27 14:01:18 NetPlex dnsmasq-dhcp[10842]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h
Jul 27 14:01:18 NetPlex dnsmasq-dhcp[10842]: DHCP, sockets bound exclusively to interface virbr0
Jul 27 14:01:18 NetPlex dnsmasq[10842]: reading /etc/resolv.conf
Jul 27 14:01:18 NetPlex dnsmasq[10842]: using nameserver 75.75.75.75#53
Jul 27 14:01:18 NetPlex dnsmasq[10842]: using nameserver 75.75.76.76#53
Jul 27 14:01:18 NetPlex dnsmasq[10842]: read /etc/hosts - 2 addresses
Jul 27 14:01:18 NetPlex dnsmasq[10842]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Jul 27 14:01:18 NetPlex dnsmasq-dhcp[10842]: read /var/lib/libvirt/dnsmasq/default.hostsfile
Jul 27 14:01:19 NetPlex avahi-daemon[10089]: Service "NetPlex" (/services/sftp-ssh.service) successfully established.
Jul 27 14:01:19 NetPlex avahi-daemon[10089]: Service "NetPlex" (/services/smb.service) successfully established.
Jul 27 14:01:19 NetPlex avahi-daemon[10089]: Service "NetPlex" (/services/ssh.service) successfully established.
Jul 27 14:01:50 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:01:50 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:19 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:19 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:44 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:44 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host adapter abort request.
Jul 27 14:02:47 NetPlex kernel: aacraid: Outstanding commands on (2,1,0,0):
Jul 27 14:02:47 NetPlex kernel: aacraid: Host bus reset request. SCSI hang ?
Jul 27 14:02:47 NetPlex kernel: aacraid 0000:03:00.0: outstanding cmd: midlevel-0
Jul 27 14:02:47 NetPlex kernel: aacraid 0000:03:00.0: outstanding cmd: lowlevel-0
Jul 27 14:02:47 NetPlex kernel: aacraid 0000:03:00.0: outstanding cmd: error handler-31
Jul 27 14:02:47 NetPlex kernel: aacraid 0000:03:00.0: outstanding cmd: firmware-0
Jul 27 14:02:47 NetPlex kernel: aacraid 0000:03:00.0: outstanding cmd: kernel-0
Jul 27 14:53:22 NetPlex nginx: 2022/07/27 14:53:22 [error] 2663#2663: *29311 upstream timed out (110: Connection timed out) while reading upstream, client: 10.0.0.34, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket:/update.htm", host: "10.0.0.126", referrer: "http://10.0.0.126/Main"
Jul 27 15:27:36 NetPlex nginx: 2022/07/27 15:27:36 [error] 2663#2663: *47513 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.0.0.34, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket/update.htm", host: "10.0.0.126", referrer: "http://10.0.0.126/Main"
Jul 27 15:39:45 NetPlex nginx: 2022/07/27 15:39:45 [error] 2663#2663: *53937 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.0.0.34, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket/update.htm", host: "10.0.0.126", referrer: "http://10.0.0.126/Main"
Jul 27 15:53:37 NetPlex shutdown[22682]: shutting down for system reboot
Jul 27 16:04:17 NetPlex nginx: 2022/07/27 16:04:17 [error] 2663#2663: *62953 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.0.0.34, server: , request: "POST /webGui/include/Boot.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock", host: "10.0.0.126", referrer: "http://10.0.0.126/Dashboard"
 

[Kilrah]

Post the full diagnostics zip, but this already seems to show controller / drive communication issues. What is connected to the mobo ports and what to the HBA?

 

Are you being careful of not accessing the array during the parity build?

[NetPlex]

Here is the full log.

netplex-syslog-20220727-2357.zip

[trurl]

That is not diagnostics. The word diagnostics appearing in every post that has requested your diagnostics is a link explaining how to get your diagnostics.

[trurl]

3 hours ago, NetPlex said:

I know the its no sense using HDD for cache. My thought to make this cache is to transfer the data from my old Trunas server

Cache, even SSD, just gets in the way for initial data load unless it has the capacity for the entire transfer since you eventually have to pause so it can be moved to the array. So you wind up needing to wait for the array anyway.

 

It is impossible to move from fast cache (even HDD) to slower array as fast as you can write to cache.

 

Better idea is to not install and build parity until after initial data load has been done.

[NetPlex]

I tried that too 1st I loaded the data and then added parity. For sync it shows around 9000 days.

[trurl]

Still no diagnostics

[trurl]

20 minutes ago, NetPlex said:

I tried that too 1st I loaded the data and then added parity. For sync it shows around 9000 days.

The point was to make initial data load faster since parity isn't involved. Remains to be discovered what is the problem with your parity sync.

[NetPlex]

I shutdown the server. I'm restarting once done I can share

[NetPlex]

netplex-diagnostics-20220727-2136.zip

[trurl]

All of your drives have disconnected.

 

RAID controllers are NOT recommended.

 

Are you sure you have enough power?

 

 

[NetPlex]

for my drives enclosures i'm using Corsair 1050W and for server 750W psu

[NetPlex]

So How can I connect 17 drives with out raid controller

[Kilrah]

It's not that you don't want "a controller", it's that you don't want "a RAID controller". Should be a controller that can expose the disks directly to the host, often referred to as "IT mode". 

 

3 hours ago, NetPlex said:

I tried that too 1st I loaded the data and then added parity. For sync it shows around 9000 days.

Parity build with a correct setup and the kind of drives you have should take at most about 2 days if the array is left alone during build.

[ChatNoir]

1 hour ago, NetPlex said:

So How can I connect 17 drives with out raid controller

[trurl]

5 hours ago, NetPlex said:

for my drives enclosures i'm using Corsair 1050W and for server 750W psu

Does that enclosure have separate connections to the server for each drive? 

[NetPlex]

Yes I'm using SFF-8644 to SFF-8644 Cables, SFF-8644 to SFF-8643 Adapter and SFF-8643 to sata cables