I recently decided to switch from Windows to Unraid for a number of reasons, but mostly because it seemed like a simpler method of managing/protecting drives.

 

I have several drives in the array with parity fully calculated on the single parity drive. I have reconstructive writes turned on and the md_num_stripes parameter set to 4096 because I saw some posts saying increasing it could make things faster if you have more memory (I have 32gb). All drives are formatted with ZFS. There is also a pool of SSDs I intend to use for cache formatted with BTRFS, but I currently have all shares writing directly to the array since I am having to move a lot of data right now to put everything on the array initially.

 

Due to size disparity, only 1 drive is having anything written to it right now. This will presumably change as more is written to the array, but it's prioritizing the 8tb drive at the beginning. I have tried 3 different ways of writing, and all display this behavior to some extent. Copying over the network, it will actually time out regularly. I have also used Krusader and mc over ssh. mc definitely performs the best with it usually going several gb before dropping down to 0 for a bit and then resuming. Krusader will say Stalled as the copy status when this happens and it generally persists for longer than with mc.

 

What seems to be happening from monitoring the array's disk activity is that everything is writing just fine at full speed for a while. It properly reads from the disks that it isn't writing to and writes to the parity/data disks at the appropriate speed. Then it just falls off. It isn't a gradual decline or anything like that. It just writes appropriately for a while then stops for a while. I got a plugin to show me some graphs of activity over time, and during the "no activity" time, there appears for be a very small amount of activity on the array of between 5-20mbps while the transfer itself will be stalled out. During this stall time, there will be 2 and sometimes 3 random cpu cores at 100% utilization with the rest at minimal usage. This is compared to during peak transfer where no single core is stuck at 100% with the average of all 12 threads sitting around 30-40%.

 

The further into a large job I get, the more frequently it will happen. The memory is overclocked, but it's still under the spec of the memory (like overclocked from the default 2133mhz). I also ran memtest for about a day continuously before starting this whole process. The CPU is a Ryzen 5600X and I put it in eco mode in the bios.

pterbomb-misc-diagnostics-20241002-1619.zip

Disk1 is SMR, so just that can cause what you describe, also note that there's a known issue with zfs on the array and write speed performance, pools are not affected.

So it's better to have array drives formatted in xfs or btrfs?

4 minutes ago, cloaknsmoke said:

So it's better to have array drives formatted in xfs or btrfs?

At the moment yes.   

 

Whether that will stray true in the future is not clear, but I would not be surprised if it does due to the interaction of ZFS and the way Unraid handles the parity drive.

46 minutes ago, cloaknsmoke said:

So it's better to have array drives formatted in xfs or btrfs?

I'm hopeful that LT can fix the zfs issue for v7 stable, but it may not be an easy fix, it has to do with the way zfs interacts with the md driver, so it can take longer to fix it.

I reformatted all the array drives to XFS instead, and that appears to have fixed the issues based on some limited testing. Thank you for your help.