After rebuilding my Unraid array using all SSD drives, initial generation of parity ran at around only 25MB/sec. I was expecting a lot more speed than that. Parity checks don't run much faster (maybe around 35MB/sec).

The only thing that runs quickly is transfers from the SSD array to an attached USB hard drive (as an Unassigned Device) -- that's about 10 times faster, ~250MB/sec.

Network transfers from my array are now much slower than when the array was a pure HD array, from either the SSD array or from the attached USB hard drive, at only 25-35MB/sec. I have a backup Unraid array that's still built solely from HDs. I get network transfer speeds of around 190MB/sec using that array.

If this were simply due to having sucky SSDs, I shouldn't be able to get that ~250MB/sec SSD->HD internal transfer rate, and network transfers from the HD shouldn't be slower than they had been before either. If this was just a network problem parity building and parity checks shouldn't be so slow, since those don't require traffic over the network.

This doesn't make a lot of sense.

 

I'm wondering if I've accidentally messed up my configuration, or if I should configure things differently for SSDs than I used to with do with HDs.

I've found an earlier thread on this topic here: 

 

 

...but I'm not sure I understand the issues involved in that thread.

I set up my new SSD array the same way I'd set up HD arrays before, albeit this is the first time I've used 2 parity drives instead of just one. There are 9 data drives, all XFS formatted. Each drive is 4TB, for a total of 36TB storage.

Any ideas about what, if anything, I might have screwed up, or could improve?

terminus-diagnostics-20231011-2141.zip

Edited October 12, 20232 yr by RasterEyes

Could power consumption be an issue? If my SSDs are over taxing my power supply, how would I know? Would the symptoms be outright crashing and failure, or could the slow performance I'm seeing be a symptom of needing more power?

 

I've already thrown away the data sheets that were packaged with my drives, and I have figured out where to find that info online yet.

Start a parity check, wait 2 minutes and download and post diags

Just to make a liar out of me, this time a parity check is running at lot faster, around 240MB/sec. There's also a backup to my other array running now (and while these diags were collected) that was still creeping along at around 25MB/sec.

terminus-diagnostics-20231012-0512.zip

If the problem is just with LAN transfers it could be a network issue, run iperf form one server to the other.

iperf is reporting a speed of around 935Mbits/sec in either direction, so pretty close to the full bandwidth of my wired 1G network.

Make sure nothing else is using the array during the transfer, if a local transfer is fast it should also be fast over LAN, assuming LAN is fine, and since you appear to be having intermittent issues, like with the parity check there could be other issues.

Stranger and stranger... at the very same time my rsync back up from my main array to my backup is still crawling along at around 25MB/sec, I was able to copy an individual file at around 100MB/sec. I know rsync has a bandwidth-limiting option, but I'm not using it.

22 minutes ago, RasterEyes said:

I know rsync has a bandwidth-limiting option, but I'm not using it.

Are you using the compression flag (-z)? That can slowdown LAN transfers a lot.

Nope, just running rsync the same way I did before.

 

rsync -rltOuv --exclude={'*.bak.mkv','*.upd.mkv','*.aac'} --delete --force /mnt/user/video/ [email protected]:/mnt/user/video/

 

Now this is really crazy. I'm getting a much faster rsync backup like this:

rsync -rltOuv --exclude={'*.bak.mkv','*.upd.mkv','*.aac'} --delete --force /mnt/user/video/ /mnt/remotes/TRANTOR_video/

 

...accessing the backup array via SMB instead of using the rsync protocol. Something wrong with rsync on the backup array???

18 hours ago, RasterEyes said:

If this were simply due to having sucky SSDs

I think so.

 

 

Edited October 12, 20232 yr by Vr2Io

42 minutes ago, Vr2Io said:

19 hours ago, RasterEyes said:

If this were simply due to having sucky SSDs

I think so.

I'm not sure I'm getting your meaning (if you're saying you think they do suck, or they don't). The review sounded positive, however, although it was about a much smaller 512MB drive.

At any rate, even though things still seem weird (like using SMB being so much faster than rsync over SSH), I'm pretty sure I've safely moved beyond worrying about the quality of the new SSD drives being an issue.

I mean SSD were suck, you can try use DD to make some test on individual SSD. Suck SSD usually with very low writing speed and always busy on that, but after writing complete or some period, then it will resume normal read / write speed again. I also have some suck SATA SSD, no matter what you do, they still suck.

 

If rsync in local transfer still slow, then it is storage issue.

 

I also implement some SATA SSD / NVMe in array / pool / UD, they generally work well.

Edited October 12, 20232 yr by Vr2Io

Ah-hah!

 

RECONSTRUCT WRITE!!!

 

That's the secret to getting great performance out of my SSDs. By switching to reconstruct write for generating parity instead of auto (which must have always auto-selected read/modify/write), things are going MUCH faster. Along with upgrading my network to 2.5G, I'm getting nearly 250MB/s, ten times the speed I started with.

No need to dis the cheap Fikwot drives either. Use the right settings, and they work great.