[6.8.3] Speed bottlenecks?


Recommended Posts

I'm relatively new to Unraid (and Linux) so be gentle. As a photographer and always in need of data storage, I decided to replace my 2 old Synology's with one big server. Might keep one just to have another backup together with one that is not in my home.

 

I upgraded my network with 2 Netgear 10GbE switches (both with 2x 10GbE ports). I am still on cat6 cables but plan to upgrade that to 6A, just to rule out problems there. My hardware is a Ryzen 5 3600, 16GB RAM, Asrock Rack X470D4U2-2T, a HBA from Broadcom 9400-16i, a Nvidia 1660GTX for the Plex transcoding, a 960 Pro 512GB as cache and for now 4x WD Gold 14TB's. So I might expect a pretty decent performing server, right?

 

When I test speeds with iPerf3 and Diskspeed everything seems normal. I get 3.9Gbit/s network speeds (not full full speed but decent enough considering the length and being cat6) and my disks seem normal with 255MB/s (with no drops) and 1.1Gbit/s for the cache drive.

 

When I move something from my computer (SMB shares) to the server (share with cache only) I'm averaging 50~70MB/s. The parity sync is at ~90MB/s. I get the same speeds when I pause the sync. When I move something to the array I get like 140MB/s in the beginning (first 30%) just to drop down to 50~70MB/s. Turbo write is on. I'm not expecting GB/s here but this just seem slow?

I have 2 10GbE ports and one Gbit port on the motherboard but I can only use one 10GbE for now. I'm not going to invest into a new very expensive 10GbE switch just to get "under Gbit" speeds. So I'm kinda out of ideas for now.

 

Not sure if there's something wrong in the hardware (RAM? Memtest seemed fine) or network settings (which I don't understand fully, bonding is on but I'm only running one port?) so I turn to the forum. I included the diagnosis file. 

marcel-diagnostics-20200323-1010.zip

Edited by jonasdegent
Link to comment

9GB file. It was faster, a lot faster (240MB/s) and then it passes the 30% mark, it just drops dramatically to 55~80MB/s going up and down... Downloading from the array was fast all the way (230MB/s).

Could this be HBA related? Should I try to flash it to newer firmware? I don't know how but I'll probably figure it out.

 

Link to comment

The initial high speed is while it's being cache to RAM, limited by network/source device, then it's limited by the actual device speed, 55-80MB/s is still slow for those disks with turbo write enable, they should be handle around 200MB/s when empty, if you can connect the disks to the onboard SATA ports and repeat the test.

Link to comment

Just attached the 4 drives to the onboard SATA ports (good thing I had a reverse breakout cable lying around. With that Corona quarantaine it ain't easy to get stuff) however Unraid won't let me start the array? There isn't an option to start it. I can see the drives in the BIOS and in Unraid

 

Nevermind, a reboot did the trick

Edited by jonasdegent
Link to comment

OK, wow... just switched to another backplane off my Inter-tech case. Speeds are what they should be (and even better than I expected). Sheeesh... I was about to give this thing up. Averaging 295MB/s to the array, 588MB/s to the cache. Maxing out my Cat6 cables. Still on onboard SATA though, so next test is the HBA.

Link to comment

Pulled out the 2 DIMMs of RAM and replaced it by one DIMM of 16GB (tried multiple slots). The first time I booted up, it seemed to do the trick (I was still on SATA onboard). However, after one reboot, it stopped working again. Did manage to get it back fast again but I can't pinpoint where the problem is. It appears to be random. I'm now thinking either the processor or the board itself. When the transfers are fast like they should, the Plex transcoding from the 1660GTX also works like it should. When the speeds are slow, Plex has issues. So could this be related to PCI-E ports being faulty? Or processor? Maybe try to reseat the CPU now, but then I'm out of ideas.

 

Also booting from the USB stick takes a LONG time, usually stuck for a while at 'initramfs'

Edited by jonasdegent
Link to comment

Could someone think of a BIOS option that I could be overlooking? When I plug in 2 PCIe cards (NVidia card and the LSI), the server becomes slow and everything just takes a little longer. It works but far from ideal. If I remove one, it's mega fast. Super responsive. I already made an RMA for the motherboard but I just want to try one more time.

Link to comment
52 minutes ago, jonasdegent said:

Looking into the IPMI the power supply status says Power Supply error

Its a consumer grade p/s so it doesn't talk to the ipmi. That can be ignored. Do the other values look right? (12v rail, 5v rail, 3.3v rail, etc)

 

On 3/25/2020 at 8:15 AM, jonasdegent said:

I've got a "B2" on the Dr Debug

That's "b2", right? It could be related to "option rom" issues. Check if the bios is set to load option rom.  Maybe your video card needs a bios update.

 

You said you tried taking the video card out and it still wouldn't work? What about the HBA? Have you tried taking that out and seeing if you get the same "b2" code.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.