jonasdegent Posted March 23, 2020 Share Posted March 23, 2020 (edited) I'm relatively new to Unraid (and Linux) so be gentle. As a photographer and always in need of data storage, I decided to replace my 2 old Synology's with one big server. Might keep one just to have another backup together with one that is not in my home. I upgraded my network with 2 Netgear 10GbE switches (both with 2x 10GbE ports). I am still on cat6 cables but plan to upgrade that to 6A, just to rule out problems there. My hardware is a Ryzen 5 3600, 16GB RAM, Asrock Rack X470D4U2-2T, a HBA from Broadcom 9400-16i, a Nvidia 1660GTX for the Plex transcoding, a 960 Pro 512GB as cache and for now 4x WD Gold 14TB's. So I might expect a pretty decent performing server, right? When I test speeds with iPerf3 and Diskspeed everything seems normal. I get 3.9Gbit/s network speeds (not full full speed but decent enough considering the length and being cat6) and my disks seem normal with 255MB/s (with no drops) and 1.1Gbit/s for the cache drive. When I move something from my computer (SMB shares) to the server (share with cache only) I'm averaging 50~70MB/s. The parity sync is at ~90MB/s. I get the same speeds when I pause the sync. When I move something to the array I get like 140MB/s in the beginning (first 30%) just to drop down to 50~70MB/s. Turbo write is on. I'm not expecting GB/s here but this just seem slow? I have 2 10GbE ports and one Gbit port on the motherboard but I can only use one 10GbE for now. I'm not going to invest into a new very expensive 10GbE switch just to get "under Gbit" speeds. So I'm kinda out of ideas for now. Not sure if there's something wrong in the hardware (RAM? Memtest seemed fine) or network settings (which I don't understand fully, bonding is on but I'm only running one port?) so I turn to the forum. I included the diagnosis file. marcel-diagnostics-20200323-1010.zip Edited March 23, 2020 by jonasdegent Quote Link to comment
JorgeB Posted March 23, 2020 Share Posted March 23, 2020 Wait for the parity sync to finish, even with turbo write performance isn't the same while parity disk is invalid. 1 Quote Link to comment
jonasdegent Posted March 23, 2020 Author Share Posted March 23, 2020 also, any ideas why this is appearing on my dashboard? Quote Link to comment
testdasi Posted March 23, 2020 Share Posted March 23, 2020 3 minutes ago, jonasdegent said: also, any ideas why this is appearing on my dashboard? Are you running any kind of ad blocker? If so whitelist your Unraid GUI IP and server name. Quote Link to comment
jonasdegent Posted March 23, 2020 Author Share Posted March 23, 2020 Hmm, I do have Ad blocker disabled in Google Chrome but still the same error. No problems with it in Firefox and Edge. Quote Link to comment
dukiethecorgi Posted March 23, 2020 Share Posted March 23, 2020 Flush the Chrome cache and it should work 1 Quote Link to comment
jonasdegent Posted March 23, 2020 Author Share Posted March 23, 2020 3 hours ago, dukiethecorgi said: Flush the Chrome cache and it should work That did the trick! Thanks! Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 Parity build just finished. Rebooted just to be sure. Same identical "slow" speeds. 😕 Turbo write is on. Quote Link to comment
JorgeB Posted March 24, 2020 Share Posted March 24, 2020 Start a copy to the array and grab diags during it. Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 I presume you just do it the same way? Made one during a copy to the share that suppose to be to the cache only and one during a copy to the array to array marcel-diagnostics-20200324-1428.zip To cached marcel-diagnostics-20200324-1427.zip Quote Link to comment
JorgeB Posted March 24, 2020 Share Posted March 24, 2020 Nothing jumps out, try enabling disk shares (Settings -> Global Share Settings), then transfer directly to a disk, e.g. \\tower\disk1, see if there's any difference. Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 9GB file. It was faster, a lot faster (240MB/s) and then it passes the 30% mark, it just drops dramatically to 55~80MB/s going up and down... Downloading from the array was fast all the way (230MB/s). Could this be HBA related? Should I try to flash it to newer firmware? I don't know how but I'll probably figure it out. Quote Link to comment
JorgeB Posted March 24, 2020 Share Posted March 24, 2020 The initial high speed is while it's being cache to RAM, limited by network/source device, then it's limited by the actual device speed, 55-80MB/s is still slow for those disks with turbo write enable, they should be handle around 200MB/s when empty, if you can connect the disks to the onboard SATA ports and repeat the test. Quote Link to comment
JorgeB Posted March 24, 2020 Share Posted March 24, 2020 BTW, you can also test to \\tower\cache, this should also be much faster than before, because it's bypassing Unraid share system, but while it's normal to see a difference in performance here it's not normal to see such low speeds writing to the user shares. Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 (edited) Just attached the 4 drives to the onboard SATA ports (good thing I had a reverse breakout cable lying around. With that Corona quarantaine it ain't easy to get stuff) however Unraid won't let me start the array? There isn't an option to start it. I can see the drives in the BIOS and in Unraid Nevermind, a reboot did the trick Edited March 24, 2020 by jonasdegent Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 (edited) Copy via disk share to disk 1 and 2 and 3 is around 100MB/s average, to the cache drive via disk share it's 205MB/s. To an array share it's averaging 93MB/s. All copies were fast the first 40-ish percentages. Edited March 24, 2020 by jonasdegent Quote Link to comment
JorgeB Posted March 24, 2020 Share Posted March 24, 2020 Seems the user share overhead is higher than normal, though speeds still seam on the low side even when using disk shares, so likely something else at play here. Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 OK, wow... just switched to another backplane off my Inter-tech case. Speeds are what they should be (and even better than I expected). Sheeesh... I was about to give this thing up. Averaging 295MB/s to the array, 588MB/s to the cache. Maxing out my Cat6 cables. Still on onboard SATA though, so next test is the HBA. Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 OK, back to square one... seemed to work, now I'm back to slow speeds. Quote Link to comment
jonasdegent Posted March 24, 2020 Author Share Posted March 24, 2020 (edited) Pulled out the 2 DIMMs of RAM and replaced it by one DIMM of 16GB (tried multiple slots). The first time I booted up, it seemed to do the trick (I was still on SATA onboard). However, after one reboot, it stopped working again. Did manage to get it back fast again but I can't pinpoint where the problem is. It appears to be random. I'm now thinking either the processor or the board itself. When the transfers are fast like they should, the Plex transcoding from the 1660GTX also works like it should. When the speeds are slow, Plex has issues. So could this be related to PCI-E ports being faulty? Or processor? Maybe try to reseat the CPU now, but then I'm out of ideas. Also booting from the USB stick takes a LONG time, usually stuck for a while at 'initramfs' Edited March 24, 2020 by jonasdegent Quote Link to comment
JorgeB Posted March 25, 2020 Share Posted March 25, 2020 Difficult to guess where the problem is with those symptoms, you'll need to swap some hardware around. 11 hours ago, jonasdegent said: usually stuck for a while at 'initramfs' This usually takes a few seconds. Quote Link to comment
jonasdegent Posted March 25, 2020 Author Share Posted March 25, 2020 I've got a "B2" on the Dr Debug which should be the GPU according to some sources, off course it's not even in the manual. Removed the 1660GTX but keeps failing. Too bad I can't test CPU, haven't got another one lying around. Quote Link to comment
jonasdegent Posted March 26, 2020 Author Share Posted March 26, 2020 Could someone think of a BIOS option that I could be overlooking? When I plug in 2 PCIe cards (NVidia card and the LSI), the server becomes slow and everything just takes a little longer. It works but far from ideal. If I remove one, it's mega fast. Super responsive. I already made an RMA for the motherboard but I just want to try one more time. Quote Link to comment
jonasdegent Posted March 26, 2020 Author Share Posted March 26, 2020 Looking into the IPMI the power supply status says Power Supply error. Is that because it isnt detected or is there really something wrong? Quote Link to comment
civic95man Posted March 26, 2020 Share Posted March 26, 2020 52 minutes ago, jonasdegent said: Looking into the IPMI the power supply status says Power Supply error Its a consumer grade p/s so it doesn't talk to the ipmi. That can be ignored. Do the other values look right? (12v rail, 5v rail, 3.3v rail, etc) On 3/25/2020 at 8:15 AM, jonasdegent said: I've got a "B2" on the Dr Debug That's "b2", right? It could be related to "option rom" issues. Check if the bios is set to load option rom. Maybe your video card needs a bios update. You said you tried taking the video card out and it still wouldn't work? What about the HBA? Have you tried taking that out and seeing if you get the same "b2" code. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.