Supermicro X12SCA-5F Networking Performance Issues (RESOLVED)

[spasszeit]

I upgraded my very old Unraid servers to X11SCA-F and X12SCA-5F boards. The X11SCA-F server is great, but the X12SCA-5F is giving me issues and I am having a hard time understanding whether those are driver or hardware related, so need your help guys. Deciding whether I should exchange the board (not easy with me being in Canada and the board came from the US) while I still a have a few days to do so.

 

The X12SCA-5F has three NICs - an IPMI NIC and Intel's i225LM and PHY I219LM NICs, both should be supported by Unraid. The issue is that Unraid doesn't recognize the  I219LM NIC (tested with 6.9.2 and next RC1). The i225LM is recognized and works but file transfers to the Unraid server are erratic, starting at around 113MB/s, going down to 40-50, up to 80-90, then to zero, back to 40-50, up to 113, then 10-20, etc. From the server to Windows it's a perfect ~113 MB/s line. Why could this be happening? On the other server it's a perfect 113MB/s line for a while then drop to ~69MB/s and it stays there until the file transfer is done. Any thoughts?

 

P.S. Installed windows on this machine too, updated drivers to the latest. The i225LM NIC works fine, wither transmitting or receiving traffic. The I219LM gets speeds of 300 Kbps for a minute than errors out. Accessing the internet is fine though, getting max speeds with speedtest.net. 

 

 

 

Edited August 23, 2021 by spasszeit

[spasszeit]

Anyone?

[JorgeB]

Do you have another NIC you could try with, just to confirm if it's a NIC driver issue or not?

[spasszeit]

Yes, I tried another NIC. Pulled a working one Intel Gigabit CT from my workstation. Works fine with that motherboard when booting into windows 10 but not recognized by Unraid for some reason. 

[JorgeB]

I meant try a NIC that works with Unraid.

[spasszeit]

In case someone runs into this, this was not NIC related. In another thread, I saw someone having similar issues and reporting that copying files directly to hard drives or the cache drive did not cause the issues. I did a test by copying the same large file to every disk share and found that in about half the cases I'd get perfect file transfer jobs that start right away and  stay at 113Mbps and then drop to 69Mbps once RAM is space is filled up, and in the other half of the cases file transfers to the server would have delayed starts and once they start, you get wild speed swings all the way to zero. Interestingly, these wild transfer speeds happened on every drive that was formatted to reiserfs. None of the XFS formatted drives had this issue. There's got to be a solution to this as my other server has a mix of reiserfs and xfs formatted drives but doesn't have the same issues when copying files to the share spanning all disks, but I don't know what it is. My solution is to reformat all the remaining drives on the server with the issues to xfs.

[ChatNoir]

6 hours ago, spasszeit said:

My solution is to reformat all the remaining drives on the server with the issues to xfs.

In any case, ReiserFS is only supported for legacy reasons and is not recommended anymore (for some years I guess).

 

https://wiki.unraid.net/File_System_Conversion

[itimpi]

ReiserFS is known to perform very badly when disks start getting anywhere near full.   One of the reasons it is now deprecated.    Also it has an absolute maximum of 16TB supported on a drive and modern drives have started to be available that are larger than this.