July 21, 20241 yr Hi all, cross posting from my reddit post. Got some helpful tips on how to diagnose but still not sure what is going on or how to fix it. When building my server I bought a pair of HPE 10Gb 561T NIC's for both the server and my main PC. I just got a 10G network switch (Netgear GS110EMX), and when testing everything I'm getting a weird speed issue. When moving a few single large files from my PC to the server, it starts out at 800-900MB/s and drops to around 220MB/s almost instantly (writing to a Samsung 990 Pro 2TB cache) but when moving from server to PC I get the full 1.1GB/s. I am not running any kind of VM/reverse proxy or using any web based FTP. It's simply a networked connection from the pc to the server via the 10g switch and the router (both PC and server are direct connected to the 10g ports). Specs for server are: Cache Drive Samsung 990 Pro 2TB Main PC Drives are: C drive Seagate Firecuda 530 2TB (the file I was moving was from this drive) D drive (Dynamic drive for 4TB minus headroom) SK Hynix P41 & Samsung 980 Pro I ran iperf and got these results which mimic what I saw when transferring the files. I just can't wrap my head around what could be going on since the issue is only one way. Any thoughts?
July 21, 20241 yr Community Expert 9 hours ago, TerpMike28 said: I ran iperf and got these results which mimic what I saw when transferring the files. That suggests a network issue, could be NICs, cables, client PC, etc
July 21, 20241 yr On the PC run the following iperf commands which runs the test for 60 secs and link to the results on pastebin. iperf3 -c <server IP> -P 4 -O 3 -t 60 > iperf_results_client_send.txt iperf3 -c <server IP> -P 4 -O 3 -t 60 -R > iperf_results_server_send.txt Might show up something
July 21, 20241 yr Author 8 hours ago, JorgeB said: That suggests a network issue, could be NICs, cables, client PC, etc If it was network, wouldn't the issue persist when downloading from the server?
July 21, 20241 yr Author 6 hours ago, giveitago said: On the PC run the following iperf commands which runs the test for 60 secs and link to the results on pastebin. iperf3 -c <server IP> -P 4 -O 3 -t 60 > iperf_results_client_send.txt iperf3 -c <server IP> -P 4 -O 3 -t 60 -R > iperf_results_server_send.txt Might show up something Here you go https://pastebin.com/cfT71GRx. Thanks for the suggestion. I'm still new to networking so I'm not 100% sure what I'm looking for, but it looks normal?
July 21, 20241 yr Community Expert 2 hours ago, TerpMike28 said: wouldn't the issue persist when downloading from the server? Not necessarily, problem can be only in one direction, iperf only tests the LAN bandwidth, nothing else.
July 22, 20241 yr Author 3 hours ago, JorgeB said: Not necessarily, problem can be only in one direction, iperf only tests the LAN bandwidth, nothing else. I gotcha...I'll start working on swapping around stuff. By chance, does the extended iperf report I posted earlier show any red flags to you? Edited July 22, 20241 yr by TerpMike28
July 22, 20241 yr Community Expert 6 hours ago, TerpMike28 said: does the extended iperf report I posted earlier show any red flags to you? No, but I recommend running a single stream test, it gives more realistic results.
July 22, 20241 yr 20 hours ago, TerpMike28 said: Here you go https://pastebin.com/cfT71GRx. Thanks for the suggestion. I'm still new to networking so I'm not 100% sure what I'm looking for, but it looks normal? Looks like you are getting consistent 10g in both directions on the network. However this is different from OP. Are you still getting different speeds when you run it as you did originally?
July 23, 20241 yr Author On 7/22/2024 at 11:43 AM, giveitago said: Looks like you are getting consistent 10g in both directions on the network. However this is different from OP. Are you still getting different speeds when you run it as you did originally? Yeah, I think I've figured out that it is a buffer issue (I've eliminated any hardware being the issue by directly connecting the server and PC. Got all of the same speeds, even with different cables/ports). Turned off a bunch of limits in the NIC properties (flow control, interrupt moderation, etc.) and maxed out receive buffers to 4096 and transmit buffers to 16384 and jumbo packet to 9014 bytes and that increased the speed from 220-250ish MB/s to around 400-425MB/s from PC to Server but I am still getting full 1.1GB/s when going Server to PC. Also, when I ran the short iperf test it still showed the same bottleneck. On 7/22/2024 at 3:24 AM, JorgeB said: No, but I recommend running a single stream test, it gives more realistic results. I had a thought that maybe increasing the buffer on the server might increase the speed but I saw your post here and based on what I think is being said, it doesn't look like that is an option/would help. Would you agree?
July 23, 20241 yr Community Expert 18 minutes ago, TerpMike28 said: Would you agree? Yep, that should not interfere with iperf results.
July 23, 20241 yr Author 16 minutes ago, JorgeB said: Yep, that should not interfere with iperf results. side question...would using a file transfer software like filezilla solve the issue?
July 24, 20241 yr Community Expert If that does multiple transfers for a single file, it should help.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.