TerpMike28 Posted July 21 Share Posted July 21 Hi all, cross posting from my reddit post. Got some helpful tips on how to diagnose but still not sure what is going on or how to fix it. When building my server I bought a pair of HPE 10Gb 561T NIC's for both the server and my main PC. I just got a 10G network switch (Netgear GS110EMX), and when testing everything I'm getting a weird speed issue. When moving a few single large files from my PC to the server, it starts out at 800-900MB/s and drops to around 220MB/s almost instantly (writing to a Samsung 990 Pro 2TB cache) but when moving from server to PC I get the full 1.1GB/s. I am not running any kind of VM/reverse proxy or using any web based FTP. It's simply a networked connection from the pc to the server via the 10g switch and the router (both PC and server are direct connected to the 10g ports). Specs for server are: Cache Drive Samsung 990 Pro 2TB Main PC Drives are: C drive Seagate Firecuda 530 2TB (the file I was moving was from this drive) D drive (Dynamic drive for 4TB minus headroom) SK Hynix P41 & Samsung 980 Pro I ran iperf and got these results which mimic what I saw when transferring the files. I just can't wrap my head around what could be going on since the issue is only one way. Any thoughts? Quote Link to comment
JorgeB Posted July 21 Share Posted July 21 9 hours ago, TerpMike28 said: I ran iperf and got these results which mimic what I saw when transferring the files. That suggests a network issue, could be NICs, cables, client PC, etc Quote Link to comment
giveitago Posted July 21 Share Posted July 21 On the PC run the following iperf commands which runs the test for 60 secs and link to the results on pastebin. iperf3 -c <server IP> -P 4 -O 3 -t 60 > iperf_results_client_send.txt iperf3 -c <server IP> -P 4 -O 3 -t 60 -R > iperf_results_server_send.txt Might show up something Quote Link to comment
TerpMike28 Posted July 21 Author Share Posted July 21 8 hours ago, JorgeB said: That suggests a network issue, could be NICs, cables, client PC, etc If it was network, wouldn't the issue persist when downloading from the server? Quote Link to comment
TerpMike28 Posted July 21 Author Share Posted July 21 6 hours ago, giveitago said: On the PC run the following iperf commands which runs the test for 60 secs and link to the results on pastebin. iperf3 -c <server IP> -P 4 -O 3 -t 60 > iperf_results_client_send.txt iperf3 -c <server IP> -P 4 -O 3 -t 60 -R > iperf_results_server_send.txt Might show up something Here you go https://pastebin.com/cfT71GRx. Thanks for the suggestion. I'm still new to networking so I'm not 100% sure what I'm looking for, but it looks normal? Quote Link to comment
JorgeB Posted July 21 Share Posted July 21 2 hours ago, TerpMike28 said: wouldn't the issue persist when downloading from the server? Not necessarily, problem can be only in one direction, iperf only tests the LAN bandwidth, nothing else. Quote Link to comment
TerpMike28 Posted July 22 Author Share Posted July 22 (edited) 3 hours ago, JorgeB said: Not necessarily, problem can be only in one direction, iperf only tests the LAN bandwidth, nothing else. I gotcha...I'll start working on swapping around stuff. By chance, does the extended iperf report I posted earlier show any red flags to you? Edited July 22 by TerpMike28 Quote Link to comment
JorgeB Posted July 22 Share Posted July 22 6 hours ago, TerpMike28 said: does the extended iperf report I posted earlier show any red flags to you? No, but I recommend running a single stream test, it gives more realistic results. Quote Link to comment
giveitago Posted July 22 Share Posted July 22 20 hours ago, TerpMike28 said: Here you go https://pastebin.com/cfT71GRx. Thanks for the suggestion. I'm still new to networking so I'm not 100% sure what I'm looking for, but it looks normal? Looks like you are getting consistent 10g in both directions on the network. However this is different from OP. Are you still getting different speeds when you run it as you did originally? Quote Link to comment
TerpMike28 Posted July 23 Author Share Posted July 23 On 7/22/2024 at 11:43 AM, giveitago said: Looks like you are getting consistent 10g in both directions on the network. However this is different from OP. Are you still getting different speeds when you run it as you did originally? Yeah, I think I've figured out that it is a buffer issue (I've eliminated any hardware being the issue by directly connecting the server and PC. Got all of the same speeds, even with different cables/ports). Turned off a bunch of limits in the NIC properties (flow control, interrupt moderation, etc.) and maxed out receive buffers to 4096 and transmit buffers to 16384 and jumbo packet to 9014 bytes and that increased the speed from 220-250ish MB/s to around 400-425MB/s from PC to Server but I am still getting full 1.1GB/s when going Server to PC. Also, when I ran the short iperf test it still showed the same bottleneck. On 7/22/2024 at 3:24 AM, JorgeB said: No, but I recommend running a single stream test, it gives more realistic results. I had a thought that maybe increasing the buffer on the server might increase the speed but I saw your post here and based on what I think is being said, it doesn't look like that is an option/would help. Would you agree? Quote Link to comment
JorgeB Posted July 23 Share Posted July 23 18 minutes ago, TerpMike28 said: Would you agree? Yep, that should not interfere with iperf results. Quote Link to comment
TerpMike28 Posted July 23 Author Share Posted July 23 16 minutes ago, JorgeB said: Yep, that should not interfere with iperf results. side question...would using a file transfer software like filezilla solve the issue? Quote Link to comment
JorgeB Posted July 24 Share Posted July 24 If that does multiple transfers for a single file, it should help. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.