Optimise SMB performance

[DaniloSilva]

After moving to 5.0-rc16c my SMB performance has dropped by around 50%, is there anything I can change to restore its former glory?

 

Using SMB on 5.0rc4 I was able to upload files at around 60MB/s and download at 90MB/s (without a cache disk), since the upgrade I'm uploading at around 30MB/s and downloading at 45MB/s.

 

To verify that this wasn't a general performance issue I tested using FTP and NFS,  FTP is stellar (64MB/s up and 120MB/s down), and NFS (using a Mac) is also great (60MB/s up and 95MB/s down).

 

Update: I installed a cache disk to see if that would improve write performance. SMB remains the same at around 30MB/s, NFS is now uploading as around 100MB/s, so this is definitely a bottleneck in the SMB functionality.

[ZeroK]

I've noticed the same thing though I was running rc8a and never had any issues or needed to upgrade to the latest rc but figured we were so close to a final that I bit the bullet and I was seeing 30MB to the array and 50MB from the array with rc8a.  Now with rc16c I am seeing 15-25MB to the array and 30-35MB from the array.  I'm tempted to go back to rc8a to see if it increases but not sure the new permissions that we had to do to go from rc8a to rc16c would have any ill effect on it going back to rc8a.

[peter_sm]

I have the same numbers, this is sad, maybe there are fix on next RC

 

 

Sent from my iPhone using Tapatalk 2

[defected07]

Have to tried AFP? Curious to see your results, since you were using a Mac for NFS.

[mr-hexen]

what network card are you guys all using??

[dirtysanchez]

I have noticed the same issue. Haven't had time lately to do much testing, but quick tests show an SMB slowdown of at least 30%.

 

Server has Realtek RTL8168F/8111F NIC.

[ZeroK]

Realtek originally with rc8a, then moved to rc16c and saw the decrease, put in an intel thinking it may solve it, same thing, no difference, went back to Realtek.  I'm still thinking of going back to rc8a to see if it increases but not sure if it will mess anything up if I do since permissions are different but more importantly I really don't know if I have time to test it at the moment.

[Thornwood]

Is this on single stream of files or multitude of streams? I have seen when I have lots of streams running parallel my performance dips drastically but with just a single stream it stays around 120 mb/sec

 

Thornwood

[lishpy]

I thought this was just normal SMB performance. 

 

I've been using SMB to hook up to my unRAID server via a few Macs since unRAID 4.3 days. I thought 35MB/s was normal speed. Just recently upgraded my server with a new Mobo/CPU/RAM and the highest I get on SMB performance is peaking at 37.1MB/s over Gigabit LAN, and a lower average then that. I'd kill for 90MB/s.

 

 

[Thornwood]

Could it be the mac side? Do you have any pc's you could try testing with? Just for testing if there is a difrance? Allso what is the version of smb in the mac? Also do you have a cashe drive?

 

Sent from my YP-G70 using Tapatalk 2

 

 

[Schar]

I have noticed similar drops in speed. My write peaks at high 20's / low 30's wherease previously I could get 50+ (from the same source PC). Running a stock Intel motherboard a few years old but with only a P4 CPU (which is a system constraint).

 

Reading, I can get anywhere from 50 to 100/sec depending on the PC doing the reading.

 

All SMB

 

 

[RobJ]

The problem does not seem well characterized above, and it's hard to fix a problem if you don't understand it.  So perhaps we can come up with a good and repeatable test, that isolates and demonstrates the issue.  The important questions seem to me to be: what types of transfers are affected, what systems are affected, what UnRAID versions are affected.

 

1. What types of SMB transfers are affected, and what types do not matter?

  My initial impression from the posts above is that the issue affects reading and writing equally, and if that is true, then we can simplify the testing by timing transfers *from* the server only, eliminating the impact of parity writing, cache drive usage, etc.  If someone knows of a reason (or discovers a reason) why testing other transfers is useful, please tell us.

  I cannot tell from the comments above whether the file sizes matter, that is, whether a transfer of a few very large files better demonstrates the problem than a large number of small files, or if size does not matter.  Can we get away with simplifying this by specifying the test transfer should be 10GB of files, any files?

 

2. What users' systems are affected, and what systems are not affected?

  I may be wrong, but it seems odd that no one reported this issue before now, which makes it hard to believe it affects very many systems.  Once we have decided on a reliable test that truly demonstrates the SMB slowdown, then we need to know which systems are and are not affected, and then try to figure out why.

 

3. Which UnRAID versions are affected with significantly slower SMB transfers, and which are fast?

  I'd like to see the UnRAID version reported for each test, and then see the test repeated with other UnRAID versions.  It should only require copying the appropriate bzimage and bzroot to the flash drive, and rebooting.  I don't personally believe there should be any permissions issues with this, but if in doubt, you can always rerun the permissions tool once you return to your desired version.

 

My initial suggestion for an SMB test would be: transfer 10GB of files from a fast UnRAID drive to a fast drive on a destination machine, time it and calculate average transfer speed in MB/s, then repeat with one or more different UnRAID versions, and report the results here, with estimates of the relative differences between the versions.  This test will probably evolve as results come in.

 

Confession: while I'm not aware of any problems with my own system, I haven't actually tested!

Confession #2: I am terrible at staying on top of an issue or topic, so while I hope I've started the ball rolling, others should feel free to carry it forward.

[lishpy]

Replying from my phone so I will make this quick, but like I mentioned before, I didn't realize this was even an issue. I thought around 30-40MB/s write time was the norm. I had some old hardware in my server before, but upgrading last week, speeds are the same.

 

Definitely willing to do more tests, but I've seen these speeds consistently for years. It hasn't changed for me regardless of what Mac computer I'm using, what OS it's been running, what unRAID version I'm running or now what server hardware I have.

[dirtysanchez]

I will attempt to find some time this weekend to do some thorough testing.

 

That said, here's what I can tell you from memory. Firstly, I was testing using LANSpeed Test, therefore ruling out HD speed on the client side, testing only read/write speeds on the server side using an all Gb/CAT6 network. Previous tests were done on rc12a (see results in build thread in sig), 1GB single file transfer with LANSpeed Test. Reads from cache drive under rc12a were 800Mbps, but are 650Mbps under rc16c. Don't recall write speeds off the top of my head. Also tested read/write to array, same read results as cache drive, and can't remember write tests.

 

Again, I'll do what I can to do some thorough testing and post some results this weekend.

[dirtysanchez]

I did some testing this morning.  I was wrong in my initial post in this thread.  At least in my case, there is nothing wrong with SMB transfer speeds under rc16c. 

 

My initial testing on rc12a was done with a laptop I no longer own.  The rc16c tests were performed with my desktop computer.  The desktop has 2 onboard NICs.  Turns out the onboard NIC I was using appears to have issues and gave results of ~50MB/s writes and ~82MB/s reads to/from the cache drive (the approximate 30% decrease I described earlier).  I switched to the other onboard NIC and transfer speeds returned to normal, ~85MB/s writes and 110MB/s reads to/from cache drive.  BTW, the cache drive is the limiting factor for writes (it's max write spec is 85MB/s) and the Gb network is the limiting factor for reads.  Tests to the array with the good NIC result in ~43MB/s writes and 110MB/s reads.  Again, right where it should be. 

 

Sorry for the confusion, this was a mistake on my part.  As I stated earlier, with my particular setup on rc16c, SMB transfer rates are completely normal.

 

TL;DR - One of my two desktop NICs apparently has issues, switching to the other NIC resolved my slow SMB transfers.

[RobJ]

My own tests do not indicate any issue at all on my system.  In fact, my 3 fastest timings were all on rc16c, but only by a small and insignificant amount.  I tested 5 UnRAID releases, rc16c, rc8a, rc11, rc12a, and rc15.  If anyone thinks there is a better SMB implementation in a different release, I'll be happy to test it on my system.

 

I found an easy way to test top speed for transfers *from* the UnRAID server, using a hash testing tool, that is essentially equivalent to copying to null, thereby eliminating almost all of the local Windows impacts.  I tried TeraCopy and Total Commander, but had wildly divergent and inconsistent results, with speeds rollercoastering between 17MB/s and 63MB/s, and overall averages between 36MB/s and 54MB/s.  Since I was too lazy to research how to do a true copy to null from the server, I thought of using a hash tool, because it removes almost all of the Windows overhead, the file creation time, file growth issues, anti-virus scanning delays, etc.  And I discovered when I tried it, it was amazingly consistent, with my first 3 timings all about 140.3 seconds, within a few hundredths of seconds apart, equal to 71.2MB/s.  That time is the CRC or MD5 hash calculation time for a 9.99GB file, that is around the middle of my fastest array drive, accessed from a Windows 7 desktop.

 

Averaged speeds (with only tiny variations):

rc16c - 71.2MB/s

rc8a  - 71.1MB/s

rc11  - 70.8MB/s

rc12a - 71.0MB/s

rc15  - 71.1MB/s

 

From what I could see, Windows often caused severe slowdowns, with long periods of 25MB/s and 40MB/s, but as far as I can tell, the slowdowns were all caused on the Windows side, not the UnRAID side (I could be wrong though).

 

I strongly recommend any who feel they have SMB issues to repeat their testing on an earlier UnRAID release, to both prove the issue and eliminate the possibility of current hardware issues causing their slowdowns.

 

For those interested, I use the fast and free HashOnClick from 2BrightSparks, makers of the excellent SyncBack tools.  Download the freeware version and install it, and you can right-click any file on your server, using Windows Explorer or Total Commander or most any file manager, select a hash type (doesn't matter which, the CPU usage is negligible), and when done it will report the time in seconds, from which you will calculate the MB/s.

 

One other thing I've done is to apply the Windows network throttling tweaks.  For those who may not have seen it, Tom reports here about tweaks to disable Windows network throttling (for Win 7, 8?, Vista?).  He said that after this 'fix', "This is actually the best performance I've ever measured".  I can say the same.  My write performance to parity protected drives (very subjectively estimated) is about 10% better, between 30MB/s and 35MB/s.  My read and copy speed from UnRAID array drives (also very subjectively estimated) is about 50% to 70% better, generally between 60MB/s and 65MB/s.  Unfortunately, this involves editing the Windows registry...  If you aren't confident in your registry skills, don't!  I've added this tip to the Improving unRAID Performance page, here.

[nars]

I did just tested it and I can't really reproduce any smb slowdown on rc16c with w7 as client, see attached screenshots, showing copy to and from cache drive on unraid server. And I'm using the crap onboard realtek nic (details on my sign) and a very very cheap TP-Link TL-SG1008D switch.

 

P.S. I did double checked speed values with iftop to make sure they match the ones showing on Windows file copy window.

 

P.S.2. Repeating test on XP SP3 as client I can still get ~105MB/s when copying to unraid but only near ~50MB/s copying from unraid, however this is not new and I did always seen this on XP as client. I did searched for that on the past and it seems usual, apparently mostly because XP doesn't have smb2 and thus efficiency is not good for a gigabit network, understandable for XP age, but pity that it didn't got smb2 as update, along with tcp rescale window auto-tuning...

[garycase]

Ditto.  I easily max out the Gb network with copies from the array => ~ 120GB/s for large files.

 

I've simply not seen this slowdown issue.

 

 

[peter_sm]

I have played with regedit and settings for my network card..... and suddenly I have these speed ?? don't know what exactly changes made this performance

 

W7 to cache = 70MB/s

W7 to disk share = 35MB/s

cache to W7 = 65MB/s

 

I hope it will stay like this ......

 

 

My old transfer data is here .. http://lime-technology.com/forum/index.php?topic=28382.msg255558#msg255558

 

 

But on my MAC I get over 100MB/s over AFP 

[limetech]

I don't see SMB slowdown, but if any mod can confirm this appears to be an unRaid-specific issue, please move this thread to the Issue List board.

[bkastner]

Just throwing in my $0.02 here. I seem to have speed issues with UnRaid compared to other Windows boxes on my network. I have a mix of Windows boxes (Win7, Win8 and Server 2012) and I can copy from any one to the other and get 80-100MB/s no problem. When I copy to UnRaid I get 20-25MB/s. The UnRaid server is on the same switch as my Server 2012 machines, and I've swapped cables back and forth with no effect.

 

I am currently using RC 15a, but did notice this was earlier RCs as well. It's not a showstopper, but it is annoying when I actually realize how big a difference there is.

 

I am using an Intel NIC in my UnRaid server and it is configured as 1000MB/Full Duplex:

 

NIC info (from ethtool)Settings for eth0:

Supported ports: [ TP ]

Supported link modes:  10baseT/Half 10baseT/Full

                                    100baseT/Half 100baseT/Full

                      1000baseT/Full

Supports auto-negotiation: Yes

Advertised link modes:  10baseT/Half 10baseT/Full

                        100baseT/Half 100baseT/Full

                        1000baseT/Full

Advertised pause frame use: No

Advertised auto-negotiation: Yes

Speed: 1000Mb/s

Duplex: Full

Port: Twisted Pair

PHYAD: 0

Transceiver: internal

Auto-negotiation: on

MDI-X: off

Supports Wake-on: umbg

Wake-on: g

Current message level: 0x00000007 (7)

Link detected: yes

NIC driver info (from ethtool -i)

driver: e1000

version: 7.3.21-k8-NAPI

firmware-version:

bus-info: 0000:06:05.0

 

I don't know if this information helps anyone else trying to look into this, but it would be great to get this fixed (assuming it can be).

 

[dirtysanchez]

Just throwing in my $0.02 here. I seem to have speed issues with UnRaid compared to other Windows boxes on my network. I have a mix of Windows boxes (Win7, Win8 and Server 2012) and I can copy from any one to the other and get 80-100MB/s no problem. When I copy to UnRaid I get 20-25MB/s. The UnRaid server is on the same switch as my Server 2012 machines, and I've swapped cables back and forth with no effect.

 

I am currently using RC 15a, but did notice this was earlier RCs as well. It's not a showstopper, but it is annoying when I actually realize how big a difference there is.

 

I am using an Intel NIC in my UnRaid server and it is configured as 1000MB/Full Duplex:

 

NIC info (from ethtool)Settings for eth0:

Supported ports: [ TP ]

Supported link modes:  10baseT/Half 10baseT/Full

                                    100baseT/Half 100baseT/Full

                      1000baseT/Full

Supports auto-negotiation: Yes

Advertised link modes:  10baseT/Half 10baseT/Full

                        100baseT/Half 100baseT/Full

                        1000baseT/Full

Advertised pause frame use: No

Advertised auto-negotiation: Yes

Speed: 1000Mb/s

Duplex: Full

Port: Twisted Pair

PHYAD: 0

Transceiver: internal

Auto-negotiation: on

MDI-X: off

Supports Wake-on: umbg

Wake-on: g

Current message level: 0x00000007 (7)

Link detected: yes

NIC driver info (from ethtool -i)

driver: e1000

version: 7.3.21-k8-NAPI

firmware-version:

bus-info: 0000:06:05.0

 

I don't know if this information helps anyone else trying to look into this, but it would be great to get this fixed (assuming it can be).

 

When you say "copy to unRaid" are you referring to copying data to the parity-protected array?  If so, it will always be slower due to the overhead of parity calculation and parity writes.  With my server I can copy to the cache drive (which is not parity-protected) at the full speed the drive is capable of (85MB/s), but writes to the parity-protected array usually top out around 40MB/s.  In addition that 40MB/s number is obtained with LANSpeed Test, which cuts out all the other overhead associated with Windows copies.  Writing to the array from Windows Explorer via drag-and-drop results in closer to 25 - 30 MB/s as reported by Windows.

[bkastner]

Good point... after I sent this I was lying in bed and realized I didn't factor in the parity write (as I was talking about non-cache drive), but I am still somewhat surprised it sucks 75% of the speed, but I guess my experience isn't that abnormal from what you mention.

 

I guess I should be getting a cache drive in place to solve my own issue.

 

Thanks for helping clear things up.