10gbps over SMB on a 45Drives Storinator


48 posts in this topic Last Reply

Recommended Posts

Unraid 6.2 will include the tweaks we provided for Linus to achieve those speeds.

 

Good to hear, I know optimizing SMB (even on Windows, but especially outside of it) can be a pain!

The SMB tuning actually all came down to two tweaks to aio read and write speed.  It took us a bit to figure that out, trying many other combos first, but yeah, that was the easy part.

 

It was the tuning to sysctl.conf for TCP settings and whatnot that took the longest to figure out.

Link to post

Hi guys! I'm new over here but just wanted to say that I'm impressed you were able to get SMB running at a full 10gbe! I don't follow Linus, but happened to catch the first video of the dual gaming pc build and was interested in unRAID after seeing that work. I stopped by to check in again as I'm thinking about testing it out to see if I can add features to my all in one system. This video has me even more interested as one of the reasons why I have Windows as the primary OS is because SMB performance is much worse on any nix distro I've ever used. I definitely still need (and want) Windows in my environment, but it seems like unRAID might have all of the components needed to take over the virtualization role while still providing a solid NAS solution and full GPU acceleration.

Link to post

The SMB tuning actually all came down to two tweaks to aio read and write speed.  It took us a bit to figure that out, trying many other combos first, but yeah, that was the easy part.

 

It was the tuning to sysctl.conf for TCP settings and whatnot that took the longest to figure out.

 

Always interesting to know what fixed it. Most companies will just throw out a 'Fixed SMB speeds' in a changelog, and that's it.

 

See, that's exactly why I'm buying an unRAID license instead of setting up a distro myself using something like SnapRAID. I'll leave it up to you to figure out the fine-tuning of the 'framework', while I can concern myself with my data and applications instead!

Link to post

The SMB tuning actually all came down to two tweaks to aio read and write speed.  It took us a bit to figure that out, trying many other combos first, but yeah, that was the easy part.

 

It was the tuning to sysctl.conf for TCP settings and whatnot that took the longest to figure out.

 

Always interesting to know what fixed it. Most companies will just throw out a 'Fixed SMB speeds' in a changelog, and that's it.

 

See, that's exactly why I'm buying an unRAID license instead of setting up a distro myself using something like SnapRAID. I'll leave it up to you to figure out the fine-tuning of the 'framework', while I can concern myself with my data and applications instead!

 

+1

Link to post
  • 3 weeks later...
  • 2 months later...

Silly question: how would having all the drives in the cache pool improve speed?

 

I thought unRAID uses RAID1 btrfs which I can't think of how it would be faster. Maybe I misread something.

Link to post

Silly question: how would having all the drives in the cache pool improve speed?

 

I thought unRAID uses RAID1 btrfs which I can't think of how it would be faster. Maybe I misread something.

Cache pool can be manually configured to other modes, including RAID0/5/6

Link to post

Silly question: how would having all the drives in the cache pool improve speed?

 

I thought unRAID uses RAID1 btrfs which I can't think of how it would be faster. Maybe I misread something.

Cache pool can be manually configured to other modes, including RAID0/5/6

 

Thank you. Silly question indeed.  ;D

Link to post
  • 4 weeks later...

New to unRAID here, in the process of buying components for my move to my new unRAID server.

 

What I was wondering is, what tweaks did Linus actually perform?

And will these be included in future versions of unRAID?

 

I'm guessing even without them I won't hit any sort of limit on my config, but I do love to optimize the crap out of my systems!

Unraid 6.2 will include the tweaks we provided for Linus to achieve those speeds.

 

Anyway those of us unable to upgrade to 6.2 (Running UnRAID in a VM on ESXi) could be provided with these tweaks?

Link to post
  • 2 weeks later...

Unraid 6.2 will include the tweaks we provided for Linus to achieve those speeds.

 

Hmmm... I'm ruining 6.2B21 and am hitting the same ~450MB/s ceiling, even when copying ramdisk to ramdisk.  What am I missing?

Link to post
  • 2 months later...

I can get write @ 10gbps to my server while it's caching to RAM but only when copying directly to a disk share, cache or disk, if I copy to a user share I can "only" get between ~650/720MB/s, same result if the user share uses cache or not, I can repeat the test 10 times and results are consistent, same server, same desktop and source file.

 

While I'm very happy with the 10GbE upgrade, well worth the 90€ I spent on it, is there a reason for this speed drop when using a user share? Any chance this can be tweaked or is it some kind of FUSE overhead?

cache.png.5d5d07485f92435680c52c5caa1bb8fe.png

disk.png.cfc550982fdab1f7b05eb4c7f966bf52.png

share1.png.c836b6fb97d24c91baefcae6eab2c232.png

share2.png.9dba9204d3bccb671dfd480fdb83c4a3.png

Link to post

I can get write @ 10gbps to my server while it's caching to RAM but only when copying directly to a disk share, cache or disk, if I copy to a user share I can "only" get between ~650/720MB/s, same result if the user share uses cache or not, I can repeat the test 10 times and results are consistent, same server, same desktop and source file.

 

While I'm very happy with the 10GbE upgrade, well worth the 90€ I spent on it, is there a reason for this speed drop when using a user share? Any chance this can be tweaked or is it some kind of FUSE overhead?

 

Are you suggesting that your infrastructure upgrade only cost you 90 Euros? Switches, cables, cards etc? I NEED to know your supplier!?!?  ;):)

Link to post

Are you suggesting that your infrastructure upgrade only cost you 90 Euros? Switches, cables, cards etc? I NEED to know your supplier!?!?  ;):)

 

I didn't upgrade my switch, I've bough two kits similar to these, 2 NICs on my desktop directly connected to my 2 fastest unRAID servers, don't plan to change my switch anytime soon as they are very expensive and don't really see the point for my normal HDD only servers, as even with turbo write it's difficult to exceed gigabit speed.

 

 

Link to post

While $599 is indeed not all that bad for a 16 port 10Gb switch; when you factor in the additional need to replace all your NICs, etc. with 10Gb units; and in some cases replace the network cabling; it's still a fairly pricing endeavor to upgrade everything to 10Gb.

 

I simply can't imagine the NEED for anything better than 1Gb in a home network ... although I admit I often saturate that bandwidth and indeed wish it was a bit faster  :)    [Then again, there's often a difference between what you NEED and what you WANT  8) ]

 

Link to post

While $599 is indeed not all that bad for a 16 port 10Gb switch; when you factor in the additional need to replace all your NICs, etc. with 10Gb units; and in some cases replace the network cabling; it's still a fairly pricing endeavor to upgrade everything to 10Gb.

 

I simply can't imagine the NEED for anything better than 1Gb in a home network ... although I admit I often saturate that bandwidth and indeed wish it was a bit faster  :)    [Then again, there's often a difference between what you NEED and what you WANT  8) ]

 

For long we thought that fast-ethernet (100 Mbps) was more than enough, then came along gigabit-ethernet. Initially very expensive, but those days are long gone.

 

Guess in some years time we will say the same about 10 gigabit-ethernet  :)

 

(professionally we work already quite extensively with 100 gigabit connections)

 

Link to post

While $599 is indeed not all that bad for a 16 port 10Gb switch; when you factor in the additional need to replace all your NICs, etc. with 10Gb units; and in some cases replace the network cabling; it's still a fairly pricing endeavor to upgrade everything to 10Gb.

 

I simply can't imagine the NEED for anything better than 1Gb in a home network ... although I admit I often saturate that bandwidth and indeed wish it was a bit faster  :)    [Then again, there's often a difference between what you NEED and what you WANT  8) ]

 

A "home" network is very subjective at this point.  I for one am not running a 4-node vSAN cluster on my home network which requires 10Gb.  Is this the norm?  Certainly not.  But I do expect home users to continue to push the envelope with regard to their home networks especially as cord cutting and home automation becomes more and more popular.

 

I can tell you this, now that I've been running 10Gb on my home network, everytime I'm at work my file transfers feel SOOOOO slow.  You get used to the speed quickly.  It's just like when you buy a new TV.  There's a reason they say to always buy the biggest TV you can afford that doesn't look ridiculous in the room, it's because you get used to the size so quickly.

Link to post

It's just like when you buy a new TV.  There's a reason they say to always buy the biggest TV you can afford that doesn't look ridiculous in the room, it's because you get used to the size so quickly.

 

Ain't that the truth!  My fiancee said something last week that almost caused me to have a heart attack!  She said, "Don't you think we should get a bigger TV for the bedroom?"  I about died! I never thought I would hear her say something like that.  We already have a 50" in there. 

Link to post

Ain't that the truth!  My fiancee said something last week that almost caused me to have a heart attack!  She said, "Don't you think we should get a bigger TV for the bedroom?"  I about died! I never thought I would hear her say something like that.  We already have a 50" in there.

She's a keeper  ;D
Link to post

While $599 is indeed not all that bad for a 16 port 10Gb switch; when you factor in the additional need to replace all your NICs, etc. with 10Gb units; and in some cases replace the network cabling; it's still a fairly pricing endeavor to upgrade everything to 10Gb.

 

I simply can't imagine the NEED for anything better than 1Gb in a home network ... although I admit I often saturate that bandwidth and indeed wish it was a bit faster  :)    [Then again, there's often a difference between what you NEED and what you WANT  8) ]

 

A "home" network is very subjective at this point.  I for one am not running a 4-node vSAN cluster on my home network which requires 10Gb.  Is this the norm?  Certainly not.  But I do expect home users to continue to push the envelope with regard to their home networks especially as cord cutting and home automation becomes more and more popular.

 

I can tell you this, now that I've been running 10Gb on my home network, everytime I'm at work my file transfers feel SOOOOO slow.  You get used to the speed quickly.  It's just like when you buy a new TV.  There's a reason they say to always buy the biggest TV you can afford that doesn't look ridiculous in the room, it's because you get used to the size so quickly.

 

I assume you meant "now" instead of "not"  :)

 

And Yes, I easily understand that.  As I noted, there's a different between "need" and "want".  I'm also someone who will almost certainly have 10Gb in the not-so-distant future ... but that's purely a "want" and not a "need".    It's simply something that I CAN do and the cost really doesn't matter if I get the whim.    But except for when I'm doing file transfers, that's not a change that will really make ANY difference.    You can stream a LOT of simultaneous video streams on a Gb network ... and as long as they're "fast enough" it won't make any difference whether the infrastructure is 1Gb or 10Gb.    And Home Automation network speed requirements don't tend to be significant ... how much data does it take to tell your thermostat to adjust; unlock the front door; or even have a look at a security feed.

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.