I'm at a loss as what else to try for getting good SMB transfers from my Windows 11 pc to unRAID 6.12.11 using the following:

 

Windows 11

• intel x520 t1 running on x16 slot
• MTU set to 9000
• Threadripper 3960x
• Gigabyte TRX40 Designare
• 64 GB Corsair Dominator Platinum ram at 3600 speed
• Samsung 980 Pro nvme

 

unRAID

• intel x540 t2 running on x16 slot
• MTU set to 9000
• Threadripper 3960x
• Asus ROG Zenith II Extreme Alpha
• 64 GB ECC ram
• nvme cache drives are Corsair Force MP600 1 TB and Sabrent Rocket 4.0 1 TB 

 

unifi network gear

• UDMP
• USW Pro Aggregation
• Jumbo Frames enabled
• Win11 and unRAID machines are connected to USW Pro Aggregation through rj45 sfp+ modules from Ubiquiti

 

I have tried different switch ports, different ethernet cables, different sfp+ modules, different nic adapter settings.

I finally got iperf3 to show full 10 GbE speeds after disabling Interruppt Moderation.

I can't, however, get my SMB transfers of movie files from my Win11 nvme drive to unRAID above 450 MB/sec.

 

I'm attaching screenshots of relavent info you might ask for.

I've gotten 900 MB/sec in a previous version of unRAID, but I don't remember which version. 

Any help is appreciated. 

 

 

 

threadripper19-diagnostics-20240827-2126.zip

 

 

 

 

 

 

 

 

 

Try making a transfer to a disk share (or exclusive share)

10 hours ago, JorgeB said:

Try making a transfer to a disk share (or exclusive share)

Thanks for looking.

 

I made a new public share called 'file transfer test' on the array cache nvmes. 

It is shown as 'Exclusive'.

Here is the file transher speed:

 

 

You can see the speed is oscillating between 920 MB/s and 600 MB/s.

Now the question is why the oscillating?

And will I need to stop using the Array and change over to using Pools?

 

I would be fine with using the Array and have the transfer speed oscillate, but I don't want to go through the hassle of switching to Exclusive shares. 

Was there an unRAID version change that caused this slow array write speeds? 

Tried with the normal MTU value of 1500? Running with anything other than 1500 might/will cause a myriad of problems for home networks, using 9000 won't gain you any real speed in a 10Gpbs network anyway.

Might fix the oscillating speed.

Just now, Yock said:

Tried with the normal MTU value of 1500? Running with anything other than 1500 might/will cause a myriad of problems for home networks, using 9000 won't gain you any real speed in a 10Gpbs network anyway.

Might fix the oscillating speed.

 

MTU of 1500 results in 120 MB/s max. 

19 minutes ago, FQs19 said:

 

MTU of 1500 results in 120 MB/s max. 

That can't be right, then there's something wrong some where.

Using a default value of 1500 i saturate my 10Gpbs network.

1 minute ago, Yock said:

That can't be right, then there's something wrong some where.

Using a default value of 1500 i saturate my 10Gpbs network.

I think it's a Ubiquiti thing. 

I've never been able to get above 1 GbE speeds on Ubiquiti gear without enabling Jumbo Frames on their switches. 

Using Ubiquity router and switch my self, granted it's not the newest of gear so might be why.

 

Well if 9000 works and 1500 for some reason don't then you might as well keep using it.

Using anything other than 1500 in a home network is just not normally recommended.

8 minutes ago, Yock said:

Using Ubiquity router and switch my self, granted it's not the newest of gear so might be why.

 

Well if 9000 works and 1500 for some reason don't then you might as well keep using it.

Using anything other than 1500 in a home network is just not normally recommended.

 

Could you tell me what settings you had to change in Unifi to get 10 GbE speeds and are you using VLANs?

 

Like I said, for me I had to enable Jumbo Frames in Unifi and disable Interrupt Moderation and enable jumbo frames on the NIC adapters. 

5 minutes ago, FQs19 said:

 

Could you tell me what settings you had to change in Unifi to get 10 GbE speeds and are you using VLANs?

 

Like I said, for me I had to enable Jumbo Frames in Unifi and disable Interrupt Moderation and enable jumbo frames on the NIC adapters. 

My stuff is to old to be running Unifi. So i'm afraid i'm of little help setting wise.

I pretty much just had the router find my ISP, set up the ports as dumb as possible as that's it. My home network is small so no vlans or anythin,. i try and keep things as simple as possible or my OCD runs away with me.

 

I saw a post where @JorgeB suggested to verify the output of 

lspci -d 1000: -vv

to get the HBA speed/width. 

 

I am getting the full 8GT/s and Width of x8.

So now, I really have no idea why I can't write to an NVME drive above 450 MB/s.

 

14 minutes ago, FQs19 said:

I saw a post where @JorgeB suggested to verify the output of 

lspci -d 1000: -vv

to get the HBA speed/width. 

 

I am getting the full 8GT/s and Width of x8.

So now, I really have no idea why I can't write to an NVME drive above 450 MB/s.

 

I rarely get over 550MB/s on my server on a fuse share.
SMB and fuse transfers takes a lot of resources and are single thread, maybe your CPU doesn't boost for some reason doing transfers or just doesn't have the single threaded needed for higher transfers over the fuse layer.

The fact you can get 900MB/s+ on a disk share kinda suggest that, though the oscillating speed is kinda weird and why i suggested trying changing MTU in case you had some equipment causing problems, though it could be something as simple as SSD cache filling up.

Kernel 6.10 has fuse pass-through and should help a lot when/if we get that.

1 hour ago, Yock said:

I rarely get over 550MB/s on my server on a fuse share.
SMB and fuse transfers takes a lot of resources and are single thread, maybe your CPU doesn't boost for some reason doing transfers or just doesn't have the single threaded needed for higher transfers over the fuse layer.

The fact you can get 900MB/s+ on a disk share kinda suggest that, though the oscillating speed is kinda weird and why i suggested trying changing MTU in case you had some equipment causing problems, though it could be something as simple as SSD cache filling up.

Kernel 6.10 has fuse pass-through and should help a lot when/if we get that.

What's weird is that I can transfer 1025MB/s over to another unRAID server I have until the ssd cache fills up. 

 

It's sounding like I just need to start over with this server. I need to read up on zfs so I can try that out. 

 

Thanks for the help. 

30 minutes ago, FQs19 said:

What's weird is that I can transfer 1025MB/s over to another unRAID server I have until the ssd cache fills up. 

 

It's sounding like I just need to start over with this server. I need to read up on zfs so I can try that out. 

 

Thanks for the help. 

To be 100% honest i don't think a fresh install will work but it's ofc. worth a try.
I'm very curious as to if it will work so if you go that route please write an update.

12 minutes ago, Yock said:

To be 100% honest i don't think a fresh install will work but it's ofc. worth a try.
I'm very curious as to if it will work so if you go that route please write an update.

It'll be a while before I can try zfs. 

If I do, I'll try and remember to post back here. 

CPU can make a big difference, especially its single thread performance, since Samba will only use one thread.

1 minute ago, JorgeB said:

CPU can make a big difference, especially its single thread performance, since Samba will only use one thread.

Understood, but I've used this same CPU since the beginning and I know at one point for a long time, I was getting at least 900 MB/s. 

Hence my confusion. 

I just haven't been doing large transfers transfers for a while. 

 

I will go into the bios and confirm all settings are correct. I will also adjust the performance settings in Tips & Tweaks to see if that charges anything.