Slow speed writing to cache. (SOLVED)


Recommended Posts

Hi.

 

I've read several posts and topics regarding slow cache, and I have ran out of ideas.

 

My system is:

Dell Poweredge R720XD with Perc710 in IT mode.

5x 2TB SATA HGST with 2 parity disks.

1x NVMe on PCI-e adapter. Recognized and working fine.

10GbE

 

Writing to folders Multimedia with cache gives me only 250MB/s which is very slow, compared to my other UR NAS which writes 1,12GB/s. 

 

After reading some other posts here, I enabled "Disk Shares" from within settings.

I could now se ALL my drives, and also my cache drive. When writing directly to cache, I get full speed 1,12GB/s.

 

Why is it so, that when I write to my Multimedia folder (which I enabled to use cache) I get a slower speed then when I write directly to cache?

I also tried to write directly to disk1 which also gives me 1,12GB/s. How is this possible, when this is a SATA-3 6Gb/s disk?

 

 

Edited by j44rs4
Link to comment
18 hours ago, j44rs4 said:

Why is it so, that when I write to my Multimedia folder (which I enabled to use cache) I get a slower speed then when I write directly to cache?

There's always some overhead when writing to an user share, but for some reason some users see a much more large impact tan others, e.g. I get around 800MB/s vs 1GBs writing to user vs disk share, enabling direct i/o helps in some cases.

 

18 hours ago, j44rs4 said:

I also tried to write directly to disk1 which also gives me 1,12GB/s. How is this possible, when this is a SATA-3 6Gb/s disk?

Because first Unraid caches the writes to RAM, it will eventually drop to device speed if it's a large enough transfer.

Link to comment

Thx for your reply.

 

Unfortunately I've already activated Direct IO, with no better outcome.

 

I've tried serveral different disks (from nvme, SAS, SSD and SATA) as cache disk, but with the same performance.

 

Is seems like it's not using the cache drive correctly when writing to shared folder. Since I get better speed when writing directly to cache, than writing to shared folder through cache. 

 

1.PNG - Shows copy directly to cache

2.PNG - Shows copy to sharefolder

3.PNG -  Shows copy directly to disk1

 

1.PNG

2.PNG

3.PNG

Link to comment

If direct i/o doesn't help I don't know of anything else you can do other than trying different hardware, ideally hardware you already have availble, and that doesn't necessarily mean faster/better hardware, just different, though since SMB is single thread CPUs with better performance on a single core are usually better, i.e., a 3.5Ghz quad core is usually faster than an 8 core 2.5GHz CPU for this.

Link to comment
On 8/24/2020 at 2:28 AM, j44rs4 said:

Thx for your reply.

 

Unfortunately I've already activated Direct IO, with no better outcome.

 

I've tried serveral different disks (from nvme, SAS, SSD and SATA) as cache disk, but with the same performance.

 

Is seems like it's not using the cache drive correctly when writing to shared folder. Since I get better speed when writing directly to cache, than writing to shared folder through cache. 

 

1.PNG - Shows copy directly to cache

2.PNG - Shows copy to sharefolder

3.PNG -  Shows copy directly to disk1

 

1.PNG

2.PNG

3.PNG

We’re you able to find a work around or fix for this....

Link to comment

@Tekminute Sorry, no fix so far. the HW seems to be handling speed as you can see, but not when writing to cache. The only bottleneck I can think of must be the CPU (Intel Xeon CPU E5-2670 v2 @ 2.50GHz)

I have 64gigs of RAM, so that should be plenty enough.

 

But still I don't understand why writing to share through cache could make such a problem, while writing directly to all other disks, give me full speed.

Link to comment

Another test done this moment:

 

Writing to share Without cache drive, gives me a speed of 450-500MB/s, while activating the cache only give me 260MB/s.

 

Can someone explain to me why writing to share with cache enabled, is slower than writing directly to share?

 

I have 3 different cache disks I've been playing with. The first one, is a NVMe WDS250G3X0C, and the second one, is a SATA SSDd from Dell MZ7LH960HBJR0D3, and the third one is a SAS 300Gig disk 15000RPM.

 

All these disks can only perform a speed of 260MB/s

 

As for now, my system is slower with cache disk than without.

Link to comment

I'm talking MegaBytes (MB)

Hmmm, let med post you some pics then.

 

Pictures 1-3 is copy to share Test, without cache drive, on a 10Gb/s ethernet. As you can se on pic2, you'll see that I've disabled writing to cache.

 

If enable cache on the share Test, I get lower speed.

1.PNG

2.PNG

3.PNG

Link to comment
On 8/24/2020 at 2:22 PM, JorgeB said:

If direct i/o doesn't help I don't know of anything else you can do other than trying different hardware, ideally hardware you already have availble, and that doesn't necessarily mean faster/better hardware, just different, though since SMB is single thread CPUs with better performance on a single core are usually better, i.e., a 3.5Ghz quad core is usually faster than an 8 core 2.5GHz CPU for this.

 

Replaced my Intel Xeon E5-2670v2 2.5GHz with a Intel Xeon E5-2637v2 3.5GHz, and the writing is still the same.

 

Any other suggestions? Anyone?

Link to comment
  • 1 month later...

Sorry, no.

 

I'm working out a new way to test, the speed.

 

I've figured out that Perc h710p controller only supports PCIe 2.0 even though they say, "When you flash your H710p to IT-mode, it will start to work as a PCIe 3.0"

Maybe that's for the PCIe adapter card and not the internal module.

https://www.dell.com/support/article/no-no/sln292279/list-of-poweredge-raid-controller-perc-types-for-dell-emc-systems?lang=en

I Also important to notice , is that this controller only supports 6Gb/s, so there is no way this controller can do 10Gb/s alone.

 

As for the m.2 disk I inserted, I used a PCIe-adapter, and still no perfomance.

 

I will try to add a H730P PCIe-card to my server, which indeed is a PCIe 3.0, and handles 12Gb/s.

I have 4x 2.5' SAS SSD, that also support 12Gb/s, and will try and see if that makes any impact on my performance. 

 

Will report back, when I'm done testing, so it might take a while. Hopefully before X-mas. :D

  • Like 1
Link to comment
  • 1 month later...

UPDATE.

 

After many ways of testing, I figured out that none of the suggestions mentioned in this thread had any impact on the system.

 

I also tried out with a Perc H730, to see if that could make any difference, since it has a PCIe 3.0 bus confirmed by DELL, and do support 12Gb/s bandwith. The test turned out to have no impact.

 

Did some more testing, and reset my BIOS to FACTORY settings, and still nothing.

 

Solution:

Made some minor changes in Bios, as you can see from posted pics.

1. Changed the System BIOS Settings -> System Profile Settings -> Performance

2. Changed the System BIOS Settings -> Processor Settings -> Dell Controlled Turbo - ENABLED

3. Changed the System BIOS Settings -> Integrated Devices -> Memory Mapped I/O above 4GB - ENABLED

 

Setting number 3. made the biggest impact.

 

1.png

2.png

3.png

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.