Large copy/write on btrfs cache pool locking up server temporarily


Recommended Posts

Running the latest beta 6.9.0-beta25

Formatted my SSD's to the new partition alignment.

 

Massive boost in speed on my Samsung 860Evo and Qvo drives!

And not locking up when I do alot of transfers as they have done earlier.

Edited by Allram
  • Like 1
  • Thanks 2
Link to comment
  • 3 weeks later...

Think I was seeing this same thing on a smaller Adata SSD the SU635 / 240GB.  MakeMKV docker pinned to 1 core writing just 30MB/s to cache SSD would get all cores/threads to a high iowait after 15 minutes or so.  Found this thread and swapped the cache to XFS and the iowait impact is much reduced now.

 

Reads from the SSD to array were 200-300MB/s no problem when I was moving stuff off cache so I could reformat, no iowait at all.

 

Looking forward to update, but if you don't need a cache pool I guess just sticking with XFS isn't a bad solution seems to work well this way.

Link to comment
  • 4 weeks later...

Hi,

 

I have 2 NVME Drives both capable of 3000 mb/s write speed

 

Samsung 970 EVO Plus 1TB

Sabrent 1TB Rocket NVMe PCIe 1TB

 

I could write to both of those individually at 1000 mb/s using my fibre connection from my Desktop

 

Together as a BTRFS cache pool I am only getting 230 mb/s 

 

I have installed the latest beta, removed and reinstalled both nvme drives as a cache pool and reformatted to the MBR: 1MiB-aligned  

 

I have attached my diagnostics in case anyone can help please?

 

 

unraid1-diagnostics-20200827-0900.zip

Link to comment
29 minutes ago, johnnie.black said:

There are no multi device xfs pools, you can have single device xfs "pools", multi device pools only with btrfs, or zfs with the zfs plugin, but those won't be available on the GUI.

OK, Thanks for that, So it is looking like BTRFS doesn't work for me on the latest beta so is that likely to be due to the Samsung Drive? I can purchase another Sabrent Rocket?

Link to comment
30 minutes ago, johnnie.black said:

Difficult to say if it will help, some users have bad performance with pools, others not, Samsung should work fine with the new alignment though.

OK, I can only presume it is because of the mix of Sabrent and Samsung If the below looks ok to you I will purchase another Sabrent to see if it makes a difference.

 

image.thumb.png.c1ce6a599be71a789d6c49423bdc56eb.pngimage.png.ed2c571acb6e774abf5ff13df52e3654.png

image.png.ecbb59c1fdb5bccf76a4f5c53aef836a.png

 

 

Link to comment
5 hours ago, BRiT said:

Did you newly create and format the drives in 6.9 beta or were they carry overs from earlier?

 

On 8/27/2020 at 9:01 AM, mbc0 said:

I have installed the latest beta, removed and reinstalled both nvme drives as a cache pool and reformatted to the MBR: 1MiB-aligned 

 

Link to comment

@johnnie.black

 

I have removed the Samsung from the cache pool leaving a sole Sabrent Rocket but I am still only getting 230mb/s could this be due to the partition alignement? I can get 1000mb/s from this drive as an unassigned device or because unraid still thinks it is a pool rather than a single drive?

 

is this prooving that the sabrent rocket is the problem here?

 

image.thumb.png.d94fd8abac8218b0d90e95282891b607.png

 

image.png.64ded40fb6356409362de9183c70936e.png

 

 

Link to comment
19 hours ago, mbc0 said:

I am still only getting 230mb/s could this be due to the partition alignement? I can get 1000mb/s from this drive as an unassigned device or because unraid still thinks it is a pool rather than a single drive?

No, this just means the pool itself wasn't the reason for the performance issues.

Link to comment
19 hours ago, mbc0 said:

@johnnie.black

 

I have removed the Samsung from the cache pool leaving a sole Sabrent Rocket but I am still only getting 230mb/s could this be due to the partition alignement? I can get 1000mb/s from this drive as an unassigned device or because unraid still thinks it is a pool rather than a single drive?

 

is this prooving that the sabrent rocket is the problem here?

 

image.thumb.png.d94fd8abac8218b0d90e95282891b607.png

 

image.png.64ded40fb6356409362de9183c70936e.png

 

 

Since you are running a single disk now. What speeds do you get if you format the drive to XFS?

Link to comment
3 hours ago, Allram said:

Since you are running a single disk now. What speeds do you get if you format the drive to XFS?

OK, can't believe this but me setting up a cache pool has coincided with an update from avast that has dropped my speeds to 230mb/s! 

 

I tried a transfer from VM to VM (unraid1 to unraid2) and speeds were spot on, looked through all my Windows programs and stopped them one by one and as soon as I stopped avast speeds are back to 1000mb/s 

 

So sorry to have wasted yours and anyone else's time but hope this helps someone else at some point!

  • Like 2
Link to comment
  • 2 years later...

This is an old thread (apologies), but this still doesn't appear to be fixed on the latest `6.11.5`?

 

When I switch from btrfs raid 1 to XFS non raid, can I do that encrypted, or does it have be be an unencrypted XFS disk? I read something in this thread about encryption possibly attributing to this issue.

Link to comment
9 minutes ago, trurl said:

You have to reformat, so new format can be encrypted or not.

 

Thank you for the response. I understand that part though.

 

I default to encrypt everything, but I'm wondering if doing an encrypted XFS drive is going to fix this, or if I need to ultimately do an unencrypted XFS disk. Normally encryption doesn't impact perceived performance... but normally raid1 doesn't cause huge iowaits with freezing docker containers :).

Link to comment
On 1/15/2020 at 10:58 PM, robobub said:

I had similar symptoms, using an older Samsung 830 SSD as a single Btrfs LUKS-encrypted cache.  When copying very large file, iowait would hit the 80's and then at some point the system became unresponsive, and write speeds were around 80 MB/s.  Howerver, moving to XFS LUKS-encrypted did not help things at all.

 

In my case, it had to do with LUKS-encryption.  Moving to non-encrypted cache, either Btrfs or XFS, iowait would be much lower, and write speeds at 200.  However, I'm on an i7-3770 which has AES acceleration and have barely any CPU utilization  

 

One guess is that the 830 controller doesn't handle incompressible data as well, but looking at reviews, that's where it shined compared to Sandforce controllers.  

 

Some searching lead me to this post:

 

Setting the IO Scheduler to none for my cache drive helped a bit, but lowering nr_requests with any IO scheduler helped more, at least in my case.

 

This post for example.

Link to comment
9 hours ago, guitarlp said:

I default to encrypt everything, but I'm wondering if doing an encrypted XFS drive is going to fix this, or if I need to ultimately do an unencrypted XFS disk.

Try for yourself, there's no clear reason why this issue happens to some users, so also no clear solution, another thing that might be worth trying is this:

https://forums.unraid.net/topic/91883-tracking-down-iowait-cause/?do=findComment&comment=1221804

 

Link to comment
  • 9 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.