• Writing to encrypted xfs very slow (10-15MB/s)


    Niklas
    • 6.7.0-rc4 Solved Minor

    I get speeds 10-15 MB/s when writing to encrypted xfs. Even with no parity. Problem still there in safe mode. 

     

    The slow speed makes transferring files from windows computers almost unusable (explorer freezes and transfers stops until the data cached in ram is done writing to disk and that could take some time at 10MB/s). 

     

    Writing to unencrypted xfs gives me full speed. 

     

    Writing to encrypted btrfs (cache ssd raid1) gives me full speed (cache only share). 

     

    This with RC2 and RC3. Can't say if it was a problem in RC1.




    User Feedback

    Recommended Comments

    My controller is IBM M1015 crossflashed to LSI9211-IT.

    I don't think it is a controller problem. I'm getting full speed to drives that is unencrypted xfs.

    Edited by Niklas
    Link to comment
    4 minutes ago, JoeUnraidUser said:

    I just rolled back to 6.6.6 and the problem is gone.  Insted 10-15MB/s in the RC releases, I am now reading and writing at a solid 113MB/s.  My processor utilization has also gone down significantly.

    Good to know. :)

    This is to encrypted btrfs (SSD). Small dips where ram is writing to disk (i guess). The problem seem to be limited to the array or enc xfs.
    970322066_Capture_86_klart_2019-02-10_19-55-52_58614426.png.1c0c51f914458d3e7bcb45376025f35f.png

    Edited by Niklas
    Link to comment

    Yes, i also encrypted my whole array. And have VERY slow speeds. Fione with 6.6.6, since RC2, only problems.

     

    Like radar improt of 6 files normally <5 min, currently its working for 6 hours or something like that 2/6 files (!)

    Edited by nuhll
    Link to comment

    There's something wrong with encryption on 6.7.0-rc - CPU utilization is very low (2% on 6.7.0-rc and 12% on 6.6.6) and writing to encrypted array is about 1/10 of speed that I get on 6.6.6

    Link to comment

    My server reported 200 (and more) avg load when writing data. CPU at low usage but yeah, the load was crazy.

    Edited by Niklas
    Link to comment

    Btw. Another thing. I am streaming from disk1 but ui shows all disks as spun down. Writing to disk1 show parity and disk1 spun up. 

     

    My array is not encrypted for the time being. Going to switch back when the problem is fixed. 

    Edited by Niklas
    Link to comment
    26 minutes ago, limetech said:

    Changed Status to Retest

     

    Better! It still drops down to 0 but just a couple of seconds. Like ram to disk not flushed or something?

    Starts at about 50 sec (this is with reconstruct write). I don't see this writing to the unencrypted drive.

    Edited by Niklas
    Link to comment

    Don't see your diags, how much RAM in your server?  Also how big is the file, 9GB?

    This is probably normal now.

     

    Edit: actually please post diagnostics.zip, would like to look at your config.

    Link to comment

    16GB ram. File is 10. Some specs in signature.

    Load avg was over 4 when transferring to encrypted xfs. I have 4 cores. No ht. CPU % at 30-40.

     

    Edited by Niklas
    Diags removed. *paranoid* ;)
    Link to comment

    Looks like you have 2 data, 1 parity disk, with md_write_method set to "reconstruct write".  That means you should get fairly close to speed of disks with sustained write.  Not sure what's happening in those 'dips', could be a number of things.  Maybe interesting to compare to 6.6.6 release, but I think this Bug Report is "solved", agreed?

    • Like 1
    Link to comment

    I did a similar test and copied a 14GB file directly to my encrypted array.

     

    image.png.ac576f6848086d11a8647524c3377271.png

     

    I achieve a constant speed of around 114 MB/s without any speed drops.

     

    Note: my disk caching size is set to 2 GB

    Link to comment
    Just now, limetech said:

    Looks like you have 2 data, 1 parity disk, with md_write_method set to "reconstruct write".  That means you should get fairly close to speed of disks with sustained write.  Not sure what's happening in those 'dips', could be a number of things.  Maybe interesting to compare to 6.6.6 release, but I think this Bug Report is "solved", agreed?

    Yes, agreed. I don't see 10-15MB/s now. 

    Link to comment
    1 minute ago, bonienl said:

    I did a similar test and copied a 14GB file directly to my encrypted array.

     

    image.png.ac576f6848086d11a8647524c3377271.png

     

    I achieve a constant speed of around 114 MB/s without any speed drops.

     

    Note: my disk caching size is set to 2 GB

    vm.dirty_background_ratio and vm.dirty_ratio? What do I need to change? ;)

    Link to comment
    1 minute ago, Niklas said:

    vm.dirty_background_ratio and vm.dirty_ratio? What do I need to change? ;)

    I have set the following in my go file

    # limit disk caching memory (2GiB/1GiB)
    sysctl -qw vm.dirty_bytes=2147483648
    sysctl -qw vm.dirty_background_bytes=1073741824

     

    Link to comment

    Ok :D

     

    2007990921_Capture_99_klart_2019-02-15_23-05-11_38003770.png.545b8b9d708bb7aa0a14c7220a64862f.png

    Tried this in Tips And Tweaks (default 10 and 20)
    Capture_ServerTipsAndTweaks_-_Google_Chrome_2019-02-15_23-05-41_80815781.png.e65d2d40ca02524fd77ee750cc519b5a.png

    Edit: Is this bad in any way?

    Edited by Niklas
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.