• MacOS Optimization


    limetech
    • Minor

    Starting with 6.11.0-rc5 we have added some features to help us figure out the proper mix of SMB settings which will achieve the best performance/functionality with MacOS.  This mainly involves tuning the so-called "fruit" SMB parameters.  Please refer to the Samba "vfs_fruit" doc:

    https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html

     

    First you will notice there are Global options and Per-Share options.  The global options are set in /etc/samba/smb.conf file on your server.  All the options are listed near the bottom, and ones which vary from the default setting are uncommented.  I don't think we will need to change these options but if you want to experiment then add your changes to /boot/config/smb-extra.conf file.

     

    Next are the set of per-share options.  These are set in /etc/samba/smb-fruit.conf file on your server.  Again all options are listed there and only the ones which deviate from default are uncommented.

     

    In addition, you may create the file /boot/config/smb-fruit.conf on your flash device and when Samba starts or is restarted, those options will override the options in /etc/samba/smb-fruit.conf.  Thus a good staring point would be to:

    cp /etc/samba/smb-fruit.conf /boot/config

    and now you can make changes to /boot/config/smb-fruit.conf

    After making a change you can type this command to to restart Samba:

    samba restart

     

    For any of this to be applied, you first need to ensure that "Settings/SMB/Enhanced macOS interoperability" is set to Yes.  This will tell Unraid OS to include the contents of the smb-fruit.conf file in each share (except for the 'flash' share, see below).

     

    Note: we actually have per-share hidden settings for enabling 'fruit', however from the documentation we read:

    Quote

    Be careful when mixing shares with and without vfs_fruit. OS X clients negotiate SMB2 AAPL protocol extensions on the first tcon, so mixing shares with and without fruit will globally disable AAPL if the first tcon is without fruit.

     

    Thus there is no UI for these settings, instead either all shares or no shares have 'fruit' extensions.  As for the 'flash' share, again from the doc we read:

    Quote

    The module enables alternate data streams (ADS) support for a share, intercepts the OS X special streams "AFP_AfpInfo" and "AFP_Resource" and handles them in a special way. All other named streams are deferred to vfs_streams_xattr which must be loaded together with vfs_fruit.

    (emphasis mine).

     

    They say 'streams_xattr' must be loaded, but later in the doc they talk about options that are incompatible with this.  In addition certain file system types are inherently incompatible, notably FAT and exFAT because they don't support xattr at all...  Hence we can't (or shouldn't) include the 'flash' share in 'fruit' but then if the 'flash' share is first share to be accessed on a Mac client, APPL extensions will be disabled for the duration... Yes this is confusing, maybe best bet is to not export the 'flash' share in MacOS environment.  If it is exported, the 'testparm' command outputs this warning:

    WARNING: some services use vfs_fruit, others don't. Mounting them in conjunction on OS X clients results in undefined behaviour.

     

    Time machine - if enabled for a share these parameters are automatically added (independent of smb-fruit.conf):

    fruit:time machine = yes
    fruit:time machine max size = SIZE # if "vol size limit" specified

     

    Finally Spotlight - our build of Samba includes spotlight support but only for 'elasticsearch'.  But 'elasticsearch' is not installed in Unraid OS.  In other places on the forum people have expressed success by setting the spotlight backend to 'tracker' - but this requires "gnone tracker" which also is not installed in Unraid OS - so I'm not sure what's going on here.  Nevertheless, you can add 'spotlight = on' to the smb-fruit.conf file to play around with this.

     

    Please limit discussion in this topic to MacOS issues only.

     

    • Like 10
    • Thanks 1
    • Upvote 1



    User Feedback

    Recommended Comments



    Here are the current contents of smb-fruit.conf

     

    # global parameters are defined in /etc/rc.d/rc.samba
    # current per-share Unraid OS defaults
    vfs objects = catia fruit streams_xattr
    #fruit:resource = file
    #fruit:metadata = netatalk
    #fruit:locking = none
    #fruit:encoding = private
    fruit:encoding = native
    #fruit:veto_appledouble = yes
    #fruit:posix_rename = yes
    #readdir_attr:aapl_rsize = yes
    #readdir_attr:aapl_finder_info = yes
    #readdir_attr:aapl_max_access = yes
    #fruit:wipe_intentionally_left_blank_rfork = no
    #fruit:delete_empty_adfiles = no
    #fruit:zero_file_id = no
    # these are added automatically if TimeMachine enabled for a share:
    #fruit:time machine
    #fruit:time machine max size = SIZE

     

    I see that there aren't any global or share name tags so I may be getting confused by what I've read on smb.org. Are those tags not required at all here?

     

    Edit: Nevermind

    Quote

    If config/smb-fruit.conf exists then it is added to the settings for every share automatically,

    I guess I'm a little dense tonight. 🤪

     

    Would still like to know if I should delete my current settings from smb-extra though.

    Edited by wgstarks
    Link to comment
    11 hours ago, wgstarks said:

    Would still like to know if I should delete my current settings from smb-extra though.

    Yes

    • Thanks 1
    Link to comment
    11 hours ago, wgstarks said:

    I see that there aren't any global or share name tags so I may be getting confused by what I've read on smb.org. Are those tags not required at all here?

    The options specified in /etc/samba/smb.conf and config/smb-extra.conf are all put into the [global] section.

    The options specified in /etc/samba/smb-fruit.conf or config/smb-fruit.conf are added to each share.  You don't  have to specify any sections.

    • Thanks 1
    Link to comment
    59 minutes ago, limetech said:

    The options specified in /etc/samba/smb.conf and config/smb-extra.conf are all put into the [global] section.

    The options specified in /etc/samba/smb-fruit.conf or config/smb-fruit.conf are added to each share.  You don't  have to specify any sections.

    Thanks. I eventually figured that out last night. It was obvious once I re-read your previous post regarding how those settings are applied.

    Link to comment
    On 10/4/2022 at 9:44 PM, dlandon said:

    Copy the /etc/samba/smb-fruit.conf to /flash/config/smb-fruit.conf.  Then make changes to the /flash/config/smb-fruit.conf file.  After making changes, you need to restart samba:

    /etc/rc.d/rc.samba restart

     

    You need to make changes to the copied smb-fruit.conf because it contains the "vfs objects = catia fruit streams_xattr".  You'll see uncommented lines.  Those are the defaults.  The commented lines are lines you can uncomment to try out different settings.

     

    Just a note because it threw me off - restarting samba didn't apply my /(flash|boot)/config/smb-fruit.conf changes. Stopping the array and restarting did though.

    Link to comment
    9 minutes ago, CS01-HS said:

    Just a note because it threw me off - restarting samba didn't apply my /(flash|boot)/config/smb-fruit.conf changes. Stopping the array and restarting did though.

    Yes,  I stand corrected.

    • Like 1
    Link to comment
    47 minutes ago, CS01-HS said:

     

    Just a note because it threw me off - restarting samba didn't apply my /(flash|boot)/config/smb-fruit.conf changes. Stopping the array and restarting did though.

    I ran into the same issue. Not sure if this is a bug or if the OP is wrong.

    Link to comment
    8 minutes ago, wgstarks said:

    I ran into the same issue. Not sure if this is a bug or if the OP is wrong.

    Now that I look at it, I don't think it's a bug, it's wrong in the OP.  Unraid doesn't apply the settings on a samba restart, it applies the changes when there is a change in the share settings - when changing the share, or restarting the server.

    • Thanks 1
    Link to comment
    1 minute ago, dlandon said:

    Now that I look at it, I don't think it's a bug, it's wrong in the OP.  Unraid doesn't apply the settings on a samba restart, it applies the changes when there is a change in the share settings - when changing the share, or restarting the server.

    So stop/start array is really the correct method?

    Link to comment

    Still no luck on getting Time Machine to work with Unraid 6.11.1. Nothing shows up in the syslog to indicate a crash of SMB. The typical "Operation not supported by device" UserInfo={DIErrorVerboseInfo=Failed to initialize IO manager: Failed opening folder for entries reading}" still shows up on the MacOS (12.6) side.

     

    I've got zero extra Samba custom configurations/files, so this is running with vanilla Unraid as far as I'm aware. 

     

    testparm -sv shows the following: https://pastebin.com/mmYPTz9n

     

    Specifically, here is the share for TimeMachine

     

    [TimeMachine]
    	path = /mnt/user/TimeMachine
    	valid users = backup
    	vfs objects = catia fruit streams_xattr
    	write list = backup
    	fruit:time machine max size = 1200000M
    	fruit:time machine = yes
    	fruit:encoding = native

     

    @limetech As requested, I've output diagnostics from Unraid as well immediately after a reboot and failing to back up from two different computers.

    pbox-diagnostics-20221017-1212.zip

    • Thanks 1
    Link to comment
    5 hours ago, kubed_zero said:

    I've got zero extra Samba custom configurations/files, so this is running with vanilla Unraid as far as I'm aware. 

    Thank you for your report.  I have TM backups running ok with Monterey (12.6).  At first I thought it was because the share is marked 'public'.  Changed to 'private' and there was some connectivity issues but eventually poked around the TM preferences and got it to work again... so the investigation continues...

    Link to comment
    20 minutes ago, limetech said:

    poked around the TM preferences and got it to work again

    Got it. Well, if there's anything I can help with (different settings, additional logs on the macOS or Unraid side, reverting to 6.9.2 and diffing outputs, etc) let me know!

    • Thanks 1
    Link to comment

    My TimeMachine got all screwed up after the 6.11.1 upgrade :(  I am redoing the backup on this MacBook 16 but my other 15 won't backup to its existing instance or a new instance. Very stubborn. I will wait for the 16 to finish and try the 15 one again. Something substantially changed in 6.11.1 that will cause this weird behavior?

     

    So the 16" runs macOS Ventura and the 15" runs Big Sur if that makes a difference? 

     

    Edited by johnwhicker
    Link to comment

    Anyone got some good fruit settings to have good SMB performance for file shares on Mac, 10Gbps and Mac is so shit can barley open a folder. One Thing apple have in the toilet.

    • Thanks 1
    Link to comment
    28 minutes ago, Nano said:

    Anyone got some good fruit settings to have good SMB performance for file shares on Mac, 10Gbps and Mac is so shit can barley open a folder. One Thing apple have in the toilet.

     

    This is what I'm using-

    # global parameters are defined in /etc/rc.d/rc.samba
    # current per-share Unraid OS defaults
    vfs objects = catia fruit streams_xattr
    #fruit:resource = file
    fruit:metadata = stream
    #fruit:locking = none
    #fruit:encoding = private
    fruit:encoding = native
    #fruit:veto_appledouble = yes
    fruit:posix_rename = yes
    #readdir_attr:aapl_rsize = yes
    #readdir_attr:aapl_finder_info = yes
    #readdir_attr:aapl_max_access = yes
    #fruit:wipe_intentionally_left_blank_rfork = no
    #fruit:delete_empty_adfiles = no
    #fruit:zero_file_id = no
    # these are added automatically if TimeMachine enabled for a share:
    #fruit:time machine
    #fruit:time machine max size = SIZE

     

    Been working fairly well for me.

    Link to comment

    We need an easy way to enable spotlight search for share Aswell, I found 1 thread on it but its crap and doesn't work.

     

    EDIT that actually has improved massively thank you, Samba was not rebooting properly

     

    Search are still really slow - 10-15 seconds to search for a file in a directory - maybe a good/easy spotlight integration would fix it? If anyone knows the easiest/best way or if its possible please let me/thread know!

     

    Once the folders show up, My transfer speed is perfect it's just the indexing/loading of the file structure!

    Edited by Nano
    • Like 1
    Link to comment
    1 minute ago, Nano said:

    We need an easy way to enable spotlight search for share Aswell, I found 1 thread on it but its crap and doesn't work.

    According to the OP spotlight shouldn’t work at all. I have seen a few claims that it works but nothing describing how it was accomplished.

     

    4 minutes ago, Nano said:

    It still seems to take over 30 seconds to open a folder with 100+ Folders in it.

    I see pretty much the same when the share is initially mounted and accessed but when opening folders after that it’s just a second or two.

    Link to comment
    Just now, wgstarks said:

    According to the OP spotlight shouldn’t work at all. I have seen a few claims that it works but nothing describing how it was accomplished.

     

    I see pretty much the same when the share is initially mounted and accessed but when opening folders after that it’s just a second or two.

    Sorry edited post after yes it helped thanks! 

    Link to comment
    Just now, Nano said:

    Sorry edited post after yes it helped thanks! 

    I think overall it’s a huge improvement but still more room for enhancements.

    Link to comment

    I removed all my "Samba extra configuration" as i was going backwards when i add the extras i used before 6.11

     

    My main issue was speed, when connected to a shared drive it would take 1 minute to move 2 x pdfs that are 47Kb in size from one folder to another folder in the same share, copying new files (10 x small pdfs to the share i could go make a coffee and still see the files copying to the share from my MAC.

    Using Quick Look on the Mac to view the pdf's in the share would take 10-30 seconds to preview.

    It was becoming unusable.

     

    Searches would take a minimum 1 minute and some times way longer. But would work.

     

    The above shares are on an NVME cache format XFS and SSD cache made of 2 x SSD format BTRFS Mirror and

    contained between 200 - 300 pdfs.

     

    I had problems with folders that contained more than 600 files in earlier version of UNRAID 6.9 and 6.10 but now it seems to be any thing with more than 60 files in a folder.

     

    Reducing the number of files down to 30 - 40 in a folder I was able to move files between folders very quickly, copy files to the share without the need for a coffee break and the search was almost instant.

     

    So the question is, why should the number of files in a folder have such an impact on the performance of the SMB share ?

     

    The SMB setting are set to default.

     

     

     

     

     

     

     

    Screenshot 2022-11-19 at 01.57.09.png

    Edited by wow001
    Link to comment

    I can't answer any of your questions but I can suggest settings that are working fairly well for me.

     

    Stop your array and edit boot/config/smb-fruit.conf to match these-

    # global parameters are defined in /etc/rc.d/rc.samba
    # current per-share Unraid OS defaults
    vfs objects = catia fruit streams_xattr
    #fruit:resource = file
    fruit:metadata = stream
    #fruit:locking = none
    fruit:encoding = native
    #fruit:veto_appledouble = yes
    fruit:posix_rename = yes
    #readdir_attr:aapl_rsize = yes
    #readdir_attr:aapl_finder_info = yes
    #readdir_attr:aapl_max_access = yes
    #fruit:wipe_intentionally_left_blank_rfork = no
    #fruit:delete_empty_adfiles = no
    #fruit:zero_file_id = no
    # these are added automatically if TimeMachine enabled for a share:
    #fruit:time machine
    #fruit:time machine max size = SIZE
    #spotlight settings
    spotlight = yes

     

    In your smb extra configuration add this-

    #spotlight settings
    [global]
       spotlight backend = tracker
    #end spotlight settings

     

    Start the array.

     

    I’m getting fair transfer speeds and Finder searches are working great. No spotlight indexing though. This is with macOS Ventura (13.0.1) and unRAID 6.11.4. Your results may vary.

    Link to comment

    Thanks wgstarks.

    I created the boot/config/smb-fruit.conf on my flash drive and added your suggested settings, restarted the server and they was a big improvement in copying files to folders with large number of files in them.

     

    From my tests you do not need the:

     

    #spotlight settings

    spotlight = yes

     

    in the boot/config/smb-fruit.conf

     

    or 

     

    #spotlight settings

    [global] spotlight backend = tracker

    #end spotlight settings

     

    In your smb extra configuration

     

    It seemed to make the search slower and in folders that have 700 files or more the search timed out before finding the file.

     

    So the setting i am using at the moment are:

    (in the boot/config/smb-fruit.conf)

     

    # global parameters are defined in /etc/samba/smb.conf

    # current per-share Unraid OS defaults

    vfs objects = catia fruit streams_xattr

    #fruit:resource = file

    fruit:metadata = stream

    #fruit:locking = none

    #fruit:encoding = private

    fruit:encoding = native

    #fruit:veto_appledouble = yes

    fruit:posix_rename = yes

    #readdir_attr:aapl_rsize = yes

    #readdir_attr:aapl_finder_info = yes

    #readdir_attr:aapl_max_access = yes

    #fruit:wipe_intentionally_left_blank_rfork = no

    #fruit:delete_empty_adfiles = no

    #fruit:zero_file_id = no

    # these are added automatically if TimeMachine enabled for a share:

    #fruit:time machine

    #fruit:time machine max size = SIZE

     

    Although this has massively improved copying to the share via SMB and previewing pdfs with Quick Look, as soon as you exceed approx 150 files in a folder you start getting slower copy times and quick look preview starts taking time to load.

     

    So it seems the speed of the SMB is linked directly to the quantity of files in a folder.

    Link to comment

    I had to actually use even more options to get it to work with folders containing hundreds of files:

     

    vfs objects = catia fruit streams_xattr
    fruit:metadata = stream
    fruit:encoding = native
    readdir_attr:aapl_rsize = no
    readdir_attr:aapl_finder_info = no
    readdir_attr:aapl_max_access = no
    

     

    I didn't need POSIX rename so far, but I did not rename any files yet.

    Also I do not know how well this works with TimeMachine, I may test this at a later point in time.

     

    Link to comment
    On 10/6/2022 at 9:19 AM, CS01-HS said:

     

    Just a note because it threw me off - restarting samba didn't apply my /(flash|boot)/config/smb-fruit.conf changes. Stopping the array and restarting did though.

    @limetech - fantastic work here - I've said it before and I'll say it again - I'm constantly impressed with the way you guys focus the features on the customer in a way that nobody else does.  Regarding the above comment, is this something to do with a licensing dependency?  Otherwise I have noted some similar behaviour requiring array restart before and it would certainly be much nicer if the whole docker system and vm's didn't have to be stopped for these kinds of small changes.  Though I must say I'm not sure that this is normally required for SMB changes.

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.