• MacOS Optimization


    limetech
    • Minor

    Starting with 6.11.0-rc5 we have added some features to help us figure out the proper mix of SMB settings which will achieve the best performance/functionality with MacOS.  This mainly involves tuning the so-called "fruit" SMB parameters.  Please refer to the Samba "vfs_fruit" doc:

    https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html

     

    First you will notice there are Global options and Per-Share options.  The global options are set in /etc/samba/smb.conf file on your server.  All the options are listed near the bottom, and ones which vary from the default setting are uncommented.  I don't think we will need to change these options but if you want to experiment then add your changes to /boot/config/smb-extra.conf file.

     

    Next are the set of per-share options.  These are set in /etc/samba/smb-fruit.conf file on your server.  Again all options are listed there and only the ones which deviate from default are uncommented.

     

    In addition, you may create the file /boot/config/smb-fruit.conf on your flash device and when Samba starts or is restarted, those options will override the options in /etc/samba/smb-fruit.conf.  Thus a good staring point would be to:

    cp /etc/samba/smb-fruit.conf /boot/config

    and now you can make changes to /boot/config/smb-fruit.conf

    After making a change you can type this command to to restart Samba:

    samba restart

     

    For any of this to be applied, you first need to ensure that "Settings/SMB/Enhanced macOS interoperability" is set to Yes.  This will tell Unraid OS to include the contents of the smb-fruit.conf file in each share (except for the 'flash' share, see below).

     

    Note: we actually have per-share hidden settings for enabling 'fruit', however from the documentation we read:

    Quote

    Be careful when mixing shares with and without vfs_fruit. OS X clients negotiate SMB2 AAPL protocol extensions on the first tcon, so mixing shares with and without fruit will globally disable AAPL if the first tcon is without fruit.

     

    Thus there is no UI for these settings, instead either all shares or no shares have 'fruit' extensions.  As for the 'flash' share, again from the doc we read:

    Quote

    The module enables alternate data streams (ADS) support for a share, intercepts the OS X special streams "AFP_AfpInfo" and "AFP_Resource" and handles them in a special way. All other named streams are deferred to vfs_streams_xattr which must be loaded together with vfs_fruit.

    (emphasis mine).

     

    They say 'streams_xattr' must be loaded, but later in the doc they talk about options that are incompatible with this.  In addition certain file system types are inherently incompatible, notably FAT and exFAT because they don't support xattr at all...  Hence we can't (or shouldn't) include the 'flash' share in 'fruit' but then if the 'flash' share is first share to be accessed on a Mac client, APPL extensions will be disabled for the duration... Yes this is confusing, maybe best bet is to not export the 'flash' share in MacOS environment.  If it is exported, the 'testparm' command outputs this warning:

    WARNING: some services use vfs_fruit, others don't. Mounting them in conjunction on OS X clients results in undefined behaviour.

     

    Time machine - if enabled for a share these parameters are automatically added (independent of smb-fruit.conf):

    fruit:time machine = yes
    fruit:time machine max size = SIZE # if "vol size limit" specified

     

    Finally Spotlight - our build of Samba includes spotlight support but only for 'elasticsearch'.  But 'elasticsearch' is not installed in Unraid OS.  In other places on the forum people have expressed success by setting the spotlight backend to 'tracker' - but this requires "gnone tracker" which also is not installed in Unraid OS - so I'm not sure what's going on here.  Nevertheless, you can add 'spotlight = on' to the smb-fruit.conf file to play around with this.

     

    Please limit discussion in this topic to MacOS issues only.

     

    • Like 10
    • Thanks 1
    • Upvote 1



    User Feedback

    Recommended Comments



    I switched my TimeMachine to an old MacMini with an external drive.  After 6.11.1 upgrade my UnRaid TimeMachine setup went belly up   I am done trying 

    Link to comment
    1 hour ago, johnwhicker said:

    I switched my TimeMachine to an old MacMini with an external drive.  After 6.11.1 upgrade my UnRaid TimeMachine setup went belly up   I am done trying 

    That’s not the latest stable though. 6.11.5 is, and time machine has been great with the configs listed in the thread for what it’s worth 

    Link to comment
    2 hours ago, Jclendineng said:

    time machine has been great with the configs listed in the thread for what it’s worth 

    I stand by my decision to stick with 6.9.2 until the DEFAULT settings for Unraid's SMB configs work correctly for the BUILT-IN functionality of Time Machine. 6.9.2 did that just fine, and there's no valid excuse in my mind for 6.11 and onwards to not include it. Especially given people have been able to hack their way to something functional. 

    I still challenge Unraid to raise the bar and get Time Machine backups working by default on a fresh install of 6.11/6.12 with zero additional manual config necessary. AKA:

    - Install Unraid to a flash drive
    - Boot, create a one disk no-parity array
    - Create a Time Machine compatible share after enabling macOS optimization in the SMB settings
    - Perform a successful backup
    - Perform successful incremental backups (which is the part that I've not yet seen working correctly)

    Bonus points if Time Machine shares created on 6.9 and earlier can continue to work with 6.11 and onwards. 

    • Upvote 2
    Link to comment
    On 3/16/2023 at 7:23 PM, kubed_zero said:

    I stand by my decision to stick with 6.9.2 until the DEFAULT settings for Unraid's SMB configs work correctly for the BUILT-IN functionality of Time Machine. 6.9.2 did that just fine, and there's no valid excuse in my mind for 6.11 and onwards to not include it. Especially given people have been able to hack their way to something functional. 

    I still challenge Unraid to raise the bar and get Time Machine backups working by default on a fresh install of 6.11/6.12 with zero additional manual config necessary. AKA:

    - Install Unraid to a flash drive
    - Boot, create a one disk no-parity array
    - Create a Time Machine compatible share after enabling macOS optimization in the SMB settings
    - Perform a successful backup
    - Perform successful incremental backups (which is the part that I've not yet seen working correctly)

    Bonus points if Time Machine shares created on 6.9 and earlier can continue to work with 6.11 and onwards. 

    I can do nothing but 100% agree with this. In this world of ubiquitous free/open source software, it's easy to forget that we are actually paying customers for this product. As paying customers, it only makes sense to make certain demands and set some expectations on what we're purchasing.

     

    Make no mistake, I like UnRAID a lot, but this whole thing with letting the customers figure out solutions to bugs in the product they paid for... It's just not right. A NAS software product which doesn't support Time Machine backups out of the box? Again, it's just not right.

    • Like 3
    Link to comment
    On 3/16/2023 at 1:23 PM, kubed_zero said:

    I stand by my decision to stick with 6.9.2 until the DEFAULT settings for Unraid's SMB configs work correctly for the BUILT-IN functionality of Time Machine. 6.9.2 did that just fine, and there's no valid excuse in my mind for 6.11 and onwards to not include it. Especially given people have been able to hack their way to something functional. 

    I still challenge Unraid to raise the bar and get Time Machine backups working by default on a fresh install of 6.11/6.12 with zero additional manual config necessary. AKA:

    - Install Unraid to a flash drive
    - Boot, create a one disk no-parity array
    - Create a Time Machine compatible share after enabling macOS optimization in the SMB settings
    - Perform a successful backup
    - Perform successful incremental backups (which is the part that I've not yet seen working correctly)

    Bonus points if Time Machine shares created on 6.9 and earlier can continue to work with 6.11 and onwards. 

    I also completely agree with this.  I have been fumbling around until I found this post, I went to 6.10.3, had to go back to 6.9.2, but I do want some of the newer unraid features, so I tired yesterday to go to 6.11.5, and again my time machines do not work, when they had been working for a decade since Unraid's beginnings.  I also would prefer to not have to hack my way to get them to work, since it was built in for so so long, and I had a perfect Time Capsule at home for my 4+ macs to backup via timemachine to flawlessly. 

    Link to comment
    On 10/17/2022 at 5:10 PM, limetech said:

    Thank you for your report.  I have TM backups running ok with Monterey (12.6).  At first I thought it was because the share is marked 'public'.  Changed to 'private' and there was some connectivity issues but eventually poked around the TM preferences and got it to work again... so the investigation continues...

    Not sure if it would help diagnose, but I think many of us that have issues have had Unraid for a long time, can you try upgrading from 6.9.2 to a newer version and see if you get the same problems? 

     

    For some reason I have seen a few people in the forums that say they created this time machine on 6.10.3 or above and have something working, so perhaps attempting on an older version then upgrading to see if you can replicate the issues? 

    Link to comment

    Hi, I hope it‘s okay to join your discussion having issues with TM backups too.

    9 minutes ago, manolodf said:

    Not sure if it would help diagnose, but I think many of us that have issues have had Unraid for a long time

    In my case I only started using Unraid with 6.10, and it never worked reliable. (I have this strange behaviour that TM backups are only working when the CPU of the server is under load.)

     

    12 minutes ago, manolodf said:

    For some reason I have seen a few people in the forums that say they created this time machine on 6.10.3 or above and have something working, so perhaps attempting on an older version then upgrading to see if you can replicate the issues? 

    I‘ve tried that. (Actually made a clean Unraid 6.10.3 installation in order to try that.) The initial backup was (kind of) working. It aborted at ~50%, but after changing some SMB configuration in Unraid it was resumed. Incremental backups though was never working reliable, no matter of the configuration I tried.

    Link to comment
    1 hour ago, Simon Edelmann said:

    Hi, I hope it‘s okay to join your discussion having issues with TM backups too.

    In my case I only started using Unraid with 6.10, and it never worked reliable. (I have this strange behaviour that TM backups are only working when the CPU of the server is under load.)

     

    I‘ve tried that. (Actually made a clean Unraid 6.10.3 installation in order to try that.) The initial backup was (kind of) working. It aborted at ~50%, but after changing some SMB configuration in Unraid it was resumed. Incremental backups though was never working reliable, no matter of the configuration I tried.

    I see, ok so maybe those are few and far between, but I promise you prior to 6.10 TM backups were amazing, I had hourly backups for I believe at least 10yrs, since around 2012 when I got my first mac.  All of a sudden this broke in 6.10 and is still not resolved (I tried 6.11.5, but I am about to revert back to 6.9.2 once again) 

     

    Update:  Downgraded to 6.9.2 and Time Machine Backups working great again instantly upon downgrade, I do not foresee incrementals having an issue either but will update if so.   

    Edited by manolodf
    Updated after downgrade
    Link to comment

    I gave up on unraid time machine (I now use a time capsule) so my comment's not based on experience with the recent versions but it seems to me the later samba versions made MacOS less compatible generally – search doesn't work out of the box, folders don't always refresh when their content changes (which sometimes causes fuse errors that take down the array.)

     

    I'd be tempted to blame MacOS but I have an old Raspberry Pi running samba 4.9.5 that works just fine.

    Link to comment

    Does anybody know if it is possible to install a newer/older version of samba in Unraid?

    I tried to get a precompiled binary for Slackware and used ‚upgradepkg‘ to install it, but when I was running smb commands then I always got „Required file not found“, although the file was there and I could inspect its content via ‚cat‘.

    I would like to try different versions of samba and check if any of those works..

    Link to comment
    On 12/13/2022 at 8:59 PM, b0m541 said:

    I had to actually use even more options to get it to work with folders containing hundreds of files:

     

    vfs objects = catia fruit streams_xattr
    fruit:metadata = stream
    fruit:encoding = native
    readdir_attr:aapl_rsize = no
    readdir_attr:aapl_finder_info = no
    readdir_attr:aapl_max_access = no
    

     

    I didn't need POSIX rename so far, but I did not rename any files yet.

    Also I do not know how well this works with TimeMachine, I may test this at a later point in time.

     

     

    Thank you!

     

    I've been having a *nightmare* with a new machine (Mini M2 Pro, Ventura) and these settings have given me the 'best' results so far of 24s~ to open a directory of 1,100 files/folders down from 1 minute+ (stock "Enhanced macOS interoperability"). I know that's still very slow but I only have a few directories like that so it's bearable. Another machine (MBP 2016, Mojave) manages the same in 12s~ and that's over wifi rather than ethernet.

    For reference not having anything in /etc/nsmb.conf (including the classic signing_required=no) made both a few seconds faster.

     

    "ls" in Terminal gives a result in <2s (4s~ for "ls- la") which makes Finder's sloth so frustrating - fyi unchecking "Show icon preview" as a default didn't help. This really does need more investigation.

     

    NFS was quicker (can't be bothered to go back and time it), but for every item Finder displayed "Date created." of "1 Jan 2001" and I'd like both created and modified times ideally. Anyone know a way to do this? I found this article but I don't know about Unraid/macOS supporting it.

     

     

    Edited by wibble
    Link to comment

    I wanted to share that I too have mostly figured out the Samba macOS nuance. I'm on Unraid 6.12.3 with macOS Ventura 13.5.1

     

    To start, setting fruit:metadata = stream in the SMB Extras in the Unraid UI was the single biggest contributor to getting things working. Here's exactly what I have, in its entirety:

    [Global]
    fruit:metadata = stream

     

    Note that I don't use Unassigned Devices, which I think would add to these lines.

     

    After adding this and stopping/starting the array, pre-existing Time Machine backups were NOT working reliably, so I also had to create new Time Machine backups from scratch. I kept the old sparsebundles around just in case.

     

    Once new initial backups were made successfully, one of my MacBooks was able to reliably back up on a daily cadence. It's been running this way for a couple months. Meanwhile, one of my other MacBooks refused to work well with Time Machine, making one successful backup every few weeks, contingent on a recent Unraid reboot. I couldn't deal with this, so I factory reset it (reinstalling macOS) and created an additional new Time Machine backup on Unraid. Then it worked flawlessly.

     

    Then one of my MacBooks died, so I needed to restore data from Time Machine. I first tried to connect to Unraid and mount the sparsebundle through Finder, but it would time out, beachball, and overall never end up working. I was however able to get it mounted and accessible through the Terminal/CLI using the command `hdiutil attach /Volumes/path/to/sparsebundle` and with that, access the list of Time Machine snapshots and the files I wanted to recover.

     

    Then, I tried to use Apple's Migration Assistant to attempt to fully restore from a Time Machine backup. I was able to connect to the Unraid share and it was able to list the sparsebundles, but it would get stuck with "Loading Backup..." indefinitely. I moved some of the other computers' sparsebundles out of the share so it could focus on just the one sparsebundle I wanted, but even after waiting 24 hours, it would still say that it was loading backups. Looking on the Open Files plugin's tab in Unraid, I would see it reading one band file at a time.

     

    After enough of this, I tried to access a different sparsebundle that only had two backups, instead of months of backups, and "Loading Backups..." went away within 10 minutes and I was able to proceed with the Time Machine restoration, albeit slowly, and not with the data I wanted.

     

    This did clue me in to something, though. Using `find /path/to/sparsebundle/bands/ -type f | wc -l` to get the file count inside the sparsebundle, the one that made it through Migration Assistant was only 111 files, and the one that stalled for 24h was over 9000 files.

    I then went back to the Unraid SMB settings and tried to fiddle around a bit more. I found, as others did, that changing the following settings in smb-fruit.conf caused big improvements. The defaults for these settings are `yes` so I changed them to `no`: 

    readdir_attr:aapl_rsize = no
    readdir_attr:aapl_finder_info = no
    readdir_attr:aapl_max_access = no

     

    As the Samba Fruit man page notes in https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html, `readdir_attr:aapl_max_access = no` is probably the most significant of these, as the setting is described: "Return the user's effective maximum permissions in SMB2 FIND responses. This is an expensive computation. Enabled by default"

     

    My suspicion is that the thousands of files that make up a sparsebundle end up getting bottlenecked when read through Samba, causing Migration Assistant to fail. 

     

    After adding these lines to `/etc/samba/smb-fruit.conf` and copying that updated file over to `/boot/config/smb-fruit.conf` and stopping and starting the array, I confirmed the settings were applied with `testparm -s` and looking at the output:

    [global]
            ~~~shortened~~~
            fruit:metadata = stream
            fruit:nfs_aces = No
            ~~~shortened~~~
    
    [TimeMachine]
            path = /mnt/user/TimeMachine
            valid users = backup
            vfs objects = catia fruit streams_xattr
            write list = backup
            fruit:time machine max size = 1250000M
            fruit:time machine = yes
            readdir_attr:aapl_max_access = no
            readdir_attr:aapl_finder_info = no
            readdir_attr:aapl_rsize = no
            fruit:encoding = native

     

    Now that the new settings were in place, Migration Assistant got through the "Loading Backups" stage within a minute or two, and I was able to successfully fully restore the old backup sparseimage with thousands of files. 

     

    I know there's some nuance around setting Apple/fruit settings depending on the first device to connect to Samba, so this entire experiment took place with only Macs connecting to Unraid. I did not yet repeat the experiment with Windows connecting first or in parallel, but I hope the behavior is the same as I cannot guarantee Macs will always connect before Windows computers in my network. 

     

    Anyway, I wanted to share as I avoided updating Unraid 6.9.2 for literal years to keep a working Time Machine backup. I then jumped for joy at the MacOS improvements forum post a year ago just to find it didn't help in any way, and was again excited to update to 6.12, just to find it STILL didn't work reliably with default settings. Very disappointing, LimeTech. 

    And a huge thanks to the folks in these threads that have shared their updates and what has or has not worked for them. Let's keep that tradition going, as it's clear we are on our own here. 


    Some Time Machine related posts from over the years. I'll make update posts in each directing here. 

     


    TLDR: Working Time Machine integration. Adding fruit:metadata = stream to the global settings and then readdir_attr:aapl_max_access = no and readdir_attr:aapl_finder_info = no and readdir_attr:aapl_rsize = no into the smb-fruit settings allowed me to run Time Machine backups AND restore from or mount them using Finder and Migration Assistant. 


     

    Edited by kubed_zero
    Added the other thread from below
    • Like 1
    • Thanks 1
    • Upvote 1
    Link to comment

     

    ^ Good information about disk shares being far faster. I tried this myself and a disk share directory of 1,500~ directories took a second or two to display rather than 30s+ for a user share.

     

    So is FUSE the problem and is this fixable?

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.