kubed_zero

Community Developer
  • Posts

    166
  • Joined

  • Last visited

Report Comments posted by kubed_zero

  1. 6 hours ago, JorgeB said:

    Does the same thing happen if you run Unraid baremetal?

    I can't say for certain, this is not a scenario I can test another VM on the machine is providing me network access to Unraid. 

    I want to emphasize though that in both of these cases, Unraid ran fine as a VM for 3-5 years, and it's only in the past few months that I've been seeing this. 

  2. I wanted to share that I too have mostly figured out the Samba macOS nuance. I'm on Unraid 6.12.3 with macOS Ventura 13.5.1

     

    To start, setting fruit:metadata = stream in the SMB Extras in the Unraid UI was the single biggest contributor to getting things working. Here's exactly what I have, in its entirety:

    [Global]
    fruit:metadata = stream

     

    Note that I don't use Unassigned Devices, which I think would add to these lines.

     

    After adding this and stopping/starting the array, pre-existing Time Machine backups were NOT working reliably, so I also had to create new Time Machine backups from scratch. I kept the old sparsebundles around just in case.

     

    Once new initial backups were made successfully, one of my MacBooks was able to reliably back up on a daily cadence. It's been running this way for a couple months. Meanwhile, one of my other MacBooks refused to work well with Time Machine, making one successful backup every few weeks, contingent on a recent Unraid reboot. I couldn't deal with this, so I factory reset it (reinstalling macOS) and created an additional new Time Machine backup on Unraid. Then it worked flawlessly.

     

    Then one of my MacBooks died, so I needed to restore data from Time Machine. I first tried to connect to Unraid and mount the sparsebundle through Finder, but it would time out, beachball, and overall never end up working. I was however able to get it mounted and accessible through the Terminal/CLI using the command `hdiutil attach /Volumes/path/to/sparsebundle` and with that, access the list of Time Machine snapshots and the files I wanted to recover.

     

    Then, I tried to use Apple's Migration Assistant to attempt to fully restore from a Time Machine backup. I was able to connect to the Unraid share and it was able to list the sparsebundles, but it would get stuck with "Loading Backup..." indefinitely. I moved some of the other computers' sparsebundles out of the share so it could focus on just the one sparsebundle I wanted, but even after waiting 24 hours, it would still say that it was loading backups. Looking on the Open Files plugin's tab in Unraid, I would see it reading one band file at a time.

     

    After enough of this, I tried to access a different sparsebundle that only had two backups, instead of months of backups, and "Loading Backups..." went away within 10 minutes and I was able to proceed with the Time Machine restoration, albeit slowly, and not with the data I wanted.

     

    This did clue me in to something, though. Using `find /path/to/sparsebundle/bands/ -type f | wc -l` to get the file count inside the sparsebundle, the one that made it through Migration Assistant was only 111 files, and the one that stalled for 24h was over 9000 files.

    I then went back to the Unraid SMB settings and tried to fiddle around a bit more. I found, as others did, that changing the following settings in smb-fruit.conf caused big improvements. The defaults for these settings are `yes` so I changed them to `no`: 

    readdir_attr:aapl_rsize = no
    readdir_attr:aapl_finder_info = no
    readdir_attr:aapl_max_access = no

     

    As the Samba Fruit man page notes in https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html, `readdir_attr:aapl_max_access = no` is probably the most significant of these, as the setting is described: "Return the user's effective maximum permissions in SMB2 FIND responses. This is an expensive computation. Enabled by default"

     

    My suspicion is that the thousands of files that make up a sparsebundle end up getting bottlenecked when read through Samba, causing Migration Assistant to fail. 

     

    After adding these lines to `/etc/samba/smb-fruit.conf` and copying that updated file over to `/boot/config/smb-fruit.conf` and stopping and starting the array, I confirmed the settings were applied with `testparm -s` and looking at the output:

    [global]
            ~~~shortened~~~
            fruit:metadata = stream
            fruit:nfs_aces = No
            ~~~shortened~~~
    
    [TimeMachine]
            path = /mnt/user/TimeMachine
            valid users = backup
            vfs objects = catia fruit streams_xattr
            write list = backup
            fruit:time machine max size = 1250000M
            fruit:time machine = yes
            readdir_attr:aapl_max_access = no
            readdir_attr:aapl_finder_info = no
            readdir_attr:aapl_rsize = no
            fruit:encoding = native

     

    Now that the new settings were in place, Migration Assistant got through the "Loading Backups" stage within a minute or two, and I was able to successfully fully restore the old backup sparseimage with thousands of files. 

     

    I know there's some nuance around setting Apple/fruit settings depending on the first device to connect to Samba, so this entire experiment took place with only Macs connecting to Unraid. I did not yet repeat the experiment with Windows connecting first or in parallel, but I hope the behavior is the same as I cannot guarantee Macs will always connect before Windows computers in my network. 

     

    Anyway, I wanted to share as I avoided updating Unraid 6.9.2 for literal years to keep a working Time Machine backup. I then jumped for joy at the MacOS improvements forum post a year ago just to find it didn't help in any way, and was again excited to update to 6.12, just to find it STILL didn't work reliably with default settings. Very disappointing, LimeTech. 

    And a huge thanks to the folks in these threads that have shared their updates and what has or has not worked for them. Let's keep that tradition going, as it's clear we are on our own here. 


    Some Time Machine related posts from over the years. I'll make update posts in each directing here. 

     


    TLDR: Working Time Machine integration. Adding fruit:metadata = stream to the global settings and then readdir_attr:aapl_max_access = no and readdir_attr:aapl_finder_info = no and readdir_attr:aapl_rsize = no into the smb-fruit settings allowed me to run Time Machine backups AND restore from or mount them using Finder and Migration Assistant. 


     

    • Like 1
    • Thanks 1
    • Upvote 1
  3. I wanted to update to say I've gotten things semi-reliably working. I'm using macOS Ventura 13.4.1 and macOS Sonoma 14 Public Beta 1, connecting to Unraid 6.12.3. On Unraid, the only modification I made was to add the typical recommendation to the SMB Extra settings in the Unraid Settings>SMB>Samba extra configuration:

    [Global]
    fruit:metadata = stream


    I created a new Share and started new Time Machine backups, which succeeded. Incremental backups work more reliably from some Macs than others. Rebooting the Mac won't help get it working again. Part of what I think has helped is to make sure the Mac is the first thing to connect to Unraid after a reboot. AKA if I get the typical failure to connect to Time Machine to back up, I

     

    • Reboot Unraid
    • Run the TM backup or just connect to Unraid from the Mac, before any Windows computer gets a chance
    • Run the backup, hopefully succeeding

     

    It seems to me that the key is making sure the Mac is the first device to connect to Samba, and ensuring the fruit:metadata = stream customization is present. I haven't found any of the other smb-fruit config options to affect the ability to back up. 

  4. Still no luck. I tried the following on 6.12-rc3 with macOS 13.3.1. I added these settings to /boot/config/smb-fruit.conf and did an array stop and start each time. I checked /etc/samba/smb-shares.conf to confirm the settings were applied each time. 

    No luck, from 

    vfs objects = acl_xattr fruit streams_xattr
    fruit:encoding = native
    fruit:metadata = stream
    fruit:posix_rename = yes

     

    No luck, from 

     

     

    vfs objects = catia fruit streams_xattr
    fruit:encoding = native
    fruit:metadata = stream
    fruit:posix_rename = yes



    No luck, from https://wiki.samba.org/index.php/Configure_Samba_to_Work_Better_with_Mac_OS_X

     

    vfs objects = fruit streams_xattr  
    fruit:metadata = stream
    fruit:model = MacSamba
    fruit:posix_rename = yes 
    fruit:veto_appledouble = no
    fruit:nfs_aces = no
    fruit:wipe_intentionally_left_blank_rfork = yes 
    fruit:delete_empty_adfiles = yes 


    In all cases, I can't mount the sparsebundle, the Time Machine backup does not complete, etc. 

    Back to 6.9.2 🤦‍♀️

    Edit: One more note, since I only saw one comment on this that I now can't find. https://www.samba.org/samba/security/CVE-2021-44142.html versions of Samba prior to 4.13.17 had a vulnerability if the use of vfs_fruit and fruit:metadata=netatalk or fruit:resource=file. Considering 6.9.2 used Samba 4.12.14, it was affected by this, and it's possible the fix has affected functionality. 

  5. 2 hours ago, Jclendineng said:

    time machine has been great with the configs listed in the thread for what it’s worth 

    I stand by my decision to stick with 6.9.2 until the DEFAULT settings for Unraid's SMB configs work correctly for the BUILT-IN functionality of Time Machine. 6.9.2 did that just fine, and there's no valid excuse in my mind for 6.11 and onwards to not include it. Especially given people have been able to hack their way to something functional. 

    I still challenge Unraid to raise the bar and get Time Machine backups working by default on a fresh install of 6.11/6.12 with zero additional manual config necessary. AKA:

    - Install Unraid to a flash drive
    - Boot, create a one disk no-parity array
    - Create a Time Machine compatible share after enabling macOS optimization in the SMB settings
    - Perform a successful backup
    - Perform successful incremental backups (which is the part that I've not yet seen working correctly)

    Bonus points if Time Machine shares created on 6.9 and earlier can continue to work with 6.11 and onwards. 

    • Upvote 2
  6. 20 minutes ago, limetech said:

    poked around the TM preferences and got it to work again

    Got it. Well, if there's anything I can help with (different settings, additional logs on the macOS or Unraid side, reverting to 6.9.2 and diffing outputs, etc) let me know!

    • Thanks 1
  7. Still no luck on getting Time Machine to work with Unraid 6.11.1. Nothing shows up in the syslog to indicate a crash of SMB. The typical "Operation not supported by device" UserInfo={DIErrorVerboseInfo=Failed to initialize IO manager: Failed opening folder for entries reading}" still shows up on the MacOS (12.6) side.

     

    I've got zero extra Samba custom configurations/files, so this is running with vanilla Unraid as far as I'm aware. 

     

    testparm -sv shows the following: https://pastebin.com/mmYPTz9n

     

    Specifically, here is the share for TimeMachine

     

    [TimeMachine]
    	path = /mnt/user/TimeMachine
    	valid users = backup
    	vfs objects = catia fruit streams_xattr
    	write list = backup
    	fruit:time machine max size = 1200000M
    	fruit:time machine = yes
    	fruit:encoding = native

     

    @limetech As requested, I've output diagnostics from Unraid as well immediately after a reboot and failing to back up from two different computers.

    pbox-diagnostics-20221017-1212.zip

    • Thanks 1
  8. 1 minute ago, T4ke said:

    incompatibilities between Apples latest changes in their SMB stack and the different Samba versions

    Yeah, this could definitely be the case. I agree completely that Apple could be going off the rails in how things are built. But considering upgrading the server causes the breakage while the Apple device stays the same, and the Time Machine functionality in Unraid is built exclusively for Apple devices, I would have hoped that Limetech could have pinned the Samba version on something functional (or at least tested it in 6.10 to a thorough degree). 

     

    It's not a good look for Limetech if upgrades to 6.10.x result in failures of core functionality such as the built-in Time Machine backups, and then a reversion to the previous stable release can fix any issues seen. That's just my two cents, though I think I'm not alone in that sentiment.

    At some point in the next few weeks I'm going to try experimenting with installing 6.10.3 and then modifying my Go script to:

    • Stop the Samba process, I think with "/etc/rc.d/rc.samba stop"
    • Install the Samba version from 6.9.2 and hopefully overwrite the default version. I found 6.9.2 used version 4.12.14, as opposed to 4.15.7 in 6.10.3
    • Start the Samba process, I think with "/etc/rc.d/rc.samba start"
    • See if initial and then incremental backups still work

    I'd also want to try rolling forward and seeing if Samba 4.16.x packages would work, which should be straightforward to test once this Go script stuff is in place. 

    If anyone else gets a chance to do this, please share your results! It will be curious to see if the Samba package upgrade broke the functionality between 6.9 and 6.10, or if it was kernel-related modification, or something else. 

  9. Just now, T4ke said:

    It's pretty obvious that Apple made changes in the SMB implementation of macOS that seem to cause these issues.
    Its not Unraids nor Sambas fault

    I'm not sure I follow. If that were the case, why (in my setup) does upgrading to 6.10.x break backups, but reverting to 6.9.2 100% of the time fix backups? 

     

    1 minute ago, T4ke said:

    This is only a temporary solution since AFP is insecure and was depreciated by Apple

    Yep, I wanted to avoid going back to using an AFP share because of Apples deprecation of it. 

  10. 1 hour ago, itimpi said:

    The problem I see is that with combinations of Linux kernel and Samba updates in new releases there may well be changes affecting this that are outside Limtech’s control and difficult to identify and correct and thus re-occurring stability problems in the capability.   The advantage of using a docker container is that it can provide a stable environment independently of changes happening at the host level.

    Eh, my opinion is that if Limetech includes the feature in the base OS, it should work properly and not break on a version upgrade. Perhaps this is a result of updating the Samba package, and not a kernel update issue. We'll have to do more testing to evaluate what broke it. Anyway, I don't want to run Docker on my NAS (would prefer a separate system considering Unraid is already a VM on my ESXi cluster) so I'll just stick on 6.9.2 until the built-in feature works again 

    • Upvote 1
  11. 27 minutes ago, kubed_zero said:
    1 hour ago, saber1 said:

    Directly after downgrade to 6.9.2 the job starts without issues.

    I'm going to give this a shot, since there's nothing in 6.10.x that is beneficial to me. Will report back. 

     

    Yep this worked 100%. All Macs, M1/Intel, are backing up successfully to preexisting and new backups on a single Time Machine Unraid share now that I've downgraded to 6.9.2 from 6.10.3. Unfortunate that this is necessary, hopefully @limetech will reopen these bugs and investigate further. 

     

    So just to summarize:

    • Unraid 6.10.3 uses smbd version 4.15.7, while Unraid 6.9.2 uses smbd version 4.12.14 (checked with "smbd --version")
    • All other SMB-related configs seem identical, such as /etc/samba/smb-shares.conf, /etc/samba/smb-names.conf, /etc/samba/smb.conf, and *a lack of* /boot/config/smb-extra.conf (meaning no SMB Extras defined)
    • Time Machine backups proceed normally on Unraid 6.9.2 from both Intel and M1 Macs running MacOS 12.4, while the same incremental backups fail on Unraid 6.10.3 with a Console log Mac-side along the lines of "Operation not supported by device" UserInfo={DIErrorVerboseInfo=Failed to initialize IO manager: Failed opening folder for entries reading}
    • The below SMB Extras config added to 6.10.3 allowed some but not all Macs to back up. Removing it caused then-working Macs to start failing again.
    [Global]
       vfs objects = catia fruit streams_xattr
       fruit:nfs_aces = no
       fruit:zero_file_id = yes
       fruit:metadata = stream
       fruit:encoding = native
       spotlight backend = tracker
    [TimeMachine]
            path = /mnt/user/TimeMachine
            comment =
            browseable = yes
            # Private
            writeable = no
            read list =
            write list = backup
            valid users =  backup
            vfs objects = catia fruit streams_xattr
            fruit:time machine = yes
            fruit:time machine max size = 1200000M
            case sensitive = auto
            preserve case = yes
            short preserve case = yes
    • I also keep my Time Machine share constrained to a single disk, as I've had cross-disk issues in the distant past
    • Like 2
  12. 22 minutes ago, wgstarks said:

    That appears to be the same errors I had. Fixed it by reinstalling macOS from the restore system. Easy process that preserves all your data and prefs. Similar to an OS update.

     

    Another option might be to install the TimeMachine docker on unRAID.

     

    Eh... neither of those are acceptable solutions in my mind. The out of the box Time Machine functionality should work without a need for a third party solution, and reinstalling MacOS after just having done so in the past month also seems less-than-ideal.

     

    52 minutes ago, saber1 said:

    Directly after downgrade to 6.9.2 the job starts without issues.

    I'm going to give this a shot, since there's nothing in 6.10.x that is beneficial to me. Will report back. 

  13. Just wanted to say I had this issue on both Intel and M1 Macs. Time Machine backups to Unraid were going swimmingly, with the Unraid share hosting an APFS sparseimage. I'm not sure if it was the Unraid 6.10 update or MacOS 12.3/12.4 or just bad timing, but then all the computers stopped being able to back up a month or so ago. I was able to make first (non-incremental) backups without issue, but subsequent incremental backups after that did not work. I had the same MacOS console issues as mentioned above: "Operation not supported by device" UserInfo={DIErrorVerboseInfo=Failed to initialize IO manager: Failed opening folder for entries reading}

     

    I had tried deleting the local snapshots on each Mac as well, to no avail. For reference, that's "tmutil listlocalsnapshots" to find the snapshot names and then "tmutil deletelocalsnapshots <snapshot-name>" to actually delete them. 

    Anyway, I applied the SMB Extra Config settings as mentioned above, and then re-attempted the incremental backups. Success! (Update: Time Machine now works on two of the three Macs. The last one still gets the "Failed opening folder for entries reading" error) The following SMB Extras config seems to have done the trick:

     

    [Global]
       vfs objects = catia fruit streams_xattr
       fruit:nfs_aces = no
       fruit:zero_file_id = yes
       fruit:metadata = stream
       fruit:encoding = native
       spotlight backend = tracker

     

    I did browse through the documentation pages to understand what these were doing  https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html  https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html and the only callout I have is that the zero_file_id setting is yes by default, and may not be necessary. The rest of the options check out, though I'm not sure why this combination of options makes Time Machine work. I saw another post say that just setting two of these worked: 

     

    As a sidenote, I DID see that https://wiki.unraid.net/UnRAID_6/Configuring_Apple_Time_Machine recommends having a separate share per computer, but that seems to be outdated (bad?) advice and perhaps should be changed. I've got three Macs backing up to the same Time Machine share with no issues.



  14. I experienced the same issue in the SNMP plugin. An old version of the plugin did not check standby state before getting temperature, leading to drives always being spun up if they didn't report temperature during standby. By adding "--nocheck standby" to  the flags  passed to smartctl, drives will only report temperature while spun up. I'm guessing something similar is happening here. By default, the Web UI only checks the disk temperature every 30 minutes.

  15. If the primary means of configuring Unraid (the web interface) can easily create corner cases that the boot media can't support, should the UI consider that limitation and make adjustments as necessary? As you noted, substitution could just open up another can of works with regard to how this is supported. Other approaches are to leave it as it is with known exposed corner cases that break the UI/functionality, or further limit the UI so those corner cases can't be reached. 

     

    There are  probably more important things to fix, all things considered, but at absolute least I'd hope there would be a warning in the UI saying that share naming containing certain characters could cause complicated conditions (😊) to arise. 

    • Like 1