Copying files from a Mac to unRAID - speed vs. integrity dilemma

[Glassed Silver]

Hey guys!

 

So I'm in the process of moving massive amounts of data from my Mac (High Sierra, because iMac '11 is no longer supported for major version upgrades to macOS) to my unRAID server trying to empty one drive on my Mac, pre-clear it on unRAID and then extend the array to make room for the next batch... So far so good. Here's my MAJOR issue however:

 

SMB protocol - speedy, mostly error-free copying of files with the samples I tried so far. Nice! But... it won't keep metadata I rely on like creation date of file or folder, file description (in the file Get Info dialogue on Mac there's this multi-line field to enter info about a file or folder), tags/color tags, ... basically macOS-specific file metadata. Alright, so let's use...

 

AFP then, right? Apple's own old, but meta data-aware protocol! Yeah... well... PERFORMANCE. It's practically not existing. It's not a drive issue on either end, again, the same transfer over SMB will work quick and easy, there's just this nasty loss of metadata (that I need).

 

Little example: create new empty folder, delete said folder... Takes me ~40secs. I shit you not.

 

Watch:

 

I need some help from fellow Mac users.

Edited May 11, 2019 by Glassed Silver

[CHBMB]

Not a Mac user, and I don't know if this would work.  Any possibility of pulling the drive out the Mac and mounting it locally in Unraid using Unassigned Devices.  (I don't know if it can mount Mac filesystems but thought it might be worth exploring)

[Glassed Silver]

13 minutes ago, CHBMB said:

Not a Mac user, and I don't know if this would work.  Any possibility of pulling the drive out the Mac and mounting it locally in Unraid using Unassigned Devices.  (I don't know if it can mount Mac filesystems but thought it might be worth exploring)

I know it can mount HFS+, not sure about APFS formatted drives.

 

This is what I’m trying out next though would you believe it. Already formatted an SD card as APFS to see if unRAID is able to digest it. If not I’ll make do with HFS+ and see if my test files appear in unRAID with all meta data intact.

 

If that’s a go, I’ll do just that.

 

TIL: Apple oversimplified yet another thing. At least Windows will return warnings when moving files from one place to another that will not store extended attributes.

 

Also TIL: Apple hates industry standards and making them work even for something so crucial as networked storage.

 

And it’s NOT like they still offer products anymore like the XServe where you could say “well it’s DESIGNED for you to use Apple stuff”.

 

There is no Apple stuff in network storage and if I see another user somewhere in a thread similar to this suggesting buying a Mac Mini as server I’ll die of a laughing attack.

[Glassed Silver]

Update: Enhanced OS X interoperability solved my issues, I get tags, get info file descriptions, etc... Creation dates still get screwed, but I also discovered that if I put a file over AFP into a share and access it over SMB either from a Mac or from Windows that date is getting screwed either way... so... since I want to rely less on my Mac anyways and use Windows primarily in the future I guess there simply is no way to keep creation dates intact whatsoever no matter the route you take.

 

 

Guess I'll have to pull this tooth already and be done with it. Sucks, but so be it. At least SMB transferred files retain their tags in macOS when touched by Windows. (BIG ASTERISK: e.g. an rtf doc opened and saved again in Word that you created in TextEdit on Mac is saved as a new file so to say (this will get rid of any extended proprietary macOS metadata)... Meanwhile a .txt stays unchanged as far as that metadata is concerned. At least if I remember correctly. I did some random testing like that same night I last posted, bottom line is: SMB, Enhanced interoperability activated and then completely forgetting AFP exists is the way to go if you want to work with the files on both macOS and Windows. Otherwise only use AFP at all times, iirc that got a little speedier too after upgrading to unRAID 6.7)

Edited May 14, 2019 by Glassed Silver
Edited: Conclusions better worded. Fixed second sentence talking about modification instead of creation dates. Oops.

[Maxrad]

My testing was less comprehensive (I hadn't noticed modification dates) but came to same conclusion. I'm not longer using AFP. I'm happy with unRAID SMB setting "Enhanced OS X interoperability" set to "Yes" and macOS Finder connections to unRAID using SMB (Connect to Server smb://[email protected]/Media) 🙂

Edited May 14, 2019 by Maxrad

[Glassed Silver]

1 hour ago, Maxrad said:

My testing was less comprehensive (I hadn't noticed modification dates) but came to same conclusion. I'm not longer using AFP. I'm happy with unRAID SMB setting "Enhanced OS X interoperability" set to "Yes" and macOS Finder connections to unRAID using SMB (Connect to Server smb://[email protected]/Media) 🙂

I typo'd. What I meant to write in my second sentence was that creation dates get screwed (basically they get equalized with the modification dates (and times obviously))

 

Other than that, the lesson remains the same. Don't touch AFP and you get a fairly okay experience cross-plat. I guess it pays off that in some files eventually I started using the initial date as part of the file name, still leaves some gaps, but alright. Going forward with the new files coming in from Windows or Linux I guess this shouldn't be an issue anymore.

 

Don't forget the setting to hide . files and folders. Works wonderfully! Windows doesn't display them, but you can still view them if needed with "Show hidden files and folders" in the Windows Explorer settings. And a Mac can still access them and store its auxiliary data there just fine.

 

If you ask me, this should be the default setup for anyone doing cross-plat. So far very pleased!