Hi there,

 

I have a, from my perspective, really bizarre problem.

I run a project backup to the unRAID Server daily; there's a folder structure on my work-machine (macOS Monterey 12.6.7), and the same on the unRAID server. I usually use FreeFileSync, but the Finder shows the same behaviour. I can't copy contents of a folder, no matter from where I copy it.

 

I attached a screenshot from FreeFileSync. As you might be able to tell from the green graph, some files went through completely fine and when I reach that folder it starts to copy a file, around 99/100% it just holds, times out and gives that error. If I hit ignore the next file gets copied and the same happens again. I tried to copy that folder to my desktop and copy from there, no avail. I tried to copy by finder, nothing. I synced the entire folder structure to an external harddrive (the backup of that project) which worked fine, and then synced/copied from there, the same happens. I also tried to copy the contents somewhere else on unRAID to then move it with Krusader, but that also doesn't work.

 

I thought the data might be corrupt, but it's just a few Quicktime MOV which work fine (in QT as well as video editors), the entire folder is no 10GB large, unRAID has more than 7TB of available space.

 

It's really something, I can't even think on where to tackle this. As the files always copy but won't be finished, I figured it's some rights-management maybe, as FreeFileSync writes tmp files first. But as the Finder does the same, it doesn't seem like it?

I restartet the unRAID server, I restartet my work machine, but just no avail.

What the heck is up here?

Diagnostics are also attached.

 

Thanks for any pointers

 

Best Regards

 

postamt-diagnostics-20231206-1231.zip

These problems can be difficult to solve as times but let's start with this possible solution:

 

IT could be that there is not enough space on the disk that you are trying to copy the file to.  So, go to   Shares   and click on the blue Share name in the left column.  Than look for this parameter:

 

 

Make this parameter larger than the largest file, you will ever write to that share.  (I did a quick check on a couple of your shares and this parameter was not set on either one.)  Quick explanation, when Unraid starts to write a file to the disk, it has no idea how large the file actually is.  If the disk it elects for the write does not have enough free space for that file, the write will fail.  With this parameter set, Unraid will first check to disk to see that the disk has at least that amount of free space before it starts.  If it does not, it will check the next available disk in the array and repeats the check until it finds one with enough space.  It is import to realize that It is not the free space in the share that is important, it is the free space on the actual drive chosen for the write operation as the entire file must fit on that single disk! 

Edited December 6, 20232 yr by Frank1940

Fro more information, see here:

 

      https://docs.unraid.net/unraid-os/manual/shares/user-shares/#allocation-method

Hi there Frank,

 

thank you so much for your answer!

 

In this case, we only steadily use the Projekte share, where this setting was already set to 80GB. As we work in a film production, it's hard to tell what the max size of a file will ever be, but 80 seems reasonable. The fullest Disk is Disk 1, which still has 1,36TB of available space - my files are less than 10GB combined, each file is around 1 to 2GB.

So I assume this can't be the issue, as it's setup and the individual disks all have enough space? (I wanted to balance out the disks a bit over night, but unBalance seems to just make one drive less and another one full, at least that's when I hit "plan", so I won't do that for now)

 

For a quick test, I set it to 100GB but also set the array as the primary storage, just in case the Cache acts up. But no avail either, it's the same as before - copy to 100%, halting, error, rinse and repeat. But only with those files.

 

Best Regards

 

//edit - something I also noticed: when I copy through the finder and it "breaks", the unRaid server gets "soft disconnected" from my computer. Soft in the sense that it is still connected, but I can't access any share, until I reconnect to it

 

//edit2 - just to rule it out, I tried to copy from another machine. Doesn't work either

Edited December 6, 20232 yr by CameraRick

So you are bypassing the cache entirely for these copies and they are still failing.   I seem to recall that you should make a similar min free space setting on the cache from the  Main   tab.)

 

It might be that there is a  permissions on your Unraid sever.  Let's try this to try to fix them:.

 

1 -- Open up the   Tools    tab. 

 

2--  Find the    New Permissions   tool and open it:

 

3-- Look at the screenshot below.  Select   Shares   from the left dropdown box and the Share with the issue ( Projekte ) from the right dropdown box.  Click  START.

 

It will take some time as it is Bash script and will set the owner/group names and the permissions on each file and directory in the entire share.  

 

 

Edited December 6, 20232 yr by Frank1940

One more question, you are using SMB for these transfers and not NFS?

20 hours ago, Frank1940 said:

One more question, you are using SMB for these transfers and not NFS?

Yes, I am using SMB. I recall we decided against NFS as it's a bit of a pain to setup, or something like that.

 

I just started the permission-fixing.

 

Yesterday, I was able to copy the files by hooking them up to the server directly, through Unassigned Devices. When the rights-thing is done, I will try to delete the files and copy them again through my machine

 

Best,

Rick

I looked and you have one Unraid share that is Public.  How are you logging onto Unraid? (It appears that you have an Apple PC.)  With Windows, you are basically allowed only only one login to each server.  If you establish a login as a 'guest' by accessing a Public share, you can never gain access to Private share (without going to the Windows command line).  And the attempt to access a Private share generally presents an error message that is completely cryptic... 

 

Are mapping that share as a drive on your PC?  Double check that you have not changed the permissions on that drive on the PC side of the setup. 

 

There is also a sub-forum for MacOS/SMB issues.  For some reason, MacOS often requires some special Samba settings.   You can find it here:

 

        https://forums.unraid.net/forum/103-macossmb/

 

 

Hi there Frank,

 

yes, all of company is Mac-based. We just "see" the server inside our network area in the finder, click on it, and connect with username/login.

Looking at the Shares tab, all shares are either Private, Private in italics or have a dash (see screenshot)-

The unRAID runs here for close to two years, so I think if that public-login-issue was given, we'd have already ran into it?

It's not a general issue I encounter, as it was visible before it was only about a certain folder, not the whole server, hence my utter confusion, haha

 

The share is not accessed as a drive, I also don't think that macOS can do it like that? I mean, in the sidebar of the Finder any network-adress is visually close how external media is mounted, but it's not like it's a drive-letter as you can do in Windows. I am not even sure you can connect to network drives any other way in macOS.

I can't access any settings from my workmachine, the preferences window only shows info on the server, nothing to edit

 

I will study the sticky-thread in the link you provided, thanks for bringing this up, maybe it's one of these occasions where macOS "just works"; I encountered those quite often lately, haha

 

On another note: the permission-thing ran through (only took 3mins), yet I wasn't able to delete/copy things a new. HOWEVER, I had to update some new shots in the same location (I can't tell if the culprit was the "root" for my rendered files or just the subfolders) and that just copied through like it should.

A colleague just came up to me and told me he wasn't able to copy stuff in the last few days either; so maybe here is indeed a larger problem. He just started another sync, we will see if it runs through, and if the rights-thing was already it

 

 

On another note: at least to my personal feeling, more and more issues arose since I recently updated the server software to 6.12.4; I also recently updated my personal machine and installed 6.12.6, since then I also had some issues there. Seemingly some things were changed how you setup shares, and at least at home the settings were also a bit messed with. No idea if there's just a tiny thing that got adjusted for the worse during the upgrade, but so far it feels connected, not sure if it is though

 

Best,

Rick

As

1 hour ago, CameraRick said:

A colleague just came up to me and told me he wasn't able to copy stuff in the last few days either; so maybe here is indeed a larger problem. He just started another sync, we will see if it runs through, and if the rights-thing was already it

 

After reading this, I had an other look at your Diagnostics file and noticed that you only have a 250GB cache drive  and.  Since you are moving files of Projekte share through the cache drive, you should probably be concerned about the Minimum free space setting on the cache drive itself.  (I am not certain that Unraid honors the free space setting on the share when using the cache drive...)  You can set it here (after double-clicking on the 'cache' drive in MAIN):

You  do have to stop the array before you can change it. This may be a belt-and-suspenders type of setting but it worth checking out.  (Many folks have sized their cache  drives and set the Mover schedule so that this would never become an issue but with a 250GB drive, multiple users, and video files, you could be having an issue.) 

Hi there Frank,

 

thank you so much for your continued support.

 

The copy on the colleague didn't went through, so back to the drawing board, haha

 

Maybe I misunderstand the Cache and the Minimum free space entirely - the sync that stops working is around 300GB, but files are only around 10GB max in this scenario. Does this mean the Cache runs full (because it's set to 0GB I just saw)? I thought when this happens, it will just copy to the Array directly.

It's recommended to have at least as much "minimum free space" set to the max file size I expect - but to my understanding the minimum free wouldn't be touched, so it shouldn't matter if a 10GB or 50GB goes over that limit? I'm a bit challenged by this, haha

 

I just invoked the Mover, and will set the cache to 80GB as well; mostly we don't sync 300GB at once, mostly it's 10-30GB but then more often (like twice a day), with the Mover working every hour or so. If that doesn't work, I will try to bypass the cache completely. Maybe this is a different issue from the one I had earlier, after all

 

Best,

Rick

2 hours ago, CameraRick said:

I thought when this happens, it will just copy to the Array directly.

It's recommended to have at least as much "minimum free space" set to the max file size I expect - but to my understanding the minimum free wouldn't be touched, so it shouldn't matter if a 10GB or 50GB goes over that limit? I'm a bit challenged by this, haha

It will only revert to copying to the array if the free space is below the Minimum Free Space value BEFORE the next file is copied.   Having a value of 0 will effectively stop it ever reverting to the array.   That is why the it is recommended that the value is larger than the biggest single file you expect to copy.

 

I believe that in the current implementation the larger of the Minimum Free Space value set on a pool (cache) and the User Share is currently used.   I am hoping that these will be decoupled at some point as I have never understood why they are not applied independently depending on whether the next file is going to the pool (primary storage) or the array (secondary storage).

 

Thanks for the clarification!

 

And sorry for my late reply. As this is a work-issue, I can only check this during business hours, and only while machines aren't in use.

 

By now, everything went through, so it's either the free-space thing (shouldn't be with the original issue though) or the rights-thing, which I will mark as solution for now. Thanks again to both of you, especially Frank, for guiding me through this!

 

Best,

Rick