How does unraid deal with files that are too big to fit on the first drive in the pool free space?

[m00nman]

Hello everyone,

 

I recently decided to take the plunge and migrate from Proxmox to unraid to save time on setting up new services etc. With proxmox I was using mergerfs + ext4 drives and I'm now slowly copying the data to unraid and then refromatting the drives and adding them to the array. The issues I've encountered is when I'm using regular "cp -a" on command prompt to copy the data from ext4 drives to unraid pool, and when a drive fills up and there is no space left to hold the next file that's being copied, it just fails to copy this file and moves on to the next file which gets allocated to the next drive (with more free space) in the pool. With mergerfs the behaviour was that if the file doesn't fit in the free space in the drive, it will get allocated on a different drive without failing. Did I miss a setting  or is this the expected behaviour?

Edited October 27, 2022 by m00nman
Added image

[itimpi]

This is expected behaviour!   Once a drive has been chosen Unraid does not switch to a different drive if it turns out there is not enough space for it.

 

There is a Minimum Free Space setting that can be used to set criteria for when the next file should go to a different drive.  It is normally recommended that this should be set to a larger value than the biggest file you expect to copy just to avoid such failures.

[m00nman]

Thanks. That's unexpected. I expected a more graceful handling of this issue. I suppose I'll contribute to the "Feature requests" subforum now

[tjb_altf4]

The main workaround for a Chia copy workload is to copy to one drive at a time, if you use "most free" and multiple drives of same free space, it will round robin to minimize issues.
I generally set min free space to 2x plot size, and copy in the last 1-2 plots manually.

 

[m00nman]

Thanks for the suggestions. I'm using "Fill-up" allocation method right now, and just watch which plots return errors in cli and copy them to a specific drive manually, then go back and remove partially copied plots. I have set 110GB min free space now for the Chia share. Unfortunately not all drives are the same size. I mostly have 14TB, but there are 18TB and 8TB in the mix. What complicates things more is that I used ext4 fs previously with reduced inode count, and xfs seems to have less free space in the end compared to that (on the same drives). I guess I took the mergerfs' "moveonenospc" mount option for granted before

Quote

moveonenospc=BOOL|POLICY: When enabled if a write fails with ENOSPC (no space left on device) or EDQUOT (disk quota exceeded) the policy selected will run to find a new location for the file. An attempt to move the file to that branch will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false, true = mfs)

 

Edited October 28, 2022 by m00nman