Rsync runs out of cache space, and doesn't switch to the array

[MrChip]

I'm setting up a new Unraid server and using rsync to bring files to it from another server. Rsync operates as I expected at the start - it writes all the new files to the cache, up to the time it reaches the minimum free space setting. At that point I'm expecting it to continue by writing to the array, but it doesn't. It continues writing to the cache until the cache fills. Then rsync reports a disk full error and dies.

 

I have both the share minimum free space and the cache minimum free space set to 1GB. The last time this happened rsync was copying a collection of 6 megapixel photos, jpg format, so the files were well below the set minimum.

 

The rsync command I run is:

rsync -av user@host:/mnt/user/NAS/    /mnt/user/NAS   >   /tmp/bb

 

From the .cfg for the share:

shareFloor="1000000"

 

From the .cfg for the cache:

diskShareFloor="1GB"

 

The cache is a pool of 2 SSDs, RAID1 with btrfs.

 

I'm attaching a diagnostic file.

 

What am I missing here, please?

 

Thanks for any guidance.

 

Bob

 

 

bailey-diagnostics-20210626-1920.zip

[trurl]

What I have experienced with rsync, and others have experienced with other applications such as krusader. The application will create the empty folders all on one disk before beginning to fill those folders, then it will copy the files to the already created folders.

 

It happened to me when first loading my backup server. I noticed it was working in alphabetical order, so I just started at the other end of the alphabet and moved empty folders to other disks.

 

For your situation, I would have recommended not caching to begin with. Best if you don't cache if you intend to write more than cache can hold.

[MrChip]

Okay, so it's about rsync, in this case. That eases my mind - I was looking for the setting I got wrong....

 

So if a folder is created on the cache before the minimum free space is passed, Unraid will later write a file into that folder on the cache, even if the minimum has been passed in the interim?

 

(edited to correct a typo)

Edited June 27, 2021 by MrChip

[JorgeB]

Like mentioned rsync will create all the folders before start copying, but that only affects split level, i.e. it must be set to any, minimum free space will still be respected, so something else is going on, possibly 1GB for cache floor is too low, especially if there's anything else using it.

[MrChip]

When the disk filled up the rsync was copying my photo library - everything in there is nowhere near 1 GB in size. I suppose the floor could be too low, but I don't see it.

 

I followed trurl's suggestion and turned off the cache and then finished the rsync. That completed without interruption. 

 

Thank you for your inputs.

 

Bob