banterer Posted June 4, 2022 Share Posted June 4, 2022 (edited) I have an array with plenty of space on it, and a share called `media` set to `cache: yes`. However, when the cache fills up, my script (using rsync to copy to `/mnt/user/media`) generates a disk full error: No space left on device (28) Yet cache: yes says If there is insufficient space on the Cache disk/pool, then new files and directories are created on the array Why is this?? Do cache rules not work for rsync / bash scripts? Edited June 4, 2022 by banterer Quote Link to comment
ChatNoir Posted June 5, 2022 Share Posted June 5, 2022 I do not use RSYNC but the result would probably change depending of the command you are using. You should share an exemple of your command. Did you set up a Minimum free space for your cache Pool ? This should solve your problem As a general rule, you should not cache a share if you regularly add more data than the cache pool can hold. Quote Link to comment
banterer Posted June 5, 2022 Author Share Posted June 5, 2022 The command is: rsync -a /mnt/disks/main/backups/* /mnt/user/backups/ I can't find the minimum free space setting, where is it exactly? Re your point on not adding more data than the cache pool can hold - absolutely. This is just the initial copy, future syncs will be incremental. I'm caught in a situation where none of the cache options 'fit'. If I set it to 'no', it will leave data it's already copied on the cache... so I have to set it to yes! Could do with another option which will move from the cache, but not put any further data onto it.... Quote Link to comment
itimpi Posted June 5, 2022 Share Posted June 5, 2022 8 minutes ago, banterer said: I can't find the minimum free space setting, where is it exactly? You can find the information here in the online documentation that can be accessed via the Manual link at the bottom of the GUI. Quote Link to comment
strike Posted June 5, 2022 Share Posted June 5, 2022 12 minutes ago, banterer said: 'm caught in a situation where none of the cache options 'fit'. If I set it to 'no', it will leave data it's already copied on the cache... so I have to set it to yes! Could do with another option which will move from the cache, but not put any further data onto it.... Just run the mover and it will move data already copied from cache to the array, then set cache to no and do the rest of the copy. When finished set cache to yes if you want. Also be sure to set your split level to split any dir. you can change it after the initial copy if you want. Split level is important because rsync will create all the directories first before it copies any files, so if the split level is set to anything other than to split any directory on the initial copy you will run into the same issue with the disk filling up. Quote Link to comment
banterer Posted June 5, 2022 Author Share Posted June 5, 2022 I have split any directory enabled - settings below. I will set the min space to 100G or something like that, when move has finished and I can stop the array. I still don't understand why a disk full error is being generated though, because surely the min space setting shouldn't matter - if it's X% full, or 100% full, further writes should go to the array according to the help / docs? Quote Link to comment
strike Posted June 5, 2022 Share Posted June 5, 2022 52 minutes ago, banterer said: I still don't understand why a disk full error is being generated though It might be because of the way rsync is copying directories/files. As I said in my previous post rsync will create the entire directory structure before copying any files. And thus will most likely try to copy all the files into the already created directories. If you do a normal copy I think you will find the cache setting is working as intended. I haven't tested it tho, as I never use the cache feature. I have cache no set on all my shares except the appdata and VM share which is set to only. Quote Link to comment
JorgeB Posted June 5, 2022 Share Posted June 5, 2022 1 hour ago, banterer said: because surely the min space setting shouldn't matter It does matter, you must set a minimum free space for the share and should also set one for the pool, the one for the share should be set at about twice the size of the largest file you expect to copy to that share. Quote Link to comment
strike Posted June 5, 2022 Share Posted June 5, 2022 (edited) 45 minutes ago, JorgeB said: It does matter And now I remembered WHY it matters just to put that out there as well. It's because unraid does not know how big the file you're going to copy is, it only knows how much space is left on the disk. So if the file is bigger than the space left on the disk it will fail if the minimum free space is not set. If unraid sees that there is less than the minimum free space left it will choose another disk IF the split level permits it. If not it will continue to fill the disk until it runs out of space or files are manually moved to another disk to free up space. Edit: Paraphraseing, unraid do know how big your files are. But when creating a file unraid does not know how big it's going to be before it created. And when copying/moving a file you're essentially making a new file just with the same data. Edited June 5, 2022 by strike Quote Link to comment
banterer Posted June 5, 2022 Author Share Posted June 5, 2022 Ah yes I guess that makes sense. Now I just need to find out why Mover is taking 12 hours (so far) to move <1TB Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.