Hello,

hope somebody can point me in the right direction.

I'm having problems with the "data-fillup" to my disks.

There is one huge share with 6 8TB disks with a total of 43TB, high water and automatically split level only top two directorys. Now first disk 8TB is filled up with more then half (4.3TB) of its space and the datas are copied still to this first disk. (min. free disk space is set to 50GB)

I do coppy with freefilesync running in a docker from one unraid server to the other as a backup server.

folder structure: share-->movies a-z, then main folder movies a-z, and then every movie in a subfolder (user/movies_a-z/movies_a-z/movie1/1.mkv)

But something I must do wrong, because array does not switch to fill up the secound drive.

Could it be wrong when I setted disk minimum free space manually to 50GB?

Additional infos:

First I configured the split level to "top only" and when I realised that this might be wrong with my folder structure I did select "top two" and restarted the server.

My parity drive has 10TB and all array disks have 8TB

Edited April 6Apr 6 by NoRaid99

Perhaps it would be a good idea if you gave an example of an actual path to a movie as it is not clear to me what your path’s actually look like.

An important point to realise is that in the event of any contention then Split Level wins over allocation method and can cause files to be written to a drive that might not have enough room. This can be an issue in particular if the application doing the copying creates the folders before copying the actual files.

thanks for reply, just edited my post.

path example: user/movies_a-z/movies_a-z/movie1/1.mkv

Perhaps I dont understand that spliting feature.

If I choose "manual: no automatic splitting" then my share shows only 8TB of space with disk1 only. If I choos automatic split level I get all selected disks and space.

---

edit:

could it be because FreeFileSync creates first all the directorys before copying the files?

---

edit2:

with setting "split any directory as required" it works.

so what split level would I need to set as minimum to make it work with my filesystem?

Edited April 6Apr 6 by NoRaid99

1 hour ago, NoRaid99 said:

If I choose "manual: no automatic splitting" thewn my share shows only 8TB of space with disk1 only.

That would be expected with Manual as only disks where you manually create the top level folder corresponding to share name are included in the share.

With the example you gave then once

1 hour ago, NoRaid99 said:

user/movies_a-z/movies_a-z/

Not convinced that is the actual path for a particular movie? Do you really have folders called ‘movies_a-z’ or is that a description? If you do not sure why it is repeated twice. Also do you really have a share called ‘user’? Once the path you give has been created (which has 3 levels) then anything that shares the third level is constrained to the disk on the third level is created.

okay, sorry. the path of the share is "user/movies_a-z/movies_a-z/..." the name of the share is movies_a-z with a subfolder with the same name.

don't really remember why I started with folder the same as the share folder. I think it was because of syncing with freefilesync. but thats allready a long time ago since I had to do it like this.

anyways, if I change to "user/movies_a-z/..." and wanting all disks included of the share, what split level I have to choose to not split the folder of a specific movie?

I still don't understand why it was not working at first try.

Edited April 6Apr 6 by NoRaid99

Honestly, you probably just want to use the "automatically split any directory" split level unless you have a reason to restrict how data is split.

If you restrict it too much like youve done already, you'll never write data to a second disk because unraid wont split due to folders that data is wrote to existing on the disk already.

I think the thing is with "automatically split any directory" that I will loose the control of my datas.

Actually I dont want any splitting.

Slowly I start to understand, hopefully .

If I get this done:

58 minutes ago, itimpi said:

That would be expected with Manual as only disks where you manually create the top level folder corresponding to share name are included in the share.

first I could not create the first level folder, but now I have seen that I have to tick "type" to be able to. I'll try it with "manual split". I'll try.

---

edit:

got split level now setted to manually.

had to create the folders with mc and changed the owner to nobody. now there is all the space of the 6 drives with 43TB listed in the share. started to fill up the share and will see if it is working now.

Edited April 6Apr 6 by NoRaid99

okay, trial and error brought me a bit further. "manual: no auto split" does not what I want, even when I create the top level folder on all the drives.

(the structure: share"movie "/folder "movie1"/any subfolders .. and I would like that the folder "movie1" does not get splitted at all.)

"auto split only top one level folder" didn't work as well, so I did go with the next possibility, the "auto split only two top level folders" and it started to fill up the secound disk.

so as résumé at the first go I would have setted the "auto split top tree level folders" because there was one more folder in the structure befor the actual movie.

but I still didn't get what is what level.

could someone eplain to me how is the hirarchy... first level is what? (sorry to ask that stupid question, but I don't get it)

got even more questions: what will be the level to choose if I want all my movies like "movie1" til "movie255" being stored all over my disks of the share in its unique folder and nothing splitted?

---

maybe it was best, letting me figure it out my self...

now I got the cause making me missunderstanding anything from before.

that what I want is "manual: no automatic split" and finally it is working. only had to create the main share directory on every disk of the share.

my problem was FreeFileSync. this tool did create all the folders as its first action, before copying the datas in them. so all the folders where created on disk one and just got filled up. thats why unraids "high water" could not work. now I was copying trough the file browser and the folders where copied one by one and it works as it should.

thanks a lot to the victims. :)

Edited April 6Apr 6 by NoRaid99

Say you have a level 2 split, /mnt/user/share(l1)/subfolder(l2)/lastfolder(l3)/files.txt

If you write subfolder(l2) to Disk1, all future folders and files that would be wrote to subfolder(l2) will forever and always be wrote to disk1 regardless of free space or the allocation method.

That's why the manual split was never writing to any other disk.

3 hours ago, MowMdown said:

That's why the manual split was never writing to any other disk.

No, finally I figured out it's not about splitting, it is about writing to all the disks of the share. this has to do with the allocation method.

The problem is that if I copied a bunch of folders with FreeFileSync they where all created first as empty folders in disk one of the share and thats why the datas where filled into these folders, without respecting the allocation method.

When I copy the hole bunch of files by linux filemanager, one folder gets copied with its content and then the next an so on. That way the allocation method gets respected and all works with "high water" now.

My only problem now is to find out what syncing tool uses a copy program that does copy file by file including contents.

Thanks a lot for the example to explane the levels.

I was to confused to understand this because the behavior of the way that FFS is copying. (first create all folders and then fill them up)

But I know now, that I do not need auto split. The first level folders won't grow, they stay, get deletet or substituted completely. I only want to fill the disks with "high water" allocation.

If somebody knows a syncing tool, that works different (one complete folder by the other), I would love the know.

Edited April 6Apr 6 by NoRaid99

Just keep in mind, split level is the highest priority/takes precedence above fill allocation and minimum free space.

I personally just use rsync to move files, not sure if there is a "better" way.

Thanks again for your help guys.

I will continue in future to sync with FreeFileSync, somehow I like this tool.

The thing is, I did change this backup server with a different hhd config, so I had to rebuild the shares and now it was the first time that I did choose a share "pool" with 6 hdds. And for now it is just to fill the backup disks again with the old datas and this caused all my missunderstandings and problems, because it didn't run like I expected.

I'm copying at the moment with krusader one disk by one from the main server to the backup server. Afterwoods when this is finished there will be only a few files getting updated, removed or added at once and it wont be a problem anymore with FFS and "high water".

And with your help I understand now how split level works and understand what the benefit is, but I don't want it. I have the feeling that I will loose control of my folders and what's in them. I am lucky to not needing every bit on my drives. As well they are less stressed when they have some free space to deal with.

For rsync I would prefer a GUI. But for regular jobs with the same parameters I could imagine to use it.

Edited April 7Apr 7 by NoRaid99