Running out of space, but I'm not

[malaki86]

I'm in the process of moving media folders from one folder to a new one with a split-level set to "Automatically split the top level as required" (movie folder). All drives are included in the share, with no exclusions and I have a minimum free space set to 200Gb. I can see that new folder on every drive, and the File Browser confirms it, and that the split-level is correct. I originally set to "High Water", but it does the same when set to "Most Free".

 

The problem is that it when using the File Browser to move the media folders, I started getting "not enough free space" errors. Sure enough, one drive was completely full, with 85kb (or something like that free), and a 2nd drive was down to 145Gb.

 

I'm currently using the Unbalanced plugin to force move folders from the 2 full drives onto the least full drives.

 

Any clue what's going on with it?

[trurl]

Attach Diagnostics to your NEXT post in this thread.

[malaki86]

Diagnostics are attached

neo-diagnostics-20240211-0440.zip

[malaki86]

I cleared up some space on the 2 completely full drives with Unbalanced, leaving roughly 500Gb each. I'm going to try using the File Browser to Move the media folders that are split among multiple drives again.

 

Edit: I started the Mover manually to clear it, and started manually moving the split media folders again. I instantly saw the available free space dropping on the drive that is almost completely full (280Gb) while there are multiple drives with 1.25Tb+ of free space. The share is set to use Most Free space.

Edited February 11 by malaki86

[JorgeB]

Split level overrides allocation method, try setting to split all.

[malaki86]

2 minutes ago, JorgeB said:

Split level overrides allocation method, try setting to split all.

I'm using the split level so that all files for a single movie, or all episodes of a tv show season, are physically on the same drive. That's exactly what the split level is for.

[malaki86]

but, if the split level overrides the allocation method, then it's a screwed up implementation of it and needs to be redone.

 

[malaki86]

Quote

 

INFO

In the event of there being conflicts between the Minimum free space, Split Level and the Allocation method settings in deciding which would be an appropriate drive to use, the Split level setting always wins. This means that you can get an out-of-space error even though there is plenty of space on other array drives that the share can logically use.

 

 

What that means is that the split-level feature is completely worthless, unless you have a single drive with unlimited space on it. It's set to the lowest level split, but it's insisting on putting EVERYTHING in that one share on that one drive, while there are 9 more drives, with more than enough space available AND the same share on every drive.

 

Something needs fixed here, or just remove the split level completely

[JorgeB]

31 minutes ago, malaki86 said:

I'm using the split level so that all files for a single movie, or all episodes of a tv show season, are physically on the same drive.

And that will work if correctly set up and if the folders don't already exist on destination, also depending on your folder structure, the same split level may not work for movies and TV seasons, what is the current folder structure?

[itimpi]

Maybe you are misunderstanding Split Level?   What is the path for the files for a given movie?   If they are all in a folder named after the movie then you might need a split level of 2 to keep them all together.
 

  If you do have everything at the top level, then you want to be using the Highwater allocation method as that would keep files together except in the rare cases where a particular file reaches a trigger point so Unraid switches to the next disk for the next file.  The Most Free allocation method would almost certainly put related files on different drives a lot of the time.

 

 

[malaki86]

For the Movies, that I'm doing now, I have it set to "Top Level": /mnt/user/media_Movies/<movie>, which splits it at <movie> as it's supposed to do.

 

11 minutes ago, itimpi said:

Maybe you are misunderstanding Split Level?   What is the path for the files for a given movie?   If they are all in a folder named after the movie then you would need a split level of 2 to keep them all together.
 

  If you do have everything at the top level, then you want to be using the Highwater allocation method as that would keep files together except in the rare cases where a particular file reaches a trigger point so Unraid switches to the next disk for the next file.

 

 

Example: \Movies\Movie1\Movie1.mkv

Split only the Top Level:
        (Splits folders at this level apart from each other)
                              ↓
Disk 1 may contain \Movies\Movie 1\All Movie 1 files
Disk 2 may contain \Movies\Movie 2\All Movie 2 files

Split only the top two:
                               (Splits Here)
                                     ↓
Disk 1 may contain \Movies\Movie 1\some of the files for Movie 1
Disk 2 may contain \Movies\Movie 1\some of the files for Movie 1

for TV Shows:
        (\share\)
            ↓
Example:\TvShows\Tvshow 1\Season 1\Tvshow file(s)

Split only the Top Level:
           (Splits folders at same level as this)
                               ↓
Disk 1 may contain \TvShows\Tvshow 1\All Seasons\All Tvshow 1 files
Disk 2 may contain \TvShows\Tvshow 2\All Seasons\All Tvshow 2 files

Split only the top two:
                           (Splits folders at this same level)
                                         ↓
Disk 1 may contain \TvShows\Tvshow 1\Season 1\All Season 1 files
Disk 2 may contain \TvShows\Tvshow 1\Season 2\All Season 2 files

 

Edited February 11 by malaki86

[malaki86]

What is driving me nuts is that I can't move anything now until Unbalance clears more space on that one single drive because that's the ONLY place that the File Browser Move is putting the files now.

[trurl]

Dynamix File Manager will let you work directly with the disks so you don't have to rely on user share settings.

[malaki86]

1 minute ago, trurl said:

Dynamix File Manager will let you work directly with the disks so you don't have to rely on user share settings.

So, basically select stuff from the share (/mnt/user/media/movies) and move it to a drive with enough space (/mnt/disk2/media_Movies)?

[itimpi]

1 minute ago, malaki86 said:

So, basically select stuff from the share (/mnt/user/media/movies) and move it to a drive with enough space (/mnt/disk2/media_Movies)?

No - select from /mnt/disk1/media_Movies and move it to /mnt/disk2/media_Movies, etc.   do not try to mix User Shares And Disk Shares in the same operation.

 

Dynamix File Manager will not let you transfer between User Shares and Disk shares (or vice Vera) as this can lead to data loss if you are not careful.

[malaki86]

Just now, itimpi said:

No - select from /mnt/disk1/media_Movies and move it to /mnt/disk2/media_Movies, etc.   do not try to mix User Shares And Disk Shares in the same operation.

 

Dynamix File Manager will not let you transfer between User Shares and Disk shares (or vice Vera) as this can lead to data loss if you are not careful.

That doesn't solve the problem with the files being split up, which is why I enabled the split level option.

[itimpi]

Just now, malaki86 said:

That doesn't solve the problem with the files being split up, which is why I enabled the split level option.

Not sure I understand you.   That allows you to control exactly which disk any file ends up on so you are manually making sure files are not split when not wanted.

[trurl]

I've never bothered with split level. Highwater allocation will usually keep things together when they are written together. On the rare occasion where allocation changes disks, the only penalty would be a slight delay when another disk has to spinup.

[malaki86]

I have THOUSANDS of individual files spread across 11 drives, with 2-4 files per movie. Unraid has a split level feature, a minimum space feature and there's zero reason that they can't work together automatically.

 

[malaki86]

1 minute ago, trurl said:

I've never bothered with split level. Highwater allocation will usually keep things together when they are written together. On the rare occasion where allocation changes disks, the only penalty would be a slight delay when another disk has to spinup.

But, when files are written, then years later more files for that movie are downloaded, like subtitles, posters, etc, they end up scattered everywhere. That's what the split level is for.

[malaki86]

And, even now, I'm trying to move a single movie folder from '/mnt/user/media/movies' to '/mnt/user/media_Movie/movie' and it's putting it on the drive that is down to 85Gb, while the minimum space on that share is set to 200Gb.

 

WHY? 

 

There are 10 more drives on that array, with all but one having 1Tb+ of free space. Why is it ONLY going to that ONE drive?

[itimpi]

Just now, malaki86 said:

I have THOUSANDS of individual files spread across 11 drives, with 2-4 files per movie. Unraid has a split level feature, a minimum space feature and there's zero reason that they can't work together automatically.

 

Those two settings are not really related to each other.   one controls grouping of items while the other controls overflow to another dtive.
 

I do not see why a Split Level of 1 is not working for all new files if you are using High Water Allocation method.    Unraid will not move files between drives, so if your previous settings scattered files then it is a manual clean up activity.

 

 

[itimpi]

What are you using to try and move the files?

 

[malaki86]

Just now, itimpi said:

Those two settings are not really related to each other.   one controls grouping of items while the other controls overflow to another dtive.
 

I do not see why a Split Level of 1 is not working for all new files if you are using High Water Allocation method.    Unraid will not move files between drives, so if your previous settings scattered files then it is a manual clean up activity.

 

 

The files regroup into a single drive in a Move from the File Browser, BUT, file browser is ONLY writing to ONE of the ELEVEN drives.

[malaki86]

1 minute ago, itimpi said:

What are you using to try and move the files?

 

Dynamix File Manager plugin

Edited February 11 by malaki86
wrong name