New Allocation Method - Pattern Match

[noski]

Maybe I am a bit OCD, but for particular Shares, I like to place files on specific disks.

 

The way I manage this currently is either using the disk specific shares (disk1, disk2, etc) or ensuring the top level directory in the share only exists on 1 disk and using a "Split level" of "Manual".

 

What I propose is a new "Allocation Method" (in Share Settings) that provides the ability to define a path pattern for each disk.
The procedure for selecting the disk for writing would work like this:
- iterate over each defined path pattern
- if there is a match, then write the file to that disk (and break/end the loop if successful)
- else continue until all definitions are tested

The patterns would use standard regex rules.

Potential issues:
- After iterating over all path pattern definitions, a match is not found. Expectation - this simply results in a failure writing the file.
- A path could match multiple definitions. Potential Solution 1 - only the first match is used. If the disk is full, it fails. Potential Solution 2 - if there is a match, but the disk is full, continuing looking for another match.

- Not sure how this would work with the "Split level" setting. Perhaps that needs to be disabled when using this Allocation Method.

- Perhaps the "Included disk(s):" and "Excluded disk(s):" need to be disabled also.

UI Design:
For me personally, I am fine with the webpage presenting a list of fields to define the patterns in normal disk order (ie disk1, disk2, ... ).
However, if it is possible to provide controls on the page to reorder the definitions (so that disks higher up the list take precedence), I see that as potentially being useful to people also.

An simple example:

Share name: Movies

disk1: "^[aA].*$" - Match all paths under the share that start with 'a' or 'A'. Example: "Movies/Avatar (2009)"
disk2: "^[bB].*$" - Match all paths under the share that start with 'b' or 'B'. Example: "Movies/Battleship (2012)"
...
disk26: "^[zZ].*$" - Match all paths under the share that start with 'z' or 'Z'. Example: "Movies/Zombieland (2009)"
disk27: ".*" - Catch all the rest. Example: "Movies/2012 (2009)"

 

I understand this may not result in drives filling evenly or efficiently. That is not the intention of this allocation method. The other existing allocation methods already serve those purposes. The 'Pattern Match' Allocation Method that I have proposed is more for those OCD people that want more granular control of where files are placed within the Array when using a User Share.

[BRiT]

Possible workarounds today...

 

Use sub directories and split levels appropriately, simply create the A, B, etc directories only on the drives you want them to be on.

/mmt/user/Movies/A/

/mnt/user/Movies/B/

...

/mnt/user/Movies/Z/

 

Other way is to use cache drive and replace mover script with your own script using your regex pattern matching logic or what have you.

[itimpi]

3 hours ago, noski said:

The way I manage this currently is either using the disk specific shares (disk1, disk2, etc) or ensuring the top level directory in the share only exists on 1 disk and using a "Split level" of "Manual".

Just in case you are not aware of it with Split Level set to manual it is still possible to split a a sub folder of a share across more than one disk by creating the top level on the disks to be used.   The allocation method then kicks on the disks holding the path

[noski]

Thanks BRiT for the ideas on possible workarounds.

Yes having the top level directory in the share works (and can be used to span more than one disk as itimpi indicated), but I am after a solution that doesn't require that top level directory. Mainly for applications that don't really work with that setup - for example, Sonarr/Radarr.

 

Thanks for the suggestion of a custom mover script. I hadn't thought of that. It sounds do-able as a workaround in light of not having this feature. I will attempt to give it a crack and will post back if I come up with something.

 

 

[trott]

personally I think the whole purpose of disk pool /array solution is you don't need to care about which disk your data actual on

[BRiT]

For myself, I fill one disk then move onto the next when it's full and I have to expand the array. The only time I care about which disks movies are on is when I replace multiple smaller drives with a larger drive, like 5 2TB to 3 4TB, or 4 4TB to 3 8TB. That's when I take the opportunity to group Collections together then attempt to split by movie Year or movie Letter/Name. I did this differently between those 2 moves, where it seems Release Year is the more natural split, it enables all but the newest drive can slow-down as majority of the time users watch newer movies.

 

But I can understand someone's CDO and not wanting to have to manually manage this all the time.

[lionelhutz]

I can't see this being implemented. I bet the majority of users barely understand the basic allocation methods.

 

I hope you get the script figured out. But then, do some searching because I bet someone has a solution to sort like you want.

Edited September 1, 2019 by lionelhutz