Request: More Advanced Data Allocation procedures

[Helmonder]

In the current situation you can set up rules in the unraid webgui that tell the system where data should be placed and how to handle the disk filling up, these are:

 

Per share:

- Included disks: the disks that data for this share may reside on (default: all)

- Excluded disks: the disks that may not be used for this share (default: none)

- Allocation Method (High Water: fill up disks evenly, Most-Free: place data on the disk that has the most available space, Fill up: first fill up one disk, then move to the next.

- Min.Free space: The amount of space that needs to remain on the disk (when 0 the disk is filled up to the max)

 

These settings are all used whenever you write new data to the system.

 

Whenever you change the settings, the new settings will be used from that point but the data that was not allready written is not re-allocated.

 

Data that is written by moving data around from disk to disk (by copying using disk shares, remote or on console) does not follow these rules.

 

Basically the way unraid works is that it does not care about the allocation rules when giving you your shares, it only uses them when you write new data.

 

Although the latter is a good thing it also means that during the course of time, whenever you change allocation rules or whenever you move files around using disk shares, your array will eventually have files everywhere.

 

There are several reasons why you could not want this (although some people would also not care):

 

- You might want to empty a disk because you want to take it of the array (use-case: you have a disk giving you smart errors that might point to a future failure, you can then take it out and let parity recreate the drive on a new volume, but if the drive is still ok you could also be even more safe and copy off al data and then let parity reconstruct the empty drive)

 

- You might want to make sure specific data is really only on one disk (use-case: you want to have as little disks as possible spinning)

 

I propose additions to the unraid gui that I think are not that difficult to implement (though I myself would absolutely not be able to do it,  I can think it up, not create it):

 

1) An "empty drive" option.

What this would do is simply move off (rsync I guess) all data of the drive to other locations the data may reside.

 

Example:

 

Empty disk1

on disk resides a folder called MOVIES

MOVIES has an include setting for disk3 and disk4

the process would now rsync the contents of MOVIES on disk1 to disk3 and disk4 according to the allocation rules already present in the gui (either highwater, most free or fill-up keeping the minimum disk space in account).

This process would need to be run in the background

 

2) A "Fill up" drive option

This process would do the opposite of 1), it would ask for a diskname, see what shares are allowed to use this disk and then move contents from those shares on other disks to itself.

 

3) A "Redistribute data" option

This process would go thru every drive and would see if there is a share on there that is not supposed to be there, if so then that share is moved according to allocation rules already present in the gui (not unlike used in the "empty drive" option, but limited to one share.

 

The logic behind this seems pretty simple, the physical action is doing copies/moves/rsyncs on linux level and although it might be a scary thought that the system is doing stuff to our data it is the exact base functionality of the system to do those actions, if they would be dangerous our problems would be large..

 

[sureguy]

A consolidate files function would be nice.  I'm sure many users have changed split level sometime after discovering they'd chosen the wrong split level. 

[Helmonder]

I actually excluded split level from the above because that makes it more complex, and the first part seems pretty easy to implement.. But indeed.. split level is also something that could be part of the reallocation.

[hawihoney]

+1 for 1), 2) and 3).

 

Regards.