I've 'sort of' found similar requests, but none that really address what I'm after.  Before I start, I appreciate that this might not be for everyone, but I throw files at shares and don't mind to which drives these shares go on, just as long as they adhere to the share's rules.

 

I have a share called "movies" with a split level of 1.  Occasionally, I put movies deeper folders on that share (keeping Lord of the Rings together, for example).  Because I've been buying drives as I've been filling them, pretty much all discs except for the last one is about full.

 

A simple issue arose the other day when putting Quantum of Solace into my James Bond subfolder.  Cannot copy - disc full.  Fair enough, I thought.  unraid1 (my server name) not full, just disc it was copying on to, since all the James Bond movies went on early.

 

After much juggling about to free space, I figured it would be handy if unRaid could move things around on my behalf in order to make space.  I appreciate that one caveat might be the insistence of a cache drive, but it would have saved me a lot of waiting around.

 

This leads on to another thought - which if I've understood correctly, I've read in another thread.  For those like me that only add drives when they need to (I would guess quite a few people), would there be any benefit in adding a "drives balance" feature to move folders around the drives to load them more evenly?  In point of fact, such a feature might have pre-empted my first problem.

 

Just a few thoughts - thanks for reading.  Simon

would there be any benefit in adding a "drives balance" feature to move folders around the drives to load them more evenly?  In point of fact, such a feature might have pre-empted my first problem.

 

I wonder about the value of this.

Seems like a customized preference to the installation.

 

I thought unRAID already does balancing based on split levels.

I'm not sure I want unRAID moving my data after it has established it's home on a disk.

I would rather grow onto a disk rather then have the OS (or a button) spread the data on my disk in an automated fashion.

I would not want to see my data become a victim of infant mortality on a new drive.

 

 

You also need to think of the bandwidth required to move the data.

 

Your cache will be flushed constantly. It would be a bit difficult getting anything done on the server.

To combat this I have a tool called rsyncmvd

 

I run it as

 

rsyncmvd /mnt/disk2/movies/Documentaries file.iso file.iso2 file.iso3 ...

 

Where it moves the files with rsync --remove-sent-files --bwlimit=6072

This keeps the file moving at a steady pace without flushing the cache so much that everything else stops.

 

I also fire it of with  nice -19 ionice -c3 to keep it at the lowest priorty so it does not affect torrenting.

 

Not that this satisfies your request.

However, it does provide a way for you to move files at the command line enabling you to "re-balance" your array as needed with respect to current io needs.

 

 

#!/bin/sh

if [ -z "${1}" ] 
   then echo "$0: Usage [destination dir] file[s]..."
        exit
fi

DIR=$1
shift

if [ ! -d "${DIR}" ]
   then echo "$0: argumment 1:${DIR} is not a directory"
        exit
fi

if [ -x /usr/bin/nice ]; then
   NICE="/usr/bin/nice -19"
fi

if [ -x /usr/bin/ionice ]; then
   IONICE="/usr/bin/ionice -c3"
fi

exec ${NICE} ${IONICE} rsync -avP --bwlimit=${BWLIMIT:=6400} --remove-sent-files "$@" ${DIR}

I thought unRAID already does balancing based on split levels.

It does, but it does not work that well when you incrementally purchase drives as the current array fills.

(really hard to balance the files into disks you do not yet own)

Because I've been buying drives as I've been filling them, pretty much all discs except for the last one is about full.

 

Joe L.

As the others have said the only way you are going to get something like this is to do it yourself, or make some sort of script to do it for you.

 

I personally have no need for a script like this.  I control what disk the data gets written to via include and exclude, but i also do not bother combining movies that are under the same Area (i.e. James Bond); I usually leave the HTPC application to do that.

it does not work that well when you incrementally purchase drives as the current array fills.

With drive sizes constantly increasing and prices constantly falling, the odds are good that your new drive is going to be larger than at least one of your existing drives.  You could simply replace your smallest (or fullest - your choice) drive with the new one, let unRAID rebuild the contents onto the new drive (which gets all the extra drive capacity to grow into), then add your old drive back into the array and reformat it which makes its capacity available for any new content.

 

I do agree that some sort of balancing script or process would be nice, but in the meantime, while not a perfect solution, that may get you by.

Thanks for that guys - all food for thought.

 

- I appreciate that the people might not want files moving once they're written, but I've needed to a couple of times.  Also appreciate that people might put everything at the top level, but I'm not at the moment.

- Yep - the split level works, but not when the drives aren't in the system yet.  With prices falling I figured it best to only buy when needed

- The thought of putting larger drives in holds some water - at present they're all Western Digital 1tb Green Power drives, so the parity would be the first to grow.  This would only really come in handy though when all drives were full (and hence I was upgrading).  The last time it happened I still had 3/4 spare on one drive

 

Coincidentally I've been splitting up my folders recently from just one share (DVDs) to three (movies, TV and music).  This appears to have balanced the discs naturally as it's been moving.

 

As a side thought - is there a quick way to find out what disc a particular file/folder is on?  I only really use XP/Vista and have only dabbled with the unRaid box when it's been playing up.  With more and more discs in the array, it's taking longer to find the folder I want to move.

 

Thanks again, very helpful (as usual)

As a side thought - is there a quick way to find out what disc a particular file/folder is on?  I only really use XP/Vista and have only dabbled with the unRaid box when it's been playing up.  With more and more discs in the array, it's taking longer to find the folder I want to move.

 

I use something called mlocate.

I compiled my own version, but there is one here.

http://slackware.rol.ru/linuxpackages//Slackware/Slackware-12.1/Console/mlocate/

 

First you run a scan after booting up.

Then you can schedule a rescan of the db when you feel necessary.

 

To fnd a file or directory type

 

locate filename

 

It will print all versions of the file or directory that match a pattern.

Very handle for large arrays.

Note the initial scan will take some time.

Cool, thank you.

Cool, thank you.

 

I've run into the same problem as you, and I've got my files organised in a similar way - normally movies are a split level 1, but some I like to group together, e.g. Lord of the Rings series, Godfather series etc.

 

As I grew my array, I found that a lot of drives were spinning up unnecessarily because my movies share spanned all 9 drives, as did my TV_Shows share. So I bit the bullet and spent 3 weeks moving and reorganising my data so that all my movies fit onto disks 1-7, TV shows were on disks 7 to 9.

 

However, since then the same problem has recurred as I've uploaded a ton more video files onto my server. I'm jumping from 11 to 15 data drives shortly, with another 4 data drives awaiting the 15+ drive bugfixes. So, I'll again spend a week or so with Midnight Commander moving things whenever I get a chance.

 

Last time, I did "lose" 4 documentaries in the move - somehow deleted them without moving.