Balancing storage

[barrygordon]

When I set up my unraid I decided that I wanted to control where a file went.  I later decided that user shares were a very good thing.  For reading I use user shares, for writing I use mnt/diskn/...

 

I am thinking I may want to change my strategy to balance my storage across all available drives.

 

1) What are the opinions on this?  obviously less housekeeping on my part, A failure might require less effort  to restore especially if it is a fairly full drive

 

2) Has any one got a program or a script that would do this taking files from the drive with the least free space and moving them to the drive with the most free space (but in the same share folder).  I could do this from a PC but the network overhead of the interdriove moves will cause it to take too long.

 

I guess I could have the PC do the analysis and then put out a bash script that would do the work on the unRaid.

 

Thoughts/advice?

[BRiT]

I think Joe L may have a script or two that does something close to this. I sort of remember him (or was it someone else) that was re-balancing their movie collection.

[Rajahal]

I organize my data so that each share spans the fewest number of drives possible.  This helps minimize the number of disks that need to be spun up to browse a share (since I run my server stock, no cache_dirs).  I use include/exclude disk settings to specify where I want each share to reside.  At this point all of my disks are nearly full so my organizational scheme doesn't matter too much.  As soon as I add a new drive, I always have to change my share settings to include it lest I run out of space on another drive.

 

So basically my system isn't very good because none of my shares are static

 

When I reorganized my data way back when I just did the standard drag'n'drop from a Windows computer and let it run overnight.  Probably took me a few nights of that before I was done.

[NAS]

My script will help you make the decisions on what to merge (best to use mc)

 

http://lime-technology.com/forum/index.php?topic=2689.0

 

unfortunately it still requires a large amount of manual effort to make happen. But at least its easier to use.

 

Feel free to make it better, it was a hack to solve an immediate problem.

[armbrust]

How about create a new share with the parameters/settings you want, and just move all files over from the old share to it?  You could rename it back to the original share after the move completed.

 

Of course the distribution of the original files would affect the selection of the disk in the new share to write to, but this would be less and less of a factor as the move processed the files.

 

You could easily run it right on the unraid server.

[BRiT]

You may have unspecified results with such an approach. There's a slight issue when reading from and writing to user shares (/mnt/user/) in that the underlying file i/o used by programs like 'mv' will keep files on the same physical device. They will merely move a directory entry on the device and be done with it instead of copying multiple blocks.