I would say the later. I think the compromise of the allocation not being perfectly sequential is a small price to pay to keep disks spun down.

Another idea:

 

"Allocate small files to a single drive"

 

On the face of it this doesnt seem to help but if you assume alot of unRAID user will be HTPC/media users it could actually help alot. Files like folder.jpg, *.nfo, *.sfv etc are used by apps for specific things. For instance XBMC uses folder.jpg, *.nfo and file listsings for its library lookup. When you do an update because it is reading all the nfos it causes all the drives to spin up. But if small files were on a single drive only one drive would spin up (assuming dentrys are cached).

I am revitalising this thread with an example. Before i started this thread I merged all my data at the TV SHow letter i.e. all letter C shows were teogether on 2 drives five and ten.

 

Today this is what it looks like:

 

137G    /mnt/disk5/TV/TV - C

27G    /mnt/disk12/TV/TV - C

3.7G    /mnt/disk9/TV/TV - C

37G    /mnt/disk4/TV/TV - C

8.4G    /mnt/disk2/TV/TV - C

351M    /mnt/disk3/TV/TV - C

14G    /mnt/disk6/TV/TV - C

16G    /mnt/disk11/TV/TV - C

8.1G    /mnt/disk14/TV/TV - C

56G    /mnt/disk10/TV/TV - C

29G    /mnt/disk13/TV/TV - C

 

 

So the current highwater allocation has spread my data since then over 11 drives. Never should we have to spin up 11 drives for anything day to day and is the crux of my argument that highwater has an annoying limitation.

 

Are those sizes of the DISK or the sizes of the DIRECTORY.

My assumption is high-water is based on DISK size. not directory size.

 

I still feel I prefer the suggested "ordered" allocation up until X% of disk is used.

thats the sizes of the folders. I cobbled together a tool to quickly show it and make life easier for recovering from highwaters annoying allocations

It seems to me high-water allocation is doing what it's supposed to.

 

For me, I've begun to redefine my shares to use included disks or just use the disk shares.

The automated allocation within user share is spinning disks up all over the place needlessly.

 

For example, I moved all my data off disk1 and writing a DVD still caused all 9 disks to spin up.

I believe this is because all shared disks need to be tested for size to determine re-allocation.

 

 

 

 

It seems to me high-water allocation is doing what it's supposed to.

 

I think it is doing what it is designed to do but not what it should do, a subtle but important difference The current implementation does not take into account the design criteria that spreading across disks is not always a good thing.

 

For me, I've begun to redefine my shares to use included disks or just use the disk shares.

The automated allocation within user share is spinning disks up all over the place needlessly.

 

Yes that is one solution. Not one i want to do though as its too much like hard work to overcome a program limitation that could be removed with some extra thought.

Yes that is one solution. Not one i want to do though as its too much like hard work to overcome a program limitation that could be removed with some extra thought.

 

Agreed, I would really love to see ordered allocation. I think it's easier to tell when you need to purchase/upgrade a new disk.

On a similar ilk...

 

i use mc to merge the folders but do you know of a way from the command line that runs no risk of losing data

 

i.e. mv /mnt/disk5/TV/TV - C /mnt/disk12/TV/TV - C

 

that merges?

rsync

this is my shell to do an rsync mv with delete. It's syntax is a lil backwards from mv, but I prefer it this way.

It also controls the rsync via nice, ionice and bwlimit to avoid process interaction and complete cache flushing.

 

Arg1 is destination directory

Other args are filenames.

 

rsync will not overwrite an existing file "if the destination matches or is newer"

 

Adjust BWLIMIT below according to your cache. 6400 works well with 8G, so 3072 should be good for 4G (although try it).

 

My reason and design was to prevent complete cache flush, thereby interfering with rtorrent.

The current method allows for a slow smooth controlled flow of data.

 

 

#!/bin/sh

if [ -z "${1}" ] 
   then echo "$0: Usage [destination dir] file[s]..."
        exit
fi

DIR=$1
shift

if [ ! -d "${DIR}" ]
   then echo "$0: argumment 1:${DIR} is not a directory"
        exit
fi

if [ -x /usr/bin/nice ]; then
   NICE="/usr/bin/nice -19"
fi

if [ -x /usr/bin/ionice ]; then
   IONICE="/usr/bin/ionice -c3"
fi

exec ${NICE} ${IONICE} rsync -avP --bwlimit=${BWLIMIT:=6400} --remove-sent-files "$@" ${DIR}

Sorry to bring an older thread, but is a parity check necessary after moving files from disk to disk?

 

I am just doing a straight move, that should be ok right:

mv /mnt/disk8/Movies/* /mnt/disk3/Movies/

 

Thanks!!

Sorry to bring an older thread, but is a parity check necessary after moving files from disk to disk?

 

I am just doing a straight move, that should be ok right:

mv /mnt/disk8/Movies/* /mnt/disk3/Movies/

 

Thanks!!

No need for you to do anything. 

 

Parity is calculated as files are created, moved, deleted, appended, etc as you perform the operation.

 

Only time you need to do a parity check/calc is initially, and periodically to read every sector on every disk to give the SMART capabilities built into the disk drives the chance to detect any error.  I have a monthly parity check scheduled on my machine. 

One other time you will see a parity check is if you power down without stopping the array first. Upon power up, a full parity check will occur.

 

Other than those times, no need to do any parity check for normal file operations.

 

Joe L.

Cool thanks.

 

I did notice the parity drive being accessed while I was moving files around.

 

Sorry but one other question:

 

If I telnet into the unraid machine and initiate files to be moved from one disk to the other, do I have to leave the telnet session open, or will it continue to move the files if I close telnet?

 

Thanks,

Cool thanks.

 

I did notice the parity drive being accessed while I was moving files around.

 

Sorry but one other question:

 

If I telnet into the unraid machine and initiate files to be moved from one disk to the other, do I have to leave the telnet session open, or will it continue to move the files if I close telnet?

 

Thanks,

There is a way to do it where you can log off, but it involves installing the "screen" utility, a slackware package.  This is a bit advanced though, so...  You probably need to leave the telnet session open.

 

Joe L.

Cool thanks.

 

I did notice the parity drive being accessed while I was moving files around.

 

Sorry but one other question:

 

If I telnet into the unraid machine and initiate files to be moved from one disk to the other, do I have to leave the telnet session open, or will it continue to move the files if I close telnet?

 

Thanks,

There is a way to do it where you can log off, but it involves installing the "screen" utility, a slackware package.  This is a bit advanced though, so...  You probably need to leave the telnet session open.

 

Joe L.

 

Whenever I want to move large amounts of data or run some other time consuming process which I don't want to wait to finish I just use the 'nohup' command.  Works great.

http://en.wikipedia.org/wiki/Nohup

Thanks xbit...I will give it a try.