After reading the following thread and segment.

http://lime-technology.com/forum/index.php?topic=4612.0

 

a big part of this fix was a change in the 'mover' script to use 'rsync' to copy files from the cache drive to the array.  This has required extensive testing on my part so that it works correctly with customer data.

 

Hey Tom,  I was wondering if there is any chance of publishing just the mover script so we can inspect and test it before release?

 

I'm wondering if there is a way to include a BWLIMIT option to keep the moves at a specific pace rather then full speed to prevent filling the cache completely.

... and some way to exclude directories from mover too (and not just with a '.' prefix!).

... and some way to exclude directories from mover too (and not just with a '.' prefix!).

 

This would be brilliant!!

 

I have modified my mover script so that folders with an underscore ("_") are also ignored.  On my Mac i could not see folders with the "." prefix which made it incredibly inconvenient to place stuff on the cache drive.

Agreed. Directory exclusions would be great, as well as being able to set the 'ignore character' via a global variable (perhaps on the config page), so we can use whatever character we want.

rsync does have an include and exclude parameters.

 

http://www.samba.org/ftp/rsync/rsync.html

 

-f, --filter=RULE           add a file-filtering RULE

-F                          same as --filter='dir-merge /.rsync-filter'

                             repeated: --filter='- .rsync-filter'

     --exclude=PATTERN       exclude files matching PATTERN

     --exclude-from=FILE     read exclude patterns from FILE

     --include=PATTERN       don't exclude files matching PATTERN

     --include-from=FILE     read include patterns from FILE

     --files-from=FILE       read list of source-file names from FILE

 

On one of my jobs I tried using --filter ${CONFDIR}/${P}.rsync-filter

 

which has rules in the context of

+ task/

+ *.py

+ *.odt

- *

 

http://blogs.verilab.com/partain/2009/08/rsyncs-filter-rules.html