To Cache drive or not to Cache drive?


Recommended Posts

  • Replies 366
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Posted Images

Hi,

I've just added a 1TB Cache drive to my 5x 2TB disks, what's the best / easiest way to use it as a temp storage area for files and apps such as Transmission and sabnzb.

 

I would like the files to remain on the cache drive until I manually send them to my shares.

Is this possible?

 

 

Thanks! :)

 

 

 

parity WDC_WD20EARS-00M_WD-WMAZA0268048 27°C 1,953,514,552 - 4,983 5,077 0

disk1 WDC_WD20EARS-00M_WD-WMAZA0420587 28°C 1,953,514,552 502,687,492 9,497 38 0

disk2 WDC_WD20EARS-00M_WD-WMAZA0605248 28°C 1,953,514,552 973,482,380 127 38 0

disk3 WDC_WD20EADS-00S2B0_WD-WCAVY1999822 31°C 1,953,514,552 1,167,248,672 14,508 4,933 0

disk4 WDC_WD20EARS-00MVWB0_WD-WCAZA5804819 28°C 1,953,514,552 1,953,422,088 65 36 0

cache SAMSUNG_HD103UJ_S13PJ1KQ411406 22°C 976,762,552 976,699,896 311 7,861 0

Link to comment

Just disable Cache drive for each of your shares if enabled.

 

Prefix any folders you don't want moved with a '.'

 

And you'd need to somehow disable the mover script--such as creating a script which removes it from crontab

 

I know the last one has been discussed on the forum, if not this thread.

 

Or you can name every top level folder (including where you put your folders) prefixed with a '.' so the mover script ignores everything on the drive.

 

But why put files on the cache if you want to manually move them off?

Link to comment

Right, for applications. Just create a folder ".transmisson" and ".sabnzb" and put those applications' files under there.

 

Have any temp files you download to be also saved under those directories, and they won't be moved off by the mover script.

 

Only files not prefixed with "." (or not under a folder at the top prefixed with '.') will be moved off by the mover script.

Link to comment

Only files not prefixed with "." (or not under a folder at the top prefixed with '.') will be moved off by the mover script.

Incorrect.  The mover ignores files under a top level folder named with a leading period.  It has no special logic for files in other directories named with a leading period.
Link to comment

so if I make a folder ".name" in windows on that drive at the top level available it will be ignored by the moving script?

 

 

 

exactly.  It also appears as if it will ignore any files at the very top level, regardless of their name.  In other words, a file at

/mnt/cache/ignored_file.txt

will be ignored because the mover pattern is only looking at files one level below the top, and only if they are not in a top level directory whose name does not start with a period.

 

You can run these commands and it will print the files the mover WILL move.

cd /mnt/cache

find -depth -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -print

 

 

Link to comment

So is that regex saying, print all the files that do NOT apply to that regex? That is, print every file under a directory which doesn't contain a '.'?

No, it says print all that match the pattern that are NOT currently open for reading or writing  ( ! exec fuser -s {} \; ) does that logic.  the mover will not move a file being written or one being read.

 

file names would be output like this

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

 

The pattern is:

[.]/[^.].*/.*

 

[.]  matched the leading period

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

 

[.]/[^.].*/.*

 

/  matched the following "/"

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

 

[.]/[^.].*/.*

 

[^.]  matches any single character EXCEPT a period "."

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

 

[.]/[[^.].*/.*

 

.*  matches any number of any character  (in this case, in the top level directory name)

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

 

[.]/[[^.].*/.*

 

/  matches the "/" in the directory path.

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

 

and lastly,

[.]/[[^.].*/.*

 

.*  matches  any number of any character.

./Movies/ALIEN.ISO

./Movies/ALIEN-2.ISO

./Music/Santana/track1.mp3

Link to comment

so if I make a folder ".name" in windows on that drive at the top level available it will be ignored by the moving script?

 

 

 

exactly.   It also appears as if it will ignore any files at the very top level, regardless of their name.  In other words, a file at

/mnt/cache/ignored_file.txt

will be ignored because the mover pattern is only looking at files one level below the top, and only if they are not in a top level directory whose name does not start with a period.

 

You can run these commands and it will print the files the mover WILL move.

cd /mnt/cache

find -depth -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -print

 

 

 

I am a little confused. By top level do you mean "root" because all the files I place in the root of the cache drive are moved, yet you said that mover ignores any files at the top level implying that files to be moved have to be one directory down.

Link to comment

so if I make a folder ".name" in windows on that drive at the top level available it will be ignored by the moving script?

 

 

 

exactly.   It also appears as if it will ignore any files at the very top level, regardless of their name.  In other words, a file at

/mnt/cache/ignored_file.txt

will be ignored because the mover pattern is only looking at files one level below the top, and only if they are not in a top level directory whose name does not start with a period.

 

You can run these commands and it will print the files the mover WILL move.

cd /mnt/cache

find -depth -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -print

 

 

 

I am a little confused. By top level do you mean "root" because all the files I place in the root of the cache drive are moved, yet you said that mover ignores any files at the top level implying that files to be moved have to be one directory down.

The top level of the cache drive is mounted at /mnt/cache

 

I just defined a cache drive on my test server, created a test file on it, then ran the test "find" command I showed earlier.  It did not "find" the file and would not move it.

 

An "ls -l" shows the cache drive empty

root@Tower2:/boot# ls -l /mnt/cache

total 0

 

I create a file named "joe.txt" on it.

root@Tower2:/boot# touch /mnt/cache/joe.txt

 

An "ls -l" now shows the new file I created.

root@Tower2:/boot# ls -l /mnt/cache/joe.txt

-rw-r--r-- 1 root root 0 Jun  8 16:37 /mnt/cache/joe.txt

I changed directory to it, just like the "mover" script.

root@Tower2:/boot# cd /mnt/cache

And I ran the test "find" ... it prints nothing, since the file does not match the "regex" pattern of being in a top level directory on the cache drive.

root@Tower2:/mnt/cache# find -depth -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -print

 

I create a top level directory

mkdir /mnt/cache/data

I create a file in it

touch /mnt/cache/data/test.txt

 

I run the "find" again, similar to the mover, and this time it finds the file, so it would be moved.

root@Tower2:/mnt/cache# find -depth -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -print

./data/test.txt

 

I rename the top level "data" directory I created as ".data" and re-run the "find" ... it ignores the file.

root@Tower2:/mnt/cache# mv data .data

root@Tower2:/mnt/cache# find -depth -type f -regex '[.]/[^.].*/.*' ! -exec fuser -s {} \; -print

root@Tower2:/mnt/cache#

 

 

 

 

Link to comment

Thanks, are you able to share the Cache drive via smb?

 

When you use the commands

 

"mkdir .<foldername>"

 

Does that make a folder on the Cache drive, which stays there and that I use for downloads etc.

or, you can type:

mkdir /mnt/cache/.<foldername>

giving it the full path to the folder.

Link to comment

HI Jo

 

Just figured out why my files do move. They are all actually under a top level directory called Media. I copy to the cache drive using the user share media so they automatically are created under a directory one level down and hence move. A subtle point that I had failed to appreciate earlier.

Link to comment

HI Jo

 

Just figured out why my files do move. They are all actually under a top level directory called Media. I copy to the cache drive using the user share media so they automatically are created under a directory one level down and hence move. A subtle point that I had failed to appreciate earlier.

 

The files you copy to "Media" should be files you want moved into the array. Use a temp storage directory such as ".temp" or ".downloaded" if you want to have a place to store files that don't get copied to the array.

 

Peter

 

Link to comment

Thanks, are you able to share the Cache drive via smb?

 

When you use the commands

 

"mkdir .<foldername>"

 

Does that make a folder on the Cache drive, which stays there and that I use for downloads etc.

or, you can type:

mkdir /mnt/cache/.<foldername>

giving it the full path to the folder.

 

That's if he's on the console, or in through telnet/SSH.

 

Through Windows command prompt, he'd have to either be on the mounted drive of his server's cache disk share, or opened a command prompt through Explorer in "\\tower", for example.

Link to comment

need some help trying to understand this cache drive.

 

I have Sab, SickBeard, and Couch potato all up and running via cache drive  in a ".custom" folder so those temporary files stop getting written to my flash drive. I have 3 disks in my array plus my parity drive on each disk I have "Movies" folders I have programs  pointing to my user share of \\tower\Movies to scrape my movies getting my artwork, movie trailers, nfo files etc.  

 

if I have a movie residing in /mnt/disk3/Movies/Sucker_Punch  and I scrape for artwork using my program that is pulling from my usershare/Movies why does it create a  directory on my cache drive like so /mnt/cache/Movies/Sucker_Punch instead of on /mnt/disk3/Movies/Sucker-Punch.

 

what I am getting at is, how do those programs even know about my cache drive when I am not telling them send downloaded files there?

and if I press the move button those files are indeed moved to /mnt/disk3/Movies  disk1 and disk2 are full and don't have room for any more files.

 

 

 

-=jason=-

 

 

EDIT: I just had a look at my Movies share settings and I had "use cache disk" set to yes. I just changed it to no, removed a trailer and rescraped and downloaded it and this time the trailer was downloaded in /mnt/disk3/Movies/Sucker-Punch

 

so should I just let it download to cache disk and then use the mover script when done, or keep it disabled. I have all my downloads to a ".downloads" directory so the mover script doesnt touch them anyways

Link to comment

The cache drive is designed to operate entirely transparently.  I suppose you could use the mover script but the intended operation is to schedule the mover to occur once per night (the default setting, it runs at 3:40am) or once per week, and not have to think about it at all.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.