Is it possible to move just some file types to cache?

[TheFreemancer]

I have an emby folder with TV Shows, Movies, and to speed things up, specially from my rclone mount point, I decided to use jpg and nfo files directly into the media folder. 

This is way faster when using Kodi without EmbyCon, which is the way I prefer to access my library.

 

What I would like to know is, is it possible to have a copy, that would be accessed first of all my .jpg and .nfo files into my ssd cache drive?

I believe there's a plugin that can move things around from a share to another hard drive which I don't remember the name now.

Not sure if that would do what I want, not sure even if it`s possible.

 

 

[Squid]

3 hours ago, TheFreemancer said:

I believe there's a plugin that can move things around from a share to another hard drive which I don't remember the name now.

unbalance

[TheFreemancer]

1 hour ago, Squid said:

unbalance

That's it but it doesn't actually do what I want. At least no automatically.

 

1. So what I would need to do would be, enable cache for the Emby share to "prefer".

This will create an Emby folder into the Cache disk.

 

2. move *.jpg and *.nfo from /mnt/user/Emby to /mnt/Cache/Emby

 

And would it work?

 

Also I have one more question, if I use a cache with my Emby share, and my cache is 240gb in size and I copy a 500gb batch of files, when it starts to get full will it start moving the files to the array while still copying to the cache? Or I would need to do it in parts of 240gb?

[BRiT]

Entire thread on this, Accelerator Drives.

 

 

[TheFreemancer]

1 hour ago, BRiT said:

Entire thread on this, Accelerator Drives.

 

 

 

I don't understand this thread last post is from 2016 and someone asking exactly the same question that I'm asking now in 2019.

 

Does it mean it's possible? does it mean it's not implemented yet?

[itimpi]

7 minutes ago, TheFreemancer said:

 

I don't understand this thread last post is from 2016 and someone asking exactly the same question that I'm asking now in 2019.

 

Does it mean it's possible? does it mean it's not implemented yet?

If you have a share set to ‘Use Cache’ = No then Unraid will see the files on the cache for read purposes but will never write new files there or try to move them back to the array.    However there is no automated way to get the required files onto the cache - the user has to do this themselves.

[TheFreemancer]

1 minute ago, itimpi said:

If you have a share set to ‘Use Cache’ = No then Unraid will see the files on the cache for read purposes but will never write new files there or try to move them back to the array.    However there is no automated way to get the required files onto the cache - the user has to do this themselves.

Ok this sounds promissing!

 

I don't mind that this is a manual job since I don`t add stuff all the time. 

 

Just, please, help me figure this out before I break something on the array.

 

All the .jpg and .nfo files are in folders inside /mnt/user/Emby/TV\ Shows/

So I need to manually create a folder inside /mnt/Cache named Emby.

 

Now I know in bash how to move stuff with mv *.jpg *.nfo /mnt/cache

 

But the problem of running this command is that it will put all the nfo and jpg files together. 

There are tons of folder.jpg, so I would need to figure out how to keep the folder structure and that's where my Bash terminal skills end. 

 

How do you do that?

And also, if I keep Use Cache = No, but keep the files in the cache with the same name and the same folder structure, unRaid will use those files without accessing the ones inside the array?

[itimpi]

You have to create the folder structure to exactly match the folder structure on the disks - but I am not sure how one does that.

 

if a file exists on both cache and array the cache copy takes precedence.    This can sometimes be confusing as if you try and delete the file it appears the delete has not worked as once the cache copy disappears the copy on the unraid then takes over.    Whether you still keep a copy on the array us up to you.

[TheFreemancer]

Ok, found out how to copy things keeping the folder structure.

 

$ mkdir /mnt/cache/Emby

$ cd /mnt/user/Emby

$ find . -name '*.jpg' | cpio -pdm /mnt/cache/Emby

-updm if you want to overwrite. 

 

now I can see the folder structure is exactly the same as the other drive. 

If unRaid even with 'Use Cache' = No see the files, I'm really really happy. 

 

Wish there was a way to test this besides waiting all the drives to spin down and access the library from a remote Kodi instance.

[itimpi]

14 minutes ago, TheFreemancer said:

Ok, found out how to copy things keeping the folder structure.

 

$ mkdir /mnt/cache/Emby

$ cd /mnt/user/Emby

$ find . -name '*.jpg' | cpio -pdm /mnt/cache/Emby

-updm if you want to overwrite. 

 

now I can see the folder structure is exactly the same as the other drive. 

If unRaid even with 'Use Cache' = No see the files, I'm really really happy. 

 

Wish there was a way to test this besides waiting all the drives to spin down and access the library from a remote Kodi instance.

You could have two files with the same name but different content - one on the cache and one on the array.    If you now look at the file via the User Share you can check which version you see.

[TheFreemancer]

Good idea!

 

But something really weird happened.

 

So I created a test.txt file in /mnt/user/Emby/TV\ Shows/test.txt

wrote 1234 into test.txt

$ cp /mnt/user/Emby/TV\ Shows/test.txt /mnt/cache/Emby/TV\ Shows/test.xt

so now I edit the original test.txt in /mnt/user/Emby/TV\ Shows/test.txt add 12345678

 

And when I look into the file inside the cache the modification is the same.

 

It seems that the files are linked? That's kind of fantastic and weird at the same time. 

[remotevisitor]

If you are going to do this then you should use /mnt/user0/Emby rather than /mnt/user/Emby otherwise you will almost certainly end up with a load of zero length files if your are very careful/lucky.

 

if any of those files you copied to the cache drive were already on the cache drive then they are now probably zero length.

Edited May 19, 2019 by remotevisitor

[TheFreemancer]

The files were not on the cache drive before and they seem fine. 

 

So if I want to overwrite or copy new files I should use the user0 folder? 

I'll keep that in mind because I intend to make a small 2 line script that I run once in a while to copy all nfo and jpg files to the cache drive.

[itimpi]

The files you see under /mnt/user are the amalgamation of the files on the disks and the cache so what you describe is what is expected.   If you want cache to be omitted then use /mnt/user0 as that is restricted to array disks.   That behavior is the whole point of this exercise     It is also the reason you are told to never use both disk and user shares in the same copy command.

[TheFreemancer]

Thank you for teaching me this. I'm on the begining of my learning curve on linux and unraid. 

 

Just to see if I understand, if Emby server changes something in a nfo or jpg file that where copied to cache, this same change will reflect into the nfo or jpg file on cache, correct?

 

Also, I don't need to impose a overwrite command since the files are an amalgamation of both. 

So if new nfo and jpg files get added, all that is needed is to perform the copy command again and those will have a first copy on cache and remain on the array. 

 

if what I think is correct, this is even better than what I had imagined! 

[TheFreemancer]

I need some further information about this.

 

So the first time I ran the cp command everything went fine.

Now I ran a test with that test.txt file and the file is 0 bytes just like @remotevisitor said.

 

So I did a test using the user0 folder and copied it to cache and edited the file.

I ended up with 2 different files after the edit. 

 

So what are my options? 

I see this in three ways, please correct me if I'm wrong

 

1. I can delete *.nfo and *.jpg from cache and copy everything every time I need to copy a new .nfo and .jpg file. 

 

2. I can run the cp command from cache to the array so all the previews files match and after that run from the array to the cache.

 

3. run a cp command and try to find a syntax or something that would skip matching files.

 

 

Any idea what should I do here?

 

 

EDIT: Still doing some tests here, it seems the files on user0 will still be different because those are in the array.

So, if .nfo and .jpg files get updated and new files are added, the solution would be this sequence:

- copy *.jpg and *.nfo from cache to user0 and overwrite, this will place any modifications into the array.

- delete the entire Emby folder from cache. 

- copy *.jpg and *.nfo back into the cache from the regular /mnt/user/ folder or user0 it won't matter since changes are not applied into the array without a cp command. 

 

EDIT 2: OK, found out that cpio command is smarter than I imagined.

If I try to run the same command again

$ find . -name '*.jpg' | cpio -pdm /mnt/cache/Emby

It will not change any existing files :))))

"************.jpg not created: newer or same age version exists"

 

So I believe it might be safe to run the command every time a new .nfo or new .jpg shows up because it will not mess up with previews files resulting in 0 bytes files.

Edited May 21, 2019 by TheFreemancer