Can UNRAID only cache small files in folders?

[imsoz0]

As the title, I recently migrated the Jellyfin library to unraid. After using it for a period of time, I found that when the webpage accesses Jellyfin, the poster will take a few seconds to load. When new movies and TV series are searched and sorted by tiny media manager, the speed is also obviously slowed down (not the problem of the host). It wasn't so slow when I used DSM RAID10 array before. So I considered moving small files such as posters, subtitle files, and nfo files to the cache pool, and large files such as video files that start at 1G will continue to be stored in the array. But I didn't find related options in folder cache settings.

If I choose “Prefer”, will both small files and video files be moved to the cache? My cache SSD is only 500 GB, which is completely insufficient if the video files will also be moved.

The caching scheme I want is, for example, in the movie folder, files smaller than 20 MB are put into the cache pool so that I can access them faster when I need to modify them (usually posters, subtitles, nfo files). Files exceeding this size are kept on the array (usually video, audio files). Is there a way to satisfy this requirement?

 

unRAID 6.11.5

[trurl]

See if mover tuning plugin will work for you

[imsoz0]

5 minutes ago, trurl said:

See if mover tuning plugin will work for you

I'm using the tuning plugin to help me manage the cache, and I've also seen that files that exceed the size can be moved out of the cache by setting the size. The “Prefer” option as I understand it is to move files from the array to the cache as long as the cache has capacity available. What I'm curious about is if in the movies folder cache is set to “Prefer”, and in the tuning plugin set the files over 20 MB are moved out of cache. Does such a setup cause video and audio files to be frequently moved back and forth between array and cache?

[Kilrah]

The standard way of doing this is to NOT have Jellyfin set to store posters etc in the media folders, then you have them in the appdata on cache and only the actual media on the array.

Edited January 24, 2023 by Kilrah

[imsoz0]

18 minutes ago, Kilrah said:

The standard way of doing this is to NOT have Jellyfin set to store posters etc in the media folders, then you have all the appdata on cache and only the actual media on the array.

I usually manage my movies and TV shows through TinyMediaManager on windows, and all posters and nfo files are already in the corresponding media folders when the arrangement is completed.
Now I have nearly 1,000 movies and TV series, and there may be tens of thousands of various small files. I don't know how to quickly transfer all but these files to appdata, nor how to set it up in Jellyfin. If all these operations need to be done manually, it will be a huge project.

[trurl]

6 hours ago, imsoz0 said:

moved back and forth

No way to move back and forth. Prefer moves to cache, Yes moves to array. The only way to make it move the other way is to change that setting.

[imsoz0]

6 hours ago, trurl said:

No way to move back and forth. Prefer moves to cache, Yes moves to array. The only way to make it move the other way is to change that setting.

Under the “Prefer” setting, both large file A and small file B in the folder will be moved to the cache. And “Mover Tuning” is set to move out of the cache to the array if the size is exceeded. Then the steps to move the file should be as follows.

 

Step1. The large and small files in the folder are moved to the cache.
Step2. “Mover Tuning” moves the large file A that exceeds the size to the array.
Step3. The system finds that there is still free space in the cache, and moves the large file A to the cache again.
Step4. “Mover Tuning” finds files A that exceed the size, so they are moved to the array.

 

My concern is that step3 and step4 steps will keep happening. The logic of “Prefer” is to continue to move file A to the cache as long as there is still available cache space. Also, “Mover Tuning” only works after file A has been written to the cache.

 

If “Prefer” always takes precedence over “Mover Tuning”, then the cycle of step3 and step4 seems to go on forever.

[itimpi]

Unless something has changed very recently then your scenario cannot happen!   I do not believe that Mover Tuning over-rides the Use Cache setting so it only makes decisions on whether to move a file in the direction designated  by the Use Cache setting and never moves them back and forth.

[imsoz0]

6 minutes ago, itimpi said:

Unless something has changed very recently then your scenario cannot happen!   I do not believe that Mover Tuning over-rides the Use Cache setting so it only makes decisions on whether to move a file in the direction designated  by the Use Cache setting and never moves them back and forth.

So when Step2 is done, neither Step3 nor Step4 will happen, right? I'll give it a try and hopefully everything works as I envision it.

[Kilrah]

No, only step 1 would happen, and supposing you set mover tuning to only move large files then it would do the opposite of what you want by moving large files to cache and leaving small ones on the array. 

Mover tuning is not a "second mover", it's just a filter on what gets moved, when the mover runs. That will always be in the direction that is specified in the share.

Edited January 25, 2023 by Kilrah

[imsoz0]

I found out that I made a mistake, I should have set the shared folder to "YES" from the beginning.

 

I tried it according to the method mentioned in this post, and set the ignore file type in Mover Tuning to: Then set the shared folder to “Prefer”, tried it and found that the video file was not filtered, and it was also moved to the cache. After looking at the contents of the files in the cache, I found that I just copied the files on the array to the cache again.

I originally wanted to filter out all other non-video files directly in Krusader and copy them to the cache. But I don't know how to create the corresponding folder hierarchy by the way when the files in the filtered results are copied to the cache. So by manually copying the folder to the cache, every time a part is copied, all the video files are deleted from the cache, and then the next batch of files is copied. (It’s a stupid way, and I don’t know if there is any more labor-saving way)

But after all, it can be used, and Jellyfin's posters are loaded much faster, and the problem of needing to refresh the page if the posters cannot be loaded is also solved.

[Grobalt]

Is there a new way, easier, in the meantime? I have like 2000 movies l, managing with TMM and all small files are in the movies folder. Would be nice to have a fast Kodi etc that can access the small files on SSD instead spinning the harddrives

[imsoz0]

10 hours ago, Grobalt said:

Is there a new way, easier, in the meantime? I have like 2000 movies l, managing with TMM and all small files are in the movies folder. Would be nice to have a fast Kodi etc that can access the small files on SSD instead spinning the harddrives

I'm not sure if there's a more convenient way. As you can see, I used a particularly clumsy method, essentially simulating an external data import process. I think the most troublesome part isn't selecting the smaller files, but how to move them while still retaining their file hierarchy structure.

[imsoz0]

On 10/26/2023 at 7:49 AM, Grobalt said:

Is there a new way, easier, in the meantime? I have like 2000 movies l, managing with TMM and all small files are in the movies folder. Would be nice to have a fast Kodi etc that can access the small files on SSD instead spinning the harddrives

Hi, I think I've found a better method. You can directly use the "rsync" command in the terminal, which is very handy.

 

Here's a tutorial from the Chinese post. You can translate it while reading or find an English tutorial. I hope this can help you.