July 17, 201510 yr This proposal is similar as http://lime-technology.com/forum/index.php?topic=34434.0 I want my array drive only to spin up when I want to watch a movie, not when i am browsing my library. To accomplish this I want to use my (large) cache drive. I also would like that browsing my library is as smooth as possible. Thus, waiting for a drive to spin up is not good. This is basically also a replacement for the cache_dirs script. What I propose is the following: I have a share called BR-Movies, spanning my whole array. It contains dirs, each dir contains a movie, so the share looks like: BR-Movie (= located on array) ----------Movie1 -----------------movie.nfo -----------------Movie1.iso -----------------poster.jpg -----------------fanart.jpg ----------Movie2 -----------------movie.nfo -----------------Movie2.iso -----------------poster.jpg -----------------fanart.jpg .... I would like to go to the following file layout: .BR-Movie (= located on cache disc) ----------Movie1 -----------------movie.nfo -----------------poster.jpg -----------------fanart.jpg ----------Movie2 -----------------movie.nfo -----------------poster.jpg -----------------fanart.jpg ... BR-Movie (= located on array) ----------Movie1 -----------------Movie1.iso ----------Movie2 -----------------Movie2.iso .... How to implement?: =============== This would work independent from caching a share, in other words, you could have: - A share that is cached - A share that is "accelerated" (this proposel) - A share that is cached AND accelerated (Big ??) disadvantage: the ".share_name" dir is not included in the protected array, in other words, if you lose the cache drive, then you lose the *.jpg and *.nfo files. User needs to do a manual backup of this ".share_name" directory (or use a cache pool?) Deleting files from the ".share_name" dir would work the same as now, meaning if you delete the "Movie1" dir THROUGH THE USER-SHARE, then the whole dir would be deleted, otherwise the user would have to go first to the cachedrive to delete those files, and then to "Diskx/..." to delete the iso file. Advanced users could make a script that would backup the ".share_name" dir to an external USB drive/dir on the protected array/... **************************************************** First implementation (basic implementation, for advanced users): ------------------------------------------------------------------ Let the user create a ".share_name" directory manually on the cache drive. Unraid: Check if a ".share_name" directory exists on the cache drive, if it does, include it in the share The user copies manually files into this ".share_name" directory **************************************************** Second implementation: -------------------------- As first implementation + Unraid: Add a setting on the "Share Settings" page: Accelerate Share : Yes/No Selecting yes would autocreate the ".share_name" dir on the cache drive The user would still need to manually copy files to the ".share_name" dir. A help text could explain this feature on the "Share Settings" page, more users would be aware of the feature. Explain in BOLD letters that the files on the cache drive are not protected! **************************************************** Third implementation: -------------------------- As second implementation + Unraid: Add a setting on the "Share Settings" page: Move following files automattically to the cache drive: Text box where the user would fill in : "*.nfo, *.jpg" Unraid: Mover script needed to move these files from the array to the cache drive (cron job (daily/weekly???)) **************************************************** Fourth implementation: -------------------------- As third implementation + Unraid: expand the power of the mover script so the user could do: "*.* < 110KB, *.jpg, *.nfo" **************************************************** First (or second ) implementation would for me be enough, third implementation would be very handy, fourth implementatin is for me the cherry on top of the cake... Comments please...
July 17, 201510 yr Sounds like you should look at dm-cache or a flashcache derived project like EnhancedIO.
July 17, 201510 yr Some thoughts - * You mention this is "a replacement for the cache_dirs script", but I don't see any connection with what CacheDirs does. CacheDirs does not cache files, and has nothing to do with the Cache drive. * I may not be fully understanding the whole concept or its implementation, but I *think* that much of what you are asking for may already be there, and if you add one tweak to the Mover script, it would automatically do the rest. If a Share is set to use the Cache drive, then the folders are automatically set up correctly when you copy a set of movie files to the share. All of them initially go to the Cache drive. Then if you add a --min-size=100000 parameter to the rsync command in the Mover script, it will only move the largest files to the array, to folders that mirror the Cache drive. I think that does what you want, keeps the small files on the Cache drive and the large ones on the array, but shows you a single folder with all of the files. * To set it up, you would do the Mover script in reverse, with a modified version of the Mover that had a --max-size=100000 parameter in the rsync command. That should mirror the share folders on the Cache drive, but only move the smaller files. Then just stop and restart the array. * If your Cache drive/pool is one or more SSDs, then this keeps all of your spinners from spinning, except to play a movie, and this may mean that you don't need the CacheDirs script. What have I missed?
July 17, 201510 yr Kinda getting confused what the key difference between this and my accelerator drive proposal is?
July 17, 201510 yr Author Kinda getting confused what the key difference between this and my accelerator drive proposal is? Basic difference is that in your proposal the *.jpg and other files are PROTECTED on the array (if I read that thread correctly). This makes the implementation more difficult (and maybe not happening because of that). It could also mean that we go from 24 data drives to 23 data drives and a accelerator SSD, depending on how your proposal would be implement (max config for a pro license).
July 17, 201510 yr Author Some thoughts - * You mention this is "a replacement for the cache_dirs script", but I don't see any connection with what CacheDirs does. CacheDirs does not cache files, and has nothing to do with the Cache drive. Cachedirs caches the file/directory structure in memory, so that if you traverse the your hd's, no new data (of the file/ dirtectory structure) would have to be read from the HD (in other words, the drives don't have to spin up while you are browsing your hd) * I may not be fully understanding the whole concept or its implementation, but I *think* that much of what you are asking for may already be there, and if you add one tweak to the Mover script, it would automatically do the rest. If a Share is set to use the Cache drive, then the folders are automatically set up correctly when you copy a set of movie files to the share. All of them initially go to the Cache drive. Then if you add a --min-size=100000 parameter to the rsync command in the Mover script, it will only move the largest files to the array, to folders that mirror the Cache drive. I think that does what you want, keeps the small files on the Cache drive and the large ones on the array, but shows you a single folder with all of the files. * To set it up, you would do the Mover script in reverse, with a modified version of the Mover that had a --max-size=100000 parameter in the rsync command. That should mirror the share folders on the Cache drive, but only move the smaller files. Then just stop and restart the array. * If your Cache drive/pool is one or more SSDs, then this keeps all of your spinners from spinning, except to play a movie, and this may mean that you don't need the CacheDirs script. What have I missed? This would do the same, correct.
July 18, 201510 yr Kinda getting confused what the key difference between this and my accelerator drive proposal is? Basic difference is that in your proposal the *.jpg and other files are PROTECTED on the array (if I read that thread correctly). ... Accelerator drives are protected as they are on the array... hence my confusion
Archived
This topic is now archived and is closed to further replies.