I am new to Unraid, loving it so far, but have quite some trouble now figuring out the best way of caching my media downloads.

 

Share for media downloads (NZBGet) is created from the WebUI, called media-dl.

Use cache pool: Yes, so i want to have the mover handling those files on a daily basis.

 

NZBGet uses a folder for intermediate or complete downloads, therefore I set up the following folder structure yesterday:

- mnt/user/media-dl/

-- .nzbget/

-- movies/

-- shows/

 

thinking that Mover will not move directories which begin with a '.' character, but:

"Mover will not move any top-level directories which begin with a '.' character"

 

In terms of mounting volumes to my docker container I am not quite sure how to handle this:

 

• If I create a directory /mnt/cache_ssd/.nzbget/ it is linked to /mnt/user/.nzbget/
• touch /mnt/cache_ssd/.nzbget/test.txt creates the file visible in /mnt/user/.nzbget/ too, since it is essentially a hidden share (? Can't see it in the shares section in WebUI)
• touch /mnt/user/.nzbget/test2.txt creates the file only visible in /mnt/user/.nzbget/, not in /mnt/cache_ssd/.nzbget/

 

I believe this is wanted behaviour, because one would need to set 'prefer' or 'Yes' to the share ".nzbget" to actually store a file created in /mnt/user/.nzbget to the cache_ssd pool.

 

So, this is where I'm not sure what is the most performant way to solve this in terms of mounting my docker volumes for NZBGet:

• I need to have /mnt/user/media-dl/ mounted; everything in media-dl/ is not top-level anymore and is moved, like right now, media-dl/.nzbget/ is moved;
• -> I need to mount /mnt/user/media-dl/ to have NZBGet categories (or sonarr/radarr) moving it to the appropriate folders (e.g. shows/rome/S01/...)
• I need to put .nzbget/ to /mnt/cache_ssd/.nzbget/ to prevent the mover seeing it; /mnt/user/.nzbget/ is created, but I can't mount this folder, since it is stored anywhere, but not on the cache_ssd/, where I want the files to be extracted and processed;
• -> I need to specifically mount /mnt/cache_ssd/.nzbget/ and configure NZBGet accordingly to have tmp, complete and intermediate files there, not seen by the mover.

 

Then it looks like this (remember: media-dl is a cache_ssd 'Yes' enabled share):

- mnt/cache_ssd/.nzbget -> mounted to NZBGet docker /nzbget/

- mnt/user/.nzbget          -> link directory; essentially a 'hidden share'; this path cannot be used, since it is not configurable in the shares WebUI

- mnt/user/media-dl/        -> mounted to NZBGet docker /media-dl/

-- movies/

-- shows/

 

But here, i have some questions: If I mount it this way, even though both volumes mounted reside on the same physical ssd drive, pool and filesystem (cache_ssd), what would happen if the files are moved from /mnt/cache_ssd/.nzbget/intermediate/ to /mnt/user/media-dl/movies/HD/ ?

 

Does the system actually unneccesarily move the file and not hardlink (or whatever it is called in xfs?) the file in the filesystem?

If yes, what is the better solution here?

Edited January 23, 20233 yr by theuema
typo zfs vs. xfs

After reading this TRaSH Guide (again), I am not very positive that there is a way to achieve a useful location for .nzbget/ that avoids atomic moves other than directly in my media-dl/ (cache_ssd 'Yes' enabled share), like this:

- mnt/user/media-dl/

-- .nzbget/

---- intermediate/

---- complete/

-- movies/

---- HD.movies/

-- shows/

---- TV.shows/

 

So, I am considering changing the mover script the following way and I hope not to break important stuff here^^

# Check for objects to move from pools to array
  for POOL in /boot/config/pools/*.cfg ; do
    for SHAREPATH in /mnt/$(basename "$POOL" .cfg)/*/ ; do
      SHARE=$(basename "$SHAREPATH")
      if grep -qs 'shareUseCache="yes"' "/boot/config/shares/${SHARE}.cfg" ; then
        find "${SHAREPATH%/}" -not \( -path "*.nzbget/*" -prune \) -not \( -path "*.nzbget" -prune \) -depth | /usr/local/sbin/move -d $LOGLEVEL
      fi
    done
  done

 

From testing the initial find command i got the following:

root@UNRAID:/mnt/user# find media-dl/ -depth
media-dl/.nzbget/test.txt
media-dl/.nzbget
media-dl/movies/HD.movies/Movie.1.mkv
media-dl/movies/HD.movies
media-dl/movies
media-dl/shows/TV.shows/Show1/S01/Show1.S01E01.mkv
media-dl/shows/TV.shows/
media-dl/shows
media-dl/.DS_Store
media-dl/._.DS_Store
media-dl/

 

Using the new command gives this:

media-dl/movies/HD.movies/Movie.1.mkv
media-dl/movies/HD.movies
media-dl/movies
media-dl/shows/TV.shows/Show1/S01/Show1.S01E01.mkv
media-dl/shows/TV.shows/
media-dl/shows
media-dl/.DS_Store
media-dl/._.DS_Store
media-dl/

 

 

I am not planning to do anything on the "# Check for objects to move from array to pools"-find command some lines below, I hope I don't forget anything here.

 

Please let me know what you think about this solution.

Edited January 23, 20233 yr by theuema