Hardlinks/Atomic Moves - Multiple Shares

[Indi]

Please don't scald me, but I am looking to see if anyone has advice if I may be thinking unclear about solving some issues.

Quick recap - I had my media all under data/media/movies etc, but after my library grew larger I had issues with: 
1. Managing split level was a pain with anime/movies/tv as they all needed a different split level. 

2. I wanted all of the same shows, the same anime to be on a single drive. I didn't use split level at all before, so I had .nfo files and random episodes, all split across 3-4 drives and it is inefficient. 

3. I like having my array organized. Disks 1-4 are for TV, Disk 4 is for Anime, 5-7 are for Movies. 

Now, I have 3 shares: 

 

anime/animefolder/seasonfolder/episodes

tv/showfolder/seasonfolder/episodes

movies/moviefolder/movies

 

Everything is working fine, but I am now noticing the benefits of hardlinks and atomic moves. I have a 4TB cache drive, where mover tuner is set to move files to the array after 15 days so all new files are accessed from the cache. 

However, when something is downloaded, it's incredibly inefficient. The process is now: 

 

Download (Write Process to downloads share)
Unpack 
Import (Read Process from downloads share)

Import (Write Process to tv share)
Mover after 15 days (Read from cache, write to array)

So for a download, it will read/write to itself multiple times and a 50GB file for example, can write 150+GB to the same drive. 

Now that I've sorta explained my setup, to solve this I am thinking of doing the following. 

Change shares to be something like anime/media/animefolders/animeseasons/episodes
Then under "media" create another directory for downloads. 

 

Next, setup 2 instances of Sonarr, 1 instance of Radarr. Sonarr1 for TV, Sonarr2 for Anime, Radarr for Movies. 
Since SabZNBD looks at a parent download directory, categories wouldn't work I don't think. But maybe I can map each share in Sab to point to the correct media, if not, I will setup 3 Sab containers to point to each arr instance. 

 

With the above, I should be able to restore hardlinks for each appropriate share, right? Unless I am missing something. 

 

If anyone might have a different way of doing this, I'd love to know. I understand now the performance benefits of a single share, but my OCD will no longer allow that and I am much happier being able to properly manage media across disks and maintain split level for each media type. 

Edited March 16, 2023 by Indi
spelling mistake

[BRiT]

1 hour ago, Indi said:

Download (Write Process to downloads share)
Unpack 
Import (Read Process from downloads share)

Import (Write Process to tv share)
Mover after 15 days (Read from cache, write to array)

So for a download, it will read/write to itself multiple times and a 50GB file for example, can write 150+GB to the same drive. 

Nope. Moving files on the cache pool does not require reading and writing the entire file, it is couple of simple entry changes in the directory tree. If you use something like /mnt/cache/downloads to /mnt/cache/TV.

 

May also be fine is you use /mnt/user/downloads and /mnt/user/TV too. I don't ever use /mnt/user/ for my download workflows, so not certain on that aspect.

 

 

Edited March 16, 2023 by BRiT

[Indi]

25 minutes ago, BRiT said:

Nope. Moving files on the cache pool does not require reading and writing the entire file, it is couple of simple entry changes in the directory tree. If you use something like /mnt/cache/downloads to /mnt/cache/TV.

 

May also be fine is you use /mnt/user/downloads and /mnt/user/TV too. I don't ever use /mnt/user/ for my download workflows, so not certain on that aspect.

 

 

Thank you! I guess I was misunderstanding the dashboard, in that during an import operation from Radarr or Sonarr etc, the cache shows Reads of 450Mbps and Write 450Mbps as if it was reading from /mnt/cache/downloads and then writing back to itself to /mnt/cache/TV

 

If that isn't the case, I might be okay. My SSD has a 2400TBW warranty, so my only thought was to help prevent unnecessary writes, if that were indeed happening 

[JonathanM]

11 hours ago, Indi said:

during an import operation from Radarr or Sonarr etc, the cache shows Reads of 450Mbps and Write 450Mbps

Whether a copy / erase or a rename is used by a move action depends on how the application views the file systems. If it sees the two locations as being part of the same file system it will rename, if it thinks there are two different file systems it will copy / delete. From the *arr's viewpoint, do the two paths share a common root? Or is it two root folders, /downloads and /TV?

[Indi]

3 hours ago, JonathanM said:

Whether a copy / erase or a rename is used by a move action depends on how the application views the file systems. If it sees the two locations as being part of the same file system it will rename, if it thinks there are two different file systems it will copy / delete. From the *arr's viewpoint, do the two paths share a common root? Or is it two root folders, /downloads and /TV?

So the *arr's viewpoint, they are two root folders. For example

Sonarr Root Folders
/tv

/anime
Downloads has no remote path mapping configured 


Then SabNZBD 
Completed Downloads Folder - /downloads


I notice now that /downloads for Sonarr is /mnt/cache/downloads while Sab is /mnt/user/downloads, could this be my issue? 

Appreciate the reply! 

[JonathanM]

9 minutes ago, Indi said:

So the *arr's viewpoint, they are two root folders.

That explains why you are getting a copy / delete.

10 minutes ago, Indi said:

I notice now that /downloads for Sonarr is /mnt/cache/downloads while Sab is /mnt/user/downloads, could this be my issue? 

Not directly, but that could cause some strange behaviour if the downloads share is set to cache:yes

 

Before you start changing things around I think you need to get a deeper understanding of user shares and how they interact with the disk paths. hard links cannot be valid across the two types of paths.

[Indi]

8 minutes ago, JonathanM said:

That explains why you are getting a copy / delete.

Not directly, but that could cause some strange behaviour if the downloads share is set to cache:yes

 

Before you start changing things around I think you need to get a deeper understanding of user shares and how they interact with the disk paths. hard links cannot be valid across the two types of paths.

Gotcha, thank you! I will see if I can find more resources on this. I thought I had an understanding of how it worked based off YT videos I've seen and other forum posts, where if they share a root path, hardlinks will work (which is where my idea came in). 

I appreciate the response, seems like I have a lot of research to do