DZMM Posted July 10, 2018 Share Posted July 10, 2018 Let's say I have two 8TB disks, disk 1 is 90% full and the other is empty. If I move and rename a file stored on disk 1 stored at /mnt/user/share1/file.txt to /mnt/user/share2/new name.txt, will it move it quickly on disk 1 between shares and ignore the high water rule, or will it move and copy (slower) to disk 2, obeying the high water rule? I'm getting into bottlenecks with renaming and moving downloads, and I'm wondering if my setup where I keep imports, movie_adults, movie_kids, tv_adults etc on different disks is causing problems i.e. a new TV show is currently going /mnt/disk1/imports/show1.mkv to /mnt/disk2/TV shows/show_renamed.mkv which takes time to copy across, whereas mnt/disk1/imports/show1.mkv to /mnt/disk1/TV shows/show_renamed.mkv should be faster if unraid always moves a file on the same disk even if another disk has more space. Thanks Quote Link to comment
Squid Posted July 11, 2018 Share Posted July 11, 2018 2 hours ago, DZMM said: If I move and rename a file stored on disk 1 stored at /mnt/user/share1/file.txt to /mnt/user/share2/new name.txt, will it move it quickly on disk 1 between shares and ignore the high water rule, or will it move and copy (slower) to disk 2, obeying the high water rule? Depends how you do it. If you move it via the network, then it will obey high water. If you do it via command line, mc, krusader etc then it won't Quote Link to comment
DZMM Posted July 11, 2018 Author Share Posted July 11, 2018 Thanks. What about dockers? I guess if a disk is selected to extract a file from nzbget based on HW, then when it's time to move and rename that file that disk will probably still win the HW selection a few mins later. Quote Link to comment
Squid Posted July 11, 2018 Share Posted July 11, 2018 It'll honor it as you're going through using 2 different mount points in the container. /Downloads and /moviesSent via Tapatalk because I'm either at work or enjoying the summer Quote Link to comment
DZMM Posted July 11, 2018 Author Share Posted July 11, 2018 Thanks. I'm hoping the changes I've made to not exclude disks from shares, or at least matching them up, and also to turn hardlinks on so hopefully less copying, will improve my import speeds as my backlog is approaching a couple of days which is annoying when you know something is there, but plex can't see it yet. Quote Link to comment
SSD Posted July 11, 2018 Share Posted July 11, 2018 This might help to understand when the allocation method and include/exclude settings come into play. Quote Link to comment
DZMM Posted July 11, 2018 Author Share Posted July 11, 2018 Thanks. I think I've got my flows setup the way I want and optimised: nzbget: downloads to /mnt/user/downloads on disk 4 - /mnt/downloads/ set to disks 4 only extracts to /mnt/user/import on disks 1-7 based on High Water choosing a disk. 99% of the time this shouldn't be on disk 4 as this has the least amount of space, so extraction should be faster as not occurring on same disk (I'm tempted to do downloads to an UD to ensure this is always the case) sonarr/radarr/lidarr/lazylibrarian move files to respective media shares. HW rules should mean this is on the same disk that was chosen for the import, so just a rename and not a copy so fast Deluge: downloads direct to /mnt/user/import on disks 1-7 sonarr/radarr/lidarr/lazylibrarian copys/hardlinks files to respective media shares. When hardlinks works should be fast as no copy The bit I'm not sure about yet, is for some of my files the media share is a unionfs mount e.g. for kids_movies it's /mnt/user/movies_kids_upload=RW:/mnt/disks/rclone_vfs/movies_kids_gd=RO /mnt/disks/unionfs_movies_kids unionfs -o cow,allow_other,direct_io,auto_cache,sync_read so sonarr/radarr etc are moving the file to /mnt/disks/unionfs_movies_kids i.e. not a /mnt/user location, although in reality the file behind the scenes is being written to /mnt/user/movies_kids_upload (rclone move job then moves it to google drive mounted at /mnt/disks/rclone_vfs/......). Do the dockers/unraid know this and still follow the HW rule i.e. the files are just renamed rather than copied? I'm assuming hardlinks won't work as the docker will think /mnt/disks/ is on a different filesystem? Quote Link to comment
Squid Posted July 11, 2018 Share Posted July 11, 2018 (edited) 12 minutes ago, DZMM said: Do the dockers/unraid know this and still follow the HW rule i.e. the files are just renamed rather than copied? I'm assuming hardlinks won't work as the docker will think /mnt/disks/ is on a different filesystem? Rather than spend forever trying to understand your workflow and inferring the host/container mappings, here's the rules in a nutshell If you passthrough to the containers 2 mount points, one for downloads, and one for the final destination then: Highwater / include / exclude rules will be honored Hardlinks between downloads and final destinations will not work (because the mountpoints between the source and destination are different) If you passthrough to the containers a single mount point, and reference downloads and the final destination from it then, (assuming everything in the container winds up referencing /mnt/user/) Highwater / include / exclude rules will not be honored (files on the final destination will wind up being on the same disk that the download happened on) Hardlinks between downloads and final destinations will work My thoughts on hardlinks between a rclone mount and /mnt/user would be that even if you passthrough /mnt mapped to /mnt then it will not work at all as ultimately /mnt/disks and /mnt/user are different filesystems. Edited July 11, 2018 by Squid 1 Quote Link to comment
DZMM Posted July 11, 2018 Author Share Posted July 11, 2018 Thanks that helped. tweaking slightly to get desired result of nzb extraction onto different disk to the acutal download then a hardlink to the media folder, and torrents hardlinking for all, except for /mnt/disk rclone moves: nzbget: /downloads mount point --> /mnt/user/import ---> /mnt/user/media deluge: /mnt/user/import ---> /mnt/user/media Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.