Jump to content

DZMM

Members
  • Posts

    2,801
  • Joined

  • Last visited

  • Days Won

    9

Everything posted by DZMM

  1. Have you tried it without the address bit and just --rc? My firewall setup is a bit complicated, so I'm not sure if other users need to add the address
  2. DZMM

    Plexdrive

    Rather than continuing to discuss rclone mounts in a plexdrive thread, I've started a new rclone thread
  3. Unionfs works 'ok' but it's a bit clunky as per the scripts above. Rclone are working on their own union which would hopefully include hardlink support unlike unionfs. It possibly will also remove the need for a seperate rclone move script, automating transfers from the local drive to the cloud https://forum.rclone.org/t/advantage-of-new-union-remote/7049/1
  4. Key elements of my rclone mount command: rclone mount \ --allow-other \ --buffer-size 256M \ --dir-cache-time 720h \ --drive-chunk-size 512M \ --log-level INFO \ --vfs-read-chunk-size 128M \ --vfs-read-chunk-size-limit off \ --vfs-cache-mode writes \ --bind=$RCloneMountIP $RcloneRemoteName: \ $RcloneMountLocation & --buffer-size: determines the amount of memory, that will be used to buffer data in advance. I think this is per stream --dir-cache-time: sets how long a directory should be considered up to date and not refreshed from the backend. Changes made locally in the mount may appear immediately or invalidate the cache, so if you upload via rclone you can set this to a very high number. If you make changes direct to the remote they won't be picked up until the cache expires --drive-chunk-size: for files uploaded via the mount NOT via the upload script i.e. if you add files directly to /mnt/user/mount_rclone/yourremote. I rarely do this and it's not a great idea --vfs-read-chunk-size: this is the key variable. This controls how much data is requested in the first chunk of playback - too big and your start times will be too slow, too small and you might get stuttering at the start of playback. 128M seems to work for most but try 64M and 32M --vfs-read-chunk-size-limit: each successive vfs-read-chunk-size doubles in size until this limit is hit e.g. for me 128M, 256M,512M,1G etc. I've set the limit as off to not cap how much is requested so that rclone downloads the biggest chunks it can for my connection Read more on vfs-read-chunk-size: https://forum.rclone.org/t/new-feature-vfs-read-chunk-size/5683
  5. 3/1/20 UPDATE: TO MIGRATE FROM UNIONFS TO MERGERFS READ THIS POST. New users continue reading 13/3/20 Update: For a clean version of the 'How To' please use the github site https://github.com/BinsonBuzz/unraid_rclone_mount 17/11/21 Update: Poll to see how much people are storing I've added a Paypal.me upon request if anyone wants to buy me a beer. There’s been a number of scattered discussions around the forum on how to use rclone to mount cloud media and play it locally via Plex, Emby etc. After discussions with @Kaizac @slimshizn and a few others, we thought it’d be useful to start a thread where we can all share and improve our setups. Why do this? Well, if set-up correctly Plex can play cloud files regardless of size e.g. I play 4K media with no issues, with start times of under 5 seconds i.e. comparable to spinning up a local disk. With access to unlimited cloud space available for the cost of a domain name and around $510/pm, then this becomes a very interesting proposition as it reduces local storage requirements, noise etc etc. At the moment I have about 80% of my library in the cloud and I struggle to tell if a file is local or in the cloud when playback starts. To kick the thread off, I’ll share my current setup using gdrive. I’ll try and keep this initial thread updated. Update: I've moved my scripts to github to make it easier to keep them updated https://github.com/BinsonBuzz/unraid_rclone_mount Changelog 6/11/18 – Initial setup (updated to include rclone rc refresh) 7/11/18 - updated mount script to fix rc issues 10/11/18 - added creation of extra user directories ( /mnt/user/appdata/other/rclone & /mnt/user/rclone_upload/google_vfs) to mount script. Also fixed typo for filepath 11/11/18 - latest scripts added to https://github.com/BinsonBuzz/unraid_rclone_mount for easier editing 3/1/20 - switched from unionfs to mergerfs 4/2/20 - updated the scripts to make easier to use and control. Thanks to @senpaibox for the inspiration My Setup Plugins needed Rclone – installs rclone and allows the creation of remotes and mounts. New scripts require V1.5.1+ User Scripts – controls how mounts get created How It Works Rclone is used to access files on your google drive and to mount them in a folder on your server e.g. mount a gdrive remote called gdrive_vfs: at /mnt/user/mount_rlone/gdrive_vfs Mergerfs is used to merge files from your rclone mount (/mnt/user/mount_rlone/gdrive_vfs) with local files that exist on your server and haven't been uploaded yet (e.g. /mnt/user/local/gdrive_vfs) in a new mount /mnt/user/mount_unionfs/gdrive_vfs This mergerfs mount allows files to be played by dockers such as Plex, or added to by dockers like radarr etc without the dockers even being aware that some files are local and some are remote. It just doesn't matter The use of a rclone vfs remote allows fast playback, with files streaming within seconds New files added to the mergerfs share are actually written to the local share, where they will stay until the upload script processes them An upload script is used to upload files in the background from the local folder to the remote. This activity is masked by mergerfs i.e. to plex, radarr etc files haven't 'moved' Getting Started Install the rclone plugin and via command line run rclone config and create 2 remotes: gdrive: - a drive remote that connects to your gdrive account. Recommend creating your own client_id gdrive_media_vfs: - a crypt remote that is mounted locally and decrypts the encrypted files uploaded to gdrive: It is advisable to create your own client_id to avoid API bans. Mount Script - see https://github.com/BinsonBuzz/unraid_rclone_mount for latest script Create a new script using the the user scripts plugin and paste in the rclone_mount script Edit the config lines at the start of the script to choose your remote name, paths etc Choose a suitable cron job. I run this script on a 10 min */10 * * * * schedule so that it automatically remounts if there’s a problem. The script: Checks if an instance is already running, remounts (if cron job set) automatically if mount drops Mounts your rclone gdrive remote Installs mergerfs and creates a mergerfs mount Starts dockers that need the mergerfs mount e.g. plex, radarr Upload Script - see https://github.com/BinsonBuzz/unraid_rclone_mount for latest script Create a new script using the the user scripts plugin and paste in the rclone_mount script Edit the config lines at the start of the script to choose your remote name, paths etc - USE THE SAME PATHS Choose a suitable cron job e.g hourly Features: Checks if rclone is installed correctly sets bwlimits There is a cap on uploads by google of 750GB/day. I have added bandwidth scheduling to the script so you can e.g. set an overnight job to upload the daily quota at 30MB/s, have it trickle up over the day at a constant 10MB/s, or set variable speeds over the day The script now stops once the 750GB/day limit is hit (rclone 1.5.1+ required) so there is more flexibility over upload strategies I've also added --min age 10mins to stop any premature uploads and exclusions to stop partial files etc getting uploaded. Cleanup Script - see https://github.com/BinsonBuzz/unraid_rclone_mount for latest script Create a new script using the the user scripts plugin and set to run at array start (recommended) or array stop In the next post I'll explain my rclone mount command in a bit more detail, to hopefully get the discussion going!
  6. @smdion the link is broken - can you share the script again please.
  7. I'm glad I found this thread as I have two drives throwing up this error. Is there a fix yet?
  8. Help please! I've read https://github.com/Novik/ruTorrent/wiki/Config#authentication but I still don't see how I add a username and password to this docker. I've just moved from Deluge (slow downloads and keeps freezing) and qbittorrent (won't upload) Thanks
  9. How far away are the rooms located? HDMI over ethernet is a potential solution, as well as long HDMI/USB cables
  10. docker stop transmission docker start transmission
  11. Anyone ever installed mergerfs on unRAID? I need it to merge a local folder with a rclone mount.
  12. It's ok - I've managed to get around using mergerfs this evening - sorry for any hassle caused with any research.
  13. Hi I'm trying this docker for the first time and I'm getting an error on first run: Help please root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='binhex-krusader' --net='br0' --ip='172.30.12.3' --privileged=true -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'TCP_PORT_6080'='6080' -e 'TEMP_FOLDER'='/config/krusader/tmp' -e 'WEBPAGE_TITLE'='Highlander' -e 'VNC_PASSWORD'='Removed' -e 'UMASK'='000' -e 'PUID'='99' -e 'PGID'='100' -v '/mnt/user':'/media':'rw' -v '/mnt/user/':'/user':'rw' -v '/mnt/disks/':'/disks':'rw' -v '/boot/':'/flash':'rw' -v '/mnt/cache/appdata/dockers/binhex-krusader':'/config':'rw' 'binhex/arch-krusader'
  14. Quick poll - what do you guys have for your Bandwidth settings?I haven't touched the default half-open connections or the max connections attempts per second on my 200/200 line, and I wonder if I should. I use two private trackers IPT and Torrentday
  15. I don't know if you do requests, but is it possible to add mergerfs please as an addition to unionfs?
  16. Bump. This would be major result for me - I just need a bit of help please
  17. Hi I'm getting tonnes of wrong csrf_token errors in my logs - fixable or not worth worrying about? Thanks Oct 10 23:11:15 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:16 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:17 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:18 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:19 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:21 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:22 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:23 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:24 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:25 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:26 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:27 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:28 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:29 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:30 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:31 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:32 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:33 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:34 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:35 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:36 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:37 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:38 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:39 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token Oct 10 23:11:41 Highlander root: error: /plugins/unassigned.devices/UnassignedDevices.php: wrong csrf_token highlander-diagnostics-20181011-1559.zip
  18. I'm posting here as I wasn't sure which board this fits in. I currently use unionfs to combine local files /rclone_upload/google_vfs with gdrive files I mount using rclone /mount_rclone/google_vfs : unionfs -o cow,allow_other,direct_io,auto_cache,sync_read /mnt/user/rclone_upload/google_vfs=RW:/mnt/user/mount_rclone/google_vfs=RO /mnt/user/mount_unionfs/google_vfs I'd like to switch to MergerFS as it supports hardlinking, whereas Unionfs doesn't. I've installed unionfs via Nerd Pack - can someone give me pointers on how to install Mergerfs please?
  19. In the docker settings change the name e.g plex-2, the appdata path e.g. /mnt/user/appdata/plex-2 and ports or ip address
  20. It was uploading files in the background so it was 'working'. However, stopped it as I need to be able to add several sub-folders. I worked out how to get the client working on my VM (changed mac) so using this method.
×
×
  • Create New...