teh0wner

Did you ever find a solution to this?

Any ideas why Authelia seems to be creating these docker containers automatically but serve no purpose?

I'm using SA for that and uploads are working fine. The problem is with "downloading"

So it seems I've bit hit with a ban - "Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded" when trying to acess my files although I can't figure out why exactly. Looking at the Quota stats on https://console.developers.google.com/apis/api/drive.googleapis.com/quotas I don't see me even getting close to the quota. I've also tried creating a new client id/secret to by-pass this, but I'm still getting the same error back. I also have a completely different Team Drive using different credentials, and that seems to have been hit with a ban as well. Any ideas?

Did you figure this out? Having the same issue here too.

Mind pointing my in the right direction? Not quite sure how the conf should look like. I've got everything up and running with linuxserver letsencrypt.

Partly one of the issues I was having - not sure what the best way of checking there's an upload currently running, or whether the mount script is currently running (and attempting to mount a drive - i.e. rclone_mount).

I've written a small user script that reliably stops the array 100%. I'm not quite sure how 'safe' it is though - I haven't noticed any issues since I've been using it though. Here it is : #!/bin/bash ########################## ### fusermount Script #### ########################## RcloneRemoteName="google_drive_encrypted_vfs" RcloneMountShare="/mnt/user/data/remote_storage" MergerfsMountShare="/mnt/user/data/merged_storage" echo "$(date "+%d.%m.%Y %T") INFO: *** Starting rclone_fusermount script ***" while [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; do echo "$(date "+%d.%m.%Y %T") INFO: mount is running, sleeping" sleep 5 done while [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running" ]]; do echo "$(date "+%d.%m.%Y %T") INFO: upload is running, sleeping" sleep 15 done fusermount -uz "$MergerfsMountShare/$RcloneRemoteName" fusermount -uz "$RcloneMountShare/$RcloneRemoteName" echo "$(date "+%d.%m.%Y %T") INFO: ***rclone_fusermount script finished ***" exit Essentially what I'm doing, is checking whether mount is currently running and/or upload is currently running. If either of them are, then the script (and stopping the array) is paused for a few seconds and tries again. Once mount and upload has finished, it will proceed to fusermount -uz (both rclone and mergefs), and then the array stops just fine. I've been using this for the past week with no issues and the array stops always. Let me know what you think if you get to use it

How safe would manually doing fusermount -uz (on array stop) be if mount_running and upload_running don't exist? (i.e. those 2 scripts aren't running)?

I'm not sure whether this is even possible, but here's my question. Is there a way I can make the domain I use for my unRAID server, if accessed internally, to point to an IP:PORT instead? I'm using letsencrypt and nginx to achieve a reverse proxy, where sonarr.mydomain.com, for example, works. However, if accessing within my LAN, I want to be able to bypass authentication set-up by nginx, and go directly into sonarr. My idea was to somehow, if that URL is accessed from my LAN, then make it points to SONARR-IP:PORT directly. I hope this makes sense. Thanks

Oh wow, amazing. Didn't know this existed - exactly what I was looking for. Thanks

+1 - Starting to use more and more dockers and the GUI is getting a bit messy to manage. A way of grouping 'related' Dockers would be great.

I'm having some issues with the permissions as well. Even adding the chmod in go, it doesn't seem to apply on boot. root@XXX-XXXX:/usr/local/emhttp/webGui/event/starting# ls -ltra total 4 drwxr-xr-x 9 root root 180 May 3 10:54 ../ -rw------- 1 root root 164 May 3 10:54 fetch_key drwxrwxrwx 2 root root 60 May 3 10:54 ./ And my go looks like this root@XXX-XXX:/boot/config# cat go #!/bin/bash # auto unlock array mkdir -p /usr/local/emhttp/webGui/event/starting mkdir -p /usr/local/emhttp/webGui/event/started mkdir -p /usr/local/emhttp/webGui/event/stopped cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/starting cp -f /boot/custom/bin/delete_key /usr/local/emhttp/webGui/event/started cp -f /boot/custom/bin/fetch_key /usr/local/emhttp/webGui/event/stopped chmod a+x /usr/local/emhttpd/webGui/event/starting/fetch_key chmod a+x /usr/local/emhttpd/webGui/event/started/delete_key chmod a+x /usr/local/emhttpd/webGui/event/stopped/fetch_key # Start the Management Utility /usr/local/sbin/emhttp & The actual fetch_key and delete_key scripts work fine, as when I chmod them manually and run, they work.

I've not even tried uploading. Fresh reboot, manually run rclone_mount, stop. Always fails 100%. I've not got to the bottom of it either, but will revert if I do. I was thinking maybe writing another script that checks if an upload if running, and if not, then run fusermount -uz on the rclone mounts, and then stop the array. Do you think that'll cause issues? Edit: Something along the lines of the below: #!/bin/bash ########################## ### fusermount Script #### ########################## RcloneRemoteName="google_drive_encrypted_vfs" RcloneMountShare="/mnt/user/XYZ/remote_storage" MergerfsMountShare="/mnt/user/XYZ/merged_storage" echo "$(date "+%d.%m.%Y %T") INFO: *** Starting rclone_fusermount script ***" while [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; do echo "$(date "+%d.%m.%Y %T") INFO: mount is running, sleeping" sleep 5 done while [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneUploadRemoteName/upload_running" ]]; do echo "$(date "+%d.%m.%Y %T") INFO: upload is running, sleeping" sleep 15 done fusermount -uz "$MergerfsMountShare" fusermount -uz "$RcloneMountShare" echo "$(date "+%d.%m.%Y %T") INFO: ***rclone_fusermount script finished ***" exit

It's more of question of what if I'm unavailable to enter the passphrase, than not being able to. People reliant on the server would have to wait for me to enter the passphrase, whereas, I would prefer to automate this process with FTP or similar. Only downside, is I don't have a keyfile to follow the tutorials. Unless it's just a matter of echo "passphrase" > keyfile ? Edit: Indeed, that's the way.

How would one go on about auto-unlocking from a remote source if encryption was set-up with a passphrase?

Have you ever figured this out? Fell into the same trap myself.

Has anyone had issues with Stopping/Rebooting/Shutting down the Array when rclone mount is active? I've been testing this for the day, and stopping the array after a fresh reboot (before I run rclone_mount) works fine. As soon as I run rclone_mount and I try and stop the array, it doesn't work. Syslogs May 2 17:34:15 XXX-XXX emhttpd: shcmd (440): rmdir /mnt/user May 2 17:34:15 XXX -XXX root: rmdir: failed to remove '/mnt/user': Device or resource busy May 2 17:34:15 XXX -XXX emhttpd: shcmd (440): exit status: 1 May 2 17:34:15 XXX -XXX emhttpd: shcmd (442): /usr/local/sbin/update_cron May 2 17:34:15 XXX -XXX emhttpd: Retry unmounting user share(s)... May 2 17:34:20 XXX -XXX emhttpd: shcmd (443): umount /mnt/user May 2 17:34:20 XXX -XXX root: umount: /mnt/user: target is busy. May 2 17:34:20 XXX -XXX emhttpd: shcmd (443): exit status: 32 May 2 17:34:20 XXX -XXX emhttpd: shcmd (444): rmdir /mnt/user May 2 17:34:20 XXX -XXX root: rmdir: failed to remove '/mnt/user': Device or resource busy May 2 17:34:20 XXX -XXX emhttpd: shcmd (444): exit status: 1 May 2 17:34:20 XXX -XXX emhttpd: shcmd (446): /usr/local/sbin/update_cron May 2 17:34:20 XXX -XXX emhttpd: Retry unmounting user share(s)... lsof gives nothing back root@XXX -XXX :~# lsof /mnt/* root@XXX -XXX :~# lsof /mnt/user/ The only way I can get the array to stop, is by manually running fusermount -uz /mnt/user/ Can someone shed some light please? Thanks

Actually, after the upload script has finished it's own thing, it looks like the hard-links are gone again, and the inode count is 1.

I think I've figured it out, although not quite sure why the behaviour is as is.. My rclone_mount set-up looks as follows (relevant parts only) : RcloneRemoteName="google_drive_encrypted_vfs" LocalFilesShare="/mnt/user/data/staged_storage" RcloneMountShare="/mnt/user/data/remote_storage" MergerfsMountShare="/mnt/user/data/merged_storage" LocalFilesShare2="/mnt/user/data/local_storage" The following configuration fails to create a hard-link: Deluge downloads to LocalFileShare2/incomplete Deluge moves once complete to LocalFileShare2/complete Sonarr attemps to create hard-link to MergerfsMountShare/media The following configuration creates the hard-link successfully: Deluge downloads to LocalFileShare2/complete Deluge moves once complete to MergerfsMountShare/complete (and it's painfully slow) Sonarr create hard-link to MergerfsMountShare/media successfully I don't see though why Mergerfs wouldn't work in the first scenario, as LocalFilesShare2 is still 'merged' into a mount on MergerfsMountShare

Managed to grab some logs from Sonarr.. The error is quite obvious, but not quite sure what the solution is. 20-3-3 22:02:41.3|Debug|EpisodeFileMovingService|Hardlinking episode file: /data/local_storage/downloads/deluge/complete/ABC.mkv to /data/merged_storage/google_drive_encrypted_vfs/media/ABC.mkv 20-3-3 22:02:41.3|Debug|DiskTransferService|HardLinkOrCopy [/data/local_storage/downloads/deluge/complete/ABC.mkv] > [/data/merged_storage/google_drive_encrypted_vfs/media/ABC.mkv] 20-3-3 22:02:41.3|Debug|DiskProvider|Hardlink '/data/local_storage/downloads/deluge/complete/ABC.mkv' to '/data/merged_storage/google_drive_encrypted_vfs/media/ABC.mkv' failed. [v2.0.0.5338] Mono.Unix.UnixIOException: Invalid cross-device link [EXDEV]. at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <4a040cc44eb54354b3d289eb2bbc1e23>:0 at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <4a040cc44eb54354b3d289eb2bbc1e23>:0 at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <4a040cc44eb54354b3d289eb2bbc1e23>:0 at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Mono\Disk\DiskProvider.cs:182

Glad it worked for you! I've been playing around with rclone and mergerfs a lot the past few days, so if you have any other questions fire away. The only thing I can't figure out which maybe @DZMM might be able to help, is getting hard-links to work with Sonarr/Radarr. It seems completely random when they work and when they dont. I have the following setup on my Sonarr/Radarr and Deluge /data -> /mnt/user/data/ so all docker containers are pointing to the same location with the same exact 'root mount' of /data. Deluge downloads to /data/gdrive_encrypt/vfs/downloads and media lives in /data/gdrive_encrypt_vfs/media, so everything pretty much 'local'. Sometimes a hard-link is created (as I can see the inode count is 2 using ls -li, and sometimes it's 1, which means hard link failed and Sonarr/Radarr had to copy). Any ideas why this might be happening?

I finally managed to get this working. Try and make a list of all docker images/containers you have, and nuke all the mergerfs ones. That will force pull the latest one, which seems to be working.

Indeed - even that though is exactly the same issue. mergerfs doesn't seem to install properly. I'm wondering if it has to do anything with the issues mentioned above with the branches. Looking at the docker images I have.. trapexit/mergerfs-static-build latest 9fd5a8a2dbc2 23 hours ago 5.58MB This looks like it could contain the dodgy version. I tried doing a docker pull though, so I would assume it would pick up the actual latest with the fix.

The error comes at mv. root@R2-D2:~# mkdir -p /mnt/user/appdata/other/test/mergerfs root@R2-D2:~# docker run -v /mnt/user/appdata/other/test/mergerfs:/build --rm trapexit/mergerfs-static-build root@R2-D2:~# mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin mv: cannot stat '/mnt/user/appdata/other/rclone/mergerfs/mergerfs': No such file or directory root@R2-D2:~# ls -ltra /mnt/user/appdata/other/rclone/mergerfs/ total 0 drwxrwxrwx 1 root root 61 Feb 27 11:22 ../ drwxrwxrwx 1 root root 6 Mar 1 13:34 ./ root@R2-D2:~#