privateer

The analyze files is checked. How are you installing mergerfs? Mine runs through the script as noted above. In order to run mergerfs like you have I can run the partial set of commands in the script but I want to make sure we're doing things 1:1 since this is so problematic. EDIT: additional permissions question. File path is /mnt/user/mount_unionfs/gdrive_media_vfs/tv_shows/. Should /mount_unionfs read 99/100 and /gdrive_media_vfs read 99/100 etc all the way down to the individual file?

So I'm on the same unraid OS and have been testing both on linuxserver develop and stable. So that seems stable between our deployments. One other thing - for a long time now (years?) the first time my mount script runs it fails and the second time it works. Not sure what to make of that but I'm trying to put everything out there. Should I have 99:100 permissions for all the folders below tv_shows? What should file permissions be for individual files sonarr needs to access?

What OS version are you on? and what sonarr version

This still seems like a permissions issue to me and I think it's with the mounting script/mergerfs because I feel like we're continuing to eliminate other options. 1) Nuked the docker and appdata. Redownloaded sonarr, stripped out UMASK, /tv, /downloads, and added /gdrive. Permissions had reset themselves on /mnt/user/mount_unionfs/gdrive_media_vfs/tv_shows so I re-ran newperms /mnt/user/mount_unionfs/gdrive_media_vfs/tv_shows. Permissions set on, chmod Folder 777. Tried to import library and it had the same result as shown above. 2) Nuked the docker and appdata. Redownloaded sonarr, stripped out UMASK and /tv, but left /downloads and added /gdrive. Permissions had reset themselves on /mnt/user/mount_unionfs/gdrive_media_vfs/tv_shows so I re-ran newperms /mnt/user/mount_unionfs/gdrive_media_vfs/tv_shows. Permissions set on, chmod Folder 777. Added new show, downloaded episode (goes to local), successfully uploaded. It is imported into Sonarr successfully. I couldn't get it to play on my computer though - every time I tried to open the file through the explorer, the window froze and crashed/had to be killed. As an aside, it also appears that everytime the mounting script is run, it resets the permissions I modified with the newperms command. EDIT: When the mount command runs for the first time, this is the log after mergerfs installs: build/.src/policy_epall.o build/.src/policy_lus.o build/Script Starting Apr 06, 2023 14:10.01 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount_new/log.txt 06.04.2023 14:10:01 INFO: Creating local folders. 06.04.2023 14:10:01 INFO: Creating MergerFS folders. 06.04.2023 14:10:01 INFO: *** Starting mount of remote gdrive_media_vfs 06.04.2023 14:10:01 INFO: Checking if this script is already running. 06.04.2023 14:10:01 INFO: Exiting script as already running. Script Finished Apr 06, 2023 14:10.01 Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_mount_new/log.txt mergerfs version: 2.35.1 'build/mergerfs' -> '/build/mergerfs' 06.04.2023 14:10:02 INFO: *sleeping for 5 seconds 06.04.2023 14:10:07 INFO: Mergerfs installed successfully, proceeding to create mergerfs mount. 06.04.2023 14:10:07 INFO: Creating gdrive_media_vfs mergerfs mount. 06.04.2023 14:10:07 INFO: Checking if gdrive_media_vfs mergerfs mount created. 06.04.2023 14:10:07 INFO: Check successful, gdrive_media_vfs mergerfs mount created. 06.04.2023 14:10:07 INFO: Starting dockers. binhex-rtorrentvpn sonarr sonarr4k radarr radarr4k overseerr jackett Just want to make sure you don't see anything here... tautulli Grafana Influxdb telegraf Varken autoscan 06.04.2023 14:10:18 INFO: Script complete

I had checked for API ban earlier but was pretty sure it wasn't the case. Here are the results: Correct - library import is /gdrive/mount_unionfs/gdrive_media_vfs/tv_shows. This is also listed as the root folder on sonarr. Sonarr docker console returns the correct amount of folders and matches the ones I tried to import. I'm able to play files from inside /gdrive/mount_unionfs/gdrive_media_vfs/tv_shows on my own computer. My guess was that it was either 1) TVDB API issues, 2) Sonarr DB issues 3) permissions issues 4) mount script issues. I was hitting a wall with progress though - once again thanks for helping out. Mount script below: #!/bin/bash ###################### #### Mount Script #### ###################### ## Version 0.96.9.3 ## ###################### ####### EDIT ONLY THESE SETTINGS ####### # INSTRUCTIONS # 1. Change the name of the rclone remote and shares to match your setup # 2. NOTE: enter RcloneRemoteName WITHOUT ':' # 3. Optional: include custom command and bind mount settings # 4. Optional: include extra folders in mergerfs mount # REQUIRED SETTINGS RcloneRemoteName="gdrive_media_vfs" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data RcloneMountShare="/mnt/user/mount_rclone" # where your rclone remote will be located without trailing slash e.g. /mnt/user/mount_rclone RcloneMountDirCacheTime="720h" # rclone dir cache time LocalFilesShare="/mnt/user/rclone_upload" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable RcloneCacheShare="/mnt/user/mount_rclone" # location of rclone cache files without trailing slash e.g. /mnt/user0/mount_rclone RcloneCacheMaxSize="400G" # Maximum size of rclone cache RcloneCacheMaxAge="336h" # Maximum age of cache files MergerfsMountShare="/mnt/user/mount_unionfs" # location without trailing slash e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable DockerStart="binhex-rtorrentvpn sonarr sonarr4k radarr radarr4k overseerr jackett tautulli Grafana Influxdb telegraf Varken autoscan" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page MountFolders=\{"4kmovies,movies,tv_shows,audiobooks"\} # comma separated list of folders to create within the mount # Note: Again - remember to NOT use ':' in your remote name above # OPTIONAL SETTINGS # Add extra paths to mergerfs mount in addition to LocalFilesShare LocalFilesShare2="ignore" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder. Enter 'ignore' to disable LocalFilesShare3="ignore" LocalFilesShare4="ignore" # Add extra commands or filters Command1="--rc" Command2="" Command3="" Command4="" Command5="" Command6="" Command7="" Command8="" CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter RCloneMountIP="192.168.1.252" # My unraid IP is 172.30.12.2 so I create another similar IP address NetworkAdapter="eth0" # choose your network adapter. eth0 recommended VirtualIPNumber="2" # creates eth0:x e.g. eth0:1. I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them ####### END SETTINGS ####### ############################################################################### ##### DO NOT EDIT ANYTHING BELOW UNLESS YOU KNOW WHAT YOU ARE DOING ####### ############################################################################### ####### Preparing mount location variables ####### RcloneMountLocation="$RcloneMountShare/$RcloneRemoteName" # Location for rclone mount LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" # Location for local files to be merged with rclone mount MergerFSMountLocation="$MergerfsMountShare/$RcloneRemoteName" # Rclone data folder location ####### create directories for rclone mount and mergerfs mounts ####### mkdir -p /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName # for script files mkdir -p $RcloneCacheShare/cache/$RcloneRemoteName # for cache files if [[ $LocalFilesShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating local folders as requested." LocalFilesLocation="/tmp/$RcloneRemoteName" eval mkdir -p $LocalFilesLocation else echo "$(date "+%d.%m.%Y %T") INFO: Creating local folders." eval mkdir -p $LocalFilesLocation/"$MountFolders" fi mkdir -p $RcloneMountLocation if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating MergerFS folders as requested." else echo "$(date "+%d.%m.%Y %T") INFO: Creating MergerFS folders." mkdir -p $MergerFSMountLocation fi ####### Check if script is already running ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Starting mount of remote ${RcloneRemoteName}" echo "$(date "+%d.%m.%Y %T") INFO: Checking if this script is already running." if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Exiting script as already running." exit else echo "$(date "+%d.%m.%Y %T") INFO: Script not running - proceeding." touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running fi ####### Checking have connectivity ####### echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if online" ping -q -c2 google.com > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") PASSED: *** Internet online" else echo "$(date "+%d.%m.%Y %T") FAIL: *** No connectivity. Will try again on next run" rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi ####### Create Rclone Mount ####### # Check If Rclone Mount Already Created if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Success ${RcloneRemoteName} remote is already mounted." else echo "$(date "+%d.%m.%Y %T") INFO: Mount not running. Will now mount ${RcloneRemoteName} remote." # Creating mountcheck file in case it doesn't already exist echo "$(date "+%d.%m.%Y %T") INFO: Recreating mountcheck file for ${RcloneRemoteName} remote." touch mountcheck rclone copy mountcheck $RcloneRemoteName: -vv --no-traverse # Check bind option if [[ $CreateBindMount == 'Y' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: *** Checking if IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" ping -q -c2 $RCloneMountIP > /dev/null # -q quiet, -c number of pings to perform if [ $? -eq 0 ]; then # ping returns exit status 0 if successful echo "$(date "+%d.%m.%Y %T") INFO: *** IP address ${RCloneMountIP} already created for remote ${RcloneRemoteName}" else echo "$(date "+%d.%m.%Y %T") INFO: *** Creating IP address ${RCloneMountIP} for remote ${RcloneRemoteName}" ip addr add $RCloneMountIP/24 dev $NetworkAdapter label $NetworkAdapter:$VirtualIPNumber fi echo "$(date "+%d.%m.%Y %T") INFO: *** Created bind mount ${RCloneMountIP} for remote ${RcloneRemoteName}" else RCloneMountIP="" echo "$(date "+%d.%m.%Y %T") INFO: *** Creating mount for remote ${RcloneRemoteName}" fi # create rclone mount rclone mount \ $Command1 $Command2 $Command3 $Command4 $Command5 $Command6 $Command7 $Command8 \ --allow-other \ --umask 000 \ --dir-cache-time $RcloneMountDirCacheTime \ --attr-timeout $RcloneMountDirCacheTime \ --log-level INFO \ --poll-interval 10s \ --cache-dir=$RcloneCacheShare/cache/$RcloneRemoteName \ --drive-pacer-min-sleep 10ms \ --drive-pacer-burst 1000 \ --vfs-cache-mode full \ --vfs-cache-max-size $RcloneCacheMaxSize \ --vfs-cache-max-age $RcloneCacheMaxAge \ --vfs-read-ahead 1G \ --bind=$RCloneMountIP \ $RcloneRemoteName: $RcloneMountLocation & # Check if Mount Successful echo "$(date "+%d.%m.%Y %T") INFO: sleeping for 5 seconds" # slight pause to give mount time to finalise sleep 5 echo "$(date "+%d.%m.%Y %T") INFO: continuing..." if [[ -f "$RcloneMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Successful mount of ${RcloneRemoteName} mount." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mount failed - please check for problems. Stopping dockers" docker stop $DockerStart rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi fi ####### Start MergerFS Mount ####### if [[ $MergerfsMountShare == 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Not creating mergerfs mount as requested." else if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount in place." else # check if mergerfs already installed if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs already installed, proceeding to create mergerfs mount" else # Build mergerfs binary echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs not installed - installing now." mkdir -p /mnt/user/appdata/other/rclone/mergerfs docker run -v /mnt/user/appdata/other/rclone/mergerfs:/build --rm trapexit/mergerfs-static-build mv /mnt/user/appdata/other/rclone/mergerfs/mergerfs /bin # check if mergerfs install successful echo "$(date "+%d.%m.%Y %T") INFO: *sleeping for 5 seconds" sleep 5 if [[ -f "/bin/mergerfs" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Mergerfs installed successfully, proceeding to create mergerfs mount." else echo "$(date "+%d.%m.%Y %T") ERROR: Mergerfs not installed successfully. Please check for errors. Exiting." rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi fi # Create mergerfs mount echo "$(date "+%d.%m.%Y %T") INFO: Creating ${RcloneRemoteName} mergerfs mount." # Extra Mergerfs folders if [[ $LocalFilesShare2 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare2} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare2=":$LocalFilesShare2" else LocalFilesShare2="" fi if [[ $LocalFilesShare3 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare3} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare3=":$LocalFilesShare3" else LocalFilesShare3="" fi if [[ $LocalFilesShare4 != 'ignore' ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Adding ${LocalFilesShare4} to ${RcloneRemoteName} mergerfs mount." LocalFilesShare4=":$LocalFilesShare4" else LocalFilesShare4="" fi # make sure mergerfs mount point is empty mv $MergerFSMountLocation $LocalFilesLocation mkdir -p $MergerFSMountLocation # mergerfs mount command mergerfs $LocalFilesLocation:$RcloneMountLocation$LocalFilesShare2$LocalFilesShare3$LocalFilesShare4 $MergerFSMountLocation -o rw,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true # check if mergerfs mount successful echo "$(date "+%d.%m.%Y %T") INFO: Checking if ${RcloneRemoteName} mergerfs mount created." if [[ -f "$MergerFSMountLocation/mountcheck" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ${RcloneRemoteName} mergerfs mount created." else echo "$(date "+%d.%m.%Y %T") CRITICAL: ${RcloneRemoteName} mergerfs mount failed. Stopping dockers." docker stop $DockerStart rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running exit fi fi fi ####### Starting Dockers That Need Mergerfs Mount To Work Properly ####### # only start dockers once if [[ -f "/mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started" ]]; then echo "$(date "+%d.%m.%Y %T") INFO: dockers already started." else # Check CA Appdata plugin not backing up or restoring if [ -f "/tmp/ca.backup2/tempFiles/backupInProgress" ] || [ -f "/tmp/ca.backup2/tempFiles/restoreInProgress" ] ; then echo "$(date "+%d.%m.%Y %T") INFO: Appdata Backup plugin running - not starting dockers." else touch /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/dockers_started echo "$(date "+%d.%m.%Y %T") INFO: Starting dockers." docker start $DockerStart fi fi rm /mnt/user/appdata/other/rclone/remotes/$RcloneRemoteName/mount_running echo "$(date "+%d.%m.%Y %T") INFO: Script complete" exit

ran the command, which looks like it worked: still no dice though. stays frozen here: I do notice that if I reboot it loads the next 3 (in terms of updating from 0 / 0 to 0 / [Correct Number]) but still doesn't import the files. Giving you that info in case it's helpful

Really appreciate the help @Kaizac! This has now had me down for day vs no problems for the last few years. permissions from unraid console: permissions from console inside the docker container: Just out of curiosity I ran ls -la inside the containers for my 2 radarr images and my other, working sonarr image, and the permissions are the same.

mount_unionfs is the mergerfs folder. Didn't bother to change folder name when the shift to mergerfs happened.

An update: I can now add my cloud drive as a root folder to sonarr. However, when I go to import library sonarr breaks. It gets stuck refreshing the first 3 series. It loads the total number of episodes in each of those series and dies. All of my remaining series show 0 / 0 and when you open the total number of seasons has loaded but each season says no episodes. At this point I believe this likely has to be a permissions issue, but I don't know where to go from here. I have trace logs running - did a search for error and warn and nothing came up. Some screen shots:

Installing a clean copy of Sonarr and I can't link up my cloud folder that was linked to the prior version. When I attempt to add the folder I get the error: Inside Sonarr I see this: 2023-04-02 16:51:21.0|Trace|DiskProviderBase|Directory '/gdrive/mount_unionfs/gdrive_media_vfs/tv_shows/' isn't writable. Access to the path '/gdrive/mount_unionfs/gdrive_media_vfs/tv_shows/sonarr_write_test.txt' is denied. Has anyone come across this or know any solutions? I'm stuck and can't re-add my cloud mount... The mounted folder I'm having issues with appears to be mounted with permissions 755. Not sure if that could be causing problems (e.g. it needs to be 777).

Data rebuild completed, however I received a yellow notification (warning) saying it had successfully completed but the description was 'Canceled'. From the archived notices: 10-03-2023 02:21 Unraid Data-Rebuild Notice [TOWER] - Data-Rebuild finished (0 errors) Canceled warning However, when I go into the system log, I see this: Mar 10 02:20:47 Tower kernel: md: sync done. time=130479sec Mar 10 02:20:47 Tower kernel: md: recovery thread: exit status: 0 Is the cancelled notice an error or bug or something? Not sure how to reconcile these.

Drive rebuild is underway, no issues there. Went through my logs and have found another instance of this error. Mar 8 15:09:36 Tower kernel: traps: lsof[26299] general protection fault ip:14ec3865f4ee sp:9d0015fe713527f1 error:0 in libc-2.36.so[14ec38647000+16b000] Is this unrelated? If so happy to discuss separately.

Unplugged the cables, plugged everything back in. Seems to work right now. I'm going to wait a bit to mark this solved just to make sure it doesn't pop up as a problem again in a few days.

A few quick follow-up questions: Do I have to shut down the server to deal with this? If not, should I stop the parity check, take the array offline, check connections, and restart array? Or what are the next steps?

Are you running this script using the User Scripts plugin? I'm trying to figure out why the scripts are being run by root instead of nobody.

I was in the middle of a scheduled parity check and 66% of the way through it encountered 2112 errors on a drive on the array and paused. I ran a short SMART check which the drive passed, then tried to run an extended test several times. I may have accidentally cancelled once, but the next time I saw it at 50% and then it looked like it had never been run. Right now, if I click to run a smart test, nothing happens, and nothing is shown in the log. Additionally, the drive that has the red X is also displayed under Unassigned Devices. When I check the logs I have loads of errors such as below: Mar 7 21:03:37 Tower kernel: traps: lsof[7196] general protection fault ip:14fe13e634ee sp:6690f0b91e1b046c error:0 in libc-2.36.so[14fe13e4b000+16b000] Mar 7 21:12:23 Tower kernel: traps: lsof[29820] general protection fault ip:151d585114ee sp:86baf5ee546675bc error:0 in libc-2.36.so[151d584f9000+16b000] Mar 7 21:24:35 Tower kernel: traps: lsof[29136] general protection fault ip:152f3c6314ee sp:152a85e9f914a6b9 error:0 in libc-2.36.so[152f3c619000+16b000] Mar 7 21:41:46 Tower kernel: traps: lsof[3668] general protection fault ip:14fd3f0824ee sp:780278895a35ec7 error:0 in libc-2.36.so[14fd3f06a000+16b000] Any thoughts on what might be causing any of this? Attached are my diagnostics. tower-diagnostics-20230307-2048.zip

Have you tried it without the uid and gid flags? I believe unraid should default to the nobody:users for everything.

Have you tried --umask 000? Is this error still happening?

The problem went away after some time. Maybe a gdrive or rclone issue.

Updated my Unraid and now am having some permission issues. Radarr and Sonarr are unable to create new folders when importing to my mounted drives. I can manually create folders, so I know the process works. Any idea why radarr and sonarr can't? I went down to trace level and there wasn't anything useful I could see. Anyone run into this before and know what caused it, and how to solve it? EDIT: error from sonarr Couldn't import episode /downloads/completed/[tv episode name].mkv: Access to the path '/gdrive/mount_unionfs/gdrive_media_vfs/tv_shows/[Name of Series]' is denied. The folder ('/gdrive/mount_unionfs/gdrive_media_vfs/tv_shows/[Name of Series]') has not been created for the series and does not exist. No clue what's causing this.

Anyone here use a QuickSync / Quick Sync box with their Unraid as a cheap way to increase the number of transcodes? I've asked before, but wanted to check back in. If so, I have a few specific questions for managing the QS box.

Think I'm in the same place as you. I am running 11.2.1 - updated today to 11.4 and now I'm in a boot loop. In 11.2 I had adjusted my cores, RAM, and some light edits to allow the iMessage service to work. Believe this was the config.plist. It took me a while to get everything up and running the first time for Big Sur. Now I get a bootloop (but can still boot the old image). I'm new to Hackintosh. Can you give me some more basic steps and info about this new Penryn patch and what else I would need to do to allow the upgrade to be successful? Thanks!

Does "At Stopping of Array" mean it executes once the array has stopped, immediately prior to the array stopping, or does it happen simultaneously with the array stopping? Trying to figure out how to have a command execute right before the array is stopped or at least simultaneously.

Ok so you aren't actually mounting anything. Thanks for clarifying. I'll update here if I ever figure out how to mount the mergerfs folder on another linux box. Or if anyone else knows how, let me know!