slimshizn

Current setup I have two servers. Main and backup. Main: Backup: Using a 10Gbe switch, the 16-XG, I'm able to connect everything and use as normal. Main has 2 SSD's in Raid1, Backup has a single SSD in XFS. So here's the problem: I can transfer files throughout my network up to 112MB/s no problem for the most part, when on my windows PC trying to move files from Main --> Backup the speed is sitting at 65MB/s typically. Going on krusader on my main server and connecting to backup it's roughly 70-75MB/s. Most I've been able to get is roughly 112MB/s while transferring a 20GB file, no more. Steps I've taken: Changed MTU to 9000 on both servers, changed 10G switch to Jumbo frame, changed RX/TX buffers to 8192 (Max), rebooted each server and switch. I've ran iperf on both servers and got this FULL 10G speeds. Just to make sure SSD was working correctly I moved a file from a UD SSD to it and got 500+MB/s no problem, from server to server via network, nothing over 112MB/s. Need help here! tower-diagnostics-20181018-2141.zip

I've updated to the latest, still having issues after rebooting with the folder caching being disabled and the function on enabled. Stopping and starting it still fixes it.

I had the same issue, removing and reinstalling fixed the issue, works fine. Haven't rebooted to see if it happened again.

This has been awesome, getting speeds up to 40 MB/s with this on ( after delete and re download ).

/tmp/user.scripts/tmpScripts/Rclone_Upload/script* should work then if that's the case, I'm not sure how you have your setup with a dummy file being created. But before I started entering all the information I wanted to see if it was correct. I know that scripts go to tmp after they are started and are ran from there, so with you using a dummy file I could just remove the "rm" portion and just check if that script is being ran at the location I posted. Edit: I could also remove the "touch" command as well then right? Assuming this would be correct for the unmount then as well.

Right okay so I don't want to remove /boot/config/plugins/user.scripts/scripts/Rclone_Upload/script so how would I "remove" it without actually destroying the script?

I do like this idea of activating cache first and then moving on to each disk, smart and efficient. I'll add the --exclude unionfs section and see how that helps. I see you have your plugin location is different than mine. So would this location be correct? /tmp/user.scripts/tmpScripts/Rclone_Upload/script* Also every time you have the rm, is that going to keep the mount still active? I thought it had to be running in the background?

So media on the RO(cloud)folder will be hidden from plex as well correct?

Does it hide the old media when it's actually transferred over to the cloud? I just had something "upgraded" and it never removed or deleted the old media/created a unionfs folder to "hide" the old one. Edit: Just realized that radarr never actually saw the old file that was there. So that's probably why it didn't happen.

Right, I'm talking about when there's an upgrade and it didn't sync or remove the old media yet.

Trying something new, going to have both local and cloud media due to never knowing what google is going to do. So I'm going to sync folders, local to cloud. At least that's the idea, syncing between the /mnt/users/Media/Movies --> /mnt/users/Media/Cloud/Movies and /mnt/users/Media/TVShows --> /mnt/users/Media/Cloud/Series. I think that will stop issues with radarr acting up when it finds there are two media files in a folder.

I haven't had any slow downs on speeds when transferring files over at all. I have sonarr/radarr/deluge/nzbget all pointed towards a UD with a download folder on it, appdata is also on a UD. Plex is also on a UD. Media and updated media from sonarr/radarr is downloaded and moved to the cache and then to the array automatically via the mover at 3AM, and if I want new media I just select the cloud folder and it gets uploaded and updated with zero issues as its all done behind the scenes. Manually moving files to my cloud unionfs folder goes from about 130 MB/s to 230 MB/s. I'm unsure why there would be any slow downs in speeds unless there was high IO.

They were very temperamental with plexdrive for me. Are you running the mounts as rw slave?

So I was looking at your cleanup script. You created ADDITIONAL mounts to have that work? Probably why mine is not working. Edit: I found my issue. Had _gd at the end of one of my mounts. I took it out, ran it, and it deleted files that weren't supposed to be there. Nice.

Okay so I removed --unmask 002, and re ran the script, hit manual transfer again and VOILA it's transfering. So that seems to be the issue I was having. Thank you.

RW

Would turning on privileged do it?

That's already where I have it pointed actually. Still getting the perm error and uploads and deletes are working fine. Not sure why it wont transfer.

So maybe that's the problem? Do I have the directories mixed up there?

Here's the entire thing. I'm using a separate folder to upload what I want to the cloud which is mnt/user/Media/Cloud/Movies and then a /Series dir as well. I move whatever I want there and then have sonarr and radarr point to that directory. I'm not sure what I'm doing wrong here. mnt/disks/crypt is the gdrive encrypted mount, then mnt/user/Media/Cloud/Seriestmp and Moviestmp are for the upload, correct. Wonder what's going on?? Edit: I'm currently uploading so would that have any effect on that?

Is this the culprit?

Keep getting things like this happening with the /mnt/users/Media/Cloud folders. Almost every time an update happens. Running docker safe new perms again see if It will fix it. Edit: It did not.

I use a ssd cache and the unassigned drive is ssd. If you use a single no raid config xfs is said to have the best performance.

Try moving your download folder to an unassigned drive, as well as appdata to the unassigned drive. After I did my io issues are almost non existent.

Getting can't follow symlink without -L/--copy-links. I've searched up on rclone forums and came across a github issue. Seems this wasn't working as of April, anyone know if adding -L fixes this issue yet?