midofru

I'm tempted to delete my earlier post, but wanted to post a follow-up instead in case my folly may help someone else. Of the two systems: A. Unraid -- Send-Only B. Synology -- Receive Only I believe it was simply a folder permissions issue where Syncthing on A didn't have write access to large swaths of the content (via container permissions), and so Syncthing would copy/transfer files to B it would accumulating a bunch of unsuccessful/pending write-requests to the share folder I wanted to sync (send to B) -- causing SHFS memory usage to balloon with the unresolved requests. Interestingly I was able to mostly finish the sync by creating enough swap space to accommodate the ballooning memory use; it was arguably a good example of treating symptoms vs. disease. Now I'm on to the next challenge; why I keep finding suggestions that setting B to Send/Receive does a better job reconciling changes from A than leaving B as Receive Only (as I look at why B says Out of Sync but isn't doing anything about it, possibly because it has some files I added to A's ignore list (e.g. *.tmp)).

Any updates or new discoveries? I setup the Syncthing docker container from linuxserver to try to sync a single folder pair for: - About 10TB of content from a bare metal Unraid build (ASRock J5005-ITX w/ 16GB ram, 2x 1TB SSD cache; currently running latest: 7.1.3) to - A mostly empty Synology DS214+ I find that after a period of file scanning by Syncthing where memory usage stays pretty typical/expected, memory usage starts climbing at a rate comparable to what others have described (perhaps when it actually starts trying to copy files). I have found no meaningful impact from any tweaks to reduce the burden on SHFS (e.g. disabling/not-starting all other containers, changing settings in Syncthing, disabling recycle bin plugin, setting up exclusive shares for everything EXCEPT the actual content to be synched given its size and existence across multiple drives). I have been able to keep Unraid from running out of memory and killing SHFS by using a swapfile; when I use the swapfile Unraid keeps going with 92-97% memory utilization and the swapfile holds about 10-18GB, suggesting that if I had 32GB of ram I might more often get away with the high ram utilization but it would still periodically exceed the ram available. I've been limping towards completed synchronization with the idea/hope that once the synchronization is completed (I've gotten to about 99%) then perhaps the periodic small changes needed to maintain would be more manageable and stay within memory bounds, but I'm increasingly less confident that this will be the case and so am thinking I'll need to look at a different solution (e.g. VPN and rsync), or commit to limping along indefinitely with a swapfile solution until something is improved. PS - I am not running the Mover Tuning plugin; I do have the following plugins installed: Appdata Backup Community Applications Dynamix Cache Directories Dynamix System Information Dynamix System Temperature Fix Common Problems rclone Recycle Bin RTL8168 Drivers SwapFile Unassigned Devices Unassigned Devices Plus Unassigned Devices Preclear unbalanced

These are additional notes in line with tr3bjockey's observations, and expanding on what I think the failing mechanism is. 16gb USB2 data traveler by Kingston, stuck on synching. While I was waiting I plugged in another already configured UNRAID usb flash drive to look at the file structure -- and the flash drive creator tool immediate saw the old already named and configured UNRAID flash drive appear and overwrote it with a new installation (this was disappointing as it had been the configured installation of a trial). I removed my old UNRAID flash drive that had just been overwritten by the creator utility, and tried running the creator utility again. It again hung on synching, so I tried renaming the flash drive UNRAID (it was called something generic like "USB Drive") and the utility immediately saw it and completed. So .. it appears that the flash drive creator is programmed to rename the flash drive to UNRAID and then to continue working on whatever drive is called UNRAID. For some drives, perhaps some that don't have a name it understands, it fails to rename it and therefore waits forever for an UNRAID drive to appear --- unless you rename it yourself. Also -- It gave my 16gb drive a single partition of 8gb. My understanding is that this doesn't really matter as the space needs are unlikely to exceed a few megabytes, but it is still disappointing. I would imagine doing a manual install would fix this as well. The irony is that I chose this drive as I believed it was one of annointed (recommended) ones. I could have had a larger and faster (USB3) drive for about the same price, but I was trying to play it safe. Hopefully it will be rock solid going forward. I'm hoping the metal construction and its size (small but not too small) will mean it won't get too hot or burn out early.