ken-ji Posted March 5, 2019 Share Posted March 5, 2019 Looks about right. do take note -c option can make rsync seem very slow... as it needs to read and compare. both sides. Quote Link to comment
comet424 Posted March 5, 2019 Author Share Posted March 5, 2019 ok sounds good well I have a really slow internet connection anyways so doesn't matter lol 600kbs upload 5megabits download and do you know if user scripts can handle if statesments so I wanted to do if OpenVPN = connected then run the rsync if not then wait 5 min and reconnect OpenVPN and do this 3 times if it cant initialized 3 times just forget it just like a fax machine.... it will try to send a fax 3 times with 2 or 3 min pause before trying again and then it just quits if it cant establish is that possible in your user scripts and OpenVPN with rysnc or am I asking too much Quote Link to comment
ken-ji Posted March 5, 2019 Share Posted March 5, 2019 Should be doable - you can have openvpn establish the connection, the script will ping the remote unraid and if it works... it will run the rsync. This is a bit beyond me (the openVPN part) as I use router to router GRE tunnels that are up 24/7 Maybe you should open a new thread for the OpenVPN part. so that the other users who use openvpn on unraid / or pfsense can give you help. Quote Link to comment
comet424 Posted March 5, 2019 Author Share Posted March 5, 2019 I did OpenVPN was told had to do it in the support page for OpenVPN.. well I asked and no one replied not even the coder for OpenVPN unraid . ya you helped a lot as for the other guy he said run memtest for days how long do you run it.. and how do you tell if a motherboard or power supply droppin power like you mentioned does unraid monitor this stuff and record it.. is there an app the says power is dropping etc like the bios shows running voltages.. what is a GRE tunnel.. and how much bandwith does that use.. Quote Link to comment
ken-ji Posted March 5, 2019 Share Posted March 5, 2019 About the memtest and monitoring of power, I ran memtest for about a few cycles when I first built my machine. as for power monitoring, I just made sure that my components can never overload my PSU without going way over. then my UPS does report power consumption of the PSU, so I have a fairly good idea if there are issues - but there is no monitoring tool. unless you get expensive industrial component testing tools. My previous PSU did fail without warning and I never figured out why. GRE is a network protocol that like OpenVPN can be used to create virtual network links across the internet. key difference is by it self, there is no encryption, but the link will feel like a slower LAN connection. It doesn't really consume bandwidth unless I try to send traffic across it, mainly because I didn't use the same subnet on both sides. I actually have 3 locations linked up this way. Quote Link to comment
comet424 Posted March 5, 2019 Author Share Posted March 5, 2019 ah I used memtest for 10 cycles when I first did unraid on my gaming machine.. I have unraid plugged into a ups now. my power suppy is a 750 watt corsair. powers 6 hard drives so the GRE does that mean your ISP can see what your files are whe you transfer no encryption... oh 3 locations and how often you run rysnc? that's my goal... 1 Server sisters house... 1 Server in My house 1 Server in my shop and they replicated to each other with the rsync… GRE is that part of unraid? Quote Link to comment
comet424 Posted March 5, 2019 Author Share Posted March 5, 2019 (edited) and doesn't rsync send encrypted? reason for the private and public key... I not at home to try it now anyways.. but was thinking instead of root@mitchsserver:/ can I not do [email protected]:/ and then it be encrypted from the public and private key of rysnc? but probably doesn't work that way Edited March 5, 2019 by comet424 Quote Link to comment
tr0910 Posted March 5, 2019 Share Posted March 5, 2019 14 hours ago, ken-ji said: @tr0910 How do you to want us to truncate the thread? should we have done the discussion as PMs next time? Ha, I was just kidding. It's great to have it all out in the open like this. But we really need to simplify this process. I haven't seen many things in unRaid that can get this ugly. Thank goodness some of the extraneous material you resolved for him wasn't normally required. This thread will really scare off the iwannabackup crowd now. lol Quote Link to comment
tr0910 Posted March 5, 2019 Share Posted March 5, 2019 (edited) 9 hours ago, comet424 said: ah I used memtest for 10 cycles when I first did unraid on my gaming machine.. I have unraid plugged into a ups now. my power suppy is a 750 watt corsair. powers 6 hard drives so the GRE does that mean your ISP can see what your files are whe you transfer no encryption... oh 3 locations and how often you run rysnc? that's my goal... 1 Server sisters house... 1 Server in My house 1 Server in my shop and they replicated to each other with the rsync… GRE is that part of unraid? If you did memtest on both machines at both ends, your corruption is strange. Could be ISP related, but I have never seen that even to China. Yes, the backup is encrypted over the wire via ssh encryption. Over the wire you don't need to worry about your data being visible. GRE is a permanent connection between 2 sites. Don't worry about it for now. This is advanced stuff for later. I would suggest loading TeamViewer on a Windows machine on the destination. That way you can connect over the net and get in and do all the things you would normally, just like being there. Later once you are bored and want cooler advanced always connected connections look at GRE or OpenVPN. The bunker utiliity I mentioned was the early command line version of the Dynamix File Integrity Plugin (find it on the unRaid APP store). They are fully compatible and work together nicely. Originally designed to identify file bit rot, it is super helpful for you in detecting any rsync transfer corruption. First make sure the file hashes are calculated and stored in the extended attributes before you rsync. Then make sure that you use "rsync -avuX" The X tells rsync to transfer the extended attributes where those file hashes are stored. Then on the destination computer, you can ssh in and execute the bunker command to verify if there were any files that arrived in different form than they should have. Check it out here: Edited March 5, 2019 by tr0910 Quote Link to comment
comet424 Posted March 5, 2019 Author Share Posted March 5, 2019 @tr0910ok ill look that stuff up.. as for rsyncing over the net haven't done that yet... what I was doing with all the issues is I had my sisters unraid at my place and I was trying to get rsync to work properly … it was working properly at first.. but we had a few days of power outages on and off and unraid kept shutting down due to no ups …. and then I was having rysnc issues it wasn't working.. so the I was asking help.. and the @ken-ji was helpful to help which was great as I guessing it corrupted files reason I couldn't ssh or all that good stuff... so since it was working yesterday.. I took it over to mysisters house and pluged it in and went home after to try it out that's where I wasn't able to rysnc over the internet now maybe your advanced too and may shed light too? as I tried to get OpenVPN to to work on unraid I cant seem to connect to it I figured unraid (my server) be client and unraid (my sisters) be host for OpenVPN it would connect and transfer... but that didn't work I tried changing root@mitchsserver:/mnt/,,,,, to [email protected]:/mnt.... in the script file or command line and that failed. now the current setup I do to connect to her House and her unraid is.. in Windows (my computer ) I have OpenVPN Client on her network I have PFsense Router with OpenVPN Server running... so I can OpenVPN to her router... and then remote access her unraid and pfsense.. and im guessing that similar to your TeamViewer but if rsync ssh is encrypted.. then I wouldn't need OpenVPN correct? I can skip that .. as I was trying but I found if I have OpenVPN client on unraid and I tried to connect to my sisters pfsense router OpenVPN... it locked out unraid… id have to physical shut it down.. it locked me out of the shares too... but if I rysnc is encrypted data transfer what do I need to change or edit? if I cant get OpenVPN to connect and then just rsync root@mitchsserver sorry if I sound confusing... with dyslexia and learning disability my head knows what I saying I just never explain it right for people sometimes Quote Link to comment
comet424 Posted March 6, 2019 Author Share Posted March 6, 2019 so I got it to rsync had to get help for port forwarding on pfsense.. but that worked.. now Id like to know @ken-ji have you gotten the email to text you once its complete to send you a email or text if there was errors.. as I cant get notifications either to email me or text me Quote Link to comment
comet424 Posted March 6, 2019 Author Share Posted March 6, 2019 (edited) and when rysnc is done to make sure I never have to run new permissions would I do after the rysncs are done to do … will both these ways work chmod 777 root:mitchsserver:/mnt/* or would I do chmod before rysnc starts so it makes it full rights of the Tower files before it starts chmod 777 /mnt/* Edited March 6, 2019 by comet424 Quote Link to comment
ken-ji Posted March 6, 2019 Share Posted March 6, 2019 2 hours ago, comet424 said: so I got it to rsync had to get help for port forwarding on pfsense.. but that worked.. I don't think you should be using port forwarding for rsync over ssh as this would expose the unraid servers ssh to the internet and with the weak password you probably have set (based on our long debugging exercise), your server would get hacked in a matter of minutes. 2 hours ago, comet424 said: now Id like to know @ken-ji have you gotten the email to text you once its complete to send you a email or text if there was errors.. as I cant get notifications either to email me or text me This means you need to add to the rsync script some stuff that will email/ notify you. And no, I don't have such a thing setup on my end either. but since I do run something every day to backup to Dropbox, my PC would show a notice about files being added or deleted daily. Finally, you can try running this after you rsync depending on which is the appropriate one. chmod -R 777 /mnt/user/Mikes Files/Documents chmod -R 777 /mnt/user/Mitchs Files/Documents Quote Link to comment
tr0910 Posted March 6, 2019 Share Posted March 6, 2019 (edited) Regarding hacking ssh, yes your server will get probed if you have port 22 (default port for ssh) forwarded in your router firewall. Without a port being forwarded, you cannot connect via ssh and do ssh rsync to my knowledge. I use the ssh plugin and change the ssh port to something between 50000 and 60000. This eliminates the probing. I also remove password access to ssh once I have this working well and test by rebooting a few times on both servers. Only then is it safe to remove password login, as you might lock yourself out. See the plugin on the unRaid app store. As @ken-ji has warned you, you should not use this plugin, until you have your server working stable and rebooting it a few times using the standard port 22. Edited March 6, 2019 by tr0910 Quote Link to comment
comet424 Posted March 6, 2019 Author Share Posted March 6, 2019 @ken-ji@tr0910 as for port forwarding I have no other means of getting behind the router if I don't forward port ... and if I change the password since yes its weak.. does that mean I gotta change the keys too.. I haven't tested many reboots I did before I took to my sisters.. and I tried 3 times rysnc over internet and script so that worked.. I did the rysnc mitchsserver.dnsaddress.com and did the ssh mitchsserver.dnsaddress.com and I was able to do that so ok pick a port in the high numbers not 22.. and I thought using ssh with private and public key you cant get hacked.. because you are using the ssh key... and ok as for the plugin haven't reinstalled it yet... but what does it have the regular ssh doesn't ? Quote Link to comment
John_M Posted March 6, 2019 Share Posted March 6, 2019 32 minutes ago, comet424 said: I have no other means of getting behind the router if I don't forward port The safe way to do it is via a VPN, as already discussed in this thread. Quote Link to comment
ken-ji Posted March 7, 2019 Share Posted March 7, 2019 @comet424 Would like to recommend you look up pfsense site to site VPN as that will make your network and your sister's seem like directly connected subnets and allow you to administer stuff as if it was all at home, with just additional latency. Like I said I use three Mikrotik routers providing site to site VPNs. and since we have the same ISP is in the same city, I can access my server content as if I was at home from my parent's and sibling's place. I don't have an offsite backup plan yet and am still working on it, but really important stuff (for me) is backedup to dropbox and google drive so if I loose my NAS I will get annoyed, but its not the end of the world. Quote Link to comment
ken-ji Posted March 7, 2019 Share Posted March 7, 2019 On 3/6/2019 at 10:01 PM, comet424 said: and ok as for the plugin haven't reinstalled it yet... but what does it have the regular ssh doesn't ? The plugin is supposed to provide a UI for tweaking the SSH configuration, like preventing SSH access via passwords, which if you port forwarded the SSH service, will prevent access unless the private key is present. There a few more settings, but I never bothered with the plugin, as I know enough to just edit the config files directly. Quote Link to comment
comet424 Posted March 12, 2019 Author Share Posted March 12, 2019 sorry for delay... @John_M @ken-ji so I find that rysncing over internet to my sisters server doesn't work if her ip address changes.. I find I have to keep reading the yes in the SSH command to add the ip address it doesn't add the dns adder name instead.. as for the VPN I was the one that brought it up about using a VPN but OPen VPN doesn't work on unraid.. 1.. unable to connect Client to Server Unraid by peter_Sm so for its been usless it can never connect when I set it up locally on my network or when I tried to connect over the internet to my sisters house.. also there client unraid locks out Unraid when you connect Unraid OpenVPN to PFSENSE OpenVPN Server 2.. I tried the Open VPN AS and web gui fails 99 percent of the time page cant be found worked once but tired again gone again.. 3.. I don't think I can do pfsense to pfsense as it problem drain my internet connection and I find it mess's up my own network DNS for naming computer IPS usually disconnect it just to use my own... and I only have a 5Mbps/500k Download/Upload connection so im sure its just going to kill my connection when I don't need to be connected 4.. I have tried couple topics no one could help me for a OpenVPN -Connect run rysnc - OpenVPN - Disconnect.... no one could help me with that issue so I gave up trying OpenVPN Quote Link to comment
JorgeB Posted March 12, 2019 Share Posted March 12, 2019 23 minutes ago, comet424 said: I have tried couple topics no one could help me for a OpenVPN -Connect run rysnc - OpenVPN - Disconnect.... no one could help me with that issue so I gave up trying OpenVPN Have you tried zerotier? There's a docker for it, very easy to use and configure. Quote Link to comment
comet424 Posted March 12, 2019 Author Share Posted March 12, 2019 @johnnie.black nope ill look it up... All I ever knew was OpenVPN as a Vpn program.... and I followed Space Invaders Video For Open VPN AS for unraid.. didn't matter page cant be found.. and I tried it on 2 different Web browsers... but ill look up this Zerotier now Quote Link to comment
ken-ji Posted March 13, 2019 Share Posted March 13, 2019 (edited) You can try this video on pfsense and site to site via openvpn. But I did read great things about zerotier Edited March 13, 2019 by ken-ji Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.