Unraid v6.12.13

Hi All - wanted to put this to the community to get recommendations for the best and easiest way to achieve the outcome I want.

My friend and I have both recently spun up our own Unraid servers at our respective houses. Part of the appeal for moving to Unraid was to become "self hosted" and have a solid platform to run the Arrs suite on so we could cancel our subscriptions to the various streaming services and save money. We've since followed the trend and migrated to Immach to replace Google Photos and NextCloud to replace OneDrive.

Now we're reviewing our backup strategy and we know any good backup solution follows the 3-2-1 methodology. When considering our options for the 'offsite backup' part, and in keeping with the self hosted theme we've been going for, we had the idea to setup an SMB share on each others Unraid server that the other could do a once a week backup too. 

The connectivity to each others server was handled very easily thanks to the server-to-server built in Wireguard VPN, but the backup to SMB share has me scratching my head. It should be simple in theory, but because of the following considerations it's become unexpectedly difficult.

Considerations:
- Each server should have full RW access to a dedicated share on the remote server setup for this purpose
- Each servers should be unable to read data inside any additional shares the remote server is hosting
- The backed up data should be encrypted so that the owner of each server cannot (without great difficulty) read the contents of the others backup data

To meet those considerations, we each created the other a local SMB share with a user account for the other person set with full RW access to the share. I came across Duplicati after some research which supports client side encryption - great!

The initial strategy was to enable the FTP service and have the other person use Duplicati to backup to the share via FTP. I soon found unfortunately that the FTP functionality built into Unraid was super basic and that granting a user account FTP access gave them permission to traverse the whole server. This is a deal breaker as while we don't suspect the other of malicious intent, we do want to follow best practices and minimise our attack surface by not giving the other person more access than is needed.

Then we thought to leverage the SMB protocol and its permissions by backing up directly to the SMB share and inputting the required user credentials. Unfortunately Duplicati doesn't seem to support backing up to an SMB share and I haven't had any luck finding a backup app that does.

So I'm putting it to the communicaty - given our setup and considerations, how would you achieve what we're trying to do?

Thanks very much in advance

there quite a few docker solution from rclone to synching to user scripts...
I'm still partial to rsync over ftp...

It sounds like you need a vpn instance or are both systems on the same lan?

In whcih case looking into free VPN: twingate docker / tailscale plugin to make a vpn connection to each other...

 

It sounds like you're on the right track with a self-hosted backup solution, but the specific requirements around permissions, encryption, and protocol limitations with Unraid have made things a bit complex. Let’s break down a practical approach:

 

1. Wireguard VPN and SMB for Controlled Access

Since you’re using Wireguard for VPN, your connection to each other's servers is secure, which is a solid start. To limit access to specific shares on each Unraid server, consider creating a dedicated shared folder on each server and assigning strict permissions to it.

SMB Permissions: Ensure each server’s user has access only to this dedicated share by carefully setting SMB share permissions within Unraid. Unraid's user system is limited, but a specific user account with exclusive access to a single share on each server should help prevent accidental access to other data.

 

2. Client-Side Encryption with Duplicati

Duplicati is a good choice for handling the client-side encryption requirement, as it will allow for backups to be encrypted before being stored on the remote server. While Duplicati doesn’t support SMB directly as a destination, there’s a workaround that could address this limitation:

Mount SMB with rclone: Use rclone to mount the SMB share locally on each Unraid server. Duplicati can then back up to the mounted location, as rclone allows connecting to an SMB share and making it appear as a local folder. This way, Duplicati will see the SMB share as a local path.

Setting Up rclone for SMB: On each server, install rclone, configure it to mount the remote SMB share, and then point Duplicati to the local path where rclone mounts the remote share. This allows you to retain SMB’s permission handling while still backing up with Duplicati's encryption.

Automate rclone Mounting: You can create a script to mount the remote share at boot or on demand to make sure the connection is persistent whenever a backup job runs.

 

3. Testing and Automating the Backup Process

Initial Setup Test: Once you’ve configured rclone and tested Duplicati’s backup to the mounted SMB share, perform a few test backups to ensure everything is functioning as expected. Check the encryption settings in Duplicati to verify that the data is being stored encrypted.

Scheduling the Backup Jobs: Duplicati allows you to set up a backup schedule, so configure it to run your backups weekly as you planned.

 

4. Additional Considerations

Health Checks and Monitoring: Enable email notifications in Duplicati to alert you of any issues with the backups. Also, consider running regular tests to ensure the backups are accessible and valid, though encrypted, for peace of mind.

Data Privacy and Access Control: While Duplicati’s encryption will make it very hard for either of you to access each other’s backups, it’s worth reviewing the encryption passphrases and using strong, unique keys for each backup job.

 

This setup should meet your requirements and keep everything relatively simple and secure, leveraging existing tools without needing FTP.

 

I ask as if on the same ISP and group like my friends and I we can use the ISP switch to make a loop on their device to act as a su lan to connect... Regardless, for you and your friend both unraid instance will need to be tied to a vpn to make a this IP = this machine and make a smb Mount...

To start i would make a ftp copy onsite with both of you present as it better to have 1 full done local before setting up a remote access and backup...

Edited November 11, 20241 yr by bmartino1

This is what I do. I'm more of the sysadmin between my friends so i have all access to the servers... in your case you want to setup limited access...

Step 1 vpn. WireGuard alone on unraid for me was too open... especial with sharing diag and pictures from the test machine...

I tried tail scale its not as secure but uses WireGuard underneath ( It worked, but left due to security issues....) I personally currently use twin gate as a vpn with brokerage server... (meaning you have to trust twingate / tailscale to Handel the ssh/tunnel....)

Step one guarantee a remote vpn to ping and see devices over the lan:
You can try a vpn corkage system as a test.

*Unraid recently partnered with tailscale... 

Both are free... Tailscale uses WireGuard underneath ...
tailscale and twingate
Both are free with upto 3 users and ?atleast 5 or more devices...

I would recommend tailscale for a test... seeing as Unraid recently partnered with them...

tailscale:
Make a tailscale account:
https://login.tailscale.com/admin/machines 
install unraid plugin.

install the app on phone.
remote device install and connect:
https://tailscale.com/download/windows

or Try Twingate:
Make free account: https://auth.twingate.com/signup-v2
install unraid twingate docker from community app store
add the auth and login to docker
install twingate on phone/remote device...
https://www.twingate.com/download

in twingate you can make 2 admin users and setup remote lan or sinagle IP to access xyz target from each other. 

as example you can setup single user on premison policy for the vpn to only allow user (Machine...) and access to 1 IP on your lan:

 

this way when they are on the "vpn tunnel" they can only talk to this machine...
In my case will has a iphone using photsync app that talks to this to auto backup pictures at night/bedtime...

I would also recomend the docker sftp. and not use unraids ftp / ssh in favor of that.
You can then add paths and users to a secure ftp server with fail to ban prebuilt...



run default sftp setting appdata pathing first. after first run it wills top with error no accounts...

then open the sshd folder:
Make user conf (this makes the useers who can login...)

root@BMM-Unraid:/mnt/user/Dockers/SFTP/sshd# cat users.conf 
admin:password:1000:100
brandon:password:1001:100
will:password:1002:100
elliott:password:1003:100
logan:password:1004:100
root@BMM-Unraid:/mnt/user/Dockers/SFTP/sshd# 

Then edit docker and add paths for folder you want shared... and put it in container path /home/%username%/%random_folder_name%
as you can see I structured my data into a user folder...

I have a sftp user password for me, will, logan elliot. and can use ftp, if they ftp they only have access to that one folder... nothing more... --Yest you can grant more than one folder....

with sftp anything in the home user folder as a folder name will be in ftp:

 

I often use file zilla as a go between form auto back scripts between my friends in a ring network. but it starts with being able to see them as if it was another machine in your home... 

so VPN in my case twingate...
Next sftp docker. (some filing to get users) 

userscirpt plugin, rclone plugin, duplicate

Edited November 11, 20241 yr by bmartino1

Thanks all for the fantastic advice!

I will look into RClone and an associated startup script to map each others SMB share locally.

 

Any advice on installing this would be amazing but I'm about to google-educate myself regardless.

Looks like I can get it to mount with the below command:

mount -t cifs -o username="username",password="password" //[remote_IP]/[remote_share] /mnt/remote_backup_folder

 

Now I'm looking for a script that can check whether the share is mounted already and if not then re-run the above command to mount it. I've installed the User Scripts plugin but am googling for now.

Question: why would I need to use Rclone if I can mount the remote SMB perfectly fine using the command above natively?

Quote

Question: why would I need to use Rclone if I can mount the remote SMB perfectly fine using the command above natively?

As explined in my first reply... you don't... 
that to use Duplicati as it doesn’t support SMB directly as a destination, Thats a workaround...

 

Quote

Looks like I can get it to mount with the below command:

mount -t cifs -o username="username",password="password" //[remote_IP]/[remote_share] /mnt/remote_backup_folder

 

Now I'm looking for a script that can check whether the share is mounted already and if not then re-run the above command to mount it. I've installed the User Scripts plugin but am googling for now.

here you go:
 

Here's a simple script you can use in the User Scripts plugin to check if the remote share is already mounted, and if not, mount it. This script will check if the mount point exists and, if it’s not already mounted, will execute the mount command.

Script to Check and Mount SMB Share

 

#!/bin/bash

# Define variables
REMOTE_IP="your_remote_IP"
REMOTE_SHARE="your_remote_share"
MOUNT_POINT="/mnt/remote_backup_folder"
USERNAME="your_username"
PASSWORD="your_password"

# Check if the share is already mounted
if mountpoint -q "$MOUNT_POINT"; then
    echo "Share is already mounted at $MOUNT_POINT."
else
    echo "Share is not mounted. Attempting to mount..."
    mount -t cifs -o username="$USERNAME",password="$PASSWORD" //$REMOTE_IP/$REMOTE_SHARE $MOUNT_POINT
    
    # Check if the mount was successful
    if mountpoint -q "$MOUNT_POINT"; then
        echo "Share successfully mounted at $MOUNT_POINT."
    else
        echo "Failed to mount share at $MOUNT_POINT."
    fi
fi

 

Explanation of the Script

Variables: The script defines variables for the remote IP, share name, mount point, username, and password.

Check if Mounted: It uses mountpoint -q to check if the specified mount point is already mounted.

Mount Command: If the share isn’t mounted, it attempts to mount it using the mount -t cifs command with the specified credentials.

Verification: After attempting to mount, it checks again if the mount was successful and outputs an appropriate message.

Adding to User Scripts

Open the User Scripts plugin in Unraid.

Create a new script and paste the above code.

Set the script to run on a schedule that suits your backup needs or run it manually as needed.

This script should help keep the mount active and automate the mounting process whenever it's needed.

bmartino1, you absolute legend.

This is exactly what I need and will suit my purpose fine. I'll just have the script you supplied execute at array start and every hour after that to ensure the mount point sticks.

I'm sorry I still don't understand what advantage Rclone offered over this solution. I get your point that Duplicati doesn't support backing up to SMB shares out of the box and so we need to mount the remote SMB share to a local folder that Duplicati can use as a backup destination.

It appears to me we can use the inbuilt 'mount' command as demonstrated in your script above to accomplish this. So what's the benefit of using Rclone? Is it just an alternative way to mounting SMB shares locally? Is there any advantage to using Rclone for this purpose over the built in 'mount' command?

rclone uses rsycn commands and rsync can copy things such as file permission if needed. I'm going back though a lot of old post with other users on similar setups. I personally use SFTP docker for user and control and duplicati via sftp over ftp connections.

web rsync per other users is what rclone is. it may not be needed in your use case.

its to answer the questions of ACLS and file permissions:

who touched what where. and Who can touch what where.

Edited November 11, 20241 yr by bmartino1