bmartino1

in terms of setup and install review:
 
 
 
 

use lxc move your dockers to teh lxc for applciation to installer adtional packages.

Docker is desinged to remove all non essential packages.

if its ubuntu/debain based..
 
# Use your base Docker image (replace with your custom image) FROM your-docker-image:latest # Install iperf3 RUN apt-get update && apt-get install -y iperf3 && rm -rf /var/lib/apt/lists/* # Expose the iPerf3 server port EXPOSE 5201 # Run iPerf3 server by default CMD ["iperf3", "-s"] you can grab your telgraph docker image and pull it and the add the binary and 3rdparty docker run your own docker image....

But for ease of managment your better off installing telgraph etc in a vm/lxc... that way you can install extra binaries..

Conclusion/Summary:
 
Your current strategy is robust, and seems well implemented.... with Duplicati handling both onsite and offsite backups. To optimize your setup:
Consider increasing Duplicati's volume and block sizes to improve performance for large backups.
Use Resilio Sync for real-time synchronization needs, but rely on Duplicati for comprehensive backups and versioning.
Regularly monitor your backup destinations to ensure adequate storage space and verify the integrity of your backups.
 
So Let's review your current backup strategy and address your questions to ensure a robust 3-2-1 backup approach.
 
Onsite Backup:
You've implemented Duplicati for onsite backups, utilizing its smart retention feature. This is a solid choice, as Duplicati offers efficient backup management with customizable retention policies.
Regarding your use of Resilio Sync locally, it's important to note that while Resilio Sync facilitates real-time file synchronization, its versioning capabilities are somewhat limited. By default, Resilio Sync stores previous versions of files in the .sync/Archive directory for 30 days on desktops. This duration can be adjusted using the sync_trash_ttl parameter in the advanced settings.
 
Review/See more here: https://help.resilio.com/hc/en-us/articles/204754239-Using-Archive-for-file-versioning-and-restoring-deleted-files
 
Addressing Your Questions:
Yes, your understanding is correct. Resilio Sync's versioning is managed through the .sync/Archive folder. To ensure this feature is active, verify that the "Store deleted files in Archive" option is enabled in the folder preferences. You can adjust the retention period by setting the sync_trash_ttl parameter to your desired number of days.
 
Default...
The default is 30 days, which is suitable for most users. However, depending on your storage capacity and recovery needs, you can adjust this period. Setting it to zero will retain versions indefinitely, but be cautious as this can consume significant storage space over time.
 
Since Duplicati provides comprehensive backup solutions with advanced retention policies, maintaining Resilio Sync locally might be redundant. However, if you require real-time synchronization across devices, Resilio Sync can complement Duplicati by ensuring immediate file consistency, while Duplicati handles scheduled backups and versioning.
 
###########

Next lets review:
Offsite Backup
For offsite backups, relying solely on Resilio Sync's versioning may not be sufficient due to its limited versioning capabilities. Implementing Duplicati for offsite backups would enhance your data protection strategy. SFTP is a secure method for transferring backups offsite, and Duplicati supports SFTP, making it a viable option for your needs.
 
Additional Considerations:
Duplicati Volume Size:
The default volume size in Duplicati is 50MB. For large backups, increasing the volume size can improve performance. Users with substantial data sets have reported better performance with volume sizes ranging from 250MB to 1GB. Adjusting the deduplication block size from the default 100KB to a larger size (e.g., 1MB) can also enhance performance, especially for large backups.
As seen here: https://forum.duplicati.com/t/backing-up-2tb-50mb-remote-volume-size-okay/11912
 
Backup Compression
Duplicati applies compression to backups, which can result in smaller backup sizes compared to the original data. The extent of compression depends on the data types being backed up. For instance, text files compress well, while already compressed files like videos or images may see little to no reduction in size.
 
Warning Message
The warning regarding backend quota being close to exceeded is a known issue in Duplicati, where it may misreport available space. If your destination drive has ample space, this warning can typically be disregarded. However, it's advisable to monitor your backup destination to ensure sufficient space is available.
 
 

plex realy should be its on folders...

I would run a script and symlink the .flac files to a folder and pass that to plex.
Let Qbit access the /mnt/user/music ...

so my suggestion would be to add a share lets call it plex.
then using termianl maeka  music folder within... this will be the plex docker path for music...
 
#!/bin/bash # Variables SOURCE_DIR="/mnt/user/music" TARGET_DIR="/mnt/user/plex/music" # Ensure target directory exists mkdir -p "$TARGET_DIR" # Traverse the source directory to find all .flac files find "$SOURCE_DIR" -type f -name "*.flac" | while read -r file; do # Get the relative path of the file relative_path="${file#$SOURCE_DIR/}" # Get the directory of the file relative to the source file_dir=$(dirname "$relative_path") # Create the corresponding directory structure in the target directory mkdir -p "$TARGET_DIR/$file_dir" # Create a symlink to the .flac file in the corresponding target directory ln -s "$file" "$TARGET_DIR/$relative_path" done echo "Symlinks for .flac files have been created in $TARGET_DIR"  
This will create a symlinked structure under the target directory....
Example output:
/mnt/user/music/ ├── Artist1/ │ ├── Album1/ │ │ ├── track1.flac │ │ ├── track1.mp3 │ │ └── track2.flac │ └── Album2/ │ └── track3.flac └── Artist2/ └── Album1/ ├── track1.mp3 └── track2.flac the symlink example output:
/mnt/user/plexFLAC/ ├── Artist1/ │ ├── Album1/ │ │ ├── track1.flac │ │ └── track2.flac │ └── Album2/ │ └── track3.flac └── Artist2/ └── Album1/ └── track2.flac This way, only .flac files are visible to Plex while maintaining the original folder structure.
 
mby install the user script plugin and cron that script to run every 3 hours or so.... if your source changes from time to time. this way plex doesnt' touch the real files but can use the files form that directory...


usualy what is known as teh *arrs have a folder structure hierachy...
so snar, qbit torrent, plex all have there directies shared with in 1 location...
what your fighting atm is file structure and who has access to what.

you may need to add extra option like puig,pgid and umask when sharing the same folder across mutiple dockers...

look into the rclone plugin for web rsync
 
https://rclone.org/docs/

Review:
 
 
 
 
 

the reason for tailscale is to use a tailscale VPn to connect to your reverse proxy with the use or need to portforward your SAWG to the public internet.

Yes you don't need tailscale in that setup, but this would require you to port froward. Most ISP have moved towards CGNAT. you would need a public IP that is not CGNAT to use your public IP as the "SWAG" reverser proxy

Lets Review:
 
Understanding Your Current Setup
You are using:
SWAG as a reverse proxy: To route traffic from a domain to specific containers (e.g., Nextcloud, Emby).
Tailscale: To create a secure, private VPN mesh network for remote access without opening ports on your router.
Cloudflare DNS: To map a domain name to the Tailscale IP.
 
Key Points to Consider:
Can You Replace SWAG with Tailscale?
Yes, you can eliminate SWAG if Tailscale meets all your needs. With Tailscale, you don't need to expose services directly to the internet, removing the need for a reverse proxy for secure access.
Using Tailscale IPs or Tailscale's built-in domain (e.g., container-name.ts.net) allows direct access to your containers.
 
Drawbacks of Forgoing SWAG
No Centralized Proxy: Without SWAG, each container must independently handle its own HTTPS (via Tailscale’s HTTPS serve feature or container-level SSL/TLS configuration).
Complexity for Custom Domains: While Cloudflare DNS can map a custom domain to a Tailscale IP, managing certificates for HTTPS on custom domains becomes more challenging.
Limited Features: SWAG offers advanced features like URL rewrites, subdomain routing, and security headers. Without it, you’d need to replicate this functionality manually.
 
When to Use Tailscale Serve
If you forgo SWAG, you can use Tailscale's "Serve" feature to map specific HTTP paths to your container services. For example:
tailscale serve /nextcloud http://localhost:8080 tailscale serve /emby http://localhost:8096 This method is simple and works for internal traffic but doesn’t natively support complex configurations like SWAG.
 
Port Forwarding vs. Tailscale
Tailscale removes the need for port forwarding by creating a private network accessible from anywhere.
If your ISP uses CGNAT (Carrier-Grade NAT), Tailscale is a more practical solution since CGNAT prevents direct access via a public IP.
 
Recommendation Based on Your Use Case
 
Use SWAG with Tailscale: If you value SWAG's advanced reverse proxy capabilities, you can keep SWAG behind Tailscale. This setup involves:
Cloudflare DNS -> Tailscale IP -> SWAG -> Containers.
SWAG still handles routing, HTTPS, and other features, but external traffic reaches it securely through Tailscale.
 
Replace SWAG with Tailscale: If your needs are simpler (direct container access), Tailscale can fully replace SWAG:
Cloudflare DNS -> Tailscale IP -> Containers.
Use Tailscale Serve or direct mapping of Tailscale IPs for container access.

docker setting and vlans is a known issue if you have a misconfiguration.
Some had issues with missing disk again due to misconfigurations...

Things you need to know...

Read the release notes and announcement page...
https://docs.unraid.net/unraid-os/release-notes/7.0.0/

And know issue per the announcement side of the forum!
Example post:

Many have updated without issues. I recommend updating sonnet then latter...


ALWAYS MAKE A FLASH BACKUP...
WebUI > Main > Flash > Flash Backup

as this can overwrite the current flash drive and bring you to a working state!

Other than that, no... no issues from me in regard to upgrading. Per base info you shouldn't have an issue.

Zfs addons for vdev and metadata holding to help increase disk speed read write and data replication/deduplication.

Overview of ZFS Sub-Pool Drive Options in Unraid:
1. Special Devices
Purpose:
Special devices are used to store metadata, file system metadata (like directory entries), and optionally small files separate from the main data vdevs.
Benefit:
Offloading metadata to faster storage (e.g., NVMe drives) improves performance for metadata-intensive workloads, such as managing a large number of small files or directories.
Use Case:
Use an NVMe drive as a special device to boost metadata handling, especially in pools with HDDs or slower SSDs.
 
2. Logs (ZFS Intent Log or ZIL)
Purpose:
ZIL is used for synchronous writes, ensuring data consistency in the event of a crash or power failure.
Adding a dedicated log device (often called an SLOG) separates this functionality from the main pool, improving performance for workloads with frequent small, synchronous writes.
Benefit:
Reduces latency for synchronous writes, which is critical for databases or virtual machine storage.
Use Case:
Use a high-speed, low-latency NVMe or Optane drive as a dedicated log device.
 
3. Cache (L2ARC)
Purpose:
Acts as a second-level Adaptive Replacement Cache (L2ARC), extending the primary memory-based cache (ARC) using a faster storage medium (e.g., NVMe or SSD).
Benefit:
Speeds up read performance for frequently accessed data that doesn’t fit into the system’s RAM.
Use Case:
Add an NVMe drive as a cache to accelerate read-heavy workloads when the ARC in RAM is insufficient.
 
####################
Your Setup and Cache Drive Question
Current Setup:
ZFS RAID of 4x 2TB Samsung 870 SSDs (3.7TB usable space).
Adding an NVMe Cache Drive:
Would it speed up your ZFS pool?
Yes, adding an NVMe drive as a special device or cache (L2ARC) would likely improve performance.
However, adding an NVMe drive as a log (ZIL) would mainly help if you have workloads with significant synchronous writes, like databases or VMs.
Specific Recommendations:
Use the NVMe drive as a special device to offload metadata and small files, especially if you manage a lot of smaller files.
If your workload is read-heavy, configure the NVMe drive as cache (L2ARC) to improve read performance.
Only configure the NVMe drive as a log (ZIL) if you know your workloads require fast synchronous write operations.
Compatibility:
Ensure the NVMe drive has sufficient endurance and reliability, especially if it’s used as a log device since it will handle many write operations.
 

read the release notes...

you may need to use overlay 2 ...
https://docs.unraid.net/unraid-os/release-notes/7.0.0/#add-support-for-overlay2-storage-driver
 
I have not had issues using native and a xfs docker image...

this will be fixed with 7.1 hoepfully with the move to kernal 6.12 lts that has teh driver support I belive there is a drive b=pluign to install in the meantime.

you can add uid and guid via adding variables.

per the docks to use tail scale you need to run the docker as previlaged.

please post tailscale quetion on the tailescale side of the forum: 
https://forums.unraid.net/forum/111-tailscale/

review docs found in here:
 

? is luck backup a docker you are running or is this on the host system?

unraid has a plugin to called user scripts to set scripts up on the host level and to run crons...

?Docker:
I assume the docker...
 
did you follow the warning and check the checkbox?

Cron: If you create a cron job please be sure to tick the "Console Mode" checkbox, otherwise the cron jobs will not work.

your on unriad 7 yes?

there have been some changes to the unraid docker system...

I recommend deleting the docker image and recreating it using XFS...

 

 
Depending on your pool setup You may need to use overlay 2 as per the release notes and new docs

Apple changed something in macOS with a update, you may need to clear browser data in chrome and in safari...

as chromes web engine on apple OS use safari web browser engine and data can get crossed due to that engine through another web browser through the app...

MacBar at top > safari > in the menu click "Settings" or "Preference" (Depending on your apple OS version...)
In the new window clcik the hand icon [Privacy tab] > then click the button [Delete Website Data]
In this new windows, may take some time to populate...
Button bottom right remove all

Then close the windows and go to macbar > sarfi > quit

it can. very unlikley though... if its in a hosed state the heads never left and leaving it in that state i would say hurts the disk more as that just adds unecessay power on hours and rpm drive rotation to HDD and SDD... Hard shut it down...

grub option consolebalnk=0

v7 unraid web ui > settings > console > disable balnking...

you will need to stop the array to edit settings. I think you need permit exclusive shares set to yes.
the global share setting look correct.
 
 

sounds like you have a revers dns loop goin on.

pihole and unbound get a resolv.conf which is their dns servers from unraid. unraid dns settings may not have a public ip our gateway router IP correct for the docker resolve config to work correctly.

try setting unraid dns ip to google, Cloudflare and cisco public ip
8.8.8.8
1.1.1.1
208.67.222.222
 
and turn off pihole and unbound to see if it works bring them on one at a time for testing...
 
as this seems more like a ISP internet connection issues or internal dns misconfiguration issue.

you may need a 3rd party driver install. Unraid uses a compiled and striped down verison of the kernel. Doesn't mean the driver made it into unraids Slackware kernel...

 

Yes, WireGuard works perfectly with Fully Qualified Domain Names (FQDN) as the peer endpoint. This is a common setup for dynamic DNS (DDNS) configurations, where the server’s IP address may change due to the ISP.
 
Here’s how it works and considerations regarding security
 
########
How FQDN Works in WireGuard
WireGuard will resolve the FQDN to an IP address when the tunnel is initiated.
If the IP address changes during an active session, WireGuard won't automatically re-resolve the FQDN. A manual restart of the tunnel or rekeying is required to update the endpoint IP address.
#########
 
Using a Cloudflare DDNS + CNAME for WireGuard

 
Your approach with a non-proxied (DNS only) CNAME record is correct. Here’s why and what to check:
Advantages:
Dynamic IP Support: By pointing your client to the FQDN (e.g., ddns.yourdomain.com), it will always resolve to your Unraid server’s updated WAN IP.
Direct Connection: A non-proxied (DNS-only) CNAME avoids Cloudflare’s CDN and proxies, allowing the WireGuard client to reach your server directly.
 
Security Considerations
Non-Proxied Exposure: Since the CNAME is set to DNS-only, your server’s WAN IP address is directly exposed. This is necessary for WireGuard but increases exposure to potential scanning and attacks.
-- Mitigation: Use a strong WireGuard private/public key setup. Only authorized peers with the correct keys can establish a connection.
 
TLS vs. WireGuard: Unlike HTTPS behind a Cloudflare proxy, WireGuard handles encryption at the network layer. It doesn’t rely on Cloudflare for security, so the lack of proxying does not reduce WireGuard’s security.
 
DNS Security: Ensure that the DNS settings for your Cloudflare domain are properly secured (e.g., strong API keys, two-factor authentication). This prevents an attacker from modifying DNS records.
 
Alternative Approach:
If you’re concerned about exposing your WAN IP:
Consider using a dedicated subdomain for WireGuard, e.g., vpn.yourdomain.com, to avoid using the main domain’s records.
Set up Cloudflare Zero Trust (if you want additional security layers) to protect DNS changes or other services.
nginx reverse proxy server...
 
Review your steps and Configure WireGuard with FQDN:
Set up a non-proxied (DNS-only) CNAME or A record in Cloudflare for your dynamic IP.
Example:
CNAME: ddns.yourdomain.com → <dynamic IP updated by DDNS container>
Use this domain (e.g., ddns.yourdomain.com:<port>) in your WireGuard client configuration as the Peer Endpoint.
Ensure your DDNS Docker container updates the A record reliably.

(Commands like dig and nslookup can be useful here)
 
It sounds like you found the area to set....

you need to edit your domain share and eithe have the data set to cache or disk1 per your setup.
 

^as you have this under domains configured... as i'm not sure which way you pointed it... but mover will move the file from 1 location to the next.

you need the vm off and have mover run to move the data per its current configuration.
(this is to rescue the VM vdisk image as it currently split in 2 disk locations...) otherwise that VM is now dead ...

as atm this should only exist either on disk 1 or cahce per your previous picture data...

To fix so this doesn't happen again...
Then edit the domain share and ONLY HAVE Primary storage set.

Now that mover has it in one location and the domain share is fixed, the vidsk as 1 file is located somewhere that need fixed in the VMs XML.
(not enoth info to assit. need picture of your share tab...)
 
As Normally vms are stored on the cache disk.
You shouldn't use the fuse system /mnt/user it should be a direct path /mnt/disk1/domains etc... when fixing hte vms xml to restore where teh vdisk1.img is located...

olama like open web UI are jsut front end interfaces AI depending on the modle may need more Gcard Vram. a single nvida 2080 can barely run a 2Billion ai modle with decent performace. 
 
local AI docker.
open web Ui Docker
etc are alos resource intensvie when runnign a AI system you need ALOT of resources. the delay is most likely caused by ram loading and unlaoding in the gcrads vram you need antoher gcard to help with the AI tasks.

For what file type ?rar 7z tar gz ...

unzip is installed by default...
 
example
unzip filename.zip -d /path/to/extract/
Replace filename.zip with the name of your file.
Replace /path/to/extract/ with the destination folder.

3rd party app install:
https://pkgs.org/search/?q=unrar
 
cd /boot/extra wget https://slackers.it/repository/slackware64-current/unrar/unrar-7.1.2-x86_64-1cf.txz
?Unrar Files
unrar x filename.rar
 
example
unrar x filename.rar /path/to/extract/
Replace filename.rar with the name of your RAR file.
Replace /path/to/extract/ with your desired directory.

 

For space and preformance 4 disk in a raid z1 with 1 vdev...
 
RecommendationL
If your priority is performance, choose the dual mirror vdev pool.
If your priority is maximum fault tolerance, RAIDZ2 is the better option.

Using ZFS with your proposed configuration (a dual mirror vdev pool) provides a robust level of fault tolerance and performance, but it comes with specific trade-offs and behaviors. Let's break down your questions
 
If a single disk is lost from one of the mirror vdevs (e.g., disk1 or disk2 from mirror-0, or disk3 or disk4 from mirror-1), the pool remains online and fully functional.
 
-The remaining disk in the affected mirror will continue to serve data for that vdev.
 
-The pool can tolerate this scenario because mirrors are designed to provide redundancy.
 
-It's crucial to replace the failed disk and rebuild the mirror as soon as possible to restore redundancy.
 
No, you cannot lose an entire mirror vdev and still have a functioning pool.
 
-In ZFS, the pool is only as resilient as its individual vdevs. If a vdev (e.g., mirror-0 or mirror-1) is entirely lost, the pool becomes unusable because the data distribution across vdevs requires all vdevs to be available.
 
-To ensure pool integrity, at least one disk from each vdev must remain operational.
 
 
In this scenario, the pool remains functional:
Each vdev still has one healthy disk (disk2 for mirror-0 and disk4 for mirror-1), so the pool can serve data without interruption.
However, redundancy is completely lost for both vdevs. If another disk fails before replacements are introduced and rebuilt, the pool will fail.
 
The pool becomes unusable in these situations:
 
Complete failure of a single vdev:
If both disks in either mirror-0 (disk1 and disk2) or mirror-1 (disk3 and disk4) fail, the pool is unrecoverable.
 
Simultaneous loss of the last operational disk in each mirror:
For example, if disk1 and disk4 fail after disk2 and disk3 have already failed, the pool will be lost.
 
Corruption across multiple vdevs beyond what redundancy can handle:
While ZFS offers mechanisms to detect and correct errors (e.g., checksumming), if corruption impacts data across all vdevs, recovery may not be possible.
 
comparisons:
 
 
 
Dual Mirror Vdev Pool vs. RAIDZ2
If you're weighing a dual mirror vdev pool against RAIDZ2, here are some points to consider:
 
Performance:
Dual mirrors generally provide better random read and write performance than RAIDZ2 because of their simpler structure and increased IOPS (input/output operations per second).
 
Capacity:
Dual mirrors give you 2 TB of usable space (50% of the total capacity).
RAIDZ2 gives you 2 TB of usable space as well, but with the ability to survive any two disk failures, regardless of which disks fail.
 
Fault Tolerance:
RAIDZ2 offers better fault tolerance than dual mirrors because it can survive two simultaneous disk failures anywhere in the pool.
Dual mirrors can only survive a single disk failure per vdev.
 
Rebuild Times:
Rebuilding a mirror is faster and involves less stress on surviving drives compared to RAIDZ2, which requires recalculating parity for the entire stripe during a rebuild.