May 19, 20251 yr Setup: Hello all, I started my unraid build a couple of weeks ago and built my pc, put unraid on the usb and started my journey. I tried doing a bunch of research and "best practice" seemed to be have an array but use a cache drive to move data onto the array (when? That is preference) which i did just that. Put in four 14tb 3.5 hdd's (2 disks for parity), two 1tb nvme's in raid for a cache drive. I setup an appdata folder to hold my docker configs and have /data on cache to move to array after a day. In theory this would download data to cache then after a day it would move to array and not break links. Great it should be all setup. Problem: I needed to move all my data that is stored on a 14tb drive in my gaming pc for temporary storage until my unraid was setup. Problem is, i couldnt move onto the cache share as it would fill up the 1tb and im not sure if i can overflow into the array with that? Plus its almost 14tb so that would be a lot of wasted writing to an nvme drive. So in my quick thinking i just made a share directly on the array itself and robocopied from the 14tb to my array that i mapped onto my gaming pc. After about 2 days its all on there. My current problem is though, i dont know how to fix my setup to have my cache drive know of the existing data and continue working with my docker containers to grab downloads and after a day move it onto the array. Any help would be appreciated. I didnt get to have a thorough look through everything and how to set it up as i just got home last night and now im at work but i thought id throw the question out there in case anyone would have a quick and obvious solution im missing (im new and still learning).
May 20, 20251 yr Community Expert the issue is the existing data at this point and where data and how mush has been written given the size and # of disk I would have recommend ditching the array and going zfs only. 4x 14TB raidz1 1TB Cache disk since you went the disk 1, disk 2 parity route and have data in between disk it will be hard to recommended a path forward. There is nothing wrong with going the default array route you did, just mentioning some extra option for more space and maybe some performance difference... https://docs.unraid.net/category/zfs/ With the zfs mention out of the way, lets see if we can dig into the actual problem you are having. It is my understanding that you have a (Please post diagnostic to help further troubleshoot or a picture of your main) Array: parity 14TB d1 14TB d2 14 TB pool Cache disk 1TB do to how you originally setup unraid, and a 14TB disk in another PC with data your trying to transfer into/on unraid... You attempted to copy data to the pre-made share cached now you made a share tied to ? disk 1? and robo copied data off the gaming machine 14TB to the array? --If I'm under standing you correctly... ...for more info on unraid and there shares, please review the docs: https://docs.unraid.net/unraid-os/manual/shares/ Edited May 20, 20251 yr by bmartino1 typo
May 20, 20251 yr Author 17 minutes ago, bmartino1 said: It is my understanding that you have a (Please post diagnostic to help further troubleshoot or a picture of your main) Array: parity 14TB d1 14TB d2 14 TB pool Cache disk 1TB do to how you originally setup unraid, and a 14TB disk in another PC with data your trying to transfer into/on unraid... You attempted to copy data to the pre-made share cached now you made a share tied to ? disk 1? and robo copied data off the gaming machine 14TB to the array? --If I'm under standing you correctly... ...for more info on unraid and there shares, please review the docs: https://docs.unraid.net/unraid-os/manual/shares/ Heya, thanks, i appreciate you taking time to help me with this. First off, yes i went the array (2 disks, 2 parity) route with cache pool (two 1tb in raid) as seen in the pictures i have attached. Second, I created data share to temporarily store data then have mover move it from cache to array. I then realized i need my data to be seen by my docker containers and i cannot move my data onto my "data" share as it will fill up the 1tb cache and reduce its life span. So, i created the "tmp_data" share that is directly on the array, mapped it to a letter on my gaming pc that has the 14tb drive in it with all my current data that i want on my unraid system, used robocopy to move it all from gaming pc drive to unraid array. Now i have the data on "tmp_data" share and not the "data" share where my docker configs are pointed to. Third, if i have to start all over its no big deal, i have backups of all my docker containers, i still have my data on my 14tb in my gaming pc, nothing is dire. I'm still trying to learn unraid and figure out a "best setup" for my use case. Let me know what else you need to help. I have read the documents and thread you have attached in your reply by the way, thank you.
May 20, 20251 yr Community Expert Solution I would highly recommend starting over then. in which case you can keep the cache mirror setup for btrfs. While we could go over share and disk setting per the unraid doc there some inherit concerns and questions that need to be done and some fixes would not be easy wiht out chaing and invoking mover which I don't trust atm due to recent build pushes and potential data loss...So, where your at now. We can use the unraid tools option and run a new disk configuration...Please stop all dockers. go to setting docker offplease go to vm manger and make sure vms are off.then on the main tab stop the array.Next we go to toolsnew configwe can keep the cace disk pool...as the goal here is to remove the arrayremoving the disk 1, disk 2, parity, parit 2.then at the main tab set array slots to 0(you may need to unassign the disk from the array at the top)make a zfs pool with that grouping of disks.Then under pool devices. you will add a 4 slot disk I don't know call it data. Come up with a name... as this will become the array moving forward...Click data(the pool name you chose) and select filesystem type zfs and chose raidz1 (or raidz2 you can lose 2 disks and keep all the data similar to your current setup)Doing this will set your disk up as a true nas raid array for redundancy in disk.(this will make a raid 5 type array where you can lose 1 disk and still have all the data on all drives.) data is striped and mirrored across the multiple disks...the reason for zfs in your case is due to disk access and its ability to increase disk read speed and increase overall data...as in your current array setup your leaving 14 TB of space where you can potently get atleast 7 TB if not more on ZFS.Review video over unraid zfshttps://www.youtube.com/watch?v=UR5RCItyCswRAID IS NOT A BACKUP ITS! Redundancy and uptime to keep system services running...with that done. We can look at some add-on plugin as linked in the top post as the plugin adds additional functionality into unraid...we can then make "datasets" which become shares and use datasets for the start of your file hierarchyreview for examples as this need to be though out and setup before data is compiled making location where xyz will be stored...https://trash-guides.info/File-and-Folder-Structure/With each dataset, we can leverage zfs for snapshots of the data for backup purposes..Example OF my main screen and some datasets...Then we can go over editing the shares tab and fixing the sahres to make sure primary is set to the corect disk example of my dataset file hiracrythen at each share and other unraid setting, we can leverage nfs/samba to share that off the machine...I would advise setting the docker and vm system image on cache and potential install the swap plugin and making a 2 GB swap volume for a just in case Out of Memory...then we can go over some side improvement script such as automatic backups, tweaks and user script settings to run automatic snapshots...Otherwise we can skip running zfs. You keep the potential ram resource that would be used running zfs, you leave the 7TB if not more on the table and relie on other script to simulate data snapshot of data that would exisit in zfs... then we can try to leverage the uinrad share tab and look into mover to move the data off.as you will be fighting the setting here between the sahres and ther data..-as the easeit way would be to come up with a file hireachy first make the new folders where you wnat to put them and refactor your data to be in taht file dataset...and then we may need to do some weird sym linking to fix which can become a mess of things... where the dataset in zfs and backups and snapshots give you more data and ease access to setting file hireachy early fixing where your placing data...As example, then all the Docker data can then become a dataset and date for docker can be stored in that dataset...as, because you have a tmp_data and data shareThis will complicate how we would need to proceed moving the data off and correcting where the data needs to be saved on the disk...in zfs all datasets would exist at /mnt/pollname/datasenameso I have a pool called vm-zfs and a dataset called dockerUnriad system host is called Docker...root@Docker:~# cd /mntroot@Docker:/mnt# lsaddons/ cache/ disks/ remotes/ rootshare/ user/ vm-zfs/root@Docker:/mnt# using unraids fuse system after making the file hierarchy uses the datasets and setting up the shares of where data exist...root@Docker:/mnt# cd vm-zfs/Dockers/root@Docker:/mnt/vm-zfs/Dockers# lsPlex/ appdata/ immich/ local_ai/ netbox-docker/ rocketchat/root@Docker:/mnt/vm-zfs/Dockers# cd /mnt/user/Dockers/root@Docker:/mnt/user/Dockers# lsPlex/ appdata/ immich/ local_ai/ netbox-docker/ rocketchat/root@Docker:/mnt/user/Dockers# the unraid fuse system for default pathing will be hard/sym link to the array for data... then its a mater of updating docker setting and vm setting where on disk or fuse system is the data to be used...as example plex temple pathing: Edited May 20, 20251 yr by bmartino1 typo-data
May 21, 20251 yr Author 7 hours ago, bmartino1 said:I would highly recommend starting over then. in which case you can keep the cache mirror setup for btrfs. While we could go over share and disk setting per the unraid doc there some inherit concerns and questions that need to be done and some fixes would not be easy wiht out chaing and invoking mover which I don't trust atm due to recent build pushes and potential data loss...So, where your at now. We can use the unraid tools option and run a new disk configuration...Please stop all dockers. go to setting docker offplease go to vm manger and make sure vms are off.then on the main tab stop the array.Next we go to toolsnew configwe can keep the cace disk pool...as the goal here is to remove the arrayremoving the disk 1, disk 2, parity, parit 2.then at the main tab set array slots to 0(you may need to unassign the disk from the array at the top)make a zfs pool with that grouping of disks.Then under pool devices. you will add a 4 slot disk I don't know call it data. Come up with a name... as this will become the array moving forward...Click data(the pool name you chose) and select filesystem type zfs and chose raidz1 (or raidz2 you can lose 2 disks and keep all the data similar to your current setup)Doing this will set your disk up as a true nas raid array for redundancy in disk.(this will make a raid 5 type array where you can lose 1 disk and still have all the data on all drives.) data is striped and mirrored across the multiple disks...the reason for zfs in your case is due to disk access and its ability to increase disk read speed and increase overall data...as in your current array setup your leaving 14 TB of space where you can potently get atleast 7 TB if not more on ZFS.Review video over unraid zfshttps://www.youtube.com/watch?v=UR5RCItyCswRAID IS NOT A BACKUP ITS! Redundancy and uptime to keep system services running...with that done. We can look at some add-on plugin as linked in the top post as the plugin adds additional functionality into unraid...we can then make "datasets" which become shares and use datasets for the start of your file hierarchyreview for examples as this need to be though out and setup before data is compiled making location where xyz will be stored...https://trash-guides.info/File-and-Folder-Structure/With each dataset, we can leverage zfs for snapshots of the data for backup purposes..Example OF my main screen and some datasets...Then we can go over editing the shares tab and fixing the sahres to make sure primary is set to the corect disk example of my dataset file hiracrythen at each share and other unraid setting, we can leverage nfs/samba to share that off the machine...I would advise setting the docker and vm system image on cache and potential install the swap plugin and making a 2 GB swap volume for a just in case Out of Memory...then we can go over some side improvement script such as automatic backups, tweaks and user script settings to run automatic snapshots...Otherwise we can skip running zfs. You keep the potential ram resource that would be used running zfs, you leave the 7TB if not more on the table and relie on other script to simulate data snapshot of data that would exisit in zfs... then we can try to leverage the uinrad share tab and look into mover to move the data off.as you will be fighting the setting here between the sahres and ther data..-as the easeit way would be to come up with a file hireachy first make the new folders where you wnat to put them and refactor your data to be in taht file dataset...and then we may need to do some weird sym linking to fix which can become a mess of things... where the dataset in zfs and backups and snapshots give you more data and ease access to setting file hireachy early fixing where your placing data...As example, then all the Docker data can then become a dataset and date for docker can be stored in that dataset...as, because you have aThis will complicate how we would need to proceed moving the data off and correcting where the data needs to be saved on the disk...in zfs all datasets would exist at /mnt/pollname/datasenameso I have a pool called vm-zfs and a dataset called dockerUnriad system host is called Docker...root@Docker:~# cd /mntroot@Docker:/mnt# lsaddons/ cache/ disks/ remotes/ rootshare/ user/ vm-zfs/root@Docker:/mnt#using unraids fuse system after making the file hierarchy uses the datasets and setting up the shares of where data exist...root@Docker:/mnt# cd vm-zfs/Dockers/root@Docker:/mnt/vm-zfs/Dockers# lsPlex/ appdata/ immich/ local_ai/ netbox-docker/ rocketchat/root@Docker:/mnt/vm-zfs/Dockers# cd /mnt/user/Dockers/root@Docker:/mnt/user/Dockers# lsPlex/ appdata/ immich/ local_ai/ netbox-docker/ rocketchat/root@Docker:/mnt/user/Dockers#the unraid fuse system for default pathing will be hard/sym link to the array for data... then its a mater of updating docker setting and vm setting where on disk or fuse system is the data to be used...as example plex temple pathing:Alright so i started over and didn't go with array and did a zfs pool as you suggested (set it up as raidz2 for more redundancy), have my cache pool, no shares/data sets, installed all the plugins you suggested in your first post linked thread. As for backups/snapshots and the like ill worry about that after the fact cause right now i don't have any data to snapshot/backup and i have an external machine i will back up everything to once i get my unraid setup and going. My question is: where do i go from here? 1. How do i move my data from my gaming pc to unraid? do i just make a share (or dataset?) and map it to my gaming pc and move it over?2. make sure i have my docker containers point to that data3. do i still need mover? 4. will i just use my cache pool for docker configs and the zfs pool for the actual data?i appreciate your help so far! Let me know if you need anything to help you help me.
May 21, 20251 yr Community Expert Perfect now that your on zfs we need to come up with a file hireachy. yes using the zfs mater plugin you can make a single dataset called samba and use unraid to share the one folder. per examplereview for mor on the zfs master pluginhttps://www.youtube.com/watch?v=phCccX--HY4Later review for space invaders zfs autoback scriptshttps://youtu.be/RTMMPHc9OoU?si=vGRhV-Qkw2ZB3qjkQ/A1. How do i move my data from my gaming pc to unraid? do i just make a share (or dataset?) and map it to my gaming pc and move it over?2. make sure i have my docker containers point to that data3. do i still need mover?4. will i just use my cache pool for docker configs and the zfs pool for the actual data?1. First we need soem datasets and come up witha file hiarchy.We will want to enable some protcall samba, nfs, sftp via ssh etc..genearl saved data, docker data, backup stagin data, and even sub fodlers...As shared earlier, as I can name them in the share tab:I have a Backup Dataset where I can place files for stagin off the machine... such as the appdata backup plugin to flash backup adn backup dockers...I have a cloud dataset, this is my general samba share that all pc conect to dump files.I have a Docker dataset this is where docker media data is set for my use such as large video file for jellyfin/embry/plex or storage for docker appication like pictures from immich...I have a dataset for the LXC Plugin for VMsI have a specail dataset PBS use to connect proxmox and use it as a nfs backup sertver for VMsI have a SFTP dataset I use with a specal Docker I run sftp-fail2ban that the ring network I have between friends as we rsyn via sftp and use each other as remote backup...Each dataset then becomes a share and edits are done to make them avalbe to share to antoher machine.Your needs may be different.Per Unraid.Some default shares exisit:Default Shares:appdata - This is the default location for storing working files associated with docker containers. Typically there will be a sub-folder for each docker container.system - This is the default location for storing the docker application binaries, and VM XML templatesdomains - This is the default location for storing virtual disk images (vdisks) that are used by VMs.isos - This is the default location for storing CD ISO images for use with VMs.Usually the CA Docker Template will auto point to the appdata folder usually this folder share is on the cache disk only. only a primay data disk is set.So if you are using the extra options form unraid and not only be a NAS, you now need a Dataset for each service, system...Unraid Doen't support subfolder sharing. you can use zfs terminal comnad and make a subdataset with n a folder.as icon shown above example my Users datraset. I could have made the names a dataset there icon is folders. as I have zfs directory listing enabled to see data disk status.I Don't know your needs. I don't know how you want to struture your data. how you want to interact with it and what you want it to do.I will advise you to make a appdata dataset for dockers(You may need to delete or remove exisitng sahre folders off disk)this is why I have a dataset called Docker as it a matter of telling the template in docker to where you want it to look at and use xyz file...so you should have atleast theses datasetsBackup: staging for latter backups such as plugin appdata backupData or Cloud: Master Dataset and Main Share from Unraid. This is where you will copy the data form the gaming machjin off and into unriad.Docker: Master Dataset for docker DataVMs: Master Dataset for LXCs/VMsIf you want You can make a dataset for system(system is where teh docker vdisk image and VMs libvirt image exisit on disk)I recommend this being on the Cached pool disk drive.As we first need dataset then we can go to the share tab and setup samba/nfs access.just fill the top partionleave everything else defaultemptythen hit createThis will make a dataset used later for snapshotsthis will make a share that needs edited latter.Make your Necessary Dataset as this is the pre file structure. as later we can make sub folder as example my sftp docker uses dataset user but acautl uses data in the usernames folder...Next we need to go to setting and fix some things...Under network services click smb and nfsNFS is the share protcal to share data via Network File System.Here we want you to mimic the same settings...At the very least turn it on.Next Settings SMB:This is Samba server the default windows sharing protcalNote the warning the array needs stopped before you can edit these settings..Notes:In your case you may not want the apple addtions with so macOS interoperability should be no (adds adational appl settign to smb share latter) non configruatble in unraid web UIIf your unraid only has 1 nic turn off SMB Multi ChannelIts fine if you don't some just some side performance tuning...this tells me that your basic NAS needs are now enabled and we are ready to edit sahres.Unless your want to use file zilla and tranfer your file to unraid as it can be quite long and ftp has some nice protcal pause features...While too ealy to use teh sftp-fail 2 ban as we are just setting up... You can use Unraid sbuilt in ssh server and conect with your root user and passowrd you use to loign to the unraid web ui...Setting > mamagnemt access this will allo you to conect to you unriad via filezilla at sftp://x.x.x.xx.x.x.x being the unraid ip and port 22where you can go to /mnt/zfs/DATASET NAMEand start dropping files...We need to fix share web UI tab settings frist before any file copy as this will dictate where files are stored..So we made a data datase:ZFS Master will show the new datasetthis will be located at root@Docker:~# cd /mnt/vm-zfs/data/root@Docker:/mnt/vm-zfs/data# lsroot@Docker:/mnt/vm-zfs/data# on disk /mnt pool name per picture its zfs / dataset name this is case sensative in my case its datadata will now exist in teh share tab its contens and subfodlers will notusing smb exta option we can make sub folder share there own share latter...Now lets look at teh share tab...Note the storage files this share option is teling new data to write to the 1TB nvme cahe poolhere we need to corect the share setting:click dataand under primary stoarge select zfsthen hit apply under itnow when data write to /mnt/user/data it write to the corect location /mnt/zfs/data with is teh 4x 14 TB disksotherwise the unriad fuse system will write to /mnt/cache/data which is the 1 TB nvmewe then scroll down we see nfs setting and smb setting...we care about 2 things 1 is this being shared off unraid and how secure is the shareto use windwos expolre to conect to data we need to export the smb settingI want all user be abvle to access dataclick apply to save that setting (If you didn't clcik apply above it will not save the settings as this page is ficle and will only save the setting of that section...clicking the sahre tab you should now see the trabel filled out with the corect dataRepreat for each share setting and each dataset!since you reused the cache disk (if you didn't format it) If you made a appdata dataset we can leverge unraid mover and secodary setting...YOU MUST HAVE A DATA SET MADE FIRST! IT IS BEST PRACTICE TO MAKE IT VIA THE ZFS MASTER PLUGIN!Here we can invkde unraid array mover task to move data off teh cache disk and into the zfs arrayNote VM-ZFS in your case it will be zfsthat zfs is primary any new folder or file will be stored in that dataset.since appdata existed previous and is on cace now when we invoke mover it will move the files off Cache and put them into the dataset.exampleIve been told that unriad has fixed adding share and it should make a zfs dataset if told... but some other option exist at the dataset creation such as encryption...It is better to make dataset first then fix share settings...once mover is done all files will exist on the zfs dataset.as we are esentail fixing the unraid fuse file system whcih "symlnks" the disk /mnt location.they are physical on teh cacehdisk at /mnt/cahce r /mnt/zfshowever docker, vm and other pathing will be set tothe fuse system of /mnt/user/data or /mnt/user/appdata
May 21, 20251 yr Author I just wanna say i appreciate the help! I looked at this briefly and it seems super in depth which i like. Im at work right now so ill view it more in depth later and ill get back to you if i have any questions or issues!
May 21, 20251 yr Community Expert as explained aboveNow that you have the dataset and share correctly configured where you want the dat to be at..2. make sure i have my docker containers point to that data (This post continued below)3. do I still need mover? (Maybe depends on your file hierarchy and how you want the data stored) this is more based on having it on cache and mover to zfs I don't recommend it as move is more fixing the /mnt/user data paths to the actual diskyou may start calling the disk path and bypass the fuse system all together.. This still requires the correct storage set! 4. will I just use my cache pool for docker configs and the zfs pool for the actual data?Correct. Essentially the 1 tb cache disk exist for 3 things-1 the swap plugin for OOM resources-2 the VMs libvirt system vdisk img file-3 the docker xfs vdisk image file-4 Some docker data that shouldn't be run off a zfs array(Database like mongdb, redis, postgress, mysql etc...) due to constant read writes...here is where the primary can be cache, but mover moves it to zfs so the data is stored in zfs but is active on the cache disk...While you can still run them from the zfs array, you can latter add special disk to zfs to handle the work load of database docker and applications.*the same was true for the unraid array and xfs! - More advance and latter zfs configurations...Q/A 2:so now that the Q/A is over lets visit the docker setting and template settings to correct setup If you chose to run VMs on unraid some recommend Generic settings.Note I'm using the direct disk path and not the fuse system /mnt/userthis is also set to use 20 GB on the 1 TB NVME for general kvm/qemu libvrit application required by unriad to run VMswith ZFS full support on kernel 6.12 on unriad v7.1.2 following the release notes and announcements.They recommend using the xfs vdisk image and a different overlap driver.I Personlay Prefer macvlan due to lan identiy and mac different address set per each macvaln docker containerThe big one here is the vDisk Location and the storage driver. I highly recomend using the xfs docker img file...see docs: https://docs.unraid.net/unraid-os/release-notes/7.0.0/#add-support-for-overlay2-storage-driverIf storing on zfs use overley2Note I went with a direct disk path and bypassed the unriad fuse system of /mnt/userThis way I can guarantee that the storage for new file and folders doesn't breaks as long as the storage is set to a single primary disk location and we bypass the need to use mover out right.docker tempalteslets lookg at CA sftp-fail2banAs I prefer transferring large files with filezillaas you should now have a dataset and share setting and docker dataset and share setting corrected...With your docker settings set we can now edit temples and add dockersUsually a template will have a folder for the docker data called app dataThe folder icon is a paththese need mapped to folders on unraid.So I would open my zfs disk array, cd into the docker dataset and mkdir folders.thus I now have a /mnt/zfs/docker/sftp/<any folders ftp would need at docker level>and any folder I want mapped...Here is where I would stick to and use the unraid fuse system... You can bypass it and go direct disk as with the vms libvirt image and docker xfs vdisk image...at this point its just telling the docker where on disk the data is and how you want to image your dis data
May 22, 20251 yr Author Thank you! I have thoroughly read through both of your replies and made a lot of the recommended changes. I would say the only changes i have not made (as they are currently not applicable) is setting up nfs/ssh/and sftp fail2ban. I have no current need for nfs connections, don't plan on sshing to my unraid machine, and wont be sftping. I don't think what you said is bad advice and i will definitely keep them in mind if the time comes for me to need these services. This is my new current main screen with zfs setup and using data sets. Its not 1:1 how you explained i should set it up as i have my data/config files in different folders but i do have the cloud and backup datasets and think those will be useful. And this is my shares page:The only question i so far have i guess is why is zfs master showing 50.9tb but the zfs pool itself shows 23.2? is that intended or?
May 22, 20251 yr Community Expert Awesome. Looks good. Glad i could help. yeas, nfs and sftp / ssh are all optional. They are things I would do and use thus mentioned.Looks like your coming along nicely...I see your running the "*ars" for plex dockers:Trash guides has a lot of good info with unraid data setup adn docker interactions with unraid.https://trash-guides.info/https://trash-guides.info/File-and-Folder-Structure/How-to-set-up/Unraid/https://trash-guides.info/File-and-Folder-Structure/Hardlinks-and-Instant-Moves/under main > zfs mast > actionI recommend taking a snapshot.-This is the webui manual way latter you can use a user script to automate on a cron schedule.Since your using zfs you need to monthly automte and monitor from time to time a zfs scrub between diskif you click zfs you will get additional disk option above the zfs master in the main web ui tabyou can set a scrub schedule:this is when zfs will scrub the disk and confirm parity recommend doing one once a month.using the user script plugin i run this once a month too to aut make a snap shot of all datasets.ZFSSNAPSHOT#!/bin/bash#v0.2########################simple-snapshot-zfs############################################# User Defined Options ####################### List your ZFS datasets. You can add/remove setsDATASETS=("zfs/backup" "zfs/cloud" "zfs/data" "zfs/dockers")# readarray -t DATASETS < <(zfs list -o name -H) # when replacing DATASETS above, should return all pools/Datasets, but not thouroughly tested yet# Set Number of Snapshots to KeepSHANPSHOT_QTY=12###### Don't change below unless you know what you're doing ########################################################################timestamp=$(date "+%Y-%m-%d-%H:%M")echo "Starting Snapshot ${timestamp}"echo "_____________________________________________________________"# Function to create snapshot if there is changed datacreate_snapshot_if_changed() { local DATASET="$1" local WRITTEN WRITTEN=$(zfs get -H -o value written "${DATASET}") if [[ "${WRITTEN}" != "0" ]]; then local TIMESTAMP TIMESTAMP="$(date '+%Y-%m-%d-%H%M')" zfs snapshot "${DATASET}@${TIMESTAMP}" echo "Snapshot created: ${DATASET}@${TIMESTAMP}" else echo "No changes detected in ${DATASET}. No snapshot created." fi}# Function to prune snapshotsprune_snapshots() { local DATASET="$1" local KEEP="${SHANPSHOT_QTY}" local SNAPSHOTS=( $(zfs list -t snapshot -o name -s creation -r "${DATASET}" | grep "^${DATASET}@") ) local SNAPSHOTS_COUNT=${#SNAPSHOTS[@]} echo "Total snapshots for ${DATASET}: ${SNAPSHOTS_COUNT}" local SNAPSHOTS_SPACE SNAPSHOTS_SPACE=$(zfs get -H -o value usedbysnapshots "${DATASET}") echo "Space used by snapshots for ${DATASET}: ${SNAPSHOTS_SPACE}" if [[ ${SNAPSHOTS_COUNT} -gt ${KEEP} ]]; then local TO_DELETE=$((SNAPSHOTS_COUNT - KEEP)) for i in "${SNAPSHOTS[@]:0:${TO_DELETE}}"; do zfs destroy "${i}" echo "Deleted snapshot: ${i}" echo "_____________________________________________________________" done else echo "_____________________________________________________________" fi}# Iterate over each dataset and call the functionsfor dataset in "${DATASETS[@]}"; do create_snapshot_if_changed "${dataset}" prune_snapshots "${dataset}"doneecho "----------------------------Done!----------------------------"This will keep 12 snapshots. (1 for each month) and auto snapshot backup using the snapshot plugin and or the zfs action snapshot manger you can view and restore data to a earlier snapshot.A system is only as good on how you handle recovery form a backup.https://www.youtube.com/watch?v=UR5RCItyCswI find zfs to be easier in maintaining data and drive replacement then the array.https://www.youtube.com/watch?v=36moTWdxjnIsome other specal disk configuration. Additional storage latter adding nvme as l2arc or special vdes to zfs to improve performancehttps://forums.unraid.net/topic/185453-zfs-special-logs-and-cache-drive-options/#findComment-1514833A side mention for future...as a x8 pcie device to add additional nve drive as l2ark or special disk...-I still receommend keeping atleast a nvme as a btrfs caceh disk for legacy unraid setting.
May 22, 20251 yr Author Thanks a lot! I have implemented the script (actually fairly easy to do when reading the directions for user scripts) so now i have scrubbing and snapshots. Tomorrow i will watch those videos and learn more. I appreciate your help and patience.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.