cloth-sprites

Was a bit of a pain getting this working, so thought I'd share some notes Normal array, individual disks are ZFS formatted, pooled disks are shared Each disk has a dataset manually created at the root level for each share that will be on it Auto snapshots are made against each dataset on each disk, windows previous versions sees the merged snapshots just fine Tried to make each disk snapshot for a dataset run at close to the same time, with the same name for cleaner browsing via fused file system Manual snapshots made via zfs-master also work in windows previous-versions, with a tweak to get the right time zone Note: All changes done on UNRAID 6.12.14, Win11 23H2. Will be back in a few weeks with any changes required for 7 If this works out of the box on unraid-7 I'll be vexed at myself for hacking at it now The windows UI shows fused snapshots of all devices in the share. Browsing and restoring works. Deleting could be unpredictable, as the snapshot dir will exist on more than one disk. ### SMB CONFIG ##Setup Global Config nano /boot/config/smb-extra.conf [global] shadow:sort = desc shadow:format = GMT-%Y.%m.%d-%H.%M.%S shadow:snapprefix = ^[A-Za-z0-9_]\{0,\}$ shadow:delimiter = GMT- vfs objects = shadow_copy2 I saw comments that using localtime would break VFS on windows #shadow:localtime = yes <= don't have this ## Configure Per Share Config for VFS Add a USER SCRIPT "Enable VFS on SMB shares" - Run at array start - This adds basedir and snapdir settings to each share in the autogenerated smb-shares.conf file - Necessary, at least for me, as relative snapdir settings did not work, so could not set in global config #!/bin/bash sleep 30 until [ -f /etc/samba/smb-shares.conf ]; do sleep 3; done && sed -i -E 's/(.*)(path)( = \/mnt\/user\/)(.*)/\1\2\3\4\n\1shadow:basedir\3\4\n\1shadow:snapdir\3\4\/\.zfs\/snapshot/' /etc/samba/smb-shares.conf sleep 1 /etc/rc.d/rc.samba restart ### MANUAL SNAPSHOTS Install zfs master config => snapshot prefix => to manual_GMT- config => snapshot format => Y.m.d-H.i.00 Patch source to use UTC in snapshot name, not local TZ nano /usr/local/emhttp/plugins/zfs.master/backend/ZFSMAdmin.php -> add the 3 timezone lines below ( without the '+' ) case 'snapshotdataset': + $script_tz = date_default_timezone_get(); date_default_timezone_set('UTC'); + $snapshot = $zfsm_cfg['snap_prefix'].date($zfsm_cfg['snap_pattern']); $ret = createDatasetSnapshot( $_POST['zdataset'], $snapshot, $_POST['recursive']); + date_default_timezone_set($script_tz); returnAnswer($ret, "ZFS Snapshot Create", "Snapshot created successfully", "Unable to create snapshot", true, false); break; ## If the world was fair we would be done Any snapshot that looks like this: "disk1/test@daily_GMT-2025.01.25-03.27.00" will now be showing up in windows previous version. Create some manually to confirm Rest of this post is just about getting auto snapshots working in the required format ### AUTO SNAPSHOTS I used SANOID, but had to edit it a fair bit 😒 Any script that can create snapshots in a format windows understands could work I never got Previous versions working well with anything but GMT and dates like Y.m.d-H.i.s. There are other other packages to help with illegal chars like ":" in dates, but c.b.a NOTE: The patches below have possibly broken features of sanoid that I am not using. Buyer beware. - I have not used 'autosnap' in config, and removed it from snapshot names - I do not use syncoid, do not know if it expects snapshots in any set format Install Sanoid Package in unraid, then make your own copy. Sanoid updates will not touch this copy. cd /mnt/system mkdir sanoid cd sanoid cp /usr/local/sbin/sanoid . chmod a+x sanoid cp /etc/sanoid/sanoid.conf . cp /etc/sanoid/sanoid.defaults.conf . nano sanoid #apply edits below Add a Userscript to invoke Sanoid ( change the path lol ). I run this at 1 minute past the hour #!/bin/bash /mnt/user/system/sanoid/sanoid --configdir=/mnt/user/system/sanoid - I avoided recursive snapshots as I want all backing disks for each dataset to be created at close to the same time So, my config is like: [disk1/test] use_template = daily [disk2/test] use_template = daily [disk3/test] use_template = daily [disk1/isos] use_template = daily [disk2/isos] use_template = daily Tedious to set up , but doesn't change often for me ## SANOID CHANGES [approx line#, see patch below] - [600, 890] change filename format to /disk1/datasetname@daily_GMT-yyyy-mm-dd hh:ii:00 - [1135] seconds are zeroed, minutes floored to nearest 5 min ( so snapshots 'fuse' together ) You won't want this change if you take snapshots really frequently! - [592] change ordering when creating snapshots. disk1/name sorts by 'name' ignoring 'disk1' - [1125] change snapshot name to use UTC. Windows converts to users local time when displaying Go through it by hand - there are not many lines changed. Best to understand what is being altered too @@ -592,7 +592,10 @@ <-- this is the approx line number a change is made - while ((my $path, my $snapData)... <-- "-" at start: deleted + #while ((my $path, my $snapData)... <-- "+" at start: added Index: sanoid IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/sanoid b/sanoid --- a/sanoid +++ b/sanoid (date 1737791459443) @@ -592,7 +592,10 @@ } if (%newsnapsgroup) { - while ((my $path, my $snapData) = each(%newsnapsgroup)) { + #if ($args{'debug'}) { print Dumper(\%newsnapsgroup); } + #while ((my $path, my $snapData) = each(%newsnapsgroup)) { + foreach my $key (sort {$a =~ s/[^\/]+\///r cmp $b =~ s/[^\/]+\///r} keys %newsnapsgroup){ + my $snapData = $newsnapsgroup{$key}; my $recursiveFlag = $snapData->{recursive}; my $dstHandling = $snapData->{handleDst}; @@ -605,7 +608,8 @@ my @snapshots; foreach my $type (@types) { - my $snapname = "autosnap_$datestamp{'sortable'}_$type"; + #my $snapname = "autosnap_$datestamp{'sortable'}_$type"; + my $snapname = "${type}_GMT-$datestamp{'sortable'}"; push(@snapshots, $snapname); } @@ -886,13 +890,17 @@ close FH; } + # disk2/test@manual_GMT-2025.01.24-10.22.00 creation 1737714177 - + # disk5/test@autosnap_2025-01-24_22:47:06_hourly creation 1737712026 - foreach my $snap (@rawsnaps) { - my ($fs,$snapname,$snapdate) = ($snap =~ m/(.*)\@(.*ly)\t*creation\t*(\d*)/); + # my ($fs,$snapname,$snapdate) = ($snap =~ m/(.*)\@(.*ly)\t*creation\t*(\d*)/); + my ($fs,$snapname,$snapdate) = ($snap =~ m/(.*)\@(\S*)\s*creation\s*(\d*)/); # avoid pissing off use warnings if (defined $snapname) { - my ($snaptype) = ($snapname =~ m/.*_(\w*ly)/); - if ($snapname =~ /^autosnap/) { + #my ($snaptype) = ($snapname =~ m/.*_(\w*ly)/); + my ($snaptype) = ($snapname =~ m/(\w*ly)_.*/); + if ($snapname =~ /^(frequently|hourly|daily|weekly|monthly|yearly)/) { $snaps{$fs}{$snapname}{'ctime'}=$snapdate; $snaps{$fs}{$snapname}{'type'}=$snaptype; } @@ -1125,16 +1133,17 @@ sub get_date { my %datestamp; - ($datestamp{'sec'},$datestamp{'min'},$datestamp{'hour'},$datestamp{'mday'},$datestamp{'mon'},$datestamp{'year'},$datestamp{'wday'},$datestamp{'yday'},$datestamp{'isdst'}) = localtime(time); + ($datestamp{'sec'},$datestamp{'min'},$datestamp{'hour'},$datestamp{'mday'},$datestamp{'mon'},$datestamp{'year'},$datestamp{'wday'},$datestamp{'yday'},$datestamp{'isdst'}) = gmtime(time); $datestamp{'year'} += 1900; $datestamp{'unix_time'} = (((((((($datestamp{'year'} - 1971) * 365) + $datestamp{'yday'}) * 24) + $datestamp{'hour'}) * 60) + $datestamp{'min'}) * 60) + $datestamp{'sec'}; - $datestamp{'sec'} = sprintf ("%02u", $datestamp{'sec'}); - $datestamp{'min'} = sprintf ("%02u", $datestamp{'min'}); + $datestamp{'sec'} = sprintf ("%02u", 0); + #$datestamp{'min'} = sprintf ("%02u", $datestamp{'min'}); + $datestamp{'min'} = sprintf ("%02u", 5* int($datestamp{'min'} / 5)); # floor to 5min. Start jobs at 1min past hour. Slow jobs have 4 minutes to run before name changes $datestamp{'hour'} = sprintf ("%02u", $datestamp{'hour'}); $datestamp{'mday'} = sprintf ("%02u", $datestamp{'mday'}); $datestamp{'mon'} = sprintf ("%02u", ($datestamp{'mon'} + 1)); - $datestamp{'noseconds'} = "$datestamp{'year'}-$datestamp{'mon'}-$datestamp{'mday'}_$datestamp{'hour'}:$datestamp{'min'}"; - $datestamp{'sortable'} = "$datestamp{'noseconds'}:$datestamp{'sec'}"; + $datestamp{'noseconds'} = "$datestamp{'year'}.$datestamp{'mon'}.$datestamp{'mday'}-$datestamp{'hour'}.$datestamp{'min'}"; + $datestamp{'sortable'} = "$datestamp{'noseconds'}.$datestamp{'sec'}"; return %datestamp; }