vindictive-voyeur2295

I am clearly being dumb with this I think, I have been transfering a lot of files from my local Windows machine to my Unraid tower and it has only copied around 500gb, the remaining 500gb was returning the error that it cannot find the location specified in Teracopy, I then tried manually moving some files in windows explorer to the share and it told me there was not enough free space to complete. I have anywhere from 1.4TB to 1.84TB free on my 6 disks (total of 8, the other 2 are parity drives) I wanted to get the task complete so I turned on disk sharing and shared disk 4, entered that in windows and can copy all my required files to that disk without any issues or warnings. Do you guys have any thoughts on this? All shares have a minimum space set of 1.8TB on them with the high water setting, this was automatically set by Unraid, I changed the specific share I was trying to copy to down to 1TB to test a transfer and it still failed when trying to write to the share so I am not sure if the minimum free space is the issue here.

I have ran btrfs dev stats -z /mnt/cache and its not showing any errors. I also ran a scrub at the weekend on the drive, I think I know where the error stems from as I overloaded the cache a few weeks ago stupidly.

Could anyone help me narrow down the below please? Appears to be 2 BTRFS warnings about space, the only BTRFS drive in my system which I am aware of is my cache NVME, its 2tb in size and not even half way full: warning BTRFS allocated space utilization btrfs.disk btrfs.disk_c5f631a8-3c02-42f7-a4fd-aa886cf7d228 r3l-unraid 100 % 7 seconds ago2/11/25 - 09:16:14 100 % 67 seconds ago2/11/25 - 09:15:14 - critical BTRFS data space utilization btrfs.data btrfs.disk_c5f631a8-3c02-42f7-a4fd-aa886cf7d228 r3l-unraid 100 % 7 seconds ago2/11/25 - 09:16:14 100 % 67 seconds ago2/11/25 - 09:15:14 -

Thanks for this - sorry for the delay I have had to deal with a CPU that decided to fail this week in my backup server, but all working again. Duplicati is now running on both my onsite and offsite machine with smart retention in place, volume size in duplicati is now 500mb, I went to change the deduplication block size to 1MB but when I clicked the option it showed the default is 1MB already. I have not changed anything in resilio sync yet as I am deciding wheather I want to keep them running or not, I think I may turn them off, at least at my onsite machine, having them accesible on my offsite machine is handy since I spend more time on it than my home one right now. Seems the final thing for me now is to get backblaze setup at the end of the month.

First backup in Dupicati seems to have completed sucesfully: Source files - 1.9TB Backup size - 1.4TB Does above sound correct, are backups slightly compressed, is it showing I have no clue about backup software so far I also got 1 warning: 2025-01-22 04:19:16 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-BackendQuotaNear]: Backend quota is close to being exceeded: Using 1.458 TB of 120.000 GB (78.881 GB available) * Apprenatly this is an issue with Duplicati looking on there forums below is taken from there: "Not sure why this part of the code was changed, but right now it reports the space from /, so whatever disk it “thinks” is mounted there is reported." This part was wrote prior to reading duplicatis forum - Dont understand above as there is plenty space on the drive the backup is being sent to as its an 8tb drive.

Thanks! Its something thats been in the back of my mind for a while to sort. I have settled on duplicati for my onsite backup - reason being it is much easier to setup to backup from array to an unassinged device / local computer compared to urbackup (could not figure it out) So on my main onsite backup I now have a resilio sync local backup as well as a duplicati one running, duplicati has smart retention in place, it seemed like the best choice but I may be wrong, if so let me know your thoughts if you wouldnt mind? I still have resillio sync running locally, I am not entirely sure what versioning is as it does not stand out in the gui - doing some reading online I can see some people refering to a .SyncArchive folder and that this should persist for however long the advanced setting "sync_trash_ttl" is set to - some questions: 1. Is my above understanding correct and I need to turn this on? 2. What is a good number to set this to? 3. Is it even worth continuing to use this locally now that I have duplicati setup and running locally? Once above is sorted that should hopefully sort out my onsite backup. Onto my offsite backup: Is resillio sync enough for this IF my above understanding of versioning is correct? If so I will set this to the recomended and continue to let it chug along as it is. If not I could do another backup from duplicati to my offsite machine as well, just not sure on the method - is sftp still safe or do you recomend another way? To your second post on the app data backup tool, I am currently using that to backup my flash drive, dockers, app data etc. I am going to invest in backblaze b2 at the end of the month I think, I like the idea of being able to resotre direct from disk providing they still offer this, I am also going to invest in another HDD, stick it in an external enclosure and do a manual backup of my important files maybe every 3 months, not sure where I will keep it though possibly at work, at that rate I would be as well as just using an old NUC running linux to do another backup with duplicati and save lugging the disk around. Edit - Sorry one more question, my volumes for my backup are set at the defualt 50MByte, any issue with with this or should it be changed to something else?

I really need to get a grip on my backups and decide on a cloud provider as well, hoping you helpfull people can aid me with this. My array is 104tb with a little over 80tb used, I am not looking to back all this up, the only thing I am worried about is family photos, RAW photos and footage, familys phone backups etc, this comes at the moment to a bit over 2tb. Whilst I would love to just backup all 80tb to another server I dont see the point in spending so much money on more drives, at least just now. At the moment I have the data I truly care about on my array, I have resillio sync one way syncing the data to an external drive on another computer in the same home and I then have it also syncing to another unraid machine I built with a couple of drives in it to basically serve as a backup in another location. So far how flawed is my logic? I feel like by using resillio sync I might be going down a slippery path but so far it has ran for a few months *knock on wood* without issue. Is there anything else I could be doing to improve this locally? Is there another way I should sync the data at least locally in my home to the other external drive? Is there anything wrong with me syncing to the other machine in another location? My next thought is that I am not truly following the 3-2-1 method, I also need a cloud backup, I have 1tb in Onedrive, and a family Google plan but I feel for this much data which will likely grow as time goes on I need something with more space, what would you guys recomend? I am based in the UK if that makes any difference. 2025 is the year I get my backups in order...hopefully

Thanks for this, I will do so just now and continue to monitor.

Some more info, looks like they are real after all. Ran btrfs dev stats /mnt/cache and got the below results [/dev/nvme0n1p1].write_io_errs 1575422 [/dev/nvme0n1p1].read_io_errs 1059 [/dev/nvme0n1p1].flush_io_errs 26143 [/dev/nvme0n1p1].corruption_errs 0 [/dev/nvme0n1p1].generation_errs 0 Edit - More info: Result of smart test below: UUID: 825a839c-92ce-4512-82f1-2718426725b5 Scrub started: Thu Oct 10 23:11:35 2024 Status: finished Duration: 0:01:41 Total to scrub: 544.39GiB Rate: 5.39GiB/s Error summary: no errors found Downloaded results: === START OF INFORMATION SECTION === Model Number: KINGSTON SFYRD2000G Serial Number: 50026B7686D58263 Firmware Version: EIFK31.7 PCI Vendor/Subsystem ID: 0x2646 IEEE OUI Identifier: 0x0026b7 Total NVM Capacity: 2,000,398,934,016 [2.00 TB] Unallocated NVM Capacity: 0 Controller ID: 1 NVMe Version: 1.4 Number of Namespaces: 1 Namespace 1 Size/Capacity: 2,000,398,934,016 [2.00 TB] Namespace 1 Formatted LBA Size: 512 Namespace 1 IEEE EUI-64: 0026b7 686d582635 Local Time is: Thu Oct 10 23:06:51 2024 BST Firmware Updates (0x12): 1 Slot, no Reset required Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test Optional NVM Commands (0x005d): Comp DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp Log Page Attributes (0x08): Telmtry_Lg Maximum Data Transfer Size: 512 Pages Warning Comp. Temp. Threshold: 84 Celsius Critical Comp. Temp. Threshold: 89 Celsius Supported Power States St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat 0 + 8.80W - - 0 0 0 0 0 0 1 + 7.10W - - 1 1 1 1 0 0 2 + 5.20W - - 2 2 2 2 0 0 3 - 0.0620W - - 3 3 3 3 2500 7500 4 - 0.0620W - - 4 4 4 4 2500 7500 Supported LBA Sizes (NSID 0x1) Id Fmt Data Metadt Rel_Perf 0 + 512 0 2 1 - 4096 0 1 === START OF SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED SMART/Health Information (NVMe Log 0x02) Critical Warning: 0x00 Temperature: 42 Celsius Available Spare: 100% Available Spare Threshold: 10% Percentage Used: 0% Data Units Read: 1,720,439 [880 GB] Data Units Written: 4,067,836 [2.08 TB] Host Read Commands: 13,936,702 Host Write Commands: 31,345,141 Controller Busy Time: 146 Power Cycles: 2 Power On Hours: 173 Unsafe Shutdowns: 0 Media and Data Integrity Errors: 0 Error Information Log Entries: 478,986 Warning Comp. Temperature Time: 0 Critical Comp. Temperature Time: 0 Temperature Sensor 2: 55 Celsius Error Information (NVMe Log 0x01, 16 of 63 entries) Num ErrCount SQId CmdId Status PELoc LBA NSID VS 0 478986 0 0x4019 0x4004 0x102c 0 0 - 1 478985 0 0x4018 0x4004 0x102c 0 0 - 2 478984 0 0x701a 0x4004 0x102c 0 0 - 3 478983 0 0x2019 0x4004 0x102c 0 0 - 4 478982 0 0x1019 0x4004 0x102c 0 0 - 5 478981 0 0x1018 0x4004 0x102c 0 0 - 6 478980 0 0x7006 0x4004 0x102c 0 0 - 7 478979 0 0x6005 0x4004 0x102c 0 0 - 8 478978 0 0x0018 0x4004 0x102c 0 0 - 9 478977 0 0x101b 0x4004 0x102c 0 0 - 10 478976 0 0xe018 0x4004 0x102c 0 0 - 11 478975 0 0xf01b 0x4004 0x102c 0 0 - 12 478974 0 0x4011 0x4004 0x102c 0 0 - 13 478973 0 0x4010 0x4004 0x102c 0 0 - 14 478972 0 0x3000 0x4004 0x102c 0 0 - 15 478971 0 0xf003 0x4004 0x102c 0 0 - ... (47 entries not read) Should also note this is just a single cache pool.

I just noticed these while browsing netdata: Loads of errors on my cache drive showing in Netdata yet nothing in the Unraid system log - surely they would be there if true right? Any thoughts on how to clear this or investigate further? I have recently changed to a new cache drive, done everything correctly and its been working fine ever since for over 2-3 weeks now. Log file attached. Edit - Some additional info, I am on the stable branch of Netdata, I was on latest but changed it to stable to check if this kept up. r3l-unraid-syslog-20241010-2118.zip

Great stuff, thank you. This forum has been a great help this last week becuase of you guys. Thanks agin.

Yes single. Thanks for the replies guys. The new drives will be larger (2tb) than current one (512gb) will this still be OK? If so I will attach one new drive, let it mirror, then remove old and replace with new.

It is btrfs

I will be swapping out my current cache drive for a new setup - 2 x 2tb nvme in raid 1. What is the correct procedure for this? I have found a few differet things and want to make sure I use the correct official method so I dont screw anything up.

Appreciate the reply. Yes I think that is what I will be doing, thought if I was looking at things the wrong way I would likely be told on here. 2 x 2 in raid 1 will be the plan, see what I can find on a deal during the prime sale Will re-purpose the other drives for other things as well.