nicduv

Anyone have an idea of why Duplicati is writing so much to my cache disk during backup operations? I have the /run/duplicati-temp/ directory remapped to RAM at /dev/shm/ and I can confirm that it is putting temp files in there, however, I am still seeing tons of disk writes to my cache when it's backing up. I've confirmed that these writes stop after the backup process finishes.

I didn't end up getting around to restarting my server until just now and it looks like that was the issue. After removing the references from that docker container, the old disk folders were removed from the mnt directory after the reboot.

Ah, that makes sense. I forgot to update my docker for my homepage dashbaord that was referencing them for individual disk usage stats. I've now removed those references. Is it safe to manually remove those folders that are still there or will they get removed automatically on the next system reboot?

I recently shrank my array of three old 8TB HDDs and now the fix common problems plugin is giving me errors saying "Invalid folder disk6/7/8 contained within /mnt". I'm assuming these are just leftover from the three disks I removed from the array and I can safely delete the old empty folders, but I just wanted to confirm that this is OK to do in order to get rid of these errors, instead of just ignoring them.

Got it. I'm still on 6.12.14, is this issue still present in version 7?

This is replicating from my raidz 4 nvme drive cache pool to a single disk HDD zpool in the array. The cache peaks at 25MB/s read sometimes while running the replication. The array drive sits around 10MB/s write. I don't think it's actually anything to do with compression, as I disabled compression on the array drive and am still seeing the same slow speed and single core of the CPU getting pegged.

I've been following many of the incredible guides from @SpaceInvaderOne, and recently decided to set up zfs auto snapshots and replication from my pool to a zfs formatted drive on my array using his script. Everything seems to be working correctly, but the speed seems extremely slow, averaging around 10MB/s. It took over two hours to replicate a dataset around 83GB. Does this seem right? I notice that while the replication is happening, one of my cores on my processor is pegged at 100% the entire time and core that's pegged at 100% will switch at random.

I'm currently testing out the Spaceinvader One Mastering ZFS: Easily Auto Snapshot & Replicate a ZFS Dataset and I'm seeing extremely slow transfer speed, around 10-15MB/sec. I believe it might have to do with the fact that the pool I'm replicating to has compression enabled. When I start a replication I can see the CPU usage of one single core get pegged to 100%. The core that is being used will jump to a new one after a bit, but it will always only use a single core. Is this expected, or is there a way to change something in zfs settings to utilize all cores during the compression process?

Nothing showing in the syslog after logging to a remote server and experiencing a crash. I think there's more than one issue going on here. The first is the USB connectivity issue that was causing the USB devices to reset. This seems to be fixed by simply starting Unraid with both DAS powered off, then starting one and letting Unraid register the drives, then starting the other and again letting Unraid register the drives, then starting the array. The other issue is with restoring the Plex appdata. I'm not sure why the Beelink PC is crashing when extracting this, but I switched to a Minisforum MS-01 and it is able to successfully extract the appdata folder without issue. I think there must be some issue with the USB controller on the Beelink so I'm sending that one back and am going to keep the MS-01 rather than try and chase ghosts with the Beelink. I'll mark this as solved for now.

I'm trying to do the same thing, make an exact mirror of my appdata to a new pool, and I'm wondering if you ever figured this out.

I'm able to get the computer to consistently crash by trying to restore a backup of my Plex appdata. Every time I try to use the appdata backup lougin to restore the plex.tar.zst, the computer immediately crashes. Likewise if I try to use Unbalance plugin to move the Plex appdata folder to a different drive it crashes. It seems to be crashing when trying to move or restore files in the Database directory for Plex. Any idea why this might be happening?

I had run into some issue starting up Unraid when I had both DAS plugged in at once before this recent issue started. My solution was to just turn one of them off and then start it once Unraid had booted and this had seemed to work previously. Interestingly, I now just restarted the computer and instead of having the TERRAMASTER D5-300C turned off during boot, I had the D8 Hybrid (where the cache pool is located) turned off. Now when I am copying the same files I had been before, where it was running very slow (60MB/s) and giving the I/O errors, it is copying from/to the same locations at full speed (260MB/s on a spinning disk on the array) and there are no errors.

Sorry for the long post, I wanted to include as much info as possible off the bat. I've recently setup my first Unraid server and everything was going well for the past couple of weeks as I set it all up and migrated data over to the array. I'm very new to Unraid so I've mostly been following a lot of the guides Spaceinvader One has put out. Due to space issues I am running Unraid on a Beelink GTi12 mini PC and have two DAS setup with my drives in them attached through USB-C 10gbs ports. A TERRAMASTER D5-300C with 2x14TB and 3x8TB drives included in the main array and the TERRAMASTER D8 Hybrid with 1x20TB HDD in the array and 4x2TB NVME drives as the cache pool. The 4x2TB drives are setup in raidz1. There is no parity setup on the array yet, I was going to do that sometime next week after getting all of my data transferred over. Yesterday while doing a backup of my main Windows desktop to the cache pool, the mini PC randomly shut off. There was no issue with power as I have it connected to a UPS and the UPS didn't register any events. I checked the zfs pool status and there were no errors listed, but I decided to run a scrub of the pool. It completed and didn't show any issues. I also tried to check the logs, but they had been cleared due to the shutdown. I set the syslog server to localhost to a share on the cache pool to see if I could capture any data if it happened again. It did end up happening again while doing another write to the cache pool and the log didn't show anything as far as I could tell, there were no entries at the time of the power off. I tried writing some more to the cache and it unfortunately shut down again. At this point after restarting the computer, I was unable to start the array. There was a yellow error in the logs to the effect of unable to mount cache due to uncorrectable I/O failure. I stupidly didn't save the error as I expected it to be saved to the syslog I was recoding locally, but I now realize that was not the case since the syslog was trying to write to the share on the cache, which was unmountable. I now have the syslog mirroring to the flash as I probably should have done from the beginning. I gave up trying to troubleshoot the issue as it was late and wanted to look at it fresh in the morning. This morning I was able to get the array to start up without issue, and started copying an appdata backup off of the cache array since I (again, stupidly) hadn't saved the appdata backups anywhere else but the cache pool. I noticed that the copy was going unusually slow, so I checked the logs and didn't see anything initially so I just let it continue copying. At one point I noticed the transfer halted so I checked the dashbaord/syslog and I noticed I was getting errors such as the following: Jan 19 11:10:08 MiniPC kernel: usb 3-2.3.1: reset high-speed USB device number 12 using xhci_hcd Jan 19 11:10:08 MiniPC kernel: sd 6:0:0:0: [sdh] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=30s Jan 19 11:10:08 MiniPC kernel: sd 6:0:0:0: [sdh] tag#0 CDB: opcode=0x28 28 00 ac 8e bf 60 00 00 f0 00 Jan 19 11:10:08 MiniPC kernel: I/O error, dev sdh, sector 2895036256 op 0x0:(READ) flags 0x4700 phys_seg 4 prio class 2 Jan 19 11:10:09 MiniPC kernel: zio pool=cache vdev=/dev/sdh1 error=5 type=1 offset=1482258407424 size=270336 flags=40080c80 This would always be proceeded by some of the cores on my processor getting pinned at 100%. I checked the zpool status and it is showing unrecoverable errors as well as that note about the resilver. The resilver note is from first thing this morning when I started up after getting the "unable to mount cache due to uncorrectable I/O failure" from the night before: pool: cache state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P scan: resilvered 7.69M in 00:00:00 with 0 errors on Sun Jan 19 09:10:05 2025 config: NAME STATE READ WRITE CKSUM cache ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 /dev/sdh1 ONLINE 21 0 0 /dev/sdi1 ONLINE 62 0 0 /dev/sdf1 ONLINE 0 0 0 /dev/sdg1 ONLINE 0 0 0 At this point I'm pretty out of my depth and don't know what I should do to correct/troubleshoot. My current thought is that it's something to do with the cache pool being attached over USB even though this had been working without issue for the past couple of weeks with TBs of data being written to and read from the cache and the array. I've attached my diagnostics. Any insight or help would be much appreciated. minipc-diagnostics-20250119-1205.zip

There is one USB-C in the front and one in the back and I have one DAS connected to each. I'll see if switching one of them over to one of the 10Gbps USB-A ports fixes it once I have a chance to restart the server next.

I've just begun my first Unraid journey and have been loving it so far. My current setup is a Beelink GTi12 mini PC with two external USB connected DAS. I know the purists will say not to use USB attached storage, but this it what I have right now and what works for me due to limited space. The DAS I'm using are the TERRAMASTER D5-300C with 2x14TB and 3x8TB drives and the TERRAMASTER D8 Hybrid with 2x20TB HDDs and 4x2TB NVME drives. There is also the Unraid USB stick and a internal NVME drive inside the mini PC that is currently unassigned. This has been working perfectly without issue as I've been transferring data to the new array and setting everything up. I just got another 20TB HDD and went to add it to the D8 Hybrid. I had some issues hot swapping it in so I decided to restart the server to see if that would fix it. Upon restarting, Unraid failed to boot. This is the output when it failed the first time. Upon subsequent restarts, it still fails to boot, but the can't set config #1 errors no longer show up. If I power down that D5-300C prior to booting, the server will start without issue and I can then power it up and Unraid will recognize all of the drives including the new 20TB I have added to the D8 Hybrid and I can successfully start the array. Have I somehow hit a limit of external USB devices that my mobo or Unradid can start up with? The current fix is fine for now as I don't plan to restart the server too frequently, but I'd obviously like to get this addressed so that, should I need to restart remotely, or there is a power outage that I need to recover from and I'm on vacation or away from the server for an extended period of time, I'd be able to do so without issue.