abhi.ko

Thank you - appreciate the quick responses @Syco54645. If anyone else feels that I should not attempt this with the VM running please let me know - otherwise I plan to proceed with this plan.

Thank you. Edited steps to add the format disk step. My docker image & libvirt image, vm domains (vdisk location) and isos (boot iso's for VM's) are all cache primary and always on the cache disk and not on the array. As far as containers that write to the array yes I do have a few like Nextcloud (data location). I am okay with shutting docker off. Would it be okay to leave the VM running (just HA) while I do this. Also with not copying the content back to original disk - as per the docs this is a step, assume you did not do this in your original case was there any issues with shares and files etc.

Apologies for dusting up this old thread - but I am trying to change FS on multiple data disks from reiserfs to xfs. So, I believe the above steps should work for me, I just have to do them multiple times for each of the reiserfs disks. Is that accurate? Questions - Would I have to stop all dockers and VM's while doing this? Do I need to copy the contents of the disk back to the original disk after formatting it to xfs. If I don't, is there anything in config that i need to change? Steps to follow for multiple disks: Shutdown VM and docker From Disk 1 (reiserfs) move contents to empty disk 20 (xfs) Stop Array Change Disk 1 to xfs Restart array Format Disk 1 Repeat Steps 2-6 for next disk, until all are xfs Restart Docker and VM One of my VM's is Home Assistant - so this would mean that all the smart home related stuff will go down during this process, is there another way?

I am happy to - is there a beta plg file or something that I need to download? BTW - I removed the exclusions to /mnt/user/ and it ran without errors.

Can you share the script that i need to add to user scripts for this please?

Thanks for pointing that out.

Thank you for a taking a look. I am sure it is user error. The reason why I excluded that was in an effort to get rid of the Multi-mapping detected warning. It did not help, but any pointers on what I can do differently to avoid the warning and have the backup run without errors? Are these containers still backed up even though they failed verification since I am getting a Backup created without issues message right before the verification starts?

I'm getting a lot of tar verification failed errors in the log after my first run (switched from old plugin just last week). Some of the containers don't have the errors, but most of them do. Not sure what if anything I can do to resolve these? I am also confused about what the previous errors referred to in the due to previous errors statement is. Log snippet below: [04.03.2024 02:38:10][ℹ️][radarr] Stopping radarr... done! (took 4 seconds) [04.03.2024 02:38:14][ℹ️][radarr] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:38:14][ℹ️][radarr] Calculated volumes to back up: /mnt/user/appdata/radarr [04.03.2024 02:38:14][ℹ️][radarr] Backing up radarr... [04.03.2024 02:38:14][ℹ️][radarr] Backup created without issues [04.03.2024 02:38:14][ℹ️][radarr] Verifying backup... [04.03.2024 02:38:14][❌][radarr] tar verification failed! Tar said: tar: /mnt/user/appdata/radarr: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:38:29][ℹ️][radarr] Starting radarr... (try #1) done! [04.03.2024 02:38:31][ℹ️][Plex-Media-Server] Stopping Plex-Media-Server... done! (took 7 seconds) [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] '/mnt/user/appdata/Plex-Media-Server/Transcode' is within mapped volume '/mnt/user/appdata/Plex-Media-Server'! Ignoring! [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Calculated volumes to back up: /mnt/user/appdata/Plex-Media-Server [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Backing up Plex-Media-Server... [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Backup created without issues [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Verifying backup... [04.03.2024 02:38:38][❌][Plex-Media-Server] tar verification failed! Tar said: tar: /mnt/user/appdata/Plex-Media-Server: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:47:16][ℹ️][Plex-Media-Server] Starting Plex-Media-Server... (try #1) done! [04.03.2024 02:47:19][ℹ️][nzbget] Stopping nzbget... done! (took 5 seconds) [04.03.2024 02:47:24][ℹ️][nzbget] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:47:24][ℹ️][nzbget] Calculated volumes to back up: /mnt/user/appdata/nzbget [04.03.2024 02:47:24][ℹ️][nzbget] Backing up nzbget... [04.03.2024 02:47:24][ℹ️][nzbget] Backup created without issues [04.03.2024 02:47:24][ℹ️][nzbget] Verifying backup... [04.03.2024 02:47:24][❌][nzbget] tar verification failed! Tar said: tar: /mnt/user/appdata/nzbget: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:47:28][ℹ️][nzbget] Starting nzbget... (try #1) done! [04.03.2024 02:47:30][ℹ️][sabnzbd] Stopping sabnzbd... done! (took 4 seconds) [04.03.2024 02:47:34][ℹ️][sabnzbd] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:47:34][ℹ️][sabnzbd] Calculated volumes to back up: /mnt/user/appdata/sabnzbd/config [04.03.2024 02:47:34][ℹ️][sabnzbd] Backing up sabnzbd... [04.03.2024 02:47:34][ℹ️][sabnzbd] Backup created without issues [04.03.2024 02:47:34][ℹ️][sabnzbd] Verifying backup... [04.03.2024 02:47:34][❌][sabnzbd] tar verification failed! Tar said: tar: /mnt/user/appdata/sabnzbd/config: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:47:38][ℹ️][sabnzbd] Starting sabnzbd... (try #1) done! ... ... ... [04.03.2024 04:04:11][ℹ️][Main] Backing up the flash drive. [04.03.2024 04:05:23][ℹ️][Main] Flash backup created! [04.03.2024 04:05:23][ℹ️][Main] Copying the flash backup to '/mnt/user/Other Media/unRAID/Flash Backup/' as well... [04.03.2024 04:05:28][ℹ️][Main] VM meta backup enabled! Backing up... [04.03.2024 04:05:28][ℹ️][Main] Done! [04.03.2024 04:05:28][⚠️][Main] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum. [04.03.2024 04:05:28][ℹ️][Main] DONE! Thanks for using this plugin and have a safe day ;) [04.03.2024 04:05:28][ℹ️][Main] ❤️ Here is the debug log id for a more recent manually triggered backup with same errors but different timestamps - f73d93f6-bece-4fe4-97b6-8477b9440833

I'm getting a lot of tar verification failed errors in the log after my first run (switched from old plugin just last week). Some of the containers don't have the errors, but most of them do. Not sure what if anything I can do to resolve these? I am also confused about what the previous errors referred to in the due to previous errors statement is. Log snippet below: [04.03.2024 02:38:10][ℹ️][radarr] Stopping radarr... done! (took 4 seconds) [04.03.2024 02:38:14][ℹ️][radarr] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:38:14][ℹ️][radarr] Calculated volumes to back up: /mnt/user/appdata/radarr [04.03.2024 02:38:14][ℹ️][radarr] Backing up radarr... [04.03.2024 02:38:14][ℹ️][radarr] Backup created without issues [04.03.2024 02:38:14][ℹ️][radarr] Verifying backup... [04.03.2024 02:38:14][❌][radarr] tar verification failed! Tar said: tar: /mnt/user/appdata/radarr: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:38:29][ℹ️][radarr] Starting radarr... (try #1) done! [04.03.2024 02:38:31][ℹ️][Plex-Media-Server] Stopping Plex-Media-Server... done! (took 7 seconds) [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] '/mnt/user/appdata/Plex-Media-Server/Transcode' is within mapped volume '/mnt/user/appdata/Plex-Media-Server'! Ignoring! [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Calculated volumes to back up: /mnt/user/appdata/Plex-Media-Server [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Backing up Plex-Media-Server... [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Backup created without issues [04.03.2024 02:38:38][ℹ️][Plex-Media-Server] Verifying backup... [04.03.2024 02:38:38][❌][Plex-Media-Server] tar verification failed! Tar said: tar: /mnt/user/appdata/Plex-Media-Server: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:47:16][ℹ️][Plex-Media-Server] Starting Plex-Media-Server... (try #1) done! [04.03.2024 02:47:19][ℹ️][nzbget] Stopping nzbget... done! (took 5 seconds) [04.03.2024 02:47:24][ℹ️][nzbget] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:47:24][ℹ️][nzbget] Calculated volumes to back up: /mnt/user/appdata/nzbget [04.03.2024 02:47:24][ℹ️][nzbget] Backing up nzbget... [04.03.2024 02:47:24][ℹ️][nzbget] Backup created without issues [04.03.2024 02:47:24][ℹ️][nzbget] Verifying backup... [04.03.2024 02:47:24][❌][nzbget] tar verification failed! Tar said: tar: /mnt/user/appdata/nzbget: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:47:28][ℹ️][nzbget] Starting nzbget... (try #1) done! [04.03.2024 02:47:30][ℹ️][sabnzbd] Stopping sabnzbd... done! (took 4 seconds) [04.03.2024 02:47:34][ℹ️][sabnzbd] Should NOT backup external volumes, sanitizing them... [04.03.2024 02:47:34][ℹ️][sabnzbd] Calculated volumes to back up: /mnt/user/appdata/sabnzbd/config [04.03.2024 02:47:34][ℹ️][sabnzbd] Backing up sabnzbd... [04.03.2024 02:47:34][ℹ️][sabnzbd] Backup created without issues [04.03.2024 02:47:34][ℹ️][sabnzbd] Verifying backup... [04.03.2024 02:47:34][❌][sabnzbd] tar verification failed! Tar said: tar: /mnt/user/appdata/sabnzbd/config: Not found in archive; tar: Exiting with failure status due to previous errors [04.03.2024 02:47:38][ℹ️][sabnzbd] Starting sabnzbd... (try #1) done! ... ... ... [04.03.2024 04:04:11][ℹ️][Main] Backing up the flash drive. [04.03.2024 04:05:23][ℹ️][Main] Flash backup created! [04.03.2024 04:05:23][ℹ️][Main] Copying the flash backup to '/mnt/user/Other Media/unRAID/Flash Backup/' as well... [04.03.2024 04:05:28][ℹ️][Main] VM meta backup enabled! Backing up... [04.03.2024 04:05:28][ℹ️][Main] Done! [04.03.2024 04:05:28][⚠️][Main] An error occurred during backup! RETENTION WILL NOT BE CHECKED! Please review the log. If you need further assistance, ask in the support forum. [04.03.2024 04:05:28][ℹ️][Main] DONE! Thanks for using this plugin and have a safe day ;) [04.03.2024 04:05:28][ℹ️][Main] ❤️

Yes tried both with no luck, stuck in boot loop for non-UEFI mode (folder name in the manual install left as EFI-) as well.

Updated my BIOS to the latest available version for my MB and tried upgrading to 6.12.3 and still has the same issue. Stuck in a boot loop, getting to bzroot loading - gets an OK pretty quickly and then reboots. Tried with a stock UNRAID trial version of 6.12.3 on a different USB (manual install method used) and had the same result. So issue still exists. So I am unable to update to versions higher than 6.11.5, which is disappointing and frustrating as more time passes and newer versions come out but this issue still remains unsolved. @limetech is there any work being done to identify what is causing the issue for all these systems and to fix it? Happy to provide diagnostics or anything else that you guys might need.

UnRaid experts - any updates on this hardware compatibility issue (based on my troubleshooting). It doesn't seem isolated to one machine and seems to impact a good number of users. Any and all help appreciated. I'm still on 11.5 and I see that 12.2 is out now.

Are there any know hardware computability issues with the Linux kernel in 6.12?

USB creator did not work for me always. It was hit or miss. Just got stuck midway most of the time but completed a few times. So I have been using the manual install method and followed the same instructions. The flash drive is Bootable (I think) since it gets to the screen with the boot options, but when it tries to load bzroot it fails, is my limited understanding. Happy to try something else.

So coming back to this, I just tried with a stock 6.12.1 version of Unraid on a different USB and it had the exact same result. It got to Loading bzroot.... and then rebooted. So does that mean something within my hardware is not compatible with the new version of Unraid? Any next steps to identify what exactly is causing the issue?

Attached. tower-diagnostics-20230619-0801.zip

Okay will do. Looks good for now. root@Tower:~# df -h Filesystem Size Used Avail Use% Mounted on rootfs 47G 2.1G 45G 5% / tmpfs 32M 6.8M 26M 22% /run Another thing that just popped up - BTRFS Operation is running. Same as this issue here. Screenshot below.

Thank you @JorgeB - what would I do to investigate/fix this? The problem seem to have gone away, at least for now. CA backup ran yesterday night as per the cron shedule and it stopped and started all my containers, which is what I am suspecting did it. Plex was also restarted. Anything more I need to do now or just watch if it happens again.

What are these temp file systems? That is the only one that is full or close to it.

df - h output below: Filesystem Size Used Avail Use% Mounted on rootfs 47G 2.1G 45G 5% / tmpfs 32M 32M 0 100% /run /dev/sda1 15G 979M 14G 7% /boot overlay 47G 2.1G 45G 5% /lib/firmware overlay 47G 2.1G 45G 5% /lib/modules devtmpfs 8.0M 0 8.0M 0% /dev tmpfs 47G 0 47G 0% /dev/shm cgroup_root 8.0M 0 8.0M 0% /sys/fs/cgroup tmpfs 128M 8.3M 120M 7% /var/log tmpfs 1.0M 0 1.0M 0% /mnt/disks tmpfs 1.0M 0 1.0M 0% /mnt/remotes tmpfs 1.0M 0 1.0M 0% /mnt/addons tmpfs 1.0M 0 1.0M 0% /mnt/rootshare /dev/md1 4.6T 3.8T 838G 83% /mnt/disk1 /dev/md2 3.7T 2.6T 1.2T 69% /mnt/disk2 /dev/md3 3.7T 2.6T 1.2T 70% /mnt/disk3 /dev/md4 4.6T 3.5T 1.2T 76% /mnt/disk4 /dev/md5 2.8T 1.9T 900G 68% /mnt/disk5 /dev/md6 3.7T 2.2T 1.5T 61% /mnt/disk6 /dev/md7 5.5T 3.6T 2.0T 65% /mnt/disk7 /dev/md8 2.8T 1.2T 1.7T 42% /mnt/disk8 /dev/md9 3.7T 1.9T 1.9T 51% /mnt/disk9 /dev/md10 7.3T 5.1T 2.3T 69% /mnt/disk10 /dev/md11 2.8T 1.4T 1.4T 51% /mnt/disk11 /dev/md12 7.3T 5.1T 2.3T 69% /mnt/disk12 /dev/md13 4.6T 2.7T 1.9T 60% /mnt/disk13 /dev/md14 7.3T 5.1T 2.3T 69% /mnt/disk14 /dev/md15 7.3T 5.4T 2.0T 74% /mnt/disk15 /dev/md16 7.3T 5.2T 2.2T 71% /mnt/disk16 /dev/md17 7.3T 4.8T 2.5T 66% /mnt/disk17 /dev/md18 7.3T 5.2T 2.2T 71% /mnt/disk18 /dev/md19 9.1T 5.8T 3.4T 64% /mnt/disk19 /dev/md20 7.3T 4.4T 3.0T 60% /mnt/disk20 /dev/sdu1 1.9T 947G 961G 50% /mnt/cache shfs 110T 73T 37T 67% /mnt/user0 shfs 110T 73T 37T 67% /mnt/user /dev/loop2 100G 50G 50G 51% /var/lib/docker /dev/loop3 1.0G 6.6M 903M 1% /etc/libvirt Diags attached as well. tower-diagnostics-20230618-1118.zip

Now a docker container is refusing to start because of a similar "no space left on device" error. Not sure what is going on. All drives including the flash device and the cache drive (appdata) have enough space on them. docker: Error response from daemon: failed to start shim: symlink /var/lib/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/f619bbfa000746382d32a164bac78f53035dc52c88d74ae2936d0b58e91a2149 /var/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/f619bbfa000746382d32a164bac78f53035dc52c88d74ae2936d0b58e91a2149/work: no space left on device: unknown. I'm think that this is somehow related to me copying the config folder after installing 6.11.5, but I am not sure what is going on or how to fix this. All help is appreciated.

I haven't done the above step yet, have the USB drive ready but waiting for some stuff to finish on the server before I shutdown. I had another question: Ever since the upgrade issue and manual revert to 6.11.5, i am seeing this error on the GUI Warning: file_put_contents(): Only -1 of 154 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php on line 715 What is this warning about - anything I can do to fix this?

Thank You!

Hi @JorgeB how do I restore if I do have a backup from CA Backup. I don't see a restore option for VM's, only for appdata within that plugin. Is it just a copy and paste the backup file to the actual location?

I will try that, but just to be clear - I did try booting into safe mode from the GUI options - Unraid OS Safe Mode (no plugins, no GUI). That also did not work.