eXorQue

I will try that. In the mean time I had another issue. Server wasn’t rebooted (at least not by me) and docker failed to start even though it was on before. I’ve taken diagnostics before rebooting supermicro-diagnostics-20250807-1006.zip

Scrub completed ``` UUID: 38dc75e4-1cf6-45cd-bb76-e15d45da80d9 Scrub started: Fri Aug 1 12:13:48 2025 Status: finished Duration: 0:21:02 Total to scrub: 469.60GiB Rate: 381.03MiB/s Error summary: verify=38140 csum=30545648 Corrected: 30583788 Uncorrectable: 0 Unverified: 0 ```

I already have that added to my syslinux.cfg /syslinux/syslinux.cfg ``` default menu.c32 menu title Lime Technology, Inc. prompt 0 timeout 50 label Unraid OS menu default kernel /bzimage append initrd=/bzroot nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off label Unraid OS GUI Mode kernel /bzimage append initrd=/bzroot,/bzroot-gui label Unraid OS Safe Mode (no plugins, no GUI) kernel /bzimage append initrd=/bzroot unraidsafemode label Unraid OS GUI Safe Mode (no plugins) kernel /bzimage append initrd=/bzroot,/bzroot-gui unraidsafemode label Memtest86+ kernel /memtest ```

Reboot > Docker broken > Settings > Docker > [x] delete vdisk file > [Delete] > Enable Docker "No" > [Apply] > Enable Docker "Yes" > [Apply] > "No Docker containers installed" > Apps > Previous apps > Oh don't forget to create the "proxynetwork" > Terminal > docker network create proxynetwork > Previous apps > Docker > Select All on page > Install > Wait... Wait... Wait... (no idea if this can be done in the background?) > Start docker compose stack > Wait > Start another stack > wait > start the last stack > Done supermicro-diagnostics-20250730-1949.zip

supermicro-diagnostics-20241125-2322.zipAgain cache issues. Should I replace my nvme disk?

I've restarted. Docker didn't start again. I have scrubbed and repaired Scrub Status btrfs scrub status: UUID: 38dc75e4-1cf6-45cd-bb76-e15d45da80d9 Scrub started: Mon Nov 25 11:27:13 2024 Status: finished Duration: 0:32:15 Total to scrub: 1.61TiB Rate: 870.75MiB/s Error summary: verify=6424 csum=408435 Corrected: 414859 Uncorrectable: 0 Unverified: 0 I then tried restarting docker by setting "Enabled: True" to false and back to true in settings > Docker. That didn't start it. So I restarted the array (not the server itself). Still no restart. I made a Diagnostics file that I added to this post. supermicro-diagnostics-20241125-1203.zip Starting service with `/etc/rc.d/rc.docker start` Gives: `no image mounted at /var/lib/docker` So I restarted to see if it would start docker afterwards, but it didn't... Do I need to delete the image again?

I've applied the changes. Let's see how it goes. Thank you for the quick support

@JorgeBI had notifications on the Cache drive. I just got errors Nov 24 20:47:59 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:47:59 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:47:59 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:47:59 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900898, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900899, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900900, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900901, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900902, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900903, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900904, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900905, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900906, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1900907, rd 194623731, flush 68123, corrupt 3, gen 0 Nov 24 20:48:07 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:07 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:07 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:07 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:10 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:10 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:11 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:11 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:11 supermicro kernel: BTRFS warning (device sdf1): lost page write due to IO error on /dev/nvme0n1p1 (-5) Nov 24 20:48:12 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1901436, rd 194623731, flush 68130, corrupt 3, gen 0 Nov 24 20:48:12 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 1901437, rd 194623731, flush 68130, corrupt 3, gen 0 I included diagnostics supermicro-diagnostics-20241124-2053.zip

For further readers: restarting nginx can sometimes help. At least it did for me. Login to the server via ssh (web terminal doesn't work when websockets don't work) Run the following command. This will restart nginx /etc/rc.d/rc.nginx restart Or if you don't have ssh access somehow (I haven't tested this, but this is an idea) Use the `user scripts` plugin and create a script with the command above. Run the script, and it'll restart nginx, hopefully fixing websockets Edit: hahaha I just saw that I was the original poster years ago! 😂

I wanted to avoid relying on Docker to run my Bind9 server, so I found a way to use the `dnsmasq` that `virsh` manages. This allows me to configure DNS in a more integrated way without Docker. I haven't tested if it survives reboots, but you can always add these changes to a userscript to run on boot. Without further ado... How to Set Up Wildcard DNS with `dnsmasq` I wanted `*.local.co` DNS requests to resolve to my server at `192.168.1.3` (replace with your relevant IP). Here’s how I configured it: 1. Edit the Network Configuration with `virsh` To configure `dnsmasq` to handle `*.local.co`, start by editing your `libvirt` network config: ```bash virsh net-edit default ``` 2. Add Custom `dnsmasq` Options Inside the `<network>` XML configuration, add the `dnsmasq:options` block with the wildcard DNS entry. Also, make sure to include the `dnsmasq` namespace declaration at the top of the `<network>` element: ```xml <network xmlns:dnsmasq="http://libvirt.org/schemas/network/dnsmasq/1.0"> ... some other stuff.... <dns> <forwarder addr='8.8.4.4'/> <forwarder addr='1.1.1.1'/> </dns> <dnsmasq:options> <dnsmasq:option value='address=/.local.co/192.168.1.3'/> <dnsmasq:option value='listen-address=192.168.1.3'/> <dnsmasq:option value='listen-address=192.168.122.1'/> </dnsmasq:options> </network> ``` - Namespace: The `xmlns:dnsmasq="http://libvirt.org/schemas/network/dnsmasq/1.0"` declaration tells `libvirt` to interpret the `dnsmasq` options correctly. - dns: to make sure you can point your router to the server and have other addresses resolved via another DNS - listen-address: make sure to have it listen to both the virtual address but also on the host ip address. You can check what the virtual address is by running `lsof -i :53 | grep dnsmasq` - dnsmasq:option: The `address=/.local.co/192.168.1.3` option resolves all `*.local.co` requests to `192.168.1.3`. 3. Restart the Network After saving the changes, restart the `default` network to apply them: ```bash virsh net-destroy default virsh net-start default ``` 4. Test Your Configuration Run a quick test to ensure the DNS resolution works as expected: ```bash nslookup test.local.co 192.168.1.3 ``` And that’s it! Now, `dnsmasq` on `virsh` should be configured to handle `*.local.co` requests. If it doesn’t persist through reboots, you can set up a userscript to reapply these settings at startup.

syslog-20241111-103928.txt ``` Nov 11 10:00:02 supermicro kernel: btrfs_dev_stat_inc_and_print: 337 callbacks suppressed Nov 11 10:00:02 supermicro kernel: BTRFS error (device sdf1): bdev /dev/nvme0n1p1 errs: wr 335938737, rd 2125849244, flush 5057466, corrupt 72, gen 0 ```

They dropped out once. On start of UnRAID after rebooting for upgrade it couldn't find one of the pool drives. So it wasn't even started yet. The array hadn't been started either. I then rebooted and the array booted just fine, except for the docker service. The whole idea of having multiple disks in the pool is to not loose data on the "cache" pool. I don't understand why a device dropping out would cause these issues. Should I add another disk to the pool? Addition: is there a way to not have to reinstall all containers and images and everything when the docker.img is corrupted? Some of the images are created from source on my pc (when I use a remote DOCKER_HOST to my server) and I need to recreate them all...... PS2: I have older syslogs if that's useful

So after scrub and repair it got broken? I'll recreate. It's getting annoying though. I've had this issue earlier. I don't blame anyone, but I'd like to know what I can do to prevent this

I just did a reboot, just to verify it's really not working. supermicro-diagnostics-20241112-1359.zip

When I go to the Docker tab, the message still says "Docker Service failed to start." When I run `docker ps` there's none started ``` docker ps Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? ```

My bad. I expected the diagnostics to contain the results. Here the results: ``` UUID: 38dc75e4-1cf6-45cd-bb76-e15d45da80d9 Scrub started: Mon Nov 11 20:46:49 2024 Status: finished Duration: 0:13:29 Total to scrub: 671.41GiB Rate: 849.77MiB/s Error summary: no errors found ```

It's still not starting unfortunately. I've added the diagnostics. supermicro-diagnostics-20241111-2141.zip

Sorry for my ignorance. How to run a correcting scrub. And what could cause this? Why does it happen after reboot for upgrade? Shouldn’t it just work after a simple reboot? Feels a bit unsettling to me

After updating and reboot, docker doesn't start anymore. In the docker tab I see the message "Docker Service failed to start." In the docker settings there's no place to start it. What to do? supermicro-diagnostics-20241111-1832.zip

So that's something that I should do from the user script as well then? Everytime I run the script, I copy over the executable file, and it's configs and run it from the userscript. Seems cumbersome, but if it's the best practice, I suppose we'll go with it

I made a small script that I need to add to unraid to run periodically. I know of the user scripts package, and I want to use it to execute the script. I have a config.json and the executable. Where to put it? In /boot there I cannot run `chmod +x` on the executable. What is the best practice to make this work?

@JorgeB Thanks, everything works again I had to recreate my docker containers that I build from source from my dev machine again, but other than that, everything works.

@JorgeB First of all, thank you for helping me so far. Everything seems to work out great I've executed both Scrub and Balance. Those disks seem to be fine. It had corrected a bunch of things. I then tried to start docker again by disabling and enabling the service in Settings > Docker. This gave me the message in the docker tab "Docker Service failed to start." Then with this command, but got the following error: ``` /etc/rc.d/rc.docker start no image mounted at /var/lib/docker ``` Could it be that the docker.img isn't mounted? What to do Edit: added diags supermicro-diagnostics-20240503-0944.zip

@JorgeB Cache pool seems to be fine from what I can see now. Rebuild is in progress for Disk 1. However, Docker doesn't start: "Docker Service failed to start." Diags attached supermicro-diagnostics-20240502-2124.zip Edit: Maybe some useful addition: I see that there's a rootshare/cache/system/docker/docker.img on the cache pool.

Whe I unassign the cache devices, can I mount them to see what the contents are? I have my photoprism importing on the cache Also, where can I see whether the mover has ran lately? @JorgeB Now, after unassigning the "Cache 2" I see the notice "Start will remove the missing cache disk and then bring the array on-line". Why wasn't this shown when "Cache 1" was missing? Is it possible that the Cache Pool is a duplicate from each other? If yes, could this be holding my old cache data?