BTRFS critical: unable to find logical [x] length [y]

[updatenil]

Hello, first time writer here on the forum.

 

I installed Unraid last year in the midst of some unusual power outages. Despite my best efforts there were some interruptions to my Unraid server during the transfer of some files. I saw some errors during parity checks and within an NVME pool, but did not really visibly impact my experience.

 

Today, I recently had some docker updates done and now my Unraid server GUI is relatively less responsive or unresponsive. I am able to access the server itself and input commands - mainly powerdown - in an effort to see if a quick reboot would do the trick. So far no luck.

 

In the terminal are these messages repeated over and over:

 

BTRFS critical (device nvmeOn1p1): unable to find logical 9223372038827163648 length 4096
BTRFS critical (device nvmeOn1p1): unable to find logical 9223372038827163648 length 16384

 

Are there any terminal commands I can use to address this issue?

 

I can sign into the GUI, but mileage varies as far as how far I can even load certain pages.

 

Under the GUI, Fix Common Problems indicates that errors have been found with the sever, but I'm not able to load the plugin properly to see what it says. I've also tried to download the diagnostics through the GUI with no luck.

 

 

Edited June 13, 2023 by updatenil

[trurl]

Try to get Diagnostics from command line 

[updatenil]

Thanks, I had some time to run the diagnostics command.atlus-diagnostics-20230613-1434.zip

[JorgeB]

There's some corruption detected on cache, run a correcting scrub and post the output.

[updatenil]

I took a photo of the output.

 

[JorgeB]

You could have used the GUI, please post new diags.

[updatenil]

atlus-diagnostics-20230614-0831.zip

 

Thanks. I normally interact with the GUI by accessing it on another computer on the network.

 

When I booted up the server into the GUI it works normally. I was able to get a new diagnostics file.

 

I realize I'm just not able to easily access the Unraid server on the network. I also notice that the server isn't able to reach GitHub.

 

Thanks for your kind attention to this.

[JorgeB]

Jun 14 08:26:41 Atlus kernel: br0: received packet on bond0 with own address as source address (addr:d8:5e:d3:0d:30:a9, vlan:0)

This usually means a network configuration error, re-configure or reset the settings.

[updatenil]

I haven't really interacted with network settings.

 

Are there some settings I should look over?

 

Is there a how-to to reset the network settings?

[JorgeB]

You LAN settings look pretty standard, possibly it's coming from a docker container, you can stop the service, reboot then start one container at a time to see if it's one of them. 

[updatenil]

I tried to turn the docker service back on and I am unable to.

 

I also rebooted the server and the docker service doesn't come back. 

[trurl]

Post new diagnostics

[updatenil]

I rebooted my router.

 

I also shut off enable bonding under the Network Settings. (no idea if that helped with the earlier messages about bond0)

 

At some point I am now able to log into the server from within the network, which as far as I'm concerned is progress.

 

atlus-diagnostics-20230614-2037.zip

Edited June 15, 2023 by updatenil

[JorgeB]

Cache filesystem crashed due to corruption and went read-only, you should backup and re-format the pool.

[updatenil]

I'll back up what's on the cache.

 

For my reference, I notice that there's a file share named system that is set to prefer cache. 

 

Is there a safe procedure for me to back up the contents, and then reformat the cache without causing any needless disruption?

[JorgeB]

Basically just copy anything important that you can elsewhere, like the array, you can use your favorite tool like midnight commander or the Dynamix file manager for example, then disable docker and VM services if still enabled, format the pool and copy the data back, and re-enable the services, and see if all works, docker may require a new image, but that's simple to do.

[updatenil]

@JorgeB and @trurl

 

I thank you both for your kind attention to my issue. 

 

I am very much a layperson with a keen interest in setting up my humble homelab that's grown to this point. I would never have reached this point without these communities. Thank you again.

 

In summary, due to a long standing corruption in the cache from some power outages last year, the docker service finally stopped working. 

 

Solution: [1] stop docker and vm, [2] backup all important files from the impacted cache, [3] stop the array, [4] format the target cache, [5] return the files to the cache, and [5] restart docker.

 

My issues resolved at this point and I had no other steps to take.

Edited June 16, 2023 by updatenil