BTRFS errors

[CyberMew]

I am getting lots of these and I can't any of my media in Plex.

 

Jan 15 13:45:02 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys_seg 3 prio class 0
Jan 15 13:45:29 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:45:29 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:45:33 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys seg 3 prio class 0
Jan 15 13:46:00 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:46:00 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:46:04 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys_ seg 3 prio class 0
Jan 15 13:46:04 Tower kernel: blk update_ request: critical space allocation error, dev loop2, sector 35829304 op 0x1:
(WRITE) flags 0x100000 phys seg 2 prio class 0
Jan 15 13:46:31 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:46:31 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:46:34 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys_seg 3 prio class 0
Jan 15 13:46:34 Tower kernel: blk update_request: critical space allocation error, dev loop2, sector 35842912 op 0x1:
(WRITE) flags 0x100000 phys_seg 4 prio class 0
Jan 15 13:47:02 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:47:02 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:47:05 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys seg 3 prio class 0
Jan 15 13:47:33 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:47:33 Tower kernel: BTRFS error (device loop2) : bad tree block start, want 58310656 have 0
Jan 15 13:47:36 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys_ seg 3 prio class 0
Jan 15 13:48:04 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:48:04 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:48:06 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35829304 op 0x1:
(WRITE) flags 0x100000 phys seg 2 prio class 0
Jan 15 13:48:06 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys_seg 3 prio class 0
Jan 15 13:48:35 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:48:35 Tower kernel: BTRFS error (device loop2): bad tree block start, want 58310656 have 0
Jan 15 13:48:37 Tower kernel: blk update request: critical space allocation error, dev loop2, sector 35842496 op 0x1:
(WRITE) flags 0x100000 phys_ seg 3 prio class 0

Have attached the diagnostics in the thread.

 

Any idea why it happened and how to recover from it?

tower-diagnostics-20230115-1349.zip

[itimpi]

You seem to have completely run out of space on your cache drive and it looks like this has corrupted the docker image file.   Btrfs file system seems to be prone to corruption if they run out of free space.   You also seem to have all your array drives completely full so I think that will also need resolving to stop a similar issue arising shortly after resolving the current one.

 

you also seem to have 50GB allocated to the docker image file?   Is there any reason you made it this large - it would be very rare to need anything like that much.  If it is because it was running out of free space this would suggest a docker container was misconfigured so it was writing to a path internal to the docker image because it was not mapped to external storage.

 

to stop drives completely filling you should use the Minimum Free Space setting that is available for both pools and shares.

 

to resolve your current issue you are going to have to free up space on the cache drive; recreate the docker image file; and then reinstall your docker apps using the Previoys Apps section of Community Applications.   

[CyberMew]

Thanks for checking! Yea I guess Sabnzbd was downloading a very large file into cache and it got full..  

 

And yes I’m ordering a new 18tb drive and remove my smallest drive so I will need to do the swap thing after I resolve this current situation.

 

I have set minimum free space to 5GB but I guess that is not sufficient if all the drives only have 5GB left 😂 

As for the docker image file I previously have this problem so I increased it. I don’t think it’s a mapping issue because my data don’t disappear after I delete and recreate it. Maybe some app is filling the logs internally. I don’t know how to check which app is causing it to happen. 
 

So the errors are related to the docker image file and not my unraid file system correct?

[trurl]

30 minutes ago, CyberMew said:

I have set minimum free space to 5GB

You must set Minimum Free for each of your user shares to larger than the largest file you expect to write to the share. If a disk has more than Minimum, Unraid can choose the disk even if the file won't fit. If a disk has less than Minimum, Unraid will choose a different disk.

 

Similarly, you must set Minimum Free for cache or any other pools to larger than the largest file you expect to write to the pool. If a pool has more than Minimum, Unraid can choose the pool even if the file won't fit. If a pool has less than Minimum, Unraid will choose an array disk instead (overflow) for cache:prefer and cache:yes shares.

 

33 minutes ago, CyberMew said:

ordering a new 18tb drive and remove my smallest drive so I will need to do the swap thing after I resolve this current situation.

Just in case you aren't clear on the way to do this (your terminology is a bit ambiguous), you would replace the small drive with the larger one and rebuild.

 

You may need to do this before you have enough space to do anything else.