A few of my Docker containers recently stopped working and when i check their logs, i see that i am getting errors due to no space left on device. Here is the log from Postgres 12.5:

2025-12-22 22:46:38.262 EST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432

2025-12-22 22:46:38.262 EST [1] LOG: listening on IPv6 address "::", port 5432

2025-12-22 22:46:38.481 EST [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

2025-12-22 22:46:38.535 EST [1] FATAL: could not access status of transaction 0

2025-12-22 22:46:38.535 EST [1] DETAIL: Could not open file "pg_notify/0000": No space left on device.

2025-12-22 22:46:38.544 EST [1] LOG: database system is shut down

2025-12-23 11:04:19.285 EST [1] FATAL: could not create lock file "postmaster.pid": No space left on device

All of my shares, including appdata have free space in them, including free space on the cache drive.

If i try to create the postmaster.pid file manually, i get the same error message:

root@dachora ~ # touch /mnt/cache/appdata/postgres/data/postmaster.pid

touch: cannot touch '/mnt/cache/appdata/postgres/data/postmaster.pid': No space left on device

root@dachora ~ # df -h /mnt/cache/appdata/*

Filesystem Size Used Avail Use% Mounted on

/dev/nvme0n1p1 932G 597G 335G 65% /mnt/cache

df reports the correct free space

Here are the settings for my appdata share

Any ideas what else i can try to troubleshoot this issue?

Attach Diagnostics ZIP to your NEXT post in this thread.

Diagnostics.zip: dachora-diagnostics-20251223-1147.zip

Looks like cache may be readonly

How would i check and change it back to read/write?

There appears to be a container constantly restarting; see if you can find out which by looking at their uptimes.

Then reboot and post new diags after array start.

Hello again,

i've kind of let this issue drag on and have just not been using my unraid server. I just checked on it again and it's giving the same problems after a fresh restart. Here is the diagnostics files after a fresh restart.

dachora-diagnostics-20260426-2247.zip

Thanks

On 12/23/2025 at 1:30 PM, JorgeB said:

There appears to be a container constantly restarting; see if you can find out which by looking at their uptimes.

Then reboot and post new diags after array start.

It looks like Plex Media Server is trying to restart after failing to write to the cache drive. If i look at the logs, it's just constantly trying to stop and start.

Starting Plex Media Server.
Stopping Plex Media Server.
Starting Plex Media Server.
Stopping Plex Media Server.
Starting Plex Media Server.
Stopping Plex Media Server.
Starting Plex Media Server.
Stopping Plex Media Server.
Starting Plex Media Server.
Stopping Plex Media Server.
Starting Plex Media Server.
PMS: failure detected. Read/write access is required for path: /config/Library/Application Support/Plex Media Server
Stopping Plex Media Server.
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
Starting Plex Media Server.
PMS: failure detected. Read/write access is required for path: /config/Library/Application Support/Plex Media Server
Stopping Plex Media Server.
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Edited April 27Apr 27 by oro
Added info about PMS logs

Disable that container for now, and post new diags if you still see the issue.

10 hours ago, oro said:

failing to write to the cache drive

Apr 26 22:33:36 dachora webgui: File_put_contents_atomic failed to write / rename /mnt/cache/1232932549.tmp
Apr 26 22:33:36 dachora root: Fix Common Problems: Error: Unable to write to cache

FCP can't write to it too, didn't you get notified that your cache was unwritable?

5 hours ago, trurl said:

Apr 26 22:33:36 dachora webgui: File_put_contents_atomic failed to write / rename /mnt/cache/1232932549.tmp
Apr 26 22:33:36 dachora root: Fix Common Problems: Error: Unable to write to cache

FCP can't write to it too, didn't you get notified that your cache was unwritable?

I didn't check FCP before, but yeah it is also saying that the cache can't be written to. FCP lead me to this post which suggested to run memtest if the cache disk is unwritable, so i ran memtest86 and it didn't find any errors.

1 hour ago, oro said:

didn't check FCP before

Setup Notifications to alert you immediately by email or other agent as soon as a problem is detected.

See if scrub will fix cache.

3 hours ago, trurl said:

Setup Notifications to alert you immediately by email or other agent as soon as a problem is detected.

See if scrub will fix cache.

Thanks, i'll set that up when i get things back to normal working order. I ran a scrub on the cache and it didn't find any errors and the problem persists.

UUID:             5b4b98af-a025-495d-9a7a-2b52a6595ed5
Scrub started:    Mon Apr 27 19:56:32 2026
Status:           finished
Duration:         0:06:56
Total to scrub:   585.70GiB
Rate:             1.41GiB/s
Error summary:    no errors found

If the container that was always restarting is disabled now, reboot to clear the logs and post new diags after there are errors again.

Ok, i set Plex to not auto start and my array to not start on boot and did a clean reboot and took diagnostics after a fresh boot before the array started and after.

Before: dachora-diagnostics-20260428-2016_disks_unmounted.zip

After: dachora-diagnostics-20260428-2039_array_started.zip

Hopefully these help narrow down the issue, let me know if there are any more tests i can run.

Go to the Docker page and check the UPTIME column of each of your containers and see which have short uptime.

Ok. Crowdsec had an uptime of about a minute, invidious had an uptime of 34 seconds, and immich server had an uptime of 3 minutes. Everything else either failed to start or had an uptime that matched the server's full uptime. i set each of the ones with a short uptime to no longer start and did a reboot of the server and started the array. For some reason, once the array is started, one of the disks is now not recognized as installed. It's a USB drive and for sure worth replacing if it's a problem. Here's the latest diagnostics after rebooting: dachora-diagnostics-20260429-0135.zip

dachora-diagnostics-20260429-0135.zip

Edited April 29Apr 29 by oro
clarity

The pool filesystem is completely allocated, preventing a new metadata chunk from being created. This will result in a write error; see if a balance helps.

https://forums.unraid.net/topic/62230-out-of-space-errors-on-cache-drive/?do=findComment&comment=610551

It's been a while since I've seen this, possibly related to you still being on 7.0.0. You should update to 7.2 as the newer kernel should help with this not happening in the future.

7 minutes ago, JorgeB said:

The pool filesystem is completely allocated, preventing a new metadata chunk from being created. This will result in a write error; see if a balance helps.

https://forums.unraid.net/topic/62230-out-of-space-errors-on-cache-drive/?do=findComment&comment=610551

It's been a while since I've seen this, possibly related to you still being on 7.0.0. You should update to 7.2 as the newer kernel should help with this not happening in the future.

Results of attempting a balance failed:

root@dachora ~ # btrfs balance start -dusage=75 /mnt/cache
ERROR: error during balancing '/mnt/cache': No space left on device
There may be more info in syslog - try dmesg | tail
root@dachora ~ # btrfs balance start -dusage=50 /mnt/cache                                                                    1 ↵
ERROR: error during balancing '/mnt/cache': No space left on device
There may be more info in syslog - try dmesg | tail
root@dachora ~ # btrfs balance start -dusage=25 /mnt/cache                                                                    1 ↵
ERROR: error during balancing '/mnt/cache': No space left on device
There may be more info in syslog - try dmesg | tail
root@dachora ~ # dmesg | tail                                                                                                 1 ↵
[ 4101.373688] docker0: port 5(veth87add45) entered forwarding state
[ 4628.678551] BTRFS info (device nvme0n1p1): balance: start -dusage=75
[ 4640.702180] BTRFS info (device nvme0n1p1): 523 enospc errors during balance
[ 4640.702184] BTRFS info (device nvme0n1p1): balance: ended with status: -28
[ 4667.333983] BTRFS info (device nvme0n1p1): balance: start -dusage=50
[ 4671.893465] BTRFS info (device nvme0n1p1): 358 enospc errors during balance
[ 4671.893470] BTRFS info (device nvme0n1p1): balance: ended with status: -28
[ 4682.782415] BTRFS info (device nvme0n1p1): balance: start -dusage=25
[ 4692.565441] BTRFS info (device nvme0n1p1): 195 enospc errors during balance
[ 4692.565447] BTRFS info (device nvme0n1p1): balance: ended with status: -28

Is this a known issue in 7.0.0? Usually i like to avoid making updates while the system is in a non-working state, unless the update specifically addresses the issue

It's a known btrfs issue, but it should no longer happen with more recent kernels.

If the balance is not working, best bet is to back up and reformat the pool

8 hours ago, oro said:

one of the disks is now not recognized as installed. It's a USB drive

USB NOT recommended for assigned disks for many reasons.

On 4/29/2026 at 3:57 AM, JorgeB said:

It's a known btrfs issue, but it should no longer happen with more recent kernels.

If the balance is not working, best bet is to back up and reformat the pool

Ok, i haven't done a full back up and restore before. Fortunately, there isn't anything stored on the server that can't be re-ripped/downloaded/restored some other way, so it wouldn't be catastrophic if a backup and restore failed, it'd just mean a lot more time and effort to get everything back on there. i was thinking of setting up Borg and borgbase as a longterm offsite backup solution as well, do you have any suggestions?

On 4/29/2026 at 10:05 AM, trurl said:

USB NOT recommended for assigned disks for many reasons.

it's on my list of things to replace when i get things back in order 😅