Write Error: no space left on device

[grattu]

 

Having an issue when I try to spin up a new VM in unraid. I have PXE boot working and when the VM goes to do an automated installation of Centos, I'm getting this error. My btrfs cache drive isn't anywhere close to being full(22.6GB of 500GB), so I know that isn't correct. I've been following along some of the steps described in this thread:

https://forums.unraid.net/topic/54533-624-cache-drive-reporting-quotno-space-left-on-diskquot/

 

There doesn't seem to have been a resolution at the time. Here is my output when I run the commands:

btrfs fi show
Label: none  uuid: 53deda07-5dc7-4d40-b7e8-22fa6d940c33
        Total devices 1 FS bytes used 21.03GiB
        devid    1 size 465.76GiB used 27.06GiB path /dev/sdb1

Label: none  uuid: 97d8ec08-e955-4602-a14e-33743bf27d6e
        Total devices 1 FS bytes used 760.00KiB
        devid    1 size 20.00GiB used 1.52GiB path /dev/loop2

Label: none  uuid: c142d18c-c212-4286-a041-5a1d2addf959
        Total devices 1 FS bytes used 404.00KiB
        devid    1 size 1.00GiB used 126.38MiB path /dev/loop3

 

btrfs fi df /mnt/cache
Data, single: total=23.00GiB, used=21.03GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=2.00GiB, used=304.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B

btrfs fi usage /mnt/cache
Overall:
    Device size:                 465.76GiB
    Device allocated:             27.06GiB
    Device unallocated:          438.70GiB
    Device missing:                  0.00B
    Used:                         21.03GiB
    Free (estimated):            440.67GiB      (min: 221.32GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:               16.00MiB      (used: 0.00B)

Data,single: Size:23.00GiB, Used:21.03GiB (91.42%)
   /dev/sdb1      23.00GiB

Metadata,DUP: Size:2.00GiB, Used:304.00KiB (0.01%)
   /dev/sdb1       4.00GiB

System,DUP: Size:32.00MiB, Used:16.00KiB (0.05%)
   /dev/sdb1      64.00MiB

Unallocated:
   /dev/sdb1     438.70GiB

I can see that there is 438GBs that are unallocated, how do I add that space to Data,single? Currently the 21.03GB of data occupying Data,single is my VMs so hopefully I can do this without having to format the drive 😅

 

Atttached is my diagnostics zip.

 

Cheers

tower-diagnostics-20200425-1016.zip

[JorgeB]

Cache has plenty of space, both the system and appdata shares exist on both cache and array, and there are some duplicate files, decide where you want them (set the appropriate use cache option) and move/delete the other ones, the domains share exists only on the array, I assume you want cache for that, so change it also.

[grattu]

Okay, I've now done that and I am still experiencing the same issue with the same results when running the btrfs commands. This is something I had noticed when I had moved my domains share, that data, single didn't change at all. I had moved the domains share in hopes that would free up space so I could continue trying to build my vms. 

[JorgeB]

Please post current diags.

[grattu]

Here you go

tower-diagnostics-20200428-1012.zip

[JorgeB]

Domains and system shares are still on the array, but that's OK if that's what you want.

 

Cache still has plenty of space, the problem is likely with VM itself, possibly a wrong path somewhere, see if you can copy a large file to cache.

[grattu]

Yep, that seems to have worked. I moved a few large files over and they transferred. The data, single partition expanded to Data,single: Size:36.00GiB, Used:28.58GiB (79.39%) and after I deleted those files it shrank to Data,single: Size:28.00GiB, Used:21.02GiB (75.06%)

 

I'm not sure what any of this means tbh. One of the first things I did try was moving my VMs from /mnt/user/domains to mnt/disk2/domains to no avail. 

[JorgeB]

The allocated chunks are created and deleted as needed, as long as there is unallocated space on the device(s) you should never get out of space errors.

[grattu]

Sooooooo, it turns out I'm an idiot 😅 at some point in time I changed the config I'm using to only use 1GB of ram instead of 2