Jump to content
  • [6.10.0RC2] VM doesn't start because has no space for logs (but does have).


    NLS
    • Minor

    So I try to start a general use Windows VM I have, that was running fine days ago (also in RC2), with error

    "Unable to write to file /var/log/libvirt/qemu/Windows.log: No space left on device"

     

    Not sure where /var/log is, but I have plenty gigabytes empty both in my boot stick and my Cache disk.

    Clicking "logs" in the VM menu, doesn't list any huge log.

     

    I doubt the issue relates to RC2, it just happens I use RC2. VM was working fine a few days ago (in RC2 and before).

     

    Any ideas?

     




    User Feedback

    Recommended Comments

    Squid

    Posted

    That folder is in RAM, and maxes out at 128M which should be sufficient for years of uptime if there are no issues anywhere

     

    Post your diagnostics

    Rushtallica

    Posted

    I'm having the same issue. Here are my diagnostics

     

    And running the command du -h -d 1 /var/log  initially showed this after rebooting:

    0       /var/log/pwfail
    4.0K    /var/log/swtpm
    20K     /var/log/samba
    0       /var/log/plugins
    0       /var/log/pkgtools
    0       /var/log/nginx
    0       /var/log/nfsd
    12K     /var/log/libvirt
    328K    /var/log

     

     

    BUT it showed this just after shutting down a VM I couldn't get to pass through again and had been showing 84 degrees C! (I've been having problems trying to pass through VM's on my new motherboard/setup, though I also added a cache drive and may have messed things up when trying to move around info and to separate VM info onto the new drive.

    and after doing the command line that you requested earlier now shows (after shutting down the VM but not rebooting yet)

    0       /var/log/pwfail
    0       /var/log/swtpm
    20K     /var/log/samba
    0       /var/log/plugins
    0       /var/log/pkgtools
    0       /var/log/nginx
    0       /var/log/nfsd
    7.6M    /var/log/libvirt
    128M    /var/log

    rushtallica-diagnostics-20211119-1008.zip

    NLS

    Posted (edited)

    Well log bar in dashboard does show 100%.

     

    Attached are my diags.

      

    root@quasar-ultima:/var/log# ls -l
    total 75236
    -rw-r--r-- 1 root   root    24200 Nov 11 09:21 Xorg.0.log
    -rw------- 1 root   root        0 Oct 19 00:05 btmp
    -rw-r--r-- 1 root   root        0 Apr 28  2021 cron
    -rw-r--r-- 1 root   root        0 Apr 28  2021 debug
    -rw-r--r-- 1 root   root    70702 Nov 11 09:20 dmesg
    -rw-r--r-- 1 root   root     8192 Nov 22 22:40 docker.log
    -rw-r--r-- 1 root   root        0 Feb 13  2021 faillog
    -rw-rw-rw- 1 root   root      618 Nov 11 09:21 fluxbox.log
    -rw-r--r-- 1 root   root        0 Apr  8  2000 lastlog
    drwxr-xr-x 3 root   root      120 Nov 11 09:22 libvirt/
    -rw-r--r-- 1 root   root     4096 Nov 23 00:20 maillog
    -rw-r--r-- 1 root   root        0 Nov 11 09:20 mcelog
    -rw-r--r-- 1 root   root        0 Apr 28  2021 messages
    drwxr-xr-x 2 root   root       40 Jun 15 21:38 nfsd/
    drwxr-x--- 2 nobody root       80 Nov 13 04:40 nginx/
    lrwxrwxrwx 1 root   root       24 Oct 19 00:06 packages -> ../lib/pkgtools/packages/
    drwxr-xr-x 5 root   root      100 Nov 13 02:09 pkgtools/
    drwxr-xr-x 2 root   root      600 Nov 22 16:14 plugins/
    drwxr-xr-x 2 root   root       40 Nov 11 10:18 pwfail/
    lrwxrwxrwx 1 root   root       25 Oct 19 00:05 removed_packages -> pkgtools/removed_packages/
    lrwxrwxrwx 1 root   root       24 Oct 19 00:05 removed_scripts -> pkgtools/removed_scripts/
    lrwxrwxrwx 1 root   root       34 Nov 13 02:09 removed_uninstall_scripts -> pkgtools/removed_uninstall_scripts/
    drwxr-xr-x 3 root   root      160 Nov 11 09:21 samba/
    lrwxrwxrwx 1 root   root       23 Oct 19 00:06 scripts -> ../lib/pkgtools/scripts/
    -rw-r--r-- 1 root   root        0 Apr 28  2021 secure
    lrwxrwxrwx 1 root   root       21 Oct 19 00:06 setup -> ../lib/pkgtools/setup/
    -rw-r--r-- 1 root   root        0 Apr 28  2021 spooler
    drwxr-xr-x 3 root   root       60 Oct 11 19:01 swtpm/
    -rw-r--r-- 1 root   root        0 Nov 23 02:23 syslog
    -rw-r--r-- 1 root   root 70582272 Nov 13 04:37 syslog.1
    -rw-r--r-- 1 root   root  6334751 Nov 12 03:22 syslog.2
    -rw-r--r-- 1 root   root        0 Nov 11 09:20 vfio-pci
    -rw-rw-r-- 1 root   utmp     7680 Nov 11 09:21 wtmp
    root@quasar-ultima:/var/log#  du -h -d 1 /var/log
    0       /var/log/pwfail
    0       /var/log/swtpm
    20K     /var/log/samba
    0       /var/log/plugins
    0       /var/log/pkgtools
    55M     /var/log/nginx
    0       /var/log/nfsd
    20K     /var/log/libvirt
    128M    /var/log

     

    Looks like syslog is filling up the space?

    I deleted syslog files and the VM fired up ok.

    But WHO is filling syslog and why syslog is not managed some way to not fill up the quota of the directory?

     

    quasar-ultima-diagnostics-20211123-0222.zip

    Edited by NLS
    ChatNoir

    Posted

    6 hours ago, NLS said:

    But WHO is filling syslog

     

    Nov 12 08:04:47 quasar-ultima rpc.mountd[6064]: refused mount request from 172.17.2.111 for /rootshare (/): not exported

    and other paths.

     

    also plenty of

    Nov 12 08:04:49 quasar-ultima nginx: 2021/11/12 08:04:49 [crit] 10941#10941: ngx_slab_alloc() failed: no memory
    Nov 12 08:04:49 quasar-ultima nginx: 2021/11/12 08:04:49 [error] 10941#10941: shpool alloc failed
    Nov 12 08:04:49 quasar-ultima nginx: 2021/11/12 08:04:49 [error] 10941#10941: nchan: Out of shared memory while allocating message of size 8328. Increase nchan_max_reserved_memory.
    Nov 12 08:04:49 quasar-ultima nginx: 2021/11/12 08:04:49 [error] 10941#10941: *821458 nchan: error publishing message (HTTP status code 500), client: unix:, server: , request: "POST /pub/disks?buffer_length=1 HTTP/1.1", host: "localhost"

     

    TPNuts

    Posted

    I'm having the same issue (on Unraid 6.10.0 rc2) with mine filling up. I have to restart the server to get the log cleared. Mine has to do with the vbios misconfigured for passthrough which results in this error
    2021-12-27T03:40:02.510435Z qemu-system-x86_64: vfio_region_write(0000:21:00.0:region1+0xe5340, 0x0,1) failed: Device or resource busy

    over and over

    When you try to restart the VM after the log is dumped over and over it errors out with this

    Unable to write to file /var/log/libvirt/qemu/Windows 11.log: No space left on device

     

    I am amazed that we do not have a way to stop it from flooding the log so fast.

    BBLV

    Posted (edited)

    On 12/26/2021 at 7:53 PM, TPNuts said:

    I'm having the same issue (on Unraid 6.10.0 rc2) with mine filling up. I have to restart the server to get the log cleared. Mine has to do with the vbios misconfigured for passthrough which results in this error
    2021-12-27T03:40:02.510435Z qemu-system-x86_64: vfio_region_write(0000:21:00.0:region1+0xe5340, 0x0,1) failed: Device or resource busy

    over and over

    When you try to restart the VM after the log is dumped over and over it errors out with this

    Unable to write to file /var/log/libvirt/qemu/Windows 11.log: No space left on device

     

    I am amazed that we do not have a way to stop it from flooding the log so fast.

    I'm having this exact same issue. Did you ever find a fix?

     

    I deleted my libvirt file and rebooted to see if that would reset something, but still running into this same issue with the exact error message as you.

    Edited by BBLV
    BBLV

    Posted

    17 hours ago, tjw said:

    I'm having the same problem today. Can't start Windows 11 VM. Attached are my diagnostics. I have rebooted several times with no luck!

    phoenix-z690-diagnostics-20220319-1722.zip

    You on RC2 or the recently released RC3? I just updated to RC3 and about to do more troubleshooting today. I’ll you know if I can figure it out.

    tjw

    Posted

    I'm on RC3. Tried RC4 but didn't like it. This is a dual boot machine. Running Windows natively has no problems.

     

    BBLV

    Posted (edited)

    FYI I updated to RC4, created a new Windows 11 VM and it fired up right away with VNC - no error messages (I'm remote right now, but will try assigning my new 3090 GPU as soon as I get home later today).

     

    Update: switched over from VNC to the 3090 GPU and the new Windows 11 VM fired up immediately. Loving RC4 so far!

    Edited by BBLV
    tjw

    Posted

    Has anyone resolved this issue? I'm still having the problem. VM won't start and after awhile fix common problems says no space left. I tried creating a separate Windows 11 VM and still get the same issue. Also, now when I try to log into my native Windows 11 I get a diagnosing computer error! After rebooting everything works okay in Windows 11 native.

    BBLV

    Posted (edited)

    6 minutes ago, tjw said:

    Has anyone resolved this issue? I'm still having the problem. VM won't start and after awhile fix common problems says no space left. I tried creating a separate Windows 11 VM and still get the same issue. Also, now when I try to log into my native Windows 11 I get a diagnosing computer error! After rebooting everything works okay in Windows 11 native.

    You update to RC4? I had the issue in RC2, can’t remember if I did troubleshooting in RC3, and RC4 just works for me (I created a brand new Windows 11 install via VM/VNC from scratch on a freshly reformatted NVME drive after updating to RC4 FWIW).

    Edited by BBLV
    tjw

    Posted

    55 minutes ago, BBLV said:

    You update to RC4? I had the issue in RC2, can’t remember if I did troubleshooting in RC3, and RC4 just works for me (I created a brand new Windows 11 install via VM/VNC from scratch on a freshly reformatted NVME drive after updating to RC4 FWIW).

    I'll try it again tonight. Strange though, it was working on rc3 for the past several weeks.

    tjw

    Posted

    Updated to RC4 but no joy! I'm starting to get frustrated. This is the only forum or thread I can find that mentions this problem.

    tjw

    Posted

    I got it figured out. I had both monitors plugged in to the video card. When I pugged the second monitor into the motherboard everything is working as expected. I just don't have dual monitors on the VM. So Happy!

    BBLV

    Posted

    4 hours ago, tjw said:

    I got it figured out. I had both monitors plugged in to the video card. When I pugged the second monitor into the motherboard everything is working as expected. I just don't have dual monitors on the VM. So Happy!

    Weird! Glad you figured it out.



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.

×
×
  • Create New...