Jarsky
Members
-
Joined
-
Last visited
Everything posted by Jarsky
-
Turns out my agentless backups complete fine...or when the VM is powered off. The problem is when trying to do one with qemu-guest-agent so I can do a snapshot backup of a running VM. It then kills libvirtd. So the workaround for me since this is just a home setup, is to allow the VM's to be shutdown to complete the backup. Not a fan of that, but better than killing my whole server. 2020-09-08 14:07:15 information: Elements can be found on the system. attempting backup. 2020-09-08 14:07:15 information: creating local Elements.xml to work with during backup. 2020-09-08 14:07:15 information: /mnt/user/share/BACKUP/VMBackups/Elements exists. continuing. 2020-09-08 14:07:15 information: skip_vm_shutdown is false and use_snapshots is 1. skipping vm shutdown procedure. Elements is running. can_backup_vm set to y. 2020-09-08 14:07:15 information: actually_copy_files is 1. 2020-09-08 14:07:15 information: can_backup_vm flag is y. starting backup of Elements configuration, nvram, and vdisk(s). sending incremental file list Elements.xml sent 6,545 bytes received 35 bytes 13,160.00 bytes/sec total size is 6,455 speedup is 0.98 2020-09-08 14:07:15 information: copy of Elements.xml to /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_Elements.xml complete. 2020-09-08 14:07:15 information: Elements does not appear to have an nvram file. skipping. 2020-09-08 14:07:15 information: able to perform snapshot for disk /mnt/nvme_mirror/VM/Elements/elements.img on Elements. use_snapshots is 1. vm_state is running. vdisk_type is raw 2020-09-08 14:07:15 information: qemu agent found. enabling quiesce on snapshot. Domain snapshot Elements-elements.snap created 2020-09-08 14:07:15 information: snapshot command succeeded on elements.snap for Elements. /mnt/nvme_mirror/VM/Elements/elements.img : 40.77% (21474836480 => 8754302341 bytes, /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_elements.img.zst) 2020-09-08 14:07:54 information: copy of /mnt/nvme_mirror/VM/Elements/elements.img to /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_elements.img.zst complete. 2020-09-08 14:07:54 information: backup of /mnt/nvme_mirror/VM/Elements/elements.img vdisk to /mnt/user/share/BACKUP/VMBackups/Elements/20200908_1406_elements.img.zst complete. error: Disconnected from qemu:///system due to end of file error: internal error: client socket is closed
-
Just checking in to see if anyone has got this working OK on 6.9.0 without it crashing libvirtd?
-
Is there someone that will be carrying this plugin forward? As of the new UnRAID 6.9.0 this plugin is having major issues for me. When I run a backup, it breaks the libvirtd service, so it can not discover the VM's. They are still running, however I then have to shutdown all the running VM's from within the guest and use lsof to close any further open files and reboot the entire server so I can get the libvirtd service back up and manage my VM's again. I have snapshots enabled and qemu-guest-agent installed on all the VM's. Sometimes it gets through 1 or 2 VM's, sometimes none. If a snapshot has been taken, when it fails, it also doesnt remove the snap disk from the VM's XML so after reboot I have to manually edit the XML back to get the VM working again (since the snap disk no longer exists) I see nothing in the /var/log/libvirt/libvirtd.log Here are the 3 last backup logs from one of my scheduled configs 20200705_0000_unraid-vmbackup_error.log 20200706_0551_unraid-vmbackup_error.log 20200703_1700_unraid-vmbackup_error.log
-
I wonder if anyone else is having this issue? I have several Ubuntu VM's mounting my primary share from the UnRAID pool via SMB. One of them in particular which runs qBitTorrent has high writes (up to approx 100MB/s). Since upgrading to the 6.9.0 build (from 6.8.3) the speed is constantly dropping as smbd appears to panic which loses connection. From my Ubuntu VM I can see smbd loses connection and blocks the app for 2 minutes [18358.622226] cifs_vfs_err: 23 callbacks suppressed [18358.622229] CIFS VFS: Server tower has not responded in 180 seconds. Reconnecting... [18363.876639] CIFS VFS: Free previous auth_key.response = 00000000ee7c66f6 [18367.456112] INFO: task qbittorrent-nox:16780 blocked for more than 120 seconds. When this occurs, I see the below errors on UnRAID in the SMB logs [2020/06/30 18:16:34.725129, 0] ../../source3/smbd/close.c:648(assert_no_pending_aio) assert_no_pending_aio: fsp->num_aio_requests=1 [2020/06/30 18:16:34.725152, 0] ../../source3/lib/util.c:829(smb_panic_s3) PANIC (pid 28606): can not close with outstanding aio requests [2020/06/30 18:16:34.725236, 0] ../../lib/util/fault.c:222(log_stack_trace) BACKTRACE: #0 log_stack_trace + 0x39 [ip=0x14abc3605e39] [sp=0x7fffb7a7f030] #1 smb_panic_s3 + 0x23 [ip=0x14abc3127f73] [sp=0x7fffb7a7f970] #2 smb_panic + 0x2f [ip=0x14abc360604f] [sp=0x7fffb7a7f990] #3 create_file_default + 0x71f [ip=0x14abc34361cf] [sp=0x7fffb7a7faa0] #4 close_file + 0xc3 [ip=0x14abc3436b53] [sp=0x7fffb7a7fab0] #5 file_close_user + 0x35 [ip=0x14abc33dc485] [sp=0x7fffb7a7fcd0] #6 smbXsrv_session_logoff + 0x4d [ip=0x14abc347dfdd] [sp=0x7fffb7a7fcf0] #7 smbXsrv_session_logoff + 0x3e2 [ip=0x14abc347e372] [sp=0x7fffb7a7fd40] #8 dbwrap_unmarshall + 0x186 [ip=0x14abc21606b6] [sp=0x7fffb7a7fd60] #9 dbwrap_unmarshall + 0x3bb [ip=0x14abc21608eb] [sp=0x7fffb7a7fe20] #10 dbwrap_traverse + 0x7 [ip=0x14abc215ef37] [sp=0x7fffb7a7fe50] #11 smbXsrv_session_logoff_all + 0x5c [ip=0x14abc347e52c] [sp=0x7fffb7a7fe60] #12 smbXsrv_open_cleanup + 0x4d2 [ip=0x14abc3483ab2] [sp=0x7fffb7a7fea0] #13 smbd_exit_server_cleanly + 0x10 [ip=0x14abc3484050] [sp=0x7fffb7a7ff00] #14 exit_server_cleanly + 0x14 [ip=0x14abc2a44284] [sp=0x7fffb7a7ff10] #15 smbd_server_connection_terminate_ex + 0x111 [ip=0x14abc345fe91] [sp=0x7fffb7a7ff20] #16 smbd_smb2_request_dispatch_immediate + 0x569 [ip=0x14abc3462ca9] [sp=0x7fffb7a7ff50] #17 tevent_common_invoke_fd_handler + 0x7d [ip=0x14abc29f070d] [sp=0x7fffb7a7ffc0] #18 tevent_wakeup_recv + 0x1097 [ip=0x14abc29f6a77] [sp=0x7fffb7a7fff0] #19 tevent_cleanup_pending_signal_handlers + 0xb7 [ip=0x14abc29f4c07] [sp=0x7fffb7a80050] #20 _tevent_loop_once + 0x94 [ip=0x14abc29efdf4] [sp=0x7fffb7a80070] #21 tevent_common_loop_wait + 0x1b [ip=0x14abc29f009b] [sp=0x7fffb7a800a0] #22 tevent_cleanup_pending_signal_handlers + 0x57 [ip=0x14abc29f4ba7] [sp=0x7fffb7a800c0] #23 smbd_process + 0x7a7 [ip=0x14abc34522f7] [sp=0x7fffb7a800e0] #24 samba_tevent_glib_glue_create + 0x2291 [ip=0x563fff42feb1] [sp=0x7fffb7a80170] #25 tevent_common_invoke_fd_handler + 0x7d [ip=0x14abc29f070d] [sp=0x7fffb7a80240] #26 tevent_wakeup_recv + 0x1097 [ip=0x14abc29f6a77] [sp=0x7fffb7a80270] #27 tevent_cleanup_pending_signal_handlers + 0xb7 [ip=0x14abc29f4c07] [sp=0x7fffb7a802d0] #28 _tevent_loop_once + 0x94 [ip=0x14abc29efdf4] [sp=0x7fffb7a802f0] #29 tevent_common_loop_wait + 0x1b [ip=0x14abc29f009b] [sp=0x7fffb7a80320] #30 tevent_cleanup_pending_signal_handlers + 0x57 [ip=0x14abc29f4ba7] [sp=0x7fffb7a80340] #31 main + 0x1b2f [ip=0x563fff429c1f] [sp=0x7fffb7a80360] #32 __libc_start_main + 0xeb [ip=0x14abc2700e5b] [sp=0x7fffb7a80710] #33 _start + 0x2a [ip=0x563fff429ffa] [sp=0x7fffb7a807d0] I have tried adjusting my mount command to smb3 and also to disable cache but still the same, heres my /etc/fstab //tower/share /mnt/share cifs vers=3.0,cache=none,credentials=/home/user/.smbcredentials,uid=1000,gid=1010,iocharset=utf8,noperm 0 0 //tower/plexmediaserver /mnt/plexmediaserver cifs vers=3.0,cache=none,credentials=/home/user/.smbcredentials,uid=1000,gid=1010,iocharset=utf8,noperm 0 0
