Everything posted by bastl
-
Unraid OS version 6.7.2 available
This thing is still running after all that time? 😮
-
[Support] cheesemarathons repo
I also randomly spotted another Bitwarden container popping up in the community apps and was kinda confused. Maybe it's that easy to point it to the already existing data folder. Did someone already tested this?
-
[Plugin] unbalanced
@Thebadmf Either enter the following into a command line directly on unraid Or go to the "Main" page in unraid and click the folder icon all the way on the right behind your unraid flash device and go into the "logs" folder to find the unbalance.log file
-
Unraid Feature Request Wishlist
Snapshots as argument for preventing ransomware is a bit out of date. Almost all actual ransomware I read about deletes all versioning the user has access to provided by snapshots on the share level. Nontheless snapshots for VMs are really helpful and quick and I use it for almost a year with a script already. Reading through all the features in the vote, I WANT THEM ALL 😁
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
@jbartlett Is there a reason, why lstopo is only available via the unraid gui boot. Can't remember which Unraid version first implemented it, I think short after SIO's video Limetech added it.
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
@jbartlett This looks really promising and might help a lot of people to get a better understanding of the topology of their CPUs. Let's hope this will make it into a final release in the future 👍 Edit: You now have to add more infos from lstopo to show where the RAM and all the PCIE devices are connected. 😂 Just an small button for running LSTOPO to present a popup with the result would be enough for me, or even better, parse the PCI adresses and read them from the devices list and show also the names besides the depending blocks 😁
-
unraid-autovmbackup: automate backup of virtual machines in unRAID - v0.4
@JTok Thanks for the explanation. If you have some test builds and need someone to test, feel free to ask me 😉
-
unraid-autovmbackup: automate backup of virtual machines in unRAID - v0.4
Thanks @JTok for the quick fix. In my scenario the backup is created and compressed as it should be now. But there are a couple warnings when the script is running. Maybe it's something important for other setups. Not sure if the "namespace warning" has an impact on the xml if the script is run with snapshots. During testing the old script with snapshots turned on I noticed that the xml has been modified to add the "vdisk1.snap". Currently running 6.8 RC6. Maybe something has changed with the newer Qemu 4.1.1 version. I have used the exact same settings as posted before with the fixed script. Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute The line 5 in the xml looks like the following in my VM: <vmtemplate xmlns="unraid" name="Debian" icon="linux-mint.png" os="debian"/> Another "maybe" warning is at the end of the logs: /tmp/user.scripts/tmpScripts/vm_sicherung/script: line 3301: [: : integer expression expected In case I should test something or you need some more info, let me know! Full log: Script location: /tmp/user.scripts/tmpScripts/vm_sicherung/script Note that closing this window will abort the execution of this script 2019-11-21 21:01 information: official_script_name is script. script name is valid. continuing. 2019-11-21 21:01 information: enabled is 1. script is enabled. continuing. 2019-11-21 21:01 information: backup_location is /mnt/user/Backup/vm_sicherung. this location exists. continuing. 2019-11-21 21:01 information: backup_location is /mnt/user/Backup/vm_sicherung. this location is writable. continuing. 2019-11-21 21:01 information: timestamp_files is 1. timestamp will be added to backup files. 2019-11-21 21:01 information: /mnt/user/Backup/vm_sicherung/logs/ exists. continuing. 2019-11-21 21:01 information: log_file_subfolder is /mnt/user/Backup/vm_sicherung/logs/. this location exists. continuing. 2019-11-21 21:01 information: log_file_subfolder is /mnt/user/Backup/vm_sicherung/logs/. this location is writable. continuing. 2019-11-21 21:01 Start logging to log file. 2019-11-21 21:01 information: send_notifications is 1. notifications will be sent. 2019-11-21 21:01 information: only_send_error_notifications is 0. normal notifications will be sent if send_notifications is enabled. 2019-11-21 21:01 information: keep_log_file is 1. log files will be kept. 2019-11-21 21:01 information: number_of_log_files_to_keep is 12. this is probably a sufficient number of log files to keep. 2019-11-21 21:01 information: snap extension not found in vdisk_extensions_to_skip. extension was added. 2019-11-21 21:01 information: use_snapshots is 0. vms will not be backed up using snapshots. 2019-11-21 21:01 information: kill_vm_if_cant_shutdown is 1. vms will be forced to shutdown if a clean shutdown can not be detected. 2019-11-21 21:01 information: set_vm_to_original_state is 1. vms will be set to their original state after backup. 2019-11-21 21:01 information: number_of_days_to_keep_backups is 30. this is probably a sufficient number of days to keep your backups. 2019-11-21 21:01 information: number_of_backups_to_keep is 5. this is probably a sufficient number of backups to keep. 2019-11-21 21:01 information: compress_backups is 1. backups will be compressed. 2019-11-21 21:01 information: enable_reconstruct_write is 0. reconstruct write will not be enabled by this script. 2019-11-21 21:01 information: compare_files is 0. files will not be compared after backups. 2019-11-21 21:01 information: backup_xml is 1. vms will have their xml configurations backed up. 2019-11-21 21:01 information: backup_nvram is 1. vms will have their nvram backed up. 2019-11-21 21:01 information: backup_vdisks is 1. vms will have their vdisks backed up. 2019-11-21 21:01 information: start_vm_after_backup is 0. vms will not be started following successful backup. 2019-11-21 21:01 information: start_vm_after_failure is 0. vms will not be started following an unsuccessful backup. 2019-11-21 21:01 information: disable_delta_sync is 0. rsync will be used to perform delta sync backups. 2019-11-21 21:01 information: rsync_only is 0. cp will be used when applicable. 2019-11-21 21:01 information: actually_copy_files is 1. files will be copied. 2019-11-21 21:01 information: clean_shutdown_checks is 20. this is probably a sufficient number of shutdown checks. 2019-11-21 21:01 information: seconds_to_wait is 30. this is probably a sufficient number of seconds to wait between shutdown checks. 2019-11-21 21:01 information: keep_error_log_file is 1. error log files will be kept. 2019-11-21 21:01 information: number_of_error_log_files_to_keep is 12. this is probably a sufficient error number of log files to keep. 2019-11-21 21:01 information: started attempt to backup Mint to /mnt/user/Backup/vm_sicherung 2019-11-21 21:01 information: Mint can be found on the system. attempting backup. 2019-11-21 21:01 information: /mnt/user/Backup/vm_sicherung/Mint exists. continuing. Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ 2019-11-21 21:01 information: skip_vm_shutdown is false. beginning vm shutdown procedure. 2019-11-21 21:01 action: Mint is running. shutting down. Domain Mint is being shutdown 2019-11-21 21:01 information: performing 20 30 second cycles waiting for Mint to shutdown cleanly 2019-11-21 21:01 information: cycle 1 of 20: waiting 30 seconds before checking if the vm has shutdown 2019-11-21 21:02 information: Mint is shut off. can_backup_vm set to y 2019-11-21 21:02 action: actually_copy_files is 1. 2019-11-21 21:02 action: can_backup_vm flag is y. starting backup of Mint configuration, nvram, and vdisk(s). sending incremental file list Mint.xml sent 6,483 bytes received 35 bytes 13,036.00 bytes/sec total size is 6,389 speedup is 0.98 2019-11-21 21:02 information: backup of Mint xml configuration to /mnt/user/Backup/vm_sicherung/Mint/20191121_2101_Mint.xml complete. Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ sending incremental file list 065a6081-e954-0913-370d-b6001262fb61_VARS-pure-efi.fd sent 131,240 bytes received 35 bytes 262,550.00 bytes/sec total size is 131,072 speedup is 1.00 2019-11-21 21:02 information: backup of Mint nvram to /mnt/user/Backup/vm_sicherung/Mint/20191121_2101_065a6081-e954-0913-370d-b6001262fb61_VARS-pure-efi.fd complete. Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ Mint.xml:5: namespace warning : xmlns: URI unraid is not absolute ^ '/mnt/user/VMs/Mint/vdisk1.img' -> '/mnt/user/Backup/vm_sicherung/Mint/20191121_2101_vdisk1.img' 2019-11-21 21:05 information: backup of vdisk1.img vdisk to /mnt/user/Backup/vm_sicherung/Mint/20191121_2101_vdisk1.img complete. 2019-11-21 21:05 information: the extensions of the vdisks that were backed up are img. 2019-11-21 21:05 action: vm_original_state is running. starting Mint. Domain Mint started 2019-11-21 21:05 information: started creating new tarball. 2019-11-21 21:05 information: creating blank backup file list at /mnt/user/Backup/vm_sicherung/Mint/backup_file_list.txt. 20191121_2101_vdisk1.img 20191121_2101_Mint.xml 20191121_2101_065a6081-e954-0913-370d-b6001262fb61_VARS-pure-efi.fd 2019-11-21 21:24 information: removing backup file list at /mnt/user/Backup/vm_sicherung/Mint/backup_file_list.txt. 2019-11-21 21:24 information: finished creating new tarball. 2019-11-21 21:24 information: removed xml, nvram, and image files that were compressed. 2019-11-21 21:24 information: backup of Mint to /mnt/user/Backup/vm_sicherung/Mint completed. 2019-11-21 21:24 information: cleaning out backups older than 30 days in location ONLY if newer files exist in /mnt/user/Backup/vm_sicherung/Mint/ 2019-11-21 21:24 information: cleaning out backups over 5 in location /mnt/user/Backup/vm_sicherung/Mint/ find: '/mnt/user/Backup/vm_sicherung/Mint/*.xml': No such file or directory 2019-11-21 21:24 information: did not find any config files to remove. find: '/mnt/user/Backup/vm_sicherung/Mint/*.fd': No such file or directory 2019-11-21 21:24 information: did not find any nvram files to remove. /tmp/user.scripts/tmpScripts/vm_sicherung/script: line 3301: [: : integer expression expected /tmp/user.scripts/tmpScripts/vm_sicherung/script: line 3301: [: : integer expression expected /tmp/user.scripts/tmpScripts/vm_sicherung/script: line 3301: [: : integer expression expected /tmp/user.scripts/tmpScripts/vm_sicherung/script: line 3301: [: : integer expression expected 2019-11-21 21:24 information: did not find any image files to remove. 2019-11-21 21:24 information: did not find any compressed files to remove. 2019-11-21 21:24 information: finished attempt to backup Mint to /mnt/user/Backup/vm_sicherung. 2019-11-21 21:24 information: cleaning out logs over 12. 2019-11-21 21:24 information: removed '/mnt/user/Backup/vm_sicherung/logs/20191027_0630_unraid-vmbackup.log' 2019-11-21 21:24 information: cleaning out error logs over 12. 2019-11-21 21:24 information: did not find any error log files to remove. 2019-11-21 21:24 Stop logging to log file.
-
unraid-autovmbackup: automate backup of virtual machines in unRAID - v0.4
@JTok I had the time now to test the new script. Unfortunatly it didn't worked with all my settings I used before. Snapshots are turned off and the following is shown in the error log. 2019-11-21 13:58 information: use_snapshots is 0. vms will not be backed up using snapshots. . . 2019-11-21 13:59 failure: unable to perform snapshot for disk /mnt/user/VMs/Mint/vdisk1.img on Mint. skipping backup for Mint to prevent data loss. no cleanup will be performed for this vm. use_snapshots is 0. vm_state is shut off. vdisk_type is qcow2. VM is shutdown correctly, XML and vram is transferred to the destination and at the point where the vdisk should be copied, it errors out the script. VM isn't returned to it's original state but it's setup in the script to do so. Full log: 2019-11-21 13:58 Start logging to log file. 2019-11-21 13:58 information: keep_log_file is 1. log files will be kept. 2019-11-21 13:58 information: number_of_log_files_to_keep is 12. this is probably a sufficient number of log files to keep. 2019-11-21 13:58 information: snap extension not found in vdisk_extensions_to_skip. extension was added. 2019-11-21 13:58 information: use_snapshots is 0. vms will not be backed up using snapshots. 2019-11-21 13:58 information: kill_vm_if_cant_shutdown is 1. vms will be forced to shutdown if a clean shutdown can not be detected. 2019-11-21 13:58 information: set_vm_to_original_state is 1. vms will be set to their original state after backup. 2019-11-21 13:58 information: number_of_days_to_keep_backups is 30. this is probably a sufficient number of days to keep your backups. 2019-11-21 13:58 information: number_of_backups_to_keep is 5. this is probably a sufficient number of backups to keep. 2019-11-21 13:58 information: compress_backups is 1. backups will be compressed. 2019-11-21 13:58 information: enable_reconstruct_write is 0. reconstruct write will not be enabled by this script. 2019-11-21 13:58 information: compare_files is 0. files will not be compared after backups. 2019-11-21 13:58 information: backup_xml is 1. vms will have their xml configurations backed up. 2019-11-21 13:58 information: backup_nvram is 1. vms will have their nvram backed up. 2019-11-21 13:58 information: backup_vdisks is 1. vms will have their vdisks backed up. 2019-11-21 13:58 information: start_vm_after_backup is 0. vms will not be started following successful backup. 2019-11-21 13:58 information: start_vm_after_failure is 0. vms will not be started following an unsuccessful backup. 2019-11-21 13:58 information: disable_delta_sync is 0. rsync will be used to perform delta sync backups. 2019-11-21 13:58 information: rsync_only is 0. cp will be used when applicable. 2019-11-21 13:58 information: actually_copy_files is 1. files will be copied. 2019-11-21 13:58 information: clean_shutdown_checks is 20. this is probably a sufficient number of shutdown checks. 2019-11-21 13:58 information: seconds_to_wait is 30. this is probably a sufficient number of seconds to wait between shutdown checks. 2019-11-21 13:58 information: keep_error_log_file is 1. error log files will be kept. 2019-11-21 13:58 information: number_of_error_log_files_to_keep is 12. this is probably a sufficient error number of log files to keep. 2019-11-21 13:58 information: started attempt to backup Mint to /mnt/user/Backup/vm_sicherung 2019-11-21 13:58 information: Mint can be found on the system. attempting backup. 2019-11-21 13:58 information: /mnt/user/Backup/vm_sicherung/Mint exists. continuing. 2019-11-21 13:58 information: skip_vm_shutdown is false. beginning vm shutdown procedure. 2019-11-21 13:58 action: Mint is running. shutting down. 2019-11-21 13:58 information: performing 20 30 second cycles waiting for Mint to shutdown cleanly 2019-11-21 13:58 information: cycle 1 of 20: waiting 30 seconds before checking if the vm has shutdown 2019-11-21 13:59 information: Mint is shut off. can_backup_vm set to y 2019-11-21 13:59 action: actually_copy_files is 1. 2019-11-21 13:59 action: can_backup_vm flag is y. starting backup of Mint configuration, nvram, and vdisk(s). 2019-11-21 13:59 information: backup of Mint xml configuration to /mnt/user/Backup/vm_sicherung/Mint/20191121_1358_Mint.xml complete. 2019-11-21 13:59 information: backup of Mint nvram to /mnt/user/Backup/vm_sicherung/Mint/20191121_1358_065a6081-e954-0913-370d-b6001262fb61_VARS-pure-efi.fd complete. 2019-11-21 13:59 failure: unable to perform snapshot for disk /mnt/user/VMs/Mint/vdisk1.img on Mint. skipping backup for Mint to prevent data loss. no cleanup will be performed for this vm. use_snapshots is 0. vm_state is shut off. vdisk_type is qcow2. 2019-11-21 13:59 information: finished attempt to backup Mint to /mnt/user/Backup/vm_sicherung. 2019-11-21 13:59 information: cleaning out logs over 12. 2019-11-21 13:59 information: removed '/mnt/user/Backup/vm_sicherung/logs/20190901_0630_unraid-vmbackup.log' 2019-11-21 13:59 information: cleaning out error logs over 12. 2019-11-21 13:59 information: did not find any error log files to remove. 2019-11-21 13:59 warning: errors found. creating error log file. 2019-11-21 13:59 Stop logging to error log file. Full script with all my settings: #!/bin/bash #backgroundOnly=true #arrayStarted=true #noParity=true # v1.1.5 - 2019/11/21 #### DISCLAIMER #### # Use at your own risk. This is a work-in-progress and provided as is. # I have tested this on my own server, as best as I am able, but YMMV. # -jtok # what is the scripts' official name. official_script_name="script" # set the name of the script to a variable so it can be used. me=$(basename "$0") # this script copies unRAID vm's vdisks and their configurations to a specified location. ################################################## script variables start ###################################################### # default 0 but set the master switch to 1 if you want to enable the script otherwise it will not run. enabled="1" # backup location to put vdisks. backup_location="/mnt/user/Backup/vm_sicherung/" # list of vms that will be backed up separated by a new line. vms_to_backup=" Mint " # list of specific vdisks to be skipped. use the full path. vdisks_to_skip=" " # list of specific vdisk extensions to be skipped. this replaces the old ignore_isos variable. vdisk_extensions_to_skip=" iso " # use snapshots to backup vms. # NOTE: vms that are backed up using snapshots will not be shutdown. if a vm is already shutdown the default backup method will be used. # WARNING: this will fail if the config path for the virtual disk is /mnt/user/. you must use /mnt/cache/ or /mnt/diskX/ for snapshots to work. use_snapshots="0" # default is 0. set this to 1 if you would like to kill a vm if it cant be shutdown cleanly. kill_vm_if_cant_shutdown="1" # default is 1. set this to 0 if you do not want a vm to be started if it was running before the backup started. Paused VMs will be left stopped. set_vm_to_original_state="1" # default is 0. set this to the number of days backups should be kept. 0 means indefinitely. number_of_days_to_keep_backups="30" # default is 0. set this to the number of backups that should be kept. 0 means infinitely. # WARNING: If VM has multiple vdisks, then they must end in sequential numbers in order to be correctly backed up (i.e. vdisk1.img, vdisk2.img, etc.). number_of_backups_to_keep="5" # default is 0. set this to 1 if you would like to compress backups. This can add a significant amount of time to the backup process. uses tar.gz for sparse file compatibility. # WARNING: do not turn on if you already have uncompressed backups. You will need to move or delete uncompressed backups before using. this will compress all config, nvram, and vdisk images in the backup directory into ONE tarball. compress_backups="1" # default is 1. set this to 0 if you would like to have backups without a timestamp. Timestamps are dropped only when number_of_backups_to_keep is equal to 1. timestamp_files="1" #### logging and notifications #### # default is 1. set to 0 to have log file deleted after the backup has completed. # NOTE: error logs are separate. settings for error logs can be found in the advanced variables. keep_log_file="1" # default is 1. number of successful log files to keep. 0 means infinitely. number_of_log_files_to_keep="12" # default is "logs". set to "" to put in root of backups folder. set to "logs/<subfolder>" to keep logs separate if running multiple versions of this script. log_file_subfolder="logs" # default is 1. set to 0 to prevent notification system from being used. Script failures that occur before logging can start, and before this variable is validated will still be sent. send_notifications="1" # default is 0. set to 1 to receive more detailed notifications. will not work with send_notifications disabled or only_send_error_notifications enabled. detailed_notifications="0" #### advanced variables #### # list of vms that will be backed up WITHOUT first shutting down separated by a new line. These must also be listed in vms_to_backup. # NOTE: vms backed up via snapshot will not be shutdown (see use_snapshots option). # WARNING: using this setting can result in an unusable backup. not recommended. vms_to_backup_running=" " # default is 0. set to 1 to have reconstruct write (a.k.a. turbo write) enabled during the backup and then disabled after the backup completes. # NOTE: may break auto functionality when it is implemented. do not use if reconstruct write is already enabled. backups may run faster with this enabled. enable_reconstruct_write="0" # default is 0. set this to 1 to compare files after copy and run rsync in the event of failure. could add significant amount of time depending on the size of vms. compare_files="0" # default is 1. set to 0 if you would like to skip backing up xml configuration files. backup_xml="1" # default is 1. set to 0 if you would like to skip backing up nvram files. backup_nvram="1" # default is 1. set to 0 if you would like to skip backing up vdisks. setting this to 0 will automatically disable compression. backup_vdisks="1" # default is 0. set this to 1 if you would like to start a vm after it has successfully been backed up. will override set_vm_to_original_state when set to 1. start_vm_after_backup="0" # default is 0. set this to 1 if you would like to start a vm after it has failed to have been backed up. will override set_vm_to_original_state when set to 1. start_vm_after_failure="0" # default is 0. set this to 1 to disable rsync delta syncs. disable_delta_sync="0" # default is 0. set this to 1 to use rsync instead of cp. # NOTE: rsync was significantly slower in my tests. rsync_only="0" # default is 1. set this to 0 if you would like to perform a dry-run backup. # WARNING: will not work unless rsync_only is set to 1. if this is set to 1 and rsync_only is set to 0, then backups will be partial and the script won't work correctly. actually_copy_files="1" # default is 20. set this to the number of times you would like to check if a clean shutdown of a vm has been successful. clean_shutdown_checks="20" # default is 30. set this to the number of seconds to wait in between checks to see if a clean shutdown has been successful. seconds_to_wait="30" # default is 1. set to 0 to have error log files deleted after the backup has completed. keep_error_log_file="1" # default is 10. number of error log files to keep. 0 means infinitely. number_of_error_log_files_to_keep="12" # default is 0. set to 1 to only send error notifications. only_send_error_notifications="0" ################################################## script variables end ######################################################### ###################################################### script start ############################################################# EDIT: removed the part bellow "script start" to not overload the page with code
-
unraid-autovmbackup: automate backup of virtual machines in unRAID - v0.4
First of all, big big thanks for the updated Script. I use it without issues for a long time now. I appreciate your work for the community. It saved my ass a couple times. But I have a quick questions. I never used the snapshot feature before # use snapshots to backup vms. # NOTE: vms that are backed up using snapshots will not be shutdown. if a vm is already shutdown the default backup method will be used. # WARNING: this will fail if the config path for the virtual disk is /mnt/user/. you must use /mnt/cache/ or /mnt/diskX/ for snapshots to work. use_snapshots="0" The default path created by Unraid for the vdisk in the xml is /mnt/user/VMSHARE. I have to change every VM's xml config by hand to make the snapshot backups to work, am I right??? So for my example <source file='/mnt/user/VMs/Win7_Outlook/WIN7_OUTLOOK.qcow2'/> I have to change it to <source file='/mnt/cache/VMs/Win7_Outlook/WIN7_OUTLOOK.qcow2'/> for a VM on a cache drive or to /disk2 for example if the vdisk is sitting on disk 2 in the array.
-
**VIDEO GUIDE** How To Use LSTOPO for Better VM Performance on Multi CPU and Threadripper Systems
@DayspringGaming You ever wondered what the Game mode in Ryzen Master does? Exactly this, otherwise for windows there is only 1 chip with all the cores. It doesn't know that there are 2 nodes underneath and it doesn't know the exact topology where the devices are directly attached to. The memory management is a different and also the related memory latency issue especially on the first gen TR4. NUMA vs. UMA if you search for that you will find a couple things to understand it better. For virtualisation especially if you want to fine tune it and optimise the performance of a VM you have to know about it, otherwise you will always have some bottlenecks and wont know where they come from.
-
**VIDEO GUIDE** How To Use LSTOPO for Better VM Performance on Multi CPU and Threadripper Systems
NO. Every Threadripper chip has at least 2 nodes (2 chips). Otherwise you wouldn't have quad channel memory or more PCI lanes available on x399 compared to the Ryzen chips. Go to your BIOS and change the memory configuration from UMA to NUMA. The setting is somewhere in the AMD CBS settings, at least for me. Auto, die, channel, socket and none is available and on default it's set to "auto" and reports only 1 node. Set it to "channel" (NUMA mode) and it will report the both dies separated. Depending on board manufacturer and BIOS versions it's different how the core pairings are presented to unraid. Could be possible by changing to NUMA mode the core pairings are shown differently than in UMA mode. After changing to NUMA mode you should see in LSTOPO which device is connected to which node directly.
-
[Support] Linuxserver.io - Nextcloud
This came up already multiple times. Please use the search function or read a couple pages back and you will find the answer.
-
[Support] Linuxserver.io - Nextcloud
Is there an extra option in the settings for indexing? If so, I can't find it or did you changed some config file by hand to stop Nextcloud indexing symlinks?
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
LUL How could I forgot about that. The custom lines stay in there and the strict and fallback are back on default. Now you mentioned it, I had that before 😂
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
@jbartlett I'am testing RC6 right now. Your mentioned issue that on first boot with the custom EPYC tweak the the CPU isn't reported as hyperthreaded I can't reproduce. As soon as I add the tweaks and boot it up, the CPU is detected as it should be. Used the following for testing on a default W10 template. Nothing special, 4GB of RAM, 30GB vdisk currently on the XFS array. Btw no corruptions so far. <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>EPYC</model> <topology sockets='1' cores='2' threads='2'/> <cache level='3' mode='emulate'/> <feature policy='require' name='topoext'/> <feature policy='disable' name='monitor'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='svm'/> <feature policy='disable' name='x2apic'/> </cpu> Did you checked the device manager if the CPU with the new tweaks is completly initialized? Remember, applying this tweaks the first time, Windows recognizes it as new CPU and has to install the drivers first. Maybe the combination of changing the CPU type AND the core count produces this issue. Did you searched for newly installed hardware in the device manager to trigger a rescan manually? Maybe that helps to detect the new cores. Just an idea.
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
Only VM I've tested it, is a SeaBIOS Windows 7 VM and I can copy it without problems and use it with a clean new VM template. Never had any issues with it before. Had to do it a couple times during my testing depending qcow2 corruptions on 6.8 RC1-5. I also have an old Windows7 SeaBIOS VM as template, also a qcow2 file which I use for testing and also never had that issue. Unfortunatly I don't have a i440fx Win10 vdisk anymore to test with. Maybe in an old backup if I'am lucky. I'll have a look tomorrow. I usually create the new VM first without booting it up, size of the vdisk doesn't matter and after that, I copy over the old vdisk only and overwrite the newly created one. Maybe try it this way. Will check this tomorrow. Unfortunately you have to reconfigure almost all custom settings you made after using the GUI every time. 😥 Would be nice to have a custom field where you could enter custom lines in the GUI for the let's say extra CPU settings, devices, interfaces etc. and keep them in the GUI to be able to re-add it with saving it via the editor automatically. In theory this should be possible, but I'am not the coding guy who could provide such a feature 🙄
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
Keep us up informed how it plays out for you and your usecase.
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
@jbartlett I didn't found any usecase, where I benefit from a higher memory bandwith. Not in gaming, not in general workflow and not in any cad software I use. Less latency in most situations should end in a quicker and lets say snappier work with any software at least in my case. I don't have any high bandwith related tasks. Don't know if it makes any differences in decoding/encoding video for example but in most cases the faster access to RAM is the more preferred scenario. Correct me if I'am wrong.
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
@jbartlett And don't forget to play around with the other 10ish options 🤣
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
The basic architecture didn't really changed between first and second gen TR4. They only added 2 more nodes and improved the interconnection latencies of the first gen. The rest is basically the same. Inside the cputune tag. <vcpu placement='static'>14</vcpu> <iothreads>1</iothreads> <cputune> <vcpupin vcpu='0' cpuset='9'/> <vcpupin vcpu='1' cpuset='25'/> <vcpupin vcpu='2' cpuset='10'/> <vcpupin vcpu='3' cpuset='26'/> <vcpupin vcpu='4' cpuset='11'/> <vcpupin vcpu='5' cpuset='27'/> <vcpupin vcpu='6' cpuset='12'/> <vcpupin vcpu='7' cpuset='28'/> <vcpupin vcpu='8' cpuset='13'/> <vcpupin vcpu='9' cpuset='29'/> <vcpupin vcpu='10' cpuset='14'/> <vcpupin vcpu='11' cpuset='30'/> <vcpupin vcpu='12' cpuset='15'/> <vcpupin vcpu='13' cpuset='31'/> <emulatorpin cpuset='8,24'/> <iothreadpin iothread='1' cpuset='8,24'/> </cputune> <numatune> <memory mode='strict' nodeset='1'/> </numatune> <resource>
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
Thats wrong. All the first gen TR4 have 2 nodes, each with it's own memory controller to get quad channel memory working on that platform. Starting with 4 cores per node (1900x), 6 cores (1920x) and the 8 core per node 1950x. 4 nodes where only available on first gen Epyc and where introduced on second gen TR4 like on yours.
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
It looks like it only works if you pass trough extra infos for the cache like I did. https://git.qemu.org/?p=qemu.git;a=commit;h=7210a02c58572b2686a3a8d610c6628f87864aed https://www.reddit.com/r/VFIO/wiki/known_issues#wiki_enabling_smt_on_amd_processors_with_qemu_3.1.2B
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
I know that feeling. There is always that one guy that has another little tweak 😂 Already seeing a 19% improvement in the memory score with this set. I'll also test hugepages. By using "interleave" you spread the RAM accross all memory controllers from all nodes, even the ones from the node you're maybe not using in the VM. On first gen TR4 this was a big issue, because it added a lot of RAM latency. Sure you get the higher memory bandwith by using "quad channel" but in most scenarios in my tests the lower latency was the preferred option. Not exactly sure how big of a difference it is on second gen TR4, but using "Preferred" or "Strict" was the better choice for me. Every program, game or benchmark is more or less affected by the lower bandwith by basically turning the RAM into a dual channel configuration. The bigger impact I saw by reducing the latency by using the "strict" setting. Maybe have a look into the "Cache & Memory Benchmark" which comes with AIDA64 to test this. This is a part of the extra CPU flags I use for a while now. <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>EPYC</model> <topology sockets='1' cores='7' threads='2'/> <cache level='3' mode='emulate'/> <feature policy='require' name='topoext'/> <feature policy='disable' name='monitor'/> <feature policy='require' name='hypervisor'/> <feature policy='disable' name='svm'/> <feature policy='disable' name='x2apic'/> </cpu> By forcing Windows into recognizing the CPU as an EPYC with these tweaks, it also recognizes the correct L1, L2 and L3 cache sizes which the node has to offer. Without it showed wrong cache sizes and wrong mapping numbers. Without these tweaks and the correct readings, starting up 3DMark for example always crashed or frooze the VM completly at the point, where it gathers the system infos. Not sure which other software might be affected, but this helped me in this scenario. Obvisiously the vcore is reported wrong, but the cache info is reported correctly with this tweak. 1 core is used for iothread and emulatorpin <emulatorpin cpuset='8,24'/> <iothreadpin iothread='1' cpuset='8,24'/> and the rest only specifically for this one VM. One 8 core die of the 2 from the 1950x is dedicated to this VM only and by adding up the numbers it exactly matches the specs of AMD. BUT this isn't the complete list of tweaks. There are way more you can play around with 😂😂😂 <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>EPYC-IBPB</model> <vendor>AMD</vendor> <topology sockets='1' cores='4' threads='2'/> <feature policy='require' name='tsc-deadline'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='arch-capabilities'/> <feature policy='require' name='cmp_legacy'/> <feature policy='require' name='perfctr_core'/> <feature policy='require' name='virt-ssbd'/> <feature policy='require' name='skip-l1dfl-vmentry'/> <feature policy='require' name='invtsc'/> </cpu> At some point I stopped, because I had no time back than to fiddle arround with it even further and the system was stable enough anyways. Main programs run fine and games performed great. Edit: Forgot to mention it reports "Hyperthreaded" for me in CoreInfo.
-
Performance Improvements in VMs by adjusting CPU pinning and assignment
@jbartlett Did you by any chance set a strict RAM allocation to the node which cores you're using? If not, you might have to test this again. By not setting this up, unraid will use RAM from all nodes. <numatune> <memory mode='strict' nodeset='1'/> </numatune> The following shows you from which node the VMs taking their RAM numastat qemu