July 29, 20196 yr In PVE 5.4 (or lartest 6), just need passthrough (IOMMU) the booting USB and HBA, that's all ( I haven't try onboard SATA controller). You can freely run Unraid under PVE or baremetel by boot selection, all work, just if Unraid under PVE, then no VM function available. Edited July 29, 20196 yr by Benson
July 29, 20196 yr thanks @Benson Dump question but, but I just want to make sure that I am reading this read HBA = host bus adapter and PVE = Proxmox. and is the setting up of the passthrough (IOMMU) easy to do ?
July 29, 20196 yr 17 minutes ago, chris_netsmart said: thanks @Benson Dump question but, but I just want to make sure that I am reading this read HBA = host bus adapter and PVE = Proxmox. and is the setting up of the passthrough (IOMMU) easy to do ? Yes and quite easy, just need setting in PVE side. In general, Unraid nothing need to change. Edited July 29, 20196 yr by Benson
July 29, 20196 yr 4 hours ago, chris_netsmart said: reading through the posts, I am liking the ideal of hosting my UNRAID within Proxmox. but I didn't see the following question answer, or if there are then I missed them regarding the spin down of HD's I like at @s.Oliver say, by passing through the SATA Controller, but does this work and allow the HD's to spin down. ? if I build the Promox environment and I have a Unraid Server environment, will I be able to transfer over my Unraid HD's as easy as moving over the HD's. Adding them to a VM UNraid guest and power up the guest and then set up Unraid as normal. ? I see posted that people mentioning " plopkexec " to fool Unraid, in making it think that it has a USB flash drive plug in, I see that uldise posted a link to a ISO. my question is. how to you install it ? I am still at the research stage so any advice would be appreciative. well, i answered all of your questions actually when you do passthrough your hardware to unRAID (HBA, or SATA controllers, or whatever) then unRAID has full control of it's features – and yes, it spins down all HDDs, which are connected to these passthrough'ed hardware. and this includes things like hot-swap (if supported by your hardware) or other things. if done in the right way, your 2nd question could be answered with yes! unRAID see's the drives as they would be on bare metal booted hardware (because you passthrough the hardware, which your drives are connected to then). your 3rd question references to my hint using plopkexec to boot natively into your unRAID installation on a USB stick (this is needed, because the hypervisor can't boot any VM from a usb stick; so plopkexec is a super small VM on ISO and it's then boots unRAID from the USB stick in stage 2). this method allows for native unRAID setup (with your USB key), without any changes/modifications, whatsoever. and the bonus is, that you never need to think about it in the future. all changes which need to be written to the usb stick (where unRAID is installed on) are written to the stick (and not to the ISO in the other guys setup). so no worries in the future – and you could boot unRAID on bare metal (without modifications). probably soon i'll tweak my setup again to go this virtualization route – because now i need the features of Proxmox, but i can't loose my unRAID-server – so i'll redo all this on my newer rig. hope this helps for your decision making. Edited July 29, 20196 yr by s.Oliver
July 29, 20196 yr 1 hour ago, s.Oliver said: because the hypervisor can't boot any VM from a usb stick You can't specify Proxmox to boot USB stick, but if USB stick is the only bootable device in guest, then it will boot. No plopkexec step also work. Edited July 29, 20196 yr by Benson
July 30, 20196 yr @Benson thanks for the post, I think I get the ideal. basically we are doing the following steps move over Unraid Hard drivers and USB Flash Drive into new case Install a blank hard drive for Proxmox Install Proxmox onto new blank hard drive download Unraid VM ISO file Create VM for UNraid add USB Flash Drive and Hard drivers from old Unraid make USB pen drive the boot drive ? - Need to look at the software power up VM once Unraid is up and running then add Hard drives to Unraid and start array. if this wrong then please point it out, and again thanks for your help.
July 30, 20196 yr 18 hours ago, Benson said: You can't specify Proxmox to boot USB stick, but if USB stick is the only bootable device in guest, then it will boot. No plopkexec step also work. you're using it this way? (no special prepared bootable image, pure unRAID usb-stick and it boots directly from that?) Edited July 30, 20196 yr by s.Oliver
July 30, 20196 yr 4 hours ago, chris_netsmart said: @Benson thanks for the post, I think I get the ideal. basically we are doing the following steps move over Unraid Hard drivers and USB Flash Drive into new case Install a blank hard drive for Proxmox Install Proxmox onto new blank hard drive download Unraid VM ISO file Create VM for UNraid add USB Flash Drive and Hard drivers from old Unraid make USB pen drive the boot drive ? - Need to look at the software power up VM once Unraid is up and running then add Hard drives to Unraid and start array. if this wrong then please point it out, and again thanks for your help. Step almost the same, but I think best should be prepare unraid USB stick first then unplug it before prepare Proxmox. Below are some detail 1. Unplug well prepare Unraid USB, best ( not must ) confirm it can boot in UEFI mode. 2. Install Proxmox to harddisk, make IOMMU enable. 3. Replug Unraid USB 4. Create VM 4a. Tweak ( Some step may not necessary ) - Click CD/DVD drive, remove it - Click Hard Disk, detach and remove it - Add phyical disk controller and physical Unraid USB stick 4b. Tweak - Edit boot order, only retain CD-ROM in boot device_1 That's all. Edited July 30, 20196 yr by Benson
July 30, 20196 yr 13 minutes ago, s.Oliver said: you're using it this way? (no special prepared bootable image, pure unRAID usb-stick and it boots directly from that?) Yes, even passthrough a NVMe as a Windows VM, freely boot under PVE or baremetel.
July 30, 20196 yr 29 minutes ago, Benson said: Yes, even passthrough a NVMe as a Windows VM, freely boot under PVE or baremetel. hmm… maybe i read the screenshots wrong... but i guess, because of no other specified bootable device (boot device 1 = cd-rom with empty 'tray', device 2 & 3 are set to nothing, or have non-bootable devices) PROXMOX is looking at it's usb devices (for this VM) and boots from it, if bootable?
July 30, 20196 yr 7 minutes ago, s.Oliver said: hmm… maybe i read the screenshots wrong... but i guess, because of no other specified bootable device (boot device 1 = cd-rom with empty 'tray', device 2 & 3 are set to nothing, or have non-bootable devices) PROXMOX is looking at it's usb devices (for this VM) and boots from it, if bootable? Yes, boot device 2,3 set to none. That's why I said 19 hours ago, Benson said: You can't specify Proxmox to boot USB stick, but if USB stick is the only bootable device in guest, then it will boot. No plopkexec step also work. Edited July 30, 20196 yr by Benson
July 30, 20196 yr 1 minute ago, Benson said: Yes, that's why I said alright man thx. i'll try this once i'm ready for my new PROXMOX installation.
November 23, 20196 yr so today was the day for PROXMOX installation and virtualizing unRAID again. most of the stuff works as expected, but some things aren't up to expectations: a) booting unRAID (directly from USB stick) brings me every time into the EFI shell, even though i set all the right boot options in the PROXMOX BIOS. so with every reboot i need to enter the BIOS, set the USB stick as boot option and then it boots unRAID. looks like PROXMOX doesn't save the changes into the EFI partitions or so. b) even though i enabled nested virtualization (PROXMOX) my macOS VM (under unRAID) is drastically slower than before. well, i had expected a little loss, but not that much. played around a bit with different options in the XML, but can't see any improvements. has someone any ideas, how to improve? well, i'll try to setup the macOS VM under PROXMOX, so no need for the nested virtualization then, but it can be a hassle
November 23, 20196 yr On 7/30/2019 at 8:31 PM, Benson said: You can't specify Proxmox to boot USB stick, but if USB stick is the only bootable device in guest, then it will boot. No plopkexec step also work. Pls set boot device 1 be CDROM, others none. I haven't run PVE now, mainly because I turn two Unraid in one, it also easy for management. Edited November 23, 20196 yr by Benson
November 24, 20196 yr 19 hours ago, Benson said: Pls set boot device 1 be CDROM, others none. I haven't run PVE now, mainly because I turn two Unraid in one, it also easy for management. thx., but i did that from beginning on. it never booted automatically from the USB stick – it entered the EFI shell. when intercepted while initializing BIOS and selecting the proper shown boot device (unRAID USB stick) it boots and everything is fine. not sure, why it doesn't automatically choose it. i'm running on Proxmox v6 and use UEFI/EFI boot because of passthrough'ing serveral PCI devices. Edited November 24, 20196 yr by s.Oliver
March 23, 20215 yr On 11/24/2019 at 2:50 PM, s.Oliver said: thx., but i did that from beginning on. it never booted automatically from the USB stick – it entered the EFI shell. when intercepted while initializing BIOS and selecting the proper shown boot device (unRAID USB stick) it boots and everything is fine. not sure, why it doesn't automatically choose it. i'm running on Proxmox v6 and use UEFI/EFI boot because of passthrough'ing serveral PCI devices. UPDATE: since one of the latest PROXMOX updates native booting from an USB device is directly support. No hacks, tricks needed anymore.
October 9, 20214 yr I'm looking at virtualising my unraid and opnsense under proxmox also. My aim is to pass through the onboard sata controller with my drives to unraid (it sits in its own IOMMU group). But then for cache, I would like to partition my NVMe drive for the proxmox partition, opnsense, unraid cache, other guests etc. Is this possible to do? I.e. give unraid a nvme partition to play with as the cache, instead of doing direct passthrough? Edit: answered with google I understand that things like SMART etc won't be available for this partition. Also how does fan control work for array drives? Is it possible to give unraid access to the fan headers, or is temperature management done in proxmox? Edit: some google-fu tells me that fan control will need to happen on proxmox side either via a fancontrol plugin or directly from BIOS. Edited October 11, 20214 yr by mishmash-
June 6, 20224 yr I have now virtualised my unraid in proxmox. I passed through the sas falcon controller (ibm something flashed to IT mode), GTX 1060 for Emby transcoding, a couple of usb devices including the unraid usb stick, enocean usb and usb zigbee controller. I created a BOND network interface and used this as an E1000 network card. The paravirtualised network card dropped packets (as detailed before). It is all working great! On the same proxmox - Opnsense (in is dedicated network port, out is network BOND), MQTT LXC container, Home Assistant Virtual machine, NodeRed LXC for testing, and Zorin VM. Server is a Asus Strix x570-F board, AMD 5900 with 64GB ram. Edited June 6, 20224 yr by spants
September 25, 20223 yr Timestamped - if it doesn't work, 9:08 This virtualized truenas video is the most closely representative of unraid drive usage that i could find for passing through individual drives without the entire controller. I'm not really looking to pass through the whole controller, as i want remaining drives to be accessible to proxmox. (my unit uses a sas expander, so i'm sorta stuck). TLDW: He sorta did stubbing like we would to pass a gpu through, but with the sda/sdb/sdc etc into proxmox's virtIO SCSI by id So, lsbk to list the drives with model/serial and attach them to the unraid VM's scsi controller. In my case, this would be the drives that i would potentially pass to the unraid VM's virtio scsi controller. ata-SanDisk_SDSSDH3_1T02_211135801590 ata-WDC_WD80EFBX-68AZZN0_VGKJ451G ata-WDC_WD80EFBX-68AZZN0_VGKJ5H3G ata-WDC_WD80EFBX-68AZZN0_VGKJD86G ata-WDC_WD80EFBX-68AZZN0_VYGAZANM ata-WDC_WD80EFBX-68AZZN0_VYGAZYMM For those with some experience virtualizing unraid, is this a viable solution? *added later* I have a new server coming tomorrow, i think i'll grab an unraid trial license and my cold spare, put some data on it, swap to proxmox and try passing that drive directly to unraid vm to see if it can 'remember' it. If it does work, i'll drop an update. Edited September 25, 20223 yr by jonfive Additional info
September 28, 20223 yr UPDATE: It DOES work. Though, i need to test it's long term viability since there's complaints from people on the forum about i/o errors using scsiblock (see below) For this test, I created another usb with a trial license on bare metal, made a few folders and stuck some isos on it. Installed proxmox, attached the usb device, set first boot to that usb device. Ran the below qm set line. Started the vm, disk showed up correctly, pre-placed in the right disk# slot without showing "missing". Opened the 6.11 file manager, disk 1, files were there. qm set vm# -scsi# /dev/disk/by-id/YourDiskInformation,scsiblock=1 Without scsiblock, it just shows up as a qemuharddisk, so it's necessary for the time being. The particular part of the proxmox docs that mentions the scsiblock potential issue: Quote scsiblock=<boolean> (default = 0) whether to use scsi-block for full passthrough of host block device ! can lead to I/O errors in combination with low memory or high memory fragmentation on host If anyone has any further information or knowledge on how high/low memory fragmentation occurs in unraid, that'd be handy
October 10, 20223 yr Has anyone managed to reproduce @grumpybutfun unRAID vhd file? The original link is dead (given it's 8 years old, it's no real surprise). I'm trying to do a similar thing, but currently, unRAID is saying "Error accessing your physical USB Flash boot device", even though the virtual USB drive is writable and boots to the unRAID GUI. Steps performed: 1. Create VHD and SCP it across to Proxmox. 1a. VHD created as required by unraid instructions. 2. Run the following script: VMNUM=1112 VMSCSINUM=0 VMDISKLOC=/mnt/prox/images DISK_STOR=Proxmox USBARG="args: -drive 'file=$VMDISKLOC/$VMNUM/vm-$VMNUM-disk-$VMSCSINUM.raw,if=none,id=drive-usb0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' -device 'usb-storage,drive=drive-usb0,bootindex=1,removable=on'" cd $VMDISKLOC/$VMNUM/ qemu-img convert -O raw UNRAID.vhd unraid.img && \ rm UNRAID.vhd && \ qm importdisk $VMNUM unraid.img $DISK_STOR && \ qm set $VMNUM --scsihw virtio-scsi-pci --scsi$VMSCSINUM $DISK_STOR:$VMNUM/vm-$VMNUM-disk-$VMSCSINUM.raw && \ sed -i -e "1i${USBARG} ${1}" /etc/pve/qemu-server/$VMNUM.conf I've tried both: args: -drive 'file=/mnt/prox/images/1112/vm-1112-disk-0.raw,if=none,id=drive-usb0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' -device 'usb-storage,drive=drive-usb0,bootindex=1,removable=on' and args: -drive id=stick,if=none,format=raw,file=/mnt/prox/images/1112/unraid.img -device usb-storage,drive=stick,bootindex=1,removable=on And whilst the VM boots into UNRAID with both, it throws up a warning that it cannot write to the "disk". I have confirmed that the "disk" is writable. Is img/raw the right way to go here?
October 11, 20223 yr On 10/10/2022 at 7:12 PM, modem7 said: Has anyone managed to reproduce @grumpybutfun unRAID vhd file? The original link is dead (given it's 8 years old, it's no real surprise). I'm trying to do a similar thing, but currently, unRAID is saying "Error accessing your physical USB Flash boot device", even though the virtual USB drive is writable and boots to the unRAID GUI. Steps performed: 1. Create VHD and SCP it across to Proxmox. 1a. VHD created as required by unraid instructions. 2. Run the following script: VMNUM=1112 VMSCSINUM=0 VMDISKLOC=/mnt/prox/images DISK_STOR=Proxmox USBARG="args: -drive 'file=$VMDISKLOC/$VMNUM/vm-$VMNUM-disk-$VMSCSINUM.raw,if=none,id=drive-usb0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' -device 'usb-storage,drive=drive-usb0,bootindex=1,removable=on'" cd $VMDISKLOC/$VMNUM/ qemu-img convert -O raw UNRAID.vhd unraid.img && \ rm UNRAID.vhd && \ qm importdisk $VMNUM unraid.img $DISK_STOR && \ qm set $VMNUM --scsihw virtio-scsi-pci --scsi$VMSCSINUM $DISK_STOR:$VMNUM/vm-$VMNUM-disk-$VMSCSINUM.raw && \ sed -i -e "1i${USBARG} ${1}" /etc/pve/qemu-server/$VMNUM.conf I've tried both: args: -drive 'file=/mnt/prox/images/1112/vm-1112-disk-0.raw,if=none,id=drive-usb0,format=raw,cache=none,aio=io_uring,detect-zeroes=on' -device 'usb-storage,drive=drive-usb0,bootindex=1,removable=on' and args: -drive id=stick,if=none,format=raw,file=/mnt/prox/images/1112/unraid.img -device usb-storage,drive=stick,bootindex=1,removable=on And whilst the VM boots into UNRAID with both, it throws up a warning that it cannot write to the "disk". I have confirmed that the "disk" is writable. Is img/raw the right way to go here? just quickly jumped here to give a hint: if your goal is, to boot unraid as a VM inside of Proxmox see my last post (just a few upwards from here). there's no need anymore, to fake/virtualize the usb-stick as vm file or such things. simple create the VM without (or delete later on) a disk associated. passthrough the original usb-stick ("UNRAID" labeled) to that VM and choose this one as boot order 1 (see Options) to boot from. i've set the machine type to Q35 with UEFI for passthrough of hardware hba controllers and other stuff. works perfectly.
October 11, 20223 yr 5 minutes ago, s.Oliver said: just quickly jumped here to give a hint: if your goal is, to boot unraid as a VM inside of Proxmox see my last post (just a few upwards from here). there's no need anymore, to fake/virtualize the usb-stick as vm file or such things. simple create the VM without (or delete later on) a disk associated. passthrough the original usb-stick ("UNRAID" labeled) to that VM and choose this one as boot order 1 (see Options) to boot from. i've set the machine type to Q35 with UEFI for passthrough of hardware hba controllers and other stuff. works perfectly. Heya, Aye, I've managed to successfully virtualise (with qemu guest agent) with no issues, I was merely looking for a way to forego the USB stick as it's a single point of failure. Would be great if there was an official vhd/iso for those who just wanted to virtualise the NAS aspect, and be able to backup/snapshot the configs as part of their standard work flows. Cheers though!
October 11, 20223 yr 6 hours ago, modem7 said: Heya, Aye, I've managed to successfully virtualise (with qemu guest agent) with no issues, I was merely looking for a way to forego the USB stick as it's a single point of failure. Would be great if there was an official vhd/iso for those who just wanted to virtualise the NAS aspect, and be able to backup/snapshot the configs as part of their standard work flows. Cheers though! well, unRAID uses the USB-stick as a way of validating your license (because it is bound to the hardware UUID of that stick). don't know how to circumvent that, even though i also don't like the single point of failure associated to that way of doing it. once (quite awhile back) i had it setup with in a different way (with a synchronisation between virtual image and the USB-stick, but you had to have it mounted while booting unRAID and in the long run it was prone to failures when you forgot to sync back to the stick, before rebooting, etc.). Edited October 11, 20223 yr by s.Oliver
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.