January 3, 201412 yr So I'm fairly new to linux, however I really want to help advance some of the unraid tools; but not enough to risk the data I have on my 40TB unraid server. So with that, I would like to have a dev unraid server, and I would like to do it on the cheap since this will be used just to dick around; not to do anything very computationally taxing, or holding a large number of drives... I suspect some of the cheapest options available would be some older server from eBay, however if someone here could point me in the right direction, that would be great. I would like to be able to do some virtualization. I understand some CPUs back from the Core2Duo line had Vt-d? I wouldn't be looking to run more than a few drives. I can go rack mounted or not, doesn't matter too much. If rack mounted, IPMI would be nice (as I don't have a small monitor and keyboard to lug around to attach to a local machine), but far from necessary. I've underwent a number of moves recently, so I don't have that much spare hardware to mess with (may have some spare older/smaller hard drives if I dig around for them). Anyway, I welcome any suggestions.
January 3, 201412 yr I'm playing around with this setup: MB: Intel DQ45CB bios 0129 | CPU: Intel Core 2 Quad Q9300 | RAM: 8 GB DDR2 | Controller: SYBA SY-PEX40039 | Disks: WD1600BEVS or WD1001FALS The mobo I got off of eBay for around $38.00 used. It supports VT-d (IOMMU). The controller was new from Newegg, maybe $25. The rest of the stuff I had from other builds. The processor supports VT-d. I haven't had any real luck with PCI-passthrough, but USB passthrough worked fine using KVM/openSUSE. I'm more hopeful with Xen. I quickly tried ESXi 4.1 and PCI-passthrough is supported with the PCIe controller card. DDR2 might be getting more expensive now compared to DDR3. So far I'm happy!
January 3, 201412 yr Author I'm playing around with this setup: MB: Intel DQ45CB bios 0129 | CPU: Intel Core 2 Quad Q9300 | RAM: 8 GB DDR2 | Controller: SYBA SY-PEX40039 | Disks: WD1600BEVS or WD1001FALS The mobo I got off of eBay for around $38.00 used. It supports VT-d (IOMMU). The controller was new from Newegg, maybe $25. The rest of the stuff I had from other builds. The processor supports VT-d. I haven't had any real luck with PCI-passthrough, but USB passthrough worked fine using KVM/openSUSE. I'm more hopeful with Xen. I quickly tried ESXi 4.1 and PCI-passthrough is supported with the PCIe controller card. DDR2 might be getting more expensive now compared to DDR3. So far I'm happy! Looks like a sweet setup! What kind of case did you end up getting?
January 3, 201412 yr Author After looking at the potential cost, I think buying an old dell/hp server of some form (preferably a smaller tower) might be the way to go. Now, just to figure out what kind of tower will play nice with unraid, kvm, and so on...
January 3, 201412 yr You're probably right, if you look at going from scratch, then a complete unit is a better choice. It seems to me anything that can run VMware will work in the other venues; and there should be a "Hardware Compatibility List (HCL) for older server hardware. But, you might have to rework some components, fans, hard drives or controllers to your liking. Well, the ESXi forums here should be loaded with possible candidates. You have to be careful about pci-passthrough, though. Good luck with the quest!!
January 3, 201412 yr Author You're probably right, if you look at going from scratch, then a complete unit is a better choice. It seems to me anything that can run VMware will work in the other venues; and there should be a "Hardware Compatibility List (HCL) for older server hardware. But, you might have to rework some components, fans, hard drives or controllers to your liking. Well, the ESXi forums here should be loaded with possible candidates. You have to be careful about pci-passthrough, though. Good luck with the quest!! I checked out the HCL, just found some pretty expensive servers, regardless of what combination of features/support I tried to select. Just posted on the serve the home forums, and that same motherboard that you are using got recommended to me, perhaps I should take the hint. I would be curious what the problem with PCI passthrough is... that's the only reluctance I have to following up with a similar build. Ogi
January 3, 201412 yr What I've read is "stuff" behind the PCI bridge can't be passed through but PCIe oriented devices can be passed. I build up an openSUSE KVM system as described in this thread: http://lime-technology.com/forum/index.php?topic=30715.0. Now virtualization was all very new to me, so while I was waiting for the controller card to arrive I did find USB passthrough worked pretty well. After the controller arrived, I reconfigured my guest machine to be connected to the new SATA controller card. However, I still got the same negative results to get the VM to boot with the controller attached. So I moved on for the time being. This thread: http://lime-technology.com/forum/index.php?topic=30932.0 seemed to indicate there may be a bug in KVM getting it to work. With the Xen build I'm trying now, http://lime-technology.com/forum/index.php?topic=30980.0 I haven't quite got a guest, domU, running but I understand more about how one might get PCIe passed to the guest and I feel more confident it will work. But, the way things are going over at the Virtualization forum, there may not need to be any need for PCI passthrough. Again, my brief experience with ESXi and configuring PCI passthrough gave me hope the board can do it. I wish I could report "mission accomplished" but for now I'll have to keep plodding along and I hope to have more news in a week or so.
January 3, 201412 yr Author What I've read is "stuff" behind the PCI bridge can't be passed through but PCIe oriented devices can be passed. I build up an openSUSE KVM system as described in this thread: http://lime-technology.com/forum/index.php?topic=30715.0. Now virtualization was all very new to me, so while I was waiting for the controller card to arrive I did find USB passthrough worked pretty well. After the controller arrived, I reconfigured my guest machine to be connected to the new SATA controller card. However, I still got the same negative results to get the VM to boot with the controller attached. So I moved on for the time being. This thread: http://lime-technology.com/forum/index.php?topic=30932.0 seemed to indicate there may be a bug in KVM getting it to work. With the Xen build I'm trying now, http://lime-technology.com/forum/index.php?topic=30980.0 I haven't quite got a guest, domU, running but I understand more about how one might get PCIe passed to the guest and I feel more confident it will work. But, the way things are going over at the Virtualization forum, there may not need to be any need for PCI passthrough. Again, my brief experience with ESXi and configuring PCI passthrough gave me hope the board can do it. I wish I could report "mission accomplished" but for now I'll have to keep plodding along and I hope to have more news in a week or so. Thanks for the detailed reply. I also found a standard bit of hardware that seems to support everything, and that's the HP DC7900. They have the right intel chipset (Intel Q45), have the right processor and seem to be selling for sub $100 on eBay. That said, I'm not sure how many hard drives they support (external optical drive? LOL), but that may be the way to go.
January 3, 201412 yr That looks like a pretty sweet deal! It looks to have two pcie slots. I'll be checking that puppy out!
January 4, 201412 yr Author That looks like a pretty sweet deal! It looks to have two pcie slots. I'll be checking that puppy out! Not that this would be an expensive investment, but I haven't found any evidence of people being able to have PCI passthrough (however all signs indicate that it SHOULD work). Ogi EDIT: Upon closer look, it only houses 1 internal 3.5in drive, and while I could replace the CD drive most likely with another 3.5in drive, I'm not sure 2 drives would be adequate to do any kind of virtual machine testing (I imagine I would need a minimum of 3 drives, one for the VM manager, 1 unRAID parity drive, and 1 unRAID data drive).
January 4, 201412 yr Oops, the mid tower case looked bigger; good catch! Sent from my Nexus 7 using Tapatalk
January 5, 201412 yr Author So after spending way too much looking at potential candidates, it looks like the best bet is the Dell OptiPlex 780 Mini-Tower. The Mini tower has 2x internal 3.5in bays, and 2x external 5.25in bays (which can have a 2 5.25 -> 3 2.5 adapter installed making room for a total of 5 drives). That platform has an Intel Q45 platform, it came with a quad-core option, but what really set it apart from the rest of the candidates was that it supported DDR3 memory. I think I'm going to just sit tight on eBay and wait for a deal too good to pass up and jump on it then. Ogi
January 5, 201412 yr I have an Optiplex 620 MiniTower, and it looks like the same case that was used in the '780. From what I can tell, getting a 2X3 3.5" (is that what you meant?) adapter to fit in the 5.25 bay is possible. Dell uses an integrated standoff (IS) mounting screw and a spring-loaded retaining mech. to hold the drives. They thoughtfully include 3 additional of those IS screws on the back of the filler plate that covers the empty bay. It seems to me you should also be able to mount a SSD with adapter in the wasted floppy drive bay. I think there are adapters out there, but I couldn't easily find "THE ONE" in a quick Google search. However, dealing with the cost of those adapters.....well, you know what I'm getting at. Looking at the images of the mobo, it has two PCIe slots 1 @ 1x and 1 @ 16x (probably for video) and that makes me wonder if the larger 16x slot can be used for something other than video. Maybe that's not a problem with the Q45 chipset? Which brings up the point of doing the research to see if there's been win/win cases with the virtualization crowd. On the other hand there might be some AMD systems that would fit the bill... Ah, the thrill of the hunt!! --Cheers!
January 5, 201412 yr Author I have an Optiplex 620 MiniTower, and it looks like the same case that was used in the '780. From what I can tell, getting a 2X3 3.5" (is that what you meant?) adapter to fit in the 5.25 bay is possible. Dell uses an integrated standoff (IS) mounting screw and a spring-loaded retaining mech. to hold the drives. They thoughtfully include 3 additional of those IS screws on the back of the filler plate that covers the empty bay. It seems to me you should also be able to mount a SSD with adapter in the wasted floppy drive bay. I think there are adapters out there, but I couldn't easily find "THE ONE" in a quick Google search. However, dealing with the cost of those adapters.....well, you know what I'm getting at. Looking at the images of the mobo, it has two PCIe slots 1 @ 1x and 1 @ 16x (probably for video) and that makes me wonder if the larger 16x slot can be used for something other than video. Maybe that's not a problem with the Q45 chipset? Which brings up the point of doing the research to see if there's been win/win cases with the virtualization crowd. On the other hand there might be some AMD systems that would fit the bill... Ah, the thrill of the hunt!! --Cheers! This search has been frustrating, as there is no clear answer. If you search for the quadCore optiplex 780 platforms on ebay (and select see only "completed listings") you can see they can be had for under $100, so even with an adapter or two, I still think from a cost perspective, that would be the way to go (as a quad core CPU, motherboard power supply and case would run me more than that on its own). I'm moving next week, so I can be patient and wait for another inexpensive box to pop up on eBay... Awesome tip for the extra spring-loaded screws. Ogi
January 7, 201412 yr Ogi-- Just a note to say I got PCI passthrough going on my Q45 setup. This build is centered around openSUSE and KVM. The thread is here: Virtualizing unRAID in KVM on openSUSE 13.1 (64Bit) The fix for me is covered in the thread, but if references this article : PCI passthrough regression caused by the fix for CVE-2011-1898 The way I read it (short version), with the older Q45 chipset, interrupt remapping is not supported and thus the failure to launch message. Luckily there's a patch. I still haven't fully worked all the bugs to get a working Xen guest, but that's next on the list. Cheers!
January 7, 201412 yr Author Ogi-- Just a note to say I got PCI passthrough going on my Q45 setup. This build is centered around openSUSE and KVM. The thread is here: Virtualizing unRAID in KVM on openSUSE 13.1 (64Bit) The fix for me is covered in the thread, but if references this article : PCI passthrough regression caused by the fix for CVE-2011-1898 The way I read it (short version), with the older Q45 chipset, interrupt remapping is not supported and thus the failure to launch message. Luckily there's a patch. I still haven't fully worked all the bugs to get a working Xen guest, but that's next on the list. Cheers! Thanks for the follow up, I did see you made progress in the other thread. I'm still waiting for a cheap optiplex on eBay
Archived
This topic is now archived and is closed to further replies.