Help designing/picking a setup for unraid, need to upgrade in a bad way :)

live4soccer7 · December 1, 2015

I've decided to go with a standalone pfsense box vs virtualization because I've been reading that there are some issues with NIC passthroughs and from what I've read unraid won't natively support it in v6.x. I still want to get the machine I have (dell dimension 4700) running with pfsense.

I have worked around the one Network interface port that I have so I can atleast log in to the webconfigurator. I simply setup a vlan on pfsense and assigned the wan to that and then assigned the LAN to the one physical port I have and then assigned it an IP etc... Voila, I'm in pfsense webconfig for the first time now.

I purchased an Intel dual port Pro 1000 NIC, but the x16 PCI-e does not accept anything but a video card for some dumb reason.

You can see the manual here:

http://downloads.dell.com/Manuals/all-products/esuprt_desktop/esuprt_dimension_desktops/dimension-4700_service%20manual_en-us.pdf

There are TWO normal PCI slots (I can't recal the correct nomenclature right now) and then there is a 1 x PCI-e slot. I'm wondering if the x 1 pci-e slot can accept a NIC or even the if the normal PCI slots can. I just hate to buy another card simply for the sake of using this machine and having it not work because this thing is antiquated. Not to mention that by the time I order it, intsall it, etc... it will be a week from the time it is ordered.

I know this is kind of straying from the unraid build, but you guys have been such great help and the project has grown a TON since inception on the first page of this thread.

live4soccer7 · December 1, 2015

Hopefully someone here may know this answer. It looks like a PCI NIC will work for the temporary setup I have going with pfsense on the dell dimension 4700.

The PCI slot I have have is notched for a 5v 32 bit card. I've been reading that a PCI-X card can/will actually fit in a PCI slot and will hang over the back edge. I ask because I can find some used intel/dell cards (PCI-X) on Ebay for pretty cheap and can't find a genuine intel PCI card that is fairly priced. I had one purchased, but the guy refunded me because it wasn't a genuine card.

I realize it likely isn't the ideal situation if this option will work, but I don't really care to much if it will work reliably while I setup the machine and put it in to use for a couple weeks and see how I like it.

I'd imagine that since the PCI slot is setup for 5v that the PCI-x would have to work on 5v. I'm still looking in to this.

Alternately, would a 64 bit pci card work in 32 bit, assuming the same voltage?

Anyone put some insight on this?

Found one of these laying around the house: http://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=6&Level=5&Conn=4&ProdID=18

Was in an even older machine (HP). Popped it in the dell and it recognizes it and was able to asign it to WAN. I'm hoping it won't bottleneck at the WAN simply based on my connection speed anyways. Hopefully I can get some internet going through the pfsense box this evening. Until then i have a lot of actual work to get done.

live4soccer7 · December 3, 2015

I've been trying to decide whether to flash the onboard LSI2308 controller to IT or IR and to the latest software. From what I can read in the below thread, it seems that the current IR Firmware allows drives to work properly without having to flash to IT, however I'm looking down the road and know that I will need another SAS controller in order to fill out this machine (HDD Slots) and I believe that most of the controllers you end up having to flash to IT mode, so would it be better to have both the onboard LSI2308 flashed to IT mode as well since the added PCI-e SAS controller will also have to be flashed to IT (most likely from what I've been reading).

I made a post in the thread that discusses the X10SL7-F board and the onboard SAS controller. You can see my post at the end of the page I have linked here: https://lime-technology.com/forum/index.php?topic=27800.135

live4soccer7 · December 3, 2015

I just flashed it. Ended up finding my answers. Guess I was just a bit tired when reading through all the threads. If anyone cares, here are my quick and dirty notes on the LSI2308 FW flash/upgrade and change to IT mode.

ftp://ftp.supermicro.com/Driver/SAS/LSI/2308/Firmware/ - place to get IT firmware

Unplug all media/usb/hdds before attempting

Make sure to get LSI controller address:

put on blank USB drive (contents of UEFI folder)

push F11 on boot

boot from built in UEFI

change to USB fs0 - fs0:

check contents - dir

invoke script: SMC2308T.NSH

Input last nine digits of address when asked: enter last 8 numbers (does not have to be caps and does not include : colon)

remove USB once it says done

reboot and check in LSI controller utility that update applied and address correct - disabled boot option in lsi config util. based on this thread: https://lime-technology.com/forum/index.php?topic=12767.0

power down and plug all drives and unraid USB back in.

Boot and check everything in unraid

It recognized the disk on the controller and it's clearing it now. All in all, it seems good so far.

live4soccer7 · December 12, 2015

Just a little update. The onboard controller flashed to IT with the latest FW has been working flawlessly.

I ordered up the UPS that was discussed earier in the thread and also the Radeon graphics card. Picked up some extra ram as I was utilizing quite a bit of the initial 8GB I had through the windows VM. The dockers and unraid hardly use any.

Are you guys running the pre-clear script still or is there something new built in to Unraid 6.x that does this now? Picked up a new hdd for the array that will be here next week or the week after and want to figure out how I'm going to "check" it out before adding it to the array.

BobPhoenix · December 12, 2015

There is a plugin for preclear but you still have to use the JoeL script or the modified faster bjp999 script with the plugin.

live4soccer7 · December 15, 2015

The UPS and video card showed up today!

Anyone have suggestions on what UPS settings I should use within unraid. I am running unraid baremetal. This is the first UPS that I have owned and would really like to get it set up properly so it can do its job well. Running unraid 6.1.3

UPS: http://www.newegg.com/Product/Product.aspx?Item=N82E16842102134

garycase · December 15, 2015

With the built-in support in v6, it's very easy to configure.

Just go to Settings - UPS Settings

You can set it to automatically shut down when the battery is down to a specific %, or there's a limited amount of runtime left; or after the UPS has been on battery for a specified time. The latter is what I use -- and I think most folks do -- as it will preserve the charge in case you have additional outages, and in my experience if the power's out for more than a few minutes, it's likely to be out for a moderately long time. I use 420 seconds (7 minutes) ... but 300 or 600 seconds are also common choices.

Not really anything else to configure, unless you want the UPS to be turned off after the shutdown. This lets you configure your server to automatically come back on by setting the BIOS power to "Always On" ... when power is restored the UPS will come on, and the application of power to the PC will cause it to boot. I don't do this -- not only because when the server's shut down due to an outage I'd prefer to turn it back on myself, but also because I have other devices on the UPS I do NOT want to shut down automatically [cable modem, router, and a few switches].

live4soccer7 · December 15, 2015

Excuse my ignorance on the topic.

I'd imagine that I'd want to start the daemon for one.

UPS cable: USB

Custom UPS cable: leave blank

UPS Type: USB

Device: What do I use for this or just leave blank?

Battery level to initiate shutdown (%): Do I change this to 0 so time on battery stays valid or does it work in conjunction with that setting?

Runtime left to initiate shutdown (minutes): Do I change this to 0 so time on battery stays valid or does it work in conjunction with that setting?

Time on battery before shutdown (seconds): 420

trurl · December 15, 2015

Excuse my ignorance on the topic.

I'd imagine that I'd want to start the daemon for one.

UPS cable: USB

Custom UPS cable: leave blank

UPS Type: USB

Device: What do I use for this or just leave blank?

Battery level to initiate shutdown (%): Do I change this to 0 so time on battery stays valid or does it work in conjunction with that setting?

Runtime left to initiate shutdown (minutes): Do I change this to 0 so time on battery stays valid or does it work in conjunction with that setting?

Time on battery before shutdown (seconds): 420

Turn on Help

live4soccer7 · December 15, 2015

I honestly had no idea that was even there!! Holy smokes that is awesome!

JonathanM · December 15, 2015

Not really anything else to configure, unless you want the UPS to be turned off after the shutdown. This lets you configure your server to automatically come back on by setting the BIOS power to "Always On" ... when power is restored the UPS will come on, and the application of power to the PC will cause it to boot. I don't do this -- not only because when the server's shut down due to an outage I'd prefer to turn it back on myself, but also because I have other devices on the UPS I do NOT want to shut down automatically [cable modem, router, and a few switches].

If you choose to try this, be sure that particular model of UPS actually works correctly with the turn off option. I remember reading somewhere that cyberpower units don't correctly implement the shutdown option in apcupsd, instead they quit charging the battery until the unit is hard cycled.

One very good way of testing your battery backup system is to plug the server into one surge protector, and the battery backup into another. Load the UPS with a similar wattage draw as your server under full load with table lamps, and turn off the surge protector feeding the battery backup. The lamps should stay lit, and your server should respond by indicating a power outage, and execute a proper shutdown when the conditions you set are reached, and complete the shutdown before the lamps drain the battery.

garycase · December 16, 2015

... One very good way of testing your battery backup system is to plug the server into one surge protector, and the battery backup into another.

Testing the UPS unit by simulating a power outage while it's not actually connected to the UPS is a good idea. However, you should NOT plug a UPS unit into a surge protector ... it should be plugged directly into a power outlet. Cascading surge protectors it not electrically a good idea, and can actually damage the circuitry in the UPS.

JonathanM · December 16, 2015

... One very good way of testing your battery backup system is to plug the server into one surge protector, and the battery backup into another.

Testing the UPS unit by simulating a power outage while it's not actually connected to the UPS is a good idea. However, you should NOT plug a UPS unit into a surge protector ... it should be plugged directly into a power outlet. Cascading surge protectors it not electrically a good idea, and can actually damage the circuitry in the UPS.

Pulling the plug so the ground is disconnected is a worse idea. You need to simulate a power failure, which means all the grounds and neutrals are still connected, while killing power to the hot line. A power strip is the easiest way to do that, an outlet controlled by a wall switch is second easiest, flipping the breaker is also an option.

I agree that cascading surge protectors isn't a good option, but it's better than cold pulling the plug, which creates a power island that is only grounded through the USB cable into the server.

live4soccer7 · December 16, 2015

The UPS is up and running. Recognized in Unraid as well. That was easy.

I'm trying to pass through the video card that I just received to a windows7 install and I get the following error:

internal error: early end of file from monitor: possible problem:

2015-12-16T06:17:42.502093Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

2015-12-16T06:17:42.502112Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 1

2015-12-16T06:17:42.502119Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed

2015-12-16T06:17:42.502125Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

Any insight on this?

As soon as I try to change video or audio in the settings of the VM, I get an error. When I received this error, I had both audio and video selected on the new card that was just acquired.

http://www.newegg.com/Product/Product.aspx?Item=N82E16814102933&cm_re=SAPPHIRE_Radeon_HD_6450-_-14-102-933-_-Product

garycase · December 16, 2015

... One very good way of testing your battery backup system is to plug the server into one surge protector, and the battery backup into another.

Testing the UPS unit by simulating a power outage while it's not actually connected to the UPS is a good idea. However, you should NOT plug a UPS unit into a surge protector ... it should be plugged directly into a power outlet. Cascading surge protectors it not electrically a good idea, and can actually damage the circuitry in the UPS.

Pulling the plug so the ground is disconnected is a worse idea. You need to simulate a power failure, which means all the grounds and neutrals are still connected, while killing power to the hot line. A power strip is the easiest way to do that, an outlet controlled by a wall switch is second easiest, flipping the breaker is also an option.

I agree that cascading surge protectors isn't a good option, but it's better than cold pulling the plug, which creates a power island that is only grounded through the USB cable into the server.

Best choice for this kind of a test is an el-cheapo ($2.98 is typical) Home Depot or Lowes power strip -- no surge protector involved Or perhaps a holiday extension cord or timer -- both have switches on them which can be used to turn the power on/off [Probably wouldn't have though of that if I hadn't just spent the weekend lighting up the house ]

JonathanM · December 16, 2015

2015-12-16T06:17:42.502093Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

Any insight on this?

Go to settings, vm manager, and set PCIe ACS Override: to yes, and reboot the server.

You may want to research that setting a little bit to become aware of a remotely possible gotcha, but many people are using the setting with no ill effects noted. I have never seen any problems reported from using it, just that there could be a corner case for the remote chance of possible data loss. Since you are using proven hardware, I think the odds are probably 99.99999% in your favor.

live4soccer7 · December 16, 2015

Is there a better way to do it or is that the only option I'm looking at to get the Video card passed through?

When you say data loss, are you referring to data loss on the VM that is using the video card that is being passed through or is it data loss in the unraid array?

JonathanM · December 16, 2015

Is there a better way to do it or is that the only option I'm looking at to get the Video card passed through?

When you say data loss, are you referring to data loss on the VM that is using the video card that is being passed through or is it data loss in the unraid array?

This has been discussed elsewhere on the forum and internet in general. Google pcie acs override alex williamson.

Bottom line, it opens the door to allow a misbehaving device or driver to corrupt data at the motherboard device level. I don't remember seeing any actual issues, only discussion of the possibility of it happening.

No other options that I am aware of.

live4soccer7 · December 16, 2015

Thanks, I've been reading around and I'll let ya'll know what I end up doing. Not making any hasty decisions as the data in the arrary is the most important part of the whole server and the actual purpose of it. The rest of the added features and functionality are just icing on the cake and can be had without if they will compromise the reliability of the system.

JonathanM · December 17, 2015

Thanks, I've been reading around and I'll let ya'll know what I end up doing. Not making any hasty decisions as the data in the arrary is the most important part of the whole server and the actual purpose of it. The rest of the added features and functionality are just icing on the cake and can be had without if they will compromise the reliability of the system.

I'm in the same boat as you, I don't like the feeling of opening myself up for an unknown issue, but I decided to go ahead with the override, since I have yet to see any actual corruption issues posted. I just wanted you to be aware of the possible issues, instead of telling you to make the change without an explanation. Since I have it enabled on one of my servers, I am extra diligent with following up on any discussions about it. The first time I see a confirmed report of an actual issue, I'll be turning it off. Until then, I'm ok with the very small risk.

live4soccer7 · December 17, 2015

I appreciate the thoroughness and honesty. I am usually pretty thorough when doing projects, to a fault. Haha. I'll probably end up enabling it. From what I've been reading the issue is typically when the bus is shared with a data storage type of bus. I'm running the x10sl7-f board, so I feel this is most likely. The gpu is using the PCI-e x 16 slot.

live4soccer7 · December 17, 2015

Jonathanm: I read a post with you and jonp talking about the override and saw you have the same board as myself. Are you using the onboard LSI controller? If so have you updated the firmware and are you using IR or IT firmware? I flashed to IT and am at FW 20.x.

JonathanM · December 17, 2015

Jonathanm: I read a post with you and jonp talking about the override and saw you have the same board as myself. Are you using the onboard LSI controller? If so have you updated the firmware and are you using IR or IT firmware? I flashed to IT and am at FW 20.x.

I am using the board as shipped, no updates. All SATA connections occupied. I work on computers for a living, and don't flash bios / firmware unless there is a direct issue to address. Unraid is running well, so I've not messed with it. I gather that I may have to flash it if I ever want to run a PCIe HBA card with an LSI chip, but I'll burn that bridge when I come to it.

live4soccer7 · December 17, 2015

Yeah, I'm not one for updates if things worked. I had read some information on the LSI controller and good improvements and caparibility with the updated firmwares. I also had read that have an sas controller flashed w the same mode was recommended when being used in conjunction with one another. With these things in mind, I updated firmware on the controller to the latest version and flashed to IT because most SAS controllers from what I've seen end up being flashed to IT to work best with unraid.

Any reason why the PCI acs setting would have any affect on the onboard LSI controller?? What else is on the same bus as the PCI-e x 16 bus on the x10sl7-f board??

Again, excuse my ignorance here. I haven't worked with a lot of this before, but am definitely learning a lot. With the below information that I have acquired form tool --> system devices in unraid, does this mean there are 6 different busses on the board in total or that are being utilized? If so, then if this issue is isolated with being shared with another device on the same bus then that should be an issue if I'm reading this right because the Radeon 6450 would be on its own bus. Again, I have no idea if this is correct or my train of thought is correct either, which is why I'm asking.

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller (rev 06)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)

00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)

00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)

00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)

00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)

00:1c.2 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 (rev d5)

00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d5)

00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)

00:1f.0 ISA bridge: Intel Corporation C222 Series Chipset Family Server Essential SKU LPC Controller (rev 05)

00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)

00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)

00:1f.6 Signal processing controller: Intel Corporation 8 Series Chipset Family Thermal Management Controller (rev 05)

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM]

01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series]

02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05)

03:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 03)

04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30)

05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

Help designing/picking a setup for unraid, need to upgrade in a bad way :)

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Archived