bnevets27 Posted December 30, 2013 Share Posted December 30, 2013 Excelsior is a Latin adjective meaning "higher". See second post for initial purchase and plans. This server started life as a server from TAMS (popular dealer for second hand servers, master thread in over at avsforums) As time has gone on and unRAID has changed so has this build. Current specs: Chassis: AIC 24 Bay - Modified to house the following Motherboard: Supermicro X7DWN+ Processors: Intel Xeon L5420 @ 2.50GHz X2 Heatsinks: Apple Mac Pro A1186 (w/drilled out mounting legs) X2 RAM: 40GB (2GBx12) + (4GBx4) Fully Buffered ECC DDR2 667 MHz SATA Expansion Card(s): SAT2-MV8 Raid Controller Cards X2 IPMI Card w/KVM: Supermicro AOC-SIMSO+ Power Supply: SeaSonic X-1250 unRAID USB: Kingston MobileLite G3 w/8GB microSD running latest v6 HVM: Enabled IOMMU: Enabled Modifications: Case: This server of course came stock with 3 power supplies that are very loud. The AIC case is a bit more difficult to modify, compared to the other server TAMS was selling, the supermicro, due to the fact the power supplies are on the bottom. There are a few ways to go about getting a ATX power supply into to case but for various reasons I decided to "roll my own" solution. I need to get the following to fit inside the case: Enhanced Extended ATX motherboard Apple Mac Pro heatsink ATX power supply - SeaSonic x-1250 I also wanted it to be quiet, I knew it wouldn't be silent but wanted it to be tolerable to be in the same room and of course it needed to keep everything cool under load. To accomplish the above I decided I only wanted to use 120mm fans and no CPU fan would be ideal. This of course is why I had purchased the apple heat sink, though I had no idea if it was going to be too big and if it would keep the CPU cool. I had looked up as much info as possible before hand but I didn't find anyone else that had tried to use it in this configuration. First I removed the back plate, then the motherboard tray, and finally the power supplies, which left just the sides and hard drive backplane. Thankfully AIC made everything very modular and everything is screwed together. This allowed me to remove the I/O port plate and PCI bracket piece from the rest of the backplate and reuse it. After a couple different ideas and mock ups, I ended up with two 120mm fans. Which is what I wanted to do. Mocking up I started with an non modular PSU, but quickly realized I didn't want, or have room for, the extra cables. I also cut down the motherboard tray so it would sit flat on the bottom and fit to one side. I did this so I didn't need to deal with stand offs. Cut up backplate/IO bracket Re-enforcing and re-attaching backplate to IO bracket The heat sink: Can't remember how I stumbled upon the heat sink but I don't think there is many passive heat sinks for socket 771. Anyhow, everything I could find about heat sink made it sound like it would at least fit the socket and should cool the chip. Though it is tall so I still wasn't sure if it would fit the 4U case. Fast forward to the installation of the heat sink and I found out apple, being typical apple, decided to use a non standard screw for attaching the heat sink to the motherboard. Well not the motherboard but the standoffs under the motherboard is what the heat sink attaches to. I decided to get some 1in 6-32 screws and drill out the heat sink to allow the bolts to pass through the M3 threads inside the legs of the heat sink. Things still to do on the case: The heat sinks are about a 1/4in too tall. But the cover does kind of fit, it just bends a bit. Haven't decided how to fit it but I will make it fit better. Fan wall needs mount brackets Various other final touches PSU: Made all custom cables except the main 24+4 motherboard connector. (Don't tell SeaSonic) PSU modular connector to molex adapters Backplane power cables, later shorten the connections between plugs My cable on the left, stock right. Made the cable shorter and combined the 8pin and 4 pin MB power plugs into one cable/PSU plug Motherboard: I only have PCIe x8 slots but I want to use a GPU for a VM. So I cut the back out of one of the PCIe slots. Big video card fits Reused one of the squirrel cage fans (they are quite and have good air flow) over the PCI slots just because there was room. Its removable if it gets in the way. I was short 2 long sata cables which is why there are two running over the top of the fans. There is still a bunch of little stuff to clean up. Will post finished pictures eventually. Now for the plan: I have an interest in a lot of the dockers that are available. I am also planning on running a VM and seeing how that can be utilized any may expand to more. Plugins: Installed: Community Applications Dynamix Plugins: System Temp System Stats System Info Preclear IPMItool Server Layout Powerdown Package (2.18) Enhanced Log Viewer To install: cache dirs Dockers: CouchPotato - linuxserver.io - Installed Sabnzbd - linuxserver.io - Installed SickRage - linuxserver.io - Installed Plex - linuxserver.io - Installed PlexPy - linuxserver.io - Installed Hydra - linuxserver.io - Installed Apache - linuxserver.io - Installed Plex Request - aptalca - Installed Dolphin - aptalca - Installed ownCloud synthing/BTSync/Sync - not sure which one yet CrashPlan Deluge - or any other BT client One of these two: Emby Kodi - Headless w/mariadb (I have been running xbmc headless with mysql on my current v5 unRAID server) Dockers I may or may not keep, need to test them: SNMP WebSync - not going to use this probably, sad that this isn't as functional as it needs to be darkstat mochad Krusader - use this or dolphin Filebot BrewPi nzbget - never tried it, will try and see if it suits me better then sabnzbd Sonarr - never tried it, will try and see if it suits me better then sickrage ------------- BackupSQL - if i use kodi headless and mariadb Amule FlexGet Headphones - w/musicbrainz docker? JDownlaoder2 MovieGrabber Calibre Google Music Manager H5AI IRC Client MakeMKV HTPC Manager Maraschino Mosh sonerezh Configurations I still need to do: Link current "production" server (which will become a backup/archive) to new server via direct gigabit connection setup rsync scripts: backup important files from main server backup cache appdata and docker.img to array (caveat, I've copied the docker.img off the cache then back on and get the "please recreate the docker image (i'll paste the actual error here later)) backup usb flash to array Script to print all file names to text file Improve Apache configuration: Add more users Force user to log in to see index page? Create nice index page with links to services Move away from Apache to ngix? run: unraid-tunables-tester.sh Test UPS Setup windows (10?) VM Current hang ups: Guides I used and various other info: Quote Link to comment
bnevets27 Posted December 30, 2013 Author Share Posted December 30, 2013 The following info is now deprecated with the introduction of v6 of unRAID. Updated specs and plans are in the first post. I wanted a second server to start playing with and I was able to snag a server from TAMS. After seeing all this virtualization around the forums for awhile and seeing what flexibility it allows I was definitely interested in getting into it. Unfortunately the hardware that came with my TAMS server does not support VT-d. My hope was to replace the motherboard with a motherboard that supported VT-d but still was able to use all the same components. Warning: All photos are taking with my phone in terrible lighting and in haste, sorry. Here is the server specifications as received from TAMS: Chassis: AIC 24 Bay Motherboard: Supermicro X7DBE-X Processor: Intel Xeon E5120 - 1.86Ghz Dual-Core RAM: 8GB (4x2GB) Fully Buffered DDR2 SATA Expansion Card(s): 3 x SAT2-MV8 Raid Controller Cards IPMI Card: Kira 100 SIMLC Power Supplies: 3 x Emacs M1W-6500P unRAID USB: Kingston MobileLite G3 w/8GB microSD Upgrades I bought, all from ebay: Motherboard: Supermicro X7DWN+ IPMI Card w/KVM: Supermicro AOC-SIMSO+ CPU: Intel Xeon L5420 - 2.50GHz Quad-Core There is some evidence/talk about ESXi be successfully run on the X7DWN+ which is outlined in this post. Which is why I chose it, also because it could be found for cheap on ebay. My first worry was if it wouldn't fit into the case as the X7DBE-X is an extended ATX and the X7DWN+ is an Enhanced Extended ATX which is 1 inch wider. Luck would have it that the X7DWN+ does fit into the AIC case. One of the side fans and one of the standoffs has to be removed (this is really important as it would sit on the pins of the motherboard and cause a short). Most of the rest of the standoffs line up and even better the I/O panel was an exact fit. X7DWN+ beside the stock X7DBE-X Why the fan had to be removed All wired up (AOC-SIMSO+ and L5420 not installed yet) There is a few other things to keep in mind about the X7DWN+ there is only two PCI-X slots so only two of the three SAT2-MV8 cards can be used and therefore your limited to 18 drives using the existing components. The positive that is has three PCI-e slots so you can use newer controller cards (ie the M1015), the X7DBE-X has none. The X7DWN+ also has more memory slots and the IPMI w/KVM card is cheaper. The server booted and ran unraid fine on both the X7DBE-X and the X7DWN+. I wasn't able to do any extensive testing as I don't have a lot of extra drives laying around. I had 2 data and 1 parity drives, ran parity checks a few times and nothing odd came up. I purchased a pro key for my Kingston MobileLite G3, it took a little be of trial and error to get it to boot properly on the X7DWN+ and outlined in this thread. As many people with the TAMS servers have mentioned this server is LOUD. My server is in the basement so it's less of an issue but it's something that should not be over looked. I currently only have one PSU plugged in which helps a little with noise, the warning buzzer has to be reset every time you power on the server if only running one PSU. The two exhaust fans can actually be louder then the PSU fans. They can be throttled in the BIOS with preset options, I chose "workstation", but they are still loud. I think the PSU fans are thermally throttled and can be surprisingly quite when cold, unplug the rear fans and its actually very quite for a server but its not practical to run like this as it doesn't draw any air through the case. This isn't a big deal for me at the moment but I make make some modifications in the future to quite the server down. Now the question is which virtualization software should I use, ESXi or Xen/KVM? I looks like Xen/KVM is getting a lot of attention on the forum as of late. But virtualization is a whole new world for me. There also seems to be a few XEN/KVM options (thanks to grumpybutfun) . [GUIDE] Virtualizing unRAID in Xen / KVM on Arch Linux [GUIDE] Virtualizing unRAID in KVM on openSUSE 13.1 (64Bit) <--- Completed 12/19 [GUIDE] Virtualizing unRAID in XenServer 6.2 - Updated to 5.0.4 New Any advice on which software would be best suited for me? Current plans are just to install unraid and a linux distro to run all the programs currently running as pluggins. I would also like a few VM's to play around with on linux and windows. I also might have some interest in running XBMC on one of them, if that matters. Currently I have the following plugins installed: APC UPS MySQL Plex Media Server Subsonic XBMC SaLUd Couchpotato Headphones SABnzbd Sickbeard Transmission I'm hoping to find out if hardware passthrough works correctly with the X7DWN+ as it looks like it would be a cheap and easy upgrade to get hardware passthrough on TAMS servers (Intel versions only of course). Quote Link to comment
bnevets27 Posted December 30, 2013 Author Share Posted December 30, 2013 Reserved for future use Quote Link to comment
bnevets27 Posted December 30, 2013 Author Share Posted December 30, 2013 Reserved for future use Quote Link to comment
bobmagnuson Posted December 30, 2013 Share Posted December 30, 2013 I'm looking forward to the results of your build. I've got the same TAM setup and was disappointed to see that VT-d wasn't supported with the motherboard/cpu combo I've got. Quote Link to comment
bnevets27 Posted January 6, 2014 Author Share Posted January 6, 2014 Followed this guide: [GUIDE] Virtualizing unRAID in KVM on openSUSE 13.1 (64Bit) <--- Completed 12/19 And can report hardware passthough works! I'll have to iron out some details and post some screen shots in the coming days. Then when I get a change I'm going to try out ESXi. Only hiccups I found were: 1. During the openSUSE install: non-existent floppy drive enabled in BIOS causes hang while searching for partitions 2. Had to add this to the bootloader: intel_iommu=on 3. Had to add this: options kvm allow_unsafe_assigned_interrupts=1 to /etc/modprobe.d/kvm.conf (create this file, if it does not exist) I was able to pass just the USB device itself through. I installed openSUSE on a 16GB usb stick. Quote Link to comment
bobmagnuson Posted January 6, 2014 Share Posted January 6, 2014 Great News!!!! Keep the info coming. Quote Link to comment
bnevets27 Posted February 12, 2016 Author Share Posted February 12, 2016 Well 2 years later I finally ripped the server apart to rebuild it. Currently testing and working on configuring all the different programs. It will then be used as my primary server and will retire my current one to a backup. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.