RussellinSacto Posted August 13, 2015 Share Posted August 13, 2015 Hi Everyone, I want to run several VM/Dockers to replace several physical computers I own: 1 - Docker (hoping I can get several things to run that were never reliable for me as UnRaid plugins): CrashPlan Plex BT Sync Anti-Virus 2 - Main Desktop (Windows 7) with the following peripherals: Wacom Cintiq 12" LCD Graphics Tablet Fujitsu ScanSnap Scanner (highly recommended) old (reliable) ScanJet 6300C Typical USB CD drives, Printer, etc - as needed Using the following software: Photoshop (with HUGE 25 GB Image files) Daminion Photo Manager (highly recommended, but has been SLOW for me) which runs postGRE SQL) Media Center Master Gigapan and other Image Stitchers Visual Studio / Corona SDK 3 - Testing VM Used to evaluate software before I buy/install/commit to my Main Desktop. That's the core... I might spin up a VM for a specific task/project for a while, but those would be my standard... hopefully either booting fast, or always running. I am fortunate that I can afford whatever makes sense for my needs - but I don't want to just waste money either. Several contenders to me: i7-4790K (Quad Core, 4.0 GHz) with 32GB of RAM i7-6700K (Quad Core, 4.0 GHz Skylake) with 64 GB of RAM i7-5960X (Eight Core, 3.0 GHz) with 64 GB of RAM What makes the most sense? Have I forgotten another worthy contender, like a Xeon or dual Xeon setup? Will I benefit from 64GB of RAM? I don't play any video games (though I might play Cities: Skylines or others if I had the horsepower). Thanks, Russell Link to comment
tr0910 Posted August 13, 2015 Share Posted August 13, 2015 You will run one app per docker, so expect to have 4+dockers. Biggest issue is Photoshop. You will be a pioneer with testing Photoshop at this level with the size of files and complexity of connected hardware. Get @jonp involved and follow his suggestions. What are you running now for hardware and version of Photoshop and what is your level of pain with your current setup? Scratch disk tuning in unRaid, and other requirements for large image processing are potential challenges ahead. Link to comment
garycase Posted August 13, 2015 Share Posted August 13, 2015 •i7-4790K (Quad Core, 4.0 GHz) with 32GB of RAM •i7-6700K (Quad Core, 4.0 GHz Skylake) with 64 GB of RAM •i7-5960X (Eight Core, 3.0 GHz) with 64 GB of RAM For the serious computing horsepower you want, I'd go with a Socket 2011-v3 setup using the 5960X. This has 50% more CPU "horsepower" than the 4790k or 6700k CPUs (which have nearly identical PassMark scores), with the added benefit of much higher bus bandwidth and a quad channel memory controller. The 5960X actually has slightly lower single-thread performance than the lesser CPUs, but easily outperforms the others on any application that takes advantage of more than one core (which Photoshop does nicely) ... and with 8 cores to spread around you'll have plenty of "horsepower" to spread around as needed. The only appreciable advantage of a 6700k vs a 4790k is the much better on-chip graphics, but with your needs I'm sure you'll be using an add-in video adapter anyway, so that's somewhat moot. You may also want to consider using a Xeon CPU, which would let you use a server class motherboard and provide a more reliable RAM subsystem with ECC. The E3's are less expensive, but the E5's can give you some very significant "horsepower" and use buffered RAM, so you can use prodigious amounts of RAM with no bus loading issues. Link to comment
RussellinSacto Posted August 13, 2015 Author Share Posted August 13, 2015 Thank you guys for the feedback. It sounds like the 5960X is the way to go - unless I can learn which Xeons would be a better solution. I follow that I need one Docker per app - does each Docker need it's own thread? So effectively, each of those four apps require a separate thread... If it's going to take four threads for those four apps, am I any better just running another Windows VM with four threads to run those tasks? Excellent question about my current hardware and my current pain - I have a Core2 Quad at 2.4GHz with 8GB of RAM. I don't use a separate video card - but if it makes sense to get one, I will. My pain is severe... I've been shooting Gigapixel Panoramic images (I take dozens, sometimes literally hundreds of 21 Megapixel photos - Canon 5D Mark II), for about 12 years - and I have only processed about 20% of my images into panoramics - because it is soooooo much work.... and soooooooo slow. Even properly cataloging the images takes forever... So I'm sitting on a mostly unprocessed collection of images that I've never really seen... and I'm sad... and tired of that. :-) I currently have Photoshop CS6; I think there's a newer version - I can upgrade if that helps take advantage of extra horsepower/RAM/etc. (I can share an image if someone is really interested in testing.) My OS, scratch disk, and image files are all on SSD when I do my editing. I've been reading about "RAM Disk" where, as I understand it, I would purchase enough RAM to use part of it as a Hard Disk... Is that something that might help? Or is it impossible to get enough of it to make much of a difference without spending north of $3K? Cause I probably don't want to spend more than that. Thanks for the awesome help, Russell Link to comment
jimbobulator Posted August 13, 2015 Share Posted August 13, 2015 You will use one docker container per application, this is due to the way docker is intended to work. It compartmentalizes, or containerizes your applications so they are isolated from each other. This makes it easier to update and manage them. It has nothing to do with threads, which you don't really need to worry about. As far as how docker containers takes advantage of your processor, unraid lets you decide which CPU cores and how many to make available to each application. Hope that helps. For performance questions regarding PS and your images, you might have better luck asking for feedback on a photography forum. You won't find as my computer experts but surely someone would have some performance feedback working with similar images on comparable hardware for a reference point. Link to comment
RussellinSacto Posted August 13, 2015 Author Share Posted August 13, 2015 Thanks Jimbobulator, So, I understand each Docker app goes in a Docker container - does each Docker container require its own CPU/Hyperthread - or do they share? The Photoshop feedback pretty much says to have as much computer as you can. :-) I'm hoping I can go with the 8 core (16 threads), have 2-4 threads stream Plex, do Crashplan, etc., have 2-4 threads for my Testing VM, and then have the remaining 8+ threads available for my "Desktop" experience. Does that sound reasonable? Thanks, Russell Link to comment
tr0910 Posted August 13, 2015 Share Posted August 13, 2015 My pain is severe... I've been shooting Gigapixel Panoramic images (I take dozens, sometimes literally hundreds of 21 Megapixel photos - Canon 5D Mark II), for about 12 years - and I have only processed about 20% of my images into panoramics - because it is soooooo much work.... and soooooooo slow. One step at a time.... First shall we get rid of the pain, before we introduce you into another potential pain point??? I think you are starving your CPU for RAM working with such huge files. The SSD will help, but a healthy dose of RAM will be better. My suggestion, increase your RAM to the max you can on the current motherboard first. (especially if this RAM also what you will be installing in the new system). Make sure you are running the 64 bit version of Photoshop. Then, do some more processing.... You could play around with other things like installing a discrete GPU, but Photoshop doesn't get great usage from them. While the latest CC versions might utilize a hot graphics card, CS5 and CS6 get marginal benefit. Is Photoshop the dog in this process or something else??? Once you have a respectable improvement in image processing, then start to think about your end goal. I expect that you may lose a bit of performance or find a nasty roadblock ahead by going all in with unRaid. However this is all unknown territory with this kind of usage application. Can you use your old Photoshop workstation as your unRaid server and run all the image processing on the new box if this plan falls apart?? Link to comment
RussellinSacto Posted August 13, 2015 Author Share Posted August 13, 2015 Hi TR, Thanks for the good advice... My current system, as I understand it, doesn't support anywhere near 64GB of memory - and I don't think they're compatible (DDR2 on my current machine, I think - DDR4 for the 5960X). I currently have two Unraid boxes (Celeron based), so I was thinking this change would be the upgrade of one of them... and should the plan not work, for whatever reason, I'd put it back the way it was... and instead upgrade my current desktop with the new processor, motherboard, and RAM. I am running 64 bit Photoshop. Photoshop is a dog in the process, but for a lot of that work I have other solutions - I can walk away and let it run... I can split the image into smaller chunks for most work and then do final corrections on the re-joined image... but your inquiry really has me thinking deep about my pain point... and I think it's not Photoshop. I love Daminion for photo management (www.daminion.net), but it's been pretty slow for me to work with - even on the 21 Megapixel JPGs. I believe this slowdown is because my images are on UnRaid and not local... I can't build panoramics with hundreds of images before I've organized the images for the stitching software. This post is what got me thinking of combining my Workstation and my UnRaid - because of the hard disk speeds of local versus network (http://mediaserver8.blogspot.com/2014/01/unraid-60-vm-speed-comparisons.html). The example shows File transfer of 11MB/second across the network from Unraid - and File transfer of 104MB/second from a VM on Unraid. Is this typical? (Writes are slower, but still a 3X of the network comparison). If it is... I must do this! :-) I appreciate your suggestion to have a backup plan... I will pay careful attention to vet out my plans before moving forward. Russell Link to comment
garycase Posted August 13, 2015 Share Posted August 13, 2015 ... have 2-4 threads stream Plex, do Crashplan, etc., have 2-4 threads for my Testing VM, and then have the remaining 8+ threads available for my "Desktop" experience. Does that sound reasonable? Yes, that's a good mix and will give Photoshop the benefit of both plenty of RAM and plenty of CPU "horsepower." r.e. your comment: "... File transfer of 11MB/second across the network from Unraid ..." ==> That's what you'll see on a 100Mb network; but on a Gb network you should get well over 100MB/second. Link to comment
WeeboTech Posted August 13, 2015 Share Posted August 13, 2015 My personal take, I would relegate the core2 duo to unRAID as a file server and re-build the windows desktop workstation with the new CPU, large amount of ram and some SSD's in RAID0 for the scratch work area. There's all those peripherals to consider who knows how well that's going to work under a virtualized environment. Also, You can achieve around 90MB/s over a good gigabit network. With a smaller array, fast 7200 RPM drives and turbo write you can get that speed on writes (or to a cache). I get this moving mp3's back and forth and/or tagging them. The i7-5960X has a 20MB cache from what I read, That's going to make it scream for data maniupulation (images) and/or virtualization. However I would keep the fileserver task separate from the desktop/workstation. (but that's just me). In fact I do this now. I have a bunch (lil farm) of HP microservers and a screaming set of Laptops with high end i7's and 32GB of ram. If I have to work on the server or laptop, one does not impact the other for most tasks. The only issue with relegating the Core2Duo to unRAID duty, might be plex. I don't know enough about it and that may be a show stopper for what you intend to do. As far as a test vm for testing out new software, You can do that on unRAID or on the upgraded workstation with VMware workstation. (or some other product). I'm familiar with Vmware workstation (which is what I use on my laptops). When you switch it to full screen mode, it's almost like having bare metal. In fact I ran Windows 2000 and XP under Vmware workstation under CentOS for many years after my windows machine died. It ran pretty well for basic desktop usage. Link to comment
RussellinSacto Posted August 14, 2015 Author Share Posted August 14, 2015 Thanks guys, Apparently I have a BAD gigabit network... I'm going to have to look, but I didn't think I was getting anything north of 20MB/sec. I hope I'm wrong, but I'll check tomorrow. Weebo, you provide a great idea... I swear my network transfer speed has been horrible - that was really one of the great ideas to merge the workstation onto UnRaid was for file transfer speed. The Core2 Quad (not Duo) does a decent job of running Plex on Windows 7, so hopefully it would be okay running on UnRaid 6 as a Docker app. This might be because I only stream to one or two devices at a time. Thanks for the continued ideas and advice. Russell Link to comment
garycase Posted August 14, 2015 Share Posted August 14, 2015 Agree with Weebo => repurposing your old workstation as the UnRAID box would REALLY give you some serious horsepower for your main workstation with a new 5960X system. And the 5960X will also run VM's VERY nicely under VMware Workstation (or even the free VMware Player -- which is good enough for most purposes, but is missing one VERY nice feature of Workstation -- snapshots). Link to comment
WeeboTech Posted August 14, 2015 Share Posted August 14, 2015 (or even the free VMware Player -- which is good enough for most purposes, but is missing one VERY nice feature of Workstation -- snapshots). True, I've used these on more then one occasion to test software installations and then back them out. Link to comment
garycase Posted August 14, 2015 Share Posted August 14, 2015 I use snapshots ALL the time ... they are IMHO easily worth the price of Workstation. Link to comment
RussellinSacto Posted August 14, 2015 Author Share Posted August 14, 2015 Thanks guys. This has been great... I've made the discovery that I only get 10-12MB/second transfer from either of my UnRaid boxes. I thought I had all gigabit switches in the house... but I've discovered I have one that is 10/100... I presume that's slowing the whole thing. :-( I moved a few cords and I'm up to 75-95MB/second... WOW! :-) So, the general consensus seems to be to keep the workhorse and UnRaid separate (provided I can get the network up to something more reasonable). Thanks, Russell Link to comment
tdallen Posted August 14, 2015 Share Posted August 14, 2015 Personally, I upgraded my original Core2Duo unRAID server with a Core2Quad off eBay, and 8GB of DDR2 RAM. That does a good job running unRAID and several dockers - in fact the RAM is almost overkill. Then I built myself a new Core i7 4790 small form factor workstation for Windows, test VMs, and all the general heavy lifting including photo manipulation. I'm happy, and while I like the idea of fewer boxes I guess I'm not quite ready to try and do it all in one. Oh, and my server is always on and stashed in a closet downstairs, while the Core i7 just goes to sleep when it isn't needed. That works well for me - server duties tend to be a bit noisy and hot, something I don't want upstairs where I work. As to your network speeds - yep, sounds like you had some 100MB hardware in there, get newer/cleaner Gigabit hardware everywhere and you'll be much happier. Link to comment
bubbaQ Posted August 14, 2015 Share Posted August 14, 2015 Remember, Intel handicaps the K processors for doing virtualization... no Vt-d. K serie4s is for overclocking.... VT-d (allegedly) becomes (or may become) unstable when overclocking. Link to comment
saarg Posted August 14, 2015 Share Posted August 14, 2015 Remember, Intel handicaps the K processors for doing virtualization... no Vt-d. K serie4s is for overclocking.... VT-d (allegedly) becomes (or may become) unstable when overclocking. That is not true. Se specs in the link. http://ark.intel.com/m/products/80807/Intel-Core-i7-4790K-Processor-8M-Cache-up-to-4_40-GHz#@product/specifications http://ark.intel.com/m/products/88195/Intel-Core-i7-6700K-Processor-8M-Cache-up-to-4_20-GHz#@product/specifications Link to comment
garycase Posted August 14, 2015 Share Posted August 14, 2015 Remember, Intel handicaps the K processors for doing virtualization... no Vt-d. K serie4s is for overclocking.... VT-d (allegedly) becomes (or may become) unstable when overclocking. That was true for some of the early K-series CPU's, but is not true for any of the processors being considered here. http://ark.intel.com/products/82930/Intel-Core-i7-5960X-Processor-Extreme-Edition-20M-Cache-up-to-3_50-GHz http://ark.intel.com/products/88195/Intel-Core-i7-6700K-Processor-8M-Cache-up-to-4_20-GHz http://ark.intel.com/products/80807/Intel-Core-i7-4790K-Processor-8M-Cache-up-to-4_40-GHz Link to comment
RussellinSacto Posted August 15, 2015 Author Share Posted August 15, 2015 Thanks everyone for helping me work through this... Seems my network is dropping from Gigabit to 100mbps because of my Roku 3. Does that make sense? I've been digging into my two Unraid boxes. They have similar motherboards: Supermicro MBD-C2SEE-0 Supermicro MBD-C2SEA-0 It seems like I could take the 8GB of RAM and the Core2 Quad Q6600 (2.40 GHz) in my existing workstation and put it into one of these. I'm hoping that would give me enough horsepower for at least my Docker needs (Plex, Crashplan, BTSync, hopefully an Antivirus). Do you think that would work? (If so, I think I could get another Core 2 Quad for about $100 for the other UnRaid if needed.) 1. Is it my Roku? :-( Is there a suggested Network Switch that won't slow down with it on the network? 2. Does the processor/RAM move make Docker possible? VMs too? (only 8GB RAM max supported on motherboards) Thanks, Russell Link to comment
garycase Posted August 15, 2015 Share Posted August 15, 2015 Those are pretty dated, but rock-solid reliable motherboards (I still have one UnRAID system on a C2SEA). If you pop in a quad core CPU and 8GB it should be just fine for Plex with one stream. Certainly wouldn't hurt to give it a try. You could run Dockers with no problem, and should even be able to run a VM or 2, although the limited memory would restrict just how well that would work (and without vt-d support, you couldn't pass through any resource that required hardware pass-through support). As for the Roku restricting the network bandwidth -- not sure why this is happening. What's the topology of your network? Link to comment
RussellinSacto Posted August 15, 2015 Author Share Posted August 15, 2015 Thanks for the advice Gary... They are dated... So far, I've only bought hardware that was exactly a match (except perhaps the case) as what Lime was specifically selling at the time. I like the practical guarantee of reliability and compatibility that comes with that. I could stream Plex from my new desktop if I need to... I'm really, deeply hoping that I could Docker CrashPlan (I have 5TB in the cloud) and maybe a Sync solution (BTSync or other) to keep my two UnRaid's synced. Previously, the plugins have crashed for me - especially the CrashPlan one (they have a page specifically on uploading lots of files and maybe there was a memory leak or something: http://support.code42.com/CrashPlan/Latest/Troubleshooting/Adjusting_CrashPlan_Settings_For_Memory_Usage_With_Large_Backups) I don't need VMs on those boxes... was just hopeful I could tinker. :-) You say topology like I planned it. LOL Master Bedroom Closet has the telco router with one port - to a DLINK DIR-655 wireless router which has several physical ports that branch throughout my household wiring (CAT5e). One of these lines goes to the Roku, one goes to a computer in a bedroom, and one goes to my main computer room, where I have a Netgear GS605 which feeds my two UnRaids and my main desktop computer. (I also DID have a NetGear connected to it - a 10/100 FS605 to feed my iMac, Printer, highspeed scanner... but I took that out of the mix. Need to buy a new switch to replace the pair in the main computer room for sure!). So, I think it must be the Roku... but the wife is watching it right now... So, I'll have to wait and test things later. Appreciate all the help, Russell Link to comment
WeeboTech Posted August 15, 2015 Share Posted August 15, 2015 Both of those devices are ethernet switches that support 10/100/1000 - 5-port 10/100/1000 MbpsGigabit Ethernet Switch (GS605) - Xtreme N Gigabit Router (DIR-655) includes a built-in 4-port 10/100/1000 Gigabit Ethernet switch. I don't think that's the problem. There might still be a hub in there or the port on your unRAID server is not at gigabit speed. You can try and run ethtool on the ethernet port to determine connection speed. Here are some example commands root@unRAIDb:/var/log# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.254 netmask 255.255.255.0 broadcast 192.168.1.255 ether 0c:c4:7a:31:46:12 txqueuelen 1000 (Ethernet) RX packets 514291593 bytes 761943197958 (709.6 GiB) RX errors 0 dropped 142230 overruns 0 frame 0 TX packets 38427177 bytes 3729715590 (3.4 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device memory 0xf7400000-f747ffff lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 0 (Local Loopback) RX packets 1890588 bytes 205896399 (196.3 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1890588 bytes 205896399 (196.3 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 root@unRAIDb:/var/log# ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on MDI-X: off (auto) Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000007 (7) drv probe link Link detected: yes Look on each switch and see if you see color variances, Some yellow or green. There could be a bad cable between one of them. FWIW, you could be at full gigabit speed, but then you have other situations inside the machine that limit how fast you go. I had to tune the kernel a bit to get the high speeds I get. The speed of your data drive and parity drive play a crucial role as well. How full the file system is,etc, etc. For me, I select the fastest, largest parity drive I can afford. Currently the 6TB HGST 7200 RPM drives get around 225 MB/s on the outer tracks. If you have an equally matched data drive you could achieve up to 100MB/s on the other tracks in unRAID. There are tools one comes up called iperf. There's probably windows versions and linux versions so you can test between them. That would provide a test of just network speed alone thus helping alleviate the questions. What version of unRAID are you on? Are you using a cache drive? Link to comment
WeeboTech Posted August 15, 2015 Share Posted August 15, 2015 So, I think it must be the Roku... but the wife is watching it right now... So, I'll have to wait and test things later. If it's on a switch, that would not do it unless you are using it in some kind of pass through mode. Link to comment
garycase Posted August 15, 2015 Share Posted August 15, 2015 From what you've described the issue almost certainly is NOT the Roku. Both the router's built-in switch and your additional switch are (as WeeboTech noted) Gigabit switches ... so something else is causing this issue. Note that ONE bad pair in any of your connecting cables will result in degradation of the transfers to 100Mb speed, as will a bad switch port, or a bad Ethernet adapter (typically with one bad line = the equivalent of a faulty cable). First question is are you sure all your computers have Gb adapters? If so, I'd check the speed of a transfer between your "computer in a bedroom" and your "main computer". If that's at Gb speed (just how fast that is depends on a variety of factors -- the areal density of the disks involved; the speed of the PC's; etc. ... but I'd expect 90-110MB/s.). If that's running "at speed", then the cables between the computers and their respective switches are fine. Next try one of those PC's to one of your UnRAID servers ... and then the other server. If BOTH are limited to 100Mb speed (~ 11MB/s) then try replacing one of the cables to the servers. You might also test the switch port the server is connected to => move one of the PCs you know works at Gb speed to that port and see if the speed drops. Basically it involves a lot of checking (to confirm everything's Gb capable) and a lot of testing (to isolate WHICH component is slowing things down). The one thing that's almost certain is that it is NOT the Roku Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.