Home network upgrading

[monden2]

Hi all!

 

I have seen many good things happening on this forum, so I am hoping I will be able to get some feedback.

 

Firstly, the setup I am currently running. I have a dedicated system running my Unraid OS with a Windows 10 VM. The main reason for the setup compared to a dedicated PC and separate NAS was experimentation and practice with using dedicated storage server and media server in one. Firstly, the server is meant to be used as a storage location for all me media. Secondly, the server is meant as a media server using dockers like PLEX (with SABnzbd, Radarr and Sonarr powering it). The Windows 10 VM on the server is used as my 'daily driver' for photo/video editing with a GPU passthrough for rendering and gaming. Though the system works fine, certain issues with specifically USB (many gaming peripherals connected to the MB USB controllers) made me decide to ditch the VM part as a daily driver, and go back to a dedicated PC with recycling some of the RAM and GPU I currently have. This will free up the server to fully work for storage and media.

 

My current systems looks like this:

 

Motherboard: AsRock X99 Xtreme4

CPU: Intel 5820k

RAM: 32GB G-Skill DDR4-2400

GPU: ASUS Strix GTX 1080 8GB

SSD: Crucial M500 240GB

HDD: 2x WD Red 4TB, 1x WD Blue 2TB, 1x Seagate 'something' 4TB (Parity)

PSU: Corsair RM850

 

For the future, I am planning the following things for my server:

1. Reduce RAM to 16GB and move to my dedicated rig

2. Move the GPU to my dedicated rig

3. Move the PSU to my dedicated rig and replace the server one with a 450 watt PSU)

4. Upgrade with 10Gbe PCI-E card (not sure yet to go with RJ45 or SPF+ yet)

5. Expand the SSD pool with another 240GB drive, and run the setup in RAID0 (current transfer speeds via my VM and Wifi max out at a 2/3 MBps)

6. Find a decent WiFi router to provide me better speeds than the crappy one I currently have from my ISP

 

My future plans for the server will be to store all my media, and run as a media server (hence I want to keep the 5820k in there). I also want to use the server on the side to run game servers (e.g. Minecraft) as well. In addition, I want to install a 1000/10000 Mbps switch to ensure that I could wire up all my regular gigabit devices (laptops, TV's, printers etc.), and my dedicated rig could get a 10Gb connection to my server whenever I need to do my photo/video uploads/downloads. Also, I don't really want to spend thousands of Euro's doing so. So my questions would be the following:

 

1. Would you go with RJ45 or SFP+ for the 10Gbe connection between my server/rig and the switch, and why? (I see X540-T2 cards for about EUR 100, and Mellanox SFP+ Cards for about EUR 50-75)

2. Which switch would be recommended for the proposed solution? (e.g. I am highly interested in going with the ASUS XG-U2008 due to the relative low costs of EUR 250 for 2x 10Gbe and 8x 1Gb RJ45 ports)

3. Which WiFi solution would you go with to get the best WiFi network and internet speeds? (e.g. hard wire multiple AP's to a router, or wire one high performance router to the router with potential range extenders)

4. Would it make sense to also build an additional router using pfSense to optimise network throughput?

 

Please feel free to let me know if you need any further information.

Edited February 8, 2018 by monden2

[JonathanM]

20 hours ago, monden2 said:

3. Which WiFi solution would you go with to get the best WiFi network and internet speeds?

unifi, if you can get them localized to where you are. Each AP should be hardwired to your switch for best results. Start with a single, they are fairly powerful and you may not need more depending on topography. They use POE, so you can put them anywhere you can drop a CAT5e-6 without having to run power.

https://www.ubnt.com/unifi/unifi-ap/

20 hours ago, monden2 said:

4. Would it make sense to also build an additional router using pfSense to optimise network throughput?

not sure what you mean by additional router, you should only be using one router normally. pfSense is a great choice for throughput and features, you can use as little or as much cpu as you need for the features you want.

[monden2]

Hi Jonathan, 

 

Thank you for the reply. The unifi ap's look good, I'll have a look at them. 

 

As for the 'additional' router, I meant to replace the crappy ISP 'wireless' modem/router with a pfSense one, and use the ISP's one just for WAN

[Hoopster]

I have a completely Ubiquiti UniFi home netork.  Two UAP-AC-LR APs, two 8-port PoE switches and a UniFI USG router/firewall.  It's a great system.  pfSense is also a great router choice, but, the USG works very well for me and my network is much faster than it was with a consumer router/switch/wifi all-in-one.  Ubiquiti also makes the Edge router and switch line which some prefer.

[Chris Pollard]

You could direct connect your storage (ie direct cable, no switch) to your PC if you only need 10Gb between those two devices,  that would give you greater flexibility with regard to switches and NIC's .

 

[monden2]

20 hours ago, Hoopster said:

I have a completely Ubiquiti UniFi home netork.  Two UAP-AC-LR APs, two 8-port PoE switches and a UniFI USG router/firewall.

 

Hi Hoopster. Thank you for the input. As much as I would like to get a full UniFi system, as I want to have a 10GBe uplink from my server to the rest of the network, as well as a 10GBe connection with my main rig, I think I would only be able to get the US-48 500w switch, which would be roughly 800 USD. And then I would still have to buy the router. That's a but above my budget at the moment. Hence I think a pfSense router with a dual 10GBe ethernet or SFP+ would be much cheaper, which would allow me to get a suitable PoE network switch. 

[wayner]

Me too on Unifi - I have a USG plus five APs.

 

Are you sure that you need a 10Gig network?  Do you know if your components can take advantage of that much speed?  If you have fast PCs they might but I doubt anything else will need that much speed - even 100Mbps would be more than enough for printers, TVs, streaming boxes, etc.  Wifi devices will obviously not get anywhere near those speeds in the real world.  The bottleneck might be somewhere other than the network and then you have needless spent a lot of money.

[monden2]

14 hours ago, wayner said:

Me too on Unifi - I have a USG plus five APs.

 

Are you sure that you need a 10Gig network?  Do you know if your components can take advantage of that much speed?  If you have fast PCs they might but I doubt anything else will need that much speed - even 100Mbps would be more than enough for printers, TVs, streaming boxes, etc.  Wifi devices will obviously not get anywhere near those speeds in the real world.  The bottleneck might be somewhere other than the network and then you have needless spent a lot of money.

Hi Wayner, 

 

The idea is not to get a full 10G network. I just want my server to have a 10G uplink to a switch so that multiple gigabit devices can connect to it (e.g. For Media consumption) at the same time without any bandwidth issues. I was also thinking about running my future 10G main rig over the switch for my photo and video work. However, given some of the feedback it might seem a better idea to have 2x 10G NICs in my server, one for the 10G uplink to the switch which connects all my gigabit devices, and the second one for a direct 10G connection to my main rig. I'll also connect a UniFI AP Pro to my switch to ensure my WiFi performance is better than the crappy 10/100mbit wireless router I got from my ISP

[pwm]

You need serious power in the server to need two 10Gbit/s links - it's hard work to fill a single 10Gbit/s link.

[monden2]

3 hours ago, pwm said:

You need serious power in the server to need two 10Gbit/s links - it's hard work to fill a single 10Gbit/s link.

Hi PWM,

 

The idea is not to fully saturate both 10Ge NIC's (at the same time), this would be quick exhaustive on the system yes. Its more from a pricing perspective.

The price of the additional 10Ge (@50-100USD)  to make a direct link to my dedicated rig is usually lower than the price difference between a switch with only a 10Ge uplink, and a switch with both a 10Ge uplink and 10Ge output port.

[uldise]

Yes for today, but you wanna start build 10GbE network, switches are at reasonable prices now, this for example will cost about 400usd https://mikrotik.com/product/crs317_1g_16s_rm

[wayner]

9 hours ago, monden2 said:

Hi Wayner, 

 

The idea is not to get a full 10G network. I just want my server to have a 10G uplink to a switch so that multiple gigabit devices can connect to it (e.g. For Media consumption) at the same time without any bandwidth issues. I was also thinking about running my future 10G main rig over the switch for my photo and video work. However, given some of the feedback it might seem a better idea to have 2x 10G NICs in my server, one for the 10G uplink to the switch which connects all my gigabit devices, and the second one for a direct 10G connection to my main rig. I'll also connect a UniFI AP Pro to my switch to ensure my WiFi performance is better than the crappy 10/100mbit wireless router I got from my ISP

How much media will you be streaming simultaneously?  MPEG-2 broadcast streams use some of the highest bitrates at 19Mbps.  But with 1Gig you could stream 50 of those and still have a bit of room left over.  Even if you have some high bitrate 4K content I can't see it going above 50Mbps.

[pwm]

5 hours ago, monden2 said:

Hi PWM,

 

The idea is not to fully saturate both 10Ge NIC's (at the same time), this would be quick exhaustive on the system yes. Its more from a pricing perspective.

The price of the additional 10Ge (@50-100USD)  to make a direct link to my dedicated rig is usually lower than the price difference between a switch with only a 10Ge uplink, and a switch with both a 10Ge uplink and 10Ge output port.

I understand you don't plan to saturate both 10Gbit/s NIC:s - but do you plan to saturate a single NIC?

 

If your workstation performs multiple streams and you have a supervised gbit switch with a 10Gbit/s uplink you could use link aggregation to be able to split the streams over two gbit network cards and switch ports and still have 8 gbit/s capacity from your server to other consumers. Or if your workstation does SMB transfers then you could use SMB multichannel mode where you split the SMB communication over two links - and in this case the switch doesn't need to know about it since the two network interfaces on the workstation would have individual IP and all multiplexing is handled by the client and server.

 

And as noted - the price of 10Gbit/s switches are dropping quite fast, so a 4-port or 8-port switch isn't so terribly high.

[c3]

I am using lots of 10G now, but it is never full. However it is often beyond the 1G mark. Isn't that the real question? Not will 10G be filled, but has the 1G limit been reached? Similarly, the use of (2) 10G (or 1G) is not about capacity, but availability. Everything possible is connected via mlag.

 

[pwm]

1 hour ago, c3 said:

I am using lots of 10G now, but it is never full. However it is often beyond the 1G mark. Isn't that the real question? Not will 10G be filled, but has the 1G limit been reached? Similarly, the use of (2) 10G (or 1G) is not about capacity, but availability. Everything possible is connected via mlag.

 

 

The use of dual links can be for several reasons. In some situations it's about availability. In some situations it's about capacity - that you double the bandwidth by being allowed to run more concurrent streams. The aggregation of two 1Gbit/s links doesn't allow a single data stream to run at 2Gbit/s but it allows multiple streams to aggregate past 1Gbit/s of total bandwidth so you can have a server stream out more data spread over multiple clients.

 

The use of SMB multichannel is a way to increase the capacity by having the SMB framework multiplex the file copying over multiple links, which means a single client can get twice the bandwidth from the same server even without performing multiple, concurrent, file copy operations.