10 gbe network card

[Pera78]

hi, i am building my first server in a 10 gb network and i would like to install unraid, can you recommend a 10 gb ethernet network card and its 10 gbe switch. Thanks

[MAM59]

What kind of 10GB Network do you plan, fiber or copper?

If copper, keep in mind, that line lenght is limited (currently I would consider everything >30m as a challenge), ports get very hot, sometimes too hot and there are only 2 serious cards on the market.

One is by Intel, they offer different types with one or two ports. But you have to look carefully. The modern ones use 4 lanes of PCIe 3.0, the older types use 8 lanes of PCIe 2.0 (slots which are common in serverboards, but barely seen in normal Desktop environments)

The other one comes from Aquatic, and is sold under many different brands/names and companies. Its a "user device", meant for high speed gaming, not for file serving. Its cheaper but also a lot slower if used for many concurrent transfers of large files.

The intels are about double of the price of Aquatic, so its a hard desicion.

 

Switches for 10GBe copper get hot and usuall contain one or more annoying fans. Not good for inhouse usage. There are rare switches with a very limited number of ports that come without a fan. But usally they are a dead end.

 

If you look at fiber, there is not much choice either. This is, 10GB on fiber is almost done already, the companies move up to 40 or 100GB. So you have to search for used or refurbished cards. (or buy a 40GN card and use it only with 10GB). When it comes to fiber, you wont find any equipment that is "hard wired". Instead they come with so called SFP+ cages (look for the "+". this means 10GB, without + its only 1GB). You need to buy the card, the switch and two modules that are slipped into the cages and connect the real fiber.

(theres is one shortcut/cheaper way: if it comes to connect two devices on very short ranges, 0,5, 2 or 3m, there are special "direct attach" cables available that consist of 2 modules with a fixed cable running between them. Because it plugs into both ends, it substitutes both modules and the cable between, saving a lot of money)

Fiber modules come with different frequencies and for different line lengths. Of course, you need 2 matching ones!. And "very strong" is usally not the best choice. The "normal" 850nm "Multimode" ones are good for lenghts up to 300m, if you use higher ones with max lenghts of 10km or even 80km you have to know, that if you have a shorter distance you may need to cut the cable in the middle and insert a kind of "dimmer" there to make the signal less strong. These devices dont even work at all if the lenght is below 1km.

So the usual 300m/850nm is most common inhouse or insite.

 

Of course, your switches also need theses cages. You get a great amount of choices in the upper price regions, if you want to play it cheaper I would recomment the company "MikroTik" from Estonia. Although their have raised their prices quite hard right now and they are currently hard to get now, they offer great quality, no fans, and a choice of two different OSses for the boxes. You can either play it safe and easy and go with SwitchOS, which offers the usual functions you know from other companies too (and it boots with a few seconds), or try your luck with "RouterOS" which will transfer the box into a full featured layer 3 router with everything you know and many things you have never heard of before. But, its really complicated and, unless you dont need to put VPN connections onto a bundle with 3 interet lines going to different providers and a fallback to cell phone (you get the idea.. you can do everything but you may need an asylum to recover afterwards)

 

The sad news: 10Gb LAN is much much more sensible compared to 1Gb stuff. Buying the correct and working cables alone can take you a year of experimenting (if you go with copper, you can easily get working CAT.7 cables, but there are NO matching plugs on the market. You have to use CAT.6 and pray...)

Fiber is more stable. I started with copper (coming up from the 1st 10Mbit/s Coax Ethernet and going to 100Mb copper, then 1Gb copper I naturally went to 10Gb copper over the decades) but then mysterious things happened like loss of carriers, lockups of cards and so on. Nothing predictable. I've changed cards, cables, switches (was a real expensive "learning curve") until I got "something working". But not for long, after a few hours the Microtik shut down my line to the garage, the modules were over 90°C and he did a safety shutdown. So I ended up with a fanless switch with a big fan in front :-)))

Then I threw everything out and switched to fiber. No problems anymore and the heat is low as 43°C or so

 

[Pera78]

thanks for the answer but I have even more doubts than before. The idea is to use copper for the network and the PCs are at most 10 meters away

 

[MAM59]

17 hours ago, Pera78 said:

The idea is to use copper for the network and the PCs are at most 10 meters away

Oh, sorry, I did not intend to scare you off 😘

If around 10m its not really a big problem with copper. The biggest challenge is to buy the correct patch cables, You will notice that the plugs for 10GBe have changed slighty, they are a bit longer now. Normal cables fit, but are not really tight (you can move them back and forth). As I pointed out, there are no suitable plugs yet. But if you fiddle around a bit and maybe use some hot glue its no big deal.

But dont buy those MikroTiks I have meantioned for copper! The modules are twice as expensive and you are not allowed to use all modules for copper because of the overheating problem. You have to leave out every 2nd port, makeing the buy twice as expensive because you need more switches but one.

 

And, I was afraid too before, "fiber to the box" is no problem anymore today. You get pretty much cheap fiber patch cables too and they are surprisibly durable even if bended or moved. I had my wall cabeling done with "keystone" modules everwhere, so it was no big deal to throw out the copper wallets and click in the fiber ones.