Moving to drastic new hardware

[alitech]

Hi guys

 

I hope the issue is pretty simple to deal with and this post is an attempt to just confirm the changes I plan to make. 

On the current unraid box, I currently have a Asrock Rack D1541 server motherboard 32GB RAM. Quadro p600,  and a 4 port Intel Nic for VMs so each VM gets its own IP address. I have 10 HDDs connected with the built in SAS and additional SATA ports along with 2 NVME drives.  I run a bunch of VMs and dockers and I am seriously killing the system with the load, it cannot handle it. Have a look at the load in the attached image. It is also running very hot as a whole. The CPU never goes below 75c and hard disks are also always hot at around 50C.  This is all in a Node 804 case. 

 

I am going to use parts from my underused monster machine and port the following things over into a new Antec tower case that I was able to grab off ebay. Image attached. 

 

Ryzen 3950x

Asus Rog X570 F-Gaming mobo

64GB DDR4

Watercooling custom loop

10GB SFP + card

LSI SAS9361-16i (to purchase)

 

I will use the HDDs, NVMEs and USB from the older machine. I understand I will be losing IPMI etc and 10GB SFP+ ports but I am going to put in an SFP+ connect X 3 in there. 

 

Now in my view, this is quite a drastic change on the hardware side. Can I port everything over and expect it to run as it did with the previous hardware? The drives are the same, the USB is the same but almost everything else will change. I just need to change the boot drive in the BIOS. 

 

Should I be worried about anything? Should I be looking at anything else to deal with? Is this smart? 

 

Edited January 13, 2021 by alitech

[Vr2Io]

1 hour ago, alitech said:

LSI SAS9361-16i (to purchase)

You should buy HBA instead RAID adapter, such 9305-16i. You need confirm previous disks setup in RAID mode or not. Does cable are SFF8643 ?

 

Don't set array auto start during migration, try your best to confirm all work normal before start array.

Edited January 13, 2021 by Vr2Io

[alitech]

6 hours ago, Vr2Io said:

You should buy HBA instead RAID adapter, such 9305-16i. You need confirm previous disks setup in RAID mode or not. Does cable are SFF8643 ?

 

Don't set array auto start during migration, try your best to confirm all work normal before start array.

Thanks for responding. I am unsure which mode the disks are setup as. Can you check my syslog? I tried to look in the syslog for this data but cannot locate it. I am not an expert with unraid. So you are saying I need the HBA card instead then...

 

I guess the real question that I am asking is (apart from confirming which card I need to purchase), should there be a cause for concern when moving my whole set up to the new hardware above? What sort of issues can I encounter when moving everything?

 

 

history-syslog-20210113-1022.zip

[Vr2Io]

SAS controller run in HBA mode, so disks should plug and play with HBA.

 

3 hours ago, alitech said:

What sort of issues can I encounter when moving everything?

This hard to say have issue or not, but Unraid support those hardware (X570)

[alitech]

Is there a cheap HBA you guys can recommend? I have 10 HDDs. I can fit 2 of these if needed.

 

[Vr2Io]

LSI 9211-8i cheap and common. You have 3 16x slot only.

Edited January 14, 2021 by Vr2Io

[alitech]

5 hours ago, Vr2Io said:

LSI 9211-8i cheap and common. You have 3 16x slot only.

Thanks for that. Do you think the newer 12GBPS HBA cards are worth the extra cost? is there a performance boost?

 

Also, I think you mean I have 16x PCI lanes to play with. Would you say 2 of these cards would be ok at 8x each?

 

[Zonediver]

53 minutes ago, alitech said:

Thanks for that. Do you think the newer 12GBPS HBA cards are worth the extra cost? is there a performance boost?

 

Also, I think you mean I have 16x PCI lanes to play with. Would you say 2 of these cards would be ok at 8x each?

 

 

I recommend the LSI SAS 9207-8i with active cooling.

Edited January 14, 2021 by Zonediver

[alitech]

1 hour ago, Zonediver said:

 

I recommend the LSI SAS 9207-8i with active cooling.

Thanks for that. I will have a look at that. Would you recommend 2 of these for my build? Is there is performance boost on 12gb cards. 

[Zonediver]

2 hours ago, alitech said:

Thanks for that. I will have a look at that. Would you recommend 2 of these for my build? Is there is performance boost on 12gb cards. 

 

The SAS 9207-8i is an older HBA and can handle 2x 4 SATA 6GBit (SATA3) Drives directly.

But the max. bandwith is 7880MB/s over PCIe 3.0 x8.

That means if you use two SAS Port-Expanders, you can connect 40x 4TB SATA3-HDDs without a bottleneck (or 20 HDDs with one expander).

 

The Problem: This expanders are terrible expensive...

 

So its cheaper to buy two 9207-8i if you like to connect 16 HDDs.

 

Calculation example:

If you connect 8x 10TB-HDDs (250MB/s max), you reach a total peak transfer of 2000MB/s - far away from 7880MB/s the HBA can handle 😉

Edited January 15, 2021 by Zonediver

[Vr2Io]

5 hours ago, alitech said:

Thanks for that. I will have a look at that. Would you recommend 2 of these for my build? Is there is performance boost on 12gb cards. 

This depends on how much bandwidth you need, for example if connect 8 disks, then 6G or 12G HBA would perform same ( calc 2Gbps per disk ), but if connect 10+ disks then you need 12G HBA & Expander product.

 

Read through below post will got more idea

 

 

Edited January 15, 2021 by Vr2Io

[Vr2Io]

8 hours ago, alitech said:

Also, I think you mean I have 16x PCI lanes to play with. Would you say 2 of these cards would be ok at 8x each?

Ryzen haven't GPU ( display ) and you plan 10G NIC, so no PCIe slot for 2 HBA, but you may use 1x slot for display or NIC.

[alitech]

18 minutes ago, Vr2Io said:

Ryzen haven't GPU ( display ) and you plan 10G NIC, so no PCIe slot for 2 HBA, but you may use 1x slot for display or NIC.

Yes I plan to put in a quadro p600 which I eventually will need to do the heavy lifting on plex if needed. 

The following cards will be needed.

 

1x 10GB nic

1x Quadro p600

1x HBA SAS card. I could go for a 12GBPS with 16 ports

[Vr2Io]

5 minutes ago, alitech said:

Yes I plan to put in a quadro p600 which I eventually will need to do the heavy lifting on plex if needed. 

The following cards will be needed.

 

1x 10GB nic

1x Quadro p600

1x HBA SAS card. I could go for a 12GBPS with 16 ports

 

Go to 12G HBA not a bad idea, because future-prove, but expensive,  btw I use two 12G HBA , one for internal 16 disks and one for external 26 disks. Previous 6G product in spare now.

[alitech]

2 minutes ago, Vr2Io said:

 

Go to 12G HBA not a bad idea, because future-prove, but expensive,  btw I use two 12G HBA , one for internal 16 disks and one for external 26 disks. Previous 6G product in spare now.

Can you recommend the one you are using? Which one is it? I just need 16 internal disks at this point 

[Vr2Io]

4 minutes ago, alitech said:

Can you recommend the one you are using? Which one is it? I just need 16 internal disks at this point 

It is a 3U case with backplane expander build-in, only external have standalone expander, the model is Adaptec Aec-82885t

Edited January 15, 2021 by Vr2Io

[alitech]

2 minutes ago, Vr2Io said:

It is a 3U case with backplane expander build-in, only external have standalone expander, the model is Adaptec Aec-82885t

So Adaptec Aec-82885t is the HBA card inside the machine? 

[Vr2Io]

6 minutes ago, alitech said:

So Adaptec Aec-82885t is the HBA card inside the machine? 

No, expander ( work like a network switch ) Aec-82885t were in external enclosure, btw if you found good price of 8 port HBA + expander, then you may use this instead of 16 port HBA, HBA+expander have flexibility, expander would work even no PCIe slot, you can places it anywhere.

Edited January 15, 2021 by Vr2Io

[alitech]

23 hours ago, Vr2Io said:

No, expander ( work like a network switch ) Aec-82885t were in external enclosure, btw if you found good price of 8 port HBA + expander, then you may use this instead of 16 port HBA, HBA+expander have flexibility, expander would work even no PCIe slot, you can places it anywhere.

Thanks for all your help. I have looked at the HBA card that you recommended and it seems to be expensive on its own. The expanders are 4 times the price also. This is going to be an expensive set up. 

 

Here is what I think I need now, please correct me if I am not correct. 

 

Modern HBA card compatible with Unraid with consistent firmware updates

16 ports to start with and option to use expander in the future

6GBPS (12GPBS are too expensive)

External expansion possible

 

can anyone recommend one to me now?

 

I found a few options on ebay but none the wiser. There is a mix of advice in this thread. 

 

Here is what I have found that would fit the profile I think

 

HBA card : https://www.ebay.co.uk/itm/133410484262

 

Expander:  https://www.ebay.co.uk/itm/133196317041

 

There was recommendation here for an actively cooled 9207-8i card. I could not find one with active cooling, but I did find this cooling solution https://www.ebay.co.uk/itm/223516272977

 

Anyway, a little confused as to which cards I need to buy still. 

 

Just need to run 12 to 14 HDDs inside the case. The less PCI slots used the better. 

 

[Vr2Io]

8 minutes ago, alitech said:

Expander:  https://www.ebay.co.uk/itm/133196317041

I use same expander before ( I have three ), no problem and work well. You can use it without PCIe slot, just power it up by mining PCIe adapter would be fine.

 

9240-8i ( IT firmware ) or 9207-8i also common and work well too. Better got 9207-8i because it is PCIe 3.0, but it not means you will got high performance in general.

Edited January 16, 2021 by Vr2Io

[alitech]

1 minute ago, Vr2Io said:

I use this expander before ( I have three ), no problem and work well.

 

9240-8i ( IT firmware ) or 9207-8i also common and work well too. Better got 9207-8i because it is PCIe 3.0, but it not means you will got high performance in general.

Awesome, thank you. 

So basically, i should get 9207-8i with the expander I linked to above and they will work well together.. thats great. 

 

Now another basic question. You said I dont need a PCI slot for the expander and can put it anywhere. I dont think I will have a spare PCI slot for this card, does it not need any power from the PCI slot? 

 

Which cable would I use to connect the expander to the HBA card?

 

Thanks for your patience.

[Vr2Io]

29 minutes ago, alitech said:

I dont think I will have a spare PCI slot for this card

Previous POST have update, you can found those adapter in very cheap price. ( No need connect that USB cable ) . It just use to provide power to the expander. ( SATA , Molex, PCIe power source in either one )

Edited January 16, 2021 by Vr2Io

[Vr2Io]

10 minutes ago, alitech said:

Which cable would I use to connect the expander to the HBA card?

Simple 8087 to 8087 cable in two ( dual link )

Edited January 16, 2021 by Vr2Io

[alitech]

20 minutes ago, Vr2Io said:

I use same expander before ( I have three ), no problem and work well. You can use it without PCIe slot, just power it up by mining PCIe adapter would be fine.

 

9240-8i ( IT firmware ) or 9207-8i also common and work well too. Better got 9207-8i because it is PCIe 3.0, but it not means you will got high performance in general.

Thanks for this, dont you think that the USB cable will restrict the data transfer speeds of this expander?

 

[alitech]

Sorry, my bad, its just for power. I understand now