50 minutes ago, mokopoko said:

I just got this one for testing:

https://www.amazon.de/-/en/dp/B0BDMSZR41?psc=1&smid=A3JWKAKR8XB7XF&ref_=chk_typ_imgToDp

This uses a port multiplier and will reduce available throughput.

JMB585 is a 5 ports chip and you are using it for 8.

Thanks for this thread @JorgeB as it has been extremely helpful!

 

I am just starting out my first Unraid build and wanted to confirm a few choices as I find myself getting confused sometimes regarding lanes and throughput etc. 

 

My motherboard is an Asrock Z790 PRO RS WIFI which has 4 m.2 slots.  I would like to add 10 SATA ports.  Based on reading this thread, I think I should be good with 2x JMB585-based cards (avoiding port multipliers).  Here is the current plan for my m.2 slots:

• M.2 Slot M2_1: 1TB NVMe SSD (WD Black SN850X) - zfs mirrored cache

• M.2 Slot M2_2: JMB585 5-port SATA expander (IOCREST SATA Expander)

• M.2 Slot M2_3: 1TB NVMe SSD (WD Black SN850X)  - zfs mirrored cache

• M.2 Slot M2_4: JMB585 5-port SATA expander (IOCREST SATA Expander)

 

My questions:

• Thoughts on the JMB585 card selected? It looks to match the criteria of the original recommendations, but it does mention "Can support port Multiplier that is not in a RAID configuration" in the description but not sure what that means since it is only 5 ports and is JMB585. Does that sound like a red flag?
• Thoughts on my m.2 slot plan? Would I have any bottlenecks with this setup?
• None of my PCIe slots are in use today, but I would like to maybe add a mid-range GPU down the road in the PCIe5x16 slot.  Would my m.2 and PCIe slots be competing with each other and hurting performance?

 

Thanks again in advance to any one who might be able to help!

Edited September 21, 20241 yr by Galooch

11 hours ago, Galooch said:

Thoughts on the JMB585 card selected?

Can't speak for that specific model, but JMB585 works very well generally, if you want one extra port you can also use ASM1166 based controllers. 

 

11 hours ago, Galooch said:

Thoughts on my m.2 slot plan? Would I have any bottlenecks with this setup?

No bottlenecks if you are going to use HDDs with the controllers. 

 

11 hours ago, Galooch said:

None of my PCIe slots are in use today, but I would like to maybe add a mid-range GPU down the road in the PCIe5x16 slot. 

That would be a CPU slot, so the PCIe lanes are dedicated. 

Perfect! Thank you again @JorgeB.   Thanks for the ASM116 reminder as I do plan on using 6 of the SATA ports to start giving me 4 slots for future expansion. However, the ASM116 will give me those 6 so I only need one for now leaving one of my nvme slots open if/when I need it.

 

One last question on potential expansion...you mentioned HDDs but I my final 4 slots in the future will be SSDs.  When that time comes, I assume I could simply plug those SSDs directly into my MB SATA ports and move 4 HDDs to the ASM1166s so that those two controllers will only be handling HDDs.  Is that the best way to handle that and avoid bottlenecks?

18 hours ago, Galooch said:

Is that the best way to handle that and avoid bottlenecks?

Yep.

This thread has been extremely useful - I never even knew this was a thing with expansion cards/ controllers and Unraid, it definitely needs to be more prominent to users with a warning as they add drives, even when adding them for the first time!!!

 

I suspect me using a incompatible expansion card/ controller was, at least partly, to blame for a cache drive failure (albeit I was messing around a bit).  I was receiving CRC errors and warnings on my SSD cache which I had plugged into my controller which had a multiplier - but I didn't really comprehend or think it was serious until I tried to mass write to my cache one day and boom, cache drive dead. 

 

I purchased this ASM1166 MZHOU product from Amazon: https://www.amazon.co.uk/dp/B08F56WKW7 for £35.

 

It has previously been mentioned and linked in this thread. Thank you all for commenting on it, it gave me enough confidence to purchase it! 

 

MZHOU PCIE SATA Card 6 Ports 4X PCIe SATA Expansion Card - 6 Gbps SATA 3.0 PCI Express Card - PCIe to SATA Controller Expansion Card with 6 SATA Cables and Low Profile Bracket

 

I wanted to write and confirm that it already came installed with firmware 221118-003E-00, which at time of writing seems to be the latest FW for ASM1166.

 

I checked it following the guide posted previously by @PhilBarker here (thank you!), which directed me to find the ROM checker and installer here (which contains outdated FW, but I used it to scan for it and check which FW mine had), the download you need is called "Fix compatibility issue and support Intel 600 series chipset motherboard" from the Sliverstone ECS06 site.

 

Run this on your normal windows PC with the controller installed and you can scan for the expansion card. I had to plug it in and out a couple of times before it was recognised, (it eventually worked when I did this as my PC was on, a no-no I know). 

 

Plugged into my Unraid server Mobo, and works fine!  Thank you all for your posts and help - I hope my post also contributes and helps people that come after me - it took me a long time to read this thread and consolidate my understanding of everything!

 

@Gegga - I saw your post from May this year, FYI it looks like you flashed it with FW 11050000.ROM, which is a fair few years out of date just as a FYI. To you and everyone else who is searching for the FW file 221118-003E-00, or to check the latest conversation on these chips and FW; I found a thread here which seems to be alive and kicking of users uploading FW versions and commenting on them as they learn, (possibly posted very early on during this thread but can't remember): https://winraid.level1techs.com/t/latest-firmware-for-asm1064-1166-sata-controllers/98543/50.

It appears you could get the latest rom file from the above winraid thread, and then replace the rom file to flash it with the FW of your choice with the tool from Silverstone. Just an observation for you, if you, or anyone in the future was interested.

 

Thanks also to @JorgeB for his persistent replies, expertise and commitment to the community. A true Unraid legend.

 

Rom checker/ Installer I used (note FW is out of date):ECS06 firmware for Intel 600series chipset.rar

Hey @Zee-Unraid, thanks for your post. I just stumbled upon the ASM1166 as well

 

Have you by chance checked the c-states? I put a lot of time into getting my server to C10, so I'm wondering if this card card would undo my effort

 

I've seen @PhilBarker 's blog post, but there he used a PCIx1 card. Not sure if that could make a difference, but I thought I'd ask anyways ^^' 

Edited October 18, 20241 yr by OddMagnet

Hello,

 I've got my hands 3 slightly used 16TB Seagate Exos x18 drives with a SAS interface. Unfortunately, I do not have any controller for these drives. And here's my question: 

 

Does a controller like the LSI 9207 work with drives of this capacity and with SAS connector?

 And the second question: 

If so, can I connect these drives via a SATA to SAS adapter?

 

Here is a link to the auction I am considering:

https://www.ebay.co.uk/itm/226415675129?chn=ps&_ul=GB&google_free_listing_action=view_item

 

 

From what I read, apparently there is no capacity limit on these cards... only speed, I can only hope that  
 the card will work with drives on the SAS interface. I ordered it so I guess I'll find out

Edited November 2, 20241 yr by Dragonovx

17 hours ago, Dragonovx said:

apparently there is no capacity limit on these cards

That is correct.

 

 

On 10/18/2024 at 6:51 AM, OddMagnet said:

Hey @Zee-Unraid, thanks for your post. I just stumbled upon the ASM1166 as well

 

Have you by chance checked the c-states? I put a lot of time into getting my server to C10, so I'm wondering if this card card would undo my effort

 

I've seen @PhilBarker 's blog post, but there he used a PCIx1 card. Not sure if that could make a difference, but I thought I'd ask anyways ^^' 

 

Glad you found it useful.

 

I personally haven't delved into c-states to be honest with you. Would be interested to hear of your findings in regards to controllers, (and thus if it's something I should be considering/ getting into)! Feel free to DM me to update me, if you feel it would not be relevant to the controller topic of this thread. 

22 hours ago, Zee-Unraid said:

 

Glad you found it useful.

 

I personally haven't delved into c-states to be honest with you. Would be interested to hear of your findings in regards to controllers, (and thus if it's something I should be considering/ getting into)! Feel free to DM me to update me, if you feel it would not be relevant to the controller topic of this thread. 

 

 

Any chance you could do a measurement on the board for C-States?
I am quite interested if it is entering C10 state, so it would be a great controller to purchase.

 

 

Hello guys can I take the takeaways from this Thread (update Firmware and ASMedia1166) for NVME Slots too?

ie.: one of those: NVME to SATA

Or are there other problems?

 

 

Would you rather take an NVME to Sata or PCIe to SATA?

My Board only has one PCIe and one NVME and I am thinking about taking the Controller for the NVME SLot so that I can later add an PCIE to two NVME Cards Adapter.

Would that be an good idea?

Well, based on this thread, I started looking for a NVMe>SATA adaptor using a ASM1166.

I found several on Amazon UK, including this one for just £15! (They're usually about £30)

I figured that at that price I could afford to be wrong , and I'm happy to report, that even this really cheap version works fine.

Currently I'm only running one drive on it just to be sure, but it's running at full speed.

 

On 11/4/2024 at 7:04 PM, Torlew said:

Hello guys can I take the takeaways from this Thread (update Firmware and ASMedia1166) for NVME Slots too?

ie.: one of those: NVME to SATA

 

Yep, that's identical to the one I have.

 

I used the firmware I found in this post.

I'm in need of additional 3 x SATA ports on top of the 4 x SATA ports on motherboard, I did some reading on this thread and found the PCIe 3.0 x1 ASM1064 4 ports seems to be a good option.

 

https://www.amazon.com/Port-SATA-Expansion-Profile-Bracket/dp/B08J2PK1HF?th=1

 

Can comment/advise if this will work well without much compromise on the bandwidth, anyway its just mechanical 7200rpm drive connected to these SATA ports.

2 hours ago, uniqf said:

Can comment/advise if this will work well without much compromise on the bandwidth, anyway its just mechanical 7200rpm drive connected to these SATA ports.

It's fine with HDDs.

On 11/4/2024 at 12:47 PM, autox said:

 

 

Any chance you could do a measurement on the board for C-States?
I am quite interested if it is entering C10 state, so it would be a great controller to purchase.

 

 

 

I've never ventured into that area, if it's relatively simple to do and you can point me to a guide of some sort; I'll give it a go for you.

On 12/6/2024 at 1:20 AM, Zee-Unraid said:

 

I've never ventured into that area, if it's relatively simple to do and you can point me to a guide of some sort; I'll give it a go for you.

@Zee-Unraid
Yes sure, there is a quite good guide here:

Reduce Power Consumption with Unraid

 

Its mainly using the powertop command by having an measurement socket to check what your power consumption is

 

Thanks in advance!

On 11/3/2024 at 3:26 PM, Zee-Unraid said:

 

Glad you found it useful.

 

I personally haven't delved into c-states to be honest with you. Would be interested to hear of your findings in regards to controllers, (and thus if it's something I should be considering/ getting into)! Feel free to DM me to update me, if you feel it would not be relevant to the controller topic of this thread. 

 

Hey @Zee-Unraid, sorry for the late reply, haven't checked the forums in some times.

 

I actually decided on an Inspur LSI 9207-8i, so I won't be able to share my findings on the ASM1166.

 

Might post my findings on C-States with the 9207-8i once it arrives

Question on controllers, I have a LSI IT Mode SAS9300-16i  and it is attached to a SAS3 Epxander on a Supermicro 846 chassis. I am wondering, would I be better off running multiple 9300-8i cards to run 24 drives. My motherboard is an MSI PRO B760-P with an i5 14600k.

On 4/2/2024 at 8:53 AM, Wody said:

Since heat has to do with how much power something uses, I did some math. The 9300-8i uses 13w, so 1,6w per port (rounded). They made a more efficient version of it, the 9305-16i, which uses 1,01w pet port.

The 9400-8i uses 1,25w per port, which seems a lot more, but the 9400-16i only uses 0,75w.

The 9500-8i also uses 0,75w, but the 9500-16i uses 0,56w. The 9600-16i uses more again, 1,06w but that's a sas4 (22.5G) card (also drops support for anything below 6Gb/s so old sata 1 & 2 drives won't work. Sata 3 drives will though)

In my experience, you'll need some airflow, but if your drives aren't active 24/7, and in server cases with a fan-wall, you don't really need any extra fans or such, not even with a 9300-8i. (I'm currently using a 9500-16i and a 9600-24i but I also have a 9300-8i I use with a SAS expander and a 9400-16i)

So, you'll probably want a 9400 or a 9500 card. There is an issue with the 9500 and 9600 though, they use 'new' sff-8654 (slimsas 8i) connectors, and cables for that to sff-8643 work fine (if you have HDD cables, they also have PCIe cables which won't work), but as far as I can tell, all cables to sff-8087 are wired for PCIe even if they specify they are for HDDs (at least the ones I tried).

Thanks for the info.  I ended up not doing anything yet.  But I am using a fractal Define Meshify 2 XL with 7 fans not a server wall but it's always got A/C running.

8 hours ago, crowdx42 said:

I am wondering, would I be better off running multiple 9300-8i cards to run 24 drives.

If the backpane has an expander, you cannot use multiple HBAs with it for better performance, you could for redundancy, but that's not supported by Unraid, and you should still get >200MB/s with 24 devices and a single HBA with two ports connected, so still good.

So I am expecting to receive a 9500 8i tomorrow, I have a single cable which I intend to connect the HBA to the SAS3 backplane expander. Will this work or should I get a cable which has 𝐒𝐅𝐅-𝟖𝟔𝟓𝟒 to two 𝐒𝐅𝐅-𝟖𝟔𝟒𝟑? 24 spinning disks, now SSDs.

5 hours ago, crowdx42 said:

Will this work or should I get a cable which has 𝐒𝐅𝐅-𝟖𝟔𝟓𝟒 to two 𝐒𝐅𝐅-𝟖𝟔𝟒𝟑? 24 spinning disks, now SSDs.

It should work with all devices, assuming there's an expander in the backplane, dual cables will double the bandwidth. 

15 hours ago, JorgeB said:

It should work with all devices, assuming there's an expander in the backplane, dual cables will double the bandwidth. 

Will it make a difference if the spinning drives are already a bottleneck? I get around 185mbs on a parity check with my current 9300 with 4 cables attached to the expander backplane.

 

9 hours ago, crowdx42 said:

9300 with 4 cables attached to the expander backplane.

With a SAS expander you can use one or two wide connections, more than two cables won't make any difference. 

 

A single wide connection should be good for around 4500MB/s total, so if the disks max speed is 185MB/s it should be enough.