HOW TO: Configure a NetApp DS4243 Shelf in Unraid


Recommended Posts

So I am thinking about getting another one, but all the cable guides show using 2 controller cards and 2 stacks and cabling them all together a special way., I am assuming for redundancy. I only have one controller, with 2 ports on the controller. This is how I would set it up?

 

first port on Controller to first diskshelf in first IOM in the Square input,

Then from first diskshelf it first IOM in the Round input to the second diskshelf into the first IOM in the square imput

and thats it?

 

or do I also take take

second port on controller to fist diskshelf in second IOM in square imput,

Then from first diskshelf it second IOM in the Round input to the second diskshelf into the second IOM in the square imput

Link to comment
On 12/3/2021 at 7:42 AM, almulder said:

So I am thinking about getting another one, but all the cable guides show using 2 controller cards and 2 stacks and cabling them all together a special way., I am assuming for redundancy. I only have one controller, with 2 ports on the controller. This is how I would set it up?

 

I only have the one shelf currently connected, but had 2 in the past.

 

Since you have 2 ports on your controller, just connect each port to the square SAS port on both your IOMs, that would give you 2 independent SAS paths for each shelf and they would be treated as separate entities by unraid.

 

I think that would be better than trying to daisy chain the shelves which would have no benefit in Unraid.

 

  • Like 1
Link to comment

So I just got my ds2246 in today and got a chance to do some testing,

 

From unraid LSI Controller to my  DS4246 (Square port, top card) to my DS2246 (Round Port Left side) they connect and work great

 

However as you suggested having them each go separately to the Controller rather and daisy chained, I did see a performance increases (DS4246 with 8+ TB Mechanical Drives)  (DS2246 with 500GB+ SSDs)

 

When many drives on DS4246 were accessed along with DS2246 being accessed, there was a drop in speed on both types of drives, When separated (Not daisy chained)  there was a 10%-15% increase in speed. but normal loads I do it made almost 0 difference.

 

But i have the cables so they stay separate until I get another DS4246, then the two of them will be daisy chained while the DS2246 will stay separate so I keep the speeds of the SSDs.

 

Thanks again for the posts.

Link to comment
  • 2 months later...

This tutorial helped a ton, @DannyG

Thank you!

Quick question on wiring from my HBA to the NetApp.

My HBA has two ports and my NetApp has 4 ports (2 in the top thing, 2 in the bottom thing).

I understand that I need to go from my HBA to the "square icon" port on my NetApp

But should I connect two cables from my HBA controller to my single 4246?

Is there a benefit?

Here's an image.

https://imgur.com/a/hKeQ2Ai

Link to comment
  • 1 month later...

Did you figure this out @chansearrington?

I don't have a Netapp myself, but I have been thinking about getting one so I am researching and came across this thread.

I did find this post which maybe helpful to you - https://community.netapp.com/t5/ONTAP-Hardware/Cabled-correctly/td-p/12303

An interesting comment was "Always Circle to Square".

If I imagine a 2 port HBA card as being a circle and a square, I believe you can wire it up like the bottom left component from the diagram in that post (and ignore the bottom right component which is there for redundancy).

Disclaimer - this is just my thoughts. I don't have one myself to verify. Just trying to help.

 

And now a question of my own...

I have seen this item listed on ebay https://www.ebay.com/p/2136573122?iid=294860204489 with a description "NetApp Class 5350 Model 0834 24-Bay SFF Storage Array w/2*Drive Module 2*PSU".

I am not quite sure the exact name of this device. I believe it is something of a 'Disk Expander' as opposed to a 'Disk Shelf'.

Would the Netapp Disk Expansion device work like the Disk Shelf devices when connecting to a HBA?

To me it seems like it would work but I can't find any reference working examples like the DS4243 and DS4246.

The advantage to me is that the 'Disk Expander' device has standard SFF-8088 ports which eliminates some of the hassle I read about the IOM modules. Thanks for any help.

Edited by noski
Link to comment

@noski thanks!

I'm definitely not an expert in this area. 

I decided to go the same route as everyone else and just get a DS4246 from here -> https://www.ebay.com/itm/202404952486

I assume that what you are talking about doing would work the same, but I don't know.

If yours works the same as mine, I only connected 1x cable from the Disk Shelf to my Server. Works fine. And I don't use the second disk manager module at all. Doing so, as I understand it, would make my disksehlf show up as two separate drives vs 1x combined one. 

Still not sure if this is how I should be doing it, lol. 

Link to comment
  • 7 months later...
On 3/30/2022 at 8:02 PM, chansearrington said:

@noski thanks!

I'm definitely not an expert in this area. 

I decided to go the same route as everyone else and just get a DS4246 from here -> https://www.ebay.com/itm/202404952486

I assume that what you are talking about doing would work the same, but I don't know.

If yours works the same as mine, I only connected 1x cable from the Disk Shelf to my Server. Works fine. And I don't use the second disk manager module at all. Doing so, as I understand it, would make my disksehlf show up as two separate drives vs 1x combined one. 

Still not sure if this is how I should be doing it, lol. 

So I ran all of the cables... It's for redundancy. I started with a 4243 and upgraded the modules (for rather cheap later on) to 4246 and noticed a big difference in my speeds... Because I had the cables daisy changed I was able to pull 1 module at a time and not loose connection to any of my hard drives on my disk shelf.

On 3/6/2020 at 8:11 AM, DannyG said:

Parts used in this Build

  • Netapp DS4243 Shelf (no controller)
  • QUAD SAS Card: PM8003 
  •  SAS Cable QSFP to QSFP

    If you have a different SAS card, I'm sure it will still work. This is the card that I was able to source.

    netapp.jpg.1a317630743234e050bccecc3cb538fa.jpg
    2020-03-06_08-57-49.png.339a06c78820c3574e7667cf8300fd74.png
    2020-03-06_08-57-33.thumb.png.e84cd672db842b462d72bf2d4f6eea78.png

     
  • There's no Power button on these shelves, they just turn on when you give them power. from my testing, you only need 1 power Supply to be on and all 24 drives will still work. the other power supplies are only to be used for redundancy (ideally from different power sources) and is over kill in a home lab. 
  • EDIT: I was getting "random" drives disconnecting form my array, to re-add the drives, you had to rebuild the array which took a long time and put my data at risk. In the logs it was found that my drives would go missing each time my UPS did a self check. the fix was to use 2 power supplies, I put them on their own UPS.

 

 

Here are some power stats for fun:

Empty Shelf with all 4 Power Supplies ON. (207 Watts)
2020-03-06_09-20-22.thumb.png.ca6d66a0a70d2f5d518e8562b90e182c.png
 

  • The Full Shelf, with 24 Disks on takes:
  • 305 Watts with 1 PowerSupply on.
  • 323 Watts with 2.
  • 348 Watts with 3, an
  • 378 Watts with all 4.


    STEP 1) Install your SAS card in your unraid server. leave the server off for now.

    STEP 2) - With the Netapp turned OFF; Physically plug the SAS Cable in the port with the "Square icon" on the top controller & the other end of the cable to your SAS Card in your unraid server.
    - Turn on a Power Supply.
    - Give it a good minute to fully initialize.
    2020-03-06_09-18-14.thumb.png.8e0fefeeca176c6fd38a4149cdf50b69.png

 

STEP 3) Power on your Unraid Server & login to its interface.
 - If you're lucky. you're Done. The drives are already showing up in the MAIN menu, and you can start assigning your Disks.

if you don't see your Drives... Keep reading.
 

 

STEP 4)  Go to TOOLS > System devices.

- Do you see your disks? (yes = good) (No? Go to Step 5)
- Your Disks don't have a "Size"? (Let's fix this)

2020-03-06_09-53-28.png.2fa26858fa5b23fc4b29dd166f06d4c2.png

 

STEP 5) Sanity Check - Check the Logs

- Go to: TOOLS > SYSTEM LOGS

- CTRL-F (find), search for "Netapp"
- Finding "Netapp" confirms that your card is working. (Great!)
- Some Netapp drives are formatted with 520b sectors instead of the needed 512b. we need to reformat them for Unraid to see them properly.
2020-03-06_09-49-03.thumb.png.47e15c9e532427248ba8683132ee6391.png

 

 

STEP 6)
 

- Click the Terminal icon

- Run this command to download the formatting utility: 

wget http://slackware.cs.utah.edu/pub/slackware/slackware64-14.1/slackware64/l/sg3_utils-1.36-x86_64-1.txz

- Run this after the download to install sg3_utils

upgradepkg --install-new sg3_utils-1.36-x86_64-1.txz

- Use this command to show SAS HDD's

sg_scan -i

this command to format. PLEASE READ /dev/XXX should be the HDD in the NetApp that needs to be formatted. (AKA double check before you format the wrong drive) 

sg_format --format --size=512 -v /dev/XXX

 

From my testing, I was able to open 24 terminal windows and format all 24 drives at the same time. This might have slowed things down a bit.. not sure.. but I just went to bed.
2020-03-06_10-32-07.thumb.png.8044a69d1fdeb6b5cdb47e5e0969cf3e.png

STEP 7) 

The following morning, I woke up, went to MAIN, but still didn't see my disks :( 

I REBOOTED the System.... and the 24 Disks appeared. :) 

Checked the Logs... no more errors... and I confirmed it was formatted in 512 sectors now. yes!


2020-03-06_09-53-28.png.c001b4b3c08b3d181776762587f19c8a.png

 

 

 

I just picked up three 4Tb drives and had to reformat them. I am on the most recent unraid build and when I tried to run the wget I got an error about there not being the file. So that might need to be updated.. I ended up figuring out which drive it was and ran the sg_format command. I have 2 real questions. Once ran do I need to keep the terminal open until it's done or is it a situation that can just be closed and it will finish when it finishes?

Also is there a way to speed up the process? Looking to get more and bigger drives as time keeps moving along

Edited by UtahDeLorean
Link to comment
1 hour ago, UtahDeLorean said:

Is there a way to speed up the process? Looking to get more and bigger drives as time keeps moving along

 

If you are re-cycling Netapp or some other modified sector size drives, then you will need the same process each time.

 

If you are buying new drives, the sector size should be fine as Unraid handles most standard sector sizes out of the box.

  • Like 1
Link to comment
  • 2 weeks later...

thanks for the thread,  a year ago I also bought a netapp 4246 IOM6 with 24x3TB SATA (WD3000FYYZ+MG03ACA300+HUA723030ALA640 from Netapp) + interposer and the PM8003 QSFP with driver patch for 6.9.2 . I tested different SATA HDDs. More than 6TB will not work with the interposer. WD60EFZX, WD60EFAX, ST4000VN008 run without problems with interposer. Interposer is for SATA to SAS two-channel connection. ST10000VN0008, MG07ACA12TE run without an interposer. Mixed w/o and with interposer no problem.

512 sectors or 512e / 4096 is no problem. But 4k doesn't work

Edited by tagchen
Link to comment
  • 2 months later...

@DannyG I had a situation where I was formatting bigger drives that I got. I got the 1st drive formatted and swapped out a drive that was tossing out error's.

I was on the 2nd drive and it was running over night and when I check it the window was closed.

I put in the 3rd drive and the window crashed/closed while I was watching it... When I tried to run the format line again I get this

image.png.a31b5a34bc985c02e3480e78b2345095.png

 

Does this mean that the format is still going or did I do something wrong? I have only complete 1 out of the 5 drives that I picked up. (I already did this on 4 other drives I purchased before with out any problems like this)

 

Link to comment
  • 2 weeks later...
On 2/17/2022 at 6:41 PM, chansearrington said:

This tutorial helped a ton, @DannyG

Thank you!

Quick question on wiring from my HBA to the NetApp.

My HBA has two ports and my NetApp has 4 ports (2 in the top thing, 2 in the bottom thing).

I understand that I need to go from my HBA to the "square icon" port on my NetApp

But should I connect two cables from my HBA controller to my single 4246?

Is there a benefit?

Here's an image.

https://imgur.com/a/hKeQ2Ai

Hey @chansearrington, There's no benefit with unraid.
Keep in mind that a real Netapp, is more then just this shelf. In a production environment it's 2 controllers + a shelf (or more).
NetApps do really fancy things that are needed to mission critical applications.
My NetApp training days are far behind me now, but I believe the extra ports at the bottom is for something called "High Availability multipath" (don't quote me) - This is something proprietary to their controllers + software. it's not something that will work out of the box, and probably not worth the effort required to get it to work.

Link to comment
On 2/3/2023 at 3:58 PM, UtahDeLorean said:

@DannyG I had a situation where I was formatting bigger drives that I got. I got the 1st drive formatted and swapped out a drive that was tossing out error's.

I was on the 2nd drive and it was running over night and when I check it the window was closed.

I put in the 3rd drive and the window crashed/closed while I was watching it... When I tried to run the format line again I get this

image.png.a31b5a34bc985c02e3480e78b2345095.png

 

Does this mean that the format is still going or did I do something wrong? I have only complete 1 out of the 5 drives that I picked up. (I already did this on 4 other drives I purchased before with out any problems like this)

 

Did you ever get this sorted out? I have one drive that is throwing this same error.

Link to comment
  • 3 weeks later...
  • 4 months later...
  • 1 month later...
  • 3 months later...

FYI: I got 11 10tb drives (4 Seagate and 7 WD sas models) gifted to me from a fellow IT guy that I didn't know that they were from a netapp appliance, and after connecting them up and running those commands I was able to start formatting them. 

My setup is a Chenbro-Cleversafe-Slicestor-1440 (48 Bay SAS) with to a LSI SAS2008 Fusion-MPT SAS2 [Falcon] (rev 03) card in IT mode in my Epyc based server. I didn't have the drives in the Netapp enclosure but instead in an expander on one side (the other 24 bay side has the 17 10TB WD WD100EMAZ drives connected). At first when I connected them I had them mixed on the expanders and didn't realize until I realized they might be a) not able to mix right, and b)oddball formatted/firmware enterprise drives, and found them to be option b, they were Netapp drives. After putting them on their own side apart from the sata drives, I started looking for information on how to flash their firmware because I remembered that they might have custom firmware, but then I just tried the commands shown here and it worked! They are formatting without any other steps needed!  Thanks for this writeup! It did the trick!

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.