HOW TO: Configure a NetApp DS4243 Shelf in Unraid


Recommended Posts

57 minutes ago, MMTim said:

How/what should I do for the DS? Should I buy a 2nd UPS for the DS? And how do I make sure it only initiates shutdown AFTER the server has been shut down succesfully?

Or, is it OK if it just looses power without a shutdown after the server itself was shutdown correctly? (and therefore all id need is a bigger one, for which it is ok if it's jsut a dumb one, which outlasts the server's UPS)

 

 

The DS has no concept of 'shutdown' it is either on or off.

 

Therefore I keep my DS on a seperate UPS - so that the Unraid server and its mounts get shut down cleanly before the DS runs out of juice.

 

Link to comment

tldr; Get a large enough single UPS to treat your server as a single entity all on the same UPS. You can use the small UPS to keep your modem, router, and switches running.

 

Keep in mind that if you have the computer and the DS on separate supplies, you must ensure that the DS is always powered up before the OS boots, and powered down after the OS has completed a clean shutdown. How you manage that is up to you, but be mindful of what happens during all plausible scenarios.

 

Normal startup and shutdown

 

All power loss scenarios assume you aren't around to manage things.

 

Power loss short term, power restored in less than a minute

 

Power loss long term, power restored after 10 minutes, subsequent power loss within a minute or 2.

 

Power returns after several hours.

 

I'd advise planning for the array to stay down until you can supervise and manage the startup, there are just too many moving parts and interconnected events to try to coordinate.

  • Like 1
Link to comment

Thanks both!

 

@jonathanm Getting both on one UPS; how would that work? I am not too familiar with UPSes, but I have mine hooked up to a usb port of my unraid server. This is currently working. When the juice is gone, it will, based on my settings, at some point in time initiate a shutdown. However, when I add the DS to this (or any other) UPS, what will that practically change? it will still initiate a shutdown at point X in time, but it will still have no way to initiate a shutdown on the DS, right? So it will jsut keep it powered on, until there's nothing left and then everything will just turn off... Right? Or am I missing something here? if I buy a 'dumb' high capacity 2nd UPS for my DS, of which I know for sure it will last longer than my Unraid UPS, then the result is practically the same right?

 

Aside from that, I do agree on the planning for power recovery/startup! thanks for the tip!!

Link to comment
2 hours ago, MMTim said:

it will jsut keep it powered on, until there's nothing left and then everything will just turn off... Right?

Some UPS's can be configured to kill power to the outputs a short period of time after a shutdown has been commanded. Support for that feature is spotty, you will need to confirm the specific model you are looking at supports that feature properly through apcupsd or NUT.

 

Otherwise, yes, it will continue to run, draining the batteries until the UPS turns off to save itself. That won't hurt the DS, but it will reduce the useful lifespan of the batteries.

 

How often do you have power outages?

Link to comment
  • 1 month later...

I seem to be having issues getting the Netapp to show on my server after the recent 6.9 update. Everything was working great prior to the new update. (I have all the hardware listed on the first post). I updated unraid to the new software, rebooted the server, and my drives didn't show up. I then powered down both the Netapp (switch the power switches), and powered down the server.  I then switched on the Netapp, waited several minutes (tried times between 1 min and 5 min), then powered on my server, and my drives no longer show up.  I have a whole shelf full of data, so i'm not sure what's going on. It was working on the previous update, so I have a feeling the new update may have broken something. 

Edited by kennygunit
  • Like 1
Link to comment
1 hour ago, kennygunit said:

I seem to be having issues getting the Netapp to show on my server after the recent 6.9 update. Everything was working great prior to the new update. (I have all the hardware listed on the first post).

 

Oh no - I know it is not much help but I've just upgraded to 6.9 and use a LSI 9206-16e card and it all just worked and appeared as before.

 

I think you can downgrade to verify (not done it myself though).

Link to comment

@kennygunit

Shelf is a NetApp DS4243 / IOM6.

The controller is a NetApp PMC-Sierra PM8003 SCC 4-Port QSFP PCIe x8.

The 6.8.3 Version has the kernel: pm80xx 0000:27:00.0: pm80xx: driver version 0.1.39.

The 6.9.0 Version stable has the kernel: pm80xx 0000:27:00.0: pm80xx: driver version 0.1.40

The controller didn't find my drives with the new driver.

It's a new driver problem.

Link to comment

I updated unraid to 6.9 and did not experience problems. 

 

Is the shelf/disk showing up on bios?

13 hours ago, kennygunit said:

I seem to be having issues getting the Netapp to show on my server after the recent 6.9 update. Everything was working great prior to the new update. (I have all the hardware listed on the first post). I updated unraid to the new software, rebooted the server, and my drives didn't show up. I then powered down both the Netapp (switch the power switches), and powered down the server.  I then switched on the Netapp, waited several minutes (tried times between 1 min and 5 min), then powered on my server, and my drives no longer show up.  I have a whole shelf full of data, so i'm not sure what's going on. It was working on the previous update, so I have a feeling the new update may have broken something. 

I updated unraid to 6.9 and did not experience problems with my shelf.

 

-Is the shelf/disks showing up on bios?

-what HBA/LSI are you running?

Edited by Tolete
  • Like 1
Link to comment

I did experience the same issue as the others. 

 

Fought with the shelf for a few hours and after rolling back to 6.8.3 everything came back up.

 

I tried swapping the IOM modules around, Moving the HBA card to a different PCI-E slot, replacing the qsfp cable, pulling all drives. Trying drives with and without interposers. Swapped the PSU's as one of mine appear to be dead anyways. None of it was able to restore drives on the new version of Unraid. But the second I rolled unraid back the drives were working again.

 

Given I just setup the disk shelf a couple of days ago I thought the HBA or Shelf may have had something wrong with them. But it looks like Tigerherz may be correct with the driver issue.

 

I imagine there's a way to downgrade the driver in 6.9.0 if others continue to experience the same issues. 

Edited by CrashMedia
  • Like 1
Link to comment
4 hours ago, Tolete said:

I updated unraid to 6.9 and did not experience problems. 

 

Is the shelf/disk showing up on bios?

I updated unraid to 6.9 and did not experience problems with my shelf.

 

-Is the shelf/disks showing up on bios?

-what HBA/LSI are you running?

 

 

I'm going to need to find out the bios info tomorrow since i don't have a screen tied to the server ATM. 

 

As for the HBA/LSI, i'm using the PMC-SIERRA NETAPP PM8003 SCC Quad Port Network Controller 111-00341

 

I rolled my server back to 6.8.3, and it still doesn't show up.  Is there a way to see via logs what driver my PM8003 is running? I'm assuming it's the following under System Devices

 

Quote

IOMMU group 19:[11f8:8001] 0c:00.0 Serial Attached SCSI controller: PMC-Sierra Inc. Device 8001 (rev 05)

IOMMU group 20:[11f8:8001] 0d:00.0 Serial Attached SCSI controller: PMC-Sierra Inc. Device 8001 (rev 05)

 

Edited by kennygunit
Link to comment
1 hour ago, kennygunit said:

 

 

I'm going to need to find out the bios info tomorrow since i don't have a screen tied to the server ATM. 

 

As for the HBA/LSI, i'm using the PMC-SIERRA NETAPP PM8003 SCC Quad Port Network Controller 111-00341

 

I rolled my server back to 6.8.3, and it still doesn't show up.  Is there a way to see via logs what driver my PM8003 is running? I'm assuming it's the following under System Devices

 

 

 

ideally you would want to compare the two drivers on your system from before the update and after. you can take note of the driver version you have now, perform update again and see if the driver ver changes. that will give you some indication if your driver ver reverted back or not.

A more stable solution would be a different HBA/LSI. While back when i was looking into which HBA for the NetApp shelf i came across some incompatibility issues with the actual name brand NetApp HBA controllers.

 

currently i can tell you im on SAS2008 in IT MODE w/

[1000:0072] 03:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

 

  • Like 1
Link to comment
  • 2 weeks later...

And this is why I haven't upgraded to 6.9.0 yet...

 

Q: Anyone use alternate firmware or have a way to even update the firmware on a Netapp X2065A-R6?

 

It works but per another post, two preclears worked on with this card on a DS4246 but then rebuilding both 4TB drives as Parity 2 failed and the drives got red X'd. So I figure I'd try to update firmware perhaps.

 

Preclear ran for 27 hours and completed successfully so it's a bit of a puzzler to me.

 

Running a preclear now using the internal backplane on my T310. So far, 4 hrs in and it looks fine.

 

 

Edited by RealActorRob
Link to comment
  • 1 month later...

Just an update to others who may have similar issues.

 

I bought a server for another storage server that came with 2 H200E dell external HBA's.

 

After flashing them to IT Mode this seems to have resolved the issues. I tried updating to 6.9.2 and the PM8003 still wasn't discovered and I was having a ton of reliability issues with it with the drives dropping out.

 

You will also need a different DAC you will need a SFF-8436 to SFF-8088

Link to comment
22 minutes ago, CrashMedia said:

Just an update to others who may have similar issues.

 

I bought a server for another storage server that came with 2 H200E dell external HBA's.

 

After flashing them to IT Mode this seems to have resolved the issues. I tried updating to 6.9.2 and the PM8003 still wasn't discovered and I was having a ton of reliability issues with it with the drives dropping out.

 

You will also need a different DAC you will need a SFF-8436 to SFF-8088

 

I had the same problem using the PM8003. With V6.9.0 Stable. The PM8003 also used to drop drives occasionally on 6.8.3, but only when unraid was started, requiring drives to be pulled and re-seated to refresh the array. It was rock sold once running.

 

Rather than wait for a fix or remain on 6.8.3, I purchased an LSI 9212-4i4e 6Gb SAS Controller Card set on IT mode, and a QSFP SFF-8436 to Mini SAS SFF-8088 Cable.

 

I have not seen any issues with the LSI controller on 6.9.2.

Link to comment
2 minutes ago, pete69 said:

 

I had the same problem using the PM8003. With V6.9.0 Stable. The PM8003 also used to drop drives occasionally on 6.8.3, but only when unraid was started, requiring drives to be pulled and re-seated to refresh the array. It was rock sold once running.

 

Rather than wait for a fix or remain on 6.8.3, I purchased an LSI 9212-4i4e 6Gb SAS Controller Card set on IT mode, and a QSFP SFF-8436 to Mini SAS SFF-8088 Cable.

 

I have not seen any issues with the LSI controller on 6.9.2.

It looks to be the best option. I got the LSI card for secondary project, but haven't tested it. 

 

I had the H200E's and didn't have any plans for them so I figured I would test it. I'm swapping out all my old 4TB drives from the server and switching to the Diskshelf with 8TB drives, I'm now on disk 2 of 8 and it's been stable so far.

 

The PM8003 was dropping the drives right after the rebuild. So I'd think everything was good, and then I would leave it for an hour and the drive suddenly wasn't attached anymore. The first time I noticed I got half way through the rebuild on the second drive and they both dropped out.

 

Because I am migrating to the Disk shelf I had the old drives in the server still so I was able to just clear the config and change the drive back to the original drive.

 

Hopefully anyone else who may have these issues reads through this and saves themselves some money. 

 

It's a great solution if you get the right HBA.

Link to comment
  • 2 weeks later...
On 4/20/2021 at 3:18 PM, SideSh0wWilly said:

I can confirm that the NetApp PM80xx series cards do not work with the 0.1.40 driver on version 6.9 and above.  I have just setup a new system with a DS4243 and the NetApp card.  Could not see the drives at all.  As soon as I downgraded to 6.8.3 all is back and visible. 

I was wondering what was going wrong..... well that is frustrating oh well. Thanks for confirming my suspicions.

Link to comment
  • 4 weeks later...

I have this same netapp setup and the pcie card shows up with unraid 6.8.3 but only my SAS Dell / Toshiba Drives are being detected. My WD Red Sata drives I get nothing.

 

SCSI Devices

[0:0:0:0]disk Verbatim STORE N GO PMAP /dev/sda 15.4GB

[2:0:10:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdb 3.00TB

[2:0:12:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdc 3.00TB

[2:0:13:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdd 3.00TB

[2:0:14:0]disk TOSHIBA MG03SCA300 DG04 /dev/sde 3.00TB

[2:0:16:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdf 3.00TB

[2:0:17:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdg 3.00TB

Link to comment
On 4/20/2021 at 1:18 PM, SideSh0wWilly said:

I can confirm that the NetApp PM80xx series cards do not work with the 0.1.40 driver on version 6.9 and above.  I have just setup a new system with a DS4243 and the NetApp card.  Could not see the drives at all.  As soon as I downgraded to 6.8.3 all is back and visible. 

See here everyone: https://forums.unraid.net/bug-reports/stable-releases/690-691-692-netapp-pmc-sierra-pm8003-rev5-and-adaptec-6805h-hba-controller-didnt-find-the-hdds-r1300/page/2/?tab=comments#comment-14700

 

It appears there is resolution for PM8XXX in Unraid 6.9.2 after all.

 

Cheers.

Link to comment

What could I be doing wrong or is there an issue with WD Red's and the NetApp PM80xx series cards? 

 

My start up process is,

 

  1. Turn on the two PSU's for the storage self wait 1 min for disks to initialize.
  2. Power on Unraid Server.

 

My Dell SAS drives and my Seagate Sata drives show up every time. But my 6 WD Reds sometimes show up in Devices but most times they do not. 

 

Anyone seen this issue with REDs?

 

Unraid 6.8.2 with PM80xx patch.

 

SCSI Devices

[0:0:0:0]disk Lexar USB Flash Drive 1100 /dev/sda 32.0GB

[2:0:0:0]disk ATA WDC WD40EZRZ-00G 0A80 /dev/sdd 4.00TB

[3:0:0:0]disk ATA Crucial_CT525MX3 R021 /dev/sdb 525GB

[4:0:0:0]disk ATA ADATA SU800 8B /dev/sdc 512GB

[9:0:0:0]disk TOSHIBA MG03SCA300 DG04 /dev/sde 3.00TB

[9:0:1:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdf 3.00TB

[9:0:2:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdg 3.00TB

[9:0:3:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdh 3.00TB

[9:0:4:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdi 3.00TB

[9:0:5:0]disk ATA ST8000DM004-2CX1 0001 /dev/sdj 8.00TB

[9:0:6:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdk 3.00TB

[9:0:7:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdl 3.00TB

[9:0:8:0]disk ATA ST8000DM004-2CX1 0001 /dev/sdm 8.00TB

[9:0:9:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdn 3.00TB

[9:0:10:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdo 3.00TB

[9:0:11:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdp 3.00TB

[9:0:12:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdq 3.00TB

[9:0:13:0]disk TOSHIBA MG03SCA300 DG04 /dev/sdr 3.00TB

 

Edited by sendas
Link to comment
12 minutes ago, sendas said:

What could I be doing wrong or is there an issue with WD Red's and the NetApp PM80xx series cards? 

 

I had similar issues on start up running 6.8.3, with some drives not showing (sometimes reds sometimes others).

 

Usually pulling the drives and reseating or moving drives to other locations got them online.

 

Once online it was rock solid until I needed to restart unraid and go through the process again.

 

I initially thought it might be a backplane issue with the disk shelf, but didn't care too much as I don't need to restart unraid very often.

 

When I upgraded unraid to 6.9.0 none of the drives were recognised due to the kernel issue.

 

Rather than wait for a fix or remain on 6.8.3, I purchased an LSI 9212-4i4e 6Gb SAS Controller Card set on IT mode, and a QSFP SFF-8436 to Mini SAS SFF-8088 Cable.

 

Since then it works fine on 6.9.x and the drives are all online at each restart without any fiddling.

 

Pete.

  • Thanks 1
Link to comment
  • 3 months later...

 

I have moved some existing 8-10TB disks from an old 8 bay USB cabinet (USB stuff disabled) to a NetApp DS4246 with two IOM. In the old cabinet the disks were connected to a LSI SAS 9207-8e card (in IT mode) using two Mini SAS to SATA cables and that has worked very stable for the last year.

 

I connected the NetApp enclosure with a Mini SAS to QSFP cable from 10GTek ( QSFP to MiniSAS DDR Cable, 2 Metre 6.5ft: Amazon.de: Computer & Accessories ). After moving the disks I turned on the NetApp disk shelve and waited a few minutes. All lights are green on the back and the drives are turned on. No faults seem to be reported.

 

- But the disks are not visible neither in the LSI BIOS and therefore neither in UNRAID. Is there anybody that has any hints to what the problem might be or what I can try out?

 

Already tried:

 

  • Changing the QSFP cable.
  • Switching the IOM6 around.

 

 

EDIT: It turned out to be an old firmware on the HBA and upgrading it solved the issues.

 

The upgrade process however was a nightmare since it was a HP H221 variant, so I had to locate an old LSI firmware update program that allowed flashing of HP cards. During the update it came with an error. The error was about missing BIOS support for some DOS call, so had to use the UEFI version. Then it said something about the card being the wrong type - back to Google that kindly told me  HP apparently sold  two different cards under the same name, so my HBA wasn't a 9207-8e, but instead a 9205-8e. After locating that firmware it finally worked.

 

 

Edited by RecycledBits
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.