almulder Posted December 2, 2021 Share Posted December 2, 2021 So I am thinking about getting another one, but all the cable guides show using 2 controller cards and 2 stacks and cabling them all together a special way., I am assuming for redundancy. I only have one controller, with 2 ports on the controller. This is how I would set it up? first port on Controller to first diskshelf in first IOM in the Square input, Then from first diskshelf it first IOM in the Round input to the second diskshelf into the first IOM in the square imput and thats it? or do I also take take second port on controller to fist diskshelf in second IOM in square imput, Then from first diskshelf it second IOM in the Round input to the second diskshelf into the second IOM in the square imput Quote Link to comment
pete69 Posted December 5, 2021 Share Posted December 5, 2021 On 12/3/2021 at 7:42 AM, almulder said: So I am thinking about getting another one, but all the cable guides show using 2 controller cards and 2 stacks and cabling them all together a special way., I am assuming for redundancy. I only have one controller, with 2 ports on the controller. This is how I would set it up? I only have the one shelf currently connected, but had 2 in the past. Since you have 2 ports on your controller, just connect each port to the square SAS port on both your IOMs, that would give you 2 independent SAS paths for each shelf and they would be treated as separate entities by unraid. I think that would be better than trying to daisy chain the shelves which would have no benefit in Unraid. 1 Quote Link to comment
almulder Posted December 10, 2021 Share Posted December 10, 2021 So I just got my ds2246 in today and got a chance to do some testing, From unraid LSI Controller to my DS4246 (Square port, top card) to my DS2246 (Round Port Left side) they connect and work great However as you suggested having them each go separately to the Controller rather and daisy chained, I did see a performance increases (DS4246 with 8+ TB Mechanical Drives) (DS2246 with 500GB+ SSDs) When many drives on DS4246 were accessed along with DS2246 being accessed, there was a drop in speed on both types of drives, When separated (Not daisy chained) there was a 10%-15% increase in speed. but normal loads I do it made almost 0 difference. But i have the cables so they stay separate until I get another DS4246, then the two of them will be daisy chained while the DS2246 will stay separate so I keep the speeds of the SSDs. Thanks again for the posts. Quote Link to comment
chansearrington Posted February 17, 2022 Share Posted February 17, 2022 This tutorial helped a ton, @DannyG Thank you! Quick question on wiring from my HBA to the NetApp. My HBA has two ports and my NetApp has 4 ports (2 in the top thing, 2 in the bottom thing). I understand that I need to go from my HBA to the "square icon" port on my NetApp But should I connect two cables from my HBA controller to my single 4246? Is there a benefit? Here's an image. https://imgur.com/a/hKeQ2Ai Quote Link to comment
noski Posted March 28, 2022 Share Posted March 28, 2022 (edited) Did you figure this out @chansearrington? I don't have a Netapp myself, but I have been thinking about getting one so I am researching and came across this thread. I did find this post which maybe helpful to you - https://community.netapp.com/t5/ONTAP-Hardware/Cabled-correctly/td-p/12303 An interesting comment was "Always Circle to Square". If I imagine a 2 port HBA card as being a circle and a square, I believe you can wire it up like the bottom left component from the diagram in that post (and ignore the bottom right component which is there for redundancy). Disclaimer - this is just my thoughts. I don't have one myself to verify. Just trying to help. And now a question of my own... I have seen this item listed on ebay https://www.ebay.com/p/2136573122?iid=294860204489 with a description "NetApp Class 5350 Model 0834 24-Bay SFF Storage Array w/2*Drive Module 2*PSU". I am not quite sure the exact name of this device. I believe it is something of a 'Disk Expander' as opposed to a 'Disk Shelf'. Would the Netapp Disk Expansion device work like the Disk Shelf devices when connecting to a HBA? To me it seems like it would work but I can't find any reference working examples like the DS4243 and DS4246. The advantage to me is that the 'Disk Expander' device has standard SFF-8088 ports which eliminates some of the hassle I read about the IOM modules. Thanks for any help. Edited March 28, 2022 by noski Quote Link to comment
chansearrington Posted March 31, 2022 Share Posted March 31, 2022 @noski thanks! I'm definitely not an expert in this area. I decided to go the same route as everyone else and just get a DS4246 from here -> https://www.ebay.com/itm/202404952486 I assume that what you are talking about doing would work the same, but I don't know. If yours works the same as mine, I only connected 1x cable from the Disk Shelf to my Server. Works fine. And I don't use the second disk manager module at all. Doing so, as I understand it, would make my disksehlf show up as two separate drives vs 1x combined one. Still not sure if this is how I should be doing it, lol. Quote Link to comment
UtahDeLorean Posted November 22, 2022 Share Posted November 22, 2022 (edited) On 3/30/2022 at 8:02 PM, chansearrington said: @noski thanks! I'm definitely not an expert in this area. I decided to go the same route as everyone else and just get a DS4246 from here -> https://www.ebay.com/itm/202404952486 I assume that what you are talking about doing would work the same, but I don't know. If yours works the same as mine, I only connected 1x cable from the Disk Shelf to my Server. Works fine. And I don't use the second disk manager module at all. Doing so, as I understand it, would make my disksehlf show up as two separate drives vs 1x combined one. Still not sure if this is how I should be doing it, lol. So I ran all of the cables... It's for redundancy. I started with a 4243 and upgraded the modules (for rather cheap later on) to 4246 and noticed a big difference in my speeds... Because I had the cables daisy changed I was able to pull 1 module at a time and not loose connection to any of my hard drives on my disk shelf. On 3/6/2020 at 8:11 AM, DannyG said: Parts used in this Build Netapp DS4243 Shelf (no controller) QUAD SAS Card: PM8003 SAS Cable QSFP to QSFP If you have a different SAS card, I'm sure it will still work. This is the card that I was able to source. There's no Power button on these shelves, they just turn on when you give them power. from my testing, you only need 1 power Supply to be on and all 24 drives will still work. the other power supplies are only to be used for redundancy (ideally from different power sources) and is over kill in a home lab. EDIT: I was getting "random" drives disconnecting form my array, to re-add the drives, you had to rebuild the array which took a long time and put my data at risk. In the logs it was found that my drives would go missing each time my UPS did a self check. the fix was to use 2 power supplies, I put them on their own UPS. Here are some power stats for fun: Empty Shelf with all 4 Power Supplies ON. (207 Watts) The Full Shelf, with 24 Disks on takes: 305 Watts with 1 PowerSupply on. 323 Watts with 2. 348 Watts with 3, an 378 Watts with all 4. STEP 1) Install your SAS card in your unraid server. leave the server off for now. STEP 2) - With the Netapp turned OFF; Physically plug the SAS Cable in the port with the "Square icon" on the top controller & the other end of the cable to your SAS Card in your unraid server. - Turn on a Power Supply. - Give it a good minute to fully initialize. STEP 3) Power on your Unraid Server & login to its interface. - If you're lucky. you're Done. The drives are already showing up in the MAIN menu, and you can start assigning your Disks. if you don't see your Drives... Keep reading. STEP 4) Go to TOOLS > System devices. - Do you see your disks? (yes = good) (No? Go to Step 5) - Your Disks don't have a "Size"? (Let's fix this) STEP 5) Sanity Check - Check the Logs - Go to: TOOLS > SYSTEM LOGS - CTRL-F (find), search for "Netapp" - Finding "Netapp" confirms that your card is working. (Great!) - Some Netapp drives are formatted with 520b sectors instead of the needed 512b. we need to reformat them for Unraid to see them properly. STEP 6) - Click the Terminal icon - Run this command to download the formatting utility: wget http://slackware.cs.utah.edu/pub/slackware/slackware64-14.1/slackware64/l/sg3_utils-1.36-x86_64-1.txz - Run this after the download to install sg3_utils upgradepkg --install-new sg3_utils-1.36-x86_64-1.txz - Use this command to show SAS HDD's sg_scan -i this command to format. PLEASE READ /dev/XXX should be the HDD in the NetApp that needs to be formatted. (AKA double check before you format the wrong drive) sg_format --format --size=512 -v /dev/XXX From my testing, I was able to open 24 terminal windows and format all 24 drives at the same time. This might have slowed things down a bit.. not sure.. but I just went to bed. STEP 7) The following morning, I woke up, went to MAIN, but still didn't see my disks I REBOOTED the System.... and the 24 Disks appeared. Checked the Logs... no more errors... and I confirmed it was formatted in 512 sectors now. yes! I just picked up three 4Tb drives and had to reformat them. I am on the most recent unraid build and when I tried to run the wget I got an error about there not being the file. So that might need to be updated.. I ended up figuring out which drive it was and ran the sg_format command. I have 2 real questions. Once ran do I need to keep the terminal open until it's done or is it a situation that can just be closed and it will finish when it finishes? Also is there a way to speed up the process? Looking to get more and bigger drives as time keeps moving along Edited November 22, 2022 by UtahDeLorean Quote Link to comment
pete69 Posted November 22, 2022 Share Posted November 22, 2022 1 hour ago, UtahDeLorean said: Is there a way to speed up the process? Looking to get more and bigger drives as time keeps moving along If you are re-cycling Netapp or some other modified sector size drives, then you will need the same process each time. If you are buying new drives, the sector size should be fine as Unraid handles most standard sector sizes out of the box. 1 Quote Link to comment
tagchen Posted December 3, 2022 Share Posted December 3, 2022 (edited) thanks for the thread, a year ago I also bought a netapp 4246 IOM6 with 24x3TB SATA (WD3000FYYZ+MG03ACA300+HUA723030ALA640 from Netapp) + interposer and the PM8003 QSFP with driver patch for 6.9.2 . I tested different SATA HDDs. More than 6TB will not work with the interposer. WD60EFZX, WD60EFAX, ST4000VN008 run without problems with interposer. Interposer is for SATA to SAS two-channel connection. ST10000VN0008, MG07ACA12TE run without an interposer. Mixed w/o and with interposer no problem. 512 sectors or 512e / 4096 is no problem. But 4k doesn't work Edited December 3, 2022 by tagchen Quote Link to comment
UtahDeLorean Posted February 3, 2023 Share Posted February 3, 2023 @DannyG I had a situation where I was formatting bigger drives that I got. I got the 1st drive formatted and swapped out a drive that was tossing out error's. I was on the 2nd drive and it was running over night and when I check it the window was closed. I put in the 3rd drive and the window crashed/closed while I was watching it... When I tried to run the format line again I get this Does this mean that the format is still going or did I do something wrong? I have only complete 1 out of the 5 drives that I picked up. (I already did this on 4 other drives I purchased before with out any problems like this) Quote Link to comment
UtahDeLorean Posted February 4, 2023 Share Posted February 4, 2023 So I just noticed that it looks like it's still working in the background and see the progress of 13.99% I will check on it in the morning Quote Link to comment
DannyG Posted February 15, 2023 Author Share Posted February 15, 2023 On 2/17/2022 at 6:41 PM, chansearrington said: This tutorial helped a ton, @DannyG Thank you! Quick question on wiring from my HBA to the NetApp. My HBA has two ports and my NetApp has 4 ports (2 in the top thing, 2 in the bottom thing). I understand that I need to go from my HBA to the "square icon" port on my NetApp But should I connect two cables from my HBA controller to my single 4246? Is there a benefit? Here's an image. https://imgur.com/a/hKeQ2Ai Hey @chansearrington, There's no benefit with unraid. Keep in mind that a real Netapp, is more then just this shelf. In a production environment it's 2 controllers + a shelf (or more). NetApps do really fancy things that are needed to mission critical applications. My NetApp training days are far behind me now, but I believe the extra ports at the bottom is for something called "High Availability multipath" (don't quote me) - This is something proprietary to their controllers + software. it's not something that will work out of the box, and probably not worth the effort required to get it to work. Quote Link to comment
drwtrippy Posted February 18, 2023 Share Posted February 18, 2023 On 2/3/2023 at 3:58 PM, UtahDeLorean said: @DannyG I had a situation where I was formatting bigger drives that I got. I got the 1st drive formatted and swapped out a drive that was tossing out error's. I was on the 2nd drive and it was running over night and when I check it the window was closed. I put in the 3rd drive and the window crashed/closed while I was watching it... When I tried to run the format line again I get this Does this mean that the format is still going or did I do something wrong? I have only complete 1 out of the 5 drives that I picked up. (I already did this on 4 other drives I purchased before with out any problems like this) Did you ever get this sorted out? I have one drive that is throwing this same error. Quote Link to comment
UtahDeLorean Posted March 5, 2023 Share Posted March 5, 2023 On 2/17/2023 at 5:08 PM, drwtrippy said: Did you ever get this sorted out? I have one drive that is throwing this same error. Yes... I let it work through and it took a reboot the next day, but it worked! 1 Quote Link to comment
UtahDeLorean Posted July 21, 2023 Share Posted July 21, 2023 I got 2 more disk shelf's that was gifted to me. I found some drives, but they are formatted to 4096. Would this process work the same to make them 512? Quote Link to comment
Mr.Lobo Posted August 28, 2023 Share Posted August 28, 2023 thank you very much. was just plug and play for me using a, LSI SAS9207-8e + DS4246 Quote Link to comment
KiiskiDR Posted December 12, 2023 Share Posted December 12, 2023 FYI: I got 11 10tb drives (4 Seagate and 7 WD sas models) gifted to me from a fellow IT guy that I didn't know that they were from a netapp appliance, and after connecting them up and running those commands I was able to start formatting them. My setup is a Chenbro-Cleversafe-Slicestor-1440 (48 Bay SAS) with to a LSI SAS2008 Fusion-MPT SAS2 [Falcon] (rev 03) card in IT mode in my Epyc based server. I didn't have the drives in the Netapp enclosure but instead in an expander on one side (the other 24 bay side has the 17 10TB WD WD100EMAZ drives connected). At first when I connected them I had them mixed on the expanders and didn't realize until I realized they might be a) not able to mix right, and b)oddball formatted/firmware enterprise drives, and found them to be option b, they were Netapp drives. After putting them on their own side apart from the sata drives, I started looking for information on how to flash their firmware because I remembered that they might have custom firmware, but then I just tried the commands shown here and it worked! They are formatting without any other steps needed! Thanks for this writeup! It did the trick! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.