Hello all,

I have recently switched from 24x 1TB with SATA to SAS adapters in my NetApp DS4243 to 4x 16TB with SATA to SAS adapters. My issue is, that only one of the 4 Seagate Exos drives show up in the GUI. The other three are listed under system devices, but show "-" as size. When looking at the log, it displays "emhttpd: device /dev/sdk has size zero".

What am I missing here? What is wrong? Sidenote: I previously had the diskshelf connected with two SAS-cables, but found out through another post, that multipathing is not supported in Unraid. Interestingly though, when I had the shelf filled up with 1TB disks, it did not through any errors regarding multipathing.

There are other errors when Unraid is detecting the drives upon insertion as well. Log file is attached. If you need anything else. Please let me know.

Thank you very much in advance.

Regards,
Niggo

storinator-syslog-20231111-1701.zip

Edit: Here is a screentshot of the system devices tab.

Please post the diagnostics, also, are the disks SATA and you using SATA to SAS interposers with the disks? If yes try without them.

Diagnostics attached below.

Yes, I am using SATA to SAS interposers and have tried without them. Still no change, sadly.

storinator-diagnostics-20231113-1736.zip

Logs are full of controller errors, I know there have been issues with these NetAPP controllers before, but I'm not familiar with them, I assume you don't have an LSI and correct cable to try with that? And these enclosures usually work better without the interposers, so that you can get actual SMART info.

I do have an LSI controller, though I am unsure whther I can flash it to IT mode since it is from an IBM System X and is branded as ServeRAID (model number is ServeRAID M5110). No adapter-cables though.

I tried without the interposers before, but then it just doesn't detect any of the drives in any way shape or form.

 

Strangely though, I did not have any problems with the NetApp drives, that were in the JBOD before. Probably because they were the "same" brand (AFAIK the drives as well as the controller are just rebranded, WD-Black and Chelsio respectively)

11 hours ago, Niggo said:

I am unsure whther I can flash it to IT mode

Looks like it can:

 

https://lazymocha.com/blog/2020/06/05/cross-flash-ibm-servraid-m5110-and-h1110-to-it-hba-mode/

Thanks for the hint. I am really unsure whether it is worth the effort though. Since the controller only has internal ports, and I don't have any adapter cables, I might just go out and buy an actual SAS controller that does have external ports with the default external SAS connector and buy some adapter cables.

What do you think?

I would still recommend an LSI, and the external post they use, e.g., sff-8088 probably won't be the same you have, will still need a cable, but probably much easier to find for a HBA with external ports.

On 11/11/2023 at 12:04 PM, Niggo said:

Hello all,

I have recently switched from 24x 1TB with SATA to SAS adapters in my NetApp DS4243 to 4x 16TB with SATA to SAS adapters. My issue is, that only one of the 4 Seagate Exos drives show up in the GUI. The other three are listed under system devices, but show "-" as size. When looking at the log, it displays "emhttpd: device /dev/sdk has size zero".

What am I missing here? What is wrong? Sidenote: I previously had the diskshelf connected with two SAS-cables, but found out through another post, that multipathing is not supported in Unraid. Interestingly though, when I had the shelf filled up with 1TB disks, it did not through any errors regarding multipathing.

There are other errors when Unraid is detecting the drives upon insertion as well. Log file is attached. If you need anything else. Please let me know.

Thank you very much in advance.

Regards,
Niggo

storinator-syslog-20231111-1701.zip 47.48 kB · 2 downloads

 

Hello,

 

-All you have to do is to run the following command

-MAKE SURE 'sdk sdm sdn' are the 3 16TB drives you are missing.

 

for disk in sdk sdm sdn; do nohup sg_format -v -F /dev/$disk > /tmp/format_$disk.log 2>&1 & done

 

-This command will start the formatting process for each disk listed (sdk sdm sdn) in the background, with the output and any errors being logged to separate files in the /tmp directory for each disk.

 

-Remember, this is a disk formatting operation, so it will ERASE ALL DATA on the specified disks.

 

-BE PATIENT the process for 16TB will take a while 24hr+-

-You can check the progress by ruining following command for each drive.

 

tail -f /tmp/format_sdk.log
tail -f /tmp/format_sdm.log
tail -f /tmp/format_sdn.log

 

-The output should be something like this after the command has been running for some time (not right away) you will start seen the progress in the logs. BE PATIENT let it do is thing, come back and check the logs you will start seen it in the logs.

 

 

-Cntrl + C to quit and check a different drive.

-Once the process is complete drives will appear and be available for the array or under Unassigned devices,

 

Edited November 17, 20232 yr by Tolete
typo

Thank you, I will try that.

Though it should be of note, this is what my system log looks like now:

storinator-syslog-20231118-1504.zip

Hello,

 

sadly I still am unable to figure this out. I tried formatting, but that still did not help. Though now two drives show up consistently. As far as the lohs are concerned, it first says that the drive has a few million 512 byte sectors, then it tries to initialize / spin up the drive again and then it shows as 0 512 byte sectors.

 

I should note, I wasn't able to get the formatting working on the drives that do not show up, other than in the system devices tab, with a - as their size reading.

 

Logs attached.

storinator-syslog-20231216-1338.zip

[0:0:0:0]disk SMI USB DISK 1100 /dev/sda 8.17GB

[1:0:4:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdo 1.00TB

[1:0:5:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdp 1.00TB

[1:0:6:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdq 1.00TB

[1:0:7:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdr 1.00TB

[1:0:8:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sds 1.00TB

[1:0:9:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdt 1.00TB

[1:0:10:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdu 1.00TB

[1:0:11:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdv 1.00TB

[1:0:12:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdw 1.00TB

[1:0:13:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdx 1.00TB

[1:0:14:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdy 1.00TB

[1:0:15:0]disk WDC WD1003FBYX-05YSA NA03 /dev/sdz 1.00TB

[1:0:18:0]disk ATA SA 4321 /dev/sdk 16.0TB

[1:0:21:0]disk ATA SA 4321 /dev/sdn -

[1:0:22:0]disk ATA SA 4321 /dev/sdl 16.0TB

[1:0:23:0]disk ATA SA 4321 /dev/sdm -

[2:0:0:0]disk ATA Patriot P210 204 916a /dev/sdb 2.04TB

[3:0:0:0]disk ATA WDC WD40EFZX-68A 0B81 /dev/sdc 4.00TB

[5:0:0:0]disk ATA Patriot P210 204 916a /dev/sdd 2.04TB

[6:0:0:0]disk ATA WDC WD40EFZX-68A 0B81 /dev/sde 4.00TB

[7:0:0:0]disk ATA WDC WD40EFZX-68A 0B81 /dev/sdf 4.00TB

[8:0:0:0]disk ATA WDC WD40EFZX-68A 0B81 /dev/sdg 4.00TB

[9:0:0:0]disk ATA Patriot P210 204 916a /dev/sdh 2.04TB

[10:0:0:0]disk ATA Patriot P210 204 916a /dev/sdi 2.04TB

[11:0:0:0]disk ATA WDC WD40EFZX-68A 0B81 /dev/sdj 4.00TB

[N:0:0:1]disk KINGSTON SNV2S1000G__1 /dev/nvme0n1 1.00TB

[N:1:0:1]disk KINGSTON SNV2S1000G__1 /dev/nvme1n1 1.00TB

 

This is the output of the SCSI Devices section under Tools -> System Devices

no luck?  i just woke up to this issue.  Different controller and disk shelf. 
Everything was working great when i went to bed last night!  i have x7 identical disks in the same shelf...x3 of them are missing/zero-size.  Seems like the server hard rebooted before this as well, as i woke up to the array not started, and the x3 out of x6 disks missing from the pool they were in.  I might have over filled them with a backup last night...but that shouldn't make unraid crash reboot itself and forget drives right?  

 

[1000:0072] 82:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

[12:0:1:0] disk SEAGATE ST3000NM0023 GS0D /dev/sdaa -

[12:0:2:0] disk SEAGATE ST33000650SS RE12 /dev/sdab 3.00TB

[12:0:3:0] disk SEAGATE ST3000NM0023 GS0D /dev/sdac -

[12:0:4:0] disk SEAGATE ST3000NM0023 GS0D /dev/sdad -

[12:0:5:0] disk SEAGATE ST33000650SS RE12 /dev/sdae 3.00TB

[12:0:6:0] disk SEAGATE ST33000650SS RE12 /dev/sdaf 3.00TB

[12:0:0:0] disk SEAGATE ST33000650SS RE12 /dev/sdz 3.00TB

 

 

Again, these drives were fully precleared, and were running in the pool last night without issue!  The ONLY bad thing is i didn't calculate the backup i was running and it was bigger then the pool.  But again, that should have caused a write error, NOT a hard crash/reboot and disks forgetting their size...right?
I've attached a bunch of stuff here...feel free to kick me to a new topic if i'm hijacking this thread.

elmstorage-diagnostics-20240102-0933.zip elmstorage-syslog-20240102-1524.zip

Edited January 2, 20242 yr by miicar
more info

Check if the disks are being detected by the HBA BIOS.

12 minutes ago, JorgeB said:

Check if the disks are being detected by the HBA BIOS.

The HBA bios was removed by the previous owner, and i didn't bother putting it back.  the disks do show up in undaid's sys devices under that controller tho (see above). 

Of course a reboot fixed it, but i have no idea why it happened in the first place...which is a big problem for me.

Edited January 2, 20242 yr by miicar

48 minutes ago, miicar said:

The HBA bios was removed by the previous owner, and i didn't bother putting it back. 

I see that the HBA is using the MegaRAID driver, you should flash it to IT mode, mpt3sas driver is much more reliable.

1 hour ago, JorgeB said:

I see that the HBA is using the MegaRAID driver, you should flash it to IT mode, mpt3sas driver is much more reliable.

That was the first thing i verified, and flashed on to this card, before i even started this (i didn't re-add the bios though, cuz i had to make a bootable USB, and find a mobo that both recognized the HBA, and liked to boot in legacy mode.  It didn't seem worth the effort, as i will never be booting from any disks connected to this HBA)!  Maybe you are confusing with the onboard (LSI SAS3-3108) MegaRAID?  The card with all the current issues is listed as "[1000:0072] 82:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)" in sysdevices.

My onboard HBA is MegaRAID capable, also in JBOD mode, is running fine (besides the silly x16 drive cap surprise, that, annoyingly, forced me to used this LSI-9200-8e in the first place!!)...but maybe there is a way to flash it to even more IT mode?  i have JBOD and RAID as the options in the onboard HBA's bios settings.

(FYI @JorgeBthis is the same card you were helping me with, in another thread, where i imported this pool the wrong way at first and was getting clones appearing in UD.  This happened after i seemed to fix it and get it working properly, then ran a backup to that pool.)

Edited January 2, 20242 yr by miicar

13 minutes ago, miicar said:

Maybe you are confusing with the onboard (LSI SAS3-3108) MegaRAID?

Yes, that's the one I was seeing, since it's in use didn't notice the other one.

6 minutes ago, JorgeB said:

Yes, that's the one I was seeing, since it's in use didn't notice the other one.

all good.  i have allot going on in this server! 

BUT, as a side note, should i be trying to flash my onboard HBA to a different/better firmware? Its worked for 2 years without any issues, but now i have added an expander to it, but it seems to be fine with everything still (as long as i keep it x16 disks or less...otherwise both myself and it has a mental breakdown and thats another 26 hour story).

I did see this note under the mpt3sas driver info, "# limetech - Workaround for kernel crash with LSI 92xx based HBA cards (and may be others)
options mpt3sas max_queue_depth=10000".   And my addon card is a lsi-9200-8e.  I'm guessing i have to set that in the cards bios?  or is this in unraid somewhere??

Edited January 2, 20242 yr by miicar

Like mentioned the other driver is more used and usually more stable, but if it's working fine...

 

No need to do anything for the mpt3sas driver, it's corrected automatically.

I would like to add, mine still don't work. The same two of the four devices show up intermittently, while the two other 16TB disks are just not happening. Meanwhile I've replaced the HBA with an Adaptec one (78165) since it has HBA mode by default and you can just change it to whatever mode you need. I've also replaced the IOM3 modules in the shelf with IOM6 ones. Still, the netapp branded drives show up fine, two (or three, depending on whatever) do not show up at all. Now that I've changed the controller, they do not show up as zero size, rather they don't show up at all in Unraid.

 

It should be of note though, rebooting (thus plugging the cables out and back into) the shelf will change the detected drives between one and two, regarding the Exos 16TB disks. All 1TB Netapp disks show up though. They always do. I'm getting the feeling there's something wrong with those disks. Because not even the controller will detect them in it's bios. When I plug them into my Windows machine though, all of them show up and work just fine.

 

I am out of ideas here.

Edited January 2, 20242 yr by Niggo

1 hour ago, JorgeB said:

No need to do anything for the mpt3sas driver, it's corrected automatically.

Thats good to hear...i guess the note is just left overs from before the workaround/fix was baked into unraid.  
 

1 hour ago, JorgeB said:

Like mentioned the other driver is more used and usually more stable, but if it's working fine...

 

I'll look into that later...unless its maybe causing conflict between the mpt3sas driver?

 

UPDATE: So LSI called me back (and gained a ton of respect for doing so)!  Apparently the LSI-9200 cards i have do not have thermal throttling, and could be the reason it hard crashed my kernel (and rebooted the machine) while running the backup last night.  I'm going to have to keep an eye on that, and was advised to add a fan to the controller heat sync.

Edited January 2, 20242 yr by miicar

Sooo, after all this time... an update.

At some point I realized I was using SAS and SATA disks in the same enclosure. Same exact disks except for the connector. Now, I did know that, I was just not fully aware of the implications. The SAS disks, obviously, showed up just fine when plugging in two controller into the disk shelf. You might already know where this is going. The SATA disks, however, were not. This is due to the fact that SAS disks are capable of connecting to two controllers/HBAs simultaneously but SATA disks are not. Thus, whenever I plugged in the second IOM on the disk shelf, the SATA disks kept randomly deciding which controller they were going to talk to. The SAS disks did not mind at all, since well, they are dual-controller capable.

Well, so much for that.