Device is Disabled Contents Emulated - what to do

[hecky]

Hi

 

after nearly finishing the transfer from one to another server (truenas > unraid) i got some squeeking noise out of one of drives.

I spun all drives down and did a reboot.

Now one of discs says "Device is Disabled Contents Emulated".

 

What are my options?

Does a simple parity-check do the trick or do i have to rebuild the one drive (unassign > erase > preclear > re-assign > reconstruct)

 

TOSHIBA_MG09ACA18TE-20230330-0205.txt

[JorgeB]

Please post the diagnostics.

[hecky]

Hey and thanks for the replied.

Sadly i was realy busy

 

I tried to move the most important data from my server.

I want to rebuild disk 2.

At the moment it is spun down.

I am zeroing a disk of the same type just in case of emergency.

 

Sadly the following page is not accessable for me

https://wiki.unraid.net/Troubleshooting#Re-enable_the_drive

 

unraid-diagnostics-20230403-1048.zip

 

[JorgeB]

Diags after are rebooting so we can't see what happened, but disk looks healthy so possibly a power/connection problem, assuming the emulated disk is mounting and contents look correct you can rebuild on top, check/replace cables first to rule that out:

 

https://wiki.unraid.net/Manual/Storage_Management#Rebuilding_a_drive_onto_itself

 

[hecky]

The link to the how-to-steps did the job.

 

Thanks alot

[905jay]

I woke up this morning to a similar issue.

12TB Ironwolf Parity

5x8TB Ironwolf data

 

/dev/sdc is disabled contents emulated.

Server is in a 42u cabinet and hasn't moved or otherwise jostled so while I presently have not checked SATA and Power to the disk, I don't suspect that being an issue, but will be happy to. I just don't want to power down the server unless I know it's safe to do so.

 

The disks aren't 2 years old, and were all pre-cleared prior to adding to the server.

 

 

 

I started in Maintenance and did an fs check and repair, and SMART shows no errors.

Extended SMART has been running for about 6 hours and is currently at 80%, but was hoping I could seek some expert assistance in the meantime.

Diagnostic from yesterday is attached, running another for today also.

 

 

 

This is the Disk Log for /dev/sdc

Mar 17 00:25:44 unraid emhttpd: spinning down /dev/sdc
Mar 17 01:36:25 unraid emhttpd: read SMART /dev/sdc
Mar 17 02:37:08 unraid emhttpd: spinning down /dev/sdc
Mar 17 04:00:01 unraid emhttpd: read SMART /dev/sdc
Mar 17 05:41:54 unraid emhttpd: spinning down /dev/sdc
Mar 17 09:16:33 unraid emhttpd: read SMART /dev/sdc
Mar 17 10:20:19 unraid emhttpd: spinning down /dev/sdc
Mar 17 13:36:33 unraid emhttpd: read SMART /dev/sdc
Mar 17 14:36:54 unraid emhttpd: spinning down /dev/sdc
Mar 17 21:21:35 unraid emhttpd: read SMART /dev/sdc
Mar 17 22:23:48 unraid emhttpd: spinning down /dev/sdc
Mar 17 23:26:36 unraid emhttpd: read SMART /dev/sdc
Mar 18 00:28:39 unraid emhttpd: spinning down /dev/sdc
Mar 18 01:36:51 unraid kernel: sd 11:0:1:0: [sdc] tag#737 CDB: opcode=0x85 85 06 20 00 00 00 00 00 00 00 00 00 00 40 e5 00
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#752 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=7s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#752 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#752 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#752 CDB: opcode=0x88 88 00 00 00 00 00 c6 4e 37 48 00 00 00 08 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 3327014728 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#753 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=18s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#753 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#753 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#753 CDB: opcode=0x88 88 00 00 00 00 00 c6 4d 6d 30 00 00 00 08 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 3326962992 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#754 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#754 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#754 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#754 CDB: opcode=0x88 88 00 00 00 00 00 c5 97 90 d8 00 00 00 20 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 3315044568 op 0x0:(READ) flags 0x0 phys_seg 4 prio class 2
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#757 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#757 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#757 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#757 CDB: opcode=0x88 88 00 00 00 00 00 c6 13 63 b8 00 00 00 20 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 3323159480 op 0x0:(READ) flags 0x0 phys_seg 4 prio class 2
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#758 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#758 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#758 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#758 CDB: opcode=0x88 88 00 00 00 00 02 00 05 a0 60 00 00 00 10 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 8590303328 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 2
Mar 18 01:36:55 unraid emhttpd: read SMART /dev/sdc
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#761 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#761 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#761 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#761 CDB: opcode=0x8a 8a 00 00 00 00 00 c6 4e 37 48 00 00 00 08 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 3327014728 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 2
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#763 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=0s
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#763 Sense Key : 0x2 [current] 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#763 ASC=0x4 ASCQ=0x0 
Mar 18 01:36:55 unraid kernel: sd 11:0:1:0: [sdc] tag#763 CDB: opcode=0x8a 8a 00 00 00 00 00 c5 97 90 d8 00 00 00 20 00 00
Mar 18 01:36:55 unraid kernel: I/O error, dev sdc, sector 3315044568 op 0x1:(WRITE) flags 0x0 phys_seg 4 prio class 2
Mar 18 02:36:58 unraid emhttpd: spinning down /dev/sdc
Mar 21 21:07:43 unraid emhttpd: spinning up /dev/sdc
Mar 21 21:07:58 unraid kernel: sd 11:0:1:0: [sdc] tag#1285 CDB: opcode=0x85 85 06 20 00 00 00 00 00 00 00 00 00 00 40 e3 00
Mar 21 21:08:02 unraid emhttpd: sdspin /dev/sdc up: 22
Mar 21 21:08:02 unraid emhttpd: read SMART /dev/sdc
Mar 22 14:20:14 unraid emhttpd: read SMART /dev/sdc
Mar 22 14:23:10 unraid emhttpd: ST8000VN004-2M2101_WSD2XVZ0 (sdc) 512 15628053168
Mar 22 14:23:10 unraid kernel: mdcmd (5): import 4 sdc 64 7814026532 0 ST8000VN004-2M2101_WSD2XVZ0
Mar 22 14:23:10 unraid kernel: md: import disk4: (sdc) ST8000VN004-2M2101_WSD2XVZ0 size: 7814026532 
Mar 22 14:23:10 unraid emhttpd: read SMART /dev/sdc
Mar 22 14:47:04 unraid emhttpd: read SMART /dev/sdc
Mar 22 14:47:07 unraid emhttpd: ST8000VN004-2M2101_WSD2XVZ0 (sdc) 512 15628053168
Mar 22 14:47:07 unraid kernel: mdcmd (5): import 4 sdc 64 7814026532 0 ST8000VN004-2M2101_WSD2XVZ0
Mar 22 14:47:07 unraid kernel: md: import disk4: (sdc) ST8000VN004-2M2101_WSD2XVZ0 size: 7814026532 
Mar 22 14:47:07 unraid emhttpd: read SMART /dev/sdc

 

 

 

 

 

unraid-diagnostics-20240321-2118.zip

[JorgeB]

Looks like this issue:

https://forums.unraid.net/topic/103938-69x-lsi-controllers-ironwolf-disks-disabling-summary-fix/

 

[905jay]

Thank you @JorgeB would it also make sense to have the dives never spin down?

I am on unraid 6.12.8, and this was reported as being an issue on 6.9

[905jay]

For anyone just coming across this issue, this is what I did to fix it.

 

Array was started in Maintenance Mode

I ran the following commands outlined below taken from this post  linked below relating to Seagate Ironwolf disks and LSI HBA cards due to a drive showing disabled contents emulated

 

 

These are my mappings only related to Seagate Ironwolf Drives Only

I pasted the output from sg_map into notepad, and made notes as to what the sg_map output relates to which drive in my array and it's use (Parity and Data) just for my own record keeping

root@unraid:/tmp/SeaChest# sg_map
/dev/sg7  /dev/sdh Parity
/dev/sg2  /dev/sdc Data Array (drive that was disabled)
/dev/sg3  /dev/sdd Data Array
/dev/sg4  /dev/sde Data Array
/dev/sg1  /dev/sdb Data Array
/dev/sg9  /dev/sdj Data Array

 

 

The SeaChest folder structure is different than noted in the link above, Seagate seems to change this more often that I think they should (IMO)

This is the folder structure as of 2024-03-23

 

C:\USERS\JAY\DOWNLOADS\
└───SeaChestUtilities
    ├───doc
    ├───Linux
    │   ├───Non-RAID
    │   │   ├───aarch64
    │   │   └───x86_64
    │   └───RAID
    │       ├───aarch64-RAID
    │       └───x86_64-RAID
    ├───parallel_testing
    ├───USB boot maker
    └───Windows
        ├───Win64-Non-RAID
        └───Win64-RAID

 

These are files I will be working with. They reference alpine linux in the name, but when made executable they worked perfectly on unraid.

C:\USERS\JAY\DOWNLOADS\SeaChestUtilities\Linux\Non-RAID\x86_64\

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---l        2024-03-19   3:33 PM         657216 SeaChest_Basics_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         616256 SeaChest_Configure_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         706432 SeaChest_Erase_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         600168 SeaChest_Firmware_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         628544 SeaChest_Format_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         595808 SeaChest_GenericTests_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         712896 SeaChest_Info_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         583488 SeaChest_Lite_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         612160 SeaChest_NVMe_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         608064 SeaChest_PowerControl_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         554816 SeaChest_Reservations_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         653184 SeaChest_Security_x86_64-alpine-linux-musl_static
-a---l        2024-03-19   3:33 PM         853856 SeaChest_SMART_x86_64-alpine-linux-musl_static

 

 

From Windows 11 Terminal I opened 2 terminal tabs (I use ssh keys for access to all my Linux servers)

TAB1: (Keep tab open after running these commands)

ssh into unraid server and create a folder in /tmp called SeaChest

ssh unraid
mkdir /tmp/SeaChest

 

TAB2: Copy all the SeaChest utilities over to unraid in /tmp/SeaChest from the download location using scp command

scp C:\Users\jay\OneDrive\Downloads\SeaChestUtilities\Linux\Non-RAID\x86_64\* unraid:/tmp/SeaChest

 

TAB1: mark the files copied to unraid as executable

chmod +x /tmp/SeaChest/*

 

I disabled EPC as well as the lowCurrentSpinup (just to be safe but as noted in the link, not confirmed if necessary)
 

go to /tmp/SeaChest folder and run the 3 utilities below

• The first command will verify on /dev/sg2 (my problem disk) that epc is enabled
• Second command will disable EPC on /dev/sg2
• Third command will disable lowCurrentSpinup (not sure if this is actually needed but ran it regardless)

./SeaChest_Info_x86_64-alpine-linux-musl_static -d /dev/sg2 -i |grep -i epc
./SeaChest_PowerControl_x86_64-alpine-linux-musl_static -d /dev/sg2 --EPCfeature disable
./SeaChest_Configure_x86_64-alpine-linux-musl_static -d /dev/sg2 --lowCurrentSpinup disable

 

 

I ran those commands above for all of the /dev/sgX disks that were Seagate Ironwolf connected to my LSI card.

 

 

---

 

Once that portion was completed, I followed the guide below to rebuild the drive from parity

There was one caveat, for some reason I could not stop the array while it was is maintenance mode.

It kept saying Retry Unmounting Shares (for about 40 minutes) so I did a powerdown from terminal

powerdown

 

Once the server came back up I did the following procedure outlined in the guide linked below (TL;DR)

1. Stop array

2. Unassign disabled disk

3. Start array so the missing disk is registered

4. Stop array

5. Reassign disabled Disk (the symbol turned to a blue square)

6. Start array in maintenance mode

7. Clicked Sync button to rebuild the disk from parity

 

 

 

https://docs.unraid.net/unraid-os/manual/storage-management/#rebuilding-a-drive-onto-itself

Edited March 23 by 905jay
added further details on rebuilding disk onto itself