Jump to content

  • [6.11.0] NVMEs unavailable after upgrade 6.9.2 => 6.11.0

    Traumfaenger

    By Traumfaenger

      • Closed

    once I have updated unraid os from 6.9.2 to 6.11.0 my nvme drives were gone.

    They are still listed in systems devies though.

    When I do an os restore to 6.9.2 my drives are available again.

     

    I have also created a new stick with 6.11.0 and ran it. There my drives are available.

     

    I don't know what's going on here and hope to get some advice.

     

    ***I attach the diagnostics made under both versions***

     

    tower-diagnostics-20220926-1603_6.9.2.zip tower-diagnostics-20220926-1544_6.11.0.zip

    Go to reports Stable Releases

    User Feedback

    Recommended Comments



    limetech

    Please report output of this command:

     

    v /dev/disk/by-id

     

    Also, is it possible the controller those device are attached to has been 'stubbed' via vfio-pci.cfg (System Devices page)?  Could be with 6.11 PCI device ID's changed.

    Link to comment
    Traumfaenger

    Under 6.9.2

      

    root@Tower:~# v /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root  9 Sep 26 15:56 ata-OOS3000G-0x5000c500a820af3e -> ../../sdh
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-OOS3000G-0x5000c500a820af3e-part1 -> ../../sdh1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 ata-OOS3000G-0x5000c500aacc5c00 -> ../../sdi
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-OOS3000G-0x5000c500aacc5c00-part1 -> ../../sdi1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1LURDHU -> ../../sdj
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1LURDHU-part1 -> ../../sdj1
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1LURDHU-part9 -> ../../sdj9
lrwxrwxrwx 1 root root  9 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5VPZUU8 -> ../../sdg
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5VPZUU8-part1 -> ../../sdg1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6KUDJTD -> ../../sdk
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6KUDJTD-part1 -> ../../sdk1
lrwxrwxrwx 1 root root 10 Sep 26 15:56 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6KUDJTD-part9 -> ../../sdk9
lrwxrwxrwx 1 root root 13 Sep 26 15:53 nvme-CT1000P5PSSD8_21343107F76B -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 Sep 26 15:53 nvme-CT1000P5PSSD8_21343107F76B-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 13 Sep 26 15:53 nvme-CT1000P5PSSD8_2135313B508C -> ../../nvme1n1
lrwxrwxrwx 1 root root 15 Sep 26 15:53 nvme-CT1000P5PSSD8_2135313B508C-part1 -> ../../nvme1n1p1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 usb-058f_6387_15031291001177-0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Sep 26 15:56 usb-058f_6387_15031291001177-0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root  9 Sep 26 15:53 usb-Generic_STORAGE_DEVICE_000000001203-0:0 -> ../../sdb
lrwxrwxrwx 1 root root  9 Sep 26 15:53 usb-Generic_STORAGE_DEVICE_000000001203-0:1 -> ../../sdc
lrwxrwxrwx 1 root root  9 Sep 26 15:53 usb-Generic_STORAGE_DEVICE_000000001203-0:2 -> ../../sdd
lrwxrwxrwx 1 root root  9 Sep 26 15:53 usb-Generic_STORAGE_DEVICE_000000001203-0:3 -> ../../sde
lrwxrwxrwx 1 root root  9 Sep 26 15:53 usb-Generic_STORAGE_DEVICE_000000001203-0:4 -> ../../sdf
lrwxrwxrwx 1 root root  9 Sep 26 15:56 wwn-0x5000c500a820af3e -> ../../sdh
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x5000c500a820af3e-part1 -> ../../sdh1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 wwn-0x5000c500aacc5c00 -> ../../sdi
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x5000c500aacc5c00-part1 -> ../../sdi1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 wwn-0x50014ee20cb05f41 -> ../../sdj
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x50014ee20cb05f41-part1 -> ../../sdj1
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x50014ee20cb05f41-part9 -> ../../sdj9
lrwxrwxrwx 1 root root  9 Sep 26 15:56 wwn-0x50014ee20cb079c3 -> ../../sdg
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x50014ee20cb079c3-part1 -> ../../sdg1
lrwxrwxrwx 1 root root  9 Sep 26 15:56 wwn-0x50014ee2b75b5120 -> ../../sdk
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x50014ee2b75b5120-part1 -> ../../sdk1
lrwxrwxrwx 1 root root 10 Sep 26 15:56 wwn-0x50014ee2b75b5120-part9 -> ../../sdk9

     

    Under 6.11.0

      

    root@Tower:~# v /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root  9 Sep 26 22:32 ata-OOS3000G-0x5000c500a820af3e -> ../../sdh
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-OOS3000G-0x5000c500a820af3e-part1 -> ../../sdh1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 ata-OOS3000G-0x5000c500aacc5c00 -> ../../sdi
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-OOS3000G-0x5000c500aacc5c00-part1 -> ../../sdi1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 ata-OOS3000G_00000000 -> ../../sdh
lrwxrwxrwx 1 root root  9 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1LURDHU -> ../../sdj
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1LURDHU-part1 -> ../../sdj1
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N1LURDHU-part9 -> ../../sdj9
lrwxrwxrwx 1 root root  9 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5VPZUU8 -> ../../sdg
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N5VPZUU8-part1 -> ../../sdg1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6KUDJTD -> ../../sdk
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6KUDJTD-part1 -> ../../sdk1
lrwxrwxrwx 1 root root 10 Sep 26 22:32 ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N6KUDJTD-part9 -> ../../sdk9
lrwxrwxrwx 1 root root  9 Sep 26 22:32 usb-058f_6387_15031291001177-0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 Sep 26 22:32 usb-058f_6387_15031291001177-0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 usb-Generic_STORAGE_DEVICE_000000001203-0:0 -> ../../sdb
lrwxrwxrwx 1 root root  9 Sep 26 22:32 usb-Generic_STORAGE_DEVICE_000000001203-0:1 -> ../../sdc
lrwxrwxrwx 1 root root  9 Sep 26 22:32 usb-Generic_STORAGE_DEVICE_000000001203-0:2 -> ../../sdd
lrwxrwxrwx 1 root root  9 Sep 26 22:32 usb-Generic_STORAGE_DEVICE_000000001203-0:3 -> ../../sde
lrwxrwxrwx 1 root root  9 Sep 26 22:32 usb-Generic_STORAGE_DEVICE_000000001203-0:4 -> ../../sdf
lrwxrwxrwx 1 root root  9 Sep 26 22:32 wwn-0x5000c500a820af3e -> ../../sdh
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x5000c500a820af3e-part1 -> ../../sdh1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 wwn-0x5000c500aacc5c00 -> ../../sdi
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x5000c500aacc5c00-part1 -> ../../sdi1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 wwn-0x50014ee20cb05f41 -> ../../sdj
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x50014ee20cb05f41-part1 -> ../../sdj1
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x50014ee20cb05f41-part9 -> ../../sdj9
lrwxrwxrwx 1 root root  9 Sep 26 22:32 wwn-0x50014ee20cb079c3 -> ../../sdg
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x50014ee20cb079c3-part1 -> ../../sdg1
lrwxrwxrwx 1 root root  9 Sep 26 22:32 wwn-0x50014ee2b75b5120 -> ../../sdk
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x50014ee2b75b5120-part1 -> ../../sdk1
lrwxrwxrwx 1 root root 10 Sep 26 22:32 wwn-0x50014ee2b75b5120-part9 -> ../../sdk9

     

    Edited by Traumfaenger
    Link to comment
    mark949

    I also have a detection problem after updating to 6.11 of my Intel® SSD DC P4608 Series nvme drive, everything worked fine before the update. I rolled back the version 6.10 now and the disk is working again, please help!

    6.10.jpg

    6.11.jpg

    Edited by mark949
    Link to comment
    mark949
    • on 6.10.3:

    lrwxrwxrwx 1 root root 13 Sep 27 09:10 nvme-7335943:ICDPC5ED2ORA6.4T_PHLE9191007U6P4BGN-1 -> ../../nvme0n1
    lrwxrwxrwx 1 root root 15 Sep 27 09:10 nvme-7335943:ICDPC5ED2ORA6.4T_PHLE9191007U6P4BGN-1-part1 -> ../../nvme0n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 09:10 nvme-7335943:ICDPC5ED2ORA6.4T_PHLE9191007U6P4BGN-2 -> ../../nvme1n1
    lrwxrwxrwx 1 root root 15 Sep 27 09:10 nvme-7335943:ICDPC5ED2ORA6.4T_PHLE9191007U6P4BGN-2-part1 -> ../../nvme1n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 09:10 nvme-INTEL_SSDPE2MD016T4_PHFT6063001A1P6JGN -> ../../nvme2n1
    lrwxrwxrwx 1 root root 15 Sep 27 09:10 nvme-INTEL_SSDPE2MD016T4_PHFT6063001A1P6JGN-part1 -> ../../nvme2n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 09:10 nvme-eui.01000000010000005cd2e4c78ede5051 -> ../../nvme1n1
    lrwxrwxrwx 1 root root 15 Sep 27 09:10 nvme-eui.01000000010000005cd2e4c78ede5051-part1 -> ../../nvme1n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 09:10 nvme-nvme.8086-5048465436303633303031413150364a474e-494e54454c205353445045324d443031365434-00000001 -> ../../nvme2n1
    lrwxrwxrwx 1 root root 15 Sep 27 09:10 nvme-nvme.8086-5048465436303633303031413150364a474e-494e54454c205353445045324d443031365434-00000001-part1 -> ../../nvme2n1p1

     

    •  on 6.11

    lrwxrwxrwx 1 root root 13 Sep 27 23:39 nvme-7335943:ICDPC5ED2ORA6.4T_PHLE9191007U6P4BGN-1 -> ../../nvme0n1
    lrwxrwxrwx 1 root root 15 Sep 27 23:39 nvme-7335943:ICDPC5ED2ORA6.4T_PHLE9191007U6P4BGN-1-part1 -> ../../nvme0n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 23:39 nvme-INTEL_SSDPE2MD016T4_PHFT6063001A1P6JGN -> ../../nvme2n1
    lrwxrwxrwx 1 root root 15 Sep 27 23:39 nvme-INTEL_SSDPE2MD016T4_PHFT6063001A1P6JGN-part1 -> ../../nvme2n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 23:39 nvme-eui.01000000010000005cd2e4c78ede5051 -> ../../nvme0n1
    lrwxrwxrwx 1 root root 15 Sep 27 23:39 nvme-eui.01000000010000005cd2e4c78ede5051-part1 -> ../../nvme0n1p1
    lrwxrwxrwx 1 root root 13 Sep 27 23:39 nvme-nvme.8086-5048465436303633303031413150364a474e-494e54454c205353445045324d443031365434-00000001 -> ../../nvme2n1
    lrwxrwxrwx 1 root root 15 Sep 27 23:39 nvme-nvme.8086-5048465436303633303031413150364a474e-494e54454c205353445045324d443031365434-00000001-part1 -> ../../nvme2n1p1

     

    Quote

    Just to be sure, please try a 'safe mode' boot in 6.11 and verify your nvme devices still don't show up.

    safe mode - does not help, I even tried to update from this mode and the first download in this mode is in the photo

     

     

    error on 6.11.jpg

    Edited by mark949
    Link to comment
    JorgeB 
    Sep 28 15:40:38 unraid kernel: nvme nvme0: globally duplicate IDs for nsid 1

     

    This is a device problem, and a "quirk" needs to be added to the kernel, LT did it for another device before, they might be able to do it for yours, but it's still a device problem:

     

    Quote

    Well, that commit just caught your disk ingoring the standard and reporting non-unique identifiers, which are a big data loss risk. Please send the output of lspci -nn so that we can add a quirk, and please also contact the vendor to tell them that their device is broken and/or leave a review to that affect on the website where you bought it if possible.

    https://lore.kernel.org/all/[email protected]/t/

     

    • Thanks 1
    Link to comment
    mark949
    3 minutes ago, JorgeB said: 
    Sep 28 15:40:38 unraid kernel: nvme nvme0: globally duplicate IDs for nsid 1

     

    This is a device problem, and a "quirk" needs to be added to the kernel, LT did it for another device before, they might be able to do it for yours, but it's still a device problem:

     

    https://lore.kernel.org/all/[email protected]/t/

    IDs for nsid are actually the same only at the beginning, if you read my posts above you will see that they are still different and work just as well in all previous versions of unraid. Why did Unraid separate them perfectly before, and now suddenly the problem with my device has become? To whom should I report this? to Intel? funny to hear this...
    Do I have to edit the kernel for purchased software myself?

    3 minutes ago, JorgeB said:

     

     

    Link to comment
    mark949

    Where are the identical identifiers?

     

    on 6.10 this device is not problematic, but on 6.11 it is already problematic and needs to be thrown out ...

    top-level device if that Intel® SSD DC P4608 Series

     

    6.11.jpg

    Edited by mark949
    Link to comment
    JorgeB
    1 hour ago, mark949 said:

    Where are the identical identifiers?

    It's not the identifiers it's the nsdi

    1 hour ago, JorgeB said: 
    nvme nvme0: globally duplicate IDs for nsid 1

     

    1 hour ago, mark949 said:

    on 6.10 this device is not problematic

    Because the kernel in v6.10 didn't check that, newer kernel does, like mentioned Tom  created a quirk for a different device, I'm sure he can do the same for yours, but it's still a device problem.

    • Thanks 1
    Link to comment
    JorgeB
    6 minutes ago, mark949 said:

    I can not understand what I need to do and who should I write to now?

    Just wait to see if Tom can add that quirk to the next release, he's already following this thread, so no need for anything else for now, and if he needs more info he will ask for it.

    • Like 1
    Link to comment
    Traumfaenger
    On 9/27/2022 at 1:04 AM, limetech said:

    Just to be sure, please try a 'safe mode' boot in 6.11 and verify your nvme devices still don't show up.

    I just tried. Still not available.

    Link to comment
    JorgeB

    Yours is a different case than the others, others have a problem when there are two devices installed because they have the same nsid, so Unraid only shows one device, and that can be fixed with a "quirk", you lose both devices because they have no bran/model output with this kernel, not sure if/how that can be fixed by LT.

     

     

    Link to comment
    Traumfaenger
    1 hour ago, JorgeB said:

    Yours is a different case than the others, others have a problem when there are two devices installed because they have the same nsid, so Unraid only shows one device, and that can be fixed with a "quirk", you lose both devices because they have no bran/model output with this kernel, not sure if/how that can be fixed by LT.

     

     

    Yeah, okay. But then they should open another thread since I have opened this one. :) We shouldn't talk about different things in one thread, right? 

    Link to comment
    JorgeB
    3 minutes ago, Traumfaenger said:

    We shouldn't talk about different things in one thread, right? 

    You're right, the other reported issue here should be fixed on the next related, so you can leave this one open for yours, hopefully a newer kernel will help, so you should also text any new release that includes one.

    Link to comment
    sebz29a

    Hi, i'm in the same case. I have 2 polls each 2 Nvme drive in my unraid server.

    6.10.3 evrything is ok

    6.11 one if the Nvme cache drive is not recognized.

    Rollback in 6.10.3 solve the problem.

     

    I can't see enything in 6.11.1 changelog about this.

    What does i have to do to solve this problem ? Thanks for tour help

     

    • Like 1
    Link to comment
    JorgeB
    1 hour ago, sebz29a said:

    Hi, i'm in the same case.

    It's not the same case, in the OP's case all devices are missing, not just one of a pair, and v6.11.1 does add some quirks to known bad NVMe devices, but please post the diagnostics to see which model(s) you are using.

    Link to comment
    Traumfaenger

    I just tried 6.11.1 but still:

      

    Oct  7 13:06:14 Tower  emhttpd: device /dev/nvme1n1 problem getting id
Oct  7 13:06:14 Tower  emhttpd: device /dev/nvme0n1 problem getting id

     

    Guess I have to wait longer. :( ...back to 6.9.2 then...

    Link to comment
    thetadawg

    Tried 6.11 and lost one of my ADATA SX8200 nvmes, I think that I have the same issue.  So I reverted back to 6.10.3 and both nvmes showed back up.  I then saw the there was an newer update (6.11.1), so I figured that I would give it a shot again.  Looks like it is still happening...not sure if/how to make an quirk entry.  Hopefully someone can shed some light on a possible fix. Going to revert back again and keep checking back here.

     

    Attached my diagnostic

    6.11.1_diagnostics-20221007-1902.zip

    • Upvote 1
    Link to comment
    JorgeB
    8 hours ago, thetadawg said:

    I then saw the there was an newer update (6.11.1)

    Some quirks were added for v6.11.1, but LT can only add the ones that are reported, I added yours to the list, it should be added for the next release.

    Link to comment
    CablDeViL

    I have the same issue with my Samsung 970 evo 1tb cache drive. But it was fine at 6.11.0. I upgraded to 6.11.1 this am and now this nvme is missing.  Rolled back the 6.11 and still missing.  Let me know if there is any logs you would want.

     

    Edit:  Ran v /dev/disk/by-id. nvme missing in 6.11 and 6.11.1  

     

    CD

    unraid-diagnostics-20221008-1145.zip

    Edited by CablDeViL
    added nvme output
    Link to comment


    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.

    ×

  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.

    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.

×
×
  • Create New...