Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

ZFS RAIDZ2 after new config, unmountable: unsupported or no file system

Featured Replies

Hello everyone, I unfortunately misread something while changing an array and ejected/deleted my ZFS z2 pool with the “new config” tool. I then rewrote the config file from a USB backup onto the Unraid USB stick, but unfortunately it does not recognize the Raidz2 and always writes “Unmountable: wrong or no file system.” Would anyone be so kind as to help me? Thanks 🙂

in the pool import I see that de device sdj is two times in there and in the unread main gui is it not!

root@Diskstation:~# zpool import
  pool: zfs.spinners
    id: 263348690834153820
 state: FAULTED
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
config:

        zfs.spinners  FAULTED  corrupted data
          raidz2-0    DEGRADED
            sdj1      ONLINE
            sdg1      ONLINE
            sdh1      ONLINE
            sdi1      ONLINE
            sdd1      ONLINE
            sde1      ONLINE
            sdf1      ONLINE
            sdj1      FAULTED  corrupted data
root@Diskstation:~# 

ZFS.jpg

  • Community Expert

The pool is damaged; this would not be caused by just doing a the new config, post the output from zdb -l /dev/sdX for all pool devices.

  • Author
Just now, Vonghofa said:

Hello everyone, I unfortunately misread something while changing an array and ejected/deleted my ZFS z2 pool with the “new config” tool. I then rewrote the config file from a USB backup onto the Unraid USB stick, but unfortunately it does not recognize the Raidz2 and always writes “Unmountable: wrong or no file system.” Would anyone be so kind as to help me? Thanks 🙂

in the pool import I see that de device sdj is two times in there and in the unread main gui is it not!

root@Diskstation:~# zpool import
  pool: zfs.spinners
    id: 263348690834153820
 state: FAULTED
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
config:

        zfs.spinners  FAULTED  corrupted data
          raidz2-0    DEGRADED
            sdj1      ONLINE
            sdg1      ONLINE
            sdh1      ONLINE
            sdi1      ONLINE
            sdd1      ONLINE
            sde1      ONLINE
            sdf1      ONLINE
            sdj1      FAULTED  corrupted data
root@Diskstation:~# 

ZFS.jpg

Just now, Vonghofa said:

Hello everyone, I unfortunately misread something while changing an array and ejected/deleted my ZFS z2 pool with the “new config” tool. I then rewrote the config file from a USB backup onto the Unraid USB stick, but unfortunately it does not recognize the Raidz2 and always writes “Unmountable: wrong or no file system.” Would anyone be so kind as to help me? Thanks 🙂

in the pool import I see that de device sdj is two times in there and in the unread main gui is it not!

root@Diskstation:~# zpool import
  pool: zfs.spinners
    id: 263348690834153820
 state: FAULTED
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
config:

        zfs.spinners  FAULTED  corrupted data
          raidz2-0    DEGRADED
            sdj1      ONLINE
            sdg1      ONLINE
            sdh1      ONLINE
            sdi1      ONLINE
            sdd1      ONLINE
            sde1      ONLINE
            sdf1      ONLINE
            sdj1      FAULTED  corrupted data
root@Diskstation:~# 

ZFS.jpg

root@Diskstation:~# zdb -l /dev/mapper/sdX1
cannot open '/dev/mapper/sdX1': No such file or directory
root@Diskstation:~# ls /dev/mapper
control  md1p1@  sdd1@  sde1@  sdf1@  sdg1@  sdh1@  sdi1@  sdj1@
root@Diskstation:~# zdb -l /dev/mapper/sdj1
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2601592
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 12008641290493221446
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdf1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdf1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdg1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdg1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 2 3 
------------------------------------
LABEL 1 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 0
    txg: 2601593
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 12008641290493221446
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 1 
root@Diskstation:~# 
  • Community Expert

And the other pool devices?

  • Author

I also changed the SATA controller beforehand, and everything ran normally afterwards. Then I wanted to replace the Force MP510 SSD and misread something, doing exactly the wrong thing in the new configuration. So, for the ZFS mirror raid ss.cache and for the ZFS raidz2 1vdev of 8 devices. I then imported the configuration file again and the ssd.cache was successfully recognized, but unfortunately not the zfs.spinner.

Even though all 11 hard drives are displayed in the Unraid GUI, the command *ls /dev/mapper only finds 10 hard drives.

See screenshot of the main GUI in Unraid. Here is my hard drive configuration. Oh, and they are all ZFS encrypted.

unraid.jpg

  • Author
root@Diskstation:~# ls /dev/mapper
control  md1p1@  sdb1@  sdc1@  sdd1@  sde1@  sdf1@  sdg1@  sdh1@  sdi1@  sdj1@
root@Diskstation:~# zdb -l /dev/mapper/sdj1
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2601592
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 12008641290493221446
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdf1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdf1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdg1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdg1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 2 3 
------------------------------------
LABEL 1 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 0
    txg: 2601593
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 12008641290493221446
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 1 
root@Diskstation:~# 
  • Community Expert
15 minutes ago, Vonghofa said:

I then imported the configuration file again and the ssd.cache was successfully recognized, but unfortunately not the zfs.spinner.

Yes, because there's a problem with the pool, the new config didn't cause this.

16 minutes ago, Vonghofa said:

Even though all 11 hard drives are displayed in the Unraid GUI, the command *ls /dev/mapper only finds 10 hard drives.

I only need the ones from the pool that is not working, post the ouptut from:

zdb -l /dev/mapper/sdd1

zdb -l /dev/mapper/sde1

zdb -l /dev/mapper/sdf1

zdb -l /dev/mapper/sdg1

zdb -l /dev/mapper/sdh1

zdb -l /dev/mapper/sdi1

zdb -l /dev/mapper/sdj1

zdb -l /dev/mapper/sdk1

(if this last one doesn't work post the diagnostics)

  • Author
root@Diskstation:~# zdb -l /dev/mapper/sdd1
zdb -l /dev/mapper/sde1
zdb -l /dev/mapper/sdf1
zdb -l /dev/mapper/sdg1
zdb -l /dev/mapper/sdh1
zdb -l /dev/mapper/sdi1
zdb -l /dev/mapper/sdj1
zdb -l /dev/mapper/sdk1
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2609501
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 7114031952368967726
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 22
            create_txg: 4
            aux_state: 'err_exceeded'
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3 
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2601592
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 16103837880165441879
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdf1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdf1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdg1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdg1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3 
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2609501
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 2600655244259309535
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 22
            create_txg: 4
            aux_state: 'err_exceeded'
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3 
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2609501
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 825157947067356691
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 22
            create_txg: 4
            aux_state: 'err_exceeded'
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3 
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2609501
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 8543962664714574884
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 22
            create_txg: 4
            aux_state: 'err_exceeded'
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3 
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2609501
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 10158990157654651413
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 22
            create_txg: 4
            aux_state: 'err_exceeded'
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3 
------------------------------------
LABEL 0 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 1
    txg: 2601592
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 12008641290493221446
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdf1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdf1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdg1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdg1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 2 3 
------------------------------------
LABEL 1 
------------------------------------
    version: 5000
    name: 'zfs.spinners'
    state: 0
    txg: 2601593
    pool_guid: 263348690834153820
    errata: 0
    hostname: 'Diskstation'
    top_guid: 14800982098715042233
    guid: 12008641290493221446
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 14800982098715042233
        nparity: 2
        metaslab_array: 76
        metaslab_shift: 34
        ashift: 12
        asize: 112003983081472
        is_log: 0
        create_txg: 4
        children[0]:
            type: 'disk'
            id: 0
            guid: 12008641290493221446
            path: '/dev/mapper/sdc1'
            devid: 'dm-uuid-CRYPT-LUKS2-cee2983101f44e258469c419074ee955-sdc1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[1]:
            type: 'disk'
            id: 1
            guid: 825157947067356691
            path: '/dev/mapper/sdi1'
            devid: 'dm-uuid-CRYPT-LUKS2-081d4c6a47364044989e9e11dbdfaab9-sdi1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[2]:
            type: 'disk'
            id: 2
            guid: 8543962664714574884
            path: '/dev/mapper/sdh1'
            devid: 'dm-uuid-CRYPT-LUKS2-90e0c88a0e6a40ed83af5981ed53369f-sdh1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[3]:
            type: 'disk'
            id: 3
            guid: 10158990157654651413
            path: '/dev/mapper/sdk1'
            devid: 'dm-uuid-CRYPT-LUKS2-23a28273f7dc4608bfcdbaf572cd356d-sdk1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[4]:
            type: 'disk'
            id: 4
            guid: 7114031952368967726
            path: '/dev/mapper/sdd1'
            devid: 'dm-uuid-CRYPT-LUKS2-3ec9fe477ec348edbe9883b52310606e-sdd1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[5]:
            type: 'disk'
            id: 5
            guid: 16103837880165441879
            path: '/dev/mapper/sde1'
            devid: 'dm-uuid-CRYPT-LUKS2-0dec3fab3fb7454b9e8e290eb9665a1b-sde1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            DTL: 8
            create_txg: 4
            aux_state: 'err_exceeded'
        children[6]:
            type: 'disk'
            id: 6
            guid: 2600655244259309535
            path: '/dev/mapper/sdb1'
            devid: 'dm-uuid-CRYPT-LUKS2-7ca469ab1aed4137bef118d8d5545600-sdb1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
        children[7]:
            type: 'disk'
            id: 7
            guid: 10090287586978045426
            path: '/dev/mapper/sdj1'
            devid: 'dm-uuid-CRYPT-LUKS2-e65507a1f26d42fd9465bbbdee7526e3-sdj1'
            phys_path: '/dev/disk/by-uuid/263348690834153820'
            whole_disk: 0
            create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 1 
cannot open '/dev/mapper/sdk1': No such file or directory
root@Diskstation:~# 

diskstation-diagnostics-20251027-2048.zip

  • Community Expert

/dev/sdk lost its partition:

Oct 27 11:17:25 Diskstation emhttpd: /bin/lsblk -lnbo TYPE,NAME,START,SIZE,PARTLABEL /dev/sdk 2>&1

Oct 27 11:17:25 Diskstation emhttpd: disk sdk 14000519643136

Oct 27 11:17:25 Diskstation emhttpd: device sdk: no partitions

Oct 27 11:17:25 Diskstation emhttpd: import 39 pool device: (sdk) ST14000NE0008-2RX103_ZL2A9SRA

Assuming you haven't rebooted yet, and that disk is still sdk, type sgdisk -o -a 8 -n 1:32K:0 /dev/sdk then post output from blkid /dev/sdk1

  • Author

Do you mean since we solved the problem? If so, then I haven't restarted, so SDK should still be SDK.

I'll try the command and see what happens.

Thanks for your help.

  • Community Expert
2 minutes ago, Vonghofa said:

Do you mean since we solved the problem?

What do you mean by that, is the pool working now?

  • Author

No, my mistake, the pool isn't ready yet. Here is the output of your commands.

root@Diskstation:~# sgdisk -o -a 8 -n 1:32K:0 /dev/sdk
Creating new GPT entries in memory.
The operation has completed successfully.
root@Diskstation:~# blkid /dev/sdk1
/dev/sdk1: PARTUUID="fd6ee637-33b0-426a-be3d-774c58d7c56e"
root@Diskstation:~# 
  • Community Expert
13 minutes ago, Vonghofa said:
/dev/sdk1: PARTUUID="fd6ee637-33b0-426a-be3d-774c58d7c56e"

This is no good, it's not detecting the LUKS signature, meaning it wasn't just the partition that was missing; the disk was wiped/erased/cleared or similar.

The results from zdb -l are also not good:

zdb -l /dev/mapper/sdd1 txg: 2609501
zdb -l /dev/mapper/sde1 txg: 2601592
zdb -l /dev/mapper/sdf1 txg: 2609501
zdb -l /dev/mapper/sdg1 txg: 2609501
zdb -l /dev/mapper/sdh1 txg: 2609501
zdb -l /dev/mapper/sdi1 txg: 2609501
zdb -l /dev/mapper/sdj1 txg: 2601592
zdb -l /dev/mapper/sdk1 NO partition/FS SIG

Two of the devices are on a much older transaation group, if it was a difference of just a few txgs, you could try to rewind the pool, but each txg typically represents 5 seconds, so those devices are missing over 10H of transactions.

So there are two pool devices that are completely out of sync and sdk is invalid, so there are 3 bad devices, and raidz2 can only recover from 2, I'm afraid you will need to destroy the pool and recover from a backup, if there's no backup, you can try a file recovery app, but they are not cheap with ZFS pools.

  • Community Expert

Just for the hell of it, also post the output from:

zpool import -o readonly=on -f -F -n zfs.spinners

Most likely it won't be able to rewind, but it won't hurt to try.

  • Author

All right, thanks for your help and the information. Yes, I have a backup. First, I need to check what I set up for the backup back then and whether everything I need is there. Otherwise, I'll probably resort to klennet's solution or something similar and try to restore the pool with that. The price is high, but I haven't found anything cheaper. I'll run a test of the software to see what it can find and restore.

Thanks

  • Author

Unfortunately, the software cannot find any data :/ I have now entered the above command to rewind, but nothing seems to be happening. So no output and no read/write access to the hard drives.

I could send the hard drive that lost the partition to Seagate; maybe they can recover it. Data recovery is still included in the warranty.

  • Author
On 10/28/2025 at 8:19 PM, JorgeB said:

Just for the hell of it, also post the output from:

zpool import -o readonly=on -f -F -n zfs.spinners

Most likely it won't be able to rewind, but it won't hurt to try.

Unfortunately, the software cannot find any data :/ I have now entered the above command to rewind, but nothing seems to be happening. So no output and no read/write access to the hard drives.

I could send the hard drive that lost the partition to Seagate; maybe they can recover it. Data recovery is still included in the warranty.

  • Community Expert
53 minutes ago, Vonghofa said:

but nothing seems to be happening

Do you get the cursor back, or does it just keep running? In any case, post the output from tail -n 50 /proc/spl/kstat/zfs/dbgmsg after running the import command.

  • Author
root@Diskstation:~# zpool import -o readonly=on -f -F -n zfs.spinners
root@Diskstation:~# tail -n 50 /proc/spl/kstat/zfs/dbgmsg
1762706872   ffff8881d9258000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881d9258000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdb1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881d9258000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdc1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881d9258000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881d9258000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdd1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881d9258000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sde1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819dda90c0 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': vdev_validate: config guid doesn't match label guid
1762706872   ffff88819dda90c0 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': CONFIG: guid 10090287586978045426, top_guid 14800982098715042233
1762706872   ffff88819dda90c0 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': LABEL: guid 825157947067356691, top_guid 14800982098715042233, aux_guid 0
1762706872   ffff8881e5558000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': best uberblock found for spa $import-ffff8881e5558000-zfs.spinners. txg 2609501
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff8881e5558000-zfs.spinners, config untrusted): using uberblock with txg=2609501
1762706872   ffff8881e5558000 spa_misc.c:416:spa_load_failed(): spa_load($import-ffff8881e5558000-zfs.spinners, config untrusted): FAILED: couldn't get 'config' value in MOS directory [error=5]
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff8881e5558000-zfs.spinners, config untrusted): UNLOADING
1762706872   ffff8881e5558000 metaslab.c:1703:spa_set_allocator(): spa allocator: dynamic
1762706872   ffff8881e5558000 spa.c:6766:spa_import(): spa_import: importing zfs.spinners
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config trusted): LOADING
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdi1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdk1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdb1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdc1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdd1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sde1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cbc4300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819cb1d3c0 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': vdev_validate: config guid doesn't match label guid
1762706872   ffff88819cb1d3c0 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': CONFIG: guid 10090287586978045426, top_guid 14800982098715042233
1762706872   ffff88819cb1d3c0 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': LABEL: guid 825157947067356691, top_guid 14800982098715042233, aux_guid 0
1762706872   ffff8881e5558000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdk1': best uberblock found for spa zfs.spinners. txg 2609501
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config untrusted): using uberblock with txg=2609501
1762706872   ffff8881e5558000 spa_misc.c:416:spa_load_failed(): spa_load(zfs.spinners, config untrusted): FAILED: couldn't get 'config' value in MOS directory [error=5]
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config untrusted): UNLOADING
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config untrusted): spa_load_retry: rewind, max txg: 2609500
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config untrusted): LOADING
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdk1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdi1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdb1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdc1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdd1': probe done, cant_read=0 cant_write=1
1762706872   ffff8881e50d2180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sde1': probe done, cant_read=0 cant_write=1
1762706872   ffff88819e35b240 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': vdev_validate: config guid doesn't match label guid
1762706872   ffff88819e35b240 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': CONFIG: guid 10090287586978045426, top_guid 14800982098715042233
1762706872   ffff88819e35b240 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': LABEL: guid 825157947067356691, top_guid 14800982098715042233, aux_guid 0
1762706872   ffff8881e5558000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': best uberblock found for spa zfs.spinners. txg 2609492
1762706872   ffff8881e5558000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': label discarded as txg is too large (2609501 > 2609492)
1762706872   ffff8881e5558000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/mapper/sdj1': failed to read label config
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config untrusted): using uberblock with txg=2609492
1762706872   ffff8881e5558000 spa_misc.c:416:spa_load_failed(): spa_load(zfs.spinners, config untrusted): FAILED: label config unavailable
1762706872   ffff8881e5558000 spa_misc.c:430:spa_load_note(): spa_load(zfs.spinners, config untrusted): UNLOADING
root@Diskstation:~# 
  • Community Expert

It tried reqiendo to txg=2609492, but that is still too far away from the two devices with a much older one, I think data recovery app is the only possible route to try and recover some data from this pool.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.