Jump to content

Nokedr

Members
  • Posts

    7
  • Joined

  • Last visited

Posts posted by Nokedr

  1. After a power loss, I have not been able to start the array, i have isolated the problem to when the system is trying to mount the "mainzfs" pool and have successfullyu started the array with all drives in "mainzfs" unassigned

    Jul 24 19:08:07 KWUN emhttpd: shcmd (249): touch /boot/config/forcesync
    Jul 24 19:08:07 KWUN emhttpd: Mounting disks...
    Jul 24 19:08:07 KWUN emhttpd: mounting /mnt/disk1
    Jul 24 19:08:07 KWUN emhttpd: shcmd (250): mkdir -p /mnt/disk1
    Jul 24 19:08:07 KWUN emhttpd: shcmd (251): mount -t xfs -o noatime,nouuid /dev/md1p1 /mnt/disk1
    Jul 24 19:08:08 KWUN kernel: SGI XFS with ACLs, security attributes, no debug enabled
    Jul 24 19:08:08 KWUN kernel: XFS (md1p1): Mounting V5 Filesystem
    Jul 24 19:08:08 KWUN kernel: XFS (md1p1): Starting recovery (logdev: internal)
    Jul 24 19:08:08 KWUN kernel: XFS (md1p1): Ending recovery (logdev: internal)
    Jul 24 19:08:08 KWUN emhttpd: shcmd (252): xfs_growfs /mnt/disk1
    Jul 24 19:08:08 KWUN root: meta-data=/dev/md1p1             isize=512    agcount=4, agsize=7551902 blks
    Jul 24 19:08:08 KWUN root:          =                       sectsz=512   attr=2, projid32bit=1
    Jul 24 19:08:08 KWUN root:          =                       crc=1        finobt=1, sparse=1, rmapbt=0
    Jul 24 19:08:08 KWUN root:          =                       reflink=1    bigtime=1 inobtcount=1 nrext64=0
    Jul 24 19:08:08 KWUN root: data     =                       bsize=4096   blocks=30207606, imaxpct=25
    Jul 24 19:08:08 KWUN root:          =                       sunit=0      swidth=0 blks
    Jul 24 19:08:08 KWUN root: naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
    Jul 24 19:08:08 KWUN root: log      =internal log           bsize=4096   blocks=16384, version=2
    Jul 24 19:08:08 KWUN root:          =                       sectsz=512   sunit=0 blks, lazy-count=1
    Jul 24 19:08:08 KWUN root: realtime =none                   extsz=4096   blocks=0, rtextents=0
    Jul 24 19:08:08 KWUN emhttpd: mounting /mnt/big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (253): mkdir -p /mnt/big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (254): /usr/sbin/zpool import -N -o autoexpand=on  -d /dev/sdc1 -d /dev/sdd1 8947813539389484345 big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (255): /usr/sbin/zfs set mountpoint=/mnt/big_cache big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (256): /usr/sbin/zfs set atime=off big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (257): /usr/sbin/zfs mount big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (258): /usr/sbin/zpool set autotrim=on big_cache
    Jul 24 19:08:08 KWUN emhttpd: shcmd (259): /usr/sbin/zfs set compression=on big_cache
    Jul 24 19:08:08 KWUN emhttpd: /mnt/big_cache root profile: 
    Jul 24 19:08:08 KWUN emhttpd: /mnt/big_cache root groups: 2
    Jul 24 19:08:08 KWUN emhttpd: /mnt/big_cache root width: 1
    Jul 24 19:08:08 KWUN emhttpd: /usr/sbin/zpool status -PL big_cache 2>&1
    Jul 24 19:08:08 KWUN emhttpd:   pool: big_cache
    Jul 24 19:08:08 KWUN emhttpd:  state: ONLINE
    Jul 24 19:08:08 KWUN emhttpd: config:
    Jul 24 19:08:08 KWUN emhttpd:  NAME         STATE     READ WRITE CKSUM
    Jul 24 19:08:08 KWUN emhttpd:  big_cache    ONLINE       0     0     0
    Jul 24 19:08:08 KWUN emhttpd:    /dev/sdc1  ONLINE       0     0     0
    Jul 24 19:08:08 KWUN emhttpd:    /dev/sdd1  ONLINE       0     0     0
    Jul 24 19:08:08 KWUN emhttpd: errors: No known data errors
    Jul 24 19:08:08 KWUN emhttpd: mounting /mnt/mainzfs
    Jul 24 19:08:08 KWUN emhttpd: shcmd (260): mkdir -p /mnt/mainzfs
    Jul 24 19:08:08 KWUN emhttpd: /sbin/btrfs filesystem show /dev/sdj1 2>&1
    Jul 24 19:08:08 KWUN emhttpd: ERROR: no btrfs on /dev/sdj1
    Jul 24 19:08:08 KWUN emhttpd: /usr/sbin/zpool import -d /dev/sdj1 2>&1
    Jul 24 19:08:09 KWUN emhttpd:    pool: mainzfs
    Jul 24 19:08:09 KWUN emhttpd:      id: 17865510998351045675
    Jul 24 19:08:09 KWUN emhttpd: shcmd (261): /usr/sbin/zpool import -N -o autoexpand=on  -d /dev/sdj1 -d /dev/sdf1 -d /dev/sdg1 -d /dev/sdh1 -d /dev/sdm1 -d /dev/sdk1 -d /dev/sdi1 -d /dev/sdl1 17865510998351045675 mainzfs
    Jul 24 19:08:12 KWUN kernel: VERIFY3(range_tree_space(smla->smla_rt) + sme->sme_run <= smla->smla_sm->sm_size) failed (17237200896 <= 17179869184)
    Jul 24 19:08:12 KWUN kernel: PANIC at space_map.c:405:space_map_load_callback()
    Jul 24 19:08:12 KWUN kernel: Showing stack for process 6653
    Jul 24 19:08:12 KWUN kernel: CPU: 1 PID: 6653 Comm: z_wr_iss Tainted: P           O       6.1.38-Unraid #2
    Jul 24 19:08:12 KWUN kernel: Hardware name: Micro-Star International Co., Ltd MS-7C02/B450 TOMAHAWK MAX II (MS-7C02), BIOS H.90 07/25/2022
    Jul 24 19:08:12 KWUN kernel: Call Trace:
    Jul 24 19:08:12 KWUN kernel: <TASK>
    Jul 24 19:08:12 KWUN kernel: dump_stack_lvl+0x44/0x5c
    Jul 24 19:08:12 KWUN kernel: spl_panic+0xd0/0xe8 [spl]
    Jul 24 19:08:12 KWUN kernel: ? rs_get_start+0xc/0x1d [zfs]
    Jul 24 19:08:12 KWUN kernel: ? range_tree_stat_incr+0x28/0x43 [zfs]
    Jul 24 19:08:12 KWUN kernel: ? range_tree_remove_impl+0x3b7/0x406 [zfs]
    Jul 24 19:08:12 KWUN kernel: ? zio_wait+0x1ee/0x1fd [zfs]
    Jul 24 19:08:12 KWUN kernel: space_map_load_callback+0x50/0x79 [zfs]
    Jul 24 19:08:12 KWUN kernel: space_map_iterate+0x2d6/0x324 [zfs]
    Jul 24 19:08:12 KWUN kernel: ? spa_stats_destroy+0x16c/0x16c [zfs]
    Jul 24 19:08:12 KWUN kernel: space_map_load_length+0x93/0xcb [zfs]
    Jul 24 19:08:12 KWUN kernel: metaslab_load+0x33b/0x6e3 [zfs]
    Jul 24 19:08:12 KWUN kernel: ? slab_post_alloc_hook+0x4d/0x15e
    Jul 24 19:08:12 KWUN kernel: ? rs_get_start+0xc/0x1d [zfs]
    Jul 24 19:08:12 KWUN kernel: ? spl_kmem_alloc_impl+0xc1/0xf2 [spl]
    Jul 24 19:08:12 KWUN kernel: ? __kmem_cache_alloc_node+0x118/0x147
    Jul 24 19:08:12 KWUN kernel: metaslab_activate+0x36/0x1f1 [zfs]
    Jul 24 19:08:12 KWUN kernel: metaslab_alloc_dva+0x8bc/0xfce [zfs]
    Jul 24 19:08:12 KWUN kernel: ? preempt_latency_start+0x2b/0x46
    Jul 24 19:08:12 KWUN kernel: metaslab_alloc+0x107/0x1fd [zfs]
    Jul 24 19:08:12 KWUN kernel: zio_dva_allocate+0xee/0x73f [zfs]
    Jul 24 19:08:12 KWUN kernel: ? spl_kmem_alloc_impl+0xc1/0xf2 [spl]
    Jul 24 19:08:12 KWUN kernel: ? preempt_latency_start+0x2b/0x46
    Jul 24 19:08:12 KWUN kernel: ? _raw_spin_lock+0x13/0x1c
    Jul 24 19:08:12 KWUN kernel: ? _raw_spin_unlock+0x14/0x29
    Jul 24 19:08:12 KWUN kernel: ? tsd_hash_search+0x70/0x7d [spl]
    Jul 24 19:08:12 KWUN kernel: zio_execute+0xb4/0xdf [zfs]
    Jul 24 19:08:12 KWUN kernel: taskq_thread+0x269/0x38a [spl]
    Jul 24 19:08:12 KWUN kernel: ? wake_up_q+0x44/0x44
    Jul 24 19:08:12 KWUN kernel: ? zio_subblock+0x22/0x22 [zfs]
    Jul 24 19:08:12 KWUN kernel: ? taskq_dispatch_delay+0x106/0x106 [spl]
    Jul 24 19:08:12 KWUN kernel: kthread+0xe7/0xef
    Jul 24 19:08:12 KWUN kernel: ? kthread_complete_and_exit+0x1b/0x1b
    Jul 24 19:08:12 KWUN kernel: ret_from_fork+0x22/0x30
    Jul 24 19:08:12 KWUN kernel: </TASK>
    Jul 24 19:11:05 KWUN nginx: 2023/07/24 19:11:05 [error] 2375#2375: *1071 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.2.2, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket:/update.htm", host: "kwun.local", referrer: "http://kwun.local/Main"

     

    root@KWUN:~# zpool import
       pool: mainzfs
         id: 17865510998351045675
      state: ONLINE
     action: The pool can be imported using its name or numeric identifier.
     config:
    
            mainzfs     ONLINE
              raidz1-0  ONLINE
                sdj1    ONLINE
                sdf1    ONLINE
                sdg1    ONLINE
                sdh1    ONLINE
              raidz1-1  ONLINE
                sdm1    ONLINE
                sdk1    ONLINE
                sdi1    ONLINE
                sdl1    ONLINE

     

    I could not get the diagnostics while it was stuck in mounting but here is the diagnostics with the unassigned disks

    I have snapshots, are they usable?

    Can anyone help me fix this problem please?

    kwun-diagnostics-20230724-2027.zip

  2. After messing up an assigning my only gpu to a vm i had to hard power off. Upon rebooting one of my disk was disabled, I followed the storage management on the manual to check / repair a disabled disk

    Phase 1 - find and verify superblock...
    Phase 2 - using internal log
            - zero log...
            - scan filesystem freespace and inode maps...
            - found root inode chunk
    Phase 3 - for each AG...
            - scan and clear agi unlinked lists...
            - process known inodes and perform inode discovery...
            - agno = 0
            - agno = 1
            - agno = 2
            - agno = 3
            - agno = 4
            - process newly discovered inodes...
    Phase 4 - check for duplicate blocks...
            - setting up duplicate extent list...
            - check for inodes claiming duplicate blocks...
            - agno = 1
            - agno = 3
            - agno = 2
            - agno = 4
            - agno = 0
    Phase 5 - rebuild AG headers and trees...
            - reset superblock...
    Phase 6 - check inode connectivity...
            - resetting contents of realtime bitmap and summary inodes
            - traversing filesystem ...
            - traversal finished ...
            - moving disconnected inodes to lost+found ...
    Phase 7 - verify and correct link counts...
    done

    Restarting the array normally still showed the disk as disabled, the .trash file on the disk was also empty

  3. Hey @GonjaT,

     

    I've been running your RagnamodVI, and it's been great, I just had one question:

    Is there a way to issue server commands? As in "list" or "op" or "say" through the server terminal.

    The log doesn't seem to respond to anything I type, and the console is a system console and not the one of the server, so I can find the server process, but I don't know how to access it.

     

    Thanks in advance for any help you can give.

    Nokedr

     

×
×
  • Create New...