macmanluke

Members
  • Posts

    64
  • Joined

  • Last visited

Posts posted by macmanluke

  1. Over the week end I got 4 read errors on one of my drives 
    It is one of the older drives in the system with 30273 power on hours (WD Red 6TB)

    Has since passed a extended smart test and no more errors

    Following error in syslog looks to me more like a disk problem than cabling (which has not been touched/moved in 6-12months)
     

    Nov 13 04:36:23 Vault kernel: ata4.00: exception Emask 0x0 SAct 0x2100 SErr 0x0 action 0x0
    Nov 13 04:36:23 Vault kernel: ata4.00: irq_stat 0x40000008
    Nov 13 04:36:23 Vault kernel: ata4.00: failed command: READ FPDMA QUEUED
    Nov 13 04:36:23 Vault kernel: ata4.00: cmd 60/c0:40:f8:b9:16/00:00:be:00:00/40 tag 8 ncq dma 98304 in
    Nov 13 04:36:23 Vault kernel:         res 41/40:00:98:ba:16/00:00:be:00:00/00 Emask 0x409 (media error) <F>
    Nov 13 04:36:23 Vault kernel: ata4.00: status: { DRDY ERR }
    Nov 13 04:36:23 Vault kernel: ata4.00: error: { UNC }
    Nov 13 04:36:23 Vault kernel: ata4.00: configured for UDMA/133
    Nov 13 04:36:23 Vault kernel: sd 4:0:0:0: [sde] tag#8 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=DRIVER_OK cmd_age=4s
    Nov 13 04:36:23 Vault kernel: sd 4:0:0:0: [sde] tag#8 Sense Key : 0x3 [current] 
    Nov 13 04:36:23 Vault kernel: sd 4:0:0:0: [sde] tag#8 ASC=0x11 ASCQ=0x4 
    Nov 13 04:36:23 Vault kernel: sd 4:0:0:0: [sde] tag#8 CDB: opcode=0x88 88 00 00 00 00 00 be 16 b9 f8 00 00 00 c0 00 00
    Nov 13 04:36:23 Vault kernel: I/O error, dev sde, sector 3189160600 op 0x0:(READ) flags 0x0 phys_seg 4 prio class 0
    Nov 13 04:36:23 Vault kernel: md: disk1 read error, sector=3189160536
    Nov 13 04:36:23 Vault kernel: md: disk1 read error, sector=3189160544
    Nov 13 04:36:23 Vault kernel: md: disk1 read error, sector=3189160552
    Nov 13 04:36:23 Vault kernel: md: disk1 read error, sector=3189160560
    Nov 13 04:36:23 Vault kernel: ata4: EH complete


    Thinking I should still swap out the drive?  thoughts?


     

  2. 24 minutes ago, ggfools said:

    The Tailscale Plugin keeps causing me to completely lose access to the unraid web UI as soon as I start my array, if I remove the plugin then it immediately works again, I can re-add the tailscale plugin and it will work ok for a while (week or 2) then I have the same problem again. I'm on unraid 6.12.3 still which I suppose may be part of the problem, don't think I can get the logs since i've had to uninstall the plugin so I could access the UI so unfortunately I cannot provide those, i'll try to next time if it happens again. I have attached a syslog if that helps at all, at the end you can see a ton of "SIGKILL" errors, and as soon as I remove the tailscale plugin everything goes back to normal.

    syslog-192.168.1.8.log 1.3 MB · 0 downloads

     

    I have also been loosing my GUI this week and use Tailscale, nothing useful in my syslog though

  3. Hello all

    Came home tonight to my unraid box playing up First noticed my home assistant seemed to not be functioning Logged in and found a bunch of dockers stopped that are normally running (unfortunately didn't think to take notice of what was and was not. Most were not running anymore)

    Noticed the CPU was maxed and top revealed a process 'node' (x12) maxing the CPU https://imgur.com/a/hAsl3M8

     

    Stopped docker trying to narrow it down and that stopped the CPU usage

    but overall the server was acting up - all my shares were not showing up in the UI etc

    Rebooted and all appears good but like to find out what happened

     

    Any suggestions? not sure what would cause it

    List of dockers I run: https://imgur.com/a/Srul7nx

    Only thing in Tailscale before the reboot and the usual nightly stuff is from Tailscale but does not look like anything of concern

    Still running 6.11.5

     

    thanks!

  4. Well its been ~10 days and still up (best so far in a while)

    Only change i made this time was unplugging the monitor from the integrated graphics - not the first time iv had this issue
    Had forgotten about it and had only plugged i back in to troubleshoot the previous crashing issue

    So i think i had 2 problems:

    1) macvlan crashes that appeared out of no where. 

    2) monitor plugged into intel igpu causes crashes. Have had the issue on 2 seperate systems since 6.9 or 6.10 (was fine in 6.8 or 6.9, cant remember when it started)

  5. after updating to the latest version on one of my machines im getting

     

    Quote

     

    /usr/local/emhttp/plugins/tailscale/update-settings.sh: line 3: /boot/config/plugins/tailscale/tailscale.cfg: No such file or directory
    Taildrop not configured or share not available, disabling
    rm: cannot remove '/etc/config/uLinux.conf': No such file or directory
    Stopping tailscaled.
    Starting tailscaled:  /usr/local/sbin/tailscaled
     

     

    Tried removing and reinstalling no luck. Is on 6.12rc2
    Other machine is fine (but 6.11.5)

  6. My unraid server started crashing approx 1 month ago
    Its crashing on average once every 2-7 days.

    Has been super stable for 1 year plus prior. No new hardware additions.
    Last software update was 6.11.5 around the time that came out
    Intel 10400, z590, 32GB DDR4 no XMP.

    iGPU used for plex
    RTX3060 passed to VM (thats not been running during this time)

    Syslog has either had nothing around time of crash, or btrfs errors (which i think is more a symptom of the crash than the cause)
    See thread below: 


    Have run memtest on the ram that was in the server (24hrs, 16 passes) all passed
    Ram swapped with 32gb from my desktop. Still crashed.

    Latest crash was this morning at 7:13am
    (only know time because my security server notifies me when it comes backup when it went down)
    Last syslog entries before i rebooted it were at 3:30am from CA Backup running and completing.
    Nothing much is happening on the server at 7:13

    Any suggestions before i start swapping out more hardware?
    lack of anything in syslogs makes troubleshooting a nightmare

    Diags attached
     

     

  7. Im starting to think its related to macvlan which for some reason has suddenly become an issue on both machines after being fine with the same config for years.

    Problem is if i swap to ipvlan it causes weird connectivity issues with connectivity coming and going from the unraid box to the internet (lan connectivity seems fine)
    Box at home is 95% functional in ipvlan mode, opnsense router seems to handle it ok but id does have issues eg app store drops in and out.
    Box at work is very not functional in ipvlan mode but thats connected to an old ubiquiti edge router that does not seem to handle it as well.


    Any suggestions to get functional network connectivity in ipvlan mode?
    Still not 100% confirmed thats the issue but being i need to wait a week to have any confidence.

    or should i start a new thread about that?

  8. 18 hours ago, JorgeB said:

    Macvlan call traces are usually the result of having dockers with a custom IP address and will end up crashing the server, upgrading to v6.10 or later and switching to ipvlan should fix it (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right))

    This is something i just picked up on after fix common problems started alerting about it
    What has changed to make that an issue? iv had dockers with custom IPs since i started using unraid in 6.8 which was default to macvlan

    all the way up to this week when fix common problems warned me (and iv been having crashing issues lately but previously months of stability etc)
     

  9. Ok this is very strange

    My other system at work has done basically the exact same issue...
    Similar setup but only a single drive cache. Far less dockers running and a single VM.

    leading me to think its a software/plugin something issue

    This system is running Version: 6.12.0-rc2 as of a few days ago (might revert it back just in case...)
     

    Quote

    Mar 28 10:22:56 NAS kernel: BTRFS critical (device nvme0n1p1): corrupt leaf: block=389159174144 slot=6 extent bytenr=401120174080 len=4096 invalid generation, have 9313056887493137007 expect (0, 1193044]
    Mar 28 10:22:56 NAS kernel: BTRFS info (device nvme0n1p1): leaf 389159174144 gen 1193043 total ptrs 183 free space 2025 owner 2
    Mar 28 10:22:56 NAS kernel:     item 0 key (401120129024 168 4096) itemoff 16230 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697035 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853249024 count 1
    Mar 28 10:22:56 NAS kernel:     item 1 key (401120133120 168 16384) itemoff 16177 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 471887 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 7150911488 count 1
    Mar 28 10:22:56 NAS kernel:     item 2 key (401120149504 168 8192) itemoff 16124 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561771 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577906688 count 1
    Mar 28 10:22:56 NAS kernel:     item 3 key (401120157696 168 4096) itemoff 16071 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561772 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577918976 count 1
    Mar 28 10:22:56 NAS kernel:     item 4 key (401120161792 168 4096) itemoff 16018 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561773 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577923072 count 1
    Mar 28 10:22:56 NAS kernel:     item 5 key (401120165888 168 8192) itemoff 15965 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561774 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577927168 count 1
    Mar 28 10:22:56 NAS kernel:     item 6 key (401120174080 168 4096) itemoff 15912 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 9313056887493137007 flags 16777096
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577935360 count 1
    Mar 28 10:22:56 NAS kernel:     item 7 key (401120178176 168 4096) itemoff 15859 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561775 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577939456 count 1
    Mar 28 10:22:56 NAS kernel:     item 8 key (401120182272 168 4096) itemoff 15806 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561775 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577943552 count 1
    Mar 28 10:22:56 NAS kernel:     item 9 key (401120186368 168 8192) itemoff 15753 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561777 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577947648 count 1
    Mar 28 10:22:56 NAS kernel:     item 10 key (401120194560 168 4096) itemoff 15700 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561777 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577955840 count 1
    Mar 28 10:22:56 NAS kernel:     item 11 key (401120198656 168 4096) itemoff 15647 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561778 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577959936 count 1
    Mar 28 10:22:56 NAS kernel:     item 12 key (401120202752 168 8192) itemoff 15594 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561779 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577964032 count 1
    Mar 28 10:22:56 NAS kernel:     item 13 key (401120210944 168 4096) itemoff 15541 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561780 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577972224 count 1
    Mar 28 10:22:56 NAS kernel:     item 14 key (401120215040 168 4096) itemoff 15488 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697035 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853253120 count 1
    Mar 28 10:22:56 NAS kernel:     item 15 key (401120219136 168 8192) itemoff 15435 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478586 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2781671424 count 1
    Mar 28 10:22:56 NAS kernel:     item 16 key (401120227328 168 4096) itemoff 15382 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478586 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 17277304832 count 1
    Mar 28 10:22:56 NAS kernel:     item 17 key (401120231424 168 4096) itemoff 15329 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478586 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 17277308928 count 1
    Mar 28 10:22:56 NAS kernel:     item 18 key (401120235520 168 4096) itemoff 15276 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478587 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2756775936 count 1
    Mar 28 10:22:56 NAS kernel:     item 19 key (401120239616 168 8192) itemoff 15223 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478587 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2781679616 count 1
    Mar 28 10:22:56 NAS kernel:     item 20 key (401120247808 168 4096) itemoff 15170 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478587 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2729680896 count 1
    Mar 28 10:22:56 NAS kernel:     item 21 key (401120251904 168 36864) itemoff 15117 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478555 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728398848 count 1
    Mar 28 10:22:56 NAS kernel:     item 22 key (401120288768 168 32768) itemoff 15064 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478556 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728456192 count 1
    Mar 28 10:22:56 NAS kernel:     item 23 key (401120321536 168 32768) itemoff 15011 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478557 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728505344 count 1
    Mar 28 10:22:56 NAS kernel:     item 24 key (401120354304 168 36864) itemoff 14958 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478559 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728546304 count 1
    Mar 28 10:22:56 NAS kernel:     item 25 key (401120391168 168 36864) itemoff 14905 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478560 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728599552 count 1
    Mar 28 10:22:56 NAS kernel:     item 26 key (401120428032 168 32768) itemoff 14852 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478561 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728648704 count 1
    Mar 28 10:22:56 NAS kernel:     item 27 key (401120460800 168 36864) itemoff 14799 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478562 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728693760 count 1
    Mar 28 10:22:56 NAS kernel:     item 28 key (401120497664 168 36864) itemoff 14746 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478564 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728751104 count 1
    Mar 28 10:22:56 NAS kernel:     item 29 key (401120534528 168 8192) itemoff 14693 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478564 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728787968 count 1
    Mar 28 10:22:56 NAS kernel:     item 30 key (401120542720 168 12288) itemoff 14640 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478566 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2781515776 count 1
    Mar 28 10:22:56 NAS kernel:     item 31 key (401120555008 168 8192) itemoff 14587 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478566 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728882176 count 1
    Mar 28 10:22:56 NAS kernel:     item 32 key (401120563200 168 4096) itemoff 14534 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478572 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2729095168 count 1
    Mar 28 10:22:56 NAS kernel:     item 33 key (401120567296 168 8192) itemoff 14481 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561780 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577976320 count 1
    Mar 28 10:22:56 NAS kernel:     item 34 key (401120575488 168 8192) itemoff 14428 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561781 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577984512 count 1
    Mar 28 10:22:56 NAS kernel:     item 35 key (401120583680 168 4096) itemoff 14375 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561782 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577992704 count 1
    Mar 28 10:22:56 NAS kernel:     item 36 key (401120587776 168 4096) itemoff 14322 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561783 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20577996800 count 1
    Mar 28 10:22:56 NAS kernel:     item 37 key (401120591872 168 4096) itemoff 14269 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561783 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578000896 count 1
    Mar 28 10:22:56 NAS kernel:     item 38 key (401120595968 168 4096) itemoff 14216 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561784 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578004992 count 1
    Mar 28 10:22:56 NAS kernel:     item 39 key (401120600064 168 4096) itemoff 14163 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561784 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578009088 count 1
    Mar 28 10:22:56 NAS kernel:     item 40 key (401120604160 168 8192) itemoff 14110 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561785 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578013184 count 1
    Mar 28 10:22:56 NAS kernel:     item 41 key (401120612352 168 4096) itemoff 14057 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561786 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578021376 count 1
    Mar 28 10:22:56 NAS kernel:     item 42 key (401120616448 168 204800) itemoff 14004 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 497529 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20537409536 count 1
    Mar 28 10:22:56 NAS kernel:     item 43 key (401120821248 168 155648) itemoff 13951 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 497529 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20545798144 count 1
    Mar 28 10:22:56 NAS kernel:     item 44 key (401120976896 168 4096) itemoff 13898 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561006 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572012544 count 1
    Mar 28 10:22:56 NAS kernel:     item 45 key (401120980992 168 4096) itemoff 13845 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561006 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572016640 count 1
    Mar 28 10:22:56 NAS kernel:     item 46 key (401120985088 168 8192) itemoff 13792 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561007 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572020736 count 1
    Mar 28 10:22:56 NAS kernel:     item 47 key (401120993280 168 8192) itemoff 13739 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561009 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572028928 count 1
    Mar 28 10:22:56 NAS kernel:     item 48 key (401121001472 168 4096) itemoff 13686 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561010 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572037120 count 1
    Mar 28 10:22:56 NAS kernel:     item 49 key (401121005568 168 4096) itemoff 13633 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561014 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572065792 count 1
    Mar 28 10:22:56 NAS kernel:     item 50 key (401121009664 168 4096) itemoff 13580 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697036 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853289984 count 1
    Mar 28 10:22:56 NAS kernel:     item 51 key (401121013760 168 8192) itemoff 13527 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478563 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2728742912 count 1
    Mar 28 10:22:56 NAS kernel:     item 52 key (401121021952 168 8192) itemoff 13474 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478564 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2781507584 count 1
    Mar 28 10:22:56 NAS kernel:     item 53 key (401121030144 168 69632) itemoff 13421 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 497529 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20549599232 count 1
    Mar 28 10:22:56 NAS kernel:     item 54 key (401121099776 168 8192) itemoff 13368 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561011 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572041216 count 1
    Mar 28 10:22:56 NAS kernel:     item 55 key (401121107968 168 8192) itemoff 13315 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561011 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572049408 count 1
    Mar 28 10:22:56 NAS kernel:     item 56 key (401121116160 168 12288) itemoff 13262 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697033 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853191680 count 1
    Mar 28 10:22:56 NAS kernel:     item 57 key (401121128448 168 135168) itemoff 13209 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 497529 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20546584576 count 1
    Mar 28 10:22:56 NAS kernel:     item 58 key (401121263616 168 8192) itemoff 13156 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561012 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572057600 count 1
    Mar 28 10:22:56 NAS kernel:     item 59 key (401121271808 168 4096) itemoff 13103 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561014 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572069888 count 1
    Mar 28 10:22:56 NAS kernel:     item 60 key (401121275904 168 4096) itemoff 13050 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561014 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572073984 count 1
    Mar 28 10:22:56 NAS kernel:     item 61 key (401121280000 168 4096) itemoff 12997 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697036 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853294080 count 1
    Mar 28 10:22:56 NAS kernel:     item 62 key (401121284096 168 4096) itemoff 12944 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697036 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853298176 count 1
    Mar 28 10:22:56 NAS kernel:     item 63 key (401121288192 168 24576) itemoff 12891 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1243807744 count 1
    Mar 28 10:22:56 NAS kernel:     item 64 key (401121312768 168 32768) itemoff 12838 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 566929 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 21213986816 count 1
    Mar 28 10:22:56 NAS kernel:     item 65 key (401121345536 168 16384) itemoff 12785 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 471887 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 7150927872 count 1
    Mar 28 10:22:56 NAS kernel:     item 66 key (401121361920 168 4096) itemoff 12732 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697037 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853302272 count 1
    Mar 28 10:22:56 NAS kernel:     item 67 key (401121366016 168 4096) itemoff 12679 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697038 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853351424 count 1
    Mar 28 10:22:56 NAS kernel:     item 68 key (401121370112 168 4096) itemoff 12626 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561786 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578025472 count 1
    Mar 28 10:22:56 NAS kernel:     item 69 key (401121374208 168 4096) itemoff 12573 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561787 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578029568 count 1
    Mar 28 10:22:56 NAS kernel:     item 70 key (401121378304 168 8192) itemoff 12520 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561788 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578033664 count 1
    Mar 28 10:22:56 NAS kernel:     item 71 key (401121386496 168 8192) itemoff 12467 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561789 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578041856 count 1
    Mar 28 10:22:56 NAS kernel:     item 72 key (401121394688 168 4096) itemoff 12414 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561790 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578050048 count 1
    Mar 28 10:22:56 NAS kernel:     item 73 key (401121398784 168 4096) itemoff 12361 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561790 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578054144 count 1
    Mar 28 10:22:56 NAS kernel:     item 74 key (401121402880 168 4096) itemoff 12308 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561791 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578058240 count 1
    Mar 28 10:22:56 NAS kernel:     item 75 key (401121406976 168 4096) itemoff 12255 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561791 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578062336 count 1
    Mar 28 10:22:56 NAS kernel:     item 76 key (401121411072 168 4096) itemoff 12202 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561792 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578066432 count 1
    Mar 28 10:22:56 NAS kernel:     item 77 key (401121415168 168 8192) itemoff 12149 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561793 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578070528 count 1
    Mar 28 10:22:56 NAS kernel:     item 78 key (401121423360 168 4096) itemoff 12096 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561794 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578078720 count 1
    Mar 28 10:22:56 NAS kernel:     item 79 key (401121427456 168 8192) itemoff 12043 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561794 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578082816 count 1
    Mar 28 10:22:56 NAS kernel:     item 80 key (401121435648 168 8192) itemoff 11990 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561795 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578091008 count 1
    Mar 28 10:22:56 NAS kernel:     item 81 key (401121443840 168 8192) itemoff 11937 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561796 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578099200 count 1
    Mar 28 10:22:56 NAS kernel:     item 82 key (401121452032 168 4096) itemoff 11884 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561798 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578107392 count 1
    Mar 28 10:22:56 NAS kernel:     item 83 key (401121456128 168 8192) itemoff 11831 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561798 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578111488 count 1
    Mar 28 10:22:56 NAS kernel:     item 84 key (401121464320 168 4096) itemoff 11778 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561799 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578119680 count 1
    Mar 28 10:22:56 NAS kernel:     item 85 key (401121468416 168 4096) itemoff 11725 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561799 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578123776 count 1
    Mar 28 10:22:56 NAS kernel:     item 86 key (401121472512 168 4096) itemoff 11672 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561800 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578127872 count 1
    Mar 28 10:22:56 NAS kernel:     item 87 key (401121476608 168 4096) itemoff 11619 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561800 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578131968 count 1
    Mar 28 10:22:56 NAS kernel:     item 88 key (401121480704 168 4096) itemoff 11566 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561801 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578136064 count 1
    Mar 28 10:22:56 NAS kernel:     item 89 key (401121484800 168 8192) itemoff 11513 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561802 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578140160 count 1
    Mar 28 10:22:56 NAS kernel:     item 90 key (401121492992 168 8192) itemoff 11460 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561803 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578148352 count 1
    Mar 28 10:22:56 NAS kernel:     item 91 key (401121501184 168 4096) itemoff 11407 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561804 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578156544 count 1
    Mar 28 10:22:56 NAS kernel:     item 92 key (401121505280 168 4096) itemoff 11354 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561804 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578160640 count 1
    Mar 28 10:22:56 NAS kernel:     item 93 key (401121509376 168 4096) itemoff 11301 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561805 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578164736 count 1
    Mar 28 10:22:56 NAS kernel:     item 94 key (401121513472 168 8192) itemoff 11248 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561805 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578168832 count 1
    Mar 28 10:22:56 NAS kernel:     item 95 key (401121521664 168 4096) itemoff 11195 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561806 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578177024 count 1
    Mar 28 10:22:56 NAS kernel:     item 96 key (401121525760 168 4096) itemoff 11142 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561807 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578181120 count 1
    Mar 28 10:22:56 NAS kernel:     item 97 key (401121529856 168 4096) itemoff 11089 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561808 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578185216 count 1
    Mar 28 10:22:56 NAS kernel:     item 98 key (401121533952 168 8192) itemoff 11036 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561808 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578189312 count 1
    Mar 28 10:22:56 NAS kernel:     item 99 key (401121542144 168 8192) itemoff 10983 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578197504 count 1
    Mar 28 10:22:56 NAS kernel:     item 100 key (401121550336 168 4096) itemoff 10930 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561810 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578205696 count 1
    Mar 28 10:22:56 NAS kernel:     item 101 key (401121554432 168 8192) itemoff 10877 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561811 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578209792 count 1
    Mar 28 10:22:56 NAS kernel:     item 102 key (401121562624 168 8192) itemoff 10824 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561812 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578217984 count 1
    Mar 28 10:22:56 NAS kernel:     item 103 key (401121570816 168 8192) itemoff 10771 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561813 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578226176 count 1
    Mar 28 10:22:56 NAS kernel:     item 104 key (401121579008 168 8192) itemoff 10718 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561814 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578234368 count 1
    Mar 28 10:22:56 NAS kernel:     item 105 key (401121587200 168 4096) itemoff 10665 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561815 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578242560 count 1
    Mar 28 10:22:56 NAS kernel:     item 106 key (401121591296 168 8192) itemoff 10612 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561816 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578246656 count 1
    Mar 28 10:22:56 NAS kernel:     item 107 key (401121599488 168 4096) itemoff 10559 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561816 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578254848 count 1
    Mar 28 10:22:56 NAS kernel:     item 108 key (401121603584 168 4096) itemoff 10506 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561817 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578258944 count 1
    Mar 28 10:22:56 NAS kernel:     item 109 key (401121607680 168 4096) itemoff 10453 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561817 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578263040 count 1
    Mar 28 10:22:56 NAS kernel:     item 110 key (401121611776 168 4096) itemoff 10400 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561817 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578267136 count 1
    Mar 28 10:22:56 NAS kernel:     item 111 key (401121615872 168 8192) itemoff 10347 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561818 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578271232 count 1
    Mar 28 10:22:56 NAS kernel:     item 112 key (401121624064 168 8192) itemoff 10294 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561820 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578279424 count 1
    Mar 28 10:22:56 NAS kernel:     item 113 key (401121632256 168 4096) itemoff 10241 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561821 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578287616 count 1
    Mar 28 10:22:56 NAS kernel:     item 114 key (401121636352 168 4096) itemoff 10188 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561821 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578291712 count 1
    Mar 28 10:22:56 NAS kernel:     item 115 key (401121640448 168 4096) itemoff 10135 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561822 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578295808 count 1
    Mar 28 10:22:56 NAS kernel:     item 116 key (401121644544 168 4096) itemoff 10082 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561822 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20578299904 count 1
    Mar 28 10:22:56 NAS kernel:     item 117 key (401121648640 168 8192) itemoff 10029 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697038 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853355520 count 1
    Mar 28 10:22:56 NAS kernel:     item 118 key (401121656832 168 4096) itemoff 9976 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 562778 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20585508864 count 1
    Mar 28 10:22:56 NAS kernel:     item 119 key (401121660928 168 4096) itemoff 9923 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697038 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853392384 count 1
    Mar 28 10:22:56 NAS kernel:     item 120 key (401121665024 168 4096) itemoff 9870 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697039 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853396480 count 1
    Mar 28 10:22:56 NAS kernel:     item 121 key (401121673216 168 8192) itemoff 9817 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 562777 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20585496576 count 1
    Mar 28 10:22:56 NAS kernel:     item 122 key (401121681408 168 8192) itemoff 9764 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 562779 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20585512960 count 1
    Mar 28 10:22:56 NAS kernel:     item 123 key (401121689600 168 4096) itemoff 9711 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697039 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853400576 count 1
    Mar 28 10:22:56 NAS kernel:     item 124 key (401121693696 168 4096) itemoff 9658 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697039 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853404672 count 1
    Mar 28 10:22:56 NAS kernel:     item 125 key (401121697792 168 4096) itemoff 9605 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182830 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 8447094784 count 1
    Mar 28 10:22:56 NAS kernel:     item 126 key (401121701888 168 4096) itemoff 9552 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697039 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2850439168 count 1
    Mar 28 10:22:56 NAS kernel:     item 127 key (401121714176 168 4096) itemoff 9499 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561015 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572078080 count 1
    Mar 28 10:22:56 NAS kernel:     item 128 key (401121718272 168 4096) itemoff 9446 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561015 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572082176 count 1
    Mar 28 10:22:56 NAS kernel:     item 129 key (401121722368 168 4096) itemoff 9393 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561016 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572086272 count 1
    Mar 28 10:22:56 NAS kernel:     item 130 key (401121726464 168 8192) itemoff 9340 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561016 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572090368 count 1
    Mar 28 10:22:56 NAS kernel:     item 131 key (401121734656 168 4096) itemoff 9287 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561017 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572098560 count 1
    Mar 28 10:22:56 NAS kernel:     item 132 key (401121738752 168 8192) itemoff 9234 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561018 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572102656 count 1
    Mar 28 10:22:56 NAS kernel:     item 133 key (401121746944 168 16384) itemoff 9181 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 567367 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 21240987648 count 1
    Mar 28 10:22:56 NAS kernel:     item 134 key (401121763328 168 4096) itemoff 9128 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697040 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853445632 count 1
    Mar 28 10:22:56 NAS kernel:     item 135 key (401121767424 168 98304) itemoff 9075 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1192903 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 320159744 count 1
    Mar 28 10:22:56 NAS kernel:     item 136 key (401121865728 168 65536) itemoff 9022 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1193043 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 320126976 count 1
    Mar 28 10:22:56 NAS kernel:     item 137 key (401121935360 168 32768) itemoff 8969 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182831 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 7435976704 count 1
    Mar 28 10:22:56 NAS kernel:     item 138 key (401121968128 168 8192) itemoff 8916 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182830 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 4736851968 count 1
    Mar 28 10:22:56 NAS kernel:     item 139 key (401122000896 168 40960) itemoff 8863 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1244200960 count 1
    Mar 28 10:22:56 NAS kernel:     item 140 key (401122050048 168 49152) itemoff 8810 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1190888 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 10972266496 count 1
    Mar 28 10:22:56 NAS kernel:     item 141 key (401122131968 168 16384) itemoff 8757 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1244475392 count 1
    Mar 28 10:22:56 NAS kernel:     item 142 key (401122168832 168 12288) itemoff 8704 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182830 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 3312988160 count 1
    Mar 28 10:22:56 NAS kernel:     item 143 key (401122181120 168 4096) itemoff 8651 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561019 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572110848 count 1
    Mar 28 10:22:56 NAS kernel:     item 144 key (401122185216 168 4096) itemoff 8598 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561019 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572114944 count 1
    Mar 28 10:22:56 NAS kernel:     item 145 key (401122189312 168 4096) itemoff 8545 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561019 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572119040 count 1
    Mar 28 10:22:56 NAS kernel:     item 146 key (401122193408 168 4096) itemoff 8492 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561020 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572123136 count 1
    Mar 28 10:22:56 NAS kernel:     item 147 key (401122197504 168 4096) itemoff 8439 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561021 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572127232 count 1
    Mar 28 10:22:56 NAS kernel:     item 148 key (401122201600 168 4096) itemoff 8386 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561021 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572131328 count 1
    Mar 28 10:22:56 NAS kernel:     item 149 key (401122205696 168 4096) itemoff 8333 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561021 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572135424 count 1
    Mar 28 10:22:56 NAS kernel:     item 150 key (401122209792 168 4096) itemoff 8280 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561023 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572147712 count 1
    Mar 28 10:22:56 NAS kernel:     item 151 key (401122213888 168 8192) itemoff 8227 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697039 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853437440 count 1
    Mar 28 10:22:56 NAS kernel:     item 152 key (401122222080 168 28672) itemoff 8174 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1243553792 count 1
    Mar 28 10:22:56 NAS kernel:     item 153 key (401122250752 168 4096) itemoff 8121 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182830 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 8329170944 count 1
    Mar 28 10:22:56 NAS kernel:     item 154 key (401122254848 168 4096) itemoff 8068 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 489287 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20087255040 count 1
    Mar 28 10:22:56 NAS kernel:     item 155 key (401122258944 168 4096) itemoff 8015 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478840 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2737500160 count 1
    Mar 28 10:22:56 NAS kernel:     item 156 key (401122263040 168 8192) itemoff 7962 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 478840 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2747977728 count 1
    Mar 28 10:22:56 NAS kernel:     item 157 key (401122271232 168 4096) itemoff 7909 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 471887 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 17253982208 count 1
    Mar 28 10:22:56 NAS kernel:     item 158 key (401122275328 168 4096) itemoff 7856 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 471887 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 9984004096 count 1
    Mar 28 10:22:56 NAS kernel:     item 159 key (401122279424 168 4096) itemoff 7819 itemsize 37
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 471887 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: shared data backref parent 389535547392 count 1
    Mar 28 10:22:56 NAS kernel:     item 160 key (401122283520 168 4096) itemoff 7766 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697040 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853449728 count 1
    Mar 28 10:22:56 NAS kernel:     item 161 key (401122287616 168 28672) itemoff 7713 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1243656192 count 1
    Mar 28 10:22:56 NAS kernel:     item 162 key (401122316288 168 4096) itemoff 7660 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182830 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 8191340544 count 1
    Mar 28 10:22:56 NAS kernel:     item 163 key (401122320384 168 4096) itemoff 7607 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1012911 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 7040253952 count 1
    Mar 28 10:22:56 NAS kernel:     item 164 key (401122324480 168 28672) itemoff 7554 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1244303360 count 1
    Mar 28 10:22:56 NAS kernel:     item 165 key (401122353152 168 24576) itemoff 7501 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 2628239360 count 1
    Mar 28 10:22:56 NAS kernel:     item 166 key (401122377728 168 4096) itemoff 7448 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1182830 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 7860662272 count 1
    Mar 28 10:22:56 NAS kernel:     item 167 key (401122381824 168 4096) itemoff 7395 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 474455 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 10583728128 count 1
    Mar 28 10:22:56 NAS kernel:     item 168 key (401122385920 168 4096) itemoff 7342 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 474470 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 10583732224 count 1
    Mar 28 10:22:56 NAS kernel:     item 169 key (401122390016 168 20480) itemoff 7289 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1244266496 count 1
    Mar 28 10:22:56 NAS kernel:     item 170 key (401122410496 168 4096) itemoff 7236 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 479158 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2739933184 count 1
    Mar 28 10:22:56 NAS kernel:     item 171 key (401122414592 168 4096) itemoff 7183 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 479160 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2739953664 count 1
    Mar 28 10:22:56 NAS kernel:     item 172 key (401122418688 168 4096) itemoff 7130 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 477880 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 17257160704 count 1
    Mar 28 10:22:56 NAS kernel:     item 173 key (401122422784 168 4096) itemoff 7077 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697040 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853453824 count 1
    Mar 28 10:22:56 NAS kernel:     item 174 key (401122426880 168 28672) itemoff 7024 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1243725824 count 1
    Mar 28 10:22:56 NAS kernel:     item 175 key (401122471936 168 4096) itemoff 6971 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697041 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853490688 count 1
    Mar 28 10:22:56 NAS kernel:     item 176 key (401122476032 168 24576) itemoff 6918 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 1171809 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 465 offset 1243832320 count 1
    Mar 28 10:22:56 NAS kernel:     item 177 key (401122508800 168 8192) itemoff 6865 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561023 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572139520 count 1
    Mar 28 10:22:56 NAS kernel:     item 178 key (401122516992 168 8192) itemoff 6812 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 561024 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 20572151808 count 1
    Mar 28 10:22:56 NAS kernel:     item 179 key (401122525184 168 32768) itemoff 6759 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697035 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2853257216 count 1
    Mar 28 10:22:56 NAS kernel:     item 180 key (401122557952 168 4096) itemoff 6706 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697440 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2850295808 count 1
    Mar 28 10:22:56 NAS kernel:     item 181 key (401122562048 168 4096) itemoff 6653 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697445 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2850754560 count 1
    Mar 28 10:22:56 NAS kernel:     item 182 key (401122566144 168 4096) itemoff 6600 itemsize 53
    Mar 28 10:22:56 NAS kernel:         extent refs 1 gen 697445 flags 1
    Mar 28 10:22:56 NAS kernel:         ref#0: extent data backref root 5 objectid 473 offset 2850299904 count 1
    Mar 28 10:22:56 NAS kernel: BTRFS error (device nvme0n1p1): block=389159174144 write time tree block corruption detected
    Mar 28 10:22:56 NAS kernel: BTRFS: error (device nvme0n1p1) in btrfs_commit_transaction:2447: errno=-5 IO failure (Error while writing out transaction)
    Mar 28 10:22:56 NAS kernel: BTRFS info (device nvme0n1p1: state E): forced readonly
    Mar 28 10:22:56 NAS kernel: BTRFS warning (device nvme0n1p1: state E): Skipping commit of aborted transaction.
    Mar 28 10:22:56 NAS kernel: BTRFS: error (device nvme0n1p1: state EA) in cleanup_transaction:1958: errno=-5 IO failure

     

  10. Crashed again today

     

    Nothing written to the log at time of crash again today

    On boot only thing that caught my eye was

    "kernel: BTRFS error (device nvme0n1p1): incorrect extent count for 4118023241728; counted 5432, expected 5419"

     

    Pool scrubbed again with no errors.
    Another parity check lol :(

    Going to swap the RAM tonight with some from my desktop and see if that is any better.

    If that does not fix it might be time for a new MB and CPU (upgrade from 10th gen to 13th gen wouldnt be a bad thing...)

  11. 8 hours ago, JorgeB said:

    Nothing obvious in the hardware used, I would try using just 2 of the 4 RAM sticks, if issues continue try the other two.

     

    Might have to buy new ram as im often over 50% usage. Seems strange being its not new ram.
    Guess passing memtest does not really mean anything unless i ran it for a week (which i cant)
     

    Will see if it happens now XMP is off. If thats no good i guess ill buy some more ram.

  12. Hello all

     

    Over the past week or so my unraid server has been acting up (seems about every 4-5 days)
    To start with it was filling the syslog up so bad it stopped working giving me no clues but finally managed get a log of the crash/lag out twice with similar messages.
    It starts acting up with dockers/vms etc stopping responding properly, machine cant be cleanly shutdown.

    Searches all mostly seemed to lead to "run a memory test" which i did, 4 pass memtest passed no problems.
    Been no recent hardware changes. Ram has been in the system over 2 seperate setups over the past 4-5 years.
     

    Each time i have run scrubs / file system checks with no issues.
    Cache is a pair of 1TB NVME drives in btrfs raid 1

     

    Turned XMP off now but id be surprised if that changes anything as thats been the same for years.

    Only real thought now is need to move the data off and reformat?
    If thats the case im tempted to upgrade to the RC and go ZFS for the cache.

     

    Any thoughts?

     

    Running 6.11.5 currently


    Section of the syslog below

     

     

    Mar 20 16:45:21 Vault kernel: BTRFS critical (device nvme0n1p1): corrupt leaf: block=4012333727744 slot=87 extent bytenr=4061243559936 len=16384 invalid generation, have 1099503796567 expect (0, 108298103]
    Mar 20 16:45:21 Vault kernel: BTRFS info (device nvme0n1p1): leaf 4012333727744 gen 108298102 total ptrs 119 free space 7001 owner 2
    Mar 20 16:45:21 Vault kernel: #011item 0 key (4061239910400 168 16384) itemoff 16230 itemsize 53
    Mar 20 16:45:21 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:21 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31604736 count 1
    Mar 20 16:45:21 Vault kernel: #011item 1 key (4061239943168 168 32768) itemoff 16177 itemsize 53
    Mar 20 16:45:21 Vault kernel: #011#011extent refs 1 gen 108295902 flags 1
    Mar 20 16:45:21 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 213247458 offset 6852608 count 1
    Mar 20 16:45:21 Vault kernel: #011item 2 key (4061239975936 168 16384) itemoff 16124 itemsize 53
    Mar 20 16:45:21 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:21 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31621120 count 1
    Mar 20 16:45:21 Vault kernel: #011item 3 key (4061239992320 168 16384) itemoff 16071 itemsize 53
    Mar 20 16:45:21 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 917504 count 1
    Mar 20 16:45:22 Vault kernel: #011item 4 key (4061240008704 168 16384) itemoff 16018 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31637504 count 1
    Mar 20 16:45:22 Vault kernel: #011item 5 key (4061240025088 168 24576) itemoff 15965 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108248237 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 210020962 offset 27832320 count 1
    Mar 20 16:45:22 Vault kernel: #011item 6 key (4061240049664 168 32768) itemoff 15912 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108295883 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 213247458 offset 6791168 count 1
    Mar 20 16:45:22 Vault kernel: #011item 7 key (4061240098816 168 4096) itemoff 15859 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107847983 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 995572 offset 726970368 count 1
    Mar 20 16:45:22 Vault kernel: #011item 8 key (4061240102912 168 163840) itemoff 15806 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108130530 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 191687199 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 9 key (4061240270848 168 16384) itemoff 15753 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31653888 count 1
    Mar 20 16:45:22 Vault kernel: #011item 10 key (4061240287232 168 16384) itemoff 15700 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 933888 count 1
    Mar 20 16:45:22 Vault kernel: #011item 11 key (4061240303616 168 16384) itemoff 15647 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31670272 count 1
    Mar 20 16:45:22 Vault kernel: #011item 12 key (4061240320000 168 16384) itemoff 15594 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 950272 count 1
    Mar 20 16:45:22 Vault kernel: #011item 13 key (4061240336384 168 16384) itemoff 15541 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31686656 count 1
    Mar 20 16:45:22 Vault kernel: #011item 14 key (4061240352768 168 16384) itemoff 15488 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 966656 count 1
    Mar 20 16:45:22 Vault kernel: #011item 15 key (4061240369152 168 16384) itemoff 15435 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31703040 count 1
    Mar 20 16:45:22 Vault kernel: #011item 16 key (4061240385536 168 24576) itemoff 15382 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108247954 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 210020962 offset 24190976 count 1
    Mar 20 16:45:22 Vault kernel: #011item 17 key (4061240422400 168 36864) itemoff 15329 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108295912 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 213247458 offset 6881280 count 1
    Mar 20 16:45:22 Vault kernel: #011item 18 key (4061240463360 168 77824) itemoff 15276 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108024352 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 196604783 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 19 key (4061240541184 168 200704) itemoff 15223 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108130527 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 191686973 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 20 key (4061240745984 168 77824) itemoff 15170 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108024352 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 196604437 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 21 key (4061240823808 168 16384) itemoff 15117 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 983040 count 1
    Mar 20 16:45:22 Vault kernel: #011item 22 key (4061240848384 168 114688) itemoff 15064 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108024351 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 196599287 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 23 key (4061240963072 168 16384) itemoff 15011 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31719424 count 1
    Mar 20 16:45:22 Vault kernel: #011item 24 key (4061240979456 168 16384) itemoff 14958 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 999424 count 1
    Mar 20 16:45:22 Vault kernel: #011item 25 key (4061240995840 168 16384) itemoff 14905 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31735808 count 1
    Mar 20 16:45:22 Vault kernel: #011item 26 key (4061241012224 168 16384) itemoff 14852 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1015808 count 1
    Mar 20 16:45:22 Vault kernel: #011item 27 key (4061241028608 168 16384) itemoff 14799 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31752192 count 1
    Mar 20 16:45:22 Vault kernel: #011item 28 key (4061241044992 168 16384) itemoff 14746 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1032192 count 1
    Mar 20 16:45:22 Vault kernel: #011item 29 key (4061241061376 168 16384) itemoff 14693 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31768576 count 1
    Mar 20 16:45:22 Vault kernel: #011item 30 key (4061241077760 168 16384) itemoff 14640 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1048576 count 1
    Mar 20 16:45:22 Vault kernel: #011item 31 key (4061241094144 168 12288) itemoff 14587 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108096313 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 139231763 offset 59559936 count 1
    Mar 20 16:45:22 Vault kernel: #011item 32 key (4061241106432 168 12288) itemoff 14534 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108196557 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 139231756 offset 17641472 count 1
    Mar 20 16:45:22 Vault kernel: #011item 33 key (4061241118720 168 16384) itemoff 14481 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31784960 count 1
    Mar 20 16:45:22 Vault kernel: #011item 34 key (4061241135104 168 16384) itemoff 14428 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1064960 count 1
    Mar 20 16:45:22 Vault kernel: #011item 35 key (4061241151488 168 16384) itemoff 14375 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31801344 count 1
    Mar 20 16:45:22 Vault kernel: #011item 36 key (4061241167872 168 16384) itemoff 14322 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1081344 count 1
    Mar 20 16:45:22 Vault kernel: #011item 37 key (4061241184256 168 16384) itemoff 14269 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31817728 count 1
    Mar 20 16:45:22 Vault kernel: #011item 38 key (4061241200640 168 65536) itemoff 14216 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108024354 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 196611522 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 39 key (4061241266176 168 16384) itemoff 14163 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1097728 count 1
    Mar 20 16:45:22 Vault kernel: #011item 40 key (4061241286656 168 16384) itemoff 14110 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31834112 count 1
    Mar 20 16:45:22 Vault kernel: #011item 41 key (4061241303040 168 155648) itemoff 14057 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108130530 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 191687214 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 42 key (4061241458688 168 4096) itemoff 14004 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108297052 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 199645416 offset 966656 count 1
    Mar 20 16:45:22 Vault kernel: #011item 43 key (4061241466880 168 16384) itemoff 13951 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1114112 count 1
    Mar 20 16:45:22 Vault kernel: #011item 44 key (4061241483264 168 4096) itemoff 13898 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108248234 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 210020962 offset 27795456 count 1
    Mar 20 16:45:22 Vault kernel: #011item 45 key (4061241487360 168 4096) itemoff 13845 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108248239 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 210020962 offset 27852800 count 1
    Mar 20 16:45:22 Vault kernel: #011item 46 key (4061241491456 168 16384) itemoff 13792 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31850496 count 1
    Mar 20 16:45:22 Vault kernel: #011item 47 key (4061241507840 168 16384) itemoff 13739 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1130496 count 1
    Mar 20 16:45:22 Vault kernel: #011item 48 key (4061241524224 168 131072) itemoff 13686 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108245763 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 209799051 offset 32108544 count 1
    Mar 20 16:45:22 Vault kernel: #011item 49 key (4061241655296 168 28672) itemoff 13633 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108295922 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 213247458 offset 6918144 count 1
    Mar 20 16:45:22 Vault kernel: #011item 50 key (4061241683968 168 16384) itemoff 13580 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1146880 count 1
    Mar 20 16:45:22 Vault kernel: #011item 51 key (4061241700352 168 667648) itemoff 13527 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108056473 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 198467545 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 52 key (4061242368000 168 204800) itemoff 13474 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108130523 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 191686728 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 53 key (4061242572800 168 98304) itemoff 13421 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108130593 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 191694163 offset 0 count 1
    Mar 20 16:45:22 Vault kernel: #011item 54 key (4061242675200 168 262144) itemoff 13368 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108019222 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 196226739 offset 2050686976 count 1
    Mar 20 16:45:22 Vault kernel: #011item 55 key (4061242937344 168 16384) itemoff 13315 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1163264 count 1
    Mar 20 16:45:22 Vault kernel: #011item 56 key (4061242961920 168 32768) itemoff 13262 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108094909 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 139231763 offset 59392000 count 1
    Mar 20 16:45:22 Vault kernel: #011item 57 key (4061242994688 168 16384) itemoff 13209 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1179648 count 1
    Mar 20 16:45:22 Vault kernel: #011item 58 key (4061243011072 168 16384) itemoff 13156 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1196032 count 1
    Mar 20 16:45:22 Vault kernel: #011item 59 key (4061243027456 168 16384) itemoff 13103 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31522816 count 1
    Mar 20 16:45:22 Vault kernel: #011item 60 key (4061243043840 168 16384) itemoff 13050 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1212416 count 1
    Mar 20 16:45:22 Vault kernel: #011item 61 key (4061243129856 168 20480) itemoff 12997 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108248231 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 210020962 offset 27779072 count 1
    Mar 20 16:45:22 Vault kernel: #011item 62 key (4061243150336 168 16384) itemoff 12944 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1228800 count 1
    Mar 20 16:45:22 Vault kernel: #011item 63 key (4061243166720 168 16384) itemoff 12891 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1245184 count 1
    Mar 20 16:45:22 Vault kernel: #011item 64 key (4061243183104 168 16384) itemoff 12838 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31866880 count 1
    Mar 20 16:45:22 Vault kernel: #011item 65 key (4061243199488 168 16384) itemoff 12785 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1261568 count 1
    Mar 20 16:45:22 Vault kernel: #011item 66 key (4061243215872 168 16384) itemoff 12732 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31883264 count 1
    Mar 20 16:45:22 Vault kernel: #011item 67 key (4061243232256 168 16384) itemoff 12679 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1277952 count 1
    Mar 20 16:45:22 Vault kernel: #011item 68 key (4061243248640 168 16384) itemoff 12626 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31899648 count 1
    Mar 20 16:45:22 Vault kernel: #011item 69 key (4061243265024 168 16384) itemoff 12573 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1294336 count 1
    Mar 20 16:45:22 Vault kernel: #011item 70 key (4061243281408 168 16384) itemoff 12520 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31916032 count 1
    Mar 20 16:45:22 Vault kernel: #011item 71 key (4061243297792 168 16384) itemoff 12467 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1310720 count 1
    Mar 20 16:45:22 Vault kernel: #011item 72 key (4061243314176 168 16384) itemoff 12414 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31932416 count 1
    Mar 20 16:45:22 Vault kernel: #011item 73 key (4061243330560 168 16384) itemoff 12361 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1327104 count 1
    Mar 20 16:45:22 Vault kernel: #011item 74 key (4061243346944 168 16384) itemoff 12308 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31948800 count 1
    Mar 20 16:45:22 Vault kernel: #011item 75 key (4061243363328 168 16384) itemoff 12255 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1343488 count 1
    Mar 20 16:45:22 Vault kernel: #011item 76 key (4061243379712 168 16384) itemoff 12202 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31965184 count 1
    Mar 20 16:45:22 Vault kernel: #011item 77 key (4061243396096 168 16384) itemoff 12149 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1359872 count 1
    Mar 20 16:45:22 Vault kernel: #011item 78 key (4061243412480 168 16384) itemoff 12096 itemsize 53
    Mar 20 16:45:22 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:22 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31981568 count 1
    Mar 20 16:45:22 Vault kernel: #011item 79 key (4061243428864 168 16384) itemoff 12043 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1376256 count 1
    Mar 20 16:45:23 Vault kernel: #011item 80 key (4061243445248 168 16384) itemoff 11990 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31997952 count 1
    Mar 20 16:45:23 Vault kernel: #011item 81 key (4061243461632 168 16384) itemoff 11937 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1392640 count 1
    Mar 20 16:45:23 Vault kernel: #011item 82 key (4061243478016 168 16384) itemoff 11884 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32014336 count 1
    Mar 20 16:45:23 Vault kernel: #011item 83 key (4061243494400 168 16384) itemoff 11831 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1409024 count 1
    Mar 20 16:45:23 Vault kernel: #011item 84 key (4061243510784 168 16384) itemoff 11778 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32030720 count 1
    Mar 20 16:45:23 Vault kernel: #011item 85 key (4061243527168 168 16384) itemoff 11725 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1425408 count 1
    Mar 20 16:45:23 Vault kernel: #011item 86 key (4061243543552 168 16384) itemoff 11672 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32047104 count 1
    Mar 20 16:45:23 Vault kernel: #011item 87 key (4061243559936 168 16384) itemoff 11619 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 7590289406351638529 gen 1099503796567 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1441792 count 1
    Mar 20 16:45:23 Vault kernel: #011item 88 key (4061243576320 168 16384) itemoff 11566 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32063488 count 1
    Mar 20 16:45:23 Vault kernel: #011item 89 key (4061243592704 168 16384) itemoff 11513 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1458176 count 1
    Mar 20 16:45:23 Vault kernel: #011item 90 key (4061243609088 168 16384) itemoff 11460 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32079872 count 1
    Mar 20 16:45:23 Vault kernel: #011item 91 key (4061243625472 168 16384) itemoff 11407 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1474560 count 1
    Mar 20 16:45:23 Vault kernel: #011item 92 key (4061243641856 168 16384) itemoff 11354 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32096256 count 1
    Mar 20 16:45:23 Vault kernel: #011item 93 key (4061243658240 168 16384) itemoff 11301 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1490944 count 1
    Mar 20 16:45:23 Vault kernel: #011item 94 key (4061243674624 168 16384) itemoff 11248 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32112640 count 1
    Mar 20 16:45:23 Vault kernel: #011item 95 key (4061243691008 168 16384) itemoff 11195 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1507328 count 1
    Mar 20 16:45:23 Vault kernel: #011item 96 key (4061243707392 168 16384) itemoff 11142 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32129024 count 1
    Mar 20 16:45:23 Vault kernel: #011item 97 key (4061243723776 168 16384) itemoff 11089 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1523712 count 1
    Mar 20 16:45:23 Vault kernel: #011item 98 key (4061243740160 168 16384) itemoff 11036 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32145408 count 1
    Mar 20 16:45:23 Vault kernel: #011item 99 key (4061243756544 168 16384) itemoff 10983 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306654 offset 1540096 count 1
    Mar 20 16:45:23 Vault kernel: #011item 100 key (4061243772928 168 16384) itemoff 10930 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32161792 count 1
    Mar 20 16:45:23 Vault kernel: #011item 101 key (4061243789312 168 16384) itemoff 10877 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31457280 count 1
    Mar 20 16:45:23 Vault kernel: #011item 102 key (4061243805696 168 16384) itemoff 10824 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32178176 count 1
    Mar 20 16:45:23 Vault kernel: #011item 103 key (4061243822080 168 16384) itemoff 10771 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31473664 count 1
    Mar 20 16:45:23 Vault kernel: #011item 104 key (4061243838464 168 16384) itemoff 10718 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32194560 count 1
    Mar 20 16:45:23 Vault kernel: #011item 105 key (4061243854848 168 16384) itemoff 10665 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31490048 count 1
    Mar 20 16:45:23 Vault kernel: #011item 106 key (4061243871232 168 16384) itemoff 10612 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32210944 count 1
    Mar 20 16:45:23 Vault kernel: #011item 107 key (4061243887616 168 16384) itemoff 10559 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108128561 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 31506432 count 1
    Mar 20 16:45:23 Vault kernel: #011item 108 key (4061243904000 168 16384) itemoff 10506 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32227328 count 1
    Mar 20 16:45:23 Vault kernel: #011item 109 key (4061243920384 168 16384) itemoff 10453 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108245765 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 209799051 offset 32235520 count 1
    Mar 20 16:45:23 Vault kernel: #011item 110 key (4061243936768 168 16384) itemoff 10400 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32243712 count 1
    Mar 20 16:45:23 Vault kernel: #011item 111 key (4061243969536 168 16384) itemoff 10347 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32260096 count 1
    Mar 20 16:45:23 Vault kernel: #011item 112 key (4061244002304 168 16384) itemoff 10294 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32276480 count 1
    Mar 20 16:45:23 Vault kernel: #011item 113 key (4061244035072 168 16384) itemoff 10241 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32292864 count 1
    Mar 20 16:45:23 Vault kernel: #011item 114 key (4061244051456 168 16384) itemoff 10188 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108298028 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 199645446 offset 24576 count 1
    Mar 20 16:45:23 Vault kernel: #011item 115 key (4061244067840 168 16384) itemoff 10135 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32309248 count 1
    Mar 20 16:45:23 Vault kernel: #011item 116 key (4061244084224 168 36864) itemoff 10082 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108248241 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 210020962 offset 27869184 count 1
    Mar 20 16:45:23 Vault kernel: #011item 117 key (4061244133376 168 16384) itemoff 10029 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 107852245 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 181306704 offset 32325632 count 1
    Mar 20 16:45:23 Vault kernel: #011item 118 key (4061244149760 168 12288) itemoff 9976 itemsize 53
    Mar 20 16:45:23 Vault kernel: #011#011extent refs 1 gen 108297858 flags 1
    Mar 20 16:45:23 Vault kernel: #011#011ref#0: extent data backref root 5 objectid 199645704 offset 487424 count 1
    Mar 20 16:45:23 Vault kernel: BTRFS error (device nvme0n1p1): block=4012333727744 write time tree block corruption detected

     

     

     

  13. Does frigate have a memory leak of some kind? noticed my unraid server was at 93% memory usage this morning which is unusual
    Frigate happened to be the first docker i restarted and it dropped memory usage back to 33%. (machine with 16GB Ram)

     

    Wonder if that was what was causing my frigate crashes...

  14. 16 hours ago, yayitazale said:

    Can be a faulty device... Did you see the TPU in the list of devices on unraid (tools -> System devices)?

     

    You have to see something like "Bus 002 Device 002 Port 2-4 ID 18d1:9302 Google Inc"

     

    Can you show how are you mapping the device and how did you configured the device on the config file?

     

    You can try also in another USB port.

     

    Yea it shows in system devices. It also works fine once i start frigate again each time after i notice its stopped.

    mapped to the docker with /dev/bus/usb

    config is:

    detectors:
      coral:
        type: edgetpu
        device: usb

     

    Will try another USB port see if there is any difference

  15. Having an issue with my docker crashing about once a week, looked at the log today but it does not seem to be anything helpful

    Any suggestions?
     

    [2023-03-06 23:12:50] frigate.watchdog               INFO    : Detection appears to be stuck. Restarting detection process...
    [2023-03-06 23:12:50] root                           INFO    : Waiting for detection process to exit gracefully...
    [2023-03-06 23:13:20] root                           INFO    : Detection process didnt exit. Force killing...
    [2023-03-06 23:13:20] detector.coral                 INFO    : Starting detection process: 8443
    [2023-03-06 23:13:20] frigate.edgetpu                INFO    : Attempting to load TPU as usb
    Process detector:coral:
    [2023-03-06 23:13:45] frigate.edgetpu                ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
        delegate = Delegate(library, options)
      File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
        raise ValueError(capture.message)
    ValueError
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
      File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
      File "/opt/frigate/frigate/edgetpu.py", line 135, in run_detector
        object_detector = LocalObjectDetector(
      File "/opt/frigate/frigate/edgetpu.py", line 43, in __init__
        edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
      File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
        raise ValueError('Failed to load delegate from {}\n{}'.format(
    ValueError: Failed to load delegate from libedgetpu.so.1.0
    
    [2023-03-06 23:13:50] frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...