KingFishy492

We believe we have found the source of the corruption. One of the 16GB RAM sticks was chucking out all the corrution errors when running the memtest. Tried it with only 1 stick in, instant errors on one of the sticks. switched the stick out (into the same RAM slot as the other RAM that was on chucking out the errors) left that running for over 30+ minutes no errors. booted into unraid no errors so far everything seems to be reporting nominal

Attached updated diags. zero-diagnostics-20221229-1341.zip Cache drive 1/2 Cache drive 2/2 Console repeating logs Dec 29 14:05:35 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:35 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:35 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:50 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:50 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:50 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:50 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:50 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0) Dec 29 14:05:50 Zero kernel: BTRFS error (device sde1): tree first key mismatch detected, bytenr=1300365312 parent_transid=112 key expected=(77321,1,0) has=(117629,1,0)

scrub finished Error summary: csum=6 Corrected: 0 Uncorrectable: 6 Unverified: 0

Several days ago started noticing issues with a VM. Logs are showing BTRFS errors such as Dec 28 12:38:39 Zero kernel: BTRFS info (device sde1): bdev /dev/sdf1 errs: wr 0, rd 0, flush 0, corrupt 12, gen 0 and this over and over again... Dec 28 12:56:58 Zero kernel: BTRFS critical (device sde1): unable to find logical 4504458284793856 length 16384 Dec 28 12:56:58 Zero kernel: BTRFS critical (device sde1): unable to find logical 4504458284793856 length 4096 Dec 28 12:56:58 Zero kernel: BTRFS critical (device sde1): unable to find logical 4504458284793856 length 16384 Dec 28 12:56:58 Zero kernel: BTRFS critical (device sde1): unable to find logical 4504458284793856 length 4096 Dec 28 12:56:58 Zero kernel: BTRFS critical (device sde1): unable to find logical 4504458284793856 length 16384 SMART attribs show fine. Currently performing a BTRFS scrub. which has found no errors so far. Attached diags. zero-diagnostics-20221228-1255.zip