I just noticed these while browsing netdata:

 

 

Loads of errors on my cache drive showing in Netdata yet nothing in the Unraid system log - surely they would be there if true right?

 

Any thoughts on how to clear this or investigate further?

 

I have recently changed to a new cache drive, done everything correctly and its been working fine ever since for over 2-3 weeks now.

 

Log file attached.

 

Edit - Some additional info, I am on the stable branch of Netdata, I was on latest but changed it to stable to check if this kept up.

r3l-unraid-syslog-20241010-2118.zip

Edited October 10, 20241 yr by vindictive-voyeur2295
More info.

Some more info, looks like they are real after all.

 

Ran btrfs dev stats /mnt/cache and got the below results  

[/dev/nvme0n1p1].write_io_errs    1575422
[/dev/nvme0n1p1].read_io_errs     1059
[/dev/nvme0n1p1].flush_io_errs    26143
[/dev/nvme0n1p1].corruption_errs  0
[/dev/nvme0n1p1].generation_errs  0

 

Edit - More info:

 

Result of smart test below:

 

 

UUID: 825a839c-92ce-4512-82f1-2718426725b5

Scrub started: Thu Oct 10 23:11:35 2024

Status: finished Duration: 0:01:41 Total to scrub: 544.39GiB Rate: 5.39GiB/s Error summary: no errors found

 

Downloaded results:

 

=== START OF INFORMATION SECTION ===
Model Number:                       KINGSTON SFYRD2000G
Serial Number:                      50026B7686D58263
Firmware Version:                   EIFK31.7
PCI Vendor/Subsystem ID:            0x2646
IEEE OUI Identifier:                0x0026b7
Total NVM Capacity:                 2,000,398,934,016 [2.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
NVMe Version:                       1.4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          2,000,398,934,016 [2.00 TB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            0026b7 686d582635
Local Time is:                      Thu Oct 10 23:06:51 2024 BST
Firmware Updates (0x12):            1 Slot, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005d):     Comp DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x08):         Telmtry_Lg
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     84 Celsius
Critical Comp. Temp. Threshold:     89 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     8.80W       -        -    0  0  0  0        0       0
 1 +     7.10W       -        -    1  1  1  1        0       0
 2 +     5.20W       -        -    2  2  2  2        0       0
 3 -   0.0620W       -        -    3  3  3  3     2500    7500
 4 -   0.0620W       -        -    4  4  4  4     2500    7500

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         1

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        42 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    1,720,439 [880 GB]
Data Units Written:                 4,067,836 [2.08 TB]
Host Read Commands:                 13,936,702
Host Write Commands:                31,345,141
Controller Busy Time:               146
Power Cycles:                       2
Power On Hours:                     173
Unsafe Shutdowns:                   0
Media and Data Integrity Errors:    0
Error Information Log Entries:      478,986
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 2:               55 Celsius

Error Information (NVMe Log 0x01, 16 of 63 entries)
Num   ErrCount  SQId   CmdId  Status  PELoc          LBA  NSID    VS
  0     478986     0  0x4019  0x4004 0x102c            0     0     -
  1     478985     0  0x4018  0x4004 0x102c            0     0     -
  2     478984     0  0x701a  0x4004 0x102c            0     0     -
  3     478983     0  0x2019  0x4004 0x102c            0     0     -
  4     478982     0  0x1019  0x4004 0x102c            0     0     -
  5     478981     0  0x1018  0x4004 0x102c            0     0     -
  6     478980     0  0x7006  0x4004 0x102c            0     0     -
  7     478979     0  0x6005  0x4004 0x102c            0     0     -
  8     478978     0  0x0018  0x4004 0x102c            0     0     -
  9     478977     0  0x101b  0x4004 0x102c            0     0     -
 10     478976     0  0xe018  0x4004 0x102c            0     0     -
 11     478975     0  0xf01b  0x4004 0x102c            0     0     -
 12     478974     0  0x4011  0x4004 0x102c            0     0     -
 13     478973     0  0x4010  0x4004 0x102c            0     0     -
 14     478972     0  0x3000  0x4004 0x102c            0     0     -
 15     478971     0  0xf003  0x4004 0x102c            0     0     -
... (47 entries not read)
 

Should also note this is just a single cache pool.

Edited October 10, 20241 yr by vindictive-voyeur2295
More info.

10 hours ago, vindictive-voyeur2295 said:

[/dev/nvme0n1p1].write_io_errs    1575422
[/dev/nvme0n1p1].read_io_errs     1059
[/dev/nvme0n1p1].flush_io_errs    26143

These suggest the device dropped offline in the past, see here for how to reset the stats and better monitoring, then if it happens again post the diags before rebooting.

Thanks for this, I will do so just now and continue to monitor.