December 17, 20169 yr On running a scrub on my BTFFS cache pool (set to correct file system errors) I have noticed the following errors, it is still running at the moment so that is just a snapshot I am running 2 250GB disks in RAID1 scrub status for 4a036a94-dbe6-4599-b53c-35f384e2f99f scrub started at Sat Dec 17 18:22:46 2016, running for 00:00:28 total bytes scrubbed: 1.17GiB with 155426 errors error details: read=155424 super=2 corrected errors: 153269, uncorrectable errors: 128, unverified errors: 0 Assuming the scrub finishes with uncorrectable errors, what is my move?
December 17, 20169 yr Community Expert If there are uncorrectable errors probably best to format the pool and start over. Post your diagnostics and the output of both: btrfs fi show /mnt/cache and btrfs fi df /mnt/cache
December 17, 20169 yr Author Will do I have also have the following message on trying to get a sys log Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 132370432 bytes) in /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(300) : eval()'d code on line 73
December 17, 20169 yr Author Please find attached diagnostics file Running btrfs fi show /mnt/cache gives:- Label: none uuid: 4a036a94-dbe6-4599-b53c-35f384e2f99f Total devices 2 FS bytes used 108.91GiB devid 2 size 232.89GiB used 112.03GiB path /dev/sdc1 devid 4 size 232.89GiB used 112.03GiB path /dev/sdb1 Running btrfs fi df /mnt/cache gives:- Data, RAID1: total=110.00GiB, used=108.68GiB System, RAID1: total=32.00MiB, used=16.00KiB Metadata, RAID1: total=2.00GiB, used=234.52MiB GlobalReserve, single: total=80.00MiB, used=0.00B tower-diagnostics-20161217-1848.zip
December 17, 20169 yr Author I have run mount -o remount,size=256m /var/log using SSH but still get the same message on trying access Syslog?
December 17, 20169 yr Community Expert Btrfs fi output looks ok, on phone so can't look at the diags now, will look later if no one respondes before.
December 17, 20169 yr Author Thanks, the scrub has now finished with the following result scrub status for 4a036a94-dbe6-4599-b53c-35f384e2f99f scrub started at Sat Dec 17 18:22:46 2016 and finished after 00:28:28 total bytes scrubbed: 217.81GiB with 28503778 errors error details: read=28503776 super=2 corrected errors: 3028658, uncorrectable errors: 25475118, unverified errors: 0
December 17, 20169 yr Author I have run a SMART test on Cache disk which shows no errors I have tried to run a SMART test on Cache 2 disk but it will not start, looking at the SMART error log shows scsiModePageOffset: response length too short, resp_len=47 offset=50 bd_len=46
December 17, 20169 yr Community Expert Looks like one of your cache disks dropped offline, despite the pool showing two active disks, reboot and post new diags.
December 17, 20169 yr Author Please find attached new diagnostics file The server hung on "Unmounting Disks" on stopping it, and so I had to carry out a hard reset. As such it is now carrying out a Parity Check tower-diagnostics-20161217-2010.zip
December 17, 20169 yr Community Expert Had a chance to look at the syslog, both cache disks were having issues, and cache2 ended up being dropped. They are both alone on one of the SASLP, so I would start be checking that controller and cable. You could use the onboard SATA ports for those, looks like they are disable? SMART looks ok for both cache disks, you could try another scrub now (but don't forget to check that controller/cable)
December 17, 20169 yr Author On stopping I disconnected and reconnected the cable connections to the drive caddies and HBA. I also took the drives out of the 5 x 3 caddies and reseated tehm too. I notice problems in the attached syslog too. syslog.txt
December 17, 20169 yr Community Expert New syslog has no ata errors, at least not for now, btrfs errors only, normal after what happened, try a new correcting scrub.
December 17, 20169 yr Author Running correcting scrub at the moment and looks like errors are being corrected so far scrub status for 4a036a94-dbe6-4599-b53c-35f384e2f99f scrub started at Sat Dec 17 20:24:39 2016, running for 00:10:45 total bytes scrubbed: 82.60GiB with 222 errors error details: csum=222 corrected errors: 222, uncorrectable errors: 0, unverified errors: 0
December 17, 20169 yr Author Looks like all errors have now been fixed, thanks for your help scrub status for 4a036a94-dbe6-4599-b53c-35f384e2f99f scrub started at Sat Dec 17 20:24:39 2016 and finished after 00:35:22 total bytes scrubbed: 217.83GiB with 4008 errors error details: csum=4008 corrected errors: 4008, uncorrectable errors: 0, unverified errors: 0 On rebooting 2 data drives have redballed preventing a parity check, but that is another problem which I know is down to a flaky AOC-SASL-MV8 card I am using 3 of them, and thankfully I have some DELL PERC H310 cards in the post at the moment to replace them!
Archived
This topic is now archived and is closed to further replies.