Fix Common Problems: Drive mounted read-only or nearly full when its not

NateRules · April 6, 2018

So, I am starting preparation to upgrade my Unraid from 6.4 to 6.5, and went to run the Fix Common problems plugin and am getting the following error:

However, when I go to the Main screen all is good. My drive is Green, I absolutely can read and write from my shares. So I'm not sure what is going on here. My drive was marked as "Warning" because it was over 70% full, but I changed the threshold to 85% and it went back to Normal, but the Fix Common Problems plugin still has the above error after multiple Rescans and Extended Tests. I'll post some screenshots about the drive, let me know if you have any ideas:

trurl · April 6, 2018

Tools - Diagnostics, post complete zip

NateRules · April 6, 2018

Quote

diagnostics-20180406-0937.zip

JorgeB · April 6, 2018

There's filesystem corruption on disk1, check filesystem:

https://lime-technology.com/wiki/Check_Disk_Filesystems#Checking_and_fixing_drives_in_the_webGui

NateRules · April 6, 2018

Wow, not sure how that happened. Where did you see that in the logs? Here is the results of a check with -nv flag, do I need to do a repair?

Phase 1 - find and verify superblock...
        - reporting progress in intervals of 15 minutes
        - block cache size set to 1474928 entries
Phase 2 - using internal log
        - zero log...
zero_log: head block 107499 tail block 107499
        - scan filesystem freespace and inode maps...
agf_freeblks 187296, counted 190870 in ag 0
agi_count 3072, counted 2560 in ag 0
agi_freecount 4294967121, counted 0 in ag 0
agi_freecount 4294967121, counted 0 in ag 0 finobt
sb_icount 111232, counted 110720
sb_ifree 4486, counted 4661
sb_fdblocks 450416816, counted 450420390
        - 10:03:16: scanning filesystem freespace - 100 of 100 allocation groups done
        - found root inode chunk
Phase 3 - for each AG...
        - scan (but don't clear) agi unlinked lists...
        - 10:03:16: scanning agi unlinked lists - 100 of 100 allocation groups done
        - process known inodes and perform inode discovery...
        - agno = 75
        - agno = 0
        - agno = 45
        - agno = 15
        - agno = 90
        - agno = 30
        - agno = 60
        - agno = 31
        - agno = 61
        - agno = 46
        - agno = 76
        - agno = 1
        - agno = 32
        - agno = 62
        - agno = 47
        - agno = 77
        - agno = 33
        - agno = 63
        - agno = 48
        - agno = 34
        - agno = 64
        - agno = 91
        - agno = 35
        - agno = 49
        - agno = 65
        - agno = 92
        - agno = 50
        - agno = 36
        - agno = 66
        - agno = 93
        - agno = 37
        - agno = 94
        - agno = 67
        - agno = 51
        - agno = 38
        - agno = 68
        - agno = 52
        - agno = 39
        - agno = 16
        - agno = 17
        - agno = 69
        - agno = 18
        - agno = 53
        - agno = 40
        - agno = 70
        - agno = 54
        - agno = 41
        - agno = 71
        - agno = 78
        - agno = 79
        - agno = 42
        - agno = 80
        - agno = 72
        - agno = 55
        - agno = 43
        - agno = 81
        - agno = 56
        - agno = 73
        - agno = 44
        - agno = 57
        - agno = 74
        - agno = 82
        - agno = 58
        - agno = 95
        - agno = 96
        - agno = 83
        - agno = 59
        - agno = 19
        - agno = 84
        - agno = 97
        - agno = 98
        - agno = 85
        - agno = 99
        - agno = 86
        - agno = 20
        - agno = 21
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - 10:03:47: process known inodes and inode discovery - 110720 of 111232 inodes done
        - process newly discovered inodes...
        - 10:03:47: process newly discovered inodes - 100 of 100 allocation groups done
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - 10:03:47: setting up duplicate extent list - 100 of 100 allocation groups done
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 3
        - agno = 2
        - agno = 1
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
        - agno = 32
        - agno = 33
        - agno = 34
        - agno = 35
        - agno = 36
        - agno = 37
        - agno = 38
        - agno = 39
        - agno = 40
        - agno = 41
        - agno = 42
        - agno = 43
        - agno = 44
        - agno = 45
        - agno = 46
        - agno = 47
        - agno = 48
        - agno = 49
        - agno = 50
        - agno = 51
        - agno = 52
        - agno = 53
        - agno = 54
        - agno = 55
        - agno = 56
        - agno = 57
        - agno = 58
        - agno = 59
        - agno = 60
        - agno = 61
        - agno = 62
        - agno = 63
        - agno = 64
        - agno = 65
        - agno = 66
        - agno = 67
        - agno = 68
        - agno = 69
        - agno = 70
        - agno = 71
        - agno = 72
        - agno = 73
        - agno = 74
        - agno = 75
        - agno = 76
        - agno = 77
        - agno = 78
        - agno = 79
        - agno = 80
        - agno = 81
        - agno = 82
        - agno = 83
        - agno = 84
        - agno = 85
        - agno = 86
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 90
        - agno = 91
        - agno = 92
        - agno = 93
        - agno = 94
        - agno = 95
        - agno = 96
        - agno = 97
        - agno = 98
        - agno = 99
        - 10:03:47: check for inodes claiming duplicate blocks - 110720 of 111232 inodes done
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
        - traversing filesystem ...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
        - agno = 32
        - agno = 33
        - agno = 34
        - agno = 35
        - agno = 36
        - agno = 37
        - agno = 38
        - agno = 39
        - agno = 40
        - agno = 41
        - agno = 42
        - agno = 43
        - agno = 44
        - agno = 45
        - agno = 46
        - agno = 47
        - agno = 48
        - agno = 49
        - agno = 50
        - agno = 51
        - agno = 52
        - agno = 53
        - agno = 54
        - agno = 55
        - agno = 56
        - agno = 57
        - agno = 58
        - agno = 59
        - agno = 60
        - agno = 61
        - agno = 62
        - agno = 63
        - agno = 64
        - agno = 65
        - agno = 66
        - agno = 67
        - agno = 68
        - agno = 69
        - agno = 70
        - agno = 71
        - agno = 72
        - agno = 73
        - agno = 74
        - agno = 75
        - agno = 76
        - agno = 77
        - agno = 78
        - agno = 79
        - agno = 80
        - agno = 81
        - agno = 82
        - agno = 83
        - agno = 84
        - agno = 85
        - agno = 86
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 90
        - agno = 91
        - agno = 92
        - agno = 93
        - agno = 94
        - agno = 95
        - agno = 96
        - agno = 97
        - agno = 98
        - agno = 99
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
        - 10:03:58: verify and correct link counts - 100 of 100 allocation groups done
No modify flag set, skipping filesystem flush and exiting.

        XFS_REPAIR Summary    Fri Apr  6 10:03:58 2018

Phase		Start		End		Duration
Phase 1:	04/06 10:03:15	04/06 10:03:16	1 second
Phase 2:	04/06 10:03:16	04/06 10:03:16
Phase 3:	04/06 10:03:16	04/06 10:03:47	31 seconds
Phase 4:	04/06 10:03:47	04/06 10:03:47
Phase 5:	Skipped
Phase 6:	04/06 10:03:47	04/06 10:03:58	11 seconds
Phase 7:	04/06 10:03:58	04/06 10:03:58

Total run time: 43 seconds

JorgeB · April 6, 2018

3 minutes ago, NateRules said:

Where did you see that in the logs?

Apr  6 09:21:07 drogo kernel: XFS (md1): Internal error XFS_WANT_CORRUPTED_GOTO at line 1423 of file fs/xfs/libxfs/xfs_ialloc.c.  Caller xfs_dialloc_ag+0xdd/0x23f [xfs]

3 minutes ago, NateRules said:

do I need to do a repair?

Yes

NateRules · April 6, 2018

Ok, I did the repair, but can't tell what it did, it mentions "- moving disconnected inodes to lost+found ...", but when I do a 'ls -la /mnt/disks/disk1' after restarting the array in normal mode, I dont see a lost+found folder.

root@dXXYY:~# xfs_repair -v /dev/md1
Phase 1 - find and verify superblock...
        - reporting progress in intervals of 15 minutes
        - block cache size set to 1474928 entries
Phase 2 - using internal log
        - zero log...
zero_log: head block 107499 tail block 107499
        - scan filesystem freespace and inode maps...
agf_freeblks 187296, counted 190870 in ag 0
agi_count 3072, counted 2560 in ag 0
agi_freecount 4294967121, counted 0 in ag 0
agi_freecount 4294967121, counted 0 in ag 0 finobt
sb_icount 111232, counted 110720
sb_ifree 4486, counted 4661
sb_fdblocks 450416816, counted 450420390
        - 10:21:11: scanning filesystem freespace - 100 of 100 allocation groups done
        - found root inode chunk
Phase 3 - for each AG...
        - scan and clear agi unlinked lists...
        - 10:21:11: scanning agi unlinked lists - 100 of 100 allocation groups done
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 15
        - agno = 60
        - agno = 90
        - agno = 45
        - agno = 30
        - agno = 75
        - agno = 61
        - agno = 46
        - agno = 31
        - agno = 76
        - agno = 1
        - agno = 62
        - agno = 32
        - agno = 47
        - agno = 77
        - agno = 63
        - agno = 33
        - agno = 48
        - agno = 34
        - agno = 64
        - agno = 91
        - agno = 49
        - agno = 35
        - agno = 65
        - agno = 92
        - agno = 50
        - agno = 36
        - agno = 66
        - agno = 93
        - agno = 94
        - agno = 37
        - agno = 67
        - agno = 51
        - agno = 38
        - agno = 68
        - agno = 52
        - agno = 39
        - agno = 16
        - agno = 17
        - agno = 69
        - agno = 53
        - agno = 18
        - agno = 70
        - agno = 40
        - agno = 54
        - agno = 41
        - agno = 71
        - agno = 78
        - agno = 79
        - agno = 42
        - agno = 80
        - agno = 72
        - agno = 55
        - agno = 43
        - agno = 81
        - agno = 56
        - agno = 73
        - agno = 44
        - agno = 57
        - agno = 74
        - agno = 82
        - agno = 58
        - agno = 95
        - agno = 96
        - agno = 83
        - agno = 59
        - agno = 19
        - agno = 84
        - agno = 97
        - agno = 98
        - agno = 85
        - agno = 99
        - agno = 86
        - agno = 20
        - agno = 21
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - 10:21:41: process known inodes and inode discovery - 110720 of 111232 inodes done
        - process newly discovered inodes...
        - 10:21:41: process newly discovered inodes - 100 of 100 allocation groups done
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - 10:21:41: setting up duplicate extent list - 100 of 100 allocation groups done
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 3
        - agno = 1
        - agno = 2
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
        - agno = 32
        - agno = 33
        - agno = 34
        - agno = 35
        - agno = 36
        - agno = 37
        - agno = 38
        - agno = 39
        - agno = 40
        - agno = 41
        - agno = 42
        - agno = 43
        - agno = 44
        - agno = 45
        - agno = 46
        - agno = 47
        - agno = 48
        - agno = 49
        - agno = 50
        - agno = 51
        - agno = 52
        - agno = 53
        - agno = 54
        - agno = 55
        - agno = 56
        - agno = 57
        - agno = 58
        - agno = 59
        - agno = 60
        - agno = 61
        - agno = 62
        - agno = 63
        - agno = 64
        - agno = 65
        - agno = 66
        - agno = 67
        - agno = 68
        - agno = 69
        - agno = 70
        - agno = 71
        - agno = 72
        - agno = 73
        - agno = 74
        - agno = 75
        - agno = 76
        - agno = 77
        - agno = 78
        - agno = 79
        - agno = 80
        - agno = 81
        - agno = 82
        - agno = 83
        - agno = 84
        - agno = 85
        - agno = 86
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 90
        - agno = 91
        - agno = 92
        - agno = 93
        - agno = 94
        - agno = 95
        - agno = 96
        - agno = 97
        - agno = 98
        - agno = 99
        - 10:21:41: check for inodes claiming duplicate blocks - 110720 of 111232 inodes done
Phase 5 - rebuild AG headers and trees...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
        - agno = 32
        - agno = 33
        - agno = 34
        - agno = 35
        - agno = 36
        - agno = 37
        - agno = 38
        - agno = 39
        - agno = 40
        - agno = 41
        - agno = 42
        - agno = 43
        - agno = 44
        - agno = 45
        - agno = 46
        - agno = 47
        - agno = 48
        - agno = 49
        - agno = 50
        - agno = 51
        - agno = 52
        - agno = 53
        - agno = 54
        - agno = 55
        - agno = 56
        - agno = 57
        - agno = 58
        - agno = 59
        - agno = 60
        - agno = 61
        - agno = 62
        - agno = 63
        - agno = 64
        - agno = 65
        - agno = 66
        - agno = 67
        - agno = 68
        - agno = 69
        - agno = 70
        - agno = 71
        - agno = 72
        - agno = 73
        - agno = 74
        - agno = 75
        - agno = 76
        - agno = 77
        - agno = 78
        - agno = 79
        - agno = 80
        - agno = 81
        - agno = 82
        - agno = 83
        - agno = 84
        - agno = 85
        - agno = 86
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 90
        - agno = 91
        - agno = 92
        - agno = 93
        - agno = 94
        - agno = 95
        - agno = 96
        - agno = 97
        - agno = 98
        - agno = 99
        - 10:21:41: rebuild AG headers and trees - 100 of 100 allocation groups done
        - reset superblock...
Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - traversing filesystem ...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - agno = 4
        - agno = 5
        - agno = 6
        - agno = 7
        - agno = 8
        - agno = 9
        - agno = 10
        - agno = 11
        - agno = 12
        - agno = 13
        - agno = 14
        - agno = 15
        - agno = 16
        - agno = 17
        - agno = 18
        - agno = 19
        - agno = 20
        - agno = 21
        - agno = 22
        - agno = 23
        - agno = 24
        - agno = 25
        - agno = 26
        - agno = 27
        - agno = 28
        - agno = 29
        - agno = 30
        - agno = 31
        - agno = 32
        - agno = 33
        - agno = 34
        - agno = 35
        - agno = 36
        - agno = 37
        - agno = 38
        - agno = 39
        - agno = 40
        - agno = 41
        - agno = 42
        - agno = 43
        - agno = 44
        - agno = 45
        - agno = 46
        - agno = 47
        - agno = 48
        - agno = 49
        - agno = 50
        - agno = 51
        - agno = 52
        - agno = 53
        - agno = 54
        - agno = 55
        - agno = 56
        - agno = 57
        - agno = 58
        - agno = 59
        - agno = 60
        - agno = 61
        - agno = 62
        - agno = 63
        - agno = 64
        - agno = 65
        - agno = 66
        - agno = 67
        - agno = 68
        - agno = 69
        - agno = 70
        - agno = 71
        - agno = 72
        - agno = 73
        - agno = 74
        - agno = 75
        - agno = 76
        - agno = 77
        - agno = 78
        - agno = 79
        - agno = 80
        - agno = 81
        - agno = 82
        - agno = 83
        - agno = 84
        - agno = 85
        - agno = 86
        - agno = 87
        - agno = 88
        - agno = 89
        - agno = 90
        - agno = 91
        - agno = 92
        - agno = 93
        - agno = 94
        - agno = 95
        - agno = 96
        - agno = 97
        - agno = 98
        - agno = 99
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
        - 10:21:51: verify and correct link counts - 100 of 100 allocation groups done

        XFS_REPAIR Summary    Fri Apr  6 10:21:54 2018

Phase           Start           End             Duration
Phase 1:        04/06 10:21:10  04/06 10:21:11  1 second
Phase 2:        04/06 10:21:11  04/06 10:21:11
Phase 3:        04/06 10:21:11  04/06 10:21:41  30 seconds
Phase 4:        04/06 10:21:41  04/06 10:21:41
Phase 5:        04/06 10:21:41  04/06 10:21:41
Phase 6:        04/06 10:21:41  04/06 10:21:51  10 seconds
Phase 7:        04/06 10:21:51  04/06 10:21:51

Total run time: 41 seconds
done

JorgeB · April 6, 2018

23 minutes ago, NateRules said:

Ok, I did the repair, but can't tell what it did, it mentions "- moving disconnected inodes to lost+found

Thta's normal, if there's no folder nothing was moved there.

NateRules · April 6, 2018

Thanks for the help!

trurl · April 6, 2018

3 hours ago, NateRules said:

but when I do a 'ls -la /mnt/disks/disk1' after restarting the array in normal mode, I dont see a lost+found folder.

This is not the correct path to disk1. It is at /mnt/disk1.

/mnt/disks/disk1 would be an Unassigned Device you had mounted and named disk1. Or possibly a folder created in /mnt by some misconfigured app. Does that path even exist on your server?

NateRules · April 6, 2018

You're right! I did check /mnt/disk1 and didn't find a lost+found directory.

I mistakenly typed /mnt/disks/disk1 into the forum. Again, thanks for the quick help on this, my server is humming along again and I will upgrade it soon.

VictorCharlie · September 16, 2021

I had received a warning for common problems with disk4 us mounted read-only. I inspected the syslog.1.txt (syslog.txt is full of the same errors) and found this error, "XFS (md4): Metadata CRC error detected at xfs_dir3_block_read_verify+0x7d/0xc6 [xfs], xfs_dir3_block block 0x1a8106c8' , 'Sep 14 02:19:27 Pensieve kernel: XFS (md4): Unmount and run xfs_repair'

I when here and found instructions how to run the repair. https://wiki.unraid.net/Check_Disk_Filesystems#Checking_and_fixing_drives_in_the_webGui

I ran the check first and there were errors (see log attached XFS_Check_log.txt):

In the instructions it says that, "If however issues were found, the display of results will indicate the recommended action to take. Typically, that will involve repeating the command with a specific option, clearly stated, which you will type into the options box (including any hyphens, usually 2 leading hyphens)."

I do not see any instruction in the output to do the repair. The instructions send me here: https://wiki.unraid.net/Check_Disk_Filesystems#xfs_repair. I am not sure which switches to use. Can I get an assist in running the repair or overall checking my work to find the root cause? There could be one issue or many I do not know.

Thank you in advance for your time and lending your expertise.

pensieve-diagnostics-20210915-1958.zip XFS_check_output.txt

Squid · September 16, 2021

Run the exact same command without the "-n" flag (no modify).

If it suggests at the end of it to unmount it and replay the log and use the -L flag, then re-run with the -L flag

VictorCharlie · September 17, 2021

@squid Thank you! I had to run it with the -L to fix the issue. It all seems to be running correctly. I check all of my disks and the cache too.

Fix Common Problems: Drive mounted read-only or nearly full when its not

Recommended Posts

NateRules

Link to comment

trurl

Link to comment

NateRules

Link to comment

JorgeB

Link to comment

NateRules

Link to comment

JorgeB

Link to comment

NateRules

Link to comment

JorgeB

Link to comment

NateRules

Link to comment

trurl

Link to comment

NateRules

Link to comment

VictorCharlie

Link to comment

Squid

Link to comment

VictorCharlie

Link to comment

Join the conversation