February 2, 20215 yr I can't find any relevant log data, but I'm worried I'm looking in the wrong place. Instead, I hooked up a VGA monitor to the server and took a photo (below). The symptoms are benign mostly, but can actually take down my home network in some rare cases. System boots well. Runs flawlessly for 1-2 days. But, eventually, something like this appears on the monitor. More rarely, it generates HUGE amounts of network traffic, and grinds everything to a halt. I'm considering running a repair on the xfs filesystem on disc1 (md1). But I also understand that can come with risks. To date, I've not lost any data that I know of. However, I believe this issue has randomly claimed some of my Dockers in the past. Happy to provide any/all other info if someone could provide some guidance. EDIT: was wrong... evidence is in the log file. See attached. athena_log_jan29.txt Edited February 2, 20215 yr by FlexGunship
February 2, 20215 yr Community Expert Go to Tools - Diagnostics and attach the complete Diagnostics ZIP file to your NEXT post in this thread.
February 3, 20215 yr Community Expert Repair filesystem on disk1, be sure to capture the output so you can post it. Also corruption on docker.img. You need to delete and recreate that anyway because it is ridiculously large. 20G is more than enough. The Previous Apps feature on the Apps page will reinstall your dockers just as they were.
February 4, 20215 yr Author On 2/2/2021 at 11:07 PM, trurl said: Repair filesystem on disk1, be sure to capture the output so you can post it. Thank you. After taking proper precautions, I'm running it now. Will update when complete. On 2/2/2021 at 11:07 PM, trurl said: Also corruption on docker.img. You need to delete and recreate that anyway because it is ridiculously large. 20G is more than enough. The Previous Apps feature on the Apps page will reinstall your dockers just as they were. I'll research this. But I believe I've done it before. The large docker.img size is likely due to album art, etc. for Plex.
February 5, 20215 yr 51 minutes ago, FlexGunship said: The large docker.img size is likely due to album art, etc. for Plex. None of that should be in the docker.img if things are configured correctly.
February 5, 20215 yr Community Expert 15 hours ago, FlexGunship said: The large docker.img size is likely due to album art, etc. for Plex. I have plex and 17 other dockers and they take less than half of 20G docker.img. All that plex stuff is in appdata and the plex templates have it that way by default. 14 hours ago, jonathanm said: None of that should be in the docker.img if things are configured correctly. 20G should be plenty. Making it larger won't fix anything it will only make it take longer to fill. The usual cause of filling docker.img is an application writing to a path that isn't mapped. Linux is case-sensitive.
February 7, 20215 yr Author Material update #1: Performed repair on filesystem, looks good. Blew away all of my dockers, and rebuilt everything. Mostly back to normal. Previous problem h as begun expressing itself in the old way again. Image of monitor on server attached as well as logs after a reboot (had to reboot, response was too slow). athena-diagnostics-20210207-1644.zip
February 8, 20215 yr Community Expert Diags are just after rebooting so not much to see, you can try this and then post that log.
February 8, 20215 yr Author 3 hours ago, JorgeB said: Diags are just after rebooting so not much to see, you can try this and then post that log. Thanks, just enabled. I can probably induce the issue later by redownloading artwork for Plex, or just doing a heavy file transfer. Will post results after.
February 8, 20215 yr Author 9 hours ago, JorgeB said: Diags are just after rebooting so not much to see, you can try this and then post that log. 6 hours ago, FlexGunship said: Thanks, just enabled. I can probably induce the issue later by redownloading artwork for Plex, or just doing a heavy file transfer. Will post results after. Thar she blows....syslog
February 8, 20215 yr Community Expert There are some xfs related crashes, run a filesystem check (without -n) on disk2.
February 8, 20215 yr Author 13 minutes ago, JorgeB said: There are some xfs related crashes, run a filesystem check (without -n) on disk2. Put it in maintenance mode, and run on disk2 (removed the -n argument). Results below: Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 2 - agno = 0 - agno = 1 - agno = 3 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... done
February 8, 20215 yr Author 21 minutes ago, JorgeB said: Try again, leave remote syslog running. Test in progress. Transcoding some Plex, copying some trivial files, etc. Will report back.
May 16, 20215 yr Author Hey all, that seemed to improve things, but I've seen the issue again. Sorry for the delay. I moved 3000 mile, my daughter was born, and I started a new job.
Archived
This topic is now archived and is closed to further replies.