Rojen Mcche Posted May 15, 2020 Share Posted May 15, 2020 Hello all. I have been having an issue that seemed to start with Plex and now seems to be effecting other aspects of my server. I keep getting either "500 Internal Server Error nginx" and "502 Bad Gateway nginx" both of which seem to lock up my server. From what I have been able to deduce from reading other posts, is that it started by moving a large amount of data onto my server and potentially corrupting my nvme cache drive. So now whether it is plex or more recently backing up personal photos, either of those tasks seem to give me trouble, which then requires me to do a hard shutdown of the system. I am fairly new to unraid and am looking for any support with this issue. Thank you in advance for any assistance. Quote Link to comment
testdasi Posted May 15, 2020 Share Posted May 15, 2020 Start with attaching diagnostics (Tools -> Diagnostics -> attach entire zip file). 1 Quote Link to comment
Rojen Mcche Posted May 15, 2020 Author Share Posted May 15, 2020 (edited) Thank you @testdasi. See attached... jenkinsnas-diagnostics-20200515-0833.zip Edited May 15, 2020 by Rojen Mcche Quote Link to comment
testdasi Posted May 15, 2020 Share Posted May 15, 2020 That looks to be after the crash so probably isn't much info. Go to Settings -> Syslog server and turn on mirroring to flash. Then simulate a crash and make sure it actually crashes (i.e. don't be too quick to hard reset, wait maybe 5 minutes and make sure it's still hanging). Then attach the syslog + diagnostics, which hopefully now contains stuff before the crash. 1 Quote Link to comment
Rojen Mcche Posted May 15, 2020 Author Share Posted May 15, 2020 On it. Thank you for your help. Will report back soon Quote Link to comment
Rojen Mcche Posted May 15, 2020 Author Share Posted May 15, 2020 @testdasi Started backing up family photos and am now locked out. Interesting as I am not getting either of the errors from my title, but I am locked out of the server and the monitor that is hooked up to the server is going crazy in a command line with text scrolling rapidly across the screen. With no access to the web gui now, not sure what to do. Quote Link to comment
testdasi Posted May 15, 2020 Share Posted May 15, 2020 Take a picture of the screen? Quote Link to comment
Rojen Mcche Posted May 15, 2020 Author Share Posted May 15, 2020 @testdasi see attached... Quote Link to comment
Rojen Mcche Posted May 15, 2020 Author Share Posted May 15, 2020 one thing I noticed in that still is a reference to Wireguard. I did recently set that up to use as a way for my family to back up remotely to this server. Anyone know if wireguard causes stability issues? I do remember something about this not being fully supported or something to that effect Quote Link to comment
Rojen Mcche Posted May 15, 2020 Author Share Posted May 15, 2020 this is what I get from the syslog whenever I start the plex docker... May 15 16:12:18 JenkinsNAS kernel: BTRFS warning (device nvme0n1p1): csum failed root 5 ino 721987 off 16384 csum 0x505100ca expected csum 0xdc5e0680 mirror 1 May 15 16:12:18 JenkinsNAS kernel: BTRFS warning (device nvme0n1p1): csum failed root 5 ino 721987 off 16384 csum 0x505100ca expected csum 0xdc5e0680 mirror 1 May 15 16:12:18 JenkinsNAS kernel: BTRFS warning (device nvme0n1p1): csum failed root 5 ino 721987 off 16384 csum 0x505100ca expected csum 0xdc5e0680 mirror 1 May 15 16:12:18 JenkinsNAS kernel: BTRFS warning (device nvme0n1p1): csum failed root 5 ino 721987 off 16384 csum 0x505100ca expected csum 0xdc5e0680 mirror 1 May 15 16:12:18 JenkinsNAS kernel: BTRFS warning (device nvme0n1p1): csum failed root 5 ino 721987 off 16384 csum 0x505100ca expected csum 0xdc5e0680 mirror 1 Quote Link to comment
testdasi Posted May 15, 2020 Share Posted May 15, 2020 Force shutdown your server. Grab the syslog from the USB stick and post it here. Boot to BIOS and make sure there's no overclock and your RAM is running at 2133MHz (i.e. stock DDR4 speed). Then run memtest for 24 hours. BTRFS is very sensitive to RAM issues so need to eliminate that first. 1 Quote Link to comment
Rojen Mcche Posted May 16, 2020 Author Share Posted May 16, 2020 @testdasi can confirm both processor and ram are running stock. syslog attached. will report back what i find from memtest. syslog Quote Link to comment
Rojen Mcche Posted May 16, 2020 Author Share Posted May 16, 2020 and I am not exactly sure what a successful memtest looks like, but something tells me this doesnt look good. see attached... Quote Link to comment
JonathanM Posted May 16, 2020 Share Posted May 16, 2020 22 minutes ago, Rojen Mcche said: but something tells me this doesnt look good. Did you set the memory speed in the BIOS? I'm not sure how to interpret the numbers I'm seeing, does CLK: 3593 MHz mean anything? And yes, any fail on memtest is severe, and must be eliminated before things will stabilize. Assuming the memory is not running an overclock (XMP) profile, I would try swapping sticks around and running on one stick at a time in memtest until you get zero errors over a period of not less than 12 hours, I wouldn't trust my data to a server that didn't run at least 24 hours with a clean memtest. 1 Quote Link to comment
Rojen Mcche Posted May 16, 2020 Author Share Posted May 16, 2020 the clock speed shown is the processor. I am letting this run over night and will swap sticks around in the morning. unfortunately I can't monitor for a 24 hour period as I am going out of town Quote Link to comment
JonathanM Posted May 16, 2020 Share Posted May 16, 2020 11 minutes ago, Rojen Mcche said: I am letting this run over night and will swap sticks around in the morning. Really no point in continuing a test with a failure shown. 1 Quote Link to comment
Rojen Mcche Posted May 16, 2020 Author Share Posted May 16, 2020 got it. thank you for your support Quote Link to comment
Rojen Mcche Posted May 19, 2020 Author Share Posted May 19, 2020 @testdasiback in town and did a 24 memtest with one ram stick. no issues. however, still getting the BTRFS issue whenever I run the Plex Docker. No issues as of yet with the server locking up with data transfer, but I am still monitoring that for now. Any thoughts regarding plex? Thanks again for your help Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.