Vcent

Pretty sure everything has been copied, and I do still have the original drives. That being said, I am being told multiple, somewhat conflicting things by unraid by now (I've re-added the drive to the array, got the blue dot, started the array/rebuild). Unraid is simultaneously telling me that there's an unmountable disk present (disk 6 now, for some reason), and that it's doing a data rebuild. Unassigned devices is showing the problematic disk, and no writes are being noted by either UD or the array GUI (It's just reading the working disks, writing is a solid 0.0B/s, and disk 5 is shown with the yellow warning triangle denoting an emulated disk). I'm unsure if this is unraid being cryptic, and if it's working as intended, or if it's failing. Edit: and I'm getting a boatload of errors from drive 6 now, zero correct reads. Paused the rebuild so far. tower-diagnostics-20260626-0617.zip

Yeah, I'm learning that now. Mounted perfectly fine, once I remembered to point towards the mapper. Is the next step to move the data and nuke the drive?

I'm having a hard time figuring out if I've done it correctly - attempting to mount it with mount -o rescue=all,ro /dev/mapper/sdf1 /temp results in >mount: /temp: special device /dev/mapper/sdf1 does not exist. dmesg(1) may have more information after failed mount system call. Which seems reasonable enough (the drive was unassigned after all). mount -o rescue=all,ro /dev/sdf1 /temp Gets >mount: /temp: unknown filesystem type 'crypto_LUKS'. dmesg(1) may have more information after failed mount system call Which also seems reasonable enough, since the drive is encrypted. Hunting for a different mountpoint with df -h and btfrs fi show found me /dev/mapper/m5p1, which does indeed match the missing disk in size and used amount, and will allow itself to mount to /temp quite happily - and it does contain data.

It would appear that the answer to that question is no, it can't mount with the UD plugin.

My apologies.

I've disconnected it, but the drive is still shown as disabled/emulated/unmountable, even after a reboot. I must admit I'm blanking a bit on the correct procedure to bring it back online - maintenance mode -> drive properties -> BTFRS check --readonly gives an error suggesting corruption (expected XXXX found YYYYY, and then fails to open the filesystem as it can't be found). *Exact output was : parent transid verify failed on 3350031482880 wanted 502589 found 502438 parent transid verify failed on 3350031482880 wanted 502589 found 502438 parent transid verify failed on 3350031482880 wanted 502589 found 502438 Ignoring transid failure ERROR: child eb corrupted: parent bytenr=3350034989056 item=190 parent level=1 child bytenr=3350031482880 child level=1 ERROR: failed to read block groups: Input/output error ERROR: cannot open file system Opening filesystem to check...

SATA to motherboard.

Seems like it's my turn to bork something up, and worse yet, be swamped at work so little time for looking into what's happened (hence the almost three-month-old diagnostics). I had an old, slow HDD that I figured was up for replacement - I replaced it with a (very) occasionally flaky new, larger drive (which was dumb, but I figured it was a connector issue, not a drive issue). At first, it worked OK, until the drive dropped out of the array. Then it got rebuilt onto itself, and I can't remember if it worked OK again, or if it crapped out before that and wound up dropped again. Either way, I ran a few more cycles of preclear on the drive, rebuilt it, and the drive wound up dropping out of the array again once it was back up running, so I replaced that drive with a second new drive (presumed good, freshly bought - it did three passes of preclear with zero issues, and S.M.A.R.T testing was OK as well), rebuilt the array onto the second new drive, and the second new drive is now exhibiting the same problem - it drops out of the array and gets emulated quite quickly. TL;DR: Changed old small drive to a bigger HDD, bigger HDD gets dropped from the array some time after starting it, test bigger HDD with 3x preclear which is OK, rebuilt bigger HDD onto itself, drops out of array quite quickly, test x3 and rebuild same story, replace bigger drive with new drive, pre-cleared x3 with no issues, now new drive is dropping out of the array as unmountable. At skill limit for diagnosis. Connectors have been switched around, with no changes noted. I think the diagnostics are from one of the first times it happened with the new drive.

Update, in case someone else hits this lovely snag in the future: The problem was indeed a ram-stick, which worked perfectly fine until stressed juuust right, at which point it started outputting gibberish garbage, which crashed the system. Unfortunately, none of the memtests identified the ram as being faulty, passing with no errors every time (unless I started faffing about in the options while a test ran, which would sometimes make a ton of errors appear - I'm guessing that's a bug in memtest though, rather than indicative of this particular flaw). I ended up finding it by removing one stick, running it for a day, stressing the system, and when it didn't crash, tried swapping the sticks to verify the problem - and indeed, it promptly wound up crashing the system once everything got loaded hard enough. Curiously all memtests regardless of runtime still insist that everything is peachy, and that there's nothing wrong with the working, or the faulty stick. System has been both stable and dependable after removal of the problematic stick.

Probably too late to help you, but the solution to that is to type in 'digikam' without the quotes, and being slightly patient as it is launched. Unfortunately launching it that way, leads to a tiny window with digikam opening, since Guacamole for some godforsaken reason assumes window size should remain static, and squashes digikam into the same window size as a terminal window. I found some sort of way to fix that, but unfortunately it's been long enough by now, that I've forgotten how I did it.

It's currently on the last 25% of the final (fourth) pass of MemTest86 Free Version 9.3, which much like the included memtest86+ on unRaid has found .. diddly squat, except that there are no issues with the ram. I would be highly surprised if that changes during the last hour of the test, seeing as the included memtest was run several times previously, and also didn't find anything during any of the passes either. Unless unRaid is somehow significantly harder on the ram, since it manages to kill the machine in a much shorter period of time than any of the test runs. It's done with its run, finding nothing. The advice it gives to run it again in multi-CPU mode is nice, but not possible on my motherboard/CPU combo, due to some UEFI limitation on it. Don't have any other AMD boards lying around to test with either. I'm letting it run once more through the night, but I'd be gobsmacked if that changed the result.

Did you ever figure this out? Having the same problem, and have had it for a while now - tried changing my OVPN file to a different endpoint, didn't change anything - figure it's the torrent client trying to contact itself through the VPN, which OpenVPN doesn't like (for obvious reasons).

Been having the syslog server (or rather mirror) up for a while now, problem is that it rarely captures anything particularly interesting - a docker will drop a net connection, then make up a new one, and so on, until eventually .. the server just stops responding to anything, usually starts producing heat, and spins up the fans. Oftentimes the last message will either be about the dropped (local) IPv6 address, or about the disks spinning down, then nothing else gets logged. And that's it, nothing more was logged. For once it actually managed to log ..something, rather than stop around 13:25:59 (withdrawing address, making a new one, yadda yadda, server unresponsive). DM-3 shows up in the log, which is as interesting as it is annoying, since I still don't have a cache device, although it might come from the system having a misconception somewhere about having a swap file (which it doesn't anymore, and swap is indicated at 0kb)? I actually thought for a while that I had figured out the problem, as a misbehaving docker got corrupted, and started spewing out files into its config directory, and since I was running minimal plugins, nothing ever reported that it was doing so, or filling up. Fairly sure I've fixed said docker, or at least directed it at a suitable target, but still - server goes space heater less often now, but it's still happening far too often to be useful. I'm guessing that the almost constant "Old network address died, making a new one" is due to a docker VPN, which itself keeps detecting a loopback which makes it kill off the packet ..this happens not infrequently, but interestingly enough everything works ok(or at least as expected), and I can't quite figure out how to stop it from happening - the issue is that a client ends up trying to send packets to itself, through the tunnel, which uhh.. OpenVPN obviously doesn't like, and kills the packets. Currently I'm guessing the shutdown is due to a thermal issue, but I can't conclusively say that it is - There's fairly decent cooling overall, although the area around the USB slots/North bridge does seem to get fairly hot for some reason. I do however have some pictures of the on screen/console output once the server goes down, sadly they're pretty much all the end of a trace, with a bunch of register addresses which mean ..nothing to me. Most understandable was > Kernel panic - not syncing: fatal exception in interrupt > Kernel Offset: disabled > ---[end kernel panic - not syncing: Fatal exception in interrupt ]---

And I guess more bumping. System is by now killing itself multiple times a day, although I did manage to trace the php killing to a docker container, which has been removed. At this point even thinking about a parity check is a joke, as it just slows down the server before the next kill restarts the process. There's nothing particularly consistent about it - sometimes I'm doing something that uses a good amount of resources and it works fine, other times the server dies, often it's just left on its own, then dies, one time I've even just left it at the unRaid login prompt, with nothing mounted or done (not even logged in), and yet it's managed to kill itself in the ~10 hours or so it was just ..standing at an idle login prompt, with no workload at all. The logs are of little help to me, and I can't claim to be able to decipher the kernel panic log that remains on screen whenever the server crashes - it's not even the exact same every time either, although it does consistently appear to be of "Not syncing: fatal exception in interrupt" type, whatever that means beyond a fatal error in an interrupt though, 'i haven't the foggiest.

Update, I guess. Not going to be helpful for anyone with a similar problem, I suspect. Got parity upgraded by rebuilding it, then upgraded drive by rebuilding onto the old parity. So parity swap, manual style. Quite annoying, but worked first time I tried it. Currently the server is busy crashing/freezing itself about every 1-2 days, killing some flavour of php (php-7 I think?) for using too much memory, due to a pathetically low limit being set ..somewhere, that I can't find. Apparently it can only use ~270 something mb of ram, despite the system having 16gb available, yet that is above the limit, so it gets reaped by the oom killer. I'm also getting errors for a DM-3 device, which is curious, as I don't have a cache drive installed (never have), and I can only find mentions of that address/designation in threads about cache SSDs. Dockers have been nerfed, all running memory limits, none of them approaching them, only running a handful of dockers changes ..nothing, and at this point essentially all plugins have been uninstalled, to no help. System still kills itself randomly, with regularity, and displaying the same symptoms each time it happens: Fans are running at a decent clip, pumping hot air out of the case (despite unraid being frozen and unresponsive to even a basic ping), interestingly the network interface lights still light up, but overall apart from functioning as a space heater, the server is not functional.