codefaux

(Snipped for brevity) What does "some trouble" mean? Were you unable to delete things you should have had the rights to? How did you resolve it? What other actions did you take after it would not delete? (ie; closing/killing the container, recreating the container, reading logs, etc) When you copied the folder, was the container still running? This is a Very Bad Thing(tm), as files are not in a "ready to be archived" state during operation; lock files exist which should not, temporary files exist which are not necessary, in-progress operations can be clipped, files are very much not properly committed to disk (ESPECIALLY with a database engine) etc. The -first- error you mentioned indicates that DBAL was unable to connect to the database. PHP was working, or DBAL would not have been running. DBAL was working; it was merely unable to connect to the databse, which is not an internal failure to DBAL or PHP. This was your indication that the database was your first problem. Changing the PHP version was not a good idea and not relevant to repair, and it should be changed back. The actual error is indicated in your DB message. `"mysqld was not shut down properly last time and critical recovery information (last binlog or tc.log file) was manually deleted after a crash"` -- aka, you copied a bunch of files from a running databse system and later restored them by erasing the existing files and copying the "while-running" files back, or it crashed and a bunch of files were lost/damaged. (PS: Never, ever restore database files over other database files after an update/downgrade; between versions, database engines and software using databases sometimes make broad, sweeping changes to database structure and/or database layout. Simply wiping and replacing the files does not prepare them to be understood by a different version. Sometimes this is safe, when it isn't it breaks absolutely everything. It's not worth the risk.) `How can I do this within the container? -> start mysqld with --tc-heuristic-recover switch to commit or rollback pending transactions` Before I answer that part -- this may or may not actually be a good idea now. After having so many things changed, and doubtless changes have been made which you have not mentioned, the system is in a very unknown state. Someone needs to sit down with you, talk over everything you did, decide if any of it needs to be reverted (and in what order) and settle the best next action from that point. That said -- Given that this is an all-in-one container, mysqld is instantly restarted as soon as it goes down. As such, this container can not be used to fix this problem. It is possible for an experienced database/Docker user to use a mysql container with your cleanest copy of the database files to recover from this situation. If you're that user; - You need to create a new container using the same mysqld (or mariadb, I haven't checked) version, using the same database config file this container does, with the same volumes mounted to the same places. - When launching the container, you need to override the startup of the container to instead run only a shell. - Manually launch the database daemon with the `-tc-heuristic-recover` flag added to the usual start-up command. This could also be done using WSL or a Linux boot medium, the process is more or less the same, and definitely not approached by someone who doesn't understand what's going on. TL;DR -- find a pro, or wipe it and start over.

Super great work getting it all working in one box. Two notes. Respectfully; I strongly suggest putting this in the Template's "Overview" section for the Container Template itself. None of this is intuitive or safely assumed. There's no limit on how much text you can put there, and you already have sections for configuration -and- notes there, which are also required to set up the container -- but not listed in the Install text above. Putting this information all together and in the Overview would remove "go to the Support forum for more information" from the unlisted requirements to using this container. Also, regarding "Change the data folder to /data" -- why not just bind the user's data volume to `/config/www/owncloud/data` (which is OwnCloud's actual default) instead? If the template default is to bind to `/config/www/owncloud/data` instead of `/data` new users don't need to change the data folder, it will simply "just work" without that step. The user won't see any difference. Presently, if the user doesn't know to do this or misses/misconfigures this step, their database gets wiped when the container updates and they just suddenly lose stuff.

I'm not sure what the question on the first line was, seems like it got deleted. Your terminal is configured to behave as an xterm supporting 256 colors. xterm is an xorg terminal emulator, and does not support automatic blanking / blanking control via the routes a direct console terminal does, as inidicated by the "not supported" part. setterm is designed to comminicate behaviors to hardware terminals, not software ones (like xterm, screen, tmux, ssh, etc) and thus cannot set or unset powersave mode or blanking as indicated.. If you're using screen or tmux from a hardware terminal and they're using xterm-256color due to misconfiguration, the two are terminal emulator shells which cannot be controlled by setterm in any way, and they must adhere to the actual terminal's properties. IE, setterm before screen or tmux. If screen or tmux are doing blanking, configure them directly, setterm does not impact their behavior. If you're using xorg, setterm will not prevent xorg blanking, as setterm is meant for a hardware terminal connection, not an x session. That requires different commands which aren't applicable here. If you're using ssh, it shouldn't -be- blanking unless one of the other layers is adding it, but just for the written record, setterm won't work over ssh either.

OK but I need you to understand that the dd is specifically done in my suggestion for a very direct and considered purpose. When doing parity calculation, two exactly identical drives are nullified, and can be remove from any parity-protected array while preserving parity data. That's why I'm seeking people with experience in what I'm trying to do. The dd is critical to my intent; not having to do another parity sweep, after replacing my drives and migrating my data. Second, I stated that my intent is to migrate the data off of the drives before beginning this entire process. The drives will have no important data on them. I just want them out of the array without doing a parity sync which is entirely possible, if I can figure out how to get the array to start or bring up shfs without those two drives mountable. I don't need someone to refine my intent, or correct what I want to do. I don't need advice on how to proceed. I need advice from someone who understood everything I've said, and knows the answer. I need someone who can tell me if there's any way to start the array or bring up shfs without those two drives mounted. I sincerely and greatly appreciate the efforts of anyone willing to so much as read this all, but if you don't have the answer to my question, you're wasting your time. My time isn't worth anything, I'm the one asking the favor.

My primary concern with using dd on a live system is that I need for Unraid to not have those drives mounted. I can remove them from shares, but I suspect shfs still mounts the underlying filesystems. With a filesystem mounted, a dd on the underlying partition can leave the kernel [writing to | reading from] the disk still before it realizes inconsistency, and that can cause panic and/or just random bytes written on the disk I'm trying to mirror. The secret sauce to option 3 is, I think, figuring out how to prevent it from being mounted when starting the array, which `wipefs -a` will accomplish, but I'm not clear on wether Unraid will allow the array to start in this state -- it's an odd mix of "new disk" and "unexpected configuration" that will probably cause issues. I may just need to build a VM with a ton of empty disks and see what happens when I poke it while it's running, rather than theorizing.

I assume you mean the Unassigned Devices plugin by dlandon? I'm familiar with it, but that doesn't prevent me from needing to do a full parity rebuild, which is my primary goal.

This is not helpful. I would love to have over two thousand dollars to burn on hard drives and just make it done right now. I'm not talking about days for these upgrades. I'm talking about years. It takes several months for me to afford a spare drive right now. I need for people willing to discuss the details of the options I presented. I'm looking for discussion of the inner workings of Unraid, its technical and presentational response to the actions I considered. Deep knowledge. Experience with the things I discussed. The literal most obvious solution has been considered and discarded, as well as the next few. If you can't provide that thing, please don't waste your time trying to answer this. If I've put you off and you can provide that information, I accept that but I was pretty clear about my ask.

Before I start -- If your reply is going to be in the format of "Suck it up and do Option One" I would rather you left the thread clear for replies more directed toward my actual questions than airing criticism of my decisions. I know how it's Supposed To Be Done(tm) but have my own excuses for not doing it that way. While I will explain my reasoning, I'm not here to defend them; I'm here to discuss the nuance and methodology of the other options with users interested in or experienced with fiddling under the hood manually, and to document my experiences with it, for the goal of helping the next fellow like myself. Hi all. I'm a long time Unraid user, I spend literally most of my time under the hood of either this server or one of my other spares. I need advice from some deep knowledge folks. When I'm done I'll try to rework this post to share some of that deep knowledge for similar-seekers. It is, after all, our way. If you're looking for the path to my goal, it's edited at the bottom of this first post IF I have found it. (It isn't. Yet.) My server has grown to too many disks, and I'm looking to prune it. Power demand and parallel disk IO loads are getting a bit out of control. Too many SAS cards, can't add a second 1650 Super for transcoding without losing my 4x NVME-PCIE bifurcation card for the Docker containers and game servers. Homelab problems, am I right? Thusfar, I've got a 16tb disk replacing a single 3tb disk in-system. Data rebuild in progress, properly emulated, all is happy because Unraid is awesome like that. I -need- to get the 16tb up before removing the other two 3tb disks because they're pretty full and I don't have 6tb spare elsewhere. After this disk comes online, I intend to remove another two disks. My next few upgrades will be the same -- install a single 16tb drive with the goal of removing three 3tb drives. Obviously step one is remove the data from the disks. From here I know of three options, but I only know how to accomplish two. What I'd like is for a sanity check (I've got time for advice before I can act, lol) and possibly a solution for Option Three from an experienced poweruser, maybe even a dev. Option One: The Unraid GUI End User way -- Remove the two drives. New Config, rebuild parity. - I don't want to do this. This means a 16tb parity rebuild, removing two 3tb disks. Option Two: "The Math Works Out(tm)" -- In parity calculations, two identical disks cancel each other out. - This means no parity rebuild, just downtime during the 3tb disk-to-disk clone. - Here's my plan in Maintenance Mode. We'll pretend I'm using /dev/sda and /dev/sdb as disks to be removed, for simplicity. 1 - Enter Maint. Mode (obviously) 2 - Wipe filesystem and partition table identifiers (same uuid on two filesystems can cause problems) and clone drives wipefs -a /dev/sd{a,b}{1,} dd if=/dev/sda of=/dev/sdb bs=32k 3 - Stop array, New Config, trust old Parity. Option Three: "The White Unicorn" -- same as Option Two, but while the array is fully running, NOT in Maintenance Mode. My sole idea on how to accomplish this is -- - Enter Maint. Mode - wipefs -a so drives can't be identified - Stop Array, Start Array -- Will the drives present as two new disks? Maybe? Unmountable and prompting to format? Or will Unraid refuse to start? - Run dd between disks while Unraid runs. Is this a thing? Is there any other way to bring up shfs with the two drives unmounted, but keeping them in md for parity? Thanks in advance.

That doesn't support any of the other metrics from any of the other dozen or so scripts at the link I mentioned, a few of which I'm interested in/using now manually. That's why I suggested adding the text collector as a supported feature on the plugin by default. It won't impact anyone except people who want to use it if it merely watches a folder for scripts to execute. Could you be bothered to explain why? I don't mean to sound dismissive or aggressive or anything, but would you follow a recommendation from a stranger with no justification or explanation? I'm not looking for suggestions; I'm not the plugin author here lol I'll allow them to be run from anywhere you require them to be placed. I'm suggesting that you add in support for the text collector to your prometheus_node_exporter plugin, since prometheus_node_exporter already supports it. I'm suggesting where you should place them. You disagree; place them anywhere you like.

Could you be convinced to add a variable to pipe stdout to a specific path? My target: prometheus_node_exporter contains a textfile collector. There are a handful of community scripts designed to - be run from cron (ie your plugin on a custom schedule) - collect useful (I'd argue critical) system metrics (ie smartmon stats like temps and failure indicators) for prometheus-node-exporter - output data to stdout, to be piped to a file The inconvenience is, to use these is not straightforward. They would require extensive modification most users can't grok. Thanks to your plugin, an alternative is to manually save them to disk someplace, then use your plugin to write a script which runs them with stdout piped to a file, scheduled cron-alike. EDIT: These are the same script, ignore the name difference I used an old screenshot for the one by mistake This is workable but some users may still find this unapproachable. If the user could paste the entire script (in my case, /boot/config/scripts/smartmon.sh aka smartmon.sh [raw] among others) into a User Script, schedule them as cron-alikes, and mark their output to be sent to a specific path, it would make these more approachable. It could be implemented similar to the variables you've recently added; an STDOUT variable could be changed to request redirection. Regardless of your decision, keep up the great work. The plugin has been quite valuable for many of us!

Hey here's another suggestion for an easy addition full of feature delivery. There exists a handful of community scripts to ammend data provided by node_exporter. Most of them seem to be intended to pipe to a text file and read via node_exporter's textfile collector. The metrics go into a file wherever, then to the exporter as a launch parameter, you add `--collector.textfile.directory=/var/lib/node_exporter` for example, and all of the readable files within /var/lib will be exported as metrics if they're in the correct format. For example, smartmon.sh writes smartmon statistics such as temperature and unallocated blocks. nvme_metrics.sh might be of interest, btrfs_stats.py, maybe even directory-size.sh for some folks. The most simple way I can think of is for your plugin to create a directory in the ram filesystem at /var/log and add `--collector.textfile.directory=/var/log/node_exporter` and suggest users execute the desired scripts, writing into /var/log/node_exporter in per-script files. I can see two ways of doing this. - One, users copy script files to someplace like /boot/config/scripts (one-time write for the scripts, no flash wear) and execute them via the User Scripts plugin as such; Scheduled similarly; The /var/log filesystem will exist on any system, won't cause flash wear, is wiped on reboot. The path should have plenty of space for a few kb (or even dozen MB) of metrics being rewritten every few minutes. If it doesn't, failure case is that logging fails on the system -- not ideal but it's mounted with 128MB by Unraid and should never be near full unless a user has a serious problem elsewhere. If it's filling, the absense of this plugin's proposed metrics won't prevent it or even delay it by much. These metrics are designed to be overwritten, not appeneded, so they should never grow more than a few dozen MB in the most obscene scenario. Plugins seem to run as root, so permissions shouldn't be a problem. I'm also going to ping the user scripts developer to allow stdout to be piped to a file per-script, so users can simply paste the scripts into User Scripts and forward the stdout, instead of needing to save them to /boot/config/scripts manually and write a User Script to run it manually.

Might be worth running it from the console to see if there's any ignored error spam like in my comment. Could be an error being dropped instead of forwarded to system logs, like on my (and everyone else's) system using the Node Exporter.

lol it's ok, I was just very confused. I'm not even using the Docker containers, I JUST needed the exporter; I'm running the rest on another system. That's the main reason I went into this so sideways; I only wanted the exporter, but it had all these extra steps and I had to assume which went to what. I did so incorrectly, but that's ok. For sure! I thought the file went with the plugin, that's all. Yup, but I didn't know that and it was all in one giant block so I assumed (incorrectly) and here we are, lol That's very much what I had expected, literally everyone using this plugin is silently ignoring repeated errors because the stdout isn't directed anywhere people know about... Synonymous with "remember that" -- it's something you say to people who know the thing you're about to say. COMPLETELY did not know that, but given the reply upstream that does not surprise me anymore lol. I'll remember that. Since the logs aren't kept, it won't be filling up a logfile. I haven't restarted my array in a few months and I don't intend to soon, so I'll likely just leave them running in a detached tmux until the issue is properly resolved. I'd already opened a bug report upstream, I'll link them here and add this information but it seems for now the best bet would be to patch the plugin to disable the md collector on your end. Edit: Github issue https://github.com/prometheus/node_exporter/issues/2642

I already explained that, but I'll be more verbose. I downloaded the plugin before I wrote its config file. You wrote the instructions. Steps four and five. I did not perform steps four and five until after I had installed the plugin. I also did not do anything beyond those steps, regrading modifying settings, configuration or parameters. There's no "restart" control that I could find, and I didn't feel like restarting my entire server (or the array and all of my services) simply to restart a single plugin. Thus, I used the console, killed the running process, and restarted it by hand. No custom parameters, I didn't change some settings. You'll note that I never said there was a user-facing issue, or that I couldn't connect or report metrics from it. It functions just fine, but on my system it's burping a line at the console about an internal (non-critical) error, every time Prometheus connects to it and retrieves metrics. The only difference between now and if I uninstall/reinstall/reboot is that the errors will be sent to a logfile someplace or discarded entirely -- I have no idea which -- instead of being sent to the console, since I ran it by hand. What I'm realizing though, is this is above your head, so to speak. If you run it by hand yourself, does it throw an error every time Prometheus polls it? I'll file a bug upstream with the actual node collector's maintainer, as it's now clear to me that the actual collector is mishandling the output of /proc/mdstat on my system and it has nothing to do with the small wrapper plugin you wrote. Mmmmmmm no, though. It's not "only meant to be installed." It's meant to be learned about, your system is meant to be manually prepared for it, and then it's meant to be installed. Steps four and five could/should be handled by either the previous step's container, or this plugin itself if it notices the configuration file is not present. Furthermore, installing the plugin starts the collector, which already expects the config file to be present, so steps four and five should actually be before step three. If the installation process weren't so complicated, I would've noticed that this wasn't your problem earlier. I installed the plugin by finding it in the CA app and going "hey that does what I need" and then discovering that it wasn't working. And in NO situation do you merely "install the plugin and that's it" so that's just a flat inaccurate thing to claim.

Literally just run the binary from the ssh terminal. I hadn't written its config file yet so I noted how it was executed, killed it, wrote the config file, and executed it by hand in a terminal.