engels74

I solved it - I'm an idiot. The Unraid GUI shows in TB, while the df on the VM uses TiB.

Hello Unraid community! I really hope you can help me out. I've tried attaching as much information as possible to this post. If you need any more information, please tell me and I will update the post accordingly. ## My Setup I've been running Unraid v7.0.1 for about 7-8 months and have been very satisfied with it. My array consists of: - 16x 20TB data drives - 2x 20TB parity drives - Total array size: 320TB ## The Issue I've mounted my entire array to an Ubuntu 24.04.2 LTS VM using VirtioFS. However, I'm noticing a significant discrepancy in reported free space: - Unraid dashboard shows 185TB free - Ubuntu VM shows only 172TB free (using `df -Th`) That's approximately ~12TB of "missing" free space, which seems excessive for normal overhead. Is this expected behavior for VirtioFS mounts? ## Mount Configuration - Created a share named `vmdata` in Unraid - Share settings: `Virtiofs Mode`, `User:vmdata` for "Unraid Share Mode" - Ubuntu VM mounts via `/etc/fstab` entry: `vmdata /data/mount virtiofs defaults 0 0` ## Screenshots and Logs Array status from Unraid Main tab showing 320TB total with 185TB free: My Ubuntu VM is showing different free space: ➜ ~ df -Th /data/mount Filesystem Type Size Used Avail Use% Mounted on vmdata virtiofs 295T 124T 172T 42% /data/mount ### Additional Logs I've attached comprehensive diagnostic logs to help troubleshoot this issue: - Full Unraid diagnostic logs (attached to post) - Full `lsblk` output (Unraid OS): https://logs.notifiarr.com/?25c8ca9014dc6d19#F971spXoWs5W3zCmVit5ScSfrsxfkkeHVXTS8N3EXzLB - Full `df -Th` output (Unraid OS): https://logs.notifiarr.com/?7503c1897084ff40#7X2ajmC6vyKNZdzsYfk1hWpUNGNtVp9m5Vvm1dvg1DCM - Ubuntu VM XML configuration: https://logs.notifiarr.com/?bb9887c3a4f2cb4a#DenZ7LFdqkJm5K7Lc4SgAUpm4aS2L6jmzsS3oP9sp2hh - VM logs from Unraid OS: https://logs.notifiarr.com/?a5458d695e993f90#5ZTjDyJ32AQtirfVRRmACCyt9NW1rkvMgbXkB6pZ8Fdq - Full `lsblk` output (Ubuntu VM): https://logs.notifiarr.com/?b5d44327407193ad#HaAXDNc5MRY4KghupnVsFVjAfQ1TyB5zEDY82x61qstR - Full `df -Th` output (Ubuntu VM): https://logs.notifiarr.com/?f76070f711795c9a#48Q54QC6K8jAukpN5igHYj7KxkcAhtuyHJXQYezXGQ6T Any insight into whether this space discrepancy is normal or if there's a configuration issue would be greatly appreciated! Thanks in advance for your help. tower-diagnostics-20250307-1921.zip

I'm not the admin of the site / forum, if that's what you are referring to. I'm the developer of the moverStatus.sh script and that's it. If you need help, don't ask to ask, just ask.

Hello again, I had to reboot my system today to update the BIOS, so I got to test it - it works great with that option. It starts as it should, and the User Scripts plugin does detect that it's running. I'll probably update the GitHub's page readme to include this option Thank you for showing that little option. It makes the script easier to utilize, now that it's no longer required to manually launch it! 😁

I'm not adding more providers right now. I would love Apprise, Gotify and all those as well, but currently I'm not happy with how the bash script is getting more and more "complex", with each provider having it's own way of formatting, and the script having to account for that. I've made a more lengthy answer here. Short answer is that I'm probably going to restructure this Mover Status project into a more fully fledged, modular Python project. Hopefully I can make it work. But it'll be a while, before I get to it. I'll gladly accept PRs Are you referring to the "arrayStarted=true" variable in the User Scripts plugin? If so, then it's not how that works unfortunately. The "arrayStarted" variable only creates a "check" for the User Scripts plugin, to only allow the script to be started, if the array has been started. It won't start automatically. If you're referring to anything else, please tell me :). I mean, it should work fine, as long as it's the User Scripts plug-in that starts the script, as the plugin is the one controlling the lockfile, and the status of whether or not the script is running. Sadly I've yet to find a way to circumvent or work with the User Scripts plugin's lockfile. I have tried explaining it on the GitHub page, on why I can't make it work. Trust me, I think it's annoying as well. My current long-term plan is to move from a bash script to a more fully fledged Python project, that will work as a Docker image for Unraid. That should hopefully work a lot better, if I can get it working. But that will be a fair while before I have time and energy to do proper work on that sadly. Edit: Oh wait, I see what you mean! I haven't seen that option before lol, I'm apparently blind 😎. For anyone else, that is as blind as me, you can find the option here. That could work, hopefully. I haven't tested it,. I'm going to apply that setting, and I'll check next time I reboot my system. It can still be the case of User Scripts plugin losing track of whether or not the script is running. To check yourself, you can always do this command in the Unraid console: ps aux | grep moverStatus The "moverStatus" should be replaced with the name of your script in the User Scripts plugin. The output should be something like this, if it is running: root@Tower:~# ps aux | grep moverStatus root 5763 0.0 0.0 4052 2268 pts/0 S+ 22:45 0:00 grep moverStatus root 19454 0.0 0.0 95616 29964 ? SL Jul17 0:00 /usr/bin/php /usr/local/emhttp/plugins/user.scripts/startBackground.php /tmp/user.scripts/tmpScripts/moverStatus/script root 19455 0.0 0.0 3980 2976 ? S Jul17 0:00 sh -c /tmp/user.scripts/tmpScripts/moverStatus/script >> /tmp/user.scripts/tmpScripts/moverStatus/log.txt 2>&1 root 19456 0.0 0.0 4188 3360 ? S Jul17 1:48 /bin/bash /tmp/user.scripts/tmpScripts/moverStatus/script If it only shows this: root@Tower:~# ps aux | grep moverStatus root 5763 0.0 0.0 4052 2268 pts/0 S+ 22:45 0:00 grep moverStatus Then the script is not running. Again, I'll test it next time I reboot my system and update this post. Thanks for making me aware of the option :). Here's hoping it works!

That's exactly what I do - it works wonderfully

Very happy that it works on your end ! Spent so many hours on it, so much trial and error haha. I have a fairly basic understanding of bash, and that's it. I would be glad to spread the word, but I wouldn't know where to start honestly

I finally got to a proper release of the v0.0.5 of this script. A lot of work went into it. If you are upgrading to v0.0.5 from v0.0.4, I recommend re-reading the installation guide on GitHub, as the script no longer supports the "cron" scheduling in the User Scripts plugin. The full changelog can be found here: https://github.com/engels74/mover-status/releases/tag/0.0.5

Hey mate, I think the current version (v0.0.5-alpha16) is in a pretty good state: https://github.com/engels74/mover-status/blob/feat-combinedVersion/moverStatus-combined.sh I did a bunch of work on it, but it should work as intended now. Can I get you to test it out, see how it works out in your end? It still has some debugging stuff enabled, I'll probably remove that for the proper "release". I still haven't tested it for multiple consecutive runs (letting it run multiple loops), but hopefully that works as well. Edit: The loop seems to work. I've yet to test: - If the script works, if only "USE_DISCORD" or "USE_TELEGRAM" is enabled - What will happen with the log in Unraid's User Scripts plugin, if you try and run the script, while it's already running. A few versions prior, it removed the entire previous log.

Yeah, that's what I'm seeing as well. I'll add some more debugging, and then hopefully get some kind of useful "response" from Telegrams API. I'm guessing something's being sent malformed. Especially having "USE_DISCORD" and "USE_TELEGRAM" set to true at the same time, seems to cause some weird behaviour. On my Discord server I got an odd amount of updates, nothing really consistent, and on my Telegram channel I only received the initial message, and the last completion one. I'll push an update to the script soon, with some more debugging, and then I can hopefully do something with whatever response I get from Telegram and/or Discord. Edit: v0.0.5-alpha10: So I added some debugging logging all over the script, hopefully that'll show where / how it fails. I also tried adjusting the "LAST_NOTIFICATION" logic a bit, not sure if it'll be fruitful or a failure. I've had a rough night with little sleep, but here's to hoping v0.0.5-alpha11: Added even more debugging / logging, since I can't really reproduce why or when it's happening . I'll let it run overnight again, and take a look at the log tomorrow. Don't have too much time this week to work on it sadly. v0.0.5-alpha14: I think we're getting there. Something was malformed being sent to both Discord and Telegram, if both were enabled. I tried to mitigate this. Also made the logging a bit smarter.. I hope. The percent increments should be more accurate now as well (again, hopefully). v0.0.5-alpha15: The alpha14 didn't solve the issue completely, tried another approach. Also introduced a new variable, that should help the percentage calculation being more accurate. Man, I'm hoping it'll work now lol.

Okay, so I did a lot of work on this one :). Hopefully it'll work now. Changes: - I added a DRY_RUN option, it's set to "false" by default. If you set it to true, it'll try and post to the webhooks, to see if that part works, and exit the script. It'll also do the calculations for the /mnt/cache, but everything else it just bogus data - I added a cooldown for the echo'ing to the logs for 10 seconds, since it would spam up the log, when it didn't find the Mover process. So now it'll look for the Mover process each 60 seconds, instead of each mili-second. I think that's.. Fine. There's probably a better way to do this, but for now, it works fine without spamming up the log too much. You should probably clear the log file at some point, as it will probably fill up the log over time. Again... I'll try and figure something out. - I think I fixed the previous issues, but I haven't had a "real" try with the new version yet. So hopefully it'll now start at 0%, like it's supposed to. - Probably fixed some other stuff, but I really hope it'll work now. I'm leaving mine overnight and see what happens in the morning. Link to script: https://github.com/engels74/mover-status/blob/feat-combinedVersion/moverStatus-combined.sh I made it into a separate branch on the GitHub. Current issues: - If USE_DISCORD and USE_TELEGRAM are both set to true, it'll only post the first webhook to Telegram, and not any more. Meanwhile the Discord webhook(s) will keep being posted?? I added some debugging in the latest version on the GitHub repo. Hopefully that'll show something. Other notes: (these are mainly for myself to remember) - Right now, the script should not be setup with cron. The user scripts plugin will re-launch it at the cron interval, overwrite the log with the "script is already running" error message. At least that's what I noticed. But that was also with the "clearLog" variable set to "true", so that might've something to do with it.

Yeah the script definitely needs some more work / fixing 😁. It also looks malformed in Discord. I thought I had made it to post a beginning message, at 0%, like before, but that also doesn't work properly. I might have some time later today or tomorrow, to look at it again.

Thanks for the feedback and everything :). I haven't tested this yet, but if you want, you can try the new version out here: https://logs.notifiarr.com/?9eaa332e3616a02f#FYLHcQpht3cQ9QnLZC7cjnvTtfs6gGhiXbqK3qcQmZ3 Again, word of caution, there's a chance it'll not work at all . I literally haven't even tried running it yet. But "on paper", it could work out of the box. There's a few changes, mainly: 1) Discord and Telegram versions are now combined Combined the Telegram and Discord scripts, hopefully will be easier to maintain one script instead of several different ones. Both variables are false by default, you should enable at least one of them. Note: I haven't tested what happens if you enable both of them, if it actually posts to both webhooks. But again, "on paper" it could work. 2) Percentage Progress only No more "notification per second", it's percentages only. I think you've made a great point. The user can always set the percentage increment lower, if they want more frequent updates posted to their webhook. So I think that should satisfy most users (including myself haha) 3) Loop instead of exit The script will now (hopefully) loop, instead of exiting. It'll continuously monitor when the Unraid Mover runs (with a "sleep 1"), and begin anew when it detects it has started/launched. 4) Output to console Made a few outputs, so the log the User Script plugin creates actually will have something semi-useful information. Like when the script starts, when it detects the mover, when it has completed, and when webhooks have been posted successfully to either Discord or Telegram. Here's hoping it'll work. I might do some testing myself later, but until then, you can have fun trying it out . Made quite a few large changes, so there's a chance I broke something along the way. Edit: To your question regarding making this work with Unraids notification system, instead of its own script entirely.. That might be feasible, though I have absolutely no experience with Unraids plugin system. I'm better / more comfortable just using straight bash.. Not good at much else in terms of programming.

I initially made this script for myself, so that's why it might be a bit weird Currently it's just configured to post every X second, set by the "NOTIFICATION_FREQUENCY" variable, and not in percentages. This was my personal preference, as I usually move huge amounts of data (1-2TB+) per mover session, and that it can take a fair while from 0% -> 25% -> 50% and so on, so it made more sense for me, to make it post every X second instead, as the user following along, could have a better sense of things actually "moving". I'm guessing I could make it an option to do either the notification pr second or by percentages. But the bash script will become increasingly complex, with all these options.. Hopefully I can figure something out. The next pain point of yours is that the script exits when it's done. This was also by design, as I only run my mover once every day, at 03:00AM, and it was easy to setup a cron-job to make the script launch a few minutes before, and exit when it's done. I think I should be able to just loop the script continuously. I would probably recommend still keeping a cronjob for the script, since it'll not relaunch by itself, if you have to reboot the Unraid server or whatever. I'll try and work a bit on the script. I have the following goals: - Combining the Discord and Telegram scripts, and make it togglable if you want to use both Telegram and Discord webhooks or only one of them, e.g. "USE_TELEGRAM=true/false", "USE_DISCORD=true/false" - Making the script use a loop so it starts anew, instead of exiting, when it has detected either the calculations have reached 100% or Unraid's Mover script has exited - Having the option to do either a notification per second or by percentages in increments of 25: 0%/25%/50%/75%/100% - or definable by the user (the increments) - Or.. Either that, or ditch the notification per second option completely, and make it in percentages instead only. Might be the easier approach, in terms of not having the bash script growing increasingly complex. Not sure how much of it I can get working, but I'll give it a shot. Thanks for the feedback

Awesome ! Does the "Estimated completion time" work? Does it actually update the timestamp for each post? I'll probably do a proper release tomorrow on GitHub.