engels74

Yeah, that's what I'm seeing as well. I'll add some more debugging, and then hopefully get some kind of useful "response" from Telegrams API. I'm guessing something's being sent malformed. Especially having "USE_DISCORD" and "USE_TELEGRAM" set to true at the same time, seems to cause some weird behaviour. On my Discord server I got an odd amount of updates, nothing really consistent, and on my Telegram channel I only received the initial message, and the last completion one. I'll push an update to the script soon, with some more debugging, and then I can hopefully do something with whatever response I get from Telegram and/or Discord. Edit: v0.0.5-alpha10: So I added some debugging logging all over the script, hopefully that'll show where / how it fails. I also tried adjusting the "LAST_NOTIFICATION" logic a bit, not sure if it'll be fruitful or a failure. I've had a rough night with little sleep, but here's to hoping v0.0.5-alpha11: Added even more debugging / logging, since I can't really reproduce why or when it's happening . I'll let it run overnight again, and take a look at the log tomorrow. Don't have too much time this week to work on it sadly. v0.0.5-alpha14: I think we're getting there. Something was malformed being sent to both Discord and Telegram, if both were enabled. I tried to mitigate this. Also made the logging a bit smarter.. I hope. The percent increments should be more accurate now as well (again, hopefully).

Okay, so I did a lot of work on this one :). Hopefully it'll work now. Changes: - I added a DRY_RUN option, it's set to "false" by default. If you set it to true, it'll try and post to the webhooks, to see if that part works, and exit the script. It'll also do the calculations for the /mnt/cache, but everything else it just bogus data - I added a cooldown for the echo'ing to the logs for 10 seconds, since it would spam up the log, when it didn't find the Mover process. So now it'll look for the Mover process each 60 seconds, instead of each mili-second. I think that's.. Fine. There's probably a better way to do this, but for now, it works fine without spamming up the log too much. You should probably clear the log file at some point, as it will probably fill up the log over time. Again... I'll try and figure something out. - I think I fixed the previous issues, but I haven't had a "real" try with the new version yet. So hopefully it'll now start at 0%, like it's supposed to. - Probably fixed some other stuff, but I really hope it'll work now. I'm leaving mine overnight and see what happens in the morning. Link to script: https://github.com/engels74/mover-status/blob/feat-combinedVersion/moverStatus-combined.sh I made it into a separate branch on the GitHub. Current issues: - If USE_DISCORD and USE_TELEGRAM are both set to true, it'll only post the first webhook to Telegram, and not any more. Meanwhile the Discord webhook(s) will keep being posted?? I added some debugging in the latest version on the GitHub repo. Hopefully that'll show something. Other notes: (these are mainly for myself to remember) - Right now, the script should not be setup with cron. The user scripts plugin will re-launch it at the cron interval, overwrite the log with the "script is already running" error message. At least that's what I noticed. But that was also with the "clearLog" variable set to "true", so that might've something to do with it.

Yeah the script definitely needs some more work / fixing 😁. It also looks malformed in Discord. I thought I had made it to post a beginning message, at 0%, like before, but that also doesn't work properly. I might have some time later today or tomorrow, to look at it again.

Thanks for the feedback and everything :). I haven't tested this yet, but if you want, you can try the new version out here: https://logs.notifiarr.com/?9eaa332e3616a02f#FYLHcQpht3cQ9QnLZC7cjnvTtfs6gGhiXbqK3qcQmZ3 Again, word of caution, there's a chance it'll not work at all . I literally haven't even tried running it yet. But "on paper", it could work out of the box. There's a few changes, mainly: 1) Discord and Telegram versions are now combined Combined the Telegram and Discord scripts, hopefully will be easier to maintain one script instead of several different ones. Both variables are false by default, you should enable at least one of them. Note: I haven't tested what happens if you enable both of them, if it actually posts to both webhooks. But again, "on paper" it could work. 2) Percentage Progress only No more "notification per second", it's percentages only. I think you've made a great point. The user can always set the percentage increment lower, if they want more frequent updates posted to their webhook. So I think that should satisfy most users (including myself haha) 3) Loop instead of exit The script will now (hopefully) loop, instead of exiting. It'll continuously monitor when the Unraid Mover runs (with a "sleep 1"), and begin anew when it detects it has started/launched. 4) Output to console Made a few outputs, so the log the User Script plugin creates actually will have something semi-useful information. Like when the script starts, when it detects the mover, when it has completed, and when webhooks have been posted successfully to either Discord or Telegram. Here's hoping it'll work. I might do some testing myself later, but until then, you can have fun trying it out . Made quite a few large changes, so there's a chance I broke something along the way. Edit: To your question regarding making this work with Unraids notification system, instead of its own script entirely.. That might be feasible, though I have absolutely no experience with Unraids plugin system. I'm better / more comfortable just using straight bash.. Not good at much else in terms of programming.

I initially made this script for myself, so that's why it might be a bit weird Currently it's just configured to post every X second, set by the "NOTIFICATION_FREQUENCY" variable, and not in percentages. This was my personal preference, as I usually move huge amounts of data (1-2TB+) per mover session, and that it can take a fair while from 0% -> 25% -> 50% and so on, so it made more sense for me, to make it post every X second instead, as the user following along, could have a better sense of things actually "moving". I'm guessing I could make it an option to do either the notification pr second or by percentages. But the bash script will become increasingly complex, with all these options.. Hopefully I can figure something out. The next pain point of yours is that the script exits when it's done. This was also by design, as I only run my mover once every day, at 03:00AM, and it was easy to setup a cron-job to make the script launch a few minutes before, and exit when it's done. I think I should be able to just loop the script continuously. I would probably recommend still keeping a cronjob for the script, since it'll not relaunch by itself, if you have to reboot the Unraid server or whatever. I'll try and work a bit on the script. I have the following goals: - Combining the Discord and Telegram scripts, and make it togglable if you want to use both Telegram and Discord webhooks or only one of them, e.g. "USE_TELEGRAM=true/false", "USE_DISCORD=true/false" - Making the script use a loop so it starts anew, instead of exiting, when it has detected either the calculations have reached 100% or Unraid's Mover script has exited - Having the option to do either a notification per second or by percentages in increments of 25: 0%/25%/50%/75%/100% - or definable by the user (the increments) - Or.. Either that, or ditch the notification per second option completely, and make it in percentages instead only. Might be the easier approach, in terms of not having the bash script growing increasingly complex. Not sure how much of it I can get working, but I'll give it a shot. Thanks for the feedback

Awesome ! Does the "Estimated completion time" work? Does it actually update the timestamp for each post? I'll probably do a proper release tomorrow on GitHub.

Okay, so I end up at least making .. something .. work. I made a bot, using the BotFather on Telegram. I created a simple channel, invited the bot as an administrator, gave it the default rights (post/delete stuff). I copied the bot id, that BotFather gave me I found the channel id, using web.telegram.org: simple jpg of how to or stackoverflow comment I made a "dummy" version of the script, that basically just tests the same method as before - except the markdown didn't work, so I made a simple html version instead: https://logs.notifiarr.com/?74dfa0fae8d7b2af#D6Y8i4VhoC8PTP9tBFvgQzz9tfG3FNNW7DKJHyEyZinu I used the bot id and the channel id in that script, and that seems to work: Bear in mind it's just a dummy script with no real data, but the posting itself seems to work, which is promising . Stil not sure how it'll deal with the formatting of the real thing. Edit: I tried updating the script, with the updated code, that uses the same HTML formatting, as the dummy script: https://logs.notifiarr.com/?df0d69f2afe05113#GXhzEptFEweKj7Q3Jm6ndX4oVA3CMaiK6waqCeR2oFAG But I have not tested it :). I would say it should work, but at this point, I should probably say that maybe it could work

So at least the bot token is working. And you've put both the bot token and the channel id into the script from above, right? And the bot is invited into the Telegram chat-id/channel and/or have access to the Telegram channel/chat-id? (sorry if the terminology is off, I'm not that experienced with Telegram)

You need to insert your bot token into that script yeah, I don't know yours . Also yeah, it might not be your fault that the script is not working btw, I could've easily messed something up.

Definitely shows that there's something not working. It's probably sending the message, but Telegram responds with a 404. Could be a wide range of things.. Like the wrong bot token input to the script, to the payload being delivered via the webhook is still wrongly formatted, or something else entirely. You can try executing this script, to see if the bot token is valid: https://logs.notifiarr.com/?14191f2c8033ada0#7RLNfxFKe1XPs8KQBSCCoapci7UZtBD1oeVSX5x6SBgH It should give you some kind of response, with some info of the Telegram Bot. If it does respond with basically anything other than an error, it's probably the JSON payload in the script that's acting up, or the curl thing still not working properly. I would test all this myself, but I can't really at the moment, sorry. I'll try and get something done in the upcoming weekend :). Thanks for helping out though.

Well, you can try this modified version: https://logs.notifiarr.com/?be6900374c58312c#F2ZesyL12fuvWpwAEczh8yxBkknWjP7joj8H98Ds99hK Not sure if it'll work out, it's a quite quick and dirty version

I'll probably need to do some testing myself, and set up a Telegram Bot/Channel.. Maybe I'll have time during this weekend. Sorry it didn't work out of the box.

You should remove the "/path/to/excluded/folder" manually from the script, if you don't have any folders. Otherwise the script will fail / exit. So it should say EXCLUDE_PATH_01="" without anything between the quotation marks. You should be able to just restart the moverStatus script, and it should detect the Unraid mover process is running, and go from there. There is not much error logging in the script, so let's see what happens 😁 I'll edit the next version, so the paths are empty by default.

I have made something, but I haven't tested it, as I don't really use Telegram. I'll probably test it later myself. I made a few changes, since the Unix timestamp thing that I use for Discord, is not supported in Telegram. So it posts that part a bit differently. In the meantime, you can try using this: https://logs.notifiarr.com/?cb3a4af59222b9e5#g8oTX2JN7Co9aBGjbNJsEZLWfDyAEHx3evnCcPuegVZ A word of caution, it might not work at all. I haven't worked with Telegram webhooks/API before. If it does work for you, please report back in here, and I'll add it properly to the Github repository.

Changelog (YYYY-MM-DD) 2024-05-09 - v0.0.4: - Added a small amount of error handling. The script will check if the Discord Webhook is set properly, and if the webhook responds with a HTTP 200 (OK), else it'll log the error. - Edited the way the variables are set, so it should be easier to figure out - Made the EXCLUSION_PATH_XX variables empty by default 2024-05-08 - v0.0.3: Now you can change the message it posts with variables: - Added MOVING_MESSAGE and COMPLETION_MESSAGE variables 2024-05-07 - v0.0.2: - Fixed the script, in terms of calculating the right amount of data in the "Remaining Data" bit. In the previous version, only the first excluded path actually got included. Now it should work properly. Hopefully . - Added a footer text, which reports the current version of the script. - Added a very simple version check function. It'll compare the current version of the script, to the latest release on the GitHub repo. If there's an update, it'll be included in the footer text, e.g.: "Version: v0.0.1 (update available)". - The version check function is only included from v0.0.2 and beyond. It won't work with the first version of the script, sorry. - Changed the colours a bit. Previously it was "Light Red -> Light Blue -> Light Green". Now it's "Light Red -> Light Orange -> Light Green". GH Repo Link: - https://github.com/engels74/mover-status

Hello! I created this script (moverStatus.sh), that'll post how far along the Mover is, from moving from Cache --> Array, to a Discord webhook. I'm not sure if it'll be useful for anyone else. I've tested it for a few runs, and it seems to work fine on my setup. I don't think I'm going to do much more work on it, so if you want to help improve it, you can help by doing a Pull Request :). Description The Bash script is designed to monitor the progress of the UnraidOS Mover utility, which transfers data from SSD cache to HDD storage in an UnraidOS system. The script checks the status of the mover process and calculates the amount of data left to move. It then formats this information into a readable format, including the percentage of the task completed and an estimated time of completion. Notifications about the mover's progress are periodically sent to a Discord channel via a webhook. These notifications include detailed progress updates and an estimated completion time, helping users track the mover process effectively. The script supports customization for notification frequency and allows users to specify directory paths to exclude from the data transfer calculation. The purpose of the script is for users who want real-time updates about the Mover, can easily follow the progress on Discord. Image preview: Installation + instructions: The GitHub link is here: https://github.com/engels74/mover-status Kind regards, Engels74

Just wanted to chime in, with some very simple bash scripts. I'm using these, since I'm running SABnzbd behind a reverse proxy, but I'm sure they work fine using local addresses as well. pauseSAB.sh (before mover starts) #!/bin/bash # SABnzbd API key API_KEY="insert-api-key-here" # SABnzbd URL URL="https://sabnzbd.domain.com/api" # Pause command to SABnzbd curl -s "$URL?apikey=$API_KEY&mode=pause" > /dev/null resumeSAB.sh (after mover ends) #!/bin/bash # SABnzbd API key API_KEY="insert-api-key-here" # SABnzbd URL URL="https://sabnzbd.domain.com/api" # Pause command to SABnzbd curl -s "$URL?apikey=$API_KEY&mode=resume" > /dev/null The scripts work fine and pauses SABnzbd instantly. I haven't tested them extensively with the Mover running yet, though. But they do as advertised - pauses and resumes SABnzbd's downloads :)