[Script] Mover Status (posts Unraid Mover status to Discord and Telegram webhooks)

[couzin2000]

So far I've been able to insert the new script with my credentials. I left it at 25% increments. 
Would be a teensy tiny bit interesting to have a "Mover starting (0% completed)" message... just so we know it's actually started.

 

I'll let you know once I get to 25% completed.

[couzin2000]

No dice. But you're right, we're getting more info now from the log. Here's what happened:
 

Sending notification...
jq: error: $TELEGRAM_CHAT_ID is not defined at , line 1:
{chat_id: $TELEGRAM_CHAT_ID, text: $text, disable_notification: "false", parse_mode: "HTML"}
jq: 1 compile error
Notification sent to Telegram.

 

I'm trying to read the corresponding lines, I can see the lines but not finding the error. My Chat ID was copied and pasted from the last script, so it's the right one.

Any ideas?

[engels74]

Yeah the script definitely needs some more work / fixing 😁. It also looks malformed in Discord.

I thought I had made it to post a beginning message, at 0%, like before, but that also doesn't work properly.

I might have some time later today or tomorrow, to look at it again.

[engels74]

On 5/11/2024 at 4:29 AM, couzin2000 said:

No dice. But you're right, we're getting more info now from the log. Here's what happened:
 

Sending notification...
jq: error: $TELEGRAM_CHAT_ID is not defined at , line 1:
{chat_id: $TELEGRAM_CHAT_ID, text: $text, disable_notification: "false", parse_mode: "HTML"}
jq: 1 compile error
Notification sent to Telegram.

 

I'm trying to read the corresponding lines, I can see the lines but not finding the error. My Chat ID was copied and pasted from the last script, so it's the right one.

Any ideas?

 

Okay, so I did a lot of work on this one :). Hopefully it'll work now.

 

Changes:

- I added a DRY_RUN option, it's set to "false" by default. If you set it to true, it'll try and post to the webhooks, to see if that part works, and exit the script. It'll also do the calculations for the /mnt/cache, but everything else it just bogus data

- I added a cooldown for the echo'ing to the logs for 10 seconds, since it would spam up the log, when it didn't find the Mover process. So now it'll look for the Mover process each 60 seconds, instead of each mili-second. I think that's.. Fine. There's probably a better way to do this, but for now, it works fine without spamming up the log too much. You should probably clear the log file at some point, as it will probably fill up the log over time. Again... I'll try and figure something out.

- I think I fixed the previous issues, but I haven't had a "real" try with the new version yet. So hopefully it'll now start at 0%, like it's supposed to.

- Probably fixed some other stuff, but I really hope it'll work now. I'm leaving mine overnight and see what happens in the morning.

 

Link to script:

https://github.com/engels74/mover-status/blob/feat-combinedVersion/moverStatus-combined.sh

 

I made it into a separate branch on the GitHub.

 

Current issues:

- If USE_DISCORD and USE_TELEGRAM are both set to true, it'll only post the first webhook to Telegram, and not any more. Meanwhile the Discord webhook(s) will keep being posted?? I added some debugging in the latest version on the GitHub repo. Hopefully that'll show something.

 

Other notes: (these are mainly for myself to remember)

- Right now, the script should not be setup with cron. The user scripts plugin will re-launch it at the cron interval, overwrite the log with the "script is already running" error message. At least that's what I noticed. But that was also with the "clearLog" variable set to "true", so that might've something to do with it.

Edited May 12 by engels74
GitHub repo link, alpha branch

[couzin2000]

Great work amigo. For the moment, I'm getting response from DRY_RUN and from running the standard script while Mover is running. So far so good. I left INCREMENT at 25 to see how well that part works, and I did get a notification at 0% which is great. You could package and sell this with Unraid, it's just that excellent.

Keep you posted on the whole move!

[couzin2000]

Completed without a hitch. Great work!
If you can get this script constantly running, it'll be incredible.

 

EDIT: Whoa snap! It DOES work on a loop!
Congrats! I'm sold!

Edited May 13 by couzin2000
new result

[couzin2000]

Ok, one bug; once you've brought the remaining percentage down to zero, the loop doesn't seem to reset. I do get a 0% notification and a Completed notification, but nothing in betwen. For that to happen, I can stop the script and run it again. Then the increments will be notified. 

[engels74]

Yeah, that's what I'm seeing as well. I'll add some more debugging, and then hopefully get some kind of useful "response" from Telegrams API. I'm guessing something's being sent malformed.

 

Especially having "USE_DISCORD" and "USE_TELEGRAM" set to true at the same time, seems to cause some weird behaviour. On my Discord server I got an odd amount of updates, nothing really consistent, and on my Telegram channel I only received the initial message, and the last completion one.

 

I'll push an update to the script soon, with some more debugging, and then I can hopefully do something with whatever response I get from Telegram and/or Discord.

 

Edit:

v0.0.5-alpha10: So I added some debugging logging all over the script, hopefully that'll show where / how it fails.

I also tried adjusting the "LAST_NOTIFICATION" logic a bit, not sure if it'll be fruitful or a failure. I've had a rough night with little sleep, but here's to hoping

v0.0.5-alpha11: Added even more debugging / logging, since I can't really reproduce why or when it's happening . I'll let it run overnight again, and take a look at the log tomorrow. Don't have too much time this week to work on it sadly.

v0.0.5-alpha14: I think we're getting there. Something was malformed being sent to both Discord and Telegram, if both were enabled. I tried to mitigate this. Also made the logging a bit smarter.. I hope. The percent increments should be more accurate now as well (again, hopefully).

v0.0.5-alpha15: The alpha14 didn't solve the issue completely, tried another approach. Also introduced a new variable, that should help the percentage calculation being more accurate. Man, I'm hoping it'll work now lol. 

Edited May 17 by engels74
v0.0.5-alpha15

[engels74]

On 5/13/2024 at 7:22 PM, couzin2000 said:

Ok, one bug; once you've brought the remaining percentage down to zero, the loop doesn't seem to reset. I do get a 0% notification and a Completed notification, but nothing in betwen. For that to happen, I can stop the script and run it again. Then the increments will be notified. 

 

Hey mate, I think the current version (v0.0.5-alpha16) is in a pretty good state:

https://github.com/engels74/mover-status/blob/feat-combinedVersion/moverStatus-combined.sh

 

I did a bunch of work on it, but it should work as intended now.

 

Can I get you to test it out, see how it works out in your end? It still has some debugging stuff enabled, I'll probably remove that for the proper "release".

 

I still haven't tested it for multiple consecutive runs (letting it run multiple loops), but hopefully that works as well.

 

Edit:

The loop seems to work. I've yet to test:

- If the script works, if only "USE_DISCORD" or "USE_TELEGRAM" is enabled

- What will happen with the log in Unraid's User Scripts plugin, if you try and run the script, while it's already running. A few versions prior, it removed the entire previous log.

Edited May 22 by engels74

[couzin2000]

Hi there,
Sorry for the delays, I've been rather busy. 
I've installed the script in its final state on my Unraid 6.12.10 about 3-4 days ago. So far, the script works exactly as intended on multiple consecutive file transfers! Outstanding work my friend, thank you! You should definitely talk about this script and make it go wide - one of the better more useful features I'd been asking for on Unraid. 
Thanks again!

[engels74]

I finally got to a proper release of the v0.0.5 of this script. A lot of work went into it.

 

If you are upgrading to v0.0.5 from v0.0.4, I recommend re-reading the installation guide on GitHub, as the script no longer supports the "cron" scheduling in the User Scripts plugin.

 

The full changelog can be found here:
https://github.com/engels74/mover-status/releases/tag/0.0.5

[engels74]

6 hours ago, couzin2000 said:

Hi there,
Sorry for the delays, I've been rather busy. 
I've installed the script in its final state on my Unraid 6.12.10 about 3-4 days ago. So far, the script works exactly as intended on multiple consecutive file transfers! Outstanding work my friend, thank you! You should definitely talk about this script and make it go wide - one of the better more useful features I'd been asking for on Unraid. 
Thanks again!

 

Very happy that it works on your end ! Spent so many hours on it, so much trial and error haha. I have a fairly basic understanding of bash, and that's it.

 

I would be glad to spread the word, but I wouldn't know where to start honestly 

Edited Monday at 07:28 PM by engels74