Pikvm monitoring and powercycling script

I have a Pikvm attached to my Unraid server, I wanted to see if I could make pikvm powercycle the server if it becomes unresponsive.
I asked ChatGPT to help me make a script, I've tested it a little bit, and it seems to get the job done.
I wanted to share it if anyone else is interested.
It is running directly on my PiKVM, but in theory it could run on a different raspberry pi or similar.

It uses ping, so it will work for anything, not just Unraid.

I have very little knowledge of shell scripts, so I am not really able to review the script properly, but as I stated previously, it seems to get the job done.

Here is the post from ChatGPT:

Disclaimer:
This script and service were generated and refined with help from ChatGPT. Always review and test scripts before using them in production to ensure they suit your specific environment.

🖥 PiKVM Server Monitor – Auto Power Cycle via systemd

This solution allows your server to be automatically power-cycled via PiKVM if it becomes unresponsive to ping. It's ideal for headless or remote setups where physical access isn't always possible.

✅ Features

• Monitors server responsiveness via ping
• Tries soft power-off via PiKVM, with fallback to hard-off if needed
• Confirms power states before continuing
• Includes grace period logic to avoid false power cycles (e.g. during reboots)
• Waits for power state transitions instead of using fixed sleep times
• All settings configurable through systemd Environment variables

📂 Systemd Service File
Save this in: /etc/systemd/system/pikvm-monitor.service

[Unit]
Description=PiKVM Server Monitor Script
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/local/bin/manage_server.sh
Restart=always
RestartSec=10
User=root

Environment=PIKVM_USER=your_kvm_username
Environment=PIKVM_PASS=your_kvm_password
Environment=PIKVM_URL=https://<your-kvm-ip>/api
Environment=TARGET_IP=<your-server-ip>

Environment=MAX_RETRIES=5
Environment=RETRY_DELAY=10
Environment=TIMEOUT=5

Environment=WAIT_AFTER_ON_TIME=90
Environment=WAIT_AFTER_OFF_BEFORE_ON=30
Environment=SOFT_OFF_TIMEOUT=60
Environment=GRACE_PERIOD=180

[Install]
WantedBy=multi-user.target

🔧 Shell Script
Save this as: /usr/local/bin/manage_server.sh
Make it executable:

 chmod +x /usr/local/bin/manage_server.sh 

#!/bin/bash

: "${PIKVM_URL:?Missing PIKVM_URL}"
: "${PIKVM_USER:?Missing PIKVM_USER}"
: "${PIKVM_PASS:?Missing PIKVM_PASS}"
: "${TARGET_IP:?Missing TARGET_IP}"
: "${MAX_RETRIES:?Missing MAX_RETRIES}"
: "${RETRY_DELAY:?Missing RETRY_DELAY}"
: "${TIMEOUT:?Missing TIMEOUT}"
: "${WAIT_AFTER_OFF_BEFORE_ON:?Missing WAIT_AFTER_OFF_BEFORE_ON}"
: "${WAIT_AFTER_ON_TIME:?Missing WAIT_AFTER_ON_TIME}"
: "${SOFT_OFF_TIMEOUT:?Missing SOFT_OFF_TIMEOUT}"
: "${GRACE_PERIOD:?Missing GRACE_PERIOD}"

POWER_CYCLES=0
LAST_SEEN_ALIVE=$(date +%s)

check_server_alive() {
    retries=0
    while [ $retries -lt $MAX_RETRIES ]; do
        echo "Pinging $TARGET_IP (attempt $((retries + 1)) of $MAX_RETRIES)..."
        ping -c 1 -W $TIMEOUT "$TARGET_IP" > /dev/null 2>&1
        if [ $? -eq 0 ]; then
            echo "Server at $TARGET_IP is alive!"
            LAST_SEEN_ALIVE=$(date +%s)
            return 0
        fi
        retries=$((retries + 1))
        echo "No response. Retrying in $RETRY_DELAY seconds..."
        sleep $RETRY_DELAY
    done
    echo "Server is unreachable after $MAX_RETRIES attempts."
    return 1
}

check_power_state() {
    RESPONSE=$(curl -s -k -u "$PIKVM_USER:$PIKVM_PASS" "$PIKVM_URL/atx")
    if [ -z "$RESPONSE" ]; then echo "Error: No response from PiKVM"; return 2; fi
    if [[ "$RESPONSE" == *"Unauthorized"* ]]; then echo "Unauthorized - check credentials"; exit 1; fi
    POWER_STATE=$(echo "$RESPONSE" | grep -o '"power": [^,]*' | awk -F': ' '{print $2}' | tr -d '"')
    [[ "$POWER_STATE" == "true" ]]
}

wait_until_power_state_on() {
    echo "Waiting until power state is ON..."
    until check_power_state; do
        echo "Still OFF... waiting..."
        sleep 1
    done
    echo "Power state confirmed ON."
}

perform_power_cycle() {
    echo "Sending soft power off..."
    curl -X POST -k -u "$PIKVM_USER:$PIKVM_PASS" "$PIKVM_URL/atx/power?action=off"

    echo "Waiting for power off (timeout: $SOFT_OFF_TIMEOUT sec)..."
    START=$(date +%s)
    while check_power_state; do
        NOW=$(date +%s)
        if [ $((NOW - START)) -ge "$SOFT_OFF_TIMEOUT" ]; then
            echo "Soft off timed out. Sending hard power off..."
            break
        fi
        echo "Still ON... waiting..."
        sleep 1
    done

    if check_power_state; then
        echo "Sending HARD power off..."
        curl -X POST -k -u "$PIKVM_USER:$PIKVM_PASS" "$PIKVM_URL/atx/power?action=off_hard"
        until ! check_power_state; do
            echo "Waiting for hard off to complete..."
            sleep 1
        done
    fi

    echo "Waiting $WAIT_AFTER_OFF_BEFORE_ON seconds before power on..."
    sleep "$WAIT_AFTER_OFF_BEFORE_ON"
    echo "Powering on..."
    curl -X POST -k -u "$PIKVM_USER:$PIKVM_PASS" "$PIKVM_URL/atx/power?action=on"
    wait_until_power_state_on
    echo "Waiting $WAIT_AFTER_ON_TIME seconds for server to boot..."
    sleep "$WAIT_AFTER_ON_TIME"
}

echo "Startup: Waiting for ping (max wait: $((GRACE_PERIOD * 2)) sec)..."
START=$(date +%s)
while true; do
    if check_server_alive; then
        echo "Server is alive. Starting monitor."
        break
    fi
    NOW=$(date +%s)
    ELAPSED=$((NOW - START))
    if [ "$ELAPSED" -ge $((GRACE_PERIOD * 2)) ]; then
        echo "Startup timeout ($ELAPSED sec). Exiting."
        exit 1
    fi
    echo "Still no response ($ELAPSED sec)..."
    sleep "$RETRY_DELAY"
done

while true; do
    echo "Checking server status..."
    if check_power_state; then
        if check_server_alive; then
            echo "Server responsive."
        else
            NOW=$(date +%s)
            SINCE_ALIVE=$((NOW - LAST_SEEN_ALIVE))
            if [ "$SINCE_ALIVE" -lt "$GRACE_PERIOD" ]; then
                echo "Unresponsive, but within grace period ($SINCE_ALIVE/$GRACE_PERIOD sec)."
            else
                echo "Server unresponsive. Starting power cycle..."
                perform_power_cycle
                if check_server_alive; then
                    echo "Server recovered after power cycle."
                    POWER_CYCLES=0
                else
                    POWER_CYCLES=$((POWER_CYCLES + 1))
                    echo "Still unresponsive. Attempt $POWER_CYCLES"
                    if [ "$POWER_CYCLES" -ge 3 ]; then
                        echo "Failed after 3 power cycles. Stopping."
                        break
                    fi
                fi
            fi
        fi
    else
        echo "Server is OFF. Waiting to power on..."
        wait_until_power_state_on
        echo "Waiting $WAIT_AFTER_ON_TIME seconds for boot..."
        sleep "$WAIT_AFTER_ON_TIME"
        check_server_alive && LAST_SEEN_ALIVE=$(date +%s)
    fi
    echo "Sleeping 60 seconds before next check..."
    sleep 60
done

🚀 Enable the Service
After saving both files, reload systemd and enable the service:

sudo systemctl daemon-reload
sudo systemctl enable --now pikvm-monitor.service

🪵 Monitor the Service Log
To follow the log output live:

 journalctl -u pikvm-monitor.service -f 

Or to check logs after reboot:

 journalctl -u pikvm-monitor.service -b