jeffreywhunter Posted May 15, 2015 Share Posted May 15, 2015 Trying to diagnose some hardware issues - starting with NO internet (aka no webgui). So I need a couple tools/techniques. 1. Is there a command that can display the progress of a parity check? 2. Is there a script that copies the syslog to the boot folder whenever it changes? Found this command /root/mdcmd status LOTS of information. Which info out of this command would be appropriate to understand the progress of a parity check? mdResync? mdState? sbSync? something else? If I do this: /root/mdcmd status | egrep "mdResync|mdState|sbSync" I get this: sbSynced=0 sbSyncErrs=0 mdState=STARTED mdResync=1953514552 mdResyncCorr=1 mdResyncPos=631452132 mdResyncDt=31 mdResyncDb=2185144 Is there anything that would give a percent complete? mdResync/mdResyncPos? Any way I could put that in division in a script and format it? Or is there a better, already done script I could download? Quote Link to comment
trurl Posted May 15, 2015 Share Posted May 15, 2015 My reply is OT but I just wanted to tell you about the link that appears in every post that says "Report to moderator". I used that to report your other thread about this that you wanted the mods to delete because you had posted in the wrong subforum. You can use that link in the future to report your own post. Quote Link to comment
Squid Posted May 15, 2015 Share Posted May 15, 2015 Trying to diagnose some hardware issues - starting with NO internet (aka no webgui). So I need a couple tools/techniques. 1. Is there a command that can display the progress of a parity check? 2. Is there a script that copies the syslog to the boot folder whenever it changes? Found this command /root/mdcmd status LOTS of information. Which info out of this command would be appropriate to understand the progress of a parity check? mdResync? mdState? sbSync? something else? If I do this: /root/mdcmd status | egrep "mdResync|mdState|sbSync" I get this: sbSynced=0 sbSyncErrs=0 mdState=STARTED mdResync=1953514552 mdResyncCorr=1 mdResyncPos=631452132 mdResyncDt=31 mdResyncDb=2185144 Is there anything that would give a percent complete? mdResync/mdResyncPos? Any way I could put that in division in a script and format it? Or is there a better, already done script I could download? Sticking with bash, you could do: A=$(/root/mdcmd status | egrep mdResyncPos= | sed -r 's/mdResyncPos=(\w+)/\1/') B=$(/root/mdcmd status | egrep mdResync= | sed -r 's/mdResync=(\w+)/\1/') RESULT=$(awk "BEGIN {printf \"%.2f\",${A}/${B}*100}") echo $RESULT Its not pretty but it works (bash doesn't support floating point, so the division winds up being done in awk) Quote Link to comment
Squid Posted May 15, 2015 Share Posted May 15, 2015 For your syslog you could do tail -f /var/log/syslog >/boot/syslogtail.txt Quote Link to comment
jeffreywhunter Posted May 15, 2015 Author Share Posted May 15, 2015 Perfect thanks! Now to learn about bash scripts... @Trurl - thanks also, I use that in the future... Quote Link to comment
jgc Posted May 11, 2016 Share Posted May 11, 2016 I was running a parity sync and had an issue with one of my off-line drives which caused the web gui to stop responding. I found this post helpful in confirming that the parity check was still running. Thank you all. I anticipated not being able to access the gui for the duration of the sync, so I took the details from this thread and produced a bash script that replicates the information provided in the web gui: #!/bin/bash status=$(mdcmd status | sed -n 's/mdState=//p') size=$(mdcmd status | sed -n 's/mdResync=//p') pos=$(mdcmd status | sed -n 's/mdResyncPos=//p') dt=$(mdcmd status | sed -n 's/mdResyncDt=//p') db=$(mdcmd status | sed -n 's/mdResyncDb=//p') gbsize=$(awk "BEGIN{ printf \"%.0f\",$size * 1024 / 1000^3}") gbpos=$(awk "BEGIN{ printf \"%.0f\",$pos * 1024 / 1000^3}") progress=$(awk "BEGIN{ printf \"%.1f\",($pos / $size) * 100}") speed=$(awk "BEGIN{ printf \"%.1f\",($db/$dt) * 1024 / 1000^2}") finish=$(awk "BEGIN{ m=(($dt*(($size-$pos)/($db/100+1)))/100)/60 print int(m/60) \"h \" int(m%60) \"m\" }") echo "Status: $status" echo "Progress: $gbpos GB of $gbsize GB (${progress}%)" echo "Speed: $speed MB/sec" echo "Finish: $finish" You can create this from the console using nano or vi. I saved as /boot/syncstatus, and made executable with `chmod +x /boot/syncstatus` If you run this once it will output the current status: Status: STARTED Progress: 1101 GB of 4001 GB (27.5%) Speed: 138.3 MB/sec Finish: 5h 49m If you want the information to continually update then you can use the `watch` command: watch /boot/syncstatus This will print the sync information and refresh the information every 2 seconds. You can change the polling period using the -n option (`watch -n 10 /boot/syncstatus` for every 10 seconds). 4 2 Quote Link to comment
tallguydirk Posted July 14, 2019 Share Posted July 14, 2019 On 5/10/2016 at 11:26 PM, jgc said: I was running a parity sync and had an issue with one of my off-line drives which caused the web gui to stop responding. I found this post helpful in confirming that the parity check was still running. Thank you all. I anticipated not being able to access the gui for the duration of the sync, so I took the details from this thread and produced a bash script that replicates the information provided in the web gui: #!/bin/bash status=$(mdcmd status | sed -n 's/mdState=//p') size=$(mdcmd status | sed -n 's/mdResync=//p') pos=$(mdcmd status | sed -n 's/mdResyncPos=//p') dt=$(mdcmd status | sed -n 's/mdResyncDt=//p') db=$(mdcmd status | sed -n 's/mdResyncDb=//p') gbsize=$(awk "BEGIN{ printf \"%.0f\",$size * 1024 / 1000^3}") gbpos=$(awk "BEGIN{ printf \"%.0f\",$pos * 1024 / 1000^3}") progress=$(awk "BEGIN{ printf \"%.1f\",($pos / $size) * 100}") speed=$(awk "BEGIN{ printf \"%.1f\",($db/$dt) * 1024 / 1000^2}") finish=$(awk "BEGIN{ m=(($dt*(($size-$pos)/($db/100+1)))/100)/60 print int(m/60) \"h \" int(m%60) \"m\" }") echo "Status: $status" echo "Progress: $gbpos GB of $gbsize GB (${progress}%)" echo "Speed: $speed MB/sec" echo "Finish: $finish" You can create this from the console using nano or vi. I saved as /boot/syncstatus, and made executable with `chmod +x /boot/syncstatus` If you run this once it will output the current status: Status: STARTED Progress: 1101 GB of 4001 GB (27.5%) Speed: 138.3 MB/sec Finish: 5h 49m If you want the information to continually update then you can use the `watch` command: watch /boot/syncstatus This will print the sync information and refresh the information every 2 seconds. You can change the polling period using the -n option (`watch -n 10 /boot/syncstatus` for every 10 seconds). Thank you very much for this! I'm in similar situation as OP and no programming experience but with your post I was able to run your script and see parity sync status from the console. Quote Link to comment
Furby8704 Posted August 1, 2020 Share Posted August 1, 2020 not sure what happened overnight but my boot drive dropped and webgui is unresponsive. this bash script has me looking at my parity check status. will reboot server once it finishes up. thanks for the write up! Quote Link to comment
blender50 Posted September 15, 2023 Share Posted September 15, 2023 I took the above script as a starting point, added a few things that made its use more useful to me. Disclaimer: I'm no BASH god, and I'm sure it can be improved upon, but for me it does the trick. There are some use remarks in the comments at the top of the script. #!/bin/bash # It takes a number of seconds as an argument. That controls how often the script updates. # # To run the script save it to /tmp and set it to execute (chmod +x scriptname) # then do # "/tmp/syncscript xx" where xx is the refresh time in seconds (without the quotes ;-) # # You can leave this script running while pausing and resuming the parity check from the GUI Main page without issue. :-) # # To stop the script Press [CTRL+C] # # This script will display like this: # # RUNNING # Press [CTRL+C] to stop. # Array Status: STARTED # Parity Check: Running # Speed: 119.9 MB/sec # Progress: 1 GB of 12000 GB # # (0.0%) # Completing in: 27h 48m # On approximately: Sat Sep 16 18:38:31 PDT 2023 # Total Errors: 0 # Refreshing in 0 # # PAUSED # Press [CTRL+C] to stop. # Array Status: STARTED # Parity Check: Paused # Speed: 0 MB/sec # Progress: 13 GB of 0 GB # # (0%) # Completing in: N/A # On approximately: N/A # Total Errors: 0 # Refreshing in 9 # # STOPPED # Press [CTRL+C] to stop. # Array Status: STARTED # Parity Check: Not Running # Speed: 0 MB/sec # Progress: 0 GB of 0 GB # # (0%) # Completing in: N/A # On approximately: N/A # Total Errors: 0 # Refreshing in 6 # Check to see the argument is provided. Otherwise exit with helpful message. if [ -z "$1" ] then echo " To run this script please supply a refresh time in x seconds, like: # /tmp/syncstatus x" exit 0 fi # Define Colored text for use in countdown RED='\033[0;31m' NC='\033[0m' # No Color # Main loop of script while true do # Set refresh to the arguments value plus 1 (necessary for the countdown) refresh=$(($1+1)) # clear the screen and display the script at the top of the window (one row down for readability) tput clear tput cup 1 0 # grab various datapoints from the output of mdcmd status command status=$(mdcmd status | sed -n 's/mdState=//p' ) size=$(mdcmd status | sed -n 's/mdResync=//p' ) pos=$(mdcmd status | sed -n 's/mdResyncPos=//p' ) dt=$(mdcmd status | sed -n 's/mdResyncDt=//p' ) db=$(mdcmd status | sed -n 's/mdResyncDb=//p' ) # calculating sizes and data processed gbsize=$(awk "BEGIN{printf \"%.0f\", $size * 1024 / 1000^3}" ) gbpos=$(awk "BEGIN{printf \"%.0f\", $pos * 1024 / 1000^3}" ) # error checking for stopped, paused, ... if [ $size == 0 ] then progress="0" else progress=$(awk "BEGIN{printf \"%.1f\", ($pos / $size) * 100}" ) fi # checking for 0 speed. Indicating stopped or paused if [ $dt == 0 ] then speed="0" else speed=$(awk "BEGIN{printf \"%.1f\", ($db / $dt) * 1024 / 1000^2}" ) fi # if there is a speed value greater than 0, then the check is running and we report the amount of time left if [ $speed == 0 ] then finish="N/A" else # since it is running we can calculate and display the projected end date finish=$(awk "BEGIN{ m=(($dt*(($size-$pos)/($db/100+1)))/100)/60 print int(m/60) \"h \" int(m%60) \"m\" }") fi # check to see if Parity check is paused, stopped or running if [[ $gbpos -gt "0" ]] && [[ $gbsize == 0 ]] then parityCheck="Paused" elif [ $size == 0 ] then parityCheck="Not Running" else parityCheck="Running" fi # Begin outputting info echo "Press [CTRL+C] to stop." echo "Array Status: $status" echo "Parity Check: $parityCheck" echo "Speed: $speed MB/sec" echo "Progress: $gbpos GB of $gbsize GB" # round progress to nearest whole number for progress bar display progressRounded=$(printf %.0f $progress) # calc progress bar and display %age completed for i in $(eval echo "{0..$progressRounded}") do echo -n "#" done echo " (${progress}%)" echo "Completing in: $finish" # account for stopped or paused processs if [[ $parityCheck="Running" ]] then # calc end date # Extract hours and minutes hours=$(echo "$finish" | awk '{print $1}' | sed 's/h//') minutes=$(echo "$finish" | awk '{print $2}' | sed 's/m//') # Convert to seconds if [[ -n "$hours" ]] && [[ -n "$minutes" ]] then seconds_left=$((hours*3600 + minutes*60)) endDate=$(date -d "now + $seconds_left seconds") else endDate="N/A" fi echo "On approximately: $endDate" fi # Parse all mdcmd status data and summing all rdevNumErrors values input_string=$(mdcmd status) # Use grep to filter lines starting with rdevNumErrors rdev_errors=$(echo "$input_string" | grep -Eo 'rdevNumErrors\.[0-9]+=[0-9]+' | awk -F= '{sum+=$2} END{print sum}') # Print the total value echo "Total Errors: $rdev_errors" # Countdown display count displays in red from 5 to 0 before refresh while [ $refresh -gt 0 ] do refresh=$((refresh-1)) if [ $refresh -le 5 ] then printf " Refreshing in ${RED}$refresh${NC} " printf "\r" sleep 1 else printf " Refreshing in $refresh " printf "\r" sleep 1 fi done # end of countdown while done # end of 'whole script' while I hope others find it useful. 1 Quote Link to comment
furian Posted October 7, 2023 Share Posted October 7, 2023 (edited) On 9/16/2023 at 12:20 AM, blender50 said: I took the above script as a starting point, added a few things that made its use more useful to me. Disclaimer: I'm no BASH god, and I'm sure it can be improved upon, but for me it does the trick. There are some use remarks in the comments at the top of the script. #!/bin/bash # It takes a number of seconds as an argument. That controls how often the script updates. # # To run the script save it to /tmp and set it to execute (chmod +x scriptname) # then do # "/tmp/syncscript xx" where xx is the refresh time in seconds (without the quotes ;-) # # You can leave this script running while pausing and resuming the parity check from the GUI Main page without issue. :-) # # To stop the script Press [CTRL+C] # # This script will display like this: # # RUNNING # Press [CTRL+C] to stop. # Array Status: STARTED # Parity Check: Running # Speed: 119.9 MB/sec # Progress: 1 GB of 12000 GB # # (0.0%) # Completing in: 27h 48m # On approximately: Sat Sep 16 18:38:31 PDT 2023 # Total Errors: 0 # Refreshing in 0 # # PAUSED # Press [CTRL+C] to stop. # Array Status: STARTED # Parity Check: Paused # Speed: 0 MB/sec # Progress: 13 GB of 0 GB # # (0%) # Completing in: N/A # On approximately: N/A # Total Errors: 0 # Refreshing in 9 # # STOPPED # Press [CTRL+C] to stop. # Array Status: STARTED # Parity Check: Not Running # Speed: 0 MB/sec # Progress: 0 GB of 0 GB # # (0%) # Completing in: N/A # On approximately: N/A # Total Errors: 0 # Refreshing in 6 # Check to see the argument is provided. Otherwise exit with helpful message. if [ -z "$1" ] then echo " To run this script please supply a refresh time in x seconds, like: # /tmp/syncstatus x" exit 0 fi # Define Colored text for use in countdown RED='\033[0;31m' NC='\033[0m' # No Color # Main loop of script while true do # Set refresh to the arguments value plus 1 (necessary for the countdown) refresh=$(($1+1)) # clear the screen and display the script at the top of the window (one row down for readability) tput clear tput cup 1 0 # grab various datapoints from the output of mdcmd status command status=$(mdcmd status | sed -n 's/mdState=//p' ) size=$(mdcmd status | sed -n 's/mdResync=//p' ) pos=$(mdcmd status | sed -n 's/mdResyncPos=//p' ) dt=$(mdcmd status | sed -n 's/mdResyncDt=//p' ) db=$(mdcmd status | sed -n 's/mdResyncDb=//p' ) # calculating sizes and data processed gbsize=$(awk "BEGIN{printf \"%.0f\", $size * 1024 / 1000^3}" ) gbpos=$(awk "BEGIN{printf \"%.0f\", $pos * 1024 / 1000^3}" ) # error checking for stopped, paused, ... if [ $size == 0 ] then progress="0" else progress=$(awk "BEGIN{printf \"%.1f\", ($pos / $size) * 100}" ) fi # checking for 0 speed. Indicating stopped or paused if [ $dt == 0 ] then speed="0" else speed=$(awk "BEGIN{printf \"%.1f\", ($db / $dt) * 1024 / 1000^2}" ) fi # if there is a speed value greater than 0, then the check is running and we report the amount of time left if [ $speed == 0 ] then finish="N/A" else # since it is running we can calculate and display the projected end date finish=$(awk "BEGIN{ m=(($dt*(($size-$pos)/($db/100+1)))/100)/60 print int(m/60) \"h \" int(m%60) \"m\" }") fi # check to see if Parity check is paused, stopped or running if [[ $gbpos -gt "0" ]] && [[ $gbsize == 0 ]] then parityCheck="Paused" elif [ $size == 0 ] then parityCheck="Not Running" else parityCheck="Running" fi # Begin outputting info echo "Press [CTRL+C] to stop." echo "Array Status: $status" echo "Parity Check: $parityCheck" echo "Speed: $speed MB/sec" echo "Progress: $gbpos GB of $gbsize GB" # round progress to nearest whole number for progress bar display progressRounded=$(printf %.0f $progress) # calc progress bar and display %age completed for i in $(eval echo "{0..$progressRounded}") do echo -n "#" done echo " (${progress}%)" echo "Completing in: $finish" # account for stopped or paused processs if [[ $parityCheck="Running" ]] then # calc end date # Extract hours and minutes hours=$(echo "$finish" | awk '{print $1}' | sed 's/h//') minutes=$(echo "$finish" | awk '{print $2}' | sed 's/m//') # Convert to seconds if [[ -n "$hours" ]] && [[ -n "$minutes" ]] then seconds_left=$((hours*3600 + minutes*60)) endDate=$(date -d "now + $seconds_left seconds") else endDate="N/A" fi echo "On approximately: $endDate" fi # Parse all mdcmd status data and summing all rdevNumErrors values input_string=$(mdcmd status) # Use grep to filter lines starting with rdevNumErrors rdev_errors=$(echo "$input_string" | grep -Eo 'rdevNumErrors\.[0-9]+=[0-9]+' | awk -F= '{sum+=$2} END{print sum}') # Print the total value echo "Total Errors: $rdev_errors" # Countdown display count displays in red from 5 to 0 before refresh while [ $refresh -gt 0 ] do refresh=$((refresh-1)) if [ $refresh -le 5 ] then printf " Refreshing in ${RED}$refresh${NC} " printf "\r" sleep 1 else printf " Refreshing in $refresh " printf "\r" sleep 1 fi done # end of countdown while done # end of 'whole script' while I hope others find it useful. thank you! My GUI stopped responding after starting the array when i replaced the parity disk (14tb) for a new 18tb disk i had no idea what the server was doing.. i selected the 14TB disk as a replacement for a old 4tb disk, i can't format it right now as the gui stopped responding and i had no idea how long it would take.. Now i see this, i think i will have to wait for it to complete... but at least now i know what is happening! Edited October 7, 2023 by furian 1 Quote Link to comment
gbrl Posted November 23, 2023 Share Posted November 23, 2023 On 10/7/2023 at 11:30 AM, furian said: thank you! My GUI stopped responding after starting the array when i replaced the parity disk (14tb) for a new 18tb disk i had no idea what the server was doing.. i selected the 14TB disk as a replacement for a old 4tb disk, i can't format it right now as the gui stopped responding and i had no idea how long it would take.. Now i see this, i think i will have to wait for it to complete... but at least now i know what is happening! Same exact thing is happening to me right now. GUI stopped responding/refusing to connect when replacing 14TB parity disk for a new 18TB disk. What ended up happening with yours? Did the GUI eventually come back after completion? Quote Link to comment
shpitz461 Posted Sunday at 06:45 PM Share Posted Sunday at 06:45 PM Is there a way to pause parity check from commandline? My GUI is unreponsive, but I was able to get a command window to open. I can't access any GUI pages, so I cannot install the parity tuning plugin. I've ran the script @blender50 posted, so I know parity is running: Quote Array Status: STARTED Parity Check: Running Speed: 225.5 MB/sec Progress: 3018 GB of 18000 GB ################## (16.8%) Completing in: 18h 27m On approximately: Mon Apr 15 09:09:21 EDT 2024 Total Errors: 0 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.