Fireball3 Posted March 7, 2014 Author Share Posted March 7, 2014 OK, so how would our code look like? You can have the check_hour function set a global variable and ALSO return that via the return command. Then you can have the best of both worlds. A variable to check the last status of the call and a return code when needed. I will always have to check if hours match. So that call will always be necessary. Quote Link to comment
WeeboTech Posted March 7, 2014 Share Posted March 7, 2014 Here's some example code. #!/bin/bash noCountdownHours="18 19 20 21 22 23" check_hour_list() { HOUR=$(date +%H) HOUR_MATCHED_RC=1 for CountdownHour in ${noCountdownHours} do if [ ${HOUR} = ${CountdownHour} ] ; then HOUR_MATCHED_RC=0 return ${HOUR_MATCHED_RC} fi done return ${HOUR_MATCHED_RC} } hour=$(date +%H) hourMatch=$(echo "$noCountdownHours" | grep "$hour" | wc -l) echo "noCountdownHours: ${noCountdownHours}, current hour: $hour, match? $hourMatch" if check_hour_list then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ $? -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ ${HOUR_MATCHED_RC} -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi Test output root@unRAID:/tmp# ./x.sh noCountdownHours: 18 19 20 21 22 23, current hour: 16, match? 0 check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH root@unRAID:/tmp# ./x.sh noCountdownHours: 16 18 19 20 21 22 23, current hour: 16, match? 1 check_hour_list current hour matched check_hour_list current hour matched check_hour_list current hour matched Quote Link to comment
Fireball3 Posted March 7, 2014 Author Share Posted March 7, 2014 OK, now we have a variable set and don't need to call the function. But how do we notice if the hour has passed and we need to check again? Edit: wait - I did not scroll down in the code window... Edit2: no, question still valid By the way, I rewrote the logger passage and now it works. The "-" was odd. How can I log multiple lines: log_message " ----------- $program_name version $version --------------------- Configuration settings. Power down instead of sleep: $powerDownInsteadOfSleep Check HDD activity: $checkHDDs Check TCP activity: $checkTCP Check SSH connections: $checkSSH Check TELNET connections: $checkTELNET Check for online clients: $pingIPs Check local logins: $checkSHELL Check for lock file (tmp/nos3sleep.lock): $checkLockFile No sleep/powerdown during hours: $noCountdownHours Save SMART values before powerdown: $save_SMART --- After wake up from sleep: ---- Renew DHCP: $doDhcpRenewal Force GB LAN: $forceGb -----------------------------------------------------------------" I don't have line feed when doing it like this. Quote Link to comment
WeeboTech Posted March 7, 2014 Share Posted March 7, 2014 I rewrote the logger passage and now it works. The "-" was odd. How can I log multiple lines: while read MESSAGE do log_message "${MESSAGE}" done <-EOF ----------- $program_name version $version --------------------- Configuration settings. Power down instead of sleep: $powerDownInsteadOfSleep Check HDD activity: $checkHDDs Check TCP activity: $checkTCP Check SSH connections: $checkSSH Check TELNET connections: $checkTELNET Check for online clients: $pingIPs Check local logins: $checkSHELL Check for lock file (tmp/nos3sleep.lock): $checkLockFile No sleep/powerdown during hours: $noCountdownHours Save SMART values before powerdown: $save_SMART --- After wake up from sleep: ---- Renew DHCP: $doDhcpRenewal Force GB LAN: $forceGb ----------------------------------------------------------------- EOF It's not efficient doing it like that. You could logger directly with logger -t$program <-EOF EOF Quote Link to comment
Fireball3 Posted March 7, 2014 Author Share Posted March 7, 2014 It's not efficient doing it like that. Yes, I thought that too, I have it like | logger -t... What's this <-EOF for? (end of file)? Quote Link to comment
WeeboTech Posted March 7, 2014 Share Posted March 7, 2014 It's not efficient doing it like that. Yes, I thought that too, I have it like | logger -t... What's this <-EOF for? (end of file)? It's a HERE document. http://tldp.org/LDP/abs/html/here-docs.html Quote Link to comment
Fireball3 Posted March 8, 2014 Author Share Posted March 8, 2014 Here's some example code. #!/bin/bash noCountdownHours="18 19 20 21 22 23" check_hour_list() { HOUR=$(date +%H) HOUR_MATCHED_RC=1 for CountdownHour in ${noCountdownHours} do if [ ${HOUR} = ${CountdownHour} ] ; then HOUR_MATCHED_RC=0 return ${HOUR_MATCHED_RC} fi done return ${HOUR_MATCHED_RC} } hour=$(date +%H) hourMatch=$(echo "$noCountdownHours" | grep "$hour" | wc -l) echo "noCountdownHours: ${noCountdownHours}, current hour: $hour, match? $hourMatch" if check_hour_list then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ $? -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ ${HOUR_MATCHED_RC} -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi OK, now we have a variable set and don't need to call the function. But how do we notice if the hour has passed and we need to check again? Quote Link to comment
WeeboTech Posted March 8, 2014 Share Posted March 8, 2014 Here's some example code. #!/bin/bash noCountdownHours="18 19 20 21 22 23" check_hour_list() { HOUR=$(date +%H) HOUR_MATCHED_RC=1 for CountdownHour in ${noCountdownHours} do if [ ${HOUR} = ${CountdownHour} ] ; then HOUR_MATCHED_RC=0 return ${HOUR_MATCHED_RC} fi done return ${HOUR_MATCHED_RC} } hour=$(date +%H) hourMatch=$(echo "$noCountdownHours" | grep "$hour" | wc -l) echo "noCountdownHours: ${noCountdownHours}, current hour: $hour, match? $hourMatch" if check_hour_list then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ $? -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ ${HOUR_MATCHED_RC} -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi OK, now we have a variable set and don't need to call the function. But how do we notice if the hour has passed and we need to check again? Run the function again. Quote Link to comment
Fireball3 Posted March 8, 2014 Author Share Posted March 8, 2014 Run the function again. Then I'm here again - or not?: Edit: According to my findings this should be correct: (command expansion for check_hour) if [[ $(check_hour) -eq 0 && "$LockFile" -eq 0 ]] Quote Link to comment
WeeboTech Posted March 9, 2014 Share Posted March 9, 2014 Run the function again. Then I'm here again - or not?: Edit: According to my findings this should be correct: (command expansion for check_hour) if [[ $(check_hour) -eq 0 && "$LockFile" -eq 0 ]] That will work. It does actually fork another process to get the value of the shell function. it could be done a lil more awkwardly like if check_hour_list && [ $lockfile -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi or check_hour_list if [[ $CHECK_HOUR_RC -eq 0 && $lockfile -eq 0 ]] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi or check_hour_list if [[ $? -eq 0 && $lockfile -eq 0 ]] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi if you are interested in the shell programming sub-process example, I can post it so you get visual of what happens. I don't want to put more in here then you need. Let me know. Quote Link to comment
Fireball3 Posted March 9, 2014 Author Share Posted March 9, 2014 Thank you, it's already overwhelming what you provided. So after all, there is no other way then forking another process in this case - just for confirmation. I will have to study the last posts and change that accordingly in my script. And then I have to check if everything works. I need to look into the debug function of the shell (set -x -v). Seems useful from what I read. Are you familiar with the DOS shell also? Anyway, I will open another thread for that problem tomorrow. Quote Link to comment
WeeboTech Posted March 9, 2014 Share Posted March 9, 2014 Thank you, it's already overwhelming what you provided. So after all, there is no other way then forking another process in this case - just for confirmation. I will have to study the last posts and change that accordingly in my script. And then I have to check if everything works. I need to look into the debug function of the shell (set -x -v). Seems useful from what I read. Are you familiar with the DOS shell also? Anyway, I will open another thread for that problem tomorrow. The 3 versions I've shown only fork to run the date command, other then that they do not fork a shell as a sub process. whenever you do $(command) or `command` the process splits(forks) after a pipe is set up. It's not normally an issue these days unless you are doing this in a loop many many times. Example: if a file is being read in the shell for many lines and each line has a bunch of VAR=`cut` commands. I had a shell program once checking a file's age via external call. When the file was small it ran in under a minute. As the control file got larger and larger the time to run exponentially grew until it was taking over 3 minutes to test all the files. I re-wrote it in perl and the process now ran in under 3 seconds. In the case for the autosleep there's probably little degradation. I just saw an opportunity to make it more efficient and provide food for thought on an easy habit that will bite someone later on. Quote Link to comment
Fireball3 Posted March 9, 2014 Author Share Posted March 9, 2014 The 3 versions I've shown only fork to run the date command, other then that they do not fork a shell as a sub process. whenever you do $(command) or `command` the process splits(forks) after a pipe is set up. If it's that easy and makes such a difference, mabe you show me that example you mentioned. Quote Link to comment
WeeboTech Posted March 9, 2014 Share Posted March 9, 2014 here is my test script, you can comment uncomment as you want to test. The goal was to test/show variances between usage syntax. #!/bin/bash noCountdownHours="16 18 19 20 21 22 23" check_hour_list() { HOUR=$(date +%H) HOUR_MATCHED_RC=1 echo "check_hour's pid: $$ ppid: $PPID, BASHPID: $BASHPID">&2 for CountdownHour in ${noCountdownHours} do if [ ${HOUR} = ${CountdownHour} ] ; then HOUR_MATCHED_RC=0 return ${HOUR_MATCHED_RC} fi done return ${HOUR_MATCHED_RC} } hour=$(date +%H) hourMatch=$(echo "$noCountdownHours" | grep "$hour" | wc -l) echo "noCountdownHours: ${noCountdownHours}, current hour: $hour, match? $hourMatch" if check_hour_list then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ $? -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [ ${HOUR_MATCHED_RC} -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi if [[ $(check_hour_list) -eq 0 ]] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi echo "check_hour_list: $(check_hour_list)" >&2 lockfile=0 if check_hour_list && [ $lockfile -eq 0 ] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [[ $? -eq 0 && $lockfile -eq 0 ]] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi check_hour_list if [[ ${HOUR_MATCHED_RC} -eq 0 && $lockfile -eq 0 ]] then echo "check_hour_list current hour matched" else echo "check_hour_list current hour DID NOT MATCH" fi exit let i=1 lockfile=0 while [[ $i -lt 10000 ]] do (( i++ )) # if check_hour_list # then : echo -e "$i \c" # else : echo -e "$i \c" # fi if [[ $(check_hour_list) -eq 0 ]] then : echo -e "$i \c" else : echo -e "$i \c" fi done First Run root@unRAID:/boot/bin# ./check_hour.sh noCountdownHours: 16 18 19 20 21 22 23, current hour: 13, match? 0 check_hour's pid: 19006 ppid: 29047, BASHPID: 19006 check_hour_list current hour DID NOT MATCH check_hour's pid: 19006 ppid: 29047, BASHPID: 19006 check_hour_list current hour DID NOT MATCH check_hour's pid: 19006 ppid: 29047, BASHPID: 19006 check_hour_list current hour DID NOT MATCH check_hour's pid: 19006 ppid: 29047, BASHPID: 19015 check_hour_list current hour matched check_hour's pid: 19006 ppid: 29047, BASHPID: 19018 check_hour_list: check_hour's pid: 19006 ppid: 29047, BASHPID: 19006 check_hour_list current hour DID NOT MATCH check_hour's pid: 19006 ppid: 29047, BASHPID: 19006 check_hour_list current hour DID NOT MATCH check_hour's pid: 19006 ppid: 29047, BASHPID: 19006 check_hour_list current hour DID NOT MATCH It can be seen that some syntax usage causes a fork when examining BASHPID. Now to prove/disprove how this affects things we have to iterate many many times over the same code. As I said before, in this particular usage and on an idle system it doesn't matter all that much. When you have other jobs running it becomes even more apparent and the CPU load will spike rapidly. This is a test of check_hour doing 1000 iterations with if [[ $(check_hour) ]] root@unRAID:/boot/bin# time ./check_hour.sh noCountdownHours: 16 18 19 20 21 22 23, current hour: 13, match? 0 check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour matched check_hour_list: check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH real 0m8.335s user 0m0.120s sys 0m0.270s And another doing if check_hour root@unRAID:/boot/bin# time ./check_hour.sh noCountdownHours: 16 18 19 20 21 22 23, current hour: 13, match? 0 check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour matched check_hour_list: check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH real 0m6.884s user 0m0.740s sys 0m0.170s small potatoes at the moment. When we bump this up to 10,000 iterations. let i=1 lockfile=0 while [[ $i -lt 10000 ]] do (( i++ )) # if check_hour_list # then : echo -e "$i \c" # else : echo -e "$i \c" # fi if [[ $(check_hour_list) -eq 0 ]] then : echo -e "$i \c" else : echo -e "$i \c" fi doneroot@unRAID:/boot/bin# time ./check_hour.sh noCountdownHours: 16 18 19 20 21 22 23, current hour: 13, match? 0 check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour matched check_hour_list: check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH real 1m22.140s user 0m1.550s sys 0m1.930s and if check_hours let i=1 lockfile=0 while [[ $i -lt 10000 ]] do (( i++ )) if check_hour_list then : echo -e "$i \c" else : echo -e "$i \c" fi # if [[ $(check_hour_list) -eq 0 ]] # then : echo -e "$i \c" # else : echo -e "$i \c" # fi done root@unRAID:/boot/bin# time ./check_hour.sh noCountdownHours: 16 18 19 20 21 22 23, current hour: 13, match? 0 check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour matched check_hour_list: check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH check_hour_list current hour DID NOT MATCH real 1m8.321s user 0m6.870s sys 0m2.020s Again in this example, small potatoes, but in a production system with many processes, the gap widens as the CPU has to context switch. Quote Link to comment
Fireball3 Posted March 17, 2014 Author Share Posted March 17, 2014 Thanks for taking your time to write those examples. Finally found some time to work through them. 1. Can we say that there is no possibility to call a function inside an IF without forking? The function has to run prior to the IF and the IF itself is only comparing return values or variables. This is probably the same for while-loops? What would be a workaround to avoid this fork? while [[ $(hour_match) -eq 1 ]] # do Probably starting with "while [1 ]" and calling the hour_match after the first loop-run? 2. check_LockFile hour_match if [[ $? -eq 0 && $LockFile -eq 0 ]] When doing it like this: "$? -eq 0" is working only if the function was called in the prior line? So comparing multiple return codes won't be possible? The second value has to be a variable? 3. Edit: I noticed you use the variables like: if [[ ${HOUR_MATCHED_RC} -eq 0 && $lockfile -eq 0 ]] while in the whole script the variables are within quotes. if [[ "$hourMatch" -eq 0 && "$LockFile" -eq 0 ]] Quotes = string, no quotes = integer? If this is true, then I will have to stick either to the string-writing or to the integer-writing. Then I would have to do: if [[ "$?" -eq 0 && "$LockFile" -eq 0 ]] or if [[ $? -eq 0 && $LockFile -eq 0 ]] Right? 4. Conclusion: While working with return codes seems pretty slight/slender it is not very transparent to the noob user. Working with variables seems better especially if someone else is supposed to decipher it. Quote Link to comment
WeeboTech Posted March 17, 2014 Share Posted March 17, 2014 This only really matters if you iterate over the same code many times in a loop. Hence the reason why it's easy to get away with. i.e. until a system starts to bog and you see why. $(function or command) causes a fork(split) no matter what. calling a shell function and storing $? (return code) or storing a testable value in global variable is the only way to avoid the split. As I mention, this matters most when you are calling something many times in a loop. In the prior code I examined, there were 3 splits, to get the value of a string match. Here you can go with what works best. I.E. it's either best for the human (understand-ability) or the computer (efficiency). if [[ $? -eq 0 && "$LockFile" -eq 0 ]] Is correct syntax. The quotes are to prevent an empty variable from causing a script to fail. $? never needs to be quoted. I use braces to be explicitly clear. Especially since I do this all day long 7 days a week. Quote Link to comment
Fireball3 Posted March 17, 2014 Author Share Posted March 17, 2014 OK, I think I can test the modifications now. What is the best way to go for testing. I read something about the debug function. bash -x auto_s3_sleep.sh I have to test that. I would probably need to cycle through each variable... Quote Link to comment
WeeboTech Posted March 17, 2014 Share Posted March 17, 2014 OK, I think I can test the modifications now. What is the best way to go for testing. I read something about the debug function. bash -x auto_s3_sleep.sh I have to test that. I would probably need to cycle through each variable... That's one way to do it. With the line as follows you can do it on demand [ ${DEBUG:=0} -gt 0 ] && set -x -v you can enable it with export DEBUG=1 ./auto_s3_sleep.sh or DEBUG=1 ./auto_s3_sleep.sh Quote Link to comment
Fireball3 Posted March 17, 2014 Author Share Posted March 17, 2014 You mentioned that already. Obviously I didn't fully understand it. ...going back and looking up "export"...OK Globally set DEBUG=1 I suppose this debug is different to the script $debug that is used for logging options. The difference is only the way the script is called, isn't it? sh -x -v auto_s3_sleep.sh vs. DEBUG=1 ./auto_s3_sleep.sh Where is the benefit? Quote Link to comment
WeeboTech Posted March 17, 2014 Share Posted March 17, 2014 You mentioned that already. Obviously I didn't fully understand it. ...going back and looking up "export"...OK Globally set DEBUG=1 I suppose this debug is different to the script $debug that is used for logging options. The difference is only the way the script is called, isn't it? sh -x -v auto_s3_sleep.sh vs. DEBUG=1 ./auto_s3_sleep.sh Where is the benefit? I have not looked at how $debug was used in the script You can have different levels of debugging. DEBUG=3 could mean something to sub shells or functions. Yes I've used it in that way. I don't have examples. Certain loops and shell functions constructs do not show up fully in trace mode. so I would put another one liner in the shell function and set the number higher in that function. Then by doing DEBUG=2 or 3 they would show up. Here's an interesting snippet to review. Again, remember I always set ${P} to the script basename. if [ ${DEBUG:=0} -gt 0 ] then set -x -v exec 9>/tmp/${P}.trace.$$ export BASH_XTRACEFD=9 export PS4='+ (${BASH_SOURCE}{LINENO}{FUNCNAME[0]-} ' fi Quote Link to comment
Fireball3 Posted March 17, 2014 Author Share Posted March 17, 2014 Puh, that's really hardcore... I have to google everything and puzzle something together. And use the PS4 prompt to give more useful information like: export PS4='+(${BASH_SOURCE}{LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' http://wiki.bash-hackers.org/scripting/debuggingtips ...munching... Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.