shell scripts

[JunkMale]

Any idea what shell scripts are written in?

The web host was kind enough to provide me with a script to help run webalizer to update the webalize.hist file

I wrote a script that scrapes this .hist file for its stats for the web site main page.

All was fine until the site started getting more than 40k a month in visits as I was starting to get cronjob error messages because of some locking a file.

So how does one write a shell script? I found some sites byt they all do the same thing, launch off in to technical jargon assuming that you have prior systems knowledge of what square peg gets hammered in to whar round hole.

The shell script has...

#! /bin/bash
cd /home/sites/****.***.**/public_html/stats
/usr/bin/webalizer -D dns.cache -nlocalhost -p -o . /home/sites/****.***.**/logs/****.***.**-access_log*

I would like to add to this sctipt the ability to repeat 20 times maximum attempts to read the dns.cache when theirs a lock error or is their a way to tell webalizer to access the file with a shared lock?

[scragar]

Dude, bash is easy:

#! /bin/bash

# cd = change directory.
cd /home/sites/****.***.**/public_html/stats

# for(I = 1; I <= 20; I++){
for I in $(seq 20); do
  ## Try the script, if it returns true we break out of the loop
  /usr/bin/webalizer -D dns.cache -nlocalhost -p -o . /home/sites/****.***.**/logs/****.***.**-access_log* && break;

  ## we couldn't lock the file. Wait a second, then try again:
  sleep 1s;

 #end for loop
done

Oh, and to answer your very first question, shell scripts are written in whatever it says to use as a parser:

#!/bin/bash

Effectively means "Run me using the parser bash, which can be found in /bin/ ". If you wanted to you could use PHP, Perl, Python, or almost any language you could think of as a parser(including other scripts, which are handled in the same way).

[JunkMale]

OK, thanks, I just about followed that, why they cant just have a for loop?!

What about making it pause for a one second limit per loop? as the thing currently stands, it will execute immediately one after another? or will it wait until the result of the last execution of the webalizer command?

[scragar]

Originally Posted by JunkMale

OK, thanks, I just about followed that, why they cant just have a for loop?!

Bash supports C style for loops. I just find typing a few less characters and using seq is easier(especially given the `break` usage).

http://bash-hackers.org/wiki/doku.php/syntax/ccmd/c_for

What about making it pause for a one second limit per loop? as the thing currently stands, it will execute immediately one after another? or will it wait until the result of the last execution of the webalizer command?

Bash will always wait for each command to return success or failure before continuing to the next command, UNLESS the first command is switched to the background.

So the 1second wait isn't really needed, I just included it because I have no idea how long webaliser tries to run for before it encounters the error, if it's a matter of a few seconds then the wait isn't needed, but if the script tries to run for a millisecond, and returns failure the instant it finds the file lock then all the iterations are likely to run before the file lock would be released.

[JunkMale]

Thanks.

[JunkMale]

I have hit a snag and need to have this script run every hour but only at certain times the webalizer needs to run like every 3 hours..

The times are in hours 00,06,09,12,15,18,21

Would I be right in thinking that this would work if thehour has the hour value? and next question would be how do you set thehour to the current hour?

#! /bin/bash

THEHOUR = ???

# THEHOUR will be the current hour of the day.
case THEHOUR in (00 | 06 | 09 | 12 | 15 | 18 | 21 ) 
        
  # cd = change directory.
  cd /home/sites/****.***.**/public_html/stats

  # for(I = 1; I <= 20; I++){
  for I in $(seq 20); do
    ## Try the script, if it returns true we break out of the loop
    /usr/bin/webalizer -D dns.cache -nlocalhost -p -o . /home/sites/****.***.**/logs/****.***.**-access_log* && break;

    ## we couldn't lock the file. Wait a second, then try again:
    sleep 1s;

   #end for loop
  done;;
esac

What I need is to check if the hours are any in the list of hours and if so eun the update routine. This will help me free up the rather limited access to cronjobs as I need to run a PHP script as a shell at 9pm and that means I need to modify this shell so the daily image roatate routine can be run automatically.

Or can you point me to a dummies guide? I have been trying to find examples but I havent a clue what to put as using shell scripts as a search term turns up all other scripting that can run as a shell.

[scragar]

Firstly I'd like to point out that cron supports comma seperated lists for time, provided there's no spaces.

0 0,3,6,9,12,15,18,21 * * * your_bash_script.sh

Alternatively you can get the time with date and test it that way:

HOUR=$(date +%l); ## date [options] +FORMAT
## %l = hour 1-12, no leading 0s

IS3HOUR=$(($HOUR % 3));## modulus maths :p

if [ $IS3HOUR -eq 0 ]; then

  ## loop in here

fi

Personally I'd do it all as a 1 liner

[ $(( $(date +%l) % 3)) -ne 0 ] && exit;
## exit the script if the current hour is not an exactly multiple of 3

[JunkMale]

First off, thanks for the rocket science lessons...

I neglected to mention that the host limits each domain to a maximum of 3 crons and they are only settable via the account CP and that they are also limited to what can be set as times.


So if

[ $(( $(date +%l) % 3)) -ne 0 ] && exit;
## exit the script if the current hour is not an exactly multiple of 3

is placed as the first line it will end the script if its not a division of 3.

#! /bin/bash

[ $(( $(date +%l) % 3)) -ne 0 ] && exit;
## exit the script if the current hour is not an exactly multiple of 3

# cd = change directory.
cd /home/sites/****.***.**/public_html/stats

# for(I = 1; I <= 20; I++){
for I in $(seq 20); do
  ## Try the script, if it returns true we break out of the loop
  /usr/bin/webalizer -D dns.cache -nlocalhost -p -o . /home/sites/****.***.**/logs/****.***.**-access_log* && break;

  ## we couldn't lock the file. Wait a second, then try again:
  sleep 1s;

 #end for loop
done

I guess this is going to take me some time to learn a new trick.

[scragar]

If there's anything I've not explained, or you have a hard time understanding I'd love to try and explain it to you(again?).

Bash is rather hard to explain though when working with more complex code, if you start with bash where most people do(it's similar to batch files in windows, a list of commands to be executed in the shell) it's a lot easier.

For reference:
$( something ) execute something
$(( something )) evaluate something using the maths rules
[ something ] test something and return success(true 0) or failure(false 1)

So some of the parts that might have been confusing explained:

for I in $(seq 20);
++ `seq 20` == 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
+ for I in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20;

[ $(( $(date +%l) % 3)) -ne 0 ] && exit;
+++ `date +%l` == " 8"
++ `8%3` == 2
+ [2 -ne 0] == true
+ exit
## && means if the previous exited on success do this

[JunkMale]

I may have to sleep on that and look at it with fresh eyes.

[JunkMale]

	
	
[ $(( $(date +%l) % 3)) -ne 0 ] && exit;
Breaking this down, date +%l how does the +%l work? I am lost on the GNU BASH site, can't even find 'date' 

Anyway, I guess this is roughly what it does is it gets the datetime stamp from the system, extracts the hour,  passes it throught a mod of 3 that is tested to see if it is not equal to 0 then return value. The value returned depends on the hour, if a true is returned then the script is terminated.

I dont suppose you have any pointers on how the following problem.

I wrote a PHP script. I tested it, it worked, I then converted it to a shell running as a PHP file. They have said I had to remove the shebang which I did, put in to the cron job the shebang path followed by the script with the php extention.

I hit the test script button in the admin CP and I get a green light and when I check the result in web space, it has done its job of rotating a picture on a test URL. The aim is to pick at random from an image pool, call the image by a new name that is potd.jpg for example then when its time is up, the image is copied under a new name, has a date and time stamp in the name, potd_0909112100.jpg for example.

The new image is picked at random ind is copied with the new name... you guessed it potd.jpg where the extention of the file is generated automatically.

Now, the web host, as usual the tech dept is dragging its heels and want proof that the script is not working when it is automated as a cron.

So my question is this, can BASH run a PHP script because BASH does not seem to have any issues as the other script works fine. I get the odd error message but the script has still worked and done its job.

Anyway, what can I do to resolve this?

[scragar]

I love your understanding of bash, it come from the same understanding of scripting languages, which I should stress bash is only sort of a scripting language, bash is both a scripting language and a command shell(like cmd.exe for windows(or dos if you want to call it that)).
The call to date is a program called date. It doesn't exist in the bash documentation because it's to a program called date, not a feature of bash called date.
http://unixhelp.ed.ac.uk/CGI/man-cgi?date
If there's ever a line you don't understand look at the first word, and try it in that url in place of date, so if I was to use echo like so:

echo "foo\nbar"
echo -n "foo\nbar"
echo -e "foo\nbar"
echo -en "foo\nbar"

You could look up echo and see exactly what it does and why the output is different for all of those lines. Man pages can be a little complex to read, but that's because of the power given to most commands(Do one thing and do it really, really well, as opposed to do loads of things rather badly, so you'll find echo supports so many options and features you never expected that it'll make your eyes water ), rather than a flaw in the language itself.

As for running a PHP script from bash, there are a good number of ways to do it, if you know the path to the PHP executable:

#!/usr/bin/php
<?php
//.......

Or maybe if you don't know that just trust it's in the $PATH

#!/bin/bash
php $0; exit
<?php
//.........

Either options should work, although I have more faith in the second if you don't know the setup of the box.

I'm assuming in both of those that your wanting to add the .php script directly to the cron, and not, wanting a wrapper. if you want a wrapper you'll have to use the second, and just replace $0; exit with the path to your PHP script, and don't put anything else after it.

[JunkMale]

OK, so + is a pad 0 ?

%l is the hours 1 thru 12 so if %k was subbed, we would have 24 hour version of that test.


Anyway, I now get this in my email and I guess that this is the message I was looking for.

X-Powered-By: PHP/5.2.10
Content-type: text/html

The output I guessed was indication that the script executed and I checked and saw that the image had changed. So I guess my moan about the fact that the script was working when I manually executed it and asked why the cron failed sort of got them looking in to it.

I will keep the BASH option in mind and set it up like that if the cron fails again.

Cheers.

[scragar]

The + is part of dates syntax, it uses the + to make the start of the format, since the format can include spaces and such.

%k would give you the hour in 24 hour format, but for the purposes of dividing by 3 the hour format makes no difference, 12 is just easier to work with than 24.

If you do wish to learn more about bash I recommend you download a liveCD for a linux distro or two and run it on your home PC from time to time, take a look at the terminals and play around with a few things(You can't cause any damage unless you choose to mount your hard drives or run commands like `dd if=/dev/random of/dev/sda` which would overwrite your first hard drive(s for sata, d for device, a because it's first) with random data(/dev/random, obviously)).

My personal recommendation for a first run liveCD would be either linux mint( http://www.linuxmint.com/ ) or ubuntu ( http://www.ubuntu.com/ ), both are debian based, which is what you'll find on a lot of servers.

It's also very nice to have a linux liveCD if you want to set bash scripts in future, that way you can avoid people who might want to give you bad commands(watch out for anything involving the keywords sudo(gives root perms), dd(can write to devices, and with random data as an input device, not nice), yes(repeatedly sends 'y' as output, great for some things, but also very bad for people planning to cripple your ram) or with lot's of brackets and braces(likely fork bomb))

[JunkMale]

Yep, have ubuntu. Very new to Linux.

Back to the BASH script.

So if I had the following code

#! /bin/bash

[ $(( $(date +%l) % 3)) -ne 0 ] && exit;
## exit the script if the current hour is not an exactly multiple of 3

# cd = change directory.
cd /home/sites/****.***.**/public_html/stats

# for(I = 1; I <= 20; I++){
for I in $(seq 20); do
  ## Try the script, if it returns true we break out of the loop
  /usr/bin/webalizer -D dns.cache -nlocalhost -p -o . /home/sites/****.***.**/logs/****.***.**-access_log* && break;

  ## we couldn't lock the file. Wait a second, then try again:
  sleep 1s;

 #end for loop
done
PHP $0; exit;
<?php

// my script here 

?>

would it run the appended PHP script?