Posts Tagged cron
Monitor s3sync with Zabbix
s3sync is a ruby program that easily transfers directories between a local directory and an S3 bucket:prefix. It behaves somewhat, but not precisely, like the rsync program.
I am using this tool to automatically backup the important data from Debian servers to Amazon S3. I am not going to explain here how to install s3sync as it is not the purpose of this article. However, you can read this very useful article from John Eberly’s blog: How I automated my backups to Amazon S3 using s3sync.
If you followed the steps from John Eberly’s post, you should have an upload.sh script and a crontab job which executes this script periodically.
From this point, here is what you need to do to monitor the success of the synchronisation with Zabbix:
- Add the following code at the end of your
upload.shscript:# print the exit code RETVAL=$? [ $RETVAL -eq 0 ] && echo "Synchronization succeed" [ $RETVAL -ne 0 ] && echo "Synchronization failed"
- Log the output of the cron script as follow:
30 2 * * sun /path/to/upload.sh > /var/log/s3sync.log 2>&1
- On Zabbix, create a new item which will check the existence of the sentence “Synchronization failed” in the file
/var/log/s3sync.log:
Item key:vfs.file.regmatch[/var/log/s3sync.log,Synchronization failed]
- Still on Zabbix, define a new trigger for the previously created item:
Trigger expression:{Template_AmazonCloud_Debian:vfs.file.regmatch[/var/log/s3sync.log,Synchronization failed].last(0)}=1
With these few steps, you should now receive Zabbix alerts when a backup on S3 fails. 🙂
Refresh GeoIP automatically
GeoIP is a very useful tool provided by MaxMind. It can determine which country, region, city, postal code, and area code the visitor is coming from in real-time. For more information, visit MaxMind website.
This tool is also coming with an Apache module allowing to redirect users depending on their location. For example, we could redirect all users from France to the French home page of a multi-language website, or we could block the traffic to users from a specific country.
To install this module on a Debian server, you simply need to run the following command:
apt-get install libapache2-mod-geoip
But, how does this module work? How does it know where the user comes from? 😯
It is actually quite simple: GeoIP is using a mapping file of IP address by country. On Debian, this file is stored in the folder /usr/share/GeoIP and is named GeoIP.dat.
However, the IP addresses are something which change all the time. So this file will get out-of-date very quickly. This is why MaxMind provides an updated file at the beginning of each month for free. To read the installation instructions, please click the following link: http://www.maxmind.com/app/installation
This is good and well, but who will remember or even have the time to update this file every month? And imagine if you have to do this on hundreds of servers?
The solution is to use a shell script which will download, extract and install the updated GeoIP file automatically once a month:
#!/bin/sh # Go in the GeoIP folder cd /usr/share/GeoIP # Remove the previous GeoIP file (if present) rm GeoIP.dat.gz # Download the new GeoIP file wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz # Remove the previous GeoIP backup file rm GeoIP.dat.bak # Backup the existing GeoIP file mv GeoIP.dat GeoIP.dat.bak # Extract the new GeoIP file gunzip GeoIP.dat.gz # Change the permission of the GeoIP file chmod 644 GeoIP.dat # Reload Apache service apache2 reload
You can place this file in your root folder and set up the following crontab job:
0 0 3 * * /root/update_geoip.sh
This will execute the script automatically on the third day of every month.
Using the ‘date’ command in your crontab
Crontab, as most people know, enables users to schedule commands or shell scripts to run periodically at certain times or dates.
The other day, this very useful Linux tool gave me a hard time! 🙁
Indeed, one of my commands wasn’t working in cron but was working perfectly fine when written in a shell console.
The faulty command looked like this:
0 5 * * 3 /data/script.sh > /data/script_`date +%y%m%d`.log 2>&1
If I run this command in a shell console, everything works fine and I get a log file containing today’s date in its filename. However, if I set this command line in my crontab, it doesn’t work and no log file is even created!
Reading the documentation of cron, I discovered the following statement:
Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input.
Well, this is good to know, isn’t it? 😉
We need to escape the percent-signs on our command line.
So in order to get our ‘faulty’ command to run in cron, it needs to look like the following:
0 5 * * 3 /data/script.sh > /data/script_`date +\%y\%m\%d`.log 2>&1
Locale settings for your cron job
Do you get special characters problem when executing your bash script from a cron job?
And does the same script work fine when it is directly executed from the command line?
If yes, continue reading this article! 😉
The reason of this characters problem is probably because of your locale settings.
Indeed, If you try to run the command locale from the command line and from a cron job, you may get different results such as:
| From the command line | From a cron job |
LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= |
LANG= LC_CTYPE="POSIX" LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE="POSIX" LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_PAPER="POSIX" LC_NAME="POSIX" LC_ADDRESS="POSIX" LC_TELEPHONE="POSIX" LC_MEASUREMENT="POSIX" LC_IDENTIFICATION="POSIX" LC_ALL= |
As you can see, the cron job is not using UTF-8. That must be the problem! 🙂
So the question now is how to change the locale settings for the cron job?
Some people say that you need to add the following environment variables to the crontab entry:
SHELL=/bin/bash LANG=en_US.UTF-8 LANGUAGE=en LC_CTYPE=en_US.UTF-8
But this actually didn’t work for me. 🙁
What you can do instead is create (if not already present) the file /etc/environment and add the following line:
LANG=en_US.UTF-8
The cron process will read this file when it starts, so you need to restart it in order to apply the change:
service cron restart
Hope this will fix your characters problem. 😉
WWF – Help us
