README
1------------------------------------------------------------------------
2amavis-stats - generate amavis scan statistics from logfiles
3------------------------------------------------------------------------
4
5Licence
6-------
7Copyright (C) 2003, Mark Lawrence (nomad@null.net)
8
9amavis-stats is free software; you can redistribute it and/or modify
10it under the terms of the GNU General Public License as published by
11the Free Software Foundation; either version 2 of the License, or
12(at your option) any later version.
13
14
15Author
16------
17Mark Lawrence <nomad@null.net>. Aussie living in Switzerland.
18Comments are welcome. So would somewhere else to host this.
19
20
21Description
22-----------
23amavis-stats is a simple amavis statistics generator based on rrdtool. It
24produces graphs of clean emails, spam emails and infected emails broken
25down by virus, from amavis log entries (such as /var/log/mail.info).
26
27You can see an example of this in action here:
28
29 http://rekudos.net/amavis-stats/
30
31The RRD files are created and updated by a perl script. Since the script
32uses the date/time values from the log entries it can be run (ir)regularly
33from cron(8) or by hand as desired. Graphs are generated by a php script.
34
35
36Dependencies
37------------
38As of version 0.1.3 amavis-stats works with amavisd and amavisd-new on linux
39and Solaris. There has also been one confirmed report of this working on
40MacOS 10.3! For amavisd I believe you need to have the log level set at 5.
41
42The perl script uses nothing which probably isn't in the standard perl
43installation.
44
45Graphs are generated by php in combination with either the command-line
46rrdtool or the php4-rrdtool shared library.
47
48
49Download
50--------
51Raw Source:
52 http://rekudos.net/download/amavis-stats.tar.gz
53
54Debian Apt (package name "amavis-stats"):
55 sources.list: "deb http://rekudos.net/debian ./"
56 root# apt-get update; apt-get install amavis-stats
57
58#RPM (EXPERIMENTAL!):
59# http://rekudos.net/amavis-stats/amavis-stats.rpm
60# root# rpm -i amavis-stats.rpm
61
62
63Install (not debian or rpm)
64-------
651. Download and extract the source:
66
67 root# wget http://rekudos.net/download/amavis-stats.tar.gz
68 root# tar -xvf amavis-stats.tar.gz
69 root# cd amavis-stats-0.X.X/
70
712. Install the program by typing (as root):
72
73 root# ./configure [options]
74 root# make
75 root# make -n install (optional)
76 root# make install
77 root# make installapache
78
79The options to configure (apart from the standard prefix's) can be:
80
81 --prefix=DIR installation prefix [/usr/local]
82 --with-install-prefix=DIR (not normally used - mostly for Debian)
83 --with-var-prefix=DIR /var prefix
84 --with-amavis-user=USER specify user to run as [amavis]
85 --with-amavis-group=GROUP specify group to run as [amavis]
86 --with-web-user=USER specify user which webserver runs as [www]
87 --with-web-group=GROUP specify group which webserver runs as [www]
88
89The amavis-user,group and web-user,group must already exist - they will
90*not* be automatically created.
91
924. If you do not already have a user "amavis" you should create one (or
93decide on another user to run amavis-stats as). You should also check that
94the user has permissions to read your particular log file. On debian, this
95means that in the .deb I have to add the "amavis" user to the "adm" group.
96Otherwise you can run it as root, but no responsibility lies here for that.
97
985. As your user "amavis" simply run amavis-stats on the log file that
99contains the amavis entries. You should see something like the following:
100
101 root# su amavis
102 amavis# /usr/sbin/amavis-stats /var/log/mail.info
103 ./amavis-stats: First Time Run
104 ./amavis-stats: First seen virus #1 (passed) at 1064126856
105 ./amavis-stats: First seen virus #2 (infected) at 1064127319
106 ./amavis-stats: First seen virus #3 (W32/Swen.A@mm) at 1064127319
107 ./amavis-stats: First seen virus #4 (Forten.F@m) at 1064131924
108 ./amavis-stats: First seen virus #5 (W32/Lentin.H@mm) at 1064133728
109 ./amavis-stats: First seen virus #6 (W32/Bugbear.B@mm) at 1064146880
110 ...
111
112You should then see a bunch of rrd files in /var/lib/amavis-stats.
113
1144. If this looks right then you can copy the debian/amavis-stats.cron.d
115file to the appropriate location (sometimes /etc/cron.d/amavis-stats)
116so that it runs every five minutes. Check that this format of contab
117is compatible with whatever version you are running.
118
1195. If all has gone well you should be able to hit
120http://localhost/amavis-stats/ with your web-browser and see some nice
121looking graphs.
122
1236. You could also run the php script by hand to check for obvious
124errors. Afterwards there should be eight "png" files in your directory:
125
126 root# php4 /usr/share/amavis-stats/amavis-stats.php
127 X-Powered-By: PHP/4.1.2
128 Content-type: text/html
129
130
131 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
132 <html>
133 <head>
134 ...
135 </html>
136
137 root# ls *png
138 passed-day.png passed-week.png virus-day.png virus-week.png
139 passed-month.png passed-year.png virus-month.png virus-year.png
140
1417. I suggest that you send mail for the "amavis" user to another account on
142the system as amavis-stats generates text each time it sees a new virus.
143
144I am also thinking of possibly generating further statistical information
145for a days worth of events. Such things as:
146
147- Top 10 recipients of SPAM, BANNED, INFECTED, Passed emails
148- Top 10 senders of SPAM, BANNED, INFECTED, Passed emails
149
150If you would find this useful please let me know.
151
152
153Problems/FAQ
154------------
155Q. What is the best way to handle rotated (*.gz) logfiles? How do I
156get all of the history in the graphs and not just the latest log?
157
158A. amavis-stats doesn't handle .tar or .gz files at the moment. What it
159expects is just the name of a single uncompressed file. So when I first
160install it on a machine I usually do the following to get the history:
161
162amavis$ ls mail.info*
163mail.info mail.info.0 mail.info.1.gz mail.info.2.gz mail.info.3.gz
164
165amavis$ sudo gzip -d mail.info.*.gz
166amavis$ /usr/sbin/amavis-stats -r mail.info.3
167amavis$ /usr/sbin/amavis-stats -r mail.info.2
168amavis$ /usr/sbin/amavis-stats -r mail.info.1
169amavis$ /usr/sbin/amavis-stats -r mail.info.0
170amavis$ /usr/sbin/amavis-stats -r mail.info
171amavis$ sudo gzip mail.info.[1-3]
172
173ie, you have to manually run ammavis-stats (as your "amavis" user) on each
174file in the correct chronological order
175
176
177Q. We restarted the amavisd with another loglevel (the default one: 0) and
178now I dont get any spam info in the amavis-stats. Where did it go?
179
180A. Spam information is only generated in some versions of amavis when the
181log level is set to a higher value. There is an effort underway to
182standardise the log output of all the Amavis, but please be patient, this
183will take some time.
184
185
186Q. The rrd files seem to get generated ok but I when I try and display the
187web page I get the error: amavis-stats::error: rrd_graph(): 127
188
189A. I believe that the error code 127 is returned from bash when it can't find
190the program to execute. I guess in this case that your web user (www-data
191or www or web) is unable to find the "rrdtool" executable. Perhaps it is
192not in the standard location, or the path for that user is not set?
193
194You could try the following to see if the tool can be found.
195
196 root# su -c env - www | grep PATH
197 root# su -c rrdtool - www
198
199
200
README.FreeBSD
1Quick installation guide for amavis-stats on FreeBSD
2----------------------------------------------------
3(c) Loic Pefferkorn (pef) <loic AT dev DOT erodia DOT net>
4
5The original document can be found here : http://www.erodia.net/doc/
6
7This file contains the following informations :
8
91.Introduction
101.1 About this document
111.2 What is amavis-stats ?
121.3 Requirements
132.Installation
142.1 Getting sources and installing them
152.2 Setting up Apache
162.3 Setting permissions
172.4 Creating user amavis
182.5 Modifying files
193.Final steps
203.1 Checking if all is ok
213.2 Cron job
224.Misc
23
24
251.Introdution
26-------------
27
281.1 About this document
29-----------------------
30This is a little guide which explains you how to get amavis-stats working on
31FreeBSD. I have STABLE-4.9 release, with others releases or *BSD, instructions may change.
32
33Since there is no package for amavis-stats, all is done "by hand".
34
351.2 What is Amavis-stats ?
36--------------------------
37Amavis-stats is a simple AMaViS statistics generator based on rrdtool.
38It produces graphs of clean emails, spam emails and infected emails broken
39down by virus, from amavis log entries.
40
41You can see example graphs on the screenshots page.
42http://rekudos.net/amavis-stats/node/view/7
43
441.3 Requirements
45----------------
46Amavis-stats needs some perl modules provided by the rrdtool package.
47
48You can install it by source :
49 cd /usr/ports/net/rrdtool ; make all && make install
50
51Or if you prefer binary package :
52 pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-4-stable/net/rrdtool-1.0.45.tgz
53
54
55note : please use a FreeBSD mirror http://mirrorlist.freebsd.org/
56
57You also need a running apache webserver, with php enabled.
58
59My system :
60-apache2 with mod_php4 enabled.
61-Postfix-2
62-Amavis-new
63-Clamav-0.65
64
65I consider Amavisd is configured as follow :
66-$DO_SYSLOG = 1;
67
68note : this setting can be modified in amavisd.conf file, usually located in /usr/local/etc/
69
70You can use a separate log file, just check that Amavis user can read it.
71
722.Installation
73--------------
74
752.1 Getting sources and installing them
76----------------------------------------
77Download the latest sources tarball :
78 wget http://rekudos.net/download/amavis-stats.tar.gz
79
80Uncompress it :
81 tar xvzf amavis-stats.tar.gz
82 cd amavis-stats-xx
83
84Copy amavis-stats to /usr/local/sbin :
85 cp amavis-stats /usr/local/sbin
86
87Create the necessary directories :
88 mkdir /usr/local/www/amavis-stats
89 mkdir -p /usr/local/lib/amavis-stats/img
90 ln -s /usr/local/lib/amavis-stats/img /usr/local/www/amavis-stats/img
91
92Copy amavis-stats.php to /usr/local/www/amavis-stats and make an index link :
93 cp amavis-stats.php /usr/local/www/amavis-stats
94 cd /usr/local/www/amavis-stats && ln -s amavis-stats.php index.php
95
96note : index.php is an index file, specified by "DirectoryIndex" in apache configuration.
97
98
992.2 Setting up Apache
100---------------------
101
102In the apache.conf provided change the paths to our :
103
104Alias /amavis-stats /usr/local/www/amavis-stats
105
106<Directory /usr/local/www/amavis-stats>
107(...)
108
109Add this to your real httpd.conf file :
110 cat apache.conf >> /usr/local/etc/apache2/httpd.conf
111
112note : instead of adding it, you can include it, rename apache.conf in amavis.httpd.conf,
113 move it to /usr/local/etc/apache2, and put in apache's httpd.conf file:
114 Include etc/apache2/amavis-stats.httpd.conf
115
116Restart apache
117 apachectl restart
118
119
1202.3 Set permissions
121-------------------
122
123Getting the user and the group whom apache is running under :
124 grep ^User\ /usr/local/etc/apache2/httpd.conf (2 spaces after User\)
125 grep ^Group\ /usr/local/etc/apache2/httpd.conf (2 spaces after User\)
126
127
128I found www as user and www as group. If you don't have the same,
129just remplace www:www by YourUser:YourGroup in the chown command.
130
131Modify permissions for img directory with user and group you had just found :
132(the php file will wrote img files on it, so apache must have write access)
133 chown www:www /usr/local/lib/amavis-stats/img
134
135
1362.4 Creating user amavis
137------------------------
138We need an amavis user, who will run cron job to update the data.
139
140Now creating him with adduser or sysinstall
141 useradd amavis
142
143Choose defaults values, for home choose /usr/local/lib/amavis-stats
144
145note : additionnal group wheel is necessary for being able to read /var/maillog file,
146 if you have setup amavisd for sending logs to syslog.
147
148Permissions for other directories, where amavis user must have write access for running his cron job :
149 chown amavis /usr/local/lib/amavis-stats
150 chown amavis /usr/local/www/amavis-stats
151
152
1532.5 Modifying files
154------------------
155Now we have to edit files :
156
157Edit amavis-stats and change :
158line 79 to :
159 $lockfile = "/var/tmp/$pkg";
160line 80 to :
161 $libdir = "/usr/local/lib/$pkg";
162
163
164Edit amavis-stats.php and change :
165line 282 to :
166 $as_libdir = "/usr/local/lib/amavis-stats";
167
168
1693.Final steps
170-------------
171
1723.1 Checking if all is ok
173-------------------------
174note : for the following instructions, put /var/maillog only if amavis send logs to syslog,
175 otherwise put the filelog you defined.
176
177Su as amavis user, and check if all is ok :
178 su amavis
179 /usr/local/sbin/amavis-stats /var/log/maillog
180
181Now you must have the following files in /usr/local/lib/amavis-stats :
182
1831.rrd amavis-stats.count amavis-stats.state
1842.rrd amavis-stats.names img
1853.rrd amavis-stats.seen
186
187
188Now check if the php page works, go to http://YourServerName/amavis-stats with your favorite browser.
189If "denied errors" is written you, check the permissions on directories. (see 2.3 Setting permissions)
190
191
1923.2 Cron job
193------------
194We put a cronjob now for having data up to date :
195 crontab -e -u amavis
196
197Write this :
198
199*/5 * * * * /usr/local/sbin/amavis-stats /var/log/maillog 2>&1 > /dev/null
200
201
202
203Amavis user doesn't need a valid shell, for more security you can disable it :
204 chsh -s /sbin/nologin amavis
205
206
2074.Misc
208------
209
210Feedbacks
211---------
212I am interested in any feedback, positive or negative, regarding the content of this document via e-mail.
213Definitely contact me if you find errors or obvious omissions.
214
215
216<loic AT dev DOT erodia DOT net>
217
218
219Version history
220---------------
221v 1.0 (Jan 31 2004 ) initial release
222v 1.1 (Feb 2 2004 ) minor changes and language corrections
223v 1.2 (Feb 6 2004 ) layout improvements, minor changes
224
225
226Copyright
227---------
228(c) 2004 Loic Pefferkorn
229
230This documentation may be reproduced in whole or in part, without fee, subject to the following restrictions:
231
232 * The copyright notice above and this permission notice must be preserved complete on all complete or partial copies.
233 * Any translation or derived work must be approved by the author in writing before distribution.
234 * If you distribute this work in part, instructions for obtaining the complete version of this documentation must be included,
235 and a means for obtaining a complete version provided.
236 * This document is placed under the GNU General Public License (see http://www.gnu.org/licenses/gpl.html)
237
238
239Thanks
240------
241-Authors of amavisd.
242-Mark Lawrence, for amavis-stats.
243-Bosco L Parisi for his very usefull advices ;)
244
245
README.FreeBSD.fr
1Petit guide pour l'installation d'amavis-stats sur FreeBSD
2----------------------------------------------------
3(c) Loic Pefferkorn (pef) <loic AT dev DOT erodia DOT net>
4
5Le document original est disponible ici : http://www.erodia.net/doc/
6
7Ce guide comporte les informations suivantes :
8
91.Introduction
101.1 A propos du guide
111.2 Qu'est-ce que amavis-stats ?
121.3 El�ments requis
132.Installation
142.1 T�l�charger les sources et les installer
152.2 Configurer Apache
162.3 Mettre les droits ad�quats
172.4 Cr�er l'utilisateur amavis
182.5 Modifier les fichiers
193.Etapes finales
203.1 Test de l'installation
213.2 Cron job
224.Divers
23
24
251.Introdution
26-------------
27
281.1 A propos du guide
29---------------------
30Ceci est un petit guide expliquant l'installation d'amavis-stats sur FreeBSD.
31J'ai la version STABLE-4.9, avec une autre version ou un autre *BSD, il ne doit
32pas y avoir de grandes diff�rences.
33
34Comme il n'y a pas de package tout pr�t, tout sera fait "� la main".
35
361.2 Qu'est-ce que amavis-stats ?
37--------------------------------
38Amavis-stats sert � g�n�rer des statistiques sur amavis, en utilisant rrdtool.
39Il g�n�re des graphiques des e-mails "propres", des spams et des virus class�s par nom.
40Pour cela il utilise les logs produits par amavis.
41
42Vous pouvez voir des exemples de graph produits � la page suivante :
43
44http://rekudos.net/amavis-stats/node/view/7
45
461.3 El�ments requis
47-------------------
48Amavis-stats a besoin d'un module perl fourni par le package rrdtool.
49
50Vous pouvez l'installer en utilisant les sources :
51 cd /usr/ports/net/rrdtool ; make all && make install
52
53Ou avec le package binaire :
54 pkg_add ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-4-stable/net/rrdtool-1.0.45.tgz
55
56note : merci d'utiliser un mirroir FreeBSD http://mirrorlist.freebsd.org/
57
58Un serveur apache fonctionnel est n�cessaire, avec php.
59
60Ma configuration :
61-apache2 avec mod_php4.
62-Postfix-2
63-Amavis-new
64-Clamav-0.65
65
66Je consid�re qu'Amavisd est configur� de la sorte :
67-$DO_SYSLOG = 1;
68
69note : ce param�tre est modifiable dans le fichier amavisd.conf, habituellement situ� dans /usr/local/etc
70
71Les logs peuvent �tre envoy�s dans un fichier s�par�, dans ce cas veillez � ce que l'utilisateur amavis puisse le lire.
72
732.Installation
74-------------
75
762.1 T�l�charger les sources et les installer
77--------------------------------------------
78T�l�chargez le tarball contenant la derni�re version :
79 wget http://rekudos.net/download/amavis-stats.tar.gz
80
81D�compressez-la :
82 tar xvzf amavis-stats.tar.gz
83 cd amavis-stats-xx
84
85Copiez le fichier amavis-stats dans /usr/local/sbin :
86 cp amavis-stats /usr/local/sbin
87
88Cr�ez les r�pertoires n�cessaires :
89 mkdir /usr/local/www/amavis-stats
90 mkdir -p /usr/local/lib/amavis-stats/img
91 ln -s /usr/local/lib/amavis-stats/img /usr/local/www/amavis-stats/img
92
93Copiez le fichier amavis-stats.php dans /usr/local/www/amavis-stats
94et faites un lien symbolique pointant vers un fichier index :
95 cp amavis-stats.php /usr/local/www/amavis-stats
96 cd /usr/local/www/amavis-stats && ln -s amavis-stats.php index.php
97
98note : index.php est un fichier d'index de r�pertoire,
99 d�fini par "DirectoryIndex" dans la configuration d'apache.
100
1012.2 Configurer d'Apache
102-----------------------
103Dans le fichier apache.conf fourni modifiez les chemins :
104
105Alias /amavis-stats /usr/local/www/amavis-stats
106
107<Directory /usr/local/www/amavis-stats>
108(...)
109
110Ajoutez le dans le httpd.conf d'Apache :
111 cat apache.conf >> /usr/local/etc/apache2/httpd.conf
112
113note : au lieu d'ajouter le fichier, vous pouvez l'inclure, renommez apache.conf en amavis.httpd.conf,
114 d�placez-le dans /usr/local/etc/apache2, puis ajoutez dans le fichier httpd.conf d'apache :
115 Include etc/apache2/amavis.httpd.conf
116
117Relancez apache
118 apachectl restart
119
1202.3 Mettre les droits ad�quats
121------------------------------
122Maintenant il faut l'utilisateur et le groupe sous lesquels tourne apache :
123 grep ^User\ /usr/local/etc/apache2/httpd.conf (2 espaces apr�s User\)
124 grep ^Group\ /usr/local/etc/apache2/httpd.conf (2 espaces apr�s User\)
125
126
127Je trouve www comme utilisateur et www comme groupe. Si vous n'avez pas la m�me chose,
128remplacez juste www:www par VotreUtilisateur:VotreGroupe dans la commande chown qui suit.
129
130Mettez les droits ad�quats sur le r�pertoire img avec ce que nous venons de trouver :
131(le fichier php va �crire les images dans ce r�pertoire, donc apache doit pouvoir y �crire)
132 chown www:www /usr/local/lib/amavis-stats/img
133
1342.4 Cr�er l'utilisateur amavis
135------------------------------
136Il faut un utilisateur amavis, qui va maintenir les statistiques � jour en lancant un cron job toutes les 5 minutes.
137
138Cr�ons le � l'aide de adduser ou de sysinstall :
139 useradd amavis
140
141Choisissez les valeurs par d�faut, pour son r�pertoire personnel mettez /usr/ocal/lib/amavis-stats.
142
143note : Le groupe additionnel wheel est indispensable pour avoir le droit de lire /var/log/maillog,
144 si toutefois amavisd est configur� pour envoyer ses log � syslog.
145
146Maintenant les droits pour les r�pertoires auxquels l'utilisateur amavis
147doit avoir acc�s en �criture durant le cron job :
148 chown amavis /usr/local/lib/amavis-stats
149 chown amavis /usr/local/www/amavis-stats
150
151
1522.5 Modifier les fichiers
153-------------------------
154Maintenant modifions certains fichiers :
155
156Editez le fichier amavis-stats et changez :
157ligne 79 en:
158 $lockfile = "/var/tmp/$pkg";
159ligne 80 en:
160 $libdir = "/usr/local/lib/$pkg";
161
162
163Editez le fichier amavis-stats.php et changez :
164ligne 282 en :
165 $as_libdir = "/usr/local/lib/amavis-stats";
166
1673. Etapes finales
168-----------------
169
1703.1 Test de l'installation
171--------------------------
172note : pour les instructions suivantes, mettez /var/log/maillog dans le cas ou amavisd est configur�
173 pour envoyer les logs � syslog, sinon mettez le fichier de log que vous avez d�fini.
174
175Logguez vous sous le compte amavis, et v�rifiez que tout se passe bien :
176 su amavis
177 /usr/local/sbin/amavis-stats /var/log/maillog
178
179Cette commande a cr�e des fichiers dans /usr/local/lib/amavis-stats :
180
1811.rrd amavis-stats.count amavis-stats.state
1822.rrd amavis-stats.names img
1833.rrd amavis-stats.seen
184
185
186A pr�sent regardez si la page php marche, allez � http://LeNomDeVotreServeur/amavis-stats avec votre naviguateur favori.
187Si vous avez des erreurs du genre "write denied" rev�rifiez les droits sur les r�pertoires
188(voir section 2.3 Mettre les droits ad�quats).
189
1903.2 Cron job
191-----------
192Mettons en place un cronjob pour que les donn�es soient mises � jour r�guli�rement :
193 crontab -e -u amavis
194
195Ecrivez ceci dans l'�diteur qui vient de s'ouvrir :
196
197*/5 * * * * /usr/local/sbin/amavis-stats /var/log/maillog 2>&1 > /dev/null
198
199
200
201Comme l'utilisateur amavis n'a pas besoin de se logguer, vous pouvez par mesure de s�curit� lui �ter son shell :
202 chsh -s /sbin/nologin amavis
203
204
2054. Divers
206---------
207
208Retours d'exp�rience
209--------------------
210Je suis int�r�ss� par tous vos retours d'exp�rience, bons ou mauvais concernant ce guide.
211N'h�sitez pas � me contacter si vous trouvez des erreurs ou si quelque chose manque.
212
213<loic AT dev DOT erodia DOT net>
214
215
216Historique
217----------
218v 1.0 (31 jan 2004) version initiale
219v 1.1 (2 fev 2004) modifications mineures, corrections version anglaise
220v 1.2 (6 fev 2004) mise en page, modifications mineures
221
222
223Copyright
224---------
225(c) 2004 Loic Pefferkorn
226
227Cette documentation peut �tre reproduite enti�rement ou partiellement, sans frais, mais doit suivre les restrictions suivantes :
228
229 *La mention copyright et cette notification de la license doivent �tre pr�serv�es sur toutes les copies.
230 *Toute traduction ou travail d�riv� doit �tre approuv� par l'auteur avant d'�tre distribu�.
231 *Si vous distribuez des parties de ce document, il vous faut mettre les instructions n�cessaires pour obtenir la version compl�te.
232 *Ce document est plac� sous la license GPL (voir http://www.gnu.org/licenses/gpl.html)
233
234Remerciements
235-------------
236-Aux auteurs d'amavis.
237-A Mark Lawrence, pour amavis-stats
238-A Bosco L Parisi, pour ses conseils avis�s.
239
240
241