1$Id: README 736 2014-09-30 13:49:12Z bw $
2
3fwlogwatch is a security tool written in C by Boris Wesslowski originally for
4RUS-CERT. It is a packet filter/firewall/IDS log analyzer with support for a
5lot of log formats and has many analysis options. It also features realtime
6response capabilities and an interactive web interface.
7
8It is available at http://fwlogwatch.inside-security.de/
9
10
11FEATURES
12- General features:
13 - Can detect and process log entries in the following formats:
14 - Linux ipchains
15 - Linux netfilter/iptables
16 - Solaris/BSD/IRIX/HP-UX ipfilter
17 - BSD ipfw
18 - Cisco IOS
19 - Cisco PIX/FWSM/ASA
20 - NetScreen
21 - Elsa Lancom router
22 - Snort IDS
23 - Entries can be parsed from single, multiple and combined log files, the
24 parsers to be used can be selected.
25 - Gzip-compressed logs are supported transparently.
26 - Can separate recent from old entries and detects timewarps in log
27 files.
28 - Can recognize 'last message repeated' entries concerning the firewall.
29 - Integrated resolver for protocols, services and host names.
30 - Can do lookups in the GeoIP and whois databases.
31 - Own DNS and whois information cache and GNU adns support for faster
32 lookups.
33 - Hosts, networks, ports, chains and branches (targets) can be selected
34 or excluded as needed.
35 - Support for internationalization (available in english, german,
36 portuguese, simplified and traditional chinese, swedish and japanese).
37 - Supports IPv6 (currently only the netfilter parser, dns cache and web
38 interface make use of it).
39- Log summary mode:
40 - A lot of options to find and display relevant patterns in connection
41 attempts.
42 - Intelligent selection of certain fields (e.g. the host name column is
43 omitted and the host mentioned in the header of the summary if the log
44 is from a single host, the same happens with chains, targets and
45 interfaces).
46 - Output as plain text or HTML (W3C XHTML 1.1 with inline or linked CSS)
47 with limit and sort options.
48 - Can send summaries by email.
49- Realtime response mode:
50 - The program detaches and stays in background as a daemon.
51 - For ipchains setups detection of necessary rules with logging turned on
52 can be configured.
53 - Can catch up reading existing entries to provide up-to-date state
54 information from program start on.
55 - Response can be a notification (in form of a log file entry, an email,
56 a remote winpopup message or whatever you can put into a shell script),
57 or a customizable firewall modification.
58 - The included response script adds a new chain for fwlogwatch to
59 ipchains or netfilter setups and attackers are blocked with new
60 firewall rules.
61 - Supports trusted hosts (anti-spoofing).
62 - The current status of the program can be followed and controlled
63 through a web interface.
64
65The commented configuration file supports and explains all options and will
66get you started quickly. Please read the man page for details on the
67command line options.
68
69
70PARSER NOTES
71 - Cisco PIX/ASA support focuses on denied packets, knows some permitted
72 packet log entries and ignores all other kinds of log entries. It
73 expects log entries as from a syslog host. If your PIX/ASA uses names
74 or objects in the log you can use the script asa-hosts.sh from the
75 contrib directory to extract them from a saved PIX/ASA configuration
76 and format them as a hosts file, which you can use to initialize the
77 dns cache. If fwlogwatch is not able to resolve the names it will
78 discard the corresponding log entries.
79 - The ipfilter parser does not support logs with resolved service names
80 and tcp/udp entries without ports.
81 - The Snort parser does not analyze portscan entries.
82 - The NetScreen parser does not recognize packet-filter-unrelated entries
83 and some icmp types which the NetScreen does not seem to recognize
84 itself...
85
86
87INSTALLATION
88- General
89 The Makefile assumes you use Linux, to compile on Mac OS X, Solaris,
90 OpenBSD, FreeBSD or IRIX look for the corresponding lines at the top of
91 the Makefile. fwlogwatch may also be compiled and run on Windows with
92 help of cygwin or mingw. You may want to have a look at main.h if you
93 want to change some default values.
94
95 flex is required to build fwlogwatch. Besides that a simple 'make' should
96 be enough to obtain a working binary. If your make (like the one on
97 OpenBSD) thinks flex can only produce files called lex.yy.c type make
98 several times, after all parsers are generated linking will work.
99
100 If you use the realtime response mode you will need to install fwlogwatch
101 with superuser permissions for certain configurations. If all you need is
102 read access to the system's standard log file (e.g. /var/log/messages)
103 you can use group permissions. You can also bind a nonprivileged port for
104 the status server. 'make install' will install the binary (in
105 /usr/local/sbin) and the man page, 'make install-config' will also
106 install a sample configuration file in /etc.
107
108- Zlib/Gettext/GNU adns/GeoIP
109 If you define HAVE_ZLIB, HAVE_GETTEXT, HAVE_ADNS and HAVE_GEOIP
110 fwlogwatch will be compiled with zlib, gettext, adns and GeoIP support.
111 The output of fwlogwatch -V will contain an overview of the compiler
112 options used.
113
114 You will need the zlib compression library version 1.0.9 or newer and
115 it's header files to be able to use zlib support.
116
117 To enable the internationalization support you will need to have the
118 gettext package (sometimes with it's surroundings libtool, GNU m4,
119 autoconf and automake) installed.
120
121 The GNU adns library and header files must be installed to build fwlogwatch
122 with support for faster, asynchronous DNS lookups.
123
124 The GeoIP legacy library and header files are required to compile and use
125 fwlogwatch with IP-to-country-lookup support. The current GeoLite Country
126 databases for IPv4 and IPv6 can be downloaded from
127 http://dev.maxmind.com/geoip/legacy/geolite/
128
129- Linux
130 For ipchains you will need at least kernel 2.2.10 (which you should have
131 updated for security reasons anyway), kernel versions before 2.2.10 don't
132 log enough.
133
134
135BASICS (with examples for iptables)
136- You should use a whitelist (meaning your firewall only allows connections
137 that are explicitly permitted and should be as specific as possible), so
138 create a packet filter with ACCEPT rules for all connections you need.
139
140- Your default policies can be DROP if you want your firewall to block all
141 connections in case all rules are deleted:
142
143 iptables -P INPUT DROP
144 iptables -P FORWARD DROP
145
146 Otherwise you should add a DROP rule at the end of all chains:
147
148 iptables -A INPUT -j DROP
149 iptables -A FORWARD -j DROP
150
151- Before those DROP rules (or the end of chains with DROP policy) you
152 should have a LOG rule with some descriptive text like the context and
153 chain name:
154
155 iptables -A INPUT -j LOG --log-prefix "fw input drop "
156 iptables -A FORWARD -j LOG --log-prefix "fw forward drop "
157
158 So the last 2 lines in your INPUT, FORWARD and custom chains should be
159
160 pkts bytes target prot opt in out source destination
161 0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 LOG flags 0 level 4 prefix `fw forward drop `
162 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
163
164- If you have a persistent log spammer drop his packets by inserting a DROP
165 rule before the logging rule above (or use the block response mode).
166
167
168CONFIGURATION AND EXAMPLES
169- You should make a configuration file for each function you want, look at
170 the included sample file, it should be easy to adapt to your needs.
171
172- Command line example:
173 (The options are in the order they are mentioned in the text)
174 If you want very verbose generation of a HTML summary in the file
175 'log.html' of all packet filter entries at most one day old representing
176 at least two connection attempts with output including start and end
177 timestamps, time intervals, resolved IP addresses and service names and
178 with connections separated by protocol, source and destination ports and
179 TCP options using the files that match the expression "messages*.gz" as
180 input you would use the command
181
182 fwlogwatch -v -v -w -o log.html -l 1d -m 2 -t -e -z -n -N -p -s -d -y messages*.gz
183
184- If you want to use fwlogwatch as a CGI, e.g. to have a quick look at
185 what happened in the last hour when you get a notification:
186 Copy the file fwlogsummary_small.cgi from the contrib directory to a
187 place where your web server can execute it (fwlogwatch must be reachable
188 and have enough permissions to read the log file).
189
190- A script for generation of 8 general summaries (fwlogsummary.cgi) is also
191 included. You can use it as above or if you comment a few lines also on
192 the command line or from cron. It will use the output directory
193 /var/www/html/fwlogwatch as default. Look at the index.html file to
194 select the level of detail you want.
195
196- Contrib also contains a web frontend written in PHP (fwlogwatch.php) to
197 apply fwlogwatch to selected files and test different options. Change the
198 header of the script to adapt it to your system (e.g. prefix of the log
199 file names, location of fwlogwatch).
200
201- Sample init files (for Red Hat and openSUSE Linux systems) to start
202 fwlogwatch in realtime response mode at system start are also included.
203
204- You might want to replace your /etc/services file by the one supplied
205 with nmap (http://www.insecure.org/nmap/), a lot more services will be
206 recognized, you can also add the ICMP types (don't assign one to type 0
207 since it is the ipchains default). The same applies to /etc/protocols,
208 you can use RFC 1700 (Assigned Numbers) to extend your version.
209
210
211FEEDBACK
212 If you find a bug or have an idea for a new feature please send an email
213 to Boris Wesslowski <bw@inside-security.de>.
214
215 If you want to report a parser problem or submit unrecognized entries
216 please use the unrecognized entry submission page:
217 http://fwlogwatch.inside-security.de/unrecognized.php
218