NOTE: this documents the original intent behind libwiretap.  Currently,
it is being developed solely as a library for reading capture files,
rather than packet capture.  The list of file formats is also
out-of-date.

Wiretap is a library that is being developed as a future replacement for
libpcap, the current standard Unix library for packet capturing. Libpcap
is great in that it is very platform independent and has a wonderful
BPF optimizing engine. But it has some shortcomings as well. These
shortcomings came to a head during the development of Wireshark
(https://www.wireshark.org/), a packet analyzer. As such, I began developing
wiretap so that:

1. The library can easily be amended with new packet filtering objects.
Libpcap is very TCP/IP-oriented. I want to filter on IPX objects, SNA objects,
etc. I also want any decent programmer to be able to add new filters to the
library.

2. The library can read file formats from many packet-capturing utilities.
Libpcap only reads Libpcap files.

3. The library can capture on more than one network interface at a time, and
save this trace in one file.

4. Network names can be resolved immediately after a trace and saved in the
trace file. That way, I can ship a trace of my firewall-protected network to a
colleague, and he'll see the proper hostnames for the IP addresses in the
packet capture, even though he doesn't have access to the DNS server behind my
LAN's firewall.

5. I want to look into the possibility of compressing packet data when saved
to a file, like Sniffer.

6. The packet-filter can be optimized for the host OS. Not all OSes have BPF;
SunOS has NIT and Solaris has DLPI, which both use the CMU/Stanford
packet-filter pseudomachine. RMON has another type of packet-filter syntax
which we could support.

Wiretap is very good at reading many file formats, as per #2
above. Wiretap has no filter capability at present; it currently doesn't
support packet capture, so it wouldn't be useful there, and filtering
when reading a capture file is done by Wireshark, using a more powerful
filtering mechanism than that provided by BPF.


File Formats
============

Libpcap
-------
The "libpcap" file format was determined by reading the "libpcap" code;
wiretap reads the "libpcap" file format with its own code, rather than
using the "libpcap" library's code to read it.

Sniffer (compressed and uncompressed)
-------
The uncompressed Sniffer format is documented in the Sniffer manual.
Unfortunately, Sniffer manuals tend to document only the format for
the Sniffer model they document. Token-Ring and ethernet seems to work
well, though.  If you have an ATM Sniffer file, both Guy and Gilbert
would be *very* interested in receiving a sample. (see 'AUTHORS' file
for our e-mail addresses).

LANalyzer
---------
The LANalyzer format is available from http://www.novell.com. Search
their knowledge base for "Trace File Format".

Network Monitor
---------------
Microsoft's Network Monitor file format is supported, at least under
Ethernet and token-ring. If you have capture files of other datalink
types, please send them to Guy.

"snoop"
-------
The Solaris 2.x "snoop" program's format is documented in RFC 1761.

"iptrace"
---------
This is the capture program that comes with AIX 3.x and 4.x.  AIX 3 uses
the iptrace 1.0 file format, while AIX4 uses iptrace 2.0.  iptrace has
an undocumented, yet very simple, file format.  The interesting thing
about iptrace is that it will record packets coming in from all network
interfaces; a single iptrace file can contain multiple datalink types.

Sniffer Basic (NetXRay)/Windows Sniffer Pro
-------------------------------------------
Network Associates' Sniffer Basic (formerly NetXRay from Cinco Networks)
file format is now supported, at least for Ethernet and token-ring.
Network Associates' Windows Sniffer Pro appears to use a variant of that
format; it's supported to the same extent.

RADCOM WAN/LAN Analyzers
------------------------
Olivier Abad has added code to read Ethernet and LAPB captures from
RADCOM WAN/LAN Analyzers (see https://web.archive.org/web/20031231213434/http://www.radcom-inc.com/).

Lucent/Ascend access products
-----------------------------
Gerald

HP-UX nettl
-----------
nettl is used on HP-UX to trace various streams based subsystems.  Wiretap
can read nettl files containing IP frames (NS_LS_IP subsystem) and LAPB
frames (SX25L2 subsystem). It has been tested with files generated on
HP-UX 9.04 and 10.20.
Use the following commands to generate a trace :
# IP capture. 0x30000000 means PDU in and PDU out :
nettl -tn 0x30000000 -e NS_LS_IP -f tracefile
# X25 capture. You must specify an interface :
nettl -tn 0x30000000 -e SX25l2 -d /dev/x25_0 -f tracefile
# stop capture. subsystem is NS_LS_IP or SX25L2 :
nettl -tf -e subsystem

One may be able to specify "-tn pduin pduout" rather than
"-tn 0x30000000"; the nettl man page for HP-UX 10.30 implies that it
should work.

There is also basic support for nettl files containing NS_LS_DRIVER,
NS_LS_TCP, NS_LS_UDP, NS_LS_LOOPBACK, unknown type 0xb9, and NS_LS_ICMP.
However, NS_LS_ICMP will not be decoded since WTAP lacks a raw ICMP
encapsulation type.


Toshiba ISDN Router
-------------------
An under-documented command that the router supports in a telnet session
is "snoop" (not related to the Solaris "snoop" command). If you give
it the "dump" option (either by letting "snoop" query you for its next
argument, or typing "snoop dump" on the command line), you'll get a hex
dump of all packets across the router (except of your own telnet session
-- good thinking Toshiba!). You can select a certain channel to sniff
(LAN, B1, B2, D), but the default is all channels.  You save this hex
dump to disk with 'script' or by 'telnet | tee'. Wiretap will read the
ASCII hex dump and convert it to binary data.

ISDN4BSD "i4btrace" utility
---------------------------
Bert Driehuis

Cisco Secure Intrusion Detection System iplogging facility
-----------------------------------------------------------
Mike Hall

pppd logs (pppdump-format files)
--------------------------------
Gilbert

VMS TCPTRACE
------------
Compaq VMS's TCPIPTRACE format is supported.  This is the capture program
that comes with TCP/IP or UCX as supplied by Compaq or Digital Equipment
Corporation.

Under UCX 4.x, it is invoked as TCPIPTRACE.  Under TCPIP 5.x, it is invoked
as TCPTRACE.

TCPTRACE produces an ascii text based format, that has changed slightly over
time.

DBS Etherwatch (text format)
----------------------------
Text output from DBS Etherwatch is supported.  DBS Etherwatch is available
from: https://web.archive.org/web/20070612033348/http://www.users.bigpond.com/dbsneddon/software.htm.

Catapult DCT2000 (.out files)
-----------------------------
DCT2000 test systems produce ascii text-based .out files for ports
that have logging enabled. When being read, the data part of the message is
prefixed with a short header that provides some context (context+port,
direction, original timestamp, etc).

You can choose to suppress the reading of non-standard protocols
(i.e. messages between layers rather than the well-known link-level protocols
usually found on board ports).


Gilbert Ramirez <gram@alumni.rice.edu>
Guy Harris <guy@alum.mit.edu>

STANAG 4607
-----------
Initial support for the STANAG 4607 protocol.  Documentation at:
https://web.archive.org/web/20130223054955/http://www.nato.int/structur/AC/224/standard/4607/4607.htm

AMC: Wireless Analyzer Captured Data (AirMagnet)
------------------------------------------------
This is just a braindump from looking at some Airmagnet capture files,
in one case having a look at a decoded display in Airmagnet itself.
Lots of things are still unknown.
This is NOT the intention to write a file importer in the foreseeable
future.

Exact timestamp decoding still unknown:
From a different decoded example file:
06 51 49 1b     Timestamp  (105990427)  =  03:30:27.497544
06 51 4a cd     Timestamp  (105990861)  =  03:30:27.497978  (+434)
06 51 4b ce     Timestamp  (105991118)  =  03:30:27.498235  (+257)

Timestamps this file:
Frame1: 15a5 2fb4	363147188	-
Frame2: 15a5 3a8e	363149966	+ 1778
Frame3: 15a5 3c50	363150416	+  550
Frame4: 15a5 487d	363153533
Frame5: 15a5 49d9	363153881
Frame6: 15a5 4bcf	363154383
Frame7: 15a5 53da	363156442
Frame8: 15a5 59e4	363157988
Frame9: 15cc 9da8	365731240
FrameA: 15db dd60	366730592
FrameB: 15dc 04f6	366740726
FrameC: 15df 5d29	366959913

Unknown stuff:
Header: 0000 0000 0002 | 3b93 d886
Frame1: da9d | 0000 0000 0002
Frame2: bf9c | 0000 0000 0002
Frame3: d59c | 0000 0000 0002
Frame4: c09c | 0000 0000 0002
Frame5: c09c | 0000 0000 0002
Frame6: d69c | 0000 0000 0002
Frame7: d79c | 0000 0000 0002
Frame8: c09c | 0000 0000 0002
Frame9: d79c | 0000 0000 0002
FrameA: d89c | 0000 0000 0002
FrameB: d69d | 0000 0000 0002
FrameC: d79c | 0000 0000 0002

Headerstructure:
 6 Bytes: 'N2GINC'
44 Bytes: unknown (0x0)
 6 Bytes: 'N2GINC'
 2 Bytes: unknown (0x0)
 2 Bytes: #Frames (BE)
 2 Bytes: unknown (0x0)
 2 Bytes: unknown (0x0002)
 4 Bytes: Timestamp
 4 Bytes: unknown (Timestamp date part ?)
28 Bytes: unknown (0x0)
==================
Total: 100 Bytes

Recordstructure:
 2 Bytes: unknown
 2 Bytes: Bytes "on wire" (BE)
 2 Bytes: LL Bytes in capturefile (BE)
 4 Bytes: unknown (0x0) (02 00 00 00 = Frame displayed in RED)
 2 Bytes: unknown (0x0002)
 4 Bytes: Timestamp
 1 Byte:  DataRate ((AND 7f) / 2 in Mbit/s)
 1 Byte:  Channel (1-13: Chan 1-13, 15: Chan 36, 16: Chan 40, ..., 19: Chan 52, ...)
 1 Byte:  SignalStrength (%)
 1 Byte:  NoiseLevel (%)
LL Bytes: Capturedata
 6 Bytes: unknown
 4 Bytes: 0x33333333
(1 Byte:  0x33) in case LL is an odd number

Filelength: 0x57e Bytes
12 Frames
============== Header ===================
0000000: 4e32 4749 4e43 0000 0000 0000 0000 0000
0000010: 0000 0000 0000 0000 0000 0000 0000 0000
0000020: 0000 0000 0000 0000 0000 0000 0000 0000
0000030: 0000 4e32 4749 4e43 0000 000c 0000 0002
0000040: 1545 5198 3b93 d886 0000 0000 0000 0000
0000050: 0000 0000 0000 0000 0000 0000 0000 0000
0000060: 0000 0000
============== Frame 1 ==================
Length: 0x36
-------- ???
0000064: da9d
0000066: 0026 0026
000006a: 0000 0000 0002
0000070: 15a5 2fb4		Timestamp
0000074: 02			DataRate
0000075: 06			Channel
0000076: 64			SignalStrength (%)
0000077: 01			NoiseLevel (%)
-------- ieee 802.11
0000078: b000			Type/Subtype/Frame Control
000007a: 3a01			Duration
000007c: 0040 9647 71a3		Destaddr
0000082: 0040 9631 d395		Sourceaddr
0000088: 0040 9647 71a3		BSS Id
000008e: b000			Fragment nr/Seq nr
-------- ???
0000090: 0000 0100 0000 3333 3333
============== Frame 2===================
Length: 0x36
-------- ???
000009a: bf9c 0026 0026
00000a0: 0000 0000 0002 15a5 3a8e 1606 3c00
-------- ieee 802.11
00000ae: b000			Type/Subtype/Frame Control
00000b0: 7500
00000b2: 0040 9631 d395
00000b8: 0040 9647 71a3
00000be: 0040 9647 71a3
00000c4: 806d
-------- ???
00000c6: 0000 0200 0000 3333 3333
============== Frame 3 ==================
Length: 0x62
-------- ???
00000d0: d59c 0051 0051
00000d6: 0000 0000 0002 15a5 3c50 0206 6400
-------- ieee 802.11
00000e4: 0000
00000e6: 3a01
00000e8: 0040 9647 71a3
00000ee: 0040 9631 d395
00000f4: 0040 9647 71a3
00000fa: c000
-------- ???
00000fc: 2100 c800
0000100: 0005 616c 616d 6f01 0402 040b 1685 1e00
0000110: 016d 0d1f 00ff 0318 0049 6e73 7472 7563
0000120: 746f 7200 0000 00
-------- ???
0000127: 00 0000 0000 2133 3333 3333
================= Frame 4 ===============
Length: 0x68
-------- ???
0000132: c09c 0058 0058
0000138: 0000 0000 0002 15a5 487d 1606 3e00
-------- ieee 802.11
000014a: 1000
000014c: 7500
000014e: 0040 9631 d395
0000154: 0040 9647 71a3
000015a: 0040 9647 71a3
000015c: 906d
-------- ???
000015e: 2100
0000160: 0000 1d00 0104 8284 8b96 851c 0000 4c0d
0000170: 0000 0000 0100 416c 616d 6f20 436c 6173
0000180: 7372 6f6f 6d20 0002 880c 80f3 0300 8137
-------- ???
0000190: 0300 0000 0000 3333 3333
=================== Frame 5 =============
-------- ???
000019a: c09c 005a 005a
00001a0: 0000 0000 0002 15a5 49d9 1606 3e00
-------- ieee 802.11
00001ae: 0802
00001b0: 7500 0040 9631 d395 0040 9647 71a3 0040
00001c0: 9647 71a3 a06d aaaa 0300 4096 0000 0032
00001d0: 4001 0040 9631 d395 0040 9647 71a3 0100
00001e0: 0000 0000 0000 0000 0000 0000 0000 0000
00001f0: 0000 0000 0000 0000 0000
-------- ???
00001fa: 0000 0000 0000 3333 3333
=================== Frame 6 =============
-------- ???
0000204: d69c 0072 0072
000020a: 0000 0000 0002 15a5 4bcf 0206 6400
-------- ieee 802.11
0000218: 0801 3a01 0040 9647
0000220: 71a3 0040 9631 d395 0040 9647 71a3 d000
0000230: aaaa 0300 4096 0000 004a 4081 0040 9647
0000240: 71a3 0040 9631 d395 016d 0004 1900 0040
0000250: 9647 71a3 0000 0000 0000 0000 0000 0000
0000260: 0000 001e 496e 7374 7275 6374 6f72 0000
0000270: 0000 0000 0000 0000 0000 0000
-------- ???
000027c: 0000 0000 0000 3333 3333
=================== Frame 7 =============
-------- ???
0000286: d79c 0039 0039
000028c: 0000 0000 0002 15a5 53da 0206 6400
-------- ieee 802.11
000029a: 0801 3a01 0040
00002a0: 9647 71a3 0040 9631 d395 0040 9647 71a3
00002b0: e000 aaaa 0300 4096 0000 0011 4001 0040
00002c0: 9647 71a3 00
00002c5: 40 9631 d395 0133 3333 3333
====================== Frame 8 ==========
-------- ???
00002d0: c09c 0072 0072
00002d6: 0000 0000 0002 15a5 59e4 1606 3e00
-------- ieee 802.11
00002e4: 0802 7500 0040 9631 d395 0040
00002f0: 9647 71a3 0040 9647 71a3 b06d aaaa 0300
0000300: 4096 0000 004a 4081 0040 9631 d395 0040
0000310: 9647 71a3 014c 030b 1500 0000 0000 0000
0000320: 0000 0000 0000 0002 0000 0a00 0001 0000
0000330: 416c 616d 6f20 436c 6173 7372 6f6f 6d00
0000340: 0000 0000 0000 0000
0000348: 0000 0000 2200 3333 3333
=================== Frame 9 =============
-------- ???
0000352: d79c 0170 00a0
0000358: 0020 0000 0002 15cc 9da8 1606 6400
-------- ieee 802.11
0000366: 0801 7500 0040 9647 71a3
0000370: 0040 9631 d395 ffff ffff ffff f000 aaaa
0000380: 0300 0000 0800 4500 0148 42a1 0000 8011
0000390: f704 0000 0000 ffff ffff 0044 0043 0134
00003a0: d991 0101 0600 7728 b62a 0000 0000 0000
00003b0: 0000 0000 0000 0000 0000 0000 0000 0040
00003c0: 9631 d395 0000 0000 0000 0000 0000 0000
00003d0: 0000 0000 0000 0000 0000 0000 0000 0000
00003e0: 0000 0000 0000 0000 0000 0000 0000 0000
00003f0: 0000 0000 0000 0000
-------- ???
00003f8: 0000 0000 0000 3333 3333
=================== Frame A =============
-------- ???
0000402: d89c 0182 00a0
0000408: 0020 0000 0002 15db dd60 1606 6400
-------- ieee 802.11
0000416: 0801 7500
000041a: 0040 9647 71a3
0000420: 0040 9631 d395
0000426: ffff ffff ffff
000042c: 0001
-------- LLC
000042e: aaaa 0300 0000 0800
-------- IP
0000436: 4500 015a 42a3 0000 8011 f6f0 0000 0000 ffff ffff
-------- UDP
000044a: 0044 0043 0146 57bc
-------- DHCP
0000452: 0101 0600 7728 b62a 0000 0000 0000
0000460: 0000 0000 0000 0000 0000 0000 0000 0040
0000470: 9631 d395 0000 0000 0000 0000 0000 0000
0000480: 0000 0000 0000 0000 0000 0000 0000 0000
0000490: 0000 0000 0000 0000 0000 0000 0000 0000
00004a0: 0000 0000 0000 0000
-------- ???
00004a8: 0000 0000 0000 3333 3333
=================== Frame B =============
-------- ???
00004b2: d69d 0056 0056
00004b8: 0000 0000 0002 15dc 04f6 1606 6401
-------- ieee 802.11
00004c6: 0801
00004c8: 7500
00004ca: 0040 9647 71a3
00004d0: 0040 9631 d395
00004d6: ffff ffff ffff
0000edc: 1001
-------- LLC
00004de: aaaa 0300 0000 0806
-------- ARP
00004e6: 0001 0800 0604 0001
00004ee: 0040 9631 d395 0a00 0065
00004f8: 0000 0000 0000 0a00 0065
-------- ???
0000502: 7b00 e097 7b00 e097 7b00 e097
000050e: 7b00 e097 7b00 3333 3333
=================== Frame C =============
-------- ???
0000518: d79c 0056 0056
000051e: 0000 0000 0002 15df 5d29 1606 6400
-------- ieee 802.11
000052c: 0801
000052e: 7500
0000530: 0040 9647 71a3
0000536: 0040 9631 d395
000053a: ffff ffff ffff
0000540: 2001
-------- LLC
0000542: aaaa 0300 0000 0806
-------- ARP
000054a: 0001			Hw
000054c: 0800			Protocol
0000550: 06			Hw-Size
0000551: 04			Protocolsize
0000552: 0001			Opcode
0000554: 0040 9631 d395		Sender MAC
000055a: 0a00 0065		Sender IP
000055e: 0000 0000 0000		Destination MAC
0000564: 0a00 0065		Destination IP
-------- ???
0000568: 0000 8602 0000 ffff ffff 0cc3
0000574: 4b82 58a1 1d82 3333 3333
=========================================
000057e: EOF

This is a very quick and very dirty guide to adding support for new
capture file formats.  If you see any errors or have any improvements,
submit patches - free software is a community effort....

To add the ability to read a new capture file format, you have to:

	add a new WTAP_FILE_ value for the file type to
	"wiretap/wtap.h";

	write an "open" routine that can read the beginning of the
	capture file and figure out if it's in that format or not,
	either by looking at a magic number at the beginning or by using
	some form of heuristic to determine if it's a file of that type
	(if the file format has a magic number, that's what should be
	used);

	write a "read" routine that can read a packet from the file and
	supply the packet length, captured data length, time stamp, and
	packet pseudo-header (if any) and data, and have the "open"
	routine set the "subtype_read" member of the "wtap" structure
	supplied to it to point to that routine;

	write a "seek and read" routine that can seek to a specified
	location in the file for a packet and supply the packet
	pseudo-header (if any) and data, and have the "open" routine set
	the "subtype_seek_read" member of the "wtap" structure to point
	to that routine;

	write a "close" routine, if necessary (if, for example, the
	"open" routine allocates any memory), and set the
	"subtype_close" member of the "wtap" structure to point to it,
	otherwise leave it set to NULL;

	add a pointer to the "open" routine to the "open_routines_base[]"
	table in "wiretap/file_access.c" - if it uses a magic number, put
	it in the first section of that list, and, if it uses a heuristic,
	put it in the second section, preferably putting the heuristic
	routines for binary files before the heuristic routines for text
	files;

	add an entry for that file type in the "dump_open_table_base[]" in
	"wiretap/file_access.c", giving a descriptive name, a short name
	that's convenient to type on a command line (no blanks or capital
	letters, please), common file extensions to open and save, a flag
	if it can be compressed with gzip (currently unused) and pointers
	to the "can_write_encap" and "dump_open" routines if writing that
	file is supported (see below), otherwise just null pointers.

Wiretap applications typically first perform sequential reads through
the capture file and may later do "seek and read" for individual frames.
The "read" routine should set the variable data_offset to the byte
offset within the capture file from which the "seek and read" routine
will read.  If the capture records consist of:

	capture record header
	pseudo-header (e.g., for ATM)
	frame data

then data_offset should point to the pseudo-header.  The first
sequential read pass will process and store the capture record header
data, but it will not store the pseudo-header.  Note that the
seek_and_read routine should work with the "random_fh" file handle
of the passed in wtap struct, instead of the "fh" file handle used
in the normal read routine.

To add the ability to write a new capture file format, you have to:

	add a "can_write_encap" routine that returns an indication of
	whether a given packet encapsulation format is supported by the
	new capture file format;

	add a "dump_open" routine that starts writing a file (writing
	headers, allocating data structures, etc.);

	add a "dump" routine to write a packet to a file, and have the
	"dump_open" routine set the "subtype_write" member of the
	"wtap_dumper" structure passed to it to point to it;

	add a "dump_close" routine, if necessary (if, for example, the
	"dump_open" routine allocates any memory, or if some of the file
	header can be written only after all the packets have been
	written), and have the "dump_open" routine set the
	"subtype_close" member of the "wtap_dumper" structure to point
	to it;

	put pointers to the "can_write_encap" and "dump_open" routines
	in the "dump_open_table_base[]" entry for that file type.

In the wiretap directory, add your source file to CMakelists.txt.