1<!DOCTYPE refentry PUBLIC "-//Davenport//DTD DocBook V3.0//EN"> 2<refentry> 3 4<refmeta> 5<refentrytitle> 6<application>flow-tools</application> 7</refentrytitle> 8<manvolnum>1</manvolnum> 9</refmeta> 10 11<refnamediv> 12<refname> 13<application>flow-tools</application> 14</refname> 15<refpurpose> 16Tool set for working with NetFlow data. 17</refpurpose> 18</refnamediv> 19 20<refsect1> 21<title>DESCRIPTION</title> 22<para> 23Flow-tools is library and a collection of programs used to collect, 24send, process, and generate reports from NetFlow data. The tools 25can be used together on a single server or distributed to multiple 26servers for large deployments. The flow-toools library provides an 27API for development of custom applications for NetFlow export versions 281,5,6 and the 14 currently defined version 8 subversions. A Perl and 29Python interface have been contributed and are included in the distribution. 30</para> 31<para> 32Flow data is collected and stored by default in host byte order, yet 33the files are portable across big and little endian architectures. 34</para> 35<para> 36Commands that utilize the network use a localip/remoteip/port designation 37for communication. "localip" is the IP address the host will use as a 38source for sending or bind to when receiving NetFlow PDU's (ie the destination 39address of the exporter. Configuring the "localip" to 0 will force the kernel 40to decide what IP address to use for sending and listen on all IP addresses 41for receiving. "remoteip" is the destination IP address used for sending or 42the expected address of the source when receiving. If the "remoteip" is 430 then the application will accept flows from any source address. The "port" 44is the UDP port number used for sending or receiving. When using multicast 45addresses the localip/remoteip/port is used to represent the source, group, 46and port respectively. 47</para> 48<para> 49Flows are exported from a router in a number of different configurable 50versions. A flow is a collection of key fields and additional data. 51The flow key is {srcaddr, dstaddr, input, output, srcport, dstport, prot, 52ToS}. Flow-tools supports one export version per file. 53</para> 54<para> 55Export versions 1, 5, 6, and 7 all maintain {nexthop, dPkts, dOctets, 56First, Last, flags}, ie the next-hop IP address, number of packets, number 57of octets (bytes), start time, end time, and flags such as the TCP header 58bits. Version 5 adds the additional fields {src_as, dst_as, src_mask, 59dst_mask}, ie source AS, destination AS, source network mask, and 60destination network mask. Version 7 which is specific to the Catalyst 61switches adds in addition to the version 5 fields {router_sc}, which is 62the Router IP address which populates the flow cache shortcut in the 63Supervisor. Version 6 which is not officially supported by Cisco adds 64in addition to the version 5 fields {in_encaps, out_encaps, peer_nexthop}, 65ie the input and output interface encapsulation size, and the IP address 66of the next hop within the peer. Version 1 exports do not contain a 67sequence number and therefore should be avoided, although it is safe 68to store the data as version 1 if the additional fields are not used. 69</para> 70<para> 71Version 8 IOS NetFlow is a second level flow cache that reduces the 72data exported from the router. There are currently 11 formats, all 73of which provide {dFlows, dOctets, dPkts, First, Last} for the key 74fields. 75</para> 76<para> 77<literallayout> 78 8.1 - Source and Destination AS, Input and Output interface 79 8.2 - Protocol and Port 80 8.3 - Source Prefix and Input interface 81 8.4 - Destination Prefix and Output interface 82 8.5 - Source/Destination Prefix and Input/Output interface 83 8.9 - 8.1 + ToS 84 8.10 - 8.2 + ToS 85 8.11 - 8.3 + ToS 86 8.12 - 8.5 + ToS 87 8.13 - 8.2 + ToS 88 8.14 - 8.3 + ports + ToS 89</literallayout> 90</para> 91<para> 92Version 8 CatIOS NetFlow appears to be a less fine grained first level 93flow cache. 94</para> 95<para> 96<literallayout> 97 8.6 - Destination IP, ToS, Marked ToS, 98 8.7 - Source/Destination IP, Input/Output interface, ToS, Marked ToS, 99 8.8 - Source/Destination IP, Source/Destination Port, 100 Input/Output interface, ToS, Marked ToS, 101</literallayout> 102</para> 103<para> 104</para> 105<para> 106The following programs are included in the flow-tools distribution. 107</para> 108<para> 109<command>flow-capture</command> - Collect, compress, store, and 110manage disk space for exported flows from a router. 111</para> 112<para> 113<command>flow-cat</command> - Concatenate flow files. Typically flow files 114will contain a small window of 5 or 15 minutes of exports. Flow-cat 115can be used to append files for generating reports that span longer time 116periods. 117</para> 118<para> 119<command>flow-fanout</command> - Replicate NetFlow datagrams to unicast or 120multicast destinations. Flow-fanout is used to facilitate 121multiple collectors attached to a single router. 122</para> 123<para> 124<command>flow-report</command> - Generate reports for NetFlow data sets. 125Reports include source/destination IP pairs, source/destination AS, 126and top talkers. Over 50 reports are currently supported. 127</para> 128<para> 129<command>flow-tag</command> - Tag flows based on IP address or AS #. 130Flow-tag is used to group flows by customer network. The tags 131can later be used with flow-fanout or flow-report 132to generate customer based traffic reports. 133</para> 134<para> 135<command>flow-filter</command> - Filter flows based on any of the export 136fields. Flow-filter is used in-line with other programs 137to generate reports based on flows matching filter expressions. 138</para> 139<para> 140<command>flow-import</command> - Import data from ASCII or cflowd format. 141</para> 142<para> 143<command>flow-export</command> - Export data to ASCII or cflowd format. 144</para> 145<para> 146<command>flow-send</command> - Send data over the network using the NetFlow 147protocol. 148</para> 149<para> 150<command>flow-receive</command> - Receive exports using the NetFlow protocol 151without storing to disk like flow-capture. 152</para> 153<para> 154<command>flow-gen</command> - Generate test data. 155</para> 156<para> 157<command>flow-dscan</command> - Simple tool for detecting some types of network 158scanning and Denial of Service attacks. 159</para> 160<para> 161<command>flow-merge</command> - Merge flow files in chronoligical order. 162</para> 163<para> 164<command>flow-xlate</command> - Perform translations on some flow fields. 165</para> 166<para> 167<command>flow-expire</command> - Expire flows using the same policy of 168flow-capture. 169</para> 170<para> 171<command>flow-header</command> - Display meta information in flow file. 172</para> 173<para> 174<command>flow-split</command> - Split flow files into smaller files based on 175size, time, or tags. 176</para> 177 178</refsect1> 179 180<refsect1> 181<title>AUTHOR</title> 182<para> 183<author> 184<firstname>Mark</firstname> 185<surname>Fullmer</surname> 186</author> 187<email>maf@splintered.net</email> 188</para> 189<para> 190<command>flow-merge</command> by 191<author> 192<firstname>Larry</firstname> 193<surname>Lidz</surname> 194</author> 195<email>ellidz@eridu.uchicago.edu</email> 196</para> 197<para> 198Patches and other contribitions by a list too long to mention here. 199</para> 200<para> 201<command>flow-tools</command> is avalable at 202<ulink url="http://www.splintered.net/sw/flow-tools"></ulink>. 203</para> 204<para> 205A mailing list is maintained at <email>flow-tools@splintered.net</email> 206</para> 207</refsect1> 208 209<refsect1> 210<title>SEE ALSO</title> 211<para> 212<application>flow-capture</application>(1) 213<application>flow-cat</application>(1) 214<application>flow-dscan</application>(1) 215<application>flow-expire</application>(1) 216<application>flow-export</application>(1) 217<application>flow-fanout</application>(1) 218<application>flow-filter</application>(1) 219<application>flow-nfilter</application>(1) 220<application>flow-gen</application>(1) 221<application>flow-header</application>(1) 222<application>flow-import</application>(1) 223<application>flow-merge</application>(1) 224<application>flow-print</application>(1) 225<application>flow-receive</application>(1) 226<application>flow-report</application>(1) 227<application>flow-send</application>(1) 228<application>flow-split</application>(1) 229<application>flow-stat</application>(1) 230<application>flow-tag</application>(1) 231<application>flow-xlate</application>(1) 232</para> 233</refsect1> 234 235</refentry> 236