• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..03-May-2022-

.gitignoreH A D04-Mar-20216 21

MakefileH A D04-Mar-2021146 118

README.kernelH A D04-Mar-20215.5 KiB146118

README.nhrpdH A D04-Mar-20215 KiB138102

debug.hH A D04-Mar-20211.2 KiB4433

linux.cH A D04-Mar-20213.2 KiB161126

list.hH A D04-Mar-20216.1 KiB210161

netlink.hH A D04-Mar-2021879 2713

netlink_arp.cH A D04-Mar-20218.2 KiB320261

netlink_gre.cH A D04-Mar-20213.6 KiB153119

nhrp_cache.cH A D04-Mar-202112.1 KiB454376

nhrp_errors.cH A D04-Mar-20211.3 KiB4418

nhrp_errors.hH A D04-Mar-20211 KiB338

nhrp_event.cH A D04-Mar-20216.8 KiB299247

nhrp_interface.cH A D04-Mar-202110.1 KiB411326

nhrp_main.cH A D04-Mar-20213.2 KiB167121

nhrp_nhs.cH A D04-Mar-202111.3 KiB417333

nhrp_packet.cH A D04-Mar-20218.3 KiB346279

nhrp_peer.cH A D04-Mar-202126.7 KiB1,038858

nhrp_protocol.hH A D04-Mar-20213.5 KiB13091

nhrp_route.cH A D04-Mar-20219.4 KiB391323

nhrp_shortcut.cH A D04-Mar-202114 KiB503405

nhrp_vc.cH A D04-Mar-20214.9 KiB226176

nhrp_vty.cH A D04-Mar-202129.5 KiB1,152975

nhrpd.hH A D04-Mar-202112.5 KiB448362

os.hH A D04-Mar-2021299 86

reqid.cH A D04-Mar-20211.1 KiB5345

subdir.amH A D04-Mar-2021812 4642

vici.cH A D04-Mar-202113.6 KiB570487

vici.hH A D04-Mar-2021414 2521

zbuf.cH A D04-Mar-20214.2 KiB233180

zbuf.hH A D04-Mar-20214.4 KiB200157

znl.cH A D04-Mar-20213.4 KiB171128

znl.hH A D04-Mar-20211.1 KiB3014

README.kernel

1KERNEL REQUIREMENTS
2===================
3
4The linux kernel has had various major regressions, performance
5issues and subtle bugs (especially in pmtu). Here is a short list
6of some -stable kernels and the first point release that is supposedly
7working well with opennhrp/dmvpn:
8  3.12.8 or later
9  3.14.54 or later
10  3.18.22 or later[1]
11
12[1] But you need to apply the following two backported commits:
13    3cdaa5be9e ipv4: Don't increase PMTU with Datagram Too Big message
14    cb6ccf09d6 route: Use ipv4_mtu instead of raw rt_pmtu
15
16See below for list of known issues in various kernel versions.
17
18Kernels earlier than 3.12 need CONFIG_ARPD enabled in the configuration.
19Many distributions do not enable it by default, and you may need to
20compile your own kernel.
21
22KERNEL BUGS
23===========
24
25DMVPN and mGRE support in the kernel has been brittle. There are various
26regressions in multiple kernel versions.
27
28This list tries to collect them to one source of information:
29
30- forward pmtu is disabled intentionally (but tunnel devices rely on it)
31  Broken since 3.14-rc1:
32    commit "ipv4: introduce ip_dst_mtu_maybe_forward and protect forwarding path against pmtu spoofing"
33  Workaround:
34    Set sysctl net.ipv4.ip_forward_use_pmtu=1
35    (Should fix kernel to have this by default on for tunnel devices)
36
37- subtle path mtu mishandling issues
38  Broken since (uncertain)
39  Fixed in 4.1-rc2:
40    commit "ipv4: Don't increase PMTU with Datagram Too Big message."
41    commit "route: Use ipv4_mtu instead of raw rt_pmtu"
42
43- fragmentation of large packets inside tunnel not working
44  Broken since 3.11-rc1
45    commit "ip_tunnels: Use skb-len to PMTU check."
46  Fixed in 3.14.54, 3.18.22, 4.1.9, 4.2-rc3
47    commit "ip_tunnel: fix ipv4 pmtu check to honor inner ip header df"
48
49- ipsec will crash during xfrm gc
50  Broke since 3.15-rc1
51    commit "flowcache: Make flow cache name space aware"
52  Fixed in 3.18.10, 4.0
53    commit "flowcache: Fix kernel panic in flow_cache_flush_task"
54
55- TSO on GRE tunnels failed, and resulted in very slow performance
56  Broke since 3.14.24, 3.18-rc3
57    commit "gre: Use inner mac length when computing tunnel length"
58  Fixed in 3.14.30, 3.18.4
59    commit "gre: fix the inner mac header in nbma tunnel xmit path"
60    commit "gre: Set inner mac header in gro complete"
61
62- NAPI GRO handling was broken; causing immediate crash (32-bit only?)
63  Broken since 3.13-rc1
64    commit "net: gro: allow to build full sized skb"
65  Fixed 3.14.5, 3.15-rc7
66    commit "net: gro: make sure skb->cb[] initial content has not to be zero"
67
68- ip_gre dst caching broke NBMA GRE tunnels
69  Broken since 3.14-rc1
70  Fixed in 3.14.5, 3.15-rc6
71    commit "ipv4: ip_tunnels: disable cache for nbma gre tunnels"
72
73- Few packets can be lost when neighbor entry is in NUD_PROBE state,
74  and there is continuous traffic to it.
75  Broken since dawn of time
76  Fixed in 3.15-rc1
77    commit "neigh: probe application via netlink in NUD_PROBE"
78
79- GRO was implemented for GRE, but the hw capabilities were not updated
80  correctly. In practice forwarding from non-GRE (physical) interface
81  to GRE interface with gro/gso/tx offloads enabled (also on the target
82  interface) does not work properly.
83  Broken around 3.9 to 3.11, need to check details.
84
85- recvfrom() returned incorrect NBMA address, breaking NAT detection
86  Broken since 3.10-rc1
87    commit "GRE: Refactor GRE tunneling code."
88  Fixed in 3.10.27, 3.12.8, 3.13-rc7
89    commit "ip_gre: fix msg_name parsing for recvfrom/recvmsg"
90
91- sendto() was broken causing opennhrp not work at all
92  Broken since 3.10-rc1
93    commit "GRE: Refactor GRE tunneling code."
94  Fixed in 3.10.12, 3.11-rc6
95    commit "ip_gre: fix ipgre_header to return correct offset"
96
97- PMTU was broken due to GRE driver rewrite
98  Broken since 3.10-rc1
99    commit "GRE: Refactor GRE tunneling code."
100  Fixed in 3.11-rc1
101    commit "ip_tunnels: Use skb-len to PMTU check."
102
103- PMTU was broken due to routing cache removal
104  Broken since 3.6-rc1
105    commit "ipv4: Cache input routes in fib_info nexthops"
106  Fixed in 3.11-rc1
107    commit "ipv4: use next hop exceptions also for input routes"
108    + 3 other commits
109    Patches exist for 3.10, but they were not approved to 3.10-stable.
110
111- Race condition during bootup: changing ARP flag did not flush
112  existing neighbor entries, causing problems if traffic was routed
113  to gre interface before opennhrp was running.
114  Broken since dawn of time
115  Fixed in 3.11-rc1
116    commit "arp: flush arp cache on IFF_NOARP change"
117
118- Crash in IPsec
119  Broken since 3.9-rc1
120    commit "xfrm: removes a superfluous check and add a statistic"
121  Fixed in 3.10-rc3
122    commit "xfrm: properly handle invalid states as an error"
123
124- An incorrect ip_gre change broke NHRP traffic over GRE
125  Broken since 3.8-rc2
126    commit "ip_gre: make ipgre_tunnel_xmit() not parse network header as IP unconditionally"
127  Fixed in 3.8.5, 3.9-rc4
128    commit "Revert "ip_gre: make ipgre_tunnel_xmit() not parse network header as IP unconditionally""
129
130- Multicast traffic over mGRE was broken.
131  Broken since 2.6.34-rc2
132    commit "gre: fix hard header destination address checking"
133  Fixed in 2.6.39-rc2
134    commit "net: gre: provide multicast mappings for ipv4 and ipv6"
135
136- Serious performance issues causing small throughput on medium to large DMVPN networks
137  Broken since dawn of time
138  Fixed in 2.6.35
139    multiple commits rewriting ipsec caching
140
141- Even though around 2.6.24 is the first version where opennhrp started
142  to work, there has been various PMTU, performance, and functionality
143  bugs before 2.6.34. That's one of the first version I consider stable
144  wrt. to opennhrp functionality.
145
146

README.nhrpd

1Quagga / NHRP Design and Configuration Notes
2============================================
3
4Quagga/NHRP is an NHRP (RFC2332) implementation for Linux. The primary
5use case is to implement DMVPN. The aim is thus to be compatible with
6Cisco DMVPN (and potentially with FlexVPN in the future).
7
8
9Current Status
10--------------
11
12- IPsec integration with strongSwan (requires patched strongSwan)
13- IPv4 over IPv4 NBMA GRE
14- IPv6 over IPv4 NBMA GRE -- majority of code exist; but is not tested
15- Spoke (NHC) functionality complete
16- Hub (NHS) functionality complete
17- Multicast support is not done yet
18  (so OSPF will not work, use BGP for now)
19
20The code is not (yet) compatible with Cisco FlexVPN style DMVPN. It
21would require relaying IKEv2 routing messages from strongSwan to nhrpd
22and parsing that. It is doable, but not implemented for the time being.
23
24
25Routing Design
26--------------
27
28In contrast to opennhrp routing design, Quagga/NHRP routes each NHRP
29domain address individually (similar to Cisco FlexVPN).
30
31To create NBMA GRE tunnel you might use following:
32	ip tunnel add gre1 mode gre key 42 ttl 64 dev eth0
33	ip addr add 10.255.255.2/32 dev gre1
34	ip link set gre1 up
35
36This has two important differences compared to opennhrp setup:
37 1. The 'tunnel add' now specifies physical device binding. Quagga/NHRP
38    wants to know stable protocol address to NBMA address mapping. Thus,
39    add 'dev <physdev>' binding, or specify 'local <nbma-address>'. If
40    neither of this is specified, NHRP will not be enabled on the interface.
41    Alternatively you can skip 'dev' binding on tunnel if you allow
42    nhrpd to manage it using 'tunnel source' command (see below).
43
44 2. The 'addr add' now has host prefix. In opennhrp you would have used
45    the GRE subnet prefix length here instead, e.g. /24.
46
47Quagga/NHRP will automatically create additional host routes pointing to
48gre1 when a connection with these hosts is established. The gre1 subnet
49should be announced by routing protocol. This allows routing protocol
50to decide which is the closest hub and get the gre addresses' traffic.
51
52The second benefit is that hubs can then easily exchange host prefixes
53of directly connected gre addresses. And thus routing of gre addresses
54inside hubs is based on routing protocol's shortest path choice -- not
55on random choice from next hop server list.
56
57
58Configuring nhrpd
59-----------------
60
61The configuration is done using vtysh, and most commands do what they
62do in Cisco. As minimal configuration example one can do:
63 configure terminal
64 interface gre1
65   tunnel protection vici profile dmvpn
66   tunnel source eth0
67   ip nhrp network-id 1
68   ip nhrp shortcut
69   ip nhrp registration no-unique
70   ip nhrp nhs dynamic nbma hubs.example.com
71
72There's important notes about the "ip nhrp nhs" command:
73
74 1. The 'dynamic' works only against Cisco (or nhrpd), but is not
75    compatible with opennhrp. To use dynamic detection of opennhrp hub's
76    protocol address use the GRE broadcast address there. For the above
77    example of 10.255.255.0/24 the configuration should read instead:
78      ip nhrp nhs 10.255.255.255 nbma hubs.example.com
79
80 2. nbma <FQDN> works like opennhrp dynamic-map. That is, all of the
81    A-records are configured as NBMA addresses of different hubs, and
82    each hub protocol address will be dynamically detected.
83
84
85Hub functionality
86-----------------
87
88Sending Traffic Indication (redirect) notifications is now accomplished
89using NFLOG.
90
91Use:
92iptables -A FORWARD -i gre1 -o gre1 \
93	-m hashlimit --hashlimit-upto 4/minute --hashlimit-burst 1 \
94	--hashlimit-mode srcip,dstip --hashlimit-srcmask 16 --hashlimit-dstmask 16 \
95	--hashlimit-name loglimit-0 -j NFLOG --nflog-group 1 --nflog-range 128
96
97or similar to get rate-limited samples of the packets that match traffic
98flow needing redirection. This kernel NFLOG target's nflog-group is configured
99in global nhrp config with:
100	nhrp nflog-group 1
101
102To start sending these traffic notices out from hubs, use the nhrp per-interface
103directive:
104	ip nhrp redirect
105
106opennhrp used PF_PACKET and tried to create packet filter to get only
107the packets of interest. Though, this was bad if shortcut fails to
108establish (remote policy, or both are behind NAT or restrictive
109firewalls), all of the relayaed traffic would match always.
110
111
112Getting information via vtysh
113-----------------------------
114
115Some commands of interest:
116 - show dmvpn
117 - show ip nhrp cache
118 - show ip nhrp shortcut
119 - show ip route nhrp
120 - clear ip nhrp cache
121 - clear ip nhrp shortcut
122
123
124Integration with strongSwan
125---------------------------
126
127Contrary to opennhrp, Quagga/NHRP has tight integration with IKE daemon.
128Currently strongSwan is supported using the VICI protocol. strongSwan
129is connected using UNIX socket (hardcoded now as /var/run/charon.vici).
130Thus nhrpd needs to be run as user that can open that file.
131
132Currently, you will need patched strongSwan. The working tree is at:
133	http://git.alpinelinux.org/cgit/user/tteras/strongswan/log/?h=tteras
134
135And the branch with patches against latest release are:
136	http://git.alpinelinux.org/cgit/user/tteras/strongswan/log/?h=tteras-release
137
138