1KERNEL REQUIREMENTS 2=================== 3 4The linux kernel has had various major regressions, performance 5issues and subtle bugs (especially in pmtu). Here is a short list 6of some -stable kernels and the first point release that is supposedly 7working well with opennhrp/dmvpn: 8 3.12.8 or later 9 3.14.54 or later 10 3.18.22 or later[1] 11 12[1] But you need to apply the following two backported commits: 13 3cdaa5be9e ipv4: Don't increase PMTU with Datagram Too Big message 14 cb6ccf09d6 route: Use ipv4_mtu instead of raw rt_pmtu 15 16See below for list of known issues in various kernel versions. 17 18Kernels earlier than 3.12 need CONFIG_ARPD enabled in the configuration. 19Many distributions do not enable it by default, and you may need to 20compile your own kernel. 21 22KERNEL BUGS 23=========== 24 25DMVPN and mGRE support in the kernel has been brittle. There are various 26regressions in multiple kernel versions. 27 28This list tries to collect them to one source of information: 29 30- forward pmtu is disabled intentionally (but tunnel devices rely on it) 31 Broken since 3.14-rc1: 32 commit "ipv4: introduce ip_dst_mtu_maybe_forward and protect forwarding path against pmtu spoofing" 33 Workaround: 34 Set sysctl net.ipv4.ip_forward_use_pmtu=1 35 (Should fix kernel to have this by default on for tunnel devices) 36 37- subtle path mtu mishandling issues 38 Broken since (uncertain) 39 Fixed in 4.1-rc2: 40 commit "ipv4: Don't increase PMTU with Datagram Too Big message." 41 commit "route: Use ipv4_mtu instead of raw rt_pmtu" 42 43- fragmentation of large packets inside tunnel not working 44 Broken since 3.11-rc1 45 commit "ip_tunnels: Use skb-len to PMTU check." 46 Fixed in 3.14.54, 3.18.22, 4.1.9, 4.2-rc3 47 commit "ip_tunnel: fix ipv4 pmtu check to honor inner ip header df" 48 49- ipsec will crash during xfrm gc 50 Broke since 3.15-rc1 51 commit "flowcache: Make flow cache name space aware" 52 Fixed in 3.18.10, 4.0 53 commit "flowcache: Fix kernel panic in flow_cache_flush_task" 54 55- TSO on GRE tunnels failed, and resulted in very slow performance 56 Broke since 3.14.24, 3.18-rc3 57 commit "gre: Use inner mac length when computing tunnel length" 58 Fixed in 3.14.30, 3.18.4 59 commit "gre: fix the inner mac header in nbma tunnel xmit path" 60 commit "gre: Set inner mac header in gro complete" 61 62- NAPI GRO handling was broken; causing immediate crash (32-bit only?) 63 Broken since 3.13-rc1 64 commit "net: gro: allow to build full sized skb" 65 Fixed 3.14.5, 3.15-rc7 66 commit "net: gro: make sure skb->cb[] initial content has not to be zero" 67 68- ip_gre dst caching broke NBMA GRE tunnels 69 Broken since 3.14-rc1 70 Fixed in 3.14.5, 3.15-rc6 71 commit "ipv4: ip_tunnels: disable cache for nbma gre tunnels" 72 73- Few packets can be lost when neighbor entry is in NUD_PROBE state, 74 and there is continuous traffic to it. 75 Broken since dawn of time 76 Fixed in 3.15-rc1 77 commit "neigh: probe application via netlink in NUD_PROBE" 78 79- GRO was implemented for GRE, but the hw capabilities were not updated 80 correctly. In practice forwarding from non-GRE (physical) interface 81 to GRE interface with gro/gso/tx offloads enabled (also on the target 82 interface) does not work properly. 83 Broken around 3.9 to 3.11, need to check details. 84 85- recvfrom() returned incorrect NBMA address, breaking NAT detection 86 Broken since 3.10-rc1 87 commit "GRE: Refactor GRE tunneling code." 88 Fixed in 3.10.27, 3.12.8, 3.13-rc7 89 commit "ip_gre: fix msg_name parsing for recvfrom/recvmsg" 90 91- sendto() was broken causing opennhrp not work at all 92 Broken since 3.10-rc1 93 commit "GRE: Refactor GRE tunneling code." 94 Fixed in 3.10.12, 3.11-rc6 95 commit "ip_gre: fix ipgre_header to return correct offset" 96 97- PMTU was broken due to GRE driver rewrite 98 Broken since 3.10-rc1 99 commit "GRE: Refactor GRE tunneling code." 100 Fixed in 3.11-rc1 101 commit "ip_tunnels: Use skb-len to PMTU check." 102 103- PMTU was broken due to routing cache removal 104 Broken since 3.6-rc1 105 commit "ipv4: Cache input routes in fib_info nexthops" 106 Fixed in 3.11-rc1 107 commit "ipv4: use next hop exceptions also for input routes" 108 + 3 other commits 109 Patches exist for 3.10, but they were not approved to 3.10-stable. 110 111- Race condition during bootup: changing ARP flag did not flush 112 existing neighbor entries, causing problems if traffic was routed 113 to gre interface before opennhrp was running. 114 Broken since dawn of time 115 Fixed in 3.11-rc1 116 commit "arp: flush arp cache on IFF_NOARP change" 117 118- Crash in IPsec 119 Broken since 3.9-rc1 120 commit "xfrm: removes a superfluous check and add a statistic" 121 Fixed in 3.10-rc3 122 commit "xfrm: properly handle invalid states as an error" 123 124- An incorrect ip_gre change broke NHRP traffic over GRE 125 Broken since 3.8-rc2 126 commit "ip_gre: make ipgre_tunnel_xmit() not parse network header as IP unconditionally" 127 Fixed in 3.8.5, 3.9-rc4 128 commit "Revert "ip_gre: make ipgre_tunnel_xmit() not parse network header as IP unconditionally"" 129 130- Multicast traffic over mGRE was broken. 131 Broken since 2.6.34-rc2 132 commit "gre: fix hard header destination address checking" 133 Fixed in 2.6.39-rc2 134 commit "net: gre: provide multicast mappings for ipv4 and ipv6" 135 136- Serious performance issues causing small throughput on medium to large DMVPN networks 137 Broken since dawn of time 138 Fixed in 2.6.35 139 multiple commits rewriting ipsec caching 140 141- Even though around 2.6.24 is the first version where opennhrp started 142 to work, there has been various PMTU, performance, and functionality 143 bugs before 2.6.34. That's one of the first version I consider stable 144 wrt. to opennhrp functionality. 145 146