1.. _topotests-json: 2 3Topotests with JSON 4=================== 5 6Overview 7-------- 8 9On top of current topotests framework following enhancements are done: 10 11 12* Creating the topology and assigning IPs to router' interfaces dynamically. 13 It is achieved by using json file, in which user specify the number of 14 routers, links to each router, interfaces for the routers and protocol 15 configurations for all routers. 16 17* Creating the configurations dynamically. It is achieved by using 18 :file:`/usr/lib/frr/frr-reload.py` utility, which takes running configuration 19 and the newly created configuration for any particular router and creates a 20 delta file(diff file) and loads it to router. 21 22 23Logging of test case executions 24------------------------------- 25 26* The user can enable logging of testcases execution messages into log file by 27 adding ``frrtest_log_dir = /tmp/topotests/`` in :file:`pytest.ini`. 28* Router's current configuration can be displyed on console or sent to logs by 29 adding ``show_router_config = True`` in :file:`pytest.ini`. 30 31Log file name will be displayed when we start execution: 32 33.. code-block:: console 34 35 root@test:# python ./test_topo_json_single_link.py 36 37 Logs will be sent to logfile: 38 /tmp/topotests/test_topo_json_single_link_11:57:01.353797 39 40Note: directory "/tmp/topotests/" is created by topotests by default, making 41use of same directory to save execution logs. 42 43Guidelines 44---------- 45 46Writing New Tests 47^^^^^^^^^^^^^^^^^ 48 49This section will guide you in all recommended steps to produce a standard 50topology test. 51 52This is the recommended test writing routine: 53 54* Create a json file , which will have routers and protocol configurations 55* Create topology from json 56* Create configuration from json 57* Write the tests 58* Format the new code using `black <https://github.com/psf/black>`_ 59* Create a Pull Request 60 61.. Note:: 62 63 BGP tests MUST use generous convergence timeouts - you must ensure 64 that any test involving BGP uses a convergence timeout of at least 65 130 seconds. 66 67File Hierarchy 68^^^^^^^^^^^^^^ 69 70Before starting to write any tests one must know the file hierarchy. The 71repository hierarchy looks like this: 72 73.. code-block:: console 74 75 $ cd path/to/topotests 76 $ find ./* 77 ... 78 ./example-topojson-test # the basic example test topology-1 79 ./example-topojson-test/test_example_topojson.json # input json file, having 80 topology, interfaces, bgp and other configuration 81 ./example-topojson-test/test_example_topojson.py # test script to write and 82 execute testcases 83 ... 84 ./lib # shared test/topology functions 85 ./lib/topojson.py # library to create topology and configurations dynamically 86 from json file 87 ./lib/common_config.py # library to create protocol's common configurations ex- 88 static_routes, prefix_lists, route_maps etc. 89 ./lib/bgp.py # library to create only bgp configurations 90 91Defining the Topology and initial configuration in JSON file 92^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 93 94The first step to write a new test is to define the topology and initial 95configuration. User has to define topology and initial configuration in JSON 96file. Here is an example of JSON file:: 97 98 BGP neihghborship with single phy-link, sample JSON file: 99 { 100 "ipv4base": "192.168.0.0", 101 "ipv4mask": 30, 102 "ipv6base": "fd00::", 103 "ipv6mask": 64, 104 "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, 105 "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, 106 "routers": { 107 "r1": { 108 "links": { 109 "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, 110 "r2": {"ipv4": "auto", "ipv6": "auto"}, 111 "r3": {"ipv4": "auto", "ipv6": "auto"} 112 }, 113 "bgp": { 114 "local_as": "64512", 115 "address_family": { 116 "ipv4": { 117 "unicast": { 118 "neighbor": { 119 "r2": { 120 "dest_link": { 121 "r1": {} 122 } 123 }, 124 "r3": { 125 "dest_link": { 126 "r1": {} 127 } 128 } 129 } 130 } 131 } 132 } 133 } 134 }, 135 "r2": { 136 "links": { 137 "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, 138 "r1": {"ipv4": "auto", "ipv6": "auto"}, 139 "r3": {"ipv4": "auto", "ipv6": "auto"} 140 }, 141 "bgp": { 142 "local_as": "64512", 143 "address_family": { 144 "ipv4": { 145 "unicast": { 146 "redistribute": [ 147 { 148 "redist_type": "static" 149 } 150 ], 151 "neighbor": { 152 "r1": { 153 "dest_link": { 154 "r2": {} 155 } 156 }, 157 "r3": { 158 "dest_link": { 159 "r2": {} 160 } 161 } 162 } 163 } 164 } 165 } 166 } 167 } 168 ... 169 170 171BGP neighboship with loopback interface, sample JSON file:: 172 173 { 174 "ipv4base": "192.168.0.0", 175 "ipv4mask": 30, 176 "ipv6base": "fd00::", 177 "ipv6mask": 64, 178 "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, 179 "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, 180 "routers": { 181 "r1": { 182 "links": { 183 "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback", 184 "add_static_route":"yes"}, 185 "r2": {"ipv4": "auto", "ipv6": "auto"} 186 }, 187 "bgp": { 188 "local_as": "64512", 189 "address_family": { 190 "ipv4": { 191 "unicast": { 192 "neighbor": { 193 "r2": { 194 "dest_link": { 195 "lo": { 196 "source_link": "lo" 197 } 198 } 199 } 200 } 201 } 202 } 203 } 204 }, 205 "static_routes": [ 206 { 207 "network": "1.0.2.17/32", 208 "next_hop": "192.168.0.1 209 } 210 ] 211 }, 212 "r2": { 213 "links": { 214 "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback", 215 "add_static_route":"yes"}, 216 "r1": {"ipv4": "auto", "ipv6": "auto"}, 217 "r3": {"ipv4": "auto", "ipv6": "auto"} 218 }, 219 "bgp": { 220 "local_as": "64512", 221 "address_family": { 222 "ipv4": { 223 "unicast": { 224 "redistribute": [ 225 { 226 "redist_type": "static" 227 } 228 ], 229 "neighbor": { 230 "r1": { 231 "dest_link": { 232 "lo": { 233 "source_link": "lo" 234 } 235 } 236 }, 237 "r3": { 238 "dest_link": { 239 "lo": { 240 "source_link": "lo" 241 } 242 } 243 } 244 } 245 } 246 } 247 } 248 }, 249 "static_routes": [ 250 { 251 "network": "192.0.20.1/32", 252 "no_of_ip": 9, 253 "admin_distance": 100, 254 "next_hop": "192.168.0.1", 255 "tag": 4001 256 } 257 ], 258 } 259 ... 260 261BGP neighborship with Multiple phy-links, sample JSON file:: 262 263 { 264 "ipv4base": "192.168.0.0", 265 "ipv4mask": 30, 266 "ipv6base": "fd00::", 267 "ipv6mask": 64, 268 "link_ip_start": {"ipv4": "192.168.0.0", "v4mask": 30, "ipv6": "fd00::", "v6mask": 64}, 269 "lo_prefix": {"ipv4": "1.0.", "v4mask": 32, "ipv6": "2001:DB8:F::", "v6mask": 128}, 270 "routers": { 271 "r1": { 272 "links": { 273 "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, 274 "r2-link1": {"ipv4": "auto", "ipv6": "auto"}, 275 "r2-link2": {"ipv4": "auto", "ipv6": "auto"} 276 }, 277 "bgp": { 278 "local_as": "64512", 279 "address_family": { 280 "ipv4": { 281 "unicast": { 282 "neighbor": { 283 "r2": { 284 "dest_link": { 285 "r1-link1": {} 286 } 287 } 288 } 289 } 290 } 291 } 292 } 293 }, 294 "r2": { 295 "links": { 296 "lo": {"ipv4": "auto", "ipv6": "auto", "type": "loopback"}, 297 "r1-link1": {"ipv4": "auto", "ipv6": "auto"}, 298 "r1-link2": {"ipv4": "auto", "ipv6": "auto"}, 299 "r3-link1": {"ipv4": "auto", "ipv6": "auto"}, 300 "r3-link2": {"ipv4": "auto", "ipv6": "auto"} 301 }, 302 "bgp": { 303 "local_as": "64512", 304 "address_family": { 305 "ipv4": { 306 "unicast": { 307 "redistribute": [ 308 { 309 "redist_type": "static" 310 } 311 ], 312 "neighbor": { 313 "r1": { 314 "dest_link": { 315 "r2-link1": {} 316 } 317 }, 318 "r3": { 319 "dest_link": { 320 "r2-link1": {} 321 } 322 } 323 } 324 } 325 } 326 } 327 } 328 } 329 ... 330 331 332JSON File Explained 333""""""""""""""""""" 334 335Mandatory keywords/options in JSON: 336 337* ``ipv4base`` : base ipv4 address to generate ips, ex - 192.168.0.0 338* ``ipv4mask`` : mask for ipv4 address, ex - 30 339* ``ipv6base`` : base ipv6 address to generate ips, ex - fd00: 340* ``ipv6mask`` : mask for ipv6 address, ex - 64 341* ``link_ip_start`` : physical interface base ipv4 and ipv6 address 342* ``lo_prefix`` : loopback interface base ipv4 and ipv6 address 343* ``routers`` : user can add number of routers as per topology, router's name 344 can be any logical name, ex- r1 or a0. 345* ``r1`` : name of the router 346* ``lo`` : loopback interface dict, ipv4 and/or ipv6 addresses generated automatically 347* ``type`` : type of interface, to identify loopback interface 348* ``links`` : physical interfaces dict, ipv4 and/or ipv6 addresses generated 349 automatically 350* ``r2-link1`` : it will be used when routers have multiple links. 'r2' is router 351 name, 'link' is any logical name, '1' is to identify link number, 352 router name and link must be seperated by hyphen (``-``), ex- a0-peer1 353 354Optional keywords/options in JSON: 355 356* ``bgp`` : bgp configuration 357* ``local_as`` : Local AS number 358* ``unicast`` : All SAFI configuration 359* ``neighbor``: All neighbor details 360* ``dest_link`` : Destination link to which router will connect 361* ``router_id`` : bgp router-id 362* ``source_link`` : if user wants to establish bgp neighborship with loopback 363 interface, add ``source_link``: ``lo`` 364* ``keepalivetimer`` : Keep alive timer for BGP neighbor 365* ``holddowntimer`` : Hold down timer for BGP neighbor 366* ``static_routes`` : create static routes for routers 367* ``redistribute`` : redistribute static and/or connected routes 368* ``prefix_lists`` : create Prefix-lists for routers 369 370Building topology and configurations 371"""""""""""""""""""""""""""""""""""" 372 373Topology and initial configuration will be created in setup_module(). Following 374is the sample code:: 375 376 class TemplateTopo(Topo): 377 def build(self, *_args, **_opts): 378 "Build function" 379 tgen = get_topogen(self) 380 381 # Building topology from json file 382 build_topo_from_json(tgen, topo) 383 384 def setup_module(mod): 385 tgen = Topogen(TemplateTopo, mod.__name__) 386 387 # Starting topology, create tmp files which are loaded to routers 388 # to start deamons and then start routers 389 start_topology(tgen) 390 391 # Creating configuration from JSON 392 build_config_from_json(tgen, topo) 393 394 def teardown_module(mod): 395 tgen = get_topogen() 396 397 # Stop toplogy and Remove tmp files 398 stop_topology(tgen) 399 400 401* Note: Topology will be created in setup module but routers will not be 402 started until we load zebra.conf and bgpd.conf to routers. For all routers 403 dirs will be created in /tmp/topotests/<test_folder_name>/<router_name> 404 zebra.conf and bgpd.conf empty files will be created and laoded to routers. 405 All folder and files are deleted in teardown module.. 406 407Creating configuration files 408"""""""""""""""""""""""""""" 409 410Router's configuration would be saved in config file frr_json.conf. Common 411configurations are like, static routes, prefixlists and route maps etc configs, 412these configs can be used by any other protocols as it is. 413BGP config will be specific to BGP protocol testing. 414 415* JSON file is passed to API build_config_from_json(), which looks for 416 configuration tags in JSON file. 417* If tag is found in JSON, configuration is created as per input and written 418 to file frr_json.conf 419* Once JSON parsing is over, frr_json.conf is loaded onto respective router. 420 Config loading is done using 'vtysh -f <file>'. Initial config at this point 421 is also saved frr_json_initial.conf. This file can be used to reset 422 configuration on router, during the course of execution. 423* Reset of configuration is done using frr "reload.py" utility, which 424 calculates the difference between router's running config and user's config 425 and loads delta file to router. API used - reset_config_on_router() 426 427Writing Tests 428""""""""""""" 429 430Test topologies should always be bootstrapped from the 431example-test/test_example.py, because it contains important boilerplate code 432that can't be avoided, like: 433 434imports: os, sys, pytest, topotest/topogen and mininet topology class 435 436The global variable CWD (Current Working directory): which is most likely going 437to be used to reference the routers configuration file location 438 439Example: 440 441 442* The topology class that inherits from Mininet Topo class; 443 444 .. code-block:: python 445 446 class TemplateTopo(Topo): 447 def build(self, *_args, **_opts): 448 tgen = get_topogen(self) 449 # topology build code 450 451 452* pytest setup_module() and teardown_module() to start the topology: 453 454 .. code-block:: python 455 456 def setup_module(_m): 457 tgen = Topogen(TemplateTopo) 458 459 # Starting topology, create tmp files which are loaded to routers 460 # to start deamons and then start routers 461 start_topology(tgen, CWD) 462 463 def teardown_module(_m): 464 tgen = get_topogen() 465 466 # Stop toplogy and Remove tmp files 467 stop_topology(tgen, CWD) 468 469 470* ``__main__`` initialization code (to support running the script directly) 471 472 .. code-block:: python 473 474 if **name** == '\ **main**\ ': 475 sys.exit(pytest.main(["-s"])) 476 477