1""" 2A non-blocking REST API for Salt 3================================ 4 5.. py:currentmodule:: salt.netapi.rest_tornado.saltnado 6 7:depends: - tornado Python module 8 9:configuration: All authentication is done through Salt's :ref:`external auth 10 <acl-eauth>` system which requires additional configuration not described 11 here. 12 13 14In order to run rest_tornado with the salt-master 15add the following to the Salt master config file. 16 17.. code-block:: yaml 18 19 rest_tornado: 20 # can be any port 21 port: 8000 22 # address to bind to (defaults to 0.0.0.0) 23 address: 0.0.0.0 24 # socket backlog 25 backlog: 128 26 ssl_crt: /etc/pki/api/certs/server.crt 27 # no need to specify ssl_key if cert and key 28 # are in one single file 29 ssl_key: /etc/pki/api/certs/server.key 30 debug: False 31 disable_ssl: False 32 webhook_disable_auth: False 33 cors_origin: null 34 35.. _rest_tornado-auth: 36 37Authentication 38-------------- 39 40Authentication is performed by passing a session token with each request. 41Tokens are generated via the :py:class:`SaltAuthHandler` URL. 42 43The token may be sent in one of two ways: 44 45* Include a custom header named :mailheader:`X-Auth-Token`. 46* Sent via a cookie. This option is a convenience for HTTP clients that 47 automatically handle cookie support (such as browsers). 48 49.. seealso:: You can bypass the session handling via the :py:class:`RunSaltAPIHandler` URL. 50 51CORS 52---- 53 54rest_tornado supports Cross-site HTTP requests out of the box. It is by default 55deactivated and controlled by the `cors_origin` config key. 56 57You can allow all origins by settings `cors_origin` to `*`. 58 59You can allow only one origin with this configuration: 60 61.. code-block:: yaml 62 63 rest_tornado: 64 cors_origin: http://salt.yourcompany.com 65 66You can also be more specific and select only a few allowed origins by using 67a list. For example: 68 69.. code-block:: yaml 70 71 rest_tornado: 72 cors_origin: 73 - http://salt.yourcompany.com 74 - http://salt-preprod.yourcampany.com 75 76The format for origin are full URL, with both scheme and port if not standard. 77 78In this case, rest_tornado will check if the Origin header is in the allowed 79list if it's the case allow the origin. Else it will returns nothing, 80effectively preventing the origin to make request. 81 82For reference, CORS is a mechanism used by browser to allow (or disallow) 83requests made from browser from a different origin than salt-api. It's 84complementary to Authentication and mandatory only if you plan to use 85a salt client developed as a Javascript browser application. 86 87Usage 88----- 89 90Commands are sent to a running Salt master via this module by sending HTTP 91requests to the URLs detailed below. 92 93.. admonition:: Content negotiation 94 95 This REST interface is flexible in what data formats it will accept as well 96 as what formats it will return (e.g., JSON, YAML, x-www-form-urlencoded). 97 98 * Specify the format of data in the request body by including the 99 :mailheader:`Content-Type` header. 100 * Specify the desired data format for the response body with the 101 :mailheader:`Accept` header. 102 103Data sent in :http:method:`post` and :http:method:`put` requests must be in 104the format of a list of lowstate dictionaries. This allows multiple commands to 105be executed in a single HTTP request. 106 107.. glossary:: 108 109 lowstate 110 A dictionary containing various keys that instruct Salt which command 111 to run, where that command lives, any parameters for that command, any 112 authentication credentials, what returner to use, etc. 113 114 Salt uses the lowstate data format internally in many places to pass 115 command data between functions. Salt also uses lowstate for the 116 :ref:`LocalClient() <python-api>` Python API interface. 117 118The following example (in JSON format) causes Salt to execute two commands:: 119 120 [{ 121 "client": "local", 122 "tgt": "*", 123 "fun": "test.fib", 124 "arg": ["10"] 125 }, 126 { 127 "client": "runner", 128 "fun": "jobs.lookup_jid", 129 "jid": "20130603122505459265" 130 }] 131 132Multiple commands in a Salt API request will be executed in serial and makes 133no guarantees that all commands will run. Meaning that if test.fib (from the 134example above) had an exception, the API would still execute "jobs.lookup_jid". 135 136Responses to these lowstates are an in-order list of dicts containing the 137return data, a yaml response could look like:: 138 139 - ms-1: true 140 ms-2: true 141 - ms-1: foo 142 ms-2: bar 143 144In the event of an exception while executing a command the return for that lowstate 145will be a string, for example if no minions matched the first lowstate we would get 146a return like:: 147 148 - No minions matched the target. No command was sent, no jid was assigned. 149 - ms-1: true 150 ms-2: true 151 152.. admonition:: x-www-form-urlencoded 153 154 Sending JSON or YAML in the request body is simple and most flexible, 155 however sending data in urlencoded format is also supported with the 156 caveats below. It is the default format for HTML forms, many JavaScript 157 libraries, and the :command:`curl` command. 158 159 For example, the equivalent to running ``salt '*' test.ping`` is sending 160 ``fun=test.ping&arg&client=local&tgt=*`` in the HTTP request body. 161 162 Caveats: 163 164 * Only a single command may be sent per HTTP request. 165 * Repeating the ``arg`` parameter multiple times will cause those 166 parameters to be combined into a single list. 167 168 Note, some popular frameworks and languages (notably jQuery, PHP, and 169 Ruby on Rails) will automatically append empty brackets onto repeated 170 parameters. E.g., ``arg=one``, ``arg=two`` will be sent as ``arg[]=one``, 171 ``arg[]=two``. This is not supported; send JSON or YAML instead. 172 173 174.. |req_token| replace:: a session token from :py:class:`~SaltAuthHandler`. 175.. |req_accept| replace:: the desired response format. 176.. |req_ct| replace:: the format of the request body. 177 178.. |res_ct| replace:: the format of the response body; depends on the 179 :mailheader:`Accept` request header. 180 181.. |200| replace:: success 182.. |400| replace:: bad request 183.. |401| replace:: authentication required 184.. |406| replace:: requested Content-Type not available 185.. |500| replace:: internal server error 186""" 187 188import cgi 189import fnmatch 190import logging 191import time 192from collections import defaultdict 193from copy import copy 194 195import salt.auth 196import salt.client 197import salt.ext.tornado.escape 198import salt.ext.tornado.gen 199import salt.ext.tornado.httpserver 200import salt.ext.tornado.ioloop 201import salt.ext.tornado.web 202import salt.netapi 203import salt.runner 204import salt.utils.args 205import salt.utils.event 206import salt.utils.json 207import salt.utils.minions 208import salt.utils.yaml 209from salt.exceptions import ( 210 AuthenticationError, 211 AuthorizationError, 212 EauthAuthenticationError, 213) 214from salt.ext.tornado.concurrent import Future 215from salt.utils.event import tagify 216 217_json = salt.utils.json.import_json() 218log = logging.getLogger(__name__) 219 220 221def _json_dumps(obj, **kwargs): 222 """ 223 Invoke salt.utils.json.dumps using the alternate json module loaded using 224 salt.utils.json.import_json(). This ensures that we properly encode any 225 strings in the object before we perform the serialization. 226 """ 227 return salt.utils.json.dumps(obj, _json_module=_json, **kwargs) 228 229 230# The clients rest_cherrypi supports. We want to mimic the interface, but not 231# necessarily use the same API under the hood 232# # all of these require coordinating minion stuff 233# - "local" (done) 234# - "local_async" (done) 235 236# # master side 237# - "runner" (done) 238# - "wheel" (need asynchronous api...) 239 240 241AUTH_TOKEN_HEADER = "X-Auth-Token" 242AUTH_COOKIE_NAME = "session_id" 243 244 245class TimeoutException(Exception): 246 pass 247 248 249class Any(Future): 250 """ 251 Future that wraps other futures to "block" until one is done 252 """ 253 254 def __init__(self, futures): 255 super().__init__() 256 for future in futures: 257 future.add_done_callback(self.done_callback) 258 259 def done_callback(self, future): 260 # Any is completed once one is done, we don't set for the rest 261 if not self.done(): 262 self.set_result(future) 263 264 265class EventListener: 266 """ 267 Class responsible for listening to the salt master event bus and updating 268 futures. This is the core of what makes this asynchronous, this allows us to do 269 non-blocking work in the main processes and "wait" for an event to happen 270 """ 271 272 def __init__(self, mod_opts, opts): 273 self.mod_opts = mod_opts 274 self.opts = opts 275 self.event = salt.utils.event.get_event( 276 "master", 277 opts["sock_dir"], 278 opts["transport"], 279 opts=opts, 280 listen=True, 281 io_loop=salt.ext.tornado.ioloop.IOLoop.current(), 282 ) 283 284 # tag -> list of futures 285 self.tag_map = defaultdict(list) 286 287 # request_obj -> list of (tag, future) 288 self.request_map = defaultdict(list) 289 290 # map of future -> timeout_callback 291 self.timeout_map = {} 292 293 self.event.set_event_handler(self._handle_event_socket_recv) 294 295 def clean_by_request(self, request): 296 """ 297 Remove all futures that were waiting for request `request` since it is done waiting 298 """ 299 if request not in self.request_map: 300 return 301 for tag, matcher, future in self.request_map[request]: 302 # timeout the future 303 self._timeout_future(tag, matcher, future) 304 # remove the timeout 305 if future in self.timeout_map: 306 salt.ext.tornado.ioloop.IOLoop.current().remove_timeout( 307 self.timeout_map[future] 308 ) 309 del self.timeout_map[future] 310 311 del self.request_map[request] 312 313 @staticmethod 314 def prefix_matcher(mtag, tag): 315 if mtag is None or tag is None: 316 raise TypeError("mtag or tag can not be None") 317 return mtag.startswith(tag) 318 319 @staticmethod 320 def exact_matcher(mtag, tag): 321 if mtag is None or tag is None: 322 raise TypeError("mtag or tag can not be None") 323 return mtag == tag 324 325 def get_event( 326 self, 327 request, 328 tag="", 329 matcher=prefix_matcher.__func__, 330 callback=None, 331 timeout=None, 332 ): 333 """ 334 Get an event (asynchronous of course) return a future that will get it later 335 """ 336 # if the request finished, no reason to allow event fetching, since we 337 # can't send back to the client 338 if request._finished: 339 future = Future() 340 future.set_exception(TimeoutException()) 341 return future 342 343 future = Future() 344 if callback is not None: 345 346 def handle_future(future): 347 salt.ext.tornado.ioloop.IOLoop.current().add_callback( 348 callback, future 349 ) # pylint: disable=E1102 350 351 future.add_done_callback(handle_future) 352 # add this tag and future to the callbacks 353 self.tag_map[(tag, matcher)].append(future) 354 self.request_map[request].append((tag, matcher, future)) 355 356 if timeout: 357 timeout_future = salt.ext.tornado.ioloop.IOLoop.current().call_later( 358 timeout, self._timeout_future, tag, matcher, future 359 ) 360 self.timeout_map[future] = timeout_future 361 362 return future 363 364 def _timeout_future(self, tag, matcher, future): 365 """ 366 Timeout a specific future 367 """ 368 if (tag, matcher) not in self.tag_map: 369 return 370 if not future.done(): 371 future.set_exception(TimeoutException()) 372 self.tag_map[(tag, matcher)].remove(future) 373 if len(self.tag_map[(tag, matcher)]) == 0: 374 del self.tag_map[(tag, matcher)] 375 376 def _handle_event_socket_recv(self, raw): 377 """ 378 Callback for events on the event sub socket 379 """ 380 mtag, data = self.event.unpack(raw) 381 382 # see if we have any futures that need this info: 383 for (tag, matcher), futures in self.tag_map.items(): 384 try: 385 is_matched = matcher(mtag, tag) 386 except Exception: # pylint: disable=broad-except 387 log.error("Failed to run a matcher.", exc_info=True) 388 is_matched = False 389 390 if not is_matched: 391 continue 392 393 for future in futures: 394 if future.done(): 395 continue 396 future.set_result({"data": data, "tag": mtag}) 397 self.tag_map[(tag, matcher)].remove(future) 398 if future in self.timeout_map: 399 salt.ext.tornado.ioloop.IOLoop.current().remove_timeout( 400 self.timeout_map[future] 401 ) 402 del self.timeout_map[future] 403 404 405class BaseSaltAPIHandler(salt.ext.tornado.web.RequestHandler): # pylint: disable=W0223 406 ct_out_map = ( 407 ("application/json", _json_dumps), 408 ("application/x-yaml", salt.utils.yaml.safe_dump), 409 ) 410 411 def _verify_client(self, low): 412 """ 413 Verify that the client is in fact one we have 414 """ 415 if "client" not in low or low.get("client") not in self.saltclients: 416 self.set_status(400) 417 self.write("400 Invalid Client: Client not found in salt clients") 418 self.finish() 419 return False 420 return True 421 422 def initialize(self): 423 """ 424 Initialize the handler before requests are called 425 """ 426 if not hasattr(self.application, "event_listener"): 427 log.debug("init a listener") 428 self.application.event_listener = EventListener( 429 self.application.mod_opts, 430 self.application.opts, 431 ) 432 433 if not hasattr(self, "saltclients"): 434 local_client = salt.client.get_local_client(mopts=self.application.opts) 435 self.saltclients = { 436 "local": local_client.run_job_async, 437 # not the actual client we'll use.. but its what we'll use to get args 438 "local_async": local_client.run_job_async, 439 "runner": salt.runner.RunnerClient( 440 opts=self.application.opts 441 ).cmd_async, 442 "runner_async": None, # empty, since we use the same client as `runner` 443 } 444 445 if not hasattr(self, "ckminions"): 446 self.ckminions = salt.utils.minions.CkMinions(self.application.opts) 447 448 @property 449 def token(self): 450 """ 451 The token used for the request 452 """ 453 # find the token (cookie or headers) 454 if AUTH_TOKEN_HEADER in self.request.headers: 455 return self.request.headers[AUTH_TOKEN_HEADER] 456 else: 457 return self.get_cookie(AUTH_COOKIE_NAME) 458 459 def _verify_auth(self): 460 """ 461 Boolean whether the request is auth'd 462 """ 463 464 return self.token and bool(self.application.auth.get_tok(self.token)) 465 466 def prepare(self): 467 """ 468 Run before get/posts etc. Pre-flight checks: 469 - verify that we can speak back to them (compatible accept header) 470 """ 471 # Find an acceptable content-type 472 accept_header = self.request.headers.get("Accept", "*/*") 473 # Ignore any parameter, including q (quality) one 474 parsed_accept_header = [ 475 cgi.parse_header(h)[0] for h in accept_header.split(",") 476 ] 477 478 def find_acceptable_content_type(parsed_accept_header): 479 for media_range in parsed_accept_header: 480 for content_type, dumper in self.ct_out_map: 481 if fnmatch.fnmatch(content_type, media_range): 482 return content_type, dumper 483 return None, None 484 485 content_type, dumper = find_acceptable_content_type(parsed_accept_header) 486 487 # better return message? 488 if not content_type: 489 self.send_error(406) 490 491 self.content_type = content_type 492 self.dumper = dumper 493 494 # do the common parts 495 self.start = time.time() 496 self.connected = True 497 498 self.lowstate = self._get_lowstate() 499 500 def timeout_futures(self): 501 """ 502 timeout a session 503 """ 504 # TODO: set a header or something??? so we know it was a timeout 505 self.application.event_listener.clean_by_request(self) 506 507 def on_finish(self): 508 """ 509 When the job has been done, lets cleanup 510 """ 511 # timeout all the futures 512 self.timeout_futures() 513 # clear local_client objects to disconnect event publisher's IOStream connections 514 del self.saltclients 515 516 def on_connection_close(self): 517 """ 518 If the client disconnects, lets close out 519 """ 520 self.finish() 521 522 def serialize(self, data): 523 """ 524 Serlialize the output based on the Accept header 525 """ 526 self.set_header("Content-Type", self.content_type) 527 528 return self.dumper(data) 529 530 def _form_loader(self, _): 531 """ 532 function to get the data from the urlencoded forms 533 ignore the data passed in and just get the args from wherever they are 534 """ 535 data = {} 536 for key in self.request.arguments: 537 val = self.get_arguments(key) 538 if len(val) == 1: 539 data[key] = val[0] 540 else: 541 data[key] = val 542 return data 543 544 def deserialize(self, data): 545 """ 546 Deserialize the data based on request content type headers 547 """ 548 ct_in_map = { 549 "application/x-www-form-urlencoded": self._form_loader, 550 "application/json": salt.utils.json.loads, 551 "application/x-yaml": salt.utils.yaml.safe_load, 552 "text/yaml": salt.utils.yaml.safe_load, 553 # because people are terrible and don't mean what they say 554 "text/plain": salt.utils.json.loads, 555 } 556 557 try: 558 # Use cgi.parse_header to correctly separate parameters from value 559 value, parameters = cgi.parse_header(self.request.headers["Content-Type"]) 560 return ct_in_map[value](salt.ext.tornado.escape.native_str(data)) 561 except KeyError: 562 self.send_error(406) 563 except ValueError: 564 self.send_error(400) 565 566 def _get_lowstate(self): 567 """ 568 Format the incoming data into a lowstate object 569 """ 570 if not self.request.body: 571 return 572 data = self.deserialize(self.request.body) 573 self.request_payload = copy(data) 574 575 if data and "arg" in data and not isinstance(data["arg"], list): 576 data["arg"] = [data["arg"]] 577 578 if not isinstance(data, list): 579 lowstate = [data] 580 else: 581 lowstate = data 582 583 return lowstate 584 585 def set_default_headers(self): 586 """ 587 Set default CORS headers 588 """ 589 mod_opts = self.application.mod_opts 590 591 if mod_opts.get("cors_origin"): 592 origin = self.request.headers.get("Origin") 593 594 allowed_origin = _check_cors_origin(origin, mod_opts["cors_origin"]) 595 596 if allowed_origin: 597 self.set_header("Access-Control-Allow-Origin", allowed_origin) 598 599 def options(self, *args, **kwargs): 600 """ 601 Return CORS headers for preflight requests 602 """ 603 # Allow X-Auth-Token in requests 604 request_headers = self.request.headers.get("Access-Control-Request-Headers") 605 allowed_headers = request_headers.split(",") 606 607 # Filter allowed header here if needed. 608 609 # Allow request headers 610 self.set_header("Access-Control-Allow-Headers", ",".join(allowed_headers)) 611 612 # Allow X-Auth-Token in responses 613 self.set_header("Access-Control-Expose-Headers", "X-Auth-Token") 614 615 # Allow all methods 616 self.set_header("Access-Control-Allow-Methods", "OPTIONS, GET, POST") 617 618 self.set_status(204) 619 self.finish() 620 621 622class SaltAuthHandler(BaseSaltAPIHandler): # pylint: disable=W0223 623 """ 624 Handler for login requests 625 """ 626 627 def get(self): # pylint: disable=arguments-differ 628 """ 629 All logins are done over post, this is a parked endpoint 630 631 .. http:get:: /login 632 633 :status 401: |401| 634 :status 406: |406| 635 636 **Example request:** 637 638 .. code-block:: bash 639 640 curl -i localhost:8000/login 641 642 .. code-block:: text 643 644 GET /login HTTP/1.1 645 Host: localhost:8000 646 Accept: application/json 647 648 **Example response:** 649 650 .. code-block:: text 651 652 HTTP/1.1 401 Unauthorized 653 Content-Type: application/json 654 Content-Length: 58 655 656 {"status": "401 Unauthorized", "return": "Please log in"} 657 """ 658 self.set_status(401) 659 self.set_header("WWW-Authenticate", "Session") 660 661 ret = {"status": "401 Unauthorized", "return": "Please log in"} 662 663 self.write(self.serialize(ret)) 664 665 # TODO: make asynchronous? Underlying library isn't... and we ARE making disk calls :( 666 def post(self): # pylint: disable=arguments-differ 667 """ 668 :ref:`Authenticate <rest_tornado-auth>` against Salt's eauth system 669 670 .. http:post:: /login 671 672 :reqheader X-Auth-Token: |req_token| 673 :reqheader Accept: |req_accept| 674 :reqheader Content-Type: |req_ct| 675 676 :form eauth: the eauth backend configured for the user 677 :form username: username 678 :form password: password 679 680 :status 200: |200| 681 :status 400: |400| 682 :status 401: |401| 683 :status 406: |406| 684 :status 500: |500| 685 686 **Example request:** 687 688 .. code-block:: bash 689 690 curl -si localhost:8000/login \\ 691 -H "Accept: application/json" \\ 692 -d username='saltuser' \\ 693 -d password='saltpass' \\ 694 -d eauth='pam' 695 696 .. code-block:: text 697 698 POST / HTTP/1.1 699 Host: localhost:8000 700 Content-Length: 42 701 Content-Type: application/x-www-form-urlencoded 702 Accept: application/json 703 704 username=saltuser&password=saltpass&eauth=pam 705 706 **Example response:** 707 708 .. code-block:: text 709 710 HTTP/1.1 200 OK 711 Content-Type: application/json 712 Content-Length: 206 713 X-Auth-Token: 6d1b722e 714 Set-Cookie: session_id=6d1b722e; expires=Sat, 17 Nov 2012 03:23:52 GMT; Path=/ 715 716 {"return": { 717 "token": "6d1b722e", 718 "start": 1363805943.776223, 719 "expire": 1363849143.776224, 720 "user": "saltuser", 721 "eauth": "pam", 722 "perms": [ 723 "grains.*", 724 "status.*", 725 "sys.*", 726 "test.*" 727 ] 728 }} 729 """ 730 try: 731 if not isinstance(self.request_payload, dict): 732 self.send_error(400) 733 return 734 735 creds = { 736 "username": self.request_payload["username"], 737 "password": self.request_payload["password"], 738 "eauth": self.request_payload["eauth"], 739 } 740 # if any of the args are missing, its a bad request 741 except KeyError: 742 self.send_error(400) 743 return 744 745 token = self.application.auth.mk_token(creds) 746 if "token" not in token: 747 # TODO: nicer error message 748 # 'Could not authenticate using provided credentials') 749 self.send_error(401) 750 # return since we don't want to execute any more 751 return 752 self.set_cookie(AUTH_COOKIE_NAME, token["token"]) 753 754 # Grab eauth config for the current backend for the current user 755 try: 756 eauth = self.application.opts["external_auth"][token["eauth"]] 757 # Get sum of '*' perms, user-specific perms, and group-specific perms 758 perms = eauth.get(token["name"], []) 759 perms.extend(eauth.get("*", [])) 760 761 if "groups" in token and token["groups"]: 762 user_groups = set(token["groups"]) 763 eauth_groups = {i.rstrip("%") for i in eauth.keys() if i.endswith("%")} 764 765 for group in user_groups & eauth_groups: 766 perms.extend(eauth["{}%".format(group)]) 767 768 perms = sorted(list(set(perms))) 769 # If we can't find the creds, then they aren't authorized 770 except KeyError: 771 self.send_error(401) 772 return 773 774 except (AttributeError, IndexError): 775 log.debug( 776 "Configuration for external_auth malformed for eauth '%s', " 777 "and user '%s'.", 778 token.get("eauth"), 779 token.get("name"), 780 exc_info=True, 781 ) 782 # TODO better error -- 'Configuration for external_auth could not be read.' 783 self.send_error(500) 784 return 785 786 ret = { 787 "return": [ 788 { 789 "token": token["token"], 790 "expire": token["expire"], 791 "start": token["start"], 792 "user": token["name"], 793 "eauth": token["eauth"], 794 "perms": perms, 795 } 796 ] 797 } 798 799 self.write(self.serialize(ret)) 800 801 802class SaltAPIHandler(BaseSaltAPIHandler): # pylint: disable=W0223 803 """ 804 Main API handler for base "/" 805 """ 806 807 def get(self): # pylint: disable=arguments-differ 808 """ 809 An endpoint to determine salt-api capabilities 810 811 .. http:get:: / 812 813 :reqheader Accept: |req_accept| 814 815 :status 200: |200| 816 :status 401: |401| 817 :status 406: |406| 818 819 **Example request:** 820 821 .. code-block:: bash 822 823 curl -i localhost:8000 824 825 .. code-block:: text 826 827 GET / HTTP/1.1 828 Host: localhost:8000 829 Accept: application/json 830 831 **Example response:** 832 833 .. code-block:: text 834 835 HTTP/1.1 200 OK 836 Content-Type: application/json 837 Content-Legnth: 83 838 839 {"clients": ["local", "local_async", "runner", "runner_async"], "return": "Welcome"} 840 """ 841 ret = {"clients": list(self.saltclients.keys()), "return": "Welcome"} 842 self.write(self.serialize(ret)) 843 844 @salt.ext.tornado.web.asynchronous 845 def post(self): # pylint: disable=arguments-differ 846 """ 847 Send one or more Salt commands (lowstates) in the request body 848 849 .. http:post:: / 850 851 :reqheader X-Auth-Token: |req_token| 852 :reqheader Accept: |req_accept| 853 :reqheader Content-Type: |req_ct| 854 855 :resheader Content-Type: |res_ct| 856 857 :status 200: |200| 858 :status 401: |401| 859 :status 406: |406| 860 861 :term:`lowstate` data describing Salt commands must be sent in the 862 request body. 863 864 **Example request:** 865 866 .. code-block:: bash 867 868 curl -si https://localhost:8000 \\ 869 -H "Accept: application/x-yaml" \\ 870 -H "X-Auth-Token: d40d1e1e" \\ 871 -d client=local \\ 872 -d tgt='*' \\ 873 -d fun='test.ping' \\ 874 -d arg 875 876 .. code-block:: text 877 878 POST / HTTP/1.1 879 Host: localhost:8000 880 Accept: application/x-yaml 881 X-Auth-Token: d40d1e1e 882 Content-Length: 36 883 Content-Type: application/x-www-form-urlencoded 884 885 fun=test.ping&arg&client=local&tgt=* 886 887 **Example response:** 888 889 Responses are an in-order list of the lowstate's return data. In the 890 event of an exception running a command the return will be a string 891 instead of a mapping. 892 893 .. code-block:: text 894 895 HTTP/1.1 200 OK 896 Content-Length: 200 897 Allow: GET, HEAD, POST 898 Content-Type: application/x-yaml 899 900 return: 901 - ms-0: true 902 ms-1: true 903 ms-2: true 904 ms-3: true 905 ms-4: true 906 907 .. admonition:: multiple commands 908 909 Note that if multiple :term:`lowstate` structures are sent, the Salt 910 API will execute them in serial, and will not stop execution upon failure 911 of a previous job. If you need to have commands executed in order and 912 stop on failure please use compound-command-execution. 913 914 """ 915 # if you aren't authenticated, redirect to login 916 if not self._verify_auth(): 917 self.redirect("/login") 918 return 919 920 self.disbatch() 921 922 @salt.ext.tornado.gen.coroutine 923 def disbatch(self): 924 """ 925 Disbatch all lowstates to the appropriate clients 926 """ 927 ret = [] 928 929 # check clients before going, we want to throw 400 if one is bad 930 for low in self.lowstate: 931 if not self._verify_client(low): 932 return 933 934 # Make sure we have 'token' or 'username'/'password' in each low chunk. 935 # Salt will verify the credentials are correct. 936 if self.token is not None and "token" not in low: 937 low["token"] = self.token 938 939 if not ( 940 ("token" in low) 941 or ("username" in low and "password" in low and "eauth" in low) 942 ): 943 ret.append("Failed to authenticate") 944 break 945 946 # disbatch to the correct handler 947 try: 948 chunk_ret = yield getattr(self, "_disbatch_{}".format(low["client"]))( 949 low 950 ) 951 ret.append(chunk_ret) 952 except (AuthenticationError, AuthorizationError, EauthAuthenticationError): 953 ret.append("Failed to authenticate") 954 break 955 except Exception as ex: # pylint: disable=broad-except 956 ret.append("Unexpected exception while handling request: {}".format(ex)) 957 log.error("Unexpected exception while handling request:", exc_info=True) 958 959 self.write(self.serialize({"return": ret})) 960 self.finish() 961 962 @salt.ext.tornado.gen.coroutine 963 def _disbatch_local(self, chunk): 964 """ 965 Dispatch local client commands 966 """ 967 # Generate jid and find all minions before triggering a job to subscribe all returns from minions 968 chunk["jid"] = ( 969 salt.utils.jid.gen_jid(self.application.opts) 970 if not chunk.get("jid", None) 971 else chunk["jid"] 972 ) 973 minions = set( 974 self.ckminions.check_minions(chunk["tgt"], chunk.get("tgt_type", "glob")) 975 ) 976 977 def subscribe_minion(minion): 978 salt_evt = self.application.event_listener.get_event( 979 self, 980 tag="salt/job/{}/ret/{}".format(chunk["jid"], minion), 981 matcher=EventListener.exact_matcher, 982 ) 983 syndic_evt = self.application.event_listener.get_event( 984 self, 985 tag="syndic/job/{}/ret/{}".format(chunk["jid"], minion), 986 matcher=EventListener.exact_matcher, 987 ) 988 return salt_evt, syndic_evt 989 990 # start listening for the event before we fire the job to avoid races 991 events = [] 992 for minion in minions: 993 salt_evt, syndic_evt = subscribe_minion(minion) 994 events.append(salt_evt) 995 events.append(syndic_evt) 996 997 f_call = self._format_call_run_job_async(chunk) 998 # fire a job off 999 pub_data = yield self.saltclients["local"]( 1000 *f_call.get("args", ()), **f_call.get("kwargs", {}) 1001 ) 1002 1003 # if the job didn't publish, lets not wait around for nothing 1004 # TODO: set header?? 1005 if "jid" not in pub_data: 1006 for future in events: 1007 try: 1008 future.set_result(None) 1009 except Exception: # pylint: disable=broad-except 1010 pass 1011 raise salt.ext.tornado.gen.Return( 1012 "No minions matched the target. No command was sent, no jid was" 1013 " assigned." 1014 ) 1015 1016 # get_event for missing minion 1017 for minion in list(set(pub_data["minions"]) - set(minions)): 1018 salt_evt, syndic_evt = subscribe_minion(minion) 1019 events.append(salt_evt) 1020 events.append(syndic_evt) 1021 1022 # Map of minion_id -> returned for all minions we think we need to wait on 1023 minions = {m: False for m in pub_data["minions"]} 1024 1025 # minimum time required for return to complete. By default no waiting, if 1026 # we are a syndic then we must wait syndic_wait at a minimum 1027 min_wait_time = Future() 1028 min_wait_time.set_result(True) 1029 1030 # wait syndic a while to avoid missing published events 1031 if self.application.opts["order_masters"]: 1032 min_wait_time = salt.ext.tornado.gen.sleep( 1033 self.application.opts["syndic_wait"] 1034 ) 1035 1036 # To ensure job_not_running and all_return are terminated by each other, communicate using a future 1037 is_finished = Future() 1038 1039 # ping until the job is not running, while doing so, if we see new minions returning 1040 # that they are running the job, add them to the list 1041 salt.ext.tornado.ioloop.IOLoop.current().spawn_callback( 1042 self.job_not_running, 1043 pub_data["jid"], 1044 chunk["tgt"], 1045 f_call["kwargs"]["tgt_type"], 1046 minions, 1047 is_finished, 1048 ) 1049 1050 def more_todo(): 1051 """ 1052 Check if there are any more minions we are waiting on returns from 1053 """ 1054 return any(x is False for x in minions.values()) 1055 1056 # here we want to follow the behavior of LocalClient.get_iter_returns 1057 # namely we want to wait at least syndic_wait (assuming we are a syndic) 1058 # and that there are no more jobs running on minions. We are allowed to exit 1059 # early if gather_job_timeout has been exceeded 1060 chunk_ret = {} 1061 while True: 1062 to_wait = events + [is_finished] 1063 if not min_wait_time.done(): 1064 to_wait += [min_wait_time] 1065 1066 def cancel_inflight_futures(): 1067 for event in to_wait: 1068 if not event.done(): 1069 event.set_result(None) 1070 1071 f = yield Any(to_wait) 1072 try: 1073 # When finished entire routine, cleanup other futures and return result 1074 if f is is_finished: 1075 cancel_inflight_futures() 1076 raise salt.ext.tornado.gen.Return(chunk_ret) 1077 elif f is min_wait_time: 1078 if not more_todo(): 1079 cancel_inflight_futures() 1080 raise salt.ext.tornado.gen.Return(chunk_ret) 1081 continue 1082 1083 f_result = f.result() 1084 if f in events: 1085 events.remove(f) 1086 # if this is a start, then we need to add it to the pile 1087 if f_result["tag"].endswith("/new"): 1088 for minion_id in f_result["data"]["minions"]: 1089 if minion_id not in minions: 1090 minions[minion_id] = False 1091 else: 1092 chunk_ret[f_result["data"]["id"]] = f_result["data"]["return"] 1093 # clear finished event future 1094 minions[f_result["data"]["id"]] = True 1095 # if there are no more minions to wait for, then we are done 1096 if not more_todo() and min_wait_time.done(): 1097 cancel_inflight_futures() 1098 raise salt.ext.tornado.gen.Return(chunk_ret) 1099 1100 except TimeoutException: 1101 pass 1102 1103 @salt.ext.tornado.gen.coroutine 1104 def job_not_running(self, jid, tgt, tgt_type, minions, is_finished): 1105 """ 1106 Return a future which will complete once jid (passed in) is no longer 1107 running on tgt 1108 """ 1109 local_client = self.saltclients["local"] 1110 ping_pub_data = yield local_client( 1111 tgt, "saltutil.find_job", [jid], tgt_type=tgt_type 1112 ) 1113 ping_tag = tagify([ping_pub_data["jid"], "ret"], "job") 1114 1115 minion_running = False 1116 while True: 1117 try: 1118 event = self.application.event_listener.get_event( 1119 self, 1120 tag=ping_tag, 1121 timeout=self.application.opts["gather_job_timeout"], 1122 ) 1123 event = yield event 1124 except TimeoutException: 1125 if not event.done(): 1126 event.set_result(None) 1127 1128 if not minion_running: 1129 raise salt.ext.tornado.gen.Return(True) 1130 else: 1131 ping_pub_data = yield local_client( 1132 tgt, "saltutil.find_job", [jid], tgt_type=tgt_type 1133 ) 1134 ping_tag = tagify([ping_pub_data["jid"], "ret"], "job") 1135 minion_running = False 1136 continue 1137 1138 # Minions can return, we want to see if the job is running... 1139 if event["data"].get("return", {}) == {}: 1140 continue 1141 if event["data"]["id"] not in minions: 1142 minions[event["data"]["id"]] = False 1143 minion_running = True 1144 1145 @salt.ext.tornado.gen.coroutine 1146 def _disbatch_local_async(self, chunk): 1147 """ 1148 Disbatch local client_async commands 1149 """ 1150 f_call = self._format_call_run_job_async(chunk) 1151 # fire a job off 1152 pub_data = yield self.saltclients["local_async"]( 1153 *f_call.get("args", ()), **f_call.get("kwargs", {}) 1154 ) 1155 1156 raise salt.ext.tornado.gen.Return(pub_data) 1157 1158 @salt.ext.tornado.gen.coroutine 1159 def _disbatch_runner(self, chunk): 1160 """ 1161 Disbatch runner client commands 1162 """ 1163 full_return = chunk.pop("full_return", False) 1164 pub_data = self.saltclients["runner"](chunk) 1165 tag = pub_data["tag"] + "/ret" 1166 try: 1167 event = yield self.application.event_listener.get_event(self, tag=tag) 1168 1169 # only return the return data 1170 ret = event if full_return else event["data"]["return"] 1171 raise salt.ext.tornado.gen.Return(ret) 1172 except TimeoutException: 1173 raise salt.ext.tornado.gen.Return("Timeout waiting for runner to execute") 1174 1175 @salt.ext.tornado.gen.coroutine 1176 def _disbatch_runner_async(self, chunk): 1177 """ 1178 Disbatch runner client_async commands 1179 """ 1180 pub_data = self.saltclients["runner"](chunk) 1181 raise salt.ext.tornado.gen.Return(pub_data) 1182 1183 # salt.utils.args.format_call doesn't work for functions having the 1184 # annotation salt.ext.tornado.gen.coroutine 1185 def _format_call_run_job_async(self, chunk): 1186 f_call = salt.utils.args.format_call( 1187 salt.client.LocalClient.run_job, chunk, is_class_method=True 1188 ) 1189 f_call.get("kwargs", {})["io_loop"] = salt.ext.tornado.ioloop.IOLoop.current() 1190 return f_call 1191 1192 1193class MinionSaltAPIHandler(SaltAPIHandler): # pylint: disable=W0223 1194 """ 1195 A convenience endpoint for minion related functions 1196 """ 1197 1198 @salt.ext.tornado.web.asynchronous 1199 def get(self, mid=None): # pylint: disable=W0221 1200 """ 1201 A convenience URL for getting lists of minions or getting minion 1202 details 1203 1204 .. http:get:: /minions/(mid) 1205 1206 :reqheader X-Auth-Token: |req_token| 1207 :reqheader Accept: |req_accept| 1208 1209 :status 200: |200| 1210 :status 401: |401| 1211 :status 406: |406| 1212 1213 **Example request:** 1214 1215 .. code-block:: bash 1216 1217 curl -i localhost:8000/minions/ms-3 1218 1219 .. code-block:: text 1220 1221 GET /minions/ms-3 HTTP/1.1 1222 Host: localhost:8000 1223 Accept: application/x-yaml 1224 1225 **Example response:** 1226 1227 .. code-block:: text 1228 1229 HTTP/1.1 200 OK 1230 Content-Length: 129005 1231 Content-Type: application/x-yaml 1232 1233 return: 1234 - ms-3: 1235 grains.items: 1236 ... 1237 """ 1238 # if you aren't authenticated, redirect to login 1239 if not self._verify_auth(): 1240 self.redirect("/login") 1241 return 1242 1243 self.lowstate = [{"client": "local", "tgt": mid or "*", "fun": "grains.items"}] 1244 self.disbatch() 1245 1246 @salt.ext.tornado.web.asynchronous 1247 def post(self): 1248 """ 1249 Start an execution command and immediately return the job id 1250 1251 .. http:post:: /minions 1252 1253 :reqheader X-Auth-Token: |req_token| 1254 :reqheader Accept: |req_accept| 1255 :reqheader Content-Type: |req_ct| 1256 1257 :resheader Content-Type: |res_ct| 1258 1259 :status 200: |200| 1260 :status 401: |401| 1261 :status 406: |406| 1262 1263 :term:`lowstate` data describing Salt commands must be sent in the 1264 request body. The ``client`` option will be set to 1265 :py:meth:`~salt.client.LocalClient.local_async`. 1266 1267 **Example request:** 1268 1269 .. code-block:: bash 1270 1271 curl -sSi localhost:8000/minions \\ 1272 -H "Accept: application/x-yaml" \\ 1273 -d tgt='*' \\ 1274 -d fun='status.diskusage' 1275 1276 .. code-block:: text 1277 1278 POST /minions HTTP/1.1 1279 Host: localhost:8000 1280 Accept: application/x-yaml 1281 Content-Length: 26 1282 Content-Type: application/x-www-form-urlencoded 1283 1284 tgt=*&fun=status.diskusage 1285 1286 **Example response:** 1287 1288 .. code-block:: text 1289 1290 HTTP/1.1 202 Accepted 1291 Content-Length: 86 1292 Content-Type: application/x-yaml 1293 1294 return: 1295 - jid: '20130603122505459265' 1296 minions: [ms-4, ms-3, ms-2, ms-1, ms-0] 1297 """ 1298 # if you aren't authenticated, redirect to login 1299 if not self._verify_auth(): 1300 self.redirect("/login") 1301 return 1302 1303 # verify that all lowstates are the correct client type 1304 for low in self.lowstate: 1305 # if you didn't specify, its fine 1306 if "client" not in low: 1307 low["client"] = "local_async" 1308 continue 1309 # if you specified something else, we don't do that 1310 if low.get("client") != "local_async": 1311 self.set_status(400) 1312 self.write("We don't serve your kind here") 1313 self.finish() 1314 return 1315 1316 self.disbatch() 1317 1318 1319class JobsSaltAPIHandler(SaltAPIHandler): # pylint: disable=W0223 1320 """ 1321 A convenience endpoint for job cache data 1322 """ 1323 1324 @salt.ext.tornado.web.asynchronous 1325 def get(self, jid=None): # pylint: disable=W0221 1326 """ 1327 A convenience URL for getting lists of previously run jobs or getting 1328 the return from a single job 1329 1330 .. http:get:: /jobs/(jid) 1331 1332 List jobs or show a single job from the job cache. 1333 1334 :status 200: |200| 1335 :status 401: |401| 1336 :status 406: |406| 1337 1338 **Example request:** 1339 1340 .. code-block:: bash 1341 1342 curl -i localhost:8000/jobs 1343 1344 .. code-block:: text 1345 1346 GET /jobs HTTP/1.1 1347 Host: localhost:8000 1348 Accept: application/x-yaml 1349 1350 **Example response:** 1351 1352 .. code-block:: text 1353 1354 HTTP/1.1 200 OK 1355 Content-Length: 165 1356 Content-Type: application/x-yaml 1357 1358 return: 1359 - '20121130104633606931': 1360 Arguments: 1361 - '3' 1362 Function: test.fib 1363 Start Time: 2012, Nov 30 10:46:33.606931 1364 Target: jerry 1365 Target-type: glob 1366 1367 **Example request:** 1368 1369 .. code-block:: bash 1370 1371 curl -i localhost:8000/jobs/20121130104633606931 1372 1373 .. code-block:: text 1374 1375 GET /jobs/20121130104633606931 HTTP/1.1 1376 Host: localhost:8000 1377 Accept: application/x-yaml 1378 1379 **Example response:** 1380 1381 .. code-block:: text 1382 1383 HTTP/1.1 200 OK 1384 Content-Length: 73 1385 Content-Type: application/x-yaml 1386 1387 info: 1388 - Arguments: 1389 - '3' 1390 Function: test.fib 1391 Minions: 1392 - jerry 1393 Start Time: 2012, Nov 30 10:46:33.606931 1394 Target: '*' 1395 Target-type: glob 1396 User: saltdev 1397 jid: '20121130104633606931' 1398 return: 1399 - jerry: 1400 - - 0 1401 - 1 1402 - 1 1403 - 2 1404 - 6.9141387939453125e-06 1405 """ 1406 # if you aren't authenticated, redirect to login 1407 if not self._verify_auth(): 1408 self.redirect("/login") 1409 return 1410 1411 if jid: 1412 self.lowstate = [{"fun": "jobs.list_job", "jid": jid, "client": "runner"}] 1413 else: 1414 self.lowstate = [{"fun": "jobs.list_jobs", "client": "runner"}] 1415 1416 self.disbatch() 1417 1418 1419class RunSaltAPIHandler(SaltAPIHandler): # pylint: disable=W0223 1420 """ 1421 Endpoint to run commands without normal session handling 1422 """ 1423 1424 @salt.ext.tornado.web.asynchronous 1425 def post(self): 1426 """ 1427 Run commands bypassing the :ref:`normal session handling 1428 <rest_cherrypy-auth>` 1429 1430 .. http:post:: /run 1431 1432 This entry point is primarily for "one-off" commands. Each request 1433 must pass full Salt authentication credentials. Otherwise this URL 1434 is identical to the :py:meth:`root URL (/) <LowDataAdapter.POST>`. 1435 1436 :term:`lowstate` data describing Salt commands must be sent in the 1437 request body. 1438 1439 :status 200: |200| 1440 :status 401: |401| 1441 :status 406: |406| 1442 1443 **Example request:** 1444 1445 .. code-block:: bash 1446 1447 curl -sS localhost:8000/run \\ 1448 -H 'Accept: application/x-yaml' \\ 1449 -d client='local' \\ 1450 -d tgt='*' \\ 1451 -d fun='test.ping' \\ 1452 -d username='saltdev' \\ 1453 -d password='saltdev' \\ 1454 -d eauth='pam' 1455 1456 .. code-block:: text 1457 1458 POST /run HTTP/1.1 1459 Host: localhost:8000 1460 Accept: application/x-yaml 1461 Content-Length: 75 1462 Content-Type: application/x-www-form-urlencoded 1463 1464 client=local&tgt=*&fun=test.ping&username=saltdev&password=saltdev&eauth=pam 1465 1466 **Example response:** 1467 1468 .. code-block:: text 1469 1470 HTTP/1.1 200 OK 1471 Content-Length: 73 1472 Content-Type: application/x-yaml 1473 1474 return: 1475 - ms-0: true 1476 ms-1: true 1477 ms-2: true 1478 ms-3: true 1479 ms-4: true 1480 """ 1481 self.disbatch() 1482 1483 1484class EventsSaltAPIHandler(SaltAPIHandler): # pylint: disable=W0223 1485 """ 1486 Expose the Salt event bus 1487 1488 The event bus on the Salt master exposes a large variety of things, notably 1489 when executions are started on the master and also when minions ultimately 1490 return their results. This URL provides a real-time window into a running 1491 Salt infrastructure. 1492 1493 .. seealso:: :ref:`events` 1494 """ 1495 1496 @salt.ext.tornado.gen.coroutine 1497 def get(self): 1498 r""" 1499 An HTTP stream of the Salt master event bus 1500 1501 This stream is formatted per the Server Sent Events (SSE) spec. Each 1502 event is formatted as JSON. 1503 1504 .. http:get:: /events 1505 1506 :status 200: |200| 1507 :status 401: |401| 1508 :status 406: |406| 1509 1510 **Example request:** 1511 1512 .. code-block:: bash 1513 1514 curl -NsS localhost:8000/events 1515 1516 .. code-block:: text 1517 1518 GET /events HTTP/1.1 1519 Host: localhost:8000 1520 1521 **Example response:** 1522 1523 .. code-block:: text 1524 1525 HTTP/1.1 200 OK 1526 Connection: keep-alive 1527 Cache-Control: no-cache 1528 Content-Type: text/event-stream;charset=utf-8 1529 1530 retry: 400 1531 data: {'tag': '', 'data': {'minions': ['ms-4', 'ms-3', 'ms-2', 'ms-1', 'ms-0']}} 1532 1533 data: {'tag': '20130802115730568475', 'data': {'jid': '20130802115730568475', 'return': True, 'retcode': 0, 'success': True, 'cmd': '_return', 'fun': 'test.ping', 'id': 'ms-1'}} 1534 1535 The event stream can be easily consumed via JavaScript: 1536 1537 .. code-block:: javascript 1538 1539 <!-- Note, you must be authenticated! --> 1540 var source = new EventSource('/events'); 1541 source.onopen = function() { console.debug('opening') }; 1542 source.onerror = function(e) { console.debug('error!', e) }; 1543 source.onmessage = function(e) { console.debug(e.data) }; 1544 1545 Or using CORS: 1546 1547 .. code-block:: javascript 1548 1549 var source = new EventSource('/events', {withCredentials: true}); 1550 1551 Some browser clients lack CORS support for the ``EventSource()`` API. Such 1552 clients may instead pass the :mailheader:`X-Auth-Token` value as an URL 1553 parameter: 1554 1555 .. code-block:: bash 1556 1557 curl -NsS localhost:8000/events/6d1b722e 1558 1559 It is also possible to consume the stream via the shell. 1560 1561 Records are separated by blank lines; the ``data:`` and ``tag:`` 1562 prefixes will need to be removed manually before attempting to 1563 unserialize the JSON. 1564 1565 curl's ``-N`` flag turns off input buffering which is required to 1566 process the stream incrementally. 1567 1568 Here is a basic example of printing each event as it comes in: 1569 1570 .. code-block:: bash 1571 1572 curl -NsS localhost:8000/events |\ 1573 while IFS= read -r line ; do 1574 echo $line 1575 done 1576 1577 Here is an example of using awk to filter events based on tag: 1578 1579 .. code-block:: bash 1580 1581 curl -NsS localhost:8000/events |\ 1582 awk ' 1583 BEGIN { RS=""; FS="\\n" } 1584 $1 ~ /^tag: salt\/job\/[0-9]+\/new$/ { print $0 } 1585 ' 1586 tag: salt/job/20140112010149808995/new 1587 data: {"tag": "salt/job/20140112010149808995/new", "data": {"tgt_type": "glob", "jid": "20140112010149808995", "tgt": "jerry", "_stamp": "2014-01-12_01:01:49.809617", "user": "shouse", "arg": [], "fun": "test.ping", "minions": ["jerry"]}} 1588 tag: 20140112010149808995 1589 data: {"tag": "20140112010149808995", "data": {"fun_args": [], "jid": "20140112010149808995", "return": true, "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2014-01-12_01:01:49.819316", "fun": "test.ping", "id": "jerry"}} 1590 """ 1591 # if you aren't authenticated, redirect to login 1592 if not self._verify_auth(): 1593 self.redirect("/login") 1594 return 1595 # set the streaming headers 1596 self.set_header("Content-Type", "text/event-stream") 1597 self.set_header("Cache-Control", "no-cache") 1598 self.set_header("Connection", "keep-alive") 1599 1600 self.write("retry: {}\n".format(400)) 1601 self.flush() 1602 1603 while True: 1604 try: 1605 event = yield self.application.event_listener.get_event(self) 1606 self.write("tag: {}\n".format(event.get("tag", ""))) 1607 self.write("data: {}\n\n".format(_json_dumps(event))) 1608 self.flush() 1609 except TimeoutException: 1610 break 1611 1612 1613class WebhookSaltAPIHandler(SaltAPIHandler): # pylint: disable=W0223 1614 """ 1615 A generic web hook entry point that fires an event on Salt's event bus 1616 1617 External services can POST data to this URL to trigger an event in Salt. 1618 For example, Amazon SNS, Jenkins-CI or Travis-CI, or GitHub web hooks. 1619 1620 .. note:: Be mindful of security 1621 1622 Salt's Reactor can run any code. A Reactor SLS that responds to a hook 1623 event is responsible for validating that the event came from a trusted 1624 source and contains valid data. 1625 1626 **This is a generic interface and securing it is up to you!** 1627 1628 This URL requires authentication however not all external services can 1629 be configured to authenticate. For this reason authentication can be 1630 selectively disabled for this URL. Follow best practices -- always use 1631 SSL, pass a secret key, configure the firewall to only allow traffic 1632 from a known source, etc. 1633 1634 The event data is taken from the request body. The 1635 :mailheader:`Content-Type` header is respected for the payload. 1636 1637 The event tag is prefixed with ``salt/netapi/hook`` and the URL path is 1638 appended to the end. For example, a ``POST`` request sent to 1639 ``/hook/mycompany/myapp/mydata`` will produce a Salt event with the tag 1640 ``salt/netapi/hook/mycompany/myapp/mydata``. 1641 1642 The following is an example ``.travis.yml`` file to send notifications to 1643 Salt of successful test runs: 1644 1645 .. code-block:: yaml 1646 1647 language: python 1648 script: python -m unittest tests 1649 after_success: 1650 - 'curl -sS http://saltapi-url.example.com:8000/hook/travis/build/success -d branch="${TRAVIS_BRANCH}" -d commit="${TRAVIS_COMMIT}"' 1651 1652 .. seealso:: :ref:`Events <events>`, :ref:`Reactor <reactor>` 1653 """ 1654 1655 def post(self, tag_suffix=None): # pylint: disable=W0221 1656 """ 1657 Fire an event in Salt with a custom event tag and data 1658 1659 .. http:post:: /hook 1660 1661 :status 200: |200| 1662 :status 401: |401| 1663 :status 406: |406| 1664 :status 413: request body is too large 1665 1666 **Example request:** 1667 1668 .. code-block:: bash 1669 1670 curl -sS localhost:8000/hook -d foo='Foo!' -d bar='Bar!' 1671 1672 .. code-block:: text 1673 1674 POST /hook HTTP/1.1 1675 Host: localhost:8000 1676 Content-Length: 16 1677 Content-Type: application/x-www-form-urlencoded 1678 1679 foo=Foo&bar=Bar! 1680 1681 **Example response**: 1682 1683 .. code-block:: text 1684 1685 HTTP/1.1 200 OK 1686 Content-Length: 14 1687 Content-Type: application/json 1688 1689 {"success": true} 1690 1691 As a practical example, an internal continuous-integration build 1692 server could send an HTTP POST request to the URL 1693 ``http://localhost:8000/hook/mycompany/build/success`` which contains 1694 the result of a build and the SHA of the version that was built as 1695 JSON. That would then produce the following event in Salt that could be 1696 used to kick off a deployment via Salt's Reactor: 1697 1698 .. code-block:: text 1699 1700 Event fired at Fri Feb 14 17:40:11 2014 1701 ************************* 1702 Tag: salt/netapi/hook/mycompany/build/success 1703 Data: 1704 {'_stamp': '2014-02-14_17:40:11.440996', 1705 'headers': { 1706 'X-My-Secret-Key': 'F0fAgoQjIT@W', 1707 'Content-Length': '37', 1708 'Content-Type': 'application/json', 1709 'Host': 'localhost:8000', 1710 'Remote-Addr': '127.0.0.1'}, 1711 'post': {'revision': 'aa22a3c4b2e7', 'result': True}} 1712 1713 Salt's Reactor could listen for the event: 1714 1715 .. code-block:: yaml 1716 1717 reactor: 1718 - 'salt/netapi/hook/mycompany/build/*': 1719 - /srv/reactor/react_ci_builds.sls 1720 1721 And finally deploy the new build: 1722 1723 .. code-block:: jinja 1724 1725 {% set secret_key = data.get('headers', {}).get('X-My-Secret-Key') %} 1726 {% set build = data.get('post', {}) %} 1727 1728 {% if secret_key == 'F0fAgoQjIT@W' and build.result == True %} 1729 deploy_my_app: 1730 cmd.state.sls: 1731 - tgt: 'application*' 1732 - arg: 1733 - myapp.deploy 1734 - kwarg: 1735 pillar: 1736 revision: {{ revision }} 1737 {% endif %} 1738 """ 1739 disable_auth = self.application.mod_opts.get("webhook_disable_auth") 1740 if not disable_auth and not self._verify_auth(): 1741 self.redirect("/login") 1742 return 1743 1744 # if you have the tag, prefix 1745 tag = "salt/netapi/hook" 1746 if tag_suffix: 1747 tag += tag_suffix 1748 1749 # TODO: consolidate?? 1750 self.event = salt.utils.event.get_event( 1751 "master", 1752 self.application.opts["sock_dir"], 1753 self.application.opts["transport"], 1754 opts=self.application.opts, 1755 listen=False, 1756 ) 1757 1758 arguments = {} 1759 for argname in self.request.query_arguments: 1760 value = self.get_arguments(argname) 1761 if len(value) == 1: 1762 value = value[0] 1763 arguments[argname] = value 1764 ret = self.event.fire_event( 1765 { 1766 "post": self.request_payload, 1767 "get": arguments, 1768 # In Tornado >= v4.0.3, the headers come 1769 # back as an HTTPHeaders instance, which 1770 # is a dictionary. We must cast this as 1771 # a dictionary in order for msgpack to 1772 # serialize it. 1773 "headers": dict(self.request.headers), 1774 }, 1775 tag, 1776 ) 1777 1778 self.write(self.serialize({"success": ret})) 1779 1780 1781def _check_cors_origin(origin, allowed_origins): 1782 """ 1783 Check if an origin match cors allowed origins 1784 """ 1785 if isinstance(allowed_origins, list): 1786 if origin in allowed_origins: 1787 return origin 1788 elif allowed_origins == "*": 1789 return allowed_origins 1790 elif allowed_origins == origin: 1791 # Cors origin is either * or specific origin 1792 return allowed_origins 1793