1 2.. _expressions: 3 4*********** 5Expressions 6*********** 7 8.. index:: expression, BNF 9 10This chapter explains the meaning of the elements of expressions in Python. 11 12**Syntax Notes:** In this and the following chapters, extended BNF notation will 13be used to describe syntax, not lexical analysis. When (one alternative of) a 14syntax rule has the form 15 16.. productionlist:: python-grammar 17 name: `othername` 18 19and no semantics are given, the semantics of this form of ``name`` are the same 20as for ``othername``. 21 22 23.. _conversions: 24 25Arithmetic conversions 26====================== 27 28.. index:: pair: arithmetic; conversion 29 30When a description of an arithmetic operator below uses the phrase "the numeric 31arguments are converted to a common type", this means that the operator 32implementation for built-in types works as follows: 33 34* If either argument is a complex number, the other is converted to complex; 35 36* otherwise, if either argument is a floating point number, the other is 37 converted to floating point; 38 39* otherwise, both must be integers and no conversion is necessary. 40 41Some additional rules apply for certain operators (e.g., a string as a left 42argument to the '%' operator). Extensions must define their own conversion 43behavior. 44 45 46.. _atoms: 47 48Atoms 49===== 50 51.. index:: atom 52 53Atoms are the most basic elements of expressions. The simplest atoms are 54identifiers or literals. Forms enclosed in parentheses, brackets or braces are 55also categorized syntactically as atoms. The syntax for atoms is: 56 57.. productionlist:: python-grammar 58 atom: `identifier` | `literal` | `enclosure` 59 enclosure: `parenth_form` | `list_display` | `dict_display` | `set_display` 60 : | `generator_expression` | `yield_atom` 61 62 63.. _atom-identifiers: 64 65Identifiers (Names) 66------------------- 67 68.. index:: name, identifier 69 70An identifier occurring as an atom is a name. See section :ref:`identifiers` 71for lexical definition and section :ref:`naming` for documentation of naming and 72binding. 73 74.. index:: exception: NameError 75 76When the name is bound to an object, evaluation of the atom yields that object. 77When a name is not bound, an attempt to evaluate it raises a :exc:`NameError` 78exception. 79 80.. _private-name-mangling: 81 82.. index:: 83 pair: name; mangling 84 pair: private; names 85 86**Private name mangling:** When an identifier that textually occurs in a class 87definition begins with two or more underscore characters and does not end in two 88or more underscores, it is considered a :dfn:`private name` of that class. 89Private names are transformed to a longer form before code is generated for 90them. The transformation inserts the class name, with leading underscores 91removed and a single underscore inserted, in front of the name. For example, 92the identifier ``__spam`` occurring in a class named ``Ham`` will be transformed 93to ``_Ham__spam``. This transformation is independent of the syntactical 94context in which the identifier is used. If the transformed name is extremely 95long (longer than 255 characters), implementation defined truncation may happen. 96If the class name consists only of underscores, no transformation is done. 97 98 99.. _atom-literals: 100 101Literals 102-------- 103 104.. index:: single: literal 105 106Python supports string and bytes literals and various numeric literals: 107 108.. productionlist:: python-grammar 109 literal: `stringliteral` | `bytesliteral` 110 : | `integer` | `floatnumber` | `imagnumber` 111 112Evaluation of a literal yields an object of the given type (string, bytes, 113integer, floating point number, complex number) with the given value. The value 114may be approximated in the case of floating point and imaginary (complex) 115literals. See section :ref:`literals` for details. 116 117.. index:: 118 triple: immutable; data; type 119 pair: immutable; object 120 121All literals correspond to immutable data types, and hence the object's identity 122is less important than its value. Multiple evaluations of literals with the 123same value (either the same occurrence in the program text or a different 124occurrence) may obtain the same object or a different object with the same 125value. 126 127 128.. _parenthesized: 129 130Parenthesized forms 131------------------- 132 133.. index:: 134 single: parenthesized form 135 single: () (parentheses); tuple display 136 137A parenthesized form is an optional expression list enclosed in parentheses: 138 139.. productionlist:: python-grammar 140 parenth_form: "(" [`starred_expression`] ")" 141 142A parenthesized expression list yields whatever that expression list yields: if 143the list contains at least one comma, it yields a tuple; otherwise, it yields 144the single expression that makes up the expression list. 145 146.. index:: pair: empty; tuple 147 148An empty pair of parentheses yields an empty tuple object. Since tuples are 149immutable, the same rules as for literals apply (i.e., two occurrences of the empty 150tuple may or may not yield the same object). 151 152.. index:: 153 single: comma 154 single: , (comma) 155 156Note that tuples are not formed by the parentheses, but rather by use of the 157comma operator. The exception is the empty tuple, for which parentheses *are* 158required --- allowing unparenthesized "nothing" in expressions would cause 159ambiguities and allow common typos to pass uncaught. 160 161 162.. _comprehensions: 163 164Displays for lists, sets and dictionaries 165----------------------------------------- 166 167.. index:: single: comprehensions 168 169For constructing a list, a set or a dictionary Python provides special syntax 170called "displays", each of them in two flavors: 171 172* either the container contents are listed explicitly, or 173 174* they are computed via a set of looping and filtering instructions, called a 175 :dfn:`comprehension`. 176 177.. index:: 178 single: for; in comprehensions 179 single: if; in comprehensions 180 single: async for; in comprehensions 181 182Common syntax elements for comprehensions are: 183 184.. productionlist:: python-grammar 185 comprehension: `assignment_expression` `comp_for` 186 comp_for: ["async"] "for" `target_list` "in" `or_test` [`comp_iter`] 187 comp_iter: `comp_for` | `comp_if` 188 comp_if: "if" `or_test` [`comp_iter`] 189 190The comprehension consists of a single expression followed by at least one 191:keyword:`!for` clause and zero or more :keyword:`!for` or :keyword:`!if` clauses. 192In this case, the elements of the new container are those that would be produced 193by considering each of the :keyword:`!for` or :keyword:`!if` clauses a block, 194nesting from left to right, and evaluating the expression to produce an element 195each time the innermost block is reached. 196 197However, aside from the iterable expression in the leftmost :keyword:`!for` clause, 198the comprehension is executed in a separate implicitly nested scope. This ensures 199that names assigned to in the target list don't "leak" into the enclosing scope. 200 201The iterable expression in the leftmost :keyword:`!for` clause is evaluated 202directly in the enclosing scope and then passed as an argument to the implicitly 203nested scope. Subsequent :keyword:`!for` clauses and any filter condition in the 204leftmost :keyword:`!for` clause cannot be evaluated in the enclosing scope as 205they may depend on the values obtained from the leftmost iterable. For example: 206``[x*y for x in range(10) for y in range(x, x+10)]``. 207 208To ensure the comprehension always results in a container of the appropriate 209type, ``yield`` and ``yield from`` expressions are prohibited in the implicitly 210nested scope. 211 212.. index:: 213 single: await; in comprehensions 214 215Since Python 3.6, in an :keyword:`async def` function, an :keyword:`!async for` 216clause may be used to iterate over a :term:`asynchronous iterator`. 217A comprehension in an :keyword:`!async def` function may consist of either a 218:keyword:`!for` or :keyword:`!async for` clause following the leading 219expression, may contain additional :keyword:`!for` or :keyword:`!async for` 220clauses, and may also use :keyword:`await` expressions. 221If a comprehension contains either :keyword:`!async for` clauses or 222:keyword:`!await` expressions or other asynchronous comprehensions it is called 223an :dfn:`asynchronous comprehension`. An asynchronous comprehension may 224suspend the execution of the coroutine function in which it appears. 225See also :pep:`530`. 226 227.. versionadded:: 3.6 228 Asynchronous comprehensions were introduced. 229 230.. versionchanged:: 3.8 231 ``yield`` and ``yield from`` prohibited in the implicitly nested scope. 232 233.. versionchanged:: 3.11 234 Asynchronous comprehensions are now allowed inside comprehensions in 235 asynchronous functions. Outer comprehensions implicitly become 236 asynchronous. 237 238 239.. _lists: 240 241List displays 242------------- 243 244.. index:: 245 pair: list; display 246 pair: list; comprehensions 247 pair: empty; list 248 object: list 249 single: [] (square brackets); list expression 250 single: , (comma); expression list 251 252A list display is a possibly empty series of expressions enclosed in square 253brackets: 254 255.. productionlist:: python-grammar 256 list_display: "[" [`starred_list` | `comprehension`] "]" 257 258A list display yields a new list object, the contents being specified by either 259a list of expressions or a comprehension. When a comma-separated list of 260expressions is supplied, its elements are evaluated from left to right and 261placed into the list object in that order. When a comprehension is supplied, 262the list is constructed from the elements resulting from the comprehension. 263 264 265.. _set: 266 267Set displays 268------------ 269 270.. index:: 271 pair: set; display 272 pair: set; comprehensions 273 object: set 274 single: {} (curly brackets); set expression 275 single: , (comma); expression list 276 277A set display is denoted by curly braces and distinguishable from dictionary 278displays by the lack of colons separating keys and values: 279 280.. productionlist:: python-grammar 281 set_display: "{" (`starred_list` | `comprehension`) "}" 282 283A set display yields a new mutable set object, the contents being specified by 284either a sequence of expressions or a comprehension. When a comma-separated 285list of expressions is supplied, its elements are evaluated from left to right 286and added to the set object. When a comprehension is supplied, the set is 287constructed from the elements resulting from the comprehension. 288 289An empty set cannot be constructed with ``{}``; this literal constructs an empty 290dictionary. 291 292 293.. _dict: 294 295Dictionary displays 296------------------- 297 298.. index:: 299 pair: dictionary; display 300 pair: dictionary; comprehensions 301 key, datum, key/datum pair 302 object: dictionary 303 single: {} (curly brackets); dictionary expression 304 single: : (colon); in dictionary expressions 305 single: , (comma); in dictionary displays 306 307A dictionary display is a possibly empty series of key/datum pairs enclosed in 308curly braces: 309 310.. productionlist:: python-grammar 311 dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" 312 key_datum_list: `key_datum` ("," `key_datum`)* [","] 313 key_datum: `expression` ":" `expression` | "**" `or_expr` 314 dict_comprehension: `expression` ":" `expression` `comp_for` 315 316A dictionary display yields a new dictionary object. 317 318If a comma-separated sequence of key/datum pairs is given, they are evaluated 319from left to right to define the entries of the dictionary: each key object is 320used as a key into the dictionary to store the corresponding datum. This means 321that you can specify the same key multiple times in the key/datum list, and the 322final dictionary's value for that key will be the last one given. 323 324.. index:: 325 unpacking; dictionary 326 single: **; in dictionary displays 327 328A double asterisk ``**`` denotes :dfn:`dictionary unpacking`. 329Its operand must be a :term:`mapping`. Each mapping item is added 330to the new dictionary. Later values replace values already set by 331earlier key/datum pairs and earlier dictionary unpackings. 332 333.. versionadded:: 3.5 334 Unpacking into dictionary displays, originally proposed by :pep:`448`. 335 336A dict comprehension, in contrast to list and set comprehensions, needs two 337expressions separated with a colon followed by the usual "for" and "if" clauses. 338When the comprehension is run, the resulting key and value elements are inserted 339in the new dictionary in the order they are produced. 340 341.. index:: pair: immutable; object 342 hashable 343 344Restrictions on the types of the key values are listed earlier in section 345:ref:`types`. (To summarize, the key type should be :term:`hashable`, which excludes 346all mutable objects.) Clashes between duplicate keys are not detected; the last 347datum (textually rightmost in the display) stored for a given key value 348prevails. 349 350.. versionchanged:: 3.8 351 Prior to Python 3.8, in dict comprehensions, the evaluation order of key 352 and value was not well-defined. In CPython, the value was evaluated before 353 the key. Starting with 3.8, the key is evaluated before the value, as 354 proposed by :pep:`572`. 355 356 357.. _genexpr: 358 359Generator expressions 360--------------------- 361 362.. index:: 363 pair: generator; expression 364 object: generator 365 single: () (parentheses); generator expression 366 367A generator expression is a compact generator notation in parentheses: 368 369.. productionlist:: python-grammar 370 generator_expression: "(" `expression` `comp_for` ")" 371 372A generator expression yields a new generator object. Its syntax is the same as 373for comprehensions, except that it is enclosed in parentheses instead of 374brackets or curly braces. 375 376Variables used in the generator expression are evaluated lazily when the 377:meth:`~generator.__next__` method is called for the generator object (in the same 378fashion as normal generators). However, the iterable expression in the 379leftmost :keyword:`!for` clause is immediately evaluated, so that an error 380produced by it will be emitted at the point where the generator expression 381is defined, rather than at the point where the first value is retrieved. 382Subsequent :keyword:`!for` clauses and any filter condition in the leftmost 383:keyword:`!for` clause cannot be evaluated in the enclosing scope as they may 384depend on the values obtained from the leftmost iterable. For example: 385``(x*y for x in range(10) for y in range(x, x+10))``. 386 387The parentheses can be omitted on calls with only one argument. See section 388:ref:`calls` for details. 389 390To avoid interfering with the expected operation of the generator expression 391itself, ``yield`` and ``yield from`` expressions are prohibited in the 392implicitly defined generator. 393 394If a generator expression contains either :keyword:`!async for` 395clauses or :keyword:`await` expressions it is called an 396:dfn:`asynchronous generator expression`. An asynchronous generator 397expression returns a new asynchronous generator object, 398which is an asynchronous iterator (see :ref:`async-iterators`). 399 400.. versionadded:: 3.6 401 Asynchronous generator expressions were introduced. 402 403.. versionchanged:: 3.7 404 Prior to Python 3.7, asynchronous generator expressions could 405 only appear in :keyword:`async def` coroutines. Starting 406 with 3.7, any function can use asynchronous generator expressions. 407 408.. versionchanged:: 3.8 409 ``yield`` and ``yield from`` prohibited in the implicitly nested scope. 410 411 412.. _yieldexpr: 413 414Yield expressions 415----------------- 416 417.. index:: 418 keyword: yield 419 keyword: from 420 pair: yield; expression 421 pair: generator; function 422 423.. productionlist:: python-grammar 424 yield_atom: "(" `yield_expression` ")" 425 yield_expression: "yield" [`expression_list` | "from" `expression`] 426 427The yield expression is used when defining a :term:`generator` function 428or an :term:`asynchronous generator` function and 429thus can only be used in the body of a function definition. Using a yield 430expression in a function's body causes that function to be a generator, 431and using it in an :keyword:`async def` function's body causes that 432coroutine function to be an asynchronous generator. For example:: 433 434 def gen(): # defines a generator function 435 yield 123 436 437 async def agen(): # defines an asynchronous generator function 438 yield 123 439 440Due to their side effects on the containing scope, ``yield`` expressions 441are not permitted as part of the implicitly defined scopes used to 442implement comprehensions and generator expressions. 443 444.. versionchanged:: 3.8 445 Yield expressions prohibited in the implicitly nested scopes used to 446 implement comprehensions and generator expressions. 447 448Generator functions are described below, while asynchronous generator 449functions are described separately in section 450:ref:`asynchronous-generator-functions`. 451 452When a generator function is called, it returns an iterator known as a 453generator. That generator then controls the execution of the generator 454function. The execution starts when one of the generator's methods is called. 455At that time, the execution proceeds to the first yield expression, where it is 456suspended again, returning the value of :token:`~python-grammar:expression_list` 457to the generator's caller. By suspended, we mean that all local state is 458retained, including the current bindings of local variables, the instruction 459pointer, the internal evaluation stack, and the state of any exception handling. 460When the execution is resumed by calling one of the generator's methods, the 461function can proceed exactly as if the yield expression were just another 462external call. The value of the yield expression after resuming depends on the 463method which resumed the execution. If :meth:`~generator.__next__` is used 464(typically via either a :keyword:`for` or the :func:`next` builtin) then the 465result is :const:`None`. Otherwise, if :meth:`~generator.send` is used, then 466the result will be the value passed in to that method. 467 468.. index:: single: coroutine 469 470All of this makes generator functions quite similar to coroutines; they yield 471multiple times, they have more than one entry point and their execution can be 472suspended. The only difference is that a generator function cannot control 473where the execution should continue after it yields; the control is always 474transferred to the generator's caller. 475 476Yield expressions are allowed anywhere in a :keyword:`try` construct. If the 477generator is not resumed before it is 478finalized (by reaching a zero reference count or by being garbage collected), 479the generator-iterator's :meth:`~generator.close` method will be called, 480allowing any pending :keyword:`finally` clauses to execute. 481 482.. index:: 483 single: from; yield from expression 484 485When ``yield from <expr>`` is used, the supplied expression must be an 486iterable. The values produced by iterating that iterable are passed directly 487to the caller of the current generator's methods. Any values passed in with 488:meth:`~generator.send` and any exceptions passed in with 489:meth:`~generator.throw` are passed to the underlying iterator if it has the 490appropriate methods. If this is not the case, then :meth:`~generator.send` 491will raise :exc:`AttributeError` or :exc:`TypeError`, while 492:meth:`~generator.throw` will just raise the passed in exception immediately. 493 494When the underlying iterator is complete, the :attr:`~StopIteration.value` 495attribute of the raised :exc:`StopIteration` instance becomes the value of 496the yield expression. It can be either set explicitly when raising 497:exc:`StopIteration`, or automatically when the subiterator is a generator 498(by returning a value from the subgenerator). 499 500 .. versionchanged:: 3.3 501 Added ``yield from <expr>`` to delegate control flow to a subiterator. 502 503The parentheses may be omitted when the yield expression is the sole expression 504on the right hand side of an assignment statement. 505 506.. seealso:: 507 508 :pep:`255` - Simple Generators 509 The proposal for adding generators and the :keyword:`yield` statement to Python. 510 511 :pep:`342` - Coroutines via Enhanced Generators 512 The proposal to enhance the API and syntax of generators, making them 513 usable as simple coroutines. 514 515 :pep:`380` - Syntax for Delegating to a Subgenerator 516 The proposal to introduce the :token:`~python-grammar:yield_from` syntax, 517 making delegation to subgenerators easy. 518 519 :pep:`525` - Asynchronous Generators 520 The proposal that expanded on :pep:`492` by adding generator capabilities to 521 coroutine functions. 522 523.. index:: object: generator 524.. _generator-methods: 525 526Generator-iterator methods 527^^^^^^^^^^^^^^^^^^^^^^^^^^ 528 529This subsection describes the methods of a generator iterator. They can 530be used to control the execution of a generator function. 531 532Note that calling any of the generator methods below when the generator 533is already executing raises a :exc:`ValueError` exception. 534 535.. index:: exception: StopIteration 536 537 538.. method:: generator.__next__() 539 540 Starts the execution of a generator function or resumes it at the last 541 executed yield expression. When a generator function is resumed with a 542 :meth:`~generator.__next__` method, the current yield expression always 543 evaluates to :const:`None`. The execution then continues to the next yield 544 expression, where the generator is suspended again, and the value of the 545 :token:`~python-grammar:expression_list` is returned to :meth:`__next__`'s 546 caller. If the generator exits without yielding another value, a 547 :exc:`StopIteration` exception is raised. 548 549 This method is normally called implicitly, e.g. by a :keyword:`for` loop, or 550 by the built-in :func:`next` function. 551 552 553.. method:: generator.send(value) 554 555 Resumes the execution and "sends" a value into the generator function. The 556 *value* argument becomes the result of the current yield expression. The 557 :meth:`send` method returns the next value yielded by the generator, or 558 raises :exc:`StopIteration` if the generator exits without yielding another 559 value. When :meth:`send` is called to start the generator, it must be called 560 with :const:`None` as the argument, because there is no yield expression that 561 could receive the value. 562 563 564.. method:: generator.throw(type[, value[, traceback]]) 565 566 Raises an exception of type ``type`` at the point where the generator was paused, 567 and returns the next value yielded by the generator function. If the generator 568 exits without yielding another value, a :exc:`StopIteration` exception is 569 raised. If the generator function does not catch the passed-in exception, or 570 raises a different exception, then that exception propagates to the caller. 571 572.. index:: exception: GeneratorExit 573 574 575.. method:: generator.close() 576 577 Raises a :exc:`GeneratorExit` at the point where the generator function was 578 paused. If the generator function then exits gracefully, is already closed, 579 or raises :exc:`GeneratorExit` (by not catching the exception), close 580 returns to its caller. If the generator yields a value, a 581 :exc:`RuntimeError` is raised. If the generator raises any other exception, 582 it is propagated to the caller. :meth:`close` does nothing if the generator 583 has already exited due to an exception or normal exit. 584 585.. index:: single: yield; examples 586 587Examples 588^^^^^^^^ 589 590Here is a simple example that demonstrates the behavior of generators and 591generator functions:: 592 593 >>> def echo(value=None): 594 ... print("Execution starts when 'next()' is called for the first time.") 595 ... try: 596 ... while True: 597 ... try: 598 ... value = (yield value) 599 ... except Exception as e: 600 ... value = e 601 ... finally: 602 ... print("Don't forget to clean up when 'close()' is called.") 603 ... 604 >>> generator = echo(1) 605 >>> print(next(generator)) 606 Execution starts when 'next()' is called for the first time. 607 1 608 >>> print(next(generator)) 609 None 610 >>> print(generator.send(2)) 611 2 612 >>> generator.throw(TypeError, "spam") 613 TypeError('spam',) 614 >>> generator.close() 615 Don't forget to clean up when 'close()' is called. 616 617For examples using ``yield from``, see :ref:`pep-380` in "What's New in 618Python." 619 620.. _asynchronous-generator-functions: 621 622Asynchronous generator functions 623^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 624 625The presence of a yield expression in a function or method defined using 626:keyword:`async def` further defines the function as an 627:term:`asynchronous generator` function. 628 629When an asynchronous generator function is called, it returns an 630asynchronous iterator known as an asynchronous generator object. 631That object then controls the execution of the generator function. 632An asynchronous generator object is typically used in an 633:keyword:`async for` statement in a coroutine function analogously to 634how a generator object would be used in a :keyword:`for` statement. 635 636Calling one of the asynchronous generator's methods returns an :term:`awaitable` 637object, and the execution starts when this object is awaited on. At that time, 638the execution proceeds to the first yield expression, where it is suspended 639again, returning the value of :token:`~python-grammar:expression_list` to the 640awaiting coroutine. As with a generator, suspension means that all local state 641is retained, including the current bindings of local variables, the instruction 642pointer, the internal evaluation stack, and the state of any exception handling. 643When the execution is resumed by awaiting on the next object returned by the 644asynchronous generator's methods, the function can proceed exactly as if the 645yield expression were just another external call. The value of the yield 646expression after resuming depends on the method which resumed the execution. If 647:meth:`~agen.__anext__` is used then the result is :const:`None`. Otherwise, if 648:meth:`~agen.asend` is used, then the result will be the value passed in to that 649method. 650 651If an asynchronous generator happens to exit early by :keyword:`break`, the caller 652task being cancelled, or other exceptions, the generator's async cleanup code 653will run and possibly raise exceptions or access context variables in an 654unexpected context--perhaps after the lifetime of tasks it depends, or 655during the event loop shutdown when the async-generator garbage collection hook 656is called. 657To prevent this, the caller must explicitly close the async generator by calling 658:meth:`~agen.aclose` method to finalize the generator and ultimately detach it 659from the event loop. 660 661In an asynchronous generator function, yield expressions are allowed anywhere 662in a :keyword:`try` construct. However, if an asynchronous generator is not 663resumed before it is finalized (by reaching a zero reference count or by 664being garbage collected), then a yield expression within a :keyword:`!try` 665construct could result in a failure to execute pending :keyword:`finally` 666clauses. In this case, it is the responsibility of the event loop or 667scheduler running the asynchronous generator to call the asynchronous 668generator-iterator's :meth:`~agen.aclose` method and run the resulting 669coroutine object, thus allowing any pending :keyword:`!finally` clauses 670to execute. 671 672To take care of finalization upon event loop termination, an event loop should 673define a *finalizer* function which takes an asynchronous generator-iterator and 674presumably calls :meth:`~agen.aclose` and executes the coroutine. 675This *finalizer* may be registered by calling :func:`sys.set_asyncgen_hooks`. 676When first iterated over, an asynchronous generator-iterator will store the 677registered *finalizer* to be called upon finalization. For a reference example 678of a *finalizer* method see the implementation of 679``asyncio.Loop.shutdown_asyncgens`` in :source:`Lib/asyncio/base_events.py`. 680 681The expression ``yield from <expr>`` is a syntax error when used in an 682asynchronous generator function. 683 684.. index:: object: asynchronous-generator 685.. _asynchronous-generator-methods: 686 687Asynchronous generator-iterator methods 688^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 689 690This subsection describes the methods of an asynchronous generator iterator, 691which are used to control the execution of a generator function. 692 693 694.. index:: exception: StopAsyncIteration 695 696.. coroutinemethod:: agen.__anext__() 697 698 Returns an awaitable which when run starts to execute the asynchronous 699 generator or resumes it at the last executed yield expression. When an 700 asynchronous generator function is resumed with an :meth:`~agen.__anext__` 701 method, the current yield expression always evaluates to :const:`None` in the 702 returned awaitable, which when run will continue to the next yield 703 expression. The value of the :token:`~python-grammar:expression_list` of the 704 yield expression is the value of the :exc:`StopIteration` exception raised by 705 the completing coroutine. If the asynchronous generator exits without 706 yielding another value, the awaitable instead raises a 707 :exc:`StopAsyncIteration` exception, signalling that the asynchronous 708 iteration has completed. 709 710 This method is normally called implicitly by a :keyword:`async for` loop. 711 712 713.. coroutinemethod:: agen.asend(value) 714 715 Returns an awaitable which when run resumes the execution of the 716 asynchronous generator. As with the :meth:`~generator.send()` method for a 717 generator, this "sends" a value into the asynchronous generator function, 718 and the *value* argument becomes the result of the current yield expression. 719 The awaitable returned by the :meth:`asend` method will return the next 720 value yielded by the generator as the value of the raised 721 :exc:`StopIteration`, or raises :exc:`StopAsyncIteration` if the 722 asynchronous generator exits without yielding another value. When 723 :meth:`asend` is called to start the asynchronous 724 generator, it must be called with :const:`None` as the argument, 725 because there is no yield expression that could receive the value. 726 727 728.. coroutinemethod:: agen.athrow(type[, value[, traceback]]) 729 730 Returns an awaitable that raises an exception of type ``type`` at the point 731 where the asynchronous generator was paused, and returns the next value 732 yielded by the generator function as the value of the raised 733 :exc:`StopIteration` exception. If the asynchronous generator exits 734 without yielding another value, a :exc:`StopAsyncIteration` exception is 735 raised by the awaitable. 736 If the generator function does not catch the passed-in exception, or 737 raises a different exception, then when the awaitable is run that exception 738 propagates to the caller of the awaitable. 739 740.. index:: exception: GeneratorExit 741 742 743.. coroutinemethod:: agen.aclose() 744 745 Returns an awaitable that when run will throw a :exc:`GeneratorExit` into 746 the asynchronous generator function at the point where it was paused. 747 If the asynchronous generator function then exits gracefully, is already 748 closed, or raises :exc:`GeneratorExit` (by not catching the exception), 749 then the returned awaitable will raise a :exc:`StopIteration` exception. 750 Any further awaitables returned by subsequent calls to the asynchronous 751 generator will raise a :exc:`StopAsyncIteration` exception. If the 752 asynchronous generator yields a value, a :exc:`RuntimeError` is raised 753 by the awaitable. If the asynchronous generator raises any other exception, 754 it is propagated to the caller of the awaitable. If the asynchronous 755 generator has already exited due to an exception or normal exit, then 756 further calls to :meth:`aclose` will return an awaitable that does nothing. 757 758.. _primaries: 759 760Primaries 761========= 762 763.. index:: single: primary 764 765Primaries represent the most tightly bound operations of the language. Their 766syntax is: 767 768.. productionlist:: python-grammar 769 primary: `atom` | `attributeref` | `subscription` | `slicing` | `call` 770 771 772.. _attribute-references: 773 774Attribute references 775-------------------- 776 777.. index:: 778 pair: attribute; reference 779 single: . (dot); attribute reference 780 781An attribute reference is a primary followed by a period and a name: 782 783.. productionlist:: python-grammar 784 attributeref: `primary` "." `identifier` 785 786.. index:: 787 exception: AttributeError 788 object: module 789 object: list 790 791The primary must evaluate to an object of a type that supports attribute 792references, which most objects do. This object is then asked to produce the 793attribute whose name is the identifier. This production can be customized by 794overriding the :meth:`__getattr__` method. If this attribute is not available, 795the exception :exc:`AttributeError` is raised. Otherwise, the type and value of 796the object produced is determined by the object. Multiple evaluations of the 797same attribute reference may yield different objects. 798 799 800.. _subscriptions: 801 802Subscriptions 803------------- 804 805.. index:: 806 single: subscription 807 single: [] (square brackets); subscription 808 809.. index:: 810 object: sequence 811 object: mapping 812 object: string 813 object: tuple 814 object: list 815 object: dictionary 816 pair: sequence; item 817 818Subscription of a sequence (string, tuple or list) or mapping (dictionary) 819object usually selects an item from the collection: 820 821.. productionlist:: python-grammar 822 subscription: `primary` "[" `expression_list` "]" 823 824The primary must evaluate to an object that supports subscription (lists or 825dictionaries for example). User-defined objects can support subscription by 826defining a :meth:`__getitem__` method. 827 828For built-in objects, there are two types of objects that support subscription: 829 830If the primary is a mapping, the expression list must evaluate to an object 831whose value is one of the keys of the mapping, and the subscription selects the 832value in the mapping that corresponds to that key. (The expression list is a 833tuple except if it has exactly one item.) 834 835If the primary is a sequence, the expression list must evaluate to an integer 836or a slice (as discussed in the following section). 837 838The formal syntax makes no special provision for negative indices in 839sequences; however, built-in sequences all provide a :meth:`__getitem__` 840method that interprets negative indices by adding the length of the sequence 841to the index (so that ``x[-1]`` selects the last item of ``x``). The 842resulting value must be a nonnegative integer less than the number of items in 843the sequence, and the subscription selects the item whose index is that value 844(counting from zero). Since the support for negative indices and slicing 845occurs in the object's :meth:`__getitem__` method, subclasses overriding 846this method will need to explicitly add that support. 847 848.. index:: 849 single: character 850 pair: string; item 851 852A string's items are characters. A character is not a separate data type but a 853string of exactly one character. 854 855Subscription of certain :term:`classes <class>` or :term:`types <type>` 856creates a :ref:`generic alias <types-genericalias>`. 857In this case, user-defined classes can support subscription by providing a 858:meth:`__class_getitem__` classmethod. 859 860 861.. _slicings: 862 863Slicings 864-------- 865 866.. index:: 867 single: slicing 868 single: slice 869 single: : (colon); slicing 870 single: , (comma); slicing 871 872.. index:: 873 object: sequence 874 object: string 875 object: tuple 876 object: list 877 878A slicing selects a range of items in a sequence object (e.g., a string, tuple 879or list). Slicings may be used as expressions or as targets in assignment or 880:keyword:`del` statements. The syntax for a slicing: 881 882.. productionlist:: python-grammar 883 slicing: `primary` "[" `slice_list` "]" 884 slice_list: `slice_item` ("," `slice_item`)* [","] 885 slice_item: `expression` | `proper_slice` 886 proper_slice: [`lower_bound`] ":" [`upper_bound`] [ ":" [`stride`] ] 887 lower_bound: `expression` 888 upper_bound: `expression` 889 stride: `expression` 890 891There is ambiguity in the formal syntax here: anything that looks like an 892expression list also looks like a slice list, so any subscription can be 893interpreted as a slicing. Rather than further complicating the syntax, this is 894disambiguated by defining that in this case the interpretation as a subscription 895takes priority over the interpretation as a slicing (this is the case if the 896slice list contains no proper slice). 897 898.. index:: 899 single: start (slice object attribute) 900 single: stop (slice object attribute) 901 single: step (slice object attribute) 902 903The semantics for a slicing are as follows. The primary is indexed (using the 904same :meth:`__getitem__` method as 905normal subscription) with a key that is constructed from the slice list, as 906follows. If the slice list contains at least one comma, the key is a tuple 907containing the conversion of the slice items; otherwise, the conversion of the 908lone slice item is the key. The conversion of a slice item that is an 909expression is that expression. The conversion of a proper slice is a slice 910object (see section :ref:`types`) whose :attr:`~slice.start`, 911:attr:`~slice.stop` and :attr:`~slice.step` attributes are the values of the 912expressions given as lower bound, upper bound and stride, respectively, 913substituting ``None`` for missing expressions. 914 915 916.. index:: 917 object: callable 918 single: call 919 single: argument; call semantics 920 single: () (parentheses); call 921 single: , (comma); argument list 922 single: = (equals); in function calls 923 924.. _calls: 925 926Calls 927----- 928 929A call calls a callable object (e.g., a :term:`function`) with a possibly empty 930series of :term:`arguments <argument>`: 931 932.. productionlist:: python-grammar 933 call: `primary` "(" [`argument_list` [","] | `comprehension`] ")" 934 argument_list: `positional_arguments` ["," `starred_and_keywords`] 935 : ["," `keywords_arguments`] 936 : | `starred_and_keywords` ["," `keywords_arguments`] 937 : | `keywords_arguments` 938 positional_arguments: positional_item ("," positional_item)* 939 positional_item: `assignment_expression` | "*" `expression` 940 starred_and_keywords: ("*" `expression` | `keyword_item`) 941 : ("," "*" `expression` | "," `keyword_item`)* 942 keywords_arguments: (`keyword_item` | "**" `expression`) 943 : ("," `keyword_item` | "," "**" `expression`)* 944 keyword_item: `identifier` "=" `expression` 945 946An optional trailing comma may be present after the positional and keyword arguments 947but does not affect the semantics. 948 949.. index:: 950 single: parameter; call semantics 951 952The primary must evaluate to a callable object (user-defined functions, built-in 953functions, methods of built-in objects, class objects, methods of class 954instances, and all objects having a :meth:`__call__` method are callable). All 955argument expressions are evaluated before the call is attempted. Please refer 956to section :ref:`function` for the syntax of formal :term:`parameter` lists. 957 958.. XXX update with kwonly args PEP 959 960If keyword arguments are present, they are first converted to positional 961arguments, as follows. First, a list of unfilled slots is created for the 962formal parameters. If there are N positional arguments, they are placed in the 963first N slots. Next, for each keyword argument, the identifier is used to 964determine the corresponding slot (if the identifier is the same as the first 965formal parameter name, the first slot is used, and so on). If the slot is 966already filled, a :exc:`TypeError` exception is raised. Otherwise, the 967argument is placed in the slot, filling it (even if the expression is 968``None``, it fills the slot). When all arguments have been processed, the slots 969that are still unfilled are filled with the corresponding default value from the 970function definition. (Default values are calculated, once, when the function is 971defined; thus, a mutable object such as a list or dictionary used as default 972value will be shared by all calls that don't specify an argument value for the 973corresponding slot; this should usually be avoided.) If there are any unfilled 974slots for which no default value is specified, a :exc:`TypeError` exception is 975raised. Otherwise, the list of filled slots is used as the argument list for 976the call. 977 978.. impl-detail:: 979 980 An implementation may provide built-in functions whose positional parameters 981 do not have names, even if they are 'named' for the purpose of documentation, 982 and which therefore cannot be supplied by keyword. In CPython, this is the 983 case for functions implemented in C that use :c:func:`PyArg_ParseTuple` to 984 parse their arguments. 985 986If there are more positional arguments than there are formal parameter slots, a 987:exc:`TypeError` exception is raised, unless a formal parameter using the syntax 988``*identifier`` is present; in this case, that formal parameter receives a tuple 989containing the excess positional arguments (or an empty tuple if there were no 990excess positional arguments). 991 992If any keyword argument does not correspond to a formal parameter name, a 993:exc:`TypeError` exception is raised, unless a formal parameter using the syntax 994``**identifier`` is present; in this case, that formal parameter receives a 995dictionary containing the excess keyword arguments (using the keywords as keys 996and the argument values as corresponding values), or a (new) empty dictionary if 997there were no excess keyword arguments. 998 999.. index:: 1000 single: * (asterisk); in function calls 1001 single: unpacking; in function calls 1002 1003If the syntax ``*expression`` appears in the function call, ``expression`` must 1004evaluate to an :term:`iterable`. Elements from these iterables are 1005treated as if they were additional positional arguments. For the call 1006``f(x1, x2, *y, x3, x4)``, if *y* evaluates to a sequence *y1*, ..., *yM*, 1007this is equivalent to a call with M+4 positional arguments *x1*, *x2*, 1008*y1*, ..., *yM*, *x3*, *x4*. 1009 1010A consequence of this is that although the ``*expression`` syntax may appear 1011*after* explicit keyword arguments, it is processed *before* the 1012keyword arguments (and any ``**expression`` arguments -- see below). So:: 1013 1014 >>> def f(a, b): 1015 ... print(a, b) 1016 ... 1017 >>> f(b=1, *(2,)) 1018 2 1 1019 >>> f(a=1, *(2,)) 1020 Traceback (most recent call last): 1021 File "<stdin>", line 1, in <module> 1022 TypeError: f() got multiple values for keyword argument 'a' 1023 >>> f(1, *(2,)) 1024 1 2 1025 1026It is unusual for both keyword arguments and the ``*expression`` syntax to be 1027used in the same call, so in practice this confusion does not often arise. 1028 1029.. index:: 1030 single: **; in function calls 1031 1032If the syntax ``**expression`` appears in the function call, ``expression`` must 1033evaluate to a :term:`mapping`, the contents of which are treated as 1034additional keyword arguments. If a keyword is already present 1035(as an explicit keyword argument, or from another unpacking), 1036a :exc:`TypeError` exception is raised. 1037 1038Formal parameters using the syntax ``*identifier`` or ``**identifier`` cannot be 1039used as positional argument slots or as keyword argument names. 1040 1041.. versionchanged:: 3.5 1042 Function calls accept any number of ``*`` and ``**`` unpackings, 1043 positional arguments may follow iterable unpackings (``*``), 1044 and keyword arguments may follow dictionary unpackings (``**``). 1045 Originally proposed by :pep:`448`. 1046 1047A call always returns some value, possibly ``None``, unless it raises an 1048exception. How this value is computed depends on the type of the callable 1049object. 1050 1051If it is--- 1052 1053a user-defined function: 1054 .. index:: 1055 pair: function; call 1056 triple: user-defined; function; call 1057 object: user-defined function 1058 object: function 1059 1060 The code block for the function is executed, passing it the argument list. The 1061 first thing the code block will do is bind the formal parameters to the 1062 arguments; this is described in section :ref:`function`. When the code block 1063 executes a :keyword:`return` statement, this specifies the return value of the 1064 function call. 1065 1066a built-in function or method: 1067 .. index:: 1068 pair: function; call 1069 pair: built-in function; call 1070 pair: method; call 1071 pair: built-in method; call 1072 object: built-in method 1073 object: built-in function 1074 object: method 1075 object: function 1076 1077 The result is up to the interpreter; see :ref:`built-in-funcs` for the 1078 descriptions of built-in functions and methods. 1079 1080a class object: 1081 .. index:: 1082 object: class 1083 pair: class object; call 1084 1085 A new instance of that class is returned. 1086 1087a class instance method: 1088 .. index:: 1089 object: class instance 1090 object: instance 1091 pair: class instance; call 1092 1093 The corresponding user-defined function is called, with an argument list that is 1094 one longer than the argument list of the call: the instance becomes the first 1095 argument. 1096 1097a class instance: 1098 .. index:: 1099 pair: instance; call 1100 single: __call__() (object method) 1101 1102 The class must define a :meth:`__call__` method; the effect is then the same as 1103 if that method was called. 1104 1105 1106.. index:: keyword: await 1107.. _await: 1108 1109Await expression 1110================ 1111 1112Suspend the execution of :term:`coroutine` on an :term:`awaitable` object. 1113Can only be used inside a :term:`coroutine function`. 1114 1115.. productionlist:: python-grammar 1116 await_expr: "await" `primary` 1117 1118.. versionadded:: 3.5 1119 1120 1121.. _power: 1122 1123The power operator 1124================== 1125 1126.. index:: 1127 pair: power; operation 1128 operator: ** 1129 1130The power operator binds more tightly than unary operators on its left; it binds 1131less tightly than unary operators on its right. The syntax is: 1132 1133.. productionlist:: python-grammar 1134 power: (`await_expr` | `primary`) ["**" `u_expr`] 1135 1136Thus, in an unparenthesized sequence of power and unary operators, the operators 1137are evaluated from right to left (this does not constrain the evaluation order 1138for the operands): ``-1**2`` results in ``-1``. 1139 1140The power operator has the same semantics as the built-in :func:`pow` function, 1141when called with two arguments: it yields its left argument raised to the power 1142of its right argument. The numeric arguments are first converted to a common 1143type, and the result is of that type. 1144 1145For int operands, the result has the same type as the operands unless the second 1146argument is negative; in that case, all arguments are converted to float and a 1147float result is delivered. For example, ``10**2`` returns ``100``, but 1148``10**-2`` returns ``0.01``. 1149 1150Raising ``0.0`` to a negative power results in a :exc:`ZeroDivisionError`. 1151Raising a negative number to a fractional power results in a :class:`complex` 1152number. (In earlier versions it raised a :exc:`ValueError`.) 1153 1154This operation can be customized using the special :meth:`__pow__` method. 1155 1156.. _unary: 1157 1158Unary arithmetic and bitwise operations 1159======================================= 1160 1161.. index:: 1162 triple: unary; arithmetic; operation 1163 triple: unary; bitwise; operation 1164 1165All unary arithmetic and bitwise operations have the same priority: 1166 1167.. productionlist:: python-grammar 1168 u_expr: `power` | "-" `u_expr` | "+" `u_expr` | "~" `u_expr` 1169 1170.. index:: 1171 single: negation 1172 single: minus 1173 single: operator; - (minus) 1174 single: - (minus); unary operator 1175 1176The unary ``-`` (minus) operator yields the negation of its numeric argument; the 1177operation can be overridden with the :meth:`__neg__` special method. 1178 1179.. index:: 1180 single: plus 1181 single: operator; + (plus) 1182 single: + (plus); unary operator 1183 1184The unary ``+`` (plus) operator yields its numeric argument unchanged; the 1185operation can be overridden with the :meth:`__pos__` special method. 1186 1187.. index:: 1188 single: inversion 1189 operator: ~ (tilde) 1190 1191The unary ``~`` (invert) operator yields the bitwise inversion of its integer 1192argument. The bitwise inversion of ``x`` is defined as ``-(x+1)``. It only 1193applies to integral numbers or to custom objects that override the 1194:meth:`__invert__` special method. 1195 1196 1197 1198.. index:: exception: TypeError 1199 1200In all three cases, if the argument does not have the proper type, a 1201:exc:`TypeError` exception is raised. 1202 1203 1204.. _binary: 1205 1206Binary arithmetic operations 1207============================ 1208 1209.. index:: triple: binary; arithmetic; operation 1210 1211The binary arithmetic operations have the conventional priority levels. Note 1212that some of these operations also apply to certain non-numeric types. Apart 1213from the power operator, there are only two levels, one for multiplicative 1214operators and one for additive operators: 1215 1216.. productionlist:: python-grammar 1217 m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "@" `m_expr` | 1218 : `m_expr` "//" `u_expr` | `m_expr` "/" `u_expr` | 1219 : `m_expr` "%" `u_expr` 1220 a_expr: `m_expr` | `a_expr` "+" `m_expr` | `a_expr` "-" `m_expr` 1221 1222.. index:: 1223 single: multiplication 1224 operator: * (asterisk) 1225 1226The ``*`` (multiplication) operator yields the product of its arguments. The 1227arguments must either both be numbers, or one argument must be an integer and 1228the other must be a sequence. In the former case, the numbers are converted to a 1229common type and then multiplied together. In the latter case, sequence 1230repetition is performed; a negative repetition factor yields an empty sequence. 1231 1232This operation can be customized using the special :meth:`__mul__` and 1233:meth:`__rmul__` methods. 1234 1235.. index:: 1236 single: matrix multiplication 1237 operator: @ (at) 1238 1239The ``@`` (at) operator is intended to be used for matrix multiplication. No 1240builtin Python types implement this operator. 1241 1242.. versionadded:: 3.5 1243 1244.. index:: 1245 exception: ZeroDivisionError 1246 single: division 1247 operator: / (slash) 1248 operator: // 1249 1250The ``/`` (division) and ``//`` (floor division) operators yield the quotient of 1251their arguments. The numeric arguments are first converted to a common type. 1252Division of integers yields a float, while floor division of integers results in an 1253integer; the result is that of mathematical division with the 'floor' function 1254applied to the result. Division by zero raises the :exc:`ZeroDivisionError` 1255exception. 1256 1257This operation can be customized using the special :meth:`__truediv__` and 1258:meth:`__floordiv__` methods. 1259 1260.. index:: 1261 single: modulo 1262 operator: % (percent) 1263 1264The ``%`` (modulo) operator yields the remainder from the division of the first 1265argument by the second. The numeric arguments are first converted to a common 1266type. A zero right argument raises the :exc:`ZeroDivisionError` exception. The 1267arguments may be floating point numbers, e.g., ``3.14%0.7`` equals ``0.34`` 1268(since ``3.14`` equals ``4*0.7 + 0.34``.) The modulo operator always yields a 1269result with the same sign as its second operand (or zero); the absolute value of 1270the result is strictly smaller than the absolute value of the second operand 1271[#]_. 1272 1273The floor division and modulo operators are connected by the following 1274identity: ``x == (x//y)*y + (x%y)``. Floor division and modulo are also 1275connected with the built-in function :func:`divmod`: ``divmod(x, y) == (x//y, 1276x%y)``. [#]_. 1277 1278In addition to performing the modulo operation on numbers, the ``%`` operator is 1279also overloaded by string objects to perform old-style string formatting (also 1280known as interpolation). The syntax for string formatting is described in the 1281Python Library Reference, section :ref:`old-string-formatting`. 1282 1283The *modulo* operation can be customized using the special :meth:`__mod__` method. 1284 1285The floor division operator, the modulo operator, and the :func:`divmod` 1286function are not defined for complex numbers. Instead, convert to a floating 1287point number using the :func:`abs` function if appropriate. 1288 1289.. index:: 1290 single: addition 1291 single: operator; + (plus) 1292 single: + (plus); binary operator 1293 1294The ``+`` (addition) operator yields the sum of its arguments. The arguments 1295must either both be numbers or both be sequences of the same type. In the 1296former case, the numbers are converted to a common type and then added together. 1297In the latter case, the sequences are concatenated. 1298 1299This operation can be customized using the special :meth:`__add__` and 1300:meth:`__radd__` methods. 1301 1302.. index:: 1303 single: subtraction 1304 single: operator; - (minus) 1305 single: - (minus); binary operator 1306 1307The ``-`` (subtraction) operator yields the difference of its arguments. The 1308numeric arguments are first converted to a common type. 1309 1310This operation can be customized using the special :meth:`__sub__` method. 1311 1312 1313.. _shifting: 1314 1315Shifting operations 1316=================== 1317 1318.. index:: 1319 pair: shifting; operation 1320 operator: << 1321 operator: >> 1322 1323The shifting operations have lower priority than the arithmetic operations: 1324 1325.. productionlist:: python-grammar 1326 shift_expr: `a_expr` | `shift_expr` ("<<" | ">>") `a_expr` 1327 1328These operators accept integers as arguments. They shift the first argument to 1329the left or right by the number of bits given by the second argument. 1330 1331This operation can be customized using the special :meth:`__lshift__` and 1332:meth:`__rshift__` methods. 1333 1334.. index:: exception: ValueError 1335 1336A right shift by *n* bits is defined as floor division by ``pow(2,n)``. A left 1337shift by *n* bits is defined as multiplication with ``pow(2,n)``. 1338 1339 1340.. _bitwise: 1341 1342Binary bitwise operations 1343========================= 1344 1345.. index:: triple: binary; bitwise; operation 1346 1347Each of the three bitwise operations has a different priority level: 1348 1349.. productionlist:: python-grammar 1350 and_expr: `shift_expr` | `and_expr` "&" `shift_expr` 1351 xor_expr: `and_expr` | `xor_expr` "^" `and_expr` 1352 or_expr: `xor_expr` | `or_expr` "|" `xor_expr` 1353 1354.. index:: 1355 pair: bitwise; and 1356 operator: & (ampersand) 1357 1358The ``&`` operator yields the bitwise AND of its arguments, which must be 1359integers or one of them must be a custom object overriding :meth:`__and__` or 1360:meth:`__rand__` special methods. 1361 1362.. index:: 1363 pair: bitwise; xor 1364 pair: exclusive; or 1365 operator: ^ (caret) 1366 1367The ``^`` operator yields the bitwise XOR (exclusive OR) of its arguments, which 1368must be integers or one of them must be a custom object overriding :meth:`__xor__` or 1369:meth:`__rxor__` special methods. 1370 1371.. index:: 1372 pair: bitwise; or 1373 pair: inclusive; or 1374 operator: | (vertical bar) 1375 1376The ``|`` operator yields the bitwise (inclusive) OR of its arguments, which 1377must be integers or one of them must be a custom object overriding :meth:`__or__` or 1378:meth:`__ror__` special methods. 1379 1380 1381.. _comparisons: 1382 1383Comparisons 1384=========== 1385 1386.. index:: 1387 single: comparison 1388 pair: C; language 1389 operator: < (less) 1390 operator: > (greater) 1391 operator: <= 1392 operator: >= 1393 operator: == 1394 operator: != 1395 1396Unlike C, all comparison operations in Python have the same priority, which is 1397lower than that of any arithmetic, shifting or bitwise operation. Also unlike 1398C, expressions like ``a < b < c`` have the interpretation that is conventional 1399in mathematics: 1400 1401.. productionlist:: python-grammar 1402 comparison: `or_expr` (`comp_operator` `or_expr`)* 1403 comp_operator: "<" | ">" | "==" | ">=" | "<=" | "!=" 1404 : | "is" ["not"] | ["not"] "in" 1405 1406Comparisons yield boolean values: ``True`` or ``False``. Custom 1407:dfn:`rich comparison methods` may return non-boolean values. In this case 1408Python will call :func:`bool` on such value in boolean contexts. 1409 1410.. index:: pair: chaining; comparisons 1411 1412Comparisons can be chained arbitrarily, e.g., ``x < y <= z`` is equivalent to 1413``x < y and y <= z``, except that ``y`` is evaluated only once (but in both 1414cases ``z`` is not evaluated at all when ``x < y`` is found to be false). 1415 1416Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *op1*, *op2*, ..., 1417*opN* are comparison operators, then ``a op1 b op2 c ... y opN z`` is equivalent 1418to ``a op1 b and b op2 c and ... y opN z``, except that each expression is 1419evaluated at most once. 1420 1421Note that ``a op1 b op2 c`` doesn't imply any kind of comparison between *a* and 1422*c*, so that, e.g., ``x < y > z`` is perfectly legal (though perhaps not 1423pretty). 1424 1425.. _expressions-value-comparisons: 1426 1427Value comparisons 1428----------------- 1429 1430The operators ``<``, ``>``, ``==``, ``>=``, ``<=``, and ``!=`` compare the 1431values of two objects. The objects do not need to have the same type. 1432 1433Chapter :ref:`objects` states that objects have a value (in addition to type 1434and identity). The value of an object is a rather abstract notion in Python: 1435For example, there is no canonical access method for an object's value. Also, 1436there is no requirement that the value of an object should be constructed in a 1437particular way, e.g. comprised of all its data attributes. Comparison operators 1438implement a particular notion of what the value of an object is. One can think 1439of them as defining the value of an object indirectly, by means of their 1440comparison implementation. 1441 1442Because all types are (direct or indirect) subtypes of :class:`object`, they 1443inherit the default comparison behavior from :class:`object`. Types can 1444customize their comparison behavior by implementing 1445:dfn:`rich comparison methods` like :meth:`__lt__`, described in 1446:ref:`customization`. 1447 1448The default behavior for equality comparison (``==`` and ``!=``) is based on 1449the identity of the objects. Hence, equality comparison of instances with the 1450same identity results in equality, and equality comparison of instances with 1451different identities results in inequality. A motivation for this default 1452behavior is the desire that all objects should be reflexive (i.e. ``x is y`` 1453implies ``x == y``). 1454 1455A default order comparison (``<``, ``>``, ``<=``, and ``>=``) is not provided; 1456an attempt raises :exc:`TypeError`. A motivation for this default behavior is 1457the lack of a similar invariant as for equality. 1458 1459The behavior of the default equality comparison, that instances with different 1460identities are always unequal, may be in contrast to what types will need that 1461have a sensible definition of object value and value-based equality. Such 1462types will need to customize their comparison behavior, and in fact, a number 1463of built-in types have done that. 1464 1465The following list describes the comparison behavior of the most important 1466built-in types. 1467 1468* Numbers of built-in numeric types (:ref:`typesnumeric`) and of the standard 1469 library types :class:`fractions.Fraction` and :class:`decimal.Decimal` can be 1470 compared within and across their types, with the restriction that complex 1471 numbers do not support order comparison. Within the limits of the types 1472 involved, they compare mathematically (algorithmically) correct without loss 1473 of precision. 1474 1475 The not-a-number values ``float('NaN')`` and ``decimal.Decimal('NaN')`` are 1476 special. Any ordered comparison of a number to a not-a-number value is false. 1477 A counter-intuitive implication is that not-a-number values are not equal to 1478 themselves. For example, if ``x = float('NaN')``, ``3 < x``, ``x < 3`` and 1479 ``x == x`` are all false, while ``x != x`` is true. This behavior is 1480 compliant with IEEE 754. 1481 1482* ``None`` and ``NotImplemented`` are singletons. :PEP:`8` advises that 1483 comparisons for singletons should always be done with ``is`` or ``is not``, 1484 never the equality operators. 1485 1486* Binary sequences (instances of :class:`bytes` or :class:`bytearray`) can be 1487 compared within and across their types. They compare lexicographically using 1488 the numeric values of their elements. 1489 1490* Strings (instances of :class:`str`) compare lexicographically using the 1491 numerical Unicode code points (the result of the built-in function 1492 :func:`ord`) of their characters. [#]_ 1493 1494 Strings and binary sequences cannot be directly compared. 1495 1496* Sequences (instances of :class:`tuple`, :class:`list`, or :class:`range`) can 1497 be compared only within each of their types, with the restriction that ranges 1498 do not support order comparison. Equality comparison across these types 1499 results in inequality, and ordering comparison across these types raises 1500 :exc:`TypeError`. 1501 1502 Sequences compare lexicographically using comparison of corresponding 1503 elements. The built-in containers typically assume identical objects are 1504 equal to themselves. That lets them bypass equality tests for identical 1505 objects to improve performance and to maintain their internal invariants. 1506 1507 Lexicographical comparison between built-in collections works as follows: 1508 1509 - For two collections to compare equal, they must be of the same type, have 1510 the same length, and each pair of corresponding elements must compare 1511 equal (for example, ``[1,2] == (1,2)`` is false because the type is not the 1512 same). 1513 1514 - Collections that support order comparison are ordered the same as their 1515 first unequal elements (for example, ``[1,2,x] <= [1,2,y]`` has the same 1516 value as ``x <= y``). If a corresponding element does not exist, the 1517 shorter collection is ordered first (for example, ``[1,2] < [1,2,3]`` is 1518 true). 1519 1520* Mappings (instances of :class:`dict`) compare equal if and only if they have 1521 equal `(key, value)` pairs. Equality comparison of the keys and values 1522 enforces reflexivity. 1523 1524 Order comparisons (``<``, ``>``, ``<=``, and ``>=``) raise :exc:`TypeError`. 1525 1526* Sets (instances of :class:`set` or :class:`frozenset`) can be compared within 1527 and across their types. 1528 1529 They define order 1530 comparison operators to mean subset and superset tests. Those relations do 1531 not define total orderings (for example, the two sets ``{1,2}`` and ``{2,3}`` 1532 are not equal, nor subsets of one another, nor supersets of one 1533 another). Accordingly, sets are not appropriate arguments for functions 1534 which depend on total ordering (for example, :func:`min`, :func:`max`, and 1535 :func:`sorted` produce undefined results given a list of sets as inputs). 1536 1537 Comparison of sets enforces reflexivity of its elements. 1538 1539* Most other built-in types have no comparison methods implemented, so they 1540 inherit the default comparison behavior. 1541 1542User-defined classes that customize their comparison behavior should follow 1543some consistency rules, if possible: 1544 1545* Equality comparison should be reflexive. 1546 In other words, identical objects should compare equal: 1547 1548 ``x is y`` implies ``x == y`` 1549 1550* Comparison should be symmetric. 1551 In other words, the following expressions should have the same result: 1552 1553 ``x == y`` and ``y == x`` 1554 1555 ``x != y`` and ``y != x`` 1556 1557 ``x < y`` and ``y > x`` 1558 1559 ``x <= y`` and ``y >= x`` 1560 1561* Comparison should be transitive. 1562 The following (non-exhaustive) examples illustrate that: 1563 1564 ``x > y and y > z`` implies ``x > z`` 1565 1566 ``x < y and y <= z`` implies ``x < z`` 1567 1568* Inverse comparison should result in the boolean negation. 1569 In other words, the following expressions should have the same result: 1570 1571 ``x == y`` and ``not x != y`` 1572 1573 ``x < y`` and ``not x >= y`` (for total ordering) 1574 1575 ``x > y`` and ``not x <= y`` (for total ordering) 1576 1577 The last two expressions apply to totally ordered collections (e.g. to 1578 sequences, but not to sets or mappings). See also the 1579 :func:`~functools.total_ordering` decorator. 1580 1581* The :func:`hash` result should be consistent with equality. 1582 Objects that are equal should either have the same hash value, 1583 or be marked as unhashable. 1584 1585Python does not enforce these consistency rules. In fact, the not-a-number 1586values are an example for not following these rules. 1587 1588 1589.. _in: 1590.. _not in: 1591.. _membership-test-details: 1592 1593Membership test operations 1594-------------------------- 1595 1596The operators :keyword:`in` and :keyword:`not in` test for membership. ``x in 1597s`` evaluates to ``True`` if *x* is a member of *s*, and ``False`` otherwise. 1598``x not in s`` returns the negation of ``x in s``. All built-in sequences and 1599set types support this as well as dictionary, for which :keyword:`!in` tests 1600whether the dictionary has a given key. For container types such as list, tuple, 1601set, frozenset, dict, or collections.deque, the expression ``x in y`` is equivalent 1602to ``any(x is e or x == e for e in y)``. 1603 1604For the string and bytes types, ``x in y`` is ``True`` if and only if *x* is a 1605substring of *y*. An equivalent test is ``y.find(x) != -1``. Empty strings are 1606always considered to be a substring of any other string, so ``"" in "abc"`` will 1607return ``True``. 1608 1609For user-defined classes which define the :meth:`__contains__` method, ``x in 1610y`` returns ``True`` if ``y.__contains__(x)`` returns a true value, and 1611``False`` otherwise. 1612 1613For user-defined classes which do not define :meth:`__contains__` but do define 1614:meth:`__iter__`, ``x in y`` is ``True`` if some value ``z``, for which the 1615expression ``x is z or x == z`` is true, is produced while iterating over ``y``. 1616If an exception is raised during the iteration, it is as if :keyword:`in` raised 1617that exception. 1618 1619Lastly, the old-style iteration protocol is tried: if a class defines 1620:meth:`__getitem__`, ``x in y`` is ``True`` if and only if there is a non-negative 1621integer index *i* such that ``x is y[i] or x == y[i]``, and no lower integer index 1622raises the :exc:`IndexError` exception. (If any other exception is raised, it is as 1623if :keyword:`in` raised that exception). 1624 1625.. index:: 1626 operator: in 1627 operator: not in 1628 pair: membership; test 1629 object: sequence 1630 1631The operator :keyword:`not in` is defined to have the inverse truth value of 1632:keyword:`in`. 1633 1634.. index:: 1635 operator: is 1636 operator: is not 1637 pair: identity; test 1638 1639 1640.. _is: 1641.. _is not: 1642 1643Identity comparisons 1644-------------------- 1645 1646The operators :keyword:`is` and :keyword:`is not` test for an object's identity: ``x 1647is y`` is true if and only if *x* and *y* are the same object. An Object's identity 1648is determined using the :meth:`id` function. ``x is not y`` yields the inverse 1649truth value. [#]_ 1650 1651 1652.. _booleans: 1653.. _and: 1654.. _or: 1655.. _not: 1656 1657Boolean operations 1658================== 1659 1660.. index:: 1661 pair: Conditional; expression 1662 pair: Boolean; operation 1663 1664.. productionlist:: python-grammar 1665 or_test: `and_test` | `or_test` "or" `and_test` 1666 and_test: `not_test` | `and_test` "and" `not_test` 1667 not_test: `comparison` | "not" `not_test` 1668 1669In the context of Boolean operations, and also when expressions are used by 1670control flow statements, the following values are interpreted as false: 1671``False``, ``None``, numeric zero of all types, and empty strings and containers 1672(including strings, tuples, lists, dictionaries, sets and frozensets). All 1673other values are interpreted as true. User-defined objects can customize their 1674truth value by providing a :meth:`__bool__` method. 1675 1676.. index:: operator: not 1677 1678The operator :keyword:`not` yields ``True`` if its argument is false, ``False`` 1679otherwise. 1680 1681.. index:: operator: and 1682 1683The expression ``x and y`` first evaluates *x*; if *x* is false, its value is 1684returned; otherwise, *y* is evaluated and the resulting value is returned. 1685 1686.. index:: operator: or 1687 1688The expression ``x or y`` first evaluates *x*; if *x* is true, its value is 1689returned; otherwise, *y* is evaluated and the resulting value is returned. 1690 1691Note that neither :keyword:`and` nor :keyword:`or` restrict the value and type 1692they return to ``False`` and ``True``, but rather return the last evaluated 1693argument. This is sometimes useful, e.g., if ``s`` is a string that should be 1694replaced by a default value if it is empty, the expression ``s or 'foo'`` yields 1695the desired value. Because :keyword:`not` has to create a new value, it 1696returns a boolean value regardless of the type of its argument 1697(for example, ``not 'foo'`` produces ``False`` rather than ``''``.) 1698 1699 1700Assignment expressions 1701====================== 1702 1703.. productionlist:: python-grammar 1704 assignment_expression: [`identifier` ":="] `expression` 1705 1706An assignment expression (sometimes also called a "named expression" or 1707"walrus") assigns an :token:`~python-grammar:expression` to an 1708:token:`~python-grammar:identifier`, while also returning the value of the 1709:token:`~python-grammar:expression`. 1710 1711One common use case is when handling matched regular expressions: 1712 1713.. code-block:: python 1714 1715 if matching := pattern.search(data): 1716 do_something(matching) 1717 1718Or, when processing a file stream in chunks: 1719 1720.. code-block:: python 1721 1722 while chunk := file.read(9000): 1723 process(chunk) 1724 1725.. versionadded:: 3.8 1726 See :pep:`572` for more details about assignment expressions. 1727 1728 1729.. _if_expr: 1730 1731Conditional expressions 1732======================= 1733 1734.. index:: 1735 pair: conditional; expression 1736 pair: ternary; operator 1737 single: if; conditional expression 1738 single: else; conditional expression 1739 1740.. productionlist:: python-grammar 1741 conditional_expression: `or_test` ["if" `or_test` "else" `expression`] 1742 expression: `conditional_expression` | `lambda_expr` 1743 1744Conditional expressions (sometimes called a "ternary operator") have the lowest 1745priority of all Python operations. 1746 1747The expression ``x if C else y`` first evaluates the condition, *C* rather than *x*. 1748If *C* is true, *x* is evaluated and its value is returned; otherwise, *y* is 1749evaluated and its value is returned. 1750 1751See :pep:`308` for more details about conditional expressions. 1752 1753 1754.. _lambdas: 1755.. _lambda: 1756 1757Lambdas 1758======= 1759 1760.. index:: 1761 pair: lambda; expression 1762 pair: lambda; form 1763 pair: anonymous; function 1764 single: : (colon); lambda expression 1765 1766.. productionlist:: python-grammar 1767 lambda_expr: "lambda" [`parameter_list`] ":" `expression` 1768 1769Lambda expressions (sometimes called lambda forms) are used to create anonymous 1770functions. The expression ``lambda parameters: expression`` yields a function 1771object. The unnamed object behaves like a function object defined with: 1772 1773.. code-block:: none 1774 1775 def <lambda>(parameters): 1776 return expression 1777 1778See section :ref:`function` for the syntax of parameter lists. Note that 1779functions created with lambda expressions cannot contain statements or 1780annotations. 1781 1782 1783.. _exprlists: 1784 1785Expression lists 1786================ 1787 1788.. index:: 1789 pair: expression; list 1790 single: , (comma); expression list 1791 1792.. productionlist:: python-grammar 1793 expression_list: `expression` ("," `expression`)* [","] 1794 starred_list: `starred_item` ("," `starred_item`)* [","] 1795 starred_expression: `expression` | (`starred_item` ",")* [`starred_item`] 1796 starred_item: `assignment_expression` | "*" `or_expr` 1797 1798.. index:: object: tuple 1799 1800Except when part of a list or set display, an expression list 1801containing at least one comma yields a tuple. The length of 1802the tuple is the number of expressions in the list. The expressions are 1803evaluated from left to right. 1804 1805.. index:: 1806 pair: iterable; unpacking 1807 single: * (asterisk); in expression lists 1808 1809An asterisk ``*`` denotes :dfn:`iterable unpacking`. Its operand must be 1810an :term:`iterable`. The iterable is expanded into a sequence of items, 1811which are included in the new tuple, list, or set, at the site of 1812the unpacking. 1813 1814.. versionadded:: 3.5 1815 Iterable unpacking in expression lists, originally proposed by :pep:`448`. 1816 1817.. index:: pair: trailing; comma 1818 1819The trailing comma is required only to create a single tuple (a.k.a. a 1820*singleton*); it is optional in all other cases. A single expression without a 1821trailing comma doesn't create a tuple, but rather yields the value of that 1822expression. (To create an empty tuple, use an empty pair of parentheses: 1823``()``.) 1824 1825 1826.. _evalorder: 1827 1828Evaluation order 1829================ 1830 1831.. index:: pair: evaluation; order 1832 1833Python evaluates expressions from left to right. Notice that while evaluating 1834an assignment, the right-hand side is evaluated before the left-hand side. 1835 1836In the following lines, expressions will be evaluated in the arithmetic order of 1837their suffixes:: 1838 1839 expr1, expr2, expr3, expr4 1840 (expr1, expr2, expr3, expr4) 1841 {expr1: expr2, expr3: expr4} 1842 expr1 + expr2 * (expr3 - expr4) 1843 expr1(expr2, expr3, *expr4, **expr5) 1844 expr3, expr4 = expr1, expr2 1845 1846 1847.. _operator-summary: 1848 1849Operator precedence 1850=================== 1851 1852.. index:: 1853 pair: operator; precedence 1854 1855The following table summarizes the operator precedence in Python, from highest 1856precedence (most binding) to lowest precedence (least binding). Operators in 1857the same box have the same precedence. Unless the syntax is explicitly given, 1858operators are binary. Operators in the same box group left to right (except for 1859exponentiation, which groups from right to left). 1860 1861Note that comparisons, membership tests, and identity tests, all have the same 1862precedence and have a left-to-right chaining feature as described in the 1863:ref:`comparisons` section. 1864 1865 1866+-----------------------------------------------+-------------------------------------+ 1867| Operator | Description | 1868+===============================================+=====================================+ 1869| ``(expressions...)``, | Binding or parenthesized | 1870| | expression, | 1871| ``[expressions...]``, | list display, | 1872| ``{key: value...}``, | dictionary display, | 1873| ``{expressions...}`` | set display | 1874+-----------------------------------------------+-------------------------------------+ 1875| ``x[index]``, ``x[index:index]``, | Subscription, slicing, | 1876| ``x(arguments...)``, ``x.attribute`` | call, attribute reference | 1877+-----------------------------------------------+-------------------------------------+ 1878| :keyword:`await` ``x`` | Await expression | 1879+-----------------------------------------------+-------------------------------------+ 1880| ``**`` | Exponentiation [#]_ | 1881+-----------------------------------------------+-------------------------------------+ 1882| ``+x``, ``-x``, ``~x`` | Positive, negative, bitwise NOT | 1883+-----------------------------------------------+-------------------------------------+ 1884| ``*``, ``@``, ``/``, ``//``, ``%`` | Multiplication, matrix | 1885| | multiplication, division, floor | 1886| | division, remainder [#]_ | 1887+-----------------------------------------------+-------------------------------------+ 1888| ``+``, ``-`` | Addition and subtraction | 1889+-----------------------------------------------+-------------------------------------+ 1890| ``<<``, ``>>`` | Shifts | 1891+-----------------------------------------------+-------------------------------------+ 1892| ``&`` | Bitwise AND | 1893+-----------------------------------------------+-------------------------------------+ 1894| ``^`` | Bitwise XOR | 1895+-----------------------------------------------+-------------------------------------+ 1896| ``|`` | Bitwise OR | 1897+-----------------------------------------------+-------------------------------------+ 1898| :keyword:`in`, :keyword:`not in`, | Comparisons, including membership | 1899| :keyword:`is`, :keyword:`is not`, ``<``, | tests and identity tests | 1900| ``<=``, ``>``, ``>=``, ``!=``, ``==`` | | 1901+-----------------------------------------------+-------------------------------------+ 1902| :keyword:`not` ``x`` | Boolean NOT | 1903+-----------------------------------------------+-------------------------------------+ 1904| :keyword:`and` | Boolean AND | 1905+-----------------------------------------------+-------------------------------------+ 1906| :keyword:`or` | Boolean OR | 1907+-----------------------------------------------+-------------------------------------+ 1908| :keyword:`if <if_expr>` -- :keyword:`!else` | Conditional expression | 1909+-----------------------------------------------+-------------------------------------+ 1910| :keyword:`lambda` | Lambda expression | 1911+-----------------------------------------------+-------------------------------------+ 1912| ``:=`` | Assignment expression | 1913+-----------------------------------------------+-------------------------------------+ 1914 1915 1916.. rubric:: Footnotes 1917 1918.. [#] While ``abs(x%y) < abs(y)`` is true mathematically, for floats it may not be 1919 true numerically due to roundoff. For example, and assuming a platform on which 1920 a Python float is an IEEE 754 double-precision number, in order that ``-1e-100 % 1921 1e100`` have the same sign as ``1e100``, the computed result is ``-1e-100 + 1922 1e100``, which is numerically exactly equal to ``1e100``. The function 1923 :func:`math.fmod` returns a result whose sign matches the sign of the 1924 first argument instead, and so returns ``-1e-100`` in this case. Which approach 1925 is more appropriate depends on the application. 1926 1927.. [#] If x is very close to an exact integer multiple of y, it's possible for 1928 ``x//y`` to be one larger than ``(x-x%y)//y`` due to rounding. In such 1929 cases, Python returns the latter result, in order to preserve that 1930 ``divmod(x,y)[0] * y + x % y`` be very close to ``x``. 1931 1932.. [#] The Unicode standard distinguishes between :dfn:`code points` 1933 (e.g. U+0041) and :dfn:`abstract characters` (e.g. "LATIN CAPITAL LETTER A"). 1934 While most abstract characters in Unicode are only represented using one 1935 code point, there is a number of abstract characters that can in addition be 1936 represented using a sequence of more than one code point. For example, the 1937 abstract character "LATIN CAPITAL LETTER C WITH CEDILLA" can be represented 1938 as a single :dfn:`precomposed character` at code position U+00C7, or as a 1939 sequence of a :dfn:`base character` at code position U+0043 (LATIN CAPITAL 1940 LETTER C), followed by a :dfn:`combining character` at code position U+0327 1941 (COMBINING CEDILLA). 1942 1943 The comparison operators on strings compare at the level of Unicode code 1944 points. This may be counter-intuitive to humans. For example, 1945 ``"\u00C7" == "\u0043\u0327"`` is ``False``, even though both strings 1946 represent the same abstract character "LATIN CAPITAL LETTER C WITH CEDILLA". 1947 1948 To compare strings at the level of abstract characters (that is, in a way 1949 intuitive to humans), use :func:`unicodedata.normalize`. 1950 1951.. [#] Due to automatic garbage-collection, free lists, and the dynamic nature of 1952 descriptors, you may notice seemingly unusual behaviour in certain uses of 1953 the :keyword:`is` operator, like those involving comparisons between instance 1954 methods, or constants. Check their documentation for more info. 1955 1956.. [#] The power operator ``**`` binds less tightly than an arithmetic or 1957 bitwise unary operator on its right, that is, ``2**-1`` is ``0.5``. 1958 1959.. [#] The ``%`` operator is also used for string formatting; the same 1960 precedence applies. 1961