1 2Documentation 3============= 4 5.. toctree:: 6 :maxdepth: 2 7 8Data model for ASN.1 types 9-------------------------- 10 11All ASN.1 types could be categorized into two groups: scalar (also 12called simple or primitive) and constructed. The first group is 13populated by well-known types like Integer or String. Members of 14constructed group hold other types (simple or constructed) as their 15inner components, thus they are semantically close to a programming 16language records or lists. 17 18In pyasn1, all ASN.1 types and values are implemented as Python 19objects. The same pyasn1 object can represent either ASN.1 type 20and/or value depending of the presence of value initializer on object 21instantiation. We will further refer to these as *pyasn1 type object* 22versus *pyasn1 value object*. 23 24Primitive ASN.1 types are implemented as immutable scalar objects. 25There values could be used just like corresponding native Python 26values (integers, strings/bytes etc) and freely mixed with them in 27expressions. 28 29.. code-block:: pycon 30 31 >>> from pyasn1.type import univ 32 >>> asn1IntegerValue = univ.Integer(12) 33 >>> asn1IntegerValue - 2 34 10 35 >>> univ.OctetString('abc') == 'abc' 36 True # Python 2 37 >>> univ.OctetString(b'abc') == b'abc' 38 True # Python 3 39 40It would be an error to perform an operation on a pyasn1 type object 41as it holds no value to deal with: 42 43.. code-block:: pycon 44 45 >>> from pyasn1.type import univ 46 >>> asn1IntegerType = univ.Integer() 47 >>> asn1IntegerType - 2 48 ... 49 pyasn1.error.PyAsn1Error: No value for __coerce__() 50 51 52Scalar types 53------------ 54 55In the sub-sections that follow we will explain pyasn1 mapping to 56those primitive ASN.1 types. Both, ASN.1 notation and corresponding 57pyasn1 syntax will be given in each case. 58 59Boolean type 60++++++++++++ 61 62*BOOLEAN* is the simplest type those values could be either True or 63False. 64 65.. code-block:: bash 66 67 ;; type specification 68 FunFactorPresent ::= BOOLEAN 69 70 ;; values declaration and assignment 71 pythonFunFactor FunFactorPresent ::= TRUE 72 cobolFunFactor FunFactorPresent :: FALSE 73 74And here's pyasn1 version of :py:class:`~pyasn1.type.univ.Boolean`: 75 76.. code-block:: pycon 77 78 >>> from pyasn1.type import univ 79 >>> class FunFactorPresent(univ.Boolean): pass 80 ... 81 >>> pythonFunFactor = FunFactorPresent(True) 82 >>> cobolFunFactor = FunFactorPresent(False) 83 >>> pythonFunFactor 84 FunFactorPresent('True(1)') 85 >>> cobolFunFactor 86 FunFactorPresent('False(0)') 87 >>> pythonFunFactor == cobolFunFactor 88 False 89 >>> 90 91Null type 92+++++++++ 93 94The *NULL* type is sometimes used to express the absence of 95information. 96 97.. code-block:: bash 98 99 ;; type specification 100 Vote ::= CHOICE { 101 agreed BOOLEAN, 102 skip NULL 103 } 104 105 ;; value declaration and assignment 106 myVote Vote ::= skip:NULL 107 108We will explain the CHOICE type later on, meanwhile the 109:py:class:`~pyasn1.type.univ.Null` type: 110 111.. code-block:: pycon 112 113 >>> from pyasn1.type import univ 114 >>> skip = univ.Null() 115 >>> skip 116 Null('') 117 >>> 118 119Integer type 120++++++++++++ 121 122ASN.1 defines the values of *INTEGER* type as negative or positive of 123whatever length. This definition plays nicely with Python as the 124latter places no limit on Integers. However, some ASN.1 125implementations may impose certain limits of integer value ranges. 126Keep that in mind when designing new data structures. 127 128.. code-block:: bash 129 130 ;; values specification 131 age-of-universe INTEGER ::= 13750000000 132 mean-martian-surface-temperature INTEGER ::= -63 133 134A rather straightforward mapping into pyasn1 - 135:py:class:`~pyasn1.type.univ.Integer`: 136 137.. code-block:: pycon 138 139 >>> from pyasn1.type import univ 140 >>> ageOfUniverse = univ.Integer(13750000000) 141 >>> ageOfUniverse 142 Integer(13750000000) 143 >>> 144 >>> meanMartianSurfaceTemperature = univ.Integer(-63) 145 >>> meanMartianSurfaceTemperature 146 Integer(-63) 147 >>> 148 149ASN.1 allows to assign human-friendly names to particular values of 150an INTEGER type. 151 152.. code-block:: bash 153 154 Temperature ::= INTEGER { 155 freezing(0), 156 boiling(100) 157 } 158 159The Temperature type expressed in pyasn1: 160 161.. code-block:: pycon 162 163 >>> from pyasn1.type import univ, namedval 164 >>> class Temperature(univ.Integer): 165 ... namedValues = namedval.NamedValues(('freezing', 0), ('boiling', 100)) 166 ... 167 >>> t = Temperature(0) 168 >>> t 169 Temperature('freezing(0)') 170 >>> t + 1 171 Temperature(1) 172 >>> t + 100 173 Temperature('boiling(100)') 174 >>> t = Temperature('boiling') 175 >>> t 176 Temperature('boiling(100)') 177 >>> Temperature('boiling') / 2 178 Temperature(50) 179 >>> -1 < Temperature('freezing') 180 True 181 >>> 47 > Temperature('boiling') 182 False 183 184These values labels have no effect on Integer type operations, any value 185still could be assigned to a type (information on value constraints will 186follow further in the documentation). 187 188Enumerated type 189+++++++++++++++ 190 191ASN.1 *ENUMERATED* type differs from an Integer type in a number of 192ways. Most important is that its instance can only hold a value that 193belongs to a set of values specified on type declaration. 194 195.. code-block:: bash 196 197 error-status ::= ENUMERATED { 198 no-error(0), 199 authentication-error(10), 200 authorization-error(20), 201 general-failure(51) 202 } 203 204When constructing :py:class:`~pyasn1.type.univ.Enumerated` type we 205will use two pyasn1 features: values labels (as mentioned above) and 206value constraint (will be described in more details later on). 207 208.. code-block:: pycon 209 210 >>> from pyasn1.type import univ, namedval, constraint 211 >>> class ErrorStatus(univ.Enumerated): 212 ... namedValues = namedval.NamedValues( 213 ... ('no-error', 0), 214 ... ('authentication-error', 10), 215 ... ('authorization-error', 20), 216 ... ('general-failure', 51) 217 ... ) 218 ... subtypeSpec = univ.Enumerated.subtypeSpec + \ 219 ... constraint.SingleValueConstraint(0, 10, 20, 51) 220 ... 221 >>> errorStatus = univ.ErrorStatus('no-error') 222 >>> errorStatus 223 ErrorStatus('no-error(0)') 224 >>> errorStatus == univ.ErrorStatus('general-failure') 225 False 226 >>> univ.ErrorStatus('non-existing-state') 227 Traceback (most recent call last): 228 ... 229 pyasn1.error.PyAsn1Error: Can't coerce non-existing-state into integer 230 >>> 231 232Particular integer values associated with Enumerated value states have 233no meaning. They should not be used as such or in any kind of math 234operation. Those integer values are only used by codecs to transfer 235state from one entity to another. 236 237Real type 238+++++++++ 239 240Values of the *REAL* type are a three-component tuple of mantissa, 241base and exponent. All three are integers. 242 243.. code-block:: bash 244 245 pi ::= REAL { mantissa 314159, base 10, exponent -5 } 246 247Corresponding pyasn1 :py:class:`~pyasn1.type.univ.Real` objects can be 248initialized with either a three-component tuple or a Python float. 249Infinite values could be expressed in a way, compatible with Python 250float type. 251 252.. code-block:: pycon 253 254 >>> from pyasn1.type import univ 255 >>> pi = univ.Real((314159, 10, -5)) 256 >>> pi 257 Real((314159, 10,-5)) 258 >>> float(pi) 259 3.14159 260 >>> pi == univ.Real(3.14159) 261 True 262 >>> univ.Real('inf') 263 Real('inf') 264 >>> univ.Real('-inf') == float('-inf') 265 True 266 >>> 267 268If a Real object is initialized from a Python float or yielded by a math 269operation, the base is set to decimal 10 (what affects encoding). 270 271Bit string type 272+++++++++++++++ 273 274ASN.1 *BIT STRING* type holds opaque binary data of an arbitrarily 275length. A BIT STRING value could be initialized by either a binary 276(base 2) or hex (base 16) value. 277 278.. code-block:: bash 279 280 public-key BIT STRING ::= '1010111011110001010110101101101 281 1011000101010000010110101100010 282 0110101010000111101010111111110'B 283 284 signature BIT STRING ::= 'AF01330CD932093392100B39FF00DE0'H 285 286The pyasn1 :py:class:`~pyasn1.type.univ.BitString` objects can 287initialize from native ASN.1 notation (base 2 or base 16 strings) or 288from a Python tuple of binary components. 289 290.. code-block:: pycon 291 292 >>> from pyasn1.type import univ 293 >>> publicKey = univ.BitString( 294 ... binValue='1010111011110001010110101101101' 295 ... '1011000101010000010110101100010' 296 ... '0110101010000111101010111111110' 297 ) 298 >>> publicKey 299 BitString(binValue='101011101111000101011010110110110110001010100000101101011000100110101010000111101010111111110') 300 >>> signature = univ.BitString( 301 ... hexValue='AF01330CD932093392100B39FF00DE0' 302 ... ) 303 >>> signature 304 BitString(binValue='1010111100000001001100110000110011011001001100100000100100110011100100100001000000001011001110011111111100000000110111100000') 305 >>> fingerprint = univ.BitString( 306 ... (1, 0, 1, 1 ,0, 1, 1, 1, 0, 1, 0, 1) 307 ... ) 308 >>> fingerprint 309 BitString(binValue='101101110101') 310 >>> 311 312Another BIT STRING initialization method supported by ASN.1 notation 313is to specify only 1-th bits along with their human-friendly label and 314bit offset relative to the beginning of the bit string. With this 315method, all not explicitly mentioned bits are doomed to be zeros. 316 317.. code-block:: bash 318 319 bit-mask BIT STRING ::= { 320 read-flag(0), 321 write-flag(2), 322 run-flag(4) 323 } 324 325To express this in pyasn1, we will employ the named values feature (as 326with Enumeration type). 327 328.. code-block:: pycon 329 330 >>> from pyasn1.type import univ, namedval 331 >>> class BitMask(univ.BitString): 332 ... namedValues = namedval.NamedValues( 333 ... ('read-flag', 0), 334 ... ('write-flag', 2), 335 ... ('run-flag', 4) 336 ... ) 337 >>> bitMask = BitMask('read-flag,run-flag') 338 >>> bitMask 339 BitMask(binValue='10001') 340 >>> tuple(bitMask) 341 (1, 0, 0, 0, 1) 342 >>> bitMask[4] 343 1 344 >>> 345 346The BitString objects mimic the properties of Python tuple type in 347part of immutable sequence object protocol support. 348 349OctetString type 350++++++++++++++++ 351 352The *OCTET STRING* type is a confusing subject. According to ASN.1 353specification, this type is similar to BIT STRING, the major 354difference is that the former operates in 8-bit chunks of data. What 355is important to note, is that OCTET STRING was NOT designed to handle 356text strings - the standard provides many other types specialized for 357text content. For that reason, ASN.1 forbids to initialize OCTET 358STRING values with "quoted text strings", only binary or hex 359initializers, similar to BIT STRING ones, are allowed. 360 361.. code-block:: bash 362 363 thumbnail OCTET STRING ::= '1000010111101110101111000000111011'B 364 thumbnail OCTET STRING ::= 'FA9823C43E43510DE3422'H 365 366However, ASN.1 users (e.g. protocols designers) seem to ignore the 367original purpose of the OCTET STRING type - they used it for handling 368all kinds of data, including text strings. 369 370.. code-block:: bash 371 372 welcome-message OCTET STRING ::= "Welcome to ASN.1 wilderness!" 373 374In pyasn1, we have taken a liberal approach and allowed both BIT 375STRING style and quoted text initializers for the 376:py:class:`~pyasn1.type.univ.OctetString` objects. To avoid possible 377collisions, quoted text is the default initialization syntax. 378 379.. code-block:: pycon 380 381 >>> from pyasn1.type import univ 382 >>> thumbnail = univ.OctetString( 383 ... binValue='1000010111101110101111000000111011' 384 ... ) 385 >>> thumbnail 386 OctetString(hexValue='85eebcec0') 387 >>> thumbnail = univ.OctetString( 388 ... hexValue='FA9823C43E43510DE3422' 389 ... ) 390 >>> thumbnail 391 OctetString(hexValue='fa9823c43e4351de34220') 392 >>> 393 394Most frequent usage of the OctetString class is to instantiate it with 395a text string. 396 397.. code-block:: pycon 398 399 >>> from pyasn1.type import univ 400 >>> welcomeMessage = univ.OctetString('Welcome to ASN.1 wilderness!') 401 >>> welcomeMessage 402 OctetString(b'Welcome to ASN.1 wilderness!') 403 >>> print('%s' % welcomeMessage) 404 Welcome to ASN.1 wilderness! 405 >>> welcomeMessage[11:16] 406 OctetString(b'ASN.1') 407 >>> 408 409OctetString objects support the immutable sequence object protocol. 410In other words, they behave like Python 3 bytes (or Python 2 strings). 411When running pyasn1 on Python 3, it's better to use the bytes objects for 412OctetString instantiation, as it's more reliable and efficient. 413 414Additionally, OctetString's can also be instantiated with a sequence of 4158-bit integers (ASCII codes). 416 417.. code-block:: pycon 418 419 >>> univ.OctetString((77, 101, 101, 103, 111)) 420 OctetString(b'Meego') 421 422It is sometimes convenient to express OctetString instances as 8-bit 423characters (Python 3 bytes or Python 2 strings) or 8-bit integers. 424 425.. code-block:: pycon 426 427 >>> octetString = univ.OctetString('ABCDEF') 428 >>> octetString.asNumbers() 429 (65, 66, 67, 68, 69, 70) 430 >>> octetString.asOctets() 431 b'ABCDEF' 432 433ObjectIdentifier type 434+++++++++++++++++++++ 435 436Values of the *OBJECT IDENTIFIER* type are sequences of integers that 437could be used to identify virtually anything in the world. Various 438ASN.1-based protocols employ OBJECT IDENTIFIERs for their own 439identification needs. 440 441.. code-block:: bash 442 443 internet-id OBJECT IDENTIFIER ::= { 444 iso(1) identified-organization(3) dod(6) internet(1) 445 } 446 447One of the natural ways to map OBJECT IDENTIFIER type into a Python 448one is to use Python tuples of integers. So this approach is taken by 449pyasn1's :py:class:`~pyasn1.type.univ.ObjectIdentifier` class. 450 451.. code-block:: pycon 452 453 >>> from pyasn1.type import univ 454 >>> internetId = univ.ObjectIdentifier((1, 3, 6, 1)) 455 >>> internetId 456 ObjectIdentifier('1.3.6.1') 457 >>> internetId[2] 458 6 459 >>> internetId[1:3] 460 ObjectIdentifier('3.6') 461 462A more human-friendly "dotted" notation is also supported. 463 464.. code-block:: pycon 465 466 >>> from pyasn1.type import univ 467 >>> univ.ObjectIdentifier('1.3.6.1') 468 ObjectIdentifier('1.3.6.1') 469 470Symbolic names of the arcs of object identifier, sometimes present in 471ASN.1 specifications, are not preserved and used in pyasn1 objects. 472 473The ObjectIdentifier objects mimic the properties of Python tuple type in 474part of immutable sequence object protocol support. 475 476Any type 477++++++++ 478 479The ASN.1 ANY type is a kind of wildcard or placeholder that matches 480any other type without knowing it in advance. ANY has no base tag. 481 482.. code-block:: bash 483 484 Error ::= SEQUENCE { 485 code INTEGER, 486 parameter ANY DEFINED BY code 487 } 488 489The ANY type is frequently used in specifications, where exact type is 490not yet agreed upon between communicating parties or the number of 491possible alternatives of a type is infinite. Sometimes an auxiliary 492selector is kept around to help parties indicate the kind of ANY 493payload in effect ("code" in the example above). 494 495Values of the ANY type contain serialized ASN.1 value(s) in form of an 496octet string. Therefore pyasn1 :py:class:`~pyasn1.type.univ.Any` value 497object share the properties of pyasn1 OctetString object. 498 499.. code-block:: pycon 500 501 >>> from pyasn1.type import univ 502 >>> someValue = univ.Any(b'\x02\x01\x01') 503 >>> someValue 504 Any(b'\x02\x01\x01') 505 >>> str(someValue) 506 '\x02\x01\x01' 507 >>> bytes(someValue) 508 b'\x02\x01\x01' 509 >>> 510 511Receiving application is supposed to explicitly deserialize the 512content of Any value object, possibly using auxiliary selector for 513figuring out its ASN.1 type to pick appropriate decoder. 514 515There will be some more talk and code snippets covering Any type in 516the codecs chapters that follow. 517 518Character string types 519++++++++++++++++++++++ 520 521ASN.1 standard introduces a diverse set of text-specific types. All of 522them were designed to handle various types of characters. Some of 523these types seem be obsolete now days, as their target technologies are 524gone. Another issue to be aware of is that raw OCTET STRING type is 525sometimes used in practice by ASN.1 users instead of specialized 526character string types, despite explicit prohibition imposed by ASN.1 527specification. 528 529The two types are specific to ASN.1 are NumericString and PrintableString. 530 531.. code-block:: bash 532 533 welcome-message ::= PrintableString { 534 "Welcome to ASN.1 text types" 535 } 536 537 dial-pad-numbers ::= NumericString { 538 "0", "1", "2", "3", "4", "5", "6", "7", "8", "9" 539 } 540 541Their pyasn1 implementations are 542:py:class:`~pyasn1.type.char.PrintableString` and 543:py:class:`~pyasn1.type.char.NumericString`: 544 545.. code-block:: pycon 546 547 >>> from pyasn1.type import char 548 >>> '%s' % char.PrintableString("Welcome to ASN.1 text types") 549 'Welcome to ASN.1 text types' 550 >>> dialPadNumbers = char.NumericString( 551 "0" "1" "2" "3" "4" "5" "6" "7" "8" "9" 552 ) 553 >>> dialPadNumbers 554 NumericString(b'0123456789') 555 >>> 556 557The :py:class:`~pyasn1.type.char.VisibleString`, 558:py:class:`~pyasn1.type.char.IA5String`, 559:py:class:`~pyasn1.type.char.TeletexString`, 560:py:class:`~pyasn1.type.char.VideotexString`, 561:py:class:`~pyasn1.type.char.GraphicString` and 562:py:class:`~pyasn1.type.char.GeneralString` types came to ASN.1 from 563ISO standards on character sets. 564 565.. code-block:: pycon 566 567 >>> from pyasn1.type import char 568 >>> char.VisibleString("abc") 569 VisibleString(b'abc') 570 >>> char.IA5String('abc') 571 IA5String(b'abc') 572 >>> char.TeletexString('abc') 573 TeletexString(b'abc') 574 >>> char.VideotexString('abc') 575 VideotexString(b'abc') 576 >>> char.GraphicString('abc') 577 GraphicString(b'abc') 578 >>> char.GeneralString('abc') 579 GeneralString(b'abc') 580 >>> 581 582The last three types are relatively recent addition to the family of 583character string types: :py:class:`~pyasn1.type.char.UniversalString`, 584:py:class:`~pyasn1.type.char.BMPString` and 585:py:class:`~pyasn1.type.char.UTF8String`. 586 587.. code-block:: pycon 588 589 >>> from pyasn1.type import char 590 >>> char.UniversalString("abc") 591 UniversalString(b'abc') 592 >>> char.BMPString('abc') 593 BMPString(b'abc') 594 >>> char.UTF8String('abc') 595 UTF8String(b'abc') 596 >>> utf8String = char.UTF8String('У попа была собака') 597 >>> utf8String 598 UTF8String(b'\xd0\xa3 \xd0\xbf\xd0\xbe\xd0\xbf\xd0\xb0 \xd0\xb1\xd1\x8b\xd0\xbb\xd0\xb0\xd1\x81\xd0\xbe\xd0\xb1\xd0\xb0\xd0\xba\xd0\xb0') 599 >>> print(utf8String) 600 У попа была собака 601 >>> 602 603In pyasn1, all character type objects behave like Python strings. None 604of them is currently constrained in terms of valid alphabet so it's up 605to the data source to keep an eye on data validation for these types. 606 607Useful types 608++++++++++++ 609 610There are three so-called useful types defined in the standard: 611:py:class:`~pyasn1.type.useful.ObjectDescriptor`, 612:py:class:`~pyasn1.type.useful.GeneralizedTime` and 613:py:class:`~pyasn1.type.useful.UTCTime`. They all are subtypes of 614GraphicString or VisibleString types therefore useful types are 615character string types. 616 617It's advised by the ASN.1 standard to have an instance of 618ObjectDescriptor type holding a human-readable description of 619corresponding instance of OBJECT IDENTIFIER type. There are no formal 620linkage between these instances and provision for ObjectDescriptor 621uniqueness in the standard. 622 623.. code-block:: pycon 624 625 >>> from pyasn1.type import useful 626 >>> descrBER = useful.ObjectDescriptor( 627 "Basic encoding of a single ASN.1 type" 628 ) 629 >>> 630 631GeneralizedTime and UTCTime types are designed to hold a 632human-readable timestamp in a universal and unambiguous form. The 633former provides more flexibility in notation while the latter is more 634strict but has Y2K issues. 635 636.. code-block:: bash 637 638 ;; Mar 8 2010 12:00:00 MSK 639 moscow-time GeneralizedTime ::= "20110308120000.0" 640 ;; Mar 8 2010 12:00:00 UTC 641 utc-time GeneralizedTime ::= "201103081200Z" 642 ;; Mar 8 1999 12:00:00 UTC 643 utc-time UTCTime ::= "9803081200Z" 644 645In pyasn1 parlance: 646 647.. code-block:: pycon 648 649 >>> from pyasn1.type import useful 650 >>> moscowTime = useful.GeneralizedTime("20110308120000.0") 651 >>> utcTime = useful.UTCTime("9803081200Z") 652 >>> 653 654Despite their intended use, these types possess no special, time-related, 655handling in pyasn1. They are just printable strings. 656 657Tagging 658------- 659 660In order to proceed to the Constructed ASN.1 types, we will first have 661to introduce the concept of tagging (and its pyasn1 implementation), as 662some of the Constructed types rely upon the tagging feature. 663 664When a value is coming into an ASN.1-based system (received from a network 665or read from some storage), the receiving entity has to determine the 666type of the value to interpret and verify it accordingly. 667 668Historically, the first data serialization protocol introduced in 669ASN.1 was BER (Basic Encoding Rules). According to BER, any serialized 670value is packed into a triplet of (Type, Length, Value) where Type is a 671code that identifies the value (which is called *tag* in ASN.1), 672length is the number of bytes occupied by the value in its serialized form 673and value is ASN.1 value in a form suitable for serial transmission or storage. 674For that reason almost every ASN.1 type has a tag (which is actually a 675BER type) associated with it by default. 676 677An ASN.1 tag could be viewed as a tuple of three numbers: 678(Class, Format, Number). While Number identifies a tag, Class component 679is used to create scopes for Numbers. Four scopes are currently defined: 680UNIVERSAL, context-specific, APPLICATION and PRIVATE. The Format component 681is actually a one-bit flag - zero for tags associated with scalar types, 682and one for constructed types (will be discussed later on). 683 684.. code-block:: bash 685 686 MyIntegerType ::= [12] INTEGER 687 MyOctetString ::= [APPLICATION 0] OCTET STRING 688 689In pyasn1, tags are implemented as immutable, tuple-like objects: 690 691.. code-block:: pycon 692 693 >>> from pyasn1.type import tag 694 >>> myTag = tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 10) 695 >>> myTag 696 Tag(tagClass=128, tagFormat=0, tagId=10) 697 >>> tuple(myTag) 698 (128, 0, 10) 699 >>> myTag[2] 700 10 701 >>> myTag == tag.Tag(tag.tagClassApplication, tag.tagFormatSimple, 10) 702 False 703 >>> 704 705Default tag, associated with any ASN.1 type, could be extended or 706replaced to make new type distinguishable from its ancestor. The 707standard provides two modes of tag mangling - IMPLICIT and EXPLICIT. 708 709EXPLICIT mode works by appending new tag to the existing ones thus 710creating an ordered set of tags. This set will be considered as a 711whole for type identification and encoding purposes. Important 712property of EXPLICIT tagging mode is that it preserves base type 713information in encoding what makes it possible to completely recover 714type information from encoding. 715 716When tagging in IMPLICIT mode, the outermost existing tag is dropped 717and replaced with a new one. 718 719.. code-block:: bash 720 721 MyIntegerType ::= [12] IMPLICIT INTEGER 722 MyOctetString ::= [APPLICATION 0] EXPLICIT OCTET STRING 723 724To model both modes of tagging, a specialized container TagSet object 725(holding zero, one or more Tag objects) is used in pyasn1. 726 727.. code-block:: pycon 728 729 >>> from pyasn1.type import tag 730 >>> tagSet = tag.TagSet( 731 ... # base tag (OBSOLETE AND NOT USED ANYMORE) 732 ... (), 733 ... # effective tag 734 ... tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 10) 735 ... ) 736 >>> tagSet 737 TagSet((), Tag(tagClass=128, tagFormat=0, tagId=10)) 738 >>> tagSet.getBaseTag() 739 Tag(tagClass=128, tagFormat=0, tagId=10) 740 >>> tagSet = tagSet.tagExplicitly(tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 20)) 741 >>> tagSet 742 TagSet((), Tag(tagClass=128, tagFormat=0, tagId=10), 743 Tag(tagClass=128, tagFormat=32, tagId=20)) 744 >>> tagSet = tagSet.tagExplicitly(tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 30)) 745 >>> tagSet 746 TagSet((), Tag(tagClass=128, tagFormat=0, tagId=10), 747 Tag(tagClass=128, tagFormat=32, tagId=20), 748 Tag(tagClass=128, tagFormat=32, tagId=30)) 749 >>> tagSet = tagSet.tagImplicitly(tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 40)) 750 >>> tagSet 751 TagSet((), Tag(tagClass=128, tagFormat=0, tagId=10), 752 Tag(tagClass=128, tagFormat=32, tagId=20), 753 Tag(tagClass=128, tagFormat=32, tagId=40)) 754 >>> 755 756As a side note: the "base tag" concept is now obsolete and not used. 757The "effective tag" is the one that always appears in encoding and is 758used on tagSets comparison. 759 760Any two TagSet objects could be compared to see if one is a derivative 761of the other. Figuring this out is also useful in cases when a type-specific 762data processing algorithms are to be chosen. 763 764.. code-block:: pycon 765 766 >>> from pyasn1.type import tag 767 >>> tagSet1 = tag.TagSet( 768 ... # base tag (OBSOLETE AND NOT USED ANYMORE) 769 ... (), 770 ... # effective tag 771 ... tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 10) 772 ... ) 773 >>> tagSet2 = tagSet1.tagExplicitly(tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 20)) 774 >>> tagSet1.isSuperTagSetOf(tagSet2) 775 True 776 >>> tagSet2.isSuperTagSetOf(tagSet1) 777 False 778 >>> 779 780We will complete this discussion on tagging with a real-world example. The 781following ASN.1 tagged type: 782 783.. code-block:: bash 784 785 MyIntegerType ::= [12] EXPLICIT INTEGER 786 787could be expressed in pyasn1 like this: 788 789.. code-block:: pycon 790 791 >>> from pyasn1.type import univ, tag 792 >>> class MyIntegerType(univ.Integer): 793 ... tagSet = univ.Integer.tagSet.tagExplicitly(tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 12)) 794 >>> myInteger = MyIntegerType(12345) 795 >>> myInteger.tagSet 796 TagSet((), Tag(tagClass=0, tagFormat=0, tagId=2), 797 Tag(tagClass=128, tagFormat=32, tagId=12)) 798 >>> 799 800Referring to the above code, the tagSet class attribute is a property 801of any pyasn1 type object that assigns default tagSet to a pyasn1 802value object. This default tagSet specification can be ignored and 803effectively replaced by some other tagSet value passed on object 804instantiation. 805 806It's important to understand that the tag set property of pyasn1 type/value 807object can never be modified in place. In other words, a pyasn1 type/value 808object can never change its tags. The only way is to create a new pyasn1 809type/value object and associate different tag set with it. 810 811Constructed types 812----------------- 813 814Besides scalar types, ASN.1 specifies so-called constructed ones - these 815are capable of holding one or more values of other types, both scalar 816and constructed. 817 818In pyasn1 implementation, constructed ASN.1 types behave like 819Python sequences, and also support additional component addressing methods, 820specific to particular constructed type. 821 822Sequence and Set types 823++++++++++++++++++++++ 824 825The *SEQUENCE* and *SET* types have many similar properties: 826 827* Both can hold any number of inner components of different types. 828* Every component has a human-friendly identifier. 829* Any component can have a default value. 830* Some components can be absent. 831 832However, :py:class:`~pyasn1.type.univ.Sequence` type guarantees the 833ordering of Sequence value components to match their declaration 834order. By contrast, components of the 835:py:class:`~pyasn1.type.univ.Set` type can be ordered to best suite 836application's needs. 837 838.. code-block:: bash 839 840 Record ::= SEQUENCE { 841 id INTEGER, 842 room [0] INTEGER OPTIONAL, 843 house [1] INTEGER DEFAULT 0 844 } 845 846Up to this moment, the only method we used for creating new pyasn1 847types is Python sub-classing. With this method, a new, named Python 848class is created what mimics type derivation in ASN.1 grammar. 849However, ASN.1 also allows for defining anonymous subtypes (room and 850house components in the example above). To support anonymous 851subtyping in pyasn1, a cloning operation on an existing pyasn1 type 852object can be invoked what creates a new instance of original object 853with possibly modified properties. 854 855.. code-block:: pycon 856 857 >>> from pyasn1.type import univ, namedtype, tag 858 >>> class Record(univ.Sequence): 859 ... componentType = namedtype.NamedTypes( 860 ... namedtype.NamedType('id', univ.Integer()), 861 ... namedtype.OptionalNamedType( 862 ... 'room', 863 ... univ.Integer().subtype( 864 ... implicitTag=tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 0) 865 ... ) 866 ... ), 867 ... namedtype.DefaultedNamedType( 868 ... 'house', 869 ... univ.Integer(0).subtype( 870 ... implicitTag=tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 1) 871 ... ) 872 ... ) 873 ... ) 874 >>> 875 876All pyasn1 constructed type classes have a class attribute 877**componentType** that represent default type specification. Its 878value is a NamedTypes object. 879 880The NamedTypes class instance holds a sequence of NameType, 881OptionalNamedType or DefaultedNamedType objects which, in turn, refer 882to pyasn1 type objects that represent inner SEQUENCE components 883specification. 884 885Finally, invocation of a subtype() method of pyasn1 type objects in 886the code above returns an implicitly tagged copy of original object. 887 888Once a SEQUENCE or SET type is declared with pyasn1, it can be 889instantiated and initialized (continuing the above code): 890 891.. code-block:: pycon 892 893 >>> record = Record() 894 >>> record['id'] = 123 895 >>> print(record.prettyPrint()) 896 Record: 897 id=123 898 >>> 899 >>> record[1] = 321 900 >>> print(record.prettyPrint()) 901 Record: 902 id=123 903 room=321 904 >>> 905 >>> record.setDefaultComponents() 906 >>> print(record.prettyPrint()) 907 Record: 908 id=123 909 room=321 910 house=0 911 912Inner components of pyasn1 Sequence/Set objects could be accessed 913using the following methods: 914 915.. code-block:: pycon 916 917 >>> record['id'] 918 Integer(123) 919 >>> record[1] 920 Integer(321) 921 >>> record[2] 922 Integer(0) 923 >>> for idx, field in enumerate(record): 924 ... print(record.componentType[idx].name, field) 925 id 123 926 room 321 927 house 0 928 >>> 929 930The Set type share all the properties of Sequence type, and additionally 931support by-tag component addressing (as all Set components have distinct 932types). 933 934.. code-block:: pycon 935 936 >>> from pyasn1.type import univ, namedtype, tag 937 >>> class Gamer(univ.Set): 938 ... componentType = namedtype.NamedTypes( 939 ... namedtype.NamedType('score', univ.Integer()), 940 ... namedtype.NamedType('player', univ.OctetString()), 941 ... namedtype.NamedType('id', univ.ObjectIdentifier()) 942 ... ) 943 >>> gamer = Gamer() 944 >>> gamer.setComponentByType(univ.Integer().tagSet, 121343) 945 >>> gamer.setComponentByType(univ.OctetString().tagSet, 'Pascal') 946 >>> gamer.setComponentByType(univ.ObjectIdentifier().tagSet, (1,3,7,2)) 947 >>> print(gamer.prettyPrint()) 948 Gamer: 949 score=121343 950 player=b'Pascal' 951 id=1.3.7.2 952 953SequenceOf and SetOf types 954++++++++++++++++++++++++++ 955 956Both, *SEQUENCE OF* and *SET OF* types resemble an unlimited size list of 957components. All the components must be of the same type. 958 959.. code-block:: bash 960 961 Progression ::= SEQUENCE OF INTEGER 962 963 arithmeticProgression Progression ::= { 1, 3, 5, 7 } 964 965:py:class:`~pyasn1.type.univ.SequenceOf` and 966:py:class:`~pyasn1.type.univ.SetOf` types are expressed by the very 967similar pyasn1 `list` type objects. Their components can only be addressed by 968position and they both have a property of automatic resize. 969 970To specify inner component type, the **componentType** class 971attribute should refer to another pyasn1 type object. 972 973.. code-block:: pycon 974 975 >>> from pyasn1.type import univ 976 >>> class Progression(univ.SequenceOf): 977 ... componentType = univ.Integer() 978 >>> arithmeticProgression = Progression() 979 >>> arithmeticProgression[1] = 111 980 >>> print(arithmeticProgression.prettyPrint()) 981 Progression: 982 -empty- 111 983 >>> arithmeticProgression[0] = 100 984 >>> print(arithmeticProgression.prettyPrint()) 985 Progression: 986 100 111 987 >>> 988 >>> for element in arithmeticProgression: 989 ... element 990 Integer(100) 991 Integer(111) 992 >>> 993 994Any scalar or constructed pyasn1 type object can serve as an inner 995component. Missing components are prohibited in SequenceOf/SetOf 996value objects. 997 998Choice type 999+++++++++++ 1000 1001Values of ASN.1 *CHOICE* type can contain only a single value of a type 1002from a list of possible alternatives. Alternatives must be ASN.1 types 1003with distinct tags for the whole structure to remain unambiguous. 1004Unlike most other types, CHOICE is an untagged one, e.g. it has no 1005base tag of its own. 1006 1007.. code-block:: bash 1008 1009 CodeOrMessage ::= CHOICE { 1010 code INTEGER, 1011 message OCTET STRING 1012 } 1013 1014In pyasn1 implementation, 1015:py:class:`~pyasn1.type.univ.Choice` object behaves like Set but 1016accepts only a single inner component at a time. It also offers a few 1017additional methods specific to its behaviour. 1018 1019.. code-block:: pycon 1020 1021 >>> from pyasn1.type import univ, namedtype 1022 >>> class CodeOrMessage(univ.Choice): 1023 ... componentType = namedtype.NamedTypes( 1024 ... namedtype.NamedType('code', univ.Integer()), 1025 ... namedtype.NamedType('message', univ.OctetString()) 1026 ... ) 1027 >>> 1028 >>> codeOrMessage = CodeOrMessage() 1029 >>> print(codeOrMessage.prettyPrint()) 1030 CodeOrMessage: 1031 >>> codeOrMessage['code'] = 123 1032 >>> print(codeOrMessage.prettyPrint()) 1033 CodeOrMessage: 1034 code=123 1035 >>> codeOrMessage['message'] = 'my string value' 1036 >>> print(codeOrMessage.prettyPrint()) 1037 CodeOrMessage: 1038 message=b'my string value' 1039 >>> 1040 1041Since there could be only a single inner component value in the pyasn1 1042Choice value object, either of the following methods could be used for 1043fetching it (continuing previous code): 1044 1045.. code-block:: pycon 1046 1047 >>> codeOrMessage.getName() 1048 'message' 1049 >>> codeOrMessage.getComponent() 1050 OctetString(b'my string value') 1051 >>> 1052 1053Subtype constraints 1054------------------- 1055 1056Most ASN.1 types can correspond to an infinite set of values. To adapt 1057to particular application's data model and needs, ASN.1 provides a 1058mechanism for limiting the infinite set to values, that make sense in 1059particular case. Imposing value constraints on an ASN.1 type can also 1060be seen as creating a subtype from its base type. 1061 1062In pyasn1, constraints take shape of immutable objects capable 1063of evaluating given value against constraint-specific requirements. 1064Constraint object is a property of pyasn1 type. Like TagSet property, 1065associated with every pyasn1 type, constraints can never be modified 1066in place. The only way to modify pyasn1 type constraint is to associate 1067new constraint object to a new pyasn1 type object. 1068 1069A handful of different flavors of *constraints* are defined in 1070ASN.1. We will discuss them one by one in the following chapters and 1071also explain how to combine and apply them to types. 1072 1073Single value constraint 1074+++++++++++++++++++++++ 1075 1076This kind of constraint allows for limiting type to a finite, specified set 1077of values. 1078 1079.. code-block:: bash 1080 1081 DialButton ::= OCTET STRING ( 1082 "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" 1083 ) 1084 1085Its pyasn1 implementation would look like: 1086 1087.. code-block:: pycon 1088 1089 >>> from pyasn1.type import constraint 1090 >>> c = constraint.SingleValueConstraint('0','1','2','3','4','5','6','7','8','9') 1091 >>> c 1092 SingleValueConstraint(0, 1, 2, 3, 4, 5, 6, 7, 8, 9) 1093 >>> c('0') 1094 >>> c('A') 1095 Traceback (most recent call last): 1096 ... 1097 ValueConstraintError: 1098 SingleValueConstraint(0, 1, 2, 3, 4, 5, 6, 7, 8, 9) failed at: A 1099 >>> 1100 1101As can be seen in the snippet above, if a value violates the 1102constraint, an exception will be thrown. A constrained pyasn1 type 1103object holds a reference to a constraint object (or their combination, 1104as will be explained later) and calls it for value verification. 1105 1106.. code-block:: pycon 1107 1108 >>> from pyasn1.type import univ, constraint 1109 >>> class DialButton(univ.OctetString): 1110 ... subtypeSpec = constraint.SingleValueConstraint( 1111 ... '0','1','2','3','4','5','6','7','8','9' 1112 ... ) 1113 >>> DialButton('0') 1114 DialButton(b'0') 1115 >>> DialButton('A') 1116 Traceback (most recent call last): 1117 ... 1118 ValueConstraintError: 1119 SingleValueConstraint(0, 1, 2, 3, 4, 5, 6, 7, 8, 9) failed at: A 1120 >>> 1121 1122Constrained pyasn1 value object can never hold a violating value. 1123 1124Value range constraint 1125++++++++++++++++++++++ 1126 1127A pair of values, compliant to a type to be constrained, denote low 1128and upper bounds of allowed range of values of a type. 1129 1130.. code-block:: bash 1131 1132 Teenagers ::= INTEGER (13..19) 1133 1134And in pyasn1 terms: 1135 1136.. code-block:: pycon 1137 1138 >>> from pyasn1.type import univ, constraint 1139 >>> class Teenagers(univ.Integer): 1140 ... subtypeSpec = constraint.ValueRangeConstraint(13, 19) 1141 >>> Teenagers(14) 1142 Teenagers(14) 1143 >>> Teenagers(20) 1144 Traceback (most recent call last): 1145 ... 1146 ValueConstraintError: 1147 ValueRangeConstraint(13, 19) failed at: 20 1148 >>> 1149 1150ASN.1 MIN and MAX operands can be substituted with floating point 1151infinity values. 1152 1153.. code-block:: bash 1154 1155 NegativeInt ::= INTEGER (MIN..-1) 1156 PositiveInt ::= INTEGER (1..MAX) 1157 1158And in pyasn1 terms: 1159 1160.. code-block:: pycon 1161 1162 >>> from pyasn1.type import univ, constraint 1163 >>> class NegativeInt(univ.Integer): 1164 ... subtypeSpec = constraint.ValueRangeConstraint(float('-inf'), -1) 1165 >>> NegativeInt(-1) 1166 NegativeInt(-1) 1167 >>> NegativeInt(0) 1168 Traceback (most recent call last): 1169 ... 1170 ValueConstraintError: 1171 ValueConstraintError: ValueRangeConstraint() failed at: "0" at NegativeInt 1172 >>> class PositiveInt(univ.Integer): 1173 ... subtypeSpec = constraint.ValueRangeConstraint(1, float('inf')) 1174 >> PositiveInt(1) 1175 PositiveInt(1) 1176 >> PositiveInt(4) 1177 PositiveInt(4) 1178 >> PositiveInt(-1) 1179 Traceback (most recent call last): 1180 ... 1181 ValueConstraintError: 1182 ValueConstraintError: ValueRangeConstraint() failed at: "-1" at PositiveInt 1183 1184Value range constraint usually applies to numeric types. 1185 1186Size constraint 1187+++++++++++++++ 1188 1189It is sometimes convenient to set or limit the allowed size of a data 1190item to be sent from one application to another to manage bandwidth 1191and memory consumption issues. Size constraint specifies the lower and 1192upper bounds of the size of a valid value. 1193 1194.. code-block:: bash 1195 1196 TwoBits ::= BIT STRING (SIZE (2)) 1197 1198Express the same grammar in pyasn1: 1199 1200.. code-block:: pycon 1201 1202 >>> from pyasn1.type import univ, constraint 1203 >>> class TwoBits(univ.BitString): 1204 ... subtypeSpec = constraint.ValueSizeConstraint(2, 2) 1205 >>> TwoBits((1,1)) 1206 TwoBits("'11'B") 1207 >>> TwoBits((1,1,0)) 1208 Traceback (most recent call last): 1209 ... 1210 ValueConstraintError: ValueSizeConstraint(2, 2) failed at: (1, 1, 0) 1211 >>> 1212 1213Size constraint can be applied to potentially massive values - bit or 1214octet strings, SEQUENCE OF/SET OF values. 1215 1216Alphabet constraint 1217+++++++++++++++++++ 1218 1219The permitted alphabet constraint is similar to Single value 1220constraint but constraint applies to individual characters of a value. 1221 1222.. code-block:: bash 1223 1224 MorseCode ::= PrintableString (FROM ("."|"-"|" ")) 1225 1226And in pyasn1: 1227 1228.. code-block:: pycon 1229 1230 >>> from pyasn1.type import char, constraint 1231 >>> class MorseCode(char.PrintableString): 1232 ... subtypeSpec = constraint.PermittedAlphabetConstraint(".", "-", " ") 1233 >>> MorseCode("...---...") 1234 MorseCode('...---...') 1235 >>> MorseCode("?") 1236 Traceback (most recent call last): 1237 ... 1238 ValueConstraintError: PermittedAlphabetConstraint(".", "-", " ") failed at: "?" 1239 >>> 1240 1241Current implementation does not handle ranges of characters in 1242constraint (FROM "A".."Z" syntax), one has to list the whole set in a 1243range. 1244 1245Constraint combinations 1246+++++++++++++++++++++++ 1247 1248Up to this moment, we used a single constraint per ASN.1 type. The 1249standard, however, allows for combining multiple individual 1250constraints into intersections, unions and exclusions. 1251 1252In pyasn1 data model, all of these methods of constraint combinations 1253are implemented as constraint-like objects holding individual 1254constraint (or combination) objects. Like terminal constraint objects, 1255combination objects are capable to perform value verification at its 1256set of enclosed constraints according to the logic of particular 1257combination. 1258 1259Constraints intersection verification succeeds only if a value is 1260compliant to each constraint in a set. To begin with, the following 1261specification will constitute a valid telephone number: 1262 1263.. code-block:: bash 1264 1265 PhoneNumber ::= NumericString (FROM ("0".."9")) (SIZE 11) 1266 1267Constraint intersection object serves the logic above: 1268 1269.. code-block:: pycon 1270 1271 >>> from pyasn1.type import char, constraint 1272 >>> class PhoneNumber(char.NumericString): 1273 ... subtypeSpec = constraint.ConstraintsIntersection( 1274 ... constraint.PermittedAlphabetConstraint('0','1','2','3','4','5','6', '7','8','9'), 1275 ... constraint.ValueSizeConstraint(11, 11) 1276 ... ) 1277 >>> PhoneNumber('79039343212') 1278 PhoneNumber('79039343212') 1279 >>> PhoneNumber('?9039343212') 1280 Traceback (most recent call last): 1281 ... 1282 ValueConstraintError: ConstraintsIntersection(PermittedAlphabetConstraint('0','1','2','3','4','5','6','7','8','9'), ValueSizeConstraint(11, 11)) failed at: PermittedAlphabetConstraint('0','1','2','3','4','5','6','7','8','9') failed at: "?039343212" 1283 >>> PhoneNumber('9343212') 1284 Traceback (most recent call last): 1285 ... 1286 ValueConstraintError: 1287 ConstraintsIntersection(PermittedAlphabetConstraint('0','1','2','3','4','5','6','7','8','9'), ValueSizeConstraint(11, 11)) failed at: ValueSizeConstraint(10, 10) failed at: "9343212" 1288 >>> 1289 1290Union of constraints works by making sure that a value is compliant 1291to any of the constraint in a set. For instance: 1292 1293.. code-block:: bash 1294 1295 CapitalOrSmall ::= IA5String (FROM ('A','B','C') | FROM ('a','b','c')) 1296 1297It's important to note, that a value must fully comply to any single 1298constraint in a set. In the specification above, a value of all small 1299or all capital letters is compliant, but a mix of small&capitals is 1300not. Here's its pyasn1 analogue: 1301 1302.. code-block:: pycon 1303 1304 >>> from pyasn1.type import char, constraint 1305 >>> class CapitalOrSmall(char.IA5String): 1306 ... subtypeSpec = constraint.ConstraintsUnion( 1307 ... constraint.PermittedAlphabetConstraint('A','B','C'), 1308 ... constraint.PermittedAlphabetConstraint('a','b','c') 1309 ... ) 1310 >>> CapitalOrSmall('ABBA') 1311 CapitalOrSmall('ABBA') 1312 >>> CapitalOrSmall('abba') 1313 CapitalOrSmall('abba') 1314 >>> CapitalOrSmall('Abba') 1315 Traceback (most recent call last): 1316 ... 1317 ValueConstraintError: ConstraintsUnion(PermittedAlphabetConstraint('A', 'B', 'C'), PermittedAlphabetConstraint('a', 'b', 'c')) failed at: failed for "Abba" 1318 >>> 1319 1320Finally, the exclusion constraint simply negates the logic of value 1321verification at a constraint. In the following example, any integer 1322value is allowed in a type but not zero. 1323 1324.. code-block:: bash 1325 1326 NoZero ::= INTEGER (ALL EXCEPT 0) 1327 1328In pyasn1 the above definition would read: 1329 1330.. code-block:: pycon 1331 1332 >>> from pyasn1.type import univ, constraint 1333 >>> class NoZero(univ.Integer): 1334 ... subtypeSpec = constraint.ConstraintsExclusion( 1335 ... constraint.SingleValueConstraint(0) 1336 ... ) 1337 >>> NoZero(1) 1338 NoZero(1) 1339 >>> NoZero(0) 1340 Traceback (most recent call last): 1341 ... 1342 ValueConstraintError: ConstraintsExclusion(SingleValueConstraint(0)) failed at: 0 1343 >>> 1344 1345The depth of such a constraints tree, built with constraint 1346combination objects at its nodes, has not explicit limit. Value 1347verification is performed in a recursive manner till a definite 1348solution is found. 1349 1350Types relationships 1351+++++++++++++++++++ 1352 1353In the course of data processing in an application, it is sometimes 1354convenient to figure out the type relationships between pyasn1 type or 1355value objects. Formally, two things influence pyasn1 types 1356relationship: *tag set* and *subtype constraints*. One 1357pyasn1 type is considered to be a derivative of another if their 1358TagSet and Constraint objects are a derivation of one another. 1359 1360The following example illustrates the concept (we use the same tagset 1361but different constraints for simplicity): 1362 1363.. code-block:: pycon 1364 1365 >>> from pyasn1.type import univ, constraint 1366 >>> i1 = univ.Integer(subtypeSpec=constraint.ValueRangeConstraint(3,8)) 1367 >>> i2 = univ.Integer(subtypeSpec=constraint.ConstraintsIntersection( 1368 ... constraint.ValueRangeConstraint(3,8), 1369 ... constraint.ValueRangeConstraint(4,7) 1370 ... ) ) 1371 >>> i1.isSameTypeWith(i2) 1372 False 1373 >>> i1.isSuperTypeOf(i2) 1374 True 1375 >>> i1.isSuperTypeOf(i1) 1376 True 1377 >>> i2.isSuperTypeOf(i1) 1378 False 1379 >>> 1380 1381As can be seen in the above code snippet, there are two methods of any 1382pyasn1 type/value object that test types for their relationship: 1383*isSameTypeWith()* and *isSuperTypeOf()*. The former is 1384self-descriptive while the latter yields true if the argument appears 1385to be a pyasn1 object which has tagset and constraints derived from 1386those of the object being called. 1387 1388Serialization codecs 1389-------------------- 1390 1391In ASN.1 context, `codec <http://en.wikipedia.org/wiki/Codec>`_ 1392is a program that transforms between concrete data structures and a stream 1393of octets, suitable for transmission over the wire. This serialized form of 1394data is sometimes called *substrate* or *essence*. 1395 1396In pyasn1 implementation, substrate takes shape of Python 3 bytes or 1397Python 2 string objects. 1398 1399One of the properties of a codec is its ability to cope with 1400incomplete data and/or substrate what implies codec to be stateful. In 1401other words, when decoder runs out of substrate and data item being 1402recovered is still incomplete, stateful codec would suspend and 1403complete data item recovery whenever the rest of substrate becomes 1404available. Similarly, stateful encoder would encode data items in 1405multiple steps waiting for source data to arrive. Codec restartability 1406is especially important when application deals with large volumes of 1407data and/or runs on low RAM. For an interesting discussion on codecs 1408options and design choices, refer to `Apache ASN.1 project 1409<http://directory.apache.org/subprojects/asn1/>`_ . 1410 1411As of this writing, codecs implemented in pyasn1 are all stateless, 1412mostly to keep the code simple. 1413 1414The pyasn1 package currently supports 1415`BER <http://en.wikipedia.org/wiki/Basic_encoding_rules>`_ codec and 1416its variations -- 1417`CER <http://en.wikipedia.org/wiki/Canonical_encoding_rules>`_ and 1418`DER <http://en.wikipedia.org/wiki/Distinguished_encoding_rules>`_. 1419More ASN.1 codecs are planned for implementation in the future. 1420 1421Encoders 1422++++++++ 1423 1424Encoder is used for transforming pyasn1 value objects into substrate. 1425Only pyasn1 value objects could be serialized, attempts to process 1426pyasn1 type objects will cause encoder failure. 1427 1428The following code will create a pyasn1 Integer object and serialize 1429it with BER encoder: 1430 1431.. code-block:: pycon 1432 1433 >>> from pyasn1.type import univ 1434 >>> from pyasn1.codec.ber import encoder 1435 >>> encoder.encode(univ.Integer(123456)) 1436 b'\x02\x03\x01\xe2@' 1437 >>> 1438 1439BER standard also defines a so-called *indefinite length* 1440encoding form which makes large data items processing more memory 1441efficient. It is mostly useful when encoder does not have the whole 1442value all at once and the length of the value can not be determined at 1443the beginning of encoding. 1444 1445*Constructed encoding* is another feature of BER closely related to 1446the indefinite length form. In essence, a large scalar value (such as 1447ASN.1 character BitString type) could be chopped into smaller chunks 1448by encoder and transmitted incrementally to limit memory consumption. 1449Unlike indefinite length case, the length of the whole value must be 1450known in advance when using constructed, definite length encoding 1451form. 1452 1453Since pyasn1 codecs are not restartable, pyasn1 encoder may only 1454encode data item all at once. However, even in this case, generating 1455indefinite length encoding may help a low-memory receiver, running a 1456restartable decoder, to process a large data item. 1457 1458.. code-block:: pycon 1459 1460 >>> from pyasn1.type import univ 1461 >>> from pyasn1.codec.ber import encoder 1462 >>> encoder.encode( 1463 ... univ.OctetString('The quick brown fox jumps over the lazy dog'), 1464 ... defMode=False, 1465 ... maxChunkSize=8 1466 ... ) 1467 b'$\x80\x04\x08The quic\x04\x08k brown \x04\x08fox jump\x04\x08s over \t\x04\x08he lazy \x04\x03dog\x00\x00' 1468 >>> 1469 >>> encoder.encode( 1470 ... univ.OctetString('The quick brown fox jumps over the lazy dog'), 1471 ... maxChunkSize=8 1472 ... ) 1473 b'$7\x04\x08The quic\x04\x08k brown \x04\x08fox jump\x04\x08s over \t\x04\x08he lazy \x04\x03dog' 1474 1475The *defMode* encoder parameter disables definite length encoding 1476mode, while the optional *maxChunkSize* parameter specifies desired 1477substrate chunk size that influences memory requirements at the 1478decoder's end. 1479 1480To use CER or DER encoders one needs to explicitly import and call them - the 1481APIs are all compatible. 1482 1483.. code-block:: pycon 1484 1485 >>> from pyasn1.type import univ 1486 >>> from pyasn1.codec.ber import encoder as ber_encoder 1487 >>> from pyasn1.codec.cer import encoder as cer_encoder 1488 >>> from pyasn1.codec.der import encoder as der_encoder 1489 >>> ber_encoder.encode(univ.Boolean(True)) 1490 b'\x01\x01\x01' 1491 >>> cer_encoder.encode(univ.Boolean(True)) 1492 b'\x01\x01\xff' 1493 >>> der_encoder.encode(univ.Boolean(True)) 1494 b'\x01\x01\xff' 1495 >>> 1496 1497Decoders 1498++++++++ 1499 1500In the process of decoding, pyasn1 value objects are created and 1501linked to each other, based on the information contained in the 1502substrate. Thus, the original pyasn1 value object(s) are recovered. 1503 1504.. code-block:: pycon 1505 1506 >>> from pyasn1.type import univ 1507 >>> from pyasn1.codec.ber import encoder, decoder 1508 >>> substrate = encoder.encode(univ.Boolean(True)) 1509 >>> decoder.decode(substrate) 1510 (Boolean('True(1)'), b'') 1511 >>> 1512 1513Commenting on the code snippet above, pyasn1 decoder accepts substrate 1514as an argument and returns a tuple of pyasn1 value object (possibly a 1515top-level one in case of constructed object) and unprocessed part of 1516input substrate. 1517 1518All pyasn1 decoders can handle both definite and indefinite length 1519encoding modes automatically, explicit switching into one mode to 1520another is not required. 1521 1522.. code-block:: pycon 1523 1524 >>> from pyasn1.type import univ 1525 >>> from pyasn1.codec.ber import encoder, decoder 1526 >>> substrate = encoder.encode( 1527 ... univ.OctetString('The quick brown fox jumps over the lazy dog'), 1528 ... defMode=False, 1529 ... maxChunkSize=8 1530 ... ) 1531 >>> decoder.decode(substrate) 1532 (OctetString(b'The quick brown fox jumps over the lazy dog'), b'') 1533 >>> 1534 1535Speaking of BER/CER/DER encoding, in many situations substrate may not 1536contain all necessary information needed for complete and accurate 1537ASN.1 values recovery. The most obvious cases include implicitly 1538tagged ASN.1 types and constrained types. 1539 1540As discussed earlier in this tutorial, when an ASN.1 type is implicitly 1541tagged, previous outermost tag is lost and never appears in substrate. 1542If it is the base tag that gets lost, decoder is unable to pick type-specific 1543value decoder at its table of built-in types, and therefore recover 1544the value part, based only on the information contained in substrate. The 1545approach taken by pyasn1 decoder is to use a prototype pyasn1 type object (or 1546a set of them) to *guide* the decoding process by matching [possibly 1547incomplete] tags recovered from substrate with those found in prototype pyasn1 1548type objects (also called pyasn1 specification object further in this 1549document). 1550 1551.. code-block:: pycon 1552 1553 >>> from pyasn1.codec.ber import decoder 1554 >>> decoder.decode(b'\x02\x01\x0c', asn1Spec=univ.Integer()) 1555 Integer(12), b'' 1556 >>> 1557 1558Decoder would neither modify pyasn1 specification object nor use its 1559current values (if it's a pyasn1 value object), but rather use it as a 1560hint for choosing proper decoder and as a pattern for creating new 1561objects: 1562 1563.. code-block:: pycon 1564 1565 >>> from pyasn1.type import univ, tag 1566 >>> from pyasn1.codec.ber import encoder, decoder 1567 >>> i = univ.Integer(12345).subtype( 1568 ... implicitTag=tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 40) 1569 ... ) 1570 >>> substrate = encoder.encode(i) 1571 >>> substrate 1572 b'\x9f(\x0209' 1573 >>> decoder.decode(substrate) 1574 Traceback (most recent call last): 1575 ... 1576 pyasn1.error.PyAsn1Error: TagSet(Tag(tagClass=128, tagFormat=0, tagId=40)) not in asn1Spec 1577 >>> decoder.decode(substrate, asn1Spec=i) 1578 (Integer(12345), b'') 1579 >>> 1580 1581Notice in the example above, that an attempt to run decoder without 1582passing pyasn1 specification object fails because recovered tag does 1583not belong to any of the built-in types. 1584 1585Another important feature of guided decoder operation is the use of 1586values constraints possibly present in pyasn1 specification object. 1587To explain this, we will decode a random integer object into generic Integer 1588and the constrained one. 1589 1590.. code-block:: pycon 1591 1592 >>> from pyasn1.type import univ, constraint 1593 >>> from pyasn1.codec.ber import encoder, decoder 1594 >>> class DialDigit(univ.Integer): 1595 ... subtypeSpec = constraint.ValueRangeConstraint(0,9) 1596 >>> substrate = encoder.encode(univ.Integer(13)) 1597 >>> decoder.decode(substrate) 1598 (Integer(13), b'') 1599 >>> decoder.decode(substrate, asn1Spec=DialDigit()) 1600 Traceback (most recent call last): 1601 ... 1602 ValueConstraintError: 1603 ValueRangeConstraint(0, 9) failed at: 13 1604 >>> 1605 1606Similarly to encoders, to use CER or DER decoders application has to 1607explicitly import and call them - all APIs are compatible. 1608 1609.. code-block:: pycon 1610 1611 >>> from pyasn1.type import univ 1612 >>> from pyasn1.codec.ber import encoder as ber_encoder 1613 >>> substrate = ber_encoder.encode(univ.OctetString('http://pyasn1.sf.net')) 1614 >>> 1615 >>> from pyasn1.codec.ber import decoder as ber_decoder 1616 >>> from pyasn1.codec.cer import decoder as cer_decoder 1617 >>> from pyasn1.codec.der import decoder as der_decoder 1618 >>> 1619 >>> ber_decoder.decode(substrate) 1620 (OctetString(b'http://pyasn1.sf.net'), b'') 1621 >>> cer_decoder.decode(substrate) 1622 (OctetString(b'http://pyasn1.sf.net'), b'') 1623 >>> der_decoder.decode(substrate) 1624 (OctetString(b'http://pyasn1.sf.net'), b'') 1625 >>> 1626 1627Advanced topics 1628--------------- 1629 1630Certain, non-trivial, ASN.1 data structures may require special 1631treatment, especially when running deserialization. 1632 1633Decoding untagged types 1634+++++++++++++++++++++++ 1635 1636It has already been mentioned, that ASN.1 has two "special case" 1637types: CHOICE and ANY. They are different from other types in part of 1638tagging - unless these two are additionally tagged, neither of them 1639will have their own tag. Therefore these types become invisible in 1640substrate and can not be recovered without passing pyasn1 1641specification object to decoder. 1642 1643To explain the issue, we will first prepare a Choice object to deal with: 1644 1645.. code-block:: pycon 1646 1647 >>> from pyasn1.type import univ, namedtype 1648 >>> class CodeOrMessage(univ.Choice): 1649 ... componentType = namedtype.NamedTypes( 1650 ... namedtype.NamedType('code', univ.Integer()), 1651 ... namedtype.NamedType('message', univ.OctetString()) 1652 ... ) 1653 >>> 1654 >>> codeOrMessage = CodeOrMessage() 1655 >>> codeOrMessage['message'] = 'my string value' 1656 >>> print(codeOrMessage.prettyPrint()) 1657 CodeOrMessage: 1658 message=b'my string value' 1659 >>> 1660 1661Let's now encode this Choice object and then decode its substrate 1662with and without pyasn1 specification object: 1663 1664.. code-block:: pycon 1665 1666 >>> from pyasn1.codec.ber import encoder, decoder 1667 >>> substrate = encoder.encode(codeOrMessage) 1668 >>> substrate 1669 b'\x04\x0fmy string value' 1670 >>> encoder.encode(univ.OctetString('my string value')) 1671 b'\x04\x0fmy string value' 1672 >>> 1673 >>> decoder.decode(substrate) 1674 (OctetString(b'my string value'), b'') 1675 >>> codeOrMessage, substrate = decoder.decode(substrate, 1676 asn1Spec=CodeOrMessage()) 1677 >>> print(codeOrMessage.prettyPrint()) 1678 CodeOrMessage: 1679 message=b'my string value' 1680 >>> 1681 1682First thing to notice in the listing above is that the substrate 1683produced for our Choice value object is equivalent to the substrate 1684for an OctetString object initialized to the same value. In other 1685words, any information about the Choice component is absent in 1686encoding. 1687 1688Sure enough, that kind of substrate will decode into an OctetString 1689object, unless original Choice type object is passed to decoder to 1690guide the decoding process. 1691 1692Similarly untagged ANY type behaves differently on decoding phase - 1693when decoder bumps into an Any object in pyasn1 specification, it 1694stops decoding and puts all the substrate into a new Any value object 1695in form of an octet string. Concerned application could then re-run 1696decoder with an additional, more exact pyasn1 specification object to 1697recover the contents of Any object. 1698 1699As it was mentioned elsewhere in this documentation, Any type allows 1700for incomplete or changing ASN.1 specification to be handled 1701gracefully by decoder and applications. 1702 1703To illustrate the working of Any type, we'll have to make the stage by 1704encoding a pyasn1 object and then putting its substrate into an any 1705object. 1706 1707.. code-block:: pycon 1708 1709 >>> from pyasn1.type import univ 1710 >>> from pyasn1.codec.ber import encoder, decoder 1711 >>> innerSubstrate = encoder.encode(univ.Integer(1234)) 1712 >>> innerSubstrate 1713 b'\x02\x02\x04\xd2' 1714 >>> any = univ.Any(innerSubstrate) 1715 >>> any 1716 Any(b'\x02\x02\x04\xd2') 1717 >>> substrate = encoder.encode(any) 1718 >>> substrate 1719 b'\x02\x02\x04\xd2' 1720 >>> 1721 1722As with Choice type encoding, there is no traces of Any type in 1723substrate. Obviously, the substrate we are dealing with, will decode 1724into the inner [Integer] component, unless pyasn1 specification is 1725given to guide the decoder. Continuing previous code: 1726 1727.. code-block:: pycon 1728 1729 >>> from pyasn1.type import univ 1730 >>> from pyasn1.codec.ber import encoder, decoder 1731 1732 >>> decoder.decode(substrate) 1733 (Integer(1234), b'') 1734 >>> any, substrate = decoder.decode(substrate, asn1Spec=univ.Any()) 1735 >>> any 1736 Any(b'\x02\x02\x04\xd2') 1737 >>> decoder.decode(str(any)) 1738 (Integer(1234), b'') 1739 >>> 1740 1741Both CHOICE and ANY types are widely used in practice. Reader is welcome to 1742take a look at 1743`ASN.1 specifications of X.509 applications 1744<http://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt>`_ 1745for more information. 1746 1747Ignoring unknown types 1748++++++++++++++++++++++ 1749 1750When dealing with a loosely specified ASN.1 structure, the receiving 1751end may not be aware of some types present in the substrate. It may be 1752convenient then to turn decoder into a recovery mode. Whilst there, 1753decoder will not bail out when hit an unknown tag but rather treat it 1754as an Any type. 1755 1756.. code-block:: pycon 1757 1758 >>> from pyasn1.type import univ, tag 1759 >>> from pyasn1.codec.ber import encoder, decoder 1760 >>> taggedInt = univ.Integer(12345).subtype( 1761 ... implicitTag=tag.Tag(tag.tagClassContext, tag.tagFormatSimple, 40) 1762 ... ) 1763 >>> substrate = encoder.encode(taggedInt) 1764 >>> decoder.decode(substrate) 1765 Traceback (most recent call last): 1766 ... 1767 pyasn1.error.PyAsn1Error: TagSet(Tag(tagClass=128, tagFormat=0, tagId=40)) 1768 not in asn1Spec 1769 >>> 1770 >>> decoder.decode.defaultErrorState = decoder.stDumpRawValue 1771 >>> decoder.decode(substrate) 1772 (Any(b'\x9f(\x0209'), '') 1773 >>> 1774 1775It's also possible to configure a custom decoder, to handle unknown 1776tags found in substrate. This can be done by means of 1777*defaultRawDecoder* attribute holding a reference to type decoder 1778object. Refer to the source for API details. 1779