readme.md
1# asn1crypto
2
3A fast, pure Python library for parsing and serializing ASN.1 structures.
4
5 - [Features](#features)
6 - [Why Another Python ASN.1 Library?](#why-another-python-asn1-library)
7 - [Related Crypto Libraries](#related-crypto-libraries)
8 - [Current Release](#current-release)
9 - [Dependencies](#dependencies)
10 - [Installation](#installation)
11 - [License](#license)
12 - [Documentation](#documentation)
13 - [Continuous Integration](#continuous-integration)
14 - [Testing](#testing)
15 - [Development](#development)
16 - [CI Tasks](#ci-tasks)
17
18[](https://github.com/wbond/asn1crypto/actions?workflow=CI)
19[](https://travis-ci.org/wbond/asn1crypto)
20[](https://ci.appveyor.com/project/wbond/asn1crypto)
21[](https://circleci.com/gh/wbond/asn1crypto)
22[](https://pypi.org/project/asn1crypto/)
23
24## Features
25
26In addition to an ASN.1 BER/DER decoder and DER serializer, the project includes
27a bunch of ASN.1 structures for use with various common cryptography standards:
28
29| Standard | Module | Source |
30| ---------------------- | ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
31| X.509 | [`asn1crypto.x509`](asn1crypto/x509.py) | [RFC 5280](https://tools.ietf.org/html/rfc5280) |
32| CRL | [`asn1crypto.crl`](asn1crypto/crl.py) | [RFC 5280](https://tools.ietf.org/html/rfc5280) |
33| CSR | [`asn1crypto.csr`](asn1crypto/csr.py) | [RFC 2986](https://tools.ietf.org/html/rfc2986), [RFC 2985](https://tools.ietf.org/html/rfc2985) |
34| OCSP | [`asn1crypto.ocsp`](asn1crypto/ocsp.py) | [RFC 6960](https://tools.ietf.org/html/rfc6960) |
35| PKCS#12 | [`asn1crypto.pkcs12`](asn1crypto/pkcs12.py) | [RFC 7292](https://tools.ietf.org/html/rfc7292) |
36| PKCS#8 | [`asn1crypto.keys`](asn1crypto/keys.py) | [RFC 5208](https://tools.ietf.org/html/rfc5208) |
37| PKCS#1 v2.1 (RSA keys) | [`asn1crypto.keys`](asn1crypto/keys.py) | [RFC 3447](https://tools.ietf.org/html/rfc3447) |
38| DSA keys | [`asn1crypto.keys`](asn1crypto/keys.py) | [RFC 3279](https://tools.ietf.org/html/rfc3279) |
39| Elliptic curve keys | [`asn1crypto.keys`](asn1crypto/keys.py) | [SECG SEC1 V2](http://www.secg.org/sec1-v2.pdf) |
40| PKCS#3 v1.4 | [`asn1crypto.algos`](asn1crypto/algos.py) | [PKCS#3 v1.4](ftp://ftp.rsasecurity.com/pub/pkcs/ascii/pkcs-3.asc) |
41| PKCS#5 v2.1 | [`asn1crypto.algos`](asn1crypto/algos.py) | [PKCS#5 v2.1](http://www.emc.com/collateral/white-papers/h11302-pkcs5v2-1-password-based-cryptography-standard-wp.pdf) |
42| CMS (and PKCS#7) | [`asn1crypto.cms`](asn1crypto/cms.py) | [RFC 5652](https://tools.ietf.org/html/rfc5652), [RFC 2315](https://tools.ietf.org/html/rfc2315) |
43| TSP | [`asn1crypto.tsp`](asn1crypto/tsp.py) | [RFC 3161](https://tools.ietf.org/html/rfc3161) |
44| PDF signatures | [`asn1crypto.pdf`](asn1crypto/pdf.py) | [PDF 1.7](http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/pdf/pdfs/PDF32000_2008.pdf) |
45
46## Why Another Python ASN.1 Library?
47
48Python has long had the [pyasn1](https://pypi.org/project/pyasn1/) and
49[pyasn1_modules](https://pypi.org/project/pyasn1-modules/) available for
50parsing and serializing ASN.1 structures. While the project does include a
51comprehensive set of tools for parsing and serializing, the performance of the
52library can be very poor, especially when dealing with bit fields and parsing
53large structures such as CRLs.
54
55After spending extensive time using *pyasn1*, the following issues were
56identified:
57
58 1. Poor performance
59 2. Verbose, non-pythonic API
60 3. Out-dated and incomplete definitions in *pyasn1-modules*
61 4. No simple way to map data to native Python data structures
62 5. No mechanism for overridden universal ASN.1 types
63
64The *pyasn1* API is largely method driven, and uses extensive configuration
65objects and lowerCamelCase names. There were no consistent options for
66converting types of native Python data structures. Since the project supports
67out-dated versions of Python, many newer language features are unavailable
68for use.
69
70Time was spent trying to profile issues with the performance, however the
71architecture made it hard to pin down the primary source of the poor
72performance. Attempts were made to improve performance by utilizing unreleased
73patches and delaying parsing using the `Any` type. Even with such changes, the
74performance was still unacceptably slow.
75
76Finally, a number of structures in the cryptographic space use universal data
77types such as `BitString` and `OctetString`, but interpret the data as other
78types. For instance, signatures are really byte strings, but are encoded as
79`BitString`. Elliptic curve keys use both `BitString` and `OctetString` to
80represent integers. Parsing these structures as the base universal types and
81then re-interpreting them wastes computation.
82
83*asn1crypto* uses the following techniques to improve performance, especially
84when extracting one or two fields from large, complex structures:
85
86 - Delayed parsing of byte string values
87 - Persistence of original ASN.1 encoded data until a value is changed
88 - Lazy loading of child fields
89 - Utilization of high-level Python stdlib modules
90
91While there is no extensive performance test suite, the
92`CRLTests.test_parse_crl` test case was used to parse a 21MB CRL file on a
93late 2013 rMBP. *asn1crypto* parsed the certificate serial numbers in just
94under 8 seconds. With *pyasn1*, using definitions from *pyasn1-modules*, the
95same parsing took over 4,100 seconds.
96
97For smaller structures the performance difference can range from a few times
98faster to an order of magnitude or more.
99
100## Related Crypto Libraries
101
102*asn1crypto* is part of the modularcrypto family of Python packages:
103
104 - [asn1crypto](https://github.com/wbond/asn1crypto)
105 - [oscrypto](https://github.com/wbond/oscrypto)
106 - [csrbuilder](https://github.com/wbond/csrbuilder)
107 - [certbuilder](https://github.com/wbond/certbuilder)
108 - [crlbuilder](https://github.com/wbond/crlbuilder)
109 - [ocspbuilder](https://github.com/wbond/ocspbuilder)
110 - [certvalidator](https://github.com/wbond/certvalidator)
111
112## Current Release
113
1141.4.0 - [changelog](changelog.md)
115
116## Dependencies
117
118Python 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8 or pypy. *No third-party
119packages required.*
120
121## Installation
122
123```bash
124pip install asn1crypto
125```
126
127## License
128
129*asn1crypto* is licensed under the terms of the MIT license. See the
130[LICENSE](LICENSE) file for the exact license text.
131
132## Documentation
133
134The documentation for *asn1crypto* is composed of tutorials on basic usage and
135links to the source for the various pre-defined type classes.
136
137### Tutorials
138
139 - [Universal Types with BER/DER Decoder and DER Encoder](docs/universal_types.md)
140 - [PEM Encoder and Decoder](docs/pem.md)
141
142### Reference
143
144 - [Universal types](asn1crypto/core.py), `asn1crypto.core`
145 - [Digest, HMAC, signed digest and encryption algorithms](asn1crypto/algos.py), `asn1crypto.algos`
146 - [Private and public keys](asn1crypto/keys.py), `asn1crypto.keys`
147 - [X509 certificates](asn1crypto/x509.py), `asn1crypto.x509`
148 - [Certificate revocation lists (CRLs)](asn1crypto/crl.py), `asn1crypto.crl`
149 - [Online certificate status protocol (OCSP)](asn1crypto/ocsp.py), `asn1crypto.ocsp`
150 - [Certificate signing requests (CSRs)](asn1crypto/csr.py), `asn1crypto.csr`
151 - [Private key/certificate containers (PKCS#12)](asn1crypto/pkcs12.py), `asn1crypto.pkcs12`
152 - [Cryptographic message syntax (CMS, PKCS#7)](asn1crypto/cms.py), `asn1crypto.cms`
153 - [Time stamp protocol (TSP)](asn1crypto/tsp.py), `asn1crypto.tsp`
154 - [PDF signatures](asn1crypto/pdf.py), `asn1crypto.pdf`
155
156## Continuous Integration
157
158Various combinations of platforms and versions of Python are tested via:
159
160 - [AppVeyor](https://ci.appveyor.com/project/wbond/asn1crypto/history)
161 - [CircleCI](https://circleci.com/gh/wbond/asn1crypto)
162 - [GitHub Actions](https://github.com/wbond/asn1crypto/actions)
163 - [Travis CI](https://travis-ci.org/wbond/asn1crypto/builds)
164
165## Testing
166
167Tests are written using `unittest` and require no third-party packages.
168
169Depending on what type of source is available for the package, the following
170commands can be used to run the test suite.
171
172### Git Repository
173
174When working within a Git working copy, or an archive of the Git repository,
175the full test suite is run via:
176
177```bash
178python run.py tests
179```
180
181To run only some tests, pass a regular expression as a parameter to `tests`.
182
183```bash
184python run.py tests ocsp
185```
186
187### PyPi Source Distribution
188
189When working within an extracted source distribution (aka `.tar.gz`) from
190PyPi, the full test suite is run via:
191
192```bash
193python setup.py test
194```
195
196### Package
197
198When the package has been installed via pip (or another method), the package
199`asn1crypto_tests` may be installed and invoked to run the full test suite:
200
201```bash
202pip install asn1crypto_tests
203python -m asn1crypto_tests
204```
205
206## Development
207
208To install the package used for linting, execute:
209
210```bash
211pip install --user -r requires/lint
212```
213
214The following command will run the linter:
215
216```bash
217python run.py lint
218```
219
220Support for code coverage can be installed via:
221
222```bash
223pip install --user -r requires/coverage
224```
225
226Coverage is measured by running:
227
228```bash
229python run.py coverage
230```
231
232To change the version number of the package, run:
233
234```bash
235python run.py version {pep440_version}
236```
237
238To install the necessary packages for releasing a new version on PyPI, run:
239
240```bash
241pip install --user -r requires/release
242```
243
244Releases are created by:
245
246 - Making a git tag in [PEP 440](https://www.python.org/dev/peps/pep-0440/#examples-of-compliant-version-schemes) format
247 - Running the command:
248
249 ```bash
250 python run.py release
251 ```
252
253Existing releases can be found at https://pypi.org/project/asn1crypto/.
254
255## CI Tasks
256
257A task named `deps` exists to download and stage all necessary testing
258dependencies. On posix platforms, `curl` is used for downloads and on Windows
259PowerShell with `Net.WebClient` is used. This configuration sidesteps issues
260related to getting pip to work properly and messing with `site-packages` for
261the version of Python being used.
262
263The `ci` task runs `lint` (if flake8 is available for the version of Python) and
264`coverage` (or `tests` if coverage is not available for the version of Python).
265If the current directory is a clean git working copy, the coverage data is
266submitted to codecov.io.
267
268```bash
269python run.py deps
270python run.py ci
271```
272
){kind=link}
){kind=link}
){kind=link}
){kind=link}