README.md
1# normality
2
3[](https://travis-ci.org/pudo/normality)
4
5Normality is a Python micro-package that contains a small set of text
6normalization functions for easier re-use. These functions accept a
7snippet of unicode or utf-8 encoded text and remove various classes
8of characters, such as diacritics, punctuation etc. This is useful as
9a preparation to further text analysis.
10
11**WARNING**: This library works much better when used in combination
12with ``pyicu``, a Python binding for the International Components for
13Unicode C library. ICU provides much better text transliteration than
14the default ``text-unidecode``.
15
16## Example
17
18```python
19# coding: utf-8
20from normality import normalize, slugify, collapse_spaces
21
22text = normalize('Nie wieder "Grüne Süppchen" kochen!')
23assert text == 'nie wieder grune suppchen kochen'
24
25slug = slugify('My first blog post!')
26assert slug == 'my-first-blog-post'
27
28text = 'this \n\n\r\nhas\tlots of \nodd spacing.'
29assert collapse_spaces(text) == 'this has lots of odd spacing.'
30```
31
32## License
33
34``normality`` is open source, licensed under a standard MIT license
35(included in this repository as ``LICENSE``).
36
){kind=link}