1[![Downloads](https://pepy.tech/badge/benford-py)](https://pepy.tech/project/benford-py)
2
3# Benford for Python
4
5--------------------------------------------------------------------------------
6
7**Citing**
8
9
10If you find *Benford_py* useful in your research, please consider adding the following citation:
11
12```bibtex
13@misc{benford_py,
14      author = {Marcel, Milcent},
15      title = {{Benford_py: a Python Implementation of Benford's Law Tests}},
16      year = {2017},
17      publisher = {GitHub},
18      journal = {GitHub repository},
19      howpublished = {\url{https://github.com/milcent/benford_py}},
20}
21```
22
23--------------------------------------------------------------------------------
24
25`current version = 0.5.0`
26
27### See [release notes](https://github.com/milcent/benford_py/releases/) for features in this and in older versions
28
29### Python versions >= 3.6
30
31### Installation
32
33Benford_py is a package in PyPi, so you can install with pip:
34
35`pip install benford_py`
36
37or
38
39`pip install benford-py`
40
41Or you can cd into the site-packages subfolder of your python distribution (or environment) and git clone from there:
42
43`git clone https://github.com/milcent/benford_py`
44
45For a quick start, please go to the [Demo notebook](https://github.com/milcent/benford_py/blob/master/Demo.ipynb), in which I show examples on how to run the tests with the SPY (S&P 500 ETF) daily returns.
46
47For more fine-grained details of the functions and classes, see the [docs](https://benford-py.readthedocs.io/en/latest/index.html).
48
49### Background
50
51The first digit of a number is [its leftmost digit](https://github.com/milcent/benford_py/blob/master/img/First_Digits.png)
52
53Since the first digit of any number can range from "1" to "9"
54(not considering "0"), it would be intuitively expected that the
55proportion of each occurrence in a set of numerical records would
56be uniformly distributed at 1/9, i.e., approximately 0.1111,
57or 11.11%.
58
59[Benford's Law](https://en.wikipedia.org/wiki/Benford%27s_law),
60also known as the Law of First Digits or the Phenomenon of
61Significant Digits, is the finding that the first digits of the
62numbers found in series of records of the most varied sources do
63not display a uniform distribution, but rather are arranged in such
64a way that the digit "1" is the most frequent, followed by "2",
65"3", and so in a successive and decremental way down to "9",
66which presents the lowest frequency as the first digit.
67
68The expected distributions of the First Digits in a
69Benford-compliant data set are the ones shown [here](https://github.com/milcent/benford_py/blob/master/img/First.png)
70
71The first record on the subject dates from 1881, in the work of
72[Simon Newcomb](https://github.com/milcent/benford_py/blob/master/img/Simon_Newcomb_APS.jpg), an American-Canadian astronomer and mathematician,
73who noted that in the logarithmic tables the first pages, which
74contained logarithms beginning with the numerals "1" and "2",
75were more worn out, that is, more consulted.
76
77In that same article, Newcomb proposed the [formula](https://github.com/milcent/benford_py/blob/master/img/formula.png) for the probability of a certain digit "d"
78being the first digit of a number, given by the following equation.
79
80In 1938, the American physicist [Frank Benford](https://github.com/milcent/benford_py/blob/master/img/2429_Benford-Frank.jpg) revisited the
81phenomenon, which he called the "Law of Anomalous Numbers," in
82a survey with more than 20,000 observations of empirical data
83compiled from various sources, ranging from areas of rivers to
84molecular weights of chemical compounds, including cost data,
85address numbers, population sizes and physical constants. All
86of them, to a greater or lesser extent, followed such
87distribution.
88
89The extent of Benford's work seems to have been one good reason
90for the phenomenon to be popularized with his name, though
91described by Newcomb 57 years earlier.
92
93Derivations of the original formula were also applied in the
94expected findings of the proportions of digits in other
95positions in the number, as in the case of the second digit
96(BENFORD, 1938), as well as combinations, such as the first
97two digits of a number (NIGRINI, 2012, p.5).
98
99Only in 1995, however, was the phenomenon proven by Hill.
100His proof was based on the fact that numbers in data series
101following the Benford Law are, in effect, "second generation"
102distributions, ie combinations of other distributions.
103The union of randomly drawn samples from various distributions
104forms a distribution that respects Benford's Law (HILL, 1995).
105
106When grouped in ascending order, data that obey Benford's Law
107must approximate a geometric sequence (NIGRINI, 2012, page 21).
108From this it follows that the logarithms of this ordered series
109must form a straight line. In addition, the mantissas (decimal
110parts) of the logarithms of these numbers must be uniformly
111distributed in the interval [0,1] (NIGRINI, 2012, p.10).
112
113In general, a series of numerical records follows Benford's Law
114when (NIGRINI, 2012, p.21):
115* it represents magnitudes of events or events, such as populations
116of cities, flows of water in rivers or sizes of celestial bodies;
117* it does not have pre-established minimum or maximum limits;
118* it is not made up of numbers used as identifiers, such as
119identity or social security numbers, bank accounts, telephone numbers; and
120* its mean is less than the median, and the data is not
121concentrated around the mean.
122
123It follows from this expected distribution that, if the set of
124numbers in a series of records that usually respects the Law
125shows a deviation in the proportions found, there may be
126distortions, whether intentional or not.
127
128Benford's Law has been used in [several fields](http://www.benfordonline.net/).
129Afer asserting that the usual data type is Benford-compliant,
130one can study samples from the same data type tin search of
131inconsistencies, errors or even [fraud](https://www.amazon.com.br/Benfords-Law-Applications-Accounting-Detection/dp/1118152859).
132
133This open source module is an attempt to facilitate the
134performance of Benford's Law-related tests by people using
135Python, whether interactively or in an automated, scripting way.
136
137It uses the versatility of numpy and pandas, along with
138matplotlib for vizualization, to deliver results like [this one](https://github.com/milcent/benford_py/blob/master/img/SPY-f2d-conf_level-95.png) and much more.
139
140
141It has been a long time since I last tested it in Python 2. The death clock has stopped ticking, so officially it is for Python 3 now. It should work on Linux, Windows and Mac, but please file a bug report if you run into some trouble.
142
143Also, if you have some nice data set that we can run these tests on, let'us try it.
144
145Thanks!
146
147Milcent
148