1######################################################################
2 Data::Throttler 0.08
3######################################################################
4
5NAME
6 Data::Throttler - Limit data throughput
7
8SYNOPSIS
9 use Data::Throttler;
10
11 ### Simple: Limit throughput to 100 per hour
12
13 my $throttler = Data::Throttler->new(
14 max_items => 100,
15 interval => 3600,
16 );
17
18 if($throttler->try_push()) {
19 print "Item can be pushed\n";
20 } else {
21 print "Item needs to wait\n";
22 }
23
24 ### Advanced: Use a persistent data store and throttle by key:
25
26 my $throttler = Data::Throttler->new(
27 max_items => 100,
28 interval => 3600,
29 backend => "YAML",
30 backend_options => {
31 db_file => "/tmp/mythrottle.yml",
32 },
33 );
34
35 if($throttler->try_push(key => "somekey")) {
36 print "Item can be pushed\n";
37 }
38
39DESCRIPTION
40 "Data::Throttler" helps solving throttling tasks like "allow a single IP
41 only to send 100 emails per hour". It provides an optionally persistent
42 data store to keep track of what happened before and offers a simple
43 yes/no interface to an application, which can then focus on performing
44 the actual task (like sending email) or suppressing/postponing it.
45
46 When defining a throttler, you can tell it to keep its internal data
47 structures in memory:
48
49 # in-memory throttler
50 my $throttler = Data::Throttler->new(
51 max_items => 100,
52 interval => 3600,
53 );
54
55 However, if the data structures need to be maintained across different
56 invocations of a script or several instances of scripts using the
57 throttler, using a persistent database is required:
58
59 # persistent throttler
60 my $throttler = Data::Throttler->new(
61 max_items => 100,
62 interval => 3600,
63 backend => "YAML",
64 backend_options => {
65 db_file => "/tmp/mythrottle.yml",
66 },
67 );
68
69 The call above will reuse an existing backend store, given that the
70 "max_items" and "interval" settings are compatible and leave the stored
71 counter bucket chain contained therein intact. To specify that the
72 backend store should be rebuilt and all counters be reset, use the
73 "reset => 1" option of the Data::Throttler object constructor.
74
75 In the simplest case, "Data::Throttler" just keeps track of single
76 events. It allows a certain number of events per time frame to succeed
77 and it recommends to block the rest:
78
79 if($throttler->try_push()) {
80 print "Item can be pushed\n";
81 } else {
82 print "Item needs to wait\n";
83 }
84
85 the "force => 1" option of the try_push() method will cause the counter
86 to be incremented regardless of threshold for use in scenarios where
87 max_items is a threshold rather than throttle condition:
88
89 if($throttler->try_push('force' => 1)) {
90 print "Item can be pushed\n";
91 } else {
92 print "Counter incremented, Item needs to wait\n";
93 }
94
95 When throttling different categories of items, like attempts to send
96 emails by IP address of the sender, a key can be used:
97
98 if($throttler->try_push( key => "192.168.0.1" )) {
99 print "Item can be pushed\n";
100 } else {
101 print "Item needs to wait\n";
102 }
103
104 In this case, each key will be tracked separately, even if the quota for
105 one key is maxed out, other keys will still succeed until their quota is
106 reached.
107
108 HOW IT WORKS
109 To keep track of what happened within the specified time frame,
110 "Data::Throttler" maintains a round-robin data store, either in memory
111 or on disk. It splits up the controlled time interval into buckets and
112 maintains counters in each bucket:
113
114 1 hour ago Now
115 +-----------------------------+
116 | 3 | 7 | 0 | 0 | 4 | 1 |
117 +-----------------------------+
118 4:10 4:20 4:30 4:40 4:50 5:00
119
120 To decide whether to allow a new event to happen or not,
121 "Data::Throttler" adds up all counters (3+7+4+1 = 15) and then compares
122 the result to the defined threshold. If the event is allowed, the
123 corresponding counter is increased (last column):
124
125 1 hour ago Now
126 +-----------------------------+
127 | 3 | 7 | 0 | 0 | 4 | 2 |
128 +-----------------------------+
129 4:10 4:20 4:30 4:40 4:50 5:00
130
131 While time progresses, old buckets are expired and then reused for new
132 data. 10 minutes later, the bucket layout would look like this:
133
134 1 hour ago Now
135 +-----------------------------+
136 | 7 | 0 | 0 | 4 | 2 | 0 |
137 +-----------------------------+
138 4:20 4:30 4:40 4:50 5:00 5:10
139
140 LOCKING
141 When used with a persistent data store, "Data::Throttler" protects
142 competing applications from clobbering the database by using the locking
143 mechanism offered with "DBM::Deep". Both the "try_push()" and the
144 "buckets_dump" function already perform locking behind the scenes.
145
146 If you see a need to lock the data store yourself, i.e. when trying to
147 push counters for several keys simultaneously, use
148
149 $throttler->lock();
150
151 and
152
153 $throttler->unlock();
154
155 to protect the data store against competing applications.
156
157 RESETTING
158 Sometimes, you may need to reset a specific counter, e.g. if an IP
159 address has been unintentionally throttled:
160
161 my $count = $throttler->reset_key(key => "192.168.0.1");
162
163 The "reset_key" method returns the total number of attempts so far.
164
165 ADVANCED USAGE
166 By default, "Data::Throttler" will decide on the number of buckets by
167 dividing the time interval by 10. It won't handle sub-seconds, though,
168 so if the time interval is less then 10 seconds, the number of buckets
169 will be equal to the number of seconds in the time interval.
170
171 If the default bucket allocation is unsatisfactory, you can specify it
172 yourself:
173
174 my $throttler = Data::Throttler->new(
175 max_items => 100,
176 interval => 3600,
177 nof_buckets => 42,
178 );
179
180 Mainly for debugging and testing purposes, you can specify a different
181 time than *now* when trying to push an item:
182
183 if($throttler->try_push(
184 key => "somekey",
185 time => time() - 600 )) {
186 print "Item can be pushed in the past\n";
187 }
188
189 Also for debugging and testing purposes, you can obtain the current
190 value of an item:
191
192 my $val = $throttler->current_value(key => "somekey");
193
194 Speaking of debugging, there's a utility method "buckets_dump" which
195 returns a string containing a formatted representation of what's in each
196 bucket. It requires the CPAN module Text::ASCIITable, so make sure to
197 have it installed before calling the method. The module is not a
198 requirement for Data::Throttler on purpose.
199
200 So the code
201
202 use Data::Throttler;
203
204 my $throttler = Data::Throttler->new(
205 interval => 3600,
206 max_items => 10,
207 );
208
209 $throttler->try_push(key => "foobar");
210 $throttler->try_push(key => "foobar");
211 $throttler->try_push(key => "barfoo");
212 print $throttler->buckets_dump();
213
214 will print out something like
215
216 .----+-----+---------------------+--------+-------.
217 | # | idx | Time: 14:43:00 | Key | Count |
218 |=---+-----+---------------------+--------+------=|
219 | 1 | 0 | 13:49:00 - 13:54:59 | | |
220 | 2 | 1 | 13:55:00 - 14:00:59 | | |
221 | 3 | 2 | 14:01:00 - 14:06:59 | | |
222 | 4 | 3 | 14:07:00 - 14:12:59 | | |
223 | 5 | 4 | 14:13:00 - 14:18:59 | | |
224 | 6 | 5 | 14:19:00 - 14:24:59 | | |
225 | 7 | 6 | 14:25:00 - 14:30:59 | | |
226 | 8 | 7 | 14:31:00 - 14:36:59 | | |
227 | 9 | 8 | 14:37:00 - 14:42:59 | | |
228 | 10 | 9 | 14:43:00 - 14:48:59 | barfoo | 1 |
229 | | | | foobar | 2 |
230 '----+-----+---------------------+--------+-------'
231
232 and allow for further investigation.
233
234LICENSE
235 Copyright 2007 by Mike Schilli, all rights reserved. This program is
236 free software, you can redistribute it and/or modify it under the same
237 terms as Perl itself.
238
239AUTHOR
240 2007, Mike Schilli <cpan@perlmeister.com>
241
242