1GRPC Connection Backoff Protocol
2================================
3
4When we do a connection to a backend which fails, it is typically desirable to
5not retry immediately (to avoid flooding the network or the server with
6requests) and instead do some form of exponential backoff.
7
8We have several parameters:
9 1. INITIAL_BACKOFF (how long to wait after the first failure before retrying)
10 1. MULTIPLIER (factor with which to multiply backoff after a failed retry)
11 1. JITTER (by how much to randomize backoffs).
12 1. MAX_BACKOFF (upper bound on backoff)
13 1. MIN_CONNECT_TIMEOUT (minimum time we're willing to give a connection to
14    complete)
15
16## Proposed Backoff Algorithm
17
18Exponentially back off the start time of connection attempts up to a limit of
19MAX_BACKOFF, with jitter.
20
21```
22ConnectWithBackoff()
23  current_backoff = INITIAL_BACKOFF
24  current_deadline = now() + INITIAL_BACKOFF
25  while (TryConnect(Max(current_deadline, now() + MIN_CONNECT_TIMEOUT))
26         != SUCCESS)
27    SleepUntil(current_deadline)
28    current_backoff = Min(current_backoff * MULTIPLIER, MAX_BACKOFF)
29    current_deadline = now() + current_backoff +
30      UniformRandom(-JITTER * current_backoff, JITTER * current_backoff)
31
32```
33
34With specific parameters of
35MIN_CONNECT_TIMEOUT = 20 seconds
36INITIAL_BACKOFF = 1 second
37MULTIPLIER = 1.6
38MAX_BACKOFF = 120 seconds
39JITTER = 0.2
40
41Implementations with pressing concerns (such as minimizing the number of wakeups
42on a mobile phone) may wish to use a different algorithm, and in particular
43different jitter logic.
44
45Alternate implementations must ensure that connection backoffs started at the
46same time disperse, and must not attempt connections substantially more often
47than the above algorithm.
48
49## Reset Backoff
50
51The back off should be reset to INITIAL_BACKOFF at some time point, so that the
52reconnecting behavior is consistent no matter the connection is a newly started
53one or a previously disconnected one.
54
55We choose to reset the Backoff when the SETTINGS frame is received, at that time
56point, we know for sure that this connection was accepted by the server.
57