• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..03-May-2022-

bench/H08-Aug-2018-4841

config/H08-Aug-2018-3127

lib/H08-Aug-2018-671579

test/H08-Aug-2018-140122

.gitignoreH A D08-Aug-2018457 2014

LICENSE.mdH A D08-Aug-20181.1 KiB2217

README.mdH A D08-Aug-20185.7 KiB13499

mix.exsH A D08-Aug-20181.4 KiB5953

mix.lockH A D08-Aug-20181.6 KiB1211

README.md

1# libring - A fast consistent hash ring for Elixir
2
3[![Hex.pm Version](http://img.shields.io/hexpm/v/libring.svg?style=flat)](https://hex.pm/packages/libring)
4
5This library implements a stateful consistent hash ring. It's extremely fast
6(in benchmarks it's faster than all other implementations I've tested against,
7namely [voicelayer/hash-ring](https://github.com/voicelayer/hash-ring) and
8[sile/hash_ring](https://github.com/sile/hash_ring)), it has no external dependencies,
9and is written in Elixir.
10
11The algorithm is based on [libketama](https://github.com/rj/ketama). Nodes on the
12ring are broken into shards and each one is assigned an integer value in the keyspace, which
13is the set of integers from 1 to 2^32-1. The distribution of these shards is random, but
14deterministic.
15
16Keys are then mapped to a shard by converting the key to a binary, hashing it with SHA-256,
17converting the hash to an integer in the keyspace, then finding the shard which is assigned
18the next highest value, if there is no next highest value, the lowest integer is used, which
19is how the "ring" is formed.
20
21This implementation uses a general balanced tree, via Erlang's `:gb_tree` module. Each shard
22is inserted into the tree, and we use this data structure to efficiently lookup next-highest
23key and smallest key. I suspect this is why `libring` is faster than other implementations I've
24benchmarked against.
25
26## Usage
27
28Add `:libring` to your deps, and run `mix deps.get`.
29
30```elixir
31def deps do
32  [{:libring, "~> 1.0"}]
33end
34```
35
36You have two choices for managing hash rings in your application:
37
38## HashRing
39
40This API works with the raw ring datastructure. It is the fastest implementation,
41and is best suited for when you have a single process which will need to access the
42ring, and which can hold the ring in it's internal state.
43
44```elixir
45ring = HashRing.new()
46       |> HashRing.add_node("a")
47       |> HashRing.add_node("b")
48
49"a" = HashRing.key_to_node(ring, {:myworker, 123})
50```
51
52You can also specify the weight of each node, and add nodes in bulk:
53
54```elixir
55ring = HashRing.new()
56       |> HashRing.add_nodes(["a", {"b", 64}])
57       |> HashRing.add_node("c", 200)
58"c" = HashRing.key_to_node(ring, {:myworker, 123})
59```
60
61**NOTE**: Node names do not have to be strings, they can be atoms, tuples, etc.
62
63## HashRing.Managed
64
65This API works with rings which are held in the internal state of a GenServer process.
66It supports the same API as `HashRing`. Because of this, there is a performance overhead
67due to the messaging, and the GenServer can be a potential bottleneck. If this is the case
68you are better off exploring ways to use the raw `HashRing` API. However this API is best suited
69for situations where you have multiple processes accessing the ring, or need to maintain multiple
70rings.
71
72**NOTE**: You must have the `:libring` application started to use this API.
73
74```elixir
75{:ok, pid} = HashRing.Managed.new(:myring)
76:ok = HashRing.Managed.add_node(:myring, "a")
77:ok = HashRing.Managed.add_node(:myring, "b", 64)
78:ok = HashRing.Managed.add_node(:myring, "c", 200)
79"c" = HashRing.Managed.key_to_node(:myring, {:myworker, 123})
80```
81
82You can configure managed rings in `config.exs`, and they will be created and initialized
83when the `:libring` application starts. Configured rings take two shapes, static and dynamic
84rings. Static rings are simply those where the nodes are provided up front, although you can
85always add/remove nodes manually at runtime; dynamic rings have Erlang node monitoring enabled,
86and add or remove nodes on the ring based on cluster membership.
87
88You can whitelist/blacklist nodes when using dynamic rings, so that only those nodes which you
89actually want to distribute work to are used in calculations. This configuration is shown below as well.
90
91If you provide a whitelist, the blacklist will have no effect, and only nodes matching the whitelist
92will be added. If you do not provide a whitelist, the blacklist will be used to filter nodes. If you
93do not provide either, a default blacklist containing the `~r/^remsh.*$/` pattern from the example below,
94which is a good default to prevent remote shell sessions (at least those done via releases) from causing
95the ring to change.
96
97The whitelist and blacklist only have an effect when `monitor_nodes: true`.
98
99## Configuration
100
101Below is an example configuration:
102
103```elixir
104config :libring,
105  rings: [
106    # A ring which automatically changes based on Erlang cluster membership,
107    # but does not allow nodes named "a" or "remsh*" to be added to the ring
108    ring_a: [monitor_nodes: true,
109             node_blacklist: ["a", ~r/^remsh.*$/]],
110    # A ring which is composed of three nodes, of which "c" has a non-default weight of 200
111    # The default weight is 128
112    ring_b: [nodes: ["a", "b", {"c", 200}]]
113  ]
114```
115
116## Contributing
117
118To run the test suite you will need to run `mix eqc.install --mini` once you've cloned the repo and fetched dependencies.
119
120If you have changes in mind that are significant or potentially time consuming, please open a RFC-style PR first, where we
121can discuss your plans first. I don't want you to spend all your time crafting a PR that I ultimately reject because I don't
122think it's a good fit or is too large for me to review. Not that I plan to reject PRs in general, but I have to be careful to
123balance features with maintenance burden, or I will quickly be unable to manage the project.
124
125Please ensure that you adhere to a commit style where logically related changes are in a single commit, or broken up in a way that
126eases review if necessary. Keep commit subject lines informative, but short, and provide additional detail in the extended message text
127if needed. If you can, mention relevant issue numbers in either the subject or the extended message.
128
129## License
130
131MIT
132
133Please see the `LICENSE` file for more info.
134