1# Funnel
2
3[![Build Status](https://travis-ci.org/AF83/funnel.png?branch=master)](https://travis-ci.org/AF83/funnel)
4
5Funnel is for building Streaming APIs build upon ElasticSearch’s
6[percolation](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html).
7
8Funnel supports ElasticSearch >= 1.1.
9
10Funnel allow to register users / devices, associates some queries to user.
11
12The common usecase is to store a query from a user and notify this user when a
13new document matching this query is available.
14
15Check out this [example](https://github.com/AF83/funnel_http) for a complete http api using Funnel dsl.
16
17## Installing things
18
19You can use Funnel in your projects with the following steps:
20
211. Adding Funnel to your `mix.exs` dependencies:
22
23  ```elixir
24  def deps do
25    [
26      {:funnel, "~> 0.1"}
27    ]
28  end
29  ```
30
312. List the `:funnel` as your application dependencies:
32
33  ```elixir
34  def application do
35    [applications: [:funnel]]
36  end
37  ```
38
393. Add this sample configuration to `config/config.exs`.
40
41  ```elixir
42  use Mix.Config
43
44  config :funnel,
45    es_host: "http://localhost:9200",
46    percolator_pool_size: 100,
47    percolator_pool_max_overflow: 1000
48  ```
49
50## Testing things
51
52``` shell
53mix test
54```
55
56## Doing things
57
58### Transport
59
60Funnel has this notion of `Funnel.Transport`. Anything implementing the
61`Funnel.Transport` protocol can be a transport.
62
63For the [Plug](https://github.com/elixir-lang/plug), the protocol would looks
64like:
65
66```elixir
67defmodule EventStreamMessage do
68  @moduledoc ""
69  This module serialize a given id and body to a ServerSent Events message.
70  ""
71  def to_message(id, data) do
72    "id:#{id}\ndata: #{data}\n\n"
73  end
74end
75
76defimpl Funnel.Transport, for: Elixir.Plug.Conn do
77  import Plug.Conn
78
79  def write(conn, %{:id => id, :item => item}) do
80    chunk conn, EventStreamMessage.to_message(id, item)
81  end
82end
83```
84
85### Register
86
87A user, or a device, can register to funnel by using the `/register` endpoint.
88This will return a token. This token must be used in all communications with the
89funnel's API.
90
91The first argument is a `transport`, the second one is optionnal, and allow to
92write onto the transport the message seen since the `last_id given`.
93
94```elixir
95{:ok, token} = Funnel.register(transport)
96{:ok, "1a9dc09879374878bd7aab27c7be6bc7"}
97
98{:ok, token} = Funnel.register(self, "422f779c759244d4aad45ac94c83b7da")
99{:ok, "80cfd5a3e1324db8b076defec2ddc1b2"}
100```
101
102### Creating Indexes
103
104This example will create an empty index:
105
106```elixir
107{:ok, status_code, body} = Funnel.Index.create
108{:ok, 200, %{"body" => %{"acknowledged" => true}, "index_id" => "e4680d88db914ed4854acb8a1a8f317d"}}
109```
110
111This example will create an index with specific settings:
112
113``` elixir
114settings = '{"settings" : {"number_of_shards" : 1},"mappings" : {"type1" :{"_source" : { "enabled" : false },"properties" : {"field1" : { "type" :"string", "index" : "not_analyzed" }}}}}' |> IO.iodata_to_binary
115{:ok, _status_code, body} = Funnel.Index.create(settings)
116{:ok, 200, %{"body" => %{"acknowledged" => true},"index_id" => "3994bf6c03df412e8b1b05d4aca7a83c"}}
117```
118
119
120### Deleting an index
121
122``` elixir
123{:ok, status_code, body} = Funnel.Index.destroy("3994bf6c03df412e8b1b05d4aca7a83c")
124{:ok, 200, %{"acknowledged" => true}}
125```
126
127### Creating a query
128
129``` elixir
130query = '{"query" : {"match" : {"message" : "funnel"}}}' |> IO.iodata_to_binary
131{:ok, status_code, body} = Funnel.Query.create(index_id, token, query)
132{:ok, 201, %{"body" => %{"_id" => "287eae87d5774f2a9d02f5a5dd66856d-0398f4e1a6a34ea4b7ede0c1b7f40f38", "_index" => "3994bf6c03df412e8b1b05d4aca7a83c_dev", "_type" => ".percolator", "_version" => 1, "created" => true}, "index_id" => "3994bf6c03df412e8b1b05d4aca7a83c", "query_id" => "0398f4e1a6a34ea4b7ede0c1b7f40f38"}}
133```
134
135### Updating a query
136
137``` elixir
138{:ok, status_code, body} = Funnel.Query.update(index_id, token, query_id, query)
139{:ok, 200, %{"body" => %{"_id" => "287eae87d5774f2a9d02f5a5dd66856d-0398f4e1a6a34ea4b7ede0c1b7f40f38", "_index" => "3994bf6c03df412e8b1b05d4aca7a83c_dev", "_type" => ".percolator", "_version" => 2, "created" => false}, "index_id" => "3994bf6c03df412e8b1b05d4aca7a83c", "query_id" => "0398f4e1a6a34ea4b7ede0c1b7f40f38"}}
140```
141
142### Deleting a query
143
144``` elixir
145{:ok, status_code, body} = Funnel.Query.destroy(index_id, token, query_id)
146{:ok, 200, %{"_id" => "287eae87d5774f2a9d02f5a5dd66856d-0398f4e1a6a34ea4b7ede0c1b7f40f38", "_index" => "3994bf6c03df412e8b1b05d4aca7a83c_dev", "_type" => ".percolator", "_version" => 3, "found" => true}}
147```
148
149### Finding queries
150
151``` elixir
152{:ok, status_code, body} = Funnel.Query.find(token)
153{:ok, 200, [%{"query_id" => "4f122313862e494b8810f073c27cf43d", "index_id" => "b79d2e9ff8c949e08ba98c4d8c216547", "score" => 1.0}]}
154
155{:ok, status_code, body} = Funnel.Query.find(token, %{index_id: index_id})
156{:ok, 200, [%{"query_id" => "4f122313862e494b8810f073c27cf43d", "index_id" => "b79d2e9ff8c949e08ba98c4d8c216547", "score" => 1.0}]}
157```
158
159### Submitting a document to the percolator
160
161``` elixir
162message = '{"message" : "this new elasticsearch percolator feature is nice, borat style"}' |> IO.iodata_to_binary
163Funnel.percolate(index_id, message)
164{:ok}
165```
166