README.md
1## void-zones-tools for FreeBSD
2### Prepare a list of void zones that can be readily feed into *Unbound*.
3
4### What is a void zone?
5
6In terms of *Unbound*, a void zone is a static empty local-zone, i.e. one without any local-data directives, for example:
7
8 local-zone: "void.example.com" static
9
10With this in place, *Unbound* would answer any DNS requests for `void.example.com` with `NXDOMAIN`.
11
12
13### For what are void zones useful?
14
15Void zones are the most straightforward way of blocking ad, tracking and other malware domains.
16While the method is similar to the *Hosts* file approach, the void zone method got 3 advantages:
17
181. Requests for any subdomain of the void zone result in `NXDOMAIN`.
19 In *Hosts* files every subdomain needs its own entry.
20
212. In *Hosts* files, each entry does consist of an IP address and a fully qualified domain,
22 so DNS requests for a given domain would return the listed IP address. For blocking
23 purposes either `127.0.0.1` or `0.0.0.0` is used, which may result in undesired side effects:
24
25 - in the case of `127.0.0.1`, developers won't be able to run a test web-server on localhost,
26 and any software is supposed to try to establish a connection to localhost, and this
27 usually ends up in a timeout error.
28 - in the case of `0.0.0.0` ill-behaved software still tries to establish a connection, which
29 cannot work, but resources are utilized until timeout.
30
31 The void zone response is `NXDOMAIN`, and no software would try to establish a connection
32 to a non-existing address.
33
343. The void zone method can be deployed on the gateway, and any number of clients behind the
35 gateway do benefit automatically from ad, tracking and malware domain blocking. While *Hosts*
36 files need to be deployed on any single machine. Even if this is not that difficult on
37 desktops and notebooks, this may easily become a PITA for mobile clients. You would need
38 `root` login on your Androids, iPhones and/or iPads.
39
40
41### How does this compare to Browser Plugins?
42
431. Browser plugins are destined to one piece of software and not to the whole machine.
44 Void zones are active for the whole machine or in the case of a gateway, for any
45 number of clients, and even for those (Android) which don't allow ad-blocking plugins.
46
472. Browser plugins are active filters, that means, beside the advertised behaviour, they
48 are able to do something in the background. This is a matter of trust, which may
49 sometimes miserably trapped -- see the WoT incident. Void zones are passive. The actual
50 filtering is done by the DNS resolver, here *Unbound*, which is much less likely of doing
51 undesired things behind your back.
52
53
54### How do I deploy the void zone method on my FreeBSD machine?
55
56On the FreeBSD machine, either clone the present `void-zones-tools` project using *Git* or check it out with *Subversion*:
57
58 # git clone https://github.com/cyclaero/void-zones-tools.git
59
60or
61
62 # svn checkout https://github.com/cyclaero/void-zones-tools.git/trunk/ void-zones-tools
63
64Enter the directory of the working copy of the `void-zones-tools` and build & install the tools:
65
66 # cd void-zones-tools
67 # make install clean CDEFS="-march=native"
68
69The tools consist of the *Hosts* file converter & consolidator `hosts2zones` and the shell script
70`void-zones-update.sh` which is used to download suitable *Hosts* files from 6 pre-defined providers:
71
72* [PGL - Ad blocking with ad server hostnames and IP addresses](http://pgl.yoyo.org/adservers/)
73* [SomeOneWhoCares - How to make the internet not suck (as much)](http://someonewhocares.org/hosts/zero/)
74* [MVPS - A detailed guide for using the MVPS HOSTS file](http://winhelp2002.mvps.org/)
75* [MDL - Malware Domain List](http://www.malwaredomainlist.com/)
76* [AdAway - Hosts](https://github.com/AdAway/AdAway/)
77* [DNS-BH – Malware Domain Blocklist](http://www.malwaredomains.com/)
78* [FadeMind - UncheckyAds](https://github.com/FadeMind/hosts.extras/)
79* [WindowsSpyBlocker - Spy Hosts](https://github.com/crazy-max/WindowsSpyBlocker/)
80
81
82The tools are placed by the above command sequence into `/usr/local/bin/`.
83
84On the first run of `void-zones-update.sh`, a directory is created at `/usr/local/etc/void-zones/`,
85which serves as the storage location for the downloaded *Hosts* files and/or *Domain* listings. In
86addition a template for a custom white/black list `my_void_hosts.txt` is placed into that directory,
87and this may be used for whitelisting some zones that are inadvertantly part of the downloaded *Hosts* files,
88or for blacklisting addtional zones, which are missing from the downloads. Now execute said shell script:
89
90 # void-zones-update.sh
91 # nano /usr/local/etc/void-zones/my_void_hosts.txt
92
93 # white list
94 1.1.1.1 my.white.dom
95
96 # black list
97 0.0.0.0 my.black.dom
98
99For whitelisting use the IP address `1.1.1.1`, and for blacklisting `0.0.0.0` shall be used.
100
101The downloaded *Hosts* files are placed into `/usr/local/etc/void-zones/` as well:
102
103 # ls -l /usr/local/etc/void-zones
104
105 total 1876
106 -rw-r--r-- 1 root wheel 13722 Jan 31 2017 away_void_hosts.txt
107 -rw-r--r-- 1 root wheel 640858 Aug 17 19:16 jdom_void_list.txt
108 -rw-r--r-- 1 root wheel 36982 Jun 29 19:52 mdl_void_hosts.txt
109 -rw-r--r-- 1 root wheel 497673 Aug 7 11:07 mvps_void_hosts.txt
110 -rw-r--r-- 1 root wheel 60257 Aug 21 05:43 pgl_void_hosts.txt
111 -rw-r--r-- 1 root wheel 376421 Aug 20 14:40 sowc_void_hosts.txt
112 -rw-r--r-- 1 root wheel 618 Aug 22 09:29 ucky_void_host.txt
113 -rw-r--r-- 1 root wheel 9977 Aug 22 09:29 w10telm_void_hosts.txt
114 -rw-r--r-- 1 root wheel 886 Aug 22 09:29 w7telm_void_hosts.txt
115 -rw-r--r-- 1 root wheel 1142 Aug 22 09:29 w81telm_void_hosts.txt
116
117And finally the `void-zones-update.sh` compiles (converts & consolidates) all *Hosts* files
118and *Domain* listings into one single `local-void.zones` include file, and moves this into
119`/var/unbound/` for direct usage with *Unbound*:
120
121 # head /var/unbound/local-void.zones
122
123 local-zone: "clk.cloudyisland.com" static
124 local-zone: "click.silvercash.com" static
125 local-zone: "oascentral.pressdemocrat.com" static
126 local-zone: "s29.cnzz.com" static
127 local-zone: "www.spywarespy.com" static
128 local-zone: "republika.onet.pl" static
129 local-zone: "preview.msn.com" static
130 local-zone: "pos.baidu.com" static
131 ...
132
133For using the void zones method, of course *Unbound* must be up and running already on the given FreeBSD machine.
134Then edit the configuration file `/var/unbound/unbound.conf` in order to activate ad, tracking, malware and telemetry domain
135filtering by *Unbound*:
136
137 # nano /var/unbound/unbound.conf
138
139**Before** any forwarder directives, e.g. `forward-zone:` or `include: /var/unbound/forward.conf` add the following line:
140
141 include: /var/unbound/local-void.zones
142
143Then restart *Unbound*:
144
145 # service local_unbound restart
146
147For future updates execute the following command sequence which may be placed into a cron job:
148
149 # /usr/local/bin/void-zones-update.sh && /usr/sbin/service local_unbound restart
150
151In order to facilitate inclusion of listings which are not part of the automated updating, 3 additional input files are
152passed by `void-zones-update.sh` to the conversion tool `hosts2zones`:
153
154 x_void_list.txt
155 y_void_list.txt
156 z_void_list.txt
157
158This mechanism can be used to include for example the [Disconnect.me](https://github.com/chrisaljoudi/uBlock/issues/1406)
159listings to the `hosts2zones` processing by executing the following command before updating the other zones:
160
161 # fetch -o - \
162 https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt \
163 https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt \
164 https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt \
165 https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt \
166 > /usr/local/etc/void-zones/x_void_list.txt
167
168Said command would place the respective lists joined together into `/usr/local/etc/void-zones/x_void_list.txt`, and on the
169next run of `void-zones-update.sh` that one would be converted & consolidated & included into the `local-void.zones` for
170filtering by *Unbound*. In the case these additional files are missing, the tool simply ignores these parameters.
171