1FreeBSD users will need to change the Perl shebangs at the top of the executables from #!/usr/bin/perl
2to #!/usr/local/bin/perl in most cases.
3
4Sorry folks, but if I set this with #!/usr/bin/env perl as suggested, then nothing works properly
5from a typical cron environment on EITHER operating system, Linux or BSD. I'm mostly using Linux
6systems, so I get to set the shebang for my use and give you folks a FREEBSD readme rather than
7the other way around. =)
8
9If you don't want to have to change the shebangs, your other option is to drop a symlink on your system:
10
11root@bsd:~# ln -s /usr/local/bin/perl /usr/bin/perl
12
13After putting this symlink in place, ANY perl script shebanged for Linux will work on your system too.
14
15Syncoid assumes a bourne style shell on remote hosts. Using (t)csh (the default for root under FreeBSD)
16has some known issues:
17
18* If mbuffer is present, syncoid will fail with an "Ambiguous output redirect." error. So if you:
19 root@bsd:~# ln -s /usr/local/bin/mbuffer /usr/bin/mbuffer
20 make sure the remote user is using an sh compatible shell.
21
22To change to a compatible shell, use the chsh command:
23
24root@bsd:~# chsh -s /bin/sh
25
README.md
1<p align="center"><img src="http://www.openoid.net/wp-content/themes/openoid/images/sanoid_logo.png" alt="sanoid logo" title="sanoid logo"></p>
2
3<img src="http://openoid.net/gplv3-127x51.png" width=127 height=51 align="right">Sanoid is a policy-driven snapshot management tool for ZFS filesystems. When combined with the Linux KVM hypervisor, you can use it to make your systems <a href="http://openoid.net/transcend" target="_blank">functionally immortal</a>.
4
5<p align="center"><a href="https://youtu.be/ZgowLNBsu00" target="_blank"><img src="http://www.openoid.net/sanoid_video_launcher.png" alt="sanoid rollback demo" title="sanoid rollback demo"></a><br clear="all"><sup>(Real time demo: rolling back a full-scale cryptomalware infection in seconds!)</sup></p>
6
7More prosaically, you can use Sanoid to create, automatically thin, and monitor snapshots and pool health from a single eminently human-readable TOML config file at /etc/sanoid/sanoid.conf. (Sanoid also requires a "defaults" file located at /etc/sanoid/sanoid.defaults.conf, which is not user-editable.) A typical Sanoid system would have a single cron job:
8```
9* * * * * TZ=UTC /usr/local/bin/sanoid --cron
10```
11
12`Note`: Using UTC as timezone is recommend to prevent problems with daylight saving times
13
14And its /etc/sanoid/sanoid.conf might look something like this:
15
16```
17[data/home]
18 use_template = production
19[data/images]
20 use_template = production
21 recursive = yes
22 process_children_only = yes
23[data/images/win7]
24 hourly = 4
25
26#############################
27# templates below this line #
28#############################
29
30[template_production]
31 frequently = 0
32 hourly = 36
33 daily = 30
34 monthly = 3
35 yearly = 0
36 autosnap = yes
37 autoprune = yes
38```
39
40Which would be enough to tell sanoid to take and keep 36 hourly snapshots, 30 dailies, 3 monthlies, and no yearlies for all datasets under data/images (but not data/images itself, since process_children_only is set). Except in the case of data/images/win7, which follows the same template (since it's a child of data/images) but only keeps 4 hourlies for whatever reason.
41
42##### Sanoid Command Line Options
43
44+ --cron
45
46 This will process your sanoid.conf file, create snapshots, then purge expired ones.
47
48+ --configdir
49
50 Specify a location for the config file named sanoid.conf. Defaults to /etc/sanoid
51
52+ --take-snapshots
53
54 This will process your sanoid.conf file, create snapshots, but it will NOT purge expired ones. (Note that snapshots taken are atomic in an individual dataset context, <i>not</i> a global context - snapshots of pool/dataset1 and pool/dataset2 will each be internally consistent and atomic, but one may be a few filesystem transactions "newer" than the other.)
55
56+ --prune-snapshots
57
58 This will process your sanoid.conf file, it will NOT create snapshots, but it will purge expired ones.
59
60+ --force-prune
61
62 Purges expired snapshots even if a send/recv is in progress
63
64+ --monitor-snapshots
65
66 This option is designed to be run by a Nagios monitoring system. It reports on the health of your snapshots.
67
68+ --monitor-health
69
70 This option is designed to be run by a Nagios monitoring system. It reports on the health of the zpool your filesystems are on. It only monitors filesystems that are configured in the sanoid.conf file.
71
72+ --monitor-capacity
73
74 This option is designed to be run by a Nagios monitoring system. It reports on the capacity of the zpool your filesystems are on. It only monitors pools that are configured in the sanoid.conf file.
75
76+ --force-update
77
78 This clears out sanoid's zfs snapshot listing cache. This is normally not needed.
79
80+ --version
81
82 This prints the version number, and exits.
83
84+ --quiet
85
86 Supress non-error output.
87
88+ --verbose
89
90 This prints additional information during the sanoid run.
91
92+ --debug
93
94 This prints out quite alot of additional information during a sanoid run, and is normally not needed.
95
96+ --readonly
97
98 Skip creation/deletion of snapshots (Simulate).
99
100+ --help
101
102 Show help message.
103
104----------
105
106# Syncoid
107
108Sanoid also includes a replication tool, syncoid, which facilitates the asynchronous incremental replication of ZFS filesystems. A typical syncoid command might look like this:
109
110```
111syncoid data/images/vm backup/images/vm
112```
113
114Which would replicate the specified ZFS filesystem (aka dataset) from the data pool to the backup pool on the local system, or
115
116```
117syncoid data/images/vm root@remotehost:backup/images/vm
118```
119
120Which would push-replicate the specified ZFS filesystem from the local host to remotehost over an SSH tunnel, or
121
122```
123syncoid root@remotehost:data/images/vm backup/images/vm
124```
125
126Which would pull-replicate the filesystem from the remote host to the local system over an SSH tunnel.
127
128Syncoid supports recursive replication (replication of a dataset and all its child datasets) and uses mbuffer buffering, lzop compression, and pv progress bars if the utilities are available on the systems used.
129If ZFS supports resumeable send/receive streams on both the source and target those will be enabled as default.
130
131As of 1.4.18, syncoid also automatically supports and enables resume of interrupted replication when both source and target support this feature.
132
133##### Syncoid Dataset Properties
134
135+ syncoid:sync
136
137 Available values:
138
139 + `true` (default if unset)
140
141 This dataset will be synchronised to all hosts.
142
143 + `false`
144
145 This dataset will not be synchronised to any hosts - it will be skipped. This can be useful for preventing certain datasets from being transferred when recursively handling a tree.
146
147 + `host1,host2,...`
148
149 A comma separated list of hosts. This dataset will only be synchronised by hosts listed in the property.
150
151 _Note_: this check is performed by the host running `syncoid`, thus the local hostname must be present for inclusion during a push operation // the remote hostname must be present for a pull.
152
153 _Note_: this will also prevent syncoid from handling the dataset if given explicitly on the command line.
154
155 _Note_: syncing a child of a no-sync dataset will currently result in a critical error.
156
157 _Note_: empty properties will be handled as if they were unset.
158
159##### Syncoid Command Line Options
160
161+ [source]
162
163 This is the source dataset. It can be either local or remote.
164
165+ [destination]
166
167 This is the destination dataset. It can be either local or remote.
168
169+ --identifier=
170
171 Adds the given identifier to the snapshot name after "syncoid_" prefix and before the hostname. This enables the use case of reliable replication to multiple targets from the same host. The following chars are allowed: a-z, A-Z, 0-9, _, -, : and . .
172
173+ -r --recursive
174
175 This will also transfer child datasets.
176
177+ --skip-parent
178
179 This will skip the syncing of the parent dataset. Does nothing without '--recursive' option.
180
181+ --compress <compression type>
182
183 Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used.
184
185+ --source-bwlimit <limit t|g|m|k>
186
187 This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired.
188
189+ --target-bw-limit <limit t|g|m|k>
190
191 This is the bandwidth limit in bytes (kbytes, mbytesm etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.
192
193+ --no-command-checks
194
195 Does not check the existence of commands before attempting the transfer, providing administrators a way to run the tool with minimal overhead and maximum speed, at risk of potentially failed replication, or other possible edge cases. It assumes all programs are available, and should not be used in most situations. This is an not an officially supported run mode.
196
197+ --no-stream
198
199 This argument tells syncoid to use -i incrementals, not -I. This updates the target with the newest snapshot from the source, without replicating the intermediate snapshots in between. (If used for an initial synchronization, will do a full replication from newest snapshot and exit immediately, rather than starting with the oldest and then doing an immediate -i to the newest.)
200
201+ --no-sync-snap
202
203 This argument tells syncoid to restrict itself to existing snapshots, instead of creating a semi-ephemeral syncoid snapshot at execution time. Especially useful in multi-target (A->B, A->C) replication schemes, where you might otherwise accumulate a large number of foreign syncoid snapshots.
204
205+ --create-bookmark
206
207 This argument tells syncoid to create a zfs bookmark for the newest snapshot after it got replicated successfully. The bookmark name will be equal to the snapshot name. Only works in combination with the --no-sync-snap option. This can be very useful for irregular replication where the last matching snapshot on the source was already deleted but the bookmark remains so a replication is still possible.
208
209+ --no-clone-rollback
210
211 Do not rollback clones on target
212
213+ --no-rollback
214
215 Do not rollback anything (clones or snapshots) on target host
216
217+ --exclude=REGEX
218
219 The given regular expression will be matched against all datasets which would be synced by this run and excludes them. This argument can be specified multiple times.
220
221+ --no-resume
222
223 This argument tells syncoid to not use resumeable zfs send/receive streams.
224
225+ --force-delete
226
227 Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks.
228
229+ --no-clone-handling
230
231 This argument tells syncoid to not recreate clones on the targe on initial sync and doing a normal replication instead.
232
233+ --dumpsnaps
234
235 This prints a list of snapshots during the run.
236
237+ --no-privilege-elevation
238
239 Bypass the root check and assume syncoid has the necessary permissions (for use with ZFS permission delegation).
240
241+ --sshport
242
243 Allow sync to/from boxes running SSH on non-standard ports.
244
245+ --sshcipher
246
247 Instruct ssh to use a particular cipher set.
248
249+ --sshoption
250
251 Passes option to ssh. This argument can be specified multiple times.
252
253+ --sshkey
254
255 Use specified identity file as per ssh -i.
256
257+ --quiet
258
259 Supress non-error output.
260
261+ --debug
262
263 This prints out quite alot of additional information during a sanoid run, and is normally not needed.
264
265+ --help
266
267 Show help message.
268
269+ --version
270
271 Print the version and exit.
272
273+ --monitor-version
274
275 This doesn't do anything right now.
276
277Note that the sync snapshots syncoid creates are not atomic in a global context: sync snapshots of pool/dataset1 and pool/dataset2 will each be internally consistent, but one may be a few filesystem transactions "newer" than the other. (This does not affect the consistency of snapshots already taken in other ways, which syncoid replicates in the overall stream unless --no-stream is specified. So if you want to manually zfs snapshot -R pool@1 before replicating with syncoid, the global atomicity of pool/dataset1@1 and pool/dataset2@1 will still be intact.)
278
