1.\" Copyright (c) 2004,2005 The DragonFly Project. All rights reserved. 2.\" 3.\" This code is derived from software contributed to The DragonFly Project 4.\" by Matthew Dillon <dillon@backplane.com> 5.\" 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in 15.\" the documentation and/or other materials provided with the 16.\" distribution. 17.\" 3. Neither the name of The DragonFly Project nor the names of its 18.\" contributors may be used to endorse or promote products derived 19.\" from this software without specific, prior written permission. 20.\" 21.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 22.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 23.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 24.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 25.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 26.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 27.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 28.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 29.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 30.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 31.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 32.\" SUCH DAMAGE. 33.\" 34.\" $DragonFly: src/sbin/jscan/jscan.8,v 1.17 2007/09/14 23:47:53 swildner Exp $ 35.\" 36.Dd February 8, 2009 37.Dt JSCAN 8 38.Os 39.Sh NAME 40.Nm jscan 41.Nd journal file processing program 42.Sh SYNOPSIS 43.Nm 44.Op Fl 2dfuvF 45.Op Fl c Ar count[k,m,g,t] 46.Op Fl D Ar directory 47.Op Fl m Ar mirror_transid_file/none 48.Op Fl o/O Ar output_transid_file/none 49.Op Fl s Ar size[k,m,g,t] 50.Op Fl w/W Ar journal_prefix 51.Op Ar journal_prefix/file 52.Sh DESCRIPTION 53The 54.Nm 55utility scans journal file or input stream for the purposes of debugging 56dumps, restoration, undo, mirroring, and other journaling features. 57.Bl -tag -width indent 58.It Fl 2 59Implement the full-duplex acknowledgement protocol on the input descriptor. 60Note that shell pipes are full-duplex and can be used with this option. 61.It Fl c Ar count 62Specify the number of transaction records which should be scanned, then exit. 63This option is typically used along with 64.Fl m 65to limit the amount of work that 66.Nm 67does, giving you the ability to incrementally run a mirror forwards or 68backwards. It is not usually used when piping in a live journal, but it 69can be. 70.It Fl d 71Display the contents of the journaling file or stream in a human readable 72format on stderr. Note that stdout is used only for 73.Fl o . 74.It Fl f 75.Nm 76will sleep for 5 seconds and loop when it hits EOF on file or prefix 77set input rather than exit. This option is typically used when running 78on an input file or prefix set which is live (being written to by 79another 80.Nm 81instance). 82.It Fl D Ar directory 83Specify the base directory for the mirroring option. 84.It Fl m Ar mirror_transid_file/none 85Generate a mirror in the directory specified by 86.Fl D 87or, if not specified, the current directory. 88The 89.Ar mirror_transid_file 90will be used to track the transaction id representing the current 91synchronization point for the mirror. The keyword 92.Ar none 93may be specified if no tracking file is desired. However, if no tracking 94file is specified it will not be possible to roll the mirror forwards or 95backwards or restart the journaling stream being used to generate the mirror. 96.Pp 97It is important to note that journaling streams can contain meta-transactions 98representing huge, multi-gigabyte operations. If the journaling data is 99not being recorded to regular files via 100.Fl w/W 101it is possible that 102.Nm 103could run itself out of memory trying to record the meta-transactions. 104In addition, the mirror would not be restartable. If the journaling data 105is being recorded via 106.Fl w/W 107and a mirroring transaction id file is being kept, the mirror can be 108restarted. 109.Pp 110While it is possible to run a journaling stream directly into a mirror, 111it is more typical to file the jornaling stream with 112.Fl w 113and catch the mirror up as a batch job with the journaling file set prefix 114specified as the input every so often. This way the system operator can 115use other 116.Nm 117commands to, for example, run a mirror backwards and forwards in time. 118.It Fl o/O Ar output_transid_file/none 119Generate a journaling stream on stdout using the specified file to track 120the transaction id to help with restarts. 121The 122.Fl o 123option indicates a half-duplex output stream while the 124.Fl O 125option indicates a full-duplex (ACK protocol) output stream. 126.Pp 127This option is not really designed to output to regular files because it 128does NOT necessarily weed out duplicate records. When both the input 129stream and output stream are full-duplex and 130.Fl w/W 131is not specified, 132.Nm 133acts as a stateless transceiver and the input stream is not acked until 134an ack is received from the output stream. 135.Pp 136This option is most typically used in conjunction with 137.Fl w/W . 138In this case the ACK protocol is handled independently for the input side 139and the output side uses the journaling data recorded by 140.Fl w/W 141as a buffer. 142.Pp 143In half-duplex output mode the output transaction id file is updated 144after a raw transaction record has been successfully written to stdout. 145In full-duplex output mode the file is only updated with ACK data returned 146on the stdout descriptor. 147.Pp 148As with the 149.Fl m 150option, you can combine 151.Fl o 152in a journaling pipe with other options, but if you are trying to use it 153as a buffer it may be better to have it separately pull its data off of 154a journaling file set generated via 155.Fl w . 156.It Fl s Ar size 157Change the size limit for rotating files created via 158.Fl w . 159The default is 100M. Values are in bytes or may be suffixed with k, 160m, or g. 161If a raw transaction causes the file's size limit to be exceeded, a new file 162will be created. If a raw transaction is, in-whole, larger the the file's 163size limit, the raw transaction will still be fully written to the file before 164a new file is created. Raw transactions are typically limited to the size 165of the source system's memory FIFO. This option is typically used to size 166journaling files to fit onto the appropriate backup media or to provide 167bite-sized chunks for other programs to ingest. 168.Pp 169When restarting a journal, a new sequence number will always be chosen for 170the resumption of data recording. No existing file will be appended to when 171.Nm 172is reinvoked. 173.It Fl u 174Will cause the journal to be scanned backwards (requires seekable media). 175Transactions will be dumped in reverse order. If mirroring, the UNDO 176data will be executed. If not specified, 1 hour's worth of data will be 177undone. Can only be used with a journaling file or journaling prefix 178as the input. 179.It Fl v 180Increase verbosity on stderr. This option is primarily used for debugging. 181.It Fl w Ar prefix 182The received journaling stream is recorded in journaling files named 183.Ar <prefix>.<seq> 184and the current transaction id is tracked in a file named 185.Ar <prefix>.transid . 186A journaling file is closed out and a new file with the next sequence 187number is created once the file surpasses 100MB. 188.Pp 189This option is robust across restarts. The current transaction id 190will be read and the input stream will be skipped until it is reached. 191If the input is a journaling file or prefix set, 192.Nm 193will be able to quickly seek to the restart point. 194.Pp 195NOTE: If 196you are generating a mirror with the same command via 197.Fl m , 198and the journaling data input is a stream rather than a file or prefix 199set, you must use 200.Fl w/W 201if you want the mirror to be restartable. This is because while we can 202pick up the transaction id where we left off, that raw transaction id may 203have cut a larger meta-transaction in half and the mirroring code will 204not be able to access the whole of the transaction unless it has a file 205or prefix set to work with. 206.It Fl W Ar prefix 207Similar to 208.Fl w 209except that the journaling files created are strictly temporary and will 210be deleted once they exceed the size limit AND the related meta-transactions 211have been completed. 212.Pp 213If combined with 214.Fl m , 215the meta-transactions are considered to be completed only when the mirror 216finishes executing them. It is possible for several sequence number files 217to build if a particularly large meta-transaction is coming down the pipe. 218.Pp 219If combined with 220.Fl o/O , 221the meta-transactions are considered to be completed when the data has 222been successfully written out to the pipe in half duplex mode, or when 223the ACK has been received in full-duplex mode. 224.Pp 225If both 226.Fl m 227and 228.Fl w/W 229is used, the journaling data files are only deleted when both actions 230no longer need the data. 231.It Fl F 232Forces 233.Nm 234to 235.Fn fsync 236after updating a journaling file prior to acknowledging the 237data or updating a transaction-id-tracking file. If specified twice, 238.Nm 239will also 240.Fn fsync 241after updating the transaction-id-tracking file. 242.It Ar journal_prefix/file 243Specify the input to 244.Nm . 245This can be a journaling file set prefix 246or it can be a plain file. If no input file is specified, stdin is 247assumed. Note that when generating a mirror from a stdin stream, the 248mirror will not be restartable unless 249.Fl w/W 250is also used. 251.El 252.Sh OPERATIONAL NOTES 253It is often important to be able to quickly stage journaled data through 254a dedicated backup machine on a LAN. There are several places where data 255can be buffered and staged out. 256.Pp 257The machine generating the journal typically buffers several megabytes of 258journal data in the kernel. This local machine can pipe that data to 259.Nm 260or some other locally run program to add another buffering stage, or you 261can directly attach a TCP connection to the kernel's journaling output. 262.Pp 263The LAN backup box typically buffers gigabytes worth of data by running 264multiple jscans. The 265.Nm 266on the receiving end of the TCP or pipe (for 267example, via ssh) typically records the data via the 268.Fl w 269option, and then runs other 270.Nm 271programs from scripts or cron to take that data and copy it to your 272off-site backup machine. Other 273.Nm 274programs may use the same data 275set to generate mirrors or other backup streams. 276.Pp 277It should be noted that if 278.Fl w/W 279is specified, both mirroring mode and output mode will internally 280fork the program once the appropriate synchronization point has been reached, 281effectively decoupling their operation, and read all of their data via 282the journaling files written out by the master program. In particular, 283blockages in the mirroring and output code will not affect our ability 284to buffer the journaling input data via 285.Fl w/W . 286If 287.Fl w/W 288is not specified then neither the mirroring or output modes will fork. Under 289these conditions, if the input is a stream rather than a file 290.Nm 291will be forced to buffer meta-transactions (for mirroring) entirely in 292memory, which could present a serious problem since a single meta-transaction 293can exceed a gigabyte (e.g. if someone were to do a single 294.Fn write 295system call writing a gigabyte all in one go). 296.Sh SEE ALSO 297.Xr mountctl 8 298.Sh CAVEATS 299This utility is currently under construction and not all features have been 300implemented yet. 301In fact, most have not. 302.Sh HISTORY 303The 304.Nm 305utility first appeared in 306.Dx 1.3 . 307