1.\" Copyright (c) 2004,2005 The DragonFly Project. All rights reserved. 2.\" 3.\" This code is derived from software contributed to The DragonFly Project 4.\" by Matthew Dillon <dillon@backplane.com> 5.\" 6.\" 7.\" Redistribution and use in source and binary forms, with or without 8.\" modification, are permitted provided that the following conditions 9.\" are met: 10.\" 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in 15.\" the documentation and/or other materials provided with the 16.\" distribution. 17.\" 3. Neither the name of The DragonFly Project nor the names of its 18.\" contributors may be used to endorse or promote products derived 19.\" from this software without specific, prior written permission. 20.\" 21.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 22.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 23.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 24.\" FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 25.\" COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 26.\" INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, 27.\" BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 28.\" LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED 29.\" AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 30.\" OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 31.\" OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 32.\" SUCH DAMAGE. 33.\" 34.\" $DragonFly: src/sbin/jscan/jscan.8,v 1.9 2006/03/08 09:53:30 swildner Exp $ 35.\" 36.Dd March 6, 2005 37.Dt JSCAN 8 38.Os 39.Sh NAME 40.Nm jscan 41.Nd journal file processing program 42.Sh SYNOPSIS 43.Nm 44.Fl 2dfuvF 45.Op Fl c Ar count[k,m,g,t] 46.Op Fl D Ar directory 47.Op Fl m Ar mirror_transid_file/none 48.Op Fl o/O Ar output_transid_file/none 49.Op Fl s Ar size[k,m,g,t] 50.Op Fl w/W Ar journal_prefix 51.Op Ar journal_prefix/file 52.Pp 53.Sh DESCRIPTION 54The 55.Nm 56utility scans journal file or input stream for the purposes of debugging 57dumps, restoration, undo, mirroring, and other journaling features. 58.Bl -tag -width indent 59.It Fl 2 60Implement the full-duplex acknowledgement protocol on the input descriptor. 61Note that shell pipes are full-duplex and can be used with this option. 62.It Fl c Ar count 63Specify the number of transaction records which should be scanned, then exit. 64This option is typically used along with 65.Fl m 66to limit the amount of work that 67.Nm 68does, giving you the ability to incrementally run a mirror forwards or 69backwards. It is not usually used when piping in a live journal, but it 70can be. 71.It Fl d 72Display the contents of the journaling file or stream in a human readable 73format on stderr. Note that stdout is used only for 74.Fl o . 75.It Fl f 76.Nm 77will sleep for 5 seconds and loop when it hits EOF on file or prefix 78set input rather then exit. This option is typically used when running 79on an input file or prefix set which is live (being written to by 80another 81.Nm 82instance). 83.It Fl D Ar directory 84Specify the base directory for the mirroring option. 85.It Fl m Ar mirror_transid_file/none 86Generate a mirror in the directory specified by 87.Fl D 88or, if not specified, the current directory. 89The 90.Ar mirror_transid_file 91will be used to track the transaction id representing the current 92syncnronization point for the mirror. The keyword 93.Ar none 94may be specified if no tracking file is desired. However, if no tracking 95file is specified it will not be possible to roll the mirror forwards or 96backwards or restart the journaling stream being used to generate the mirror. 97.Pp 98It is important to note that journaling streams can contain meta-transactions 99representing huge, multi-gigabyte operations. If the journaling data is 100not being recorded to regular files via 101.Fl w/W 102it is possible that 103.Nm 104could run itself out of memory trying to record the meta-transactions. 105In addition, the mirror would not be restartable. If the journaling data 106is being recorded via 107.Fl w/W 108and a mirroring transaction id file is being kept, the mirror can be 109restarted. 110.Pp 111While it is possible to run a journaling stream directly into a mirror, 112it is more typical to file the jornaling stream with 113.Fl w 114and catch the mirror up as a batch job with the journaling file set prefix 115specified as the input every so often. This way the system operator can 116use other 117.Nm 118commands to, for example, run a mirror backwards and forwards in time. 119.It Fl o/O Ar output_transid_file/none 120Generate a journaling stream on stdout using the specified file to track 121the transaction id to help with restarts. 122The 123.Fl o 124option indicates a half-duplex output stream while the 125.Fl O 126option indicates a full-duplex (ACK protocol) output stream. 127.Pp 128This option is not really designed to output to regular files because it 129does NOT necessarily weed out duplicate records. When both the input 130stream and output stream are full-duplex and 131.Fl w/W 132is not specified, 133.Nm 134acts as a stateless transceiver and the input stream is not acked until 135an ack is received from the output stream. 136.Pp 137This option is most typically used in conjuction with 138.Fl w/W . 139In this case the ACK protocol is handled independantly for the input side 140and the output side uses the journaling data recorded by 141.Fl w/W 142as a buffer. 143.Pp 144In half-duplex output mode the output transaction id file is updated 145after a raw transaction record has been successfully written to stdout. 146In full-duplex output mode the file is only updated with ACK data returned 147on the stdout descriptor. 148.Pp 149As with the 150.Fl m 151option, you can combine 152.Fl o 153in a journaling pipe with other options, but if you are trying to use it 154as a buffer it may be better to have it separately pull its data off of 155a journaling file set generated via 156.Fl w . 157.It Fl s Ar size 158Change the size limit for rotating files created via 159.Fl w . 160The default is 100M. Values are in bytes or may be suffixed with k, 161m, or g. 162If a raw transaction causes the file's size limit to be exceeded, a new file 163will be created. If a raw transaction is, in-whole, larger the the file's 164size limit, the raw transaction will still be fully written to the file before 165a new file is created. Raw transactions are typically limited to the size 166of the source system's memory FIFO. This option is typically used to size 167journaling files to fit onto the appropriate backup media or to provide 168bite-sized chunks for other programs to injest. 169.Pp 170When restarting a journal, a new sequence number will always be chosen for 171the resumption of data recording. No existing file will be appended to when 172.Nm 173is reinvoked. 174.It Fl u 175Will cause the journal to be scanned backwards (requires seekable media). 176Transactions will be dumped in reverse order. If mirroring, the UNDO 177data will be executed. If not specified, 1 hour's worth of data will be 178undone. Can only be used with a journaling file or journaling prefix 179as the input. 180.It Fl v 181Increase verbosity on stderr. This option is primarily used for debugging. 182.It Fl w Ar prefix 183The received journaling stream is recorded in journaling files named 184.Ar <prefix>.<seq> 185and the current transaction id is tracked in a file named 186.Ar <prefix>.transid . 187A journaling file is closed out and a new file with the next sequence 188number is created once the file surpasses 100MB. 189.Pp 190This option is robust across restarts. The current transaction id 191will be read and the input stream will be skipped until it is reached. 192If the input is a journaling file or prefix set, 193.Nm 194will be able to quickly seek to the restart point. 195.Pp 196NOTE: If 197you are generating a mirror with the same command via 198.Fl m , 199and the journaling data input is a stream rather then a file or prefix 200set, you must use 201.Fl w/W 202if you want the mirror to be restartable. This is because while we can 203pick up the transaction id where we left off, that raw transaction id may 204have cut a larger meta-transaction in half and the mirroring code will 205not be able to access the whole of the transaction unless it has a file 206or prefix set to work with. 207.It Fl W Ar prefix 208Similar to 209.Fl w 210except that the journaling files created are strictly temporary and will 211be deleted once they exceed the size limit AND the related meta-transactions 212have been completed. 213.Pp 214If combined with 215.Fl m , 216the meta-transactions are considered to be completed only when the mirror 217finishes executing them. It is possible for several sequence number files 218to build if a particularly large meta-transaction is coming down the pipe. 219.Pp 220If combined with 221.Fl o/O , 222the meta-transactions are considered to be completed when the data has 223been successfully written out to the pipe in half duplex mode, or when 224the ACK has been received in full-duplex mode. 225.Pp 226If both 227.Fl m 228and 229.Fl w/W 230is used, the journaling data files are only deleted when both actions 231no longer need the data. 232.It Fl F 233Forces 234.Nm 235to fsync() after updating a journaling file prior to acknowledging the 236data or updating a transaction-id-tracking file. If specified twice, 237.Nm 238will also fsync() after updating the transaction-id-tracking file. 239.It Ar journal_prefix/file 240Specify the input to jscan. This can be a journaling file set prefix 241or it can be a plain file. If no input file is specified, stdin is 242assumed. Note that when generating a mirror from a stdin stream, the 243mirror will not be restartable unless 244.Fl w/W 245is also used. 246.Pp 247.El 248.Pp 249.Sh OPERATIONAL NOTES 250It is often important to be able to quickly stage journaled data through 251a dedicated backup machine on a LAN. There are several places where data 252can be buffered and staged out. 253.Pp 254The machine generating the journal typically buffers several megabytes of 255journal data in the kernel. This local machine can pipe that data to 256.Nm 257or some other locally run program to add another buffering stage, or you 258can directly attach a TCP connection to the kernel's journaling output. 259.Pp 260The LAN backup box typically buffers gigabytes worth of data by running 261multiple jscan's. The jscan on the receiving end of the TCP or pipe (for 262example, via ssh) typically records the data via the 263.Fl w 264option, and then runs other 265.Nm 266programs from scripts or cron to take that data and copy it to your 267off-site backup machine. Other jscan programs may use the same data 268set to generate mirrors or other backup streams. 269.Pp 270It should be noted that if 271.Fl w/W 272is specified, both mirroring mode and output mode will internally 273fork the program once the appropriate synchronization point has been reached, 274effectively decoupling their operation, and read all of their data via 275the journaling files written out by the master program. In particular, 276blockages in the mirroring and output code will not effect our ability 277to buffer the journaling input data via 278.Fl w/W . 279If 280.Fl w/W 281is not specified then neither the mirroring or output modes will fork. Under 282these conditions, if the input is a stream rather then a file 283.Nm 284will be forced to buffer meta-transactions (for mirroring) entirely in 285memory, which could present a serious problem since a single meta-transaction 286can exceed a gigabyte (e.g. if someone were to do a single write() system 287call writing a gigabyte all in one go). 288.Pp 289.Sh SEE ALSO 290.Xr mountctl 8 291.Sh CAVEATS 292This utility is currently under construction and not all features have been 293implemented yet. 294In fact, most have not. 295.Sh HISTORY 296The 297.Nm 298utility first appeared in 299.Dx 1.3 . 300