1\ 2.\" This man page was generated by the Netpbm tool 'makeman' from HTML source. 3.\" Do not hand-hack it! If you have bug fixes or improvements, please find 4.\" the corresponding HTML page on the Netpbm website, generate a patch 5.\" against that, and send it to the Netpbm maintainer. 6.TH "Ppmtompeg User Manual" 0 "23 July 2006" "netpbm documentation" 7 8.SH NAME 9ppmtompeg - encode an MPEG-1 bitstream 10 11.UN synopsis 12.SH SYNOPSIS 13 14\fBppmtompeg\fP 15[\fIoptions\fP] 16\fIparameter-file\fP 17 18.UN description 19.SH DESCRIPTION 20.PP 21This program is part of 22.BR "Netpbm" (1)\c 23\&. 24.PP 25\fBppmtompeg\fP produces an MPEG-1 video stream. MPEG-1 is the 26first great video compression method, and is what is used in Video CDs 27(VCD). \fBppmtompeg\fP originated in the year 1995. DVD uses a more 28advanced method, MPEG-2. There is an even newer method called MPEG-4 29which is also called Divx. I don't know where one finds that used. 30.PP 31There's technically a difference between a compression method for 32video and an actual file (stream) format for a movie, and I don't know 33if it can be validly said that the format of the stream 34\fBppmtompeg\fP produces is MPEG-1. 35.PP 36Mencoder from the 37.UR http://www.mplayerhq.hu 38Mplayer package 39.UE 40\& is probably superior for most video format generation 41needs, if for no other reason than that it is more popular. 42.PP 43The programming library 44.UR http://pm2v.free.fr 45\fBPM2V\fP 46.UE 47\& 48generates MPEG-2 streams. 49.PP 50Use 51.UR http://www.mplayerhq.hu 52Mplayer 53.UE 54\& (not part of Netpbm) 55to do the reverse conversion: to create a series of PNM files from an MPEG 56stream. 57.PP 58\fIparam_file\fP is a parameter file which includes a list of 59input files and other parameters. The file is described in detail 60below. 61.PP 62To understand this program, you need to understand something about 63the complex MPEG-1 format. One source of information about this 64standard format is the section Introduction to MPEG in the 65.UR http://www.faqs.org/faqs/compression-faq 66Compression FAQ 67.UE 68\&. 69 70.UN options 71.SH OPTIONS 72.PP 73The \fB-gop\fP, \fB-combine_gops\fP, \fB-frames\fP, and 74\fB-combine_frames\fP options are all mutually exclusive. 75 76 77.TP 78\fB-stat stat_file\fP 79This option causes \fBppmtompeg\fP to append the statistics that 80it write to Standard Output to the file \fIstat_file\fP as well. The 81statistics use the following abbreviations: bits per block (bpb), bits 82per frame (bpf), seconds per frame (spf), and bits per second (bps). 83.sp 84These statistics include how many I, P, and B frames there were, 85and information about compression and quality. 86 87 88.TP 89\fB-quiet\fP \fInum_seconds\fP 90 causes \fBppmtompeg\fP not to report remaining time more often 91than every \fInum_seconds\fP seconds (unless the time estimate rises, 92which will happen near the beginning of the run). A negative value 93tells \fBppmtompeg\fP not to report at all. 0 is the default 94(reports once after each frame). Note that the time remaining is an 95estimate and does not take into account time to read in frames. 96 97.TP 98\fB-realquiet\fP 99 causes \fBppmtompeg\fP to run silently, 100with the only screen output being errors. Particularly useful when 101reading input from stdin. 102 103.TP 104 105\fB-no_frame_summary\fP 106 This option prevents \fBppmtompeg\fP from printing a summary 107line for each frame 108 109.TP 110\fB-float_dct\fP 111 forces \fBppmtompeg\fP to use a more accurate, yet more 112computationally expensive version of the DCT. 113 114.TP 115\fB-gop\fP \fIgop_num\fP 116causes \fBppmtompeg\fP to encode only the numbered GOP (first GOP is 0). The 117parameter file is the same as for normal usage. The output file will be 118the normal output file with the suffix \fB.gop.\fP\fIgop_num\fP. 119\fBppmtompeg\fP does not output any sequence information. 120 121.TP 122\fB-combine_gops\fP 123 causes \fBppmtompeg\fP simply to combine some GOP files into a 124single MPEG output stream. \fBppmtompeg\fP inserts a sequence header 125and trailer. In this case, the parameter file needs only to contain 126the SIZE value, an output file, and perhaps a list of input GOP 127files (see below). 128 129If you don't supply a list of input GOP files is used, then 130\fBppmtompeg\fP assumes you're using the same parameter file you used 131when you created the input (with the \fB-gop\fP option) and 132calculates the corresponding gop filenames itself. If this is not the 133case, you can specify input GOP files in the same manner as normal 134input files -- except instead of using INPUT_DIR, INPUT, and 135END_INPUT, use GOP_INPUT_DIR, GOP_INPUT, and GOP_END_INPUT. If no 136input GOP files are specified, then the default is to use the output 137file name with suffix \fB.gop.\fP\fIgop_num\fP, with \fIgop_num\fP 138starting from 0, as the input files. 139.sp 140Thus, unless you're mixing and matching GOP files from different 141sources, you can simply use the same parameter file for creating the 142GOP files (\fB-gop\fP) and for later turning them into an MPEG stream 143(\fB-combine_gops\fP). 144 145 146.TP 147\fB-frames \fIfirst_frame\fP \fIlast_frame\fP\fP 148This option causes \fBppmtompeg\fP to encode only the frames numbered 149\fIfirst_frame\fP to \fIlast_frame\fP, inclusive. The parameter 150file is the same as for normal usage. The output will be placed in 151separate files, one per frame, with the file names being the normal 152output file name with the suffix \fB.frame.\fP\fIframe_num\fP. No 153GOP header information is output. (Thus, the parameter file need not 154include the GOP_SIZE value) 155.sp 156Use \fBppmtompeg -combine_frames\fP to combine these frames later into 157an MPEG stream. 158 159 160.TP 161\fB-combine_frames\fP 162 This option causes \fBppmtompeg\fP simply to combine some 163individual MPEG frames (such as you might have created with an earlier 164run of \fBppmtompeg -frames\fP) into a single MPEG stream. Sequence 165and GOP headers are inserted appropriately. In this case, the 166parameter file needs to contain only the SIZE value, the GOP_SIZE 167value, an output file, and perhaps a list of frame files (see below). 168.sp 169The parameter file may specify input frame files in the same manner 170as normal input files -- except instead of using INPUT_DIR, INPUT, and 171END_INPUT, use FRAME_INPUT_DIR, FRAME_INPUT, and FRAME_END_INPUT. If 172no input frame files are specified, then the default is to use the 173output file name with suffix \fB.frame.\fP\fIframe_num\fP, with 174\fIframe_num\fP starting from 0, as the input files. 175 176 177 178.TP 179\fB-nice\fP 180This option causes \fBppmtompeg\fP to run any remote processes 181"nicely," i.e. at low priority. (This is relevant only if you are 182running \fBppmtompeg\fP in parallel mode. Otherwise, there are no 183remote processes). See 'man nice.' 184 185.TP 186\fB-max_machines \fInum_machines\fP\fP 187This option causes \fBppmtompeg\fP to use no more than 188\fInum_machines\fP machines as slaves for use in parallel encoding. 189 190.TP 191\fB-snr\fP 192This option causes \fBppmtompeg\fP to include the signal-to-noise 193ratio in the reported statistics. Prints SNR (Y U V) and peak SNR (Y 194U V) for each frame. In summary, prints averages of luminance only 195(Y). SNR is defined as 10*log(variance of original/variance of 196error). Peak SNR is defined as 20*log(255/RMSE). Note that 197\fBppmtompeg\fP runs a little slower when you use this option. 198 199.TP 200\fB-mse\fP 201This option causes \fBppmtompeg\fP to report the mean squared 202error per block. It also automatically reports the quality of the 203images, so there is no need to specify \fB-snr\fP then. 204 205.TP 206\fB-bit_rate_info\fP \fIrate_file\fP 207 This option makes \fBppmtompeg\fP write bit rate information 208into the file \fIrate_file\fP. Bit rate information is bits per frame, and 209also bits per I-frame-to-I-frame. 210 211.TP 212\fB-mv_histogram\fP 213 This option causes \fBppmtompeg\fP to print a histogram of the 214motion vectors as part of statistics. There are three histograms -- 215one for P frame, one for forward B frame, and one for backward B frame 216motion vectors. 217.sp 218The output is in the form of a matrix, each entry corresponding to one 219motion vector in the search window. The center of the matrix 220represents (0,0) motion vectors. 221 222.TP 223\fB-debug_sockets\fP 224This option causes \fBppmtompeg\fP to print to Standard Output 225messages that narrate the communication between the machines when you run 226\fBppmtompeg\fP in 227.UR #parallel 228parallel mode 229.UE 230\&. 231 232.TP 233\fB-debug_machines\fP 234This option causes \fBppmtompeg\fP to print to Standard Output 235messages that narrate the progress of the conversion on the various 236machines when you run \fBppmtompeg\fP in 237.UR #parallel 238parallel mode 239.UE 240\&. 241 242 243 244.UN parmfile 245.SH PARAMETER FILE 246.PP 247The parameter file \fBmust\fP contain the following 248lines (except when using the \fB-combine_gops\fP or \fB-combine_frames\fP 249options): 250 251 252 253.TP 254\fBPATTERN\fP \fIpattern\fP 255This statement specifies the pattern (sequence) of I frames, P frames, 256and B frames. \fIpattern\fP is just a sequence of the letters I, P, and 257B with nothing between. Example: 258 259.nf 260 PATTERN IBBPBBPBBPBBPBB 261.fi 262.sp 263See 264.UR #ipb 265I Frames, P Frames, B Frames 266.UE 267\&. 268 269.TP 270\fBOUTPUT\fP \fIoutput file\fP 271This names the file where the output MPEG stream goes. 272 273.TP 274\fBINPUT_DIR\fP \fIdirectory\fP 275This statement tells where the input images (frames) come from. 276If each frame is in a separate file, \fIdirectory\fP is the directory 277where they all are. You may use \fB.\fP to refer to the current 278directory. A null \fIdirectory\fP refers to the root directory of the 279system file tree. 280.sp 281To have \fBppmtompeg\fP read all the frames serially from Standard 282Input, specify 283.nf 284 INPUT_DIR stdin 285.fi 286 287.TP 288\fBINPUT\fP 289This line must be followed by a list of the input files (in display order) 290and then the line \fBEND_INPUT\fP. 291.sp 292There are three types of lines between INPUT and END_INPUT. First, 293a line may simply be the name of an input file. Second, the line 294may be of the form \fIsingle_star_expr\fP 295\fB[\fP\fIx\fP\fB-\fP\fIy\fP\fB]\fP. 296\fIsingle_star_expr\fP can have a single \fB*\fP in it. It is 297replaced by all the numbers between x and y inclusive. So, for 298example, the line \fBtennis*.ppm [12-15]\fP refers to the files 299tennis12.ppm, tennis13.ppm, tennis14.ppm, tennis15.ppm. 300.sp 301Uniform zero-padding occurs, as well. For example, the line 302\fBfootball.*.ppm [001-130]\fP refers to the files football.001.ppm, 303football.002.ppm, ..., football.009.ppm, football.010.ppm, ..., 304football.130.ppm. 305.sp 306The third type of line is: \fIsingle_star_expr\fP 307\fB[\fP\fIx\fP\fB-\fP\fIy\fP\fB+\fP\fIs\fP\fB]\fP, where the 308line is treated exactly as above, except that we skip by \fIs\fP. Thus, the 309line \fBfootball.*.ppm [001-130+4]\fP refers to the files 310football.001.ppm, football.005.ppm, football.009.ppm, 311football.013.ppm, etc. 312.sp 313Furthermore, a line may specify a shell command to execute to 314generate lines to be interpreted as described above, as if those lines 315were in the parameter file instead. Use back ticks, like in the 316Bourne Shell, like this: 317 318.nf 319 `cat myfilelist` 320.fi 321.sp 322If input is from Standard Input (per the \fBINPUT_DIR\fP statement), 323\fBppmtompeg\fP ignores the \fBINPUT\fP/\fBEND_INPUT\fP block, but 324it still must be present. 325 326.TP 327\fBBASE_FILE_FORMAT\fP {\fBPPM\fP | \fBPNM\fP | \fBYUV\fP | 328 \fBJPEG\fP | \fBJMOVIE\fP} 329\fBppmtompeg\fP must convert all input files to one of the 330following formats as a first step of processing: PNM, YUV, JPEG(v4), 331or JMOVIE. (The conversion may be trivial if your input files are 332already in one of these formats). This line specifies which of the 333four formats. PPM is actually a subset of PNM. The separate 334specification is allowed for backward compatibility. Use PNM instead 335of PPM in new applications. 336 337.TP 338\fBINPUT_CONVERT\fP \fIconversion_command\fP 339You must specify how to convert a file to the base file format. 340If no conversion is necessary, then you would just say: 341 342.nf 343 INPUT_CONVERT * 344.fi 345.sp 346Otherwise, \fIconversion_command\fP is a shell command that causes 347an image in the format your specified with \fBBASE_FILE_FORMAT\fP to 348be written to Standard Output. \fBppmtompeg\fP executes the command 349once for each line between \fBINPUT\fP and \fBEND_INPUT\fP (which is 350normally, but not necessarily, a file name). In the conversion 351command, \fBppmtompeg\fP replaces each '*' with the contents of that 352line. 353 354 If you had a bunch of gif files, you might say: 355.nf 356 INPUT_CONVERT giftopnm * 357.fi 358 359 If you have a bunch of separate a.Y, a.U, and a.V files (where 360 the U and V have already been subsampled), then you might say: 361 362.nf 363 INPUT_CONVERT cat *.Y *.U *.V 364.fi 365.sp 366Input conversion is not allowed with input from stdin, so use 367 368.nf 369 INPUT_CONVERT * 370.fi 371 372as described above. 373 374.TP 375\fBSIZE\fP \fIwidth\fP\fBx\fP\fIheight\fP 376.sp 377\fIwidth\fP and \fIheight\fP are the width and height of each 378frame in pixels. 379.sp 380When \fBppmtompeg\fP can get this information from the input image 381files, it ignores the \fBSIZE\fP parameter and you may omit it. 382.sp 383When the image files are in YUV format, the files don't contain 384dimension information, so \fBSIZE\fP is required. 385.sp 386When \fBppmtompeg\fP is running in parallel mode, not all of the 387processes in the network have access to the image files, so 388\fBSIZE\fP is required and must give the same dimensions as the 389input image files. 390 391.TP 392\fBYUV_SIZE\fP \fIwidth\fP\fBx\fP\fIheight\fP 393This is an obsolete synonym of \fBSIZE\fP. 394 395.TP 396\fBYUV_FORMAT\fP {\fBABEKAS\fP | \fBPHILLIPS\fP | \fBUCB\fP | 397 \fBEYUV\fP | \fIpattern\fP} 398This is meaningful only when \fBBASE_FILE_FORMAT\fP specifies 399YUV format, and then it is required. It specifies the sub-format of 400the YUV class. 401 402 403.TP 404\fBGOP_SIZE\fP \fIn\fP 405\fIn\fP is the number of frames in a Group of Pictures. Except that 406because a GOP must start with an I frame, \fBppmtompeg\fP makes a GOP as 407much longer than \fIn\fP as it has to to make the next GOP start with an 408I frame. 409.sp 410Normally, it makes sense to make your GOP size a multiple of your 411pattern length (the latter is determined by the PATTERN parameter file 412statement). 413.sp 414See 415.UR #gop 416Group Of Pictures 417.UE 418\&. 419 420.TP 421\fBSLICES_PER_FRAME\fP \fIn\fP 422\fIn\fP is roughly the number of slices per frame. Note, at 423least one MPEG player may complain if slices do not start at the left 424side of an image. To ensure this does not happen, make sure the 425number of rows is divisible by SLICES_PER_FRAME. 426 427.TP 428\fBPIXEL\fP {\fBFULL\fP | \fBHALF\fP} 429use half-pixel motion vectors, or just full-pixel ones It is 430usually important that you use half-pixel motion vectors, because it 431results in both better quality and better compression. 432 433 434.TP 435\fBRANGE\fP \fIn\fP 436Use a search range of \fIn\fP pixels in each of the four directions 437from a subject pixel. (So the search window is a square \fIn\fP*2 pixels 438on a side). 439 440.TP 441\fBPSEARCH_ALG\fP {\fBEXHAUSTIVE\fP | \fBTWOLEVEL\fP | 442 \fBSUBSAMPLE\fP | \fBLOGARITHMIC\fP} 443This statement tells \fBppmtompeg\fP what kind of search 444 technique (algorithm) to use for P frames. You select the desired 445 combination of speed and compression. \fBEXHAUSTIVE\fP gives the 446 best compression, but \fBLOGARITHMIC\fP is the fastest. 447 \fBTWOLEVEL\fP is an exhaustive full-pixel search, followed by a 448 local half- pixel search around the best full-pixel vector (the 449 PIXEL option is ignored for this search technique). 450 451.TP 452\fBBSEARCH_ALG\fP {\fBSIMPLE\fP | \fBCROSS2\fP | \fBEXHAUSTIVE\fP} 453This statement tells \fBppmtompeg\fP what kind of search 454 technique (algorithm) to use for B frames. \fBSIMPLE\fP means 455 find best forward and backward vectors, then interpolate. 456 \fBCROSS2\fP means find those two vectors, then see what backward 457 vector best matches the best forward vector, and vice versa. 458 \fBEXHAUSTIVE\fP does an n-squared search and is 459 \fIextremely\fP slow in relation to the others (\fBCROSS2\fP 460 is about half as fast as \fBSIMPLE\fP). 461 462.TP 463\fBIQSCALE\fP \fIn\fP 464Use \fIn\fP as the qscale for I frames. 465 See 466.UR #qscale 467Qscale 468.UE 469\&. 470 471.TP 472\fBPQSCALE\fP \fIn\fP 473Use \fIn\fP as the qscale for P frames. 474 See 475.UR #qscale 476Qscale 477.UE 478\&. 479 480.TP 481\fBBQSCALE\fP \fIn\fP 482Use \fIn\fP as the qscale for B frames. 483 See 484.UR #qscale 485Qscale 486.UE 487\&. 488 489.TP 490\fBREFERENCE_FRAME\fP {\fBORIGINAL\fP | \fBDECODED\fP} 491This 492statement determines whether \fBppmtompeg\fP uses the original images 493or the decoded images when computing motion vectors. Using decoded 494images is more accurate and should increase the playback quality of 495the output, but it makes the encoding take longer and seems to give 496worse compression. It also causes some complications with parallel 497encoding. (see the section on parallel encoding). One thing you can 498do as a trade-off is select \fBORIGINAL\fP here, and lower the 499qscale (see \fBQSCALE\fP if the quality is not good enough. 500 501.B Original or Decoded? (Normalized) 502.TS 503r c c c c c. 504_ 505Reference Compression Speed Quality I Quality P Quality B 506Decoded 1000 1000 1000 969 919 507Original 885 1373 1000 912 884 508.TE 509 510 511 512 513 514.PP 515The following lines are optional: 516 517 518 519.TP 520\fBFORCE_ENCODE_LAST_FRAME\fP 521This statement is obsolete. It does nothing. 522.sp 523Before Netpbm 10.26 (January 2005), \fBppmtompeg\fP would drop 524trailing B frames from your movie, since a movie can't end with a B 525frame. (See 526.UR #ipb 527I Frames, P Frames, B Frames 528.UE 529\&. 530You would have to specify \fBFORCE_ENCODE_LAST_FRAME\fP to stop 531that from happening and get the same function that \fBppmtompeg\fP 532has today. 533 534 535.TP 536\fBNIQTABLE\fP 537This statement specifies a custom non-intra quantization table. 538If you don't specify this statement, \fBppmtompeg\fP uses a default 539non-intra quantization table. 540.sp 541The 8 lines immediately following \fBNIQTABLE\fP specify the quantization 542table. Each line defines a table row and consists of 8 integers, 543whitespace-delimited, which define the table columns. 544 545.TP 546\fBIQTABLE\fP 547This is analogous to NIQTABLE, but for the intra quantization table. 548 549.TP 550\fBASPECT_RATIO\fP \fIratio\fP 551This statement specifies the aspect ratio for \fBppmtompeg\fP to 552specify in the MPEG output. I'm not sure what this is used for. 553.sp 554\fIratio\fP must be 1.0, 0.6735, 0.7031, 0.7615, 0.8055, 0.8437, 5550.8935, 0.9157, 0.9815, 1.0255, 1.0695, 1.0950, 1.1575, or 1.2015. 556 557.TP 558\fBFRAME_RATE\fP \fIrate\fP 559This specifies the frame rate for \fBppmtompeg\fP to specify in the 560MPEG output. Some players use this value to determine the playback rate. 561.sp 562\fIrate\fP must be 23.976, 24, 25, 29.97, 30, 50, 59.94, or 60. 563 564.TP 565\fBBIT_RATE\fP \fIrate\fP 566This specifies the bit rate for Constant Bit Rate (CBR) encoding. 567.sp 568\fIrate\fP must be an integer. 569 570.TP 571\fBBUFFER_SIZE\fP \fIsize\fP 572This specifies the value 573\fBppmtompeg\fP is to specify in the MPEG output for the Video 574Buffering Verifier (VBV) buffer size needed to decode the sequence. 575.sp 576A Video Verifying Buffer is a buffer in which a decoder keeps the 577decoded bits in order to match the uneven speed of the decoding with 578the required constant playback speed. 579.sp 580As \fBppmtompeg\fP encodes the image, it simulates the decoding 581process in terms of how many bits would be in the VBV as each frame gets 582decoded, assuming a VBV of the size you indicate. 583.sp 584If you specify the \fBWARN_VBV_UNDERFLOW\fP statement, 585\fBppmtompeg\fP issues a warning each time the simulation underflows 586the buffer, which suggests that an underflow would occur on playback, 587which suggests the buffer is too small. 588.sp 589If you specify the \fBWARN_VBV_OVERFLOW\fP statement, 590\fBppmtompeg\fP issues a warning each time the simulation overflows 591the buffer, which suggests that an overflow would occur on playback, 592which suggests the buffer is too small. 593 594.TP 595\fBWARN_VBV_UNDERFLOW\fP 596.TP 597\fBWARN_VBV_OVERFLOW\fP 598See \fBBUFFER_SIZE\fP. 599.sp 600These options were new in Netpbm 10.26 (January 2005). Before that, 601\fBppmtompeg\fP issued the warnings always. 602 603 604 605 606The following statements apply only to parallel operation: 607 608 609 610.TP 611\fBPARALLEL\fP 612This statement, paired with \fBEND PARALLEL\fP, is what causes 613\fBppmtompeg\fP to operate in parallel mode. See 614.UR #parallel 615Parallel Operation 616.UE 617\&. 618 619.TP 620\fBEND PARALLEL\fP 621This goes with \fBPARALLEL\fP. 622 623.TP 624\fBPARALLEL_TEST_FRAMES\fP \fIn\fP 625The master starts off by measuring each slave's speed. It does 626this by giving each slave \fIn\fP frames to encode and noting how 627long the slave takes to finish. These are not just test frames, 628though -- they're real frames and the results become part of the 629output. 630\fBppmtompeg\fP is old and measures time in undivided seconds, so 631to get useful timings, specify enough frames that it will take at 632least 5 seconds to process them. The default is 10. 633.sp 634If you specify \fBFORCE_I_ALIGN\fP, \fBppmtompeg\fP will increase 635the test frames value enough to maintain the alignment. 636.sp 637If there aren't enough frames for every slave to have the indicated 638number of test frames, \fBppmtompeg\fP will give some slaves fewer. 639 640 641.TP 642\fBPARALLEL_TIME_CHUNKS\fP \fIt\fP 643When you specify this statement, the master attempts to feed work 644to the slaves in chunks that take \fIt\fP seconds to process. It uses 645the speed measurement it made when it started up (see PARALLEL_TEST_FRAMES) 646to decide how many frames to put in the chunk. This statement obviously 647doesn't affect the first batch of work sent to each slave, which is the 648one used to measure the slave's speed. 649.sp 650Smaller values of \fIt\fP increase communication, but improve load 651balancing. The default is 30 seconds. 652.sp 653You may specify only one of PARALLEL_TIME_CHUNKS, PARALLEL_CHUNK_TAPER, 654and PARALLEL_PERFECT. PARALLEL_CHUNK_TAPER is usually best. 655 656.TP 657\fBPARALLEL_CHUNK_TAPER\fP 658When you specify this statement, the master distributes work like 659with PARALLEL_TIME_CHUNKS, except that the master chooses the number 660of seconds for the chunks. It starts with a large number and, as it 661gets closer to finishing the job, reduces it. That way, it reduces 662scheduling overhead when precise scheduling isn't helpful, but still 663prevents a slave from finishing early after all the work has already 664been handed out to the other slaves, and then sitting idle while 665there's still work to do. 666.sp 667You may specify only one of PARALLEL_TIME_CHUNKS, PARALLEL_CHUNK_TAPER, 668and PARALLEL_PERFECT. PARALLEL_CHUNK_TAPER is usually best. 669 670 671.TP 672\fBPARALLEL_PERFECT\fP 673If this statement is present, \fBppmtompeg\fP schedules on the 674assumption that each machine is about the same speed. The master will 675simply divide up the frames evenly between the slaves -- each 676slave gets the same number of frames. If some slaves are faster than 677others, they will finish first and remain idle while the slower slaves 678continue. 679.sp 680This has the advantage of minimal scheduling overhead. Where slaves 681have different speeds, though, it makes inefficient use of the fast 682ones. Where slaves are the same speed, it also has the disadvantage 683that they all finish at the same time and feed their output to the 684single Combine Server in a burst, which makes less efficient use of 685the Combine Server and thus can increase the total elapsed time. 686.sp 687You may specify only one of PARALLEL_TIME_CHUNKS, PARALLEL_CHUNK_TAPER, 688and PARALLEL_PERFECT. PARALLEL_CHUNK_TAPER is usually best. 689 690.TP 691\fBRSH\fP \fIremote_shell_command\fP 692\fBppmtompeg\fP executes the shell command 693\fIremote_shell_command\fP to start a process on another machine. 694The default command is \fBrsh\fP, and whatever command you specify 695must have compatible semantics. \fBssh\fP is usually compatible. 696The command \fBppmtompeg\fP uses is one like this: 697\fBssh remote.host.com -l username shellcommand\fP. 698.sp 699Be sure to set up \fB.rhosts\fP files or SSH key authorizations 700where needed. Otherwise, you'll have to type in passwords. 701.sp 702On some HP machines, \fBrsh\fP is the restricted shell, and you want 703to specify \fBremsh\fP. 704 705.TP 706\fBFORCE_I_ALIGN\fP 707This statement forces each slave to encode a chunk of frames which 708is a multiple of the pattern length (see \fBPATTERN\fP). Since the 709first frame in any pattern is an I frame, this forces each chunk 710encoded by a slave to begin with an I frame. 711.sp 712This document used to say there was an argument to 713\fBFORCE_I_ALIGN\fP which was the number of frames \fBppmtompeg\fP 714would use (and was required to be a multiple of the pattern length). 715But \fBppmtompeg\fP has apparently always ignored that argument, and 716it does now. 717 718.TP 719\fBKEEP_TEMP_FILES\fP 720This statement causes \fBppmtompeg\fP not to delete the temporary 721files it uses to transmit encoded frames to the combine server. This 722means you will be left with a file for each frame, the same as you 723would get with the \fB-frames\fP option. 724.sp 725This is mostly useful for debugging. 726.sp 727This works only if you're using a shared filesystem to communicate 728between the servers. 729.sp 730This option was new in Netpbm 10.26 (January 2005). 731 732 733 734 735.SS Parameter File Notes 736.PP 737 If you use the \fB-combine_gops\fP option, then you need to specify 738only the SIZE and OUTPUT values in the parameter file. In 739addition, the parameter file may specify input GOP files in the same 740manner as normal input files -- except instead of using INPUT_DIR, 741INPUT, and END_INPUT, use GOP_INPUT_DIR, GOP_INPUT, and GOP_END_INPUT. 742If you specify no input GOP files, then \fBppmtompeg\fP uses by default the 743output file name with suffix \fB.gop.\fP\fIgop_num\fP, with \fIgop_num\fP 744starting from 0, as the input files. 745.PP 746If you use the \fB-combine_frames\fP option, then you need to 747specify only the SIZE, GOP_SIZE, and OUTPUT values in the 748parameter file. In addition, the parameter file may specify input 749frame files in the same manner as normal input files -- except instead 750of using INPUT_DIR, INPUT, and END_INPUT, use FRAME_INPUT_DIR, 751FRAME_INPUT, and FRAME_END_INPUT. If no input frame files are 752specified, then the default is to use the output file name with suffix 753\fB.frame.\fP\fIframe_num\fP, with \fIframe_num\fP starting from 0, 754as the input files. 755.PP 756Any number of spaces and tabs may come between each option and value. Lines 757beginning with \fB#\fP are ignored. Any other lines are ignored except for 758those between INPUT and END_INPUT. This allows you to use the same 759parameter file for normal usage and for \fB-combine_gops\fP and 760\fB-combine_frames\fP. 761.PP 762The file format is case-sensitive so all keywords should be in 763upper case. 764.PP 765The statements may appear in any order, except that the order within 766a block statement (such as INPUT ... END INPUT) is significant. 767.PP 768\fBppmtompeg\fP is prepared to handle up to 16 B frames between 769reference frames when encoding with input from stdin. (To build a 770modified \fBppmtompeg\fP with a higher limit, change the constant 771B_FRAME_RUN in frame.c and recompile). 772 773.UN general 774.SH GENERAL USAGE INFORMATION 775 776.UN qscale 777.SS Qscale 778.PP 779The quantization scale values (qscale) give a trade-off between 780quality and compression. Using different Qscale values has very little 781effect on speed. The qscale values can be set separately for I, P, and 782B frames. 783.PP 784You select the qscale values with the \fBIQSCALE\fP, 785\fBPQSCALE\fP, and \fBBSCALE\fP parameter file statements. 786.PP 787A qscale value is an integer from 1 to 31. Larger numbers give 788better compression, but worse quality. In the following, the quality 789numbers are peak signal-to-noise ratio, defined as: 790.B signal-to-noise formula 791.IMG -C ppmtompeg-snr.gif 792where MSE is the mean squared error. 793 794.PP 795Flower garden tests: 796 797.B Qscale vs Quality 798.TS 799r r r r. 800_ 801Qscale I Frames P Frames B Frames 8021 43.2 46.3 46.5 8036 32.6 34.6 34.3 80411 28.6 29.5 30.0 80516 26.3 26.8 28.6 80621 24.7 25.0 27.9 80726 23.5 23.9 27.5 80831 22.6 23.0 27.3 809.TE 810 811.B Qscale vs Compression 812.TS 813r r r r. 814_ 815Qscale I Frames P Frames B Frames 8161 2 2 2 8176 7 10 15 81811 11 18 43 81916 15 29 97 82021 19 41 173 82126 24 56 256 82231 28 73 330 823.TE 824 825 826.SS Search Techniques 827 828.PP 829There are several different motion vector search techniques 830available. There are different techniques available for P frame 831search and B frame search. Using different search techniques present 832little difference in quality, but a large difference in compression 833and speed. 834 835.PP 836There are 4 types of P frame search: Exhaustive, TwoLevel, 837SubSample, and Logarithmic. 838 839.PP 840There are 3 types of B frame search: Exhaustive, Cross2, and 841Simple. 842 843The recommended search techniques are TwoLevel and Logarithmic for 844P frame search, and Cross2 and Simple for B frame search. Here are 845some numbers comparing the different search methods: 846 847.B P frame Motion Vector Search (Normalized) 848.TS 849r c c c. 850_ 851Technique T{ 852Compression 853.UR #smallbetter 854\u1\d 855.UE 856T} T{ 857Speed 858.UR #largefaster 859\u2\d 860.UE 861T} T{ 862Quality 863.UR #largebetter 864\u3\d 865.UE 866T} 867Exhaustive 1000 1000 1000 868SubSample 1008 2456 1000 869TwoLevel 1009 3237 1000 870Logarithmic 1085 8229 998 871.TE 872 873.B B frame Motion Vector Search (Normalized) 874.TS 875r c c c. 876_ 877Technique T{ 878Compression 879.UR #smallbetter 880\u1\d 881.UE 882T} T{ 883Speed 884.UR #largefaster 885\u2\d 886.UE 887T} T{ 888Quality 889.UR #largebetter 890\u3\d 891.UE 892T} 893Exhaustive 1000 1000 1000 894Cross2 975 1000 996 895Simple 938 1765 991 896.TE 897 898.UN smallbetter 899\u1\dSmaller numbers are better 900compression. 901 902.UN largefaster 903\u2\dLarger numbers mean faster 904execution. 905 906.UN largebetter 907\u3\dLarger numbers mean better quality. 908.PP 909For some reason, Simple seems to give better compression, but it 910depends on the image sequence. 911.PP 912Select the search techniques with the \fBPSEARCH_ALG\fP and 913\fBBSEARCH_ALG\fP parameter file statements. 914 915 916.UN gop 917.SS Group Of Pictures (GOP) 918.PP 919A Group of Pictures (GOP) is a roughly independently decodable 920sequence of frames. An MPEG video stream is made of one or more 921GOP's. You may specify how many frames should be in each GOP with the 922\fBGOP_SIZE\fP parameter file statement. A GOP always starts with an 923I frame. 924.PP 925Instead of encoding an entire sequence, you can encode a single 926GOP. To do this, use the \fB-gop\fP command option. You can later 927join the resulting GOP files at any time by running \fBppmtompeg\fP 928with the \fB-combine_gops\fP command option. 929 930 931.SS Slices 932.PP 933A slice is an independently decodable unit in a frame. It can be 934as small as one macroblock, or it can be as big as the entire frame. 935Barring transmission error, adding slices does not change quality or 936speed; the only effect is slightly worse compression. More slices are 937used for noisy transmission so that errors are more recoverable. Since 938usually errors are not such a problem, we usually just use one slice 939per frame. 940 941.PP 942Control the slice size with the \fBSLICES_PER_FRAME\fP parameter 943file statement. 944.PP 945Some MPEG playback systems require that each slice consist of whole 946rows of macroblocks. If you are encoding for this kind of player, if 947the height of the image is H pixels, then you should set the 948SLICES_PER_FRAME to some number which divides H/16. For example, if 949the image is 240 pixels (15 macroblocks) high, then you should use 950only 15, 5, 3, or 1 slices per frame. 951 952.PP 953Note: these MPEG playback systems are really wrong, since the MPEG 954standard says this doesn't have to be so. 955 956 957 958.SS Search Window 959 960.PP 961The search window is the window in which \fBppmtompeg\fP searches 962for motion vectors. The window is a square. You can specify the size 963of the square, and whether to allow half-pixel motion vectors or not, 964with the \fBRANGE\fP and \fBPIXEL\fP parameter file statements. 965 966.UN ipb 967.SS I Frames, P Frames, B Frames 968.PP 969In MPEG-1, a movie is represented as a sequence of MPEG frames, 970each of which is an I Frame, a P Frame, or a B Frame. Each represents 971an actual frame of the movie (don't get confused by the dual use of 972the word "frame." A movie frame is a graphical image. An MPEG frame 973is a set of data that describes a movie frame). 974.PP 975An I frame ("intra" frame) describes a movie frame in isolation -- 976without respect to any other frame in the movie. A P frame 977("predictive" frame) describes a movie frame by describing how it 978differs from the movie frame described by the latest preceding I or 979P frame. A B frame ("bidirectional" frame) describes a movie frame by 980describing how it differs from the movie frames described by the 981nearest I or P frame before \fIand\fP after it. 982.PP 983Note that the first frame of a movie must be described by an I 984frame (because there is no previous movie frame) and the last movie 985frame must be described by an I or P frame (because there is no 986subsequent movie frame). 987.PP 988Beyond that, you can choose which frames are represented by which 989types. You specify a pattern, such as IBPBP and \fBppmtompeg\fP 990simply repeats it over and over throughout the movie. The pattern 991affects speed, quality, and stream size. Here is a chart which shows 992some of the trade-offs: 993 994.B Comparison of I/P/B Frames (Normalized) 995.TS 996r c c c. 997_ 998Frame Type Size Speed Quality 999I frames 1000 1000 1000 1000P frames 409 609 969 1001B frames 72 260 919 1002.TE 1003 1004(this is with constant qscale) 1005 1006.PP 1007A standard sequence is IBBPBBPBBPBBPBB. 1008 1009.PP 1010Select the sequence with the \fBPATTERN\fP parameter file statement. 1011.PP 1012Since the last MPEG frame cannot be a B frame (see above), if the 1013pattern you specify indicates a B frame for the last movie frame of 1014the movie, \fBppmtompeg\fP makes it an I frame instead. 1015.PP 1016Before Netpbm 10.26 (January 2005), \fBppmtompeg\fP instead drops 1017the trailing B frames by default, and you need the 1018\fBFORCE_ENCODE_LAST_FRAME\fP parameter file statement to make it do 1019this. 1020.PP 1021The MPEG frames don't appear in the MPEG-1 stream in the same order that 1022the corresponding movie frames appear in the movie -- the B frames come after 1023the I and P frames on which they are based. For example, if the movie is 10244 frames that you will represent with the pattern IBBP, the MPEG-1 stream 1025will start with an I frame describing movie frame 0. The next frame in 1026the MPEG-1 stream is a P frame describing movie frame 3. The last two 1027frames in the MPEG-1 stream are B frames describing movie frames 1 and 2, 1028respectively. 1029 1030 1031.SS Specifying Input and Output Files 1032.PP 1033Specify the input frame images with the \fBINPUT_DIR\fP, 1034\fBINPUT\fP, \fBEND_INPUT\fP, \fBBASE_FILE_FORMAT\fP, 1035\fBSIZE\fP, \fBYUV_FORMAT\fP and \fBINPUT_CONVERT\fP parameter 1036file statements. 1037.PP 1038Specify the output file with the \fBOUTPUT\fP parameter file statement. 1039 1040 1041.SS Statistics 1042.PP 1043\fBppmtompeg\fP can generate a variety of statistics about the 1044encoding. See the \fB-stat\fP, \fB-snr\fP, \fB-mv_histogram\fP, 1045\fB-quiet\fP, \fB-no_frame_summary\fP, and \fB-bit_rate_info\fP 1046options. 1047 1048 1049.UN parallel 1050.SH PARALLEL OPERATION 1051.PP 1052You can run \fBppmtompeg\fP on multiple machines at once, encoding 1053the same MPEG stream. When you do, the machines are used as shown in 1054the following diagram. We call this "parallel mode." 1055.PP 1056.B ppmtompeg-par.gif 1057.IMG -C ppmtompeg-par.gif 1058.PP 1059To do parallel processing, put the statement 1060 1061.nf 1062 PARALLEL 1063.fi 1064 1065in the parameter file, followed by a listing of the machines, one 1066machine per line, then 1067 1068.nf 1069 END_PARALLEL 1070.fi 1071 1072Each of the machine lines must be in one of two forms. If the machine 1073has filesystem access to the input files, then the line is: 1074.PP 1075\fImachine\fP \fIuser\fP \fIexecutable\fP 1076.PP 1077The executable is normally \fBppmtompeg\fP (you may need to give 1078the complete path if you've built for different architectures). If 1079the machine does not have filesystem access to the input files, the line 1080is: 1081.PP 1082\fBREMOTE\fP \fImachine\fP \fIuser\fP \fIexecutable\fP 1083\fIparameter file\fP 1084.PP 1085The \fB-max_machines\fP command option limits the number of 1086machines \fBppmtompeg\fP will use. If you specify more machines in 1087the parameter file than \fB-max_machines\fP allows, \fBppmtompeg\fP 1088uses only the machines listed first. This is handy if you want to 1089experiment with different amounts of parallelism. 1090.PP 1091In general, you should use full path file names when describing 1092executables and parameter files. This \fIincludes\fP the parameter 1093file argument on the original invocation of \fBppmtompeg\fP. 1094.PP 1095All file names must be the same on all systems (so if e.g. you're 1096using an NFS filesystem, you must make sure it is mounted at the same 1097mountpoint on all systems). 1098.PP 1099Because not all of the processes involved in parallel operation 1100have easy access to the input files, you must specify the \fBSIZE\fP 1101parameter file statement when you do parallel operation. 1102.PP 1103The machine on which you originally invoke \fBppmtompeg\fP is the 1104master machine. It hosts a "combine server,", a 1105"decode server," and a number of "i/o servers," 1106all as separate processes. The other machines in the network (listed 1107in the parameter file) are slave machines. Each hosts a single 1108process that continuously requests work from the master and does it. 1109The slave process does the computation to encode MPEG frames. It 1110processes frames in batches identified by the master. 1111.PP 1112The master uses a remote shell command to start a process on a 1113slave machine. By default, it uses an \fBrsh\fP shell command to do 1114this. But use the \fBRSH\fP parameter file statement to control 1115this. The shell command the master executes remotely is 1116\fBppmtompeg\fP, but with options to indicate that it is to perform 1117slave functions. 1118.PP 1119The various machines talk to each other over TCP connections. Each 1120machine finds and binds to a free TCP port number and tells its 1121partners the port number. These port numbers are at least 2048. 1122.PP 1123Use the PARALLEL_TEST_FRAMES, PARALLEL_TIME_CHUNKS, and 1124PARALLEL_PERFECT parameter file statements to control the way the 1125master divides up work among the slaves. 1126.PP 1127Use the \fB-nice\fP command option to cause all slave processes to run 1128"nicely," i.e. as low priority processes. That way, this substantial and 1129long-running CPU load will have minimal impact on other, possibly 1130interactive, users of the systems. 1131 1132.UN speed 1133.SH SPEED 1134.PP 1135Here is a look at \fBppmtompeg\fP speed, in single-node (not parallel) 1136operation: 1137 1138.B Compression Speed 1139.TS 1140r c. 1141_ 1142Machine Type Macroblocks per second\u1\d 1143HP 9000/755 280 1144DEC 3000/400 247 1145HP 9000/750 191 1146Sparc 10 104 1147DEC 5000 68 1148.TE 1149\u1\dA macroblock is a 16x16 pixel square 1150.PP 1151The measurements in the table are with inputs and outputs via a 1152conventional locally attached filesystem. If you are using a network 1153filesystem over a single 10 MB/s Ethernet, that constrains your speed more 1154than your CPU speed. In that case, don't expect to get better than 4 1155or 5 frames per second no matter how fast your CPUs are. 1156.PP 1157Network speed is even more of a bottleneck when the slaves do not 1158have filesystem access to the input files -- i.e. you declare them 1159REMOTE. 1160.PP 1161Where I/O is the bottleneck, size of the input frames can make a big 1162difference. So YUV input is better than PPM, and JPEG is better than 1163both. 1164.PP 1165When you're first trying to get parallel mode working, be sure to 1166use the \fB-debug_machines\fP option so you can see what's going on. 1167Also, \fB-debug_sockets\fP can help you diagnose communication 1168problems. 1169 1170 1171.UN authors 1172.SH AUTHORS 1173 1174 1175 1176.IP \(bu 1177Kevin Gong - University of California, Berkeley, \fIkeving@cs.berkeley.edu\fP 1178 1179.IP \(bu 1180Ketan Patel - University of California, Berkeley, \fIkpatel@cs.berkeley.edu\fP 1181 1182.IP \(bu 1183Dan Wallach - University of California, Berkeley, \fIdwallach@cs.berkeley.edu\fP 1184 1185.IP \(bu 1186Darryl Brown - University of California, Berkeley, \fIdarryl@cs.berkeley.edu\fP 1187 1188.IP \(bu 1189Eugene Hung - University of California, Berkeley, \fIeyhung@cs.berkeley.edu\fP 1190 1191.IP \(bu 1192Steve Smoot - University of California, Berkeley, \fIsmoot@cs.berkeley.edu\fP 1193.SH DOCUMENT SOURCE 1194This manual page was generated by the Netpbm tool 'makeman' from HTML 1195source. The master documentation is at 1196.IP 1197.B http://netpbm.sourceforge.net/doc/ppmtompeg.html 1198.PP
