1 2libng overview 3============== 4 5 6some general notes 7------------------ 8 9The text below is not a complete reference (yet?), it just tries to 10give a overview and explain the design of the library. Have a look at 11grab-ng.h, this is the header file where everything is defined. 12 13If you are looking for some simple sample code check out 14console/webcam.c. More complex usages of libng can be found everythere 15in the xawtv source code: In common/capture.c for example, where most of 16the threaded movie recording code for xawtv+streamer is. 17 18There are two types of structs: Those which carrying some data (like 19ng_video_fmt or ng_video_buf) and those which define a interface (like 20ng_driver). 21 22The interfaces all have a initialization function which returns a void 23pointer as handle. All other interface functions expect getting that 24handle passed in. The complete state information is kept there. 25Global variables are a no-no, the interfaces need to be reentrant (so 26you can use multiple instances of them at the same time). 27 28 29video buffers (struct ng_video_buf) 30----------------------------------- 31 32One video buffer holds one frame. There are some rules for video 33frames: 34 35 (1) Multiple instances (threads for example) may hold a pointer to 36 the buffer. The refcount variable is used to keep track of the 37 number of references. If you hand out a pointer to the buffer to 38 someone else *and* keep a pointer to the buffer yourself to use 39 it later refcount must be increased by one. If you don't need 40 the buffer any more, just free it with ng_release_video_buf(). 41 That function will take care about count down the reference 42 counter and freeing the buffer if refcount is zero. 43 44 (2) The above implies you should not write to ng_video_buf->data 45 because another thread might use the image data while you are 46 modifying it. Allocate a new buffer and put the new data in there 47 instead if you want process the image data. You can use 48 ng_malloc_video_buf() to get a buffer. Don't forget to copy the 49 frame meta data to the new buffer (ng_video_buf->info). 50 51Within the current implementation two types of buffers exist: Those 52malloc()ed ones which are returned by ng_malloc_video_buf() and 53buffers provided by the hardware drivers, where ng_video_buf->data is 54a pointer directly to the mmap()ed buffer(s). 55 56 57video capture / overlay drivers (struct ng_vid_driver) 58------------------------------------------------------ 59 60This is a interface to a capture driver. Right now three different 61ones exist: 62 63 (1) video4linux (current linux API, see plugins/drv1-v4l.c). 64 (2) video4linux two (work in progress - new API for linux, see 65 plugins/drv0-v4l2.c). 66 (3) bktr (bt848/878 driver for FreeBSD + OpenBSD, see 67 plugins/drv0-bsd.c). 68 69xawtv uses struct ng_vid_driver for Xvideo support too (see x11/xv.c). 70 71open()/close() should be clear. The capabilities() function returns a 72bitfield which specifies the capabilities of the driver. The *attr* 73functions can be used to control several (hardware) settings (see 74below for more on attributes). 75 76To for a overlay use setupfb() and overlay(). For linux setupfb() 77just verifies the driver parameters, it expects the setup is done with 78a extern utility like v4l-conf (because one needs root priviliges for 79that). 80 81To capture frames you have to configure the format with setformat() 82first. Be aware that the image size might have changed on return, 83depending on the capabilities of the underlying hardware. For video 84recording use startvideo(), multiple nextframe() calls and 85stopvideo(), for single frames use getimage(). 86 87 88output drivers (struct ng_writer) 89--------------------------------- 90 91This is a interface for movie writer code. Four different ones exist: 92 93 (1) Microsoft's AVI 94 (2) Apple Quicktime (using the libquicktime library). 95 (3) raw, uncompressed data (one big file). 96 (4) one image file/frame (jpeg or ppm). 97 98The first two write audio and video data into the same stream, the 99last two can write audio data to a separate wav file. 100 101The audio/video args hold a list of supported formats. Usage should 102be pretty straight forward: wr_open(), write data with wr_video() + 103wr_audio(), then wr_close(). You can write both audio-only and 104video-only streams if you want. 105 106 107attributes (struct ng_attribute) 108-------------------------------- 109 110Attributes can be used to control properties attributes. Right now 111they are only used for video drivers (struct ng_driver), but that 112might change in the future. There are a number of standard attributes 113defined (tv norm, volume and the like), but it is also possible to 114specify non-standard attributes. 115 116struct ng_attribute also has functions to list/read/modify the given 117attribute. A number of helper functions to search a attribute list by 118id / name and for multiple choice attribute handling are available 119too. 120 121 122audio recording + mixer control 123------------------------------- 124 125There are only some structs for audio formats and audio data chunks. 126The movie writers use these. There are also interfaces for sound 127recording and mixer control (struct ng_dsp_driver + struct 128ng_mix_driver) and a implementation for the OSS API (in 129plugins/snd-oss.c). 130 131I'm not that happy with the current design of the mixer stuff, it 132likely will change in the future. 133 134 135color space conversion and compression 136-------------------------------------- 137 138struct ng_video_conv describes a converter. There are plenty built-in 139into libng, but it is also possible to add more using plugins. These 140converters handle (a) color space conversion (yuv -> rgb), (b) convert 141rgb with different color depths, (c) handle compression. 142 143 144image filtering 145--------------- 146 147I've recently added a interfaces for image filtering (i.e. on-the-fly 148image processing). struct ng_filter is it, in plugins/flt-*.c is some 149sample code. There is no complex stuff yet. 150 151For now filters are limited: They work on a frame-by-frame base, 152i.e. for one frame which gets passed in one is expected to come out. 153Read: you can't (yet?) merge multiple frames into one. The input and 154output format is expected to be the same. 155 156Filters should be able to handle as much formats as possible, neither 157xawtv nor libng attempt to do any conversions (i.e. do yuv->rgb -> 158filter -> rgb->yuv for you because the filter works in RGB space 159only). ng_filter->fmts lists the supported formats of a given filter, 160and if the required format isn't supported by some filter it is simply 161skipped and no filtering takes place. 162 163Most frequently formats are: 164 * RGB (various depts), to display images on the X11 screen, for ppm 165 capture, ... 166 * packed pixel yuv (If xawtv/motv blits images using the Xvideo 167 extention instead of normal X11 ximages). 168 * planar yuv (for recording compressed video, libjpeg can be feeded 169 directly with planar yuv to save some CPU cycles for rgb->yuv color 170 space conversion). 171 172 173misc 174---- 175 176There are some helper functions for various stuff, see grab-ng.[ch]. 177There are also some arrays with text descriptions and other 178informations about audio/video formats: ng_[va]fmt_to_*. 179