1//////////////////////////////////////////////////////////////////////////////////////// 2/// \file 3/// \brief Doxygen documentation 4/// \author Lubomir Bourdev and Hailin Jin \n 5/// Adobe Systems Incorporated 6/// 7/// 8//////////////////////////////////////////////////////////////////////////////////////// 9 10/** 11\page GILTutorial Generic Image Library Tutorial 12 13\author Lubomir Bourdev (lbourdev@adobe.com) and Hailin Jin (hljin@adobe.com) \n 14 Adobe Systems Incorporated 15 16\version 2.1 17\date September 15, 2007 18 19The Generic Image Library (GIL) is a C++ library that abstracts image representations from algorithms and allows writing code that can work on 20a variety of images with performance similar to hand-writing for a specific image type. 21<p>This document will give you a jump-start in using GIL. It does not discuss the underlying design 22of the library and does not cover all aspects of it. You can find a detailed library design document on the main GIL web page 23at http://stlab.adobe.com/gil 24 25- \ref InstallSec 26- \ref ExampleSec 27 - \ref InterfaceSec 28 - \ref FirstImplementationSec 29 - \ref LocatorsSec 30 - \ref GenericVersionSec 31 - \ref ImageViewTransformationSec 32 - \ref OneDIteratorsSec 33 - \ref STLEquivalentsSec 34 - \ref ColorConversionSec 35 - \ref ImagesSec 36 - \ref VirtualViewSec 37 - \ref DynamicImageSec 38 - \ref ConclusionSec 39- \ref AppendixSec 40 - \ref AppendixConventionSec 41 42\section InstallSec Installation 43 44The latest version of GIL can be downloaded from GIL's web page, at http://stlab.adobe.com/gil. 45GIL is approved for integration into Boost and in the future will be installed simply by installing Boost from http://www.boost.org. 46GIL consists of header files only and does not require any libraries to link against. It does not require Boost to be built. 47Including \p boost/gil/gil_all.hpp will be sufficient for most projects. 48 49\section ExampleSec Example - Computing the Image Gradient 50 51This tutorial will walk through an example of using GIL to compute the image gradients. 52We will start with some very simple and non-generic code and make it more generic as we go along. 53Let us start with a horizontal gradient and use the simplest possible approximation to a gradient - central difference. 54The gradient at pixel x can be approximated with the half-difference of its two neighboring pixels: 55D[x] = (I[x-1] - I[x+1]) / 2 56 57For simplicity, we will also ignore the boundary cases - the pixels along the edges of the image for which one of the neighbors is not defined. 58The focus of this document is how to use GIL, not how to create a good gradient generation algorithm. 59 60\subsection InterfaceSec Interface and Glue Code 61 62Let us first start with 8-bit unsigned grayscale image as the input and 8-bit signed grayscale image as the output. 63Here is how the interface to our algorithm looks like: 64 65\code 66#include <boost/gil/gil_all.hpp> 67using namespace boost::gil; 68 69void x_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 70 assert(src.dimensions() == dst.dimensions()); 71 ... // compute the gradient 72} 73\endcode 74 75\p gray8c_view_t is the type of the source image view - an 8-bit grayscale view, whose pixels are read-only (denoted by the \p "c"). The output 76is a grayscale view with a 8-bit signed (denoted by the \p "s") integer channel type. See Appendix 1 for the complete convension GIL uses to name concrete types. 77 78GIL makes a distinction between an image and an image view. A GIL <em>image view</em>, is a shallow, lightweight view of a rectangular grid of pixels. It provides access to the pixels 79but does not own the pixels. Copy-constructing a view does not deep-copy the pixels. Image views do not propagate their constness to the pixels and should 80always be taken by a const reference. Whether a view is mutable or read-only (immutable) is a property of the view type. 81 82A GIL \e image, on the other hand, is a view with associated ownership. It is a container of pixels; its constructor/destructor allocates/deallocates the pixels, its copy-constructor 83performs deep-copy of the pixels and its operator== performs deep-compare of the pixels. Images also propagate their constness to their pixels - a constant reference to an image will not 84allow for modifying its pixels. 85 86Most GIL algorithms operate on image views; images are rarely needed. GIL's design is very similar to that of the STL. The STL equivalent of GIL's image is a container, like \p std::vector, whereas 87GIL's image view corresponds to STL's range, which is often represented with a pair of iterators. STL algorithms operate on ranges, just like GIL algorithms operate on image views. 88 89GIL's image views can be constructed from raw data - the dimensions, the number of bytes per row and the pixels, which for chunky views are represented with one pointer. Here is how to provide 90the glue between your code and GIL: 91 92\code 93void ComputeXGradientGray8(const unsigned char* src_pixels, ptrdiff_t src_row_bytes, int w, int h, 94 signed char* dst_pixels, ptrdiff_t dst_row_bytes) { 95 gray8c_view_t src = interleaved_view(w, h, (const gray8_pixel_t*)src_pixels,src_row_bytes); 96 gray8s_view_t dst = interleaved_view(w, h, ( gray8s_pixel_t*)dst_pixels,dst_row_bytes); 97 x_gradient(src,dst); 98} 99\endcode 100 101This glue code is very fast and views are lightweight - in the above example the views have a size of 16 bytes. They consist of a pointer to the top left pixel and three integers - the width, height, 102and number of bytes per row. 103 104\subsection FirstImplementationSec First Implementation 105 106Focusing on simplicity at the expense of speed, we can compute the horizontal gradient like this: 107 108\code 109void x_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 110 for (int y=0; y<src.height(); ++y) 111 for (int x=1; x<src.width()-1; ++x) 112 dst(x,y) = (src(x-1,y) - src(x+1,y)) / 2; 113} 114\endcode 115 116We use image view's \p operator(x,y) to get a reference to the pixel at a given location and we set it to the half-difference of its left and right neighbors. 117operator() returns a reference to a grayscale pixel. A grayscale pixel is convertible to its channel type (<tt>unsigned char</tt> for \p src) and it can be copy-constructed from a channel. 118(This is only true for grayscale pixels). 119While the above code is easy to read, it is not very fast, because the binary \p operator() computes the location of the pixel in a 2D grid, which involves addition and multiplication. Here is 120a faster version of the above: 121 122\code 123void x_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 124 for (int y=0; y<src.height(); ++y) { 125 gray8c_view_t::x_iterator src_it = src.row_begin(y); 126 gray8s_view_t::x_iterator dst_it = dst.row_begin(y); 127 128 for (int x=1; x<src.width()-1; ++x) 129 dst_it[x] = (src_it[x-1] - src_it[x+1]) / 2; 130 } 131} 132\endcode 133 134We use pixel iterators initialized at the beginning of each row. GIL's iterators are Random Access Traversal iterators. If you are not familiar with random access iterators, think of them as if they 135were pointers. In fact, in the above example the two iterator types are raw C pointers and their \p operator[] is a fast pointer indexing operator. 136 137The code to compute gradient in the vertical direction is very similar: 138 139\code 140void y_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 141 for (int x=0; x<src.width(); ++x) { 142 gray8c_view_t::y_iterator src_it = src.col_begin(x); 143 gray8s_view_t::y_iterator dst_it = dst.col_begin(x); 144 145 for (int y=1; y<src.height()-1; ++y) 146 dst_it[y] = (src_it[y-1] - src_it[y+1])/2; 147 } 148} 149\endcode 150 151Instead of looping over the rows, we loop over each column and create a \p y_iterator, an iterator moving vertically. In this case a simple pointer cannot be used because the distance 152between two adjacent pixels equals the number of bytes in each row of the image. GIL uses here a special step iterator class whose size is 8 bytes - it contains a raw C pointer and a step. 153Its \p operator[] multiplies the index by its step. 154 155The above version of \p y_gradient, however, is much slower (easily an order of magnitude slower) than \p x_gradient because of the memory access pattern; traversing an image vertically 156results in lots of cache misses. A much more efficient and cache-friendly version will iterate over the columns in the inner loop: 157 158\code 159void y_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 160 for (int y=1; y<src.height()-1; ++y) { 161 gray8c_view_t::x_iterator src1_it = src.row_begin(y-1); 162 gray8c_view_t::x_iterator src2_it = src.row_begin(y+1); 163 gray8s_view_t::x_iterator dst_it = dst.row_begin(y); 164 165 for (int x=0; x<src.width(); ++x) { 166 *dst_it = ((*src1_it) - (*src2_it))/2; 167 ++dst_it; 168 ++src1_it; 169 ++src2_it; 170 } 171 } 172} 173\endcode 174 175This sample code also shows an alternative way of using pixel iterators - instead of \p operator[] one could use increments and dereferences. 176 177 178 179 180 181\subsection LocatorsSec Using Locators 182 183Unfortunately this cache-friendly version requires the extra hassle of maintaining two separate iterators in the source view. For every pixel, 184we want to access its neighbors above and below it. Such relative access can be done with GIL locators: 185 186\code 187void y_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 188 gray8c_view_t::xy_locator src_loc = src.xy_at(0,1); 189 for (int y=1; y<src.height()-1; ++y) { 190 gray8s_view_t::x_iterator dst_it = dst.row_begin(y); 191 192 for (int x=0; x<src.width(); ++x) { 193 (*dst_it) = (src_loc(0,-1) - src_loc(0,1)) / 2; 194 ++dst_it; 195 ++src_loc.x(); // each dimension can be advanced separately 196 } 197 src_loc+=point2<std::ptrdiff_t>(-src.width(),1); // carriage return 198 } 199} 200\endcode 201 202The first line creates a locator pointing to the first pixel of the second row of the source view. A GIL pixel locator is very similar to an iterator, 203except that it can move both horizontally and vertically. \p src_loc.x() and \p src_loc.y() return references to a horizontal and a vertical iterator 204respectively, which can be used to move the locator along the desired dimension, as shown above. Additionally, the locator can be advanced in both dimensions 205simultaneously using its \p operator+= and \p operator-=. Similar to image views, locators provide binary \p operator() which returns a reference to a pixel 206with a relative offset to the current locator position. For example, \p src_loc(0,1) returns a reference to the neighbor below the current pixel. 207Locators are very lightweight objects - in the above example the locator has a size of 8 bytes - it consists of a raw pointer to the current pixel and an int 208indicating the number of bytes from one row to the next (which is the step when moving vertically). The call to \p ++src_loc.x() corresponds to a single C pointer increment. 209However, the example above performs more computations than necessary. The code src_loc(0,1) has to compute the offset of the pixel in two dimensions, which is slow. 210Notice though that the offset of the two neighbors is the same, regardless of the pixel location. To improve the performance, GIL can cache and reuse this offset: 211 212\code 213void y_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 214 gray8c_view_t::xy_locator src_loc = src.xy_at(0,1); 215 gray8c_view_t::xy_locator::cached_location_t above = src_loc.cache_location(0,-1); 216 gray8c_view_t::xy_locator::cached_location_t below = src_loc.cache_location(0, 1); 217 218 for (int y=1; y<src.height()-1; ++y) { 219 gray8s_view_t::x_iterator dst_it = dst.row_begin(y); 220 221 for (int x=0; x<src.width(); ++x) { 222 (*dst_it) = (src_loc[above] - src_loc[below])/2; 223 ++dst_it; 224 ++src_loc.x(); 225 } 226 src_loc+=point2<std::ptrdiff_t>(-src.width(),1); 227 } 228} 229\endcode 230 231In this example \p "src_loc[above]" corresponds to a fast pointer indexing operation and the code is efficient. 232 233\subsection GenericVersionSec Creating a Generic Version of GIL Algorithms 234 235Let us make our \p x_gradient more generic. It should work with any image views, as long as they have the same number of channels. 236The gradient operation is to be computed for each channel independently. Here is how the new interface looks like: 237 238\code 239template <typename SrcView, typename DstView> 240void x_gradient(const SrcView& src, const DstView& dst) { 241 gil_function_requires<ImageViewConcept<SrcView> >(); 242 gil_function_requires<MutableImageViewConcept<DstView> >(); 243 gil_function_requires<ColorSpacesCompatibleConcept< 244 typename color_space_type<SrcView>::type, 245 typename color_space_type<DstView>::type> >(); 246 247 ... // compute the gradient 248} 249\endcode 250 251The new algorithm now takes the types of the input and output image views as template parameters. 252That allows using both built-in GIL image views, as well as any user-defined image view classes. 253The first three lines are optional; they use \p boost::concept_check to ensure that the two arguments 254are valid GIL image views, that the second one is mutable and that their color spaces are compatible (i.e. have the same set of channels). 255 256GIL does not require using its own built-in constructs. You are free to use your own channels, color spaces, iterators, locators, views and images. 257However, to work with the rest of GIL they have to satisfy a set of requirements; in other words, they have to \e model the corresponding GIL \e concept. 258GIL's concepts are defined in the user guide. 259 260One of the biggest drawbacks of using 261templates and generic programming in C++ is that compile errors can be very difficult to comprehend. 262This is a side-effect of the lack of early type checking - a generic argument may not satisfy the requirements of a function, 263but the incompatibility may be triggered deep into a nested call, in code unfamiliar and hardly related to the problem. 264GIL uses \p boost::concept_check to mitigate this problem. The above three lines of code check whether the 265template parameters are valid models of their corresponding concepts. 266If a model is incorrect, the compile error will be inside \p gil_function_requires, which is much closer to the problem 267and easier to track. Furthermore, such checks get compiled out and have zero performance overhead. The disadvantage of using 268concept checks is the sometimes severe impact they have on compile time. This is why GIL performs concept checks only in 269debug mode, and only if \p BOOST_GIL_USE_CONCEPT_CHECK is defined (off by default). 270 271The body of the generic function is very similar to that of the concrete one. The biggest difference is that we need to loop over the 272channels of the pixel and compute the gradient for each channel: 273 274\code 275template <typename SrcView, typename DstView> 276void x_gradient(const SrcView& src, const DstView& dst) { 277 for (int y=0; y<src.height(); ++y) { 278 typename SrcView::x_iterator src_it = src.row_begin(y); 279 typename DstView::x_iterator dst_it = dst.row_begin(y); 280 281 for (int x=1; x<src.width()-1; ++x) 282 for (int c=0; c<num_channels<SrcView>::value; ++c) 283 dst_it[x][c] = (src_it[x-1][c]- src_it[x+1][c])/2; 284 } 285} 286\endcode 287 288Having an explicit loop for each channel could be a performance problem. GIL allows us to abstract out such per-channel operations: 289 290\code 291template <typename Out> 292struct halfdiff_cast_channels { 293 template <typename T> Out operator()(const T& in1, const T& in2) const { 294 return Out((in1-in2)/2); 295 } 296}; 297 298template <typename SrcView, typename DstView> 299void x_gradient(const SrcView& src, const DstView& dst) { 300 typedef typename channel_type<DstView>::type dst_channel_t; 301 302 for (int y=0; y<src.height(); ++y) { 303 typename SrcView::x_iterator src_it = src.row_begin(y); 304 typename DstView::x_iterator dst_it = dst.row_begin(y); 305 306 for (int x=1; x<src.width()-1; ++x) 307 static_transform(src_it[x-1], src_it[x+1], dst_it[x], 308 halfdiff_cast_channels<dst_channel_t>()); 309 } 310} 311\endcode 312 313\p static_transform is an example of a channel-level GIL algorithm. Other such algorithms are \p static_generate, \p static_fill and \p static_for_each. They are the channel-level equivalents 314of STL's \p generate, \p transform, \p fill and \p for_each respectively. GIL channel algorithms use static recursion to unroll the loops; they never loop over the channels explicitly. 315Note that sometimes modern compilers (at least Visual Studio 8) already unroll channel-level loops, such as the one above. However, another advantage of using 316GIL's channel-level algorithms is that they pair the channels semantically, not based on their order in memory. For example, the above example will properly match an RGB source 317with a BGR destination. 318 319Here is how we can use our generic version with images of different types: 320 321\code 322// Calling with 16-bit grayscale data 323void XGradientGray16_Gray32(const unsigned short* src_pixels, ptrdiff_t src_row_bytes, int w, int h, 324 signed int* dst_pixels, ptrdiff_t dst_row_bytes) { 325 gray16c_view_t src=interleaved_view(w,h,(const gray16_pixel_t*)src_pixels,src_row_bytes); 326 gray32s_view_t dst=interleaved_view(w,h,( gray32s_pixel_t*)dst_pixels,dst_row_bytes); 327 x_gradient(src,dst); 328} 329 330// Calling with 8-bit RGB data into 16-bit BGR 331void XGradientRGB8_BGR16(const unsigned char* src_pixels, ptrdiff_t src_row_bytes, int w, int h, 332 signed short* dst_pixels, ptrdiff_t dst_row_bytes) { 333 rgb8c_view_t src = interleaved_view(w,h,(const rgb8_pixel_t*)src_pixels,src_row_bytes); 334 rgb16s_view_t dst = interleaved_view(w,h,( rgb16s_pixel_t*)dst_pixels,dst_row_bytes); 335 x_gradient(src,dst); 336} 337 338// Either or both the source and the destination could be planar - the gradient code does not change 339void XGradientPlanarRGB8_RGB32( 340 const unsigned short* src_r, const unsigned short* src_g, const unsigned short* src_b, 341 ptrdiff_t src_row_bytes, int w, int h, 342 signed int* dst_pixels, ptrdiff_t dst_row_bytes) { 343 rgb16c_planar_view_t src=planar_rgb_view (w,h, src_r,src_g,src_b, src_row_bytes); 344 rgb32s_view_t dst=interleaved_view(w,h,(rgb32s_pixel_t*)dst_pixels,dst_row_bytes); 345 x_gradient(src,dst); 346} 347\endcode 348 349As these examples illustrate, both the source and the destination can be interleaved or planar, of any channel depth (assuming the destination channel is 350assignable to the source), and of any compatible color spaces. 351 352GIL 2.1 can also natively represent images whose channels are not byte-aligned, such as 6-bit RGB222 image or a 1-bit Gray1 image. 353GIL algorithms apply to these images natively. See the design guide or sample files for more on using such images. 354 355 356 357 358 359 360 361 362 363 364 365 366\subsection ImageViewTransformationSec Image View Transformations 367 368One way to compute the y-gradient is to rotate the image by 90 degrees, compute the x-gradient and rotate the result back. Here is how to do this in GIL: 369 370\code 371template <typename SrcView, typename DstView> 372void y_gradient(const SrcView& src, const DstView& dst) { 373 x_gradient(rotated90ccw_view(src), rotated90ccw_view(dst)); 374} 375\endcode 376 377\p rotated90ccw_view takes an image view and returns an image view representing 90-degrees counter-clockwise rotation of its input. It is an example of a GIL view transformation function. GIL provides 378a variety of transformation functions that can perform any axis-aligned rotation, transpose the view, flip it vertically or horizontally, extract a rectangular subimage, 379perform color conversion, subsample view, etc. The view transformation functions are fast and shallow - they don't copy the pixels, they just change the "coordinate system" of 380accessing the pixels. \p rotated90cw_view, for example, returns a view whose horizontal iterators are the vertical iterators of the original view. The above code to compute \p y_gradient 381is slow because of the memory access pattern; using \p rotated90cw_view does not make it any slower. 382 383Another example: suppose we want to compute the gradient of the N-th channel of a color image. Here is how to do that: 384 385\code 386template <typename SrcView, typename DstView> 387void nth_channel_x_gradient(const SrcView& src, int n, const DstView& dst) { 388 x_gradient(nth_channel_view(src, n), dst); 389} 390\endcode 391 392\p nth_channel_view is a view transformation function that takes any view and returns a single-channel (grayscale) view of its N-th channel. 393For interleaved RGB view, for example, the returned view is a step view - a view whose horizontal iterator skips over two channels when incremented. 394If applied on a planar RGB view, the returned type is a simple grayscale view whose horizontal iterator is a C pointer. 395Image view transformation functions can be piped together. For example, to compute the y gradient of the second channel of the even pixels in the view, use: 396 397\code 398y_gradient(subsampled_view(nth_channel_view(src, 1), 2,2), dst); 399\endcode 400 401GIL can sometimes simplify piped views. For example, two nested subsampled views (views that skip over pixels in X and in Y) can be represented as a single subsampled view whose step 402is the product of the steps of the two views. 403 404\subsection OneDIteratorsSec 1D pixel iterators 405 406Let's go back to \p x_gradient one more time. 407Many image view algorithms apply the same operation for each pixel and GIL provides an abstraction to handle them. However, our algorithm has an unusual access pattern, as it skips the 408first and the last column. It would be nice and instructional to see how we can rewrite it in canonical form. The way to do that in GIL is to write a version that works for every pixel, but 409apply it only on the subimage that excludes the first and last column: 410 411\code 412void x_gradient_unguarded(const gray8c_view_t& src, const gray8s_view_t& dst) { 413 for (int y=0; y<src.height(); ++y) { 414 gray8c_view_t::x_iterator src_it = src.row_begin(y); 415 gray8s_view_t::x_iterator dst_it = dst.row_begin(y); 416 417 for (int x=0; x<src.width(); ++x) 418 dst_it[x] = (src_it[x-1] - src_it[x+1]) / 2; 419 } 420} 421 422void x_gradient(const gray8c_view_t& src, const gray8s_view_t& dst) { 423 assert(src.width()>=2); 424 x_gradient_unguarded(subimage_view(src, 1, 0, src.width()-2, src.height()), 425 subimage_view(dst, 1, 0, src.width()-2, src.height())); 426} 427\endcode 428 429\p subimage_view is another example of a GIL view transformation function. It takes a source view and a rectangular region (in this case, defined as x_min,y_min,width,height) and 430returns a view operating on that region of the source view. The above implementation has no measurable performance degradation from the version that operates on the original views. 431 432Now that \p x_gradient_unguarded operates on every pixel, we can rewrite it more compactly: 433 434\code 435void x_gradient_unguarded(const gray8c_view_t& src, const gray8s_view_t& dst) { 436 gray8c_view_t::iterator src_it = src.begin(); 437 for (gray8s_view_t::iterator dst_it = dst.begin(); dst_it!=dst.end(); ++dst_it, ++src_it) 438 *dst_it = (src_it.x()[-1] - src_it.x()[1]) / 2; 439} 440\endcode 441 442GIL image views provide \p begin() and \p end() methods that return one dimensional pixel iterators which iterate over each pixel in the view, 443left to right and top to bottom. They do a proper "carriage return" - they skip any unused bytes at the end of a row. As such, they are slightly suboptimal, because they need to keep 444track of their current position with respect to the end of the row. Their increment operator performs one extra check (are we at the end of the row?), a check that is avoided if two 445nested loops are used instead. These iterators have a method \p x() which returns the more lightweight horizontal iterator that we used previously. Horizontal iterators have no 446notion of the end of rows. In this case, the horizontal iterators are raw C pointers. In our example, we must use the horizontal iterators to access the two neighbors properly, since they 447could reside outside the image view. 448 449\subsection STLEquivalentsSec STL Equivalent Algorithms 450 451GIL provides STL equivalents of many algorithms. For example, \p std::transform is an STL algorithm that sets each element in a destination range the result of a generic function taking the 452corresponding element of the source range. In our example, we want to assign to each destination pixel the value of the half-difference of the horizontal neighbors of the corresponding source pixel. 453If we abstract that operation in a function object, we can use GIL's \p transform_pixel_positions to do that: 454 455\code 456struct half_x_difference { 457 int operator()(const gray8c_loc_t& src_loc) const { 458 return (src_loc.x()[-1] - src_loc.x()[1]) / 2; 459 } 460}; 461 462void x_gradient_unguarded(const gray8c_view_t& src, const gray8s_view_t& dst) { 463 transform_pixel_positions(src, dst, half_x_difference()); 464} 465\endcode 466 467GIL provides the algorithms \p for_each_pixel and \p transform_pixels which are image view equivalents of STL's \p std::for_each and \p std::transform. It also provides 468\p for_each_pixel_position and \p transform_pixel_positions, which instead of references to pixels, pass to the generic function pixel locators. This allows for more powerful functions 469that can use the pixel neighbors through the passed locators. 470GIL algorithms iterate through the pixels using the more efficient two nested loops (as opposed to the single loop using 1-D iterators) 471 472\subsection ColorConversionSec Color Conversion 473 474Instead of computing the gradient of each color plane of an image, we often want to compute the gradient of the luminosity. In other words, we want to convert the 475color image to grayscale and compute the gradient of the result. Here how to compute the luminosity gradient of a 32-bit float RGB image: 476 477\code 478void x_gradient_rgb_luminosity(const rgb32fc_view_t& src, const gray8s_view_t& dst) { 479 x_gradient(color_converted_view<gray8_pixel_t>(src), dst); 480} 481\endcode 482 483\p color_converted_view is a GIL view transformation function that takes any image view and returns a view in a target color space and channel depth (specified 484as template parameters). In our example, it constructs an 8-bit integer grayscale view over 32-bit float RGB pixels. Like all other view transformation functions, \p color_converted_view is very 485fast and shallow. It doesn't copy the data or perform any color conversion. Instead it returns a view that performs color conversion every time its pixels are accessed. 486 487In the generic version of this algorithm we might like to convert the color space to grayscale, but keep the channel depth the same. We do that by constructing the 488type of a GIL grayscale pixel with the same channel as the source, and color convert to that pixel type: 489 490\code 491template <typename SrcView, typename DstView> 492void x_luminosity_gradient(const SrcView& src, const DstView& dst) { 493 typedef pixel<typename channel_type<SrcView>::type, gray_layout_t> gray_pixel_t; 494 x_gradient(color_converted_view<gray_pixel_t>(src), dst); 495} 496\endcode 497 498When the destination color space and channel type happens to be the same as the source one, color conversion is unnecessary. GIL detects this case and avoids calling the color conversion 499code at all - i.e. \p color_converted_view returns back the source view unchanged. 500 501 502\subsection ImagesSec Image 503 504The above example has a performance problem - \p x_gradient dereferences most source pixels twice, which will cause the above code to perform color conversion twice. 505Sometimes it may be more efficient to copy the color converted image into a temporary buffer and use it to compute the gradient - that way color conversion is invoked once per pixel. 506Using our non-generic version we can do it like this: 507 508\code 509void x_luminosity_gradient(const rgb32fc_view_t& src, const gray8s_view_t& dst) { 510 gray8_image_t ccv_image(src.dimensions()); 511 copy_pixels(color_converted_view<gray8_pixel_t>(src), view(ccv_image)); 512 513 x_gradient(const_view(ccv_image), dst); 514} 515\endcode 516 517First we construct an 8-bit grayscale image with the same dimensions as our source. Then we copy a color-converted view of the source into the temporary image. 518Finally we use a read-only view of the temporary image in our \p x_gradient algorithm. As the example shows, GIL provides global functions \p view and \p const_view 519that take an image and return a mutable or an immutable view of its pixels. 520 521Creating a generic version of the above is a bit trickier: 522 523\code 524template <typename SrcView, typename DstView> 525void x_luminosity_gradient(const SrcView& src, const DstView& dst) { 526 typedef typename channel_type<DstView>::type d_channel_t; 527 typedef typename channel_convert_to_unsigned<d_channel_t>::type channel_t; 528 typedef pixel<channel_t, gray_layout_t> gray_pixel_t; 529 typedef image<gray_pixel_t, false> gray_image_t; 530 531 gray_image_t ccv_image(src.dimensions()); 532 copy_pixels(color_converted_view<gray_pixel_t>(src), view(ccv_image)); 533 x_gradient(const_view(ccv_image), dst); 534} 535\endcode 536 537First we use the \p channel_type metafunction to get the channel type of the destination view. A metafunction is a function operating on types. In GIL metafunctions 538are structs which take their parameters as template parameters and return their result in a nested typedef called \p type. In this case, \p channel_type is 539a unary metafunction which in this example is called with the type of an image view and returns the type of the channel associated with that image view. 540 541GIL constructs that have an associated pixel type, such as pixels, pixel iterators, locators, views and images, all model \p PixelBasedConcept, which means 542that they provide a set of metafunctions to query the pixel properties, such as \p channel_type, \p color_space_type, \p channel_mapping_type, and \p num_channels. 543 544After we get the channel type of the destination view, we use another metafunction to remove its sign (if it is a signed integral type) and then use it 545to generate the type of a grayscale pixel. From the pixel type we create the image type. GIL's image class is templated over the pixel type and a boolean 546indicating whether the image should be planar or interleaved. 547Single-channel (grayscale) images in GIL must always be interleaved. There are multiple ways of constructing types in GIL. Instead of instantiating the classes 548directly we could have used type factory metafunctions. The following code is equivalent: 549 550\code 551template <typename SrcView, typename DstView> 552void x_luminosity_gradient(const SrcView& src, const DstView& dst) { 553 typedef typename channel_type<DstView>::type d_channel_t; 554 typedef typename channel_convert_to_unsigned<d_channel_t>::type channel_t; 555 typedef typename image_type<channel_t, gray_layout_t>::type gray_image_t; 556 typedef typename gray_image_t::value_type gray_pixel_t; 557 558 gray_image_t ccv_image(src.dimensions()); 559 copy_and_convert_pixels(src, view(ccv_image)); 560 x_gradient(const_view(ccv_image), dst); 561} 562\endcode 563 564GIL provides a set of metafunctions that generate GIL types - \p image_type is one such meta-function that constructs the type of an image from 565a given channel type, color layout, and planar/interleaved option (the default is interleaved). There are also similar meta-functions to 566construct the types of pixel references, iterators, locators and image views. GIL also has metafunctions \p derived_pixel_reference_type, \p derived_iterator_type, 567\p derived_view_type and \p derived_image_type that construct the type of a GIL construct from a given source one by changing one or more properties of 568the type and keeping the rest. 569 570From the image type we can use the nested typedef \p value_type to obtain the type of a pixel. GIL images, image views and locators have nested typedefs 571\p value_type and \p reference to obtain the type of the pixel and a reference to the pixel. If you have a pixel iterator, you can get these types from its 572\p iterator_traits. Note also the algorithm \p copy_and_convert_pixels, which is an abbreviated version of \p copy_pixels with a color converted source view. 573 574\subsection VirtualViewSec Virtual Image Views 575 576So far we have been dealing with images that have pixels stored in memory. GIL allows you to create an image view of an arbitrary image, including 577a synthetic function. To demonstrate this, let us create a view of the Mandelbrot set. 578First, we need to create a function object that computes the value of the Mandelbrot set at a given location (x,y) in the image: 579\code 580// models PixelDereferenceAdaptorConcept 581struct mandelbrot_fn { 582 typedef point2<ptrdiff_t> point_t; 583 584 typedef mandelbrot_fn const_t; 585 typedef gray8_pixel_t value_type; 586 typedef value_type reference; 587 typedef value_type const_reference; 588 typedef point_t argument_type; 589 typedef reference result_type; 590 BOOST_STATIC_CONSTANT(bool, is_mutable=false); 591 592 mandelbrot_fn() {} 593 mandelbrot_fn(const point_t& sz) : _img_size(sz) {} 594 595 result_type operator()(const point_t& p) const { 596 // normalize the coords to (-2..1, -1.5..1.5) 597 double t=get_num_iter(point2<double>(p.x/(double)_img_size.x*3-2, p.y/(double)_img_size.y*3-1.5f)); 598 return value_type((bits8)(pow(t,0.2)*255)); // raise to power suitable for viewing 599 } 600private: 601 point_t _img_size; 602 603 double get_num_iter(const point2<double>& p) const { 604 point2<double> Z(0,0); 605 for (int i=0; i<100; ++i) { // 100 iterations 606 Z = point2<double>(Z.x*Z.x - Z.y*Z.y + p.x, 2*Z.x*Z.y + p.y); 607 if (Z.x*Z.x + Z.y*Z.y > 4) 608 return i/(double)100; 609 } 610 return 0; 611 } 612}; 613\endcode 614 615We can now use GIL's \p virtual_2d_locator with this function object to construct a Mandelbrot view of size 200x200 pixels: 616\code 617typedef mandelbrot_fn::point_t point_t; 618typedef virtual_2d_locator<mandelbrot_fn,false> locator_t; 619typedef image_view<locator_t> my_virt_view_t; 620 621point_t dims(200,200); 622 623// Construct a Mandelbrot view with a locator, taking top-left corner (0,0) and step (1,1) 624my_virt_view_t mandel(dims, locator_t(point_t(0,0), point_t(1,1), mandelbrot_fn(dims))); 625\endcode 626 627We can treat the synthetic view just like a real one. For example, let's invoke our \p x_gradient algorithm to compute 628the gradient of the 90-degree rotated view of the Mandelbrot set and save the original and the result: 629 630\code 631gray8s_image_t img(dims); 632x_gradient(rotated90cw_view(mandel), view(img)); 633 634// Save the Mandelbrot set and its 90-degree rotated gradient (jpeg cannot save signed char; must convert to unsigned char) 635jpeg_write_view("mandel.jpg",mandel); 636jpeg_write_view("mandel_grad.jpg",color_converted_view<gray8_pixel_t>(const_view(img))); 637\endcode 638 639Here is what the two files look like: 640 641\image html mandel.jpg 642 643\subsection DynamicImageSec Run-Time Specified Images and Image Views 644 645So far we have created a generic function that computes the image gradient of a templated image view. 646Sometimes, however, the properties of an image view, such as its color space and channel depth, may not be available at compile time. 647GIL's \p dynamic_image extension allows for working with GIL constructs that are specified at run time, also called \e variants. GIL provides 648models of a run-time instantiated image, \p any_image, and a run-time instantiated image view, \p any_image_view. The mechanisms are in place to create 649other variants, such as \p any_pixel, \p any_pixel_iterator, etc. 650Most of GIL's algorithms and all of the view transformation functions also work with run-time instantiated image views and binary algorithms, such 651as \p copy_pixels can have either or both arguments be variants. 652 653Lets make our \p x_luminosity_gradient algorithm take a variant image view. For simplicity, let's assume that only the source view can be a variant. 654(As an example of using multiple variants, see GIL's image view algorithm overloads taking multiple variants.) 655 656First, we need to make a function object that contains the templated destination view and has an application operator taking a templated source view: 657 658\code 659#include <boost/gil/extension/dynamic_image/dynamic_image_all.hpp> 660 661template <typename DstView> 662struct x_gradient_obj { 663 typedef void result_type; // required typedef 664 665 const DstView& _dst; 666 x_gradient_obj(const DstView& dst) : _dst(dst) {} 667 668 template <typename SrcView> 669 void operator()(const SrcView& src) const { x_luminosity_gradient(src, _dst); } 670}; 671\endcode 672 673The second step is to provide an overload of \p x_luminosity_gradient that takes image view variant and calls GIL's \p apply_operation 674passing it the function object: 675 676\code 677template <typename SrcViews, typename DstView> 678void x_luminosity_gradient(const any_image_view<SrcViews>& src, const DstView& dst) { 679 apply_operation(src, x_gradient_obj<DstView>(dst)); 680} 681\endcode 682 683\p any_image_view<SrcViews> is the image view variant. It is templated over \p SrcViews, an enumeration of all possible view types the variant can take. 684\p src contains inside an index of the currently instantiated type, as well as a block of memory containing the instance. 685\p apply_operation goes through a switch statement over the index, each case of which casts the memory to the correct view type and invokes the 686function object with it. Invoking an algorithm on a variant has the overhead of one switch statement. Algorithms that perform an operation for 687each pixel in an image view have practically no performance degradation when used with a variant. 688 689Here is how we can construct a variant and invoke the algorithm: 690 691\code 692#include <boost/mpl/vector.hpp> 693#include <boost/gil/extension/io/jpeg_dynamic_io.hpp> 694 695typedef mpl::vector<gray8_image_t, gray16_image_t, rgb8_image_t, rgb16_image_t> my_img_types; 696any_image<my_img_types> runtime_image; 697jpeg_read_image("input.jpg", runtime_image); 698 699gray8s_image_t gradient(runtime_image.dimensions()); 700x_luminosity_gradient(const_view(runtime_image), view(gradient)); 701jpeg_write_view("x_gradient.jpg", color_converted_view<gray8_pixel_t>(const_view(gradient))); 702\endcode 703 704In this example, we create an image variant that could be 8-bit or 16-bit RGB or grayscale image. We then use GIL's I/O extension to load the image from file 705in its native color space and channel depth. If none of the allowed image types matches the image on disk, an exception will be thrown. 706We then construct a 8 bit signed (i.e. \p char) image to store the gradient and invoke \p x_gradient on it. Finally we save the result into another file. 707We save the view converted to 8-bit unsigned, because JPEG I/O does not support signed char. 708 709Note how free functions and methods such as \p jpeg_read_image, \p dimensions, \p view and \p const_view work on both templated and variant types. 710For templated images \p view(img) returns a templated view, whereas for image variants it returns a view variant. 711For example, the return type of \p view(runtime_image) is \p any_image_view<Views> where \p Views enumerates four views corresponding to the four image types. 712\p const_view(runtime_image) returns a \p any_image_view of the four read-only view types, etc. 713 714A warning about using variants: instantiating an algorithm with a variant effectively instantiates it with every possible type the variant can take. 715For binary algorithms, the algorithm is instantiated with every possible combination of the two input types! This can take a toll on both the compile time 716and the executable size. 717 718\section ConclusionSec Conclusion 719 720This tutorial provides a glimpse at the challenges associated with writing generic and efficient image processing algorithms in GIL. 721We have taken a simple algorithm and shown how to make it work with image representations that vary in bit depth, color space, ordering of the 722channels, and planar/interleaved structure. We have demonstrated that the algorithm can work with fully abstracted virtual images, and even images 723whose type is specified at run time. The associated video presentation also demonstrates that even for complex scenarios the generated assembly 724is comparable to that of a C version of the algorithm, hand-written for the specific image types. 725 726Yet, even for such a simple algorithm, we are far from making a fully generic and optimized code. In particular, the presented algorithms work on homogeneous 727images, i.e. images whose pixels have channels that are all of the same type. There are examples of images, such as a packed 565 RGB format, which contain 728channels of different types. While GIL provides concepts and algorithms operating on heterogeneous pixels, we leave the task of extending x_gradient as an 729exercise for the reader. 730Second, after computing the value of the gradient we are simply casting it to the destination channel type. This may not always be the desired operation. For 731example, if the source channel is a float with range [0..1] and the destination is unsigned char, casting the half-difference to unsigned char will result in 732either 0 or 1. Instead, what we might want to do is scale the result into the range of the destination channel. GIL's channel-level algorithms might be useful 733in such cases. For example, \p channel_convert converts between channels by linearly scaling the source channel value into the range of the destination channel. 734 735There is a lot to be done in improving the performance as well. Channel-level operations, such as the half-difference, could be abstracted out into atomic 736channel-level algorithms and performance overloads could be provided for concrete channel types. Processor-specific operations could be used, for example, 737to perform the operation over an entire row of pixels simultaneously, or the data could be prefetched. All of these optimizations can be realized as performance 738specializations of the generic algorithm. Finally, compilers, while getting better over time, are still failing to fully optimize generic code in some cases, such 739as failing to inline some functions or put some variables into registers. If performance is an issue, it might be worth trying your code with different compilers. 740 741\section AppendixSec Appendix 742 743\subsection AppendixConventionSec Naming convention for GIL concrete types 744 745Concrete (non-generic) GIL types follow this naming convention: 746 747<p> 748\e ColorSpace + \e BitDepth + [\p f | \p s]+ [\p c] + [\p _planar] + [\p _step] + \e ClassType + \p _t 749<p> 750 751Where \e ColorSpace also indicates the ordering of components. Examples are \p rgb, \p bgr, \p cmyk, \p rgba. 752\e BitDepth indicates the bit depth of the color channel. Examples are \p 8,\p 16,\p 32. By default the type of channel is unsigned integral; using \p s indicates 753signed integral and \p f - a floating point type, which is always signed. \p c indicates object operating over immutable pixels. \p _planar indicates planar organization 754(as opposed to interleaved). \p _step indicates special image views, 755locators and iterators which traverse the data in non-trivial way (for example, backwards or every other pixel). 756\e ClassType is \p _image (image), \p _view (image view), \p _loc (pixel 2D locator) \p _ptr (pixel iterator), \p _ref (pixel reference), 757\p _pixel (pixel value). 758 759\code 760bgr8_image_t a; // 8-bit interleaved BGR image 761cmyk16_pixel_t; b; // 16-bit CMYK pixel value; 762cmyk16c_planar_ref_t c(b); // const reference to a 16-bit planar CMYK pixel x. 763rgb32f_planar_step_ptr_t d; // step pointer to a 32-bit planar RGB pixel. 764\endcode 765 766<div id="footerrow"><!--give footer 25px of white above--></div><div id="footer" title="footer: links to copyright and other legal information"><p><a href="licenses.html" class="el">Copyright © 2005 Adobe Systems Incorporated</a></p><ul id="list1"><!-- due to a rendering error in IE, these links should all be on one line without returns --><li id="terms"><a title="Terms of Use" href="http://www.adobe.com/misc/copyright.html">Terms of Use</a></li><li><a title="Privacy Policy" href="http://www.adobe.com/misc/privacy.html">Privacy Policy</a></li><li><a href="http://access.adobe.com">Accessibility</a></li><li><a title="Avoid software piracy" href="http://www.adobe.com/aboutadobe/antipiracy/main.html">Avoid software piracy</a></li><li id="tms"><a title="Permissions and trademarks" href="http://www.adobe.com/misc/agreement.html">Permissions and trademarks</a></li><li><a title="Product License Agreements" href="http://www.adobe.com/products/eulas/main.html">Product License Agreements</a></li></ul></div> 767 768*/ 769