1.. _rfc-51: 2 3======================================================================================= 4RFC 51: RasterIO() improvements : resampling and progress callback 5======================================================================================= 6 7Author: Even Rouault 8 9Contact: even dot rouault at spatialys dot com 10 11Status: Adopted, implemented in GDAL 2.0 12 13Summary 14------- 15 16This RFC aims at extending the RasterIO() API to allow specifying a 17resampling algorithm when doing requests involving subsampling or 18oversampling. A progress callback can also be specified to be notified 19of progression and allow the user to interrupt the operation. 20 21Core changes 22------------ 23 24Addition of GDALRasterIOExtraArg structure 25~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 26 27A new structure GDALRasterIOExtraArg is added to contain the new 28options. 29 30:: 31 32 /** Structure to pass extra arguments to RasterIO() method 33 * @since GDAL 2.0 34 */ 35 typedef struct 36 { 37 /*! Version of structure (to allow future extensions of the structure) */ 38 int nVersion; 39 40 /*! Resampling algorithm */ 41 GDALRIOResampleAlg eResampleAlg; 42 43 /*! Progress callback */ 44 GDALProgressFunc pfnProgress; 45 /*! Progress callback user data */ 46 void *pProgressData; 47 48 /*! Indicate if dfXOff, dfYOff, dfXSize and dfYSize are set. 49 Mostly reserved from the VRT driver to communicate a more precise 50 source window. Must be such that dfXOff - nXOff < 1.0 and 51 dfYOff - nYOff < 1.0 and nXSize - dfXSize < 1.0 and nYSize - dfYSize < 1.0 */ 52 int bFloatingPointWindowValidity; 53 /*! Pixel offset to the top left corner. Only valid if bFloatingPointWindowValidity = TRUE */ 54 double dfXOff; 55 /*! Line offset to the top left corner. Only valid if bFloatingPointWindowValidity = TRUE */ 56 double dfYOff; 57 /*! Width in pixels of the area of interest. Only valid if bFloatingPointWindowValidity = TRUE */ 58 double dfXSize; 59 /*! Height in pixels of the area of interest. Only valid if bFloatingPointWindowValidity = TRUE */ 60 double dfYSize; 61 } GDALRasterIOExtraArg; 62 63 #define RASTERIO_EXTRA_ARG_CURRENT_VERSION 1 64 65 /** Macro to initialize an instance of GDALRasterIOExtraArg structure. 66 * @since GDAL 2.0 67 */ 68 #define INIT_RASTERIO_EXTRA_ARG(s) \ 69 do { (s).nVersion = RASTERIO_EXTRA_ARG_CURRENT_VERSION; \ 70 (s).eResampleAlg = GRIORA_NearestNeighbour; \ 71 (s).pfnProgress = NULL; \ 72 (s).pProgressData = NULL; \ 73 (s).bFloatingPointWindowValidity = FALSE; } while(0) 74 75There are several reasons to prefer a structure rather than new 76parameters to the RasterIO() methods : 77 78- code readability (GDALDataset::IRasterIO() has already 14 79 parameters...) 80- allow future extensions without changing the prototype in all drivers 81- to a lesser extent, efficiency: it is common for RasterIO() calls to 82 be chained between generic/specific and/or dataset/rasterband 83 implementations. Passing just the pointer is more efficient. 84 85The structure is versioned. In the future if further options are added, 86the new members will be added at the end of the structure and the 87version number will be incremented. Code in GDAL core&drivers can check 88the version number to determine which options are available. 89 90Addition of GDALRIOResampleAlg structure 91~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 92 93The following resampling algorithms are available : 94 95:: 96 97 /** RasterIO() resampling method. 98 * @since GDAL 2.0 99 */ 100 typedef enum 101 { 102 /*! Nearest neighbour */ GRIORA_NearestNeighbour = 0, 103 /*! Bilinear (2x2 kernel) */ GRIORA_Bilinear = 1, 104 /*! Cubic Convolution Approximation (4x4 kernel) */ GRIORA_Cubic = 2, 105 /*! Cubic B-Spline Approximation (4x4 kernel) */ GRIORA_CubicSpline = 3, 106 /*! Lanczos windowed sinc interpolation (6x6 kernel) */ GRIORA_Lanczos = 4, 107 /*! Average */ GRIORA_Average = 5, 108 /*! Mode (selects the value which appears most often of all the sampled points) */ 109 GRIORA_Mode = 6, 110 /*! Gauss blurring */ GRIORA_Gauss = 7 111 } GDALRIOResampleAlg; 112 113Those new resampling methods can be used by the 114GDALRasterBand::IRasterIO() default implementation when the size of the 115buffer (nBufXSize x nBufYSize) is different from the size of the area of 116interest (nXSize x nYSize). The code heavily relies on the algorithms 117used for overview computation, with adjustments to be also able to deal 118with oversampling. Bilinear, CubicSpline and Lanczos are now available 119in overview computation as well, and rely on the generic infrastructure 120for convolution computation introduced lately for improved cubic 121overviews. Some algorithms are not available on raster bands with color 122palette. A warning will be emitted if an attempt of doing so is done, 123and nearest neighbour will be used as a fallback. 124 125The GDAL_RASTERIO_RESAMPLING configuration option can be set as an 126alternate way of specifying the resampling algorithm. Mainly useful for 127tests with applications that do not yet use the new API. 128 129Currently, the new resampling methods are only available for GF_Read 130operations. The use case for GF_Write operations isn't obvious, but 131could be added without API changes if needed. 132 133C++ changes 134~~~~~~~~~~~ 135 136GDALDataset and GDALRasterBand (non virtual) RasterIO() and (virtual) 137IRasterIO() methods have a new final argument psExtraArg of type 138GDALRasterIOExtraArg*. This extra argument defaults to NULL for code 139using GDAL, but is required for all in-tree code, so as to avoid that 140in-tree code forgets to forwards psExtraArg it might have returned from 141a caller. 142 143GDALDataset::RasterIO() and GDALRasterBand::RasterIO() can accept a NULL 144pointer for that argument in which case they will instantiate a default 145GDALRasterIOExtraArg structure to be passed to IRasterIO(). Any other 146code that calls IRasterIO() directly (a few IReadBlock() 147implementations) should make sure of doing so, so that IRasterIO() can 148assume that its psExtraArg is not NULL. 149 150As a provision to be able to deal with very large requests with buffers 151larger than several gigabytes, the nPixelSpace, nLineSpace and 152nBandSpace parameters have been promoted from the int datatype to the 153new GSpacing datatype, which is an alias of a signed 64 bit integer. 154 155GDALRasterBand::IRasterIO() and GDALDataset::BlockBasedRasterIO() now 156use the progress callback when available. 157 158C API changes 159~~~~~~~~~~~~~ 160 161Only additions : 162 163:: 164 165 CPLErr CPL_DLL CPL_STDCALL GDALDatasetRasterIOEx( 166 GDALDatasetH hDS, GDALRWFlag eRWFlag, 167 int nDSXOff, int nDSYOff, int nDSXSize, int nDSYSize, 168 void * pBuffer, int nBXSize, int nBYSize, GDALDataType eBDataType, 169 int nBandCount, int *panBandCount, 170 GSpacing nPixelSpace, GSpacing nLineSpace, GSpacing nBandSpace, 171 GDALRasterIOExtraArg* psExtraArg); 172 173 CPLErr CPL_DLL CPL_STDCALL 174 GDALRasterIOEx( GDALRasterBandH hRBand, GDALRWFlag eRWFlag, 175 int nDSXOff, int nDSYOff, int nDSXSize, int nDSYSize, 176 void * pBuffer, int nBXSize, int nBYSize,GDALDataType eBDataType, 177 GSpacing nPixelSpace, GSpacing nLineSpace, 178 GDALRasterIOExtraArg* psExtraArg ); 179 180Those are the same as the existing functions with a final 181GDALRasterIOExtraArg\* psExtraArg argument, and the spacing parameters 182promoted to GSpacing. 183 184Changes in drivers 185------------------ 186 187- All in-tree drivers that implemented or used RasterIO have been 188 edited to accept the GDALRasterIOExtraArg\* psExtraArg parameter, and 189 forward it when needed. Those who had a custom RasterIO() 190 implementation now use the progress callback when available. 191- VRT: the and elements can accept a 'resampling' attribute. The VRT 192 driver will also set the dfXOff, dfYOff, dfXSize and dfYSize fields 193 of GDALRasterIOExtraArg\* to have source sub-pixel accuracy, so that 194 GDALRasterBand::IRasterIO() leads to consistent results when 195 operating on a small area of interest or the whole raster. If that 196 was not done, chunking done in GDALDatasetCopyWholeRaster() or other 197 algorithms could lead to repeated lines due to integer rounding 198 issues. 199 200Changes in utilities 201-------------------- 202 203- gdal_translate: accept a -r parameter to specify the resampling 204 algorithm. Defaults to NEAR. Can be set to bilinear, cubic, 205 cubicspline, lanczos, average or mode. (Under the hood, this sets the 206 new resampling property at the VRT source level.) 207- gdaladdo: -r parameter now accepts bilinear, cubicspline and lanczos 208 as additional algorithms to the existing ones. 209 210Changes in SWIG bindings 211------------------------ 212 213- For Python and Perl bindings: Band.ReadRaster(), Dataset.ReadRaster() 214 now accept optional resample_alg, callback and callback_data 215 arguments. (untested for Perl, but the existing tests pass) 216- For Python bindings, Band.ReadAsArray() and Dataset.ReadAsArray() now 217 accept optional resample_alg, callback and callback_data arguments. 218 219Compatibility 220------------- 221 222- C API/ABI preserved. 223 224- C++ users of the GDALRasterBand::RasterIO() and 225 GDALDataset::RasterIO() API do not need to change their code, since 226 the new GDALRasterIOExtraArg\* psExtraArg argument is optional for 227 out-of-tree code. 228 229- Out-of-tree drivers that implement IRasterIO() must be changed to 230 accept the new GDALRasterIOExtraArg\* psExtraArg argument. Note: 231 failing to do so will be undetected at compile time (due to how C++ 232 virtual method overloading work). 233 234Both issues will be mentioned in MIGRATION_GUIDE.TXT 235 236Documentation 237------------- 238 239All new methods are documented. 240 241Testing 242------- 243 244The various aspects of this RFC are tested in the Python bindings: 245 246- use of the new options of Band.ReadRaster(), Dataset.ReadRaster(), 247 Band.ReadAsArray() and Dataset.ReadAsArray(). 248- resampling algorithms in subsampling and oversampling RasterIO() 249 requests. 250- "-r" option of gdal_translate 251 252Implementation 253-------------- 254 255Implementation will be done by Even Rouault 256(`Spatialys <http://spatialys.com>`__), and sponsored by `R3 257GIS <http://r3-gis.com>`__. 258 259The proposed implementation lies in the "rasterio" branch of the 260`https://github.com/rouault/gdal2/tree/rasterio <https://github.com/rouault/gdal2/tree/rasterio>`__ 261repository. 262 263The list of changes : 264`https://github.com/rouault/gdal2/compare/rasterio <https://github.com/rouault/gdal2/compare/rasterio>`__ 265 266Voting history 267-------------- 268 269+1 from FrankW, JukkaR, HowardB, DanielM, TamasS and EvenR 270