1.. _rfc-46: 2 3======================================================================================= 4RFC 46: GDAL/OGR unification 5======================================================================================= 6 7Author: Even Rouault 8 9Contact: even dot rouault at spatialys.com 10 11Status: Adopted, implemented in GDAL 2.0 12 13Summary 14------- 15 16In the 1.X series of GDAL/OGR, the GDAL/raster and OGR/vector sides are 17quite different on some aspects even where there is no strong reason for 18them to be different, particularly in the structure of drivers. This RFC 19aims at unifying the OGR driver structure with the GDAL driver 20structure. The main advantages of using the GDAL driver structure are : 21 22- metadata capabilities : description of driver, extensions, creation 23 options, virtual IO capability ... 24- efficient driver identification and opening. 25 26Similarly, OGR datasource and layer classes lack the metadata mechanisms 27offered by the corresponding GDAL dataset and raster band classes. 28 29Another aspect is that the separation between GDAL "datasets" and OGR 30"datasources" is sometimes artificial. Various data containers can 31accept both data types. The list of drivers that have a GDAL side and 32OGR side is : SDTS, PDS, GRASS, KML, Spatialite/Rasterlite, GeoPackage 33(raster side not yet implemented), PostGIS/PostGIS Raster, PDF, PCIDSK, 34FileGDB (raster side not yet implemented). For applications that are 35interested in both, this currently means to open the file twice with 36different API. And for update mode, for file-based drivers, the updates 37must be done sequentially to avoid opening a file twice simultaneously 38in update mode and making conflicting changes. 39 40Related RFCs 41------------ 42 43There are a few related past RFCs that have never been adopted but 44strongly relate to RFC 46 : 45 46- `RFC 10: OGR Open Parameters <./rfc10_ogropen>`__. All the 47 functionality described in RFC 10 is included in RFC 46, mainly the 48 GDALOpenEx() new API 49- `RFC 25: Fast Open <./rfc25_fast_open>`__. RFC 25 mentions avoiding 50 to systematically listing the sibling files to the file being opened. 51 This can now achieved in RFC 46 by lazy loading with 52 GDALOpenInfo::GetSiblingFiles(). At least Identify() should not 53 trigger GetSiblingFiles(). 54- `RFC 36: Allow specification of intended on 55 GDALOpen <./rfc36_open_by_drivername>`__. The new GDALOpenEx() 56 accepts a list of a subset drivers that must be probed, as suggested 57 by RFC36. The specification of the drivers on the command line of 58 utilities could be easily done through a new option, but that's not in 59 the scope of RFC 46. 60- `RFC 38: OGR Faster Open <./rfc38_ogr_faster_open>`__ is completely 61 included in RFC 46 through the possibility of using 62 Open(GDALOpenInfo*) in OGR drivers 63 64Self-assigned development constraints 65------------------------------------- 66 67The changes should have moderate impact on the existing GDAL/OGR code 68base, and particularly on most of its code, that lies in drivers. 69Existing users of the GDAL/OGR API should also be moderately impacted by 70the changes, if they do not need to use the new offered capabilities. 71 72Core changes: summary 73--------------------- 74 75- OGRSFDriver extends GDALDriver. 76- Vector drivers can be implemented as GDALDriver. 77- OGRSFDriverRegistrar is a compatibility wrapper around 78 GDALDriverManager for legacy OGRSFDriver. 79- OGRDataSource extends GDALDataSource. 80- GDALOpenEx() API is added to be able to open "mixed" datasets. 81- OGRLayer extends GDALMajorObject, thus adding metadata capability. 82- The methods of OGRDataSource related to layers are moved to 83 GDALDataset, making it both a raster and vector capable container. 84- Performance improvements in GDALOpenInfo() mechanism. 85- New driver metadata item to describe open options (i.e. deprecate the 86 use of configuration option). 87- New driver metadata item to describe layer creation options. 88 89Core changes: details 90--------------------- 91 92Drivers and driver registration 93~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 94 95- The OGRSFDriver now extends GDALDriver and is meant as being a legacy 96 way of implementing a vector driver. It is kept mainbly because, in 97 the current implementation, not all drivers have been migrated to 98 being "pure" GDALDriver. The CopyDataSource() virtual method has been 99 removed since no in-tree drivers implement it. The inheritance to 100 GDALDriver make it possible to manage vector drivers by the 101 GDALDriverManager, and to be able to attach metadata to them, to 102 document driver long name, link to documentation, file extension, 103 datasource creation options with the existing GDAL\_DMD\_\* metadata 104 items. 105 106- Drivers directly inheriting from GDALDriver (to be opposed to those 107 inheriting from OGRSFDriver) should : - declare SetMetadataItem( 108 GDAL_DCAP_VECTOR, "YES" ). - implement pfnOpen() for dataset opening 109 - optionally, implement pfnCreate() for dataset creation. For vector 110 drivers, the nBands parameter of Create() is supposed to be passed to 111 0. - optionally, implement pfnDelete() for dataset deletion 112 113- The *C* OGR Driver API will still work with drivers that have been 114 converted as "pure" GDALDrivers (this is not true of the C++ OGR 115 Driver API). For example OGR_Dr_GetName() calls 116 GDALDriver::GetDescription(), OGR_Dr_CreateDatasource() calls 117 GDALDriver::Create(), etc... 118 119- The C++ definition of GDALDriver is extended with the following 120 function pointers so that it can work with legacy OGRSFDriver. 121 122:: 123 124 /* For legacy OGR drivers */ 125 GDALDataset *(*pfnOpenWithDriverArg)( GDALDriver*, GDALOpenInfo * ); 126 GDALDataset *(*pfnCreateVectorOnly)( GDALDriver*, 127 const char * pszName, 128 char ** papszOptions ); 129 CPLErr (*pfnDeleteDataSource)( GDALDriver*, 130 const char * pszName ); 131 132:: 133 134 They are used by GDALOpenEx(), GDALDriver::Create() and GDALDriver::Delete() 135 if the pfnOpen, pfnCreate or pfnDelete pointers are NULL. The OGRSFDriverRegistrar 136 class has an implementation of those function pointers that calls the 137 legacy C++ OGRSFDriver::Open(), OGRSFDriver::CreateDataSource() and 138 OGRSFDriver::DeleteDataSource() virtual methods. 139 140- GDALDriver::Create() can accept nBands == 0 for a vector capable 141 driver. 142 143- GDALDriver::DefaultCreateCopy() can accept a dataset with 0 bands for 144 a vector capable driver, and if the output dataset has layer creation 145 capability and the source dataset has layers, it copies the layers 146 from the source dataset into the target dataset. 147 148- GDALDriver::Identify() now iterates over all kinds of drivers. It has 149 been modified to do a first pass on drivers that have an 150 implementation of Identify(). If no match is found, it does a second 151 pass on all drivers and use the potentially slower Open() as the 152 identification method. 153 154- Related to the above point, the implementations of 155 GDALDriver::pfnIdentify function pointer used to return a boolean 156 value to indicate if the passed GDALOpenInfo was a match for the 157 driver. For some drivers, this was too restrictive so that they were 158 able to implement Identify(). For example where the detection logic 159 can return "yes, I definitely recognize that file", "no, it is not 160 for me" or "I have not enough elements in GDALOpenInfo to be able to 161 tell". That last state can now be advertized with a negative return 162 value. 163 164- The OGRSFDriverRegistrar is trimmed down to be mostly a wrapper 165 around GDALDriverManager. In particular, it does not contain any 166 longer a list of drivers. The Open(), OpenShared(), 167 ReleaseDataSource(), DeregisterDriver() and AutoLoadDrivers() methods 168 are removed from the class. This change can have impact on C++ code. 169 A few adaptations in OGR utilities have been done to accommodate for 170 those changes. The RegisterDriver() API has been kept for legacy OGR 171 drivers and it automatically sets SetMetadataItem( GDAL_DCAP_VECTOR, 172 "YES" ). The GetDriverCount(), GetDriver() and GetDriverByName() 173 methods delegate to GDALDriverManager and make sure to only take into 174 account drivers that have the GDAL_DCAP_VECTOR capability. In the 175 case a driver has the same name as GDAL and OGR driver, the OGR 176 variant is internally prefixed with OGR\_, and GetDriverByName() will 177 first try the OGR\_ variant. The GetOpenDSCount() and GetOpenDS() 178 have now a dummy implementation returning 0/NULL. For reference, 179 neither MapServer nor QGIS use those functions. 180 181- OGRRegisterAll() is now an alias of GDALAllRegister(). The past 182 OGRRegisterAll() is now renamed OGRRegisterAllInternal() and called 183 by GDALAllRegister(). So, GDALAllRegister() and OGRRegisterAll() are 184 now equivalent and register all drivers. 185 186- GDALDriverManager has received a few changes : 187 188 - use of a map from driver name to driver object to speed-up 189 GetDriverByName() 190 - accept OGR_SKIP and OGR_DRIVER_PATH configuration options for 191 backward compatibility. 192 - The recommended separator for driver names in GDAL_SKIP is now 193 comma instead of space (similarly to what OGR_SKIP does). This is 194 to make it possible to define OGR driver names in GDAL_SKIP that 195 have spaces in their names like "ESRI Shapefile" or "MapInfo 196 File". If there is no comma in the GDAL_SKIP value, then space 197 separator is assumed (backward compatibility). 198 - removal of GetHome()/SetHome() methods whose purpose seemed to 199 define an alternate path for the search directory of plugins. 200 Those methods only existed at the C++ level, and are redundant 201 with GDAL_DRIVER_PATH configuration option 202 203- Raster-capable drivers should declare SetMetadataItem( 204 GDAL_DCAP_RASTER, "YES" ). All in-tree GDAL drivers have been patched 205 to declare it. But the registration code detects if a driver does not 206 declare any of GDAL_DCAP_RASTER nor GDAL_DCAP_VECTOR, in which case 207 it declares GDAL_DCAP_RASTER on behalf of the un-patched driver, with 208 a debug message inviting to explicitly set it. 209 210- New metadata items : 211 212 - GDAL_DCAP_RASTER=YES / GDAL_DCAP_VECTOR=YES at driver level. To 213 declare that a driver has raster/vector capabilities. A driver can 214 declare both. 215 - GDAL_DMD_EXTENSIONS (with a final S) at driver level. This is a 216 small evolution of GDAL_DMD_EXTENSION where one can specify 217 several extensions in the value string. The extensions are 218 space-separated. For example "shp dbf", "tab mif mid", etc... For 219 ease of use, GDALDriver::SetMetadataItem(GDAL_DMD_EXTENSION) also 220 sets the passed value as GDAL_DMD_EXTENSIONS, if it is not already 221 set. So new code can always use GDAL_DMD_EXTENSIONS. 222 - GDAL_DMD_OPENOPTIONLIST at driver level. The value of this item is 223 an XML snippet with a format similar to creation options. 224 GDALOpenEx(), once it has identified with Identify() that a driver 225 accepts the file, will validate the passed open option list with 226 the authorized open option list. Below an example of such an 227 authorized open option list in the S57 driver 228 229:: 230 231 <OpenOptionList> 232 <Option name="UPDATES" type="string-select" 233 description="Should update files be incorporated into the base data on the fly" default="APPLY"> 234 <Value>APPLY</Value> 235 <Value>IGNORE</Value> 236 </Option> 237 <Option name="SPLIT_MULTIPOINT" type="boolean" 238 description="Should multipoint soundings be split into many single point " 239 "sounding features" default="NO" /> 240 <Option name="ADD_SOUNDG_DEPTH" type="boolean" 241 description="Should a DEPTH attribute be added on SOUNDG features and " 242 "assign the depth of the sounding" default="NO" /> 243 <Option name="RETURN_PRIMITIVES" type="boolean" 244 description="Should all the low level geometry primitives be returned as " 245 "special IsolatedNode, ConnectedNode, Edge and Face layers" default="NO" /> 246 <Option name="PRESERVE_EMPTY_NUMBERS" type="boolean" 247 description="If enabled, numeric attributes assigned an empty string as a " 248 "value will be preserved as a special numeric value" default="NO" /> 249 <Option name="LNAM_REFS" type="boolean" 250 description="Should LNAM and LNAM_REFS fields be attached to features " 251 "capturing the feature to feature relationships in the FFPT " 252 "group of the S-57 file" default="YES" /> 253 <Option name="RETURN_LINKAGES" type="boolean" 254 description="Should additional attributes relating features to their underlying " 255 "geometric primtives be attached" default="NO" /> 256 <Option name="RECODE_BY_DSSI" type="boolean" 257 description="Should attribute values be recoded to UTF-8 from the character " 258 "encoding specified in the S57 DSSI record." default="NO" /> 259 </OpenOptionList> 260 261:: 262 263 - GDAL_DS_LAYER_CREATIONOPTIONLIST at dataset level. But can also be set at 264 driver level because, in practice, layer creation options do not depend on the 265 dataset instance. 266 The value of this item is an XML snippet with a format similar to dataset creation 267 options. 268 If specified, the passed creation options to CreateLayer() are validated 269 against that authorized creation option list. 270 Below an example of such an authorized open option list in the Shapefile driver. 271 272:: 273 274 <LayerCreationOptionList> 275 <Option name="SHPT" type="string-select" description="type of shape" default="automatically detected"> 276 <Value>POINT</Value> 277 <Value>ARC</Value> 278 <Value>POLYGON</Value> 279 <Value>MULTIPOINT</Value> 280 <Value>POINTZ</Value> 281 <Value>ARCZ</Value> 282 <Value>POLYGONZ</Value> 283 <Value>MULTIPOINTZ</Value> 284 <Value>NONE</Value> 285 <Value>NULL</Value> 286 </Option> 287 <Option name="2GB_LIMIT" type="boolean" description="Restrict .shp and .dbf to 2GB" default="NO" /> 288 <Option name="ENCODING" type="string" description="DBF encoding" default="LDID/87" /> 289 <Option name="RESIZE" type="boolean" description="To resize fields to their optimal size." default="NO" /> 290 </LayerCreationOptionList> 291 292.. _datasets--datasources: 293 294Datasets / Datasources 295~~~~~~~~~~~~~~~~~~~~~~ 296 297- The main methods from OGRDataSource have been moved to GDALDataset : 298 299:: 300 301 virtual int GetLayerCount() { return 0; } 302 virtual OGRLayer *GetLayer(int) { return NULL; } 303 virtual OGRLayer *GetLayerByName(const char *); 304 virtual OGRErr DeleteLayer(int); 305 306 virtual int TestCapability( const char * ) { return FALSE; } 307 308 virtual OGRLayer *CreateLayer( const char *pszName, 309 OGRSpatialReference *poSpatialRef = NULL, 310 OGRwkbGeometryType eGType = wkbUnknown, 311 char ** papszOptions = NULL ); 312 virtual OGRLayer *CopyLayer( OGRLayer *poSrcLayer, 313 const char *pszNewName, 314 char **papszOptions = NULL ); 315 316 virtual OGRStyleTable *GetStyleTable(); 317 virtual void SetStyleTableDirectly( OGRStyleTable *poStyleTable ); 318 319 virtual void SetStyleTable(OGRStyleTable *poStyleTable); 320 321 virtual OGRLayer * ExecuteSQL( const char *pszStatement, 322 OGRGeometry *poSpatialFilter, 323 const char *pszDialect ); 324 virtual void ReleaseResultSet( OGRLayer * poResultsSet ); 325 326 int GetRefCount() const; 327 int GetSummaryRefCount() const; 328 OGRErr Release(); 329 330:: 331 332 The following matching C API is available : 333 334:: 335 336 int CPL_DLL GDALDatasetGetLayerCount( GDALDatasetH ); 337 OGRLayerH CPL_DLL GDALDatasetGetLayer( GDALDatasetH, int ); 338 OGRLayerH CPL_DLL GDALDatasetGetLayerByName( GDALDatasetH, const char * ); 339 OGRErr CPL_DLL GDALDatasetDeleteLayer( GDALDatasetH, int ); 340 OGRLayerH CPL_DLL GDALDatasetCreateLayer( GDALDatasetH, const char *, 341 OGRSpatialReferenceH, OGRwkbGeometryType, 342 char ** ); 343 OGRLayerH CPL_DLL GDALDatasetCopyLayer( GDALDatasetH, OGRLayerH, const char *, 344 char ** ); 345 int CPL_DLL GDALDatasetTestCapability( GDALDatasetH, const char * ); 346 OGRLayerH CPL_DLL GDALDatasetExecuteSQL( GDALDatasetH, const char *, 347 OGRGeometryH, const char * ); 348 void CPL_DLL GDALDatasetReleaseResultSet( GDALDatasetH, OGRLayerH ); 349 OGRStyleTableH CPL_DLL GDALDatasetGetStyleTable( GDALDatasetH ); 350 void CPL_DLL GDALDatasetSetStyleTableDirectly( GDALDatasetH, OGRStyleTableH ); 351 void CPL_DLL GDALDatasetSetStyleTable( GDALDatasetH, OGRStyleTableH ); 352 353:: 354 355 OGRDataSource definition is now reduced to : 356 357:: 358 359 class CPL_DLL OGRDataSource : public GDALDataset 360 { 361 public: 362 OGRDataSource(); 363 364 virtual const char *GetName() = 0; 365 366 static void DestroyDataSource( OGRDataSource * ); 367 }; 368 369:: 370 371 The existing OGR_DS_* API is preserved. The implementation of those functions 372 casts the OGRDataSourceH opaque pointer to GDALDataset*, so it is possible to 373 consider GDALDatasetH and OGRDataSourceH as equivalent from the C API point of 374 view. Note that it is not true at the C++ level ! 375 376- OGRDataSource::SyncToDisk() has been removed. The equivalent 377 functionality should be implemented in existing FlushCache(). 378 GDALDataset::FlushCache() nows does the job of the previous generic 379 implementation of OGRDataSource::SyncToDisk(), i.e. iterate over all 380 layers and call SyncToDisk() on them. 381 382- GDALDataset has now a protected ICreateLayer() method. 383 384:: 385 386 virtual OGRLayer *ICreateLayer( const char *pszName, 387 OGRSpatialReference *poSpatialRef = NULL, 388 OGRwkbGeometryType eGType = wkbUnknown, 389 char ** papszOptions = NULL ); 390 391:: 392 393 This method is what used to be CreateLayer(), i.e. that drivers should 394 rename their specialized CreateLayer() implementations as ICreateLayer(). 395 CreateLayer() is kept at GDALDataset level, but its implementation does a 396 prior validation of passed creation options against an optional authorized 397 creation option list (GDAL_DS_LAYER_CREATIONOPTIONLIST), before calling 398 ICreateLayer() (this is similar to RasterIO() / IRasterIO() ) 399 A global pass on all in-tree OGR drivers has been made to rename CreateLayer() 400 as ICreateLayer(). 401 402- GDALOpenEx() is added to be able to open raster-only, vector-only, or 403 raster-vector datasets. It accepts read-only/update mode, 404 shared/non-shared mode. A list of potential candidate drivers can be 405 passed. If NULL, all drivers are probed. A list of open options 406 (NAME=VALUE syntax) can be passed. If the list of sibling files has 407 already been established, it can also be passed. Otherwise 408 GDALOpenInfo will establish it. 409 410:: 411 412 GDALDatasetH CPL_STDCALL GDALOpenEx( const char* pszFilename, 413 unsigned int nOpenFlags, 414 const char* const* papszAllowedDrivers, 415 const char* const* papszOpenOptions, 416 const char* const* papszSiblingFiles ); 417 418:: 419 420 The nOpenFlags argument is a 'or-able' combination of the following values : 421 422:: 423 424 /* Note: we define GDAL_OF_READONLY and GDAL_OF_UPDATE to be on purpose */ 425 /* equals to GA_ReadOnly and GA_Update */ 426 427 /** Open in read-only mode. */ 428 #define GDAL_OF_READONLY 0x00 429 /** Open in update mode. */ 430 #define GDAL_OF_UPDATE 0x01 431 432 /** Allow raster and vector drivers. */ 433 #define GDAL_OF_ALL 0x00 434 435 /** Allow raster drivers. */ 436 #define GDAL_OF_RASTER 0x02 437 /** Allow vector drivers. */ 438 #define GDAL_OF_VECTOR 0x04 439 /* Some space for GDAL 3.0 new types ;-) */ 440 /*#define GDAL_OF_OTHER_KIND1 0x08 */ 441 /*#define GDAL_OF_OTHER_KIND2 0x10 */ 442 443 /** Open in shared mode. */ 444 #define GDAL_OF_SHARED 0x20 445 446 /** Emit error message in case of failed open. */ 447 #define GDAL_OF_VERBOSE_ERROR 0x40 448 449:: 450 451 The existing GDALOpen(), GDALOpenShared(), OGROpen(), OGROpenShared(), 452 OGR_Dr_Open() are just wrappers of GDALOpenEx() with appropriate open flags. 453 From the user point of view, their behavior is identical to the existing one, 454 i.e. GDALOpen() family will only returns datasets of drivers with declared raster 455 capabilities, and similarly with OGROpen() family with vector. 456 457- GDALOpenInfo class. The following changes are done : 458 459 - the second argument of the constructor is now nOpenFlags instead 460 of GDALAccess, with same semantics as GDALOpenEx(). GDALOpenInfo 461 uses the read-only/update bit to "compute" the eAccess flag that 462 is heavily used in existing drivers. Drivers with both raster and 463 vector capabilities can use the GDAL_OF_VECTOR/GDAL_OF_RASTER bits 464 to determine the intent of the caller. For example if a caller 465 opens with GDAL_OF_RASTER only and the dataset only contains 466 vector data, the driver might decide to not open the dataset (if 467 it is a read-only driver. If it is a driver with update 468 capability, it should do that only if the opening is done in 469 read-only mode). 470 - the open options passed to GDALOpenEx() are stored into a 471 papszOpenOptions member of GDALOpenInfo, so that drivers can use 472 them. 473 - the "FILE\* fp" member is transformed into "VSILFILE\* fpL". This 474 change is motivated by the fact that most popular drivers now use 475 the VSI Virtual File API, so they can now directly use the fpL 476 member instead of re-opening again the file. A global pass on all 477 in-tree GDAL drivers that used fp has been made. 478 - A VSIStatExL() was done previously to determine the nature of the 479 file passed. Now, we optimistically begin with a VSIFOpenL(), 480 assuming that in most use cases the passed filename is a file. If 481 the opening fails, VSIStatExL() is done to determine the nature of 482 the filename. 483 - If the requested access mode is update, the opening of the file 484 with VSIFOpenL() is done with "rb+" permissions to be directly 485 usable. 486 - The papszSiblingFiles member is now private. It is accessed by a 487 GetSiblingFiles() method that does the ReadDir() on demand. This 488 can speed up the Identify() method that generally does not require 489 to know sibling files. 490 - A new method, TryToIngest(), is added to read more than the first 491 1024 bytes of a file. This is useful for a few vector drivers, 492 like GML or NAS, that must fetch a bit more bytes to be able to 493 identify the file. 494 495Layer 496~~~~~ 497 498- OGRLayer extends GDALMajorObject. Drivers can now define layer 499 metadata items that can be retrieved with the usual 500 GetMetadata()/GetMetadateItem() API. 501 502- The GetInfo() method has been removed. It has never been implemented 503 in any in-tree drivers and has been deprecated for a long time. 504 505Other 506~~~~~ 507 508- The deprecated and unused GDALProjDefH and GDALOptionDefinition types 509 have been removed from gdal.h 510 511- GDALGeneralCmdLineProcessor() now interprets the nOptions 512 (combination of GDAL_OF_RASTER and GDAL_OF_RASTER) argument as the 513 type of drivers that should be displayed with the --formats option. 514 If set to 0, GDAL_OF_RASTER is assumed. 515 516- the --formats option of GDAL utilities outputs whether drivers have 517 raster and/or vector capabilities 518 519- the --format option of GDAL utilities outputs GDAL_DMD_EXTENSIONS, 520 GDAL_DMD_OPENOPTIONLIST, GDAL_DS_LAYER_CREATIONOPTIONLIST. 521 522- OGRGeneralCmdLineProcessor() use GDALGeneralCmdLineProcessor() 523 implementation, restricting --formats to vector capable drivers. 524 525Changes in drivers 526------------------ 527 528- OGR PCIDSK driver has been merged into GDAL PCIDSK driver. 529 530- OGR PDF driver has been merged into GDAL PDF driver. 531 532- A global pass has been made to in-tree OGR drivers that have to open 533 a file to determine if they recognize it. They have been converted to 534 GDALDriver to accept a GDALOpenInfo argument and they now use its 535 pabyHeader field to examine the first bytes of files. The number of 536 system calls realated to file access (open/stat), in order to 537 determine that a file is not recognized by any OGR driver, has now 538 dropped from 46 in GDAL 1.11 to 1. The converted drivers are : 539 AeronavFAA, ArcGEN, AVCBin, AVCE00, BNA, CSV, DGN, EDIGEO, ESRI 540 Shapefile, GeoJSON, GeoRSS, GML, GPKG, GPSBabel, GPX, GTM, HTF, ILI1, 541 ILI2, KML, LIBKML, MapInfo File, MySQL, NAS, NTF, OpenAIR, OSM, PDS, 542 REC, S57, SDTS, SEGUKOOA, SEGY, SOSI, SQLite, SUA, SVG, TIGER, VFK, 543 VRT, WFS 544 545- Long driver description is set for most OGR drivers. 546 547- All classes deriving from OGRLayer have been modified to call 548 SetDescription() with the value of 549 GetName()/poFeatureDefn->GetName(). test_ogrsf tests that it is 550 properly set. 551 552- Following drivers are kept as OGRSFDriver, but their Open() method 553 does early extension/prefix testing to avoid datasource object to be 554 instantiated : CartoDB, CouchDB, DXF, EDIGEO, GeoConcept, GFT, GME, 555 IDRISI, OGDI, PCIDSK, PG, XPlane. 556 557- Identify() has been implemented for CSV, DGN, DXF, EDIGEO, GeoJSON, 558 GML, KML, LIBKML, MapInfo File, NAS, OpenFileGDB, OSM, S57, Shape, 559 SQLite, VFK, VRT. 560 561- GDAL_DMD_EXTENSION/GDAL_DMD_EXTENSIONS set for following drivers: 562 AVCE00, BNA, CSV, DGN, DWG, DXF, EDIGEO, FileGDB, Geoconcept, 563 GeoJSON, Geomedia, GML, GMT, GPKG, GPX, GPSTrackMaker, IDRISI Vector, 564 Interlis 1, Interlis 2, KML, LIBKML, MDB, MapInfo File, NAS, ODS, 565 OpenFileGDB, OSM, PGDump, PGeo, REC, S57, ESRI Shapefile, SQLite, 566 SVG, WaSP, XLS, XLSX, XPlane. 567 568- Document dataset and layer creation options of BNA, DGN, FileGDB, 569 GeoConccept, GeoJSON, GeoRSS, GML, GPKG, KML, LIBKML, PG, PGDump and 570 ESRI Shapefile drivers as GDAL_DMD_CREATIONOPTIONLIST / 571 GDAL_DS_LAYER_CREATIONOPTIONLIST. 572 573- Add open options AAIGRID, PDF, S57 and ESRI Shapefile drivers. 574 575- GetFileList() implemented in OpenFileGDB, Shapefile and OGR VRT 576 drivers. 577 578- Rename datasource SyncToDisk() as FlushCache() for LIBKML, OCI, ODS, 579 XLSX drivers. 580 581- Use poOpenInfo->fpL to avoid useless file re-opening in GTiff, PNG, 582 JPEG, GIF, VRT, NITF, DTED. 583 584- HTTP driver: declared as GDAL_DCAP_RASTER and GDAL_DCAP_VECTOR 585 driver. 586 587- RIK: implement Identify() 588 589- Note: the compilation and good working of the following OGR drivers 590 (mostly proprietary) could not be tested: ArcObjects, DWG, DODS, SDE, 591 FME, GRASS, IDB, OCI, MSSQLSpatial(compilation OK, but not runtime 592 tested) 593 594Changes in utilities 595-------------------- 596 597- gdalinfo accepts a -oo option to define open options 598- ogrinfo accepts a -oo option to define open options 599- ogr2ogr accepts a -oo option to define input dataset open options, 600 and -doo to define destination dataset open options 601 602Changes in SWIG bindings 603------------------------ 604 605- Python and Java bindings: 606 607 - add new GDALDataset methods taken from OGRDataSource : 608 CreateLayer(), CopyLayer(), DeleteLayer(), GetLayerCount(), 609 GetLayerByIndex(), GetLayerByName(), TestCapability(), 610 ExecuteSQL(), ReleaseResultSet(), GetStyleTable() and 611 SetStyleTable(). 612 - make OGR Driver, DataSource and Layer objects derive from 613 MajorObject 614 615- Perl and CSharp: make sure that it still compiles but some work would 616 have to be done by their mainteners to be able to use the new 617 capabilities 618 619Potential changes that are *NOT* included in this RFC 620----------------------------------------------------- 621 622"Natural" evolutions of current RFC : 623 624- Unifying the GDAL MEM and OGR Memory drivers. 625- Unifying the GDAL VRT and OGR VRT drivers. 626 627Further unification steps : 628 629- Source tree changes to move OGR drivers from ogr/ogrsf_frmts/ to 630 frmts/ , to move ogr/ogrsf_frmts/generic/\* to gcore/\*, etc... 631- Documentation unification (pages with list of drivers, etc...) 632- Renaming to remove traces of OGR namespace : OGRLayer -> GDALLayer, 633 etc... 634- Kill --without-ogr compilation option ? It has been preserved in a 635 working state even if it embeds now ogr/ogrsf_frmts/generic and 636 ogr/ogrsf_frmts/mitab for conveniency. 637- Unification of some utilities : "gdal info XXX", "gdal convert XXX" 638 that would work on all kind of datasets. 639 640Backward compatibility 641---------------------- 642 643GDALDriverManager::GetDriverCount(), GetDriver() now returns OGR 644drivers, as well as GDAL drivers 645 646The reference counting in GDAL datasets and GDAL 1.X OGR datasources was 647a bit different. It starts at 1 for GDAL datasets, and started at 0 for 648OGR datasources. Now that OGRDataSource is basically a GDALDataset, it 649starts at 1 for both cases. Hopefully there are very few users of the 650OGR_DS_GetRefCount() API. If it was deemed necessary we could restore 651the previous behavior at the C API, but that would not be possible at 652the C++ level. For reference, neither MapServer nor QGIS use 653OGR_DS_GetRefCount(). 654 655Documentation 656------------- 657 658A pass should be made on the documentation to check that all new methods 659are properly documented. The OGR general documentation (especially C++ 660API Read/Write tutorial, Driver implementation tutorial and OGR 661architecture) should be updated to reflect the changes. 662 663Testing 664------- 665 666Very few changes have been made so that the existing autotest suite 667still passes. Additions have been made to test the GDALOpenEx() API and 668the methods "imported" from OGRDataSource into GDALDataset. 669 670Version numbering 671----------------- 672 673Although the above describes changes should have very few impact on 674existing applications of the C API, some behavior changes, C++ level 675changes and the conceptual changes are thought to deserve a 2.0 version 676number. 677 678Implementation 679-------------- 680 681Implementation will be done by Even Rouault. 682 683The proposed implementation lies in the "unification" branch of the 684`https://github.com/rouault/gdal2/tree/unification <https://github.com/rouault/gdal2/tree/unification>`__ 685repository. 686 687The list of changes : 688`https://github.com/rouault/gdal2/compare/unification <https://github.com/rouault/gdal2/compare/unification>`__ 689 690Voting history 691-------------- 692 693+1 from JukkaR, FrankW, DanielM, TamasS and EvenR. 694