1.. _rfc-6: 2 3======================================================================================= 4RFC 6: Geometry and Feature Style as OGR Special Fields 5======================================================================================= 6 7Author: Tamas Szekeres 8 9Contact: szekerest@gmail.com 10 11Status: Adopted 12 13Summary 14------- 15 16This proposal addresses and issue have been discovered long ago, and OGR 17provides no equivalent solution so far. 18 19Some of the supported formats like Mapinfo.tab may contain multiple 20geometry types and style information. In order to handle this kind of 21data sources properly a support for selecting the layers by geometry 22type or by the style info would be highly required. For more details see 23the following MapServer related bugs later in this document. 24 25All of the proposed changes can be found at the tracking bug of this RFC 26referenced later in this document. 27 28Main concepts 29------------- 30 31The most reasonable way to support this feature is to extend the 32currently existing 'special field' approach to allow specifying more 33than one fields. Along with the already defined 'FID' field we will add 34the following ones: 35 36- 'OGR_GEOMETRY' containing the geometry type like 'POINT' or 37 'POLYGON'. 38- 'OGR_STYLE' containing the style string. 39- 'OGR_GEOM_WKT' containing the full WKT of the geometry. 40 41By providing the aforementioned fields one can make for example the 42following selections: 43 44- select FID, OGR_GEOMETRY, OGR_STYLE, OGR_GEOM_WKT, \* from MyTable 45 where OGR_GEOMETRY='POINT' OR OGR_GEOMETRY='POLYGON' 46- select FID, OGR_GEOMETRY, OGR_STYLE, OGR_GEOM_WKT, \* from MyTable 47 where OGR_STYLE LIKE '%BRUSH%' 48- select FID, OGR_GEOMETRY, OGR_STYLE, OGR_GEOM_WKT, \* from MyTable 49 where OGR_GEOM_WKT LIKE 'POLYGON%' 50- select distinct OGR_GEOMETRY from MyTable order by OGR_GEOMETRY desc 51 52Implementation 53-------------- 54 55There are two distinct areas where this feature plays a role 56 57- Feature query implemented at ogrfeaturequery.cpp 58 59- SQL based selection implemented at ogr_gensql.cpp and 60 ogrdatasource.cpp 61 62To specify arbitrary number of special fields we will declare an array 63for the field names and types in ogrfeaturequery.cpp as 64 65:: 66 67 char* SpecialFieldNames[SPECIAL_FIELD_COUNT] 68 = {"FID", "OGR_GEOMETRY", "OGR_STYLE", "OGR_GEOM_WKT"}; 69 swq_field_type SpecialFieldTypes[SPECIAL_FIELD_COUNT] 70 = {SWQ_INTEGER, SWQ_STRING, SWQ_STRING, SWQ_STRING}; 71 72So as to make this array accessible to the other files the followings 73will be added to ogr_p.h 74 75:: 76 77 CPL_C_START 78 #include "ogr_swq.h" 79 CPL_C_END 80 81 #define SPF_FID 0 82 #define SPF_OGR_GEOMETRY 1 83 #define SPF_OGR_STYLE 2 84 #define SPF_OGR_GEOM_WKT 3 85 #define SPECIAL_FIELD_COUNT 4 86 87 extern char* SpecialFieldNames[SPECIAL_FIELD_COUNT]; 88 extern swq_field_type SpecialFieldTypes[SPECIAL_FIELD_COUNT]; 89 90In ogrfeature.cpp the field accessor functions (GetFieldAsString, 91GetFieldAsInteger, GetFieldAsDouble) will be modified providing the 92values of the special fields by the field index 93 94The following code will be added to the beginning of 95OGRFeature::GetFieldAsInteger: 96 97:: 98 99 int iSpecialField = iField - poDefn->GetFieldCount(); 100 if (iSpecialField >= 0) 101 { 102 // special field value accessors 103 switch (iSpecialField) 104 { 105 case SPF_FID: 106 return GetFID(); 107 default: 108 return 0; 109 } 110 } 111 112The following code will be added to the beginning of 113OGRFeature::GetFieldAsDouble: 114 115:: 116 117 int iSpecialField = iField - poDefn->GetFieldCount(); 118 if (iSpecialField >= 0) 119 { 120 // special field value accessors 121 switch (iSpecialField) 122 { 123 case SPF_FID: 124 return GetFID(); 125 default: 126 return 0.0; 127 } 128 } 129 130The following code will be added to the beginning of 131OGRFeature::GetFieldAsString: 132 133:: 134 135 int iSpecialField = iField - poDefn->GetFieldCount(); 136 if (iSpecialField >= 0) 137 { 138 // special field value accessors 139 switch (iSpecialField) 140 { 141 case SPF_FID: 142 sprintf( szTempBuffer, "%d", GetFID() ); 143 return m_pszTmpFieldValue = CPLStrdup( szTempBuffer ); 144 case SPF_OGR_GEOMETRY: 145 return poGeometry->getGeometryName(); 146 case SPF_OGR_STYLE: 147 return GetStyleString(); 148 case SPF_OGR_GEOM_WKT: 149 { 150 if (poGeometry->exportToWkt( &m_pszTmpFieldValue ) == OGRERR_NONE ) 151 return m_pszTmpFieldValue; 152 else 153 return ""; 154 } 155 default: 156 return ""; 157 } 158 } 159 160The current implementation of OGRFeature::GetFieldAsString uses a static 161string to hold the const char\* return value that is highly avoidable 162and makes the code thread unsafe. In this regard the 'static char 163szTempBuffer[80]' will be changed to non static and a new member will be 164added to OGRFeature in ogrfeature.h as: 165 166:: 167 168 char * m_pszTmpFieldValue; 169 170This member will be initialized to NULL at the constructor, and will be 171freed using CPLFree() at the destructor of OGRFeature. 172 173In OGRFeature::GetFieldAsString all of the occurrences of 'return 174szTempBuffer;' will be changed to 'return m_pszTmpFieldValue = 175CPLStrdup( szTempBuffer );' 176 177OGRFeature::GetFieldAsString is responsible to destroy the old value of 178m_pszTmpFieldValue at the beginning of the function: 179 180:: 181 182 CPLFree(m_pszTmpFieldValue); 183 m_pszTmpFieldValue = NULL; 184 185In ogrfeaturequery.cpp we should change OGRFeatureQuery::Compile to add 186the special fields like: 187 188:: 189 190 iField = 0; 191 while (iField < SPECIAL_FIELD_COUNT) 192 { 193 papszFieldNames[poDefn->GetFieldCount() + iField] = SpecialFieldNames[iField]; 194 paeFieldTypes[poDefn->GetFieldCount() + iField] = SpecialFieldTypes[iField]; 195 ++iField; 196 } 197 198In ogrfeaturequery.cpp OGRFeatureQueryEvaluator() should be modifyed 199according to the field specific actions like 200 201:: 202 203 int iSpecialField = op->field_index - poFeature->GetDefnRef()->GetFieldCount(); 204 if( iSpecialField >= 0 ) 205 { 206 if ( iSpecialField < SPECIAL_FIELD_COUNT ) 207 { 208 switch ( SpecialFieldTypes[iSpecialField] ) 209 { 210 case SWQ_INTEGER: 211 sField.Integer = poFeature->GetFieldAsInteger( op->field_index ); 212 case SWQ_STRING: 213 sField.String = (char*) poFeature->GetFieldAsString( op->field_index ); 214 } 215 } 216 else 217 { 218 CPLDebug( "OGRFeatureQuery", "Illegal special field index."); 219 return FALSE; 220 } 221 psField = &sField; 222 } 223 else 224 psField = poFeature->GetRawFieldRef( op->field_index ); 225 226In ogrfeaturequery.cpp OGRFeatureQuery::FieldCollector should be 227modifyed to add the field names like: 228 229:: 230 231 if( op->field_index >= poTargetDefn->GetFieldCount() 232 && op->field_index < poTargetDefn->GetFieldCount() + SPECIAL_FIELD_COUNT) 233 pszFieldName = SpecialFieldNames[op->field_index]; 234 235In ogrdatasource.cpp ExecuteSQL() will allocate the arrays according to 236the number of the special fields: 237 238:: 239 240 sFieldList.names = (char **) 241 CPLMalloc( sizeof(char *) * (nFieldCount+SPECIAL_FIELD_COUNT) ); 242 sFieldList.types = (swq_field_type *) 243 CPLMalloc( sizeof(swq_field_type) * (nFieldCount+SPECIAL_FIELD_COUNT) ); 244 sFieldList.table_ids = (int *) 245 CPLMalloc( sizeof(int) * (nFieldCount+SPECIAL_FIELD_COUNT) ); 246 sFieldList.ids = (int *) 247 CPLMalloc( sizeof(int) * (nFieldCount+SPECIAL_FIELD_COUNT) ); 248 249And the fields will be added as 250 251:: 252 253 for (iField = 0; iField < SPECIAL_FIELD_COUNT; iField++) 254 { 255 sFieldList.names[sFieldList.count] = SpecialFieldNames[iField]; 256 sFieldList.types[sFieldList.count] = SpecialFieldTypes[iField]; 257 sFieldList.table_ids[sFieldList.count] = 0; 258 sFieldList.ids[sFieldList.count] = nFIDIndex + iField; 259 sFieldList.count++; 260 } 261 262For supporting the SQL based queries we should also modify the 263constructor of OGRGenSQLResultsLayer in ogr_gensql.cpp and set the field 264type properly: 265 266:: 267 268 else if ( psColDef->field_index >= iFIDFieldIndex ) 269 { 270 switch ( SpecialFieldTypes[psColDef->field_index - iFIDFieldIndex] ) 271 { 272 case SWQ_INTEGER: 273 oFDefn.SetType( OFTInteger ); 274 break; 275 case SWQ_STRING: 276 oFDefn.SetType( OFTString ); 277 break; 278 case SWQ_FLOAT: 279 oFDefn.SetType( OFTReal ); 280 break; 281 } 282 } 283 284Some of the queries will require to modify 285OGRGenSQLResultsLayer::PrepareSummary in ogr_gensql.cpp will be 286simplified (GetFieldAsString will be used in all cases to access the 287field values): 288 289:: 290 291 pszError = swq_select_summarize( psSelectInfo, iField, 292 poSrcFeature->GetFieldAsString( psColDef->field_index ) ); 293 294OGRGenSQLResultsLayer::TranslateFeature should also be modifyed when 295copying the fields from primary record to the destination feature 296 297:: 298 299 if ( psColDef->field_index >= iFIDFieldIndex && 300 psColDef->field_index < iFIDFieldIndex + SPECIAL_FIELD_COUNT ) 301 { 302 switch (SpecialFieldTypes[psColDef->field_index - iFIDFieldIndex]) 303 { 304 case SWQ_INTEGER: 305 poDstFeat->SetField( iField, poSrcFeat->GetFieldAsInteger(psColDef->field_index) ); 306 case SWQ_STRING: 307 poDstFeat->SetField( iField, poSrcFeat->GetFieldAsString(psColDef->field_index) ); 308 } 309 } 310 311For supporting the 'order by' queries we should also modify 312OGRGenSQLResultsLayer::CreateOrderByIndex() as: 313 314:: 315 316 317 if ( psKeyDef->field_index >= iFIDFieldIndex) 318 { 319 if ( psKeyDef->field_index < iFIDFieldIndex + SPECIAL_FIELD_COUNT ) 320 { 321 switch (SpecialFieldTypes[psKeyDef->field_index - iFIDFieldIndex]) 322 { 323 case SWQ_INTEGER: 324 psDstField->Integer = poSrcFeat->GetFieldAsInteger(psKeyDef->field_index); 325 case SWQ_STRING: 326 psDstField->String = CPLStrdup( poSrcFeat->GetFieldAsString(psKeyDef->field_index) ); 327 } 328 } 329 continue; 330 } 331 332All of the strings allocated previously should be deallocated later in 333the same function as: 334 335:: 336 337 338 if ( psKeyDef->field_index >= iFIDFieldIndex ) 339 { 340 /* warning: only special fields of type string should be deallocated */ 341 if (SpecialFieldTypes[psKeyDef->field_index - iFIDFieldIndex] == SWQ_STRING) 342 { 343 for( i = 0; i < nIndexSize; i++ ) 344 { 345 OGRField *psField = pasIndexFields + iKey + i * nOrderItems; 346 CPLFree( psField->String ); 347 } 348 } 349 continue; 350 } 351 352When ordering by the field values the OGRGenSQLResultsLayer::Compare 353should also be modifyed: 354 355:: 356 357 if( psKeyDef->field_index >= iFIDFieldIndex ) 358 poFDefn = NULL; 359 else 360 poFDefn = poSrcLayer->GetLayerDefn()->GetFieldDefn( 361 psKeyDef->field_index ); 362 363 if( (pasFirstTuple[iKey].Set.nMarker1 == OGRUnsetMarker 364 && pasFirstTuple[iKey].Set.nMarker2 == OGRUnsetMarker) 365 || (pasSecondTuple[iKey].Set.nMarker1 == OGRUnsetMarker 366 && pasSecondTuple[iKey].Set.nMarker2 == OGRUnsetMarker) ) 367 nResult = 0; 368 else if ( poFDefn == NULL ) 369 { 370 switch (SpecialFieldTypes[psKeyDef->field_index - iFIDFieldIndex]) 371 { 372 case SWQ_INTEGER: 373 if( pasFirstTuple[iKey].Integer < pasSecondTuple[iKey].Integer ) 374 nResult = -1; 375 else if( pasFirstTuple[iKey].Integer > pasSecondTuple[iKey].Integer ) 376 nResult = 1; 377 break; 378 case SWQ_STRING: 379 nResult = strcmp(pasFirstTuple[iKey].String, 380 pasSecondTuple[iKey].String); 381 break; 382 } 383 } 384 385Adding New Special Fields 386------------------------- 387 388Adding a new special field in a subsequent development phase is fairly 389straightforward and the following steps should be made: 390 3911. In ogr_p.h a new constant should be added with the value of the 392 SPECIAL_FIELD_COUNT and SPECIAL_FIELD_COUNT should be incremented by 393 one. 394 3952. In ogrfeaturequery.cpp the special field string and the type should 396 be added to SpecialFieldNames and SpecialFieldTypes respectively 397 3983. The field value accessors (OGRFeature::GetFieldAsString, 399 OGRFeature::GetFieldAsInteger, OGRFeature::GetFieldAsDouble) should 400 be modifyed to provide the value of the new special field. All of 401 these functions provide const return values so GetFieldAsString 402 should retain the value in the m_pszTmpFieldValue member. 403 4044. When adding a new value with a type other than SWQ_INTEGER and 405 SWQ_STRING the following functions might also be modified 406 accordingly: 407 408- OGRGenSQLResultsLayer::OGRGenSQLResultsLayer 409- OGRGenSQLResultsLayer::TranslateFeature 410- OGRGenSQLResultsLayer::CreateOrderByIndex 411- OGRGenSQLResultsLayer::Compare 412- OGRFeatureQueryEvaluator 413 414Backward Compatibility 415---------------------- 416 417In most cases the backward compatibility of the OGR library will be 418retained. However the special fields will potentially conflict with 419regard fields with the given names. When accessing the field values the 420special fields will take pecedence over the other fields with the same 421names. 422 423When using OGRFeature::GetFieldAsString the returned value will be 424stored as a member variable instead of a static variable. The string 425will be deallocated and will no longer be usable after the destruction 426of the feature. 427 428Regression Testing 429------------------ 430 431A new gdalautotest/ogr/ogr_sqlspecials.py script to test support for all 432special fields in the ExecuteSQL() call and with WHERE clauses. 433 434Documentation 435------------- 436 437The OGR SQL document will be updated to reflect the support for special 438fields. 439 440Implementation Staffing 441----------------------- 442 443Tamas Szekeres will implement the bulk of the RFC in time for GDAL/OGR 4441.4.0. 445 446Frank Warmerdam will consider how the backward compatibility issues 447(with special regard to the modified lifespan of the GetFieldAsString 448returned value) will affect the other parts of the OGR project and will 449write the Python regression testing script. 450 451References 452---------- 453 454- Tracking bug for this feature (containing all of the proposed code 455 changes): #1333 456 457- MapServer related bugs: 458 459 - `1129 <http://trac.osgeo.org/mapserver/ticket/1129>`__ 460 - `1438 <http://trac.osgeo.org/mapserver/ticket/1438>`__ 461 462Voting History 463-------------- 464 465Frank Warmerdam +1 466 467Daniel Morissette +1 468 469Howard Butler +0 470 471Andrey Kiselev +1 472