1.. _rfc-35: 2 3================================================================================ 4RFC 35: Delete, reorder and alter field definitions of OGR layers 5================================================================================ 6 7Authors: Even Rouault 8 9Contact: even dot rouault at spatialys.com 10 11Status: Adopted 12 13Summary 14------- 15 16This document proposes changes in OGR to add the capability to delete 17fields, reorder fields and alter field definitions, in OGR layer 18definitions. 19 20Rationale 21--------- 22 23Currently, an OGR layer definition can only be altered to add a new 24field definition with OGRLayer::CreateField(). 25 26It is desirable to extend OGR capabilities to be able to delete, reorder 27and alter field definitions of existing layers. Such wish has been 28expressed in ticket #2671 and comes back regularly on QGIS mailing list 29(e.g. 30`http://lists.osgeo.org/pipermail/qgis-user/2011-May/011935.html <http://lists.osgeo.org/pipermail/qgis-user/2011-May/011935.html>`__). 31QGIS currently has a "Table Manager" extension to work around the lack 32of DeleteField(), so a proper solution is clearly needed. 33 34Planned Changes 35--------------- 36 37The OGRLayer class will be extended with the following methods : 38 39:: 40 41 virtual OGRErr DeleteField( int iField ); 42 virtual OGRErr ReorderFields( int* panMap ); 43 virtual OGRErr AlterFieldDefn( int iField, OGRFieldDefn* poNewFieldDefn, int nFlags ); 44 45 /* non virtual : conveniency wrapper for ReorderFields() */ 46 OGRErr ReorderField( int iOldFieldPos, int iNewFieldPos ); 47 48The documentation of those new methods is : 49 50:: 51 52 53 /** 54 \fn OGRErr OGRLayer::DeleteField( int iField ); 55 56 \brief Delete an existing field on a layer. 57 58 You must use this to delete existing fields 59 on a real layer. Internally the OGRFeatureDefn for the layer will be updated 60 to reflect the deleted field. Applications should never modify the OGRFeatureDefn 61 used by a layer directly. 62 63 This method should not be called while there are feature objects in existence that 64 were obtained or created with the previous layer definition. 65 66 Not all drivers support this method. You can query a layer to check if it supports it 67 with the OLCDeleteField capability. Some drivers may only support this method while 68 there are still no features in the layer. When it is supported, the existings features of the 69 backing file/database should be updated accordingly. 70 71 This function is the same as the C function OGR_L_DeleteField(). 72 73 @param iField index of the field to delete. 74 75 @return OGRERR_NONE on success. 76 77 @since OGR 1.9.0 78 */ 79 80 /** 81 \fn OGRErr OGRLayer::ReorderFields( int* panMap ); 82 83 \brief Reorder all the fields of a layer. 84 85 You must use this to reorder existing fields 86 on a real layer. Internally the OGRFeatureDefn for the layer will be updated 87 to reflect the reordering of the fields. Applications should never modify the OGRFeatureDefn 88 used by a layer directly. 89 90 This method should not be called while there are feature objects in existence that 91 were obtained or created with the previous layer definition. 92 93 panMap is such that,for each field definition at position i after reordering, 94 its position before reordering was panMap[i]. 95 96 For example, let suppose the fields were "0","1","2","3","4" initially. 97 ReorderFields([0,2,3,1,4]) will reorder them as "0","2","3","1","4". 98 99 Not all drivers support this method. You can query a layer to check if it supports it 100 with the OLCReorderFields capability. Some drivers may only support this method while 101 there are still no features in the layer. When it is supported, the existings features of the 102 backing file/database should be updated accordingly. 103 104 This function is the same as the C function OGR_L_ReorderFields(). 105 106 @param panMap an array of GetLayerDefn()->GetFieldCount() elements which 107 is a permutation of [0, GetLayerDefn()->GetFieldCount()-1]. 108 109 @return OGRERR_NONE on success. 110 111 @since OGR 1.9.0 112 */ 113 114 /** 115 \fn OGRErr OGRLayer::ReorderField( int iOldFieldPos, int iNewFieldPos ); 116 117 \brief Reorder an existing field on a layer. 118 119 This method is a conveniency wrapper of ReorderFields() dedicated to move a single field. 120 It is a non-virtual method, so drivers should implement ReorderFields() instead. 121 122 You must use this to reorder existing fields 123 on a real layer. Internally the OGRFeatureDefn for the layer will be updated 124 to reflect the reordering of the fields. Applications should never modify the OGRFeatureDefn 125 used by a layer directly. 126 127 This method should not be called while there are feature objects in existence that 128 were obtained or created with the previous layer definition. 129 130 The field definition that was at initial position iOldFieldPos will be moved at 131 position iNewFieldPos, and elements between will be shuffled accordingly. 132 133 For example, let suppose the fields were "0","1","2","3","4" initially. 134 ReorderField(1, 3) will reorder them as "0","2","3","1","4". 135 136 Not all drivers support this method. You can query a layer to check if it supports it 137 with the OLCReorderFields capability. Some drivers may only support this method while 138 there are still no features in the layer. When it is supported, the existings features of the 139 backing file/database should be updated accordingly. 140 141 This function is the same as the C function OGR_L_ReorderField(). 142 143 @param iOldFieldPos previous position of the field to move. Must be in the range [0,GetFieldCount()-1]. 144 @param iNewFieldPos new position of the field to move. Must be in the range [0,GetFieldCount()-1]. 145 146 @return OGRERR_NONE on success. 147 148 @since OGR 1.9.0 149 */ 150 151 /** 152 \fn OGRErr OGRLayer::AlterFieldDefn( int iField, OGRFieldDefn* poNewFieldDefn, int nFlags ); 153 154 \brief Alter the definition of an existing field on a layer. 155 156 You must use this to alter the definition of an existing field of a real layer. 157 Internally the OGRFeatureDefn for the layer will be updated 158 to reflect the altered field. Applications should never modify the OGRFeatureDefn 159 used by a layer directly. 160 161 This method should not be called while there are feature objects in existence that 162 were obtained or created with the previous layer definition. 163 164 Not all drivers support this method. You can query a layer to check if it supports it 165 with the OLCAlterFieldDefn capability. Some drivers may only support this method while 166 there are still no features in the layer. When it is supported, the existings features of the 167 backing file/database should be updated accordingly. Some drivers might also not support 168 all update flags. 169 170 This function is the same as the C function OGR_L_AlterFieldDefn(). 171 172 @param iField index of the field whose definition must be altered. 173 @param poNewFieldDefn new field definition 174 @param nFlags combination of ALTER_NAME_FLAG, ALTER_TYPE_FLAG and ALTER_WIDTH_PRECISION_FLAG 175 to indicate which of the name and/or type and/or width and precision fields from the new field 176 definition must be taken into account. 177 178 @return OGRERR_NONE on success. 179 180 @since OGR 1.9.0 181 */ 182 183Three new layer capabilities are added : 184 185:: 186 187 OLCDeleteField / "DeleteField": TRUE if this layer can delete 188 existing fields on the current layer using DeleteField(), otherwise FALSE. 189 190 OLCReorderFields / "ReorderFields": TRUE if this layer can reorder 191 existing fields on the current layer using ReorderField() or ReorderFields(), otherwise FALSE. 192 193 OLCAlterFieldDefn / "AlterFieldDefn": TRUE if this layer can alter 194 the definition of an existing field on the current layer using AlterFieldDefn(), otherwise FALSE. 195 196The new methods are mapped to the C API : 197 198:: 199 200 OGRErr CPL_DLL OGR_L_DeleteField( OGRLayerH, int iField ); 201 OGRErr CPL_DLL OGR_L_ReorderFields( OGRLayerH, int* panMap ); 202 OGRErr CPL_DLL OGR_L_ReorderField( OGRLayerH, int iOldFieldPos, int iNewFieldPos ); 203 OGRErr CPL_DLL OGR_L_AlterFieldDefn( OGRLayerH, int iField, OGRFieldDefnH hNewFieldDefn, int nFlags ); 204 205For the purpose of the implementation, new methods are also added to the 206OGRFeatureDefn class : 207 208:: 209 210 OGRErr DeleteFieldDefn( int iField ); 211 OGRErr ReorderFieldDefns( int* panMap ); 212 213A OGRErr OGRCheckPermutation(int\* panPermutation, int nSize) function 214is added to ogrutils.cpp to check that the array is a permutation of 215[0,nSize-1]. It is used by OGRFeatureDefn::ReorderFieldDefns() and can 216be used by all drivers implementing OGRLayer::ReorderFields() to 217validate the panMap argument. 218 219Altering field types 220-------------------- 221 222This RFC does not attempt to guarantee which type conversions will be 223possible. It will depend on the capabilities of the implementing 224drivers. For example, for database drivers, the operation will be 225directly done on the server side (through a 'ALTER TABLE my_table ALTER 226COLUMN my_column TYPE new_type' command for the PG driver). So some 227conversions might be possible, others not... 228 229It is however expected that converting from any type to OFTString will 230be supported in most cases when AlterFieldDefn() is supported. 231 232Drivers that don't support a conversion and that were required to do it 233(ALTER_TYPE_FLAG set and new_type != old_type) should emit an explicit 234error. 235 236Compatibility Issues 237-------------------- 238 239None 240 241Changed drivers 242--------------- 243 244The shapefile driver will implement DeleteField(), ReorderFields() and 245AlterFieldDefn(). Shapelib will be extended with DBFReorderFields() and 246DBFAlterFieldDefn(). 247 248Note: The implementation of AlterFieldDefn() in the Shapefile driver 249does not support altering the field type, except when converting to 250OFTString. It will not reformat numeric values of existing features if 251width or precision are changed. However, appropriate field truncation or 252expansion will occur if the width is altered. 253 254Other drivers, mainly database drivers (PG, MySQL, SQLite), could be 255easily extended to implement the new API by issuing the appropriate SQL 256command (ALTER TABLE foo DROP COLUMN bar, ALTER TABLE foo ALTER COLUMN 257bar, ...). The implementation of DeleteField() and AlterFieldDefn() in 258the PG driver is indeed planned, provided this RFC is adopted. The 259Memory driver will also updated to support DeleteField(), 260ReorderFields() and AlterFieldDefn(). 261 262SWIG bindings 263------------- 264 265DeleteField(), ReorderField(), ReorderFields() and AlterFieldDefn() will 266be mapped to SWIG. 267 268Test Suite 269---------- 270 271The autotest suite will be extended to test the implementation of the 272new API for the Shapefile driver. An example of the use of the new API 273is attached to ticket #2671 274(`rfc35_test.py <http://trac.osgeo.org/gdal/attachment/ticket/2671/rfc35_test.py>`__) 275and will be turned into unit tests. 276 277Implementation 278-------------- 279 280Implementation will be done by Even Rouault in GDAL/OGR trunk. Changes 281in Shapelib will need to be pushed into upstream CVS by a Shapelib 282committer. The proposed implementation is attached as a patch in ticket 283#2671 284(`rfc35_v3.patch <http://trac.osgeo.org/gdal/attachment/ticket/2671/rfc35_v3.patch>`__). 285 286Voting history 287-------------- 288 289+1 from FrankW, DanielM, HowardB, TamasS and EvenR 290