1.. _rfc-35:
2
3================================================================================
4RFC 35: Delete, reorder and alter field definitions of OGR layers
5================================================================================
6
7Authors: Even Rouault
8
9Contact: even dot rouault at spatialys.com
10
11Status: Adopted
12
13Summary
14-------
15
16This document proposes changes in OGR to add the capability to delete
17fields, reorder fields and alter field definitions, in OGR layer
18definitions.
19
20Rationale
21---------
22
23Currently, an OGR layer definition can only be altered to add a new
24field definition with OGRLayer::CreateField().
25
26It is desirable to extend OGR capabilities to be able to delete, reorder
27and alter field definitions of existing layers. Such wish has been
28expressed in ticket #2671 and comes back regularly on QGIS mailing list
29(e.g.
30`http://lists.osgeo.org/pipermail/qgis-user/2011-May/011935.html <http://lists.osgeo.org/pipermail/qgis-user/2011-May/011935.html>`__).
31QGIS currently has a "Table Manager" extension to work around the lack
32of DeleteField(), so a proper solution is clearly needed.
33
34Planned Changes
35---------------
36
37The OGRLayer class will be extended with the following methods :
38
39::
40
41       virtual OGRErr      DeleteField( int iField );
42       virtual OGRErr      ReorderFields( int* panMap );
43       virtual OGRErr      AlterFieldDefn( int iField, OGRFieldDefn* poNewFieldDefn, int nFlags );
44
45       /* non virtual : conveniency wrapper for ReorderFields() */
46       OGRErr              ReorderField( int iOldFieldPos, int iNewFieldPos );
47
48The documentation of those new methods is :
49
50::
51
52
53   /**
54   \fn OGRErr OGRLayer::DeleteField( int iField );
55
56   \brief Delete an existing field on a layer.
57
58   You must use this to delete existing fields
59   on a real layer. Internally the OGRFeatureDefn for the layer will be updated
60   to reflect the deleted field.  Applications should never modify the OGRFeatureDefn
61   used by a layer directly.
62
63   This method should not be called while there are feature objects in existence that
64   were obtained or created with the previous layer definition.
65
66   Not all drivers support this method. You can query a layer to check if it supports it
67   with the OLCDeleteField capability. Some drivers may only support this method while
68   there are still no features in the layer. When it is supported, the existings features of the
69   backing file/database should be updated accordingly.
70
71   This function is the same as the C function OGR_L_DeleteField().
72
73   @param iField index of the field to delete.
74
75   @return OGRERR_NONE on success.
76
77   @since OGR 1.9.0
78   */
79
80   /**
81   \fn OGRErr OGRLayer::ReorderFields( int* panMap );
82
83   \brief Reorder all the fields of a layer.
84
85   You must use this to reorder existing fields
86   on a real layer. Internally the OGRFeatureDefn for the layer will be updated
87   to reflect the reordering of the fields.  Applications should never modify the OGRFeatureDefn
88   used by a layer directly.
89
90   This method should not be called while there are feature objects in existence that
91   were obtained or created with the previous layer definition.
92
93   panMap is such that,for each field definition at position i after reordering,
94   its position before reordering was panMap[i].
95
96   For example, let suppose the fields were "0","1","2","3","4" initially.
97   ReorderFields([0,2,3,1,4]) will reorder them as "0","2","3","1","4".
98
99   Not all drivers support this method. You can query a layer to check if it supports it
100   with the OLCReorderFields capability. Some drivers may only support this method while
101   there are still no features in the layer. When it is supported, the existings features of the
102   backing file/database should be updated accordingly.
103
104   This function is the same as the C function OGR_L_ReorderFields().
105
106   @param panMap an array of GetLayerDefn()->GetFieldCount() elements which
107   is a permutation of [0, GetLayerDefn()->GetFieldCount()-1].
108
109   @return OGRERR_NONE on success.
110
111   @since OGR 1.9.0
112   */
113
114   /**
115   \fn OGRErr OGRLayer::ReorderField( int iOldFieldPos, int iNewFieldPos );
116
117   \brief Reorder an existing field on a layer.
118
119   This method is a conveniency wrapper of ReorderFields() dedicated to move a single field.
120   It is a non-virtual method, so drivers should implement ReorderFields() instead.
121
122   You must use this to reorder existing fields
123   on a real layer. Internally the OGRFeatureDefn for the layer will be updated
124   to reflect the reordering of the fields.  Applications should never modify the OGRFeatureDefn
125   used by a layer directly.
126
127   This method should not be called while there are feature objects in existence that
128   were obtained or created with the previous layer definition.
129
130   The field definition that was at initial position iOldFieldPos will be moved at
131   position iNewFieldPos, and elements between will be shuffled accordingly.
132
133   For example, let suppose the fields were "0","1","2","3","4" initially.
134   ReorderField(1, 3) will reorder them as "0","2","3","1","4".
135
136   Not all drivers support this method. You can query a layer to check if it supports it
137   with the OLCReorderFields capability. Some drivers may only support this method while
138   there are still no features in the layer. When it is supported, the existings features of the
139   backing file/database should be updated accordingly.
140
141   This function is the same as the C function OGR_L_ReorderField().
142
143   @param iOldFieldPos previous position of the field to move. Must be in the range [0,GetFieldCount()-1].
144   @param iNewFieldPos new position of the field to move. Must be in the range [0,GetFieldCount()-1].
145
146   @return OGRERR_NONE on success.
147
148   @since OGR 1.9.0
149   */
150
151   /**
152   \fn OGRErr OGRLayer::AlterFieldDefn( int iField, OGRFieldDefn* poNewFieldDefn, int nFlags );
153
154   \brief Alter the definition of an existing field on a layer.
155
156   You must use this to alter the definition of an existing field of a real layer.
157   Internally the OGRFeatureDefn for the layer will be updated
158   to reflect the altered field.  Applications should never modify the OGRFeatureDefn
159   used by a layer directly.
160
161   This method should not be called while there are feature objects in existence that
162   were obtained or created with the previous layer definition.
163
164   Not all drivers support this method. You can query a layer to check if it supports it
165   with the OLCAlterFieldDefn capability. Some drivers may only support this method while
166   there are still no features in the layer. When it is supported, the existings features of the
167   backing file/database should be updated accordingly. Some drivers might also not support
168   all update flags.
169
170   This function is the same as the C function OGR_L_AlterFieldDefn().
171
172   @param iField index of the field whose definition must be altered.
173   @param poNewFieldDefn new field definition
174   @param nFlags combination of ALTER_NAME_FLAG, ALTER_TYPE_FLAG and ALTER_WIDTH_PRECISION_FLAG
175   to indicate which of the name and/or type and/or width and precision fields from the new field
176   definition must be taken into account.
177
178   @return OGRERR_NONE on success.
179
180   @since OGR 1.9.0
181   */
182
183Three new layer capabilities are added :
184
185::
186
187   OLCDeleteField / "DeleteField": TRUE if this layer can delete
188   existing fields on the current layer using DeleteField(), otherwise FALSE.
189
190   OLCReorderFields / "ReorderFields": TRUE if this layer can reorder
191   existing fields on the current layer using ReorderField() or ReorderFields(), otherwise FALSE.
192
193   OLCAlterFieldDefn / "AlterFieldDefn": TRUE if this layer can alter
194   the definition of an existing field on the current layer using AlterFieldDefn(), otherwise FALSE.
195
196The new methods are mapped to the C API :
197
198::
199
200   OGRErr CPL_DLL OGR_L_DeleteField( OGRLayerH, int iField );
201   OGRErr CPL_DLL OGR_L_ReorderFields( OGRLayerH, int* panMap );
202   OGRErr CPL_DLL OGR_L_ReorderField( OGRLayerH, int iOldFieldPos, int iNewFieldPos );
203   OGRErr CPL_DLL OGR_L_AlterFieldDefn( OGRLayerH, int iField, OGRFieldDefnH hNewFieldDefn, int nFlags );
204
205For the purpose of the implementation, new methods are also added to the
206OGRFeatureDefn class :
207
208::
209
210       OGRErr      DeleteFieldDefn( int iField );
211       OGRErr      ReorderFieldDefns( int* panMap );
212
213A OGRErr OGRCheckPermutation(int\* panPermutation, int nSize) function
214is added to ogrutils.cpp to check that the array is a permutation of
215[0,nSize-1]. It is used by OGRFeatureDefn::ReorderFieldDefns() and can
216be used by all drivers implementing OGRLayer::ReorderFields() to
217validate the panMap argument.
218
219Altering field types
220--------------------
221
222This RFC does not attempt to guarantee which type conversions will be
223possible. It will depend on the capabilities of the implementing
224drivers. For example, for database drivers, the operation will be
225directly done on the server side (through a 'ALTER TABLE my_table ALTER
226COLUMN my_column TYPE new_type' command for the PG driver). So some
227conversions might be possible, others not...
228
229It is however expected that converting from any type to OFTString will
230be supported in most cases when AlterFieldDefn() is supported.
231
232Drivers that don't support a conversion and that were required to do it
233(ALTER_TYPE_FLAG set and new_type != old_type) should emit an explicit
234error.
235
236Compatibility Issues
237--------------------
238
239None
240
241Changed drivers
242---------------
243
244The shapefile driver will implement DeleteField(), ReorderFields() and
245AlterFieldDefn(). Shapelib will be extended with DBFReorderFields() and
246DBFAlterFieldDefn().
247
248Note: The implementation of AlterFieldDefn() in the Shapefile driver
249does not support altering the field type, except when converting to
250OFTString. It will not reformat numeric values of existing features if
251width or precision are changed. However, appropriate field truncation or
252expansion will occur if the width is altered.
253
254Other drivers, mainly database drivers (PG, MySQL, SQLite), could be
255easily extended to implement the new API by issuing the appropriate SQL
256command (ALTER TABLE foo DROP COLUMN bar, ALTER TABLE foo ALTER COLUMN
257bar, ...). The implementation of DeleteField() and AlterFieldDefn() in
258the PG driver is indeed planned, provided this RFC is adopted. The
259Memory driver will also updated to support DeleteField(),
260ReorderFields() and AlterFieldDefn().
261
262SWIG bindings
263-------------
264
265DeleteField(), ReorderField(), ReorderFields() and AlterFieldDefn() will
266be mapped to SWIG.
267
268Test Suite
269----------
270
271The autotest suite will be extended to test the implementation of the
272new API for the Shapefile driver. An example of the use of the new API
273is attached to ticket #2671
274(`rfc35_test.py <http://trac.osgeo.org/gdal/attachment/ticket/2671/rfc35_test.py>`__)
275and will be turned into unit tests.
276
277Implementation
278--------------
279
280Implementation will be done by Even Rouault in GDAL/OGR trunk. Changes
281in Shapelib will need to be pushed into upstream CVS by a Shapelib
282committer. The proposed implementation is attached as a patch in ticket
283#2671
284(`rfc35_v3.patch <http://trac.osgeo.org/gdal/attachment/ticket/2671/rfc35_v3.patch>`__).
285
286Voting history
287--------------
288
289+1 from FrankW, DanielM, HowardB, TamasS and EvenR
290