1src/backend/utils/fmgr/README 2 3Function Manager 4================ 5 6[This file originally explained the transition from the V0 to the V1 7interface. Now it just explains some internals and rationale for the V1 8interface, while the V0 interface has been removed.] 9 10The V1 Function-Manager Interface 11--------------------------------- 12 13The core of the design is data structures for representing the result of a 14function lookup and for representing the parameters passed to a specific 15function invocation. (We want to keep function lookup separate from 16function call, since many parts of the system apply the same function over 17and over; the lookup overhead should be paid once per query, not once per 18tuple.) 19 20 21When a function is looked up in pg_proc, the result is represented as 22 23typedef struct 24{ 25 PGFunction fn_addr; /* pointer to function or handler to be called */ 26 Oid fn_oid; /* OID of function (NOT of handler, if any) */ 27 short fn_nargs; /* number of input args (0..FUNC_MAX_ARGS) */ 28 bool fn_strict; /* function is "strict" (NULL in => NULL out) */ 29 bool fn_retset; /* function returns a set (over multiple calls) */ 30 unsigned char fn_stats; /* collect stats if track_functions > this */ 31 void *fn_extra; /* extra space for use by handler */ 32 MemoryContext fn_mcxt; /* memory context to store fn_extra in */ 33 Node *fn_expr; /* expression parse tree for call, or NULL */ 34} FmgrInfo; 35 36For an ordinary built-in function, fn_addr is just the address of the C 37routine that implements the function. Otherwise it is the address of a 38handler for the class of functions that includes the target function. 39The handler can use the function OID and perhaps also the fn_extra slot 40to find the specific code to execute. (fn_oid = InvalidOid can be used 41to denote a not-yet-initialized FmgrInfo struct. fn_extra will always 42be NULL when an FmgrInfo is first filled by the function lookup code, but 43a function handler could set it to avoid making repeated lookups of its 44own when the same FmgrInfo is used repeatedly during a query.) fn_nargs 45is the number of arguments expected by the function, fn_strict is its 46strictness flag, and fn_retset shows whether it returns a set; all of 47these values come from the function's pg_proc entry. fn_stats is also 48set up to control whether or not to track runtime statistics for calling 49this function. 50 51If the function is being called as part of a SQL expression, fn_expr will 52point to the expression parse tree for the function call; this can be used 53to extract parse-time knowledge about the actual arguments. Note that this 54field really is information about the arguments rather than information 55about the function, but it's proven to be more convenient to keep it in 56FmgrInfo than in FunctionCallInfoBaseData where it might more logically go. 57 58 59During a call of a function, the following data structure is created 60and passed to the function: 61 62typedef struct 63{ 64 FmgrInfo *flinfo; /* ptr to lookup info used for this call */ 65 Node *context; /* pass info about context of call */ 66 Node *resultinfo; /* pass or return extra info about result */ 67 Oid fncollation; /* collation for function to use */ 68 bool isnull; /* function must set true if result is NULL */ 69 short nargs; /* # arguments actually passed */ 70 NullableDatum args[]; /* Arguments passed to function */ 71} FunctionCallInfoBaseData; 72typedef FunctionCallInfoBaseData* FunctionCallInfo; 73 74flinfo points to the lookup info used to make the call. Ordinary functions 75will probably ignore this field, but function class handlers will need it 76to find out the OID of the specific function being called. 77 78context is NULL for an "ordinary" function call, but may point to additional 79info when the function is called in certain contexts. (For example, the 80trigger manager will pass information about the current trigger event here.) 81If context is used, it should point to some subtype of Node; the particular 82kind of context is indicated by the node type field. (A callee should 83always check the node type before assuming it knows what kind of context is 84being passed.) fmgr itself puts no other restrictions on the use of this 85field. 86 87resultinfo is NULL when calling any function from which a simple Datum 88result is expected. It may point to some subtype of Node if the function 89returns more than a Datum. (For example, resultinfo is used when calling a 90function that returns a set, as discussed below.) Like the context field, 91resultinfo is a hook for expansion; fmgr itself doesn't constrain the use 92of the field. 93 94fncollation is the input collation derived by the parser, or InvalidOid 95when there are no inputs of collatable types or they don't share a common 96collation. This is effectively a hidden additional argument, which 97collation-sensitive functions can use to determine their behavior. 98 99nargs and args[] hold the arguments being passed to the function. 100Notice that all the arguments passed to a function (as well as its result 101value) will now uniformly be of type Datum. As discussed below, callers 102and callees should apply the standard Datum-to-and-from-whatever macros 103to convert to the actual argument types of a particular function. The 104value in args[i].value is unspecified when args[i].isnull is true. 105 106It is generally the responsibility of the caller to ensure that the 107number of arguments passed matches what the callee is expecting; except 108for callees that take a variable number of arguments, the callee will 109typically ignore the nargs field and just grab values from args[]. 110 111The isnull field will be initialized to "false" before the call. On 112return from the function, isnull is the null flag for the function result: 113if it is true the function's result is NULL, regardless of the actual 114function return value. Note that simple "strict" functions can ignore 115both isnull and args[i].isnull, since they won't even get called when there 116are any TRUE values in args[].isnull. 117 118FunctionCallInfo replaces FmgrValues plus a bunch of ad-hoc parameter 119conventions, global variables (fmgr_pl_finfo and CurrentTriggerData at 120least), and other uglinesses. 121 122 123Callees, whether they be individual functions or function handlers, 124shall always have this signature: 125 126Datum function (FunctionCallInfo fcinfo); 127 128which is represented by the typedef 129 130typedef Datum (*PGFunction) (FunctionCallInfo fcinfo); 131 132The function is responsible for setting fcinfo->isnull appropriately 133as well as returning a result represented as a Datum. Note that since 134all callees will now have exactly the same signature, and will be called 135through a function pointer declared with exactly that signature, we 136should have no portability or optimization problems. 137 138 139Function Coding Conventions 140--------------------------- 141 142Here are the proposed macros and coding conventions: 143 144The definition of an fmgr-callable function will always look like 145 146Datum 147function_name(PG_FUNCTION_ARGS) 148{ 149 ... 150} 151 152"PG_FUNCTION_ARGS" just expands to "FunctionCallInfo fcinfo". The main 153reason for using this macro is to make it easy for scripts to spot function 154definitions. However, if we ever decide to change the calling convention 155again, it might come in handy to have this macro in place. 156 157A nonstrict function is responsible for checking whether each individual 158argument is null or not, which it can do with PG_ARGISNULL(n) (which is 159just "fcinfo->args[n].isnull"). It should avoid trying to fetch the value 160of any argument that is null. 161 162Both strict and nonstrict functions can return NULL, if needed, with 163 PG_RETURN_NULL(); 164which expands to 165 { fcinfo->isnull = true; return (Datum) 0; } 166 167Argument values are ordinarily fetched using code like 168 int32 name = PG_GETARG_INT32(number); 169 170For float4, float8, and int8, the PG_GETARG macros will hide whether the 171types are pass-by-value or pass-by-reference. For example, if float8 is 172pass-by-reference then PG_GETARG_FLOAT8 expands to 173 (* (float8 *) DatumGetPointer(fcinfo->args[number].value)) 174and would typically be called like this: 175 float8 arg = PG_GETARG_FLOAT8(0); 176For what are now historical reasons, the float-related typedefs and macros 177express the type width in bytes (4 or 8), whereas we prefer to label the 178widths of integer types in bits. 179 180Non-null values are returned with a PG_RETURN_XXX macro of the appropriate 181type. For example, PG_RETURN_INT32 expands to 182 return Int32GetDatum(x) 183PG_RETURN_FLOAT4, PG_RETURN_FLOAT8, and PG_RETURN_INT64 hide whether their 184data types are pass-by-value or pass-by-reference, by doing a palloc if 185needed. 186 187fmgr.h will provide PG_GETARG and PG_RETURN macros for all the basic data 188types. Modules or header files that define specialized SQL datatypes 189(eg, timestamp) should define appropriate macros for those types, so that 190functions manipulating the types can be coded in the standard style. 191 192For non-primitive data types (particularly variable-length types) it won't 193be very practical to hide the pass-by-reference nature of the data type, 194so the PG_GETARG and PG_RETURN macros for those types won't do much more 195than DatumGetPointer/PointerGetDatum plus the appropriate typecast (but see 196TOAST discussion, below). Functions returning such types will need to 197palloc() their result space explicitly. I recommend naming the GETARG and 198RETURN macros for such types to end in "_P", as a reminder that they 199produce or take a pointer. For example, PG_GETARG_TEXT_P yields "text *". 200 201When a function needs to access fcinfo->flinfo or one of the other auxiliary 202fields of FunctionCallInfo, it should just do it. I doubt that providing 203syntactic-sugar macros for these cases is useful. 204 205 206Support for TOAST-Able Data Types 207--------------------------------- 208 209For TOAST-able data types, the PG_GETARG macro will deliver a de-TOASTed 210data value. There might be a few cases where the still-toasted value is 211wanted, but the vast majority of cases want the de-toasted result, so 212that will be the default. To get the argument value without causing 213de-toasting, use PG_GETARG_RAW_VARLENA_P(n). 214 215Some functions require a modifiable copy of their input values. In these 216cases, it's silly to do an extra copy step if we copied the data anyway 217to de-TOAST it. Therefore, each toastable datatype has an additional 218fetch macro, for example PG_GETARG_TEXT_P_COPY(n), which delivers a 219guaranteed-fresh copy, combining this with the detoasting step if possible. 220 221There is also a PG_FREE_IF_COPY(ptr,n) macro, which pfree's the given 222pointer if and only if it is different from the original value of the n'th 223argument. This can be used to free the de-toasted value of the n'th 224argument, if it was actually de-toasted. Currently, doing this is not 225necessary for the majority of functions because the core backend code 226releases temporary space periodically, so that memory leaked in function 227execution isn't a big problem. However, as of 7.1 memory leaks in 228functions that are called by index searches will not be cleaned up until 229end of transaction. Therefore, functions that are listed in pg_amop or 230pg_amproc should be careful not to leak detoasted copies, and so these 231functions do need to use PG_FREE_IF_COPY() for toastable inputs. 232 233A function should never try to re-TOAST its result value; it should just 234deliver an untoasted result that's been palloc'd in the current memory 235context. When and if the value is actually stored into a tuple, the 236tuple toaster will decide whether toasting is needed. 237 238 239Functions Accepting or Returning Sets 240------------------------------------- 241 242If a function is marked in pg_proc as returning a set, then it is called 243with fcinfo->resultinfo pointing to a node of type ReturnSetInfo. A 244function that desires to return a set should raise an error "called in 245context that does not accept a set result" if resultinfo is NULL or does 246not point to a ReturnSetInfo node. 247 248There are currently two modes in which a function can return a set result: 249value-per-call, or materialize. In value-per-call mode, the function returns 250one value each time it is called, and finally reports "done" when it has no 251more values to return. In materialize mode, the function's output set is 252instantiated in a Tuplestore object; all the values are returned in one call. 253Additional modes might be added in future. 254 255ReturnSetInfo contains a field "allowedModes" which is set (by the caller) 256to a bitmask that's the OR of the modes the caller can support. The actual 257mode used by the function is returned in another field "returnMode". For 258backwards-compatibility reasons, returnMode is initialized to value-per-call 259and need only be changed if the function wants to use a different mode. 260The function should ereport() if it cannot use any of the modes the caller is 261willing to support. 262 263Value-per-call mode works like this: ReturnSetInfo contains a field 264"isDone", which should be set to one of these values: 265 266 ExprSingleResult /* expression does not return a set */ 267 ExprMultipleResult /* this result is an element of a set */ 268 ExprEndResult /* there are no more elements in the set */ 269 270(the caller will initialize it to ExprSingleResult). If the function simply 271returns a Datum without touching ReturnSetInfo, then the call is over and a 272single-item set has been returned. To return a set, the function must set 273isDone to ExprMultipleResult for each set element. After all elements have 274been returned, the next call should set isDone to ExprEndResult and return a 275null result. (Note it is possible to return an empty set by doing this on 276the first call.) 277 278Value-per-call functions MUST NOT assume that they will be run to completion; 279the executor might simply stop calling them, for example because of a LIMIT. 280Therefore, it's unsafe to attempt to perform any resource cleanup in the 281final call. It's usually not necessary to clean up memory, anyway. If it's 282necessary to clean up other types of resources, such as file descriptors, 283one can register a shutdown callback function in the ExprContext pointed to 284by the ReturnSetInfo node. (But note that file descriptors are a limited 285resource, so it's generally unwise to hold those open across calls; SRFs 286that need file access are better written to do it in a single call using 287Materialize mode.) 288 289Materialize mode works like this: the function creates a Tuplestore holding 290the (possibly empty) result set, and returns it. There are no multiple calls. 291The function must also return a TupleDesc that indicates the tuple structure. 292The Tuplestore and TupleDesc should be created in the context 293econtext->ecxt_per_query_memory (note this will *not* be the context the 294function is called in). The function stores pointers to the Tuplestore and 295TupleDesc into ReturnSetInfo, sets returnMode to indicate materialize mode, 296and returns null. isDone is not used and should be left at ExprSingleResult. 297 298The Tuplestore must be created with randomAccess = true if 299SFRM_Materialize_Random is set in allowedModes, but it can (and preferably 300should) be created with randomAccess = false if not. Callers that can support 301both ValuePerCall and Materialize mode will set SFRM_Materialize_Preferred, 302or not, depending on which mode they prefer. 303 304If available, the expected tuple descriptor is passed in ReturnSetInfo; 305in other contexts the expectedDesc field will be NULL. The function need 306not pay attention to expectedDesc, but it may be useful in special cases. 307 308There is no support for functions accepting sets; instead, the function will 309be called multiple times, once for each element of the input set. 310 311 312Notes About Function Handlers 313----------------------------- 314 315Handlers for classes of functions should find life much easier and 316cleaner in this design. The OID of the called function is directly 317reachable from the passed parameters; we don't need the global variable 318fmgr_pl_finfo anymore. Also, by modifying fcinfo->flinfo->fn_extra, 319the handler can cache lookup info to avoid repeat lookups when the same 320function is invoked many times. (fn_extra can only be used as a hint, 321since callers are not required to re-use an FmgrInfo struct. 322But in performance-critical paths they normally will do so.) 323 324If the handler wants to allocate memory to hold fn_extra data, it should 325NOT do so in CurrentMemoryContext, since the current context may well be 326much shorter-lived than the context where the FmgrInfo is. Instead, 327allocate the memory in context flinfo->fn_mcxt, or in a long-lived cache 328context. fn_mcxt normally points at the context that was 329CurrentMemoryContext at the time the FmgrInfo structure was created; 330in any case it is required to be a context at least as long-lived as the 331FmgrInfo itself. 332