1= OFX Programming Guide : Spatial Effects 2Author:Bruno Nicoletti 32014-10-17 4:toc: 5:data-uri: 6:source-highlighter: coderay 7 8This guide will introduce the spatial coordinate system used by OFX and will illustrate that with a 9simple circle drawing plugin. 10Its source can be found in the pass:[C++] 11file `Guide/Code/Example5/circle.cpp`. 12This plugin takes a clip and draws a circle over it. The colour, size and position of the circle 13are controlled by several parameters. 14 15== Spatial Coordinate Systems 16There are two main coordinate systems in OFX, these are the **pixel coordinate system** and the **canonical coordinate system**. 17I'll describe them both, but first a slight digression. 18 19=== Pixel Aspect Ratios 20Some display formats (for example standard definition PAL and NTSC) have non square pixels, which is quite annoying 21in my opinion. The https://en.wikipedia.org/wiki/Pixel_aspect_ratio[**pixel aspect ratio**] defines how non-square 22your pixel is. 23 24image::Pics/kinder720.jpg[ role = "thumb", align=center, title=Addressable Pixels In An PAL 16:9 Image] 25 26For example, a digital PAL 16:9 wide-screen image has 720 by 576 actual addressable pixels, however it has a pixel aspect 27ratio of ~1.42 footnote:[Yes, it can also be 1.46, depending on who you talk to. 28Today I'm picking 1.42 29to force an exact 16 by 9 aspect on a PAL's 720x576 pixels]. 30 31image::Pics/kinderCanonical.jpg[ role = "thumb", align=center, title=PAL 16:9 As It Appears On Your Telly] 32This means on the display device, each of those pixels 33are stretched horizontally by a factor of ~1.42. If it were square pixels, our displayed image would actually have 1024 34pixels. 35 36Looking at the two images above you can distinctly see the affect that the PAR has, the image appears 'squashed' when 37viewed as raw pixels, but these stretch out to look correct on our final display device. 38 39=== Render Scales 40Applications typically let a user generate low resolution proxy previews of their work, either to save time 41or space until they have finished. In OFX we call this applying a **render scale**, which has two values 42one for X and one for Y. 43 44If we were doing a half resolution proxy render of our PAL 16:9 project, we'd have a renderscale of (0.5,0.5), and 45360 by 288 addressable pixels in an image, with a PAR of 1.42. 46 47image::Pics/kinderProxy.jpg[ role = "thumb", align=center, title=Addressable Pixels In An PAL 16:9 Image at Half Proxy ] 48=== Coordinate Systems 49We call the coordinate system which allows us to index honest to goodness pixels in memory, the **pixel coordinate system**. 50It is usually represented 51with integers. For example the **bounds** of our image data are in pixel coordinates. 52 53Now expressing positions or sizes on the image plane in pixel coordinates is problematic, both because 54of pixel aspect ratios and render scales. The problem with sizes is that a circle drawn with a constant radius in pixel coordinates 55will not necessarily be circular on the final display device, it will be stretched by the pixel aspect ratio. Circles won't be round. 56 57A similar problem applies to render scales, when you say something is at a position, it has to be independent of the renderscale. 58 59To get around this, we have the **canonical coordinate system**. This is on an idealised image plane, where all pixels 60are square and we have no scales applied to any data. Canonical coordinates are typically represented by doubles. 61 62Back to our PAL 16:9 example. The canonical coordinate system for that image would always has x==0 at the left and x==1023 63at the right, circles are always appear to be round and the arithmetic is easy. We use the canonical coordinate system 64to express render scale and PAR invariant data. This is the coordinate system we express spatial parameters in. 65 66There was a third coordinate system, the **Normalised Coordinate System**, but in practice this proved to be problematic 67and has been deprecated. 68 69=== Mapping Between Coordinate Systems 70Obviously on render you will need to map parameters from Canonical Coordinates to the required Pixel Coordinates, or vice versa. That 71is fortunately very easy, you just need to do a bit of multiplying via the pixel aspect ratio and the renderscale. 72 73Now, given... 74 75 - the pixel aspect ratio as _PAR_, which is available on the `**kOfxImagePropPixelAspectRatio**` double property on images and clips, 76 - the renderscale, as _SX_ and _SY_ , which is passed with in `inArgs` to a variety of actions in the 2D double property `**kOfxImageEffectPropRenderScale**`, 77 78==== Mapping from Canonical Coordinates to Pixel Coordinate 79---- 80x' = x * SX/PAR; 81y' = y * SY; 82---- 83 84==== Mapping from Pixel Coordinate to Canonical Coordinates 85---- 86x' = x * PAR/SX; 87y' = y/SY; 88---- 89 90== Loading Our Plugin 91This plugin highlights the fact that the OFX API is really a way a plugin and a host can have a discussion so they 92can both figure out how they should operate. It allows plugins to modify their behaviour depending on what the host 93says it can do. 94 95Here is the source for the load action... 96 97[source, c++] 98.circle.cpp 99---- 100 //////////////////////////////////////////////////////////////////////////////// 101 // The first _action_ called after the binary is loaded (three boot strapper functions will be howeever) 102 OfxStatus LoadAction(void) 103 { 104 // fetch our three suites 105 FetchSuite(gPropertySuite, kOfxPropertySuite, 1); 106 FetchSuite(gImageEffectSuite, kOfxImageEffectSuite, 1); 107 FetchSuite(gParameterSuite, kOfxParameterSuite, 1); 108 109 int verSize = 0; 110 if(gPropertySuite->propGetDimension(gHost->host, kOfxPropAPIVersion, &verSize) == kOfxStatOK) { 111 verSize = verSize > 2 ? 2 : verSize; 112 gPropertySuite->propGetIntN(gHost->host, 113 kOfxPropAPIVersion, 114 2, 115 gAPIVersion); 116 } 117 118 // we only support 1.2 and above 119 if(gAPIVersion[0] == 1 && gAPIVersion[1] < 2) { 120 return kOfxStatFailed; 121 } 122 123 /// does the host support multi-resolution images 124 gPropertySuite->propGetInt(gHost->host, 125 kOfxImageEffectPropSupportsMultiResolution, 126 0, 127 &gHostSupportsMultiRes); 128 129 return kOfxStatOK; 130 } 131---- 132 133It fetches three suites then it checks to see if the **kOfxPropAPIVersion** property exists on the host, if it does it 134then checks that the version is at least "1.2", as we later rely on features only available in that version of the API. 135 136The next thing it does is to check that the host is supports multiple resolutions. This is short hand for saying that 137the host allows input and output clips to have different regions of definition, and images may be passed 138to the plugin that have differing bounds. This is also a property of the plugin descriptor, but we've left it at the default value, 139which is 'true', as our plugin does support multiple resolutions. 140 141We are checking for multiple resolution support to conditionally modify our plugin's behaviour in later actions. 142 143 144== Description 145Now, onto our plugin. The description action is pretty standard, as is the describe in context action. I'll just show you 146snippets of the interesting bits. 147 148Note, we are relying on a parameter type that is only available with the 1.2 version of OFX. Our plugin checks for this version 149of the API the host supports and will fail gracefully during the load action. 150 151[source, c++] 152.circle.cpp 153---- 154 // set the properties on the radius param 155 gParameterSuite->paramDefine(paramSet, 156 kOfxParamTypeDouble, 157 RADIUS_PARAM_NAME, 158 &radiusParamProps); 159 160 gPropertySuite->propSetString(radiusParamProps, 161 kOfxParamPropDoubleType, 162 0, 163 kOfxParamDoubleTypeX); 164 165 gPropertySuite->propSetString(radiusParamProps, 166 kOfxParamPropDefaultCoordinateSystem, 167 0, 168 kOfxParamCoordinatesNormalised); 169 170 gPropertySuite->propSetDouble(radiusParamProps, 171 kOfxParamPropDefault, 172 0, 173 0.25); 174 gPropertySuite->propSetDouble(radiusParamProps, 175 kOfxParamPropMin, 176 0, 177 0); 178 gPropertySuite->propSetDouble(radiusParamProps, 179 kOfxParamPropDisplayMin, 180 0, 181 0.0); 182 gPropertySuite->propSetDouble(radiusParamProps, 183 kOfxParamPropDisplayMax, 184 0, 185 2.0); 186 gPropertySuite->propSetString(radiusParamProps, 187 kOfxPropLabel, 188 0, 189 "Radius"); 190 gPropertySuite->propSetString(radiusParamProps, 191 kOfxParamPropHint, 192 0, 193 "The radius of the circle."); 194---- 195 196Here we are defining the parameter that controls the radius of our circle we will draw. It's a double param, and the type of double param is **kOfxParamDoubleTypeX**, 197footnote:[this double parameter type is only available API versions 1.2 or above] which 198says to the host, this represents a size in X in canonical coordinates. The host can display that however it like, but to the API, it needs to pass values back in canonical coordinates. 199 200The other thing we do is to set up the default value. Which is 0.25, which seems to be a mighty small circle, as is the display maximum value of 2.0. However, 201note the property **kOfxParamPropDefaultCoordinateSystem** being set to **kOfxParamCoordinatesNormalised**, this says that defaults/mins/maxes are being described 202relative to the project size. So our circle's radius will default to be a quarter of the nominal project size's x dimension. For a 1080 HD project, this would be a value of 480. 203 204[source, c++] 205.circle.cpp 206---- 207 // set the properties on the centre param 208 OfxPropertySetHandle centreParamProps; 209 static double centreDefault[] = {0.5, 0.5}; 210 211 gParameterSuite->paramDefine(paramSet, 212 kOfxParamTypeDouble2D, 213 CENTRE_PARAM_NAME, 214 ¢reParamProps); 215 216 gPropertySuite->propSetString(centreParamProps, 217 kOfxParamPropDoubleType, 218 0, 219 kOfxParamDoubleTypeXYAbsolute); 220 gPropertySuite->propSetString(centreParamProps, 221 kOfxParamPropDefaultCoordinateSystem, 222 0, 223 kOfxParamCoordinatesNormalised); 224 gPropertySuite->propSetDoubleN(centreParamProps, 225 kOfxParamPropDefault, 226 2, 227 centreDefault); 228 gPropertySuite->propSetString(centreParamProps, 229 kOfxPropLabel, 230 0, 231 "Centre"); 232 gPropertySuite->propSetString(centreParamProps, 233 kOfxParamPropHint, 234 0, 235 "The centre of the circle."); 236---- 237Here we are defining the parameter that controls the position of the centre of our circle. It's a 2D double parameter and we are telling the host that 238it represents an absolute position in the canonical coordinate system 239footnote:[this double parameter type is only available API versions 1.2 or above]. Some hosts will automatically add user interface handles for such parameters to 240let you simply drag such positions around. We are also setting the default values relative to the project size, and in this case (0.5, 0.5), it should 241appear in the centre of the final image. 242 243[source, c++] 244.circle.cpp 245---- 246 // set the properties on the colour param 247 OfxPropertySetHandle colourParamProps; 248 static double colourDefault[] = {1.0, 1.0, 1.0, 0.5}; 249 250 gParameterSuite->paramDefine(paramSet, 251 kOfxParamTypeRGBA, 252 COLOUR_PARAM_NAME, 253 &colourParamProps); 254 gPropertySuite->propSetDoubleN(colourParamProps, 255 kOfxParamPropDefault, 256 4, 257 colourDefault); 258 gPropertySuite->propSetString(colourParamProps, 259 kOfxPropLabel, 260 0, 261 "Colour"); 262 gPropertySuite->propSetString(centreParamProps, 263 kOfxParamPropHint, 264 0, 265 "The colour of the circle."); 266---- 267This is obvious, we are defining an RGBA parameter to control the colour and transparency of our circle. Colours are always normalised 0 to 1, so when 268you get and set the colour, you need to scale the values up to the nominal white point of your image, which is implicitly defined by the data type of 269the image. 270 271[source, c++] 272.circle.cpp 273---- 274 if(gHostSupportsMultiRes) { 275 OfxPropertySetHandle growRoDParamProps; 276 gParameterSuite->paramDefine(paramSet, 277 kOfxParamTypeBoolean, 278 GROW_ROD_PARAM_NAME, 279 &growRoDParamProps); 280 gPropertySuite->propSetInt(growRoDParamProps, 281 kOfxParamPropDefault, 282 0, 283 0); 284 gPropertySuite->propSetString(growRoDParamProps, 285 kOfxParamPropHint, 286 0, 287 "Whether to grow the output's Region of Definition to include the circle."); 288 gPropertySuite->propSetString(growRoDParamProps, 289 kOfxPropLabel, 290 0, 291 "Grow RoD"); 292 } 293---- 294Finally, we are conditionally defining a boolean parameter that controls whether our circle affects the region of definition 295of our output image. We only able to modify the region of definition if the host has an architecture that supports that 296behaviour, which we checked at load time where we set the **gHostSupportsMultiRes** global variable. 297 298== Get Region Of Definition Action 299What is this region of definition action? Easy, an effect and a clip have a region of definition (RoD). This is the 300maximum rectangle for which an effect or clip can produce pixels. You can ask for RoD of a clip via the `**clipGetRegionOfDefinition**` 301function in the image effect suite. The RoD is currently defined in canonical coordinates footnote:[we are debating whether to modifying 302that to be in pixel coordinates]. 303 304Note that the RoD is independent of the **bounds** of a image, an image's bounds may be less than, more than or equal to the RoD. It is up to 305host how or why it wants to manage the RoD differently. As noted above, some hosts don't have the ability to do any such thing. 306 307By default the RoD of the output is the union of all the RoDs from all the mandatory input clips. In our example, we want to 308be able to set the RoD to be the union of the input clip with the area the circle we are drawing. Whether we do that or not is controlled by 309the "growRoD" parameter which is conditionally defined in the describe in context action. 310 311To set the output rod, we need to trap the `**kOfxImageEffectActionGetRegionOfDefinition**` action. Our MainEntry function now has an 312extra conditional in there.... 313 314[source, c++] 315.circle.cpp 316---- 317 ... 318 else if(gHostSupportsMultiRes && strcmp(action, kOfxImageEffectActionGetRegionOfDefinition) == 0) { 319 returnStatus = GetRegionOfDefinitionAction(effect, inArgs, outArgs); 320 } 321 ... 322---- 323 324Note that we dont trap this on hosts that aren't multi-resolution, as by definition on those hosts RoDs are fixed. 325 326The code for the action itself is quite simple... 327 328.circle.cpp 329---- 330 // tells the host what region we are capable of filling 331 OfxStatus 332 GetRegionOfDefinitionAction( OfxImageEffectHandle effect, 333 OfxPropertySetHandle inArgs, 334 OfxPropertySetHandle outArgs) 335 { 336 // retrieve any instance data associated with this effect 337 MyInstanceData *myData = FetchInstanceData(effect); 338 339 OfxTime time; 340 gPropertySuite->propGetDouble(inArgs, kOfxPropTime, 0, &time); 341 342 int growingRoD; 343 gParameterSuite->paramGetValueAtTime(myData->growRoD, time, 344 &growingRoD); 345 346 // are we growing the RoD to include the circle? 347 if(not growingRoD) { 348 return kOfxStatReplyDefault; 349 } 350 else { 351 double radius = 0.0; 352 gParameterSuite->paramGetValueAtTime(myData->radiusParam, time, 353 &radius); 354 355 double centre[2]; 356 gParameterSuite->paramGetValueAtTime(myData->centreParam, time, 357 ¢re[0], 358 ¢re[1]); 359 360 // get the source rod 361 OfxRectD rod; 362 gImageEffectSuite->clipGetRegionOfDefinition(myData->sourceClip, time, &rod); 363 364 if(rod.x1 > centre[0] - radius) rod.x1 = centre[0] - radius; 365 if(rod.y1 > centre[1] - radius) rod.y1 = centre[1] - radius; 366 367 if(rod.x2 < centre[0] + radius) rod.x2 = centre[0] + radius; 368 if(rod.y2 < centre[1] + radius) rod.y2 = centre[1] + radius; 369 370 // set the rod in the out args 371 gPropertySuite->propSetDoubleN(outArgs, kOfxImageEffectPropRegionOfDefinition, 4, &rod.x1); 372 373 // and say we trapped the action and we are at the identity 374 return kOfxStatOK; 375 } 376 } 377---- 378 379We are being asked to calculate the RoD at a specific time, which means that RoDs are time varying in 380OFX. 381 382We check our "growRoD" parameter to see if we are going to actually modify the RoD. If we do, we find out, in canonical coordinates, 383where we are drawing our circle. We then fetch the region of definition and make a union of those two 384regions. We then set the **kOfxImageEffectPropRegionOfDefinition** return property on **outArgs** and say 385that we trapped the action. 386 387All fairly easy. 388 389== Is Identity Action 390Our identity checking action is fairly obvious, we check to see if our circle has a non zero radius, and to see if we are 391not growing the RoD and our circle is outside the RoD. 392 393== Rendering 394The action code is fairly boiler plate, it fetches parameter values and images from clips before calling the templated 395PixelProcessing function. Which is below... 396 397.circle.cpp 398---- 399 template <class T, int MAX> 400 void PixelProcessing(OfxImageEffectHandle instance, 401 Image &src, 402 Image &output, 403 double centre[2], 404 double radius, 405 double colour[4], 406 double renderScale[2], 407 OfxRectI renderWindow) 408 { // pixel aspect of our output 409 float PAR = output.pixelAspectRatio(); 410 411 T colourQuantised[4]; 412 for(int c = 0; c < 4; ++c) { 413 colourQuantised[c] = Clamp<T, MAX>(colour[c] * MAX); 414 } 415 416 // now do some processing 417 for(int y = renderWindow.y1; y < renderWindow.y2; y++) { 418 if(y % 20 == 0 && gImageEffectSuite->abort(instance)) break; 419 420 // get our y coord in canonical space 421 float yCanonical = (y + 0.5f)/renderScale[1]; 422 423 // how far are we from the centre in y, canonical 424 float dy = yCanonical - centre[1]; 425 426 // get the row start for the output image 427 T *dstPix = output.pixelAddress<T>(renderWindow.x1, y); 428 429 for(int x = renderWindow.x1; x < renderWindow.x2; x++) { 430 // get our x pixel coord in canonical space, 431 float xCanonical = (x + 0.5) * PAR/renderScale[0]; 432 433 // how far are we from the centre in x, canonical 434 float dx = xCanonical - centre[0]; 435 436 // distance to the centre of our circle, canonical 437 float d = sqrtf(dx * dx + dy * dy); 438 439 // this will hold the antialiased value 440 float alpha = colour[3]; 441 442 // Is the square of the distance to the centre 443 // less than the square of the radius? 444 if(d < radius) { 445 if(d > radius - 1) { 446 // we are within 1 pixel of the edge, modulate 447 // our alpha with an anti-aliasing value 448 alpha *= radius - d; 449 } 450 } 451 else { 452 // outside, so alpha is 0 453 alpha = 0; 454 } 455 456 // get the source pixel 457 const T *srcPix = src.pixelAddressWithFallback<T>(x, y); 458 459 // scale each component around that average 460 for(int c = 0; c < output.nComponents(); ++c) { 461 // use the mask to control how much original we should have 462 dstPix[c] = Blend(srcPix[c], colourQuantised[c], alpha); 463 } 464 dstPix += output.nComponents(); 465 } 466 } 467 } 468---- 469Please don't think I actually write production code as slow as this, I'm just making the whole thing as clear as possible 470in my example. 471 472The first thing we do is to scale the normalised value for our circle colour up to a quantised value based on our data type. So 473multiplying up by 255 for 8 bit data types, 65536 for 16bit ints and 1 for floats. 474 475To draw the circle we are transforming a pixel's position in pixel space into a canonical coordinate. We then calculate 476the distance to the centre of the circle, again in canonical coordinates. We use that distance to see if we are inside or out 477of the circle, with a bit of anti-aliasing thrown in. This gives us a normalised alpha value. 478 479Our output value is our source pixel blended with our circle colour based on the intensity of the calculated alpha. 480 481 482== Summary 483This example plugin has shown ... 484 485 - the two main OFX spatial coordinate systems, 486 - how to use the region of definition action, 487 - that the API is a negotiation between a host and a plugiun, 488 - mapping between coordinate systems for rendering. 489 490 491
