1Resampling {#dev_guide_resampling} 2===================================== 3 4> 5> [API reference](@ref dnnl_api_resampling) 6> 7 8## General 9 10The resampling primitive computes forward or backward resampling operation on 111D, 2D, or 3D spatial data. Resampling performs spatial scaling of original 12tensor using one of the supported interpolation algorithms: 13- Nearest Neighbor 14- Linear (or Bilinear for 2D spatial tensor, Trilinear for 3D spatial tensor). 15 16Resampling operation is defined by the source tensor and scaling factors in 17each spatial dimension. Upsampling and downsampling are the alternative terms 18for resampling that are used when all scaling factors are greater (upsampling) 19or less (downsampling) than one. 20 21The resampling operation is defined by the following formulas. We show formulas 22only for 2D spatial data which are straightforward to generalize to cases of 23higher and lower dimensions. Variable names follow the standard 24@ref dev_guide_conventions. 25 26Let \src and \dst be \f$N \times C \times IH \times IW\f$ and \f$N 27\times C \times OH \times OW\f$ tensors respectively. Let 28\f$ F_h = \frac{OH}{IH} \f$ and \f$ F_w = \frac{OW}{IW} \f$ define scaling 29factors in each spatial dimension. 30 31The following formulas show how oneDNN computes resampling for nearest neighbor 32and bilinear interpolation methods. 33To further simplify the formulas, we assume the following: 34\f$\src(n, ic, ih, iw) = \begin{cases} 35\src(n, ic, ih, 0), & \text{if}\ iw < 0 \\ 36\src(n, ic, ih, iw), & \text{if}\ IW - 1 \geq iw \geq 0 \\ 37\src(n, ic, ih, IW - 1), & \text{if}\ iw > IW - 1 38\end{cases}\f$ 39 40Same assumptions apply for \f$ih\f$. Definitions of \f$ih\f$ and \f$iw\f$ are 41provided below with a correspondent algorithm. 42 43### Forward 44 45#### Nearest Neighbor Resampling 46 47\f[\dst(n, c, oh, ow) = \src(n, c, ih, iw)\f] 48 49where 50 51- \f$ih = [\frac{oh + 0.5} {F_h} - 0.5]\f$, 52- \f$iw = [\frac{ow + 0.5} {F_w} - 0.5]\f$. 53 54#### Bilinear Resampling 55 56\f[ 57 \dst(n, c, oh, ow) = 58 \src(n, c, ih_0, iw_0) \cdot (1 - W_{ih}) \cdot (1 - W_{iw}) + \\ 59 \src(n, c, ih_1, iw_0) \cdot W_{ih} \cdot (1 - W_{iw}) + \\ 60 \src(n, c, ih_0, iw_1) \cdot (1 - W_{ih}) \cdot W_{iw} + \\ 61 \src(n, c, ih_1, iw_1) \cdot W_{ih} \cdot W_{iw} \\ 62\f] 63 64where 65- \f$ih_0 = \left\lfloor{\frac {oh + 0.5} {F_h} - 0.5}\right\rfloor\f$, 66- \f$ih_1 = \left\lceil {\frac {oh + 0.5} {F_h} - 0.5}\right\rceil\f$, 67- \f$iw_0 = \left\lfloor{\frac {ow + 0.5} {F_w} - 0.5}\right\rfloor\f$, 68- \f$iw_1 = \left\lceil {\frac {ow + 0.5} {F_w} - 0.5}\right\rceil\f$, 69- \f$W_{ih} = \frac{oh + 0.5}{F_h} - 0.5 - ih_0\f$, 70- \f$W_{iw} = \frac{ow + 0.5}{F_w} - 0.5 - iw_0\f$. 71 72 73#### Difference Between Forward Training and Forward Inference 74 75There is no difference between the #dnnl_forward_training 76and #dnnl_forward_inference propagation kinds. 77 78### Backward 79 80The backward propagation computes \diffsrc based on \diffdst. 81 82## Execution Arguments 83 84When executed, the inputs and outputs should be mapped to an execution 85argument index as specified by the following table. 86 87| Primitive input/output | Execution argument index | 88| --- | --- | 89| \src | DNNL_ARG_SRC | 90| \dst | DNNL_ARG_DST | 91| \diffsrc | DNNL_ARG_DIFF_SRC | 92| \diffdst | DNNL_ARG_DIFF_DST | 93| \f$\text{binary post-op}\f$ | DNNL_ARG_ATTR_MULTIPLE_POST_OP(binary_post_op_position) \| DNNL_ARG_SRC_1 | 94 95## Implementation Details 96 97### General Notes 981. Resampling implementation supports data with arbitrary data tag (nchw, nhwc, 99 nChw16c, etc.) but memory tags for `src` and `dst` are expected to be the 100 same. Resampling primitive supports `dst` and `diff_src` memory tag 101 #dnnl::memory::format_tag::any and can define destination format based on 102 source format. 1032. Resampling descriptor can be created by specifying the source and 104 destination memory descriptors, only the source descriptor and floating 105 point factors, or the source and destination memory descriptors and factors. 106 In case when user does not provide the destination descriptor, the 107 destination dimensions are deduced using the factors: 108 \f$ 109 output\_spatial\_size = \left\lfloor{ 110 \frac{input\_spatial\_size} {F} 111 }\right\rfloor 112 \f$. 113 114@note 115 Implementation of resampling algorithm uses factors as defined by the 116 relation \f$F = \frac{output\_spatial\_ size} { 117 input\_spatial\_size}\f$ that do not necessarily equal to the ones passed 118 by the user. 119 120 121### Data Types 122 123Resampling primitive supports the following combination of data types for 124source and destination memory objects: 125 126| Propagation | Source | Destination | 127| :-- | :-- | :-- | 128| forward / backward | f32, s32, bf16, s8, u8 | f32, s32, bf16, s8, u8 | 129| forward | f16 | f16 | 130 131### Post-ops and Attributes 132 133The following attributes are supported: 134 135| Type | Operation | Description | Restrictions 136| :-- | :-- | :-- | :-- 137| Post-op | [Sum](@ref dnnl::post_ops::append_sum) | Adds the operation result to the destination tensor instead of overwriting it. | | 138| Post-op | [Eltwise](@ref dnnl::post_ops::append_eltwise) | Applies an @ref dnnl_api_eltwise operation to the result. | | 139| Post-op | [Binary](@ref dnnl::post_ops::append_binary) | Applies a @ref dnnl_api_binary operation to the result | General binary post-op restrictions | 140 141## Implementation Limitations 142 1431. No primitive specific limitations. Refer to @ref dev_guide_data_types for 144 limitations related to data types support. 145 146## Performance Tips 147 148N/A 149 150## Examples 151 152| Engine | Name | Comments 153| :-- | :-- | :-- 154| CPU/GPU | @ref resampling_example_cpp | @copydetails resampling_example_cpp_short 155