1Resampling {#dev_guide_resampling} 2===================================== 3 4> 5> [API reference](@ref dnnl_api_resampling) 6> 7 8## General 9 10The resampling primitive computes forward or backward resampling operation on 111D, 2D, or 3D spatial data. Resampling performs spatial scaling of original 12tensor using one of the supported interpolation algorithms: 13- Nearest Neighbor 14- Linear (or Bilinear for 2D spatial tensor, Trilinear for 3D spatial tensor). 15 16Resampling operation is defined by the source tensor and scaling factors in 17each spatial dimension. Upsampling and downsampling are the alternative terms 18for resampling that are used when all scaling factors are greater (upsampling) 19or less (downsampling) than one. 20 21The resampling operation is defined by the following formulas. We show formulas 22only for 2D spatial data which are straightforward to generalize to cases of 23higher and lower dimensions. Variable names follow the standard 24@ref dev_guide_conventions. 25 26Let \src and \dst be \f$N \times C \times IH \times IW\f$ and \f$N 27\times C \times OH \times OW\f$ tensors respectively. Let 28\f$ F_h = \frac{OH}{IH} \f$ and \f$ F_w = \frac{OW}{IW} \f$ define scaling 29factors in each spatial dimension. 30 31The following formulas show how oneDNN computes resampling for nearest neighbor 32and bilinear interpolation methods. 33To further simplify the formulas, we assume the following: 34\f$\src(n, ic, ih, iw) = \begin{cases} 35\src(n, ic, ih, 0), & \text{if}\ iw < 0 \\ 36\src(n, ic, ih, iw), & \text{if}\ IW - 1 \geq iw \geq 0 \\ 37\src(n, ic, ih, IW - 1), & \text{if}\ iw > IW - 1 38\end{cases}\f$ 39 40Same assumptions apply for \f$ih\f$. Definitions of \f$ih\f$ and \f$iw\f$ are 41provided below with a correspondent algorithm. 42 43### Forward 44 45#### Nearest Neighbor Resampling 46 47\f[\dst(n, c, oh, ow) = \src(n, c, ih, iw)\f] 48 49where 50 51- \f$ih = [\frac{oh + 0.5} {F_h} - 0.5]\f$, 52- \f$iw = [\frac{ow + 0.5} {F_w} - 0.5]\f$. 53 54#### Bilinear Resampling 55 56\f[ 57 \dst(n, c, oh, ow) = 58 \src(n, c, ih_0, iw_0) \cdot (1 - W_{ih}) \cdot (1 - W_{iw}) + \\ 59 \src(n, c, ih_1, iw_0) \cdot W_{ih} \cdot (1 - W_{iw}) + \\ 60 \src(n, c, ih_0, iw_1) \cdot (1 - W_{ih}) \cdot W_{iw} + \\ 61 \src(n, c, ih_1, iw_1) \cdot W_{ih} \cdot W_{iw} \\ 62\f] 63 64where 65- \f$ih_0 = \left\lfloor{\frac {oh + 0.5} {F_h} - 0.5}\right\rfloor\f$, 66- \f$ih_1 = \left\lceil {\frac {oh + 0.5} {F_h} - 0.5}\right\rceil\f$, 67- \f$iw_0 = \left\lfloor{\frac {ow + 0.5} {F_w} - 0.5}\right\rfloor\f$, 68- \f$iw_1 = \left\lceil {\frac {ow + 0.5} {F_w} - 0.5}\right\rceil\f$, 69- \f$W_{ih} = \frac{oh + 0.5}{F_h} - 0.5 - ih_0\f$, 70- \f$W_{iw} = \frac{ow + 0.5}{F_w} - 0.5 - iw_0\f$. 71 72 73#### Difference Between Forward Training and Forward Inference 74 75There is no difference between the #dnnl_forward_training 76and #dnnl_forward_inference propagation kinds. 77 78### Backward 79 80The backward propagation computes \diffsrc based on \diffdst. 81 82## Execution Arguments 83 84When executed, the inputs and outputs should be mapped to an execution 85argument index as specified by the following table. 86 87| Primitive input/output | Execution argument index | 88| --- | --- | 89| \src | DNNL_ARG_SRC | 90| \dst | DNNL_ARG_DST | 91| \diffsrc | DNNL_ARG_DIFF_SRC | 92| \diffdst | DNNL_ARG_DIFF_DST | 93| \f$\text{binary post-op}\f$ | DNNL_ARG_ATTR_MULTIPLE_POST_OP(binary_post_op_position) \| DNNL_ARG_SRC_1 | 94 95## Implementation Details 96 97### General Notes 98 991. Resampling implementation supports data with arbitrary data tag (nchw, nhwc, 100 nChw16c, etc.) but memory tags for `src` and `dst` are expected to be the 101 same. Resampling primitive supports `dst` and `diff_src` memory tag 102 #dnnl::memory::format_tag::any and can define destination format based on 103 source format. 1042. Resampling descriptor can be created by specifying the source and 105 destination memory descriptors, only the source descriptor and floating 106 point factors, or the source and destination memory descriptors and factors. 107 In case when user does not provide the destination descriptor, the 108 destination dimensions are deduced using the factors: 109 \f$ 110 output\_spatial\_size = \left\lfloor{ 111 \frac{input\_spatial\_size} {F} 112 }\right\rfloor 113 \f$. 114 115@note 116 Implementation of resampling algorithm uses factors as defined by the 117 relation \f$F = \frac{output\_spatial\_ size} { 118 input\_spatial\_size}\f$ that do not necessarily equal to the ones passed 119 by the user. 120 121 122### Data Types 123 124Resampling primitive supports the following combination of data types for 125source and destination memory objects: 126 127| Propagation | Source | Destination | 128| :-- | :-- | :-- | 129| forward / backward | f32, s32, bf16, s8, u8 | f32, s32, bf16, s8, u8 | 130| forward | f16 | f16 | 131 132### Post-Ops and Attributes 133 134The following attributes are supported: 135 136| Type | Operation | Description | Restrictions | 137| :-- | :-- | :-- | :-- | 138| Post-op | [Sum](@ref dnnl::post_ops::append_sum) | Adds the operation result to the destination tensor instead of overwriting it. | | 139| Post-op | [Eltwise](@ref dnnl::post_ops::append_eltwise) | Applies an @ref dnnl_api_eltwise operation to the result. | | 140| Post-op | [Binary](@ref dnnl::post_ops::append_binary) | Applies a @ref dnnl_api_binary operation to the result | General binary post-op restrictions | 141 142## Implementation Limitations 143 1441. No primitive specific limitations. Refer to @ref dev_guide_data_types for 145 limitations related to data types support. 146 147## Performance Tips 148 149N/A 150 151## Example 152 153[Resampling Primitive Example](@ref resampling_example_cpp) 154 155@copydetails resampling_example_cpp_short 156