1 /*#****************************************************************************** 2 ** IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING. 3 ** 4 ** By downloading, copying, installing or using the software you agree to this license. 5 ** If you do not agree to this license, do not download, install, 6 ** copy or use the software. 7 ** 8 ** 9 ** bioinspired : interfaces allowing OpenCV users to integrate Human Vision System models. Presented models originate from Jeanny Herault's original research and have been reused and adapted by the author&collaborators for computed vision applications since his thesis with Alice Caplier at Gipsa-Lab. 10 ** Use: extract still images & image sequences features, from contours details to motion spatio-temporal features, etc. for high level visual scene analysis. Also contribute to image enhancement/compression such as tone mapping. 11 ** 12 ** Maintainers : Listic lab (code author current affiliation & applications) and Gipsa Lab (original research origins & applications) 13 ** 14 ** Creation - enhancement process 2007-2011 15 ** Author: Alexandre Benoit (benoit.alexandre.vision@gmail.com), LISTIC lab, Annecy le vieux, France 16 ** 17 ** Theses algorithm have been developped by Alexandre BENOIT since his thesis with Alice Caplier at Gipsa-Lab (www.gipsa-lab.inpg.fr) and the research he pursues at LISTIC Lab (www.listic.univ-savoie.fr). 18 ** Refer to the following research paper for more information: 19 ** Benoit A., Caplier A., Durette B., Herault, J., "USING HUMAN VISUAL SYSTEM MODELING FOR BIO-INSPIRED LOW LEVEL IMAGE PROCESSING", Elsevier, Computer Vision and Image Understanding 114 (2010), pp. 758-773, DOI: http://dx.doi.org/10.1016/j.cviu.2010.01.011 20 ** This work have been carried out thanks to Jeanny Herault who's research and great discussions are the basis of all this work, please take a look at his book: 21 ** Vision: Images, Signals and Neural Networks: Models of Neural Processing in Visual Perception (Progress in Neural Processing),By: Jeanny Herault, ISBN: 9814273686. WAPI (Tower ID): 113266891. 22 ** 23 ** The retina filter includes the research contributions of phd/research collegues from which code has been redrawn by the author : 24 ** _take a look at the retinacolor.hpp module to discover Brice Chaix de Lavarene color mosaicing/demosaicing and the reference paper: 25 ** ====> B. Chaix de Lavarene, D. Alleysson, B. Durette, J. Herault (2007). "Efficient demosaicing through recursive filtering", IEEE International Conference on Image Processing ICIP 2007 26 ** _take a look at imagelogpolprojection.hpp to discover retina spatial log sampling which originates from Barthelemy Durette phd with Jeanny Herault. A Retina / V1 cortex projection is also proposed and originates from Jeanny's discussions. 27 ** ====> more informations in the above cited Jeanny Heraults's book. 28 ** 29 ** License Agreement 30 ** For Open Source Computer Vision Library 31 ** 32 ** Copyright (C) 2000-2008, Intel Corporation, all rights reserved. 33 ** Copyright (C) 2008-2011, Willow Garage Inc., all rights reserved. 34 ** 35 ** For Human Visual System tools (bioinspired) 36 ** Copyright (C) 2007-2011, LISTIC Lab, Annecy le Vieux and GIPSA Lab, Grenoble, France, all rights reserved. 37 ** 38 ** Third party copyrights are property of their respective owners. 39 ** 40 ** Redistribution and use in source and binary forms, with or without modification, 41 ** are permitted provided that the following conditions are met: 42 ** 43 ** * Redistributions of source code must retain the above copyright notice, 44 ** this list of conditions and the following disclaimer. 45 ** 46 ** * Redistributions in binary form must reproduce the above copyright notice, 47 ** this list of conditions and the following disclaimer in the documentation 48 ** and/or other materials provided with the distribution. 49 ** 50 ** * The name of the copyright holders may not be used to endorse or promote products 51 ** derived from this software without specific prior written permission. 52 ** 53 ** This software is provided by the copyright holders and contributors "as is" and 54 ** any express or implied warranties, including, but not limited to, the implied 55 ** warranties of merchantability and fitness for a particular purpose are disclaimed. 56 ** In no event shall the Intel Corporation or contributors be liable for any direct, 57 ** indirect, incidental, special, exemplary, or consequential damages 58 ** (including, but not limited to, procurement of substitute goods or services; 59 ** loss of use, data, or profits; or business interruption) however caused 60 ** and on any theory of liability, whether in contract, strict liability, 61 ** or tort (including negligence or otherwise) arising in any way out of 62 ** the use of this software, even if advised of the possibility of such damage. 63 *******************************************************************************/ 64 65 #ifndef MagnoRetinaFilter_H_ 66 #define MagnoRetinaFilter_H_ 67 68 /** 69 * @class MagnoRetinaFilter 70 * @brief class which describes the magnocellular channel of the retina: 71 * -> performs a moving contours extraction with powerfull local data enhancement 72 * 73 * TYPICAL USE: 74 * 75 * // create object at a specified picture size 76 * MagnoRetinaFilter *movingContoursExtractor; 77 * movingContoursExtractor =new MagnoRetinaFilter(frameSizeRows, frameSizeColumns); 78 * 79 * // init gain, spatial and temporal parameters: 80 * movingContoursExtractor->setCoefficientsTable(0, 0.7, 5, 3); 81 * 82 * // during program execution, call the filter for contours extraction for an input picture called "FrameBuffer": 83 * movingContoursExtractor->runfilter(FrameBuffer); 84 * 85 * // get the output frame, check in the class description below for more outputs: 86 * const float *movingContours=movingContoursExtractor->getMagnoYsaturated(); 87 * 88 * // at the end of the program, destroy object: 89 * delete movingContoursExtractor; 90 91 * @author Alexandre BENOIT, benoit.alexandre.vision@gmail.com, LISTIC : www.listic.univ-savoie.fr, Gipsa-Lab, France: www.gipsa-lab.inpg.fr/ 92 * Creation date 2007 93 * Based on Alexandre BENOIT thesis: "Le système visuel humain au secours de la vision par ordinateur" 94 */ 95 96 #include "basicretinafilter.hpp" 97 98 //#define _IPL_RETINA_ELEMENT_DEBUG 99 100 namespace cv 101 { 102 namespace bioinspired 103 { 104 class MagnoRetinaFilter: public BasicRetinaFilter 105 { 106 public: 107 /** 108 * constructor parameters are only linked to image input size 109 * @param NBrows: number of rows of the input image 110 * @param NBcolumns: number of columns of the input image 111 */ 112 MagnoRetinaFilter(const unsigned int NBrows, const unsigned int NBcolumns); 113 114 115 /** 116 * destructor 117 */ 118 virtual ~MagnoRetinaFilter(); 119 120 /** 121 * function that clears all buffers of the object 122 */ 123 void clearAllBuffers(); 124 125 /** 126 * resize retina magno filter object (resize all allocated buffers) 127 * @param NBrows: the new height size 128 * @param NBcolumns: the new width size 129 */ 130 void resize(const unsigned int NBrows, const unsigned int NBcolumns); 131 132 /** 133 * set parameters values 134 * @param parasolCells_beta: the low pass filter gain used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), typical value is 0 135 * @param parasolCells_tau: the low pass filter time constant used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), unit is frame, typical value is 0 (immediate response) 136 * @param parasolCells_k: the low pass filter spatial constant used for local contrast adaptation at the IPL level of the retina (for ganglion cells local adaptation), unit is pixels, typical value is 5 137 * @param amacrinCellsTemporalCutFrequency: the time constant of the first order high pass fiter of the magnocellular way (motion information channel), unit is frames, tipicall value is 5 138 * @param localAdaptIntegration_tau: specifies the temporal constant of the low pas filter involved in the computation of the local "motion mean" for the local adaptation computation 139 * @param localAdaptIntegration_k: specifies the spatial constant of the low pas filter involved in the computation of the local "motion mean" for the local adaptation computation 140 */ 141 void setCoefficientsTable(const float parasolCells_beta, const float parasolCells_tau, const float parasolCells_k, const float amacrinCellsTemporalCutFrequency, const float localAdaptIntegration_tau, const float localAdaptIntegration_k); 142 143 /** 144 * launch filter that runs all the IPL magno filter (model of the magnocellular channel of the Inner Plexiform Layer of the retina) 145 * @param OPL_ON: the output of the bipolar ON cells of the retina (available from the ParvoRetinaFilter class (getBipolarCellsON() function) 146 * @param OPL_OFF: the output of the bipolar OFF cells of the retina (available from the ParvoRetinaFilter class (getBipolarCellsOFF() function) 147 * @return the processed result without post-processing 148 */ 149 const std::valarray<float> &runFilter(const std::valarray<float> &OPL_ON, const std::valarray<float> &OPL_OFF); 150 151 /** 152 * @return the Magnocellular ON channel filtering output 153 */ getMagnoON() const154 inline const std::valarray<float> &getMagnoON() const { return _magnoXOutputON; } 155 156 /** 157 * @return the Magnocellular OFF channel filtering output 158 */ getMagnoOFF() const159 inline const std::valarray<float> &getMagnoOFF() const { return _magnoXOutputOFF; } 160 161 /** 162 * @return the Magnocellular Y (sum of the ON and OFF magno channels) filtering output 163 */ getMagnoYsaturated() const164 inline const std::valarray<float> &getMagnoYsaturated() const { return *_magnoYsaturated; } 165 166 /** 167 * applies an image normalization which saturates the high output values by the use of an assymetric sigmoide 168 */ normalizeGrayOutputNearZeroCentreredSigmoide()169 inline void normalizeGrayOutputNearZeroCentreredSigmoide() 170 { _filterOutput.normalizeGrayOutputNearZeroCentreredSigmoide(&(*_magnoYOutput)[0], &(*_magnoYsaturated)[0]); } 171 172 /** 173 * @return the horizontal cells' temporal constant 174 */ getTemporalConstant()175 inline float getTemporalConstant() { return _filteringCoeficientsTable[2]; } 176 177 private: 178 179 // related pointers to these buffers 180 std::valarray<float> _previousInput_ON; 181 std::valarray<float> _previousInput_OFF; 182 std::valarray<float> _amacrinCellsTempOutput_ON; 183 std::valarray<float> _amacrinCellsTempOutput_OFF; 184 std::valarray<float> _magnoXOutputON; 185 std::valarray<float> _magnoXOutputOFF; 186 std::valarray<float> _localProcessBufferON; 187 std::valarray<float> _localProcessBufferOFF; 188 // reference to parent buffers and allow better readability 189 TemplateBuffer<float> *_magnoYOutput; 190 std::valarray<float> *_magnoYsaturated; 191 192 // varialbles 193 float _temporalCoefficient; 194 195 // amacrine cells filter : high pass temporal filter 196 void _amacrineCellsComputing(const float *ONinput, const float *OFFinput); 197 #ifdef MAKE_PARALLEL 198 /****************************************************** 199 ** IF some parallelizing thread methods are available, then, main loops are parallelized using these functors 200 ** ==> main idea paralellise main filters loops, then, only the most used methods are parallelized... TODO : increase the number of parallelised methods as necessary 201 ** ==> functors names = Parallel_$$$ where $$$= the name of the serial method that is parallelised 202 ** ==> functors constructors can differ from the parameters used with their related serial functions 203 */ 204 class Parallel_amacrineCellsComputing: public cv::ParallelLoopBody 205 { 206 private: 207 const float *OPL_ON, *OPL_OFF; 208 float *previousInput_ON, *previousInput_OFF, *amacrinCellsTempOutput_ON, *amacrinCellsTempOutput_OFF; 209 float temporalCoefficient; 210 public: Parallel_amacrineCellsComputing(const float * OPL_ON_PTR,const float * OPL_OFF_PTR,float * previousInput_ON_PTR,float * previousInput_OFF_PTR,float * amacrinCellsTempOutput_ON_PTR,float * amacrinCellsTempOutput_OFF_PTR,float temporalCoefficientVal)211 Parallel_amacrineCellsComputing(const float *OPL_ON_PTR, const float *OPL_OFF_PTR, float *previousInput_ON_PTR, float *previousInput_OFF_PTR, float *amacrinCellsTempOutput_ON_PTR, float *amacrinCellsTempOutput_OFF_PTR, float temporalCoefficientVal) 212 :OPL_ON(OPL_ON_PTR), OPL_OFF(OPL_OFF_PTR), previousInput_ON(previousInput_ON_PTR), previousInput_OFF(previousInput_OFF_PTR), amacrinCellsTempOutput_ON(amacrinCellsTempOutput_ON_PTR), amacrinCellsTempOutput_OFF(amacrinCellsTempOutput_OFF_PTR), temporalCoefficient(temporalCoefficientVal) {} 213 operator ()(const Range & r) const214 virtual void operator()( const Range& r ) const CV_OVERRIDE { 215 const float *OPL_ON_PTR=OPL_ON+r.start; 216 const float *OPL_OFF_PTR=OPL_OFF+r.start; 217 float *previousInput_ON_PTR= previousInput_ON+r.start; 218 float *previousInput_OFF_PTR= previousInput_OFF+r.start; 219 float *amacrinCellsTempOutput_ON_PTR= amacrinCellsTempOutput_ON+r.start; 220 float *amacrinCellsTempOutput_OFF_PTR= amacrinCellsTempOutput_OFF+r.start; 221 222 for (int IDpixel=r.start ; IDpixel!=r.end; ++IDpixel) 223 { 224 225 /* Compute ON and OFF amacrin cells high pass temporal filter */ 226 float magnoXonPixelResult = temporalCoefficient*(*amacrinCellsTempOutput_ON_PTR+ *OPL_ON_PTR-*previousInput_ON_PTR); 227 *(amacrinCellsTempOutput_ON_PTR++)=((float)(magnoXonPixelResult>0))*magnoXonPixelResult; 228 229 float magnoXoffPixelResult = temporalCoefficient*(*amacrinCellsTempOutput_OFF_PTR+ *OPL_OFF_PTR-*previousInput_OFF_PTR); 230 *(amacrinCellsTempOutput_OFF_PTR++)=((float)(magnoXoffPixelResult>0))*magnoXoffPixelResult; 231 232 /* prepare next loop */ 233 *(previousInput_ON_PTR++)=*(OPL_ON_PTR++); 234 *(previousInput_OFF_PTR++)=*(OPL_OFF_PTR++); 235 236 } 237 } 238 239 }; 240 #endif 241 }; 242 243 }// end of namespace bioinspired 244 }// end of namespace cv 245 246 #endif /*MagnoRetinaFilter_H_*/ 247