1# tifffile.py 2 3# Copyright (c) 2008-2021, Christoph Gohlke 4# All rights reserved. 5# 6# Redistribution and use in source and binary forms, with or without 7# modification, are permitted provided that the following conditions are met: 8# 9# 1. Redistributions of source code must retain the above copyright notice, 10# this list of conditions and the following disclaimer. 11# 12# 2. Redistributions in binary form must reproduce the above copyright notice, 13# this list of conditions and the following disclaimer in the documentation 14# and/or other materials provided with the distribution. 15# 16# 3. Neither the name of the copyright holder nor the names of its 17# contributors may be used to endorse or promote products derived from 18# this software without specific prior written permission. 19# 20# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 23# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE 24# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 25# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 26# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 27# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 28# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 29# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 30# POSSIBILITY OF SUCH DAMAGE. 31 32"""Read and write TIFF files. 33 34Tifffile is a Python library to 35 36(1) store numpy arrays in TIFF (Tagged Image File Format) files, and 37(2) read image and metadata from TIFF-like files used in bioimaging. 38 39Image and metadata can be read from TIFF, BigTIFF, OME-TIFF, STK, LSM, SGI, 40NIHImage, ImageJ, MicroManager, FluoView, ScanImage, SEQ, GEL, SVS, SCN, SIS, 41ZIF (Zoomable Image File Format), QPTIFF (QPI), NDPI, and GeoTIFF files. 42 43Image data can be read as numpy arrays or zarr arrays/groups from strips, 44tiles, pages (IFDs), SubIFDs, higher order series, and pyramidal levels. 45 46Numpy arrays can be written to TIFF, BigTIFF, OME-TIFF, and ImageJ hyperstack 47compatible files in multi-page, volumetric, pyramidal, memory-mappable, tiled, 48predicted, or compressed form. 49 50A subset of the TIFF specification is supported, mainly 8, 16, 32 and 64-bit 51integer, 16, 32 and 64-bit float, grayscale and multi-sample images. 52Specifically, CCITT and OJPEG compression, chroma subsampling without JPEG 53compression, color space transformations, samples with differing types, or 54IPTC and XMP metadata are not implemented. 55 56TIFF, the Tagged Image File Format, was created by the Aldus Corporation and 57Adobe Systems Incorporated. BigTIFF allows for files larger than 4 GB. 58STK, LSM, FluoView, SGI, SEQ, GEL, QPTIFF, NDPI, SCN, SVS, ZIF, and OME-TIFF, 59are custom extensions defined by Molecular Devices (Universal Imaging 60Corporation), Carl Zeiss MicroImaging, Olympus, Silicon Graphics International, 61Media Cybernetics, Molecular Dynamics, PerkinElmer, Hamamatsu, Leica, 62ObjectivePathology, and the Open Microscopy Environment consortium, 63respectively. 64 65For command line usage run ``python -m tifffile --help`` 66 67:Author: 68 `Christoph Gohlke <https://www.lfd.uci.edu/~gohlke/>`_ 69 70:Organization: 71 Laboratory for Fluorescence Dynamics, University of California, Irvine 72 73:License: BSD 3-Clause 74 75:Version: 2021.8.30 76 77Requirements 78------------ 79This release has been tested with the following requirements and dependencies 80(other versions may work): 81 82* `CPython 3.7.9, 3.8.10, 3.9.7 64-bit <https://www.python.org>`_ 83* `Numpy 1.20.3 <https://pypi.org/project/numpy/>`_ 84* `Imagecodecs 2021.8.26 <https://pypi.org/project/imagecodecs/>`_ 85 (required only for encoding or decoding LZW, JPEG, etc.) 86* `Matplotlib 3.4.3 <https://pypi.org/project/matplotlib/>`_ 87 (required only for plotting) 88* `Lxml 4.6.3 <https://pypi.org/project/lxml/>`_ 89 (required only for validating and printing XML) 90* `Zarr 2.9.4 <https://pypi.org/project/zarr/>`_ 91 (required only for opening zarr storage) 92 93Revisions 94--------- 952021.8.30 96 Pass 4723 tests. 97 Fix horizontal differencing with non-native byte order. 98 Fix multi-threaded access of memory-mappable, multi-page Zarr stores (#67). 992021.8.8 100 Fix tag offset and valueoffset for NDPI > 4 GB (#96). 1012021.7.30 102 Deprecate first parameter to TiffTag.overwrite (no longer required). 103 TiffTag init API change (breaking). 104 Detect Ventana BIF series and warn that tiles are not stitched. 105 Enable reading PreviewImage from RAW formats (#93, #94). 106 Work around numpy.ndarray.tofile is very slow for non-contiguous arrays. 107 Fix issues with PackBits compression (requires imagecodecs 2021.7.30). 1082021.7.2 109 Decode complex integer images found in SAR GeoTIFF. 110 Support reading NDPI with JPEG-XR compression. 111 Deprecate TiffWriter RGB auto-detection, except for RGB24/48 and RGBA32/64. 1122021.6.14 113 Set stacklevel for deprecation warnings (#89). 114 Fix svs_description_metadata for SVS with double header (#88, breaking). 115 Fix reading JPEG compressed CMYK images. 116 Support ALT_JPEG and JPEG_2000_LOSSY compression found in Bio-Formats. 117 Log warning if TiffWriter auto-detects RGB mode (specify photometric). 1182021.6.6 119 Fix TIFF.COMPESSOR typo (#85). 120 Round resolution numbers that do not fit in 64-bit rationals (#81). 121 Add support for JPEG XL compression. 122 Add numcodecs compatible TIFF codec. 123 Rename ZarrFileStore to ZarrFileSequenceStore (breaking). 124 Add method to export fsspec ReferenceFileSystem from ZarrFileStore. 125 Fix fsspec ReferenceFileSystem v1 for multifile series. 126 Fix creating OME-TIFF with micron character in OME-XML. 1272021.4.8 128 Fix reading OJPEG with wrong photometric or samplesperpixel tags (#75). 129 Fix fsspec ReferenceFileSystem v1 and JPEG compression. 130 Use TiffTagRegistry for NDPI_TAGS, EXIF_TAGS, GPS_TAGS, IOP_TAGS constants. 131 Make TIFF.GEO_KEYS an Enum (breaking). 1322021.3.31 133 Use JPEG restart markers as tile offsets in NDPI. 134 Support version 1 and more codecs in fsspec ReferenceFileSystem (untested). 1352021.3.17 136 Fix regression reading multi-file OME-TIFF with missing files (#72). 137 Fix fsspec ReferenceFileSystem with non-native byte order (#56). 1382021.3.16 139 TIFF is no longer a defended trademark. 140 Add method to export fsspec ReferenceFileSystem from ZarrTiffStore (#56). 1412021.3.5 142 Preliminary support for EER format (#68). 143 Do not warn about unknown compression (#68). 1442021.3.4 145 Fix reading multi-file, multi-series OME-TIFF (#67). 146 Detect ScanImage 2021 files (#46). 147 Shape new version ScanImage series according to metadata (breaking). 148 Remove Description key from TiffFile.scanimage_metadata dict (breaking). 149 Also return ScanImage version from read_scanimage_metadata (breaking). 150 Fix docstrings. 1512021.2.26 152 Squeeze axes of LSM series by default (breaking). 153 Add option to preserve single dimensions when reading from series (WIP). 154 Do not allow appending to OME-TIFF files. 155 Fix reading STK files without name attribute in metadata. 156 Make TIFF constants multi-thread safe and pickleable (#64). 157 Add detection of NDTiffStorage MajorVersion to read_micromanager_metadata. 158 Support ScanImage v4 files in read_scanimage_metadata. 1592021.2.1 160 Fix multi-threaded access of ZarrTiffStores using same TiffFile instance. 161 Use fallback zlib and lzma codecs with imagecodecs lite builds. 162 Open Olympus and Panasonic RAW files for parsing, albeit not supported. 163 Support X2 and X4 differencing found in DNG. 164 Support reading JPEG_LOSSY compression found in DNG. 1652021.1.14 166 Try ImageJ series if OME series fails (#54) 167 Add option to use pages as chunks in ZarrFileStore (experimental). 168 Fix reading from file objects with no readinto function. 1692021.1.11 170 Fix test errors on PyPy. 171 Fix decoding bitorder with imagecodecs >= 2021.1.11. 1722021.1.8 173 Decode float24 using imagecodecs >= 2021.1.8. 174 Consolidate reading of segments if possible. 1752020.12.8 176 Fix corrupted ImageDescription in multi shaped series if buffer too small. 177 Fix libtiff warning that ImageDescription contains null byte in value. 178 Fix reading invalid files using JPEG compression with palette colorspace. 1792020.12.4 180 Fix reading some JPEG compressed CFA images. 181 Make index of SubIFDs a tuple. 182 Pass through FileSequence.imread arguments in imread. 183 Do not apply regex flags to FileSequence axes patterns (breaking). 1842020.11.26 185 Add option to pass axes metadata to ImageJ writer. 186 Pad incomplete tiles passed to TiffWriter.write (#38). 187 Split TiffTag constructor (breaking). 188 Change TiffTag.dtype to TIFF.DATATYPES (breaking). 189 Add TiffTag.overwrite method. 190 Add script to change ImageDescription in files. 191 Add TiffWriter.overwrite_description method (WIP). 1922020.11.18 193 Support writing SEPARATED color space (#37). 194 Use imagecodecs.deflate codec if available. 195 Fix SCN and NDPI series with Z dimensions. 196 Add TiffReader alias for TiffFile. 197 TiffPage.is_volumetric returns True if ImageDepth > 1. 198 Zarr store getitem returns numpy arrays instead of bytes. 1992020.10.1 200 Formally deprecate unused TiffFile parameters (scikit-image #4996). 2012020.9.30 202 Allow to pass additional arguments to compression codecs. 203 Deprecate TiffWriter.save method (use TiffWriter.write). 204 Deprecate TiffWriter.save compress parameter (use compression). 205 Remove multifile parameter from TiffFile (breaking). 206 Pass all is_flag arguments from imread to TiffFile. 207 Do not byte-swap JPEG2000, WEBP, PNG, JPEGXR segments in TiffPage.decode. 2082020.9.29 209 Fix reading files produced by ScanImage > 2015 (#29). 2102020.9.28 211 Derive ZarrStore from MutableMapping. 212 Support zero shape ZarrTiffStore. 213 Fix ZarrFileStore with non-TIFF files. 214 Fix ZarrFileStore with missing files. 215 Cache one chunk in ZarrFileStore. 216 Keep track of already opened files in FileCache. 217 Change parse_filenames function to return zero-based indices. 218 Remove reopen parameter from asarray (breaking). 219 Rename FileSequence.fromfile to imread (breaking). 2202020.9.22 221 Add experimental zarr storage interface (WIP). 222 Remove unused first dimension from TiffPage.shaped (breaking). 223 Move reading of STK planes to series interface (breaking). 224 Always use virtual frames for ScanImage files. 225 Use DimensionOrder to determine axes order in OmeXml. 226 Enable writing striped volumetric images. 227 Keep complete dataoffsets and databytecounts for TiffFrames. 228 Return full size tiles from Tiffpage.segments. 229 Rename TiffPage.is_sgi property to is_volumetric (breaking). 230 Rename TiffPageSeries.is_pyramid to is_pyramidal (breaking). 231 Fix TypeError when passing jpegtables to non-JPEG decode method (#25). 2322020.9.3 233 Do not write contiguous series by default (breaking). 234 Allow to write to SubIFDs (WIP). 235 Fix writing F-contiguous numpy arrays (#24). 2362020.8.25 237 Do not convert EPICS timeStamp to datetime object. 238 Read incompletely written Micro-Manager image file stack header (#23). 239 Remove tag 51123 values from TiffFile.micromanager_metadata (breaking). 2402020.8.13 241 Use tifffile metadata over OME and ImageJ for TiffFile.series (breaking). 242 Fix writing iterable of pages with compression (#20). 243 Expand error checking of TiffWriter data, dtype, shape, and tile arguments. 2442020.7.24 245 Parse nested OmeXml metadata argument (WIP). 246 Do not lazy load TiffFrame JPEGTables. 247 Fix conditionally skipping some tests. 2482020.7.22 249 Do not auto-enable OME-TIFF if description is passed to TiffWriter.save. 250 Raise error writing empty bilevel or tiled images. 251 Allow to write tiled bilevel images. 252 Allow to write multi-page TIFF from iterable of single page images (WIP). 253 Add function to validate OME-XML. 254 Correct Philips slide width and length. 2552020.7.17 256 Initial support for writing OME-TIFF (WIP). 257 Return samples as separate dimension in OME series (breaking). 258 Fix modulo dimensions for multiple OME series. 259 Fix some test errors on big endian systems (#18). 260 Fix BytesWarning. 261 Allow to pass TIFF.PREDICTOR values to TiffWriter.save. 2622020.7.4 263 Deprecate support for Python 3.6 (NEP 29). 264 Move pyramidal subresolution series to TiffPageSeries.levels (breaking). 265 Add parser for SVS, SCN, NDPI, and QPI pyramidal series. 266 Read single-file OME-TIFF pyramids. 267 Read NDPI files > 4 GB (#15). 268 Include SubIFDs in generic series. 269 Preliminary support for writing packed integer arrays (#11, WIP). 270 Read more LSM info subrecords. 271 Fix missing ReferenceBlackWhite tag for YCbCr photometrics. 272 Fix reading lossless JPEG compressed DNG files. 2732020.6.3 274 ... 275 276Refer to the CHANGES file for older revisions. 277 278Notes 279----- 280The API is not stable yet and might change between revisions. 281 282Tested on little-endian platforms only. 283 284Python 32-bit versions are deprecated. Python <= 3.7 are no longer supported. 285 286Tifffile relies on the `imagecodecs <https://pypi.org/project/imagecodecs/>`_ 287package for encoding and decoding LZW, JPEG, and other compressed image 288segments. 289 290Several TIFF-like formats do not strictly adhere to the TIFF6 specification, 291some of which allow file or data sizes to exceed the 4 GB limit: 292 293* *BigTIFF* is identified by version number 43 and uses different file 294 header, IFD, and tag structures with 64-bit offsets. It adds more data types. 295 Tifffile can read and write BigTIFF files. 296* *ImageJ* hyperstacks store all image data, which may exceed 4 GB, 297 contiguously after the first IFD. Files > 4 GB contain one IFD only. 298 The size (shape and dtype) of the up to 6-dimensional image data can be 299 determined from the ImageDescription tag of the first IFD, which is Latin-1 300 encoded. Tifffile can read and write ImageJ hyperstacks. 301* *OME-TIFF* stores up to 8-dimensional data in one or multiple TIFF of BigTIFF 302 files. The 8-bit UTF-8 encoded OME-XML metadata found in the ImageDescription 303 tag of the first IFD defines the position of TIFF IFDs in the high 304 dimensional data. Tifffile can read OME-TIFF files, except when the OME-XML 305 metadata are stored in a separate file. Tifffile can write numpy arrays 306 to single-file OME-TIFF. 307* *LSM* stores all IFDs below 4 GB but wraps around 32-bit StripOffsets. 308 The StripOffsets of each series and position require separate unwrapping. 309 The StripByteCounts tag contains the number of bytes for the uncompressed 310 data. Tifffile can read large LSM files. 311* *STK* (MetaMorph Stack) contains additional image planes stored contiguously 312 after the image data of the first page. The total number of planes 313 is equal to the counts of the UIC2tag. Tifffile can read STK files. 314* *NDPI* uses some 64-bit offsets in the file header, IFD, and tag structures. 315 Tag values/offsets can be corrected using high bits stored after IFD 316 structures. Tifffile can read NDPI files > 4 GB. 317 JPEG compressed segments with dimensions >65530 or missing restart markers 318 are not decodable with libjpeg. Tifffile works around this limitation by 319 separately decoding the MCUs between restart markers. 320 BitsPerSample, SamplesPerPixel, and PhotometricInterpretation tags may 321 contain wrong values, which can be corrected using the value of tag 65441. 322* *Philips* TIFF slides store wrong ImageWidth and ImageLength tag values for 323 tiled pages. The values can be corrected using the DICOM_PIXEL_SPACING 324 attributes of the XML formatted description of the first page. Tifffile can 325 read Philips slides. 326* *Ventana BIF* slides store tiles and metadata in a BigTIFF container. 327 Tiles may overlap and require stitching based on the TileJointInfo elements 328 in the XMP tag. Tifffile can read BigTIFF and decode individual tiles, 329 but does not perform stitching. 330* *ScanImage* optionally allows corrupt non-BigTIFF files > 2 GB. The values 331 of StripOffsets and StripByteCounts can be recovered using the constant 332 differences of the offsets of IFD and tag values throughout the file. 333 Tifffile can read such files if the image data are stored contiguously in 334 each page. 335* *GeoTIFF* sparse files allow strip or tile offsets and byte counts to be 0. 336 Such segments are implicitly set to 0 or the NODATA value on reading. 337 Tifffile can read GeoTIFF sparse files. 338 339Other libraries for reading scientific TIFF files from Python: 340 341* `Python-bioformats <https://github.com/CellProfiler/python-bioformats>`_ 342* `Imread <https://github.com/luispedro/imread>`_ 343* `GDAL <https://github.com/OSGeo/gdal/tree/master/gdal/swig/python>`_ 344* `OpenSlide-python <https://github.com/openslide/openslide-python>`_ 345* `Slideio <https://gitlab.com/bioslide/slideio>`_ 346* `PyLibTiff <https://github.com/pearu/pylibtiff>`_ 347* `SimpleITK <https://github.com/SimpleITK/SimpleITK>`_ 348* `PyLSM <https://launchpad.net/pylsm>`_ 349* `PyMca.TiffIO.py <https://github.com/vasole/pymca>`_ (same as fabio.TiffIO) 350* `BioImageXD.Readers <http://www.bioimagexd.net/>`_ 351* `CellCognition <https://cellcognition-project.org/>`_ 352* `pymimage <https://github.com/ardoi/pymimage>`_ 353* `pytiff <https://github.com/FZJ-INM1-BDA/pytiff>`_ 354* `ScanImageTiffReaderPython 355 <https://gitlab.com/vidriotech/scanimagetiffreader-python>`_ 356* `bigtiff <https://pypi.org/project/bigtiff>`_ 357* `Large Image <https://github.com/girder/large_image>`_ 358 359Some libraries are using tifffile to write OME-TIFF files: 360 361* `Zeiss Apeer OME-TIFF library 362 <https://github.com/apeer-micro/apeer-ometiff-library>`_ 363* `Allen Institute for Cell Science imageio 364 <https://pypi.org/project/aicsimageio>`_ 365* `xtiff <https://github.com/BodenmillerGroup/xtiff>`_ 366 367Other tools for inspecting and manipulating TIFF files: 368 369* `tifftools <https://github.com/DigitalSlideArchive/tifftools>`_ 370* `Tyf <https://github.com/Moustikitos/tyf>`_ 371 372References 373---------- 374* TIFF 6.0 Specification and Supplements. Adobe Systems Incorporated. 375 https://www.adobe.io/open/standards/TIFF.html 376* TIFF File Format FAQ. https://www.awaresystems.be/imaging/tiff/faq.html 377* The BigTIFF File Format. 378 https://www.awaresystems.be/imaging/tiff/bigtiff.html 379* MetaMorph Stack (STK) Image File Format. 380 http://mdc.custhelp.com/app/answers/detail/a_id/18862 381* Image File Format Description LSM 5/7 Release 6.0 (ZEN 2010). 382 Carl Zeiss MicroImaging GmbH. BioSciences. May 10, 2011 383* The OME-TIFF format. 384 https://docs.openmicroscopy.org/ome-model/latest/ 385* UltraQuant(r) Version 6.0 for Windows Start-Up Guide. 386 http://www.ultralum.com/images%20ultralum/pdf/UQStart%20Up%20Guide.pdf 387* Micro-Manager File Formats. 388 https://micro-manager.org/wiki/Micro-Manager_File_Formats 389* ScanImage BigTiff Specification - ScanImage 2019. 390 http://scanimage.vidriotechnologies.com/display/SI2019/ 391 ScanImage+BigTiff+Specification 392* ZIF, the Zoomable Image File format. http://zif.photo/ 393* GeoTIFF File Format https://gdal.org/drivers/raster/gtiff.html 394* Cloud optimized GeoTIFF. 395 https://github.com/cogeotiff/cog-spec/blob/master/spec.md 396* Tags for TIFF and Related Specifications. Digital Preservation. 397 https://www.loc.gov/preservation/digital/formats/content/tiff_tags.shtml 398* CIPA DC-008-2016: Exchangeable image file format for digital still cameras: 399 Exif Version 2.31. 400 http://www.cipa.jp/std/documents/e/DC-008-Translation-2016-E.pdf 401* The EER (Electron Event Representation) file format. 402 https://github.com/fei-company/EerReaderLib 403* Digital Negative (DNG) Specification. Version 1.4.0.0, June 2012. 404 https://www.adobe.com/content/dam/acom/en/products/photoshop/pdfs/ 405 dng_spec_1.4.0.0.pdf 406 407Examples 408-------- 409Write a numpy array to a single-page RGB TIFF file: 410 411>>> data = numpy.random.randint(0, 255, (256, 256, 3), 'uint8') 412>>> imwrite('temp.tif', data, photometric='rgb') 413 414Read the image from the TIFF file as numpy array: 415 416>>> image = imread('temp.tif') 417>>> image.shape 418(256, 256, 3) 419 420Write a 3D numpy array to a multi-page, 16-bit grayscale TIFF file: 421 422>>> data = numpy.random.randint(0, 2**12, (64, 301, 219), 'uint16') 423>>> imwrite('temp.tif', data, photometric='minisblack') 424 425Read the whole image stack from the TIFF file as numpy array: 426 427>>> image_stack = imread('temp.tif') 428>>> image_stack.shape 429(64, 301, 219) 430>>> image_stack.dtype 431dtype('uint16') 432 433Read the image from the first page in the TIFF file as numpy array: 434 435>>> image = imread('temp.tif', key=0) 436>>> image.shape 437(301, 219) 438 439Read images from a selected range of pages: 440 441>>> image = imread('temp.tif', key=range(4, 40, 2)) 442>>> image.shape 443(18, 301, 219) 444 445Iterate over all pages in the TIFF file and successively read images: 446 447>>> with TiffFile('temp.tif') as tif: 448... for page in tif.pages: 449... image = page.asarray() 450 451Get information about the image stack in the TIFF file without reading 452the image data: 453 454>>> tif = TiffFile('temp.tif') 455>>> len(tif.pages) # number of pages in the file 45664 457>>> page = tif.pages[0] # get shape and dtype of the image in the first page 458>>> page.shape 459(301, 219) 460>>> page.dtype 461dtype('uint16') 462>>> page.axes 463'YX' 464>>> series = tif.series[0] # get shape and dtype of the first image series 465>>> series.shape 466(64, 301, 219) 467>>> series.dtype 468dtype('uint16') 469>>> series.axes 470'QYX' 471>>> tif.close() 472 473Inspect the "XResolution" tag from the first page in the TIFF file: 474 475>>> with TiffFile('temp.tif') as tif: 476... tag = tif.pages[0].tags['XResolution'] 477>>> tag.value 478(1, 1) 479>>> tag.name 480'XResolution' 481>>> tag.code 482282 483>>> tag.count 4841 485>>> tag.dtype 486<DATATYPES.RATIONAL: 5> 487 488Iterate over all tags in the TIFF file: 489 490>>> with TiffFile('temp.tif') as tif: 491... for page in tif.pages: 492... for tag in page.tags: 493... tag_name, tag_value = tag.name, tag.value 494 495Overwrite the value of an existing tag, e.g. XResolution: 496 497>>> with TiffFile('temp.tif', mode='r+b') as tif: 498... _ = tif.pages[0].tags['XResolution'].overwrite((96000, 1000)) 499 500Write a floating-point ndarray and metadata using BigTIFF format, tiling, 501compression, and planar storage: 502 503>>> data = numpy.random.rand(2, 5, 3, 301, 219).astype('float32') 504>>> imwrite('temp.tif', data, bigtiff=True, photometric='minisblack', 505... compression='zlib', planarconfig='separate', tile=(32, 32), 506... metadata={'axes': 'TZCYX'}) 507 508Write a 10 fps time series of volumes with xyz voxel size 2.6755x2.6755x3.9474 509micron^3 to an ImageJ hyperstack formatted TIFF file: 510 511>>> volume = numpy.random.randn(6, 57, 256, 256).astype('float32') 512>>> imwrite('temp.tif', volume, imagej=True, resolution=(1./2.6755, 1./2.6755), 513... metadata={'spacing': 3.947368, 'unit': 'um', 'finterval': 1/10, 514... 'axes': 'TZYX'}) 515 516Read the volume and metadata from the ImageJ file: 517 518>>> with TiffFile('temp.tif') as tif: 519... volume = tif.asarray() 520... axes = tif.series[0].axes 521... imagej_metadata = tif.imagej_metadata 522>>> volume.shape 523(6, 57, 256, 256) 524>>> axes 525'TZYX' 526>>> imagej_metadata['slices'] 52757 528>>> imagej_metadata['frames'] 5296 530 531Create an empty TIFF file and write to the memory-mapped numpy array: 532 533>>> memmap_image = memmap( 534... 'temp.tif', shape=(3, 256, 256), photometric='rgb', dtype='float32' 535... ) 536>>> memmap_image[1, 255, 255] = 1.0 537>>> memmap_image.flush() 538>>> del memmap_image 539 540Memory-map image data of the first page in the TIFF file: 541 542>>> memmap_image = memmap('temp.tif', page=0) 543>>> memmap_image[1, 255, 255] 5441.0 545>>> del memmap_image 546 547Write two numpy arrays to a multi-series TIFF file: 548 549>>> series0 = numpy.random.randint(0, 255, (32, 32, 3), 'uint8') 550>>> series1 = numpy.random.randint(0, 1023, (4, 256, 256), 'uint16') 551>>> with TiffWriter('temp.tif') as tif: 552... tif.write(series0, photometric='rgb') 553... tif.write(series1, photometric='minisblack') 554 555Read the second image series from the TIFF file: 556 557>>> series1 = imread('temp.tif', series=1) 558>>> series1.shape 559(4, 256, 256) 560 561Successively write the frames of one contiguous series to a TIFF file: 562 563>>> data = numpy.random.randint(0, 255, (30, 301, 219), 'uint8') 564>>> with TiffWriter('temp.tif') as tif: 565... for frame in data: 566... tif.write(frame, contiguous=True) 567 568Append an image series to the existing TIFF file: 569 570>>> data = numpy.random.randint(0, 255, (301, 219, 3), 'uint8') 571>>> imwrite('temp.tif', data, photometric='rgb', append=True) 572 573Create a TIFF file from a generator of tiles: 574 575>>> data = numpy.random.randint(0, 2**12, (31, 33, 3), 'uint16') 576>>> def tiles(data, tileshape): 577... for y in range(0, data.shape[0], tileshape[0]): 578... for x in range(0, data.shape[1], tileshape[1]): 579... yield data[y : y + tileshape[0], x : x + tileshape[1]] 580>>> imwrite('temp.tif', tiles(data, (16, 16)), tile=(16, 16), 581... shape=data.shape, dtype=data.dtype, photometric='rgb') 582 583Write two numpy arrays to a multi-series OME-TIFF file: 584 585>>> series0 = numpy.random.randint(0, 255, (32, 32, 3), 'uint8') 586>>> series1 = numpy.random.randint(0, 1023, (4, 256, 256), 'uint16') 587>>> with TiffWriter('temp.ome.tif') as tif: 588... tif.write(series0, photometric='rgb') 589... tif.write(series1, photometric='minisblack', 590... metadata={'axes': 'ZYX', 'SignificantBits': 10, 591... 'Plane': {'PositionZ': [0.0, 1.0, 2.0, 3.0]}}) 592 593Write a tiled, multi-resolution, pyramidal, OME-TIFF file using 594JPEG compression. Sub-resolution images are written to SubIFDs: 595 596>>> data = numpy.arange(1024*1024*3, dtype='uint8').reshape((1024, 1024, 3)) 597>>> with TiffWriter('temp.ome.tif', bigtiff=True) as tif: 598... options = dict(tile=(256, 256), photometric='rgb', compression='jpeg') 599... tif.write(data, subifds=2, **options) 600... # save pyramid levels to the two subifds 601... # in production use resampling to generate sub-resolutions 602... tif.write(data[::2, ::2], subfiletype=1, **options) 603... tif.write(data[::4, ::4], subfiletype=1, **options) 604 605Access the image levels in the pyramidal OME-TIFF file: 606 607>>> baseimage = imread('temp.ome.tif') 608>>> second_level = imread('temp.ome.tif', series=0, level=1) 609>>> with TiffFile('temp.ome.tif') as tif: 610... baseimage = tif.series[0].asarray() 611... second_level = tif.series[0].levels[1].asarray() 612 613Iterate over and decode single JPEG compressed tiles in the TIFF file: 614 615>>> with TiffFile('temp.ome.tif') as tif: 616... fh = tif.filehandle 617... for page in tif.pages: 618... for index, (offset, bytecount) in enumerate( 619... zip(page.dataoffsets, page.databytecounts) 620... ): 621... fh.seek(offset) 622... data = fh.read(bytecount) 623... tile, indices, shape = page.decode( 624... data, index, jpegtables=page.jpegtables 625... ) 626 627Use zarr to read parts of the tiled, pyramidal images in the TIFF file: 628 629>>> import zarr 630>>> store = imread('temp.ome.tif', aszarr=True) 631>>> z = zarr.open(store, mode='r') 632>>> z 633<zarr.hierarchy.Group '/' read-only> 634>>> z[0] # base layer 635<zarr.core.Array '/0' (1024, 1024, 3) uint8 read-only> 636>>> z[0][256:512, 512:768].shape # read a tile from the base layer 637(256, 256, 3) 638>>> store.close() 639 640Read images from a sequence of TIFF files as numpy array: 641 642>>> imwrite('temp_C001T001.tif', numpy.random.rand(64, 64)) 643>>> imwrite('temp_C001T002.tif', numpy.random.rand(64, 64)) 644>>> image_sequence = imread(['temp_C001T001.tif', 'temp_C001T002.tif']) 645>>> image_sequence.shape 646(2, 64, 64) 647 648Read an image stack from a series of TIFF files with a file name pattern 649as numpy or zarr arrays: 650 651>>> image_sequence = TiffSequence('temp_C001*.tif', pattern='axes') 652>>> image_sequence.shape 653(1, 2) 654>>> image_sequence.axes 655'CT' 656>>> data = image_sequence.asarray() 657>>> data.shape 658(1, 2, 64, 64) 659>>> with image_sequence.aszarr() as store: 660... zarr.open(store, mode='r') 661<zarr.core.Array (1, 2, 64, 64) float64 read-only> 662>>> image_sequence.close() 663 664""" 665 666__version__ = '2021.8.30' 667 668__all__ = ( 669 'OmeXml', 670 'OmeXmlError', 671 'TIFF', 672 'TiffFile', 673 'TiffFileError', 674 'TiffFrame', 675 'TiffPage', 676 'TiffPageSeries', 677 'TiffReader', 678 'TiffSequence', 679 'TiffTag', 680 'TiffTags', 681 'TiffTagRegistry', 682 'TiffWriter', 683 'ZarrFileSequenceStore', 684 'ZarrStore', 685 'ZarrTiffStore', 686 'imread', 687 'imshow', 688 'imwrite', 689 'lsm2bin', 690 'memmap', 691 'read_micromanager_metadata', 692 'read_scanimage_metadata', 693 'tiff2fsspec', 694 'tiffcomment', 695 # utility classes and functions used by oiffile, czifile, etc. 696 'FileCache', 697 'FileHandle', 698 'FileSequence', 699 'Timer', 700 'askopenfilename', 701 'astype', 702 'create_output', 703 'enumarg', 704 'enumstr', 705 'format_size', 706 'lazyattr', 707 'matlabstr2py', 708 'natural_sorted', 709 'nullfunc', 710 'parse_kwargs', 711 'pformat', 712 'product', 713 'repeat_nd', 714 'reshape_axes', 715 'reshape_nd', 716 'squeeze_axes', 717 'stripnull', 718 'transpose_axes', 719 'update_kwargs', 720 'xml2dict', 721 # deprecated 722 'imsave', 723 '_app_show', 724) 725 726import binascii 727import collections 728import datetime 729import enum 730import glob 731import io 732import json 733import math 734import os 735import re 736import struct 737import sys 738import threading 739import time 740import warnings 741from collections.abc import Iterable, MutableMapping 742from concurrent.futures import ThreadPoolExecutor 743 744import numpy 745 746try: 747 import imagecodecs 748except Exception: 749 imagecodecs = None 750 751# delay import of mmap, pprint, fractions, xml, lxml, matplotlib, tkinter, 752# logging, subprocess, multiprocessing, tempfile, zipfile, fnmatch 753 754 755def imread(files=None, aszarr=False, **kwargs): 756 """Return image data from TIFF file(s) as numpy array or zarr storage. 757 758 Refer to the TiffFile and TiffSequence classes and their asarray 759 functions for documentation. 760 761 Parameters 762 ---------- 763 files : str, path-like, binary stream, or sequence 764 File name, seekable binary stream, glob pattern, or sequence of 765 file names. 766 aszarr : bool 767 If True, return file sequences, series, or single pages as 768 zarr storage instead of numpy array (experimental). 769 kwargs : dict 770 Parameters 'name', 'offset', 'size', and 'is_' flags are passed to 771 TiffFile or TiffSequence.imread. 772 The 'pattern', 'sort', 'container', and 'axesorder' parameters are 773 passed to TiffSequence(). 774 Other parameters are passed to the asarray or aszarr functions. 775 The first image series in the file is returned if no arguments are 776 provided. 777 778 Returns 779 ------- 780 numpy.ndarray or zarr storage 781 Image data from the specified pages. 782 Zarr storage instances must be closed after use. 783 See TiffPage.asarray for operations that are applied (or not) 784 to the raw data stored in the file. 785 786 """ 787 kwargs_file = parse_kwargs( 788 kwargs, 789 'name', 790 'offset', 791 'size', 792 # private 793 '_multifile', 794 '_useframes', 795 # deprecated, ignored 796 # TODO: remove 797 'fastij', 798 'movie', 799 'multifile', 800 'multifile_close', 801 # is_flags 802 *(key for key in kwargs if key[:3] == 'is_'), 803 ) 804 kwargs_seq = parse_kwargs( 805 kwargs, 'pattern', 'sort', 'container', 'imread', 'axesorder' 806 ) 807 808 if kwargs.get('pages', None) is not None: 809 # TODO: remove 810 if kwargs.get('key', None) is not None: 811 raise TypeError( 812 "the 'pages' and 'key' parameters cannot be used together" 813 ) 814 warnings.warn( 815 "imread: the 'pages' parameter is deprecated since 2017.9.29. " 816 "Use the 'key' parameter", 817 DeprecationWarning, 818 stacklevel=2, 819 ) 820 kwargs['key'] = kwargs.pop('pages') 821 822 if kwargs_seq.get('container', None) is None: 823 if isinstance(files, str) and ('*' in files or '?' in files): 824 files = glob.glob(files) 825 if not files: 826 raise ValueError('no files found') 827 if ( 828 not hasattr(files, 'seek') 829 and not isinstance(files, (str, os.PathLike)) 830 and len(files) == 1 831 ): 832 files = files[0] 833 834 if isinstance(files, (str, os.PathLike)) or hasattr(files, 'seek'): 835 with TiffFile(files, **kwargs_file) as tif: 836 if aszarr: 837 return tif.aszarr(**kwargs) 838 return tif.asarray(**kwargs) 839 840 with TiffSequence(files, **kwargs_seq) as imseq: 841 if aszarr: 842 return imseq.aszarr(**kwargs, **kwargs_file) 843 return imseq.asarray(**kwargs, **kwargs_file) 844 845 846def imwrite(file, data=None, shape=None, dtype=None, **kwargs): 847 """Write numpy array to TIFF file. 848 849 Refer to the TiffWriter class and its write function for documentation. 850 851 A BigTIFF file is created if the data's size is larger than 4 GB minus 852 32 MB (for metadata), and 'bigtiff' is not specified, and 'imagej' or 853 'truncate' are not enabled. 854 855 Parameters 856 ---------- 857 file : str, path-like, or binary stream 858 File name or writable binary stream, such as an open file or BytesIO. 859 data : array-like 860 Input image. The last dimensions are assumed to be image depth, 861 length, width, and samples. 862 If None, an empty array of the specified shape and dtype is 863 saved to file. 864 Unless 'byteorder' is specified in 'kwargs', the TIFF file byte order 865 is determined from the data's dtype or the dtype argument. 866 shape : tuple 867 If 'data' is None, shape of an empty array to save to the file. 868 dtype : numpy.dtype 869 If 'data' is None, datatype of an empty array to save to the file. 870 kwargs : dict 871 Parameters 'append', 'byteorder', 'bigtiff', 'imagej', and 'ome', 872 are passed to TiffWriter(). 873 Other parameters are passed to TiffWriter.write(). 874 875 Returns 876 ------- 877 offset, bytecount : tuple or None 878 If the 'returnoffset' argument is True and the image data are written 879 contiguously, return offset and bytecount of image data in the file. 880 881 """ 882 tifargs = parse_kwargs( 883 kwargs, 'append', 'bigtiff', 'byteorder', 'imagej', 'ome' 884 ) 885 if data is None: 886 dtype = numpy.dtype(dtype) 887 datasize = product(shape) * dtype.itemsize 888 byteorder = dtype.byteorder 889 else: 890 try: 891 datasize = data.nbytes 892 byteorder = data.dtype.byteorder 893 except Exception: 894 datasize = 0 895 byteorder = None 896 bigsize = kwargs.pop('bigsize', 2 ** 32 - 2 ** 25) 897 if ( 898 'bigtiff' not in tifargs 899 and datasize > bigsize 900 and not tifargs.get('imagej', False) 901 and not tifargs.get('truncate', False) 902 and not kwargs.get('compression', False) 903 and not kwargs.get('compress', False) # TODO: remove deprecated 904 ): 905 tifargs['bigtiff'] = True 906 if 'byteorder' not in tifargs: 907 tifargs['byteorder'] = byteorder 908 909 with TiffWriter(file, **tifargs) as tif: 910 result = tif.write(data, shape, dtype, **kwargs) 911 return result 912 913 914def memmap( 915 filename, 916 shape=None, 917 dtype=None, 918 page=None, 919 series=0, 920 level=0, 921 mode='r+', 922 **kwargs, 923): 924 """Return memory-mapped numpy array stored in TIFF file. 925 926 Memory-mapping requires data stored in native byte order, without tiling, 927 compression, predictors, etc. 928 If 'shape' and 'dtype' are provided, existing files are overwritten or 929 appended to depending on the 'append' parameter. 930 Otherwise the image data of a specified page or series in an existing 931 file are memory-mapped. By default, the image data of the first 932 series are memory-mapped. 933 Call flush() to write any changes in the array to the file. 934 Raise ValueError if the image data in the file are not memory-mappable. 935 936 Parameters 937 ---------- 938 filename : str or path-like 939 Name of the TIFF file which stores the array. 940 shape : tuple 941 Shape of the empty array. 942 dtype : numpy.dtype 943 Datatype of the empty array. 944 page : int 945 Index of the page which image data to memory-map. 946 series, level : int 947 Index of the page series and pyramid level which image data to 948 memory-map. 949 mode : {'r+', 'r', 'c'} 950 The file open mode. Default is to open existing file for reading and 951 writing ('r+'). 952 kwargs : dict 953 Additional parameters passed to imwrite() or TiffFile(). 954 955 Returns 956 ------- 957 numpy.memmap 958 Image data in TIFF file. 959 960 """ 961 if shape is not None and dtype is not None: 962 # create a new, empty array 963 kwargs.update( 964 data=None, 965 shape=shape, 966 dtype=dtype, 967 align=TIFF.ALLOCATIONGRANULARITY, 968 returnoffset=True, 969 ) 970 result = imwrite(filename, **kwargs) 971 if result is None: 972 # TODO: fail before creating file or writing data 973 raise ValueError('image data are not memory-mappable') 974 offset = result[0] 975 else: 976 # use existing file 977 with TiffFile(filename, **kwargs) as tif: 978 if page is not None: 979 page = tif.pages[page] 980 if not page.is_memmappable: 981 raise ValueError('image data are not memory-mappable') 982 offset, _ = page.is_contiguous 983 shape = page.shape 984 dtype = page.dtype 985 else: 986 series = tif.series[series] 987 if series.offset is None: 988 raise ValueError('image data are not memory-mappable') 989 shape = series.shape 990 dtype = series.dtype 991 offset = series.offset 992 dtype = tif.byteorder + dtype.char 993 return numpy.memmap(filename, dtype, mode, offset, shape, 'C') 994 995 996class lazyattr: 997 """Attribute whose value is computed on first access. 998 999 Lazyattrs are not thread-safe. 1000 1001 """ 1002 1003 # TODO: replace with functools.cached_property? requires Python >= 3.8 1004 __slots__ = ('func', '__dict__') 1005 1006 def __init__(self, func): 1007 """Initialize instance from decorated function.""" 1008 self.func = func 1009 self.__doc__ = func.__doc__ 1010 self.__module__ = func.__module__ 1011 self.__name__ = func.__name__ 1012 self.__qualname__ = func.__qualname__ 1013 # self.lock = threading.RLock() 1014 1015 def __get__(self, instance, owner): 1016 # with self.lock: 1017 if instance is None: 1018 return self 1019 try: 1020 value = self.func(instance) 1021 except AttributeError as exc: 1022 raise RuntimeError(exc) 1023 if value is NotImplemented: 1024 return getattr(super(owner, instance), self.func.__name__) 1025 setattr(instance, self.func.__name__, value) 1026 return value 1027 1028 1029class TiffFileError(Exception): 1030 """Exception to indicate invalid TIFF structure.""" 1031 1032 1033class TiffWriter: 1034 """Write numpy arrays to TIFF file. 1035 1036 TiffWriter's main purpose is saving nD numpy array's as TIFF, not to 1037 create any possible TIFF format. Specifically, ExifIFD and GPSIFD tags. 1038 1039 TiffWriter instances must be closed using the 'close' method, which is 1040 automatically called when using the 'with' context manager. 1041 1042 TiffWriter instances are not thread-safe. 1043 1044 """ 1045 1046 def __init__( 1047 self, 1048 file, 1049 bigtiff=False, 1050 byteorder=None, 1051 append=False, 1052 imagej=False, 1053 ome=None, 1054 ): 1055 """Open TIFF file for writing. 1056 1057 An empty TIFF file is created if the file does not exist, else the 1058 file is overwritten with an empty TIFF file unless 'append' 1059 is true. Use 'bigtiff=True' when creating files larger than 4 GB. 1060 1061 Parameters 1062 ---------- 1063 file : str, path-like, binary stream, or FileHandle 1064 File name or writable binary stream, such as an open file 1065 or BytesIO. 1066 bigtiff : bool 1067 If True, the BigTIFF format is used. 1068 byteorder : {'<', '>', '=', '|'} 1069 The endianness of the data in the file. 1070 By default, this is the system's native byte order. 1071 append : bool 1072 If True and 'file' is an existing standard TIFF file, image data 1073 and tags are appended to the file. 1074 Appending data may corrupt specifically formatted TIFF files 1075 such as OME-TIFF, LSM, STK, ImageJ, or FluoView. 1076 imagej : bool 1077 If True and not 'ome', write an ImageJ hyperstack compatible file. 1078 This format can handle data types uint8, uint16, or float32 and 1079 data shapes up to 6 dimensions in TZCYXS order. 1080 RGB images (S=3 or S=4) must be uint8. 1081 ImageJ's default byte order is big-endian but this implementation 1082 uses the system's native byte order by default. 1083 ImageJ hyperstacks do not support BigTIFF or compression. 1084 The ImageJ file format is undocumented. 1085 When using compression, use ImageJ's Bio-Formats import function. 1086 ome : bool 1087 If True, write an OME-TIFF compatible file. If None (default), 1088 the value is determined from the file name extension, the value of 1089 the 'description' parameter in the first call of the write 1090 function, and the value of 'imagej'. 1091 Refer to the OME model for restrictions of this format. 1092 1093 """ 1094 if append: 1095 # determine if file is an existing TIFF file that can be extended 1096 try: 1097 with FileHandle(file, mode='rb', size=0) as fh: 1098 pos = fh.tell() 1099 try: 1100 with TiffFile(fh) as tif: 1101 if append != 'force' and not tif.is_appendable: 1102 raise ValueError( 1103 'cannot append to file containing metadata' 1104 ) 1105 byteorder = tif.byteorder 1106 bigtiff = tif.is_bigtiff 1107 self._ifdoffset = tif.pages.next_page_offset 1108 finally: 1109 fh.seek(pos) 1110 except (OSError, FileNotFoundError): 1111 append = False 1112 1113 if imagej and bigtiff: 1114 warnings.warn( 1115 'TiffWriter: writing nonconformant BigTIFF ImageJ', UserWarning 1116 ) 1117 1118 if byteorder in (None, '=', '|'): 1119 byteorder = '<' if sys.byteorder == 'little' else '>' 1120 elif byteorder not in ('<', '>'): 1121 raise ValueError(f'invalid byteorder {byteorder}') 1122 1123 if byteorder == '<': 1124 self.tiff = TIFF.BIG_LE if bigtiff else TIFF.CLASSIC_LE 1125 else: 1126 self.tiff = TIFF.BIG_BE if bigtiff else TIFF.CLASSIC_BE 1127 1128 self._truncate = False 1129 self._metadata = None 1130 self._colormap = None 1131 self._tags = None 1132 self._datashape = None # shape of data in consecutive pages 1133 self._datadtype = None # data type 1134 self._dataoffset = None # offset to data 1135 self._databytecounts = None # byte counts per plane 1136 self._dataoffsetstag = None # strip or tile offset tag code 1137 self._descriptiontag = None # TiffTag for updating comment 1138 self._subifds = 0 # number of subifds 1139 self._subifdslevel = -1 # index of current subifd level 1140 self._subifdsoffsets = [] # offsets to offsets to subifds 1141 self._nextifdoffsets = [] # offsets to offset to next ifd 1142 self._ifdindex = 0 # index of current ifd 1143 1144 # normalized shape of data in consecutive pages 1145 # (pages, separate_samples, depth, length, width, contig_samples) 1146 self._storedshape = None 1147 1148 if append: 1149 self._fh = FileHandle(file, mode='r+b', size=0) 1150 self._fh.seek(0, os.SEEK_END) 1151 else: 1152 self._fh = FileHandle(file, mode='wb', size=0) 1153 self._fh.write({'<': b'II', '>': b'MM'}[byteorder]) 1154 if bigtiff: 1155 self._fh.write(struct.pack(byteorder + 'HHH', 43, 8, 0)) 1156 else: 1157 self._fh.write(struct.pack(byteorder + 'H', 42)) 1158 # first IFD 1159 self._ifdoffset = self._fh.tell() 1160 self._fh.write(struct.pack(self.tiff.offsetformat, 0)) 1161 1162 self._ome = None if ome is None else bool(ome) 1163 self._imagej = False if self._ome else bool(imagej) 1164 if self._imagej: 1165 self._ome = False 1166 1167 @property 1168 def filehandle(self): 1169 """Return file handle.""" 1170 return self._fh 1171 1172 def write( 1173 self, 1174 data=None, 1175 shape=None, 1176 dtype=None, 1177 photometric=None, 1178 planarconfig=None, 1179 extrasamples=None, 1180 volumetric=False, 1181 tile=None, 1182 contiguous=False, 1183 truncate=False, 1184 align=None, 1185 rowsperstrip=None, 1186 bitspersample=None, 1187 compression=None, 1188 predictor=None, 1189 subsampling=None, 1190 jpegtables=None, 1191 colormap=None, 1192 description=None, 1193 datetime=None, 1194 resolution=None, 1195 subfiletype=0, 1196 software=None, 1197 subifds=None, 1198 metadata={}, 1199 extratags=(), 1200 returnoffset=False, 1201 ijmetadata=None, # deprecated: use metadata 1202 compress=None, # deprecated: use compression 1203 ): 1204 """Write numpy ndarray to a series of TIFF pages. 1205 1206 The ND image data are written to a series of TIFF pages/IFDs. 1207 By default, metadata in JSON, ImageJ, or OME-XML format are written 1208 to the ImageDescription tag of the first page to describe the series 1209 such that the image data can later be read back as a ndarray of same 1210 shape. 1211 1212 The data shape's last dimensions are assumed to be image depth, 1213 length (height), width, and samples. 1214 If a colormap is provided, the data's dtype must be uint8 or uint16 1215 and the data values are indices into the last dimension of the 1216 colormap. 1217 If 'shape' and 'dtype' are specified instead of 'data', an empty array 1218 is saved. This option cannot be used with compression, predictors, 1219 packed integers, bilevel images, or multiple tiles. 1220 If 'shape', 'dtype', and 'tile' are specified, 'data' must be an 1221 iterable of all tiles in the image. 1222 If 'shape', 'dtype', and 'data' are specified but not 'tile', 'data' 1223 must be an iterable of all single planes in the image. 1224 Image data are written uncompressed in one strip per plane by default. 1225 Dimensions larger than 2 to 4 (depending on photometric mode, planar 1226 configuration, and volumetric mode) are flattened and saved as separate 1227 pages. 1228 If the data size is zero, a single page with shape (0, 0) is saved. 1229 The SampleFormat tag is derived from the data type or dtype. 1230 1231 A UserWarning is logged if RGB colorspace is auto-detected. Specify 1232 the 'photometric' parameter to avoid the warning. 1233 1234 Parameters 1235 ---------- 1236 data : numpy.ndarray, iterable of numpy.ndarray, or None 1237 Input image or iterable of tiles or images. 1238 A copy of the image data is made if 'data' is not a C-contiguous 1239 numpy array with the same byteorder as the TIFF file. 1240 Iterables must yield C-contiguous numpy array of TIFF byteorder. 1241 Iterable tiles must match 'dtype' and the shape specified in 1242 'tile'. Incomplete tiles are zero-padded. 1243 Iterable images must match 'dtype' and 'shape[1:]'. 1244 shape : tuple or None 1245 Shape of the empty or iterable data to save. 1246 Use only if 'data' is None or an iterable of tiles or images. 1247 dtype : numpy.dtype or None 1248 Datatype of the empty or iterable data to save. 1249 Use only if 'data' is None or an iterable of tiles or images. 1250 photometric : {MINISBLACK, MINISWHITE, RGB, PALETTE, SEPARATED, CFA} 1251 The color space of the image data according to TIFF.PHOTOMETRIC. 1252 By default, this setting is inferred from the data shape, dtype, 1253 and the value of colormap. Always specify this parameter to avoid 1254 ambiguities. 1255 For CFA images, the CFARepeatPatternDim, CFAPattern, and other 1256 DNG or TIFF/EP tags must be specified in 'extratags' to produce a 1257 valid file. 1258 planarconfig : {CONTIG, SEPARATE} 1259 Specifies if samples are stored interleaved or in separate planes. 1260 By default, this setting is inferred from the data shape. 1261 If this parameter is set, extra samples are used to store grayscale 1262 images. 1263 CONTIG: last dimension contains samples. 1264 SEPARATE: third (or fourth) last dimension contains samples. 1265 extrasamples : tuple of {UNSPECIFIED, ASSOCALPHA, UNASSALPHA} 1266 Defines the interpretation of extra components in pixels. 1267 UNSPECIFIED: no transparency information (default). 1268 ASSOCALPHA: single, true transparency with pre-multiplied color. 1269 UNASSALPHA: independent transparency masks. 1270 volumetric : bool 1271 If True, the SGI ImageDepth tag is used to save volumetric data 1272 in one page. The volumetric format is not officially specified, 1273 and few software can read it. OME and ImageJ formats are not 1274 compatible with volumetric storage. 1275 tile : tuple of int 1276 The shape ([depth,] length, width) of image tiles to write. 1277 If None (default), image data are written in strips. 1278 The tile length and width must be a multiple of 16. 1279 If a tile depth is provided, the SGI ImageDepth and TileDepth 1280 tags are used to save volumetric data. 1281 Tiles cannot be used to write contiguous series, except if tile 1282 matches the data shape. 1283 contiguous : bool 1284 If False (default), save data to a new series. 1285 If True and the data and parameters are compatible with previous 1286 saved ones (same shape, no compression, etc.), the image data 1287 are stored contiguously after the previous one. In that case, 1288 'photometric', 'planarconfig', and 'rowsperstrip' are ignored. 1289 Metadata such as 'description', 'metadata', 'datetime', and 1290 'extratags' are written to the first page of a contiguous series 1291 only. Cannot be used with the OME or ImageJ formats. 1292 truncate : bool 1293 If True, only write the first page of a contiguous series if 1294 possible (uncompressed, contiguous, not tiled). 1295 Other TIFF readers will only be able to read part of the data. 1296 Cannot be used with the OME or ImageJ formats. 1297 align : int 1298 Byte boundary on which to align the image data in the file. 1299 Default 16. Use mmap.ALLOCATIONGRANULARITY for memory-mapped data. 1300 Following contiguous writes are not aligned. 1301 rowsperstrip : int 1302 The number of rows per strip. By default, strips are ~64 KB if 1303 compression is enabled, else rowsperstrip is set to the image 1304 length. 1305 bitspersample : int 1306 Number of bits per sample. By default, this is the number of 1307 bits of the data dtype. Different values for different samples 1308 are not supported. Unsigned integer data are packed into bytes 1309 as tightly as possible. Valid values are 1-8 for uint8, 9-16 for 1310 uint16 and 17-32 for uint32. Cannot be used with compression, 1311 contiguous series, or empty files. 1312 compression : str, (str, int), (str, int, dict) 1313 If None (default), data are written uncompressed. 1314 If a str, one of TIFF.COMPRESSION, e.g. 'JPEG' or 'ZSTD'. 1315 If a tuple, the first item is one of TIFF.COMPRESSION, the 1316 second item is the compression level, and the third item is a dict 1317 of arguments passed to the compression codec. 1318 Compression cannot be used to write contiguous series. 1319 Compressors may require certain data shapes, types or value ranges. 1320 For example, JPEG requires grayscale or RGB(A), uint8 or 12-bit 1321 uint16. JPEG compression is experimental. JPEG markers and TIFF 1322 tags may not match. 1323 Only a limited set of compression shemes are implemented. 1324 predictor : bool or TIFF.PREDICTOR 1325 If True, apply horizontal differencing or floating-point predictor 1326 before compression. Predictors are disabled for 64-bit integers. 1327 subsampling : {(1, 1), (2, 1), (2, 2), (4, 1)} 1328 The horizontal and vertical subsampling factors used for the 1329 chrominance components of images. The default is (2, 2). 1330 Currently applies to JPEG compression of RGB images only. 1331 Images are stored in YCbCr color space. 1332 Segment widths must be a multiple of 8 times the horizontal factor. 1333 Segment lengths and rowsperstrip must be a multiple of 8 times the 1334 vertical factor. 1335 jpegtables : bytes 1336 JPEG quantization and/or Huffman tables. Use for copying 1337 pre-compressed JPEG segments. 1338 colormap : numpy.ndarray 1339 RGB color values for the corresponding data value. 1340 Must be of shape (3, 2**(data.itemsize*8)) and dtype uint16. 1341 description : str or encoded bytes 1342 The subject of the image. Must be 7-bit ASCII. Cannot be used with 1343 the ImageJ or OME formats. Saved with the first page of a series 1344 only. 1345 datetime : datetime, str, or bool 1346 Date and time of image creation in '%Y:%m:%d %H:%M:%S' format or 1347 datetime object. Else if True, the current date and time is used. 1348 Saved with the first page of a series only. 1349 resolution : (float, float[, str]) or ((int, int), (int, int)[, str]) 1350 X and Y resolutions in pixels per resolution unit as float or 1351 rational numbers. A third, optional parameter specifies the 1352 resolution unit, which must be None (default for ImageJ), 1353 'INCH' (default), or 'CENTIMETER'. 1354 subfiletype : int 1355 Bitfield to indicate the kind of data. Set bit 0 if the image 1356 is a reduced-resolution version of another image. Set bit 1 if 1357 the image is part of a multi-page image. Set bit 2 if the image 1358 is transparency mask for another image (photometric must be 1359 MASK, SamplesPerPixel and BitsPerSample must be 1). 1360 software : str or bool 1361 Name of the software used to create the file. 1362 If None (default), 'tifffile.py'. Must be 7-bit ASCII. 1363 Saved with the first page of a series only. 1364 subifds : int 1365 Number of child IFDs. If greater than 0, the following 'subifds' 1366 number of series are written as child IFDs of the current 1367 series. The number of IFDs written for each SubIFD level must match 1368 the number of IFDs written for the current series. All pages 1369 written to a certain SubIFD level of the current series must have 1370 the same hash. SubIFDs cannot be used with truncated or ImageJ 1371 files. SubIFDs in OME-TIFF files must be sub-resolutions of the 1372 main IFDs. 1373 metadata : dict 1374 Additional metadata describing the image data, saved along with 1375 shape information in JSON, OME-XML, or ImageJ formats in 1376 ImageDescription or IJMetadata tags. 1377 If None, do not write an ImageDescription tag with shape in JSON 1378 format. 1379 If ImageJ format, values for keys 'Info', 'Labels', 'Ranges', 1380 'LUTs', 'Plot', 'ROI', and 'Overlays' are saved in IJMetadata and 1381 IJMetadataByteCounts tags. Refer to the imagej_metadata_tag 1382 function for valid values. 1383 Refer to the OmeXml class for supported keys when writing OME-TIFF. 1384 Strings must be 7-bit ASCII. 1385 Saved with the first page of a series only. 1386 extratags : sequence of tuples 1387 Additional tags as [(code, dtype, count, value, writeonce)]. 1388 1389 code : int 1390 The TIFF tag Id. 1391 dtype : int or str 1392 Data type of items in 'value'. One of TIFF.DATATYPES. 1393 count : int 1394 Number of data values. Not used for string or bytes values. 1395 value : sequence 1396 'Count' values compatible with 'dtype'. 1397 Bytes must contain count values of dtype packed as binary data. 1398 writeonce : bool 1399 If True, the tag is written to the first page of a series only. 1400 1401 returnoffset : bool 1402 If True and the image data in the file are memory-mappable, return 1403 the offset and number of bytes of the image data in the file. 1404 1405 Returns 1406 ------- 1407 offset, bytecount : tuple or None 1408 If 'returnoffset' is true and the image data in the file are 1409 memory-mappable, return the offset and number of bytes of the 1410 image data in the file. 1411 1412 """ 1413 # TODO: refactor this function 1414 fh = self._fh 1415 byteorder = self.tiff.byteorder 1416 1417 if data is None: 1418 # empty 1419 dataiter = None 1420 datashape = tuple(shape) 1421 datadtype = numpy.dtype(dtype).newbyteorder(byteorder) 1422 datadtypechar = datadtype.char 1423 elif ( 1424 shape is not None 1425 and dtype is not None 1426 and hasattr(data, '__iter__') 1427 ): 1428 # iterable pages or tiles 1429 if hasattr(data, '__next__'): 1430 dataiter = data 1431 else: 1432 dataiter = iter(data) 1433 datashape = tuple(shape) 1434 datadtype = numpy.dtype(dtype).newbyteorder(byteorder) 1435 datadtypechar = datadtype.char 1436 elif hasattr(data, '__next__'): 1437 # generator 1438 raise TypeError( 1439 "generators require 'shape' and 'dtype' parameters" 1440 ) 1441 else: 1442 # whole image data 1443 # must be C-contiguous numpy array of TIFF byteorder 1444 if hasattr(data, 'dtype'): 1445 data = numpy.asarray(data, byteorder + data.dtype.char, 'C') 1446 else: 1447 datadtype = numpy.dtype(dtype).newbyteorder(byteorder) 1448 data = numpy.asarray(data, datadtype, 'C') 1449 1450 if dtype is not None and dtype != data.dtype: 1451 warnings.warn( 1452 "TiffWriter: ignoring 'dtype' argument", UserWarning 1453 ) 1454 if shape is not None and shape != data.shape: 1455 warnings.warn( 1456 "TiffWriter: ignoring 'shape' argument", UserWarning 1457 ) 1458 dataiter = None 1459 datashape = data.shape 1460 datadtype = data.dtype 1461 datadtypechar = data.dtype.char 1462 1463 returnoffset = returnoffset and datadtype.isnative 1464 1465 if compression is None and compress is not None: 1466 # TODO: remove 1467 warnings.warn( 1468 "TiffWriter: the 'compress' parameter is deprecated " 1469 "since 2020.9.30. Use the 'compression' parameter", 1470 DeprecationWarning, 1471 stacklevel=2, 1472 ) 1473 if isinstance(compress, (int, numpy.integer)) and compress > 0: 1474 # ADOBE_DEFLATE 1475 compression = 8, int(compress) 1476 if not 0 < compress <= 9: 1477 raise ValueError(f'invalid compression level {compress}') 1478 else: 1479 compression = compress 1480 del compress 1481 1482 bilevel = datadtypechar == '?' 1483 if bilevel: 1484 index = -1 if datashape[-1] > 1 else -2 1485 datasize = product(datashape[:index]) 1486 if datashape[index] % 8: 1487 datasize *= datashape[index] // 8 + 1 1488 else: 1489 datasize *= datashape[index] // 8 1490 else: 1491 datasize = product(datashape) * datadtype.itemsize 1492 1493 if datasize == 0: 1494 data = None 1495 compression = False 1496 bitspersample = None 1497 if metadata is not None: 1498 truncate = True 1499 elif compression in (None, 0, 1, 'NONE', 'none'): 1500 compression = False 1501 1502 inputshape = datashape 1503 1504 packints = ( 1505 bitspersample is not None 1506 and bitspersample != datadtype.itemsize * 8 1507 ) 1508 1509 # just append contiguous data if possible 1510 if self._datashape is not None: 1511 if ( 1512 not contiguous 1513 or self._datashape[1:] != datashape 1514 or self._datadtype != datadtype 1515 or not numpy.array_equal(colormap, self._colormap) 1516 ): 1517 # incompatible shape, dtype, or colormap 1518 self._write_remaining_pages() 1519 1520 if self._imagej: 1521 raise ValueError( 1522 'ImageJ does not support non-contiguous series' 1523 ) 1524 elif self._ome: 1525 if self._subifdslevel < 0: 1526 # add image to OME-XML 1527 self._ome.addimage( 1528 self._datadtype, 1529 self._datashape[ 1530 0 if self._datashape[0] != 1 else 1 : 1531 ], 1532 self._storedshape, 1533 **self._metadata, 1534 ) 1535 elif metadata is not None: 1536 self._write_image_description() 1537 # description might have been appended to file 1538 fh.seek(0, os.SEEK_END) 1539 1540 if self._subifds: 1541 if self._truncate or truncate: 1542 raise ValueError( 1543 'SubIFDs cannot be used with truncated series' 1544 ) 1545 self._subifdslevel += 1 1546 if self._subifdslevel == self._subifds: 1547 # done with writing SubIFDs 1548 self._nextifdoffsets = [] 1549 self._subifdsoffsets = [] 1550 self._subifdslevel = -1 1551 self._subifds = 0 1552 self._ifdindex = 0 1553 elif subifds: 1554 raise ValueError( 1555 'SubIFDs in SubIFDs are not supported' 1556 ) 1557 1558 self._datashape = None 1559 self._colormap = None 1560 1561 elif compression or packints or tile: 1562 raise ValueError( 1563 'contiguous cannot be used with compression, tiles, etc.' 1564 ) 1565 1566 else: 1567 # consecutive mode 1568 # write contiguous data, write IFDs/tags later 1569 self._datashape = (self._datashape[0] + 1,) + datashape 1570 offset = fh.tell() 1571 if data is None: 1572 fh.write_empty(datasize) 1573 else: 1574 fh.write_array(data) 1575 if returnoffset: 1576 return offset, datasize 1577 return None 1578 1579 if self._ome is None: 1580 if description is None: 1581 self._ome = '.ome.tif' in fh.name 1582 else: 1583 self._ome = False 1584 self._truncate = False if self._ome else bool(truncate) 1585 1586 if datasize == 0: 1587 # write single placeholder TiffPage for arrays with size=0 1588 datashape = (0, 0) 1589 warnings.warn( 1590 'TiffWriter: writing zero size array to nonconformant TIFF', 1591 UserWarning, 1592 ) 1593 # TODO: reconsider this 1594 # raise ValueError('cannot save zero size array') 1595 1596 valueformat = f'{self.tiff.offsetsize}s' 1597 tagnoformat = self.tiff.tagnoformat 1598 offsetformat = self.tiff.offsetformat 1599 offsetsize = self.tiff.offsetsize 1600 tagsize = self.tiff.tagsize 1601 1602 MINISBLACK = TIFF.PHOTOMETRIC.MINISBLACK 1603 MINISWHITE = TIFF.PHOTOMETRIC.MINISWHITE 1604 RGB = TIFF.PHOTOMETRIC.RGB 1605 YCBCR = TIFF.PHOTOMETRIC.YCBCR 1606 PALETTE = TIFF.PHOTOMETRIC.PALETTE 1607 CONTIG = TIFF.PLANARCONFIG.CONTIG 1608 SEPARATE = TIFF.PLANARCONFIG.SEPARATE 1609 1610 # parse input 1611 if photometric is not None: 1612 photometric = enumarg(TIFF.PHOTOMETRIC, photometric) 1613 if planarconfig: 1614 planarconfig = enumarg(TIFF.PLANARCONFIG, planarconfig) 1615 if predictor: 1616 if not isinstance(predictor, bool): 1617 predictor = bool(enumarg(TIFF.PREDICTOR, predictor)) 1618 if extrasamples is None: 1619 extrasamples_ = None 1620 else: 1621 extrasamples_ = tuple( 1622 enumarg(TIFF.EXTRASAMPLE, es) for es in sequence(extrasamples) 1623 ) 1624 1625 if compression: 1626 if isinstance(compression, (tuple, list)): 1627 if len(compression) == 2: 1628 compressionargs = {'level': compression[1]} 1629 else: 1630 compressionargs = dict(compression[2]) 1631 if compression[1] is not None: 1632 compressionargs['level'] = compression[1] 1633 compression = compression[0] 1634 else: 1635 compressionargs = {} 1636 if isinstance(compression, str): 1637 compression = compression.upper() 1638 if compression == 'ZLIB': 1639 compression = 8 # ADOBE_DEFLATE 1640 compressiontag = enumarg(TIFF.COMPRESSION, compression) 1641 compression = compressiontag > 1 1642 else: 1643 compression = False 1644 compressiontag = 1 1645 1646 if not compression: 1647 compressionargs = {} 1648 predictor = False # TODO: support predictors without compression? 1649 predictortag = 1 1650 1651 if predictor: 1652 if compressiontag in ( 1653 7, 1654 33003, 1655 33004, 1656 33005, 1657 33007, 1658 34712, 1659 34892, 1660 34933, 1661 34934, 1662 50001, 1663 50002, 1664 ): 1665 # disable predictor for JPEG, JPEG2000, WEBP, PNG, JPEGXR 1666 predictor = False 1667 elif datadtype.kind in 'iu': 1668 if datadtype.itemsize > 4: 1669 predictor = False # disable predictor for 64 bit 1670 else: 1671 predictortag = 2 1672 predictor = TIFF.PREDICTORS[2] 1673 elif datadtype.kind == 'f': 1674 predictortag = 3 1675 predictor = TIFF.PREDICTORS[3] 1676 else: 1677 raise ValueError(f'cannot apply predictor to {datadtype}') 1678 1679 if self._ome: 1680 if description is not None: 1681 warnings.warn( 1682 'TiffWriter: not writing description to OME-TIFF', 1683 UserWarning, 1684 ) 1685 description = None 1686 if not isinstance(self._ome, OmeXml): 1687 self._ome = OmeXml(**metadata) 1688 if volumetric or (tile and len(tile) > 2): 1689 raise ValueError('OME-TIFF does not support ImageDepth') 1690 volumetric = False 1691 1692 elif self._imagej: 1693 # if tile is not None or predictor or compression: 1694 # warnings.warn( 1695 # 'ImageJ does not support tiles, predictors, compression' 1696 # ) 1697 if description is not None: 1698 warnings.warn( 1699 'TiffWriter: not writing description to ImageJ file', 1700 UserWarning, 1701 ) 1702 description = None 1703 if datadtypechar not in 'BHhf': 1704 raise ValueError( 1705 f'ImageJ does not support data type {datadtypechar!r}' 1706 ) 1707 if volumetric or (tile and len(tile) > 2): 1708 raise ValueError('ImageJ does not support ImageDepth') 1709 volumetric = False 1710 ijrgb = photometric == RGB if photometric else None 1711 if datadtypechar not in 'B': 1712 ijrgb = False 1713 ijshape = imagej_shape( 1714 datashape, ijrgb, metadata.get('axes', None) 1715 ) 1716 if ijshape[-1] in (3, 4): 1717 photometric = RGB 1718 if datadtypechar not in 'B': 1719 raise ValueError( 1720 'ImageJ does not support ' 1721 f'data type {datadtypechar!r} for RGB' 1722 ) 1723 elif photometric is None: 1724 photometric = MINISBLACK 1725 planarconfig = None 1726 if planarconfig == SEPARATE: 1727 raise ValueError('ImageJ does not support planar images') 1728 planarconfig = CONTIG if ijrgb else None 1729 1730 # verify colormap and indices 1731 if colormap is not None: 1732 if datadtypechar not in 'BH': 1733 raise ValueError('invalid data dtype for palette mode') 1734 colormap = numpy.asarray(colormap, dtype=byteorder + 'H') 1735 if colormap.shape != (3, 2 ** (datadtype.itemsize * 8)): 1736 raise ValueError('invalid color map shape') 1737 self._colormap = colormap 1738 1739 if tile: 1740 # verify tile shape 1741 tile = tuple(int(i) for i in tile[:3]) 1742 if ( 1743 len(tile) < 2 1744 or tile[-1] % 16 1745 or tile[-2] % 16 1746 or any(i < 1 for i in tile) 1747 ): 1748 raise ValueError('invalid tile shape') 1749 if volumetric and len(tile) == 2: 1750 tile = (1,) + tile 1751 volumetric = len(tile) == 3 1752 else: 1753 tile = () 1754 volumetric = bool(volumetric) 1755 1756 # normalize data shape to 5D or 6D, depending on volume: 1757 # (pages, separate_samples, [depth,] length, width, contig_samples) 1758 storedshape = reshape_nd( 1759 datashape, TIFF.PHOTOMETRIC_SAMPLES.get(photometric, 2) 1760 ) 1761 shape = storedshape 1762 ndim = len(storedshape) 1763 1764 samplesperpixel = 1 1765 extrasamples = 0 1766 1767 if volumetric and ndim < 3: 1768 volumetric = False 1769 1770 if colormap is not None: 1771 photometric = PALETTE 1772 planarconfig = None 1773 1774 if photometric is None: 1775 deprecate = False 1776 photometric = MINISBLACK 1777 if bilevel: 1778 photometric = MINISWHITE 1779 elif planarconfig == CONTIG: 1780 if ndim > 2 and shape[-1] in (3, 4): 1781 photometric = RGB 1782 deprecate = datadtypechar not in 'BH' 1783 elif planarconfig == SEPARATE: 1784 if volumetric and ndim > 3 and shape[-4] in (3, 4): 1785 photometric = RGB 1786 deprecate = True 1787 elif ndim > 2 and shape[-3] in (3, 4): 1788 photometric = RGB 1789 deprecate = True 1790 elif ndim > 2 and shape[-1] in (3, 4): 1791 photometric = RGB 1792 planarconfig = CONTIG 1793 deprecate = datadtypechar not in 'BH' 1794 elif self._imagej or self._ome: 1795 photometric = MINISBLACK 1796 planarconfig = None 1797 elif volumetric and ndim > 3 and shape[-4] in (3, 4): 1798 photometric = RGB 1799 planarconfig = SEPARATE 1800 deprecate = True 1801 elif ndim > 2 and shape[-3] in (3, 4): 1802 photometric = RGB 1803 planarconfig = SEPARATE 1804 deprecate = True 1805 1806 if deprecate: 1807 if planarconfig == CONTIG: 1808 msg = 'contiguous samples', "parameter is" 1809 else: 1810 msg = ( 1811 'separate component planes', 1812 "and 'planarconfig' parameters are", 1813 ) 1814 warnings.warn( 1815 f"TiffWriter: data with shape {datashape} and dtype " 1816 f"'{datadtype}'' are stored as RGB with {msg[0]}. " 1817 "Future versions will store such data as MINISBLACK in " 1818 "separate pages by default unless the 'photometric' " 1819 f"{msg[1]} specified.", 1820 DeprecationWarning, 1821 stacklevel=2, 1822 ) 1823 del msg 1824 del deprecate 1825 1826 del datashape 1827 photometricsamples = TIFF.PHOTOMETRIC_SAMPLES[photometric] 1828 1829 if planarconfig and len(shape) <= (3 if volumetric else 2): 1830 # TODO: raise error? 1831 planarconfig = None 1832 if photometricsamples > 1: 1833 photometric = MINISBLACK 1834 1835 if photometricsamples > 1: 1836 if len(shape) < 3: 1837 raise ValueError(f'not a {photometric!r} image') 1838 if len(shape) < 4: 1839 volumetric = False 1840 if planarconfig is None: 1841 if photometric == RGB: 1842 samples_ = (photometricsamples, 4) # allow common alpha 1843 else: 1844 samples_ = (photometricsamples,) 1845 if shape[-1] in samples_: 1846 planarconfig = CONTIG 1847 elif shape[-4 if volumetric else -3] in samples_: 1848 planarconfig = SEPARATE 1849 elif shape[-1] > shape[-4 if volumetric else -3]: 1850 # TODO: deprecated this? 1851 planarconfig = SEPARATE 1852 else: 1853 planarconfig = CONTIG 1854 if planarconfig == CONTIG: 1855 storedshape = (-1, 1) + shape[(-4 if volumetric else -3) :] 1856 samplesperpixel = storedshape[-1] 1857 else: 1858 storedshape = ( 1859 (-1,) + shape[(-4 if volumetric else -3) :] + (1,) 1860 ) 1861 samplesperpixel = storedshape[1] 1862 if samplesperpixel > photometricsamples: 1863 extrasamples = samplesperpixel - photometricsamples 1864 1865 elif photometric == TIFF.PHOTOMETRIC.CFA: 1866 if len(shape) != 2: 1867 raise ValueError('invalid CFA image') 1868 volumetric = False 1869 planarconfig = None 1870 storedshape = (-1, 1) + shape[-2:] + (1,) 1871 # if all(et[0] != 50706 for et in extratags): 1872 # raise ValueError('must specify DNG tags for CFA image') 1873 1874 elif planarconfig and len(shape) > (3 if volumetric else 2): 1875 if planarconfig == CONTIG: 1876 storedshape = (-1, 1) + shape[(-4 if volumetric else -3) :] 1877 samplesperpixel = storedshape[-1] 1878 else: 1879 storedshape = ( 1880 (-1,) + shape[(-4 if volumetric else -3) :] + (1,) 1881 ) 1882 samplesperpixel = storedshape[1] 1883 extrasamples = samplesperpixel - 1 1884 1885 else: 1886 planarconfig = None 1887 while len(shape) > 2 and shape[-1] == 1: 1888 shape = shape[:-1] # remove trailing 1s 1889 if len(shape) < 3: 1890 volumetric = False 1891 if extrasamples_ is None: 1892 storedshape = ( 1893 (-1, 1) + shape[(-3 if volumetric else -2) :] + (1,) 1894 ) 1895 else: 1896 storedshape = (-1, 1) + shape[(-4 if volumetric else -3) :] 1897 samplesperpixel = storedshape[-1] 1898 extrasamples = samplesperpixel - 1 1899 1900 if subfiletype & 0b100: 1901 # FILETYPE_MASK 1902 if not ( 1903 bilevel and samplesperpixel == 1 and photometric in (0, 1, 4) 1904 ): 1905 raise ValueError('invalid SubfileType MASK') 1906 photometric = TIFF.PHOTOMETRIC.MASK 1907 1908 packints = False 1909 if bilevel: 1910 if bitspersample is not None and bitspersample != 1: 1911 raise ValueError('bitspersample must be 1 for bilevel') 1912 bitspersample = 1 1913 elif compressiontag == 7 and datadtype == 'uint16': 1914 if bitspersample is not None and bitspersample != 12: 1915 raise ValueError( 1916 'bitspersample must be 12 for JPEG compressed uint16' 1917 ) 1918 bitspersample = 12 # use 12-bit JPEG compression 1919 elif bitspersample is None: 1920 bitspersample = datadtype.itemsize * 8 1921 elif ( 1922 datadtype.kind != 'u' or datadtype.itemsize > 4 1923 ) and bitspersample != datadtype.itemsize * 8: 1924 raise ValueError('bitspersample does not match dtype') 1925 elif not ( 1926 bitspersample > {1: 0, 2: 8, 4: 16}[datadtype.itemsize] 1927 and bitspersample <= datadtype.itemsize * 8 1928 ): 1929 raise ValueError('bitspersample out of range of dtype') 1930 elif compression: 1931 if bitspersample != datadtype.itemsize * 8: 1932 raise ValueError( 1933 'bitspersample cannot be used with compression' 1934 ) 1935 elif bitspersample != datadtype.itemsize * 8: 1936 packints = True 1937 1938 # normalize storedshape to 6D 1939 if len(storedshape) not in (5, 6): 1940 raise RuntimeError('len(storedshape) not in (5, 6)') 1941 if len(storedshape) == 5: 1942 storedshape = storedshape[:2] + (1,) + storedshape[2:] 1943 if storedshape[0] == -1: 1944 s0 = product(storedshape[1:]) 1945 s0 = 1 if s0 == 0 else product(inputshape) // s0 1946 storedshape = (s0,) + storedshape[1:] 1947 try: 1948 data = data.reshape(storedshape) 1949 except AttributeError: 1950 pass # data is None or iterator 1951 1952 if photometric == PALETTE: 1953 if ( 1954 samplesperpixel != 1 1955 or extrasamples 1956 or storedshape[1] != 1 1957 or storedshape[-1] != 1 1958 ): 1959 raise ValueError('invalid data shape for palette mode') 1960 1961 if photometric == RGB and samplesperpixel == 2: 1962 raise ValueError('not a RGB image (samplesperpixel=2)') 1963 1964 tags = [] # list of (code, ifdentry, ifdvalue, writeonce) 1965 1966 if tile: 1967 tagbytecounts = 325 # TileByteCounts 1968 tagoffsets = 324 # TileOffsets 1969 else: 1970 tagbytecounts = 279 # StripByteCounts 1971 tagoffsets = 273 # StripOffsets 1972 self._dataoffsetstag = tagoffsets 1973 1974 def pack(fmt, *val): 1975 if fmt[0] not in '<>': 1976 fmt = byteorder + fmt 1977 return struct.pack(fmt, *val) 1978 1979 def addtag(code, dtype, count, value, writeonce=False): 1980 # compute ifdentry & ifdvalue bytes from code, dtype, count, value 1981 # append (code, ifdentry, ifdvalue, writeonce) to tags list 1982 if not isinstance(code, int): 1983 code = TIFF.TAGS[code] 1984 try: 1985 datatype = dtype 1986 dataformat = TIFF.DATA_FORMATS[datatype][-1] 1987 except KeyError as exc: 1988 try: 1989 dataformat = dtype 1990 if dataformat[0] in '<>': 1991 dataformat = dataformat[1:] 1992 datatype = TIFF.DATA_DTYPES[dataformat] 1993 except (KeyError, TypeError): 1994 raise ValueError(f'unknown dtype {dtype}') from exc 1995 del dtype 1996 1997 rawcount = count 1998 if datatype == 2: 1999 # string 2000 if isinstance(value, str): 2001 # enforce 7-bit ASCII on Unicode strings 2002 try: 2003 value = value.encode('ascii') 2004 except UnicodeEncodeError as exc: 2005 raise ValueError( 2006 'TIFF strings must be 7-bit ASCII' 2007 ) from exc 2008 elif not isinstance(value, bytes): 2009 raise ValueError('TIFF strings must be 7-bit ASCII') 2010 if len(value) == 0 or value[-1] != b'\x00': 2011 value += b'\x00' 2012 count = len(value) 2013 if code == 270: 2014 self._descriptiontag = TiffTag( 2015 self, 0, 270, 2, count, None, 0 2016 ) 2017 rawcount = value.find(b'\x00\x00') 2018 if rawcount < 0: 2019 rawcount = count 2020 else: 2021 # length of string without buffer 2022 rawcount = max(offsetsize + 1, rawcount + 1) 2023 rawcount = min(count, rawcount) 2024 else: 2025 rawcount = count 2026 value = (value,) 2027 2028 elif isinstance(value, bytes): 2029 # packed binary data 2030 itemsize = struct.calcsize(dataformat) 2031 if len(value) % itemsize: 2032 raise ValueError('invalid packed binary data') 2033 count = len(value) // itemsize 2034 rawcount = count 2035 2036 if datatype in (5, 10): # rational 2037 count *= 2 2038 dataformat = dataformat[-1] 2039 2040 ifdentry = [ 2041 pack('HH', code, datatype), 2042 pack(offsetformat, rawcount), 2043 ] 2044 2045 ifdvalue = None 2046 if struct.calcsize(dataformat) * count <= offsetsize: 2047 # value(s) can be written directly 2048 if isinstance(value, bytes): 2049 ifdentry.append(pack(valueformat, value)) 2050 elif count == 1: 2051 if isinstance(value, (tuple, list, numpy.ndarray)): 2052 value = value[0] 2053 ifdentry.append(pack(valueformat, pack(dataformat, value))) 2054 else: 2055 ifdentry.append( 2056 pack(valueformat, pack(f'{count}{dataformat}', *value)) 2057 ) 2058 else: 2059 # use offset to value(s) 2060 ifdentry.append(pack(offsetformat, 0)) 2061 if isinstance(value, bytes): 2062 ifdvalue = value 2063 elif isinstance(value, numpy.ndarray): 2064 if value.size != count: 2065 raise RuntimeError('value.size != count') 2066 if value.dtype.char != dataformat: 2067 raise RuntimeError('value.dtype.char != dtype') 2068 ifdvalue = value.tobytes() 2069 elif isinstance(value, (tuple, list)): 2070 ifdvalue = pack(f'{count}{dataformat}', *value) 2071 else: 2072 ifdvalue = pack(dataformat, value) 2073 tags.append((code, b''.join(ifdentry), ifdvalue, writeonce)) 2074 2075 def rational(arg): 2076 # return numerator and denominator from float or two integers 2077 from fractions import Fraction # delayed import 2078 2079 try: 2080 f = Fraction.from_float(arg) 2081 except TypeError: 2082 f = Fraction(arg[0], arg[1]) 2083 try: 2084 numerator, denominator = f.as_integer_ratio() 2085 except AttributeError: 2086 # Python 3.7 2087 f = f.limit_denominator(4294967294) 2088 numerator, denominator = f.numerator, f.denominator 2089 if numerator > 4294967295 or denominator > 4294967295: 2090 s = 4294967295 / max(numerator, denominator) 2091 numerator = round(numerator * s) 2092 denominator = round(denominator * s) 2093 return numerator, denominator 2094 2095 if description is not None: 2096 # ImageDescription: user provided description 2097 addtag(270, 2, 0, description, writeonce=True) 2098 2099 # write shape and metadata to ImageDescription 2100 self._metadata = {} if not metadata else metadata.copy() 2101 if self._ome: 2102 if len(self._ome.images) == 0: 2103 description = '' # rewritten later at end of file 2104 else: 2105 description = None 2106 elif self._imagej: 2107 if ijmetadata is None: 2108 ijmetadata = parse_kwargs( 2109 self._metadata, 2110 'Info', 2111 'Labels', 2112 'Ranges', 2113 'LUTs', 2114 'Plot', 2115 'ROI', 2116 'Overlays', 2117 'info', 2118 'labels', 2119 'ranges', 2120 'luts', 2121 'plot', 2122 'roi', 2123 'overlays', 2124 ) 2125 else: 2126 # TODO: remove 2127 warnings.warn( 2128 "TiffWriter: the 'ijmetadata' parameter is deprecated " 2129 "since 2020.5.5. Use the 'metadata' parameter", 2130 DeprecationWarning, 2131 stacklevel=2, 2132 ) 2133 for t in imagej_metadata_tag(ijmetadata, byteorder): 2134 addtag(*t) 2135 description = imagej_description( 2136 inputshape, 2137 storedshape[-1] in (3, 4), 2138 self._colormap is not None, 2139 **self._metadata, 2140 ) 2141 description += '\x00' * 64 # add buffer for in-place update 2142 elif metadata or metadata == {}: 2143 if self._truncate: 2144 self._metadata.update(truncated=True) 2145 description = json_description(inputshape, **self._metadata) 2146 description += '\x00' * 16 # add buffer for in-place update 2147 # elif metadata is None and self._truncate: 2148 # raise ValueError('cannot truncate without writing metadata') 2149 else: 2150 description = None 2151 2152 if description is not None: 2153 description = description.encode('ascii') 2154 addtag(270, 2, 0, description, writeonce=True) 2155 del description 2156 2157 if software is None: 2158 software = 'tifffile.py' 2159 if software: 2160 addtag(305, 2, 0, software, writeonce=True) 2161 if datetime: 2162 if isinstance(datetime, str): 2163 if len(datetime) != 19 or datetime[16] != ':': 2164 raise ValueError('invalid datetime string') 2165 else: 2166 try: 2167 datetime = datetime.strftime('%Y:%m:%d %H:%M:%S') 2168 except AttributeError: 2169 datetime = self._now().strftime('%Y:%m:%d %H:%M:%S') 2170 addtag(306, 2, 0, datetime, writeonce=True) 2171 addtag(259, 3, 1, compressiontag) # Compression 2172 if compressiontag == 34887: 2173 # LERC without additional compression 2174 addtag(50674, 4, 2, (4, 0)) 2175 if predictor: 2176 addtag(317, 3, 1, predictortag) 2177 addtag(256, 4, 1, storedshape[-2]) # ImageWidth 2178 addtag(257, 4, 1, storedshape[-3]) # ImageLength 2179 if tile: 2180 addtag(322, 4, 1, tile[-1]) # TileWidth 2181 addtag(323, 4, 1, tile[-2]) # TileLength 2182 if volumetric: 2183 addtag(32997, 4, 1, storedshape[-4]) # ImageDepth 2184 if tile: 2185 addtag(32998, 4, 1, tile[0]) # TileDepth 2186 if subfiletype: 2187 addtag(254, 4, 1, subfiletype) # NewSubfileType 2188 if (subifds or self._subifds) and self._subifdslevel < 0: 2189 if self._subifds: 2190 subifds = self._subifds 2191 else: 2192 try: 2193 self._subifds = subifds = int(subifds) 2194 except TypeError: 2195 # allow TiffPage.subifds tuple 2196 self._subifds = subifds = len(subifds) 2197 addtag(330, 18 if offsetsize > 4 else 13, subifds, [0] * subifds) 2198 if not bilevel and not datadtype.kind == 'u': 2199 sampleformat = {'u': 1, 'i': 2, 'f': 3, 'c': 6}[datadtype.kind] 2200 addtag(339, 3, samplesperpixel, (sampleformat,) * samplesperpixel) 2201 if colormap is not None: 2202 addtag(320, 3, colormap.size, colormap) 2203 addtag(277, 3, 1, samplesperpixel) 2204 if bilevel: 2205 pass 2206 elif planarconfig and samplesperpixel > 1: 2207 addtag(284, 3, 1, planarconfig.value) # PlanarConfiguration 2208 addtag( # BitsPerSample 2209 258, 3, samplesperpixel, (bitspersample,) * samplesperpixel 2210 ) 2211 else: 2212 addtag(258, 3, 1, bitspersample) 2213 if extrasamples: 2214 if extrasamples_ is not None: 2215 if extrasamples != len(extrasamples_): 2216 raise ValueError('wrong number of extrasamples specified') 2217 addtag(338, 3, extrasamples, extrasamples_) 2218 elif photometric == RGB and extrasamples == 1: 2219 # Unassociated alpha channel 2220 addtag(338, 3, 1, 2) 2221 else: 2222 # Unspecified alpha channel 2223 addtag(338, 3, extrasamples, (0,) * extrasamples) 2224 2225 if jpegtables is not None: 2226 addtag(347, 7, len(jpegtables), jpegtables) 2227 2228 if ( 2229 compressiontag == 7 2230 and planarconfig == 1 2231 and photometric in (RGB, YCBCR) 2232 ): 2233 # JPEG compression with subsampling 2234 # TODO: use JPEGTables for multiple tiles or strips 2235 if subsampling is None: 2236 subsampling = (2, 2) 2237 elif subsampling not in ((1, 1), (2, 1), (2, 2), (4, 1)): 2238 raise ValueError('invalid subsampling factors') 2239 maxsampling = max(subsampling) * 8 2240 if tile and (tile[-1] % maxsampling or tile[-2] % maxsampling): 2241 raise ValueError(f'tile shape not a multiple of {maxsampling}') 2242 if extrasamples > 1: 2243 raise ValueError('JPEG subsampling requires RGB(A) images') 2244 addtag(530, 3, 2, subsampling) # YCbCrSubSampling 2245 # use PhotometricInterpretation YCBCR by default 2246 outcolorspace = enumarg( 2247 TIFF.PHOTOMETRIC, compressionargs.get('outcolorspace', 6) 2248 ) 2249 compressionargs['subsampling'] = subsampling 2250 compressionargs['colorspace'] = photometric.name 2251 compressionargs['outcolorspace'] = outcolorspace.name 2252 addtag(262, 3, 1, outcolorspace) 2253 # ReferenceBlackWhite is required for YCBCR 2254 if all(et[0] != 532 for et in extratags): 2255 addtag( 2256 532, 5, 6, (0, 1, 255, 1, 128, 1, 255, 1, 128, 1, 255, 1) 2257 ) 2258 else: 2259 if subsampling not in (None, (1, 1)): 2260 log_warning('TiffWriter: cannot apply subsampling') 2261 subsampling = None 2262 maxsampling = 1 2263 addtag(262, 3, 1, photometric.value) # PhotometricInterpretation 2264 if photometric == YCBCR: 2265 # YCbCrSubSampling and ReferenceBlackWhite 2266 addtag(530, 3, 2, (1, 1)) 2267 if all(et[0] != 532 for et in extratags): 2268 addtag( 2269 532, 2270 5, 2271 6, 2272 (0, 1, 255, 1, 128, 1, 255, 1, 128, 1, 255, 1), 2273 ) 2274 if resolution is not None: 2275 addtag(282, 5, 1, rational(resolution[0])) # XResolution 2276 addtag(283, 5, 1, rational(resolution[1])) # YResolution 2277 if len(resolution) > 2: 2278 unit = resolution[2] 2279 unit = 1 if unit is None else enumarg(TIFF.RESUNIT, unit) 2280 elif self._imagej: 2281 unit = 1 2282 else: 2283 unit = 2 2284 addtag(296, 3, 1, unit) # ResolutionUnit 2285 elif not self._imagej: 2286 addtag(282, 5, 1, (1, 1)) # XResolution 2287 addtag(283, 5, 1, (1, 1)) # YResolution 2288 addtag(296, 3, 1, 1) # ResolutionUnit 2289 2290 def bytecount_format( 2291 bytecounts, compression=compression, size=offsetsize 2292 ): 2293 # return small bytecount format 2294 if len(bytecounts) == 1: 2295 return {4: 'I', 8: 'Q'}[size] 2296 bytecount = bytecounts[0] 2297 if compression: 2298 bytecount = bytecount * 10 2299 if bytecount < 2 ** 16: 2300 return 'H' 2301 if bytecount < 2 ** 32: 2302 return 'I' 2303 if size == 4: 2304 return 'I' 2305 return 'Q' 2306 2307 # can save data array contiguous 2308 contiguous = not (compression or packints or bilevel) 2309 if tile: 2310 # one chunk per tile per plane 2311 if len(tile) == 2: 2312 tiles = ( 2313 (storedshape[3] + tile[0] - 1) // tile[0], 2314 (storedshape[4] + tile[1] - 1) // tile[1], 2315 ) 2316 contiguous = ( 2317 contiguous 2318 and storedshape[3] == tile[0] 2319 and storedshape[4] == tile[1] 2320 ) 2321 else: 2322 tiles = ( 2323 (storedshape[2] + tile[0] - 1) // tile[0], 2324 (storedshape[3] + tile[1] - 1) // tile[1], 2325 (storedshape[4] + tile[2] - 1) // tile[2], 2326 ) 2327 contiguous = ( 2328 contiguous 2329 and storedshape[2] == tile[0] 2330 and storedshape[3] == tile[1] 2331 and storedshape[4] == tile[2] 2332 ) 2333 numtiles = product(tiles) * storedshape[1] 2334 databytecounts = [ 2335 product(tile) * storedshape[-1] * datadtype.itemsize 2336 ] * numtiles 2337 bytecountformat = bytecount_format(databytecounts) 2338 addtag(tagbytecounts, bytecountformat, numtiles, databytecounts) 2339 addtag(tagoffsets, offsetformat, numtiles, [0] * numtiles) 2340 bytecountformat = bytecountformat * numtiles 2341 if contiguous or dataiter is not None: 2342 pass 2343 else: 2344 dataiter = iter_tiles(data, tile, tiles) 2345 2346 elif contiguous and rowsperstrip is None: 2347 count = storedshape[1] * storedshape[2] 2348 databytecounts = [ 2349 product(storedshape[3:]) * datadtype.itemsize 2350 ] * count 2351 bytecountformat = bytecount_format(databytecounts) 2352 addtag(tagbytecounts, bytecountformat, count, databytecounts) 2353 addtag(tagoffsets, offsetformat, count, [0] * count) 2354 addtag(278, 4, 1, storedshape[-3]) # RowsPerStrip 2355 bytecountformat = bytecountformat * storedshape[1] 2356 if contiguous or dataiter is not None: 2357 pass 2358 else: 2359 dataiter = iter_images(data) 2360 2361 else: 2362 # use rowsperstrip 2363 rowsize = product(storedshape[-2:]) * datadtype.itemsize 2364 if rowsperstrip is None: 2365 # compress ~64 KB chunks by default 2366 # TIFF-EP requires <= 64 KB 2367 if compression: 2368 rowsperstrip = 65536 // rowsize 2369 else: 2370 rowsperstrip = storedshape[-3] 2371 if rowsperstrip < 1: 2372 rowsperstrip = maxsampling 2373 elif rowsperstrip > storedshape[-3]: 2374 rowsperstrip = storedshape[-3] 2375 elif subsampling and rowsperstrip % maxsampling: 2376 rowsperstrip = ( 2377 math.ceil(rowsperstrip / maxsampling) * maxsampling 2378 ) 2379 addtag(278, 4, 1, rowsperstrip) # RowsPerStrip 2380 2381 numstrips1 = (storedshape[-3] + rowsperstrip - 1) // rowsperstrip 2382 numstrips = numstrips1 * storedshape[1] * storedshape[2] 2383 # TODO: save bilevel data with rowsperstrip 2384 stripsize = rowsperstrip * rowsize 2385 databytecounts = [stripsize] * numstrips 2386 stripsize -= rowsize * ( 2387 numstrips1 * rowsperstrip - storedshape[-3] 2388 ) 2389 for i in range(numstrips1 - 1, numstrips, numstrips1): 2390 databytecounts[i] = stripsize 2391 bytecountformat = bytecount_format(databytecounts) 2392 addtag(tagbytecounts, bytecountformat, numstrips, databytecounts) 2393 addtag(tagoffsets, offsetformat, numstrips, [0] * numstrips) 2394 bytecountformat = bytecountformat * numstrips 2395 2396 if contiguous or dataiter is not None: 2397 pass 2398 else: 2399 dataiter = iter_images(data) 2400 2401 if data is None and not contiguous: 2402 raise ValueError('cannot write non-contiguous empty file') 2403 2404 # add extra tags from user 2405 for t in extratags: 2406 addtag(*t) 2407 2408 # TODO: check TIFFReadDirectoryCheckOrder warning in files containing 2409 # multiple tags of same code 2410 # the entries in an IFD must be sorted in ascending order by tag code 2411 tags = sorted(tags, key=lambda x: x[0]) 2412 2413 # define compress function 2414 if bilevel: 2415 if compressiontag == 1: 2416 2417 def compress(data, level=None): 2418 return numpy.packbits(data, axis=-2).tobytes() 2419 2420 elif compressiontag in (5, 32773): 2421 # LZW, PackBits 2422 def compress( 2423 data, 2424 compressor=TIFF.COMPRESSORS[compressiontag], 2425 kwargs=compressionargs, 2426 ): 2427 data = numpy.packbits(data, axis=-2).tobytes() 2428 return compressor(data, **kwargs) 2429 2430 else: 2431 raise ValueError('cannot compress bilevel image') 2432 2433 elif compression: 2434 compressor = TIFF.COMPRESSORS[compressiontag] 2435 2436 if compressiontag == 32773: # PackBits 2437 compressionargs['axis'] = -2 2438 2439 if subsampling: 2440 # JPEG with subsampling 2441 def compress( 2442 data, compressor=compressor, kwargs=compressionargs 2443 ): 2444 return compressor(data, **kwargs) 2445 2446 elif predictor: 2447 2448 def compress( 2449 data, 2450 predictor=predictor, 2451 compressor=compressor, 2452 kwargs=compressionargs, 2453 ): 2454 data = predictor(data, axis=-2) 2455 return compressor(data, **kwargs) 2456 2457 elif compressionargs: 2458 2459 def compress( 2460 data, compressor=compressor, kwargs=compressionargs 2461 ): 2462 return compressor(data, **kwargs) 2463 2464 else: 2465 compress = compressor 2466 2467 elif packints: 2468 2469 def compress(data, bps=bitspersample): 2470 return packints_encode(data, bps, axis=-2) 2471 2472 else: 2473 compress = False 2474 2475 del compression 2476 2477 fhpos = fh.tell() 2478 if ( 2479 not (offsetsize > 4 or self._imagej or compress) 2480 and fhpos + datasize > 2 ** 32 - 1 2481 ): 2482 raise ValueError('data too large for standard TIFF file') 2483 2484 # if not compressed or multi-tiled, write the first IFD and then 2485 # all data contiguously; else, write all IFDs and data interleaved 2486 for pageindex in range(1 if contiguous else storedshape[0]): 2487 2488 ifdpos = fhpos 2489 if ifdpos % 2: 2490 # location of IFD must begin on a word boundary 2491 fh.write(b'\x00') 2492 ifdpos += 1 2493 2494 if self._subifdslevel < 0: 2495 # update pointer at ifdoffset 2496 fh.seek(self._ifdoffset) 2497 fh.write(pack(offsetformat, ifdpos)) 2498 2499 fh.seek(ifdpos) 2500 2501 # create IFD in memory 2502 if pageindex < 2: 2503 subifdsoffsets = None 2504 ifd = io.BytesIO() 2505 ifd.write(pack(tagnoformat, len(tags))) 2506 tagoffset = ifd.tell() 2507 ifd.write(b''.join(t[1] for t in tags)) 2508 ifdoffset = ifd.tell() 2509 ifd.write(pack(offsetformat, 0)) # offset to next IFD 2510 # write tag values and patch offsets in ifdentries 2511 for tagindex, tag in enumerate(tags): 2512 offset = tagoffset + tagindex * tagsize + 4 + offsetsize 2513 code = tag[0] 2514 value = tag[2] 2515 if value: 2516 pos = ifd.tell() 2517 if pos % 2: 2518 # tag value is expected to begin on word boundary 2519 ifd.write(b'\x00') 2520 pos += 1 2521 ifd.seek(offset) 2522 ifd.write(pack(offsetformat, ifdpos + pos)) 2523 ifd.seek(pos) 2524 ifd.write(value) 2525 if code == tagoffsets: 2526 dataoffsetsoffset = offset, pos 2527 elif code == tagbytecounts: 2528 databytecountsoffset = offset, pos 2529 elif code == 270: 2530 self._descriptiontag.offset = ( 2531 ifdpos + tagoffset + tagindex * tagsize 2532 ) 2533 self._descriptiontag.valueoffset = ifdpos + pos 2534 elif code == 330: 2535 subifdsoffsets = offset, pos 2536 elif code == tagoffsets: 2537 dataoffsetsoffset = offset, None 2538 elif code == tagbytecounts: 2539 databytecountsoffset = offset, None 2540 elif code == 270: 2541 self._descriptiontag.offset = ( 2542 ifdpos + tagoffset + tagindex * tagsize 2543 ) 2544 self._descriptiontag.valueoffset = ( 2545 self._descriptiontag.offset + offsetsize + 4 2546 ) 2547 elif code == 330: 2548 subifdsoffsets = offset, None 2549 ifdsize = ifd.tell() 2550 if ifdsize % 2: 2551 ifd.write(b'\x00') 2552 ifdsize += 1 2553 2554 # write IFD later when strip/tile bytecounts and offsets are known 2555 fh.seek(ifdsize, os.SEEK_CUR) 2556 2557 # write image data 2558 dataoffset = fh.tell() 2559 if align is None: 2560 align = 16 2561 skip = (align - (dataoffset % align)) % align 2562 fh.seek(skip, os.SEEK_CUR) 2563 dataoffset += skip 2564 if contiguous: 2565 if data is None: 2566 fh.write_empty(datasize) 2567 elif dataiter is not None: 2568 for pagedata in dataiter: 2569 if pagedata.dtype != datadtype: 2570 raise ValueError( 2571 'dtype of iterable does not match dtype' 2572 ) 2573 fh.write_array(pagedata.reshape(storedshape[1:])) 2574 else: 2575 fh.write_array(data) 2576 elif tile: 2577 if storedshape[-1] == 1: 2578 tileshape = tile 2579 else: 2580 tileshape = tile + (storedshape[-1],) 2581 tilesize = product(tileshape) * datadtype.itemsize 2582 if data is None: 2583 fh.write_empty(numtiles * tilesize) 2584 elif compress: 2585 isbytes = True 2586 for tileindex in range(storedshape[1] * product(tiles)): 2587 chunk = next(dataiter) 2588 if chunk is None: 2589 databytecounts[tileindex] = 0 2590 continue 2591 if isbytes and isinstance(chunk, bytes): 2592 # pre-compressed 2593 pass 2594 else: 2595 if chunk.nbytes != tilesize: 2596 chunk = pad_tile(chunk, tileshape, datadtype) 2597 isbytes = False 2598 chunk = compress(chunk) 2599 fh.write(chunk) 2600 databytecounts[tileindex] = len(chunk) 2601 else: 2602 for tileindex in range(storedshape[1] * product(tiles)): 2603 chunk = next(dataiter) 2604 if chunk is None: 2605 # databytecounts[tileindex] = 0 # not contiguous 2606 fh.write_empty(tilesize) 2607 continue 2608 if chunk.nbytes != tilesize: 2609 chunk = pad_tile(chunk, tileshape, datadtype) 2610 fh.write_array(chunk) 2611 elif compress: 2612 # write one strip per rowsperstrip 2613 numstrips = ( 2614 storedshape[-3] + rowsperstrip - 1 2615 ) // rowsperstrip 2616 stripindex = 0 2617 pagedata = next(dataiter).reshape(storedshape[1:]) 2618 if pagedata.dtype != datadtype: 2619 raise ValueError('dtype of iterable does not match dtype') 2620 for plane in pagedata: 2621 for depth in plane: 2622 for i in range(numstrips): 2623 strip = depth[ 2624 i * rowsperstrip : (i + 1) * rowsperstrip 2625 ] 2626 strip = compress(strip) 2627 fh.write(strip) 2628 databytecounts[stripindex] = len(strip) 2629 stripindex += 1 2630 else: 2631 pagedata = next(dataiter).reshape(storedshape[1:]) 2632 if pagedata.dtype != datadtype: 2633 raise ValueError('dtype of iterable does not match dtype') 2634 fh.write_array(pagedata) 2635 2636 # update strip/tile offsets 2637 offset, pos = dataoffsetsoffset 2638 ifd.seek(offset) 2639 if pos: 2640 ifd.write(pack(offsetformat, ifdpos + pos)) 2641 ifd.seek(pos) 2642 offset = dataoffset 2643 for size in databytecounts: 2644 ifd.write(pack(offsetformat, offset)) 2645 offset += size 2646 else: 2647 ifd.write(pack(offsetformat, dataoffset)) 2648 2649 if compress: 2650 # update strip/tile bytecounts 2651 offset, pos = databytecountsoffset 2652 ifd.seek(offset) 2653 if pos: 2654 ifd.write(pack(offsetformat, ifdpos + pos)) 2655 ifd.seek(pos) 2656 ifd.write(pack(bytecountformat, *databytecounts)) 2657 2658 if subifdsoffsets is not None: 2659 # update and save pointer to SubIFDs tag values if necessary 2660 offset, pos = subifdsoffsets 2661 if pos is not None: 2662 ifd.seek(offset) 2663 ifd.write(pack(offsetformat, ifdpos + pos)) 2664 self._subifdsoffsets.append(ifdpos + pos) 2665 else: 2666 self._subifdsoffsets.append(ifdpos + offset) 2667 2668 fhpos = fh.tell() 2669 fh.seek(ifdpos) 2670 fh.write(ifd.getbuffer()) 2671 fh.flush() 2672 2673 if self._subifdslevel < 0: 2674 self._ifdoffset = ifdpos + ifdoffset 2675 else: 2676 # update SubIFDs tag values 2677 fh.seek( 2678 self._subifdsoffsets[self._ifdindex] 2679 + self._subifdslevel * offsetsize 2680 ) 2681 fh.write(pack(offsetformat, ifdpos)) 2682 2683 # update SubIFD chain offsets 2684 if self._subifdslevel == 0: 2685 self._nextifdoffsets.append(ifdpos + ifdoffset) 2686 else: 2687 fh.seek(self._nextifdoffsets[self._ifdindex]) 2688 fh.write(pack(offsetformat, ifdpos)) 2689 self._nextifdoffsets[self._ifdindex] = ifdpos + ifdoffset 2690 self._ifdindex += 1 2691 self._ifdindex %= len(self._subifdsoffsets) 2692 2693 fh.seek(fhpos) 2694 2695 # remove tags that should be written only once 2696 if pageindex == 0: 2697 tags = [tag for tag in tags if not tag[-1]] 2698 2699 self._storedshape = storedshape 2700 self._datashape = (1,) + inputshape 2701 self._datadtype = datadtype 2702 self._dataoffset = dataoffset 2703 self._databytecounts = databytecounts 2704 2705 if contiguous: 2706 # write remaining IFDs/tags later 2707 self._tags = tags 2708 # return offset and size of image data 2709 if returnoffset: 2710 return dataoffset, sum(databytecounts) 2711 return None 2712 2713 def overwrite_description(self, description): 2714 """Overwrite the value of the last ImageDescription tag. 2715 2716 Can be used to write OME-XML after writing the image data. 2717 Ends a contiguous series. 2718 2719 """ 2720 if self._descriptiontag is None: 2721 raise ValueError('no ImageDescription tag found') 2722 self._write_remaining_pages() 2723 self._descriptiontag.overwrite(description, erase=False) 2724 self._descriptiontag = None 2725 2726 def _write_remaining_pages(self): 2727 """Write outstanding IFDs and tags to file.""" 2728 if not self._tags or self._truncate or self._datashape is None: 2729 return 2730 2731 pageno = self._storedshape[0] * self._datashape[0] - 1 2732 if pageno < 1: 2733 self._tags = None 2734 self._dataoffset = None 2735 self._databytecounts = None 2736 return 2737 2738 fh = self._fh 2739 fhpos = fh.tell() 2740 if fhpos % 2: 2741 fh.write(b'\x00') 2742 fhpos += 1 2743 2744 pack = struct.pack 2745 offsetformat = self.tiff.offsetformat 2746 offsetsize = self.tiff.offsetsize 2747 tagnoformat = self.tiff.tagnoformat 2748 tagsize = self.tiff.tagsize 2749 dataoffset = self._dataoffset 2750 pagedatasize = sum(self._databytecounts) 2751 subifdsoffsets = None 2752 2753 # construct template IFD in memory 2754 # must patch offsets to next IFD and data before writing to file 2755 ifd = io.BytesIO() 2756 ifd.write(pack(tagnoformat, len(self._tags))) 2757 tagoffset = ifd.tell() 2758 ifd.write(b''.join(t[1] for t in self._tags)) 2759 ifdoffset = ifd.tell() 2760 ifd.write(pack(offsetformat, 0)) # offset to next IFD 2761 # tag values 2762 for tagindex, tag in enumerate(self._tags): 2763 offset = tagoffset + tagindex * tagsize + offsetsize + 4 2764 code = tag[0] 2765 value = tag[2] 2766 if value: 2767 pos = ifd.tell() 2768 if pos % 2: 2769 # tag value is expected to begin on word boundary 2770 ifd.write(b'\x00') 2771 pos += 1 2772 ifd.seek(offset) 2773 try: 2774 ifd.write(pack(offsetformat, fhpos + pos)) 2775 except Exception: # struct.error 2776 if self._imagej: 2777 warnings.warn( 2778 'TiffWriter: truncating ImageJ file', UserWarning 2779 ) 2780 self._truncate = True 2781 return 2782 raise ValueError('data too large for non-BigTIFF file') 2783 ifd.seek(pos) 2784 ifd.write(value) 2785 if code == self._dataoffsetstag: 2786 # save strip/tile offsets for later updates 2787 dataoffsetsoffset = offset, pos 2788 elif code == 330: 2789 # save subifds offsets for later updates 2790 subifdsoffsets = offset, pos 2791 elif code == self._dataoffsetstag: 2792 dataoffsetsoffset = offset, None 2793 elif code == 330: 2794 subifdsoffsets = offset, None 2795 2796 ifdsize = ifd.tell() 2797 if ifdsize % 2: 2798 ifd.write(b'\x00') 2799 ifdsize += 1 2800 2801 # check if all IFDs fit in file 2802 if offsetsize < 8 and fhpos + ifdsize * pageno > 2 ** 32 - 32: 2803 if self._imagej: 2804 warnings.warn( 2805 'TiffWriter: truncating ImageJ file', UserWarning 2806 ) 2807 self._truncate = True 2808 return 2809 raise ValueError('data too large for non-BigTIFF file') 2810 2811 # assemble IFD chain in memory from IFD template 2812 ifds = io.BytesIO(bytes(ifdsize * pageno)) 2813 ifdpos = fhpos 2814 for _ in range(pageno): 2815 # update strip/tile offsets in IFD 2816 dataoffset += pagedatasize # offset to image data 2817 offset, pos = dataoffsetsoffset 2818 ifd.seek(offset) 2819 if pos is not None: 2820 ifd.write(pack(offsetformat, ifdpos + pos)) 2821 ifd.seek(pos) 2822 offset = dataoffset 2823 for size in self._databytecounts: 2824 ifd.write(pack(offsetformat, offset)) 2825 offset += size 2826 else: 2827 ifd.write(pack(offsetformat, dataoffset)) 2828 2829 if subifdsoffsets is not None: 2830 offset, pos = subifdsoffsets 2831 self._subifdsoffsets.append( 2832 ifdpos + (pos if pos is not None else offset) 2833 ) 2834 2835 if self._subifdslevel < 0: 2836 if subifdsoffsets is not None: 2837 # update pointer to SubIFDs tag values if necessary 2838 offset, pos = subifdsoffsets 2839 if pos is not None: 2840 ifd.seek(offset) 2841 ifd.write(pack(offsetformat, ifdpos + pos)) 2842 2843 # update pointer at ifdoffset to point to next IFD in file 2844 ifdpos += ifdsize 2845 ifd.seek(ifdoffset) 2846 ifd.write(pack(offsetformat, ifdpos)) 2847 2848 else: 2849 # update SubIFDs tag values in file 2850 fh.seek( 2851 self._subifdsoffsets[self._ifdindex] 2852 + self._subifdslevel * offsetsize 2853 ) 2854 fh.write(pack(offsetformat, ifdpos)) 2855 2856 # update SubIFD chain 2857 if self._subifdslevel == 0: 2858 self._nextifdoffsets.append(ifdpos + ifdoffset) 2859 else: 2860 fh.seek(self._nextifdoffsets[self._ifdindex]) 2861 fh.write(pack(offsetformat, ifdpos)) 2862 self._nextifdoffsets[self._ifdindex] = ifdpos + ifdoffset 2863 self._ifdindex += 1 2864 self._ifdindex %= len(self._subifdsoffsets) 2865 ifdpos += ifdsize 2866 2867 # write IFD entry 2868 ifds.write(ifd.getbuffer()) 2869 2870 # terminate IFD chain 2871 ifdoffset += ifdsize * (pageno - 1) 2872 ifds.seek(ifdoffset) 2873 ifds.write(pack(offsetformat, 0)) 2874 # write IFD chain to file 2875 fh.seek(fhpos) 2876 fh.write(ifds.getbuffer()) 2877 2878 if self._subifdslevel < 0: 2879 # update file to point to new IFD chain 2880 pos = fh.tell() 2881 fh.seek(self._ifdoffset) 2882 fh.write(pack(offsetformat, fhpos)) 2883 fh.flush() 2884 fh.seek(pos) 2885 self._ifdoffset = fhpos + ifdoffset 2886 2887 self._tags = None 2888 self._dataoffset = None 2889 self._databytecounts = None 2890 # do not reset _storedshape, _datashape, _datadtype 2891 2892 def _write_image_description(self): 2893 """Write metadata to ImageDescription tag.""" 2894 if self._datashape is None or self._descriptiontag is None: 2895 self._descriptiontag = None 2896 return 2897 2898 if self._ome: 2899 if self._subifdslevel < 0: 2900 self._ome.addimage( 2901 self._datadtype, 2902 self._datashape[0 if self._datashape[0] != 1 else 1 :], 2903 self._storedshape, 2904 **self._metadata, 2905 ) 2906 description = self._ome.tostring(declaration=True).encode() 2907 elif self._datashape[0] == 1: 2908 # description already up-to-date 2909 self._descriptiontag = None 2910 return 2911 # elif self._subifdslevel >= 0: 2912 # # don't write metadata to SubIFDs 2913 # return 2914 elif self._imagej: 2915 colormapped = self._colormap is not None 2916 isrgb = self._storedshape[-1] in (3, 4) 2917 description = imagej_description( 2918 self._datashape, isrgb, colormapped, **self._metadata 2919 ) 2920 else: 2921 description = json_description(self._datashape, **self._metadata) 2922 2923 self._descriptiontag.overwrite(description, erase=False) 2924 self._descriptiontag = None 2925 2926 def _now(self): 2927 """Return current date and time.""" 2928 return datetime.datetime.now() 2929 2930 def close(self): 2931 """Write remaining pages and close file handle.""" 2932 if not self._truncate: 2933 self._write_remaining_pages() 2934 self._write_image_description() 2935 self._fh.close() 2936 2937 def __enter__(self): 2938 return self 2939 2940 def __exit__(self, exc_type, exc_value, traceback): 2941 self.close() 2942 2943 2944class TiffFile: 2945 """Read image and metadata from TIFF file. 2946 2947 TiffFile instances must be closed using the 'close' method, which is 2948 automatically called when using the 'with' context manager. 2949 2950 TiffFile instances are not thread-safe. 2951 2952 Attributes 2953 ---------- 2954 pages : TiffPages 2955 Sequence of TIFF pages in file. 2956 series : list of TiffPageSeries 2957 Sequences of closely related TIFF pages. These are computed 2958 from OME, LSM, ImageJ, etc. metadata or based on similarity 2959 of page properties such as shape, dtype, and compression. 2960 is_flag : bool 2961 If True, file is of a certain format. 2962 Flags are: bigtiff, uniform, shaped, ome, imagej, stk, lsm, fluoview, 2963 nih, vista, micromanager, metaseries, mdgel, mediacy, tvips, fei, 2964 sem, scn, svs, scanimage, andor, epics, ndpi, pilatus, qpi. 2965 2966 All attributes are read-only. 2967 2968 """ 2969 2970 def __init__( 2971 self, 2972 arg, 2973 mode=None, 2974 name=None, 2975 offset=None, 2976 size=None, 2977 _multifile=True, 2978 _useframes=None, 2979 _parent=None, 2980 **kwargs, 2981 ): 2982 """Initialize instance from file. 2983 2984 Parameters 2985 ---------- 2986 arg : str or open file 2987 Name of file or open file object. 2988 The file objects are closed in TiffFile.close(). 2989 mode : str 2990 File open mode in case 'arg' is a file name. Must be 'rb' or 'r+b'. 2991 Default is 'rb'. 2992 name : str 2993 Optional name of file in case 'arg' is a file handle. 2994 offset : int 2995 Optional start position of embedded file. By default, this is 2996 the current file position. 2997 size : int 2998 Optional size of embedded file. By default, this is the number 2999 of bytes from the 'offset' to the end of the file. 3000 kwargs : bool 3001 'is_ome': If False, disable processing of OME-XML metadata. 3002 3003 """ 3004 if kwargs: 3005 # TODO: remove; formally deprecated in 2020.10.1 3006 for key in ('fastij', 'movie', 'multifile', 'multifile_close'): 3007 if key in kwargs: 3008 del kwargs[key] 3009 warnings.warn( 3010 f'TiffFile: the {key!r} argument is ignored', 3011 DeprecationWarning, 3012 stacklevel=2, 3013 ) 3014 if 'pages' in kwargs: 3015 raise TypeError( 3016 "the TiffFile 'pages' parameter is no longer supported." 3017 "\n\nUse TiffFile.asarray(key=[...]) to read image data " 3018 "from specific pages.\n" 3019 ) 3020 3021 for key, value in kwargs.items(): 3022 if key[:3] == 'is_' and key[3:] in TIFF.FILE_FLAGS: 3023 if value is not None: 3024 setattr(self, key, bool(value)) 3025 else: 3026 raise TypeError(f'unexpected keyword argument: {key}') 3027 3028 if mode not in (None, 'rb', 'r+b'): 3029 raise ValueError(f'invalid mode {mode!r}') 3030 3031 fh = FileHandle(arg, mode=mode, name=name, offset=offset, size=size) 3032 self._fh = fh 3033 self._multifile = bool(_multifile) 3034 self._files = {fh.name: self} # cache of TiffFile instances 3035 self._decoders = {} # cache of TiffPage.decode functions 3036 self._parent = self if _parent is None else _parent # OME master file 3037 3038 try: 3039 fh.seek(0) 3040 header = fh.read(4) 3041 try: 3042 byteorder = {b'II': '<', b'MM': '>', b'EP': '<'}[header[:2]] 3043 except KeyError: 3044 raise TiffFileError('not a TIFF file') 3045 3046 version = struct.unpack(byteorder + 'H', header[2:4])[0] 3047 if version == 43: 3048 # BigTiff 3049 offsetsize, zero = struct.unpack(byteorder + 'HH', fh.read(4)) 3050 if zero != 0 or offsetsize != 8: 3051 raise TiffFileError('invalid BigTIFF file') 3052 if byteorder == '>': 3053 self.tiff = TIFF.BIG_BE 3054 else: 3055 self.tiff = TIFF.BIG_LE 3056 elif version == 42: 3057 # Classic TIFF 3058 if byteorder == '>': 3059 self.tiff = TIFF.CLASSIC_BE 3060 elif kwargs.get('is_ndpi', False) or fh.name.endswith('ndpi'): 3061 # NDPI uses 64 bit IFD offsets 3062 self.tiff = TIFF.NDPI_LE 3063 else: 3064 self.tiff = TIFF.CLASSIC_LE 3065 elif version == 0x55 or version == 0x4F52 or version == 0x5352: 3066 # Panasonic or Olympus RAW 3067 log_warning(f'RAW format 0x{version:04X} not supported') 3068 if byteorder == '>': 3069 self.tiff = TIFF.CLASSIC_BE 3070 else: 3071 self.tiff = TIFF.CLASSIC_LE 3072 else: 3073 raise TiffFileError(f'invalid TIFF version {version}') 3074 3075 # file handle is at offset to offset to first page 3076 self.pages = TiffPages(self) 3077 3078 if self.is_lsm and ( 3079 self.filehandle.size >= 2 ** 32 3080 or self.pages[0].compression != 1 3081 or self.pages[1].compression != 1 3082 ): 3083 self._lsm_load_pages() 3084 3085 elif self.is_scanimage and not self.is_bigtiff: 3086 # ScanImage <= 2015 3087 try: 3088 self.pages._load_virtual_frames() 3089 except Exception as exc: 3090 log_warning( 3091 f'load_virtual_frames: {exc.__class__.__name__}: {exc}' 3092 ) 3093 3094 elif self.is_philips: 3095 try: 3096 self._philips_load_pages() 3097 except Exception as exc: 3098 log_warning( 3099 f'philips_load_pages: {exc.__class__.__name__}: {exc}' 3100 ) 3101 3102 elif self.is_ndpi: 3103 try: 3104 self._ndpi_load_pages() 3105 except Exception as exc: 3106 log_warning( 3107 f'_ndpi_load_pages: {exc.__class__.__name__}: {exc}' 3108 ) 3109 3110 elif _useframes: 3111 self.pages.useframes = True 3112 3113 except Exception: 3114 fh.close() 3115 raise 3116 3117 @property 3118 def byteorder(self): 3119 return self.tiff.byteorder 3120 3121 @property 3122 def filehandle(self): 3123 """Return file handle.""" 3124 return self._fh 3125 3126 @property 3127 def filename(self): 3128 """Return name of file handle.""" 3129 return self._fh.name 3130 3131 @lazyattr 3132 def fstat(self): 3133 """Return status of file handle as stat_result object.""" 3134 try: 3135 return os.fstat(self._fh.fileno()) 3136 except Exception: # io.UnsupportedOperation 3137 return None 3138 3139 def close(self): 3140 """Close open file handle(s).""" 3141 for tif in self._files.values(): 3142 tif.filehandle.close() 3143 3144 def asarray( 3145 self, 3146 key=None, 3147 series=None, 3148 level=None, 3149 squeeze=None, 3150 out=None, 3151 maxworkers=None, 3152 ): 3153 """Return image data from selected TIFF page(s) as numpy array. 3154 3155 By default, the data from the first series is returned. 3156 3157 Parameters 3158 ---------- 3159 key : int, slice, or sequence of indices 3160 Defines which pages to return as array. 3161 If None (default), data from a series (default 0) is returned. 3162 If not None, data from the specified pages in the whole file 3163 (if 'series' is None) or a specified series are returned as a 3164 stacked array. 3165 Requesting an array from multiple pages that are not compatible 3166 wrt. shape, dtype, compression etc. is undefined, i.e. may crash 3167 or return incorrect values. 3168 series : int or TiffPageSeries 3169 Defines which series of pages to return as array. 3170 level : int 3171 Defines which pyramid level of a series to return as array. 3172 squeeze : bool 3173 If True, all length-1 dimensions (except X and Y) are squeezed 3174 out from the array. 3175 If False, single pages are returned as 5D array (TiffPage.shaped). 3176 For series, the shape of the returned array also includes singlet 3177 dimensions specified in some file formats. E.g. ImageJ series, and 3178 most commonly also OME series, are returned in TZCYXS order. 3179 If None (default), all but "shaped" series are squeezed. 3180 out : numpy.ndarray, str, or file-like object 3181 Buffer where image data are saved. 3182 If None (default), a new array is created. 3183 If numpy.ndarray, a writable array of compatible dtype and shape. 3184 If 'memmap', directly memory-map the image data in the TIFF file 3185 if possible; else create a memory-mapped array in a temporary file. 3186 If str or open file, the file name or file object used to 3187 create a memory-map to an array stored in a binary file on disk. 3188 maxworkers : int or None 3189 Maximum number of threads to concurrently get data from multiple 3190 pages or compressed segments. 3191 If None (default), up to half the CPU cores are used. 3192 If 1, multi-threading is disabled. 3193 Reading data from file is limited to a single thread. 3194 Using multiple threads can significantly speed up this function 3195 if the bottleneck is decoding compressed data, e.g. in case of 3196 large LZW compressed LSM files or JPEG compressed tiled slides. 3197 If the bottleneck is I/O or pure Python code, using multiple 3198 threads might be detrimental. 3199 3200 Returns 3201 ------- 3202 numpy.ndarray 3203 Image data from the specified pages. 3204 See TiffPage.asarray for operations that are applied (or not) 3205 to the raw data stored in the file. 3206 3207 """ 3208 if not self.pages: 3209 return numpy.array([]) 3210 if key is None and series is None: 3211 series = 0 3212 if series is None: 3213 pages = self.pages 3214 else: 3215 try: 3216 series = self.series[series] 3217 except (KeyError, TypeError): 3218 pass 3219 if level is not None: 3220 series = series.levels[level] 3221 pages = series.pages 3222 3223 if key is None: 3224 pass 3225 elif series is None: 3226 pages = self.pages._getlist(key) 3227 elif isinstance(key, (int, numpy.integer)): 3228 pages = [pages[key]] 3229 elif isinstance(key, slice): 3230 pages = pages[key] 3231 elif isinstance(key, Iterable): 3232 pages = [pages[k] for k in key] 3233 else: 3234 raise TypeError('key must be an int, slice, or sequence') 3235 3236 if not pages: 3237 raise ValueError('no pages selected') 3238 3239 if key is None and series and series.offset: 3240 typecode = self.byteorder + series.dtype.char 3241 if ( 3242 pages[0].is_memmappable 3243 and isinstance(out, str) 3244 and out == 'memmap' 3245 ): 3246 # direct mapping 3247 shape = series.get_shape(squeeze) 3248 result = self.filehandle.memmap_array( 3249 typecode, shape, series.offset 3250 ) 3251 else: 3252 # read into output 3253 shape = series.get_shape(squeeze) 3254 if out is not None: 3255 out = create_output(out, shape, series.dtype) 3256 self.filehandle.seek(series.offset) 3257 result = self.filehandle.read_array( 3258 typecode, product(shape), out=out 3259 ) 3260 elif len(pages) == 1: 3261 result = pages[0].asarray(out=out, maxworkers=maxworkers) 3262 else: 3263 result = stack_pages(pages, out=out, maxworkers=maxworkers) 3264 3265 if result is None: 3266 return None 3267 3268 if key is None: 3269 shape = series.get_shape(squeeze) 3270 try: 3271 result.shape = shape 3272 except ValueError: 3273 try: 3274 log_warning( 3275 'TiffFile.asarray: ' 3276 f'failed to reshape {result.shape} to {shape}' 3277 ) 3278 # try series of expected shapes 3279 result.shape = (-1,) + shape 3280 except ValueError: 3281 # revert to generic shape 3282 result.shape = (-1,) + pages[0].shape 3283 elif len(pages) == 1: 3284 if squeeze is None: 3285 squeeze = True 3286 result.shape = pages[0].shape if squeeze else pages[0].shaped 3287 else: 3288 if squeeze is None: 3289 squeeze = True 3290 result.shape = (-1,) + ( 3291 pages[0].shape if squeeze else pages[0].shaped 3292 ) 3293 return result 3294 3295 def aszarr(self, key=None, series=None, level=None, **kwargs): 3296 """Return image data from selected TIFF page(s) as zarr storage.""" 3297 if not self.pages: 3298 raise NotImplementedError('empty zarr arrays not supported') 3299 if key is None and series is None: 3300 return self.series[0].aszarr(level=level, **kwargs) 3301 if series is None: 3302 pages = self.pages 3303 else: 3304 try: 3305 series = self.series[series] 3306 except (KeyError, TypeError): 3307 pass 3308 if key is None: 3309 return series.aszarr(level=level, **kwargs) 3310 pages = series.pages 3311 if isinstance(key, (int, numpy.integer)): 3312 return pages[key].aszarr(**kwargs) 3313 raise TypeError('key must be an integer index') 3314 3315 @lazyattr 3316 def series(self): 3317 """Return related pages as TiffPageSeries. 3318 3319 Side effect: after calling this function, TiffFile.pages might contain 3320 TiffPage and TiffFrame instances. 3321 3322 """ 3323 if not self.pages: 3324 return [] 3325 3326 useframes = self.pages.useframes 3327 keyframe = self.pages.keyframe.index 3328 series = [] 3329 for name in ( 3330 'shaped', 3331 'lsm', 3332 'ome', 3333 'imagej', 3334 'fluoview', 3335 'stk', 3336 'sis', 3337 'svs', 3338 'scn', 3339 'qpi', 3340 'ndpi', 3341 'bif', 3342 'scanimage', 3343 'mdgel', # adds second page to cache 3344 'uniform', 3345 ): 3346 if getattr(self, 'is_' + name, False): 3347 series = getattr(self, '_series_' + name)() 3348 if not series and name == 'ome' and self.is_imagej: 3349 # try ImageJ series if OME series fails. 3350 # clear pages cache since _series_ome() might leave some 3351 # frames without keyframe 3352 self.pages._clear() 3353 continue 3354 break 3355 self.pages.useframes = useframes 3356 self.pages.keyframe = keyframe 3357 if not series: 3358 series = self._series_generic() 3359 3360 # remove empty series, e.g. in MD Gel files 3361 # series = [s for s in series if product(s.shape) > 0] 3362 3363 for i, s in enumerate(series): 3364 s.index = i 3365 return series 3366 3367 def _series_uniform(self): 3368 """Return all images in file as single series.""" 3369 page = self.pages[0] 3370 validate = not (page.is_scanimage or page.is_nih) 3371 pages = self.pages._getlist(validate=validate) 3372 shape = (len(pages),) + page.shape 3373 axes = 'I' + page.axes 3374 dtype = page.dtype 3375 if page.is_nih: 3376 kind = 'NIHImage' 3377 else: 3378 kind = 'Uniform' 3379 return [TiffPageSeries(pages, shape, dtype, axes, kind=kind)] 3380 3381 def _series_generic(self): 3382 """Return image series in file. 3383 3384 A series is a sequence of TiffPages with the same hash. 3385 3386 """ 3387 pages = self.pages 3388 pages._clear(False) 3389 pages.useframes = False 3390 if pages.cache: 3391 pages._load() 3392 3393 series = [] 3394 keys = [] 3395 seriesdict = {} 3396 3397 def addpage(page): 3398 # add page to seriesdict 3399 if not page.shape: # or product(page.shape) == 0: 3400 return 3401 key = page.hash 3402 if key in seriesdict: 3403 for p in seriesdict[key]: 3404 if p.offset == page.offset: 3405 break # remove duplicate page 3406 else: 3407 seriesdict[key].append(page) 3408 else: 3409 keys.append(key) 3410 seriesdict[key] = [page] 3411 3412 for page in pages: 3413 addpage(page) 3414 if page.subifds is not None: 3415 for i, offset in enumerate(page.subifds): 3416 if offset < 8: 3417 continue 3418 try: 3419 self._fh.seek(offset) 3420 subifd = TiffPage(self, (page.index, i)) 3421 except Exception as exc: 3422 log_warning( 3423 f'Generic series: {exc.__class__.__name__}: {exc}' 3424 ) 3425 else: 3426 addpage(subifd) 3427 3428 for key in keys: 3429 pages = seriesdict[key] 3430 page = pages[0] 3431 shape = (len(pages),) + page.shape 3432 axes = 'I' + page.axes 3433 if 'S' not in axes: 3434 shape += (1,) 3435 axes += 'S' 3436 series.append( 3437 TiffPageSeries(pages, shape, page.dtype, axes, kind='Generic') 3438 ) 3439 3440 self.is_uniform = len(series) == 1 3441 pyramidize_series(series) 3442 return series 3443 3444 def _series_shaped(self): 3445 """Return image series in "shaped" file.""" 3446 3447 def append(series, pages, axes, shape, reshape, name, truncated): 3448 # append TiffPageSeries to series 3449 page = pages[0] 3450 if not axes: 3451 shape = page.shape 3452 axes = page.axes 3453 if len(pages) > 1: 3454 shape = (len(pages),) + shape 3455 axes = 'Q' + axes 3456 size = product(shape) 3457 resize = product(reshape) 3458 if page.is_contiguous and resize > size and resize % size == 0: 3459 if truncated is None: 3460 truncated = True 3461 axes = 'Q' + axes 3462 shape = (resize // size,) + shape 3463 try: 3464 axes = reshape_axes(axes, shape, reshape) 3465 shape = reshape 3466 except ValueError as exc: 3467 log_warning(f'Shaped series: {exc.__class__.__name__}: {exc}') 3468 series.append( 3469 TiffPageSeries( 3470 pages, 3471 shape, 3472 page.dtype, 3473 axes, 3474 name=name, 3475 kind='Shaped', 3476 truncated=truncated, 3477 squeeze=False, 3478 ) 3479 ) 3480 3481 def detect_series(pages, series, issubifds=False): 3482 lenpages = len(pages) 3483 keyframe = axes = shape = reshape = name = None 3484 index = 0 3485 while True: 3486 if index >= lenpages: 3487 break 3488 if issubifds: 3489 keyframe = pages[0] 3490 else: 3491 # new keyframe; start of new series 3492 pages.keyframe = index 3493 keyframe = pages.keyframe 3494 if not keyframe.is_shaped: 3495 log_warning( 3496 'Shaped series: invalid metadata or corrupted file' 3497 ) 3498 return None 3499 # read metadata 3500 axes = None 3501 shape = None 3502 metadata = json_description_metadata(keyframe.is_shaped) 3503 name = metadata.get('name', '') 3504 reshape = metadata['shape'] 3505 truncated = None if keyframe.subifds is None else False 3506 truncated = metadata.get('truncated', truncated) 3507 if 'axes' in metadata: 3508 axes = metadata['axes'] 3509 if len(axes) == len(reshape): 3510 shape = reshape 3511 else: 3512 axes = '' 3513 log_warning('Shaped series: axes do not match shape') 3514 # skip pages if possible 3515 spages = [keyframe] 3516 size = product(reshape) 3517 if size > 0: 3518 npages, mod = divmod(size, product(keyframe.shape)) 3519 else: 3520 npages = 1 3521 mod = 0 3522 if mod: 3523 log_warning( 3524 'Shaped series: series shape does not match page shape' 3525 ) 3526 return None 3527 if 1 < npages <= lenpages - index: 3528 size *= keyframe._dtype.itemsize 3529 if truncated: 3530 npages = 1 3531 elif ( 3532 keyframe.is_final 3533 and keyframe.offset + size < pages[index + 1].offset 3534 and keyframe.subifds is None 3535 ): 3536 truncated = False 3537 else: 3538 # must read all pages for series 3539 truncated = False 3540 for j in range(index + 1, index + npages): 3541 page = pages[j] 3542 page.keyframe = keyframe 3543 spages.append(page) 3544 append(series, spages, axes, shape, reshape, name, truncated) 3545 index += npages 3546 3547 # create series from SubIFDs 3548 if keyframe.subifds: 3549 for i, offset in enumerate(keyframe.subifds): 3550 if offset < 8: 3551 continue 3552 subifds = [] 3553 for j, page in enumerate(spages): 3554 # if page.subifds is not None: 3555 try: 3556 self._fh.seek(page.subifds[i]) 3557 if j == 0: 3558 subifd = TiffPage(self, (page.index, i)) 3559 keysubifd = subifd 3560 else: 3561 subifd = TiffFrame( 3562 self, 3563 (page.index, i), 3564 keyframe=keysubifd, 3565 ) 3566 except Exception as exc: 3567 log_warning( 3568 f'Generic series: {exc.__class__.__name__}' 3569 f': {exc}' 3570 ) 3571 return None 3572 subifds.append(subifd) 3573 if subifds: 3574 series = detect_series(subifds, series, True) 3575 if series is None: 3576 return None 3577 return series 3578 3579 self.pages.useframes = True 3580 series = detect_series(self.pages, []) 3581 if series is None: 3582 return None 3583 self.is_uniform = len(series) == 1 3584 pyramidize_series(series, isreduced=True) 3585 return series 3586 3587 def _series_imagej(self): 3588 """Return image series in ImageJ file.""" 3589 # ImageJ's dimension order is TZCYXS 3590 # TODO: fix loading of color, composite, or palette images 3591 pages = self.pages 3592 pages.useframes = True 3593 pages.keyframe = 0 3594 page = pages[0] 3595 meta = self.imagej_metadata 3596 3597 def is_virtual(): 3598 # ImageJ virtual hyperstacks store all image metadata in the first 3599 # page and image data are stored contiguously before the second 3600 # page, if any 3601 if not page.is_final: 3602 return False 3603 images = meta.get('images', 0) 3604 if images <= 1: 3605 return False 3606 offset, count = page.is_contiguous 3607 if ( 3608 count != product(page.shape) * page.bitspersample // 8 3609 or offset + count * images > self.filehandle.size 3610 ): 3611 raise ValueError 3612 # check that next page is stored after data 3613 if len(pages) > 1 and offset + count * images > pages[1].offset: 3614 return False 3615 return True 3616 3617 try: 3618 isvirtual = is_virtual() 3619 except (ValueError, RuntimeError) as exc: 3620 log_warning( 3621 f'ImageJ series: invalid metadata or corrupted file ({exc})' 3622 ) 3623 return None 3624 if isvirtual: 3625 # no need to read other pages 3626 pages = [page] 3627 else: 3628 pages = pages[:] 3629 3630 images = meta.get('images', len(pages)) 3631 frames = meta.get('frames', 1) 3632 slices = meta.get('slices', 1) 3633 channels = meta.get('channels', 1) 3634 3635 shape = (frames, slices, channels) 3636 axes = 'TZC' 3637 3638 remain = images // product(shape) 3639 if remain > 1: 3640 log_warning("ImageJ series: detected extra dimension 'I'") 3641 shape = (remain,) + shape 3642 axes = 'I' + axes 3643 3644 if page.shaped[0] > 1: 3645 # planar storage, S == C, saved by Bio-Formats 3646 if page.shaped[0] != channels: 3647 log_warning( 3648 f'ImageJ series: number of channels {channels} ' 3649 f'do not match separate samples {page.shaped[0]}' 3650 ) 3651 shape = shape[:-1] + page.shape 3652 axes += page.axes[1:] 3653 elif page.shaped[-1] == channels and channels > 1: 3654 # keep contig storage, C = 1 3655 shape = (frames, slices, 1) + page.shape 3656 axes += page.axes 3657 else: 3658 shape += page.shape 3659 axes += page.axes 3660 3661 if 'S' not in axes: 3662 shape += (1,) 3663 axes += 'S' 3664 # assert axes.endswith('TZCYXS'), axes 3665 3666 truncated = ( 3667 isvirtual 3668 and len(self.pages) == 1 3669 and page.is_contiguous[1] 3670 != (product(shape) * page.bitspersample // 8) 3671 ) 3672 3673 self.is_uniform = True 3674 3675 return [ 3676 TiffPageSeries( 3677 pages, 3678 shape, 3679 page.dtype, 3680 axes, 3681 kind='ImageJ', 3682 truncated=truncated, 3683 ) 3684 ] 3685 3686 def _series_scanimage(self): 3687 """Return image series in ScanImage file.""" 3688 pages = self.pages._getlist(validate=False) 3689 page = pages[0] 3690 dtype = page.dtype 3691 shape = None 3692 3693 framedata = self.scanimage_metadata.get('FrameData', {}) 3694 if 'SI.hChannels.channelSave' in framedata: 3695 try: 3696 channels = framedata['SI.hChannels.channelSave'] 3697 try: 3698 # channelSave is a list 3699 channels = len(channels) 3700 except TypeError: 3701 # channelSave is an int 3702 channels = int(channels) 3703 # slices = framedata.get( 3704 # 'SI.hStackManager.actualNumSlices', 3705 # framedata.get('SI.hStackManager.numSlices', None), 3706 # ) 3707 # if slices is None: 3708 # raise ValueError('unable to determine numSlices') 3709 slices = None 3710 try: 3711 frames = int(framedata['SI.hStackManager.framesPerSlice']) 3712 except Exception: 3713 # framesPerSlice is inf 3714 slices = 1 3715 if len(pages) % channels: 3716 raise ValueError('unable to determine framesPerSlice') 3717 frames = len(pages) // channels 3718 if slices is None: 3719 slices = max(len(pages) // (frames * channels), 1) 3720 shape = (slices, frames, channels) + page.shape 3721 axes = 'ZTC' + page.axes 3722 except Exception as exc: 3723 log_warning(f'ScanImage series: {exc}') 3724 3725 # TODO: older versions of ScanImage store non-varying frame data in 3726 # the ImageDescription tag. Candidates are scanimage.SI5.channelsSave, 3727 # scanimage.SI5.stackNumSlices, scanimage.SI5.acqNumFrames 3728 # scanimage.SI4., state.acq.numberOfFrames, state.acq.numberOfFrames... 3729 3730 if shape is None: 3731 shape = (len(pages),) + page.shape 3732 axes = 'I' + page.axes 3733 3734 return [TiffPageSeries(pages, shape, dtype, axes, kind='ScanImage')] 3735 3736 def _series_fluoview(self): 3737 """Return image series in FluoView file.""" 3738 pages = self.pages._getlist(validate=False) 3739 3740 mm = self.fluoview_metadata 3741 mmhd = list(reversed(mm['Dimensions'])) 3742 axes = ''.join(TIFF.MM_DIMENSIONS.get(i[0].upper(), 'Q') for i in mmhd) 3743 shape = tuple(int(i[1]) for i in mmhd) 3744 self.is_uniform = True 3745 return [ 3746 TiffPageSeries( 3747 pages, 3748 shape, 3749 pages[0].dtype, 3750 axes, 3751 name=mm['ImageName'], 3752 kind='FluoView', 3753 ) 3754 ] 3755 3756 def _series_mdgel(self): 3757 """Return image series in MD Gel file.""" 3758 # only a single page, scaled according to metadata in second page 3759 self.pages.useframes = False 3760 self.pages.keyframe = 0 3761 md = self.mdgel_metadata 3762 if md['FileTag'] in (2, 128): 3763 dtype = numpy.dtype('float32') 3764 scale = md['ScalePixel'] 3765 scale = scale[0] / scale[1] # rational 3766 if md['FileTag'] == 2: 3767 # squary root data format 3768 def transform(a): 3769 return a.astype('float32') ** 2 * scale 3770 3771 else: 3772 3773 def transform(a): 3774 return a.astype('float32') * scale 3775 3776 else: 3777 transform = None 3778 page = self.pages[0] 3779 self.is_uniform = False 3780 return [ 3781 TiffPageSeries( 3782 [page], 3783 page.shape, 3784 dtype, 3785 page.axes, 3786 transform=transform, 3787 kind='MDGel', 3788 ) 3789 ] 3790 3791 def _series_ndpi(self): 3792 """Return pyramidal image series in NDPI file.""" 3793 series = self._series_generic() 3794 for s in series: 3795 s.kind = 'NDPI' 3796 if s.axes[0] == 'I': 3797 s.axes = 'Z' + s.axes[1:] 3798 if s.is_pyramidal: 3799 name = s.pages[0].tags.get(65427, None) 3800 s.name = 'Baseline' if name is None else name.value 3801 continue 3802 mag = s.pages[0].tags.get(65421, None) 3803 if mag is not None: 3804 mag = mag.value 3805 if mag == -1.0: 3806 s.name = 'Macro' 3807 elif mag == -2.0: 3808 s.name = 'Map' 3809 return series 3810 3811 def _series_sis(self): 3812 """Return image series in Olympus SIS file.""" 3813 pages = self.pages._getlist(validate=False) 3814 page = pages[0] 3815 lenpages = len(pages) 3816 md = self.sis_metadata 3817 if 'shape' in md and 'axes' in md: 3818 shape = md['shape'] + page.shape 3819 axes = md['axes'] + page.axes 3820 else: 3821 shape = (lenpages,) + page.shape 3822 axes = 'I' + page.axes 3823 self.is_uniform = True 3824 return [TiffPageSeries(pages, shape, page.dtype, axes, kind='SIS')] 3825 3826 def _series_qpi(self): 3827 """Return image series in PerkinElmer QPI file.""" 3828 series = [] 3829 pages = self.pages 3830 pages.cache = True 3831 pages.useframes = False 3832 pages.keyframe = 0 3833 pages._load() 3834 3835 # Baseline 3836 # TODO: get name from ImageDescription XML 3837 ifds = [] 3838 index = 0 3839 axes = 'C' + pages[0].axes 3840 dtype = pages[0].dtype 3841 pshape = pages[0].shape 3842 while index < len(pages): 3843 page = pages[index] 3844 if page.shape != pshape: 3845 break 3846 ifds.append(page) 3847 index += 1 3848 shape = (len(ifds),) + pshape 3849 series.append( 3850 TiffPageSeries( 3851 ifds, shape, dtype, axes, name='Baseline', kind='QPI' 3852 ) 3853 ) 3854 3855 if index < len(pages): 3856 # Thumbnail 3857 page = pages[index] 3858 series.append( 3859 TiffPageSeries( 3860 [page], 3861 page.shape, 3862 page.dtype, 3863 page.axes, 3864 name='Thumbnail', 3865 kind='QPI', 3866 ) 3867 ) 3868 index += 1 3869 3870 if pages[0].is_tiled: 3871 # Resolutions 3872 while index < len(pages): 3873 pshape = (pshape[0] // 2, pshape[1] // 2) + pshape[2:] 3874 ifds = [] 3875 while index < len(pages): 3876 page = pages[index] 3877 if page.shape != pshape: 3878 break 3879 ifds.append(page) 3880 index += 1 3881 if len(ifds) != len(series[0].pages): 3882 break 3883 shape = (len(ifds),) + pshape 3884 series[0].levels.append( 3885 TiffPageSeries( 3886 ifds, shape, dtype, axes, name='Resolution', kind='QPI' 3887 ) 3888 ) 3889 3890 if series[0].is_pyramidal and index < len(pages): 3891 # Macro 3892 page = pages[index] 3893 series.append( 3894 TiffPageSeries( 3895 [page], 3896 page.shape, 3897 page.dtype, 3898 page.axes, 3899 name='Macro', 3900 kind='QPI', 3901 ) 3902 ) 3903 index += 1 3904 # Label 3905 if index < len(pages): 3906 page = pages[index] 3907 series.append( 3908 TiffPageSeries( 3909 [page], 3910 page.shape, 3911 page.dtype, 3912 page.axes, 3913 name='Label', 3914 kind='QPI', 3915 ) 3916 ) 3917 3918 self.is_uniform = False 3919 return series 3920 3921 def _series_svs(self): 3922 """Return image series in Aperio SVS file.""" 3923 if not self.pages[0].is_tiled: 3924 return None 3925 3926 series = [] 3927 self.is_uniform = False 3928 self.pages.cache = True 3929 self.pages.useframes = False 3930 self.pages.keyframe = 0 3931 self.pages._load() 3932 3933 # Baseline 3934 index = 0 3935 page = self.pages[index] 3936 series.append( 3937 TiffPageSeries( 3938 [page], 3939 page.shape, 3940 page.dtype, 3941 page.axes, 3942 name='Baseline', 3943 kind='SVS', 3944 ) 3945 ) 3946 # Thumbnail 3947 index += 1 3948 if index == len(self.pages): 3949 return series 3950 page = self.pages[index] 3951 series.append( 3952 TiffPageSeries( 3953 [page], 3954 page.shape, 3955 page.dtype, 3956 page.axes, 3957 name='Thumbnail', 3958 kind='SVS', 3959 ) 3960 ) 3961 # Resolutions 3962 # TODO: resolutions not by two 3963 index += 1 3964 while index < len(self.pages): 3965 page = self.pages[index] 3966 if not page.is_tiled or page.is_reduced: 3967 break 3968 series[0].levels.append( 3969 TiffPageSeries( 3970 [page], 3971 page.shape, 3972 page.dtype, 3973 page.axes, 3974 name='Resolution', 3975 kind='SVS', 3976 ) 3977 ) 3978 index += 1 3979 # Label, Macro; subfiletype 1, 9 3980 for name in ('Label', 'Macro'): 3981 if index == len(self.pages): 3982 break 3983 page = self.pages[index] 3984 series.append( 3985 TiffPageSeries( 3986 [page], 3987 page.shape, 3988 page.dtype, 3989 page.axes, 3990 name=name, 3991 kind='SVS', 3992 ) 3993 ) 3994 index += 1 3995 3996 return series 3997 3998 def _series_scn(self): 3999 """Return pyramidal image series in Leica SCN file.""" 4000 # TODO: support collections 4001 from xml.etree import ElementTree as etree # delayed import 4002 4003 scnxml = self.pages[0].description 4004 root = etree.fromstring(scnxml) 4005 4006 series = [] 4007 self.is_uniform = False 4008 self.pages.cache = True 4009 self.pages.useframes = False 4010 self.pages.keyframe = 0 4011 self.pages._load() 4012 4013 for collection in root: 4014 if not collection.tag.endswith('collection'): 4015 continue 4016 for image in collection: 4017 if not image.tag.endswith('image'): 4018 continue 4019 name = image.attrib.get('name', 'Unknown') 4020 for pixels in image: 4021 if not pixels.tag.endswith('pixels'): 4022 continue 4023 resolutions = {} 4024 for dimension in pixels: 4025 if not dimension.tag.endswith('dimension'): 4026 continue 4027 if int(image.attrib.get('sizeZ', 1)) > 1: 4028 raise NotImplementedError( 4029 'SCN series: Z-Stacks not supported' 4030 ) 4031 sizex = int(dimension.attrib['sizeX']) 4032 sizey = int(dimension.attrib['sizeY']) 4033 c = int(dimension.attrib.get('c', 0)) 4034 z = int(dimension.attrib.get('z', 0)) 4035 r = int(dimension.attrib.get('r', 0)) 4036 ifd = int(dimension.attrib['ifd']) 4037 if r in resolutions: 4038 level = resolutions[r] 4039 if c > level['channels']: 4040 level['channels'] = c 4041 if z > level['sizez']: 4042 level['sizez'] = z 4043 level['ifds'][(c, z)] = ifd 4044 else: 4045 resolutions[r] = { 4046 'size': [sizey, sizex], 4047 'channels': c, 4048 'sizez': z, 4049 'ifds': {(c, z): ifd}, 4050 } 4051 if not resolutions: 4052 continue 4053 levels = [] 4054 for r, level in sorted(resolutions.items()): 4055 shape = (level['channels'] + 1, level['sizez'] + 1) 4056 axes = 'CZ' 4057 4058 ifds = [None] * product(shape) 4059 for (c, z), ifd in sorted(level['ifds'].items()): 4060 ifds[c * shape[1] + z] = self.pages[ifd] 4061 4062 axes += ifds[0].axes 4063 shape += ifds[0].shape 4064 dtype = ifds[0].dtype 4065 4066 levels.append( 4067 TiffPageSeries( 4068 ifds, 4069 shape, 4070 dtype, 4071 axes, 4072 parent=self, 4073 name=name, 4074 kind='SCN', 4075 ) 4076 ) 4077 levels[0].levels.extend(levels[1:]) 4078 series.append(levels[0]) 4079 4080 return series 4081 4082 def _series_bif(self): 4083 """Return image series in Ventana BIF file.""" 4084 if not self.pages[0].is_tiled: 4085 return None 4086 4087 series = [] 4088 baseline = None 4089 self.is_uniform = False 4090 self.pages.cache = True 4091 self.pages.useframes = False 4092 self.pages.keyframe = 0 4093 self.pages._load() 4094 4095 for page in self.pages: 4096 if page.description == 'Label Image': 4097 series.append( 4098 TiffPageSeries( 4099 [page], 4100 page.shape, 4101 page.dtype, 4102 page.axes, 4103 name='Label', 4104 kind='BIF', 4105 ) 4106 ) 4107 elif page.description == 'Thumbnail': 4108 series.append( 4109 TiffPageSeries( 4110 [page], 4111 page.shape, 4112 page.dtype, 4113 page.axes, 4114 name='Thumbnail', 4115 kind='BIF', 4116 ) 4117 ) 4118 elif 'level' not in page.description: 4119 # TODO: is this necessary? 4120 series.append( 4121 TiffPageSeries( 4122 [page], 4123 page.shape, 4124 page.dtype, 4125 page.axes, 4126 name='Unknown', 4127 kind='BIF', 4128 ) 4129 ) 4130 elif baseline is None: 4131 baseline = TiffPageSeries( 4132 [page], 4133 page.shape, 4134 page.dtype, 4135 page.axes, 4136 name='Baseline', 4137 kind='BIF', 4138 ) 4139 series.insert(0, baseline) 4140 else: 4141 baseline.levels.append( 4142 TiffPageSeries( 4143 [page], 4144 page.shape, 4145 page.dtype, 4146 page.axes, 4147 name='Resolution', 4148 kind='SVS', 4149 ) 4150 ) 4151 4152 log_warning('BIF series: beware, tiles are not stiched') 4153 return series 4154 4155 def _series_ome(self): 4156 """Return image series in OME-TIFF file(s).""" 4157 # xml.etree found to be faster than lxml 4158 from xml.etree import ElementTree as etree # delayed import 4159 4160 omexml = self.pages[0].description 4161 try: 4162 root = etree.fromstring(omexml) 4163 except etree.ParseError as exc: 4164 # TODO: test badly encoded OME-XML 4165 log_warning(f'OME series: {exc.__class__.__name__}: {exc}') 4166 try: 4167 omexml = omexml.decode(errors='ignore').encode() 4168 root = etree.fromstring(omexml) 4169 except Exception: 4170 return None 4171 4172 self.pages.cache = True 4173 self.pages.useframes = True 4174 self.pages.keyframe = 0 4175 self.pages._load(keyframe=None) 4176 4177 root_uuid = root.attrib.get('UUID', None) 4178 self._files = {root_uuid: self} 4179 dirname = self._fh.dirname 4180 moduloref = [] 4181 modulo = {} 4182 series = [] 4183 for element in root: 4184 if element.tag.endswith('BinaryOnly'): 4185 # TODO: load OME-XML from master or companion file 4186 log_warning('OME series: not an ome-tiff master file') 4187 break 4188 if element.tag.endswith('StructuredAnnotations'): 4189 for annot in element: 4190 if not annot.attrib.get('Namespace', '').endswith( 4191 'modulo' 4192 ): 4193 continue 4194 modulo[annot.attrib['ID']] = mod = {} 4195 for value in annot: 4196 for modul in value: 4197 for along in modul: 4198 if not along.tag[:-1].endswith('Along'): 4199 continue 4200 axis = along.tag[-1] 4201 newaxis = along.attrib.get('Type', 'other') 4202 newaxis = TIFF.AXES_LABELS[newaxis] 4203 if 'Start' in along.attrib: 4204 step = float(along.attrib.get('Step', 1)) 4205 start = float(along.attrib['Start']) 4206 stop = float(along.attrib['End']) + step 4207 labels = numpy.arange(start, stop, step) 4208 else: 4209 labels = [ 4210 label.text 4211 for label in along 4212 if label.tag.endswith('Label') 4213 ] 4214 mod[axis] = (newaxis, labels) 4215 4216 if not element.tag.endswith('Image'): 4217 continue 4218 4219 for annot in element: 4220 if annot.tag.endswith('AnnotationRef'): 4221 annotationref = annot.attrib['ID'] 4222 break 4223 else: 4224 annotationref = None 4225 4226 attr = element.attrib 4227 name = attr.get('Name', None) 4228 4229 for pixels in element: 4230 if not pixels.tag.endswith('Pixels'): 4231 continue 4232 attr = pixels.attrib 4233 # dtype = attr.get('PixelType', None) 4234 axes = ''.join(reversed(attr['DimensionOrder'])) 4235 shape = [int(attr['Size' + ax]) for ax in axes] 4236 ifds = [] 4237 spp = 1 # samples per pixel 4238 first = True 4239 4240 for data in pixels: 4241 if data.tag.endswith('Channel'): 4242 attr = data.attrib 4243 if first: 4244 first = False 4245 spp = int(attr.get('SamplesPerPixel', spp)) 4246 if spp > 1: 4247 # correct channel dimension for spp 4248 shape = [ 4249 shape[i] // spp if ax == 'C' else shape[i] 4250 for i, ax in enumerate(axes) 4251 ] 4252 elif int(attr.get('SamplesPerPixel', 1)) != spp: 4253 raise ValueError( 4254 'OME series: cannot handle differing ' 4255 'SamplesPerPixel' 4256 ) 4257 continue 4258 4259 if not data.tag.endswith('TiffData'): 4260 continue 4261 4262 attr = data.attrib 4263 ifd = int(attr.get('IFD', 0)) 4264 num = int(attr.get('NumPlanes', 1 if 'IFD' in attr else 0)) 4265 num = int(attr.get('PlaneCount', num)) 4266 idx = [int(attr.get('First' + ax, 0)) for ax in axes[:-2]] 4267 try: 4268 idx = int(numpy.ravel_multi_index(idx, shape[:-2])) 4269 except ValueError: 4270 # ImageJ produces invalid ome-xml when cropping 4271 log_warning('OME series: invalid TiffData index') 4272 continue 4273 for uuid in data: 4274 if not uuid.tag.endswith('UUID'): 4275 continue 4276 if root_uuid is None and uuid.text is not None: 4277 # no global UUID, use this file 4278 root_uuid = uuid.text 4279 self._files[root_uuid] = self._files[None] 4280 elif uuid.text not in self._files: 4281 if not self._multifile: 4282 # abort reading multifile OME series 4283 # and fall back to generic series 4284 return [] 4285 fname = uuid.attrib['FileName'] 4286 try: 4287 tif = TiffFile( 4288 os.path.join(dirname, fname), _parent=self 4289 ) 4290 tif.pages.cache = True 4291 tif.pages.useframes = True 4292 tif.pages.keyframe = 0 4293 tif.pages._load(keyframe=None) 4294 except (OSError, FileNotFoundError, ValueError): 4295 log_warning( 4296 f'OME series: failed to read {fname!r}' 4297 ) 4298 # assume that size is same as in previous file 4299 # if no NumPlanes or PlaneCount are given 4300 size = num if num else size # noqa: undefined 4301 ifds.extend([None] * (size + idx - len(ifds))) 4302 break 4303 self._files[uuid.text] = tif 4304 tif.close() 4305 pages = self._files[uuid.text].pages 4306 try: 4307 size = num if num else len(pages) 4308 ifds.extend([None] * (size + idx - len(ifds))) 4309 for i in range(size): 4310 ifds[idx + i] = pages[ifd + i] 4311 except IndexError: 4312 log_warning('OME series: index out of range') 4313 # only process first UUID 4314 break 4315 else: 4316 # no uuid found 4317 pages = self.pages 4318 try: 4319 size = num if num else len(pages) 4320 ifds.extend([None] * (size + idx - len(ifds))) 4321 for i in range(size): 4322 ifds[idx + i] = pages[ifd + i] 4323 except IndexError: 4324 log_warning('OME series: index out of range') 4325 4326 if not ifds or all(i is None for i in ifds): 4327 # skip images without data 4328 continue 4329 4330 # find a keyframe 4331 keyframe = None 4332 for ifd in ifds: 4333 # try find a TiffPage 4334 if ifd and ifd == ifd.keyframe: 4335 keyframe = ifd 4336 break 4337 if keyframe is None: 4338 # reload a TiffPage from file 4339 for i, keyframe in enumerate(ifds): 4340 if keyframe: 4341 isclosed = keyframe.parent.filehandle.closed 4342 if isclosed: 4343 keyframe.parent.filehandle.open() 4344 keyframe.parent.pages.keyframe = keyframe.index 4345 keyframe = keyframe.parent.pages[keyframe.index] 4346 ifds[i] = keyframe 4347 if isclosed: 4348 keyframe.parent.filehandle.close() 4349 break 4350 4351 # does the series spawn multiple files 4352 multifile = False 4353 for ifd in ifds: 4354 if ifd and ifd.parent != keyframe.parent: 4355 multifile = True 4356 break 4357 4358 if spp > 1: 4359 if keyframe.planarconfig == 1: 4360 shape += [spp] 4361 axes += 'S' 4362 else: 4363 shape = shape[:-2] + [spp] + shape[-2:] 4364 axes = axes[:-2] + 'S' + axes[-2:] 4365 if 'S' not in shape: 4366 shape += [1] 4367 axes += 'S' 4368 4369 # there might be more pages in the file than referenced in XML 4370 # e.g. Nikon-cell011.ome.tif 4371 size = max(product(shape) // keyframe.size, 1) 4372 if size != len(ifds): 4373 log_warning( 4374 'OME series: expected %s frames, got %s', 4375 size, 4376 len(ifds), 4377 ) 4378 ifds = ifds[:size] 4379 4380 # FIXME: this implementation assumes the last dimensions are 4381 # stored in TIFF pages. Apparently that is not always the case. 4382 # E.g. TCX (20000, 2, 500) is stored in 2 pages of (20000, 500) 4383 # in 'Image 7.ome_h00.tiff'. 4384 # For now, verify that shapes of keyframe and series match. 4385 # If not, skip series. 4386 squeezed = squeeze_axes(shape, axes)[0] 4387 if keyframe.shape != tuple(squeezed[-len(keyframe.shape) :]): 4388 log_warning( 4389 'OME series: cannot handle discontiguous storage ' 4390 '%s != %s', 4391 keyframe.shape, 4392 tuple(squeezed[-len(keyframe.shape) :]), 4393 ) 4394 del ifds 4395 continue 4396 4397 # set keyframe on all IFDs 4398 keyframes = {keyframe.parent.filehandle.name: keyframe} 4399 for i, page in enumerate(ifds): 4400 if page is None: 4401 continue 4402 fh = page.parent.filehandle 4403 if fh.name not in keyframes: 4404 if page.keyframe != page: 4405 # reload TiffPage from file 4406 isclosed = fh.closed 4407 if isclosed: 4408 fh.open() 4409 page.parent.pages.keyframe = page.index 4410 page = page.parent.pages[page.index] 4411 ifds[i] = page 4412 if isclosed: 4413 fh.close() 4414 keyframes[fh.name] = page 4415 if page.keyframe != page: 4416 page.keyframe = keyframes[fh.name] 4417 4418 moduloref.append(annotationref) 4419 series.append( 4420 TiffPageSeries( 4421 ifds, 4422 shape, 4423 keyframe.dtype, 4424 axes, 4425 parent=self, 4426 name=name, 4427 multifile=multifile, 4428 kind='OME', 4429 ) 4430 ) 4431 del ifds 4432 4433 for serie, annotationref in zip(series, moduloref): 4434 if annotationref not in modulo: 4435 continue 4436 shape = list(serie.get_shape(False)) 4437 axes = serie.get_axes(False) 4438 for axis, (newaxis, labels) in modulo[annotationref].items(): 4439 i = axes.index(axis) 4440 size = len(labels) 4441 if shape[i] == size: 4442 axes = axes.replace(axis, newaxis, 1) 4443 else: 4444 shape[i] //= size 4445 shape.insert(i + 1, size) 4446 axes = axes.replace(axis, axis + newaxis, 1) 4447 serie.set_shape_axes(shape, axes) 4448 4449 # pyramids 4450 for serie in series: 4451 keyframe = serie.keyframe 4452 if keyframe.subifds is None: 4453 continue 4454 if len(self._files) > 1: 4455 # TODO: support multi-file pyramids; must re-open/close 4456 log_warning('OME series: cannot read multi-file pyramids') 4457 break 4458 for level in range(len(keyframe.subifds)): 4459 keyframe = None 4460 ifds = [] 4461 for page in serie.pages: 4462 if page is None: 4463 ifds.append(None) 4464 continue 4465 page.parent.filehandle.seek(page.subifds[level]) 4466 if page.keyframe == page: 4467 ifd = keyframe = TiffPage(self, (page.index, level)) 4468 elif keyframe is None: 4469 raise RuntimeError('no keyframe') 4470 else: 4471 ifd = TiffFrame(self, page.index, keyframe=keyframe) 4472 ifds.append(ifd) 4473 # fix shape 4474 shape = [] 4475 for i, ax in enumerate(serie.axes): 4476 if ax == 'X': 4477 shape.append(keyframe.imagewidth) 4478 elif ax == 'Y': 4479 shape.append(keyframe.imagelength) 4480 else: 4481 shape.append(serie.shape[i]) 4482 # add series 4483 serie.levels.append( 4484 TiffPageSeries( 4485 ifds, 4486 tuple(shape), 4487 keyframe.dtype, 4488 serie.axes, 4489 parent=self, 4490 name=f'level {level + 1}', 4491 kind='OME', 4492 ) 4493 ) 4494 4495 self.is_uniform = len(series) == 1 and len(series[0].levels) == 1 4496 4497 return series 4498 4499 def _series_stk(self): 4500 """Return series in STK file.""" 4501 page = self.pages[0] 4502 meta = self.stk_metadata 4503 planes = meta['NumberPlanes'] 4504 name = meta.get('Name', '') 4505 if planes == 1: 4506 shape = (1,) + page.shape 4507 axes = 'I' + page.axes 4508 elif numpy.all(meta['ZDistance'] != 0): 4509 shape = (planes,) + page.shape 4510 axes = 'Z' + page.axes 4511 elif numpy.all(numpy.diff(meta['TimeCreated']) != 0): 4512 shape = (planes,) + page.shape 4513 axes = 'T' + page.axes 4514 else: 4515 # TODO: determine other/combinations of dimensions 4516 shape = (planes,) + page.shape 4517 axes = 'I' + page.axes 4518 self.is_uniform = True 4519 series = TiffPageSeries( 4520 [page], 4521 shape, 4522 page.dtype, 4523 axes, 4524 name=name, 4525 truncated=planes > 1, 4526 kind='STK', 4527 ) 4528 return [series] 4529 4530 def _series_lsm(self): 4531 """Return main and thumbnail series in LSM file.""" 4532 lsmi = self.lsm_metadata 4533 axes = TIFF.CZ_LSMINFO_SCANTYPE[lsmi['ScanType']] 4534 if self.pages[0].photometric == 2: # RGB; more than one channel 4535 axes = axes.replace('C', '').replace('XY', 'XYC') 4536 if lsmi.get('DimensionP', 0) > 0: 4537 axes += 'P' 4538 if lsmi.get('DimensionM', 0) > 0: 4539 axes += 'M' 4540 axes = axes[::-1] 4541 shape = tuple(int(lsmi[TIFF.CZ_LSMINFO_DIMENSIONS[i]]) for i in axes) 4542 4543 name = lsmi.get('Name', '') 4544 pages = self.pages._getlist(slice(0, None, 2), validate=False) 4545 dtype = pages[0].dtype 4546 series = [ 4547 TiffPageSeries(pages, shape, dtype, axes, name=name, kind='LSM') 4548 ] 4549 4550 page = self.pages[1] 4551 if page.is_reduced: 4552 pages = self.pages._getlist(slice(1, None, 2), validate=False) 4553 dtype = page.dtype 4554 cp = 1 4555 i = 0 4556 while cp < len(pages) and i < len(shape) - 2: 4557 cp *= shape[i] 4558 i += 1 4559 shape = shape[:i] + page.shape 4560 axes = axes[:i] + 'SYX' 4561 series.append( 4562 TiffPageSeries( 4563 pages, shape, dtype, axes, name=name, kind='LSMreduced' 4564 ) 4565 ) 4566 4567 self.is_uniform = False 4568 return series 4569 4570 def _lsm_load_pages(self): 4571 """Load and fix all pages from LSM file.""" 4572 # cache all pages to preserve corrected values 4573 pages = self.pages 4574 pages.cache = True 4575 pages.useframes = True 4576 # use first and second page as keyframes 4577 pages.keyframe = 1 4578 pages.keyframe = 0 4579 # load remaining pages as frames 4580 pages._load(keyframe=None) 4581 # fix offsets and bytecounts first 4582 # TODO: fix multiple conversions between lists and tuples 4583 self._lsm_fix_strip_offsets() 4584 self._lsm_fix_strip_bytecounts() 4585 # assign keyframes for data and thumbnail series 4586 keyframe = pages[0] 4587 for page in pages[::2]: 4588 page.keyframe = keyframe 4589 keyframe = pages[1] 4590 for page in pages[1::2]: 4591 page.keyframe = keyframe 4592 4593 def _lsm_fix_strip_offsets(self): 4594 """Unwrap strip offsets for LSM files greater than 4 GB. 4595 4596 Each series and position require separate unwrapping (undocumented). 4597 4598 """ 4599 if self.filehandle.size < 2 ** 32: 4600 return 4601 4602 pages = self.pages 4603 npages = len(pages) 4604 series = self.series[0] 4605 axes = series.axes 4606 4607 # find positions 4608 positions = 1 4609 for i in 0, 1: 4610 if series.axes[i] in 'PM': 4611 positions *= series.shape[i] 4612 4613 # make time axis first 4614 if positions > 1: 4615 ntimes = 0 4616 for i in 1, 2: 4617 if axes[i] == 'T': 4618 ntimes = series.shape[i] 4619 break 4620 if ntimes: 4621 div, mod = divmod(npages, 2 * positions * ntimes) 4622 if mod != 0: 4623 raise RuntimeError('mod != 0') 4624 shape = (positions, ntimes, div, 2) 4625 indices = numpy.arange(product(shape)).reshape(shape) 4626 indices = numpy.moveaxis(indices, 1, 0) 4627 else: 4628 indices = numpy.arange(npages).reshape(-1, 2) 4629 4630 # images of reduced page might be stored first 4631 if pages[0].dataoffsets[0] > pages[1].dataoffsets[0]: 4632 indices = indices[..., ::-1] 4633 4634 # unwrap offsets 4635 wrap = 0 4636 previousoffset = 0 4637 for i in indices.flat: 4638 page = pages[int(i)] 4639 dataoffsets = [] 4640 for currentoffset in page.dataoffsets: 4641 if currentoffset < previousoffset: 4642 wrap += 2 ** 32 4643 dataoffsets.append(currentoffset + wrap) 4644 previousoffset = currentoffset 4645 page.dataoffsets = tuple(dataoffsets) 4646 4647 def _lsm_fix_strip_bytecounts(self): 4648 """Set databytecounts to size of compressed data. 4649 4650 The StripByteCounts tag in LSM files contains the number of bytes 4651 for the uncompressed data. 4652 4653 """ 4654 pages = self.pages 4655 if pages[0].compression == 1: 4656 return 4657 # sort pages by first strip offset 4658 pages = sorted(pages, key=lambda p: p.dataoffsets[0]) 4659 npages = len(pages) - 1 4660 for i, page in enumerate(pages): 4661 if page.index % 2: 4662 continue 4663 offsets = page.dataoffsets 4664 bytecounts = page.databytecounts 4665 if i < npages: 4666 lastoffset = pages[i + 1].dataoffsets[0] 4667 else: 4668 # LZW compressed strips might be longer than uncompressed 4669 lastoffset = min( 4670 offsets[-1] + 2 * bytecounts[-1], self._fh.size 4671 ) 4672 bytecounts = list(bytecounts) 4673 for j in range(len(bytecounts) - 1): 4674 bytecounts[j] = offsets[j + 1] - offsets[j] 4675 bytecounts[-1] = lastoffset - offsets[-1] 4676 page.databytecounts = tuple(bytecounts) 4677 4678 def _ndpi_load_pages(self): 4679 """Load and fix pages from NDPI slide file if CaptureMode > 6. 4680 4681 If the value of the CaptureMode tag is greater than 6, change the 4682 attributes of the TiffPages that are part of the pyramid to match 4683 16-bit grayscale data. TiffTags are not corrected. 4684 4685 """ 4686 pages = self.pages 4687 capturemode = pages[0].tags.get(65441, None) 4688 if capturemode is None or capturemode.value < 6: 4689 return 4690 4691 pages.cache = True 4692 pages.useframes = False 4693 pages._load() 4694 4695 for page in pages: 4696 mag = page.tags.get(65421, None) 4697 if mag is None or mag.value > 0: 4698 page.photometric = TIFF.PHOTOMETRIC.MINISBLACK 4699 page.samplesperpixel = 1 4700 page.sampleformat = 1 4701 page.bitspersample = 16 4702 page.dtype = page._dtype = numpy.dtype('uint16') 4703 if page.shaped[-1] > 1: 4704 page.axes = page.axes[:-1] 4705 page.shape = page.shape[:-1] 4706 page.shaped = page.shaped[:-1] + (1,) 4707 4708 def _philips_load_pages(self): 4709 """Load and fix all pages from Philips slide file. 4710 4711 The imagewidth and imagelength values of all tiled pages are corrected 4712 using the DICOM_PIXEL_SPACING attributes of the XML formatted 4713 description of the first page. 4714 4715 """ 4716 from xml.etree import ElementTree as etree # delayed import 4717 4718 pages = self.pages 4719 pages.cache = True 4720 pages.useframes = False 4721 pages._load() 4722 npages = len(pages) 4723 4724 root = etree.fromstring(pages[0].description) 4725 4726 imagewidth = pages[0].imagewidth 4727 imagelength = pages[0].imagelength 4728 sizes = None 4729 for elem in root.iter(): 4730 if ( 4731 elem.tag != 'Attribute' 4732 or elem.attrib['Name'] != 'DICOM_PIXEL_SPACING' 4733 ): 4734 continue 4735 w, h = (float(v) for v in elem.text.replace('"', '').split()) 4736 if sizes is None: 4737 imagelength *= h 4738 imagewidth *= w 4739 sizes = [] 4740 else: 4741 sizes.append( 4742 ( 4743 int(math.ceil(imagelength / h)), 4744 int(math.ceil(imagewidth / w)), 4745 ) 4746 ) 4747 4748 i = 0 4749 for imagelength, imagewidth in sizes: 4750 while i < npages and pages[i].tilewidth == 0: 4751 # Label, Macro 4752 i += 1 4753 continue 4754 if i == npages: 4755 break 4756 page = pages[i] 4757 page.imagewidth = imagewidth 4758 page.imagelength = imagelength 4759 if page.shaped[-1] > 1: 4760 page.shape = (imagelength, imagewidth, page.shape[-1]) 4761 elif page.shaped[0] > 1: 4762 page.shape = (page.shape[0], imagelength, imagewidth) 4763 else: 4764 page.shape = (imagelength, imagewidth) 4765 page.shaped = ( 4766 page.shaped[:2] + (imagelength, imagewidth) + page.shaped[-1:] 4767 ) 4768 i += 1 4769 4770 def __getattr__(self, name): 4771 """Return 'is_flag' attributes from first page.""" 4772 if name[3:] in TIFF.FILE_FLAGS: 4773 if not self.pages: 4774 return False 4775 value = bool(getattr(self.pages[0], name)) 4776 setattr(self, name, value) 4777 return value 4778 raise AttributeError( 4779 f'{self.__class__.__name__!r} object has no attribute {name!r}' 4780 ) 4781 4782 def __enter__(self): 4783 return self 4784 4785 def __exit__(self, exc_type, exc_value, traceback): 4786 self.close() 4787 4788 def __str__(self, detail=0, width=79): 4789 """Return string containing information about TiffFile. 4790 4791 The detail parameter specifies the level of detail returned: 4792 4793 0: file only. 4794 1: all series, first page of series and its tags. 4795 2: large tag values and file metadata. 4796 3: all pages. 4797 4798 """ 4799 info = [ 4800 "TiffFile '{}'", 4801 format_size(self._fh.size), 4802 '' 4803 if byteorder_isnative(self.byteorder) 4804 else {'<': 'little-endian', '>': 'big-endian'}[self.byteorder], 4805 ] 4806 if self.is_bigtiff: 4807 info.append('BigTiff') 4808 info.append(' '.join(f.lower() for f in self.flags)) 4809 if len(self.pages) > 1: 4810 info.append(f'{len(self.pages)} Pages') 4811 if len(self.series) > 1: 4812 info.append(f'{len(self.series)} Series') 4813 if len(self._files) > 1: 4814 info.append(f'{len(self._files)} Files') 4815 info = ' '.join(info) 4816 info = info.replace(' ', ' ').replace(' ', ' ') 4817 info = info.format( 4818 snipstr(self._fh.name, max(12, width + 2 - len(info))) 4819 ) 4820 if detail <= 0: 4821 return info 4822 info = [info] 4823 info.append('\n'.join(str(s) for s in self.series)) 4824 if detail >= 3: 4825 for p in self.pages: 4826 if p is None: 4827 continue 4828 info.append(TiffPage.__str__(p, detail=detail, width=width)) 4829 for s in p.pages: 4830 info.append( 4831 TiffPage.__str__(s, detail=detail, width=width) 4832 ) 4833 elif self.series: 4834 info.extend( 4835 TiffPage.__str__(s.pages[0], detail=detail, width=width) 4836 for s in self.series 4837 if s.pages[0] is not None 4838 ) 4839 elif self.pages and self.pages[0]: 4840 info.append( 4841 TiffPage.__str__(self.pages[0], detail=detail, width=width) 4842 ) 4843 if detail >= 2: 4844 for name in sorted(self.flags): 4845 if hasattr(self, name + '_metadata'): 4846 m = getattr(self, name + '_metadata') 4847 if m: 4848 info.append( 4849 '{}_METADATA\n{}'.format( 4850 name.upper(), 4851 pformat(m, width=width, height=detail * 24), 4852 ) 4853 ) 4854 return '\n\n'.join(info).replace('\n\n\n', '\n\n') 4855 4856 @lazyattr 4857 def flags(self): 4858 """Return set of file flags, a potentially expensive operation.""" 4859 return { 4860 name.lower() 4861 for name in sorted(TIFF.FILE_FLAGS) 4862 if getattr(self, 'is_' + name) 4863 } 4864 4865 @property 4866 def is_bigtiff(self): 4867 """Return if file has BigTIFF format.""" 4868 return self.tiff.version == 43 4869 4870 @lazyattr 4871 def is_mdgel(self): 4872 """Return if file has MD Gel format.""" 4873 # side effect: add second page, if exists, to cache 4874 try: 4875 ismdgel = ( 4876 self.pages[0].is_mdgel 4877 or self.pages.get(1, cache=True).is_mdgel 4878 ) 4879 if ismdgel: 4880 self.is_uniform = False 4881 return ismdgel 4882 except IndexError: 4883 return False 4884 4885 @lazyattr 4886 def is_uniform(self): 4887 """Return if file contains a uniform series of pages.""" 4888 # the hashes of IFDs 0, 7, and -1 are the same 4889 pages = self.pages 4890 page = pages[0] 4891 if page.subifds: 4892 return False 4893 if page.is_scanimage or page.is_nih: 4894 return True 4895 try: 4896 useframes = pages.useframes 4897 pages.useframes = False 4898 h = page.hash 4899 for i in (1, 7, -1): 4900 if pages[i].aspage().hash != h: 4901 return False 4902 except IndexError: 4903 return False 4904 finally: 4905 pages.useframes = useframes 4906 return True 4907 4908 @property 4909 def is_appendable(self): 4910 """Return if pages can be appended to file without corrupting.""" 4911 # TODO: check other formats 4912 return not ( 4913 self.is_ome 4914 or self.is_lsm 4915 or self.is_stk 4916 or self.is_imagej 4917 or self.is_fluoview 4918 or self.is_micromanager 4919 ) 4920 4921 @lazyattr 4922 def shaped_metadata(self): 4923 """Return tifffile metadata from JSON descriptions as dicts.""" 4924 if not self.is_shaped: 4925 return None 4926 return tuple( 4927 json_description_metadata(s.pages[0].is_shaped) 4928 for s in self.series 4929 if s.kind.lower() == 'shaped' 4930 ) 4931 4932 @property 4933 def ome_metadata(self): 4934 """Return OME XML.""" 4935 if not self.is_ome: 4936 return None 4937 # return xml2dict(self.pages[0].description)['OME'] 4938 return self.pages[0].description 4939 4940 @property 4941 def scn_metadata(self): 4942 """Return Leica SCN XML.""" 4943 if not self.is_scn: 4944 return None 4945 return self.pages[0].description 4946 4947 @property 4948 def philips_metadata(self): 4949 """Return Philips DP XML.""" 4950 if not self.is_philips: 4951 return None 4952 return self.pages[0].description 4953 4954 @property 4955 def lsm_metadata(self): 4956 """Return LSM metadata from CZ_LSMINFO tag as dict.""" 4957 if not self.is_lsm: 4958 return None 4959 return self.pages[0].tags[34412].value # CZ_LSMINFO 4960 4961 @lazyattr 4962 def stk_metadata(self): 4963 """Return STK metadata from UIC tags as dict.""" 4964 if not self.is_stk: 4965 return None 4966 page = self.pages[0] 4967 result = {} 4968 result['NumberPlanes'] = page.tags[33629].count # UIC2tag 4969 if page.description: 4970 result['PlaneDescriptions'] = page.description.split('\x00') 4971 # result['plane_descriptions'] = stk_description_metadata( 4972 # page.image_description) 4973 tag = page.tags.get(33628) # UIC1tag 4974 if tag is not None: 4975 result.update(tag.value) 4976 tag = page.tags.get(33630) # UIC3tag 4977 if tag is not None: 4978 result.update(tag.value) # wavelengths 4979 tag = page.tags.get(33631) # UIC4tag 4980 if tag is not None: 4981 result.update(tag.value) # override UIC1 tags 4982 uic2tag = page.tags[33629].value 4983 result['ZDistance'] = uic2tag['ZDistance'] 4984 result['TimeCreated'] = uic2tag['TimeCreated'] 4985 result['TimeModified'] = uic2tag['TimeModified'] 4986 try: 4987 result['DatetimeCreated'] = numpy.array( 4988 [ 4989 julian_datetime(*dt) 4990 for dt in zip( 4991 uic2tag['DateCreated'], uic2tag['TimeCreated'] 4992 ) 4993 ], 4994 dtype='datetime64[ns]', 4995 ) 4996 result['DatetimeModified'] = numpy.array( 4997 [ 4998 julian_datetime(*dt) 4999 for dt in zip( 5000 uic2tag['DateModified'], uic2tag['TimeModified'] 5001 ) 5002 ], 5003 dtype='datetime64[ns]', 5004 ) 5005 except ValueError as exc: 5006 log_warning(f'STK metadata: {exc.__class__.__name__}: {exc}') 5007 return result 5008 5009 @lazyattr 5010 def imagej_metadata(self): 5011 """Return consolidated ImageJ metadata as dict.""" 5012 if not self.is_imagej: 5013 return None 5014 page = self.pages[0] 5015 result = imagej_description_metadata(page.is_imagej) 5016 tag = page.tags.get(50839) # IJMetadata 5017 if tag is not None: 5018 try: 5019 result.update(tag.value) 5020 except Exception: 5021 pass 5022 return result 5023 5024 @lazyattr 5025 def fluoview_metadata(self): 5026 """Return consolidated FluoView metadata as dict.""" 5027 if not self.is_fluoview: 5028 return None 5029 result = {} 5030 page = self.pages[0] 5031 result.update(page.tags[34361].value) # MM_Header 5032 # TODO: read stamps from all pages 5033 result['Stamp'] = page.tags[34362].value # MM_Stamp 5034 # skip parsing image description; not reliable 5035 # try: 5036 # t = fluoview_description_metadata(page.image_description) 5037 # if t is not None: 5038 # result['ImageDescription'] = t 5039 # except Exception as exc: 5040 # log_warning( 5041 # 'FluoView metadata: failed to parse image description ' 5042 # f'({exc})' 5043 # ) 5044 return result 5045 5046 @property 5047 def nih_metadata(self): 5048 """Return NIH Image metadata from NIHImageHeader tag as dict.""" 5049 if not self.is_nih: 5050 return None 5051 return self.pages[0].tags[43314].value # NIHImageHeader 5052 5053 @property 5054 def fei_metadata(self): 5055 """Return FEI metadata from SFEG or HELIOS tags as dict.""" 5056 if not self.is_fei: 5057 return None 5058 tags = self.pages[0].tags 5059 tag = tags.get(34680, tags.get(34682)) # FEI_SFEG or FEI_HELIOS 5060 return None if tag is None else tag.value 5061 5062 @property 5063 def sem_metadata(self): 5064 """Return SEM metadata from CZ_SEM tag as dict.""" 5065 if not self.is_sem: 5066 return None 5067 return self.pages[0].tags[34118].value 5068 5069 @property 5070 def sis_metadata(self): 5071 """Return Olympus SIS metadata from SIS and INI tags as dict.""" 5072 if not self.is_sis: 5073 return None 5074 tags = self.pages[0].tags 5075 result = {} 5076 try: 5077 result.update(tags[33471].value) # OlympusINI 5078 except Exception: 5079 pass 5080 try: 5081 result.update(tags[33560].value) # OlympusSIS 5082 except Exception: 5083 pass 5084 return result 5085 5086 @lazyattr 5087 def mdgel_metadata(self): 5088 """Return consolidated metadata from MD GEL tags as dict.""" 5089 for page in self.pages[:2]: 5090 if 33445 in page.tags: # MDFileTag 5091 tags = page.tags 5092 break 5093 else: 5094 return None 5095 result = {} 5096 for code in range(33445, 33453): 5097 if code not in tags: 5098 continue 5099 name = TIFF.TAGS[code] 5100 result[name[2:]] = tags[code].value 5101 return result 5102 5103 @property 5104 def andor_metadata(self): 5105 """Return Andor tags as dict.""" 5106 return self.pages[0].andor_tags 5107 5108 @property 5109 def epics_metadata(self): 5110 """Return EPICS areaDetector tags as dict.""" 5111 return self.pages[0].epics_tags 5112 5113 @property 5114 def tvips_metadata(self): 5115 """Return TVIPS tag as dict.""" 5116 if not self.is_tvips: 5117 return None 5118 return self.pages[0].tags[37706].value 5119 5120 @lazyattr 5121 def metaseries_metadata(self): 5122 """Return MetaSeries metadata from image description as dict.""" 5123 if not self.is_metaseries: 5124 return None 5125 return metaseries_description_metadata(self.pages[0].description) 5126 5127 @lazyattr 5128 def pilatus_metadata(self): 5129 """Return Pilatus metadata from image description as dict.""" 5130 if not self.is_pilatus: 5131 return None 5132 return pilatus_description_metadata(self.pages[0].description) 5133 5134 @lazyattr 5135 def micromanager_metadata(self): 5136 """Return MicroManager non-TIFF settings from file as dict.""" 5137 if not self.is_micromanager: 5138 return None 5139 # from file header 5140 return read_micromanager_metadata(self._fh) 5141 5142 @lazyattr 5143 def scanimage_metadata(self): 5144 """Return ScanImage non-varying frame and ROI metadata as dict. 5145 5146 The returned dict may be empty or contain 'FrameData', 'RoiGroups', 5147 and 'version' keys. 5148 5149 The varying frame data can be found in the ImageDescription tags. 5150 5151 """ 5152 if not self.is_scanimage: 5153 return None 5154 result = {} 5155 try: 5156 framedata, roidata, version = read_scanimage_metadata(self._fh) 5157 result['version'] = version 5158 result['FrameData'] = framedata 5159 result.update(roidata) 5160 except ValueError: 5161 pass 5162 return result 5163 5164 @property 5165 def geotiff_metadata(self): 5166 """Return GeoTIFF metadata from first page as dict.""" 5167 if not self.is_geotiff: 5168 return None 5169 return self.pages[0].geotiff_tags 5170 5171 @property 5172 def eer_metadata(self): 5173 """Return EER metadata from first page as XML.""" 5174 if not self.is_eer: 5175 return None 5176 return self.pages[0].tags[65001].value.decode() 5177 5178 5179class TiffPages: 5180 """Sequence of TIFF image file directories (IFD chain). 5181 5182 Instances of TiffPages have a state (cache, keyframe, etc.) and are not 5183 thread-safe. 5184 5185 """ 5186 5187 def __init__(self, arg, index=None): 5188 """Initialize instance and read first TiffPage from file. 5189 5190 If arg is a TiffFile, the file position must be at an offset to an 5191 offset to a TiffPage. If arg is a TiffPage, page offsets are read 5192 from the SubIFDs tag. 5193 5194 """ 5195 self.parent = None 5196 self.pages = [] # cache of TiffPages, TiffFrames, or their offsets 5197 self._indexed = False # True if offsets to all pages were read 5198 self._cached = False # True if all pages were read into cache 5199 self._tiffpage = TiffPage # class used for reading pages 5200 self._keyframe = None # page that is currently used as keyframe 5201 self._cache = False # do not cache frames or pages (if not keyframe) 5202 self._nextpageoffset = None 5203 self._index = (index,) if isinstance(index, int) else index 5204 5205 if isinstance(arg, TiffFile): 5206 # read offset to first page from current file position 5207 self.parent = arg 5208 fh = self.parent.filehandle 5209 self._nextpageoffset = fh.tell() 5210 offset = struct.unpack( 5211 self.parent.tiff.offsetformat, 5212 fh.read(self.parent.tiff.offsetsize), 5213 )[0] 5214 if offset == 0: 5215 log_warning('TiffPages: file contains no pages') 5216 self._indexed = True 5217 return 5218 elif 330 in arg.tags: 5219 # use offsets from SubIFDs tag 5220 self.parent = arg.parent 5221 fh = self.parent.filehandle 5222 offsets = arg.tags[330].value 5223 offset = offsets[0] 5224 if offset == 0: 5225 log_warning('TiffPages: TiffPage contains invalid SubIFDs') 5226 self._indexed = True 5227 return 5228 else: 5229 self._indexed = True 5230 return 5231 5232 if offset >= fh.size: 5233 log_warning(f'TiffPages: invalid page offset {offset!r}') 5234 self._indexed = True 5235 return 5236 5237 pageindex = 0 if self._index is None else self._index + (0,) 5238 5239 # read and cache first page 5240 fh.seek(offset) 5241 page = TiffPage(self.parent, index=pageindex) 5242 self.pages.append(page) 5243 self._keyframe = page 5244 if self._nextpageoffset is None: 5245 # offsets from SubIFDs tag 5246 self.pages.extend(offsets[1:]) 5247 self._indexed = True 5248 self._cached = True 5249 5250 @property 5251 def cache(self): 5252 """Return if pages/frames are currently being cached.""" 5253 return self._cache 5254 5255 @cache.setter 5256 def cache(self, value): 5257 """Enable or disable caching of pages/frames. Clear cache if False.""" 5258 value = bool(value) 5259 if self._cache and not value: 5260 self._clear() 5261 self._cache = value 5262 5263 @property 5264 def useframes(self): 5265 """Return if currently using TiffFrame (True) or TiffPage (False).""" 5266 return self._tiffpage == TiffFrame and TiffFrame is not TiffPage 5267 5268 @useframes.setter 5269 def useframes(self, value): 5270 """Set to use TiffFrame (True) or TiffPage (False).""" 5271 self._tiffpage = TiffFrame if value else TiffPage 5272 5273 @property 5274 def keyframe(self): 5275 """Return current keyframe.""" 5276 return self._keyframe 5277 5278 @keyframe.setter 5279 def keyframe(self, index): 5280 """Set current keyframe. Load TiffPage from file if necessary.""" 5281 index = int(index) 5282 if index < 0: 5283 index %= len(self) 5284 if self._keyframe.index == index: 5285 return 5286 if index == 0: 5287 self._keyframe = self.pages[0] 5288 return 5289 if self._indexed or index < len(self.pages): 5290 page = self.pages[index] 5291 if isinstance(page, TiffPage): 5292 self._keyframe = page 5293 return 5294 if isinstance(page, TiffFrame): 5295 # remove existing TiffFrame 5296 self.pages[index] = page.offset 5297 # load TiffPage from file 5298 tiffpage = self._tiffpage 5299 self._tiffpage = TiffPage 5300 try: 5301 self._keyframe = self._getitem(index) 5302 finally: 5303 self._tiffpage = tiffpage 5304 # always cache keyframes 5305 self.pages[index] = self._keyframe 5306 5307 @property 5308 def next_page_offset(self): 5309 """Return offset where offset to a new page can be stored.""" 5310 if not self._indexed: 5311 self._seek(-1) 5312 return self._nextpageoffset 5313 5314 def get(self, key, default=None, validate=False, cache=None, aspage=True): 5315 """Return specified page from cache or file.""" 5316 try: 5317 return self._getitem( 5318 key, validate=validate, cache=cache, aspage=aspage 5319 ) 5320 except IndexError: 5321 if default is None: 5322 raise 5323 return default 5324 5325 def _load(self, keyframe=True): 5326 """Read all remaining pages from file.""" 5327 if self._cached: 5328 return 5329 pages = self.pages 5330 if not pages: 5331 return 5332 if not self._indexed: 5333 self._seek(-1) 5334 if not self._cache: 5335 return 5336 fh = self.parent.filehandle 5337 if keyframe is not None: 5338 keyframe = self._keyframe 5339 for i, page in enumerate(pages): 5340 if isinstance(page, (int, numpy.integer)): 5341 pageindex = i if self._index is None else self._index + (i,) 5342 fh.seek(page) 5343 page = self._tiffpage( 5344 self.parent, index=pageindex, keyframe=keyframe 5345 ) 5346 pages[i] = page 5347 self._cached = True 5348 5349 def _load_virtual_frames(self): 5350 """Calculate virtual TiffFrames.""" 5351 pages = self.pages 5352 try: 5353 if len(pages) > 1: 5354 raise ValueError('pages already loaded') 5355 page = pages[0] 5356 if not page.is_contiguous: 5357 raise ValueError('data not contiguous') 5358 self._seek(4) 5359 delta = pages[2] - pages[1] 5360 if pages[3] - pages[2] != delta or pages[4] - pages[3] != delta: 5361 raise ValueError('page offsets not equidistant') 5362 page1 = self._getitem(1, validate=page.hash) 5363 offsetoffset = page1.dataoffsets[0] - page1.offset 5364 if offsetoffset < 0 or offsetoffset > delta: 5365 raise ValueError('page offsets not equidistant') 5366 pages = [page, page1] 5367 filesize = self.parent.filehandle.size - delta 5368 5369 for index, offset in enumerate( 5370 range(page1.offset + delta, filesize, delta) 5371 ): 5372 pageindex = index + 2 5373 d = pageindex * delta 5374 offsets = tuple(i + d for i in page.dataoffsets) 5375 offset = offset if offset < 2 ** 31 - 1 else None 5376 if self._index is not None: 5377 pageindex = self._index + (pageindex,) 5378 pages.append( 5379 TiffFrame( 5380 parent=page.parent, 5381 index=pageindex, 5382 offset=offset, 5383 offsets=offsets, 5384 bytecounts=page.databytecounts, 5385 keyframe=page, 5386 ) 5387 ) 5388 self.pages = pages 5389 self._cache = True 5390 self._cached = True 5391 self._indexed = True 5392 except Exception as exc: 5393 if self.parent.filehandle.size >= 2147483648: 5394 log_warning(f'TiffPages: failed to load virtual frames: {exc}') 5395 5396 def _clear(self, fully=True): 5397 """Delete all but first page from cache. Set keyframe to first page.""" 5398 pages = self.pages 5399 if not pages: 5400 return 5401 self._keyframe = pages[0] 5402 if fully: 5403 # delete all but first TiffPage/TiffFrame 5404 for i, page in enumerate(pages[1:]): 5405 if not isinstance(page, int) and page.offset is not None: 5406 pages[i + 1] = page.offset 5407 elif TiffFrame is not TiffPage: 5408 # delete only TiffFrames 5409 for i, page in enumerate(pages): 5410 if isinstance(page, TiffFrame) and page.offset is not None: 5411 pages[i] = page.offset 5412 self._cached = False 5413 5414 def _seek(self, index, maxpages=None): 5415 """Seek file to offset of page specified by index.""" 5416 pages = self.pages 5417 lenpages = len(pages) 5418 if lenpages == 0: 5419 raise IndexError('index out of range') 5420 5421 fh = self.parent.filehandle 5422 if fh.closed: 5423 raise ValueError('seek of closed file') 5424 5425 if self._indexed or 0 <= index < lenpages: 5426 page = pages[index] 5427 offset = page if isinstance(page, int) else page.offset 5428 fh.seek(offset) 5429 return 5430 5431 tiff = self.parent.tiff 5432 offsetformat = tiff.offsetformat 5433 offsetsize = tiff.offsetsize 5434 tagnoformat = tiff.tagnoformat 5435 tagnosize = tiff.tagnosize 5436 tagsize = tiff.tagsize 5437 unpack = struct.unpack 5438 5439 page = pages[-1] 5440 offset = page if isinstance(page, int) else page.offset 5441 5442 if maxpages is None: 5443 maxpages = 2 ** 22 5444 while lenpages < maxpages: 5445 # read offsets to pages from file until index is reached 5446 fh.seek(offset) 5447 # skip tags 5448 try: 5449 tagno = unpack(tagnoformat, fh.read(tagnosize))[0] 5450 if tagno > 4096: 5451 raise TiffFileError(f'suspicious number of tags {tagno!r}') 5452 except Exception: 5453 log_warning( 5454 'TiffPages: corrupted tag list of page ' 5455 f'{lenpages} @ {offset}' 5456 ) 5457 del pages[-1] 5458 lenpages -= 1 5459 self._indexed = True 5460 break 5461 self._nextpageoffset = offset + tagnosize + tagno * tagsize 5462 fh.seek(self._nextpageoffset) 5463 5464 # read offset to next page 5465 offset = unpack(offsetformat, fh.read(offsetsize))[0] 5466 if offset == 0: 5467 self._indexed = True 5468 break 5469 if offset >= fh.size: 5470 log_warning(f'TiffPages: invalid page offset {offset!r}') 5471 self._indexed = True 5472 break 5473 5474 pages.append(offset) 5475 lenpages += 1 5476 if 0 <= index < lenpages: 5477 break 5478 5479 # detect some circular references 5480 if lenpages == 100: 5481 for p in pages[:-1]: 5482 if offset == (p if isinstance(p, int) else p.offset): 5483 raise TiffFileError('invalid circular IFD reference') 5484 5485 if index >= lenpages: 5486 raise IndexError('index out of range') 5487 5488 page = pages[index] 5489 fh.seek(page if isinstance(page, int) else page.offset) 5490 5491 def _getlist(self, key=None, useframes=True, validate=True): 5492 """Return specified pages as list of TiffPages or TiffFrames. 5493 5494 The first item is a TiffPage, and is used as a keyframe for 5495 following TiffFrames. 5496 5497 """ 5498 getitem = self._getitem 5499 _useframes = self.useframes 5500 5501 if key is None: 5502 key = iter(range(len(self))) 5503 elif isinstance(key, Iterable): 5504 key = iter(key) 5505 elif isinstance(key, slice): 5506 start, stop, _ = key.indices(2 ** 31 - 1) 5507 if not self._indexed and max(stop, start) > len(self.pages): 5508 self._seek(-1) 5509 key = iter(range(*key.indices(len(self.pages)))) 5510 elif isinstance(key, (int, numpy.integer)): 5511 # return single TiffPage 5512 self.useframes = False 5513 if key == 0: 5514 return [self.pages[key]] 5515 try: 5516 return [getitem(key)] 5517 finally: 5518 self.useframes = _useframes 5519 else: 5520 raise TypeError('key must be an integer, slice, or iterable') 5521 5522 # use first page as keyframe 5523 keyframe = self._keyframe 5524 self.keyframe = next(key) 5525 if validate: 5526 validate = self._keyframe.hash 5527 if useframes: 5528 self.useframes = True 5529 try: 5530 pages = [getitem(i, validate) for i in key] 5531 pages.insert(0, self._keyframe) 5532 finally: 5533 # restore state 5534 self._keyframe = keyframe 5535 if useframes: 5536 self.useframes = _useframes 5537 5538 return pages 5539 5540 def _getitem(self, key, validate=False, cache=None, aspage=None): 5541 """Return specified page from cache or file.""" 5542 key = int(key) 5543 pages = self.pages 5544 5545 if key < 0: 5546 key %= len(self) 5547 elif self._indexed and key >= len(pages): 5548 raise IndexError(f'index {key} out of range({len(pages)})') 5549 5550 tiffpage = TiffPage if aspage else self._tiffpage 5551 5552 if key < len(pages): 5553 page = pages[key] 5554 if self._cache and not aspage: 5555 if not isinstance(page, (int, numpy.integer)): 5556 if validate and validate != page.hash: 5557 raise RuntimeError('page hash mismatch') 5558 return page 5559 elif isinstance(page, (TiffPage, tiffpage)): 5560 if validate and validate != page.hash: 5561 raise RuntimeError('page hash mismatch') 5562 return page 5563 5564 pageindex = key if self._index is None else self._index + (key,) 5565 self._seek(key) 5566 page = tiffpage(self.parent, index=pageindex, keyframe=self._keyframe) 5567 if validate and validate != page.hash: 5568 raise RuntimeError('page hash mismatch') 5569 if self._cache or cache: 5570 pages[key] = page 5571 return page 5572 5573 def __getitem__(self, key): 5574 """Return specified page(s).""" 5575 pages = self.pages 5576 getitem = self._getitem 5577 5578 if isinstance(key, (int, numpy.integer)): 5579 if key == 0: 5580 return pages[key] 5581 return getitem(key) 5582 5583 if isinstance(key, slice): 5584 start, stop, _ = key.indices(2 ** 31 - 1) 5585 if not self._indexed and max(stop, start) > len(pages): 5586 self._seek(-1) 5587 return [getitem(i) for i in range(*key.indices(len(pages)))] 5588 5589 if isinstance(key, Iterable): 5590 return [getitem(k) for k in key] 5591 5592 raise TypeError('key must be an integer, slice, or iterable') 5593 5594 def __iter__(self): 5595 """Return iterator over all pages.""" 5596 i = 0 5597 while True: 5598 try: 5599 yield self._getitem(i) 5600 i += 1 5601 except IndexError: 5602 break 5603 if self._cache: 5604 self._cached = True 5605 5606 def __bool__(self): 5607 """Return True if file contains any pages.""" 5608 return len(self.pages) > 0 5609 5610 def __len__(self): 5611 """Return number of pages in file.""" 5612 if not self._indexed: 5613 self._seek(-1) 5614 return len(self.pages) 5615 5616 5617class TiffPage: 5618 """TIFF image file directory (IFD). 5619 5620 Attributes 5621 ---------- 5622 index : int 5623 Index of the page in file. 5624 dtype : numpy.dtype or None 5625 Data type (native byte order) of the image in IFD. 5626 shape : tuple of int 5627 Dimensions of the image in IFD, as returned by asarray. 5628 axes : str 5629 Axes label codes for each dimension in shape: 5630 'S' sample, 5631 'X' width, 5632 'Y' length, 5633 'Z' depth, 5634 tags : TiffTags 5635 Multidict like interface to tags in IFD. 5636 colormap : numpy.ndarray 5637 Color look up table, if exists. 5638 shaped : tuple of int 5639 Normalized 5-dimensional shape of the image in IFD: 5640 0 : separate samplesperpixel or 1. 5641 1 : imagedepth Z or 1. 5642 2 : imagelength Y. 5643 3 : imagewidth X. 5644 4 : contig samplesperpixel or 1. 5645 5646 All attributes are read-only. 5647 5648 """ 5649 5650 # default properties; will be updated from tags 5651 subfiletype = 0 5652 imagewidth = 0 5653 imagelength = 0 5654 imagedepth = 1 5655 tilewidth = 0 5656 tilelength = 0 5657 tiledepth = 1 5658 bitspersample = 1 5659 samplesperpixel = 1 5660 sampleformat = 1 5661 rowsperstrip = 2 ** 32 - 1 5662 compression = 1 5663 planarconfig = 1 5664 fillorder = 1 5665 photometric = 0 5666 predictor = 1 5667 extrasamples = () 5668 subsampling = None 5669 subifds = None 5670 jpegtables = None 5671 jpegheader = None # NDPI only 5672 colormap = None 5673 software = '' 5674 description = '' 5675 description1 = '' 5676 nodata = 0 5677 5678 def __init__(self, parent, index, keyframe=None): 5679 """Initialize instance from file. 5680 5681 The file handle position must be at offset to a valid IFD. 5682 5683 """ 5684 self.parent = parent 5685 self.index = index 5686 self.shape = () 5687 self.shaped = () 5688 self.dtype = None 5689 self._dtype = None 5690 self.axes = '' 5691 self.tags = tags = TiffTags() 5692 self.dataoffsets = () 5693 self.databytecounts = () 5694 5695 tiff = parent.tiff 5696 5697 # read TIFF IFD structure and its tags from file 5698 fh = parent.filehandle 5699 self.offset = fh.tell() # offset to this IFD 5700 try: 5701 tagno = struct.unpack(tiff.tagnoformat, fh.read(tiff.tagnosize))[0] 5702 if tagno > 4096: 5703 raise ValueError(f'suspicious number of tags {tagno}') 5704 except Exception as exc: 5705 raise TiffFileError( 5706 f'TiffPage {self.index}: ' 5707 f'corrupted tag list at offset {self.offset}' 5708 ) from exc 5709 5710 tagoffset = self.offset + tiff.tagnosize # fh.tell() 5711 tagsize = tagsize_ = tiff.tagsize 5712 5713 data = fh.read(tagsize * tagno) 5714 5715 if tiff.version == 42 and tiff.offsetsize == 8: 5716 # patch offsets/values for 64-bit NDPI file 5717 tagsize = 16 5718 fh.seek(8, os.SEEK_CUR) 5719 ext = fh.read(4 * tagno) # high bits 5720 data = b''.join( 5721 data[i * 12 : i * 12 + 12] + ext[i * 4 : i * 4 + 4] 5722 for i in range(tagno) 5723 ) 5724 5725 tagindex = -tagsize 5726 for i in range(tagno): 5727 tagindex += tagsize 5728 tagdata = data[tagindex : tagindex + tagsize] 5729 try: 5730 tag = TiffTag.fromfile( 5731 parent, tagoffset + i * tagsize_, tagdata 5732 ) 5733 except TiffFileError as exc: 5734 log_warning( 5735 f'TiffPage {self.index}: {exc.__class__.__name__}: {exc}' 5736 ) 5737 continue 5738 tags.add(tag) 5739 5740 if not tags: 5741 return # found in FIBICS 5742 5743 for code, name in TIFF.TAG_ATTRIBUTES.items(): 5744 tag = tags.get(code) 5745 if tag is not None: 5746 if code in (270, 305) and not isinstance(tag.value, str): 5747 # wrong string type for software or description 5748 continue 5749 setattr(self, name, tag.value) 5750 5751 tag = tags.get(270, index=1) 5752 if tag: 5753 self.description1 = tag.value 5754 5755 if self.subfiletype == 0: 5756 tag = tags.get(255) # SubfileType 5757 if tag: 5758 if tag.value == 2: 5759 self.subfiletype = 0b1 # reduced image 5760 elif tag.value == 3: 5761 self.subfiletype = 0b10 # multi-page 5762 5763 # consolidate private tags; remove them from self.tags 5764 # if self.is_andor: 5765 # self.andor_tags 5766 # elif self.is_epics: 5767 # self.epics_tags 5768 # elif self.is_ndpi: 5769 # self.ndpi_tags 5770 # if self.is_sis and 34853 in tags: 5771 # # TODO: can't change tag.name 5772 # tags[34853].name = 'OlympusSIS2' 5773 5774 # dataoffsets and databytecounts 5775 if 324 in tags: # TileOffsets 5776 self.dataoffsets = tags[324].value 5777 elif 273 in tags: # StripOffsets 5778 self.dataoffsets = tags[273].value 5779 elif 513 in tags: # JPEGInterchangeFormat et al. 5780 self.dataoffsets = tags[513].value 5781 if 325 in tags: # TileByteCounts 5782 self.databytecounts = tags[325].value 5783 elif 279 in tags: # StripByteCounts 5784 self.databytecounts = tags[279].value 5785 elif 514 in tags: # JPEGInterchangeFormatLength et al. 5786 self.databytecounts = tags[514].value 5787 5788 if ( 5789 self.imagewidth == 0 5790 and self.imagelength == 0 5791 and self.dataoffsets 5792 and self.databytecounts 5793 ): 5794 # dimensions may be missing in some RAW formats 5795 # read dimensions from assumed JPEG encoded segment 5796 try: 5797 fh.seek(self.dataoffsets[0]) 5798 ( 5799 precision, 5800 imagelength, 5801 imagewidth, 5802 samplesperpixel, 5803 ) = jpeg_shape(fh.read(min(self.databytecounts[0], 4096))) 5804 except Exception: 5805 pass 5806 else: 5807 self.imagelength = imagelength 5808 self.imagewidth = imagewidth 5809 self.samplesperpixel = samplesperpixel 5810 if 258 not in tags: 5811 self.bitspersample = 8 if precision <= 8 else 16 5812 if 262 not in tags and samplesperpixel == 3: 5813 self.photometric = 6 # YCbCr 5814 if 259 not in tags: 5815 self.compression = 6 # OJPEG 5816 if 278 not in tags: 5817 self.rowsperstrip = imagelength 5818 5819 elif self.compression == 6: 5820 # OJPEG hack. See libtiff v4.2.0 tif_dirread.c#L4082 5821 if 262 not in tags: 5822 # PhotometricInterpretation missing 5823 self.photometric = 6 # YCbCr 5824 elif self.photometric == 2: 5825 # RGB -> YCbCr 5826 self.photometric = 6 5827 if 258 not in tags: 5828 # BitsPerSample missing 5829 self.bitspersample = 8 5830 if 277 not in tags: 5831 # SamplesPerPixel missing 5832 if self.photometric in (2, 6): 5833 self.samplesperpixel = 3 5834 elif self.photometric in (0, 1): 5835 self.samplesperpixel = 3 5836 5837 elif self.is_lsm or (self.index != 0 and self.parent.is_lsm): 5838 # correct non standard LSM bitspersample tags 5839 tags[258]._fix_lsm_bitspersample() 5840 if self.compression == 1 and self.predictor != 1: 5841 # work around bug in LSM510 software 5842 self.predictor = 1 5843 5844 elif self.is_vista or (self.index != 0 and self.parent.is_vista): 5845 # ISS Vista writes wrong ImageDepth tag 5846 self.imagedepth = 1 5847 5848 elif self.is_stk: 5849 tag = tags.get(33628) # UIC1tag 5850 if tag is not None and not tag.value: 5851 # read UIC1tag now that plane count is known 5852 fh.seek(tag.valueoffset) 5853 tag.value = read_uic1tag( 5854 fh, 5855 tiff.byteorder, 5856 tag.dtype, 5857 tag.count, 5858 None, 5859 tags[33629].count, # UIC2tag 5860 ) 5861 5862 if 50839 in tags: 5863 # decode IJMetadata tag 5864 try: 5865 tags[50839].value = imagej_metadata( 5866 tags[50839].value, 5867 tags[50838].value, # IJMetadataByteCounts 5868 tiff.byteorder, 5869 ) 5870 except Exception as exc: 5871 log_warning( 5872 f'TiffPage {self.index}: {exc.__class__.__name__}: {exc}' 5873 ) 5874 5875 # BitsPerSample 5876 tag = tags.get(258) 5877 if tag is not None: 5878 if self.bitspersample != 1: 5879 pass # bitspersample was set by ojpeg hack 5880 elif tag.count == 1: 5881 self.bitspersample = tag.value 5882 else: 5883 # LSM might list more items than samplesperpixel 5884 value = tag.value[: self.samplesperpixel] 5885 if any(v - value[0] for v in value): 5886 self.bitspersample = value 5887 else: 5888 self.bitspersample = value[0] 5889 5890 # SampleFormat 5891 tag = tags.get(339) 5892 if tag is not None: 5893 if tag.count == 1: 5894 self.sampleformat = tag.value 5895 else: 5896 value = tag.value[: self.samplesperpixel] 5897 if any(v - value[0] for v in value): 5898 self.sampleformat = value 5899 else: 5900 self.sampleformat = value[0] 5901 5902 if 322 in tags: # TileWidth 5903 self.rowsperstrip = None 5904 elif 257 in tags: # ImageLength 5905 if 278 not in tags or tags[278].count > 1: # RowsPerStrip 5906 self.rowsperstrip = self.imagelength 5907 self.rowsperstrip = min(self.rowsperstrip, self.imagelength) 5908 # self.stripsperimage = int(math.floor( 5909 # float(self.imagelength + self.rowsperstrip - 1) / 5910 # self.rowsperstrip)) 5911 5912 # determine dtype 5913 dtype = TIFF.SAMPLE_DTYPES.get( 5914 (self.sampleformat, self.bitspersample), None 5915 ) 5916 if dtype is not None: 5917 dtype = numpy.dtype(dtype) 5918 self.dtype = self._dtype = dtype 5919 5920 # determine shape of data 5921 imagelength = self.imagelength 5922 imagewidth = self.imagewidth 5923 imagedepth = self.imagedepth 5924 samplesperpixel = self.samplesperpixel 5925 5926 if self.photometric == 2 or samplesperpixel > 1: # PHOTOMETRIC.RGB 5927 if self.planarconfig == 1: 5928 self.shaped = ( 5929 1, 5930 imagedepth, 5931 imagelength, 5932 imagewidth, 5933 samplesperpixel, 5934 ) 5935 if imagedepth == 1: 5936 self.shape = (imagelength, imagewidth, samplesperpixel) 5937 self.axes = 'YXS' 5938 else: 5939 self.shape = ( 5940 imagedepth, 5941 imagelength, 5942 imagewidth, 5943 samplesperpixel, 5944 ) 5945 self.axes = 'ZYXS' 5946 else: 5947 self.shaped = ( 5948 samplesperpixel, 5949 imagedepth, 5950 imagelength, 5951 imagewidth, 5952 1, 5953 ) 5954 if imagedepth == 1: 5955 self.shape = (samplesperpixel, imagelength, imagewidth) 5956 self.axes = 'SYX' 5957 else: 5958 self.shape = ( 5959 samplesperpixel, 5960 imagedepth, 5961 imagelength, 5962 imagewidth, 5963 ) 5964 self.axes = 'SZYX' 5965 else: 5966 self.shaped = (1, imagedepth, imagelength, imagewidth, 1) 5967 if imagedepth == 1: 5968 self.shape = (imagelength, imagewidth) 5969 self.axes = 'YX' 5970 else: 5971 self.shape = (imagedepth, imagelength, imagewidth) 5972 self.axes = 'ZYX' 5973 5974 if not self.databytecounts: 5975 self.databytecounts = ( 5976 product(self.shape) * (self.bitspersample // 8), 5977 ) 5978 if self.compression != 1: 5979 log_warning( 5980 f'TiffPage {self.index}: ByteCounts tag is missing' 5981 ) 5982 5983 if imagelength and self.rowsperstrip and not self.is_lsm: 5984 # fix incorrect number of strip bytecounts and offsets 5985 maxstrips = ( 5986 int( 5987 math.floor(imagelength + self.rowsperstrip - 1) 5988 / self.rowsperstrip 5989 ) 5990 * self.imagedepth 5991 ) 5992 if self.planarconfig == 2: 5993 maxstrips *= self.samplesperpixel 5994 if maxstrips != len(self.databytecounts): 5995 log_warning( 5996 f'TiffPage {self.index}: incorrect StripByteCounts count' 5997 ) 5998 self.databytecounts = self.databytecounts[:maxstrips] 5999 if maxstrips != len(self.dataoffsets): 6000 log_warning( 6001 f'TiffPage {self.index}: incorrect StripOffsets count' 6002 ) 6003 self.dataoffsets = self.dataoffsets[:maxstrips] 6004 6005 tag = tags.get(42113) # GDAL_NODATA 6006 if tag is not None: 6007 try: 6008 pytype = type(dtype.type(0).item()) 6009 self.nodata = pytype(tag.value) 6010 except Exception: 6011 pass 6012 6013 if 65426 in tags and self.is_ndpi: 6014 # use NDPI JPEG McuStarts as tile offsets 6015 mcustarts = tags[65426].value 6016 if 65432 in tags: 6017 # McuStartsHighBytes 6018 high = tags[65432].value.astype('uint64') 6019 high <<= 32 6020 mcustarts = mcustarts.astype('uint64') 6021 mcustarts += high 6022 fh.seek(self.dataoffsets[0]) 6023 jpegheader = fh.read(mcustarts[0]) 6024 try: 6025 ( 6026 self.tilelength, 6027 self.tilewidth, 6028 self.jpegheader, 6029 ) = ndpi_jpeg_tile(jpegheader) 6030 except ValueError as exc: 6031 log_warning( 6032 f'TiffPage {self.index}: ndpi_jpeg_tile failed ({exc})' 6033 ) 6034 else: 6035 self.databytecounts = ( 6036 mcustarts[1:] - mcustarts[:-1] 6037 ).tolist() + [self.databytecounts[0] - int(mcustarts[-1])] 6038 self.dataoffsets = (mcustarts + self.dataoffsets[0]).tolist() 6039 6040 @lazyattr 6041 def decode(self): 6042 """Return decoded segment, its shape, and indices in image. 6043 6044 The decode function is implemeted as a closure. 6045 6046 Parameters 6047 ---------- 6048 data : bytes 6049 Encoded bytes of a segment (aka strile, strip or tile) 6050 or None for empty segments. 6051 index : int 6052 The index of the segment in the Offsets and Bytecount tag values. 6053 jpegtables : bytes or None 6054 For JPEG compressed segments only, the value of the JPEGTables tag 6055 if any. 6056 6057 Returns 6058 ------- 6059 segment : numpy.ndarray 6060 Decoded segment or None for empty segments. 6061 indices : tuple of int 6062 The position of the segment in the image array of normalized shape: 6063 (separate sample, depth, length, width, contig sample). 6064 shape : tuple of int 6065 The shape of the segment: (depth, length, width, contig samples). 6066 The shape of strips depends on their linear index. 6067 6068 Raises ValueError or NotImplementedError if decoding is not supported. 6069 6070 """ 6071 if self.hash in self.parent._parent._decoders: 6072 return self.parent._parent._decoders[self.hash] 6073 6074 def cache(decode): 6075 self.parent._parent._decoders[self.hash] = decode 6076 return decode 6077 6078 if self.dtype is None: 6079 6080 def decode(*args, **kwargs): 6081 raise ValueError( 6082 f'TiffPage {self.index}: data type not supported: ' 6083 f'SampleFormat {self.sampleformat}, ' 6084 f'{self.bitspersample}-bit' 6085 ) 6086 6087 return cache(decode) 6088 6089 if 0 in self.shaped: 6090 6091 def decode(*args, **kwargs): 6092 raise ValueError(f'TiffPage {self.index}: empty image') 6093 6094 return cache(decode) 6095 6096 try: 6097 if self.compression == 1: 6098 decompress = None 6099 else: 6100 decompress = TIFF.DECOMPRESSORS[self.compression] 6101 except KeyError as exc: 6102 6103 def decode(*args, exc=str(exc)[1:-1], **kwargs): 6104 raise ValueError(f'TiffPage {self.index}: {exc}') 6105 6106 return cache(decode) 6107 6108 try: 6109 if self.predictor == 1: 6110 unpredict = None 6111 else: 6112 unpredict = TIFF.UNPREDICTORS[self.predictor] 6113 except KeyError as exc: 6114 6115 def decode(*args, exc=str(exc)[1:-1], **kwargs): 6116 raise ValueError(f'TiffPage {self.index}: {exc}') 6117 6118 return cache(decode) 6119 6120 if self.tags.get(339) is not None: 6121 tag = self.tags[339] # SampleFormat 6122 if tag.count != 1 and any(i - tag.value[0] for i in tag.value): 6123 6124 def decode(*args, **kwargs): 6125 raise ValueError( 6126 f'TiffPage {self.index}: ' 6127 f'sample formats do not match {tag.value}' 6128 ) 6129 6130 return cache(decode) 6131 6132 if self.is_subsampled and ( 6133 self.compression not in (6, 7) or self.planarconfig == 2 6134 ): 6135 6136 def decode(*args, **kwargs): 6137 raise NotImplementedError( 6138 f'TiffPage {self.index}: chroma subsampling not supported' 6139 ) 6140 6141 return cache(decode) 6142 6143 # normalize segments shape to [depth, length, length, contig] 6144 if self.is_tiled: 6145 stshape = [self.tiledepth, self.tilelength, self.tilewidth, 1] 6146 else: 6147 stshape = [1, self.rowsperstrip, self.imagewidth, 1] 6148 if self.planarconfig == 1: 6149 stshape[-1] = self.samplesperpixel 6150 stshape = tuple(stshape) 6151 6152 stdepth, stlength, stwidth, samples = stshape 6153 imdepth, imlength, imwidth, samples = self.shaped[1:] 6154 6155 if self.is_tiled: 6156 6157 width = (imwidth + stwidth - 1) // stwidth 6158 length = (imlength + stlength - 1) // stlength 6159 depth = (imdepth + stdepth - 1) // stdepth 6160 6161 def indices(tileindex): 6162 # return indices and shape of tile in image array 6163 return ( 6164 ( 6165 tileindex // (width * length * depth), 6166 (tileindex // (width * length)) % depth * stdepth, 6167 (tileindex // width) % length * stlength, 6168 tileindex % width * stwidth, 6169 0, 6170 ), 6171 stshape, 6172 ) 6173 6174 def reshape(data, indices, shape): 6175 # return reshaped tile 6176 if data is None: 6177 return data 6178 size = shape[0] * shape[1] * shape[2] * shape[3] 6179 if data.size > size: 6180 # decompression / unpacking might return too many bytes 6181 data.shape = -1 6182 data = data[:size] 6183 if data.size == size: 6184 # complete tile 6185 # data might be non-contiguous; cannot reshape inplace 6186 data = data.reshape(shape) 6187 else: 6188 # data fills remaining space 6189 # found in some JPEG/PNG compressed tiles 6190 try: 6191 data = data.reshape( 6192 ( 6193 min(imdepth - indices[1], shape[0]), 6194 min(imlength - indices[2], shape[1]), 6195 min(imwidth - indices[3], shape[2]), 6196 samples, 6197 ) 6198 ) 6199 except ValueError: 6200 # incomplete tile; see gdal issue #1179 6201 log_warning( 6202 f'reshape: incomplete tile {data.shape} {shape}' 6203 ) 6204 t = numpy.zeros(size, data.dtype) 6205 size = min(data.size, size) 6206 t[:size] = data[:size] 6207 data = t.reshape(shape) 6208 return data 6209 6210 def pad(data, shape, nodata=self.nodata): 6211 # pad tile to shape 6212 if data is None or data.shape == shape: 6213 return data, shape 6214 padwidth = [(0, i - j) for i, j in zip(shape, data.shape)] 6215 data = numpy.pad(data, padwidth, constant_values=nodata) 6216 return data, data.shape 6217 6218 else: 6219 # strips 6220 length = (imlength + stlength - 1) // stlength 6221 6222 def indices(stripindex): 6223 # return indices and shape of strip in image array 6224 indices = ( 6225 stripindex // (length * imdepth), 6226 (stripindex // length) % imdepth * stdepth, 6227 stripindex % length * stlength, 6228 0, 6229 0, 6230 ) 6231 shape = ( 6232 stdepth, 6233 min(stlength, imlength - indices[2]), 6234 stwidth, 6235 samples, 6236 ) 6237 return indices, shape 6238 6239 def reshape(data, indices, shape): 6240 # return reshaped strip 6241 if data is None: 6242 return data 6243 size = shape[0] * shape[1] * shape[2] * shape[3] 6244 if data.size > size: 6245 # decompression / unpacking might return too many bytes 6246 data.shape = -1 6247 data = data[:size] 6248 if data.size == size: 6249 # expected size 6250 data.shape = shape 6251 else: 6252 # should not happen, but try different length 6253 data.shape = shape[0], -1, shape[2], shape[3] 6254 # raise RuntimeError( 6255 # f'invalid strip shape {data.shape} or size {size}' 6256 # ) 6257 return data 6258 6259 def pad(data, shape, nodata=self.nodata): 6260 # pad strip length to rowsperstrip 6261 shape = (shape[0], stlength, shape[2], shape[3]) 6262 if data is None or data.shape == shape: 6263 return data, shape 6264 padwidth = [ 6265 (0, 0), 6266 (0, stlength - data.shape[1]), 6267 (0, 0), 6268 (0, 0), 6269 ] 6270 data = numpy.pad(data, padwidth, constant_values=nodata) 6271 return data, data.shape 6272 6273 if self.compression in (6, 7, 34892, 33007): 6274 # JPEG needs special handling 6275 if self.fillorder == 2: 6276 log_warning( 6277 f'TiffPage {self.index}: disabling LSB2MSB for JPEG' 6278 ) 6279 if unpredict: 6280 log_warning( 6281 f'TiffPage {self.index}: disabling predictor for JPEG' 6282 ) 6283 if 28672 in self.tags: # SonyRawFileType 6284 log_warning( 6285 f'TiffPage {self.index}: ' 6286 'SonyRawFileType might need additional unpacking ' 6287 '(see issue #95)' 6288 ) 6289 6290 colorspace, outcolorspace = jpeg_decode_colorspace( 6291 self.photometric, self.planarconfig, self.extrasamples 6292 ) 6293 6294 def decode( 6295 data, 6296 segmentindex, 6297 jpegtables=None, 6298 jpegheader=None, 6299 _fullsize=False, 6300 bitspersample=self.bitspersample, 6301 colorspace=colorspace, 6302 outcolorspace=outcolorspace, 6303 ): 6304 # return decoded segment, its shape, and indices in image 6305 index, shape = indices(segmentindex) 6306 if data is None: 6307 if _fullsize: 6308 data, shape = pad(data, shape) 6309 return data, index, shape 6310 data = imagecodecs.jpeg_decode( 6311 data, 6312 bitspersample=bitspersample, 6313 tables=jpegtables, 6314 header=jpegheader, 6315 colorspace=colorspace, 6316 outcolorspace=outcolorspace, 6317 shape=shape[1:3], 6318 ) 6319 data = reshape(data, index, shape) 6320 if _fullsize: 6321 data, shape = pad(data, shape) 6322 return data, index, shape 6323 6324 return cache(decode) 6325 6326 if self.compression in ( 6327 33003, 6328 33004, 6329 33005, 6330 34712, 6331 34933, 6332 34934, 6333 22610, 6334 50001, 6335 50002, 6336 ): 6337 # JPEG2000, WEBP, PNG, JPEGXR 6338 # presume codecs always return correct dtype, native byte order... 6339 if self.fillorder == 2: 6340 log_warning( 6341 f'TiffPage {self.index}: disabling LSB2MSB for ' 6342 f'compression {self.compression}' 6343 ) 6344 if unpredict: 6345 log_warning( 6346 f'TiffPage {self.index}: disabling predictor for ' 6347 f'compression {self.compression}' 6348 ) 6349 6350 def decode(data, segmentindex, jpegtables=None, _fullsize=False): 6351 # return decoded segment, its shape, and indices in image 6352 index, shape = indices(segmentindex) 6353 if data is None: 6354 if _fullsize: 6355 data, shape = pad(data, shape) 6356 return data, index, shape 6357 data = decompress(data) 6358 data = reshape(data, index, shape) 6359 if _fullsize: 6360 data, shape = pad(data, shape) 6361 return data, index, shape 6362 6363 return cache(decode) 6364 6365 dtype = numpy.dtype(self.parent.byteorder + self._dtype.char) 6366 6367 if self.sampleformat == 5: 6368 # complex integer 6369 if unpredict is not None: 6370 raise NotImplementedError( 6371 'unpredicting complex integers not supported' 6372 ) 6373 6374 itype = numpy.dtype( 6375 f'{self.parent.byteorder}i{self.bitspersample // 16}' 6376 ) 6377 ftype = numpy.dtype( 6378 f'{self.parent.byteorder}f{dtype.itemsize // 2}' 6379 ) 6380 6381 def unpack(data): 6382 # return complex integer as numpy.complex 6383 data = numpy.frombuffer(data, itype) 6384 return data.astype(ftype).view(dtype) 6385 6386 elif self.bitspersample in (8, 16, 32, 64, 128): 6387 # regular data types 6388 6389 if (self.bitspersample * stwidth * samples) % 8: 6390 raise ValueError( 6391 f'TiffPage {self.index}: data and sample size mismatch' 6392 ) 6393 if self.predictor == 3: # PREDICTOR.FLOATINGPOINT 6394 # floating-point horizontal differencing decoder needs 6395 # raw byte order 6396 dtype = numpy.dtype(self._dtype.char) 6397 6398 def unpack(data): 6399 # return numpy array from buffer 6400 try: 6401 # read only numpy array 6402 return numpy.frombuffer(data, dtype) 6403 except ValueError: 6404 # e.g. LZW strips may be missing EOI 6405 bps = self.bitspersample // 8 6406 size = (len(data) // bps) * bps 6407 return numpy.frombuffer(data[:size], dtype) 6408 6409 elif isinstance(self.bitspersample, tuple): 6410 # e.g. RGB 565 6411 def unpack(data): 6412 # return numpy array from packed integers 6413 return unpack_rgb(data, dtype, self.bitspersample) 6414 6415 elif self.bitspersample == 24 and dtype.char == 'f': 6416 # float24 6417 if unpredict is not None: 6418 # floatpred_decode requires numpy.float24, which does not exist 6419 raise NotImplementedError('unpredicting float24 not supported') 6420 6421 def unpack(data, byteorder=self.parent.byteorder): 6422 # return numpy.float32 array from float24 6423 return float24_decode(data, byteorder) 6424 6425 else: 6426 # bilevel and packed integers 6427 def unpack(data): 6428 # return numpy array from packed integers 6429 return packints_decode( 6430 data, dtype, self.bitspersample, stwidth * samples 6431 ) 6432 6433 def decode(data, segmentindex, jpegtables=None, _fullsize=False): 6434 # return decoded segment, its shape, and indices in image 6435 index, shape = indices(segmentindex) 6436 if data is None: 6437 if _fullsize: 6438 data, shape = pad(data, shape) 6439 return data, index, shape 6440 if self.fillorder == 2: 6441 data = bitorder_decode(data) 6442 if decompress is not None: 6443 # TODO: calculate correct size for packed integers 6444 size = shape[0] * shape[1] * shape[2] * shape[3] 6445 data = decompress(data, out=size * dtype.itemsize) 6446 data = unpack(data) 6447 data = reshape(data, index, shape) 6448 data = data.astype('=' + dtype.char) 6449 if unpredict is not None: 6450 # unpredict is faster with native byte order 6451 data = unpredict(data, axis=-2, out=data) 6452 if _fullsize: 6453 data, shape = pad(data, shape) 6454 return data, index, shape 6455 6456 return cache(decode) 6457 6458 def segments( 6459 self, lock=None, maxworkers=None, func=None, sort=False, _fullsize=None 6460 ): 6461 """Return iterator over decoded segments in TiffPage. 6462 6463 See the decode function for return values. 6464 6465 """ 6466 keyframe = self.keyframe # self or keyframe 6467 fh = self.parent.filehandle 6468 if lock is None: 6469 lock = fh.lock 6470 if _fullsize is None: 6471 _fullsize = keyframe.is_tiled 6472 6473 decodeargs = {'_fullsize': bool(_fullsize)} 6474 if keyframe.compression in (6, 7, 34892, 33007): # JPEG 6475 decodeargs['jpegtables'] = self.jpegtables 6476 decodeargs['jpegheader'] = keyframe.jpegheader 6477 6478 if func is None: 6479 6480 def decode(args, decodeargs=decodeargs, keyframe=keyframe): 6481 return keyframe.decode(*args, **decodeargs) 6482 6483 else: 6484 6485 def decode( 6486 args, decodeargs=decodeargs, keyframe=keyframe, func=func 6487 ): 6488 return func(keyframe.decode(*args, **decodeargs)) 6489 6490 if maxworkers is None or maxworkers < 1: 6491 maxworkers = keyframe.maxworkers 6492 if maxworkers < 2: 6493 for segment in fh.read_segments( 6494 self.dataoffsets, 6495 self.databytecounts, 6496 lock=lock, 6497 sort=sort, 6498 flat=True, 6499 ): 6500 yield decode(segment) 6501 else: 6502 # reduce memory overhead by processing chunks of up to 6503 # ~64 MB of segments because ThreadPoolExecutor.map is not 6504 # collecting iterables lazily 6505 with ThreadPoolExecutor(maxworkers) as executor: 6506 for segments in fh.read_segments( 6507 self.dataoffsets, 6508 self.databytecounts, 6509 lock=lock, 6510 sort=sort, 6511 flat=False, 6512 ): 6513 yield from executor.map(decode, segments) 6514 6515 def asarray(self, out=None, squeeze=True, lock=None, maxworkers=None): 6516 """Read image data from file and return as numpy array. 6517 6518 Raise ValueError if format is not supported. 6519 6520 Parameters 6521 ---------- 6522 out : numpy.ndarray, str, or file-like object 6523 Buffer where image data are saved. 6524 If None (default), a new array is created. 6525 If numpy.ndarray, a writable array of compatible dtype and shape. 6526 If 'memmap', directly memory-map the image data in the TIFF file 6527 if possible; else create a memory-mapped array in a temporary file. 6528 If str or open file, the file name or file object used to 6529 create a memory-map to an array stored in a binary file on disk. 6530 squeeze : bool 6531 If True (default), all length-1 dimensions (except X and Y) are 6532 squeezed out from the array. 6533 If False, the shape of the returned array is the normalized 6534 5-dimensional shape (TiffPage.shaped). 6535 lock : {RLock, NullContext} 6536 A reentrant lock used to synchronize seeks and reads from file. 6537 If None (default), the lock of the parent's filehandle is used. 6538 maxworkers : int or None 6539 Maximum number of threads to concurrently decode strips ot tiles. 6540 If None (default), up to half the CPU cores are used. 6541 See remarks in TiffFile.asarray. 6542 6543 Returns 6544 ------- 6545 numpy.ndarray 6546 Numpy array of decompressed, unpredicted, and unpacked image data 6547 read from Strip/Tile Offsets/ByteCounts, formatted according to 6548 shape and dtype metadata found in tags and parameters. 6549 Photometric conversion, pre-multiplied alpha, orientation, and 6550 colorimetry corrections are not applied. Specifically, CMYK images 6551 are not converted to RGB, MinIsWhite images are not inverted, 6552 and color palettes are not applied. Exception are YCbCr JPEG 6553 compressed images, which are converted to RGB. 6554 6555 """ 6556 keyframe = self.keyframe # self or keyframe 6557 6558 if not keyframe.shaped or product(keyframe.shaped) == 0: 6559 return None 6560 6561 fh = self.parent.filehandle 6562 if lock is None: 6563 lock = fh.lock 6564 with lock: 6565 closed = fh.closed 6566 if closed: 6567 fh.open() 6568 6569 if ( 6570 isinstance(out, str) 6571 and out == 'memmap' 6572 and keyframe.is_memmappable 6573 ): 6574 # direct memory map array in file 6575 with lock: 6576 result = fh.memmap_array( 6577 keyframe.parent.byteorder + keyframe._dtype.char, 6578 keyframe.shaped, 6579 offset=self.dataoffsets[0], 6580 ) 6581 6582 elif keyframe.is_contiguous: 6583 # read contiguous bytes to array 6584 if keyframe.is_subsampled: 6585 raise NotImplementedError( 6586 f'TiffPage {self.index}: chroma subsampling not supported' 6587 ) 6588 if out is not None: 6589 out = create_output(out, keyframe.shaped, keyframe._dtype) 6590 with lock: 6591 fh.seek(self.dataoffsets[0]) 6592 result = fh.read_array( 6593 keyframe.parent.byteorder + keyframe._dtype.char, 6594 product(keyframe.shaped), 6595 out=out, 6596 ) 6597 if keyframe.fillorder == 2: 6598 bitorder_decode(result, out=result) 6599 if keyframe.predictor != 1: 6600 # predictors without compression 6601 unpredict = TIFF.UNPREDICTORS[keyframe.predictor] 6602 if keyframe.predictor == 1: 6603 unpredict(result, axis=-2, out=result) 6604 else: 6605 # floatpred cannot decode in-place 6606 out = unpredict(result, axis=-2, out=result) 6607 result[:] = out 6608 6609 elif ( 6610 keyframe.jpegheader is not None 6611 and keyframe is self 6612 and 273 in self.tags # striped ... 6613 and self.is_tiled # but reported as tiled 6614 and self.imagewidth <= 65500 6615 and self.imagelength <= 65500 6616 ): 6617 # decode the whole NDPI JPEG strip 6618 with lock: 6619 fh.seek(self.tags[273].value[0]) # StripOffsets 6620 data = fh.read(self.tags[279].value[0]) # StripByteCounts 6621 decompress = TIFF.DECOMPRESSORS[self.compression] 6622 result = decompress( 6623 data, bitspersample=self.bitspersample, out=out 6624 ) 6625 del data 6626 6627 else: 6628 # decode individual strips or tiles 6629 result = create_output(out, keyframe.shaped, keyframe._dtype) 6630 keyframe.decode # init TiffPage.decode function 6631 6632 def func(decoderesult, keyframe=keyframe, out=out): 6633 # copy decoded segments to output array 6634 segment, (s, d, l, w, _), shape = decoderesult 6635 if segment is None: 6636 segment = keyframe.nodata 6637 else: 6638 segment = segment[ 6639 : keyframe.imagedepth - d, 6640 : keyframe.imagelength - l, 6641 : keyframe.imagewidth - w, 6642 ] 6643 result[ 6644 s, d : d + shape[0], l : l + shape[1], w : w + shape[2] 6645 ] = segment 6646 # except IndexError: 6647 # pass # corrupted files e.g. with too many strips 6648 6649 for _ in self.segments( 6650 func=func, 6651 lock=lock, 6652 maxworkers=maxworkers, 6653 sort=True, 6654 _fullsize=False, 6655 ): 6656 pass 6657 6658 result.shape = keyframe.shaped 6659 if squeeze: 6660 try: 6661 result.shape = keyframe.shape 6662 except ValueError: 6663 log_warning( 6664 f'TiffPage {self.index}: ' 6665 f'failed to reshape {result.shape} to {keyframe.shape}' 6666 ) 6667 6668 if closed: 6669 # TODO: file should remain open if an exception occurred above 6670 fh.close() 6671 return result 6672 6673 def aszarr(self, **kwargs): 6674 """Return image data as zarr storage.""" 6675 return ZarrTiffStore(self, **kwargs) 6676 6677 def asrgb( 6678 self, 6679 uint8=False, 6680 alpha=None, 6681 colormap=None, 6682 dmin=None, 6683 dmax=None, 6684 **kwargs, 6685 ): 6686 """Return image data as RGB(A). 6687 6688 Work in progress. 6689 6690 """ 6691 data = self.asarray(**kwargs) 6692 keyframe = self.keyframe # self or keyframe 6693 6694 if keyframe.photometric == TIFF.PHOTOMETRIC.PALETTE: 6695 colormap = keyframe.colormap 6696 if ( 6697 colormap.shape[1] < 2 ** keyframe.bitspersample 6698 or keyframe.dtype.char not in 'BH' 6699 ): 6700 raise ValueError( 6701 f'TiffPage {self.index}: cannot apply colormap' 6702 ) 6703 if uint8: 6704 if colormap.max() > 255: 6705 colormap >>= 8 6706 colormap = colormap.astype('uint8') 6707 if 'S' in keyframe.axes: 6708 data = data[..., 0] if keyframe.planarconfig == 1 else data[0] 6709 data = apply_colormap(data, colormap) 6710 6711 elif keyframe.photometric == TIFF.PHOTOMETRIC.RGB: 6712 if keyframe.extrasamples: 6713 if alpha is None: 6714 alpha = TIFF.EXTRASAMPLE 6715 for i, exs in enumerate(keyframe.extrasamples): 6716 if exs in alpha: 6717 if keyframe.planarconfig == 1: 6718 data = data[..., [0, 1, 2, 3 + i]] 6719 else: 6720 data = data[:, [0, 1, 2, 3 + i]] 6721 break 6722 else: 6723 if keyframe.planarconfig == 1: 6724 data = data[..., :3] 6725 else: 6726 data = data[:, :3] 6727 # TODO: convert to uint8? 6728 6729 elif keyframe.photometric == TIFF.PHOTOMETRIC.MINISBLACK: 6730 raise NotImplementedError 6731 elif keyframe.photometric == TIFF.PHOTOMETRIC.MINISWHITE: 6732 raise NotImplementedError 6733 elif keyframe.photometric == TIFF.PHOTOMETRIC.SEPARATED: 6734 raise NotImplementedError 6735 else: 6736 raise NotImplementedError 6737 return data 6738 6739 def _gettags(self, codes=None, lock=None): 6740 """Return list of (code, TiffTag).""" 6741 return [ 6742 (tag.code, tag) 6743 for tag in self.tags 6744 if codes is None or tag.code in codes 6745 ] 6746 6747 def _nextifd(self): 6748 """Return offset to next IFD from file.""" 6749 fh = self.parent.filehandle 6750 tiff = self.parent.tiff 6751 fh.seek(self.offset) 6752 tagno = struct.unpack(tiff.tagnoformat, fh.read(tiff.tagnosize))[0] 6753 fh.seek(self.offset + tiff.tagnosize + tagno * tiff.tagsize) 6754 return struct.unpack(tiff.offsetformat, fh.read(tiff.offsetsize))[0] 6755 6756 def aspage(self): 6757 """Return self.""" 6758 return self 6759 6760 @property 6761 def keyframe(self): 6762 """Return keyframe, self.""" 6763 return self 6764 6765 @keyframe.setter 6766 def keyframe(self, index): 6767 """Set keyframe, NOP.""" 6768 return 6769 6770 @property 6771 def ndim(self): 6772 """Return number of array dimensions.""" 6773 return len(self.shape) 6774 6775 @property 6776 def size(self): 6777 """Return number of elements in array.""" 6778 return product(self.shape) 6779 6780 @property 6781 def nbytes(self): 6782 """Return number of bytes in array.""" 6783 return product(self.shape) * self.dtype.itemsize 6784 6785 @lazyattr 6786 def chunks(self): 6787 """Return shape of tiles or stripes.""" 6788 shape = [] 6789 if self.tiledepth > 1: 6790 shape.append(self.tiledepth) 6791 if self.is_tiled: 6792 shape.extend((self.tilelength, self.tilewidth)) 6793 else: 6794 shape.extend((self.rowsperstrip, self.imagewidth)) 6795 if self.planarconfig == 1 and self.samplesperpixel > 1: 6796 shape.append(self.samplesperpixel) 6797 return tuple(shape) 6798 6799 @lazyattr 6800 def chunked(self): 6801 """Return shape of chunked image.""" 6802 shape = [] 6803 if self.planarconfig == 2 and self.samplesperpixel > 1: 6804 shape.append(self.samplesperpixel) 6805 if self.is_tiled: 6806 if self.imagedepth > 1: 6807 shape.append( 6808 (self.imagedepth + self.tiledepth - 1) // self.tiledepth 6809 ) 6810 shape.append( 6811 (self.imagelength + self.tilelength - 1) // self.tilelength 6812 ) 6813 shape.append( 6814 (self.imagewidth + self.tilewidth - 1) // self.tilewidth 6815 ) 6816 else: 6817 if self.imagedepth > 1: 6818 shape.append(self.imagedepth) 6819 shape.append( 6820 (self.imagelength + self.rowsperstrip - 1) // self.rowsperstrip 6821 ) 6822 shape.append(1) 6823 if self.planarconfig == 1 and self.samplesperpixel > 1: 6824 shape.append(1) 6825 return tuple(shape) 6826 6827 @lazyattr 6828 def hash(self): 6829 """Return checksum to identify pages in same series. 6830 6831 Pages with the same hash can use the same decode function. 6832 6833 """ 6834 return hash( 6835 self.shaped 6836 + ( 6837 self.parent.byteorder, 6838 self.tilewidth, 6839 self.tilelength, 6840 self.tiledepth, 6841 self.sampleformat, 6842 self.bitspersample, 6843 self.rowsperstrip, 6844 self.fillorder, 6845 self.predictor, 6846 self.extrasamples, 6847 self.photometric, 6848 self.planarconfig, 6849 self.compression, 6850 ) 6851 ) 6852 6853 @lazyattr 6854 def pages(self): 6855 """Return sequence of sub-pages, SubIFDs.""" 6856 if 330 not in self.tags: 6857 return () 6858 return TiffPages(self, index=self.index) 6859 6860 @lazyattr 6861 def maxworkers(self): 6862 """Return maximum number of threads for decoding segments. 6863 6864 Return 0 to disable multi-threading also for stacking pages. 6865 6866 """ 6867 if self.is_contiguous or self.dtype is None: 6868 return 0 6869 if self.compression in ( 6870 6, 6871 7, 6872 33003, 6873 33004, 6874 33005, 6875 33007, 6876 34712, 6877 34892, 6878 34933, 6879 34934, 6880 22610, 6881 50001, 6882 50002, 6883 ): 6884 # image codecs 6885 return min(TIFF.MAXWORKERS, len(self.dataoffsets)) 6886 bytecount = product(self.chunks) * self.dtype.itemsize 6887 if bytecount < 2048: 6888 # disable multi-threading for small segments 6889 return 0 6890 if self.compression != 1 or self.fillorder != 1 or self.predictor != 1: 6891 if self.compression == 5 and bytecount < 16384: 6892 # disable multi-threading for small LZW compressed segments 6893 return 0 6894 if len(self.dataoffsets) < 4: 6895 return 1 6896 if self.compression != 1 or self.fillorder != 1 or self.predictor != 1: 6897 if imagecodecs is not None: 6898 return min(TIFF.MAXWORKERS, len(self.dataoffsets)) 6899 return 2 # optimum for large number of uncompressed tiles 6900 6901 @lazyattr 6902 def is_contiguous(self): 6903 """Return offset and size of contiguous data, else None. 6904 6905 Excludes prediction and fill_order. 6906 6907 """ 6908 if self.sampleformat == 5: 6909 return None 6910 if self.compression != 1 or self.bitspersample not in (8, 16, 32, 64): 6911 return None 6912 if 322 in self.tags: # TileWidth 6913 if ( 6914 self.imagewidth != self.tilewidth 6915 or self.imagelength % self.tilelength 6916 or self.tilewidth % 16 6917 or self.tilelength % 16 6918 ): 6919 return None 6920 if ( 6921 32997 in self.tags 6922 and 32998 in self.tags # ImageDepth 6923 and ( # TileDepth 6924 self.imagelength != self.tilelength 6925 or self.imagedepth % self.tiledepth 6926 ) 6927 ): 6928 return None 6929 offsets = self.dataoffsets 6930 bytecounts = self.databytecounts 6931 if len(offsets) == 1: 6932 return offsets[0], bytecounts[0] 6933 if self.is_stk or self.is_lsm: 6934 return offsets[0], sum(bytecounts) 6935 if all( 6936 bytecounts[i] != 0 and offsets[i] + bytecounts[i] == offsets[i + 1] 6937 for i in range(len(offsets) - 1) 6938 ): 6939 return offsets[0], sum(bytecounts) 6940 return None 6941 6942 @lazyattr 6943 def is_final(self): 6944 """Return if page's image data are stored in final form. 6945 6946 Excludes byte-swapping. 6947 6948 """ 6949 return ( 6950 self.is_contiguous 6951 and self.fillorder == 1 6952 and self.predictor == 1 6953 and not self.is_subsampled 6954 ) 6955 6956 @lazyattr 6957 def is_memmappable(self): 6958 """Return if page's image data in file can be memory-mapped.""" 6959 return ( 6960 self.parent.filehandle.is_file 6961 and self.is_final 6962 # and (self.bitspersample == 8 or self.parent.isnative) 6963 # aligned? 6964 and self.is_contiguous[0] % self.dtype.itemsize == 0 6965 ) 6966 6967 def __str__(self, detail=0, width=79): 6968 """Return string containing information about TiffPage.""" 6969 if self.keyframe != self: 6970 return TiffFrame.__str__(self, detail, width) 6971 attr = '' 6972 for name in ('memmappable', 'final', 'contiguous'): 6973 attr = getattr(self, 'is_' + name) 6974 if attr: 6975 attr = name.upper() 6976 break 6977 6978 def tostr(name, skip=1): 6979 obj = getattr(self, name) 6980 try: 6981 value = getattr(obj, 'name') 6982 except AttributeError: 6983 return '' 6984 if obj != skip: 6985 return value 6986 return '' 6987 6988 info = ' '.join( 6989 s.lower() 6990 for s in ( 6991 'x'.join(str(i) for i in self.shape), 6992 '{}{}'.format( 6993 TIFF.SAMPLEFORMAT(self.sampleformat).name, 6994 self.bitspersample, 6995 ), 6996 ' '.join( 6997 i 6998 for i in ( 6999 TIFF.PHOTOMETRIC(self.photometric).name, 7000 'REDUCED' if self.is_reduced else '', 7001 'MASK' if self.is_mask else '', 7002 'TILED' if self.is_tiled else '', 7003 tostr('compression'), 7004 tostr('planarconfig'), 7005 tostr('predictor'), 7006 tostr('fillorder'), 7007 ) 7008 + tuple(f.upper() for f in self.flags) 7009 + (attr,) 7010 if i 7011 ), 7012 ) 7013 if s 7014 ) 7015 info = f'TiffPage {self.index} @{self.offset} {info}' 7016 if detail <= 0: 7017 return info 7018 info = [info, self.tags.__str__(detail + 1, width=width)] 7019 if detail > 1: 7020 for name in ('ndpi',): 7021 name = name + '_tags' 7022 attr = getattr(self, name, False) 7023 if attr: 7024 info.append( 7025 '{}\n{}'.format( 7026 name.upper(), 7027 pformat(attr, width=width, height=detail * 8), 7028 ) 7029 ) 7030 if detail > 3: 7031 try: 7032 info.append( 7033 'DATA\n{}'.format( 7034 pformat(self.asarray(), width=width, height=detail * 8) 7035 ) 7036 ) 7037 except Exception: 7038 pass 7039 return '\n\n'.join(info) 7040 7041 @lazyattr 7042 def flags(self): 7043 """Return set of flags.""" 7044 return { 7045 name.lower() 7046 for name in sorted(TIFF.FILE_FLAGS) 7047 if getattr(self, 'is_' + name) 7048 } 7049 7050 @lazyattr 7051 def andor_tags(self): 7052 """Return consolidated metadata from Andor tags as dict.""" 7053 if not self.is_andor: 7054 return None 7055 result = {'Id': self.tags[4864].value} # AndorId 7056 for tag in self.tags: # list(self.tags.values()): 7057 code = tag.code 7058 if not 4864 < code < 5031: 7059 continue 7060 name = tag.name 7061 name = name[5:] if len(name) > 5 else name 7062 result[name] = tag.value 7063 # del self.tags[code] 7064 return result 7065 7066 @lazyattr 7067 def epics_tags(self): 7068 """Return consolidated metadata from EPICS areaDetector tags as dict. 7069 7070 Use epics_datetime() to get a datetime object from the epicsTSSec and 7071 epicsTSNsec tags. 7072 7073 """ 7074 if not self.is_epics: 7075 return None 7076 result = {} 7077 for tag in self.tags: # list(self.tags.values()): 7078 code = tag.code 7079 if not 65000 <= code < 65500: 7080 continue 7081 value = tag.value 7082 if code == 65000: 7083 # not a POSIX timestamp 7084 # https://github.com/bluesky/area-detector-handlers/issues/20 7085 result['timeStamp'] = float(value) 7086 elif code == 65001: 7087 result['uniqueID'] = int(value) 7088 elif code == 65002: 7089 result['epicsTSSec'] = int(value) 7090 elif code == 65003: 7091 result['epicsTSNsec'] = int(value) 7092 else: 7093 key, value = value.split(':', 1) 7094 result[key] = astype(value) 7095 # del self.tags[code] 7096 return result 7097 7098 @lazyattr 7099 def ndpi_tags(self): 7100 """Return consolidated metadata from Hamamatsu NDPI as dict.""" 7101 # TODO: parse 65449 ini style comments 7102 if not self.is_ndpi: 7103 return None 7104 tags = self.tags 7105 result = {} 7106 for name in ('Make', 'Model', 'Software'): 7107 result[name] = tags[name].value 7108 for code, name in TIFF.NDPI_TAGS.items(): 7109 if code in tags: 7110 result[name] = tags[code].value 7111 # del tags[code] 7112 if 'McuStarts' in result: 7113 mcustarts = result['McuStarts'] 7114 if 'McuStartsHighBytes' in result: 7115 high = result['McuStartsHighBytes'].astype('uint64') 7116 high <<= 32 7117 mcustarts = mcustarts.astype('uint64') 7118 mcustarts += high 7119 del result['McuStartsHighBytes'] 7120 result['McuStarts'] = mcustarts 7121 return result 7122 7123 @lazyattr 7124 def geotiff_tags(self): 7125 """Return consolidated metadata from GeoTIFF tags as dict.""" 7126 if not self.is_geotiff: 7127 return None 7128 tags = self.tags 7129 7130 gkd = tags[34735].value # GeoKeyDirectoryTag 7131 if gkd[0] != 1: 7132 log_warning('GeoTIFF tags: invalid GeoKeyDirectoryTag') 7133 return {} 7134 7135 result = { 7136 'KeyDirectoryVersion': gkd[0], 7137 'KeyRevision': gkd[1], 7138 'KeyRevisionMinor': gkd[2], 7139 # 'NumberOfKeys': gkd[3], 7140 } 7141 # deltags = ['GeoKeyDirectoryTag'] 7142 geokeys = TIFF.GEO_KEYS 7143 geocodes = TIFF.GEO_CODES 7144 for index in range(gkd[3]): 7145 try: 7146 keyid, tagid, count, offset = gkd[ 7147 4 + index * 4 : index * 4 + 8 7148 ] 7149 except Exception as exc: 7150 log_warning(f'GeoTIFF tags: {exc}') 7151 continue 7152 if tagid == 0: 7153 value = offset 7154 else: 7155 try: 7156 value = tags[tagid].value[offset : offset + count] 7157 except KeyError: 7158 log_warning(f'GeoTIFF tags: {tagid} not found') 7159 continue 7160 if tagid == 34737 and count > 1 and value[-1] == '|': 7161 value = value[:-1] 7162 value = value if count > 1 else value[0] 7163 if keyid in geocodes: 7164 try: 7165 value = geocodes[keyid](value) 7166 except Exception: 7167 pass 7168 try: 7169 key = geokeys(keyid).name 7170 except ValueError: 7171 key = keyid 7172 result[key] = value 7173 7174 tag = tags.get(33920) # IntergraphMatrixTag 7175 if tag is not None: 7176 value = numpy.array(tag.value) 7177 if len(value) == 16: 7178 value = value.reshape((4, 4)).tolist() 7179 result['IntergraphMatrix'] = value 7180 7181 tag = tags.get(33550) # ModelPixelScaleTag 7182 if tag is not None: 7183 result['ModelPixelScale'] = numpy.array(tag.value).tolist() 7184 7185 tag = tags.get(33922) # ModelTiepointTag 7186 if tag is not None: 7187 value = numpy.array(tag.value).reshape((-1, 6)).squeeze().tolist() 7188 result['ModelTiepoint'] = value 7189 7190 tag = tags.get(34264) # ModelTransformationTag 7191 if tag is not None: 7192 value = numpy.array(tag.value).reshape((4, 4)).tolist() 7193 result['ModelTransformation'] = value 7194 7195 # if 33550 in tags and 33922 in tags: 7196 # sx, sy, sz = tags[33550].value # ModelPixelScaleTag 7197 # tiepoints = tags[33922].value # ModelTiepointTag 7198 # transforms = [] 7199 # for tp in range(0, len(tiepoints), 6): 7200 # i, j, k, x, y, z = tiepoints[tp : tp + 6] 7201 # transforms.append( 7202 # [ 7203 # [sx, 0.0, 0.0, x - i * sx], 7204 # [0.0, -sy, 0.0, y + j * sy], 7205 # [0.0, 0.0, sz, z - k * sz], 7206 # [0.0, 0.0, 0.0, 1.0], 7207 # ] 7208 # ) 7209 # if len(tiepoints) == 6: 7210 # transforms = transforms[0] 7211 # result['ModelTransformation'] = transforms 7212 7213 tag = tags.get(50844) # RPCCoefficientTag 7214 if tag is not None: 7215 rpcc = tag.value 7216 result['RPCCoefficient'] = { 7217 'ERR_BIAS': rpcc[0], 7218 'ERR_RAND': rpcc[1], 7219 'LINE_OFF': rpcc[2], 7220 'SAMP_OFF': rpcc[3], 7221 'LAT_OFF': rpcc[4], 7222 'LONG_OFF': rpcc[5], 7223 'HEIGHT_OFF': rpcc[6], 7224 'LINE_SCALE': rpcc[7], 7225 'SAMP_SCALE': rpcc[8], 7226 'LAT_SCALE': rpcc[9], 7227 'LONG_SCALE': rpcc[10], 7228 'HEIGHT_SCALE': rpcc[11], 7229 'LINE_NUM_COEFF': rpcc[12:33], 7230 'LINE_DEN_COEFF ': rpcc[33:53], 7231 'SAMP_NUM_COEFF': rpcc[53:73], 7232 'SAMP_DEN_COEFF': rpcc[73:], 7233 } 7234 return result 7235 7236 @property 7237 def is_reduced(self): 7238 """Page is reduced image of another image.""" 7239 return self.subfiletype & 0b1 7240 7241 @property 7242 def is_multipage(self): 7243 """Page is part of multi-page image.""" 7244 return self.subfiletype & 0b10 7245 7246 @property 7247 def is_mask(self): 7248 """Page is transparency mask for another image.""" 7249 return self.subfiletype & 0b100 7250 7251 @property 7252 def is_mrc(self): 7253 """Page is part of Mixed Raster Content.""" 7254 return self.subfiletype & 0b1000 7255 7256 @property 7257 def is_tiled(self): 7258 """Page contains tiled image.""" 7259 return self.tilewidth > 0 # return 322 in self.tags # TileWidth 7260 7261 @property 7262 def is_subsampled(self): 7263 """Page contains chroma subsampled image.""" 7264 if self.subsampling is not None: 7265 return self.subsampling != (1, 1) 7266 return ( 7267 self.compression == 7 7268 and self.planarconfig == 1 7269 and self.photometric in (2, 6) 7270 ) 7271 7272 @lazyattr 7273 def is_imagej(self): 7274 """Return ImageJ description if exists, else None.""" 7275 for description in (self.description, self.description1): 7276 if not description: 7277 return None 7278 if description[:7] == 'ImageJ=': 7279 return description 7280 return None 7281 7282 @lazyattr 7283 def is_shaped(self): 7284 """Return description containing array shape if exists, else None.""" 7285 for description in (self.description, self.description1): 7286 if not description or '"mibi.' in description: 7287 return None 7288 if description[:1] == '{' and '"shape":' in description: 7289 return description 7290 if description[:6] == 'shape=': 7291 return description 7292 return None 7293 7294 @property 7295 def is_mdgel(self): 7296 """Page contains MDFileTag tag.""" 7297 return 33445 in self.tags # MDFileTag 7298 7299 @property 7300 def is_mediacy(self): 7301 """Page contains Media Cybernetics Id tag.""" 7302 tag = self.tags.get(50288) # MC_Id 7303 try: 7304 return tag is not None and tag.value[:7] == b'MC TIFF' 7305 except Exception: 7306 return False 7307 7308 @property 7309 def is_stk(self): 7310 """Page contains UIC2Tag tag.""" 7311 return 33629 in self.tags 7312 7313 @property 7314 def is_lsm(self): 7315 """Page contains CZ_LSMINFO tag.""" 7316 return 34412 in self.tags 7317 7318 @property 7319 def is_fluoview(self): 7320 """Page contains FluoView MM_STAMP tag.""" 7321 return 34362 in self.tags 7322 7323 @property 7324 def is_nih(self): 7325 """Page contains NIHImageHeader tag.""" 7326 return 43314 in self.tags 7327 7328 @property 7329 def is_volumetric(self): 7330 """Page contains SGI ImageDepth tag with value > 1.""" 7331 return self.imagedepth > 1 7332 7333 @property 7334 def is_vista(self): 7335 """Software tag is 'ISS Vista'.""" 7336 return self.software == 'ISS Vista' 7337 7338 @property 7339 def is_metaseries(self): 7340 """Page contains MDS MetaSeries metadata in ImageDescription tag.""" 7341 if self.index != 0 or self.software != 'MetaSeries': 7342 return False 7343 d = self.description 7344 return d.startswith('<MetaData>') and d.endswith('</MetaData>') 7345 7346 @property 7347 def is_ome(self): 7348 """Page contains OME-XML in ImageDescription tag.""" 7349 if self.index != 0 or not self.description: 7350 return False 7351 return self.description[-4:] == 'OME>' # and [:13] == '<?xml version' 7352 7353 @property 7354 def is_scn(self): 7355 """Page contains Leica SCN XML in ImageDescription tag.""" 7356 if self.index != 0 or not self.description: 7357 return False 7358 return self.description[-6:] == '</scn>' 7359 7360 @property 7361 def is_micromanager(self): 7362 """Page contains MicroManagerMetadata tag.""" 7363 return 51123 in self.tags 7364 7365 @property 7366 def is_andor(self): 7367 """Page contains Andor Technology tags 4864-5030.""" 7368 return 4864 in self.tags 7369 7370 @property 7371 def is_pilatus(self): 7372 """Page contains Pilatus tags.""" 7373 return self.software[:8] == 'TVX TIFF' and self.description[:2] == '# ' 7374 7375 @property 7376 def is_epics(self): 7377 """Page contains EPICS areaDetector tags.""" 7378 return ( 7379 self.description == 'EPICS areaDetector' 7380 or self.software == 'EPICS areaDetector' 7381 ) 7382 7383 @property 7384 def is_tvips(self): 7385 """Page contains TVIPS metadata.""" 7386 return 37706 in self.tags 7387 7388 @property 7389 def is_fei(self): 7390 """Page contains FEI_SFEG or FEI_HELIOS tags.""" 7391 return 34680 in self.tags or 34682 in self.tags 7392 7393 @property 7394 def is_sem(self): 7395 """Page contains CZ_SEM tag.""" 7396 return 34118 in self.tags 7397 7398 @property 7399 def is_svs(self): 7400 """Page contains Aperio metadata.""" 7401 return self.description[:7] == 'Aperio ' 7402 7403 @property 7404 def is_bif(self): 7405 """Page contains Ventana metadata.""" 7406 try: 7407 return ( 7408 self.description == 'Label Image' 7409 or ('Ventana' in self.software) 7410 ) and ( 7411 700 in self.tags and b'<iScan' in self.tags[700].value[:4096] 7412 ) 7413 except Exception: 7414 return False 7415 7416 @property 7417 def is_scanimage(self): 7418 """Page contains ScanImage metadata.""" 7419 return ( 7420 self.software[:3] == 'SI.' 7421 or self.description[:6] == 'state.' 7422 or 'scanimage.SI' in self.description[-256:] 7423 ) 7424 7425 @property 7426 def is_qpi(self): 7427 """Page contains PerkinElmer tissue images metadata.""" 7428 # The ImageDescription tag contains XML with a top-level 7429 # <PerkinElmer-QPI-ImageDescription> element 7430 return self.software[:15] == 'PerkinElmer-QPI' 7431 7432 @property 7433 def is_geotiff(self): 7434 """Page contains GeoTIFF metadata.""" 7435 return 34735 in self.tags # GeoKeyDirectoryTag 7436 7437 @property 7438 def is_tiffep(self): 7439 """Page contains TIFF/EP metadata.""" 7440 return 37398 in self.tags # TIFF/EPStandardID 7441 7442 @property 7443 def is_sis(self): 7444 """Page contains Olympus SIS metadata.""" 7445 return 33560 in self.tags or 33471 in self.tags 7446 7447 @property 7448 def is_ndpi(self): 7449 """Page contains NDPI metadata.""" 7450 return 65420 in self.tags and 271 in self.tags 7451 7452 @property 7453 def is_philips(self): 7454 """Page contains Philips DP metadata.""" 7455 return ( 7456 self.software[:10] == 'Philips DP' 7457 and self.description[-13:] == '</DataObject>' 7458 ) 7459 7460 @property 7461 def is_eer(self): 7462 """Page contains EER metadata.""" 7463 return ( 7464 self.parent.is_bigtiff 7465 and self.compression in (65000, 65001) 7466 and 65001 in self.tags 7467 ) 7468 7469 7470class TiffFrame: 7471 """Lightweight TIFF image file directory (IFD). 7472 7473 Only a limited number of tag values are read from file. 7474 Other tag values are assumed to be identical with a specified TiffPage 7475 instance, the keyframe. 7476 7477 TiffFrame is intended to reduce resource usage and speed up reading image 7478 data from file, not for introspection of metadata. 7479 7480 """ 7481 7482 __slots__ = ( 7483 'index', 7484 'parent', 7485 'offset', 7486 'dataoffsets', 7487 'databytecounts', 7488 'subifds', 7489 'jpegtables', 7490 '_keyframe', 7491 ) 7492 7493 is_mdgel = False 7494 pages = () 7495 # tags = {} 7496 7497 def __init__( 7498 self, 7499 parent, 7500 index, 7501 offset=None, 7502 keyframe=None, 7503 offsets=None, 7504 bytecounts=None, 7505 ): 7506 """Initialize TiffFrame from file or values. 7507 7508 The file handle position must be at the offset to a valid IFD. 7509 7510 """ 7511 self._keyframe = None 7512 self.parent = parent 7513 self.index = index 7514 self.offset = offset 7515 self.subifds = None 7516 self.jpegtables = None 7517 self.dataoffsets = () 7518 self.databytecounts = () 7519 7520 if offsets is not None: 7521 # initialize "virtual frame" from offsets and bytecounts 7522 self.dataoffsets = offsets 7523 self.databytecounts = bytecounts 7524 self._keyframe = keyframe 7525 return 7526 7527 if offset is None: 7528 self.offset = self.parent.filehandle.tell() 7529 else: 7530 self.parent.filehandle.seek(offset) 7531 7532 if keyframe is None: 7533 tags = {273, 279, 324, 325, 330, 347} 7534 elif keyframe.is_contiguous: 7535 # use databytecounts from keyframe 7536 tags = {256, 273, 324, 330} 7537 self.databytecounts = keyframe.databytecounts 7538 else: 7539 tags = {256, 273, 279, 324, 325, 330, 347} 7540 7541 for code, tag in self._gettags(tags): 7542 if code == 273 or code == 324: 7543 self.dataoffsets = tag.value 7544 elif code == 279 or code == 325: 7545 self.databytecounts = tag.value 7546 elif code == 330: 7547 self.subifds = tag.value 7548 elif code == 347: 7549 self.jpegtables = tag.value 7550 elif code == 256 and keyframe.imagewidth != tag.value: 7551 raise RuntimeError( 7552 f'TiffFrame {self.index} incompatible keyframe' 7553 ) 7554 7555 if not self.dataoffsets: 7556 log_warning(f'TiffFrame {self.index}: missing required tags') 7557 7558 self.keyframe = keyframe 7559 7560 def _gettags(self, codes=None, lock=None): 7561 """Return list of (code, TiffTag) from file.""" 7562 fh = self.parent.filehandle 7563 tiff = self.parent.tiff 7564 unpack = struct.unpack 7565 lock = NullContext() if lock is None else lock 7566 tags = [] 7567 7568 with lock: 7569 fh.seek(self.offset) 7570 try: 7571 tagno = unpack(tiff.tagnoformat, fh.read(tiff.tagnosize))[0] 7572 if tagno > 4096: 7573 raise ValueError(f'suspicious number of tags {tagno}') 7574 except Exception as exc: 7575 raise TiffFileError( 7576 f'TiffFrame {self.index}: ' 7577 f'corrupted tag list at offset {self.offset}' 7578 ) from exc 7579 7580 tagoffset = self.offset + tiff.tagnosize # fh.tell() 7581 tagsize = tiff.tagsize 7582 tagindex = -tagsize 7583 codeformat = tiff.tagformat1[:2] 7584 tagbytes = fh.read(tagsize * tagno) 7585 7586 for _ in range(tagno): 7587 tagindex += tagsize 7588 code = unpack(codeformat, tagbytes[tagindex : tagindex + 2])[0] 7589 if codes and code not in codes: 7590 continue 7591 try: 7592 tag = TiffTag.fromfile( 7593 self.parent, 7594 tagoffset + tagindex, 7595 tagbytes[tagindex : tagindex + tagsize], 7596 ) 7597 except TiffFileError as exc: 7598 log_warning( 7599 f'TiffFrame {self.index}: ' 7600 f'{exc.__class__.__name__}: {exc}' 7601 ) 7602 continue 7603 tags.append((code, tag)) 7604 7605 return tags 7606 7607 def _nextifd(self): 7608 """Return offset to next IFD from file.""" 7609 return TiffPage._nextifd(self) 7610 7611 def aspage(self): 7612 """Return TiffPage from file.""" 7613 if self.offset is None: 7614 raise ValueError( 7615 f'TiffFrame {self.index}: cannot return virtual frame as page' 7616 ) 7617 self.parent.filehandle.seek(self.offset) 7618 return TiffPage(self.parent, index=self.index) 7619 7620 def asarray(self, *args, **kwargs): 7621 """Read image data from file and return as numpy array.""" 7622 return TiffPage.asarray(self, *args, **kwargs) 7623 7624 def aszarr(self, **kwargs): 7625 """Return image data as zarr storage.""" 7626 return TiffPage.aszarr(self, **kwargs) 7627 7628 def asrgb(self, *args, **kwargs): 7629 """Read image data from file and return RGB image as numpy array.""" 7630 return TiffPage.asrgb(self, *args, **kwargs) 7631 7632 def segments(self, *args, **kwargs): 7633 """Return iterator over decoded segments in TiffFrame.""" 7634 return TiffPage.segments(self, *args, **kwargs) 7635 7636 @property 7637 def keyframe(self): 7638 """Return keyframe.""" 7639 return self._keyframe 7640 7641 @keyframe.setter 7642 def keyframe(self, keyframe): 7643 """Set keyframe.""" 7644 if self._keyframe == keyframe: 7645 return 7646 if self._keyframe is not None: 7647 raise RuntimeError( 7648 f'TiffFrame {self.index}: cannot reset keyframe' 7649 ) 7650 if len(self.dataoffsets) != len(keyframe.dataoffsets): 7651 raise RuntimeError( 7652 f'TiffFrame {self.index}: incompatible keyframe' 7653 ) 7654 if keyframe.is_contiguous: 7655 self.databytecounts = keyframe.databytecounts 7656 self._keyframe = keyframe 7657 7658 @property 7659 def is_contiguous(self): 7660 """Return offset and size of contiguous data, else None.""" 7661 # if self._keyframe is None: 7662 # raise RuntimeError(f'TiffFrame {self.index}: keyframe not set') 7663 if self._keyframe.is_contiguous: 7664 return self.dataoffsets[0], self._keyframe.is_contiguous[1] 7665 return None 7666 7667 @property 7668 def is_memmappable(self): 7669 """Return if page's image data in file can be memory-mapped.""" 7670 # if self._keyframe is None: 7671 # raise RuntimeError(f'TiffFrame {self.index}: keyframe not set') 7672 return self._keyframe.is_memmappable 7673 7674 @property 7675 def hash(self): 7676 """Return checksum to identify pages in same series.""" 7677 # if self._keyframe is None: 7678 # raise RuntimeError(f'TiffFrame {self.index}: keyframe not set') 7679 return self._keyframe.hash 7680 7681 def __getattr__(self, name): 7682 """Return attribute from keyframe.""" 7683 if name in TIFF.FRAME_ATTRS: 7684 return getattr(self._keyframe, name) 7685 # this error could be raised because an AttributeError was 7686 # raised inside a @property function 7687 raise AttributeError( 7688 f'{self.__class__.__name__!r} object has no attribute {name!r}' 7689 ) 7690 7691 def __str__(self, detail=0, width=79): 7692 """Return string containing information about TiffFrame.""" 7693 if self._keyframe is None: 7694 info = '' 7695 kf = None 7696 else: 7697 info = ' '.join( 7698 s 7699 for s in ( 7700 'x'.join(str(i) for i in self.shape), 7701 str(self.dtype), 7702 ) 7703 ) 7704 kf = TiffPage.__str__(self._keyframe, width=width - 11) 7705 if detail > 3: 7706 of = pformat(self.dataoffsets, width=width - 9, height=detail - 3) 7707 bc = pformat( 7708 self.databytecounts, width=width - 13, height=detail - 3 7709 ) 7710 info = f'\n Keyframe {kf}\n Offsets {of}\n Bytecounts {bc}' 7711 return f'TiffFrame {self.index} @{self.offset} {info}' 7712 7713 7714class TiffTag: 7715 """TIFF tag structure. 7716 7717 Attributes 7718 ---------- 7719 name : string 7720 Name of tag, TIFF.TAGS[code]. 7721 code : int 7722 Decimal code of tag. 7723 dtype : int 7724 Datatype of tag data. One of TIFF.DATATYPES. 7725 count : int 7726 Number of values. 7727 value : various types 7728 Tag data as Python object. 7729 valueoffset : int 7730 Location of value in file. 7731 offset : int 7732 Location of tag structure in file. 7733 parent : TiffFile or TiffWriter 7734 Reference to parent TIFF file. 7735 7736 All attributes are read-only. 7737 7738 """ 7739 7740 __slots__ = ( 7741 'parent', 7742 'offset', 7743 'code', 7744 'dtype', 7745 'count', 7746 'value', 7747 'valueoffset', 7748 ) 7749 7750 def __init__(self, parent, offset, code, dtype, count, value, valueoffset): 7751 """Initialize TiffTag instance from values.""" 7752 self.parent = parent 7753 self.offset = offset 7754 self.code = code 7755 self.dtype = TIFF.DATATYPES(dtype) 7756 self.count = count 7757 self.value = value 7758 self.valueoffset = valueoffset 7759 7760 @classmethod 7761 def fromfile(cls, parent, offset=None, header=None): 7762 """Return TiffTag instance read from file.""" 7763 fh = parent.filehandle 7764 tiff = parent.tiff 7765 unpack = struct.unpack 7766 7767 if header is None: 7768 if offset is None: 7769 offset = fh.tell() 7770 else: 7771 fh.seek(offset) 7772 header = fh.read(tiff.tagsize) 7773 elif offset is None: 7774 offset = fh.tell() 7775 7776 valueoffset = offset + tiff.tagsize - tiff.tagoffsetthreshold 7777 code, dtype = unpack(tiff.tagformat1, header[:4]) 7778 count, value = unpack(tiff.tagformat2, header[4:]) 7779 7780 try: 7781 dataformat = TIFF.DATA_FORMATS[dtype] 7782 except KeyError: 7783 raise TiffFileError(f'unknown tag data type {dtype!r}') 7784 7785 fmt = '{}{}{}'.format( 7786 tiff.byteorder, count * int(dataformat[0]), dataformat[1] 7787 ) 7788 size = struct.calcsize(fmt) 7789 if size > tiff.tagoffsetthreshold or code in TIFF.TAG_READERS: 7790 valueoffset = unpack(tiff.offsetformat, value)[0] 7791 if valueoffset < 8 or valueoffset > fh.size - size: 7792 raise TiffFileError('invalid tag value offset') 7793 # if valueoffset % 2: 7794 # log_warning('TiffTag: value does not begin on word boundary') 7795 fh.seek(valueoffset) 7796 if code in TIFF.TAG_READERS: 7797 readfunc = TIFF.TAG_READERS[code] 7798 value = readfunc( 7799 fh, tiff.byteorder, dtype, count, tiff.offsetsize 7800 ) 7801 elif dtype == 7 or (count > 1 and dtype == 1): 7802 value = read_bytes( 7803 fh, tiff.byteorder, dtype, count, tiff.offsetsize 7804 ) 7805 # elif code in TIFF.TAGS or dtype == 2: 7806 else: 7807 value = unpack(fmt, fh.read(size)) 7808 # else: 7809 # value = read_numpy( 7810 # fh, tiff.byteorder, dtype, count, tiff.offsetsize 7811 # ) 7812 elif dtype in (1, 7): 7813 # BYTES, UNDEFINED 7814 value = value[:size] 7815 elif ( 7816 tiff.version == 42 7817 and tiff.offsetsize == 8 7818 and count == 1 7819 and dtype in (4, 13) 7820 and value[4:] != b'\x00\x00\x00\x00' 7821 ): 7822 # NDPI LONG or IFD 7823 # dtype = 16 if dtype == 4 else 18 7824 # dataformat = '1Q' 7825 value = unpack('<Q', value) 7826 else: 7827 value = unpack(fmt, value[:size]) 7828 7829 process = ( 7830 code not in TIFF.TAG_READERS 7831 and code not in TIFF.TAG_TUPLE 7832 and dtype != 7 # UNDEFINED 7833 ) 7834 if process and dtype == 2 and isinstance(value[0], bytes): 7835 # TIFF ASCII fields can contain multiple strings, 7836 # each terminated with a NUL 7837 value = value[0] 7838 try: 7839 value = bytes2str(stripnull(value, first=False).strip()) 7840 except UnicodeDecodeError: 7841 log_warning(f'TiffTag {code}: coercing invalid ASCII to bytes') 7842 # dtype = 1 7843 # dataformat = '1B' 7844 else: 7845 if code in TIFF.TAG_ENUM: 7846 t = TIFF.TAG_ENUM[code] 7847 try: 7848 value = tuple(t(v) for v in value) 7849 except ValueError as exc: 7850 if code not in (259, 317): # ignore compression/predictor 7851 log_warning(f'TiffTag {code}: {exc}') 7852 if process: 7853 if len(value) == 1: 7854 value = value[0] 7855 return cls(parent, offset, code, dtype, count, value, valueoffset) 7856 7857 @property 7858 def name(self): 7859 """Return name of tag from TIFF.TAGS registry.""" 7860 return TIFF.TAGS.get(self.code, str(self.code)) 7861 7862 @property 7863 def dataformat(self): 7864 """Return data type as Python struct format.""" 7865 return TIFF.DATA_FORMATS[self.dtype] 7866 7867 @property 7868 def valuebytecount(self): 7869 """Return size of value in file.""" 7870 return self.count * struct.calcsize(TIFF.DATA_FORMATS[self.dtype]) 7871 7872 def _astuple(self): 7873 """Return tag code, dtype, count, and encoded value. 7874 7875 The encoded value is read from file if necessary. 7876 7877 """ 7878 # TODO: make this method public 7879 if isinstance(self.value, bytes): 7880 value = self.value 7881 else: 7882 dataformat = TIFF.DATA_FORMATS[self.dtype] 7883 count = self.count * int(dataformat[0]) 7884 fmt = '{}{}{}'.format( 7885 self.parent.tiff.byteorder, count, dataformat[1] 7886 ) 7887 try: 7888 if count == 1: 7889 value = struct.pack(fmt, self.value) 7890 else: 7891 value = struct.pack(fmt, *self.value) 7892 except Exception: 7893 tiff = self.parent.tiff 7894 if tiff.version == 42 and tiff.offsetsize == 8: 7895 raise NotImplementedError('cannot read NDPI > 4 GB files') 7896 fh = self.parent.filehandle 7897 pos = fh.tell() 7898 fh.seek(self.valueoffset) 7899 value = fh.read(struct.calcsize(fmt)) 7900 fh.seek(pos) 7901 return self.code, self.dtype.value, self.count, value 7902 7903 def overwrite(self, value, _arg=None, dtype=None, erase=True): 7904 """Write new tag value to file and return new TiffTag instance. 7905 7906 The value must be compatible with the struct.pack formats in 7907 TIFF.DATA_FORMATS. 7908 7909 The new packed value is appended to the file if it is longer than the 7910 old value. The old value is zeroed. The file position is left where it 7911 was. 7912 7913 """ 7914 if self.offset < 8 or self.valueoffset < 8: 7915 raise ValueError('cannot rewrite tag at offset < 8') 7916 7917 if hasattr(value, 'filehandle'): 7918 value = _arg 7919 warnings.warn( 7920 "TiffTag.overwrite: passing a TiffFile instance is deprecated " 7921 "and no longer required since 2021.7.x.", 7922 DeprecationWarning, 7923 stacklevel=2, 7924 ) 7925 7926 fh = self.parent.filehandle 7927 tiff = self.parent.tiff 7928 7929 if tiff.version == 42 and tiff.offsetsize == 8: 7930 # TODO: support patching NDPI > 4 GB files 7931 raise NotImplementedError('cannot patch NDPI > 4 GB files') 7932 7933 if value is None: 7934 value = b'' 7935 if dtype is None: 7936 dtype = self.dtype 7937 7938 try: 7939 dataformat = TIFF.DATA_FORMATS[dtype] 7940 except KeyError as exc: 7941 try: 7942 dataformat = dtype[-1:] 7943 if dataformat[0] in '<>': 7944 dataformat = dataformat[1:] 7945 dtype = TIFF.DATA_DTYPES[dataformat] 7946 except (KeyError, TypeError): 7947 raise ValueError(f'unknown dtype {dtype}') from exc 7948 7949 if dtype == 2: 7950 # strings 7951 if isinstance(value, str): 7952 # enforce 7-bit ASCII on Unicode strings 7953 try: 7954 value = value.encode('ascii') 7955 except UnicodeEncodeError as exc: 7956 raise ValueError( 7957 'TIFF strings must be 7-bit ASCII' 7958 ) from exc 7959 elif not isinstance(value, bytes): 7960 raise ValueError('TIFF strings must be 7-bit ASCII') 7961 if len(value) == 0 or value[-1] != b'\x00': 7962 value += b'\x00' 7963 count = len(value) 7964 value = (value,) 7965 elif isinstance(value, bytes): 7966 # pre-packed binary data 7967 dtsize = struct.calcsize(dataformat) 7968 if len(value) % dtsize: 7969 raise ValueError('invalid packed binary data') 7970 count = len(value) // dtsize 7971 value = (value,) 7972 else: 7973 try: 7974 count = len(value) 7975 except TypeError: 7976 value = (value,) 7977 count = 1 7978 7979 if dtype in (5, 10): 7980 if count < 2 or count % 2: 7981 raise ValueError('invalid RATIONAL value') 7982 count //= 2 # rational 7983 7984 packedvalue = struct.pack( 7985 '{}{}{}'.format( 7986 tiff.byteorder, count * int(dataformat[0]), dataformat[1] 7987 ), 7988 *value, 7989 ) 7990 newsize = len(packedvalue) 7991 oldsize = self.count * struct.calcsize(TIFF.DATA_FORMATS[self.dtype]) 7992 valueoffset = self.valueoffset 7993 7994 pos = fh.tell() 7995 try: 7996 if dtype != self.dtype: 7997 # rewrite data type 7998 fh.seek(self.offset + 2) 7999 fh.write(struct.pack(tiff.byteorder + 'H', dtype)) 8000 8001 if oldsize <= tiff.tagoffsetthreshold: 8002 if newsize <= tiff.tagoffsetthreshold: 8003 # inline -> inline: overwrite 8004 fh.seek(self.offset + 4) 8005 fh.write(struct.pack(tiff.tagformat2, count, packedvalue)) 8006 else: 8007 # inline -> separate: append to file 8008 fh.seek(0, os.SEEK_END) 8009 valueoffset = fh.tell() 8010 if valueoffset % 2: 8011 # value offset must begin on a word boundary 8012 fh.write(b'\x00') 8013 valueoffset += 1 8014 fh.write(packedvalue) 8015 fh.seek(self.offset + 4) 8016 fh.write( 8017 struct.pack( 8018 tiff.tagformat2, 8019 count, 8020 struct.pack(tiff.offsetformat, valueoffset), 8021 ) 8022 ) 8023 elif newsize <= tiff.tagoffsetthreshold: 8024 # separate -> inline: erase old value 8025 valueoffset = self.offset + 4 + tiff.offsetsize 8026 fh.seek(self.offset + 4) 8027 fh.write(struct.pack(tiff.tagformat2, count, packedvalue)) 8028 if erase: 8029 fh.seek(self.valueoffset) 8030 fh.write(b'\x00' * oldsize) 8031 elif newsize <= oldsize or self.valueoffset + oldsize == fh.size: 8032 # separate -> separate smaller: overwrite, erase remaining 8033 fh.seek(self.offset + 4) 8034 fh.write(struct.pack(tiff.offsetformat, count)) 8035 fh.seek(self.valueoffset) 8036 fh.write(packedvalue) 8037 if erase and oldsize - newsize > 0: 8038 fh.write(b'\x00' * (oldsize - newsize)) 8039 else: 8040 # separate -> separate larger: erase old value, append to file 8041 if erase: 8042 fh.seek(self.valueoffset) 8043 fh.write(b'\x00' * oldsize) 8044 fh.seek(0, os.SEEK_END) 8045 valueoffset = fh.tell() 8046 if valueoffset % 2: 8047 # value offset must begin on a word boundary 8048 fh.write(b'\x00') 8049 valueoffset += 1 8050 fh.write(packedvalue) 8051 fh.seek(self.offset + 4) 8052 fh.write( 8053 struct.pack( 8054 tiff.tagformat2, 8055 count, 8056 struct.pack(tiff.offsetformat, valueoffset), 8057 ) 8058 ) 8059 finally: 8060 fh.seek(pos) # must restore file position 8061 8062 return TiffTag( 8063 self.parent, 8064 self.offset, 8065 self.code, 8066 dtype, 8067 count, 8068 value, 8069 valueoffset, 8070 ) 8071 8072 def _fix_lsm_bitspersample(self): 8073 """Correct LSM bitspersample tag. 8074 8075 Old LSM writers may use a separate region for two 16-bit values, 8076 although they fit into the tag value element of the tag. 8077 8078 """ 8079 if self.code != 258 or self.count != 2: 8080 return 8081 # TODO: test this case; need example file 8082 log_warning(f'TiffTag {self.code}: correcting LSM bitspersample tag') 8083 value = struct.pack('<HH', *self.value) 8084 self.valueoffset = struct.unpack('<I', value)[0] 8085 self.parent.filehandle.seek(self.valueoffset) 8086 self.value = struct.unpack('<HH', self.parent.filehandle.read(4)) 8087 8088 def __str__(self, detail=0, width=79): 8089 """Return string containing information about TiffTag.""" 8090 height = 1 if detail <= 0 else 8 * detail 8091 dtype = self.dtype.name 8092 if self.count > 1: 8093 dtype += f'[{self.count}]' 8094 name = '|'.join(TIFF.TAGS.getall(self.code, ())) 8095 if name: 8096 name = f'{self.code} {name} @{self.offset}' 8097 else: 8098 name = f'{self.code} @{self.offset}' 8099 line = f'TiffTag {name} {dtype} @{self.valueoffset} ' 8100 line = line[:width] 8101 try: 8102 if self.count == 1: 8103 value = enumstr(self.value) 8104 else: 8105 value = pformat(tuple(enumstr(v) for v in self.value)) 8106 except Exception: 8107 value = pformat(self.value, width=width, height=height) 8108 if detail <= 0: 8109 line += value 8110 line = line[:width] 8111 else: 8112 line += '\n' + value 8113 return line 8114 8115 8116class TiffTags: 8117 """Multidict like interface to TiffTag instances in TiffPage. 8118 8119 Differences to a regular dict: 8120 8121 * values are instances of TiffTag. 8122 * keys are TiffTag.code (int). 8123 * multiple values can be stored per key. 8124 * can be indexed with TiffTag.name (str), although slower than by key. 8125 * iter() returns values instead of keys. 8126 * values() and items() contain all values sorted by offset stored in file. 8127 * len() returns the number of all values. 8128 * get() takes an optional index argument. 8129 * some functions are not implemented, e.g. update, setdefault, pop. 8130 8131 """ 8132 8133 __slots__ = ('_dict', '_list') 8134 8135 def __init__(self): 8136 """Initialize empty instance.""" 8137 self._dict = {} 8138 self._list = [self._dict] 8139 8140 def add(self, tag): 8141 """Add a tag.""" 8142 code = tag.code 8143 for d in self._list: 8144 if code not in d: 8145 d[code] = tag 8146 break 8147 else: 8148 self._list.append({code: tag}) 8149 8150 def keys(self): 8151 """Return new view of all codes.""" 8152 return self._dict.keys() 8153 8154 def values(self): 8155 """Return all tags in order they are stored in file.""" 8156 tags = (t for d in self._list for t in d.values()) 8157 return sorted(tags, key=lambda t: t.offset) 8158 8159 def items(self): 8160 """Return all (code, tag) pairs in order tags are stored in file.""" 8161 items = (i for d in self._list for i in d.items()) 8162 return sorted(items, key=lambda i: i[1].offset) 8163 8164 def get(self, key, default=None, index=None): 8165 """Return tag of code or name if exists, else default.""" 8166 if index is None: 8167 if key in self._dict: 8168 return self._dict[key] 8169 if not isinstance(key, str): 8170 return default 8171 index = 0 8172 try: 8173 tags = self._list[index] 8174 except IndexError: 8175 return default 8176 if key in tags: 8177 return tags[key] 8178 if not isinstance(key, str): 8179 return default 8180 for tag in tags.values(): 8181 if tag.name == key: 8182 return tag 8183 return default 8184 8185 def getall(self, key, default=None): 8186 """Return list of all tags of code or name if exists, else default.""" 8187 result = [] 8188 for tags in self._list: 8189 if key in tags: 8190 result.append(tags[key]) 8191 else: 8192 break 8193 if result: 8194 return result 8195 if not isinstance(key, str): 8196 return default 8197 for tags in self._list: 8198 for tag in tags.values(): 8199 if tag.name == key: 8200 result.append(tag) 8201 break 8202 if not result: 8203 break 8204 return result if result else default 8205 8206 def __getitem__(self, key): 8207 """Return first tag of code or name. Raise KeyError if not found.""" 8208 if key in self._dict: 8209 return self._dict[key] 8210 if not isinstance(key, str): 8211 raise KeyError(key) 8212 for tag in self._dict.values(): 8213 if tag.name == key: 8214 return tag 8215 raise KeyError(key) 8216 8217 def __setitem__(self, code, tag): 8218 """Add a tag.""" 8219 self.add(tag) 8220 8221 def __delitem__(self, key): 8222 """Delete all tags of code or name.""" 8223 found = False 8224 for tags in self._list: 8225 if key in tags: 8226 found = True 8227 del tags[key] 8228 else: 8229 break 8230 if found: 8231 return None 8232 if not isinstance(key, str): 8233 raise KeyError(key) 8234 for tags in self._list: 8235 for tag in tags.values(): 8236 if tag.name == key: 8237 del tags[tag.code] 8238 found = True 8239 break 8240 else: 8241 break 8242 if not found: 8243 raise KeyError(key) 8244 return None 8245 8246 def __contains__(self, item): 8247 """Return if tag is in map.""" 8248 if item in self._dict: 8249 return True 8250 if not isinstance(item, str): 8251 return False 8252 for tag in self._dict.values(): 8253 if tag.name == item: 8254 return True 8255 return False 8256 8257 def __iter__(self): 8258 """Return iterator over all tags.""" 8259 return iter(self.values()) 8260 8261 def __len__(self): 8262 """Return number of tags.""" 8263 size = 0 8264 for d in self._list: 8265 size += len(d) 8266 return size 8267 8268 def __str__(self, detail=0, width=79): 8269 """Return string with information about TiffTags.""" 8270 info = [] 8271 tlines = [] 8272 vlines = [] 8273 for tag in self: 8274 value = tag.__str__(width=width + 1) 8275 tlines.append(value[:width].strip()) 8276 if detail > 0 and len(value) > width: 8277 if detail < 2 and tag.code in (273, 279, 324, 325): 8278 value = pformat(tag.value, width=width, height=detail * 4) 8279 else: 8280 value = pformat(tag.value, width=width, height=detail * 12) 8281 vlines.append(f'{tag.name}\n{value}') 8282 info.append('\n'.join(tlines)) 8283 if detail > 0 and vlines: 8284 info.append('\n') 8285 info.append('\n\n'.join(vlines)) 8286 return '\n'.join(info) 8287 8288 8289class TiffTagRegistry: 8290 """Registry of TIFF tag codes and names. 8291 8292 The registry allows to look up tag codes and names by indexing with names 8293 and codes respectively. 8294 One tag code may be registered with several names, e.g. 34853 is used for 8295 GPSTag or OlympusSIS2. 8296 Different tag codes may be registered with the same name, e.g. 37387 and 8297 41483 are both named FlashEnergy. 8298 8299 """ 8300 8301 __slots__ = ('_dict', '_list') 8302 8303 def __init__(self, arg): 8304 self._dict = {} 8305 self._list = [self._dict] 8306 self.update(arg) 8307 8308 def update(self, arg): 8309 """Add codes and names from sequence or dict to registry.""" 8310 if isinstance(arg, TiffTagRegistry): 8311 self._list.extend(arg._list) 8312 return 8313 if isinstance(arg, dict): 8314 arg = arg.items() 8315 for code, name in arg: 8316 self.add(code, name) 8317 8318 def add(self, code, name): 8319 """Add code and name to registry.""" 8320 for d in self._list: 8321 if code in d and d[code] == name: 8322 break 8323 if code not in d and name not in d: 8324 d[code] = name 8325 d[name] = code 8326 break 8327 else: 8328 self._list.append({code: name, name: code}) 8329 8330 def items(self): 8331 """Return all registry items as (code, name).""" 8332 items = ( 8333 i for d in self._list for i in d.items() if isinstance(i[0], int) 8334 ) 8335 return sorted(items, key=lambda i: i[0]) 8336 8337 def get(self, key, default=None): 8338 """Return first code/name if exists, else default.""" 8339 for d in self._list: 8340 if key in d: 8341 return d[key] 8342 return default 8343 8344 def getall(self, key, default=None): 8345 """Return list of all codes/names if exists, else default.""" 8346 result = [d[key] for d in self._list if key in d] 8347 return result if result else default 8348 8349 def __getitem__(self, key): 8350 """Return first code/name. Raise KeyError if not found.""" 8351 for d in self._list: 8352 if key in d: 8353 return d[key] 8354 raise KeyError(key) 8355 8356 def __delitem__(self, key): 8357 """Delete all tags of code or name.""" 8358 found = False 8359 for d in self._list: 8360 if key in d: 8361 found = True 8362 value = d[key] 8363 del d[key] 8364 del d[value] 8365 if not found: 8366 raise KeyError(key) 8367 8368 def __contains__(self, item): 8369 """Return if code or name is in registry.""" 8370 for d in self._list: 8371 if item in d: 8372 return True 8373 return False 8374 8375 def __iter__(self): 8376 """Return iterator over all items in registry.""" 8377 return iter(self.items()) 8378 8379 def __len__(self): 8380 """Return number of registered tags.""" 8381 size = 0 8382 for d in self._list: 8383 size += len(d) 8384 return size // 2 8385 8386 def __str__(self): 8387 """Return string with information about TiffTags.""" 8388 return 'TiffTagRegistry(((\n {}\n))'.format( 8389 ',\n '.join(f'({code}, {name!r})' for code, name in self.items()) 8390 ) 8391 8392 8393class TiffPageSeries: 8394 """Series of TIFF pages with compatible shape and data type. 8395 8396 Attributes 8397 ---------- 8398 pages : list of TiffPage 8399 Sequence of TiffPages in series. 8400 dtype : numpy.dtype 8401 Data type (native byte order) of the image array in series. 8402 shape : tuple 8403 Dimensions of the image array in series. 8404 axes : str 8405 Labels of axes in shape. See TIFF.AXES_LABELS. 8406 offset : int or None 8407 Position of image data in file if memory-mappable, else None. 8408 levels : list of TiffPageSeries 8409 Pyramid levels. levels[0] is 'self'. 8410 8411 """ 8412 8413 def __init__( 8414 self, 8415 pages, 8416 shape=None, 8417 dtype=None, 8418 axes=None, 8419 parent=None, 8420 name=None, 8421 transform=None, 8422 kind=None, 8423 truncated=False, 8424 multifile=False, 8425 squeeze=True, 8426 ): 8427 """Initialize instance.""" 8428 self.index = 0 8429 self._pages = pages # might contain only first of contiguous pages 8430 self.levels = [self] 8431 if shape is None: 8432 shape = pages[0].shape 8433 if axes is None: 8434 axes = pages[0].axes 8435 if dtype is None: 8436 dtype = pages[0].dtype 8437 8438 self.set_shape_axes(shape, axes, squeeze) 8439 8440 self.dtype = numpy.dtype(dtype) 8441 self.kind = kind if kind else '' 8442 self.name = name if name else '' 8443 self.transform = transform 8444 self.keyframe = next(p.keyframe for p in pages if p is not None) 8445 self.is_multifile = bool(multifile) 8446 8447 if parent: 8448 self.parent = parent 8449 elif pages: 8450 self.parent = self.keyframe.parent 8451 else: 8452 self.parent = None 8453 if not truncated and len(pages) == 1: 8454 s = product(pages[0].shape) 8455 if s > 0: 8456 self._len = int(product(self.shape) // s) 8457 else: 8458 self._len = len(pages) 8459 else: 8460 self._len = len(pages) 8461 8462 def set_shape_axes(self, shape, axes, squeeze=True): 8463 """Set shape and axes.""" 8464 shape = tuple(shape) 8465 axes = ''.join(axes) 8466 # expanded shape according to metadata 8467 self._shape_expanded = shape 8468 self._axes_expanded = axes 8469 # squeezed shape and axes 8470 self._shape_squeezed, self._axes_squeezed = squeeze_axes(shape, axes) 8471 # default shape and axes returned by asarray 8472 self.shape = self._shape_squeezed if squeeze else self._shape_expanded 8473 self.axes = self._axes_squeezed if squeeze else self._axes_expanded 8474 8475 def get_shape(self, squeeze=None): 8476 """Return default, squeezed, or expanded shape.""" 8477 if squeeze is None: 8478 return self.shape 8479 return self._shape_squeezed if squeeze else self._shape_expanded 8480 8481 def get_axes(self, squeeze=None): 8482 """Return default, squeezed, or expanded axes.""" 8483 if squeeze is None: 8484 return self.axes 8485 return self._axes_squeezed if squeeze else self._axes_expanded 8486 8487 def asarray(self, level=None, **kwargs): 8488 """Return image data from series of TIFF pages as numpy array.""" 8489 if level is not None: 8490 return self.levels[level].asarray(**kwargs) 8491 if self.parent: 8492 result = self.parent.asarray(series=self, **kwargs) 8493 if self.transform is not None: 8494 result = self.transform(result) 8495 return result 8496 return None 8497 8498 def aszarr(self, level=None, **kwargs): 8499 """Return image data from series of TIFF pages as zarr storage.""" 8500 if self.parent: 8501 return ZarrTiffStore(self, level=level, **kwargs) 8502 return None 8503 8504 @lazyattr 8505 def offset(self): 8506 """Return offset to series data in file, if any.""" 8507 if not self._pages: 8508 return None 8509 8510 pos = 0 8511 for page in self._pages: 8512 if page is None: 8513 return None 8514 if not page.is_final: 8515 return None 8516 if not pos: 8517 pos = page.is_contiguous[0] + page.is_contiguous[1] 8518 continue 8519 if pos != page.is_contiguous[0]: 8520 return None 8521 pos += page.is_contiguous[1] 8522 8523 page = self._pages[0] 8524 offset = page.is_contiguous[0] 8525 if (page.is_imagej or page.is_shaped or page.is_stk) and len( 8526 self._pages 8527 ) == 1: 8528 # truncated files 8529 return offset 8530 if pos == offset + product(self.shape) * self.dtype.itemsize: 8531 return offset 8532 return None 8533 8534 @property 8535 def is_pyramidal(self): 8536 """Return if series contains several levels.""" 8537 return len(self.levels) > 1 8538 8539 @property 8540 def ndim(self): 8541 """Return number of array dimensions.""" 8542 return len(self.shape) 8543 8544 @property 8545 def size(self): 8546 """Return number of elements in array.""" 8547 return int(product(self.shape)) 8548 8549 @property 8550 def pages(self): 8551 """Return sequence of all pages in series.""" 8552 # a workaround to keep the old interface working 8553 return self 8554 8555 def _getitem(self, key): 8556 """Return specified page of series from cache or file.""" 8557 key = int(key) 8558 if key < 0: 8559 key %= self._len 8560 if len(self._pages) == 1 and 0 < key < self._len: 8561 index = self._pages[0].index 8562 return self.parent.pages._getitem(index + key) 8563 return self._pages[key] 8564 8565 def __getitem__(self, key): 8566 """Return specified page(s).""" 8567 getitem = self._getitem 8568 if isinstance(key, (int, numpy.integer)): 8569 return getitem(key) 8570 if isinstance(key, slice): 8571 return [getitem(i) for i in range(*key.indices(self._len))] 8572 if isinstance(key, Iterable): 8573 return [getitem(k) for k in key] 8574 raise TypeError('key must be an integer, slice, or iterable') 8575 8576 def __iter__(self): 8577 """Return iterator over pages in series.""" 8578 if len(self._pages) == self._len: 8579 yield from self._pages 8580 else: 8581 pages = self.parent.pages 8582 index = self._pages[0].index 8583 for i in range(self._len): 8584 yield pages[index + i] 8585 8586 def __len__(self): 8587 """Return number of pages in series.""" 8588 return self._len 8589 8590 def __str__(self): 8591 """Return string with information about TiffPageSeries.""" 8592 s = ' '.join( 8593 s 8594 for s in ( 8595 snipstr(f'{self.name!r}', 20) if self.name else '', 8596 'x'.join(str(i) for i in self.shape), 8597 str(self.dtype), 8598 self.axes, 8599 self.kind, 8600 (f'{len(self.levels)} Levels') if self.is_pyramidal else '', 8601 f'{len(self)} Pages', 8602 (f'@{self.offset}') if self.offset else '', 8603 ) 8604 if s 8605 ) 8606 return f'TiffPageSeries {self.index} {s}' 8607 8608 8609class ZarrStore(MutableMapping): 8610 """Zarr storage base class. 8611 8612 ZarrStore instances must be closed using the 'close' method, which is 8613 automatically called when using the 'with' context manager. 8614 8615 https://zarr.readthedocs.io/en/stable/spec/v2.html 8616 https://forum.image.sc/t/multiscale-arrays-v0-1/37930 8617 8618 """ 8619 8620 def __init__(self, fillvalue=None, chunkmode=None): 8621 """Initialize ZarrStore.""" 8622 self._store = {} 8623 self._fillvalue = 0 if fillvalue is None else fillvalue 8624 if chunkmode is None: 8625 self._chunkmode = TIFF.CHUNKMODE(0) 8626 else: 8627 self._chunkmode = enumarg(TIFF.CHUNKMODE, chunkmode) 8628 8629 def __enter__(self): 8630 return self 8631 8632 def __exit__(self, exc_type, exc_value, traceback): 8633 self.close() 8634 8635 def __del__(self): 8636 self.close() 8637 8638 def close(self): 8639 """Close ZarrStore.""" 8640 8641 def flush(self): 8642 """Flush ZarrStore.""" 8643 raise PermissionError('ZarrStore is read-only') 8644 8645 def clear(self): 8646 """Clear ZarrStore.""" 8647 raise PermissionError('ZarrStore is read-only') 8648 8649 def keys(self): 8650 """Return keys in ZarrStore.""" 8651 return self._store.keys() 8652 8653 def items(self): 8654 """Return items in ZarrStore.""" 8655 return self._store.items() 8656 8657 def values(self): 8658 """Return values in ZarrStore.""" 8659 return self._store.values() 8660 8661 def __iter__(self): 8662 return iter(self._store.keys()) 8663 8664 def __len__(self): 8665 return len(self._store) 8666 8667 def __delitem__(self, key): 8668 raise PermissionError('ZarrStore is read-only') 8669 8670 def __contains__(self, key): 8671 return key in self._store 8672 8673 def __setitem__(self, key, value): 8674 raise PermissionError('ZarrStore is read-only') 8675 8676 def __getitem__(self, key): 8677 if key in self._store: 8678 return self._store[key] 8679 return self._getitem(key) 8680 8681 def _getitem(self, key): 8682 """Return chunk from file.""" 8683 raise NotImplementedError 8684 8685 @property 8686 def is_multiscales(self): 8687 """Return if ZarrStore is multiscales.""" 8688 return b'multiscales' in self._store['.zattrs'] 8689 8690 @staticmethod 8691 def _empty_chunk(shape, dtype, fillvalue): 8692 """Return empty chunk.""" 8693 if fillvalue is None or fillvalue == 0: 8694 return bytes(product(shape) * dtype.itemsize) 8695 chunk = numpy.empty(shape, dtype) 8696 chunk[:] = fillvalue 8697 return chunk # .tobytes() 8698 8699 @staticmethod 8700 def _dtype(dtype): 8701 """Return dtype as string with native byte order.""" 8702 if dtype.itemsize == 1: 8703 byteorder = '|' 8704 else: 8705 byteorder = {'big': '>', 'little': '<'}[sys.byteorder] 8706 return byteorder + dtype.str[1:] 8707 8708 @staticmethod 8709 def _json(obj): 8710 """Serialize obj to a JSON formatted string.""" 8711 return json.dumps( 8712 obj, 8713 indent=1, 8714 sort_keys=True, 8715 ensure_ascii=True, 8716 separators=(',', ': '), 8717 ).encode('ascii') 8718 8719 @staticmethod 8720 def _value(value, dtype): 8721 """Return value which is serializable to JSON.""" 8722 if value is None: 8723 return value 8724 if dtype.kind == 'b': 8725 return bool(value) 8726 if dtype.kind in 'ui': 8727 return int(value) 8728 if dtype.kind == 'f': 8729 if numpy.isnan(value): 8730 return 'NaN' 8731 if numpy.isposinf(value): 8732 return 'Infinity' 8733 if numpy.isneginf(value): 8734 return '-Infinity' 8735 return float(value) 8736 if dtype.kind in 'c': 8737 value = numpy.array(value, dtype) 8738 return ( 8739 ZarrStore._value(value.real, dtype.type().real.dtype), 8740 ZarrStore._value(value.imag, dtype.type().imag.dtype), 8741 ) 8742 return value 8743 8744 @staticmethod 8745 def _ndindex(shape, chunks): 8746 """Return iterator over all chunk index strings.""" 8747 assert len(shape) == len(chunks) 8748 chunked = tuple( 8749 i // j + (1 if i % j else 0) for i, j in zip(shape, chunks) 8750 ) 8751 for indices in numpy.ndindex(chunked): 8752 yield '.'.join(str(index) for index in indices) 8753 8754 8755class ZarrTiffStore(ZarrStore): 8756 """Zarr storage interface to image data in TiffPage or TiffPageSeries. 8757 8758 ZarrTiffStore instances are using a TiffFile instance for reading and 8759 decoding chunks. Therefore ZarrTiffStore instances cannot be pickled. 8760 8761 """ 8762 8763 def __init__( 8764 self, 8765 arg, 8766 level=None, 8767 chunkmode=None, 8768 fillvalue=None, 8769 lock=None, 8770 squeeze=None, 8771 _openfiles=None, 8772 ): 8773 """Initialize Zarr storage from TiffPage or TiffPageSeries.""" 8774 super().__init__(fillvalue=fillvalue, chunkmode=chunkmode) 8775 8776 if self._chunkmode not in (0, 2): 8777 raise NotImplementedError(f'{self._chunkmode!r} not implemented') 8778 8779 self._squeeze = None if squeeze is None else bool(squeeze) 8780 self._transform = getattr(arg, 'transform', None) 8781 self._data = getattr(arg, 'levels', [TiffPageSeries([arg])]) 8782 if level is not None: 8783 self._data = [self._data[level]] 8784 8785 if lock is None: 8786 fh = self._data[0].keyframe.parent._parent.filehandle 8787 fh.lock = True 8788 lock = fh.lock 8789 self._filecache = FileCache(size=_openfiles, lock=lock) 8790 8791 if len(self._data) > 1: 8792 # multiscales 8793 self._store['.zgroup'] = ZarrStore._json({'zarr_format': 2}) 8794 self._store['.zattrs'] = ZarrStore._json( 8795 { 8796 'multiscales': [ 8797 { 8798 'datasets': [ 8799 {'path': str(i)} 8800 for i in range(len(self._data)) 8801 ], 8802 'metadata': {}, 8803 'name': arg.name, 8804 # 'type': 'gaussian', 8805 'version': '0.1', 8806 } 8807 ] 8808 } 8809 ) 8810 for level, series in enumerate(self._data): 8811 series.keyframe.decode # cache decode function 8812 shape = series.get_shape(squeeze) 8813 dtype = series.dtype 8814 if fillvalue is None: 8815 self._fillvalue = fillvalue = series.keyframe.nodata 8816 if self._chunkmode: 8817 chunks = series.keyframe.shape 8818 else: 8819 chunks = series.keyframe.chunks 8820 self._store[f'{level}/.zarray'] = ZarrStore._json( 8821 { 8822 'zarr_format': 2, 8823 'shape': shape, 8824 'chunks': ZarrTiffStore._chunks(chunks, shape), 8825 'dtype': ZarrStore._dtype(dtype), 8826 'compressor': None, 8827 'fill_value': ZarrStore._value(fillvalue, dtype), 8828 'order': 'C', 8829 'filters': None, 8830 } 8831 ) 8832 else: 8833 series = self._data[0] 8834 series.keyframe.decode # cache decode function 8835 shape = series.get_shape(squeeze) 8836 dtype = series.dtype 8837 if fillvalue is None: 8838 self._fillvalue = fillvalue = series.keyframe.nodata 8839 if self._chunkmode: 8840 chunks = series.keyframe.shape 8841 else: 8842 chunks = series.keyframe.chunks 8843 self._store['.zattrs'] = ZarrStore._json({}) 8844 self._store['.zarray'] = ZarrStore._json( 8845 { 8846 'zarr_format': 2, 8847 'shape': shape, 8848 'chunks': ZarrTiffStore._chunks(chunks, shape), 8849 'dtype': ZarrStore._dtype(dtype), 8850 'compressor': None, 8851 'fill_value': ZarrStore._value(fillvalue, dtype), 8852 'order': 'C', 8853 'filters': None, 8854 } 8855 ) 8856 8857 def close(self): 8858 """Close ZarrTiffStore.""" 8859 if hasattr(self, '_filecache'): 8860 self._filecache.clear() 8861 8862 def write_fsspec(self, arg, url, compressors=None, version=None): 8863 """Write fsspec ReferenceFileSystem as JSON to file. 8864 8865 Url is the remote location of the TIFF file without the file name(s). 8866 8867 Raise ValueError if TIFF store can not be represented as 8868 ReferenceFileSystem due to features that are not supported by zarr 8869 or numcodecs: 8870 8871 * compressors, e.g. CCITT 8872 * filters, e.g. predictors, bitorder reversal, packed integers 8873 * dtypes, e.g. float24 8874 * JPEGTables in multi-page files 8875 * incomplete chunks, e.g. if imagelength % rowsperstrip != 0 8876 8877 Files containing incomplete tiles may fail at runtime. 8878 8879 https://github.com/intake/fsspec-reference-maker 8880 8881 """ 8882 compressors = { 8883 1: None, 8884 8: 'zlib', 8885 32946: 'zlib', 8886 34925: 'lzma', 8887 50000: 'zstd', 8888 5: 'imagecodecs_lzw', 8889 7: 'imagecodecs_jpeg', 8890 22610: 'imagecodecs_jpegxr', # NDPI 8891 32773: 'imagecodecs_packbits', 8892 33003: 'imagecodecs_jpeg2k', 8893 33004: 'imagecodecs_jpeg2k', 8894 33005: 'imagecodecs_jpeg2k', 8895 33007: 'imagecodecs_jpeg', # ALT_JPG 8896 34712: 'imagecodecs_jpeg2k', 8897 34887: 'imagecodecs_lerc', 8898 34892: 'imagecodecs_jpeg', # DNG lossy 8899 34933: 'imagecodecs_png', 8900 34934: 'imagecodecs_jpegxr', 8901 50001: 'imagecodecs_webp', 8902 50002: 'imagecodecs_jpegxl', 8903 **({} if compressors is None else compressors), 8904 } 8905 8906 for series in self._data: 8907 errormsg = ' not supported by the fsspec ReferenceFileSystem' 8908 keyframe = series.keyframe 8909 if keyframe.compression not in compressors: 8910 raise ValueError(f'{keyframe.compression!r} is' + errormsg) 8911 if keyframe.predictor != 1: 8912 raise ValueError(f'{keyframe.predictor!r} is' + errormsg) 8913 if keyframe.fillorder != 1: 8914 raise ValueError(f'{keyframe.fillorder!r} is' + errormsg) 8915 if keyframe.sampleformat not in (1, 2, 3, 6): 8916 raise ValueError(f'{keyframe.sampleformat!r} is' + errormsg) 8917 if keyframe.bitspersample not in ( 8918 8, 8919 16, 8920 32, 8921 64, 8922 128, 8923 ) and keyframe.compression not in ( 8924 7, 8925 33007, 8926 34892, 8927 ): # JPEG 8928 raise ValueError( 8929 f'BitsPerSample {keyframe.bitspersample} is' + errormsg 8930 ) 8931 if ( 8932 not self._chunkmode 8933 and not keyframe.is_tiled 8934 and keyframe.imagelength % keyframe.rowsperstrip 8935 ): 8936 raise ValueError('incomplete chunks are' + errormsg) 8937 if self._chunkmode and not keyframe.is_final: 8938 raise ValueError(f'{self._chunkmode!r} is' + errormsg) 8939 if keyframe.jpegtables is not None and len(series.pages) > 1: 8940 raise ValueError( 8941 'JPEGTables in multi-page files are' + errormsg 8942 ) 8943 8944 if url is None: 8945 url = '' 8946 elif url and url[-1] != '/': 8947 url += '/' 8948 8949 byteorder = '<' if sys.byteorder == 'big' else '>' 8950 if ( 8951 self._data[0].keyframe.parent.byteorder != byteorder 8952 or self._data[0].keyframe.dtype.itemsize == 1 8953 ): 8954 byteorder = None 8955 8956 refs = dict() 8957 if version == 1: 8958 refs['version'] = 1 8959 refs['templates'] = {} 8960 refs['gen'] = [] 8961 templates = {} 8962 if self._data[0].is_multifile: 8963 i = 0 8964 for page in self._data[0].pages: 8965 if page is None: 8966 continue 8967 fname = page.keyframe.parent.filehandle.name 8968 if fname in templates: 8969 continue 8970 key = f'u{i}' 8971 templates[fname] = '{{%s}}' % key 8972 refs['templates'][key] = url + fname 8973 i += 1 8974 else: 8975 fname = self._data[0].keyframe.parent.filehandle.name 8976 key = 'u' 8977 templates[fname] = '{{%s}}' % key 8978 refs['templates'][key] = url + fname 8979 8980 refs['refs'] = refzarr = dict() 8981 else: 8982 refzarr = refs 8983 8984 for key, value in self._store.items(): 8985 if '.zarray' in key: 8986 level = int(key.split('/')[0]) if '/' in key else 0 8987 keyframe = self._data[level].keyframe 8988 value = json.loads(value) 8989 codec_id = compressors[keyframe.compression] 8990 if codec_id == 'imagecodecs_jpeg': 8991 # TODO: handle JPEG colorspaces 8992 tables = keyframe.jpegtables 8993 if tables is not None: 8994 import base64 8995 8996 tables = base64.b64encode(tables).decode() 8997 header = keyframe.jpegheader 8998 if header is not None: 8999 import base64 9000 9001 header = base64.b64encode(header).decode() 9002 colorspace_jpeg, colorspace_data = jpeg_decode_colorspace( 9003 keyframe.photometric, 9004 keyframe.planarconfig, 9005 keyframe.extrasamples, 9006 ) 9007 value['compressor'] = { 9008 'id': codec_id, 9009 'tables': tables, 9010 'header': header, 9011 'bitspersample': keyframe.bitspersample, 9012 'colorspace_jpeg': colorspace_jpeg, 9013 'colorspace_data': colorspace_data, 9014 } 9015 elif codec_id is not None: 9016 value['compressor'] = {'id': codec_id} 9017 if byteorder is not None: 9018 value['dtype'] = byteorder + value['dtype'][1:] 9019 value = ZarrStore._json(value) 9020 9021 refzarr[key] = value.decode() 9022 9023 if hasattr(arg, 'write'): 9024 fh = arg 9025 else: 9026 fh = open(arg, 'w') 9027 9028 if version == 1: 9029 fh.write(json.dumps(refs, indent=1).rsplit('}"', 1)[0] + '}"') 9030 indent = ' ' 9031 else: 9032 fh.write(json.dumps(refs, indent=1)[:-2]) 9033 indent = ' ' 9034 9035 for key, value in self._store.items(): 9036 if '.zarray' in key: 9037 value = json.loads(value) 9038 shape = value['shape'] 9039 chunks = value['chunks'] 9040 level = (key.split('/')[0] + '/') if '/' in key else '' 9041 for chunkindex in ZarrStore._ndindex(shape, chunks): 9042 key = level + chunkindex 9043 keyframe, page, _, offset, bytecount = self._parse_key(key) 9044 if page and self._chunkmode and offset is None: 9045 offset = page.dataoffsets[0] 9046 bytecount = keyframe.nbytes 9047 if offset and bytecount: 9048 fname = keyframe.parent.filehandle.name 9049 if version == 1: 9050 fname = templates[fname] 9051 else: 9052 fname = f'{url}{fname}' 9053 fh.write( 9054 f',\n{indent}"{key}": ' 9055 f'["{fname}", {offset}, {bytecount}]' 9056 ) 9057 9058 if version == 1: 9059 fh.write('\n }') 9060 fh.write('\n}') 9061 9062 if not hasattr(arg, 'write'): 9063 fh.close() 9064 9065 def _getitem(self, key): 9066 """Return chunk from file.""" 9067 keyframe, page, chunkindex, offset, bytecount = self._parse_key(key) 9068 9069 if self._chunkmode: 9070 chunks = keyframe.shape 9071 else: 9072 chunks = keyframe.chunks 9073 9074 if page is None or offset == 0 or bytecount == 0: 9075 chunk = ZarrStore._empty_chunk( 9076 chunks, keyframe.dtype, self._fillvalue 9077 ) 9078 if self._transform is not None: 9079 chunk = self._transform(chunk) 9080 return chunk 9081 9082 if self._chunkmode and offset is None: 9083 chunk = page.asarray(lock=self._filecache.lock) # maxworkers=1 ? 9084 if self._transform is not None: 9085 chunk = self._transform(chunk) 9086 return chunk 9087 9088 chunk = self._filecache.read(page.parent.filehandle, offset, bytecount) 9089 9090 decodeargs = {'_fullsize': True} 9091 if page.jpegtables is not None: 9092 decodeargs['jpegtables'] = page.jpegtables 9093 if keyframe.jpegheader is not None: 9094 decodeargs['jpegheader'] = keyframe.jpegheader 9095 9096 chunk = keyframe.decode(chunk, chunkindex, **decodeargs)[0] 9097 if self._transform is not None: 9098 chunk = self._transform(chunk) 9099 9100 if chunk.size != product(chunks): 9101 raise RuntimeError 9102 return chunk # .tobytes() 9103 9104 def _parse_key(self, key): 9105 """Return keyframe, page, index, offset, and bytecount from key.""" 9106 if len(self._data) > 1: 9107 # multiscales 9108 level, key = key.split('/') 9109 series = self._data[int(level)] 9110 else: 9111 series = self._data[0] 9112 keyframe = series.keyframe 9113 pageindex, chunkindex = self._indices(key, series) 9114 if pageindex > 0 and len(series) == 1: 9115 # truncated ImageJ, STK, or shaped 9116 if series.offset is None: 9117 raise RuntimeError('truncated series is not contiguous') 9118 page = series[0] 9119 if page is None: 9120 return keyframe, None, chunkindex, 0, 0 9121 offset = pageindex * page.size * page.dtype.itemsize 9122 offset += page.dataoffsets[chunkindex] 9123 if self._chunkmode: 9124 bytecount = page.size * page.dtype.itemsize 9125 return page.keyframe, page, chunkindex, offset, bytecount 9126 elif self._chunkmode: 9127 with self._filecache.lock: 9128 page = series[pageindex] 9129 if page is None: 9130 return keyframe, None, None, 0, 0 9131 return page.keyframe, page, None, None, None 9132 else: 9133 with self._filecache.lock: 9134 page = series[pageindex] 9135 if page is None: 9136 return keyframe, None, chunkindex, 0, 0 9137 offset = page.dataoffsets[chunkindex] 9138 bytecount = page.databytecounts[chunkindex] 9139 return page.keyframe, page, chunkindex, offset, bytecount 9140 9141 def _indices(self, key, series): 9142 """Return page and strile indices from zarr chunk index.""" 9143 keyframe = series.keyframe 9144 shape = series.get_shape(self._squeeze) 9145 indices = [int(i) for i in key.split('.')] 9146 assert len(indices) == len(shape) 9147 if self._chunkmode: 9148 chunked = (1,) * len(keyframe.shape) 9149 else: 9150 chunked = keyframe.chunked 9151 p = 1 9152 for i, s in enumerate(shape[::-1]): 9153 p *= s 9154 if p == keyframe.size: 9155 i = len(indices) - i - 1 9156 frames_indices = indices[:i] 9157 strile_indices = indices[i:] 9158 frames_chunked = shape[:i] 9159 strile_chunked = list(shape[i:]) # updated later 9160 break 9161 else: 9162 raise RuntimeError 9163 if len(strile_chunked) == len(keyframe.shape): 9164 strile_chunked = chunked 9165 else: 9166 # get strile_chunked including singleton dimensions 9167 i = len(strile_indices) - 1 9168 j = len(keyframe.shape) - 1 9169 while True: 9170 if strile_chunked[i] == keyframe.shape[j]: 9171 strile_chunked[i] = chunked[j] 9172 i -= 1 9173 j -= 1 9174 elif strile_chunked[i] == 1: 9175 i -= 1 9176 else: 9177 raise RuntimeError('shape does not match page shape') 9178 if i < 0 or j < 0: 9179 break 9180 assert product(strile_chunked) == product(chunked) 9181 if len(frames_indices) > 0: 9182 frameindex = int( 9183 numpy.ravel_multi_index(frames_indices, frames_chunked) 9184 ) 9185 else: 9186 frameindex = 0 9187 if len(strile_indices) > 0: 9188 strileindex = int( 9189 numpy.ravel_multi_index(strile_indices, strile_chunked) 9190 ) 9191 else: 9192 strileindex = 0 9193 return frameindex, strileindex 9194 9195 @staticmethod 9196 def _chunks(chunks, shape): 9197 """Return chunks with same length as shape.""" 9198 ndim = len(shape) 9199 if ndim == 0: 9200 return () # empty array 9201 if 0 in shape: 9202 return (1,) * ndim 9203 newchunks = [] 9204 i = ndim - 1 9205 j = len(chunks) - 1 9206 while True: 9207 if j < 0: 9208 newchunks.append(1) 9209 i -= 1 9210 elif shape[i] > 1 and chunks[j] > 1: 9211 newchunks.append(chunks[j]) 9212 i -= 1 9213 j -= 1 9214 elif shape[i] == chunks[j]: # both 1 9215 newchunks.append(1) 9216 i -= 1 9217 j -= 1 9218 elif shape[i] == 1: 9219 newchunks.append(1) 9220 i -= 1 9221 elif chunks[j] == 1: 9222 newchunks.append(1) 9223 j -= 1 9224 else: 9225 raise RuntimeError 9226 if i < 0 or ndim == len(newchunks): 9227 break 9228 # assert ndim == len(newchunks) 9229 return tuple(newchunks[::-1]) 9230 9231 9232class ZarrFileSequenceStore(ZarrStore): 9233 """Zarr storage interface to image data in FileSequence.""" 9234 9235 def __init__(self, arg, fillvalue=None, chunkmode=None, **kwargs): 9236 """Initialize Zarr storage from FileSequence.""" 9237 super().__init__(fillvalue=fillvalue, chunkmode=chunkmode) 9238 9239 if self._chunkmode not in (0, 3): 9240 raise NotImplementedError(f'{self._chunkmode!r} not implemented') 9241 9242 if not isinstance(arg, FileSequence): 9243 raise TypeError('not a FileSequence') 9244 9245 if arg._container: 9246 raise NotImplementedError('cannot open container as zarr storage') 9247 9248 image = arg.imread(arg.files[0], **kwargs) 9249 dtype = image.dtype 9250 chunks = image.shape 9251 shape = arg.shape + chunks 9252 chunks = (1,) * len(arg.shape) + chunks 9253 9254 self._store['.zattrs'] = ZarrStore._json({}) 9255 self._store['.zarray'] = ZarrStore._json( 9256 { 9257 'zarr_format': 2, 9258 'shape': shape, 9259 'chunks': chunks, 9260 'dtype': ZarrStore._dtype(dtype), 9261 'compressor': None, 9262 'fill_value': ZarrStore._value(fillvalue, dtype), 9263 'order': 'C', 9264 'filters': None, 9265 } 9266 ) 9267 9268 self._kwargs = kwargs 9269 self._imread = arg.imread 9270 self._lookup = dict(zip(arg.indices, arg.files)) 9271 self._chunks = image.shape 9272 self._dtype = image.dtype 9273 self._cache = {arg.indices[0]: image} # TODO: cache MRU number chunks 9274 self._commonpath = arg.commonpath() 9275 9276 def _getitem(self, key): 9277 """Return chunk from file.""" 9278 indices = tuple(int(i) for i in key.split('.')) 9279 indices = indices[: -len(self._chunks)] 9280 if indices in self._cache: 9281 return self._cache[indices] 9282 self._cache.clear() 9283 filename = self._lookup.get(indices, None) 9284 if filename is None: 9285 chunk = ZarrStore._empty_chunk( 9286 self._chunks, self._dtype, self._fillvalue 9287 ) 9288 else: 9289 chunk = self._imread(filename, **self._kwargs).tobytes() 9290 self._cache[indices] = chunk 9291 return chunk 9292 9293 def close(self): 9294 """Clear chunk cache.""" 9295 self._cache.clear() 9296 9297 def write_fsspec(self, arg, url, codec_id=None, version=None): 9298 """Write fsspec ReferenceFileSystem as JSON to file. 9299 9300 Url is the remote location of the files without the file names. 9301 9302 """ 9303 from urllib.parse import quote 9304 9305 kwargs = self._kwargs.copy() 9306 9307 if codec_id is not None: 9308 pass 9309 elif self._imread == imread: 9310 codec_id = 'tifffile' 9311 elif 'imagecodecs.' in self._imread.__module__: 9312 if ( 9313 self._imread.__name__ != 'imread' 9314 or 'codec' not in self._kwargs 9315 ): 9316 raise ValueError('can not determine codec_id') 9317 codec = kwargs.pop('codec') 9318 if isinstance(codec, (list, tuple)): 9319 codec = codec[0] 9320 if callable(codec): 9321 codec = codec.__name__.split('_')[0] 9322 codec_id = { 9323 'avif': 'imagecodecs_avif', 9324 'gif': 'imagecodecs_gif', 9325 'jpeg': 'imagecodecs_jpeg', 9326 'jpeg8': 'imagecodecs_jpeg', 9327 'jpeg12': 'imagecodecs_jpeg', 9328 'jpeg2k': 'imagecodecs_jpeg2k', 9329 'jpegls': 'imagecodecs_jpegls', 9330 'jpegxl': 'imagecodecs_jpegxl', 9331 'jpegxr': 'imagecodecs_jpegxr', 9332 'ljpeg': 'imagecodecs_ljpeg', 9333 # 'npy': 'imagecodecs_npy', 9334 'png': 'imagecodecs_png', 9335 'tiff': 'imagecodecs_tiff', 9336 'webp': 'imagecodecs_webp', 9337 'zfp': 'imagecodecs_zfp', 9338 }[codec] 9339 else: 9340 raise ValueError('can not determine codec_id') 9341 9342 if url is None: 9343 url = '' 9344 elif url and url[-1] != '/': 9345 url += '/' 9346 9347 refs = dict() 9348 if version == 1: 9349 refs['version'] = 1 9350 refs['templates'] = {'u': url} 9351 refs['gen'] = [] 9352 refs['refs'] = refzarr = dict() 9353 url = '{{u}}' 9354 else: 9355 refzarr = refs 9356 9357 for key, value in self._store.items(): 9358 if '.zarray' in key: 9359 value = json.loads(value) 9360 # TODO: make kwargs serializable 9361 value['compressor'] = {'id': codec_id, **kwargs} 9362 value = ZarrStore._json(value) 9363 refzarr[key] = value.decode() 9364 9365 if hasattr(arg, 'write'): 9366 fh = arg 9367 else: 9368 fh = open(arg, 'w') 9369 9370 if version == 1: 9371 fh.write(json.dumps(refs, indent=1).rsplit('}"', 1)[0] + '}"') 9372 indent = ' ' 9373 else: 9374 fh.write(json.dumps(refs, indent=1)[:-2]) 9375 indent = ' ' 9376 9377 prefix = len(self._commonpath) 9378 9379 for key, value in self._store.items(): 9380 if '.zarray' in key: 9381 value = json.loads(value) 9382 shape = value['shape'] 9383 chunks = value['chunks'] 9384 for key in ZarrStore._ndindex(shape, chunks): 9385 indices = tuple(int(i) for i in key.split('.')) 9386 indices = indices[: -len(self._chunks)] 9387 filename = self._lookup.get(indices, None) 9388 if filename is not None: 9389 filename = quote(filename[prefix:].replace('\\', '/')) 9390 if filename[0] == '/': 9391 filename = filename[1:] 9392 fh.write(f',\n{indent}"{key}": ["{url}{filename}"]') 9393 9394 if version == 1: 9395 fh.write('\n }') 9396 fh.write('\n}') 9397 9398 if not hasattr(arg, 'write'): 9399 fh.close() 9400 9401 9402class FileSequence: 9403 """Series of files containing array data of compatible shape and data type. 9404 9405 Attributes 9406 ---------- 9407 files : list 9408 List of file names. 9409 shape : tuple 9410 Shape of file series. Excludes shape of individual arrays. 9411 axes : str 9412 Labels of axes in shape. 9413 indices : tuple of tuples 9414 ND indices of files in shape. 9415 9416 """ 9417 9418 def __init__( 9419 self, 9420 imread, 9421 files, 9422 container=None, 9423 sort=None, 9424 pattern=None, 9425 axesorder=None, 9426 ): 9427 r"""Initialize instance from multiple files. 9428 9429 Parameters 9430 ---------- 9431 imread : function or class 9432 Array read function or class with asarray function returning numpy 9433 array from single file. 9434 files : str, path-like, or sequence thereof 9435 Glob filename pattern or sequence of file names. Default: \*. 9436 Binary streams are not supported. 9437 container : str or container instance 9438 Name or open instance of ZIP file in which files are stored. 9439 sort : function 9440 Sort function used to sort file names when 'files' is a pattern. 9441 The default (None) is natural_sorted. Use sort=False to disable 9442 sorting. 9443 pattern : str 9444 Regular expression pattern that matches axes and sequence indices 9445 in file names. By default (None), no pattern matching is performed. 9446 Axes can be specified by matching groups preceding the index groups 9447 in the file name, be provided as group names for the index groups, 9448 or be omitted. The predefined 'axes' pattern matches Olympus OIF 9449 and Leica TIFF series. 9450 axesorder : sequence of int 9451 Indices of axes in pattern. 9452 9453 """ 9454 if files is None: 9455 files = '*' 9456 if sort is None: 9457 sort = natural_sorted 9458 self._container = container 9459 if container: 9460 import fnmatch 9461 9462 if isinstance(container, (str, os.PathLike)): 9463 import zipfile 9464 9465 self._container = zipfile.ZipFile(container) 9466 elif not hasattr(self._container, 'open'): 9467 raise ValueError('invalid container') 9468 if isinstance(files, str): 9469 files = fnmatch.filter(self._container.namelist(), files) 9470 if sort: 9471 files = sort(files) 9472 elif isinstance(files, os.PathLike): 9473 files = [os.fspath(files)] 9474 elif isinstance(files, str): 9475 files = glob.glob(files) 9476 if sort: 9477 files = sort(files) 9478 9479 files = [os.fspath(f) for f in files] 9480 if not files: 9481 raise ValueError('no files found') 9482 9483 if hasattr(imread, 'asarray'): 9484 # redefine imread to use asarray from imread class 9485 if not callable(imread.asarray): 9486 raise ValueError('invalid imread function') 9487 _imread_ = imread 9488 9489 def imread(fname, **kwargs): 9490 with _imread_(fname) as handle: 9491 return handle.asarray(**kwargs) 9492 9493 elif not callable(imread): 9494 raise ValueError('invalid imread function') 9495 9496 if container: 9497 # redefine imread to read from container 9498 _imread_ = imread 9499 9500 def imread(fname, **kwargs): 9501 with self._container.open(fname) as handle1: 9502 with io.BytesIO(handle1.read()) as handle2: 9503 return _imread_(handle2, **kwargs) 9504 9505 axes = 'I' 9506 shape = (len(files),) 9507 indices = tuple((i,) for i in range(len(files))) 9508 startindex = (0,) 9509 9510 pattern = TIFF.FILE_PATTERNS.get(pattern, pattern) 9511 if pattern: 9512 try: 9513 axes, shape, indices, startindex = parse_filenames( 9514 files, pattern, axesorder 9515 ) 9516 except ValueError as exc: 9517 log_warning( 9518 f'FileSequence: failed to parse file names ({exc})' 9519 ) 9520 axesorder = None 9521 9522 if product(shape) != len(files): 9523 log_warning( 9524 'FileSequence: files are missing. Missing data are zeroed' 9525 ) 9526 9527 self.imread = imread 9528 self.files = files 9529 self.pattern = pattern 9530 self.axes = axes.upper() 9531 self.shape = shape 9532 self.indices = indices 9533 self._startindex = startindex 9534 9535 def __str__(self): 9536 """Return string with information about file FileSequence.""" 9537 file = str(self._container) if self._container else self.files[0] 9538 file = os.path.split(file)[-1] 9539 return '\n '.join( 9540 ( 9541 self.__class__.__name__, 9542 file, 9543 f'files: {len(self.files)}', 9544 'shape: {}'.format(', '.join(str(i) for i in self.shape)), 9545 f'axes: {self.axes}', 9546 ) 9547 ) 9548 9549 def __len__(self): 9550 return len(self.files) 9551 9552 def __enter__(self): 9553 return self 9554 9555 def __exit__(self, exc_type, exc_value, traceback): 9556 self.close() 9557 9558 def close(self): 9559 if self._container: 9560 self._container.close() 9561 self._container = None 9562 9563 def asarray(self, file=None, ioworkers=1, out=None, **kwargs): 9564 """Read image data from files and return as numpy array. 9565 9566 Raise IndexError or ValueError if array shapes do not match. 9567 9568 Parameters 9569 ---------- 9570 file : int or None 9571 Index or name of single file to read. 9572 ioworkers : int or None 9573 Maximum number of threads to execute the array read function 9574 asynchronously. Default: 1. 9575 If None, default to the number of processors multiplied by 5. 9576 Using threads can significantly improve runtime when 9577 reading many small files from a network share. 9578 out : numpy.ndarray, str, or file-like object 9579 Buffer where image data are saved. 9580 If None (default), a new array is created. 9581 If numpy.ndarray, a writable array of compatible dtype and shape. 9582 If 'memmap', create a memory-mapped array in a temporary file. 9583 If str or open file, the file name or file object used to 9584 create a memory-map to an array stored in a binary file on disk. 9585 kwargs : dict 9586 Additional parameters passed to the array read function. 9587 9588 """ 9589 if file is not None: 9590 if isinstance(file, int): 9591 return self.imread(self.files[file], **kwargs) 9592 return self.imread(file, **kwargs) 9593 9594 im = self.imread(self.files[0], **kwargs) 9595 shape = self.shape + im.shape 9596 result = create_output(out, shape, dtype=im.dtype) 9597 result = result.reshape(-1, *im.shape) 9598 9599 def func(index, fname): 9600 """Read single image from file into result.""" 9601 index = int(numpy.ravel_multi_index(index, self.shape)) 9602 im = self.imread(fname, **kwargs) 9603 result[index] = im 9604 9605 if len(self.files) < 2: 9606 ioworkers = 1 9607 elif ioworkers is None or ioworkers < 1: 9608 import multiprocessing 9609 9610 ioworkers = max(multiprocessing.cpu_count() * 5, 1) 9611 9612 if ioworkers < 2: 9613 for index, fname in zip(self.indices, self.files): 9614 func(index, fname) 9615 else: 9616 with ThreadPoolExecutor(ioworkers) as executor: 9617 for _ in executor.map(func, self.indices, self.files): 9618 pass 9619 9620 result.shape = shape 9621 return result 9622 9623 def aszarr(self, **kwargs): 9624 """Return image data from files as zarr storage.""" 9625 return ZarrFileSequenceStore(self, **kwargs) 9626 9627 def commonpath(self): 9628 """Return longest common sub-path of each file in sequence.""" 9629 if len(self.files) == 1: 9630 commonpath = os.path.dirname(self.files[0]) 9631 else: 9632 commonpath = os.path.commonpath(self.files) 9633 return commonpath 9634 9635 9636class TiffSequence(FileSequence): 9637 """Series of TIFF files.""" 9638 9639 def __init__( 9640 self, 9641 files=None, 9642 container=None, 9643 sort=None, 9644 pattern=None, 9645 axesorder=None, 9646 imread=imread, 9647 ): 9648 """Initialize instance from multiple TIFF files.""" 9649 super().__init__( 9650 imread, 9651 '*.tif' if files is None else files, 9652 container=container, 9653 sort=sort, 9654 pattern=pattern, 9655 axesorder=axesorder, 9656 ) 9657 9658 9659class FileHandle: 9660 """Binary file handle. 9661 9662 A limited, special purpose file handle that can: 9663 9664 * handle embedded files (e.g. for LSM within LSM files) 9665 * re-open closed files (for multi-file formats, such as OME-TIFF) 9666 * read and write numpy arrays and records from file like objects 9667 9668 Only 'rb', 'r+b', and 'wb' modes are supported. Concurrently reading and 9669 writing of the same stream is untested. 9670 9671 When initialized from another file handle, do not use it unless this 9672 FileHandle is closed. 9673 9674 Attributes 9675 ---------- 9676 name : str 9677 Name of the file. 9678 path : str 9679 Absolute path to file. 9680 size : int 9681 Size of file in bytes. 9682 is_file : bool 9683 If True, file has a fileno and can be memory-mapped. 9684 9685 All attributes are read-only. 9686 9687 """ 9688 9689 __slots__ = ( 9690 '_fh', 9691 '_file', 9692 '_mode', 9693 '_name', 9694 '_dir', 9695 '_lock', 9696 '_offset', 9697 '_size', 9698 '_close', 9699 'is_file', 9700 ) 9701 9702 def __init__(self, file, mode=None, name=None, offset=None, size=None): 9703 """Initialize file handle from file name or another file handle. 9704 9705 Parameters 9706 ---------- 9707 file : str, path-like, binary stream, or FileHandle 9708 File name or seekable binary stream, such as an open file 9709 or BytesIO. 9710 mode : str 9711 File open mode in case 'file' is a file name. Must be 'rb', 'r+b', 9712 or 'wb'. Default is 'rb'. 9713 name : str 9714 Optional name of file in case 'file' is a binary stream. 9715 offset : int 9716 Optional start position of embedded file. By default, this is 9717 the current file position. 9718 size : int 9719 Optional size of embedded file. By default, this is the number 9720 of bytes from the 'offset' to the end of the file. 9721 9722 """ 9723 self._fh = None 9724 self._file = file 9725 self._mode = 'rb' if mode is None else mode 9726 self._name = name 9727 self._dir = '' 9728 self._offset = offset 9729 self._size = size 9730 self._close = True 9731 self.is_file = None 9732 self._lock = NullContext() 9733 self.open() 9734 9735 def open(self): 9736 """Open or re-open file.""" 9737 if self._fh is not None: 9738 return # file is open 9739 9740 if isinstance(self._file, os.PathLike): 9741 self._file = os.fspath(self._file) 9742 if isinstance(self._file, str): 9743 # file name 9744 self._file = os.path.realpath(self._file) 9745 self._dir, self._name = os.path.split(self._file) 9746 self._fh = open(self._file, self._mode) 9747 self._close = True 9748 if self._offset is None: 9749 self._offset = 0 9750 elif isinstance(self._file, FileHandle): 9751 # FileHandle 9752 self._fh = self._file._fh 9753 if self._offset is None: 9754 self._offset = 0 9755 self._offset += self._file._offset 9756 self._close = False 9757 if not self._name: 9758 if self._offset: 9759 name, ext = os.path.splitext(self._file._name) 9760 self._name = f'{name}@{self._offset}{ext}' 9761 else: 9762 self._name = self._file._name 9763 if self._mode and self._mode != self._file._mode: 9764 raise ValueError('FileHandle has wrong mode') 9765 self._mode = self._file._mode 9766 self._dir = self._file._dir 9767 elif hasattr(self._file, 'seek'): 9768 # binary stream: open file, BytesIO 9769 try: 9770 self._file.tell() 9771 except Exception: 9772 raise ValueError('binary stream is not seekable') 9773 self._fh = self._file 9774 if self._offset is None: 9775 self._offset = self._file.tell() 9776 self._close = False 9777 if not self._name: 9778 try: 9779 self._dir, self._name = os.path.split(self._fh.name) 9780 except AttributeError: 9781 self._name = 'Unnamed binary stream' 9782 try: 9783 self._mode = self._fh.mode 9784 except AttributeError: 9785 pass 9786 else: 9787 raise ValueError( 9788 'the first parameter must be a file name, ' 9789 'seekable binary stream, or FileHandle' 9790 ) 9791 9792 if self._offset: 9793 self._fh.seek(self._offset) 9794 9795 if self._size is None: 9796 pos = self._fh.tell() 9797 self._fh.seek(self._offset, os.SEEK_END) 9798 self._size = self._fh.tell() 9799 self._fh.seek(pos) 9800 9801 if self.is_file is None: 9802 try: 9803 self._fh.fileno() 9804 self.is_file = True 9805 except Exception: 9806 self.is_file = False 9807 9808 def close(self): 9809 """Close file.""" 9810 if self._close and self._fh: 9811 self._fh.close() 9812 self._fh = None 9813 9814 def tell(self): 9815 """Return file's current position.""" 9816 return self._fh.tell() - self._offset 9817 9818 def seek(self, offset, whence=0): 9819 """Set file's current position.""" 9820 if self._offset: 9821 if whence == 0: 9822 self._fh.seek(self._offset + offset, whence) 9823 return 9824 if whence == 2 and self._size > 0: 9825 self._fh.seek(self._offset + self._size + offset, 0) 9826 return 9827 self._fh.seek(offset, whence) 9828 9829 def read(self, size=-1): 9830 """Read 'size' bytes from file, or until EOF is reached.""" 9831 if size < 0 and self._offset: 9832 size = self._size 9833 return self._fh.read(size) 9834 9835 def readinto(self, b): 9836 """Read up to len(b) bytes into b and return number of bytes read.""" 9837 return self._fh.readinto(b) 9838 9839 def write(self, bytestring): 9840 """Write bytes to file.""" 9841 return self._fh.write(bytestring) 9842 9843 def flush(self): 9844 """Flush write buffers if applicable.""" 9845 return self._fh.flush() 9846 9847 def memmap_array(self, dtype, shape, offset=0, mode='r', order='C'): 9848 """Return numpy.memmap of data stored in file.""" 9849 if not self.is_file: 9850 raise ValueError('cannot memory-map file without fileno') 9851 return numpy.memmap( 9852 self._fh, 9853 dtype=dtype, 9854 mode=mode, 9855 offset=self._offset + offset, 9856 shape=shape, 9857 order=order, 9858 ) 9859 9860 def read_array(self, dtype, count=-1, out=None): 9861 """Return numpy array from file in native byte order.""" 9862 dtype = numpy.dtype(dtype) 9863 9864 if count < 0: 9865 nbytes = self._size if out is None else out.nbytes 9866 count = nbytes // dtype.itemsize 9867 else: 9868 nbytes = count * dtype.itemsize 9869 9870 result = numpy.empty(count, dtype) if out is None else out 9871 9872 if result.nbytes != nbytes: 9873 raise ValueError('size mismatch') 9874 9875 try: 9876 n = self._fh.readinto(result) 9877 except AttributeError: 9878 result[:] = numpy.frombuffer(self._fh.read(nbytes), dtype).reshape( 9879 result.shape 9880 ) 9881 n = nbytes 9882 9883 if n != nbytes: 9884 raise ValueError(f'failed to read {nbytes} bytes') 9885 9886 if not result.dtype.isnative: 9887 if not dtype.isnative: 9888 result.byteswap(True) 9889 result = result.newbyteorder() 9890 elif result.dtype.isnative != dtype.isnative: 9891 result.byteswap(True) 9892 9893 if out is not None: 9894 if hasattr(out, 'flush'): 9895 out.flush() 9896 9897 return result 9898 9899 def read_record(self, dtype, shape=1, byteorder=None): 9900 """Return numpy record from file.""" 9901 rec = numpy.rec 9902 try: 9903 record = rec.fromfile(self._fh, dtype, shape, byteorder=byteorder) 9904 except Exception: 9905 dtype = numpy.dtype(dtype) 9906 if shape is None: 9907 shape = self._size // dtype.itemsize 9908 size = product(sequence(shape)) * dtype.itemsize 9909 # data = bytearray(size) 9910 # n = self._fh.readinto(data) 9911 # data = data[:n] 9912 # TODO: record is not writable 9913 data = self._fh.read(size) 9914 record = rec.fromstring(data, dtype, shape, byteorder=byteorder) 9915 return record[0] if shape == 1 else record 9916 9917 def write_empty(self, size): 9918 """Append size bytes to file. Position must be at end of file.""" 9919 if size < 1: 9920 return 9921 self._fh.seek(size - 1, os.SEEK_CUR) 9922 self._fh.write(b'\x00') 9923 9924 def write_array(self, data): 9925 """Write numpy array to binary file.""" 9926 try: 9927 # writing non-contiguous arrays is very slow 9928 numpy.ascontiguousarray(data).tofile(self._fh) 9929 except Exception: 9930 # numpy can't write to BytesIO 9931 self._fh.write(data.tobytes()) 9932 9933 def read_segments( 9934 self, 9935 offsets, 9936 bytecounts, 9937 indices=None, 9938 sort=True, 9939 lock=None, 9940 buffersize=None, 9941 flat=True, 9942 ): 9943 """Return iterator over segments read from file and their indices. 9944 9945 The purpose of this function is to 9946 9947 * reduce small or random reads 9948 * reduce acquiring reentrant locks 9949 * synchronize seeks and reads 9950 * limit the size of segments read into memory at once 9951 (ThreadPoolExecutor.map is not collecting iterables lazily). 9952 9953 Parameters 9954 ---------- 9955 offsets, bytecounts : sequence of int 9956 offsets and bytecounts of the segments to read from file. 9957 indices : sequence of int 9958 Indices of the segments in the image. Default: range(len(offsets)). 9959 sort : bool 9960 If True (default), segments are read from file in the order of 9961 their offsets. 9962 lock: 9963 A reentrant lock used to synchronize seeks and reads. 9964 buffersize : int 9965 Approximate number of bytes to read from file in one pass. 9966 Default: 64 MB. 9967 flat : bool 9968 If True (default), return an iterator over individual 9969 (segment, index) tuples. Else return an iterator over a list 9970 of (segment, index) tuples that were acquired in one pass. 9971 9972 Returns 9973 ------- 9974 items : (bytes, int) or [(bytes, int)] 9975 Iterator over individual or lists of (segment, index) tuples. 9976 9977 """ 9978 # TODO: Cythonize this? 9979 length = len(offsets) 9980 if length < 1: 9981 return 9982 if length == 1: 9983 index = 0 if indices is None else indices[0] 9984 if bytecounts[index] > 0 and offsets[index] > 0: 9985 if lock is None: 9986 lock = self._lock 9987 with lock: 9988 self.seek(offsets[index]) 9989 data = self._fh.read(bytecounts[index]) 9990 else: 9991 data = None 9992 yield (data, index) if flat else [(data, index)] 9993 return 9994 9995 if lock is None: 9996 lock = self._lock 9997 if buffersize is None: 9998 buffersize = 67108864 # 2 ** 26, 64 MB 9999 10000 if indices is None: 10001 segments = [(i, offsets[i], bytecounts[i]) for i in range(length)] 10002 else: 10003 segments = [ 10004 (indices[i], offsets[i], bytecounts[i]) for i in range(length) 10005 ] 10006 if sort: 10007 segments = sorted(segments, key=lambda x: x[1]) 10008 10009 iscontig = True 10010 for i in range(length - 1): 10011 _, offset, bytecount = segments[i] 10012 nextoffset = segments[i + 1][1] 10013 if offset == 0 or bytecount == 0 or nextoffset == 0: 10014 continue 10015 if offset + bytecount != nextoffset: 10016 iscontig = False 10017 break 10018 10019 seek = self.seek 10020 read = self._fh.read 10021 10022 if iscontig: 10023 # consolidate reads 10024 i = 0 10025 while i < length: 10026 j = i 10027 offset = None 10028 bytecount = 0 10029 while bytecount < buffersize and i < length: 10030 _, o, b = segments[i] 10031 if o > 0 and b > 0: 10032 if offset is None: 10033 offset = o 10034 bytecount += b 10035 i += 1 10036 10037 if offset is None: 10038 data = None 10039 else: 10040 with lock: 10041 seek(offset) 10042 data = read(bytecount) 10043 start = 0 10044 stop = 0 10045 result = [] 10046 while j < i: 10047 index, offset, bytecount = segments[j] 10048 if offset > 0 and bytecount > 0: 10049 stop += bytecount 10050 result.append((data[start:stop], index)) 10051 start = stop 10052 else: 10053 result.append((None, index)) 10054 j += 1 10055 if flat: 10056 yield from result 10057 else: 10058 yield result 10059 return 10060 10061 i = 0 10062 while i < length: 10063 result = [] 10064 size = 0 10065 with lock: 10066 while size < buffersize and i < length: 10067 index, offset, bytecount = segments[i] 10068 if offset > 0 and bytecount > 0: 10069 seek(offset) 10070 result.append((read(bytecount), index)) 10071 # buffer = bytearray(bytecount) 10072 # n = fh.readinto(buffer) 10073 # data.append(buffer[:n]) 10074 size += bytecount 10075 else: 10076 result.append((None, index)) 10077 i += 1 10078 if flat: 10079 yield from result 10080 else: 10081 yield result 10082 10083 def __enter__(self): 10084 return self 10085 10086 def __exit__(self, exc_type, exc_value, traceback): 10087 self.close() 10088 10089 def __getattr__(self, name): 10090 """Return attribute from underlying file object.""" 10091 if self._offset: 10092 warnings.warn( 10093 f'FileHandle: {name!r} not implemented for embedded files', 10094 UserWarning, 10095 ) 10096 return getattr(self._fh, name) 10097 10098 @property 10099 def name(self): 10100 return self._name 10101 10102 @property 10103 def dirname(self): 10104 return self._dir 10105 10106 @property 10107 def path(self): 10108 return os.path.join(self._dir, self._name) 10109 10110 @property 10111 def size(self): 10112 return self._size 10113 10114 @property 10115 def closed(self): 10116 return self._fh is None 10117 10118 @property 10119 def lock(self): 10120 """Return current lock instance.""" 10121 return self._lock 10122 10123 @lock.setter 10124 def lock(self, value): 10125 if bool(value) == isinstance(self._lock, NullContext): 10126 self._lock = threading.RLock() if value else NullContext() 10127 10128 @property 10129 def has_lock(self): 10130 """Return if a RLock is used.""" 10131 return not isinstance(self._lock, NullContext) 10132 10133 10134class FileCache: 10135 """Keep FileHandles open.""" 10136 10137 __slots__ = ('files', 'keep', 'past', 'lock', 'size') 10138 10139 def __init__(self, size=None, lock=None): 10140 """Initialize open file cache.""" 10141 self.past = [] # FIFO of opened files 10142 self.files = {} # refcounts of opened file handles 10143 self.keep = set() # files to keep open 10144 self.lock = NullContext() if lock is None else lock 10145 self.size = 8 if size is None else int(size) 10146 10147 def __len__(self): 10148 """Return number of open files.""" 10149 return len(self.files) 10150 10151 def open(self, filehandle): 10152 """Open file, re-open if necessary.""" 10153 with self.lock: 10154 if filehandle in self.files: 10155 self.files[filehandle] += 1 10156 elif filehandle.closed: 10157 filehandle.open() 10158 self.files[filehandle] = 1 10159 self.past.append(filehandle) 10160 else: 10161 self.files[filehandle] = 2 10162 self.keep.add(filehandle) 10163 self.past.append(filehandle) 10164 10165 def close(self, filehandle): 10166 """Close least recently used open files.""" 10167 with self.lock: 10168 if filehandle in self.files: 10169 self.files[filehandle] -= 1 10170 self._trim() 10171 10172 def clear(self): 10173 """Close all opened files if not in use when opened first.""" 10174 with self.lock: 10175 for filehandle, refcount in list(self.files.items()): 10176 if filehandle not in self.keep: 10177 filehandle.close() 10178 del self.files[filehandle] 10179 del self.past[self.past.index(filehandle)] 10180 10181 def read(self, filehandle, offset, bytecount, whence=0): 10182 """Return bytes read from binary file.""" 10183 with self.lock: 10184 b = filehandle not in self.files 10185 if b: 10186 if filehandle.closed: 10187 filehandle.open() 10188 self.files[filehandle] = 0 10189 else: 10190 self.files[filehandle] = 1 10191 self.keep.add(filehandle) 10192 self.past.append(filehandle) 10193 filehandle.seek(offset, whence) 10194 data = filehandle.read(bytecount) 10195 if b: 10196 self._trim() 10197 return data 10198 10199 def _trim(self): 10200 """Trim the file cache.""" 10201 index = 0 10202 size = len(self.past) 10203 while size > self.size and index < size: 10204 filehandle = self.past[index] 10205 if filehandle not in self.keep and self.files[filehandle] <= 0: 10206 filehandle.close() 10207 del self.files[filehandle] 10208 del self.past[index] 10209 size -= 1 10210 else: 10211 index += 1 10212 10213 10214class NullContext: 10215 """Null context manager. 10216 10217 >>> with NullContext(): 10218 ... pass 10219 10220 """ 10221 10222 __slots = () 10223 10224 def __enter__(self): 10225 return self 10226 10227 def __exit__(self, exc_type, exc_value, traceback): 10228 pass 10229 10230 10231class Timer: 10232 """Stopwatch for timing execution speed.""" 10233 10234 __slots__ = ('started', 'stopped', 'duration') 10235 10236 clock = time.perf_counter 10237 10238 def __init__(self, message=None, end=' '): 10239 """Initialize timer and print message.""" 10240 if message is not None: 10241 print(message, end=end, flush=True) 10242 self.duration = 0 10243 self.started = self.stopped = Timer.clock() 10244 10245 def start(self, message=None, end=' '): 10246 """Start timer and return current time.""" 10247 if message is not None: 10248 print(message, end=end, flush=True) 10249 self.duration = 0 10250 self.started = self.stopped = Timer.clock() 10251 return self.started 10252 10253 def stop(self, message=None, end=' '): 10254 """Return duration of timer till start.""" 10255 self.stopped = Timer.clock() 10256 if message is not None: 10257 print(message, end=end, flush=True) 10258 self.duration = self.stopped - self.started 10259 return self.duration 10260 10261 def print(self, message=None, end=None): 10262 """Print duration from timer start till last stop or now.""" 10263 msg = str(self) 10264 if message is not None: 10265 print(message, end=' ') 10266 print(msg, end=end, flush=True) 10267 10268 def __str__(self): 10269 """Return duration from timer start till last stop or now as string.""" 10270 if self.duration <= 0: 10271 # not stopped 10272 duration = Timer.clock() - self.started 10273 else: 10274 duration = self.duration 10275 s = str(datetime.timedelta(seconds=duration)) 10276 i = 0 10277 while i < len(s) and s[i : i + 2] in '0:0010203040506070809': 10278 i += 1 10279 if s[i : i + 1] == ':': 10280 i += 1 10281 return f'{s[i:]} s' 10282 10283 def __enter__(self): 10284 return self 10285 10286 def __exit__(self, exc_type, exc_value, traceback): 10287 self.print() 10288 10289 10290class OmeXmlError(Exception): 10291 """Exception to indicate invalid OME-XML or unsupported cases.""" 10292 10293 10294class OmeXml: 10295 """OME-TIFF XML.""" 10296 10297 def __init__(self, **metadata): 10298 """Create a new instance. 10299 10300 Creator : str (optional) 10301 Name of the creating application. Default 'tifffile.py'. 10302 UUID : str (optional) 10303 Unique identifier. 10304 10305 """ 10306 if 'OME' in metadata: 10307 metadata = metadata['OME'] 10308 10309 self.ifd = 0 10310 self.images = [] 10311 self.annotations = [] 10312 self.elements = [] 10313 # TODO: parse other OME elements from metadata 10314 # Project 10315 # Dataset 10316 # Folder 10317 # Experiment 10318 # Plate 10319 # Screen 10320 # Experimenter 10321 # ExperimenterGroup 10322 # Instrument 10323 # StructuredAnnotations 10324 # ROI 10325 if 'UUID' in metadata: 10326 self.uuid = metadata['UUID'].split(':')[-1] 10327 else: 10328 from uuid import uuid1 # noqa: delayed import 10329 10330 self.uuid = str(uuid1()) 10331 creator = OmeXml._attribute( 10332 metadata, 'Creator', default=f'tifffile.py {__version__}' 10333 ) 10334 schema = 'http://www.openmicroscopy.org/Schemas/OME/2016-06' 10335 self.xml = ( 10336 '{declaration}' 10337 f'<OME xmlns="{schema}" ' 10338 f'xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ' 10339 f'xsi:schemaLocation="{schema} {schema}/ome.xsd" ' 10340 f'UUID="urn:uuid:{self.uuid}" {creator}>' 10341 '{images}' 10342 '{annotations}' 10343 '{elements}' 10344 f'</OME>' 10345 ) 10346 10347 def addimage(self, dtype, shape, storedshape, axes=None, **metadata): 10348 """Add image to OME-XML. 10349 10350 The OME model can handle up to 9 dimensional images for selected 10351 axes orders. Refer to the OME-XML specification for details. 10352 Non-TZCYXS (modulo) dimensions must be after a TZC dimension or 10353 require an unused TZC dimension. 10354 10355 Parameters 10356 ---------- 10357 dtype : numpy.dtype 10358 Data type of image array. 10359 shape : tuple 10360 Shape of image array. 10361 storedshape: tuple 10362 Normalized shape describing how the image array is stored in TIFF: 10363 (pages, separate_samples, depth, length, width, contig_samples). 10364 axes : str (optional) 10365 Axes labels for each dimension in shape. 10366 By default, axes are matched to the shape in reverse order of 10367 TZC(S)YX(S) based on storedshape. 10368 The following axes codes are supported: 'S' sample, 'X' width, 10369 'Y' length, 'Z' depth, 'C' channel, 'T' time, 'A' angle, 'P' phase, 10370 'R' tile, 'H' lifetime, 'E' lambda, 'Q' other. 10371 metadata : miscellaneous (optional) 10372 Additional OME-XML attributes or elements to be stored. 10373 Image/Pixels: Name, AcquisitionDate, Description, 10374 PhysicalSizeX, PhysicalSizeXUnit, PhysicalSizeY, PhysicalSizeYUnit, 10375 PhysicalSizeZ, PhysicalSizeZUnit, TimeIncrement, TimeIncrementUnit. 10376 Per Plane: DeltaTUnit, ExposureTime, ExposureTimeUnit, 10377 PositionX, PositionXUnit, PositionY, PositionYUnit, PositionZ, 10378 PositionZUnit. 10379 Per Channel: Name, AcquisitionMode, Color, ContrastMethod, 10380 EmissionWavelength, EmissionWavelengthUnit, ExcitationWavelength, 10381 ExcitationWavelengthUnit, Fluor, IlluminationType, NDFilter, 10382 PinholeSize, PinholeSizeUnit, PockelCellSetting. 10383 10384 """ 10385 index = len(self.images) 10386 10387 # get Image and Pixels metadata 10388 metadata = metadata.get('OME', metadata) 10389 metadata = metadata.get('Image', metadata) 10390 if isinstance(metadata, (list, tuple)): 10391 # multiple images 10392 metadata = metadata[index] 10393 if 'Pixels' in metadata: 10394 # merge with Image 10395 if 'ID' in metadata['Pixels']: 10396 del metadata['Pixels']['ID'] 10397 metadata.update(metadata['Pixels']) 10398 del metadata['Pixels'] 10399 10400 try: 10401 dtype = numpy.dtype(dtype).name 10402 dtype = { 10403 'int8': 'int8', 10404 'int16': 'int16', 10405 'int32': 'int32', 10406 'uint8': 'uint8', 10407 'uint16': 'uint16', 10408 'uint32': 'uint32', 10409 'float32': 'float', 10410 'float64': 'double', 10411 'complex64': 'complex', 10412 'complex128': 'double-complex', 10413 'bool': 'bit', 10414 }[dtype] 10415 except KeyError: 10416 raise OmeXmlError(f'data type {dtype!r} not supported') 10417 10418 if metadata.get('Type', dtype) != dtype: 10419 raise OmeXmlError( 10420 f'metadata Pixels Type {metadata["Type"]!r} ' 10421 f'does not match array dtype {dtype!r}' 10422 ) 10423 10424 samples = 1 10425 planecount, separate, depth, length, width, contig = storedshape 10426 if depth != 1: 10427 raise OmeXmlError('ImageDepth not supported') 10428 if not (separate == 1 or contig == 1): 10429 raise ValueError('invalid stored shape') 10430 10431 shape = tuple(int(i) for i in shape) 10432 ndim = len(shape) 10433 if ndim < 1 or product(shape) <= 0: 10434 raise OmeXmlError('empty arrays not supported') 10435 10436 if axes is None: 10437 # get axes from shape, stored shape, and DimensionOrder 10438 if contig != 1 or shape[-3:] == (length, width, 1): 10439 axes = 'YXS' 10440 samples = contig 10441 elif separate != 1 or ( 10442 ndim == 6 and shape[-3:] == (1, length, width) 10443 ): 10444 axes = 'SYX' 10445 samples = separate 10446 else: 10447 axes = 'YX' 10448 if not len(axes) <= ndim <= (6 if 'S' in axes else 5): 10449 raise OmeXmlError(f'{ndim} dimensions not supported') 10450 hiaxes = metadata.get('DimensionOrder', 'XYCZT')[:1:-1] 10451 axes = hiaxes[(6 if 'S' in axes else 5) - ndim :] + axes 10452 assert len(axes) == len(shape) 10453 10454 else: 10455 # validate axes against shape and stored shape 10456 axes = axes.upper() 10457 if len(axes) != len(shape): 10458 raise ValueError('axes do not match shape') 10459 if not (axes.endswith('YX') or axes.endswith('YXS')): 10460 raise OmeXmlError('dimensions must end with YX or YXS') 10461 unique = [] 10462 for ax in axes: 10463 if ax not in 'TZCYXSAPRHEQ': 10464 raise OmeXmlError(f'dimension {ax!r} not supported') 10465 if ax in unique: 10466 raise OmeXmlError(f'multiple {ax!r} dimensions') 10467 unique.append(ax) 10468 if ndim > (9 if 'S' in axes else 8): 10469 raise OmeXmlError('more than 8 dimensions not supported') 10470 if contig != 1: 10471 samples = contig 10472 if ndim < 3: 10473 raise ValueError('dimensions do not match stored shape') 10474 if axes[-1] != 'S': 10475 raise ValueError('axes do not match stored shape') 10476 if shape[-1] != contig or shape[-2] != width: 10477 raise ValueError('shape does not match stored shape') 10478 elif separate != 1: 10479 samples = separate 10480 if ndim < 3: 10481 raise ValueError('dimensions do not match stored shape') 10482 if axes[-3] != 'S': 10483 raise ValueError('axes do not match stored shape') 10484 if shape[-3] != separate or shape[-1] != length: 10485 raise ValueError('shape does not match stored shape') 10486 10487 if shape[axes.index('X')] != width or shape[axes.index('Y')] != length: 10488 raise ValueError('shape does not match stored shape') 10489 10490 if 'S' in axes: 10491 hiaxes = axes[: min(axes.index('S'), axes.index('Y'))] 10492 else: 10493 hiaxes = axes[: axes.index('Y')] 10494 10495 if any(ax in 'APRHEQ' for ax in hiaxes): 10496 # modulo axes 10497 modulo = {} 10498 dimorder = [] 10499 axestype = { 10500 'A': 'angle', 10501 'P': 'phase', 10502 'R': 'tile', 10503 'H': 'lifetime', 10504 'E': 'lambda', 10505 'Q': 'other', 10506 } 10507 for i, ax in enumerate(hiaxes): 10508 if ax in 'APRHEQ': 10509 x = hiaxes[i - 1 : i] 10510 if x and x in 'TZC': 10511 # use previous axis 10512 modulo[x] = axestype[ax], shape[i] 10513 else: 10514 # use next unused axis 10515 for x in 'TZC': 10516 if x not in dimorder and x not in modulo: 10517 modulo[x] = axestype[ax], shape[i] 10518 dimorder.append(x) 10519 break 10520 else: 10521 # TODO: support any order of axes, e.g. APRTZC 10522 raise OmeXmlError('more than 3 modulo dimensions') 10523 else: 10524 dimorder.append(ax) 10525 hiaxes = ''.join(dimorder) 10526 10527 # TODO: use user-specified start, stop, step, or labels 10528 moduloalong = ''.join( 10529 f'<ModuloAlong{ax} Type="{axtype}" Start="0" End="{size-1}"/>' 10530 for ax, (axtype, size) in modulo.items() 10531 ) 10532 annotationref = f'<AnnotationRef ID="Annotation:{index}"/>' 10533 annotations = ( 10534 f'<XMLAnnotation ID="Annotation:{index}" ' 10535 'Namespace="openmicroscopy.org/omero/dimension/modulo">' 10536 '<Value>' 10537 '<Modulo namespace=' 10538 '"http://www.openmicroscopy.org/Schemas/Additions/2011-09">' 10539 f'{moduloalong}' 10540 '</Modulo>' 10541 '</Value>' 10542 '</XMLAnnotation>' 10543 ) 10544 self.annotations.append(annotations) 10545 else: 10546 modulo = {} 10547 annotationref = '' 10548 10549 hiaxes = hiaxes[::-1] 10550 for dimorder in ( 10551 metadata.get('DimensionOrder', 'XYCZT'), 10552 'XYCZT', 10553 'XYZCT', 10554 'XYZTC', 10555 'XYCTZ', 10556 'XYTCZ', 10557 'XYTZC', 10558 ): 10559 if hiaxes in dimorder: 10560 break 10561 else: 10562 raise OmeXmlError(f'dimension order {axes!r} not supported') 10563 10564 dimsizes = [] 10565 for ax in dimorder: 10566 if ax == 'S': 10567 continue 10568 if ax in axes: 10569 size = shape[axes.index(ax)] 10570 else: 10571 size = 1 10572 if ax == 'C': 10573 sizec = size 10574 size *= samples 10575 if ax in modulo: 10576 size *= modulo[ax][1] 10577 dimsizes.append(size) 10578 sizes = ''.join( 10579 f' Size{ax}="{size}"' for ax, size in zip(dimorder, dimsizes) 10580 ) 10581 10582 # verify DimensionOrder in metadata is compatible 10583 if 'DimensionOrder' in metadata: 10584 omedimorder = metadata['DimensionOrder'] 10585 omedimorder = ''.join( 10586 ax for ax in omedimorder if dimsizes[dimorder.index(ax)] > 1 10587 ) 10588 if hiaxes not in omedimorder: 10589 raise OmeXmlError( 10590 f'metadata DimensionOrder does not match {axes!r}' 10591 ) 10592 10593 # verify metadata Size values match shape 10594 for ax, size in zip(dimorder, dimsizes): 10595 if metadata.get(f'Size{ax}', size) != size: 10596 raise OmeXmlError( 10597 f'metadata Size{ax} does not match {shape!r}' 10598 ) 10599 10600 dimsizes[dimorder.index('C')] //= samples 10601 if planecount != product(dimsizes[2:]): 10602 raise ValueError('shape does not match stored shape') 10603 10604 planes = [] 10605 planeattributes = metadata.get('Plane', '') 10606 if planeattributes: 10607 cztorder = tuple(dimorder[2:].index(ax) for ax in 'CZT') 10608 for p in range(planecount): 10609 attributes = OmeXml._attributes( 10610 planeattributes, 10611 p, 10612 'DeltaTUnit', 10613 'ExposureTime', 10614 'ExposureTimeUnit', 10615 'PositionX', 10616 'PositionXUnit', 10617 'PositionY', 10618 'PositionYUnit', 10619 'PositionZ', 10620 'PositionZUnit', 10621 ) 10622 unraveled = numpy.unravel_index(p, dimsizes[2:], order='F') 10623 c, z, t = (int(unraveled[i]) for i in cztorder) 10624 planes.append( 10625 f'<Plane TheC="{c}" TheZ="{z}" TheT="{t}"{attributes}/>' 10626 ) 10627 # TODO: if possible, verify c, z, t match planeattributes 10628 planes = ''.join(planes) 10629 10630 channels = [] 10631 for c in range(sizec): 10632 lightpath = '<LightPath/>' 10633 # TODO: use LightPath elements from metadata 10634 # 'AnnotationRef', 10635 # 'DichroicRef', 10636 # 'EmissionFilterRef', 10637 # 'ExcitationFilterRef' 10638 attributes = OmeXml._attributes( 10639 metadata.get('Channel', ''), 10640 c, 10641 'Name', 10642 'AcquisitionMode', 10643 'Color', 10644 'ContrastMethod', 10645 'EmissionWavelength', 10646 'EmissionWavelengthUnit', 10647 'ExcitationWavelength', 10648 'ExcitationWavelengthUnit', 10649 'Fluor', 10650 'IlluminationType', 10651 'NDFilter', 10652 'PinholeSize', 10653 'PinholeSizeUnit', 10654 'PockelCellSetting', 10655 ) 10656 channels.append( 10657 f'<Channel ID="Channel:{index}:{c}" ' 10658 f'SamplesPerPixel="{samples}"' 10659 f'{attributes}>' 10660 f'{lightpath}' 10661 '</Channel>' 10662 ) 10663 channels = ''.join(channels) 10664 10665 # TODO: support more Image elements 10666 elements = OmeXml._elements(metadata, 'AcquisitionDate', 'Description') 10667 10668 name = OmeXml._attribute(metadata, 'Name', default=f'Image{index}') 10669 attributes = OmeXml._attributes( 10670 metadata, 10671 None, 10672 'SignificantBits', 10673 'PhysicalSizeX', 10674 'PhysicalSizeXUnit', 10675 'PhysicalSizeY', 10676 'PhysicalSizeYUnit', 10677 'PhysicalSizeZ', 10678 'PhysicalSizeZUnit', 10679 'TimeIncrement', 10680 'TimeIncrementUnit', 10681 ) 10682 if separate > 1 or contig > 1: 10683 interleaved = 'false' if separate > 1 else 'true' 10684 interleaved = f' Interleaved="{interleaved}"' 10685 else: 10686 interleaved = '' 10687 10688 self.images.append( 10689 f'<Image ID="Image:{index}"{name}>' 10690 f'{elements}' 10691 f'<Pixels ID="Pixels:{index}" ' 10692 f'DimensionOrder="{dimorder}" ' 10693 f'Type="{dtype}"' 10694 f'{sizes}' 10695 f'{interleaved}' 10696 f'{attributes}>' 10697 f'{channels}' 10698 f'<TiffData IFD="{self.ifd}" PlaneCount="{planecount}"/>' 10699 f'{planes}' 10700 f'</Pixels>' 10701 f'{annotationref}' 10702 f'</Image>' 10703 ) 10704 self.ifd += planecount 10705 10706 def tostring(self, declaration=False): 10707 """Return OME-XML string.""" 10708 # TODO: support other top-level elements 10709 elements = ''.join(self.elements) 10710 images = ''.join(self.images) 10711 annotations = ''.join(self.annotations) 10712 if annotations: 10713 annotations = ( 10714 f'<StructuredAnnotations>{annotations}</StructuredAnnotations>' 10715 ) 10716 if declaration: 10717 declaration = '<?xml version="1.0" encoding="UTF-8"?>' 10718 else: 10719 declaration = '' 10720 xml = self.xml.format( 10721 declaration=declaration, 10722 images=images, 10723 annotations=annotations, 10724 elements=elements, 10725 ) 10726 return xml 10727 10728 def __str__(self): 10729 """Return OME-XML string.""" 10730 xml = self.tostring() 10731 try: 10732 from lxml import etree # noqa: delayed import 10733 10734 parser = etree.XMLParser(remove_blank_text=True) 10735 tree = etree.fromstring(xml, parser) 10736 xml = etree.tostring( 10737 tree, encoding='utf-8', pretty_print=True, xml_declaration=True 10738 ).decode() 10739 except Exception as exc: 10740 warnings.warn(f'OmeXml.__str__: {exc}', UserWarning) 10741 except ImportError: 10742 pass 10743 return xml 10744 10745 @staticmethod 10746 def _escape(value): 10747 """Return escaped string of value.""" 10748 if not isinstance(value, str): 10749 value = str(value) 10750 elif '&' in value or '>' in value or '<' in value: 10751 return value 10752 value = value.replace('&', '&') 10753 value = value.replace('>', '>') 10754 value = value.replace('<', '<') 10755 return value 10756 10757 @staticmethod 10758 def _element(metadata, name, default=None): 10759 """Return XML formatted element if name in metadata.""" 10760 value = metadata.get(name, default) 10761 if value is None: 10762 return None 10763 return f'<{name}>{OmeXml._escape(value)}</{name}>' 10764 10765 @staticmethod 10766 def _elements(metadata, *names): 10767 """Return XML formatted elements.""" 10768 if not metadata: 10769 return '' 10770 elements = (OmeXml._element(metadata, name) for name in names) 10771 return ''.join(e for e in elements if e) 10772 10773 @staticmethod 10774 def _attribute(metadata, name, index=None, default=None): 10775 """Return XML formatted attribute if name in metadata.""" 10776 value = metadata.get(name, default) 10777 if value is None: 10778 return None 10779 if index is not None: 10780 if isinstance(value, (list, tuple)): 10781 value = value[index] 10782 elif index > 0: 10783 raise TypeError( 10784 f'{type(value).__name__!r} is not a list or tuple' 10785 ) 10786 return f' {name}="{OmeXml._escape(value)}"' 10787 10788 @staticmethod 10789 def _attributes(metadata, index_, *names): 10790 """Return XML formatted attributes.""" 10791 if not metadata: 10792 return '' 10793 if index_ is None: 10794 attributes = (OmeXml._attribute(metadata, name) for name in names) 10795 elif isinstance(metadata, (list, tuple)): 10796 metadata = metadata[index_] 10797 attributes = (OmeXml._attribute(metadata, name) for name in names) 10798 elif isinstance(metadata, dict): 10799 attributes = ( 10800 OmeXml._attribute(metadata, name, index_) for name in names 10801 ) 10802 return ''.join(a for a in attributes if a) 10803 10804 @staticmethod 10805 def _reference(metadata, name): 10806 """Return XML formatted reference element.""" 10807 value = metadata.get(name, None) 10808 if value is None: 10809 return '' 10810 try: 10811 value = value['ID'] 10812 except KeyError: 10813 pass 10814 return f'<{name} ID="{OmeXml._escape(value)}"/>' 10815 10816 @staticmethod 10817 def validate(omexml, omexsd=None, assert_=True, _schema=[]): 10818 """Return if OME-XML is valid according to XMLSchema. 10819 10820 If 'assert_' is True, raise an AssertionError if validation fails. 10821 10822 On first run, this function takes several seconds to download and 10823 parse the 2016-06 OME XMLSchema. 10824 10825 """ 10826 from lxml import etree # noqa: delay import 10827 10828 if not _schema: 10829 if omexsd is None: 10830 omexsd = os.path.join(os.path.dirname(__file__), 'ome.xsd') 10831 if os.path.exists(omexsd): 10832 with open(omexsd, 'rb') as fh: 10833 omexsd = fh.read() 10834 else: 10835 import urllib.request # noqa: delay import 10836 10837 with urllib.request.urlopen( 10838 'https://www.openmicroscopy.org/' 10839 'Schemas/OME/2016-06/ome.xsd' 10840 ) as fh: 10841 omexsd = fh.read() 10842 if omexsd.startswith(b'<?xml'): 10843 omexsd = omexsd.split(b'>', 1)[-1] 10844 try: 10845 _schema.append( 10846 etree.XMLSchema(etree.fromstring(omexsd.decode())) 10847 ) 10848 except Exception: 10849 # raise 10850 _schema.append(None) 10851 if _schema and _schema[0] is not None: 10852 if omexml.startswith('<?xml'): 10853 omexml = omexml.split('>', 1)[-1] 10854 tree = etree.fromstring(omexml) 10855 if assert_: 10856 _schema[0].assert_(tree) 10857 return True 10858 return _schema[0].validate(tree) 10859 return None 10860 10861 10862class LazyConst: 10863 """Class whose attributes are computed on first access from its methods.""" 10864 10865 def __init__(self, cls): 10866 self._cls = cls 10867 self.__doc__ = cls.__doc__ 10868 self.__module__ = cls.__module__ 10869 self.__name__ = cls.__name__ 10870 self.__qualname__ = cls.__qualname__ 10871 self.lock = threading.RLock() 10872 10873 def __reduce__(self): 10874 # decorated class will be pickled by name 10875 return self._cls.__qualname__ 10876 10877 def __getattr__(self, name): 10878 with self.lock: 10879 if name in self.__dict__: 10880 # another thread set attribute while awaiting lock 10881 return self.__dict__[name] 10882 func = getattr(self._cls, name) 10883 value = func() if callable(func) else func 10884 for attr in ( 10885 '__module__', 10886 '__name__', 10887 '__qualname__', 10888 # '__doc__', 10889 # '__annotations__' 10890 ): 10891 try: 10892 setattr(value, attr, getattr(func, attr)) 10893 except AttributeError: 10894 pass 10895 setattr(self, name, value) 10896 return value 10897 10898 10899@LazyConst 10900class TIFF: 10901 """Namespace for module constants.""" 10902 10903 def CLASSIC_LE(): 10904 class ClassicTiffLe: 10905 __slots__ = () 10906 version = 42 10907 byteorder = '<' 10908 offsetsize = 4 10909 offsetformat = '<I' 10910 tagnosize = 2 10911 tagnoformat = '<H' 10912 tagsize = 12 10913 tagformat1 = '<HH' 10914 tagformat2 = '<I4s' 10915 tagoffsetthreshold = 4 10916 10917 return ClassicTiffLe 10918 10919 def CLASSIC_BE(): 10920 class ClassicTiffBe: 10921 __slots__ = () 10922 version = 42 10923 byteorder = '>' 10924 offsetsize = 4 10925 offsetformat = '>I' 10926 tagnosize = 2 10927 tagnoformat = '>H' 10928 tagsize = 12 10929 tagformat1 = '>HH' 10930 tagformat2 = '>I4s' 10931 tagoffsetthreshold = 4 10932 10933 return ClassicTiffBe 10934 10935 def BIG_LE(): 10936 class BigTiffLe: 10937 __slots__ = () 10938 version = 43 10939 byteorder = '<' 10940 offsetsize = 8 10941 offsetformat = '<Q' 10942 tagnosize = 8 10943 tagnoformat = '<Q' 10944 tagsize = 20 10945 tagformat1 = '<HH' 10946 tagformat2 = '<Q8s' 10947 tagoffsetthreshold = 8 10948 10949 return BigTiffLe 10950 10951 def BIG_BE(): 10952 class BigTiffBe: 10953 __slots__ = () 10954 version = 43 10955 byteorder = '>' 10956 offsetsize = 8 10957 offsetformat = '>Q' 10958 tagnosize = 8 10959 tagnoformat = '>Q' 10960 tagsize = 20 10961 tagformat1 = '>HH' 10962 tagformat2 = '>Q8s' 10963 tagoffsetthreshold = 8 10964 10965 return BigTiffBe 10966 10967 def NDPI_LE(): 10968 class NdpiTiffLe: 10969 __slots__ = () 10970 version = 42 10971 byteorder = '<' 10972 offsetsize = 8 # NDPI uses 8 bytes IFD and tag offsets 10973 offsetformat = '<Q' 10974 tagnosize = 2 10975 tagnoformat = '<H' 10976 tagsize = 12 # 16 after patching 10977 tagformat1 = '<HH' 10978 tagformat2 = '<I8s' # after patching 10979 tagoffsetthreshold = 4 10980 10981 return NdpiTiffLe 10982 10983 def TAGS(): 10984 # TIFF tag codes and names from TIFF6, TIFF/EP, EXIF, and other specs 10985 # TODO: divide into baseline, exif, private, ... tags 10986 return TiffTagRegistry( 10987 ( 10988 (11, 'ProcessingSoftware'), 10989 (254, 'NewSubfileType'), 10990 (255, 'SubfileType'), 10991 (256, 'ImageWidth'), 10992 (257, 'ImageLength'), 10993 (258, 'BitsPerSample'), 10994 (259, 'Compression'), 10995 (262, 'PhotometricInterpretation'), 10996 (263, 'Thresholding'), 10997 (264, 'CellWidth'), 10998 (265, 'CellLength'), 10999 (266, 'FillOrder'), 11000 (269, 'DocumentName'), 11001 (270, 'ImageDescription'), 11002 (271, 'Make'), 11003 (272, 'Model'), 11004 (273, 'StripOffsets'), 11005 (274, 'Orientation'), 11006 (277, 'SamplesPerPixel'), 11007 (278, 'RowsPerStrip'), 11008 (279, 'StripByteCounts'), 11009 (280, 'MinSampleValue'), 11010 (281, 'MaxSampleValue'), 11011 (282, 'XResolution'), 11012 (283, 'YResolution'), 11013 (284, 'PlanarConfiguration'), 11014 (285, 'PageName'), 11015 (286, 'XPosition'), 11016 (287, 'YPosition'), 11017 (288, 'FreeOffsets'), 11018 (289, 'FreeByteCounts'), 11019 (290, 'GrayResponseUnit'), 11020 (291, 'GrayResponseCurve'), 11021 (292, 'T4Options'), 11022 (293, 'T6Options'), 11023 (296, 'ResolutionUnit'), 11024 (297, 'PageNumber'), 11025 (300, 'ColorResponseUnit'), 11026 (301, 'TransferFunction'), 11027 (305, 'Software'), 11028 (306, 'DateTime'), 11029 (315, 'Artist'), 11030 (316, 'HostComputer'), 11031 (317, 'Predictor'), 11032 (318, 'WhitePoint'), 11033 (319, 'PrimaryChromaticities'), 11034 (320, 'ColorMap'), 11035 (321, 'HalftoneHints'), 11036 (322, 'TileWidth'), 11037 (323, 'TileLength'), 11038 (324, 'TileOffsets'), 11039 (325, 'TileByteCounts'), 11040 (326, 'BadFaxLines'), 11041 (327, 'CleanFaxData'), 11042 (328, 'ConsecutiveBadFaxLines'), 11043 (330, 'SubIFDs'), 11044 (332, 'InkSet'), 11045 (333, 'InkNames'), 11046 (334, 'NumberOfInks'), 11047 (336, 'DotRange'), 11048 (337, 'TargetPrinter'), 11049 (338, 'ExtraSamples'), 11050 (339, 'SampleFormat'), 11051 (340, 'SMinSampleValue'), 11052 (341, 'SMaxSampleValue'), 11053 (342, 'TransferRange'), 11054 (343, 'ClipPath'), 11055 (344, 'XClipPathUnits'), 11056 (345, 'YClipPathUnits'), 11057 (346, 'Indexed'), 11058 (347, 'JPEGTables'), 11059 (351, 'OPIProxy'), 11060 (400, 'GlobalParametersIFD'), 11061 (401, 'ProfileType'), 11062 (402, 'FaxProfile'), 11063 (403, 'CodingMethods'), 11064 (404, 'VersionYear'), 11065 (405, 'ModeNumber'), 11066 (433, 'Decode'), 11067 (434, 'DefaultImageColor'), 11068 (435, 'T82Options'), 11069 (437, 'JPEGTables'), # 347 11070 (512, 'JPEGProc'), 11071 (513, 'JPEGInterchangeFormat'), 11072 (514, 'JPEGInterchangeFormatLength'), 11073 (515, 'JPEGRestartInterval'), 11074 (517, 'JPEGLosslessPredictors'), 11075 (518, 'JPEGPointTransforms'), 11076 (519, 'JPEGQTables'), 11077 (520, 'JPEGDCTables'), 11078 (521, 'JPEGACTables'), 11079 (529, 'YCbCrCoefficients'), 11080 (530, 'YCbCrSubSampling'), 11081 (531, 'YCbCrPositioning'), 11082 (532, 'ReferenceBlackWhite'), 11083 (559, 'StripRowCounts'), 11084 (700, 'XMP'), # XMLPacket 11085 (769, 'GDIGamma'), # GDI+ 11086 (770, 'ICCProfileDescriptor'), # GDI+ 11087 (771, 'SRGBRenderingIntent'), # GDI+ 11088 (800, 'ImageTitle'), # GDI+ 11089 (999, 'USPTO_Miscellaneous'), 11090 (4864, 'AndorId'), # TODO, Andor Technology 4864 - 5030 11091 (4869, 'AndorTemperature'), 11092 (4876, 'AndorExposureTime'), 11093 (4878, 'AndorKineticCycleTime'), 11094 (4879, 'AndorAccumulations'), 11095 (4881, 'AndorAcquisitionCycleTime'), 11096 (4882, 'AndorReadoutTime'), 11097 (4884, 'AndorPhotonCounting'), 11098 (4885, 'AndorEmDacLevel'), 11099 (4890, 'AndorFrames'), 11100 (4896, 'AndorHorizontalFlip'), 11101 (4897, 'AndorVerticalFlip'), 11102 (4898, 'AndorClockwise'), 11103 (4899, 'AndorCounterClockwise'), 11104 (4904, 'AndorVerticalClockVoltage'), 11105 (4905, 'AndorVerticalShiftSpeed'), 11106 (4907, 'AndorPreAmpSetting'), 11107 (4908, 'AndorCameraSerial'), 11108 (4911, 'AndorActualTemperature'), 11109 (4912, 'AndorBaselineClamp'), 11110 (4913, 'AndorPrescans'), 11111 (4914, 'AndorModel'), 11112 (4915, 'AndorChipSizeX'), 11113 (4916, 'AndorChipSizeY'), 11114 (4944, 'AndorBaselineOffset'), 11115 (4966, 'AndorSoftwareVersion'), 11116 (18246, 'Rating'), 11117 (18247, 'XP_DIP_XML'), 11118 (18248, 'StitchInfo'), 11119 (18249, 'RatingPercent'), 11120 (20481, 'ResolutionXUnit'), # GDI+ 11121 (20482, 'ResolutionYUnit'), # GDI+ 11122 (20483, 'ResolutionXLengthUnit'), # GDI+ 11123 (20484, 'ResolutionYLengthUnit'), # GDI+ 11124 (20485, 'PrintFlags'), # GDI+ 11125 (20486, 'PrintFlagsVersion'), # GDI+ 11126 (20487, 'PrintFlagsCrop'), # GDI+ 11127 (20488, 'PrintFlagsBleedWidth'), # GDI+ 11128 (20489, 'PrintFlagsBleedWidthScale'), # GDI+ 11129 (20490, 'HalftoneLPI'), # GDI+ 11130 (20491, 'HalftoneLPIUnit'), # GDI+ 11131 (20492, 'HalftoneDegree'), # GDI+ 11132 (20493, 'HalftoneShape'), # GDI+ 11133 (20494, 'HalftoneMisc'), # GDI+ 11134 (20495, 'HalftoneScreen'), # GDI+ 11135 (20496, 'JPEGQuality'), # GDI+ 11136 (20497, 'GridSize'), # GDI+ 11137 (20498, 'ThumbnailFormat'), # GDI+ 11138 (20499, 'ThumbnailWidth'), # GDI+ 11139 (20500, 'ThumbnailHeight'), # GDI+ 11140 (20501, 'ThumbnailColorDepth'), # GDI+ 11141 (20502, 'ThumbnailPlanes'), # GDI+ 11142 (20503, 'ThumbnailRawBytes'), # GDI+ 11143 (20504, 'ThumbnailSize'), # GDI+ 11144 (20505, 'ThumbnailCompressedSize'), # GDI+ 11145 (20506, 'ColorTransferFunction'), # GDI+ 11146 (20507, 'ThumbnailData'), 11147 (20512, 'ThumbnailImageWidth'), # GDI+ 11148 (20513, 'ThumbnailImageHeight'), # GDI+ 11149 (20514, 'ThumbnailBitsPerSample'), # GDI+ 11150 (20515, 'ThumbnailCompression'), 11151 (20516, 'ThumbnailPhotometricInterp'), # GDI+ 11152 (20517, 'ThumbnailImageDescription'), # GDI+ 11153 (20518, 'ThumbnailEquipMake'), # GDI+ 11154 (20519, 'ThumbnailEquipModel'), # GDI+ 11155 (20520, 'ThumbnailStripOffsets'), # GDI+ 11156 (20521, 'ThumbnailOrientation'), # GDI+ 11157 (20522, 'ThumbnailSamplesPerPixel'), # GDI+ 11158 (20523, 'ThumbnailRowsPerStrip'), # GDI+ 11159 (20524, 'ThumbnailStripBytesCount'), # GDI+ 11160 (20525, 'ThumbnailResolutionX'), 11161 (20526, 'ThumbnailResolutionY'), 11162 (20527, 'ThumbnailPlanarConfig'), # GDI+ 11163 (20528, 'ThumbnailResolutionUnit'), 11164 (20529, 'ThumbnailTransferFunction'), 11165 (20530, 'ThumbnailSoftwareUsed'), # GDI+ 11166 (20531, 'ThumbnailDateTime'), # GDI+ 11167 (20532, 'ThumbnailArtist'), # GDI+ 11168 (20533, 'ThumbnailWhitePoint'), # GDI+ 11169 (20534, 'ThumbnailPrimaryChromaticities'), # GDI+ 11170 (20535, 'ThumbnailYCbCrCoefficients'), # GDI+ 11171 (20536, 'ThumbnailYCbCrSubsampling'), # GDI+ 11172 (20537, 'ThumbnailYCbCrPositioning'), 11173 (20538, 'ThumbnailRefBlackWhite'), # GDI+ 11174 (20539, 'ThumbnailCopyRight'), # GDI+ 11175 (20545, 'InteroperabilityIndex'), 11176 (20546, 'InteroperabilityVersion'), 11177 (20624, 'LuminanceTable'), 11178 (20625, 'ChrominanceTable'), 11179 (20736, 'FrameDelay'), # GDI+ 11180 (20737, 'LoopCount'), # GDI+ 11181 (20738, 'GlobalPalette'), # GDI+ 11182 (20739, 'IndexBackground'), # GDI+ 11183 (20740, 'IndexTransparent'), # GDI+ 11184 (20752, 'PixelUnit'), # GDI+ 11185 (20753, 'PixelPerUnitX'), # GDI+ 11186 (20754, 'PixelPerUnitY'), # GDI+ 11187 (20755, 'PaletteHistogram'), # GDI+ 11188 (28672, 'SonyRawFileType'), # Sony ARW 11189 (28722, 'VignettingCorrParams'), # Sony ARW 11190 (28725, 'ChromaticAberrationCorrParams'), # Sony ARW 11191 (28727, 'DistortionCorrParams'), # Sony ARW 11192 # Private tags >= 32768 11193 (32781, 'ImageID'), 11194 (32931, 'WangTag1'), 11195 (32932, 'WangAnnotation'), 11196 (32933, 'WangTag3'), 11197 (32934, 'WangTag4'), 11198 (32953, 'ImageReferencePoints'), 11199 (32954, 'RegionXformTackPoint'), 11200 (32955, 'WarpQuadrilateral'), 11201 (32956, 'AffineTransformMat'), 11202 (32995, 'Matteing'), 11203 (32996, 'DataType'), # use SampleFormat 11204 (32997, 'ImageDepth'), 11205 (32998, 'TileDepth'), 11206 (33300, 'ImageFullWidth'), 11207 (33301, 'ImageFullLength'), 11208 (33302, 'TextureFormat'), 11209 (33303, 'TextureWrapModes'), 11210 (33304, 'FieldOfViewCotangent'), 11211 (33305, 'MatrixWorldToScreen'), 11212 (33306, 'MatrixWorldToCamera'), 11213 (33405, 'Model2'), 11214 (33421, 'CFARepeatPatternDim'), 11215 (33422, 'CFAPattern'), 11216 (33423, 'BatteryLevel'), 11217 (33424, 'KodakIFD'), 11218 (33434, 'ExposureTime'), 11219 (33437, 'FNumber'), 11220 (33432, 'Copyright'), 11221 (33445, 'MDFileTag'), 11222 (33446, 'MDScalePixel'), 11223 (33447, 'MDColorTable'), 11224 (33448, 'MDLabName'), 11225 (33449, 'MDSampleInfo'), 11226 (33450, 'MDPrepDate'), 11227 (33451, 'MDPrepTime'), 11228 (33452, 'MDFileUnits'), 11229 (33471, 'OlympusINI'), 11230 (33550, 'ModelPixelScaleTag'), 11231 (33560, 'OlympusSIS'), # see also 33471 and 34853 11232 (33589, 'AdventScale'), 11233 (33590, 'AdventRevision'), 11234 (33628, 'UIC1tag'), # Metamorph Universal Imaging Corp STK 11235 (33629, 'UIC2tag'), 11236 (33630, 'UIC3tag'), 11237 (33631, 'UIC4tag'), 11238 (33723, 'IPTCNAA'), 11239 (33858, 'ExtendedTagsOffset'), # DEFF points IFD with tags 11240 (33918, 'IntergraphPacketData'), # INGRPacketDataTag 11241 (33919, 'IntergraphFlagRegisters'), # INGRFlagRegisters 11242 (33920, 'IntergraphMatrixTag'), # IrasBTransformationMatrix 11243 (33921, 'INGRReserved'), 11244 (33922, 'ModelTiepointTag'), 11245 (33923, 'LeicaMagic'), 11246 (34016, 'Site'), # 34016..34032 ANSI IT8 TIFF/IT 11247 (34017, 'ColorSequence'), 11248 (34018, 'IT8Header'), 11249 (34019, 'RasterPadding'), 11250 (34020, 'BitsPerRunLength'), 11251 (34021, 'BitsPerExtendedRunLength'), 11252 (34022, 'ColorTable'), 11253 (34023, 'ImageColorIndicator'), 11254 (34024, 'BackgroundColorIndicator'), 11255 (34025, 'ImageColorValue'), 11256 (34026, 'BackgroundColorValue'), 11257 (34027, 'PixelIntensityRange'), 11258 (34028, 'TransparencyIndicator'), 11259 (34029, 'ColorCharacterization'), 11260 (34030, 'HCUsage'), 11261 (34031, 'TrapIndicator'), 11262 (34032, 'CMYKEquivalent'), 11263 (34118, 'CZ_SEM'), # Zeiss SEM 11264 (34152, 'AFCP_IPTC'), 11265 (34232, 'PixelMagicJBIGOptions'), # EXIF, also TI FrameCount 11266 (34263, 'JPLCartoIFD'), 11267 (34122, 'IPLAB'), # number of images 11268 (34264, 'ModelTransformationTag'), 11269 (34306, 'WB_GRGBLevels'), # Leaf MOS 11270 (34310, 'LeafData'), 11271 (34361, 'MM_Header'), 11272 (34362, 'MM_Stamp'), 11273 (34363, 'MM_Unknown'), 11274 (34377, 'ImageResources'), # Photoshop 11275 (34386, 'MM_UserBlock'), 11276 (34412, 'CZ_LSMINFO'), 11277 (34665, 'ExifTag'), 11278 (34675, 'InterColorProfile'), # ICCProfile 11279 (34680, 'FEI_SFEG'), # 11280 (34682, 'FEI_HELIOS'), # 11281 (34683, 'FEI_TITAN'), # 11282 (34687, 'FXExtensions'), 11283 (34688, 'MultiProfiles'), 11284 (34689, 'SharedData'), 11285 (34690, 'T88Options'), 11286 (34710, 'MarCCD'), # offset to MarCCD header 11287 (34732, 'ImageLayer'), 11288 (34735, 'GeoKeyDirectoryTag'), 11289 (34736, 'GeoDoubleParamsTag'), 11290 (34737, 'GeoAsciiParamsTag'), 11291 (34750, 'JBIGOptions'), 11292 (34821, 'PIXTIFF'), # ? Pixel Translations Inc 11293 (34850, 'ExposureProgram'), 11294 (34852, 'SpectralSensitivity'), 11295 (34853, 'GPSTag'), # GPSIFD also OlympusSIS2 11296 (34853, 'OlympusSIS2'), 11297 (34855, 'ISOSpeedRatings'), 11298 (34855, 'PhotographicSensitivity'), 11299 (34856, 'OECF'), # optoelectric conversion factor 11300 (34857, 'Interlace'), 11301 (34858, 'TimeZoneOffset'), 11302 (34859, 'SelfTimerMode'), 11303 (34864, 'SensitivityType'), 11304 (34865, 'StandardOutputSensitivity'), 11305 (34866, 'RecommendedExposureIndex'), 11306 (34867, 'ISOSpeed'), 11307 (34868, 'ISOSpeedLatitudeyyy'), 11308 (34869, 'ISOSpeedLatitudezzz'), 11309 (34908, 'HylaFAXFaxRecvParams'), 11310 (34909, 'HylaFAXFaxSubAddress'), 11311 (34910, 'HylaFAXFaxRecvTime'), 11312 (34911, 'FaxDcs'), 11313 (34929, 'FedexEDR'), 11314 (34954, 'LeafSubIFD'), 11315 (34959, 'Aphelion1'), 11316 (34960, 'Aphelion2'), 11317 (34961, 'AphelionInternal'), # ADCIS 11318 (36864, 'ExifVersion'), 11319 (36867, 'DateTimeOriginal'), 11320 (36868, 'DateTimeDigitized'), 11321 (36873, 'GooglePlusUploadCode'), 11322 (36880, 'OffsetTime'), 11323 (36881, 'OffsetTimeOriginal'), 11324 (36882, 'OffsetTimeDigitized'), 11325 # TODO, Pilatus/CHESS/TV6 36864..37120 conflicting with Exif 11326 (36864, 'TVX_Unknown'), 11327 (36865, 'TVX_NumExposure'), 11328 (36866, 'TVX_NumBackground'), 11329 (36867, 'TVX_ExposureTime'), 11330 (36868, 'TVX_BackgroundTime'), 11331 (36870, 'TVX_Unknown'), 11332 (36873, 'TVX_SubBpp'), 11333 (36874, 'TVX_SubWide'), 11334 (36875, 'TVX_SubHigh'), 11335 (36876, 'TVX_BlackLevel'), 11336 (36877, 'TVX_DarkCurrent'), 11337 (36878, 'TVX_ReadNoise'), 11338 (36879, 'TVX_DarkCurrentNoise'), 11339 (36880, 'TVX_BeamMonitor'), 11340 (37120, 'TVX_UserVariables'), # A/D values 11341 (37121, 'ComponentsConfiguration'), 11342 (37122, 'CompressedBitsPerPixel'), 11343 (37377, 'ShutterSpeedValue'), 11344 (37378, 'ApertureValue'), 11345 (37379, 'BrightnessValue'), 11346 (37380, 'ExposureBiasValue'), 11347 (37381, 'MaxApertureValue'), 11348 (37382, 'SubjectDistance'), 11349 (37383, 'MeteringMode'), 11350 (37384, 'LightSource'), 11351 (37385, 'Flash'), 11352 (37386, 'FocalLength'), 11353 (37387, 'FlashEnergy'), # 37387 11354 (37388, 'SpatialFrequencyResponse'), # 37388 11355 (37389, 'Noise'), 11356 (37390, 'FocalPlaneXResolution'), 11357 (37391, 'FocalPlaneYResolution'), 11358 (37392, 'FocalPlaneResolutionUnit'), 11359 (37393, 'ImageNumber'), 11360 (37394, 'SecurityClassification'), 11361 (37395, 'ImageHistory'), 11362 (37396, 'SubjectLocation'), 11363 (37397, 'ExposureIndex'), 11364 (37398, 'TIFFEPStandardID'), 11365 (37399, 'SensingMethod'), 11366 (37434, 'CIP3DataFile'), 11367 (37435, 'CIP3Sheet'), 11368 (37436, 'CIP3Side'), 11369 (37439, 'StoNits'), 11370 (37500, 'MakerNote'), 11371 (37510, 'UserComment'), 11372 (37520, 'SubsecTime'), 11373 (37521, 'SubsecTimeOriginal'), 11374 (37522, 'SubsecTimeDigitized'), 11375 (37679, 'MODIText'), # Microsoft Office Document Imaging 11376 (37680, 'MODIOLEPropertySetStorage'), 11377 (37681, 'MODIPositioning'), 11378 (37706, 'TVIPS'), # offset to TemData structure 11379 (37707, 'TVIPS1'), 11380 (37708, 'TVIPS2'), # same TemData structure as undefined 11381 (37724, 'ImageSourceData'), # Photoshop 11382 (37888, 'Temperature'), 11383 (37889, 'Humidity'), 11384 (37890, 'Pressure'), 11385 (37891, 'WaterDepth'), 11386 (37892, 'Acceleration'), 11387 (37893, 'CameraElevationAngle'), 11388 (40000, 'XPos'), # Janelia 11389 (40001, 'YPos'), 11390 (40002, 'ZPos'), 11391 (40001, 'MC_IpWinScal'), # Media Cybernetics 11392 (40001, 'RecipName'), # MS FAX 11393 (40002, 'RecipNumber'), 11394 (40003, 'SenderName'), 11395 (40004, 'Routing'), 11396 (40005, 'CallerId'), 11397 (40006, 'TSID'), 11398 (40007, 'CSID'), 11399 (40008, 'FaxTime'), 11400 (40100, 'MC_IdOld'), 11401 (40106, 'MC_Unknown'), 11402 (40965, 'InteroperabilityTag'), # InteropOffset 11403 (40091, 'XPTitle'), 11404 (40092, 'XPComment'), 11405 (40093, 'XPAuthor'), 11406 (40094, 'XPKeywords'), 11407 (40095, 'XPSubject'), 11408 (40960, 'FlashpixVersion'), 11409 (40961, 'ColorSpace'), 11410 (40962, 'PixelXDimension'), 11411 (40963, 'PixelYDimension'), 11412 (40964, 'RelatedSoundFile'), 11413 (40976, 'SamsungRawPointersOffset'), 11414 (40977, 'SamsungRawPointersLength'), 11415 (41217, 'SamsungRawByteOrder'), 11416 (41218, 'SamsungRawUnknown'), 11417 (41483, 'FlashEnergy'), 11418 (41484, 'SpatialFrequencyResponse'), 11419 (41485, 'Noise'), # 37389 11420 (41486, 'FocalPlaneXResolution'), # 37390 11421 (41487, 'FocalPlaneYResolution'), # 37391 11422 (41488, 'FocalPlaneResolutionUnit'), # 37392 11423 (41489, 'ImageNumber'), # 37393 11424 (41490, 'SecurityClassification'), # 37394 11425 (41491, 'ImageHistory'), # 37395 11426 (41492, 'SubjectLocation'), # 37395 11427 (41493, 'ExposureIndex '), # 37397 11428 (41494, 'TIFF-EPStandardID'), 11429 (41495, 'SensingMethod'), # 37399 11430 (41728, 'FileSource'), 11431 (41729, 'SceneType'), 11432 (41730, 'CFAPattern'), # 33422 11433 (41985, 'CustomRendered'), 11434 (41986, 'ExposureMode'), 11435 (41987, 'WhiteBalance'), 11436 (41988, 'DigitalZoomRatio'), 11437 (41989, 'FocalLengthIn35mmFilm'), 11438 (41990, 'SceneCaptureType'), 11439 (41991, 'GainControl'), 11440 (41992, 'Contrast'), 11441 (41993, 'Saturation'), 11442 (41994, 'Sharpness'), 11443 (41995, 'DeviceSettingDescription'), 11444 (41996, 'SubjectDistanceRange'), 11445 (42016, 'ImageUniqueID'), 11446 (42032, 'CameraOwnerName'), 11447 (42033, 'BodySerialNumber'), 11448 (42034, 'LensSpecification'), 11449 (42035, 'LensMake'), 11450 (42036, 'LensModel'), 11451 (42037, 'LensSerialNumber'), 11452 (42080, 'CompositeImage'), 11453 (42081, 'SourceImageNumberCompositeImage'), 11454 (42082, 'SourceExposureTimesCompositeImage'), 11455 (42112, 'GDAL_METADATA'), 11456 (42113, 'GDAL_NODATA'), 11457 (42240, 'Gamma'), 11458 (43314, 'NIHImageHeader'), 11459 (44992, 'ExpandSoftware'), 11460 (44993, 'ExpandLens'), 11461 (44994, 'ExpandFilm'), 11462 (44995, 'ExpandFilterLens'), 11463 (44996, 'ExpandScanner'), 11464 (44997, 'ExpandFlashLamp'), 11465 (48129, 'PixelFormat'), # HDP and WDP 11466 (48130, 'Transformation'), 11467 (48131, 'Uncompressed'), 11468 (48132, 'ImageType'), 11469 (48256, 'ImageWidth'), # 256 11470 (48257, 'ImageHeight'), 11471 (48258, 'WidthResolution'), 11472 (48259, 'HeightResolution'), 11473 (48320, 'ImageOffset'), 11474 (48321, 'ImageByteCount'), 11475 (48322, 'AlphaOffset'), 11476 (48323, 'AlphaByteCount'), 11477 (48324, 'ImageDataDiscard'), 11478 (48325, 'AlphaDataDiscard'), 11479 (50003, 'KodakAPP3'), 11480 (50215, 'OceScanjobDescription'), 11481 (50216, 'OceApplicationSelector'), 11482 (50217, 'OceIdentificationNumber'), 11483 (50218, 'OceImageLogicCharacteristics'), 11484 (50255, 'Annotations'), 11485 (50288, 'MC_Id'), # Media Cybernetics 11486 (50289, 'MC_XYPosition'), 11487 (50290, 'MC_ZPosition'), 11488 (50291, 'MC_XYCalibration'), 11489 (50292, 'MC_LensCharacteristics'), 11490 (50293, 'MC_ChannelName'), 11491 (50294, 'MC_ExcitationWavelength'), 11492 (50295, 'MC_TimeStamp'), 11493 (50296, 'MC_FrameProperties'), 11494 (50341, 'PrintImageMatching'), 11495 (50495, 'PCO_RAW'), # TODO, PCO CamWare 11496 (50547, 'OriginalFileName'), 11497 (50560, 'USPTO_OriginalContentType'), # US Patent Office 11498 (50561, 'USPTO_RotationCode'), 11499 (50648, 'CR2Unknown1'), 11500 (50649, 'CR2Unknown2'), 11501 (50656, 'CR2CFAPattern'), 11502 (50674, 'LercParameters'), # ESGI 50674 .. 50677 11503 (50706, 'DNGVersion'), # DNG 50706 .. 51114 11504 (50707, 'DNGBackwardVersion'), 11505 (50708, 'UniqueCameraModel'), 11506 (50709, 'LocalizedCameraModel'), 11507 (50710, 'CFAPlaneColor'), 11508 (50711, 'CFALayout'), 11509 (50712, 'LinearizationTable'), 11510 (50713, 'BlackLevelRepeatDim'), 11511 (50714, 'BlackLevel'), 11512 (50715, 'BlackLevelDeltaH'), 11513 (50716, 'BlackLevelDeltaV'), 11514 (50717, 'WhiteLevel'), 11515 (50718, 'DefaultScale'), 11516 (50719, 'DefaultCropOrigin'), 11517 (50720, 'DefaultCropSize'), 11518 (50721, 'ColorMatrix1'), 11519 (50722, 'ColorMatrix2'), 11520 (50723, 'CameraCalibration1'), 11521 (50724, 'CameraCalibration2'), 11522 (50725, 'ReductionMatrix1'), 11523 (50726, 'ReductionMatrix2'), 11524 (50727, 'AnalogBalance'), 11525 (50728, 'AsShotNeutral'), 11526 (50729, 'AsShotWhiteXY'), 11527 (50730, 'BaselineExposure'), 11528 (50731, 'BaselineNoise'), 11529 (50732, 'BaselineSharpness'), 11530 (50733, 'BayerGreenSplit'), 11531 (50734, 'LinearResponseLimit'), 11532 (50735, 'CameraSerialNumber'), 11533 (50736, 'LensInfo'), 11534 (50737, 'ChromaBlurRadius'), 11535 (50738, 'AntiAliasStrength'), 11536 (50739, 'ShadowScale'), 11537 (50740, 'DNGPrivateData'), 11538 (50741, 'MakerNoteSafety'), 11539 (50752, 'RawImageSegmentation'), 11540 (50778, 'CalibrationIlluminant1'), 11541 (50779, 'CalibrationIlluminant2'), 11542 (50780, 'BestQualityScale'), 11543 (50781, 'RawDataUniqueID'), 11544 (50784, 'AliasLayerMetadata'), 11545 (50827, 'OriginalRawFileName'), 11546 (50828, 'OriginalRawFileData'), 11547 (50829, 'ActiveArea'), 11548 (50830, 'MaskedAreas'), 11549 (50831, 'AsShotICCProfile'), 11550 (50832, 'AsShotPreProfileMatrix'), 11551 (50833, 'CurrentICCProfile'), 11552 (50834, 'CurrentPreProfileMatrix'), 11553 (50838, 'IJMetadataByteCounts'), 11554 (50839, 'IJMetadata'), 11555 (50844, 'RPCCoefficientTag'), 11556 (50879, 'ColorimetricReference'), 11557 (50885, 'SRawType'), 11558 (50898, 'PanasonicTitle'), 11559 (50899, 'PanasonicTitle2'), 11560 (50908, 'RSID'), # DGIWG 11561 (50909, 'GEO_METADATA'), # DGIWG XML 11562 (50931, 'CameraCalibrationSignature'), 11563 (50932, 'ProfileCalibrationSignature'), 11564 (50933, 'ProfileIFD'), # EXTRACAMERAPROFILES 11565 (50934, 'AsShotProfileName'), 11566 (50935, 'NoiseReductionApplied'), 11567 (50936, 'ProfileName'), 11568 (50937, 'ProfileHueSatMapDims'), 11569 (50938, 'ProfileHueSatMapData1'), 11570 (50939, 'ProfileHueSatMapData2'), 11571 (50940, 'ProfileToneCurve'), 11572 (50941, 'ProfileEmbedPolicy'), 11573 (50942, 'ProfileCopyright'), 11574 (50964, 'ForwardMatrix1'), 11575 (50965, 'ForwardMatrix2'), 11576 (50966, 'PreviewApplicationName'), 11577 (50967, 'PreviewApplicationVersion'), 11578 (50968, 'PreviewSettingsName'), 11579 (50969, 'PreviewSettingsDigest'), 11580 (50970, 'PreviewColorSpace'), 11581 (50971, 'PreviewDateTime'), 11582 (50972, 'RawImageDigest'), 11583 (50973, 'OriginalRawFileDigest'), 11584 (50974, 'SubTileBlockSize'), 11585 (50975, 'RowInterleaveFactor'), 11586 (50981, 'ProfileLookTableDims'), 11587 (50982, 'ProfileLookTableData'), 11588 (51008, 'OpcodeList1'), 11589 (51009, 'OpcodeList2'), 11590 (51022, 'OpcodeList3'), 11591 (51023, 'FibicsXML'), # 11592 (51041, 'NoiseProfile'), 11593 (51043, 'TimeCodes'), 11594 (51044, 'FrameRate'), 11595 (51058, 'TStop'), 11596 (51081, 'ReelName'), 11597 (51089, 'OriginalDefaultFinalSize'), 11598 (51090, 'OriginalBestQualitySize'), 11599 (51091, 'OriginalDefaultCropSize'), 11600 (51105, 'CameraLabel'), 11601 (51107, 'ProfileHueSatMapEncoding'), 11602 (51108, 'ProfileLookTableEncoding'), 11603 (51109, 'BaselineExposureOffset'), 11604 (51110, 'DefaultBlackRender'), 11605 (51111, 'NewRawImageDigest'), 11606 (51112, 'RawToPreviewGain'), 11607 (51113, 'CacheBlob'), 11608 (51114, 'CacheVersion'), 11609 (51123, 'MicroManagerMetadata'), 11610 (51125, 'DefaultUserCrop'), 11611 (51159, 'ZIFmetadata'), # Objective Pathology Services 11612 (51160, 'ZIFannotations'), # Objective Pathology Services 11613 (51177, 'DepthFormat'), 11614 (51178, 'DepthNear'), 11615 (51179, 'DepthFar'), 11616 (51180, 'DepthUnits'), 11617 (51181, 'DepthMeasureType'), 11618 (51182, 'EnhanceParams'), 11619 (59932, 'Padding'), 11620 (59933, 'OffsetSchema'), 11621 # Reusable Tags 65000-65535 11622 # (65000, DimapDocumentXML'), 11623 # (65001, 'EER_XML'), 11624 # 65000-65112, Photoshop Camera RAW EXIF tags 11625 # (65000, 'OwnerName'), 11626 # (65001, 'SerialNumber'), 11627 # (65002, 'Lens'), 11628 # (65024, 'KodakKDCPrivateIFD'), 11629 # (65100, 'RawFile'), 11630 # (65101, 'Converter'), 11631 # (65102, 'WhiteBalance'), 11632 # (65105, 'Exposure'), 11633 # (65106, 'Shadows'), 11634 # (65107, 'Brightness'), 11635 # (65108, 'Contrast'), 11636 # (65109, 'Saturation'), 11637 # (65110, 'Sharpness'), 11638 # (65111, 'Smoothness'), 11639 # (65112, 'MoireFilter'), 11640 (65200, 'FlexXML'), 11641 ) 11642 ) 11643 11644 def TAG_READERS(): 11645 # map tag codes to import functions 11646 return { 11647 320: read_colormap, 11648 # 700: read_bytes, # read_utf8, 11649 # 34377: read_bytes, 11650 33723: read_bytes, 11651 # 34675: read_bytes, 11652 33628: read_uic1tag, # Universal Imaging Corp STK 11653 33629: read_uic2tag, 11654 33630: read_uic3tag, 11655 33631: read_uic4tag, 11656 34118: read_cz_sem, # Carl Zeiss SEM 11657 34361: read_mm_header, # Olympus FluoView 11658 34362: read_mm_stamp, 11659 34363: read_numpy, # MM_Unknown 11660 34386: read_numpy, # MM_UserBlock 11661 34412: read_cz_lsminfo, # Carl Zeiss LSM 11662 34680: read_fei_metadata, # S-FEG 11663 34682: read_fei_metadata, # Helios NanoLab 11664 37706: read_tvips_header, # TVIPS EMMENU 11665 37724: read_bytes, # ImageSourceData 11666 33923: read_bytes, # read_leica_magic 11667 43314: read_nih_image_header, 11668 # 40001: read_bytes, 11669 40100: read_bytes, 11670 50288: read_bytes, 11671 50296: read_bytes, 11672 50839: read_bytes, 11673 51123: read_json, 11674 33471: read_sis_ini, 11675 33560: read_sis, 11676 34665: read_exif_ifd, 11677 34853: read_gps_ifd, # conflicts with OlympusSIS 11678 40965: read_interoperability_ifd, 11679 65426: read_numpy, # NDPI McuStarts 11680 65432: read_numpy, # NDPI McuStartsHighBytes 11681 65439: read_numpy, # NDPI unknown 11682 65459: read_bytes, # NDPI bytes, not string 11683 } 11684 11685 def TAG_TUPLE(): 11686 # tags whose values must be stored as tuples 11687 return frozenset( 11688 (273, 279, 324, 325, 330, 338, 513, 514, 530, 531, 34736) 11689 ) 11690 11691 def TAG_ATTRIBUTES(): 11692 # map tag codes to TiffPage attribute names 11693 return { 11694 254: 'subfiletype', 11695 256: 'imagewidth', 11696 257: 'imagelength', 11697 # 258: 'bitspersample', # set manually 11698 259: 'compression', 11699 262: 'photometric', 11700 266: 'fillorder', 11701 270: 'description', 11702 277: 'samplesperpixel', 11703 278: 'rowsperstrip', 11704 284: 'planarconfig', 11705 305: 'software', 11706 320: 'colormap', 11707 317: 'predictor', 11708 322: 'tilewidth', 11709 323: 'tilelength', 11710 330: 'subifds', 11711 338: 'extrasamples', 11712 # 339: 'sampleformat', # set manually 11713 347: 'jpegtables', 11714 530: 'subsampling', 11715 32997: 'imagedepth', 11716 32998: 'tiledepth', 11717 } 11718 11719 def TAG_ENUM(): 11720 # map tag codes to Enums 11721 class TAG_ENUM: 11722 11723 __slots__ = ('_codes',) 11724 11725 def __init__(self): 11726 self._codes = { 11727 254: None, 11728 255: None, 11729 259: TIFF.COMPRESSION, 11730 262: TIFF.PHOTOMETRIC, 11731 263: None, 11732 266: None, 11733 274: None, 11734 284: TIFF.PLANARCONFIG, 11735 290: None, 11736 296: None, 11737 300: None, 11738 317: None, 11739 338: None, 11740 339: TIFF.SAMPLEFORMAT, 11741 } 11742 11743 def __contains__(self, key): 11744 return key in self._codes 11745 11746 def __getitem__(self, key): 11747 value = self._codes[key] 11748 if value is not None: 11749 return value 11750 if key == 254: 11751 value = TIFF.FILETYPE 11752 elif key == 255: 11753 value = TIFF.OFILETYPE 11754 elif key == 263: 11755 value = TIFF.THRESHHOLD 11756 elif key == 266: 11757 value = TIFF.FILLORDER 11758 elif key == 274: 11759 value = TIFF.ORIENTATION 11760 elif key == 290: 11761 value = TIFF.GRAYRESPONSEUNIT 11762 # elif key == 292: 11763 # value = TIFF.GROUP3OPT 11764 # elif key == 293: 11765 # value = TIFF.GROUP4OPT 11766 elif key == 296: 11767 value = TIFF.RESUNIT 11768 elif key == 300: 11769 value = TIFF.COLORRESPONSEUNIT 11770 elif key == 317: 11771 value = TIFF.PREDICTOR 11772 elif key == 338: 11773 value = TIFF.EXTRASAMPLE 11774 # elif key == 512: 11775 # TIFF.JPEGPROC 11776 # elif key == 531: 11777 # TIFF.YCBCRPOSITION 11778 else: 11779 raise KeyError(key) 11780 self._codes[key] = value 11781 return value 11782 11783 return TAG_ENUM() 11784 11785 def FILETYPE(): 11786 class FILETYPE(enum.IntFlag): 11787 UNDEFINED = 0 11788 REDUCEDIMAGE = 1 11789 PAGE = 2 11790 MASK = 4 11791 MACRO = 8 # Aperio SVS, or DNG Depth map 11792 ENHANCED = 16 # DNG 11793 DNG = 65536 # 65537: Alternative, 65540: Semantic mask 11794 11795 return FILETYPE 11796 11797 def OFILETYPE(): 11798 class OFILETYPE(enum.IntEnum): 11799 UNDEFINED = 0 11800 IMAGE = 1 11801 REDUCEDIMAGE = 2 11802 PAGE = 3 11803 11804 return OFILETYPE 11805 11806 def COMPRESSION(): 11807 class COMPRESSION(enum.IntEnum): 11808 NONE = 1 # Uncompressed 11809 CCITTRLE = 2 # CCITT 1D 11810 CCITT_T4 = 3 # T4/Group 3 Fax 11811 CCITT_T6 = 4 # T6/Group 4 Fax 11812 LZW = 5 11813 OJPEG = 6 # old-style JPEG 11814 JPEG = 7 11815 ADOBE_DEFLATE = 8 11816 JBIG_BW = 9 11817 JBIG_COLOR = 10 11818 JPEG_99 = 99 11819 KODAK_262 = 262 11820 JPEGXR_NDPI = 22610 11821 NEXT = 32766 11822 SONY_ARW = 32767 11823 PACKED_RAW = 32769 11824 SAMSUNG_SRW = 32770 11825 CCIRLEW = 32771 11826 SAMSUNG_SRW2 = 32772 11827 PACKBITS = 32773 11828 THUNDERSCAN = 32809 11829 IT8CTPAD = 32895 11830 IT8LW = 32896 11831 IT8MP = 32897 11832 IT8BL = 32898 11833 PIXARFILM = 32908 11834 PIXARLOG = 32909 11835 DEFLATE = 32946 11836 DCS = 32947 11837 APERIO_JP2000_YCBC = 33003 # Leica Aperio 11838 JPEG_2000_LOSSY = 33004 # BioFormats 11839 APERIO_JP2000_RGB = 33005 # Leica Aperio 11840 ALT_JPEG = 33007 # BioFormats 11841 JBIG = 34661 11842 SGILOG = 34676 11843 SGILOG24 = 34677 11844 JPEG2000 = 34712 11845 NIKON_NEF = 34713 11846 JBIG2 = 34715 11847 MDI_BINARY = 34718 # Microsoft Document Imaging 11848 MDI_PROGRESSIVE = 34719 # Microsoft Document Imaging 11849 MDI_VECTOR = 34720 # Microsoft Document Imaging 11850 LERC = 34887 # ESRI Lerc 11851 JPEG_LOSSY = 34892 # DNG 11852 LZMA = 34925 11853 ZSTD_DEPRECATED = 34926 11854 WEBP_DEPRECATED = 34927 11855 PNG = 34933 # Objective Pathology Services 11856 JPEGXR = 34934 # Objective Pathology Services 11857 ZSTD = 50000 11858 WEBP = 50001 11859 JPEGXL = 50002 # JXL 11860 PIXTIFF = 50013 11861 # EER_V0 = 65000 11862 # EER_V1 = 65001 11863 # KODAK_DCR = 65000 11864 # PENTAX_PEF = 65535 11865 11866 def __bool__(self): 11867 return self != 1 11868 11869 return COMPRESSION 11870 11871 def PHOTOMETRIC(): 11872 class PHOTOMETRIC(enum.IntEnum): 11873 MINISWHITE = 0 11874 MINISBLACK = 1 11875 RGB = 2 11876 PALETTE = 3 11877 MASK = 4 11878 SEPARATED = 5 # CMYK 11879 YCBCR = 6 11880 CIELAB = 8 11881 ICCLAB = 9 11882 ITULAB = 10 11883 CFA = 32803 # Color Filter Array 11884 LOGL = 32844 11885 LOGLUV = 32845 11886 LINEAR_RAW = 34892 11887 DEPTH_MAP = 51177 # DNG 1.5 11888 SEMANTIC_MASK = 52527 # DNG 1.6 11889 11890 return PHOTOMETRIC 11891 11892 def PHOTOMETRIC_SAMPLES(): 11893 return { 11894 0: 1, # MINISWHITE 11895 1: 1, # MINISBLACK 11896 2: 3, # RGB 11897 3: 1, # PALETTE 11898 4: 1, # MASK 11899 5: 4, # SEPARATED 11900 6: 3, # YCBCR 11901 8: 3, # CIELAB 11902 9: 3, # ICCLAB 11903 10: 3, # ITULAB 11904 32803: 1, # CFA 11905 32844: 1, # LOGL ? 11906 32845: 3, # LOGLUV 11907 34892: 3, # LINEAR_RAW ? 11908 51177: 1, # DEPTH_MAP ? 11909 52527: 1, # SEMANTIC_MASK ? 11910 } 11911 11912 def THRESHHOLD(): 11913 class THRESHHOLD(enum.IntEnum): 11914 BILEVEL = 1 11915 HALFTONE = 2 11916 ERRORDIFFUSE = 3 11917 11918 return THRESHHOLD 11919 11920 def FILLORDER(): 11921 class FILLORDER(enum.IntEnum): 11922 MSB2LSB = 1 11923 LSB2MSB = 2 11924 11925 return FILLORDER 11926 11927 def ORIENTATION(): 11928 class ORIENTATION(enum.IntEnum): 11929 TOPLEFT = 1 11930 TOPRIGHT = 2 11931 BOTRIGHT = 3 11932 BOTLEFT = 4 11933 LEFTTOP = 5 11934 RIGHTTOP = 6 11935 RIGHTBOT = 7 11936 LEFTBOT = 8 11937 11938 return ORIENTATION 11939 11940 def PLANARCONFIG(): 11941 class PLANARCONFIG(enum.IntEnum): 11942 CONTIG = 1 # CHUNKY 11943 SEPARATE = 2 11944 11945 return PLANARCONFIG 11946 11947 def GRAYRESPONSEUNIT(): 11948 class GRAYRESPONSEUNIT(enum.IntEnum): 11949 _10S = 1 11950 _100S = 2 11951 _1000S = 3 11952 _10000S = 4 11953 _100000S = 5 11954 11955 return GRAYRESPONSEUNIT 11956 11957 def GROUP4OPT(): 11958 class GROUP4OPT(enum.IntEnum): 11959 UNCOMPRESSED = 2 11960 11961 return GROUP4OPT 11962 11963 def RESUNIT(): 11964 class RESUNIT(enum.IntEnum): 11965 NONE = 1 11966 INCH = 2 11967 CENTIMETER = 3 11968 MILLIMETER = 4 # DNG 11969 MICROMETER = 5 # DNG 11970 11971 def __bool__(self): 11972 return self != 1 11973 11974 return RESUNIT 11975 11976 def COLORRESPONSEUNIT(): 11977 class COLORRESPONSEUNIT(enum.IntEnum): 11978 _10S = 1 11979 _100S = 2 11980 _1000S = 3 11981 _10000S = 4 11982 _100000S = 5 11983 11984 return COLORRESPONSEUNIT 11985 11986 def PREDICTOR(): 11987 class PREDICTOR(enum.IntEnum): 11988 NONE = 1 11989 HORIZONTAL = 2 11990 FLOATINGPOINT = 3 11991 HORIZONTALX2 = 34892 # DNG 11992 HORIZONTALX4 = 34893 11993 FLOATINGPOINTX2 = 34894 11994 FLOATINGPOINTX4 = 34895 11995 11996 def __bool__(self): 11997 return self != 1 11998 11999 return PREDICTOR 12000 12001 def EXTRASAMPLE(): 12002 class EXTRASAMPLE(enum.IntEnum): 12003 UNSPECIFIED = 0 12004 ASSOCALPHA = 1 12005 UNASSALPHA = 2 12006 12007 return EXTRASAMPLE 12008 12009 def SAMPLEFORMAT(): 12010 class SAMPLEFORMAT(enum.IntEnum): 12011 UINT = 1 12012 INT = 2 12013 IEEEFP = 3 12014 VOID = 4 12015 COMPLEXINT = 5 12016 COMPLEXIEEEFP = 6 12017 12018 return SAMPLEFORMAT 12019 12020 def DATATYPES(): 12021 class DATATYPES(enum.IntEnum): 12022 BYTE = 1 # 8-bit unsigned integer 12023 ASCII = 2 # 8-bit byte that contains a 7-bit ASCII code; 12024 # the last byte must be NULL (binary zero) 12025 SHORT = 3 # 16-bit (2-byte) unsigned integer 12026 LONG = 4 # 32-bit (4-byte) unsigned integer 12027 RATIONAL = 5 # two LONGs: the first represents the numerator 12028 # of a fraction; the second, the denominator 12029 SBYTE = 6 # an 8-bit signed (twos-complement) integer 12030 UNDEFINED = 7 # an 8-bit byte that may contain anything, 12031 # depending on the definition of the field 12032 SSHORT = 8 # A 16-bit (2-byte) signed (twos-complement) integer 12033 SLONG = 9 # a 32-bit (4-byte) signed (twos-complement) integer 12034 SRATIONAL = 10 # two SLONGs: the first represents the numerator 12035 # of a fraction, the second the denominator 12036 FLOAT = 11 # single precision (4-byte) IEEE format 12037 DOUBLE = 12 # double precision (8-byte) IEEE format 12038 IFD = 13 # unsigned 4 byte IFD offset 12039 UNICODE = 14 12040 COMPLEX = 15 12041 LONG8 = 16 # unsigned 8 byte integer (BigTiff) 12042 SLONG8 = 17 # signed 8 byte integer (BigTiff) 12043 IFD8 = 18 # unsigned 8 byte IFD offset (BigTiff) 12044 12045 return DATATYPES 12046 12047 def DATA_FORMATS(): 12048 # map TIFF DATATYPES to Python struct formats 12049 return { 12050 1: '1B', 12051 2: '1s', 12052 3: '1H', 12053 4: '1I', 12054 5: '2I', 12055 6: '1b', 12056 7: '1B', 12057 8: '1h', 12058 9: '1i', 12059 10: '2i', 12060 11: '1f', 12061 12: '1d', 12062 13: '1I', 12063 # 14: '', 12064 # 15: '', 12065 16: '1Q', 12066 17: '1q', 12067 18: '1Q', 12068 } 12069 12070 def DATA_DTYPES(): 12071 # map numpy dtypes to TIFF DATATYPES 12072 return { 12073 'B': 1, 12074 's': 2, 12075 'H': 3, 12076 'I': 4, 12077 '2I': 5, 12078 'b': 6, 12079 'h': 8, 12080 'i': 9, 12081 '2i': 10, 12082 'f': 11, 12083 'd': 12, 12084 'Q': 16, 12085 'q': 17, 12086 } 12087 12088 def SAMPLE_DTYPES(): 12089 # map SampleFormat and BitsPerSample to numpy dtype 12090 return { 12091 # UINT 12092 (1, 1): '?', # bitmap 12093 (1, 2): 'B', 12094 (1, 3): 'B', 12095 (1, 4): 'B', 12096 (1, 5): 'B', 12097 (1, 6): 'B', 12098 (1, 7): 'B', 12099 (1, 8): 'B', 12100 (1, 9): 'H', 12101 (1, 10): 'H', 12102 (1, 11): 'H', 12103 (1, 12): 'H', 12104 (1, 13): 'H', 12105 (1, 14): 'H', 12106 (1, 15): 'H', 12107 (1, 16): 'H', 12108 (1, 17): 'I', 12109 (1, 18): 'I', 12110 (1, 19): 'I', 12111 (1, 20): 'I', 12112 (1, 21): 'I', 12113 (1, 22): 'I', 12114 (1, 23): 'I', 12115 (1, 24): 'I', 12116 (1, 25): 'I', 12117 (1, 26): 'I', 12118 (1, 27): 'I', 12119 (1, 28): 'I', 12120 (1, 29): 'I', 12121 (1, 30): 'I', 12122 (1, 31): 'I', 12123 (1, 32): 'I', 12124 (1, 64): 'Q', 12125 # VOID : treat as UINT 12126 (4, 1): '?', # bitmap 12127 (4, 2): 'B', 12128 (4, 3): 'B', 12129 (4, 4): 'B', 12130 (4, 5): 'B', 12131 (4, 6): 'B', 12132 (4, 7): 'B', 12133 (4, 8): 'B', 12134 (4, 9): 'H', 12135 (4, 10): 'H', 12136 (4, 11): 'H', 12137 (4, 12): 'H', 12138 (4, 13): 'H', 12139 (4, 14): 'H', 12140 (4, 15): 'H', 12141 (4, 16): 'H', 12142 (4, 17): 'I', 12143 (4, 18): 'I', 12144 (4, 19): 'I', 12145 (4, 20): 'I', 12146 (4, 21): 'I', 12147 (4, 22): 'I', 12148 (4, 23): 'I', 12149 (4, 24): 'I', 12150 (4, 25): 'I', 12151 (4, 26): 'I', 12152 (4, 27): 'I', 12153 (4, 28): 'I', 12154 (4, 29): 'I', 12155 (4, 30): 'I', 12156 (4, 31): 'I', 12157 (4, 32): 'I', 12158 (4, 64): 'Q', 12159 # INT 12160 (2, 8): 'b', 12161 (2, 16): 'h', 12162 (2, 32): 'i', 12163 (2, 64): 'q', 12164 # IEEEFP 12165 (3, 16): 'e', 12166 (3, 24): 'f', # float24 bit not supported by numpy 12167 (3, 32): 'f', 12168 (3, 64): 'd', 12169 # COMPLEXIEEEFP 12170 (6, 64): 'F', 12171 (6, 128): 'D', 12172 # RGB565 12173 (1, (5, 6, 5)): 'B', 12174 # COMPLEXINT : not supported by numpy 12175 (5, 16): 'E', 12176 (5, 32): 'F', 12177 (5, 64): 'D', 12178 } 12179 12180 def PREDICTORS(): 12181 # map PREDICTOR to predictor encode functions 12182 12183 class PREDICTORS: 12184 def __init__(self): 12185 self._codecs = {None: identityfunc, 1: identityfunc} 12186 if imagecodecs is None: 12187 self._codecs[2] = delta_encode 12188 12189 def __getitem__(self, key): 12190 if key in self._codecs: 12191 return self._codecs[key] 12192 try: 12193 if key == 2: 12194 codec = imagecodecs.delta_encode 12195 elif key == 3: 12196 codec = imagecodecs.floatpred_encode 12197 elif key == 34892: 12198 12199 def codec(data, axis=-1, out=None): 12200 return imagecodecs.delta_encode( 12201 data, axis=axis, out=out, dist=2 12202 ) 12203 12204 elif key == 34893: 12205 12206 def codec(data, axis=-1, out=None): 12207 return imagecodecs.delta_encode( 12208 data, axis=axis, out=out, dist=4 12209 ) 12210 12211 elif key == 34894: 12212 12213 def codec(data, axis=-1, out=None): 12214 return imagecodecs.floatpred_encode( 12215 data, axis=axis, out=out, dist=2 12216 ) 12217 12218 elif key == 34895: 12219 12220 def codec(data, axis=-1, out=None): 12221 return imagecodecs.floatpred_encode( 12222 data, axis=axis, out=out, dist=4 12223 ) 12224 12225 else: 12226 raise KeyError(f'{key} is not a known PREDICTOR') 12227 except AttributeError: 12228 raise KeyError( 12229 f'{TIFF.PREDICTOR(key)!r}' 12230 " requires the 'imagecodecs' package" 12231 ) 12232 self._codecs[key] = codec 12233 return codec 12234 12235 return PREDICTORS() 12236 12237 def UNPREDICTORS(): 12238 # map PREDICTOR to predictor decode functions 12239 12240 class UNPREDICTORS: 12241 def __init__(self): 12242 self._codecs = {None: identityfunc, 1: identityfunc} 12243 if imagecodecs is None: 12244 self._codecs[2] = delta_decode 12245 12246 def __getitem__(self, key): 12247 if key in self._codecs: 12248 return self._codecs[key] 12249 try: 12250 if key == 2: 12251 codec = imagecodecs.delta_decode 12252 elif key == 3: 12253 codec = imagecodecs.floatpred_decode 12254 elif key == 34892: 12255 12256 def codec(data, axis=-1, out=None): 12257 return imagecodecs.delta_decode( 12258 data, axis=axis, out=out, dist=2 12259 ) 12260 12261 elif key == 34893: 12262 12263 def codec(data, axis=-1, out=None): 12264 return imagecodecs.delta_decode( 12265 data, axis=axis, out=out, dist=4 12266 ) 12267 12268 elif key == 34894: 12269 12270 def codec(data, axis=-1, out=None): 12271 return imagecodecs.floatpred_decode( 12272 data, axis=axis, out=out, dist=2 12273 ) 12274 12275 elif key == 34895: 12276 12277 def codec(data, axis=-1, out=None): 12278 return imagecodecs.floatpred_decode( 12279 data, axis=axis, out=out, dist=4 12280 ) 12281 12282 else: 12283 raise KeyError(f'{key} is not a known PREDICTOR') 12284 except AttributeError: 12285 raise KeyError( 12286 f'{TIFF.PREDICTOR(key)!r}' 12287 " requires the 'imagecodecs' package" 12288 ) 12289 self._codecs[key] = codec 12290 return codec 12291 12292 return UNPREDICTORS() 12293 12294 def COMPRESSORS(): 12295 # map COMPRESSION to compress functions 12296 12297 class COMPRESSORS: 12298 def __init__(self): 12299 self._codecs = {None: identityfunc, 1: identityfunc} 12300 if imagecodecs is None: 12301 self._codecs[8] = zlib_encode 12302 self._codecs[32946] = zlib_encode 12303 self._codecs[34925] = lzma_encode 12304 12305 def __getitem__(self, key): 12306 if key in self._codecs: 12307 return self._codecs[key] 12308 try: 12309 if key == 5: 12310 codec = imagecodecs.lzw_encode 12311 elif key == 7: 12312 codec = imagecodecs.jpeg_encode 12313 elif key == 8 or key == 32946: 12314 if ( 12315 hasattr(imagecodecs, 'DEFLATE') 12316 and imagecodecs.DEFLATE 12317 ): 12318 codec = imagecodecs.deflate_encode 12319 elif imagecodecs.ZLIB: 12320 codec = imagecodecs.zlib_encode 12321 else: 12322 codec = zlib_encode 12323 elif key == 32773: 12324 codec = imagecodecs.packbits_encode 12325 elif ( 12326 key == 33003 12327 or key == 33004 12328 or key == 33005 12329 or key == 34712 12330 ): 12331 codec = imagecodecs.jpeg2k_encode 12332 elif key == 34887: 12333 codec = imagecodecs.lerc_encode 12334 elif key == 34892: 12335 codec = imagecodecs.jpeg8_encode # DNG lossy 12336 elif key == 34925: 12337 if imagecodecs.LZMA: 12338 codec = imagecodecs.lzma_encode 12339 else: 12340 codec = lzma_encode 12341 elif key == 34933: 12342 codec = imagecodecs.png_encode 12343 elif key == 34934 or key == 22610: 12344 codec = imagecodecs.jpegxr_encode 12345 elif key == 50000: 12346 codec = imagecodecs.zstd_encode 12347 elif key == 50001: 12348 codec = imagecodecs.webp_encode 12349 elif key == 50002: 12350 codec = imagecodecs.jpegxl_encode 12351 else: 12352 try: 12353 msg = f'{TIFF.COMPRESSION(key)!r} not supported' 12354 except ValueError: 12355 msg = f'{key} is not a known COMPRESSION' 12356 raise KeyError(msg) 12357 except AttributeError: 12358 raise KeyError( 12359 f'{TIFF.COMPRESSION(key)!r} ' 12360 "requires the 'imagecodecs' package" 12361 ) 12362 self._codecs[key] = codec 12363 return codec 12364 12365 return COMPRESSORS() 12366 12367 def DECOMPRESSORS(): 12368 # map COMPRESSION to decompress functions 12369 12370 class DECOMPRESSORS: 12371 def __init__(self): 12372 self._codecs = {None: identityfunc, 1: identityfunc} 12373 if imagecodecs is None: 12374 self._codecs[8] = zlib_decode 12375 self._codecs[32773] = packbits_decode 12376 self._codecs[32946] = zlib_decode 12377 self._codecs[34925] = lzma_decode 12378 12379 def __getitem__(self, key): 12380 if key in self._codecs: 12381 return self._codecs[key] 12382 try: 12383 if key == 5: 12384 codec = imagecodecs.lzw_decode 12385 elif key == 6 or key == 7 or key == 33007: 12386 codec = imagecodecs.jpeg_decode 12387 elif key == 8 or key == 32946: 12388 if ( 12389 hasattr(imagecodecs, 'DEFLATE') 12390 and imagecodecs.DEFLATE 12391 ): 12392 codec = imagecodecs.deflate_decode 12393 elif imagecodecs.ZLIB: 12394 codec = imagecodecs.zlib_decode 12395 else: 12396 codec = zlib_decode 12397 elif key == 32773: 12398 codec = imagecodecs.packbits_decode 12399 elif ( 12400 key == 33003 12401 or key == 33004 12402 or key == 33005 12403 or key == 34712 12404 ): 12405 codec = imagecodecs.jpeg2k_decode 12406 elif key == 34887: 12407 codec = imagecodecs.lerc_decode 12408 elif key == 34892: 12409 codec = imagecodecs.jpeg8_decode # DNG lossy 12410 elif key == 34925: 12411 if imagecodecs.LZMA: 12412 codec = imagecodecs.lzma_decode 12413 else: 12414 codec = lzma_decode 12415 elif key == 34933: 12416 codec = imagecodecs.png_decode 12417 elif key == 34934 or key == 22610: 12418 codec = imagecodecs.jpegxr_decode 12419 elif key == 50000 or key == 34926: # 34926 deprecated 12420 codec = imagecodecs.zstd_decode 12421 elif key == 50001 or key == 34927: # 34927 deprecated 12422 codec = imagecodecs.webp_decode 12423 elif key == 50002: 12424 codec = imagecodecs.jpegxl_decode 12425 else: 12426 try: 12427 msg = f'{TIFF.COMPRESSION(key)!r} not supported' 12428 except ValueError: 12429 msg = f'{key} is not a known COMPRESSION' 12430 raise KeyError(msg) 12431 except AttributeError: 12432 raise KeyError( 12433 f'{TIFF.COMPRESSION(key)!r} ' 12434 "requires the 'imagecodecs' package" 12435 ) 12436 self._codecs[key] = codec 12437 return codec 12438 12439 def __contains__(self, key): 12440 try: 12441 self[key] 12442 except KeyError: 12443 return False 12444 return True 12445 12446 return DECOMPRESSORS() 12447 12448 def FRAME_ATTRS(): 12449 # attributes that a TiffFrame shares with its keyframe 12450 return {'shape', 'ndim', 'size', 'dtype', 'axes', 'is_final', 'decode'} 12451 12452 def FILE_FLAGS(): 12453 # TiffFile and TiffPage 'is_\*' attributes 12454 exclude = { 12455 'reduced', 12456 'mask', 12457 'final', 12458 'memmappable', 12459 'contiguous', 12460 'tiled', 12461 'subsampled', 12462 } 12463 return { 12464 a[3:] 12465 for a in dir(TiffPage) 12466 if a[:3] == 'is_' and a[3:] not in exclude 12467 } 12468 12469 def FILE_PATTERNS(): 12470 # predefined FileSequence patterns 12471 return { 12472 'axes': r"""(?ix) 12473 # matches Olympus OIF and Leica TIFF series 12474 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4})) 12475 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))? 12476 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))? 12477 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))? 12478 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))? 12479 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))? 12480 _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))? 12481 """ 12482 } 12483 12484 def FILE_EXTENSIONS(): 12485 # TIFF file extensions 12486 return ( 12487 'tif', 12488 'tiff', 12489 'ome.tif', 12490 'lsm', 12491 'stk', 12492 'qpi', 12493 'pcoraw', 12494 'qptiff', 12495 'gel', 12496 'seq', 12497 'svs', 12498 'scn', 12499 'zif', 12500 'ndpi', 12501 'bif', 12502 'tf8', 12503 'tf2', 12504 'btf', 12505 'eer', 12506 ) 12507 12508 def FILEOPEN_FILTER(): 12509 # string for use in Windows File Open box 12510 return [ 12511 (f'{ext.upper()} files', f'*.{ext}') 12512 for ext in TIFF.FILE_EXTENSIONS 12513 ] + [('allfiles', '*')] 12514 12515 def AXES_LABELS(): 12516 12517 # TODO: is there a standard for character axes labels? 12518 axes = { 12519 'X': 'width', 12520 'Y': 'length', # height 12521 'Z': 'depth', 12522 'S': 'sample', # rgb(a), cmyk 12523 'I': 'series', # general sequence of frames/planes/pages/IFDs 12524 'T': 'time', 12525 'C': 'channel', # color, emission wavelength 12526 'A': 'angle', 12527 'P': 'phase', # formerly F # P is Position in LSM! 12528 'R': 'tile', # region, point, mosaic 12529 'H': 'lifetime', # histogram 12530 'E': 'lambda', # excitation wavelength 12531 'L': 'exposure', # lux 12532 'V': 'event', 12533 'Q': 'other', 12534 'M': 'mosaic', # LSM 6 12535 } 12536 axes.update({v: k for k, v in axes.items()}) 12537 return axes 12538 12539 def NDPI_TAGS(): 12540 # 65420 - 65458 Private Hamamatsu NDPI tags 12541 # TODO: obtain specification 12542 return TiffTagRegistry( 12543 ( 12544 (65324, 'OffsetHighBytes'), 12545 (65325, 'ByteCountHighBytes'), 12546 (65420, 'FileFormat'), 12547 (65421, 'Magnification'), # SourceLens 12548 (65422, 'XOffsetFromSlideCenter'), 12549 (65423, 'YOffsetFromSlideCenter'), 12550 (65424, 'ZOffsetFromSlideCenter'), # FocalPlane 12551 (65425, 'TissueIndex'), 12552 (65426, 'McuStarts'), 12553 (65427, 'SlideLabel'), 12554 (65428, 'AuthCode'), # ? 12555 (65429, '65429'), 12556 (65430, '65430'), 12557 (65431, '65431'), 12558 (65432, 'McuStartsHighBytes'), 12559 (65433, '65433'), 12560 (65434, 'Fluorescence'), # FilterSetName 12561 (65435, 'ExposureRatio'), 12562 (65436, 'RedMultiplier'), 12563 (65437, 'GreenMultiplier'), 12564 (65438, 'BlueMultiplier'), 12565 (65439, 'FocusPoints'), 12566 (65440, 'FocusPointRegions'), 12567 (65441, 'CaptureMode'), 12568 (65442, 'ScannerSerialNumber'), 12569 (65443, '65443'), 12570 (65444, 'JpegQuality'), 12571 (65445, 'RefocusInterval'), 12572 (65446, 'FocusOffset'), 12573 (65447, 'BlankLines'), 12574 (65448, 'FirmwareVersion'), 12575 (65449, 'Comments'), # PropertyMap, CalibrationInfo 12576 (65450, 'LabelObscured'), 12577 (65451, 'Wavelength'), 12578 (65452, '65452'), 12579 (65453, 'LampAge'), 12580 (65454, 'ExposureTime'), 12581 (65455, 'FocusTime'), 12582 (65456, 'ScanTime'), 12583 (65457, 'WriteTime'), 12584 (65458, 'FullyAutoFocus'), 12585 (65500, 'DefaultGamma'), 12586 ) 12587 ) 12588 12589 def EXIF_TAGS(): 12590 # 65000 - 65112 Photoshop Camera RAW EXIF tags 12591 tags = TiffTagRegistry( 12592 ( 12593 (65000, 'OwnerName'), 12594 (65001, 'SerialNumber'), 12595 (65002, 'Lens'), 12596 (65100, 'RawFile'), 12597 (65101, 'Converter'), 12598 (65102, 'WhiteBalance'), 12599 (65105, 'Exposure'), 12600 (65106, 'Shadows'), 12601 (65107, 'Brightness'), 12602 (65108, 'Contrast'), 12603 (65109, 'Saturation'), 12604 (65110, 'Sharpness'), 12605 (65111, 'Smoothness'), 12606 (65112, 'MoireFilter'), 12607 ) 12608 ) 12609 tags.update(TIFF.TAGS) 12610 return tags 12611 12612 def GPS_TAGS(): 12613 return TiffTagRegistry( 12614 ( 12615 (0, 'GPSVersionID'), 12616 (1, 'GPSLatitudeRef'), 12617 (2, 'GPSLatitude'), 12618 (3, 'GPSLongitudeRef'), 12619 (4, 'GPSLongitude'), 12620 (5, 'GPSAltitudeRef'), 12621 (6, 'GPSAltitude'), 12622 (7, 'GPSTimeStamp'), 12623 (8, 'GPSSatellites'), 12624 (9, 'GPSStatus'), 12625 (10, 'GPSMeasureMode'), 12626 (11, 'GPSDOP'), 12627 (12, 'GPSSpeedRef'), 12628 (13, 'GPSSpeed'), 12629 (14, 'GPSTrackRef'), 12630 (15, 'GPSTrack'), 12631 (16, 'GPSImgDirectionRef'), 12632 (17, 'GPSImgDirection'), 12633 (18, 'GPSMapDatum'), 12634 (19, 'GPSDestLatitudeRef'), 12635 (20, 'GPSDestLatitude'), 12636 (21, 'GPSDestLongitudeRef'), 12637 (22, 'GPSDestLongitude'), 12638 (23, 'GPSDestBearingRef'), 12639 (24, 'GPSDestBearing'), 12640 (25, 'GPSDestDistanceRef'), 12641 (26, 'GPSDestDistance'), 12642 (27, 'GPSProcessingMethod'), 12643 (28, 'GPSAreaInformation'), 12644 (29, 'GPSDateStamp'), 12645 (30, 'GPSDifferential'), 12646 (31, 'GPSHPositioningError'), 12647 ) 12648 ) 12649 12650 def IOP_TAGS(): 12651 return TiffTagRegistry( 12652 ( 12653 (1, 'InteroperabilityIndex'), 12654 (2, 'InteroperabilityVersion'), 12655 (4096, 'RelatedImageFileFormat'), 12656 (4097, 'RelatedImageWidth'), 12657 (4098, 'RelatedImageLength'), 12658 ) 12659 ) 12660 12661 def GEO_KEYS(): 12662 try: 12663 from .tifffile_geodb import GeoKeys # delayed import 12664 except ImportError: 12665 try: 12666 from tifffile_geodb import GeoKeys # delayed import 12667 except ImportError: 12668 12669 class GeoKeys(enum.IntEnum): 12670 pass 12671 12672 return GeoKeys 12673 12674 def GEO_CODES(): 12675 try: 12676 from .tifffile_geodb import GEO_CODES # delayed import 12677 except ImportError: 12678 try: 12679 from tifffile_geodb import GEO_CODES # delayed import 12680 except ImportError: 12681 GEO_CODES = {} 12682 return GEO_CODES 12683 12684 def CZ_LSMINFO(): 12685 return [ 12686 ('MagicNumber', 'u4'), 12687 ('StructureSize', 'i4'), 12688 ('DimensionX', 'i4'), 12689 ('DimensionY', 'i4'), 12690 ('DimensionZ', 'i4'), 12691 ('DimensionChannels', 'i4'), 12692 ('DimensionTime', 'i4'), 12693 ('DataType', 'i4'), # DATATYPES 12694 ('ThumbnailX', 'i4'), 12695 ('ThumbnailY', 'i4'), 12696 ('VoxelSizeX', 'f8'), 12697 ('VoxelSizeY', 'f8'), 12698 ('VoxelSizeZ', 'f8'), 12699 ('OriginX', 'f8'), 12700 ('OriginY', 'f8'), 12701 ('OriginZ', 'f8'), 12702 ('ScanType', 'u2'), 12703 ('SpectralScan', 'u2'), 12704 ('TypeOfData', 'u4'), # TYPEOFDATA 12705 ('OffsetVectorOverlay', 'u4'), 12706 ('OffsetInputLut', 'u4'), 12707 ('OffsetOutputLut', 'u4'), 12708 ('OffsetChannelColors', 'u4'), 12709 ('TimeIntervall', 'f8'), 12710 ('OffsetChannelDataTypes', 'u4'), 12711 ('OffsetScanInformation', 'u4'), # SCANINFO 12712 ('OffsetKsData', 'u4'), 12713 ('OffsetTimeStamps', 'u4'), 12714 ('OffsetEventList', 'u4'), 12715 ('OffsetRoi', 'u4'), 12716 ('OffsetBleachRoi', 'u4'), 12717 ('OffsetNextRecording', 'u4'), 12718 # LSM 2.0 ends here 12719 ('DisplayAspectX', 'f8'), 12720 ('DisplayAspectY', 'f8'), 12721 ('DisplayAspectZ', 'f8'), 12722 ('DisplayAspectTime', 'f8'), 12723 ('OffsetMeanOfRoisOverlay', 'u4'), 12724 ('OffsetTopoIsolineOverlay', 'u4'), 12725 ('OffsetTopoProfileOverlay', 'u4'), 12726 ('OffsetLinescanOverlay', 'u4'), 12727 ('ToolbarFlags', 'u4'), 12728 ('OffsetChannelWavelength', 'u4'), 12729 ('OffsetChannelFactors', 'u4'), 12730 ('ObjectiveSphereCorrection', 'f8'), 12731 ('OffsetUnmixParameters', 'u4'), 12732 # LSM 3.2, 4.0 end here 12733 ('OffsetAcquisitionParameters', 'u4'), 12734 ('OffsetCharacteristics', 'u4'), 12735 ('OffsetPalette', 'u4'), 12736 ('TimeDifferenceX', 'f8'), 12737 ('TimeDifferenceY', 'f8'), 12738 ('TimeDifferenceZ', 'f8'), 12739 ('InternalUse1', 'u4'), 12740 ('DimensionP', 'i4'), 12741 ('DimensionM', 'i4'), 12742 ('DimensionsReserved', '16i4'), 12743 ('OffsetTilePositions', 'u4'), 12744 ('', '9u4'), # Reserved 12745 ('OffsetPositions', 'u4'), 12746 # ('', '21u4'), # must be 0 12747 ] 12748 12749 def CZ_LSMINFO_READERS(): 12750 # import functions for CZ_LSMINFO sub-records 12751 # TODO: read more CZ_LSMINFO sub-records 12752 return { 12753 'ScanInformation': read_lsm_scaninfo, 12754 'TimeStamps': read_lsm_timestamps, 12755 'EventList': read_lsm_eventlist, 12756 'ChannelColors': read_lsm_channelcolors, 12757 'Positions': read_lsm_positions, 12758 'TilePositions': read_lsm_positions, 12759 'VectorOverlay': None, 12760 'InputLut': read_lsm_lookuptable, 12761 'OutputLut': read_lsm_lookuptable, 12762 'TimeIntervall': None, 12763 'ChannelDataTypes': read_lsm_channeldatatypes, 12764 'KsData': None, 12765 'Roi': None, 12766 'BleachRoi': None, 12767 'NextRecording': None, # read with TiffFile(fh, offset=) 12768 'MeanOfRoisOverlay': None, 12769 'TopoIsolineOverlay': None, 12770 'TopoProfileOverlay': None, 12771 'ChannelWavelength': read_lsm_channelwavelength, 12772 'SphereCorrection': None, 12773 'ChannelFactors': None, 12774 'UnmixParameters': None, 12775 'AcquisitionParameters': None, 12776 'Characteristics': None, 12777 } 12778 12779 def CZ_LSMINFO_SCANTYPE(): 12780 # map CZ_LSMINFO.ScanType to dimension order 12781 return { 12782 0: 'XYZCT', # 'Stack' normal x-y-z-scan 12783 1: 'XYZCT', # 'Z-Scan' x-z-plane Y=1 12784 2: 'XYZCT', # 'Line' 12785 3: 'XYTCZ', # 'Time Series Plane' time series x-y XYCTZ ? Z=1 12786 4: 'XYZTC', # 'Time Series z-Scan' time series x-z 12787 5: 'XYTCZ', # 'Time Series Mean-of-ROIs' 12788 6: 'XYZTC', # 'Time Series Stack' time series x-y-z 12789 7: 'XYCTZ', # Spline Scan 12790 8: 'XYCZT', # Spline Plane x-z 12791 9: 'XYTCZ', # Time Series Spline Plane x-z 12792 10: 'XYZCT', # 'Time Series Point' point mode 12793 } 12794 12795 def CZ_LSMINFO_DIMENSIONS(): 12796 # map dimension codes to CZ_LSMINFO attribute 12797 return { 12798 'X': 'DimensionX', 12799 'Y': 'DimensionY', 12800 'Z': 'DimensionZ', 12801 'C': 'DimensionChannels', 12802 'T': 'DimensionTime', 12803 'P': 'DimensionP', 12804 'M': 'DimensionM', 12805 } 12806 12807 def CZ_LSMINFO_DATATYPES(): 12808 # description of CZ_LSMINFO.DataType 12809 return { 12810 0: 'varying data types', 12811 1: '8 bit unsigned integer', 12812 2: '12 bit unsigned integer', 12813 5: '32 bit float', 12814 } 12815 12816 def CZ_LSMINFO_TYPEOFDATA(): 12817 # description of CZ_LSMINFO.TypeOfData 12818 return { 12819 0: 'Original scan data', 12820 1: 'Calculated data', 12821 2: '3D reconstruction', 12822 3: 'Topography height map', 12823 } 12824 12825 def CZ_LSMINFO_SCANINFO_ARRAYS(): 12826 return { 12827 0x20000000: 'Tracks', 12828 0x30000000: 'Lasers', 12829 0x60000000: 'DetectionChannels', 12830 0x80000000: 'IlluminationChannels', 12831 0xA0000000: 'BeamSplitters', 12832 0xC0000000: 'DataChannels', 12833 0x11000000: 'Timers', 12834 0x13000000: 'Markers', 12835 } 12836 12837 def CZ_LSMINFO_SCANINFO_STRUCTS(): 12838 return { 12839 # 0x10000000: 'Recording', 12840 0x40000000: 'Track', 12841 0x50000000: 'Laser', 12842 0x70000000: 'DetectionChannel', 12843 0x90000000: 'IlluminationChannel', 12844 0xB0000000: 'BeamSplitter', 12845 0xD0000000: 'DataChannel', 12846 0x12000000: 'Timer', 12847 0x14000000: 'Marker', 12848 } 12849 12850 def CZ_LSMINFO_SCANINFO_ATTRIBUTES(): 12851 return { 12852 # Recording 12853 0x10000001: 'Name', 12854 0x10000002: 'Description', 12855 0x10000003: 'Notes', 12856 0x10000004: 'Objective', 12857 0x10000005: 'ProcessingSummary', 12858 0x10000006: 'SpecialScanMode', 12859 0x10000007: 'ScanType', 12860 0x10000008: 'ScanMode', 12861 0x10000009: 'NumberOfStacks', 12862 0x1000000A: 'LinesPerPlane', 12863 0x1000000B: 'SamplesPerLine', 12864 0x1000000C: 'PlanesPerVolume', 12865 0x1000000D: 'ImagesWidth', 12866 0x1000000E: 'ImagesHeight', 12867 0x1000000F: 'ImagesNumberPlanes', 12868 0x10000010: 'ImagesNumberStacks', 12869 0x10000011: 'ImagesNumberChannels', 12870 0x10000012: 'LinscanXySize', 12871 0x10000013: 'ScanDirection', 12872 0x10000014: 'TimeSeries', 12873 0x10000015: 'OriginalScanData', 12874 0x10000016: 'ZoomX', 12875 0x10000017: 'ZoomY', 12876 0x10000018: 'ZoomZ', 12877 0x10000019: 'Sample0X', 12878 0x1000001A: 'Sample0Y', 12879 0x1000001B: 'Sample0Z', 12880 0x1000001C: 'SampleSpacing', 12881 0x1000001D: 'LineSpacing', 12882 0x1000001E: 'PlaneSpacing', 12883 0x1000001F: 'PlaneWidth', 12884 0x10000020: 'PlaneHeight', 12885 0x10000021: 'VolumeDepth', 12886 0x10000023: 'Nutation', 12887 0x10000034: 'Rotation', 12888 0x10000035: 'Precession', 12889 0x10000036: 'Sample0time', 12890 0x10000037: 'StartScanTriggerIn', 12891 0x10000038: 'StartScanTriggerOut', 12892 0x10000039: 'StartScanEvent', 12893 0x10000040: 'StartScanTime', 12894 0x10000041: 'StopScanTriggerIn', 12895 0x10000042: 'StopScanTriggerOut', 12896 0x10000043: 'StopScanEvent', 12897 0x10000044: 'StopScanTime', 12898 0x10000045: 'UseRois', 12899 0x10000046: 'UseReducedMemoryRois', 12900 0x10000047: 'User', 12901 0x10000048: 'UseBcCorrection', 12902 0x10000049: 'PositionBcCorrection1', 12903 0x10000050: 'PositionBcCorrection2', 12904 0x10000051: 'InterpolationY', 12905 0x10000052: 'CameraBinning', 12906 0x10000053: 'CameraSupersampling', 12907 0x10000054: 'CameraFrameWidth', 12908 0x10000055: 'CameraFrameHeight', 12909 0x10000056: 'CameraOffsetX', 12910 0x10000057: 'CameraOffsetY', 12911 0x10000059: 'RtBinning', 12912 0x1000005A: 'RtFrameWidth', 12913 0x1000005B: 'RtFrameHeight', 12914 0x1000005C: 'RtRegionWidth', 12915 0x1000005D: 'RtRegionHeight', 12916 0x1000005E: 'RtOffsetX', 12917 0x1000005F: 'RtOffsetY', 12918 0x10000060: 'RtZoom', 12919 0x10000061: 'RtLinePeriod', 12920 0x10000062: 'Prescan', 12921 0x10000063: 'ScanDirectionZ', 12922 # Track 12923 0x40000001: 'MultiplexType', # 0 After Line; 1 After Frame 12924 0x40000002: 'MultiplexOrder', 12925 0x40000003: 'SamplingMode', # 0 Sample; 1 Line Avg; 2 Frame Avg 12926 0x40000004: 'SamplingMethod', # 1 Mean; 2 Sum 12927 0x40000005: 'SamplingNumber', 12928 0x40000006: 'Acquire', 12929 0x40000007: 'SampleObservationTime', 12930 0x4000000B: 'TimeBetweenStacks', 12931 0x4000000C: 'Name', 12932 0x4000000D: 'Collimator1Name', 12933 0x4000000E: 'Collimator1Position', 12934 0x4000000F: 'Collimator2Name', 12935 0x40000010: 'Collimator2Position', 12936 0x40000011: 'IsBleachTrack', 12937 0x40000012: 'IsBleachAfterScanNumber', 12938 0x40000013: 'BleachScanNumber', 12939 0x40000014: 'TriggerIn', 12940 0x40000015: 'TriggerOut', 12941 0x40000016: 'IsRatioTrack', 12942 0x40000017: 'BleachCount', 12943 0x40000018: 'SpiCenterWavelength', 12944 0x40000019: 'PixelTime', 12945 0x40000021: 'CondensorFrontlens', 12946 0x40000023: 'FieldStopValue', 12947 0x40000024: 'IdCondensorAperture', 12948 0x40000025: 'CondensorAperture', 12949 0x40000026: 'IdCondensorRevolver', 12950 0x40000027: 'CondensorFilter', 12951 0x40000028: 'IdTransmissionFilter1', 12952 0x40000029: 'IdTransmission1', 12953 0x40000030: 'IdTransmissionFilter2', 12954 0x40000031: 'IdTransmission2', 12955 0x40000032: 'RepeatBleach', 12956 0x40000033: 'EnableSpotBleachPos', 12957 0x40000034: 'SpotBleachPosx', 12958 0x40000035: 'SpotBleachPosy', 12959 0x40000036: 'SpotBleachPosz', 12960 0x40000037: 'IdTubelens', 12961 0x40000038: 'IdTubelensPosition', 12962 0x40000039: 'TransmittedLight', 12963 0x4000003A: 'ReflectedLight', 12964 0x4000003B: 'SimultanGrabAndBleach', 12965 0x4000003C: 'BleachPixelTime', 12966 # Laser 12967 0x50000001: 'Name', 12968 0x50000002: 'Acquire', 12969 0x50000003: 'Power', 12970 # DetectionChannel 12971 0x70000001: 'IntegrationMode', 12972 0x70000002: 'SpecialMode', 12973 0x70000003: 'DetectorGainFirst', 12974 0x70000004: 'DetectorGainLast', 12975 0x70000005: 'AmplifierGainFirst', 12976 0x70000006: 'AmplifierGainLast', 12977 0x70000007: 'AmplifierOffsFirst', 12978 0x70000008: 'AmplifierOffsLast', 12979 0x70000009: 'PinholeDiameter', 12980 0x7000000A: 'CountingTrigger', 12981 0x7000000B: 'Acquire', 12982 0x7000000C: 'PointDetectorName', 12983 0x7000000D: 'AmplifierName', 12984 0x7000000E: 'PinholeName', 12985 0x7000000F: 'FilterSetName', 12986 0x70000010: 'FilterName', 12987 0x70000013: 'IntegratorName', 12988 0x70000014: 'ChannelName', 12989 0x70000015: 'DetectorGainBc1', 12990 0x70000016: 'DetectorGainBc2', 12991 0x70000017: 'AmplifierGainBc1', 12992 0x70000018: 'AmplifierGainBc2', 12993 0x70000019: 'AmplifierOffsetBc1', 12994 0x70000020: 'AmplifierOffsetBc2', 12995 0x70000021: 'SpectralScanChannels', 12996 0x70000022: 'SpiWavelengthStart', 12997 0x70000023: 'SpiWavelengthStop', 12998 0x70000026: 'DyeName', 12999 0x70000027: 'DyeFolder', 13000 # IlluminationChannel 13001 0x90000001: 'Name', 13002 0x90000002: 'Power', 13003 0x90000003: 'Wavelength', 13004 0x90000004: 'Aquire', 13005 0x90000005: 'DetchannelName', 13006 0x90000006: 'PowerBc1', 13007 0x90000007: 'PowerBc2', 13008 # BeamSplitter 13009 0xB0000001: 'FilterSet', 13010 0xB0000002: 'Filter', 13011 0xB0000003: 'Name', 13012 # DataChannel 13013 0xD0000001: 'Name', 13014 0xD0000003: 'Acquire', 13015 0xD0000004: 'Color', 13016 0xD0000005: 'SampleType', 13017 0xD0000006: 'BitsPerSample', 13018 0xD0000007: 'RatioType', 13019 0xD0000008: 'RatioTrack1', 13020 0xD0000009: 'RatioTrack2', 13021 0xD000000A: 'RatioChannel1', 13022 0xD000000B: 'RatioChannel2', 13023 0xD000000C: 'RatioConst1', 13024 0xD000000D: 'RatioConst2', 13025 0xD000000E: 'RatioConst3', 13026 0xD000000F: 'RatioConst4', 13027 0xD0000010: 'RatioConst5', 13028 0xD0000011: 'RatioConst6', 13029 0xD0000012: 'RatioFirstImages1', 13030 0xD0000013: 'RatioFirstImages2', 13031 0xD0000014: 'DyeName', 13032 0xD0000015: 'DyeFolder', 13033 0xD0000016: 'Spectrum', 13034 0xD0000017: 'Acquire', 13035 # Timer 13036 0x12000001: 'Name', 13037 0x12000002: 'Description', 13038 0x12000003: 'Interval', 13039 0x12000004: 'TriggerIn', 13040 0x12000005: 'TriggerOut', 13041 0x12000006: 'ActivationTime', 13042 0x12000007: 'ActivationNumber', 13043 # Marker 13044 0x14000001: 'Name', 13045 0x14000002: 'Description', 13046 0x14000003: 'TriggerIn', 13047 0x14000004: 'TriggerOut', 13048 } 13049 13050 def CZ_LSM_LUTTYPE(): 13051 class CZ_LSM_LUTTYPE(enum.IntEnum): 13052 NORMAL = 0 13053 ORIGINAL = 1 13054 RAMP = 2 13055 POLYLINE = 3 13056 SPLINE = 4 13057 GAMMA = 5 13058 13059 return CZ_LSM_LUTTYPE 13060 13061 def CZ_LSM_SUBBLOCK_TYPE(): 13062 class CZ_LSM_SUBBLOCK_TYPE(enum.IntEnum): 13063 END = 0 13064 GAMMA = 1 13065 BRIGHTNESS = 2 13066 CONTRAST = 3 13067 RAMP = 4 13068 KNOTS = 5 13069 PALETTE_12_TO_12 = 6 13070 13071 return CZ_LSM_SUBBLOCK_TYPE 13072 13073 def NIH_IMAGE_HEADER(): 13074 return [ 13075 ('FileID', 'a8'), 13076 ('nLines', 'i2'), 13077 ('PixelsPerLine', 'i2'), 13078 ('Version', 'i2'), 13079 ('OldLutMode', 'i2'), 13080 ('OldnColors', 'i2'), 13081 ('Colors', 'u1', (3, 32)), 13082 ('OldColorStart', 'i2'), 13083 ('ColorWidth', 'i2'), 13084 ('ExtraColors', 'u2', (6, 3)), 13085 ('nExtraColors', 'i2'), 13086 ('ForegroundIndex', 'i2'), 13087 ('BackgroundIndex', 'i2'), 13088 ('XScale', 'f8'), 13089 ('Unused2', 'i2'), 13090 ('Unused3', 'i2'), 13091 ('UnitsID', 'i2'), # NIH_UNITS_TYPE 13092 ('p1', [('x', 'i2'), ('y', 'i2')]), 13093 ('p2', [('x', 'i2'), ('y', 'i2')]), 13094 ('CurveFitType', 'i2'), # NIH_CURVEFIT_TYPE 13095 ('nCoefficients', 'i2'), 13096 ('Coeff', 'f8', 6), 13097 ('UMsize', 'u1'), 13098 ('UM', 'a15'), 13099 ('UnusedBoolean', 'u1'), 13100 ('BinaryPic', 'b1'), 13101 ('SliceStart', 'i2'), 13102 ('SliceEnd', 'i2'), 13103 ('ScaleMagnification', 'f4'), 13104 ('nSlices', 'i2'), 13105 ('SliceSpacing', 'f4'), 13106 ('CurrentSlice', 'i2'), 13107 ('FrameInterval', 'f4'), 13108 ('PixelAspectRatio', 'f4'), 13109 ('ColorStart', 'i2'), 13110 ('ColorEnd', 'i2'), 13111 ('nColors', 'i2'), 13112 ('Fill1', '3u2'), 13113 ('Fill2', '3u2'), 13114 ('Table', 'u1'), # NIH_COLORTABLE_TYPE 13115 ('LutMode', 'u1'), # NIH_LUTMODE_TYPE 13116 ('InvertedTable', 'b1'), 13117 ('ZeroClip', 'b1'), 13118 ('XUnitSize', 'u1'), 13119 ('XUnit', 'a11'), 13120 ('StackType', 'i2'), # NIH_STACKTYPE_TYPE 13121 # ('UnusedBytes', 'u1', 200) 13122 ] 13123 13124 def NIH_COLORTABLE_TYPE(): 13125 return ( 13126 'CustomTable', 13127 'AppleDefault', 13128 'Pseudo20', 13129 'Pseudo32', 13130 'Rainbow', 13131 'Fire1', 13132 'Fire2', 13133 'Ice', 13134 'Grays', 13135 'Spectrum', 13136 ) 13137 13138 def NIH_LUTMODE_TYPE(): 13139 return ( 13140 'PseudoColor', 13141 'OldAppleDefault', 13142 'OldSpectrum', 13143 'GrayScale', 13144 'ColorLut', 13145 'CustomGrayscale', 13146 ) 13147 13148 def NIH_CURVEFIT_TYPE(): 13149 return ( 13150 'StraightLine', 13151 'Poly2', 13152 'Poly3', 13153 'Poly4', 13154 'Poly5', 13155 'ExpoFit', 13156 'PowerFit', 13157 'LogFit', 13158 'RodbardFit', 13159 'SpareFit1', 13160 'Uncalibrated', 13161 'UncalibratedOD', 13162 ) 13163 13164 def NIH_UNITS_TYPE(): 13165 return ( 13166 'Nanometers', 13167 'Micrometers', 13168 'Millimeters', 13169 'Centimeters', 13170 'Meters', 13171 'Kilometers', 13172 'Inches', 13173 'Feet', 13174 'Miles', 13175 'Pixels', 13176 'OtherUnits', 13177 ) 13178 13179 def TVIPS_HEADER_V1(): 13180 # TVIPS TemData structure from EMMENU Help file 13181 return [ 13182 ('Version', 'i4'), 13183 ('CommentV1', 'a80'), 13184 ('HighTension', 'i4'), 13185 ('SphericalAberration', 'i4'), 13186 ('IlluminationAperture', 'i4'), 13187 ('Magnification', 'i4'), 13188 ('PostMagnification', 'i4'), 13189 ('FocalLength', 'i4'), 13190 ('Defocus', 'i4'), 13191 ('Astigmatism', 'i4'), 13192 ('AstigmatismDirection', 'i4'), 13193 ('BiprismVoltage', 'i4'), 13194 ('SpecimenTiltAngle', 'i4'), 13195 ('SpecimenTiltDirection', 'i4'), 13196 ('IlluminationTiltDirection', 'i4'), 13197 ('IlluminationTiltAngle', 'i4'), 13198 ('ImageMode', 'i4'), 13199 ('EnergySpread', 'i4'), 13200 ('ChromaticAberration', 'i4'), 13201 ('ShutterType', 'i4'), 13202 ('DefocusSpread', 'i4'), 13203 ('CcdNumber', 'i4'), 13204 ('CcdSize', 'i4'), 13205 ('OffsetXV1', 'i4'), 13206 ('OffsetYV1', 'i4'), 13207 ('PhysicalPixelSize', 'i4'), 13208 ('Binning', 'i4'), 13209 ('ReadoutSpeed', 'i4'), 13210 ('GainV1', 'i4'), 13211 ('SensitivityV1', 'i4'), 13212 ('ExposureTimeV1', 'i4'), 13213 ('FlatCorrected', 'i4'), 13214 ('DeadPxCorrected', 'i4'), 13215 ('ImageMean', 'i4'), 13216 ('ImageStd', 'i4'), 13217 ('DisplacementX', 'i4'), 13218 ('DisplacementY', 'i4'), 13219 ('DateV1', 'i4'), 13220 ('TimeV1', 'i4'), 13221 ('ImageMin', 'i4'), 13222 ('ImageMax', 'i4'), 13223 ('ImageStatisticsQuality', 'i4'), 13224 ] 13225 13226 def TVIPS_HEADER_V2(): 13227 return [ 13228 ('ImageName', 'V160'), # utf16 13229 ('ImageFolder', 'V160'), 13230 ('ImageSizeX', 'i4'), 13231 ('ImageSizeY', 'i4'), 13232 ('ImageSizeZ', 'i4'), 13233 ('ImageSizeE', 'i4'), 13234 ('ImageDataType', 'i4'), 13235 ('Date', 'i4'), 13236 ('Time', 'i4'), 13237 ('Comment', 'V1024'), 13238 ('ImageHistory', 'V1024'), 13239 ('Scaling', '16f4'), 13240 ('ImageStatistics', '16c16'), 13241 ('ImageType', 'i4'), 13242 ('ImageDisplaType', 'i4'), 13243 ('PixelSizeX', 'f4'), # distance between two px in x, [nm] 13244 ('PixelSizeY', 'f4'), # distance between two px in y, [nm] 13245 ('ImageDistanceZ', 'f4'), 13246 ('ImageDistanceE', 'f4'), 13247 ('ImageMisc', '32f4'), 13248 ('TemType', 'V160'), 13249 ('TemHighTension', 'f4'), 13250 ('TemAberrations', '32f4'), 13251 ('TemEnergy', '32f4'), 13252 ('TemMode', 'i4'), 13253 ('TemMagnification', 'f4'), 13254 ('TemMagnificationCorrection', 'f4'), 13255 ('PostMagnification', 'f4'), 13256 ('TemStageType', 'i4'), 13257 ('TemStagePosition', '5f4'), # x, y, z, a, b 13258 ('TemImageShift', '2f4'), 13259 ('TemBeamShift', '2f4'), 13260 ('TemBeamTilt', '2f4'), 13261 ('TilingParameters', '7f4'), # 0: tiling? 1:x 2:y 3: max x 13262 # 4: max y 5: overlap x 6: overlap y 13263 ('TemIllumination', '3f4'), # 0: spotsize 1: intensity 13264 ('TemShutter', 'i4'), 13265 ('TemMisc', '32f4'), 13266 ('CameraType', 'V160'), 13267 ('PhysicalPixelSizeX', 'f4'), 13268 ('PhysicalPixelSizeY', 'f4'), 13269 ('OffsetX', 'i4'), 13270 ('OffsetY', 'i4'), 13271 ('BinningX', 'i4'), 13272 ('BinningY', 'i4'), 13273 ('ExposureTime', 'f4'), 13274 ('Gain', 'f4'), 13275 ('ReadoutRate', 'f4'), 13276 ('FlatfieldDescription', 'V160'), 13277 ('Sensitivity', 'f4'), 13278 ('Dose', 'f4'), 13279 ('CamMisc', '32f4'), 13280 ('FeiMicroscopeInformation', 'V1024'), 13281 ('FeiSpecimenInformation', 'V1024'), 13282 ('Magic', 'u4'), 13283 ] 13284 13285 def MM_HEADER(): 13286 # Olympus FluoView MM_Header 13287 MM_DIMENSION = [ 13288 ('Name', 'a16'), 13289 ('Size', 'i4'), 13290 ('Origin', 'f8'), 13291 ('Resolution', 'f8'), 13292 ('Unit', 'a64'), 13293 ] 13294 return [ 13295 ('HeaderFlag', 'i2'), 13296 ('ImageType', 'u1'), 13297 ('ImageName', 'a257'), 13298 ('OffsetData', 'u4'), 13299 ('PaletteSize', 'i4'), 13300 ('OffsetPalette0', 'u4'), 13301 ('OffsetPalette1', 'u4'), 13302 ('CommentSize', 'i4'), 13303 ('OffsetComment', 'u4'), 13304 ('Dimensions', MM_DIMENSION, 10), 13305 ('OffsetPosition', 'u4'), 13306 ('MapType', 'i2'), 13307 ('MapMin', 'f8'), 13308 ('MapMax', 'f8'), 13309 ('MinValue', 'f8'), 13310 ('MaxValue', 'f8'), 13311 ('OffsetMap', 'u4'), 13312 ('Gamma', 'f8'), 13313 ('Offset', 'f8'), 13314 ('GrayChannel', MM_DIMENSION), 13315 ('OffsetThumbnail', 'u4'), 13316 ('VoiceField', 'i4'), 13317 ('OffsetVoiceField', 'u4'), 13318 ] 13319 13320 def MM_DIMENSIONS(): 13321 # map FluoView MM_Header.Dimensions to axes characters 13322 return { 13323 'X': 'X', 13324 'Y': 'Y', 13325 'Z': 'Z', 13326 'T': 'T', 13327 'CH': 'C', 13328 'WAVELENGTH': 'C', 13329 'TIME': 'T', 13330 'XY': 'R', 13331 'EVENT': 'V', 13332 'EXPOSURE': 'L', 13333 } 13334 13335 def UIC_TAGS(): 13336 # map Universal Imaging Corporation MetaMorph internal tag ids to 13337 # name and type 13338 from fractions import Fraction # delayed import 13339 13340 return [ 13341 ('AutoScale', int), 13342 ('MinScale', int), 13343 ('MaxScale', int), 13344 ('SpatialCalibration', int), 13345 ('XCalibration', Fraction), 13346 ('YCalibration', Fraction), 13347 ('CalibrationUnits', str), 13348 ('Name', str), 13349 ('ThreshState', int), 13350 ('ThreshStateRed', int), 13351 ('tagid_10', None), # undefined 13352 ('ThreshStateGreen', int), 13353 ('ThreshStateBlue', int), 13354 ('ThreshStateLo', int), 13355 ('ThreshStateHi', int), 13356 ('Zoom', int), 13357 ('CreateTime', julian_datetime), 13358 ('LastSavedTime', julian_datetime), 13359 ('currentBuffer', int), 13360 ('grayFit', None), 13361 ('grayPointCount', None), 13362 ('grayX', Fraction), 13363 ('grayY', Fraction), 13364 ('grayMin', Fraction), 13365 ('grayMax', Fraction), 13366 ('grayUnitName', str), 13367 ('StandardLUT', int), 13368 ('wavelength', int), 13369 ('StagePosition', '(%i,2,2)u4'), # N xy positions as fract 13370 ('CameraChipOffset', '(%i,2,2)u4'), # N xy offsets as fract 13371 ('OverlayMask', None), 13372 ('OverlayCompress', None), 13373 ('Overlay', None), 13374 ('SpecialOverlayMask', None), 13375 ('SpecialOverlayCompress', None), 13376 ('SpecialOverlay', None), 13377 ('ImageProperty', read_uic_image_property), 13378 ('StageLabel', '%ip'), # N str 13379 ('AutoScaleLoInfo', Fraction), 13380 ('AutoScaleHiInfo', Fraction), 13381 ('AbsoluteZ', '(%i,2)u4'), # N fractions 13382 ('AbsoluteZValid', '(%i,)u4'), # N long 13383 ('Gamma', 'I'), # 'I' uses offset 13384 ('GammaRed', 'I'), 13385 ('GammaGreen', 'I'), 13386 ('GammaBlue', 'I'), 13387 ('CameraBin', '2I'), 13388 ('NewLUT', int), 13389 ('ImagePropertyEx', None), 13390 ('PlaneProperty', int), 13391 ('UserLutTable', '(256,3)u1'), 13392 ('RedAutoScaleInfo', int), 13393 ('RedAutoScaleLoInfo', Fraction), 13394 ('RedAutoScaleHiInfo', Fraction), 13395 ('RedMinScaleInfo', int), 13396 ('RedMaxScaleInfo', int), 13397 ('GreenAutoScaleInfo', int), 13398 ('GreenAutoScaleLoInfo', Fraction), 13399 ('GreenAutoScaleHiInfo', Fraction), 13400 ('GreenMinScaleInfo', int), 13401 ('GreenMaxScaleInfo', int), 13402 ('BlueAutoScaleInfo', int), 13403 ('BlueAutoScaleLoInfo', Fraction), 13404 ('BlueAutoScaleHiInfo', Fraction), 13405 ('BlueMinScaleInfo', int), 13406 ('BlueMaxScaleInfo', int), 13407 # ('OverlayPlaneColor', read_uic_overlay_plane_color), 13408 ] 13409 13410 def PILATUS_HEADER(): 13411 # PILATUS CBF Header Specification, Version 1.4 13412 # map key to [value_indices], type 13413 return { 13414 'Detector': ([slice(1, None)], str), 13415 'Pixel_size': ([1, 4], float), 13416 'Silicon': ([3], float), 13417 'Exposure_time': ([1], float), 13418 'Exposure_period': ([1], float), 13419 'Tau': ([1], float), 13420 'Count_cutoff': ([1], int), 13421 'Threshold_setting': ([1], float), 13422 'Gain_setting': ([1, 2], str), 13423 'N_excluded_pixels': ([1], int), 13424 'Excluded_pixels': ([1], str), 13425 'Flat_field': ([1], str), 13426 'Trim_file': ([1], str), 13427 'Image_path': ([1], str), 13428 # optional 13429 'Wavelength': ([1], float), 13430 'Energy_range': ([1, 2], float), 13431 'Detector_distance': ([1], float), 13432 'Detector_Voffset': ([1], float), 13433 'Beam_xy': ([1, 2], float), 13434 'Flux': ([1], str), 13435 'Filter_transmission': ([1], float), 13436 'Start_angle': ([1], float), 13437 'Angle_increment': ([1], float), 13438 'Detector_2theta': ([1], float), 13439 'Polarization': ([1], float), 13440 'Alpha': ([1], float), 13441 'Kappa': ([1], float), 13442 'Phi': ([1], float), 13443 'Phi_increment': ([1], float), 13444 'Chi': ([1], float), 13445 'Chi_increment': ([1], float), 13446 'Oscillation_axis': ([slice(1, None)], str), 13447 'N_oscillations': ([1], int), 13448 'Start_position': ([1], float), 13449 'Position_increment': ([1], float), 13450 'Shutter_time': ([1], float), 13451 'Omega': ([1], float), 13452 'Omega_increment': ([1], float), 13453 } 13454 13455 def ALLOCATIONGRANULARITY(): 13456 # alignment for writing contiguous data to TIFF 13457 import mmap # delayed import 13458 13459 return mmap.ALLOCATIONGRANULARITY 13460 13461 def MAXWORKERS(): 13462 # half of CPU cores 13463 import multiprocessing # delayed import 13464 13465 return max(multiprocessing.cpu_count() // 2, 1) 13466 13467 def CHUNKMODE(): 13468 class CHUNKMODE(enum.IntEnum): 13469 NONE = 0 13470 PLANE = 1 13471 PAGE = 2 13472 FILE = 3 13473 13474 return CHUNKMODE 13475 13476 13477def read_tags( 13478 fh, byteorder, offsetsize, tagnames, customtags=None, maxifds=None 13479): 13480 """Read tags from chain of IFDs and return as list of dicts. 13481 13482 The file handle position must be at a valid IFD header. 13483 Does not work with NDPI. 13484 13485 """ 13486 if offsetsize == 4: 13487 offsetformat = byteorder + 'I' 13488 tagnosize = 2 13489 tagnoformat = byteorder + 'H' 13490 tagsize = 12 13491 tagformat1 = byteorder + 'HH' 13492 tagformat2 = byteorder + 'I4s' 13493 elif offsetsize == 8: 13494 offsetformat = byteorder + 'Q' 13495 tagnosize = 8 13496 tagnoformat = byteorder + 'Q' 13497 tagsize = 20 13498 tagformat1 = byteorder + 'HH' 13499 tagformat2 = byteorder + 'Q8s' 13500 else: 13501 raise ValueError('invalid offset size') 13502 13503 if customtags is None: 13504 customtags = {} 13505 if maxifds is None: 13506 maxifds = 2 ** 32 13507 13508 result = [] 13509 unpack = struct.unpack 13510 offset = fh.tell() 13511 while len(result) < maxifds: 13512 # loop over IFDs 13513 try: 13514 tagno = unpack(tagnoformat, fh.read(tagnosize))[0] 13515 if tagno > 4096: 13516 raise TiffFileError('suspicious number of tags') 13517 except Exception: 13518 log_warning(f'read_tags: corrupted tag list at offset {offset}') 13519 break 13520 13521 tags = {} 13522 data = fh.read(tagsize * tagno) 13523 pos = fh.tell() 13524 index = 0 13525 for _ in range(tagno): 13526 code, dtype = unpack(tagformat1, data[index : index + 4]) 13527 count, value = unpack( 13528 tagformat2, data[index + 4 : index + tagsize] 13529 ) 13530 index += tagsize 13531 name = tagnames.get(code, str(code)) 13532 try: 13533 dataformat = TIFF.DATA_FORMATS[dtype] 13534 except KeyError: 13535 raise TiffFileError(f'unknown tag data type {dtype}') 13536 13537 fmt = '{}{}{}'.format( 13538 byteorder, count * int(dataformat[0]), dataformat[1] 13539 ) 13540 size = struct.calcsize(fmt) 13541 if size > offsetsize or code in customtags: 13542 offset = unpack(offsetformat, value)[0] 13543 if offset < 8 or offset > fh.size - size: 13544 raise TiffFileError(f'invalid tag value offset {offset}') 13545 fh.seek(offset) 13546 if code in customtags: 13547 readfunc = customtags[code][1] 13548 value = readfunc(fh, byteorder, dtype, count, offsetsize) 13549 elif dtype == 7 or (count > 1 and dtype == 1): 13550 value = read_bytes(fh, byteorder, dtype, count, offsetsize) 13551 elif code in tagnames or dtype == 2: 13552 value = unpack(fmt, fh.read(size)) 13553 else: 13554 value = read_numpy(fh, byteorder, dtype, count, offsetsize) 13555 elif dtype in (1, 7): 13556 # BYTES, UNDEFINED 13557 value = value[:size] 13558 else: 13559 value = unpack(fmt, value[:size]) 13560 13561 if code not in customtags and code not in TIFF.TAG_TUPLE: 13562 if len(value) == 1: 13563 value = value[0] 13564 if dtype == 2 and isinstance(value, bytes): 13565 # TIFF ASCII fields can contain multiple strings, 13566 # each terminated with a NUL 13567 try: 13568 value = bytes2str(stripnull(value, first=False).strip()) 13569 except UnicodeDecodeError: 13570 log_warning( 13571 'read_tags: coercing invalid ASCII to bytes ' 13572 f'(tag {code})' 13573 ) 13574 tags[name] = value 13575 13576 result.append(tags) 13577 # read offset to next page 13578 fh.seek(pos) 13579 offset = unpack(offsetformat, fh.read(offsetsize))[0] 13580 if offset == 0: 13581 break 13582 if offset >= fh.size: 13583 log_warning(f'read_tags: invalid page offset ({offset})') 13584 break 13585 fh.seek(offset) 13586 13587 if result and maxifds == 1: 13588 result = result[0] 13589 return result 13590 13591 13592def read_exif_ifd(fh, byteorder, dtype, count, offsetsize): 13593 """Read EXIF tags from file and return as dict.""" 13594 exif = read_tags(fh, byteorder, offsetsize, TIFF.EXIF_TAGS, maxifds=1) 13595 for name in ('ExifVersion', 'FlashpixVersion'): 13596 try: 13597 exif[name] = bytes2str(exif[name]) 13598 except Exception: 13599 pass 13600 if 'UserComment' in exif: 13601 idcode = exif['UserComment'][:8] 13602 try: 13603 if idcode == b'ASCII\x00\x00\x00': 13604 exif['UserComment'] = bytes2str(exif['UserComment'][8:]) 13605 elif idcode == b'UNICODE\x00': 13606 exif['UserComment'] = exif['UserComment'][8:].decode('utf-16') 13607 except Exception: 13608 pass 13609 return exif 13610 13611 13612def read_gps_ifd(fh, byteorder, dtype, count, offsetsize): 13613 """Read GPS tags from file and return as dict.""" 13614 return read_tags(fh, byteorder, offsetsize, TIFF.GPS_TAGS, maxifds=1) 13615 13616 13617def read_interoperability_ifd(fh, byteorder, dtype, count, offsetsize): 13618 """Read Interoperability tags from file and return as dict.""" 13619 return read_tags(fh, byteorder, offsetsize, TIFF.IOP_TAGS, maxifds=1) 13620 13621 13622def read_bytes(fh, byteorder, dtype, count, offsetsize): 13623 """Read tag data from file and return as bytes.""" 13624 dtype = 'B' if dtype == 2 else byteorder + TIFF.DATA_FORMATS[dtype][-1] 13625 count *= numpy.dtype(dtype).itemsize 13626 data = fh.read(count) 13627 if len(data) != count: 13628 log_warning( 13629 f'read_bytes: failed to read all bytes ({len(data)} < {count})' 13630 ) 13631 return data 13632 13633 13634def read_utf8(fh, byteorder, dtype, count, offsetsize): 13635 """Read tag data from file and return as Unicode string.""" 13636 return fh.read(count).decode() 13637 13638 13639def read_numpy(fh, byteorder, dtype, count, offsetsize): 13640 """Read tag data from file and return as numpy array.""" 13641 dtype = 'b' if dtype == 2 else byteorder + TIFF.DATA_FORMATS[dtype][-1] 13642 return fh.read_array(dtype, count) 13643 13644 13645def read_colormap(fh, byteorder, dtype, count, offsetsize): 13646 """Read ColorMap data from file and return as numpy array.""" 13647 cmap = fh.read_array(byteorder + TIFF.DATA_FORMATS[dtype][-1], count) 13648 cmap.shape = (3, -1) 13649 return cmap 13650 13651 13652def read_json(fh, byteorder, dtype, count, offsetsize): 13653 """Read JSON tag data from file and return as object.""" 13654 data = fh.read(count) 13655 try: 13656 return json.loads(stripnull(data).decode()) 13657 except ValueError: 13658 log_warning('read_json: invalid JSON') 13659 return None 13660 13661 13662def read_mm_header(fh, byteorder, dtype, count, offsetsize): 13663 """Read FluoView mm_header tag from file and return as dict.""" 13664 mmh = fh.read_record(TIFF.MM_HEADER, byteorder=byteorder) 13665 mmh = recarray2dict(mmh) 13666 mmh['Dimensions'] = [ 13667 (bytes2str(d[0]).strip(), d[1], d[2], d[3], bytes2str(d[4]).strip()) 13668 for d in mmh['Dimensions'] 13669 ] 13670 d = mmh['GrayChannel'] 13671 mmh['GrayChannel'] = ( 13672 bytes2str(d[0]).strip(), 13673 d[1], 13674 d[2], 13675 d[3], 13676 bytes2str(d[4]).strip(), 13677 ) 13678 return mmh 13679 13680 13681def read_mm_stamp(fh, byteorder, dtype, count, offsetsize): 13682 """Read FluoView mm_stamp tag from file and return as numpy.ndarray.""" 13683 return fh.read_array(byteorder + 'f8', 8) 13684 13685 13686def read_uic1tag(fh, byteorder, dtype, count, offsetsize, planecount=None): 13687 """Read MetaMorph STK UIC1Tag from file and return as dict. 13688 13689 Return empty dictionary if planecount is unknown. 13690 13691 """ 13692 if dtype not in (4, 5) or byteorder != '<': 13693 raise ValueError(f'invalid UIC1Tag {byteorder}{dtype}') 13694 result = {} 13695 if dtype == 5: 13696 # pre MetaMorph 2.5 (not tested) 13697 values = fh.read_array('<u4', 2 * count).reshape(count, 2) 13698 result = {'ZDistance': values[:, 0] / values[:, 1]} 13699 elif planecount: 13700 for _ in range(count): 13701 tagid = struct.unpack('<I', fh.read(4))[0] 13702 if tagid in (28, 29, 37, 40, 41): 13703 # silently skip unexpected tags 13704 fh.read(4) 13705 continue 13706 name, value = read_uic_tag(fh, tagid, planecount, offset=True) 13707 result[name] = value 13708 return result 13709 13710 13711def read_uic2tag(fh, byteorder, dtype, planecount, offsetsize): 13712 """Read MetaMorph STK UIC2Tag from file and return as dict.""" 13713 if dtype != 5 or byteorder != '<': 13714 raise ValueError('invalid UIC2Tag') 13715 values = fh.read_array('<u4', 6 * planecount).reshape(planecount, 6) 13716 return { 13717 'ZDistance': values[:, 0] / values[:, 1], 13718 'DateCreated': values[:, 2], # julian days 13719 'TimeCreated': values[:, 3], # milliseconds 13720 'DateModified': values[:, 4], # julian days 13721 'TimeModified': values[:, 5], # milliseconds 13722 } 13723 13724 13725def read_uic3tag(fh, byteorder, dtype, planecount, offsetsize): 13726 """Read MetaMorph STK UIC3Tag from file and return as dict.""" 13727 if dtype != 5 or byteorder != '<': 13728 raise ValueError('invalid UIC3Tag') 13729 values = fh.read_array('<u4', 2 * planecount).reshape(planecount, 2) 13730 return {'Wavelengths': values[:, 0] / values[:, 1]} 13731 13732 13733def read_uic4tag(fh, byteorder, dtype, planecount, offsetsize): 13734 """Read MetaMorph STK UIC4Tag from file and return as dict.""" 13735 if dtype != 4 or byteorder != '<': 13736 raise ValueError('invalid UIC4Tag') 13737 result = {} 13738 while True: 13739 tagid = struct.unpack('<H', fh.read(2))[0] 13740 if tagid == 0: 13741 break 13742 name, value = read_uic_tag(fh, tagid, planecount, offset=False) 13743 result[name] = value 13744 return result 13745 13746 13747def read_uic_tag(fh, tagid, planecount, offset): 13748 """Read a single UIC tag value from file and return tag name and value. 13749 13750 UIC1Tags use an offset. 13751 13752 """ 13753 13754 def read_int(count=1): 13755 value = struct.unpack(f'<{count}I', fh.read(4 * count)) 13756 return value[0] if count == 1 else value 13757 13758 try: 13759 name, dtype = TIFF.UIC_TAGS[tagid] 13760 except IndexError: 13761 # unknown tag 13762 return f'_TagId{tagid}', read_int() 13763 13764 Fraction = TIFF.UIC_TAGS[4][1] 13765 13766 if offset: 13767 pos = fh.tell() 13768 if dtype not in (int, None): 13769 off = read_int() 13770 if off < 8: 13771 if dtype is str: 13772 return name, '' 13773 log_warning( 13774 f'read_uic_tag: invalid offset for tag {name!r} @{off}' 13775 ) 13776 return name, off 13777 fh.seek(off) 13778 13779 if dtype is None: 13780 # skip 13781 name = '_' + name 13782 value = read_int() 13783 elif dtype is int: 13784 # int 13785 value = read_int() 13786 elif dtype is Fraction: 13787 # fraction 13788 value = read_int(2) 13789 value = value[0] / value[1] 13790 elif dtype is julian_datetime: 13791 # datetime 13792 value = julian_datetime(*read_int(2)) 13793 elif dtype is read_uic_image_property: 13794 # ImagePropertyEx 13795 value = read_uic_image_property(fh) 13796 elif dtype is str: 13797 # pascal string 13798 size = read_int() 13799 if 0 <= size < 2 ** 10: 13800 value = struct.unpack(f'{size}s', fh.read(size))[0][:-1] 13801 value = bytes2str(stripnull(value)) 13802 elif offset: 13803 value = '' 13804 log_warning(f'read_uic_tag: corrupt string in tag {name!r}') 13805 else: 13806 raise ValueError(f'read_uic_tag: invalid string size {size}') 13807 elif dtype == '%ip': 13808 # sequence of pascal strings 13809 value = [] 13810 for _ in range(planecount): 13811 size = read_int() 13812 if 0 <= size < 2 ** 10: 13813 string = struct.unpack(f'{size}s', fh.read(size))[0][:-1] 13814 string = bytes2str(stripnull(string)) 13815 value.append(string) 13816 elif offset: 13817 log_warning(f'read_uic_tag: corrupt string in tag {name!r}') 13818 else: 13819 raise ValueError(f'read_uic_tag: invalid string size: {size}') 13820 else: 13821 # struct or numpy type 13822 dtype = '<' + dtype 13823 if '%i' in dtype: 13824 dtype = dtype % planecount 13825 if '(' in dtype: 13826 # numpy type 13827 value = fh.read_array(dtype, 1)[0] 13828 if value.shape[-1] == 2: 13829 # assume fractions 13830 value = value[..., 0] / value[..., 1] 13831 else: 13832 # struct format 13833 value = struct.unpack(dtype, fh.read(struct.calcsize(dtype))) 13834 if len(value) == 1: 13835 value = value[0] 13836 13837 if offset: 13838 fh.seek(pos + 4) 13839 13840 return name, value 13841 13842 13843def read_uic_image_property(fh): 13844 """Read UIC ImagePropertyEx tag from file and return as dict.""" 13845 # TODO: test this 13846 size = struct.unpack('B', fh.read(1))[0] 13847 name = struct.unpack(f'{size}s', fh.read(size))[0][:-1] 13848 flags, prop = struct.unpack('<IB', fh.read(5)) 13849 if prop == 1: 13850 value = struct.unpack('II', fh.read(8)) 13851 value = value[0] / value[1] 13852 else: 13853 size = struct.unpack('B', fh.read(1))[0] 13854 value = struct.unpack(f'{size}s', fh.read(size))[0] 13855 return dict(name=name, flags=flags, value=value) 13856 13857 13858def read_cz_lsminfo(fh, byteorder, dtype, count, offsetsize): 13859 """Read CZ_LSMINFO tag from file and return as dict.""" 13860 if byteorder != '<': 13861 raise ValueError('invalid CZ_LSMINFO structure') 13862 magic_number, structure_size = struct.unpack('<II', fh.read(8)) 13863 if magic_number not in (50350412, 67127628): 13864 raise ValueError('invalid CZ_LSMINFO structure') 13865 fh.seek(-8, os.SEEK_CUR) 13866 13867 if structure_size < numpy.dtype(TIFF.CZ_LSMINFO).itemsize: 13868 # adjust structure according to structure_size 13869 lsminfo = [] 13870 size = 0 13871 for name, dtype in TIFF.CZ_LSMINFO: 13872 size += numpy.dtype(dtype).itemsize 13873 if size > structure_size: 13874 break 13875 lsminfo.append((name, dtype)) 13876 else: 13877 lsminfo = TIFF.CZ_LSMINFO 13878 13879 lsminfo = fh.read_record(lsminfo, byteorder=byteorder) 13880 lsminfo = recarray2dict(lsminfo) 13881 13882 # read LSM info subrecords at offsets 13883 for name, reader in TIFF.CZ_LSMINFO_READERS.items(): 13884 if reader is None: 13885 continue 13886 offset = lsminfo.get('Offset' + name, 0) 13887 if offset < 8: 13888 continue 13889 fh.seek(offset) 13890 try: 13891 lsminfo[name] = reader(fh) 13892 except ValueError: 13893 pass 13894 return lsminfo 13895 13896 13897def read_lsm_channeldatatypes(fh): 13898 """Read LSM channel data type.""" 13899 size = struct.unpack('<I', fh.read(4))[0] 13900 return fh.read_array('<u4', count=size) 13901 13902 13903def read_lsm_channelwavelength(fh): 13904 """Read LSM channel wavelength ranges from file and return as list.""" 13905 size = struct.unpack('<i', fh.read(4))[0] 13906 return fh.read_array('<2f8', count=size) 13907 13908 13909def read_lsm_positions(fh): 13910 """Read LSM positions from file and return as list.""" 13911 size = struct.unpack('<I', fh.read(4))[0] 13912 return fh.read_array('<3f8', count=size) 13913 13914 13915def read_lsm_timestamps(fh): 13916 """Read LSM time stamps from file and return as list.""" 13917 size, count = struct.unpack('<ii', fh.read(8)) 13918 if size != (8 + 8 * count): 13919 log_warning('read_lsm_timestamps: invalid LSM TimeStamps block') 13920 return [] 13921 # return struct.unpack(f'<{count}d', fh.read(8 * count)) 13922 return fh.read_array('<f8', count=count) 13923 13924 13925def read_lsm_eventlist(fh): 13926 """Read LSM events from file and return as list of (time, type, text).""" 13927 count = struct.unpack('<II', fh.read(8))[1] 13928 events = [] 13929 while count > 0: 13930 esize, etime, etype = struct.unpack('<IdI', fh.read(16)) 13931 etext = bytes2str(stripnull(fh.read(esize - 16))) 13932 events.append((etime, etype, etext)) 13933 count -= 1 13934 return events 13935 13936 13937def read_lsm_channelcolors(fh): 13938 """Read LSM ChannelColors structure from file and return as dict.""" 13939 result = {'Mono': False, 'Colors': [], 'ColorNames': []} 13940 pos = fh.tell() 13941 (size, ncolors, nnames, coffset, noffset, mono) = struct.unpack( 13942 '<IIIIII', fh.read(24) 13943 ) 13944 if ncolors != nnames: 13945 log_warning( 13946 'read_lsm_channelcolors: invalid LSM ChannelColors structure' 13947 ) 13948 return result 13949 result['Mono'] = bool(mono) 13950 # Colors 13951 fh.seek(pos + coffset) 13952 colors = fh.read_array('uint8', count=ncolors * 4).reshape((ncolors, 4)) 13953 result['Colors'] = colors.tolist() 13954 # ColorNames 13955 fh.seek(pos + noffset) 13956 buffer = fh.read(size - noffset) 13957 names = [] 13958 while len(buffer) > 4: 13959 size = struct.unpack('<I', buffer[:4])[0] 13960 names.append(bytes2str(buffer[4 : 3 + size])) 13961 buffer = buffer[4 + size :] 13962 result['ColorNames'] = names 13963 return result 13964 13965 13966def read_lsm_lookuptable(fh): 13967 """Read LSM lookup tables from file and return as dict.""" 13968 result = {} 13969 ( 13970 size, 13971 nsubblocks, 13972 nchannels, 13973 luttype, 13974 advanced, 13975 currentchannel, 13976 ) = struct.unpack('<iiiiii', fh.read(24)) 13977 if size < 60: 13978 log_warning('read_lsm_lookuptable: invalid LSM LookupTables structure') 13979 return result 13980 fh.read(9 * 4) # reserved 13981 result['LutType'] = TIFF.CZ_LSM_LUTTYPE(luttype) 13982 result['Advanced'] = advanced 13983 result['NumberChannels'] = nchannels 13984 result['CurrentChannel'] = currentchannel 13985 result['SubBlocks'] = subblocks = [] 13986 for i in range(nsubblocks): 13987 sbtype = struct.unpack('<i', fh.read(4))[0] 13988 if sbtype <= 0: 13989 break 13990 size = struct.unpack('<i', fh.read(4))[0] - 8 13991 if sbtype == 1: 13992 data = fh.read_array('<f8', count=nchannels) 13993 elif sbtype == 2: 13994 data = fh.read_array('<f8', count=nchannels) 13995 elif sbtype == 3: 13996 data = fh.read_array('<f8', count=nchannels) 13997 elif sbtype == 4: 13998 # the data type is wrongly documented as f8 13999 data = fh.read_array('<i4', count=nchannels * 4) 14000 data = data.reshape((-1, 2, 2)) 14001 elif sbtype == 5: 14002 # the data type is wrongly documented as f8 14003 nknots = struct.unpack('<i', fh.read(4))[0] # undocumented 14004 data = fh.read_array('<i4', count=nchannels * nknots * 2) 14005 data = data.reshape((nchannels, nknots, 2)) 14006 elif sbtype == 6: 14007 data = fh.read_array('<i2', count=nchannels * 4096) 14008 data = data.reshape((-1, 4096)) 14009 else: 14010 log_warning( 14011 f'read_lsm_lookuptable: invalid LSM SubBlock type {sbtype}' 14012 ) 14013 break 14014 subblocks.append( 14015 {'Type': TIFF.CZ_LSM_SUBBLOCK_TYPE(sbtype), 'Data': data} 14016 ) 14017 return result 14018 14019 14020def read_lsm_scaninfo(fh): 14021 """Read LSM ScanInfo structure from file and return as dict.""" 14022 block = {} 14023 blocks = [block] 14024 unpack = struct.unpack 14025 if struct.unpack('<I', fh.read(4))[0] != 0x10000000: 14026 # not a Recording sub block 14027 log_warning('read_lsm_scaninfo: invalid LSM ScanInfo structure') 14028 return block 14029 fh.read(8) 14030 while True: 14031 entry, dtype, size = unpack('<III', fh.read(12)) 14032 if dtype == 2: 14033 # ascii 14034 value = bytes2str(stripnull(fh.read(size))) 14035 elif dtype == 4: 14036 # long 14037 value = unpack('<i', fh.read(4))[0] 14038 elif dtype == 5: 14039 # rational 14040 value = unpack('<d', fh.read(8))[0] 14041 else: 14042 value = 0 14043 if entry in TIFF.CZ_LSMINFO_SCANINFO_ARRAYS: 14044 blocks.append(block) 14045 name = TIFF.CZ_LSMINFO_SCANINFO_ARRAYS[entry] 14046 newobj = [] 14047 block[name] = newobj 14048 block = newobj 14049 elif entry in TIFF.CZ_LSMINFO_SCANINFO_STRUCTS: 14050 blocks.append(block) 14051 newobj = {} 14052 block.append(newobj) 14053 block = newobj 14054 elif entry in TIFF.CZ_LSMINFO_SCANINFO_ATTRIBUTES: 14055 name = TIFF.CZ_LSMINFO_SCANINFO_ATTRIBUTES[entry] 14056 block[name] = value 14057 elif entry == 0xFFFFFFFF: 14058 # end sub block 14059 block = blocks.pop() 14060 else: 14061 # unknown entry 14062 block[f'Entry0x{entry:x}'] = value 14063 if not blocks: 14064 break 14065 return block 14066 14067 14068def read_sis(fh, byteorder, dtype, count, offsetsize): 14069 """Read OlympusSIS structure and return as dict. 14070 14071 No specification is avaliable. Only few fields are known. 14072 14073 """ 14074 result = {} 14075 14076 (magic, minute, hour, day, month, year, name, tagcount) = struct.unpack( 14077 '<4s6xhhhhh6x32sh', fh.read(60) 14078 ) 14079 14080 if magic != b'SIS0': 14081 raise ValueError('invalid OlympusSIS structure') 14082 14083 result['name'] = bytes2str(stripnull(name)) 14084 try: 14085 result['datetime'] = datetime.datetime( 14086 1900 + year, month + 1, day, hour, minute 14087 ) 14088 except ValueError: 14089 pass 14090 14091 data = fh.read(8 * tagcount) 14092 for i in range(0, tagcount * 8, 8): 14093 tagtype, count, offset = struct.unpack('<hhI', data[i : i + 8]) 14094 fh.seek(offset) 14095 if tagtype == 1: 14096 # general data 14097 (lenexp, xcal, ycal, mag, camname, pictype) = struct.unpack( 14098 '<10xhdd8xd2x34s32s', fh.read(112) # 220 14099 ) 14100 m = math.pow(10, lenexp) 14101 result['pixelsizex'] = xcal * m 14102 result['pixelsizey'] = ycal * m 14103 result['magnification'] = mag 14104 result['cameraname'] = bytes2str(stripnull(camname)) 14105 result['picturetype'] = bytes2str(stripnull(pictype)) 14106 elif tagtype == 10: 14107 # channel data 14108 continue 14109 # TODO: does not seem to work? 14110 # (length, _, exptime, emv, _, camname, _, mictype, 14111 # ) = struct.unpack('<h22sId4s32s48s32s', fh.read(152)) # 720 14112 # result['exposuretime'] = exptime 14113 # result['emvoltage'] = emv 14114 # result['cameraname2'] = bytes2str(stripnull(camname)) 14115 # result['microscopename'] = bytes2str(stripnull(mictype)) 14116 14117 return result 14118 14119 14120def read_sis_ini(fh, byteorder, dtype, count, offsetsize): 14121 """Read OlympusSIS INI string and return as dict.""" 14122 inistr = fh.read(count) 14123 inistr = bytes2str(stripnull(inistr)) 14124 try: 14125 return olympusini_metadata(inistr) 14126 except Exception as exc: 14127 log_warning(f'olympusini_metadata: {exc.__class__.__name__}: {exc}') 14128 return {} 14129 14130 14131def read_tvips_header(fh, byteorder, dtype, count, offsetsize): 14132 """Read TVIPS EM-MENU headers and return as dict.""" 14133 result = {} 14134 header = fh.read_record(TIFF.TVIPS_HEADER_V1, byteorder=byteorder) 14135 for name, typestr in TIFF.TVIPS_HEADER_V1: 14136 result[name] = header[name].tolist() 14137 if header['Version'] == 2: 14138 header = fh.read_record(TIFF.TVIPS_HEADER_V2, byteorder=byteorder) 14139 if header['Magic'] != int(0xAAAAAAAA): 14140 log_warning('read_tvips_header: invalid TVIPS v2 magic number') 14141 return {} 14142 # decode utf16 strings 14143 for name, typestr in TIFF.TVIPS_HEADER_V2: 14144 if typestr.startswith('V'): 14145 s = header[name].tobytes().decode('utf-16', errors='ignore') 14146 result[name] = stripnull(s, null='\x00') 14147 else: 14148 result[name] = header[name].tolist() 14149 # convert nm to m 14150 for axis in 'XY': 14151 header['PhysicalPixelSize' + axis] /= 1e9 14152 header['PixelSize' + axis] /= 1e9 14153 elif header.version != 1: 14154 log_warning('read_tvips_header: unknown TVIPS header version') 14155 return {} 14156 return result 14157 14158 14159def read_fei_metadata(fh, byteorder, dtype, count, offsetsize): 14160 """Read FEI SFEG/HELIOS headers and return as dict.""" 14161 result = {} 14162 section = {} 14163 data = bytes2str(stripnull(fh.read(count))) 14164 for line in data.splitlines(): 14165 line = line.strip() 14166 if line.startswith('['): 14167 section = {} 14168 result[line[1:-1]] = section 14169 continue 14170 try: 14171 key, value = line.split('=') 14172 except ValueError: 14173 continue 14174 section[key] = astype(value) 14175 return result 14176 14177 14178def read_cz_sem(fh, byteorder, dtype, count, offsetsize): 14179 """Read Zeiss SEM tag and return as dict. 14180 14181 See https://sourceforge.net/p/gwyddion/mailman/message/29275000/ for 14182 unnamed values. 14183 14184 """ 14185 result = {'': ()} 14186 key = None 14187 data = bytes2str(stripnull(fh.read(count))) 14188 for line in data.splitlines(): 14189 if line.isupper(): 14190 key = line.lower() 14191 elif key: 14192 try: 14193 name, value = line.split('=') 14194 except ValueError: 14195 try: 14196 name, value = line.split(':', 1) 14197 except Exception: 14198 continue 14199 value = value.strip() 14200 unit = '' 14201 try: 14202 v, u = value.split() 14203 number = astype(v, (int, float)) 14204 if number != v: 14205 value = number 14206 unit = u 14207 except Exception: 14208 number = astype(value, (int, float)) 14209 if number != value: 14210 value = number 14211 if value in ('No', 'Off'): 14212 value = False 14213 elif value in ('Yes', 'On'): 14214 value = True 14215 result[key] = (name.strip(), value) 14216 if unit: 14217 result[key] += (unit,) 14218 key = None 14219 else: 14220 result[''] += (astype(line, (int, float)),) 14221 return result 14222 14223 14224def read_nih_image_header(fh, byteorder, dtype, count, offsetsize): 14225 """Read NIH_IMAGE_HEADER tag from file and return as dict.""" 14226 a = fh.read_record(TIFF.NIH_IMAGE_HEADER, byteorder=byteorder) 14227 a = a.newbyteorder(byteorder) 14228 a = recarray2dict(a) 14229 a['XUnit'] = a['XUnit'][: a['XUnitSize']] 14230 a['UM'] = a['UM'][: a['UMsize']] 14231 return a 14232 14233 14234def read_scanimage_metadata(fh): 14235 """Read ScanImage BigTIFF v3 or v4 static and ROI metadata from open file. 14236 14237 Return non-varying frame data, ROI group data, and version as 14238 tuple(dict, dict, int). 14239 14240 The settings can be used to read image data and metadata without parsing 14241 the TIFF file. 14242 14243 Raise ValueError if file does not contain valid ScanImage metadata. 14244 14245 Frame data and ROI groups can alternatively be obtained from the Software 14246 and Artist tags of any TIFF page. 14247 14248 """ 14249 fh.seek(0) 14250 try: 14251 byteorder, version = struct.unpack('<2sH', fh.read(4)) 14252 if byteorder != b'II' or version != 43: 14253 raise ValueError('not a BigTIFF file') 14254 fh.seek(16) 14255 magic, version, size0, size1 = struct.unpack('<IIII', fh.read(16)) 14256 if magic != 117637889 or version not in (3, 4): 14257 raise ValueError( 14258 f'invalid magic {magic} or version {version} number' 14259 ) 14260 except UnicodeDecodeError as exc: 14261 raise ValueError('file must be opened in binary mode') from exc 14262 except Exception as exc: 14263 raise ValueError('not a ScanImage BigTIFF v3 or v4 file') from exc 14264 14265 frame_data = matlabstr2py(bytes2str(fh.read(size0)[:-1])) 14266 roi_data = read_json(fh, '<', None, size1, None) if size1 > 1 else {} 14267 return frame_data, roi_data, version 14268 14269 14270def read_micromanager_metadata(fh): 14271 """Read MicroManager non-TIFF settings from open file and return as dict. 14272 14273 The settings can be used to read image data without parsing the TIFF file. 14274 14275 """ 14276 fh.seek(0) 14277 try: 14278 byteorder = {b'II': '<', b'MM': '>'}[fh.read(2)] 14279 except IndexError: 14280 raise ValueError('not a MicroManager TIFF file') 14281 14282 result = {} 14283 fh.seek(8) 14284 ( 14285 index_header, # Index map 14286 index_offset, 14287 display_header, 14288 display_offset, 14289 comments_header, 14290 comments_offset, 14291 summary_header, 14292 summary_length, 14293 ) = struct.unpack(byteorder + 'IIIIIIII', fh.read(32)) 14294 14295 if summary_header == 2355492: 14296 result['Summary'] = read_json( 14297 fh, byteorder, None, summary_length, None 14298 ) 14299 else: 14300 log_warning('invalid MicroManager summary header') 14301 14302 if index_header == 54773648: 14303 fh.seek(index_offset) 14304 header, count = struct.unpack(byteorder + 'II', fh.read(8)) 14305 if header == 3453623: 14306 data = struct.unpack( 14307 byteorder + 'IIIII' * count, fh.read(20 * count) 14308 ) 14309 result['IndexMap'] = { 14310 'Channel': data[::5], 14311 'Slice': data[1::5], 14312 'Frame': data[2::5], 14313 'Position': data[3::5], 14314 'Offset': data[4::5], 14315 } 14316 else: 14317 log_warning('invalid MicroManager index header') 14318 else: 14319 log_warning('invalid MicroManager index header') 14320 14321 if display_header == 483765892: 14322 fh.seek(display_offset) 14323 header, count = struct.unpack(byteorder + 'II', fh.read(8)) 14324 if header == 347834724: 14325 result['DisplaySettings'] = read_json( 14326 fh, byteorder, None, count, None 14327 ) 14328 else: 14329 log_warning('invalid MicroManager display header') 14330 else: 14331 log_warning('invalid MicroManager display header') 14332 14333 result['MajorVersion'] = 0 14334 if comments_header == 99384722: 14335 # Micro-Manager multipage TIFF 14336 fh.seek(comments_offset) 14337 header, count = struct.unpack(byteorder + 'II', fh.read(8)) 14338 if header == 84720485: 14339 result['Comments'] = read_json(fh, byteorder, None, count, None) 14340 else: 14341 log_warning('invalid MicroManager comments header') 14342 elif comments_header == 483729: 14343 # NDTiffStorage 14344 result['MajorVersion'] = comments_offset 14345 else: 14346 log_warning('invalid MicroManager comments header') 14347 14348 return result 14349 14350 14351def read_metaseries_catalog(fh): 14352 """Read MetaSeries non-TIFF hint catalog from file. 14353 14354 Raise ValueError if the file does not contain a valid hint catalog. 14355 14356 """ 14357 # TODO: implement read_metaseries_catalog 14358 raise NotImplementedError 14359 14360 14361def imagej_metadata_tag(metadata, byteorder): 14362 """Return IJMetadata and IJMetadataByteCounts tags from metadata dict. 14363 14364 The tags can be passed to TiffWriter.write() as extratags. 14365 14366 The metadata dict may contain the following keys and values: 14367 14368 Info : str 14369 Human-readable information as string. 14370 Labels : sequence of str 14371 Human-readable labels for each channel. 14372 Ranges : sequence of doubles 14373 Lower and upper values for each channel. 14374 LUTs : sequence of (3, 256) uint8 ndarrays 14375 Color palettes for each channel. 14376 Plot : bytes 14377 Undocumented ImageJ internal format. 14378 ROI: bytes 14379 Undocumented ImageJ internal region of interest format. 14380 Overlays : bytes 14381 Undocumented ImageJ internal format. 14382 14383 """ 14384 if not metadata: 14385 return () 14386 header = [{'>': b'IJIJ', '<': b'JIJI'}[byteorder]] 14387 bytecounts = [0] 14388 body = [] 14389 14390 def _string(data, byteorder): 14391 return data.encode('utf-16' + {'>': 'be', '<': 'le'}[byteorder]) 14392 14393 def _doubles(data, byteorder): 14394 return struct.pack(byteorder + ('d' * len(data)), *data) 14395 14396 def _ndarray(data, byteorder): 14397 return data.tobytes() 14398 14399 def _bytes(data, byteorder): 14400 return data 14401 14402 metadata_types = ( 14403 ('Info', b'info', _string), 14404 ('Labels', b'labl', _string), 14405 ('Ranges', b'rang', _doubles), 14406 ('LUTs', b'luts', _ndarray), 14407 ('Plot', b'plot', _bytes), 14408 ('ROI', b'roi ', _bytes), 14409 ('Overlays', b'over', _bytes), 14410 ) 14411 14412 for key, mtype, func in metadata_types: 14413 if key.lower() in metadata: 14414 key = key.lower() 14415 elif key not in metadata: 14416 continue 14417 if byteorder == '<': 14418 mtype = mtype[::-1] 14419 values = metadata[key] 14420 if isinstance(values, list): 14421 count = len(values) 14422 else: 14423 values = [values] 14424 count = 1 14425 header.append(mtype + struct.pack(byteorder + 'I', count)) 14426 for value in values: 14427 data = func(value, byteorder) 14428 body.append(data) 14429 bytecounts.append(len(data)) 14430 14431 if not body: 14432 return () 14433 body = b''.join(body) 14434 header = b''.join(header) 14435 data = header + body 14436 bytecounts[0] = len(header) 14437 bytecounts = struct.pack(byteorder + ('I' * len(bytecounts)), *bytecounts) 14438 return ( 14439 (50839, 1, len(data), data, True), 14440 (50838, 4, len(bytecounts) // 4, bytecounts, True), 14441 ) 14442 14443 14444def imagej_metadata(data, bytecounts, byteorder): 14445 """Return IJMetadata tag value as dict. 14446 14447 The 'Info' string can have multiple formats, e.g. OIF or ScanImage, 14448 that might be parsed into dicts using the matlabstr2py or 14449 oiffile.SettingsFile functions. 14450 'ROI' and 'Overlays' are returned as bytes, which can be parsed with the 14451 ImagejRoi.frombytes() function of the roifile package. 14452 14453 """ 14454 14455 def _string(data, byteorder): 14456 return data.decode('utf-16' + {'>': 'be', '<': 'le'}[byteorder]) 14457 14458 def _doubles(data, byteorder): 14459 return struct.unpack(byteorder + ('d' * (len(data) // 8)), data) 14460 14461 def _lut(data, byteorder): 14462 return numpy.frombuffer(data, 'uint8').reshape(-1, 256) 14463 14464 def _bytes(data, byteorder): 14465 return data 14466 14467 # big-endian 14468 metadata_types = { 14469 b'info': ('Info', _string), 14470 b'labl': ('Labels', _string), 14471 b'rang': ('Ranges', _doubles), 14472 b'luts': ('LUTs', _lut), 14473 b'plot': ('Plot', _bytes), 14474 b'roi ': ('ROI', _bytes), 14475 b'over': ('Overlays', _bytes), 14476 } 14477 # little-endian 14478 metadata_types.update({k[::-1]: v for k, v in metadata_types.items()}) 14479 14480 if not bytecounts: 14481 raise ValueError('no ImageJ metadata') 14482 14483 if not data[:4] in (b'IJIJ', b'JIJI'): 14484 raise ValueError('invalid ImageJ metadata') 14485 14486 header_size = bytecounts[0] 14487 if header_size < 12 or header_size > 804: 14488 raise ValueError('invalid ImageJ metadata header size') 14489 14490 ntypes = (header_size - 4) // 8 14491 header = struct.unpack( 14492 byteorder + '4sI' * ntypes, data[4 : 4 + ntypes * 8] 14493 ) 14494 pos = 4 + ntypes * 8 14495 counter = 0 14496 result = {} 14497 for mtype, count in zip(header[::2], header[1::2]): 14498 values = [] 14499 name, func = metadata_types.get(mtype, (bytes2str(mtype), read_bytes)) 14500 for _ in range(count): 14501 counter += 1 14502 pos1 = pos + bytecounts[counter] 14503 values.append(func(data[pos:pos1], byteorder)) 14504 pos = pos1 14505 result[name.strip()] = values[0] if count == 1 else values 14506 return result 14507 14508 14509def imagej_description_metadata(description): 14510 r"""Return metatata from ImageJ image description as dict. 14511 14512 Raise ValueError if not a valid ImageJ description. 14513 14514 >>> description = 'ImageJ=1.11a\nimages=510\nhyperstack=true\n' 14515 >>> imagej_description_metadata(description) # doctest: +SKIP 14516 {'ImageJ': '1.11a', 'images': 510, 'hyperstack': True} 14517 14518 """ 14519 14520 def _bool(val): 14521 return {'true': True, 'false': False}[val.lower()] 14522 14523 result = {} 14524 for line in description.splitlines(): 14525 try: 14526 key, val = line.split('=') 14527 except Exception: 14528 continue 14529 key = key.strip() 14530 val = val.strip() 14531 for dtype in (int, float, _bool): 14532 try: 14533 val = dtype(val) 14534 break 14535 except Exception: 14536 pass 14537 result[key] = val 14538 14539 if 'ImageJ' not in result: 14540 raise ValueError('not an ImageJ image description') 14541 return result 14542 14543 14544def imagej_description( 14545 shape, 14546 rgb=None, 14547 colormaped=False, 14548 version=None, 14549 hyperstack=None, 14550 mode=None, 14551 loop=None, 14552 **kwargs, 14553): 14554 """Return ImageJ image description from data shape. 14555 14556 ImageJ can handle up to 6 dimensions in order TZCYXS. 14557 14558 >>> imagej_description((51, 5, 2, 196, 171)) # doctest: +SKIP 14559 ImageJ=1.11a 14560 images=510 14561 channels=2 14562 slices=5 14563 frames=51 14564 hyperstack=true 14565 mode=grayscale 14566 loop=false 14567 14568 """ 14569 if colormaped: 14570 raise NotImplementedError('ImageJ colormapping not supported') 14571 if version is None: 14572 version = kwargs.pop('ImageJ', '1.11a') 14573 axes = kwargs.pop('axes', None) 14574 shape = imagej_shape(shape, rgb=rgb, axes=axes) 14575 rgb = shape[-1] in (3, 4) 14576 14577 append = [] 14578 result = [f'ImageJ={version}'] 14579 result.append(f'images={product(shape[:-3])}') 14580 if hyperstack is None: 14581 hyperstack = True 14582 append.append('hyperstack=true') 14583 else: 14584 append.append(f'hyperstack={bool(hyperstack)}') 14585 if shape[2] > 1: 14586 result.append(f'channels={shape[2]}') 14587 if mode is None and not rgb: 14588 mode = 'grayscale' 14589 if hyperstack and mode: 14590 append.append(f'mode={mode}') 14591 if shape[1] > 1: 14592 result.append(f'slices={shape[1]}') 14593 if shape[0] > 1: 14594 result.append(f'frames={shape[0]}') 14595 if loop is None: 14596 append.append('loop=false') 14597 if loop is not None: 14598 append.append(f'loop={bool(loop)}'.lower()) 14599 14600 for key, value in kwargs.items(): 14601 if key not in ('images', 'channels', 'slices', 'frames'): 14602 append.append(f'{key.lower()}={value}') 14603 14604 return '\n'.join(result + append + ['']) 14605 14606 14607def imagej_shape(shape, rgb=None, axes=None): 14608 """Return shape normalized to 6D ImageJ hyperstack TZCYXS. 14609 14610 Raise ValueError if not a valid ImageJ hyperstack shape or axes order. 14611 14612 >>> imagej_shape((2, 3, 4, 5, 3), False) 14613 (2, 3, 4, 5, 3, 1) 14614 14615 """ 14616 shape = tuple(int(i) for i in shape) 14617 ndim = len(shape) 14618 if 1 > ndim > 6: 14619 raise ValueError('ImageJ hyperstack must be 2-6 dimensional') 14620 14621 if axes: 14622 if len(axes) != ndim: 14623 raise ValueError('ImageJ hyperstack shape and axes do not match') 14624 i = 0 14625 axes = axes.upper() 14626 for ax in axes: 14627 j = 'TZCYXS'.find(ax) 14628 if j < i: 14629 raise ValueError( 14630 'ImageJ hyperstack axes must be in TZCYXS order' 14631 ) 14632 i = j 14633 ndims = len(axes) 14634 newshape = [] 14635 i = 0 14636 for ax in 'TZCYXS': 14637 if i < ndims and ax == axes[i]: 14638 newshape.append(shape[i]) 14639 i += 1 14640 else: 14641 newshape.append(1) 14642 if newshape[-1] not in (1, 3, 4): 14643 raise ValueError( 14644 'ImageJ hyperstack must contain 1, 3, or 4 samples' 14645 ) 14646 return tuple(newshape) 14647 14648 if rgb is None: 14649 rgb = shape[-1] in (3, 4) and ndim > 2 14650 if rgb and shape[-1] not in (3, 4): 14651 raise ValueError('ImageJ hyperstack is not a RGB image') 14652 if not rgb and ndim == 6 and shape[-1] != 1: 14653 raise ValueError('ImageJ hyperstack is not a grayscale image') 14654 if rgb or shape[-1] == 1: 14655 return (1,) * (6 - ndim) + shape 14656 return (1,) * (5 - ndim) + shape + (1,) 14657 14658 14659def jpeg_decode_colorspace(photometric, planarconfig, extrasamples): 14660 """Return JPEG and output colorspace for jpeg_decode function.""" 14661 colorspace = None 14662 outcolorspace = None 14663 if extrasamples: 14664 pass 14665 elif photometric == 6: 14666 # YCBCR -> RGB 14667 outcolorspace = 2 # RGB 14668 elif photometric == 2: 14669 if planarconfig == 1: 14670 colorspace = outcolorspace = 2 # RGB 14671 elif photometric == 5: 14672 # CMYK 14673 outcolorspace = 4 14674 elif photometric > 3: 14675 outcolorspace = TIFF.PHOTOMETRIC(photometric).name 14676 return colorspace, outcolorspace 14677 14678 14679def jpeg_shape(jpeg): 14680 """Return bitdepth and shape of JPEG image.""" 14681 i = 0 14682 while True and i < len(jpeg): 14683 marker = struct.unpack('>H', jpeg[i : i + 2])[0] 14684 i += 2 14685 14686 if marker == 0xFFD8: 14687 # start of image 14688 continue 14689 if marker == 0xFFD9: 14690 # end of image 14691 break 14692 if 0xFFD0 <= marker <= 0xFFD7: 14693 # restart marker 14694 continue 14695 if marker == 0xFF01: 14696 # private marker 14697 continue 14698 14699 length = struct.unpack('>H', jpeg[i : i + 2])[0] 14700 i += 2 14701 14702 if 0xFFC0 <= marker <= 0xFFC3: 14703 # start of frame 14704 return struct.unpack('>BHHB', jpeg[i : i + 6]) 14705 if marker == 0xFFDA: 14706 # start of scan 14707 break 14708 14709 # skip to next marker 14710 i += length - 2 14711 14712 raise ValueError('no SOF marker found') 14713 14714 14715def ndpi_jpeg_tile(jpeg): 14716 """Return tile shape and JPEG header from JPEG with restart markers.""" 14717 restartinterval = 0 14718 sofoffset = 0 14719 sosoffset = 0 14720 i = 0 14721 while True and i < len(jpeg): 14722 marker = struct.unpack('>H', jpeg[i : i + 2])[0] 14723 i += 2 14724 14725 if marker == 0xFFD8: 14726 # start of image 14727 continue 14728 if marker == 0xFFD9: 14729 # end of image 14730 break 14731 if 0xFFD0 <= marker <= 0xFFD7: 14732 # restart marker 14733 continue 14734 if marker == 0xFF01: 14735 # private marker 14736 continue 14737 14738 length = struct.unpack('>H', jpeg[i : i + 2])[0] 14739 i += 2 14740 14741 if marker == 0xFFDD: 14742 # define restart interval 14743 restartinterval = struct.unpack('>H', jpeg[i : i + 2])[0] 14744 14745 elif marker == 0xFFC0: 14746 # start of frame 14747 sofoffset = i + 1 14748 precision, imlength, imwidth, ncomponents = struct.unpack( 14749 '>BHHB', jpeg[i : i + 6] 14750 ) 14751 i += 6 14752 mcuwidth = 1 14753 mcuheight = 1 14754 for _ in range(ncomponents): 14755 cid, factor, table = struct.unpack('>BBB', jpeg[i : i + 3]) 14756 i += 3 14757 if factor >> 4 > mcuwidth: 14758 mcuwidth = factor >> 4 14759 if factor & 0b00001111 > mcuheight: 14760 mcuheight = factor & 0b00001111 14761 mcuwidth *= 8 14762 mcuheight *= 8 14763 i = sofoffset - 1 14764 14765 elif marker == 0xFFDA: 14766 # start of scan 14767 sosoffset = i + length - 2 14768 break 14769 14770 # skip to next marker 14771 i += length - 2 14772 14773 if restartinterval == 0 or sofoffset == 0 or sosoffset == 0: 14774 raise ValueError('missing required JPEG markers') 14775 14776 # patch jpeg header for tile size 14777 tilelength = mcuheight 14778 tilewidth = restartinterval * mcuwidth 14779 jpegheader = ( 14780 jpeg[:sofoffset] 14781 + struct.pack('>HH', tilelength, tilewidth) 14782 + jpeg[sofoffset + 4 : sosoffset] 14783 ) 14784 return tilelength, tilewidth, jpegheader 14785 14786 14787def json_description(shape, **metadata): 14788 """Return JSON image description from data shape and other metadata. 14789 14790 Return UTF-8 encoded JSON. 14791 14792 >>> json_description((256, 256, 3), axes='YXS') # doctest: +SKIP 14793 b'{"shape": [256, 256, 3], "axes": "YXS"}' 14794 14795 """ 14796 metadata.update(shape=shape) 14797 return json.dumps(metadata) # .encode() 14798 14799 14800def json_description_metadata(description): 14801 """Return metatata from JSON formated image description as dict. 14802 14803 Raise ValuError if description is of unknown format. 14804 14805 >>> description = '{"shape": [256, 256, 3], "axes": "YXS"}' 14806 >>> json_description_metadata(description) # doctest: +SKIP 14807 {'shape': [256, 256, 3], 'axes': 'YXS'} 14808 >>> json_description_metadata('shape=(256, 256, 3)') 14809 {'shape': (256, 256, 3)} 14810 14811 """ 14812 if description[:6] == 'shape=': 14813 # old-style 'shaped' description; not JSON 14814 shape = tuple(int(i) for i in description[7:-1].split(',')) 14815 return dict(shape=shape) 14816 if description[:1] == '{' and description[-1:] == '}': 14817 # JSON description 14818 return json.loads(description) 14819 raise ValueError('invalid JSON image description', description) 14820 14821 14822def fluoview_description_metadata(description, ignoresections=None): 14823 r"""Return metatata from FluoView image description as dict. 14824 14825 The FluoView image description format is unspecified. Expect failures. 14826 14827 >>> descr = ('[Intensity Mapping]\nMap Ch0: Range=00000 to 02047\n' 14828 ... '[Intensity Mapping End]') 14829 >>> fluoview_description_metadata(descr) 14830 {'Intensity Mapping': {'Map Ch0: Range': '00000 to 02047'}} 14831 14832 """ 14833 if not description.startswith('['): 14834 raise ValueError('invalid FluoView image description') 14835 if ignoresections is None: 14836 ignoresections = {'Region Info (Fields)', 'Protocol Description'} 14837 14838 result = {} 14839 sections = [result] 14840 comment = False 14841 for line in description.splitlines(): 14842 if not comment: 14843 line = line.strip() 14844 if not line: 14845 continue 14846 if line[0] == '[': 14847 if line[-5:] == ' End]': 14848 # close section 14849 del sections[-1] 14850 section = sections[-1] 14851 name = line[1:-5] 14852 if comment: 14853 section[name] = '\n'.join(section[name]) 14854 if name[:4] == 'LUT ': 14855 a = numpy.array(section[name], dtype=numpy.uint8) 14856 a.shape = -1, 3 14857 section[name] = a 14858 continue 14859 # new section 14860 comment = False 14861 name = line[1:-1] 14862 if name[:4] == 'LUT ': 14863 section = [] 14864 elif name in ignoresections: 14865 section = [] 14866 comment = True 14867 else: 14868 section = {} 14869 sections.append(section) 14870 result[name] = section 14871 continue 14872 # add entry 14873 if comment: 14874 section.append(line) 14875 continue 14876 line = line.split('=', 1) 14877 if len(line) == 1: 14878 section[line[0].strip()] = None 14879 continue 14880 key, value = line 14881 if key[:4] == 'RGB ': 14882 section.extend(int(rgb) for rgb in value.split()) 14883 else: 14884 section[key.strip()] = astype(value.strip()) 14885 return result 14886 14887 14888def pilatus_description_metadata(description): 14889 """Return metatata from Pilatus image description as dict. 14890 14891 Return metadata from Pilatus pixel array detectors by Dectris, created 14892 by camserver or TVX software. 14893 14894 >>> pilatus_description_metadata('# Pixel_size 172e-6 m x 172e-6 m') 14895 {'Pixel_size': (0.000172, 0.000172)} 14896 14897 """ 14898 result = {} 14899 if not description.startswith('# '): 14900 return result 14901 for c in '#:=,()': 14902 description = description.replace(c, ' ') 14903 for line in description.split('\n'): 14904 if line[:2] != ' ': 14905 continue 14906 line = line.split() 14907 name = line[0] 14908 if line[0] not in TIFF.PILATUS_HEADER: 14909 try: 14910 result['DateTime'] = datetime.datetime.strptime( 14911 ' '.join(line), '%Y-%m-%dT%H %M %S.%f' 14912 ) 14913 except Exception: 14914 result[name] = ' '.join(line[1:]) 14915 continue 14916 indices, dtype = TIFF.PILATUS_HEADER[line[0]] 14917 if isinstance(indices[0], slice): 14918 # assumes one slice 14919 values = line[indices[0]] 14920 else: 14921 values = [line[i] for i in indices] 14922 if dtype is float and values[0] == 'not': 14923 values = ['NaN'] 14924 values = tuple(dtype(v) for v in values) 14925 if dtype == str: 14926 values = ' '.join(values) 14927 elif len(values) == 1: 14928 values = values[0] 14929 result[name] = values 14930 return result 14931 14932 14933def svs_description_metadata(description): 14934 """Return metatata from Aperio image description as dict. 14935 14936 The Aperio image description format is unspecified. Expect failures. 14937 14938 >>> svs_description_metadata('Aperio Image Library v1.0') 14939 {'Header': 'Aperio Image Library v1.0'} 14940 14941 """ 14942 if not description.startswith('Aperio '): 14943 raise ValueError('invalid Aperio image description') 14944 result = {} 14945 items = description.split('|') 14946 result['Header'] = items[0] 14947 if len(items) == 1: 14948 return result 14949 for item in items[1:]: 14950 key, value = item.split(' = ') 14951 result[key.strip()] = astype(value.strip()) 14952 return result 14953 14954 14955def stk_description_metadata(description): 14956 """Return metadata from MetaMorph image description as list of dict. 14957 14958 The MetaMorph image description format is unspecified. Expect failures. 14959 14960 """ 14961 description = description.strip() 14962 if not description: 14963 return [] 14964 try: 14965 description = bytes2str(description) 14966 except UnicodeDecodeError as exc: 14967 log_warning( 14968 f'stk_description_metadata: {exc.__class__.__name__}: {exc}' 14969 ) 14970 return [] 14971 result = [] 14972 for plane in description.split('\x00'): 14973 d = {} 14974 for line in plane.split('\r\n'): 14975 line = line.split(':', 1) 14976 if len(line) > 1: 14977 name, value = line 14978 d[name.strip()] = astype(value.strip()) 14979 else: 14980 value = line[0].strip() 14981 if value: 14982 if '' in d: 14983 d[''].append(value) 14984 else: 14985 d[''] = [value] 14986 result.append(d) 14987 return result 14988 14989 14990def metaseries_description_metadata(description): 14991 """Return metatata from MetaSeries image description as dict.""" 14992 if not description.startswith('<MetaData>'): 14993 raise ValueError('invalid MetaSeries image description') 14994 14995 from xml.etree import ElementTree as etree # delayed import 14996 14997 root = etree.fromstring(description) 14998 types = { 14999 'float': float, 15000 'int': int, 15001 'bool': lambda x: asbool(x, 'on', 'off'), 15002 } 15003 15004 def parse(root, result): 15005 # recursive 15006 for child in root: 15007 attrib = child.attrib 15008 if not attrib: 15009 result[child.tag] = parse(child, {}) 15010 continue 15011 if 'id' in attrib: 15012 i = attrib['id'] 15013 t = attrib['type'] 15014 v = attrib['value'] 15015 if t in types: 15016 result[i] = types[t](v) 15017 else: 15018 result[i] = v 15019 return result 15020 15021 adict = parse(root, {}) 15022 if 'Description' in adict: 15023 adict['Description'] = adict['Description'].replace(' ', '\n') 15024 return adict 15025 15026 15027def scanimage_description_metadata(description): 15028 """Return metatata from ScanImage image description as dict.""" 15029 return matlabstr2py(description) 15030 15031 15032def scanimage_artist_metadata(artist): 15033 """Return metatata from ScanImage artist tag as dict.""" 15034 try: 15035 return json.loads(artist) 15036 except ValueError as exc: 15037 log_warning( 15038 f'scanimage_artist_metadata: {exc.__class__.__name__}: {exc}' 15039 ) 15040 return None 15041 15042 15043def olympusini_metadata(inistr): 15044 """Return OlympusSIS metadata from INI string. 15045 15046 No documentation is available. 15047 15048 """ 15049 15050 def keyindex(key): 15051 # split key into name and index 15052 index = 0 15053 i = len(key.rstrip('0123456789')) 15054 if i < len(key): 15055 index = int(key[i:]) - 1 15056 key = key[:i] 15057 return key, index 15058 15059 result = {} 15060 bands = [] 15061 zpos = None 15062 tpos = None 15063 for line in inistr.splitlines(): 15064 line = line.strip() 15065 if line == '' or line[0] == ';': 15066 continue 15067 if line[0] == '[' and line[-1] == ']': 15068 section_name = line[1:-1] 15069 result[section_name] = section = {} 15070 if section_name == 'Dimension': 15071 result['axes'] = axes = [] 15072 result['shape'] = shape = [] 15073 elif section_name == 'ASD': 15074 result[section_name] = [] 15075 elif section_name == 'Z': 15076 if 'Dimension' in result: 15077 result[section_name]['ZPos'] = zpos = [] 15078 elif section_name == 'Time': 15079 if 'Dimension' in result: 15080 result[section_name]['TimePos'] = tpos = [] 15081 elif section_name == 'Band': 15082 nbands = result['Dimension']['Band'] 15083 bands = [{'LUT': []} for _ in range(nbands)] 15084 result[section_name] = bands 15085 iband = 0 15086 else: 15087 key, value = line.split('=') 15088 if value.strip() == '': 15089 value = None 15090 elif ',' in value: 15091 value = tuple(astype(v) for v in value.split(',')) 15092 else: 15093 value = astype(value) 15094 15095 if section_name == 'Dimension': 15096 section[key] = value 15097 axes.append(key) 15098 shape.append(value) 15099 elif section_name == 'ASD': 15100 if key == 'Count': 15101 result['ASD'] = [{}] * value 15102 else: 15103 key, index = keyindex(key) 15104 result['ASD'][index][key] = value 15105 elif section_name == 'Band': 15106 if key[:3] == 'LUT': 15107 lut = bands[iband]['LUT'] 15108 value = struct.pack('<I', value) 15109 lut.append( 15110 [ord(value[0:1]), ord(value[1:2]), ord(value[2:3])] 15111 ) 15112 else: 15113 key, iband = keyindex(key) 15114 bands[iband][key] = value 15115 elif key[:4] == 'ZPos' and zpos is not None: 15116 zpos.append(value) 15117 elif key[:7] == 'TimePos' and tpos is not None: 15118 tpos.append(value) 15119 else: 15120 section[key] = value 15121 15122 if 'axes' in result: 15123 sisaxes = {'Band': 'C'} 15124 axes = [] 15125 shape = [] 15126 for i, x in zip(result['shape'], result['axes']): 15127 if i > 1: 15128 axes.append(sisaxes.get(x, x[0].upper())) 15129 shape.append(i) 15130 result['axes'] = ''.join(axes) 15131 result['shape'] = tuple(shape) 15132 try: 15133 result['Z']['ZPos'] = numpy.array( 15134 result['Z']['ZPos'][: result['Dimension']['Z']], 'float64' 15135 ) 15136 except Exception: 15137 pass 15138 try: 15139 result['Time']['TimePos'] = numpy.array( 15140 result['Time']['TimePos'][: result['Dimension']['Time']], 'int32' 15141 ) 15142 except Exception: 15143 pass 15144 for band in bands: 15145 band['LUT'] = numpy.array(band['LUT'], 'uint8') 15146 return result 15147 15148 15149def unpack_rgb(data, dtype=None, bitspersample=None, rescale=True): 15150 """Return array from bytes containing packed samples. 15151 15152 Use to unpack RGB565 or RGB555 to RGB888 format. 15153 Works on little-endian platforms only. 15154 15155 Parameters 15156 ---------- 15157 data : byte str 15158 The data to be decoded. Samples in each pixel are stored consecutively. 15159 Pixels are aligned to 8, 16, or 32 bit boundaries. 15160 dtype : numpy.dtype 15161 The sample data type. The byteorder applies also to the data stream. 15162 bitspersample : tuple 15163 Number of bits for each sample in a pixel. 15164 rescale : bool 15165 Upscale samples to the number of bits in dtype. 15166 15167 Returns 15168 ------- 15169 numpy.ndarray 15170 Flattened array of unpacked samples of native dtype. 15171 15172 Examples 15173 -------- 15174 >>> data = struct.pack('BBBB', 0x21, 0x08, 0xff, 0xff) 15175 >>> print(unpack_rgb(data, '<B', (5, 6, 5), False)) 15176 [ 1 1 1 31 63 31] 15177 >>> print(unpack_rgb(data, '<B', (5, 6, 5))) 15178 [ 8 4 8 255 255 255] 15179 >>> print(unpack_rgb(data, '<B', (5, 5, 5))) 15180 [ 16 8 8 255 255 255] 15181 15182 """ 15183 if bitspersample is None: 15184 bitspersample = (5, 6, 5) 15185 if dtype is None: 15186 dtype = '<B' 15187 dtype = numpy.dtype(dtype) 15188 bits = int(numpy.sum(bitspersample)) 15189 if not ( 15190 bits <= 32 and all(i <= dtype.itemsize * 8 for i in bitspersample) 15191 ): 15192 raise ValueError(f'sample size not supported: {bitspersample}') 15193 dt = next(i for i in 'BHI' if numpy.dtype(i).itemsize * 8 >= bits) 15194 data = numpy.frombuffer(data, dtype.byteorder + dt) 15195 result = numpy.empty((data.size, len(bitspersample)), dtype.char) 15196 for i, bps in enumerate(bitspersample): 15197 t = data >> int(numpy.sum(bitspersample[i + 1 :])) 15198 t &= int('0b' + '1' * bps, 2) 15199 if rescale: 15200 o = ((dtype.itemsize * 8) // bps + 1) * bps 15201 if o > data.dtype.itemsize * 8: 15202 t = t.astype('I') 15203 t *= (2 ** o - 1) // (2 ** bps - 1) 15204 t //= 2 ** (o - (dtype.itemsize * 8)) 15205 result[:, i] = t 15206 return result.reshape(-1) 15207 15208 15209def float24_decode(data, byteorder): 15210 """Return float32 array from float24.""" 15211 raise NotImplementedError('float24_decode') 15212 15213 15214def zlib_encode(data, level=None, out=None): 15215 """Compress Zlib DEFLATE.""" 15216 import zlib 15217 15218 return zlib.compress(data, 6 if level is None else level) 15219 15220 15221def zlib_decode(data, out=None): 15222 """Decompress Zlib DEFLATE.""" 15223 import zlib 15224 15225 return zlib.decompress(data) 15226 15227 15228def lzma_encode(data, level=None, out=None): 15229 """Compress LZMA.""" 15230 import lzma 15231 15232 return lzma.compress(data) 15233 15234 15235def lzma_decode(data, out=None): 15236 """Decompress LZMA.""" 15237 import lzma 15238 15239 return lzma.decompress(data) 15240 15241 15242if imagecodecs is None: 15243 15244 def delta_encode(data, axis=-1, dist=1, out=None): 15245 """Encode Delta.""" 15246 if dist != 1: 15247 raise NotImplementedError(f'dist {dist} not implemented') 15248 if isinstance(data, (bytes, bytearray)): 15249 data = numpy.frombuffer(data, dtype=numpy.uint8) 15250 diff = numpy.diff(data, axis=0) 15251 return numpy.insert(diff, 0, data[0]).tobytes() 15252 15253 dtype = data.dtype 15254 if dtype.kind == 'f': 15255 data = data.view(f'u{dtype.itemsize}') 15256 15257 diff = numpy.diff(data, axis=axis) 15258 key = [slice(None)] * data.ndim 15259 key[axis] = 0 15260 diff = numpy.insert(diff, 0, data[tuple(key)], axis=axis) 15261 15262 if dtype.kind == 'f': 15263 return diff.view(dtype) 15264 return diff 15265 15266 def delta_decode(data, axis=-1, dist=1, out=None): 15267 """Decode Delta.""" 15268 if dist != 1: 15269 raise NotImplementedError(f'dist {dist} not implemented') 15270 if out is not None and not out.flags.writeable: 15271 out = None 15272 if isinstance(data, (bytes, bytearray)): 15273 data = numpy.frombuffer(data, dtype=numpy.uint8) 15274 return numpy.cumsum( 15275 data, axis=0, dtype=numpy.uint8, out=out 15276 ).tobytes() 15277 if data.dtype.kind == 'f': 15278 view = data.view(f'u{data.dtype.itemsize}') 15279 view = numpy.cumsum(view, axis=axis, dtype=view.dtype) 15280 return view.view(data.dtype) 15281 return numpy.cumsum(data, axis=axis, dtype=data.dtype, out=out) 15282 15283 def bitorder_decode(data, out=None, _bitorder=[]): 15284 r"""Reverse bits in each byte of bytes or numpy array. 15285 15286 Decode data where pixels with lower column values are stored in the 15287 lower-order bits of the bytes (TIFF FillOrder is LSB2MSB). 15288 15289 Parameters 15290 ---------- 15291 data : bytes or ndarray 15292 The data to be bit reversed. If bytes, a new bit-reversed 15293 bytes is returned. Numpy arrays are bit-reversed in-place. 15294 15295 Examples 15296 -------- 15297 >>> bitorder_decode(b'\x01\x64') 15298 b'\x80&' 15299 >>> data = numpy.array([1, 666], dtype='uint16') 15300 >>> bitorder_decode(data) 15301 >>> data 15302 array([ 128, 16473], dtype=uint16) 15303 15304 """ 15305 if not _bitorder: 15306 _bitorder.append( 15307 b'\x00\x80@\xc0 \xa0`\xe0\x10\x90P\xd00\xb0p\xf0\x08\x88H' 15308 b'\xc8(\xa8h\xe8\x18\x98X\xd88\xb8x\xf8\x04\x84D\xc4$\xa4d' 15309 b'\xe4\x14\x94T\xd44\xb4t\xf4\x0c\x8cL\xcc,\xacl\xec\x1c\x9c' 15310 b'\\\xdc<\xbc|\xfc\x02\x82B\xc2"\xa2b\xe2\x12\x92R\xd22' 15311 b'\xb2r\xf2\n\x8aJ\xca*\xaaj\xea\x1a\x9aZ\xda:\xbaz\xfa' 15312 b'\x06\x86F\xc6&\xa6f\xe6\x16\x96V\xd66\xb6v\xf6\x0e\x8eN' 15313 b'\xce.\xaen\xee\x1e\x9e^\xde>\xbe~\xfe\x01\x81A\xc1!\xa1a' 15314 b'\xe1\x11\x91Q\xd11\xb1q\xf1\t\x89I\xc9)\xa9i\xe9\x19' 15315 b'\x99Y\xd99\xb9y\xf9\x05\x85E\xc5%\xa5e\xe5\x15\x95U\xd55' 15316 b'\xb5u\xf5\r\x8dM\xcd-\xadm\xed\x1d\x9d]\xdd=\xbd}\xfd' 15317 b'\x03\x83C\xc3#\xa3c\xe3\x13\x93S\xd33\xb3s\xf3\x0b\x8bK' 15318 b'\xcb+\xabk\xeb\x1b\x9b[\xdb;\xbb{\xfb\x07\x87G\xc7\'\xa7g' 15319 b'\xe7\x17\x97W\xd77\xb7w\xf7\x0f\x8fO\xcf/\xafo\xef\x1f\x9f_' 15320 b'\xdf?\xbf\x7f\xff' 15321 ) 15322 _bitorder.append(numpy.frombuffer(_bitorder[0], dtype=numpy.uint8)) 15323 try: 15324 view = data.view('uint8') 15325 numpy.take(_bitorder[1], view, out=view) 15326 return data 15327 except AttributeError: 15328 return data.translate(_bitorder[0]) 15329 except ValueError: 15330 raise NotImplementedError('slices of arrays not supported') 15331 return None 15332 15333 def packints_encode(data, bitspersample, axis=-1, out=None): 15334 """Tightly pack integers.""" 15335 raise NotImplementedError('packints_encode') 15336 15337 def packints_decode(data, dtype, bitspersample, runlen=0, out=None): 15338 """Decompress bytes to array of integers. 15339 15340 This implementation only handles itemsizes 1, 8, 16, 32, and 64 bits. 15341 Install the imagecodecs package for decoding other integer sizes. 15342 15343 Parameters 15344 ---------- 15345 data : byte str 15346 Data to decompress. 15347 dtype : numpy.dtype or str 15348 A numpy boolean or integer type. 15349 bitspersample : int 15350 Number of bits per integer. 15351 runlen : int 15352 Number of consecutive integers, after which to start at next byte. 15353 15354 Examples 15355 -------- 15356 >>> packints_decode(b'a', 'B', 1) 15357 array([0, 1, 1, 0, 0, 0, 0, 1], dtype=uint8) 15358 15359 """ 15360 if bitspersample == 1: # bitarray 15361 data = numpy.frombuffer(data, '|B') 15362 data = numpy.unpackbits(data) 15363 if runlen % 8: 15364 data = data.reshape(-1, runlen + (8 - runlen % 8)) 15365 data = data[:, :runlen].reshape(-1) 15366 return data.astype(dtype) 15367 if bitspersample in (8, 16, 32, 64): 15368 return numpy.frombuffer(data, dtype) 15369 raise NotImplementedError( 15370 f'unpacking {bitspersample}-bit integers ' 15371 f'to {numpy.dtype(dtype)} not supported' 15372 ) 15373 15374 def packbits_decode(encoded, out=None): 15375 r"""Decompress PackBits encoded byte string. 15376 15377 >>> packbits_decode(b'\x80\x80') # NOP 15378 b'' 15379 >>> packbits_decode(b'\x02123') 15380 b'123' 15381 >>> packbits_decode( 15382 ... b'\xfe\xaa\x02\x80\x00\x2a\xfd\xaa\x03\x80\x00\x2a\x22\xf7\xaa' 15383 ... )[:-5] 15384 b'\xaa\xaa\xaa\x80\x00*\xaa\xaa\xaa\xaa\x80\x00*"\xaa\xaa\xaa\xaa\xaa' 15385 15386 """ 15387 out = [] 15388 out_extend = out.extend 15389 i = 0 15390 try: 15391 while True: 15392 n = ord(encoded[i : i + 1]) + 1 15393 i += 1 15394 if n > 129: 15395 # replicate 15396 out_extend(encoded[i : i + 1] * (258 - n)) 15397 i += 1 15398 elif n < 129: 15399 # literal 15400 out_extend(encoded[i : i + n]) 15401 i += n 15402 except TypeError: 15403 pass 15404 return bytes(out) 15405 15406 15407else: 15408 bitorder_decode = imagecodecs.bitorder_decode # noqa 15409 packints_decode = imagecodecs.packints_decode # noqa 15410 packints_encode = imagecodecs.packints_encode # noqa 15411 try: 15412 float24_decode = imagecodecs.float24_decode # noqa 15413 except AttributeError: 15414 pass 15415 15416 15417def apply_colormap(image, colormap, contig=True): 15418 """Return palette-colored image. 15419 15420 The image values are used to index the colormap on axis 1. The returned 15421 image is of shape image.shape+colormap.shape[0] and dtype colormap.dtype. 15422 15423 Parameters 15424 ---------- 15425 image : numpy.ndarray 15426 Indexes into the colormap. 15427 colormap : numpy.ndarray 15428 RGB lookup table aka palette of shape (3, 2**bits_per_sample). 15429 contig : bool 15430 If True, return a contiguous array. 15431 15432 Examples 15433 -------- 15434 >>> image = numpy.arange(256, dtype='uint8') 15435 >>> colormap = numpy.vstack([image, image, image]).astype('uint16') * 256 15436 >>> apply_colormap(image, colormap)[-1] 15437 array([65280, 65280, 65280], dtype=uint16) 15438 15439 """ 15440 image = numpy.take(colormap, image, axis=1) 15441 image = numpy.rollaxis(image, 0, image.ndim) 15442 if contig: 15443 image = numpy.ascontiguousarray(image) 15444 return image 15445 15446 15447def parse_filenames(files, pattern, axesorder=None): 15448 r"""Return shape and axes from sequence of file names matching pattern. 15449 15450 >>> parse_filenames( 15451 ... ['c1001.ext', 'c2002.ext'], r'([^\d])(\d)(?P<t>\d+)\.ext' 15452 ... ) 15453 ('ct', (2, 2), [(0, 0), (1, 1)], (1, 1)) 15454 15455 """ 15456 if not pattern: 15457 raise ValueError('invalid pattern') 15458 if isinstance(pattern, str): 15459 pattern = re.compile(pattern) 15460 15461 def parse(fname, pattern=pattern): 15462 # return axes and indices from file name 15463 # fname = os.path.split(fname)[-1] 15464 axes = [] 15465 indices = [] 15466 groupindex = {v: k for k, v in pattern.groupindex.items()} 15467 match = pattern.search(fname) 15468 if not match: 15469 raise ValueError('pattern does not match file name') 15470 ax = None 15471 for i, m in enumerate(match.groups()): 15472 if m is None: 15473 continue 15474 if i + 1 in groupindex: 15475 ax = groupindex[i + 1] # names axis 15476 if not m[0].isdigit(): 15477 m = ord(m) # index letter to number 15478 if m < 65 or m > 122: 15479 raise ValueError(f'invalid index {m!r}') 15480 elif m[0].isalpha(): 15481 ax = m # axis letter for next index 15482 continue 15483 if ax is None: 15484 ax = 'Q' # no preceding axis letter 15485 try: 15486 m = int(m) 15487 except Exception: 15488 raise ValueError(f'invalid index {m!r}') 15489 indices.append(m) 15490 axes.append(ax) 15491 ax = None 15492 return ''.join(axes), indices 15493 15494 files = [os.path.normpath(f) for f in files] 15495 if len(files) == 1: 15496 prefix = os.path.dirname(files[0]) 15497 else: 15498 prefix = os.path.commonpath(files) 15499 prefix = len(prefix) 15500 15501 axes = None 15502 indices = [] 15503 for fname in files: 15504 ax, idx = parse(fname[prefix:]) 15505 if axes is None: 15506 axes = ax 15507 if axesorder is not None and ( 15508 len(axesorder) != len(axes) 15509 or any(i not in axesorder for i in range(len(axes))) 15510 ): 15511 raise ValueError('invalid axisorder') 15512 elif axes != ax: 15513 raise ValueError('axes do not match within image sequence') 15514 if axesorder is not None: 15515 idx = [idx[i] for i in axesorder] 15516 indices.append(idx) 15517 15518 if axesorder is not None: 15519 axes = ''.join(axes[i] for i in axesorder) 15520 15521 indices = numpy.array(indices, dtype=numpy.intp) 15522 startindex = numpy.min(indices, axis=0) 15523 shape = numpy.max(indices, axis=0) 15524 shape -= startindex 15525 shape += 1 15526 shape = tuple(shape.tolist()) 15527 indices -= startindex 15528 indices = indices.tolist() 15529 indices = [tuple(index) for index in indices] 15530 startindex = tuple(startindex.tolist()) 15531 return axes, shape, indices, startindex 15532 15533 15534def iter_images(data): 15535 """Return iterator over pages in data array of normalized shape.""" 15536 yield from data 15537 15538 15539def iter_tiles(data, tile, tiles): 15540 """Return iterator over tiles in data array of normalized shape.""" 15541 shape = data.shape 15542 chunk = numpy.empty(tile + (shape[-1],), dtype=data.dtype) 15543 if not 1 < len(tile) < 4: 15544 raise ValueError('invalid tile shape') 15545 if len(tile) == 2: 15546 for page in data: 15547 for plane in page: 15548 for ty in range(tiles[0]): 15549 for tx in range(tiles[1]): 15550 c1 = min(tile[0], shape[3] - ty * tile[0]) 15551 c2 = min(tile[1], shape[4] - tx * tile[1]) 15552 chunk[c1:, c2:] = 0 15553 chunk[:c1, :c2] = plane[ 15554 0, 15555 ty * tile[0] : ty * tile[0] + c1, 15556 tx * tile[1] : tx * tile[1] + c2, 15557 ] 15558 yield chunk 15559 else: 15560 for page in data: 15561 for plane in page: 15562 for tz in range(tiles[0]): 15563 for ty in range(tiles[1]): 15564 for tx in range(tiles[2]): 15565 c0 = min(tile[0], shape[2] - tz * tile[0]) 15566 c1 = min(tile[1], shape[3] - ty * tile[1]) 15567 c2 = min(tile[2], shape[4] - tx * tile[2]) 15568 chunk[c0:, c1:, c2:] = 0 15569 chunk[:c0, :c1, :c2] = plane[ 15570 tz * tile[0] : tz * tile[0] + c0, 15571 ty * tile[1] : ty * tile[1] + c1, 15572 tx * tile[2] : tx * tile[2] + c2, 15573 ] 15574 if tile[0] == 1: 15575 # squeeze for image compressors 15576 yield chunk[0] 15577 else: 15578 yield chunk 15579 15580 15581def pad_tile(tile, shape, dtype): 15582 """Return tile padded to tile shape.""" 15583 if tile.dtype != dtype or tile.nbytes > product(shape) * dtype.itemsize: 15584 raise ValueError('invalid tile shape or dtype') 15585 pad = tuple((0, i - j) for i, j in zip(shape, tile.shape)) 15586 return numpy.pad(tile, pad) 15587 15588 15589def reorient(image, orientation): 15590 """Return reoriented view of image array. 15591 15592 Parameters 15593 ---------- 15594 image : numpy.ndarray 15595 Non-squeezed output of asarray() functions. 15596 Axes -3 and -2 must be image length and width respectively. 15597 orientation : int or str 15598 One of TIFF.ORIENTATION names or values. 15599 15600 """ 15601 orient = TIFF.ORIENTATION 15602 orientation = enumarg(orient, orientation) 15603 15604 if orientation == orient.TOPLEFT: 15605 return image 15606 if orientation == orient.TOPRIGHT: 15607 return image[..., ::-1, :] 15608 if orientation == orient.BOTLEFT: 15609 return image[..., ::-1, :, :] 15610 if orientation == orient.BOTRIGHT: 15611 return image[..., ::-1, ::-1, :] 15612 if orientation == orient.LEFTTOP: 15613 return numpy.swapaxes(image, -3, -2) 15614 if orientation == orient.RIGHTTOP: 15615 return numpy.swapaxes(image, -3, -2)[..., ::-1, :] 15616 if orientation == orient.RIGHTBOT: 15617 return numpy.swapaxes(image, -3, -2)[..., ::-1, :, :] 15618 if orientation == orient.LEFTBOT: 15619 return numpy.swapaxes(image, -3, -2)[..., ::-1, ::-1, :] 15620 return image 15621 15622 15623def repeat_nd(a, repeats): 15624 """Return read-only view into input array with elements repeated. 15625 15626 Zoom nD image by integer factors using nearest neighbor interpolation 15627 (box filter). 15628 15629 Parameters 15630 ---------- 15631 a : array-like 15632 Input array. 15633 repeats : sequence of int 15634 The number of repetitions to apply along each dimension of input array. 15635 15636 Examples 15637 -------- 15638 >>> repeat_nd([[1, 2], [3, 4]], (2, 2)) 15639 array([[1, 1, 2, 2], 15640 [1, 1, 2, 2], 15641 [3, 3, 4, 4], 15642 [3, 3, 4, 4]]) 15643 15644 """ 15645 a = numpy.asarray(a) 15646 reshape = [] 15647 shape = [] 15648 strides = [] 15649 for i, j, k in zip(a.strides, a.shape, repeats): 15650 shape.extend((j, k)) 15651 strides.extend((i, 0)) 15652 reshape.append(j * k) 15653 return numpy.lib.stride_tricks.as_strided( 15654 a, shape, strides, writeable=False 15655 ).reshape(reshape) 15656 15657 15658def reshape_nd(data_or_shape, ndim): 15659 """Return image array or shape with at least ndim dimensions. 15660 15661 Prepend 1s to image shape as necessary. 15662 15663 >>> reshape_nd(numpy.empty(0), 1).shape 15664 (0,) 15665 >>> reshape_nd(numpy.empty(1), 2).shape 15666 (1, 1) 15667 >>> reshape_nd(numpy.empty((2, 3)), 3).shape 15668 (1, 2, 3) 15669 >>> reshape_nd(numpy.empty((3, 4, 5)), 3).shape 15670 (3, 4, 5) 15671 >>> reshape_nd((2, 3), 3) 15672 (1, 2, 3) 15673 15674 """ 15675 is_shape = isinstance(data_or_shape, tuple) 15676 shape = data_or_shape if is_shape else data_or_shape.shape 15677 if len(shape) >= ndim: 15678 return data_or_shape 15679 shape = (1,) * (ndim - len(shape)) + shape 15680 return shape if is_shape else data_or_shape.reshape(shape) 15681 15682 15683def squeeze_axes(shape, axes, skip=None): 15684 """Return shape and axes with single-dimensional entries removed. 15685 15686 Remove unused dimensions unless their axes are listed in 'skip'. 15687 15688 >>> squeeze_axes((5, 1, 2, 1, 1), 'TZYXC') 15689 ((5, 2, 1), 'TYX') 15690 15691 >>> squeeze_axes((1,), 'Q') 15692 ((1,), 'Q') 15693 15694 """ 15695 if len(shape) != len(axes): 15696 raise ValueError('dimensions of axes and shape do not match') 15697 if skip is None: 15698 skip = 'XY' 15699 try: 15700 shape_squeezed, axes_squeezed = zip( 15701 *(i for i in zip(shape, axes) if i[0] > 1 or i[1] in skip) 15702 ) 15703 except ValueError: 15704 # not enough values to unpack, return last axis 15705 shape_squeezed = shape[-1:] 15706 axes_squeezed = axes[-1:] 15707 return tuple(shape_squeezed), ''.join(axes_squeezed) 15708 15709 15710def transpose_axes(image, axes, asaxes=None): 15711 """Return image with its axes permuted to match specified axes. 15712 15713 A view is returned if possible. 15714 15715 >>> transpose_axes(numpy.zeros((2, 3, 4, 5)), 'TYXC', asaxes='CTZYX').shape 15716 (5, 2, 1, 3, 4) 15717 15718 """ 15719 for ax in axes: 15720 if ax not in asaxes: 15721 raise ValueError(f'unknown axis {ax}') 15722 # add missing axes to image 15723 if asaxes is None: 15724 asaxes = 'CTZYX' 15725 shape = image.shape 15726 for ax in reversed(asaxes): 15727 if ax not in axes: 15728 axes = ax + axes 15729 shape = (1,) + shape 15730 image = image.reshape(shape) 15731 # transpose axes 15732 image = image.transpose([axes.index(ax) for ax in asaxes]) 15733 return image 15734 15735 15736def reshape_axes(axes, shape, newshape, unknown=None): 15737 """Return axes matching new shape. 15738 15739 By default, unknown dimensions are labelled 'Q'. 15740 15741 >>> reshape_axes('YXS', (219, 301, 1), (219, 301)) 15742 'YX' 15743 >>> reshape_axes('IYX', (12, 219, 301), (3, 4, 219, 1, 301, 1)) 15744 'QQYQXQ' 15745 15746 """ 15747 shape = tuple(shape) 15748 newshape = tuple(newshape) 15749 if len(axes) != len(shape): 15750 raise ValueError('axes do not match shape') 15751 15752 size = product(shape) 15753 newsize = product(newshape) 15754 if size != newsize: 15755 raise ValueError(f'cannot reshape {shape} to {newshape}') 15756 if not axes or not newshape: 15757 return '' 15758 15759 lendiff = max(0, len(shape) - len(newshape)) 15760 if lendiff: 15761 newshape = newshape + (1,) * lendiff 15762 15763 i = len(shape) - 1 15764 prodns = 1 15765 prods = 1 15766 result = [] 15767 for ns in newshape[::-1]: 15768 prodns *= ns 15769 while i > 0 and shape[i] == 1 and ns != 1: 15770 i -= 1 15771 if ns == shape[i] and prodns == prods * shape[i]: 15772 prods *= shape[i] 15773 result.append(axes[i]) 15774 i -= 1 15775 elif unknown: 15776 result.append(unknown) 15777 else: 15778 unknown = 'Q' 15779 result.append(unknown) 15780 15781 return ''.join(reversed(result[lendiff:])) 15782 15783 15784def subresolution(a, b, p=2, n=16): 15785 """Return level of subresolution of series or page b vs a.""" 15786 if a.axes != b.axes or a.dtype != b.dtype: 15787 return None 15788 level = None 15789 for ax, i, j in zip(a.axes.lower(), a.shape, b.shape): 15790 if ax in 'xyz': 15791 if level is None: 15792 for r in range(n): 15793 d = p ** r 15794 if d > i: 15795 return None 15796 if abs((i / d) - j) < 1.0: 15797 level = r 15798 break 15799 else: 15800 return None 15801 else: 15802 d = p ** level 15803 if d > i: 15804 return None 15805 if abs((i / d) - j) >= 1.0: 15806 return None 15807 elif i != j: 15808 return None 15809 return level 15810 15811 15812def pyramidize_series(series, isreduced=False): 15813 """Pyramidize list of TiffPageSeries in-place. 15814 15815 TiffPageSeries that are a subresolution of another TiffPageSeries are 15816 appended to the other's TiffPageSeries levels and removed from the list. 15817 Levels are to be ordered by size using the same downsampling factor. 15818 TiffPageSeries of subifds cannot be pyramid top levels. 15819 15820 """ 15821 samplingfactors = (2, 3, 4) 15822 i = 0 15823 while i < len(series): 15824 a = series[i] 15825 p = None 15826 j = i + 1 15827 if isinstance(a.keyframe.index, tuple): 15828 # subifds cannot be pyramid top levels 15829 i += 1 15830 continue 15831 while j < len(series): 15832 b = series[j] 15833 if isreduced and not b.keyframe.is_reduced: 15834 # pyramid levels must be reduced 15835 j += 1 15836 continue # not a pyramid level 15837 if p is None: 15838 for f in samplingfactors: 15839 if subresolution(a.levels[-1], b, p=f) == 1: 15840 p = f 15841 break # not a pyramid level 15842 else: 15843 j += 1 15844 continue # not a pyramid level 15845 elif subresolution(a.levels[-1], b, p=p) != 1: 15846 j += 1 15847 continue 15848 a.levels.append(b) 15849 del series[j] 15850 i += 1 15851 15852 15853def stack_pages(pages, out=None, maxworkers=None, **kwargs): 15854 """Read data from sequence of TiffPage and stack them vertically. 15855 15856 Additional parameters are passsed to the TiffPage.asarray function. 15857 15858 """ 15859 npages = len(pages) 15860 if npages == 0: 15861 raise ValueError('no pages') 15862 15863 if npages == 1: 15864 kwargs['maxworkers'] = maxworkers 15865 return pages[0].asarray(out=out, **kwargs) 15866 15867 page0 = next(p.keyframe for p in pages if p is not None) 15868 shape = (npages,) + page0.shape 15869 dtype = page0.dtype 15870 out = create_output(out, shape, dtype) 15871 15872 # TODO: benchmark and optimize this 15873 if maxworkers is None or maxworkers < 1: 15874 # auto-detect 15875 page_maxworkers = page0.maxworkers 15876 maxworkers = min(npages, TIFF.MAXWORKERS) 15877 if maxworkers == 1 or page_maxworkers < 1: 15878 maxworkers = page_maxworkers = 1 15879 elif npages < 3: 15880 maxworkers = 1 15881 elif ( 15882 page_maxworkers <= 2 15883 and page0.compression == 1 15884 and page0.fillorder == 1 15885 and page0.predictor == 1 15886 ): 15887 maxworkers = 1 15888 else: 15889 page_maxworkers = 1 15890 elif maxworkers == 1: 15891 maxworkers = page_maxworkers = 1 15892 elif npages > maxworkers or page0.maxworkers < 2: 15893 page_maxworkers = 1 15894 else: 15895 page_maxworkers = maxworkers 15896 maxworkers = 1 15897 15898 kwargs['maxworkers'] = page_maxworkers 15899 15900 filehandle = page0.parent.filehandle 15901 haslock = filehandle.has_lock 15902 if not haslock and maxworkers > 1 or page_maxworkers > 1: 15903 filehandle.lock = True 15904 filecache = FileCache(size=max(4, maxworkers), lock=filehandle.lock) 15905 15906 def func(page, index, out=out, filecache=filecache, kwargs=kwargs): 15907 # read, decode, and copy page data 15908 if page is not None: 15909 filecache.open(page.parent.filehandle) 15910 page.asarray(lock=filecache.lock, out=out[index], **kwargs) 15911 filecache.close(page.parent.filehandle) 15912 15913 if maxworkers < 2: 15914 for i, page in enumerate(pages): 15915 func(page, i) 15916 else: 15917 page0.decode # init TiffPage.decode function 15918 with ThreadPoolExecutor(maxworkers) as executor: 15919 for _ in executor.map(func, pages, range(npages)): 15920 pass 15921 15922 filecache.clear() 15923 if not haslock: 15924 filehandle.lock = False 15925 return out 15926 15927 15928def create_output(out, shape, dtype, mode='w+', suffix=None): 15929 """Return numpy array where image data of shape and dtype can be copied. 15930 15931 The 'out' parameter may have the following values or types: 15932 15933 None 15934 An empty array of shape and dtype is created and returned. 15935 numpy.ndarray 15936 An existing writable array of compatible dtype and shape. A view of 15937 the same array is returned after verification. 15938 'memmap' or 'memmap:tempdir' 15939 A memory-map to an array stored in a temporary binary file on disk 15940 is created and returned. 15941 str or open file 15942 The file name or file object used to create a memory-map to an array 15943 stored in a binary file on disk. The created memory-mapped array is 15944 returned. 15945 15946 """ 15947 if out is None: 15948 return numpy.zeros(shape, dtype) 15949 if isinstance(out, numpy.ndarray): 15950 if product(shape) != product(out.shape): 15951 raise ValueError('incompatible output shape') 15952 if not numpy.can_cast(dtype, out.dtype): 15953 raise ValueError('incompatible output dtype') 15954 return out.reshape(shape) 15955 if isinstance(out, str) and out[:6] == 'memmap': 15956 import tempfile 15957 15958 tempdir = out[7:] if len(out) > 7 else None 15959 if suffix is None: 15960 suffix = '.memmap' 15961 with tempfile.NamedTemporaryFile(dir=tempdir, suffix=suffix) as fh: 15962 return numpy.memmap(fh, shape=shape, dtype=dtype, mode=mode) 15963 return numpy.memmap(out, shape=shape, dtype=dtype, mode=mode) 15964 15965 15966def matlabstr2py(string): 15967 r"""Return Python object from Matlab string representation. 15968 15969 Return str, bool, int, float, list (Matlab arrays or cells), or 15970 dict (Matlab structures) types. 15971 15972 Use to access ScanImage metadata. 15973 15974 >>> matlabstr2py('1') 15975 1 15976 >>> matlabstr2py("['x y z' true false; 1 2.0 -3e4; NaN Inf @class]") 15977 [['x y z', True, False], [1, 2.0, -30000.0], [nan, inf, '@class']] 15978 >>> d = matlabstr2py( 15979 ... "SI.hChannels.channelType = {'stripe' 'stripe'}\n" 15980 ... "SI.hChannels.channelsActive = 2" 15981 ... ) 15982 >>> d['SI.hChannels.channelType'] 15983 ['stripe', 'stripe'] 15984 15985 """ 15986 # TODO: handle invalid input 15987 # TODO: review unboxing of multidimensional arrays 15988 15989 def lex(s): 15990 # return sequence of tokens from matlab string representation 15991 tokens = ['['] 15992 while True: 15993 t, i = next_token(s) 15994 if t is None: 15995 break 15996 if t == ';': 15997 tokens.extend((']', '[')) 15998 elif t == '[': 15999 tokens.extend(('[', '[')) 16000 elif t == ']': 16001 tokens.extend((']', ']')) 16002 else: 16003 tokens.append(t) 16004 s = s[i:] 16005 tokens.append(']') 16006 return tokens 16007 16008 def next_token(s): 16009 # return next token in matlab string 16010 length = len(s) 16011 if length == 0: 16012 return None, 0 16013 i = 0 16014 while i < length and s[i] == ' ': 16015 i += 1 16016 if i == length: 16017 return None, i 16018 if s[i] in '{[;]}': 16019 return s[i], i + 1 16020 if s[i] == "'": 16021 j = i + 1 16022 while j < length and s[j] != "'": 16023 j += 1 16024 return s[i : j + 1], j + 1 16025 if s[i] == '<': 16026 j = i + 1 16027 while j < length and s[j] != '>': 16028 j += 1 16029 return s[i : j + 1], j + 1 16030 j = i 16031 while j < length and not s[j] in ' {[;]}': 16032 j += 1 16033 return s[i:j], j 16034 16035 def value(s, fail=False): 16036 # return Python value of token 16037 s = s.strip() 16038 if not s: 16039 return s 16040 if len(s) == 1: 16041 try: 16042 return int(s) 16043 except Exception: 16044 if fail: 16045 raise ValueError() 16046 return s 16047 if s[0] == "'": 16048 if fail and s[-1] != "'" or "'" in s[1:-1]: 16049 raise ValueError() 16050 return s[1:-1] 16051 if s[0] == '<': 16052 if fail and s[-1] != '>' or '<' in s[1:-1]: 16053 raise ValueError() 16054 return s 16055 if fail and any(i in s for i in " ';[]{}"): 16056 raise ValueError() 16057 if s[0] == '@': 16058 return s 16059 if s in ('true', 'True'): 16060 return True 16061 if s in ('false', 'False'): 16062 return False 16063 if s[:6] == 'zeros(': 16064 return numpy.zeros([int(i) for i in s[6:-1].split(',')]).tolist() 16065 if s[:5] == 'ones(': 16066 return numpy.ones([int(i) for i in s[5:-1].split(',')]).tolist() 16067 if '.' in s or 'e' in s: 16068 try: 16069 return float(s) 16070 except Exception: 16071 pass 16072 try: 16073 return int(s) 16074 except Exception: 16075 pass 16076 try: 16077 return float(s) # nan, inf 16078 except Exception: 16079 if fail: 16080 raise ValueError() 16081 return s 16082 16083 def parse(s): 16084 # return Python value from string representation of Matlab value 16085 s = s.strip() 16086 try: 16087 return value(s, fail=True) 16088 except ValueError: 16089 pass 16090 result = add2 = [] 16091 levels = [add2] 16092 for t in lex(s): 16093 if t in '[{': 16094 add2 = [] 16095 levels.append(add2) 16096 elif t in ']}': 16097 x = levels.pop() 16098 if len(x) == 1 and isinstance(x[0], (list, str)): 16099 x = x[0] 16100 add2 = levels[-1] 16101 add2.append(x) 16102 else: 16103 add2.append(value(t)) 16104 if len(result) == 1 and isinstance(result[0], (list, str)): 16105 result = result[0] 16106 return result 16107 16108 if '\r' in string or '\n' in string: 16109 # structure 16110 d = {} 16111 for line in string.splitlines(): 16112 line = line.strip() 16113 if not line or line[0] == '%': 16114 continue 16115 k, v = line.split('=', 1) 16116 k = k.strip() 16117 if any(c in k for c in " ';[]{}<>"): 16118 continue 16119 d[k] = parse(v) 16120 return d 16121 return parse(string) 16122 16123 16124def stripnull(string, null=b'\x00', first=True): 16125 r"""Return string truncated at first null character. 16126 16127 Clean NULL terminated C strings. For Unicode strings use null='\0'. 16128 16129 >>> stripnull(b'string\x00\x00') 16130 b'string' 16131 >>> stripnull(b'string\x00string\x00\x00', first=False) 16132 b'string\x00string' 16133 >>> stripnull('string\x00', null='\0') 16134 'string' 16135 16136 """ 16137 if first: 16138 i = string.find(null) 16139 return string if i < 0 else string[:i] 16140 null = null[0] 16141 i = len(string) 16142 while i: 16143 i -= 1 16144 if string[i] != null: 16145 break 16146 else: 16147 i = -1 16148 return string[: i + 1] 16149 16150 16151def stripascii(string): 16152 r"""Return string truncated at last byte that is 7-bit ASCII. 16153 16154 Clean NULL separated and terminated TIFF strings. 16155 16156 >>> stripascii(b'string\x00string\n\x01\x00') 16157 b'string\x00string\n' 16158 >>> stripascii(b'\x00') 16159 b'' 16160 16161 """ 16162 # TODO: pythonize this 16163 i = len(string) 16164 while i: 16165 i -= 1 16166 if 8 < string[i] < 127: 16167 break 16168 else: 16169 i = -1 16170 return string[: i + 1] 16171 16172 16173def asbool(value, true=None, false=None): 16174 """Return string as bool if possible, else raise TypeError. 16175 16176 >>> asbool(b' False ') 16177 False 16178 >>> asbool('ON', ['on'], ['off']) 16179 True 16180 16181 """ 16182 value = value.strip().lower() 16183 isbytes = False 16184 if true is None: 16185 if isinstance(value, bytes): 16186 if value == b'true': 16187 return True 16188 isbytes = True 16189 elif value == 'true': 16190 return True 16191 if false is None: 16192 if isbytes or isinstance(value, bytes): 16193 if value == b'false': 16194 return False 16195 elif value == 'false': 16196 return False 16197 if value in true: 16198 return True 16199 if value in false: 16200 return False 16201 raise TypeError() 16202 16203 16204def astype(value, types=None): 16205 """Return argument as one of types if possible. 16206 16207 >>> astype('42') 16208 42 16209 >>> astype('3.14') 16210 3.14 16211 >>> astype('True') 16212 True 16213 >>> astype(b'Neee-Wom') 16214 'Neee-Wom' 16215 16216 """ 16217 if types is None: 16218 types = int, float, asbool, bytes2str 16219 for typ in types: 16220 try: 16221 return typ(value) 16222 except (ValueError, AttributeError, TypeError, UnicodeEncodeError): 16223 pass 16224 return value 16225 16226 16227def format_size(size, threshold=1536): 16228 """Return file size as string from byte size. 16229 16230 >>> format_size(1234) 16231 '1234 B' 16232 >>> format_size(12345678901) 16233 '11.50 GiB' 16234 16235 """ 16236 if size < threshold: 16237 return f'{size} B' 16238 for unit in ('KiB', 'MiB', 'GiB', 'TiB', 'PiB'): 16239 size /= 1024.0 16240 if size < threshold: 16241 return f'{size:.2f} {unit}' 16242 return 'ginormous' 16243 16244 16245def identityfunc(arg, *args, **kwargs): 16246 """Single argument identity function. 16247 16248 >>> identityfunc('arg') 16249 'arg' 16250 16251 """ 16252 return arg 16253 16254 16255def nullfunc(*args, **kwargs): 16256 """Null function. 16257 16258 >>> nullfunc('arg', kwarg='kwarg') 16259 16260 """ 16261 return 16262 16263 16264def sequence(value): 16265 """Return tuple containing value if value is not a tuple or list. 16266 16267 >>> sequence(1) 16268 (1,) 16269 >>> sequence([1]) 16270 [1] 16271 >>> sequence('ab') 16272 ('ab',) 16273 16274 """ 16275 return value if isinstance(value, (tuple, list)) else (value,) 16276 16277 16278def product(iterable): 16279 """Return product of sequence of numbers. 16280 16281 Equivalent of functools.reduce(operator.mul, iterable, 1). 16282 Multiplying numpy integers might overflow. 16283 16284 >>> product([2**8, 2**30]) 16285 274877906944 16286 >>> product([]) 16287 1 16288 16289 """ 16290 prod = 1 16291 for i in iterable: 16292 prod *= i 16293 return prod 16294 16295 16296def natural_sorted(iterable): 16297 """Return human sorted list of strings. 16298 16299 E.g. for sorting file names. 16300 16301 >>> natural_sorted(['f1', 'f2', 'f10']) 16302 ['f1', 'f2', 'f10'] 16303 16304 """ 16305 16306 def sortkey(x): 16307 return [(int(c) if c.isdigit() else c) for c in re.split(numbers, x)] 16308 16309 numbers = re.compile(r'(\d+)') 16310 return sorted(iterable, key=sortkey) 16311 16312 16313def epics_datetime(sec, nsec): 16314 """Return datetime object from epicsTSSec and epicsTSNsec tag values.""" 16315 return datetime.datetime.fromtimestamp(sec + 631152000 + nsec / 1e9) 16316 16317 16318def excel_datetime(timestamp, epoch=None): 16319 """Return datetime object from timestamp in Excel serial format. 16320 16321 Convert LSM time stamps. 16322 16323 >>> excel_datetime(40237.029999999795) 16324 datetime.datetime(2010, 2, 28, 0, 43, 11, 999982) 16325 16326 """ 16327 if epoch is None: 16328 epoch = datetime.datetime.fromordinal(693594) 16329 return epoch + datetime.timedelta(timestamp) 16330 16331 16332def julian_datetime(julianday, milisecond=0): 16333 """Return datetime from days since 1/1/4713 BC and ms since midnight. 16334 16335 Convert Julian dates according to MetaMorph. 16336 16337 >>> julian_datetime(2451576, 54362783) 16338 datetime.datetime(2000, 2, 2, 15, 6, 2, 783) 16339 16340 """ 16341 if julianday <= 1721423: 16342 # no datetime before year 1 16343 return None 16344 16345 a = julianday + 1 16346 if a > 2299160: 16347 alpha = math.trunc((a - 1867216.25) / 36524.25) 16348 a += 1 + alpha - alpha // 4 16349 b = a + (1524 if a > 1721423 else 1158) 16350 c = math.trunc((b - 122.1) / 365.25) 16351 d = math.trunc(365.25 * c) 16352 e = math.trunc((b - d) / 30.6001) 16353 16354 day = b - d - math.trunc(30.6001 * e) 16355 month = e - (1 if e < 13.5 else 13) 16356 year = c - (4716 if month > 2.5 else 4715) 16357 16358 hour, milisecond = divmod(milisecond, 1000 * 60 * 60) 16359 minute, milisecond = divmod(milisecond, 1000 * 60) 16360 second, milisecond = divmod(milisecond, 1000) 16361 16362 return datetime.datetime( 16363 year, month, day, hour, minute, second, milisecond 16364 ) 16365 16366 16367def byteorder_isnative(byteorder): 16368 """Return if byteorder matches the system's byteorder. 16369 16370 >>> byteorder_isnative('=') 16371 True 16372 16373 """ 16374 if byteorder in ('=', sys.byteorder): 16375 return True 16376 keys = {'big': '>', 'little': '<'} 16377 return keys.get(byteorder, byteorder) == keys[sys.byteorder] 16378 16379 16380def byteorder_compare(byteorder, byteorder2): 16381 """Return if byteorders match. 16382 16383 >>> byteorder_compare('<', '<') 16384 True 16385 >>> byteorder_compare('>', '<') 16386 False 16387 16388 """ 16389 if byteorder == byteorder2 or byteorder == '|' or byteorder2 == '|': 16390 return True 16391 if byteorder == '=': 16392 byteorder = {'big': '>', 'little': '<'}[sys.byteorder] 16393 elif byteorder2 == '=': 16394 byteorder2 = {'big': '>', 'little': '<'}[sys.byteorder] 16395 return byteorder == byteorder2 16396 16397 16398def recarray2dict(recarray): 16399 """Return numpy.recarray as dict.""" 16400 # TODO: subarrays 16401 result = {} 16402 for descr, value in zip(recarray.dtype.descr, recarray): 16403 name, dtype = descr[:2] 16404 if dtype[1] == 'S': 16405 value = bytes2str(stripnull(value)) 16406 elif value.ndim < 2: 16407 value = value.tolist() 16408 result[name] = value 16409 return result 16410 16411 16412def xml2dict(xml, sanitize=True, prefix=None): 16413 """Return XML as dict. 16414 16415 >>> xml2dict('<?xml version="1.0" ?><root attr="name"><key>1</key></root>') 16416 {'root': {'key': 1, 'attr': 'name'}} 16417 >>> xml2dict('<level1><level2>3.5322</level2></level1>') 16418 {'level1': {'level2': 3.5322}} 16419 16420 """ 16421 from xml.etree import ElementTree as etree # delayed import 16422 16423 at = tx = '' 16424 if prefix: 16425 at, tx = prefix 16426 16427 def astype(value): 16428 # return string value as int, float, bool, or unchanged 16429 if not isinstance(value, (str, bytes)): 16430 return value 16431 for t in (int, float, asbool): 16432 try: 16433 return t(value) 16434 except Exception: 16435 pass 16436 return value 16437 16438 def etree2dict(t): 16439 # adapted from https://stackoverflow.com/a/10077069/453463 16440 key = t.tag 16441 if sanitize: 16442 key = key.rsplit('}', 1)[-1] 16443 d = {key: {} if t.attrib else None} 16444 children = list(t) 16445 if children: 16446 dd = collections.defaultdict(list) 16447 for dc in map(etree2dict, children): 16448 for k, v in dc.items(): 16449 dd[k].append(astype(v)) 16450 d = { 16451 key: { 16452 k: astype(v[0]) if len(v) == 1 else astype(v) 16453 for k, v in dd.items() 16454 } 16455 } 16456 if t.attrib: 16457 d[key].update((at + k, astype(v)) for k, v in t.attrib.items()) 16458 if t.text: 16459 text = t.text.strip() 16460 if children or t.attrib: 16461 if text: 16462 d[key][tx + 'value'] = astype(text) 16463 else: 16464 d[key] = astype(text) 16465 return d 16466 16467 return etree2dict(etree.fromstring(xml)) 16468 16469 16470def hexdump(bytestr, width=75, height=24, snipat=-2, modulo=2, ellipsis=None): 16471 """Return hexdump representation of bytes. 16472 16473 >>> hexdump(binascii.unhexlify('49492a00080000000e00fe0004000100')) 16474 '49 49 2a 00 08 00 00 00 0e 00 fe 00 04 00 01 00 II*.............' 16475 16476 """ 16477 size = len(bytestr) 16478 if size < 1 or width < 2 or height < 1: 16479 return '' 16480 if height == 1: 16481 addr = b'' 16482 bytesperline = min( 16483 modulo * (((width - len(addr)) // 4) // modulo), size 16484 ) 16485 if bytesperline < 1: 16486 return '' 16487 nlines = 1 16488 else: 16489 addr = b'%%0%ix: ' % len(b'%x' % size) 16490 bytesperline = min( 16491 modulo * (((width - len(addr % 1)) // 4) // modulo), size 16492 ) 16493 if bytesperline < 1: 16494 return '' 16495 width = 3 * bytesperline + len(addr % 1) 16496 nlines = (size - 1) // bytesperline + 1 16497 16498 if snipat is None or snipat == 1: 16499 snipat = height 16500 elif 0 < abs(snipat) < 1: 16501 snipat = int(math.floor(height * snipat)) 16502 if snipat < 0: 16503 snipat += height 16504 16505 if height == 1 or nlines == 1: 16506 blocks = [(0, bytestr[:bytesperline])] 16507 addr = b'' 16508 height = 1 16509 width = 3 * bytesperline 16510 elif height is None or nlines <= height: 16511 blocks = [(0, bytestr)] 16512 elif snipat <= 0: 16513 start = bytesperline * (nlines - height) 16514 blocks = [(start, bytestr[start:])] # (start, None) 16515 elif snipat >= height or height < 3: 16516 end = bytesperline * height 16517 blocks = [(0, bytestr[:end])] # (end, None) 16518 else: 16519 end1 = bytesperline * snipat 16520 end2 = bytesperline * (height - snipat - 1) 16521 blocks = [ 16522 (0, bytestr[:end1]), 16523 (size - end1 - end2, None), 16524 (size - end2, bytestr[size - end2 :]), 16525 ] 16526 16527 ellipsis = b'...' if ellipsis is None else ellipsis.encode('cp1252') 16528 result = [] 16529 for start, bytestr in blocks: 16530 if bytestr is None: 16531 result.append(ellipsis) # 'skip %i bytes' % start) 16532 continue 16533 hexstr = binascii.hexlify(bytestr) 16534 strstr = re.sub(br'[^\x20-\x7f]', b'.', bytestr) 16535 for i in range(0, len(bytestr), bytesperline): 16536 h = hexstr[2 * i : 2 * i + bytesperline * 2] 16537 r = (addr % (i + start)) if height > 1 else addr 16538 r += b' '.join(h[i : i + 2] for i in range(0, 2 * bytesperline, 2)) 16539 r += b' ' * (width - len(r)) 16540 r += strstr[i : i + bytesperline] 16541 result.append(r) 16542 result = b'\n'.join(result) 16543 result = result.decode('ascii') 16544 return result 16545 16546 16547def isprintable(string): 16548 r"""Return if all characters in string are printable. 16549 16550 >>> isprintable('abc') 16551 True 16552 >>> isprintable(b'\01') 16553 False 16554 16555 """ 16556 string = string.strip() 16557 if not string: 16558 return True 16559 try: 16560 return string.isprintable() 16561 except Exception: 16562 pass 16563 try: 16564 return string.decode().isprintable() 16565 except Exception: 16566 pass 16567 16568 16569def clean_whitespace(string, compact=False): 16570 """Return string with compressed whitespace.""" 16571 for a, b in ( 16572 ('\r\n', '\n'), 16573 ('\r', '\n'), 16574 ('\n\n', '\n'), 16575 ('\t', ' '), 16576 (' ', ' '), 16577 ): 16578 string = string.replace(a, b) 16579 if compact: 16580 for a, b in ( 16581 ('\n', ' '), 16582 ('[ ', '['), 16583 (' ', ' '), 16584 (' ', ' '), 16585 (' ', ' '), 16586 ): 16587 string = string.replace(a, b) 16588 return string.strip() 16589 16590 16591def pformat_xml(xml): 16592 """Return pretty formatted XML.""" 16593 try: 16594 from lxml import etree # delayed import 16595 16596 if not isinstance(xml, bytes): 16597 xml = xml.encode() 16598 tree = etree.parse(io.BytesIO(xml)) 16599 xml = etree.tostring( 16600 tree, 16601 pretty_print=True, 16602 xml_declaration=True, 16603 encoding=tree.docinfo.encoding, 16604 ) 16605 xml = bytes2str(xml) 16606 except Exception: 16607 if isinstance(xml, bytes): 16608 xml = bytes2str(xml) 16609 xml = xml.replace('><', '>\n<') 16610 return xml.replace(' ', ' ').replace('\t', ' ') 16611 16612 16613def pformat(arg, width=79, height=24, compact=True): 16614 """Return pretty formatted representation of object as string. 16615 16616 Whitespace might be altered. 16617 16618 """ 16619 if height is None or height < 1: 16620 height = 1024 16621 if width is None or width < 1: 16622 width = 256 16623 16624 npopt = numpy.get_printoptions() 16625 numpy.set_printoptions(threshold=100, linewidth=width) 16626 16627 if isinstance(arg, bytes): 16628 if arg[:5].lower() == b'<?xml' or arg[-4:] == b'OME>': 16629 arg = bytes2str(arg) 16630 16631 if isinstance(arg, bytes): 16632 if isprintable(arg): 16633 arg = bytes2str(arg) 16634 arg = clean_whitespace(arg) 16635 else: 16636 numpy.set_printoptions(**npopt) 16637 return hexdump(arg, width=width, height=height, modulo=1) 16638 arg = arg.rstrip() 16639 elif isinstance(arg, str): 16640 if arg[:5].lower() == '<?xml' or arg[-4:] == 'OME>': 16641 arg = arg[: 4 * width] if height == 1 else pformat_xml(arg) 16642 arg = arg.rstrip() 16643 elif isinstance(arg, numpy.record): 16644 arg = arg.pprint() 16645 else: 16646 import pprint # delayed import 16647 16648 arg = pprint.pformat(arg, width=width, compact=compact) 16649 16650 numpy.set_printoptions(**npopt) 16651 16652 if height == 1: 16653 arg = clean_whitespace(arg, compact=True) 16654 return arg[:width] 16655 16656 argl = list(arg.splitlines()) 16657 if len(argl) > height: 16658 arg = '\n'.join(argl[: height // 2] + ['...'] + argl[-height // 2 :]) 16659 return arg 16660 16661 16662def snipstr(string, width=79, snipat=None, ellipsis=None): 16663 """Return string cut to specified length. 16664 16665 >>> snipstr('abcdefghijklmnop', 8) 16666 'abc...op' 16667 16668 """ 16669 if snipat is None: 16670 snipat = 0.5 16671 if ellipsis is None: 16672 if isinstance(string, bytes): 16673 ellipsis = b'...' 16674 else: 16675 ellipsis = '\u2026' 16676 esize = len(ellipsis) 16677 16678 splitlines = string.splitlines() 16679 # TODO: finish and test multiline snip 16680 16681 result = [] 16682 for line in splitlines: 16683 if line is None: 16684 result.append(ellipsis) 16685 continue 16686 linelen = len(line) 16687 if linelen <= width: 16688 result.append(string) 16689 continue 16690 16691 split = snipat 16692 if split is None or split == 1: 16693 split = linelen 16694 elif 0 < abs(split) < 1: 16695 split = int(math.floor(linelen * split)) 16696 if split < 0: 16697 split += linelen 16698 if split < 0: 16699 split = 0 16700 16701 if esize == 0 or width < esize + 1: 16702 if split <= 0: 16703 result.append(string[-width:]) 16704 else: 16705 result.append(string[:width]) 16706 elif split <= 0: 16707 result.append(ellipsis + string[esize - width :]) 16708 elif split >= linelen or width < esize + 4: 16709 result.append(string[: width - esize] + ellipsis) 16710 else: 16711 splitlen = linelen - width + esize 16712 end1 = split - splitlen // 2 16713 end2 = end1 + splitlen 16714 result.append(string[:end1] + ellipsis + string[end2:]) 16715 16716 if isinstance(string, bytes): 16717 return b'\n'.join(result) 16718 return '\n'.join(result) 16719 16720 16721def enumstr(enum): 16722 """Return short string representation of Enum instance.""" 16723 name = enum.name 16724 if name is None: 16725 name = str(enum) 16726 return name 16727 16728 16729def enumarg(enum, arg): 16730 """Return enum member from its name or value. 16731 16732 >>> enumarg(TIFF.PHOTOMETRIC, 2) 16733 <PHOTOMETRIC.RGB: 2> 16734 >>> enumarg(TIFF.PHOTOMETRIC, 'RGB') 16735 <PHOTOMETRIC.RGB: 2> 16736 16737 """ 16738 try: 16739 return enum(arg) 16740 except Exception: 16741 try: 16742 return enum[arg.upper()] 16743 except Exception: 16744 raise ValueError(f'invalid argument {arg}') 16745 16746 16747def parse_kwargs(kwargs, *keys, **keyvalues): 16748 """Return dict with keys from keys|keyvals and values from kwargs|keyvals. 16749 16750 Existing keys are deleted from kwargs. 16751 16752 >>> kwargs = {'one': 1, 'two': 2, 'four': 4} 16753 >>> kwargs2 = parse_kwargs(kwargs, 'two', 'three', four=None, five=5) 16754 >>> kwargs == {'one': 1} 16755 True 16756 >>> kwargs2 == {'two': 2, 'four': 4, 'five': 5} 16757 True 16758 16759 """ 16760 result = {} 16761 for key in keys: 16762 if key in kwargs: 16763 result[key] = kwargs[key] 16764 del kwargs[key] 16765 for key, value in keyvalues.items(): 16766 if key in kwargs: 16767 result[key] = kwargs[key] 16768 del kwargs[key] 16769 else: 16770 result[key] = value 16771 return result 16772 16773 16774def update_kwargs(kwargs, **keyvalues): 16775 """Update dict with keys and values if keys do not already exist. 16776 16777 >>> kwargs = {'one': 1, } 16778 >>> update_kwargs(kwargs, one=None, two=2) 16779 >>> kwargs == {'one': 1, 'two': 2} 16780 True 16781 16782 """ 16783 for key, value in keyvalues.items(): 16784 if key not in kwargs: 16785 kwargs[key] = value 16786 16787 16788def log_warning(msg, *args, **kwargs): 16789 """Log message with level WARNING.""" 16790 import logging 16791 16792 logging.getLogger(__name__).warning(msg, *args, **kwargs) 16793 16794 16795def validate_jhove(filename, jhove=None, ignore=None): 16796 """Validate TIFF file using jhove -m TIFF-hul. 16797 16798 Raise ValueError if jhove outputs an error message unless the message 16799 contains one of the strings in 'ignore'. 16800 16801 JHOVE does not support bigtiff or more than 50 IFDs. 16802 16803 See `JHOVE TIFF-hul Module <http://jhove.sourceforge.net/tiff-hul.html>`_ 16804 16805 """ 16806 import subprocess 16807 16808 if ignore is None: 16809 ignore = ['More than 50 IFDs'] 16810 if jhove is None: 16811 jhove = 'jhove' 16812 out = subprocess.check_output([jhove, filename, '-m', 'TIFF-hul']) 16813 if b'ErrorMessage: ' in out: 16814 for line in out.splitlines(): 16815 line = line.strip() 16816 if line.startswith(b'ErrorMessage: '): 16817 error = line[14:].decode() 16818 for i in ignore: 16819 if i in error: 16820 break 16821 else: 16822 raise ValueError(error) 16823 break 16824 16825 16826def tiffcomment(arg, comment=None, index=None, code=None): 16827 """Return or replace ImageDescription value in first page of TIFF file.""" 16828 if index is None: 16829 index = 0 16830 if code is None: 16831 code = 270 16832 mode = None if comment is None else 'r+b' 16833 with TiffFile(arg, mode=mode) as tif: 16834 tag = tif.pages[index].tags.get(code, None) 16835 if tag is None: 16836 raise ValueError(f'no {TIFF.TAGS[code]} tag found') 16837 if comment is None: 16838 return tag.value 16839 tag.overwrite(comment) 16840 16841 16842def tiff2fsspec( 16843 filename, 16844 url, 16845 out=None, 16846 key=None, 16847 series=None, 16848 level=None, 16849 chunkmode=None, 16850 version=None, 16851): 16852 """Write fsspec ReferenceFileSystem JSON from TIFF file.""" 16853 if out is None: 16854 out = filename + '.json' 16855 with TiffFile(filename) as tif: 16856 with tif.aszarr( 16857 key=key, series=series, level=level, chunkmode=chunkmode 16858 ) as store: 16859 store.write_fsspec(out, url, version=version) 16860 16861 16862def lsm2bin(lsmfile, binfile=None, tile=None, verbose=True): 16863 """Convert [MP]TZCYX LSM file to series of BIN files. 16864 16865 One BIN file containing 'ZCYX' data are created for each position, time, 16866 and tile. The position, time, and tile indices are encoded at the end 16867 of the filenames. 16868 16869 """ 16870 verbose = print if verbose else nullfunc 16871 16872 if tile is None: 16873 tile = (256, 256) 16874 16875 if binfile is None: 16876 binfile = lsmfile 16877 elif binfile.lower() == 'none': 16878 binfile = None 16879 if binfile: 16880 binfile += '_(z%ic%iy%ix%i)_m%%ip%%it%%03iy%%ix%%i.bin' 16881 16882 verbose('\nOpening LSM file... ', end='', flush=True) 16883 timer = Timer() 16884 16885 with TiffFile(lsmfile) as lsm: 16886 if not lsm.is_lsm: 16887 verbose('\n', lsm, flush=True) 16888 raise ValueError('not a LSM file') 16889 series = lsm.series[0] # first series contains the image data 16890 shape = series.get_shape(False) 16891 axes = series.get_axes(False) 16892 dtype = series.dtype 16893 size = product(shape) * dtype.itemsize 16894 16895 verbose(timer) 16896 # verbose(lsm, flush=True) 16897 verbose( 16898 'Image\n axes: {}\n shape: {}\n dtype: {}\n size: {}'.format( 16899 axes, shape, dtype, format_size(size) 16900 ), 16901 flush=True, 16902 ) 16903 if not series.axes.endswith('TZCYX'): 16904 raise ValueError('not a *TZCYX LSM file') 16905 16906 verbose('Copying image from LSM to BIN files', end='', flush=True) 16907 timer.start() 16908 tiles = shape[-2] // tile[-2], shape[-1] // tile[-1] 16909 if binfile: 16910 binfile = binfile % (shape[-4], shape[-3], tile[0], tile[1]) 16911 shape = (1,) * (7 - len(shape)) + shape 16912 # cache for ZCYX stacks and output files 16913 data = numpy.empty(shape[3:], dtype=dtype) 16914 out = numpy.empty( 16915 (shape[-4], shape[-3], tile[0], tile[1]), dtype=dtype 16916 ) 16917 # iterate over Tiff pages containing data 16918 pages = iter(series.pages) 16919 for m in range(shape[0]): # mosaic axis 16920 for p in range(shape[1]): # position axis 16921 for t in range(shape[2]): # time axis 16922 for z in range(shape[3]): # z slices 16923 data[z] = next(pages).asarray() 16924 for y in range(tiles[0]): # tile y 16925 for x in range(tiles[1]): # tile x 16926 out[:] = data[ 16927 ..., 16928 y * tile[0] : (y + 1) * tile[0], 16929 x * tile[1] : (x + 1) * tile[1], 16930 ] 16931 if binfile: 16932 out.tofile(binfile % (m, p, t, y, x)) 16933 verbose('.', end='', flush=True) 16934 verbose(timer, flush=True) 16935 16936 16937def imshow( 16938 data, 16939 photometric=None, 16940 planarconfig=None, 16941 bitspersample=None, 16942 nodata=0, 16943 interpolation=None, 16944 cmap=None, 16945 vmin=None, 16946 vmax=None, 16947 figure=None, 16948 title=None, 16949 dpi=96, 16950 subplot=None, 16951 maxdim=None, 16952 **kwargs, 16953): 16954 """Plot n-dimensional images using matplotlib.pyplot. 16955 16956 Return figure, subplot, and plot axis. 16957 Requires pyplot already imported C{from matplotlib import pyplot}. 16958 16959 Parameters 16960 ---------- 16961 data : nd array 16962 The image data. 16963 photometric : {'MINISWHITE', 'MINISBLACK', 'RGB', or 'PALETTE'} 16964 The color space of the image data. 16965 planarconfig : {'CONTIG' or 'SEPARATE'} 16966 Defines how components of each pixel are stored. 16967 bitspersample : int 16968 Number of bits per channel in integer RGB images. 16969 interpolation : str 16970 The image interpolation method used in matplotlib.imshow. By default, 16971 'nearest' is used for image dimensions <= 512, else 'bilinear'. 16972 cmap : str or matplotlib.colors.Colormap 16973 The colormap maps non-RGBA scalar data to colors. 16974 vmin, vmax : scalar 16975 Data range covered by the colormap. By default, the complete 16976 range of the data is covered. 16977 figure : matplotlib.figure.Figure 16978 Matplotlib figure to use for plotting. 16979 title : str 16980 Window and subplot title. 16981 subplot : int 16982 A matplotlib.pyplot.subplot axis. 16983 maxdim : int 16984 Maximum image width and length. 16985 kwargs : dict 16986 Additional arguments for matplotlib.pyplot.imshow. 16987 16988 """ 16989 # TODO: rewrite detection of isrgb, iscontig 16990 # TODO: use planarconfig 16991 if photometric is None: 16992 photometric = 'RGB' 16993 if maxdim is None: 16994 maxdim = 2 ** 16 16995 isrgb = photometric in ('RGB', 'YCBCR') # 'PALETTE', 'YCBCR' 16996 16997 if data.dtype == 'float16': 16998 data = data.astype('float32') 16999 17000 if data.dtype.kind == 'b': 17001 isrgb = False 17002 17003 if isrgb and not ( 17004 data.shape[-1] in (3, 4) 17005 or (data.ndim > 2 and data.shape[-3] in (3, 4)) 17006 ): 17007 isrgb = False 17008 photometric = 'MINISBLACK' 17009 17010 data = data.squeeze() 17011 if photometric in ('MINISWHITE', 'MINISBLACK', None): 17012 data = reshape_nd(data, 2) 17013 else: 17014 data = reshape_nd(data, 3) 17015 17016 dims = data.ndim 17017 if dims < 2: 17018 raise ValueError('not an image') 17019 if dims == 2: 17020 dims = 0 17021 isrgb = False 17022 else: 17023 if isrgb and data.shape[-3] in (3, 4): 17024 data = numpy.swapaxes(data, -3, -2) 17025 data = numpy.swapaxes(data, -2, -1) 17026 elif not isrgb and ( 17027 data.shape[-1] < data.shape[-2] // 8 17028 and data.shape[-1] < data.shape[-3] // 8 17029 ): 17030 data = numpy.swapaxes(data, -3, -1) 17031 data = numpy.swapaxes(data, -2, -1) 17032 isrgb = isrgb and data.shape[-1] in (3, 4) 17033 dims -= 3 if isrgb else 2 17034 17035 if interpolation is None: 17036 threshold = 512 17037 elif isinstance(interpolation, int): 17038 threshold = interpolation 17039 else: 17040 threshold = 0 17041 17042 if isrgb: 17043 data = data[..., :maxdim, :maxdim, :maxdim] 17044 if threshold: 17045 if data.shape[-2] > threshold or data.shape[-3] > threshold: 17046 interpolation = 'bilinear' 17047 else: 17048 interpolation = 'nearest' 17049 else: 17050 data = data[..., :maxdim, :maxdim] 17051 if threshold: 17052 if data.shape[-1] > threshold or data.shape[-2] > threshold: 17053 interpolation = 'bilinear' 17054 else: 17055 interpolation = 'nearest' 17056 17057 if photometric == 'PALETTE' and isrgb: 17058 try: 17059 datamax = numpy.max(data) 17060 except ValueError: 17061 datamax = 1 17062 if datamax > 255: 17063 data = data >> 8 # possible precision loss 17064 data = data.astype('B') 17065 elif data.dtype.kind in 'ui': 17066 if not (isrgb and data.dtype.itemsize <= 1) or bitspersample is None: 17067 try: 17068 bitspersample = int(math.ceil(math.log(data.max(), 2))) 17069 except Exception: 17070 bitspersample = data.dtype.itemsize * 8 17071 elif not isinstance(bitspersample, (int, numpy.integer)): 17072 # bitspersample can be tuple, e.g. (5, 6, 5) 17073 bitspersample = data.dtype.itemsize * 8 17074 datamax = 2 ** bitspersample 17075 if isrgb: 17076 if bitspersample < 8: 17077 data = data << (8 - bitspersample) 17078 elif bitspersample > 8: 17079 data = data >> (bitspersample - 8) # precision loss 17080 data = data.astype('B') 17081 elif data.dtype.kind == 'f': 17082 if nodata: 17083 data = data.copy() 17084 data[data > 1e30] = 0.0 17085 try: 17086 datamax = numpy.max(data) 17087 except ValueError: 17088 datamax = 1 17089 if isrgb and datamax > 1.0: 17090 if data.dtype.char == 'd': 17091 data = data.astype('f') 17092 data /= datamax 17093 else: 17094 data = data / datamax 17095 elif data.dtype.kind == 'b': 17096 datamax = 1 17097 elif data.dtype.kind == 'c': 17098 data = numpy.absolute(data) 17099 try: 17100 datamax = numpy.max(data) 17101 except ValueError: 17102 datamax = 1 17103 17104 if isrgb: 17105 vmin = 0 17106 else: 17107 if vmax is None: 17108 vmax = datamax 17109 if vmin is None: 17110 if data.dtype.kind == 'i': 17111 dtmin = numpy.iinfo(data.dtype).min 17112 try: 17113 vmin = numpy.min(data) 17114 except ValueError: 17115 vmin = -1 17116 if vmin == dtmin: 17117 vmin = numpy.min(data[data > dtmin]) 17118 elif data.dtype.kind == 'f': 17119 dtmin = numpy.finfo(data.dtype).min 17120 try: 17121 vmin = numpy.min(data) 17122 except ValueError: 17123 vmin = 0.0 17124 if vmin == dtmin: 17125 vmin = numpy.min(data[data > dtmin]) 17126 else: 17127 vmin = 0 17128 17129 pyplot = sys.modules['matplotlib.pyplot'] 17130 17131 if figure is None: 17132 pyplot.rc('font', family='sans-serif', weight='normal', size=8) 17133 figure = pyplot.figure( 17134 dpi=dpi, 17135 figsize=(10.3, 6.3), 17136 frameon=True, 17137 facecolor='1.0', 17138 edgecolor='w', 17139 ) 17140 try: 17141 figure.canvas.manager.window.title(title) 17142 except Exception: 17143 pass 17144 size = len(title.splitlines()) if title else 1 17145 pyplot.subplots_adjust( 17146 bottom=0.03 * (dims + 2), 17147 top=0.98 - size * 0.03, 17148 left=0.1, 17149 right=0.95, 17150 hspace=0.05, 17151 wspace=0.0, 17152 ) 17153 if subplot is None: 17154 subplot = 111 17155 subplot = pyplot.subplot(subplot) 17156 subplot.set_facecolor((0, 0, 0)) 17157 17158 if title: 17159 try: 17160 title = str(title, 'Windows-1252') 17161 except TypeError: 17162 pass 17163 pyplot.title(title, size=11) 17164 17165 if cmap is None: 17166 if data.dtype.char == '?': 17167 cmap = 'gray' 17168 elif data.dtype.kind in 'buf' or vmin == 0: 17169 cmap = 'viridis' 17170 else: 17171 cmap = 'coolwarm' 17172 if photometric == 'MINISWHITE': 17173 cmap += '_r' 17174 17175 image = pyplot.imshow( 17176 numpy.atleast_2d(data[(0,) * dims].squeeze()), 17177 vmin=vmin, 17178 vmax=vmax, 17179 cmap=cmap, 17180 interpolation=interpolation, 17181 **kwargs, 17182 ) 17183 17184 if not isrgb: 17185 pyplot.colorbar() # panchor=(0.55, 0.5), fraction=0.05 17186 17187 def format_coord(x, y): 17188 # callback function to format coordinate display in toolbar 17189 x = int(x + 0.5) 17190 y = int(y + 0.5) 17191 try: 17192 if dims: 17193 return f'{curaxdat[1][y, x]} @ {current} [{y:4}, {x:4}]' 17194 return f'{data[y, x]} @ [{y:4}, {x:4}]' 17195 except IndexError: 17196 return '' 17197 17198 def none(event): 17199 return '' 17200 17201 subplot.format_coord = format_coord 17202 image.get_cursor_data = none 17203 image.format_cursor_data = none 17204 17205 if dims: 17206 current = list((0,) * dims) 17207 curaxdat = [0, data[tuple(current)].squeeze()] 17208 sliders = [ 17209 pyplot.Slider( 17210 pyplot.axes([0.125, 0.03 * (axis + 1), 0.725, 0.025]), 17211 f'Dimension {axis}', 17212 0, 17213 data.shape[axis] - 1, 17214 0, 17215 facecolor='0.5', 17216 valfmt=f'%.0f [{data.shape[axis]}]', 17217 ) 17218 for axis in range(dims) 17219 ] 17220 for slider in sliders: 17221 slider.drawon = False 17222 17223 def set_image(current, sliders=sliders, data=data): 17224 # change image and redraw canvas 17225 curaxdat[1] = data[tuple(current)].squeeze() 17226 image.set_data(curaxdat[1]) 17227 for ctrl, index in zip(sliders, current): 17228 ctrl.eventson = False 17229 ctrl.set_val(index) 17230 ctrl.eventson = True 17231 figure.canvas.draw() 17232 17233 def on_changed(index, axis, data=data, current=current): 17234 # callback function for slider change event 17235 index = int(round(index)) 17236 curaxdat[0] = axis 17237 if index == current[axis]: 17238 return 17239 if index >= data.shape[axis]: 17240 index = 0 17241 elif index < 0: 17242 index = data.shape[axis] - 1 17243 current[axis] = index 17244 set_image(current) 17245 17246 def on_keypressed(event, data=data, current=current): 17247 # callback function for key press event 17248 key = event.key 17249 axis = curaxdat[0] 17250 if str(key) in '0123456789': 17251 on_changed(key, axis) 17252 elif key == 'right': 17253 on_changed(current[axis] + 1, axis) 17254 elif key == 'left': 17255 on_changed(current[axis] - 1, axis) 17256 elif key == 'up': 17257 curaxdat[0] = 0 if axis == len(data.shape) - 1 else axis + 1 17258 elif key == 'down': 17259 curaxdat[0] = len(data.shape) - 1 if axis == 0 else axis - 1 17260 elif key == 'end': 17261 on_changed(data.shape[axis] - 1, axis) 17262 elif key == 'home': 17263 on_changed(0, axis) 17264 17265 figure.canvas.mpl_connect('key_press_event', on_keypressed) 17266 for axis, ctrl in enumerate(sliders): 17267 ctrl.on_changed(lambda k, a=axis: on_changed(k, a)) 17268 17269 return figure, subplot, image 17270 17271 17272def _app_show(): 17273 """Block the GUI. For use as skimage plugin.""" 17274 pyplot = sys.modules['matplotlib.pyplot'] 17275 pyplot.show() 17276 17277 17278def askopenfilename(**kwargs): 17279 """Return file name(s) from Tkinter's file open dialog.""" 17280 from tkinter import Tk, filedialog 17281 17282 root = Tk() 17283 root.withdraw() 17284 root.update() 17285 filenames = filedialog.askopenfilename(**kwargs) 17286 root.destroy() 17287 return filenames 17288 17289 17290def main(): 17291 """Tifffile command line usage main function.""" 17292 import logging 17293 import optparse # TODO: use argparse 17294 17295 logging.getLogger(__name__).setLevel(logging.INFO) 17296 17297 parser = optparse.OptionParser( 17298 usage='usage: %prog [options] path', 17299 description='Display image data in TIFF files.', 17300 version=f'%prog {__version__}', 17301 prog='tifffile', 17302 ) 17303 opt = parser.add_option 17304 opt( 17305 '-p', 17306 '--page', 17307 dest='page', 17308 type='int', 17309 default=-1, 17310 help='display single page', 17311 ) 17312 opt( 17313 '-s', 17314 '--series', 17315 dest='series', 17316 type='int', 17317 default=-1, 17318 help='display series of pages of same shape', 17319 ) 17320 opt( 17321 '-l', 17322 '--level', 17323 dest='level', 17324 type='int', 17325 default=-1, 17326 help='display pyramid level of series', 17327 ) 17328 opt( 17329 '--nomultifile', 17330 dest='nomultifile', 17331 action='store_true', 17332 default=False, 17333 help='do not read OME series from multiple files', 17334 ) 17335 opt( 17336 '--noplots', 17337 dest='noplots', 17338 type='int', 17339 default=10, 17340 help='maximum number of plots', 17341 ) 17342 opt( 17343 '--interpol', 17344 dest='interpol', 17345 metavar='INTERPOL', 17346 default=None, 17347 help='image interpolation method', 17348 ) 17349 opt('--dpi', dest='dpi', type='int', default=96, help='plot resolution') 17350 opt( 17351 '--vmin', 17352 dest='vmin', 17353 type='int', 17354 default=None, 17355 help='minimum value for colormapping', 17356 ) 17357 opt( 17358 '--vmax', 17359 dest='vmax', 17360 type='int', 17361 default=None, 17362 help='maximum value for colormapping', 17363 ) 17364 opt( 17365 '--debug', 17366 dest='debug', 17367 action='store_true', 17368 default=False, 17369 help='raise exception on failures', 17370 ) 17371 opt( 17372 '--doctest', 17373 dest='doctest', 17374 action='store_true', 17375 default=False, 17376 help='runs the docstring examples', 17377 ) 17378 opt('-v', '--detail', dest='detail', type='int', default=2) 17379 opt('-q', '--quiet', dest='quiet', action='store_true') 17380 17381 settings, path = parser.parse_args() 17382 path = ' '.join(path) 17383 17384 if settings.doctest: 17385 import doctest 17386 17387 try: 17388 import tifffile.tifffile as m 17389 except ImportError: 17390 m = None 17391 doctest.testmod(m, optionflags=doctest.ELLIPSIS) 17392 return 0 17393 if not path: 17394 path = askopenfilename( 17395 title='Select a TIFF file', filetypes=TIFF.FILEOPEN_FILTER 17396 ) 17397 if not path: 17398 parser.error('No file specified') 17399 17400 if any(i in path for i in '?*'): 17401 path = glob.glob(path) 17402 if not path: 17403 print('No files match the pattern') 17404 return 0 17405 # TODO: handle image sequences 17406 path = path[0] 17407 17408 if not settings.quiet: 17409 print('\nReading TIFF header:', end=' ', flush=True) 17410 timer = Timer() 17411 try: 17412 tif = TiffFile(path, _multifile=not settings.nomultifile) 17413 except Exception as exc: 17414 if settings.debug: 17415 raise 17416 print(f'\n\n{exc.__class__.__name__}: {exc}') 17417 sys.exit(0) 17418 17419 if not settings.quiet: 17420 print(timer) 17421 17422 if tif.is_ome: 17423 settings.norgb = True 17424 17425 images = [] 17426 if settings.noplots > 0: 17427 if not settings.quiet: 17428 print('Reading image data: ', end=' ', flush=True) 17429 17430 def notnone(x): 17431 return next(i for i in x if i is not None) 17432 17433 timer.start() 17434 try: 17435 if settings.page >= 0: 17436 images = [ 17437 ( 17438 tif.asarray(key=settings.page), 17439 tif.pages[settings.page], 17440 None, 17441 ) 17442 ] 17443 elif settings.series >= 0: 17444 series = tif.series[settings.series] 17445 if settings.level >= 0: 17446 level = settings.level 17447 elif series.is_pyramidal and product(series.shape) > 2 ** 32: 17448 level = -1 17449 for r in series.levels: 17450 level += 1 17451 if product(r.shape) < 2 ** 32: 17452 break 17453 else: 17454 level = 0 17455 images = [ 17456 ( 17457 tif.asarray(series=settings.series, level=level), 17458 notnone(tif.series[settings.series]._pages), 17459 tif.series[settings.series], 17460 ) 17461 ] 17462 else: 17463 for i, s in enumerate(tif.series[: settings.noplots]): 17464 if settings.level < 0: 17465 level = -1 17466 for r in s.levels: 17467 level += 1 17468 if product(r.shape) < 2 ** 31: 17469 break 17470 else: 17471 level = 0 17472 try: 17473 images.append( 17474 ( 17475 tif.asarray(series=i, level=level), 17476 notnone(s._pages), 17477 tif.series[i], 17478 ) 17479 ) 17480 except Exception as exc: 17481 images.append((None, notnone(s.pages), None)) 17482 if settings.debug: 17483 raise 17484 print( 17485 '\nSeries {} failed with {}: {}... '.format( 17486 i, exc.__class__.__name__, exc 17487 ), 17488 end='', 17489 ) 17490 except Exception as exc: 17491 if settings.debug: 17492 raise 17493 print(f'{exc.__class__.__name__}: {exc}') 17494 17495 if not settings.quiet: 17496 print(timer) 17497 17498 if not settings.quiet: 17499 print('Generating report:', end=' ', flush=True) 17500 timer.start() 17501 info = TiffFile.__str__(tif, detail=int(settings.detail)) 17502 print(timer) 17503 print() 17504 print(info) 17505 print() 17506 tif.close() 17507 17508 if images and settings.noplots > 0: 17509 try: 17510 import matplotlib 17511 17512 matplotlib.use('TkAgg') 17513 from matplotlib import pyplot 17514 except ImportError as exc: 17515 log_warning(f'tifffile.main: {exc.__class__.__name__}: {exc}') 17516 else: 17517 for img, page, series in images: 17518 if img is None: 17519 continue 17520 keyframe = page.keyframe 17521 vmin, vmax = settings.vmin, settings.vmax 17522 if keyframe.nodata: 17523 try: 17524 vmin = numpy.min(img[img > keyframe.nodata]) 17525 except ValueError: 17526 pass 17527 if tif.is_stk: 17528 try: 17529 vmin = tif.stk_metadata['MinScale'] 17530 vmax = tif.stk_metadata['MaxScale'] 17531 except KeyError: 17532 pass 17533 else: 17534 if vmax <= vmin: 17535 vmin, vmax = settings.vmin, settings.vmax 17536 if series: 17537 title = f'{tif}\n{page}\n{series}' 17538 else: 17539 title = f'{tif}\n {page}' 17540 photometric = 'MINISBLACK' 17541 if keyframe.photometric not in (3,): 17542 photometric = TIFF.PHOTOMETRIC(keyframe.photometric).name 17543 imshow( 17544 img, 17545 title=title, 17546 vmin=vmin, 17547 vmax=vmax, 17548 bitspersample=keyframe.bitspersample, 17549 nodata=keyframe.nodata, 17550 photometric=photometric, 17551 interpolation=settings.interpol, 17552 dpi=settings.dpi, 17553 ) 17554 pyplot.show() 17555 return 0 17556 17557 17558def bytes2str(b, encoding=None, errors='strict'): 17559 """Return Unicode string from encoded bytes.""" 17560 if encoding is not None: 17561 return b.decode(encoding, errors) 17562 try: 17563 return b.decode('utf-8', errors) 17564 except UnicodeDecodeError: 17565 return b.decode('cp1252', errors) 17566 17567 17568def bytestr(s, encoding='cp1252'): 17569 """Return bytes from Unicode string, else pass through.""" 17570 return s.encode(encoding) if isinstance(s, str) else s 17571 17572 17573# aliases and deprecated 17574imsave = imwrite 17575TiffWriter.save = TiffWriter.write 17576TiffReader = TiffFile 17577 17578if __name__ == '__main__': 17579 sys.exit(main()) 17580